Searching for Time Series Patterns at Big Data Scale


Duration: 50 mins
Ananth Gundabattula
Senior Architect, Commonwealth Bank of Australia

Time series patterns are pretty common in everyday life. Extracting patterns on these data sets and providing interactive search responses has immense value across many software systems. This talk presents an overview of how such a search system can be built for very large time series data points.

This talk focuses on the STAMP algorithm (Scalable Time Series Anytime Matrix profile) to showcase how time series patterns can be extracted for big data sets. The talk covers concepts of Matrix profile and how motifs/patterns are detected. Once the patterns are analyzed, the results are converted into a document format that is amenable for indexing via a search engine. The session then proceeds to extend the idea to detecting time series across multiple dimensions. A part of the session also covers the distributed approach to extract these time series patterns to tackle big data sets. Further, you will also learn about mechanisms to calculate time series segmentations, time series chains and discords or anomalies thereby enabling ideas for users to search for anomalies and segments of time series.

Dask, Stumpy and Elasticsearch stack is used to describe a reference implementation.

You may also be interested in

50 mins
Leading & Guiding Development Teams

By definition, a tech leader is responsible for leading and guiding development teams. In this session we will take a...

50 mins
Definition of Ready & Done - A Guide to Achieving Predictability

Delivering software often takes longer than we anticipate. Why is that? Part of the reason is not understanding the nuances...

50 mins
Mental Bookmarks and the Fractal Nature of Success

Good discussions are supposed to diverge from their intended path. Free association is a feature, not a bug, and helps...

180 mins
Design Principles for the Effective Developer

How many design patterns and principles can you name as developer? Are they important? Should we not rather focus on...

50 mins
Building Antifragile Teams

Antifragile systems thrive under stress and through failure. How can we help our teams – systems made up of people...

25 mins
Writing Professionally

The most important thing you do in your job is write. It's in every email you send, every commit you...