Searching for Time Series Patterns at Big Data Scale


Duration: 50 mins
Ananth Gundabattula
Senior Architect, Commonwealth Bank of Australia

Time series patterns are pretty common in everyday life. Extracting patterns on these data sets and providing interactive search responses has immense value across many software systems. This talk presents an overview of how such a search system can be built for very large time series data points.

This talk focuses on the STAMP algorithm (Scalable Time Series Anytime Matrix profile) to showcase how time series patterns can be extracted for big data sets. The talk covers concepts of Matrix profile and how motifs/patterns are detected. Once the patterns are analyzed, the results are converted into a document format that is amenable for indexing via a search engine. The session then proceeds to extend the idea to detecting time series across multiple dimensions. A part of the session also covers the distributed approach to extract these time series patterns to tackle big data sets. Further, you will also learn about mechanisms to calculate time series segmentations, time series chains and discords or anomalies thereby enabling ideas for users to search for anomalies and segments of time series.

Dask, Stumpy and Elasticsearch stack is used to describe a reference implementation.

You may also be interested in

25 mins
Developer is 'King' - Unleashing Innovation by Unblocking your Developers

As each industry is disrupted by the wave of digital transformation, harnessing and unlocking new ideas can only be done...

180 mins
Foundations of Tech Leadership

According to a CareerBuilder study, only 40% of new engineering leaders receive formal training when they become a boss for...

25 mins
Designers + Developers = Best Friends Forever?

How is the relationship between your design team and your development team? Is it highly functional? Or 'just professional'? Maybe...

50 mins
Do You Know Da Wae

We build development teams based on individual ability to write code but development of a software project of any significance...

50 mins
Empowering, Growing and Coaching your Superstars

Nothing is more rewarding than when your leaders or teammates help you reach your goals or the next level of your career. ...

50 mins
On Being an Effective Developer

As developers we not only operate in different contexts, but also often have these different contexts interplay as part of...