A Journey to Choose an Enterprise Big Data Platform


Duration: 25 mins
Rajib Deysarkar
Head of Enterprise Data, IG

IG has an on-premises Hadoop system to support its MIS and Data Science capabilities. However, with the changing landscape of technology and tools required by Data Science, there was an urgent need to revisit the Big Data capabilities of the organization, to check the pulse of the existing estate and prepare the runaway for the multi-million pound Data Science projects waiting to get started. This is where Rajib kicked off the need to clearly define the Big Data requirement of IG and propose the choice of a platform which will shape and support the Data Science capabilities of the organization.

This session is an attempt to share the journey which involved technical people from various backgrounds like Enterprise Architecture, Data Engineering, Data Science, Cloud Architecture, Platform Engineering, Big Data & Data Warehouse Expertise, multiple External Consultants and even business stakeholders. The journey took almost a year with serious challenges from various stakeholders, but the engagement ensured that the final outcome should be as inclusive as possible and with enough rationale to support the final suggestion.

Rajib believes this journey cannot be unique to IG and many other organizations might be embarking on the same journey. Your organization might well be one such.

You may also be interested in

50 mins
Top 10 Algorithms for Coding Interviews

So we are all the best coders, but have you see the run time of the code you are writing?...

25 mins
Writing Professionally

The most important thing you do in your job is write. It's in every email you send, every commit you...

50 mins
Imposter Syndrome: Overcoming Self-Doubt in Success

Impostor Syndrome is the domain of the high achiever. Those who set the bar low are rarely it’s victim. What...

50 mins
The Dao of Tech Leadership

Effectively managing humans requires a certain level of self-awareness. Therefore, understanding your WHY and what drives you is vitally important...

180 mins
Beyond Managing Your Manager

The deep-dive workshop presents why conflicts with your manager are inevitable based on differences in priorities and perspectives, and how...

50 mins
Mental Bookmarks and the Fractal Nature of Success

Good discussions are supposed to diverge from their intended path. Free association is a feature, not a bug, and helps...