A Journey to Choose an Enterprise Big Data Platform


Duration: 25 mins
Rajib Deysarkar
Head of Enterprise Data, IG

IG has an on-premises Hadoop system to support its MIS and Data Science capabilities. However, with the changing landscape of technology and tools required by Data Science, there was an urgent need to revisit the Big Data capabilities of the organization, to check the pulse of the existing estate and prepare the runaway for the multi-million pound Data Science projects waiting to get started. This is where Rajib kicked off the need to clearly define the Big Data requirement of IG and propose the choice of a platform which will shape and support the Data Science capabilities of the organization.

This session is an attempt to share the journey which involved technical people from various backgrounds like Enterprise Architecture, Data Engineering, Data Science, Cloud Architecture, Platform Engineering, Big Data & Data Warehouse Expertise, multiple External Consultants and even business stakeholders. The journey took almost a year with serious challenges from various stakeholders, but the engagement ensured that the final outcome should be as inclusive as possible and with enough rationale to support the final suggestion.

Rajib believes this journey cannot be unique to IG and many other organizations might be embarking on the same journey. Your organization might well be one such.

You may also be interested in

180 mins
Design Principles for the Effective Developer

How many design patterns and principles can you name as developer? Are they important? Should we not rather focus on...

25 mins
Developer is 'King' - Unleashing Innovation by Unblocking your Developers

As each industry is disrupted by the wave of digital transformation, harnessing and unlocking new ideas can only be done...

50 mins
Building Antifragile Teams

Antifragile systems thrive under stress and through failure. How can we help our teams – systems made up of people...

50 mins
10x productivity for Developers and Architects

Productivity is key to success in software development. We will be exploring different principles, so you do not have to...

25 mins
Dealing with Noobs: How to Work with Non-Technical People

The secret to being the hero of your story? Helping others be the hero of theirs. Take a business problem you...

25 mins
How to Validate your Startup Idea Quickly

I work at Facebook's Innovation Lab as an Engineering Lead. As part of that, I help with rapidly prototyping and...