Data Streaming Important IT Investment, Delivers 2-5x ROI For 79% Indian Businesses

A new report on data streaming revealed 71 per cent of Indian organisations power their critical systems with data streaming and it delivers a two to fivefold return on investment (ROI) for 79 per cent of these businesses.

Driven by the increasing need for real-time insights and the proven value of the technology, data streaming is now viewed as a top priority for a majority of companies, according to Confluent’s Data Streaming report.

In India, 78 per cent of IT leaders report their organisations are seeing significant or emerging benefits from data streaming when it comes to faster strategic decision making and 79 per cent report improved customer acquisition and retention. Additionally, 80 per cent of organisations reported a faster product/service time-to-market.

"According to our report, data streaming is widely adopted across businesses in India, delivering significant ROIs, and enabling a multitude of use cases that enhance security, agility, and exceed customer expectations. The impact of data streaming is evident and we are excited to continue this momentum in India," said Rubal Sahni, Area VP and Country Manager for Confluent in India.

Globally, the Financial Services industry leads the investment in data streaming with 93 per cent of IT leaders in the industry citing data streaming as one of their top strategic priorities or an important area of investment. This was followed by technology (92 per cent) and the construction industry (90 per cent).

Given the trend in investments, 53 per cent of Financial Services leaders are seeing five to tenfold ROI. Other industries seeing high returns are technology (51 per cent), construction (46 per cent) and transport and logistics (45 per cent).

Netflix Senior Software Engineer Tejas Chopra told BW Businessworld that data streaming is critical to financial systems. "Data is generated at multiple sources today, and there is a need for surfacing it at the right place at the right time. Furthermore with ML and AI, you want to process massive amounts of data. Data streaming allows a framework to bring data from these disparate data sources together, and therefore its importance will only grow," he explained.

In fact, data streaming has started a new paradigm in computing called event driven architectures. "Many companies such as Netflix build their microservices on these architectures," Chopra added.

Data Streaming’s Biggest Challenges

As companies continue to expand their data streaming use cases, many struggle with scaling, operational challenges and organisational silos.

Around 72 per cent of IT leaders said that fragmented projects and uncoordinated teams and budgets can be a challenge or a major hurdle to advancing data streaming, and 78 per cent cite a lack of relevant skills, expertise, and experience. Additionally, 95 per cent of respondents said training and recruitment to bolster skills is a high or medium priority as they continue to invest in data streaming over the next 12 months.

“Firstly, there is a lack of understanding on how to effectively leverage streaming technology and its existing capabilities. Secondly, organisations face challenges in operating Apache Kafka (data streaming platform) at a large scale. As Kafka becomes successful within an organisation, questions arise about scaling it to handle an increasing number of streams from various sources,” Shaun Clowes, Chief Product Officer at Confluent told BW Businessworld.

Apache Kafka, an open-source event store and stream processing solution is used by more than 80 per cent of Fortune 100 companies. 

“Kafka, being a distributed system, introduces complexities when it comes to scaling. Particularly in the context of a large-scale organisation like a bank in India, scaling Kafka becomes a daunting task. It requires a significant investment of time and effort to gain a comprehensive understanding of Kafka's operations and how to effectively manage it,” Clowes added.

91social Founder and Architect Sunil Guttula believes skills gap comes into the data streaming picture in two parts.  

"The first part is to define the data streaming problem well and figure out which streaming technology is a good fit and how to use it. It is important to get this right either through in-house experts or engaging with a credible service provider," Guttula explained. 

"The second part is in keeping the technology working well in production. While an in-house team can be the first choice and in some cases the only choice, there are managed providers with this expertise and can be leveraged upon," he suggested.