<p><em>The silo approach, going it alone like the sheriff facing off against the outlaws in a Hollywood Western film, may work in the movies, but not in Big Data, writes <strong>Moshe Kranc</strong></em><br><br>In my position as Chief Data Officer for Ness Software Engineering Services, I talk with many companies about their Big Data challenges and needs. Some of these companies have chosen to analyze their Big Data to enable use cases such as online personalization or creating a 360-degree view of their users. Others have been forced into this area by one of their competitors, who announced that, thanks to their successful Big Data initiative, they now possess a competitive advantage over all their competitors; e.g., their products require significantly fewer service calls because they can monitor the performance of the equipment remotely.<br><br>If there is one common thread that links 95 per cent of these customers, it is FUD - fear, uncertainty, doubt. The field is full of competing or overlapping products, each of which claims to be the right solution for Big Data. The Big Boys (Oracle, IBM, SAP, Teradata, etc.) all want to steer you towards their (sometimes costly) offering. The upstarts (Cloudera, HortonWorks, DataBricks, DataStax, etc.) all talk about use cases with dramatic savings, but you may not be familiar with the use cases in which their product isn't designed to work as well. And, depending on your vertical, there are dozens of one stop shops who offer to take your data and come back with vertical-specific insights.</p><table style="width: 200px;" align="right" border="1" cellpadding="3" cellspacing="3"><tbody><tr><td><img alt="" src="http://bw-image.s3.amazonaws.com/Moshe-Kranc.jpg" style="width: 200px; height: 248px;"></td></tr><tr><td><span style="color:#696969;"><em><strong>Moshe Kranc</strong></em></span></td></tr></tbody></table><p>Left to evaluate this cacophony of conflicting voices is your organization. It's hardly a fair battle - you may not know the hard questions to ask. That company that promises 20,000 writes per second? That's only true if only one process is writing the data - the performance tanks when there are multiple writers. That product that claims to support full SQL? They consider little things like JOINs, transactions and nested queries to be out of scope.<br><br>No wonder we read so often about Big Data failures. "Product X let us down," announces the CTO of yet another large financial institution that made the mistake of believing the vendor hype. The real culprit is not Product X - the real culprit is the silo approach that drove the company to make a choice without the benefit of outside advice or experience. The only way to cut through the hype around a product is to try it yourself, and/or talk to someone you trust who is using it.<br><br>The best advice I can give to a company contemplating a Big Data initiative for the first time: Get help. Perhaps you can hire experts from outside your organisation who have already built such a system. Perhaps you can build up a network of friends who have made the beginner's mistakes and can share their insights with you. Another option is to partner with a company like Ness Software Engineering Services that has managed a broad range of Big Data projects and technologies, and has a proven track record of success. The silo approach, going it alone like the sheriff facing off against the outlaws in a Hollywood Western film, may work in the movies, but not in Big Data.<br><br><em>The author is chief data officer at Ness Software Engineering Services</em></p>
 (1)_20241001214828_original_image_25.webp)
