You may have heard of the term ‘big data’ – it’s one of those technologies that some people are making a lot of noise about. It’s been written about in many publications, e.g., Harvard Business Review blog, Forbes, MIT Sloan Management Review, McKinsey Quarterly. However, as one analyst has commented, the hype about it is reaching “the levels of SOA in the early 2000’s, cloud in the late 2000’s, and social in the past few years.”
A nice simple definition of big data is this:
The main characteristics of big data are described as:
“Volume – This characteristic describes the relative size of data to the processing capability … Overcoming the volume issue requires technologies that store vast amounts of data in a scalable fashion and provide distributed approaches to querying or finding that data …
Velocity – Velocity describes the frequency at which data is generated, captured, and shared. The growth in sensor data from devices, and web-based click stream analysis now creates requirements for greater real-time use cases …
Variety – A proliferation of data types from social, machine-to-machine, and mobile sources add new data types to traditional transactional data. Data no longer fits into neat, easy-to-consume structures …”
So who is using big data? Forrester Research indicates that the businesses using big data are the ones that have always handled lots of data in the past – e.g., banks and insurance companies, telcos, oil companies, large retailers, and the international CPG companies. An analysis by McKinsey defines those industries where big data has high value vs. those where its value is low, and also indicates that currently the value of big data isn’t that great for manufacturing businesses. The same McKinsey article mentions that the analysis was done for businesses with over 1000 employees, but if you look at the US Census bureau’s 2010 statistics for business you will see that businesses of this size make up less than 9000 (0.15%) of the total of over 5.7 million businesses in the US.
How relevant is big data? It seems that most of the hype is coming from database vendors and consultants, which you might expect. A recent article had the following advice:
“One of the key lessons from the history of marketing science is that when a new data source becomes available, everyone is quick to fall in love with it. But smart companies take a step back and strive for a more holistic view of their customers and markets.”
Is big data appropriate for small- and mid-size business (SMB)? Some comments:
From another external perspective, TechnologyEvaluation.com some very complementary things to say about SYSPRO:
“The company develops its own nifty gadgets and functionality, but it often waits for Microsoft’s products to get mature (i.e., Azure, SharePoint, Windows 8, etc.) and then it comes up with some great tools on top of it, often outdoing Microsoft…big data is also in SYSPRO’s roadmap, and my guess is that it is again waiting for Microsoft’s in-memory analytics technology to mature in SQL Server.”
Finally, what could make big data become more important to the SMB market?
These conditions could arise with the growth of the Internet of Things. This refers to how objects are becoming embedded with sensors and linked through networks, often communicating via the Internet. This could create huge volumes of data that flow to computers for analysis, and could therefore have an impact on SMB manufacturers and distributors.
If you are part of an SMB organization, is big data on your horizon? We would be very interested to hear from you.
