Define It! Big Data vs Small Data

Deluge of “sexy” data terms got your head spinning? Here’s what you need to understand about two important ones.

 

The evolution of technology has unlocked new levels of data exploration, and one of the most exciting is Big Data. Essentially the term Big Data refers to both the size of data, and the structure of the data. A few things you should know before we dive deeper: first, our world has generated more data in the last two years than in all of human existence prior. So lots of data is available. Second, advances in computing architecture and affordable storage means we can capture anything and everything, no matter how trivial it may seem. Last, we know that of all the data we collect, 80%-90% of it has yet to be used for any effective purpose, meaning it’s a race to uncover the value.

 

What can we learn from analyzing finger swipe patterns on a smart phone? How can we scan millions of product reviews for hidden insight that helps us develop the next great product? Can we detect consumer sentiment in real time across millions of tweets and feed complimentary  marketing direction to those very same customers just minutes after they post?  These are the questions being asked of Big Data. 

 

And it’s not just the questions that are new – the data itself is also very different than traditional data tables like Excel spreadsheets or SQL databases. Big Data can be anything: a picture, a tweet, a PDF, a sensor reading, etc. This data is collected and stored together, with non-traditional organization techniques and requires new skills to analyze and interpret. Common tools include Natural Language Processing (NLP) which translates text into meaning or Machine Learning, which uses myriad algorithms to interpret and predict outcomes from large data sets. Other terms you may have heard like NoSQL (a database for unstructured Big Data) or Cloudera (a Big Data platform) also fall into this category.

 

In a nutshell, Big Data is a trailblazing way to capture, analyze and make use of the estimated 44 zettabytes of data available at the start of 2020 according to the World Economic Forum.  That’s 44,000,000,000,000,000,000,000 bytes. We’re talking about every smart lightbulb that turns on, every fantasy football score request from Siri, etc.

 

Ok what about Small Data? Is that a thing? Yes! Consider this the less sexy, more manual, dare I say more universally applicable little brother.  

 

Small Data refers to smaller, well structured, and similarly formatted data sets. This should not be too foreign to business laymen as we all work in some form of accounting system or CRM. These systems collect data in fields and store them in tables and have the ability to connect related data, like orders to customers or shipments to products. This data has been around for all of the modern computing age, and it has become more sophisticated and powerful with time. Now, even your local barbershop has access to sales, appointments and website visits in digital form. Small data could also be called “Ready Data” as it is packaged and organized in a ready-to-use way. If Big Data is raw, Small Data is refined.

 

But that doesn’t mean Small Data is out and Big Data is the new king. Big Data is an entirely different approach to data application and requires resources and opportunities well beyond those currently available to most companies. Ironically what we now call Small Data is made up of exponentially larger and more diverse data sets than any person can manually manage. A data ecosystem that links procurement data with customer data and warranty claims is immensely beneficial, even if it’s considered small.

 

The danger here (caution, we’re entering the good-natured opinion zone) is that companies can feel pressure to enact a Big Data strategy when they have yet to successfully deploy a Small Data strategy. Is your data accurate? Is it clean? Is it up to date? Is it in one place? Is it accessible? Do you have tools to analyze the data? Are metrics a part of internal assessment and decision making?  Is data being used by the organization in a meaningful way? Even talking about Big Data or predictive analytics before you have those Small Data components in place is a bit like building a house with no foundation.

 

Small Data is ever-present, yet under-utilized. Big Data is truly sensational, yet more narrowly applicable. When determining your organization’s data strategy, it is important to identify the most appropriate approach to meet your objectives, even if it is a little less sexy.

 

The Define It! series provides short explanations for common data practices, written for general business understanding and edited for IT-lish.

More from the Define  It! Series:

• Define It! Probabilistic (Fuzz) Matching

 

 

Elevation Data Group is a managed data services and consulting firm helping businesses put their data to good use.