BigDataBig Data management and Advanced Analytics are at the center of a productivity transformation with real benefit across our economy. This new data science should transform our ability to improve forecasts, prioritize investments, boost productivity, extend the life of capital assets, find ways to better satisfy our customers and even improve our overall quality of life. The hype around Big Data is legitimized with a flow of stories ranging from enhanced farming practices, to traffic management, to industrial productivity as operational processes are being revolutionized to create new levels of economic value.

But the Big Data revolution and the associated awareness, expertise and tools should also drive a new level of awareness and investment in what I would call “Little Data”. In all of our businesses we have our own essential operational data that is critical to managing our transactions, our operations and our overall business performance. Now if we take the use of our “Little Data” to new levels by applying some of this new data management discipline, normalization and flexible data analytics from the Big Data world we can develop dramatically improved insight into our businesses.

Today we have data for sales, inventory, salaries, bonuses and purchases. We have data about our customers, our partners, our channels, our markets and our competitors. We have data on our productivity in manufacturing, engineering, service and sales. We follow our web page hits and SEO status versus our competitors. We have data on employee results. And it is all important. But, for the most part, it is isolated data about pieces of the business. Snapshots.
When we look at the transformational advances being made with Big Data analytics, it seems right to ask this problematic question: Are we getting all we can from our “Little Data” and smart business analytics? I agree that the problem space and the insights we hope to discover with Big Data analysis is different. But really getting our arms around everything we have at hand in managing our businesses has to be worth the effort. Can we better figure out what products to focus our investments on? Based on our data, can we create sales incentives that actually work? Can we better segment our markets? Identify our most profitable customers? Determine what marketing investments actually pay off? I think that the answer to these questions for most of us is yes. So what is the problem? Why aren’t we doing it? We aren’t lazy or stupid. So what is it?

From my perspective there are three main problems that we need to solve. First, for most of us, our data is spread across multiple systems and it seems just too disjointed to use in any holistic analytical way. Second, most of us have not equipped our businesses and organizations with the right skills, training, tools or experience to really leverage data management and analytics. And third, most of us have not created the organizational discipline to react to real time learning with agile adjustments across our functional groups, divisions or overall business based on what we discover.

Our data comes from multiple databases, created for multiple purposes and is organized for different uses. We have data from Salesforce.com, our MRP systems, our finance, invoice and purchasing systems and multiple other sources.  Any attempt to pull all this into one meaningful view immediately results in naming aliases, duplicate records, differences in data organization, and even conflicts in what the data means. So, transforming our multi-source “Little Data” into a whole to reflect all of the dimensions of our worlds is a big challenge. We have to start by figuring out the specific information we might need in our new analytics and which business system is the best “source of truth” for each dimension. Then we have to consolidate and normalize the data so it makes sense as a whole and is internally consistent. We also absolutely have to automate this. We can import data from multiple tools into one “master” tool that provides the best utilities to operate on the data. Or we can export from all sources to a new database and operate on the consolidated data with programs and automation to normalized it. But without a goal of automation, we are likely setting up a one-time run at business insight.

Next, what do we do when we have the data normalized? For many organizations, data analytics amount to standard graphs that are routinely generated to indicate performance against goals. We look to the finance team to give us year over year or month over month trends in sales, profits, regional performance, and results by business unit, or segment. Sales leadership will generate separate reports that show metrics like pipeline status, conversion rate, churn and sales person productivity. From Engineering, we see project milestones and defect closure rate. From Manufacturing, we see COGs, quality and productivity trends. But we are often ill equipped to look at how this information inter-relates, perform what-if scenarios dynamically, learn from this and then dig deeper. And most of us are not very well equipped to look at cross functional interaction and how the different business functions truly align (or don’t) to drive our overall success. This is no easy trick and will take practice. But it will pay off.

It is important to recognize that if we don’t have good data to work with, it is very hard to do any meaningful analysis anyway. As a result, most teams have only developed marginal data analytics capabilities. To make the transition, we need to drive an expectation that all our teams will use data analytics to improve the business. This needs to apply to all functions. Great places to start are with cross-functional collaboration between product management, sales and marketing to create data driven insights, or between engineering and operations on COGS and quality, or between all functions and HR on talent. The CEO or GM needs to “walk the talk” and engage in pulling this all together into “analysis and insight” sessions to make meaningful course corrections to create value.

Bringing this about will take some adjustment. We need to invest across business functions and train up staff on basic data modeling tools like (excel) pivot tables but to get good at this we need more advanced data modeling and visualization tools like Tableau, Qlik Sense or the many other platforms available. Different functions in your organization will likely benefit from different types of analysis, but you should identify a common set of tools and build from there. Establishing a few experts with complementary skills to leverage across the business is very helpful. Adding an expert from IT for application skills, from Marketing for market insights and from Product Management for business intelligence will usually make the most sense. There are a lot of great YouTube videos to help your team become adept at pivot tables and basic trend analysis tools, and this article by Sharon Machlis provides a great reference to 22 different tools that can be used for interactive analysis and data visualization.

All of the buzz on the value of analytics from Big Data may already have us convinced of the value of making this leap to improve efforts, but let’s look at just a few likely results:

  • Sales growth through sharper segmentation, customer needs focus and trend awareness
  • Much clearer contribution assessment of by-product / by-region / by-channel sales efforts
  • Improved R&D success rate and ROI through understanding customer and market trends
  • Faster bottom line growth through identification of Pareto COGS and quality targets
  • Major risk identification such as supplier, customer, segment or region concentration

The fact that most businesses are just now making these changes to better manage their data, normalize it and use data analytics to drive success should really come as no surprise. For years, we have been embracing new tools such as Salesforce.com, Oracle, SAP, and many other MRP / ERP operations and quality management systems and they have profoundly helped us drive value. But a very useful side effect of this is the sea of data we now have about our customers, markets, suppliers, and our operations. Harnessing this data, normalizing it and analyzing it to improve our operations is the natural next step, but our organizations need to evolve so that we can make this happen.

As leaders we need to take the time and energy to upgrade our operations so that our data is automatically collected and normalized, our staff is trained and equipped to manage it and our business stakeholders creatively engage in analysis as a team. And we need to respond to the insights with agility. In short, we need to Collect the Data, Manage the Data and Analyze the Data! And then we can use the “Little Data” from our operations to Optimize our businesses. From our “Little Data” insight we can appropriately tune our strategies, make better decisions and create new value for our stakeholders, our markets and our customers.