Big Value From Small Data – Guest Blog by John Tardy

Our friend and recent podcast guest John Tardy wrote this guest blog post expanding upon his discussion of focusing on Big Value not just Big Data. John is an expert in business analytics and we are delighted he is sharing his expertise with us. Read on to see why Big Data may not be necessary for your company to gain Big Value from your data!

Big data gets a lot of attention and is a game-changer for many use cases. Big data is not really a classification of data, but is about tools and techniques used to deal with data that is high volume, is generated at a high velocity, or has a high variety in its structure. Of course, there is a continuum along each of these dimensions and no definitive line between big data and normal (or “small” or “little”) data. Beyond the hype, the goal is always to solve some business problem and unlock value from the data asset, but as we move toward scenarios that could be described as big data we need to shift approaches in order to deliver practical solutions.

While the nature of big data sources provide new opportunities, the foundational analytical approaches and methods are not all that different. Instead, there are specialized technologies and techniques to deal with the nature of big data.

So, is big data better? Should we look for big data sources and big data opportunities to maximize value? While the hype points in that direction, the answer is no. Real value always starts with the business objective or challenge and responding to those needs may or may not involve big data.

There are use cases in HR Analytics that deal with big data.  Recruiting candidate portals involve many applicants, the opportunity to monitor click patterns and candidate experience, and unstructured data such as resumes.  Employee engagement efforts may involve a variety of data sources including email communications for analysis of the connections between different parts of the organization.

There are also many use cases that do not naturally deal with big data. For example, HR Analytics deals with a company’s workforce. There are few companies that exceed a million people in their workforce and none where the demographics for each person would be considered big data volume. Rather than big data solutions in search of a business problem, we should develop a data-driven culture by making the most value of the data we have.

Dependable results begin with a data strategy. We need to ensure that the business objective is clear, that we understand stakeholder responsibilities, there is a common understanding of what the data represents, and that the quality of the data will support the objective.

Once that foundation is established, what are some of the key ways that we can unlock value from data that would not be described as big data?

Reporting

It’s easy to skip over this, but business has been leveraging the value of reporting since the beginning. Reporting is the simplest form of transforming data into information that the business can use. That doesn’t mean that it is always easy. There can be challenges in collecting the data, the quality, or the way it is structured.

Visualization

Visualization takes our ability to gain information from data to another level. It allows us to explore more complex relationships and interactions between components of the data. It leverages our human capacity to understand and analyze information and relationships using our visual senses.

Statistical Analysis

Our intuitive analysis of visual information also has limits. We can only analyze what we see, or what we are presented, so the manner of that presentation can lead us toward incomplete or inaccurate conclusions.

Statistical analysis allows us to test a hypothesis with mathematical rigor. Is the trend we see statistically significant or possibly part of random variation? If we see a relationship between two variables, how strong is the correlation? With what level of confidence can we accept the hypothesis? These are techniques that are very impactful on even relatively small data sets as they help support a conclusion and give strength to a call to action (or not).

Machine Learning

Machine learning is really just more statistics and while it is often used with big data, we don’t need big data to use the approach effectively. Machine learning uses the data (and statistics) to determine the relationships between different variables. The algorithm learns these relationships, stores them in a model, and uses them for prediction in the future. This can be contrasted with explicitly coding business rules into a system. Using an algorithm to automate the learning of relationships that exist in historical data has huge advantages, and also some disadvantages. That is a topic for another blog post.

In the end, the value derived is not related to the size of the data source or the approach used to analyze it. It’s about using the right data and the right approach to answer questions that are relevant and actionable to drive business value.

John, thank you for sharing these insights with us! To read more from John Tardy visit his website, Unlock Data Value or connect with him on LinkedIn. Or listen to our recent podcast discussion with John here!