How Big Data is Revolutionizing Business

data literacy

DATA IS REVOLUTIONIZING THE WORLD. IBM estimates that the world is producing 2.5 exabytes of data each day. That’s enough hard disks to cover more than six NFL football fields when laid out flat. McKinsey estimates there’s a talent shortage of 1.5 million data-savvy product and business managers. That’s larger than the population of San Diego.

Companies are taking advantage of the data revolution, combining big data, machine learning (ML) and artificial intelligence (AI) to find predictive and valuable patterns in their data to lead them to new products and services that people haven’t even dreamt of before. In a world where data defines so many products, data literacy is increasingly necessary for business leaders.

 

Data Products are Everywhere

In today’s data-rich world, our applications are becoming increasingly data-driven. Everything from executive business decisions to automated product recommendations has data at its heart. And we see them everywhere. 

Products like Alexa and Siri leverage AI to understand our voice commands and communicate with users. Netflix uses its user preferences and ML recommendation engines to provide personalized recommendations for TV shows based on what users with similar tastes and preferences like.

Facebook uses similar technology to populate its addictive newsfeed with stories from your friends that keep you coming back. Amazon employs its wealth of commercial data to offer product recommendations and targeted discounts for goods it knows you’ll want. Google uses all the data it’s harvesting from webpages, search histories and user behavior to find everything, from menus for local restaurants to scores for historical sports games. 

 

Leveraging the Right Data

It’s not just digital-born companies based in Silicon Valley that are leveraging data. We’ve worked with many non-digital-born companies that are learning about the value of their data.

  • Banks use their data to price loans better. Their strategies go beyond traditional FICO scores and credit histories to using more sophisticated ML techniques and novel data sources.
  • Insurance companies leverage telematics data to identify dangerous-driver behavior when offering auto insurance and data from wearables or gym membership attendance records to understand exercise habits when offering life or health insurance.
  • Pharmaceutical companies comb through electronic medical records to identify potentially lucrative off-label uses of their drugs.
  • Telecom companies are merging stores, call centers, websites, email and mobile data to improve digital confinement. Identifying unclear digital interfaces or communications that drive users toward costly call-center or on-site interactions.

 

More than a Trend

This is not just a passing fad. In a world where data defines so many products, data literacy is increasingly necessary. At its heart, the data revolution comes from three macroeconomic trends that show no signs of slowing down:

  1. The relentless drop in the cost of computation. This is not confined to processing; it also applies to memory, hard disk and network bandwidth. This means that data is becoming increasingly cheaper to store, transfer and process–making the analysis of data feasible.
  2. As technology becomes more ubiquitous, more data is being captured. We are collecting data when people surf the internet not only on their computers, but also on their mobiles, tablets and wearables. And with the growth of the industrial Internet of Things (IOT), we will be collecting data not just from humans, but also from machinery all over the world.
  3. Data and AI are becoming competitive issues. As more companies are embracing data and AI to be smarter and more personalized, consumers are beginning to demand it as table stakes.

 

Incorporating Data Into Products

Data has fundamentally altered the way product managers need to think about their products. When we consider how personalized and addictive Facebook’s newsfeed is or how Amazon is able to recommend the next product you want to buy, we can understand the business value of embracing data.

At a superficial level, embracing data appears like a move toward evidence-based decision making. A move away from HiPPO (Highest Paid Person’s Opinion) decision-making toward relying on proven results. But using data in products goes far beyond that. It subtly shifts to a very different way of thinking about products.

Since grade school, we’ve been taught to apply rules of logical deduction to our reasoning and decision-making. Unsurprisingly, such reasoning has been codified into our business rules: Deliver the product to the customer once they’ve paid for the product. Grant existing customers a 10 percent loyalty discount for continuing with our product.

 

A Data-Driven World

In a data-driven world, these business decisions become much more fluid. We don’t use data simply to confirm business rules but to generate and optimize them. 

For instance, by leveraging data, we might understand how to optimize customer discounts to increase sales and revenue. We can give selective discounts on products that lure customers back or offer time-sensitive discounts to induce customers to buy as soon as possible. 

None of these ideas are new in retail, but each has a myriad of accompanying decisions, such as: 

  • How deep is the discount? 
  • On what products? 
  • To whom should we offer a discount? 
  • When should we offer a discount? 
  • When should we stop offering a discount?

By leveraging data, your strategy can continuously optimize these parameters—and many more. It can track trends to adapt to seasonal changes, discover product groupings or personalize for customer preferences. And we leverage these ideas to constantly optimize for revenue.

Rather than starting with what a product should do, product managers define what the objectives are and where the boundaries are, and then let the underlying software optimize within those bounds.

This frees up managers from thinking about the mundane day-to-day features and from meetings to decide on small product features, instead allowing them to think more strategically on:

  • How to measure success
  • New types of data to incorporate into decision-making
  • The strategic direction of the industry

 

A Change in Mindset Around Data

Much of today’s data gold comes from what was previously “data exhaust,” collected data that’s a byproduct of other business processes. 

For example, programmers used to maintain weblogs. These logs were kept around for a few days and were not well-structured. Their original purpose was for programmers to comb through customer complaints of bad interactions so they could find mistakes in their code. 

Ingenious managers realized these logs were a potential gold mine of data about their customers, telling them things such as:

  • What pages are people visiting, and in what order?
  • Are they using the product as we intended or clicking around in new or unexpected ways?
  • Can we use this data to improve our site layout or even build a recommendation engine to personalize pages for customers?

This new way of thinking requires a massive mindset shift. Rather than seeking data to answer their questions, they’re looking for the questions to pose their data.

These trailblazers began leveraging data, turning what was previously considered trash into one of the most critical competitive advantages a company can have. Identifying and harvesting value from data is the heart of how organizations drive value using data.

The pharmaceutical industry is an excellent case in point. As an industry with long-established business practices and complex government regulations, it would seem like an unlikely place to find innovative approaches to data. 

Traditionally, data in pharma have been geared toward passing multiple rounds of FDA clinical trials. The problem was clear: demonstrate the safety and efficacy of your treatment with statistical significance. The data was collected specifically for this purpose. Human test subjects were enrolled, given a treatment (or not), and then monitored for either improvement in the target condition or adverse side effects.

This data wasn’t cheap to collect—the cost can run into billions of dollars. Once the data was collected, the analysis was relatively easy. The pharma companies that are leaders in data have flipped the paradigm. They’re combing through their data exhaust and those of other medical services sniffing for cheap data that might be valuable. 

The questions they’re posing aren’t always clear or well-directed. Instead, data professionals and product managers need to understand how to leverage data science techniques to answer valuable business questions based on the data they can find.

 

Data-Literate Organizations

This mindset shift means there’s a new set of skills to master as they become better versed in data science, machine learning and artificial intelligence. As data starts to play a fundamental role in the market, organizations face an increasing need to become data literate. Below are a few questions to get your started:

  • Do you understand the difference between structured, semi-structured and unstructured data?
  • How are today’s data analytics different from their forebears?
  • How do business leaders go about cataloging their own data?
  • Once leaders understand what data they have, how do they envision potential products to be delivered in a pilot and then iterate?
  • And once pilots have confirmed value, how do they think about a product to make it robust?
  • How do business leaders scale the impact of data across an entire organization?

 

Are you making the most of your organization’s data?

An organization that leverages data strategically can uncover new opportunities, gain a competitive advantage and boost return on investment. Take the Pragmatic Institute Data Maturity Assessment to discover where your organization falls on the data maturity continuum and start building a data-driven culture. 

Take Assessment 

 

Continue Learning 

The world of data is moving fast, don’t get left behind. Our Data Science for Business Leaders course empowers leaders to leverage data in decision making and strategy. Learn how to partner with data professionals to uncover business value and make informed decisions with the data at hand. 

Learn More

Author

  • Michael Li

    Tianhui "Michael" Li, a seasoned data scientist with 22 years of experience, has left an indelible mark at NASA, Intel, Google, Bloomberg, and more. As a 2X entrepreneur, he has contributed to the tech landscape with roles at Andreessen Horowitz, Foursquare, and Cornell Tech. Michael's expertise extends to The Wall Street Journal, VentureBeat, Harvard Business Review, The Data Incubator, and Pragmatic Institute. For questions or inquiries, please contact [email protected].

    View all posts

Most Recent

Article

The Data Incubator is Now Pragmatic Data

As of 2024, The Data Incubator is now Pragmatic Data! Explore Pragmatic Institute’s new offerings, learn about team training opportunities, and more.
Category: Data Science
Article

10 Technologies You Need To Build Your Data Pipeline

Many companies realize the benefit of analyzing their data. Yet, they face one major challenge. Moving massive amounts of data from a source to a destination system causes significant wait times and discrepancies. A data...
Article

Which Machine Learning Language is better?

Python has become the go-to language for data science and machine learning because it offers a wide range of tools for building data pipelines, visualizing data, and creating interactive dashboards that are smart and intuitive. R is...
Category: Data Science
Article

Data Storytelling

Become an adept communicator by using data storytelling to share insights and spark action within your organization.
Category: Data Science
Article

AI Prompts for Data Scientists

Enhance your career with AI prompts for data scientists. We share 50 ways to automate routine tasks and get unique data insights.
Category: Data Science

OTHER ArticleS

Article

The Data Incubator is Now Pragmatic Data

As of 2024, The Data Incubator is now Pragmatic Data! Explore Pragmatic Institute’s new offerings, learn about team training opportunities, and more.
Category: Data Science
Article

10 Technologies You Need To Build Your Data Pipeline

Many companies realize the benefit of analyzing their data. Yet, they face one major challenge. Moving massive amounts of data from a source to a destination system causes significant wait times and discrepancies. A data...

Sign up to stay up to date on the latest industry best practices.

Sign up to received invites to upcoming webinars, updates on our recent podcast episodes and the latest on industry best practices.

Subscribe

Subscribe

Pragmatic Institute Resources