Enter the Industry 4.0 Era Today by Using “Dark Data” You Already Have

by Gregory Gimpel and Allan Alter

The Industrial Internet-of-Things (IoT), also known as Industry 4.0, provides new opportunities to harness data-driven insights to improve business performance. In reality, operational technology, production equipment, and other “things” already provide data that can improve – even transform – company operations. By answering four key questions, business leaders can benefit from this “dark data” today.


Enter the Industry 4.0 Era Today by Using “Dark Data” You Already Have

The Industrial Internet of Things (IoT), also known as Industry 4.0, provides a new opportunity to improve business performance through data-driven insights. A quick Internet search provides many details about the transformative potential of the IoT. Wireless sensors on patients will help hospitals to predict heart attacks before they happen, enabling preventative emergency measures.1 Companies will equip employees with uniforms that detect hazards as a way to cut down on industrial accidents.2 Sensors in roadways, sidewalks, and mass transit systems will manage traffic flow in real time, minimising traffic and the lost productivity and environmental cost associated with it.3 IoT-enabled supply chains will become adaptive, self-organising, and super-efficient networks.4

Building the new IoT infrastructure will take years and many companies that can benefit from the IoT are not prepared to use the technology.

Building the new IoT infrastructure will take years and many companies that can benefit from the IoT are not prepared to use the technology.5 Fortunately, companies can enter the IoT era today by using the “dark data” created by supply chain operations, production equipment, operational technology, and other existing “things”. Presently, over 90% of these data are dark, providing a rich opportunity for companies to begin exploiting these information resources today. 

Our interviews with senior executives suggest that companies can begin to take advantage of the opportunities enabled by dark data by asking four questions. What data are being generated? What problems can I solve with the data? Should we use the data? How do I motivate the necessary investments? The answers will provide a guide for companies to use existing information resources to enter the IoT era now. 


1 What data are being generated? 

Although Big Data and the Internet of Things have become hot topics of discussion, those who want to analyse IoT data must first identify what data are being generated within their processes. In many cases, managers are only aware of a small fraction of the total data generated by operational technology and other equipment. The CTO of an Industrial IoT application provider explains: “most of the dark data we’re probably not aware of [is dark] simply because historically we have not been looking for it. But the fact that it is dark data, it’s data we weren’t thinking as having value, we weren’t cataloging it or indexing it.” Different functional units within an organisation may be unaware of the data generated by their sister units. The managing director of an analytics service firm paints an all too common scenario: “I remember a customer being with a client and their IT department, and the client said, ‘Well, if I had X & Y, I could do that.’ He then explains the IT department’s answer: ‘Yes, we know we have this. We have the data. Why don’t you ask us?’ Well, the line of business is not going to ask what they don’t know.”

Sometimes, it is a classic situation of “not seeing the forest because of all the trees.”  Employees know about the data and use it every day.  But because they view it as common, they don’t recognise that it is a form of IoT data. 

Sometimes, it is a classic situation of “not seeing the forest because of all the trees.” Employees know about the data and use it every day. But because they view it as common, they don’t recognise that it is a form of IoT data.The managing director of a consortium of IoT companies shares how common this is in the oil refining business: “There is a significant amount of intelligence, information, or data from a petrochemical refinery that would be considered instrumentation of the physical world. Being in the middle of it could be blinding me to the possibilities of how it’s different.”

[ms-protect-content id=”9932″]

Our research finds that data audits are effective solutions for breaking down information silos and for bringing overlooked data to light. A related strategy is to start charging for data storage, which can motivate business units to take a close look and reflect upon what kinds of data they are creating. 

Companies need to educate employees that much of the IoT is old technology that they already use, not some mysterious innovation, and that the information it produces is IoT data. Sometimes, people may need an extra incentive to identify the data they generate in their operations and other processes. For example, a deputy CIO who we interviewed suggests that companies create games in which people identify data, contemplate its value, and are rewarded for finding new uses for existing data.

Straightforward discussions with techs and data scientists will help business executives understand the potential benefits they can achieve by leveraging the assets they already have.

2 What problems can I solve with the data?

Even when organisations know that they are generating IoT data, they may not understand which challenges the existing IoT data are well suited to solve. The CEO of a Healthcare IoT firm explains: “People don’t understand what to do with the data – they’re not educated enough.” He explains that many companies do not know how to leverage simple website data and information from basic forms, let alone IoT data.

IT people and data scientists need to explain to business executives what can be done with their existing data. These conversations need to focus on the business case – presenting data and technology as tools for helping solve business problems, not as ends in themselves. The managing director of an IoT technology lab explains the need to frame conversations around business issues rather than technology:

“At the end of the day it’s always about… a handful of things. It’s always about improving the product or service experience for your customers. Doing your job better. Doing it at a lower cost or being able to manage that cost so that you have fewer surprises. Enhancing your ability to drive revenues, primarily through existing services, with new or existing customers.” 

Straightforward discussions with techs and data scientists will help business executives understand the potential benefits they can achieve by leveraging the assets they already have. The CEO of a Healthcare IoT firm explains: “They [the business executives] need to start knowing what’s possible so they start dreaming bigger to create better services and products.”


3 Should we use the data?

Even when organisations know that information exists and have a strategically relevant plan for analysing the data, they still may lack the legal right to use the data. Some firms may have permission to use it only for specific purposes, cutting off the opportunity to repurpose the data to generate new insights. Others may be precluded from using the data at all. Such has been the case, at least until very recently, with heavy equipment. The original equipment manufacturers (OEM) have been embedding connected sensors for decades.They have full access to the data generated from the machines they sell. But what about the users of the equipment? Only recently have some of these end users been granted access to the data that they generate. 

Consumer-facing companies must consider customer responses to using their data. The senior data scientist at a leading consumer products company explains how concerns about customer trust have influenced their decision against using dark data generated by their consumer products: “First and foremost is the consumer trust. We have to earn that. So we can’t do anything with the data that comes anywhere close to something that would violate any privacy…” Companies may decide that the potential loss of trust outweighs the potential gains, or conversely, they may decide that they can manage the risk, thus benefitting from deeper customer relationships resulting from dark data. 

Laws differ country by country. Technology advances faster than the law. Regulations are in flux and changing rapidly. Even when laws like the GDPR aim to provide standard rules, the exact meaning of such regulations may rely on court precedents and future case law as much as the actual letter of the law. As a result, the legality of using a certain set of dark data can remain unclear. The director of an analytics think tank explains how the uncertain legal environment creates a chilling effect on the use of dark data: “I don’t want my executive to end up in court because they used somebody’s data the wrong way.”

Our research suggests ways to address trust concerns. To build trust with consumers, avoid lengthy user agreements. Instead, use short terms-of-service written in plain language, explaining clearly how data will be used and stating the ways it will never be used. In business-to-business environments where collaborators could potentially use partner data against each other, enlist trade associations to aggregate and analyse data to provide insights useful to all members. This can provide data sharing benefits while preventing unauthorised use of proprietary information.

When faced with legal concerns, companies can take a conservative approach. They can play it safe by basing policies for every territory on countries with the strictest data regulations. On the other hand, companies can actively utilise the data today to create strategies and policies that will remain useful if laws change and preclude firms from collecting data or using it in certain ways in the future.


How do I motivate the investment?

The desire to exploit current opportunities can outweigh the motivation to invest for tomorrow – including data that will create future opportunities. Successful companies may be using all their resources to meet current customer demand. They simply do not have the slack human or financial resources needed for new types of data analysis or new data-driven initiatives. The managing director of an IoT technology lab elaborates: “much of the data ends up being dark after the fact that people are busy and they don’t have excess capacity to go and hunt and peck for this data.” 

Similarly, the future value of dark data initiatives can be hard to assess with confidence. An IoT software executive explains how the nature of dark data makes forecasting ROI difficult: “One of the biggest differences between traditional data sets, and dark data sets, and potentially IoT dark data, is there’s typically a time element associated with the IoT data. […] If you don’t analyse the data or do something useful with it, you’ve lost the opportunity because time has passed.”

To encourage the investment, whether pausing current operations to install data collection capabilities or allocating financial resources, frame dark data initiatives as solutions to existing priorities.While dark data can create new opportunities, it can also provide benefits in the here and now. Whereas resources can be difficult to marshal around new initiatives, it is much easier to get buy-in for actions and investments that will help accomplish a company’s current objectives.

Additionally, emphasise that exploiting dark data uses what you already have. A key advantage – and selling point – arises from the fact that dark data are already being generated. Existing data can be analysed to support existing work. 



Substantial opportunities lay within the grasp of many companies. Leveraging existing dark data can create opportunities today, as well as build data management and analytics capabilities needed for new initiatives launched in an IoT-driven future.By answering four questions – what data are bring generated, what types of problems these data can solve, should we use the data given a particular business context, and how do I motivate the allocation of the necessary time and money – executives can effectively bring dark data into the light. 


About the Authors

Gregory Gimpel is a clinical assistant professor of computer information systems at Georgia State University. Prior to GSU, he conducted digital strategy research at the Massachusetts Institute of Technology and he designed Ball State University’s business analytics major. His research focusses on the intersection of emerging technologies, analytics, and digital business transformation. He worked in senior management positions for a decade before entering the academic world

Allan Alter is an independent consultant and researcher in the Boston area. He is a former senior principal at Accenture Research and Fellow in Artificial Intelligence and Machine Learning at the World Economic Forum’s Centre for the Fourth Industrial Revolution in San Francisco. His research focuses on emerging technology and technology management strategy. He was previously an editor at MIT Sloan Management Review, Computerworld, CIO and CIO Insight. He holds degrees in intellectual history from the University of Michigan and the University of Pennsylvania.

1. Chou T. Precision: Principles, Practices and Solutions for the Internet of Things. USA: Cloudbook, Inc.; 2016.
2. Solon O. Wearable Technology Creeps Into The Workplace. Bloomberg. Available at: https://www.bloomberg.com/news/articles/2015-08-07/wearable-technology-creeps-into-the-workplace, 2018.
3. Association G. Mobilizing Intelligent Transportation Systems: GSM Association; 2015.
4. Liongosari E, Mullan P, Muller M, Guittat P. Finding a Way Forward: How Manufacturers Can Make the Most of the Industrial Internet of Things: Accenture; 2015.
5. Markoff R, Seifert R. The Real Industry 4.0 Challenge. The European Business Review; March – April, 2018.


Please enter your comment!
Please enter your name here