The claim that more data is always a good thing has been repeated so often it has become a cliche. But data is only valuable insofar as it can be used to further the interests of the business. Otherwise, at best it is a wasted opportunity and at worst presents a legal and security risk.
The volume of data modern companies gather is enormous. It streams in from business operations, employee and customer activity, email, social media, and numerous other channels.
For many organizations, the problem is having too much data to manage and no idea how to generate value from it. But because that data has potential value and storage is inexpensive, it is kept until the company can figure out what to do with it.
Dark data is data a company stores without any clear idea why they are storing it and the value it represents. Often, data generated by an organization’s operations is stored in a data center without a human ever thinking about or seeing it. There it waits until it can be harvested by big data analytics or machine learning algorithms.
Or at least, that’s the idea.
In reality, much of the data stored by organizations isn’t used. It sits on hard drives in data centers racking up hosting bills without generating any value.
No-one in the organization knows exactly what the data represents. It may be innocuous fluff capable of nothing more than taking up space on a disk. On the other hand, it may be data covered by privacy and security regulations. Dark data is usually not encrypted or subject to the same protection as data that is known to be sensitive, posing an obvious risk to the business.
Dark data may also be a wasted opportunity. It may contain information that, if analysed properly, generates actionable insights that can be used for product development, marketing, operational optimization, and any number of other positive applications.
But if the data is dark, you can’t know which it is: dangerous, valuable, or waste of space.
Dealing With Dark Data
The first step is to understand exactly what data your organization stores, whether it presents a useful opportunity, and whether it is being stored with appropriate protections. In all cases, data should be archived with protections suitable for the most sensitive data it may include.
If you aren’t going to analyze and categorize data, delete it or encrypt it.
The best way to avoid gathering dark data is to implement carefully considered data retention policies. Know what the company is storing and why it is being stored.
Industry experts advise that small businesses collect data with a goal in mind, rather than indiscriminately. Data is only useful if it answers a question or fulfills a need. An “ask questions first” approach ensures that an organization avoids gathering data that may pose a risk, increase costs, and offer dubious value.