Transforming your data deluge from a liability into an asset

There has been a considerable focus on amassing and storing data over the past decade: the more data collected, the better. But when the volume of new data being generated is overwhelming the capacity of institutions to manage it and researchers to make use of it, it turns into a data deluge.

The world’s total digital data volume has been doubling every two years. In their Data Age report, the International Data Corporation (IDC) predicts that 175 zettabytes of data will be generated annually by 2025. Yes, zettabytes!

Apart from traditional data sources, there are numerous digital data sources that have made it difficult to aggregate, visualise and analyse information efficiently. Having all these various sources at hand has caused many organisations to collect and store as much data as possible – often way more than is actually needed.

Data is easily accessible and virtually inexpensive to store, and thus masses of data is kept with the belief that it could be useful one day. But floods of almost completely futile data cause data scientists to focus less on data analysis and more on data scrubbing. This means that most of their time is spent struggling with inefficient cleaning processes to make data usable. Data preparation accounts for about 80% of the work of data scientists – a tedious task that can be reduced by focusing on quality over quantity when data is collected.

Another contributing factor to the upsurge of data in recent years is more convenient access to the cloud. Because of the relatively low costs of storage, cloud technologies are promoting carelessness with the volume of data that is being stored. Instead of organisations sorting through it and keeping only what’s needed, data is piling up in a bigger drawer of junk data – a data deluge.

But there’s light the end of the tunnel. If you suspect that you might be guilty of data hoarding, below are 3 ways to turn your masses of data from a liability into an asset:

1. Build a data strategy
Whether you’re a big data giant or a family-run business, all successful businesses begin with a strategy. And these days, all companies of all sizes and in all industries need a data strategy. Ask yourself what business problem needs to be solved and what data is needed. From there, start thinking about how the data will be collected and analysed. Keep in mind that, like any business improvement process, things may shift or evolve along the way. You may find that you need to revisit your existing data strategy along the way and make some modifications to adapt to the organisation’s needs.

2. Identify the causes of your data challenges
In order to truly master your data, you need to identify some key challenges. Does your data live in multiple, siloed, legacy data systems? Do you have numerous sources of data? Does data cleansing and standardisation only occur in isolated data sources? To reap the benefits of useful data, relevant stakeholders in your organisation need to understand and appreciate the role of data governance, and take time to regularly review data rules and processes.

3. Simplify your data
Establish a process to simplify and standardise data collection, analysis and presentation. This will allow you to quickly sort through data lakes and find exactly what you need, while discarding chunks of futile data. Your data strategy should include automated policies to ensure data remains consistent and data quality is standardised across your organisation. As soon as internal data management guidelines are in place, simplifying data will become easier and a “less is more” culture can start developing.

Technology today has made data collection more seamless than ever. This ability is both a blessing and a curse. When you can collect anything you want, the challenge is deciding what you should collect. The focus needs to slowly start shifting towards rich data instead of big data, because bigger data isn’t always better data.