Curated Tech Content

Whether we are publishing our own original content or helping our partners get the word out about their technologies. We bring you the latest news, market trends and product innovation.

Humanised big data – your story in bits

With our unstoppable use of digital and connected devices, an invisible world is changing human lives. Sensors, GPS, and smartphones are beginning to sense, measure and understand our existence in more ways. Our devices are creating a new kind of nervous system, one that gathers and analyses data in real time.


With our unstoppable use of digital and connected devices, an invisible world is changing human lives. Sensors, GPS, and smartphones are beginning to sense, measure and understand our existence in more ways. Our devices are creating a new kind of nervous system, one that gathers and analyses data in real time.

Through the Internet of all Things (IoT), intelligence is brought into everyday appliances, resulting in a flood of data. The new word for this is Big Data. Big Data is a technology that collects and analyses the kind of data created by the numerous connected devices and systems used every day. There is data everywhere. Wherever you go, the story of your life is now in bits. Business recognises the data that is generated by connected devices and consumer activity holds potential. To realise this potential is to make Big Data, more Humanised.

Every object on Earth is collecting data, whether it be our homes, cars, and ourselves. Everything we do leaves a trail of data. We are now exposed to more information in a single day compared to thirty years ago. Data is more humanised, as it tells the story of what we do, where we go, what we buy, what we say, and it never goes away, as it is being recorded and stored. It is Humanised Big Data, where we now have the potential to go from the ‘Homo Sapien’ to the ‘Homo Economicus’.


Big Data started with one level of progression. Historically, data was accumulated by workers, employees entering data into computer systems. Then with the Internet, things evolved, and the second level of progression emerged. Users could generate their own data, for example, with Facebook. Now there is a third level of progression, where machines are now accumulating data. Anything from machines in a building monitoring electricity consumption, to the space station monitoring Earth, taking pictures and collecting data. As we are now at the machine stage, there is a major amount of data generated. Picture a whole row of servers, where each server has some small component of the entire data set. A processor is in each server individually doing its job and the data is being processed with a whole bunch of servers at the same time.


To acquire knowledge, we would write down information. With Humanised Big Data, we would look at the pile of data, and shift it this way or that way, so it turns into an interesting piece of information. Humans interacting with technology has given rise to massive amounts of data. By 2020, there will be data volumes of 40 Zettabytes. What’s even more fascinating, is that the data processing of the last two years is more than the data processing of the last thousand years, according to those who are part of this digital transformation. The moment of birth for Big Data was Search. Within a few years this tool enabled hundreds of millions of people to navigate incredible amounts of information. What was already in text was put on the web. Human knowledge expanded and in recent years, we have the technology to store and process large quantities of data. Visualising that data, we can look at how sea levels are functioning, where a Twitter tweet has gone viral, the use of public bicycles in London, to the capture of a dancer in how they move their body. It is about seeing patterns and meaning that we thought were impossible. Everything we do is measurable.


Not that long ago and certainly within living memory, the only way to detect a flu epidemic was by a doctor recording and submitting information from patient visits. A process that took its time to reach the health organisations. Health researchers wondered if they could predict a flu outbreak in real time by using data from on-line searches, such as Google. So they searched the searchers, to find the information over a five-year period. In their research, they noticed that Google Flu Trend Estimates corresponded with the search terms to accurately predict flu outbreaks through an algorithm based on what people were searching in real time. Now there are measurable ways to find out about the health of a nation and respond quickly to it. However, there could be a flip side.  The media may do a story about an upcoming flu season and searches regarding flu increase, but no one has the flu. Not a perfect science, at least not yet.


In the world of business and marketing, insights and actions of customers are obviously important when it comes to products and services. Big Data is about People. Humanising it involves design. For real insights, there are several key design principals into Humanised Big Data, according to the CITO Research White Paper on Humanizing Big Data.

Ingest and Integrate Data from anywhere:

Systems of record, social media, and sensor data are all fair game, as is the information from the data storage facility.

Seek Patterns:

Patterns hold the key to predicting future outcomes. Don’t look for a pixel-perfect report that is accurate to the penny when searching unstructured data. Fuse the qualitative structured data with the important context provided by unstructured data.

Make Insight available at the point of decision:

Insights are best when widely available. A store manager knows their market as they see it every day. With powerful analytical tools, they can make effective and informed decisions.

Reuse Analytical IP:

A “data artisan” can create a data object and share it with a full range of decision makers, who can adapt and build on it. Each time the story extends, it also gains focus.

The goal of Humanised Big Data is to get these capabilities into the hands of analysts in business units, allowing them to create reusable analytic workflows. This will allow company decisions in regards to merchandise, marketing, operations, financial and customer satisfaction, to be made in real-time.