What is the Unstructured Data conundrum — Definition, types, etc.

What is the 'Unstructured Data' conundrum?

If your organization struggles to corral and analyze unstructured data, you're not alone. In a recent survey, only 18% of organizations reported being able to take advantage of such data.

The great majority of data is unstructured information (almost 90 percent) — text, video, audio, web server logs, social media, and more. Organizations who figure out how to use it will get a definitive competitive advantage.

So, what is Unstructured Data?

Unlike structured data — which is organized in a searchable format like a database — unstructured data is information, in many different forms, that doesn't follow conventional data models, making it difficult to store and manage in a mainstream relational database.

Is it a big problem?

Unstructured data is the majority of existing data and also the vast majority of new data being generated today. The complicating factor is that unstructured data has an internal structure but does not contain a predetermined data model or schema. It can be textual or non-textual, human-generated or machine-generated.

What are the most common types of unstructured data?

One of the most common types is text — Word documents, email messages, PowerPoint presentations, survey responses, call center transcripts, blog posts, and social media. Other types include images, audio, and video files.

Another rapidly growing type is machine data. Log files from websites, servers, networks, and applications yield a trove of activity data. Data from sensors on manufacturing equipment and connected devices is yet another form of unstructured data.

Has unstructured data been harnessed successfully?

Retail and finance were ahead in processing unstructured data. Since the mid-2000s, retailers were the first to combine and analyze data from customer emails, images, voice, and store-traffic records to market to particular customers. Finance has made substantial progress in the last few years. Other industries — shipping, transportation, legal, and real estate — are now leaning into unstructured data as well.