I am addressing data types in today’s issue. Occasionally, we hear suggestions that there is no such thing as unstructured data. Proponents argue that all data has structure. “It only depends on how you view the data” . Perhaps!
Perhaps we do not have tools for all types of data.
Perhaps the structure of data as we intend it to mean, refers to the ability of computers to utilise data and produce results.
When we think of data types, it is generally in the context of how easy it is for computer systems to use the data for analysis and generate meaningful insights. Although unstructured data can be meaningful to its producers, it has not been easy to include in computational data analysis.
Below i attempt to answer some key questions relating to the blog topic;
What is structured data?
Data that is in relational format such as tables, comma delimited files which can easily be stored in a relational database. Usually in two dimensional format – rows and columns
Other forms of data which do not fit this mould are usually difficult to include in analysis which drive business decisions
What is unstructured data?
Unstructured data is the opposite of structured data. It cannot be easily stored in a database for analysis. Some examples include Audio files, videos, Pdf files, documents, images
What is semi structured data?
There is a middle ground between Structured and Unstructured data which is referred to as semi- structured. Semi structured because it has metadata and tags which make the data easily readable. Some examples are IoT data, Json format files.
Why does unstructured data pose a challenge?
Traditionally, businesses use data which is held in their data platforms to make decisions.
The information from the data is often times combined with expertise of business subject matter experts. This can lead to subjective outcomes as the information used to arrive at decisions is often not robust enough. Lack of robustness could be due to rapid business landscape changes . Such subjective outcomes can also arise because the business hasn’t taken into account other factors such as end user sentiment which could be significant. A gap therefore exists in decision making.
Do you need structured data for AI?
This is an interesting one which is answered in 2 folds;
- AI is the use of Machine learning models to solve problems or perform tasks. AI does not need structured data because the underlying model holds the algorithms which is required for AI to work.
- Machine Learning models which are used to power AI, need data in a structured or semi structured format which algorithms can process.
Ultimately, yes structured data is required for Artificial intelligence. There are deep learning models which involve heavy processing which can be considered as alternatives.
How can AI help make unstructured data structured?
There a a few AI tools which can be used to transform unstructured data such as audio files into structured data. Such as Speech and Vision AI tools which can be used to extract data from audio and video files. The data extracted can then be incorporated into more comprehensive business decision making.
Does data need to be structured for AI to work?
I hope that the brief explanation above clarifies this question. AI agents do not need structured data. This is why you can pass a document to a generative AI agent for summarisation.
However, how did we get to AI?
We needed to start with statistics and machine learning. Machine learning requires structured data for its algorithms (Regression, Classification etc). Data is collected and processed before being trained with the chosen algorithm.
Leave a comment