This is the year when "Big Data" becomes a must-have competency, according to International Data Corporation. Its forecast volume of 2.7 zettabytes for the world's digital content represents a 48 percent jump from the 2011 total. Most of that - social media files, text, images, videos, audio - is unstructured and will offer formidable challenges for business information (BI) specialists. But don't expect those firms to be daunted.
"As businesses seek to squeeze high-value insights from this data, IDC expects to see offerings that more closely integrate data and analytics technologies, such as in-memory databases and BI tools, move into the mainstream," IDC said in a report on this year's data outlook.
How do information specialists make sense of all this unstructured data and what are the key hurdles they face? In the realm of financial markets, much comes down to the techniques they use to process textual data.
BT Radianz, which focuses on telecoms for the financial markets, is seeing an increasing number of information providers coming onto the network focused on delivering unstructured data that is reshaped into structured data.
"I have a feel for it, and that feeling is that it's broadening and increasing," said Michael Cooper, BT Radianz's chief technology officer, referring to the use of unstructured data. "The base of what people are using is definitely broadening and I think that the frequency or volume of that data is increasing as well."
That presents one of the most immediate challenges to this trend: data capacity. "So one of the things you have to do with this type of data, you really have to use it intelligently," Cooper said. That means not only dealing with the scale of the volume, which he suggests is actually manageable, but also addressing its fluid nature, which is less so.