Artificial Intelligence (AI) is to big data as a magnet is to finding a needle in a haystack.
Businesses are piling up information that promises sharper, more valuable insights. But the volume is too great for humans to process. Haystacks of data are growing exponentially, and AI offers the best promise for keeping up.
What is Big Data?
Searching the question “What is big data” in Google ironically results in its own large chunk of output–over 270 million hits. IDC describes big data technologies as “a new generation of technologies and architectures designed to economically extract value from very large volumes of a wide variety of data by enabling high velocity capture, discovery, and/or analysis”.
Thus IDC associates four attributes to “data” in the Big Data definition: value (data that will yield valuable benefits), velocity (typically a speed of above 60 GBps), volume (often 100 TB or more), and variety (data derived from multiple sources for IDC’S Worldwide Storage for Big Data and Business Analytics Taxonomy 2016).
Where Does Big Data Come From?
Big data is everywhere, both on and off the Web. For example, Social media generate big data from the constant stream of updates, likes and comments. Business systems generate big data from purchases, medical record updates and all kinds of transactions. The Internet of Things (IoT) makes up a third category of sensor or machine generated data.
This may be stored as structured data in tables and databases laid out in neat columns and rows, or organized in a format that makes elements easy to process–e.g., customer sales records, results of marketing surveys, etc. Unstructured data is, for example, the content of email, documents, voice recordings, videos, and much IoT data that do not have formatted elements that can be easily accessed or addressed.
Enter Big Data Analytics
Making informed decisions through proven analytical tools is what big data analytics strives to do. Structured big data, neatly tucked away in relational databases or warehouses, can provide a partial solution, but warehouse data has its limits.
Big data analytic technologies, like Apache™ Hadoop® and its family of data mining tools, can help organizations process unstructured data. The resulting mined output is more useful.
But in this model, data analytics operations must be programmed in advance to perform planned processing tasks. Organizations need to hire or train people with the skills to develop the desired applications and integrate them with existing business systems. The programs don’t have the intelligence to respond and adjust to changes in the system.
The Limits of Data Analytics
The characteristics of big data–its volume, variety, and constant growth–can make it resistant to effective and agile analysis. The variety of the data and velocity of its growth could force data mining software into its own specialized niches.
Computerworld estimated that the amount of digital data being produced will exceed 5,200 GB per person on earth by 2020. But so far, analytics explore only a “tiny fraction” of data for valuable information. Besides sheer volume, issues of consistency and data quality also cause management headaches.
So, aside from maintaining a bank of supercomputers and hiring a team of trained data scientists, how can businesses keep up?
AI is Tearing Down Information Processing Barriers
AI departs from the paradigm of supercomputers and mimics human intelligence. An AI computer program responds to changing information and makes adjustments in its output. In short, AI programs don’t just compute; they learn and figure out what rules to follow as the programmer feeds in more data.
This rule-making process is known as machine learning. An AI program mimics human intelligence by the way it actually “teaches” itself to react to the ever changing and constantly growing influx of big data. As Edward C. Monaghan explained in Wired, “Soon we won’t program computers. We’ll train them like dogs.”
How AI Will Impact Business
AI “has become pervasive in business in every industry where decision making is being fundamentally transformed by Thinking Machines,” says CEO and global futurist James Canton. He observes that AI and big data, formerly two different disciplines, are converging.
AI has the potential to help us find meaningful trends and patterns, create new business models, and perhaps even end hunger and disease, he argues. Canton envisions that in a world of big data everywhere, AI will be the driver that will enable us to extract meaning and to monetize “data for a purpose.”
In fact, IDC predicts that by 2020, cognitive (or AI enabled) applications will yield productivity improvements in excess of $60B annually for U.S. enterprises. Cognitively enabled applications, product and services are coming to market across a variety of industries. These include shopping recommendation apps like North Face, smart home systems like Google Nest and Amazon Echo, intelligent supply and logistics systems, automated assistants such as Apple Siri and Microsoft Cortana, and much more (Source: Market Analysis Perspective: Worldwide Cognitive Systems and Content Analytics Software, 2016.)
AI is Already Transforming These Three Industries
Author and business expert Bernard Marr identifies three sectors where AI has a big role that will grow in the next decade:
Healthcare–Marr foresees a melding of smartphone technology with AI to enable individuals to self-diagnose. “AI bots” will tap into vast genome databases and assess the probability that a patient will contract a particular disease. Resulting preventative measures will result in longer and disease-free lives.
Finance–Marr predicts that soon, AI financial advisors will begin to replace humans. AI vastly exceeds one person’s capacity to access and sort through thousands of companies and investment plans and factor in everything from risk preferences to subtle changes in the stock market, for example.
Insurance–Marr says that the insurance industry will be transformed by the meeting of big data and AI. Devices like driving habit monitors or wearable health monitoring devices could result in lower premiums and discounts for safe, healthy drivers and health insurance customers.
Other top use cases across a variety of industries include diagnosis and treatment, automated customer service agents, crime investigation, financial advisors, risk analysis, threat intelligence, insurance claim investigation and processing, and manufacturing operation improvement.
Takeaways
AI picks up where human assessment and processing capabilities (such as Hadoop) reach their limits.
The fuel for AI is big data.
AI and big data are dramatically impacting a variety of industries, with large opportunities for growth expected in the healthcare, finance, and insurance industries in particular.
The ability to capture data about people and the world is driving organizations to make it increasingly useful. The convergence of AI and big data is the likely natural next step to find meaning and create better outcomes from our ever-growing data resources.
The post The Outlook for Big Data and Artificial Intelligence (AI) appeared first on IDG Research.