This article was originally published in AI & Machine Learning Tech Brief on Sep. 23, 2019
Big data continues to grow exponentially creating a critical need for solutions that can make sense and extract valuable information from it. For example, the Internet is full of a wide variety of constantly growing text sources— blog posts, forum posts, chats, message boards, item and services reviews, etc. If leveraged correctly, this data can provide valuable clues on your customers’ satisfaction, their pain points, and may help to explain past customer behaviors or predict the future ones.
Artificial Intelligence (AI) gave birth to new solutions that leverage speech and text analysis bringing valuable insights into new business areas. By taking a holistic approach to AI – analyzing content across unstructured data, text, video, imagery and audio – data scientists and software developers are building new tools that help address real business needs.
Imagine a tool that analyzes customer calls and chats; automating and speeding up customer complaint analysis.
Recent advances in AI, namely Neural Nets —a set of algorithms, modeled loosely after the human brain, that are designed to recognize patterns— have lowered the computational requirements needed for many important text analysis tasks. By leveraging Neural Nets and natural language processing algorithms, businesses can use text analysis, the automated process that allows machines to extract and classify information from text, such as tweets, emails, support tickets, product reviews, survey responses, etc. This enables businesses to collect important customer data and help reduce customer service costs without losing valuable quality.
In one of our recent efforts we created an AI system that used a combination of NLP and NN models to analyze customer interactions and condensed the data into an easy to read summary. This summary helps the company to work more efficiently with its customers, ultimately gaining a better understanding of each customer’s needs. To enhance the AI system even further, we examined a set of text classification models that increase the company’s comprehension of comparable situations. Such a component will ensure successful and efficient adaptation of the most effective solutions.
Making Sense of Your Data using MVP and AI First
Many companies are faced with the challenge that the complexity, potential impact, available mathematical methods and technology might be too overwhelming for current systems or resources. In situations like this, we recommend two methodologies: MVP and AI first.
The MVP approach stands for minimal valuable prediction, a well-known minimal valuable product concept often used by software developers but applied to Data Science. The idea is to focus on delivering a minimum viable prediction as fast as you can and iterate from there.
- Focusing only on the most-pressing problem you want to solve
- Instead of assembling all your data, assemble only the data that correlates with that problem
- You’ll work “backwards” as you identify other related data, essentially following the data breadcrumbs that lead to actionable outcomes
This approach is faster, cheaper and has a much higher success rate than typical big data projects in that it starts with small data sets and incrementally scales as you uncover actionable insights ensuring you are on the right track before committing more resources.
As the name suggests, AI first concept suggests that you begin a project by formulating a business task and choosing an approach. After that, follow by asking yourself: what data do we need? What data do we have? Which data pieces are crucial and which ones can be substituted? Next, think about the technology and the properties of your future architecture; does it support the chosen AI approach? Is it flexible to support possible future changes? At every step of this process we iterate and come back to the original business question. This approach requires an organizational mindset shift and is dramatically different than trying to incorporate AI into an existing system.
By leveraging both MVP and AI First methodologies for any of your AI projects, you will produce results faster, cheaper, and without the risk of big-bite data approaches.