This article was originally published in Database Trends and Applications on April 9, 2019
The data on big data indicates that up to 60% of analytics projects fail or are abandoned, costing companies an average of $12.5 million. That’s not the result we seek from data lakes. Instead, companies are increasingly finding themselves mired in data swamps that are overfilled and too muddy to offer any useful visibility. Or are they?
Many companies and government organizations have jumped on the data bandwagon and are hoping to analyze as much data as possible to discover something that might give them a competitive edge or change their operations. Instead of big insights, they have petabytes of big data. They’ve become compulsive data hoarders while struggling to develop an actionable plan for that data.
The motivation to gain insights from data is good but the approach isn’t. Big, traditional, analytical projects are expensive, take years to implement, and aren’t as effective as the marketing materials would lead you to believe. A survey of more than 500 executives by McKinsey Analytics found that 86% of respondents believed that their organizations were only somewhat effective at meeting the goals they set for their data and analytics initiatives.
A better approach to big data is to start small. Instead of taking a giant bite of your data, you nibble at it, keeping risks and expenses low. Time and again, we have seen some of the most valuable business insights derived from surprisingly small datasets. We call this approach “minimal viable prediction,” or MVP.
MVP is an approach replicated from best practices in software product development. The idea is to focus on delivering a minimum viable prediction as fast as you can and iterate from there.
For example, let’s say you are an ecommerce grocer. Your customer experience culminates in customers adding items to their shopping carts. You know that product affinity changes with each customer’s context, such as the time of day, day of the week, weather, and location. What if you wanted to suggest, in near-real time, items that appeal based on the customer’s specific context? Not generic buying patterns based on all Amazon users or the like, but personalized for an individual customer and real data about their real environment when shopping virtually.
You might start by gathering historical, broad market basket data. You may discover that, in general, diaper purchases have the highest affinity with beer purchases. The next data breadcrumb might be a customer’s gender data, if available. Another set of data could be weather data by ZIP code. Suddenly, you are starting to see some trends and you’ve barely started! Perhaps it turns out that product affinity of diapers and beer applies to male customers, but significantly more often during later hours of the day and has overwhelming statistical relevance when it is unusually hot. This is your first MVP that can get immediately validated in the top and bottom lines.
If the first dataset was product affinity, customer gender, and weather data, a second tier of data may include a list of brands and their profitability to you. You dig into that data, and simple A/B testing may uncover that regional brands have higher success rates. While you’re probably not going to get rid of the major consumer packaged goods suppliers, you can keep that correlation in mind, and move on to the next dataset. The key point is to establish a methodical process that iteratively examines your top headache … or opportunity.
The approach is simple and straightforward, but keeping things small is actually difficult. It’s a mindset that requires organizations to answer one question at a time and follow the data path to a conclusion. Armed with these little insights, it’s tempting to want to dive deeper.
Here are a few tips to stay the course and embrace MVP:
Collect thoughtfully. Data becomes dated—fast. Unless your questions focus on historical changes, spring-clean your data regularly and eliminate data clutter.
Iterate. Don’t waste time by taking too big a bite. Predictive analytics is a process of trial and error. So limit your focus and iterate, trying new paths until you find one that leads you in the right direction.
Prioritize. List all the problems or questions you would like to solve with data analytics. Rank them by potential business impact and ease of assembling the first set of data. Then pick the highest-ranked problem, and follow the breadcrumbs.
Stay focused. If you hit a dead end, retrace your steps and follow the next set of data breadcrumbs. You could easily assemble another set of data but don’t—it will muddle your quest to answer question number one. The more self-discipline and focus you demonstrate, the easier it is to generate insights and achieve a better outcome. As you continue to iterate, include more data, where and when it’s relevant, and only then.
Refine. Once you’ve answered your number-one problem and have your first MVP, refine it again and again so you have an ongoing stream of actionable, affordable predictions.
The cloud is your friend. Cloud-based pay-as-you-go tools, such as Microsoft’s Power BI, are ideal companions for this approach. Data management, analysis, and visualization tools in out-of-the-box, cloud-based packages make it feasible for organizations to become data savvy, regardless of budget or analysts on staff.
Once you have your first MVP, your project is already more successful than 50% of the big data projects out there. We can attest to seeing results in 2–3 months—rather than 2–3 years. And that’s where the MVP approach excels. By only focusing on the relevant data breadcrumbs and starting small, it becomes much easier to gauge whether you’re on the right track and to correct any mistakes relatively cheaply.
The MVP approach requires a mindset shift inside the organization—one that values agility and accepts trial and error. It’s as much a cultural change as it is a data approach. Initially, the MVP approach’s small scope might feel too small for organizations experiencing a revenue-impacting pain point. The approach was challenged when it was first used in software development too. Have faith that starting small—with limited questions, investment, and time—will produce results faster, less expensively, and without the risk of big-bite data approaches.
AI and cloud computing together have reshaped the digital landscape in recent years. Alone, each is transformative; together, they can revolutionize industries, driving the next tech evolution. Let's explore how to optimize this integration.Read more
Explore the intricacies of AI deployment: Why does the performance of the AI models sometimes worsen after it's been out in public?Read more
Choosing your digital transformation partner is crucial for the business's long-term success. In our many years of experience, EastBanc Technologies has built a list of factors to consider when choosing a digital transformation consulting company.Read more
AI's vast potential brings challenges and opportunities that will redefine the industry's future; let's dive into how predictive analytics can help and look at several standout examples from diverse sectors that can provide insights for IT and beyond.Read more
The rise of low-code development platforms has revolutionized the software development process.; Llet's look at how they did do it and what they provide.Read more
Artificial intelligence (AI) has the potential to revolutionize various industries and aspects of our lives. However, along with its promises, AI also presents several challenges that need to be addressed for responsible and fair development and use. This article discusses some of the top challenges for AI, including bias in data and algorithms, explainability and transparency, ethical concerns, integration with society, security and safety, and regulatory framework.Read more
Increasingly complex AI and ML models require more data to be trained effectively - what are the ways of tackling the increasing power needs?Read more
The demand for faster and more efficient software delivery has led to the emergence of DevSecOps, a combination of DevOps and security practices. In this article series, EastBanc Technologies explores some of the top trends in DevOps, starting with DevSecOps.Read more
It's no secret that the business landscape is changing. In order to stay ahead of the competition, it's necessary to undergo a digital transformation. In this blog, we'll outline the steps you need to take in order to make sure your business is prepared for the future.Read more
Get ready to witness the power of artificial intelligence as it transforms the military, retail and personalized medicine industries! From revolutionizing defense strategies to streamlining the shopping experience to providing customized healthcare solutions, AI is changing the game in ways we never thought possible.Read more
In the context of EastBanc Technologies' dev approach, MVP stands for Minimal Viable Product. An effective method of quickly establishing a framework for a digital solution, the MVP streamlines the process of a product’s initial deployment. We take a look at the fundamentals of the MVP and how to construct one.Read more
Solar energy is a promising – and green – alternative to fossil fuels. As long as the sun is shining. Check out how AI helps solar energy providers optimize output, manage supply & demand and reduce the price of electricity using predictive analytics and machine learning. This is AI at work, making gigantic strides for worldwide adoption of this renewable energy source.Read more
Wind energy has the potential to cover much of the world's insatiable thirst for electricity in a sustainable way. Unfortunately, the wind doesn't always blow -- and not always with the same intensity. Using AI and machine learning models, energy producers and scientists are finding new ways to maximize the output and efficiency of wind energy.Read more
The potential for Artificial Intelligence (AI) in the green energy industry is rapidly gaining momentum. Renewable energy sources such as solar power are complex and unreliable due to constantly changing weather conditions, but AI can help remove obstacles and unleash the true power of solar.Read more
Apple’s introduction of passkeys with the latest versions of its MacOS and IOS operating systems means is a major step forward for online identity management, but passkeys will not hand us complete control over our own online identities. For that to happen, we need to look at Self-Sovereign Identity (SSI).Read more
Cryptocurrencies are notoriously volatile. Indeed, the rapid rises and vertigo-inducing plunges can make even the most stout-hearted crypto investors tremble. But with time, Bitcoin, et al may settle into a more temperate pattern and become a stable - or even a centerpiece - of our financial systems.Read more
Artificial intelligence will continue to disrupt many industries, and the best way to maximize the impact of the technology is to start teaching it early. Weaving AI learning into high school curricula will create a strong link between technologies and curious students, fostering future employees well trained in the digital world. -- benefiting business and driving innovation.Read more
A dive into the implementation of the blockchain in finances, smart contracts and NFTs.Read more
Blockchain is a word that is now heard everywhere, but not everyone has a clear understanding and knows what is there under the hood. In our second part of the blockchain guide let's dive deeper into the technology and concepts behind it.Read more
Teaching computer science to teenagers is a no-brainer in today's digital world. Here's why weaving artificial intelligence and machine learning into the high school curriculum can increase the growth of innovative technologies like never before.Read more
While traditional computers continue to evolve and pump out more raw power, they are no match for the quantum computer, which can tackle calculations that the most powerful conventional machines would need decades to process – in a split second.Read more
HackTJ 2002 is in the books, bringing together more than 400 bright young minds eager to tackle real-world problems with creative technology solutions. As a Gold Sponsor for the event, EastBanc Technologies created three challenges for the young innovators, and we are delighted to announce this year's top contestants -- and their winning hacks.Read more
This is EastBanc Technologies 3rd year sponsoring HackTJ, and our participation includes designing three challenges for teams to hack. The challenges will explore how to alleviate some of the world’s most pressing issues impacting our personal and professional lives.Read more
One “new” technology that has stuck is Blockchain. To understand what Blockchain is, you only need to know three things. What is a block? What is a chain? What is a ledger?Read more
Modern technology brings the world closer together, but millions of people continue to be left behind. The "digital divide" is multifaceted and impacts society in a variety of ways. These are some of the technologies that are helping bridge the gap.Read more
Artificial intelligence (AI) can be found almost everywhere in modern life. Learn key lessons and best practices that help companies avoid common AI pitfalls and achieve ROI from their AI systems.Read more
Open Data fuels today's digital economy, enables communication and innovation, boosts business and generally makes our lives easier. But how do we protect privacy if everything is open? Zero-knowledge proof could be the answer.Read more
Blockchain capabilities, including fully-automated data storage and transparency, make it an essential technology for cybersecurity. In this article, we look at some of its use cases.Read more
DevOps built-in flexibility allows development teams to work at a level that suits their resources and skills without being held back by departmental barriers.Read more
Artificial Intelligence (AI) – the capability of a machine or piece of software to display human-like intelligence – permeates our daily lives, often in ways we do not notice.Read more
Data-driven software touches our lives every day. Sometimes, it is in ways you see, such as when you check your Twitter feed, pay for your bus ticket or order your latte using your phone.Read more
EastBanc Technologies is recognized on CIOReview’s list: “Most Promising Microsoft Azure Solution Providers.”Read more
In this article, we’re going to dig a bit deeper into AI-implementation. We will take our airline use case a step further, and we will describe a specific example of how EastBanc Technologies solved a particularly challenging problem through AI and machine learning.Read more
If your organization provides a product or service -- which applies to just about any business on the planet -- you, too, can benefit from Artificial Intelligence (AI). While implementing AI may sound daunting, it doesn't have to be complex or expensive. This article covers the basics of AI and looks at some easy-to-explore use cases.Read more
Digital transformation is about opportunity and survival. Businesses that transform digitally gain a significant competitive advantage.Read more
Part 2: Best practices for modernizing your company’s IT infrastructure to ensure innovation success.Read more
Best practices for modernizing your company’s IT infrastructure to ensure innovation success.Read more
Learn how machine learning engineers and data scientists collaborate and roll out models faster and with ease using Azure Machine Learning.Read more
What is DevOps, what are DevOps practices, and how do you implement DevOps? Your FAQs answered.Read more
Refactor, rewrite, or leave as is? Learn when and how to bring your legacy systems up to speed with modern application development practices.Read more
Learn how technology can better meet your business needs with this foundational understanding of how software and system architectures work.Read more
Ready to embrace AI? Explore why cloud computing is the best infrastructure for your AI model, not on-premises.Read more
Software is a strategic differentiator that can catalyze digital transformation. Organizations are investing in technology, such as modern cloud services, to drive efficiencies and increase the customer experience. To make this a reality, it’s essential that business leaders have a basic understanding of business software and applications work and the opportunities they bring.Read more
How an intelligence-driven customer technical support approach can transform your support from a reactive operation to a streamlined, efficient, and proactive operation.Read more
Kubernetes is a popular container orchestration system, but how did it come to be and why, and what role does it play in digital transformation?Read more
Continuous integration and continuous delivery (CI/CD) is integral to a DevOps approach to software development. But what is CI/CD and why is it key?Read more
This article is the third in a series that aims to demystify data science , machine learning, deep learning, and artificial intelligence (AI) – while exploring how they are interconnected.Read more
2020 has seen profound change in the way we live and work with COVID-19 accelerating the pace of digital transformation. Yet, business leaders are often confused about how to implement one of the key enablers of...Read more
Artificial intelligence (AI), together with its brethren buzzwords data science, machine learning, and deep learning have been around for some time now and are no longer future concepts. Yet misconceptions persist about the true meaning of these terms.Read more
When SUSE, the world’s largest independent open source company, announced its acquisition of Rancher Labs in early July 2020, the industry took notice. Clearly, the Kubernetes management industry is very much alive.Read more
We live in a technology-driven world. Even non-technology companies are seeing their business models increasingly shaped by technology. Led by disrupters such as Amazon and Netflix, those enterprises who recognized opportunities early have found ways to extend the analog experience into a digital one. Even creating new revenue streams that they could never have predicted.Read more
Digital transformation is about delivering core competencies in a digital, automated, and user-centric manner. Driven by data and powered by tech (e.g. cloud, cloud native stack, AI, machine learning, and deep learning), it increases business agility, competitiveness, and enhances customer value.Read more
Let’s start by understanding where DataOps falls in the line-up of current IT methodologies. DataOps is the next level up from ETL (extract, transform, and load) and MDM (master data management systems) in terms of organizing data and processes. It can also be thought of as a methodology that combines DevOps and Agile within the field of data science.Read more
The hotel industry hasn’t changed much in the past decades. While they have introduced some level of digitization such as websites and apps, they haven’t fully embraced digital transformation. Indeed, if things are working fine, why change? Because the next unforeseen disruptor may be right around the corner.Read more
The term “DataOps” has picked up momentum and is quickly becoming the new buzz word. But we want it to be more than just a buzz word for your company, after reading this article you will have the knowledge to leverage the best of DataOps for your organization.Read more
Unstructured text is found in many, if not all business functions, and can become a source of valuable insight. Product reviews will guide your customers’ preferences, customer support chats can identifyRead more
Disclaimer: We have not spoken to a WeWork executive and have no further background information. This is merely a thought experiment to exemplify what digital transformation is about.Read more
In part one of this series, we defined data science and explored the role of a data scientist — including data preparation, modeling, visualization, and discovery. We also introduced the role of a machine learning engineer who closely collaborates with the data scientist.Read more
Big data continues to grow exponentially creating a critical need for solutions that can make sense and extract valuable information from it. For example, the Internet is full of a wide variety of constantly growing text sources— blog posts, forum posts, chats, message boards, item and services reviews, etc.Read more
Kubernetes, the de facto container orchestrator, is great and should be part of any DevOps toolkit. But, just as any other open source technology, it’s not a full-fletched ready-to-use platform.Read more
With the increasing popularity of machine learning (ML), it’s becoming more difficult for data scientists to find the appropriate tools for a specific task and decide on a robust approach. Should they stick to the basics and code everything from scratch or use one of the many pre-built tools that keep popping up on the market?Read more
Blue-green deployments and canary releases mitigate application deployment risk by enabling IT to revert back to the previous version should an issue occur during the release. Switching back and forth between versionsRead more
For those who were still debating whether they should hop on the digital transformation bandwagon, the COVID-19 crisis was a wakeup call, maybe even a slap in the face.Read more
The entire business world is talking about digital transformation. IT leaders, on the other hand, talk about DevOps, cloud native, Kuberentes and containers.Read more
If your organization leverages technology as a differentiator, a DevOps approach to application and service delivery is inevitable. The benefits are just too great.Read more
Digital transformation is one of today’s biggest buzzwords. Everyone is talking about it; everyone wants it. We all know the role technology is playing in enabling businesses to innovate at an unprecedented pace.Read more
We collect data at a mind-boggling pace. In fact, as companies, we’re hoarding it. But what good is data if it can’t speak to us? Fortunately, data complexity can be broken down through design and visualization – the charts, graphs and plots that show trends, outliers and opportunities.Read more
As a company and as a team, our lives at EastBanc Technologies have always been about tackling the biggest problems for the biggest organizations.Read more
Artificial intelligence (AI) surrounds us. It unlocks our phones, creates our shopping list, navigates our commute, and cleans spam from our email. It’s making customers’ lives easier and more convenient.Read more
Nearly every week there’s something new in our industry. The pace of technology is unprecedented, the role of IT is booming, and innovation is part of our DNA.Read more
Technology is accelerating at such a rate that it permeates all industries. In fact, software is the only industry that cuts horizontally across all verticals.Read more
Innovation is a critical part of business. While prioritizing production in general makes sense, the best approaches make innovation a component of the whole production process.Read more
We recently sat down with a large pharmaceutical company to discuss their data analytics projects. What we heard wasn’t a surprise. Three of the four large analytics efforts they undertook last year had failed.Read more
AMS Group is a cohesive group of established companies that provide technology and security equipment to aerospace, defense, and security markets.Read more
A European market leader in online survey and feedback software acquired complementary companies in different Wester European countries, each of which had its own survey platform.Read more
Everyone loves their own data. Collecting it. Analyzing it. Drawing conclusions from it. But often, when you allow departments or business units within your organization to gather their own data, that data isn’t shared.Read more
Gartner predicts that through 2017 60% of big data projects will fail to go beyond piloting and experimentation and ultimately will be abandoned.Read more
Organizations generally understand the power behind analytics, but how do you make it work culturally and technically? We take a look at the barriers to data analytics success and suggest new approaches that buck the system, with dramatic results.Read more
And how to make your next data analytics project succeed?Read more
Container use is exploding right now. Developers love them and enterprises are embracing them at an unprecedented rate.Read more
If you’re making the move to containers, you’ll need a container management platform. And, if you’re reading this article, chances are you’re considering the benefits of Kubernetes.Read more
Wouldn’t it be nice to reach artificial intelligence (AI) nirvana? To have a system that provides real-time, context-aware decisions.Read more
Today’s IT environment is moving and evolving at an unprecedented pace. So, all of a sudden, your 5-year old software infrastructure can look more like it’s 50. To get your software current – and stay there – requires flexibility. Moving to containers does just that. There’s been lots of talk about containers over the past few years – so why aren’t you on the bandwagon yet?Read more
Under pressure to deliver applications faster and ensure 24/7 runtime, organizations are increasingly turning to DevOps methodologies to deliver applications quicker and in an automated fashion. But what tools should you have in your DevOps toolkit?Read more
Amazon Web Services (AWS), Azure, and Google Cloud Platform (GCP) are the public cloud market leaders, but how do you determine which of them best supports your enterprise's specific needs? For most enterprises, and for the foreseeable future, it’s going to be a multiple answer question.Read more
As the dominant movie rental service in the 90s and early 2000s, Blockbuster was the market leader, seemingly indefatigable. Until the great disruptor, Netflix, hit the scene.Read more
Big Data. Everyone’s paying for it, collecting it, and talking about it, but what are companies actually doing with it?Read more
The API management market is a hot one. As more organizations make investments in mobile, IoT, and big data, APIs are a core of their digital strategy.Read more
Big data is everywhere. Organizations are being advised to hoard it and do everything they can to derive actionable insights. This article will argue that this approach puts the cart before the horse.Read more
Let’s face it. Organizations struggle with their legacy applications. Even when they still solve some of the business’ problems, they reach a point where they can no longer keep up with market and industry demands.Read more
Let’s flash back to 2000. You’ve survived Y2K and you’re building systems for CRM, inventory, logistics, or data. They’re all state-of-the-art, and get the job done, even if they don’t talk to each other.Read more
It’s a mobile app world, and we just live in it. But for those working on the “next big thing,” there’s a conundrum – everyone knows we should be building apps in HTML, but not every device out there runs it as smoothly as it should.Read more
In technology, everyone likes to talk about “future-proofing.” But even for the most cutting-edge tech, time always catches up.Read more
The future is here. No, we don’t have flying cars or robot butlers – yet – but it’s definitely a digital world.Read more
We’re excited to announce Microsoft Azure support for the Kubernetes auto scaling module, an open source system for automating deployment, scaling, and management of containerized applications.Read more
You can’t mention enterprise technologies today without getting into a discussion about the cloud. “Are you in the cloud yet?” Why jumping headlong into cloud computing may not be the necessary move for your business.Read more
In the mad rush to move to the cloud, some organizations put the proverbial cart in front of the horse. They’re just looking for the best hosting, the preferred provider, or whatever the rest of the industry is using.Read more
2016 saw momentum in many areas – DevOps, cloud technologies, and big data- at the thrust of innovation. So, what tech predictions will define 2017?Read more
Every month, week, or day, it seems there’s buzz about yet another solution or service that will revolutionize your industry – or more simply, make your life easier.Read more
Apps. Sensors. They’re everywhere. Your phone, your car, your TV, even your refrigeratorRead more
In an increasingly commoditized market, learn how to cut through the noise and forge a cloud strategy that meets your needsRead more
Fleet management is a challenging business. This is particularly true of snow removal services where the dynamics on the ground can change fast and the pressures to perform put fleet supervisors to the test – in the toughest of conditions.Read more
Long before the first flakes fall from the sky many municipalities begin to prepare for the cold, icy, and snowy conditions that inevitably lie ahead.Read more
Fun fact: in 2014, cloud services were already a $45 billion business worldwide, and are expected to grow to $95 billion by 2017. Will you be part of that equation?Read more
Simple is good. Simple is clean. And whether I’m cooking or planning a trip, simple is always better, right? So why do so many companies make user experience (UX) so complex?Read more
Future-ready predictive analysis infrastructures hold the key to gaining insights from data today, and into tomorrow.Read more
Immersive and exciting, Virtual Reality is already part of our lives, whether it’s a plot device in a new sci-fi thriller or the best way to enjoy the latest video games or thrill rides.Read more
Now that smartphones are the most widely used tool for navigating important life activities (nearly two thirds of Americans own one), there’s pretty much an app for everything these days.Read more
If you’re tasked with choosing an API management system, Charles Dickens summed it up best: “It was the best of times, it was the worst of times.”Read more
DevOps: the panacea for all that’s wrong with enterprise IT. Where siloed teams who keep information close to their chest are replaced by agile, transparent relationships between developers and operations and fast and stable workflows that improve IT efficiency significantly and very visibly.Read more
As a technology company focused on complex project integrations that unify legacy systems as well as modular solutions that ensure lasting scalability, we work on a multitude of projects that involve custom software development; packaged, open source, and SaaS software integration; infrastructure setup; and production operations and maintenance.Read more
In an earlier blog we talked about why you need to integrate API management into your business strategyRead more
In a previous release of “What the Tech?” we discussed why you should integrate API management into your business strategy.Read more
Smart cars, smart homes, smart devices. The Internet of Things (IoT) is already transforming how we live. But very soon, the IoT will swiftly extend into the enterprise.Read more
Why you Need to Integrate API Management into your Business StrategyRead more
The promise of big data is, well, big! With terabytes of intelligence at their disposal, organizations can make faster, more accurate decisions, monitor trends, and even predict the future.Read more
Businesses accumulate data, create content, or possess unique business logic—each of which represents an untapped business opportunity. But how can organizations realize that opportunity?Read more
The Internet of Things (IoT) is much more than a consumer trend, it’s rapidly changing the way enterprises are using data to improve business decision-making.Read more
Content consumption is changing rapidly. With multiple channels and media formats, reaching target audiences is getting harder than ever.Read more
The way in which we consume content is changing rapidly and a few trends have emerged recently that we think will have a meaningful impact on media organizations this year and in years to come.Read more
Building a mobile app isn’t as simple as it used to be. With multiple devices to cater to, development teams must ask themselves a few questions:Read more