We recently sat down with a large pharmaceutical company to discuss their data analytics projects. What we
heard wasn’t a surprise. Three of the four large analytics efforts they undertook last year had
failed. The jury was still out on the fourth.
Yet, by way of contrast, here at EastBanc Technologies we are responsible for ten analytics
projects at any given moment, without the mammoth-sized resources available to the large enterprise
clients that we often serve.
The difference between our success rate and the failures we see has little to do with
the complexity of the undertaking (although that’s a part of it) or the incredible budgets allocated; it has
more to do with at what point and how frequently along the project path a success
measurement is taken.
Traditionally, complex, large-scale analytics projects are so involved and unwieldly that only at the end of the
project is a real success assessment made, often years after kick-off.
That’s not our approach. We weave success measurement into the entire process. That measurement and the completion of
tasks become two sides of the same coin.
We do this by leveraging two approaches:
1. An Agile Methodology
Agile has been around for more than 15 years, but has only recently gone mainstream. Agile
is a term used in the IT world that describes a process of software or product development that
stresses cross-functional collaboration over siloes, adaptive planning, early delivery, and continuous
Agile breaks work into small increments and short timeframes. During this period team members come together to
report on what they did the previous day towards their iteration goal; what they’ll do today; and any
impediments that may be in the way to that daily goal.
Feedback, transparency and collaboration are key and encouraged. Team members are empowered to
make suggestions about how things can be done differently and prioritize tasks more efficiently. Through
constant adaptation and improvement, Agile teams can more quickly respond to changing needs, address problems as
quickly as they arise, and iterate towards a higher quality data analytics product – and a lower
probability of failure.
To break it down, an Agile methodology incorporates the following key features and benefits:
Weekly Sprints – With Agile, success measurement is woven into the weekly
process. Weekly recaps (aka Sprint Meetings) provide a moment in time to measure
progress against your original Minimal Viable Prediction (MVP) and assess if you’re
on the right track. Because any failure caught at this stage is likely to be small in scale, it’s
also easier to rectify. Plus, each failure is a new insight that will inform the next sprint. For
a deeper dive into the anatomy of the sprint approach, check out: Sprint, Scrum, and Seize Success.
Business Stakeholders are Core to the Process, not on the Periphery – Another
big bonus of this process is that business people are part of the process from day one. Unlike
traditional waterfall approaches to software development which kept business people out of their own
projects, with an Agile approach, business stakeholders are involved every single day, or at
least at weekly Sprint meetings, rendering failure close to negligible.
Micro Goals Aid Success Measurement – A core facet of the Agile methodology is
breaking goals down into micro goals. These are measured and evaluated on a weekly basis by all
stakeholders, measuring both technical and business success.
If this process is structured right, the mere completion of a task equals a micro-success
measurement. That the task was collaboratively added to the backlog in the first place means that
its transparent completion by the team member, who took ownership of that task, added another mini-checkmark to
the success of the overall project.
2. New Lightweight Cloud-Based, Pay-As-You Go Technology
Alongside Agile, our technology approach also flips tradition on its head.
Legacy analytics technology forced teams to think sequentially rather than in parallel. New cloud-based
business intelligence (BI) technologies are paving the way for immediate measurements and real-time,
not historical, insights allowing businesses to adjust goals along the way. Unlike monolithic systems
where integration was a huge challenge, these modular, service-based architectures are relatively easy
to integrate. You can also fire them up as needed and pay-as-you-go or switch providers whenever you want.
Another benefit to cloud BI is that you can start small. In the past, any BI initiative involved a huge
investment, terabytes of data, and technology overhead that prohibited smaller, nimbler
As data analytics becomes democratized and failure rates decrease, the role of analytics will become increasingly
important. Powered by the cloud, BI technology has become an orchestrated service eliminating the need for unnecessary
redundant features while minimizing vendor lock-in. Plus, executives get real-time results faster (and at a
lower price point) and are energized to want more.
New Approaches Require New Mindsets
Of course, a new way of doing things requires a cultural mind shift. Adjusting to an Agile, product
development methodology and thinking in terms of rapid iterations, involves acting like a pivoting start-up,
agile and ready to change directions quickly. Yet decades-old traditions and hierarchies stand in the way.
The charge to change must be led by data and facts, not gut instincts and established routines. A
pay-as-you-go approach also means that you won’t negotiate a software license anymore or have hard-set deliverables.
This makes budgeting a challenge. How do you know how many licenses you’ll need the next year? The business
model of the big legacy players was to provide you with the number of licenses you’ll need from day
one. With the pay-as-you-go cloud BI tools, you may want to start with one and add more as you see success – pay-as-you-go
cloud models give you the flexibility to do that. But people who think the old way will find it hard
All Too Radical? Start with a Hybrid
If this path to data analytics success is more than you can bite off right now, don’t worry. Start with a mix of
the traditional and MVP approaches. Instead of an MVP, choose a bigger goal, perhaps something you can revise on
a quarterly basis. Even if you lose three months because your team was on the wrong track it is still
better than no iteration at all and realizing only at project completion that you failed.
If you can’t include business stakeholders on a weekly basis, loop them in each quarter.
You don’t have to adopt a pure Agile approach from the get-go. Go Agile iteratively and slowly increase adoption
as stakeholders see the benefits for themselves.
Take a Leap of Faith
Look at the disruptors in your industry and take a small leap of faith. It’s a risk, but taking iterative,
incremental success measurements will ensure a result that truly provides value.
In the words of Amazon-founder and owner of our customer The Washington Post (which has famously adopted Agile): “There is no map, and charting a path ahead will
not be easy. We will need to invent, which means we will need to experiment.”