Over the last decade, numerous in-house data analytics programs have hastily embraced the latest data trends, spanning from ‘big data’ to ‘machine learning,’ resulting in excessive spending and disappointing outcomes. This trend is now recurring with the rapid adoption of ‘AI’.
As before, boards and non-technical executives are over-pressuring their teams toward AI beyond definable problems, and unqualified vendors are marketing and selling expensive products that don’t deliver (and often aren’t really ‘AI’). You can see it in your inbox and on LinkedIn.
Nevertheless, valuable lessons can be drawn from previous adoption cycles, especially from data analytics programs that thoughtfully incorporated new technologies to solve real problems efficiently. These programs will thrive in the era of AI.
At Civis, we’ve had the privilege of providing technology for and collaborating with exceptional in-house analytics programs over the last decade. Based on the lessons from these collaborations, I’m going to share some recommendations that are more crucial than ever in the AI era.
It’s worrisome to read some of the narrative on this platform, and more broadly, about how AI can ‘solve everything’. AI promises incredible opportunity, but we should learn from past trends and adoption cycles to make sure that in-house data programs become transformational investments — not predictable failures through a reckless rush to AI.
Why Do Some Data Programs Fail?
The uncomfortable truth is that most data analytics efforts fail, careless trend adoption is often a major contributing factor, and there’s a huge risk of that happening again in the rush to AI.
Here’s how organizations from companies, to nonprofits, to governments create unsuccessful programs:
- They invest in the trend before defining the problem: During the ‘Big Data’ phase, numerous data and IT teams built ‘Data Lakes’—expensive repositories to store data from anywhere—without defining day-one program objectives. Many of these data lake investments burned out due to accelerating storage and personnel costs, posing a similar risk in the AI world.
- They neglect to invest time in data quality: AI algorithms, often leveraging large neural networks (this video is fun), can be highly sensitive to statistical outliers. Under-investing in data quality can result in machines producing consistently inaccurate answers. Non-technical users should engage in ‘what could go wrong here?’ conversations early and often.
- They overspend on home-built solutions that won’t scale: While very large enterprise organizations can justify building hyper-optimized data analytics infrastructures, medium-sized organizations susceptible to staff turnover and maintenance burdens should carefully consider their ability to maintain homebuilt stacks over the long term.
- Or buy the wrong products from trendy vendors: Non-technical buyers especially need to aggressively vet what they’re buying in the AI economy where there can be more hype than reality.
Why Do Some Data Programs Prosper, and How Will They Succeed With AI?
The best data teams we’ve worked with follow a short-list of best practices, which makes them successful through trends and ongoing executive pressures.
- They pick the best method for the problem: Smart data teams don’t default to AI. Smart data teams choose the most suitable – not the fanciest – method for a given problem. This can vary from employing straightforward approaches like linear regression for simple data to employing more intricate machine learning methods for handling vast multi-dimensional datasets. They define the exact scope of the problem and determine an approach aligned with their time constraints.
- They focus on agility and iteration: Smart data teams iterate over time, typically beginning with simple problems and data infrastructure and then building solutions toward more complex, multi-department challenges as they prove themselves and their methods correct.
- They ask ‘how is this going to get used?’ on day one: The hardest problem in analytics is not the math – it’s changing what people do at work because of what the math says. Smart data teams will evaluate new AI investments by asking how the results will (or will not) be used by real people many months in advance. If there’s no path to using the results, a good team will kill the project before it starts.
- They invest in right-sized technology for their stage: While enterprise teams can make some risky investments, medium-sized data teams will typically buy out of the box solutions that are fit for their needs vs. building from scratch.
- They see AI as a journey not a trend: Recognizing that the journey to AI is a gradual process, successful programs make early investments in fundamental aspects like data quality. They identify specific problems within the decision cycle that AI can genuinely solve and strategically implement the right solution, knowing that AI adoption is a long-term commitment.
- And they have lots of fun: Great data programs conduct open experiments with technical stakeholders to measure the merits of different approaches, and AI will be no different.
Data programs today are under incredible pressure to ‘deliver AI’, often with an unclear definition or reason of what that means, which introduces a lot of failure risk as we saw with previous trends. Great teams will partner with their executives, ignore the trends, and solve definable problems while deliberately and carefully incorporating AI into work. And, most importantly, they’ll have a lot of fun doing it. And, of course, if you’re interested in advancing your own AI journey, feel free to reach out.
