How to successfully productise AI

  • Post by Rachel
  • Aug 23, 2020

AI is not a fad. Predicted to be a $15.7 trillion game changer, its gargantuan economic impact is expected to transform productivity, increase global GDP by 14% and boost global economies by 26% by 2030.

In 2017, Vladimir Putin said that whoever leads in the development of artificial intelligence will rule the world. Jeff Bezos declared this the golden age of AI and noted we’ve only scratched the surface of what’s possible. Andrew Ng, AI industry leader, Stanford Professor and ex-Google Mind, believes that just as electricity changed the world 100 years ago when it upended transportation, manufacturing, agriculture and health care, AI is poised to have a similar impact on every industry. Microsoft billionaire Bill Gates says artificial intelligence is “so incredible, it will change society in some very deep ways.”

“While some markets, sectors and individual businesses are more advanced than others, AI is still at a very early stage of development overall. From a macroeconomic point of view, there are therefore opportunities for emerging markets to leapfrog more developed counterparts. And within your business sector, one of today’s start-ups or a business that hasn’t even been founded yet could be the market leader in ten years’ time. “

- PwC Global Artificial Intelligence Study: Exploiting the AI Revolution

But despite the hype and expectation, there’s still a significant amount of weariness and hesitation when it comes to AI. AI is still at a very early stage of development overall and its adoption is sometimes lagging - A 2019 survey by the MIT Technology Review on the barriers to AI adoption shows that companies are encountering obstacles such as challenges in leadership, vision, expertise, and data quality.

While most companies are already implementing AI strategies and mature initiatives, 40% of organisations surveyed by Google and MIT are still awaiting overall AI transformation and cautious about adoption, with a minority of “traditionalists,” - around 11% - saying they are more apt to use standard, well-established technologies and don’t have AI adoption on the horizon.

Meanwhile, Big Tech giants and industry leaders (Amazon, Alphabet, Facebook, Apple, Microsoft, Netflix, Nvidia, LinkedIn, Tesla, IBM, Tencent) are light years ahead of the game through heavy investment in AI and already revolutionising and disrupting many industries. These companies are using techniques such as machine learning and neural networks routinely as a core function of their business and delivering unparalleled customer experiences. They are betting big on AI and - unsurprisingly - their business domination continues to amplify even during a global pandemic and one of the worst recessions the world has ever seen.

“Day 2 is stasis. Followed by irrelevance. Followed by excruciating, painful decline. Followed by death. And that is why it is always Day 1…The outside world can push you into Day 2 if you won’t or can’t embrace powerful trends quickly. If you fight them, you’re probably fighting the future. Embrace them and you have a tailwind…

At Amazon, we’ve been engaged in the practical application of machine learning for many years now. Some of this work is highly visible: our autonomous Prime Air delivery drones; the Amazon Go convenience store that uses machine vision to eliminate checkout lines; and Alexa, our cloud-based AI assistant. (We still struggle to keep Echo in stock, despite our best efforts. A high-quality problem, but a problem. We’re working on it.)

But much of what we do with machine learning happens beneath the surface. Machine learning drives our algorithms for demand forecasting, product search ranking, product and deals recommendations, merchandising placements, fraud detection, translations, and much more. Though less visible, much of the impact of machine learning will be of this type — quietly but meaningfully improving core operations.”

- Jeff Bezos’ letter to shareholders (2017)

Which begs the question - if AI is so great why isn’t every company, big or small, using it? It turns out the challenges are more cultural and organisational than technical.

What’s really holding back AI adoption

The business buzz around machine learning (ML) has been steadily growing since the early days of big data. Today, ML is beginning to deliver on the potential created by big data and analytics by turning raw data into useful, predictive tools for business. Innovation-driven companies with agile, collaborative and groundbreaking cultures are launching AI initiatives that promise real benefits and significant ROI.

O’Reilly survey results on AI adoption in the enterprise (March 2020) show that while many AI efforts are maturing from prototype to production, 22% of respondents named a lack of institutional support as the biggest problem, not a skills gap (although that’s also a major challenge). Difficulties in identifying appropriate AI business use cases was cited by 20% of respondents.

The 2019 MIT Sloane-BCG Artificial Intelligence Global Executive Study shows that 40% of organisations making significant investments in AI do not report business gains from AI. Many AI initiatives fail.

“The crux is that while some companies have clearly figured out how to be successful, most companies have a hard time generating value with AI. As a result, many executives find themselves facing a set of AI realities: AI is a source of untapped opportunity, it is an existential risk, and it is difficult. Above all, it is an urgent issue to address.

- 2019 MIT Sloane-BCG AI Global Executive Report

A growing number of leaders say AI is not just an opportunity but also a strategic risk: “What if competitors, particularly unencumbered new entrants, figure out AI before we do?” In 2019, 45% perceived some risk from AI, up from an already substantial 37% in 2017. This shift suggests an increasing awareness of and concern with competitors’ use of AI. The challenges are also with large-scale implementation and deployment of AI, which are necessary to achieve value.

The bottom line is that it seems - outside of Big Tech - two things are widely misunderstood:

1/ What are AI’s capabilities and value creation potential.

2/ How to successfully productise AI.

“I know it’s there; I just don’t know how to take advantage of it”

As Google points out, when it comes to innovative and disruptive ideas, what often bridges the gap between ambitious ideas and brilliant outcomes is a company’s ability to effectively make data-driven decisions and execute at scale. That’s where machine learning (ML) comes in - it can help companies cleverly and rapidly optimise data at scale.

“ML is particularly adept at finding patterns in complex datasets to solve complex problems, including perceptual tasks, such as visual perception and speech recognition. The use cases are both wide reaching and dynamic. Manufacturers, for example, are streamlining their capital expenses by implementing predictive maintenance. Financial institutions are enhancing their risk analysis. Retailers and media providers are personalising their customer experience. And the travel industry is offering their customers dynamic pricing predictions. These advances — combined with exponential gains in the cost of data storage, compute power, AI-centric hardware, and cloud computing — have democratised AI for industry in an unprecedented way.”

- Google Cloud’s AI Adoption Framework

A recent MIT & Google Cloud survey shows that ML adoption results in 2x more data-driven decisions, 5x faster decision-making, and 3x faster execution. Enterprises that invest in building industry-specific AI solutions are proven to be better positioned as future global economic leaders. By 2030, companies that fully absorb AI could double their cash flow.

“Machine learning is basically a way for a computer to find the nuggets of information that a human can’t. Once you have your data and train and deploy your models, the machine can go through terabytes of data and get smarter and smarter—basically train itself—and ultimately make predictions for you.”

- Fausto Ibarra, director of global product management for Google Cloud Platform.

AI is set to be the key source of transformation, disruption and competitive advantage in today’s fast changing economy. The most common projects among current ML implementers are image recognition, classification and tagging (47%); emotion/behaviour analysis (47%); text classification and mining (47%); and natural language processing, or NLP (45%).

The low-hanging fruit for companies planning to leverage AI nowadays is to take unstructured data such as images, e-mail, product reviews, customer service calls - and make sense of it. But AI’s most valuable commercial opportunities could create much bigger breakthroughs and touch nearly every imaginable sector. PwC has identified more than 300 AI use cases across 8 key industries - improved medical imaging diagnosis and pandemic prediction in healthcare; cost saving autonomous fleets and predictive maintenance in automotive; personalised financial planning, swift fraud detection and 3-second claims processing in financial services; anticipating customer demand and inventory and delivery management in retail; customised content and personalised advertising and marketing in media; smart metering, more efficient grid operation and storage and predictive infrastructure maintenance in the energy sector.

AI is applicable across all elements of the value chain. But there are still gaps between expectations and outcomes, with some initiatives intended to create next-generation products or deliver internal process efficiencies, not entirely living up to expectations. So what does it take to successfully productise AI?

Productising AI is difficult

AI products are automated systems that collect and learn from data to make user-facing decisions or predictions. ML technology (a subset of AI) - which underlies many AI products today - uses statistical techniques to give computer systems the ability to “learn” by being trained on existing data. After training, the system makes predictions based on new data which it hasn’t seen before.

In the best case scenario, the trained neural network accurately produces the correct output even when presented with new input data the model didn’t see during training. For machine learning systems used in consumer internet companies, models are often continuously retrained many times a day using billions of entirely new input-output pairs. LinkedIn’s personalised job ads, Netflix’s content suggestions, Amazon’s shopping recommendations and Google’s search results are all AI products.

AI projects can be complex and challenging due to certain AI idiosyncrasies:

  1. 1/ Tactical organisational approach - tactical (rather than strategic) approaches to AI can set up a project for failure because of lack of executive sponsorship and a clear AI strategy. Tactical projects only tend to explore AI’s short term potential and focus on narrow use cases, leveraging ready-to-use AI and ML services for proofs of concept and prototyping. ML is viewed as unattainable and complex problems are outsourced. Such ML projects are often driven by individual efforts and so might not be aligned with the organisation’s business goals and may have only limited business impact. Also, a lack of process or the skill sets to scale AI solutions consistently can stall or kill the project entirely without delivering any meaningful outcomes. A preferred strategic approach focuses on delivering sustainable business value, with several ML systems deployed and maintained in production that leverage both ready-to- use and custom models. A strategic approach means that ML is not seen as the domain of a select few, but a pivotal accelerator for the business where dedicated skilled teams build solutions for various ML use cases across business functions and the entire organisation gets behind the strategy and adoption.
  2. 2/ Bad data - Lack of integrated, clean, and fresh data make your machine learning tools useless. You won’t be able to gain actionable insights tailored to your needs, which translates to murky models and undermines the power of ML. Data is often siloed across many lines of business and needs to be brought into a central, unified data lake that makes it easier to derive insights from unstructured data and easier also to perform batch integration for reporting.
  3. 3/ Schedule uncertainty - AI products may take a while to get right as models may need to be built from the ground up, trained, validated, launched and then monitored in live environments, tweaked and continuously retrained to improve. Often, data collection, cleaning and labelling can be incredibly time consuming and must take place before the training phase can start. It’s therefore tricky to guarantee results on a specific timeline.
  4. 4/ High cost - ML projects require solid data infrastructure and highly skilled staff. They’re iterative and never ending. Various costs such as compute power, hiring, training, software, business process change, cloud services and integration can add up.
  5. 5/ Accuracy and relevance - ML models typically degrade in production and require continuous iteration and training on fresh data to improve. ML models rarely deliver 100% accuracy and perfect predictions. Tradeoffs are usually made between ML-specific performance metrics (such as accuracy vs relevance), depending on the product’s use case. “The model is produced by code, but it isn’t code; it’s an artifact of the code and the training data. That data is never as stable as we’d like to think. As your user base grows, the demographics and behavior of the user population in production shift away from your initial training data, which was based on early adopters. Models also become stale and outdated over time. To make things even more challenging, the real world adapts to your model’s predictions and decisions. A model for detecting fraud will make some kinds of fraud harder to commit–and bad actors will react by inventing new kinds of fraud, invalidating the original model.”
  6. 6/ Opacity - ML models can be buggy, or difficult to understand and explain both internally and to your customers. It’s not always straightforward to know why a certain prediction was made or how to debug issues such as bias. Tools like Google’s Explainable AI help with the development of interpretable and inclusive ML models and are designed to improve data sets or model architecture and debug model performance.
  7. 7/ Inherent fairness and bias issues - Human biases can make their way into artificial intelligence systems - with harmful results. Being acutely aware of those risks and working to reduce them is a priority and part of the development and design process.
  8. 8/ Hard to estimate ROI - Unlike traditional hardware or software investments, where the costs and impacts are more neatly defined and predictable, estimating future returns for AI projects can be challenging. Some AI applications link neatly to projected returns, making ROI calculations straightforward. Other applications are more complex and unpredictable, making it challenging to use typical ROI approaches. Not all large datasets will necessarily contain useful patterns, let alone patterns that can provide more value than the cost of looking for them. Sometimes the commercial advantages gained by certain product features may not justify the effort/cost but that may not be apparent until after the product is launched. A/B testing (i.e. comparing the new application vs the status quo) may help with assessing value. Assessing AI products’ ROI is usually heavily customised to each organisation’s data and circumstances.

Why are AI projects different to standard software development

“AI systems differ from traditional software in many ways, but the biggest difference is that machine learning shifts engineering from a deterministic process to a probabilistic one. Instead of writing code with hard-coded algorithms and rules that always behave in a predictable manner, ML engineers collect a large number of examples of input and output pairs and use them as training data for their models.”

- What you need to know about product management for AI by Peter Skomoroch

Diving deeper into the technical, Trevor Back, a Google/DeepMind product manager gave an interesting talk at ProductCon in London in 2019 where he explains why AI products present such unique challenges. In a way, it helps to look at AI as more akin to R&D.

For starters, AI isn’t fairy dust - you can’t just plug into an off-the-shelf cloud API and immediately see great results. It’s still evolving (albeit at exponential pace) and, while many problems have been solved, there are still many unresolved questions that require a bespoke approach and plenty of experimentation and iteration. It may involve changing all or a significant part of your technical plumbing (unlike mobile where the stack will likely be a lot smaller).

Secondly, the success of AI projects depends on having an “extreme measurement culture” that heavily utilises analytics, gathers data in a structured way, has an experimental mindset founded on rapid iteration and can build on that to execute its AI projects. Many companies at the start of their AI journey haven’t been tracking data in any meaningful way and this can set them back from implementing AI in real products by at least a couple of years.

Thirdly, if you’re working on an AI research project, there is no product to manage yet. So your focus as a product manager would be on the problem at hand, not any specific product or feature. This means focusing on the impact of the problem you’re trying to solve on end users and operating in a much higher state of uncertainty about the ways that problem could be solved (for example there may be various ways in which AI can help with medical diagnosis or self-driving cars and there’s rarely one established technical solution to these problems).

“With machine learning, we often get a system that is statistically more accurate than simpler techniques, but with the tradeoff that some small percentage of model predictions will always be incorrect, sometimes in ways that are hard to understand. This shift requires a fundamental change in your software engineering practice. The same neural network code trained with seemingly similar datasets of input and output pairs can give entirely different results. The model outputs produced by the same code will vary with changes to things like the size of the training data (number of labeled examples), network training parameters, and training run time. This has serious implications for software testing, versioning, deployment, and other core development processes.”

- What you need to know about product management for AI

The AI skillset

So how do companies like Amazon, LinkedIn, Google and Apple launch successful AI products? Peter Skomoroch, Head of Data Products at Workday and previously a LinkedIn Research Scientist has a few valuable insights into the AI product pipeline:

  1. 1/ Have a good intuition for how ML models work and what’s possible with the data you have. It’s imperative to know what’s easy, hard and impossible to do with AI technologies and how long the product would take to build.
  2. 2/ Even if a product is feasible, you’ll need to determine whether there’s a product/market fit, i.e. would your AI product solve a real user problem and will the value justify the effort and cost? The payoff will need to match the investment. Overpromising on something that doesn’t show good or meaningful ROI can undermine the team.
  3. 3/ Start with a mission & near term strategic objectives - take data science ideas and line them up thematically; estimate T-shirt size (S/M/L) effort and do the same for impact. Work on things that matter to the business and pair applications to those objectives - for example, ML metrics such as accuracy could have an impact on business metrics so find an AI application that ties directly into business metrics.
  4. 4/ Be metrics and data driven - you’ll need to help gather the right data, know your data inside out, understand its quality, limitations and biases and be prepared to monitor your models, iterate and improve them continuously. Build a system that collects data accurately from day one.
  5. 5/ Adopt Agile ways of working - AI projects are closer to R&D in nature. Get working with data for prototyping and testing, involve UX designers early, focus on feature engineering, get Version 1 out as soon as possible as 80% of the work will happen once the product is live, feed back new training data and monitor for continuous improvement.
  6. 6/ Traditional testing will break in the AI context. The “data flywheel” principle (aka the virtuous cycle of AI) means that more users get you more data which lets you build better algorithms and ultimately a better product to get more users. Rinse & repeat). Get a core version of the product to work and test progression. You’ll need real production data from real users to improve. This will also highlight what input data needs to be adjusted and what prediction errors emerge in real time.

Take a leap of faith

AI clearly has the potential to fundamentally disrupt your market, create innovative new products and services and erect entirely new business models. It will surpass the creative destruction of the first digitisation wave of the past decade. Some of the market leaders in the next decade years’ time may be companies you’ve never heard of. Some of today’s biggest names could become irrelevant or disappear altogether after a slow painful decline, if they’re too late or too slow to the game.

Think about:

  • How vulnerable your business model might be to AI disruption;
  • What are the game-changing openings within your market and how can you take advantage;
  • Could you do things that have never been done before, rather than simply automating or accelerating existing capabilities;
  • Do you have the right talent, data and technology to execute on AI opportunities.

Get it right and, in Jeff Bezos’ words, it’ll always be Day One.

More reading

What You Need to Know About Product Management for AI, O’Reilly article by Peter Skomoroch & Mike Loukides

AI Adoption in the Enterprise, O’Reilly article

Google PAIR (People & AI Research)

Google Cloud’s AI Adoption Framework

The Data Flywheel: How Enlightened Self-Interest Drives Data Network Effects

PwC interactive tool for measuring AI impact per sector