Singh Spectrum

“In investing, what is comfortable is rarely profitable.” – Robert Arnott

…and investment in Artificial Intelligence is not a comfortable problem to tackle. The speed with which the AI related technologies are moving warrants high decision and action velocity. However, since AI defies the traditional rules of doing business (for example, majority of the value offered is qualitative), it doesn’t respond well to traditional measurement frameworks. Therefore, while all of us may want to fit the square peg of AI in the round hole of traditional business processes, there is no way to do it while maintaining profitability.

Hence, it is important to understand what works and what doesn’t when trying to measure the ROI on AI Investments, and have a robust framework to evaluate not only the quantitative, but the qualitative impact of AI on the bottomline. In this article, I will present to you an approach for measuring AI ROI that can be tailored to your business. Before we get into the specifics of the measurement framework, we must understand the potential areas of impact from AI.

Impact of AI on Enterprise Operations

According to research done by Deloitte, the areas of positive ROI from AI investments are illustrated here.

This gives us a tool to prioritize AI investments as applicable.

Types of Investments in and Returns from AI

There are 2 types of investments and returns from AI, categorized as Hard and Soft. 

Hard investments and returns are measurable directly or indirectly (using some transformation from qualitative to quantitative). These respond directly to tactical methods.

Soft investments and returns are not measurable directly or indirectly. While their effects are visible, they cannot be quantitatively connected to or related with the corresponding investments or returns. These are the unique components of AI ROI and have to be handled through robust strategic approaches.

PwC research has categorized these as follows:

Return on Investment (RoI) in Artificial Intelligence (AI)

The framework illustrated here consists of 4 pillars:

1. Evaluating AI Readiness

2. Evaluating Cost & Benefits (Hard & Soft)

3. Business Case Evaluation & Prioritization

4. Estimating Cost of not investing

Evaluating AI Readiness

The two key imperatives towards AI readiness of any organization are Data and Skill. When evaluating the data readiness, identify areas, LOBs and divisions that have high maturity with respect to Data Quality, Data Integration and Data Security. This will ensure that your AI implementation has a proper data platform to sit on. Remember that in the end an AI system analyzes data, and the principle of Garbage in-Garbage Out is very much applicable. A similar principle applies to the use of this data in AI implementation, which will be ineffective without skillful manpower. The key skills include understanding of the nature of data (as it pertains to business) and understanding of contemporary AI technologies. The parts of business that sit at the intersection of these 2 imperatives are rife for implementation of AI usecases.

Evaluating Cost and Benefit for AI Projects

The cost of an AI project is difficult to calculate due to several added dimensions like Training Time, Inference Costs, effectiveness of the models etc. Therefore, a structured approach to evaluation is required, which breaks down the subjective process of estimation to a more concrete evaluation process. This could consist of breaking down the process into concrete tasks, with the following attributes:

1. Resources needed

2. Rates

3. Duration

4. Team Composition

5. Number of Iterations

6. Data to be Collected

7. Cost of Support

8. Cost of Ongoing Maintenance

9. Cost of Errors

This would give the estimated cost of AI. This also accounts for the cost of error, which is generally ignored when evaluating traditional projects. Cost of Error is also the single most significant cost that leads to Bill Shocks when using AI. This consists of calculating the following for Classifiers and Recommenders:

1. Cost of False Positives

2. Risk of False Negatives

Evaluating the benefits of an AI project consists of evaluating Hard and Soft benefits. For Hard returns, the revenue gains should not focus only on the best case where all AI expectations are met (which is rarely the case), but on the revenues gained after accounting for the cost of errors. While soft returns cannot be evaluated quantitatively, key tenets can be established with assumption of benefits. This can be done based on common sense, past data or peer experiences. Following are some key tenets that can serve as a starting point.

1. Productivity translates to Profitability – Therefore any initiative with the promise of productivity increase can be considered profitable

2. Data Strategy before AI Strategy – Presence of a mature and effective data strategy is a must for the success of any AI project.

3. Start with Business Objective – Any AI project needs to be clearly associated with a Business Objective. There are many shiny usecases driven from hype or fear-of-missing-out. However, it is always best to focus on your specific business and what makes sense for it

4. Goals for the AI project should be SMART

5. Observability is Mandatory – Most projects struggle to justify the cost due to sheer lack of the ability to assess the performance of AI investments. A robust and well integrated observability strategy is paramount to a longer lifespan of AI implementations.

Business Case Evaluation & Prioritization

Once the area of implementation and cost of the AI projects have been evaluated, it is important to have a structure to prioritize the business cases and understand which ones need to be tackled first for maximizing value. 

Quick Wins are the Business Outcomes with a shorter timeframe and clear path to completion. These are ideal for AI implementations, as you do not want to spend too much time (and cost) on your initial AI projects. As the AI strategy becomes more well defined and has endured some test of time, the focus can shift towards longer AI projects.

Differentiating Usecases are areas that set you apart from your competition. It is not enough to implement quick wins that are run-of-the-mill as they do not unearth enough value to justify ongoing costs of AI initiatives. Therefore, these types of usecases are ideal to establish the value proposition.

Transformation Initiatives are business objectives that result in the creation of additional revenue streams. In other words, they change the way of doing business by your organization.

An ideal AI initiative will lie at the intersection of all three evaluation criteria. However, it is rare to discover such a use case right off the bat when starting with AI. The way to prioritize the Business Cases should be:

1. Top Priority to use cases in the center

2. Second Priority to any use cases that lie at the intersection of two of the three areas

3. Third Priority to any use cases that lie in at least one of the three circles

This structure ensures that usecases with most relevance to business are not overshadowed by usecases that seem more attractive due to peer pressure or market hype.

Estimating Cost of not investing

With a business that is already running profitably, it may be difficult to consider modifying the working strategy for something new that incorporates the use of AI. That is why it is important to carefully consider the cost of not investing in AI to ascertain its priority in relation to the overall enterprise business strategy. Here is how you can assess the opportunity cost of not investing in AI:

1. Observe Competition: Evaluate how competitors leveraging AI are outperforming in areas like efficiency, customer experience and innovation.

2. Identify Operational Inefficiencies: Identify the processes within your enterprise that could be automated or optimized with AI, and compare the cost with that of maintaining the current manual processes.

3. Assess revenue loss from Missed Opportunities: Identify the prospective revenue that can be gained from predictive analytics, personalized marketing or intelligent customer support and compare against the cost of implementing these using AI

4. Identify the cost of Slow Market Adaptation: Assess the financial impact of inability to respond quickly to shifting market demands, which AI-enabled insights and automation could address.

5. Consider the cost of Talent Turnover: The cost of losing top talent to competition, which is a normal occurrence due to limited long-term growth visibility, could be huge as it will cost more to train new talent while increasing competition due to attrition to competitors.

Conclusion

While AI adoption is an inevitable action for any growth/profit oriented business, plunging into it without a well thought of approach can cause more harm than good, and also burn your fingers too early to try again boldly. Keep the following in mind as you decide to welcome AI into your business strategy mix.

1. Don’t rush to solutioning. Start with Business Case

2. Identify gaps before addressing areas you may already be good at, to illustrate initial impact to the board

3. Do not over analyze

4. Don’t quantify soft benefits. Take a tenet based approach to guide you

5. Involve Finance team early to understand any financial constraints

6. Use NPV to understand long term value of your investments

7. Account for and enable AI learning curve for your AI models

8. Be transparent with stakeholders

Remember, with the right strategies in place, AI can become a powerful tool for innovation, efficiency and competitiveness.