How AI Products can have Double Impact

My Dual Impact Mental Model for AI Product Managers

Jul 21, 2024

#beyondAI - At times, it becomes challenging to assess an AI use case and determine if it presents a real business opportunity or is just an AI Use Case without real value. However, we must conduct thorough assessments for every AI use case before diving into development. This ensures that the AI solution we build evolves into a valuable AI product, rather than just a technically impressive solution with no real business or user impact.

But if we don't know exactly how AI Products impact processes and which KPIs are influenced, we might run an insufficient or even incorrect assessment. This means we could end up prioritizing based on wrong assumptions.

This issue aims to help every AI Product Manager understand how AI Products create value, making it easier to run effective assessments.

You will learn:

The inner workings of Organizations
The Dual Impact of AI Products on Organizational Processes
Why Process Efficiency is Easier to Measure
Key Takeaways for AI Product Managers

And of course, you'll read about the invaluable contributions from my AIPM community that made this issue even better. The credits go to…

Happy reading! 🛋️

Before starting: If you’ve found my posts valuable, consider supporting my work. You can help by sharing, liking, and commenting on my LinkedIn posts introducing this issue. This helps me reach more people on this journey.

Thank you for supporting and motivating me to continue sharing my experience with you. Thanks for being part of this community ❤️.

For internal AI product managers, truly understanding how your company operates is essential. This insight forms the foundation for identifying the most impactful problems that AI can solve and ensures that your solutions align with the company’s strategic goals. Comprehensively understanding both business processes and operational dynamics has always been my superpower in identifying the right needs and making it easier to talk to those experiencing the problems. Their buy-in was almost immediate—okay, that's a bit of an exaggeration. 🙂

But let's see what my understanding of the inner workings are:

The inner workings of Organizations

My understanding of the inner workings of organizations, excluding organizational politics, can be simplified to one sentence: an integrated system of interconnected processes and non-process activities designed to create and sustain value, maintained and executed by humans and machines.

At least, this is the main objective for each company.

The term integrated system emphasizes that these processes and activities are not just interconnected but are part of a cohesive whole.

Designed implies intentionality in how these processes and activities are structured to achieve organizational goals.

The phrase create and sustain value highlights both the generation of new value and the maintenance of existing value, covering the full spectrum of an organization’s purpose.

Maintained and executed by humans and machines underscores the collaboration between human expertise and technological tools in these processes and activities.

For example, the HR Department manages processes related to recruitment, training, and employee relations. These processes are part of an integrated system because they connect with other departments to ensure a competent and satisfied workforce. They are designed to attract, develop, and retain talent, directly contributing to the organization’s ability to create and sustain value. These processes are maintained and executed by humans and machines, such as HR specialists and recruitment software.

The IT Department oversees the technological infrastructure and support processes. These processes form an integrated system by enabling seamless communication and data flow across the organization. They are designed to support operational efficiency and innovation, helping the organization create and sustain value. The processes are maintained and executed by humans and machines, including IT professionals and automated systems.

The Product Department handles processes related to product development, quality assurance, and lifecycle management. These processes are an integrated system as they involve coordination with R&D, marketing, and sales. They are designed to bring high-quality products to market efficiently, thus creating and sustaining value. These processes are maintained and executed by humans and machines, such as product managers and development tools.

The Marketing Department manages processes like market research, advertising, and customer engagement. These processes constitute an integrated system by linking with sales and product development to ensure alignment with market demands. They are designed to understand and influence customer behavior, thus creating and sustaining value. The processes are maintained and executed by humans and machines, including marketing strategists and analytics platforms.

While every process can potentially be measured by Key Performance Indicators (KPIs), non-process activities are less measurable. Processes are typically structured and have defined steps and outcomes, making them easier to quantify and evaluate using KPIs.

In contrast, non-process activities are often ad-hoc, spontaneous, and not bound by a predefined structure, making them inherently more challenging to measure. These activities rely heavily on human intuition, creativity, and real-time decision-making, which are not easily quantifiable. For instance, a manager’s quick decision to resolve a conflict, a spontaneous brainstorming session that sparks innovation, or an impromptu customer interaction that builds loyalty are all valuable but difficult to capture with traditional KPIs.

However, while they are less measurable, non-process activities play a crucial role in the agility and adaptability of an organization. They enable quick responses to unexpected situations, foster innovation, and enhance the overall dynamism of the workplace.

Based on my understanding and this argumentation chain, I strive to build AI Products to improve processes as they are more easily measurable. Once my AI Product seamlessly integrates into these processes, it becomes straightforward to track its impact through established KPIs.

Well, that’s a lie 😊

I have experienced two different impact dimensions when integrating AI Products into company processes. That’s why today I want to introduce you to my DUAL Impact Mental Model for AI Products and then explain why one dimension is easier to measure than the other.

🎓 I am offering this framework under the MIT License, an open-source license allowing free use, modification, and distribution, as long as proper attribution is given. Feel free to use, adapt, and share this model to suit your needs and help drive the dual impact of AI in your own projects.

The Dual Impact of AI Products on Organizational Processes

AI products have the remarkable capability to drive significant improvements in both business performance and operational efficiency. This dual impact is what makes AI so transformative for organizations aiming to stay competitive and innovative in today's rapidly evolving market, and reaching operational excellence.

The double impact of AI products on both the top line (Process Effectivity) and bottom line (Process Efficiency) underscores their transformative potential. I always recommend that AI Product Managers seek out problems that require AI solutions addressing both aspects. However, this is easier said than done. Let's break down the key components of my mental model to understand this dual impact better:

Top Line Effect: Process Effectivity

Process Effectivity refers to enhancing business KPIs, which are crucial indicators of an organization's success and growth. AI products can improve these KPIs, such as :

Sales Growth: AI can analyze vast amounts of customer data to identify sales opportunities, optimize pricing strategies, and personalize marketing efforts, leading to increased sales.
Churn Rate: By predicting customer churn through data analysis and offering personalized retention strategies, AI helps in retaining customers and reducing churn rates.
Net Promoter Score (NPS): AI enhances customer satisfaction by providing better customer service through chatbots, personalized recommendations, and proactive issue resolution.

These improvements in business KPIs collectively enhance the overall Process Effectivity, driving the top-line effect.

Bottom Line Effect: Process Efficiency

Process Efficiency focuses on optimizing operational KPIs, which are critical for reducing costs, improving productivity, and ensuring smooth operations. AI products streamline these processes by automating repetitive tasks, optimizing resource allocation, and improving accuracy. The impact of AI Products on operational efficiency can be measured by KPIs, such as:

Error Rates: AI-powered systems reduce human error by automating complex and repetitive tasks, ensuring higher accuracy and consistency in operations.
Labor Costs: Automation through AI reduces the need for manual labor, leading to significant cost savings while allowing employees to focus on higher-value tasks.
Cycle Time: AI optimizes workflows and processes, shortening the time required to complete tasks and accelerating time-to-market for products and services.

Enhancing these operational KPIs boosts Process Efficiency, driving the bottom-line effect.

🤔 Why It’s Called Top Line and Bottom Line Effect 

The terms top line and bottom line come from financial statements. The top line refers to a company’s revenue or sales, which is listed at the top of the income statement. This is why improvements in business performance metrics, like sales growth and customer satisfaction, are called top-line effects. On the other hand, the bottom line refers to the net profit, which is listed at the bottom of the income statement after all expenses are deducted. This is why improvements in operational efficiency, like reducing costs and increasing productivity, are called bottom-line effects.

Why Process Efficiency is Easier to Measure

AI products that successfully impact both lines—Process Effectivity and Process Efficiency—are the ones every AI Product Manager strives to build. While the mental model demonstrates how AI products can benefit organizations, finding such an AI solution and proving its impact can be challenging. Based on my experience, proving the impact on the Bottom Line is often easier than on the Top Line:

Advantages in Measuring Process Efficiency

Clear, Quantifiable Metrics:
- Operational KPIs such as Error Rates, Labor Costs, and Cycle Time are straightforward to measure. These metrics are typically numerical and well-defined, making it easier to track changes and attribute improvements directly to the AI product.
- For instance, if an AI solution reduces the error rate from 5% to 1%, this 4% reduction is a clear and quantifiable improvement.
Immediate and Tangible Results:
- Changes in operational efficiency often produce immediate and observable results. When an AI product automates a manual process, the reduction in labor costs or the time saved can be seen almost instantly.
- For example, if implementing an AI-powered chatbot reduces the need for customer service agents by 20%, this impact can be measured immediately in terms of cost savings and resource reallocation.
Direct Cost Savings:
- The cost savings from improved process efficiency, such as reduced manual labor, lower error rates, and decreased material waste, are directly reflected in financial statements. This direct reflection makes it easier to validate and attribute these savings to the AI product.
- An example is when AI optimization in inventory management reduces wastage by 15%, translating directly into cost savings that can be documented and reported.
Simplified Validation:
- Validating the impact on operational efficiency often involves fewer variables compared to business performance metrics. The cause-and-effect relationship between AI implementation and efficiency improvements is typically more straightforward.
- For instance, if an AI algorithm streamlines a production line, resulting in a 10% faster cycle time, this improvement is directly linked to the AI implementation without significant external factors complicating the validation process.

As you can see, the advantages of measuring the impact on Process Efficiency are quite clear. However, when it comes to Process Effectivity, things get more complicated. What makes the bottom line easier to measure are exactly the factors that make the top line more challenging. But let’s break this down further for better clarity:

Challenges in Measuring Process Effectivity

Indirect and Multi-Faceted Impact:
- Business KPIs like Sales Growth, Churn Rate, and NPS are influenced by various factors, making it difficult to isolate the impact of the AI product. External influences such as market conditions, competitive actions, and broader economic trends can affect these metrics.
- For example, an increase in sales might be attributed to a successful marketing campaign or seasonal demand, rather than solely to an AI-driven recommendation system.
Delayed Effects:
- Improvements in business KPIs may take time to manifest. Customer satisfaction and loyalty, for example, may improve due to enhanced service quality, but this improvement might not translate into increased sales or reduced churn immediately.
- If an AI-powered customer service tool enhances customer interactions, the positive impact on NPS might be observed over several months as customer perceptions gradually improve.
Complex KPIs and Subjectivity:
- Business KPIs often involve complex, qualitative measurements. Customer satisfaction and NPS, for instance, are subjective and can be influenced by individual perceptions and experiences.
- Measuring the direct impact of AI on these metrics requires comprehensive data analysis and may involve customer feedback, surveys, and other qualitative assessment methods.

Key Takeaways

If you want to take away one thing, it's this: Measuring the impact of AI products on Process Efficiency is generally more straightforward than on Process Effectivity. The clear, quantifiable, and immediate nature of operational KPIs makes it easier to validate improvements and attribute them directly to AI implementations. In contrast, the multi-faceted, indirect, and often delayed nature of business KPIs poses greater challenges in measuring and proving the impact of AI on Process Effectivity.

Ultimately, success hinges on the ability to run effective experiments and accurately assign the impact, whether on the bottom or top line, to the implementation of an AI product in organizational processes and workflows. But this is a topic for another day.

Here are some more takeaways for quick recall:

Holistic Improvement: Aim for AI products that enhance both business KPIs and operational KPIs.
Strategic Development: Prioritize developing AI solutions that have a dual impact on both process effectiveness and efficiency.
Sustainable Growth: Using AI to boost both effectiveness and efficiency leads to long-term success and competitiveness.
Clear Metrics: Focus on quantifiable and immediate KPIs for easier validation of AI impact.
Embrace Complexity: Be prepared to navigate the complexities of measuring business KPIs and recognize the longer time frame for seeing results.
Experimentation is Key: Running well-designed experiments helps in accurately determining the impact of AI implementations.
Collaboration is Crucial: Work closely with different departments to understand their processes and identify where AI can make the most significant impact.
Continuous Learning: Keep evolving your understanding of both business processes and operational dynamics to refine AI solutions continually.

Credits to my AIPM community

I regularly refine my mental models, and this Dual Impact Model is no exception. Recently, I had a great discussion on one of my LinkedIn posts, where I sought to be challenged on this very topic:

The discussions were fantastic, and I had the chance to revise my statement thanks to the active collaboration of my network. I believe it's important to give credit where it's due:

Anna Maria: Your comment inspired me to add the terms "top line" and "bottom line" to my mental model.
Reuben Yeong: Your reminder to avoid mistaking metrics for targets, or worse, turning them into targets, was invaluable.
Maheep Gupta: Your insight about how being metrics-driven can sometimes stifle creativity and innovation was crucial for a balanced perspective.

And to all the rest who actively engaged through comments and likes, thank you!

That’s it for this week.

I hope my mental model helps you in creating your own.

JBK 🕊

P.S and a friendly reminder: If you’ve found my posts valuable, consider supporting my work. You can help by sharing, liking, and commenting on my LinkedIn posts introducing this issue. This helps me reach more people on this journey.