Over the last ten years, my work has slowly evolved from pure AI Product Management into a role that increasingly lives at the intersection of AI Strategy and AI Product Strategy. The transition did not feel like a deliberate move toward strategy. It felt more like following the problems wherever they led me.

In the beginning, my world was clear. Understand users. Shape solutions. Deliver value. Improve adoption. Measure outcomes. Like many product managers, I believed the hardest challenges lived inside the product itself. If we understood the problem deeply enough and built well enough, success would follow.

But over time, something kept happening. Good ideas struggled. Strong teams delivered promising AI solutions that never scaled. Technically sound systems disappeared after pilots. And the reasons were rarely technical.

I slowly realized that many AI problems were not product problems at all.

They were strategic ones.

Most of my experience comes from building AI inside large enterprises. Organizations with thousands of employees, deeply interconnected processes, legacy systems, governance obligations, and competing priorities. AI behaves very differently in this environment compared to startups. In startups, the central challenge is usually product–market fit. In enterprises, the challenge is often something else entirely. The question is less whether AI works, and more whether the organization is capable of letting AI work. I increasingly began to think of this as product–organization fit.

At that time, strategy still felt abstract to me. I associated it with executive discussions and long planning cycles, slightly detached from the reality of building products. Product work felt tangible. Strategy felt distant.

It took years to understand that strategy is not removed from execution. Strategy is an attempt to understand reality deeply enough so execution has a chance to work.

Today, I see strategy as a set of beliefs.

Strategy is a collection of convictions about how progress actually happens.

Every organization carries an idea of a future it wants to reach. Strategy lives between today’s constraints and that future ambition. It tries to answer a deceptively simple question:

Only later did I realize that this understanding closely aligns with how thinkers like Richard Rumelt describe strategy, not as planning activity but as diagnosing reality and choosing coherent action based on that diagnosis.

Once I began thinking about strategy this way, something unexpected became obvious.

The best product managers I had worked with were already doing strategy work long before anyone called it strategy.

Great product managers constantly form beliefs. They observe users and conclude that adoption matters more than feature completeness. They recognize that workflow integration beats technological sophistication. They learn that solving one painful problem creates more value than delivering many impressive capabilities. They understand that timing inside an organization often matters more than innovation itself.

These are not execution decisions. They are strategic judgments.

Modern product thinking, shaped by voices such as Marty Cagan and Teresa Torres, increasingly frames discovery itself as decision-making under uncertainty. Product managers constantly test beliefs against reality. Products punish wrong assumptions quickly. Adoption declines. Value disappears. Reality corrects you.

Product management, at its best, is strategy operating under fast feedback loops.

The transition into AI Strategy did not replace this mindset. It expanded it.

Instead of only understanding users, I began trying to understand organizations. Instead of asking why a product succeeds, I asked why organizations repeatedly struggle to let good products succeed.

Strategy increasingly felt less like planning and more like sense-making. Organizational theorists such as Karl Weick describe organizations as systems constantly interpreting reality rather than executing perfectly designed plans. That description resonated deeply with what I observed in enterprise AI environments.

This distinction becomes especially visible in AI.

In startup environments, product strategy and company strategy often evolve together because organizations are small and adaptable. In enterprises, these layers separate. Organizational complexity introduces friction long before product quality becomes the deciding factor. This is why AI Strategy becomes necessary even before individual AI products can succeed.

An AI Product Strategy emerges from discovery. We study how work is actually done, where friction exists, and what outcomes matter. We form beliefs about how a specific AI product creates value. These beliefs translate into product principles, experiments, epics, and roadmaps. The goal is clear: build something people adopt and trust.

AI Strategy asks a different question entirely:

How must the organization evolve so AI products can succeed repeatedly?

Here, discovery shifts from users to systems. We observe data accessibility, governance structures, delivery models, incentives, organizational maturity, and cultural readiness. We form beliefs about how AI value can realistically emerge within that environment.

One strategic belief that fundamentally changed my perspective was simple:

AI development must be treated as product development, not as one-time projects.

While this realization may sound simple and almost obvious to experienced product managers, its implications are far-reaching. Once we understand that many AI challenges are strategic rather than technical, the perspective changes. The question is no longer how to build better models, but how organizations must evolve to let those models create value. Decisions around funding, governance, ownership, and delivery suddenly look different. AI stops being a project to deliver and becomes a capability to nurture. Success moves away from completion toward sustained adoption, and learning becomes continuous rather than episodic. AI product teams therefore cannot disappear after go-live. They must remain with the product, continuously learning, improving, and evolving it instead of moving immediately to the next promising AI use case.

Another belief follows almost inevitably. If AI products evolve continuously, organizations must continuously discover AI opportunities. Innovation cannot depend on isolated initiatives. It requires structured observation and prioritization. From this belief emerges capabilities such as scalable AI Opportunity Management. Not as temporary programs, but as permanent organizational muscles connecting strategy with product creation.

This is where strategy becomes beautiful to me.

Strategy is not about inventing actions. It is about understanding consequences. When beliefs are coherent, organizations begin aligning almost naturally. Structures reinforce direction. Investments gain meaning. Decisions stop competing and start supporting each other.

At the same time, strategy work is difficult because organizations are complex systems. Improving one area often creates pressure elsewhere. Automating a workflow may accelerate one team while overwhelming another. A technically strong AI product can fail because governance was not prepared. A well-articulated AI Strategy can exist without impact if no meaningful products ever emerge.

Strategy requires living with ambiguity longer than product work usually demands. Feedback loops slow down. Impact becomes indirect. Instead of shipping features, you shape conditions. Instead of solving a single problem, you influence how many problems can be solved in the future.

Stepping into an AI Product Strategy Lead role therefore did not mean moving away from products. Quite the opposite.

The role lives exactly between strategy and product execution.

On one side, it contributes to AI Strategy by helping define organizational beliefs, capabilities, and directions that allow AI to scale responsibly and effectively. On the other side, it stays deliberately close to AI products themselves. Guiding product teams. Challenging assumptions. Supporting AI Product Managers in strengthening their product strategies. Ensuring discovery remains grounded in reality. Translating organizational ambition into adoptable solutions.

The role constantly translates in both directions.

Strategy informs products.

Product reality informs strategy.

AI Strategy without proximity to products becomes theoretical.

AI Product Strategy without organizational alignment becomes fragile.

My work increasingly focuses on connecting both worlds.

This connects deeply to something personal for me. My LinkedIn profile background carries the sentence:

Shaping a future where good AI products don’t fail for the wrong reasons.

That sentence expresses how I understand strategy today.

Most AI products do not fail because models are weak. They fail because organizations were not ready. Because incentives discouraged adoption. Because discovery was skipped. Because governance arrived too late. Because strategy and product evolved separately instead of together.

Looking back, I do not see a transition away from product management. I see an expansion of product thinking itself. The same curiosity that once helped understand users now helps understand systems. The same discovery mindset now applies to organizations.

Seeing AI Strategy through the eyes of an AI Product Manager ultimately means refusing to separate strategy from products. It means staying close enough to real work to remain grounded, while thinking broadly enough to shape the environment in which that work succeeds. Strategy then stops being an abstract exercise and becomes a continuous dialogue between organizational direction and product reality. Every strategic belief must eventually prove itself in adoption, usage, and sustained value creation. And every product insight becomes an input to strategy itself, shaping how the organization learns and evolves.

In the end, AI Strategy is not about defining a future from a distance. It is about creating the conditions in which good AI products are given a real chance to succeed.

JBK 🕊️

When we begin building an AI product inside a company, especially one aimed at solving real problems for real teams, it’s easy to assume that everyone is already aligned - after all, they brought the idea, or they nodded to ours. But what I’ve learned again and again is that even when everyone’s nodding in the same direction, their understanding of what happens next can be worlds apart.

Some teams believe that their job is done once they’ve shared the idea. Others believe that saying yes opens the door to unlimited feature requests, as if the AI product team were now an internal agency. Both perspectives are understandable, and both will quietly derail an AI product if we don’t correct course early.

The truth is, getting from idea to a mature, usable product is not a handoff. It’s a journey. A long, effortful one. And the moment someone says “yes,” the real work begins. First, we have to understand the problem deeply enough to run a Quick Assessment. If it’s promising, we move into a Detailed Assessment, which means checking data availability, defining success, shaping the first version, estimating costs, and aligning across different teams and layers of the organization. Even then, we’re just getting started.

Once we begin building, we step into an entirely different reality: turning intent into experience, building something that works in practice, with all its edge cases, user behaviors, and change management needs. And then, even after the product goes live, the journey continues. Every new feature idea, every new user group, every seemingly small request must go through some form of structured evaluation. AI products are never really finished. They evolve. And with each evolution, complexity grows. That’s why it’s so important to make expectations transparent early on.

This isn’t about bureaucracy. It’s about clarity. If we don’t name the expectations, we leave too much room for misunderstanding. The product team becomes reactive. Stakeholders become frustrated. Strategic focus slips. But if we do make things transparent, if we show what this journey really involves, and who needs to be involved at each stage, we create shared ownership. We give everyone a map, not just a promise.

The Reality of the Internal AI Product Journey

Building an AI product is not a single event. It’s not an app you build in a hackathon and hand over. In practice, AI product development has many gates, each requiring decisions, commitment, and careful attention.

• We start with Problem Discovery, where we try to understand if the problem really needs an AI solution - or if it’s a process, policy, or visibility issue in disguise.

• Then comes the Quick Assessment, where we decide whether it’s worth exploring further based on impact potential, data availability, and feasibility.

• If it’s still promising, we enter the Detailed Assessment, where we estimate the cost of development and ownership, build a lightweight prototype, and align with legal, security, and governance partners.

• And if the product idea passes all this, we enter the 0→1 phase. The first real attempt to deliver value through a working product.

• But even when the product is live, it’s not over. Every new request, every additional user group, every iteration - it all requires us to revisit what we’ve built and make sure it still holds.

AI products are not static. They are living solutions that grow with the business, and so the people involved must grow with them. Which brings us to a conversation we want to have with every team we build for.

A Letter to Our Stakeholders

We Are Partners in Crime, Which Turned Out to Be AI

Dear Colleagues,

When we agreed to explore an AI solution together, we didn’t just start a project. We entered a kind of partnership - not the ceremonial kind, but the kind that unfolds through real work, shared problem-solving, and the occasional chaos that comes with doing something new inside a big company. It turns out, our “crime” is AI. But the story we’re writing together is really about something more human: solving meaningful problems, with all the constraints and complexity that come with working in the real world.

So before we go any further, we’d like to be clear with you - not just about what to expect from us, but also what we’ll need from you. Because AI product development isn’t a service. It’s a collaboration. And the success of what we’re building depends on how well we stay aligned.

What You Can Expect From Us

• A structured approach. We won’t throw tech at your problem. We’ll take it seriously, understand it deeply, and explore whether AI is the right fit before we commit to building.

• Clear steps. From Quick Assessment to Detailed Assessment, and later into development, we’ll guide the process and make sure you always know where we are and what’s coming next.

• A real product mindset. If we agree to build something, it won’t be a one-off experiment. We aim to build something usable, valuable, and maintainable.

• Challenge and honesty. If your expectations are too high, or the timeline unrealistic, or if we believe the solution you have in mind won’t work, we’ll tell you. Respectfully, but clearly. We owe you that.

• A team that listens. We won’t disappear into our backlog. We’ll involve you in the right moments, bring you into design and testing, and keep your voice part of the journey.

What We’ll Expect From You

• Active participation. The discovery phase needs your time, your business logic, your data, and your real use cases. We can’t invent those on our own.

• Commitment beyond the idea. Saying yes to an idea is just the start. We’ll need you in refinement, decision-making, and especially during the early product iterations.

• Openness to the unknown. Not everything will go as planned. We’ll need space to experiment, sometimes fail fast, and adapt. That’s part of the process, not a sign of failure.

• Shared prioritization. Once live, your team might come with new ideas, changes, or user groups. That’s good. But each one needs to be assessed, prioritized, and resourced. We’ll need your help to make smart decisions.

• Executive sponsorship. If you’re not the decision-maker, we’ll need your leadership to help align with those who are. This kind of product journey works best when it’s visible and supported from the top.

This letter isn’t a contract, and it’s not a checklist either. It’s an invitation to approach this like a joint venture.

And if we do this right, the result won’t just be another internal tool. It will be a real capability that solves something meaningful in your world, and keeps delivering long after we’ve shipped the first version.

Let’s stay close. Let’s be honest with each other. Let’s keep learning as we go. And if we hit some bumps along the way, good. That’s how we know we’re doing something that’s worth it.

Product Maturity: What It Really Means

One of the most important things we’ve learned in our AI product work is this: just because an AI product ambition is approved, it doesn’t mean we don’t need you anymore. And just because something is live doesn’t mean it’s done. If we don’t set the right expectations around what it means to grow an AI product - not just build one - we end up with solutions that look complete on the surface, but break down when used in practice, or stall the moment new requests emerge.

Product maturity doesn’t mean feature-rich. It means: the solution is proven, adopted, stable, and trusted by the business and users. It means we’ve not only built the solution but also put the processes in place to operate it, evolve it, and support it responsibly.

That kind of maturity only comes with time, usage, learning, and adjustment. Not just from the AI team, but from everyone involved.

Why New Requests Aren’t Free

It’s completely natural that once something useful is live, more ideas emerge. And in most cases, we welcome those. But it’s important to be clear: every new request is a new mini-product. Even if it seems like “just another button” or “can we add one more user group,” each of those changes needs discovery, prioritization, testing, and enablement. And often, they carry downstream implications for governance, user support, and compliance.

We apply the same mindset to new requests that we used at the beginning:

• What’s the problem we’re solving?

• What user behavior do we expect to change?

• What is the estimated value?

• What is the cost of ownership?

If we don’t ask those again, we risk building features that slow us down more than they help. Or worse - that break something we’ve already made work.

The Total Cost of Ownership (TCO)

There’s one more thing we’d like to be honest about. Building an AI product can be relatively fast. Owning one is where the real cost shows up.

Many internal AI projects seem cheap in the beginning. A couple of developers, a use case, maybe a pre-trained model. But the true costs arrive later:

• Integrating it into real workflows

• Handling errors and exceptions

• Training and re-training users

• Monitoring for AI quality

• Meeting security and compliance standards

• Supporting expansion to more teams

That’s why we assess so much before building. We’re not trying to block innovation. We’re trying to protect our focus and prioritize what’s worth owning.

Roles Across Phases: Who’s Involved and How

To make this journey work, we can’t walk alone. Here’s a rough sketch of how collaboration shifts across phases - and what’s typically needed from you as our business partners and domain experts along the way:

1. Problem Discovery

• Our role: Facilitate the discovery process, frame the context, and start mapping how value could be created.

• Your role: Clarify the problem, walk us through the current process, and help us understand where the real pain points are.

Your time investment: Around 3–4 hours spread over one week.

2. Quick Assessment

• Our role: Evaluate the technical feasibility and early fit for AI, based on data, problem framing, and business priority.

• Your role: Share your expectations, validate that the problem is worth solving, and connect us to the right domain or technical experts.

Your time investment: 2–3 hours, focused and compact.

3. Detailed Assessment

• Our role: Architect the solution approach, run early prototypes, and estimate effort, value, and risks.

• Your role: Provide access to relevant data, co-shape the use case logic, and help us define what success looks like.

Your time investment: 6–10 hours, typically spread across two weeks.

4. 0→1 AI Product Development

• Our role: Build the product, test and fix iteratively, and refine based on feedback.

• Your role: Be part of ongoing validation, join feedback rounds, test early versions, and help with first-user onboarding.

Your time investment: Around 2 hours per week, usually for 6–8 weeks.

5. Go-Live and After

• Our role: Ensure the product is stable, monitor its performance, and maintain functionality.

• Your role: Support adoption by onboarding users, helping interpret early feedback, and guiding change in your team or department.

Your time investment: About 1 hour per week, depending on the scale of rollout.

6. New Requests

• Our role: Assess technical feasibility, estimate effort, and evaluate alignment with the product roadmap and total cost of ownership.

• Your role: Support in reframing the need, explain the user or business problem, help validate the potential impact, and be ready to prioritize.

Your time investment: Case by case - usually at least 2–3 hours per new request.

An Invitation

AI product development isn’t just about building smart solutions. It’s about building smart relationships - with each other, with the problem, and with the organization we’re trying to serve. That takes effort. It takes structure. And above all, it takes clarity. We’ve seen what happens when clarity is missing. And we’ve also seen the power of a shared journey, when roles are defined, expectations are aligned, and everyone involved understands what it really takes to bring something new into the world and make it stick.

If you’re ready for that kind of partnership, not just for the launch, but for the lifecycle, then we’re ready to build with you.

JBK 🕊️

In Part 1, I made a claim:

Not because it replaces business metrics, but because it is the only signal available early enough to matter. It tells you whether what you’ve built is being used. Because without usage, every other success metric stays out of reach.

This belief didn’t arrive all at once. It came from years of building internal AI products that technically worked, but quietly failed. And still, they were celebrated.

That never felt right.

How can we, as a team, sometimes as a department, celebrate something without knowing if it created real value for the people who were supposed to use it?

Was I wrong? Or is this simply how big companies define success?

Back when I owned a startup and worked in others, we never celebrated a launch just for launching. We celebrated because now we could finally test whether we had built something real. Something people wanted. Something people would pay for. A launch was never the end. It was the moment the real work began:

• finding product-market fit,

• solving real user problems,

• and learning if our solution deserves to grow.

But when I entered the world of large enterprises, I noticed something different.

Delivery became the final goal.

“Brilliant, team. Let’s move on to the next project.”

And honestly, I became allergic to that mindset.

Because these weren’t real products. They were internal service deliveries. We built what someone up the chain had asked for, without taking responsibility for whether it actually worked. Without knowing whether it ever made a difference.

For the last ten years, I’ve been on a mission to shift that.

To establish more solid product thinking in internal AI work. To treat these solutions not as technical outputs, but as real products. Not as projects, but as investments in value creation.

And that shift always brings me back to the same question:

How can we truly know that what we’ve built is valuable?

How can we show, without guesswork or slide decks, that something meaningful is happening?

For me, the answer begins with user adoption.

Because in internal AI, we don’t have all the signals that external products offer. There’s no revenue line. No churn metric. No market traction to point to. We can’t always follow the money, but we can follow the behavior.

And over the years, I’ve learned this:
Adoption is the only KPI worth tracking first. It’s the beginning of everything else.

So in this second part of the series, let’s go deeper. Let’s unpack what adoption really means. Let’s look at how to measure it and let’s explore how it becomes your most practical tool, not just to prove value, but to improve your product, align your stakeholders, and earn the credibility to keep building.

Because if adoption is the signal that rules them all, we owe it to ourselves to treat it like one.

What Adoption Means for Internal AI Products

Adoption is a word we all use. But most of us mean different things when we say it. Sometimes we mean access. Sometimes usage. Sometimes trust. Sometimes, just awareness. In some dashboards, adoption is binary: “Is it being used or not?” In some project meetings, it’s reduced to a launch checklist: “Did we roll it out to users?” And sometimes, worst of all, it’s assumed to be automatic. As if deployment equals adoption.

It doesn’t.

In internal AI product work, adoption is rarely immediate. And it’s never guaranteed. Not even when the model is strong. Not even when the integration is clean. Not even when the business case is sound. Because the truth is this:

It lives outside the product team. It lives in the people you’re trying to help. In how they choose to work, what they’re trying to get done, and what alternatives they already trust. That’s why we need to be much more precise about what we mean by it.

To me, internal AI product adoption means this:

People are using your product voluntarily and consistently, for the purpose it was designed to support — and sometimes even beyond that.

Let’s unpack that. And begin with the most important one.

Voluntarily means it wasn’t forced. People chose it because it made sense. Because it helped. Because the alternative was worse, or simply more work.

In an enterprise setup, it’s common that an internal tool is introduced and employees are required to use it. Think CRM systems, where there is no alternative. Usage is enforced. Compliance is easy to measure. Adoption, in this case, becomes inevitable.

But for AI products, this rarely holds true.

By nature, AI products are designed to improve workflows, not replace them. They assist, guide, recommend, or automate. But rarely do they become the only way to get a task done. The outcome still matters more than the method. As long as the deliverable arrives, no one questions how it was achieved.

That’s what makes internal AI Product Adoption harder to earn.

Consistently means the behavior is stable. Not a spike. Not a one-time test. Not a pilot use case that disappears once the sponsor moves on.

Real adoption creates patterns. Patterns you can observe, learn from, and improve. And consistent usage is what matters most. Because at the end of the day, that is how you prove your product is trusted to do the job it was built for. Not once. But every time. It becomes the default. Not just an option.

So once you’ve built a product and taken real steps to drive adoption, reaching that point tells you something powerful. It means you’ve achieved alignment. Between what you built, what users actually need, and how they truly behave.

That’s why a technically “successful” product can still fail. Because

It’s not a rollout. It’s a fit. And guess what, we have it, too. The famous product-market fit. Only here, it has a different name. Product-organization fit.

The point where your AI product doesn’t just exist inside the company. It belongs.

And fit always emerges through comparison. Users try your product. They compare it, consciously or not, to whatever they were doing before.

Manual steps. Excel hacks. Asking colleagues. Ignoring the problem.

If your product helps more than it hurts, if it feels smoother, faster, safer, or more reliable, they come back. If it doesn’t, they don’t. Adoption happens when the alternative is worse.

And once that’s true, once you’ve reached the invisible tipping point where the user stops asking, “Should I use this?” and simply starts using it, that’s when the product becomes real.

Not because you said so. Because they did.

How to Track AI Adoption in a Company

While adoption rate is the KPI we care about most, it doesn’t appear on its own. It has to be constructed. Built from the bottom up. It’s a composite, derived from smaller, observable metrics that together signal whether adoption is really happening.

It only makes sense when you know who the product is for and whether the people you had in mind are actually using it. In other words: before you can calculate adoption, you need two things.

(1) A clear picture of your initial user base.
(2) A way to measure voluntary and consistent usage across that base.

Without these two, you can’t derive a meaningful adoption rate.

01 The Initial User Base

Know the People You Choose First

Adoption rate already tells you that you need a denominator.
And here’s where most teams accidentally sabotage their own success.

In internal AI product development, you almost always start with a specific user group. Either they approached you and asked for AI help, or you pitched an idea to them and got them onboard. This team becomes your initial user base. It might be sales, ops, or marketing from a particular department. But it could just as easily be a niche team from anywhere in the organization.

Your solution is meant to support their daily work. That’s the scope. And that’s the user base you should measure adoption against, at first.

It’s not only important to define who your initial users are, but also how many there are.

For example the company may have 100 salespeople in total.
But if your product is designed for, and piloted with, just 20 in the B2B segment, stick with 20 for now. Don’t get distracted by the bigger number.

The reason is twofold:

1. You can’t assume fit across the board.
   Just because a product works for 20 users doesn’t mean it fits the workflows, needs, or systems of the other 80. Different regions, different setups, different behaviors. It’s a false assumption.

2. You’ll dilute your metric.
   If you count all 100 salespeople in your denominator while only 20 were ever involved, your adoption rate looks artificially low and you’ll find yourself explaining numbers that don’t reflect reality.

Let’s make this concrete.

You build an AI tool for the 20 people in your B2B sales team. After rollout, 18 of them use it consistently. That’s a 90% adoption rate. Strong signal. Great outcome.

But a few weeks later, someone tells you there’s another sales team with 80 people. You add them to your target base, even though they haven’t seen the tool yet. Now, your adoption rate drops to 18%.

Technically correct.
But totally misleading.

That’s why you need to define and track user groups separately. Only expand your base when it makes sense to. Otherwise, you confuse the narrative and weaken the story your metrics are telling.

02 Voluntary Usage

When People Choose Your Product

Voluntary means people are choosing your product without being forced to.

And in internal AI scenarios, that’s almost always the case. Unlike CRM systems or time-tracking tools, no one mandates the use of your AI assistant. People already have their own workarounds. Their spreadsheets. Their templates. Their muscle memory.

So if they do use your product, it’s a signal. It means:

✅ Your product makes their job easier.
✅ Your product is a better alternative.
✅ They’ve chosen it over what they had before.

That’s worth tracking.

Here’s one of the strongest indicators:
New users over time, beyond your original scope.

As described above, most internal AI products begin with a tightly scoped user group, often a single team who co-develop or pilot the idea. That’s your baseline.

But if the product has real value, you’ll begin to see interest from outside that circle.
Someone asks if they can try it. Someone else shows up in the logs. Suddenly, new names appear, without formal rollout.

That’s not just usage. That’s organic proof of value.

So here are two simple, telling metrics to track:

• Unique users / usages per month

• Unique users / usages per department per month

You don’t need perfection here. But if those numbers grow, especially from teams not part of your original user group, it’s a sign that voluntary adoption is unfolding.

And that matters.

03 Consistent Usage

When Your Product Becomes a Habit

Now let’s move to the deeper layer. If someone uses your product once, it hasn’t been adopted. Even if the demo was a hit and the pilot went well. Real adoption happens when your product becomes part of how work gets done.

That means behavior that’s repeatable. Predictable. Consistent.

But consistency is always contextual. Some products support daily tasks. Others support monthly processes. One AI product might be used 40 times a week. Another, once a quarter. Both can be adopted, if they match the rhythm of the task they support.

So before tracking anything, ask yourself:

• What’s the expected usage frequency for this product?

• Does the current behavior align with that expectation?

• Are users coming back or was it a one-time test?

Once you’ve clarified that, these are the metrics worth watching:

• Usage frequency per user

• Repeat usage across relevant timeframes (weekly, monthly, quarterly)

These show whether your product is truly becoming embedded in how people work or if it’s just another tool they tried once and left behind.

These basic signals are already enough to give you a sense of adoption rate. If people are coming back, using it again and again, you’ve done something right.

04 Calculating Adoption Rate

Let’s try to bring this all together with a working example. We’ll use what we’ve learned to create a first draft of an adoption rate formula, based on an potentially real internal AI product: The Tender Assistant. Let’s see what adoption really looks like, when it’s measured with care.

This product is designed to support teams working on public tenders, automating data collection, identifying inconsistencies in documents, and helping prepare responses more efficiently. It wasn’t meant to replace a mandated process and not to be rolled out through a mandatory toolchain. It was just an enhancement, a GenAI use case meant to improve a high-effort workflow.

So, how do you track adoption here?

Step 1: Define the Initial User Base

We start with one department: the Strategic B2B Sales team, responsible for responding to complex tenders in a specific region.

There were 12 active users involved in the process, bid managers, solution architects, and legal reviewers. That became our first user group.

We won’t include other sales regions yet and not count adjacent teams. We focus on the 12. This gives us a clear denominator for early adoption tracking.

Step 2: Track Voluntary Usage

As mentioned before, no one is forced to use an internal AI product. That’s the nature of this work. We introduce the product. We support the onboarding. We train the core users. And then, we wait. But we don’t disappear.

We stay available and close. We answer questions, fix what’s broken, and listen to what’s missing. Because adoption doesn’t happen in a moment. It unfolds over time.

In the first few weeks, we start to see the signals.

• 11 out of 12 users accessed the Tender Assistant at least once.

• 9 out of 12 returned to it more than once.

• 6 out of 12 used it across at least three separate tender processes.

That already tells us something: people are choosing the product, not just trying it, but using it when it counts.

But then came the stronger sign:
In the second month, the Large Deals unit - a completely separate team - reached out and asked for access. They had heard about the tool and wanted to see if it could help them, too. We hadn’t pitched it. They had asked for it. That’s not just growth. That’s pull.

That’s adoption trying to spread.
Still, we treat them differently and carefully.

We don’t add them to the original adoption rate formula just yet. We start a new onboarding journey, map their specific workflows, and learn what overlaps and what doesn’t. Some of the features work. Some don’t. Their usage is still low, but it’s early and more importantly, we’re tracking them in a separate bucket.

Step 3: Track Consistent Usage

We look at usage frequency aligned to the actual rhythm of tenders. This team responded to 2–3 tenders per month, typically with a 1-2 week turnaround. So we expected the tool to be used at least once per tender.

What we saw:

• 7 out of 12 users used the tool during all tenders in the second month.

• The average usage per user per month rose from 1.3 to 2.7.

• Usage logs showed task-specific prompts being reused - a good sign of habit formation.

This confirmed that it wasn’t just being tested. It was becoming part of the process.

Step 4: Build the Adoption Rate Formula

We kept it simple to start:

Adoption Rate = (Number of consistent, voluntary users in initial group) / (Total number of users in initial group)

In this case:

• 7 users showed consistent usage (aligned with tender frequency)

• 12 users in initial group

→ Adoption Rate = 58%

But that number doesn’t stand alone. We tracked growth as well.

After onboarding the Large Deals team (10 new users), we tracked their behavior separately for a month before merging them into the broader adoption metric. This kept our insights clean and our stories honest.

So far, we’ve talked about how to measure adoption rate. And in this case, we focused on signals from a GenAI use case - the Tender Assistant.

These metrics are foundational. But depending on the nature of your AI product, they might not be enough. Not every AI solution looks the same. Some are workflow assistants. Others are quiet automation. Some surface insights. Others shape decisions.

And depending on how your product is used - and by whom - different signals will matter. What counts as "real usage" for one AI product might be irrelevant for another.

That’s why we need to expand our view.

And yes - even though I said this would just be a two-part series - there’s more to unpack.

In the next article(s), we’ll explore:

• Different AI Products, Different Signals

• How to Use Adoption to Drive Product Decisions

• Using Adoption in Stakeholder Conversations

• Adoption and Long-Term Funding

Hope to see you then.

JBK 🕊️

If you build internal AI products, you’ve probably felt it: the disconnect between all the work you’re doing and the recognition you’re not getting. The pitch goes well. The prototype works. The model scores look great. But months later, you’re still struggling to answer one quiet, persistent question:

“Did it really change anything?”

This series is for those of us who carry that question around. Not just as a reporting requirement, but as a personal weight. Because building AI is no longer the hard part.

Proving it mattered is where most teams falter. And what makes it harder is that the outside world doesn’t see the difference. External AI teams get clean metrics: revenue uplift, conversion rates, churn rate. But inside the enterprise, success hides behind foggy processes, political handoffs, and silence. Sometimes your solution is used, but invisible. Sometimes it’s unused, but still alive in infrastructure. And sometimes it’s brilliant on paper, but completely irrelevant to how work actually happens.

In this series, I’ll explore why internal teams need a different approach to measuring value. Because in these enterprise environments, what looks like success can quickly become a mirage.

And unless we learn to measure what matters — early, honestly, and with the business in mind — AI will continue to be a story of missed potential.

Let’s change that.

The Value Mirage of Internal AI

Why success is harder to prove than most teams expect

Internal AI teams are growing. Their ambition is high. Their technical capabilities are maturing. Models are getting stronger. Prototypes come together faster. And the tooling, thanks to foundation models and better infrastructure, is finally catching up to the promise of enterprise AI.

But despite all this visible progress, one question still lingers at the core of almost every internal AI product conversation:

“How do we know this actually delivered business value?”

And often, there’s no clear answer. Not because people aren’t trying, but because in the world of internal AI, proving value is uniquely difficult. Much harder than it seems from the outside. And often far harder than expected by stakeholders who equate technical performance with business success.

It’s not that these products fail outright. In fact, that’s what makes this so tricky. They work. They run. They score. They process. They sometimes even deploy. But something’s missing. Something that’s felt in the silence that follows a well-rehearsed demo, or in the polite nods after a slide titled “accuracy > 90%.”

That silence is the absence of connection between what was built and what the business feels. It’s the moment when your audience is no longer listening to how it works. They’re wondering why it matters.

At the beginning of any internal AI journey, the terrain is foggy. We usually start with a hunch, a pain point, or a well-meant ambition like “let’s automate this process”. But when it comes to identifying the right KPI — the one that will later prove the solution’s worth — there’s rarely a straight answer.

And to be honest, that’s normal.

You don’t always know upfront what to measure. You don’t always know how value will show up. Maybe it’s cost avoidance. Maybe it’s faster cycle time. Maybe it’s fewer escalations, better targeting, or less manual rework. But these outcomes often sit far downstream from the product itself. They depend on behavior change, process integration, and adoption across multiple teams. And that makes them slow, indirect, and politically fragmented.

So the question of “what should we measure?” isn’t just hard. It can feel paralyzing.

And yet, most of us are still asked to provide a business case before we build. To define impact. To attach a number. We try our best. We make projections. We pick a few KPIs that might signal value later. But the truth is: we don’t know yet. We can’t know yet.

This is what makes internal AI different. In external products, success leaves a trail. Users pay, or they churn. Growth can be tracked. Revenue is visible. There are signals that tell you whether your product has found a market or not.

But inside the enterprise that trail disappears.

There is no revenue line for your AI product. No invoice to point to. No CAC or LTV. Just internal teams who may or may not use what you’ve built, and may or may not tell you when they stop. Your AI might be deployed, but bypassed. Integrated, but distrusted. Impressive, but irrelevant.

You rarely know the moment your product fails. Because internal failure is quiet. It doesn’t come with angry customers or public reviews. It comes with workarounds. With shadow spreadsheets. With tools that are “live” in infrastructure but dead in behavior. It comes with silence.

And in that silence, a deeper risk starts to grow: credibility loss.

Because the business remembers the investment. They remember the pitch. They remember the promise. But if no one can confidently say what changed, the narrative starts to shift. “AI” becomes something that consumes resources, not something that creates value. Sponsors lose interest. Teams lose momentum. And the next idea becomes harder to fund. Not because it’s worse, but because trust has eroded.

This is the real danger. Not technical failure, but the slow erosion of confidence in internal AI as a whole.

We’ve all seen it. Maybe we’ve even built it. That internal AI product ambition that did everything right on the surface — accurate predictions, beautiful dashboards, flawless deployment — but ultimately delivered nothing of value. Not because the product was broken. But because no one used it. Or because the way it was used never translated into measurable change. Or because the people it was supposed to help never changed how they worked.

These aren’t edge cases. They’re common. And they’re exhausting. Because the effort is real. The intentions are good. But the outcome? The outcome vanishes. And it leaves behind a question that haunts many internal AI teams more than they’d like to admit:

“Did we build something valuable or just something impressive?”

But i think we can overcome that struggle with just one KPI. At least at the beginning. I’d like to describe my thoughts with a Tolkien’s Lord of the Rings analogy:

Let me explain why.

One KPI to Rule Them All

The insight that reshaped how I measure internal AI success

For a long time, I searched for the one perfect metric. The one that would finally tie our internal AI products to real business value.

But the difficulty was this: internal AI products touch multiple business processes and workflows. Some of those generate revenue. Others protect it. Some AI products are meant to avoid costs by preventing additional headcount. Others reduce costs by cutting down on external spend, like consultant fees. And some act as enablers - helping the business grow revenue, indirectly. In some cases, all three apply.

So I tried cost savings metrics. I tried process efficiency metrics. I tried risk reduction and effort elimination. I even tried composite scorecards with weighted proxies.

And sometimes, those metrics helped tell the story.

But only on slides, in steering committees, or in follow-up emails.

Too often, they were fragile signals. Too slow to emerge. Too easy to challenge. Too detached from real behavior.

That’s when I realized something both practical and deeply grounding:

Used voluntarily. Used consistently. Used by the right people, in the right moments. Because in internal AI work, nothing else matters without adoption.

That was the shift for me. That was the moment the fog began to lift.

So when I say adoption is “One KPI to rule them all” what I mean is this:

Adoption is the only early signal that has the power to connect — or rule over — every other KPI you’ll eventually need.

• Model performance metrics like precision, recall, and latency

• Business metrics like time saved or compliance risk reduced

• Outcome metrics like reduced handling time, increased conversion, or fewer escalations

They — the KPIs you’ll later be asked to report on — are all governed by adoption.

If no one uses your AI product, none of those metrics matter. They become disconnected ideas in your head, or worse, misleading decorations on a dashboard.

So yes, them refers to the entire family of downstream KPIs.

And adoption is what makes them possible. It gives them form. It gives them context.

It’s the keystone.

But adoption isn’t just a positive signal. It’s not just the green light that tells you your product is alive. Adoption is also your first diagnostic tool.

It’s the earliest and most reliable sign that something might be wrong. If adoption drops. If it never starts. If it spikes in one team but not another.

You don’t need to speculate. You investigate.

And that investigation leads to real insights:

• Does the user journey feel intuitive?

• Are people skipping steps or overriding AI decisions?

• Is the integration too shallow, or too disruptive?

• Are teams using it in unintended ways that reveal new value?

• Or is the AI not accurate enough?

In this way, adoption becomes both proof of momentum and a system of early warnings. It helps you see what’s working. And it helps you find what isn’t — before failure becomes political or irreversible.

I’ve seen technically strong AI products die quietly because nobody changed how they worked. And I’ve seen modest solutions take off because adoption came early and the team had the humility to listen, iterate, and respond.

That’s when I began to treat adoption rate not as an afterthought, but as the one KPI that governs all the others. It’s the only one that’s visible from the beginning.

And it’s the only one that unlocks the rest.

No, adoption won’t tell you everything. It won’t quantify ROI or certify value to the CFO. But it will do something even more essential. It will show you if what you’ve built is real enough to be used — and alive enough to be improved.

That’s why, in internal AI product work, adoption is not just a signal. It’s the signal that rules them all.

One KPI to Find Them

Why adoption reveals the business value you were looking for all along

When we begin building an internal AI product, we always start with purpose. We don’t build blindly. We listen to the pain points. We work with users. We try to understand the operational logic behind the problem. And at the same time, we aim to connect that user problem to something bigger — to business outcomes.

• Will this solution help us reduce cost?

• Will it eliminate waste or manual rework?

• Could it avoid future risks?

• Or will it enable revenue, directly or indirectly, by improving decisions, speed, or scale?

We don’t ignore these questions. In fact, we often write them into the product brief or the business case.

But let’s be honest — defining the exact metrics and logic to measure them, especially upfront, is hard. Sometimes frustratingly so.

Because even when we’re clear on what the AI solution should help achieve, we’re rarely clear on how that impact will show up in the data.

And even if we define a KPI with a stakeholder early on, it often turns out to be:

• Too far downstream

• Owned by another team

• Mixed with dozens of other influencing factors

• Or worse, tracked in a report that nobody actually trusts

So we make our best guess. We write down the metrics we think will prove value later.

But more often than we’d like to admit, those guesses don’t hold. And we realize, three months in, that the KPI we picked either can’t be measured cleanly or doesn’t reflect the real outcome we’re driving.

This is where adoption changes everything.

Because once people start using your AI product — really using it, in live processes, under real conditions — suddenly the fog begins to lift.

You see how the product is being used. You learn which teams engage with it naturally and which ones don’t. You observe where trust builds and where friction still exists.

And most importantly, you start to see which outcomes are actually being influenced — and how.

The business KPIs you couldn’t quite measure before, now they start surfacing. Still not through your own analysis, but through the people using the product every day.

• A sales team might say, “We’re closing leads faster now.”

• A customer service leader might report, “Escalation rates have dropped.”

• A compliance officer might notice, “We’re catching issues earlier.”

These signals don’t come from the AI team. They come from the business. And that makes them powerful.

So when I say “One KPI to find them” I mean this:

Adoption helps you find the true business KPIs — them — that are affected by your product.

Not in theory and projection. But through lived usage.

Because once a product is adopted:

• It becomes observable

• It enters real workflows

• It creates data you didn’t have before

• It starts conversations you couldn’t have earlier

And that’s when the right metrics begin to emerge. Not as guesses, but as patterns. Not as assumptions, but as evidence.

It’s humbling, really.

Because it reminds us that no matter how sharp our thinking, we don’t control all the value. Some of it is outside our reach.

• It’s embedded in business metrics we don’t own.

• It’s locked in processes we can’t fully observe.

• It’s shaped by stakeholder behaviors we don’t manage.

But once our product is being used those metrics start to show up.

The business begins to bring them forward. Suddenly, people don’t just ask you what the product is doing. They start telling you what it’s changing. That’s when you know your product is real.

So yes, adoption doesn’t just rule the other KPIs. It helps you find them. It gives you access to the only thing that ever reveals business value for internal AI: the lived behavior of people who trust what you’ve built.

One KPI to Bring Them All

How adoption becomes the gravitational pull that turns an AI product into a business asset

There’s a moment in the life of a successful internal AI product when you start to notice a shift. It no longer feels like you’re pushing. It no longer feels like you’re convincing people to try it, chasing usage reports, or writing follow-up messages just to keep the spark alive. Instead, things begin to pull.

• A new team reaches out.

• A stakeholder from another business unit asks if they can join the next pilot.

• Someone you’ve never met references your product in a planning meeting.

What started as a focused product now has momentum. And that momentum did not come from technical excellence alone. It came from adoption through relevance.

So when I say “One KPI to bring them all” I still mean the business KPIs we’ve been trying to measure from the start.

• Cost reduction.

• Cost avoidance.

• Direct or indirect revenue enablement.

Them — the KPIs we struggled to define at the beginning — finally start showing signs of life once the product is used.

But something else happens, too. Adoption does not just bring the metrics into motion. It brings in the people behind the metrics. The business owners, the adjacent teams, the leaders and enablers who start building on top of what you’ve created.

It becomes something people talk about. Something that touches other systems, other teams, other goals. Adoption pulls it all together. It is the gravitational force that begins to draw the organization in.

And that pull is what activates your KPIs in a way that dashboards never could.

You start seeing shifts in:

• Cycle times

• Manual workarounds

• Time-to-resolution

• Uplift in conversion or sales readiness

• Effort allocation across roles

• Forecast accuracy

• Process completion rates

But now, it is not you reporting these shifts. The business starts noticing them.

• Finance might begin modeling cost avoidance based on changes in headcount planning.

• Operations might share how throughput has increased with no additional staff.

• Risk or Legal might recognize that an early-warning AI system is now embedded in their compliance checks.

These conversations do not happen when a product is in development. They do not even happen when it is just launched. They happen when adoption is real.

Adoption brings something else that most metrics cannot.

Alignment.

When the product is used, everyone around it starts working differently. Enablement makes sense. Feedback becomes targeted. Governance becomes active, not theoretical.

Executives stop asking why you built it — and start asking what more it could do.

Your AI product stops being an initiative. It starts becoming infrastructure.

And here is the truth I’ve learned:

All the metrics in the world are meaningless until people care. And people do not care until they use it. And once they use it — when it actually helps them — they start becoming part of the story. They bring others in. They speak for the product. They make your business case stronger than any model ever could. AND they make the outcomes transparent only they have control and access to.

And in the Outcome Bind Them

How adoption transforms usage into proof, and AI products into trusted outcomes

By the time adoption is established — when people are using your AI product not out of obligation or curiosity, but because it genuinely fits how they work — something important starts to shift. Not suddenly. Not dramatically. But steadily, and unmistakably.

The product begins to create more than activity. It begins to create results.

And yet, those results — the ones we aimed for in the beginning — are rarely immediate. They do not appear in clean, self-contained dashboards. They do not arrive in tidy before-and-after comparisons. Instead, they begin to surface in the rhythm of the business. In meetings. In feedback loops. In operational metrics that slowly start to move.

This is the moment where everything we hoped to measure — those elusive business KPIs we tried to define at the start — begin to settle into form. And they do not just emerge. They become bound to the product itself.

That is what I mean when I say: “And in the outcome bind them”. Until this point, many of those KPIs felt abstract.

• Cost avoidance.

• Cycle time reduction.

• Conversion uplift.

• Revenue growth.

We mentioned them in our business case. We tried to estimate them. But we also knew — quietly — that measuring them would be hard. That they lived downstream, in systems we did not control, owned by people we were not sure would pay attention.

But once adoption takes hold, once the product is used in daily work, something subtle but powerful changes. Those same people begin to see the impact for themselves. Not because we told them to, but because it shows up in their reality.

• A team lead starts saying, “We do not have to double-check these entries anymore.”

• A controller notes, “We are spending less time on reconciliations.”

• A compliance owner quietly shares, “We have reduced our response time without adding staff.”

These are not claims. They are experiences. And in that moment, the KPIs we sought to measure begin to belong to them. That is the binding.

Because adoption alone is not the outcome. Adoption is the start of a pattern — one that allows us to observe, learn, and begin making connections we could not make before.

Now, we are no longer speculating. We are seeing relationships between usage and impact. We are finding leading indicators. We are discovering how certain behaviors, when supported by the AI product, correlate with improved business performance.

And crucially — we are no longer the only ones doing this work. The business starts participating. They bring their own data, their own stories, their own versions of the value narrative. And suddenly, we are not measuring in isolation anymore. We are measuring together. This is the moment internal AI products move from experimental to essential. Not because everything has been quantified. But because the product is no longer defended by the product team — it is spoken for by the business.

That is when outcomes become stable, when metrics become trusted, when sponsorship becomes continuous.

And it all begins because adoption has created enough usage to make impact visible. Enough trust to make collaboration possible. Enough real-world relevance to make measurement credible.

So yes, adoption rules the KPIs. It finds them. It brings them into motion. But most importantly, it binds them to outcomes — to the kinds of results that teams can feel, leaders can report, and businesses can build on.

Final Reflection

When I look back at the internal AI products that truly made a difference, it was never the technical elegance that convinced the business. Not the architecture. Not the pilot results. It was because we found ways to make the product useful. Genuinely useful. For real people, in real moments of their work.

That usefulness led to something rare: engagement.

Participation in user acceptance tests went up. Feedback became honest. And ultimately, adoption took hold.

Adoption is where it all starts. And this beginning is not easy.

But once you understand that you do not need to focus on any other metric first, it becomes much easier to allocate your precious resources to the right tasks. You stop chasing hypothetical KPIs and start building something real.

Because without adoption, all other KPIs remain out of reach. They stay theoretical and fragile. They stay disconnected from the truth of how people work.

This article was all about that one insight: Why adoption rate is the only metric that matters in the beginning.

In the second part of this series, I will go deeper into the how. How to define adoption. How to track it. And how to use it as a compass to guide product decisions, stakeholder alignment, and even long-term funding.

Hope to see you there.

JBK 🕊️

In my last article, I shared why so many AI quick wins turn into long-term burdens. Today, let’s talk about how to avoid that illusion — and how to ensure we build only what’s truly worth owning.

✅ Why Most Quick Wins Aren’t Wins at All

✅ How to Avoid the Illusion

✅ The Service Provider Perspective

✅ Closing Thoughts

Why Most “Quick Wins” Aren’t Wins at All

It’s easy to understand why the idea of AI “quick wins” has captured the imagination of so many organizations, especially in an era where every company feels the pressure to demonstrate technological innovation and where leadership eagerly seeks visible proof points to justify investments in artificial intelligence.

The notion that a small team, armed with modern APIs and a few clever prompts, can deliver something that appears intelligent, valuable, and even transformative within a matter of days or weeks is not merely appealing — it’s intoxicating, because it suggests that the barriers to impact have all but vanished.

Yet the sobering reality, which emerges time and again in the aftermath of these rapid development sprints, is that most of these so-called quick wins are not wins at all, at least not when measured against the full economic equation required to sustain an AI product in a live business environment.

For while it is true that the cost and speed of initial development have fallen dramatically, this reduction has done little to eliminate the far more stubborn costs that arise once a solution is expected to perform reliably, safely, and in alignment with business objectives over time.

Indeed, it is precisely because prototypes can be built so quickly and with such dazzling technical fluency that organizations often rush ahead, mistaking technical feasibility for product viability, and underestimating the layers of complexity that lie between a working demo and a sustainable, value-generating product.

One of the most insidious illusions of AI quick wins is that the moment a model produces correct outputs in controlled scenarios, it is ready for real-world deployment, when in truth, the journey from demo to product is where the true costs — and risks — begin to accumulate.

Consider the governance overhead alone: the legal teams who must review every possible output to ensure regulatory compliance, the security audits required to protect sensitive data, the need for explainability if the AI’s decisions affect customers or employees, and the documentation that must be created to satisfy auditors and internal stakeholders alike.

Layered atop these governance concerns is the relentless evolution of business processes themselves, because no matter how elegant an AI model may be, the realities of enterprise life dictate that business rules change, systems are upgraded, organizational priorities shift, and what was true yesterday might be obsolete tomorrow — all of which demand ongoing adjustments, retraining, and validation work that quietly consume time, budget, and human attention.

Then there is the human side of AI, which is so often overlooked in the rush to build: the time and effort required to train users, to earn their trust, to manage their expectations, and to support them when the AI inevitably produces an output that confuses, disappoints, or outright fails to meet the nuanced needs of their real-world tasks.

Even the most straightforward tools, once released into production, attract a steady stream of enhancement requests, support tickets, edge cases, and new use scenarios that were never foreseen during the initial build, each demanding attention, prioritization, and — ultimately — additional cost.

And so what begins as a deceptively affordable technical exercise becomes, over time, an ongoing drain on organizational resources, often far exceeding any value the initial prototype seemed poised to deliver.

This is the essence of why most quick wins are not wins at all: because they are evaluated only through the narrow lens of build effort, without a sober analysis of the cost of ownership and the total economic impact required to sustain them as real products.

An AI solution that costs €10,000 to build but €150,000 annually to maintain, while delivering only €50,000 of value, is not a win — it is a liability masquerading as innovation.

The true discipline of AI Product Management, therefore, lies not in celebrating how quickly something can be built, but in developing the discernment to know which solutions are worth owning, and in having the courage to say no to ideas that, while technically feasible, cannot deliver sustainable, economic value once all costs are accounted for.

Until we embed this discipline into our processes, we risk chasing the illusion of quick wins, filling our roadmaps with projects that impress in demos but quietly erode resources, distract teams, and ultimately fail to justify their existence in the harsh light of economic reality.

How to Avoid the Illusion

It’s one thing to recognize the illusion of AI quick wins; it’s quite another to develop the discipline and the practical methods required to avoid falling into that trap, especially in organizations eager to proclaim their leadership in AI innovation and where the pressure to deliver visible results can sometimes override sober assessment of long-term sustainability.

Yet avoiding this illusion is not merely a matter of caution — it is a fundamental responsibility for anyone tasked with building AI products, because the costs of getting it wrong are not confined to technical rework but extend into wasted resources, eroded trust among stakeholders, and the opportunity cost of having pursued initiatives that were never destined to generate meaningful returns.

So how, then, does one navigate the seductive pull of building quick, impressive prototypes while protecting the organization from the hidden liabilities of ownership?

Separate the Cost of Building from the Cost of Owning

The first and perhaps most crucial step is to consciously and systematically separate the cost of building from the cost of owning, treating them as two entirely distinct phases of the product’s economic life cycle.

It is no longer sufficient to ask only how many weeks it will take a developer to connect to an LLM API or to produce a functioning prototype; we must equally ask how many people will be required to maintain it, how frequently the model or prompts will need to be updated, and how many systems it must integrate with — each of which contributes directly to the total cost of ownership.

This separation forces product teams and stakeholders alike to look beyond the glamour of the initial demo and to confront the practical realities of sustaining a product once it enters the complex, ever-changing environment of real-world operations.

Ask the Right Questions Early

Avoiding the illusion begins with asking better questions — questions designed not merely to confirm technical feasibility but to expose the true shape of the ownership burden that will inevitably follow any AI solution.

Questions such as:

• How dynamic is the problem domain? Do rules, policies, or user needs change frequently?

• How many users will this serve, and how diverse are their roles and expectations?

• Will this solution need to integrate into other systems, and if so, how tightly?

• Does the solution handle personal or regulated data, triggering privacy or compliance requirements?

• How critical is this solution to business operations, and what is the risk if it fails?

• Are there existing teams prepared to own and support this product long-term?

Each of these questions serves as a signal for potential costs that may not be visible in the initial build estimate but will certainly materialize once the product is live.

Keep ROI Front and Center

Yet even as we strive to separate costs and estimate them with as much realism as possible, there remains one final, indispensable lens through which every AI initiative must ultimately be viewed: the lens of return on investment.

For no matter how elegantly we may build, or how rigorously we may estimate the costs of owning an AI product, these efforts are meaningful only insofar as they allow us to judge whether the value created by the product will, in the end, exceed the total costs required to build and sustain it.

Having a clear ROI mindset is not a constraint but a strategic compass, one that empowers product leaders and organizations to make deliberate choices about which AI ambitions are truly worth pursuing.

When we force ourselves to ask — even at the earliest stages — how much economic benefit a solution might realistically deliver, and when we set that potential benefit against both the known costs of building and the less visible but equally real costs of ownership, we transform decision-making from an exercise in technological enthusiasm into a discipline of informed, economically grounded choices.

It is through this lens of ROI that we gain the courage and clarity to prioritize not merely what we can build, but what is genuinely worth owning — and it is this discipline that will distinguish the fleeting illusions of AI quick wins from the sustainable successes that endure and create true business value.

Align Technical and Product Perspectives

One of the most significant risks in AI development is the disconnect between technical teams, who are often eager to demonstrate what is possible, and product leaders, who are responsible for ensuring that solutions translate into sustainable value.

To avoid the illusion of quick wins, these two groups must collaborate closely from the outset, jointly evaluating both the technical feasibility and the economic sustainability of any proposed initiative.

Technical teams must be transparent about the assumptions and hidden complexities in their solutions, while product leaders must challenge optimistic timelines and push for clarity around governance, integration, and support costs.

It is in this partnership — between those who build and those who own — that the best defenses against the illusion are forged.

The Service Provider Perspective: Shifting, Not Erasing, Ownership Costs

As we grapple with the challenge of distinguishing between the cost of building and the cost of owning AI products, it’s worth pausing to consider a business model that seems, at first glance, to escape the burden of ownership altogether: the model of the service provider.

There exists a substantial segment of the technology landscape composed of companies whose business is not to own products themselves, but to build solutions on behalf of others, delivering precisely what has been specified, collecting their fees, and moving on to the next project.

For these service providers, the economic calculus appears refreshingly simple: scope is defined, requirements are gathered, code is written, the solution is delivered, and payment is rendered. In this model, the cost of ownership — with all its complexities and potential liabilities — seemingly vanishes from the service provider’s concerns, because their financial and operational obligations end the moment the solution is handed over.

Yet this absence of ownership costs in the service provider’s books does not mean those costs disappear from the world. They are merely shifted — often in full — onto the shoulders of the client who commissioned the work.

For the buying company, the reality remains unchanged: every AI solution brought into production becomes part of a living ecosystem that must be integrated, supported, governed, and maintained over time. The same challenges apply, whether the code was written by internal developers or by an external partner.

Data pipelines still require monitoring and updates. Models still drift and demand retraining. Compliance audits still loom, demanding documentation and explainability. Users still generate tickets, seek enhancements, and raise concerns when outputs fail to align with their expectations. And when the inevitable changes in business processes arrive, someone must be there to adjust the solution so that it continues to deliver value rather than becoming a liability.

In many ways, the illusion of quick wins can be even more seductive in service-driven contexts, because clients may mistakenly believe that by outsourcing the build, they have also outsourced the burden of ownership.

But the truth is unavoidable: the cost of ownership can be transferred, but it cannot be eliminated.

It is a debt that must always be paid — either by the builder or by the buyer — and wise organizations recognize that even when they choose to partner with service providers, they must plan and budget not only for the cost of building, but for the far greater and longer-lasting cost of owning the solution once it comes home.

Thus, whether acting as a product company, a service provider, or a client engaging external partners, the fundamental discipline remains the same: to distinguish the immediate thrill of building from the ongoing responsibility of ownership, and to make choices that ensure the solutions we deploy are not merely technically impressive, but economically sustainable.

A Discipline, Not a Constraint

Avoiding the illusion of AI quick wins is not about stifling innovation or rejecting the incredible possibilities that modern AI unlocks; rather, it is about practicing a discipline that ensures the solutions we choose to build are not only technologically possible but economically sustainable.

It is about remembering that in the realm of AI products, the real measure of success is not how quickly something can be built, but whether it can endure, scale, and deliver value over time without consuming more resources than it is worth.

And it is about recognizing that in the end, the true quick win is not the speed of development, but the wisdom to build only what is worth owning.

Closing Thoughts

In the race to stake a claim in the future of artificial intelligence, it is understandable — and perhaps inevitable — that we find ourselves captivated by the apparent ease and speed with which AI solutions can now be built, fueled by the astonishing capabilities of modern language models and the growing arsenal of accessible developer tools that promise to transform even modest ideas into impressive technical demonstrations.

Yet amidst this newfound agility, there lies a fundamental truth that every AI product leader must carry forward: an AI solution that can be built quickly is not necessarily an AI product worth owning.

For while it is undeniably exhilarating to create something that works, that speaks fluently, that classifies data or forecasts outcomes with mathematical precision, the real test of success lies not in the initial build, but in the far less visible domain of sustained operation, governance, and economic viability.

It lies in the long, patient work of ensuring that what we deploy today remains relevant, accurate, and trustworthy tomorrow — that it continues to deliver value without quietly accumulating costs that outweigh any benefits it was meant to produce.

This is the quiet discipline at the heart of AI Product Management: the insistence on looking beyond the glittering surface of technical feasibility to examine the deeper, more demanding questions of adoption, integration, governance, and total cost of ownership.

It is the willingness to say no to ideas that cannot justify their presence in a real-world business environment, no matter how technically brilliant they may appear in a controlled demo.

It is the courage to protect an organization’s focus and resources, to ensure that the AI products we choose to build are those that can not only be delivered swiftly but sustained wisely.

Because ultimately, the role of AI product professionals is not merely to prove what is possible, but to guide organizations toward building only those solutions that stand the test of time, that create more value than they consume, and that contribute meaningfully to the strategic objectives we exist to serve.

Now, my question is:

How many real AI Quick Wins have you ever seen?

JBK 🕊️

Nowadays, every developer feels empowered to build their own AI solution.

At least a certain type of AI solution, the kind that doesn’t require training a model from scratch.

With the rise of powerful LLM providers, building AI has become remarkably accessible. You browse the provider’s docs, write a few lines of code, and within hours you’ve built something that looks like magic.

The cost of building AI has never been lower.

An AI product is an economic proposition. It’s not just code that runs, it’s a solution that solves a real problem for a specific user group, earns adoption, and generates enough value to justify its costs. Ultimately, it’s about building something that makes more money than it costs to sustain.

And this is where many builders get blindsided.

People forget that real AI products carry two types of costs:

• The cost to build them.

• And the cost to own them.

And once you understand this, you realize that the majority of AI use cases we see today would never make it past the idea stage if we honestly accounted for the true costs of ownership.

That’s what my article today is about:

The Rise of “Cheap to Build” AI

We’re living through a remarkable moment in technology history, one in which the barrier to building AI solutions has dropped so dramatically that it’s now possible for nearly any developer, regardless of their background in machine learning, to assemble something that looks intelligent in a matter of hours.

The reason for this sudden accessibility lies, of course, in the emergence of large language models and the thriving ecosystem of APIs and developer tools that surround them, making it feasible to build conversational agents, text analyzers, summarization tools, or countless other applications without ever training a single model from scratch.

Where once deploying AI required deep knowledge of algorithms, the painstaking preparation of training data, and expensive computational resources, it now often requires little more than reading API documentation, crafting a few prompts, and wiring the output into an existing user interface or backend service.

This shift has created an intoxicating sense of speed and empowerment among developers, because for the first time, the dream of embedding intelligence into digital products feels tangible, immediate, and relatively inexpensive to prototype.

Yet this very ease has become a double-edged sword, because while the act of building has become democratized and dramatically cheaper, it can foster a dangerous illusion: that the low cost and speed of initial development somehow translates into a sustainable, low-cost product over the long term.

One can build a working demo in a hackathon, showcase it internally, and impress stakeholders with the apparent sophistication of natural language understanding or smart decision-making — and in doing so, create the impression that the AI problem is “solved” simply because the technical proof of concept runs without errors.

However, the reality that lurks beneath the surface is that the cost of writing code and hooking into an AI API is often the smallest fraction of what it takes to turn an AI solution into a real product — a product that not only functions reliably but integrates into workflows, complies with governance requirements, adapts to shifting business needs, and continues to deliver economic value year after year.

The temptation to believe in AI “quick wins” stems from this new reality: that building a prototype has become so deceptively easy, it masks the far greater costs and complexities involved in truly owning and operating an AI product over time.

And unless we consciously separate the cost of building from the cost of owning, we risk filling our backlogs and our organizations with solutions that look brilliant on the surface but quietly drain resources, erode trust, and fail to deliver a sustainable return on investment.

Why an AI Solution ≠ an AI Product

It’s a subtle but critical distinction — one that often gets overlooked in the current rush to showcase technological capability — that an AI solution, impressive though it may be from a purely technical standpoint, is not by default an AI product, because a product is defined not merely by its existence but by its ability to consistently solve a problem for real users, in a way that is sustainable, adopted, and ultimately economically viable.

Related Article

While an AI solution might be a clever script, a working demo, or a functional prototype that answers questions, generates text, or classifies data with uncanny accuracy, it remains fundamentally an internal artefact until it crosses the far more challenging threshold of delivering value to a defined group of users who choose, repeatedly and willingly, to integrate it into their daily tasks.

An AI product, in contrast, exists within an ecosystem of human expectations, business objectives, and operational realities; it must earn trust, fit seamlessly into workflows, comply with often stringent governance requirements, and deliver an experience robust enough that users are not only willing to try it once, but to depend on it over time, perhaps even to the point of paying for it — either directly or through its contribution to broader business performance.

This transition from “solution” to “product” represents the true crucible of AI product management, because it is here that the technical marvel of AI collides with the stubborn complexities of human behavior, regulatory constraints, and the shifting sands of organizational priorities.

Too often, teams celebrate the technical feasibility of an AI initiative as though that alone were sufficient proof of its value, pointing to a working chatbot, an elegant classification model, or an automated report as evidence that the problem has been solved — when in reality, these artifacts are little more than prototypes until they can prove that users actually want to adopt them, that they can survive in production environments, and that the economics of ongoing operation make sense when weighed against the benefits they bring.

I have seen, time and again, solutions that worked beautifully in a controlled testing environment but fell apart in the real world, not because the underlying AI was flawed, but because the product as a whole lacked the infrastructure, the support mechanisms, and the organizational alignment necessary to transform a clever idea into a sustainable asset.

Consider, for instance, a chatbot designed to answer policy questions within a large enterprise. While the technical implementation might be straightforward — wiring an LLM API to a document database, perhaps — the true challenge arises when policies change, when users begin asking nuanced or politically sensitive questions, or when legal and compliance teams intervene to scrutinize every possible hallucination or misinterpretation that the model might produce.

Or imagine a forecasting model built to predict customer churn, which dazzles stakeholders with its precision during a pilot phase, only to collapse under the weight of integrating into live systems, dealing with data refreshes, and explaining predictions in terms that business users can trust and act upon.

The difference between an AI solution and an AI product, therefore, is not just a matter of technical sophistication, but a question of economic sustainability and operational maturity — the capacity to deliver value continuously, safely, and in a manner that justifies both the initial investment and the ongoing cost of ownership.

This is why, in the discipline of AI Product Management, we must always look beyond the seductive glow of working demos, and insist on asking the harder questions:

• Who will use this?

• Will they truly adopt it?

• How often will it need to change?

• What governance or compliance hurdles must it clear?

• And above all — will it generate more value than it costs to build and maintain?

Because in the end, it is not the elegance of our code, nor the cleverness of our models, that defines success in AI, but our ability to build products that persist, scale, and deliver a return on the resources invested in them — products that serve real needs, in the real world.

Related article

The Two Costs of Real AI Products

When we speak of AI products, it’s tempting to focus almost entirely on the exhilarating act of building — on the prototypes, the architecture diagrams, the proof-of-concepts that light up demo days and reassure stakeholders that progress is being made.

And yet, the true story of an AI product is told not merely in the cost and time it takes to build it, but in the often invisible, relentless costs that follow long after the first lines of code are written, costs that determine whether the product becomes a sustainable asset or an expensive curiosity that fades from memory once the initial excitement has worn off.

The Cost of Building

The cost of building encompasses all the one-time efforts that go into transforming an idea into a functioning prototype or a first release.

It includes the hours spent by developers exploring data sources, selecting algorithms, testing prompts, designing user interfaces, and navigating the labyrinth of system integrations necessary to embed AI into existing business processes.

It covers the technical work of setting up pipelines, the collaboration sessions between product managers and data scientists to frame the problem correctly, and the sometimes intense periods of iteration required to move from a promising proof-of-concept to a solution robust enough to be demoed to stakeholders.

In many organizations today, thanks to the availability of powerful pre-trained models and API-driven services, this initial cost of building has plummeted, allowing teams to produce impressive prototypes at a fraction of what it would have cost only a few years ago.

And this, paradoxically, is precisely where the illusion of quick wins begins — because it creates the impression that the bulk of the work is done once a model produces credible outputs or an LLM responds with human-like fluency.

The truth, however, is that while building has become easier, it remains only the first, often smallest, part of the journey.

The Cost of Owning

It is in the cost of owning where the real weight of AI product development reveals itself — the weight that so often remains hidden during the euphoric days of building, only to emerge as an increasingly heavy burden in the months and years that follow.

Owning an AI product means maintaining not just the technical components — the models, the code, the integrations — but the entire ecosystem required to keep the product relevant, accurate, and safe in the face of continual change.

It means monitoring model performance to detect drift, updating prompts as business rules evolve, retraining models when new data becomes available, and ensuring that the AI’s outputs remain consistent with shifting regulatory requirements and legal standards.

It involves integrating the AI into production systems in a way that remains resilient even as upstream or downstream systems change, and preparing for the reality that what works perfectly in a lab environment may encounter unexpected edge cases or operational challenges in real-world conditions.

The cost of owning also includes the human side of AI: supporting users as they learn to trust and adopt new systems, providing documentation and training materials, handling requests for enhancements or bug fixes, and dealing with the inevitable questions and complaints that arise when AI makes errors or delivers results that users don’t fully understand.

Moreover, ownership carries with it the burden of governance — the processes of security reviews, legal assessments, and risk management, all of which are non-negotiable in enterprise environments, particularly when AI is involved in decisions that might affect customers, employees, or regulated business activities.

These costs of ownership are neither optional nor trivial. They are the ongoing price we pay for transforming clever prototypes into real products — products that not only work once but keep working, safely and reliably, over time.

This is why the notion of AI quick wins can be so dangerously seductive: because it blinds us to the reality that while the cost of building has indeed fallen, the cost of ownership has remained stubbornly high, and in many cases, has even increased as AI systems become more complex, regulated, and deeply integrated into the heart of business operations.

Until we account for both sides of the ledger — the cost of building and the cost of owning — we cannot truly judge whether an AI initiative is a quick win or a long-term liability in disguise.

Examples of the Ownership Trap

To truly appreciate why so many AI solutions, though cheap to build, become expensive to own, we need only look at the real-world examples that emerge time and again in enterprises attempting to harness the promise of artificial intelligence.

These are not failures of technology per se, for the algorithms often perform precisely as designed; rather, they are cautionary tales about what happens when the seductive ease of building blinds us to the relentless realities of ownership.

Example 1: The Chatbot That Kept Growing

Consider the seemingly innocuous decision to deploy a chatbot designed to help employees navigate internal policies, an initiative that, on paper, appeared to be a perfect quick win.

The technical work was modest: a few calls to an LLM API, some prompt engineering to ensure the bot referenced the correct documents, and a lightweight web interface for employees to submit questions.

Within weeks, the prototype was working well enough to be demoed to leadership, and its creators rightly felt a surge of pride — for here was an AI solution that could answer policy questions quickly and reduce the load on human support teams.

Yet, as soon as the chatbot went live, a different reality unfolded.

Employees, delighted by the initial utility, began asking increasingly complex questions that blended policy interpretation with subtle organizational politics — queries the model was never designed to handle and which introduced significant risks if answered incorrectly.

Meanwhile, the legal department intervened, demanding rigorous controls to ensure no confidential or outdated information was served, triggering a new wave of compliance reviews, prompt adjustments, and the need for an auditable log of every interaction.

Worse still, business units outside the original scope began requesting versions of the chatbot tailored to their own specialized policies, fragmenting the development effort and multiplying the maintenance burden.

What began as a small, low-cost experiment had now evolved into an ongoing product with legal risks, governance overhead, and a growing queue of change requests — a perfect example of how low initial build costs can mask the true cost of ownership.

Example 2: The Forecasting Model That Couldn’t Survive the Real World

Another example arises from the widespread enthusiasm for predictive modeling, particularly models designed to forecast critical business outcomes such as customer churn.

In one enterprise, a team built a sophisticated churn prediction model using historical customer data, leveraging advanced machine learning techniques that achieved impressive accuracy during testing.

The prototype dazzled stakeholders, who were eager to deploy it as a tool for proactive retention strategies.

However, as soon as the model was moved toward production, its creators discovered that the very data pipelines feeding it were prone to frequent schema changes, driven by evolving business definitions and new marketing initiatives.

Each time upstream systems changed, the model broke, requiring urgent intervention from data engineers and data scientists to re-map features and re-run validations.

Moreover, business users, once enthusiastic, began demanding clear explanations for why certain customers were flagged as high churn risks — explanations the model was ill-prepared to provide, especially under tight timelines.

What had seemed like a technical triumph quickly transformed into a fragile solution requiring constant care, communication, and firefighting — its ownership costs far exceeding the initial estimates.

Example 3: The One-Team Tool That Became Everyone’s Problem

A final example comes from a small tool built by a team to automate the categorization of customer feedback into topics for analysis.

Originally conceived as a simple internal solution, the AI model used text classification to tag feedback into a handful of business categories, helping one analytics team speed up their reporting.

Initially, the build was straightforward: the team fine-tuned an existing model, connected it to their feedback database, and produced a simple dashboard.

But success quickly brought attention.

Other departments, seeing the tool’s usefulness, requested their own categories, more languages, integration into enterprise reporting systems, and compliance reviews for customer data privacy.

Each request seemed small on its own — a new tag here, another language there — but together they transformed a low-maintenance script into an enterprise-grade product requiring dedicated ownership, funding, and continuous upgrades.

What started as a clever side project became a sprawling responsibility nobody had planned to sustain, draining resources that could have been focused on higher-value initiatives.

The Pattern Across All Examples

In each of these stories, the initial build was fast, inexpensive, and technologically feasible.

But the unseen costs — governance, integration complexities, user support, evolving requirements, and compliance obligations — turned these “quick wins” into enduring commitments, often without delivering proportional economic value.

This is the essence of the ownership trap: the seductive belief that because we can build AI solutions quickly and cheaply, they will naturally become sustainable products — when in reality, ownership costs often dwarf the initial effort and can transform even the most promising initiatives into long-term liabilities.

Understanding this trap is not simply a technical concern but a core responsibility of AI Product Management, because only by acknowledging and planning for the full cost of ownership can we ensure that the solutions we build become products that survive, scale, and create lasting value.

These examples reveal the hidden dangers behind so-called AI quick wins. But if building has become cheap, and owning remains expensive, how can we avoid falling into the same trap? That’s what I’ll explore in the next article.

The illusion of AI quick wins. Part 2 - The Solution.

JBK 🕊️

Some years ago, I would never have expected that almost everyone would understand what I mean when I talk about the cost of wrong AI.

These days, nearly everyone has paid that cost, at least with their nerves, while interacting with an AI. We’ve all faced frustrating moments with the most common AI tools: chatbots that give nonsense answers, recommendation engines suggesting irrelevant content, or voice assistants misunderstanding simple commands.

But there’s a cost that goes far beyond personal annoyance, frustration, or headaches. This deeper cost is felt by businesses that integrate AI into their processes and workflows.

With AI products, there’s a critical dimension that sits above all the usual measures of product success: the quality of the AI’s output.

An AI product might address a user’s pain point beautifully. It might have a sleek, intuitive interface and a high-performing AI model. But ultimately, the product’s value stands or falls on how reliable, accurate, and appropriate the AI’s output is. And no matter how well the model performs, it is never perfect.

Whether you’re working with predictive models or generative systems, the AI model is the beating heart of your solution. Its outputs define the product’s quality in ways that are far more volatile and impactful than most traditional software features.

This is why AI product success hinges first and foremost on the quality of the outputs your model generates, assuming of course that the problem you’re solving is genuinely valuable to a specific user group.

Long before UI polish, feature richness, or clever pricing strategies, the key question remains:

Can users trust what the AI produces?

Because when AI goes wrong, the cost to the business can quickly exceed all the times the AI was right and beneficial.

That’s why every AI Product Manager needs to learn how to measure, monitor, and improve AI output quality.

This is what today’s article is about: the cost of wrong AI.

All I Know: Not All AI Is Measured Alike — and That’s Enough for Now

Not all AI is created equal, and neither are its outputs.

A predictive model might forecast a sales figure, classify a customer’s sentiment, or flag a suspicious transaction. Its output is usually structured, numeric, or categorical—something you can measure directly against the truth. Metrics like accuracy, precision, recall, F1 scores, or ROC curves are well-established and relatively straightforward to track.

Generative AI, however, operates in a completely different arena. Its outputs are creative, open-ended, and often subjective. A large language model might draft marketing copy, summarize a report, or generate code. An image model might produce new artwork or product mockups. In these cases, the “correctness” of the output isn’t always a simple yes-or-no answer. Instead, it sits on a spectrum. The quality of these outputs can depend on style, tone, factual accuracy, coherence, relevance, and even subtle nuances like empathy or humor.

Because of these differences, the way we assess output performance varies dramatically across AI types:

• For predictive AI, we measure how close the output is to a known ground truth.

• For generative AI, we often have to define what “good” looks like for our specific use case, and then find ways to evaluate it—whether through human assessment, automated checks, or user feedback signals.

As AI Product Managers, we need to be fluent in these differences. And by fluency, I mean first understanding what type of AI the solution requires, and second, knowing how its performance can be measured.

Let me also clarify one important point. Even though I’ve been in the AI product management field for over ten years, I haven’t worked with every AI type out there. I’m not fluent in measuring the performance of all possible AI solutions. My experience is mainly in building classical prediction models and those generating insights, such as churn prediction and sales forecasting. And natural language processing—including large language models, which are part of NLP.

But the fact that I’m aware of the nuances and differences among AI types, and that each requires its own methodology for evaluating performance, helps me make informed decisions about where to focus my next chapters of learning. It also prepares me for tackling new AI product challenges that might require different solution types in the future.

That’s the purpose of this article: to give you greater awareness of these distinctions, so you can navigate the cost and complexity of managing AI products with more confidence.

What Is the Cost of Wrong AI?

When people talk about “AI errors,” they often imagine a chatbot saying something silly or a model predicting a slightly off number. But in the context of internal AI products, the cost of wrong AI is much deeper and far-reaching.

As you might already know, I mainly write about AI product management in the context of building AI products within and for enterprises. In this world, I have built my own latticework of mental models. One of those mental models tells me that regardless of where I want to implement AI in a company, it always touches on one of two core types of processes: those that generate revenue or those that protect revenue.

I’m yet to experience that there could be a truly separate third category.

So, in this very simplified world of enterprise processes, it quickly becomes apparent that even a single change within these processes either reduces or increases an outcome. And it’s here that the real cost of wrong AI can often be quantified.

Let’s break it down.

1. The Cost Of Wrong AI in Revenue-Generating Processes

Imagine an AI model used in a sales forecasting process. Its job is to predict how much revenue each product line will generate next quarter. If that model consistently overestimates demand:

• The business might overproduce inventory, tying up capital unnecessarily.

• Sales teams might push the wrong products, missing actual market demand.

• Marketing budgets could be allocated to lower-impact campaigns.

And the result? Missed revenue targets, higher operational costs, and reduced trust in the analytics or product teams who deployed the model. At least, if anyone ever discovers that it’s your solution causing the problem. But that’s a different topic altogether. :)

2. The Cost Of Wrong AI in Revenue-Protecting Processes

Consider fraud detection—a classic example of a revenue protection process. An AI model might analyze transactions to flag suspicious behavior. If the model generates too many false positives:

• Legitimate customer transactions get blocked.

• Call centers become overwhelmed with complaints.

• Customers lose trust and might take their business elsewhere.

I think the point should be clear now.

A Final Example: LLM-Based Tender Assistant

Let’s take one more example—this time from the world of generative AI. Imagine you’re building an internal LLM-based Tender Assistant.

The goal is to help a tender management team quickly analyze and summarize large, complex tender documents from potential partners or clients. On paper, this sounds like the perfect productivity boost. But here’s where things can go wrong:

• The LLM might hallucinate facts, inserting details about tender requirements that don’t exist in the original documents.

• Important legal or financial clauses might be omitted or misinterpreted in the summary.

• The assistant might phrase recommendations too confidently, making users trust outputs without verifying them.

In a tender process, mistakes like these can be costly:

• Teams could base their bid strategies on incorrect information.

• The company might miss critical compliance requirements.

• Misunderstandings could damage relationships with potential clients or partners.

Even if the AI only makes small errors, the cost of cleaning up the mess—through manual document reviews, legal checks, and rework—can wipe out any productivity gains the solution promised. Worse still, if decision-makers lose trust in the assistant, adoption drops, and the entire investment risks becoming shelfware.

This is exactly why the cost of wrong AI goes far beyond just technical performance. In internal enterprise products, it’s about operational disruption, financial risks, and the delicate trust between business teams and the technology they rely on.

The Hidden Costs Behind These Examples

Across all these examples, there’s a common theme:

• Errors don’t just produce slightly “off” numbers—they ripple through processes, triggering downstream costs that can far exceed any initial savings promised by AI.

• Fixing mistakes often means manual rework, retraining models, and eroding stakeholder trust, which can slow future AI adoption.

This is why, in internal enterprise environments, the cost of wrong AI is rarely just technical. It’s operational, financial, and political.

Understanding where your AI product sits in this landscape—and what processes it touches—is the first step in quantifying the true cost of errors.

You Cannot Avoid Wrong AI, But You Can Mitigate the Risk

You might already have understood: There is no such thing as a perfect AI. It simply isn’t possible.

We use machine learning algorithms for problems where ordinary algorithms fail to deliver a proper answer within a reasonable amount of time. These problems are often so complex that you can’t simply dictate rules for how to handle every single case. There are simply too many variations, exceptions, and edge cases.

Machine learning algorithms, instead of relying on predefined rules written by humans, try to make sense of data and discover as many patterns and rules as possible on their own. But this also comes at a cost.

The cost is that we will inevitably get answers with some degree of error. And this degree of error is something we, as AI product teams, need to keep in mind at every moment.

The most successful AI products are those that incorporate strategies to cope with these errors.

How to Build AI Products Ready for Mistakes

So, how do you build AI products that stay successful despite inevitable errors?

1. Know Where Errors Matter Most

Not every mistake is equally significant. Some errors are merely annoying, while others can trigger real financial, legal, or reputational damage. As an AI Product Manager, your first job is to figure out where errors in your AI system would cause the biggest harm so you can prioritize mitigation efforts where it matters most.

✅ Predictive AI:

• Critical when predictions directly drive business actions, like fraud detection, credit scoring, or forecasting.

• Errors here can have measurable financial or regulatory consequences.

✅ Generative AI:

• Equally important but different in nature. Mistakes often mean hallucinations, factual inaccuracies, or off-brand content.

• E.g. a chatbot offering incorrect legal advice, or an image model generating inappropriate visuals.

2. Keep Humans in the Loop

AI alone isn’t enough, especially in high-risk situations. Successful AI products are designed so humans can step in to review, correct, or override AI outputs where necessary. This not only prevents costly mistakes but also builds trust with users who know they’re not entirely at the mercy of the machine.

✅ Predictive AI:

• Less common in high-volume, low-risk predictions but critical in high-stakes use cases.

• E.g. financial approvals, medical diagnoses, security alerts.

✅ Generative AI:

• Essential because generative outputs can be unpredictable and subjective.

• E.g. humans reviewing marketing copy, legal summaries, or code before release.

3. Monitor Performance Continuously

AI isn’t static. Models degrade over time as real-world data shifts or new business challenges emerge. Successful AI products have monitoring systems in place to catch drops in performance early, so issues can be fixed before they cause significant harm.

✅ Predictive AI:

• Standard practice. Retrain models regularly as underlying data changes.

• E.g. changes in customer behavior affecting churn models.

✅ Generative AI:

• Also critical, but more complex.

  • Track hallucination rates.

  • Monitor factual accuracy.

  • Watch for toxic or biased outputs.

• Tools like automated evals and red-teaming are increasingly used to help.

4. Educate Your Users

A critical part of any AI product’s success is teaching users what the system can and can’t do, how to interpret its outputs, and when to be cautious.

✅ Predictive AI:

• Users need to understand that predictions are probabilities, not certainties.

• Helps avoid poor decisions based on overconfidence in model outputs.

✅ Generative AI:

• Absolutely crucial. Generative outputs can appear impressively fluent yet be entirely wrong.

• Users should treat outputs as drafts rather than final truth, and know when to verify information.

5. Design Escape Routes

When AI goes wrong, users need a way out. Successful AI products include features that let users easily reverse decisions, escalate problems, or switch back to manual processes. Designing for graceful failure prevents frustration and loss of trust.

✅ Predictive AI:

• Important for high-stakes decisions. Allow manual overrides or alternative workflows.

• E.g. letting a human analyst confirm a flagged fraud alert.

✅ Generative AI:

• Absolutely essential. Users must be able to reject, edit, or regenerate content.

• E.g. a “Regenerate” button for a chatbot answer, or clear disclaimers on sensitive outputs.

6. Quantify Risk and Communicate Transparently

Finally, successful AI product management means being honest about risk. Don’t hide limitations or pretend your AI is perfect. Instead, quantify how often errors occur, what kinds of harm they might cause, and how you’re reducing those risks. Transparency builds trust and helps stakeholders make informed decisions about using AI.

✅ Predictive AI:

• Often well-established practice. Stakeholders expect error rates and performance metrics.

• E.g. ROC curves, precision-recall trade-offs.

✅ Generative AI:

• Needs extra emphasis because errors are less predictable and often subjective.

• Stakeholders must understand risks like hallucinations, bias, and tone issues, and the cost of mitigating them.

Applying the Strategies: The Tender Assistant Example

Let’s make this real. Let‘s take that LLM-based Tender Assistant for an enterprise example from above. Its job is to analyze large, complex tender documents and produce useful outputs such as:

• Summaries of lengthy legal or technical requirements

• Lists of critical compliance obligations

• Suggested draft responses for tender submissions

• Risk highlights based on tender clauses

On paper, it sounds like a dream tool for efficiency. But here’s where wrong AI can become costly — and how each of our strategies helps manage the risk.

1. Know Where Errors Matter Most

The first step is to pinpoint exactly where mistakes from the Tender Assistant would hurt the business most.

Challenges in the Tender Assistant:

• Summaries might omit crucial requirements, leading to non-compliant bids.

• The AI could hallucinate requirements that don’t exist in the documents.

• Drafted responses might contradict company policy or misstate legal positions.

Applying the Strategy:

• Map the tender workflow and identify critical outputs where errors would have legal, financial, or reputational consequences.

• Prioritize rigorous checks for those outputs, rather than treating every output equally.

2. Keep Humans in the Loop

No AI model should independently drive high-stakes decisions in tender processes.

Challenges in the Tender Assistant:

• Tender content often involves legal, financial, and commercial nuances the AI might not fully grasp.

• Users might wrongly assume AI outputs are legally vetted.

Applying the Strategy:

• Design the product so all AI outputs are clearly marked as drafts.

• Require human review and approval before finalizing summaries or tender responses.

• Provide confidence scores or flags for sections the AI is uncertain about.

3. Monitor Performance Continuously

LLMs can degrade in quality over time as business language, legal standards, or tender formats evolve.

Challenges in the Tender Assistant:

• The model might perform well initially but start hallucinating or omitting details as document styles change.

• Undetected errors could slip into production workflows.

Applying the Strategy:

• Establish routine evaluations on fresh tender documents to check:

  • Hallucination rates

  • Omission of key clauses

  • Consistency in legal or technical terminology

• Encourage users to report errors and feed these back into model refinement.

4. Educate Your Users

A Tender Assistant seems intelligent and authoritative—but users must remember that LLMs can be confidently wrong.

Challenges in the Tender Assistant:

• Users might trust AI outputs without verifying them, especially under deadline pressure.

• Teams may assume the AI has legal or commercial authority.

Applying the Strategy:

• Train users on:

  • AI’s limitations

  • How to spot potential hallucinations

  • The need to treat outputs as drafts, not final answers

• Provide clear disclaimers on every AI-generated summary or recommendation.

5. Design Escape Routes

Users need a way to handle errors gracefully instead of getting stuck with flawed outputs.

Challenges in the Tender Assistant:

• Users may waste time editing unusable outputs instead of starting from scratch.

• Errors might silently propagate if there’s no easy way to escalate issues.

Applying the Strategy:

• Provide:

  • “Regenerate” buttons for new attempts.

  • Clear feedback channels to flag problematic outputs.

  • Options to revert to manual workflows when outputs are unreliable.

• Make it simple to trace back outputs to specific document sections for quick verification.

6. Quantify Risk and Communicate Transparently

Stakeholders must understand that while the Tender Assistant can save time, it’s not infallible.

Challenges in the Tender Assistant:

• Business leaders may overestimate the AI’s capabilities and push for higher automation than is safe.

• Legal teams might worry about liability if outputs are used without checks.

Applying the Strategy:

• Quantify:

  • Average error rates in summaries

  • Frequency of hallucinations

  • Time saved versus risk exposure

• Communicate trade-offs clearly:

  • “Using the Tender Assistant saves 60% of drafting time but requires mandatory human review to avoid compliance risks.”

• Be honest about what the AI can and cannot guarantee.

Final Thoughts

The Tender Assistant example makes one thing clear: it takes significant effort to build an AI product that truly serves users’ needs while managing the risks of wrong AI outputs in a timely and appropriate way.

I’ve become very cautious about which AI product ambitions are worth pursuing. Too often, we don’t fully see the hidden costs and side effects at the beginning. What looks like a potential million-euro opportunity can quickly require millions to build, fine-tune, and maintain. In the end, there might not be much value left on the bottom line—especially if the original business case was based on the wrong assumptions about efficiency gains.

If too much human oversight is required to validate AI outputs, that effort needs to be factored into the business case from the very start. Otherwise, we risk building products that look impressive but fail to deliver meaningful returns.

At the end of the day, a deliberate assessment, involving the right experts at the right time—and crucially, at the very beginning of each initiative—is absolutely essential.

Ultimately, the best AI products expect errors. And that’s exactly why they succeed.

JBK 🕊️

An enterprise decides to “do something with AI.” A promising use case surfaces. A team is assembled. Budgets are allocated. Timelines are drawn. And someone asks, “Who’s going to manage this project?”

That question, on its own, isn’t wrong.

AI initiatives, like any other initiative, require direction and effective delivery. But too often, we try to solve the delivery challenge before we've truly understood the problem. We treat AI initiatives as if we already know what needs to be built, and we assume that whatever gets delivered will automatically be adopted once it’s rolled out.

But if we’re being honest, many of those promising AI use cases are framed by external consultants eager to sell their services, or by managers who may understand the business context but don’t have the time to sit down with real end users. They rarely investigate what needs to be solved first or how a solution should evolve with user feedback. The thinking starts and ends with solutions, not with adoption.

And when you think in solutions, then yes, execution becomes your focus.

But the reality is this: building AI for internal use doesn’t fail because teams can’t execute. It fails, just like many technologies before it, because users don’t adopt what’s been built.

And when I talk about adoption, I don’t mean rollout. I mean real usage. I mean users solving their problems with your product, so that the business starts seeing real value in return.

Now, if you can answer the following question with a confident yes, feel free to skip the rest of this article:

Why Project Thinking Doesn’t Work for Internal AI Initiatives

I see a project as a clearly defined mandate: deliver something based on predefined requirements, with a set start and end date, and within a fixed budget and resource plan.

That’s a project.

And yes, building AI systems can be handled as if they were projects. Someone defines what needs to be delivered, what resources are available, and by when it should be done. Then the execution begins.

But we also need to ask: what is the deeper motivation behind delivering AI systems inside companies?

The main ambition — and it’s the core promise that’s fueled the AI hype — is simple. AI is expected to reduce or avoid cost, and ideally increase revenue. It’s positioned as a technology that can offer insights and generate content at scale, enabling companies to operate far more effectively and efficiently. So even though we may phrase AI use cases as technical solutions, they come with very real business expectations.

And here’s the tension: if the expectation is business value, then simply delivering an AI system is not enough. You may have executed successfully, but still failed to meet the expectation.

Project Management helps you deliver something. But it doesn’t guarantee that what you deliver will generate value. So the real question is: why are so many companies still treating AI as if it were just another IT project?

If we take the business ambition seriously — if we believe that AI is meant to deliver measurable value — then we need to stop thinking in terms of projects. We need to think in terms of products. And just by using the word “product,” we already imply a responsibility toward sustainable, economic value creation. Projects, in contrast, are primarily measured by efficiency: did we deliver on time and within scope?

Those are two entirely different goal systems.

When we treat internal AI solutions as projects, we imply that success is predictable. That if we define the scope clearly and execute well, value will follow. But if that were true, we’d all be billionaires by now. We would just need to execute AI projects efficiently, and the money would come.

Of course, reality tells us something else.

Startups are the perfect example. Some set up their entire organization, from team to product to operations, and still fail. They execute well, and yet they miss the target. Which proves something important: value is not created through flawless project execution. It is created through product-market fit, adoption, and constant iteration.

And the same is true for AI in companies.

Every internal AI initiative carries an economic ambition. But we can execute flawlessly and still fail, and many companies are now seeing exactly that. Even those who claim high adoption rates often admit they don’t see any impact on their bottom line.

That’s the real red flag.

Because if adoption is high, but the financial benefit is missing, something is off. And what that usually means is this: the solution may be valuable to users, but it is not valuable to the business.

This isn’t rare. In fact, it often traces back to the earliest phase of the initiative, where decisions are made about where to invest time and resources. Too often, those decisions aren’t grounded in a clear understanding of how value will actually show up in business terms.

So, if we want to increase the chances of creating value — not just for end users, but for the business as well — we need to change the way we approach AI initiatives. We need the right frameworks, the right mindsets, and the right operating models.

And in my experience, no successful internal AI initiative was ever truly a project. Some were labeled as such, but if you look closely, the ones that succeeded didn’t follow classical project patterns. They did everything necessary to ensure real impact, even if that meant going far beyond the original project scope.

The uncomfortable truth comes when those teams, having built something that works and delivers value, realize they’ve just taken the first step. Because once a “project” is delivered, the real work begins. Suddenly, it’s no longer a one-off project. It’s an ongoing responsibility.

A product, in the best case, runs an indefinite number of projects — each designed to keep it relevant, effective, and trusted. And once you see AI solutions through that lens, the need to build a proper product organization becomes clear. Classical project delivery setups simply don’t support these ambitions.

In the worst case, companies make AI investments based on the assumption that they’ll pay a one-time delivery cost, without considering the long-term operational or maintenance costs. That’s how bad investment decisions are made. That’s how ROI expectations go unmet.

So yes, I’m not a believer in AI projects.

They’ve never delivered on the promises the AI industry continues to make.

Those promises require a deep shift: a real product operating model, a product-driven mindset, and people who know how to manage all of this in one cohesive effort.

Only then can we build AI solutions that do more than just function.
They deliver value — and they continue to do so.

AI Project Management Still Matters

Now, just to be clear: I am not arguing that project management is useless in AI initiatives. Far from it.

Project management still matters — especially once the fog begins to clear.

Every product ambition, once it reaches a certain level of maturity, needs structure. It needs clear priorities, reliable timelines, and accountability. It needs someone to drive progress, manage complexity, align stakeholders, and make sure that what’s been decided actually gets delivered. In this sense, project management brings discipline to the chaos. It’s the engine room that ensures the ship keeps moving.

And if you’re a Product Manager, especially in the AI space, you’ll quickly realize this: at some point, you will need to act like a Project Manager (and in agile contexts they might be named a Product Owner). Not because your title says so, but because the product demands it.

Once you’ve shaped the problem space, validated the key assumptions, and defined what success looks like, the focus shifts. You move from learning to delivering. And when that happens, it’s not strategic vision that gets the product over the line. It’s a clear execution.

That’s where the strengths of project management come in.
And the best AI Product Managers know how to borrow from that skillset when the time is right.

You need to plan without becoming rigid.
You need to track progress without turning into a micromanager.
You need to manage scope and expectations while still keeping your eye on long-term value.

It’s a balancing act — and the more mature your product becomes, the more of that balance is required.

But here’s the key difference: as a Product Manager, you never fully become a Project Manager. You carry a different mandate. You are still responsible for the direction, for making sure the team is solving the right problem in the right way, for protecting the product from drifting into mere output.

So, yes, you adopt project management skills.
You just don’t lose your product mindset in the process.

In fact, I’d argue

They know when to step back and reframe the problem, and when to lean in and drive delivery. They don’t draw a hard line between “thinking” and “doing.” They understand that impact requires both.

This is especially important in internal AI product work, where the boundaries between roles often blur, and where the absence of clear product ownership makes it easy to fall into delivery for delivery’s sake.

So, rather than dismiss project management, let’s integrate its strengths.

Because once your strategy is in place and your product has a clear path forward, execution becomes the strategy.
And if you don’t own that execution, someone else will — and they might not be carrying the product's intent the way you do.

What We Really Need to Acknowledge

If there’s one thing I wish more companies would admit out loud, it’s this:

AI is not just a technical initiative. It’s a product journey.

And as long as we treat it like a project, with fixed timelines, one-time delivery expectations, and predefined scopes, we will continue to fall short of its potential. Not because the teams aren't capable. But the framing is wrong from the start.

AI products live at the intersection of uncertainty, complexity, and change. They don’t just automate tasks. They reshape how people work. They challenge established processes. They require trust, behavior change, and new ways of measuring success. And that’s exactly why they need product thinking at the core.

The uncomfortable truth is that AI success doesn’t come from good project management alone. It comes from having people who know how to manage assumptions, not just resources.
People who understand how to balance strategic ambiguity with operational clarity.
People who can translate potential into progress, even when no one is quite sure what “done” really looks like.

And that means building your AI initiatives around Product Managers, not just Project Managers.
It means giving AI efforts the same treatment you’d give any real product: a clear strategy, ownership across the lifecycle, and an operating model designed for continuous discovery and delivery.

Because

And that, by definition, can’t be a project.

JBK 🕊️

Product thinking is the critical skill. AI expertise is the optimizing one.

You need both, but only one determines whether what you build will matter.

There’s a subtle tension at the heart of modern AI product management, the kind you only begin to sense when a seemingly brilliant solution fails to land. It’s the tension between what AI makes possible and what users actually need, between technological potential and human value.

In many organizations, especially those racing to “unlock” AI, the excitement around the technology overshadows the deeper work of understanding real problems, testing what matters, and designing for adoption. There’s an unspoken belief that mastering architecture, benchmarks, and models will somehow lead to a successful product. But this is rarely the case.

The inconvenient truth is that no matter how deeply you understand AI, or how advanced your model performance is, if you don’t understand product, you won’t make AI work in the real world.

This isn’t a personal failure. It’s a recurring pattern. We’ve seen it before across technological waves—mobile, cloud, APIs, data science. Each time, technical skills surged, but adoption stalled, not because the tools weren’t powerful, but because the work of integrating them into actual behaviors, decisions, and workflows was never done.

AI adds a particular challenge. It feels like intelligence. It creates the illusion that it will naturally solve problems, because it appears to understand them. But intelligence is not understanding. And model performance is not product-market fit.

Building an AI product is not the same as using AI in a product. The second is about integrating AI into an already established product, usually as a feature that supports or enhances something that already works. The first still has to prove itself. It must solve a real problem with AI as the core of the solution, not as an addition. That makes it more complex, more fragile, and more dependent on the quality of decisions made long before the first model is ever deployed.

And those decisions begin in product.

What does it mean to speak the language of product?

It means starting with what’s broken, not with what’s impressive. It means identifying real user friction, not just chasing interesting use cases. It means defining what success looks like before deciding what kind of model might support it. It also means building for change, not just for performance. And not to be forgotten, it means building something others would be willing to pay for, or something that creates meaningful value when used internally across the company.

Product thinking teaches you to look at systems over time, not just point solutions. It demands that you test assumptions, reframe features, and integrate feedback into the foundation of what you build. It helps you create value that doesn’t just show well in a demo, but actually holds up in real-world use.

AI, for all its sophistication, doesn’t know what matters to your users. It doesn’t know whether people will trust probabilistic outcomes, whether your solution fits a real decision path, or whether adoption will quietly vanish after week two. AI doesn’t care if the user goes back to Excel.

That’s why product thinking must come first.

If I Had to Choose, I’d Choose Product

When it comes to AI Product Management, you can separate the skills — but you can’t separate their impact. In an ideal world, your AI PM brings both product intuition and technical depth. But if I had to choose just one, I’d always pick strong product thinking over semi-technical AI experience. Every. Single. Time.

Because technology does not define the outcome. Product thinking does.

I’ve seen technically skilled PMs struggle to frame a clear problem, align stakeholders, or prioritize for adoption. I’ve also seen product-driven PMs with minimal AI experience deliver far more impact, simply because they asked better questions, focused on value, and brought the right people in when it mattered. The truth is, AI can be learned. Judgment can be learned too. But knowing which one matters more — and when — is where most people get it wrong.

That doesn’t mean AI knowledge isn’t important. It is. But AI is a tool, and knowing how to build it is only part of the story. Product thinking is what tells you whether the problem is worth solving, and whether the solution will actually stick.

Without product thinking, you risk solving the wrong problem. Without AI knowledge, you risk solving the right problem too slowly or with unnecessary complexity. But if you have both, you make better decisions earlier. You shape discovery with technical realism. You speak your engineers’ language. You avoid building for novelty and aim for scale.

You stop treating AI like magic. You treat it like leverage. And you stop asking, “What can the model do?” and start asking, “What is the pain we need to address?”

Because in the end, great AI products are not built by those who know the technology best. They’re built by those who know when and why to use it.

And that begins with something far older than machine learning.

Lead with practical wisdom, not theoretical knowledge

The ancient Greeks made a distinction between two forms of knowledge that still feels surprisingly relevant today. Episteme referred to theoretical knowledge — knowing facts, concepts, and systems. Phronesis meant practical wisdom — knowing how to act, how to decide, and how to navigate uncertainty in specific, real-world contexts.

AI skills belong to the world of episteme. They are grounded in logic, in structured knowledge, in models that can be measured and improved. Product thinking, on the other hand, lives in the world of phronesis. It requires you to make sense of ambiguity, to work with incomplete information, and to guide people and ideas toward value in an unpredictable environment.

AI Product Management requires both kinds of knowledge. But it only works when guided by phronesis. When practical wisdom leads, and theoretical knowledge supports.

Either way, if you build what brings in value for the company by building things that add value for users, you have done your job.

JBK 🕊️

🧭 Continue your journey in AI Product Management

If you’re serious about building the right foundation for AI Product Management, here are some essential reads from blog:

• Before the AI Product, There’s Belief
  A personal reflection on why belief and trust are the invisible prerequisites of every internal AI product.

• Why AI Evaluations Have Never Been Optional for AI Product Managers
  A guide on how to treat evaluation as part of product thinking, not just as technical due diligence.

• Most AI Teams Ship Confidently Into the Void – Prototyping as Discovery
  Why prototyping should be a tool for learning, not validation, in internal AI development.

• A Curriculum for AI Product Management
  A comprehensive roadmap of skills, mindsets, and methods for internal AI Product Managers.

• Too Technical to Succeed? – The Peer I Was Advising Was Me
  An honest look at how over-relying on technical depth can derail product decisions.

• The Path to AI Product
  A conceptual journey from AI use case to a real, adopted AI product—with language and framing that stick.

  ---

🔁 Looking for another sharp voice in AI Product Management?

I highly recommend AIPM Guru by . With hands-on experience leading internal AI product development at large organizations like Amazon, Shaili brings a grounded perspective that balances product discipline with technical depth. She writes about stakeholder alignment, AI lifecycle management, model evaluation, and how to frame AI work in product terms—topics often ignored in overly technical discourse.

Here are a few of her must-reads:

• The Future of AI Product Management
  A look at how multi-agent systems, adaptive models, and human-machine collaboration will reshape the role.

• AI Basics: What It Actually Is (And Isn’t)
  A crisp explanation of what AI truly is, and how PMs can stay clear-eyed about its use.

• Can You Prove Your AI Moat?
  A thought-provoking piece on measuring the defensibility of AI features in product contexts.

• CRISP-ML: A Framework for AI Product Managers
  Her 8-part series laying out a comprehensive, repeatable framework for managing AI product delivery from start to finish.

Follow her at aipmguru.substack.com if you want to sharpen your practice and think more strategically about your role as an AI PM.

We love structure. We crave it, especially in large organizations. It gives us something to hold onto in the chaos of delivery deadlines, shifting priorities, and internal expectations. And when we set out to build AI products, we often reach for what looks familiar: a framework, a template, a delivery method someone else has already tested. Scrum. SAFe. The Spotify model. All tried-and-true in some context, all battle-tested, but not in our context. And that’s the point.

Why Predefined Structures Don’t Work

When we import a predefined delivery structure into our teams, we’re implicitly making a dangerous assumption: that our product context is just like someone else’s. That their constraints are our constraints. That their user reality is our user reality.

But internal AI products rarely start with clarity. They start with ambiguity. We don’t just have a vague problem. We often don’t know if there is a real problem yet. We’re not scaling a known solution. We’re trying to discover whether something is even worth solving. And that discovery isn’t just about the product. It’s also about the delivery itself.

How can we assume that any rigid delivery structure will fit a process that is, by definition, still unfolding?

Different AI Products Need Different Ways of Working

Even within the same organization, AI product teams are solving wildly different problems under wildly different conditions. One team might be building a foundational NLP model from scratch, experimenting with new architectures and custom datasets, optimizing for training efficiency, and managing GPU infrastructure. Another team might be wrapping a pre-trained foundation model with internal data, plugging it into an existing workflow, and focusing more on prompt engineering, integration logic, and user testing.

Both are AI products. Both exist under the same company umbrella. But their delivery reality couldn’t be more different.

Now add in other variations: Some AI products are meant to automate existing processes. Others aim to augment decision-making. Some are critical infrastructure. Others are lightweight experiments. Some require explainability and compliance from day one. Others can live as internal alpha tools for months.

And yet, we often try to apply the same delivery expectations — the same sprint cadence, the same roadmap templates, the same governance checkpoints — across all of them.

It’s like assuming that just because two products both use AI, they should be built the same way. But would we expect the same delivery approach for building a backend API and designing a customer-facing app? Probably not. So why do we treat all AI initiatives as if they share the same DNA?

The truth is: the nature of the problem, the data, the user type, and the maturity level all shape how a team needs to work. And so should our expectations of delivery, structure, and success criteria.

That’s why rigid delivery setups fail. Not because they’re bad, but because they assume a level of uniformity that simply doesn’t exist.

Product Discovery and Delivery Shape Each Other

Here’s a more realistic truth: we discover the problem and the way we work at the same time.

With every iteration of learning — from stakeholder interviews, user feedback, model behavior, or data friction — we not only understand the problem better, but also discover how we need to talk about the problem, how fast we can move, what kind of skills we need, who needs to be involved, and what kind of delivery rhythm fits the reality we’re in.

Our delivery structure is not something we apply to the work. It’s something that emerges with the work.

This doesn’t mean we allow anarchy. But it also doesn’t mean strict control is the answer. The goal isn’t to copy structure. It’s to design for emergence with just enough clarity to align, and just enough freedom to adapt.

Governance Demands Flexibility, Not Uniformity

Some argue that internal teams need structure because they operate within a governed environment — and they’re right. Enterprise AI product teams often navigate a web of rules: legal, compliance, data privacy, security, ethics, model monitoring. And those requirements aren’t optional.

But here’s the catch: governance is not one thing. It varies, sometimes dramatically. One company’s governance framework is light-touch and decentralized. Another’s requires seven-step approval chains and model risk committees. In some places, governance differs per business unit. In others, it’s even more granular, with each division applying its own criteria for validation, sign-off, or integration.

This means that even if two teams are building similar AI products, their delivery realities will diverge. Not because the product demands it. But because governance does.

So we’re not just dealing with product diversity. We’re dealing with governance diversity, and that reinforces the need for custom delivery setups, tailored to both the product ambition and the regulatory environment it lives in.

The mistake is thinking that a rigid delivery model will simplify governance. In reality, it often causes friction, because the model wasn’t designed to speak the language of that specific governance body or to deliver the specific artifacts required for that context.

And the truth is: you can meet governance expectations without enforcing delivery uniformity. You can build in traceability, risk assessments, ethical reviews, model explainability, and deployment guardrails without forcing teams to plan, iterate, or communicate according to a model that was never theirs to begin with.

Governance doesn’t require rigidity. It requires accountability. And accountability works best when teams are given the space to define how they meet the right outcomes, not forced into a structure that doesn’t reflect the complexity they’re actually working with.

And while we’re here, it’s worth stating the obvious: we should always look for ways to streamline governance so that it enables, rather than blocks, time to market. That’s another discussion entirely. But one worth having.

What matters most at the start is this: it’s better to begin with bad governance than to wait for perfect governance to exist. Because speed matters. And learning beats delay every time.

Use Frameworks as Inspiration, Not Instructions

This doesn’t mean we shouldn’t learn from frameworks like Scrum, SAFe, or the Spotify model. But we need to treat them as inspiration, not instruction.

Take Scrum, for example. It says: deliver in small iterations and stay close to your users. Build a cross-functional team that owns the full delivery loop. These are healthy principles. But what if you’re building an internal AI service that feeds into five other teams? Or your users are data governance officers, not end users with a UI?

You can’t just run sprints and expect value to emerge. In these settings, understanding the problem landscape and aligning across stakeholder groups takes more time and flexibility than Scrum rituals allow.

Then there’s SAFe — the Scaled Agile Framework. It promotes alignment through Program Increments and a central Agile Release Train. It defines strong role clarity between Product Management, System Architects, and Business Owners. This can work well (to be proven) in large-scale, regulated industries with lots of dependencies. But for internal AI teams exploring whether a GenAI assistant can even solve something meaningful? You don’t need a Release Train. You need a small crew, fast feedback, and the ability to ditch the project if the hypothesis breaks. SAFe is built for predictability and scale, not for discovery under uncertainty.

And then there’s the Spotify model. Famous for its Squads, Tribes, Chapters, and Guilds — often admired for its emphasis on team autonomy and cultural coherence. But here’s the thing: even Spotify itself has said that what became known as “the Spotify model” was just a snapshot, not a blueprint. Henrik Kniberg — who helped describe the model — later clarified that The Spotify model doesn’t even exist. It was a snapshot in time of how we worked.

Ironically, the model became more famous outside of Spotify than inside. Many companies adopted the vocabulary — Squads, Tribes, Guilds — but not the thinking that shaped it: constant adaptation, context over control, culture before structure.

The danger isn’t in using ideas from these frameworks. The danger is copying the form without the function.

Scrum says: ship fast.

SAFe says: align big.

Spotify said: empower the team.

All good thoughts. But none of them should be assumed to fit by default.

You’re Discovering the Product and the Process

This is the heart of it.

When we build AI products internally, we’re discovering two things at once: the problem we should solve, and the way of working that lets us solve it effectively in this environment.

And just like our understanding of the problem changes, our delivery setup must change too.

Maybe we start with fast prototyping. Then we realize we need more data governance. Maybe we start with a team of two, then bring in security and operations. Maybe we start with async feedback, then shift to daily touchpoints once we hit real complexity.

There’s no shame in that. It’s not chaos. It’s adaptation.

Even once we’ve found a delivery rhythm that works — one that fits our team size, product scope, data needs, and governance landscape — we should assume it will need to change again.

Teams change. Tools evolve. The organization reorganizes. AI capabilities shift. And with it, the way we work must shift, too.

That’s not a sign of failure. It’s a sign of life.

We wouldn’t build every product the same way. So why should we deliver them the same way?

The smartest thing we can do for our teams isn’t to give them a rigid framework. It’s to give them the freedom to discover their way of working with our guidance — just like they’re discovering the product itself and getting our feedbacks.

Not in anarchy. Not in chaos. But in conscious evolution, grounded in context.

JBK 🕊️

Especially in corporate settings, where roles are staffed because someone is available, or because someone said, “AI needs a Data Scientist.” But that is the wrong lens. AI does not necessarily need a Data Scientist. A specific type of AI product does. And unless AI Product Managers can tell the difference, we will either over-engineer simple use cases or under-staff the hard ones. That is why it is essential to understand what each role really brings, and when they are critical to delivery, to trust, or to both.

Let’s start with one of the most misunderstood ones: the Data Scientist.

The general role of a Data Scientist: What they’re trained to do

A Data Scientist’s job is not just to “do AI“. It is to make sense of data — often messy, often incomplete, often overwhelming — and turn it into something you can act on. That might mean discovering new insights from customer behavior. It might mean building a model that predicts what users will do next. Or it might mean designing experiments to understand why a system behaves the way it does.

Their toolkit includes:

• Statistical modeling and hypothesis testing

• Exploratory data analysis (EDA)

• Feature engineering and variable selection

• Machine learning algorithm development and tuning

• Model evaluation and validation

• Interpretability and fairness techniques

They combine math, code, and critical thinking to move from data to decision logic. But that does not mean every team building with AI needs them.

What a Data Scientist actually contributes to internal AI products

Inside a company, building AI products is not just a technical challenge. It is a business challenge shaped by organizational complexity. Data is spread across silos. Expectations are vague. Timelines are tight. And trust is everything. In this context, the Data Scientist brings a set of capabilities no other role quite covers.

1. They bridge messy data and model-ready inputs

Internal data is rarely clean. It comes from legacy systems, manual processes, and diverse domains. A Data Scientist does not just take what’s given. They question it. They ask:

• What does this column really mean?

• Is this bias or signal?

• Are we modeling the right problem or just what is easiest to compute?

Their ability to structure, clean, and translate that data is foundational. Because if you do not start from solid ground, no AI model, no matter how sophisticated, will deliver the impact you are looking for.

2. They build models tailored to internal realities

Most off-the-shelf models are trained on public datasets. But internal problems often require internal logic. Whether it is scoring leads based on sales history, predicting churn for B2B accounts, or classifying support tickets with enterprise-level nuance, these are not problems you solve with generic APIs. Data Scientists build models that reflect:

• Internal processes

• Business rules

• Customer behaviors specific to your organization

3. They protect your product’s credibility

An AI solution can fail even if it works — if no one trusts it. That is where a Data Scientist makes a quiet but critical difference. They:

• Test for fairness

• Quantify uncertainty

• Simulate how the model behaves under different conditions

• Help stakeholders interpret the model’s decisions

In internal environments, where decisions affect teams, budgets, and customers, trust is essential. Without explainability, adoption stalls.

4. They turn AI discovery into AI strategy

Even before you have a clear product, you often have data. And before you know what to build, you need to understand what is happening beneath the surface. A Data Scientist helps you explore patterns, test early hypotheses, and generate strategic questions like:

• Which customers are most affected by late delivery?

• What behaviors lead to high support ticket volume?

• Is there a predictable signal before a churn event?

They do not just help you build the product. They help you define what should be built.

When you need a Data Scientist on your core team

You do not need a Data Scientist for every AI initiative. But when you do, no other role can replace them. Here is when their presence goes from nice to have to essential.

You’re developing your own model

If your AI product involves training a custom model — for churn prediction, time-series forecasting, NLP classification, or anything similar — you need a Data Scientist. Even if you are fine-tuning an existing one, you will need their help with:

• Feature engineering

• Hyperparameter tuning

• Model validation and selection

• Performance benchmarking

Without this expertise, you are making educated guesses. And guessing with business-critical data is risky.

Your data is complex or proprietary

Internal data is rarely plug-and-play. It is fragmented, shaped by human behavior, and full of edge cases. A Data Scientist can:

• Identify biases or data gaps

• Select the right features and formats

• Handle imbalance or missing values

• Engineer variables that reflect actual business logic

This becomes essential when internal systems, roles, and processes do not follow clean structures.

You’re building products for insight, not just automation

Many internal AI products are not built to make decisions. They are built to uncover insights. These are discovery or sense-making use cases, and they need Data Scientists to make the results meaningful.

Examples include:

• Customer segmentation models

• Behavioral clustering (e.g., app usage or sales rep activity)

• Root cause analysis of KPI trends

• Pattern or trend detection in network data

• Journey mapping based on events or actions

Without a Data Scientist, these products remain shallow. With one, they generate value across teams.

Your product must be explainable and auditable

In regulated industries, or anywhere that decisions must be reviewed or audited, transparency is non-negotiable. A Data Scientist ensures:

• Traceable logic and clear decision paths

• Documentation of how predictions are made

• Use of techniques like SHAP or LIME

• Monitoring for fairness, accuracy, and drift

This is not just for compliance. It builds confidence.

You’re in a discovery-heavy phase

Some products begin not with specifications, but with questions. In this case, a Data Scientist is the person who helps test feasibility and shape hypotheses. They can:

• Analyze opportunity areas

• Simulate outcomes

• Estimate model performance

• Clarify whether the use case is worth building

They help turn ambiguity into informed direction.

When you don’t need a Data Scientist on your core team

Not every machine learning–powered product requires a Data Scientist from the start. In some cases, their core skill set — model development, data exploration, statistical reasoning — may not be critical to delivering value early on. Here’s when that’s likely to be the case.

You are using foundation models via API, without custom training

If your AI product relies on foundation models (like GPT, Claude, or Gemini) for capabilities such as summarization, semantic search, classification, or generation, and you are not fine-tuning or training on your own data, then a Data Scientist is not immediately necessary. What you are doing is orchestration and application, not model innovation.

You’ll likely need:

• A Prompt Engineer (if this is even a role) to structure interactions

• An AI Engineer to handle retrieval or context enrichment

• A Designer to build usable workflows on top of the model

Until you reach a point where you need to analyze performance or fine-tune with internal data, a Data Scientist would have little to contribute.

You are using pre-trained models for narrow ML tasks

Some internal products embed pre-trained models that perform a very specific ML task — like image classification, sentiment analysis, or language detection. If these models perform well enough and do not require retraining, you can build valuable products around them without needing a Data Scientist.

Examples:

• Using an email sentiment classifier to route internal tickets

• Applying an OCR model to extract structured data from documents

• Leveraging a pre-trained keyword extractor to tag customer interactions

The model is already built. Your work is around productization, not modeling. Similar to the foundation model case above.

You are still validating the problem and don’t need model development yet

If your AI product is still in the early discovery phase, and your main question is whether the problem is real, valuable, and solvable with ML, then you may not need a Data Scientist immediately, if you can derive the answer without heavy data analysis. What you then need first is:

• A clear problem framing

• An understanding of available data sources

• A simple prototype to test workflows or model fit

You might work with an AI Engineer or generalist to explore feasibility using basic, Non-ml models or prebuilt tools. A Data Scientist can come in once the product vision matures.

You need model orchestration, not model creation

Many internal AI products rely on combining multiple ML components — retrieval, embedding search, pre-trained classification — but do not involve building or training new models. The complexity lies in gluing pieces together, not in discovering patterns.

Common examples:

• Retrieval-augmented generation (RAG) for internal knowledge assistants

• Multi-step workflows using off-the-shelf models

• Semantic search powered by vector embeddings

These products still use ML, but they are integration-heavy, not data science–driven.

Your evaluation focus is on business metrics, not model performance

If your current goal is to test whether the product drives business value or user adoption, rather than tuning model performance, a Data Scientist is not the most urgent role. You may be testing:

• Whether users trust the AI assistant

• Whether recommendations improve outcomes

• Whether response time or accuracy meets baseline needs

Until model quality becomes the limiting factor, other roles will move the product forward more effectively.

What I’ve Learned From Building Internal AI Products

We often debate roles. But the more honest question is: What does the product need to succeed?

The title “Data Scientist” may sound generic. But their work is anything but. They are not just model builders. They are pattern finders, uncertainty reducers, and sometimes the only person in the room who understands whether your assumptions are statistically sound. When the product needs that, they are the right person. When it doesn’t, they’re not.

The goal isn’t to staff roles based on trends.

The goal is to solve the right problem with the right capabilities.

Most AI Products Today Don’t Need a Data Scientist—And That’s Not an Insult

We are in a wave of AI product launches. Internally. Externally. Everywhere. But most of them share a pattern. They are:

• Wrappers around foundation models

• RAG-based assistants powered by vector search

• Applications using off-the-shelf models for classification or summarization

They are products built on top of pre-trained intelligence, not new intelligence developed from scratch. Which means they succeed or fail based on:

• Prompt design

• Workflow orchestration

• User experience

• Data integration and context

In this context, a Data Scientist is often not the limiting factor. The real blockers are adoption, alignment, or usability. So no—most AI products today don’t need a Data Scientist. They need strong engineers, great designers, and clear product thinking.

But that’s not the end of the story.

The Hidden Strength of the Misunderstood Role

For years, Data Scientists were treated as the “unicorns” of AI. Everyone wanted one. Few knew what they were actually supposed to do. Expectations were sky-high: build the model, explain it, deploy it, make it scalable, make it usable, make it compliant—and do it alone.

As a result, many Data Scientists developed much broader skills than their job title suggests.

You’ll often find Data Scientists who:

• Write production-grade code

• Design evaluation pipelines

• Build dashboards and reporting layers

• Tune prompts and experiment with LLM-based architectures

• Manage experiments or run early-stage product discovery

So before we decide whether a Data Scientist belongs on the team, we should ask a more nuanced question: Can this person cover part of what we need, even if their title says otherwise?

Not every company has all the roles they need on paper. But sometimes, they already have someone who can fill the gap—quietly, competently, and creatively.

We rarely get to staff ideal teams.

But with the right awareness and flexibility, we can still build the right products.

JBK 🕊️

If you found this useful or want to share your approach to building internal AI teams, let’s talk. I’d love to hear what you’ve tried, what worked, and what didn’t.

A year ago, I wrote about why we need a curated learning repository for AI Product Managers. Today, I want to show you how we can build it together.

Over the past year, I’ve written dozens of articles on internal AI Product Management and read at least twice as many from others doing the same. Topics range from shadow governance to MVP value measurement, from cross-functional buy-in to the traps of over-promising AI capabilities.

But here's the challenge: all that wisdom is scattered.

Some of it sits in blog posts.
Some in LinkedIn threads.
Some in decks you’ll never see.

What’s missing is a place that pulls this knowledge together. Not into a theoretical textbook, but into a learning path grounded in practice.

So I created a structure for it. A curriculum.

The Curriculum I Wish Had Existed

A few months ago, I published A Curriculum for Internal AI Product Management. It’s designed like an M.Sc.-level program, but with one big difference: it’s for internal AI Product Managers. The ones navigating the complexity of building and scaling AI solutions inside large organizations. Not on a greenfield. Not in startups. But deep inside orgs with legacy tech, process silos, conflicting goals, and real impact on the line.

The curriculum covers topics like problem discovery in internal teams, AI solution prototyping and rollout, adoption metrics beyond usage, compliance, explainability, governance, and collaboration with data science, legal, and business units.

It’s a map, not a manual. And like every good map, it becomes more useful the more people contribute to it.

Why AI PMs Should Lead This

AI Product Managers are in a unique position to lead this kind of learning effort because they sit at the intersection of multiple disciplines. They work across engineering, data science, business, legal, operations, and compliance, often within the same week, sometimes within the same meeting.

That vantage point gives them an unusually broad perspective. Unlike specialists who go deep in one field, AI PMs constantly connect dots between teams, technologies, processes, and real-world problems. They translate between mindsets, resolve contradictions, and shape solutions that are not only technically sound but also usable, trustworthy, and aligned with business needs.

This role forces them to see what’s missing. It forces them to ask the uncomfortable questions others avoid. It also gives them visibility into what knowledge gaps slow down progress, where alignment breaks down, and what kinds of insights actually move things forward.

So when it comes to building a curated learning path, one that reflects the realities of shipping AI inside complex organizations, AI PMs are not just participants. They’re the ones best positioned to lead.

They know what’s essential, what’s overhyped, and what truly helps teams deliver impact.

How You Can Contribute

I’m now opening up this curriculum to contributions from anyone working in or around internal AI product delivery.

There are three easy ways to get involved:

1. Match your content to a course

Each class in the curriculum includes a short description. Based on that, you can send me your articles, articles you’ve read and recommend, talks, tutorials, or even short courses. Just let me know which class your contribution fits best. If it aligns, I’ll link it directly from the curriculum and credit your work. This way, learners not only follow a structured path but also benefit from real-world perspectives.

2. Suggest a new course

This curriculum reflects how I see internal AI Product Management, but it’s not set in stone. If you think there’s a course missing, whether it’s a topic you’ve struggled with, something you wish you had learned earlier, or an area you teach often, let me know. If it’s helpful for internal AI PMs, I’m open to adding it.

3. Propose an elective

The curriculum includes elective modules for areas like new technologies, frameworks, or methods that are still evolving. If you’ve developed expertise in something like enterprise AI agents, prompt ops, data-centric evaluation, or domain-specific architectures, you can propose a new elective. You can even co-create it with me.

How to Contact Me

I regularly share my thoughts on LinkedIn and on Substack, that’s where you’ll find my articles, reflections, and frameworks on internal AI Product Management. If you want to contribute to the curriculum or share a relevant piece of work, the best way to reach me is via the comments under my posts or articles.

I'm not very active in DMs. Not because I don’t value the input, but because there’s simply too much to respond to one-on-one. Everything I’m building here happens next to my full-time role as a Lead AI Product Manager and Strategist, and as a Co-Founder of the AI Center of Excellence at Vodafone. And beyond work, there’s also a personal life I care deeply about and try to protect.

So if you want to send me a suggestion - whether it’s an article, a new course idea, or a contribution to one of the electives - please comment under my latest LinkedIn post or Substack article, no matter which one. That’s where I’m most likely to see it and respond. Even though notifications are off, I check the comments regularly because they’re part of my publishing routine.

I know it’s not the most convenient setup. But it helps me stay focused - both in my work and outside of it.

Maybe one day I’ll find a better way. Let’s see.

PDF Download: A Visual Overview of the Curriculum

To make things easier to explore and share, I’ve also created a PDF version of the curriculum. It includes:

• An overview of all semesters and course modules

• Core and elective topics

• A clear structure that reflects the internal AI PM journey

I’ll be sharing it on LinkedIn soon, so feel free to download, share, or reference it as you explore the curriculum or think about contributing.

Help Spread the Word

If you believe in the value of this curriculum and the idea of a curated learning path for internal AI PMs, here’s how you can help it grow:

1. Share the article. Share it with your network, especially with people working in AI, product, data, or digital transformation roles inside large organizations.

2. Tag someone. Know someone who’s written something brilliant? Tag them. One mention can bring in a whole new module.

3. Leave a comment. Even a short note saying “this is needed” helps others take notice. And if you disagree with something or have ideas for improvement, even better.

4. Add your voice. Send me your article, video, course, or story and tell me which class it fits. Or propose a new one. I’ll take care of linking it where it belongs.

The more we co-create this, the more useful it becomes. Not just for learners, but for all of us trying to make AI work where it matters most.

JBK 🕊️

#beyondAI

Not long ago, the primary difficulty in AI was the model and the entire system around it. Designing, training, and deploying machine learning models at scale required specialized infrastructure and niche expertise. But with the rise of GenAI, not only were new capabilities introduced, but the arrival of foundation models shifted much of that difficulty away from internal product teams. Today, the most advanced language, vision, and reasoning capabilities can be accessed through a simple API, maintained by a handful of vendors who have taken on the challenge of scaling intelligence as a service.

So yes, one could argue that building something with AI has never been easier. But building something with AI that truly deserves to be called a product is still as hard as ever. A real product is something that earns repeated usage, delivers sustainable value, and justifies long-term investment. And none of these vendors handle the parts that matter most to your company. They don't uncover real user pain points. They don't manage internal politics. And they don't ensure that AI fits into business-critical workflows where trust must be earned, not assumed.

A real product is not a demo, not a prototype, and not a PoC. It is something that makes a measurable difference. It is something people use regularly because it is better than the alternative. It improves outcomes in a repeatable way. And over time, it either generates revenue, reduces cost, or creates undeniable internal efficiency that changes how the organization operates. That bar hasn’t changed. And neither has the difficulty of reaching it.

We may no longer need to build an AI system ourselves. But we still need to build the AI product. And that part is still hard to get right.

That is why knowing how to iterate carefully, intentionally, and always anchored in value is not just helpful for an AI Product Manager. It is foundational.

This Article Is About Internal AI Products

The focus here is on internal AI products. These are the kinds of tools that live inside enterprises, embedded in business workflows, and designed to support teams like sales, HR, customer operations, finance, or IT. In these environments, success is rarely measured by user growth or market share. It’s measured by adoption, time saved, process compliance, or impact on key business outcomes.

While many of the principles in this article, especially those around value-based iteration, also apply to external AI products, the nature of iteration is different when the users are customers. External products are shaped by market dynamics, monetization strategies, and competitive positioning. Internal AI products, on the other hand, grow within an existing system. You’re not launching into a blank canvas. You’re building into legacy workflows, informal shortcuts, and organizational expectations that were never designed with your product in mind.

That’s where this approach begins: with the reality of building AI products in the messy middle of real businesses.

Related Article

How Internal AI Products Grow: The Value Step Iteration - Method

This approach isn’t an entirely new idea. It builds on established product thinking, using the same principles you’ll find in Lean Startup, Outcome-Driven Innovation, or Continuous Discovery. What makes it worth calling out here is that it’s essential in AI product development, especially for internal use.

Internal products operate in highly entangled environments. The product doesn’t enter a greenfield. It enters a system that’s already full. It has to coexist with legacy systems, awkward handoffs, informal workarounds, and teams who already have a way of getting things done.

Even if that way is inefficient, it’s familiar.
And that familiarity has power.

Growing into that environment means the product must start by fitting into a small, real task. It needs to be narrow enough that it doesn’t disrupt the system, but valuable enough that people notice the improvement. Once it’s proven there, it can expand. But not before. Internal AI products don’t earn adoption through excitement or novelty. They earn it through consistent, grounded usefulness. That means every part of the product must make sense in the context it’s entering.

That’s why each feature, each iteration, and each technical decision must be scoped as a value hypothesis.

This changes how you prioritize, how you scope, and how you release.

You don’t build more unless the last iteration has been adopted.
You don’t expand scope unless what you’ve already built is solving a real problem.
You don’t solve downstream tasks if upstream usage is still low.

This doesn’t slow you down. It keeps your effort aligned with reality.

So while this approach is not new, and while its principles are shared across other product disciplines, what makes it non-negotiable in internal AI product development is the combination of complexity, ambiguity, and proximity to the user.

You’re not building for abstract personas.
You’re building for colleagues.
And they’ll only adopt your product if it delivers clear, immediate, and lasting value. One iteration at a time.

The Real Job of Internal AI Products: Workflow Support

Internal AI products — especially those powered by generative AI — are most effective when they support the people closest to the business problem: subject matter experts. These are the analysts, controllers, legal reviewers, strategists, and other domain professionals who carry deep institutional knowledge and apply it through structured, repeatable work.

These experts are not waiting for full automation. They are looking for support tools that reduce manual effort, eliminate routine steps, and help them move faster and more confidently through their tasks. GenAI can do exactly that. Not by replacing the expert, but by augmenting the steps where friction, repetition, or low-leverage effort slow things down.

Every task a subject matter expert performs follows a workflow. Some steps are linear. Others loop or require judgment. But each step, regardless of size or visibility, takes time. One might take thirty seconds, another half a day. One might be mentally heavy, another just tedious. The key is not to obsess over which step is more “valuable”, but the most time-consuming. Because from the expert’s point of view, the value lies in completing the entire workflow so they can produce the deliverable — a report, a contract, a presentation, a recommendation.

That’s why the priority isn’t evaluating the strategic value of each individual step. It’s identifying where the most effort accumulates. Because if we can reduce the time or complexity of just one high-effort step, we accelerate the entire task. That creates capacity. It frees up expert time. It opens the door to faster delivery. And faster delivery often has very real financial consequences — shorter billing cycles, earlier client handoffs, or quicker internal decision-making.

Each step in a workflow, then, has an indirect but very real impact on delivery — and therefore on business value.

This is exactly where Value Step Iteration becomes essential. The goal is to solve one real step in the value chain of the SME. Once that’s done — once it’s working, adopted, and useful — then you move to the next. Each supported step moves the system forward without overwhelming users or overcommitting your team.

Value Step Iteration is a way of scoping AI product development by focusing on individual workflow steps — not features, not UI screens, not model capabilities. It asks one question at a time:

Which step in the expert’s workflow consumes the most time or effort, and can AI meaningfully reduce it?

And this is exactly why the Value Step Method is so powerful in internal AI product work. It treats every iteration not as a technical milestone, but as a test of whether one step in a workflow can be supported in a way that moves the whole system forward. You don’t need to automate the full task. You need to support one step, clearly, confidently, and usefully. And when that’s done, you earn the right to move on.

How Versioning Supports the Value Step Method

Once we understand that each AI product iteration should support a meaningful step in an expert’s workflow, the next challenge is structuring how we build — and how we know when we’re ready to move forward.

This is where the Value Step Method benefits from a clear versioning model. It gives the team — and the organization — a shared language to describe progress. Not just in terms of functionality shipped, but in terms of how much trust has been earned, how deeply the product is used, and how reliably it supports the delivery of actual work.

We use a semantic versioning-like structure here not only to track internal releases, but to represent product maturity through the lens of value creation and adoption. Each version tells a story about how far the AI product has grown into its environment — and how confidently it supports a step that matters.

Here’s how the progression unfolds within the Value Step Method:

• 0.1 – 0.3: Early learning stages. You're validating that a problem exists in the workflow and that AI could reasonably support it. The system may work in parts, but lacks stability. Users are curious, but not yet relying on it. This stage is about listening, testing, and refining your understanding of what to build.

• 0.4 – 0.6: A focused, functional slice starts to emerge. You’ve solved one specific step well enough that some users begin to replace manual effort. The output may still need review, but the time savings or flow improvement is real. This is where User Acceptance Testing (UAT) becomes essential, not as a formality, but as evidence that value is landing.

• 0.7 – 0.9: The product handles one full task end-to-end with minimal oversight. It has earned trust. The expert begins to rely on it, not just experiment with it. Usage is self-sustaining. Feedback shifts from “is this useful?” to “can this be expanded?” Now the assistant is no longer a pilot — it’s becoming part of the real workflow.

• 1.0: The product is embedded. It operates independently, consistently, and meaningfully supports delivery. It’s trusted. The workflow flows better with it than without it. If your team stepped away, the users wouldn’t — because the tool is now part of how work gets done.

This structure prevents premature scaling and anchors progress to real, observed outcomes. It slows down hype and speeds up clarity. And it keeps everyone aligned on what matters: earning the right to take the next step, one clear piece of user value at a time.

That’s the essence of the Value Step Method.

Applying the Value Step Method

Let’s make the Value Step Method tangible.
Imagine you’re building a Sales Enablement Assistant to support internal sales teams in preparing for meetings. The aim isn’t to automate everything. The aim is to reduce effort where it accumulates — one workflow step at a time.

Sales reps might spend 30 to 60 minutes before each client meeting collecting scattered information. They pull CRM history, search email threads, check for open tickets, review notes, and try to stitch it all together into something actionable. It’s inefficient, inconsistent, and error-prone. But every step is required to reach the deliverable — a well-prepared, high-quality client conversation.

This is exactly where the Value Step Method applies. You’re not trying to replace the rep’s expertise or redesign the entire process. You’re trying to identify which step consumes the most effort, and reduce it meaningfully, so the whole task moves faster with less friction.

Phase 1: Deconstruct the Workflow

You begin by mapping the full set of actions the SME performs to deliver the output. For a sales rep preparing a meeting, the steps might include:

• Pulling CRM data

• Surfacing past communications

• Highlighting active deals or support escalations

• Generating talking points or reminders

• Packaging all of it into a meeting-ready brief

Each step takes time. Some are tedious, some mentally demanding, some dependent on multiple systems. The task can’t be delivered without all of them, but not all of them are equally painful. The goal is to understand where that effort stacks up, and where AI support would be immediately useful and low-risk to adopt.

Phase 2: Prioritize by Effort — Not Feature Appeal

With the workflow mapped, you now apply Value Step Prioritization. That means choosing where to start based on effort concentration and feasibility, not based on what’s technically impressive or what stakeholders ask for first.

For example:

• Step 1 - Summarize CRM activity — Structured, stable, and quick to implement.

• Step 2 - Add past interactions — Higher complexity, but high value for user context.

• Step 3 - Highlight support issues — Relevant data exists, but signals must be interpreted.

• Step 4 - Format into a briefing — Only makes sense once the content is reliable.

• Step 5 - Suggest talking points — High ambition. Trust must already be in place.

This way, you're not only identifying where support is needed in the workflow — you're also generating a product roadmap that reflects real user effort, not internal assumptions. You begin where confidence is high, time savings are visible, and the expert knows the step well enough to evaluate the quality of support. It’s a roadmap that’s earned, not imagined.

Phase 3: Iterating an AI Assistant from 0.1 to 1.0 — One Step at a Time

Now you build — but in small, scoped releases, each targeting one meaningful step in the workflow. Every release is a test: Can this specific step be reliably supported in a way that reduces effort and builds trust with the SME?

In this case, you might decide to begin with Step 1 and Step 2, since they both focus on surfacing past interactions — a critical part of meeting prep — even though they draw from different source systems. The value is clear, the data exists, and users already know what “good” looks like.

So the first version of the Sales Enablement Assistant focuses only on these two capabilities. The team iterates from 0.1 to 1.0 solely around making this functionality useful, stable, and adopted. No additional features. No unnecessary expansion. Just making sure that this slice of workflow is genuinely improved and trusted.

And because the scope is small and focused, this path can realistically lead to a strong, adopted MVP 1.0 within 1 to 3 months — one that solves a real problem, earns a place in the workflow, and gives you a solid foundation for future iterations.

Here’s what iteration looks like:

• Version 0.1 — First Exposure, First AI Touchpoint

  • The assistant is introduced to a small group of users

  • It can fetch and summarize CRM data from one system

  • The AI generates briefs with basic metadata and deal context

🎯 Goal: Validate that the AI adds value immediately, even in a narrow scope
👥 UAT: Do users trust the summaries? Does it save them any time?

• Version 0.2 – 0.3 — Expand Input Sources, Maintain Clarity

  • Add a second data source: e.g. past email conversations

  • Merge AI-generated summaries from both systems into one preview

  • Add light metadata (date, contact, topic) so users can verify without switching tools

🎯 Goal: See if users start using it unprompted#
👥 UAT: Is the assistant’s context accurate enough to reduce manual lookup?

• Version 0.4 – 0.6 — Structure, Feedback, and Flow

  • Introduce simple formatting into the briefing: sections, headings, and collapsed views

  • Add a “Was this helpful?” feedback prompt for each section

  • Begin timing usage (e.g., do users open it before meetings?)

🎯 Goal: Make the assistant’s presence feel structured, consistent, and safe
👥UAT: Are users adjusting their workflows to include the assistant?

• Version 0.7 – 0.9 — Reliable Prep, Reduced Manual Effort

  • Auto-deliver the briefing ahead of meetings via Slack or email

  • AI adjusts the summary slightly depending on meeting type

  • Users no longer open multiple systems before calls

🎯 Goal: Shift from usage to reliance
👥UAT: Does the assistant fully replace previous prep steps for most users?

• ✅ MVP 1.0 — Fully Embedded Assistant for Prep Tasks

  • The AI Assistant is now part of the meeting workflow

  • It surfaces relevant context with minimal user input

  • No major gaps remain in the selected steps (Step 1 & Step 2)

  • Success Metric: Users would notice if the assistant disappeared

  • Optional: Product team can declare MVP 1.0 before reaching 0.9 if adoption and trust are strong

🧠 Note : You Don’t Have to “Complete” Every Version
The path from 0.1 to 1.0 isn’t a checklist — it’s a maturity scale. If you reach a point by version 0.4 or 0.6 where the assistant is clearly helping, being used, and considered trustworthy, you can declare MVP 1.0 and focus on scaling, onboarding more users, or expanding to the next Value Step.

The key is not building more, but proving usefulness earlier.

Beyond 1.0: Iteration Doesn’t Stop — It Evolves

Reaching MVP 1.0 means the assistant reliably supports one or more high-effort workflow steps. It’s used. It’s trusted. And it has earned its place in the way work gets done. But with the Value Step Method, 1.0 isn’t the finish line — it’s just the moment you know the product is alive.

Beyond this point, iteration becomes more strategic. The goal is no longer to prove usefulness, but to extend value responsibly — without breaking trust, introducing unnecessary friction, or overloading the assistant with capabilities it doesn’t need.

Every next step should still follow the same principle: support one real workflow step at a time, and earn the right to build further.

Version 1.1 – 1.3: Deepen the Fit

• Improve system performance, AI output clarity, and UI polish

• Add fallback logic for missing data or integration hiccups

• Refactor internal flows to reduce manual dependencies

• Add light internal documentation to help support scale

🎯 Goal: Make the assistant more resilient, faster to use, and easier to maintain — without changing its core purpose

Version 1.4 – 1.6: Enrich the Context

• Introduce additional signals — e.g. product usage, customer satisfaction, escalation status

• Tune the AI briefings to adjust based on context (e.g. new client vs long-time customer)

• Let SMEs contribute improvements to prompt templates or adjust summary preferences

🎯 Goal: Increase the assistant’s relevance and accuracy without increasing user effort

Version 1.7 – 1.9: Expand the Workflow

• Support additional teams such as customer success, technical account managers, or partner sales

• Adapt briefing templates to their language, tasks, and workflows

• Start identifying new high-effort steps (e.g. follow-up emails, meeting documentation)

🎯 Goal: Extend the assistant to adjacent use cases that mirror the original workflow — not reinvent it

✅ Version 2.0+: Generalize the Pattern (Only If It Makes Sense)

• Codify the assistant architecture as a reusable internal product pattern

• Create a briefing generation framework that can be adapted for other domains (e.g. onboarding, procurement, incident response)

• Offer a lightweight assistant kit to teams facing similar workflow pain

🎯 Goal: Turn what worked in one context into a pattern — but only if the demand and maturity support it

Each Step Beyond 1.0 Still Follows the Same Rule: Prove It

Post-1.0, it’s tempting to grow faster, add features, or “productize” too soon. But the Value Step Method keeps you grounded. You don’t add unless the last step is adopted. You don’t expand unless the next workflow is real. And you don’t generalize until the use case has been proven in more than one place.

More is only better if it’s earned.

Final Thoughts: Adoption Is the Outcome That Matters

The hardest part about building internal AI products isn’t the model, or the integrations, or even the workflow mapping. It’s building something that people actually choose to use.

And not just once, out of curiosity — but every day, because it quietly makes their work easier.

That’s what adoption really means. It’s not feature usage. It’s not click-through rates. It’s when a subject matter expert says, “I rely on this now.” And in internal environments, that kind of adoption has to be earned, one step at a time.

The Value Step Method helps you do exactly that. It grounds your AI product decisions in reality — in the shape of real workflows, the weight of actual effort, and the friction that professionals already feel in their day-to-day work. It gives you a way to focus. A way to say no to unnecessary features. A way to align stakeholders around what matters right now, not what might matter in six months if everything goes perfectly.

And just as importantly, it gives you a way to measure real progress. Not by how much you’ve built, but by how much value you’ve delivered — and how deeply that value has embedded itself into the organization.

So if you're building internal AI products, the job isn’t to chase completeness. It’s to find that one step, in that one workflow, where AI can reduce friction in a way that builds trust.

Then you do it again.
And again.
And you stop when the product is truly part of how work gets done.

JBK 🕊️

It’s been a while since I last wrote to you. I wasn’t planning to write again so soon. But I believe it’s time. The world you’re walking into still isn’t quite ready for your role. And yet—somehow—it needs you more than it thinks it does.

This letter isn’t about certainty. It won’t offer you a playbook. But it will give you perspective, and maybe a bit of peace. These are not instructions. They’re observations, shaped over years of quiet work in spaces that reward clarity, trust, and persistence more than they do noise or speed. I’m writing to help you walk in with your eyes open, your spine straight, and your convictions intact.

You’re stepping into a role that will challenge how you think about influence, impact, and even the definition of a product. It won’t look like what you studied. It won’t behave like the case studies you admired. And it won’t reward you with the kind of visibility people usually associate with success. What it will give you, if you let it, is something much deeper. A chance to shape systems from the inside out. Slowly. Quietly. Meaningfully.

But only if you learn to see the real game being played.

Belief Comes First

You’re not here to manage a product—not in the way most people imagine it. You’re here to manage belief. Not belief in the technology, but belief in the problem, in the opportunity, in the idea that something could work better than it does today.

Your first real delivery won’t be a working prototype. It’ll be a conversation that shifts someone’s thinking. A slide that reframes a loose ambition into something concrete. A sentence that helps a team see their situation differently.

You’ll be building trust before you build anything else. And trust is a product of its own—it’s slow to earn, impossible to fake, and constantly under review.

So yes, you may not be managing a product on day one. But you are creating the conditions for one to exist. And that’s the deeper truth: belief is what precedes the product. It’s the foundation.

Eventually, yes—there will be a product. Something visible. Usable. Valuable. But it will only come into being because people believed enough to invest. To participate. To take a risk.

That’s what you manage first.

And ironically, it’s the most product-like thing you’ll ever handle—because if belief doesn’t get adopted, nothing else ever will.

Direction Over Speed

There will be pressure to ship fast. To stay visible. To justify your presence with features, demos, and deadlines. And while it’s true that real success in this role is about direction—not just velocity—that doesn’t mean speed doesn’t matter. It does. But you need to understand what kind of speed actually moves things forward.

Speed, in your world, isn’t about rushing. It’s about rhythm. It’s about knowing when to show something early—not to prove that it’s finished, but to prove that you and your team can be trusted. A fast prototype is rarely about the solution itself. It’s about keeping belief alive. It’s about showing that you understood the problem well enough to reflect it back, even in rough form.

A well-timed prototype builds credibility. It shows momentum. And when done right, it serves two purposes at once: it validates that you heard the problem correctly, and it earns you the permission to keep going.

So don’t fall into the trap of saying “we’re still aligning” for too long. Find small ways to materialize progress. Give people something to react to. Use prototypes not just to test ideas, but to test alignment.

You’ll spend more time aligning than building. That’s true. You’ll negotiate expectations more than APIs. You’ll be asked to simplify without oversimplifying. And yes, it will often feel slow.

But don’t confuse slowness with stagnation. And don’t assume that speed means shallowness. Done right, speed builds trust. Momentum keeps attention. And both will help you earn the space to do the deeper, more lasting work.

That’s your value. Not speed for its own sake—but speed with purpose. Prototypes with intention. Direction with momentum.

Navigating the Tension

You’ll live in a space no one quite mapped out for you. A space between the promise of AI and the weight of everything that came before it, the legacy. Between what’s technically achievable on paper and what’s actually permitted within the realities of enterprise policy, procurement, compliance, and regulation.

You’ll see that the pace of innovation—what you read about in articles and GitHub threads—doesn’t match the pace of production in your organization. Even when the solution is clear, you’ll hit walls: fragmented ownership, risk-averse sponsors, outdated infrastructure, unclear decision rights.

And at first, that tension will frustrate you. It will feel like a series of blockades, like a test of your patience. But over time, you’ll learn that this friction isn’t an accident. It’s the operating environment. And it’s where your role actually begins.

Because this tension is more than bureaucracy—it’s a map. It reveals where things stall, where handoffs fail, where no one ever quite took responsibility. It shows you which teams don’t talk to each other. Which tools don’t integrate. Which incentives are misaligned.

And all of that? That’s where you’re needed most.

AI in the enterprise is not just about the model. It’s about the path the model has to travel—from idea to integration to actual use. That path crosses technical systems, business processes, cultural resistance, and institutional memory. Your job is not to bulldoze through the tension. It’s to navigate it. To make it visible. To stitch together a way forward.

Sometimes that means asking the uncomfortable question. Sometimes it means slowing down just enough to bring the right people into the room. Sometimes it means offering a version one that works within constraints—so you earn the right to unlock version two.

You’re not here to erase the tension. You’re here to read it like a roadmap. Because that’s where the work is. That’s where the opportunity hides. And that’s where you’ll build real leverage—not just in the product, but in the system around it.

Define the Problem First

One of the hardest parts of your job won’t be building the solution. It will be getting people to agree on what the problem actually is.

You’ll be handed symptoms disguised as requirements. Neatly packaged requests that sound urgent but fall apart the moment you ask, “What happens if we do nothing?” You’ll see wishlists passed off as strategy decks. You’ll hear phrases like “We need an AI for that,” without clarity on what “that” even is.

And you won’t be handed a clean brief. Don’t expect one. In internal environments, real pain often lives in the handoffs, in the grey zones between departments, in the quiet workarounds people invent when processes no longer serve them. These frictions rarely make it into Jira tickets. They live in side comments. In hallway chats. In the Slack threads that keep getting reopened.

Your job is to sit with that ambiguity longer than most people are willing to. Not to panic. Not to solve it too quickly. But to stay in the discomfort long enough to understand what’s underneath.

Because what’s real is usually not loud. It rarely announces itself in numbers. It shows up in context—how someone describes their day, what they pause before saying, how three stakeholders give three versions of the same process.

That’s where your work really begins.

Because once you name the thing that no one could quite articulate—but everyone silently felt—you shift from being someone who “does AI” to someone who can actually fix things.

And that trust, more than the tech, is what earns you the right to build.

So yes—stay in the ambiguity.

Sit with it.

Map it.

Respect it.

But also—don’t get stuck there.

And I know I told you: build the prototype as early as possible.

And I still mean it.

Because sometimes the fastest way to test whether you’ve understood the problem is to show someone a rough version of the solution. A sketch. A simulation. A flow. Something they can point to and say, “Yes, but not like that.”

Do both. Stay in the problem long enough to see it clearly.

And then build something—anything—that lets others see what you’ve begun to understand.

That’s how momentum begins.

That’s how alignment forms.

That’s how trust grows.

Build What Actually Works

Your users won’t act like startup users. They’re not early adopters looking for the next big thing. They’re not customers to win over with feature launches or branding campaigns.

They are survivors of broken systems. They’ve seen tools arrive with fanfare and leave without a trace. They’ve onboarded to platforms that made their lives harder. They’ve wasted time clicking through interfaces that solved nothing. And quietly, they’ve stopped expecting that the next solution will be any different.

That’s the emotional landscape you’re walking into. You’re not just up against complexity—you’re up against disappointment.

So don’t try to impress them. Don’t sell them on the latest AI model or show off a slick UI just to get their attention. What they really want is something that finally works. Something that makes their job easier without asking them to rethink everything. Something that fits into the way they already get things done—only smoother, simpler, quieter.

And if you can give them that, you’ll earn something more valuable than praise. You’ll earn trust.

Trust that this solution will stick around. Trust that it won’t make their day harder. Trust that you understand how their world really operates.

That’s the bar.

Not innovation for innovation’s sake. Not flashy features that no one asked for.

Build the thing that makes the workflow disappear.

If you can do that, you’ve already won.

Adoption Beats Accuracy

You’ll build brilliant models no one uses.

You’ll run evaluations that hit every metric—precision, recall, F1 score—all green. You’ll have dashboards and charts that prove, objectively, that the model is right. And still, no one will use it.

Then one day, you’ll build something small. A quick automation. A simple tool that answers just one question. It won’t feel like much at first. But people will love it. They’ll start relying on it. You’ll hear things like, “I just use that now,” or “This saves me two hours every time.”

And that’s when it will click.

Accuracy alone doesn’t create value. Adoption does.

The model that gets used will always outperform the one that gets ignored—no matter how brilliant the math behind it is. Because in enterprise environments, the challenge is rarely technical. It’s behavioral. It’s emotional. It’s organizational.

People don’t adopt a tool just because it’s accurate.

They adopt it because it feels safe. Because it fits into their already overloaded day. Because it doesn’t create more friction, more questions, more need for explanations.

And here’s something you really need to understand: most AI products—at least today—are optimization tools. They’re not existential parts of the process. They’re not like the billing system, the CRM, the approval chain—systems that people have to use, whether they like it or not.

Those systems, even when poorly designed, don’t have to earn their place. They’re mandatory.

But your AI product? It’s optional.

You will have to earn every user.

So build with real-world friction in mind. Build for people who are tired, busy, skeptical. Build for the actual pace and pressure of operations—not for the clean conditions of a test environment or the assumptions in your design doc.

Know the Value, Show the Value

If you say your product creates efficiency, you’d better know how.

It’s not enough to throw the word into a slide or a stakeholder update. In enterprise environments, efficiency isn’t a concept—it’s a conversion. So ask yourself: what does it actually unlock? Does it allow revenue to come in earlier? Does it increase total revenue potential? Does it reduce costs in a measurable way? Or does it free up internal capacity that’s already overstretched?

Trace the impact, end to end. From model output to business outcome. From screen interaction to savings. From faster workflows to tangible decisions made in less time, with fewer errors.

Because if you can’t trace the value, don’t pretend it’s there.

And don’t expect others to believe in it either.

Now here’s something just as important: every product that creates value has a benefit owner. Someone inside the business who gains when your product works. A team that hits its KPIs more easily. A department that meets its targets with less stress. A leader who can finally report progress with confidence.

Find that person. Name them. Build with them—not for them. When it’s time to defend the product, you’ll want their voice in the room. Because when you speak, it’s seen as advocacy. When they speak, it’s seen as proof.

Value doesn’t speak for itself—not in companies this size.

You have to name it, translate it, and socialize it.

And if you do that well, your product won’t just survive.

It will spread.

Measure What Matters

Yes—you must measure it. Even if it’s inconvenient. Even if it’s messy. Even if the dashboards don’t exist yet and the systems don’t speak to each other. Measure anyway.

Because what you don’t measure, you can’t protect.

And what you don’t protect, you’ll eventually lose.

Measuring isn’t just a reporting task. It’s a declaration of value. It tells others that what you’re building deserves to be monitored, improved, and resourced. And it tells you—honestly—whether you’re solving a real problem or just building something interesting.

But I won’t tell you exactly how to measure. And I don’t think anyone should. Because the process of figuring it out—of chasing the signal through broken systems, of having uncomfortable conversations with finance, of getting alignment on what “impact” even means—will reveal more about your company than any training or template ever will.

You’ll learn how decisions are made. How money flows. How KPIs are chosen. What gets attention and what quietly dies.

Measurement is where product management becomes political. Strategic. Real. It’s where you stop being the person who “supports” the business and become the one who helps shape it.

And yes, it will be hard.

Understand the Power Structures

A brilliant idea can die in the wrong room.

Not because it wasn’t viable. Not because it didn’t create value.

But because the timing was off.

Because someone influential wasn’t informed early enough.

Because someone else needed to be the one to introduce it.

Because the political current wasn’t flowing in your direction that day.

This isn’t cynicism. This is structure.

In large organizations, decision-making isn’t always linear or logical. It’s relational. It’s layered. It’s shaped by past projects, previous battles, personal credibility, and invisible lines of influence that don’t show up in the org chart—but define what gets greenlit and what quietly dies.

Your job is to read the system. Learn who really owns what. Learn which departments set direction, and which ones follow. Understand who speaks in meetings, and who people look at before nodding.

And most of all, know when to push and when to pause.

Don’t waste energy trying to force alignment too early. Build it.

Don’t manipulate. Navigate.

Be the one who sees the landscape clearly, who anticipates the question before it’s asked, who gives others the space to come to the idea on their own—even if it was yours.

Influence is not earned through brilliance alone. It’s earned through clarity, patience, consistency—and knowing when to let others lead the conversation you started.

That’s how things move forward in the enterprise.

Not just because the idea is good. But because the conditions are ready.

Make Governance Your Ally

Legal, compliance, risk—these are not blockers. They’re not the people you “deal with at the end.” They’re part of the system. And if your product is meant to last, if it’s meant to scale, they need to be part of it from the beginning.

Too often, teams treat governance as a final checkpoint. A phase that starts after the product is “done.” But if you wait until then, you’re not asking for input—you’re asking for forgiveness. And in most enterprise environments, that’s not a risk you can afford to take.

So bring them in early. Treat them like partners, not gatekeepers. Ask how they think about risk. Ask what they’ve seen go wrong before. Ask what success looks like to them. Because governance is not just about preventing harm—it’s about building confidence. And confidence is what lets good products move faster.

Speak their language. If they care about audit trails, show them how you’re logging decisions. If they care about model explainability, invite them into how it’s being approached—not once it’s finished, but while it’s still being designed.

If you build that trust early, something powerful happens. Governance stops being a slowdown. It becomes a multiplier. It becomes the reason your product doesn’t just get approved—it gets championed.

Because in the enterprise, moving fast doesn’t mean skipping steps.

It means designing with every step in mind, from day one.

Your Words Will Build the Future

You’ll build more decks than prototypes. And at first, that might feel like a failure. Like you’re not doing enough of the “real work.” But it’s not a failure. It’s the job.

Because inside an enterprise, storytelling is not a soft skill. It’s a product skill.

It’s how you earn buy-in from people who’ve seen a hundred ideas come and go. It’s how you help someone far removed from the problem understand why it matters. It’s how you unlock budget, secure resources, align teams, and get one more shot to keep going.

And when it’s done well, your words become infrastructure. They shape how people talk about the problem when you’re not in the room. They turn ambiguity into something actionable. They turn scattered opinions into shared understanding.

So take your words seriously. Don’t just document—design your communication like you design your product.

Write like a business analyst. Be specific. Structured. Clear. Anticipate the questions.

Talk like a founder. With belief. With conviction. With the quiet urgency of someone who knows what this could become.

Be clear in the doc.

Be convincing in the room.

Because in the enterprise, your slides might travel further than your code.

And your words—if you use them well—will open doors your prototype never could.

You Won’t Like Everyone—And That’s Fine

Some of the most important people you’ll need to work with will frustrate you. They might be overly political. They might move too fast or too slow. They might operate from ego, from fear, from habit. They might not listen the way you want them to. And truthfully, you won’t like how they work.

And still—you will need them.

Because they’ll have access. To decision-makers. To funding. To influence that you, at this point, simply don’t have. They might be the only ones who can get your product in front of the board. The only ones who know the quiet context behind a “no” that no one explained to you. The only ones who can open the door when you’ve already knocked three times.

So do yourself a favor: don’t waste energy trying to change them. Don’t turn your frustration into resistance.

Put your ego aside.

This isn’t about admiration. It’s about progress.

You don’t have to like everyone.

They don’t all have to like you.

But you do need to find a way to work with them.

You can draw a boundary without burning a bridge. You can collaborate without compromising your values. And you can be respected, even by people who’ll never fully understand what you do—if you show up consistently, deliver what you promise, and make it easy for them to move forward with you.

That’s not selling out.

That’s strategy.

And sometimes, that’s what makes the difference between your product staying stuck and actually making it out into the world.

Start With Their Story, Not Your Tech

When you sit down with someone about a potential AI project, resist the urge to pitch. Don’t start with architecture. Don’t show slides. Don’t talk about accuracy or use cases from other departments.

Start with their story.

Ask about their world. How their day actually starts. What they check first. What slows them down. What drains their team. What decisions take longer than they should. What keeps breaking. What’s been broken for so long they’ve stopped bringing it up.

And then—just listen.

Because if you listen long enough, you’ll hear something no system or dataset can tell you. You’ll hear context. You’ll hear history. You’ll hear how this team works around the tools they were given, and how they’d work with something if it finally understood them.

And when someone feels heard—genuinely heard—they open up. They stop performing. They stop pitching back. They start trusting you.

That’s when you can introduce AI.

Not as the shiny solution.

Not as a disruptive force.

But as a quiet helper. A tool to make the pain smaller. A way to make the work smoother.

You’re not selling intelligence.

You’re offering relief.

And if you start with their story, the tech will follow—because now, it has somewhere real to land.

Slides Are Your Craft

I already told you that you’ll build more slides than prototypes. Now let me tell you why that’s not a limitation—it’s a power tool.

Slides are your programming language.

What code is to a developer, slides are to you.

They’re how you shape the narrative. How you translate complexity into clarity. How you make the invisible work visible—so people can rally behind it.

But to use them well, you need to treat them like craft. You need to learn to build for different audiences. What makes a data team nod with interest might completely miss the mark with leadership. What excites your product squad may raise red flags for legal or compliance. One message doesn’t fit all.

So learn the mechanics. Learn to use color intentionally—not to decorate, but to direct attention. Learn the basics of visual hierarchy. Learn what it means when something feels too dense, too fast, too unstructured. Understand how whitespace creates rhythm. How font size can shape priority. How structure creates trust.

Your slides aren’t just supporting materials.

They are thinking tools. Alignment tools. Influence tools.

They are how you open a conversation with someone new.

They are how you guide a room that’s halfway bought in.

They are how you follow up when you’re not in the room to speak for yourself.

You’re not building to impress.

You’re building to align.

So don’t outsource this.

Own it.

Get better at it.

And treat your slides with the same intentionality you give to your product.

Because sometimes, the right slide will do what no prototype ever could:

It will get people to believe.

Give Credit Generously

You’ll work with people who care deeply about being seen—sometimes more deeply than you’ll ever know. Even if they never say it. Even if they act like it doesn’t matter.

Don’t underestimate how far a thank-you can go. A name mentioned in a meeting. A quiet message to their manager. A sentence in a deck that says, “This happened because of their work.”

It’s easy to focus only on outcomes. On roadmaps, metrics, and prototypes. But behind every milestone, there are people solving things in the background—removing blockers, catching edge cases, smoothing tensions you never even saw.

Learn to give credit to whom it belongs. Not just because it’s the polite thing to do. But because it builds trust. It tells your team: “I see you. I know what you made possible.”

And in a world where uncertainty is high and visibility is low, that acknowledgment matters.

It creates safety.

It creates loyalty.

It creates momentum that doesn’t show up in KPIs, but absolutely shows up in what you can build together.

Even if you don’t care much about being recognized, remember this: for someone else, it might mean everything.

Your words could be the reason someone believes they belong here.

And that kind of belief is what keeps great people from walking away.

Accept Credit With Clarity

And while you give credit to others—freely, frequently, and with care—don’t forget to accept it for yourself. Not with ego. With clarity.

Because in enterprise settings, product management is often misunderstood. It’s not like in startups or product-native companies, where everyone knows what it means to define a roadmap, validate a need, or align tech and business.

Here, much of your work happens in the background. Quietly. In pre-reads, in workshops, in Slack threads, in unglamorous decisions that prevent future chaos.

So when your product succeeds, the impact won’t always trace back to you. People might thank the data science team. The sponsor. The delivery lead. And they wouldn’t be wrong—but they wouldn’t be seeing the full picture either.

Unless you help them see it.

Be clear about your contribution. Not as a claim to ownership, but as a thread in the story. Say, “Here’s how I helped move this forward.” Say, “This was where product decisions made the difference.”

Make your role visible. Especially across departments. Especially when the org doesn’t quite know what product management actually means in an AI context.

This isn’t self-promotion.

It’s context.

It ensures that your role is valued, resourced, and included when it matters most—next time.

So yes—give credit to others. And also give it to yourself.

Because this is how you make sure you—and your work—don’t stay invisible.

Find the Right Spotlights

Don’t wait for the perfect moment to show your product. Don’t wait until the first use case is finished, polished, and ready for a showcase. If you wait for perfect, you’ll miss momentum.

Start earlier.

Build visibility into your workflow—not just at the end, but all the way through. Share what’s in motion, what you’re learning, what questions you’re wrestling with. Let people follow the journey, not just the result. Because when people see something grow, they’re far more likely to care when it arrives.

And make sure you’re showing up in the right rooms. Find the spotlights that match your moment:

Brown bags. Cross-functional communities. Weekly leadership calls. Internal demos. Product forums. Use them to build narrative capital—quietly, steadily.

Don’t limit yourself to your immediate team. Speak to adjacent departments. Drop your name into spaces where curiosity outweighs urgency. Connect with people who may not need your product today but might influence its path tomorrow.

Because this is how internal awareness grows.

This is how you create optionality.

This is how scale begins—long before it’s formally requested.

You’re not marketing to the outside world. But make no mistake: inside an enterprise, you still need to market.

Because no one supports what they don’t know exists.

And no one funds what they haven’t seen for themselves.

When the World Isn’t Ready—But You Are

I’ve told you a lot, Little Jaser. Maybe more than you liked to read. Maybe not yet as much as I wanted to say. But here’s what I know for sure: the role you’re stepping into still isn’t widely understood. Not by every team. Not by every decision-maker. Not even by every product leader.

You’ll walk into rooms where people won’t know exactly what you do—or why it matters—until they’ve seen it play out. And even then, they might not be able to name it.

That’s the reality.

But it’s not the limit.

Because this job—this quiet, persistent, often invisible work of making AI real inside complex organizations—isn’t about being recognized. It’s about recognizing what’s needed before others do. It’s about holding the shape of a solution long enough for others to step into it. It’s about building trust before traction, belief before buy-in, and alignment before action.

And that’s something I know you’re built for.

Because the truth is, I trust you. Deeply.

You’ll figure it out—just like I did.

You’ll find your own rhythm. Your own way of moving between teams, shaping conversations, navigating the fog, and turning it into something real. Something useful. Something that works.

After all, it’s you writing to you.

Only with 25 more years of doing the work.

Of asking the second question.

Of learning when to wait, and when to act.

Of making mistakes that felt big at the time—and learning, with time, that they were just part of discovering what really matters.

You already carry what you need.

Even if you can’t name it yet.

Even if the world isn’t ready to recognize it just yet.

So go use it.

Go build the things that won’t exist unless you do.

Go be the one who sees what’s possible, even when the systems around you are still operating like it’s not.

And when it feels like no one sees what you’re really building—read this again.

And remember: I see you.

Elder Jaser

JBK 🕊️

In AI products, it’s dangerously easy to pass every technical test — and still fail the user.

In this article:

• Why evaluations are the hidden foundation of great AI products

• How GenAI evaluations differ radically from traditional ML

• A real-world example of evaluating a GenAI system

• Practical mistakes to avoid — and how to do it right

#beyondAI - If you don’t deeply understand how to evaluate AI behavior, you’re not managing a product. You’re gambling.

When we build AI products, one mistake can easily slip into the foundation without us even noticing: assuming that if the model works technically, the product will work for users.

That’s not true.

And this is exactly why AI evaluations (AI evals) are so critical — especially for AI Product Managers.

Evals aren’t just a technical health check. They are a way to make sure that the AI system performs in a way that serves the user, under the real conditions and expectations the product must fulfill. If we don’t understand evaluations deeply — and I mean beyond just “accuracy” or “precision” percentages — we risk building AI solutions that pass technical tests but fail spectacularly when they meet the real world.

I’ll be honest: I’m still learning.

And to stay honest — this is the first time I’m going this deep into the new world of AI evaluations.

Other things had priority.

(Real PM is speaking here… we always have to pick our battles.)

AI evaluations — especially for GenAI and LLMs — are a rapidly evolving field, and every project teaches me something new.

With this writing, I’m sharing what I’ve learned so far, hoping it helps others who are navigating the same shift.

The evaluation landscape changes depending on the type of AI system we’re dealing with. A traditional machine learning model (say, a churn prediction algorithm) is evaluated very differently from a GenAI model like a chatbot powered by a Large Language Model (LLM).

As AI Product Managers, we have to know the difference — and we have to know how to lead teams to design, interpret, and act on evaluation results that actually matter for the product.

Let’s get into it.

AI Evals and the Core Mission of Any Product Manager: Creating Value

At the end of the day, Product Management is about one thing: creating value for the business by solving problems for users.

Whether you’re building a mobile app, an internal tool, or an AI system, this doesn’t change.

The twist with AI products — and especially GenAI products — is that you can’t separate the technical behavior of the system from the user experience it creates.

If the AI model behaves poorly, feels unreliable, or simply doesn’t align with user expectations, it directly kills the value you’re trying to generate.

In traditional software, this separation is possible.

A button might technically work — it sends a request to the backend, triggers the right workflow, and returns the expected result — even if the visual design or wording isn’t perfect.

In other words: technical functionality and user experience are distinct layers.

You can fix usability later without needing to change how the system itself computes or behaves internally.

But in AI, especially in GenAI, the system’s behavior is the user experience.

There’s no “under the hood” you can separate cleanly.

When an AI writes an email reply, generates a product description, or answers a customer question, the output is the product.

There’s no layer between the user and the system’s core behavior — no abstraction shield.

And this is why evaluations in AI Product Management are not optional or secondary.

They are essential to ensuring that the product fulfills its real-world purpose — and that the business actually captures the value it hopes to create.

Traditional AI Evaluations: Clear Metrics, Narrow Scenarios

In traditional machine learning, evaluations are mostly built around structured outputs.

You might predict a binary outcome (“Will this customer churn?”) or a continuous value (“What’s the estimated price of this house?”).

To evaluate such models, the industry has relied on metrics like:

• Accuracy

• Precision and Recall

• F1 Score

• ROC-AUC

• Mean Absolute Error (MAE) and Root Mean Squared Error (RMSE)

The key thing here is: the evaluation criteria are objective and quantifiable.

You know the ground truth (whether a customer actually churned or not).

You know the prediction.

You can run the math and get a score.

The role of the AI PM in this setup is relatively straightforward:

• Define which metric matters for the business use case (e.g., precision over recall if false positives are very costly).

• Align the team on the target thresholds that mean “good enough” to ship.

• Monitor model drift or degradation over time.

In short: in traditional ML, evaluations are clean, comparable, and repeatable.

But then came GenAI — and the game changed.

GenAI and LLM Evaluations: Messy Outputs, Moving Targets

When you move into GenAI, especially with LLMs, the nature of the output changes completely.

Instead of predicting a simple label, the AI now generates free-form text, images, even code.

The output space is basically infinite.

And because there is often no single “ground truth” answer, traditional metrics break down.

You can’t easily calculate “accuracy” on a chatbot that gives slightly different but equally acceptable responses to the same question.

You can’t just look at a number and say “this model is ready.”

Evaluating GenAI models involves concepts like:

• Human-likeness

• Factual correctness

• Relevance to the query

• Completeness of answer

• Bias, toxicity, and harmful content detection

• Style and tone alignment

And to make it even more complicated: human judgment is often needed.

Human evaluators have to assess if an LLM’s response was helpful, respectful, in line with brand voice, or free of hallucinations.

In practice, evaluation setups for GenAI now involve a mix of:

• Prompt-based testing (feeding in test prompts and evaluating outputs)

• Rubrics for human raters (scoring outputs against subjective quality criteria)

• Automated evals using smaller models (“critique models”) trained to assess the main model

• Red teaming (actively trying to break the model by feeding adversarial prompts)

What AI Evaluation Looks Like in Practice

Let’s take a practical example:

Imagine you are shipping an internal GenAI tool for customer service agents. The AI suggests draft replies to customer emails. An evaluation setup might look like this:

• You define 100 typical customer emails covering different topics: billing issues, product complaints, upgrade requests, cancellations, technical support.

• You feed these emails into the GenAI system and collect the draft replies.

• Human reviewers — preferably real customer service agents — score the AI replies on dimensions like:

  • Relevance: Does the reply actually address the customer’s issue?

  • Tone: Is it polite, professional, and empathetic?

  • Accuracy: Are factual statements (e.g., refund timelines, account policies) correct?

  • Actionability: Is the reply clear on the next steps for the customer?

Each of these dimensions might be scored on a scale from 1 to 5.

You might find, for example:

• 85% of replies are relevant (good)

• 78% are in the right tone (okay)

• 65% are factually correct (problem)

• 60% are actionable (problem)

As an AI PM, you now have concrete signals.

You don’t just “hope” the model is good — you know where it’s failing for the business and where fine-tuning, guardrails, or post-processing might be needed.

That’s the real life of AI evaluation: structured, messy, human-in-the-loop, and critically important.

How AI Evals Relate to UX: The Hidden Parallel

The more you work with GenAI evaluations, the more you realize: AI evals have more in common with user experience (UX) testing than traditional software testing.

When we build normal software (no AI involved), we know that:

• The code compiles or it doesn’t.

• The feature works or it doesn’t.

• The button clicks through or it doesn’t.

But even if everything works technically, the user might still hate the experience.

Maybe the button is hidden.

Maybe the flow is confusing.

Maybe the error message feels rude.

The only way to find this out?

UX testing.

You need to observe how users interact with the product in real conditions. You need feedback that’s not about whether something works, but how well it fits into the user’s life.

With GenAI, it’s the same.

A model might respond to a prompt. Technically, it “works.”

But is it clear?

Is it respectful?

Is it helpful?

Is it concise enough, or too verbose?

Evaluations for GenAI are essentially a form of UX testing for AI behavior.

What This Means for AI Product Managers

As AI PMs, our role is to bring evaluation to the center of product thinking, not treat it like a final hurdle before release.

Here’s what this practically looks like:

• Define evaluation goals early.
  Before any data science starts, define what “good” looks like for the user — not just technical performance, but experience quality.

• Mix automated and human evaluations.
  Understand that GenAI models need both structured evals (where possible) and subjective assessments.

• Create meaningful prompt sets.
  Work with your team to design realistic, diverse prompts that represent the full range of user behavior — not just the happy paths.

• Continuously test and monitor.
  GenAI models can drift not only in technical performance but also in tone, helpfulness, and safety.
  Ongoing evaluations are not optional.

• Translate eval results into product decisions.
  Don’t just hand over evaluation reports to data scientists. Interpret them in the light of business goals and user experience expectations.

Common Mistakes in AI Evaluations

Even experienced teams can stumble when it comes to evaluating AI products.

Some of the most common mistakes I’ve seen:

• Focusing only on technical accuracy:
  High accuracy scores don’t mean users will trust, enjoy, or even accept the AI’s behavior.

• Testing only the “happy paths”:
  Evaluation sets often miss real-world edge cases, sarcasm, ambiguous queries, or hostile prompts.

• Using unrealistic test data:
  Clean, idealized prompts make the model look good. Real users don’t write like a textbook.

• Skipping human evaluation steps:
  Relying only on automated scores might be faster, but it often misses subtle but critical issues like tone, clarity, or user perception.

Avoiding these mistakes isn’t just about better testing — it’s about delivering an AI experience that feels trustworthy and valuable to real users.

First Steps Checklist for AI PMs

When you’re building your evaluation plan, keep it simple to start:

• Define your key evaluation dimensions:
  What does success look like to users — relevance, clarity, helpfulness, safety?

• Design realistic prompt sets:
  Use messy, real-world examples, not sanitized ones.

• Mix human and automated evaluations:
  Don’t rely on metrics alone — integrate human review cycles.

• Evaluate continuously, not just at launch:
  Plan for monitoring model performance after deployment.

Final Note

The more I work with AI products, the clearer it becomes: We’re not just building features. We’re shaping behaviors, expectations, and trust. Evaluation isn’t a side task. It’s how we stay honest with ourselves — and with the people we build for. It’s how we check if what we’re creating is truly helping — or just adding noise to an already noisy world.

And evaluation is how we stay close enough to see that truth — and strong enough to act when what we see isn’t good enough yet.

That’s the kind of AI Product Management I believe in.

JBK 🕊️

There’s a quiet assumption in most tech teams that feels so natural we rarely stop to question it:

Code is considered an asset. Problem understanding is not.

That single mental model — largely unspoken — silently shapes the way AI products are built. It influences what gets prioritized, who gets funded, how progress is measured, and what success looks like.

And more often than not, it’s also the reason why AI products miss the mark — not because the code was poorly written, or the model undertrained, but because we built something scalable and sophisticated… that nobody actually needed.

When Code Becomes the Hero — and Discovery Gets Forgotten

In most organizations, especially those driven by delivery milestones, code is treated as proof of progress. It’s visible. It’s documented. It’s reusable. It shows up in sprint reviews, gets archived in Git, and lives on in the roadmap. People get excited about it because it feels like something you can touch, something you can show — an asset that endures.

This perception is deeply ingrained in how product and engineering teams operate. Teams celebrate pull requests and production pushes. Roadmaps are mapped in epics and features. Burn-down charts show velocity, and demos show outcomes. We build. We deliver. We optimize.

But if we zoom out for a second, we have to ask:

Build what? Deliver why? Optimize toward what pain?

That’s where things start to unravel — and where discovery enters the picture.

The Invisible Work of Problem Understanding

Discovery is where everything starts.

But in most companies, it’s not treated like part of the real work.

It’s often a prelude, something done at the beginning of a project and then left behind. A few user interviews. A canvas workshop. Some sticky notes in Miro. Maybe even a thorough slide deck with pain points and opportunity spaces.

And then? It vanishes.

Even when discovery is done well, it rarely gets maintained, versioned, or reused. It doesn’t live inside the product backlog or inform quarterly OKRs. It’s not tracked like code, logged like bugs, or celebrated like a successful deployment.

That’s the real risk.

If discovery isn’t seen as an asset —

it doesn’t get time.

It doesn’t get attention.

It doesn’t get the company’s best minds.

And in AI Product Management, that’s not just unfortunate — it’s dangerous.

The Supposed Forgiveness of Software Development

People like to say that in classic software development, you can get away with unclear discovery.

You ship a feature, see what happens, tweak it, and eventually land on something usable.

The cycle is iterative. The stakes are lower. It’s all very forgiving.

It’s not forgiving. It’s just familiar failure.

We’ve normalized teams shipping into the dark and hoping that agile rituals will save them later.

And most of the time, it doesn’t.

Unclear discovery in software leads to the same thing it does in AI:

Wasted resources, lost time, and user problems that remain unsolved.

The difference?

AI is more expensive.

• The cost of experimentation is higher.

• The time to validate is longer.

• And the risk of eroding trust — through wrong answers, hallucinations, or unfair behavior — is much harder to recover from.

So no, it’s not that AI is more fragile than regular software.

It’s just that the price of misunderstanding the problem is paid up front — and in full.

It’s more fragile, more data-dependent, and less predictable.

You can’t always iterate your way out of a bad starting point.

You can’t just refactor your prompts and magically land in product-market fit.

And you definitely can’t backtest your way into solving a real human problem if you didn’t deeply understand the problem in the first place.

This is why so many AI teams ship confidently into the void.

They’re moving fast. They’re technically capable.

But their code is built on sand — assumptions that were never validated, pain points that were never fully understood, users that were never truly involved.

The result?

A beautiful solution to the wrong problem.

Why the Code Survives — Even When the Product Fails

The irony in most failed AI products is that the code survives.

It lives on in version control. It gets reused in other experiments. It becomes a library, a model, a reference.

But the discovery — the messy, human understanding that should’ve guided the build — is lost.

And that’s why I started asking myself:

What if we could treat discovery like we treat code?

What if problem understanding was also seen as an asset — not just a phase?

I didn’t want to fight the delivery mindset.

I wanted to work with it.

And that’s how I landed on a shift in thinking I now call:

Prototyping as Discovery

It’s not a new idea, but one not widely spread in the AI dev space.

Most people think of prototyping as a way to test a technical solution, not a user solution.

But what if the goal wasn’t to test your build — but to explore your understanding?

Prototyping as Discovery is a mindset shift.

It means building not to ship, but to learn. And yes — it goes by many names.

It’s about treating early product increments as strategic probes — ways to uncover real user behavior, real constraints, real patterns in data and usage — not just to validate assumptions, but to uncover the ones we didn’t know we had.

It’s a way of embedding discovery inside delivery.

Not as a box to tick before dev starts, but as an ongoing process that grows alongside the code.

You discover while you build.

You build while you discover.

And both outcomes — the code and the insight — become valuable assets.

What It Looks Like in Practice

You don’t need to restructure your team or get buy-in for a whole new methodology. You just need to start treating early cycles as insight engines.

Here’s one way I’ve framed it:

• 2 weeks of focused discovery: interviews, workshops, pain point mapping, data landscape review

• 6 weeks of dev: small prototype that targets the most promising problem with the lowest fidelity possible

• 2 weeks of follow-up discovery: observe what happened, run validation sessions, collect behavioral data

• 6 more weeks of dev: build on the insights and iterate toward a real solution

Each phase feeds the other.

Each step produces both code and context.

The understanding is just as valuable as the functionality.

Over time, this approach turns your discovery from a one-time effort into a continuously compounding asset.

And people know it as Continuous Discovery, coined by Teresa Torres.

Making Discovery Look Like an Asset

In many organizations, the challenge isn’t that people don’t believe in discovery — it’s that they don’t see it.

They don’t see it in dashboards.

They don’t see it in team reviews.

They don’t see it in the OKRs or the sprint metrics.

So part of the work is making discovery visible.

That might mean:

• Keeping a living repository of insights and opportunity spaces

• Capturing key learnings from prototypes as artifacts, not just lessons

• Measuring insight velocity alongside code velocity

• Including discovery-driven pivots in stakeholder updates

The more you surface these outcomes, the more credibility discovery builds — not as an idea, but as an investment.

Something worth time. Worth talent. Worth treating like code. Shareable throughout the entire company.

Why This Matters Now — Especially for AI

As AI continues to evolve, the space between technical capability and real-world impact is widening.

We have tools that can generate, classify, summarize, translate — almost instantly.

But what we often don’t have is clarity on what problems are truly worth solving.

When the tech gets easier to build, the temptation to skip discovery grows.

When everyone’s focused on prompts and models, understanding becomes the forgotten frontier.

That’s why this shift — from discovery as a phase to discovery as a product companion — is so critical.

Because in AI, it’s not enough to have working code.

We Need Working Insight

AI products don’t live or die by model performance alone.

They succeed when they solve something real.

They scale when they’re trusted.

They endure when they’re rooted in real understanding of a problem space, a user behavior, or a business gap.

And that understanding doesn’t happen by accident.

So maybe the real question isn’t:

“How do we get people to care about discovery?”

Maybe it’s:

“How do we make discovery look like an asset —

and deliver like one?”

That’s the shift I believe we need.

Not more discovery slides. Not more workshops.

But more problem understanding — embedded in the way we build.

And more building — designed to surface real insight.

So the next time someone asks you to ship quickly,

ask them what you’re shipping toward.

And if the answer is unclear?

Build something small.

Pointed.

Probing.

And let the discovery begin.

JBK 🕊️

For over a decade as an AI Product Manager, I’ve been navigating these scenarios. Whether building AI products for end-consumers or internal teams, the underlying purpose of AI remains the same: to make things easier, faster, and cheaper. The real challenge isn’t in the promise of AI but in finding the right problems for AI to solve—and doing so efficiently.

In large enterprises, where processes are complex and stakeholders are diverse, discovering AI use cases is rarely straightforward. From my experience, the requests and opportunities typically fall into three distinct scenarios:

1. Scenario 1: Someone believes they know the problem and is confident AI is the solution.

2. Scenario 2: Someone believes they know the problem but isn’t sure if AI is the right fit.

3. Scenario 3: No one recognizes the problem, let alone considers AI as a potential solution.

Of these, Scenario 1 is often the easiest to approach. When stakeholders know you as someone who delivers AI solutions, they proactively reach out to you with their ideas. However, that doesn’t mean it’s without challenges. Scenarios 2 and 3, on the other hand, require deeper exploration, closer alignment with stakeholders, and a methodical approach to uncover the real opportunities AI can address.

A Crucial Mindset: Assume They Are Wrong

Let me share one critical piece of advice, especially for Scenario 1 and 2: always assume stakeholders are wrong in their assumptions. This applies to both their understanding of the underlying problem and their belief that AI is the right solution. While this might sound overly cautious, it’s a mindset that ensures you approach every idea critically and methodically.

Stakeholders often view problems through the lens of their immediate frustrations, focusing on surface-level symptoms rather than the root cause. For instance, a department lead might say, “We need AI to make our chatbot smarter.” On the surface, this sounds reasonable, but upon closer investigation, you might uncover that the real issue lies in outdated or incomplete FAQs feeding into the chatbot. Or perhaps the department lead was actually referring to this root issue but framed it in a way that’s open to misinterpretation. Either way, relying solely on their initial framing can easily send you down the wrong path.

The same caution applies to their belief that AI is the solution. While AI is undoubtedly powerful, it’s not a one-size-fits-all tool. Many challenges can be addressed more effectively—and often more efficiently—using simpler approaches like process optimization, basic automation, or off-the-shelf software solutions. The reality is that stakeholders often perceive AI as a magic bullet without fully understanding its capabilities or limitations. That’s where you come in: to guide them toward realistic, impactful solutions.

Now, I’m not suggesting you dismiss their ideas outright. On the contrary, validating assumptions is a key part of effective collaboration. The way forward is to ask thoughtful, probing questions that help clarify the situation, such as:

• “What makes you think AI is the right solution here?”

• “What data do we have to support this idea?”

• “Is this problem repetitive or pattern-based?”

Digging deeper to validate the stakeholder’s assumption typically takes longer than just asking whether AI is the right fit. If your team’s focus is solely on delivering AI products, your first step should be to determine if the assumed problem can actually be solved by AI. If it can’t, you can politely guide the stakeholder to a different team better equipped to address their challenge.

However, there’s a tricky downside: if the assumed problem turns out to be incorrect, but the actual underlying issue is something AI could solve, you risk losing that opportunity. 🙂 Well, I never said this was an easy world to navigate.

By approaching every proposed idea with healthy skepticism, you ensure only well-founded, high-impact opportunities move forward. This mindset not only prevents wasted efforts but also positions you as a trusted advisor. Stakeholders will appreciate your thoughtfulness and rigor. They recognize that you’re not just executing their requests but carefully aligning solutions with their real needs.

Why This Mindset Matters

Adopting a skeptical mindset isn’t about being difficult or contrarian; it’s about protecting both your resources and your credibility. In the fast-paced, often ambiguous world of enterprise AI, jumping into a solution without validating assumptions can quickly lead to expensive missteps. Pause to critically evaluate the problem and its fit for AI. You will minimize your risk but also elevate the quality of your outcomes.

This mindset is especially important in large organizations, where AI use cases often face scrutiny from multiple angles: leadership wants measurable ROI, technical teams need clear feasibility, and end-users expect seamless solutions. It’s not about saying “no” to every idea but about ensuring the right problem is being solved in the right way.

Lessons from My Experience

In my 10+ years of navigating these scenarios, I’ve learned that skepticism often leads to unexpected breakthroughs. Stakeholders may approach you with a flawed framing of their problem, but asking the right questions can uncover opportunities that no one saw coming. One of the most satisfying moments in this role is when a stakeholder realizes, mid-conversation, that their initial assumption wasn’t quite right—but that together, you’ve identified a problem far more valuable and solvable.

I remember a time when a sales team came to me saying, “We need AI to analyze why we’re losing deals.” After digging in, it turned out they already had plenty of insights into why deals were being lost; the real issue was that sales reps didn’t have an easy way to act on this data in real-time. By reframing the problem, we shifted from building a generic AI analytics tool (is AI needed at all?!?) to developing an AI-powered recommendation engine that suggested the best next steps for reps to take during a deal cycle. The result? Higher adoption, faster decision-making, and a direct impact on revenue.

It’s this kind of journey—from assumption to clarity—that makes skepticism so powerful. Not only do you deliver better solutions, but you also build stronger relationships with stakeholders.

Making It Practical

If you’re an AI Product Manager or working in a similar role, here’s how you can adopt this mindset in a way that’s both efficient and collaborative:

1. Start with Curiosity, Not Criticism: Stakeholders may not articulate their problems perfectly, but they often have valuable context. Use their initial ideas as a starting point, not an endpoint.

   • Example: “That’s an interesting challenge. Can we explore what’s behind it?”

2. Be Transparent About AI’s Strengths and Limits: Educate stakeholders early about what AI can realistically achieve and where it might not be the best fit.

   • Example: “AI works great for repetitive, data-driven problems, but this challenge might be better addressed with process changes.”

3. Collaborate to Reframe the Problem: Use structured methods like workshops, hypothesis-driven discovery, or even simple brainstorming sessions to align on the real issue.

   • Example: “If the problem isn’t the chatbot’s intelligence but the content feeding it, we might need to start there before applying AI.”

4. Don’t Be Afraid to Pivot: If AI isn’t the right solution, guide the stakeholder toward other options. This honesty builds trust and sets the stage for future collaboration.

Final Thoughts

While this article focused on the mindset required for companies and, in particular, AI Product Managers when identifying the right problems for AI, there are plenty of techniques—like Hypothesis-Driven Discovery, Value Stream Mapping, Design Thinking, Job Shadowing, or Process Mining—that can help boil down to root problems and at the same time discover if AI might help. Each of these fits better with specific scenarios introduced here, and not all are necessary. But knowing what tools and methods you can leverage makes Product Discovery easier and much more structured.

Once you’re ready, I’ll write about those methods, showing how they can enhance your process. For now, let’s focus on building that mindset—it’s the foundation for everything else.

JBK 🕊️

P.S. If you’ve found my posts valuable, consider supporting my work. While I’m not accepting payments right now, you can help by sharing, liking, and commenting here or on my LinkedIn posts. This helps me reach more people on this journey, and your feedback is invaluable for improving the content. Thank you for being part of this community ❤️.

Thank you for being part of this journey ❤️.

#beyondAI - I believe time-to-market is critical for every company, whether a scrappy startup or a massive multinational corporation. For both, the speed at which they move from concept to launch determines not only how quickly they gain insights from the market but also how fast they start generating revenue.

It’s a simple equation: the faster you get your product into customers' hands, the sooner you can iterate, adapt, and deliver something even better.

To reduce time-to-market, the first step is understanding the actual processes and workflows that companies have in place to deliver.Let’s take a closer look at the typical process for delivering software, whether as a product or a service.

Most companies follow a structured process for product development, one that has evolved over time into what is often considered the standard. But let’s be honest - just because it’s the standard doesn’t mean it’s the best. And we’ll come back to that later.

These internal software processes are usually aligned with the Software Development Lifecycle Process (SDLP), which consists of six stages:

The Six Stages of the Software Development Lifecycle Process (SDLP)

1. Idea or Demand Generation

   This is where it all begins. A need for a product or service is identified, often driven by customer feedback, market opportunities, internal optimization goals, or regulatory requirements. The main goal here is to clearly define the problem and assess the potential impact of solving it.

2. Concept Validation

   At this stage, the idea is evaluated for feasibility. Teams assess whether the solution aligns with company objectives, whether the required resources are available, and if the market is ready. This phase often includes developing a preliminary business case and going through the initial approval processes.

3. Design

   This is where the idea takes shape. Teams create detailed specifications, wireframes, or prototypes to define what the product will look like, how it will function, and how it will integrate with other systems. The focus here is to provide a clear blueprint for the development phase.

4. Development

   This is the stage where the product is actually built. Developers write code, create features, and integrate systems. Whether it’s done in agile sprints or using a traditional waterfall model, this phase is all about execution.

5. Testing

   Before the product is released, it undergoes thorough testing to ensure it meets quality, security, and functionality standards. This phase includes unit testing, integration testing, and user acceptance testing (UAT). It’s where bugs are identified and resolved.

6. Delivery or Launch

   Finally, the product or service is released to the market or end users. This includes deployment, marketing efforts, and initial customer support to ensure everything runs smoothly. Feedback loops are typically established during this phase to gather early user insights.

Each of these stages is then tailored to fit an organization’s specific requirements and governance structures. And here’s where it gets tricky. These company-specific adaptations often inflate the SDLP in ways that slow it down over time, regardless of whether the process follows a waterfall, iterative, or agile model. While the SDLP in its raw form is relatively lean, organizational adjustments can make it cumbersome, leading to various pain points at the process level.

Process-Level Pain Points in Time-to-Market

Some common process-level pain points that stem from how processes are adapted to organizational needs:

• Excessive Approvals: Endless layers of sign-offs can turn what should be quick decisions into weeks of delays.

• Rigid Governance: Processes designed to ensure quality often end up becoming the biggest hurdles to speed and agility.

• Misaligned Goals Across Teams: Different departments pursuing conflicting priorities can create bottlenecks and delays.

• Inefficient Resource Allocation: Waiting on approvals for funding, personnel, or tools can completely stall progress.

These pain points stack up over time. Honestly, I don’t know a single team that hasn’t complained about them. It feels like the natural evolution of companies:

What starts as a framework for consistency and quality eventually turns into a maze of inefficiencies that drag time-to-market to a crawl.

Now, many companies tackle these pains, often investing heavily to identify where processes need streamlining and how to overcome inefficiencies. These efforts typically focus on the process level, which isn’t wrong—it can, in fact, significantly reduce time-to-market. However, focusing solely on processes is only part of the solution. There’s another critical area to optimize: workflows.

Optimizing Where the Real Work Happens

Workflows are the specific activities carried out to complete a task within a process step. While processes define the overarching structure, workflows deal with the hands-on execution of tasks. Optimizing workflows is just as important as streamlining processes, yet it’s often skipped.

This isn’t because companies don’t want to address workflows. It’s because those responsible for optimizing processes are usually strategic, high-level thinkers who lack a detailed understanding of the specialized workflows used by domain experts. These are people focused on the bigger picture—governance, approvals, or resource alignment—rather than the granular tasks being done on the ground.

For workflows to be optimized, the drive must come from within, from the very core of where tasks are actually executed. It’s about empowering the teams and individuals who know these workflows best to identify inefficiencies and implement changes. This type of optimization requires collaboration between high-level strategists and domain experts to bridge the gap between processes and practical execution.

AI Superpower Reveals at the Workflow-Level

As we entered the digitalization era, companies, guided by expert advice, began introducing new software to make workflows more efficient. This was followed by automation technologies like RPA (Robotic Process Automation) and scripting, which automated even more tasks. But then, progress seemed to plateau.

Now, with the emergence of Generative AI, we’re standing on the brink of a new wave of transformation—automation, or at the very least, expert augmentation, that can make workflows even more efficient.

This potential, however, comes with a caveat:

Optimizing workflows with GenAI isn’t about simply applying the technology wherever possible. It requires a deep understanding of routines and workflows at the working level. You can’t just plug it in and expect miracles—you have to figure out where it truly fits.

Every GenAI initiative aimed at reducing time-to-market, especially in the context of software development, must be developed in collaboration with Subject Matter Experts (SMEs) who own those workflows. These are the people who live and breathe the daily tasks. They know exactly where bottlenecks are, what areas can be improved, and what must remain untouched.

For internal AI Product Managers, SMEs are not just collaborators—they’re key stakeholders, or even better, customers in the process. AI Product Managers need to work closely with SMEs to ideate, explore, and assess whether parts of a workflow can be augmented with AI or, in some cases, fully automated.

When this deep collaboration succeeds, it opens the door to endless possibilities. AI use cases across the software development lifecycle start to reveal themselves, each one holding the potential to reduce inefficiencies and speed up the process.

How I Create AI Use Cases to Reduce Time-to-Market

Let me walk you through how I typically proceed when creating an idea to support reducing time-to-market in software development. The process is structured but flexible enough to adapt to different organizational needs:

1. Understand the Overall Process (e.g., SDLP)
   Start by mapping out the entire process you’re focusing on. In software development, this could be the Software Development Lifecycle Process (SDLP). Understand how the stages connect and identify where bottlenecks or inefficiencies might occur.

2. Understand Which Workflows Typically Exist Within a Process Step (e.g., Concept Validation)
   Dive deeper into a specific process step. For example, in the Concept Validation stage, identify the workflows involved, such as defining requirements, validating feasibility, or aligning stakeholders.

3. Ideate on Potential AI Use Cases for Specific Activities Within a Workflow
   Once the workflows are clear, focus on individual activities within them. For example, if a workflow involves gathering customer requirements, think about how AI could assist—perhaps through a Generative AI model that analyzes historical data to pre-fill requirements or suggest templates.

4. Make a Rough Estimate About the Benefit of a Potential AI Use Case
   Evaluate the impact of your ideas. Would automating a task save time, reduce errors, or free up resources? Estimate potential gains in terms of time savings, cost reduction, or improved output quality.

5. Prioritize the AI Use Cases and Find the Key Stakeholders
   Not all ideas will have the same impact, so rank them by potential value and feasibility. Then identify the stakeholders—usually SMEs or team leads—who are critical to validating the idea and implementing the solution.

6. Approach the Stakeholders to Validate My Assumptions
   Present your prioritized ideas to the stakeholders. Share your assumptions about the benefits and feasibility and gather their feedback to refine the use cases. Their input is essential for making realistic plans.

7. Get Them on Board and, If Agreed, Start the AI Product Journey
   Once stakeholders see the value and agree to proceed, bring them on board as collaborators. This is where the actual AI product journey begins—from proof of concept to full implementation.

Sure, you can adapt this approach to fit your unique situation. Every company is eager to find ways to improve time-to-market, and maybe now you can take the lead by applying a similar approach. 🚀

Final Thoughts

When I first started writing this article, I planned to simply list AI use cases for improving time-to-market. However, I quickly realized that the approach might not be very helpful. Instead, I decided to share how I think about processes, workflows, and tasks—and how I use this mental model to identify impactful AI use cases. My hope is that this resonates with you more than just a generic list ever could.

Here’s what I’ve touched on in this article:

1. Time-to-Market is Critical: In today’s fast-paced world, speed isn’t optional—it’s the difference between staying relevant or falling behind.

2. Processes and Workflows Are Different: Addressing inefficiencies at both levels is key to achieving meaningful results.

3. AI Unlocks New Possibilities: Generative AI is a game-changer, enabling smarter automation and augmenting expertise in entirely new ways.

4. Collaboration is Key: The best solutions emerge when internal AI Product Managers and SMEs work together to uncover and implement impactful AI use cases.

5. A Structured Approach Works: By following a clear, step-by-step process, you can ensure your AI initiatives are both practical and valuable.

And here’s something even more important to keep in mind: stop positioning AI as just a tool for cost reduction. That narrative is tired. Instead, highlight its potential to accelerate time-to-market—because that’s where the real competitive edge lies.

Sadly, this perspective is often overlooked. Too many companies see AI solely as a way to save money. But imagine what could happen if they realized how AI could help them get to market faster. It might just change everything.

Maybe.

JBK 🕊️

A quick note: I see the stats showing people are opening my newsletters, but it’s hard to tell if they’re truly resonating or delivering value. I would really love to hear from you — your thoughts, feedback, or even just a quick comment. 

Thank you for being part of this journey ❤️.

In the corporate world, the call for more structure and governance in managing AI initiatives is growing louder—and for good reason. Over the past two decades, we’ve seen companies enthusiastically dive into AI, hiring experts left and right. Some approached this strategically, while others took a more scattershot approach. The results? Varied, to say the least.

AI teams across organizations have been busy building models: some for internal use, others to enhance their company’s core services or products. But here’s the common thread—most have little to no visibility into what their counterparts in other teams are working on. It’s not uncommon for one team to discover—by pure accident—that another department is building the exact same thing, just for different stakeholders.

This kind of overlap isn’t a minor annoyance; it’s a serious problem. When teams unknowingly duplicate work, they multiply costs without multiplying value. Worse, they lose time—time that could have been spent improving or scaling existing solutions.

And here’s the heart of the issue: AI models, much like any other corporate asset, need to be managed with clear ownership, transparent value tracking, and robust usage guidelines.

When companies don’t think of AI as part of a broader portfolio, they miss out on the opportunity to scale. Anyone who’s been involved in building a model knows how resource-intensive it is. It’s a heavy lift, and once that lift is complete, why wouldn’t you maximize its impact?

The ideal scenario is this: build a model once and adapt it for multiple use cases with minimal effort. That’s scalability. But the reality? It’s far from this ideal.

Take enterprise IT departments as our grown-up cousins, for instance. Most large companies understand that maintaining a well-organized software portfolio isn’t a luxury—it’s business-critical. Why not borrow from these established practices? Why not take what works in IT and adapt it to AI?

This is why I’m making the case for every company with a serious investment in AI to introduce AI Portfolio Management. It’s not rocket science. The tools, processes, and frameworks we need are already at our disposal. The challenge isn’t invention—it’s adaptation and integration. We don’t need to overhaul entire systems. We just need to introduce one extra step into daily workflows—a step that saves countless others down the line. Seems like a good trade-off, doesn’t it?

For AI Product Managers, this topic is equally relevant. While this article is written for companies at large, it’s packed with insights to help you as an individual contributor. Whether or not your organization already has portfolio and inventory management in place, understanding these concepts empowers you to:

• Manage your AI products more effectively.

• Capture key details to contribute to a future portfolio.

• Align your work with broader organizational goals—and prepare for the moment your work becomes part of something bigger.

Happy reading 🛋️

What Is AI Portfolio Management?

It’s easy to have different associations when we talk about a portfolio. For me, at the beginning of my own portfolio journey, the term conjured up multiple images:

• A stock market portfolio—assets reflecting financial worth.

• An artist’s portfolio—a curated collection showcasing their skills.

• A job application portfolio—a selection of your best work proving your capabilities.

What ties all these examples together is a common theme: they’re all curated collections that reflect value. They aren’t random assortments—they’re purposeful, intentional, and designed to give a clear picture of worth.

This is exactly how we should think about an AI portfolio in a corporate context. It’s not just a list of all the models your teams have built or the AI initiatives underway. It’s a curated collection of assets that reflect the value your organization is generating with AI.

And, just like a stock portfolio or an artist’s portfolio, an AI portfolio needs to be actively managed.

Active management means making tough decisions:

• Which models are worth further investment?

• Which should be retired or repurposed?

• What new initiatives will maximize resources and align with company goals?

Every portfolio faces limiting factors:

• For the artist, it’s the finite space on a gallery wall.

• For the broker, it’s the amount of money available to invest.

• For the corporate AI space, it’s budgets, technical expertise, data availability, and more.

To make your AI portfolio as valuable as possible, you need to navigate these constraints wisely.

There are countless reasons to establish an AI Portfolio Management function. To make it easier for you to decide whether to implement this function—or not—I’ve compiled a list of the key benefits I’ve come across:

1. Strategic Alignment

• Benefit: Align AI investments with business goals. Identify and prioritize AI projects that drive measurable outcomes like cost reduction, revenue growth, or customer satisfaction, ensuring alignment with strategic business objectives.

2. AI Use Case Portfolio Optimization

• Benefit: Avoid redundancy and maximize value. Evaluate AI use cases to identify overlapping efforts, such as multiple teams working on similar models or data sets, and consolidate efforts to optimize resources.

3. AI Risk Management

• Benefit: Minimize risks in AI deployment. Track AI model performance, compliance with regulations (e.g., GDPR, AI Act), and ethical risks like bias or unintended consequences to proactively address issues.

4. Cost and Resource Optimization

• Benefit: Manage AI investments for maximum ROI. Provide visibility into the costs of training, deploying, and maintaining AI models and identify areas to reduce computational expenses or improve operational efficiency.

5. Improved Decision-Making

• Benefit: Data-driven prioritization of AI initiatives. Use key performance indicators (KPIs) like accuracy, adoption rate, or time-to-value to make informed decisions about continuing, pivoting, or stopping AI projects.

6. Enhanced AI Governance

• Benefit: Ensure compliance, ethical AI use, and accountability. Establish governance frameworks to monitor AI model usage, ensure fairness, and mitigate risks, while providing transparency into AI decision-making processes.

7. Supporting AI Transformation

• Benefit: Facilitate company-wide AI adoption. Plan and coordinate the rollout of AI initiatives across departments, ensuring infrastructure readiness and aligning transformation efforts with business needs.

8. Enterprise-Wide AI Architecture

• Benefit: Maintain a unified and scalable AI infrastructure. Design a robust AI architecture that integrates with existing IT systems and supports data pipelines, model lifecycle management, and deployment workflows.

9. Enhancing Collaboration Across Teams

• Benefit: Foster alignment between data scientists, engineers, and business teams. Provide a shared platform or repository for tracking AI models, datasets, and results to improve collaboration and reduce silos.

If you carefully went through this list, you might have realized that establishing a powerful AI Portfolio Management function requires one critical foundation: bringing all AI teams across the organization together to document their new, in-development, and live AI products in one centralized place. This step is essential to enable that new function to work effectively.

And yes, I’m not saying this is an easy endeavor. It’s challenging, time-consuming, and requires commitment across all levels of the organization. But it’s necessary.

I firmly believe that every disruptive technology—and by now, I think we can all agree that AI falls into this category—requires an organizational and processual change to reveal its true potential. Without this shift, we’ll only see the power of AI at the tech level, never fully realizing its impact on the business side.

You’ll know your efforts have been successful once you’ve built and maintained a robust AI Inventory. Only with an AI Inventory can your AI Portfolio truly unfold the benefits I’ve outlined above.

What Is an AI Inventory?

The word inventory takes me back to my school days when some of us worked part-time at a large grocery store during the summer. The task was simple: inventory. At least, that’s what they called it. What it really meant was counting every item on the shelves, documenting what was there, and noting what needed restocking. It was tedious, but also revealing. You’d discover that some products were flying off the shelves while others were collecting dust in the corner.

This is the essence of an inventory: it’s a systematic record of what you have. And while it may sound basic, having an accurate inventory is critical to making informed decisions. In retail, it helps manage stock levels. In AI, it’s a dynamic map of your AI ecosystem, detailing every asset—what it’s for, who owns it, how often it’s used, and how it might be repurposed or reused. Without this map, portfolio management is like trying to curate an art exhibit without knowing which pieces are in storage.

An AI Inventory provides visibility into the organization’s AI landscape. It ensures that every AI product and use case is accounted for, enabling the Portfolio Management function to make strategic decisions with confidence. Without an accurate and well-maintained inventory, AI efforts can quickly become fragmented, redundant, or misaligned with business goals.

So what exactly should be captured in an AI Inventory to support Portfolio Management? Here's a checklist to ensure your inventory captures everything it needs:

Key Details to Capture for Each AI Product & AI Use Case

1. General Information

   • Name of the AI Product or Use Case

   • Description of its purpose and functionality

   • Status (e.g., conceptual, in development, live, retired)

2. Ownership and Responsibilities

   • Owning team or department

   • Product Owner or main point of contact

   • Data Owner (if applicable)

3. Business Context

   • Problem it solves or opportunity it addresses

   • Stakeholders involved (internal and external)

   • Business unit(s) it supports

   • Expected business impact (e.g., cost reduction, revenue growth, efficiency improvements)

4. Technical Details

   • Type of AI model (e.g., regression, classification, generative)

   • Underlying technology or framework (e.g., TensorFlow, PyTorch)

   • Data sources used for training

   • Deployment environment (e.g., cloud, on-premises)

5. Performance Metrics

   • Key performance indicators (KPIs) for the model

   • Accuracy, precision, recall, or other relevant metrics

   • Current performance levels

6. Reusability Potential

   • Similar use cases or models it could support

   • Level of customization required for reuse

   • Dependencies or prerequisites for reuse

7. Lifecycle Management

   • Date of creation or deployment

   • Maintenance schedule

   • Version history

8. Compliance and Governance

   • Data privacy considerations (e.g., GDPR, CCPA compliance)

   • Ethical considerations (e.g., bias assessments)

   • Approval processes it has undergone

9. Cost and Resources

   • Development costs (time and budget)

   • Operational costs (e.g., compute resources, licensing fees)

   • Resource allocation (team members involved)

10. Usage and Impact

    • Frequency of use

    • Business outcomes achieved (quantitative and qualitative)

    • Feedback from end users

Final Thoughts

I’m pretty sure there’s more that could be tracked in an AI inventory, and the list will likely evolve. But I believe this is a solid starting point. And trust me, implementing even this foundational structure will already be challenging enough.

That said, I firmly believe unlocking the full potential of your organization’s AI initiatives requires more than technical innovation. It goes beyondAI. It demands organizational and processual change. The sooner companies embrace this, the sooner the benefits will materialize.

For AI Product Managers, this guide highlights the details you need to capture when building an AI product—even in the absence of formal portfolio or inventory management.

Because one thing is certain: someday, the CEO will ask, “Who can tell me how much value we’ve generated from our AI initiatives?” And in that moment, you’ll want to be the one still sitting confidently in your seat, ready to report not only on your products but on their contribution to the company’s overall success.

So, why not start now?

JBK 🕊

P.S. If you’ve found my posts valuable, consider supporting my work. While I’m not accepting payments right now, you can help by sharing, liking, and commenting here or on my LinkedIn posts. This helps me reach more people on this journey, and your feedback is invaluable for improving the content. Thank you for being part of this community ❤️.

This peer came from a deeply technical background. He was a computer scientist, completely immersed in algorithms, and obsessed with accuracy metrics. He believed that if you could build the best solution from a technical perspective, that was all that mattered. He even spent years researching Artificial Intelligence, dedicated to developing NLP systems that could push boundaries. But now, he was a Product Manager in AI—navigating a space that demanded much more than just technical skills.

Yet, he still approached the challenges with a laser focus on technology. He was always thinking about the smartest algorithms, the highest accuracy, and the most elegant code. We’d be in meetings, and his mind would jump straight to technical solutions, even when we were supposed to be talking about user experience or business needs. I watched him spend hours refining models with his data scientist when a simpler approach could have done the job just as well. Watching him reminded me so much of my own journey.

And in truth? The peer I was advising was me.

When I first transitioned to AI Product Management, I thought my technical expertise was my most valuable asset. And in many ways, it was. But it also became my biggest blind spot. I believed that if I could just architect the best solutions, everything else would naturally fall into place. I had to learn the hard way that being too technical can be just as limiting as not being technical at all.

This post is for all my technical peers out there who are making the leap into Product Management. And for those who have already made that leap, maybe you need to hear this right now too.

"Hi, my name is Jaser, and I was a 'too-technical' PM."

Happy reading 🛋️

What It Means to Be "Too Technical" as an AI Product Manager

As AIPMs, we often wear our technical expertise as a badge of honor. And it’s true—understanding the tech is super important. But here's the thing: if we focus too much on the technical side, we can lose sight of what the user and business really need. Let me share a few lessons I learned along the way:

Pitfall 1: Over-Engineering

At first, I was always striving for technical perfection, aiming to use the most cutting-edge algorithms and fine-tuning models to the smallest details. I wanted every solution to be flawless. But I quickly learned that over-engineering can slow down progress. The business didn't need perfection; it needed solutions that worked well enough to deliver value—and deliver it quickly.

Sometimes, a simpler approach could do the job instead of creating a complex neural network. My urge to use the latest tech often led me to build more than necessary. You can imagine what this meant: it delayed the path to MVP and wasted resources.

Pitfall 2: Losing Sight of the End-User’s Problem

My technical mindset often made me jump to how before I fully understood why. I was always so eager to start building a solution, even before I truly understood the user’s pain points. Over time, I learned that empathy is where effective AIPMs need to begin. You need to put yourself in the user's shoes and really see the problem through their eyes. Only then can you ensure that every solution truly addresses the real needs of the end-user.

Users might ask for a more intuitive search function, and my technical mind would push me toward building a sophisticated NLP model. But sometimes, all that was needed was an enhanced keyword search with some simple filters, which could have been delivered faster and with less effort.

Pitfall 3: Paralysis by Metrics and Data

Data was both my comfort zone and my downfall. I always felt uneasy making decisions unless every single metric was perfectly aligned. Have you ever been stuck in the data spiral, waiting for the perfect answer? You can probably guess what happened—I kept waiting for more data, analyzing, and then overanalyzing. And, as you can imagine, decision-making dragged on, and we ended up missing opportunities. Eventually, I had to face the truth: sometimes, you just need to trust your gut and be willing to test ideas quickly instead of waiting for absolute certainty. This is especially tough for those of us with a technical background because it goes against everything we were taught.

Pitfall 4: Over-focusing on Scalability Too Early

Coming from a technical background, I was obsessed with making sure every solution could scale for millions of users. And here’s the funny part—most of the time, the market wasn't even that big, but my technical mind couldn't let go of this obsession. Scalability is important, sure, but focusing on it too early led me to complicate the architecture for no good reason. I wasted so much precious time when what we really needed was just a simple MVP to validate the idea first and gather real user feedback.

In one case, instead of creating a simple backend that could handle a small user base and iterating from there, I had the wild idea of a distributed microservices architecture that ultimately would have delayed our go-to-market timeline. Thankfully, my more senior developer in the team—who would have made a brilliant Product Manager—steered us in a better direction. Scalability can come later—first, you need to prove that users even want what you're building.

Pitfall 5: Underestimating the Importance of Non-Functional Requirements

At first, I thought that if the core AI model worked well, everything else would just fall into place. But believe me, I even had to learn that non-functional requirements are crucial too, funny but the truth. Things like usability, maintainability, and ease of integration—they're just as important as the core functionality itself. Ignoring these aspects meant we ended up with 'products' that might have been impressive from a technical standpoint, but they were a nightmare to use or integrate. And honestly, can we even call them products if they can't be used? If it can't be used, it's not a real solution. And only real solutions have the potential to grow into successful products. Read more on this idea in the link below.

I worked on an AI-driven tool where I poured all my effort into creating a highly accurate recommendation engine. It felt like a huge win to get those metrics looking perfect. But then came the real challenge—integrating it with our client’s legacy system. Let me tell you, building the model was the easy part. Making it work seamlessly with old, complex infrastructure? That was a completely different beast. It’s something I wish I had thought more about from the beginning, especially in a large enterprise environment.

Why Letting Go (Just a Little, and sometimes a little more) Matters

Now that I've shared some examples of my mistakes, I truly believe they're closely tied to my technical mindset, my education, and my experiences as a data scientist and developer. But you know what? Reflection is what really helps me uncover my blind spots. I have this habit—sometimes it's good, sometimes not so much—of reflecting on almost every aspect of my life. It helps me grow, though I'll admit, it can also lead me to overthink things. And you can google (or chatgpt) to read that overthinking isn't that good. But in this particular case, reflection really helped me understand why I acted the way I did, and it ultimately helped me develop strategies to find a better balance.

As AIPMs, our mission is to deliver value and solve real problems. I think we can all agree on that now.

Technical expertise is definitely an asset as an AIPM, but it's only part of what makes us truly effective. The real impact, I found, came when I stepped back from the technical side and learned to balance it with user insights, business goals, and a willingness to prioritize progress over perfection. Once I understood that there are six key dimensions I need to focus on as an AIPM to even have a chance at success, my mindset shifted—from a purely technical one to what people often call a product mindset.

I've written an entire article about those six dimensions, and you might want to give it a try. It was my key to 'letting go.' It's about deeply understanding the interplay between Data, AI, and IT on one side—the technical trio—and Business, Governance, and People on the other—the strategic trio. If you're only strong on one side, you won't find the balance you need.

The truth is, that being an AI Product Manager means walking a fine line. Being too technical can be just as ineffective as lacking technical depth altogether. Finding that balance was key for me, and maybe it can be for you too.

And I hope you get what I mean by 'letting go.' Sometimes, you have to let go first to make room for something new. Often, that 'something new' turns out to be even better. In this case, it's about letting go of that purely technical mindset—at least for a while—so you can make space for a new, broader mindset to grow.

Where could you benefit from letting go a bit, to create that space for something new?

JBK 🕊️

P.S. If you’ve found my posts valuable, consider supporting my work. While I’m not accepting payments right now, you can help by sharing, liking, and commenting here or on my LinkedIn posts. This helps me reach more people on this journey, and your feedback is invaluable for improving the content. Thank you for being part of this community ❤️.