That title isn't clickbait: it's what actually happened. Over the last three months, I've spent 90% of my time obsessing over whether our AI search returns the right results. Not writing PRDs. Not in stakeholder meetings. Just testing, scoring, and re-testing AI outputs.

Only recently did I learn that there is a trending term for what I am doing: AI evals. I thought that was "quality assurance" or "testing". But AI Evals is a sexier name for selling courses.

Jokes aside. The main reason the industry began separating it from traditional QA frameworks is the non-deterministic nature of the output. Different results from the same input were considered as the definition of madness.

Now it is our new reality.

Introduction: The Shift in My PM Role​

As a Product manager, I am building proofs-of-concept (POC) and constantly hunting for context.

That's what my role is about now. I am reluctant to create roadmaps, write any required documents, and manage the backlog in Jira.

Previously, writing those requirements docs, sitting on mockups to ensure they matched the requirements, writing user stories, and many other related tasks were part of the creative process.

I enjoyed that. That was a reason I kept going in my craft for now more than a decade: from an L1 support engineer to a Senior product manager.

Now I am looking and constructing something that a "team can build in the near future." Everything else stepped aside. I don't enjoy it anymore.

This article is a summary of my PM experience on designing an artificial intelligence (AI)-based capability for an existing product from the proof-of-concept (POC) stage to a minimal viable product (MVP).

When you are adding an AI feature to an existing product, you can either:

• introduce an entirely new functionality
• extend existing functionality with AI possibilities.

In my case, the latter is true: we aim to introduce AI search to enhance user experience. It sounds pretty trivial, I acknowledge that. The challenge is that search is the critical functionality for that product.

Therefore, we need to introduce a new search method to provide users with more value than the existing one. If AI search works "just good enough," then a traditional one could be considered a failure.

The improvement must be substantial, because of …

The higher you climb the product manager ladder, the more noise gathers around you.

You hear multiple ideas and suggestions for improvements from various people. And that doesn't make much sense in the current picture. It would be nice to do in an ideal world, but we are too far from that.

There is a tendency for companies to lay off people but don't cut down their expectations, trying to operate as if nothing had happened and the business goes as usual.

They likely hope for AI to resolve that inconsistency between dreams and reality. But that is a gamble at the moment. It might pay off for some, but hurt others.

There are complex issues at many organizational and technical layers, whose resolution requires considerable planning and cross-team effort. Individuals who possess the ability to resolve such matters must be valuable to an organization.

However, with recent changes in the employment market and the rise of AI, that is no longer relevant to you personally. Let me explain what I mean here.

Any stakeholder management framework is ineffective if there is no trust between you, as a product owner, and your stakeholders. Trust is challenging to gain, resourceful to maintain, and very easy to lose.

For stakeholders, you represent your team, your product, and your organization. If they don't trust you and consider you an unreliable partner, then it is a chain effect with a downward spiral.

How to build and then maintain that trust? It is straightforward - deliver expectations. If you can't deliver, then manage those expectations.

Should we let users know about upcoming features? Can it make them angry if you tease them and then underdeliver?

Let's find out.

While reading Julius Caesar’s biography by Adrian Goldsworthy, I learned a surprising fact: he funded his political career as a startup would today.

Caesar was from a very noble patrician family. However, at the moment of his birth, his family was not incredibly rich and powerful compared to other Roman nobility. They have significant connections, but that alone would not be enough to build an outstanding political career.

The Roman political system was not characterized by parties but rather by individuals. Those individuals spent a large amount of their money to gain popularity among the citizens, support their lifestyle, engage in what we today consider bribery, and so on.

The Temptation to Say No​

I saw on LinkedIn a rare opportunity for 2025: a vacancy for a Junior BA. I wondered whether hiring someone for such a junior position makes sense, considering the current level of AI advancement.

My immediate answer is ^^NO^^: as a Product Manager managing a portfolio of products, AI could do the same work as a junior Business Analyst does. And sometimes, with a better quality and less involvement of someone who needs to guide that person.

Let's discuss the statement above, because the answer is not as simple as it may seem.

Composable Definition and MACH​

Composability, also known as Composable solutions or Composable commerce, is a design principle in which individual components or services are modular, independent, and interoperable. It allows them to be easily combined, reused, or replaced to build complex systems.

We have a custom-built core product for software businesses, whether a monolith or a set of microservices. In addition, you (when I mention you, I mean a product, area, or organization) have several external integrations with different levels of complexity.

For non-software businesses, several vendors of different scales have some in-house solutions. Those businesses are the primary beneficiaries, as Composable is a way to eliminate vendor lock-in and gain more control while also being flexible enough to adapt to market changes.

Composable addresses the latter case by using "best-of-breed" components available, whether market or house-built and adapting them according to business requirements.

To build such a miraculous system, you need middleware that connects several headless subsystems via API and provides governance and data consistency.