3 posts tagged with "AI"

That title isn't clickbait: it's what actually happened. Over the last three months, I've spent 90% of my time obsessing over whether our AI search returns the right results. Not writing PRDs. Not in stakeholder meetings. Just testing, scoring, and re-testing AI outputs.

Only recently did I learn that there is a trending term for what I am doing: AI evals. I thought that was "quality assurance" or "testing". But AI Evals is a sexier name for selling courses.

Jokes aside. The main reason the industry began separating it from traditional QA frameworks is the non-deterministic nature of the output. Different results from the same input were considered as the definition of madness.

Now it is our new reality.

Introduction: The Shift in My PM Role​

As a Product manager, I am building proofs-of-concept (POC) and constantly hunting for context.

That's what my role is about now. I am reluctant to create roadmaps, write any required documents, and manage the backlog in Jira.

Previously, writing those requirements docs, sitting on mockups to ensure they matched the requirements, writing user stories, and many other related tasks were part of the creative process.

I enjoyed that. That was a reason I kept going in my craft for now more than a decade: from an L1 support engineer to a Senior product manager.

Now I am looking and constructing something that a "team can build in the near future." Everything else stepped aside. I don't enjoy it anymore.

This article is a summary of my PM experience on designing an artificial intelligence (AI)-based capability for an existing product from the proof-of-concept (POC) stage to a minimal viable product (MVP).

When you are adding an AI feature to an existing product, you can either:

• introduce an entirely new functionality
• extend existing functionality with AI possibilities.

In my case, the latter is true: we aim to introduce AI search to enhance user experience. It sounds pretty trivial, I acknowledge that. The challenge is that search is the critical functionality for that product.

Therefore, we need to introduce a new search method to provide users with more value than the existing one. If AI search works "just good enough," then a traditional one could be considered a failure.

The improvement must be substantial, because of …