17 posts tagged with "Product Management"

When my team started working on the AI feature, I was mostly concerned with writing clear, precise prompts. As shown, maintaining them throughout the development and operational lifecycle becomes a larger, unanticipated problem.

You will learn which issues with updating prompts you might encounter when you push your AI agent from MVP to Production. Below, I am sharing my findings on the operational side of the prompt management.

That title isn't clickbait: it's what actually happened. Over the last three months, I've spent 90% of my time obsessing over whether our AI search returns the right results. Not writing PRDs. Not in stakeholder meetings. Just testing, scoring, and re-testing AI outputs.

Only recently did I learn that there is a trending term for what I am doing: AI evals. I thought that was "quality assurance" or "testing". But AI Evals is a sexier name for selling courses.

Jokes aside. The main reason the industry began separating it from traditional QA frameworks is the non-deterministic nature of the output. Different results from the same input were considered as the definition of madness.

Now it is our new reality.

Introduction: The Shift in My PM Role​

As a Product manager, I am building proofs-of-concept (POC) and constantly hunting for context.

That's what my role is about now. I am reluctant to create roadmaps, write any required documents, and manage the backlog in Jira.

Previously, writing those requirements docs, sitting on mockups to ensure they matched the requirements, writing user stories, and many other related tasks were part of the creative process.

I enjoyed that. That was a reason I kept going in my craft for now more than a decade: from an L1 support engineer to a Senior product manager.

Now I am looking and constructing something that a "team can build in the near future." Everything else stepped aside. I don't enjoy it anymore.

This article is a summary of my PM experience on designing an artificial intelligence (AI)-based capability for an existing product from the proof-of-concept (POC) stage to a minimal viable product (MVP).

When you are adding an AI feature to an existing product, you can either:

• introduce an entirely new functionality
• extend existing functionality with AI possibilities.

In my case, the latter is true: we aim to introduce AI search to enhance user experience. It sounds pretty trivial, I acknowledge that. The challenge is that search is the critical functionality for that product.

Therefore, we need to introduce a new search method to provide users with more value than the existing one. If AI search works "just good enough," then a traditional one could be considered a failure.

The improvement must be substantial, because of …

The higher you climb the product manager ladder, the more noise gathers around you.

You hear multiple ideas and suggestions for improvements from various people. And that doesn't make much sense in the current picture. It would be nice to do in an ideal world, but we are too far from that.

There is a tendency for companies to lay off people but don't cut down their expectations, trying to operate as if nothing had happened and the business goes as usual.

They likely hope for AI to resolve that inconsistency between dreams and reality. But that is a gamble at the moment. It might pay off for some, but hurt others.

Should we let users know about upcoming features? Can it make them angry if you tease them and then underdeliver?

Let's find out.

There is much buzz about whether AI could replace human product managers. I am not very concerned. Much of my work involves dealing with stakeholders at different levels and pursuing various objectives. AI will find it challenging to substitute that side of people interaction. I mean politics in a sense of power dynamics among multiple groups of stakeholders.

We don't like politics in the workplace, but it is inevitable when there are more than a dozen people.

Dealing with internal or external stakeholders always involves politics and personal-professional relationships. The larger the organization, the more complex the political landscape and the bigger the stakes.

The power dynamics impact everyone in the organization, including individual contributors. When an individual contributor (IC) transitions to a manager role, they become a part of the game without any choice.

For future plans, please don't create one-liner tickets in your backlog.

The only excuse is when you must "proof" some future work. I knew a project manager in an outsourcing project who told business analysts to create 500 Jira issues to showcase to a customer the scope of work for continuing a development team's assignment.

In recent years, I have been dealing with massive backlogs of outdated items written by people no longer around. That has resulted in painful and time-consuming reviews and (re)-negotiations with stakeholders.

If you have such experience at least once, you are likely a fan of focused and precise backlogs like myself.

Onboarding is a crucial process of incorporating a new person into a team, product, or domain (for simplicity, let's generalize all that as "area"). The purpose is to align or increase performance as soon as possible after the unavoidable drop due to a structure change. If approached wrongly, it may cause substantial trouble, especially when you dive into a new area as a product manager (PM).

Whether it is a new or same organization, new role, or new area - the approach to onboarding will vary. Here, we talk about horizontal growth within an organization, keeping your role but acquiring a new "area" (for simplicity, let's use this term to generalize one or several products, teams, lines of business, etc). That is the situation I am currently in, so I have something to reflect on. Also, most onboarding guides focus on vertical growth with a new role in another business. So, it is a good topic to uncover.

I follow the patterns of 3 key knowledge types in product management suggested in the "Building Products for Enterprise" book (I wrote about it here): organizational, product, and market knowledge. So, let's construct the onboarding strategy for each.

I finished reading "TRANSFORMED: Moving to the Product Operating Model" by Marty Cagan. This is not my usual a few thousand-word review—rather, it is a few aftermath thoughts.