Before launching the AI function, the product manager must first get three critical things fully clarified through proper inquiries.

"We need to launch an AI feature too" —— one casual line from the boss, and the product manager immediately starts selecting models, writing prompts, and designing interactions. After launch, the click-through rate is decent, but retention sees no improvement, with users complaining that "the answers are not accurate enough". The problem does not lie in execution, but in the three core questions that were never clarified before the project started: Which exact step are users paying the cost for? What will happen if the AI makes a mistake? How can we prove the AI performs better? There is no harm in getting these three questions answered before you start working.

In the past two years, almost every product team has received the same type of requirement: "We need to launch an AI feature too."

The requirement may come from the boss being inspired at an industry conference, a competitor just releasing an intelligent assistant, or the technical team having successfully tested a very cool demo. When raising the requirement, all people get is a single sentence and a deadline.

Then, product managers will follow their usual path dependency: which model to choose, how to write the prompt, where to place the entry, and what the interaction should look like.

When the feature is completed and launched, you check the data: the click-through rate is good, retention remains unchanged, and user feedback is full of comments saying "the answers are not accurate enough". During the retrospective, you will find that the problem is not poor execution, but that many questions have not been clarified before the project starts.

Therefore, before the project is approved, at least the following three questions need to be made clear:

First question: Which exact step are users paying the cost for?

Many AI features are proposed with the justification that "this link can be optimized with AI".

—— That sounds reasonable, but the way of asking is wrong.

The real question is: in this scenario, which exact step is the user stuck at? What cost do users have to pay at this step currently?

It can be time cost, for example, it takes forty minutes to write a weekly report.

It can be threshold cost, for example, users cannot get the data if they do not know how to write SQL.

It can also be psychological burden, for example, users do not know where to start when facing a blank document.

Only when you clarify the specific cost, can you judge whether AI is the right solution to the problem.

A very practical test method is: Remove the word "AI", and describe the value of this feature in one sentence. The statement "Help users compress the 40-minute weekly report to 5 minutes" makes sense. The statement "Use AI to improve the writing experience" does not make sense, because it describes the means rather than the value.

And that is not all, keep asking: Can this cost be solved by traditional solutions?

If a template, a filter, or a rule can solve the problem, there is no need to launch an AI feature.

The advantage of AI lies in handling ambiguous, open-ended, and hard-to-enumerate tasks, such as understanding a piece of natural language, generating a first draft, and extracting key points from a large amount of text. When the task itself is sufficiently definite, using rules is often faster, cheaper, and more controllable.

The purpose of the first question is to prevent taking AI as an end rather than a means.

Second question: What will happen if the AI makes a mistake?

The logic of traditional features is deterministic: same input leads to same output, and any problem is a bug that can be fixed.

But the underlying logic of AI features is different. It is probabilistic, there is always a certain percentage of errors, and this percentage will never drop to zero.

So you need to change the way of asking: instead of "Can it do things correctly?", ask "If it makes a mistake, how serious will the consequence be?"

You can break it down from three perspectives:

How high is the cost of errors: If a song recommended does not match the user's taste, the user can just swipe away. If a generated legal clause is incorrect and the user uses it directly, the consequence will be completely different. The higher the cost, the more the AI should step back to the auxiliary position, and the human should make the final decision.

Can errors be detected in time: If the output is a piece of text that users can read and judge immediately, the risk of errors is relatively controllable. If the output is a result hidden deep in the process that will only be exposed a few days later, it will be much more dangerous.

Is there a fallback after an error: Can users undo it with one click, can they modify it manually, can they go back to the original process. Good AI products will reserve a proper exit for "errors", for example, clearly mark "AI generated, please verify", retain the editing entry, and actively say "I am not sure" when the confidence level is low.

After clarifying all these, you will know whether this feature should be designed as "automatic execution" or "giving suggestions for users to confirm".

Many AI features get bad public reputation not because the model makes too many mistakes, but because the product designs something that should be a suggestion as a final conclusion.

Third question: How to prove that the AI performs better?

This is the question that is most easily skipped.

Traditional features have ready-made acceptance criteria: the button can be clicked, the process can go through, and the data can be stored in the database.

The output quality of AI features is a continuous grayscale, and there is no natural standard for "whether it is good or not".

If you do not define it before launch, you will fall into two embarrassing situations after going online: either everyone evaluates it based on their own feelings, and everyone talks at cross purposes in the meeting; or you only look at the click volume and usage times, but do not know whether users have obtained real value.

Therefore, we need to set these at the very beginning:

Write down the criteria for "good performance": It does not need to be very precise at the beginning, even just five judgment rules: no factual errors, the tone fits the scenario, the length is within a certain range, no sensitive content appears, and the format can be used directly. With this, you can have a common language for evaluation.

Prepare a batch of real samples: Find 50 to 100 inputs from real scenarios, including typical, marginal, and tricky cases. Every time you adjust the model or prompt, run it through this batch of samples to compare the differences before and after. For AI products without a sample set, optimization basically depends on luck.

Design a feedback loop after launch: Whether users have adopted the output, how much they have modified, whether they have clicked "dissatisfied", and whether they have regenerated. These behavior signals can better explain the effect than the simple usage times. The adoption rate and modification rate are often much more honest than the click-through rate.

If a feature cannot clarify "what counts as doing a good job", it has already lost the direction for iteration at the time of initiation.

After answering the three questions, then discuss how to implement it

Looking back at these three questions, they actually correspond to three basic judgments of product development: whether we should do it, whether we can do it with full confidence, and whether we have done it well.

They are not new concepts, but they are amplified in the AI era. Because the capability boundary of AI is vague, the output is uncertain, and the cost and effect are changing rapidly, the team is easily led by the possibility of technology, forgetting to return to the three origins: users, risks, and results.

Interestingly, after you clarify these three things, many requirements that "AI must be launched" will naturally shrink, some will be replaced by rule-based solutions, and some will be reduced to a narrower and more stable scenario. The remaining ones have clear goals, clear boundaries, and verifiable acceptance criteria, so they can be developed faster.

Therefore, the next time someone says to you "Let's make an AI feature too", don't rush to open the model document.

First ask clearly: What is the cost for users, what will happen if the AI makes a mistake, and how to prove that it works well.

There is no harm in getting these three questions answered before you start working.

This article is from the WeChat Official Account "Everyone is a Product Manager" (ID: woshipm), author: Hemu, published with authorization from 36Kr.