Thinking of switching to an AI career? First get a clear understanding of Token.

At 11 p.m., Xiao Lin is still revising his resume.

To switch to a career as a large language model product manager, he first sent his work experience to the chat tool, then supplemented the target position, recruitment requirements and project data.

The first few rounds of revisions went well, but by the dozen or so round of conversation, the tool suddenly rewrote the experience he had already deleted, and even forgot the requirement he repeatedly emphasized: "Do not package me as a technical R&D personnel."

Xiao Lin is a bit confused: Why does AI seem to get dumber the more you use it?

This is not that the tool suddenly "gets dumb", but that he has unknowingly consumed a large number of Tokens.

Token, a unit of measurement similar to the weighing unit in supermarkets

When we buy fruits, we count how many apples there are, but when checking out in the supermarket, the bill is calculated by weight;

Humans are used to counting the number of characters and sentences when reading text, while the unit of measurement that large language models use to process text is Token.

When a piece of text enters the model, it will first be split into individual Tokens. A Token can be a Chinese character, part of a word, or a punctuation mark, number or space. Different models have different splitting methods, so Token is not equal to the number of words, and there is no fixed conversion rule like "one thousand characters equals how many Tokens".

The questions you input take up Tokens, the uploaded resumes and job descriptions take up Tokens, the previous chat history takes up Tokens, and the responses generated by the model also take up Tokens.

Simply put, Token is the "text chip" that large language models use when reading and responding.

Why does it become more forgetful the longer you chat?

You can imagine a single conversation as moving house.

There is only one truck with limited capacity in front of you. The resume is one box, the job description is another box, the chat history of the first ten rounds is more than a dozen boxes, and space also needs to be reserved for the responses the model will generate later.

When the carriage is not full, all items can be taken away; when there are more and more items, it may be necessary to compress, sort, and even leave some of the old luggage behind.

The capacity of this "truck" is the context window of your chat with AI; the total amount of text it can hold is measured by Token.

Therefore, as the conversation gets longer, the model may gradually ignore the previous requirements. If you said "highlight operational experience" at the beginning, then repeatedly discussed products, technology and data later, it may only remember the latest information in the end, and the resume will be revised more and more off track.

Similar scenarios are not uncommon in daily life.

You sent fifty-nine WeChat messages in a row to a friend, with a sentence "Don't bring your computer tomorrow" in the middle, and it is very likely that the other party will miss it; you move all the ingredients in the refrigerator in front of the chef, but only say "make any dish at will", no matter how many ingredients there are, it does not mean the result will be more to your liking.

The more information there is, the less effective it may not be. For the model, what really matters is: in the limited space, which information should be prioritized to be seen.

Why is Token also related to cost?

If you only use ready-made chat products, you may not feel the cost of Token. But when enterprises connect large language models to their own products through interfaces, Token usually directly corresponds to the cost.

It is a bit like taking a taxi: passengers only care about whether they can reach the destination, but the platform has to calculate the mileage and travel time.

Users only see "help me optimize my resume", but the background may actually send personal experience, job requirements, modification rules, historical conversations and reference examples, and the model also needs to generate new content.

All these inputs and outputs will generate Tokens.

Suppose a recruitment platform provides resume optimization services for 100,000 users every day. If the entire historical records of users and dozens of job information are re-sent every time, it is like the driver takes a detour around the city before going to the destination every time he takes an order. A little extra cost in a single operation is not obvious, but when scaled up to 100,000 times, it is a considerable cost.

Therefore, developing a large language model product is not as simple as "connecting to a model". It is also necessary to consider: which information must be sent, which can be deleted, how to refine long documents, whether expensive models are needed for simple tasks, and how to balance effectiveness, speed and cost.

Does a longer prompt mean better results?

Many people will write such requirements to make the model "obedient":

"Please analyze carefully, think deeply, never be perfunctory, the content must be professional, comprehensive, wonderful and logical..."

This is like sitting down in a barber shop and repeatedly telling the barber "cut it nicely" without ever saying how short you want to cut your hair or whether you want to keep the bangs.

It seems that a lot has been said, but there is very little really useful information.

If you replace it with: "Revise the resume for the large language model product manager position, focus on highlighting requirement analysis, project promotion and data review; do not fabricate experience; each item is expressed in the format of 'what you did + what result it produced', and the content is controlled within 50 words." The model will be easier to execute.

A good prompt is not long, but specific. The target, background, constraints and output format are more valuable than repeatedly emphasizing "be careful".

If you want to switch to a new career, to what extent should you understand this concept?

Not every job seeker needs to calculate Tokens, but if you want to enter the related industry, you should at least understand its impact on three issues:

First, it affects how much information the model can process at one time; second, it affects the product invocation cost; third, it affects the response speed and final effect.

When doing content operation, you will know why long materials should be refined before creation; when working as a product manager, you will pay attention to the context, user experience and invocation cost; when doing application development, you also need to further understand model billing, content segmentation, caching and knowledge base retrieval.

In an interview, if you are asked "how to reduce the usage cost of an interview assistant", you can't just answer "replace it with a cheaper model". You can also propose: delete duplicate historical information, only retrieve the materials related to the current question, summarize long conversations first, limit invalid outputs, and match different models for tasks of different difficulty levels.

This is the way to move from "being able to use tools" to "understanding the product".

Next time, when it forgets your requirements, misses the details in a long text, or stops suddenly halfway through writing, don't rush to blame it.

Check if this truck with limited capacity has already been loaded with too many things.

Token seems to be just a technical term, but behind it lies the most basic operating logic of large language models: how much it can read, remember and write at one time, and how much cost the enterprise has to pay for it.

For people who want to switch careers, understanding Token is not for memorizing an interview question, but for learning to re-evaluate an intelligent product from three dimensions: effect, experience and cost.

This article is from the WeChat official account Liepin (ID: liepinwang), the author is Cheng Shu, and it is authorized to be released by 36Kr.