Master these 5 AI terms, and you will be ahead of 90% of people.

Shen Translation Office is a translation team under 36Kr, focusing on technology, business, workplace, life and other fields, introducing new technologies, new viewpoints and new trends from abroad.

Editor's Note: The popularity of AI continues to rise at present, but most users only stay at the basic operation level and have a vague understanding of the underlying core concepts. This article can help you quickly build basic cognition, avoid misuse, and use artificial intelligence tools more scientifically and efficiently. This article is translated from foreign materials, hoping to bring you inspiration.

AI-generated image

To be honest, when most people talk about artificial intelligence, they either rigidly recite the written definitions, or get totally confused when hearing terms like large language model and neural network.

You don't have to be either of those two groups. I believe that as long as you fully grasp the following 5 terms and 5 core concepts, no matter you are in the technology, business or education industry, or just curious about the development trend of the world, your cognition will far exceed most people around you.

Let's start the explanation now.

1. Tokens

First of all, you need to remember a core common sense: artificial intelligence models do not read complete words directly, nor do they recognize individual letters. The smallest unit for them to process information is token.

So what is a token?

You can imagine reading a book, but you don't read it word by word, you split it into fragments and read them. Some fragments are complete words, such as "cat"; some are only part of a word, such as the prefix "un-"; and punctuation marks are also counted as separate tokens. These split text fragments are tokens.

For example, the sentence "I love eating pizza" will be split into 3 tokens: "I", "love eating", "pizza".

Why is this concept important?

All AI products such as ChatGPT and Gemini continuously count the number of tokens in the background. The more tokens your input question contains, the more workload the model needs to process; the more tokens generated by AI replies, the higher the operating cost will be.

The model Context window that people often talk about measures the total number of tokens that the model can store in its "memory" at a single time. Early old models could only process 4000 tokens, while the new generation of models already support millions of tokens.

This also explains why AI forgets the previous content in long conversations: once the conversation content fills the context window, the earliest input tokens will be cleared, which is the same reason that the computer runs sluggishly when the memory is full.

Token is the smallest basic unit of the artificial intelligence language system. Understanding it, you will know why some prompts work better, why AI is prone to memory loss in long conversations, and why the interface billing standard is settled per thousand tokens.

2. Context window

Imagine you are chatting with someone, but the other person has a clear upper limit of memory, can only remember the content of the conversation in the recent period of time, and all the earlier content is completely cleared and forgotten. That is the context window.

It refers to the total amount of all text that the AI model can read and refer to at a single time. The unit of measurement is token, including your instructions, complete chat records, various uploaded documents, and all previous replies generated by the model.

You can compare it to a whiteboard, and the context window is the size of the whiteboard. You can write content on it at will, but once it is full, you have to erase the old content before you can write new content.

What's more interesting is that a small context window (for example, 4000 tokens) means that AI can only process a small amount of text at a time. If you give it a long document, it can only read it in sections; for an extra-large context window (for example, 200,000 tokens), you can paste an entire book directly and ask the model to interpret it.

This is also why the industry was excited when Claude announced support for a 200,000-token context and Gemini pushed the upper limit to 1 million tokens. Because this capability fundamentally broadens the application boundary of AI.

If you want to complete important tasks such as long document summarization and data analysis, you must be aware that AI may forget the content in the early stage of the conversation. This is not a program failure, but simply that the "whiteboard" has run out of space.

3. Temperature

This is my favorite concept to explain, and you can remember it firmly once you listen to it.

When you ask AI to generate text, there is a parameter called Temperature, which specifically determines the randomness and predictability of the output content.

• Low temperature (value close to 0): The AI output style is conservative and stable, it will choose the most probable and conventional words every time, the content is uniform and accurate, but slightly boring, just like people who always use the same set of email templates.

• High temperature (value close to 1 or even higher): AI is more willing to break away from conventions, choose novel words, unique sentence patterns and imaginative ideas, and the output content is sometimes very brilliant, but the stability cannot be guaranteed.

Here is an intuitive example: ask AI to complete the sentence "The kitten is sitting on..."

In low temperature mode, it will almost only output "the mat" and "the floor", which is safe and has no surprises;

In high temperature mode, it may write "on top of philosophical dilemma" and "on the peak of a collapsing empire".

It is full of creativity, but it is completely unsuitable for drafting legal documents.

Here is a hidden usage rule that most people don't know:

1. Fact-based work (content summarization, code writing, information extraction): lower the temperature, prioritize ensuring accuracy, and do not need extra creativity;

2. Creative work (novel writing, brainstorming, marketing copywriting): raise the temperature to get unexpected inspiration.

Public clients such as ChatGPT do not open this adjustment button, and the default setting is the middle value; but if you use the AI interface or development tools, you can see this parameter, and now you also know how to adjust it.

4. Hallucination

Everyone has heard of this term, but few people understand the underlying cause, and the cause is exactly the key.

AI hallucination refers to that artificial intelligence outputs wrong information very surely, without any hesitation, and states completely untrue content as objective facts.

For example, if you ask AI about a book, it can completely make up the title, author, publication year and story outline, and the book does not exist in reality at all, but AI speaks with such certainty as if it was excerpted from Wikipedia.

The core reason for hallucinations is ignored by many people:

Large language models are not databases and will not actively retrieve real materials. It only predicts the next most reasonable token based on the text rules learned during the training phase, and is essentially a super-sized auto-completion tool.

Therefore, when AI does not know something, it will not say "I don't know" directly, but generate a logically sounding answer, which is exactly the operating logic given to it during training.

The real risk is not that AI will make mistakes, all tools will inevitably make mistakes; the scary part is that when AI outputs wrong information, its confidence level is no different from when it outputs correct content, and it will give answers directly.

Reminder: In scenarios involving objective facts, statistical data, medical advice, legal provisions and other situations where errors will cause substantial losses, do not fully trust AI. You can take it as the starting point of information, and be sure to cross-verify the authenticity of the content.

People who really understand the concept of hallucination will not stop using AI, but learn to use it more wisely.

5. Retrieval-Augmented Generation (RAG)

Among these five concepts, it is the most easily misunderstood, but after you understand it, you will find that it is almost everywhere.

RAG is the abbreviation of Retrieval-Augmented Generation, and its core logic is actually very simple. It solves a key pain point: the training data of ordinary AI models has a cut-off date, and cannot read internal enterprise files, news that happened last week, or PDF documents you uploaded.

The underlying support for tools like "PDF Q&A" and "Document Interpretation" on the market is RAG technology.

The complete operation process:

1. After you upload the document, the system will not pour the entire content directly into the AI model; instead, it will split the document into multiple fragments and store them in a special database - vector database, which recognizes text semantics rather than simply matching keywords;

2. When you raise a question, the system first retrieves the text fragments with the highest matching degree in the vector database to complete the content retrieval;

3. The retrieved relevant fragments and your question are sent to AI together, and the model is informed: "Here is the corresponding reference material, please answer in combination with the materials."

In short, the three steps: retrieve relevant materials, input into the model, generate answers, collectively referred to as RAG.

Why is this technology so important?

Most of the practical AI products in the past two years are built on RAG at the bottom: customer service robots that can access enterprise rules and regulations, AI assistants that can interpret legal documents, and paper summarization tools, all of which are inseparable from this architecture.

Understanding RAG will completely change the way you look at AI products: AI can read your internal documents, which does not mean that it actively learns and remembers the content, but only completes an accurate retrieval, and then passes the materials to the language model for answers. The model itself has not changed, only the reference context is added.

What is the significance of understanding these concepts?

Artificial intelligence will not withdraw from public life. In the next few years, the cognitive gap between people who can only use AI superficially and those who truly understand the underlying basic logic will continue to widen.

Understanding tokens, you can write more efficient prompts;

Understanding the context window, you will know the root cause of the frequent logical confusion in AI conversations;

Understanding the temperature parameter, you can accurately adjust the output style according to different work scenarios;

Understanding hallucinations, you will not accept the facts given by AI without verification;

Understanding RAG, you will know the real operating principle of various AI tools for document interpretation.

The above are the five core terms. Mastering the underlying logic is enough to surpass 90% of users.

Welcome to join the top 10% of the crowd in cognition.

Translator: Teresa