The journalists hidden behind large language models: This is how GPTs generate their responses

In the past two years, more than 40,000 film and television jobs have vanished in Los Angeles alone. The number of journalistic positions at U.S. newspapers has been cut in half compared to 2008. A group of these displaced media professionals have moved to large model companies in Silicon Valley, using their writing experience to teach AI how to answer your questions. Every time you feel that a large model's response strikes a chord, the person who crafted that exact line is a former journalist, editor, or documentary director.

In this episode of the "Silicon Valley 101" podcast, we dive into this little-known profession — Content Engineer. We invited two senior content engineers who transitioned from journalism: former media professional and host of the "Planet Tavern" podcast Tony, and Bianca Consunji, AI Strategy Director at NBCUniversal, for an in-depth discussion on what defines a "good" conversation, and how to break it down into quantifiable, trainable, and evaluable engineering standards.

We start our conversation with this emerging role that is rapidly expanding across AI companies, exploring how content and taste can become the key to product competitiveness as model capabilities gradually level out, and how the judgment rooted in humanities and social sciences can be translated into language that engineering teams can use.

Below are selected highlights from this conversation:

01 A New Profession in Silicon Valley: Content Engineer

Software engineers give AIs like ChatGPT and Claude the ability to speak, while content engineers are responsible for defining another thing: what makes a conversation "good". The things we once thought could only be felt but not explained — inspiration, linguistic intuition, and the soulful spark we feel when we are fully engaged in a conversation — are now being broken down by these former wordsmiths into a set of rules that can be built into AI systems.

In its job posting for content engineers, Meta outlines the responsibilities as follows: define what "good content" is; demonstrate exceptional taste, creativity, and writing skills; and craft responses that make users feel delighted, surprised, and inspired.

Faith: What exactly is a content engineer?

Tony: Are we designing the AI model, the AI product, or its writing style? None of the above. What we are designing is the human perception of the experience.

At the time, Meta launched a new position called Content Engineering. The posting stated that candidates needed content experience, editorial experience, and ideally film and television production experience. I thought, that's exactly me! During the interview, I told them directly that I met every single requirement they listed, and that they couldn't find many people on the market who could meet these standards, so they might as well hire me.

What does content engineering require you to write? It could be system prompts, for example. It heavily relies on creative writing — how to craft prompts that are well-written and understandable to the model, and how to train the model when it occasionally acts stubbornly. These are the core skills of professionals in this role.

Bianca: Code is a technical language, but ordinary language is also technical. It has grammar and syntax, all of which can be quantified. So what makes good writing, good production, and good conversation? These are all technical skills. A great script or a great article can be deconstructed, analyzed, and evaluated for quality, even though people are used to reading based on intuition. People know that The New Yorker, The New York Times, and the Daily Mail all have distinct styles, but they can't clearly articulate the differences. Those of us who transitioned from journalism to AI do exactly that — we make it clear what makes The New Yorker The New Yorker. We translate tone, style, and voice into more technical concepts.

Ultimately, this role seeks to answer the question: what should content actually look like? For many of us with a journalism background, whether we were producing videos or writing in the past, we would always think about things like: who should I interview, and what tone should this story have? I first learned how to produce content, and then I learned how to reverse-engineer content.

Faith: Before discussing how to deconstruct "good content", the first step we need to take is to understand what the definition of "good" is.

Bianca: The concept of "good" depends on the product you are working on. For example, even with the same underlying model, you can build very different products: one could be an agent that helps you get work done, and the other could be a chatbot that pretends to be your boyfriend. These two products are completely different. For example, for an airline customer service chatbot, you need to design what the opening conversation looks like. You also need to clarify the goal of this interaction — is it to help users solve their problems, or to get rid of them as quickly as a real customer service agent would?

After that, you need to keep training the model and reviewing its outputs. You break down these attributes one by one and score them. This process is all about isolating variables, figuring out what is not that useful, and then gradually defining what "good" means. Some responses might be decent, but they are only 7 out of 10, so we need to explain where the missing 3 points lie.

In fact, the focus of my work is to improve writing and help AI answer questions better. These are all fundamental journalism skills. For example, did the AI actually answer the user's question, or did it fail to understand the user's true intention? So if I had to name my area of expertise, I would say it is understanding intentions, not just the literal question. Essentially, I am teaching AI what an ideal response looks like. For example, someone might ask "Will Taylor Swift get married at Madison Square Garden?"

A poor response would be: "Yes, she will get married there, and here is her bridesmaid list." Because we have no way of knowing if this is true — no one can confirm it.

Another very bad type of response is the kind we can easily recognize: responses that are obviously full of robotic tones and mechanically stitched-together information.

Madison Square Garden | Image source: Ajay Suresh, CC BY 2.0

So what makes a better response? For example: "We don't know if she is actually getting married, but there is a lot of speculation that her wedding might be held at Madison Square Garden." At this point, you can add sources and attributions, and acknowledge that there is significant uncertainty in the current information. These are all elements that make news reporting better, so we need to integrate them into the training process. In general, this boils down to factuality, tone, and the true understanding of what information the user is looking for that I just mentioned.

Think about it: a user might simply be asking "Will she get married at Madison Square Garden?" If you only answer "No", even though you literally responded to the question, the user will not have a satisfying experience. They will still want the background and context behind the question.

Faith: Which means understanding the "implied meaning" behind the questioner's words.

Bianca: This is especially necessary for non-Western cultures. I am Filipino myself, and there is a lot of truly important information that we do not say directly. When you are having a conversation with a real person, you have to read their facial expressions and notice their pauses to understand the unspoken parts of what they are saying.

So when I teach people how to use AI tools better, I encourage them to use voice input. This way, the AI can capture even the most subtle nuances. When you use voice, you don't repeatedly edit your thoughts in your head first — you directly express what you want to convey. This is very similar to interviews, because oral interviews — whether over the phone or in person — are almost always much better than written interviews conducted via email.

Because email interviews are over-edited, either by ourselves or by the other party. Your interviewee has too much time to polish their answers, so they might not share their full thought process, or they might simply be too busy to spend an hour answering your questions. More importantly, during an email interview, no one can follow up in real time, ask them to elaborate on a previous topic, and instead they just answer your pre-planned questions step by step. I think this is exactly what journalists are good at, so I strongly encourage people to use verbal input to let the model access a more complete range of thought processes. As a Model Designer or Content Engineer, we can also use this to design better responses.

In fact, one of my favorite things about AI is that it makes you realize that the speed of thinking and the speed of acting are not the same thing. The time it takes you to think of something is completely different from the time it takes you to actually execute it. So instead of feeding the AI more materials, I would advise you to first try to tell the AI model all the thoughts in your head, their background, and why you came up with those ideas directly. The model may already have extensive training, but all it needs is for you to fully express your intentions to unlock its capabilities.

02 Media Professionals Step in to Fill AI's Context Gap

Faith: What Bianca talked about is how a person can pass the context in their mind to AI as completely as possible. But context does not only exist in one person's mind — it is also embedded in a language, a culture, and the unspoken understandings that have formed over years in an industry. Sometimes, even if a sentence is translated correctly, the intended meaning is lost. So in the process of switching between multiple languages and cultures, what else can content engineers do?

Tony: After generative AI emerged, all AI labs discovered a new opportunity in internationalization. Before AI, when we distributed the same product or content around the world, the only thing we could do was translation — for example, translating "Meryl Streep just won an Oscar" into a local language.

But with generative AI, a new possibility is that AI can not only present content in different languages, but also recreate it based on local languages and contexts. For example, if we put the same piece of information in a Chinese context — who is China's equivalent of Meryl Streep? What is China's equivalent of the Oscars? The counterparts might be "Siqin Gaowa" and the "Golden Rooster Awards or Hundred Flowers Awards". This involves generative creative work.

Can AI do this? Technically, AI absolutely has the language capabilities to do it, but what it lacks is the knowledge of who the local equivalent of Meryl Streep is in different contexts. Tuning AI to do this is very difficult, because you need to find these cross-lingual and cross-regional equivalents, and if you accidentally offend fan communities or trigger other cultural conflicts, things become even more sensitive. At that point, who is best suited to do this kind of work? In the entertainment industry, entertainment journalists are obviously the most qualified, because they follow these news stories every day and have developed their own excellent vertical "small model" of the entertainment industry. Having them train, tune, and develop these strategies yields the best results.

Another point: a few days ago, when I was listening to previous episodes of "Silicon Valley 101", Director Lu Chuan mentioned that the faces generated by domestic video generation models are all the good-looking men and women that are currently popular in China. This is obviously because when you were building the model and designing the product, the people you involved and the training data itself were biased. But if you use someone who understands film, who knows what a "cinematic face" looks like, and who knows how to make the video model more diverse, you can quickly improve the quality of the model.

The interaction between journalism and AI does not have to be limited to asking and answering questions — it can also correspond to journalistic writing. Because one thing I've noticed is that many people can't ask good questions to AI because they don't understand what context is needed in the current situation — what kind of context the AI requires. But in journalistic writing, you can't just throw out a fact without context. You have to establish context: how did this happen, and what are its causes and consequences. So when you train and interact with AI, the mindset of "I must first explain the full context, causes, and consequences of this matter, and then put forward my core question and argument" — this journalistic writing approach is extremely effective, and it works every single time.

Let me share something cutting-edge. A paper that won the Best Paper Award at this year's ICLR conference focuses on how AI handles multi-turn conversations, which is usually a weak point for AI. What the paper did was break down a very complex instruction into small pieces, feed them to the AI bit by bit, and see if the quality of the output decreased compared to when the AI processed the entire complex instruction at once. It turned out that the quality dropped significantly. Why? When AI does not have enough context, it will make assumptions based on its own training data, and keep filling in the gaps, making its answers increasingly bloated. If it makes a mistake halfway, that mistake will snowball and get bigger and bigger. This is our judgment of AI in 2026.

The Duck-Rabbit Illusion: What you see depends on your context | Image source: Fliegende Blätter

A few days ago, I sent you another paper I published in 2018, which was a study on Weibo. At the time, there was an incident in Mong Kok, Hong Kong, where a young girl relieved herself in public, which sparked huge discussions on Weibo. I witnessed firsthand how the conversation shifted from friendly chatting to mutual insults. One of the main findings of this study is that conversations between people in Weibo comment sections also lack context — you don't know who the other person is, so people start making assumptions: "You must be that kind of person", "You must not have children", "That's why you would say something like that". Human assumptions and AI's assumptions are quite similar when context is fragmented. So in many cases, I think the findings from humanities and social sciences are not that different from the discoveries in today's AI industry.

Faith: As a former international journalist, one of the key skills you used for international news reporting is figuring out how to tell a story in different countries and different cultures. How did you transfer these experiences to your current job?

Tony: I think the experience of working not just as an international journalist, but in the entire journalism and media industry, is not so much transferred as completely ported over to the AI field. Because on the one hand, your personal creative writing is completely different from your regular journalistic writing. When you do journalistic writing, you always have to keep the audience in mind. How do you create a connection with the audience? What can be said, and what can't? What kind of reactions might certain statements trigger? How can you stay objective, not offend anyone, stay true to yourself, and at the same time present the facts accurately? This balance is very complex and requires training — that's one aspect.

On the other hand, even during our conversation right now, as the interviewer, you must constantly be thinking: he just said this, what should I say next to guide him to the next question? There are many such interactive moments. This kind of interaction translates into an intuitive skill in designing AI chat interactions. I think it's very important to bring this kind of intuition into conversations with AI.

Another skill is the ability to ask questions. The most important part of doing interviews and journalistic work is completely ported over to daily AI work.

Faith: I just thought of something — a very important skill for journalists is how to ask good follow-up questions. You can't strictly follow the logic of your pre-written