ByteDance Games' AI trial and error has finally found a viable path this time

On July 17, Nuverse, ByteDance's gaming division, released the second promotion video (PV) for *Beyond Mortal Dust*, its hyper-realistic Xianxia interactive work produced entirely via end-to-end AI workflows.

In this Xianxia-themed premium buy-to-play interactive title, nearly all scenes, characters, and combat animations are generated by AI models including Seedance.

From project initiation last October to releasing two PVs since the start of this year, the entire process took less than 12 months.

Recently, *Wall Street News · All-Weather Tech* conducted an in-depth dialogue with Xinjie, the lead producer of *Beyond Mortal Dust*, and his team at Jiangnan Studio.

Beneath the less-than-one-year development cycle lies the parallel growth of the development team and the AI models they utilized.

When the project launched last year, AI video models were still at a stage where generated characters barely looked human. The team had to work like they were producing Flash animations, manually inserting keyframes frame by frame before feeding them to the model for interpolation. After Seedance 2.0 was officially released during this year's Spring Festival, the videos crossed the "Uncanny Valley" threshold, and the model began to understand character performances, enabling a qualitative leap in production efficiency.

Even so, one word Xinjie repeatedly mentioned during the conversation left a deep impression: "diligence".

In an era where AI tools are evolving at an explosive pace, this word seems almost out of place.

But Xinjie shared a set of figures: to achieve better results, the team's success rate for AI generation sampling was once as low as 2%.

To produce a 15-second clip, the team might only get 5 seconds of usable material they deemed suitable from each generation attempt, and those 5 seconds required 20 to 100 sampling runs. Complex shots even needed over 100 attempts to select 3 to 5 seconds of usable content.

"While tools are improving, how you use them to create better content depends partly on waiting for tools to advance, and partly on human diligence and patience," Xinjie said.

I

Continuous Trial and Error

*Beyond Mortal Dust* is the fourth AI-powered game developed by Xinjie's team at Jiangnan Studio. The fates of the first three titles read like a condensed history of AI development trial and error.

The first project was a "Boss Simulator": players take on the role of a company boss, conversing with employees through natural language to manage their status and run a simulation business.

The team was thrilled at the time by AI's ability to recognize unstructured data. Traditional game inputs rely on structured numbers and spreadsheets, but AI can comprehend the meaning of full paragraphs of text.

However, after completing the demo, they ultimately scrapped the project.

The reason was simple: natural language interaction is a "high-friction" form of interaction. In traditional games, players press a few buttons to receive abundant visual feedback and emotional immersion; in this simulator, players had to type nonstop, drastically increasing interaction burden without a corresponding boost in feedback quality.

The team summarized a core principle: the best products feature lightweight interaction paired with intense, impactful feedback.

The second project was a "book-traversing game". Players raise different characters and lead them to travel into various fictional worlds. For example, the team once brought the character Lin Daiyu into the world of Hogwarts, where AI demonstrated surprising emergent capabilities: Lin Daiyu could not cast spells like "Wingardium Leviosa‌", and instead used poetry to perform magic.

That moment made the team think "this is amazing" — it fit perfectly with the Harry Potter universe while staying completely true to the character's historical background.

Yet this product was also shelved, due to limitations in AI's memory capacity: characters frequently suffered from "context garbling". For instance, after leading a character through a World Cup match, if the player then faced an invasion by the Three-Body Fleet, the AI would say "I will use the 4-2-3-1 formation from the World Cup final to analyze the Three-Body Fleet's tactics".

The team spent over a month trying to resolve the issue, only to discover it was a hard boundary of model capability that could not be overcome through engineering efforts alone.

The third project was called "I'm the A-List Director". Players nurture celebrities, select casts, revise scripts, and compete for box office returns, with AI generating trailers of the celebrities' films as feedback.

The team still believes this gameplay is very interesting, but the project was blocked by prohibitive operational costs — it will only become feasible once video model pricing drops by one to two orders of magnitude.

However, it was during the development of "I'm the A-List Director" that the team accidentally stumbled on a key insight: making an AI-generated real human figure move on screen and react differently based on player choices already constitutes a minimal viable game loop.

At that time, the team casually grabbed a random character image online, defined the opening and closing keyframes, and let AI fill in the intermediate frames. The moment the character started moving, it conveyed a striking sense of realism.

"This feels like a new-generation game engine," the team decided to set aside their ongoing projects and build a full game around this "engine".

That was the origin of *Beyond Mortal Dust*.

II

An AI Team Making Games

"We are an AI team that makes games, not a game team that uses AI," Xinjie's team members emphasized this distinction to *All-Weather Tech*.

The 2026 gaming industry is no stranger to AI, with many companies adding AI modules to their established production pipelines for tasks ranging from asset creation and art production to text generation. However, games built entirely around AI as their core driving force remain extremely rare.

*Beyond Mortal Dust* was different from day one: nearly all of its content, including scenes, characters, and combat animations, is AI-generated.

AI is the foundational premise that makes this game possible.

Long before *Beyond Mortal Dust* was officially greenlit last year, the team built an internal toolset that integrated the multi-modal capabilities of image generation, video generation, and other models onto a single platform. This architecture is now being fully CLI-optimized, so directors no longer need to open individual models, upload assets, and adjust parameters manually — they can simply tell the Agent what scene they want, and the system handles all model calls and workflow connections.

Meanwhile, the team established a dedicated director team. AI-generated video requires human oversight to refine details such as how to shoot combat animations for greater impact, or how to design storylines for maximum emotional resonance — challenges that cannot be solved with a few simple prompts.

The *Beyond Mortal Dust* team initially tried to partner with external collaborators, but found it very difficult. Many traditional directors had limited proficiency with AI, did not know how to use agents for production, and there were also divergent aesthetic standards across different parties.

In the end, the team's leadership decided to provide all members with training in AI tool usage to help everyone adapt to this new creative paradigm.

The architecture can be proactively adjusted, but model capabilities are not within the team's control.

When the project was first in preparation last year, AI video models were still generating footage where characters barely looked human, and the production process was more akin to making Flash animations.

"To make a character smile, you couldn't just give AI an image and say 'make him smile' — the model simply couldn't understand that. Instead, we had to draw a keyframe of the character's neutral front face, then draw another keyframe of the smiling face, and let the video model interpolate between the two. Back then, we needed a new keyframe every 5 seconds, so the team badly needed people who understood composition," a team member told *All-Weather Tech*.

The turning point came with Seedance 2.0.

According to the team, Seedance evolved from a simple keyframe interpolation tool into a "assistant director".

You can tell it two characters are having a conversation in a scene, provide it with the scene image, character turnaround views, and audio, and it will understand spatial relationships and perspective, independently complete the character performances, and deliver a full, coherent clip. Production efficiency finally advanced from working at the "frame" level to the "full clip" level.

Currently, Seedance 2.0 can generate 15-second videos in a single run, and the upcoming 2.5 version will extend that capability to 30 seconds.

"AI video is still in its infancy," a team member noted, "but it has already crossed the Uncanny Valley, and can produce impressive works that align with your creative vision and aesthetic standards."

Even with improved model capabilities, a 98% failure rate remains a daily reality. The team believes many people misunderstand AI production, assuming the gap between mediocre and exceptional results lies solely in crafting clever prompts.

"It's unrealistic to claim you have a set of magical prompts that can make a model produce outputs beyond its inherent capabilities — that era is over. Many AI production tutorials online can indeed help you get from 0 to 1," the team said, "but how do you go from 1 to 100? It still relies on hard work and total dedication. These two qualities are often overlooked in the AI era, yet they remain as essential as ever."

III

"Betting" with the Model

The biggest gamble at project initiation was the team's judgment of how quickly model capabilities would advance.

The team needed to enter mass production 6 months after completing the demo and core framework, and no one could be certain whether the models would reach a usable level by that time.

"If you bet too aggressively, the project will never see the light of day; if you bet too conservatively, you'll end up building a product that belongs to the previous generation."

AI content products have a clear window of opportunity. Models receive minor updates roughly every three months and major overhauls every six months — content produced this year may look outdated by next year.

This is completely different from the traditional film and television industry: films shot 10 years ago barely differ from recent releases in image quality, cinematography, and costume design. But the AI field is still on a steep growth curve, "you have to precisely time your product cycle to arrange the entire team's rhythm and management," the team explained.

One of the team's leads admitted he made a "pretty bold bet".

When the project was launched, some team members thought the plan was too aggressive. At that time, using the existing AI model capabilities to complete full game production seemed extremely difficult. "But the team members who were willing to work on this kind of content all had strong faith in this direction."

In hindsight, the bet "paid off".

When talking about ByteDance's patience with its gaming business — a topic often questioned by outsiders — many team members stated that internal support was more than sufficient, "the management was generally willing to invest in this direction, which made it possible for us to experiment with such an unconventional project."

*Beyond Mortal Dust* is scheduled to launch on Steam in Q3 2026, releasing approximately 60% of its content in early access, with continuous updates to follow until the full version is completed.

Right now, the team is simultaneously scaling up content production and polishing the completeness of earlier versions, in the middle of a hectic mass production phase.

The team maintains an open attitude about whether this game can drive the emergence of more similar products.

"If the product performs well, it will encourage more peers to join in and foster healthy industry competition. The market is large enough to accommodate many players," the team said. This also means the industry will no longer need to train a huge number of specialized production talents internally, and the healthy development of the entire ecosystem will make all of this much easier.

IV

Between Film & Games

One of the team's leads used a conceptual line to explain what they are building.

The far left end of the line represents traditional film: no player choices, audiences simply watch a pre-written story. Slightly to the right are live-action interactive films like *Prosperous World*, which have dozens to hundreds of choice points and several main narrative branches, but essentially remain "appreciating someone else's well-told story".

The far right end is traditional games: players make dozens of operations per minute, with skill combinations, strategic decisions, and dense interaction-feedback loops forming the core experience.

*Beyond Mortal Dust* sits on the middle-right section of this spectrum.

"The advantage of live-action filming is that it renders characters with greater subtlety, and for moving audiences through plot, live-action is currently a superior presentation method," a team lead admitted, "but what we are making is not just a compelling story, it's a fun interactive product. The game contains hundreds of performance segments, and it would be impossible to have real actors perform every single one — only AI can achieve that."

AI is bringing entirely new possibilities to the intersection of film and games.

In the team's view, the film and gaming industries are two sectors that have not yet been democratized for UGC creation. Short videos and graphic content have already entered an era where everyone can be a creator, but the threshold for making a film or a game remains so high that ordinary people can only dream of participating. AI is changing this, drastically accelerating the speed and efficiency of turning creative ideas into reality, potentially empowering more people with the ability to express themselves and delivering more diverse experiences to consumers.

The team has already been running small experiments. They revealed plans to host offline events during ChinaJoy, inviting players to visit their studio to take a photo, which AI will use to insert the player's likeness into the game, letting them experience what they would look like in the Xianxia cultivation world.

*Beyond Mortal Dust* may well become an important bellwether for AI-powered games. AI video-driven games may no longer be just a concept, but a scalable, replicable development path.

From the "Boss Simulator", to the context-garbling "Book-Traversing Game", to the delightfully fun but prohibitively expensive "A-List Director Simulator", and finally arriving at *Beyond Mortal Dust*, the tools have iterated and the models have evolved along the way — but some things have never changed: the sound judgment of technological direction, and the unglamorous, diligent hard work invested at every step.

AI is getting faster, but there are no shortcuts to diligence and patience.

This article is from the WeChat official account "All-Weather Tech" (ID: iawtmt), written by Zheng Minfang, and republished with authorization from 36Kr.