A new Scaling variable for text-to-image generation has been discovered by ByteDance Seed Team.

Text-to-image models have been continuously expanding their model scale, data volume and computing power, but the information content of captions paired with training images is rarely systematically studied as an independent variable. The ByteDance Seed team found that longer natural language captions do not necessarily mean the model obtains more usable visual supervision; compared with length, the image-bound information content in captions is a better predictor of the final converged training loss of diffusion models.

Based on this finding, the team proposed Structured Prompt, and jointly improved text conditioning from both the Diffusability and Promptability dimensions, which finally achieved significant performance gains in complex composition, reasoning and world knowledge generation tasks.

Figure 1 of the paper｜Natural language length saturates quickly; structured conditioning continuously increases image information and reduces diffusion training loss along a unified relationship.

Over the past few years, advances in text-to-image models have almost followed a familiar path: larger models, more data, and higher training computing power.

However, there is an easily overlooked difference between text-to-image models and language models. Language models can perform self-supervised learning directly from text sequences; text-to-image models rely on image-caption pairs to learn "what kind of text corresponds to what kind of visual content". An image may contain a large number of objects, attributes, positions, actions and relationships, but only the parts that are accurately described and clearly bound by the caption can be passed to the model as text condition supervision.

Thus, a fundamental question arises: in addition to continuing to expand the model, data and computing power, can we make the generative model learn better by increasing the image information carried by captions?

In this new work, the ByteDance Seed team studied this problem. The core conclusion can be summarized in one sentence:

What truly scales with text conditioning is not the number of tokens in the caption, but the image information in it that can be utilized by the model.

• Paper Title: Scaling Properties of Text Conditioning in Visual Generation
• Authors: Zilong Chen, Chaorui Deng, Kunchang Li, Hongyi Yuan, Haoqi Fan Affiliation: ByteDance Seed
• Paper: https://arxiv.org/abs/2607.29679
• Project Homepage: https://heheyas.github.io/context-scaling
• Code: https://github.com/heheyas/context-scaling
• Model: https://huggingface.co/collections/heheyas/context-scaling
• Online Demo: https://heheyas-context-scaling.hf.space/
• Hugging Face Paper: https://huggingface.co/papers/2607.29679

The Prompt is getting longer,

Why isn't the model getting stronger?

An intuitive approach is to write training captions or user prompts longer and more detailed. More tokens seem to imply more supervision, which should also help the model generate more complex images.

However, experiments give different results. On a variety of existing open-source text-to-image systems, natural language prompts quickly saturate as their length increases, and the final output performance is even lower than the performance of their respective shortest prompts. Even if the diffusion model is specially trained on the same set of long-text captions, the gain is very limited.

To observe this phenomenon more clearly, the team designed an image reconstruction experiment with a fixed backbone network. For the same reference image, the team started from the same complete annotation, generated four natural language captions with gradually increasing detail levels, and then used the same Qwen-Image model and random seed to try to reconstruct the image. These captions describe the same entities and relationships, and subsequent versions mainly increase their length by supplementing and expanding natural language expressions.

Surprisingly, although the captions are significantly longer, the quality of image reconstruction hardly improves. The newly added prose is mostly explaining, rewriting or connecting existing content, and does not continuously add new visual variables that can be stably utilized.

Figure 3 of the paper｜Fixed backbone reconstruction experiment: reconstruction tends to saturate after NL captions continue to get longer, while gradually restoring SP fields continuously improves the result.

This shows that caption length is only a very weak proxy variable. A text can be very long, but still fail to clearly explain which object a certain attribute belongs to, what the relationship is between two objects, where they are respectively, and their front-back hierarchy in the frame.

How to measure the real image information in captions?

If the number of tokens is not sufficient to measure the strength of supervision, we need to directly measure the image-bound information in captions. To this end, the team adapted two complementary metrics from existing work: Grounded Perplexity Gain (GPG) and Effective Detailness (ED).

GPG: How much more "predictable" does the image make the caption.

GPG is a white-box metric that requires reading the token probabilities of the model. For the same caption, the team respectively let a frozen vision-language model see and not see the paired image, and calculated the log-likelihood improvement of the caption's content tokens after the image appears. If the caption contains a large amount of information tightly bound to the image, seeing the image should significantly improve the model's ability to predict these tokens.

ED: How many reliable image attributes does the caption cover.

ED is a black-box semantic metric that does not depend on token probabilities. It extracts attributes with entity context from both the image and the caption, and then calculates the precision of caption attributes and the recall of image attributes. Finally, F0.5, which pays more attention to precision, is adopted to impose a stronger penalty on descriptions in captions that have no image basis.

The two metrics characterize the same problem from different perspectives: GPG focuses on the statistical dependence between images and text, while ED focuses on whether the caption accurately covers verifiable visual content.

Figure 6 of the paper｜Definition and measurement trends of GPG and ED.

Caption information content can predict the training loss of diffusion models

Next, the team fixed the images, model architecture, initialization method, optimization configuration and training budget, and only changed the training captions. The entire experiment includes 15 caption configurations: three natural language versions of different lengths, six Structured Prompt versions with gradually restored fields, and six spatial expression or field masking variants. Each configuration starts from the same BAGEL continued-training checkpoint, and independently trains a diffusion model.

The results show that there is no unified relationship between the number of tokens in natural language captions and the training results; but when the horizontal axis is changed to caption information content, configurations of different formats and detail levels fall on a highly regular curve:

• The converged diffusion loss is approximately linearly related to GPG, with Pearson r = -0.984;
• The converged diffusion loss shows a power-law trend with ED, with Pearson r = -0.971 in log-log space;
• The rankings of different caption configurations by GPG and ED are also highly consistent, with Spearman ρ = 0.96.

The team calls this the scaling properties of text conditioning. It is not a theoretical law that holds for all models, but an empirical calibration obtained under a fixed architecture and training recipe. However, it brings two direct values.

First, it turns caption information content from a vague concept of "data quality" into a controllable and measurable training variable: when the model, images and computing power remain unchanged, the information content can predict the final converged training loss of the model.

Second, after completing one calibration, candidate caption schemes can be compared first with GPG or ED under the same training recipe, before deciding whether to invest in expensive diffusion model training. For the six caption variants not involved in the fitting, the two metrics can still predict their converged loss relatively accurately.

Figure 7 of the paper｜Under a fixed training recipe, the converged loss shows a stable relationship with caption information content.

Structured Prompt:

Make information not only more, but also easier for the model to use

The previous experiments revealed a key point: simply adding more natural language prose is not enough, and the newly added information also needs to be organized in a stable and clear way.

Therefore, the team proposed Structured Prompt (SP), which uses structured JSON to represent visual variables in an image. It contains three levels:

• Global level: frame intent, scene, atmosphere, style, lighting and photographic information;
• Element level: the identity, attributes, actions, positions, optional depth and local photographic information of each subject;
• Relationship level: position, occlusion, interaction and semantic relationships between different elements.

Compared with free text, the key of SP is not just the appearance of "JSON", but to make different visual variables enter stable named fields, reducing ambiguity in attribute attribution, spatial relationship and object binding. In the fixed backbone reconstruction experiment, as SP fields are gradually restored, the reconstruction quality continues to improve; in the complete training sweep, the increase of field coverage also simultaneously improves GPG, ED, and reduces the converged diffusion loss.

Figure 5 of the paper｜Structured Prompt organizes global, element-level and cross-element visual variables into named fields.

In order to generate complete SP on large-scale training data, the team built an image-to-SP annotation pipeline. General-purpose VLM is responsible for global semantics and local content, Sapiens supplements human pose evidence, DepthAnything V2 provides relative depth, SAM 2.1 provides mask and occlusion cues, and finally VLM unifies all this information into a consistent and complete SP.

Figure 8 of the paper｜VLM together with pose, depth and segmentation experts jointly construct a complete Structured Prompt.

The team refers to the ability of a caption representation to expose and organize image supervision to the diffusion model as Diffusability. What SP improves is exactly this dimension: without changing the architecture of the diffusion model, it allows the model to learn more and clearer visual variables from text conditioning.

Promptability:

With a good structure, we also need the LLM to fill it properly

During training, complete SP can be extracted from paired images, but in actual generation, there is only a single sentence from the user, no reference image or oracle annotation. The system also needs an LLM prompter to expand the user's request into a detailed, coherent SP that does not violate the original constraints.

The team calls this ability to instantiate structured conditioning from user requests Promptability. The end-to-end generation quality depends on the joint effect of both sides:

Generation Quality = Diffusability × Promptability

The multiplication sign here is an organizational perspective, not a mathematically fitted multiplication formula: Diffusability describes what the diffusion model can learn from the caption representation, and Promptability describes whether the LLM can really generate high-quality caption instances during inference.

Figure 4 of the paper｜Structured Prompt connects annotation, measurement, diffuser training, prompter training and final generation.

First, the team fixed the SP schema and Qwen-Image diffuser, and only replaced the zero-shot LLM prompter. As Qwen3.5 scales from 0.8B to 397B, GenEval++ in thinking mode increases from 46.4% to 86.8%. Except that the smallest model tends to repeat in the thinking process and cannot output valid JSON, chain-of-thought brings further improvement at other scales. This shows that the model capability and reasoning ability of general-purpose LLMs can be directly converted into better image generation results through the caption interface.

However, zero-shot LLMs still tend to generate SPs with insufficient information and relatively simple composition. To further improve Promptability, the team adopted three-stage training:

SFT learns the SP content distribution expected by the diffusion model, not just the JSON format;

Cold-start distills "how to derive SP only from user requests" from privileged reasoning traces with paired images;

RFT continues to optimize on rollouts generated and rendered by the prompter itself, the verifier filters high-confidence trajectories, and then dense token supervision is provided through on-policy self-distillation of the image-conditioned teacher.

Ablation experiments show that the three stages play different roles: SFT brings the largest single-stage structural improvement, cold-start strengthens the derivation from user requests to SP, and verifier-gated OPSD achieves the strongest results on the prompter's own distribution.