Photoroom reveals the data strategy behind PRX, its 7B text-to-image model, including data assembly, captioning, and filtering.
Welcome back! This is Part 4 of the PRX series. Parts 1 to 3 covered model architectures, training design, and a 24-hour speedrun.
This time we're pulling back the curtain on the part that quietly underpins all of it: the data. Of all the things that shaped PRX's quality, the data pipeline was one of the least glamorous parts to build but nevertheless an important piece to get right. Here's what we did, what we'd do differently, and a few things we only learned the slow way.
In one sentence: we assemble training data from a mix of public and internal datasets, re-caption the images with a VLM, and turn the result into the streamable corpus we trained PRX on. At a high level, the data pipeline looks like this: The goal was to assemble a large, diverse dataset for pre-training. At this stage the model is learning how the world looks: the visual concepts, the objects and scenes, how things are composed and lit, and the sheer range of what images can contain.
That is a problem of coverage and diversity, not of per-image perfection. A broad, representative corpus teaches the model far more about the structure of the visual world than a smaller, prettier one would, even if many of the individual images are ordinary snapshots or slightly compressed. Over-filtering for aesthetics at this stage would actually hurt, narrowing the distribution and costing the model concepts and compositional variety it cannot recover later.
Making generations look polished is a separate, later concern, which we leave to fine-tuning and preference alignment on small, ruthlessly curated sets. Pre-training is for breadth; fine-tuning is for taste. We assemble our pre-training data from a mix of public and internal datasets.
The priority at this stage is breadth, diversity, and standing on curation that already exists rather than redoing it ourselves. Where a source already comes quality-filtered, deduplicated, and filtered for NSFW content and personal information, we build on that work instead of repeating it at scale. Sources arrive in different shapes: some come with the image data itself, others as metadata plus baseline captions that we bring into a common form.
We took a pragmatic approach: rather than build a corpus entirely from scratch, we leaned on existing datasets and our own tooling to assemble one quickly. In hindsight it is not necessarily the absolute best dataset one could build, but it was a solid and lightweight starting point for pre-training a 7B model. In our experience, what matters most for pre-training is to use long captions that accurately describe everything in the image.
We saw this directly in Part 2, where switching from short captions to long ones substantially improved sample quality. If captioning is faithful, we don't need to worry about the occasional screenshot, advertisement, logo, or bit of text in an image, because those things get described in the caption too, so the model learns them as conditioned, controllable attributes rather than reproducing them unconditionally. Accurate captioning turns "noise" into something you can prompt for, or prompt away.
This is precisely why the filtering we do later is deliberately light. We remove what is genuinely unusable, not everything that is imperfect. We have been using Mosaic Streaming and Mosaic Data Shards (MDS) as a dataset format for distributed training for a while now.
In combination with Mosaic Composer we found it to be a very low-maintenance, flexible, and well-performing framework for distributed training. However, MDS datasets are very rigid. Adding a column or creating a subset for a given filter basically means that one has to scan and rewrite the whole dataset.
That's why we use Lance for this kind of feature engineering and dataset curation. Lance is a columnar data format with cheap predicate pushdown, scalar indexes, and vector search, the right tool for building and exploring datasets with billions of rows. These two formats play off each other throughout this post and the PRX data pipeline: Lance to build, MDS to stream.
For previous training runs, we used T5Gemma as the text encoder and pre-computed the text latents, storing them in MDS as bytes. This time around, after switching the text encoder to Qwen3-VL, we decided to compute text latents on the fly during training instead. Running the text encoder inside the training loop costs throughput, but how much depends on the model: for a small denoiser it can be significant, whereas at PRX's 7B scale the text encoder's compute is negligible next to the denoiser, and we measured only a roughly 3–4% throughput cost (about 1 extra day on a 30-day run).
We encoded all images as JPEG at quality 92, instead of a lossless format such as PNG. We did not just assume quality 92 was safe, we measured it. Real-world images have usually already been JPEG-compressed several times, so the real question is whether one more re-encode hurts.
Across repeated decode/encode cycles on both high-resolution (1–2 MP) and lower-resolution (0.25–0.5 MP) real images, the first re-encode at quality 92 is essentially imperceptible. Since most source images already arrive JPEG-compressed, storing them losslessly as PNG buys little, so we convert everything to JPEG at high quality (quality 92). We also checked what matters most: whether training on JPEGs changes what the model produces.
Our corpus is mostly JPEG anyway, so the only advantage PNG could offer is not introducing new artifacts. You cannot interactively explore Parquet tables with hundreds of millions or even billions of rows. We therefore store the data in Lance, where we can index, query, and browse it.
One lesson worth sharing is about fragmentation. A Lance table is split into fragments, and several operations scale with the total number of fragments rather than the row count: a scan opens the files of every fragment, and some metadata operations are O(number of fragments). We learned this the slow way.
Our first ingest targeted only 100,000 rows per fragment, which left thousands of tiny fragments and made even simple filters and full-text queries crawl. We chose to re-caption every image ourselves rather than rely on the captions some of the datasets happened to ship with. The reason is consistency: across a mix of sources, caption length, style, and quality vary a lot, and we wanted a single uniform standard across the whole corpus The metadata a dataset already carries is still valuable, though, just for a different purpose.
Pre-existing captions, and the vector embeddings that sometimes come alongside them (for example CLIP embeddings), are what make a dataset explorable in Lance right away: full-text search over the captions and nearest-neighbor search over an embedding column let us navigate the data and quickly judge its quality and the filtering it will need, well before we have run our own captioning over it. With the data queryable, we could quickly profile it. The resolution distribution, for example, told us where to set cutoffs.
Lance tables support full-text search on text columns as well as nearest-neighbor similarity search on vector embedding columns out of the box, and both can be accelerated by building indexes. Browsing the data this way let us assess its diversity and quality qualitatively and spot issues early, simply by looking at it. We found that long, detailed captions are a very strong lever on output quality.
A small diffusion model trained on Qwen2.5-VL-7B captions (red) versus the baseline LLaVA-1.5-LLaMA3-8B captions (blue), scored every 10k steps up to 100k. Here is an example image with both our long, VLM-generated caption as well as the shorter caption used as the baseline: Our captioning runs as a streaming pipeline built on Ray Data: it reads the data from Lance, prepares each image, captions it on the GPU, and writes the caption back as a new column we can query and filter on. A tight schedule and limited resources meant that we couldn't spend too much time on ablations of different captioning models and system prompts.
We used the following prompts to generate the captions. We shortlisted three VLMs as captioner candidates: We benchmarked caption quality indirectly: we trained a small diffusion model for 100k steps on each caption variant and scored its generations with FID, CMMD, and DINO-MMD. Among the candidates, Qwen3.5-9B is best on all three metrics, with Qwen3-VL-8B close behind on FID and DINO-MMD (though weaker on CMMD) and the Relaxed captioner trailing on FID but competitive elsewhere.
We put less weight on throughput than quality, but at hundreds of millions of images it still matters, since it sets how much the captions cost to generate. At the end of the pipeline, we turn the captioned, bucketed stream into MDS, the format the trainer streams. The Mosaic Streaming library packs samples into ~128MB shards streamed during training from object storage or a local filesystem.
Diffusion training runs on fixed-size tensors, so every image in a batch must share one width and height. We bucket in two steps. First by resolution tier (pixel count): a 512px and a 1024px tier for most of the corpus, plus 2048px and 4096px tiers for high-resolution data.
Each image is then resized to its bucket with a Lanczos filter and encoded as JPEG at quality 92, and Mosaic Data Shards are written into a tree keyed by resolution and aspect ratio so the trainer can stream buckets separately: The exploration in Section 2 relied on whatever captions a dataset shipped with. Once our own detailed captions were in place, the same full-text search became far more powerful, and a few passes over the captions turned up things the sparser original metadata had hidden: more text-heavy images (screenshots, slides, documents, infographics) than we had estimated, and some NSFW content that had slipped through. To turn those approximate searches into a filter, we ran a quick classification pass with Qwen3-8B in text-only mode: it reads each caption (never the image) and labels the sample visual, text, or nsfw.
Rather than rewrite the whole corpus to drop these samples, we added a skip-list feature to the MDS data loader: a small sidecar file per shard lists the sample indices to skip, and the loader unions them and skips those samples at training time. The same explorer surfaced a fair number of duplicates: large image collections tend to accumulate exact and near-duplicate copies, which waste training compute and skew the distribution. We could have done an exact dedup on a byte hash like SHA-256, but a perceptual hash already subsumes that (it matches byte-identical images too) while also catching re-encoded or resized copies, so we used only the perceptual hash.
Once every image has a perceptual hash (fingerprint), the dedup itself is just a hash-map lookup: scanning each aspect-ratio bucket across resolutions, we keep one entry per fingerprint, so any image whose fingerprint we have already seen is a duplicate of the one we kept, The duplicates are written out as a per-shard skip list that the loader unions and skips at load time, so nothing is deleted and the dataset is never rewritten. Across the corpus, deduplication removed on the order of a few percent of images, the caption-based text filter a few percent more, and the NSFW pass a small fraction of a percent. The dataset described here is the pre-training corpus, where breadth and scale matter most.
For supervised fine-tuning and preference alignment the trade flips entirely: quality matters far more than quantity, and the problem becomes finding and curating small, high-signal subsets out of this large corpus. Build something with PRX, or find a bug in our pipeline? PRX is Apache 2.0 and the model code lives at github.com/Photoroom/PRX.
Come talk diffusion models and data with us on Discord. Why this matters: The PRX data strategy reveals the complexities and trade-offs involved in assembling a large-scale dataset for text-to-image models. By prioritizing breadth and diversity, Photoroom's approach highlights the importance of coverage and representation in pre-training.
The use of long captions and light filtering also underscores the value of accurate and detailed descriptions in guiding the model's learning process. As the AI industry continues to push the boundaries of generative models, the insights shared in this article will likely inform and influence the development of future data strategies. For developers and businesses, understanding the intricacies of data assembly and curation can help inform their own approaches to building and fine-tuning models.
Moreover, the open-sourcing of PRX and its accompanying code will likely facilitate further innovation and experimentation in the field, driving progress and advancements in text-to-image synthesis.
