Does it still look
like the product?
A proposed pilot protocol for evaluating product fidelity in generated images and short video.
1. Start with a bounded question
Can reference-guided generation change a product’s surroundings while preserving the visual details that matter to a buyer? The first pilot will focus on still images. Short video will use the same rubric with additional temporal checks.
2. Build a small reference set
Proposed scope: 6–10 owned, synthetic or permissioned products across a few everyday categories. Capture or collect clear references for each item, record permitted uses and list the details that must remain unchanged. Do not include private client material without permission.
3. Compare controlled variations
Use the same product reference and scene brief for each method. Compare reference-guided generation with a compositing baseline. Keep aspect ratio and output size consistent, generate a small fixed number of attempts, and retain every output. Log prompts, settings, model version, elapsed time, retries and estimated generation cost.
4. Review against a shared rubric
| Dimension | Check |
|---|---|
| Identity | Silhouette, proportions and recognisable details match the reference. |
| Surface & colour | Material, texture and colour remain credible under the new lighting. |
| Text & marks | Required labels and logos remain accurate and readable. |
| Scene integration | Shadows, reflections, scale and perspective agree with the setting. |
| Usability | The agreed crop is usable and product-critical details are visible. |
| Temporal stability | For video: product details stay stable across the clip. |
Score each applicable dimension as 0 (fails), 1 (needs correction) or 2 (usable). A product-critical failure blocks acceptance even if the total score is high. Where possible, use two independent reviewers and record disagreements before reconciliation.
5. Report useful measures
Report accepted outputs as a fraction of all attempts, correction time, failure categories and cost per accepted asset. Include the sample size, review criteria and uncertainty. Do not infer sales or conversion gains from visual quality scores.
6. State the limits
A small product sample cannot establish general model superiority. Results will depend on category, reference quality, model version and reviewer judgement. Local language and cultural suitability need separate review. Any public examples must be cleared for publication.
Discuss a research collaboration ↗