IC-541Code-based output (Python or TikZ) produces images with notably higher scientific style scores than direct image generation from Stable Diffusion and DALL-E

Leixin Zhang, Steffen Eger, Yinjie Cheng, WEIHE ZHAI, Jonas Belouadi, Fahimeh Moafian, Zhixue Zhao

SourceScImage: How good are multimodal large language models at scientific text-to-image generation?

Comparing the two output modes on the scientific style criterion, images generated via intermediate code (Python or TikZ) receive scores above 2.5, while direct image generation from Stable Diffusion (1.96) and DALL-E (1.55) scores below 2. The paper attributes this to code models producing structured, convention-following output, whereas visual models trained on real-life images lack scientific drawing conventions. Python code also outperforms TikZ across all three criteria (correctness 2.81 vs 2.64, relevance 2.97 vs 2.81, scientific style 3.56 vs 3.18) with a lower compile error rate (0.17 vs 0.19).

Evidence
correlational
Key metric
Scientific style: code output >2.5 (TikZ 3.18, Python 3.56) vs direct image <2 (Stable Diffusion 1.96, DALL-E 1.55); Python vs TikZ: correctness 2.81 vs 2.64, relevance 2.97 vs 2.81, scientific style 3.56 vs 3.18, compile error 0.17 vs 0.19
Caveat
The comparison is between different model families (code models vs image models), so the effect of output format is confounded with model identity. The paper notes this is an average over GPT-4o and Llama 3.1 8B for the code comparison.
Model
GPT-4o, Llama 3.1 8B, Stable Diffusion, DALL-E
Related findings
IC-539, IC-540
Extraction
automatic-extraction