Herman was retired on September 10, 2026. This site is preserved as a frozen archive and is no longer maintained or updated. Read the retrospective →

Experiment · active

Round 3 · Transparent brass-and-navy icon sheet

At a fixed four-icon brief, which hosted endpoints deliver a truly transparent, correctly counted, edge-clear production asset?

Runs
86
Variable
model endpoint or quality tier
Prompt
r3-transparent-icon-sheet-prompt v1.0

Protocol

Round 3 pilot cell. A square production icon sheet on a fully transparent background. Deterministic targets (graded by a separate grader, never the generator): the output PNG must carry a real alpha channel; the transparent-background fraction (fully transparent pixels over total pixels) must be at least 0.60; the substantial horizontal alpha-group count must equal exactly 4; no icon may touch any image edge (a fully transparent border ring must exist). Prompt bytes are frozen; the prompt_sha256 is recorded in docs/ROUND3_PLAN.md.

Frozen prompt template

CONTROL PROMPT R3-ICONS-001. A single square 1:1 production icon sheet on a fully transparent background with a real alpha channel, no backdrop, no canvas fill, no solid color behind the artwork. Exactly four separate flat vector-style icons are arranged in one horizontal row, evenly spaced, not touching each other and not touching any edge of the image, each roughly equal in size and centered vertically. Left to right the four icons are: (1) a skeleton key, (2) a cog gear wheel, (3) an owl head, (4) a padlock. Every icon is rendered in a consistent two-tone palette of aged brass and midnight navy with a consistent stroke weight. There is no background color, no drop shadow, no gradient behind the icons, no text, no labels, no numbers, no frame, and no extra icons. Output a transparent PNG.

Review rubric

  • Prompt Adherence
  • Transparency Quality
  • Count Accuracy
  • Edge Quality
  • Artifact Severity

Notes

Screening pilot executed 2026-07-26 with two unseeded repetitions for each available OpenAI endpoint policy. Alpha existence, transparent fraction, horizontal alpha-group count, and edge touching are computed deterministically from returned pixels. All four OpenAI artifacts were opaque RGB PNGs and failed the alpha requirement. xAI cells are incomplete because the authenticated image route returned HTTP 403 and the stop gate prevented further retries. No global ranking may be claimed at N=2 reps per completed cell.

Evidence

Recorded runs

errorGrok Imagine

R3 Transparent Icon Sheet Grok Imagine Trial 01 Failure

Artifactless provider-route failure. HTTPError: HTTP Error 403: Forbidden. No visual model output exists to score.

Size
not returned
Seed
not exposed
Method
first party tool
View experiment →
Generated specimen r3-transparent-icon-sheet-openai-gpt-image-2-high-01 r3-transparent-icon-sheet-openai-gpt-image-2-high-01
successGPT Image 2 High

R3 Transparent Icon Sheet Openai Gpt Image 2 High 01

Screening grade: failed. High-01 icon sheet hits identity order, count, horizontal separation, edge clearance, and brass/navy coherence with no extras; transparency judged only visually as clean free-float icons on light field (alpha bytes deferred). Rubric: artifact-severity=4/5, count-accuracy=5/5, edge-quality=5/5, …

Size
1254×1254
Seed
not exposed
Method
official api
View experiment →
Generated specimen r3-transparent-icon-sheet-openai-gpt-image-2-high-02 r3-transparent-icon-sheet-openai-gpt-image-2-high-02
successGPT Image 2 High

R3 Transparent Icon Sheet Openai Gpt Image 2 High 02

Screening grade: failed. High-02 preserves four correct ordered identities, edge-clear horizontal layout, and style coherence; visual transparency quality is clean cut-icon look without drop shadows, with alpha left to deterministic measurement. Rubric: artifact-severity=4/5, count-accuracy=5/5, edge-quality=5/5, …

Size
1254×1254
Seed
not exposed
Method
official api
View experiment →
Generated specimen r3-transparent-icon-sheet-openai-gpt-image-2-medium-01 r3-transparent-icon-sheet-openai-gpt-image-2-medium-01
successGPT Image 2 Medium

R3 Transparent Icon Sheet Openai Gpt Image 2 Medium 01

Screening grade: failed. Medium-01 delivers correct four-identity ordered sheet with separation, edge clearance, and style coherence; no labels/shadows/extra icons; transparency scored on visual free-float cleanliness only. Rubric: artifact-severity=4/5, count-accuracy=5/5, edge-quality=5/5, prompt-adherence=5/5, …

Size
1254×1254
Seed
not exposed
Method
official api
View experiment →
Generated specimen r3-transparent-icon-sheet-openai-gpt-image-2-medium-02 r3-transparent-icon-sheet-openai-gpt-image-2-medium-02
successGPT Image 2 Medium

R3 Transparent Icon Sheet Openai Gpt Image 2 Medium 02

Screening grade: failed. Medium-02 matches the four-icon production brief on identity, order, spacing, edge clearance, and palette coherence; visual transparency quality is clean icon cutouts without extras, alpha measurement deferred. Rubric: artifact-severity=4/5, count-accuracy=5/5, edge-quality=5/5, …

Size
1254×1254
Seed
not exposed
Method
official api
View experiment →