Herman was retired on September 10, 2026. This site is preserved as a frozen archive and is no longer maintained or updated. Read the retrospective →

Experiment · active

Round 3 · Five-node network diagram consistency

At a fixed five-node brief, how consistently do hosted endpoints hold the exact node set, directed-edge topology, and legend?

Runs
85
Variable
model endpoint or quality tier
Prompt
r3-network-diagram-consistency-prompt v1.0

Protocol

Round 3 pilot cell. A 16:9 five-node technical network diagram with fixed labels, edges, and a legend. Deterministic targets (graded by a separate grader, never the generator): the OCR label set must equal exactly {CLIENT, GATEWAY, AUTH, CACHE, DATABASE} plus the two legend rows 'SOLID = SYNC' and 'DASHED = ASYNC'; the node count must equal exactly 5; the directed-edge topology must equal exactly {CLIENT->GATEWAY, GATEWAY->AUTH, GATEWAY->CACHE, GATEWAY->DATABASE, CACHE->DATABASE}; legend consistency requires exactly one dashed edge and that it is CACHE->DATABASE. Prompt bytes are frozen; the prompt_sha256 is recorded in docs/ROUND3_PLAN.md.

Frozen prompt template

CONTROL PROMPT R3-NETDIAG-001. A clean 16:9 technical network diagram on a white background, flat 2D schematic style, no photorealism, no 3D. There are exactly five labeled circular nodes, each a plain circle with one centered uppercase single-word label: 'CLIENT', 'GATEWAY', 'AUTH', 'CACHE', 'DATABASE'. Directed edges are drawn as straight arrows with clear arrowheads showing exactly these connections and directions and no others: CLIENT to GATEWAY, GATEWAY to AUTH, GATEWAY to CACHE, GATEWAY to DATABASE, and CACHE to DATABASE. In the lower-right corner is a small two-row legend box containing exactly the rows 'SOLID = SYNC' and 'DASHED = ASYNC'; the CACHE to DATABASE arrow is drawn dashed and every other arrow is drawn solid. Lines and labels are black on white with a single blue accent used only for arrowheads. All labels are correctly spelled, horizontal, and sans-serif. There are no extra nodes, no extra edges, no title, no watermark, and no additional text beyond the five node labels and the two legend rows.

Review rubric

  • Prompt Adherence
  • Ocr Exact Match
  • Count Accuracy
  • Topology Accuracy
  • Legend Consistency
  • Artifact Severity

Notes

Screening pilot executed 2026-07-26 with two unseeded repetitions for each available OpenAI endpoint policy. Independent visual review recorded node count, label set, edge topology, and legend consistency per run. All four returned diagrams preserved the required topology; one medium-policy legend split each required row into two cells. xAI cells are incomplete because the authenticated image route returned HTTP 403 and the stop gate prevented further retries. No global ranking may be claimed at N=2 reps per completed cell.

Evidence

Recorded runs

errorImage-01

Cloud Wave 20260802 Minimax Wave3 R3 Priority R3 Network Diagram Consistency 066

Expanded cloud-wave variance specimen; not independently graded yet.

Size
not returned
Seed
not exposed
Method
official api
View experiment →
Generated specimen r3-network-diagram-consistency-openai-gpt-image-2-high-01 r3-network-diagram-consistency-openai-gpt-image-2-high-01
successGPT Image 2 High

R3 Network Diagram Consistency Openai Gpt Image 2 High 01

Screening grade: passed. High-01 network is topologically exact: five labels, required directed edges only, single dashed CACHE->DATABASE, and exact two-row legend on white flat schematic with blue arrowheads. Rubric: artifact-severity=5/5, count-accuracy=5/5, legend-consistency=5/5, ocr-exact-match=5/5, …

Size
1672×941
Seed
not exposed
Method
official api
View experiment →
Generated specimen r3-network-diagram-consistency-openai-gpt-image-2-high-02 r3-network-diagram-consistency-openai-gpt-image-2-high-02
successGPT Image 2 High

R3 Network Diagram Consistency Openai Gpt Image 2 High 02

Screening grade: passed. High-02 holds exact node OCR, exact directed topology, single dashed async edge CACHE->DATABASE, and matching two-row legend with no extra nodes/edges/title. Rubric: artifact-severity=5/5, count-accuracy=5/5, legend-consistency=5/5, ocr-exact-match=5/5, prompt-adherence=5/5, …

Size
1672×941
Seed
not exposed
Method
official api
View experiment →
Generated specimen r3-network-diagram-consistency-openai-gpt-image-2-medium-01 r3-network-diagram-consistency-openai-gpt-image-2-medium-01
successGPT Image 2 Medium

R3 Network Diagram Consistency Openai Gpt Image 2 Medium 01

Screening grade: passed. Medium-01 matches the frozen network brief: exact labels, exact edge set, only CACHE->DATABASE dashed, and correct legend rows without extras. Rubric: artifact-severity=5/5, count-accuracy=5/5, legend-consistency=5/5, ocr-exact-match=5/5, prompt-adherence=5/5, topology-accuracy=5/5. Strengths: …

Size
1672×941
Seed
not exposed
Method
official api
View experiment →
Generated specimen r3-network-diagram-consistency-openai-gpt-image-2-medium-02 r3-network-diagram-consistency-openai-gpt-image-2-medium-02
successGPT Image 2 Medium

R3 Network Diagram Consistency Openai Gpt Image 2 Medium 02

Screening grade: passed. Medium-02 topology and node set are exact with sole dashed CACHE->DATABASE, but legend formatting splits each required row into two cells so exact two-row legend OCR is only partial. Rubric: artifact-severity=4/5, count-accuracy=5/5, legend-consistency=4/5, ocr-exact-match=4/5, …

Size
1672×941
Seed
not exposed
Method
official api
View experiment →