GEB Benchmark
One structure, six voices.
Every sample below is generated in parallel from a single latent schema — never by chained translation. The schema is the only ground truth: each program voice is executed and asserted against the schema answer, each image is rendered deterministically from the drawing voice, and stylized images must preserve the structure exactly. Three V1 families: Graph Worlds (topology), Recursive Gardens (hierarchy), Symmetry Labs (geometry).
Graph Worldsgw_cycle_0003
cycle detection: does the graph contain a directed cycle? · answer: yes — B → F → D → E → C → B
story voice
[domain: software]
Module F imports module D. Module D imports module E. Module C imports module E. Module C imports module B. Module E imports module C. Module E imports module A. Module E imports module F. Module B imports module F.
Question: Is there a circular import among these modules?
[domain: messaging]
User F can forward messages to user D. User D can forward messages to user E. User E can forward messages to user C. User C can forward messages to user E. User E can forward messages to user A. User E can forward messages to user F. User B can forward messages to user F. User C can forward messages to user B.
Question: Can a message ever return to a user who already forwarded it?
mathematics voice
Node set $V = \{A, B, C, D, E, F\}$, edge set $$E = \{(B,F), (C,B), (C,E), (D,E), (E,A), (E,C), (E,F), (F,D)\}.$$ Adjacency matrix (rows/columns ordered $A, B, C, D, E, F$): $$M = \begin{bmatrix} 0 & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 1 \\ 0 & 1 & 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 0 & 1 & 0 \\ 1 & 0 & 1 & 0 & 0 & 1 \\ 0 & 0 & 0 & 1 & 0 & 0 \end{bmatrix}.$$
Question. Does there exist $k \ge 1$ with $\operatorname{tr}(M^k) > 0$?
Solution. Yes; a witness cycle is $$B \to F \to D \to E \to C \to B.$$
program voice (self-checking)
"""GEB benchmark — program voice. sample: gw_cycle_0003 (graph_world / cycle detection)"""
GRAPH = {
"A": [],
"B": ['F'],
"C": ['B', 'E'],
"D": ['E'],
"E": ['A', 'C', 'F'],
"F": ['D'],
}
EXPECTED = True
def has_cycle(graph):
WHITE, GRAY, BLACK = 0, 1, 2
color = {v: WHITE for v in graph}
def dfs(u):
color[u] = GRAY
for v in graph.get(u, ()):
if color[v] == GRAY or (color[v] == WHITE and dfs(v)):
return True
color[u] = BLACK
return False
return any(color[v] == WHITE and dfs(v) for v in list(graph))
if __name__ == "__main__":
result = has_cycle(GRAPH)
assert result == EXPECTED, (result, EXPECTED)
print(f"has_cycle = {result} [matches schema]")
drawing-program voice
{
"canvas": {
"width": 768,
"height": 512
},
"nodes": [
{
"id": "B",
"x": 384.0,
"y": 66.0,
"shape": "circle",
"role": "ordinary"
},
{
"id": "F",
"x": 548.5,
"y": 161.0,
"shape": "circle",
"role": "ordinary"
},
{
"id": "D",
"x": 548.5,
"y": 351.0,
"shape": "circle",
"role": "ordinary"
},
{
"id": "E",
"x": 384.0,
"y": 446.0,
"shape": "circle",
"role": "ordinary"
},
{
"id": "C",
"x": 219.5,
"y": 351.0,
"shape": "circle",
"role": "ordinary"
},
{
"id": "A",
"x": 219.5,
"y": 161.0,
"shape": "circle",
"role": "ordinary"
}
],
"edges": [
{
"source": "B",
"target": "F",
"directed": true
},
{
"source": "C",
"target": "B",
"directed": true
},
{
"source": "C",
"target": "E",
"directed": true
},
{
"source": "D",
"target": "E",
"directed": true
},
{
"source": "E",
"target": "A",
"directed": true
},
{
"source": "E",
"target": "C",
"directed": true
},
{
"source": "E",
"target": "F",
"directed": true
},
{
"source": "F",
"target": "D",
"directed": true
}
]
}latent schema (ground truth)
{
"sample_id": "gw_cycle_0003",
"family": "graph_world",
"task_type": "cycle",
"directed": true,
"nodes": [
{
"id": "A",
"role": "ordinary"
},
{
"id": "B",
"role": "ordinary"
},
{
"id": "C",
"role": "ordinary"
},
{
"id": "D",
"role": "ordinary"
},
{
"id": "E",
"role": "ordinary"
},
{
"id": "F",
"role": "ordinary"
}
],
"edges": [
[
"B",
"F"
],
[
"C",
"B"
],
[
"C",
"E"
],
[
"D",
"E"
],
[
"E",
"A"
],
[
"E",
"C"
],
[
"E",
"F"
],
[
"F",
"D"
]
],
"query": {
"property": "has_cycle"
},
"answer": {
"has_cycle": true,
"witness_cycle": [
"B",
"F",
"D",
"E",
"C",
"B"
]
},
"structural_properties": {
"num_nodes": 6,
"num_edges": 8,
"has_cycle": true
}
}Graph Worldsgw_reach_no_0002
reachability: A → F ? · answer: no
story voice
[domain: city-roads]
There is a one-way road from city D to city B. There is a one-way road from city E to city A. There is a one-way road from city D to city A. There is a one-way road from city F to city C. There is a one-way road from city C to city E. There is a one-way road from city A to city B. There is a one-way road from city E to city D. There is a one-way road from city E to city C.
Question: A traveler starts in city A. Can they reach city F by following the roads?
[domain: messaging]
User A can forward messages to user B. User E can forward messages to user C. User D can forward messages to user B. User E can forward messages to user D. User F can forward messages to user C. User C can forward messages to user E. User E can forward messages to user A. User D can forward messages to user A.
Question: A message originates with user A. Can it eventually reach user F?
mathematics voice
Node set $V = \{A, B, C, D, E, F\}$, edge set $$E = \{(A,B), (C,E), (D,A), (D,B), (E,A), (E,C), (E,D), (F,C)\}.$$ Adjacency matrix (rows/columns ordered $A, B, C, D, E, F$): $$M = \begin{bmatrix} 0 & 1 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 1 & 0 \\ 1 & 1 & 0 & 0 & 0 & 0 \\ 1 & 0 & 1 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 & 0 & 0 \end{bmatrix}.$$
Question. Does there exist $k \ge 1$ with $(M^k)_{A,F} > 0$?
Solution. No; $(M^k)_{A,F} = 0$ for all $k \ge 1$ (no directed path from $A$ to $F$).
program voice (self-checking)
"""GEB benchmark — program voice. sample: gw_reach_no_0002 (graph_world / reachability)"""
from collections import deque
GRAPH = {
"A": ['B'],
"B": [],
"C": ['E'],
"D": ['A', 'B'],
"E": ['A', 'C', 'D'],
"F": ['C'],
}
SOURCE, TARGET = 'A', 'F'
EXPECTED = False
def reachable(graph, start, target):
queue = deque([start])
seen = {start}
while queue:
node = queue.popleft()
if node == target:
return True
for nxt in graph.get(node, ()):
if nxt not in seen:
seen.add(nxt)
queue.append(nxt)
return False
if __name__ == "__main__":
result = reachable(GRAPH, SOURCE, TARGET)
assert result == EXPECTED, (result, EXPECTED)
print(f"reachable({SOURCE} -> {TARGET}) = {result} [matches schema]")
drawing-program voice
{
"canvas": {
"width": 768,
"height": 512
},
"nodes": [
{
"id": "E",
"x": 384.0,
"y": 66.0,
"shape": "circle",
"role": "ordinary"
},
{
"id": "D",
"x": 548.5,
"y": 161.0,
"shape": "circle",
"role": "ordinary"
},
{
"id": "B",
"x": 548.5,
"y": 351.0,
"shape": "circle",
"role": "ordinary"
},
{
"id": "A",
"x": 384.0,
"y": 446.0,
"shape": "circle",
"role": "start"
},
{
"id": "C",
"x": 219.5,
"y": 351.0,
"shape": "circle",
"role": "ordinary"
},
{
"id": "F",
"x": 219.5,
"y": 161.0,
"shape": "circle",
"role": "goal"
}
],
"edges": [
{
"source": "A",
"target": "B",
"directed": true
},
{
"source": "C",
"target": "E",
"directed": true
},
{
"source": "D",
"target": "A",
"directed": true
},
{
"source": "D",
"target": "B",
"directed": true
},
{
"source": "E",
"target": "A",
"directed": true
},
{
"source": "E",
"target": "C",
"directed": true
},
{
"source": "E",
"target": "D",
"directed": true
},
{
"source": "F",
"target": "C",
"directed": true
}
]
}latent schema (ground truth)
{
"sample_id": "gw_reach_no_0002",
"family": "graph_world",
"task_type": "reachability",
"directed": true,
"nodes": [
{
"id": "A",
"role": "start"
},
{
"id": "B",
"role": "ordinary"
},
{
"id": "C",
"role": "ordinary"
},
{
"id": "D",
"role": "ordinary"
},
{
"id": "E",
"role": "ordinary"
},
{
"id": "F",
"role": "goal"
}
],
"edges": [
[
"A",
"B"
],
[
"C",
"E"
],
[
"D",
"A"
],
[
"D",
"B"
],
[
"E",
"A"
],
[
"E",
"C"
],
[
"E",
"D"
],
[
"F",
"C"
]
],
"query": {
"source": "A",
"target": "F"
},
"answer": {
"reachable": false,
"witness_path": null
},
"structural_properties": {
"num_nodes": 6,
"num_edges": 8,
"has_cycle": true
}
}Graph Worldsgw_reach_yes_0001
reachability: A → F ? · answer: yes — A → B → C → F
story voice
[domain: city-roads]
There is a one-way road from city A to city B. There is a one-way road from city C to city A. There is a one-way road from city B to city D. There is a one-way road from city D to city F. There is a one-way road from city E to city F. There is a one-way road from city B to city C. There is a one-way road from city C to city F. There is a one-way road from city F to city D.
Question: A traveler starts in city A. Can they reach city F by following the roads?
[domain: messaging]
User C can forward messages to user F. User B can forward messages to user C. User B can forward messages to user D. User C can forward messages to user A. User E can forward messages to user F. User D can forward messages to user F. User A can forward messages to user B. User F can forward messages to user D.
Question: A message originates with user A. Can it eventually reach user F?
mathematics voice
Node set $V = \{A, B, C, D, E, F\}$, edge set $$E = \{(A,B), (B,C), (B,D), (C,A), (C,F), (D,F), (E,F), (F,D)\}.$$ Adjacency matrix (rows/columns ordered $A, B, C, D, E, F$): $$M = \begin{bmatrix} 0 & 1 & 0 & 0 & 0 & 0 \\ 0 & 0 & 1 & 1 & 0 & 0 \\ 1 & 0 & 0 & 0 & 0 & 1 \\ 0 & 0 & 0 & 0 & 0 & 1 \\ 0 & 0 & 0 & 0 & 0 & 1 \\ 0 & 0 & 0 & 1 & 0 & 0 \end{bmatrix}.$$
Question. Does there exist $k \ge 1$ with $(M^k)_{A,F} > 0$?
Solution. Yes; the minimal such $k$ is $3$, witnessed by $$A \to B \to C \to F.$$
program voice (self-checking)
"""GEB benchmark — program voice. sample: gw_reach_yes_0001 (graph_world / reachability)"""
from collections import deque
GRAPH = {
"A": ['B'],
"B": ['C', 'D'],
"C": ['A', 'F'],
"D": ['F'],
"E": ['F'],
"F": ['D'],
}
SOURCE, TARGET = 'A', 'F'
EXPECTED = True
def reachable(graph, start, target):
queue = deque([start])
seen = {start}
while queue:
node = queue.popleft()
if node == target:
return True
for nxt in graph.get(node, ()):
if nxt not in seen:
seen.add(nxt)
queue.append(nxt)
return False
if __name__ == "__main__":
result = reachable(GRAPH, SOURCE, TARGET)
assert result == EXPECTED, (result, EXPECTED)
print(f"reachable({SOURCE} -> {TARGET}) = {result} [matches schema]")
drawing-program voice
{
"canvas": {
"width": 768,
"height": 512
},
"nodes": [
{
"id": "F",
"x": 384.0,
"y": 66.0,
"shape": "circle",
"role": "goal"
},
{
"id": "B",
"x": 548.5,
"y": 161.0,
"shape": "circle",
"role": "ordinary"
},
{
"id": "C",
"x": 548.5,
"y": 351.0,
"shape": "circle",
"role": "ordinary"
},
{
"id": "A",
"x": 384.0,
"y": 446.0,
"shape": "circle",
"role": "start"
},
{
"id": "E",
"x": 219.5,
"y": 351.0,
"shape": "circle",
"role": "ordinary"
},
{
"id": "D",
"x": 219.5,
"y": 161.0,
"shape": "circle",
"role": "ordinary"
}
],
"edges": [
{
"source": "A",
"target": "B",
"directed": true
},
{
"source": "B",
"target": "C",
"directed": true
},
{
"source": "B",
"target": "D",
"directed": true
},
{
"source": "C",
"target": "A",
"directed": true
},
{
"source": "C",
"target": "F",
"directed": true
},
{
"source": "D",
"target": "F",
"directed": true
},
{
"source": "E",
"target": "F",
"directed": true
},
{
"source": "F",
"target": "D",
"directed": true
}
]
}latent schema (ground truth)
{
"sample_id": "gw_reach_yes_0001",
"family": "graph_world",
"task_type": "reachability",
"directed": true,
"nodes": [
{
"id": "A",
"role": "start"
},
{
"id": "B",
"role": "ordinary"
},
{
"id": "C",
"role": "ordinary"
},
{
"id": "D",
"role": "ordinary"
},
{
"id": "E",
"role": "ordinary"
},
{
"id": "F",
"role": "goal"
}
],
"edges": [
[
"A",
"B"
],
[
"B",
"C"
],
[
"B",
"D"
],
[
"C",
"A"
],
[
"C",
"F"
],
[
"D",
"F"
],
[
"E",
"F"
],
[
"F",
"D"
]
],
"query": {
"source": "A",
"target": "F"
},
"answer": {
"reachable": true,
"witness_path": [
"A",
"B",
"C",
"F"
]
},
"structural_properties": {
"num_nodes": 6,
"num_edges": 8,
"has_cycle": true
}
}Recursive Gardensrg_btree_0004
leaf count after depth-4 binary recursion? · answer: 16
story voice
[domain: plant-growth]
A plant begins as a single stem. Each season, every growing tip sprouts exactly two new shoots — one angled to the left and one to the right — each shorter than the branch it grew from. This repeats for 4 seasons.
Question: after 4 seasons, how many growing tips does the plant have?
[domain: task-decomposition]
A project starts as one task. Every task that is still too large is split into exactly two smaller subtasks, and the splitting is applied again to each new subtask, 4 times in total.
Question: after 4 rounds of splitting, how many smallest-level subtasks exist?
mathematics voice
Recursive definition: $$T_0 = \operatorname{Leaf}, \qquad T_d = \operatorname{Node}(T_{d-1}, T_{d-1}).$$ Leaf count: $L_d = 2^d$. Total node count: $N_d = 2^{d+1} - 1$. Branch length recurrence with ratio $r = 0.62$: $\ell_{d+1} = r\,\ell_d$.
Question. $L_{4} = ?$
Solution. $L_{4} = 2^{4} = 16$ (and $N_{4} = 31$).
program voice (self-checking)
"""GEB benchmark — program voice. sample: rg_btree_0004 (recursive_garden / binary_tree)"""
from dataclasses import dataclass
DEPTH = 4
EXPECTED_LEAVES = 16
@dataclass
class Node:
left: "Node | None" = None
right: "Node | None" = None
def build_tree(depth: int) -> Node:
if depth == 0:
return Node()
return Node(left=build_tree(depth - 1), right=build_tree(depth - 1))
def count_leaves(node: Node) -> int:
if node.left is None and node.right is None:
return 1
return count_leaves(node.left) + count_leaves(node.right)
if __name__ == "__main__":
result = count_leaves(build_tree(DEPTH))
assert result == EXPECTED_LEAVES, (result, EXPECTED_LEAVES)
print(f"leaf_count(depth={DEPTH}) = {result} [matches schema]")
drawing-program voice
{
"primitive": "branch",
"root": {
"x": 384.0,
"y": 482,
"angle": -90,
"length": 175
},
"recursion": {
"depth": 4,
"children": [
{
"angle_delta": -34,
"scale": 0.62
},
{
"angle_delta": 34,
"scale": 0.62
}
]
}
}latent schema (ground truth)
{
"sample_id": "rg_btree_0004",
"family": "recursive_garden",
"task_type": "binary_tree",
"depth": 4,
"branching_factor": 2,
"parameters": {
"angle": 34,
"scale_ratio": 0.62,
"initial_length": 175
},
"query": {
"property": "leaf_count"
},
"answer": {
"leaf_count": 16
},
"structural_properties": {
"recursion_depth": 4,
"branching_factor": 2,
"leaf_count": 16,
"node_count": 31,
"self_similar": true,
"bilateral_symmetry": true
}
}Symmetry Labssym_c4_0005
order of the rotation acting on the asymmetric shape? · answer: C4 (order 4)
story voice
[domain: mechanism]
A mechanical pointer is mounted on a central pivot. Each activation of the mechanism turns the pointer by exactly 90 degrees about the pivot, and the pointer itself is a rigid asymmetric piece.
Question: what is the smallest number of activations after which the pointer returns exactly to its original position and orientation?
[domain: dance]
Four dancers stand around the center of a stage. Every beat, the whole formation rotates by one step of 90 degrees about the stage center, keeping each dancer's pose fixed relative to the formation.
Question: what is the smallest number of beats after which the formation is exactly back to its starting configuration?
mathematics voice
Rotation matrix: $$R_\theta = \begin{bmatrix}\cos\theta & -\sin\theta\\ \sin\theta & \cos\theta\end{bmatrix}, \qquad \theta = \frac{2\pi}{4}.$$ Question. What is the order of $R_\theta$ in $SO(2)$, i.e. the smallest $k \ge 1$ with $R_\theta^k = I$?
Solution. $R_\theta^{4} = I$ and $R_\theta^k \ne I$ for $1 \le k < 4$, so the order is $4$; the orbit of the (asymmetric) base shape has exactly $4$ elements and generates the cyclic group $C_{4}$.
program voice (self-checking)
"""GEB benchmark — program voice. sample: sym_c4_0005 (symmetry_lab / rotation_order)"""
import math
POINTS = [(0, 0), (2, 0), (2, -1), (4, 1), (2, 3), (2, 2), (0, 2)]
ANGLE_DEG = 90.0
EXPECTED_ORDER = 4
def rotate(points, angle_deg, center=(0.0, 0.0)):
a = math.radians(angle_deg)
ca, sa = math.cos(a), math.sin(a)
cx, cy = center
return [(cx + ca * (x - cx) - sa * (y - cy),
cy + sa * (x - cx) + ca * (y - cy)) for x, y in points]
def close(ps, qs, tol=1e-9):
return all(abs(px - qx) < tol and abs(py - qy) < tol
for (px, py), (qx, qy) in zip(ps, qs))
def rotation_order(points, angle_deg, max_k=360):
pts = points
for k in range(1, max_k + 1):
pts = rotate(pts, angle_deg)
if close(pts, points):
return k
raise ValueError("no finite order found")
if __name__ == "__main__":
result = rotation_order(POINTS, ANGLE_DEG)
assert result == EXPECTED_ORDER, (result, EXPECTED_ORDER)
print(f"rotation_order(angle={ANGLE_DEG}) = {result} [matches schema]")
drawing-program voice
{
"base_primitive": {
"type": "polygon",
"points": [
[
0,
0
],
[
2,
0
],
[
2,
-1
],
[
4,
1
],
[
2,
3
],
[
2,
2
],
[
0,
2
]
]
},
"transform_set": [
{
"type": "rotate",
"angle": 0
},
{
"type": "rotate",
"angle": 90
},
{
"type": "rotate",
"angle": 180
},
{
"type": "rotate",
"angle": 270
}
],
"composition": "union",
"canvas_center": [
384,
384
]
}latent schema (ground truth)
{
"sample_id": "sym_c4_0005",
"family": "symmetry_lab",
"task_type": "rotation_order",
"base_shape": {
"type": "asymmetric_arrow",
"points": [
[
0,
0
],
[
2,
0
],
[
2,
-1
],
[
4,
1
],
[
2,
3
],
[
2,
2
],
[
0,
2
]
]
},
"group_action": {
"type": "rotation",
"angle_degrees": 90,
"center": [
0,
0
]
},
"repetitions": 4,
"query": {
"property": "rotation_order"
},
"answer": {
"rotation_order": 4
},
"structural_properties": {
"rotation_order": 4,
"reflection_symmetric": false,
"chirality_preserved": true
}
}