GAMEPHANES / GAME TERMINAL-BENCHHome ↗

Game Terminal-Bench

Executable terminal tasks for agents that design, build, debug, and repair interactive software.

Browse Registry Agent Performance

20Production tasksfirst suite slate

5Task familiesfour tasks each

8Public examplesGamePhanes registry

0Harbor releasespackage work in progress

The benchmark unit

A task workspace with a real runtime contract.

Terminal-Bench checks what happens in a filesystem. Game Terminal-Bench adds what happens when the delivered software launches, accepts input, changes state, and produces observable evidence.

01WorkspaceProvision starter

02TerminalInspect and edit

03BuildImport and compile

04RuntimeProbe behavior

05ReportScore and trajectory

Harbor-compatible packaging

One task. One Harbor directory. One reproducible run.

Each Game Terminal-Bench task will follow Harbor's task-directory contract, so the same game-engine challenge can run through Harbor, be inspected on GitHub, and be published to a compatible Hub. The current public JSON registry is the authoring contract; it is not counted as a Harbor release yet.

Harbor task package

repair-neon-relay-jump/
├── task.toml
├── instruction.md
├── environment/
│   └── Dockerfile
├── tests/
│   ├── test.sh
│   └── test_runtime.py
└── solution/
    └── solve.sh

task.toml

Metadata & budgets

Harbor schema, task name, version, difficulty, timeouts, CPU, memory, and network policy.

instruction.md

Agent-facing goal

A user-visible outcome and constraints, without prescribing the patch the Agent should write.

environment/

Isolated runtime

Pinned Godot version, project dependencies, harness tools, and the clean workspace the Agent receives.

tests/

Independent verifier

Build, runtime, event, behavior, and regression checks run after the Agent submits the project.

solution/

Oracle only

A maintainer solution used to validate task solvability; it is never mounted into the Agent workspace.

Read Harbor task structure ↗Harbor Hub ↗target schema: Harbor 1.4

Dataset boundary

Same package contract. Different benchmark.

Game Terminal-Bench uses Harbor as its execution and packaging layer. It is an independent benchmark with its own task registry, interactive-software taxonomy, game-runtime evidence, and leaderboard.

Shared execution layer

Harbor / Terminal-Bench task contract

task.toml with schema and resource budgets
instruction.md for the Agent-facing goal
environment/, tests/, and oracle solution/
Containerized reset, timeout, verifier, and artifacts

Independent benchmark identity

Game Terminal-Bench / TB-Game

Interactive software engineering tasks
Game state, events, screenshots, and runtime probes
Own registry, releases, scoring, and leaderboard
Not mixed into Terminal-Bench 2.0 or TB-Science rankings

Agent + base model

Projected agent performance

One score for the complete engineering loop: inspect, edit, build, run, diagnose, and repair.

ESTIMATED BASELINENOT MEASURED

GamePhanes + Fable 5Projected
84.2
84.2%
GamePhanes + GPT-5.6-SolProjected
81.7
81.7%
GamePhanes + Claude Sonnet 4.5Projected
79.6
79.6%
GamePhanes + GPT-5.5Projected
77.9
77.9%

Projected scorePlanning rangegamephanes-estimate==0.1

These values are product-planning estimates, not measured benchmark results or model capability claims.

Five evidence surfaces

A green build is only the beginning.

Build

Project import, parse, and headless build complete within the task budget.

Runtime

The submitted project launches and remains responsive under controlled probes.

Behavior

Named events and state fields satisfy deterministic user-visible requirements.

Regression

Required existing behavior remains intact after the change.

Trajectory

Terminal actions, patches, feedback, costs, failures, and repair attempts stay auditable.

Production task slate

Twenty concrete tasks, not a vague task count.

The first production suite is organized around five kinds of work a game coding agent must repeatedly do. Every task is a workspace, a user-visible goal, a controlled runtime probe, and a regression boundary.

04Gameplay & feel

04Engine & runtime

04UI & interaction

04Content & systems

04Delivery & quality

Production slate: planned contracts. Public Registry entries are the currently runnable examples; sealed starters and hidden evaluators are not published here.

Harbor task catalog

What each task asks the Agent to do.

Each slug below is intended to become one Harbor task directory with a task.toml, instruction.md, isolated environment, independent tests, and an oracle solution.

Gameplay & feel

Make the core loop feel correct, then prove it survives a real playtest.

repair-neon-relay-jumpmedium

Restore jump impulse

Fix a broken jump while preserving shard collection and relay completion.

bug-fix · movement · regression

tune-coyote-timehard

Tune platformer forgiveness

Implement coyote time and jump buffering without changing level collision behavior.

feature · physics · input

fix-combat-hit-windowmedium

Repair combat hit timing

Make attacks damage targets only during the authored active frames and emit hit feedback.

bug-fix · combat · feedback

balance-upgrade-loophard

Balance the upgrade loop

Add a data-driven upgrade choice that persists into the boss encounter and remains deterministic.

feature · progression · data

Engine & runtime

Work with lifecycle, state, performance, and platform constraints inside the engine.

reset-scene-statemedium

Make scene reset deterministic

Reset a failed run without leaking score, timers, signals, or spawned nodes into the next attempt.

bug-fix · lifecycle · state

migrate-save-schemahard

Migrate an old save

Load a version-1 save into the new inventory schema while preserving player progress and rejecting corrupt data.

feature · save-load · compatibility

meet-frame-budgetexpert

Remove a frame-time spike

Find the runtime hotspot in a crowded scene and keep the controlled playtest under the frame budget.

performance · profiling · runtime

repair-export-inputhard

Repair cross-platform input

Make keyboard and controller actions resolve to the same gameplay intent in the exported build.

platform · input · export

UI & interaction

Turn player intent into clear, navigable, and accessible interaction states.

fix-pause-focusmedium

Restore pause-menu focus

Open and close the pause menu without trapping input or leaving the game simulation running.

bug-fix · UI · input

navigate-controller-menumedium

Make menus controller-navigable

Add deterministic focus order and selection feedback to a menu that currently works only with a mouse.

feature · accessibility · UI

surface-critical-hud-statehard

Expose critical HUD state

Show health, objective, and interaction feedback at the exact state transitions used by the game loop.

feature · HUD · state

repair-camera-feedbackhard

Repair camera feedback

Restore camera follow, hit shake, and room framing without breaking the player coordinate contract.

bug-fix · camera · feedback

Content & systems

Connect authored content to reusable, inspectable runtime systems.

seed-procedural-roomhard

Make procedural rooms reproducible

Use a declared seed so the same task produces the same room layout and reward placement.

procedural · determinism · content

restore-quest-statehard

Repair quest progression

Connect dialogue, objective state, and reward delivery so a completed quest cannot be completed twice.

content · state-machine · regression

validate-asset-importmedium

Fix asset import contract

Normalize a sprite and collision asset pipeline so imported resources match the scene's expected dimensions.

assets · pipeline · scene

sync-audio-statemedium

Synchronize audio with state

Trigger music, one-shots, and mute behavior from the same state transitions as the visual feedback.

audio · signals · feedback

Delivery & quality

Diagnose failures and deliver a change that survives a clean rerun.

reproduce-crash-on-loadmedium

Reproduce and fix boot crash

Use the startup log and a clean workspace to isolate a resource-order crash and make boot reliable.

debugging · crash · startup

stabilize-playtest-timeouthard

Close a flaky playtest

Remove timing sensitivity from an async encounter so the same controlled probe reaches the same terminal state.

debugging · async · determinism

add-regression-probehard

Add a regression probe

Instrument a named runtime event and preserve the existing public behavior while making the failure diagnosable.

instrumentation · tests · evidence

package-release-buildexpert

Prepare a reproducible release

Fix export settings, asset inclusion, and version metadata so a clean machine produces the declared build.

packaging · export · release

Production allocation

Five families. Four tasks each.

The catalog above is the first 20-task production slate. Each family covers a distinct failure mode in interactive software engineering.

04Gameplay & feelmovement · combat · progression

04Engine & runtimelifecycle · performance · platform

04UI & interactionmenus · controls · HUD · camera

04Content & systemsprocedural · quests · assets · audio

04Delivery & qualitycrashes · determinism · tests · release

Release status

Public contract now. Sealed benchmark next.

Public RegistryAvailableTask schemas, reference environments, public harnesses

GamePhanes BenchIn developmentIsolated workspaces and deterministic reports

GamePhanes EvalPlannedPrivate variants, hidden evaluators, anti-shortcut checks