RX — Replication Experiment

Demonstrates that the Executable property of Generative Specification is reproducible by any developer with a PostgreSQL instance and an Anthropic API key.

Full specification: docs/specs/rx.md

Prerequisites

  • Docker (for PostgreSQL)
  • Node.js 20+
  • ANTHROPIC_API_KEY environment variable set

Quick Start 

# 1. Start PostgreSQL
docker compose up -d postgres

# 2. Run the full generate → build → test cycle
./runner/run.sh

# 3. Review evidence artifacts
cat evidence/score.json
cat evidence/jest-output.json

Directory Structure 

experiments/rx/
  spec/               GS document fed to the AI agent (committed verbatim)
  runner/             Orchestration scripts
  score/              Rubric scoring scripts
  evidence/           Committed output artifacts (jest --json, build log, scores)
  generated/          AI-generated project source (gitignored)
  docker-compose.yml  PostgreSQL 16 ephemeral instance

Evidence Artifacts

After a successful run, evidence/ contains:

File Description
jest-output.json Raw jest --json output — primary evidence
build-log.txt npm install + tsc --noEmit output
score.json Per-property rubric scores
run-metadata.json Model version, date, API call count, cost

These files are committed. Anyone who clones the repository gets the pre-run evidence and can verify the results without re-running, or re-run to produce their own evidence.

Pass Criteria

  • tsc --noEmit exits 0
  • jest --json → numFailedTests === 0
  • No hardcoded credentials in generated output

Relationship to Other Experiments

  • AX (complete) — adversarial AI vs AI study, v1–v5, 40 participants. Establishes the claim under controlled conditions.
  • RX (this) — automated replication. Makes the claim independently verifiable.
  • DX (April 2026) — human developer comparative study. Tests adoption in the wild.