RL Tasks

Each RL task is one real change from a private production codebase, rebuilt as a Harbor task that grades offline with a continuous reward. This page describes what a delivered task contains and how its numbers are produced.

How a task is madeCopy Anchor Link

1. A licensed repository. Private production code with its git history, cleaned of secrets and personal data before the pipeline reads it. It has passed the exposure check.
2. One real commit becomes one task. Mechanical changes are dropped. Small neighboring commits in the same module can be combined into one task.
3. The before-state is rebuilt as a Harbor task container at the parent commit. The solver’s workspace carries no git history, grading runs offline, and the original commit message is withheld.
4. A graded verifier is written. Checks run the code instead of scanning it. Fakes are used only at external service boundaries.
5. QC checks prove the task can be solved and resists gaming. See QC checks.
6. Calibration. Solver runs place the task in a difficulty band.
7. Export. The task ships as one bundle. A task that fails a QC check goes back for repair, or does not ship.

Bundle layoutCopy Anchor Link

<task>/
├── env/                    the Harbor task
│   ├── task.toml
│   ├── instruction.md      what the solver reads
│   ├── environment/        Dockerfile and build context; repo/ is the before-state
│   ├── tests/              the graded suite; test.sh is the verifier entrypoint
│   ├── solution/           solve.sh and canonical.patch: the reference solution
│   └── cheat/solve.sh      known reward-forgery attempts, for you to run
├── manifest.json           task metadata, verifier summary, calibration, file tree
├── qc-report.md            calibration table, mutation results, baselines
├── reproduction-kit.md     the Harbor commands behind every number
├── README.md               a table of the bundle's contents
├── qa-evidence/            reference, do-nothing, mutation, and cheat-probe runs
├── trajectories/           solver runs: trajectory.json, recording.cast, reward.txt
└── traces.jsonl            the same runs as chat-format rows

The container’s workspace is /app. The solver runs as the unprivileged node user. The verifier wipes and restores its own test directory before every grade, so nothing the solver leaves behind can stand in for the graded suite.

task.tomlCopy Anchor Link

schema_version = "1.1"

[task]
name = "predev-rl/t-<12 hex>"
description = "<task title>"
authors = []
keywords = []

[metadata]
author_name = "predev-rl-pipeline"
author_email = "rl@pre.dev"
difficulty = "medium"
category = "programming"
tags = ["<task type>", "<stack>", "commit-<8 hex>", "tier:medium"]
sweep_model = "<calibration solver>"
sweep_mean = 0.52
sweep_pass_at_1 = 0.0
commit_message = "" # withheld
before_sha = "<40 hex>"
after_sha = "<40 hex>"

[verifier]
timeout_sec = 1800.0

[agent]
user = "node"

[environment]
build_timeout_sec = 2400.0
cpus = 2
memory_mb = 4096
storage_mb = 20480
gpus = 0
allow_internet = false
mcp_servers = []

[verifier.env]
PD_GRADE_WALL_CAP_SEC = "900"

[environment.env]

[solution.env]

RewardCopy Anchor Link

The reward is continuous from 0 to 1:

reward = weighted checks passed / weighted checks total

• Core checks gate partial credit. Core checks are the few that prove the task was genuinely attempted. If any fails, the whole score is multiplied by the fraction of core checks that passed.
• Synthesized checks carry a fixed share. Some tasks carry an added synthesized check layer. When it is present it is 50% of the reward, and the other layers make up the rest.
• The verifier writes the reward to /logs/verifier/reward.txt. It reads results only from a private results file the solver cannot reach. Printed output is never scored.

A run scores 0 when:

• the results file is missing or empty;
• the verifier’s test tooling does not match its SHA-256 manifest;
• the verifier leaves tracked files modified;
• one grading pass runs past PD_GRADE_WALL_CAP_SEC.

QC checksCopy Anchor Link

Every shipped task clears these checks:

• The reference solution scores at least 0.95, run in a cold container.
• Doing nothing scores at most 0.1, and no scored check passes on the untouched code.
• The checks are audited for fairness against the instruction.
• Mutation checks: renaming internal variables keeps the score at 0.95 or above, and removing one specified behavior lowers it.
• Cheat probes score 0.1 or less: the reference solution changed to exit as soon as it loads, with and without printing forged test results.
• Solver scores vary across runs, and the checks order solver attempts consistently by difficulty (Loevinger H of at least 0.40). Flat or incoherent tasks go back for repair.
• Solver runs place the task in the deliverable band.

qa-evidence/ holds the reference, do-nothing, mutation, and cheat-probe runs. qc-report.md summarizes the baselines, the mutation results, and the calibration table.

Difficulty bandsCopy Anchor Link

Solver runs set each task’s band from its solver score:

Only deliverable-band tasks count as tasks. A task whose solver runs score no better than doing nothing is treated as ungradeable, not hard, and goes back for repair. Frontier-model results from several model families can be run for a delivered set on request.

Checking overlap with your corpusCopy Anchor Link

manifest.json carries a file_tree for the repository at the before-state, the code the solver starts from:

{
  "file_tree": {
    "algorithm": "sha256",
    "path": "env/environment/repo",
    "file_count": 2,
    "root": "<sha256 of the tree>",
    "files": {
      "package.json": "<sha256 of the file>",
      "src/app.ts": "<sha256 of the file>"
    }
  }
}

• files maps every regular file to the SHA-256 of its contents. Paths are relative to the repository, /-separated, and Unicode NFC. .git and symbolic links are skipped.
• root is the SHA-256 of the lines <path>\0<sha256>\n, one per file, sorted by the UTF-8 bytes of the path. The same paths and contents always give the same root.

To check overlap, hash the files in your corpus with SHA-256 and intersect the digests with the values of files. To compare a whole tree, compute its root the same way:

import hashlib, os, unicodedata

def file_tree(root):
    files = []
    for dirpath, dirnames, filenames in os.walk(root):
        dirnames[:] = [d for d in dirnames if d != ".git"]
        for name in filenames:
            full = os.path.join(dirpath, name)
            if name == ".git" or os.path.islink(full) or not os.path.isfile(full):
                continue
            rel = os.path.relpath(full, root).replace(os.sep, "/")
            with open(full, "rb") as f:
                files.append((unicodedata.normalize("NFC", rel), hashlib.sha256(f.read()).hexdigest()))
    files.sort(key=lambda entry: entry[0].encode("utf-8"))
    tree = hashlib.sha256()
    for path, digest in files:
        tree.update(f"{path}\0{digest}\n".encode("utf-8"))
    return tree.hexdigest(), dict(files)

DeliveryCopy Anchor Link

Each task ships as a .tar.gz archive with its SHA-256 checksum. Check the archive before you unpack it, then run the reference solution to confirm the verifier in your own environment:

shasum -a 256 <task>.tar.gz
tar -xzf <task>.tar.gz
harbor run -a oracle -p <task>/env

reproduction-kit.md lists the commands behind every number in the bundle, including model runs and the mutation checks. To run the cheat script, put cheat/solve.sh in place of solution/solve.sh in a copy of the task and run the oracle agent.