v0.9 · five-lane pipeline · reproducible Nix execution · visible screenshot + recording evidence · acceptance-criteria fidelity · MFA-authenticated testing · runs on any LLM

Autonomous tests, AI-graded.

Hand TFactory a finished feature on a branch — from AIFactory, Claude Code, or any tool, via the MCP control plane or a plain acceptance-criteria file (markdown / Gherkin / EARS). The agent pipeline plans, writes, sandboxes, scores, and commits the suite — autonomously — grades every acceptance criterion against a test that actually ran, and posts a triage report to your PR.

See the demo   Design plan

The five-agent pipeline

spec → graded tests
01

Planner

test_plan.json
02

Gen-Functional

tests/*
03

Executor

docker sandbox
04

Evaluator

verdicts.json
05

Triager

triage_report.md

Part of the Factory family — a governed, verified, observable autonomous software factory. PFactory plans, AIFactory builds, TFactory verifies, CFactory watches over all four. See Why Factory.

How it works

Evidence you can see

A green checkmark is not proof. For interactive acceptance criteria, the browser lane runs in a reproducible per-task Nix toolchain inside an ephemeral Kubernetes Job (RFC-0005 Tier A), drives the real deployed app, and captures a screenshot of the rendered page plus a recording of the test driving it. The Acceptance tab grades each criterion against a test that actually passed — an honest “verified X/Y”, never a blanket “done”:

The Acceptance tab — verified 5/5 acceptance criteria, each linked to its evidence

The Evidence tab renders the captured recordings and screenshots inline, so a reviewer can watch the test execute and look at the page it produced:

The Evidence tab — browser-lane recordings and screenshots

The whole pipeline — Plan, Generate, Execute, Report — is a live view in the portal, and the same evidence appears on the finished task in the CFactory cockpit:

The TFactory pipeline view — Plan, Generate, Execute, Report

Reach anything under test — including behind MFA

A governed node in the Factory line

Status by lane

The lane spine is modality-based (Decision 2). Security scanning is delegated to dedicated security pipelines and is out of scope here — TFactory focuses on functional and feature testing.

Lane Status Runtime Coverage Evidence captured
Unit Active tfactory-runner-pytest + tfactory-runner-jest line (cobertura / lcov) —
Browser Active Nix toolchain in a k8s Job (Playwright); host fallback where applicable n/a (line coverage doesn’t apply when the test drives the browser) screenshots, video, trace
API Active per-framework image + HTTP HAR recorder line where the test exercises framework code network.har
Integration Active per-framework image + AppRuntime (multi-service) line where applicable network.har, service logs
Mutation Active mutmut (Python) / Stryker (TypeScript) — one-mutation-per-run probe in the Evaluator reported per mutant (killed / survived) —

Each subtask’s lane is chosen by the Planner from its (language, framework) via the framework registry; reviewers see the lifecycle phases in the LaneStatusGrid and the per-test evidence in the Triager PR comment. New languages and additional pipelines hook into the same five-lane spine through new FrameworkDescriptors — no lane additions required.

Quickstart

# Clone and bootstrap (NixOS / flake-based)
git clone https://github.com/olafkfreund/TFactory
cd TFactory
nix develop
tfactory-minimal-venv   # creates apps/backend/.venv
tfactory-test           # backend suite, seconds

Full walkthrough in the repo README plus the end-to-end smoke guide for running real scenarios against an AIFactory project.

Documentation

Tracking