How Trayn Works

The Full Picture

flowchart TB
  subgraph RECORD["1. Record"]
    direction LR
    Ext["Browser extension"] --> Capture["Capture every screen + interaction"]
  end
 
  subgraph PROCESS["2. Process"]
    direction LR
    Anon["Anonymize offline"] --> Gen["Generate interactive playground"]
    Gen --> Task["Generate task definition"]
  end
 
  subgraph TRAIN["3. Train"]
    direction LR
    Agent["Agent attempts task"] --> Grade["Grade each step"]
    Grade --> Mem["Store memories"]
    Mem -->|"next rep"| Agent
  end
 
  RECORD --> PROCESS
  PROCESS --> TRAIN

Recording

Install the Trayn browser extension for Chrome or Firefox. Navigate to any website and record yourself completing a workflow.

Privacy-First Anonymization

Before anything is stored, Trayn detects and replaces all personal data — entirely offline in your browser. Nothing leaves your machine until anonymization is complete.

flowchart LR
  R["Raw recording"] --> D["Detect entities"]
  D --> Rep["Replace consistently"]
  Rep --> Safe["Anonymized recording"]
 
  subgraph Entities["What gets replaced"]
    direction TB
    N["Names"] ~~~ E["Emails"]
    E ~~~ A["Avatars"]
    A ~~~ C["Currency amounts"]
    C ~~~ DT["Dates & times"]
    DT ~~~ L["Locations"]
  end

• Offline Anonymization — AI models run directly in your browser for identifying and replacing sensitive information.
• Context Preservation — Anonymized identities and context are maintained and mapped to ensure consistent representation across applications.

Playground Generation

From just a few recordings, Trayn creates a hyper-realistic, anonymized, fully interactive clone of the original application. These are standalone RL environments where every button, popup, dropdown, and navigation works as they do on the real app.

Trayn analyzes the recorded interactions to understand the app's behavior — how clicks cause UI changes, how inputs flow through forms, how elements relate to each other and hundreds of micro interactions to generate the most accurate sandboxed playgrounds to train your agent.

flowchart LR
  R["A few recordings"] --> P["Fully interactive playground"]
  P --> T1["Task-specific playground"]
  P --> T2["Task-specific playground"]
  T1 & T2 --> APP["Full app playground"]
  APP --> A["Supports arbitrarily complex tasks"]

Each recording produces a task-specific playground where actions are tracked and graded with guidance on what should have been done instead. This data is packaged as memories the agent can retrieve on subsequent attempts.

Task Generation

Trayn analyzes each recording and extracts a task definition:

• Goal — Natural language description ("Change ticket priority to High and add the 'escalated' tag")
• Verifiers — Success criteria (screenshot of expected final state, DOM checks)
• Expected actions — Semantic descriptions of each step the user took

All of the above are editable and customizable in the application as well as the SDK.

Agent Training Loop

The core of Trayn is a learning loop: attempt → grade → remember → improve.

sequenceDiagram
  participant SDK as Trayn SDK
  participant Browser as Playground Browser
  participant Agent as Your Agent
  participant Grader as Grading Service
 
  SDK->>Browser: Launch playground URL
  loop Each step (up to max_steps)
    SDK->>Browser: Extract accessibility tree
    SDK->>Agent: Observation (goal, axtree, URL, history)
    Agent->>SDK: Action ("click('42')")
    SDK->>Browser: Execute action
  end
  SDK->>Grader: Submit actions for grading
  Grader->>SDK: Per-step grades + corrections
  SDK->>SDK: Store memories
  Note over SDK,Agent: Next rep retrieves memories

Repetitions

Each CLI run executes the task multiple times:

• Run 1 — Baseline run, no memory retrieval
• Run 2+ — Retrieves memories from previous reps

Memories tell the agent what worked ("REPEAT: clicked the save button") and what didn't ("AVOID: clicked the wrong dropdown — DO INSTEAD: use the priority dropdown in the sidebar").

Pluggable Adapter Architecture

Every I/O boundary in Trayn is an interface you can replace. This lets you use Trayn's learning loop with any infrastructure:

flowchart TB
  subgraph Adapters["Pluggable Adapters"]
    direction LR
    GA["GraderAdapter"]
    RA["RetrieverAdapter"]
    SA["StorageAdapter"]
    EA["EmbeddingAdapter"]
  end
 
  subgraph Defaults["Built-in Defaults"]
    direction LR
    HTTP["HttpGraderAdapter"]
    DR["DefaultRetriever"]
    S3["CloudStorageAdapter"]
    GE["GoogleEmbeddingAdapter"]
  end
 
  GA --> HTTP
  RA --> DR
  SA --> S3
  EA --> GE

Each adapter is configured via a dependency injection singleton:

import { setStorage, setEmbedding, setGrader, setRetriever } from "@trayn/memory";
 
setStorage(myStorageAdapter);
setEmbedding(myEmbeddingAdapter);
setGrader(myGraderAdapter);
setRetriever(myRetrieverAdapter);

See the individual adapter docs for interface details and custom implementation examples:

• Grading — GraderAdapter
• Retrieval — RetrieverAdapter
• Storage — StorageAdapter
• Embeddings — EmbeddingAdapter

Multi-Framework Support

Trayn is designed to work with any browser agent. The @trayn/agent-sdk provides the harness, browser automation, and action execution. The @trayn/memory package provides the learning layer. Bring your own agent — or use the built-in reference agent to get started.

flowchart TB
  subgraph Frameworks["Your Agent"]
    direction LR
    Custom["Your own agent"]
    OC["OpenClaw, Browser Use, ..."]
    Built["TraynAgent (reference)"]
  end
 
  subgraph SDK["@trayn/agent-sdk"]
    direction LR
    Harness["Harness"]
    Env["Browser environment"]
    AX["Accessibility tree"]
    Actions["Action executor"]
  end
 
  subgraph Memory["@trayn/memory"]
    direction LR
    Store["Memory storage"]
    Retrieve["Memory retrieval"]
    Embed["Embeddings"]
  end
 
  Frameworks --> SDK
  SDK --> Memory

Bring Your Own Agent

The SDK defines an Agent interface — implement it to plug any browser agent into Trayn's training loop. The harness handles the browser, accessibility tree, action execution, and grading; your agent just decides what to do next.

trayn --agent my-agent --url https://app.trayn.ai/playground/{host}/{sessionId} --max-steps 15 --reps 3

External agents like OpenClaw work out of the box:

trayn --agent openclaw --url https://app.trayn.ai/playground/{host}/{sessionId}

Built-in Agent (TraynAgent)

For quick starts, the SDK includes TraynAgent — a reference agent that implements the full observation → reasoning → action loop. It supports models from OpenAI and Google (GPT-4o, Gemini, etc.) and defaults to gpt-4.1-mini:

trayn --url https://app.trayn.ai/playground/{host}/{sessionId} --max-steps 15 --reps 3

Memory as a Standalone Package

@trayn/memory can be used independently of @trayn/agent-sdk — integrate Trayn's learning layer into any agent framework:

npm install @trayn/memory

Use it as a Claude Memory Tool or integrate directly into your agent's retrieval pipeline.

Two Packages

Installing @trayn/agent-sdk pulls in @trayn/memory automatically.

Environment Variables