Trayntrayn.ai

Observations

Observation data passed to get_action.

Fields

Each step, the harness passes an Observation to your agent:

FieldTypeDescription
goalstringTask objective from the Trayn run.
urlstringCurrent sandbox URL.
axtree_txtstring?Accessibility tree with BIDs.
axtree_objectAXTreeObject?Structured accessibility tree.
extra_element_propertiesRecord<string, ExtraElementProperties>?Visibility, bounding box, clickability, and markability by BID.
screenshotUint8Array?Screenshot when the task needs verifier evidence.
last_actionstring?Previous action.
last_action_errorstring?Execution error or verifier feedback from the previous action.
action_historystring[]?Semantic summaries of prior server-accepted steps.
elapsed_timenumber?Seconds since task start.
open_pages_urlsstring[]?Open page URLs.
active_page_indexnumber?Active page index.
focused_element_bidstring?Focused element BID.
dom_txtstring?Text DOM snapshot when available.
pruned_htmlstring?Pruned HTML snapshot when available.

BIDs

BIDs identify elements across frames:

BIDMeaning
42Element 42 in the main frame.
a5Element 5 inside iframe a.
ab3Element 3 inside iframe b, nested inside iframe a.

The SDK resolves the frame path before executing element actions.

Accessibility Tree

[42] button "Add to Cart", visible, clickable
[43] textbox "Search" value="laptop", visible, clickable
[a0] link "Help", visible, clickable

Extra Element Properties

{
  "42": {
    visibility: 0.85,
    bbox: [100, 200, 80, 32],
    clickable: true,
    setOfMarks: true,
  },
}

On this page