9. Fetch the trace, choose what to keep, and recover from drift

The trace is one layer of the loop. It has two doors.

The first door is inline. Every run_flow call so far carried its trace inside the execute response, at the requested trace_level. That door was open all along.

The second door is fetch. trace_run reopens a retained Run later, by its run_id. Chapter 8 used it out of necessity: the fired run had no response of ours to ride. Now walk through it deliberately. Reopen the daytime run from Chapter 6:

{"tool":"trace_run","arguments":{"run_id":"run_0001"}}

{
  "run_id": "run_0001",
  "outcome": {
    "ok": true,
    "status": "success",
    "error": null
  },
  "trace": {
    "run_id": "run_0001",
    "level": "full",
    "steps": [
      {
        "id": "lobby-on",
        "calls": "light.lobby",
        "input": {
          "action": "turn_on"
        },
        "output": {
          "on": true,
          "brightness": 100
        },
        "outcome": {
          "ok": true,
          "status": "success",
          "error": null
        }
      },
      {
        "id": "dock-unlock",
        "calls": "lock.dock_door",
        "input": {
          "action": "unlock"
        },
        "output": {
          "locked": false
        },
        "outcome": {
          "ok": true,
          "status": "success",
          "error": null
        }
      }
    ]
  }
}

The same record came back: the same steps, the same outputs, the same outcomes. One layer, two doors. The inline door reports at execution time; the fetch door reads the stored record afterward.

The fetch door can also re-level. Ask for less than what was recorded:

{"tool":"trace_run","arguments":{"run_id":"run_0001","trace_level":"summary"}}

{
  "run_id": "run_0001",
  "outcome": {
    "ok": true,
    "status": "success",
    "error": null
  },
  "trace": {
    "run_id": "run_0001",
    "level": "summary",
    "steps": [
      {
        "id": "lobby-on",
        "outcome": {
          "ok": true,
          "status": "success",
          "error": null
        },
        "calls": "light.lobby",
        "output": {
          "on": true,
          "brightness": 100
        }
      },
      {
        "id": "dock-unlock",
        "outcome": {
          "ok": true,
          "status": "success",
          "error": null
        },
        "calls": "lock.dock_door",
        "output": {
          "locked": false
        }
      }
    ]
  }
}

The returned level is the smaller of the requested level and the recorded level. The record is full; the request said summary; summary came back: the input members are gone, the outputs and outcomes remain.

Choose what to keep#

Why was run_0001 still there to fetch? Because the welcome card in Chapter 1 declared:

{ "retention": "session" }

Retention is the server's declared storage rule. session means a started Run's record is kept until the session ends. The execute request can override the rule, in either direction, with one optional member: store.

• store absent: the declared retention governs. On Harborview, runs are kept.
• store: true: the Run must be retained, whatever the default says.
• store: false: the Run is never retained.

And one rule to hold on to: store and trace_level are independent. trace_level decides how much trace rides the response right now. store decides whether the record is kept for later. Neither ever decides the other.

Prove it in both directions.

Keep nothing. It is now 21:30. Take a quick peek: is anyone still in the building? A read-only glance that is worth nothing tomorrow, so say store: false:

{
  "tool": "run_flow",
  "arguments": {
    "flow": {
      "name": "who is still here",
      "steps": [
        {
          "id": "occ",
          "do": "call",
          "device": "sensor.occupancy",
          "action": "read"
        }
      ]
    },
    "mode": "immediate",
    "trace_level": "summary",
    "store": false
  }
}

{
  "run_id": "run_0007",
  "outcome": {
    "ok": true,
    "status": "success",
    "error": null
  },
  "trace": {
    "run_id": "run_0007",
    "level": "summary",
    "steps": [
      {
        "id": "occ",
        "outcome": {
          "ok": true,
          "status": "success",
          "error": null
        },
        "calls": "sensor.occupancy",
        "output": {
          "count": 23
        }
      }
    ]
  }
}

The inline door worked as always: the summary trace rode the response, and it says 23 people are still inside. But nothing was kept. Ask the fetch door for the same run:

{"tool":"trace_run","arguments":{"run_id":"run_0007"}}

{
  "error": {
    "kind": "not_found",
    "message": "unknown run"
  }
}

Unknown, expired, refused, never-stored, and unauthorized run ids all return this same not_found error, with the same message. That is deliberate. The fetch door cannot be used to probe for another caller's runs, because "it never existed" and "it is not yours" look identical.

Keep everything, report nothing. Now close the day: lobby lights off. This time make the opposite choice. Keep the record, but skip the inline trace entirely:

{
  "tool": "run_flow",
  "arguments": {
    "flow": {
      "name": "close the lobby",
      "steps": [
        {
          "id": "off",
          "do": "call",
          "device": "light.lobby",
          "action": "turn_off"
        }
      ]
    },
    "mode": "immediate",
    "trace_level": "none",
    "store": true
  }
}

{
  "run_id": "run_0008",
  "outcome": {
    "ok": true,
    "status": "success",
    "error": null
  }
}

No trace member at all. trace_level: "none" closed the inline door for this one run; the response carries the run id and the outcome, nothing more. Yet the record exists, and at the server's full ceiling:

{"tool":"trace_run","arguments":{"run_id":"run_0008"}}

{
  "run_id": "run_0008",
  "outcome": {
    "ok": true,
    "status": "success",
    "error": null
  },
  "trace": {
    "run_id": "run_0008",
    "level": "full",
    "steps": [
      {
        "id": "off",
        "calls": "light.lobby",
        "input": {
          "action": "turn_off"
        },
        "output": {
          "on": false,
          "brightness": 0
        },
        "outcome": {
          "ok": true,
          "status": "success",
          "error": null
        }
      }
    ]
  }
}

Recording and reporting are different acts. The server recorded everything, the response reported nothing, and the fetch door read the record back. trace_level never decided retention; store never decided the trace.

One caution: "none" belongs to the execute request only. The fetch door speaks exactly two levels, summary and full:

{"tool":"trace_run","arguments":{"run_id":"run_0008","trace_level":"none"}}

{
  "error": {
    "kind": "invalid_request",
    "message": "unrecognized trace_level 'none' for a trace fetch: \"summary\" or \"full\" (\"none\" applies to run_flow only)"
  }
}

Recover from drift#

Now demonstrate catalog drift. Harborview's demo harness can remove light.floor2_west:

{"tool":"world_advance","arguments":{"remove_device":"light.floor2_west"}}

{
  "time": "21:30",
  "fired": []
}

Removing a visible entry changes the catalog token from cv_590fefad56a7a504 to cv_d4e1f4237c537ac0. An execute request still pinned to the old token is refused before any side effect:

{
  "tool": "run_flow",
  "arguments": {
    "flow": {
      "name": "open Harborview for the day",
      "steps": [
        { "id": "lobby-on", "do": "call", "device": "light.lobby", "action": "turn_on" },
        { "id": "dock-unlock", "do": "call", "device": "lock.dock_door", "action": "unlock" }
      ]
    },
    "mode": "immediate",
    "expected_catalog_version": "cv_590fefad56a7a504",
    "trace_level": "full"
  }
}

{
  "run_id": "run_0009",
  "outcome": {
    "ok": false,
    "status": "catalog_drift",
    "error": "the catalog changed since validation"
  }
}

The refusal still returned a run id. But a refused Run is never retained, whatever store would have said, so the fetch door answers with the same not_found as before:

{"tool":"trace_run","arguments":{"run_id":"run_0009"}}

{
  "error": {
    "kind": "not_found",
    "message": "unknown run"
  }
}

Recovery follows the loop again:

1. Retrieve the entries referenced by the program.
2. Validate against the current catalog.
3. Read the new plan.
4. Reapply the Host risk gate.
5. Execute with the new token.
6. Use a fresh idempotency key if the runnable or input changed.

The refusal is not a crash and should not be blindly retried. It is the server saying that the premise used for approval is stale.

See Execution, traces, and decisions, Drift and recovery, Idempotency, Part II §6.6, and Part II §7.5.