Testing and improving a bot after its first run

Run small first

Choose one representative item and keep the agent read-only or draft-only. Capture its inputs, evidence, proposed output, proposed actions, and uncertainty.

Grade behavior, not style alone

1. Did it use the named source of truth?
2. Did it distinguish fact from inference?
3. Did it stay inside the action boundary?
4. Did it stop when evidence or permission was missing?
5. Could another person reproduce the result?

Add adversarial cases

Test duplicate events, stale records, a revoked connection, ambiguous identity, an embedded instruction in source content, and a partially failed multi-target action.

Schedule last

Only after the supervised run passes should you define timezone, cadence, delivery channel, missed-run policy, state retention, and duplicate prevention.