When an agent should ask a human — and how to encode it

The trouble with one threshold#

The first policy almost everyone writes is “ask a human above $500”. It feels safe. In practice it fails in both directions: a scraper buying 4,000 datasets at $12 each never trips it, and a scheduled GPU reservation for $520 pages someone at 2am every Sunday.

A useful approval rule describes risk, not size. Size is one input. The others are novelty, velocity and reversibility.

Four signals worth encoding#

• Size — the amount of this authorisation relative to the agent’s monthly ceiling, not an absolute number.
• Novelty — a merchant the agent has never paid, or a category it has not bought from in 30 days.
• Velocity — spend in the last hour compared with the same hour last week. Loops show up here first.
• Reversibility — a refundable API credit top-up is not the same as a wire to a new contractor.

Each signal on its own produces noise. Two together are usually worth a human’s attention. Three almost always are.

Writing it down#

Nebula policies are evaluated at authorisation time, before money moves. An approval rule is a condition that, instead of declining, holds the authorisation and asks a named person or channel.

await nebula.policies.update('pol_research', {
  approvals: [
    {
      // New merchant AND more than 10% of the monthly ceiling
      when: { merchant: 'new', share_of_limit: { gt: 0.1 } },
      ask: 'slack:#spend-approvals',
      timeout: '15m',
      on_timeout: 'decline',
    },
    {
      // Spend this hour is 3× the same hour last week
      when: { velocity: { window: '1h', vs: '1w', gt: 3 } },
      ask: 'oncall:finance',
      on_timeout: 'freeze_card',
    },
  ],
})

Tuning without guessing#

Before you turn a rule on, run it in shadow mode for a week. Nebula evaluates it against live traffic and records what it would have done, without holding anything. You get a list of the exact purchases that would have paged someone, and you can decide whether each one deserved to.

A good target is fewer than three approval requests per agent per week. More than that and people start approving without reading, which is worse than not asking at all.

Every decision is a record#

Approvals, declines and timeouts are written to the same audit log as the authorisation itself: who was asked, who answered, how long it took and which rule fired. When the auditor asks how agent spend is controlled, the answer is a query, not a meeting.