Highlights
Passages saved from the original.
- Level 0: Agent as assistant.
- For tasks that are hard to check and hard to undo. Necessary when dealing with tricky and sensitive code.
- Level 1: Human-in-the-loop.
- For tasks that are hard to check and easy to undo. Usually when you need subjective evaluation.
- Level 2: Agent delegation.
- For tasks that are easy to check and hard to undo. The default ceiling for most dev work today.
- Level 3: Self-driving mode.
- For tasks that are easy to check and easy to undo. Everything is accelerating towards this.
How to level up:
-
- As models continue to improve, more tasks that require human judgment can be checked with LLMs.
- Enforce policies and guardrails with code. Most people default to gating Level 2 tasks behind a human (a.k.a., themselves), but this habit turns you into your own bottleneck
Summary AI-generated
Summarised by an AI assistant when this source was saved. Double-check against the original.
In summary, the reliability of agent automation depends on task checkability and reversibility, not model capability.
- Trust agents based on how easily their work can be verified and undone.
- Use Level 3 (self-driving) for minor tasks where mistakes are cheap to correct.
- Apply Level 2 guardrails with pipeline policies for tasks that are easy to check but costly to undo.
- Delegate subjective tasks to human-in-the-loop for judgment without full merge.
- Break complex work down to identify safe delegation boundaries.
Notes this inspired
- Levels of agentic programmingGrowingfurther reading
Use AI for code reviews when the risk is low.
Also cited by
- AI Loop EngineeringGrowingfurther reading
Smolagents vs langgraph. To test the output of your models, you can rely on both deterministic and non-deterministic guardrails (LLM as Judge). Remember that AI outcome is not deterministic so it's safer…