{ "module": "1 — The Execution Loop", "course": "Master Course — Harness Engineering", "version": "1.0.0", "duration_minutes": 45, "total_questions": 24, "bloom_distribution": { "target": "20% recall / 40% application / 40% analysis-design", "actual": { "recall": 6, "application": 9, "analysis": 9 } }, "passing_score_percent": 70, "questions": [ { "id": "Q01", "bloom": "recall", "type": "multiple_choice", "prompt": "How many loop architectures does Module 1 define? Name the source of the ReAct pattern.", "options": ["3; ReAct from LangChain", "5; ReAct from Yao et al. 2022 (arXiv:2210.03629)", "7; ReAct from Anthropic", "4; ReAct from AutoGen"], "answer_index": 1, "rationale": "Five architectures (ReAct, Plan-then-Execute, Graph-based, Dumb Loop, Conversation-driven). ReAct is from Yao et al. 2022, arXiv:2210.03629." }, { "id": "Q02", "bloom": "recall", "type": "multiple_choice", "prompt": "Plan-then-Execute is approximately how much faster than ReAct on complex tasks, and from what source?", "options": ["2× (Anthropic blog)", "3.6× (LLMCompiler data)", "10× (LangGraph docs)", "No meaningful difference"], "answer_index": 1, "rationale": "3.6× per LLMCompiler. Precision matters: holds on complex, decomposable tasks WHERE THE PLAN IS CORRECT. Inverts when the plan is wrong." }, { "id": "Q03", "bloom": "recall", "type": "multiple_choice", "prompt": "State the future-proof test.", "options": [ "Does the harness pass all benchmarks?", "When the underlying model upgrades, does agent performance improve? If yes, harness isn't constraining; if no, harness is the constraint.", "Does the harness work with multiple model providers?", "Can the harness be deployed without code changes?" ], "answer_index": 1, "rationale": "The test for whether your harness fights the model. Most over-engineered harnesses fail it. The dumb-loop philosophy exists to pass it." }, { "id": "Q04", "bloom": "recall", "type": "multiple_choice", "prompt": "How many stop conditions does a production loop need?", "options": ["At least one", "All five (token budget, end_turn, max-iter, error-threshold, human)", "Exactly two", "None — the model self-regulates"], "answer_index": 1, "rationale": "A production loop has all five. A toy loop has one or two. Missing any is a production defect (the infinite loop problem)." }, { "id": "Q05", "bloom": "recall", "type": "multiple_choice", "prompt": "Name the four subagent patterns.", "options": [ "Queue, stack, heap, pipe", "Agents-as-tools, Handoffs, Fork, Worktree", "Master-slave, peer-to-peer, broadcast, pipeline", "Static, dynamic, hybrid, recursive" ], "answer_index": 1, "rationale": "Agents-as-tools (sync, parent in control), Handoffs (terminal), Fork (isolated), Worktree (parallel + merge)." }, { "id": "Q06", "bloom": "recall", "type": "multiple_choice", "prompt": "Why does the per-turn observability payload use input_hash/output_hash instead of full content?", "options": [ "Hashes are faster to compute", "Tool outputs are 67.6% of context (Module 0.3) and may contain secrets. Hashes dedup, detect stuck loops, enable replay. Full content → separate access-controlled store.", "Full content is not structured", "Hashes compress better" ], "answer_index": 1, "rationale": "The two reasons: size (67.6% of context) and sensitivity (secrets). Hashes give dedup/stuck-detection/replay; full content goes elsewhere." }, { "id": "Q07", "bloom": "application", "type": "multiple_choice", "prompt": "You're building a debugging agent. ReAct or Plan-then-Execute?", "options": [ "Plan-then-Execute — it's 3.6× faster", "ReAct — debugging is unpredictable; you cannot plan upfront. Plan-then-Execute would commit to a wrong plan and inherit the error.", "Either works equally well", "Neither — use conversation-driven" ], "answer_index": 1, "rationale": "Task predictability decides. Debugging is discovered-as-you-go → ReAct. The 3.6× is irrelevant if the plan is wrong." }, { "id": "Q08", "bloom": "application", "type": "multiple_choice", "prompt": "A CI/CD pipeline agent. Which architecture?", "options": [ "ReAct — for maximum flexibility", "Plan-then-Execute — steps are predictable/known; the speed applies; brittleness doesn't matter because the plan is reliably correct", "Conversation-driven — for stakeholder input", "Dumb loop — to co-evolve with the model" ], "answer_index": 1, "rationale": "Predictable task → planning. The plan is correct (steps are known), so the 3.6× speed applies and the brittleness never bites." }, { "id": "Q09", "bloom": "application", "type": "multiple_choice", "prompt": "Your agent keeps calling the same failing tool with the same input, looping forever. Which stop condition is missing?", "options": ["Token budget", "Error threshold (N consecutive errors → halt)", "Human interrupt", "Max iterations only"], "answer_index": 1, "rationale": "Same tool + same input + same error = stuck loop. The error threshold catches it. (Max iterations would also eventually catch it, but error-threshold is the targeted fix — Module 7.2.)" }, { "id": "Q10", "bloom": "application", "type": "multiple_choice", "prompt": "You need to audit what a subagent did after it runs. Which subagent pattern must you use?", "options": ["Handoffs (parent loses visibility)", "Fork (isolated, no shared state)", "Agents-as-tools or Worktree (parent keeps visibility)", "Conversation-driven"], "answer_index": 2, "rationale": "If you need to audit, you need parent visibility. Handoffs and fork lose it; agents-as-tools and worktree keep it." }, { "id": "Q11", "bloom": "application", "type": "multiple_choice", "prompt": "You observe input_hash and output_hash identical across 5 consecutive turns. What is happening?", "options": [ "Normal operation", "A STUCK LOOP — the model calls the same tool with the same input and gets the same result. No progress. Triggers stuck-loop detection.", "The trace is corrupted", "The model has finished" ], "answer_index": 1, "rationale": "Same input + same output = no progress. This is the stuck-loop signal that observability hashes are designed to catch." }, { "id": "Q12", "bloom": "application", "type": "multiple_choice", "prompt": "You want to add permission gates to an existing loop without modifying its core. What pattern do you use?", "options": [ "Rewrite the loop from scratch", "Wrap callModel and executeTool with a higher-order function (the observability-wrapper pattern, reused for permissions)", "Add permission checks to the system prompt", "Switch to a graph-based architecture" ], "answer_index": 1, "rationale": "The loop's callModel/executeTool interface is the extension point. Wrap it — same pattern as observability. Module 6 and 11 reuse it." }, { "id": "Q13", "bloom": "application", "type": "multiple_choice", "prompt": "oh-my-opencode's hierarchy: Prometheus → Atlas → Junior. Which two subagent patterns are mixed?", "options": [ "Handoffs then fork", "Agents-as-tools (Prometheus→Atlas, structured summary returns) then worktree (Atlas→Junior, parallel + merge)", "Conversation-driven throughout", "Fork throughout" ], "answer_index": 1, "rationale": "Real systems mix patterns. Prometheus→Atlas = agents-as-tools (sync, summary). Atlas→Junior = worktree (parallel, merge). The mix is itself a scored rubric decision." }, { "id": "Q14", "bloom": "application", "type": "multiple_choice", "prompt": "A compliance-governed multi-approval agent (regulated workflow). Which architecture?", "options": [ "Dumb loop — maximal model freedom", "ReAct — for recovery", "Graph-based (LangGraph) — the process IS the product; explicit state machine, native checkpointing, interrupt() for HITL at approval nodes", "Conversation-driven" ], "answer_index": 2, "rationale": "Graph-based when the process is the product. Regulated workflows need explicit, auditable transitions and HITL checkpoints — LangGraph's native strengths." }, { "id": "Q15", "bloom": "analysis", "type": "multiple_choice", "prompt": "Why does the 3.6× Plan-then-Execute speedup NOT mean Plan-then-Execute is universally better than ReAct?", "options": [ "The benchmark is flawed", "The speedup holds only on complex, decomposable tasks WHERE THE PLAN IS CORRECT. It inverts when the plan is wrong: every step inherits the error, no recovery. Task predictability decides.", "ReAct is newer so it's better", "Plan-then-Execute uses more tokens" ], "answer_index": 1, "rationale": "Speed and robustness are on different axes. Predictable tasks reward planning; unpredictable tasks punish it. 'Faster' ≠ 'better' universally." }, { "id": "Q16", "bloom": "analysis", "type": "multiple_choice", "prompt": "A senior engineer adds a sophisticated LangGraph state machine 'for control.' What is the latent risk?", "options": [ "No risk — more control is always better", "The graph fights model capability growth. A rigid graph encoding yesterday's best practice becomes tomorrow's constraint. If the model could handle a transition naturally, the forced path is overhead.", "LangGraph is deprecated", "The graph adds too many tokens" ], "answer_index": 1, "rationale": "The 'fights model capability growth' problem. The future-proof test catches it: if model upgrades don't improve performance, the harness is the constraint. This is the senior-engineer temptation Module 1 warns against." }, { "id": "Q17", "bloom": "analysis", "type": "multiple_choice", "prompt": "Why must the token budget be cumulative across the session, not a per-call max_tokens?", "options": [ "Per-call is harder to implement", "Per-call max_tokens limits ONE completion. A session budget limits cumulative spend across ALL calls in the run. Without the cumulative budget, a runaway session = five-figure bill.", "Cumulative is more accurate", "Per-call violates the API terms" ], "answer_index": 1, "rationale": "The distinction is load-bearing for cost control. Per-call limits are not a budget — they cap individual completions. The session budget is the financial stop condition." }, { "id": "Q18", "bloom": "analysis", "type": "multiple_choice", "prompt": "A subagent returns its full context to the parent. What is the harm, and what is the cure?", "options": [ "No harm — full context is helpful", "Pollutes the parent's context window AND re-bills tokens. Cure: bound returns to 1-2k tokens; summary-only by default; full output via JIT retrieval.", "Harm: the subagent loses its memory", "Cure: increase the parent's context window" ], "answer_index": 1, "rationale": "The unbounded-return anti-pattern. Same disease as Fleet F02's orchestrator-context-trap. The 1-2k bound is the canonical cure." }, { "id": "Q19", "bloom": "analysis", "type": "multiple_choice", "prompt": "Pi logs minimally; Claude Code logs extensively. Is Pi's minimal logging a defect?", "options": [ "Yes — all production loops need full observability", "No. Observability is a thickness-spectrum decision. Pi is above the FLOOR (it logs something); a loop emitting nothing is below the floor. Pi's minimalism is correct for its use case.", "Yes — Pi should match Claude Code", "Only if Pi is used in enterprise" ], "answer_index": 1, "rationale": "There is a floor (emit SOMETHING per turn) but no ceiling. Pi is above the floor; no-logging is below it. Thickness spectrum applies to observability too." }, { "id": "Q20", "bloom": "analysis", "type": "multiple_choice", "prompt": "The loop's callModel/executeTool interface is called 'the extension point.' Why does this matter?", "options": [ "It's where bugs happen", "It's where cross-cutting concerns (observability, permissions, security, rate limiting, cost enforcement) plug in via wrappers WITHOUT modifying the core. Learn the wrapper pattern once; reuse for all of them.", "It's where the model is called", "It's documented in the API spec" ], "answer_index": 1, "rationale": "The wrapper pattern is the module's reusable engineering insight. The interface boundary is what makes a loop extensible without forking. Modules 6, 10, 11 all depend on it." }, { "id": "Q21", "bloom": "analysis", "type": "multiple_choice", "prompt": "A colleague says 'there's one correct subagent pattern — pick the best one.' What does Module 1 teach?", "options": [ "The colleague is correct — agents-as-tools is best", "Wrong. Real systems MIX patterns based on the control/visibility tradeoff. oh-my-opencode mixes agents-as-tools and worktree. The rubric scores the MIX, not a single choice.", "Worktree is always best", "Handoffs are deprecated" ], "answer_index": 1, "rationale": "The anti-pattern of seeking one true pattern. The decision is the MIX, conditioned on whether you need parent visibility (audit) and parallelism." }, { "id": "Q22", "bloom": "analysis", "type": "multiple_choice", "prompt": "Why is the 'shape predicts the failure mode' framing useful when reading a loop architecture?", "options": [ "It helps you draw diagrams", "Cycles can loop; lines can't recover; graphs can fight the model; meshes can talk past each other. Knowing the shape tells you what to defend against before reading the code.", "It's required for the rubric", "It helps with naming" ], "answer_index": 1, "rationale": "The framing lets you predict failure modes from architecture alone — before reading code. ReAct (cycle) → error compounding. Plan-then-Execute (line) → no recovery. Etc." }, { "id": "Q23", "bloom": "analysis", "type": "multiple_choice", "prompt": "The infinite loop problem has three causes. A team's agent loops forever because 'the context filled with noise and the model lost the original task.' Which module cures this?", "options": [ "Module 7 (Error Handling) — it's a stuck loop", "Module 3 (Context Management) + the token budget. Context compaction keeps the task visible; the budget halts the run.", "Module 6 (Permission)", "Module 11 (Security)" ], "answer_index": 1, "rationale": "Context rot is a Module 3 problem. The cure is compaction (keep the task visible) plus the token-budget stop (halt the runaway). This is cause #3 of the infinite loop problem." }, { "id": "Q24", "bloom": "analysis", "type": "multiple_choice", "prompt": "You're auditing a harness and find its loop emits no structured per-turn data. What is your verdict?", "options": [ "Acceptable for a thin harness", "Below the observability floor. Un-debuggable. Cured by the withObservability wrapper. There is no excuse to ship a loop with no per-turn payload.", "Acceptable if it's a prototype", "Only a problem in enterprise" ], "answer_index": 1, "rationale": "The floor is non-negotiable. Pi logs minimally but logs SOMETHING. No logging at all is below the floor and un-debuggable. The wrapper makes adding it trivial." } ] }