Highlights
What we build:
ToolSchemaValidator: automated checks that every registered tool function has a correct, non-ambiguous JSON Schema definition before code reachesmainPromptRobustnessHarness: a deterministic test suite that probes system prompts and instruction strings for injection vectors, token-budget overruns, and persona driftAgentLogicTestSuite: pytest-based unit and integration tests for routing logic, fallback chains, andTaskResultcontract adherenceSecurityComplianceScanner: static analysis layer that flags hardcoded secrets, uncheckedeval/exec, and unsafe deserialization inside agent codeCIGateOrchestrator: GitHub Actions job graph that sequences all four checks, fails fast on critical gates, and emits a machine-readable quality report consumed by the React dashboard
Connection to L62: L62 established a GitHub Actions skeleton with code linting and basic unit-test invocation. L63 plugs purpose-built AI-specific gates into that skeleton — the
on: pushtriggers, cache layers, and artifact upload patterns from L62 are reused without modification.
Enables L64: The CI output includes a build_manifest.json that L64’s Docker build step reads to select the correct base image, copy the validated test artifacts, and tag the container with a CI-verified digest.
Architecture Context
CI for traditional software tests whether code compiles and existing behaviours are preserved. For a VAIA, three additional failure modes exist that no standard linter or coverage tool catches:
Schema drift — a tool function’s Python signature changes but its JSON Schema declaration does not, causing the LLM to generate tool calls with wrong argument names at runtime.
Prompt brittleness — a prompt that passes manual review silently degrades under paraphrased or adversarially-constructed inputs, producing hallucinated tool calls or persona leakage.
Agent logic contract violation — a refactored routing branch stops emitting the
TaskResultfields that downstream sub-agents depend on, breaking the bridging contract established in L46–L57.
L63 places gates for all three failure modes between developer git push and the merge queue, making them impossible to bypass without explicit override.
The
CIGateOrchestratorruns four sequential jobs. Jobs 1 and 2 (linting and schema validation) are cheap and fail fast. Jobs 3 and 4 (prompt robustness and security) are heavier and run in parallel after jobs 1–2 pass. The React dashboard polls a FastAPI endpoint that streams real-time job state so engineers see which gate is blocking a PR within seconds.
Place in the 90-lesson arc: L63 sits at the inflection point where the curriculum shifts from building agent capability to hardening agent deployability. Every lesson from L64 onward assumes a CI-verified artifact.
Preparing for a distributed systems interview?
→Download the free Interview Pack
→ Subscribe now to access source code repository - 200 + coding lessons
