# How I Built Quizco's AI Quiz Agent > A breakdown of the planner, validation pipeline, human review loop, and admin dashboard behind Quizco's AI quiz agent. By Vishwajeet Raj · 2026-03-25 · https://vishwajeet.co/blog/how-i-built-quizco-ai-agent --- I did not want Quizco's AI agent to be a one-shot prompt that blindly spits out trivia. I wanted it to act more like a product teammate: - look at what people are already playing - decide whether new quizzes are even needed - generate drafts in a strict structure - verify and de-duplicate them - hand the final decision to a human reviewer That led me to an architecture that is much closer to an agent loop than a simple "generate quiz" button. I also wanted the post to have a single systems-view image, so I made this architecture diagram for the agent loop: This is a **single Node.js backend service** architecture; planner, writer, and verifier are internal modules inside the same backend.
Architecture diagram for the Quizco AI agent showing triggers, planner context, generation, quality gates, review, publishing, and memory feedback.
> _(interactive demo — view it on the live post)_ ## Planning before generation The most important design decision was making planning a first-class step. Instead of asking a model to directly invent quizzes, I built a planner that can inspect the state of the platform first. That planner gets a small tool belt and can decide that the correct answer is to do nothing. ```txt title="Planner tool contract" get_platform_state get_recent_agent_history get_agent_memory web_search ``` That mattered for two reasons. First, the agent needs to respect product reality. If there are already too many pending quizzes, or if a topic was just rejected, generating more of the same thing is noise. Second, freshness is uneven. Some topics can be chosen from internal demand alone, while others benefit from current web context. The planner only reaches for web search when that freshness actually improves the decision. I also added a deterministic fallback ranker. If the planner call fails, Quizco can still choose topics based on attempts, active quiz counts, and penalties for recently pending, rejected, or already-approved topics. That keeps the system resilient instead of brittle. ## Generation is only one stage of the pipeline Once the planner selects topics, the writer prompt asks for a strict quiz JSON object, not free-form prose. The draft must include: - a title and description - a topic and tags - a format such as `standard`, `speed_round`, `deep_dive`, or `streak` - an `agentConfidence` score - exactly 5 questions - exactly 4 options per question After that, the draft goes through multiple filters: - safety validation blocks harmful or tragedy-heavy topics - structure validation catches broken question counts, duplicate prompts, or invalid options - citation validation ensures web-influenced drafts still carry sources - the verifier uses web search to fact-check questions that contain factual claims - the embedding check blocks drafts that are too close to existing quizzes This was the difference between "AI content" and "reviewable product content." I wanted the agent to earn its way into the inbox. ## Human review was non-negotiable I built a dedicated React admin surface at `/agent` instead of hiding the workflow behind logs. The dashboard is split into a few focused views: - `Briefing` summarizes the latest run and current agent state - `Quiz Inbox` shows pending quizzes with confidence, topic context, and approve/reject actions - `Skipped` records why the agent chose not to generate certain candidates - `Run History` exposes recent runs for auditability - `Platform Health` shows preflight status, last run info, and run controls > _(interactive demo — view it on the live post)_ That dashboard turns the agent from a black box into an inspectable system. If a quiz is rejected, the rejection reason is saved. If a quiz is approved, it is converted into a real `Quiz` and its `Question` records. > _(interactive demo — view it on the live post)_ ## Why the data model mattered I split the system into a few purpose-built collections so the pipeline stays auditable. | Collection | Why it exists | What it stores | | --- | --- | --- | | `AgentRun` | Run-level observability | trigger, duration, planner action, topics selected, citations, skips, errors | | `AgentMemory` | Durable learning between runs | topic performance, review insights, recent runs | | `QuizPending` | Human review buffer | generated draft, citations, verification report, confidence, review status | | `Quiz` + `Question` | Published learning content | approved quizzes and their questions | | `Recommendation` | Recommendation surface for players | recommendation metadata and expiry windows | That last piece is worth calling out honestly: I already laid down the `Recommendation` model and the toast-based recommendation UI in the frontend, but the current `runAgentCycle` is still focused on generating and reviewing quizzes rather than automatically dispatching recommendations. The surface is there; the publishing loop came first. ## What I would improve next If I keep pushing Quizco's agent further, the next upgrades are pretty clear. - automatic recommendation dispatch so approved agent content can reach the right users without manual glue - richer health analytics so the dashboard matches the ambition of the backend pipeline - stronger reviewer tooling around why a draft passed validation and which sources were used to verify it The biggest takeaway from building this was simple: the useful part of an AI agent is not the model call. It is the loop around the model call. Planning, guardrails, validation, memory, and review are what made Quizco's agent feel dependable enough to ship.