Loading...
Loading...
HCIE is not just an Intelligent Tutoring System — it is a research instrument that doubles as a production ITS. Every design decision traces back to one constraint: every learner interaction must be reproducible, auditable, and traceable to a specific algorithm run — even months later.
Why event-sourcing? Because the core thesis requires it: ADC governance depends on every trajectory being replayable and attributable to an exact algorithm state.
A learner interaction is written to Postgres AND an outbox_events row atomically in a single transaction. The outbox row is the immutable record of truth. Kafka fans it out to every consumer (trajectory recording, projection updates, audit). This means: if you know the run_id and the interaction sequence, you can replay the entire learning trajectory deterministically.
The outbox pattern ensures that no interaction is ever lost between the API response and the Kafka message. The database is the source of truth; Kafka is a broadcast mechanism.
When a learner submits an answer, the API must do two things atomically: record the interaction and guarantee Kafka will eventually receive it. A naive approach — write to DB, then produce to Kafka — can fail between steps, losing the event. The transactional outbox pattern solves this: the API writes both the interactions row and an outbox_events row in a single Postgres transaction. A Kafka producer polls the outbox table, publishes pending rows, and marks them delivered. This is an at-least-once guarantee; consumers are idempotent by interaction_id.
The sealed run (run-94a3b8ba, N=96,727) was derived from experiment_trajectories which was populated by this exact pipeline. The reproducibility claim holds because the outbox guarantees no events are silently dropped. You can replay any experiment by filtering experiment_run_id.
Every learner interaction enters unified_brain.py and triggers four interleaved computations: mastery estimation, JT scoring, ensemble fusion, and recommendation policy.
core/03_ensemble/unified_brain.pyState-space mastery model. Each concept has a hidden mastery state μ with variance σ². A correct answer updates μ upward via Kalman gain; incorrect pulls it down. The gain adapts to observation noise, unlike BKT's fixed slip/guess.
core/03_ensemble/unified_brain.pyConjugate prior over mastery. Maintains Beta(α, β) per concept. Correct → α += 1, incorrect → β += 1. Population prior from Yudelson 2013: Beta(α₀, β₀) initialized from concept-level base rates, not uniform Beta(1,1).
core/03_ensemble/unified_brain.pyCombines Kalman and Bayesian estimates via learned weights. In V1 the weights are near-uniform (cv 0.03–0.06) — effectively an average. V2 redesign cuts BoundedStability (Lyapunov misnomer, corr=0.92 with Bayesian) to a 2-learner fusion.
core/03_ensemble/unified_brain.pyJoint Traversal: scores the next-best concept recommendation along 6 axes. Each axis captures a different learning signal. The JT score drives which concept gets recommended next.
| Dimension | Column | Formula | Complexity | Status |
|---|---|---|---|---|
| ΔM (Mastery delta) | jt_delta_m_contribution | μ_after − μ_before | O(1) | PASS |
| T_realized (Transfer) | jt_transfer_contribution | Σ w_ij · μ_j for j ∈ prereqs(k) | O(indeg(k)) | DISCLOSE |
| T_prospective | jt_transfer_prospective_contribution | hardcoded 0.0 (5 formulations failed) | O(1) | DORMANT |
| Challenge | jt_challenge_contribution | avg_difficulty × recent_err_rate | O(1) | WARN: latency proxy |
| Uncertainty | jt_uncertainty_contribution | 1 − |2μ − 1| (peak at 0.5) | O(1) | PASS |
| ZPD | jt_zpd_contribution | sigmoid((μ − 0.4)/0.2) | O(1) | SATURATED ≈0.97 |
T_realized is the only O(indeg(k)) dimension — requires an adjacency list walk over prerequisite concepts. In practice, mean indeg ≈ 2–4 in educational KGs, making this effectively O(1) at runtime scale.
An online multi-armed bandit selects which presentation modality (text / MCQ / video / audio) to offer each real learner. It learns per-learner per-concept preferences through closed-loop feedback.
The bandit is the only component that deliberately injects stochasticity into the recommendation path. Every other component (JT, KF, Bayesian) is deterministic given the same inputs. The bandit uses Thompson Sampling over a Beta(α, β) posterior per arm:
The bandit ONLY runs for real learners (is_research_subject=true AND NOT synthetic). Synthetic users (seeded for sealed experiments) receive fixed modality assignments. This prevents bandit learning from contaminating the sealed trajectory distribution.
selection_metrics tableEach consumer has a single responsibility. They are idempotent: replaying the same event_id twice produces the same final state. This is how sealed experiments are reconstructed.
Writes one row per interaction with full JT decomposition (6 jt_* columns), experiment_run_id, mastery snapshot. This is the primary evidence table used by ADC and all cascade grounding scripts.
Maintains the "current best estimate" of learner mastery per concept. Reads P2 recommendation authority chain. 20h snappy outage was here — snappy now baked into Dockerfile.
Updates the live mastery state used by the next recommendation cycle. Keyed by user_id+concept_id. Serves the /v3/learner/* endpoints that feed the frontend profile page.
Append-only audit trail for governance. Every event leaves a permanent record regardless of downstream consumer failures. Used by the ADC live router (designed; currently only post-hoc sealer runs).
The Adaptive Dimension Controller reads the sealed trajectory store and classifies each JT dimension as informative or floor-noise. It never writes to the runtime — it is a pure read path.
The ADC is a governance instrument: it tells researchers which JT dimensions are actually carrying signal versus which are dominated by sigmoid floor noise. The classification runs post-hoc on the sealed run, never during recommendation.
sigmoid(−2.5) ≈ 0.076. Any dimension whose formula bottoms out at sigmoid output will score above α_floor = 0.01 even for random inputs. This made ZPD and T_prospective appear informative in V1. The shuffled-DAG control (randomize graph edges, preserve degree) was the decisive experiment: real topology adds +0.020 AUC (p=5e-5) above the floor.
run-94a3b8ba, N=96,727 Junyi learners46/46 cascade steps terminal (42 PASS + 4 DEFERRED) via decision-aware reframing. The 4 DEFERRED steps reference jt_design_decisions.json — open design choices that are explicitly disclosed rather than silently resolved.
All external traffic enters through the nginx gateway on port 80. The gateway routes /api/* to the FastAPI service and /* to the Next.js frontend.
Reverse proxy. Routes /api/* → api:8011, /* → frontend:3000. Fixed: listen [::]:80 for dual-stack IPv4+IPv6.
Core application server. All /v3/* endpoints. bcrypt + snappy baked in Dockerfile.cutover.
SSR + client SPA. Reads NEXT_PUBLIC_BACKEND_URL ARG at build time (now wired).
Message bus. Topics: hcie.interactions, hcie.learner-updates. Partitioned by user_id.
Kafka coordination. Required for Kafka 2.x.
Primary data store. interactions, experiment_trajectories, learner_projections, outbox_events, selection_metrics.
trajectory-recorder, projection-consumer, learning-consumer, audit-consumer. Each is a separate long-lived Python process.
Trace a learner submitting a single answer from browser click to trajectory row.
Next.js SPA sends POST /v3/learner/attempt/{user_id} with {concept_id, answer, modality, response_time_ms}. Auth token sent as Bearer header.
nginx matches the /v3/* location block and proxy_passes to the api container. No TLS termination at gateway in local/research mode.
its_runtime_service.py validates the attempt. Looks up concept metadata (difficulty, prerequisites from the KG). Loads current learner mastery state from learner_projections.
KF update: μ, σ² → posterior. Bayesian update: α, β += 1. Ensemble fusion: ŷ. JT scorer: 6 dimensions × O(indeg) complexity. Bandit: for real learners, Thompson sample selects next modality.
Single Postgres transaction: INSERT INTO interactions + INSERT INTO outbox_events. If either fails, both roll back. The attempt is not considered recorded until this commits.
Response: {mastery: 0.73, next_concept: 'algebra-2', next_modality: 'video', jt_score: 0.81}. Browser renders new recommendation.
Outbox producer polls outbox_events WHERE status='pending', publishes to hcie.interactions topic, marks rows delivered. Runs every 500ms in background.
trajectory-recorder inserts into experiment_trajectories with full JT decomposition. projection-consumer updates learner_projections for the next recommendation cycle.
adaptive_dimension_controller.py reads experiment_trajectories for the sealed run. Classifies each JT dimension. Writes grounding reports. Cascade scripts consume these reports for paper validation.
Every table serves a specific contract. Mixing responsibilities (e.g., writing to experiment_trajectories at runtime) would break the sealed/live isolation that makes the research valid.
| Table | Partitioned by | Written by | Read by | Role |
|---|---|---|---|---|
interactions | user_id, concept_id | API (sync) | all consumers | Primary event log. Ground truth for every learner action. |
outbox_events | status, created_at | API (same tx) | Kafka producer | Delivery guarantee. At-least-once bridge to Kafka. |
experiment_trajectories | experiment_run_id, user_id | trajectory-recorder | ADC, grounding scripts, Figure Atlas | JT-annotated trajectory store. Has run-94a3b8ba sealed rows + live:: rows. |
learner_projections | user_id, concept_id | projection-consumer | API (next recommendation) | "Current best estimate" of mastery for live recommendation. |
selection_metrics | user_id, concept_id, arm | bandit (API) | bandit (API) | Thompson Sampling Beta posteriors per modality arm. |
outbox_events (audit) | aggregate_id, event_type | API (same tx) | audit-consumer, ADC live router (designed) | Governance append-only log. Never deleted. |
These invariants are not conventions — violating them breaks the research validity guarantee that the entire thesis rests on.
The ADC reads experiment_trajectories (sealed) and writes grounding reports only. It must not write to learner_projections, selection_metrics, or outbox_events. If it did, the sealed/live isolation collapses and the paper's reproducibility claim fails.
run-94a3b8ba and run-e49d92e6 are sealed. No new rows may be added to these runs. The thesis AUC, ADC verdicts, and all paper figures are derived from these exact row-sets. Adding rows would silently change every downstream number.
is_research_subject=true AND NOT synthetic. Allowing the bandit to run on synthetic users would corrupt both the Beta posteriors (leaking synthetic signal into real recommendations) and the sealed experiment (leaking live adaptive choices into the controlled run).
The entire event-sourcing guarantee rests on this. If they're ever separated, there's a window where an interaction is recorded but the Kafka event is never sent, leaving trajectory-recorder's table incomplete and making replay non-deterministic.
Kafka is at-least-once. The same event can arrive twice after a consumer crash. Every consumer must upsert or skip on duplicate interaction_id. trajectory-recorder checks before insert; projection-consumer uses ON CONFLICT UPDATE.