ADC · GeneralizedBKT Deep DiveTopology-ConditionalCross-Model

ADC as a Universal KT Governance Layer

ADC was built for HCIE — but its core question applies to any knowledge tracing model: "Which of this model's output dimensions are genuinely measuring learner state versus measuring structural artifacts of the data, the item format, or the algorithm's own floor effects?" Here we trace what ADC would do when applied to BKT's four probabilistic parameters, and then generalize to DKT, SAKT, and GKT.

What ADC Actually Does — Signal vs Artifact

ADC answers one question per dimension: is this number telling you something about the learner, or is it telling you about your model, your data collection process, or your item design?

Every KT model emits signals — scalar values that represent some aspect of the learner's state. The problem is that signals can carry artifact instead of information . An artifact is a value that varies due to something other than learner cognition: item format floors, sigmoid saturation, topology structure in the data, or parameter initialization priors.

ADC applies a two-gate classification to each signal dimension over the full trajectory distribution:

// ADC classification (applies to ANY model)

μ = mean(signal_dimension)
σ = std(signal_dimension)

if μ ≤ α_floor:          → FLOOR_NOISE
elif σ/μ < signal_ratio: → NEAR_CONSTANT
else:                    → INFORMATIVE

The key insight: α_floor is model-agnostic . For BKT the floor is the HMM identifiability constraint. For DKT it is dead neuron output. For HCIE it is sigmoid(−2.5)≈0.076. ADC adapts by letting the researcher set α_floor to the theoretical minimum for that model — then the two gates above do the rest.

3 VERDICT TYPES

INFORMATIVE

Mean above floor AND variance is meaningful. The signal is tracking real learner state differences.

NEAR_CONSTANT

Mean above floor but very low variance. The signal is present but not discriminating learners. Often structural.

FLOOR_NOISE

Mean is at or below the theoretical minimum for this model. The signal is dominated by a structural artifact.

🔑The generalizable insight

NEAR_CONSTANT is the most dangerous verdict. A FLOOR_NOISE signal is easy to spot (it's zero). A NEAR_CONSTANT signal looks meaningful (it has a real value) but its variance comes from noise, not learner state. BKT's P(S) is the canonical example: always ~0.08, looks real, but doesn't discriminate concepts.

⚠️Why this is hard to see without ADC

Most KT evaluation only reports aggregate AUC. A NEAR_CONSTANT dimension can pass AUC checks because it contributes a constant offset to every prediction — it doesn't hurt AUC, but it also doesn't help. You only discover it's inert when you ablate it. ADC automates that ablation decision by classifying the distribution before the model is evaluated.

BKT Under ADC — Parameter-by-Parameter

BKT's Hidden Markov Model has 4 parameters per Knowledge Component. Each one has a different structural floor, a different topology dependency, and a different ADC verdict profile.

📐BKT HMM Recap

BKT models each KC as a 2-state HMM: state 0 (unmastered) and state 1 (mastered). The learner transitions 0→1 with probability P(T) after each practice opportunity. The observation (correct/incorrect) is noisy: P(G) chance of correct from state 0, P(S) chance of incorrect from state 1. P(L₀) is the prior on being in state 1 before the first attempt.

STRUCTURAL FLOOR

When dataset is cold-start (few prior interactions), the estimator reverts to a population mean — often ~0.20–0.35. This creates a flat P(L₀) distribution across concepts: mean=0.25, std/mean<0.08 → NEAR_CONSTANT.

WHY THIS VERDICT

INFORMATIVE when rich dataset with varied prerequisites. NEAR_CONSTANT in cold-start — ADC detects when the prior is carrying no real per-concept signal and is just the population average.

TOPOLOGY DEPENDENCY

In diamond/convergent topology, P(L₀) for the sink concept should be higher if either incoming path has high mastery. Standard BKT ignores this — ADC flags when P(L₀) is topology-blind on convergent nodes.

ADC ALERT CONDITIONS

CONDITION

P(L₀) std/mean < 0.08 across all KCs

ALERT SIGNAL

NEAR_CONSTANT → population prior not informative

WHY IT MATTERS

Model is not using concept-level base rates. Cold-start fallback dominates. Yudelson 2013 population prior would fix this.

CONDITION

P(L₀) topology class = convergent AND P(L₀) not higher than single-prereq nodes

ALERT SIGNAL

TOPOLOGY_BLIND → transfer ignored in prior

WHY IT MATTERS

Diamond topology implies higher expected mastery at the sink. Flat prior signals BKT is not receiving transfer benefit.

CONDITION

P(L₀) > 0.85 for >30% of KCs

ALERT SIGNAL

FLOOR_CEILING → dataset may contain mastery-biased sampling

WHY IT MATTERS

High prior across many concepts means learners were sampled post-mastery. ADC flags selection bias in dataset.

ADC wrapper for BKT — how to instrument itPython

class ADCForBKT:
    """Wraps any BKT implementation. Classifies 4 parameters per KC."""

    α_floor_prior = 0.01   # P(L₀), P(T): positive probability space
    α_floor_noise = 0.001  # P(G), P(S): bounded by HMM identifiability
    signal_ratio   = 0.08  # std/mean threshold

    def classify(self, bkt_params: dict[str, dict]) -> dict:
        # bkt_params: {kc_id: {L0, T, G, S}} for all KCs
        verdicts = {}
        for dim in ['L0', 'T', 'G', 'S']:
            vals = [bkt_params[kc][dim] for kc in bkt_params]
            μ, σ = np.mean(vals), np.std(vals)
            floor = self.α_floor_prior if dim in ['L0','T'] else self.α_floor_noise

            if μ <= floor:
                verdicts[dim] = 'FLOOR_NOISE'
            elif σ / max(μ, 1e-9) < self.signal_ratio:
                verdicts[dim] = 'NEAR_CONSTANT'
            else:
                verdicts[dim] = 'INFORMATIVE'

        # Extra gate: BKT degeneracy check
        G_vals = [bkt_params[kc]['G'] for kc in bkt_params]
        S_vals = [bkt_params[kc]['S'] for kc in bkt_params]
        if np.corrcoef(G_vals, S_vals)[0,1] > 0.7:
            verdicts['_alert'] = 'REDUNDANT: P(G) corr(P(S))>0.7 — consider 1-param noise model'

        return verdicts

Topology-Conditional Alert Matrix

The most powerful ADC extension: different knowledge graph topologies predict different expected parameter distributions. ADC can flag topology blindness — when a model's parameters don't reflect the graph structure it should be using.

Standard BKT treats all KCs identically — it does not know whether a KC is a prerequisite chain root, a diamond convergence point, or an isolated concept. ADC can detect this by comparing the expected parameter pattern for each topology class against the observed parameter values. The gap is a governance alert: "your model is topology-blind here."

Topology-conditional ADC check — pseudocodePython

def topology_conditional_check(
    bkt_params: dict,      # {kc_id: {L0, T, G, S}}
    kg: KnowledgeGraph,    # adjacency list
) -> list[Alert]:
    alerts = []
    for kc_id, params in bkt_params.items():
        topo_class = kg.classify_node(kc_id)
        # Chain: expect P(T) to increase with chain depth
        if topo_class == 'chain':
            depth = kg.chain_depth(kc_id)
            expected_T = baseline_T + depth * 0.02  # empirical gradient
            if abs(params['T'] - expected_T) > 0.05:
                alerts.append(Alert(
                    kc=kc_id, param='T',
                    type='TOPOLOGY_BLIND',
                    msg=f'P(T)={params["T"]:.3f} does not reflect chain depth={depth}',
                ))

        # Diamond: expect P(L0) elevated at convergence node
        elif topo_class == 'diamond_sink':
            prereq_mastery = np.mean([
                bkt_params[p]['L0'] for p in kg.predecessors(kc_id)
            ])
            if params['L0'] < prereq_mastery * 0.9:
                alerts.append(Alert(
                    kc=kc_id, param='L0',
                    type='TRANSFER_MISSING',
                    msg=f'P(L0)={params["L0"]:.3f} not elevated despite convergent prereqs',
                ))

        # Island: P(L0) should match global prior
        elif topo_class == 'island':
            global_prior = np.mean([p['L0'] for p in bkt_params.values()])
            if abs(params['L0'] - global_prior) > 0.15:
                alerts.append(Alert(
                    kc=kc_id, param='L0',
                    type='ISOLATION_ANOMALY',
                    msg=f'Island node P(L0)={params["L0"]:.3f} deviates from global prior={global_prior:.3f}',
                ))
    return alerts

Critical Edge Cases — When ADC Saves the Model

These are situations where a model's predictions look plausible in aggregate but a signal is catastrophically wrong at the dimension level. ADC catches these before they reach the paper.

EC-1BKT Parameter Inversion — P(G) > P(S)

SCENARIO

A dataset of open-ended fill-in-the-blank questions has P(G)≈0.35 (guessable from partial knowledge) and P(S)≈0.12 (low slip because experts rarely make fill-in errors). BKT EM converges to P(G)=0.40, P(S)=0.08 for many KCs.

THE SILENT PROBLEM

When P(G) > P(S), the BKT likelihood surface inverts: correct answers provide WEAKER evidence of mastery than incorrect answers. The model silently becomes backwards. Aggregate AUC may still be 0.72 because the dataset is mostly mastered learners.

ADC ALERT + ACTION

ADC redundancy check: corr(P(G), P(S)) > 0.7 across KCs → REDUNDANT. Secondary check: mean(P(G)) > mean(P(S)) → INVERSION_WARNING. ADC does not fix this — it refuses to use P(G) as an informative signal and flags for researcher review.

EC-2Cold-Start P(L₀) Collapse

SCENARIO

A new MOOC platform has 500 learners, each attempting an average of 8 KCs. Most KCs have <50 observations. BKT EM cannot estimate P(L₀) reliably — it reverts to the global mean (0.22) for most KCs.

THE SILENT PROBLEM

P(L₀) carries no per-concept signal. Every concept is initialized the same. Early recommendations are uninformed. The model looks correct (correct/incorrect predictions are reasonable) but never differentiates cold-start from late-start learners.

ADC ALERT + ACTION

ADC verdict on P(L₀): NEAR_CONSTANT (std/mean < 0.05). Alert: COLD_START_COLLAPSE. Recommendation: use Yudelson 2013 population prior (Beta(α₀,β₀) from concept base rates) to inject topology-derived prior — exactly what HCIE implements for its Bayesian estimator.

EC-3Mesh Topology P(S) Flatness — Interference Invisible

SCENARIO

A chemistry knowledge graph has 80 concepts with dense bidirectional connections (mesh topology). Learners who know "oxidation" also know "reduction" and "electronegativity" — all concepts interfere with each other. P(S) across concepts is 0.09 ± 0.01.

THE SILENT PROBLEM

Slip in a mesh should be higher (more interference means more slip). Flat P(S)=0.09 means the model ignores concept-concept interference entirely. Recommendations are overconfident for late-stage learners: the model thinks mastery is stable when interference is actually causing random errors.

ADC ALERT + ACTION

ADC verdict on P(S): NEAR_CONSTANT. Topology check: MESH_INTERFERENCE alert — P(S) does not correlate with node degree (expected: high-degree nodes should have higher P(S)). Flag: model requires interference-aware error modeling (e.g., multi-factor slip term).

EC-4Fan-Out Foundation Not Learned

SCENARIO

A math KG has "fractions" as a fan-out root enabling 12 downstream concepts. P(T) for "fractions" is 0.28 — higher than the average 0.20 — which looks fine. But all 12 downstream concepts show P(L₀)=0.18, same as isolated concepts.

THE SILENT PROBLEM

The transfer benefit from mastering "fractions" is invisible to BKT. Learners who master fractions should have elevated P(L₀) for all 12 downstream KCs. The BKT estimates show no fan-out elevation, meaning recommendations for learners who already mastered the root are the same as for those who haven't.

ADC ALERT + ACTION

Topology check: fan-out root. Expected: downstream P(L₀) = f(P(L₀)_root). Observed: flat. Alert: FOUNDATION_EFFECT_MISSING. ADC recommendation: add transfer term to BKT prior — or switch to a topology-aware model (GKT/HCIE) for this KC cluster.

EC-5DKT Dead Neuron — Floor Noise in Hidden State

SCENARIO

A DKT model is trained on ASSISTments. After training, hidden unit h_42 has mean activation 0.003 across all timesteps. It was initialized near zero and was never activated because it competed with a more useful unit for the same variance.

THE SILENT PROBLEM

h_42 is dead. It contributes noise (near-zero) to the output layer. In a 100-unit LSTM, dead neurons are invisible in aggregate AUC but their contributions sum to a constant negative bias on some concept outputs. Hard to diagnose without per-unit analysis.

ADC ALERT + ACTION

ADC applied to DKT: treat each hidden unit as a signal dimension. mean(h_42) = 0.003 < α_floor = 0.01 → FLOOR_NOISE. ADC verdict: dead unit — prune from output layer or reinitialize. Analogous to HCIE T_prospective verdict (hardcoded 0.0 → FLOOR_NOISE).

ADC Applied Across Models — What Gets Flagged Where

Each KT model has a different signal space. ADC's classification framework adapts to each one by identifying the model-specific floor, the near-constant patterns, and the topology blindness.

BKT (HMM)

P(L₀)P(T)P(G)P(S)

⬇Floor Artifact

P(G) → item format (0.25 MCQ). P(S) → HMM constraint ≤0.4

≈Near-Constant Pattern

P(S) across KCs. P(L₀) in cold-start

◈Topology Interaction

P(T) chain depth effect. P(L₀) diamond convergence

∥Redundancy Risk

P(G) and P(S) can be collinear (corr>0.7)

ADC VALUE PROPOSITION FOR THIS MODEL

Detects parameter degeneracy before overfitting. Flags topology-blind parameter estimation.

Is ADC HCIE-Only? No — Here's the General Interface

ADC is a governance protocol, not an algorithm. Any KT model can be wrapped with ADC by implementing four hooks. The classification logic is identical for all models.

ADC Protocol — any KT model implements these 4 hooksPython

from abc import ABC, abstractmethod

class ADCCompatible(ABC):
    """
    Any KT model implementing these 4 hooks can be governed by ADC.
    The governance logic (floor/near-constant/topology checks) is
    model-agnostic — only the hooks differ.
    """

    @abstractmethod
    def signal_dimensions(self) -> list[str]:
        """Return names of all output signal dimensions.
        BKT: ['L0','T','G','S']
        DKT: ['h_0','h_1',...,'h_127']   (hidden units)
        SAKT: ['attn_0','attn_1',...]     (attention heads)
        HCIE: ['delta_m','t_realized','challenge','uncertainty','zpd','t_prospective']
        """

    @abstractmethod
    def theoretical_floor(self, dim: str) -> float:
        """Return the structural minimum for this dimension.
        BKT P(G): 1/n_choices (item format floor)
        BKT P(T): 0.01 (HMM identifiability)
        DKT h_i:  0.007 (sigmoid(-5))
        HCIE:     sigmoid(-2.5)=0.076 for sigmoid-output dims
        """

    @abstractmethod
    def emit_signals(self, trajectories: list) -> dict[str, list[float]]:
        """Run the model and return per-dimension signal distributions.
        Output: {dim: [val_1, val_2, ..., val_N]} for N learner-concept pairs
        """

    @abstractmethod
    def topology_expected(self, dim: str, topo_class: str) -> tuple[float,float]:
        """Return (expected_mean, expected_std) for this dim × topology class.
        Used by topology-conditional alert check.
        Return (None, None) if this model has no topology structure.
        """

# ─── ADC governance engine (unchanged for all models) ───────────────

class ADC:
    signal_ratio = 0.08   # std/mean threshold

    def govern(self, model: ADCCompatible, trajectories: list, kg=None) -> Report:
        signals = model.emit_signals(trajectories)
        verdicts = {}
        for dim in model.signal_dimensions():
            vals = signals[dim]
            μ, σ = np.mean(vals), np.std(vals)
            floor = model.theoretical_floor(dim)
            if μ <= floor:
                verdicts[dim] = Verdict.FLOOR_NOISE
            elif σ / max(μ, 1e-9) < self.signal_ratio:
                verdicts[dim] = Verdict.NEAR_CONSTANT
            else:
                verdicts[dim] = Verdict.INFORMATIVE
        return Report(verdicts=verdicts)

BKT implementation time

~2 hours

signal_dimensions() returns 4 strings. theoretical_floor() returns the HMM identifiability floor per parameter. emit_signals() runs the EM algorithm and returns per-KC parameter values.

DKT implementation time

~4 hours

signal_dimensions() returns hidden unit names. theoretical_floor() returns sigmoid(logit_min). emit_signals() runs a forward pass and collects per-unit activations. topology_expected() returns None (DKT is topology-blind — that itself is a governance finding).

HCIE implementation

Already done

This is the reference implementation. adaptive_dimension_controller.py is the concrete ADCCompatible for the JT engine. 6 dimensions, α_floor=0.01, signal_ratio=0.08. Shuffled-DAG provides the topology-conditional evidence baseline.

The Unifying Insight — Why ADC Exists

Every KT model is a compression of reality into a small number of scalars. ADC audits whether those scalars are compressing learner state — or compressing noise.

"A measurement that does not vary is not measuring anything."

This is the core of ADC. BKT's P(S)=0.09 across all KCs looks like a measurement — it has a value, it appears in the model, it contributes to predictions. But if it doesn't vary across concepts, it is a constant, not a measurement. Constants don't need to be learned. They can be replaced with their mean without affecting predictions.

ADC's classification is formalized statistical common sense: mean > floor means the signal is real; std/mean > threshold means the signal is discriminating. Both conditions together mean: this dimension is earning its keep in the model.

Applied to BKT: P(T) and P(L₀) usually earn their keep. P(S) usually does not (NEAR_CONSTANT). P(G) usually does not (FLOOR_NOISE from item format). Applied to HCIE: ΔM and T_realized earn their keep. T_prospective does not (FLOOR_NOISE: hardcoded 0.0). ZPD earns its mean but not its variance (NEAR_CONSTANT: saturates at 0.97). The insight is the same in both models. ADC is the instrument that surfaces it.

🔭The research contribution reframed

HCIE's primary contribution is not that it achieves AUC 0.609 — BKT achieves 0.612. HCIE's contribution is the ADC governance instrument itself: a generalizable protocol for auditing which dimensions of any KT model are informative versus artifacts. The numeric results are the evidence that the instrument works. Any future KT model can be plugged into the ADC interface and receive the same governance audit.

ADC Governance · Generalized Framework · BKT × 4 params · 5 topology classes · 5 edge cases · cross-model comparison

BKT/HMMDKT/LSTMSAKTGKT

NextTheory & Lit →Architecture →

ADC as a Universal KT Governance Layer

class ADCForBKT: """Wraps any BKT implementation. Classifies 4 parameters per KC.""" α_floor_prior = 0.01 # P(L₀), P(T): positive probability space α_floor_noise = 0.001 # P(G), P(S): bounded by HMM identifiability signal_ratio = 0.08 # std/mean threshold def classify(self, bkt_params: dict[str, dict]) -> dict: # bkt_params: {kc_id: {L0, T, G, S}} for all KCs verdicts = {} for dim in ['L0', 'T', 'G', 'S']: vals = [bkt_params[kc][dim] for kc in bkt_params] μ, σ = np.mean(vals), np.std(vals) floor = self.α_floor_prior if dim in ['L0','T'] else self.α_floor_noise if μ <= floor: verdicts[dim] = 'FLOOR_NOISE' elif σ / max(μ, 1e-9) < self.signal_ratio: verdicts[dim] = 'NEAR_CONSTANT' else: verdicts[dim] = 'INFORMATIVE' # Extra gate: BKT degeneracy check G_vals = [bkt_params[kc]['G'] for kc in bkt_params] S_vals = [bkt_params[kc]['S'] for kc in bkt_params] if np.corrcoef(G_vals, S_vals)[0,1] > 0.7: verdicts['_alert'] = 'REDUNDANT: P(G) corr(P(S))>0.7 — consider 1-param noise model' return verdicts

def topology_conditional_check( bkt_params: dict, # {kc_id: {L0, T, G, S}} kg: KnowledgeGraph, # adjacency list ) -> list[Alert]: alerts = [] for kc_id, params in bkt_params.items(): topo_class = kg.classify_node(kc_id) # Chain: expect P(T) to increase with chain depth if topo_class == 'chain': depth = kg.chain_depth(kc_id) expected_T = baseline_T + depth * 0.02 # empirical gradient if abs(params['T'] - expected_T) > 0.05: alerts.append(Alert( kc=kc_id, param='T', type='TOPOLOGY_BLIND', msg=f'P(T)={params["T"]:.3f} does not reflect chain depth={depth}', )) # Diamond: expect P(L0) elevated at convergence node elif topo_class == 'diamond_sink': prereq_mastery = np.mean([ bkt_params[p]['L0'] for p in kg.predecessors(kc_id) ]) if params['L0'] < prereq_mastery * 0.9: alerts.append(Alert( kc=kc_id, param='L0', type='TRANSFER_MISSING', msg=f'P(L0)={params["L0"]:.3f} not elevated despite convergent prereqs', )) # Island: P(L0) should match global prior elif topo_class == 'island': global_prior = np.mean([p['L0'] for p in bkt_params.values()]) if abs(params['L0'] - global_prior) > 0.15: alerts.append(Alert( kc=kc_id, param='L0', type='ISOLATION_ANOMALY', msg=f'Island node P(L0)={params["L0"]:.3f} deviates from global prior={global_prior:.3f}', )) return alerts

from abc import ABC, abstractmethod class ADCCompatible(ABC): """ Any KT model implementing these 4 hooks can be governed by ADC. The governance logic (floor/near-constant/topology checks) is model-agnostic — only the hooks differ. """ @abstractmethod def signal_dimensions(self) -> list[str]: """Return names of all output signal dimensions. BKT: ['L0','T','G','S'] DKT: ['h_0','h_1',...,'h_127'] (hidden units) SAKT: ['attn_0','attn_1',...] (attention heads) HCIE: ['delta_m','t_realized','challenge','uncertainty','zpd','t_prospective'] """ @abstractmethod def theoretical_floor(self, dim: str) -> float: """Return the structural minimum for this dimension. BKT P(G): 1/n_choices (item format floor) BKT P(T): 0.01 (HMM identifiability) DKT h_i: 0.007 (sigmoid(-5)) HCIE: sigmoid(-2.5)=0.076 for sigmoid-output dims """ @abstractmethod def emit_signals(self, trajectories: list) -> dict[str, list[float]]: """Run the model and return per-dimension signal distributions. Output: {dim: [val_1, val_2, ..., val_N]} for N learner-concept pairs """ @abstractmethod def topology_expected(self, dim: str, topo_class: str) -> tuple[float,float]: """Return (expected_mean, expected_std) for this dim × topology class. Used by topology-conditional alert check. Return (None, None) if this model has no topology structure. """ # ─── ADC governance engine (unchanged for all models) ─────────────── class ADC: signal_ratio = 0.08 # std/mean threshold def govern(self, model: ADCCompatible, trajectories: list, kg=None) -> Report: signals = model.emit_signals(trajectories) verdicts = {} for dim in model.signal_dimensions(): vals = signals[dim] μ, σ = np.mean(vals), np.std(vals) floor = model.theoretical_floor(dim) if μ <= floor: verdicts[dim] = Verdict.FLOOR_NOISE elif σ / max(μ, 1e-9) < self.signal_ratio: verdicts[dim] = Verdict.NEAR_CONSTANT else: verdicts[dim] = Verdict.INFORMATIVE return Report(verdicts=verdicts)