瀏覽代碼

docs(02): create phase plan — assembly integration with resolver wiring and difficulty_source audit

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
yemeishu 3 周之前
父節點
當前提交
7c1d2ae848

+ 81 - 0
.planning/ROADMAP.md

@@ -0,0 +1,81 @@
+# Roadmap: Math CMS — Difficulty Calibration & Intelligent Exam
+
+## Overview
+
+This roadmap transforms a difficulty calibration system that currently runs in isolation into one that drives intelligent exam assembly. The journey follows a strict gate-based progression: first validate that the calibration algorithm actually predicts outcomes (and fix the dual-scale data bug), then wire validated calibration into the production assembly pipeline, then build mastery-based adaptive matching on top, and finally add longitudinal health monitoring to catch drift before it harms students.
+
+## Phases
+
+**Phase Numbering:**
+- Integer phases (1, 2, 3, 4): Planned milestone work
+- Decimal phases (2.1, 2.2): Urgent insertions (marked with INSERTED)
+
+Decimal phases appear between their surrounding integers in numeric order.
+
+- [ ] **Phase 1: Validation & Data Audit** - Verify calibration accuracy with temporal backtesting, fix dual-scale bug, audit coverage
+- [ ] **Phase 2: Assembly Integration** - Wire validated calibrated difficulty into all exam assembly paths with fallback
+- [ ] **Phase 3: Adaptive Matching** - Map student mastery to optimal difficulty category for zone-of-proximal-development targeting
+- [ ] **Phase 4: Health Monitoring** - Detect calibration drift over time with actionable alerts
+
+## Phase Details
+
+### Phase 1: Validation & Data Audit
+**Goal**: The calibration algorithm is verified against held-out historical data, the difficulty scale is unified to 0-1, and coverage gaps are visible -- forming a PASS/FAIL gate that must open before any production wiring
+**Depends on**: Nothing (first phase)
+**Requirements**: VAL-01, VAL-02, VAL-03, VAL-04, VAL-05
+**Success Criteria** (what must be TRUE):
+  1. Running a walk-forward backtest on historical answer data produces a PASS or FAIL verdict (not an indeterminate result)
+  2. The backtest report shows Brier Skill Score, Pearson correlation, and calibration-vs-actual error rate metrics in a readable format
+  3. If the backtest FAILs (metrics below threshold), no calibrated values can enter the production assembly pipeline
+  4. All questions use a single unified 0-1 difficulty scale with no 0-5 scale values mixed in
+  5. A coverage report shows what fraction of questions per knowledge point have sufficient calibration samples
+**Plans**: 2 plans
+
+Plans:
+- [ ] 01-01-PLAN.md — Walk-forward backtest service with PASS/FAIL gate and Brier/Pearson metrics
+- [ ] 01-02-PLAN.md — Difficulty scale audit and calibration coverage report per knowledge point
+
+### Phase 2: Assembly Integration
+**Goal**: All exam assembly paths use calibrated difficulty values when available, fall back gracefully to original values when not, and the difficulty distribution strategy is active by default
+**Depends on**: Phase 1 (PASS gate must open)
+**Requirements**: ASM-01, ASM-02, ASM-03, ASM-04
+**Success Criteria** (what must be TRUE):
+  1. IntelligentExamController and LearningAnalyticsService both use calibrated difficulty as the primary value when assembling exams
+  2. Questions without sufficient calibration data automatically use the original questions.difficulty value without errors or gaps
+  3. Difficulty distribution strategy is active by default in all exam types (diagnostic, practice, error-review)
+  4. Each assembled exam records whether each question's difficulty came from calibration or original value, enabling post-hoc audit
+**Plans**: 2 plans
+
+Plans:
+- [ ] 02-01-PLAN.md — Wire resolver into AssembleExamTaskJob and LearningAnalyticsService, activate distribution by default
+- [ ] 02-02-PLAN.md — Persist difficulty_source in paper_questions and verify with feature tests
+
+### Phase 3: Adaptive Matching
+**Goal**: Exams automatically target each student's optimal learning zone by mapping their per-knowledge-point mastery to a difficulty category, closing the "answer-calibrate-assemble-re-answer" loop
+**Depends on**: Phase 2 (calibrated assembly working in production)
+**Requirements**: ADP-01, ADP-02
+**Success Criteria** (what must be TRUE):
+  1. The system computes a target difficulty range per student per knowledge point based on mastery data, targeting 60-75% expected correctness
+  2. Exam assembly automatically shifts difficulty distribution weights so the resulting exam difficulty falls within the student's target zone
+**Plans**: TBD
+
+### Phase 4: Health Monitoring
+**Goal**: Calibration quality is tracked longitudinally and drift is detected before it degrades exam quality, with actionable alerts
+**Depends on**: Phase 3 (accumulated production data from wired pipeline)
+**Requirements**: HLT-01, HLT-02
+**Success Criteria** (what must be TRUE):
+  1. The system periodically compares current calibration values against historical baselines and detects when drift exceeds a defined threshold
+  2. Drift alerts include the direction (easier/harder), magnitude, affected knowledge points, and count of affected questions
+**Plans**: TBD
+
+## Progress
+
+**Execution Order:**
+Phases execute in numeric order: 1 → 2 → 3 → 4
+
+| Phase | Plans Complete | Status | Completed |
+|-------|----------------|--------|-----------|
+| 1. Validation & Data Audit | 0/2 | Not started | - |
+| 2. Assembly Integration | 0/2 | Not started | - |
+| 3. Adaptive Matching | 0/? | Not started | - |
+| 4. Health Monitoring | 0/? | Not started | - |

+ 315 - 0
.planning/phases/02-assembly-integration/02-01-PLAN.md

@@ -0,0 +1,315 @@
+---
+phase: 02-assembly-integration
+plan: 01
+type: execute
+wave: 1
+depends_on: []
+files_modified:
+  - database/migrations/2026_04_17_100000_add_difficulty_source_to_paper_questions_table.php
+  - app/Jobs/AssembleExamTaskJob.php
+  - app/Services/LearningAnalyticsService.php
+autonomous: true
+requirements:
+  - ASM-01
+  - ASM-02
+  - ASM-03
+
+must_haves:
+  truths:
+    - "Every question that passes through AssembleExamTaskJob::hydrateQuestions() has calibrated difficulty applied when available in question_difficulty_calibrations"
+    - "Every question that passes through LearningAnalyticsService::generateIntelligentExam() has calibrated difficulty applied before distribution/sorting, regardless of which code path was taken"
+    - "Questions without a row in question_difficulty_calibrations retain their original questions.difficulty value unchanged"
+    - "Difficulty distribution strategy is active by default even when ExamTypeStrategy::buildParams() is not called or throws"
+  artifacts:
+    - path: "database/migrations/2026_04_17_100000_add_difficulty_source_to_paper_questions_table.php"
+      provides: "Adds difficulty_source column to paper_questions table"
+      contains: "difficulty_source"
+    - path: "app/Jobs/AssembleExamTaskJob.php"
+      provides: "Calls QuestionDifficultyResolver::applyCalibratedDifficulty() in hydrateQuestions()"
+      contains: "applyCalibratedDifficulty"
+    - path: "app/Services/LearningAnalyticsService.php"
+      provides: "Applies calibrated difficulty after gathering questions and defaults distribution to true"
+      contains: "applyCalibratedDifficulty"
+  key_links:
+    - from: "app/Jobs/AssembleExamTaskJob.php"
+      to: "app/Services/QuestionDifficultyResolver.php"
+      via: "resolve from container and call applyCalibratedDifficulty()"
+      pattern: "QuestionDifficultyResolver.*applyCalibratedDifficulty"
+    - from: "app/Services/LearningAnalyticsService.php"
+      to: "app/Services/QuestionDifficultyResolver.php"
+      via: "resolve from container and call applyCalibratedDifficulty()"
+      pattern: "QuestionDifficultyResolver.*applyCalibratedDifficulty"
+---
+
+<objective>
+Wire the QuestionDifficultyResolver into the two remaining assembly paths (AssembleExamTaskJob and LearningAnalyticsService) so that all exam assembly uses calibrated difficulty when available, with automatic fallback to original difficulty. Also activate difficulty distribution by default and prepare the paper_questions table for audit tracing.
+
+Purpose: Without this wiring, exams assembled via the async job path (AssembleExamTaskJob) and certain LearningAnalyticsService paths use original (potentially inaccurate) difficulty values, while other paths use calibrated values. This inconsistency means some students get poorly-calibrated exams.
+Output: Migration adding difficulty_source column, resolver calls in AssembleExamTaskJob and LearningAnalyticsService, distribution default set to true.
+</objective>
+
+<execution_context>
+@$HOME/.claude/get-shit-done/workflows/execute-plan.md
+@$HOME/.claude/get-shit-done/templates/summary.md
+</execution_context>
+
+<context>
+@.planning/PROJECT.md
+@.planning/ROADMAP.md
+@.planning/STATE.md
+@.planning/phases/02-assembly-integration/02-RESEARCH.md
+
+<interfaces>
+<!-- Key interfaces the executor needs. Extracted from codebase. -->
+
+From app/Services/QuestionDifficultyResolver.php:
+```php
+class QuestionDifficultyResolver
+{
+    // Returns questions with difficulty overridden and difficulty_source='calibrated' set
+    // for questions found in question_difficulty_calibrations.
+    // Questions NOT in the calibrations table are returned unchanged (original difficulty preserved).
+    public function applyCalibratedDifficulty(array $questions): array
+
+    // Returns map of question_bank_id => calibrated_difficulty
+    public function mapCalibratedDifficulty(array $questionIds): array
+}
+```
+
+From app/Jobs/AssembleExamTaskJob.php line 376-398:
+```php
+private function hydrateQuestions(array $questions, array $kpCodes): array
+{
+    $normalized = [];
+    foreach ($questions as $question) {
+        // ... normalization ...
+        $normalized[] = [
+            'id' => ...,
+            'difficulty' => isset($question['difficulty']) ? (float) $question['difficulty'] : 0.5,
+            // ...
+        ];
+    }
+    return array_values(array_filter($normalized, fn ($q) => ! empty($q['id'])));
+    // NOTE: No call to questionDifficultyResolver here -- THIS IS THE GAP
+}
+```
+
+From app/Http/Controllers/Api/IntelligentExamController.php line 733 (WORKING REFERENCE):
+```php
+// This is how the controller does it -- replicate this pattern in the job
+return $this->questionDifficultyResolver->applyCalibratedDifficulty($normalized);
+```
+
+From app/Services/LearningAnalyticsService.php line 1554:
+```php
+$enableDistribution = $params['enable_difficulty_distribution'] ?? false;
+// CHANGE TO: $params['enable_difficulty_distribution'] ?? true;
+```
+
+From database/migrations/2025_11_24_000000_create_base_tables_for_testing.php:
+```php
+// paper_questions base schema -- has columns: id, question_text, knowledge_point, timestamps
+// Later migrations add: question_id, question_bank_id, question_type, difficulty, score, etc.
+```
+</interfaces>
+</context>
+
+<tasks>
+
+<task type="auto">
+  <name>Task 1: Add difficulty_source migration and wire resolver into AssembleExamTaskJob</name>
+  <files>database/migrations/2026_04_17_100000_add_difficulty_source_to_paper_questions_table.php, app/Jobs/AssembleExamTaskJob.php</files>
+  <read_first>
+    app/Jobs/AssembleExamTaskJob.php
+    app/Services/QuestionDifficultyResolver.php
+    app/Http/Controllers/Api/IntelligentExamController.php
+    database/migrations/2025_11_23_090143_add_question_type_to_paper_questions_table.php
+  </read_first>
+  <acceptance_criteria>
+    - Migration file exists at `database/migrations/2026_04_17_100000_add_difficulty_source_to_paper_questions_table.php`
+    - `grep -c 'difficulty_source' database/migrations/2026_04_17_100000_add_difficulty_source_to_paper_questions_table.php` returns >= 2
+    - `grep -c 'applyCalibratedDifficulty' app/Jobs/AssembleExamTaskJob.php` returns >= 1
+    - `grep -c 'QuestionDifficultyResolver' app/Jobs/AssembleExamTaskJob.php` returns >= 1
+    - `php artisan migrate --path=database/migrations/2026_04_17_100000_add_difficulty_source_to_paper_questions_table.php` exits 0
+    - AssembleExamTaskJob::hydrateQuestions() return statement passes through applyCalibratedDifficulty()
+  </acceptance_criteria>
+  <action>
+**Step A: Create migration for difficulty_source column.**
+
+Create `database/migrations/2026_04_17_100000_add_difficulty_source_to_paper_questions_table.php` following the pattern from `2025_11_23_090143_add_question_type_to_paper_questions_table.php`:
+
+```php
+<?php
+
+use Illuminate\Database\Migrations\Migration;
+use Illuminate\Database\Schema\Blueprint;
+use Illuminate\Support\Facades\Schema;
+
+return new class extends Migration
+{
+    public function up(): void
+    {
+        if (Schema::hasTable('paper_questions')) {
+            Schema::table('paper_questions', function (Blueprint $table) {
+                $table->string('difficulty_source', 20)->nullable()->default('original')->after('difficulty');
+            });
+        }
+    }
+
+    public function down(): void
+    {
+        if (Schema::hasTable('paper_questions')) {
+            Schema::table('paper_questions', function (Blueprint $table) {
+                $table->dropColumn('difficulty_source');
+            });
+        }
+    }
+};
+```
+
+The column: string(20), nullable, default 'original', placed after the `difficulty` column. Use the `if (Schema::hasTable(...))` guard pattern from the existing migrations. Run the migration with `php artisan migrate`.
+
+**Step B: Wire QuestionDifficultyResolver into AssembleExamTaskJob::hydrateQuestions().**
+
+Open `app/Jobs/AssembleExamTaskJob.php`. Make TWO changes:
+
+1. Add import at the top of the file (after existing imports, around line 6):
+   ```php
+   use App\Services\QuestionDifficultyResolver;
+   ```
+
+2. In the private method `hydrateQuestions()` (line 376-398), modify the return statement. Currently the method returns:
+   ```php
+   return array_values(array_filter($normalized, fn ($q) => ! empty($q['id'])));
+   ```
+   Change it to resolve the resolver from the container and apply calibrated difficulty:
+   ```php
+   $result = array_values(array_filter($normalized, fn ($q) => ! empty($q['id'])));
+
+   return app(QuestionDifficultyResolver::class)->applyCalibratedDifficulty($result);
+   ```
+
+   Use `app(QuestionDifficultyResolver::class)` to resolve from the container rather than constructor injection -- this follows the research recommendation (Pattern B) and avoids changing the job's constructor signature which could affect queue serialization.
+
+IMPORTANT: Do NOT inject via constructor. The job is dispatched to a queue and constructor parameters must be serializable. Use `app()` container resolution inside the method instead. This is the same pattern used elsewhere in the codebase for optional service resolution.
+  </action>
+  <verify>
+    <automated>cd /Volumes/T9/code/math_cms && php artisan migrate --path=database/migrations/2026_04_17_100000_add_difficulty_source_to_paper_questions_table.php 2>&1 && echo "MIGRATION OK" && grep -c 'applyCalibratedDifficulty' app/Jobs/AssembleExamTaskJob.php && grep -c 'QuestionDifficultyResolver' app/Jobs/AssembleExamTaskJob.php</automated>
+  </verify>
+  <done>
+    - Migration adds difficulty_source column (string 20, nullable, default 'original') to paper_questions table
+    - Migration runs successfully without errors
+    - AssembleExamTaskJob::hydrateQuestions() calls applyCalibratedDifficulty() via app() container resolution
+    - QuestionDifficultyResolver import added to AssembleExamTaskJob.php
+    - Questions from mistake-based assembly paths (assemble_type 5, 15) now get calibrated difficulty
+  </done>
+</task>
+
+<task type="auto">
+  <name>Task 2: Wire resolver into LearningAnalyticsService and activate distribution by default</name>
+  <files>app/Services/LearningAnalyticsService.php</files>
+  <read_first>
+    app/Services/LearningAnalyticsService.php
+    app/Services/QuestionDifficultyResolver.php
+  </read_first>
+  <acceptance_criteria>
+    - `grep -c 'applyCalibratedDifficulty' app/Services/LearningAnalyticsService.php` returns >= 1
+    - `grep -c 'QuestionDifficultyResolver' app/Services/LearningAnalyticsService.php` returns >= 1
+    - Line 1554 now reads `$params['enable_difficulty_distribution'] ?? true` (not `?? false`)
+    - Resolver is called on the question pool AFTER gathering from DB and BEFORE selectQuestionsByMastery/distribution
+  </acceptance_criteria>
+  <action>
+Open `app/Services/LearningAnalyticsService.php`. Make THREE changes:
+
+**Change 1: Add import.**
+Add at the top (after existing use statements):
+```php
+use App\Services\QuestionDifficultyResolver;
+```
+
+**Change 2: Change difficulty distribution default to true.**
+Find line 1554 (inside `generateIntelligentExam()` method):
+```php
+$enableDistribution = $params['enable_difficulty_distribution'] ?? false;
+```
+Change to:
+```php
+$enableDistribution = $params['enable_difficulty_distribution'] ?? true;
+```
+
+**Change 3: Apply calibrated difficulty to the full question pool.**
+Find the location in `generateIntelligentExam()` where all questions have been gathered from the bank but BEFORE they are passed to `selectQuestionsByMastery()`. This is around the point where `$allQuestions` or similar variable holds the full candidate pool from `getQuestionsFromBank()` and any supplement questions from `fetchQuestionsForKpAssembleSupplement()`.
+
+Look for the section where questions are merged/gathered and about to enter mastery-based filtering. Add the resolver call at that point:
+
+```php
+// Apply calibrated difficulty before mastery filtering and distribution
+$allQuestions = app(QuestionDifficultyResolver::class)->applyCalibratedDifficulty($allQuestions);
+```
+
+The exact variable name depends on the code structure -- it could be `$questions`, `$allQuestions`, `$candidateQuestions`, or similar. Read the code flow carefully:
+
+1. `getQuestionsFromBank()` returns raw questions from DB (line ~1734)
+2. These questions may be supplemented with `fetchQuestionsForKpAssembleSupplement()` (line ~1650)
+3. Then `selectQuestionsByMastery()` (line ~2097) filters/reorders them
+4. Then difficulty distribution is applied
+
+Insert the resolver call AFTER steps 1-2 (all questions gathered) and BEFORE step 3 (mastery filtering). This ensures calibrated difficulty is used for both mastery-based selection and difficulty distribution.
+
+Use `app(QuestionDifficultyResolver::class)` for container resolution -- same pattern as the job. Do NOT add constructor injection because the service already has an optional-parameter constructor pattern that would need careful handling.
+
+CRITICAL: Apply the resolver ONLY ONCE per question batch. Do not apply it multiple times -- the resolver queries the DB each time. Identify the single merge point where all candidate questions are collected and apply there.
+  </action>
+  <verify>
+    <automated>cd /Volumes/T9/code/math_cms && grep -n 'applyCalibratedDifficulty' app/Services/LearningAnalyticsService.php && grep -n "enable_difficulty_distribution.*\?\?" app/Services/LearningAnalyticsService.php</automated>
+  </verify>
+  <done>
+    - LearningAnalyticsService imports QuestionDifficultyResolver
+    - Calibrated difficulty is applied to the full question pool before mastery filtering and distribution
+    - Questions from getQuestionsFromBank() and fetchQuestionsForKpAssembleSupplement() both get calibrated difficulty
+    - enable_difficulty_distribution defaults to true (not false)
+    - Distribution is active even when ExamTypeStrategy::buildParams() does not run or fails
+    - Resolver is called exactly once per assembly, not multiple times
+  </done>
+</task>
+
+</tasks>
+
+<threat_model>
+## Trust Boundaries
+
+| Boundary | Description |
+|----------|-------------|
+| Assembly pipeline -> question_difficulty_calibrations | Read-only lookup of calibrated difficulty values |
+| Assembly pipeline -> paper_questions | Write with new difficulty_source field |
+
+## STRIDE Threat Register
+
+| Threat ID | Category | Component | Disposition | Mitigation Plan |
+|-----------|----------|-----------|-------------|-----------------|
+| T-02-01 | T | AssembleExamTaskJob | accept | Resolver reads from calibration table; no user input reaches the DB query. Question IDs come from prior validated queries |
+| T-02-02 | I | difficulty_source field | accept | Field is set server-side only, never from user input. Values are hardcoded 'calibrated' or 'original' |
+| T-02-03 | D | Migration | mitigate | Migration uses hasTable() guard to prevent failure on missing table. Column is nullable with default to avoid breaking existing rows |
+| T-02-04 | E | app() resolution | accept | QuestionDifficultyResolver is stateless with no external dependencies. Container resolution is safe |
+</threat_model>
+
+<verification>
+1. `php artisan migrate` runs without errors, adding difficulty_source column
+2. `grep -c 'applyCalibratedDifficulty' app/Jobs/AssembleExamTaskJob.php` returns >= 1
+3. `grep -c 'applyCalibratedDifficulty' app/Services/LearningAnalyticsService.php` returns >= 1
+4. `grep "enable_difficulty_distribution.*\?\?" app/Services/LearningAnalyticsService.php` shows `?? true`
+5. No double-application: count of applyCalibratedDifficulty calls in each file matches expectation (1 per file)
+6. `grep -r "difficulty_source" database/migrations/2026_04_17_100000_add_difficulty_source_to_paper_questions_table.php` confirms column exists
+</verification>
+
+<success_criteria>
+1. AssembleExamTaskJob::hydrateQuestions() applies calibrated difficulty before returning questions (ASM-01, ASM-02)
+2. LearningAnalyticsService::generateIntelligentExam() applies calibrated difficulty to the full candidate pool before mastery filtering (ASM-01)
+3. Questions without calibration data retain original difficulty unchanged (ASM-02)
+4. Difficulty distribution defaults to enabled (ASM-03)
+5. difficulty_source column exists in paper_questions table (partial ASM-04 -- column ready for persistence in Plan 02)
+</success_criteria>
+
+<output>
+After completion, create `.planning/phases/02-assembly-integration/02-01-SUMMARY.md`
+</output>

+ 238 - 0
.planning/phases/02-assembly-integration/02-02-PLAN.md

@@ -0,0 +1,238 @@
+---
+phase: 02-assembly-integration
+plan: 02
+type: execute
+wave: 2
+depends_on: ["02-01"]
+files_modified:
+  - app/Services/QuestionBankService.php
+  - tests/Feature/AssemblyDifficultySourceTest.php
+autonomous: true
+requirements:
+  - ASM-04
+
+must_haves:
+  truths:
+    - "Every row inserted into paper_questions includes a difficulty_source value ('calibrated' or 'original')"
+    - "Post-hoc queries on paper_questions can distinguish which difficulty values came from calibration vs original"
+    - "Existing paper_questions rows without difficulty_source default to 'original' (migration default)"
+  artifacts:
+    - path: "app/Services/QuestionBankService.php"
+      provides: "Persists difficulty_source when saving exams"
+      contains: "difficulty_source"
+    - path: "tests/Feature/AssemblyDifficultySourceTest.php"
+      provides: "Integration test verifying difficulty_source is persisted"
+      contains: "difficulty_source"
+  key_links:
+    - from: "app/Services/QuestionDifficultyResolver.php"
+      to: "app/Services/QuestionBankService.php"
+      via: "question['difficulty_source'] set by resolver, read by saveExamToDatabase"
+      pattern: "difficulty_source.*calibrated|difficulty_source.*original"
+---
+
+<objective>
+Persist the difficulty_source field in paper_questions when exams are saved, completing the audit trail from resolver to database. Verify the full assembly pipeline end-to-end with a feature test.
+
+Purpose: Without persistence, the difficulty_source field set in memory by QuestionDifficultyResolver is silently dropped when PaperQuestion::insert() runs. Post-hoc analysis of exam quality would be impossible -- no way to know which questions used calibrated difficulty vs original.
+Output: QuestionBankService modified to persist difficulty_source, feature test verifying the full chain.
+</objective>
+
+<execution_context>
+@$HOME/.claude/get-shit-done/workflows/execute-plan.md
+@$HOME/.claude/get-shit-done/templates/summary.md
+</execution_context>
+
+<context>
+@.planning/PROJECT.md
+@.planning/ROADMAP.md
+@.planning/STATE.md
+@.planning/phases/02-assembly-integration/02-RESEARCH.md
+@.planning/phases/02-assembly-integration/02-01-SUMMARY.md
+
+<interfaces>
+<!-- Key interfaces from Plan 01 output and existing codebase -->
+
+From app/Services/QuestionDifficultyResolver.php (what sets difficulty_source):
+```php
+// Line 69: Sets difficulty_source = 'calibrated' on questions with calibration data
+$q['difficulty_source'] = 'calibrated';
+// Questions NOT in the calibrations table have NO difficulty_source key set
+```
+
+From app/Services/QuestionBankService.php lines 686-699 (current insert, NEEDS MODIFICATION):
+```php
+$questionInsertData[] = [
+    'paper_id' => $paperId,
+    'question_id' => $question['question_code'] ?? $question['question_id'] ?? null,
+    'question_bank_id' => $question['id'] ?? $question['question_id'] ?? 0,
+    'knowledge_point' => $knowledgePoint,
+    'question_type' => $questionType,
+    'question_text' => is_array($question['stem'] ?? null) ? json_encode($question['stem'], JSON_UNESCAPED_UNICODE) : ($question['stem'] ?? $question['content'] ?? $question['question_text'] ?? ''),
+    'correct_answer' => is_array($correctAnswer) ? json_encode($correctAnswer, JSON_UNESCAPED_UNICODE) : $correctAnswer,
+    'solution' => is_array($question['solution'] ?? null) ? json_encode($question['solution'], JSON_UNESCAPED_UNICODE) : ($question['solution'] ?? ''),
+    'difficulty' => $difficultyValue,
+    'score' => $question['score'] ?? 5,
+    'estimated_time' => $question['estimated_time'] ?? 300,
+    'question_number' => $question['question_number'] ?? ($index + 1),
+    // MISSING: 'difficulty_source' => $question['difficulty_source'] ?? 'original',
+];
+```
+
+From database/migrations/2026_04_17_100000_add_difficulty_source_to_paper_questions_table.php (created in Plan 01):
+```php
+$table->string('difficulty_source', 20)->nullable()->default('original')->after('difficulty');
+```
+</interfaces>
+</context>
+
+<tasks>
+
+<task type="auto">
+  <name>Task 1: Persist difficulty_source in QuestionBankService::saveExamToDatabase()</name>
+  <files>app/Services/QuestionBankService.php</files>
+  <read_first>
+    app/Services/QuestionBankService.php
+    app/Services/QuestionDifficultyResolver.php
+  </read_first>
+  <acceptance_criteria>
+    - `grep -c "difficulty_source" app/Services/QuestionBankService.php` returns >= 1
+    - The `$questionInsertData` array includes `'difficulty_source'` key with value `$question['difficulty_source'] ?? 'original'`
+    - The `difficulty_source` entry is placed immediately after the `difficulty` entry in the array
+  </acceptance_criteria>
+  <action>
+Open `app/Services/QuestionBankService.php` and find the `$questionInsertData[]` assignment block (around lines 686-699).
+
+Add a single line to the array, immediately after the `'difficulty' => $difficultyValue,` line (line 695):
+
+```php
+'difficulty_source' => $question['difficulty_source'] ?? 'original',
+```
+
+The resulting block should look like:
+```php
+$questionInsertData[] = [
+    'paper_id' => $paperId,
+    'question_id' => $question['question_code'] ?? $question['question_id'] ?? null,
+    'question_bank_id' => $question['id'] ?? $question['question_id'] ?? 0,
+    'knowledge_point' => $knowledgePoint,
+    'question_type' => $questionType,
+    'question_text' => is_array($question['stem'] ?? null) ? json_encode($question['stem'], JSON_UNESCAPED_UNICODE) : ($question['stem'] ?? $question['content'] ?? $question['question_text'] ?? ''),
+    'correct_answer' => is_array($correctAnswer) ? json_encode($correctAnswer, JSON_UNESCAPED_UNICODE) : $correctAnswer,
+    'solution' => is_array($question['solution'] ?? null) ? json_encode($question['solution'], JSON_UNESCAPED_UNICODE) : ($question['solution'] ?? ''),
+    'difficulty' => $difficultyValue,
+    'difficulty_source' => $question['difficulty_source'] ?? 'original',
+    'score' => $question['score'] ?? 5,
+    'estimated_time' => $question['estimated_time'] ?? 300,
+    'question_number' => $question['question_number'] ?? ($index + 1),
+];
+```
+
+This is the ONLY change needed. The `difficulty_source` value flows from:
+- `QuestionDifficultyResolver::applyCalibratedDifficulty()` sets `$question['difficulty_source'] = 'calibrated'` for calibrated questions
+- Questions without calibration have no `difficulty_source` key, so `?? 'original'` provides the fallback
+- The migration (Plan 01) already added the column with default 'original' as a safety net
+  </action>
+  <verify>
+    <automated>cd /Volumes/T9/code/math_cms && grep -A2 "'difficulty'" app/Services/QuestionBankService.php | grep -c "difficulty_source"</automated>
+  </verify>
+  <done>
+    - QuestionBankService includes difficulty_source in the paper_questions insert data
+    - Calibrated questions record difficulty_source='calibrated'
+    - Questions without calibration record difficulty_source='original'
+    - No other changes to saveExamToDatabase() logic
+  </done>
+</task>
+
+<task type="auto">
+  <name>Task 2: Create feature test verifying difficulty_source persistence</name>
+  <files>tests/Feature/AssemblyDifficultySourceTest.php</files>
+  <read_first>
+    app/Services/QuestionBankService.php
+    app/Services/QuestionDifficultyResolver.php
+    app/Jobs/AssembleExamTaskJob.php
+  </read_first>
+  <acceptance_criteria>
+    - File exists at `tests/Feature/AssemblyDifficultySourceTest.php`
+    - `grep -c 'difficulty_source' tests/Feature/AssemblyDifficultySourceTest.php` returns >= 3
+    - Test class extends `Tests\TestCase` (or equivalent base test class)
+    - Test verifies that: (a) calibrated questions get difficulty_source='calibrated', (b) non-calibrated questions get difficulty_source='original'
+    - `php vendor/bin/phpunit tests/Feature/AssemblyDifficultySourceTest.php` exits 0
+  </acceptance_criteria>
+  <action>
+Create `tests/Feature/AssemblyDifficultySourceTest.php` with tests that verify the full difficulty_source chain.
+
+The test should:
+1. Use the `RefreshDatabase` trait (or `DatabaseMigrations` if that is the project pattern -- check existing test files first).
+2. Test the QuestionDifficultyResolver directly:
+   - Create test data: insert a row into `question_difficulty_calibrations` for a known question_bank_id with a calibrated_difficulty value
+   - Call `applyCalibratedDifficulty()` with an array containing that question (with key `id` set to the question_bank_id)
+   - Assert the result has `difficulty_source` set to `'calibrated'`
+   - Call with a question NOT in the calibrations table
+   - Assert the result has no `difficulty_source` key (or it is not 'calibrated')
+3. Test the QuestionBankService persistence:
+   - Create a minimal paper record
+   - Call `saveExamToDatabase()` with questions that have `difficulty_source` keys set to various values ('calibrated', missing)
+   - Query `paper_questions` and assert that rows have the correct difficulty_source values
+4. Test the AssembleExamTaskJob hydration:
+   - Call `hydrateQuestions()` on the job with a mix of questions
+   - Verify that the resolver is called (questions with calibration data get difficulty_source='calibrated')
+
+If database setup for full integration test is complex, focus on the resolver + persistence chain which is the critical path. Use DB facade to create test data directly rather than factories (following the pattern of no model factories in the existing codebase).
+
+Check for existing test patterns first:
+```bash
+ls tests/Feature/
+head -30 tests/Feature/*.php 2>/dev/null | head -60
+```
+
+Follow whatever base class and trait pattern the existing feature tests use.
+  </action>
+  <verify>
+    <automated>cd /Volumes/T9/code/math_cms && php vendor/bin/phpunit tests/Feature/AssemblyDifficultySourceTest.php --testdox 2>&1 | tail -20</automated>
+  </verify>
+  <done>
+    - Feature test file exists and runs without errors
+    - Test verifies calibrated questions get difficulty_source='calibrated' in paper_questions
+    - Test verifies non-calibrated questions get difficulty_source='original' in paper_questions
+    - Test verifies QuestionDifficultyResolver sets the correct in-memory values
+    - All test assertions pass green
+  </done>
+</task>
+
+</tasks>
+
+<threat_model>
+## Trust Boundaries
+
+| Boundary | Description |
+|----------|-------------|
+| Resolver output -> QuestionBankService | In-memory question arrays pass difficulty_source |
+| QuestionBankService -> paper_questions DB | Persistence of difficulty_source field |
+
+## STRIDE Threat Register
+
+| Threat ID | Category | Component | Disposition | Mitigation Plan |
+|-----------|----------|-----------|-------------|-----------------|
+| T-02-05 | T | difficulty_source persistence | accept | Value is set server-side by resolver, not user input. Only two possible values: 'calibrated' or 'original' |
+| T-02-06 | I | Test data | accept | Feature test uses controlled test data, no production impact |
+| T-02-07 | D | Column addition | mitigate | Column is nullable with default 'original', no risk to existing data. Migration guarded with hasTable() check |
+</threat_model>
+
+<verification>
+1. `grep "difficulty_source" app/Services/QuestionBankService.php` shows the field in the insert array
+2. `php vendor/bin/phpunit tests/Feature/AssemblyDifficultySourceTest.php` passes all tests
+3. `php artisan migrate:status` shows the difficulty_source migration as run
+4. `php artisan tinker --execute="echo DB::select('DESCRIBE paper_questions difficulty_source')[0]->Default;"` returns 'original'
+</verification>
+
+<success_criteria>
+1. Every exam saved through QuestionBankService::saveExamToDatabase() records difficulty_source per question (ASM-04)
+2. Calibrated questions have difficulty_source='calibrated' in paper_questions
+3. Non-calibrated questions have difficulty_source='original' in paper_questions
+4. Feature test verifies the full chain from resolver to database
+5. No breaking changes to existing saveExamToDatabase() behavior
+</success_criteria>
+
+<output>
+After completion, create `.planning/phases/02-assembly-integration/02-02-SUMMARY.md`
+</output>