--- phase: 02-assembly-integration plan: 02 type: execute wave: 2 depends_on: ["02-01"] files_modified: - app/Services/QuestionBankService.php - tests/Feature/AssemblyDifficultySourceTest.php autonomous: true requirements: - ASM-04 must_haves: truths: - "Every row inserted into paper_questions includes a difficulty_source value ('calibrated' or 'original')" - "Post-hoc queries on paper_questions can distinguish which difficulty values came from calibration vs original" - "Existing paper_questions rows without difficulty_source default to 'original' (migration default)" artifacts: - path: "app/Services/QuestionBankService.php" provides: "Persists difficulty_source when saving exams" contains: "difficulty_source" - path: "tests/Feature/AssemblyDifficultySourceTest.php" provides: "Integration test verifying difficulty_source is persisted" contains: "difficulty_source" key_links: - from: "app/Services/QuestionDifficultyResolver.php" to: "app/Services/QuestionBankService.php" via: "question['difficulty_source'] set by resolver, read by saveExamToDatabase" pattern: "difficulty_source.*calibrated|difficulty_source.*original" --- Persist the difficulty_source field in paper_questions when exams are saved, completing the audit trail from resolver to database. Verify the full assembly pipeline end-to-end with a feature test. Purpose: Without persistence, the difficulty_source field set in memory by QuestionDifficultyResolver is silently dropped when PaperQuestion::insert() runs. Post-hoc analysis of exam quality would be impossible -- no way to know which questions used calibrated difficulty vs original. Output: QuestionBankService modified to persist difficulty_source, feature test verifying the full chain. @$HOME/.claude/get-shit-done/workflows/execute-plan.md @$HOME/.claude/get-shit-done/templates/summary.md @.planning/PROJECT.md @.planning/ROADMAP.md @.planning/STATE.md @.planning/phases/02-assembly-integration/02-RESEARCH.md @.planning/phases/02-assembly-integration/02-01-SUMMARY.md From app/Services/QuestionDifficultyResolver.php (what sets difficulty_source): ```php // Line 69: Sets difficulty_source = 'calibrated' on questions with calibration data $q['difficulty_source'] = 'calibrated'; // Questions NOT in the calibrations table have NO difficulty_source key set ``` From app/Services/QuestionBankService.php lines 686-699 (current insert, NEEDS MODIFICATION): ```php $questionInsertData[] = [ 'paper_id' => $paperId, 'question_id' => $question['question_code'] ?? $question['question_id'] ?? null, 'question_bank_id' => $question['id'] ?? $question['question_id'] ?? 0, 'knowledge_point' => $knowledgePoint, 'question_type' => $questionType, 'question_text' => is_array($question['stem'] ?? null) ? json_encode($question['stem'], JSON_UNESCAPED_UNICODE) : ($question['stem'] ?? $question['content'] ?? $question['question_text'] ?? ''), 'correct_answer' => is_array($correctAnswer) ? json_encode($correctAnswer, JSON_UNESCAPED_UNICODE) : $correctAnswer, 'solution' => is_array($question['solution'] ?? null) ? json_encode($question['solution'], JSON_UNESCAPED_UNICODE) : ($question['solution'] ?? ''), 'difficulty' => $difficultyValue, 'score' => $question['score'] ?? 5, 'estimated_time' => $question['estimated_time'] ?? 300, 'question_number' => $question['question_number'] ?? ($index + 1), // MISSING: 'difficulty_source' => $question['difficulty_source'] ?? 'original', ]; ``` From database/migrations/2026_04_17_100000_add_difficulty_source_to_paper_questions_table.php (created in Plan 01): ```php $table->string('difficulty_source', 20)->nullable()->default('original')->after('difficulty'); ``` Task 1: Persist difficulty_source in QuestionBankService::saveExamToDatabase() app/Services/QuestionBankService.php app/Services/QuestionBankService.php app/Services/QuestionDifficultyResolver.php - `grep -c "difficulty_source" app/Services/QuestionBankService.php` returns >= 1 - The `$questionInsertData` array includes `'difficulty_source'` key with value `$question['difficulty_source'] ?? 'original'` - The `difficulty_source` entry is placed immediately after the `difficulty` entry in the array Open `app/Services/QuestionBankService.php` and find the `$questionInsertData[]` assignment block (around lines 686-699). Add a single line to the array, immediately after the `'difficulty' => $difficultyValue,` line (line 695): ```php 'difficulty_source' => $question['difficulty_source'] ?? 'original', ``` The resulting block should look like: ```php $questionInsertData[] = [ 'paper_id' => $paperId, 'question_id' => $question['question_code'] ?? $question['question_id'] ?? null, 'question_bank_id' => $question['id'] ?? $question['question_id'] ?? 0, 'knowledge_point' => $knowledgePoint, 'question_type' => $questionType, 'question_text' => is_array($question['stem'] ?? null) ? json_encode($question['stem'], JSON_UNESCAPED_UNICODE) : ($question['stem'] ?? $question['content'] ?? $question['question_text'] ?? ''), 'correct_answer' => is_array($correctAnswer) ? json_encode($correctAnswer, JSON_UNESCAPED_UNICODE) : $correctAnswer, 'solution' => is_array($question['solution'] ?? null) ? json_encode($question['solution'], JSON_UNESCAPED_UNICODE) : ($question['solution'] ?? ''), 'difficulty' => $difficultyValue, 'difficulty_source' => $question['difficulty_source'] ?? 'original', 'score' => $question['score'] ?? 5, 'estimated_time' => $question['estimated_time'] ?? 300, 'question_number' => $question['question_number'] ?? ($index + 1), ]; ``` This is the ONLY change needed. The `difficulty_source` value flows from: - `QuestionDifficultyResolver::applyCalibratedDifficulty()` sets `$question['difficulty_source'] = 'calibrated'` for calibrated questions - Questions without calibration have no `difficulty_source` key, so `?? 'original'` provides the fallback - The migration (Plan 01) already added the column with default 'original' as a safety net cd /Volumes/T9/code/math_cms && grep -A2 "'difficulty'" app/Services/QuestionBankService.php | grep -c "difficulty_source" - QuestionBankService includes difficulty_source in the paper_questions insert data - Calibrated questions record difficulty_source='calibrated' - Questions without calibration record difficulty_source='original' - No other changes to saveExamToDatabase() logic Task 2: Create feature test verifying difficulty_source persistence tests/Feature/AssemblyDifficultySourceTest.php app/Services/QuestionBankService.php app/Services/QuestionDifficultyResolver.php app/Jobs/AssembleExamTaskJob.php - File exists at `tests/Feature/AssemblyDifficultySourceTest.php` - `grep -c 'difficulty_source' tests/Feature/AssemblyDifficultySourceTest.php` returns >= 3 - Test class extends `Tests\TestCase` (or equivalent base test class) - Test verifies that: (a) calibrated questions get difficulty_source='calibrated', (b) non-calibrated questions get difficulty_source='original' - `php vendor/bin/phpunit tests/Feature/AssemblyDifficultySourceTest.php` exits 0 Create `tests/Feature/AssemblyDifficultySourceTest.php` with tests that verify the full difficulty_source chain. The test should: 1. Use the `RefreshDatabase` trait (or `DatabaseMigrations` if that is the project pattern -- check existing test files first). 2. Test the QuestionDifficultyResolver directly: - Create test data: insert a row into `question_difficulty_calibrations` for a known question_bank_id with a calibrated_difficulty value - Call `applyCalibratedDifficulty()` with an array containing that question (with key `id` set to the question_bank_id) - Assert the result has `difficulty_source` set to `'calibrated'` - Call with a question NOT in the calibrations table - Assert the result has no `difficulty_source` key (or it is not 'calibrated') 3. Test the QuestionBankService persistence: - Create a minimal paper record - Call `saveExamToDatabase()` with questions that have `difficulty_source` keys set to various values ('calibrated', missing) - Query `paper_questions` and assert that rows have the correct difficulty_source values 4. Test the AssembleExamTaskJob hydration: - Call `hydrateQuestions()` on the job with a mix of questions - Verify that the resolver is called (questions with calibration data get difficulty_source='calibrated') If database setup for full integration test is complex, focus on the resolver + persistence chain which is the critical path. Use DB facade to create test data directly rather than factories (following the pattern of no model factories in the existing codebase). Check for existing test patterns first: ```bash ls tests/Feature/ head -30 tests/Feature/*.php 2>/dev/null | head -60 ``` Follow whatever base class and trait pattern the existing feature tests use. cd /Volumes/T9/code/math_cms && php vendor/bin/phpunit tests/Feature/AssemblyDifficultySourceTest.php --testdox 2>&1 | tail -20 - Feature test file exists and runs without errors - Test verifies calibrated questions get difficulty_source='calibrated' in paper_questions - Test verifies non-calibrated questions get difficulty_source='original' in paper_questions - Test verifies QuestionDifficultyResolver sets the correct in-memory values - All test assertions pass green ## Trust Boundaries | Boundary | Description | |----------|-------------| | Resolver output -> QuestionBankService | In-memory question arrays pass difficulty_source | | QuestionBankService -> paper_questions DB | Persistence of difficulty_source field | ## STRIDE Threat Register | Threat ID | Category | Component | Disposition | Mitigation Plan | |-----------|----------|-----------|-------------|-----------------| | T-02-05 | T | difficulty_source persistence | accept | Value is set server-side by resolver, not user input. Only two possible values: 'calibrated' or 'original' | | T-02-06 | I | Test data | accept | Feature test uses controlled test data, no production impact | | T-02-07 | D | Column addition | mitigate | Column is nullable with default 'original', no risk to existing data. Migration guarded with hasTable() check | 1. `grep "difficulty_source" app/Services/QuestionBankService.php` shows the field in the insert array 2. `php vendor/bin/phpunit tests/Feature/AssemblyDifficultySourceTest.php` passes all tests 3. `php artisan migrate:status` shows the difficulty_source migration as run 4. `php artisan tinker --execute="echo DB::select('DESCRIBE paper_questions difficulty_source')[0]->Default;"` returns 'original' 1. Every exam saved through QuestionBankService::saveExamToDatabase() records difficulty_source per question (ASM-04) 2. Calibrated questions have difficulty_source='calibrated' in paper_questions 3. Non-calibrated questions have difficulty_source='original' in paper_questions 4. Feature test verifies the full chain from resolver to database 5. No breaking changes to existing saveExamToDatabase() behavior After completion, create `.planning/phases/02-assembly-integration/02-02-SUMMARY.md`