|
@@ -0,0 +1,116 @@
|
|
|
|
|
+# Math CMS — Difficulty Calibration & Intelligent Exam
|
|
|
|
|
+
|
|
|
|
|
+## What This Is
|
|
|
|
|
+
|
|
|
|
|
+K12 数学智能提分系统的难度分级优化项目。系统已有题库管理、知识点匹配、组卷、答题分析、掌握度计算等核心能力,现需解决题目难度值不准确和组卷难度匹配不佳两个核心问题,形成"答题 → 校准 → 精准组卷 → 再答题"的闭环。
|
|
|
|
|
+
|
|
|
|
|
+## Core Value
|
|
|
|
|
+
|
|
|
|
|
+**让题目难度趋于真实,让组卷精准匹配学生水平。** 校准算法准确性是一切的前提,未经验证不进入生产链路。
|
|
|
|
|
+
|
|
|
|
|
+## Requirements
|
|
|
|
|
+
|
|
|
|
|
+### Validated
|
|
|
|
|
+
|
|
|
|
|
+<!-- From existing codebase — already working and relied upon -->
|
|
|
|
|
+
|
|
|
|
|
+- ✓ 题库管理(CRUD、导入、AI生成题目)
|
|
|
|
|
+- ✓ 知识点体系与章节匹配
|
|
|
|
|
+- ✓ 智能组卷(按知识点、章节、错题等多种策略)
|
|
|
|
|
+- ✓ 答题分析与评分
|
|
|
|
|
+- ✓ 掌握度计算(MasteryCalculator,按知识点追踪学生水平)
|
|
|
|
|
+- ✓ 难度分级基础架构(questions.difficulty、DifficultyDistributionService、5级分类)
|
|
|
|
|
+- ✓ 校准算法框架(QuestionDifficultyCalibrationService,stratified_residual_eb_v2 算法)
|
|
|
|
|
+- ✓ 校准数据表(question_difficulty_calibrations)
|
|
|
|
|
+- ✓ 难度校准命令行报告工具(AnalyzeQuestionDifficultyCalibrationCommand)
|
|
|
|
|
+
|
|
|
|
|
+### Active
|
|
|
|
|
+
|
|
|
|
|
+<!-- Current scope — building toward these -->
|
|
|
|
|
+
|
|
|
|
|
+- [ ] 校准算法(stratified_residual_eb_v2)准确性验证 — 历史数据回测
|
|
|
|
|
+- [ ] 难度标准统一(消除 0-1 和 0-5 双标准混用问题)
|
|
|
|
|
+- [ ] 校准值接入组卷全链路(所有组卷路径优先使用校准值)
|
|
|
|
|
+- [ ] 难度分布策略默认启用(enable_difficulty_distribution 全路径生效)
|
|
|
|
|
+- [ ] 基于学生掌握度自动推荐 difficulty_category
|
|
|
|
|
+- [ ] 后续产品方向规划(学习路径、更精准的提分策略等)
|
|
|
|
|
+
|
|
|
|
|
+### Out of Scope
|
|
|
|
|
+
|
|
|
|
|
+- 回写 questions.difficulty — 原始难度作为初始值永久不变,校准值存在独立表
|
|
|
|
|
+- 重新设计校准算法 — 当前先用 stratified_residual_eb_v2,验证后决定是否调整
|
|
|
|
|
+- 改造掌握度计算权重体系 — MasteryCalculator 的 overshoot/match/undershoot 权重暂不动
|
|
|
|
|
+
|
|
|
|
|
+## Context
|
|
|
|
|
+
|
|
|
|
|
+### 现有系统架构
|
|
|
|
|
+
|
|
|
|
|
+- **技术栈**: Laravel (PHP),MySQL,异步任务队列
|
|
|
|
|
+- **难度存储**: `questions.difficulty`(原始值)+ `question_difficulty_calibrations`(校准值)
|
|
|
|
|
+- **组卷流程**: IntelligentExamController → AssembleExamTaskJob → LearningAnalyticsService → DifficultyDistributionService
|
|
|
|
|
+- **掌握度**: MasteryCalculator 按知识点追踪,0-1 连续值
|
|
|
|
|
+- **难度分类**: 5 级(零基础/筑基/进阶/培优/竞赛),对应 0-4
|
|
|
|
|
+
|
|
|
|
|
+### 已识别的 6 个关键问题
|
|
|
|
|
+
|
|
|
|
|
+**难度不准:**
|
|
|
|
|
+1. 校准值存在独立表,但智能组卷主流程未启用校准值,多条路径仍用原始值
|
|
|
|
|
+2. 原始难度有 0-1 和 0-5 两套标准混用,导致分布计算可能出错
|
|
|
|
|
+3. (设计如此)校准结果不回写 questions.difficulty,原始值保持静态
|
|
|
|
|
+
|
|
|
|
|
+**组卷匹配差:**
|
|
|
|
|
+4. `enable_difficulty_distribution` 默认关闭,API 主链路未使用难度分布策略
|
|
|
|
|
+5. 无根据学生掌握度自动推荐 difficulty_category 的逻辑
|
|
|
|
|
+6. LearningAnalyticsService 的组卷路径未调用校准难度
|
|
|
|
|
+
|
|
|
|
|
+### 校准算法(stratified_residual_eb_v2)
|
|
|
|
|
+
|
|
|
|
|
+- 分层基线残差 + 贝叶斯收缩 + 时间衰减
|
|
|
|
|
+- 按题目类型 + 试卷难度分层计算全局基线错误率
|
|
|
|
|
+- 残差 = 观测错误率 - 期望错误率
|
|
|
|
|
+- 自适应增益/收缩随样本量和信号强度调整
|
|
|
|
|
+- 45 天半衰期时间衰减
|
|
|
|
|
+- 校准值范围 [0.01, 0.99]
|
|
|
|
|
+- 健康监控:近期 Brier score/log-loss 恶化时缩小调整幅度
|
|
|
|
|
+
|
|
|
|
|
+### 关键原则
|
|
|
|
|
+
|
|
|
|
|
+- `questions.difficulty` = 初始值,永不改变
|
|
|
|
|
+- `question_difficulty_calibrations.calibrated_difficulty` = 实时校准值,持续更新
|
|
|
|
|
+- 验证通过前,校准值不进入生产组卷链路
|
|
|
|
|
+
|
|
|
|
|
+## Constraints
|
|
|
|
|
+
|
|
|
|
|
+- **Tech Stack**: Laravel PHP 后端,不改技术栈
|
|
|
|
|
+- **数据完整性**: 历史答题数据必须可用于回测验证
|
|
|
|
|
+- **向前兼容**: 校准值接入链路不能破坏现有组卷功能
|
|
|
|
|
+- **渐进式**: 验证 → 打通 → 智能匹配,分阶段推进
|
|
|
|
|
+
|
|
|
|
|
+## Key Decisions
|
|
|
|
|
+
|
|
|
|
|
+| Decision | Rationale | Outcome |
|
|
|
|
|
+|----------|-----------|---------|
|
|
|
|
|
+| questions.difficulty 保持不变 | 作为初始参考值,便于审计和问题追溯 | ✓ Good |
|
|
|
|
|
+| 校准值存在独立表 | 解耦原始值与校准值,独立演进 | ✓ Good |
|
|
|
|
|
+| 先验证再接入生产 | 校准算法未经验证就用于组卷会引入更大误差 | — Pending |
|
|
|
|
|
+| 历史数据回测验证 | 有大量历史答题数据可用于评估校准效果 | — Pending |
|
|
|
|
|
+
|
|
|
|
|
+## Evolution
|
|
|
|
|
+
|
|
|
|
|
+This document evolves at phase transitions and milestone boundaries.
|
|
|
|
|
+
|
|
|
|
|
+**After each phase transition** (via `/gsd-transition`):
|
|
|
|
|
+1. Requirements invalidated? → Move to Out of Scope with reason
|
|
|
|
|
+2. Requirements validated? → Move to Validated with phase reference
|
|
|
|
|
+3. New requirements emerged? → Add to Active
|
|
|
|
|
+4. Decisions to log? → Add to Key Decisions
|
|
|
|
|
+5. "What This Is" still accurate? → Update if drifted
|
|
|
|
|
+
|
|
|
|
|
+**After each milestone** (via `/gsd-complete-milestone`):
|
|
|
|
|
+1. Full review of all sections
|
|
|
|
|
+2. Core Value check — still the right priority?
|
|
|
|
|
+3. Audit Out of Scope — reasons still valid?
|
|
|
|
|
+4. Update Context with current state
|
|
|
|
|
+
|
|
|
|
|
+---
|
|
|
|
|
+*Last updated: 2026-04-16 after initialization*
|