feat: refactor API key configuration and enhance application initialization
- Renamed `check_environment` to `check_api_key_configured` for clarity, simplifying the API key validation logic. - Removed the blocking behavior of the API key check during application startup, allowing the app to run while providing a prompt for configuration. - Updated `LocalAgentApp` to accept an `api_configured` parameter, enabling conditional messaging for API key setup. - Enhanced the `SandboxRunner` to support backup management and improved execution result handling with detailed metrics. - Integrated data governance strategies into the `HistoryManager`, ensuring compliance and improved data management. - Added privacy settings and metrics tracking across various components to enhance user experience and application safety.
This commit is contained in:
117
docs/P1-04-optimization-summary.md
Normal file
117
docs/P1-04-optimization-summary.md
Normal file
@@ -0,0 +1,117 @@
|
||||
# P1-04 需求分析失败处理优化方案
|
||||
|
||||
## 问题描述
|
||||
|
||||
**问题标题**: 需求分析失败时直接进入代码生成,模糊需求可能被执行
|
||||
**问题类型**: 业务规则/数据一致性
|
||||
**所在位置**: app/agent.py:467, app/agent.py:471
|
||||
|
||||
**核心问题**: 完整性检查报错时走"直接生成代码"路径,而非强制澄清/终止,导致模糊规则被执行,输出偏差和返工增加。
|
||||
|
||||
## 解决方案
|
||||
|
||||
### 1. 异常分级系统 (app/exceptions.py)
|
||||
|
||||
创建了需求分析异常分级系统,将异常分为四个级别:
|
||||
|
||||
- **CriticalInfoMissingException** (严重级): 关键信息缺失,必须澄清才能继续
|
||||
- **AmbiguousRequirementException** (高级): 需求存在歧义,强制澄清
|
||||
- **LowConfidenceException** (中级): 置信度低,建议澄清但允许用户选择
|
||||
- **CheckerFailureException** (低级): 检查器本身失败,降级处理
|
||||
|
||||
### 2. 优化需求检查回调逻辑 (app/agent.py)
|
||||
|
||||
修改了 `_on_requirement_checked` 方法,根据异常类型采取不同策略:
|
||||
|
||||
```python
|
||||
def _on_requirement_checked(self, result: Optional[Dict], error: Optional[Exception]):
|
||||
# 分类异常
|
||||
exception = classify_requirement_error(result, error)
|
||||
|
||||
# 根据异常严重程度决定处理策略
|
||||
if isinstance(exception, CriticalInfoMissingException):
|
||||
# 强制澄清
|
||||
elif isinstance(exception, AmbiguousRequirementException):
|
||||
# 强制澄清
|
||||
elif isinstance(exception, LowConfidenceException):
|
||||
# 提供选择:澄清或继续
|
||||
elif isinstance(exception, CheckerFailureException):
|
||||
# 降级处理,记录警告
|
||||
else:
|
||||
# 需求完整,直接继续
|
||||
```
|
||||
|
||||
### 3. 度量指标记录 (app/metrics_logger.py)
|
||||
|
||||
创建了度量指标记录系统,跟踪以下指标:
|
||||
|
||||
- **澄清触发率**: clarification_triggered / total_tasks
|
||||
- **直接执行率**: direct_execution / total_tasks
|
||||
- **用户二次修改率**: user_modifications / total_tasks
|
||||
- **需求歧义导致失败率**: ambiguity_failures / total_tasks
|
||||
|
||||
指标数据保存在 `workspace/metrics/requirement_analysis.json`,支持导出报告。
|
||||
|
||||
### 4. 增强需求检查 Prompt (llm/prompts.py)
|
||||
|
||||
更新了 `REQUIREMENT_CHECK_SYSTEM` prompt,明确了:
|
||||
|
||||
- **关键信息分类**: critical_fields(必需)vs missing_info(可选)
|
||||
- **严重程度判断**: 4个级别的详细判断标准
|
||||
- **输出格式**: 增加 critical_fields 字段用于标识关键缺失信息
|
||||
|
||||
## 优化效果
|
||||
|
||||
### 处理流程对比
|
||||
|
||||
**优化前**:
|
||||
```
|
||||
需求检查失败 → 显示警告 → 直接生成代码 → 可能产生偏差
|
||||
```
|
||||
|
||||
**优化后**:
|
||||
```
|
||||
需求检查失败 → 异常分级 →
|
||||
- 关键信息缺失 → 强制澄清
|
||||
- 需求歧义 → 强制澄清
|
||||
- 低置信度 → 用户选择(澄清/继续)
|
||||
- 检查器失败 → 降级处理 + 警告
|
||||
```
|
||||
|
||||
### 预期改进
|
||||
|
||||
1. **减少模糊需求执行**: 关键信息缺失时强制澄清,避免错误理解
|
||||
2. **提高代码质量**: 需求明确后生成的代码更准确
|
||||
3. **降低返工率**: 减少因需求理解偏差导致的二次修改
|
||||
4. **可追踪优化**: 通过度量指标持续改进澄清策略
|
||||
|
||||
## 使用说明
|
||||
|
||||
### 度量指标查看
|
||||
|
||||
```python
|
||||
from app.metrics_logger import MetricsLogger
|
||||
from pathlib import Path
|
||||
|
||||
logger = MetricsLogger(Path("workspace"))
|
||||
|
||||
# 获取摘要
|
||||
summary = logger.get_summary()
|
||||
print(f"澄清触发率: {summary['clarification_rate']:.1%}")
|
||||
print(f"需求歧义失败率: {summary['failure_rate']:.1%}")
|
||||
|
||||
# 导出报告
|
||||
report = logger.export_report(Path("workspace/metrics/report.md"))
|
||||
```
|
||||
|
||||
### 自定义澄清阈值
|
||||
|
||||
可以通过修改 `classify_requirement_error` 函数中的判断逻辑来调整澄清触发的阈值。
|
||||
|
||||
## 建议的后续优化
|
||||
|
||||
1. **动态阈值调整**: 根据历史成功率自动调整置信度阈值
|
||||
2. **用户反馈收集**: 在执行后询问用户是否符合预期,用于改进判断
|
||||
3. **A/B测试**: 对比不同策略的效果,找到最优平衡点
|
||||
4. **智能默认值**: 基于历史数据学习常用参数的默认值
|
||||
|
||||
Reference in New Issue
Block a user