init: AI日报 pipeline 完整代码 + 技能文档 + 运行记录
This commit is contained in:
29
skill/references/llm-config-auto-follow.md
Normal file
29
skill/references/llm-config-auto-follow.md
Normal file
@@ -0,0 +1,29 @@
|
||||
# AI Daily Pipeline — LLM Config Auto-Follow (2026-05-30)
|
||||
|
||||
## Problem
|
||||
The daily report script had hardcoded `XIAOMI_API_KEY` / `XIAOMI_BASE_URL` env vars. When the user switches Hermes' main model provider, the script would still use the old provider unless manually updated.
|
||||
|
||||
## Solution: `resolve_llm_config(env)`
|
||||
Added to `ai_daily_blog_pipeline.py` (replaces hardcoded reads in `llm_call()`):
|
||||
|
||||
```python
|
||||
def resolve_llm_config(env: dict):
|
||||
"""Read Hermes config to get the active provider's API key, base_url, and model."""
|
||||
# 1. Read ~/.hermes/config.yaml → model.provider, model.base_url, model.default
|
||||
# 2. Read ~/.hermes/auth.json → credential_pool[provider].source (e.g. "env:XIAOMI_API_KEY")
|
||||
# 3. Resolve env var name → actual key from .env
|
||||
# 4. Fallback to LLM_API_KEY / XIAOMI_API_KEY if auth.json lookup fails
|
||||
return api_key, base_url, model_name
|
||||
```
|
||||
|
||||
## Config Sources (priority order)
|
||||
1. `~/.hermes/config.yaml` → `model.provider`, `model.base_url`, `model.default`
|
||||
2. `~/.hermes/auth.json` → `credential_pool[provider][0].source` (format: `env:VAR_NAME`)
|
||||
3. `~/.hermes/.env` → actual key value
|
||||
4. Legacy fallback: `LLM_API_KEY` / `XIAOMI_API_KEY` / `LLM_BASE_URL` / `LLM_MODEL`
|
||||
|
||||
## Usage
|
||||
When user runs `hermes config set model.provider=minimax`, the daily report script automatically uses MiniMax's API key and endpoint on the next run. No script changes needed.
|
||||
|
||||
## Pitfall
|
||||
The script needs `import yaml` — ensure `PyYAML` is installed. It's available in the Hermes venv but may not be in system Python.
|
||||
55
skill/references/mimo-api-performance.md
Normal file
55
skill/references/mimo-api-performance.md
Normal file
@@ -0,0 +1,55 @@
|
||||
# MiMo-v2.5-pro API Performance Profile
|
||||
|
||||
Empirically tested on `https://token-plan-sgp.xiaomimimo.com/v1` (2026-05-29).
|
||||
|
||||
## Latency by Prompt Size
|
||||
|
||||
| Prompt Size | Items | Response Time | Status |
|
||||
|-------------|-------|---------------|--------|
|
||||
| ~500 chars | 1-2 | 2-4s | ✅ Reliable |
|
||||
| ~4,500 chars | 15 | ~73s | ✅ OK |
|
||||
| ~7,400 chars | 25 | >120s | ❌ Timeout |
|
||||
| ~10,900 chars | 35 | >120s | ❌ Timeout |
|
||||
| ~19,000 chars | 65-70 | >150s | ❌ Timeout |
|
||||
|
||||
## Key Constraints
|
||||
|
||||
- **Max reliable prompt size: ~5K chars / ~18 items** for structured output tasks
|
||||
- Output token generation is slow (~50-80 tokens/s for large JSON outputs)
|
||||
- Simple prompts (<1K) are fast and reliable (2-4s)
|
||||
- Latency is **highly variable** — same prompt can take 73s or timeout at 150s
|
||||
- Temperature 0.2 used for structured output consistency
|
||||
|
||||
## Implications for Cron Jobs
|
||||
|
||||
- **Pre-filter aggressively** before sending to LLM: dedupe + source priority + cap at 18 items
|
||||
- **Cron timeout 300s** budget: ~35s data fetch + ~80s LLM = ~115s typical, but retries can push to 250s+
|
||||
- Set LLM urllib timeout to **150s** (not 300s — it won't help, just wastes cron budget)
|
||||
- **Retry 2x max** (not 3x) to stay within 300s cron budget
|
||||
- If LLM consistently times out, check if API is rate-limited (test with simple prompt first)
|
||||
|
||||
## Workaround: Pre-filter Pattern
|
||||
|
||||
```python
|
||||
def _prefilter_items(raw_items, max_items=18):
|
||||
"""Dedupe + prioritize before LLM call."""
|
||||
seen = set()
|
||||
filtered = []
|
||||
priority_sources = {'AI HOT': 1, '橘鸦AI早报': 1, 'InfoQ AI': 2, '量子位': 2}
|
||||
sorted_items = sorted(raw_items, key=lambda r: priority_sources.get(r.get('source_group', ''), 3))
|
||||
for item in sorted_items:
|
||||
norm = re.sub(r'[^\w\u4e00-\u9fff]+', '', item['title_raw'].lower())
|
||||
if not norm or len(norm) < 3 or norm in seen:
|
||||
continue
|
||||
seen.add(norm)
|
||||
filtered.append(item)
|
||||
if len(filtered) >= max_items:
|
||||
break
|
||||
return filtered
|
||||
```
|
||||
|
||||
## Alternative Providers (tested same day)
|
||||
|
||||
- **Findmini (gpt-5.4)**: `https://api.findmini.top/gpt/v1` — returned 503
|
||||
- **OpenRouter (free models)**: returned 429 rate limit
|
||||
- **MiMo small prompts**: consistently 2-4s, reliable for simple tasks
|
||||
65
skill/references/rendering-guide.md
Normal file
65
skill/references/rendering-guide.md
Normal file
@@ -0,0 +1,65 @@
|
||||
# Rendering & Guide Formatting Reference
|
||||
|
||||
## `clean_guide_text(text)` function (in `blog_markdown()`)
|
||||
|
||||
Strips unwanted artifacts from LLM-generated guide text:
|
||||
|
||||
```python
|
||||
def clean_guide_text(text):
|
||||
# Strip all [N] reference numbers
|
||||
text = re.sub(r'\[\d+\]', '', text)
|
||||
text = re.sub(r'\[N\]', '', text).strip()
|
||||
# Strip "主线判断:" prefix
|
||||
text = re.sub(r'^主线判断[::]\s*', '', text)
|
||||
# Clean extra whitespace
|
||||
text = re.sub(r'\s+', ' ', text).strip()
|
||||
return text
|
||||
```
|
||||
|
||||
## Summary section rendering
|
||||
|
||||
Type labels map: `{'strong': '强信号', 'medium': '中信号', 'risk': '待验证'}`
|
||||
|
||||
Output format per type group:
|
||||
```
|
||||
## 总结
|
||||
|
||||
**强信号**
|
||||
|
||||
- **标题(从text第一句提取)**
|
||||
解释内容...
|
||||
|
||||
- **标题**
|
||||
解释内容...
|
||||
|
||||
**中信号**
|
||||
|
||||
- **标题**
|
||||
解释内容...
|
||||
|
||||
**待验证**
|
||||
|
||||
- **标题**
|
||||
解释内容...
|
||||
```
|
||||
|
||||
Title extraction logic:
|
||||
1. Try splitting on `:` or `:` — if prefix < 60 chars, use as title
|
||||
2. Otherwise, split on `。!?` and use first sentence as title
|
||||
|
||||
## Title translation (Stage 2a)
|
||||
|
||||
Titles are translated from English to Chinese in Stage 2a. Rules:
|
||||
- Brand names preserved: GPT-5, Codex, Gemini, OpenAI, Meta, etc.
|
||||
- Technical terms with no good Chinese equivalent: keep English
|
||||
- Everything else: translate to natural Chinese
|
||||
- LLM prompt explicitly states: "英文品牌名/模型名保留原样,其余翻译为中文"
|
||||
|
||||
## LLM prompt for guide (as of 2026-05-30)
|
||||
|
||||
Key instructions to LLM:
|
||||
- 不要空泛总结(如"行业焦点转向XX"),要指向具体事件
|
||||
- 不要引用编号如[1][3],读者看不到对应关系
|
||||
- 不要建议("开发者应该..."之类删掉)
|
||||
- 每条控制在2-3句话以内
|
||||
- 用大白话,不要学术腔
|
||||
34
skill/references/timeout-config.md
Normal file
34
skill/references/timeout-config.md
Normal file
@@ -0,0 +1,34 @@
|
||||
# Timeout Configuration Reference
|
||||
|
||||
## Timeout Locations
|
||||
|
||||
| Setting | Location | Current Value | Notes |
|
||||
|---------|----------|---------------|-------|
|
||||
| Script total timeout | `~/.hermes/config.yaml` → `cron.script_timeout_seconds` | 600s | Max time for entire script execution |
|
||||
| LLM urllib timeout | `ai_daily_blog_pipeline.py` → `llm_call()` → `urlopen(timeout=...)` | 600s | Single LLM API call timeout |
|
||||
| RSS fetch timeout | `ai_daily_blog_pipeline.py` → `fetch_text()` → `urlopen(timeout=...)` | 25s | Per-RSS-feed fetch |
|
||||
| 橘鸦 RSS timeout | `ai_daily_blog_pipeline.py` → `fetch_juya_rss()` → `urlopen(timeout=...)` | 45s | GitHub Pages can be slow; 262KB RSS |
|
||||
| 橘鸦 fallback page timeout | `ai_daily_blog_pipeline.py` → `parse_juya()` → `urlopen(timeout=...)` | 45s | Only used if content:encoded unavailable |
|
||||
| Service API timeout | `ai_daily_blog_pipeline.py` → `blog_api_request()` → `urlopen(timeout=...)` | 25s | Blog publish API call |
|
||||
| 橘鸦 wait timeout | `ai_daily_blog_pipeline.py` → sleep(120) | 120s | Wait if 橘鸦 RSS is empty |
|
||||
|
||||
## Timeout Tuning Rules
|
||||
|
||||
1. **Always set generously** — user explicitly wants 1.5-2x theoretical time minimum
|
||||
2. **MiMo API is slow** for long prompts — 18 items with 600s timeout works; 30+ items times out even at 600s
|
||||
3. **Config file is protected** — use `sed -i` via terminal, not `patch` tool
|
||||
4. **Gateway restart required** after config changes: `systemctl --user restart hermes-gateway`
|
||||
|
||||
## Theoretical Timing
|
||||
|
||||
- Script without LLM: ~10-15s (fetch + parse + publish)
|
||||
- LLM call (18 items): ~60-120s typically, can spike to 300s+
|
||||
- Total theoretical: ~80-150s
|
||||
- Recommended timeout: 600s (generous, accounts for API variability)
|
||||
|
||||
## If Timeout Still Occurs
|
||||
|
||||
1. Check `run_meta.json` → `llm_error` field
|
||||
2. If `TimeoutError: The read operation timed out` → LLM API is slow
|
||||
3. Check if `max_items` was increased — more items = longer LLM time
|
||||
4. Consider reducing `max_items` in `_prefilter_items()` back to 18
|
||||
Reference in New Issue
Block a user