init: AI日报 pipeline 完整代码 + 技能文档 + 运行记录

This commit is contained in:
2026-06-04 10:38:44 +08:00
commit 94e18ce22d
10 changed files with 1728 additions and 0 deletions

View File

@@ -0,0 +1,29 @@
# AI Daily Pipeline — LLM Config Auto-Follow (2026-05-30)
## Problem
The daily report script had hardcoded `XIAOMI_API_KEY` / `XIAOMI_BASE_URL` env vars. When the user switches Hermes' main model provider, the script would still use the old provider unless manually updated.
## Solution: `resolve_llm_config(env)`
Added to `ai_daily_blog_pipeline.py` (replaces hardcoded reads in `llm_call()`):
```python
def resolve_llm_config(env: dict):
"""Read Hermes config to get the active provider's API key, base_url, and model."""
# 1. Read ~/.hermes/config.yaml → model.provider, model.base_url, model.default
# 2. Read ~/.hermes/auth.json → credential_pool[provider].source (e.g. "env:XIAOMI_API_KEY")
# 3. Resolve env var name → actual key from .env
# 4. Fallback to LLM_API_KEY / XIAOMI_API_KEY if auth.json lookup fails
return api_key, base_url, model_name
```
## Config Sources (priority order)
1. `~/.hermes/config.yaml``model.provider`, `model.base_url`, `model.default`
2. `~/.hermes/auth.json``credential_pool[provider][0].source` (format: `env:VAR_NAME`)
3. `~/.hermes/.env` → actual key value
4. Legacy fallback: `LLM_API_KEY` / `XIAOMI_API_KEY` / `LLM_BASE_URL` / `LLM_MODEL`
## Usage
When user runs `hermes config set model.provider=minimax`, the daily report script automatically uses MiniMax's API key and endpoint on the next run. No script changes needed.
## Pitfall
The script needs `import yaml` — ensure `PyYAML` is installed. It's available in the Hermes venv but may not be in system Python.

View File

@@ -0,0 +1,55 @@
# MiMo-v2.5-pro API Performance Profile
Empirically tested on `https://token-plan-sgp.xiaomimimo.com/v1` (2026-05-29).
## Latency by Prompt Size
| Prompt Size | Items | Response Time | Status |
|-------------|-------|---------------|--------|
| ~500 chars | 1-2 | 2-4s | ✅ Reliable |
| ~4,500 chars | 15 | ~73s | ✅ OK |
| ~7,400 chars | 25 | >120s | ❌ Timeout |
| ~10,900 chars | 35 | >120s | ❌ Timeout |
| ~19,000 chars | 65-70 | >150s | ❌ Timeout |
## Key Constraints
- **Max reliable prompt size: ~5K chars / ~18 items** for structured output tasks
- Output token generation is slow (~50-80 tokens/s for large JSON outputs)
- Simple prompts (<1K) are fast and reliable (2-4s)
- Latency is **highly variable** — same prompt can take 73s or timeout at 150s
- Temperature 0.2 used for structured output consistency
## Implications for Cron Jobs
- **Pre-filter aggressively** before sending to LLM: dedupe + source priority + cap at 18 items
- **Cron timeout 300s** budget: ~35s data fetch + ~80s LLM = ~115s typical, but retries can push to 250s+
- Set LLM urllib timeout to **150s** (not 300s — it won't help, just wastes cron budget)
- **Retry 2x max** (not 3x) to stay within 300s cron budget
- If LLM consistently times out, check if API is rate-limited (test with simple prompt first)
## Workaround: Pre-filter Pattern
```python
def _prefilter_items(raw_items, max_items=18):
"""Dedupe + prioritize before LLM call."""
seen = set()
filtered = []
priority_sources = {'AI HOT': 1, '橘鸦AI早报': 1, 'InfoQ AI': 2, '量子位': 2}
sorted_items = sorted(raw_items, key=lambda r: priority_sources.get(r.get('source_group', ''), 3))
for item in sorted_items:
norm = re.sub(r'[^\w\u4e00-\u9fff]+', '', item['title_raw'].lower())
if not norm or len(norm) < 3 or norm in seen:
continue
seen.add(norm)
filtered.append(item)
if len(filtered) >= max_items:
break
return filtered
```
## Alternative Providers (tested same day)
- **Findmini (gpt-5.4)**: `https://api.findmini.top/gpt/v1` — returned 503
- **OpenRouter (free models)**: returned 429 rate limit
- **MiMo small prompts**: consistently 2-4s, reliable for simple tasks

View File

@@ -0,0 +1,65 @@
# Rendering & Guide Formatting Reference
## `clean_guide_text(text)` function (in `blog_markdown()`)
Strips unwanted artifacts from LLM-generated guide text:
```python
def clean_guide_text(text):
# Strip all [N] reference numbers
text = re.sub(r'\[\d+\]', '', text)
text = re.sub(r'\[N\]', '', text).strip()
# Strip "主线判断:" prefix
text = re.sub(r'^主线判断[:]\s*', '', text)
# Clean extra whitespace
text = re.sub(r'\s+', ' ', text).strip()
return text
```
## Summary section rendering
Type labels map: `{'strong': '强信号', 'medium': '中信号', 'risk': '待验证'}`
Output format per type group:
```
## 总结
**强信号**
- **标题从text第一句提取**
解释内容...
- **标题**
解释内容...
**中信号**
- **标题**
解释内容...
**待验证**
- **标题**
解释内容...
```
Title extraction logic:
1. Try splitting on `` or `:` — if prefix < 60 chars, use as title
2. Otherwise, split on `。!?` and use first sentence as title
## Title translation (Stage 2a)
Titles are translated from English to Chinese in Stage 2a. Rules:
- Brand names preserved: GPT-5, Codex, Gemini, OpenAI, Meta, etc.
- Technical terms with no good Chinese equivalent: keep English
- Everything else: translate to natural Chinese
- LLM prompt explicitly states: "英文品牌名/模型名保留原样,其余翻译为中文"
## LLM prompt for guide (as of 2026-05-30)
Key instructions to LLM:
- 不要空泛总结(如"行业焦点转向XX"),要指向具体事件
- 不要引用编号如[1][3],读者看不到对应关系
- 不要建议("开发者应该..."之类删掉)
- 每条控制在2-3句话以内
- 用大白话,不要学术腔

View File

@@ -0,0 +1,34 @@
# Timeout Configuration Reference
## Timeout Locations
| Setting | Location | Current Value | Notes |
|---------|----------|---------------|-------|
| Script total timeout | `~/.hermes/config.yaml``cron.script_timeout_seconds` | 600s | Max time for entire script execution |
| LLM urllib timeout | `ai_daily_blog_pipeline.py``llm_call()``urlopen(timeout=...)` | 600s | Single LLM API call timeout |
| RSS fetch timeout | `ai_daily_blog_pipeline.py``fetch_text()``urlopen(timeout=...)` | 25s | Per-RSS-feed fetch |
| 橘鸦 RSS timeout | `ai_daily_blog_pipeline.py``fetch_juya_rss()``urlopen(timeout=...)` | 45s | GitHub Pages can be slow; 262KB RSS |
| 橘鸦 fallback page timeout | `ai_daily_blog_pipeline.py``parse_juya()``urlopen(timeout=...)` | 45s | Only used if content:encoded unavailable |
| Service API timeout | `ai_daily_blog_pipeline.py``blog_api_request()``urlopen(timeout=...)` | 25s | Blog publish API call |
| 橘鸦 wait timeout | `ai_daily_blog_pipeline.py` → sleep(120) | 120s | Wait if 橘鸦 RSS is empty |
## Timeout Tuning Rules
1. **Always set generously** — user explicitly wants 1.5-2x theoretical time minimum
2. **MiMo API is slow** for long prompts — 18 items with 600s timeout works; 30+ items times out even at 600s
3. **Config file is protected** — use `sed -i` via terminal, not `patch` tool
4. **Gateway restart required** after config changes: `systemctl --user restart hermes-gateway`
## Theoretical Timing
- Script without LLM: ~10-15s (fetch + parse + publish)
- LLM call (18 items): ~60-120s typically, can spike to 300s+
- Total theoretical: ~80-150s
- Recommended timeout: 600s (generous, accounts for API variability)
## If Timeout Still Occurs
1. Check `run_meta.json``llm_error` field
2. If `TimeoutError: The read operation timed out` → LLM API is slow
3. Check if `max_items` was increased — more items = longer LLM time
4. Consider reducing `max_items` in `_prefilter_items()` back to 18