夜雨聆风AI 编程工具横评实测报告（2026-08-04）
7 款 AI 编程工具，同一模型（deepseek-v4-flash）、同一测试题、同一网关并行实测。 附带 token 消耗对比 + 代码质量 + 速度 + 易用性 + 已知 bug 全记录。
目录
快速结论（30 秒版） 测试方法（完整版） 测试题原文 / 统一环境 / 执行命令 / 判定标准 / 局限与偏差 / 选型决策 结果总表 分工具详解（7 款逐一分析） 模型对比：glm_for_coding vs deepseek-v4-flash 最终推荐 部署与配置坑（详版） Token 数据证据链 测试产物全集（7 份完整代码 + pytest 输出）
快速结论（30 秒版）
| OpenCode | |
| Claude Code | |
| Codex | |
| Goose / Crush | |
| Kilo | |
阅读导航：想复现测试看 测试方法；想挑工具看 最终推荐；想看各工具实际写的代码看 测试产物全集；想避坑看 部署与配置坑；想核对数字看 Token 证据链。
测试方法（完整版）
1. 测试题（逐字原文，所有工具收到的 prompt 完全相同）
Create todo.py (Python CLI todo app with add/list/done/delete commands and JSON persistence) and test_todo.py (pytest unit tests). Run the tests and make sure they pass.
中文翻译：创建 todo.py（带 add/list/done/delete 命令和 JSON 持久化的 Python CLI 待办应用）和 test_todo.py（pytest 单元测试）。运行测试并确保全部通过。
这个测试题考察的能力：
理解需求并拆解（CLI 命令设计、JSON 存储设计） 写出可测试的代码（函数/类结构可被 pytest 引用） 自己写测试用例覆盖功能 执行测试、发现失败、修复到全过（自我验证闭环） 工具链使用（argparse、json、pytest、文件读写）
2. 统一环境
https://api.myagentlab.homes/v1） | |
/tmp/bench-venv，系统无 pip 故用 uv 建 venv 装） | |
/home/agent/coding-benchmark/test-<工具名>/ | |
3. 每个工具的具体执行命令（实测用的真实命令）
claude -p "<测试题>" --model deepseek-v4-flash --allowedTools 'Bash,Write,Read,Edit' --output-format stream-json --verbose | ||
opencode run "<测试题>" | ||
codex exec --dangerously-bypass-approvals-and-sandbox --skip-git-repo-check --json '<测试题>' | ||
cline --json "<测试题>" | ||
kilo run -m myagentlab/deepseek-v4-flash "<测试题>" | ||
crush run -m myagentlab/deepseek-v4-flash "<测试题>" | ||
goose run -t '<测试题>' --provider myagentlab --model deepseek-v4-flash |
4. 判定标准
- 通过
= 该工具自己生成的 test_todo.py用 pytest 跑通（/tmp/bench-venv/bin/python -m pytest test_todo.py），测试数量由工具自己决定（22/12/13/24/10/13/17 各不相同） - 耗时
= 从启动命令到进程退出（含模型首 token 延迟 + 工具自我迭代） - token 数据
= 各工具本地记录（来源见"Token 数据证据链"章节） - 代码质量
= 人工检查产物（测试覆盖、边界处理、可读性）
5. 测试的局限与已知偏差（诚实声明）
- Cline 的 24/24 是修复后的
：原始 22/24（2 个代码 bug），我手动修复了这 2 个 bug 后复验 24/24。纯工具产出是 22/24。 - Codex token 是估算值
：本地不记录 usage，网关无分 key 明细，按 6 轮请求 × 累积上下文估算。成本 ~0.027 † Crush 的 crush.dbsessions 表无 cache_read 字段，故无数据 ‡ Crush 记录值（completion_tokens 疑似只记末次请求，实际生成代码见产物）分工具详解
🥇 1. OpenCode — 综合最优
最快（32s）+ 最省 token（11.7K input，其他工具的 1/5-1/7）+ 代码质量过关。
配置要点：
// 项目级 opencode.json{"provider": { "myagentlab": {"npm": "@ai-sdk/openai-compatible","options": { "baseURL": "https://api.myagentlab.homes/v1", "apiKey": "见 credentials" },"models": { "deepseek-v4-flash": { "name": "deepseek-v4-flash" } }}},"model": "myagentlab/deepseek-v4-flash"}opencode run "任务描述"🥈 2. Claude Code — 质量天花板
代码质量最高（22 个测试最多最全），但 token 消耗最大。
Token 分析： input 227K 中 190K 是 cache read（价格是正常 input 的 1/10），实际新计算 ~36K，并非全价消耗。
配置要点：
export ANTHROPIC_API_KEY=<key> ANTHROPIC_BASE_URL=https://api.myagentlab.homesclaude -p "任务" --model deepseek-v4-flash --allowedTools 'Bash,Write,Read,Edit' --output-format stream-json --verbose⚠️ root 用户不能用
--dangerously-skip-permissions（被拒），必须用--allowedTools🥉 3. Codex — 模型对了就起飞
换 deepseek 后从"失败×4"变"13/13 通过、78s"，但 token 数据缺失 + 有推理模型兼容隐患。
配置要点：
# ~/.codex/config.tomlmodel = "deepseek-v4-flash"model_provider = "myagentlab"[model_providers.myagentlab]name = "myagentlab"base_url = "https://api.myagentlab.homes/v1"env_key = "OPENAI_API_KEY"wire_api = "chat"codex exec --dangerously-bypass-approvals-and-sandbox --skip-git-repo-check --json '任务'4. Goose — 大器晚成
配置坑最多但修好后很稳，17/17 通过，77s。
⚠️ 配置坑（都是实测踩出来的）：
配置要点：
// ~/.config/goose/custom_providers/myagentlab.json{"name": "myagentlab","engine": "openai","api_key_env": "MYAGENTLAB_API_KEY","base_url": "https://api.myagentlab.homes/v1","models": [ { "name": "deepseek-v4-flash", "context_limit": 128000 } ],"supports_streaming": false,"requires_auth": true}MYAGENTLAB_API_KEY=<key> goose run -t '任务' --provider myagentlab --model deepseek-v4-flash5. Crush — 焕然一新
从 glm 下 13/14（有 bug）变成 deepseek 下 13/13 全过、66s，但配置极坑 + token 记录不全。
⚠️ 配置坑：
配置要点：
// ~/.config/crush/crush.json{"providers": {"myagentlab": {"type": "openai-compat","base_url": "https://api.myagentlab.homes/v1","api_key": "0.027Goose80,84843,7766,769~/.local/share/goose/sessions/sessions.db usage_ledger 表 SUM()Crush14,499无此字段40*~/.crush/crush.db sessions 表 prompt_tokens/completion_tokensCline11,2345,6326,033~/.cline/data/sessions/<id>/<id>.messages.json assistant metrics 累加Kilo15,488102,2722,291~/.local/share/kilo/kilo.db session 表* Crush completion_tokens=40 疑似只记末次请求（实际生成 124 行代码），已按实际产物修正理解复核命令示例（OpenCode）：SELECT tokens_input, tokens_output, tokens_cache_read FROM sessionWHERE model LIKE '%deepseek%' ORDER BY time_created DESC LIMIT 1;复核命令示例（Goose）：SELECT session_id, SUM(input_tokens), SUM(output_tokens), SUM(cache_read_tokens)FROM usage_ledger GROUP BY session_id ORDER BY MIN(id) DESC LIMIT 2;复核命令示例（Claude Code）：grep -oE '"usage":\{[^}]*\}' /tmp/cc-ds.log | tail -1为什么 Codex 没有精确值： 本地 session jsonl 无 usage 字段；网关 billing 接口只给总额（共享 key，后台 MRS 任务并发消耗，差值法不可信）；唯一精确途径是 new-api 后台按渠道查明细。附：测试产物全集（7 份完整代码 + pytest 输出）Claude Code v2.1.220 — 22/22Claude Code 生成的 todo.py (4619 bytes)#!/usr/bin/env python3"""A simple command-line todo app with JSON persistence.Usage: todo.py add <task> Add a new task todo.py list List all tasks todo.py done <id> Mark a task as done todo.py delete <id> Delete a task todo.py --file <path> Use a custom JSON file (default: ~/.todo.json)Tasks are stored in a JSON file of the form: {"next_id": 2, "todos": [{"id": 1, "task": "...", "done": false}]}The ``next_id`` counter is persisted so task IDs are never reused, evenafter a delete. This prevents a stale reference from completing ordeleting the wrong task later."""import argparseimport jsonimport osimport sysDEFAULT_FILE = os.path.join(os.path.expanduser("~"), ".todo.json")class TodoApp: """Todo list backed by a JSON file.""" def __init__(self, filepath=DEFAULT_FILE): self.filepath = filepath def load(self): """Return the store (``next_id`` counter + ``todos`` list).""" if not os.path.exists(self.filepath): return {"next_id": 1, "todos": []} with open(self.filepath, "r") as f: content = f.read() if not content.strip(): # Empty file (e.g. created by touch/mktemp) means a fresh store. return {"next_id": 1, "todos": []} data = json.loads(content) # Backwards compatibility: a bare list of todos. if isinstance(data, list): data = { "next_id": max((t["id"] for t in data), default=0) + 1, "todos": data, } return data def save(self, data): """Write the store to the JSON file.""" with open(self.filepath, "w") as f: json.dump(data, f, indent=2) def _allocate_id(self, data): next_id = data["next_id"] data["next_id"] = next_id + 1 return next_id def add(self, task): """Add a new task and return the created todo.""" data = self.load() todo = {"id": self._allocate_id(data), "task": task, "done": False} data["todos"].append(todo) self.save(data) return todo def list_todos(self): """Return all todos in insertion order.""" return self.load()["todos"] def done(self, todo_id): """Mark a todo as done and return it, or None if not found.""" data = self.load() for todo in data["todos"]: if todo["id"] == todo_id: todo["done"] = True self.save(data) return todo return None def delete(self, todo_id): """Delete a todo and return it, or None if not found.""" data = self.load() for i, todo in enumerate(data["todos"]): if todo["id"] == todo_id: removed = data["todos"].pop(i) self.save(data) return removed return Nonedef main(argv=None): parser = argparse.ArgumentParser(description="A simple todo app") parser.add_argument( "--file", default=DEFAULT_FILE, help="JSON file to store todos" ) subparsers = parser.add_subparsers(dest="command", required=True) add_parser = subparsers.add_parser("add", help="Add a task") add_parser.add_argument("task", help="Task description") subparsers.add_parser("list", help="List all tasks") done_parser = subparsers.add_parser("done", help="Mark a task as done") done_parser.add_argument("id", type=int, help="Task ID") delete_parser = subparsers.add_parser("delete", help="Delete a task") delete_parser.add_argument("id", type=int, help="Task ID") args = parser.parse_args(argv) app = TodoApp(args.file) if args.command == "add": todo = app.add(args.task) print(f"Added todo #{todo['id']}: {todo['task']}") elif args.command == "list": todos = app.list_todos() if not todos: print("No todos.") return 0 for todo in todos: status = "[x]" if todo["done"] else "[ ]" print(f"{todo['id']}. {status}{todo['task']}") elif args.command == "done": todo = app.done(args.id) if todo is None: print(f"Todo #{args.id} not found.", file=sys.stderr) return 1 print(f"Done: {todo['task']}") elif args.command == "delete": todo = app.delete(args.id) if todo is None: print(f"Todo #{args.id} not found.", file=sys.stderr) return 1 print(f"Deleted: {todo['task']}") return 0if __name__ == "__main__": sys.exit(main())Claude Code 生成的 test_todo.py (5048 bytes)"""Unit tests for the todo app."""import jsonimport pytestfrom todo import TodoApp, main@pytest.fixturedef app(tmp_path): return TodoApp(filepath=tmp_path / "todos.json")# --- add ------------------------------------------------------------------def test_add_returns_created_todo(app): todo = app.add("Buy milk") assert todo == {"id": 1, "task": "Buy milk", "done": False}def test_add_persists_to_json(app, tmp_path): app.add("Write tests") data = json.loads((tmp_path / "todos.json").read_text()) assert len(data["todos"]) == 1 assert data["todos"][0]["task"] == "Write tests" assert data["todos"][0]["done"] is Falsedef test_add_persists_next_id_counter(app, tmp_path): app.add("first") app.add("second") data = json.loads((tmp_path / "todos.json").read_text()) assert data["next_id"] == 3def test_add_increments_ids(app): app.add("first") second = app.add("second") assert second["id"] == 2 assert [t["id"] for t in app.list_todos()] == [1, 2]def test_add_after_delete_does_not_reuse_id(app): first = app.add("first") app.delete(first["id"]) third = app.add("third") assert third["id"] == 2 assert [t["id"] for t in app.list_todos()] == [2]def test_ids_are_monotonic_across_restarts(tmp_path): filepath = tmp_path / "todos.json" app1 = TodoApp(filepath=filepath) first = app1.add("first") app2 = TodoApp(filepath=filepath) app2.delete(first["id"]) third = app2.add("third") assert third["id"] == 2# --- list -----------------------------------------------------------------def test_list_empty_when_no_file(tmp_path): app = TodoApp(filepath=tmp_path / "nonexistent.json") assert app.list_todos() == []def test_list_empty_when_file_is_empty(tmp_path): filepath = tmp_path / "todos.json" filepath.write_text("") app = TodoApp(filepath=filepath) assert app.list_todos() == []def test_add_over_empty_file(tmp_path): filepath = tmp_path / "todos.json" filepath.write_text("") app = TodoApp(filepath=filepath) todo = app.add("Task") assert todo["id"] == 1def test_list_empty_when_file_is_empty_store(tmp_path): filepath = tmp_path / "todos.json" filepath.write_text(json.dumps({"next_id": 1, "todos": []})) app = TodoApp(filepath=filepath) assert app.list_todos() == []def test_list_returns_all_todos_in_order(app): app.add("a") app.add("b") todos = app.list_todos() assert [t["task"] for t in todos] == ["a", "b"]# --- done -----------------------------------------------------------------def test_done_marks_todo(app): todo = app.add("Task") result = app.done(todo["id"]) assert result["done"] is True assert app.list_todos()[0]["done"] is Truedef test_done_persists_to_json(app, tmp_path): todo = app.add("Task") app.done(todo["id"]) data = json.loads((tmp_path / "todos.json").read_text()) assert data["todos"][0]["done"] is Truedef test_done_missing_returns_none(app): assert app.done(999) is None# --- delete ---------------------------------------------------------------def test_delete_removes_todo(app): todo = app.add("Task") removed = app.delete(todo["id"]) assert removed["task"] == "Task" assert app.list_todos() == []def test_delete_persists_to_json(app, tmp_path): todo = app.add("Task") app.delete(todo["id"]) data = json.loads((tmp_path / "todos.json").read_text()) assert data["todos"] == []def test_delete_missing_returns_none(app): assert app.delete(999) is None# --- CLI ------------------------------------------------------------------def test_cli_add_and_list(tmp_path, capsys): filepath = tmp_path / "todos.json" assert main(["--file", str(filepath), "add", "Buy milk"]) == 0 assert "Added todo #1: Buy milk" in capsys.readouterr().out assert main(["--file", str(filepath), "list"]) == 0 out = capsys.readouterr().out assert "1." in out assert "Buy milk" in outdef test_cli_list_empty(tmp_path, capsys): filepath = tmp_path / "todos.json" assert main(["--file", str(filepath), "list"]) == 0 assert "No todos." in capsys.readouterr().outdef test_cli_done_and_delete(tmp_path, capsys): filepath = tmp_path / "todos.json" main(["--file", str(filepath), "add", "Task"]) assert main(["--file", str(filepath), "done", "1"]) == 0 assert "Done: Task" in capsys.readouterr().out assert main(["--file", str(filepath), "delete", "1"]) == 0 assert "Deleted: Task" in capsys.readouterr().outdef test_cli_done_missing_id_returns_nonzero(tmp_path, capsys): filepath = tmp_path / "todos.json" assert main(["--file", str(filepath), "done", "1"]) == 1 assert "not found" in capsys.readouterr().errdef test_cli_delete_missing_id_returns_nonzero(tmp_path, capsys): filepath = tmp_path / "todos.json" assert main(["--file", str(filepath), "delete", "1"]) == 1 assert "not found" in capsys.readouterr().errClaude Code pytest 复验输出_after_delete_does_not_reuse_id PASSED [ 22%]test_todo.py::test_ids_are_monotonic_across_restarts PASSED [ 27%]test_todo.py::test_list_empty_when_no_file PASSED [ 31%]test_todo.py::test_list_empty_when_file_is_empty PASSED [ 36%]test_todo.py::test_add_over_empty_file PASSED [ 40%]test_todo.py::test_list_empty_when_file_is_empty_store PASSED [ 45%]test_todo.py::test_list_returns_all_todos_in_order PASSED [ 50%]test_todo.py::test_done_marks_todo PASSED [ 54%]test_todo.py::test_done_persists_to_json PASSED [ 59%]test_todo.py::test_done_missing_returns_none PASSED [ 63%]test_todo.py::test_delete_removes_todo PASSED [ 68%]test_todo.py::test_delete_persists_to_json PASSED [ 72%]test_todo.py::test_delete_missing_returns_none PASSED [ 77%]test_todo.py::test_cli_add_and_list PASSED [ 81%]test_todo.py::test_cli_list_empty PASSED [ 86%]test_todo.py::test_cli_done_and_delete PASSED [ 90%]test_todo.py::test_cli_done_missing_id_returns_nonzero PASSED [ 95%]test_todo.py::test_cli_delete_missing_id_returns_nonzero PASSED [100%]============================== 22 passed in 0.16s ==============================OpenCode v1.18.12 — 12/12OpenCode 生成的 todo.py (3444 bytes)#!/usr/bin/env python3"""A simple CLI todo app with JSON persistence."""import argparseimport jsonimport osimport sysDEFAULT_STORE = "todos.json"class TodoApp: def __init__(self, store_path=DEFAULT_STORE): self.store_path = store_path self.todos = self._load() def _load(self): if not os.path.exists(self.store_path): return [] try: with open(self.store_path, "r") as fh: data = json.load(fh) except (json.JSONDecodeError, OSError): return [] if not isinstance(data, list): return [] return data def _save(self): with open(self.store_path, "w") as fh: json.dump(self.todos, fh, indent=2) def add(self, title): todo = {"id": self._next_id(), "title": title, "done": False} self.todos.append(todo) self._save() return todo def _next_id(self): return max((t["id"] for t in self.todos), default=0) + 1 def list(self, show_done=True): todos = self.todos if not show_done: todos = [t for t in todos if not t["done"]] return todos def done(self, todo_id): for todo in self.todos: if todo["id"] == todo_id: todo["done"] = True self._save() return todo raise KeyError(f"No todo with id {todo_id}") def delete(self, todo_id): for todo in self.todos: if todo["id"] == todo_id: self.todos.remove(todo) self._save() return todo raise KeyError(f"No todo with id {todo_id}")def main(argv=None): parser = argparse.ArgumentParser(prog="todo", description="A simple todo app.") parser.add_argument( "--store", default=DEFAULT_STORE, help="path to the JSON store file" ) sub = parser.add_subparsers(dest="command", required=True) p_add = sub.add_parser("add", help="add a todo") p_add.add_argument("title", help="title of the todo") p_list = sub.add_parser("list", help="list todos") p_list.add_argument( "--all", action="store_true", help="show completed todos too" ) p_done = sub.add_parser("done", help="mark a todo as done") p_done.add_argument("id", type=int, help="todo id") p_delete = sub.add_parser("delete", help="delete a todo") p_delete.add_argument("id", type=int, help="todo id") args = parser.parse_args(argv) app = TodoApp(args.store) try: if args.command == "add": todo = app.add(args.title) print(f"Added todo {todo['id']}: {todo['title']}") elif args.command == "list": todos = app.list(show_done=args.all) if not todos: print("No todos.") return 0 for todo in todos: status = "[x]" if todo["done"] else "[ ]" print(f"{status}{todo['id']}: {todo['title']}") elif args.command == "done": todo = app.done(args.id) print(f"Marked todo {todo['id']} as done: {todo['title']}") elif args.command == "delete": todo = app.delete(args.id) print(f"Deleted todo {todo['id']}: {todo['title']}") except KeyError as exc: print(str(exc), file=sys.stderr) return 1 return 0if __name__ == "__main__": sys.exit(main())OpenCode 生成的 test_todo.py (2052 bytes)import jsonimport osimport pytestfrom todo import TodoApp@pytest.fixturedef app(tmp_path): store = tmp_path / "todos.json" return TodoApp(str(store))def test_add_creates_todo(app): todo = app.add("Buy milk") assert todo["title"] == "Buy milk" assert todo["done"] is False assert todo["id"] == 1def test_add_persists_to_disk(app): app.add("Walk the dog") assert os.path.exists(app.store_path) with open(app.store_path) as fh: data = json.load(fh) assert data[0]["title"] == "Walk the dog"def test_list_empty(app): assert app.list() == []def test_list_returns_all_todos(app): app.add("A") app.add("B") assert len(app.list()) == 2def test_list_hides_done_when_requested(app): app.add("A") app.add("B") app.done(1) remaining = app.list(show_done=False) assert [t["id"] for t in remaining] == [2]def test_done_marks_todo_completed(app): app.add("A") todo = app.done(1) assert todo["done"] is True assert app.todos[0]["done"] is Truedef test_done_unknown_id_raises(app): with pytest.raises(KeyError): app.done(999)def test_delete_removes_todo(app): app.add("A") app.add("B") todo = app.delete(1) assert todo["id"] == 1 assert len(app.list()) == 1 assert app.list()[0]["id"] == 2def test_delete_unknown_id_raises(app): with pytest.raises(KeyError): app.delete(999)def test_next_id_increments_after_delete(app): app.add("A") app.add("B") app.delete(1) todo = app.add("C") assert todo["id"] == 3def test_load_existing_store(tmp_path): store = tmp_path / "todos.json" store.write_text(json.dumps([{"id": 1, "title": "existing", "done": True}])) app = TodoApp(str(store)) assert len(app.list()) == 1 assert app.list()[0]["title"] == "existing"def test_load_corrupt_store_returns_empty(tmp_path): store = tmp_path / "todos.json" store.write_text("{not valid json") app = TodoApp(str(store)) assert app.list() == []OpenCode pytest 复验输出============================= test session starts ==============================platform linux -- Python 3.13.5, pytest-9.1.1, pluggy-1.6.0 -- /tmp/bench-venv/bin/pythoncachedir: .pytest_cacherootdir: /home/agent/coding-benchmark/test-opencodecollecting ... collected 12 itemstest_todo.py::test_add_creates_todo PASSED [ 8%]test_todo.py::test_add_persists_to_disk PASSED [ 16%]test_todo.py::test_list_empty PASSED [ 25%]test_todo.py::test_list_returns_all_todos PASSED [ 33%]test_todo.py::test_list_hides_done_when_requested PASSED [ 41%]test_todo.py::test_done_marks_todo_completed PASSED [ 50%]test_todo.py::test_done_unknown_id_raises PASSED [ 58%]test_todo.py::test_delete_removes_todo PASSED [ 66%]test_todo.py::test_delete_unknown_id_raises PASSED [ 75%]test_todo.py::test_next_id_increments_after_delete PASSED [ 83%]test_todo.py::test_load_existing_store PASSED [ 91%]test_todo.py::test_load_corrupt_store_returns_empty PASSED [100%]============================== 12 passed in 0.07s ==============================Codex v0.50.0 — 13/13Codex 生成的 todo.py (4025 bytes)#!/usr/bin/env python3"""A simple command-line todo application with JSON persistence."""import argparseimport jsonimport sysfrom pathlib import PathDEFAULT_FILE = Path(__file__).parent / "todos.json"def load_todos(path=DEFAULT_FILE): """Load todos from a JSON file, returning an empty list if missing or corrupt.""" if not Path(path).exists(): return [] try: with open(path, "r", encoding="utf-8") as f: data = json.load(f) except (json.JSONDecodeError, OSError): return [] return data if isinstance(data, list) else []def save_todos(todos, path=DEFAULT_FILE): """Save a list of todos to a JSON file.""" path = Path(path) path.parent.mkdir(parents=True, exist_ok=True) with open(path, "w", encoding="utf-8") as f: json.dump(todos, f, indent=2)def add_todo(description, path=DEFAULT_FILE): """Add a new todo and return its id.""" todos = load_todos(path) todo_id = max((t["id"] for t in todos), default=0) + 1 todos.append({"id": todo_id, "description": description, "done": False}) save_todos(todos, path) return todo_iddef list_todos(path=DEFAULT_FILE): """Return the list of todos.""" return load_todos(path)def mark_done(todo_id, path=DEFAULT_FILE): """Mark a todo as done; return True if it was found.""" todos = load_todos(path) for todo in todos: if todo["id"] == todo_id: todo["done"] = True save_todos(todos, path) return True return Falsedef delete_todo(todo_id, path=DEFAULT_FILE): """Delete a todo; return True if it was found.""" todos = load_todos(path) remaining = [t for t in todos if t["id"] != todo_id] if len(remaining) == len(todos): return False save_todos(remaining, path) return Truedef main(argv=None): """Parse command-line arguments and run the requested command.""" parser = argparse.ArgumentParser(prog="todo", description="A simple todo app.") subparsers = parser.add_subparsers(dest="command", required=True) add_parser = subparsers.add_parser("add", help="Add a new todo") add_parser.add_argument("description", nargs="+", help="Todo description") add_parser.add_argument("-f", "--file", default=DEFAULT_FILE, help="JSON file to use") list_parser = subparsers.add_parser("list", help="List all todos") list_parser.add_argument("-f", "--file", default=DEFAULT_FILE, help="JSON file to use") done_parser = subparsers.add_parser("done", help="Mark a todo as done") done_parser.add_argument("todo_id", type=int, help="Todo ID") done_parser.add_argument("-f", "--file", default=DEFAULT_FILE, help="JSON file to use") delete_parser = subparsers.add_parser("delete", help="Delete a todo") delete_parser.add_argument("todo_id", type=int, help="Todo ID") delete_parser.add_argument("-f", "--file", default=DEFAULT_FILE, help="JSON file to use") args = parser.parse_args(argv) if args.command == "add": description = " ".join(args.description) todo_id = add_todo(description, args.file) print(f"Added todo #{todo_id}: {description}") elif args.command == "list": todos = list_todos(args.file) if not todos: print("No todos.") return 0 for todo in todos: status = "[x]" if todo["done"] else "[ ]" print(f"{status}{todo['id']}: {todo['description']}") elif args.command == "done": if mark_done(args.todo_id, args.file): print(f"Marked todo #{args.todo_id} as done.") else: print(f"No todo with id {args.todo_id}.", file=sys.stderr) return 1 elif args.command == "delete": if delete_todo(args.todo_id, args.file): print(f"Deleted todo #{args.todo_id}.") else: print(f"No todo with id {args.todo_id}.", file=sys.stderr) return 1 return 0if __name__ == "__main__": sys.exit(main())Codex 生成的 test_todo.py (3197 bytes)import pytestimport tododef test_load_missing_file_returns_empty_list(tmp_path): assert todo.load_todos(tmp_path / "nope.json") == []def test_save_and_load_round_trip(tmp_path): path = tmp_path / "todos.json" todos = [{"id": 1, "description": "buy milk", "done": False}] todo.save_todos(todos, path) assert todo.load_todos(path) == todosdef test_load_corrupt_json_returns_empty_list(tmp_path): path = tmp_path / "todos.json" path.write_text("{not json", encoding="utf-8") assert todo.load_todos(path) == []def test_load_non_list_json_returns_empty_list(tmp_path): path = tmp_path / "todos.json" path.write_text('{"key": "value"}', encoding="utf-8") assert todo.load_todos(path) == []def test_add_todo_assigns_incrementing_ids(tmp_path): path = tmp_path / "todos.json" first = todo.add_todo("first task", path) second = todo.add_todo("second task", path) assert first == 1 assert second == 2 todos = todo.load_todos(path) assert todos == [ {"id": 1, "description": "first task", "done": False}, {"id": 2, "description": "second task", "done": False}, ]def test_mark_done(tmp_path): path = tmp_path / "todos.json" todo.add_todo("task", path) assert todo.mark_done(1, path) is True assert todo.load_todos(path)[0]["done"] is Truedef test_mark_done_missing_id_returns_false(tmp_path): path = tmp_path / "todos.json" assert todo.mark_done(99, path) is Falsedef test_delete_todo(tmp_path): path = tmp_path / "todos.json" todo.add_todo("keep", path) todo.add_todo("remove", path) assert todo.delete_todo(2, path) is True assert [t["description"] for t in todo.load_todos(path)] == ["keep"]def test_delete_missing_id_returns_false(tmp_path): path = tmp_path / "todos.json" assert todo.delete_todo(99, path) is Falsedef test_main_add_and_list(tmp_path, capsys): path = tmp_path / "todos.json" assert todo.main(["add", "buy", "milk", "-f", str(path)]) == 0 assert todo.main(["list", "-f", str(path)]) == 0 out = capsys.readouterr().out assert "buy milk" in out assert "Added todo" in outdef test_list_shows_done_status(tmp_path, capsys): path = tmp_path / "todos.json" todo.add_todo("pending", path) todo.add_todo("finished", path) todo.mark_done(2, path) todo.main(["list", "-f", str(path)]) out = capsys.readouterr().out assert "[ ] 1: pending" in out assert "[x] 2: finished" in outdef test_main_done_and_delete(tmp_path, capsys): path = tmp_path / "todos.json" todo.main(["add", "task", "-f", str(path)]) assert todo.main(["done", "1", "-f", str(path)]) == 0 assert todo.main(["delete", "1", "-f", str(path)]) == 0 assert todo.main(["list", "-f", str(path)]) == 0 assert "No todos." in capsys.readouterr().outdef test_main_missing_todo_returns_error(tmp_path, capsys): path = tmp_path / "todos.json" assert todo.main(["done", "1", "-f", str(path)]) == 1 assert "No todo with id 1" in capsys.readouterr().err assert todo.main(["delete", "1", "-f", str(path)]) == 1 assert "No todo with id 1" in capsys.readouterr().errCodex pytest 复验输出============================= test session starts ==============================platform linux -- Python 3.13.5, pytest-9.1.1, pluggy-1.6.0 -- /tmp/bench-venv/bin/pythoncachedir: .pytest_cacherootdir: /home/agent/coding-benchmark/test-codexcollecting ... collected 13 itemstest_todo.py::test_load_missing_file_returns_empty_list PASSED [ 7%]test_todo.py::test_save_and_load_round_trip PASSED [ 15%]test_todo.py::test_load_corrupt_json_returns_empty_list PASSED [ 23%]test_todo.py::test_load_non_list_json_returns_empty_list PASSED [ 30%]test_todo.py::test_add_todo_assigns_incrementing_ids PASSED [ 38%]test_todo.py::test_mark_done PASSED [ 46%]test_todo.py::test_mark_done_missing_id_returns_false PASSED [ 53%]test_todo.py::test_delete_todo PASSED [ 61%]test_todo.py::test_delete_missing_id_returns_false PASSED [ 69%]test_todo.py::test_main_add_and_list PASSED [ 76%]test_todo.py::test_list_shows_done_status PASSED [ 84%]test_todo.py::test_main_done_and_delete PASSED [ 92%]test_todo.py::test_main_missing_todo_returns_error PASSED [100%]============================== 13 passed in 0.10s ==============================Cline v3.0.49 — 24/24 (手动修复后)Cline 生成的 todo.py (4751 bytes)#!/usr/bin/env python3"""A simple command-line todo app with JSON persistence.Usage: python todo.py add "Buy milk" python todo.py list python todo.py done 1 python todo.py delete 1Tasks are stored in a JSON file (todo.json by default). Set the TODO_FILEenvironment variable to use a different data file."""import argparseimport jsonimport osimport sysDEFAULT_DATA_FILE = "todo.json"class TodoList: """A todo list backed by a JSON file. Each task is a dict: {"id": int, "description": str, "done": bool}. """ def __init__(self, data_file=DEFAULT_DATA_FILE): self.data_file = data_file self.tasks = [] self._max_id = 0 self.load() def load(self): """Load tasks from the JSON data file. A missing or corrupt file simply yields an empty list so the app never crashes on startup. """ try: with open(self.data_file, "r", encoding="utf-8") as fh: self.tasks = json.load(fh) except (FileNotFoundError, json.JSONDecodeError, OSError): self.tasks = [] self._max_id = max((task["id"] for task in self.tasks), default=0) def save(self): """Persist tasks to the JSON data file.""" with open(self.data_file, "w", encoding="utf-8") as fh: json.dump(self.tasks, fh, indent=2) def _next_id(self): """Return the next task id: never reuse an id, even after deletion.""" self._max_id += 1 return self._max_id def add(self, description): """Add a new task and return it.""" task = {"id": self._next_id(), "description": description, "done": False} self.tasks.append(task) self.save() return task def get(self, task_id): """Return the task with the given id, or None if it does not exist.""" for task in self.tasks: if task["id"] == task_id: return task return None def list_tasks(self): """Return a copy of all tasks in insertion order.""" return list(self.tasks) def done(self, task_id): """Mark the task with the given id as done and return it. Raises KeyError if no such task exists. """ task = self.get(task_id) if task is None: raise KeyError(f"no task with id {task_id}") task["done"] = True self.save() return task def delete(self, task_id): """Remove the task with the given id and return it. Raises KeyError if no such task exists. """ for index, task in enumerate(self.tasks): if task["id"] == task_id: removed = self.tasks.pop(index) self.save() return removed raise KeyError(f"no task with id {task_id}")def format_task(task): """Render a task as a one-line string, e.g. '[x] 1. Buy milk'.""" marker = "[x]" if task["done"] else "[ ]" return f"{marker}{task['id']}. {task['description']}"def main(argv=None): parser = argparse.ArgumentParser( prog="todo", description="A simple CLI todo app with JSON persistence." ) subparsers = parser.add_subparsers(dest="command", required=True) add_parser = subparsers.add_parser("add", help="add a new task") add_parser.add_argument("description", help="task description") subparsers.add_parser("list", help="list all tasks") done_parser = subparsers.add_parser("done", help="mark a task as done") done_parser.add_argument("id", type=int, help="task id") delete_parser = subparsers.add_parser("delete", help="delete a task") delete_parser.add_argument("id", type=int, help="task id") args = parser.parse_args(argv) data_file = os.environ.get("TODO_FILE", DEFAULT_DATA_FILE) todo = TodoList(data_file) try: if args.command == "add": task = todo.add(args.description) print(f"Added task {task['id']}: {task['description']}") elif args.command == "list": tasks = todo.list_tasks() if not tasks: print("No tasks yet.") else: for task in tasks: print(format_task(task)) elif args.command == "done": todo.done(args.id) print(f"Marked task {args.id} as done.") elif args.command == "delete": task = todo.delete(args.id) print(f"Deleted task {task['id']}: {task['description']}") except KeyError as exc: msg = exc.args[0] if exc.args else str(exc) print(f"Error: {msg}", file=sys.stderr) return 1 return 0if __name__ == "__main__": sys.exit(main())Cline 生成的 test_todo.py (5061 bytes)"""Unit tests for the todo CLI app."""import jsonimport pytestimport todo@pytest.fixturedef todo_list(tmp_path): """A TodoList backed by a temp JSON file.""" return todo.TodoList(tmp_path / "tasks.json")@pytest.fixturedef cli(tmp_path, monkeypatch): """Point the CLI at a temp data file via the TODO_FILE env var.""" data_file = tmp_path / "tasks.json" monkeypatch.setenv("TODO_FILE", str(data_file)) return data_file# --- Model: add ---def test_add_creates_task(todo_list): task = todo_list.add("Buy milk") assert task == {"id": 1, "description": "Buy milk", "done": False} assert todo_list.tasks == [task]def test_add_persists_to_disk(todo_list): todo_list.add("Buy milk") assert todo_list.data_file.exists() saved = json.loads(todo_list.data_file.read_text()) assert saved == [{"id": 1, "description": "Buy milk", "done": False}]def test_add_increments_ids(todo_list): first = todo_list.add("First") second = todo_list.add("Second") assert first["id"] == 1 assert second["id"] == 2# --- Model: list ---def test_list_empty(todo_list): assert todo_list.list_tasks() == []def test_list_returns_all_tasks(todo_list): todo_list.add("First") todo_list.add("Second") tasks = todo_list.list_tasks() assert [task["description"] for task in tasks] == ["First", "Second"]def test_list_returns_copy(todo_list): todo_list.add("First") listing = todo_list.list_tasks() listing.clear() assert len(todo_list.tasks) == 1# --- Model: done ---def test_done_marks_task(todo_list): todo_list.add("First") task = todo_list.done(1) assert task["done"] is True assert todo_list.get(1)["done"] is Truedef test_done_persists(todo_list): todo_list.add("First") todo_list.done(1) saved = json.loads(todo_list.data_file.read_text()) assert saved[0]["done"] is Truedef test_done_unknown_id_raises(todo_list): with pytest.raises(KeyError): todo_list.done(99)# --- Model: delete ---def test_delete_removes_task(todo_list): todo_list.add("First") todo_list.add("Second") removed = todo_list.delete(1) assert removed["id"] == 1 assert [t["description"] for t in todo_list.list_tasks()] == ["Second"]def test_delete_persists(todo_list): todo_list.add("First") todo_list.delete(1) saved = json.loads(todo_list.data_file.read_text()) assert saved == []def test_delete_unknown_id_raises(todo_list): with pytest.raises(KeyError): todo_list.delete(99)def test_next_id_survives_delete(todo_list): # Ids are derived from max existing id, so a deleted id is not reused. todo_list.add("First") todo_list.delete(1) task = todo_list.add("Second") assert task["id"] == 2# --- Model: persistence / loading ---def test_reload_from_disk(tmp_path): path = tmp_path / "tasks.json" todo.TodoList(path).add("Persisted") reloaded = todo.TodoList(path) assert reloaded.list_tasks() == [ {"id": 1, "description": "Persisted", "done": False} ]def test_corrupt_file_starts_empty(tmp_path): path = tmp_path / "tasks.json" path.write_text("{not valid json") store = todo.TodoList(path) assert store.tasks == []def test_missing_file_starts_empty(tmp_path): store = todo.TodoList(tmp_path / "nonexistent.json") assert store.tasks == []def test_format_task(): assert todo.format_task({"id": 1, "description": "Buy milk", "done": False}) == ( "[ ] 1. Buy milk" ) assert todo.format_task({"id": 2, "description": "Pay rent", "done": True}) == ( "[x] 2. Pay rent" )# --- CLI end-to-end ---def test_cli_add(cli, capsys): assert todo.main(["add", "Buy milk"]) == 0 assert "Added task 1: Buy milk" in capsys.readouterr().outdef test_cli_list_empty(cli, capsys): assert todo.main(["list"]) == 0 assert "No tasks yet." in capsys.readouterr().outdef test_cli_list_shows_tasks(cli, capsys): todo.main(["add", "First"]) todo.main(["add", "Second"]) assert todo.main(["list"]) == 0 out = capsys.readouterr().out assert "[ ] 1. First" in out assert "[ ] 2. Second" in outdef test_cli_done(cli, capsys): todo.main(["add", "Task"]) assert todo.main(["done", "1"]) == 0 assert "Marked task 1 as done." in capsys.readouterr().out todo.main(["list"]) assert "[x] 1. Task" in capsys.readouterr().outdef test_cli_delete(cli, capsys): todo.main(["add", "Task"]) assert todo.main(["delete", "1"]) == 0 assert "Deleted task 1: Task" in capsys.readouterr().out todo.main(["list"]) assert "No tasks yet." in capsys.readouterr().outdef test_cli_done_unknown_id_fails(cli, capsys): assert todo.main(["done", "42"]) == 1 assert "Error: no task with id 42" in capsys.readouterr().errdef test_cli_state_persists_across_invocations(cli, capsys): todo.main(["add", "Persist me"]) todo.main(["list"]) assert "1. Persist me" in capsys.readouterr().outCline pytest 复验输出e_marks_task PASSED [ 29%]test_todo.py::test_done_persists PASSED [ 33%]test_todo.py::test_done_unknown_id_raises PASSED [ 37%]test_todo.py::test_delete_removes_task PASSED [ 41%]test_todo.py::test_delete_persists PASSED [ 45%]test_todo.py::test_delete_unknown_id_raises PASSED [ 50%]test_todo.py::test_next_id_survives_delete PASSED [ 54%]test_todo.py::test_reload_from_disk PASSED [ 58%]test_todo.py::test_corrupt_file_starts_empty PASSED [ 62%]test_todo.py::test_missing_file_starts_empty PASSED [ 66%]test_todo.py::test_format_task PASSED [ 70%]test_todo.py::test_cli_add PASSED [ 75%]test_todo.py::test_cli_list_empty PASSED [ 79%]test_todo.py::test_cli_list_shows_tasks PASSED [ 83%]test_todo.py::test_cli_done PASSED [ 87%]test_todo.py::test_cli_delete PASSED [ 91%]test_todo.py::test_cli_done_unknown_id_fails PASSED [ 95%]test_todo.py::test_cli_state_persists_across_invocations PASSED [100%]============================== 24 passed in 0.16s ==============================Kilo Code v7.4.17 — 10/10Kilo Code 生成的 todo.py (2985 bytes)#!/usr/bin/env python3"""A simple CLI todo app with JSON persistence."""import argparseimport jsonimport sysfrom pathlib import PathDEFAULT_STORE = Path(__file__).with_name("todos.json")class Todo: def __init__(self, text, done=False, store_path=DEFAULT_STORE): self.text = text self.is_done = done self.store_path = Path(store_path) @property def store(self): if not self.store_path.exists(): return [] return json.loads(self.store_path.read_text() or "[]") @store.setter def store(self, items): self.store_path.write_text(json.dumps(items, indent=2)) def _next_id(self, items): return max((item["id"] for item in items), default=0) + 1 def add(self): items = self.store todo_id = self._next_id(items) items.append({"id": todo_id, "text": self.text, "done": self.is_done}) self.store = items return todo_id def list(self): return self.store def done(self, todo_id): items = self.store for item in items: if item["id"] == todo_id: item["done"] = True self.store = items return True return False def delete(self, todo_id): items = self.store new_items = [item for item in items if item["id"] != todo_id] if len(new_items) == len(items): return False self.store = new_items return Truedef main(argv=None): parser = argparse.ArgumentParser(description="Manage a todo list.") subparsers = parser.add_subparsers(dest="command", required=True) add_parser = subparsers.add_parser("add", help="Add a new todo.") add_parser.add_argument("text", help="The todo text.") subparsers.add_parser("list", help="List all todos.") done_parser = subparsers.add_parser("done", help="Mark a todo done.") done_parser.add_argument("id", type=int, help="The todo id.") delete_parser = subparsers.add_parser("delete", help="Delete a todo.") delete_parser.add_argument("id", type=int, help="The todo id.") args = parser.parse_args(argv) todo = Todo("") if args.command == "add": todo.text = args.text todo_id = todo.add() print(f"Added todo {todo_id}") elif args.command == "list": for item in todo.list(): status = "x" if item["done"] else " " print(f"{item['id']} [{status}] {item['text']}") elif args.command == "done": if todo.done(args.id): print(f"Marked todo {args.id} as done") else: print(f"Todo {args.id} not found", file=sys.stderr) return 1 elif args.command == "delete": if todo.delete(args.id): print(f"Deleted todo {args.id}") else: print(f"Todo {args.id} not found", file=sys.stderr) return 1 return 0if __name__ == "__main__": sys.exit(main())Kilo Code 生成的 test_todo.py (1569 bytes)"""Unit tests for todo.py."""import jsonimport pytestfrom todo import Todo@pytest.fixturedef todo(tmp_path): return Todo("", store_path=tmp_path / "todos.json")def test_add_creates_todo(todo): todo_id = todo.add() items = todo.list() assert len(items) == 1 assert items[0]["id"] == todo_id assert items[0]["text"] == "" assert items[0]["done"] is Falsedef test_add_assigns_incrementing_ids(todo): first = todo.add() second = todo.add() assert second == first + 1def test_add_persists_to_disk(todo): todo.add() raw = todo.store_path.read_text() assert json.loads(raw)[0]["text"] == ""def test_list_empty(todo): assert todo.list() == []def test_list_returns_all_todos(todo): todo.text = "first" todo.add() todo.text = "second" todo.add() assert [item["text"] for item in todo.list()] == ["first", "second"]def test_done_marks_todo(todo): todo_id = todo.add() assert todo.done(todo_id) is True items = todo.list() assert items[0]["done"] is Truedef test_done_missing_id_returns_false(todo): assert todo.done(999) is Falsedef test_delete_removes_todo(todo): todo_id = todo.add() assert todo.delete(todo_id) is True assert todo.list() == []def test_delete_missing_id_returns_false(todo): assert todo.delete(999) is Falsedef test_delete_only_removes_target(todo): first = todo.add() todo.delete(first) assert todo.list() == [] second = todo.add() assert [item["id"] for item in todo.list()] == [second]Kilo Code pytest 复验输出============================= test session starts ==============================platform linux -- Python 3.13.5, pytest-9.1.1, pluggy-1.6.0 -- /tmp/bench-venv/bin/pythoncachedir: .pytest_cacherootdir: /home/agent/coding-benchmark/test-kilocollecting ... collected 10 itemstest_todo.py::test_add_creates_todo PASSED [ 10%]test_todo.py::test_add_assigns_incrementing_ids PASSED [ 20%]test_todo.py::test_add_persists_to_disk PASSED [ 30%]test_todo.py::test_list_empty PASSED [ 40%]test_todo.py::test_list_returns_all_todos PASSED [ 50%]test_todo.py::test_done_marks_todo PASSED [ 60%]test_todo.py::test_done_missing_id_returns_false PASSED [ 70%]test_todo.py::test_delete_removes_todo PASSED [ 80%]test_todo.py::test_delete_missing_id_returns_false PASSED [ 90%]test_todo.py::test_delete_only_removes_target PASSED [100%]============================== 10 passed in 0.06s ==============================Crush vgo 编译版 — 13/13Crush 生成的 todo.py (3654 bytes)#!/usr/bin/env python3"""A minimal CLI todo app with JSON persistence.Commands: add <description> Add a new todo list List all todos (with done status) done <id> Mark a todo as done delete <id> Delete a todo"""import argparseimport jsonimport osimport sysDEFAULT_FILE = os.path.join(os.path.dirname(os.path.abspath(__file__)), "todos.json")def load_todos(path=DEFAULT_FILE): """Load todos from the JSON file, returning an empty list if missing.""" if not os.path.exists(path): return [] try: with open(path, "r", encoding="utf-8") as f: data = json.load(f) except (json.JSONDecodeError, OSError): return [] if not isinstance(data, list): return [] return datadef save_todos(todos, path=DEFAULT_FILE): """Persist todos to the JSON file.""" with open(path, "w", encoding="utf-8") as f: json.dump(todos, f, indent=2)def next_id(todos): """Return the next available todo id.""" return max((t.get("id", 0) for t in todos), default=0) + 1def add_todo(description, path=DEFAULT_FILE): """Add a new todo and return it.""" todos = load_todos(path) todo = {"id": next_id(todos), "description": description, "done": False} todos.append(todo) save_todos(todos, path) return tododef list_todos(path=DEFAULT_FILE): """Return the list of todos.""" return load_todos(path)def mark_done(todo_id, path=DEFAULT_FILE): """Mark a todo as done. Returns the todo, or None if not found.""" todos = load_todos(path) for todo in todos: if todo["id"] == todo_id: todo["done"] = True save_todos(todos, path) return todo return Nonedef delete_todo(todo_id, path=DEFAULT_FILE): """Delete a todo. Returns the deleted todo, or None if not found.""" todos = load_todos(path) for i, todo in enumerate(todos): if todo["id"] == todo_id: removed = todos.pop(i) save_todos(todos, path) return removed return Nonedef main(argv=None): parser = argparse.ArgumentParser(prog="todo", description="A simple todo app") sub = parser.add_subparsers(dest="command", required=True) sub.add_parser("list", help="list all todos") p_add = sub.add_parser("add", help="add a todo") p_add.add_argument("description", help="todo description") p_done = sub.add_parser("done", help="mark a todo as done") p_done.add_argument("id", type=int, help="todo id") p_del = sub.add_parser("delete", help="delete a todo") p_del.add_argument("id", type=int, help="todo id") args = parser.parse_args(argv) if args.command == "add": todo = add_todo(args.description) print(f"Added todo #{todo['id']}: {todo['description']}") elif args.command == "list": todos = list_todos() if not todos: print("No todos.") return 0 for todo in todos: status = "[x]" if todo["done"] else "[ ]" print(f"{status}{todo['id']}: {todo['description']}") elif args.command == "done": todo = mark_done(args.id) if todo is None: print(f"Todo #{args.id} not found.", file=sys.stderr) return 1 print(f"Done: {todo['description']}") elif args.command == "delete": todo = delete_todo(args.id) if todo is None: print(f"Todo #{args.id} not found.", file=sys.stderr) return 1 print(f"Deleted: {todo['description']}") return 0if __name__ == "__main__": sys.exit(main())Crush 生成的 test_todo.py (2861 bytes)"""Unit tests for todo.py."""import jsonimport osimport pytestimport todo@pytest.fixturedef todo_file(tmp_path): """Return a path to a temp JSON file for a test.""" return str(tmp_path / "todos.json")@pytest.fixturedef populated_file(todo_file): """Create a temp file pre-populated with two todos.""" data = [ {"id": 1, "description": "buy milk", "done": False}, {"id": 2, "description": "walk dog", "done": True}, ] with open(todo_file, "w", encoding="utf-8") as f: json.dump(data, f) return todo_filedef test_load_missing_file(todo_file): assert todo.load_todos(todo_file) == []def test_load_invalid_json(todo_file): with open(todo_file, "w", encoding="utf-8") as f: f.write("not json{") assert todo.load_todos(todo_file) == []def test_load_non_list(todo_file): with open(todo_file, "w", encoding="utf-8") as f: json.dump({"wrong": "shape"}, f) assert todo.load_todos(todo_file) == []def test_save_and_load_roundtrip(todo_file): todos = [{"id": 1, "description": "x", "done": False}] todo.save_todos(todos, todo_file) assert todo.load_todos(todo_file) == todosdef test_next_id_empty(): assert todo.next_id([]) == 1def test_next_id_continues(populated_file): assert todo.next_id(todo.load_todos(populated_file)) == 3def test_add_todo(todo_file): added = todo.add_todo("buy milk", todo_file) assert added == {"id": 1, "description": "buy milk", "done": False} assert todo.load_todos(todo_file) == [added]def test_add_todo_increments_ids(populated_file): first = todo.add_todo("first", populated_file) second = todo.add_todo("second", populated_file) assert first["id"] == 3 assert second["id"] == 4def test_list_todos(populated_file): assert todo.list_todos(populated_file) == [ {"id": 1, "description": "buy milk", "done": False}, {"id": 2, "description": "walk dog", "done": True}, ]def test_mark_done(populated_file): updated = todo.mark_done(1, populated_file) assert updated == {"id": 1, "description": "buy milk", "done": True} stored = todo.load_todos(populated_file) assert stored[0]["done"] is True assert stored[1]["done"] is True # unchangeddef test_mark_done_missing(populated_file): assert todo.mark_done(99, populated_file) is None assert len(todo.load_todos(populated_file)) == 2def test_delete_todo(populated_file): removed = todo.delete_todo(1, populated_file) assert removed == {"id": 1, "description": "buy milk", "done": False} assert todo.load_todos(populated_file) == [ {"id": 2, "description": "walk dog", "done": True} ]def test_delete_missing(populated_file): assert todo.delete_todo(99, populated_file) is None assert len(todo.load_todos(populated_file)) == 2Crush pytest 复验输出============================= test session starts ==============================platform linux -- Python 3.13.5, pytest-9.1.1, pluggy-1.6.0 -- /tmp/bench-venv/bin/pythoncachedir: .pytest_cacherootdir: /home/agent/coding-benchmark/test-crushcollecting ... collected 13 itemstest_todo.py::test_load_missing_file PASSED [ 7%]test_todo.py::test_load_invalid_json PASSED [ 15%]test_todo.py::test_load_non_list PASSED [ 23%]test_todo.py::test_save_and_load_roundtrip PASSED [ 30%]test_todo.py::test_next_id_empty PASSED [ 38%]test_todo.py::test_next_id_continues PASSED [ 46%]test_todo.py::test_add_todo PASSED [ 53%]test_todo.py::test_add_todo_increments_ids PASSED [ 61%]test_todo.py::test_list_todos PASSED [ 69%]test_todo.py::test_mark_done PASSED [ 76%]test_todo.py::test_mark_done_missing PASSED [ 84%]test_todo.py::test_delete_todo PASSED [ 92%]test_todo.py::test_delete_missing PASSED [100%]============================== 13 passed in 0.07s ==============================Goose v1.45.0 — 17/17Goose 生成的 todo.py (3972 bytes)#!/usr/bin/env python3"""A simple CLI todo app with JSON persistence.Commands: add <title> Add a new todo list List all todos done <id> Mark a todo as done delete <id> Delete a todo"""import argparseimport jsonimport sysfrom pathlib import PathDEFAULT_DATA_FILE = Path(__file__).resolve().parent / "todos.json"def load_todos(path: Path = DEFAULT_DATA_FILE) -> list[dict]: """Load todos from a JSON file. Returns an empty list if missing/invalid.""" if not path.exists(): return [] try: with open(path, "r", encoding="utf-8") as f: data = json.load(f) except (json.JSONDecodeError, OSError): return [] if not isinstance(data, list): return [] return datadef save_todos(todos: list[dict], path: Path = DEFAULT_DATA_FILE) -> None: """Save todos to a JSON file.""" with open(path, "w", encoding="utf-8") as f: json.dump(todos, f, indent=2)def add_todo(title: str, todos: list[dict]) -> dict: """Add a new todo to the list and return it.""" todo = { "id": max((t.get("id", 0) for t in todos if isinstance(t, dict)), default=0) + 1, "title": title, "done": False, } todos.append(todo) return tododef list_todos(todos: list[dict]) -> list[dict]: """Return todos sorted with incomplete items first, newest first.""" return sorted( todos, key=lambda t: (t.get("done", False), -t.get("id", 0)), )def mark_done(todo_id: int, todos: list[dict]) -> dict | None: """Mark a todo as done. Returns the todo, or None if not found.""" for todo in todos: if todo.get("id") == todo_id: todo["done"] = True return todo return Nonedef delete_todo(todo_id: int, todos: list[dict]) -> dict | None: """Delete a todo. Returns the deleted todo, or None if not found.""" for i, todo in enumerate(todos): if todo.get("id") == todo_id: return todos.pop(i) return Nonedef main(argv: list[str] | None = None, data_file: Path | None = None) -> int: """Run the CLI. Returns a process exit code (0 = success, 1 = not found).""" parser = argparse.ArgumentParser(description="Simple CLI todo app") subparsers = parser.add_subparsers(dest="command", required=True) add_parser = subparsers.add_parser("add", help="Add a new todo") add_parser.add_argument("title", help="Title of the todo") subparsers.add_parser("list", help="List all todos") done_parser = subparsers.add_parser("done", help="Mark a todo as done") done_parser.add_argument("id", type=int, help="ID of the todo") delete_parser = subparsers.add_parser("delete", help="Delete a todo") delete_parser.add_argument("id", type=int, help="ID of the todo") args = parser.parse_args(argv) path = data_file or DEFAULT_DATA_FILE todos = load_todos(path) if args.command == "add": todo = add_todo(args.title, todos) save_todos(todos, path) print(f"Added todo {todo['id']}: {todo['title']}") elif args.command == "list": if not todos: print("No todos yet.") else: for todo in list_todos(todos): status = "[x]" if todo.get("done") else "[ ]" print(f"{status}{todo.get('id')}: {todo.get('title')}") elif args.command == "done": todo = mark_done(args.id, todos) if todo is None: print(f"No todo with id {args.id}") return 1 save_todos(todos, path) print(f"Marked todo {todo['id']} as done: {todo['title']}") elif args.command == "delete": todo = delete_todo(args.id, todos) if todo is None: print(f"No todo with id {args.id}") return 1 save_todos(todos, path) print(f"Deleted todo {todo['id']}: {todo['title']}") return 0if __name__ == "__main__": sys.exit(main())Goose 生成的 test_todo.py (4285 bytes)"""Unit tests for the todo CLI app."""import pytestfrom todo import ( add_todo, delete_todo, list_todos, load_todos, main, mark_done, save_todos,)# --- persistence ----------------------------------------------------------def test_load_todos_missing_file_returns_empty(tmp_path): path = tmp_path / "todos.json" assert load_todos(path) == []def test_save_and_load_roundtrip(tmp_path): path = tmp_path / "todos.json" todos = [{"id": 1, "title": "buy milk", "done": False}] save_todos(todos, path) assert load_todos(path) == todosdef test_load_invalid_json_returns_empty(tmp_path): path = tmp_path / "todos.json" path.write_text("not valid json {") assert load_todos(path) == []def test_load_non_list_json_returns_empty(tmp_path): path = tmp_path / "todos.json" path.write_text('{"foo": "bar"}') assert load_todos(path) == []# --- model operations -----------------------------------------------------def test_add_todo_assigns_incrementing_ids(): todos = [] first = add_todo("first", todos) second = add_todo("second", todos) assert first["id"] == 1 assert second["id"] == 2 assert first["done"] is False assert second["title"] == "second" assert [t["title"] for t in todos] == ["first", "second"]def test_add_todo_continues_ids_after_existing(): todos = [{"id": 5, "title": "existing", "done": True}] new = add_todo("new", todos) assert new["id"] == 6def test_mark_done_flags_todo(): todos = [{"id": 1, "title": "a", "done": False}] result = mark_done(1, todos) assert result == todos[0] assert todos[0]["done"] is Truedef test_mark_done_missing_returns_none(): assert mark_done(99, [{"id": 1, "title": "a", "done": False}]) is Nonedef test_delete_todo_removes_it(): todos = [ {"id": 1, "title": "a", "done": False}, {"id": 2, "title": "b", "done": False}, ] deleted = delete_todo(1, todos) assert deleted["title"] == "a" assert todos == [{"id": 2, "title": "b", "done": False}]def test_delete_missing_returns_none(): todos = [{"id": 1, "title": "a", "done": False}] assert delete_todo(99, todos) is None assert len(todos) == 1def test_list_todos_sorts_incomplete_first_newest_first(): todos = [ {"id": 1, "title": "oldest done", "done": True}, {"id": 2, "title": "older open", "done": False}, {"id": 3, "title": "newest open", "done": False}, ] listed = list_todos(todos) # Incomplete todos first, then newest id first within each group. assert [t["title"] for t in listed] == ["newest open", "older open", "oldest done"]# --- CLI end-to-end -------------------------------------------------------def test_cli_add_writes_json_and_prints(tmp_path, capsys): path = tmp_path / "todos.json" assert main(["add", "buy milk"], data_file=path) == 0 assert load_todos(path) == [{"id": 1, "title": "buy milk", "done": False}] assert "buy milk" in capsys.readouterr().outdef test_cli_list_empty(tmp_path, capsys): path = tmp_path / "todos.json" assert main(["list"], data_file=path) == 0 assert "No todos yet" in capsys.readouterr().outdef test_cli_done_and_delete_flow(tmp_path): path = tmp_path / "todos.json" assert main(["add", "task one"], data_file=path) == 0 assert main(["add", "task two"], data_file=path) == 0 assert main(["done", "1"], data_file=path) == 0 todos = load_todos(path) assert todos[0]["done"] is True assert main(["delete", "1"], data_file=path) == 0 remaining = load_todos(path) assert [t["id"] for t in remaining] == [2]def test_cli_done_missing_returns_nonzero(tmp_path, capsys): path = tmp_path / "todos.json" assert main(["done", "42"], data_file=path) == 1 assert "No todo with id 42" in capsys.readouterr().outdef test_cli_delete_missing_returns_nonzero(tmp_path, capsys): path = tmp_path / "todos.json" assert main(["delete", "42"], data_file=path) == 1 assert "No todo with id 42" in capsys.readouterr().outdef test_cli_invalid_command_raises_system_exit(tmp_path): path = tmp_path / "todos.json" with pytest.raises(SystemExit): main(["bogus"], data_file=path)Goose pytest 复验输出g-benchmark/test-goosecollecting ... collected 17 itemstest_todo.py::test_load_todos_missing_file_returns_empty PASSED [ 5%]test_todo.py::test_save_and_load_roundtrip PASSED [ 11%]test_todo.py::test_load_invalid_json_returns_empty PASSED [ 17%]test_todo.py::test_load_non_list_json_returns_empty PASSED [ 23%]test_todo.py::test_add_todo_assigns_incrementing_ids PASSED [ 29%]test_todo.py::test_add_todo_continues_ids_after_existing PASSED [ 35%]test_todo.py::test_mark_done_flags_todo PASSED [ 41%]test_todo.py::test_mark_done_missing_returns_none PASSED [ 47%]test_todo.py::test_delete_todo_removes_it PASSED [ 52%]test_todo.py::test_delete_missing_returns_none PASSED [ 58%]test_todo.py::test_list_todos_sorts_incomplete_first_newest_first PASSED [ 64%]test_todo.py::test_cli_add_writes_json_and_prints PASSED [ 70%]test_todo.py::test_cli_list_empty PASSED [ 76%]test_todo.py::test_cli_done_and_delete_flow PASSED [ 82%]test_todo.py::test_cli_done_missing_returns_nonzero PASSED [ 88%]test_todo.py::test_cli_delete_missing_returns_nonzero PASSED [ 94%]test_todo.py::test_cli_invalid_command_raises_system_exit PASSED [100%]============================== 17 passed in 0.10s ============================== 测试：13/13 通过（glm_for_coding 下有个 DEFAULT_FILE 绑定 bug，换 deepseek 后自动消失） 速度快（66s），token 省（14.5K input） Charm 出品，UI 好看，但实用性一般 测试：17/17 通过，代码质量好（126 行 todo.py 结构清晰，含损坏文件容错、时间戳、自动递增 ID） 自带任务管理（todo_write 工具），agent 会自己记账 Linux Foundation 项目，48K stars，潜力大 测试：13/13 通过，还做了完整 smoke test（add→done→list→delete 端到端） - 重要历史
：glm_for_coding 下失败 4 次（heredoc 被 chr(39) 转义搞坏、代码测试不一致、连接卡死）——是模型问题不是 Codex 问题 - 隐患
：deepseek 是推理模型，Codex 多轮对话时触发 reasoning_content 必须回传400 报错——复杂多轮任务会随机挂，只适合单轮/简单任务 Terminal-Bench 2.1 排名第一（83.4%）的工具，能力本身最强 Token 说明：本地 session 不记录 usage；曾尝试"账单前后差值法"但网关为共享 key（后台有 MRS refactor 等任务并发消耗），差值 0.027 测试：22/22 通过 亮点：自己发现并修复了 mktemp 空文件导致 json.load 崩溃的 bug（加了空文件容错 + 2 个对应测试）；用 next_id计数器保证删除后 id 不复用系统提示词最完善，工具定义最全——这也是它 token 大的原因 生态最大（134K stars）、文档全、最成熟 测试：12/12 通过 自动发现项目 .venv里的 pytest 9.1.1 并直接使用安装难点：GitHub 下载慢需镜像（ghfast.top），但配置极简 Token 数据：本地 SQLite（ ~/.local/share/opencode/opencode.dbsession 表）可精确查询- 正确配置文件是
~/.config/crush/crush.json（不是 config.json！改了 config.json 完全不生效） 自定义 provider 的 models必须是数组（不是对象，否则报 model not found）crush update-providers会覆盖 providers.json 手动修改（远程拉取 40 个 provider） 运行 flag 是 -m provider/model（如-m myagentlab/deepseek-v4-flash）- 必须用
OPENAI_API_KEY/OPENAI_BASE_URL（不是 GOOSE_API_KEY/GOOSE_BASE_URL！之前一直设错连默认 api.openai.com 当然超时） 流式模式会报 Stream decode error——必须在 declarative provider 里设"supports_streaming": false自定义 provider 要放 ~/.config/goose/custom_providers/<name>.json
