feat: 新增 px.sh 命令执行封装 — 支持 list[str](直接执行)/ str(shell=True),统一中文错误处理,失败返回 None 不抛异常;sglang.py 首个迁移示例
CI / Lint, Typecheck & Test (push) Failing after 2m8s

This commit is contained in:
2026-07-08 14:56:54 +08:00
parent fbabbc98ad
commit ae4cc3aeeb
6 changed files with 316 additions and 15 deletions
+70
View File
@@ -0,0 +1,70 @@
# iter-28px.sh 命令执行封装
## 本轮目标
ops/ 目录下 57 处 `subprocess.run` 调用存在重复的 `try/except CalledProcessError +
FileNotFoundError + print` 模式。封装一个轻量的 `px.sh` 辅助函数,统一错误处理
与中文输出,支持 `list[str]`(直接执行)和 `str``shell=True`)两种输入。
## 改动文件清单
| 文件 | 变更 |
|------|------|
| `src/pyflowx/shell.py` | 新建模块,实现 `sh()` 函数 |
| `src/pyflowx/__init__.py` | 导出 `sh`,更新 docstring |
| `tests/test_shell.py` | 新建,16 个测试(list/str/mocked/exported |
| `src/pyflowx/ops/infra/sglang.py` | 重构为使用 `px.sh`,移除 try/except 样板 |
| `tests/cli/test_llm.py` | 适配 sglang.py 重构后的错误信息格式 |
## 关键决策与依据
### 命名与定位
- **`px.sh` 而非 `px.run`**`px.run` 已被占用(执行 Graph DAG),`sh` 简短易记,
语义清晰("执行 shell 命令")。
- **轻量封装,不替代所有 `subprocess.run`**:需要 `capture_output` + 手动检查
返回码等精细控制的场景(如 `lscalc.py`/`piptool.py`/`which.py`)保留原样。
`px.sh` 定位为"简单执行 + 报错"场景的快捷方式。
### 输入类型
- **`list[str]``shell=False`**:直接执行,安全无注入风险。
- **`str``shell=True`**:支持管道/重定向等 shell 语法;ops 工具场景下命令可信。
- **不支持 `func`**ops 工具本身就是 Python 函数,直接 `func()` 调用即可,
封装反而增加复杂度。
### 返回值设计
- **成功返回 `CompletedProcess[str]`**`text=True` 始终启用,输出为字符串。
- **失败返回 `None` + 打印中文错误**:不抛异常,调用方用 `result is None` 判断。
统一 `CalledProcessError`(打印 returncode)和 `FileNotFoundError`(打印"命令未找到")。
- **`capture` 参数**`False`(默认)输出到终端;`True` 捕获 stdout/stderr。
### sglang.py 迁移示例
重构前(12 行 try/except 样板):
```python
try:
subprocess.run(["uv", "install", "sglang[all]"], check=True)
except subprocess.CalledProcessError as e:
print(f"安装失败 (returncode={e.returncode}): {e}")
except FileNotFoundError:
print("uv 命令未找到,请确认 uv 已安装并在 PATH 中")
```
重构后(1 行):
```python
px.sh(["uv", "install", "sglang[all]"], label="安装 sglang")
```
## 验证结果
- pytest1721 passed(新增 16 个 shell 测试)
- coverage97.20%shell.py 100%、sglang.py 100%
- ruffAll checks passed
- pyrefly0 errors
## 遗留事项
- 其余 ~55 处 `subprocess.run` 可在后续迭代中按需迁移到 `px.sh`
优先迁移"简单执行 + 报错"模式的调用;`capture_output` 等精细控制场景保留原样。
+3
View File
@@ -16,6 +16,7 @@
* :func:`compose` —— 编程式组合多图。
* :func:`switch` / :func:`branch` —— 条件分支 DAG 构造器。
* :func:`task_template` —— 批量生成相似 TaskSpec 的工厂。
* :func:`sh` —— Shell 命令执行辅助(支持 ``list[str]`` / ``str``,统一中文错误处理)。
* 状态后端::class:`StateBackend`、:class:`MemoryBackend`、:class:`JSONBackend`、:class:`SQLiteBackend`。
快速上手
@@ -92,6 +93,7 @@ from .profiling import ProfileReport, TaskProfile
from .progress import ProgressCallback, RichProgressMonitor
from .report import RunReport
from .runner import CliExitCode, CliRunner
from .shell import sh
from .storage import JSONBackend, MemoryBackend, SQLiteBackend, StateBackend
from .task import (
CacheKeyFn,
@@ -215,6 +217,7 @@ __all__ = [
"run_command",
"run_iter",
"run_tool",
"sh",
"start_metrics_server",
"switch",
"task",
+2 -13
View File
@@ -6,7 +6,6 @@
from __future__ import annotations
import shutil
import subprocess
from pathlib import Path
import pyflowx as px
@@ -29,12 +28,7 @@ def install_sglang() -> None:
return
print("正在安装 sglang[all]...")
try:
subprocess.run(["uv", "install", "sglang[all]"], check=True)
except subprocess.CalledProcessError as e:
print(f"安装失败 (returncode={e.returncode}): {e}")
except FileNotFoundError:
print("uv 命令未找到,请确认 uv 已安装并在 PATH 中")
px.sh(["uv", "install", "sglang[all]"], label="安装 sglang")
@px.tool("sglang", subcommand="run", help="启动 SGLang 服务", needs=["install"])
@@ -89,9 +83,4 @@ def run_sglang(
log_level,
]
print(f"启动 SGLang: {model_dir} (port={port}, ctx={ctx_len}, mem={mem_fraction})")
try:
subprocess.run(cmd, check=True)
except subprocess.CalledProcessError as e:
print(f"SGLang 启动失败 (returncode={e.returncode}): {e}")
except FileNotFoundError:
print(f"{python_bin} 命令未找到,请确认 Python 已安装并在 PATH 中")
px.sh(cmd, label="SGLang 启动")
+75
View File
@@ -0,0 +1,75 @@
"""Shell 命令执行辅助。
提供 :func:`sh` —— 对 :func:`subprocess.run` 的轻量封装,统一错误处理
与中文输出,支持 ``list[str]``(直接执行)和 ``str````shell=True``)两种输入。
典型用法::
import pyflowx as px
px.sh(["git", "add", "."]) # list[str],直接执行
px.sh("git add . | tee log.txt") # strshell=True(支持管道/重定向)
result = px.sh("echo hello", capture=True)
if result is not None:
print(result.stdout.strip())
设计定位:简单"执行 + 报错"场景的快捷方式。需要 ``capture_output`` +
手动检查返回码等精细控制时,直接使用 :func:`subprocess.run`。
"""
from __future__ import annotations
import subprocess
__all__ = ["sh"]
def sh(
cmd: list[str] | str,
*,
capture: bool = False,
label: str = "命令",
) -> subprocess.CompletedProcess[str] | None:
"""执行 shell 命令,统一错误处理与中文输出。
``list[str]`` 直接执行;``str`` 走 ``shell=True``(支持管道/重定向等语法)。
成功返回 :class:`subprocess.CompletedProcess`;失败时打印中文错误信息
并返回 ``None``(不抛异常),调用方用 ``result is None`` 判断是否失败。
Parameters
----------
cmd : list[str] | str
命令列表(直接执行)或 shell 字符串(``shell=True`` 执行)
capture : bool
为 ``True`` 时捕获 stdout/stderr(不直接输出到终端),
可从返回值的 ``.stdout`` / ``.stderr`` 读取(默认 ``False``
label : str
命令标签,用于错误信息中标识命令用途(默认 ``"命令"``
Returns
-------
subprocess.CompletedProcess[str] | None
成功时返回执行结果;失败时返回 ``None``
Examples
--------
>>> px.sh(["ls", "-la"]) # 列出当前目录
>>> px.sh("cat file.txt | grep pattern") # shell 管道
>>> result = px.sh("pip --version", capture=True, label="pip")
>>> if result is not None:
... print(result.stdout)
"""
try:
return subprocess.run(
cmd,
shell=isinstance(cmd, str),
capture_output=capture,
text=True,
check=True,
)
except subprocess.CalledProcessError as e:
print(f"{label}失败 (returncode={e.returncode}): {e}")
return None
except FileNotFoundError:
print(f"{label}失败: 命令未找到,请确认已安装并在 PATH 中")
return None
+2 -2
View File
@@ -130,7 +130,7 @@ class TestInstallSglang:
monkeypatch.setattr(subprocess, "run", lambda *_, **__: (_ for _ in ()).throw(err))
install_sglang()
captured = capsys.readouterr()
assert "安装失败" in captured.out
assert "安装 sglang失败" in captured.out
assert "returncode=1" in captured.out
def test_install_file_not_found_prints_no_uv(
@@ -141,7 +141,7 @@ class TestInstallSglang:
monkeypatch.setattr(subprocess, "run", lambda *_, **__: (_ for _ in ()).throw(FileNotFoundError()))
install_sglang()
captured = capsys.readouterr()
assert "uv 命令未找到" in captured.out
assert "命令未找到" in captured.out
# ---------------------------------------------------------------------- #
+164
View File
@@ -0,0 +1,164 @@
"""px.sh 命令执行封装的测试。"""
from __future__ import annotations
import subprocess
from typing import Any
import pyflowx as px
from pyflowx.shell import sh
class TestShListInput:
"""list[str] 输入(直接执行,非 shell)。"""
def test_sh_list_success(self) -> None:
"""list[str] 命令成功执行,返回 CompletedProcess。"""
result = sh(["true"])
assert result is not None
assert result.returncode == 0
def test_sh_list_success_with_capture(self) -> None:
"""capture=True 时捕获 stdout。"""
result = sh(["echo", "hello"], capture=True, label="echo")
assert result is not None
assert result.stdout == "hello\n"
def test_sh_list_no_capture_stdout_is_none(self) -> None:
"""capture=False 时 stdout 为 None(输出到终端)。"""
result = sh(["true"])
assert result is not None
assert result.stdout is None
def test_sh_list_failure_returns_none(self, capsys: Any) -> None:
"""list[str] 命令失败(非零返回码)返回 None + 打印中文错误。"""
result = sh(["false"], label="测试命令")
assert result is None
captured = capsys.readouterr()
assert "测试命令失败" in captured.out
assert "returncode=" in captured.out
def test_sh_list_file_not_found_returns_none(self, capsys: Any) -> None:
"""list[str] 命令不存在返回 None + 打印中文错误。"""
result = sh(["this_command_does_not_exist_xyz"], label="缺失命令")
assert result is None
captured = capsys.readouterr()
assert "缺失命令失败" in captured.out
assert "命令未找到" in captured.out
class TestShStrInput:
"""str 输入(shell=True)。"""
def test_sh_str_success(self) -> None:
"""str 命令成功执行(shell=True)。"""
result = sh("true")
assert result is not None
assert result.returncode == 0
def test_sh_str_with_pipe(self) -> None:
"""str 支持 shell 管道语法。"""
result = sh("echo hello | tr a-z A-Z", capture=True, label="管道")
assert result is not None
assert result.stdout == "HELLO\n"
def test_sh_str_with_redirect(self, tmp_path: Any) -> None:
"""str 支持 shell 重定向语法。"""
out_file = tmp_path / "out.txt"
sh(f"echo content > {out_file}")
assert out_file.read_text() == "content\n"
def test_sh_str_failure_returns_none(self, capsys: Any) -> None:
"""str 命令失败返回 None + 打印中文错误。"""
result = sh("false", label="shell 命令")
assert result is None
captured = capsys.readouterr()
assert "shell 命令失败" in captured.out
def test_sh_str_capture_stderr(self) -> None:
"""capture=True 时捕获 stderr。"""
result = sh("echo err >&2", capture=True, label="stderr")
assert result is not None
assert result.stderr == "err\n"
class TestShMocked:
"""用 monkeypatch mock subprocess.run 验证参数传递。"""
def test_sh_list_passes_shell_false(self, monkeypatch: Any) -> None:
"""list[str] 输入时 shell=False。"""
captured: dict[str, Any] = {}
def fake_run(cmd: Any, **kwargs: Any) -> subprocess.CompletedProcess[str]:
captured["cmd"] = cmd
captured["shell"] = kwargs.get("shell")
captured["capture_output"] = kwargs.get("capture_output")
captured["text"] = kwargs.get("text")
captured["check"] = kwargs.get("check")
return subprocess.CompletedProcess(cmd, 0)
monkeypatch.setattr(subprocess, "run", fake_run)
sh(["ls", "-la"])
assert captured["cmd"] == ["ls", "-la"]
assert captured["shell"] is False
assert captured["capture_output"] is False
assert captured["text"] is True
assert captured["check"] is True
def test_sh_str_passes_shell_true(self, monkeypatch: Any) -> None:
"""str 输入时 shell=True。"""
captured: dict[str, Any] = {}
def fake_run(cmd: Any, **kwargs: Any) -> subprocess.CompletedProcess[str]:
captured["cmd"] = cmd
captured["shell"] = kwargs.get("shell")
return subprocess.CompletedProcess(cmd, 0)
monkeypatch.setattr(subprocess, "run", fake_run)
sh("ls -la | grep foo")
assert captured["cmd"] == "ls -la | grep foo"
assert captured["shell"] is True
def test_sh_capture_passes_capture_output_true(self, monkeypatch: Any) -> None:
"""capture=True 时 capture_output=True。"""
captured: dict[str, Any] = {}
def fake_run(cmd: Any, **kwargs: Any) -> subprocess.CompletedProcess[str]:
captured["capture_output"] = kwargs.get("capture_output")
return subprocess.CompletedProcess(cmd, 0, stdout="", stderr="")
monkeypatch.setattr(subprocess, "run", fake_run)
sh(["echo"], capture=True)
assert captured["capture_output"] is True
def test_sh_called_process_error_returns_none(self, monkeypatch: Any, capsys: Any) -> None:
"""CalledProcessError 被捕获,打印中文错误并返回 None。"""
def fake_run(cmd: Any, **kwargs: Any) -> subprocess.CompletedProcess[str]:
raise subprocess.CalledProcessError(returncode=1, cmd=cmd)
monkeypatch.setattr(subprocess, "run", fake_run)
result = sh(["failing"], label="自定义标签")
assert result is None
captured = capsys.readouterr()
assert "自定义标签失败" in captured.out
assert "returncode=1" in captured.out
def test_sh_file_not_found_error_returns_none(self, monkeypatch: Any, capsys: Any) -> None:
"""FileNotFoundError 被捕获,打印中文错误并返回 None。"""
def fake_run(cmd: Any, **kwargs: Any) -> subprocess.CompletedProcess[str]:
raise FileNotFoundError("[Errno 2] No such file")
monkeypatch.setattr(subprocess, "run", fake_run)
result = sh(["missing"], label="缺失工具")
assert result is None
captured = capsys.readouterr()
assert "缺失工具失败" in captured.out
assert "命令未找到" in captured.out
class TestShExported:
"""验证 px.sh 顶层导出。"""
def test_sh_is_exported_from_px(self) -> None:
"""px.sh 等价于 pyflowx.shell.sh。"""
assert px.sh is sh