Compare commits

...

45 Commits

Author SHA1 Message Date
zhou 08149d990d bump version to 0.4.8
CI / Lint, Typecheck & Test (push) Failing after 1m49s
Release / Build, Publish & Release (push) Successful in 46s
2026-07-06 13:19:24 +08:00
zhou 54aef98f3a chore(gittool): 新增ruff缓存目录到排除列表
CI / Lint, Typecheck & Test (push) Failing after 1m52s
将.ruff_cache添加到EXCLUDE_DIRS列表中,避免git工具扫描该缓存目录
2026-07-06 13:11:59 +08:00
zhou e31646e281 feat: 新增envdev、gittool、msdownload、pymake工具模块及对应测试用例
新增四个业务工具模块:
1. envdev: 开发环境镜像源配置工具,支持Python/Conda/Rust镜像配置及Linux系统环境配置
2. gittool: Git操作工具,提供提交、初始化、清理、推送等快捷子命令
3. msdownload: ModelScope模型/数据集下载工具
4. pymake: 项目构建工具,覆盖构建、测试、发布等全流程开发操作

同时为每个工具模块编写了完整的注册验证与功能测试用例
2026-07-06 13:11:09 +08:00
zhou f7fb95af83 refactor: 重构项目架构,移除YAML编排与旧注册系统
1.  将所有YAML配置的工具迁移到`@px.tool`装饰器实现
2.  拆分`llm`模块为`msdownload`和`sglang`子模块
3.  移除废弃的`registry.py`、`yaml_loader.py`和`yamlrun.py`模块
4.  清理项目依赖,移除PyYAML相关包
5.  更新文档与测试用例适配新架构
2026-07-06 13:04:57 +08:00
zhou 6da42ec5ff refactor: 移除YAML配置,统一使用@px.tool装饰器实现CLI工具
1.  新增tools.py模块实现@px.tool装饰器与工具注册表
2.  将所有configs下的YAML工具迁移为ops/xxx.py模块形式
3.  重构cli路由逻辑,优先加载Python工具实现回退YAML
4.  删除所有YAML配置文件与旧的yaml_loader相关代码
5.  调整__init__.py导出API,移除YAML相关依赖
2026-07-06 07:53:22 +08:00
zhou c55a37173a test: 新增 YamlCliRunner 错误分支测试使覆盖率达标 95%
CI / Lint, Typecheck & Test (push) Successful in 3m5s
2026-07-05 22:25:15 +08:00
zhou 960b8672f4 test: 新增截图、PDF工具相关的单元测试
1. 新增Linux平台下截图工具回退到scrot的测试用例
2. 新增pdf加密、解密、重排、PDF转图片的完整测试用例
3. 补充各PDF工具未安装依赖时的分支测试
2026-07-05 22:21:47 +08:00
zhou 4fd1d70b58 test: 修复跨平台路径断言不兼容问题
1. 调整测试用例中的路径比较逻辑,统一替换反斜杠为正斜杠适配Windows平台
2. 将原生echo/true/false命令替换为跨平台的python执行命令,避免环境依赖问题
2026-07-05 22:18:16 +08:00
zhou 6fb9223066 fix: pymake bump 调用 pf bumpversion 而非已移除的外部 bumpversion 命令
CI / Lint, Typecheck & Test (push) Successful in 1m50s
2026-07-05 22:09:28 +08:00
zhou 1f7127357e refactor: 将 cli/system、cli/llm、cli/dev 脚本迁移为 YAML 配置 + register_fn 模式
CI / Lint, Typecheck & Test (push) Successful in 1m44s
删除 cli/system (clearscreen/taskkill/which)、cli/llm (msdownload/sglang)、cli/dev (dockercmd/envdev) 三个目录; 新增 7 个 YAML 配置; 新增 ops/llm.py 模块; 扩展 ops/system.py 和 ops/dev.py; pf.py 添加工具别名; 删除 examples/ 目录
2026-07-05 21:12:41 +08:00
zhou 58ee84ded6 bump version to 0.4.7
CI / Lint, Typecheck & Test (push) Successful in 2m2s
Release / Build, Publish & Release (push) Successful in 1m5s
2026-07-05 19:49:54 +08:00
zhou 9a96e5d052 fix: 修复 bump_project_version 版本号不同步跳号 bug 并抽离到独立模块
CI / Lint, Typecheck & Test (push) Successful in 1m26s
原实现对每个文件独立 +1, 文件版本号不同步时跳号; 改为先读取所有文件取 max 作为基准再统一写入. 同时修复 git add . 违规 (改按文件名) 与 check=False 吞错误. bumpversion 从 ops/dev.py 抽离到 ops/bumpversion.py, 测试简化为 16 个核心场景.
2026-07-05 19:34:44 +08:00
zhou c9c7529c58 bump version to 0.4.6
CI / Lint, Typecheck & Test (push) Successful in 2m4s
Release / Build, Publish & Release (push) Successful in 46s
2026-07-05 19:09:10 +08:00
zhou c498d9b1c9 feat: 实现方向 B 聚合 job 并消除 pymake/reseticoncache CLI 入口
CI / Lint, Typecheck & Test (push) Successful in 2m3s
放宽 yaml_loader 校验允许有 needs 无 cmd/fn 的聚合 job, 完善 pymake.yaml 覆盖原 pymake.py 所有别名, 新增 reset_icon_cache_run fn 与 reseticoncache.yaml, 删除 pymake.py/reseticoncache.py 及对应 scripts 入口, 修复 --list 在 subcommands 模式下的可达性 bug.
2026-07-05 18:12:59 +08:00
zhou b36e279f92 feat(cli): add pymake project build tool support
add pymake command alias to CLI, create pymake config yaml and move its legacy tool config out of _LEGACY_TOOLS dict
2026-07-05 17:45:08 +08:00
zhou 58d6f1faad refactor: 迁移 cli/_ops/ 到 ops/, 按类别保持 dev/files/media/system 分类
CI / Lint, Typecheck & Test (push) Successful in 1m20s
将 src/pyflowx/cli/_ops/ 整体迁移至 src/pyflowx/ops/, 与 cli/ 平级
(工具函数非 CLI 专属, 可被 YAML 任务编排通用引用). 分类保持不变:
dev (git/pip/bump/autofmt), files (date/level/back/zip),
media (pdf/screenshot), system (ls/pack/ssh).

同步更新 15 个引用文件 (yaml_loader + 14 个测试) 的 import 路径,
README 模块结构表与 test_registry docstring.
2026-07-05 17:32:22 +08:00
zhou d93da0d8b4 refactor: 迁移 cli/_ops/ 到 ops/, 按类别保持 dev/files/media/system 分类
CI / Lint, Typecheck & Test (push) Successful in 1m35s
将 src/pyflowx/cli/_ops/ 整体迁移至 src/pyflowx/ops/, 与 cli/ 平级
(工具函数非 CLI 专属, 可被 YAML 任务编排通用引用). 分类保持不变:
dev (git/pip/bump/autofmt), files (date/level/back/zip),
media (pdf/screenshot), system (ls/pack/ssh).

同步更新 15 个引用文件 (yaml_loader + 14 个测试) 的 import 路径,
README 模块结构表与 test_registry docstring.
2026-07-05 17:30:35 +08:00
zhou 701c455c42 refactor: 用 YamlCliRunner/PfApp class 封装 CLI 入口逻辑
CI / Lint & Typecheck (push) Failing after 11m31s
CI / Test (Python 3.11) (push) Failing after 6m1s
CI / Test (Python 3.13) (push) Failing after 12m44s
CI / Test (Python 3.8) (push) Failing after 12m2s
CI / Docs Build (push) Failing after 6m56s
yaml_loader.py 新增 YamlCliRunner class, 将 run_cli 的 90 行
流程拆为 run/_load_config/_add_global_options/_extract_variables/
_handle_list 等单一职责方法; run_cli 保留为薄包装 (公共 API).
pf.py 新增 PfApp class, 将 main 的路由逻辑拆为 run/_list_tools/
_resolve_tool/_run_legacy/_run_yaml 等方法; main 仅调用
PfApp().run().

附带修复 run_yaml 的 jobs=None 误报 bug: 原代码 jobs is None
时 raise ValueError, 但 None 本意是执行全部任务, 导致
pf folderzip 报 "jobs 不能为空".
2026-07-05 15:59:05 +08:00
zhou e174b64495 refactor(cli): 重构配置文件路径并新增多工具配置
1. 移除旧的cli/configs/__init__.py占位文件
2. 修正pf.py中配置目录的路径指向
3. 新增folderzip、bumpversion等十余种工具的配置文件
4. 更新uv.lock中的依赖版本匹配规则
2026-07-05 15:50:18 +08:00
zhou 3afb25bb5e fix: 修正 typing-extensions 依赖条件为 python_version < '3.13'
CI / Lint & Typecheck (push) Failing after 15m3s
CI / Test (Python 3.11) (push) Failing after 24m15s
CI / Test (Python 3.13) (push) Failing after 27m17s
CI / Test (Python 3.8) (push) Failing after 5m56s
CI / Docs Build (push) Failing after 31s
task.py 在 Python < 3.13 时需要 typing_extensions 的 TypeVar
(PEP 696 default= 参数), 此前条件 < '3.10' 导致 3.10-3.12
环境 import 失败, ReadTheDocs (Python 3.11) 构建报
ModuleNotFoundError: No module named 'typing_extensions'.
2026-07-05 13:16:24 +08:00
zhou fbd17536fd ci: 重写 CI/Release 为 GitHub 兼容版本并加文档构建
CI / Lint & Typecheck (push) Failing after 8m55s
CI / Test (Python 3.11) (push) Failing after 31s
CI / Test (Python 3.13) (push) Failing after 31s
CI / Test (Python 3.8) (push) Failing after 31s
CI / Docs Build (push) Failing after 31s
ci.yml 改用标准 actions (checkout/setup-uv/setup-python), 新增
pyrefly 类型检查、coverage 阈值检查 (>=95%)、Sphinx 文档构建三个
job, 多版本矩阵测试 (py38/py311/py313)。release.yml 改用标准
actions, 发布到 PyPI + GitHub Release (替代原 Gitea Release)。
2026-07-05 12:32:00 +08:00
zhou 32ca8c1208 docs: 搭建 Sphinx 文档站并清理死代码
CI / Lint, Typecheck & Test (push) Successful in 1m57s
1. 新建 docs/ Sphinx 文档结构 (conf.py + 8 个 rst 章节),
   napoleon 支持 Google/NumPy docstring, rtd 主题,
   自动生成 API 参考与错误家族文档
2. 新建 .readthedocs.yaml 配置, pyproject.toml 加 docs 依赖
3. 删除 runner.py 的 _apply_verbose_to_graph 死代码及对应测试
   (功能已移入 executors.run 统一处理)
4. 更新 README: CLI 示例改为 pf 统一入口, 模块结构表补全
   cli/pf.py/cli/configs/cli/_ops 等模块
5. 修复版本不一致 (pyproject.toml 0.3.5 → 0.4.5)
6. 加文档徽章链接到 ReadTheDocs
2026-07-05 12:17:10 +08:00
zhou a7ff68d279 feat: pf 默认显示 verbose 执行过程, --quiet 关闭
CI / Lint, Typecheck & Test (push) Successful in 1m17s
run() 在 verbose=True 时自动把 verbose 标记应用到所有 spec,
使 execute_command 打印执行命令与返回码 (此前只 callback 打印
任务生命周期)。全局选项 --verbose 改为 --quiet (默认 verbose=True,
传 --quiet 关闭)。gittool CLEAN_EXCLUDES 补全 .pytest_cache/
.ruff_cache/.vscode/.trae/.qoder/.editorconfig 等目录。
2026-07-05 08:46:15 +08:00
zhou de368ea810 refactor: 删除冗余 cli 入口脚本, gittool 用数组配置 clean excludes
CI / Lint, Typecheck & Test (push) Successful in 1m11s
1. 删除 13 个已有 YAML 配置的 cli .py 入口脚本, 统一通过 pf 调用
2. gittool.yaml 用 CLEAN_EXCLUDES 数组变量配置 git clean 的 -e 参数,
   保留 .venv/.tox/node_modules/.idea 等目录避免误删
3. run_cli 执行前打印调用信息: [gittool] 执行: c
4. 更新 pyproject.toml 移除 13 个冗余 entry points, 仅保留 pf
5. 清理测试文件中的 TestMain 类 (测 _ops 模块的测试保留)
2026-07-05 08:39:20 +08:00
zhou 6a3e3a57cd fix: cmd 任务成功时打印 stdout
CI / Lint, Typecheck & Test (push) Failing after 50s
execute_command 在非 verbose 模式下捕获 stdout 后直接 return None,
导致 git status --porcelain 等命令的输出被丢弃。
现在成功时若有 stdout 则打印到终端, 保留失败时的 stderr 信息。
2026-07-05 00:52:38 +08:00
zhou 7089944306 build: 调整pyproject.toml中pf命令的脚本位置
将pf命令的脚本配置移至文件末尾,修正脚本条目排序
2026-07-05 00:50:16 +08:00
zhou ec5e348694 feat: 新增 pf 统一入口, YAML 配置自带 CLI 参数定义
CI / Lint, Typecheck & Test (push) Failing after 47s
新增 pf 统一 CLI 入口, 通过 YAML 的 cli: 段定义参数解析规则,
逐步消除工具 .py 入口文件。yaml_loader 新增 build_cli_parser
和 run_cli 函数, 支持 subcommands/positional/options 三级 schema,
内置 --dry-run/--verbose/--strategy/--list 全局选项。
13 个工具 YAML 配置全部添加 cli: 段。
2026-07-04 20:31:40 +08:00
zhou 12d9f2f647 fix: 恢复 gittool 条件逻辑,修复 has_files 检查 git status
CI / Lint, Typecheck & Test (push) Successful in 1m56s
将 gitt a/i 命令改用 fn job 包装(git_add_commit/git_init_add_commit),
内部检查 has_files() 和 not_has_git_repo() 条件,避免无更改时 git commit
报错。修正 has_files() 实现为检查 git status --porcelain 而非目录文件。
2026-07-04 20:00:25 +08:00
zhou 6ffcbecade chore: update 2026-07-04 19:57:20 +08:00
zhou e76d93187b chore: update 2026-07-04 19:55:09 +08:00
zhou 52e20e3f93 style: 统一调整代码格式,将单行列表展开为多行缩进格式 2026-07-04 19:49:10 +08:00
zhou 3f966a230e refactor: 简化 CLI 工具入口为 YAML 加载器
CI / Lint, Typecheck & Test (push) Successful in 2m5s
将 13 个工具入口文件重构为通过 px.run_yaml 调用 YAML 配置,
辅助函数移至 _ops 模块。新增 run_yaml 便捷函数支持 job 选择
和传递依赖收集,修复 _build_cmd 列表变量展开,新增 bump_project_version
高层函数封装版本号更新+git 提交流程。
2026-07-04 19:35:08 +08:00
zhou 5d0b211a44 feat: 新增 13 个 CLI 工具的 YAML 配置并修复 _ops 函数注册
CI / Lint, Typecheck & Test (push) Successful in 1m41s
- 在 cli/configs/ 下创建 13 个 YAML 工作流配置, 覆盖 filedate/filelevel/folderback/
  folderzip/autofmt/bumpversion/piptool/gittool/pdftool/screenshot/lscalc/
  sshcopyid/packtool 工具, 共 51 个 job (cmd 与 fn 混合)
- yaml_loader 模块级导入 _ops 子模块, 使 YAML fn 字段可引用注册函数,
  try/except 守卫避免最小安装场景下的 ImportError
- 修复 test_registry 的 clear_registry fixture: 保存/恢复 _REGISTRY 原始状态,
  避免 teardown 清空 _ops 自动注册的函数导致 TestOpsModules 失败
2026-07-04 18:35:20 +08:00
zhou 6931f36fd1 feat: 新增函数注册机制与 CLI 工具函数模块
CI / Lint, Typecheck & Test (push) Successful in 2m27s
- 新增 registry.py 提供 register_fn/get_fn/has_fn 函数注册机制, 支持 @register_fn 和 @register_fn("name") 两种用法
- 新增 cli/_ops 包 (files/dev/media/system 四个子模块), 聚合 59 个可复用函数供 YAML fn 字段引用
- 扩展 yaml_loader 支持 fn 字段、args/kwargs 传参、${VAR} 变量占位符
- 新增 test_registry.py (20 个测试) 和扩展 test_yaml_loader.py
- 更新自驱动规则: 自动 commit+push, 删除需要用户明确指示的步骤
2026-07-04 18:24:52 +08:00
zhou db02443463 feat: 新增 YAML 任务编排功能
1. 新增 yaml_loader 模块,支持加载 GitHub Actions 风格的 YAML 任务图
2. 新增 Graph.from_yaml 静态方法,支持从 YAML 文件构建任务图
3. 新增 yamlrun CLI 工具,支持执行、预览 YAML 任务流水线
4. 添加 pyyaml 运行时依赖与 types-PyYAML 开发依赖
5. 更新 README 文档与对外暴露的 API 接口
2026-07-04 16:00:04 +08:00
zhou eb8e1402bc docs: 更新自驱动规则文档,补充决策判据与细节
补充自主决策的具体范围、收尾规则,新增决策判据章节,细化暂停条件与沟通要求
2026-07-04 15:29:47 +08:00
zhou c93f45dcb8 refactor: 统一使用px.task/px.cmd替代旧版TaskSpec创建任务
本次提交将项目内所有使用px.TaskSpec创建任务的代码,替换为新的px.task和px.cmd快捷API,简化了任务定义写法,同时更新了版本号到0.3.5。重构过程中保持了原有功能逻辑不变,仅调整了代码书写格式,提升了代码可读性和编写效率。
2026-07-04 15:22:27 +08:00
zhou a0b1814024 style: 格式化sshcopyid.py的列表代码,提升可读性
调整了px.Graph.from_specs的参数列表排版,将多行列表缩进优化为更简洁的单行展开格式,不改变代码实际功能。
2026-07-04 13:43:33 +08:00
zhou 3a2826d3f9 bump version to 0.3.5
CI / Lint, Typecheck & Test (push) Successful in 1m47s
Release / Build, Publish & Release (push) Successful in 1m2s
2026-07-04 11:36:07 +08:00
zhou dbd30689ab chore(ci): 更新release工作流的gitea服务地址
将GITEA_URL从10.0.16.16:3000调整为172.17.0.1:3000,适配新的内网部署地址
2026-07-04 11:36:04 +08:00
zhou 5eb59b8a66 bump version to 0.3.4
CI / Lint, Typecheck & Test (push) Successful in 1m8s
Release / Build, Publish & Release (push) Failing after 30s
2026-07-04 11:24:11 +08:00
zhou 8e7b866de2 更新 .github/workflows/release.yml
CI / Lint, Typecheck & Test (push) Has been cancelled
2026-07-04 03:23:16 +00:00
zhou 1b4f9bfa6a bump version to 0.3.3
CI / Lint, Typecheck & Test (push) Successful in 1m8s
Release / Build, Publish & Release (push) Has been cancelled
2026-07-04 11:16:31 +08:00
zhou 2d39272330 ci(github workflows): update pypi api token secret name
将PyPI发布步骤中的密钥变量名从PYPI_API_TOKEN改为PYPI_TOKEN,保持配置一致性
2026-07-04 11:16:24 +08:00
zhou f699bb9167 chore: 升级pyflowx版本到0.3.2
CI / Lint, Typecheck & Test (push) Successful in 1m19s
2026-07-04 10:50:50 +08:00
112 changed files with 10547 additions and 4714 deletions
+1 -2
View File
@@ -38,9 +38,8 @@ env
*.swp
*.swo
# 文档与示例(按需保留)
# 文档(按需保留)
docs
examples
# 系统文件
.DS_Store
+2 -2
View File
@@ -27,14 +27,14 @@ jobs:
run: uv build
- name: Publish to pypi
run: uv publish --token '${{ secrets.PYPI_API_TOKEN }}'
run: uv publish --token '${{ secrets.PYPI_TOKEN }}'
- name: Create Gitea Release & Upload Assets
env:
GITEA_TOKEN: ${{ secrets.GITHUB_TOKEN }}
TAG_NAME: ${{ github.ref_name }}
REPO: ${{ github.repository }}
GITEA_URL: http://gitea:3000
GITEA_URL: http://172.17.0.1:3000
run: |
set -e
# 1. 创建 Release
+4
View File
@@ -11,3 +11,7 @@ wheels/
.coverage
.idea
*_profile.html
# Sphinx 文档构建输出
docs/_build/
.trae/refs
+23
View File
@@ -0,0 +1,23 @@
# ReadTheDocs 配置
# https://docs.readthedocs.io/en/stable/config-file/v2.html
version: 2
# 构建配置
build:
os: ubuntu-24.04
tools:
python: "3.11"
# Python 依赖与构建命令
python:
install:
- method: pip
path: .
extra_requirements:
- docs
# Sphinx 构建
sphinx:
configuration: docs/conf.py
builder: html
fail_on_warning: false
+108
View File
@@ -0,0 +1,108 @@
# 文档整理与 Sphinx 文档搭建计划
## Context
最近完成 CLI 重构:新增 `pf` 统一入口,13 个工具迁移到 YAML 配置并删除了对应 .py 入口脚本,`run()` 的 verbose 统一应用到 spec。但文档未同步:README 仍引用旧命令(`yamlrun``python build.py`),模块结构表缺漏;`runner.py``_apply_verbose_to_graph` 成为死代码;项目缺少可发布的 Sphinx 文档。本次任务整理这些遗留,并搭建 ReadTheDocs 文档站。
## 任务范围
### 1. 清理死代码
- 删除 `src/pyflowx/runner.py``_apply_verbose_to_graph` 函数(line 38-68),功能已移入 `executors.py``run()`
- 删除 `tests/test_runner.py` 中对应测试(line 610-636`TestApplyVerboseToGraph` 类)。
- 清理 `runner.py` 顶部 `from dataclasses import replace` 若变为未使用。
### 2. 修复版本不一致
- `src/pyflowx/__init__.py:105` 硬编码 `__version__ = "0.4.5"``pyproject.toml:25``0.3.5`
- 统一为 `0.4.5``__init__.py` 为准,pyproject.toml 是源但 bumpversion 工具应同时更新两者)。
### 3. 更新 README.md
- L304-308`python build.py clean/build/test``pf pymake clean/build/test`
- L335-351、L435`yamlrun pipeline.yaml ...``pf yamlrun pipeline.yaml ...`6 处)。
- L311`verbose=True(默认)` 描述保留,但 CLI 示例改为 `pf`
- L558-574 模块结构表:补充 `cli/pf.py`(统一入口)、`cli/configs/`YAML 工具配置)、`cli/_ops/`(工具函数)、`profiling.py``registry.py`
- 顶部增加「文档」徽章链接到 ReadTheDocs。
### 4. 搭建 Sphinx 文档结构
新建 `docs/` 目录:
```
docs/
├── conf.py # Sphinx 配置
├── index.rst # 首页与目录
├── installation.rst # 安装
├── quickstart.rst # 快速上手(从 README 提炼)
├── guide/
│ ├── task.rst # TaskSpec 任务描述
│ ├── graph.rst # Graph DAG 构建
│ ├── execution.rst # 执行策略与 run()
│ ├── yaml.rst # YAML 任务编排
│ └── cli.rst # pf 统一入口与工具列表
├── api.rst # API 参考(automodule 自动生成)
└── changelog.rst # 变更日志摘要
```
**conf.py 要点**
- 扩展:`sphinx.ext.autodoc``sphinx.ext.napoleon`(支持 Google/NumPy docstring)、`sphinx.ext.viewcode``myst_parser`(支持 Markdown
- 主题:`sphinx_rtd_theme`
- 项目版本从 `pyflowx.__version__` 动态读取
- `autodoc_default_options``members: True, undoc-members: True, show-inheritance: True`
**api.rst**:用 `automodule:: pyflowx` 抓取 `__all__` 的 56 个公共符号。
### 5. ReadTheDocs 配置
- 新建 `.readthedocs.yaml`Python 3.11`pip install -e .[docs]``sphinx -b html docs/ docs/_build/`
- `.gitignore` 增加 `docs/_build/`
### 6. pyproject.toml 补充 docs 依赖
```toml
docs = [
"sphinx>=7.0",
"sphinx-rtd-theme>=2.0",
"myst-parser>=3.0",
]
```
并在 `[dependency-groups]` 的 dev 中加入 `pyflowx[docs]`
## 关键文件
| 文件 | 操作 |
|------|------|
| `src/pyflowx/runner.py` | 删除 `_apply_verbose_to_graph` |
| `tests/test_runner.py` | 删除 `TestApplyVerboseToGraph` |
| `src/pyflowx/__init__.py` | 版本统一(已 0.4.5,确认) |
| `pyproject.toml` | 版本 → 0.4.5;加 docs 依赖 |
| `README.md` | 更新 CLI 示例与模块结构表 |
| `docs/conf.py` | 新建 |
| `docs/*.rst` | 新建 |
| `.readthedocs.yaml` | 新建 |
| `.gitignore` | 加 docs/_build/ |
## 验证
1. **测试与 lint**
```bash
uv run pytest tests/ -q
uv run ruff check src/ tests/ docs/conf.py
uv run pyrefly check src/pyflowx/runner.py
```
2. **Sphinx 构建本地验证**
```bash
uv sync --extra docs
uv run sphinx-build -b html docs/ docs/_build/
```
确认无 warning,打开 `docs/_build/index.html` 检查页面。
3. **pf 功能回归**
```bash
pf gitt c
pf pymake b --dry-run
```
4. **RTD 配置校验**`.readthedocs.yaml` 语法正确,`docs/conf.py` 能独立构建。
## 不在范围
- 不统一各模块 docstring 风格(napoleon 兼容 Google/NumPy,够用)。
- 不重构现有 CLI 工具 YAML。
- 不新增中文文档翻译(文档用中文撰写,与项目既有风格一致)。
@@ -0,0 +1,171 @@
# 方案C:移除 YAML,统一 @px.tool 装饰器配置
## Context
现状:YAML 配置(22 个文件)+ `register_fn` 函数(79 个)分裂严重。fn 模式下 YAML 沦为冗余参数表——参数要三处重复声明(`variables`/`cli.options`/`jobs.args`),表达力不足(跨平台逻辑被迫下沉 `register_fn`),失类型检查/IDE 支持,函数签名与 YAML `args:` 顺序远端耦合。**矩阵 0 个实际使用、`if:` 0 个实际使用**,YAML 的声明式 DAG 优势在 fn 工具集场景完全没发挥。
目标:完全移除 YAML 支持,统一用 `@px.tool` 装饰器——函数签名驱动 CLI 生成,函数体即逻辑,DAG 编排通过装饰器参数(`needs`/`strategy`)表达。用户无感知迁移(别名/子命令/参数名全保持)。验收:全部门禁通过(ruff/pyrefly/pytest)且覆盖率 ≥95%。
## 设计要点
### 新模块 `src/pyflowx/tools.py`
职责:`@px.tool` 装饰器 + `ToolSpec`(frozen dataclass)+ 全局注册表 + CLI 生成 + 执行入口。**不扩展 `registry.py`**(职责不同,@px.tool 直接持有函数对象)。
### `@px.tool` API
```python
@px.tool("pdftool", subcommand="m", help="合并 PDF",
needs=["setup"], strategy="thread", # DAG 编排参数
cmd=["..."], # 可选:有 cmd 执行命令,无 cmd 执行函数体
hidden=False, allow_upstream_skip=False)
def pdf_merge(
inputs: list[Path], # positional(无默认值)→ nargs="+"
output: Path = Path("merged.pdf"), # option(有默认值)→ --output
password: str = "", # --password
dry_run: bool = False, # bool 默认 False → --dry-run store_true
level: int = 5, # --level type=int
mode: Literal["a","b"] = "a", # --mode choices=["a","b"]
) -> None:
"""函数体即逻辑(无 cmd 时执行;有 cmd 时签名仅驱动 CLI)."""
...
```
**装饰器关键字参数**:`subcommand``help``description``cmd``needs``strategy``cwd``allow_upstream_skip``hidden``env`/`retry`/`timeout`(透传 TaskSpec)。
**CLI 生成规则**(inspect.signature + typing.get_type_hints):
- 无默认值 → positional;有默认值 → `--kebab-case` 选项
- 类型映射:`list[X]``nargs="+"``Path``type=Path``bool`默认 False→`store_true``Literal``choices``int`/`float`→对应 type
- 全局选项 `--dry-run`/`--quiet`/`--strategy`/`--list` 注入每个 subparser
**执行分支**:
-`cmd`:TaskSpec 用 `cmd=cmd`,函数体不执行(签名仅驱动 CLI);cwd/env 从装饰器 + CLI kwargs 合并
-`cmd`:TaskSpec 用 `fn=func`,kwargs 透传
- 聚合任务(有 needs 无 cmd 无函数逻辑):fn=noop
**needs 传递依赖**:从 `_TOOL_REGISTRY[tool_name]` BFS 收集(复用 `_collect_with_deps` 思路),每个依赖生成 TaskSpec,构建子图 → `px.run(graph, strategy)`
**动态 cmd 改 fn 模式**:`piptool i`/`gittool clean` 等原 `cmd+${VAR}` 改为函数体内 `subprocess.run([...])`,消除 `${VAR}` 列表展开,模型更纯。
### pf.py 路由改造
- `_TOOL_ALIASES`(57 个)保留全量,值改为 tool_name(去掉 .yaml 含义)
- `_run_yaml``_run_tool`:`importlib.import_module(f"pyflowx.ops.{tool_name}")` 触发注册 → 从 `_TOOL_REGISTRY[tool_name]` 取 subcommand → `run_tool(tool, subcommand, argv)` → 退出码 0/1/130
- `_LEGACY_TOOLS` 保留 emlmanager/profiler;**删除 yamlrun**
- `_list_tools` 改为遍历 `_TOOL_REGISTRY` 取 description
### ops 模块重组
从 6 个功能模块(`files/dev/media/system/llm/bumpversion`)重组为 **22 个工具模块**:`ops/pdftool.py``ops/pymake.py``ops/envdev.py` 等,一文件一工具,文件内多个 `@px.tool` = 多 subcommand。
- 跨工具复用函数(如 `add_date_prefix` 被 filedate 用)放 `ops/_common.py`
- 未被 YAML 引用的约 20 个 `register_fn`(辅助函数如 `get_file_timestamp`/`has_files`)去掉装饰器,降级为模块级私有函数
- `register_fn`/`get_fn`/`has_fn` 在阶段5 一并移除(YAML 配套,失去用途)
## 迁移路径(6 阶段)
### 阶段1:核心实现 + 单元测试
- 新增 `src/pyflowx/tools.py``tests/test_tools.py`
- 覆盖:注册/重复检测/CLI 生成(各类型)/单命令执行/cmd 分支/fn 分支/聚合/DAG/hidden/全局选项/退出码三态
- 验证:`tools.py` 单元测试通过,覆盖率 ≥95%
### 阶段2:迁移简单单命令工具
- 工具:clr/which/taskkill/reseticoncache/screenshot/dockercmd/lscalc
- 新增 `ops/<tool>.py`,每个用 `@px.tool` 重写;删除对应 `configs/*.yaml`
-`tests/cli/test_*.py` 的 import 路径(函数体不变,测试基本不动)
- 验证:`pf clr`/`pf which python` 端到端 + 测试通过
### 阶段3:迁移多 subcommand fn 工具
- 工具:pdftool(14子命令)/piptool/packtool/filedate/filelevel/folderback/folderzip/sshcopyid/autofmt/bumpversion
- pdftool:函数签名已有完整类型注解,迁移最直接
- piptool:`i`/`up` 的 cmd 改 fn 模式(`subprocess.run`)
- bumpversion:positional `PART` 默认 "patch" + `--no-tag` store_true
- 验证:每工具 `pf <tool> --list` + 至少一个 subcommand 端到端
### 阶段4:迁移复杂工具
- pymake:12 cmd job + 6 聚合(ba/bump/cov/tc/p/pb)+ 4 内部 job(`hidden=True`:pyrefly_check/git_add_all/git_push/test_coverage)
- envdev:9 job,4 处 `allow_upstream_skip`;`install_rust` 用 needs + allow_upstream_skip
- gittool:`a`/`i`/`isub` 是 fn,`c`/`p`/`pl` 是 cmd,`clean` 是 hidden cmd
- sglang/msdownload:跨平台 + 多参数
- 验证:`pf pymake tc`/`pf pymake b`/`pf envdev` 端到端
### 阶段5:清理 YAML
- 删除:`yaml_loader.py``configs/*.yaml`(22 个)、`cli/yamlrun.py``tests/test_yaml_loader.py`
- 删除:`ops/{files,dev,media,system,llm,bumpversion}.py`(被 `ops/<tool>.py` 替代后)
- 删除:`registry.py``register_fn`/`get_fn`/`has_fn`/`FnRegistry`(YAML 配套)
- `__init__.py`:移除 `YamlLoadError`/`build_cli_parser`/`load_yaml`/`parse_yaml_string`/`run_cli`/`run_yaml`/`Graph.from_yaml`/`register_fn`/`get_fn`/`has_fn`/`FnRegistry`;新增 `tool`/`ToolSpec`/`run_tool`
- `graph.py`:删 `from_yaml` 类方法
- `pyproject.toml`:删 `yamlrun` 脚本入口、`pyyaml` 运行时依赖、`types-PyYAML` dev 依赖
- 验证:全局无 `import yaml`、无 `load_yaml`/`run_cli` 残留引用
### 阶段6:补充测试,覆盖率达标
- `tests/test_tools.py` 扩展:覆盖所有装饰器分支
- `tests/cli/test_*.py`(16 个):改为通过 `PfApp([tool, subcommand, ...]).run()``run_tool(...)` 测试
- 删除 `tests/test_yaml_loader.py`
- 验证:`pf pymake cov` ≥95% branch
## 关键文件
**新增**:
- `src/pyflowx/tools.py`(核心,@px.tool 装饰器)
- `src/pyflowx/ops/<tool>.py`(22 个工具模块)
- `src/pyflowx/ops/_common.py`(跨工具复用函数)
- `tests/test_tools.py`
**修改**:
- `src/pyflowx/cli/pf.py`(_run_yaml → _run_tool,加载 ops/<tool>.py)
- `src/pyflowx/__init__.py`(API 导出调整)
- `pyproject.toml`(删 yamlrun/pyyaml)
- `tests/cli/test_*.py`(import 路径 + 测试入口)
**删除**:
- `src/pyflowx/yaml_loader.py`
- `src/pyflowx/configs/*.yaml`(22 个)
- `src/pyflowx/cli/yamlrun.py`
- `tests/test_yaml_loader.py`
- `src/pyflowx/ops/{files,dev,media,system,llm,bumpversion}.py`(被替代后)
- `src/pyflowx/registry.py`(YAML 配套)
## 可复用的现有实现
- `_collect_with_deps`(yaml_loader.py:287):needs 传递依赖收集,迁到 tools.py
- `_add_global_options`(yaml_loader.py:1017):全局选项注入,思路复用
- `CliExitCode`(runner.py:30):退出码三态,直接复用
- `px.Graph.from_specs` + `px.run`:DAG 构建与执行,直接复用
- `TaskSpec`(task.py):任务描述符,@px.tool 内部构建它
## CLI 兼容性
- 57 个 pf 别名不变
- 子命令名不变(pdftool m/s/c/...、pymake b/ba/tc/...)
- 参数名不变(`--output`/`--password`/...,靠 snake_case→kebab-case 转换)
- `--dry-run`/`--quiet`/`--strategy`/`--list` 全局选项保留
- 退出码 0/1/130 三态保留
## 风险与权衡
- **动态 cmd 改 fn**:失去 cmd 任务的 verbose 输出(命令打印/returncode),但模型更统一;verbose 可在 fn 内手动 print
- **Graph.from_yaml 移除**:破坏性公共 API 变更,changelog 说明;yamlrun 用户迁移到 pf
- **ops 重组工作量大**:22 个工具模块从 6 个功能模块拆出,跨工具函数需识别;分阶段(2-4)渐进,每阶段可独立验证
- **register_fn 移除**:20 个未被引用的函数降级私有,被 @px.tool 间接调用的内联或放 `_common.py`
## 验证
每阶段结束运行:
```bash
uv run ruff check . && uv run ruff format --check .
uv run pyrefly check .
uv run pytest --cov=pyflowx --cov-branch --cov-report=term-missing
```
端到端验证(阶段4 后):
```bash
pf clr # 单命令
pf which python # 单命令 + positional
pf pdftool m a.pdf b.pdf --output merged.pdf # 多 subcommand + positional nargs+
pf pymake tc # DAG + 聚合 + 内部 job
pf envdev --python-mirror aliyun # 跨平台 fn
pf # 列出所有工具
```
最终:`pf pymake cov` 通过,覆盖率 ≥95%。
@@ -0,0 +1,228 @@
# 方案C 续接执行计划
## Summary
承接已批准的 `.trae/documents/方案C-迁移执行计划.md`。前序会话已完成**阶段1**(tools.py 测试 + 公共 API 导出)与**阶段2 的大部分工作**(7 个新 ops 模块创建 + pf.py 双路由 + 4 个测试文件 import 更新),但因上下文中断尚未收尾。本计划从当前状态起,完成阶段2 收尾,并继续执行阶段3-6,最终完全移除 YAML 支持,覆盖率 ≥95%。
## Current State Analysis
### 阶段1(已完成)
- `src/pyflowx/tools.py`:完整实现 @px.tool 装饰器、ToolSpec、_TOOL_REGISTRY、CLI 生成、依赖收集、run_tool、三态退出码
- `tests/test_tools.py`:110 测试全通过,覆盖率 96.91%
- `src/pyflowx/__init__.py`:已导出 `tool`/`ToolSpec`/`run_tool`/`list_tools`/`list_subcommands`(同时保留 YAML API 供过渡)
- `pyproject.toml`:已加 `[tool.ruff.lint.per-file-ignores]``src/pyflowx/tools.py = ["PLR0911"]`
### 阶段2(进行中,待收尾)
**已完成**:
- 7 个新 ops 模块创建完成,均使用 `@px.tool` 装饰器:
- `ops/clr.py`(`clear_screen_run`)
- `ops/which.py`(`which_run`)
- `ops/taskkill.py`(`taskkill_run`)
- `ops/reseticoncache.py`(`reset_icon_cache_run`)
- `ops/screenshot.py`(`take_screenshot_full`/`take_screenshot_area`,双 subcommand)
- `ops/dockercmd.py`(`docker_login_tencent`,subcommand="login")
- `ops/lscalc.py`(`run_ls_dyna`/`run_ls_dyna_mpi`/`check_ls_dyna_status`,三 subcommand)
- `pf.py` 已改双路由:`_run_yaml``_run_tool_or_yaml`(@px.tool 优先,YAML 回退)
- 4 个测试文件 import 路径已更新:`test_reseticoncache.py``test_screenshot.py``test_lscalc.py``test_system_run.py`
- 旧功能模块(`ops/system.py``ops/media.py``ops/dev.py`)中的原函数仍保留(`@px.register_fn`),供未迁移的 YAML 工具使用
**待完成**:
1. **test_envdev.py 无需改动**:该文件测试 `ops/dev.py` 中的 `docker_login_tencent` 等 10 个函数,这些函数在阶段4 前仍在 `dev.py` 中(`@px.register_fn` 装饰),`dockercmd.py` 中的 `@px.tool` 版本是独立副本,两者共存无冲突
2. **删除 7 个 YAML 配置文件**:`configs/{clr,which,taskkill,reseticoncache,screenshot,dockercmd,lscalc}.yaml`
3. **验证**:运行 ruff/pyrefly/pytest,确认阶段2 通过
### 阶段3-6(未开始)
- 阶段3:迁移 10 个多 subcommand fn 工具(pdftool/piptool/packtool/filedate/filelevel/folderback/folderzip/sshcopyid/autofmt/bumpversion)
- 阶段4:迁移 5 个复杂工具(pymake/envdev/gittool/sglang/msdownload)
- 阶段5:清理 YAML 与配套代码(yaml_loader.py/registry.py/configs 目录/旧 ops 功能模块)
- 阶段6:补充测试,覆盖率达标 95%
## Proposed Changes
### 阶段2 收尾:删除 YAML + 验证
**删除文件**(7 个):
- `src/pyflowx/configs/clr.yaml`
- `src/pyflowx/configs/which.yaml`
- `src/pyflowx/configs/taskkill.yaml`
- `src/pyflowx/configs/reseticoncache.yaml`
- `src/pyflowx/configs/screenshot.yaml`
- `src/pyflowx/configs/dockercmd.yaml`
- `src/pyflowx/configs/lscalc.yaml`
**验证**:
```bash
uv run ruff check . && uv run ruff format --check .
uv run pyrefly check .
uv run pytest tests/cli/test_{reseticoncache,screenshot,lscalc,system_run}.py -v
uv run pytest tests/test_tools.py -v
```
端到端冒烟(可选,若环境允许):
```bash
pf clr
pf which python
pf lscalc --list
```
### 阶段3:迁移多 subcommand fn 工具(10 个)
**工具清单**:`pdftool`(14 子命令)/`piptool`/`packtool`/`filedate`/`filelevel`/`folderback`/`folderzip`/`sshcopyid`/`autofmt`/`bumpversion`
**新建文件**(10 个,`ops/<tool>.py`,一文件一工具):
- `ops/pdftool.py`:14 个 `@px.tool("pdftool", subcommand="...")`,函数体来自 `ops/media.py`(`pdf_merge`/`pdf_split`/`pdf_compress`/`pdf_encrypt`/`pdf_decrypt`/`pdf_extract_text`/`pdf_extract_images`/`pdf_add_watermark`/`pdf_rotate`/`pdf_crop`/`pdf_info`/`pdf_ocr`/`pdf_reorder`/`pdf_to_images`/`pdf_repair`)
- `ops/piptool.py`:多 subcommand(`i`/`up`/`un`/`re`/`fr`/`dl`),`i`/`up` 原 cmd 模式改 fn 模式
- `ops/packtool.py`:来自 `ops/system.py`(`pack_source`/`pack_wheel`/`pack_dependencies`/`clean_build_dir`)
- `ops/filedate.py`/`ops/filelevel.py`/`ops/folderback.py`/`ops/folderzip.py`:来自 `ops/files.py`
- `ops/sshcopyid.py`:来自 `ops/system.py`(`ssh_copy_id`)
- `ops/autofmt.py`:4 subcommand(`fmt`/`lint`/`doc`/`sync`),`fmt`/`lint` 原 cmd 改 fn 模式
- `ops/bumpversion.py`:positional `part: str = "patch"` + `no_tag: bool = False`,直接调用 `bump_project_version`
**关键决策**:
- 函数签名保持不变(测试兼容),仅装饰器 `@px.register_fn``@px.tool(name, subcommand=...)` 变化
- 参数名 snake_case,CLI 自动转 kebab-case,**保持 CLI 兼容**
- `piptool i`/`up`:原 `cmd: ["pip","install","${PKG}"]` 改为 `def i(packages: list[str]) -> None: subprocess.run(["pip","install",*packages], check=True)`
- `autofmt fmt`/`lint`:原 `cmd: ["ruff","format","${TARGET}"]` 改为 `def fmt(target: str = ".") -> None: subprocess.run(["ruff","format",target], check=True)`
- `bumpversion`:原 `cmd: ["pf","bumpversion","patch"]` 改为直接调用 `bump_project_version` 函数(避免递归调用 pf)
**测试更新**(10 个):
- `tests/cli/test_pdftool.py``test_piptool.py``test_packtool.py``test_filedate.py``test_filelevel.py``test_folderback.py``test_folderzip.py``test_sshcopyid.py``test_autofmt.py``test_bumpversion.py`
- import 路径:`from pyflowx.ops import <tool>``from pyflowx.ops.<tool> import <func>`
- 函数调用不变(函数体未变)
**删除文件**(10 个):
- `configs/{pdftool,piptool,packtool,filedate,filelevel,folderback,folderzip,sshcopyid,autofmt,bumpversion}.yaml`
**验证**:每工具 `pf <tool> --list` + 对应 cli 测试通过
### 阶段4:迁移复杂工具(5 个)
**工具清单**:`pymake`/`envdev`/`gittool`/`sglang`/`msdownload`
**新建文件**(5 个):
- `ops/pymake.py`:
- 12 个 cmd subcommand(`b`/`bc`/`sync`/`c`/`t`/`tf`/`bumpversion`/`bumpmi`/`doc`/`lint`/`tox`):`@px.tool("pymake", subcommand="b", cmd=["uv","build"])` + 函数体 `pass` 或 docstring
- hidden 内部 job(`test_coverage`/`pyrefly_check`/`git_add_all`/`git_push`/`git_push_tags`/`twine_publish`/`publish_python`):`hidden=True`,`cmd=[...]`
- 6 个聚合(`ba`/`bump`/`cov`/`tc`/`p`/`pb`):`needs=[...]` + 函数体 `pass`
- `cwd: Path = Path(".")` 作为函数签名参数,CLI `--cwd` 覆盖
- `ops/envdev.py`:9 个 fn subcommand,来自 `ops/dev.py`;`install_rust``needs=["setup_rust","download_rustup"]` + `allow_upstream_skip=True`
- `ops/gittool.py`:`a`/`i`/`isub` 是 fn,`c`/`p`/`pl` 是 cmd,`clean``hidden=True` cmd
- `ops/sglang.py`/`ops/msdownload.py`:跨平台 fn,来自 `ops/llm.py`,`Constants.IS_LINUX`/`IS_WINDOWS` 判断在函数体内
**关键决策**:
- pymake 的 `cwd` 参数:`TaskSpec``cwd``_build_task_spec` 从函数签名参数 `cwd: Path = Path(".")` 取,CLI `--cwd` 透传
- pymake `bumpversion` 子命令:保持 cmd 模式 `cmd=["pf","bumpversion","patch"]`(调用外部 pf 命令)
- envdev 跨平台逻辑保留在函数体内,仅装饰器迁移
- gittool `c`(clean)是 hidden,被 `tc`/`p` 通过 needs 引用
**测试更新**(5 个):
- `tests/cli/test_envdev.py`:`docker_login_tencent` import 从 `pyflowx.ops.dev``pyflowx.ops.dockercmd`;其余 envdev 函数从 `pyflowx.ops.envdev` 导入
- `tests/cli/test_gittool.py``test_pymake.py`(若存在)、`test_llm.py`:import 路径改 `from pyflowx.ops import <tool>`
**删除文件**(5 个):
- `configs/{pymake,envdev,gittool,sglang,msdownload}.yaml`
**验证**:`pf pymake tc``pf envdev --python-mirror aliyun``pf gittool a -m "msg"` 端到端;cli 测试通过
### 阶段5:清理 YAML 与配套代码
**删除文件**:
- `src/pyflowx/yaml_loader.py`
- `src/pyflowx/cli/yamlrun.py`
- `src/pyflowx/registry.py`
- `src/pyflowx/configs/`(整个目录,22 个 .yaml 已在前阶段删完,删目录本身)
- `src/pyflowx/ops/{files,dev,media,system,llm,bumpversion}.py`(6 个功能模块,被 22 个工具模块替代)
- `tests/test_yaml_loader.py`
- `tests/test_registry.py`
**修改文件**:
- `src/pyflowx/__init__.py`:
- 删导入:`YamlLoadError`/`build_cli_parser`/`load_yaml`/`parse_yaml_string`/`run_cli`/`run_yaml`/`FnRegistry`/`get_fn`/`has_fn`/`register_fn`
-`__all__` 对应条目
- `src/pyflowx/graph.py`:删 `from_yaml` 类方法(若存在)
- `src/pyflowx/cli/pf.py`:
- `_run_tool_or_yaml``_run_tool`:仅走 `importlib.import_module` + `run_tool`,删除 YAML 回退分支
- `_LEGACY_TOOLS``yamlrun`(保留 `emlman`/`profiler`/`pxp`)
- `_list_tools` 改为遍历 `_TOOL_REGISTRY` 取 description(不再读 YAML)
- `_tool_description` 删除(不再读 YAML)
- 模块 docstring 更新(去掉"YAML 配置"描述)
- `src/pyflowx/ops/__init__.py`:从导入 6 个功能模块改为导入 22 个工具模块
- `pyproject.toml`:
- `[project.scripts]``yamlrun`
- `[project] dependencies``pyyaml>=6.0.1`
- `[project.optional-dependencies] dev``types-PyYAML>=6.0.12`
**全局检查**:
```bash
# 无残留 YAML 引用
grep -rE "import yaml|load_yaml|run_cli|register_fn|YamlLoadError|build_cli_parser|parse_yaml_string|from_yaml|yamlrun" src/ tests/ || echo "OK"
```
### 阶段6:补充测试 + 覆盖率达标
**扩展 `tests/test_tools.py`**:
- 补齐阶段2-4 迁移中发现的边界(如 `cwd` 从 CLI 透传、`env`/`retry`/`timeout` 透传 TaskSpec、`_has_function_logic` 对 async 函数/类方法、`_unwrap_optional``X | None` 等)
-`tools.py` 覆盖率未达 95%,补齐
**新增 `tests/cli/test_pf_routing.py`**:
- 测试 `PfApp` 路由:已知别名 → `_run_tool`、legacy → `_run_legacy`、未知 → 退出码 1
- 测试 `_list_tools` 输出格式
**更新 `tests/cli/test_*.py`**:
- 阶段2-4 已更新 import 路径,本阶段补充通过 `run_tool(...)``PfApp([tool, subcommand]).run()` 的端到端测试
**覆盖率验证**:
```bash
uv run pytest --cov=pyflowx --cov-branch --cov-report=term-missing
```
- 目标:`pyflowx.tools` ≥95%、整体 ≥95%(branch)
-`ops/<tool>.py` 因跨平台分支未覆盖,补 mock 测试(`Constants.IS_LINUX`/`IS_WINDOWS`)
**最终门禁**:
```bash
uv run ruff check . && uv run ruff format --check .
uv run pyrefly check .
uv run pytest --cov=pyflowx --cov-branch --cov-report=term-missing
```
全部门禁通过,覆盖率 ≥95%。
## Assumptions & Decisions
1. **延续已批准计划**:本计划是 `.trae/documents/方案C-迁移执行计划.md` 的续接,不改变其架构决策,仅推进执行
2. **test_envdev.py 阶段2 不改**:该文件测试 `ops/dev.py` 函数,dev.py 在阶段4 前仍存在,无需改动;阶段4 迁移 envdev 时再统一更新
3. **函数体复制策略**:阶段3-4 从旧功能模块复制函数体到新工具模块,旧模块在阶段5 统一删除,避免提前破坏未迁移工具
4. **CLI 兼容**:57 个 pf 别名不变、子命令名不变、参数名(snake_case → kebab-case)不变、全局选项保留、退出码三态保留
5. **cmd 任务改 fn 模式**:`piptool i`/`autofmt fmt` 等原 `cmd+${VAR}` 改为函数体内 `subprocess.run`,verbose 由函数内 `print` 补偿
6. **聚合任务判定**:`_has_function_logic` 用 ast 分析函数体(pass/.../docstring 视为无逻辑),迁移时聚合任务的函数体写 `pass`
7. **每阶段独立验证**:每阶段结束运行 ruff + pyrefly + pytest,确保不累积错误
8. **自动提交**:每阶段完成且门禁通过后自动 git commit + push(分支已跟踪远程),遵循中文提交信息风格
## Verification Steps
每阶段结束运行:
```bash
uv run ruff check . && uv run ruff format --check .
uv run pyrefly check .
uv run pytest tests/ --cov=pyflowx --cov-branch --cov-report=term-missing -x
```
端到端验证(阶段4 后):
```bash
pf clr # 单命令
pf which python # 单命令 + positional
pf pdftool m a.pdf b.pdf --output merged.pdf # 多 subcommand + positional nargs+
pf pdftool --list # --list 输出
pf pymake tc # DAG + 聚合 + 内部 job(hidden)
pf pymake b --cwd /tmp # cwd CLI 透传
pf envdev --python-mirror aliyun # 跨平台 fn
pf gittool a -m "msg" # fn subcommand
pf # 列出所有工具(从 _TOOL_REGISTRY)
pf unknown_tool # 退出码 1
```
阶段5 后全局清理验证:
```bash
grep -rE "import yaml|load_yaml|run_cli|register_fn|YamlLoadError|build_cli_parser|parse_yaml_string|from_yaml|yamlrun" src/ tests/ || echo "OK"
```
阶段6 最终:`pf pymake cov` 通过,覆盖率 ≥95%(branch)。
@@ -0,0 +1,251 @@
# 方案C 迁移执行计划
## Summary
承接已批准的高层设计 `.trae/documents/方案C-移除YAML-统一px-tool装饰器.md`。当前 `src/pyflowx/tools.py` 已完整实现(@px.tool 装饰器 + ToolSpec + 注册表 + CLI 生成 + 依赖收集 + run_tool 入口),其余所有代码仍处于旧 YAML 模式。本计划从当前状态起,执行 6 个阶段,完全移除 YAML 支持,统一用 `@px.tool` 装饰器,覆盖率 ≥95%。
## Current State Analysis
**已完成**:
- `src/pyflowx/tools.py`(607 行):ToolSpec/`@tool`/注册表/`_add_func_args_to_parser`/`_collect_with_deps`/`_build_task_spec`/`run_tool`,完整实现三态退出码(0/1/130)
**未完成(本计划覆盖)**:
1. `tests/test_tools.py` 不存在
2. `__init__.py` 仍导出 YAML API(`load_yaml`/`run_cli`/`YamlLoadError`/`build_cli_parser`/`parse_yaml_string`/`run_yaml`)+ `register_fn`/`get_fn`/`has_fn`/`FnRegistry`,未导出 `tool`/`ToolSpec`/`run_tool`/`list_tools`/`list_subcommands`
3. `pf.py` 仍走 `_run_yaml``px.run_cli(config_path, argv)`,57 个别名指向 .yaml 文件;`_LEGACY_TOOLS``yamlrun`
4. `ops/` 仍是 6 个功能模块(`files/dev/media/system/llm/bumpversion`),共 79 个 `@px.register_fn`;无 22 个工具模块
5. `configs/` 仍有 22 个 .yaml 文件
6. `tests/cli/test_*.py`(16 个)+ `tests/test_yaml_loader.py`(49KB)+ `tests/test_registry.py` 仍依赖旧结构
7. `pyproject.toml` 仍含 `yamlrun` 脚本入口、`pyyaml` 运行时依赖、`types-PyYAML` dev 依赖
8. `src/pyflowx/yaml_loader.py``src/pyflowx/cli/yamlrun.py``src/pyflowx/registry.py` 仍存在
9. `graph.py` 仍含 `from_yaml` 类方法(若存在)
**关键依赖关系**:
- `register_fn` 函数被 YAML `fn:` 字段引用,迁移后改用 `@px.tool` 直接持有函数对象,不再需要 `register_fn`/`get_fn`/`has_fn`
- 现有 `tests/cli/test_*.py` 直接调用 `media.pdf_merge(...)`/`dev._get_installed_packages()` 等函数体,迁移后函数体不变,仅 import 路径与装饰器变化
- `tools.py``run_tool` 依赖 `px.Graph.from_specs` + `px.run`,与 YAML 无耦合,可直接使用
## Proposed Changes
### 阶段1:tools.py 单元测试 + 公共 API 导出
**文件**:
- 新建 `tests/test_tools.py`
- 修改 `src/pyflowx/__init__.py`
**test_tools.py 覆盖点**(对应 `tools.py` 各分支):
1. 注册:`@tool` 装饰器注册到 `_TOOL_REGISTRY`,`list_tools`/`list_subcommands`/`get_tool`/`clear_tool_registry`
2. 重复检测:同名同 subcommand 二次注册抛 `ValueError`
3. CLI 生成各类型:
- positional(无默认值)/ option(有默认值)
- `list[Path]``nargs="+"`
- `Literal["a","b"]``choices`
- `int`/`float`/`Path`/`str``type=`
- `bool` 默认 False → `--flag store_true`;默认 True → `--no-flag store_false`
- `Optional[X]``_unwrap_optional`
4. 单命令工具执行:`run_tool` 返回 0(成功)/1(失败)/130(KeyboardInterrupt)
5. cmd 分支:`cmd=["..."]` 时构建 TaskSpec 用 cmd,函数体不执行
6. fn 分支:无 cmd 时执行函数体,kwargs 透传
7. 聚合任务:`_is_aggregate` 判定 pass/.../docstring,fn=`_noop`
8. DAG:`_collect_with_deps` BFS 拓扑顺序(依赖在前)
9. hidden:`list_subcommands(include_hidden=False)` 排除,`_build_parser` 不暴露
10. 全局选项:`--dry-run`/`--quiet`/`--strategy`/`--list`
11. 退出码三态:成功 0、`TaskFailedError` → 1、`PyFlowXError` → 1、`KeyboardInterrupt` → 130
12. 未注册工具:`run_tool("unknown")` → 1,stderr 提示
13. `_handle_list` 输出格式
14. `_snake_to_kebab` 转换
**__init__.py 改动**:
- 新增导入:`from .tools import ToolSpec, list_subcommands, list_tools, run_tool, tool`
- 新增 `__all__` 条目:`ToolSpec`/`list_subcommands`/`list_tools`/`run_tool`/`tool`
- 暂时**保留** YAML/register_fn 导出(阶段5 才删),避免提前破坏
**验证**:`uv run pytest tests/test_tools.py --cov=pyflowx.tools --cov-branch` 通过,`tools.py` 覆盖率 ≥95%
### 阶段2:迁移简单单命令工具(7 个)
**工具**:`clr`/`which`/`taskkill`/`reseticoncache`/`screenshot`/`dockercmd`/`lscalc`
**新建文件**(`ops/<tool>.py`,一文件一工具):
- `ops/clr.py`:`@px.tool("clr")` 包装 `clear_screen_run`(来自 `ops/system.py`)
- `ops/which.py`:`@px.tool("which")` 包装 `which_run`
- `ops/taskkill.py`:`@px.tool("taskkill")` 包装 `taskkill_run`
- `ops/reseticoncache.py`:`@px.tool("reseticoncache")` 包装 `reset_icon_cache_run`
- `ops/screenshot.py`:`@px.tool("screenshot")` 包装 `take_screenshot_full`/`take_screenshot_area`(两个 subcommand)
- `ops/dockercmd.py`:`@px.tool("dockercmd")` 包装 `docker_login_tencent`
- `ops/lscalc.py`:`@px.tool("lscalc")` 包装 `run_ls_dyna`/`run_ls_dyna_mpi`/`check_ls_dyna_status`(多 subcommand)
**函数迁移策略**:
-`ops/system.py` 把对应函数体复制到新 `ops/<tool>.py`,改 `@px.register_fn``@px.tool(...)`
- 函数签名保持不变(测试兼容),仅装饰器变化
- 跨工具复用的常量(`LS_DYNA_COMMANDS` 等)若被多工具引用,放 `ops/_common.py`;否则就地保留
**测试更新**:
- `tests/cli/test_reseticoncache.py``test_screenshot.py``test_lscalc.py``test_system_run.py`:import 路径从 `from pyflowx.ops import system` 改为 `from pyflowx.ops import <tool>`
- 函数调用不变(`system.clear_screen_run()``clr.clear_screen_run()` 或类似)
**删除**:`configs/{clr,which,taskkill,reseticoncache,screenshot,dockercmd,lscalc}.yaml`(7 个)
**验证**:`pf clr``pf which python``pf lscalc --list` 端到端可用;`uv run pytest tests/cli/test_{reseticoncache,screenshot,lscalc,system_run}.py` 通过
### 阶段3:迁移多 subcommand fn 工具(10 个)
**工具**:`pdftool`(14 子命令)/`piptool`/`packtool`/`filedate`/`filelevel`/`folderback`/`folderzip`/`sshcopyid`/`autofmt`/`bumpversion`
**新建文件**:
- `ops/pdftool.py`:14 个 `@px.tool("pdftool", subcommand="m"/"s"/"c"/...)`,函数来自 `ops/media.py`
- `ops/piptool.py`:多 subcommand(`i`/`up`/`un`/`re`/`fr`/`dl`),`i`/`up` 原 cmd 模式改 fn 模式(`subprocess.run` 在函数体内)
- `ops/packtool.py`:多 subcommand,来自 `ops/system.py`(`pack_source`/`pack_wheel`/`pack_dependencies`/`clean_build_dir`)
- `ops/filedate.py`/`filelevel.py`/`folderback.py`/`folderzip.py`:来自 `ops/files.py`
- `ops/sshcopyid.py`:来自 `ops/system.py`(`ssh_copy_id`)
- `ops/autofmt.py`:4 subcommand(`fmt`/`lint`/`doc`/`sync`),`fmt`/`lint` 原 cmd 改 fn 模式
- `ops/bumpversion.py`:positional `PART` 默认 "patch" + `--no-tag store_true`,来自 `ops/bumpversion.py`
**关键决策**:
- pdftool 函数签名已有完整类型注解,迁移最直接;`pdf_merge(input_paths: list[Path], output_path: Path)``@px.tool("pdftool", subcommand="m")` + `inputs: list[Path]`(positional nargs="+") + `output: Path = Path("merged.pdf")`
- 参数名映射:YAML 的 `INPUTS`/`OUTPUT`/`PASSWORD` → snake_case `inputs`/`output`/`password`,CLI 自动转 kebab-case `--output`/`--password`,**保持 CLI 兼容**
- `piptool i`/`up`:原 `cmd: ["pip", "install", "${PKG}"]` 改为 `def i(packages: list[str]) -> None: subprocess.run(["pip", "install", *packages], check=True)`
- `autofmt fmt`/`lint`:原 `cmd: ["ruff", "format", "${TARGET}"]` 改为 `def fmt(target: str = ".") -> None: subprocess.run(["ruff", "format", target], check=True)`
- bumpversion:positional `part: str = "patch"` + `no_tag: bool = False`(store_true),原 `cmd: ["pf", "bumpversion", "patch"]` 改为直接调用 `bump_project_version` 函数
**测试更新**:
- `tests/cli/test_pdftool.py``test_piptool.py``test_packtool.py``test_filedate.py``test_filelevel.py``test_folderback.py``test_folderzip.py``test_sshcopyid.py``test_autofmt.py``test_bumpversion.py`:import 路径改 `from pyflowx.ops import <tool>`
- 函数调用不变(函数体未变)
**删除**:`configs/{pdftool,piptool,packtool,filedate,filelevel,folderback,folderzip,sshcopyid,autofmt,bumpversion}.yaml`(10 个)
**验证**:每工具 `pf <tool> --list` + 至少一个 subcommand 端到端;对应 cli 测试通过
### 阶段4:迁移复杂工具(5 个)
**工具**:`pymake`/`envdev`/`gittool`/`sglang`/`msdownload`
**新建文件**:
- `ops/pymake.py`:
- 12 个 cmd subcommand(`b`/`bc`/`sync`/`c`/`t`/`tf`/`bumpversion`/`bumpmi`/`doc`/`lint`/`tox`):`@px.tool("pymake", subcommand="b", cmd=["uv","build"])` + `def b(cwd: Path = Path(".")) -> None: pass`
- 4 个 hidden 内部 job(`test_coverage`/`pyrefly_check`/`git_add_all`/`git_push`/`git_push_tags`/`twine_publish`/`publish_python`):`hidden=True`,`cmd=[...]`
- 6 个聚合(`ba`/`bump`/`cov`/`tc`/`p`/`pb`):`needs=[...]` + 函数体 `pass`
- `cwd: Path = Path(".")` 作为函数签名参数,CLI `--cwd` 覆盖
- `ops/envdev.py`:9 个 fn subcommand,4 处 `allow_upstream_skip=True`;`install_rust``needs=["setup_rust","download_rustup"]` + `allow_upstream_skip=True`
- `ops/gittool.py`:`a`/`i`/`isub` 是 fn,`c`/`p`/`pl` 是 cmd,`clean``hidden=True` cmd
- `ops/sglang.py`/`ops/msdownload.py`:跨平台 fn(来自 `ops/llm.py`),`Constants.IS_LINUX`/`IS_WINDOWS` 判断在函数体内
**关键决策**:
- pymake 的 `cwd` 参数:`TaskSpec``cwd``_build_task_spec``variables["cwd"]` 取,CLI `--cwd` 透传;装饰器级 `cwd=None`
- pymake `bumpversion` 子命令:原 `cmd: ["pf","bumpversion","patch"]` 改为直接 `cmd=["pf","bumpversion","patch"]`(保持 cmd 模式,因调用外部 pf 命令)
- envdev 跨平台逻辑保留在函数体内(已实现),仅装饰器迁移
- gittool `c`(clean)是 hidden,被 `tc`/`p` 通过 needs 引用
**测试更新**:
- `tests/cli/test_envdev.py``test_gittool.py``test_llm.py`:import 路径改 `from pyflowx.ops import <tool>`
- `test_pdftool.py` 等(若引用 gittool/pymake 的函数)同步改
**删除**:`configs/{pymake,envdev,gittool,sglang,msdownload}.yaml`(5 个)
**验证**:`pf pymake tc``pf pymake b``pf envdev --python-mirror aliyun``pf gittool a -m "msg"` 端到端;cli 测试通过
### 阶段5:清理 YAML 与配套代码
**删除文件**:
- `src/pyflowx/yaml_loader.py`
- `src/pyflowx/cli/yamlrun.py`
- `src/pyflowx/registry.py`
- `src/pyflowx/configs/`(整个目录,22 个 .yaml 已在前阶段删完,删目录本身)
- `src/pyflowx/ops/{files,dev,media,system,llm,bumpversion}.py`(6 个功能模块,被 22 个工具模块替代)
- `tests/test_yaml_loader.py`
- `tests/test_registry.py`(若专为 register_fn 写;若含其他内容则保留并更新)
**修改文件**:
- `src/pyflowx/__init__.py`:
- 删导入:`YamlLoadError`/`build_cli_parser`/`load_yaml`/`parse_yaml_string`/`run_cli`/`run_yaml`/`FnRegistry`/`get_fn`/`has_fn`/`register_fn`
-`__all__` 对应条目
- `src/pyflowx/graph.py`:删 `from_yaml` 类方法(若存在)
- `src/pyflowx/cli/pf.py`:
- `_TOOL_ALIASES` 保留 57 个别名(值已是工具名,无 .yaml 含义)
- `_run_yaml``_run_tool`:`importlib.import_module(f"pyflowx.ops.{tool_name}")` 触发 `@px.tool` 注册 → `from pyflowx.tools import run_tool; return run_tool(tool_name, argv)`
- `_LEGACY_TOOLS``yamlrun`(保留 `emlman`/`profiler`/`pxp`)
- `_list_tools` 改为遍历 `_TOOL_REGISTRY` 取 description
- `_tool_description` 删除(不再读 YAML)
- `src/pyflowx/ops/__init__.py`:从导入 6 个功能模块改为导入 22 个工具模块
- `pyproject.toml`:
- `[project.scripts]``yamlrun`
- `[project] dependencies``pyyaml>=6.0.1`
- `[project.optional-dependencies] dev``types-PyYAML>=6.0.12`
- `src/pyflowx/yaml_loader.py``import pyflowx.ops.*` 触发注册的副作用:迁移后由 `pf.py``importlib.import_module` 显式触发
**全局检查**:
- `Grep` 全局 `import yaml``load_yaml``run_cli``run_yaml``register_fn``get_fn``has_fn``FnRegistry``YamlLoadError``build_cli_parser``parse_yaml_string``from_yaml``yamlrun`,确认无残留引用
- `uv run ruff check .` + `uv run pyrefly check .` 通过
### 阶段6:补充测试 + 覆盖率达标
**扩展 `tests/test_tools.py`**:
- 阶段1 已覆盖核心分支,本阶段补齐阶段2-4 迁移中发现的边界(如 `cwd` 从 CLI 透传、`env`/`retry`/`timeout` 透传 TaskSpec、`_has_function_logic` 对 async 函数/类方法、`_unwrap_optional``X | None`(3.10+) 等)
- 阶段1 后若 `tools.py` 覆盖率未达 95%,本阶段补齐
**新增 `tests/cli/test_pf_routing.py`**:
- 测试 `PfApp` 路由:已知别名 → `_run_tool`、legacy → `_run_legacy`、未知 → 退出码 1
- 测试 `_list_tools` 输出格式
**更新 `tests/cli/test_*.py`**(16 个):
- 阶段2-4 已更新 import 路径,本阶段补充通过 `run_tool(...)``PfApp([tool, subcommand]).run()` 的端到端测试
- 删除 `tests/test_yaml_loader.py``tests/test_registry.py`(若纯 register_fn 测试)
**覆盖率验证**:
```bash
uv run pytest --cov=pyflowx --cov-branch --cov-report=term-missing
```
- 目标:`pyflowx.tools` ≥95%、整体 ≥95%(branch)
-`ops/<tool>.py` 因跨平台分支未覆盖,补 mock 测试(`Constants.IS_LINUX`/`IS_WINDOWS`)
**最终门禁**:
```bash
uv run ruff check . && uv run ruff format --check .
uv run pyrefly check .
uv run pytest --cov=pyflowx --cov-branch --cov-report=term-missing
```
全部门禁通过,覆盖率 ≥95%。
## Assumptions & Decisions
1. **tools.py 实现不重写**:已存在的 `tools.py` 经审查实现完整,本计划仅补测试,不改实现(除非测试发现 bug)
2. **函数体保持不变**:迁移仅改装饰器(`@px.register_fn``@px.tool`)与文件位置,函数体逻辑、签名、类型注解保持不变,最大化测试兼容
3. **CLI 兼容**:57 个 pf 别名不变、子命令名不变、参数名(snake_case → kebab-case)不变、全局选项保留、退出码三态保留
4. **cmd 任务改 fn 模式**:`piptool i`/`autofmt fmt` 等原 `cmd+${VAR}` 改为函数体内 `subprocess.run`,失去 cmd 任务的 verbose 输出,但模型更统一;verbose 由函数内 `print` 补偿
5. **聚合任务判定**:`_has_function_logic` 用 ast 分析函数体(pass/.../docstring 视为无逻辑),迁移时聚合任务的函数体写 `pass`
6. **ops 重组**:6 功能模块 → 22 工具模块,跨工具复用函数(如 `add_date_prefix` 被 filedate 用)放 `ops/_common.py`;未被 YAML 引用的约 20 个 `register_fn` 辅助函数(如 `get_file_timestamp`/`has_files`)去掉装饰器降级为模块级私有
7. **register_fn 移除时机**:阶段5 统一移除,避免阶段2-4 迁移中破坏未迁移工具
8. **pyyaml 依赖移除**:阶段5 从 `pyproject.toml` 删除,需确认无其他模块隐式依赖(全局 grep `import yaml` 验证)
9. **测试更新策略**:阶段2-4 每阶段更新对应 cli 测试 import,阶段6 补充端到端测试;`tests/test_yaml_loader.py` 阶段5 删除
## Verification Steps
每阶段结束运行:
```bash
uv run ruff check . && uv run ruff format --check .
uv run pyrefly check .
uv run pytest tests/ --cov=pyflowx --cov-branch --cov-report=term-missing -x
```
端到端验证(阶段4 后):
```bash
pf clr # 单命令
pf which python # 单命令 + positional
pf pdftool m a.pdf b.pdf --output merged.pdf # 多 subcommand + positional nargs+
pf pdftool --list # --list 输出
pf pymake tc # DAG + 聚合 + 内部 job(hidden)
pf pymake b --cwd /tmp # cwd CLI 透传
pf envdev --python-mirror aliyun # 跨平台 fn
pf gittool a -m "msg" # fn subcommand
pf # 列出所有工具(从 _TOOL_REGISTRY)
pf unknown_tool # 退出码 1
```
阶段5 后全局清理验证:
```bash
# 无残留 YAML 引用
grep -r "import yaml" src/ tests/ || echo "OK"
grep -r "load_yaml\|run_cli\|register_fn\|YamlLoadError" src/ tests/ || echo "OK"
```
阶段6 最终:`pf pymake cov` 通过,覆盖率 ≥95%(branch)。
+1 -1
View File
@@ -150,7 +150,7 @@ uvx --from pyflowx pymake cov
## Git 与提交
- **自动提交/push**:除非用户明确要求
- **自动提交**:任务完成后自动 `git add`(按文件名)+ `git commit` + `git push`(仅当分支已跟踪远程时执行 push;新分支跳过 push 并在总结中说明)
- **不修改 git config**。
- **不运行破坏性命令**`push --force`/`reset --hard`/`clean -f`)除非用户明确要求。
- **staging**:按文件名添加,不用 `git add -A`/`git add .`,避免误加敏感文件。
+134
View File
@@ -0,0 +1,134 @@
---
alwaysApply: true
---
# 自驱动开发规则
本规则定义一种"目标驱动、闭环执行"的工作模式:仅在任务开始时与用户确认一次目标与边界,后续由 Agent 自主完成"计划 → 编码 → 测试 → 文档 → 验证"的迭代循环,直到用户目标达成。
## 核心原则
- **目标导向**:始终以用户最终目标为准绳,所有阶段产出都应服务于该目标。
- **闭环执行**:每个子任务必须走完"计划 → 实现 → 测试 → 文档 → 验证"五步;禁止跳步留半成品。
- **自主决策**:初始确认之后,实现路径、API 形态、重构范围、文件命名、测试组织、错误修复策略等由 Agent 自行决断,不再逐项请示。**可逆操作(编辑文件、运行测试、修复 lint、调整实现)直接执行,不询问**;只有不可逆/高风险操作才暂停。
- **透明沟通**:每个阶段开始前用一句话说明意图;关键节点(完成、阻塞、转向)给简短更新;不复述内部思考,**不在收尾时停下询问"是否继续"或"是否提交"**——直接输出总结并结束。
- **安全边界**:仅在高风险、不可逆操作或真正阻塞时才暂停找用户。
## 初始确认(一次性,仅在最开始)
任务启动时,用 `AskUserQuestion` 一次性确认以下信息(已由项目规范覆盖的不必重复确认):
1. **目标与范围**:要解决什么问题?交付物是什么?显式列出不在范围内的内容。
2. **验收标准**:怎样算"完成"?可观测的判定条件(功能、性能、覆盖率阈值)。
3. **特殊约束**:除 `python-standards.md` 之外的约束(兼容性、依赖限制、API 兼容策略等)。
4. **测试要求**:覆盖率门槛(项目默认 ≥95%,branch);是否需要新增 `slow` 标记。
**git commit/push 不在确认范围内**:任务完成后自动 commit + push(仅当分支已跟踪远程时执行 push;新分支跳过 push 并在总结中说明),遵循 `.trae/rules/git-commit-message.md` 风格。仅 force-push、reset --hard、clean -f、修改 git config 等真正破坏性操作才需暂停确认。
确认后,将目标与验收标准固化进 `TaskCreate` 任务列表,后续不再就同一信息反复询问。
## 迭代循环
下列五个阶段构成一个完整闭环。未达验收标准时,回到「计划」开启下一轮;达标准时,进入「收尾」。
### 1. 计划(Plan
- 用 Explore/Glob/Grep 研究相关代码与既有模式,避免凭空设计。
-`TaskCreate` 把目标拆为可独立验证的子任务;每完成一项立即 `TaskUpdate` 为 completed。
- 优先复用现有抽象;不为本轮假想需求设计接口。
- 不过早抽象:三处相似才考虑提取,否则就地写。
### 2. 实现(Code
- 严格遵守 `.trae/rules/python-standards.md` 与既有代码风格。
- 优先 Edit 现有文件;新增文件需有明确职责边界。
- 不引入运行时依赖(项目零依赖原则);确需引入须在计划阶段说明。
- 公共 API 必须有完整类型注解与中文 docstring。
- 不写未被要求的功能、不为未来场景预留扩展点。
### 3. 测试(Test
- 新增/修改的公共 API 必须配套测试;优先通过公共接口测试,故障注入可访问私有属性并在 docstring 注明。
- Mock 优先级:`monkeypatch` > 内联 stub > `unittest.mock` > `pytest-mock`;禁用 `@patch` 装饰器。
- 必跑校验(每次修改后):
```bash
uvx --from pyflowx pymake tc
uvx --from pyflowx pymake cov
```
- 测试失败时定位根因再修复,不通过放宽断言或 `# pragma: no cover` 绕过。
- 覆盖率不得低于上一次的值(项目门槛 95%,branch)。
### 4. 文档(Docs
- 同步更新 docstring、README、模块结构说明。
- 行为变更须同步更新 `.agents/skills/pyflowx-development/SKILL.md` 中的对应章节。
- 跨会话有价值的设计决策、约束、陷阱,追加到 memory(`project_memory.md` 或对应 `topics.md`)。
- 不主动新建 `*.md` 文档;除非用户明确要求。
### 5. 验证(Verify
- 逐条对照初始确认的「验收标准」核验;未满足则回到「计划」继续下一轮。
- 全套门禁通过:ruff、pyrefly、pytest、coverage。
- 给出本轮变更清单(改了哪些文件、为什么)。
## 暂停条件(仅在以下情况中断自驱动找用户)
1. **歧义无法自决**:需求存在多种合理解读且无既有约定可循。
2. **高风险/不可逆操作**:删除非临时文件、`git push --force`、`reset --hard`、删表、修改 CI 配置、修改 git config、卸载依赖等。**普通 `git commit`/`push` 不属于此类**(任务完成后自动执行)。
3. **不可恢复的失败**:根因不在本仓库、需外部环境/权限配合、或经两轮尝试仍无法定位。
4. **超出初始确认范围**:用户目标在执行中发现需要显著扩大范围或改变方向。
5. **用户主动询问**:用户在对话中提出新问题或要求澄清。
**注意**"目标已达成"**不是**暂停条件——验收标准全部满足后直接进入收尾并结束任务,不询问"是否扩展范围"或"是否提交"。
非以上情况,一律继续自驱动,不要为"求确认"而暂停。
## 决策判据:该问还是自决
遇到不确定时,按以下顺序判断:
1. **是否不可逆/高风险?** 是 → 暂停确认(如删除文件、`push --force`、修改 CI 配置、卸载依赖)。否 → 继续。
2. **是否在初始确认范围内?** 是 → 按确认执行,不询问。否 → 视为"超出初始确认范围",暂停。
3. **是否有既有约定可循?** 是 → 按约定执行(参考 `python-standards.md`、`project_memory.md`)。否 → 视为"歧义无法自决",暂停。
4. **是否可逆?** 是 → 直接执行,即使结果可能不完美(可在后续迭代修正)。否 → 暂停。
**可直接自决(不询问)的典型情况**:
- 测试失败、覆盖率不达标、lint/类型检查报错 → 定位根因并修复。
- 代码风格选择(命名、模块划分、参数顺序)→ 自决。
- 文件编辑、运行测试、运行校验命令 → 直接执行。
- 任务完成后输出收尾总结 → 直接输出,不询问下一步。
- 显式指定 `name` 参数以保持测试兼容性 → 自决。
- 重命名局部变量以避免遮蔽 → 自决。
**必须暂停询问的典型情况**
- 删除非临时文件、重命名公共模块/包。
- `git push --force`、`reset --hard`、`clean -f`、修改 git config(普通 commit/push 自动执行,无需询问)。
- 引入新的运行时依赖(违反项目零依赖原则)。
- 修改 CI 配置、pre-commit 钩子、pyproject.toml 的工具链配置。
- 卸载或降级既有依赖。
## 沟通风格
- 阶段切换时一句话说明即可;不要把内部推理写给用户看。
- 完成子任务后用一两句总结改了什么、下一步做什么。
- 遇到阻塞时直接说明:卡在哪、试了什么、需要用户做什么。
- **不在收尾时询问"是否需要提交"或"是否扩展范围"**——直接输出总结并结束。用户后续若有新需求,由用户主动提出。
- 不使用 emoji,除非用户明确要求。
## 工具使用
- 独立操作尽量并行调用(多个 Read/Grep/Glob 一批发出)。
- 用 `TaskCreate`/`TaskUpdate` 维护进度,不批量推迟标记。
- 长命令用后台运行(`run_in_background`),完成会自动通知。
- 文件操作一律用专用工具:Read/Edit/Write/Glob/Grep,不用 `cat`/`sed`/`grep`/`find`。
## 收尾
- 验收标准全部满足后,**直接输出最终总结并结束任务**:交付物、关键决策、遗留事项。
- **自动提交**:收尾时自动 `git add`(按文件名)+ `git commit`(遵循 `.trae/rules/git-commit-message.md` 风格)+ `git push`(仅当分支已跟踪远程时执行;新分支跳过 push 并在总结中说明);**不询问**"是否需要提交"或"是否扩展范围"。
- 若验收标准未全部满足,回到「计划」继续下一轮,不停下询问。
- 将本次会话的关键产出与决策更新到 memory,便于后续会话续接。
+125 -18
View File
@@ -5,6 +5,7 @@
[![CI](https://github.com/gookeryoung/pyflowx/actions/workflows/ci.yml/badge.svg)](https://github.com/gookeryoung/pyflowx/actions/workflows/ci.yml)
[![PyPI](https://img.shields.io/pypi/v/pyflowx.svg)](https://pypi.org/project/pyflowx/)
[![Python](https://img.shields.io/pypi/pyversions/pyflowx.svg)](https://pypi.org/project/pyflowx/)
[![Documentation Status](https://readthedocs.org/projects/pyflowx/badge/?version=latest)](https://pyflowx.readthedocs.io/zh/latest/)
[![Coverage](https://img.shields.io/badge/coverage-100%25-brightgreen.svg)](https://github.com/gookeryoung/pyflowx)
[![License](https://img.shields.io/pypi/l/pyflowx.svg)](https://github.com/gookeryoung/pyflowx/blob/main/LICENSE)
@@ -31,7 +32,8 @@ PyFlowX 把"任务依赖"这件事做到极致简单:**参数名就是依赖
- **图级默认值** —— `GraphDefaults` 统一配置 retry/timeout/concurrency 等
- **CLI 运行器** —— `CliRunner` 把多个图映射为命令行子命令,替代 Makefile
- **可观测** —— `on_event` 回调(RUNNING/SUCCESS/FAILED/SKIPPED)、`dry_run` 预览、`verbose` 生命周期日志、Mermaid 可视化
- **零运行时依赖** —— 仅依赖标准库(3.8 需 `graphlib_backport`
- **YAML 任务编排** —— GitHub Actions 风格的声明式任务图,支持 `jobs`/`needs`/`strategy.matrix`/`if` 等 CI/CD 概念,从 YAML 文件直接加载执行
- **最小依赖** —— 仅依赖标准库 + PyYAML3.8 需 `graphlib_backport``typing-extensions`
- **97% 测试覆盖** —— 分支覆盖率 >= 95%
## 安装
@@ -300,31 +302,123 @@ runner.run_cli() # 解析 sys.argv 并执行
命令行用法:
```bash
python build.py clean # 执行 clean 图
python build.py build --strategy thread # 覆盖执行策略
python build.py test --dry-run # 仅打印执行计划
python build.py --list # 列出所有命令
python build.py --quiet # 静默模式
pf pymake clean # 执行 clean 图
pf pymake build --strategy thread # 覆盖执行策略
pf pymake test --dry-run # 仅打印执行计划
pf pymake --list # 列出所有命令
pf pymake --quiet # 静默模式
```
`verbose=True`(默认)时打印任务生命周期(开始/成功/失败/跳过)与命令输出;`--quiet` 关闭。
## 示例
## YAML 任务编排
仓库 `examples/` 目录包含完整示例:
PyFlowX 支持 GitHub Actions 风格的声明式 YAML 任务编排,从 YAML 文件直接加载任务图。
- [`etl_pipeline.py`](examples/etl_pipeline.py) —— ETL 流水线(sequential
- [`parallel_run.py`](examples/parallel_run.py) —— 并行执行对比(thread vs sequential
- [`async_aggregation.py`](examples/async_aggregation.py) —— 异步聚合 + Context 注入
### 编程式 API
运行:
```python
import pyflowx as px
# 从 YAML 文件加载任务图
graph = px.Graph.from_yaml("pipeline.yaml")
report = px.run(graph, strategy="thread")
# 或用函数式 API
graph = px.load_yaml("pipeline.yaml")
graph = px.parse_yaml_string("""
jobs:
hello:
cmd: ["echo", "hello"]
""")
```
### CLI 入口
通过 `pf` 统一入口调用(详见 [pf 工具](#cli-工具) 章节):
```bash
python examples/etl_pipeline.py
python examples/parallel_run.py
python examples/async_aggregation.py
# 执行 YAML 任务图
pf yamlrun pipeline.yaml
# 指定执行策略
pf yamlrun pipeline.yaml --strategy thread
# 仅打印任务分层,不执行
pf yamlrun pipeline.yaml --dry-run
# 列出所有任务名
pf yamlrun pipeline.yaml --list
# 静默模式
pf yamlrun pipeline.yaml --quiet
```
### YAML SchemaGitHub Actions 风格)
```yaml
strategy: thread # 图级默认策略
defaults: # 图级默认值
retry: {max_attempts: 3}
verbose: true
env: {CI: "true"}
jobs:
setup:
cmd: ["git", "clone", "..."]
runs-on: linux
build:
needs: [setup] # 依赖列表
cmd: ["python", "-m", "build"]
timeout: 300
retry: {max_attempts: 2, delay: 1.0}
test:
needs: [build]
cmd: ["python${{ matrix.version }}", "-m", "pytest"] # 矩阵占位符
strategy:
matrix: # 笛卡尔积展开为 6 个任务
version: ["3.8", "3.9", "3.10"]
os: ["linux", "macos"]
if: "env.CI" # 条件: 环境变量存在
lint:
needs: [build]
cmd: ["ruff", "check"]
if: "env.CI == 'true'" # 条件: 环境变量等于
deploy:
needs: [test, lint] # 矩阵依赖自动展开
cmd: ["twine", "upload"]
if: "env.DEPLOY_TOKEN != ''"
allow-upstream-skip: true
concurrency-key: deploy_lock
```
### 字段映射
| YAML 字段 | TaskSpec 字段 | 说明 |
|-----------|---------------|------|
| `jobs.<id>` | `name` | job ID 作为任务名 |
| `cmd` / `run` | `cmd` | `cmd` 为列表形式,`run` 为 shell 字符串 |
| `needs` | `depends_on` | 依赖列表(矩阵任务自动展开) |
| `if` | `conditions` | `success()` / `always()` / `env.VAR` / `env.VAR == 'x'` |
| `strategy.matrix` | 矩阵扇出 | 笛卡尔积展开为多个任务 |
| `${{ matrix.key }}` | 占位符 | 在 cmd/run/cwd/env 中替换 |
| `timeout` | `timeout` | 超时秒数 |
| `retry` | `retry` | `{max_attempts, delay, backoff, jitter}` |
| `cwd` | `cwd` | 工作目录 |
| `env` | `env` | 环境变量 |
| `verbose` | `verbose` | 详细输出 |
| `continue-on-error` | `continue_on_error` | 失败不中止整图 |
| `skip-if-missing` | `skip_if_missing` | 命令不存在时跳过 |
| `allow-upstream-skip` | `allow_upstream_skip` | 上游跳过时仍执行 |
| `priority` | `priority` | 同层优先级 |
| `concurrency-key` | `concurrency_key` | 并发限制键 |
| `tags` | `tags` | 自由标签 |
| `runs-on` | `tags`(追加) | 运行环境标签 |
## 断点续跑
```python
@@ -441,12 +535,14 @@ uv run pytest --cov=pyflowx --cov-fail-under=95
uv run mypy
# 代码风格
uv run ruff check src tests examples
uv run ruff format --check src tests examples
uv run ruff check src tests
uv run ruff format --check src tests
```
## 模块结构
### 核心
| 模块 | 职责 |
|------|------|
| `task.py` | 纯数据结构:`TaskSpec``RetryPolicy``TaskHooks``TaskStatus` |
@@ -455,11 +551,22 @@ uv run ruff format --check src tests examples
| `context.py` | 上下文注入:参数名→依赖解析 |
| `command.py` | 命令执行:`run_command`list/shell/Callable |
| `conditions.py` | 条件执行:内置条件与组合器 |
| `executors.py` | 执行器与 `run` 入口:四种策略共享模块级辅助 |
| `executors.py` | 执行器与 `run` 入口:四种策略共享模块级辅助verbose 统一应用到 spec |
| `storage.py` | 状态后端:`MemoryBackend` / `JSONBackend`batch flush |
| `runner.py` | CLI 运行器:`CliRunner` |
| `report.py` | 运行结果:`RunReport` / `TaskResult` |
| `tools.py` | `@px.tool` 装饰器:`ToolSpec` / `run_tool` / `list_tools` / `list_subcommands` |
| `profiling.py` | 性能分析:`Profiler` 任务耗时统计 |
| `errors.py` | 错误家族:`PyFlowXError` 子类 |
| `ops/` | CLI 工具模块集合(每个子模块用 `@px.tool` 注册一个工具) |
### CLI 工具
| 模块 | 职责 |
|------|------|
| `cli/pf.py` | 统一入口:`pf <tool> [command]`,按需导入 `ops/<tool>.py` 触发 `@px.tool` 注册 |
| `cli/profiler.py` | 性能分析 CLI |
| `cli/emlmanager.py` | 邮件管理 CLI |
## 许可证
+102
View File
@@ -0,0 +1,102 @@
API 参考
========
任务描述
--------
.. autoclass:: pyflowx.TaskSpec
:members:
:undoc-members:
:show-inheritance:
:exclude-members: args, kwargs
.. autoclass:: pyflowx.RetryPolicy
:members:
:undoc-members:
.. autoclass:: pyflowx.TaskHooks
:members:
:undoc-members:
.. autoclass:: pyflowx.TaskStatus
:members:
:undoc-members:
图构建
------
.. autoclass:: pyflowx.Graph
:members:
:undoc-members:
:exclude-members: from_specs, from_yaml
.. autoclass:: pyflowx.GraphDefaults
:members:
:undoc-members:
.. autofunction:: pyflowx.compose
.. autofunction:: pyflowx.task_template
执行
----
.. autofunction:: pyflowx.run
.. autoclass:: pyflowx.RunReport
:members:
:undoc-members:
.. autoclass:: pyflowx.TaskResult
:members:
:undoc-members:
@px.tool 工具
-------------
.. autoclass:: pyflowx.ToolSpec
:members:
:undoc-members:
.. autofunction:: pyflowx.tool
.. autofunction:: pyflowx.run_tool
.. autofunction:: pyflowx.list_tools
.. autofunction:: pyflowx.list_subcommands
命令执行
--------
.. autofunction:: pyflowx.run_command
CLI 运行器
----------
.. autoclass:: pyflowx.CliRunner
:members:
:undoc-members:
状态后端
--------
.. autoclass:: pyflowx.StateBackend
:members:
:undoc-members:
.. autoclass:: pyflowx.MemoryBackend
:members:
:undoc-members:
.. autoclass:: pyflowx.JSONBackend
:members:
:undoc-members:
错误家族
--------
.. autoexception:: pyflowx.PyFlowXError
.. autoexception:: pyflowx.DuplicateTaskError
.. autoexception:: pyflowx.MissingDependencyError
.. autoexception:: pyflowx.CycleError
.. autoexception:: pyflowx.TaskFailedError
.. autoexception:: pyflowx.TaskTimeoutError
.. autoexception:: pyflowx.InjectionError
.. autoexception:: pyflowx.StorageError
+45
View File
@@ -0,0 +1,45 @@
变更日志
========
0.4.5
-----
CLI 重构
~~~~~~~~
- 新增 ``pf`` 统一入口:通过 ``pf <tool> [command] [options]`` 调用所有工具
- 13 个工具迁移到 YAML 配置(filedate/filelevel/folderback/folderzip/screenshot/sshcopyid/lscalc/bumpversion/autofmt/piptool/packtool/pdftool/gittool
- YAML 配置支持 ``cli:`` 段声明命令行参数 schema,由 ``build_cli_parser`` 自动生成 argparse
- 删除 13 个冗余 ``.py`` 入口脚本,统一通过 ``pf`` 调用
- ``run()````verbose=True`` 时自动把 verbose 标记应用到所有 spec
- 全局选项 ``--verbose`` 改为 ``--quiet``(默认显示执行过程)
- ``cmd`` 任务成功时打印 stdout(此前被静默丢弃)
- ``gittool````CLEAN_EXCLUDES`` 数组变量配置 ``git clean -e`` 参数
YAML 任务编排
~~~~~~~~~~~~~
- 支持 ``variables`` 变量定义,``${VAR}`` 在 cmd/env/cwd 中替换
- 列表变量展开为 cmd 数组多个元素
- ``cli:`` 段支持 subcommands/positional/options 三级 schema
- 支持 ``type: path`` 自动转为 ``pathlib.Path``
文档
~~~~
- 搭建 Sphinx 文档,发布到 ReadTheDocs
- 更新 READMECLI 示例改为 ``pf`` 统一入口,模块结构表补全
0.3.x
-----
- 新增 YAML 任务编排(GitHub Actions 风格 schema
- 新增 ``fn:`` 函数引用与 ``register_fn`` / ``get_fn`` 注册中心
- 新增 ``compose`` / ``GraphComposer`` 多图组合
- 新增 ``task_template`` 任务模板工厂
- 新增 ``concurrency_key`` + ``concurrency_limits`` 并发限制
- 新增 ``JSONBackend`` 断点续跑与 ``batch()`` 批量落盘
- 新增 ``cache_key`` 缓存键函数
- 新增条件执行(``IS_WINDOWS`` / ``HAS_INSTALLED`` / ``ENV_VAR_EQUALS`` 等)
- 四种执行策略:``sequential`` / ``thread`` / ``async`` / ``dependency``
- 参数名即依赖的上下文注入机制
+65
View File
@@ -0,0 +1,65 @@
"""Sphinx 配置.
ReadTheDocs 构建 PyFlowX 文档站。
"""
from __future__ import annotations
import sys
from pathlib import Path
# 确保 src/ 在 sys.path 中, autodoc 能导入 pyflowx
sys.path.insert(0, str(Path(__file__).resolve().parent.parent / "src"))
from pyflowx import __version__
# -- 项目信息 --------------------------------------------------------------
project = "PyFlowX"
author = "pyflowx"
copyright = "2024, pyflowx"
release = __version__
version = __version__
# -- Sphinx 配置 -----------------------------------------------------------
extensions = [
"sphinx.ext.autodoc",
"sphinx.ext.napoleon",
"sphinx.ext.viewcode",
"sphinx.ext.intersphinx",
"myst_parser",
]
# -- 主题 ------------------------------------------------------------------
html_theme = "sphinx_rtd_theme"
html_static_path = ["_static"]
# -- autodoc 配置 ----------------------------------------------------------
autodoc_default_options = {
"members": True,
"undoc-members": True,
"show-inheritance": True,
"member-order": "bysource",
}
autodoc_type_hints = "description"
autodoc_typehints_format = "short"
# -- napoleon 配置 (Google/NumPy docstring 兼容) --------------------------
napoleon_google_docstring = True
napoleon_numpy_docstring = True
napoleon_include_init_with_doc = False
napoleon_include_private_with_doc = False
napoleon_include_special_with_doc = True
# -- intersphinx -----------------------------------------------------------
intersphinx_mapping = {
"python": ("https://docs.python.org/3", None),
}
# -- 全局选项 ---------------------------------------------------------------
language = "zh_CN"
master_doc = "index"
exclude_patterns = ["_build", "Thumbs.db", ".DS_Store"]
source_suffix = {
".rst": "restructuredtext",
".md": "markdown",
}
+158
View File
@@ -0,0 +1,158 @@
pf 统一 CLI 入口
================
所有工具通过 ``pf <tool> [command] [options]`` 调用。工具定义在 ``cli/configs/`` 目录下的 YAML 文件中。
基本用法
--------
.. code-block:: bash
pf # 列出所有可用工具
pf filedate # 查看 filedate 工具帮助
pf filedate add a.txt # 调用 filedate 的 add 子命令
pf gitt c # 调用 gittool 的 c 子命令
pf pymake b # 调用 pymake 的 b 别名
全局选项
--------
所有 YAML 工具支持以下全局选项:
.. list-table::
:header-rows: 1
:widths: 25 75
* - 选项
- 说明
* - ``--dry-run``
- 仅打印执行计划,不执行
* - ``--quiet`` / ``-q``
- 减少输出,不显示执行过程
* - ``--strategy``
- 执行策略(``sequential`` / ``thread`` / ``async`` / ``dependency``
* - ``--list``
- 列出所有任务名后退出
默认 ``verbose`` 开启,显示执行过程(任务开始/命令/返回码/任务成功)。``--quiet`` 关闭。
YAML 配置工具
--------------
.. list-table::
:header-rows: 1
:widths: 20 15 65
* - 工具
- 别名
- 说明
* - ``filedate``
- ``fd``
- 文件日期处理
* - ``filelevel``
- ``fl``
- 文件等级重命名
* - ``folderback``
- ``fb``
- 文件夹备份
* - ``folderzip``
- ``fz``
- 文件夹压缩
* - ``gittool``
- ``gitt``
- Git 执行工具
* - ``lscalc``
- ``ls``
- LS-DYNA 计算工具
* - ``packtool``
- ``pack``
- Python 打包工具
* - ``pdftool``
- ``pdf``
- PDF 文件工具集
* - ``piptool``
- ``pip``
- pip 包管理工具
* - ``screenshot``
- ``ss``
- 截图工具
* - ``sshcopyid``
- ``ssh``
- SSH 密钥部署工具
* - ``autofmt``
- ``af``
- 自动格式化工具
* - ``bumpversion``
- ``bump``
- 版本号自动管理工具
传统工具
--------
.. list-table::
:header-rows: 1
:widths: 20 80
* - 工具
- 说明
* - ``pymake``
- 构建工具(替代 Makefile),如 ``pf pymake b`` 构建
* - ``yamlrun``
- YAML pipeline 执行器,``pf yamlrun pipeline.yaml``
* - ``profiler``
- 性能分析
* - ``emlman``
- 邮件管理
* - ``reseticon``
- 重置图标缓存
自定义工具
----------
``cli/configs/`` 目录新建 ``<tool>.yaml`` 即可被 ``pf`` 自动发现:
.. code-block:: yaml
# cli/configs/mytool.yaml
strategy: sequential
variables:
MSG: "hello"
cli:
description: "我的工具"
usage: "pf mytool [command]"
subcommands:
greet:
help: "打招呼"
jobs:
greet:
cmd: ["echo", "${MSG}"]
执行::
pf mytool greet
CliRunner(编程式)
-------------------
``CliRunner`` 把多个 Graph 映射为命令行子命令,适合构建项目专属构建工具:
.. code-block:: python
runner = px.CliRunner(
strategy="sequential",
description="My Build Tool",
graphs={
"clean": clean_graph,
"build": build_graph,
"test": test_graph,
},
)
runner.run_cli() # 解析 sys.argv 并执行
命令行::
pf pymake clean
pf pymake build --strategy thread
pf pymake test --dry-run
pf pymake --list
pf pymake --quiet
+93
View File
@@ -0,0 +1,93 @@
执行策略与 run()
=================
``run()`` 是执行入口,支持四种策略:
.. code-block:: python
report = px.run(
graph,
strategy="async", # sequential | thread | async | dependency
max_workers=8, # thread 策略的线程池大小
concurrency_limits={"db": 2}, # 按 concurrency_key 限流
dry_run=False, # True = 仅打印计划
verbose=True, # True = 打印执行过程
on_event=callback, # 状态转换回调
state=px.JSONBackend("state.json"), # 断点续跑后端
continue_on_error=False, # True = 单任务失败不中断整体
)
策略对比
--------
.. list-table::
:header-rows: 1
:widths: 18 18 30 16 18
* - 策略
- 并发模型
- 适用场景
- 同步任务
- 异步任务
* - ``sequential``
- 串行
- 调试、CPU 密集
- 直接调用
- 事件循环
* - ``thread``
- 线程池
- I/O 密集同步
- 线程池
- 不支持
* - ``async``
- 事件循环
- I/O 密集异步
- 卸载到线程池
- 事件循环
* - ``dependency``
- 依赖驱动
- 最大化并行度
- 卸载到线程池
- 事件循环
所有策略都遵循 ``RetryPolicy````timeout``、上下文注入、状态后端、``concurrency_limits``
并发出 ``TaskEvent``RUNNING/SUCCESS/FAILED/SKIPPED)。``dependency`` 策略无层屏障:
任务在其所有硬依赖完成后立即启动。
上下文注入规则
--------------
按顺序求值:
1. **标注为 ``Context``** 的参数 → 接收完整上游结果映射
2. **名称匹配依赖** 的参数 → 接收该依赖的结果(含软依赖,缺失时注入默认值)
3. **``**kwargs``** 参数 → 接收所有依赖结果(dict)
4. **``TaskSpec.args`` / ``kwargs``** → 为非依赖参数提供静态值
.. code-block:: python
from typing import Any, Dict
def aggregate(ctx: px.Context) -> Dict[str, Any]:
"""ctx 包含所有 depends_on 任务的返回值。"""
return dict(ctx)
def merge(fetch_a: str, fetch_b: str) -> str:
"""fetch_a / fetch_b 自动注入。"""
return fetch_a + fetch_b
断点续跑
--------
.. code-block:: python
from pyflowx import JSONBackend
backend = JSONBackend("state.json", ttl=3600)
report = px.run(graph, strategy="sequential", state=backend)
``run()`` 内部以 ``backend.batch()`` 包裹整个执行:所有 ``save`` 延迟到运行结束时统一落盘一次。
缓存键:默认存储键为任务名。配置 ``cache_key`` 函数后,键为 ``"name:cache_key_value"``
完整 API 说明详见 :doc:`/api`
+50
View File
@@ -0,0 +1,50 @@
Graph —— DAG 构建
=================
``Graph`` 管理任务集合,提供建构建、校验、分层、可视化能力。
构建方式
--------
.. code-block:: python
# 图级默认值:TaskSpec 字段为 None 时回退
defaults = px.GraphDefaults(retry=px.RetryPolicy(max_attempts=2), timeout=60.0)
graph = px.Graph.from_specs([...], defaults=defaults) # 整批校验(推荐)
# 或增量构建
graph = px.Graph(defaults=defaults)
graph.add(px.TaskSpec("a", fn_a))
graph.add(px.TaskSpec("b", fn_b, ("a",)))
常用方法
--------
.. code-block:: python
graph.validate() # 显式校验(环检测)
graph.layers() # 拓扑分层(Kahn 算法)
graph.to_mermaid() # Mermaid 可视化
graph.describe() # 人类可读摘要
graph.subgraph(("api",)) # 按标签切片
graph.subgraph_by_names(("a", "b")) # 按名称切片
graph.map("fetch", [1, 2, 3], lambda i: TaskSpec(f"fetch_{i}", ...)) # 批量 fan-out
图组合
------
``compose`` / ``GraphComposer`` 把带字符串引用的多个图展开为纯 ``Graph``
.. code-block:: python
graphs = {
"build": px.Graph.from_specs([px.TaskSpec("b", cmd=["echo", "b"])]),
"all": px.Graph.from_specs(["build", px.TaskSpec("t", cmd=["echo", "t"])]),
}
resolved = px.compose(graphs) # "all" 图中的 "build" 引用被展开
引用格式:``"command_name"``(整个图)或 ``"command_name.task_name"``(特定任务)。
``CliRunner`` 内部自动调用 ``compose``
完整方法说明详见 :doc:`/api`
+89
View File
@@ -0,0 +1,89 @@
TaskSpec —— 任务描述
=====================
``TaskSpec`` 是不可变的任务描述符(``Generic[T]``,返回类型一路传到 ``RunReport``),是唯一需要配置的东西。
主要参数说明:
.. code-block:: python
px.TaskSpec(
name="fetch_user", # 唯一标识
fn=fetch_user, # 同步或异步函数
cmd=["curl", "..."], # 或: 执行命令(覆盖 fn)
depends_on=("auth",), # 硬依赖(参与拓扑分层)
soft_depends_on=("cache",), # 软依赖(仅注入,不参与分层)
args=(uid,), # 静态位置参数(追加在注入参数后)
kwargs={"timeout": 30}, # 静态关键字参数
retry=px.RetryPolicy(max_attempts=3, delay=1.0, backoff=2.0),
timeout=30.0, # 超时秒数(None = 不限制)
tags=("api", "user"), # 自由标签,用于子图过滤
conditions=(is_prod,), # 条件函数列表(全部为 True 才执行)
priority=10, # 同层内优先级(高优先执行,默认 0)
concurrency_key="db", # 并发分组键(配合 concurrency_limits 限流)
cache_key=lambda ctx: str(ctx.get("uid")), # 缓存键函数
hooks=px.TaskHooks(pre_run=..., post_run=..., on_failure=...),
cwd=Path("/tmp"), # 命令工作目录(仅 cmd 模式)
env={"DEBUG": "1"}, # 环境变量覆盖
verbose=True, # 打印命令输出(仅 cmd 模式)
skip_if_missing=True, # 命令不存在时自动跳过(仅 list[str] cmd
allow_upstream_skip=False, # 上游 SKIPPED/FAILED 时是否仍执行
continue_on_error=False, # 本任务失败是否不中断整体
)
两种任务形态
------------
- **函数任务**``fn``):普通 Python 函数,参数名驱动自动注入
- **命令任务**``cmd``):执行外部命令,支持 ``list[str]````str``shell)、``Callable`` 三种形态
``skip_if_missing=True`` 时,``list[str]`` 类型的 ``cmd`` 会通过 ``shutil.which`` 检查命令是否存在,不存在则跳过任务(标记为 ``SKIPPED``)而非失败。
重试策略
--------
``RetryPolicy`` 配置重试次数、延迟、退避:
.. code-block:: python
retry = px.RetryPolicy(
max_attempts=3, # 最大尝试次数
delay=1.0, # 初始延迟秒数
backoff=2.0, # 退避倍数
jitter=0.1, # 随机抖动(避免惊群)
retry_on=(ConnectionError,), # 仅对这些异常重试
)
任务钩子
--------
``TaskHooks`` 在任务生命周期触发(异常仅记录,不影响任务状态):
.. code-block:: python
hooks = px.TaskHooks(
pre_run=lambda spec: print(f"start {spec.name}"),
post_run=lambda spec, value: print(f"done {spec.name}"),
on_failure=lambda spec, exc: alert(spec.name, exc),
)
px.TaskSpec("task", fn=work, hooks=hooks)
任务模板
--------
``task_template`` 工厂批量生成相似 TaskSpec
.. code-block:: python
fetch = px.task_template(
fn=fetch_url,
retry=px.RetryPolicy(max_attempts=5),
timeout=30.0,
tags=("api",),
)
graph = px.Graph.from_specs([
fetch("users", url="https://api.example.com/users"),
fetch("posts", url="https://api.example.com/posts"),
])
完整字段说明详见 :doc:`/api`
+164
View File
@@ -0,0 +1,164 @@
YAML 任务编排
=============
PyFlowX 支持 GitHub Actions 风格的声明式 YAML 任务编排,从 YAML 文件直接加载任务图。
编程式 API
----------
.. code-block:: python
import pyflowx as px
# 从 YAML 文件加载任务图
graph = px.Graph.from_yaml("pipeline.yaml")
report = px.run(graph, strategy="thread")
# 或用函数式 API
graph = px.load_yaml("pipeline.yaml")
# 从字符串解析
graph = px.parse_yaml_string("""
jobs:
hello:
cmd: ["echo", "hello"]
""")
YAML Schema
-----------
.. code-block:: yaml
strategy: thread # 图级默认策略
defaults: # 图级默认值
retry: {max_attempts: 3}
verbose: true
env: {CI: "true"}
variables: # 变量定义 (可在 cmd/env 中 ${VAR} 引用)
OUTPUT: "dist"
jobs:
setup:
cmd: ["git", "clone", "..."]
runs-on: linux
build:
needs: [setup] # 依赖列表
cmd: ["python", "-m", "build"]
timeout: 300
retry: {max_attempts: 2, delay: 1.0}
test:
needs: [build]
cmd: ["python${{ matrix.version }}", "-m", "pytest"]
strategy:
matrix: # 笛卡尔积展开为 6 个任务
version: ["3.8", "3.9", "3.10"]
os: ["linux", "macos"]
if: "env.CI" # 条件: 环境变量存在
lint:
needs: [build]
cmd: ["ruff", "check"]
if: "env.CI == 'true'"
deploy:
needs: [test, lint] # 矩阵依赖自动展开
cmd: ["twine", "upload"]
if: "env.DEPLOY_TOKEN != ''"
allow-upstream-skip: true
concurrency-key: deploy_lock
字段映射
--------
.. list-table::
:header-rows: 1
:widths: 30 30 40
* - YAML 字段
- TaskSpec 字段
- 说明
* - ``jobs.<id>``
- ``name``
- job ID 作为任务名
* - ``cmd`` / ``run``
- ``cmd``
- ``cmd`` 为列表形式,``run`` 为 shell 字符串
* - ``needs``
- ``depends_on``
- 依赖列表(矩阵任务自动展开)
* - ``if``
- ``conditions``
- ``success()`` / ``always()`` / ``env.VAR`` / ``env.VAR == 'x'``
* - ``strategy.matrix``
- 矩阵扇出
- 笛卡尔积展开为多个任务
* - ``${{ matrix.key }}``
- 占位符
- 在 cmd/run/cwd/env 中替换
* - ``timeout``
- ``timeout``
- 超时秒数
* - ``retry``
- ``retry``
- ``{max_attempts, delay, backoff, jitter}``
* - ``cwd``
- ``cwd``
- 工作目录
* - ``env``
- ``env``
- 环境变量
* - ``verbose``
- ``verbose``
- 详细输出
* - ``continue-on-error``
- ``continue_on_error``
- 失败不中止整图
* - ``skip-if-missing``
- ``skip_if_missing``
- 命令不存在时跳过
* - ``allow-upstream-skip``
- ``allow_upstream_skip``
- 上游跳过时仍执行
* - ``priority``
- ``priority``
- 同层优先级
* - ``concurrency-key``
- ``concurrency_key``
- 并发限制键
* - ``tags``
- ``tags``
- 自由标签
* - ``runs-on``
- ``tags``(追加)
- 运行环境标签
CLI 配置段(``cli:``
----------------------
工具 YAML 还可定义 ``cli:`` 段,声明命令行参数 schema,由 ``pf`` 自动解析:
.. code-block:: yaml
cli:
description: "FileDate - 文件日期处理工具"
usage: "pf filedate <command> [files...]"
subcommands:
add:
help: "添加日期前缀"
positional:
- name: FILES
nargs: "+"
type: path
help: "文件路径"
options:
- name: CLEAR
flag: "--clear"
action: store_true
help: "清除已有日期前缀"
支持的 ``type````str`` / ``int`` / ``float`` / ``path``
完整 API 说明详见 :doc:`/api`
+56
View File
@@ -0,0 +1,56 @@
PyFlowX 文档
============
PyFlowX 是一个轻量、类型安全的 DAG 任务调度器:**参数名就是依赖声明**
无需装饰器、无需样板包装器,写一个普通函数,框架按参数名自动注入上游结果。
特性
----
- **零样板** —— 参数名即依赖,框架自动注入上游结果
- **四种执行策略** —— sequential(串行)、thread(线程池)、async(事件循环)、dependency(依赖驱动,最大化并行)
- **类型安全** —— ``TaskSpec[T]`` 把返回类型一路传到 ``RunReport``
- **DAG 校验** —— 构建时即时校验重名、缺失依赖、环
- **自动分层** —— Kahn 算法分组,同层任务可并行
- **重试与超时** —— 每个任务独立配置 ``RetryPolicy````timeout``
- **并发限制** —— ``concurrency_key`` + ``concurrency_limits`` 按组限流
- **断点续跑** —— ``MemoryBackend`` / ``JSONBackend``,成功结果可缓存复用
- **命令任务** —— ``cmd`` 参数直接执行外部命令
- **条件执行** —— ``conditions`` 按平台、环境变量等条件跳过任务
- **YAML 任务编排** —— GitHub Actions 风格声明式任务图
- **pf 统一 CLI** —— ``pf <tool> [command]`` 调用所有工具
- **最小依赖** —— 仅依赖标准库 + PyYAML
文档导航
--------
.. toctree::
:maxdepth: 2
:caption: 入门
installation
quickstart
.. toctree::
:maxdepth: 2
:caption: 用户指南
guide/task
guide/graph
guide/execution
guide/yaml
guide/cli
.. toctree::
:maxdepth: 2
:caption: 参考
api
changelog
索引
----
* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`
+51
View File
@@ -0,0 +1,51 @@
安装
====
PyFlowX 支持 Python 3.8+,仅依赖标准库与 PyYAML(3.8 额外需要 ``graphlib_backport````typing-extensions``)。
pip 安装
--------
.. code-block:: bash
pip install pyflowx
uv 安装
-------
推荐使用 `uv <https://docs.astral.sh/uv/>`_
.. code-block:: bash
uv add pyflowx
可选依赖
--------
``office`` —— PDF/图片处理(pdftool、screenshot 等工具需要):
.. code-block:: bash
pip install pyflowx[office]
``dev`` —— 开发工具链(ruff、pyrefly、pytest、tox 等):
.. code-block:: bash
pip install pyflowx[dev]
验证安装
--------
.. code-block:: bash
pf --version
输出示例::
PyFlowX 0.4.5
下一步
------
前往 :doc:`quickstart` 开始使用。
+87
View File
@@ -0,0 +1,87 @@
快速上手
========
核心思想:**参数名即依赖**。写一个普通函数,参数名匹配上游任务名,框架自动注入结果。
最小示例
--------
.. code-block:: python
import pyflowx as px
def extract() -> list[int]:
return [1, 2, 3]
# 参数名 extract 自动匹配上游任务名 → 自动注入
def double(extract: list[int]) -> list[int]:
return [x * 2 for x in extract]
graph = px.Graph.from_specs([
px.TaskSpec("extract", extract),
px.TaskSpec("double", double, ("extract",)),
])
report = px.run(graph, strategy="sequential")
print(report["double"]) # [2, 4, 6]
三种任务形态
------------
1. **函数任务**``fn``):普通 Python 函数,参数名驱动自动注入
2. **命令任务**``cmd``):执行外部命令,支持 ``list[str]`` / ``str``shell/ ``Callable``
3. **YAML 声明式**:从 YAML 文件加载任务图
.. code-block:: python
graph = px.Graph.from_specs([
px.TaskSpec("list", cmd=["ls", "-la"]),
px.TaskSpec("greet", fn=lambda: "hello"),
])
执行策略
--------
PyFlowX 提供四种执行策略:
.. list-table::
:header-rows: 1
:widths: 20 20 60
* - 策略
- 并发模型
- 适用场景
* - ``sequential``
- 串行
- 调试、CPU 密集
* - ``thread``
- 线程池
- I/O 密集同步
* - ``async``
- 事件循环
- I/O 密集异步
* - ``dependency``
- 依赖驱动
- 最大化并行度(默认推荐)
.. code-block:: python
report = px.run(graph, strategy="dependency")
结果访问
--------
.. code-block:: python
report["task_name"] # 任务返回值
report.result_of("task_name") # 完整 TaskResult
report.success # 整体是否成功
report.summary() # 统计字典
report.failed_tasks() # 失败任务名列表
下一步
------
- :doc:`guide/task` —— TaskSpec 详细配置
- :doc:`guide/yaml` —— YAML 声明式任务编排
- :doc:`guide/cli` —— ``pf`` 统一 CLI 入口
+10 -31
View File
@@ -13,7 +13,7 @@ classifiers = [
]
dependencies = [
"graphlib_backport >= 1.0.0; python_version < '3.9'",
"typing-extensions>=4.13.2; python_version < '3.10'",
"typing-extensions>=4.13.2; python_version < '3.13'",
]
description = "Lightweight, type-safe DAG task scheduler with multi-strategy execution."
keywords = ["async", "dag", "scheduler", "task", "workflow"]
@@ -21,35 +21,12 @@ license = { text = "MIT" }
name = "pyflowx"
readme = "README.md"
requires-python = ">=3.8"
version = "0.3.1"
version = "0.4.8"
[project.scripts]
autofmt = "pyflowx.cli.autofmt:main"
bumpversion = "pyflowx.cli.bumpversion:main"
dockercmd = "pyflowx.cli.dev.dockercmd:main"
emlman = "pyflowx.cli.emlmanager:main"
filedate = "pyflowx.cli.filedate:main"
filelvl = "pyflowx.cli.filelevel:main"
foldback = "pyflowx.cli.folderback:main"
foldzip = "pyflowx.cli.folderzip:main"
gitt = "pyflowx.cli.gittool:main"
lscalc = "pyflowx.cli.lscalc:main"
msdown = "pyflowx.cli.llm.msdownload:main"
packtool = "pyflowx.cli.packtool:main"
pdftool = "pyflowx.cli.pdftool:main"
piptool = "pyflowx.cli.piptool:main"
pxp = "pyflowx.cli.profiler:main"
pymake = "pyflowx.cli.pymake:main"
reseticon = "pyflowx.cli.reseticoncache:main"
scrcap = "pyflowx.cli.screenshot:main"
sglang = "pyflowx.cli.llm.sglang:main"
sshcopy = "pyflowx.cli.sshcopyid:main"
# dev
envdev = "pyflowx.cli.dev.envdev:main"
# system
clr = "pyflowx.cli.system.clearscreen:main"
taskk = "pyflowx.cli.system.taskkill:main"
wch = "pyflowx.cli.system.which:main"
emlman = "pyflowx.cli.emlmanager:main"
pf = "pyflowx.cli.pf:main"
pxp = "pyflowx.cli.profiler:main"
[project.optional-dependencies]
dev = [
@@ -67,6 +44,7 @@ dev = [
"tox-uv>=1.13.1",
"tox>=4.25.0",
]
docs = ["myst-parser>=3.0", "sphinx-rtd-theme>=2.0", "sphinx>=7.0"]
office = [
"pillow>=10.4.0",
"pymupdf>=1.24.11",
@@ -95,12 +73,12 @@ packages = ["src/pyflowx"]
pyflowx = { workspace = true }
[dependency-groups]
dev = ["pyflowx[dev,office]"]
dev = ["pyflowx[dev,docs,office]"]
[tool.coverage.run]
branch = true
concurrency = ["thread"]
omit = ["src/pyflowx/cli/*", "src/pyflowx/examples/*", "tests/*"]
omit = ["src/pyflowx/cli/*", "tests/*"]
source = ["pyflowx"]
[tool.coverage.report]
@@ -153,7 +131,8 @@ select = [
]
[tool.ruff.lint.per-file-ignores]
"**/tests/**" = ["ARG001", "ARG002"]
"**/tests/**" = ["ARG001", "ARG002"]
"src/pyflowx/tools.py" = ["PLR0911"]
[tool.pyrefly]
preset = "strict"
+8 -10
View File
@@ -60,15 +60,7 @@ from __future__ import annotations
from .command import run_command
from .compose import GraphComposer, compose
from .conditions import (
IS_LINUX,
IS_MACOS,
IS_POSIX,
IS_WINDOWS,
BuiltinConditions,
Condition,
Constants,
)
from .conditions import IS_LINUX, IS_MACOS, IS_POSIX, IS_WINDOWS, BuiltinConditions, Condition, Constants
from .context import Context, build_call_args, describe_injection
from .errors import (
CycleError,
@@ -99,8 +91,9 @@ from .task import (
task,
task_template,
)
from .tools import ToolSpec, list_subcommands, list_tools, run_tool, tool
__version__ = "0.4.1"
__version__ = "0.4.8"
__all__ = [
"IS_LINUX",
@@ -139,12 +132,17 @@ __all__ = [
"TaskSpec",
"TaskStatus",
"TaskTimeoutError",
"ToolSpec",
"build_call_args",
"cmd",
"compose",
"describe_injection",
"list_subcommands",
"list_tools",
"run",
"run_command",
"run_tool",
"task",
"task_template",
"tool",
]
-263
View File
@@ -1,263 +0,0 @@
"""版本号自动管理工具.
使用 TaskSpec 模式实现, 支持语义化版本管理和多文件格式的版本号更新.
"""
from __future__ import annotations
import argparse
import re
from pathlib import Path
from typing import Literal, get_args
import pyflowx as px
BumpVersionType = Literal["patch", "minor", "major"]
# 针对不同文件类型的版本号匹配模式
# pyproject.toml: version = "X.Y.Z" 或 version = 'X.Y.Z'
_PYPROJECT_VERSION_PATTERN = re.compile(
r'(?:^|\n)\s*version\s*=\s*["\']'
r"(?P<major>0|[1-9]\d*)\.(?P<minor>0|[1-9]\d*)\.(?P<patch>0|[1-9]\d*)"
r"(?:-(?P<prerelease>(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)(?:\.(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*))?"
r"(?:\+(?P<buildmetadata>[0-9a-zA-Z-]+(?:\.[0-9a-zA-Z-]+)*))?"
r'["\']',
re.MULTILINE,
)
# __init__.py: __version__ = "X.Y.Z" 或 __version__ = 'X.Y.Z'
_INIT_VERSION_PATTERN = re.compile(
r'(?:^|\n)\s*__version__\s*=\s*["\']'
r"(?P<major>0|[1-9]\d*)\.(?P<minor>0|[1-9]\d*)\.(?P<patch>0|[1-9]\d*)"
r"(?:-(?P<prerelease>(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)(?:\.(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*))?"
r"(?:\+(?P<buildmetadata>[0-9a-zA-Z-]+(?:\.[0-9a-zA-Z-]+)*))?"
r'["\']',
re.MULTILINE,
)
def _get_pattern_for_file(file_name: str) -> re.Pattern[str] | None:
"""根据文件类型获取对应的正则表达式.
Parameters
----------
file_name : str
文件名
Returns
-------
re.Pattern[str] | None
对应的正则表达式,如果无法确定则返回 None
"""
if file_name == "pyproject.toml":
return _PYPROJECT_VERSION_PATTERN
if file_name == "__init__.py":
return _INIT_VERSION_PATTERN
return None
def _calculate_new_version(major: int, minor: int, patch: int, part: BumpVersionType) -> str:
"""计算新版本号.
Parameters
----------
major : int
当前主版本号
minor : int
当前次版本号
patch : int
当前补丁版本号
part : BumpVersionType
要更新的部分
Returns
-------
str
新版本号
"""
if part == "major":
return f"{major + 1}.0.0"
if part == "minor":
return f"{major}.{minor + 1}.0"
return f"{major}.{minor}.{patch + 1}"
def _build_replacement_string(original_match: str, new_version: str, file_name: str) -> str:
"""构建替换字符串,保留原始格式.
Parameters
----------
original_match : str
原始匹配的字符串
new_version : str
新版本号
file_name : str
文件名
Returns
-------
str
替换字符串
"""
quote_char = '"' if '"' in original_match else "'"
if file_name == "pyproject.toml":
prefix_match = re.match(r'(\s*version\s*=\s*)["\']', original_match)
prefix = prefix_match.group(1) if prefix_match else "version = "
return f"{prefix}{quote_char}{new_version}{quote_char}"
if file_name == "__init__.py":
prefix_match = re.match(r'(\s*__version__\s*=\s*)["\']', original_match)
prefix = prefix_match.group(1) if prefix_match else "__version__ = "
return f"{prefix}{quote_char}{new_version}{quote_char}"
return new_version
def bump_file_version(file_path: Path, part: BumpVersionType = "patch") -> str | None:
"""更新文件中的版本号.
Parameters
----------
file_path : Path
要更新的文件路径
part : BumpVersionType
版本部分: patch, minor, major
Returns
-------
str | None
更新后的新版本号,如果文件中未找到版本号则返回 None
"""
try:
content = file_path.read_text(encoding="utf-8")
except Exception as e:
print(f"读取文件 {file_path} 时出错: {e}")
raise
# 获取文件对应的正则表达式
pattern = _get_pattern_for_file(file_path.name)
# 对于未知文件类型,尝试两种模式
if pattern:
match = pattern.search(content)
else:
match = _PYPROJECT_VERSION_PATTERN.search(content) or _INIT_VERSION_PATTERN.search(content)
if not match:
print(f"文件 {file_path} 中未找到版本号模式")
return None
# 提取当前版本号
major = int(match.group("major"))
minor = int(match.group("minor"))
patch = int(match.group("patch"))
# 计算新版本号
new_version = _calculate_new_version(major, minor, patch, part)
# 构建替换字符串
original_match = match.group(0)
replacement = _build_replacement_string(original_match, new_version, file_path.name)
# 更新文件内容
content = content.replace(original_match, replacement)
try:
file_path.write_text(content, encoding="utf-8")
except Exception as e:
print(f"更新文件 {file_path} 版本号时出错: {e}")
raise
return new_version
def main() -> None:
"""版本号管理工具主函数."""
parser = argparse.ArgumentParser(description="BumpVersion - 版本号自动管理工具")
parser.add_argument(
"part",
type=str,
nargs="?",
default="patch",
choices=get_args(BumpVersionType),
help=f"版本部分: {get_args(BumpVersionType)}",
)
parser.add_argument(
"--no-tag",
action="store_true",
help="提交后不创建 git tag",
)
args = parser.parse_args()
part = args.part
# 搜索文件,排除常见的虚拟环境和缓存目录
ignore_dirs = {".venv", "venv", ".git", "__pycache__", ".tox", "node_modules", "build", "dist", ".eggs"}
all_files = set()
for pattern in ["__init__.py", "pyproject.toml"]:
for file in Path.cwd().rglob(pattern):
# 检查路径中是否包含需要忽略的目录
if not any(ignore_dir in file.parts for ignore_dir in ignore_dirs):
all_files.add(file)
if not all_files:
print("未找到包含版本号的文件")
return
print(f"找到 {len(all_files)} 个文件需要更新版本号")
for file in sorted(all_files):
print(f" - {file.relative_to(Path.cwd())}")
# 更新所有文件的版本号(使用顺序执行避免竞争条件)
# 使用相对于 cwd 的路径作为任务名,确保唯一性
graph = px.Graph.from_specs([
px.TaskSpec(
f"bump_{file.relative_to(Path.cwd())}".replace("\\", "_").replace("/", "_").replace(".", "_"),
fn=bump_file_version,
args=(file, part),
)
for file in all_files
])
report = px.run(graph, strategy="sequential")
# 收集新版本号(取第一个成功的结果)
new_version = None
for task_name in report:
result = report[task_name]
if result is not None:
new_version = result
break
if not new_version:
print("未能获取新版本号")
return
print(f"版本号已更新为: {new_version}")
# 提交修改并创建标签
tasks = [
px.TaskSpec("git_add", cmd=["git", "add", "."]),
px.TaskSpec(
"git_commit",
cmd=["git", "commit", "-m", f"bump version to {new_version}"],
depends_on=("git_add",),
),
]
if not args.no_tag:
tag_name = f"v{new_version}"
tasks.append(
px.TaskSpec(
"git_tag",
cmd=["git", "tag", "-a", tag_name, "-m", f"Release {tag_name}"],
depends_on=("git_commit",),
)
)
graph = px.Graph.from_specs(tasks)
px.run(graph, strategy="sequential")
if not args.no_tag:
print(f"已创建标签: v{new_version}")
-26
View File
@@ -1,26 +0,0 @@
from __future__ import annotations
from typing import Literal
import pyflowx as px
DockerMirrorType = Literal["tencent"]
DOCKER_MIRROR_URLS: dict[DockerMirrorType, str] = {"tencent": "ccr.ccs.tencentyun.com"}
def main():
# parser = argparse.ArgumentParser(description="Docker 命令行工具")
# parser.add_argument("--username", nargs="?", default="", type=str, help="Docker 用户名")
# args = parser.parse_args()
tasks: list[px.TaskSpec] = [
px.cmd(["docker", "login", "--username", "xxx", DOCKER_MIRROR_URLS["tencent"]], name="docker_login_tencent"),
]
alias: dict[str, str | list[str | px.TaskSpec] | px.TaskSpec | px.Graph] = {
"login": "docker_login_tencent",
}
runner = px.CliRunner(strategy="sequential", tasks=tasks, aliases=alias)
runner.run_cli()
-357
View File
@@ -1,357 +0,0 @@
from __future__ import annotations
import argparse
import getpass
from pathlib import Path
from typing import Literal, get_args
import pyflowx as px
from pyflowx.conditions import BuiltinConditions
from pyflowx.tasks.system import setenv_group, write_file
# ============================================================================
# Mirror 配置
# ============================================================================
DOWNLOAD_MIRROR_SCRIPT: str = "curl -sSL https://linuxmirrors.cn/main.sh -o /tmp/linuxmirrors.sh"
INSTALL_MIRROR_SCRIPT: str = "sudo bash /tmp/linuxmirrors.sh"
# ============================================================================
# Python 配置
# ============================================================================
PyMirrorType = Literal["tsinghua", "aliyun", "huaweicloud", "ustc", "zju"]
PIP_INDEX_URLS: dict[PyMirrorType, str] = {
"tsinghua": "https://pypi.tuna.tsinghua.edu.cn/simple",
"aliyun": "https://mirrors.aliyun.com/pypi/simple/",
"huaweicloud": "https://mirrors.huaweicloud.com/repository/pypi/simple/",
"ustc": "https://pypi.mirrors.ustc.edu.cn/simple/",
"zju": "https://mirrors.zju.edu.cn/pypi/simple/",
}
PIP_TRUSTED_HOSTS: dict[PyMirrorType, str] = {
"tsinghua": "pypi.tuna.tsinghua.edu.cn",
"aliyun": "mirrors.aliyun.com",
"huaweicloud": "mirrors.huaweicloud.com",
"ustc": "pypi.mirrors.ustc.edu.cn",
"zju": "mirrors.zju.edu.cn",
}
PIP_CONFIG_PATH = Path.home() / ".pip" / "pip.conf" if BuiltinConditions.IS_LINUX() else Path.home() / "pip" / "pip.ini"
UV_INDEX_URLS = PIP_INDEX_URLS
UV_PYTHON_INSTALL_MIRROR: str = "https://registry.npmmirror.com/-/binary/python-build-standalone"
# ============================================================================
# Conda 配置
# ============================================================================
CondaMirrorType = Literal["tsinghua", "ustc", "bsfu", "aliyun"]
CONDA_MIRROR_URLS: dict[CondaMirrorType, list[str]] = {
"tsinghua": [
"https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/",
"https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free/",
"https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/r/",
"https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/msys2/",
"https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/pro/",
"https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/",
"https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/bioconda/",
"https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/menpo/",
"https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/pytorch/",
],
"ustc": [
"https://mirrors.ustc.edu.cn/anaconda/pkgs/main/",
"https://mirrors.ustc.edu.cn/anaconda/pkgs/free/",
"https://mirrors.ustc.edu.cn/anaconda/pkgs/r/",
"https://mirrors.ustc.edu.cn/anaconda/pkgs/msys2/",
"https://mirrors.ustc.edu.cn/anaconda/pkgs/pro/",
"https://mirrors.ustc.edu.cn/anaconda/pkgs/dev/",
"https://mirrors.ustc.edu.cn/anaconda/cloud/conda-forge/",
"https://mirrors.ustc.edu.cn/anaconda/cloud/bioconda/",
"https://mirrors.ustc.edu.cn/anaconda/cloud/menpo/",
"https://mirrors.ustc.edu.cn/anaconda/cloud/pytorch/",
],
"bsfu": [
"https://mirrors.bsfu.edu.cn/anaconda/pkgs/main/",
"https://mirrors.bsfu.edu.cn/anaconda/pkgs/free/",
"https://mirrors.bsfu.edu.cn/anaconda/pkgs/r/",
"https://mirrors.bsfu.edu.cn/anaconda/pkgs/msys2/",
"https://mirrors.bsfu.edu.cn/anaconda/pkgs/pro/",
"https://mirrors.bsfu.edu.cn/anaconda/pkgs/dev/",
"https://mirrors.bsfu.edu.cn/anaconda/cloud/conda-forge/",
"https://mirrors.bsfu.edu.cn/anaconda/cloud/bioconda/",
"https://mirrors.bsfu.edu.cn/anaconda/cloud/menpo/",
"https://mirrors.bsfu.edu.cn/anaconda/cloud/pytorch/",
],
"aliyun": [
"https://mirrors.aliyun.com/anaconda/pkgs/main/",
"https://mirrors.aliyun.com/anaconda/pkgs/free/",
"https://mirrors.aliyun.com/anaconda/pkgs/r/",
"https://mirrors.aliyun.com/anaconda/pkgs/msys2/",
"https://mirrors.aliyun.com/anaconda/pkgs/pro/",
"https://mirrors.aliyun.com/anaconda/pkgs/dev/",
"https://mirrors.aliyun.com/anaconda/cloud/conda-forge/",
"https://mirrors.aliyun.com/anaconda/cloud/bioconda/",
"https://mirrors.aliyun.com/anaconda/cloud/menpo/",
"https://mirrors.aliyun.com/anaconda/cloud/pytorch/",
],
}
CONDA_CONFIG_PATH = Path.home() / ".condarc"
# ============================================================================
# Qt 配置
# ============================================================================
QT_LIBS: list[str] = [
"build-essential",
"libgl1",
"libegl1",
"libglib2.0-0",
"libfontconfig1",
"libfreetype6",
"libxkbcommon0",
"libdbus-1-3",
"libxcb-xinerama0",
"libxcb-icccm4",
"libxcb-image0",
"libxcb-keysyms1",
"libxcb-randr0",
"libxcb-render-util0",
"libxcb-shape0",
"libxcb-xfixes0",
"libxcb-cursor0",
]
CHINESE_FONTS: list[str] = [
"fonts-noto-cjk",
"fonts-wqy-microhei",
"fonts-wqy-zenhei",
"fonts-noto-color-emoji",
]
# ============================================================================
# Rust 配置
# ============================================================================
RustMirrorType = Literal["tsinghua", "ustc", "aliyun"]
RustVersionType = Literal["stable", "nightly", "beta"]
DEFAULT_RUST_VERSION: RustVersionType = "stable"
DEFAULT_MIRROR: RustMirrorType = "tsinghua"
RUSTUP_MIRRORS: dict[RustMirrorType, dict[str, str]] = {
"tsinghua": {
"RUSTUP_DIST_SERVER": "https://mirrors.tuna.tsinghua.edu.cn/rustup",
"RUSTUP_UPDATE_ROOT": "https://mirrors.tuna.tsinghua.edu.cn/rustup/rustup",
"TOML_REGISTRY": "https://mirrors.tuna.tsinghua.edu.cn/crates.io-index/",
},
"aliyun": {
"RUSTUP_DIST_SERVER": "https://mirrors.aliyun.com/rustup",
"RUSTUP_UPDATE_ROOT": "https://mirrors.aliyun.com/rustup/rustup",
"TOML_REGISTRY": "https://mirrors.aliyun.com/crates.io-index/",
},
"ustc": {
"RUSTUP_DIST_SERVER": "https://mirrors.ustc.edu.cn/rust-static",
"RUSTUP_UPDATE_ROOT": "https://mirrors.ustc.edu.cn/rust-static/rustup",
"TOML_REGISTRY": "https://mirrors.ustc.edu.cn/crates.io-index/",
},
}
RUSTUP_DOWNLOAD_URL_LINUX = "https://mirrors.aliyun.com/repo/rust/rustup-init.sh"
RUSTUP_DOWNLOAD_URL_WINDOWS = "https://static.rust-lang.org/rustup/dist/x86_64-pc-windows-msvc/rustup-init.exe"
RUST_CONFIG_PATH = Path.home() / ".cargo" / "config.toml"
RUST_SCCACHE_DIR: Path = Path.home() / ".cargo" / "sccache"
RUST_SCCACHE_CACHE_SIZE: str = "20G"
def main() -> None:
"""主函数."""
parser = argparse.ArgumentParser(description="环境开发工具")
parser.add_argument(
"--python-mirror",
nargs="?",
type=str,
default="tsinghua",
choices=get_args(PyMirrorType),
help="Python 镜像源",
)
parser.add_argument(
"--conda-mirror",
nargs="?",
type=str,
default="tsinghua",
choices=get_args(CondaMirrorType),
help="Conda 镜镜像源",
)
parser.add_argument(
"--rust-mirror",
nargs="?",
type=str,
default=DEFAULT_MIRROR,
choices=get_args(RustMirrorType),
help="Rust 镜像源",
)
parser.add_argument(
"--rust-version",
nargs="?",
type=str,
default=DEFAULT_RUST_VERSION,
choices=get_args(RustVersionType),
help=f"Rust 版本, 推荐: {get_args(RustVersionType)}",
)
args = parser.parse_args()
python_mirror = args.python_mirror
conda_mirror_urls = CONDA_MIRROR_URLS[args.conda_mirror]
rust_mirror = args.rust_mirror
rust_version = args.rust_version
# 确保配置文件目录存在
PIP_CONFIG_PATH.parent.mkdir(parents=True, exist_ok=True)
CONDA_CONFIG_PATH.parent.mkdir(parents=True, exist_ok=True)
RUST_CONFIG_PATH.parent.mkdir(parents=True, exist_ok=True)
RUST_SCCACHE_DIR.mkdir(parents=True, exist_ok=True)
# 使用 conditions 自动控制任务执行
graph = px.Graph.from_specs([
# 系统镜像配置(仅 Linux 且未配置国内镜像)
px.TaskSpec(
"download_mirror",
cmd=DOWNLOAD_MIRROR_SCRIPT,
conditions=(
BuiltinConditions.IS_LINUX(),
BuiltinConditions.NOT(
BuiltinConditions.OR(
*[
BuiltinConditions.FILE_CONTENT_EXISTS(f, m)
for f in [
"/etc/apt/sources.list",
"/etc/apt/sources.list.d/ubuntu.sources",
]
for m in get_args(PyMirrorType)
],
)
),
),
verbose=True,
),
px.TaskSpec(
"install_mirror",
cmd=INSTALL_MIRROR_SCRIPT,
depends_on=("download_mirror",),
verbose=True,
),
# 安装 Qt 依赖(仅 Linux
px.TaskSpec(
"install_qt_libs",
cmd=["sudo", "apt", "install", "-y", *QT_LIBS],
conditions=(BuiltinConditions.IS_LINUX(),),
depends_on=("install_mirror",),
allow_upstream_skip=True,
verbose=True,
),
# 安装中文字体(仅 Linux
px.TaskSpec(
"install_fonts",
cmd=["sudo", "apt", "install", "-y", *CHINESE_FONTS],
conditions=(BuiltinConditions.IS_LINUX(),),
depends_on=("install_mirror",),
allow_upstream_skip=True,
verbose=True,
),
# 安装 Docker
px.TaskSpec(
"install_docker",
cmd=["sudo", "apt", "install", "-y", "docker-compose-v2"],
conditions=(BuiltinConditions.IS_LINUX(),),
depends_on=("install_mirror",),
allow_upstream_skip=True,
verbose=True,
),
px.TaskSpec(
"add_docker_group",
cmd=["sudo", "usermod", "-aG", "docker", getpass.getuser()],
conditions=(BuiltinConditions.IS_LINUX(),),
depends_on=("install_docker",),
allow_upstream_skip=True,
verbose=True,
),
px.TaskSpec(
"refresh_docker_group",
cmd=["newgrp", "docker"],
conditions=(BuiltinConditions.IS_LINUX(),),
depends_on=("add_docker_group",),
allow_upstream_skip=True,
verbose=True,
),
# 设置 Python 环境变量
*setenv_group({
"PIP_INDEX_URL": PIP_INDEX_URLS[python_mirror],
"PIP_TRUSTED_HOSTS": PIP_TRUSTED_HOSTS[python_mirror],
"UV_INDEX_URL": UV_INDEX_URLS[python_mirror],
"UV_PYTHON_INSTALL_MIRROR": UV_PYTHON_INSTALL_MIRROR,
"UV_HTTP_TIMEOUT": "600",
"UV_LINK_MODE": "copy",
}),
# 写入 Python 配置(仅当未配置)
write_file(
str(PIP_CONFIG_PATH),
f"[global]\nindex-url = {PIP_INDEX_URLS[python_mirror]}\ntrusted-host = {PIP_TRUSTED_HOSTS[python_mirror]}",
),
# 写入 Conda 配置(仅当未配置)
write_file(
str(CONDA_CONFIG_PATH),
"show_channel_urls: true\nchannels:\n - " + "\n - ".join(conda_mirror_urls) + "\n - defaults",
),
# 设置 Rust 镜像源
*setenv_group({
"RUSTUP_DIST_SERVER": RUSTUP_MIRRORS[rust_mirror]["RUSTUP_DIST_SERVER"],
"RUSTUP_UPDATE_ROOT": RUSTUP_MIRRORS[rust_mirror]["RUSTUP_UPDATE_ROOT"],
"RUST_SCCACHE_DIR": str(RUST_SCCACHE_DIR),
"RUST_SCCACHE_CACHE_SIZE": RUST_SCCACHE_CACHE_SIZE,
}),
# 写入 Rust 配置(仅当未配置)
write_file(
str(RUST_CONFIG_PATH),
f"""
[source.crates-io]
replace-with = '{rust_mirror}'
[source.{rust_mirror}]
registry = "sparse+{RUSTUP_MIRRORS[rust_mirror]["TOML_REGISTRY"]}"
[registries.{rust_mirror}]
index = "sparse+{RUSTUP_MIRRORS[rust_mirror]["TOML_REGISTRY"]}"
""",
),
# 下载 Rustup 安装脚本
px.TaskSpec(
"download_rustup",
cmd=["curl", "-fsSL", RUSTUP_DOWNLOAD_URL_LINUX, "-o", "rustup-init.sh"],
conditions=(BuiltinConditions.IS_LINUX(), BuiltinConditions.NOT(BuiltinConditions.HAS_INSTALLED("rustup"))),
verbose=True,
),
px.TaskSpec(
"download_rustup_win",
cmd=[
"powershell",
"-Command",
"Invoke-WebRequest",
"-Uri",
RUSTUP_DOWNLOAD_URL_WINDOWS,
"-OutFile",
"rustup-init.exe",
],
conditions=(
BuiltinConditions.IS_WINDOWS(),
BuiltinConditions.NOT(BuiltinConditions.HAS_INSTALLED("rustup")),
),
verbose=True,
),
# 安装 Rust 工具链
px.TaskSpec(
"install_rust",
cmd=["rustup", "toolchain", "install", rust_version],
conditions=(BuiltinConditions.HAS_INSTALLED("rustup"),),
depends_on=("setenv_rustup_dist_server",),
allow_upstream_skip=True,
verbose=True,
),
])
px.run(graph, strategy="thread", verbose=True)
+9 -7
View File
@@ -567,13 +567,15 @@ class EmlManagerHandler(BaseHTTPRequestHandler):
emails = self.db.search_emails(keyword, field, limit, offset)
total_count = self.db.get_email_count()
self._send_json_response({
"emails": emails,
"count": len(emails),
"total": total_count,
"limit": limit,
"offset": offset,
})
self._send_json_response(
{
"emails": emails,
"count": len(emails),
"total": total_count,
"limit": limit,
"offset": offset,
}
)
def _api_get_email(self, query_params: dict[str, list[str]]) -> None:
"""API: 获取单个邮件详情."""
-107
View File
@@ -1,107 +0,0 @@
"""Git 工具模块.
提供 Git 仓库管理的常用操作封装,
支持初始化、提交、清理、推送等功能.
"""
from __future__ import annotations
from pathlib import Path
import pyflowx as px
EXCLUDE_DIRS = [
# 编辑器相关目录
".vscode",
".idea",
".editorconfig",
".trae",
".qoder",
# 项目相关目录
".venv",
".git",
".tox",
".pytest_cache",
"node_modules",
".ruff_cache",
]
EXCLUDE_CMDS = [arg for d in EXCLUDE_DIRS for arg in ["-e", d]]
def init_sub_dirs() -> None:
"""初始化子目录的Git仓库."""
sub_dirs = [subdir for subdir in Path.cwd().iterdir() if subdir.is_dir()]
for subdir in sub_dirs:
px.run(
px.Graph.from_specs([
px.TaskSpec(
"init",
cmd=["git", "init"],
conditions=(lambda _: not_has_git_repo(),),
cwd=subdir,
),
px.TaskSpec("add", cmd=["git", "add", "."], depends_on=("init",)),
px.TaskSpec("commit", cmd=["git", "commit", "-m", "init commit"], depends_on=("add",)),
]),
)
@px.task(name="isub")
def isub() -> None:
"""初始化子目录的Git仓库."""
init_sub_dirs()
push: px.TaskSpec = px.TaskSpec("push", cmd=["git", "push"])
pull: px.TaskSpec = px.TaskSpec("pull", cmd=["git", "pull"])
kill_tgit: px.TaskSpec = px.TaskSpec("task_kill", cmd=["taskkill", "/f", "/t", "/im", "tgitcache.exe"])
def not_has_git_repo() -> bool:
"""检查当前目录没有Git仓库."""
return not Path.cwd().exists() or not (Path.cwd() / ".git").is_dir()
def has_files() -> bool:
"""检查当前目录是否有文件."""
return bool(list(Path.cwd().glob("*")))
def main() -> None:
"""Git工具主函数."""
runner = px.CliRunner(
strategy="thread",
description="Gittool - Git 执行工具.",
aliases={
# 添加并提交
"a": px.Graph.from_specs([
px.TaskSpec("add", cmd=["git", "add", "."], conditions=(lambda _: has_files(),)),
px.TaskSpec("commit", cmd=["git", "commit", "-m", "chore: update"], depends_on=("add",)),
]),
# 清理(chain: clean → status
"c": px.Graph().chain(
px.TaskSpec("clean", cmd=["git", "clean", "-xfd", *EXCLUDE_CMDS]),
px.TaskSpec("status", cmd=["git", "status", "--porcelain"]),
),
# 初始化、添加并提交
"i": px.Graph.from_specs([
px.TaskSpec("init", cmd=["git", "init"], conditions=(lambda _: not_has_git_repo(),)),
px.TaskSpec("add", cmd=["git", "add", "."], depends_on=("init",), conditions=(lambda _: has_files(),)),
px.TaskSpec(
"commit",
cmd=["git", "commit", "-m", "init commit"],
depends_on=("add",),
conditions=(lambda _: has_files(),),
),
]),
# 初始化子目录
"isub": isub,
# 推送
"p": push,
# 拉取
"pl": pull,
# 重启TGit缓存
"r": kill_tgit,
},
)
runner.run_cli()
View File
-41
View File
@@ -1,41 +0,0 @@
"""Download from ModelScopeHub."""
import argparse
from pathlib import Path
from typing import Literal, get_args
import pyflowx as px
DownloadType = Literal["model", "dataset", "space"]
def main():
parser = argparse.ArgumentParser(description="Download a model from ModelScopeHub.")
parser.add_argument("name", help="Target name.")
parser.add_argument("--type", "-t", nargs="?", default="model", choices=get_args(DownloadType), help="Target type.")
parser.add_argument("--dir", default=None, help="Download directory.")
args = parser.parse_args()
if not args.name:
parser.error("name is required")
download_dir: Path = Path(args.dir) if args.dir else Path.home() / ".models" / args.name.split("/")[-1]
download_dir.mkdir(parents=True, exist_ok=True)
graph = px.Graph.from_specs([
px.TaskSpec(
name="download",
cmd=[
"uvx",
"modelscope",
"download",
f"--{args.type}",
args.name,
"--local_dir",
str(download_dir),
],
verbose=True,
),
])
px.run(graph, strategy="thread", verbose=True)
-63
View File
@@ -1,63 +0,0 @@
"""使用 SGLang 运行本地模型."""
import argparse
from pathlib import Path
import pyflowx as px
from pyflowx.conditions import BuiltinConditions, Constants
def main():
parser = argparse.ArgumentParser(description="启动 SGLang 服务")
parser.add_argument("--model", default="~/.models/Qwen2.5-Coder-32B-Instruct-AWQ", help="模型路径")
parser.add_argument("--port", type=int, default=8000, help="服务端口")
parser.add_argument("--ctx-len", type=int, default=28672, help="最大上下文长度")
parser.add_argument("--mem", type=float, default=0.75, help="显存占比 (0-1)")
parser.add_argument("--host", default="0.0.0.0", help="主机地址")
parser.add_argument("--log-level", default="info", help="日志级别")
args = parser.parse_args()
if not args.model:
parser.error("model is required")
model_dir = Path(args.model).expanduser()
if not model_dir.exists():
parser.error(f"Model directory {model_dir} does not exist.")
graph = px.Graph.from_specs([
px.TaskSpec(
name="download",
cmd=[
"uv",
"install",
"sglang[all]",
],
conditions=(BuiltinConditions.NOT(BuiltinConditions.HAS_INSTALLED("sglang")),),
verbose=True,
),
px.TaskSpec(
name="run",
cmd=[
"python" if Constants.IS_WINDOWS else "python3",
"-m",
"sglang.launch_server",
"--model-path",
str(model_dir),
"--host",
str(args.host),
"--port",
"8000",
"--mem-fraction-static",
str(args.mem),
"--context-length",
"32768",
"--tool-call-parser",
"qwen",
"--log-level",
str(args.log_level),
],
verbose=True,
),
])
px.run(graph, strategy="sequential", verbose=True)
-523
View File
@@ -1,523 +0,0 @@
"""PDF 工具模块.
提供 PDF 文件操作的常用功能封装,
支持合并、拆分、压缩、加密、水印、OCR等功能.
"""
from __future__ import annotations
import argparse
from pathlib import Path
import pyflowx as px
try:
import fitz # PyMuPDF
HAS_PYMUPDF = True
except ImportError:
HAS_PYMUPDF = False
try:
import pypdf
HAS_PYPDF = True
except ImportError:
HAS_PYPDF = False
# ============================================================================
# 配置
# ============================================================================
PDF_SUFFIX = ".pdf"
DEFAULT_QUALITY = 75
DEFAULT_PASSWORD = ""
# ============================================================================
# 辅助函数
# ============================================================================
def pdf_merge(input_paths: list[Path], output_path: Path) -> None:
"""合并多个 PDF 文件."""
if not HAS_PYPDF:
print("未安装 pypdf 库,请安装: pip install pypdf")
return
writer = pypdf.PdfWriter()
for input_path in input_paths:
if input_path.exists():
reader = pypdf.PdfReader(str(input_path))
for page in reader.pages:
writer.add_page(page)
output_path.parent.mkdir(parents=True, exist_ok=True)
with open(output_path, "wb") as f:
writer.write(f)
print(f"合并完成: {output_path}")
def pdf_split(input_path: Path, output_dir: Path) -> None:
"""拆分 PDF 文件为单页."""
if not HAS_PYPDF:
print("未安装 pypdf 库,请安装: pip install pypdf")
return
reader = pypdf.PdfReader(str(input_path))
output_dir.mkdir(parents=True, exist_ok=True)
for i, page in enumerate(reader.pages):
writer = pypdf.PdfWriter()
writer.add_page(page)
output_file = output_dir / f"{input_path.stem}_page_{i + 1}.pdf"
with open(output_file, "wb") as f:
writer.write(f)
print(f"拆分完成: {output_dir}")
def pdf_compress(input_path: Path, output_path: Path) -> None:
"""压缩 PDF 文件."""
if not HAS_PYMUPDF:
print("未安装 PyMuPDF 库,请安装: pip install PyMuPDF")
return
doc = fitz.open(str(input_path))
output_path.parent.mkdir(parents=True, exist_ok=True)
doc.save(str(output_path), garbage=4, deflate=True, clean=True)
doc.close()
original_size = input_path.stat().st_size
new_size = output_path.stat().st_size
ratio = (1 - new_size / original_size) * 100
print(f"压缩完成: {output_path} (缩小 {ratio:.1f}%)")
def pdf_encrypt(input_path: Path, output_path: Path, password: str) -> None:
"""加密 PDF 文件."""
if not HAS_PYPDF:
print("未安装 pypdf 库,请安装: pip install pypdf")
return
reader = pypdf.PdfReader(str(input_path))
writer = pypdf.PdfWriter()
for page in reader.pages:
writer.add_page(page)
writer.encrypt(user_password=password, owner_password=password, use_128bit=True)
output_path.parent.mkdir(parents=True, exist_ok=True)
with open(output_path, "wb") as f:
writer.write(f)
print(f"加密完成: {output_path}")
def pdf_decrypt(input_path: Path, output_path: Path, password: str) -> None:
"""解密 PDF 文件."""
if not HAS_PYPDF:
print("未安装 pypdf 库,请安装: pip install pypdf")
return
reader = pypdf.PdfReader(str(input_path))
if reader.is_encrypted:
reader.decrypt(password)
writer = pypdf.PdfWriter()
for page in reader.pages:
writer.add_page(page)
output_path.parent.mkdir(parents=True, exist_ok=True)
with open(output_path, "wb") as f:
writer.write(f)
print(f"解密完成: {output_path}")
def pdf_extract_text(input_path: Path, output_path: Path) -> None:
"""提取 PDF 文本."""
if not HAS_PYMUPDF:
print("未安装 PyMuPDF 库,请安装: pip install PyMuPDF")
return
doc = fitz.open(str(input_path))
text = ""
for page in doc:
text += str(page.get_text()) + "\n\n"
doc.close()
output_path.parent.mkdir(parents=True, exist_ok=True)
output_path.write_text(text, encoding="utf-8")
print(f"文本提取完成: {output_path}")
def pdf_extract_images(input_path: Path, output_dir: Path) -> None:
"""提取 PDF 图片."""
if not HAS_PYMUPDF:
print("未安装 PyMuPDF 库,请安装: pip install PyMuPDF")
return
doc = fitz.open(str(input_path))
output_dir.mkdir(parents=True, exist_ok=True)
image_count = 0
# pyrefly: ignore [bad-argument-type]
for page_num, page in enumerate(doc):
images = page.get_images(full=True)
for img_idx, img in enumerate(images):
xref = img[0]
base_image = doc.extract_image(xref)
image_data = base_image["image"]
image_ext = base_image["ext"]
image_path = output_dir / f"page_{page_num + 1}_img_{img_idx + 1}.{image_ext}"
image_path.write_bytes(image_data)
image_count += 1
doc.close()
print(f"图片提取完成: {output_dir} (共 {image_count} 张)")
def pdf_add_watermark(input_path: Path, output_path: Path, text: str = "CONFIDENTIAL") -> None:
"""添加 PDF 水印."""
if not HAS_PYMUPDF:
print("未安装 PyMuPDF 库,请安装: pip install PyMuPDF")
return
doc = fitz.open(str(input_path))
for page in doc:
rect = page.rect
text_width = fitz.get_text_length(text, fontsize=48)
x = (rect.width - text_width) / 2
y = rect.height / 2
page.insert_text((x, y), text, fontsize=48, rotate=45, color=(0, 0, 0))
output_path.parent.mkdir(parents=True, exist_ok=True)
doc.save(str(output_path))
doc.close()
print(f"水印添加完成: {output_path}")
def pdf_rotate(input_path: Path, output_path: Path, rotation: int = 90) -> None:
"""旋转 PDF 页面."""
if not HAS_PYMUPDF:
print("未安装 PyMuPDF 库,请安装: pip install PyMuPDF")
return
doc = fitz.open(str(input_path))
for page in doc:
page.set_rotation(rotation)
output_path.parent.mkdir(parents=True, exist_ok=True)
doc.save(str(output_path))
doc.close()
print(f"旋转完成: {output_path}")
def pdf_crop(input_path: Path, output_path: Path, margins: tuple[int, int, int, int]) -> None:
"""裁剪 PDF 页面."""
if not HAS_PYMUPDF:
print("未安装 PyMuPDF 库,请安装: pip install PyMuPDF")
return
doc = fitz.open(str(input_path))
left, top, right, bottom = margins
for page in doc:
rect = page.rect
new_rect = fitz.Rect(
rect.x0 + left,
rect.y0 + top,
rect.x1 - right,
rect.y1 - bottom,
)
page.set_cropbox(new_rect)
output_path.parent.mkdir(parents=True, exist_ok=True)
doc.save(str(output_path))
doc.close()
print(f"裁剪完成: {output_path}")
def pdf_info(input_path: Path) -> None:
"""显示 PDF 信息."""
if not HAS_PYMUPDF:
print("未安装 PyMuPDF 库,请安装: pip install PyMuPDF")
return
doc = fitz.open(str(input_path))
print(f"文件: {input_path}")
print(f"页数: {doc.page_count}")
# pyrefly: ignore [missing-attribute]
print(f"标题: {doc.metadata.get('title', 'N/A')}")
# pyrefly: ignore [missing-attribute]
print(f"作者: {doc.metadata.get('author', 'N/A')}")
# pyrefly: ignore [missing-attribute]
print(f"创建日期: {doc.metadata.get('creationDate', 'N/A')}")
# pyrefly: ignore [missing-attribute]
print(f"修改日期: {doc.metadata.get('modDate', 'N/A')}")
print(f"文件大小: {input_path.stat().st_size / 1024:.1f} KB")
doc.close()
def pdf_ocr(input_path: Path, output_path: Path, lang: str = "chi_sim+eng") -> None:
"""PDF OCR 识别."""
try:
import pytesseract
from PIL import Image
except ImportError:
print("未安装 OCR 相关库,请安装: pip install pytesseract pillow")
return
if not HAS_PYMUPDF:
print("未安装 PyMuPDF 库,请安装: pip install PyMuPDF")
return
doc = fitz.open(str(input_path))
new_doc = fitz.open()
for page in doc:
pix = page.get_pixmap()
img = Image.frombytes("RGB", (pix.width, pix.height), pix.samples)
ocr_text = pytesseract.image_to_string(img, lang=lang)
new_page = new_doc.new_page(width=page.rect.width, height=page.rect.height)
new_page.insert_image(new_page.rect, pixmap=pix)
text_rect = fitz.Rect(0, 0, page.rect.width, page.rect.height)
# pyrefly: ignore [bad-argument-type]
new_page.insert_textbox(text_rect, ocr_text, fontname="china-ss", fontsize=11)
output_path.parent.mkdir(parents=True, exist_ok=True)
new_doc.save(str(output_path))
new_doc.close()
doc.close()
print(f"OCR 识别完成: {output_path}")
def pdf_reorder(input_path: Path, output_path: Path, order: list[int]) -> None:
"""重排 PDF 页面顺序."""
if not HAS_PYPDF:
print("未安装 pypdf 库,请安装: pip install pypdf")
return
reader = pypdf.PdfReader(str(input_path))
writer = pypdf.PdfWriter()
for page_num in order:
if 0 <= page_num < len(reader.pages):
writer.add_page(reader.pages[page_num])
output_path.parent.mkdir(parents=True, exist_ok=True)
with open(output_path, "wb") as f:
writer.write(f)
print(f"重排完成: {output_path}")
def pdf_to_images(input_path: Path, output_dir: Path, dpi: int = 300) -> None:
"""PDF 转图片."""
if not HAS_PYMUPDF:
print("未安装 PyMuPDF 库,请安装: pip install PyMuPDF")
return
doc = fitz.open(str(input_path))
output_dir.mkdir(parents=True, exist_ok=True)
# pyrefly: ignore [bad-argument-type]
for page_num, page in enumerate(doc):
pix = page.get_pixmap(dpi=dpi)
image_path = output_dir / f"{input_path.stem}_page_{page_num + 1}.png"
pix.save(str(image_path))
doc.close()
print(f"转换完成: {output_dir}")
def pdf_repair(input_path: Path, output_path: Path) -> None:
"""修复 PDF 文件."""
if not HAS_PYMUPDF:
print("未安装 PyMuPDF 库,请安装: pip install PyMuPDF")
return
doc = fitz.open(str(input_path))
output_path.parent.mkdir(parents=True, exist_ok=True)
doc.save(str(output_path), garbage=4, deflate=True, clean=True)
doc.close()
print(f"修复完成: {output_path}")
# ============================================================================
# CLI Runner
# ============================================================================
def main() -> None: # noqa: PLR0912
"""PDF 工具主函数."""
parser = argparse.ArgumentParser(
description="PDFTool - PDF 文件工具集",
usage="pdftool <command> [options]",
)
subparsers = parser.add_subparsers(dest="command", help="可用命令")
# 合并 PDF 命令
merge_parser = subparsers.add_parser("m", help="合并 PDF 文件")
merge_parser.add_argument("inputs", nargs="+", help="输入 PDF 文件路径")
merge_parser.add_argument("--output", type=str, default="merged.pdf", help="输出文件路径")
# 拆分 PDF 命令
split_parser = subparsers.add_parser("s", help="拆分 PDF 文件为单页")
split_parser.add_argument("input", help="输入 PDF 文件路径")
split_parser.add_argument("--output-dir", type=str, default="split", help="输出目录")
# 压缩 PDF 命令
compress_parser = subparsers.add_parser("c", help="压缩 PDF 文件")
compress_parser.add_argument("input", help="输入 PDF 文件路径")
compress_parser.add_argument("--output", type=str, default="compressed.pdf", help="输出文件路径")
# 加密 PDF 命令
encrypt_parser = subparsers.add_parser("e", help="加密 PDF 文件")
encrypt_parser.add_argument("input", help="输入 PDF 文件路径")
encrypt_parser.add_argument("--output", type=str, default="encrypted.pdf", help="输出文件路径")
encrypt_parser.add_argument("--password", type=str, required=True, help="密码")
# 解密 PDF 命令
decrypt_parser = subparsers.add_parser("d", help="解密 PDF 文件")
decrypt_parser.add_argument("input", help="输入 PDF 文件路径")
decrypt_parser.add_argument("--output", type=str, default="decrypted.pdf", help="输出文件路径")
decrypt_parser.add_argument("--password", type=str, required=True, help="密码")
# 提取文本命令
extract_text_parser = subparsers.add_parser("xt", help="提取 PDF 文本")
extract_text_parser.add_argument("input", help="输入 PDF 文件路径")
extract_text_parser.add_argument("--output", type=str, default="output.txt", help="输出文件路径")
# 提取图片命令
extract_images_parser = subparsers.add_parser("xi", help="提取 PDF 图片")
extract_images_parser.add_argument("input", help="输入 PDF 文件路径")
extract_images_parser.add_argument("--output-dir", type=str, default="images", help="输出目录")
# 添加水印命令
watermark_parser = subparsers.add_parser("w", help="添加 PDF 水印")
watermark_parser.add_argument("input", help="输入 PDF 文件路径")
watermark_parser.add_argument("--output", type=str, default="watermarked.pdf", help="输出文件路径")
watermark_parser.add_argument("--text", type=str, default="CONFIDENTIAL", help="水印文本")
# 旋转 PDF 命令
rotate_parser = subparsers.add_parser("r", help="旋转 PDF 页面")
rotate_parser.add_argument("input", help="输入 PDF 文件路径")
rotate_parser.add_argument("--output", type=str, default="rotated.pdf", help="输出文件路径")
rotate_parser.add_argument("--rotation", type=int, default=90, help="旋转角度 (90, 180, 270)")
# 裁剪 PDF 命令
crop_parser = subparsers.add_parser("crop", help="裁剪 PDF 页面")
crop_parser.add_argument("input", help="输入 PDF 文件路径")
crop_parser.add_argument("--output", type=str, default="cropped.pdf", help="输出文件路径")
crop_parser.add_argument("--left", type=int, default=10, help="左边裁剪")
crop_parser.add_argument("--top", type=int, default=10, help="顶部裁剪")
crop_parser.add_argument("--right", type=int, default=10, help="右边裁剪")
crop_parser.add_argument("--bottom", type=int, default=10, help="底部裁剪")
# 显示信息命令
info_parser = subparsers.add_parser("i", help="显示 PDF 信息")
info_parser.add_argument("input", help="输入 PDF 文件路径")
# OCR 识别命令
ocr_parser = subparsers.add_parser("ocr", help="PDF OCR 识别")
ocr_parser.add_argument("input", help="输入 PDF 文件路径")
ocr_parser.add_argument("--output", type=str, default="ocr.pdf", help="输出文件路径")
ocr_parser.add_argument("--lang", type=str, default="chi_sim+eng", help="OCR 语言")
# 转换图片命令
to_images_parser = subparsers.add_parser("img", help="PDF 转图片")
to_images_parser.add_argument("input", help="输入 PDF 文件路径")
to_images_parser.add_argument("--output-dir", type=str, default="images", help="输出目录")
to_images_parser.add_argument("--dpi", type=int, default=300, help="图片 DPI")
# 修复 PDF 命令
repair_parser = subparsers.add_parser("repair", help="修复 PDF 文件")
repair_parser.add_argument("input", help="输入 PDF 文件路径")
repair_parser.add_argument("--output", type=str, default="repaired.pdf", help="输出文件路径")
args = parser.parse_args()
if args.command == "m":
graph = px.Graph.from_specs([
px.TaskSpec("pdf_merge", fn=pdf_merge, args=([Path(p) for p in args.inputs], Path(args.output)))
])
elif args.command == "s":
graph = px.Graph.from_specs([
px.TaskSpec("pdf_split", fn=pdf_split, args=(Path(args.input), Path(args.output_dir)))
])
elif args.command == "c":
graph = px.Graph.from_specs([
px.TaskSpec("pdf_compress", fn=pdf_compress, args=(Path(args.input), Path(args.output)))
])
elif args.command == "e":
graph = px.Graph.from_specs([
px.TaskSpec("pdf_encrypt", fn=pdf_encrypt, args=(Path(args.input), Path(args.output), args.password))
])
elif args.command == "d":
graph = px.Graph.from_specs([
px.TaskSpec("pdf_decrypt", fn=pdf_decrypt, args=(Path(args.input), Path(args.output), args.password))
])
elif args.command == "xt":
graph = px.Graph.from_specs([
px.TaskSpec("pdf_extract_text", fn=pdf_extract_text, args=(Path(args.input), Path(args.output)))
])
elif args.command == "xi":
graph = px.Graph.from_specs([
px.TaskSpec("pdf_extract_images", fn=pdf_extract_images, args=(Path(args.input), Path(args.output_dir)))
])
elif args.command == "w":
graph = px.Graph.from_specs([
px.TaskSpec(
"pdf_watermark",
fn=pdf_add_watermark,
args=(Path(args.input), Path(args.output)),
kwargs={"text": args.text},
)
])
elif args.command == "r":
graph = px.Graph.from_specs([
px.TaskSpec(
"pdf_rotate",
fn=pdf_rotate,
args=(Path(args.input), Path(args.output)),
kwargs={"rotation": args.rotation},
)
])
elif args.command == "crop":
graph = px.Graph.from_specs([
px.TaskSpec(
"pdf_crop",
fn=pdf_crop,
args=(Path(args.input), Path(args.output)),
kwargs={"margins": (args.left, args.top, args.right, args.bottom)},
)
])
elif args.command == "i":
graph = px.Graph.from_specs([px.TaskSpec("pdf_info", fn=pdf_info, args=(Path(args.input),))])
elif args.command == "ocr":
graph = px.Graph.from_specs([
px.TaskSpec("pdf_ocr", fn=pdf_ocr, args=(Path(args.input), Path(args.output)), kwargs={"lang": args.lang})
])
elif args.command == "img":
graph = px.Graph.from_specs([
px.TaskSpec(
"pdf_to_images",
fn=pdf_to_images,
args=(Path(args.input), Path(args.output_dir)),
kwargs={"dpi": args.dpi},
)
])
elif args.command == "repair":
graph = px.Graph.from_specs([
px.TaskSpec("pdf_repair", fn=pdf_repair, args=(Path(args.input), Path(args.output)))
])
else:
parser.print_help()
return
px.run(graph, strategy="thread")
+212
View File
@@ -0,0 +1,212 @@
"""PyFlowX 统一 CLI 入口.
通过 ``pf <tool> [command] [options]`` 调用所有工具,
工具定义在 ``pyflowx.ops`` 子包中, 每个模块用 ``@px.tool`` 装饰器注册.
用法
----
pf # 列出所有可用工具
pf filedate # 查看 filedate 工具帮助
pf filedate add a.txt # 调用 filedate 的 add 子命令
pf pymake b # 调用 pymake 的 b 别名
"""
from __future__ import annotations
import contextlib
import importlib
import sys
from typing import Sequence
from pyflowx.tools import _TOOL_REGISTRY, run_tool
class PfApp:
"""pf 统一入口应用.
路由 ``pf <tool> [command]`` 到 ``@px.tool`` 注册的工具或传统 Python 工具.
"""
# 工具名到 ops 模块名的映射 (支持短别名)
_TOOL_ALIASES: dict[str, str] = {
"autofmt": "autofmt",
"af": "autofmt",
"bump": "bumpversion",
"bumpversion": "bumpversion",
"bv": "bumpversion",
"clr": "clr",
"clearscreen": "clr",
"dockercmd": "dockercmd",
"docker": "dockercmd",
"envdev": "envdev",
"env": "envdev",
"filedate": "filedate",
"fd": "filedate",
"filelevel": "filelevel",
"fl": "filelevel",
"folderback": "folderback",
"foldback": "folderback",
"fb": "folderback",
"folderzip": "folderzip",
"foldzip": "folderzip",
"fz": "folderzip",
"git": "gittool",
"gitt": "gittool",
"gittool": "gittool",
"gt": "gittool",
"ls": "lscalc",
"lscalc": "lscalc",
"msdown": "msdownload",
"msdownload": "msdownload",
"msd": "msdownload",
"pack": "packtool",
"packtool": "packtool",
"pk": "packtool",
"pdf": "pdftool",
"pdftool": "pdftool",
"pt": "pdftool",
"pip": "piptool",
"pymake": "pymake",
"piptool": "piptool",
"pp": "piptool",
"reseticon": "reseticoncache",
"reseticoncache": "reseticoncache",
"ric": "reseticoncache",
"screenshot": "screenshot",
"scrcap": "screenshot",
"ss": "screenshot",
"sglang": "sglang",
"sg": "sglang",
"ssh": "sshcopyid",
"sshcopy": "sshcopyid",
"sshcopyid": "sshcopyid",
"sc": "sshcopyid",
"taskk": "taskkill",
"taskkill": "taskkill",
"tk": "taskkill",
"wch": "which",
"which": "which",
}
# 传统工具: 有自己的 main() 函数 (无法 @px.tool 化的复杂逻辑)
_LEGACY_TOOLS: dict[str, str] = {
"emlman": "pyflowx.cli.emlmanager:main",
"profiler": "pyflowx.cli.profiler:main",
"pxp": "pyflowx.cli.profiler:main",
}
def __init__(self, argv: Sequence[str] | None = None) -> None:
self._argv = list(argv) if argv is not None else sys.argv[1:]
def run(self) -> int:
"""主入口, 返回退出码."""
if not self._argv:
self._list_tools()
return 0
tool_name = self._argv[0]
rest_argv = self._argv[1:]
resolved = self._resolve_tool(tool_name)
if resolved is None:
print(f"错误: 未知工具 '{tool_name}'", file=sys.stderr)
print("运行 'pf' 查看可用工具列表", file=sys.stderr)
return 1
tool_type, target = resolved
if tool_type == "legacy":
return self._run_legacy(target, rest_argv)
return self._run_tool(target, rest_argv)
def _list_tools(self) -> None:
"""列出所有可用工具."""
print("PyFlowX 工具列表:")
print()
print("@px.tool 工具:")
yaml_tools = sorted(set(self._TOOL_ALIASES.values()))
for tool in yaml_tools:
description = self._tool_description(tool)
if description:
print(f" pf {tool:<15} - {description}")
else:
print(f" pf {tool:<15}")
print()
print("传统工具:")
for tool in sorted(self._LEGACY_TOOLS):
print(f" pf {tool:<15}")
print()
print("示例:")
print(" pf filedate add a.txt")
print(" pf pymake b")
def _tool_description(self, tool_name: str) -> str:
"""获取工具描述 (从 _TOOL_REGISTRY 中已注册 ToolSpec 的 description/help)."""
# 触发 @px.tool 注册
with contextlib.suppress(ImportError):
importlib.import_module(f"pyflowx.ops.{tool_name}")
if tool_name not in _TOOL_REGISTRY:
return ""
subs = _TOOL_REGISTRY[tool_name]
# 优先取 description, 否则取任一 subcommand 的 help
for spec in subs.values():
if spec.description:
return spec.description
# 取第一个非 hidden subcommand 的 help
for spec in subs.values():
if not spec.hidden and spec.help:
return spec.help
return ""
def _resolve_tool(self, name: str) -> tuple[str, str] | None:
"""解析工具名, 返回 (类型, 目标).
类型: "tool""legacy"
目标: ops 模块名或 legacy 模块路径
"""
if name in self._TOOL_ALIASES:
return ("tool", self._TOOL_ALIASES[name])
if name in self._LEGACY_TOOLS:
return ("legacy", self._LEGACY_TOOLS[name])
return None
def _run_legacy(self, module_path: str, argv: list[str]) -> int:
"""运行传统工具的 main() 函数."""
module_name, func_name = module_path.split(":", 1)
module = importlib.import_module(module_name)
func = getattr(module, func_name)
original_argv = sys.argv
sys.argv = [f"pf {module_name.split('.')[-1]}", *argv]
try:
func()
return 0
except SystemExit as e:
return int(e.code) if e.code is not None else 0
finally:
sys.argv = original_argv
def _run_tool(self, target: str, argv: list[str]) -> int:
"""运行 @px.tool 工具.
导入 ``pyflowx.ops.<target>`` 触发 ``@px.tool`` 注册, 然后调用 ``run_tool``.
"""
with contextlib.suppress(ImportError):
importlib.import_module(f"pyflowx.ops.{target}")
if target not in _TOOL_REGISTRY:
print(f"错误: 未找到工具 '{target}'", file=sys.stderr)
print("运行 'pf' 查看可用工具列表", file=sys.stderr)
return 1
return run_tool(target, argv)
def main() -> None:
"""pf 统一入口主函数."""
sys.exit(PfApp().run())
if __name__ == "__main__":
main()
-195
View File
@@ -1,195 +0,0 @@
"""pip 包管理工具模块.
提供 pip 包管理操作的封装,
支持安装、卸载、下载等功能.
"""
from __future__ import annotations
import argparse
import fnmatch
import subprocess
from pathlib import Path
import pyflowx as px
# ============================================================================
# 配置
# ============================================================================
PACKAGE_DIR = "packages"
REQUIREMENTS_FILE = "requirements.txt"
# 受保护的包名集合
_PROTECTED_PACKAGES: frozenset[str] = frozenset({
"pyflowx",
"bitool",
})
# ============================================================================
# 辅助函数
# ============================================================================
def _get_installed_packages() -> list[str]:
"""获取当前环境中所有已安装的包名."""
try:
result = subprocess.run(
["pip", "list", "--format=freeze"],
capture_output=True,
text=True,
check=True,
)
packages: list[str] = []
for line in result.stdout.strip().split("\n"):
if line and "==" in line:
pkg_name = line.split("==")[0].strip()
packages.append(pkg_name)
except (subprocess.SubprocessError, OSError):
return []
return packages
def _expand_wildcard_packages(pattern: str) -> list[str]:
"""展开通配符模式为实际的包名列表."""
if not any(char in pattern for char in ["*", "?", "[", "]"]):
return [pattern]
installed_packages = _get_installed_packages()
matched = [pkg for pkg in installed_packages if fnmatch.fnmatchcase(pkg.lower(), pattern.lower())]
return matched
def _filter_protected_packages(packages: list[str]) -> list[str]:
"""过滤掉受保护的包名."""
safe = [p for p in packages if p.lower() not in {p.lower() for p in _PROTECTED_PACKAGES}]
filtered = [p for p in packages if p.lower() in {p.lower() for p in _PROTECTED_PACKAGES}]
if filtered:
print(f"跳过受保护的包: {', '.join(filtered)}")
return safe
def pip_uninstall(pkg_names: list[str]) -> None:
"""卸载包."""
packages_to_uninstall: list[str] = []
for pattern in pkg_names:
packages_to_uninstall.extend(_expand_wildcard_packages(pattern))
packages_to_uninstall = _filter_protected_packages(packages_to_uninstall)
if not packages_to_uninstall:
return
subprocess.run(["pip", "uninstall", "-y", *packages_to_uninstall], check=True)
def pip_reinstall(pkg_names: list[str], offline: bool = False) -> None:
"""重新安装包."""
safe_pkgs = _filter_protected_packages(pkg_names)
if not safe_pkgs:
print("所有指定的包均为受保护包, 跳过重装")
return
subprocess.run(["pip", "uninstall", "-y", *safe_pkgs], check=True)
options = ["--no-index", "--find-links", "."] if offline else []
subprocess.run(["pip", "install", *options, *safe_pkgs], check=True)
def pip_download(pkg_names: list[str], offline: bool = False) -> None:
"""下载包."""
options = ["--no-index", "--find-links", "."] if offline else []
subprocess.run(
["pip", "download", *pkg_names, *options, "-d", PACKAGE_DIR],
check=True,
)
def pip_freeze() -> None:
"""冻结依赖."""
result = subprocess.run(
["pip", "freeze", "--exclude-editable"],
capture_output=True,
text=True,
check=True,
)
Path(REQUIREMENTS_FILE).write_text(result.stdout)
# ============================================================================
# CLI Runner
# ============================================================================
def main() -> None:
"""pip 工具主函数."""
parser = argparse.ArgumentParser(
description="PipTool - pip 包管理工具",
usage="piptool <command> [options]",
)
subparsers = parser.add_subparsers(dest="command", help="可用命令")
# 安装命令
install_parser = subparsers.add_parser("i", help="安装包")
install_parser.add_argument("packages", nargs="+", help="要安装的包名")
# 卸载命令
uninstall_parser = subparsers.add_parser("u", help="卸载包")
uninstall_parser.add_argument("packages", nargs="+", help="要卸载的包名 (支持通配符)")
# 重装命令
reinstall_parser = subparsers.add_parser("r", help="重新安装包")
reinstall_parser.add_argument("packages", nargs="+", help="要重装的包名")
reinstall_parser.add_argument("--offline", action="store_true", help="使用离线模式")
# 下载命令
download_parser = subparsers.add_parser("d", help="下载包")
download_parser.add_argument("packages", nargs="+", help="要下载的包名")
download_parser.add_argument("--offline", action="store_true", help="使用离线模式")
# 升级 pip 命令
subparsers.add_parser("up", help="升级 pip")
# 冻结依赖命令
subparsers.add_parser("f", help="冻结依赖到 requirements.txt")
args = parser.parse_args()
if args.command == "i":
graph = px.Graph.from_specs([px.TaskSpec("pip_install", cmd=["pip", "install", *args.packages], verbose=True)])
elif args.command == "u":
graph = px.Graph.from_specs([
px.TaskSpec("pip_uninstall", fn=pip_uninstall, args=(args.packages,), verbose=True)
])
elif args.command == "r":
graph = px.Graph.from_specs([
px.TaskSpec(
"pip_reinstall",
fn=pip_reinstall,
args=(args.packages,),
kwargs={"offline": args.offline},
verbose=True,
)
])
elif args.command == "d":
graph = px.Graph.from_specs([
px.TaskSpec(
"pip_download",
fn=pip_download,
args=(args.packages,),
kwargs={"offline": args.offline},
verbose=True,
)
])
elif args.command == "up":
graph = px.Graph.from_specs([
px.TaskSpec("pip_upgrade", cmd=["python", "-m", "pip", "install", "--upgrade", "pip"], verbose=True)
])
elif args.command == "f":
graph = px.Graph.from_specs([px.TaskSpec("pip_freeze", fn=pip_freeze, verbose=True)])
else:
parser.print_help()
return
px.run(graph, strategy="thread")
-125
View File
@@ -1,125 +0,0 @@
"""Python 构建工具模块.
完全替代传统的 Makefile,
提供更好的跨平台兼容性和 Python 生态集成.
"""
from __future__ import annotations
from pathlib import Path
import pyflowx as px
from pyflowx.conditions import Constants
# 项目根目录(pymake.py 在 src/pyflowx/cli,向上四层到达根目录)
ROOT_DIR = Path(__file__).parent.parent.parent.parent
MATURIN_BUILD_COMMAND = ["maturin", "build", "-r"]
if Constants.IS_WINDOWS:
MATURIN_BUILD_COMMAND.extend(["--target", "x86_64-win7-windows-msvc", "-Zbuild-std", "-i", "python3.8"])
# 扁平注册所有任务(px.cmd 自动从命令前两段推导 name)
# 所有任务指定 cwd=ROOT_DIR,确保在项目根目录执行
tasks: list[px.TaskSpec] = [
px.cmd(["uv", "build"], cwd=ROOT_DIR),
px.cmd(MATURIN_BUILD_COMMAND, cwd=ROOT_DIR),
px.cmd(["uv", "sync"], cwd=ROOT_DIR),
px.cmd(["gitt", "c"], name="git_clean", cwd=ROOT_DIR),
px.cmd(
["pytest", "-m", "not slow", "-n", "8", "--dist", "loadfile", "--color=yes", "--durations=10"],
name="test",
cwd=ROOT_DIR,
),
px.cmd(
["pytest", "-m", "not slow", "--dist", "loadfile", "--color=yes", "--durations=10"],
name="test_fast",
cwd=ROOT_DIR,
),
px.cmd(
["pytest", "--cov", "-n", "8", "--dist", "loadfile", "--tb=short", "-v", "--color=yes", "--durations=10"],
name="test_coverage",
cwd=ROOT_DIR,
),
px.cmd(["pyrefly", "check", "."], cwd=ROOT_DIR),
px.cmd(["git", "add", "-A"], name="git_add_all", cwd=ROOT_DIR),
px.cmd(["bumpversion"], cwd=ROOT_DIR),
px.cmd(["bumpversion", "minor"], cwd=ROOT_DIR),
px.cmd(["git", "push"], cwd=ROOT_DIR),
px.cmd(["git", "push", "--tags"], name="git_push_tags", cwd=ROOT_DIR),
px.cmd(["hatch", "publish"], name="publish_python", cwd=ROOT_DIR),
px.cmd(["twine", "upload", "--disable-progress-bar"], name="twine_publish", cwd=ROOT_DIR),
]
# 单任务别名(alias 名与任务名相同):直接内联 TaskSpec,避免 str 自引用
aliases: dict[str, str | list[str | px.TaskSpec] | px.TaskSpec | px.Graph] = {
# 构建命令
"b": "uv_build",
"bc": "maturin_build",
"ba": ["b", "bc"],
# 安装命令
"sync": "uv_sync",
# 清理命令
"c": "git_clean",
# 开发工具
"bump": ["c", "tc", "git_add_all", "bumpversion"],
"bumpmi": "bumpversion_minor",
"cov": ["git_clean", "test_coverage"],
"doc": px.cmd(["sphinx-build", "-b", "html", "docs", "docs/_build"], name="doc", cwd=ROOT_DIR),
"lint": px.cmd(["ruff", "check", "--fix", "--unsafe-fixes"], name="lint", cwd=ROOT_DIR),
"pb": ["twine_publish", "publish_python"],
"t": "test",
"tf": "test_fast",
"tc": ["pyrefly_check", "lint"],
"tox": px.cmd(["tox", "-p", "auto"], name="tox", cwd=ROOT_DIR),
# 发布命令
"p": ["git_clean", "git_push", "git_push_tags"],
}
def main() -> None:
"""pymake 构建工具.
🔨 构建命令:
pymake b - 构建 Python 主包 (uv build)
pymake bc - 构建 Rust 核心模块 (maturin build)
pymake ba - 构建所有包 (先 Python 后 Rust)
📦 安装命令 (开发模式):
pymake sync - 安装依赖包 (uv sync)
🧹 清理命令:
pymake c - 清理所有构建产物 (gitt c)
🛠️ 开发工具:
pymake t - 运行测试 (pytest)
pymake tc - 运行测试并生成覆盖率报告
pymake tf - 运行快速测试 (pytest -m not slow)
pymake lint - 代码格式化与检查 (ruff)
pymake type - 类型检查 (mypy, ty)
pymake doc - 构建文档 (sphinx)
🔬 多版本测试:
pymake tox - 多版本 Python 测试 (tox -p auto)
📦 发布命令:
pymake pb - 发布到 PyPI (twine + hatch)
🔖 版本管理:
pymake bump - 自动升级版本号并提交修改 (清理 + 检查 + 格式化 + git add + bumpversion)
💡 常用工作流:
1. 日常开发: pymake lint && pymake t
2. 构建发布包: pymake ba
3. 多版本兼容性测试: pymake tox
4. 发布到 PyPI: pymake pb
📝 示例:
pymake ba # 构建所有包
pymake sync # 安装依赖
pymake t # 运行测试
pymake tox # 多版本兼容性测试
pymake lint # 格式化代码
pymake type # 类型检查
"""
runner = px.CliRunner(strategy="sequential", description="PyMake - Python 构建工具", tasks=tasks, aliases=aliases)
runner.run_cli()
-10
View File
@@ -1,10 +0,0 @@
from __future__ import annotations
import pyflowx as px
from pyflowx.tasks.system import reset_icon_cache
def main() -> None:
"""重启图标缓存工具主函数."""
graph = px.Graph.from_specs(reset_icon_cache())
px.run(graph, strategy="thread")
View File
-15
View File
@@ -1,15 +0,0 @@
"""清屏工具.
跨平台清屏工具, 支持终端和控制台清屏.
"""
from __future__ import annotations
import pyflowx as px
from pyflowx.tasks.system import clr
def main() -> None:
"""清屏工具主函数."""
graph = px.Graph.from_specs([clr()])
px.run(graph, strategy="thread")
-40
View File
@@ -1,40 +0,0 @@
"""进程终止工具.
跨平台进程终止工具, 支持按名称终止进程.
用法: taskkill proc_name [proc_name ...]
"""
from __future__ import annotations
import argparse
import pyflowx as px
from pyflowx.conditions import Constants
def main() -> None:
"""进程终止工具主函数."""
parser = argparse.ArgumentParser(
description="TaskKill - 进程终止工具",
usage="taskkill <process_name> [process_name ...]",
)
parser.add_argument(
"process_names",
type=str,
nargs="+",
help="进程名称 (如: chrome.exe python node)",
)
args = parser.parse_args()
if Constants.IS_WINDOWS:
cmd = ["taskkill", "/f", "/im"]
else:
cmd = ["pkill", "-f"]
graph = px.Graph.from_specs(
[
px.TaskSpec(f"kill_{proc_name}", cmd=[*cmd, f"{proc_name}*"], verbose=True)
for proc_name in args.process_names
],
)
px.run(graph, strategy="thread")
-21
View File
@@ -1,21 +0,0 @@
"""命令查找工具.
跨平台查找可执行命令路径, 类似 Unix 的 which 命令.
"""
from __future__ import annotations
import argparse
import pyflowx as px
from pyflowx.tasks.system import which
def main() -> None:
"""命令查找工具主函数."""
parser = argparse.ArgumentParser(description="Which - 命令查找工具")
parser.add_argument("commands", nargs="+", help="要查找的命令名称, 如: python ls ps gcc...")
args = parser.parse_args()
graph = px.Graph.from_specs([which(cmd) for cmd in args.commands])
px.run(graph, strategy="thread")
+2
View File
@@ -90,6 +90,8 @@ def run_command(spec: TaskSpec[Any]) -> Any: # noqa: PLR0912
print(f"[verbose] 返回码: {result.returncode}", flush=True)
if result.returncode == 0:
if not verbose and result.stdout:
print(result.stdout, end="", flush=True)
return None
err_msg = f"{label}执行失败: `{cmd_str}`, 返回码: {result.returncode}"
View File
-56
View File
@@ -1,56 +0,0 @@
"""Example 3: async aggregation with static args and Context injection.
Shows:
* async task functions executed with strategy="async".
* static positional args (TaskSpec.args) for parameterised tasks.
* Context annotation to receive the full upstream result mapping.
* on_event callback for real-time progress.
"""
from __future__ import annotations
import asyncio
from typing import Any
import pyflowx as px
async def fetch_user(uid: int) -> dict[str, Any]:
await asyncio.sleep(0.2)
return {"id": uid, "name": f"User{uid}"}
async def fetch_posts(uid: int) -> list[int]:
await asyncio.sleep(0.2)
return [uid, uid + 1]
# Context annotation → receives the full mapping of upstream results.
def aggregate(ctx: px.Context) -> dict[str, Any]:
return dict(ctx)
def main() -> None:
graph = px.Graph.from_specs([
# Static positional args parameterise the same function twice.
px.TaskSpec("fetch_user", fetch_user, args=(1,)),
px.TaskSpec("fetch_posts", fetch_posts, args=(1,)),
px.TaskSpec("aggregate", aggregate, depends_on=("fetch_user", "fetch_posts")),
])
print("=== Dry run ===")
_ = px.run(graph, strategy="async", dry_run=True)
events: list[px.TaskEvent] = []
print("\n=== Async execution ===")
report = px.run(graph, strategy="async", on_event=events.append)
for ev in events:
print(f" event: {ev.task} -> {ev.status.value}")
print(f"\naggregate = {report['aggregate']}")
print(report.describe())
if __name__ == "__main__":
main()
-77
View File
@@ -1,77 +0,0 @@
"""Example 1: ETL pipeline (sequential strategy).
Demonstrates the core PyFlowX workflow:
* Define tasks as plain functions.
* Declare the DAG with a list of TaskSpec.
* Parameter names == dependency names → automatic context injection,
no wrappers needed (contrast with flowweaver's get_task_result boilerplate).
* dry_run to preview, then execute and read typed results from RunReport.
"""
from __future__ import annotations
from typing import Any
import pyflowx as px
# --- task functions: pure, testable, no framework coupling ------------- #
def extract_customers() -> list[dict[str, Any]]:
return [
{"id": "C001", "name": "Alice"},
{"id": "C002", "name": "Bob"},
]
def extract_orders() -> list[dict[str, Any]]:
return [
{"id": "O001", "customer_id": "C001", "amount": 150.0},
{"id": "O002", "customer_id": "C002", "amount": 200.5},
]
# Parameter names match dependency names → automatic injection.
def transform(
extract_customers: list[dict[str, Any]],
extract_orders: list[dict[str, Any]],
) -> list[dict[str, Any]]:
cmap = {c["id"]: c for c in extract_customers}
return [{**o, "customer_name": cmap[o["customer_id"]]["name"]} for o in extract_orders if o["customer_id"] in cmap]
def load(transform: list[dict[str, Any]]) -> int:
print(f" loaded {len(transform)} records")
return len(transform)
def main() -> None:
graph = px.Graph.from_specs([
px.TaskSpec("extract_customers", extract_customers, tags=("extract",)),
px.TaskSpec("extract_orders", extract_orders, tags=("extract",)),
px.TaskSpec(
"transform",
transform,
depends_on=("extract_customers", "extract_orders"),
tags=("transform",),
),
px.TaskSpec(
"load", load, depends_on=("transform",), retry=px.RetryPolicy(max_attempts=1, delay=1.0), tags=("load",)
),
])
print("=== Execution plan ===")
print(graph.describe())
print("\n=== Dry run (no execution) ===")
_ = px.run(graph, strategy="sequential", dry_run=True)
print("\n=== Sequential execution ===")
report = px.run(graph, strategy="sequential")
print(report.describe())
print(f"\nload result = {report['load']}")
print(f"summary = {report.summary()}")
if __name__ == "__main__":
main()
-57
View File
@@ -1,57 +0,0 @@
"""Example 2: parallel execution (thread strategy).
Same DAG run with sequential vs. thread strategy to show layer-internal
parallelism. Tasks within a layer run concurrently; layers are barriers.
Layer 1: [fetch_a, fetch_b] (parallel)
Layer 2: [merge] (waits for both)
"""
from __future__ import annotations
import time
import pyflowx as px
def fetch_a() -> str:
time.sleep(0.5)
return "a"
def fetch_b() -> str:
time.sleep(0.5)
return "b"
def merge(fetch_a: str, fetch_b: str) -> str:
return fetch_a + fetch_b
def main() -> None:
graph = px.Graph.from_specs([
px.TaskSpec("fetch_a", fetch_a),
px.TaskSpec("fetch_b", fetch_b),
px.TaskSpec("merge", merge, depends_on=("fetch_a", "fetch_b")),
])
print("=== Mermaid diagram ===")
print(graph.to_mermaid("LR"))
print("\n=== Sequential (expect ~1.0s) ===")
start = time.time()
report_seq = px.run(graph, strategy="sequential")
t_seq = time.time() - start
print(f" result={report_seq['merge']} time={t_seq:.2f}s")
print("\n=== Threaded (expect ~0.5s) ===")
start = time.time()
report_thr = px.run(graph, strategy="thread", max_workers=2)
t_thr = time.time() - start
print(f" result={report_thr['merge']} time={t_thr:.2f}s")
print(f"\nspeedup = {t_seq / t_thr:.2f}x")
if __name__ == "__main__":
main()
+7
View File
@@ -792,6 +792,13 @@ def run(
_print_dry_run(graph, layers)
return RunReport(success=True)
# verbose 模式下, 把所有 spec 的 verbose 标记设为 True,
# 使 execute_command 打印执行命令与返回码 (任务生命周期由 callback 打印)
if verbose:
from dataclasses import replace
graph = Graph.from_specs([replace(s, verbose=True) if not s.verbose else s for s in graph.all_specs().values()])
# 入口统一校验一次:所有策略共用,避免 layers() / dependency 路径
# 各自重复调用 validate()。
graph.validate()
-1
View File
@@ -219,7 +219,6 @@ class Graph:
"""
graph = cls(defaults=defaults or GraphDefaults(), namespace=namespace)
pending_refs: list[str] = []
for spec in specs:
if isinstance(spec, str):
pending_refs.append(spec)
+34
View File
@@ -0,0 +1,34 @@
"""ops 子包 - CLI 工具模块集合.
每个子模块使用 ``@px.tool`` 装饰器注册 CLI 工具, 由 ``pf`` 入口按需
``importlib.import_module`` 触发注册. 子模块按工具名组织 (一文件一工具).
子模块
------
- :mod:`autofmt` —— 代码格式化 (ruff)
- :mod:`bumpversion` —— 版本号管理
- :mod:`clr` —— 清屏
- :mod:`dockercmd` —— Docker 镜像登录
- :mod:`envdev` —— 开发环境镜像源配置
- :mod:`filedate` —— 文件日期修改
- :mod:`filelevel` —— 文件等级
- :mod:`folderback` —— 文件夹备份
- :mod:`folderzip` —— 文件夹压缩
- :mod:`gittool` —— Git 工具
- :mod:`lscalc` —— LS-DYNA 计算
- :mod:`msdownload` —— ModelScope 下载
- :mod:`packtool` —— 打包工具
- :mod:`pdftool` —— PDF 工具
- :mod:`piptool` —— pip 工具
- :mod:`pymake` —— Python 项目构建
- :mod:`reseticoncache` —— 重置图标缓存
- :mod:`screenshot` —— 截图
- :mod:`sglang` —— SGLang 服务
- :mod:`sshcopyid` —— SSH 公钥拷贝
- :mod:`taskkill` —— 进程终止
- :mod:`which` —— 命令查找
"""
from __future__ import annotations
__all__: list[str] = []
@@ -1,25 +1,28 @@
"""自动格式化工具模块.
"""autofmt - 自动格式化工具.
提供 Python 代码自动格式化的常用功能封装,
支持 docstring 自动生成pyproject.toml 配置同步等功能.
提供格式化代码/代码检查/自动添加文档/同步配置 子命令.
"""
from __future__ import annotations
import argparse
import ast
import subprocess
from pathlib import Path
import pyflowx as px
try:
import tomllib # noqa: F401
HAS_TOMLLIB = True
except ImportError:
HAS_TOMLLIB = False
__all__ = [
"IGNORE_PATTERNS",
"add_docstring",
"auto_add_docstrings",
"fmt",
"format_all",
"format_with_ruff",
"generate_module_docstring",
"lint",
"lint_with_ruff",
"sync_pyproject_config",
]
# ============================================================================
# 配置
@@ -47,6 +50,32 @@ IGNORE_PATTERNS = [
# ============================================================================
@px.tool("autofmt", subcommand="fmt", help="格式化代码")
def fmt(target: str = ".") -> None:
"""格式化代码 (ruff format).
Parameters
----------
target : str
目标路径 (默认: .)
"""
format_with_ruff(Path(target), fix=False)
@px.tool("autofmt", subcommand="lint", help="代码检查")
def lint(target: str = ".", fix: bool = False) -> None:
"""代码检查 (ruff check).
Parameters
----------
target : str
目标路径 (默认: .)
fix : bool
自动修复问题
"""
lint_with_ruff(Path(target), fix=fix)
def format_with_ruff(target: Path, fix: bool = True) -> None:
"""使用 ruff 格式化代码.
@@ -102,12 +131,10 @@ def add_docstring(file_path: Path, docstring: str) -> bool:
content = file_path.read_text(encoding="utf-8")
tree = ast.parse(content)
# 检查是否已有 docstring
first_node = tree.body[0] if tree.body else None
if first_node and isinstance(first_node, ast.Expr) and isinstance(first_node.value, ast.Constant):
return False
# 添加 docstring
lines = content.splitlines()
doc_lines = docstring.splitlines()
doc_lines.append("")
@@ -138,7 +165,6 @@ def generate_module_docstring(file_path: Path) -> str:
stem = file_path.stem
parent = file_path.parent.name
# 关键词匹配
keywords = {
"cli": f"Command-line interface for {parent}",
"gui": f"Graphical user interface for {parent}",
@@ -155,13 +181,14 @@ def generate_module_docstring(file_path: Path) -> str:
return f'"""{stem.replace("_", " ").title()} module."""'
def auto_add_docstrings(root_dir: Path) -> int:
@px.tool("autofmt", subcommand="doc", help="自动添加文档字符串")
def auto_add_docstrings(root_dir: Path = Path()) -> int:
"""自动为所有 Python 文件添加 docstring.
Parameters
----------
root_dir : Path
根目录
根目录 (默认: 当前目录)
Returns
-------
@@ -170,7 +197,6 @@ def auto_add_docstrings(root_dir: Path) -> int:
"""
count = 0
for py_file in root_dir.rglob("*.py"):
# 跳过忽略的文件
if any(pattern in str(py_file) for pattern in IGNORE_PATTERNS):
continue
@@ -182,20 +208,20 @@ def auto_add_docstrings(root_dir: Path) -> int:
return count
def sync_pyproject_config(root_dir: Path) -> None:
@px.tool("autofmt", subcommand="sync", help="同步 pyproject 配置")
def sync_pyproject_config(root_dir: Path = Path()) -> None:
"""同步 pyproject.toml 配置到子项目.
Parameters
----------
root_dir : Path
根目录
根目录 (默认: 当前目录)
"""
main_toml = root_dir / "pyproject.toml"
if not main_toml.exists():
print(f"主项目配置文件不存在: {main_toml}")
return
# 查找所有子项目的 pyproject.toml
sub_tomls = [p for p in root_dir.rglob("pyproject.toml") if p != main_toml and ".venv" not in str(p)]
if not sub_tomls:
@@ -204,7 +230,6 @@ def sync_pyproject_config(root_dir: Path) -> None:
print(f"找到 {len(sub_tomls)} 个子项目配置文件")
# 对每个子项目调用 ruff format
for sub_toml in sub_tomls:
subprocess.run(["ruff", "format", str(sub_toml)], check=False)
@@ -219,64 +244,6 @@ def format_all(root_dir: Path) -> None:
root_dir : Path
根目录
"""
# 使用 ruff format
subprocess.run(["ruff", "format", str(root_dir)], check=True)
# 使用 ruff check
subprocess.run(["ruff", "check", "--fix", "--unsafe-fixes", str(root_dir)], check=True)
print(f"格式化完成: {root_dir}")
# ============================================================================
# CLI Runner
# ============================================================================
def main() -> None:
"""自动格式化工具主函数."""
parser = argparse.ArgumentParser(
description="AutoFmt - 自动格式化工具",
usage="autofmt <command> [options]",
)
subparsers = parser.add_subparsers(dest="command", help="可用命令")
# ruff format 命令
format_parser = subparsers.add_parser("fmt", help="使用 ruff 格式化代码")
format_parser.add_argument("--target", type=str, default=".", help="目标路径")
# ruff check 命令
lint_parser = subparsers.add_parser("lint", help="使用 ruff 检查代码")
lint_parser.add_argument("--target", type=str, default=".", help="目标路径")
lint_parser.add_argument("--fix", action="store_true", help="自动修复")
# 自动添加 docstring 命令
doc_parser = subparsers.add_parser("doc", help="自动添加 docstring")
doc_parser.add_argument("--root-dir", type=str, default=".", help="根目录")
# 同步配置命令
sync_parser = subparsers.add_parser("sync", help="同步 pyproject.toml 配置")
sync_parser.add_argument("--root-dir", type=str, default=".", help="根目录")
args = parser.parse_args()
if args.command == "fmt":
graph = px.Graph.from_specs([px.TaskSpec("ruff_format", cmd=["ruff", "format", args.target], verbose=True)])
elif args.command == "lint":
cmd = ["ruff", "check", args.target]
if args.fix:
cmd.extend(["--fix", "--unsafe-fixes"])
graph = px.Graph.from_specs([px.TaskSpec("ruff_check", cmd=cmd, verbose=True)])
elif args.command == "doc":
graph = px.Graph.from_specs([
px.TaskSpec("auto_docstring", fn=auto_add_docstrings, args=(Path(args.root_dir),), verbose=True)
])
elif args.command == "sync":
graph = px.Graph.from_specs([
px.TaskSpec("sync_config", fn=sync_pyproject_config, args=(Path(args.root_dir),), verbose=True)
])
else:
parser.print_help()
return
px.run(graph, strategy="thread")
+232
View File
@@ -0,0 +1,232 @@
"""版本号管理模块.
提供单文件版本号更新 (``bump_file_version``) 与项目级批量版本号同步
(``bump_project_version``) 能力. ``bump_project_version`` 通过 ``@px.tool``
注册为 CLI 工具, 可通过 ``pf bumpversion`` 调用.
设计要点
--------
``bump_project_version`` 采用 "先读取基准、再统一写入" 的两阶段策略:
先扫描所有 ``__init__.py`` / ``pyproject.toml`` 文件, 读取各自的版本号,
取最大值作为基准版本计算新版本号, 然后把新版本号统一写入所有文件,
避免文件间版本号不同步导致的跳号问题.
"""
from __future__ import annotations
import re
import subprocess
from pathlib import Path
from typing import Literal
import pyflowx as px
__all__ = [
"BumpVersionType",
"bump_file_version",
"bump_project_version",
]
# ============================================================================
# 配置
# ============================================================================
BumpVersionType = Literal["patch", "minor", "major"]
_PYPROJECT_VERSION_PATTERN = re.compile(
r'(?:^|\n)\s*version\s*=\s*["\']'
r"(?P<major>0|[1-9]\d*)\.(?P<minor>0|[1-9]\d*)\.(?P<patch>0|[1-9]\d*)"
r"(?:-(?P<prerelease>(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)(?:\.(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*))?"
r"(?:\+(?P<buildmetadata>[0-9a-zA-Z-]+(?:\.[0-9a-zA-Z-]+)*))?"
r'["\']',
re.MULTILINE,
)
_INIT_VERSION_PATTERN = re.compile(
r'(?:^|\n)\s*__version__\s*=\s*["\']'
r"(?P<major>0|[1-9]\d*)\.(?P<minor>0|[1-9]\d*)\.(?P<patch>0|[1-9]\d*)"
r"(?:-(?P<prerelease>(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)(?:\.(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*))?"
r"(?:\+(?P<buildmetadata>[0-9a-zA-Z-]+(?:\.[0-9a-zA-Z-]+)*))?"
r'["\']',
re.MULTILINE,
)
_IGNORE_DIRS = frozenset({".venv", "venv", ".git", "__pycache__", ".tox", "node_modules", "build", "dist", ".eggs"})
# ============================================================================
# 私有辅助函数
# ============================================================================
def _get_pattern_for_file(file_name: str) -> re.Pattern[str] | None:
"""根据文件类型获取对应的正则表达式."""
if file_name == "pyproject.toml":
return _PYPROJECT_VERSION_PATTERN
if file_name == "__init__.py":
return _INIT_VERSION_PATTERN
return None
def _calculate_new_version(major: int, minor: int, patch: int, part: BumpVersionType) -> str:
"""计算新版本号."""
if part == "major":
return f"{major + 1}.0.0"
if part == "minor":
return f"{major}.{minor + 1}.0"
return f"{major}.{minor}.{patch + 1}"
def _build_replacement_string(original_match: str, new_version: str, file_name: str) -> str:
"""构建替换字符串, 保留原始格式."""
quote_char = '"' if '"' in original_match else "'"
key = "__version__" if file_name == "__init__.py" else "version"
prefix_match = re.match(rf"(\s*{key}\s*=\s*)[\"']", original_match)
prefix = prefix_match.group(1) if prefix_match else f"{key} = "
return f"{prefix}{quote_char}{new_version}{quote_char}"
def _read_version_tuple(file_path: Path) -> tuple[int, int, int] | None:
"""从文件中读取版本号, 返回 (major, minor, patch) 元组; 未找到返回 None.
读取失败时抛出 ``OSError`` / ``UnicodeDecodeError`` 由调用方处理.
"""
pattern = _get_pattern_for_file(file_path.name)
if pattern is None:
return None
content = file_path.read_text(encoding="utf-8")
match = pattern.search(content)
if not match:
return None
return int(match.group("major")), int(match.group("minor")), int(match.group("patch"))
def _write_version_to_file(file_path: Path, new_version: str) -> bool:
"""把新版本号写入指定文件; 成功返回 True, 未匹配到版本号返回 False."""
pattern = _get_pattern_for_file(file_path.name)
if pattern is None: # pragma: no cover - 调用方已保证 pattern 不为 None
return False
content = file_path.read_text(encoding="utf-8")
match = pattern.search(content)
if not match: # pragma: no cover - 调用方已通过 _read_version_tuple 验证
return False
replacement = _build_replacement_string(match.group(0), new_version, file_path.name)
content = content.replace(match.group(0), replacement)
try:
file_path.write_text(content, encoding="utf-8")
except OSError as e:
print(f"更新文件 {file_path} 版本号时出错: {e}")
raise
return True
# ============================================================================
# 公共函数
# ============================================================================
def bump_file_version(file_path: Path, part: BumpVersionType = "patch") -> str | None:
"""更新单个文件中的版本号.
读取文件当前版本号, 按 ``part`` 指定的部分递增, 写回文件.
Parameters
----------
file_path : Path
要更新的文件路径 (``pyproject.toml`` 或 ``__init__.py``)
part : BumpVersionType
版本部分: patch, minor, major
Returns
-------
str | None
更新后的新版本号; 文件中未找到版本号或读取失败时返回 None
"""
version_tuple = _read_version_tuple(file_path)
if version_tuple is None:
print(f"文件 {file_path} 中未找到版本号模式")
return None
major, minor, patch = version_tuple
new_version = _calculate_new_version(major, minor, patch, part)
if not _write_version_to_file(file_path, new_version): # pragma: no cover - _read_version_tuple 已验证
return None
return new_version
@px.tool("bumpversion", help="版本号自动管理")
def bump_project_version(part: BumpVersionType = "patch", no_tag: bool = False) -> str | None:
"""批量同步项目所有版本号文件并提交.
扫描当前目录下所有 ``__init__.py`` 和 ``pyproject.toml`` 文件
(排除虚拟环境和缓存目录), 先读取每个文件的当前版本号取最大值作为基准,
计算新版本号后统一写入所有文件, 最后执行 git add (按文件名) + commit + tag.
采用 "先读取基准、再统一写入" 的两阶段策略, 即使某些文件版本号不同步,
也能在一次 bump 后重新对齐, 避免跳号.
Parameters
----------
part : BumpVersionType
版本部分: patch, minor, major
no_tag : bool
提交后不创建 git tag
Returns
-------
str | None
更新后的新版本号; 未找到版本号文件时返回 None
"""
all_files: set[Path] = set()
for pattern in ("__init__.py", "pyproject.toml"):
for file in Path.cwd().rglob(pattern):
if not any(ignore_dir in file.parts for ignore_dir in _IGNORE_DIRS):
all_files.add(file)
if not all_files:
print("未找到包含版本号的文件")
return None
print(f"找到 {len(all_files)} 个文件需要更新版本号")
cwd = Path.cwd()
for file in sorted(all_files):
print(f" - {file.relative_to(cwd)}")
# 阶段 1: 读取所有文件版本号, 取最大值作为基准
versions: list[tuple[int, int, int]] = []
for file in sorted(all_files):
v = _read_version_tuple(file)
if v is not None:
versions.append(v)
if not versions:
print("未能从任何文件读取版本号")
return None
major, minor, patch = max(versions)
new_version = _calculate_new_version(major, minor, patch, part)
print(f"基准版本: {major}.{minor}.{patch} -> 新版本: {new_version}")
# 阶段 2: 统一写入新版本号到所有文件
for file in sorted(all_files):
_write_version_to_file(file, new_version)
# 阶段 3: git add (按文件名) + commit + tag
relative_files = [str(file.relative_to(cwd)) for file in sorted(all_files)]
subprocess.run(["git", "add", *relative_files], check=True)
subprocess.run(["git", "commit", "-m", f"bump version to {new_version}"], check=True)
if not no_tag:
tag_name = f"v{new_version}"
subprocess.run(["git", "tag", "-a", tag_name, "-m", f"Release {tag_name}"], check=True)
print(f"已创建标签: {tag_name}")
return new_version
+21
View File
@@ -0,0 +1,21 @@
"""clr - 清屏工具.
跨平台清屏: Windows 调用 ``cls``, Linux/macOS 调用 ``clear``.
"""
from __future__ import annotations
import subprocess
import pyflowx as px
from pyflowx.conditions import Constants
@px.tool("clr", help="清屏 (跨平台)")
def clear_screen_run() -> None:
"""清屏 (跨平台).
Windows 调用 ``cls``, Linux/macOS 调用 ``clear``.
"""
cmd = ["cls"] if Constants.IS_WINDOWS else ["clear"]
subprocess.run(cmd, check=False)
+27
View File
@@ -0,0 +1,27 @@
"""dockercmd - Docker 镜像登录工具.
登录腾讯云 Docker 镜像仓库.
"""
from __future__ import annotations
import getpass
import subprocess
import pyflowx as px
_DOCKER_MIRROR_TENCENT: str = "ccr.ccs.tencentyun.com"
@px.tool("dockercmd", subcommand="login", help="登录腾讯云 Docker 镜像仓库")
def docker_login_tencent(username: str = "") -> None:
"""登录腾讯云 Docker 镜像仓库.
Parameters
----------
username : str
Docker 用户名 (为空时由 docker 交互式提示输入)
"""
user = username or getpass.getuser()
subprocess.run(["docker", "login", "--username", user, _DOCKER_MIRROR_TENCENT], check=False)
print(f"已尝试登录腾讯云镜像仓库 (用户: {user})")
+421
View File
@@ -0,0 +1,421 @@
"""envdev - 开发环境镜像源配置工具.
配置 Python / Conda / Rust 镜像源 (Linux 还会安装 Qt 库、中文字体、Docker).
所有镜像源参数互不影响, 可单独使用.
Linux 专用操作 (系统镜像/Qt/字体/Docker) 在非 Linux 平台上由函数内部跳过.
"""
from __future__ import annotations
import getpass
import os
import shutil
import subprocess
from pathlib import Path
from typing import Literal
import pyflowx as px
from pyflowx.conditions import Constants
__all__ = [
"download_rustup_script",
"install_linux_docker",
"install_linux_fonts",
"install_linux_qt_libs",
"install_rust_toolchain",
"setup_conda_mirror",
"setup_linux_system_mirror",
"setup_python_mirror",
"setup_rust_mirror",
]
# ============================================================================
# 配置常量
# ============================================================================
PyMirrorType = Literal["tsinghua", "aliyun", "huaweicloud", "ustc", "zju"]
CondaMirrorType = Literal["tsinghua", "ustc", "bsfu", "aliyun"]
RustMirrorType = Literal["tsinghua", "ustc", "aliyun"]
_PIP_INDEX_URLS: dict[str, str] = {
"tsinghua": "https://pypi.tuna.tsinghua.edu.cn/simple",
"aliyun": "https://mirrors.aliyun.com/pypi/simple/",
"huaweicloud": "https://mirrors.huaweicloud.com/repository/pypi/simple/",
"ustc": "https://pypi.mirrors.ustc.edu.cn/simple/",
"zju": "https://mirrors.zju.edu.cn/pypi/simple/",
}
_PIP_TRUSTED_HOSTS: dict[str, str] = {
"tsinghua": "pypi.tuna.tsinghua.edu.cn",
"aliyun": "mirrors.aliyun.com",
"huaweicloud": "mirrors.huaweicloud.com",
"ustc": "pypi.mirrors.ustc.edu.cn",
"zju": "mirrors.zju.edu.cn",
}
_UV_PYTHON_INSTALL_MIRROR: str = "https://registry.npmmirror.com/-/binary/python-build-standalone"
_CONDA_MIRROR_URLS: dict[str, list[str]] = {
"tsinghua": [
"https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/",
"https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free/",
"https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/r/",
"https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/msys2/",
"https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/pro/",
"https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/",
"https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/bioconda/",
"https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/menpo/",
"https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/pytorch/",
],
"ustc": [
"https://mirrors.ustc.edu.cn/anaconda/pkgs/main/",
"https://mirrors.ustc.edu.cn/anaconda/pkgs/free/",
"https://mirrors.ustc.edu.cn/anaconda/pkgs/r/",
"https://mirrors.ustc.edu.cn/anaconda/pkgs/msys2/",
"https://mirrors.ustc.edu.cn/anaconda/pkgs/pro/",
"https://mirrors.ustc.edu.cn/anaconda/pkgs/dev/",
"https://mirrors.ustc.edu.cn/anaconda/cloud/conda-forge/",
"https://mirrors.ustc.edu.cn/anaconda/cloud/bioconda/",
"https://mirrors.ustc.edu.cn/anaconda/cloud/menpo/",
"https://mirrors.ustc.edu.cn/anaconda/cloud/pytorch/",
],
"bsfu": [
"https://mirrors.bsfu.edu.cn/anaconda/pkgs/main/",
"https://mirrors.bsfu.edu.cn/anaconda/pkgs/free/",
"https://mirrors.bsfu.edu.cn/anaconda/pkgs/r/",
"https://mirrors.bsfu.edu.cn/anaconda/pkgs/msys2/",
"https://mirrors.bsfu.edu.cn/anaconda/pkgs/pro/",
"https://mirrors.bsfu.edu.cn/anaconda/pkgs/dev/",
"https://mirrors.bsfu.edu.cn/anaconda/cloud/conda-forge/",
"https://mirrors.bsfu.edu.cn/anaconda/cloud/bioconda/",
"https://mirrors.bsfu.edu.cn/anaconda/cloud/menpo/",
"https://mirrors.bsfu.edu.cn/anaconda/cloud/pytorch/",
],
"aliyun": [
"https://mirrors.aliyun.com/anaconda/pkgs/main/",
"https://mirrors.aliyun.com/anaconda/pkgs/free/",
"https://mirrors.aliyun.com/anaconda/pkgs/r/",
"https://mirrors.aliyun.com/anaconda/pkgs/msys2/",
"https://mirrors.aliyun.com/anaconda/pkgs/pro/",
"https://mirrors.aliyun.com/anaconda/pkgs/dev/",
"https://mirrors.aliyun.com/anaconda/cloud/conda-forge/",
"https://mirrors.aliyun.com/anaconda/cloud/bioconda/",
"https://mirrors.aliyun.com/anaconda/cloud/menpo/",
"https://mirrors.aliyun.com/anaconda/cloud/pytorch/",
],
}
_RUSTUP_MIRRORS: dict[str, dict[str, str]] = {
"tsinghua": {
"RUSTUP_DIST_SERVER": "https://mirrors.tuna.tsinghua.edu.cn/rustup",
"RUSTUP_UPDATE_ROOT": "https://mirrors.tuna.tsinghua.edu.cn/rustup/rustup",
"TOML_REGISTRY": "https://mirrors.tuna.tsinghua.edu.cn/crates.io-index/",
},
"aliyun": {
"RUSTUP_DIST_SERVER": "https://mirrors.aliyun.com/rustup",
"RUSTUP_UPDATE_ROOT": "https://mirrors.aliyun.com/rustup/rustup",
"TOML_REGISTRY": "https://mirrors.aliyun.com/crates.io-index/",
},
"ustc": {
"RUSTUP_DIST_SERVER": "https://mirrors.ustc.edu.cn/rust-static",
"RUSTUP_UPDATE_ROOT": "https://mirrors.ustc.edu.cn/rust-static/rustup",
"TOML_REGISTRY": "https://mirrors.ustc.edu.cn/crates.io-index/",
},
}
_RUST_SCCACHE_DIR: Path = Path.home() / ".cargo" / "sccache"
_RUST_SCCACHE_CACHE_SIZE: str = "20G"
_QT_LIBS: list[str] = [
"build-essential",
"libgl1",
"libegl1",
"libglib2.0-0",
"libfontconfig1",
"libfreetype6",
"libxkbcommon0",
"libdbus-1-3",
"libxcb-xinerama0",
"libxcb-icccm4",
"libxcb-image0",
"libxcb-keysyms1",
"libxcb-randr0",
"libxcb-render-util0",
"libxcb-shape0",
"libxcb-xfixes0",
"libxcb-cursor0",
]
_CHINESE_FONTS: list[str] = [
"fonts-noto-cjk",
"fonts-wqy-microhei",
"fonts-wqy-zenhei",
"fonts-noto-color-emoji",
]
_DOWNLOAD_MIRROR_SCRIPT: str = "curl -sSL https://linuxmirrors.cn/main.sh -o /tmp/linuxmirrors.sh"
_INSTALL_MIRROR_SCRIPT: str = "sudo bash /tmp/linuxmirrors.sh"
_RUSTUP_DOWNLOAD_URL_LINUX: str = "https://mirrors.aliyun.com/repo/rust/rustup-init.sh"
_RUSTUP_DOWNLOAD_URL_WINDOWS: str = "https://static.rust-lang.org/rustup/dist/x86_64-pc-windows-msvc/rustup-init.exe"
# ============================================================================
# 私有辅助
# ============================================================================
def _pip_config_path() -> Path:
"""返回当前平台的 pip 配置文件路径."""
if Constants.IS_LINUX:
return Path.home() / ".pip" / "pip.conf"
return Path.home() / "pip" / "pip.ini"
# ============================================================================
# 镜像源配置函数
# ============================================================================
@px.tool("envdev", subcommand="setup-python", help="配置 Python 镜像源")
def setup_python_mirror(mirror: str = "tsinghua") -> None:
"""配置 Python 镜像源 (设置环境变量 + 写入 pip 配置文件).
设置 ``PIP_INDEX_URL`` / ``PIP_TRUSTED_HOSTS`` / ``UV_INDEX_URL`` /
``UV_PYTHON_INSTALL_MIRROR`` 等环境变量, 并写入 pip 配置文件.
Parameters
----------
mirror : str
镜像源名称: tsinghua/aliyun/huaweicloud/ustc/zju (默认: tsinghua)
"""
if mirror not in _PIP_INDEX_URLS:
print(f"未知 Python 镜像源: {mirror}")
return
index_url = _PIP_INDEX_URLS[mirror]
trusted_host = _PIP_TRUSTED_HOSTS[mirror]
os.environ["PIP_INDEX_URL"] = index_url
os.environ["PIP_TRUSTED_HOSTS"] = trusted_host
os.environ["UV_INDEX_URL"] = index_url
os.environ["UV_PYTHON_INSTALL_MIRROR"] = _UV_PYTHON_INSTALL_MIRROR
os.environ["UV_HTTP_TIMEOUT"] = "600"
os.environ["UV_LINK_MODE"] = "copy"
config_path = _pip_config_path()
config_path.parent.mkdir(parents=True, exist_ok=True)
content = f"[global]\nindex-url = {index_url}\ntrusted-host = {trusted_host}\n"
config_path.write_text(content, encoding="utf-8")
print(f"Python 镜像源已配置: {mirror} -> {config_path}")
@px.tool("envdev", subcommand="setup-conda", help="配置 Conda 镜像源")
def setup_conda_mirror(mirror: str = "tsinghua") -> None:
"""配置 Conda 镜像源 (写入 ~/.condarc).
Parameters
----------
mirror : str
镜像源名称: tsinghua/ustc/bsfu/aliyun (默认: tsinghua)
"""
if mirror not in _CONDA_MIRROR_URLS:
print(f"未知 Conda 镜像源: {mirror}")
return
urls = _CONDA_MIRROR_URLS[mirror]
config_path = Path.home() / ".condarc"
config_path.parent.mkdir(parents=True, exist_ok=True)
content = "show_channel_urls: true\nchannels:\n - " + "\n - ".join(urls) + "\n - defaults\n"
config_path.write_text(content, encoding="utf-8")
print(f"Conda 镜像源已配置: {mirror} -> {config_path}")
@px.tool("envdev", subcommand="setup-rust", help="配置 Rust 镜像源")
def setup_rust_mirror(mirror: str = "tsinghua", version: str = "stable") -> None:
"""配置 Rust 镜像源 (设置环境变量 + 写入 cargo config + 创建 sccache 目录).
设置 ``RUSTUP_DIST_SERVER`` / ``RUSTUP_UPDATE_ROOT`` / ``RUST_SCCACHE_DIR``
等环境变量, 写入 ``~/.cargo/config.toml``, 并创建 sccache 缓存目录.
Parameters
----------
mirror : str
镜像源名称: tsinghua/ustc/aliyun (默认: tsinghua)
version : str
Rust 版本 (未使用, 保留以与原 envdev 参数对齐)
"""
del version # 兼容旧参数, 实际安装由独立 job 处理
if mirror not in _RUSTUP_MIRRORS:
print(f"未知 Rust 镜像源: {mirror}")
return
mirrors = _RUSTUP_MIRRORS[mirror]
os.environ["RUSTUP_DIST_SERVER"] = mirrors["RUSTUP_DIST_SERVER"]
os.environ["RUSTUP_UPDATE_ROOT"] = mirrors["RUSTUP_UPDATE_ROOT"]
os.environ["RUST_SCCACHE_DIR"] = str(_RUST_SCCACHE_DIR)
os.environ["RUST_SCCACHE_CACHE_SIZE"] = _RUST_SCCACHE_CACHE_SIZE
_RUST_SCCACHE_DIR.mkdir(parents=True, exist_ok=True)
config_path = Path.home() / ".cargo" / "config.toml"
config_path.parent.mkdir(parents=True, exist_ok=True)
registry = mirrors["TOML_REGISTRY"]
content = (
f"\n[source.crates-io]\nreplace-with = '{mirror}'\n\n"
f'[source.{mirror}]\nregistry = "sparse+{registry}"\n\n'
f'[registries.{mirror}]\nindex = "sparse+{registry}"\n'
)
config_path.write_text(content, encoding="utf-8")
print(f"Rust 镜像源已配置: {mirror} -> {config_path}")
# ============================================================================
# Rust 工具链安装
# ============================================================================
@px.tool("envdev", subcommand="download-rustup", help="下载 Rustup 安装脚本")
def download_rustup_script() -> None:
"""下载 Rustup 安装脚本 (跨平台, 已安装 rustup 时跳过).
Linux 下载 ``rustup-init.sh``, Windows 下载 ``rustup-init.exe``.
"""
if shutil.which("rustup") is not None:
print("rustup 已安装, 跳过下载")
return
if Constants.IS_WINDOWS:
print("下载 rustup-init.exe...")
subprocess.run(
[
"powershell",
"-Command",
"Invoke-WebRequest",
"-Uri",
_RUSTUP_DOWNLOAD_URL_WINDOWS,
"-OutFile",
"rustup-init.exe",
],
check=False,
)
else:
print("下载 rustup-init.sh...")
subprocess.run(
["curl", "-fsSL", _RUSTUP_DOWNLOAD_URL_LINUX, "-o", "rustup-init.sh"],
check=False,
)
@px.tool(
"envdev",
subcommand="install-rust",
help="安装 Rust 工具链",
needs=["setup-rust", "download-rustup"],
allow_upstream_skip=True,
)
def install_rust_toolchain(version: str = "stable") -> None:
"""安装 Rust 工具链 (rustup 未安装时跳过).
Parameters
----------
version : str
Rust 版本: ``stable`` / ``nightly`` / ``beta`` (默认: ``stable``)
"""
if shutil.which("rustup") is None:
print("rustup 未安装, 跳过工具链安装")
return
subprocess.run(["rustup", "toolchain", "install", version], check=False)
print(f"Rust 工具链 {version} 安装完成")
# ============================================================================
# Linux 专用函数
# ============================================================================
@px.tool("envdev", subcommand="setup-linux-mirror", help="配置 Linux 系统镜像源")
def setup_linux_system_mirror() -> None:
"""下载并安装 Linux 系统镜像源 (仅 Linux, 已配置国内镜像时跳过).
检查 ``/etc/apt/sources.list`` 与 ``/etc/apt/sources.list.d/ubuntu.sources``
是否已配置国内镜像, 已配置则跳过; 未配置则下载并执行 linuxmirrors 脚本.
"""
if not Constants.IS_LINUX:
print("setup_linux_system_mirror: 仅在 Linux 上执行")
return
apt_files = ["/etc/apt/sources.list", "/etc/apt/sources.list.d/ubuntu.sources"]
mirror_keys = list(_PIP_INDEX_URLS.keys())
already_configured = False
for apt_file in apt_files:
try:
content = Path(apt_file).read_text(encoding="utf-8")
except (OSError, UnicodeDecodeError):
continue
if any(mirror in content for mirror in mirror_keys):
already_configured = True
break
if already_configured:
print("已配置国内镜像源, 跳过系统镜像配置")
return
print("下载 linuxmirrors 脚本...")
subprocess.run(_DOWNLOAD_MIRROR_SCRIPT, shell=True, check=False)
print("安装 linuxmirrors...")
subprocess.run(_INSTALL_MIRROR_SCRIPT, shell=True, check=False)
@px.tool(
"envdev",
subcommand="install-qt-libs",
help="安装 Qt 依赖库",
needs=["setup-linux-mirror"],
allow_upstream_skip=True,
)
def install_linux_qt_libs() -> None:
"""安装 Qt 依赖库 (仅 Linux)."""
if not Constants.IS_LINUX:
print("install_linux_qt_libs: 仅在 Linux 上执行")
return
subprocess.run(["sudo", "apt", "install", "-y", *_QT_LIBS], check=False)
print("Qt 依赖库安装完成")
@px.tool(
"envdev",
subcommand="install-fonts",
help="安装中文字体",
needs=["setup-linux-mirror"],
allow_upstream_skip=True,
)
def install_linux_fonts() -> None:
"""安装中文字体 (仅 Linux)."""
if not Constants.IS_LINUX:
print("install_linux_fonts: 仅在 Linux 上执行")
return
subprocess.run(["sudo", "apt", "install", "-y", *_CHINESE_FONTS], check=False)
print("中文字体安装完成")
@px.tool(
"envdev",
subcommand="install-docker",
help="安装 Docker",
needs=["setup-linux-mirror"],
allow_upstream_skip=True,
)
def install_linux_docker() -> None:
"""安装 Docker (仅 Linux)."""
if not Constants.IS_LINUX:
print("install_linux_docker: 仅在 Linux 上执行")
return
subprocess.run(["sudo", "apt", "install", "-y", "docker-compose-v2"], check=False)
subprocess.run(["sudo", "usermod", "-aG", "docker", getpass.getuser()], check=False)
print("Docker 安装完成 (需重新登录以生效 docker 用户组)")
@@ -1,18 +1,26 @@
"""文件日期处理工具.
"""filedate - 文件日期处理工具.
自动检测文件名的日期前缀,
并根据文件的实际创建或修改时间重命名文件.
提供添加日期前缀/清除日期前缀 子命令.
"""
from __future__ import annotations
import argparse
import re
import time
from pathlib import Path
import pyflowx as px
__all__ = [
"DATE_PATTERN",
"SEP",
"add_date_prefix",
"get_file_timestamp",
"process_file_date",
"process_files_date",
"remove_date_prefix",
]
# ============================================================================
# 配置
# ============================================================================
@@ -22,7 +30,7 @@ SEP = "_"
# ============================================================================
# 辅助函数
# 公共函数
# ============================================================================
@@ -69,11 +77,34 @@ def process_file_date(filepath: Path, clear: bool = False) -> None:
if clear:
remove_date_prefix(filepath)
else:
# 先移除旧日期前缀,再添加新日期前缀
new_path = remove_date_prefix(filepath)
add_date_prefix(new_path)
@px.tool("filedate", subcommand="add", help="添加日期前缀")
def process_files_date_add(files: list[Path]) -> None:
"""添加日期前缀.
Parameters
----------
files : list[Path]
文件路径列表
"""
process_files_date(files, clear=False)
@px.tool("filedate", subcommand="clear", help="清除日期前缀")
def process_files_date_clear(files: list[Path]) -> None:
"""清除日期前缀.
Parameters
----------
files : list[Path]
文件路径列表
"""
process_files_date(files, clear=True)
def process_files_date(targets: list[Path], clear: bool = False) -> None:
"""批量处理文件日期前缀.
@@ -87,51 +118,3 @@ def process_files_date(targets: list[Path], clear: bool = False) -> None:
for target in targets:
if target.exists() and not target.name.startswith("."):
process_file_date(target, clear)
# ============================================================================
# CLI Runner
# ============================================================================
def main() -> None:
"""文件日期处理工具主函数."""
parser = argparse.ArgumentParser(
description="FileDate - 文件日期处理工具",
usage="filedate <command> [options]",
)
subparsers = parser.add_subparsers(dest="command", help="可用命令")
# 添加日期前缀命令
add_parser = subparsers.add_parser("add", help="添加日期前缀")
add_parser.add_argument("files", nargs="+", help="文件路径")
# 清除日期前缀命令
clear_parser = subparsers.add_parser("clear", help="清除日期前缀")
clear_parser.add_argument("files", nargs="+", help="文件路径")
args = parser.parse_args()
if args.command == "add":
graph = px.Graph.from_specs([
px.TaskSpec(
"process_files_date",
fn=process_files_date,
args=([Path(f) for f in args.files],),
kwargs={"clear": False},
)
])
elif args.command == "clear":
graph = px.Graph.from_specs([
px.TaskSpec(
"process_files_date",
fn=process_files_date,
args=([Path(f) for f in args.files],),
kwargs={"clear": True},
)
])
else:
parser.print_help()
return
px.run(graph, strategy="thread")
@@ -1,16 +1,22 @@
"""文件等级重命名工具.
"""filelevel - 文件等级重命名工具.
根据文件等级配置自动重命名文件,
支持多种等级标识和括号格式.
提供设置文件等级 子命令.
"""
from __future__ import annotations
import argparse
from pathlib import Path
import pyflowx as px
__all__ = [
"BRACKETS",
"LEVELS",
"process_file_level",
"process_files_level",
"remove_marks",
]
# ============================================================================
# 配置
# ============================================================================
@@ -27,7 +33,7 @@ BRACKETS: tuple[str, str] = (" ([_(【-", " )]_)】")
# ============================================================================
# 辅助函数
# 公共函数
# ============================================================================
@@ -69,72 +75,34 @@ def process_file_level(filepath: Path, level: int = 0) -> None:
filestem = filepath.stem
original_stem = filestem
# 移除所有等级标记
for level_names in LEVELS.values():
if level_names:
filestem = remove_marks(filestem, level_names.split(","))
# 移除数字标记
for digit in map(str, range(1, 10)):
filestem = remove_marks(filestem, [digit])
# 添加等级标记
if level > 0:
levelstr = LEVELS.get(str(level), "").split(",")[0]
if levelstr:
filestem = f"{filestem}({levelstr})"
# 重命名文件
if filestem != original_stem:
new_path = filepath.with_name(filestem + filepath.suffix)
filepath.rename(new_path)
print(f"重命名: {filepath} -> {new_path}")
def process_files_level(targets: list[Path], level: int = 0) -> None:
@px.tool("filelevel", subcommand="set", help="设置文件等级")
def process_files_level(files: list[Path], level: int = 0) -> None:
"""批量处理文件等级标记.
Parameters
----------
targets : list[Path]
files : list[Path]
文件路径列表
level : int
文件等级 (0-4)
"""
for target in targets:
for target in files:
process_file_level(target, level)
# ============================================================================
# CLI Runner
# ============================================================================
def main() -> None:
"""文件等级重命名工具主函数."""
parser = argparse.ArgumentParser(
description="FileLevel - 文件等级重命名工具",
usage="filelevel <command> [options]",
)
subparsers = parser.add_subparsers(dest="command", help="可用命令")
# 设置等级命令
level_parser = subparsers.add_parser("set", help="设置文件等级")
level_parser.add_argument("files", nargs="+", help="文件路径")
level_parser.add_argument("--level", type=int, choices=[0, 1, 2, 3, 4], required=True, help="文件等级 (0-4)")
args = parser.parse_args()
if args.command == "set":
graph = px.Graph.from_specs(
[
px.TaskSpec(
"process_files_level", fn=process_files_level, args=([Path(f) for f in args.files], args.level)
)
]
)
else:
parser.print_help()
return
px.run(graph, strategy="thread")
@@ -1,7 +1,6 @@
"""文件夹备份工具.
"""folderback - 文件夹备份工具.
备份文件和文件夹为 zip 文件,
自动删除超过最大数量的旧备份文件.
备份当前文件夹到指定目录, 自动清理旧备份.
"""
from __future__ import annotations
@@ -12,8 +11,16 @@ from pathlib import Path
import pyflowx as px
__all__ = [
"backup_folder",
"folderback_default",
"remove_dump",
"zip_target",
]
# ============================================================================
# 辅助函数
# 公共函数
# ============================================================================
@@ -40,17 +47,18 @@ def zip_target(src: Path, dst: Path, max_zip: int) -> None:
print(f"备份完成: {target_path}")
def backup_folder(src: str, dst: str, max_zip: int = 5) -> None:
@px.tool("folderback", help="备份文件夹")
def backup_folder(src: str = ".", dst: str = "./backup", max_zip: int = 5) -> None:
"""备份文件夹.
Parameters
----------
src : str
源文件夹路径
源文件夹路径 (默认: 当前目录)
dst : str
目标文件夹路径
目标文件夹路径 (默认: ./backup)
max_zip : int
最大备份数量
最大备份数量 (默认: 5)
"""
src_path = Path(src)
dst_path = Path(dst)
@@ -66,20 +74,6 @@ def backup_folder(src: str, dst: str, max_zip: int = 5) -> None:
zip_target(src_path, dst_path, max_zip)
@px.task
def folderback_default() -> None:
"""备份当前目录到 ./backup."""
backup_folder(".", "./backup", 5)
def main() -> None:
"""文件夹备份工具主函数."""
runner = px.CliRunner(
strategy="thread",
description="FolderBack - 文件夹备份工具",
aliases={
# 备份当前目录到 ./backup
"b": folderback_default,
},
)
runner.run_cli()
@@ -1,7 +1,6 @@
"""文件夹压缩工具.
"""folderzip - 文件夹压缩工具.
压缩目录下的所有文件/文件夹为 zip 文件,
默认压缩当前目录下的所有子文件夹.
压缩当前目录下的所有文件夹为 zip 文件.
"""
from __future__ import annotations
@@ -11,6 +10,15 @@ from pathlib import Path
import pyflowx as px
__all__ = [
"IGNORE_DIRS",
"IGNORE_EXT",
"IGNORE_FILES",
"archive_folder",
"folderzip_default",
"zip_folders",
]
# ============================================================================
# 配置
# ============================================================================
@@ -22,7 +30,7 @@ IGNORE_EXT: list[str] = [".zip", ".rar", ".7z", ".tar", ".gz"]
# ============================================================================
# 辅助函数
# 公共函数
# ============================================================================
@@ -36,13 +44,14 @@ def archive_folder(folder: Path) -> None:
print(f"压缩完成: {folder.name}.zip")
@px.tool("folderzip", help="压缩文件夹")
def zip_folders(cwd: str = ".") -> None:
"""压缩目录下的所有文件夹.
Parameters
----------
cwd : str
工作目录
工作目录 (默认: 当前目录)
"""
cwd_path = Path(cwd)
if not cwd_path.exists():
@@ -57,20 +66,6 @@ def zip_folders(cwd: str = ".") -> None:
archive_folder(dir_path)
@px.task
def folderzip_default() -> None:
"""压缩当前目录下的所有文件夹."""
zip_folders(".")
def main() -> None:
"""文件夹压缩工具主函数."""
runner = px.CliRunner(
strategy="thread",
description="FolderZip - 文件夹压缩工具",
aliases={
# 压缩当前目录下的所有文件夹
"z": folderzip_default,
},
)
runner.run_cli()
+148
View File
@@ -0,0 +1,148 @@
"""gittool - Git 执行工具.
提供添加提交/清理/初始化/推送/拉取 子命令.
"""
from __future__ import annotations
import subprocess
from pathlib import Path
import pyflowx as px
EXCLUDE_DIRS = [
# 编辑器相关目录
".vscode",
".idea",
".editorconfig",
".trae",
".qoder",
# 项目相关目录
".venv",
".git",
".ruff_cache",
".tox",
"node_modules",
]
EXCLUDE_CMDS = [arg for d in EXCLUDE_DIRS for arg in ["-e", d]]
# ============================================================================
# 私有辅助函数
# ============================================================================
def not_has_git_repo() -> bool:
"""检查当前目录没有 Git 仓库."""
return not Path.cwd().exists() or not (Path.cwd() / ".git").is_dir()
def has_files() -> bool:
"""检查当前 Git 仓库是否有未提交的更改."""
try:
result = subprocess.run(
["git", "status", "--porcelain"],
capture_output=True,
text=True,
check=False,
)
return bool(result.stdout.strip())
except (subprocess.SubprocessError, OSError):
return False
# ============================================================================
# fn 子命令
# ============================================================================
@px.tool("gittool", subcommand="a", help="添加并提交")
def git_add_commit(message: str = "chore: update") -> None:
"""执行 git add + git commit (仅当有未提交更改时).
Parameters
----------
message : str
提交信息 (默认: ``chore: update``)
"""
if not has_files():
print("没有文件需要提交")
return
subprocess.run(["git", "add", "."], check=True)
subprocess.run(["git", "commit", "-m", message], check=True)
@px.tool("gittool", subcommand="i", help="初始化并提交")
def git_init_add_commit(message: str = "init commit") -> None:
"""执行 git init (若需) + git add + git commit (若有更改).
Parameters
----------
message : str
提交信息 (默认: ``init commit``)
"""
if not_has_git_repo():
subprocess.run(["git", "init"], check=True)
if has_files():
subprocess.run(["git", "add", "."], check=True)
subprocess.run(["git", "commit", "-m", message], check=True)
else:
print("没有文件需要提交")
@px.tool("gittool", subcommand="isub", help="初始化子目录")
def init_sub_dirs() -> None:
"""初始化子目录的 Git 仓库."""
sub_dirs = [subdir for subdir in Path.cwd().iterdir() if subdir.is_dir()]
for subdir in sub_dirs:
px.run(
px.Graph().chain(
px.cmd(["git", "init"], conditions=(lambda _: not_has_git_repo(),), cwd=subdir),
px.cmd(["git", "add", "."]),
px.cmd(["git", "commit", "-m", "init commit"]),
),
)
# 兼容别名: 与 dev.py 旧函数名对齐, 便于外部调用
a = git_add_commit
i = git_init_add_commit
isub = init_sub_dirs
# ============================================================================
# cmd 子命令
# ============================================================================
@px.tool(
"gittool",
subcommand="clean",
help="清理未跟踪文件",
cmd=["git", "clean", "-xfd", *EXCLUDE_CMDS],
hidden=True,
)
def clean() -> None:
"""清理 Git 未跟踪文件 (隐藏命令, 被 c 依赖)."""
@px.tool(
"gittool",
subcommand="c",
help="清理并查看状态",
cmd=["git", "status", "--porcelain"],
needs=["clean"],
)
def c() -> None:
"""清理未跟踪文件并查看 Git 状态."""
@px.tool("gittool", subcommand="p", help="推送", cmd=["git", "push"])
def p() -> None:
"""推送代码到远程仓库."""
@px.tool("gittool", subcommand="pl", help="拉取", cmd=["git", "pull"])
def pl() -> None:
"""从远程仓库拉取代码."""
@@ -1,37 +1,19 @@
"""LS-DYNA 计算工具.
"""lscalc - LS-DYNA 计算工具.
用于管理 LS-DYNA 仿真计算任务,
支持启动监控和管理计算进程.
运行 LS-DYNA 计算 (单机/MPI) 与进程状态检查.
"""
from __future__ import annotations
import argparse
import subprocess
from pathlib import Path
import pyflowx as px
from pyflowx.conditions import Constants
# ============================================================================
# 配置
# ============================================================================
LS_DYNA_COMMANDS: dict[str, list[str]] = {
"windows": ["ls-dyna_mpp", "i=input.k", "ncpu=4"],
"linux": ["ls-dyna_mpp", "i=input.k", "ncpu=8"],
"macos": ["ls-dyna_mpp", "i=input.k", "ncpu=4"],
}
DEFAULT_INPUT_FILE: str = "input.k"
DEFAULT_NCPU: int = 4
# ============================================================================
# 辅助函数
# ============================================================================
def get_ls_dyna_command(input_file: str, ncpu: int) -> list[str]:
"""获取 LS-DYNA 命令.
@@ -53,6 +35,7 @@ def get_ls_dyna_command(input_file: str, ncpu: int) -> list[str]:
return ["ls-dyna_mpp", f"i={input_file}", f"ncpu={ncpu}"]
@px.tool("lscalc", subcommand="run", help="运行 LS-DYNA 计算")
def run_ls_dyna(input_file: str, ncpu: int = DEFAULT_NCPU) -> None:
"""运行 LS-DYNA 计算.
@@ -78,6 +61,7 @@ def run_ls_dyna(input_file: str, ncpu: int = DEFAULT_NCPU) -> None:
print(f"LS-DYNA 计算失败: {e}")
@px.tool("lscalc", subcommand="mpi", help="运行 LS-DYNA MPI 计算")
def run_ls_dyna_mpi(input_file: str, ncpu: int = DEFAULT_NCPU) -> None:
"""运行 LS-DYNA MPI 计算.
@@ -103,6 +87,7 @@ def run_ls_dyna_mpi(input_file: str, ncpu: int = DEFAULT_NCPU) -> None:
print(f"LS-DYNA MPI 计算失败: {e}")
@px.tool("lscalc", subcommand="status", help="检查 LS-DYNA 进程状态")
def check_ls_dyna_status() -> None:
"""检查 LS-DYNA 进程状态."""
try:
@@ -127,48 +112,3 @@ def check_ls_dyna_status() -> None:
print("没有运行中的 LS-DYNA 进程")
except subprocess.CalledProcessError as e:
print(f"检查进程状态失败: {e}")
# ============================================================================
# CLI Runner
# ============================================================================
def main() -> None:
"""LS-DYNA 计算工具主函数."""
parser = argparse.ArgumentParser(
description="LSCalc - LS-DYNA 计算工具",
usage="lscalc <command> [options]",
)
subparsers = parser.add_subparsers(dest="command", help="可用命令")
# 运行计算命令
run_parser = subparsers.add_parser("run", help="运行 LS-DYNA 计算")
run_parser.add_argument("input_file", help="输入文件路径")
run_parser.add_argument("--ncpu", type=int, default=DEFAULT_NCPU, help="CPU 核心数")
# 运行 MPI 计算命令
mpi_parser = subparsers.add_parser("mpi", help="运行 LS-DYNA MPI 计算")
mpi_parser.add_argument("input_file", help="输入文件路径")
mpi_parser.add_argument("--ncpu", type=int, default=DEFAULT_NCPU, help="CPU 核心数")
# 检查进程状态命令
subparsers.add_parser("status", help="检查 LS-DYNA 进程状态")
args = parser.parse_args()
if args.command == "run":
graph = px.Graph.from_specs(
[px.TaskSpec("run_ls_dyna", fn=run_ls_dyna, args=(args.input_file,), kwargs={"ncpu": args.ncpu})]
)
elif args.command == "mpi":
graph = px.Graph.from_specs(
[px.TaskSpec("run_ls_dyna_mpi", fn=run_ls_dyna_mpi, args=(args.input_file,), kwargs={"ncpu": args.ncpu})]
)
elif args.command == "status":
graph = px.Graph.from_specs([px.TaskSpec("check_ls_dyna_status", fn=check_ls_dyna_status)])
else:
parser.print_help()
return
px.run(graph, strategy="thread")
+41
View File
@@ -0,0 +1,41 @@
"""msdownload - ModelScope 模型/数据集下载工具.
从 ModelScope 下载模型、数据集或空间.
"""
from __future__ import annotations
import subprocess
from pathlib import Path
import pyflowx as px
__all__ = ["msdownload_run"]
@px.tool("msdownload", help="ModelScope 模型/数据集下载工具")
def msdownload_run(name: str, target_type: str = "model", download_dir: str | None = None) -> None:
"""从 ModelScope 下载模型/数据集/空间.
Parameters
----------
name : str
目标名称 (如: ``Qwen/Qwen2.5-Coder-32B-Instruct``)
target_type : str
目标类型: ``model`` / ``dataset`` / ``space`` (默认: ``model``)
download_dir : str | None
下载目录; 为 None 时默认 ``~/.models/<name 最后一段>``
"""
if not name:
print("msdownload: name 不能为空")
return
if download_dir:
out_dir = Path(download_dir)
else:
out_dir = Path.home() / ".models" / name.rsplit("/", 1)[-1]
out_dir.mkdir(parents=True, exist_ok=True)
cmd = ["uvx", "modelscope", "download", f"--{target_type}", name, "--local_dir", str(out_dir)]
print(f"下载 {target_type}: {name} -> {out_dir}")
subprocess.run(cmd, check=False)
@@ -1,19 +1,28 @@
"""Python 打包工具模块.
"""packtool - Python 打包工具.
提供 Python 项目打包的常用功能封装,
支持源码打包依赖打包嵌入式 Python 安装等功能.
提供源码打包/依赖打包/wheel 构建/嵌入式 Python 安装/zip 包创建/清理 子命令.
"""
from __future__ import annotations
import argparse
import platform
import shutil
import subprocess
import urllib.request
import zipfile
from pathlib import Path
import pyflowx as px
__all__ = [
"clean_build_dir",
"create_zip_package",
"install_embed_python",
"pack_dependencies",
"pack_source",
"pack_wheel",
]
# ============================================================================
# 配置
# ============================================================================
@@ -41,23 +50,23 @@ IGNORE_PATTERNS = [
# ============================================================================
# 辅助函数
# 公共函数
# ============================================================================
def pack_source(project_dir: Path, output_dir: Path) -> None:
@px.tool("packtool", subcommand="src", help="打包源码")
def pack_source(project_dir: Path = Path(), output_dir: Path = Path(".pypack")) -> None:
"""打包项目源码.
Parameters
----------
project_dir : Path
项目目录
项目目录 (默认: 当前目录)
output_dir : Path
输出目录
输出目录 (默认: .pypack)
"""
output_dir.mkdir(parents=True, exist_ok=True)
# 检测项目名称
pyproject_file = project_dir / "pyproject.toml"
project_name = project_dir.name
@@ -71,11 +80,9 @@ def pack_source(project_dir: Path, output_dir: Path) -> None:
except ImportError:
pass
# 打包源码
source_dir = output_dir / "src" / project_name
source_dir.mkdir(parents=True, exist_ok=True)
# 复制文件
src_subdir = project_dir / "src"
if src_subdir.exists():
shutil.copytree(
@@ -102,23 +109,26 @@ def pack_source(project_dir: Path, output_dir: Path) -> None:
print(f"源码打包完成: {source_dir}")
def pack_dependencies(lib_dir: Path, dependencies: list[str]) -> None:
@px.tool("packtool", subcommand="deps", help="打包依赖")
def pack_dependencies(lib_dir: Path = Path("libs"), dependencies: list[str] | None = None) -> None:
"""打包项目依赖.
Parameters
----------
lib_dir : Path
依赖库目录
dependencies : list[str]
依赖库目录 (默认: libs)
dependencies : list[str] | None
依赖列表
"""
if dependencies is None:
dependencies = []
lib_dir.mkdir(parents=True, exist_ok=True)
if not dependencies:
print("没有依赖需要打包")
return
# 使用 pip 安装依赖到目标目录
cmd = [
"pip",
"install",
@@ -133,19 +143,19 @@ def pack_dependencies(lib_dir: Path, dependencies: list[str]) -> None:
print(f"依赖打包完成: {lib_dir}")
def pack_wheel(project_dir: Path, output_dir: Path) -> None:
@px.tool("packtool", subcommand="wheel", help="构建 wheel")
def pack_wheel(project_dir: Path = Path(), output_dir: Path = Path("dist")) -> None:
"""打包项目为 wheel 文件.
Parameters
----------
project_dir : Path
项目目录
项目目录 (默认: 当前目录)
output_dir : Path
输出目录
输出目录 (默认: dist)
"""
output_dir.mkdir(parents=True, exist_ok=True)
# 使用 pip wheel 打包
cmd = [
"pip",
"wheel",
@@ -159,28 +169,25 @@ def pack_wheel(project_dir: Path, output_dir: Path) -> None:
print(f"Wheel 打包完成: {output_dir}")
def install_embed_python(version: str, output_dir: Path) -> None:
@px.tool("packtool", subcommand="embed", help="安装嵌入式 Python")
def install_embed_python(version: str = "3.10", output_dir: Path = Path("python")) -> None:
"""安装嵌入式 Python.
Parameters
----------
version : str
Python 版本 (: 3.10, 3.11)
Python 版本 (: 3.10, 3.11), 默认 3.10
output_dir : Path
输出目录
输出目录 (默认: python)
"""
import platform
output_dir.mkdir(parents=True, exist_ok=True)
# 构建下载 URL
arch = platform.machine().lower()
if arch in ["x86_64", "amd64"]:
arch = "amd64"
elif arch in ["arm64", "aarch64"]:
arch = "arm64"
# 解析完整版本号
version_map = {
"3.8": "3.8.10",
"3.9": "3.9.13",
@@ -190,36 +197,32 @@ def install_embed_python(version: str, output_dir: Path) -> None:
}
full_version = version_map.get(version, f"{version}.0")
# Windows 嵌入式 Python 下载 URL
url = f"https://www.python.org/ftp/python/{full_version}/python-{full_version}-embed-{arch}.zip"
# 下载并解压
cache_file = Path(DEFAULT_CACHE_DIR) / f"python-{full_version}-embed-{arch}.zip"
cache_file.parent.mkdir(parents=True, exist_ok=True)
if not cache_file.exists():
print(f"正在下载嵌入式 Python {full_version}...")
import urllib.request
urllib.request.urlretrieve(url, cache_file)
print(f"下载完成: {cache_file}")
# 解压
with zipfile.ZipFile(cache_file, "r") as zf:
zf.extractall(output_dir)
print(f"嵌入式 Python 安装完成: {output_dir}")
def create_zip_package(source_dir: Path, output_file: Path) -> None:
@px.tool("packtool", subcommand="zip", help="创建 zip 包")
def create_zip_package(source_dir: Path = Path(), output_file: Path = Path("package.zip")) -> None:
"""创建 ZIP 打包文件.
Parameters
----------
source_dir : Path
源目录
源目录 (默认: 当前目录)
output_file : Path
输出文件
输出文件 (默认: package.zip)
"""
output_file.parent.mkdir(parents=True, exist_ok=True)
@@ -232,118 +235,17 @@ def create_zip_package(source_dir: Path, output_file: Path) -> None:
print(f"ZIP 打包完成: {output_file}")
def clean_build_dir(build_dir: Path) -> None:
@px.tool("packtool", subcommand="clean", help="清理构建目录")
def clean_build_dir(build_dir: Path = Path(".pypack")) -> None:
"""清理构建目录.
Parameters
----------
build_dir : Path
构建目录
构建目录 (默认: .pypack)
"""
if build_dir.exists():
shutil.rmtree(build_dir)
print(f"清理完成: {build_dir}")
else:
print(f"目录不存在: {build_dir}")
# ============================================================================
# CLI Runner
# ============================================================================
def main() -> None:
"""Python 打包工具主函数."""
parser = argparse.ArgumentParser(
description="PackTool - Python 打包工具",
usage="packtool <command> [options]",
)
subparsers = parser.add_subparsers(dest="command", help="可用命令")
# 源码打包命令
src_parser = subparsers.add_parser("src", help="打包项目源码")
src_parser.add_argument("--project-dir", type=str, default=".", help="项目目录")
src_parser.add_argument("--output-dir", type=str, default=DEFAULT_BUILD_DIR, help="输出目录")
# 依赖打包命令
deps_parser = subparsers.add_parser("deps", help="打包项目依赖")
deps_parser.add_argument("--lib-dir", type=str, default=DEFAULT_LIB_DIR, help="依赖库目录")
deps_parser.add_argument("dependencies", nargs="*", help="依赖列表")
# Wheel 打包命令
wheel_parser = subparsers.add_parser("wheel", help="打包项目为 wheel 文件")
wheel_parser.add_argument("--project-dir", type=str, default=".", help="项目目录")
wheel_parser.add_argument("--output-dir", type=str, default=DEFAULT_DIST_DIR, help="输出目录")
# 嵌入式 Python 安装命令
embed_parser = subparsers.add_parser("embed", help="安装嵌入式 Python")
embed_parser.add_argument("--version", type=str, default="3.10", help="Python 版本")
embed_parser.add_argument("--output-dir", type=str, default="python", help="输出目录")
# ZIP 打包命令
zip_parser = subparsers.add_parser("zip", help="创建 ZIP 打包文件")
zip_parser.add_argument("--source-dir", type=str, default=".", help="源目录")
zip_parser.add_argument("--output-file", type=str, default="package.zip", help="输出文件")
# 清理命令
subparsers.add_parser("clean", help="清理构建目录")
args = parser.parse_args()
if args.command == "src":
graph = px.Graph.from_specs(
[
px.TaskSpec(
"pack_source",
fn=pack_source,
args=(Path(args.project_dir), Path(args.output_dir)),
)
]
)
elif args.command == "deps":
graph = px.Graph.from_specs(
[
px.TaskSpec(
"pack_deps",
fn=pack_dependencies,
args=(Path(args.lib_dir), args.dependencies),
)
]
)
elif args.command == "wheel":
graph = px.Graph.from_specs(
[
px.TaskSpec(
"pack_wheel",
fn=pack_wheel,
args=(Path(args.project_dir), Path(args.output_dir)),
)
]
)
elif args.command == "embed":
graph = px.Graph.from_specs(
[
px.TaskSpec(
"install_embed",
fn=install_embed_python,
args=(args.version, Path(args.output_dir)),
)
]
)
elif args.command == "zip":
graph = px.Graph.from_specs(
[
px.TaskSpec(
"create_zip",
fn=create_zip_package,
args=(Path(args.source_dir), Path(args.output_file)),
)
]
)
elif args.command == "clean":
graph = px.Graph.from_specs([px.TaskSpec("clean_build", fn=clean_build_dir, args=(Path(DEFAULT_BUILD_DIR),))])
else:
parser.print_help()
return
px.run(graph, strategy="thread")
+511
View File
@@ -0,0 +1,511 @@
"""pdftool - PDF 文件工具集.
提供 PDF 合并/拆分/压缩/加密/解密/提取文本/提取图片/水印/旋转/裁剪/
信息/OCR/转图片/修复 等子命令.
"""
from __future__ import annotations
from pathlib import Path
import pyflowx as px
__all__ = [
"pdf_add_watermark",
"pdf_compress",
"pdf_crop",
"pdf_decrypt",
"pdf_encrypt",
"pdf_extract_images",
"pdf_extract_text",
"pdf_info",
"pdf_merge",
"pdf_ocr",
"pdf_reorder",
"pdf_repair",
"pdf_rotate",
"pdf_split",
"pdf_to_images",
]
try:
import fitz # PyMuPDF
HAS_PYMUPDF = True
except ImportError:
HAS_PYMUPDF = False
try:
import pypdf
HAS_PYPDF = True
except ImportError:
HAS_PYPDF = False
@px.tool("pdftool", subcommand="m", help="合并 PDF")
def pdf_merge(input_paths: list[Path], output_path: Path = Path("merged.pdf")) -> None:
"""合并多个 PDF 文件.
Parameters
----------
input_paths : list[Path]
输入 PDF 文件列表
output_path : Path
输出文件 (默认: merged.pdf)
"""
if not HAS_PYPDF:
print("未安装 pypdf 库, 请安装: pip install pypdf")
return
writer = pypdf.PdfWriter()
for input_path in input_paths:
if input_path.exists():
reader = pypdf.PdfReader(str(input_path))
for page in reader.pages:
writer.add_page(page)
output_path.parent.mkdir(parents=True, exist_ok=True)
with open(output_path, "wb") as f:
writer.write(f)
print(f"合并完成: {output_path}")
@px.tool("pdftool", subcommand="s", help="拆分 PDF")
def pdf_split(input_path: Path, output_dir: Path = Path("split")) -> None:
"""拆分 PDF 文件为单页.
Parameters
----------
input_path : Path
输入 PDF 文件
output_dir : Path
输出目录 (默认: split)
"""
if not HAS_PYPDF:
print("未安装 pypdf 库, 请安装: pip install pypdf")
return
reader = pypdf.PdfReader(str(input_path))
output_dir.mkdir(parents=True, exist_ok=True)
for i, page in enumerate(reader.pages):
writer = pypdf.PdfWriter()
writer.add_page(page)
output_file = output_dir / f"{input_path.stem}_page_{i + 1}.pdf"
with open(output_file, "wb") as f:
writer.write(f)
print(f"拆分完成: {output_dir}")
@px.tool("pdftool", subcommand="c", help="压缩 PDF")
def pdf_compress(input_path: Path, output_path: Path = Path("compressed.pdf")) -> None:
"""压缩 PDF 文件.
Parameters
----------
input_path : Path
输入 PDF 文件
output_path : Path
输出文件 (默认: compressed.pdf)
"""
if not HAS_PYMUPDF:
print("未安装 PyMuPDF 库, 请安装: pip install PyMuPDF")
return
doc = fitz.open(str(input_path))
output_path.parent.mkdir(parents=True, exist_ok=True)
doc.save(str(output_path), garbage=4, deflate=True, clean=True)
doc.close()
original_size = input_path.stat().st_size
new_size = output_path.stat().st_size
ratio = (1 - new_size / original_size) * 100
print(f"压缩完成: {output_path} (缩小 {ratio:.1f}%)")
@px.tool("pdftool", subcommand="e", help="加密 PDF")
def pdf_encrypt(input_path: Path, output_path: Path = Path("encrypted.pdf"), password: str = "") -> None:
"""加密 PDF 文件.
Parameters
----------
input_path : Path
输入 PDF 文件
output_path : Path
输出文件 (默认: encrypted.pdf)
password : str
密码 (必填)
"""
if not password:
print("错误: --password 为必填参数")
return
if not HAS_PYPDF:
print("未安装 pypdf 库, 请安装: pip install pypdf")
return
reader = pypdf.PdfReader(str(input_path))
writer = pypdf.PdfWriter()
for page in reader.pages:
writer.add_page(page)
writer.encrypt(user_password=password, owner_password=password, use_128bit=True)
output_path.parent.mkdir(parents=True, exist_ok=True)
with open(output_path, "wb") as f:
writer.write(f)
print(f"加密完成: {output_path}")
@px.tool("pdftool", subcommand="d", help="解密 PDF")
def pdf_decrypt(input_path: Path, output_path: Path = Path("decrypted.pdf"), password: str = "") -> None:
"""解密 PDF 文件.
Parameters
----------
input_path : Path
输入 PDF 文件
output_path : Path
输出文件 (默认: decrypted.pdf)
password : str
密码 (必填)
"""
if not password:
print("错误: --password 为必填参数")
return
if not HAS_PYPDF:
print("未安装 pypdf 库, 请安装: pip install pypdf")
return
reader = pypdf.PdfReader(str(input_path))
if reader.is_encrypted:
reader.decrypt(password)
writer = pypdf.PdfWriter()
for page in reader.pages:
writer.add_page(page)
output_path.parent.mkdir(parents=True, exist_ok=True)
with open(output_path, "wb") as f:
writer.write(f)
print(f"解密完成: {output_path}")
@px.tool("pdftool", subcommand="xt", help="提取文本")
def pdf_extract_text(input_path: Path, output_path: Path = Path("output.txt")) -> None:
"""提取 PDF 文本.
Parameters
----------
input_path : Path
输入 PDF 文件
output_path : Path
输出文件 (默认: output.txt)
"""
if not HAS_PYMUPDF:
print("未安装 PyMuPDF 库, 请安装: pip install PyMuPDF")
return
doc = fitz.open(str(input_path))
text = ""
for page in doc:
text += str(page.get_text()) + "\n\n"
doc.close()
output_path.parent.mkdir(parents=True, exist_ok=True)
output_path.write_text(text, encoding="utf-8")
print(f"文本提取完成: {output_path}")
@px.tool("pdftool", subcommand="xi", help="提取图片")
def pdf_extract_images(input_path: Path, output_dir: Path = Path("images")) -> None:
"""提取 PDF 图片.
Parameters
----------
input_path : Path
输入 PDF 文件
output_dir : Path
输出目录 (默认: images)
"""
if not HAS_PYMUPDF:
print("未安装 PyMuPDF 库, 请安装: pip install PyMuPDF")
return
doc = fitz.open(str(input_path))
output_dir.mkdir(parents=True, exist_ok=True)
image_count = 0
# pyrefly: ignore [bad-argument-type]
for page_num, page in enumerate(doc):
images = page.get_images(full=True)
for img_idx, img in enumerate(images):
xref = img[0]
base_image = doc.extract_image(xref)
image_data = base_image["image"]
image_ext = base_image["ext"]
image_path = output_dir / f"page_{page_num + 1}_img_{img_idx + 1}.{image_ext}"
image_path.write_bytes(image_data)
image_count += 1
doc.close()
print(f"图片提取完成: {output_dir} (共 {image_count} 张)")
@px.tool("pdftool", subcommand="w", help="添加水印")
def pdf_add_watermark(
input_path: Path, output_path: Path = Path("watermarked.pdf"), text: str = "CONFIDENTIAL"
) -> None:
"""添加 PDF 水印.
Parameters
----------
input_path : Path
输入 PDF 文件
output_path : Path
输出文件 (默认: watermarked.pdf)
text : str
水印文字 (默认: CONFIDENTIAL)
"""
if not HAS_PYMUPDF:
print("未安装 PyMuPDF 库, 请安装: pip install PyMuPDF")
return
doc = fitz.open(str(input_path))
for page in doc:
rect = page.rect
text_width = fitz.get_text_length(text, fontsize=48)
x = (rect.width - text_width) / 2
y = rect.height / 2
page.insert_text((x, y), text, fontsize=48, rotate=45, color=(0, 0, 0))
output_path.parent.mkdir(parents=True, exist_ok=True)
doc.save(str(output_path))
doc.close()
print(f"水印添加完成: {output_path}")
@px.tool("pdftool", subcommand="r", help="旋转 PDF")
def pdf_rotate(input_path: Path, output_path: Path = Path("rotated.pdf"), rotation: int = 90) -> None:
"""旋转 PDF 页面.
Parameters
----------
input_path : Path
输入 PDF 文件
output_path : Path
输出文件 (默认: rotated.pdf)
rotation : int
旋转角度 (默认: 90)
"""
if not HAS_PYMUPDF:
print("未安装 PyMuPDF 库, 请安装: pip install PyMuPDF")
return
doc = fitz.open(str(input_path))
for page in doc:
page.set_rotation(rotation)
output_path.parent.mkdir(parents=True, exist_ok=True)
doc.save(str(output_path))
doc.close()
print(f"旋转完成: {output_path}")
@px.tool("pdftool", subcommand="crop", help="裁剪 PDF")
def pdf_crop(
input_path: Path,
output_path: Path = Path("cropped.pdf"),
margins: tuple[int, int, int, int] = (10, 10, 10, 10),
) -> None:
"""裁剪 PDF 页面.
Parameters
----------
input_path : Path
输入 PDF 文件
output_path : Path
输出文件 (默认: cropped.pdf)
margins : tuple[int, int, int, int]
边距 (左, 上, 右, 下), 默认 (10, 10, 10, 10)
"""
if not HAS_PYMUPDF:
print("未安装 PyMuPDF 库, 请安装: pip install PyMuPDF")
return
doc = fitz.open(str(input_path))
left, top, right, bottom = margins
for page in doc:
rect = page.rect
new_rect = fitz.Rect(
rect.x0 + left,
rect.y0 + top,
rect.x1 - right,
rect.y1 - bottom,
)
page.set_cropbox(new_rect)
output_path.parent.mkdir(parents=True, exist_ok=True)
doc.save(str(output_path))
doc.close()
print(f"裁剪完成: {output_path}")
@px.tool("pdftool", subcommand="i", help="查看 PDF 信息")
def pdf_info(input_path: Path) -> None:
"""显示 PDF 信息.
Parameters
----------
input_path : Path
输入 PDF 文件
"""
if not HAS_PYMUPDF:
print("未安装 PyMuPDF 库, 请安装: pip install PyMuPDF")
return
doc = fitz.open(str(input_path))
print(f"文件: {input_path}")
print(f"页数: {doc.page_count}")
# pyrefly: ignore [missing-attribute]
print(f"标题: {doc.metadata.get('title', 'N/A')}")
# pyrefly: ignore [missing-attribute]
print(f"作者: {doc.metadata.get('author', 'N/A')}")
# pyrefly: ignore [missing-attribute]
print(f"创建日期: {doc.metadata.get('creationDate', 'N/A')}")
# pyrefly: ignore [missing-attribute]
print(f"修改日期: {doc.metadata.get('modDate', 'N/A')}")
print(f"文件大小: {input_path.stat().st_size / 1024:.1f} KB")
doc.close()
@px.tool("pdftool", subcommand="ocr", help="PDF OCR 识别")
def pdf_ocr(input_path: Path, output_path: Path = Path("ocr.pdf"), lang: str = "chi_sim+eng") -> None:
"""PDF OCR 识别.
Parameters
----------
input_path : Path
输入 PDF 文件
output_path : Path
输出文件 (默认: ocr.pdf)
lang : str
识别语言 (默认: chi_sim+eng)
"""
try:
import pytesseract
from PIL import Image
except ImportError:
print("未安装 OCR 相关库, 请安装: pip install pytesseract pillow")
return
if not HAS_PYMUPDF:
print("未安装 PyMuPDF 库, 请安装: pip install PyMuPDF")
return
doc = fitz.open(str(input_path))
new_doc = fitz.open()
for page in doc:
pix = page.get_pixmap()
img = Image.frombytes("RGB", (pix.width, pix.height), pix.samples)
ocr_text = pytesseract.image_to_string(img, lang=lang)
new_page = new_doc.new_page(width=page.rect.width, height=page.rect.height)
new_page.insert_image(new_page.rect, pixmap=pix)
text_rect = fitz.Rect(0, 0, page.rect.width, page.rect.height)
# pyrefly: ignore [bad-argument-type]
new_page.insert_textbox(text_rect, ocr_text, fontname="china-ss", fontsize=11)
output_path.parent.mkdir(parents=True, exist_ok=True)
new_doc.save(str(output_path))
new_doc.close()
doc.close()
print(f"OCR 识别完成: {output_path}")
def pdf_reorder(input_path: Path, output_path: Path, order: list[int]) -> None:
"""重排 PDF 页面顺序.
Parameters
----------
input_path : Path
输入 PDF 文件
output_path : Path
输出文件
order : list[int]
页面顺序列表 (0-based)
"""
if not HAS_PYPDF:
print("未安装 pypdf 库, 请安装: pip install pypdf")
return
reader = pypdf.PdfReader(str(input_path))
writer = pypdf.PdfWriter()
for page_num in order:
if 0 <= page_num < len(reader.pages):
writer.add_page(reader.pages[page_num])
output_path.parent.mkdir(parents=True, exist_ok=True)
with open(output_path, "wb") as f:
writer.write(f)
print(f"重排完成: {output_path}")
@px.tool("pdftool", subcommand="img", help="PDF 转图片")
def pdf_to_images(input_path: Path, output_dir: Path = Path("images"), dpi: int = 300) -> None:
"""PDF 转图片.
Parameters
----------
input_path : Path
输入 PDF 文件
output_dir : Path
输出目录 (默认: images)
dpi : int
DPI (默认: 300)
"""
if not HAS_PYMUPDF:
print("未安装 PyMuPDF 库, 请安装: pip install PyMuPDF")
return
doc = fitz.open(str(input_path))
output_dir.mkdir(parents=True, exist_ok=True)
# pyrefly: ignore [bad-argument-type]
for page_num, page in enumerate(doc):
pix = page.get_pixmap(dpi=dpi)
image_path = output_dir / f"{input_path.stem}_page_{page_num + 1}.png"
pix.save(str(image_path))
doc.close()
print(f"转换完成: {output_dir}")
@px.tool("pdftool", subcommand="repair", help="修复 PDF")
def pdf_repair(input_path: Path, output_path: Path = Path("repaired.pdf")) -> None:
"""修复 PDF 文件.
Parameters
----------
input_path : Path
输入 PDF 文件
output_path : Path
输出文件 (默认: repaired.pdf)
"""
if not HAS_PYMUPDF:
print("未安装 PyMuPDF 库, 请安装: pip install PyMuPDF")
return
doc = fitz.open(str(input_path))
output_path.parent.mkdir(parents=True, exist_ok=True)
doc.save(str(output_path), garbage=4, deflate=True, clean=True)
doc.close()
print(f"修复完成: {output_path}")
+177
View File
@@ -0,0 +1,177 @@
"""piptool - pip 包管理工具.
提供安装/卸载/重装/下载/升级 pip/冻结依赖 子命令.
"""
from __future__ import annotations
import fnmatch
import subprocess
from pathlib import Path
import pyflowx as px
__all__ = [
"pip_download",
"pip_freeze",
"pip_install",
"pip_reinstall",
"pip_uninstall",
"pip_upgrade",
]
# ============================================================================
# 配置
# ============================================================================
PACKAGE_DIR = "packages"
REQUIREMENTS_FILE = "requirements.txt"
_PROTECTED_PACKAGES: frozenset[str] = frozenset(
{
"pyflowx",
"bitool",
}
)
# ============================================================================
# 私有辅助函数
# ============================================================================
def _get_installed_packages() -> list[str]:
"""获取当前环境中所有已安装的包名."""
try:
result = subprocess.run(
["pip", "list", "--format=freeze"],
capture_output=True,
text=True,
check=True,
)
packages: list[str] = []
for line in result.stdout.strip().split("\n"):
if line and "==" in line:
pkg_name = line.split("==")[0].strip()
packages.append(pkg_name)
except (subprocess.SubprocessError, OSError):
return []
return packages
def _expand_wildcard_packages(pattern: str) -> list[str]:
"""展开通配符模式为实际的包名列表."""
if not any(char in pattern for char in ["*", "?", "[", "]"]):
return [pattern]
installed_packages = _get_installed_packages()
matched = [pkg for pkg in installed_packages if fnmatch.fnmatchcase(pkg.lower(), pattern.lower())]
return matched
def _filter_protected_packages(packages: list[str]) -> list[str]:
"""过滤掉受保护的包名."""
safe = [p for p in packages if p.lower() not in {p.lower() for p in _PROTECTED_PACKAGES}]
filtered = [p for p in packages if p.lower() in {p.lower() for p in _PROTECTED_PACKAGES}]
if filtered:
print(f"跳过受保护的包: {', '.join(filtered)}")
return safe
# ============================================================================
# 公共函数
# ============================================================================
@px.tool("piptool", subcommand="i", help="安装包")
def pip_install(packages: list[str]) -> None:
"""安装包.
Parameters
----------
packages : list[str]
包名列表
"""
subprocess.run(["pip", "install", *packages], check=True)
print(f"安装完成: {', '.join(packages)}")
@px.tool("piptool", subcommand="u", help="卸载包")
def pip_uninstall(packages: list[str]) -> None:
"""卸载包.
Parameters
----------
packages : list[str]
包名列表
"""
packages_to_uninstall: list[str] = []
for pattern in packages:
packages_to_uninstall.extend(_expand_wildcard_packages(pattern))
packages_to_uninstall = _filter_protected_packages(packages_to_uninstall)
if not packages_to_uninstall:
return
subprocess.run(["pip", "uninstall", "-y", *packages_to_uninstall], check=True)
@px.tool("piptool", subcommand="r", help="重装包")
def pip_reinstall(packages: list[str], offline: bool = False) -> None:
"""重新安装包.
Parameters
----------
packages : list[str]
包名列表
offline : bool
离线模式
"""
safe_ps = _filter_protected_packages(packages)
if not safe_ps:
print("所有指定的包均为受保护包, 跳过重装")
return
subprocess.run(["pip", "uninstall", "-y", *safe_ps], check=True)
options = ["--no-index", "--find-links", "."] if offline else []
subprocess.run(["pip", "install", *options, *safe_ps], check=True)
@px.tool("piptool", subcommand="d", help="下载包")
def pip_download(packages: list[str], offline: bool = False) -> None:
"""下载包.
Parameters
----------
packages : list[str]
包名列表
offline : bool
离线模式
"""
options = ["--no-index", "--find-links", "."] if offline else []
subprocess.run(
["pip", "download", *packages, *options, "-d", PACKAGE_DIR],
check=True,
)
@px.tool("piptool", subcommand="up", help="升级 pip")
def pip_upgrade() -> None:
"""升级 pip 到最新版本."""
subprocess.run(["python", "-m", "pip", "install", "--upgrade", "pip"], check=True)
print("pip 升级完成")
@px.tool("piptool", subcommand="f", help="导出依赖")
def pip_freeze() -> None:
"""冻结依赖到 requirements.txt."""
result = subprocess.run(
["pip", "freeze", "--exclude-editable"],
capture_output=True,
text=True,
check=True,
)
Path(REQUIREMENTS_FILE).write_text(result.stdout)
print(f"依赖已导出到 {REQUIREMENTS_FILE}")
+231
View File
@@ -0,0 +1,231 @@
"""pymake - 项目构建工具.
提供构建/测试/清理/文档/格式化/发布等子命令.
"""
from __future__ import annotations
__all__ = [
"b",
"ba",
"bc",
"bump",
"bumpmi",
"bumpversion",
"c",
"cov",
"doc",
"git_add_all",
"git_push",
"git_push_tags",
"lint",
"p",
"pb",
"publish_python",
"pyrefly_check",
"sync",
"t",
"tc",
"test_coverage",
"tf",
"tox",
"twine_publish",
]
from pathlib import Path
import pyflowx as px
# ============================================================================
# 单任务别名 (cmd 任务)
# ============================================================================
@px.tool("pymake", subcommand="b", help="构建 Python 主包 (uv build)", cmd=["uv", "build"])
def b(cwd: Path = Path()) -> None:
"""构建 Python 主包."""
@px.tool("pymake", subcommand="bc", help="构建 Rust 核心模块 (maturin build)", cmd=["maturin", "build", "-r"])
def bc(cwd: Path = Path()) -> None:
"""构建 Rust 核心模块."""
@px.tool("pymake", subcommand="sync", help="同步依赖 (uv sync)", cmd=["uv", "sync"])
def sync(cwd: Path = Path()) -> None:
"""同步依赖."""
@px.tool("pymake", subcommand="c", help="清理构建产物 (调用 gitt c)", cmd=["pf", "gitt", "c"])
def c(cwd: Path = Path()) -> None:
"""清理构建产物."""
@px.tool(
"pymake",
subcommand="t",
help="运行测试",
cmd=["pytest", "-m", "not slow", "-n", "8", "--dist", "loadfile", "--color=yes", "--durations=10"],
)
def t(cwd: Path = Path()) -> None:
"""运行测试 (含并行)."""
@px.tool(
"pymake",
subcommand="tf",
help="快速测试 (无 slow)",
cmd=["pytest", "-m", "not slow", "--dist", "loadfile", "--color=yes", "--durations=10"],
)
def tf(cwd: Path = Path()) -> None:
"""快速测试 (无 slow, 无并行)."""
@px.tool("pymake", subcommand="bumpmi", help="升级次版本号 (bumpversion minor)", cmd=["pf", "bumpversion", "minor"])
def bumpmi(cwd: Path = Path()) -> None:
"""升级次版本号."""
@px.tool(
"pymake",
subcommand="doc",
help="构建 Sphinx 文档",
cmd=["sphinx-build", "-b", "html", "docs", "docs/_build"],
)
def doc(cwd: Path = Path()) -> None:
"""构建 Sphinx 文档."""
@px.tool("pymake", subcommand="lint", help="代码格式化与检查 (ruff)", cmd=["ruff", "check", "--fix", "--unsafe-fixes"])
def lint(cwd: Path = Path()) -> None:
"""代码格式化与检查."""
@px.tool("pymake", subcommand="tox", help="多版本测试 (tox)", cmd=["tox", "-p", "auto"])
def tox(cwd: Path = Path()) -> None:
"""多版本测试."""
# ============================================================================
# 内部 job (hidden, 不暴露为 subcommand)
# ============================================================================
@px.tool(
"pymake",
subcommand="bumpversion",
help="升级版本号 (patch)",
cmd=["pf", "bumpversion", "patch"],
needs=["git_add_all"],
hidden=True,
)
def bumpversion(cwd: Path = Path()) -> None:
"""升级版本号 (patch, 内部 job)."""
@px.tool(
"pymake",
subcommand="test_coverage",
help="测试并生成覆盖率",
cmd=["pytest", "--cov", "-n", "8", "--dist", "loadfile", "--tb=short", "-v", "--color=yes", "--durations=10"],
needs=["c"],
hidden=True,
)
def test_coverage(cwd: Path = Path()) -> None:
"""测试并生成覆盖率 (内部 job)."""
@px.tool(
"pymake",
subcommand="pyrefly_check",
help="pyrefly 类型检查",
cmd=["pyrefly", "check", "."],
hidden=True,
)
def pyrefly_check(cwd: Path = Path()) -> None:
"""pyrefly 类型检查 (内部 job)."""
@px.tool(
"pymake",
subcommand="git_add_all",
help="git add -A",
cmd=["git", "add", "-A"],
needs=["tc"],
hidden=True,
)
def git_add_all(cwd: Path = Path()) -> None:
"""git add -A (内部 job)."""
@px.tool("pymake", subcommand="git_push", help="git push", cmd=["git", "push"], hidden=True)
def git_push(cwd: Path = Path()) -> None:
"""git push (内部 job)."""
@px.tool(
"pymake",
subcommand="git_push_tags",
help="git push --tags",
cmd=["git", "push", "--tags"],
hidden=True,
)
def git_push_tags(cwd: Path = Path()) -> None:
"""git push --tags (内部 job)."""
@px.tool(
"pymake",
subcommand="twine_publish",
help="twine upload",
cmd=["twine", "upload", "--disable-progress-bar"],
hidden=True,
)
def twine_publish(cwd: Path = Path()) -> None:
"""twine upload (内部 job)."""
@px.tool("pymake", subcommand="publish_python", help="hatch publish", cmd=["hatch", "publish"], hidden=True)
def publish_python(cwd: Path = Path()) -> None:
"""hatch publish (内部 job)."""
# ============================================================================
# 聚合 job (有 needs 无 cmd)
# ============================================================================
@px.tool("pymake", subcommand="ba", help="构建所有包 (Python + Rust)", needs=["b", "bc"], strategy="thread")
def ba(cwd: Path = Path()) -> None:
"""构建所有包 (Python + Rust)."""
@px.tool("pymake", subcommand="bump", help="升级版本号 (清理 + 检查 + add + bumpversion)", needs=["bumpversion"])
def bump(cwd: Path = Path()) -> None:
"""升级版本号 (聚合)."""
@px.tool("pymake", subcommand="cov", help="测试并生成覆盖率", needs=["test_coverage"])
def cov(cwd: Path = Path()) -> None:
"""测试并生成覆盖率 (聚合)."""
@px.tool(
"pymake",
subcommand="tc",
help="类型检查 (pyrefly + ruff)",
needs=["c", "pyrefly_check", "lint"],
strategy="thread",
)
def tc(cwd: Path = Path()) -> None:
"""类型检查 (聚合)."""
@px.tool("pymake", subcommand="p", help="推送代码 (清理 + push + push tags)", needs=["c", "git_push", "git_push_tags"], strategy="thread")
def p(cwd: Path = Path()) -> None:
"""推送代码 (聚合)."""
@px.tool("pymake", subcommand="pb", help="发布到 PyPI (twine + hatch)", needs=["twine_publish", "publish_python"], strategy="thread")
def pb(cwd: Path = Path()) -> None:
"""发布到 PyPI (聚合)."""
+52
View File
@@ -0,0 +1,52 @@
"""reseticoncache - 重置 Windows 图标缓存工具.
执行流程: 杀掉 explorer → 删除 IconCache.db → 删除 iconcache* → 重启 explorer.
仅在 Windows 上执行, 非 Windows 平台打印提示并跳过.
"""
from __future__ import annotations
import os
import subprocess
from pathlib import Path
import pyflowx as px
from pyflowx.conditions import Constants
@px.tool("reseticoncache", help="重置 Windows 图标缓存")
def reset_icon_cache_run() -> None:
"""重置 Windows 图标缓存.
执行流程: 杀掉 explorer → 删除 IconCache.db → 删除 iconcache* → 重启 explorer.
仅在 Windows 上执行, 非 Windows 平台打印提示并跳过.
"""
if not Constants.IS_WINDOWS:
print("reset_icon_cache: 仅在 Windows 上支持")
return
local_app_data = os.environ.get("LOCALAPPDATA", "")
if not local_app_data:
print("reset_icon_cache: LOCALAPPDATA 环境变量未设置")
return
icon_cache_db = Path(local_app_data) / "IconCache.db"
explorer_cache_dir = Path(local_app_data) / "Microsoft" / "Windows" / "Explorer"
print("正在终止 explorer 进程...")
subprocess.run(["taskkill", "/f", "/im", "explorer.exe"], check=False)
if icon_cache_db.exists():
print(f"删除图标缓存: {icon_cache_db}")
subprocess.run(["cmd", "/c", "del", "/a", "/q", str(icon_cache_db)], check=False)
if explorer_cache_dir.exists():
print(f"清理 Explorer 缓存: {explorer_cache_dir}")
subprocess.run(
["cmd", "/c", "del", "/a", "/q", str(explorer_cache_dir / "iconcache*")],
check=False,
)
print("重启 explorer...")
subprocess.run(["cmd", "/c", "start", "explorer.exe"], check=False)
print("图标缓存已重置")
@@ -1,11 +1,10 @@
"""截图工具.
"""screenshot - 截图工具.
跨平台截图工具, 支持全屏截图和区域截图.
跨平台截图: Windows PowerShell, macOS screencapture, Linux gnome-screenshot/scrot.
"""
from __future__ import annotations
import argparse
import subprocess
from datetime import datetime
from pathlib import Path
@@ -13,10 +12,6 @@ from pathlib import Path
import pyflowx as px
from pyflowx.conditions import Constants
# ============================================================================
# 辅助函数
# ============================================================================
def get_screenshot_path(filename: str | None = None) -> Path:
"""获取截图保存路径.
@@ -40,6 +35,7 @@ def get_screenshot_path(filename: str | None = None) -> Path:
return screenshots_dir / filename
@px.tool("screenshot", subcommand="full", help="全屏截图")
def take_screenshot_full(filename: str | None = None) -> None:
"""全屏截图.
@@ -51,7 +47,6 @@ def take_screenshot_full(filename: str | None = None) -> None:
output_path = get_screenshot_path(filename)
if Constants.IS_WINDOWS:
# Windows: 使用 PowerShell 截图
ps_script = f"""
Add-Type -AssemblyName System.Windows.Forms
Add-Type -AssemblyName System.Drawing
@@ -66,10 +61,8 @@ $bitmap.Dispose()
"""
subprocess.run(["powershell", "-Command", ps_script], check=True)
elif Constants.IS_MACOS:
# macOS: 使用 screencapture
subprocess.run(["screencapture", "-x", str(output_path)], check=True)
else:
# Linux: 使用 gnome-screenshot 或 scrot
try:
subprocess.run(["gnome-screenshot", "-f", str(output_path)], check=True)
except FileNotFoundError:
@@ -78,6 +71,7 @@ $bitmap.Dispose()
print(f"截图已保存: {output_path}")
@px.tool("screenshot", subcommand="area", help="区域截图")
def take_screenshot_area(filename: str | None = None) -> None:
"""区域截图.
@@ -89,7 +83,6 @@ def take_screenshot_area(filename: str | None = None) -> None:
output_path = get_screenshot_path(filename)
if Constants.IS_WINDOWS:
# Windows: 使用 PowerShell 截图 (需要用户选择区域)
ps_script = f"""
Add-Type -AssemblyName System.Windows.Forms
Add-Type -AssemblyName System.Drawing
@@ -113,51 +106,11 @@ $bitmap.Dispose()
"""
subprocess.run(["powershell", "-Command", ps_script], check=True)
elif Constants.IS_MACOS:
# macOS: 使用 screencapture 交互模式
subprocess.run(["screencapture", "-i", str(output_path)], check=True)
else:
# Linux: 使用 gnome-screenshot 交互模式
try:
subprocess.run(["gnome-screenshot", "-a", "-f", str(output_path)], check=True)
except FileNotFoundError:
subprocess.run(["scrot", "-s", str(output_path)], check=True)
print(f"截图已保存: {output_path}")
# ============================================================================
# CLI Runner
# ============================================================================
def main() -> None:
"""截图工具主函数."""
parser = argparse.ArgumentParser(
description="Screenshot - 截图工具",
usage="screenshot <command> [options]",
)
subparsers = parser.add_subparsers(dest="command", help="可用命令")
# 全屏截图命令
full_parser = subparsers.add_parser("full", help="全屏截图")
full_parser.add_argument("--filename", type=str, help="文件名")
# 区域截图命令
area_parser = subparsers.add_parser("area", help="区域截图")
area_parser.add_argument("--filename", type=str, help="文件名")
args = parser.parse_args()
if args.command == "full":
graph = px.Graph.from_specs(
[px.TaskSpec("screenshot_full", fn=take_screenshot_full, kwargs={"filename": args.filename})]
)
elif args.command == "area":
graph = px.Graph.from_specs(
[px.TaskSpec("screenshot_area", fn=take_screenshot_area, kwargs={"filename": args.filename})]
)
else:
parser.print_help()
return
px.run(graph, strategy="thread")
+87
View File
@@ -0,0 +1,87 @@
"""sglang - SGLang 本地模型服务工具.
提供安装与启动 SGLang 服务的子命令.
"""
from __future__ import annotations
import shutil
import subprocess
from pathlib import Path
import pyflowx as px
from pyflowx.conditions import Constants
__all__ = [
"install_sglang",
"run_sglang",
]
@px.tool("sglang", subcommand="install", help="安装 sglang")
def install_sglang() -> None:
"""安装 sglang (若未安装).
通过 ``shutil.which`` 检测 sglang 是否已安装, 未安装时执行 ``uv install sglang[all]``.
"""
if shutil.which("sglang") is not None:
print("sglang 已安装, 跳过安装步骤")
return
print("正在安装 sglang[all]...")
subprocess.run(["uv", "install", "sglang[all]"], check=False)
@px.tool("sglang", subcommand="run", help="启动 SGLang 服务", needs=["install"])
def run_sglang(
model: str = "~/.models/Qwen2.5-Coder-32B-Instruct-AWQ",
port: int = 8000,
ctx_len: int = 32768,
mem_fraction: float = 0.75,
host: str = "0.0.0.0",
log_level: str = "info",
) -> None:
"""启动 SGLang 本地模型服务.
Parameters
----------
model : str
模型路径 (默认: ``~/.models/Qwen2.5-Coder-32B-Instruct-AWQ``)
port : int
服务端口 (默认: 8000)
ctx_len : int
最大上下文长度 (默认: 32768)
mem_fraction : float
显存占比 0-1 (默认: 0.75)
host : str
主机地址 (默认: 0.0.0.0)
log_level : str
日志级别 (默认: info)
"""
model_dir = Path(model).expanduser()
if not model_dir.exists():
print(f"模型目录不存在: {model_dir}")
return
python_bin = "python" if Constants.IS_WINDOWS else "python3"
cmd = [
python_bin,
"-m",
"sglang.launch_server",
"--model-path",
str(model_dir),
"--host",
host,
"--port",
str(port),
"--mem-fraction-static",
str(mem_fraction),
"--context-length",
str(ctx_len),
"--tool-call-parser",
"qwen",
"--log-level",
log_level,
]
print(f"启动 SGLang: {model_dir} (port={port}, ctx={ctx_len}, mem={mem_fraction})")
subprocess.run(cmd, check=False)
@@ -1,23 +1,20 @@
"""SSH 密钥部署工具.
"""sshcopyid - SSH 密钥部署工具.
类似 ssh-copy-id, 自动 SSH 公钥部署到远程服务器,
支持密码认证和密钥认证两种方式.
本地 SSH 公钥部署到远程服务器.
"""
from __future__ import annotations
import argparse
import subprocess
import sys
from pathlib import Path
import pyflowx as px
# ============================================================================
# 辅助函数
# ============================================================================
__all__ = ["ssh_copy_id"]
@px.tool("sshcopyid", help="部署 SSH 公钥到远程服务器")
def ssh_copy_id(
hostname: str,
username: str,
@@ -43,7 +40,6 @@ def ssh_copy_id(
timeout : int
SSH 操作超时秒数, 默认 30
"""
# 读取公钥
pub_key_path = Path(keypath).expanduser()
if not pub_key_path.exists():
print(f"公钥文件不存在: {pub_key_path}")
@@ -51,12 +47,10 @@ def ssh_copy_id(
pub_key = pub_key_path.read_text().strip()
# 构建部署脚本
script = f"""mkdir -p ~/.ssh && chmod 700 ~/.ssh
cd ~/.ssh && touch authorized_keys && chmod 600 authorized_keys
grep -qF '{pub_key.split()[1]}' authorized_keys 2>/dev/null || echo '{pub_key}' >> authorized_keys"""
# 使用 sshpass 执行
try:
subprocess.run(
[
@@ -80,7 +74,7 @@ grep -qF '{pub_key.split()[1]}' authorized_keys 2>/dev/null || echo '{pub_key}'
)
print(f"SSH 密钥已部署到 {username}@{hostname}:{port}")
except FileNotFoundError:
print(f"未找到 sshpass 工具请手动执行: ssh-copy-id -p {port} {username}@{hostname}")
print(f"未找到 sshpass 工具, 请手动执行: ssh-copy-id -p {port} {username}@{hostname}")
sys.exit(1)
except subprocess.TimeoutExpired:
print("SSH 连接超时")
@@ -88,35 +82,3 @@ grep -qF '{pub_key.split()[1]}' authorized_keys 2>/dev/null || echo '{pub_key}'
except subprocess.CalledProcessError as e:
print(f"SSH 执行失败: {e}")
sys.exit(1)
# ============================================================================
# CLI Runner
# ============================================================================
def main() -> None:
"""SSH 密钥部署工具主函数."""
parser = argparse.ArgumentParser(
description="SSHCopyID - SSH 密钥部署工具",
usage="sshcopyid <hostname> <username> <password> [--port PORT] [--keypath KEYPATH]",
)
parser.add_argument("hostname", type=str, help="远程服务器主机名或 IP 地址")
parser.add_argument("username", type=str, help="远程服务器用户名")
parser.add_argument("password", type=str, help="远程服务器密码")
parser.add_argument("--port", type=int, default=22, help="SSH 端口 (默认: 22)")
parser.add_argument("--keypath", type=str, default="~/.ssh/id_rsa.pub", help="公钥文件路径")
parser.add_argument("--timeout", type=int, default=30, help="SSH 操作超时秒数 (默认: 30)")
args = parser.parse_args()
graph = px.Graph.from_specs(
[
px.TaskSpec(
"ssh_deploy",
fn=ssh_copy_id,
args=(args.hostname, args.username, args.password),
kwargs={"port": args.port, "keypath": args.keypath, "timeout": args.timeout},
)
]
)
px.run(graph, strategy="thread")
+33
View File
@@ -0,0 +1,33 @@
"""taskkill - 进程终止工具.
跨平台按名称终止进程: Windows 用 ``taskkill``, Linux/macOS 用 ``pkill``.
"""
from __future__ import annotations
import subprocess
import pyflowx as px
from pyflowx.conditions import Constants
@px.tool("taskkill", help="按名称终止进程 (跨平台)")
def taskkill_run(process_names: list[str]) -> None:
"""按名称终止进程 (跨平台).
Windows 使用 ``taskkill /f /im <name>*``,
Linux/macOS 使用 ``pkill -f <name>*``.
Parameters
----------
process_names : list[str]
进程名称列表 (如: ``["chrome.exe", "python"]``)
"""
if Constants.IS_WINDOWS:
cmd_prefix: list[str] = ["taskkill", "/f", "/im"]
else:
cmd_prefix = ["pkill", "-f"]
for name in process_names:
print(f"终止进程: {name}")
subprocess.run([*cmd_prefix, f"{name}*"], check=False)
+35
View File
@@ -0,0 +1,35 @@
"""which - 命令查找工具.
跨平台查找可执行命令路径: Windows 用 ``where``, Linux/macOS 用 ``which``.
"""
from __future__ import annotations
import subprocess
import pyflowx as px
from pyflowx.conditions import Constants
@px.tool("which", help="查找可执行命令路径 (跨平台)")
def which_run(commands: list[str]) -> None:
"""查找可执行命令路径 (跨平台).
Windows 使用 ``where``, Linux/macOS 使用 ``which``.
对每个命令打印 ``<cmd> -> <path>`` 或 ``<cmd> -> 未找到``.
Parameters
----------
commands : list[str]
要查找的命令名称列表
"""
which_cmd = "where" if Constants.IS_WINDOWS else "which"
for cmd in commands:
result = subprocess.run([which_cmd, cmd], capture_output=True, text=True, check=False)
if result.returncode == 0:
# Windows 的 where 可能返回多行, 取第一个
path = result.stdout.strip().split("\n")[0].strip()
print(f"{cmd} -> {path}")
else:
print(f"{cmd} -> 未找到")
+2 -39
View File
@@ -14,7 +14,7 @@ from __future__ import annotations
import argparse
import enum
import sys
from dataclasses import dataclass, field, replace
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any, Sequence, get_args
@@ -35,39 +35,6 @@ class CliExitCode(enum.IntEnum):
INTERRUPTED = 130 # 与 POSIX 信号中断一致
def _apply_verbose_to_graph(graph: Graph, verbose: bool) -> Graph:
"""创建新图, 其中所有 TaskSpec 的 verbose 字段被设置为指定值.
使用 ``dataclasses.replace`` 在不可变的 TaskSpec 上创建带 verbose 标记的副本.
依赖关系、标签等元数据全部保留.
Note
-----
自 ``_wrap_cmd`` 不再闭包捕获 ``verbose`` 后,此函数不再是必需的——
直接翻转 ``spec.verbose`` 即可生效。保留是为了向后兼容现有调用与测试。
TaskSpec 仍是 frozen dataclass,故仍用 ``replace`` 创建副本。
Parameters
----------
graph : Graph
原始图.
verbose : bool
要设置的 verbose 值.
Returns
-------
Graph
所有 spec 的 verbose 字段已更新的新图.
"""
new_specs: list[TaskSpec[Any]] = []
for spec in graph.all_specs().values():
if spec.verbose == verbose:
new_specs.append(spec)
else:
new_specs.append(replace(spec, verbose=verbose))
return Graph.from_specs(new_specs)
@dataclass
class CliRunner:
"""命令行运行器: 根据用户输入执行对应的任务流图.
@@ -296,12 +263,8 @@ class CliRunner:
# 确定是否 verbose: --quiet 覆盖默认值
verbose = self.verbose and not parsed.quiet
# 对图应用 verbose 设置 (重建带 verbose 标记的 spec)
# 执行对应的图 (verbose 标记由 run() 统一应用到各 spec)
graph = self.graphs[parsed.command]
if verbose:
graph = _apply_verbose_to_graph(graph, verbose=True)
# 执行对应的图
try:
report = run(
graph,
+599
View File
@@ -0,0 +1,599 @@
"""@px.tool 装饰器: 函数签名驱动 CLI 生成 + DAG 编排.
替代 YAML 配置模式, 用 .py 装饰器统一描述工具.
函数签名 → argparse 自动生成, 函数体即任务逻辑, needs/strategy 表达 DAG.
示例
----
::
@px.tool("pdftool", subcommand="m", help="合并 PDF")
def pdf_merge(
inputs: list[Path], # positional, nargs="+"
output: Path = Path("merged.pdf"), # --output 选项
) -> None:
... # 函数体即逻辑
# CLI: pf pdftool m a.pdf b.pdf --output merged.pdf
cmd 任务 (执行外部命令, 函数体不执行)::
@px.tool("pymake", subcommand="b", help="构建", cmd=["uv", "build"])
def b(cwd: Path = Path(".")) -> None:
pass # 签名仅驱动 CLI, cwd 注入到 cmd 工作目录
聚合任务 (有 needs 无 cmd 无函数逻辑)::
@px.tool("pymake", subcommand="tc", help="类型检查",
needs=["c", "pyrefly_check", "lint"], strategy="thread")
def tc(cwd: Path = Path(".")) -> None:
pass # 仅作依赖聚合点
"""
from __future__ import annotations
__all__ = [
"ToolExitCode",
"ToolSpec",
"clear_tool_registry",
"get_tool",
"list_subcommands",
"list_tools",
"run_tool",
"tool",
]
import argparse
import ast
import enum
import inspect
import sys
import textwrap
from dataclasses import dataclass
from pathlib import Path
from typing import Any, Callable, Literal, Mapping, Sequence, Union, cast, get_args, get_origin, get_type_hints
from pyflowx.errors import PyFlowXError
from pyflowx.executors import run
from pyflowx.graph import Graph
from pyflowx.task import RetryPolicy, TaskSpec
class ToolExitCode(enum.IntEnum):
"""工具执行退出码."""
SUCCESS = 0
FAILURE = 1
INTERRUPTED = 130 # 与 POSIX 信号中断一致
def _noop() -> None:
"""聚合任务的占位函数."""
return None
# ---------------------------------------------------------------------- #
# ToolSpec: 工具描述符
# ---------------------------------------------------------------------- #
@dataclass(frozen=True)
class ToolSpec:
"""工具描述符: 由 ``@px.tool`` 装饰器注册.
封装函数 + CLI 元数据 + DAG 编排参数, 运行时映射为 :class:`TaskSpec`.
"""
name: str # 工具名 (如 "pdftool")
subcommand: str | None # 子命令名 (None=单命令工具)
func: Callable[..., Any] # 函数 (签名驱动 CLI, 体即逻辑)
help: str = "" # 帮助文本
description: str = "" # 工具描述 (pf 列表用)
cmd: tuple[str, ...] | str | None = None # 命令 (有则执行 cmd, 无则执行函数体)
needs: tuple[str, ...] = () # 依赖任务 (引用同 tool 的 subcommand)
strategy: Literal["sequential", "thread", "async", "dependency"] | None = None # 执行策略
cwd: str | Path | None = None # 工作目录 (cmd 任务, 装饰器级默认)
allow_upstream_skip: bool = False
hidden: bool = False # 不暴露为 subcommand (内部 job)
env: Mapping[str, str] | None = None
retry: RetryPolicy | None = None
timeout: float | None = None
# 全局工具注册表: {tool_name: {subcommand: ToolSpec}}
_TOOL_REGISTRY: dict[str, dict[str | None, ToolSpec]] = {}
# ---------------------------------------------------------------------- #
# @tool 装饰器 + 注册表
# ---------------------------------------------------------------------- #
def tool(
name: str,
*,
subcommand: str | None = None,
help: str = "",
description: str = "",
cmd: Sequence[str] | str | None = None,
needs: Sequence[str] | None = None,
strategy: Literal["sequential", "thread", "async", "dependency"] | None = None,
cwd: str | Path | None = None,
allow_upstream_skip: bool = False,
hidden: bool = False,
env: Mapping[str, str] | None = None,
retry: RetryPolicy | None = None,
timeout: float | None = None,
) -> Callable[[Callable[..., Any]], Callable[..., Any]]:
"""装饰器: 将函数注册为 ``@px.tool`` 工具.
Parameters
----------
name : str
工具名 (如 "pdftool"); 多个 ``@px.tool`` 共用同名即多 subcommand 工具
subcommand : str | None
子命令名; ``None`` 表示单命令工具 (整个工具仅一个函数)
help : str
子命令帮助文本; 默认用函数 docstring
description : str
工具描述, 用于 pf 工具列表
cmd : Sequence[str] | str | None
命令列表或 shell 字符串; 有 ``cmd`` 执行命令, 函数体不执行 (签名仅驱动 CLI)
needs : Sequence[str] | None
依赖任务名 (引用同 tool 的其他 subcommand)
strategy : str | None
执行策略: ``sequential`` / ``thread`` / ``async`` / ``dependency``
cwd : str | Path | None
工作目录 (cmd 任务装饰器级默认); 若函数签名有 ``cwd`` 参数则被 CLI 值覆盖
allow_upstream_skip : bool
上游 SKIPPED 时本任务仍执行
hidden : bool
不暴露为 subcommand (内部 job, 仅被 needs 引用)
env / retry / timeout
透传 :class:`TaskSpec` 对应字段
"""
def decorator(func: Callable[..., Any]) -> Callable[..., Any]:
spec = ToolSpec(
name=name,
subcommand=subcommand,
func=func,
help=help or inspect.getdoc(func) or "",
description=description,
cmd=cast("tuple[str, ...] | str | None", tuple(cmd) if isinstance(cmd, (list, tuple)) else cmd),
needs=tuple(needs) if needs else (),
strategy=strategy,
cwd=cwd,
allow_upstream_skip=allow_upstream_skip,
hidden=hidden,
env=dict(env) if env else None,
retry=retry,
timeout=timeout,
)
_register_tool(spec)
return func
return decorator
def _register_tool(spec: ToolSpec) -> None:
"""注册工具到全局注册表, 校验重复."""
if spec.name not in _TOOL_REGISTRY:
_TOOL_REGISTRY[spec.name] = {}
if spec.subcommand in _TOOL_REGISTRY[spec.name]:
raise ValueError(f"工具 {spec.name!r} 的子命令 {spec.subcommand!r} 已注册")
_TOOL_REGISTRY[spec.name][spec.subcommand] = spec
def get_tool(name: str, subcommand: str | None = None) -> ToolSpec:
"""获取已注册工具.
Raises
------
KeyError
工具或子命令未注册
"""
if name not in _TOOL_REGISTRY:
raise KeyError(f"工具 {name!r} 未注册")
subs = _TOOL_REGISTRY[name]
if subcommand not in subs:
raise KeyError(f"工具 {name!r} 没有子命令 {subcommand!r}")
return subs[subcommand]
def list_tools() -> list[str]:
"""列出所有已注册工具名."""
return sorted(_TOOL_REGISTRY.keys())
def list_subcommands(name: str, include_hidden: bool = False) -> list[str]:
"""列出工具的子命令 (hidden 默认排除).
单命令工具 (subcommand=None) 返回空列表.
"""
if name not in _TOOL_REGISTRY:
return []
return sorted(
sc for sc, spec in _TOOL_REGISTRY[name].items() if sc is not None and (include_hidden or not spec.hidden)
)
def clear_tool_registry() -> None:
"""清空注册表 (测试用)."""
_TOOL_REGISTRY.clear()
# ---------------------------------------------------------------------- #
# CLI 生成 (函数签名 → argparse)
# ---------------------------------------------------------------------- #
def _snake_to_kebab(name: str) -> str:
"""snake_case → kebab-case."""
return name.replace("_", "-")
def _unwrap_optional(annotation: Any) -> tuple[Any, bool]:
"""返回 (实际类型, 是否 Optional).
``Optional[X]`` / ``X | None`` → ``(X, True)``; 其他 → ``(annotation, False)``.
"""
origin = get_origin(annotation)
is_union = origin is Union
if not is_union and sys.version_info >= (3, 10):
import types as _types
is_union = origin is _types.UnionType
if is_union:
args = [a for a in get_args(annotation) if a is not type(None)]
if len(args) == 1:
return args[0], True
return annotation, False
def _map_param_type(annotation: Any) -> dict[str, Any]:
"""将类型注解转为 argparse add_argument 的关键字参数.
Returns
-------
dict
含 ``type`` / ``nargs`` / ``choices`` 等; bool 不在此处理 (由调用方特殊处理)
"""
kwargs: dict[str, Any] = {}
inner, _is_opt = _unwrap_optional(annotation)
origin = get_origin(inner)
# list[X] / List[X] → nargs="+"
if origin is list:
args = get_args(inner)
if args:
kwargs["type"] = args[0]
kwargs["nargs"] = "+"
return kwargs
# Literal["a", "b"] → choices
if origin is Literal:
kwargs["choices"] = list(get_args(inner))
return kwargs
# 基本类型
if inner is int:
kwargs["type"] = int
elif inner is float:
kwargs["type"] = float
elif inner is Path:
kwargs["type"] = Path
elif inner is str:
kwargs["type"] = str
return kwargs
def _add_func_args_to_parser(parser: argparse.ArgumentParser, func: Callable[..., Any]) -> None:
"""从函数签名向 parser 添加位置参数和选项参数.
- 无默认值 → positional
- 有默认值 → ``--kebab-case`` 选项
- ``bool`` 默认 False → ``--flag`` store_true; 默认 True → ``--no-flag`` store_false
- ``list[X]`` → ``nargs="+"``
- ``Literal[...]`` → ``choices``
"""
sig = inspect.signature(func)
try:
hints: dict[str, Any] = get_type_hints(func)
except Exception:
hints = {}
for pname, param in sig.parameters.items():
if pname in ("self",):
continue
if param.kind in (inspect.Parameter.VAR_POSITIONAL, inspect.Parameter.VAR_KEYWORD):
continue
annotation = hints.get(pname, param.annotation if param.annotation is not inspect.Parameter.empty else str)
has_default = param.default is not inspect.Parameter.empty
default = param.default if has_default else inspect.Parameter.empty
# bool 特殊处理
if annotation is bool:
if not has_default or default is False:
parser.add_argument(
f"--{_snake_to_kebab(pname)}",
dest=pname,
action="store_true",
default=False,
help=pname,
)
else: # 默认 True → --no-flag store_false
parser.add_argument(
f"--no-{_snake_to_kebab(pname)}",
dest=pname,
action="store_false",
default=True,
help=f"禁用 {pname}",
)
continue
type_kwargs = _map_param_type(annotation)
if has_default:
flag = f"--{_snake_to_kebab(pname)}"
kwargs: dict[str, Any] = {"dest": pname, "default": default, "help": pname}
kwargs.update(type_kwargs)
parser.add_argument(flag, **kwargs)
else:
# 位置参数; 若类型是 list[X] 则 nargs="+" 已在 type_kwargs
kwargs = {"help": pname}
kwargs.update(type_kwargs)
parser.add_argument(pname, **kwargs)
def _add_global_options(parser: argparse.ArgumentParser) -> None:
"""向 parser (及所有 subparser) 添加全局选项."""
parser.add_argument("--dry-run", action="store_true", help="仅打印执行计划, 不执行")
parser.add_argument("--quiet", "-q", action="store_false", dest="verbose", help="减少输出")
parser.add_argument("--strategy", type=str, default=None, help="执行策略 (sequential/thread/dependency)")
parser.add_argument("--list", action="store_true", dest="list_jobs", help="列出所有任务后退出")
# 子命令也加全局选项
for action in parser._actions:
if action.dest == "command" and isinstance(action.choices, dict):
for sub in action.choices.values():
sub.add_argument("--dry-run", action="store_true")
sub.add_argument("--quiet", "-q", action="store_false", dest="verbose")
sub.add_argument("--strategy", type=str, default=None)
sub.add_argument("--list", action="store_true", dest="list_jobs")
def _build_parser(name: str, all_subs: dict[str | None, ToolSpec]) -> argparse.ArgumentParser:
"""构建 argparse 解析器.
单命令工具 (仅 subcommand=None) → 简单 parser;
多 subcommand 工具 → 带 subparsers.
"""
first_spec = next(iter(all_subs.values()))
description = first_spec.description or first_spec.help or name
parser = argparse.ArgumentParser(prog=f"pf {name}", description=description)
if None in all_subs and len(all_subs) == 1:
# 单命令工具
_add_func_args_to_parser(parser, all_subs[None].func)
else:
# 多 subcommand 工具
visible = [(sc, sp) for sc, sp in all_subs.items() if sc is not None and not sp.hidden]
if visible:
subparsers = parser.add_subparsers(dest="command", help="可用命令")
for sc, sp in sorted(visible, key=lambda x: x[0] or ""):
sub = subparsers.add_parser(sc, help=sp.help, description=sp.help)
_add_func_args_to_parser(sub, sp.func)
_add_global_options(parser)
return parser
# ---------------------------------------------------------------------- #
# 依赖收集 + TaskSpec 构建
# ---------------------------------------------------------------------- #
def _collect_with_deps(name: str, target: str | None) -> list[str | None]:
"""BFS 收集 target 及其传递依赖 (subcommand 名).
返回顺序: 依赖在前, target 在后 (符合 DAG 拓扑).
"""
if name not in _TOOL_REGISTRY:
return [target]
subs = _TOOL_REGISTRY[name]
result: list[str | None] = []
seen: set[str | None] = set()
queue: list[str | None] = [target]
while queue:
sc = queue.pop(0)
if sc in seen:
continue
seen.add(sc)
result.append(sc)
if sc in subs:
queue.extend(subs[sc].needs)
# 反转: 依赖在前, target 在后
result.reverse()
return result
def _has_function_logic(func: Callable[..., Any]) -> bool:
"""判断函数体是否有实际逻辑 (非 pass/.../docstring).
用 ast 分析, 避免 exec 函数.
"""
try:
src = inspect.getsource(func)
src = textwrap.dedent(src)
tree = ast.parse(src)
except (OSError, TypeError, SyntaxError): # pragma: no cover
return True
func_def = next((n for n in ast.walk(tree) if isinstance(n, (ast.FunctionDef, ast.AsyncFunctionDef))), None)
if func_def is None: # pragma: no cover
return True
for stmt in func_def.body:
if isinstance(stmt, ast.Expr) and isinstance(stmt.value, ast.Constant):
continue # docstring
if isinstance(stmt, ast.Pass):
continue
return True
return False
def _is_aggregate(spec: ToolSpec) -> bool:
"""判断是否为聚合任务 (有 needs 无 cmd 无函数逻辑)."""
if spec.cmd is not None or not spec.needs:
return False
return not _has_function_logic(spec.func)
def _build_task_spec(spec: ToolSpec, variables: Mapping[str, Any]) -> TaskSpec:
"""将 ToolSpec + 解析后的变量转为 TaskSpec.
- cmd 任务: 执行命令, cwd 从 variables["cwd"] 或装饰器 cwd 取
- 聚合任务 (有 needs 无 cmd 无函数逻辑): fn=noop
- fn 任务: 执行函数, kwargs 按签名从 variables 取
"""
task_name = spec.subcommand if spec.subcommand is not None else spec.name
# cmd 任务
if spec.cmd is not None:
cwd_value = variables.get("cwd", spec.cwd)
cwd = Path(cwd_value) if cwd_value is not None else None
cmd_value: Any = list(spec.cmd) if isinstance(spec.cmd, tuple) else spec.cmd
return TaskSpec(
name=task_name,
cmd=cmd_value,
depends_on=spec.needs,
cwd=cwd,
env=spec.env,
retry=spec.retry or RetryPolicy(),
timeout=spec.timeout,
allow_upstream_skip=spec.allow_upstream_skip,
strategy=spec.strategy,
)
# 聚合任务
if _is_aggregate(spec):
return TaskSpec(
name=task_name,
fn=_noop,
depends_on=spec.needs,
allow_upstream_skip=spec.allow_upstream_skip,
strategy=spec.strategy,
)
# fn 任务
sig = inspect.signature(spec.func)
kwargs: dict[str, Any] = {}
for pname in sig.parameters:
if pname in variables:
kwargs[pname] = variables[pname]
cwd_value = variables.get("cwd")
cwd = Path(cwd_value) if cwd_value is not None else None
return TaskSpec(
name=task_name,
fn=spec.func,
kwargs=kwargs,
depends_on=spec.needs,
cwd=cwd,
env=spec.env,
retry=spec.retry or RetryPolicy(),
timeout=spec.timeout,
allow_upstream_skip=spec.allow_upstream_skip,
strategy=spec.strategy,
)
def _extract_variables(args: argparse.Namespace, func: Callable[..., Any]) -> dict[str, Any]:
"""从 argparse namespace 提取函数签名对应的参数."""
sig = inspect.signature(func)
variables: dict[str, Any] = {}
for pname in sig.parameters:
if hasattr(args, pname):
variables[pname] = getattr(args, pname)
return variables
# ---------------------------------------------------------------------- #
# 执行入口
# ---------------------------------------------------------------------- #
def _handle_list(name: str, all_subs: dict[str | None, ToolSpec]) -> int:
"""处理 --list, 打印任务列表."""
print(f"工具 {name} 的任务列表:")
for sc, spec in sorted(all_subs.items(), key=lambda x: (x[0] is not None, x[0] or "")):
display = sc if sc is not None else name
deps = ", ".join(spec.needs) if spec.needs else "(无依赖)"
hidden = " [隐藏]" if spec.hidden else ""
cmd_tag = " [cmd]" if spec.cmd is not None else ""
print(f" - {display} (依赖: {deps}){cmd_tag}{hidden}")
return 0
def run_tool(name: str, argv: Sequence[str] | None = None) -> int:
"""执行工具, 返回退出码 (0 成功 / 1 失败 / 130 中断).
Parameters
----------
name : str
工具名 (已通过 ``@px.tool`` 注册)
argv : Sequence[str] | None
命令行参数 (不含工具名); None 用 ``sys.argv[1:]``
Returns
-------
int
退出码
"""
if name not in _TOOL_REGISTRY:
print(f"错误: 未注册工具 {name!r}", file=sys.stderr)
return ToolExitCode.FAILURE
all_subs = _TOOL_REGISTRY[name]
argv = list(argv) if argv is not None else sys.argv[1:]
parser = _build_parser(name, all_subs)
args = parser.parse_args(argv)
# --list 优先
if getattr(args, "list_jobs", False):
return _handle_list(name, all_subs)
# 确定要执行的 subcommand
visible_subs = [sc for sc, sp in all_subs.items() if sc is not None and not sp.hidden]
is_multi = len(visible_subs) > 1
if is_multi:
subcommand = getattr(args, "command", None)
if subcommand is None:
parser.print_help()
return ToolExitCode.SUCCESS
else:
# 单命令工具: subcommand 是 None 或唯一可见的具名 subcommand
subcommand = visible_subs[0] if visible_subs else None
if subcommand not in all_subs:
print(f"错误: 未知子命令 {subcommand!r}", file=sys.stderr)
return ToolExitCode.FAILURE
target_spec = all_subs[subcommand]
variables = _extract_variables(args, target_spec.func)
# 收集依赖 + 构建子图
all_names = _collect_with_deps(name, subcommand)
specs: list[TaskSpec] = []
for sc in all_names:
dep_spec = all_subs[sc]
specs.append(_build_task_spec(dep_spec, variables))
graph = Graph.from_specs(specs)
print(f"[{name}] 执行: {subcommand or '全部任务'}", flush=True)
try:
run(
graph,
strategy=target_spec.strategy or "dependency",
dry_run=getattr(args, "dry_run", False),
verbose=getattr(args, "verbose", True),
)
return ToolExitCode.SUCCESS
except PyFlowXError as e:
print(f"错误: {e}", file=sys.stderr)
return ToolExitCode.FAILURE
except KeyboardInterrupt:
print("\n已中断", file=sys.stderr)
return ToolExitCode.INTERRUPTED
+20 -103
View File
@@ -5,8 +5,7 @@ from __future__ import annotations
from pathlib import Path
from unittest.mock import MagicMock, patch
import pyflowx as px
from pyflowx.cli import autofmt
from pyflowx.ops import autofmt as dev
# ---------------------------------------------------------------------- #
@@ -19,14 +18,14 @@ class TestFormatWithRuff:
"""Should format with ruff."""
with patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(returncode=0)
autofmt.format_with_ruff(tmp_path, fix=True)
dev.format_with_ruff(tmp_path, fix=True)
assert mock_run.called
def test_format_with_ruff_no_fix(self, tmp_path: Path) -> None:
"""Should format with ruff without fix."""
with patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(returncode=0)
autofmt.format_with_ruff(tmp_path, fix=False)
dev.format_with_ruff(tmp_path, fix=False)
# Should not include --fix flag
call_args = mock_run.call_args[0][0]
assert "--fix" not in call_args
@@ -42,14 +41,14 @@ class TestLintWithRuff:
"""Should lint with ruff."""
with patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(returncode=0)
autofmt.lint_with_ruff(tmp_path, fix=True)
dev.lint_with_ruff(tmp_path, fix=True)
assert mock_run.called
def test_lint_with_ruff_no_fix(self, tmp_path: Path) -> None:
"""Should lint with ruff without fix."""
with patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(returncode=0)
autofmt.lint_with_ruff(tmp_path, fix=False)
dev.lint_with_ruff(tmp_path, fix=False)
# Should not include --fix flag
call_args = mock_run.call_args[0][0]
assert "--fix" not in call_args
@@ -66,7 +65,7 @@ class TestAddDocstring:
py_file = tmp_path / "test.py"
py_file.write_text("def test():\n pass\n")
result = autofmt.add_docstring(py_file, '"""Test module."""')
result = dev.add_docstring(py_file, '"""Test module."""')
assert result is True
def test_add_docstring_skips_files_with_docstring(self, tmp_path: Path) -> None:
@@ -74,7 +73,7 @@ class TestAddDocstring:
py_file = tmp_path / "test.py"
py_file.write_text('"""Existing docstring."""\ndef test():\n pass\n')
result = autofmt.add_docstring(py_file, '"""New docstring."""')
result = dev.add_docstring(py_file, '"""New docstring."""')
assert result is False
def test_add_docstring_empty_file(self, tmp_path: Path) -> None:
@@ -82,7 +81,7 @@ class TestAddDocstring:
py_file = tmp_path / "test.py"
py_file.write_text("")
result = autofmt.add_docstring(py_file, '"""Test module."""')
result = dev.add_docstring(py_file, '"""Test module."""')
# Should handle empty file
assert result is True
@@ -98,7 +97,7 @@ class TestGenerateModuleDocstring:
py_file = tmp_path / "test.py"
py_file.write_text("def test():\n pass\n")
result = autofmt.generate_module_docstring(py_file)
result = dev.generate_module_docstring(py_file)
# Should contain "Tests for" since stem contains "test"
assert "Tests for" in result
@@ -108,7 +107,7 @@ class TestGenerateModuleDocstring:
py_file.parent.mkdir(parents=True)
py_file.write_text("def test():\n pass\n")
result = autofmt.generate_module_docstring(py_file)
result = dev.generate_module_docstring(py_file)
assert "mypackage" in result
def test_generate_module_docstring_cli(self, tmp_path: Path) -> None:
@@ -116,7 +115,7 @@ class TestGenerateModuleDocstring:
py_file = tmp_path / "cli.py"
py_file.write_text("def test():\n pass\n")
result = autofmt.generate_module_docstring(py_file)
result = dev.generate_module_docstring(py_file)
assert "Command-line interface" in result
def test_generate_module_docstring_util(self, tmp_path: Path) -> None:
@@ -124,7 +123,7 @@ class TestGenerateModuleDocstring:
py_file = tmp_path / "utils.py"
py_file.write_text("def test():\n pass\n")
result = autofmt.generate_module_docstring(py_file)
result = dev.generate_module_docstring(py_file)
assert "Utility functions" in result
@@ -139,8 +138,8 @@ class TestAutoAddDocstrings:
py_file = tmp_path / "test.py"
py_file.write_text("def test():\n pass\n")
with patch.object(autofmt, "add_docstring", return_value=True):
count = autofmt.auto_add_docstrings(tmp_path)
with patch.object(dev, "add_docstring", return_value=True):
count = dev.auto_add_docstrings(tmp_path)
assert count >= 0
def test_auto_add_docstrings_skips_ignored(self, tmp_path: Path) -> None:
@@ -149,7 +148,7 @@ class TestAutoAddDocstrings:
py_file.parent.mkdir()
py_file.write_text("def test():\n pass\n")
count = autofmt.auto_add_docstrings(tmp_path)
count = dev.auto_add_docstrings(tmp_path)
# Should skip __pycache__
assert count == 0
@@ -158,7 +157,7 @@ class TestAutoAddDocstrings:
txt_file = tmp_path / "test.txt"
txt_file.write_text("test content")
count = autofmt.auto_add_docstrings(tmp_path)
count = dev.auto_add_docstrings(tmp_path)
assert count == 0
@@ -179,7 +178,7 @@ class TestSyncPyprojectConfig:
with patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(returncode=0)
autofmt.sync_pyproject_config(tmp_path)
dev.sync_pyproject_config(tmp_path)
assert mock_run.called
def test_sync_pyproject_config_updates_file(self, tmp_path: Path) -> None:
@@ -193,7 +192,7 @@ class TestSyncPyprojectConfig:
with patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(returncode=0)
autofmt.sync_pyproject_config(tmp_path)
dev.sync_pyproject_config(tmp_path)
assert mock_run.called
@@ -207,95 +206,13 @@ class TestFormatAll:
"""Should run ruff format."""
with patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(returncode=0)
autofmt.format_all(tmp_path)
dev.format_all(tmp_path)
assert mock_run.called
def test_format_all_runs_ruff_check(self, tmp_path: Path) -> None:
"""Should run ruff check."""
with patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(returncode=0)
autofmt.format_all(tmp_path)
dev.format_all(tmp_path)
# Should call ruff format and ruff check
assert mock_run.call_count == 2
# ---------------------------------------------------------------------- #
# main function
# ---------------------------------------------------------------------- #
class TestMain:
"""Test main function."""
def test_main_fmt_default_target(self) -> None:
"""main() should handle fmt command with default target."""
with patch("sys.argv", ["autofmt", "fmt"]), patch.object(px, "run") as mock_run:
autofmt.main()
assert mock_run.called
def test_main_fmt_custom_target(self) -> None:
"""main() should handle fmt command with custom target."""
with patch("sys.argv", ["autofmt", "fmt", "--target", "src"]), patch.object(px, "run") as mock_run:
autofmt.main()
assert mock_run.called
def test_main_lint_default_target(self) -> None:
"""main() should handle lint command with default target."""
with patch("sys.argv", ["autofmt", "lint"]), patch.object(px, "run") as mock_run:
autofmt.main()
assert mock_run.called
def test_main_lint_with_fix(self) -> None:
"""main() should handle lint command with fix."""
with patch("sys.argv", ["autofmt", "lint", "--fix"]), patch.object(px, "run") as mock_run:
autofmt.main()
assert mock_run.called
def test_main_lint_custom_target(self) -> None:
"""main() should handle lint command with custom target."""
with patch("sys.argv", ["autofmt", "lint", "--target", "src"]), patch.object(px, "run") as mock_run:
autofmt.main()
assert mock_run.called
def test_main_doc_default_root(self) -> None:
"""main() should handle doc command with default root."""
with patch("sys.argv", ["autofmt", "doc"]), patch.object(px, "run") as mock_run:
autofmt.main()
assert mock_run.called
def test_main_doc_custom_root(self) -> None:
"""main() should handle doc command with custom root."""
with patch("sys.argv", ["autofmt", "doc", "--root-dir", "src"]), patch.object(px, "run") as mock_run:
autofmt.main()
assert mock_run.called
def test_main_sync_default_root(self) -> None:
"""main() should handle sync command with default root."""
with patch("sys.argv", ["autofmt", "sync"]), patch.object(px, "run") as mock_run:
autofmt.main()
assert mock_run.called
def test_main_sync_custom_root(self) -> None:
"""main() should handle sync command with custom root."""
with patch("sys.argv", ["autofmt", "sync", "--root-dir", "."]), patch.object(px, "run") as mock_run:
autofmt.main()
assert mock_run.called
def test_main_with_no_args_shows_help(self) -> None:
"""main() with no args should show help."""
with patch("sys.argv", ["autofmt"]), patch.object(autofmt, "main"):
# Just call main, it should show help and return
autofmt.main()
# main() should return without calling px.run
assert True
def test_main_creates_task_specs_with_verbose(self) -> None:
"""main() should create TaskSpecs with verbose=True."""
with patch("sys.argv", ["autofmt", "fmt"]), patch.object(px, "run") as mock_run:
autofmt.main()
assert mock_run.called
def test_main_uses_thread_strategy(self) -> None:
"""main() should use thread strategy."""
with patch("sys.argv", ["autofmt", "fmt"]), patch.object(px, "run") as mock_run:
autofmt.main()
# Check that strategy="thread" was used
assert mock_run.called
+143 -273
View File
@@ -1,15 +1,13 @@
"""Tests for cli.bumpversion module."""
"""Tests for ops.bumpversion module."""
from __future__ import annotations
import os
import subprocess
from pathlib import Path
from unittest.mock import patch
import pytest
import pyflowx as px
from pyflowx.cli import bumpversion
from pyflowx.ops import bumpversion
@pytest.fixture(autouse=True)
@@ -24,299 +22,171 @@ def auto_use_tmp_path(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
class TestBumpFileVersion:
"""Test bump_file_version function."""
def test_bump_patch_version(self, tmp_path: Path) -> None:
"""Should bump patch version correctly."""
test_file = tmp_path / "pyproject.toml"
test_file.write_text('version = "1.2.3"', encoding="utf-8")
@pytest.mark.parametrize(
("part", "expected"),
[("patch", "1.2.4"), ("minor", "1.3.0"), ("major", "2.0.0")],
)
def test_bump_pyproject(self, tmp_path: Path, part: str, expected: str) -> None:
"""pyproject.toml 三种 part 递增."""
f = tmp_path / "pyproject.toml"
f.write_text('version = "1.2.3"', encoding="utf-8")
assert bumpversion.bump_file_version(f, part) == expected # type: ignore[arg-type]
assert f'version = "{expected}"' in f.read_text(encoding="utf-8")
result = bumpversion.bump_file_version(test_file, "patch")
@pytest.mark.parametrize(
("part", "expected"),
[("patch", "1.2.4"), ("minor", "1.3.0"), ("major", "2.0.0")],
)
def test_bump_init_py(self, tmp_path: Path, part: str, expected: str) -> None:
"""__init__.py 三种 part 递增."""
f = tmp_path / "__init__.py"
f.write_text('__version__ = "1.2.3"', encoding="utf-8")
assert bumpversion.bump_file_version(f, part) == expected # type: ignore[arg-type]
assert f'__version__ = "{expected}"' in f.read_text(encoding="utf-8")
assert result == "1.2.4"
assert test_file.read_text(encoding="utf-8") == 'version = "1.2.4"'
def test_bump_minor_version(self, tmp_path: Path) -> None:
"""Should bump minor version correctly."""
test_file = tmp_path / "pyproject.toml"
test_file.write_text('version = "1.2.3"', encoding="utf-8")
result = bumpversion.bump_file_version(test_file, "minor")
assert result == "1.3.0"
assert test_file.read_text(encoding="utf-8") == 'version = "1.3.0"'
def test_bump_major_version(self, tmp_path: Path) -> None:
"""Should bump major version correctly."""
test_file = tmp_path / "pyproject.toml"
test_file.write_text('version = "1.2.3"', encoding="utf-8")
result = bumpversion.bump_file_version(test_file, "major")
assert result == "2.0.0"
assert test_file.read_text(encoding="utf-8") == 'version = "2.0.0"'
def test_version_pattern_with_prerelease(self, tmp_path: Path) -> None:
"""Should handle version with prerelease suffix."""
test_file = tmp_path / "pyproject.toml"
test_file.write_text('version = "1.2.3-alpha.1"', encoding="utf-8")
result = bumpversion.bump_file_version(test_file, "patch")
assert result == "1.2.4"
# 预发布版本应该被清除
content = test_file.read_text(encoding="utf-8")
def test_prerelease_and_build_metadata_stripped(self, tmp_path: Path) -> None:
"""prerelease 和 build metadata 应被清除."""
f = tmp_path / "pyproject.toml"
f.write_text('version = "1.2.3-alpha.1+build.123"', encoding="utf-8")
assert bumpversion.bump_file_version(f, "patch") == "1.2.4"
content = f.read_text(encoding="utf-8")
assert "alpha" not in content
def test_version_pattern_with_build_metadata(self, tmp_path: Path) -> None:
"""Should handle version with build metadata."""
test_file = tmp_path / "pyproject.toml"
test_file.write_text('version = "1.2.3+build.123"', encoding="utf-8")
result = bumpversion.bump_file_version(test_file, "patch")
assert result == "1.2.4"
# 构建元数据应该被清除
content = test_file.read_text(encoding="utf-8")
assert "build" not in content
def test_no_version_found(self, tmp_path: Path, capsys: pytest.CaptureFixture[str]) -> None:
"""Should return None when no version pattern found."""
test_file = tmp_path / "test.txt"
test_file.write_text("no version here", encoding="utf-8")
def test_dependencies_not_modified(self, tmp_path: Path) -> None:
"""只更新 project version, 不动 dependencies 中的版本号."""
f = tmp_path / "pyproject.toml"
f.write_text(
'[project]\nversion = "1.0.0"\ndependencies = ["lib >= 2.0.0", "other >= 3.0.0"]\n',
encoding="utf-8",
)
assert bumpversion.bump_file_version(f, "patch") == "1.0.1"
content = f.read_text(encoding="utf-8")
assert 'version = "1.0.1"' in content
assert "lib >= 2.0.0" in content
assert "other >= 3.0.0" in content
result = bumpversion.bump_file_version(test_file, "patch")
def test_no_version_pattern_returns_none(self, tmp_path: Path, capsys: pytest.CaptureFixture[str]) -> None:
"""未匹配到版本号模式返回 None (支持类型但无版本 / 不支持的文件类型)."""
f1 = tmp_path / "__init__.py"
f1.write_text("# no version here", encoding="utf-8")
assert bumpversion.bump_file_version(f1, "patch") is None
assert "未找到版本号模式" in capsys.readouterr().out
assert result is None
captured = capsys.readouterr()
assert "未找到版本号模式" in captured.out
def test_utf8_encoding(self, tmp_path: Path) -> None:
"""Should handle UTF-8 encoded files correctly."""
test_file = tmp_path / "__init__.py"
test_file.write_text('__version__ = "1.2.3"', encoding="utf-8")
result = bumpversion.bump_file_version(test_file, "patch")
assert result == "1.2.4"
assert test_file.read_text(encoding="utf-8") == '__version__ = "1.2.4"'
def test_pyproject_toml_format(self, tmp_path: Path) -> None:
"""Should handle pyproject.toml format correctly."""
test_file = tmp_path / "pyproject.toml"
content = """
[project]
name = "test"
version = "0.1.0"
description = "Test project"
"""
test_file.write_text(content, encoding="utf-8")
result = bumpversion.bump_file_version(test_file, "minor")
assert result == "0.2.0"
updated = test_file.read_text(encoding="utf-8")
assert 'version = "0.2.0"' in updated
assert 'name = "test"' in updated
def test_init_py_format(self, tmp_path: Path) -> None:
"""Should handle __init__.py format correctly."""
test_file = tmp_path / "__init__.py"
content = '''"""Package info."""
__version__ = "1.0.0"
'''
test_file.write_text(content, encoding="utf-8")
result = bumpversion.bump_file_version(test_file, "major")
assert result == "2.0.0"
updated = test_file.read_text(encoding="utf-8")
assert '__version__ = "2.0.0"' in updated
def test_multiple_versions_in_file(self, tmp_path: Path) -> None:
"""Should only bump the project version, not dependencies."""
test_file = tmp_path / "pyproject.toml"
content = """
[project]
version = "1.0.0"
dependencies = ["lib >= 2.0.0", "other >= 3.0.0"]
"""
test_file.write_text(content, encoding="utf-8")
result = bumpversion.bump_file_version(test_file, "patch")
assert result == "1.0.1"
updated = test_file.read_text(encoding="utf-8")
assert 'version = "1.0.1"' in updated
# 确保 dependencies 中的版本没有被更新
assert "lib >= 2.0.0" in updated
assert "other >= 3.0.0" in updated
def test_file_read_error(self, tmp_path: Path, capsys: pytest.CaptureFixture[str]) -> None:
"""Should handle file read errors."""
# 创建一个目录而不是文件
test_file = tmp_path / "test_dir"
test_file.mkdir()
f2 = tmp_path / "test.txt"
f2.write_text("no version here", encoding="utf-8")
assert bumpversion.bump_file_version(f2, "patch") is None
def test_read_directory_raises(self, tmp_path: Path) -> None:
"""读取目录 (名为 __init__.py) 应抛异常."""
f = tmp_path / "__init__.py"
f.mkdir()
with pytest.raises(Exception): # noqa: B017
bumpversion.bump_file_version(test_file, "patch")
bumpversion.bump_file_version(f, "patch")
def test_file_write_error(self, tmp_path: Path, capsys: pytest.CaptureFixture[str]) -> None:
"""Should handle file write errors."""
if os.geteuid() == 0:
pytest.skip("测试在 root 权限下不适用")
def test_write_failure_raises(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""写入失败应抛 OSError."""
f = tmp_path / "__init__.py"
f.write_text('__version__ = "1.0.0"', encoding="utf-8")
# 在只读目录中创建文件(这个测试在某些系统上可能不适用)
test_file = tmp_path / "readonly.toml"
test_file.write_text('version = "1.0.0"', encoding="utf-8")
# 设置为只读
test_file.chmod(0o444)
def raise_oserror(*_args: object, **_kwargs: object) -> None:
raise OSError("write failed")
try:
with pytest.raises(Exception): # noqa: B017
bumpversion.bump_file_version(test_file, "patch")
finally:
# 恢复权限以便清理
test_file.chmod(0o644)
monkeypatch.setattr(Path, "write_text", raise_oserror)
with pytest.raises(OSError, match="write failed"):
bumpversion.bump_file_version(f, "patch")
# ---------------------------------------------------------------------- #
# Version pattern tests
# bump_project_version (核心 bug 修复: 不同步文件统一同步)
# ---------------------------------------------------------------------- #
class TestVersionPattern:
"""Test version pattern matching."""
class TestBumpProjectVersion:
"""Test bump_project_version function."""
def test_simple_version(self, tmp_path: Path) -> None:
"""Should match simple version."""
test_file = tmp_path / "__init__.py"
test_file.write_text('__version__ = "1.0.0"', encoding="utf-8")
@staticmethod
def _mock_subprocess(monkeypatch: pytest.MonkeyPatch) -> list[list[str]]:
"""Mock subprocess.run, 返回调用记录列表."""
calls: list[list[str]] = []
result = bumpversion.bump_file_version(test_file, "patch")
def fake_run(cmd: list[str], **_kwargs: object) -> subprocess.CompletedProcess[bytes]:
calls.append(cmd)
return subprocess.CompletedProcess(cmd, 0, b"", b"")
assert result == "1.0.1"
monkeypatch.setattr(subprocess, "run", fake_run)
return calls
def test_version_with_zeros(self, tmp_path: Path) -> None:
"""Should handle versions with zeros correctly."""
test_file = tmp_path / "__init__.py"
test_file.write_text('__version__ = "0.0.0"', encoding="utf-8")
def test_unsynced_files_synchronized(
self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
) -> None:
"""核心 bug 修复: 不同步的文件应统一同步到同一新版本号.
result = bumpversion.bump_file_version(test_file, "patch")
场景: __init__.py = 0.4.5, pyproject.toml = 0.3.5 (历史不同步)
期望: bump patch 后两者都变为 0.4.6 (取最大值 0.4.5 作为基准 +1)
"""
init_file = tmp_path / "src" / "pkg" / "__init__.py"
init_file.parent.mkdir(parents=True)
init_file.write_text('__version__ = "0.4.5"', encoding="utf-8")
pyproj = tmp_path / "pyproject.toml"
pyproj.write_text('version = "0.3.5"', encoding="utf-8")
assert result == "0.0.1"
calls = self._mock_subprocess(monkeypatch)
def test_large_version_numbers(self, tmp_path: Path) -> None:
"""Should handle large version numbers."""
test_file = tmp_path / "__init__.py"
test_file.write_text('__version__ = "10.20.30"', encoding="utf-8")
result = bumpversion.bump_project_version("patch")
result = bumpversion.bump_file_version(test_file, "minor")
assert result == "0.4.6"
assert '__version__ = "0.4.6"' in init_file.read_text(encoding="utf-8")
assert 'version = "0.4.6"' in pyproj.read_text(encoding="utf-8")
out = capsys.readouterr().out
assert "基准版本: 0.4.5" in out
assert "新版本: 0.4.6" in out
assert result == "10.21.0"
add_calls = [c for c in calls if c[:2] == ["git", "add"]]
assert len(add_calls) == 1
# 跨平台: Windows 上 Path 转换为反斜杠, 统一用正斜杠比较
init_path = str(init_file.relative_to(tmp_path)).replace("\\", "/")
assert init_path in [arg.replace("\\", "/") for arg in add_calls[0]]
assert "pyproject.toml" in add_calls[0]
assert "." not in add_calls[0][2:]
def test_version_in_url(self, tmp_path: Path) -> None:
"""Should not match version in URL or other contexts."""
test_file = tmp_path / "test.txt"
test_file.write_text("https://example.com/v1.2.3/download", encoding="utf-8")
tag_calls = [c for c in calls if c[:2] == ["git", "tag"]]
assert len(tag_calls) == 1
assert "v0.4.6" in tag_calls[0]
result = bumpversion.bump_file_version(test_file, "patch")
# 不应该匹配 URL 中的版本号
assert result is None
# ---------------------------------------------------------------------- #
# Edge cases
# ---------------------------------------------------------------------- #
class TestEdgeCases:
"""Test edge cases and error handling."""
def test_empty_file(self, tmp_path: Path, capsys: pytest.CaptureFixture[str]) -> None:
"""Should handle empty file."""
test_file = tmp_path / "empty.txt"
test_file.write_text("", encoding="utf-8")
result = bumpversion.bump_file_version(test_file, "patch")
assert result is None
captured = capsys.readouterr()
assert "未找到版本号模式" in captured.out
def test_file_with_special_chars(self, tmp_path: Path) -> None:
"""Should handle file with special characters."""
test_file = tmp_path / "__init__.py"
content = '# 中文注释\n__version__ = "1.0.0"\n# 特殊字符: @#$%'
test_file.write_text(content, encoding="utf-8")
result = bumpversion.bump_file_version(test_file, "patch")
assert result == "1.0.1"
updated = test_file.read_text(encoding="utf-8")
assert "# 中文注释" in updated
assert "# 特殊字符: @#$%" in updated
def test_consecutive_bumps(self, tmp_path: Path) -> None:
"""Should handle consecutive version bumps correctly."""
test_file = tmp_path / "__init__.py"
test_file.write_text('__version__ = "1.0.0"', encoding="utf-8")
# 第一次 bump
result1 = bumpversion.bump_file_version(test_file, "patch")
assert result1 == "1.0.1"
# 第二次 bump
result2 = bumpversion.bump_file_version(test_file, "minor")
assert result2 == "1.1.0"
# 第三次 bump
result3 = bumpversion.bump_file_version(test_file, "major")
assert result3 == "2.0.0"
# 验证最终结果
assert test_file.read_text(encoding="utf-8") == '__version__ = "2.0.0"'
class TestBumpVersionCli:
"""Test bumpversion CLI."""
def test_minor(self, tmp_path: Path) -> None:
"""Should handle minor version bump."""
test_file = tmp_path / "__init__.py"
test_file.write_text('__version__ = "1.0.0"', encoding="utf-8")
# Mock px.run: 只真正执行第一次调用(版本更新),其余返回空 dict
with patch("sys.argv", ["bumpversion", "minor", "--no-tag"]), patch("pyflowx.run") as mock_run:
def run_side_effect(graph: px.Graph, strategy: str | None = None):
# 执行实际版本更新任务
results = {}
for spec in graph.specs.values():
if spec.fn is not None and spec.args:
results[spec.name] = spec.fn(*spec.args)
return results
mock_run.side_effect = run_side_effect
bumpversion.main()
# 验证版本号已更新
assert test_file.read_text(encoding="utf-8") == '__version__ = "1.1.0"'
def test_no_valid_files(self, tmp_path: Path, capsys: pytest.CaptureFixture[str]) -> None:
"""Should handle no valid files."""
test_file = tmp_path / "test.txt"
test_file.write_text("这是一个测试文件", encoding="utf-8")
with patch("sys.argv", ["bumpversion", "minor", "--no-tag"]), patch("pyflowx.run") as mock_run:
def run_side_effect(graph: px.Graph, strategy: str | None = None):
# 执行实际版本更新任务
results = {}
for spec in graph.specs.values():
if spec.fn is not None and spec.args:
results[spec.name] = spec.fn(*spec.args)
return results
mock_run.side_effect = run_side_effect
bumpversion.main()
# 验证未更新任何文件
assert test_file.read_text(encoding="utf-8") == "这是一个测试文件"
def test_no_files_returns_none(self, capsys: pytest.CaptureFixture[str]) -> None:
"""无版本号文件返回 None."""
assert bumpversion.bump_project_version("patch") is None
assert "未找到包含版本号的文件" in capsys.readouterr().out
def test_files_without_version_returns_none(
self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
) -> None:
"""有文件但所有文件都无版本号返回 None."""
f = tmp_path / "__init__.py"
f.write_text("# no version here", encoding="utf-8")
self._mock_subprocess(monkeypatch)
assert bumpversion.bump_project_version("patch") is None
assert "未能从任何文件读取版本号" in capsys.readouterr().out
def test_no_tag_skips_tag_creation(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""no_tag=True 跳过 tag 创建."""
pyproj = tmp_path / "pyproject.toml"
pyproj.write_text('version = "1.0.0"', encoding="utf-8")
calls = self._mock_subprocess(monkeypatch)
assert bumpversion.bump_project_version("patch", no_tag=True) == "1.0.1"
assert not any(c[:2] == ["git", "tag"] for c in calls)
def test_ignored_dirs_excluded(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
""".venv 等忽略目录中的版本号文件不被处理."""
venv_init = tmp_path / ".venv" / "lib" / "pkg" / "__init__.py"
venv_init.parent.mkdir(parents=True)
venv_init.write_text('__version__ = "0.1.0"', encoding="utf-8")
pyproj = tmp_path / "pyproject.toml"
pyproj.write_text('version = "1.0.0"', encoding="utf-8")
self._mock_subprocess(monkeypatch)
assert bumpversion.bump_project_version("patch") == "1.0.1"
assert venv_init.read_text(encoding="utf-8") == '__version__ = "0.1.0"'
-21
View File
@@ -1,21 +0,0 @@
"""Tests for cli.clearscreen module."""
from __future__ import annotations
from unittest.mock import patch
import pyflowx as px
from pyflowx.cli.system import clearscreen
# ---------------------------------------------------------------------- #
# main function
# ---------------------------------------------------------------------- #
class TestMain:
"""Test main function."""
def test_main_creates_graph_and_runs(self) -> None:
"""main() should create a Graph and run it."""
with patch.object(px, "run") as mock_run:
clearscreen.main()
assert mock_run.called
+210 -182
View File
@@ -125,19 +125,21 @@ class TestEmailDatabase:
# Insert test emails
for i in range(5):
db.insert_email({
"file_path": f"/test/path{i}.eml",
"file_hash": f"hash{i}",
"subject": f"Subject {i}",
"sender": f"sender{i}@example.com",
"recipients": "recipient@example.com",
"date": f"Mon, {i + 1} Jan 2024 12:00:00 +0000",
"date_parsed": f"2024-01-0{i + 1}T12:00:00",
"body_text": f"Body {i}",
"body_html": f"<p>Body {i}</p>",
"has_attachments": 0,
"file_size": 1024,
})
db.insert_email(
{
"file_path": f"/test/path{i}.eml",
"file_hash": f"hash{i}",
"subject": f"Subject {i}",
"sender": f"sender{i}@example.com",
"recipients": "recipient@example.com",
"date": f"Mon, {i + 1} Jan 2024 12:00:00 +0000",
"date_parsed": f"2024-01-0{i + 1}T12:00:00",
"body_text": f"Body {i}",
"body_html": f"<p>Body {i}</p>",
"has_attachments": 0,
"file_size": 1024,
}
)
results = db.search_emails(limit=3)
assert len(results) == 3
@@ -148,33 +150,37 @@ class TestEmailDatabase:
db_path = tmp_path / "test.db"
db = emlmanager.EmailDatabase(db_path)
db.insert_email({
"file_path": "/test/path1.eml",
"file_hash": "hash1",
"subject": "Important Meeting",
"sender": "sender1@example.com",
"recipients": "recipient@example.com",
"date": "Mon, 1 Jan 2024 12:00:00 +0000",
"date_parsed": "2024-01-01T12:00:00",
"body_text": "Meeting body",
"body_html": "<p>Meeting body</p>",
"has_attachments": 0,
"file_size": 1024,
})
db.insert_email(
{
"file_path": "/test/path1.eml",
"file_hash": "hash1",
"subject": "Important Meeting",
"sender": "sender1@example.com",
"recipients": "recipient@example.com",
"date": "Mon, 1 Jan 2024 12:00:00 +0000",
"date_parsed": "2024-01-01T12:00:00",
"body_text": "Meeting body",
"body_html": "<p>Meeting body</p>",
"has_attachments": 0,
"file_size": 1024,
}
)
db.insert_email({
"file_path": "/test/path2.eml",
"file_hash": "hash2",
"subject": "Casual Chat",
"sender": "sender2@example.com",
"recipients": "recipient@example.com",
"date": "Tue, 2 Jan 2024 12:00:00 +0000",
"date_parsed": "2024-01-02T12:00:00",
"body_text": "Chat body",
"body_html": "<p>Chat body</p>",
"has_attachments": 0,
"file_size": 1024,
})
db.insert_email(
{
"file_path": "/test/path2.eml",
"file_hash": "hash2",
"subject": "Casual Chat",
"sender": "sender2@example.com",
"recipients": "recipient@example.com",
"date": "Tue, 2 Jan 2024 12:00:00 +0000",
"date_parsed": "2024-01-02T12:00:00",
"body_text": "Chat body",
"body_html": "<p>Chat body</p>",
"has_attachments": 0,
"file_size": 1024,
}
)
results = db.search_emails(keyword="Meeting", field="subject")
assert len(results) == 1
@@ -186,33 +192,37 @@ class TestEmailDatabase:
db_path = tmp_path / "test.db"
db = emlmanager.EmailDatabase(db_path)
db.insert_email({
"file_path": "/test/path1.eml",
"file_hash": "hash1",
"subject": "Test",
"sender": "alice@example.com",
"recipients": "recipient@example.com",
"date": "Mon, 1 Jan 2024 12:00:00 +0000",
"date_parsed": "2024-01-01T12:00:00",
"body_text": "Body",
"body_html": "<p>Body</p>",
"has_attachments": 0,
"file_size": 1024,
})
db.insert_email(
{
"file_path": "/test/path1.eml",
"file_hash": "hash1",
"subject": "Test",
"sender": "alice@example.com",
"recipients": "recipient@example.com",
"date": "Mon, 1 Jan 2024 12:00:00 +0000",
"date_parsed": "2024-01-01T12:00:00",
"body_text": "Body",
"body_html": "<p>Body</p>",
"has_attachments": 0,
"file_size": 1024,
}
)
db.insert_email({
"file_path": "/test/path2.eml",
"file_hash": "hash2",
"subject": "Test",
"sender": "bob@example.com",
"recipients": "recipient@example.com",
"date": "Tue, 2 Jan 2024 12:00:00 +0000",
"date_parsed": "2024-01-02T12:00:00",
"body_text": "Body",
"body_html": "<p>Body</p>",
"has_attachments": 0,
"file_size": 1024,
})
db.insert_email(
{
"file_path": "/test/path2.eml",
"file_hash": "hash2",
"subject": "Test",
"sender": "bob@example.com",
"recipients": "recipient@example.com",
"date": "Tue, 2 Jan 2024 12:00:00 +0000",
"date_parsed": "2024-01-02T12:00:00",
"body_text": "Body",
"body_html": "<p>Body</p>",
"has_attachments": 0,
"file_size": 1024,
}
)
results = db.search_emails(keyword="alice", field="sender")
assert len(results) == 1
@@ -224,19 +234,21 @@ class TestEmailDatabase:
db_path = tmp_path / "test.db"
db = emlmanager.EmailDatabase(db_path)
db.insert_email({
"file_path": "/test/path1.eml",
"file_hash": "hash1",
"subject": "Project Update",
"sender": "manager@example.com",
"recipients": "team@example.com",
"date": "Mon, 1 Jan 2024 12:00:00 +0000",
"date_parsed": "2024-01-01T12:00:00",
"body_text": "Please review the quarterly report",
"body_html": "<p>Please review the quarterly report</p>",
"has_attachments": 0,
"file_size": 1024,
})
db.insert_email(
{
"file_path": "/test/path1.eml",
"file_hash": "hash1",
"subject": "Project Update",
"sender": "manager@example.com",
"recipients": "team@example.com",
"date": "Mon, 1 Jan 2024 12:00:00 +0000",
"date_parsed": "2024-01-01T12:00:00",
"body_text": "Please review the quarterly report",
"body_html": "<p>Please review the quarterly report</p>",
"has_attachments": 0,
"file_size": 1024,
}
)
# Search for keyword in subject
results = db.search_emails(keyword="Project", field="all")
@@ -253,47 +265,53 @@ class TestEmailDatabase:
db = emlmanager.EmailDatabase(db_path)
# Insert emails with same subject (different prefixes)
db.insert_email({
"file_path": "/test/path1.eml",
"file_hash": "hash1",
"subject": "Meeting Tomorrow",
"sender": "sender1@example.com",
"recipients": "recipient@example.com",
"date": "Mon, 1 Jan 2024 12:00:00 +0000",
"date_parsed": "2024-01-01T12:00:00",
"body_text": "Body 1",
"body_html": "<p>Body 1</p>",
"has_attachments": 0,
"file_size": 1024,
})
db.insert_email(
{
"file_path": "/test/path1.eml",
"file_hash": "hash1",
"subject": "Meeting Tomorrow",
"sender": "sender1@example.com",
"recipients": "recipient@example.com",
"date": "Mon, 1 Jan 2024 12:00:00 +0000",
"date_parsed": "2024-01-01T12:00:00",
"body_text": "Body 1",
"body_html": "<p>Body 1</p>",
"has_attachments": 0,
"file_size": 1024,
}
)
db.insert_email({
"file_path": "/test/path2.eml",
"file_hash": "hash2",
"subject": "Re: Meeting Tomorrow",
"sender": "sender2@example.com",
"recipients": "recipient@example.com",
"date": "Tue, 2 Jan 2024 12:00:00 +0000",
"date_parsed": "2024-01-02T12:00:00",
"body_text": "Body 2",
"body_html": "<p>Body 2</p>",
"has_attachments": 0,
"file_size": 1024,
})
db.insert_email(
{
"file_path": "/test/path2.eml",
"file_hash": "hash2",
"subject": "Re: Meeting Tomorrow",
"sender": "sender2@example.com",
"recipients": "recipient@example.com",
"date": "Tue, 2 Jan 2024 12:00:00 +0000",
"date_parsed": "2024-01-02T12:00:00",
"body_text": "Body 2",
"body_html": "<p>Body 2</p>",
"has_attachments": 0,
"file_size": 1024,
}
)
db.insert_email({
"file_path": "/test/path3.eml",
"file_hash": "hash3",
"subject": "Different Topic",
"sender": "sender3@example.com",
"recipients": "recipient@example.com",
"date": "Wed, 3 Jan 2024 12:00:00 +0000",
"date_parsed": "2024-01-03T12:00:00",
"body_text": "Body 3",
"body_html": "<p>Body 3</p>",
"has_attachments": 0,
"file_size": 1024,
})
db.insert_email(
{
"file_path": "/test/path3.eml",
"file_hash": "hash3",
"subject": "Different Topic",
"sender": "sender3@example.com",
"recipients": "recipient@example.com",
"date": "Wed, 3 Jan 2024 12:00:00 +0000",
"date_parsed": "2024-01-03T12:00:00",
"body_text": "Body 3",
"body_html": "<p>Body 3</p>",
"has_attachments": 0,
"file_size": 1024,
}
)
grouped = db.get_grouped_emails()
# Should have 2 groups: "Meeting Tomorrow" and "Different Topic"
@@ -322,19 +340,21 @@ class TestEmailDatabase:
assert db.get_email_count() == 0
for i in range(3):
db.insert_email({
"file_path": f"/test/path{i}.eml",
"file_hash": f"hash{i}",
"subject": f"Subject {i}",
"sender": f"sender{i}@example.com",
"recipients": "recipient@example.com",
"date": f"Mon, {i + 1} Jan 2024 12:00:00 +0000",
"date_parsed": f"2024-01-0{i + 1}T12:00:00",
"body_text": f"Body {i}",
"body_html": f"<p>Body {i}</p>",
"has_attachments": 0,
"file_size": 1024,
})
db.insert_email(
{
"file_path": f"/test/path{i}.eml",
"file_hash": f"hash{i}",
"subject": f"Subject {i}",
"sender": f"sender{i}@example.com",
"recipients": "recipient@example.com",
"date": f"Mon, {i + 1} Jan 2024 12:00:00 +0000",
"date_parsed": f"2024-01-0{i + 1}T12:00:00",
"body_text": f"Body {i}",
"body_html": f"<p>Body {i}</p>",
"has_attachments": 0,
"file_size": 1024,
}
)
assert db.get_email_count() == 3
db.close()
@@ -346,19 +366,21 @@ class TestEmailDatabase:
# Insert some emails
for i in range(3):
db.insert_email({
"file_path": f"/test/path{i}.eml",
"file_hash": f"hash{i}",
"subject": f"Subject {i}",
"sender": f"sender{i}@example.com",
"recipients": "recipient@example.com",
"date": f"Mon, {i + 1} Jan 2024 12:00:00 +0000",
"date_parsed": f"2024-01-0{i + 1}T12:00:00",
"body_text": f"Body {i}",
"body_html": f"<p>Body {i}</p>",
"has_attachments": 0,
"file_size": 1024,
})
db.insert_email(
{
"file_path": f"/test/path{i}.eml",
"file_hash": f"hash{i}",
"subject": f"Subject {i}",
"sender": f"sender{i}@example.com",
"recipients": "recipient@example.com",
"date": f"Mon, {i + 1} Jan 2024 12:00:00 +0000",
"date_parsed": f"2024-01-0{i + 1}T12:00:00",
"body_text": f"Body {i}",
"body_html": f"<p>Body {i}</p>",
"has_attachments": 0,
"file_size": 1024,
}
)
assert db.get_email_count() == 3
@@ -687,19 +709,21 @@ class TestEmlManagerHandler:
# Insert some emails
for i in range(3):
db.insert_email({
"file_path": f"/test/path{i}.eml",
"file_hash": f"hash{i}",
"subject": f"Subject {i}",
"sender": f"sender{i}@example.com",
"recipients": "recipient@example.com",
"date": f"Mon, {i + 1} Jan 2024 12:00:00 +0000",
"date_parsed": f"2024-01-0{i + 1}T12:00:00",
"body_text": f"Body {i}",
"body_html": f"<p>Body {i}</p>",
"has_attachments": 0,
"file_size": 1024,
})
db.insert_email(
{
"file_path": f"/test/path{i}.eml",
"file_hash": f"hash{i}",
"subject": f"Subject {i}",
"sender": f"sender{i}@example.com",
"recipients": "recipient@example.com",
"date": f"Mon, {i + 1} Jan 2024 12:00:00 +0000",
"date_parsed": f"2024-01-0{i + 1}T12:00:00",
"body_text": f"Body {i}",
"body_html": f"<p>Body {i}</p>",
"has_attachments": 0,
"file_size": 1024,
}
)
# Create a mock handler instance without calling __init__
handler = Mock(spec=emlmanager.EmlManagerHandler)
@@ -721,19 +745,21 @@ class TestEmlManagerHandler:
db = emlmanager.EmailDatabase(db_path)
# Insert test email
db.insert_email({
"file_path": "/test/path.eml",
"file_hash": "hash",
"subject": "Test Subject",
"sender": "sender@example.com",
"recipients": "recipient@example.com",
"date": "Mon, 1 Jan 2024 12:00:00 +0000",
"date_parsed": "2024-01-01T12:00:00",
"body_text": "Test body",
"body_html": "<p>Test body</p>",
"has_attachments": 0,
"file_size": 1024,
})
db.insert_email(
{
"file_path": "/test/path.eml",
"file_hash": "hash",
"subject": "Test Subject",
"sender": "sender@example.com",
"recipients": "recipient@example.com",
"date": "Mon, 1 Jan 2024 12:00:00 +0000",
"date_parsed": "2024-01-01T12:00:00",
"body_text": "Test body",
"body_html": "<p>Test body</p>",
"has_attachments": 0,
"file_size": 1024,
}
)
# Create a mock handler instance without calling __init__
handler = Mock(spec=emlmanager.EmlManagerHandler)
@@ -756,19 +782,21 @@ class TestEmlManagerHandler:
db = emlmanager.EmailDatabase(db_path)
# Insert test email
db.insert_email({
"file_path": "/test/path.eml",
"file_hash": "hash",
"subject": "Test Subject",
"sender": "sender@example.com",
"recipients": "recipient@example.com",
"date": "Mon, 1 Jan 2024 12:00:00 +0000",
"date_parsed": "2024-01-01T12:00:00",
"body_text": "Test body",
"body_html": "<p>Test body</p>",
"has_attachments": 0,
"file_size": 1024,
})
db.insert_email(
{
"file_path": "/test/path.eml",
"file_hash": "hash",
"subject": "Test Subject",
"sender": "sender@example.com",
"recipients": "recipient@example.com",
"date": "Mon, 1 Jan 2024 12:00:00 +0000",
"date_parsed": "2024-01-01T12:00:00",
"body_text": "Test body",
"body_html": "<p>Test body</p>",
"has_attachments": 0,
"file_size": 1024,
}
)
assert db.get_email_count() == 1
+389
View File
@@ -0,0 +1,389 @@
"""Tests for ops.envdev / ops.dockercmd 模块 (镜像源配置/Docker 登录)."""
from __future__ import annotations
import subprocess
from pathlib import Path
from unittest.mock import MagicMock
import pytest
from pyflowx.conditions import Constants
from pyflowx.ops.dockercmd import docker_login_tencent
from pyflowx.ops.envdev import (
download_rustup_script,
install_linux_docker,
install_linux_fonts,
install_linux_qt_libs,
install_rust_toolchain,
setup_conda_mirror,
setup_linux_system_mirror,
setup_python_mirror,
setup_rust_mirror,
)
# ---------------------------------------------------------------------- #
# setup_python_mirror
# ---------------------------------------------------------------------- #
class TestSetupPythonMirror:
"""``setup_python_mirror`` 函数测试."""
def test_unknown_mirror_skips(self, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]) -> None:
"""未知镜像源应打印提示并跳过."""
monkeypatch.setattr(subprocess, "run", lambda *_, **__: MagicMock())
setup_python_mirror("unknown_mirror")
captured = capsys.readouterr()
assert "未知 Python 镜像源" in captured.out
def test_known_mirror_writes_config(self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None:
"""已知镜像源应写入配置文件."""
monkeypatch.setattr(Path, "home", lambda: tmp_path)
setup_python_mirror("tsinghua")
# Linux 平台默认配置路径
config_path = tmp_path / ".pip" / "pip.conf"
if not config_path.exists():
config_path = tmp_path / "pip" / "pip.ini"
assert config_path.exists()
content = config_path.read_text(encoding="utf-8")
assert "pypi.tuna.tsinghua.edu.cn" in content
def test_linux_uses_pip_conf(self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None:
"""Linux 平台应写入 ~/.pip/pip.conf."""
monkeypatch.setattr(Constants, "IS_LINUX", True)
monkeypatch.setattr(Path, "home", lambda: tmp_path)
setup_python_mirror("tsinghua")
config_path = tmp_path / ".pip" / "pip.conf"
assert config_path.exists()
def test_non_linux_uses_pip_ini(self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None:
"""非 Linux 平台应写入 ~/pip/pip.ini."""
monkeypatch.setattr(Constants, "IS_LINUX", False)
monkeypatch.setattr(Path, "home", lambda: tmp_path)
setup_python_mirror("tsinghua")
config_path = tmp_path / "pip" / "pip.ini"
assert config_path.exists()
def test_sets_env_vars(self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None:
"""应设置 PIP_INDEX_URL 等环境变量."""
import os
monkeypatch.setattr(Path, "home", lambda: tmp_path)
monkeypatch.setattr(os, "environ", {})
setup_python_mirror("aliyun")
assert "PIP_INDEX_URL" in os.environ
assert "mirrors.aliyun.com" in os.environ["PIP_INDEX_URL"]
# ---------------------------------------------------------------------- #
# setup_conda_mirror
# ---------------------------------------------------------------------- #
class TestSetupCondaMirror:
"""``setup_conda_mirror`` 函数测试."""
def test_unknown_mirror_skips(self, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]) -> None:
"""未知镜像源应跳过."""
setup_conda_mirror("unknown")
captured = capsys.readouterr()
assert "未知 Conda 镜像源" in captured.out
def test_known_mirror_writes_condarc(self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None:
"""已知镜像源应写入 ~/.condarc."""
monkeypatch.setattr(Path, "home", lambda: tmp_path)
setup_conda_mirror("tsinghua")
condarc = tmp_path / ".condarc"
assert condarc.exists()
content = condarc.read_text(encoding="utf-8")
assert "tsinghua" in content
assert "channels:" in content
# ---------------------------------------------------------------------- #
# setup_rust_mirror
# ---------------------------------------------------------------------- #
class TestSetupRustMirror:
"""``setup_rust_mirror`` 函数测试."""
def test_unknown_mirror_skips(self, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]) -> None:
"""未知镜像源应跳过."""
setup_rust_mirror("unknown")
captured = capsys.readouterr()
assert "未知 Rust 镜像源" in captured.out
def test_known_mirror_writes_config(self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None:
"""已知镜像源应写入 ~/.cargo/config.toml."""
monkeypatch.setattr(Path, "home", lambda: tmp_path)
setup_rust_mirror("ustc", "nightly")
config = tmp_path / ".cargo" / "config.toml"
assert config.exists()
content = config.read_text(encoding="utf-8")
assert "ustc" in content
def test_creates_sccache_dir(self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None:
"""应创建 sccache 缓存目录."""
from pyflowx.ops import envdev as envdev_module
fake_sccache = tmp_path / ".cargo" / "sccache"
monkeypatch.setattr(envdev_module, "_RUST_SCCACHE_DIR", fake_sccache)
monkeypatch.setattr(Path, "home", lambda: tmp_path)
setup_rust_mirror("tsinghua")
assert fake_sccache.exists()
# ---------------------------------------------------------------------- #
# docker_login_tencent
# ---------------------------------------------------------------------- #
class TestDockerLoginTencent:
"""``docker_login_tencent`` 函数测试."""
def test_default_username_uses_getpass(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""未提供 username 时应使用 getpass.getuser."""
monkeypatch.setattr("getpass.getuser", lambda: "testuser")
ran_cmds: list[list[str]] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: ran_cmds.append(cmd) or MagicMock())
docker_login_tencent()
assert ran_cmds[0][0] == "docker"
assert ran_cmds[0][1] == "login"
assert "testuser" in ran_cmds[0]
assert "ccr.ccs.tencentyun.com" in ran_cmds[0]
def test_custom_username(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""提供 username 时应使用自定义用户名."""
ran_cmds: list[list[str]] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: ran_cmds.append(cmd) or MagicMock())
docker_login_tencent("myuser")
assert "myuser" in ran_cmds[0]
# ---------------------------------------------------------------------- #
# setup_linux_system_mirror
# ---------------------------------------------------------------------- #
class TestSetupLinuxSystemMirror:
"""``setup_linux_system_mirror`` 函数测试."""
def test_non_linux_skips(self, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]) -> None:
"""非 Linux 平台应跳过."""
monkeypatch.setattr(Constants, "IS_LINUX", False)
called: list[str] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: called.append(cmd))
setup_linux_system_mirror()
captured = capsys.readouterr()
assert "仅在 Linux 上执行" in captured.out
assert called == []
def test_linux_already_configured_skips(
self, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
) -> None:
"""Linux 上已配置国内镜像时应跳过."""
monkeypatch.setattr(Constants, "IS_LINUX", True)
def fake_read_text(self: Path, encoding: str = "utf-8") -> str:
return "tsinghua mirror configured"
monkeypatch.setattr(Path, "read_text", fake_read_text)
called: list[str] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: called.append(cmd))
setup_linux_system_mirror()
captured = capsys.readouterr()
assert "已配置" in captured.out
assert called == []
def test_linux_not_configured_runs_script(
self, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
) -> None:
"""Linux 上未配置镜像时应执行下载与安装脚本."""
monkeypatch.setattr(Constants, "IS_LINUX", True)
def fake_read_text(self: Path, encoding: str = "utf-8") -> str:
raise OSError("file not found")
monkeypatch.setattr(Path, "read_text", fake_read_text)
ran_cmds: list[str] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: ran_cmds.append(cmd) or MagicMock())
setup_linux_system_mirror()
captured = capsys.readouterr()
assert "下载" in captured.out
assert "安装" in captured.out
assert len(ran_cmds) == 2
def test_linux_content_without_mirror_runs_script(
self, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
) -> None:
"""Linux 上文件存在但不包含镜像关键词时应执行脚本."""
monkeypatch.setattr(Constants, "IS_LINUX", True)
def fake_read_text(self: Path, encoding: str = "utf-8") -> str:
return "deb http://archive.ubuntu.com/ubuntu/ jammy main"
monkeypatch.setattr(Path, "read_text", fake_read_text)
ran_cmds: list[str] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: ran_cmds.append(cmd) or MagicMock())
setup_linux_system_mirror()
captured = capsys.readouterr()
assert "下载" in captured.out
assert len(ran_cmds) == 2
# ---------------------------------------------------------------------- #
# install_linux_qt_libs / install_linux_fonts / install_linux_docker
# ---------------------------------------------------------------------- #
class TestLinuxInstallers:
"""Linux 专用安装函数测试."""
def test_qt_libs_non_linux_skips(self, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]) -> None:
"""非 Linux 上 install_linux_qt_libs 应跳过."""
monkeypatch.setattr(Constants, "IS_LINUX", False)
called: list[list[str]] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: called.append(cmd))
install_linux_qt_libs()
captured = capsys.readouterr()
assert "仅在 Linux" in captured.out
assert called == []
def test_qt_libs_linux_runs_apt(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""Linux 上应执行 apt install."""
monkeypatch.setattr(Constants, "IS_LINUX", True)
ran_cmds: list[list[str]] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: ran_cmds.append(cmd) or MagicMock())
install_linux_qt_libs()
assert ran_cmds[0][0] == "sudo"
assert "apt" in ran_cmds[0]
assert "install" in ran_cmds[0]
def test_fonts_non_linux_skips(self, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]) -> None:
"""非 Linux 上 install_linux_fonts 应跳过."""
monkeypatch.setattr(Constants, "IS_LINUX", False)
called: list[list[str]] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: called.append(cmd))
install_linux_fonts()
captured = capsys.readouterr()
assert "仅在 Linux" in captured.out
assert called == []
def test_fonts_linux_runs_apt(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""Linux 上应执行 apt install 字体包."""
monkeypatch.setattr(Constants, "IS_LINUX", True)
ran_cmds: list[list[str]] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: ran_cmds.append(cmd) or MagicMock())
install_linux_fonts()
assert "fonts-noto-cjk" in ran_cmds[0]
def test_docker_non_linux_skips(self, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]) -> None:
"""非 Linux 上 install_linux_docker 应跳过."""
monkeypatch.setattr(Constants, "IS_LINUX", False)
called: list[list[str]] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: called.append(cmd))
install_linux_docker()
captured = capsys.readouterr()
assert "仅在 Linux" in captured.out
assert called == []
def test_docker_linux_runs_install_and_usermod(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""Linux 上应执行 apt install docker-compose-v2 和 usermod."""
monkeypatch.setattr(Constants, "IS_LINUX", True)
monkeypatch.setattr("getpass.getuser", lambda: "testuser")
ran_cmds: list[list[str]] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: ran_cmds.append(cmd) or MagicMock())
install_linux_docker()
assert any("docker-compose-v2" in cmd for cmd in ran_cmds)
assert any("usermod" in cmd and "docker" in cmd for cmd in ran_cmds)
# ---------------------------------------------------------------------- #
# download_rustup_script
# ---------------------------------------------------------------------- #
class TestDownloadRustupScript:
"""``download_rustup_script`` 函数测试."""
def test_rustup_installed_skips(self, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]) -> None:
"""rustup 已安装时应跳过."""
monkeypatch.setattr("shutil.which", lambda _cmd: "/usr/bin/rustup")
called: list[list[str]] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: called.append(cmd))
download_rustup_script()
captured = capsys.readouterr()
assert "已安装" in captured.out
assert called == []
def test_windows_downloads_exe(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""Windows 上应下载 rustup-init.exe."""
monkeypatch.setattr("shutil.which", lambda _cmd: None)
monkeypatch.setattr(Constants, "IS_WINDOWS", True)
ran_cmds: list[list[str]] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: ran_cmds.append(cmd) or MagicMock())
download_rustup_script()
assert "powershell" in ran_cmds[0]
assert "rustup-init.exe" in ran_cmds[0]
def test_non_windows_downloads_sh(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""非 Windows 上应下载 rustup-init.sh."""
monkeypatch.setattr("shutil.which", lambda _cmd: None)
monkeypatch.setattr(Constants, "IS_WINDOWS", False)
ran_cmds: list[list[str]] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: ran_cmds.append(cmd) or MagicMock())
download_rustup_script()
assert "curl" in ran_cmds[0]
assert "rustup-init.sh" in ran_cmds[0]
# ---------------------------------------------------------------------- #
# install_rust_toolchain
# ---------------------------------------------------------------------- #
class TestInstallRustToolchain:
"""``install_rust_toolchain`` 函数测试."""
def test_rustup_not_installed_skips(
self, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
) -> None:
"""rustup 未安装时应跳过."""
monkeypatch.setattr("shutil.which", lambda _cmd: None)
called: list[list[str]] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: called.append(cmd))
install_rust_toolchain()
captured = capsys.readouterr()
assert "未安装" in captured.out
assert called == []
def test_rustup_installed_runs_toolchain_install(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""rustup 已安装时应执行 toolchain install."""
monkeypatch.setattr("shutil.which", lambda _cmd: "/usr/bin/rustup")
ran_cmds: list[list[str]] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: ran_cmds.append(cmd) or MagicMock())
install_rust_toolchain("nightly")
assert ran_cmds == [["rustup", "toolchain", "install", "nightly"]]
def test_default_version_stable(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""默认版本应为 stable."""
monkeypatch.setattr("shutil.which", lambda _cmd: "/usr/bin/rustup")
ran_cmds: list[list[str]] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: ran_cmds.append(cmd) or MagicMock())
install_rust_toolchain()
assert "stable" in ran_cmds[0]
+99
View File
@@ -0,0 +1,99 @@
"""Tests for ops.envdev / sglang / msdownload 模块 (@px.tool 注册验证)."""
from __future__ import annotations
import pyflowx as px
from pyflowx.ops import envdev, msdownload, sglang # noqa: F401 -- 触发 @px.tool 注册
from pyflowx.tools import _TOOL_REGISTRY, get_tool
# ---------------------------------------------------------------------- #
# envdev 注册验证
# ---------------------------------------------------------------------- #
class TestEnvdevRegistration:
"""``envdev`` 模块 ``@px.tool`` 注册验证."""
def test_envdev_registered(self) -> None:
"""envdev 应在 _TOOL_REGISTRY 中注册."""
assert "envdev" in _TOOL_REGISTRY
def test_visible_subcommands(self) -> None:
"""可见子命令应包含核心镜像源配置命令."""
subs = px.list_subcommands("envdev")
assert "setup-python" in subs
assert "setup-conda" in subs
assert "setup-rust" in subs
assert "setup-linux-mirror" in subs
def test_install_subcommands_visible(self) -> None:
"""安装类子命令应可见."""
subs = px.list_subcommands("envdev")
assert "download-rustup" in subs
assert "install-rust" in subs
assert "install-qt-libs" in subs
assert "install-fonts" in subs
assert "install-docker" in subs
def test_setup_python_is_fn(self) -> None:
"""setup-python 应为 fn 任务 (非 cmd)."""
spec = get_tool("envdev", "setup-python")
assert spec.func is not None
assert spec.cmd is None
def test_setup_conda_is_fn(self) -> None:
"""setup-conda 应为 fn 任务."""
spec = get_tool("envdev", "setup-conda")
assert spec.func is not None
def test_setup_rust_is_fn(self) -> None:
"""setup-rust 应为 fn 任务."""
spec = get_tool("envdev", "setup-rust")
assert spec.func is not None
def test_setup_linux_mirror_is_fn(self) -> None:
"""setup-linux-mirror 应为 fn 任务."""
spec = get_tool("envdev", "setup-linux-mirror")
assert spec.func is not None
# ---------------------------------------------------------------------- #
# sglang 注册验证
# ---------------------------------------------------------------------- #
class TestSglangRegistration:
"""``sglang`` 模块 ``@px.tool`` 注册验证."""
def test_sglang_registered(self) -> None:
"""sglang 应在 _TOOL_REGISTRY 中注册."""
assert "sglang" in _TOOL_REGISTRY
def test_visible_subcommands(self) -> None:
"""可见子命令: install, run."""
subs = px.list_subcommands("sglang")
assert "install" in subs
assert "run" in subs
def test_run_needs_install(self) -> None:
"""run 应依赖 install."""
spec = get_tool("sglang", "run")
assert "install" in spec.needs
# ---------------------------------------------------------------------- #
# msdownload 注册验证
# ---------------------------------------------------------------------- #
class TestMsdownloadRegistration:
"""``msdownload`` 模块 ``@px.tool`` 注册验证."""
def test_msdownload_registered(self) -> None:
"""msdownload 应在 _TOOL_REGISTRY 中注册."""
assert "msdownload" in _TOOL_REGISTRY
def test_msdownload_default_spec(self) -> None:
"""msdownload 默认 spec 应可获取且为 fn 任务."""
spec = get_tool("msdownload")
assert spec.func is not None
assert spec.cmd is None
def test_msdownload_run_callable(self) -> None:
"""msdownload_run 应为可调用函数."""
assert callable(msdownload.msdownload_run)
+10 -43
View File
@@ -3,10 +3,8 @@
from __future__ import annotations
from pathlib import Path
from unittest.mock import patch
import pyflowx as px
from pyflowx.cli import filedate
from pyflowx.ops import filedate as files
# ---------------------------------------------------------------------- #
@@ -20,7 +18,7 @@ class TestGetFileTimestamp:
test_file = tmp_path / "test.txt"
test_file.write_text("test content")
timestamp = filedate.get_file_timestamp(test_file)
timestamp = files.get_file_timestamp(test_file)
assert len(timestamp) == 8 # YYYYMMDD format
assert timestamp.isdigit()
@@ -36,7 +34,7 @@ class TestRemoveDatePrefix:
test_file = tmp_path / "20240101_test.txt"
test_file.write_text("test content")
new_path = filedate.remove_date_prefix(test_file)
new_path = files.remove_date_prefix(test_file)
assert new_path.name == "test.txt"
def test_remove_date_prefix_without_date(self, tmp_path: Path) -> None:
@@ -44,7 +42,7 @@ class TestRemoveDatePrefix:
test_file = tmp_path / "test.txt"
test_file.write_text("test content")
new_path = filedate.remove_date_prefix(test_file)
new_path = files.remove_date_prefix(test_file)
assert new_path == test_file
@@ -59,7 +57,7 @@ class TestAddDatePrefix:
test_file = tmp_path / "test.txt"
test_file.write_text("test content")
new_path = filedate.add_date_prefix(test_file)
new_path = files.add_date_prefix(test_file)
assert new_path.name.startswith("20") # Starts with year
assert "_test.txt" in new_path.name
@@ -75,7 +73,7 @@ class TestProcessFileDate:
test_file = tmp_path / "test.txt"
test_file.write_text("test content")
filedate.process_file_date(test_file, clear=False)
files.process_file_date(test_file, clear=False)
# File should be renamed with date prefix
def test_process_file_date_clear(self, tmp_path: Path) -> None:
@@ -83,7 +81,7 @@ class TestProcessFileDate:
test_file = tmp_path / "20240101_test.txt"
test_file.write_text("test content")
filedate.process_file_date(test_file, clear=True)
files.process_file_date(test_file, clear=True)
# File should be renamed without date prefix
@@ -95,42 +93,11 @@ class TestProcessFilesDate:
def test_process_files_date_batch(self, tmp_path: Path) -> None:
"""Should process multiple files."""
files = []
file_list = []
for i in range(3):
test_file = tmp_path / f"test{i}.txt"
test_file.write_text(f"content{i}")
files.append(test_file)
file_list.append(test_file)
filedate.process_files_date(files, clear=False)
files.process_files_date(file_list, clear=False)
# All files should be processed
# ---------------------------------------------------------------------- #
# main function
# ---------------------------------------------------------------------- #
class TestMain:
"""Test main function."""
def test_main_add_command(self, tmp_path: Path) -> None:
"""main() should handle add command."""
test_file = tmp_path / "test.txt"
test_file.write_text("test content")
with patch("sys.argv", ["filedate", "add", str(test_file)]), patch.object(px, "run") as mock_run:
filedate.main()
assert mock_run.called
def test_main_clear_command(self, tmp_path: Path) -> None:
"""main() should handle clear command."""
test_file = tmp_path / "20240101_test.txt"
test_file.write_text("test content")
with patch("sys.argv", ["filedate", "clear", str(test_file)]), patch.object(px, "run") as mock_run:
filedate.main()
assert mock_run.called
def test_main_with_no_args_shows_help(self) -> None:
"""main() with no args should show help."""
with patch("sys.argv", ["filedate"]):
filedate.main()
# Should print help and return
+12 -49
View File
@@ -3,10 +3,8 @@
from __future__ import annotations
from pathlib import Path
from unittest.mock import patch
import pyflowx as px
from pyflowx.cli import filelevel
from pyflowx.ops import filelevel as files
# ---------------------------------------------------------------------- #
@@ -18,19 +16,19 @@ class TestRemoveMarks:
def test_remove_marks_single_mark(self) -> None:
"""Should remove single mark."""
stem = "filename(PUB)"
result = filelevel.remove_marks(stem, ["PUB"])
result = files.remove_marks(stem, ["PUB"])
assert result == "filename"
def test_remove_marks_multiple_marks(self) -> None:
"""Should remove multiple marks."""
stem = "filename(PUB)(NOR)"
result = filelevel.remove_marks(stem, ["PUB", "NOR"])
result = files.remove_marks(stem, ["PUB", "NOR"])
assert result == "filename"
def test_remove_marks_no_marks(self) -> None:
"""Should not change stem without marks."""
stem = "filename"
result = filelevel.remove_marks(stem, ["PUB"])
result = files.remove_marks(stem, ["PUB"])
assert result == "filename"
@@ -45,7 +43,7 @@ class TestProcessFileLevel:
test_file = tmp_path / "test.txt"
test_file.write_text("test content")
filelevel.process_file_level(test_file, level=1)
files.process_file_level(test_file, level=1)
# File should be renamed with PUB level
def test_process_file_level_set_int(self, tmp_path: Path) -> None:
@@ -53,7 +51,7 @@ class TestProcessFileLevel:
test_file = tmp_path / "test.txt"
test_file.write_text("test content")
filelevel.process_file_level(test_file, level=2)
files.process_file_level(test_file, level=2)
# File should be renamed with INT level
def test_process_file_level_clear(self, tmp_path: Path) -> None:
@@ -61,7 +59,7 @@ class TestProcessFileLevel:
test_file = tmp_path / "test(PUB).txt"
test_file.write_text("test content")
filelevel.process_file_level(test_file, level=0)
files.process_file_level(test_file, level=0)
# File should be renamed without level
def test_process_file_level_invalid_level(self, tmp_path: Path) -> None:
@@ -69,14 +67,14 @@ class TestProcessFileLevel:
test_file = tmp_path / "test.txt"
test_file.write_text("test content")
filelevel.process_file_level(test_file, level=5)
files.process_file_level(test_file, level=5)
# Should print error message
def test_process_file_level_nonexistent_file(self, tmp_path: Path) -> None:
"""Should handle nonexistent file."""
test_file = tmp_path / "nonexistent.txt"
filelevel.process_file_level(test_file, level=1)
files.process_file_level(test_file, level=1)
# Should print error message
@@ -88,46 +86,11 @@ class TestProcessFilesLevel:
def test_process_files_level_batch(self, tmp_path: Path) -> None:
"""Should process multiple files."""
files = []
file_list = []
for i in range(3):
test_file = tmp_path / f"test{i}.txt"
test_file.write_text(f"content{i}")
files.append(test_file)
file_list.append(test_file)
filelevel.process_files_level(files, level=1)
files.process_files_level(file_list, level=1)
# All files should be processed
# ---------------------------------------------------------------------- #
# main function
# ---------------------------------------------------------------------- #
class TestMain:
"""Test main function."""
def test_main_set_command(self, tmp_path: Path) -> None:
"""main() should handle set command."""
test_file = tmp_path / "test.txt"
test_file.write_text("test content")
with patch("sys.argv", ["filelevel", "set", str(test_file), "--level", "1"]), patch.object(
px, "run"
) as mock_run:
filelevel.main()
assert mock_run.called
def test_main_set_command_level_2(self, tmp_path: Path) -> None:
"""main() should handle set command with level 2."""
test_file = tmp_path / "test.txt"
test_file.write_text("test content")
with patch("sys.argv", ["filelevel", "set", str(test_file), "--level", "2"]), patch.object(
px, "run"
) as mock_run:
filelevel.main()
assert mock_run.called
def test_main_with_no_args_shows_help(self) -> None:
"""main() with no args should show help."""
with patch("sys.argv", ["filelevel"]):
filelevel.main()
# Should print help and return
+18 -33
View File
@@ -5,8 +5,7 @@ from __future__ import annotations
from pathlib import Path
from unittest.mock import patch
import pyflowx as px
from pyflowx.cli import folderback
from pyflowx.ops import folderback as files
# ---------------------------------------------------------------------- #
@@ -22,7 +21,7 @@ class TestRemoveDump:
dst = tmp_path / "backup"
dst.mkdir()
folderback.remove_dump(src, dst, 5)
files.remove_dump(src, dst, 5)
# Should not raise error
def test_remove_dump_within_limit(self, tmp_path: Path) -> None:
@@ -37,7 +36,7 @@ class TestRemoveDump:
zip_file = dst / f"source_20240101_12000{i}.zip"
zip_file.write_bytes(b"ZIP content")
folderback.remove_dump(src, dst, 5)
files.remove_dump(src, dst, 5)
# All files should remain
assert len(list(dst.glob("*.zip"))) == 3
@@ -53,7 +52,7 @@ class TestRemoveDump:
zip_file = dst / f"source_20240101_12000{i}.zip"
zip_file.write_bytes(b"ZIP content")
folderback.remove_dump(src, dst, 5)
files.remove_dump(src, dst, 5)
# Should have only 5 files
assert len(list(dst.glob("*.zip"))) == 5
@@ -73,7 +72,7 @@ class TestZipTarget:
dst.mkdir()
with patch("time.strftime", return_value="_20240101_120000"):
folderback.zip_target(src, dst, 5)
files.zip_target(src, dst, 5)
# Should create zip file
zip_files = list(dst.glob("*.zip"))
@@ -91,7 +90,7 @@ class TestZipTarget:
dst.mkdir()
with patch("time.strftime", return_value="_20240101_120000"):
folderback.zip_target(src, dst, 5)
files.zip_target(src, dst, 5)
# Should create zip file
zip_files = list(dst.glob("*.zip"))
@@ -111,8 +110,8 @@ class TestBackupFolder:
(source_dir / "test.txt").write_text("test content")
backup_dir = tmp_path / "backup"
with patch.object(folderback, "zip_target") as mock_zip:
folderback.backup_folder(str(source_dir), str(backup_dir), 5)
with patch.object(files, "zip_target") as mock_zip:
files.backup_folder(str(source_dir), str(backup_dir), 5)
assert mock_zip.called
def test_backup_folder_with_max_backups(self, tmp_path: Path) -> None:
@@ -122,8 +121,8 @@ class TestBackupFolder:
(source_dir / "test.txt").write_text("test content")
backup_dir = tmp_path / "backup"
with patch.object(folderback, "zip_target") as mock_zip:
folderback.backup_folder(str(source_dir), str(backup_dir), 10)
with patch.object(files, "zip_target") as mock_zip:
files.backup_folder(str(source_dir), str(backup_dir), 10)
assert mock_zip.called
def test_backup_folder_source_not_exists(self, tmp_path: Path) -> None:
@@ -132,7 +131,7 @@ class TestBackupFolder:
backup_dir = tmp_path / "backup"
backup_dir.mkdir()
folderback.backup_folder(str(source_dir), str(backup_dir), 5)
files.backup_folder(str(source_dir), str(backup_dir), 5)
# Should print error message and return
def test_backup_folder_creates_dst(self, tmp_path: Path) -> None:
@@ -142,32 +141,18 @@ class TestBackupFolder:
(source_dir / "test.txt").write_text("test content")
backup_dir = tmp_path / "backup"
with patch.object(folderback, "zip_target") as mock_zip:
folderback.backup_folder(str(source_dir), str(backup_dir), 5)
with patch.object(files, "zip_target") as mock_zip:
files.backup_folder(str(source_dir), str(backup_dir), 5)
assert backup_dir.exists()
assert mock_zip.called
# ---------------------------------------------------------------------- #
# TaskSpec definitions
# 函数注册
# ---------------------------------------------------------------------- #
class TestTaskSpecDefinitions:
"""Test that all TaskSpec definitions are valid."""
class TestRegisteredFunctions:
"""Test that folderback functions are registered."""
def test_folderback_default_spec(self) -> None:
"""folderback_default spec should be properly defined."""
assert folderback.folderback_default.name == "folderback_default"
assert folderback.folderback_default.fn is not None
# ---------------------------------------------------------------------- #
# main function
# ---------------------------------------------------------------------- #
class TestMain:
"""Test main function."""
def test_main_calls_run_cli(self) -> None:
"""main() should create a CliRunner and call run_cli()."""
with patch.object(px.CliRunner, "run_cli") as mock_run_cli:
folderback.main()
assert mock_run_cli.called
"""folderback_default should be a callable function."""
assert callable(files.folderback_default)
+10 -25
View File
@@ -5,8 +5,7 @@ from __future__ import annotations
from pathlib import Path
from unittest.mock import patch
import pyflowx as px
from pyflowx.cli import folderzip
from pyflowx.ops import folderzip as files
# ---------------------------------------------------------------------- #
@@ -22,7 +21,7 @@ class TestArchiveFolder:
(folder / "test.txt").write_text("test content")
with patch("shutil.make_archive") as mock_archive:
folderzip.archive_folder(folder)
files.archive_folder(folder)
assert mock_archive.called
@@ -39,37 +38,23 @@ class TestZipFolders:
(tmp_path / "folder2").mkdir()
(tmp_path / ".git").mkdir() # Should be ignored
with patch.object(folderzip, "archive_folder") as mock_archive:
folderzip.zip_folders(str(tmp_path))
with patch.object(files, "archive_folder") as mock_archive:
files.zip_folders(str(tmp_path))
# Should archive folder1 and folder2, but not .git
assert mock_archive.call_count == 2
def test_zip_folders_nonexistent_cwd(self, tmp_path: Path) -> None:
"""Should handle nonexistent cwd."""
folderzip.zip_folders(str(tmp_path / "nonexistent"))
files.zip_folders(str(tmp_path / "nonexistent"))
# Should print error message and return
# ---------------------------------------------------------------------- #
# TaskSpec definitions
# 函数注册
# ---------------------------------------------------------------------- #
class TestTaskSpecDefinitions:
"""Test that all TaskSpec definitions are valid."""
class TestRegisteredFunctions:
"""Test that folderzip functions are registered."""
def test_folderzip_default_spec(self) -> None:
"""folderzip_default spec should be properly defined."""
assert folderzip.folderzip_default.name == "folderzip_default"
assert folderzip.folderzip_default.fn is not None
# ---------------------------------------------------------------------- #
# main function
# ---------------------------------------------------------------------- #
class TestMain:
"""Test main function."""
def test_main_calls_run_cli(self) -> None:
"""main() should create a CliRunner and call run_cli()."""
with patch.object(px.CliRunner, "run_cli") as mock_run_cli:
folderzip.main()
assert mock_run_cli.called
"""folderzip_default should be a callable function."""
assert callable(files.folderzip_default)
+23 -66
View File
@@ -8,7 +8,7 @@ from unittest.mock import patch
import pytest
import pyflowx as px
from pyflowx.cli import gittool
from pyflowx.ops import gittool as dev
# ---------------------------------------------------------------------- #
@@ -20,7 +20,7 @@ class TestNotHasGitRepo:
def test_not_has_git_repo_true(self, tmp_path: Path) -> None:
"""Should return True when no .git directory."""
with patch.object(Path, "cwd", return_value=tmp_path):
result = gittool.not_has_git_repo()
result = dev.not_has_git_repo()
assert result is True
def test_not_has_git_repo_false(self, tmp_path: Path) -> None:
@@ -29,7 +29,7 @@ class TestNotHasGitRepo:
git_dir.mkdir()
with patch.object(Path, "cwd", return_value=tmp_path):
result = gittool.not_has_git_repo()
result = dev.not_has_git_repo()
assert result is False
def test_not_has_git_repo_cwd_not_exists(self, tmp_path: Path) -> None:
@@ -37,7 +37,7 @@ class TestNotHasGitRepo:
nonexistent = tmp_path / "nonexistent"
with patch.object(Path, "cwd", return_value=nonexistent):
result = gittool.not_has_git_repo()
result = dev.not_has_git_repo()
assert result is True
@@ -47,19 +47,25 @@ class TestNotHasGitRepo:
class TestHasFiles:
"""Test has_files function."""
def test_has_files_true(self, tmp_path: Path) -> None:
"""Should return True when files exist."""
(tmp_path / "test.txt").write_text("test")
def test_has_files_true(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""Should return True when there are uncommitted changes."""
with patch.object(Path, "cwd", return_value=tmp_path):
result = gittool.has_files()
assert result is True
class _FakeResult:
stdout = " M test.txt\n"
def test_has_files_false(self, tmp_path: Path) -> None:
"""Should return False when no files."""
with patch.object(Path, "cwd", return_value=tmp_path):
result = gittool.has_files()
assert result is False
monkeypatch.setattr("subprocess.run", lambda *_, **__: _FakeResult())
result = dev.has_files()
assert result is True
def test_has_files_false(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""Should return False when no uncommitted changes."""
class _FakeResult:
stdout = ""
monkeypatch.setattr("subprocess.run", lambda *_, **__: _FakeResult())
result = dev.has_files()
assert result is False
# ---------------------------------------------------------------------- #
@@ -76,62 +82,13 @@ class TestInitSubDirs:
subdir2.mkdir()
with patch.object(Path, "cwd", return_value=tmp_path), patch.object(px, "run") as mock_run:
gittool.init_sub_dirs()
dev.init_sub_dirs()
# Should call px.run for each subdirectory
assert mock_run.call_count == 2
def test_init_sub_dirs_no_subdirectories(self, tmp_path: Path) -> None:
"""Should handle no subdirectories."""
with patch.object(Path, "cwd", return_value=tmp_path), patch.object(px, "run") as mock_run:
gittool.init_sub_dirs()
dev.init_sub_dirs()
# Should not call px.run
assert mock_run.call_count == 0
# ---------------------------------------------------------------------- #
# TaskSpec definitions
# ---------------------------------------------------------------------- #
class TestTaskSpecDefinitions:
"""Test that all TaskSpec definitions are valid."""
def test_push_spec(self) -> None:
"""push spec should be properly defined."""
assert gittool.push.name == "push"
assert gittool.push.cmd == ["git", "push"]
def test_pull_spec(self) -> None:
"""pull spec should be properly defined."""
assert gittool.pull.name == "pull"
assert gittool.pull.cmd == ["git", "pull"]
def test_kill_tgit_spec(self) -> None:
"""kill_tgit spec should be properly defined."""
assert gittool.kill_tgit.name == "task_kill"
assert isinstance(gittool.kill_tgit.cmd, list)
assert "taskkill" in gittool.kill_tgit.cmd
# ---------------------------------------------------------------------- #
# main function
# ---------------------------------------------------------------------- #
class TestMain:
"""Test main function."""
def test_main_calls_run_cli(self) -> None:
"""main() should create a CliRunner and call run_cli()."""
with pytest.raises(SystemExit) as exc_info:
gittool.main()
# run_cli() calls sys.exit(), so we should get SystemExit
assert exc_info.value.code in (0, 1, 2)
def test_main_with_list_argument(self) -> None:
"""main() should handle --list argument."""
with patch("sys.argv", ["gittool", "--list"]), pytest.raises(SystemExit) as exc_info:
gittool.main()
assert exc_info.value.code == 0
def test_main_with_no_args_shows_help(self) -> None:
"""main() with no args should show help and exit."""
with patch("sys.argv", ["gittool"]), pytest.raises(SystemExit) as exc_info:
gittool.main()
assert exc_info.value.code == 1
+267
View File
@@ -0,0 +1,267 @@
"""Tests for ops.gittool 模块 (@px.tool 注册验证)."""
from __future__ import annotations
import subprocess
from pathlib import Path
from unittest.mock import patch
import pytest
import pyflowx as px
import pyflowx.ops.gittool
from pyflowx.ops import gittool
from pyflowx.tools import _TOOL_REGISTRY, get_tool
# ---------------------------------------------------------------------- #
# @px.tool 注册验证
# ---------------------------------------------------------------------- #
class TestGittoolRegistration:
"""``gittool`` 模块 ``@px.tool`` 注册验证."""
def test_gittool_registered(self) -> None:
"""gittool 应在 _TOOL_REGISTRY 中注册."""
assert "gittool" in _TOOL_REGISTRY
def test_visible_subcommands(self) -> None:
"""可见子命令: a, c, i, isub, p, pl."""
subs = px.list_subcommands("gittool")
assert "a" in subs
assert "c" in subs
assert "i" in subs
assert "isub" in subs
assert "p" in subs
assert "pl" in subs
def test_clean_is_hidden(self) -> None:
"""clean 应为 hidden (不在可见子命令中)."""
subs = px.list_subcommands("gittool")
assert "clean" not in subs
def test_clean_hidden_included_with_flag(self) -> None:
"""include_hidden=True 时 clean 应出现."""
subs = px.list_subcommands("gittool", include_hidden=True)
assert "clean" in subs
def test_c_needs_clean(self) -> None:
"""c 应依赖 clean."""
spec = get_tool("gittool", "c")
assert "clean" in spec.needs
def test_clean_has_cmd(self) -> None:
"""clean 应有 cmd."""
spec = get_tool("gittool", "clean")
assert spec.cmd is not None
assert "git" in spec.cmd
assert "clean" in spec.cmd
assert "-xfd" in spec.cmd
def test_clean_cmd_includes_excludes(self) -> None:
"""clean cmd 应包含排除项."""
spec = get_tool("gittool", "clean")
assert spec.cmd is not None
assert "-e" in spec.cmd
assert ".venv" in spec.cmd
def test_c_has_cmd(self) -> None:
"""c 应有 cmd."""
spec = get_tool("gittool", "c")
assert spec.cmd == ("git", "status", "--porcelain")
def test_p_has_cmd(self) -> None:
"""p 应有 cmd."""
spec = get_tool("gittool", "p")
assert spec.cmd == ("git", "push")
def test_pl_has_cmd(self) -> None:
"""pl 应有 cmd."""
spec = get_tool("gittool", "pl")
assert spec.cmd == ("git", "pull")
# ---------------------------------------------------------------------- #
# 辅助函数验证 (not_has_git_repo / has_files)
# ---------------------------------------------------------------------- #
class TestGittoolHelpers:
"""``gittool`` 模块辅助函数验证."""
def test_not_has_git_repo_true(self, tmp_path: Path) -> None:
"""无 .git 目录时 not_has_git_repo 应返回 True."""
with patch.object(Path, "cwd", return_value=tmp_path):
assert gittool.not_has_git_repo() is True
def test_not_has_git_repo_false(self, tmp_path: Path) -> None:
"""有 .git 目录时 not_has_git_repo 应返回 False."""
(tmp_path / ".git").mkdir()
with patch.object(Path, "cwd", return_value=tmp_path):
assert gittool.not_has_git_repo() is False
def test_has_files_false_when_no_changes(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""无未提交更改时 has_files 应返回 False."""
class _FakeResult:
stdout = ""
monkeypatch.setattr("subprocess.run", lambda *_, **__: _FakeResult())
assert gittool.has_files() is False
def test_has_files_true_when_changes(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""有未提交更改时 has_files 应返回 True."""
class _FakeResult:
stdout = " M test.txt\n"
monkeypatch.setattr("subprocess.run", lambda *_, **__: _FakeResult())
assert gittool.has_files() is True
def test_has_files_false_on_subprocess_error(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""subprocess.SubprocessError 时 has_files 应返回 False."""
def _raise(*args: object, **kwargs: object) -> None:
raise subprocess.SubprocessError("fail")
monkeypatch.setattr("subprocess.run", _raise)
assert gittool.has_files() is False
def test_has_files_false_on_oserror(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""OSError 时 has_files 应返回 False."""
def _raise(*args: object, **kwargs: object) -> None:
raise OSError("fail")
monkeypatch.setattr("subprocess.run", _raise)
assert gittool.has_files() is False
# ---------------------------------------------------------------------- #
# git_add_commit / git_init_add_commit 函数体验证
# ---------------------------------------------------------------------- #
class TestGittoolCommit:
"""``gittool`` 模块提交函数验证."""
def test_git_add_commit_no_changes(
self,
monkeypatch: pytest.MonkeyPatch,
capsys: pytest.CaptureFixture[str],
) -> None:
"""无更改时 git_add_commit 应打印提示且不执行 git 命令."""
monkeypatch.setattr(gittool, "has_files", lambda: False)
calls: list[list[str]] = []
def _capture(*args: object, **kwargs: object) -> None:
calls.append(args[0]) # type: ignore[arg-type]
monkeypatch.setattr("subprocess.run", _capture)
gittool.git_add_commit()
assert calls == []
assert "没有文件需要提交" in capsys.readouterr().out
def test_git_add_commit_with_changes(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""有更改时 git_add_commit 应执行 git add + git commit."""
monkeypatch.setattr(gittool, "has_files", lambda: True)
calls: list[list[str]] = []
def _capture(*args: object, **kwargs: object) -> None:
calls.append(args[0]) # type: ignore[arg-type]
monkeypatch.setattr("subprocess.run", _capture)
gittool.git_add_commit(message="custom msg")
assert calls == [["git", "add", "."], ["git", "commit", "-m", "custom msg"]]
def test_git_init_add_commit_init_and_commit(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""无仓库且有更改时 git_init_add_commit 应 git init + add + commit."""
monkeypatch.setattr(gittool, "not_has_git_repo", lambda: True)
monkeypatch.setattr(gittool, "has_files", lambda: True)
calls: list[list[str]] = []
def _capture(*args: object, **kwargs: object) -> None:
calls.append(args[0]) # type: ignore[arg-type]
monkeypatch.setattr("subprocess.run", _capture)
gittool.git_init_add_commit()
assert calls == [["git", "init"], ["git", "add", "."], ["git", "commit", "-m", "init commit"]]
def test_git_init_add_commit_init_no_changes(
self,
monkeypatch: pytest.MonkeyPatch,
capsys: pytest.CaptureFixture[str],
) -> None:
"""无仓库且无更改时 git_init_add_commit 应 git init + 打印提示."""
monkeypatch.setattr(gittool, "not_has_git_repo", lambda: True)
monkeypatch.setattr(gittool, "has_files", lambda: False)
calls: list[list[str]] = []
def _capture(*args: object, **kwargs: object) -> None:
calls.append(args[0]) # type: ignore[arg-type]
monkeypatch.setattr("subprocess.run", _capture)
gittool.git_init_add_commit()
assert calls == [["git", "init"]]
assert "没有文件需要提交" in capsys.readouterr().out
def test_git_init_add_commit_no_init_with_changes(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""有仓库且有更改时 git_init_add_commit 应 add + commit (不 init)."""
monkeypatch.setattr(gittool, "not_has_git_repo", lambda: False)
monkeypatch.setattr(gittool, "has_files", lambda: True)
calls: list[list[str]] = []
def _capture(*args: object, **kwargs: object) -> None:
calls.append(args[0]) # type: ignore[arg-type]
monkeypatch.setattr("subprocess.run", _capture)
gittool.git_init_add_commit()
assert calls == [["git", "add", "."], ["git", "commit", "-m", "init commit"]]
def test_git_init_add_commit_no_init_no_changes(
self,
monkeypatch: pytest.MonkeyPatch,
capsys: pytest.CaptureFixture[str],
) -> None:
"""有仓库且无更改时 git_init_add_commit 应仅打印提示 (不 init, 不 add/commit)."""
monkeypatch.setattr(gittool, "not_has_git_repo", lambda: False)
monkeypatch.setattr(gittool, "has_files", lambda: False)
calls: list[list[str]] = []
def _capture(*args: object, **kwargs: object) -> None:
calls.append(args[0]) # type: ignore[arg-type]
monkeypatch.setattr("subprocess.run", _capture)
gittool.git_init_add_commit()
assert calls == []
assert "没有文件需要提交" in capsys.readouterr().out
# ---------------------------------------------------------------------- #
# init_sub_dirs 函数验证
# ---------------------------------------------------------------------- #
class TestGittoolInitSubDirs:
"""``init_sub_dirs`` 函数验证."""
def test_init_sub_dirs_runs_graph_for_each_subdir(
self,
monkeypatch: pytest.MonkeyPatch,
tmp_path: Path,
) -> None:
"""init_sub_dirs 应为每个子目录构建并运行图 (文件被跳过)."""
(tmp_path / "subdir1").mkdir()
(tmp_path / "subdir2").mkdir()
(tmp_path / "file.txt").write_text("test")
monkeypatch.setattr(Path, "cwd", lambda: tmp_path)
runs: list[object] = []
monkeypatch.setattr(gittool.px, "run", runs.append)
gittool.init_sub_dirs()
assert len(runs) == 2
def test_init_sub_dirs_no_subdirs(
self,
monkeypatch: pytest.MonkeyPatch,
tmp_path: Path,
) -> None:
"""无子目录时 init_sub_dirs 不应运行任何图."""
(tmp_path / "file.txt").write_text("test")
monkeypatch.setattr(Path, "cwd", lambda: tmp_path)
runs: list[object] = []
monkeypatch.setattr(gittool.px, "run", runs.append)
gittool.init_sub_dirs()
assert runs == []
+169
View File
@@ -0,0 +1,169 @@
"""Tests for ops.msdownload / ops.sglang 模块 (ModelScope 下载/SGLang 服务)."""
from __future__ import annotations
import subprocess
from pathlib import Path
from unittest.mock import MagicMock
import pytest
from pyflowx.conditions import Constants
from pyflowx.ops.msdownload import msdownload_run
from pyflowx.ops.sglang import install_sglang, run_sglang
# ---------------------------------------------------------------------- #
# msdownload_run
# ---------------------------------------------------------------------- #
class TestMsdownloadRun:
"""``msdownload_run`` 函数测试."""
def test_empty_name_does_nothing(self, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]) -> None:
"""name 为空时应直接返回."""
called: list[list[str]] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: called.append(cmd))
msdownload_run("")
captured = capsys.readouterr()
assert "name 不能为空" in captured.out
assert called == []
def test_default_download_dir(self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None:
"""未提供 download_dir 时默认使用 ~/.models/<name 最后一段>."""
monkeypatch.setattr(Path, "home", lambda: tmp_path)
ran_cmds: list[list[str]] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: ran_cmds.append(cmd) or MagicMock())
msdownload_run("Qwen/Qwen2.5-Coder")
expected_dir = tmp_path / ".models" / "Qwen2.5-Coder"
assert expected_dir.exists()
assert ran_cmds[0] == [
"uvx",
"modelscope",
"download",
"--model",
"Qwen/Qwen2.5-Coder",
"--local_dir",
str(expected_dir),
]
def test_custom_download_dir(self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None:
"""提供 download_dir 时应使用指定目录."""
custom_dir = tmp_path / "custom"
ran_cmds: list[list[str]] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: ran_cmds.append(cmd) or MagicMock())
msdownload_run("Qwen/Qwen2.5", "dataset", str(custom_dir))
assert custom_dir.exists()
assert ran_cmds[0][3] == "--dataset"
assert str(custom_dir) in ran_cmds[0]
def test_dataset_type(self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None:
"""target_type=dataset 时应传递 --dataset."""
monkeypatch.setattr(Path, "home", lambda: tmp_path)
ran_cmds: list[list[str]] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: ran_cmds.append(cmd) or MagicMock())
msdownload_run("foo/bar", "dataset")
assert "--dataset" in ran_cmds[0]
# ---------------------------------------------------------------------- #
# install_sglang
# ---------------------------------------------------------------------- #
class TestInstallSglang:
"""``install_sglang`` 函数测试."""
def test_already_installed_skips(self, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]) -> None:
"""sglang 已安装时应跳过."""
monkeypatch.setattr("shutil.which", lambda _cmd: "/usr/bin/sglang")
called: list[list[str]] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: called.append(cmd))
install_sglang()
captured = capsys.readouterr()
assert "已安装" in captured.out
assert called == []
def test_not_installed_runs_uv_install(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""sglang 未安装时应执行 uv install sglang[all]."""
monkeypatch.setattr("shutil.which", lambda _cmd: None)
ran_cmds: list[list[str]] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: ran_cmds.append(cmd) or MagicMock())
install_sglang()
assert ran_cmds == [["uv", "install", "sglang[all]"]]
# ---------------------------------------------------------------------- #
# run_sglang
# ---------------------------------------------------------------------- #
class TestRunSglang:
"""``run_sglang`` 函数测试."""
def test_model_dir_not_exist_skips(
self, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
) -> None:
"""模型目录不存在时应跳过."""
monkeypatch.setattr(Path, "exists", lambda _self: False)
called: list[list[str]] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: called.append(cmd))
run_sglang(model="/nonexistent/path")
captured = capsys.readouterr()
assert "模型目录不存在" in captured.out
assert called == []
def test_windows_uses_python(self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None:
"""Windows 上应使用 python."""
monkeypatch.setattr(Constants, "IS_WINDOWS", True)
monkeypatch.setattr(Path, "expanduser", lambda _self: tmp_path)
monkeypatch.setattr(Path, "exists", lambda _self: True)
ran_cmds: list[list[str]] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: ran_cmds.append(cmd) or MagicMock())
run_sglang(model=str(tmp_path))
assert ran_cmds[0][0] == "python"
assert "-m" in ran_cmds[0]
assert "sglang.launch_server" in ran_cmds[0]
def test_non_windows_uses_python3(self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None:
"""非 Windows 上应使用 python3."""
monkeypatch.setattr(Constants, "IS_WINDOWS", False)
monkeypatch.setattr(Path, "expanduser", lambda _self: tmp_path)
monkeypatch.setattr(Path, "exists", lambda _self: True)
ran_cmds: list[list[str]] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: ran_cmds.append(cmd) or MagicMock())
run_sglang(model=str(tmp_path), port=9000, ctx_len=4096, mem_fraction=0.5, host="127.0.0.1", log_level="debug")
cmd = ran_cmds[0]
assert cmd[0] == "python3"
assert "--port" in cmd
assert "9000" in cmd
assert "4096" in cmd
assert "0.5" in cmd
assert "127.0.0.1" in cmd
assert "debug" in cmd
def test_command_includes_qwen_parser(self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None:
"""命令应包含 --tool-call-parser qwen."""
monkeypatch.setattr(Constants, "IS_WINDOWS", True)
monkeypatch.setattr(Path, "expanduser", lambda _self: tmp_path)
monkeypatch.setattr(Path, "exists", lambda _self: True)
ran_cmds: list[list[str]] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: ran_cmds.append(cmd) or MagicMock())
run_sglang(model=str(tmp_path))
cmd = ran_cmds[0]
assert "--tool-call-parser" in cmd
qwen_idx = cmd.index("--tool-call-parser")
assert cmd[qwen_idx + 1] == "qwen"
+10 -57
View File
@@ -5,9 +5,8 @@ from __future__ import annotations
from pathlib import Path
from unittest.mock import MagicMock, patch
import pyflowx as px
from pyflowx.cli import lscalc
from pyflowx.conditions import Constants
from pyflowx.ops import lscalc as system
# ---------------------------------------------------------------------- #
@@ -19,7 +18,7 @@ class TestGetLsDynaCommand:
def test_get_ls_dyna_command_windows(self) -> None:
"""Should get LS-DYNA command for Windows."""
with patch.object(Constants, "IS_WINDOWS", True), patch.object(Constants, "IS_MACOS", False):
cmd = lscalc.get_ls_dyna_command("input.k", 4)
cmd = system.get_ls_dyna_command("input.k", 4)
assert "ls-dyna_mpp" in cmd
assert "i=input.k" in cmd
assert "ncpu=4" in cmd
@@ -27,7 +26,7 @@ class TestGetLsDynaCommand:
def test_get_ls_dyna_command_linux(self) -> None:
"""Should get LS-DYNA command for Linux."""
with patch.object(Constants, "IS_WINDOWS", False), patch.object(Constants, "IS_MACOS", False):
cmd = lscalc.get_ls_dyna_command("input.k", 8)
cmd = system.get_ls_dyna_command("input.k", 8)
assert "ls-dyna_mpp" in cmd
assert "i=input.k" in cmd
assert "ncpu=8" in cmd
@@ -46,14 +45,14 @@ class TestRunLsDyna:
with patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(returncode=0)
lscalc.run_ls_dyna(str(input_file), ncpu=4)
system.run_ls_dyna(str(input_file), ncpu=4)
assert mock_run.called
def test_run_ls_dyna_file_not_found(self, tmp_path: Path) -> None:
"""Should handle nonexistent input file."""
input_file = tmp_path / "nonexistent.k"
lscalc.run_ls_dyna(str(input_file), ncpu=4)
system.run_ls_dyna(str(input_file), ncpu=4)
# Should print error message
def test_run_ls_dyna_command_not_found(self, tmp_path: Path) -> None:
@@ -62,7 +61,7 @@ class TestRunLsDyna:
input_file.write_text("LS-DYNA input")
with patch("subprocess.run", side_effect=FileNotFoundError):
lscalc.run_ls_dyna(str(input_file), ncpu=4)
system.run_ls_dyna(str(input_file), ncpu=4)
# Should print error message
@@ -79,14 +78,14 @@ class TestRunLsDynaMpi:
with patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(returncode=0)
lscalc.run_ls_dyna_mpi(str(input_file), ncpu=8)
system.run_ls_dyna_mpi(str(input_file), ncpu=8)
assert mock_run.called
def test_run_ls_dyna_mpi_file_not_found(self, tmp_path: Path) -> None:
"""Should handle nonexistent input file."""
input_file = tmp_path / "nonexistent.k"
lscalc.run_ls_dyna_mpi(str(input_file), ncpu=8)
system.run_ls_dyna_mpi(str(input_file), ncpu=8)
# Should print error message
@@ -100,58 +99,12 @@ class TestCheckLsDynaStatus:
"""Should check LS-DYNA status on Windows."""
with patch.object(Constants, "IS_WINDOWS", True), patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(stdout="ls-dyna_mpp.exe", returncode=0)
lscalc.check_ls_dyna_status()
system.check_ls_dyna_status()
assert mock_run.called
def test_check_ls_dyna_status_linux(self) -> None:
"""Should check LS-DYNA status on Linux."""
with patch.object(Constants, "IS_WINDOWS", False), patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(stdout="1234", returncode=0)
lscalc.check_ls_dyna_status()
system.check_ls_dyna_status()
assert mock_run.called
# ---------------------------------------------------------------------- #
# main function
# ---------------------------------------------------------------------- #
class TestMain:
"""Test main function."""
def test_main_run_command(self, tmp_path: Path) -> None:
"""main() should handle run command."""
input_file = tmp_path / "input.k"
input_file.write_text("LS-DYNA input")
with patch("sys.argv", ["lscalc", "run", str(input_file)]), patch.object(px, "run") as mock_run:
lscalc.main()
assert mock_run.called
def test_main_run_command_with_ncpu(self, tmp_path: Path) -> None:
"""main() should handle run command with ncpu."""
input_file = tmp_path / "input.k"
input_file.write_text("LS-DYNA input")
with patch("sys.argv", ["lscalc", "run", str(input_file), "--ncpu", "8"]), patch.object(px, "run") as mock_run:
lscalc.main()
assert mock_run.called
def test_main_mpi_command(self, tmp_path: Path) -> None:
"""main() should handle mpi command."""
input_file = tmp_path / "input.k"
input_file.write_text("LS-DYNA input")
with patch("sys.argv", ["lscalc", "mpi", str(input_file)]), patch.object(px, "run") as mock_run:
lscalc.main()
assert mock_run.called
def test_main_status_command(self) -> None:
"""main() should handle status command."""
with patch("sys.argv", ["lscalc", "status"]), patch.object(px, "run") as mock_run:
lscalc.main()
assert mock_run.called
def test_main_with_no_args_shows_help(self) -> None:
"""main() with no args should show help."""
with patch("sys.argv", ["lscalc"]):
lscalc.main()
# Should print help and return
+16 -81
View File
@@ -7,8 +7,7 @@ from unittest.mock import MagicMock, patch
import pytest
import pyflowx as px
from pyflowx.cli import packtool
from pyflowx.ops import packtool as system
@pytest.fixture(autouse=True)
@@ -20,7 +19,7 @@ def packtool_tmp_workdir(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Non
monkeypatch: pytest monkeypatch 工具
"""
# Mock DEFAULT_CACHE_DIR 到临时目录
monkeypatch.setattr(packtool, "DEFAULT_CACHE_DIR", str(tmp_path / ".cache" / "pypack"))
monkeypatch.setattr(system, "DEFAULT_CACHE_DIR", str(tmp_path / ".cache" / "pypack"))
# ---------------------------------------------------------------------- #
@@ -36,7 +35,7 @@ class TestPackSource:
(project_dir / "main.py").write_text("print('hello')")
output_dir = tmp_path / "output"
packtool.pack_source(project_dir, output_dir)
system.pack_source(project_dir, output_dir)
assert output_dir.exists()
def test_pack_source_with_pyproject(self, tmp_path: Path) -> None:
@@ -47,7 +46,7 @@ class TestPackSource:
(project_dir / "main.py").write_text("print('hello')")
output_dir = tmp_path / "output"
packtool.pack_source(project_dir, output_dir)
system.pack_source(project_dir, output_dir)
assert output_dir.exists()
@@ -61,7 +60,7 @@ class TestPackDependencies:
"""Should handle empty dependencies."""
lib_dir = tmp_path / "libs"
packtool.pack_dependencies(lib_dir, [])
system.pack_dependencies(lib_dir, [])
# Should print message and return
def test_pack_dependencies_with_deps(self, tmp_path: Path) -> None:
@@ -70,7 +69,7 @@ class TestPackDependencies:
with patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(returncode=0)
packtool.pack_dependencies(lib_dir, ["numpy", "pandas"])
system.pack_dependencies(lib_dir, ["numpy", "pandas"])
assert mock_run.called
@@ -89,7 +88,7 @@ class TestPackWheel:
with patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(returncode=0)
packtool.pack_wheel(project_dir, output_dir)
system.pack_wheel(project_dir, output_dir)
assert mock_run.called
@@ -112,7 +111,7 @@ class TestInstallEmbedPython:
mock_zip_instance = MagicMock()
mock_zipfile.return_value.__enter__.return_value = mock_zip_instance
packtool.install_embed_python("3.10", output_dir)
system.install_embed_python("3.10", output_dir)
# Verify download was called
assert mock_urlretrieve.called
@@ -135,7 +134,7 @@ class TestInstallEmbedPython:
mock_zip_instance = MagicMock()
mock_zipfile.return_value.__enter__.return_value = mock_zip_instance
packtool.install_embed_python("3.10", output_dir)
system.install_embed_python("3.10", output_dir)
# Verify extraction was called (using cache)
assert mock_zip_instance.extractall.called
@@ -158,7 +157,7 @@ class TestInstallEmbedPython:
return
# Perform real installation
packtool.install_embed_python("3.10", output_dir)
system.install_embed_python("3.10", output_dir)
# Verify installation succeeded
assert output_dir.exists()
@@ -199,7 +198,7 @@ class TestInstallEmbedPython:
# Test different versions
for version in ["3.8", "3.9", "3.10", "3.11", "3.12"]:
packtool.install_embed_python(version, output_dir)
system.install_embed_python(version, output_dir)
assert mock_urlretrieve.called
def test_install_embed_python_creates_cache(self, tmp_path: Path) -> None:
@@ -213,10 +212,10 @@ class TestInstallEmbedPython:
mock_zip_instance = MagicMock()
mock_zipfile.return_value.__enter__.return_value = mock_zip_instance
packtool.install_embed_python("3.10", output_dir)
system.install_embed_python("3.10", output_dir)
# Verify cache directory was created (now in tmp_path)
Path(packtool.DEFAULT_CACHE_DIR)
Path(system.DEFAULT_CACHE_DIR)
# Note: In test environment, cache might not persist due to mocking
@@ -233,7 +232,7 @@ class TestCreateZipPackage:
(source_dir / "test.txt").write_text("test content")
output_file = tmp_path / "package.zip"
packtool.create_zip_package(source_dir, output_file)
system.create_zip_package(source_dir, output_file)
assert output_file.exists()
@@ -249,76 +248,12 @@ class TestCleanBuildDir:
build_dir.mkdir()
(build_dir / "test.txt").write_text("test")
packtool.clean_build_dir(build_dir)
system.clean_build_dir(build_dir)
assert not build_dir.exists()
def test_clean_build_dir_not_exists(self, tmp_path: Path) -> None:
"""Should handle nonexistent build directory."""
build_dir = tmp_path / "nonexistent"
packtool.clean_build_dir(build_dir)
system.clean_build_dir(build_dir)
# Should print message
# ---------------------------------------------------------------------- #
# main function
# ---------------------------------------------------------------------- #
class TestMain:
"""Test main function."""
def test_main_src_command(self, tmp_path: Path) -> None:
"""main() should handle src command."""
project_dir = tmp_path / "project"
project_dir.mkdir()
with patch("sys.argv", ["packtool", "src", "--project-dir", str(project_dir)]), patch.object(
px, "run"
) as mock_run:
packtool.main()
assert mock_run.called
def test_main_deps_command(self, tmp_path: Path) -> None:
"""main() should handle deps command."""
with patch("sys.argv", ["packtool", "deps", "numpy", "pandas"]), patch.object(px, "run") as mock_run:
packtool.main()
assert mock_run.called
def test_main_wheel_command(self, tmp_path: Path) -> None:
"""main() should handle wheel command."""
project_dir = tmp_path / "project"
project_dir.mkdir()
with patch("sys.argv", ["packtool", "wheel", "--project-dir", str(project_dir)]), patch.object(
px, "run"
) as mock_run:
packtool.main()
assert mock_run.called
def test_main_embed_command(self, tmp_path: Path) -> None:
"""main() should handle embed command."""
with patch("sys.argv", ["packtool", "embed", "--version", "3.10"]), patch.object(px, "run") as mock_run:
packtool.main()
assert mock_run.called
def test_main_zip_command(self, tmp_path: Path) -> None:
"""main() should handle zip command."""
source_dir = tmp_path / "source"
source_dir.mkdir()
with patch("sys.argv", ["packtool", "zip", "--source-dir", str(source_dir)]), patch.object(
px, "run"
) as mock_run:
packtool.main()
assert mock_run.called
def test_main_clean_command(self) -> None:
"""main() should handle clean command."""
with patch("sys.argv", ["packtool", "clean"]), patch.object(px, "run") as mock_run:
packtool.main()
assert mock_run.called
def test_main_with_no_args_shows_help(self) -> None:
"""main() with no args should show help."""
with patch("sys.argv", ["packtool"]):
packtool.main()
# Should print help and return
+207 -44
View File
@@ -8,8 +8,7 @@ from unittest.mock import MagicMock, patch
import pytest
import pyflowx as px
from pyflowx.cli import pdftool
from pyflowx.ops import pdftool as media
# ---------------------------------------------------------------------- #
@@ -29,7 +28,7 @@ class TestPdfMerge:
with patch("pypdf.PdfReader"), patch("pypdf.PdfWriter") as mock_writer:
mock_writer_instance = MagicMock()
mock_writer.return_value = mock_writer_instance
pdftool.pdf_merge(input_files, output_file)
media.pdf_merge(input_files, output_file)
assert mock_writer_instance.write.called
@@ -50,7 +49,7 @@ class TestPdfSplit:
mock_reader_instance = MagicMock()
mock_reader.return_value = mock_reader_instance
mock_reader_instance.pages = [MagicMock()]
pdftool.pdf_split(input_file, output_dir)
media.pdf_split(input_file, output_dir)
assert output_dir.exists()
@@ -76,7 +75,7 @@ class TestPdfCompress:
output_file.write_bytes(b"Compressed PDF")
mock_doc.save = mock_save
pdftool.pdf_compress(input_file, output_file)
media.pdf_compress(input_file, output_file)
assert output_file.exists()
@@ -99,7 +98,7 @@ class TestPdfExtractText:
mock_page.get_text.return_value = "Test text"
mock_doc.__iter__ = MagicMock(return_value=iter([mock_page]))
mock_fitz_open.return_value = mock_doc
pdftool.pdf_extract_text(input_file, output_file)
media.pdf_extract_text(input_file, output_file)
assert output_file.exists()
@@ -123,7 +122,7 @@ class TestPdfExtractImages:
mock_doc.__iter__ = MagicMock(return_value=iter([mock_page]))
mock_doc.extract_image.return_value = {"image": b"image data", "ext": "png"}
mock_fitz_open.return_value = mock_doc
pdftool.pdf_extract_images(input_file, output_dir)
media.pdf_extract_images(input_file, output_dir)
assert output_dir.exists()
@@ -147,7 +146,7 @@ class TestPdfAddWatermark:
mock_doc.__iter__ = MagicMock(return_value=iter([mock_page]))
mock_fitz_open.return_value = mock_doc
mock_text_length.return_value = 100
pdftool.pdf_add_watermark(input_file, output_file)
media.pdf_add_watermark(input_file, output_file)
assert mock_doc.save.called
@@ -169,7 +168,7 @@ class TestPdfRotate:
mock_page = MagicMock()
mock_doc.__iter__ = MagicMock(return_value=iter([mock_page]))
mock_fitz_open.return_value = mock_doc
pdftool.pdf_rotate(input_file, output_file, rotation=90)
media.pdf_rotate(input_file, output_file, rotation=90)
assert mock_doc.save.called
def test_pdf_rotate_file_180(self, tmp_path: Path) -> None:
@@ -184,7 +183,7 @@ class TestPdfRotate:
mock_page = MagicMock()
mock_doc.__iter__ = MagicMock(return_value=iter([mock_page]))
mock_fitz_open.return_value = mock_doc
pdftool.pdf_rotate(input_file, output_file, rotation=180)
media.pdf_rotate(input_file, output_file, rotation=180)
assert mock_doc.save.called
@@ -207,7 +206,7 @@ class TestPdfCrop:
mock_page.rect = MagicMock(x0=0, y0=0, x1=800, y1=600)
mock_doc.__iter__ = MagicMock(return_value=iter([mock_page]))
mock_fitz_open.return_value = mock_doc
pdftool.pdf_crop(input_file, output_file, margins=(10, 10, 10, 10))
media.pdf_crop(input_file, output_file, margins=(10, 10, 10, 10))
assert mock_doc.save.called
@@ -228,7 +227,7 @@ class TestPdfInfo:
mock_doc.page_count = 10
mock_doc.metadata = {"title": "Test", "author": "Author"}
mock_fitz_open.return_value = mock_doc
pdftool.pdf_info(input_file)
media.pdf_info(input_file)
assert mock_fitz_open.called
@@ -257,7 +256,7 @@ class TestPdfOcr:
mock_doc.__iter__ = MagicMock(return_value=iter([mock_page]))
mock_fitz_open.return_value = mock_doc
mock_ocr.return_value = "OCR text"
pdftool.pdf_ocr(input_file, output_file)
media.pdf_ocr(input_file, output_file)
# Should complete OCR
@@ -277,48 +276,212 @@ class TestPdfRepair:
with patch("fitz.open") as mock_fitz_open:
mock_doc = MagicMock()
mock_fitz_open.return_value = mock_doc
pdftool.pdf_repair(input_file, output_file)
media.pdf_repair(input_file, output_file)
assert mock_doc.save.called
# ---------------------------------------------------------------------- #
# main function
# pdf_encrypt
# ---------------------------------------------------------------------- #
class TestMain:
"""Test main function."""
class TestPdfEncrypt:
"""Test pdf_encrypt function."""
def test_main_merge_command(self, tmp_path: Path) -> None:
"""main() should handle merge command."""
input_files = [tmp_path / "input1.pdf", tmp_path / "input2.pdf"]
for f in input_files:
f.write_bytes(b"PDF content")
with patch("sys.argv", ["pdftool", "m", str(input_files[0]), str(input_files[1])]), patch.object(
px, "run"
) as mock_run:
pdftool.main()
assert mock_run.called
def test_main_split_command(self, tmp_path: Path) -> None:
"""main() should handle split command."""
def test_pdf_encrypt_file(self, tmp_path: Path) -> None:
"""Should encrypt PDF."""
pytest.importorskip("pypdf")
input_file = tmp_path / "input.pdf"
input_file.write_bytes(b"PDF content")
output_file = tmp_path / "encrypted.pdf"
with patch("sys.argv", ["pdftool", "s", str(input_file)]), patch.object(px, "run") as mock_run:
pdftool.main()
assert mock_run.called
with patch("pypdf.PdfReader") as mock_reader, patch("pypdf.PdfWriter") as mock_writer:
mock_reader_instance = MagicMock()
mock_reader.return_value = mock_reader_instance
mock_reader_instance.pages = [MagicMock()]
mock_writer_instance = MagicMock()
mock_writer.return_value = mock_writer_instance
media.pdf_encrypt(input_file, output_file, "secret")
assert mock_writer_instance.encrypt.called
assert mock_writer_instance.write.called
def test_main_compress_command(self, tmp_path: Path) -> None:
"""main() should handle compress command."""
def test_pdf_encrypt_not_installed(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""Should print message when pypdf not installed."""
monkeypatch.setattr(media, "HAS_PYPDF", False)
media.pdf_encrypt(tmp_path / "in.pdf", tmp_path / "out.pdf", "pw")
# ---------------------------------------------------------------------- #
# pdf_decrypt
# ---------------------------------------------------------------------- #
class TestPdfDecrypt:
"""Test pdf_decrypt function."""
def test_pdf_decrypt_encrypted(self, tmp_path: Path) -> None:
"""Should decrypt encrypted PDF."""
pytest.importorskip("pypdf")
input_file = tmp_path / "input.pdf"
input_file.write_bytes(b"PDF content")
output_file = tmp_path / "decrypted.pdf"
with patch("sys.argv", ["pdftool", "c", str(input_file)]), patch.object(px, "run") as mock_run:
pdftool.main()
assert mock_run.called
with patch("pypdf.PdfReader") as mock_reader, patch("pypdf.PdfWriter") as mock_writer:
mock_reader_instance = MagicMock()
mock_reader.return_value = mock_reader_instance
mock_reader_instance.is_encrypted = True
mock_reader_instance.pages = [MagicMock()]
mock_writer_instance = MagicMock()
mock_writer.return_value = mock_writer_instance
media.pdf_decrypt(input_file, output_file, "secret")
assert mock_reader_instance.decrypt.called
assert mock_writer_instance.write.called
def test_main_with_no_args_shows_help(self) -> None:
"""main() with no args should show help."""
with patch("sys.argv", ["pdftool"]):
pdftool.main()
# Should print help and return
def test_pdf_decrypt_not_encrypted(self, tmp_path: Path) -> None:
"""Should handle non-encrypted PDF."""
pytest.importorskip("pypdf")
input_file = tmp_path / "input.pdf"
input_file.write_bytes(b"PDF content")
output_file = tmp_path / "decrypted.pdf"
with patch("pypdf.PdfReader") as mock_reader, patch("pypdf.PdfWriter") as mock_writer:
mock_reader_instance = MagicMock()
mock_reader.return_value = mock_reader_instance
mock_reader_instance.is_encrypted = False
mock_reader_instance.pages = [MagicMock()]
mock_writer_instance = MagicMock()
mock_writer.return_value = mock_writer_instance
media.pdf_decrypt(input_file, output_file, "secret")
assert not mock_reader_instance.decrypt.called
def test_pdf_decrypt_not_installed(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""Should print message when pypdf not installed."""
monkeypatch.setattr(media, "HAS_PYPDF", False)
media.pdf_decrypt(tmp_path / "in.pdf", tmp_path / "out.pdf", "pw")
# ---------------------------------------------------------------------- #
# pdf_reorder
# ---------------------------------------------------------------------- #
class TestPdfReorder:
"""Test pdf_reorder function."""
def test_pdf_reorder_pages(self, tmp_path: Path) -> None:
"""Should reorder PDF pages."""
pytest.importorskip("pypdf")
input_file = tmp_path / "input.pdf"
input_file.write_bytes(b"PDF content")
output_file = tmp_path / "reordered.pdf"
with patch("pypdf.PdfReader") as mock_reader, patch("pypdf.PdfWriter") as mock_writer:
mock_reader_instance = MagicMock()
mock_reader.return_value = mock_reader_instance
mock_reader_instance.pages = [MagicMock(), MagicMock(), MagicMock()]
mock_writer_instance = MagicMock()
mock_writer.return_value = mock_writer_instance
media.pdf_reorder(input_file, output_file, [2, 0, 1])
assert mock_writer_instance.add_page.call_count == 3
def test_pdf_reorder_out_of_range(self, tmp_path: Path) -> None:
"""Should skip out-of-range page numbers."""
pytest.importorskip("pypdf")
input_file = tmp_path / "input.pdf"
input_file.write_bytes(b"PDF content")
output_file = tmp_path / "reordered.pdf"
with patch("pypdf.PdfReader") as mock_reader, patch("pypdf.PdfWriter") as mock_writer:
mock_reader_instance = MagicMock()
mock_reader.return_value = mock_reader_instance
mock_reader_instance.pages = [MagicMock()]
mock_writer_instance = MagicMock()
mock_writer.return_value = mock_writer_instance
media.pdf_reorder(input_file, output_file, [0, 5, -1])
assert mock_writer_instance.add_page.call_count == 1
def test_pdf_reorder_not_installed(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""Should print message when pypdf not installed."""
monkeypatch.setattr(media, "HAS_PYPDF", False)
media.pdf_reorder(tmp_path / "in.pdf", tmp_path / "out.pdf", [0])
# ---------------------------------------------------------------------- #
# pdf_to_images
# ---------------------------------------------------------------------- #
class TestPdfToImages:
"""Test pdf_to_images function."""
def test_pdf_to_images_convert(self, tmp_path: Path) -> None:
"""Should convert PDF to images."""
pytest.importorskip("fitz")
input_file = tmp_path / "input.pdf"
input_file.write_bytes(b"PDF content")
output_dir = tmp_path / "images"
with patch("fitz.open") as mock_fitz_open:
mock_doc = MagicMock()
mock_page = MagicMock()
mock_pixmap = MagicMock()
mock_page.get_pixmap.return_value = mock_pixmap
mock_doc.__iter__ = MagicMock(return_value=iter([mock_page]))
mock_fitz_open.return_value = mock_doc
media.pdf_to_images(input_file, output_dir, dpi=150)
assert output_dir.exists()
assert mock_pixmap.save.called
def test_pdf_to_images_not_installed(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""Should print message when PyMuPDF not installed."""
monkeypatch.setattr(media, "HAS_PYMUPDF", False)
media.pdf_to_images(tmp_path / "in.pdf", tmp_path / "out")
# ---------------------------------------------------------------------- #
# Not-installed branches for already-tested functions
# ---------------------------------------------------------------------- #
class TestNotInstalledBranches:
"""Test 'not installed' branches for PDF functions."""
def test_pdf_merge_not_installed(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""Should print message when pypdf not installed."""
monkeypatch.setattr(media, "HAS_PYPDF", False)
media.pdf_merge([tmp_path / "a.pdf"], tmp_path / "out.pdf")
def test_pdf_split_not_installed(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""Should print message when pypdf not installed."""
monkeypatch.setattr(media, "HAS_PYPDF", False)
media.pdf_split(tmp_path / "in.pdf", tmp_path / "out")
def test_pdf_compress_not_installed(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""Should print message when PyMuPDF not installed."""
monkeypatch.setattr(media, "HAS_PYMUPDF", False)
media.pdf_compress(tmp_path / "in.pdf", tmp_path / "out.pdf")
def test_pdf_extract_text_not_installed(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""Should print message when PyMuPDF not installed."""
monkeypatch.setattr(media, "HAS_PYMUPDF", False)
media.pdf_extract_text(tmp_path / "in.pdf", tmp_path / "out.txt")
def test_pdf_extract_images_not_installed(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""Should print message when PyMuPDF not installed."""
monkeypatch.setattr(media, "HAS_PYMUPDF", False)
media.pdf_extract_images(tmp_path / "in.pdf", tmp_path / "out")
def test_pdf_add_watermark_not_installed(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""Should print message when PyMuPDF not installed."""
monkeypatch.setattr(media, "HAS_PYMUPDF", False)
media.pdf_add_watermark(tmp_path / "in.pdf", tmp_path / "out.pdf")
def test_pdf_rotate_not_installed(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""Should print message when PyMuPDF not installed."""
monkeypatch.setattr(media, "HAS_PYMUPDF", False)
media.pdf_rotate(tmp_path / "in.pdf", tmp_path / "out.pdf")
def test_pdf_crop_not_installed(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""Should print message when PyMuPDF not installed."""
monkeypatch.setattr(media, "HAS_PYMUPDF", False)
media.pdf_crop(tmp_path / "in.pdf", tmp_path / "out.pdf", (1, 1, 1, 1))
def test_pdf_info_not_installed(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""Should print message when PyMuPDF not installed."""
monkeypatch.setattr(media, "HAS_PYMUPDF", False)
media.pdf_info(tmp_path / "in.pdf")
def test_pdf_repair_not_installed(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""Should print message when PyMuPDF not installed."""
monkeypatch.setattr(media, "HAS_PYMUPDF", False)
media.pdf_repair(tmp_path / "in.pdf", tmp_path / "out.pdf")
+28 -78
View File
@@ -6,8 +6,7 @@ import subprocess
from pathlib import Path
from unittest.mock import MagicMock, patch
import pyflowx as px
from pyflowx.cli import piptool
from pyflowx.ops import piptool as dev
# ---------------------------------------------------------------------- #
@@ -20,7 +19,7 @@ class TestGetInstalledPackages:
"""Should get installed packages."""
with patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(stdout="numpy==1.0.0\npandas==2.0.0\n", returncode=0)
result = piptool._get_installed_packages()
result = dev._get_installed_packages()
assert "numpy" in result
assert "pandas" in result
@@ -28,19 +27,19 @@ class TestGetInstalledPackages:
"""Should handle empty output."""
with patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(stdout="", returncode=0)
result = piptool._get_installed_packages()
result = dev._get_installed_packages()
assert result == []
def test_get_installed_packages_error(self) -> None:
"""Should handle subprocess error."""
with patch("subprocess.run", side_effect=subprocess.SubprocessError):
result = piptool._get_installed_packages()
result = dev._get_installed_packages()
assert result == []
def test_get_installed_packages_oserror(self) -> None:
"""Should handle OSError."""
with patch("subprocess.run", side_effect=OSError):
result = piptool._get_installed_packages()
result = dev._get_installed_packages()
assert result == []
@@ -52,26 +51,26 @@ class TestExpandWildcardPackages:
def test_expand_wildcard_no_pattern(self) -> None:
"""Should return package name when no wildcard."""
result = piptool._expand_wildcard_packages("numpy")
result = dev._expand_wildcard_packages("numpy")
assert result == ["numpy"]
def test_expand_wildcard_with_star(self) -> None:
"""Should expand wildcard with star."""
with patch.object(piptool, "_get_installed_packages", return_value=["numpy", "numpy-core", "pandas"]):
result = piptool._expand_wildcard_packages("numpy*")
with patch.object(dev, "_get_installed_packages", return_value=["numpy", "numpy-core", "pandas"]):
result = dev._expand_wildcard_packages("numpy*")
assert "numpy" in result
assert "numpy-core" in result
def test_expand_wildcard_with_question(self) -> None:
"""Should expand wildcard with question mark."""
with patch.object(piptool, "_get_installed_packages", return_value=["numpy", "numba"]):
result = piptool._expand_wildcard_packages("num??")
with patch.object(dev, "_get_installed_packages", return_value=["numpy", "numba"]):
result = dev._expand_wildcard_packages("num??")
assert len(result) > 0
def test_expand_wildcard_no_match(self) -> None:
"""Should return empty list when no match."""
with patch.object(piptool, "_get_installed_packages", return_value=["pandas", "scipy"]):
result = piptool._expand_wildcard_packages("numpy*")
with patch.object(dev, "_get_installed_packages", return_value=["pandas", "scipy"]):
result = dev._expand_wildcard_packages("numpy*")
assert result == []
@@ -83,19 +82,19 @@ class TestFilterProtectedPackages:
def test_filter_protected_packages_normal(self) -> None:
"""Should filter protected packages."""
result = piptool._filter_protected_packages(["numpy", "pandas", "pyflowx"])
result = dev._filter_protected_packages(["numpy", "pandas", "pyflowx"])
assert "numpy" in result
assert "pandas" in result
assert "pyflowx" not in result
def test_filter_protected_packages_all_protected(self) -> None:
"""Should filter all protected packages."""
result = piptool._filter_protected_packages(["pyflowx", "bitool"])
result = dev._filter_protected_packages(["pyflowx", "bitool"])
assert result == []
def test_filter_protected_packages_case_insensitive(self) -> None:
"""Should filter case insensitive."""
result = piptool._filter_protected_packages(["PyFlowX", "BITOOL"])
result = dev._filter_protected_packages(["PyFlowX", "BITOOL"])
assert result == []
@@ -109,35 +108,35 @@ class TestPipUninstall:
"""Should uninstall single package."""
with patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(returncode=0)
piptool.pip_uninstall(["numpy"])
dev.pip_uninstall(["numpy"])
assert mock_run.called
def test_pip_uninstall_multiple_packages(self) -> None:
"""Should uninstall multiple packages."""
with patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(returncode=0)
piptool.pip_uninstall(["numpy", "pandas", "scipy"])
dev.pip_uninstall(["numpy", "pandas", "scipy"])
# Should call pip uninstall
assert mock_run.called
def test_pip_uninstall_with_wildcard(self) -> None:
"""Should handle wildcard in package name."""
with patch.object(piptool, "_expand_wildcard_packages", return_value=["numpy", "numpy-core"]), patch(
with patch.object(dev, "_expand_wildcard_packages", return_value=["numpy", "numpy-core"]), patch(
"subprocess.run"
) as mock_run:
mock_run.return_value = MagicMock(returncode=0)
piptool.pip_uninstall(["numpy*"])
dev.pip_uninstall(["numpy*"])
assert mock_run.called
def test_pip_uninstall_empty_packages(self) -> None:
"""Should handle empty packages list."""
with patch.object(piptool, "_expand_wildcard_packages", return_value=[]):
piptool.pip_uninstall(["nonexistent*"])
with patch.object(dev, "_expand_wildcard_packages", return_value=[]):
dev.pip_uninstall(["nonexistent*"])
# Should not call subprocess.run
def test_pip_uninstall_all_protected(self) -> None:
"""Should handle all protected packages."""
piptool.pip_uninstall(["pyflowx"])
dev.pip_uninstall(["pyflowx"])
# Should not call subprocess.run
@@ -151,7 +150,7 @@ class TestPipReinstall:
"""Should reinstall single package."""
with patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(returncode=0)
piptool.pip_reinstall(["numpy"])
dev.pip_reinstall(["numpy"])
# Should call pip uninstall and pip install
assert mock_run.call_count == 2
@@ -159,13 +158,13 @@ class TestPipReinstall:
"""Should reinstall packages offline."""
with patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(returncode=0)
piptool.pip_reinstall(["numpy"], offline=True)
dev.pip_reinstall(["numpy"], offline=True)
# Should call pip install with offline flags
assert mock_run.called
def test_pip_reinstall_all_protected(self) -> None:
"""Should handle all protected packages."""
piptool.pip_reinstall(["pyflowx"])
dev.pip_reinstall(["pyflowx"])
# Should not call subprocess.run
@@ -179,14 +178,14 @@ class TestPipDownload:
"""Should download single package."""
with patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(returncode=0)
piptool.pip_download(["numpy"])
dev.pip_download(["numpy"])
assert mock_run.called
def test_pip_download_offline(self) -> None:
"""Should download packages offline."""
with patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(returncode=0)
piptool.pip_download(["numpy"], offline=True)
dev.pip_download(["numpy"], offline=True)
# Should call pip download with offline flags
assert mock_run.called
@@ -201,54 +200,5 @@ class TestPipFreeze:
"""Should freeze dependencies."""
with patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(stdout="numpy==1.0.0\npandas==2.0.0", returncode=0)
piptool.pip_freeze()
dev.pip_freeze()
assert mock_run.called
# ---------------------------------------------------------------------- #
# main function
# ---------------------------------------------------------------------- #
class TestMain:
"""Test main function."""
def test_main_install_command(self) -> None:
"""main() should handle install command."""
with patch("sys.argv", ["piptool", "i", "numpy", "pandas"]), patch.object(px, "run") as mock_run:
piptool.main()
assert mock_run.called
def test_main_uninstall_command(self) -> None:
"""main() should handle uninstall command."""
with patch("sys.argv", ["piptool", "u", "numpy"]), patch.object(px, "run") as mock_run:
piptool.main()
assert mock_run.called
def test_main_reinstall_command(self) -> None:
"""main() should handle reinstall command."""
with patch("sys.argv", ["piptool", "r", "numpy"]), patch.object(px, "run") as mock_run:
piptool.main()
assert mock_run.called
def test_main_download_command(self) -> None:
"""main() should handle download command."""
with patch("sys.argv", ["piptool", "d", "numpy"]), patch.object(px, "run") as mock_run:
piptool.main()
assert mock_run.called
def test_main_upgrade_command(self) -> None:
"""main() should handle upgrade command."""
with patch("sys.argv", ["piptool", "up"]), patch.object(px, "run") as mock_run:
piptool.main()
assert mock_run.called
def test_main_freeze_command(self) -> None:
"""main() should handle freeze command."""
with patch("sys.argv", ["piptool", "f"]), patch.object(px, "run") as mock_run:
piptool.main()
assert mock_run.called
def test_main_with_no_args_shows_help(self) -> None:
"""main() with no args should show help."""
with patch("sys.argv", ["piptool"]):
piptool.main()
# Should print help and return
+6 -4
View File
@@ -56,10 +56,12 @@ def _build_simple_profile() -> ProfileReport:
report = px.RunReport()
report.results["a"] = _result("a", start, 1.0)
report.results["b"] = _result("b", start + timedelta(seconds=1), 2.0)
graph = px.Graph.from_specs([
_spec("a"),
_spec("b", deps=("a",)),
])
graph = px.Graph.from_specs(
[
_spec("a"),
_spec("b", deps=("a",)),
]
)
return ProfileReport.from_report(report, graph)
-175
View File
@@ -1,175 +0,0 @@
"""Tests for cli.pymake module."""
from __future__ import annotations
from pathlib import Path
from unittest.mock import patch
import pytest
from pyflowx.cli import pymake
# ---------------------------------------------------------------------- #
# TaskSpec definitions
# ---------------------------------------------------------------------- #
def _find_task(name: str) -> pymake.px.TaskSpec:
"""从 pymake.tasks 或 aliases 中查找指定名称的 TaskSpec."""
for spec in pymake.tasks:
if spec.name == name:
return spec
# 单任务别名(doc/lint/tox)内联在 aliases dict 中
value = pymake.aliases.get(name)
if isinstance(value, pymake.px.TaskSpec):
return value
raise KeyError(f"任务 {name!r} 未找到")
class TestTaskSpecDefinitions:
"""Test that all TaskSpec definitions are valid."""
def test_uv_build_spec(self) -> None:
"""uv_build spec should be properly defined."""
spec = _find_task("uv_build")
assert spec.name == "uv_build"
assert spec.cmd == ["uv", "build"]
assert spec.skip_if_missing is False
def test_maturin_build_spec(self) -> None:
"""maturin_build spec should be properly defined."""
spec = _find_task("maturin_build")
assert spec.name == "maturin_build"
assert isinstance(spec.cmd, list)
assert spec.skip_if_missing is False
def test_uv_sync_spec(self) -> None:
"""uv_sync spec should be properly defined."""
spec = _find_task("uv_sync")
assert spec.name == "uv_sync"
assert spec.cmd == ["uv", "sync"]
assert spec.skip_if_missing is False
def test_git_clean_spec(self) -> None:
"""git_clean spec should be properly defined."""
spec = _find_task("git_clean")
assert spec.name == "git_clean"
assert spec.cmd == ["gitt", "c"]
assert spec.skip_if_missing is False
def test_test_spec(self) -> None:
"""test spec should be properly defined."""
spec = _find_task("test")
assert spec.name == "test"
assert isinstance(spec.cmd, list)
assert "pytest" in spec.cmd
assert "-m" in spec.cmd
assert "not slow" in spec.cmd
assert spec.skip_if_missing is False
def test_test_fast_spec(self) -> None:
"""test_fast spec should be properly defined."""
spec = _find_task("test_fast")
assert spec.name == "test_fast"
assert isinstance(spec.cmd, list)
assert "pytest" in spec.cmd
assert "-n" not in spec.cmd # test_fast doesn't use parallel
assert spec.skip_if_missing is False
def test_test_coverage_spec(self) -> None:
"""test_coverage spec should be properly defined."""
spec = _find_task("test_coverage")
assert spec.name == "test_coverage"
assert isinstance(spec.cmd, list)
assert "pytest" in spec.cmd
assert "--cov" in spec.cmd
assert spec.skip_if_missing is False
def test_ruff_lint_spec(self) -> None:
"""lint spec should be properly defined."""
spec = _find_task("lint")
assert spec.name == "lint"
assert isinstance(spec.cmd, list)
assert "ruff" in spec.cmd
assert "check" in spec.cmd
assert spec.skip_if_missing is False
def test_doc_spec(self) -> None:
"""doc spec should be properly defined."""
spec = _find_task("doc")
assert spec.name == "doc"
assert isinstance(spec.cmd, list)
assert "sphinx-build" in spec.cmd
assert spec.skip_if_missing is False
def test_hatch_publish_spec(self) -> None:
"""publish_python spec should be properly defined."""
spec = _find_task("publish_python")
assert spec.name == "publish_python"
assert spec.cmd == ["hatch", "publish"]
assert spec.skip_if_missing is False
def test_twine_publish_spec(self) -> None:
"""twine_publish spec should be properly defined."""
spec = _find_task("twine_publish")
assert spec.name == "twine_publish"
assert isinstance(spec.cmd, list)
assert "twine" in spec.cmd
assert "upload" in spec.cmd
assert spec.skip_if_missing is False
def test_tox_spec(self) -> None:
"""tox spec should be properly defined."""
spec = _find_task("tox")
assert spec.name == "tox"
assert spec.cmd == ["tox", "-p", "auto"]
assert spec.skip_if_missing is False
def test_all_tasks_have_correct_cwd(self) -> None:
"""所有任务应该有正确的 cwd 设置(指向项目根目录)."""
# 验证 ROOT_DIR 定义正确(向上三层到达项目根目录)
expected_root = Path(__file__).parent.parent.parent
assert expected_root == pymake.ROOT_DIR
# 验证 tasks 中的所有命令任务都有正确的 cwd
for spec in pymake.tasks:
if spec.cmd is not None:
assert spec.cwd == pymake.ROOT_DIR, f"任务 {spec.name} 的 cwd 应为 {pymake.ROOT_DIR}"
# 验证 aliases 中的内联任务(doc/lint/tox)也有正确的 cwd
for name in ("doc", "lint", "tox"):
spec = _find_task(name)
assert spec.cwd == pymake.ROOT_DIR, f"任务 {name} 的 cwd 应为 {pymake.ROOT_DIR}"
# ---------------------------------------------------------------------- #
# main function
# ---------------------------------------------------------------------- #
class TestMain:
"""Test main function."""
def test_main_calls_run_cli(self) -> None:
"""main() should create a CliRunner and call run_cli()."""
with pytest.raises(SystemExit) as exc_info:
pymake.main()
# run_cli() calls sys.exit(), so we should get SystemExit
# The exit code depends on whether any commands are available
assert exc_info.value.code in (0, 1, 2)
def test_main_with_list_argument(self) -> None:
"""main() should handle --list argument."""
with patch("sys.argv", ["pymake", "--list"]), pytest.raises(SystemExit) as exc_info:
pymake.main()
assert exc_info.value.code == 0
def test_main_creates_runner_with_multiple_commands(self) -> None:
"""main() should create a CliRunner with multiple commands."""
# We can't easily test the runner creation without mocking,
# but we can verify that main() doesn't raise an error for --list
with patch("sys.argv", ["pymake", "--list"]), pytest.raises(SystemExit):
pymake.main()
def test_main_with_no_args_shows_help(self) -> None:
"""main() with no args should show help and exit with failure."""
with patch("sys.argv", ["pymake"]), pytest.raises(SystemExit) as exc_info:
pymake.main()
assert exc_info.value.code == 1
+108
View File
@@ -0,0 +1,108 @@
"""Tests for ops.pymake 模块 (@px.tool 注册验证)."""
from __future__ import annotations
import pyflowx as px
import pyflowx.ops.pymake
from pyflowx.tools import _TOOL_REGISTRY, get_tool
# ---------------------------------------------------------------------- #
# @px.tool 注册验证
# ---------------------------------------------------------------------- #
class TestPymakeRegistration:
"""``pymake`` 模块 ``@px.tool`` 注册验证."""
def test_pymake_registered(self) -> None:
"""pymake 应在 _TOOL_REGISTRY 中注册."""
assert "pymake" in _TOOL_REGISTRY
def test_visible_subcommands(self) -> None:
"""可见子命令应包含核心构建/测试命令."""
subs = px.list_subcommands("pymake")
assert "b" in subs
assert "bc" in subs
assert "sync" in subs
assert "c" in subs
assert "t" in subs
assert "tf" in subs
assert "lint" in subs
assert "tox" in subs
assert "ba" in subs
assert "bump" in subs
assert "cov" in subs
assert "tc" in subs
assert "p" in subs
def test_hidden_subcommands_excluded(self) -> None:
"""hidden 子命令不应出现在可见列表中."""
subs = px.list_subcommands("pymake")
assert "pyrefly_check" not in subs
assert "git_push" not in subs
assert "git_push_tags" not in subs
assert "twine_publish" not in subs
assert "publish_python" not in subs
def test_hidden_subcommands_included_with_flag(self) -> None:
"""include_hidden=True 时 hidden 子命令应出现."""
subs = px.list_subcommands("pymake", include_hidden=True)
assert "pyrefly_check" in subs
assert "git_push" in subs
assert "git_push_tags" in subs
assert "twine_publish" in subs
assert "publish_python" in subs
def test_b_has_cmd(self) -> None:
"""b 应有 cmd (uv build)."""
spec = get_tool("pymake", "b")
assert spec.cmd is not None
assert "uv" in spec.cmd
assert "build" in spec.cmd
def test_bc_has_cmd(self) -> None:
"""bc 应有 cmd (maturin build)."""
spec = get_tool("pymake", "bc")
assert spec.cmd is not None
assert "maturin" in spec.cmd
def test_ba_needs_b_and_bc(self) -> None:
"""ba 应依赖 b 和 bc."""
spec = get_tool("pymake", "ba")
assert "b" in spec.needs
assert "bc" in spec.needs
assert spec.cmd is None
def test_tc_needs_c_pyrefly_lint(self) -> None:
"""tc 应依赖 c, pyrefly_check, lint."""
spec = get_tool("pymake", "tc")
assert "c" in spec.needs
assert "pyrefly_check" in spec.needs
assert "lint" in spec.needs
def test_cov_needs_test_coverage(self) -> None:
"""cov 应依赖 test_coverage."""
spec = get_tool("pymake", "cov")
assert "test_coverage" in spec.needs
def test_p_needs_c_git_push(self) -> None:
"""p 应依赖 c, git_push, git_push_tags."""
spec = get_tool("pymake", "p")
assert "c" in spec.needs
assert "git_push" in spec.needs
assert "git_push_tags" in spec.needs
def test_bumpversion_needs_git_add_all(self) -> None:
"""bumpversion 应依赖 git_add_all."""
spec = get_tool("pymake", "bumpversion")
assert "git_add_all" in spec.needs
def test_git_add_all_needs_tc(self) -> None:
"""git_add_all 应依赖 tc."""
spec = get_tool("pymake", "git_add_all")
assert "tc" in spec.needs
def test_aggregate_tasks_have_no_cmd(self) -> None:
"""聚合任务 (ba/bump/cov/tc/p) 应无 cmd."""
for name in ("ba", "bump", "cov", "tc", "p"):
spec = get_tool("pymake", name)
assert spec.cmd is None, f"{name} 应为聚合任务 (无 cmd)"
+58
View File
@@ -0,0 +1,58 @@
"""Tests for reseticoncache tool (reset_icon_cache_run 函数)."""
from __future__ import annotations
import subprocess
import pytest
from pyflowx.conditions import Constants
from pyflowx.ops.reseticoncache import reset_icon_cache_run
class TestResetIconCacheRun:
"""``reset_icon_cache_run`` 函数测试."""
def test_non_windows_skips(self, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]) -> None:
"""非 Windows 平台应打印提示并跳过."""
monkeypatch.setattr(Constants, "IS_WINDOWS", False)
reset_icon_cache_run()
captured = capsys.readouterr()
assert "仅在 Windows 上支持" in captured.out
def test_windows_no_localappdata_skips(
self, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
) -> None:
"""Windows 上 LOCALAPPDATA 未设置时应跳过."""
monkeypatch.setattr(Constants, "IS_WINDOWS", True)
monkeypatch.setattr("os.environ.get", lambda key, default="": "" if key == "LOCALAPPDATA" else default)
reset_icon_cache_run()
captured = capsys.readouterr()
assert "LOCALAPPDATA" in captured.out
def test_windows_executes_all_steps(
self, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
) -> None:
"""Windows 上应依次执行 kill/delete/delete_all/restart 四步."""
monkeypatch.setattr(Constants, "IS_WINDOWS", True)
local_app_data = "/tmp/fake_local_app_data"
monkeypatch.setattr(
"os.environ.get", lambda key, default="": local_app_data if key == "LOCALAPPDATA" else default
)
# 让 icon_cache_db 和 explorer_cache_dir 都不存在, 跳过删除分支
from pathlib import Path
monkeypatch.setattr(Path, "exists", lambda _self: False)
ran_cmds: list[list[str]] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: ran_cmds.append(cmd))
reset_icon_cache_run()
captured = capsys.readouterr()
# 验证执行了 kill_explorer 和 restart_explorer (delete 分支因 exists=False 被跳过)
assert any("taskkill" in str(cmd) for cmd in ran_cmds)
assert any("explorer.exe" in str(cmd) for cmd in ran_cmds)
assert any("start" in str(cmd) and "explorer.exe" in str(cmd) for cmd in ran_cmds)
assert "图标缓存已重置" in captured.out
+26 -34
View File
@@ -5,9 +5,8 @@ from __future__ import annotations
from pathlib import Path
from unittest.mock import MagicMock, patch
import pyflowx as px
from pyflowx.cli import screenshot
from pyflowx.conditions import Constants
from pyflowx.ops import screenshot as media
# ---------------------------------------------------------------------- #
@@ -19,13 +18,13 @@ class TestGetScreenshotPath:
def test_get_screenshot_path_with_filename(self, tmp_path: Path) -> None:
"""Should get screenshot path with filename."""
with patch.object(Path, "home", return_value=tmp_path):
result = screenshot.get_screenshot_path("test.png")
result = media.get_screenshot_path("test.png")
assert result.name == "test.png"
def test_get_screenshot_path_without_filename(self, tmp_path: Path) -> None:
"""Should get screenshot path without filename."""
with patch.object(Path, "home", return_value=tmp_path):
result = screenshot.get_screenshot_path()
result = media.get_screenshot_path()
assert "screenshot_" in result.name
assert result.suffix == ".png"
@@ -42,7 +41,7 @@ class TestTakeScreenshotFull:
Path, "home", return_value=tmp_path
), patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(returncode=0)
screenshot.take_screenshot_full()
media.take_screenshot_full()
assert mock_run.called
def test_take_screenshot_full_macos(self, tmp_path: Path) -> None:
@@ -51,7 +50,7 @@ class TestTakeScreenshotFull:
Path, "home", return_value=tmp_path
), patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(returncode=0)
screenshot.take_screenshot_full()
media.take_screenshot_full()
assert mock_run.called
def test_take_screenshot_full_linux(self, tmp_path: Path) -> None:
@@ -60,9 +59,18 @@ class TestTakeScreenshotFull:
Path, "home", return_value=tmp_path
), patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(returncode=0)
screenshot.take_screenshot_full()
media.take_screenshot_full()
assert mock_run.called
def test_take_screenshot_full_linux_scrot_fallback(self, tmp_path: Path) -> None:
"""Should fallback to scrot when gnome-screenshot not found."""
with patch.object(Constants, "IS_WINDOWS", False), patch.object(Constants, "IS_MACOS", False), patch.object(
Path, "home", return_value=tmp_path
), patch("subprocess.run") as mock_run:
mock_run.side_effect = [FileNotFoundError(), MagicMock(returncode=0)]
media.take_screenshot_full()
assert mock_run.call_count == 2
# ---------------------------------------------------------------------- #
# take_screenshot_area
@@ -76,7 +84,7 @@ class TestTakeScreenshotArea:
Path, "home", return_value=tmp_path
), patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(returncode=0)
screenshot.take_screenshot_area()
media.take_screenshot_area()
assert mock_run.called
def test_take_screenshot_area_macos(self, tmp_path: Path) -> None:
@@ -85,7 +93,7 @@ class TestTakeScreenshotArea:
Path, "home", return_value=tmp_path
), patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(returncode=0)
screenshot.take_screenshot_area()
media.take_screenshot_area()
assert mock_run.called
def test_take_screenshot_area_linux(self, tmp_path: Path) -> None:
@@ -94,30 +102,14 @@ class TestTakeScreenshotArea:
Path, "home", return_value=tmp_path
), patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(returncode=0)
screenshot.take_screenshot_area()
media.take_screenshot_area()
assert mock_run.called
# ---------------------------------------------------------------------- #
# main function
# ---------------------------------------------------------------------- #
class TestMain:
"""Test main function."""
def test_main_full_command(self, tmp_path: Path) -> None:
"""main() should handle full command."""
with patch("sys.argv", ["screenshot", "full"]), patch.object(px, "run") as mock_run:
screenshot.main()
assert mock_run.called
def test_main_area_command(self, tmp_path: Path) -> None:
"""main() should handle area command."""
with patch("sys.argv", ["screenshot", "area"]), patch.object(px, "run") as mock_run:
screenshot.main()
assert mock_run.called
def test_main_with_no_args_shows_help(self) -> None:
"""main() with no args should show help."""
with patch("sys.argv", ["screenshot"]):
screenshot.main()
# Should print help and return
def test_take_screenshot_area_linux_scrot_fallback(self, tmp_path: Path) -> None:
"""Should fallback to scrot -s when gnome-screenshot not found."""
with patch.object(Constants, "IS_WINDOWS", False), patch.object(Constants, "IS_MACOS", False), patch.object(
Path, "home", return_value=tmp_path
), patch("subprocess.run") as mock_run:
mock_run.side_effect = [FileNotFoundError(), MagicMock(returncode=0)]
media.take_screenshot_area()
assert mock_run.call_count == 2
+9 -75
View File
@@ -8,8 +8,7 @@ from unittest.mock import MagicMock, patch
import pytest
import pyflowx as px
from pyflowx.cli import sshcopyid
from pyflowx.ops import sshcopyid as system
# ---------------------------------------------------------------------- #
@@ -21,7 +20,7 @@ class TestSshCopyId:
def test_ssh_copy_id_pub_key_not_exists(self, tmp_path: Path) -> None:
"""Should handle nonexistent public key."""
with patch.object(Path, "expanduser", return_value=tmp_path / "nonexistent.pub"), pytest.raises(SystemExit):
sshcopyid.ssh_copy_id("localhost", "user", "password")
system.ssh_copy_id("localhost", "user", "password")
def test_ssh_copy_id_sshpass_not_found(self, tmp_path: Path) -> None:
"""Should handle sshpass not found."""
@@ -31,7 +30,7 @@ class TestSshCopyId:
with patch.object(Path, "expanduser", return_value=pub_key), patch(
"subprocess.run", side_effect=FileNotFoundError
), pytest.raises(SystemExit):
sshcopyid.ssh_copy_id("localhost", "user", "password")
system.ssh_copy_id("localhost", "user", "password")
def test_ssh_copy_id_timeout(self, tmp_path: Path) -> None:
"""Should handle SSH timeout."""
@@ -41,7 +40,7 @@ class TestSshCopyId:
with patch.object(Path, "expanduser", return_value=pub_key), patch(
"subprocess.run", side_effect=subprocess.TimeoutExpired("cmd", 30)
), pytest.raises(SystemExit):
sshcopyid.ssh_copy_id("localhost", "user", "password")
system.ssh_copy_id("localhost", "user", "password")
def test_ssh_copy_id_process_error(self, tmp_path: Path) -> None:
"""Should handle SSH process error."""
@@ -51,7 +50,7 @@ class TestSshCopyId:
with patch.object(Path, "expanduser", return_value=pub_key), patch(
"subprocess.run", side_effect=subprocess.CalledProcessError(1, "cmd")
), pytest.raises(SystemExit):
sshcopyid.ssh_copy_id("localhost", "user", "password")
system.ssh_copy_id("localhost", "user", "password")
def test_ssh_copy_id_success(self, tmp_path: Path) -> None:
"""Should deploy SSH key successfully."""
@@ -60,7 +59,7 @@ class TestSshCopyId:
with patch.object(Path, "expanduser", return_value=pub_key), patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(returncode=0)
sshcopyid.ssh_copy_id("localhost", "user", "password")
system.ssh_copy_id("localhost", "user", "password")
assert mock_run.called
def test_ssh_copy_id_with_custom_port(self, tmp_path: Path) -> None:
@@ -70,7 +69,7 @@ class TestSshCopyId:
with patch.object(Path, "expanduser", return_value=pub_key), patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(returncode=0)
sshcopyid.ssh_copy_id("localhost", "user", "password", port=2222)
system.ssh_copy_id("localhost", "user", "password", port=2222)
# Verify port is used
call_args = mock_run.call_args[0][0]
assert "2222" in call_args
@@ -82,7 +81,7 @@ class TestSshCopyId:
with patch.object(Path, "expanduser", return_value=custom_key), patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(returncode=0)
sshcopyid.ssh_copy_id("localhost", "user", "password", keypath=str(custom_key))
system.ssh_copy_id("localhost", "user", "password", keypath=str(custom_key))
assert mock_run.called
def test_ssh_copy_id_with_custom_timeout(self, tmp_path: Path) -> None:
@@ -92,72 +91,7 @@ class TestSshCopyId:
with patch.object(Path, "expanduser", return_value=pub_key), patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(returncode=0)
sshcopyid.ssh_copy_id("localhost", "user", "password", timeout=60)
system.ssh_copy_id("localhost", "user", "password", timeout=60)
# Verify timeout is used in ConnectTimeout option
call_args = mock_run.call_args[0][0]
assert "ConnectTimeout=60" in call_args
# ---------------------------------------------------------------------- #
# main function
# ---------------------------------------------------------------------- #
class TestMain:
"""Test main function."""
def test_main_with_required_args(self) -> None:
"""main() should handle required arguments."""
with patch("sys.argv", ["sshcopyid", "localhost", "user", "password"]), patch.object(
px, "run"
) as mock_run, patch.object(sshcopyid, "ssh_copy_id"):
sshcopyid.main()
assert mock_run.called
graph = mock_run.call_args[0][0]
assert isinstance(graph, px.Graph)
def test_main_with_custom_port(self) -> None:
"""main() should handle custom port argument."""
with patch("sys.argv", ["sshcopyid", "localhost", "user", "password", "--port", "2222"]), patch.object(
px, "run"
) as mock_run, patch.object(sshcopyid, "ssh_copy_id"):
sshcopyid.main()
assert mock_run.called
def test_main_with_custom_keypath(self) -> None:
"""main() should handle custom keypath argument."""
with patch(
"sys.argv", ["sshcopyid", "localhost", "user", "password", "--keypath", "/custom/key.pub"]
), patch.object(px, "run") as mock_run, patch.object(sshcopyid, "ssh_copy_id"):
sshcopyid.main()
assert mock_run.called
def test_main_with_custom_timeout(self) -> None:
"""main() should handle custom timeout argument."""
with patch("sys.argv", ["sshcopyid", "localhost", "user", "password", "--timeout", "60"]), patch.object(
px, "run"
) as mock_run, patch.object(sshcopyid, "ssh_copy_id"):
sshcopyid.main()
assert mock_run.called
def test_main_with_no_args_shows_help(self) -> None:
"""main() with no args should show help and exit."""
with patch("sys.argv", ["sshcopyid"]), pytest.raises(SystemExit) as exc_info:
sshcopyid.main()
assert exc_info.value.code == 2
def test_main_creates_task_spec_with_correct_name(self) -> None:
"""main() should create TaskSpec with correct name."""
with patch("sys.argv", ["sshcopyid", "localhost", "user", "password"]), patch.object(
px, "run"
) as mock_run, patch.object(sshcopyid, "ssh_copy_id"):
sshcopyid.main()
graph = mock_run.call_args[0][0]
task_names = list(graph.all_specs().keys())
assert "ssh_deploy" in task_names
def test_main_uses_thread_strategy(self) -> None:
"""main() should use thread strategy."""
with patch("sys.argv", ["sshcopyid", "localhost", "user", "password"]), patch.object(
px, "run"
) as mock_run, patch.object(sshcopyid, "ssh_copy_id"):
sshcopyid.main()
assert mock_run.call_args[1]["strategy"] == "thread"
+145
View File
@@ -0,0 +1,145 @@
"""Tests for ops.system 清屏/进程终止/命令查找函数."""
from __future__ import annotations
import subprocess
from unittest.mock import MagicMock
import pytest
from pyflowx.conditions import Constants
from pyflowx.ops.clr import clear_screen_run
from pyflowx.ops.taskkill import taskkill_run
from pyflowx.ops.which import which_run
# ---------------------------------------------------------------------- #
# clear_screen_run
# ---------------------------------------------------------------------- #
class TestClearScreenRun:
"""``clear_screen_run`` 函数测试."""
def test_windows_uses_cls(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""Windows 上应调用 cls."""
monkeypatch.setattr(Constants, "IS_WINDOWS", True)
ran_cmds: list[list[str]] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: ran_cmds.append(cmd) or MagicMock())
clear_screen_run()
assert ran_cmds == [["cls"]]
def test_non_windows_uses_clear(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""非 Windows 上应调用 clear."""
monkeypatch.setattr(Constants, "IS_WINDOWS", False)
ran_cmds: list[list[str]] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: ran_cmds.append(cmd) or MagicMock())
clear_screen_run()
assert ran_cmds == [["clear"]]
# ---------------------------------------------------------------------- #
# taskkill_run
# ---------------------------------------------------------------------- #
class TestTaskkillRun:
"""``taskkill_run`` 函数测试."""
def test_windows_uses_taskkill(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""Windows 上应使用 taskkill /f /im 并加通配符."""
monkeypatch.setattr(Constants, "IS_WINDOWS", True)
ran_cmds: list[list[str]] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: ran_cmds.append(cmd) or MagicMock())
taskkill_run(["chrome.exe", "python.exe"])
assert ran_cmds == [
["taskkill", "/f", "/im", "chrome.exe*"],
["taskkill", "/f", "/im", "python.exe*"],
]
def test_non_windows_uses_pkill(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""非 Windows 上应使用 pkill -f 并加通配符."""
monkeypatch.setattr(Constants, "IS_WINDOWS", False)
ran_cmds: list[list[str]] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: ran_cmds.append(cmd) or MagicMock())
taskkill_run(["chrome"])
assert ran_cmds == [["pkill", "-f", "chrome*"]]
def test_empty_list_does_nothing(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""空列表不应调用 subprocess."""
monkeypatch.setattr(Constants, "IS_WINDOWS", True)
called: list[list[str]] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: called.append(cmd) or MagicMock())
taskkill_run([])
assert called == []
def test_prints_progress(self, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]) -> None:
"""应打印每个进程的终止信息."""
monkeypatch.setattr(Constants, "IS_WINDOWS", True)
monkeypatch.setattr(subprocess, "run", lambda *_, **__: MagicMock())
taskkill_run(["chrome.exe"])
captured = capsys.readouterr()
assert "chrome.exe" in captured.out
# ---------------------------------------------------------------------- #
# which_run
# ---------------------------------------------------------------------- #
class TestWhichRun:
"""``which_run`` 函数测试."""
def test_windows_uses_where(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""Windows 上应调用 where."""
monkeypatch.setattr(Constants, "IS_WINDOWS", True)
ran_cmds: list[list[str]] = []
mock_result = MagicMock(returncode=0, stdout="C:\\Windows\\python.exe\n")
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: ran_cmds.append(cmd) or mock_result)
which_run(["python"])
assert ran_cmds == [["where", "python"]]
def test_non_windows_uses_which(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""非 Windows 上应调用 which."""
monkeypatch.setattr(Constants, "IS_WINDOWS", False)
ran_cmds: list[list[str]] = []
mock_result = MagicMock(returncode=0, stdout="/usr/bin/python\n")
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: ran_cmds.append(cmd) or mock_result)
which_run(["python"])
assert ran_cmds == [["which", "python"]]
def test_found_prints_path(self, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]) -> None:
"""找到命令时应打印 <cmd> -> <path>."""
monkeypatch.setattr(Constants, "IS_WINDOWS", False)
mock_result = MagicMock(returncode=0, stdout="/usr/bin/python\n")
monkeypatch.setattr(subprocess, "run", lambda *_, **__: mock_result)
which_run(["python"])
captured = capsys.readouterr()
assert "python" in captured.out
assert "/usr/bin/python" in captured.out
def test_not_found_prints_message(
self, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
) -> None:
"""未找到命令时应打印未找到."""
monkeypatch.setattr(Constants, "IS_WINDOWS", False)
mock_result = MagicMock(returncode=1, stdout="")
monkeypatch.setattr(subprocess, "run", lambda *_, **__: mock_result)
which_run(["nonexistent_cmd"])
captured = capsys.readouterr()
assert "未找到" in captured.out
def test_multiple_commands(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""多个命令应分别查找."""
monkeypatch.setattr(Constants, "IS_WINDOWS", False)
ran_cmds: list[list[str]] = []
mock_result = MagicMock(returncode=0, stdout="/usr/bin/python\n")
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: ran_cmds.append(cmd) or mock_result)
which_run(["python", "pip"])
assert ran_cmds == [["which", "python"], ["which", "pip"]]
def test_windows_multiline_takes_first(
self, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
) -> None:
"""Windows where 返回多行时应取第一行."""
monkeypatch.setattr(Constants, "IS_WINDOWS", True)
mock_result = MagicMock(returncode=0, stdout="C:\\first\\python.exe\nC:\\second\\python.exe\n")
monkeypatch.setattr(subprocess, "run", lambda *_, **__: mock_result)
which_run(["python"])
captured = capsys.readouterr()
assert "C:\\first\\python.exe" in captured.out
assert "C:\\second" not in captured.out
-102
View File
@@ -1,102 +0,0 @@
"""Tests for cli.taskkill module."""
from __future__ import annotations
from unittest.mock import patch
import pytest
import pyflowx as px
from pyflowx.cli.system import taskkill
from pyflowx.conditions import Constants
# ---------------------------------------------------------------------- #
# main function
# ---------------------------------------------------------------------- #
class TestMain:
"""Test main function."""
def test_main_with_single_process(self) -> None:
"""main() should handle single process argument."""
with patch("sys.argv", ["taskkill", "chrome.exe"]), patch.object(px, "run") as mock_run:
taskkill.main()
assert mock_run.called
graph = mock_run.call_args[0][0]
assert isinstance(graph, px.Graph)
def test_main_with_multiple_processes(self) -> None:
"""main() should handle multiple process arguments."""
with patch("sys.argv", ["taskkill", "chrome.exe", "python.exe", "node.exe"]), patch.object(
px, "run"
) as mock_run:
taskkill.main()
assert mock_run.called
graph = mock_run.call_args[0][0]
assert isinstance(graph, px.Graph)
def test_main_with_no_args_shows_help(self) -> None:
"""main() with no args should show help and exit."""
with patch("sys.argv", ["taskkill"]), pytest.raises(SystemExit) as exc_info:
taskkill.main()
assert exc_info.value.code == 2
def test_main_creates_task_specs_with_correct_names(self) -> None:
"""main() should create TaskSpecs with correct names."""
with patch("sys.argv", ["taskkill", "chrome.exe", "python.exe"]), patch.object(px, "run") as mock_run:
taskkill.main()
graph = mock_run.call_args[0][0]
task_names = list(graph.all_specs().keys())
assert "kill_chrome.exe" in task_names
assert "kill_python.exe" in task_names
def test_main_uses_thread_strategy(self) -> None:
"""main() should use thread strategy."""
with patch("sys.argv", ["taskkill", "chrome.exe"]), patch.object(px, "run") as mock_run:
taskkill.main()
assert mock_run.call_args[1]["strategy"] == "thread"
def test_main_windows_command_format(self) -> None:
"""main() should use Windows command format on Windows."""
if Constants.IS_WINDOWS:
with patch("sys.argv", ["taskkill", "chrome.exe"]), patch.object(px, "run") as mock_run:
taskkill.main()
graph = mock_run.call_args[0][0]
specs = graph.all_specs()
# Check that command includes Windows taskkill format
for spec in specs.values():
assert spec.cmd[0] == "taskkill"
assert spec.cmd[1] == "/f"
assert spec.cmd[2] == "/im"
def test_main_linux_command_format(self) -> None:
"""main() should use Linux command format on Linux."""
with patch.object(Constants, "IS_WINDOWS", False), patch("sys.argv", ["taskkill", "chrome.exe"]), patch.object(
px, "run"
) as mock_run:
taskkill.main()
graph = mock_run.call_args[0][0]
specs = graph.all_specs()
# Check that command includes Linux pkill format
for spec in specs.values():
assert spec.cmd[0] == "pkill"
assert spec.cmd[1] == "-f"
def test_main_tasks_have_verbose_true(self) -> None:
"""main() should create tasks with verbose=True."""
with patch("sys.argv", ["taskkill", "chrome.exe"]), patch.object(px, "run") as mock_run:
taskkill.main()
graph = mock_run.call_args[0][0]
specs = graph.all_specs()
for spec in specs.values():
assert spec.verbose is True
def test_main_adds_wildcard_to_process_name(self) -> None:
"""main() should add wildcard to process name."""
with patch("sys.argv", ["taskkill", "chrome.exe"]), patch.object(px, "run") as mock_run:
taskkill.main()
graph = mock_run.call_args[0][0]
specs = graph.all_specs()
# Check that wildcard is added
for spec in specs.values():
assert spec.cmd[-1].endswith("*")
+273 -197
View File
@@ -82,9 +82,11 @@ class TestRetryPolicy:
raise RuntimeError("not yet")
return "ok"
graph = px.Graph.from_specs([
px.TaskSpec("flaky", flaky, retry=RetryPolicy(max_attempts=3)),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("flaky", flaky, retry=RetryPolicy(max_attempts=3)),
]
)
report = px.run(graph, strategy="sequential")
assert report.success
assert report["flaky"] == "ok"
@@ -95,9 +97,11 @@ class TestRetryPolicy:
def always_fail() -> None:
raise RuntimeError("nope")
graph = px.Graph.from_specs([
px.TaskSpec("f", always_fail, retry=RetryPolicy(max_attempts=3)),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("f", always_fail, retry=RetryPolicy(max_attempts=3)),
]
)
with pytest.raises(px.TaskFailedError) as exc_info:
px.run(graph, strategy="sequential")
assert exc_info.value.attempts == 3
@@ -111,13 +115,15 @@ class TestRetryPolicy:
raise KeyError("not retried")
# retry_on=(ValueError,) -> KeyError 不应被重试
graph = px.Graph.from_specs([
px.TaskSpec(
"f",
fail_with_keyerror,
retry=RetryPolicy(max_attempts=3, retry_on=(ValueError,)),
),
])
graph = px.Graph.from_specs(
[
px.TaskSpec(
"f",
fail_with_keyerror,
retry=RetryPolicy(max_attempts=3, retry_on=(ValueError,)),
),
]
)
with pytest.raises(px.TaskFailedError) as exc_info:
px.run(graph, strategy="sequential")
# KeyError 不在 retry_on 中,应只尝试 1 次
@@ -136,13 +142,15 @@ class TestRetryPolicy:
raise RuntimeError("not yet")
return "ok"
graph = px.Graph.from_specs([
px.TaskSpec(
"flaky",
flaky,
retry=RetryPolicy(max_attempts=3, delay=0.05, backoff=2.0),
),
])
graph = px.Graph.from_specs(
[
px.TaskSpec(
"flaky",
flaky,
retry=RetryPolicy(max_attempts=3, delay=0.05, backoff=2.0),
),
]
)
report = px.run(graph, strategy="sequential")
assert report.success
# 第 2 次重试应在 delay=0.05 后,第 3 次应在 0.05*2=0.10 后
@@ -161,9 +169,11 @@ class TestRetryPolicy:
raise RuntimeError("not yet")
return "ok"
graph = px.Graph.from_specs([
px.TaskSpec("flaky", flaky, retry=RetryPolicy(max_attempts=3)),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("flaky", flaky, retry=RetryPolicy(max_attempts=3)),
]
)
report = px.run(graph, strategy="async")
assert report.success
assert report["flaky"] == "ok"
@@ -187,9 +197,11 @@ class TestTaskHooks:
return "ok"
hooks = TaskHooks(pre_run=pre_run)
graph = px.Graph.from_specs([
px.TaskSpec("t", fn, hooks=hooks),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("t", fn, hooks=hooks),
]
)
report = px.run(graph, strategy="sequential")
assert report.success
assert events == ["pre:t", "run"]
@@ -205,9 +217,11 @@ class TestTaskHooks:
return 42
hooks = TaskHooks(post_run=post_run)
graph = px.Graph.from_specs([
px.TaskSpec("t", fn, hooks=hooks),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("t", fn, hooks=hooks),
]
)
report = px.run(graph, strategy="sequential")
assert report.success
assert captured == {"name": "t", "result": 42}
@@ -223,9 +237,11 @@ class TestTaskHooks:
raise ValueError("boom")
hooks = TaskHooks(on_failure=on_failure)
graph = px.Graph.from_specs([
px.TaskSpec("t", fn, hooks=hooks, continue_on_error=True),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("t", fn, hooks=hooks, continue_on_error=True),
]
)
report = px.run(graph, strategy="sequential")
# continue_on_error=True -> 报告成功但任务失败
assert report.success
@@ -242,14 +258,16 @@ class TestTaskHooks:
events.append("post")
hooks = TaskHooks(pre_run=pre_run, post_run=post_run)
graph = px.Graph.from_specs([
px.TaskSpec(
"t",
fn=lambda: "ok",
hooks=hooks,
conditions=(lambda _ctx: False,),
),
])
graph = px.Graph.from_specs(
[
px.TaskSpec(
"t",
fn=lambda: "ok",
hooks=hooks,
conditions=(lambda _ctx: False,),
),
]
)
report = px.run(graph, strategy="sequential")
assert report.success
assert report.result_of("t").status == TaskStatus.SKIPPED
@@ -266,9 +284,11 @@ class TestTaskHooks:
return "ok"
hooks = TaskHooks(pre_run=pre_run)
graph = px.Graph.from_specs([
px.TaskSpec("t", fn, hooks=hooks),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("t", fn, hooks=hooks),
]
)
report = px.run(graph, strategy="async")
assert report.success
assert events == ["pre:t", "run"]
@@ -414,10 +434,12 @@ class TestSoftDependencies:
order.append("fast")
return f"after-{slow}"
graph = px.Graph.from_specs([
px.TaskSpec("slow", slow),
px.TaskSpec("fast", fast, soft_depends_on=("slow",)),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("slow", slow),
px.TaskSpec("fast", fast, soft_depends_on=("slow",)),
]
)
report = px.run(graph, strategy="dependency")
assert report.success
# soft 依赖应等待 slow 完成后再执行 fast
@@ -433,15 +455,17 @@ class TestSoftDependencies:
def downstream(fail: str = "default") -> str:
return f"got:{fail}"
graph = px.Graph.from_specs([
px.TaskSpec("fail", fail, continue_on_error=True),
px.TaskSpec(
"downstream",
downstream,
soft_depends_on=("fail",),
continue_on_error=True,
),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("fail", fail, continue_on_error=True),
px.TaskSpec(
"downstream",
downstream,
soft_depends_on=("fail",),
continue_on_error=True,
),
]
)
report = px.run(graph, strategy="dependency")
assert report.success
# fail 失败但下游仍执行(使用默认值)
@@ -450,9 +474,11 @@ class TestSoftDependencies:
def test_soft_dependency_validation_unknown_dep(self) -> None:
with pytest.raises(px.MissingDependencyError):
px.Graph.from_specs([
px.TaskSpec("a", lambda: "ok", soft_depends_on=("missing",)),
])
px.Graph.from_specs(
[
px.TaskSpec("a", lambda: "ok", soft_depends_on=("missing",)),
]
)
def test_soft_and_hard_dependency_combined(self) -> None:
order: list[str] = []
@@ -469,11 +495,13 @@ class TestSoftDependencies:
order.append("c")
return f"c-{b}"
graph = px.Graph.from_specs([
px.TaskSpec("a", a),
px.TaskSpec("b", b, depends_on=("a",)),
px.TaskSpec("c", c, depends_on=("b",), soft_depends_on=("a",)),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("a", a),
px.TaskSpec("b", b, depends_on=("a",)),
px.TaskSpec("c", c, depends_on=("b",), soft_depends_on=("a",)),
]
)
report = px.run(graph, strategy="dependency")
assert report.success
assert order == ["a", "b", "c"]
@@ -495,11 +523,13 @@ class TestDependencyDrivenScheduling:
def c(b: int) -> int:
return b + 1
graph = px.Graph.from_specs([
px.TaskSpec("a", a),
px.TaskSpec("b", b, depends_on=("a",)),
px.TaskSpec("c", c, depends_on=("b",)),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("a", a),
px.TaskSpec("b", b, depends_on=("a",)),
px.TaskSpec("c", c, depends_on=("b",)),
]
)
report = px.run(graph, strategy="dependency")
assert report.success
assert report["a"] == 1
@@ -514,10 +544,12 @@ class TestDependencyDrivenScheduling:
async def b(a: str) -> str:
return f"b-{a}"
graph = px.Graph.from_specs([
px.TaskSpec("a", a),
px.TaskSpec("b", b, depends_on=("a",)),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("a", a),
px.TaskSpec("b", b, depends_on=("a",)),
]
)
report = px.run(graph, strategy="dependency")
assert report.success
assert report["b"] == "b-a"
@@ -537,12 +569,14 @@ class TestDependencyDrivenScheduling:
def d(b: int, c: int) -> int:
return b + c
graph = px.Graph.from_specs([
px.TaskSpec("a", a),
px.TaskSpec("b", b, depends_on=("a",)),
px.TaskSpec("c", c, depends_on=("a",)),
px.TaskSpec("d", d, depends_on=("b", "c")),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("a", a),
px.TaskSpec("b", b, depends_on=("a",)),
px.TaskSpec("c", c, depends_on=("a",)),
px.TaskSpec("d", d, depends_on=("b", "c")),
]
)
report = px.run(graph, strategy="dependency")
assert report.success
assert report["a"] == 10
@@ -574,11 +608,13 @@ class TestConcurrencyLimits:
return fn
graph = px.Graph.from_specs([
px.TaskSpec("a", make_fn(1), concurrency_key="db"),
px.TaskSpec("b", make_fn(2), concurrency_key="db"),
px.TaskSpec("c", make_fn(3), concurrency_key="db"),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("a", make_fn(1), concurrency_key="db"),
px.TaskSpec("b", make_fn(2), concurrency_key="db"),
px.TaskSpec("c", make_fn(3), concurrency_key="db"),
]
)
report = px.run(
graph,
strategy="dependency",
@@ -604,10 +640,12 @@ class TestConcurrencyLimits:
return fn
graph = px.Graph.from_specs([
px.TaskSpec("a", make_fn("a"), concurrency_key="db1"),
px.TaskSpec("b", make_fn("b"), concurrency_key="db2"),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("a", make_fn("a"), concurrency_key="db1"),
px.TaskSpec("b", make_fn("b"), concurrency_key="db2"),
]
)
report = px.run(
graph,
strategy="dependency",
@@ -633,12 +671,14 @@ class TestConcurrencyLimits:
return fn
graph = px.Graph.from_specs([
px.TaskSpec("a", make_fn(1), concurrency_key="pool"),
px.TaskSpec("b", make_fn(2), concurrency_key="pool"),
px.TaskSpec("c", make_fn(3), concurrency_key="pool"),
px.TaskSpec("d", make_fn(4), concurrency_key="pool"),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("a", make_fn(1), concurrency_key="pool"),
px.TaskSpec("b", make_fn(2), concurrency_key="pool"),
px.TaskSpec("c", make_fn(3), concurrency_key="pool"),
px.TaskSpec("d", make_fn(4), concurrency_key="pool"),
]
)
report = px.run(
graph,
strategy="dependency",
@@ -666,11 +706,13 @@ class TestPriority:
return fn
graph = px.Graph.from_specs([
px.TaskSpec("low", make_fn("low"), priority=1),
px.TaskSpec("high", make_fn("high"), priority=10),
px.TaskSpec("mid", make_fn("mid"), priority=5),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("low", make_fn("low"), priority=1),
px.TaskSpec("high", make_fn("high"), priority=10),
px.TaskSpec("mid", make_fn("mid"), priority=5),
]
)
report = px.run(graph, strategy="sequential")
assert report.success
# 高优先级先执行
@@ -696,10 +738,12 @@ class TestContinueOnError:
def downstream() -> str:
return "ran"
graph = px.Graph.from_specs([
px.TaskSpec("fail", fail, continue_on_error=True),
px.TaskSpec("downstream", downstream, depends_on=("fail",)),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("fail", fail, continue_on_error=True),
px.TaskSpec("downstream", downstream, depends_on=("fail",)),
]
)
report = px.run(graph, strategy="sequential")
# continue_on_error 使整体报告成功(不抛异常)
assert report.success
@@ -716,10 +760,12 @@ class TestContinueOnError:
def downstream() -> str:
return "ran"
graph = px.Graph.from_specs([
px.TaskSpec("fail", fail, continue_on_error=True),
px.TaskSpec("downstream", downstream, soft_depends_on=("fail",)),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("fail", fail, continue_on_error=True),
px.TaskSpec("downstream", downstream, soft_depends_on=("fail",)),
]
)
report = px.run(graph, strategy="dependency")
assert report.success
assert report.result_of("fail").status == TaskStatus.FAILED
@@ -734,10 +780,12 @@ class TestContinueOnError:
def other() -> str:
return "ok"
graph = px.Graph.from_specs([
px.TaskSpec("fail", fail, continue_on_error=True),
px.TaskSpec("other", other),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("fail", fail, continue_on_error=True),
px.TaskSpec("other", other),
]
)
report = px.run(graph, strategy="dependency")
assert report.success
assert report.result_of("fail").status == TaskStatus.FAILED
@@ -750,10 +798,12 @@ class TestContinueOnError:
def other() -> str:
return "ok"
graph = px.Graph.from_specs([
px.TaskSpec("fail", fail),
px.TaskSpec("other", other),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("fail", fail),
px.TaskSpec("other", other),
]
)
with pytest.raises(px.TaskFailedError):
px.run(graph, strategy="sequential")
@@ -854,9 +904,11 @@ class TestCompose:
g_extract = px.Graph.from_specs([px.TaskSpec("extract", extract)])
# transform 图:通过 _pending_refs 引用 "extract" 命令
# transform 自身不声明 depends_on,由 compose 展开时自动连接
g_transform = px.Graph.from_specs([
px.TaskSpec("transform", transform),
])
g_transform = px.Graph.from_specs(
[
px.TaskSpec("transform", transform),
]
)
g_transform._pending_refs = ["extract"]
resolved = px.compose({"extract": g_extract, "transform": g_transform})
@@ -943,18 +995,22 @@ class TestCacheKey:
return key
graph1 = px.Graph.from_specs([
px.TaskSpec("t", expensive, args=(5,), cache_key=make_cache_key(5)),
])
graph1 = px.Graph.from_specs(
[
px.TaskSpec("t", expensive, args=(5,), cache_key=make_cache_key(5)),
]
)
report1 = px.run(graph1, strategy="sequential", state=backend)
assert report1.success
assert report1["t"] == 10
assert calls["n"] == 1
# 第二次运行相同输入应命中缓存
graph2 = px.Graph.from_specs([
px.TaskSpec("t", expensive, args=(5,), cache_key=make_cache_key(5)),
])
graph2 = px.Graph.from_specs(
[
px.TaskSpec("t", expensive, args=(5,), cache_key=make_cache_key(5)),
]
)
report2 = px.run(graph2, strategy="sequential", state=backend)
assert report2.success
assert report2["t"] == 10
@@ -976,16 +1032,20 @@ class TestCacheKey:
return key
graph1 = px.Graph.from_specs([
px.TaskSpec("t", expensive, args=(5,), cache_key=make_cache_key(5)),
])
graph1 = px.Graph.from_specs(
[
px.TaskSpec("t", expensive, args=(5,), cache_key=make_cache_key(5)),
]
)
px.run(graph1, strategy="sequential", state=backend)
assert calls["n"] == 1
# 不同输入应 miss
graph2 = px.Graph.from_specs([
px.TaskSpec("t", expensive, args=(7,), cache_key=make_cache_key(7)),
])
graph2 = px.Graph.from_specs(
[
px.TaskSpec("t", expensive, args=(7,), cache_key=make_cache_key(7)),
]
)
px.run(graph2, strategy="sequential", state=backend)
assert calls["n"] == 2
@@ -997,13 +1057,15 @@ class TestEnvAndCwd:
"""测试环境变量与工作目录隔离。"""
def test_env_override_for_cmd(self) -> None:
graph = px.Graph.from_specs([
px.TaskSpec(
"print_var",
cmd=[sys.executable, "-c", "import os; print(os.environ.get('PYFLOWX_TEST_VAR', 'unset'))"],
env={"PYFLOWX_TEST_VAR": "isolated"},
),
])
graph = px.Graph.from_specs(
[
px.TaskSpec(
"print_var",
cmd=[sys.executable, "-c", "import os; print(os.environ.get('PYFLOWX_TEST_VAR', 'unset'))"],
env={"PYFLOWX_TEST_VAR": "isolated"},
),
]
)
report = px.run(graph, strategy="sequential")
assert report.success
@@ -1011,13 +1073,15 @@ class TestEnvAndCwd:
# 在 tmp_path 下创建标记文件
marker = tmp_path / "marker.txt"
marker.write_text("found")
graph = px.Graph.from_specs([
px.TaskSpec(
"check_cwd",
cmd=["ls", "marker.txt"],
cwd=tmp_path,
),
])
graph = px.Graph.from_specs(
[
px.TaskSpec(
"check_cwd",
cmd=["ls", "marker.txt"],
cwd=tmp_path,
),
]
)
report = px.run(graph, strategy="sequential")
assert report.success
@@ -1027,13 +1091,15 @@ class TestEnvAndCwd:
def check_env() -> str:
return os.environ.get("PYFLOWX_LEAK_TEST", "not-set")
graph = px.Graph.from_specs([
px.TaskSpec(
"t",
check_env,
env={"PYFLOWX_LEAK_TEST": "leaked"},
),
])
graph = px.Graph.from_specs(
[
px.TaskSpec(
"t",
check_env,
env={"PYFLOWX_LEAK_TEST": "leaked"},
),
]
)
# fn 任务的环境变量隔离仅在 cmd 任务生效,fn 共享进程环境
# 这里验证 fn 任务不修改外层环境
report = px.run(graph, strategy="sequential")
@@ -1059,21 +1125,23 @@ class TestContextAwareConditions:
def path_b(decide: str = "") -> str:
return f"ran-b:{decide}"
graph = px.Graph.from_specs([
px.TaskSpec("decide", decide),
px.TaskSpec(
"path_a",
path_a,
depends_on=("decide",),
conditions=(BuiltinConditions.DEP_EQUALS("decide", "path_a"),),
),
px.TaskSpec(
"path_b",
path_b,
depends_on=("decide",),
conditions=(BuiltinConditions.DEP_EQUALS("decide", "path_b"),),
),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("decide", decide),
px.TaskSpec(
"path_a",
path_a,
depends_on=("decide",),
conditions=(BuiltinConditions.DEP_EQUALS("decide", "path_a"),),
),
px.TaskSpec(
"path_b",
path_b,
depends_on=("decide",),
conditions=(BuiltinConditions.DEP_EQUALS("decide", "path_b"),),
),
]
)
report = px.run(graph, strategy="dependency")
assert report.success
assert report.result_of("path_a").status == TaskStatus.SKIPPED
@@ -1087,15 +1155,17 @@ class TestContextAwareConditions:
def only_if_nonempty(source: list[int]) -> str:
return f"has-{len(source)}"
graph = px.Graph.from_specs([
px.TaskSpec("source", source),
px.TaskSpec(
"only_if_nonempty",
only_if_nonempty,
depends_on=("source",),
conditions=(BuiltinConditions.DEP_TRUTHY("source"),),
),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("source", source),
px.TaskSpec(
"only_if_nonempty",
only_if_nonempty,
depends_on=("source",),
conditions=(BuiltinConditions.DEP_TRUTHY("source"),),
),
]
)
report = px.run(graph, strategy="dependency")
assert report.success
assert report["only_if_nonempty"] == "has-3"
@@ -1107,15 +1177,17 @@ class TestContextAwareConditions:
def only_if_nonempty(source: list[int]) -> str:
return "should-not-run"
graph = px.Graph.from_specs([
px.TaskSpec("source", source),
px.TaskSpec(
"only_if_nonempty",
only_if_nonempty,
depends_on=("source",),
conditions=(BuiltinConditions.DEP_TRUTHY("source"),),
),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("source", source),
px.TaskSpec(
"only_if_nonempty",
only_if_nonempty,
depends_on=("source",),
conditions=(BuiltinConditions.DEP_TRUTHY("source"),),
),
]
)
report = px.run(graph, strategy="dependency")
assert report.success
assert report.result_of("only_if_nonempty").status == TaskStatus.SKIPPED
@@ -1127,15 +1199,17 @@ class TestContextAwareConditions:
def downstream(source: int) -> str:
return f"got-{source}"
graph = px.Graph.from_specs([
px.TaskSpec("source", source),
px.TaskSpec(
"downstream",
downstream,
depends_on=("source",),
conditions=(BuiltinConditions.DEP_MATCHES("source", lambda v: v > 10),),
),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("source", source),
px.TaskSpec(
"downstream",
downstream,
depends_on=("source",),
conditions=(BuiltinConditions.DEP_MATCHES("source", lambda v: v > 10),),
),
]
)
report = px.run(graph, strategy="dependency")
assert report.success
assert report["downstream"] == "got-42"
@@ -1165,10 +1239,12 @@ class TestPerTaskStrategy:
await asyncio.sleep(0.01)
return f"async-{sync}"
graph = px.Graph.from_specs([
px.TaskSpec("sync", sync_fn),
px.TaskSpec("async", async_fn, depends_on=("sync",)),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("sync", sync_fn),
px.TaskSpec("async", async_fn, depends_on=("sync",)),
]
)
report = px.run(graph, strategy="dependency")
assert report.success
assert report["async"] == "async-sync"

Some files were not shown because too many files have changed in this diff Show More