Compare commits

..

19 Commits

Author SHA1 Message Date
zhou 5e93a75987 refactor(ops): 清理无用代码与过时文档 — 删除 gittool 兼容别名、folderzip 未用 IGNORE 变量、过时迁移计划文档
CI / Lint, Typecheck & Test (push) Failing after 1m3s
2026-07-07 03:02:24 +08:00
zhou 529a024ea1 feat(cli): rich 深度集成 — 执行生命周期 ✓/✗/▸/○ 符号 + dry-run + 命令输出
CI / Lint, Typecheck & Test (push) Failing after 1m59s
2026-07-06 22:27:25 +08:00
zhou 4721b59f40 feat(cli): 重构CLI并添加富文本输出与版本/帮助优化
CI / Lint, Typecheck & Test (push) Failing after 5m41s
- 使用rich库重构工具列表展示为美观的表格形式
- 添加--version/-V参数显示版本号
- 新增未知工具的模糊匹配提示功能
- 统一使用标准错误输出和富文本样式打印错误信息
- 优化命令行参数处理逻辑,支持先解析help/version参数
2026-07-06 22:16:12 +08:00
zhou fca6f17a5c chore: 升级项目适配 Python 3.10+ 并重构代码
1.  移除对 Python 3.8/3.9 的支持,更新 tox、pyproject.toml 配置
2.  替换 typing 导入为 collections.abc 标准库类型
3.  重构 CLI 系统从 argparse 迁移到 typer
4.  优化代码格式与类型注解,修复多处类型兼容问题
5.  更新依赖声明,移除 graphlib_backport 兼容包
2026-07-06 22:14:59 +08:00
zhou d493c6233e docs(rules): 补充多阶段项目约束与更新规则说明
新增开发流程约束文档dev-workflow.md,完善self-driven.md中的多阶段项目规则,
补充阶段迭代、收尾的判定逻辑,修正目标达成的暂停条件说明
2026-07-06 21:16:07 +08:00
zhou 7a87a4177e feat(ops): 新增 _common 共享辅助模块消除跨平台重复代码
CI / Lint, Typecheck & Test (push) Failing after 3m39s
新增 src/pyflowx/ops/_common.py 提供 platform_command/ensure_platform/IGNORE_PATTERNS 三个公共 API, 配套 12 个单元测试; 修复上个提交中 8 个 ops 模块引用缺失模块导致的 ImportError.
2026-07-06 20:40:38 +08:00
zhou 04d6871e8d chore: 发布v0.4.8版本,清理冗余代码并优化跨平台实现
1. 移除folderzip和folderback模块中的默认导出函数与测试用例
2. 重构跨平台命令获取逻辑,统一使用platform_common工具函数
3. 移动IGNORE_PATTERNS常量到公共模块,精简packtool和autofmt代码
4. 重构平台检查逻辑,复用ensure_platform减少重复代码
5. 优化pdftool模块的依赖检查逻辑,提取公共检查函数
6. 修复测试用例中的字符串断言冗余内容
2026-07-06 20:22:59 +08:00
zhou 08149d990d bump version to 0.4.8
CI / Lint, Typecheck & Test (push) Failing after 1m49s
Release / Build, Publish & Release (push) Successful in 46s
2026-07-06 13:19:24 +08:00
zhou 54aef98f3a chore(gittool): 新增ruff缓存目录到排除列表
CI / Lint, Typecheck & Test (push) Failing after 1m52s
将.ruff_cache添加到EXCLUDE_DIRS列表中,避免git工具扫描该缓存目录
2026-07-06 13:11:59 +08:00
zhou e31646e281 feat: 新增envdev、gittool、msdownload、pymake工具模块及对应测试用例
新增四个业务工具模块:
1. envdev: 开发环境镜像源配置工具,支持Python/Conda/Rust镜像配置及Linux系统环境配置
2. gittool: Git操作工具,提供提交、初始化、清理、推送等快捷子命令
3. msdownload: ModelScope模型/数据集下载工具
4. pymake: 项目构建工具,覆盖构建、测试、发布等全流程开发操作

同时为每个工具模块编写了完整的注册验证与功能测试用例
2026-07-06 13:11:09 +08:00
zhou f7fb95af83 refactor: 重构项目架构,移除YAML编排与旧注册系统
1.  将所有YAML配置的工具迁移到`@px.tool`装饰器实现
2.  拆分`llm`模块为`msdownload`和`sglang`子模块
3.  移除废弃的`registry.py`、`yaml_loader.py`和`yamlrun.py`模块
4.  清理项目依赖,移除PyYAML相关包
5.  更新文档与测试用例适配新架构
2026-07-06 13:04:57 +08:00
zhou 6da42ec5ff refactor: 移除YAML配置,统一使用@px.tool装饰器实现CLI工具
1.  新增tools.py模块实现@px.tool装饰器与工具注册表
2.  将所有configs下的YAML工具迁移为ops/xxx.py模块形式
3.  重构cli路由逻辑,优先加载Python工具实现回退YAML
4.  删除所有YAML配置文件与旧的yaml_loader相关代码
5.  调整__init__.py导出API,移除YAML相关依赖
2026-07-06 07:53:22 +08:00
zhou c55a37173a test: 新增 YamlCliRunner 错误分支测试使覆盖率达标 95%
CI / Lint, Typecheck & Test (push) Successful in 3m5s
2026-07-05 22:25:15 +08:00
zhou 960b8672f4 test: 新增截图、PDF工具相关的单元测试
1. 新增Linux平台下截图工具回退到scrot的测试用例
2. 新增pdf加密、解密、重排、PDF转图片的完整测试用例
3. 补充各PDF工具未安装依赖时的分支测试
2026-07-05 22:21:47 +08:00
zhou 4fd1d70b58 test: 修复跨平台路径断言不兼容问题
1. 调整测试用例中的路径比较逻辑,统一替换反斜杠为正斜杠适配Windows平台
2. 将原生echo/true/false命令替换为跨平台的python执行命令,避免环境依赖问题
2026-07-05 22:18:16 +08:00
zhou 6fb9223066 fix: pymake bump 调用 pf bumpversion 而非已移除的外部 bumpversion 命令
CI / Lint, Typecheck & Test (push) Successful in 1m50s
2026-07-05 22:09:28 +08:00
zhou 1f7127357e refactor: 将 cli/system、cli/llm、cli/dev 脚本迁移为 YAML 配置 + register_fn 模式
CI / Lint, Typecheck & Test (push) Successful in 1m44s
删除 cli/system (clearscreen/taskkill/which)、cli/llm (msdownload/sglang)、cli/dev (dockercmd/envdev) 三个目录; 新增 7 个 YAML 配置; 新增 ops/llm.py 模块; 扩展 ops/system.py 和 ops/dev.py; pf.py 添加工具别名; 删除 examples/ 目录
2026-07-05 21:12:41 +08:00
zhou 58ee84ded6 bump version to 0.4.7
CI / Lint, Typecheck & Test (push) Successful in 2m2s
Release / Build, Publish & Release (push) Successful in 1m5s
2026-07-05 19:49:54 +08:00
zhou 9a96e5d052 fix: 修复 bump_project_version 版本号不同步跳号 bug 并抽离到独立模块
CI / Lint, Typecheck & Test (push) Successful in 1m26s
原实现对每个文件独立 +1, 文件版本号不同步时跳号; 改为先读取所有文件取 max 作为基准再统一写入. 同时修复 git add . 违规 (改按文件名) 与 check=False 吞错误. bumpversion 从 ops/dev.py 抽离到 ops/bumpversion.py, 测试简化为 16 个核心场景.
2026-07-05 19:34:44 +08:00
117 changed files with 6972 additions and 11736 deletions
+1 -2
View File
@@ -38,9 +38,8 @@ env
*.swp
*.swo
# 文档与示例(按需保留)
# 文档(按需保留)
docs
examples
# 系统文件
.DS_Store
+1
View File
@@ -14,3 +14,4 @@ wheels/
# Sphinx 文档构建输出
docs/_build/
.trae/refs
@@ -0,0 +1,62 @@
# 迭代 01: typer/rich 迁移
## 本轮目标
采用 typer + rich 简化 CLI 设计, 提高显示效果:
1. 修复 Python 3.8 兼容性测试失败
2. 全量重写 tools.py 为 typer 内核 (保留 @px.tool 装饰器 API)
3. 重写 pf.py 为 rich 驱动主入口
4. rich 深度集成 (执行生命周期/错误/工具列表/dry-run)
5. 放弃 Python 3.8/3.9 支持 (requires-python >=3.10)
## 改动文件清单
### 依赖与配置
- `pyproject.toml`: requires-python >=3.10, 添加 typer>=0.24.0 + rich>=13.7.0 依赖, ruff target-version=py310, pyrefly python-version=3.10
- `tox.ini`: envlist 改为 py310-py314
### 核心重写
- `src/pyflowx/tools.py`: 全量重写 — argparse → typer.Typer; _make_dispatcher 构建命令 dispatcher; _build_typer_app 单命令/多命令路由; _execute_dag DAG 编排; standalone_mode=False + 捕获 typer._click 异常处理退出码; _handle_list_rich rich 表格 --list
- `src/pyflowx/cli/pf.py`: 重写 — rich Panel+Table 工具列表 (含别名/说明列), --help/-h/--version/-V 全局选项, 未知工具模糊匹配建议 (difflib), rich Console stderr 错误
### rich 深度集成
- `src/pyflowx/executors.py`: _make_verbose_callback 用 ✓/✗/▸/○ 符号 + 颜色替代 [verbose] 前缀; _print_dry_run rich styled
- `src/pyflowx/command.py`: 执行命令/返回码用 rich Console + 颜色 (green=0/red=非0)
### Python 3.10+ 现代化
- `src/pyflowx/graph.py`: 移除 graphlib_backport, 直接 import graphlib
- `src/pyflowx/task.py`: Union → X|Y, typing.ContextManager → contextlib.AbstractContextManager
- `src/pyflowx/storage.py`: 同上
- `src/pyflowx/command.py`: typing.List → list, Union → X|Y
- `src/pyflowx/cli/emlmanager.py`: zip() 加 strict= 参数
- `src/pyflowx/executors.py`: zip() 加 strict=True
### 测试
- `tests/test_tools.py`: 全量重写 (84 测试) — 覆盖 typer 内核 (_make_dispatcher/_build_typer_app/_execute_dag/退出码三态)
- `tests/test_runner.py`: verbose 断言更新 (✓ 替代 [verbose])
## 关键决策与依据
1. **保留 @px.tool 装饰器 API**: 用户既有 ops/ 模块全部使用 @px.tool, 改 API 会破坏所有工具. 内部替换 argparse→typer, 外部 API 不变.
2. **standalone_mode=False**: typer 默认 standalone_mode=True 会 sys.exit() 退出进程, 测试中无法捕获. 用 False + 手动捕获 typer._click 异常 (NoArgsIsHelpError/ClickException/Abort) 处理退出码.
3. **typer._click 私有导入**: typer 内置 click (typer._click) 与公共 click 是两套独立类层次, 必须从 typer._click.exceptions 导入 NoArgsIsHelpError/ClickException 才能捕获.
4. **--list 预处理而非 typer callback**: typer 的 no_args_is_help=True 在多命令工具中会拦截 --list (要求先给 subcommand). 在 run_tool 中预处理 --list (任意位置出现即列出任务) 绕过此限制.
5. **单命令 vs 多命令**: 单命令工具 (仅 subcommand=None) 用 callback(invoke_without_command=True), no_args_is_help=False; 多命令工具用各 subcommand 注册为 command, no_args_is_help=True.
6. **legacy 工具不迁移**: emlmanager.py 是 Web 应用 (sqlite3+threading+HTTP), profiler.py 用 runpy.run_path, 均不适配 @px.tool DAG 模型. 保留 pf._run_legacy() 路由是更简方案.
## 验证结果
- ruff check: 0 errors
- ruff format: 96 files already formatted
- pyrefly: 0 errors (12 suppressed)
- pytest: 1003 passed
- coverage: 97.00% (门槛 95%)
## 遗留事项
-
-108
View File
@@ -1,108 +0,0 @@
# 文档整理与 Sphinx 文档搭建计划
## Context
最近完成 CLI 重构:新增 `pf` 统一入口,13 个工具迁移到 YAML 配置并删除了对应 .py 入口脚本,`run()` 的 verbose 统一应用到 spec。但文档未同步:README 仍引用旧命令(`yamlrun``python build.py`),模块结构表缺漏;`runner.py``_apply_verbose_to_graph` 成为死代码;项目缺少可发布的 Sphinx 文档。本次任务整理这些遗留,并搭建 ReadTheDocs 文档站。
## 任务范围
### 1. 清理死代码
- 删除 `src/pyflowx/runner.py``_apply_verbose_to_graph` 函数(line 38-68),功能已移入 `executors.py``run()`
- 删除 `tests/test_runner.py` 中对应测试(line 610-636`TestApplyVerboseToGraph` 类)。
- 清理 `runner.py` 顶部 `from dataclasses import replace` 若变为未使用。
### 2. 修复版本不一致
- `src/pyflowx/__init__.py:105` 硬编码 `__version__ = "0.4.5"``pyproject.toml:25``0.3.5`
- 统一为 `0.4.5``__init__.py` 为准,pyproject.toml 是源但 bumpversion 工具应同时更新两者)。
### 3. 更新 README.md
- L304-308`python build.py clean/build/test``pf pymake clean/build/test`
- L335-351、L435`yamlrun pipeline.yaml ...``pf yamlrun pipeline.yaml ...`6 处)。
- L311`verbose=True(默认)` 描述保留,但 CLI 示例改为 `pf`
- L558-574 模块结构表:补充 `cli/pf.py`(统一入口)、`cli/configs/`YAML 工具配置)、`cli/_ops/`(工具函数)、`profiling.py``registry.py`
- 顶部增加「文档」徽章链接到 ReadTheDocs。
### 4. 搭建 Sphinx 文档结构
新建 `docs/` 目录:
```
docs/
├── conf.py # Sphinx 配置
├── index.rst # 首页与目录
├── installation.rst # 安装
├── quickstart.rst # 快速上手(从 README 提炼)
├── guide/
│ ├── task.rst # TaskSpec 任务描述
│ ├── graph.rst # Graph DAG 构建
│ ├── execution.rst # 执行策略与 run()
│ ├── yaml.rst # YAML 任务编排
│ └── cli.rst # pf 统一入口与工具列表
├── api.rst # API 参考(automodule 自动生成)
└── changelog.rst # 变更日志摘要
```
**conf.py 要点**
- 扩展:`sphinx.ext.autodoc``sphinx.ext.napoleon`(支持 Google/NumPy docstring)、`sphinx.ext.viewcode``myst_parser`(支持 Markdown
- 主题:`sphinx_rtd_theme`
- 项目版本从 `pyflowx.__version__` 动态读取
- `autodoc_default_options``members: True, undoc-members: True, show-inheritance: True`
**api.rst**:用 `automodule:: pyflowx` 抓取 `__all__` 的 56 个公共符号。
### 5. ReadTheDocs 配置
- 新建 `.readthedocs.yaml`Python 3.11`pip install -e .[docs]``sphinx -b html docs/ docs/_build/`
- `.gitignore` 增加 `docs/_build/`
### 6. pyproject.toml 补充 docs 依赖
```toml
docs = [
"sphinx>=7.0",
"sphinx-rtd-theme>=2.0",
"myst-parser>=3.0",
]
```
并在 `[dependency-groups]` 的 dev 中加入 `pyflowx[docs]`
## 关键文件
| 文件 | 操作 |
|------|------|
| `src/pyflowx/runner.py` | 删除 `_apply_verbose_to_graph` |
| `tests/test_runner.py` | 删除 `TestApplyVerboseToGraph` |
| `src/pyflowx/__init__.py` | 版本统一(已 0.4.5,确认) |
| `pyproject.toml` | 版本 → 0.4.5;加 docs 依赖 |
| `README.md` | 更新 CLI 示例与模块结构表 |
| `docs/conf.py` | 新建 |
| `docs/*.rst` | 新建 |
| `.readthedocs.yaml` | 新建 |
| `.gitignore` | 加 docs/_build/ |
## 验证
1. **测试与 lint**
```bash
uv run pytest tests/ -q
uv run ruff check src/ tests/ docs/conf.py
uv run pyrefly check src/pyflowx/runner.py
```
2. **Sphinx 构建本地验证**
```bash
uv sync --extra docs
uv run sphinx-build -b html docs/ docs/_build/
```
确认无 warning,打开 `docs/_build/index.html` 检查页面。
3. **pf 功能回归**
```bash
pf gitt c
pf pymake b --dry-run
```
4. **RTD 配置校验**`.readthedocs.yaml` 语法正确,`docs/conf.py` 能独立构建。
## 不在范围
- 不统一各模块 docstring 风格(napoleon 兼容 Google/NumPy,够用)。
- 不重构现有 CLI 工具 YAML。
- 不新增中文文档翻译(文档用中文撰写,与项目既有风格一致)。
+23
View File
@@ -0,0 +1,23 @@
# 开发流程约束
本规则补充 `self-driven.md`,定义执行过程记录与规则变更的两条硬约束。
## 执行过程记录
- **每次迭代**(一个完整的"计划 → 实现 → 测试 → 文档 → 验证"闭环)必须记录到
`.trae/docs/` 目录,文件名形如 `iter-NN-<主题>.md``NN` 为零填充序号。
- 记录内容至少包括:本轮目标、改动文件清单、关键决策与依据、验证结果、遗留事项。
- **每 5 次迭代后**清理 `.trae/docs/`
- 把需要长期保留的整合性内容(可复用模式、踩坑总结、设计决策)
归档到 `.trae/skills/` 对应技能文档中;
- 其余过程性记录删除;
- docs 目录只保留最近未归档的迭代记录(即第 6 次迭代开始前,清理 1–5 号记录)。
- 归档动作需在迭代记录中标注"已归档至 skills/<文件>"。
## 规则变更约束
- 任何对 `.trae/rules/` 下文件的修改(新增、编辑、删除)**必须先询问用户**,
获得明确授权后方可变动;**未授权前不得擅自修改 rules**。
- 用户在对话中显式要求写入的规则内容(如"把 X 写入 rules")视为已授权,
可直接写入,写入后在回复中说明改了哪个文件、加了什么。
- 规则变更后需同步更新 `project_memory.md` 中的"开发约定"小节。
+19 -2
View File
@@ -73,6 +73,22 @@ alwaysApply: true
- 全套门禁通过:ruff、pyrefly、pytest、coverage。
- 给出本轮变更清单(改了哪些文件、为什么)。
## 多阶段项目
当项目按阶段(P0/P1/P2/...)划分时,初始确认的目标是**整个项目**,而非单个阶段。
- **阶段是里程碑,不是边界**:确认"推进 P1"只是指定下一步重点,不缩小整体目标范围;
也不意味着 P1 完成后就可以停下。整体目标 = 所有阶段交付完毕。
- **阶段完成不是停止条件**:某阶段验收标准全部满足后,**自动进入下一阶段的「计划」步骤**,
不停下来输出"是否继续"或"是否扩展范围"的询问。仅在阶段切换时用一句话说明
"进入 Pn,重点:…",然后继续自驱动迭代。
- **「收尾」仅适用于整个项目完成**:只有所有阶段(含最后一个)都交付后,才执行
「收尾」输出最终总结。单阶段完成 → 进入下一阶段的「计划」,不触发收尾。
- **跨阶段依赖**:若下一阶段依赖外部资源(如新依赖审批、环境配置),属"不可恢复的失败"
暂停条件;否则一律连续推进。
- **阶段范围超出预期**:执行中发现某阶段需要显著扩大范围或改变方向(如 P2 本来只做
表格抽取,发现必须先重写 IR),属"超出初始确认范围"暂停条件,需找用户确认。
## 暂停条件(仅在以下情况中断自驱动找用户)
1. **歧义无法自决**:需求存在多种合理解读且无既有约定可循。
@@ -81,7 +97,7 @@ alwaysApply: true
4. **超出初始确认范围**:用户目标在执行中发现需要显著扩大范围或改变方向。
5. **用户主动询问**:用户在对话中提出新问题或要求澄清。
**注意**"目标已达成"**不是**暂停条件——验收标准全部满足后直接进入收尾并结束任务,不询问"是否扩展范围"或"是否提交"
**注意**"目标已达成"**不是**暂停条件——但"目标已达成"指**整个项目目标**(所有阶段交付完毕),而非单个阶段验收满足。单阶段验收满足后应自动进入下一阶段(见「多阶段项目」),不触发收尾
非以上情况,一律继续自驱动,不要为"求确认"而暂停。
@@ -128,7 +144,8 @@ alwaysApply: true
## 收尾
- 验收标准全部满足后,**直接输出最终总结并结束任务**交付物、关键决策、遗留事项。
- **仅当整个项目所有阶段都交付后**,才执行收尾:直接输出最终总结并结束任务交付物、关键决策、遗留事项
- 单阶段完成**不属于收尾**——见「多阶段项目」,应自动进入下一阶段的「计划」。
- **自动提交**:收尾时自动 `git add`(按文件名)+ `git commit`(遵循 `.trae/rules/git-commit-message.md` 风格)+ `git push`(仅当分支已跟踪远程时执行;新分支跳过 push 并在总结中说明);**不询问**"是否需要提交"或"是否扩展范围"。
- 若验收标准未全部满足,回到「计划」继续下一轮,不停下询问。
- 将本次会话的关键产出与决策更新到 memory,便于后续会话续接。
+5 -31
View File
@@ -419,25 +419,6 @@ jobs:
| `tags` | `tags` | 自由标签 |
| `runs-on` | `tags`(追加) | 运行环境标签 |
## 示例
仓库 `examples/` 目录包含完整示例:
- [`etl_pipeline.py`](examples/etl_pipeline.py) —— ETL 流水线(sequential
- [`parallel_run.py`](examples/parallel_run.py) —— 并行执行对比(thread vs sequential
- [`async_aggregation.py`](examples/async_aggregation.py) —— 异步聚合 + Context 注入
- [`yaml_pipeline.yaml`](examples/yaml_pipeline.yaml) + [`yaml_pipeline.py`](examples/yaml_pipeline.py) —— YAML 声明式 CI/CD 流水线(矩阵扇出 + 条件执行)
运行:
```bash
python examples/etl_pipeline.py
python examples/parallel_run.py
python examples/async_aggregation.py
python -m pyflowx.examples.yaml_pipeline
pf yamlrun src/pyflowx/examples/yaml_pipeline.yaml --dry-run
```
## 断点续跑
```python
@@ -554,8 +535,8 @@ uv run pytest --cov=pyflowx --cov-fail-under=95
uv run mypy
# 代码风格
uv run ruff check src tests examples
uv run ruff format --check src tests examples
uv run ruff check src tests
uv run ruff format --check src tests
```
## 模块结构
@@ -574,25 +555,18 @@ uv run ruff format --check src tests examples
| `storage.py` | 状态后端:`MemoryBackend` / `JSONBackend`batch flush |
| `runner.py` | CLI 运行器:`CliRunner` |
| `report.py` | 运行结果:`RunReport` / `TaskResult` |
| `yaml_loader.py` | YAML 任务编排:GitHub Actions 风格 schema 解析(`load_yaml` / `parse_yaml_string` / `run_cli` |
| `registry.py` | 函数注册中心:`register_fn` / `get_fn` / `has_fn`YAML 的 `fn:` 引用) |
| `tools.py` | `@px.tool` 装饰器:`ToolSpec` / `run_tool` / `list_tools` / `list_subcommands` |
| `profiling.py` | 性能分析:`Profiler` 任务耗时统计 |
| `errors.py` | 错误家族:`PyFlowXError` 子类 |
| `ops/` | 工具函数(dev/files/media/system),被 YAML 的 `fn:` 引用 |
| `ops/` | CLI 工具模块集合(每个子模块用 `@px.tool` 注册一个工具) |
### CLI 工具
| 模块 | 职责 |
|------|------|
| `cli/pf.py` | 统一入口:`pf <tool> [command]`自动发现 `configs/*.yaml` 并路由 |
| `cli/configs/` | YAML 工具配置(gittool/filedate/pdftool 等 13 个工具的 `cli:` schema |
| `cli/pymake.py` | 构建工具(替代 Makefile),`pf pymake b` 调用 |
| `cli/yamlrun.py` | YAML pipeline 执行器,`pf yamlrun pipeline.yaml` 调用 |
| `cli/pf.py` | 统一入口:`pf <tool> [command]`按需导入 `ops/<tool>.py` 触发 `@px.tool` 注册 |
| `cli/profiler.py` | 性能分析 CLI |
| `cli/emlmanager.py` | 邮件管理 CLI |
| `cli/dev/` | 开发工具(dockercmd/envdev |
| `cli/llm/` | LLM 工具(msdownload/sglang |
| `cli/system/` | 系统工具(clearscreen/taskkill/which |
## 许可证
+9 -13
View File
@@ -50,21 +50,17 @@ API 参考
:members:
:undoc-members:
YAML 编排
---------
@px.tool 工具
-------------
.. autofunction:: pyflowx.load_yaml
.. autofunction:: pyflowx.parse_yaml_string
.. autofunction:: pyflowx.run_yaml
.. autofunction:: pyflowx.run_cli
.. autofunction:: pyflowx.build_cli_parser
.. autoclass:: pyflowx.ToolSpec
:members:
:undoc-members:
函数注册
--------
.. autofunction:: pyflowx.register_fn
.. autofunction:: pyflowx.get_fn
.. autofunction:: pyflowx.has_fn
.. autofunction:: pyflowx.tool
.. autofunction:: pyflowx.run_tool
.. autofunction:: pyflowx.list_tools
.. autofunction:: pyflowx.list_subcommands
命令执行
--------
+12 -24
View File
@@ -7,13 +7,11 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
"Programming Language :: Python :: 3.14",
"Programming Language :: Python :: 3.8",
"Programming Language :: Python :: 3.9",
"Topic :: Software Development :: Libraries :: Application Frameworks",
]
dependencies = [
"graphlib_backport >= 1.0.0; python_version < '3.9'",
"pyyaml>=6.0.1",
"rich>=13.7.0",
"typer>=0.24.0",
"typing-extensions>=4.13.2; python_version < '3.13'",
]
description = "Lightweight, type-safe DAG task scheduler with multi-strategy execution."
@@ -21,23 +19,13 @@ keywords = ["async", "dag", "scheduler", "task", "workflow"]
license = { text = "MIT" }
name = "pyflowx"
readme = "README.md"
requires-python = ">=3.8"
version = "0.4.6"
requires-python = ">=3.10"
version = "0.4.8"
[project.scripts]
dockercmd = "pyflowx.cli.dev.dockercmd:main"
emlman = "pyflowx.cli.emlmanager:main"
msdown = "pyflowx.cli.llm.msdownload:main"
pf = "pyflowx.cli.pf:main"
pxp = "pyflowx.cli.profiler:main"
sglang = "pyflowx.cli.llm.sglang:main"
yamlrun = "pyflowx.cli.yamlrun:main"
# dev
envdev = "pyflowx.cli.dev.envdev:main"
# system
clr = "pyflowx.cli.system.clearscreen:main"
taskk = "pyflowx.cli.system.taskkill:main"
wch = "pyflowx.cli.system.which:main"
emlman = "pyflowx.cli.emlmanager:main"
pf = "pyflowx.cli.pf:main"
pxp = "pyflowx.cli.profiler:main"
[project.optional-dependencies]
dev = [
@@ -54,7 +42,6 @@ dev = [
"ruff>=0.8.0",
"tox-uv>=1.13.1",
"tox>=4.25.0",
"types-PyYAML>=6.0.12",
]
docs = ["myst-parser>=3.0", "sphinx-rtd-theme>=2.0", "sphinx>=7.0"]
office = [
@@ -90,7 +77,7 @@ dev = ["pyflowx[dev,docs,office]"]
[tool.coverage.run]
branch = true
concurrency = ["thread"]
omit = ["src/pyflowx/cli/*", "src/pyflowx/examples/*", "tests/*"]
omit = ["src/pyflowx/cli/*", "tests/*"]
source = ["pyflowx"]
[tool.coverage.report]
@@ -110,7 +97,7 @@ markers = ["slow: marks tests as slow (deselect with
# Ruff 配置 - 与 .pre-commit-config.yaml 保持一致
[tool.ruff]
line-length = 120
target-version = "py38"
target-version = "py310"
[tool.ruff.lint]
ignore = [
@@ -143,9 +130,10 @@ select = [
]
[tool.ruff.lint.per-file-ignores]
"**/tests/**" = ["ARG001", "ARG002"]
"**/tests/**" = ["ARG001", "ARG002"]
"src/pyflowx/tools.py" = ["PLR0911"]
[tool.pyrefly]
preset = "strict"
project-includes = ["**/*.ipynb", "**/*.py*"]
python-version = "3.8"
python-version = "3.10"
+8 -22
View File
@@ -60,15 +60,7 @@ from __future__ import annotations
from .command import run_command
from .compose import GraphComposer, compose
from .conditions import (
IS_LINUX,
IS_MACOS,
IS_POSIX,
IS_WINDOWS,
BuiltinConditions,
Condition,
Constants,
)
from .conditions import IS_LINUX, IS_MACOS, IS_POSIX, IS_WINDOWS, BuiltinConditions, Condition, Constants
from .context import Context, build_call_args, describe_injection
from .errors import (
CycleError,
@@ -83,7 +75,6 @@ from .errors import (
from .executors import Strategy, run
from .graph import Graph, GraphDefaults
from .profiling import ProfileReport, TaskProfile
from .registry import FnRegistry, get_fn, has_fn, register_fn
from .report import RunReport
from .runner import CliExitCode, CliRunner
from .storage import JSONBackend, MemoryBackend, StateBackend
@@ -100,9 +91,9 @@ from .task import (
task,
task_template,
)
from .yaml_loader import YamlLoadError, build_cli_parser, load_yaml, parse_yaml_string, run_cli, run_yaml
from .tools import ToolSpec, list_subcommands, list_tools, run_tool, tool
__version__ = "0.4.6"
__version__ = "0.4.8"
__all__ = [
"IS_LINUX",
@@ -118,7 +109,6 @@ __all__ = [
"Context",
"CycleError",
"DuplicateTaskError",
"FnRegistry",
"Graph",
"GraphComposer",
"GraphDefaults",
@@ -142,21 +132,17 @@ __all__ = [
"TaskSpec",
"TaskStatus",
"TaskTimeoutError",
"YamlLoadError",
"ToolSpec",
"build_call_args",
"build_cli_parser",
"cmd",
"compose",
"describe_injection",
"get_fn",
"has_fn",
"load_yaml",
"parse_yaml_string",
"register_fn",
"list_subcommands",
"list_tools",
"run",
"run_cli",
"run_command",
"run_yaml",
"run_tool",
"task",
"task_template",
"tool",
]
View File
-26
View File
@@ -1,26 +0,0 @@
from __future__ import annotations
from typing import Literal
import pyflowx as px
DockerMirrorType = Literal["tencent"]
DOCKER_MIRROR_URLS: dict[DockerMirrorType, str] = {"tencent": "ccr.ccs.tencentyun.com"}
def main():
# parser = argparse.ArgumentParser(description="Docker 命令行工具")
# parser.add_argument("--username", nargs="?", default="", type=str, help="Docker 用户名")
# args = parser.parse_args()
tasks: list[px.TaskSpec] = [
px.cmd(["docker", "login", "--username", "xxx", DOCKER_MIRROR_URLS["tencent"]], name="docker_login_tencent"),
]
alias: dict[str, str | list[str | px.TaskSpec] | px.TaskSpec | px.Graph] = {
"login": "docker_login_tencent",
}
runner = px.CliRunner(strategy="sequential", tasks=tasks, aliases=alias)
runner.run_cli()
-366
View File
@@ -1,366 +0,0 @@
from __future__ import annotations
import argparse
import getpass
from pathlib import Path
from typing import Literal, get_args
import pyflowx as px
from pyflowx.conditions import BuiltinConditions
from pyflowx.tasks.system import setenv_group, write_file
# ============================================================================
# Mirror 配置
# ============================================================================
DOWNLOAD_MIRROR_SCRIPT: str = "curl -sSL https://linuxmirrors.cn/main.sh -o /tmp/linuxmirrors.sh"
INSTALL_MIRROR_SCRIPT: str = "sudo bash /tmp/linuxmirrors.sh"
# ============================================================================
# Python 配置
# ============================================================================
PyMirrorType = Literal["tsinghua", "aliyun", "huaweicloud", "ustc", "zju"]
PIP_INDEX_URLS: dict[PyMirrorType, str] = {
"tsinghua": "https://pypi.tuna.tsinghua.edu.cn/simple",
"aliyun": "https://mirrors.aliyun.com/pypi/simple/",
"huaweicloud": "https://mirrors.huaweicloud.com/repository/pypi/simple/",
"ustc": "https://pypi.mirrors.ustc.edu.cn/simple/",
"zju": "https://mirrors.zju.edu.cn/pypi/simple/",
}
PIP_TRUSTED_HOSTS: dict[PyMirrorType, str] = {
"tsinghua": "pypi.tuna.tsinghua.edu.cn",
"aliyun": "mirrors.aliyun.com",
"huaweicloud": "mirrors.huaweicloud.com",
"ustc": "pypi.mirrors.ustc.edu.cn",
"zju": "mirrors.zju.edu.cn",
}
PIP_CONFIG_PATH = Path.home() / ".pip" / "pip.conf" if BuiltinConditions.IS_LINUX() else Path.home() / "pip" / "pip.ini"
UV_INDEX_URLS = PIP_INDEX_URLS
UV_PYTHON_INSTALL_MIRROR: str = "https://registry.npmmirror.com/-/binary/python-build-standalone"
# ============================================================================
# Conda 配置
# ============================================================================
CondaMirrorType = Literal["tsinghua", "ustc", "bsfu", "aliyun"]
CONDA_MIRROR_URLS: dict[CondaMirrorType, list[str]] = {
"tsinghua": [
"https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/",
"https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free/",
"https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/r/",
"https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/msys2/",
"https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/pro/",
"https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/",
"https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/bioconda/",
"https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/menpo/",
"https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/pytorch/",
],
"ustc": [
"https://mirrors.ustc.edu.cn/anaconda/pkgs/main/",
"https://mirrors.ustc.edu.cn/anaconda/pkgs/free/",
"https://mirrors.ustc.edu.cn/anaconda/pkgs/r/",
"https://mirrors.ustc.edu.cn/anaconda/pkgs/msys2/",
"https://mirrors.ustc.edu.cn/anaconda/pkgs/pro/",
"https://mirrors.ustc.edu.cn/anaconda/pkgs/dev/",
"https://mirrors.ustc.edu.cn/anaconda/cloud/conda-forge/",
"https://mirrors.ustc.edu.cn/anaconda/cloud/bioconda/",
"https://mirrors.ustc.edu.cn/anaconda/cloud/menpo/",
"https://mirrors.ustc.edu.cn/anaconda/cloud/pytorch/",
],
"bsfu": [
"https://mirrors.bsfu.edu.cn/anaconda/pkgs/main/",
"https://mirrors.bsfu.edu.cn/anaconda/pkgs/free/",
"https://mirrors.bsfu.edu.cn/anaconda/pkgs/r/",
"https://mirrors.bsfu.edu.cn/anaconda/pkgs/msys2/",
"https://mirrors.bsfu.edu.cn/anaconda/pkgs/pro/",
"https://mirrors.bsfu.edu.cn/anaconda/pkgs/dev/",
"https://mirrors.bsfu.edu.cn/anaconda/cloud/conda-forge/",
"https://mirrors.bsfu.edu.cn/anaconda/cloud/bioconda/",
"https://mirrors.bsfu.edu.cn/anaconda/cloud/menpo/",
"https://mirrors.bsfu.edu.cn/anaconda/cloud/pytorch/",
],
"aliyun": [
"https://mirrors.aliyun.com/anaconda/pkgs/main/",
"https://mirrors.aliyun.com/anaconda/pkgs/free/",
"https://mirrors.aliyun.com/anaconda/pkgs/r/",
"https://mirrors.aliyun.com/anaconda/pkgs/msys2/",
"https://mirrors.aliyun.com/anaconda/pkgs/pro/",
"https://mirrors.aliyun.com/anaconda/pkgs/dev/",
"https://mirrors.aliyun.com/anaconda/cloud/conda-forge/",
"https://mirrors.aliyun.com/anaconda/cloud/bioconda/",
"https://mirrors.aliyun.com/anaconda/cloud/menpo/",
"https://mirrors.aliyun.com/anaconda/cloud/pytorch/",
],
}
CONDA_CONFIG_PATH = Path.home() / ".condarc"
# ============================================================================
# Qt 配置
# ============================================================================
QT_LIBS: list[str] = [
"build-essential",
"libgl1",
"libegl1",
"libglib2.0-0",
"libfontconfig1",
"libfreetype6",
"libxkbcommon0",
"libdbus-1-3",
"libxcb-xinerama0",
"libxcb-icccm4",
"libxcb-image0",
"libxcb-keysyms1",
"libxcb-randr0",
"libxcb-render-util0",
"libxcb-shape0",
"libxcb-xfixes0",
"libxcb-cursor0",
]
CHINESE_FONTS: list[str] = [
"fonts-noto-cjk",
"fonts-wqy-microhei",
"fonts-wqy-zenhei",
"fonts-noto-color-emoji",
]
# ============================================================================
# Rust 配置
# ============================================================================
RustMirrorType = Literal["tsinghua", "ustc", "aliyun"]
RustVersionType = Literal["stable", "nightly", "beta"]
DEFAULT_RUST_VERSION: RustVersionType = "stable"
DEFAULT_MIRROR: RustMirrorType = "tsinghua"
RUSTUP_MIRRORS: dict[RustMirrorType, dict[str, str]] = {
"tsinghua": {
"RUSTUP_DIST_SERVER": "https://mirrors.tuna.tsinghua.edu.cn/rustup",
"RUSTUP_UPDATE_ROOT": "https://mirrors.tuna.tsinghua.edu.cn/rustup/rustup",
"TOML_REGISTRY": "https://mirrors.tuna.tsinghua.edu.cn/crates.io-index/",
},
"aliyun": {
"RUSTUP_DIST_SERVER": "https://mirrors.aliyun.com/rustup",
"RUSTUP_UPDATE_ROOT": "https://mirrors.aliyun.com/rustup/rustup",
"TOML_REGISTRY": "https://mirrors.aliyun.com/crates.io-index/",
},
"ustc": {
"RUSTUP_DIST_SERVER": "https://mirrors.ustc.edu.cn/rust-static",
"RUSTUP_UPDATE_ROOT": "https://mirrors.ustc.edu.cn/rust-static/rustup",
"TOML_REGISTRY": "https://mirrors.ustc.edu.cn/crates.io-index/",
},
}
RUSTUP_DOWNLOAD_URL_LINUX = "https://mirrors.aliyun.com/repo/rust/rustup-init.sh"
RUSTUP_DOWNLOAD_URL_WINDOWS = "https://static.rust-lang.org/rustup/dist/x86_64-pc-windows-msvc/rustup-init.exe"
RUST_CONFIG_PATH = Path.home() / ".cargo" / "config.toml"
RUST_SCCACHE_DIR: Path = Path.home() / ".cargo" / "sccache"
RUST_SCCACHE_CACHE_SIZE: str = "20G"
def main() -> None:
"""主函数."""
parser = argparse.ArgumentParser(description="环境开发工具")
parser.add_argument(
"--python-mirror",
nargs="?",
type=str,
default="tsinghua",
choices=get_args(PyMirrorType),
help="Python 镜像源",
)
parser.add_argument(
"--conda-mirror",
nargs="?",
type=str,
default="tsinghua",
choices=get_args(CondaMirrorType),
help="Conda 镜镜像源",
)
parser.add_argument(
"--rust-mirror",
nargs="?",
type=str,
default=DEFAULT_MIRROR,
choices=get_args(RustMirrorType),
help="Rust 镜像源",
)
parser.add_argument(
"--rust-version",
nargs="?",
type=str,
default=DEFAULT_RUST_VERSION,
choices=get_args(RustVersionType),
help=f"Rust 版本, 推荐: {get_args(RustVersionType)}",
)
args = parser.parse_args()
python_mirror = args.python_mirror
conda_mirror_urls = CONDA_MIRROR_URLS[args.conda_mirror]
rust_mirror = args.rust_mirror
rust_version = args.rust_version
# 确保配置文件目录存在
PIP_CONFIG_PATH.parent.mkdir(parents=True, exist_ok=True)
CONDA_CONFIG_PATH.parent.mkdir(parents=True, exist_ok=True)
RUST_CONFIG_PATH.parent.mkdir(parents=True, exist_ok=True)
RUST_SCCACHE_DIR.mkdir(parents=True, exist_ok=True)
# 使用 conditions 自动控制任务执行
graph = px.Graph.from_specs(
[
# 系统镜像配置(仅 Linux 且未配置国内镜像)
px.TaskSpec(
"download_mirror",
cmd=DOWNLOAD_MIRROR_SCRIPT,
conditions=(
BuiltinConditions.IS_LINUX(),
BuiltinConditions.NOT(
BuiltinConditions.OR(
*[
BuiltinConditions.FILE_CONTENT_EXISTS(f, m)
for f in [
"/etc/apt/sources.list",
"/etc/apt/sources.list.d/ubuntu.sources",
]
for m in get_args(PyMirrorType)
],
)
),
),
verbose=True,
),
px.TaskSpec(
"install_mirror",
cmd=INSTALL_MIRROR_SCRIPT,
depends_on=("download_mirror",),
verbose=True,
),
# 安装 Qt 依赖(仅 Linux
px.TaskSpec(
"install_qt_libs",
cmd=["sudo", "apt", "install", "-y", *QT_LIBS],
conditions=(BuiltinConditions.IS_LINUX(),),
depends_on=("install_mirror",),
allow_upstream_skip=True,
verbose=True,
),
# 安装中文字体(仅 Linux
px.TaskSpec(
"install_fonts",
cmd=["sudo", "apt", "install", "-y", *CHINESE_FONTS],
conditions=(BuiltinConditions.IS_LINUX(),),
depends_on=("install_mirror",),
allow_upstream_skip=True,
verbose=True,
),
# 安装 Docker
px.TaskSpec(
"install_docker",
cmd=["sudo", "apt", "install", "-y", "docker-compose-v2"],
conditions=(BuiltinConditions.IS_LINUX(),),
depends_on=("install_mirror",),
allow_upstream_skip=True,
verbose=True,
),
px.TaskSpec(
"add_docker_group",
cmd=["sudo", "usermod", "-aG", "docker", getpass.getuser()],
conditions=(BuiltinConditions.IS_LINUX(),),
depends_on=("install_docker",),
allow_upstream_skip=True,
verbose=True,
),
px.TaskSpec(
"refresh_docker_group",
cmd=["newgrp", "docker"],
conditions=(BuiltinConditions.IS_LINUX(),),
depends_on=("add_docker_group",),
allow_upstream_skip=True,
verbose=True,
),
# 设置 Python 环境变量
*setenv_group(
{
"PIP_INDEX_URL": PIP_INDEX_URLS[python_mirror],
"PIP_TRUSTED_HOSTS": PIP_TRUSTED_HOSTS[python_mirror],
"UV_INDEX_URL": UV_INDEX_URLS[python_mirror],
"UV_PYTHON_INSTALL_MIRROR": UV_PYTHON_INSTALL_MIRROR,
"UV_HTTP_TIMEOUT": "600",
"UV_LINK_MODE": "copy",
}
),
# 写入 Python 配置(仅当未配置)
write_file(
str(PIP_CONFIG_PATH),
f"[global]\nindex-url = {PIP_INDEX_URLS[python_mirror]}\ntrusted-host = {PIP_TRUSTED_HOSTS[python_mirror]}",
),
# 写入 Conda 配置(仅当未配置)
write_file(
str(CONDA_CONFIG_PATH),
"show_channel_urls: true\nchannels:\n - " + "\n - ".join(conda_mirror_urls) + "\n - defaults",
),
# 设置 Rust 镜像源
*setenv_group(
{
"RUSTUP_DIST_SERVER": RUSTUP_MIRRORS[rust_mirror]["RUSTUP_DIST_SERVER"],
"RUSTUP_UPDATE_ROOT": RUSTUP_MIRRORS[rust_mirror]["RUSTUP_UPDATE_ROOT"],
"RUST_SCCACHE_DIR": str(RUST_SCCACHE_DIR),
"RUST_SCCACHE_CACHE_SIZE": RUST_SCCACHE_CACHE_SIZE,
}
),
# 写入 Rust 配置(仅当未配置)
write_file(
str(RUST_CONFIG_PATH),
f"""
[source.crates-io]
replace-with = '{rust_mirror}'
[source.{rust_mirror}]
registry = "sparse+{RUSTUP_MIRRORS[rust_mirror]["TOML_REGISTRY"]}"
[registries.{rust_mirror}]
index = "sparse+{RUSTUP_MIRRORS[rust_mirror]["TOML_REGISTRY"]}"
""",
),
# 下载 Rustup 安装脚本
px.TaskSpec(
"download_rustup",
cmd=["curl", "-fsSL", RUSTUP_DOWNLOAD_URL_LINUX, "-o", "rustup-init.sh"],
conditions=(
BuiltinConditions.IS_LINUX(),
BuiltinConditions.NOT(BuiltinConditions.HAS_INSTALLED("rustup")),
),
verbose=True,
),
px.TaskSpec(
"download_rustup_win",
cmd=[
"powershell",
"-Command",
"Invoke-WebRequest",
"-Uri",
RUSTUP_DOWNLOAD_URL_WINDOWS,
"-OutFile",
"rustup-init.exe",
],
conditions=(
BuiltinConditions.IS_WINDOWS(),
BuiltinConditions.NOT(BuiltinConditions.HAS_INSTALLED("rustup")),
),
verbose=True,
),
# 安装 Rust 工具链
px.TaskSpec(
"install_rust",
cmd=["rustup", "toolchain", "install", rust_version],
conditions=(BuiltinConditions.HAS_INSTALLED("rustup"),),
depends_on=("setenv_rustup_dist_server",),
allow_upstream_skip=True,
verbose=True,
),
]
)
px.run(graph, strategy="thread", verbose=True)
+3 -3
View File
@@ -154,7 +154,7 @@ class EmailDatabase:
cursor.execute(query, (f"%{keyword}%", f"%{keyword}%", f"%{keyword}%", f"%{keyword}%", limit, offset))
columns = [description[0] for description in cursor.description]
return [dict(zip(columns, row)) for row in cursor.fetchall()]
return [dict(zip(columns, row, strict=False)) for row in cursor.fetchall()]
def get_grouped_emails(self) -> dict[str, list[dict[str, Any]]]:
"""获取按主题分组的邮件."""
@@ -165,7 +165,7 @@ class EmailDatabase:
cursor.execute(f"SELECT * FROM {TABLE_NAME} ORDER BY subject, date_parsed DESC")
columns = [description[0] for description in cursor.description]
emails = [dict(zip(columns, row)) for row in cursor.fetchall()]
emails = [dict(zip(columns, row, strict=False)) for row in cursor.fetchall()]
# 按主题分组
grouped: dict[str, list[dict[str, Any]]] = {}
@@ -602,7 +602,7 @@ class EmlManagerHandler(BaseHTTPRequestHandler):
self._send_json_response({"error": "邮件不存在"}, 404)
return
email_data = dict(zip(columns, row))
email_data = dict(zip(columns, row, strict=False))
self._send_json_response({"email": email_data})
def _api_get_grouped_emails(self) -> None:
View File
-43
View File
@@ -1,43 +0,0 @@
"""Download from ModelScopeHub."""
import argparse
from pathlib import Path
from typing import Literal, get_args
import pyflowx as px
DownloadType = Literal["model", "dataset", "space"]
def main():
parser = argparse.ArgumentParser(description="Download a model from ModelScopeHub.")
parser.add_argument("name", help="Target name.")
parser.add_argument("--type", "-t", nargs="?", default="model", choices=get_args(DownloadType), help="Target type.")
parser.add_argument("--dir", default=None, help="Download directory.")
args = parser.parse_args()
if not args.name:
parser.error("name is required")
download_dir: Path = Path(args.dir) if args.dir else Path.home() / ".models" / args.name.split("/")[-1]
download_dir.mkdir(parents=True, exist_ok=True)
graph = px.Graph.from_specs(
[
px.TaskSpec(
name="download",
cmd=[
"uvx",
"modelscope",
"download",
f"--{args.type}",
args.name,
"--local_dir",
str(download_dir),
],
verbose=True,
),
]
)
px.run(graph, strategy="thread", verbose=True)
-65
View File
@@ -1,65 +0,0 @@
"""使用 SGLang 运行本地模型."""
import argparse
from pathlib import Path
import pyflowx as px
from pyflowx.conditions import BuiltinConditions, Constants
def main():
parser = argparse.ArgumentParser(description="启动 SGLang 服务")
parser.add_argument("--model", default="~/.models/Qwen2.5-Coder-32B-Instruct-AWQ", help="模型路径")
parser.add_argument("--port", type=int, default=8000, help="服务端口")
parser.add_argument("--ctx-len", type=int, default=28672, help="最大上下文长度")
parser.add_argument("--mem", type=float, default=0.75, help="显存占比 (0-1)")
parser.add_argument("--host", default="0.0.0.0", help="主机地址")
parser.add_argument("--log-level", default="info", help="日志级别")
args = parser.parse_args()
if not args.model:
parser.error("model is required")
model_dir = Path(args.model).expanduser()
if not model_dir.exists():
parser.error(f"Model directory {model_dir} does not exist.")
graph = px.Graph.from_specs(
[
px.TaskSpec(
name="download",
cmd=[
"uv",
"install",
"sglang[all]",
],
conditions=(BuiltinConditions.NOT(BuiltinConditions.HAS_INSTALLED("sglang")),),
verbose=True,
),
px.TaskSpec(
name="run",
cmd=[
"python" if Constants.IS_WINDOWS else "python3",
"-m",
"sglang.launch_server",
"--model-path",
str(model_dir),
"--host",
str(args.host),
"--port",
"8000",
"--mem-fraction-static",
str(args.mem),
"--context-length",
"32768",
"--tool-call-parser",
"qwen",
"--log-level",
str(args.log_level),
],
verbose=True,
),
]
)
px.run(graph, strategy="sequential", verbose=True)
+122 -59
View File
@@ -1,7 +1,7 @@
"""PyFlowX 统一 CLI 入口.
通过 ``pf <tool> [command] [options]`` 调用所有工具,
工具定义在 ``configs/`` 目录下的 YAML 文件中.
工具定义在 ``pyflowx.ops`` 子包中, 每个模块用 ``@px.tool`` 装饰器注册.
用法
----
@@ -13,29 +13,40 @@
from __future__ import annotations
import contextlib
import difflib
import importlib
import sys
from pathlib import Path
from typing import Sequence
from collections.abc import Sequence
import pyflowx as px
from rich.console import Console
from rich.panel import Panel
from rich.table import Table
from rich.text import Text
from pyflowx import __version__
from pyflowx.tools import _TOOL_REGISTRY, run_tool
class PfApp:
"""pf 统一入口应用.
路由 ``pf <tool> [command]`` 到 YAML 配置工具或传统 Python 工具.
路由 ``pf <tool> [command]`` 到 ``@px.tool`` 注册的工具或传统 Python 工具.
"""
_CONFIGS_DIR = Path(__file__).parent.parent / "configs"
# 工具名到 YAML 配置文件的映射 (支持短别名)
# 工具名到 ops 模块名的映射 (支持短别名)
_TOOL_ALIASES: dict[str, str] = {
"autofmt": "autofmt",
"af": "autofmt",
"bump": "bumpversion",
"bumpversion": "bumpversion",
"bv": "bumpversion",
"clr": "clr",
"clearscreen": "clr",
"dockercmd": "dockercmd",
"docker": "dockercmd",
"envdev": "envdev",
"env": "envdev",
"filedate": "filedate",
"fd": "filedate",
"filelevel": "filelevel",
@@ -52,6 +63,9 @@ class PfApp:
"gt": "gittool",
"ls": "lscalc",
"lscalc": "lscalc",
"msdown": "msdownload",
"msdownload": "msdownload",
"msd": "msdownload",
"pack": "packtool",
"packtool": "packtool",
"pk": "packtool",
@@ -68,22 +82,30 @@ class PfApp:
"screenshot": "screenshot",
"scrcap": "screenshot",
"ss": "screenshot",
"sglang": "sglang",
"sg": "sglang",
"ssh": "sshcopyid",
"sshcopy": "sshcopyid",
"sshcopyid": "sshcopyid",
"sc": "sshcopyid",
"taskk": "taskkill",
"taskkill": "taskkill",
"tk": "taskkill",
"wch": "which",
"which": "which",
}
# 传统工具: 有自己的 main() 函数 (无法 YAML 化的复杂逻辑)
# 传统工具: 有自己的 main() 函数 (无法 @px.tool 化的复杂逻辑)
_LEGACY_TOOLS: dict[str, str] = {
"emlman": "pyflowx.cli.emlmanager:main",
"profiler": "pyflowx.cli.profiler:main",
"pxp": "pyflowx.cli.profiler:main",
"yamlrun": "pyflowx.cli.yamlrun:main",
}
def __init__(self, argv: Sequence[str] | None = None) -> None:
self._argv = list(argv) if argv is not None else sys.argv[1:]
self._console = Console()
self._err = Console(stderr=True)
def run(self) -> int:
"""主入口, 返回退出码."""
@@ -91,64 +113,101 @@ class PfApp:
self._list_tools()
return 0
tool_name = self._argv[0]
rest_argv = self._argv[1:]
first = self._argv[0]
resolved = self._resolve_tool(tool_name)
if first in ("--help", "-h"):
self._list_tools()
return 0
if first in ("--version", "-V"):
self._console.print(f"PyFlowX [bold cyan]{__version__}[/bold cyan]")
return 0
rest = self._argv[1:]
resolved = self._resolve_tool(first)
if resolved is None:
print(f"错误: 未知工具 '{tool_name}'", file=sys.stderr)
print("运行 'pf' 查看可用工具列表", file=sys.stderr)
self._print_unknown_tool(first)
return 1
tool_type, target = resolved
if tool_type == "legacy":
return self._run_legacy(target, rest_argv)
return self._run_yaml(target, rest_argv)
kind, target = resolved
if kind == "legacy":
return self._run_legacy(target, rest)
return self._run_tool(target, rest)
# ------------------------------------------------------------------ #
# 工具列表 (rich)
# ------------------------------------------------------------------ #
def _list_tools(self) -> None:
"""列出所有可用工具."""
print("PyFlowX 工具列表:")
print()
print("YAML 配置工具:")
yaml_tools = sorted(set(self._TOOL_ALIASES.values()))
for tool in yaml_tools:
print(f" pf {tool:<15} - {self._tool_description(tool)}")
print()
print("传统工具:")
for tool in sorted(self._LEGACY_TOOLS):
print(f" pf {tool:<15}")
print()
print("示例:")
print(" pf filedate add a.txt")
print(" pf pymake b")
"""rich 表格列出所有可用工具."""
self._console.print(
Panel(
Text(f"PyFlowX v{__version__}", style="bold cyan", justify="center"),
subtitle="[dim]pf <tool> [command] [options][/dim]",
)
)
table = Table(title="@px.tool 工具", show_header=True, header_style="bold", show_lines=False)
table.add_column("命令", style="cyan", no_wrap=True)
table.add_column("别名", style="dim", no_wrap=True)
table.add_column("说明")
for tool in sorted(set(self._TOOL_ALIASES.values())):
aliases = self._aliases_for(tool)
table.add_row(f"pf {tool}", ", ".join(aliases), self._tool_description(tool))
self._console.print(table)
if self._LEGACY_TOOLS:
legacy = Table(title="传统工具", show_header=True, header_style="bold", show_lines=False)
legacy.add_column("命令", style="cyan", no_wrap=True)
for tool in sorted(self._LEGACY_TOOLS):
legacy.add_row(f"pf {tool}")
self._console.print(legacy)
self._console.print("\n[bold]示例:[/bold]")
self._console.print(" [cyan]pf filedate add a.txt[/cyan] # 给文件添加日期前缀")
self._console.print(" [cyan]pf pymake b[/cyan] # 构建 Python 包")
self._console.print(" [cyan]pf gitt c[/cyan] # 清理并查看 git 状态")
def _aliases_for(self, canonical: str) -> list[str]:
"""获取工具的别名 (不含规范名本身)."""
return sorted(a for a, t in self._TOOL_ALIASES.items() if t == canonical and a != canonical)
def _tool_description(self, tool_name: str) -> str:
"""获取工具描述 (从 YAML cli.description)."""
config_path = self._CONFIGS_DIR / f"{tool_name}.yaml"
if not config_path.exists():
return ""
try:
import yaml
"""获取工具描述 (从 _TOOL_REGISTRY 中已注册 ToolSpec 的 description/help)."""
with contextlib.suppress(ImportError):
importlib.import_module(f"pyflowx.ops.{tool_name}")
data = yaml.safe_load(config_path.read_text(encoding="utf-8"))
if isinstance(data, dict) and isinstance(data.get("cli"), dict):
return str(data["cli"].get("description", ""))
except Exception:
pass
if tool_name not in _TOOL_REGISTRY:
return ""
subs = _TOOL_REGISTRY[tool_name]
for spec in subs.values():
if spec.description:
return spec.description
for spec in subs.values():
if not spec.hidden and spec.help:
return spec.help
return ""
# ------------------------------------------------------------------ #
# 路由
# ------------------------------------------------------------------ #
def _resolve_tool(self, name: str) -> tuple[str, str] | None:
"""解析工具名, 返回 (类型, 目标).
类型: "yaml""legacy"
目标: YAML 文件名 (不含 .yaml) 或 legacy 模块路径
"""
"""解析工具名, 返回 (类型, 目标)."""
if name in self._TOOL_ALIASES:
return ("yaml", self._TOOL_ALIASES[name])
return ("tool", self._TOOL_ALIASES[name])
if name in self._LEGACY_TOOLS:
return ("legacy", self._LEGACY_TOOLS[name])
return None
def _print_unknown_tool(self, name: str) -> None:
"""打印未知工具错误 + 模糊匹配建议."""
self._err.print(f"[red]错误:[/red] 未知工具 [yellow]{name!r}[/yellow]")
suggestions = difflib.get_close_matches(name, list(self._TOOL_ALIASES), n=3, cutoff=0.5)
if suggestions:
self._err.print(f"[dim]是否想用: {', '.join(suggestions)}[/dim]")
self._err.print("[dim]运行 'pf' 查看可用工具列表[/dim]")
def _run_legacy(self, module_path: str, argv: list[str]) -> int:
"""运行传统工具的 main() 函数."""
module_name, func_name = module_path.split(":", 1)
@@ -165,16 +224,20 @@ class PfApp:
finally:
sys.argv = original_argv
def _run_yaml(self, target: str, argv: list[str]) -> int:
"""运行 YAML 配置工具."""
config_path = self._CONFIGS_DIR / f"{target}.yaml"
if not config_path.exists():
print(f"错误: 未找到配置文件 '{config_path}'", file=sys.stderr)
print("运行 'pf' 查看可用工具列表", file=sys.stderr)
def _run_tool(self, target: str, argv: list[str]) -> int:
"""运行 @px.tool 工具.
导入 ``pyflowx.ops.<target>`` 触发 ``@px.tool`` 注册, 然后调用 ``run_tool``.
"""
with contextlib.suppress(ImportError):
importlib.import_module(f"pyflowx.ops.{target}")
if target not in _TOOL_REGISTRY:
self._err.print(f"[red]错误:[/red] 未找到工具 [yellow]{target!r}[/yellow]")
self._err.print("[dim]运行 'pf' 查看可用工具列表[/dim]")
return 1
print(f"运行配置文件 '{config_path}'")
return px.run_cli(config_path, argv)
return run_tool(target, argv)
def main() -> None:
View File
-15
View File
@@ -1,15 +0,0 @@
"""清屏工具.
跨平台清屏工具, 支持终端和控制台清屏.
"""
from __future__ import annotations
import pyflowx as px
from pyflowx.tasks.system import clr
def main() -> None:
"""清屏工具主函数."""
graph = px.Graph.from_specs([clr()])
px.run(graph, strategy="thread")
-40
View File
@@ -1,40 +0,0 @@
"""进程终止工具.
跨平台进程终止工具, 支持按名称终止进程.
用法: taskkill proc_name [proc_name ...]
"""
from __future__ import annotations
import argparse
import pyflowx as px
from pyflowx.conditions import Constants
def main() -> None:
"""进程终止工具主函数."""
parser = argparse.ArgumentParser(
description="TaskKill - 进程终止工具",
usage="taskkill <process_name> [process_name ...]",
)
parser.add_argument(
"process_names",
type=str,
nargs="+",
help="进程名称 (如: chrome.exe python node)",
)
args = parser.parse_args()
if Constants.IS_WINDOWS:
cmd = ["taskkill", "/f", "/im"]
else:
cmd = ["pkill", "-f"]
graph = px.Graph.from_specs(
[
px.TaskSpec(f"kill_{proc_name}", cmd=[*cmd, f"{proc_name}*"], verbose=True)
for proc_name in args.process_names
],
)
px.run(graph, strategy="thread")
-21
View File
@@ -1,21 +0,0 @@
"""命令查找工具.
跨平台查找可执行命令路径, 类似 Unix 的 which 命令.
"""
from __future__ import annotations
import argparse
import pyflowx as px
from pyflowx.tasks.system import which
def main() -> None:
"""命令查找工具主函数."""
parser = argparse.ArgumentParser(description="Which - 命令查找工具")
parser.add_argument("commands", nargs="+", help="要查找的命令名称, 如: python ls ps gcc...")
args = parser.parse_args()
graph = px.Graph.from_specs([which(cmd) for cmd in args.commands])
px.run(graph, strategy="thread")
-109
View File
@@ -1,109 +0,0 @@
"""YAML 任务编排执行工具.
从 YAML 文件加载 GitHub Actions 风格的任务图并执行.
支持串并行编排、矩阵扇出、条件执行等 CI/CD 核心概念.
用法
----
yamlrun pipeline.yaml # 执行 YAML 任务图
yamlrun pipeline.yaml --strategy thread # 指定执行策略
yamlrun pipeline.yaml --dry-run # 仅打印任务分层, 不执行
yamlrun pipeline.yaml --list # 列出所有任务名
yamlrun pipeline.yaml --quiet # 静默模式
示例 YAML
----------
::
strategy: thread
jobs:
setup:
cmd: ["git", "clone", "https://github.com/foo/bar"]
build:
needs: [setup]
cmd: ["python", "-m", "build"]
test:
needs: [build]
cmd: ["pytest"]
strategy:
matrix:
python: ["3.8", "3.9"]
"""
from __future__ import annotations
import argparse
import sys
from pathlib import Path
from typing import cast
import pyflowx as px
from pyflowx.executors import Strategy
def main() -> None:
"""YAML 任务编排执行工具主函数."""
parser = argparse.ArgumentParser(
description="YamlRun - 从 YAML 文件加载并执行任务图",
usage="yamlrun <file.yaml> [--strategy STRATEGY] [--dry-run] [--list] [--quiet]",
)
parser.add_argument("file", type=str, help="YAML 任务图文件路径")
parser.add_argument(
"--strategy",
type=str,
default=None,
help="执行策略: sequential/thread/async/dependency (默认: YAML 中指定的策略或 dependency)",
)
parser.add_argument("--dry-run", action="store_true", help="仅打印任务分层, 不执行")
parser.add_argument("--list", action="store_true", help="列出所有任务名后退出")
parser.add_argument("--quiet", action="store_true", help="静默模式, 不打印详细输出")
args = parser.parse_args()
file_path = Path(args.file)
if not file_path.exists():
print(f"错误: 文件不存在: {file_path}", file=sys.stderr)
sys.exit(1)
try:
graph = px.Graph.from_yaml(file_path)
except px.YamlLoadError as e:
print(f"错误: YAML 加载失败: {e}", file=sys.stderr)
sys.exit(1)
if args.list:
print("任务列表:")
for name in graph.names:
spec = graph.spec(name)
deps = ", ".join(spec.depends_on) if spec.depends_on else "(无依赖)"
print(f" - {name} (依赖: {deps})")
sys.exit(0)
layers = graph.layers()
print(f"任务分层 ({len(layers)} 层):")
for i, layer in enumerate(layers):
print(f"{i + 1}: {layer}")
if args.dry_run:
print("\n[dry-run] 跳过执行")
sys.exit(0)
strategy = args.strategy or graph.defaults.strategy or "dependency"
print(f"\n执行策略: {strategy}")
print(f"任务总数: {len(graph.names)}")
print("-" * 40)
report = px.run(graph, strategy=cast(Strategy, strategy), verbose=not args.quiet)
print("-" * 40)
succeeded = report.succeeded_tasks()
failed = report.failed_tasks()
skipped = report.skipped_tasks()
print(f"完成: {len(succeeded)} 成功 / {len(failed)} 失败 / {len(skipped)} 跳过 (共 {len(graph.names)})")
if failed:
print(f"失败任务: {failed}")
sys.exit(1)
if __name__ == "__main__":
main()
+12 -7
View File
@@ -10,12 +10,16 @@ from __future__ import annotations
import os
import subprocess
from typing import Any, List, Union, cast
from typing import Any, cast
from rich.console import Console
from .task import TaskSpec
__all__ = ["run_command"]
_cmd_console = Console()
def run_command(spec: TaskSpec[Any]) -> Any: # noqa: PLR0912
"""执行 ``spec.cmd`` 指定的命令(list / shell 字符串 / 可调用对象)。
@@ -39,9 +43,9 @@ def run_command(spec: TaskSpec[Any]) -> Any: # noqa: PLR0912
if callable(cmd) and not isinstance(cmd, (list, str)):
name = getattr(cmd, "__name__", "callable")
if verbose:
print(f"[verbose] 执行可调用命令: {name}", flush=True)
_cmd_console.print(f"[cyan]▸[/cyan] 执行可调用命令: [bold]{name}[/bold]")
if cwd is not None:
print(f"[verbose] 工作目录: {cwd}", flush=True)
_cmd_console.print(f" [dim]工作目录: {cwd}[/dim]")
try:
return cmd()
except Exception as e:
@@ -58,9 +62,9 @@ def run_command(spec: TaskSpec[Any]) -> Any: # noqa: PLR0912
label = "Shell 命令"
if verbose:
print(f"[verbose] {verb}: {cmd_str}", flush=True)
_cmd_console.print(f"[cyan]▸[/cyan] {verb}: [bold]{cmd_str}[/bold]")
if cwd is not None:
print(f"[verbose] 工作目录: {cwd}", flush=True)
_cmd_console.print(f" [dim]工作目录: {cwd}[/dim]")
# 合并环境变量
run_env: dict[str, str] | None = None
@@ -70,7 +74,7 @@ def run_command(spec: TaskSpec[Any]) -> Any: # noqa: PLR0912
try:
result = subprocess.run(
cast(Union[str, List[str]], cmd),
cast(str | list[str], cmd),
shell=not is_list,
cwd=cwd,
env=run_env,
@@ -87,7 +91,8 @@ def run_command(spec: TaskSpec[Any]) -> Any: # noqa: PLR0912
raise RuntimeError(f"{label}执行异常: {cmd_str}: {e}") from e
if verbose:
print(f"[verbose] 返回码: {result.returncode}", flush=True)
style = "green" if result.returncode == 0 else "red"
_cmd_console.print(f"[{style}]返回码: {result.returncode}[/{style}]")
if result.returncode == 0:
if not verbose and result.stdout:
+2 -1
View File
@@ -16,8 +16,9 @@ import os
import shutil
import subprocess
import sys
from collections.abc import Callable
from pathlib import Path
from typing import Any, Callable
from typing import Any
from .task import Condition, Context
-65
View File
@@ -1,65 +0,0 @@
# autofmt - 自动格式化工具
# 用法:
# pf autofmt fmt --target .
# pf autofmt lint --target .
# pf autofmt lint --target . --fix
# pf autofmt doc --root-dir .
# pf autofmt sync --root-dir .
strategy: thread
variables:
TARGET: "."
ROOT_DIR: "."
FIX: false
cli:
description: "AutoFmt - 自动格式化工具"
usage: "pf autofmt <command> [options]"
subcommands:
fmt:
help: "格式化代码"
options:
- name: TARGET
flag: "--target"
type: str
default: "."
help: "目标路径 (默认: .)"
lint:
help: "代码检查"
options:
- name: TARGET
flag: "--target"
type: str
default: "."
help: "目标路径 (默认: .)"
- name: FIX
flag: "--fix"
action: "store_true"
help: "自动修复问题"
doc:
help: "自动添加文档字符串"
options:
- name: ROOT_DIR
flag: "--root-dir"
type: str
default: "."
help: "根目录 (默认: .)"
sync:
help: "同步 pyproject 配置"
options:
- name: ROOT_DIR
flag: "--root-dir"
type: str
default: "."
help: "根目录 (默认: .)"
jobs:
fmt:
cmd: ["ruff", "format", "${TARGET}"]
lint:
cmd: ["ruff", "check", "${TARGET}"]
lint_fix:
cmd: ["ruff", "check", "--fix", "--unsafe-fixes", "${TARGET}"]
doc:
fn: auto_add_docstrings
args: ["${ROOT_DIR}"]
sync:
fn: sync_pyproject_config
args: ["${ROOT_DIR}"]
-27
View File
@@ -1,27 +0,0 @@
# bumpversion - 版本号自动管理工具
# 用法:
# pf bumpversion
# pf bumpversion minor --no-tag
strategy: sequential
variables:
PART: patch
NO_TAG: false
cli:
description: "BumpVersion - 版本号自动管理工具"
usage: "pf bumpversion [part] [options]"
positional:
- name: PART
type: str
default: patch
help: "版本部分: patch, minor, major"
options:
- name: NO_TAG
flag: "--no-tag"
action: "store_true"
help: "提交后不创建 git tag"
jobs:
bump:
fn: bump_project_version
args: ["${PART}"]
kwargs:
no_tag: ${NO_TAG}
-36
View File
@@ -1,36 +0,0 @@
# filedate - 文件日期处理工具
# 用法:
# pf filedate add file1.txt file2.txt
# pf filedate clear file1.txt file2.txt
strategy: thread
variables:
FILES: []
cli:
description: "FileDate - 文件日期处理工具"
usage: "pf filedate <command> [files...]"
subcommands:
add:
help: "添加日期前缀"
positional:
- name: FILES
nargs: "+"
type: path
help: "文件路径"
clear:
help: "清除日期前缀"
positional:
- name: FILES
nargs: "+"
type: path
help: "文件路径"
jobs:
add:
fn: process_files_date
args: ["${FILES}"]
kwargs:
clear: false
clear:
fn: process_files_date
args: ["${FILES}"]
kwargs:
clear: true
-28
View File
@@ -1,28 +0,0 @@
# filelevel - 文件等级重命名工具
# 用法:
# pf filelevel set file.txt --level 2
strategy: thread
variables:
FILES: []
LEVEL: 0
cli:
description: "FileLevel - 文件等级重命名工具"
usage: "pf filelevel <command> [files...] [options]"
subcommands:
set:
help: "设置文件等级"
positional:
- name: FILES
nargs: "+"
type: path
help: "文件路径"
options:
- name: LEVEL
flag: "--level"
type: int
required: true
help: "文件等级 (0-4)"
jobs:
set:
fn: process_files_level
args: ["${FILES}", "${LEVEL}"]
-34
View File
@@ -1,34 +0,0 @@
# folderback - 文件夹备份工具
# 用法:
# pf folderback
# pf folderback --src ./project --dst ./backup --max-zip 10
strategy: thread
variables:
SRC: "."
DST: "./backup"
MAX_ZIP: 5
cli:
description: "FolderBack - 文件夹备份工具"
usage: "pf folderback [options]"
options:
- name: SRC
flag: "--src"
type: str
default: "."
help: "源文件夹路径 (默认: 当前目录)"
- name: DST
flag: "--dst"
type: str
default: "./backup"
help: "目标文件夹路径 (默认: ./backup)"
- name: MAX_ZIP
flag: "--max-zip"
type: int
default: 5
help: "最大备份数量 (默认: 5)"
jobs:
backup:
fn: backup_folder
args: ["${SRC}", "${DST}"]
kwargs:
max_zip: ${MAX_ZIP}
-21
View File
@@ -1,21 +0,0 @@
# folderzip - 文件夹压缩工具
# 用法:
# pf folderzip
# pf folderzip --cwd ./project
strategy: thread
variables:
CWD: "."
cli:
description: "FolderZip - 文件夹压缩工具"
usage: "pf folderzip [options]"
options:
- name: CWD
flag: "--cwd"
type: str
required: false
default: "."
help: "工作目录 (默认: 当前目录)"
jobs:
zip:
fn: zip_folders
args: ["${CWD}"]
-51
View File
@@ -1,51 +0,0 @@
# gittool - Git 执行工具
# 用法:
# pf gittool a
# pf gittool c
# pf gittool i
# pf gittool isub
# pf gittool p
# pf gittool pl
strategy: thread
variables:
# git clean -e 参数列表 (展开为 cmd 数组元素)
CLEAN_EXCLUDES: ["-e", ".venv", "-e", ".tox", "-e", ".pytest_cache",
"-e", ".ruff_cache", "-e", "node_modules",
"-e", ".idea", "-e", ".vscode",
"-e", ".trae", "-e", ".qoder",
"-e", ".editorconfig", "-e", "idea.config",
"-e", "idea_modules.xml", "-e", "vcs.xml"]
cli:
description: "GitTool - Git 执行工具"
usage: "pf gittool <command>"
subcommands:
a:
help: "添加并提交"
c:
help: "清理并查看状态"
i:
help: "初始化并提交"
isub:
help: "初始化子目录"
p:
help: "推送"
pl:
help: "拉取"
jobs:
a:
fn: git_add_commit
args: ["chore: update"]
clean:
cmd: ["git", "clean", "-xfd", "${CLEAN_EXCLUDES}"]
c:
needs: [clean]
cmd: ["git", "status", "--porcelain"]
i:
fn: git_init_add_commit
args: ["init commit"]
isub:
fn: init_sub_dirs
p:
cmd: ["git", "push"]
pl:
cmd: ["git", "pull"]
-51
View File
@@ -1,51 +0,0 @@
# lscalc - LS-DYNA 计算工具
# 用法:
# pf lscalc run input.k --ncpu 4
# pf lscalc status
strategy: thread
variables:
INPUT_FILE: input.k
NCPU: 4
cli:
description: "LSCalc - LS-DYNA 计算工具"
usage: "pf lscalc <command> [options]"
subcommands:
run:
help: "运行 LS-DYNA 计算"
positional:
- name: INPUT_FILE
type: str
help: "输入文件路径"
options:
- name: NCPU
flag: "--ncpu"
type: int
default: 4
help: "CPU 核心数 (默认: 4)"
mpi:
help: "运行 LS-DYNA MPI 计算"
positional:
- name: INPUT_FILE
type: str
help: "输入文件路径"
options:
- name: NCPU
flag: "--ncpu"
type: int
default: 4
help: "CPU 核心数 (默认: 4)"
status:
help: "检查 LS-DYNA 进程状态"
jobs:
run:
fn: run_ls_dyna
args: ["${INPUT_FILE}"]
kwargs:
ncpu: ${NCPU}
mpi:
fn: run_ls_dyna_mpi
args: ["${INPUT_FILE}"]
kwargs:
ncpu: ${NCPU}
status:
fn: check_ls_dyna_status
-107
View File
@@ -1,107 +0,0 @@
# packtool - Python 打包工具
# 用法:
# pf packtool src --project-dir . --output-dir .pypack
# pf packtool deps requests numpy --lib-dir libs
# pf packtool wheel --project-dir . --output-dir dist
# pf packtool embed --version 3.10 --output-dir python
# pf packtool zip --source-dir . --output-file package.zip
# pf packtool clean
strategy: thread
variables:
PROJECT_DIR: "."
OUTPUT_DIR: ".pypack"
LIB_DIR: "libs"
DEPENDENCIES: []
VERSION: "3.10"
OUTPUT_FILE: "package.zip"
SOURCE_DIR: "."
cli:
description: "PackTool - Python 打包工具"
usage: "pf packtool <command> [options]"
subcommands:
src:
help: "打包源码"
options:
- name: PROJECT_DIR
flag: "--project-dir"
type: path
default: "."
help: "项目目录 (默认: .)"
- name: OUTPUT_DIR
flag: "--output-dir"
type: str
default: ".pypack"
help: "输出目录 (默认: .pypack)"
deps:
help: "打包依赖"
positional:
- name: DEPENDENCIES
nargs: "*"
type: str
help: "依赖包列表"
options:
- name: LIB_DIR
flag: "--lib-dir"
type: path
default: "libs"
help: "依赖库目录 (默认: libs)"
wheel:
help: "构建 wheel"
options:
- name: PROJECT_DIR
flag: "--project-dir"
type: path
default: "."
help: "项目目录 (默认: .)"
- name: OUTPUT_DIR
flag: "--output-dir"
type: path
default: "dist"
help: "输出目录 (默认: dist)"
embed:
help: "安装嵌入式 Python"
options:
- name: VERSION
flag: "--version"
type: str
default: "3.10"
help: "Python 版本 (默认: 3.10)"
- name: OUTPUT_DIR
flag: "--output-dir"
type: path
default: "python"
help: "输出目录 (默认: python)"
zip:
help: "创建 zip 包"
options:
- name: SOURCE_DIR
flag: "--source-dir"
type: path
default: "."
help: "源目录 (默认: .)"
- name: OUTPUT_FILE
flag: "--output-file"
type: path
default: "package.zip"
help: "输出文件 (默认: package.zip)"
clean:
help: "清理构建目录"
jobs:
src:
fn: pack_source
args: ["${PROJECT_DIR}", "${OUTPUT_DIR}"]
deps:
fn: pack_dependencies
args: ["${LIB_DIR}", "${DEPENDENCIES}"]
wheel:
fn: pack_wheel
args: ["${PROJECT_DIR}", "${OUTPUT_DIR}"]
embed:
fn: install_embed_python
args: ["${VERSION}", "${OUTPUT_DIR}"]
zip:
fn: create_zip_package
args: ["${SOURCE_DIR}", "${OUTPUT_FILE}"]
clean:
fn: clean_build_dir
args: ["${OUTPUT_DIR}"]
-303
View File
@@ -1,303 +0,0 @@
# pdftool - PDF 文件工具集
# 用法:
# pf pdftool m a.pdf b.pdf --output merged.pdf
# pf pdftool s input.pdf --output-dir split
# pf pdftool c input.pdf --output compressed.pdf
# pf pdftool e input.pdf --output encrypted.pdf --password 123456
# pf pdftool d input.pdf --output decrypted.pdf --password 123456
# pf pdftool xt input.pdf --output output.txt
# pf pdftool xi input.pdf --output-dir images
# pf pdftool w input.pdf --output watermarked.pdf --text CONFIDENTIAL
# pf pdftool r input.pdf --output rotated.pdf --rotation 90
# pf pdftool crop input.pdf --output cropped.pdf --left 10 --top 10 --right 10 --bottom 10
# pf pdftool i input.pdf
# pf pdftool ocr input.pdf --output ocr.pdf --lang chi_sim+eng
# pf pdftool img input.pdf --output-dir images --dpi 300
# pf pdftool repair input.pdf --output repaired.pdf
strategy: thread
variables:
INPUT: input.pdf
INPUTS: []
OUTPUT: output.pdf
OUTPUT_DIR: output
PASSWORD: ""
TEXT: CONFIDENTIAL
ROTATION: 90
MARGINS: [10, 10, 10, 10]
DPI: 300
LANG: chi_sim+eng
ORDER: []
LEFT: 10
TOP: 10
RIGHT: 10
BOTTOM: 10
cli:
description: "PdfTool - PDF 文件工具集"
usage: "pf pdftool <command> [options]"
subcommands:
m:
help: "合并 PDF"
positional:
- name: INPUTS
nargs: "+"
type: path
help: "输入 PDF 文件列表"
options:
- name: OUTPUT
flag: "--output"
type: path
default: "merged.pdf"
help: "输出文件 (默认: merged.pdf)"
s:
help: "拆分 PDF"
positional:
- name: INPUT
type: path
help: "输入 PDF 文件"
options:
- name: OUTPUT_DIR
flag: "--output-dir"
type: path
default: "split"
help: "输出目录 (默认: split)"
c:
help: "压缩 PDF"
positional:
- name: INPUT
type: path
help: "输入 PDF 文件"
options:
- name: OUTPUT
flag: "--output"
type: path
default: "compressed.pdf"
help: "输出文件 (默认: compressed.pdf)"
e:
help: "加密 PDF"
positional:
- name: INPUT
type: path
help: "输入 PDF 文件"
options:
- name: OUTPUT
flag: "--output"
type: path
default: "encrypted.pdf"
help: "输出文件 (默认: encrypted.pdf)"
- name: PASSWORD
flag: "--password"
type: str
required: true
help: "密码 (必填)"
d:
help: "解密 PDF"
positional:
- name: INPUT
type: path
help: "输入 PDF 文件"
options:
- name: OUTPUT
flag: "--output"
type: path
default: "decrypted.pdf"
help: "输出文件 (默认: decrypted.pdf)"
- name: PASSWORD
flag: "--password"
type: str
required: true
help: "密码 (必填)"
xt:
help: "提取文本"
positional:
- name: INPUT
type: path
help: "输入 PDF 文件"
options:
- name: OUTPUT
flag: "--output"
type: path
default: "output.txt"
help: "输出文件 (默认: output.txt)"
xi:
help: "提取图片"
positional:
- name: INPUT
type: path
help: "输入 PDF 文件"
options:
- name: OUTPUT_DIR
flag: "--output-dir"
type: path
default: "images"
help: "输出目录 (默认: images)"
w:
help: "添加水印"
positional:
- name: INPUT
type: path
help: "输入 PDF 文件"
options:
- name: OUTPUT
flag: "--output"
type: path
default: "watermarked.pdf"
help: "输出文件 (默认: watermarked.pdf)"
- name: TEXT
flag: "--text"
type: str
default: "CONFIDENTIAL"
help: "水印文字 (默认: CONFIDENTIAL)"
r:
help: "旋转 PDF"
positional:
- name: INPUT
type: path
help: "输入 PDF 文件"
options:
- name: OUTPUT
flag: "--output"
type: path
default: "rotated.pdf"
help: "输出文件 (默认: rotated.pdf)"
- name: ROTATION
flag: "--rotation"
type: int
default: 90
help: "旋转角度 (默认: 90)"
crop:
help: "裁剪 PDF"
positional:
- name: INPUT
type: path
help: "输入 PDF 文件"
options:
- name: OUTPUT
flag: "--output"
type: path
default: "cropped.pdf"
help: "输出文件 (默认: cropped.pdf)"
- name: LEFT
flag: "--left"
type: int
default: 10
help: "左边距 (默认: 10)"
- name: TOP
flag: "--top"
type: int
default: 10
help: "上边距 (默认: 10)"
- name: RIGHT
flag: "--right"
type: int
default: 10
help: "右边距 (默认: 10)"
- name: BOTTOM
flag: "--bottom"
type: int
default: 10
help: "下边距 (默认: 10)"
i:
help: "查看 PDF 信息"
positional:
- name: INPUT
type: path
help: "输入 PDF 文件"
ocr:
help: "PDF OCR 识别"
positional:
- name: INPUT
type: path
help: "输入 PDF 文件"
options:
- name: OUTPUT
flag: "--output"
type: path
default: "ocr.pdf"
help: "输出文件 (默认: ocr.pdf)"
- name: LANG
flag: "--lang"
type: str
default: "chi_sim+eng"
help: "识别语言 (默认: chi_sim+eng)"
img:
help: "PDF 转图片"
positional:
- name: INPUT
type: path
help: "输入 PDF 文件"
options:
- name: OUTPUT_DIR
flag: "--output-dir"
type: path
default: "images"
help: "输出目录 (默认: images)"
- name: DPI
flag: "--dpi"
type: int
default: 300
help: "DPI (默认: 300)"
repair:
help: "修复 PDF"
positional:
- name: INPUT
type: path
help: "输入 PDF 文件"
options:
- name: OUTPUT
flag: "--output"
type: path
default: "repaired.pdf"
help: "输出文件 (默认: repaired.pdf)"
jobs:
m:
fn: pdf_merge
args: ["${INPUTS}", "${OUTPUT}"]
s:
fn: pdf_split
args: ["${INPUT}", "${OUTPUT_DIR}"]
c:
fn: pdf_compress
args: ["${INPUT}", "${OUTPUT}"]
e:
fn: pdf_encrypt
args: ["${INPUT}", "${OUTPUT}", "${PASSWORD}"]
d:
fn: pdf_decrypt
args: ["${INPUT}", "${OUTPUT}", "${PASSWORD}"]
xt:
fn: pdf_extract_text
args: ["${INPUT}", "${OUTPUT}"]
xi:
fn: pdf_extract_images
args: ["${INPUT}", "${OUTPUT_DIR}"]
w:
fn: pdf_add_watermark
args: ["${INPUT}", "${OUTPUT}"]
kwargs:
text: "${TEXT}"
r:
fn: pdf_rotate
args: ["${INPUT}", "${OUTPUT}"]
kwargs:
rotation: ${ROTATION}
crop:
fn: pdf_crop
args: ["${INPUT}", "${OUTPUT}"]
kwargs:
margins: "${MARGINS}"
i:
fn: pdf_info
args: ["${INPUT}"]
ocr:
fn: pdf_ocr
args: ["${INPUT}", "${OUTPUT}"]
kwargs:
lang: "${LANG}"
img:
fn: pdf_to_images
args: ["${INPUT}", "${OUTPUT_DIR}"]
kwargs:
dpi: ${DPI}
repair:
fn: pdf_repair
args: ["${INPUT}", "${OUTPUT}"]
-78
View File
@@ -1,78 +0,0 @@
# piptool - pip 包管理工具
# 用法:
# pf piptool i requests
# pf piptool u requests
# pf piptool r requests
# pf piptool d requests
# pf piptool up
# pf piptool f
strategy: thread
variables:
PACKAGES: []
OFFLINE: false
cli:
description: "PipTool - pip 包管理工具"
usage: "pf piptool <command> [packages...] [options]"
subcommands:
i:
help: "安装包"
positional:
- name: PACKAGES
nargs: "+"
type: str
help: "包名列表"
u:
help: "卸载包"
positional:
- name: PACKAGES
nargs: "+"
type: str
help: "包名列表"
r:
help: "重装包"
positional:
- name: PACKAGES
nargs: "+"
type: str
help: "包名列表"
options:
- name: OFFLINE
flag: "--offline"
action: "store_true"
help: "离线模式"
d:
help: "下载包"
positional:
- name: PACKAGES
nargs: "+"
type: str
help: "包名列表"
options:
- name: OFFLINE
flag: "--offline"
action: "store_true"
help: "离线模式"
up:
help: "升级 pip"
f:
help: "导出依赖"
jobs:
i:
cmd: ["pip", "install", "${PACKAGES}"]
u:
fn: pip_uninstall
args: ["${PACKAGES}"]
r:
fn: pip_reinstall
args: ["${PACKAGES}"]
kwargs:
offline: ${OFFLINE}
d:
fn: pip_download
args: ["${PACKAGES}"]
kwargs:
offline: ${OFFLINE}
up:
cmd: ["python", "-m", "pip", "install", "--upgrade", "pip"]
f:
fn: pip_freeze
-125
View File
@@ -1,125 +0,0 @@
# pymake - 项目构建工具
# 用法
# pf pymake <command>
# 命令
# b: 构建 Python 主包 (uv build)
# ba: 构建所有包 (Python + Rust)
# bc: 构建 Rust 核心模块 (maturin build)
# bump: 升级版本号 (清理 + 检查 + add + bumpversion)
# bumpmi: 升级次版本号 (bumpversion minor)
# c: 清理构建产物 (调用 gitt c)
# cov: 测试并生成覆盖率
# doc: 构建 Sphinx 文档
# lint: 代码格式化与检查 (ruff)
# p: 推送代码 (清理 + push + push tags)
# pb: 发布到 PyPI (twine + hatch)
# sync: 同步依赖 (uv sync)
# t: 运行测试
# tc: 类型检查 (pyrefly + ruff)
# tf: 快速测试 (无 slow)
# tox: 多版本测试 (tox)
strategy: thread
variables:
CWD: "."
cli:
description: "PyMake - 项目构建工具"
usage: "pf pymake <command>"
options:
- name: CWD
flag: "--cwd"
type: path
required: false
default: "."
help: "工作目录 (默认: 当前目录)"
subcommands:
b: {help: "构建 Python 主包 (uv build)"}
ba: {help: "构建所有包 (Python + Rust)"}
bc: {help: "构建 Rust 核心模块 (maturin build)"}
bump: {help: "升级版本号 (清理 + 检查 + add + bumpversion)"}
bumpmi: {help: "升级次版本号 (bumpversion minor)"}
c: {help: "清理构建产物 (调用 gitt c)"}
cov: {help: "测试并生成覆盖率"}
doc: {help: "构建 Sphinx 文档"}
lint: {help: "代码格式化与检查 (ruff)"}
p: {help: "推送代码 (清理 + push + push tags)"}
pb: {help: "发布到 PyPI (twine + hatch)"}
sync: {help: "同步依赖 (uv sync)"}
t: {help: "运行测试"}
tc: {help: "类型检查 (pyrefly + ruff)"}
tf: {help: "快速测试 (无 slow)"}
tox: {help: "多版本测试 (tox)"}
jobs:
# 单任务别名
b:
cmd: ["uv", "build"]
cwd: ${CWD}
bc:
cmd: ["maturin", "build", "-r"]
cwd: ${CWD}
sync:
cmd: ["uv", "sync"]
cwd: ${CWD}
c:
cmd: ["pf", "gitt", "c"]
cwd: ${CWD}
t:
cmd: ["pytest", "-m", "not slow", "-n", "8", "--dist", "loadfile", "--color=yes", "--durations=10"]
cwd: ${CWD}
tf:
cmd: ["pytest", "-m", "not slow", "--dist", "loadfile", "--color=yes", "--durations=10"]
cwd: ${CWD}
bumpversion:
cmd: ["bumpversion", "patch"]
needs: [git_add_all]
cwd: ${CWD}
bumpmi:
cmd: ["bumpversion", "minor"]
cwd: ${CWD}
doc:
cmd: ["sphinx-build", "-b", "html", "docs", "docs/_build"]
cwd: ${CWD}
lint:
cmd: ["ruff", "check", "--fix", "--unsafe-fixes"]
cwd: ${CWD}
tox:
cmd: ["tox", "-p", "auto"]
cwd: ${CWD}
# 内部 job (不暴露为 subcommand)
test_coverage:
cmd: ["pytest", "--cov", "-n", "8", "--dist", "loadfile", "--tb=short", "-v", "--color=yes", "--durations=10"]
needs: [c]
cwd: ${CWD}
pyrefly_check:
cmd: ["pyrefly", "check", "."]
cwd: ${CWD}
git_add_all:
cmd: ["git", "add", "-A"]
needs: [tc]
cwd: ${CWD}
git_push:
cmd: ["git", "push"]
cwd: ${CWD}
git_push_tags:
cmd: ["git", "push", "--tags"]
cwd: ${CWD}
twine_publish:
cmd: ["twine", "upload", "--disable-progress-bar"]
cwd: ${CWD}
publish_python:
cmd: ["hatch", "publish"]
cwd: ${CWD}
# 聚合 job (方向 B: 有 needs 无 cmd/fn)
ba:
needs: [b, bc]
bump:
needs: [bumpversion]
cov:
needs: [test_coverage]
tc:
needs: [c, pyrefly_check, lint]
p:
needs: [c, git_push, git_push_tags]
pb:
needs: [twine_publish, publish_python]
-13
View File
@@ -1,13 +0,0 @@
# reseticoncache - 重置 Windows 图标缓存
# 用法
# pf reseticon
# 说明
# 杀掉 explorer → 删除 IconCache.db → 删除 iconcache* → 重启 explorer
# 仅在 Windows 上有效, 非 Windows 平台打印提示并跳过
strategy: sequential
cli:
description: "重置 Windows 图标缓存"
usage: "pf reseticon"
jobs:
reset:
fn: reset_icon_cache_run
-34
View File
@@ -1,34 +0,0 @@
# screenshot - 截图工具
# 用法:
# pf screenshot full
# pf screenshot area --filename custom.png
strategy: thread
variables:
FILENAME: null
cli:
description: "Screenshot - 截图工具"
usage: "pf screenshot <command> [options]"
subcommands:
full:
help: "全屏截图"
options:
- name: FILENAME
flag: "--filename"
type: str
help: "文件名"
area:
help: "区域截图"
options:
- name: FILENAME
flag: "--filename"
type: str
help: "文件名"
jobs:
full:
fn: take_screenshot_full
kwargs:
filename: "${FILENAME}"
area:
fn: take_screenshot_area
kwargs:
filename: "${FILENAME}"
-49
View File
@@ -1,49 +0,0 @@
# sshcopyid - SSH 密钥部署工具
# 用法:
# pf sshcopyid hostname username password
# pf sshcopyid server user pass --port 2222
strategy: thread
variables:
HOSTNAME: ""
USERNAME: ""
PASSWORD: ""
PORT: 22
KEYPATH: "~/.ssh/id_rsa.pub"
TIMEOUT: 30
cli:
description: "SSHCopyID - SSH 密钥部署工具"
usage: "pf sshcopyid <hostname> <username> <password> [options]"
positional:
- name: HOSTNAME
type: str
help: "远程服务器主机名或 IP 地址"
- name: USERNAME
type: str
help: "远程服务器用户名"
- name: PASSWORD
type: str
help: "远程服务器密码"
options:
- name: PORT
flag: "--port"
type: int
default: 22
help: "SSH 端口 (默认: 22)"
- name: KEYPATH
flag: "--keypath"
type: str
default: "~/.ssh/id_rsa.pub"
help: "公钥文件路径"
- name: TIMEOUT
flag: "--timeout"
type: int
default: 30
help: "SSH 操作超时秒数 (默认: 30)"
jobs:
deploy:
fn: ssh_copy_id
args: ["${HOSTNAME}", "${USERNAME}", "${PASSWORD}"]
kwargs:
port: ${PORT}
keypath: "${KEYPATH}"
timeout: ${TIMEOUT}
+2 -1
View File
@@ -16,8 +16,9 @@ DAG 库中泛滥的样板包装器。
from __future__ import annotations
import inspect
from collections.abc import Mapping
from functools import lru_cache
from typing import Any, Mapping
from typing import Any
from .errors import InjectionError
from .task import Context, TaskSpec
+2 -1
View File
@@ -6,7 +6,8 @@
from __future__ import annotations
from typing import Any, Iterable
from collections.abc import Iterable
from typing import Any
class PyFlowXError(Exception):
View File
-58
View File
@@ -1,58 +0,0 @@
"""Example 3: async aggregation with static args and Context injection.
Shows:
* async task functions executed with strategy="async".
* static positional args (TaskSpec.args) for parameterised tasks.
* Context annotation to receive the full upstream result mapping.
* on_event callback for real-time progress.
"""
from __future__ import annotations
import asyncio
from typing import Any
import pyflowx as px
async def fetch_user(uid: int) -> dict[str, Any]:
await asyncio.sleep(0.2)
return {"id": uid, "name": f"User{uid}"}
async def fetch_posts(uid: int) -> list[int]:
await asyncio.sleep(0.2)
return [uid, uid + 1]
# Context annotation → receives the full mapping of upstream results.
def aggregate(ctx: px.Context) -> dict[str, Any]:
return dict(ctx)
def main() -> None:
graph = px.Graph.from_specs(
[
# Static positional args parameterise the same function twice.
px.TaskSpec("fetch_user", fetch_user, args=(1,)),
px.TaskSpec("fetch_posts", fetch_posts, args=(1,)),
px.TaskSpec("aggregate", aggregate, depends_on=("fetch_user", "fetch_posts")),
]
)
print("=== Dry run ===")
_ = px.run(graph, strategy="async", dry_run=True)
events: list[px.TaskEvent] = []
print("\n=== Async execution ===")
report = px.run(graph, strategy="async", on_event=events.append)
for ev in events:
print(f" event: {ev.task} -> {ev.status.value}")
print(f"\naggregate = {report['aggregate']}")
print(report.describe())
if __name__ == "__main__":
main()
-79
View File
@@ -1,79 +0,0 @@
"""Example 1: ETL pipeline (sequential strategy).
Demonstrates the core PyFlowX workflow:
* Define tasks as plain functions.
* Declare the DAG with a list of TaskSpec.
* Parameter names == dependency names → automatic context injection,
no wrappers needed (contrast with flowweaver's get_task_result boilerplate).
* dry_run to preview, then execute and read typed results from RunReport.
"""
from __future__ import annotations
from typing import Any
import pyflowx as px
# --- task functions: pure, testable, no framework coupling ------------- #
def extract_customers() -> list[dict[str, Any]]:
return [
{"id": "C001", "name": "Alice"},
{"id": "C002", "name": "Bob"},
]
def extract_orders() -> list[dict[str, Any]]:
return [
{"id": "O001", "customer_id": "C001", "amount": 150.0},
{"id": "O002", "customer_id": "C002", "amount": 200.5},
]
# Parameter names match dependency names → automatic injection.
def transform(
extract_customers: list[dict[str, Any]],
extract_orders: list[dict[str, Any]],
) -> list[dict[str, Any]]:
cmap = {c["id"]: c for c in extract_customers}
return [{**o, "customer_name": cmap[o["customer_id"]]["name"]} for o in extract_orders if o["customer_id"] in cmap]
def load(transform: list[dict[str, Any]]) -> int:
print(f" loaded {len(transform)} records")
return len(transform)
def main() -> None:
graph = px.Graph.from_specs(
[
px.TaskSpec("extract_customers", extract_customers, tags=("extract",)),
px.TaskSpec("extract_orders", extract_orders, tags=("extract",)),
px.TaskSpec(
"transform",
transform,
depends_on=("extract_customers", "extract_orders"),
tags=("transform",),
),
px.TaskSpec(
"load", load, depends_on=("transform",), retry=px.RetryPolicy(max_attempts=1, delay=1.0), tags=("load",)
),
]
)
print("=== Execution plan ===")
print(graph.describe())
print("\n=== Dry run (no execution) ===")
_ = px.run(graph, strategy="sequential", dry_run=True)
print("\n=== Sequential execution ===")
report = px.run(graph, strategy="sequential")
print(report.describe())
print(f"\nload result = {report['load']}")
print(f"summary = {report.summary()}")
if __name__ == "__main__":
main()
-59
View File
@@ -1,59 +0,0 @@
"""Example 2: parallel execution (thread strategy).
Same DAG run with sequential vs. thread strategy to show layer-internal
parallelism. Tasks within a layer run concurrently; layers are barriers.
Layer 1: [fetch_a, fetch_b] (parallel)
Layer 2: [merge] (waits for both)
"""
from __future__ import annotations
import time
import pyflowx as px
def fetch_a() -> str:
time.sleep(0.5)
return "a"
def fetch_b() -> str:
time.sleep(0.5)
return "b"
def merge(fetch_a: str, fetch_b: str) -> str:
return fetch_a + fetch_b
def main() -> None:
graph = px.Graph.from_specs(
[
px.TaskSpec("fetch_a", fetch_a),
px.TaskSpec("fetch_b", fetch_b),
px.TaskSpec("merge", merge, depends_on=("fetch_a", "fetch_b")),
]
)
print("=== Mermaid diagram ===")
print(graph.to_mermaid("LR"))
print("\n=== Sequential (expect ~1.0s) ===")
start = time.time()
report_seq = px.run(graph, strategy="sequential")
t_seq = time.time() - start
print(f" result={report_seq['merge']} time={t_seq:.2f}s")
print("\n=== Threaded (expect ~0.5s) ===")
start = time.time()
report_thr = px.run(graph, strategy="thread", max_workers=2)
t_thr = time.time() - start
print(f" result={report_thr['merge']} time={t_thr:.2f}s")
print(f"\nspeedup = {t_seq / t_thr:.2f}x")
if __name__ == "__main__":
main()
+17 -12
View File
@@ -48,8 +48,11 @@ import inspect
import logging
import threading
import time
from collections.abc import Awaitable, Callable, Mapping
from datetime import datetime
from typing import Any, Awaitable, Callable, Literal, Mapping, cast
from typing import Any, Literal, cast
from rich.console import Console
from .context import build_call_args, describe_injection
from .errors import TaskFailedError, TaskTimeoutError
@@ -60,6 +63,9 @@ from .task import TaskEvent, TaskHooks, TaskResult, TaskSpec, TaskStatus
logger = logging.getLogger(__name__)
# verbose 模式输出用的 Console (不抢占用户 stdout, 仅格式化)
_verbose_console = Console()
# 进程池复用:同一次 run() 内的 process 任务共享一个 ProcessPoolExecutor。
# 模块级缓存避免每次任务都创建/销毁进程池的开销。
# run() 结束后通过 _shutdown_process_pool() 关闭(shutdown(wait=False) +
@@ -666,7 +672,7 @@ class AsyncLayerRunner:
return task_ctx, result
results = await asyncio.gather(*[_run_async_task(name) for name in to_run])
for name, (task_ctx, result) in zip(to_run, results):
for name, (task_ctx, result) in zip(to_run, results, strict=True):
_store_result(name, result, graph.resolved_spec(name), task_ctx, context, report, backend, on_event)
@@ -723,23 +729,22 @@ class DependencyRunner:
# 公共 API
# ---------------------------------------------------------------------- #
def _make_verbose_callback(on_event: EventCallback | None) -> EventCallback:
"""包装 on_event 回调, 在 verbose 模式下打印任务生命周期。"""
"""包装 on_event 回调, 在 verbose 模式下用 rich 打印任务生命周期。"""
def _verbose_callback(event: TaskEvent) -> None:
dur = f" ({event.duration:.3f}s)" if event.duration is not None else ""
if event.status == TaskStatus.RUNNING:
print(f"[verbose] 任务 {event.task!r} 开始执行...", flush=True)
_verbose_console.print(f"[cyan]▸[/cyan] [bold]{event.task!r}[/bold] 开始执行...")
elif event.status == TaskStatus.SUCCESS:
print(f"[verbose] 任务 {event.task!r} 成功{dur}", flush=True)
_verbose_console.print(f"[green]✓[/green] [bold]{event.task!r}[/bold] 成功[dim]{dur}[/dim]")
elif event.status == TaskStatus.FAILED:
err = f": {event.error}" if event.error else ""
print(
f"[verbose] 任务 {event.task!r} 失败{dur} (尝试 {event.attempts} 次){err}",
flush=True,
_verbose_console.print(
f"[red]✗[/red] [bold]{event.task!r}[/bold] 失败[dim]{dur} (尝试 {event.attempts} 次)[/dim][red]{err}[/red]"
)
elif event.status == TaskStatus.SKIPPED:
reason = f" ({event.reason})" if event.reason else ""
print(f"[verbose] 任务 {event.task!r} 跳过{reason}", flush=True)
_verbose_console.print(f"[yellow]○[/yellow] [bold]{event.task!r}[/bold] 跳过[dim]{reason}[/dim]")
if on_event is not None:
on_event(event)
@@ -840,11 +845,11 @@ def run(
def _print_dry_run(graph: Graph, layers: list[list[str]]) -> None:
"""打印执行计划但不运行任何任务。"""
print(f"Dry run: {len(graph)} tasks, {len(layers)} layers")
_verbose_console.print(f"[bold]Dry run:[/bold] [cyan]{len(graph)}[/cyan] tasks, [cyan]{len(layers)}[/cyan] layers")
for idx, layer in enumerate(layers, 1):
print(f" Layer {idx}: {layer}")
_verbose_console.print(f" [dim]Layer {idx}:[/dim] {layer}")
for name in layer:
print(f" - {describe_injection(graph.resolved_spec(name))}")
_verbose_console.print(f" [cyan]-[/cyan] {describe_injection(graph.resolved_spec(name))}")
def _drive_sequential(
+7 -46
View File
@@ -1,7 +1,7 @@
"""DAG 构建、校验、分层与可视化。
使用标准库的 :mod:`graphlib`3.9+)或 :mod:`graphlib_backport`3.8
进行拓扑排序。图以增量方式构建并即时校验,使配置错误在构建时(而非执行时)快速失败。
使用标准库的 :mod:`graphlib` 进行拓扑排序。图以增量方式构建并即时校验,
使配置错误在构建时(而非执行时)快速失败。
支持:
* 图级默认值 :class:`GraphDefaults`TaskSpec 字段为 ``None`` 时回退。
@@ -17,23 +17,16 @@ __all__ = [
"GraphDefaults",
]
import graphlib
import inspect
import sys
from collections.abc import Callable, Iterable, Mapping, Sequence
from dataclasses import dataclass, field, replace
from pathlib import Path
from typing import Any, Callable, Iterable, Mapping, Sequence
from typing import Any
from .errors import CycleError, DuplicateTaskError, MissingDependencyError
from .task import Context, RetryPolicy, TaskSpec
if sys.version_info >= (3, 9): # pragma: no cover
import graphlib # pyright: ignore[reportUnreachable]
_TopologicalSorter = graphlib.TopologicalSorter
else: # pragma: no cover
import graphlib # type: ignore[import-untyped]
_TopologicalSorter = graphlib.TopologicalSorter # pragma: no cover
_TopologicalSorter = graphlib.TopologicalSorter
@dataclass
@@ -235,38 +228,6 @@ class Graph:
graph.validate()
return graph
@classmethod
def from_yaml(
cls,
path: str | Path,
variables: Mapping[str, Any] | None = None,
) -> Graph:
"""从 YAML 文件构建任务图。
参考 GitHub Actions 风格 schema, 支持 jobs/needs/strategy.matrix/if
等 CI/CD 概念。详见 :mod:`pyflowx.yaml_loader`。
Parameters
----------
path : str | Path
YAML 文件路径
variables : Mapping[str, Any] | None
运行时变量, 用于替换 ``${VAR}`` 占位符
Returns
-------
Graph
构建好的任务图
Raises
------
YamlLoadError
文件不存在、YAML 格式错误、schema 校验失败、循环依赖等
"""
from .yaml_loader import load_yaml
return load_yaml(path, variables=variables)
def add_subgraph(self, sub: Graph, *, namespace: str | None = None) -> Graph:
"""将子图合并到当前图,任务名加命名空间前缀避免冲突。
@@ -326,7 +287,7 @@ class Graph:
sorter = _TopologicalSorter(self.deps)
try:
sorter.prepare()
except graphlib.CycleError as exc: # type: ignore[name-defined]
except graphlib.CycleError as exc:
cycle: Sequence[str] = exc.args[1] if len(exc.args) > 1 else []
raise CycleError(list(cycle)) from exc
+26 -10
View File
@@ -1,18 +1,34 @@
"""工具函数模块.
"""ops 子包 - CLI 工具模块集合.
按类别组织 CLI 工具中可复用的函数, 每个子模块使用 ``@px.register_fn`` 注册函数,
供 YAML 任务编排通过 ``fn`` 字段引用.
每个子模块使用 ``@px.tool`` 装饰器注册 CLI 工具, 由 ``pf`` 入口按需
``importlib.import_module`` 触发注册. 子模块按工具名组织 (一文件一工具).
子模块
------
- :mod:`files` —— 文件日期/等级/备份/压缩相关函数
- :mod:`dev` —— 开发工具 (ruff/版本号/pip/git) 相关函数
- :mod:`media` —— PDF/截图相关函数
- :mod:`system` —— LS-DYNA/SSH/打包相关函数
- :mod:`autofmt` —— 代码格式化 (ruff)
- :mod:`bumpversion` —— 版本号管理
- :mod:`clr` —— 清屏
- :mod:`dockercmd` —— Docker 镜像登录
- :mod:`envdev` —— 开发环境镜像源配置
- :mod:`filedate` —— 文件日期修改
- :mod:`filelevel` —— 文件等级
- :mod:`folderback` —— 文件夹备份
- :mod:`folderzip` —— 文件夹压缩
- :mod:`gittool` —— Git 工具
- :mod:`lscalc` —— LS-DYNA 计算
- :mod:`msdownload` —— ModelScope 下载
- :mod:`packtool` —— 打包工具
- :mod:`pdftool` —— PDF 工具
- :mod:`piptool` —— pip 工具
- :mod:`pymake` —— Python 项目构建
- :mod:`reseticoncache` —— 重置图标缓存
- :mod:`screenshot` —— 截图
- :mod:`sglang` —— SGLang 服务
- :mod:`sshcopyid` —— SSH 公钥拷贝
- :mod:`taskkill` —— 进程终止
- :mod:`which` —— 命令查找
"""
from __future__ import annotations
from . import dev, files, media, system
__all__ = ["dev", "files", "media", "system"]
__all__: list[str] = []
+66
View File
@@ -0,0 +1,66 @@
"""ops 共享辅助: 跨平台命令选择、平台守卫、共享忽略模式."""
from __future__ import annotations
from collections.abc import Sequence
from pyflowx.conditions import Constants
__all__ = ["IGNORE_PATTERNS", "ensure_platform", "platform_command"]
IGNORE_PATTERNS: list[str] = [
"__pycache__",
"*.pyc",
"*.pyo",
".git",
".venv",
".idea",
".vscode",
"*.egg-info",
"dist",
"build",
".pytest_cache",
".tox",
".mypy_cache",
]
def platform_command(windows: Sequence[str], posix: Sequence[str]) -> list[str]:
"""按当前平台返回对应命令.
Parameters
----------
windows : Sequence[str]
Windows 平台命令 (含参数).
posix : Sequence[str]
Linux/macOS 平台命令 (含参数).
Returns
-------
list[str]
当前平台对应的命令列表 (副本, 可安全修改).
"""
return list(windows) if Constants.IS_WINDOWS else list(posix)
def ensure_platform(predicate: bool, tool_name: str, platform_name: str) -> bool:
"""平台不满足时打印提示, 返回是否满足.
Parameters
----------
predicate : bool
平台匹配谓词 (如 ``Constants.IS_WINDOWS``).
tool_name : str
工具名称, 用于提示信息前缀.
platform_name : str
目标平台名称, 用于提示信息.
Returns
-------
bool
平台满足时返回 True, 否则打印 ``{tool_name}: 仅在 {platform_name} 上支持`` 并返回 False.
"""
if not predicate:
print(f"{tool_name}: 仅在 {platform_name} 上支持")
return False
return True
+228
View File
@@ -0,0 +1,228 @@
"""autofmt - 自动格式化工具.
提供格式化代码/代码检查/自动添加文档/同步配置 子命令.
"""
from __future__ import annotations
import ast
import subprocess
from pathlib import Path
import pyflowx as px
from pyflowx.ops._common import IGNORE_PATTERNS
__all__ = [
"add_docstring",
"auto_add_docstrings",
"fmt",
"format_all",
"format_with_ruff",
"generate_module_docstring",
"lint",
"lint_with_ruff",
"sync_pyproject_config",
]
# ============================================================================
# 辅助函数
# ============================================================================
@px.tool("autofmt", subcommand="fmt", help="格式化代码")
def fmt(target: str = ".") -> None:
"""格式化代码 (ruff format).
Parameters
----------
target : str
目标路径 (默认: .)
"""
format_with_ruff(Path(target), fix=False)
@px.tool("autofmt", subcommand="lint", help="代码检查")
def lint(target: str = ".", fix: bool = False) -> None:
"""代码检查 (ruff check).
Parameters
----------
target : str
目标路径 (默认: .)
fix : bool
自动修复问题
"""
lint_with_ruff(Path(target), fix=fix)
def format_with_ruff(target: Path, fix: bool = True) -> None:
"""使用 ruff 格式化代码.
Parameters
----------
target : Path
目标路径
fix : bool
是否自动修复
"""
cmd = ["ruff", "format", str(target)]
if fix:
cmd.append("--fix")
subprocess.run(cmd, check=True)
print(f"ruff format 完成: {target}")
def lint_with_ruff(target: Path, fix: bool = True) -> None:
"""使用 ruff 检查代码.
Parameters
----------
target : Path
目标路径
fix : bool
是否自动修复
"""
cmd = ["ruff", "check", str(target)]
if fix:
cmd.extend(["--fix", "--unsafe-fixes"])
subprocess.run(cmd, check=True)
print(f"ruff check 完成: {target}")
def add_docstring(file_path: Path, docstring: str) -> bool:
"""为文件添加 docstring.
Parameters
----------
file_path : Path
文件路径
docstring : str
docstring 内容
Returns
-------
bool
是否成功添加
"""
try:
content = file_path.read_text(encoding="utf-8")
tree = ast.parse(content)
first_node = tree.body[0] if tree.body else None
if first_node and isinstance(first_node, ast.Expr) and isinstance(first_node.value, ast.Constant):
return False
lines = content.splitlines()
doc_lines = docstring.splitlines()
doc_lines.append("")
new_content = "\n".join(doc_lines + lines)
file_path.write_text(new_content, encoding="utf-8")
print(f"添加 docstring: {file_path}")
return True
except (OSError, UnicodeDecodeError, SyntaxError) as e:
print(f"处理失败: {file_path} - {e}")
return False
def generate_module_docstring(file_path: Path) -> str:
"""生成模块 docstring.
Parameters
----------
file_path : Path
文件路径
Returns
-------
str
生成的 docstring
"""
stem = file_path.stem
parent = file_path.parent.name
keywords = {
"cli": f"Command-line interface for {parent}",
"gui": f"Graphical user interface for {parent}",
"core": f"Core functionality for {parent}",
"util": f"Utility functions for {parent}",
"model": f"Data models for {parent}",
"test": f"Tests for {parent}",
}
for key, desc in keywords.items():
if key in stem.lower():
return f'"""{desc}."""'
return f'"""{stem.replace("_", " ").title()} module."""'
@px.tool("autofmt", subcommand="doc", help="自动添加文档字符串")
def auto_add_docstrings(root_dir: Path = Path()) -> int:
"""自动为所有 Python 文件添加 docstring.
Parameters
----------
root_dir : Path
根目录 (默认: 当前目录)
Returns
-------
int
添加的 docstring 数量
"""
count = 0
for py_file in root_dir.rglob("*.py"):
if any(pattern in str(py_file) for pattern in IGNORE_PATTERNS):
continue
docstring = generate_module_docstring(py_file)
if add_docstring(py_file, docstring):
count += 1
print(f"共添加 {count} 个 docstring")
return count
@px.tool("autofmt", subcommand="sync", help="同步 pyproject 配置")
def sync_pyproject_config(root_dir: Path = Path()) -> None:
"""同步 pyproject.toml 配置到子项目.
Parameters
----------
root_dir : Path
根目录 (默认: 当前目录)
"""
main_toml = root_dir / "pyproject.toml"
if not main_toml.exists():
print(f"主项目配置文件不存在: {main_toml}")
return
sub_tomls = [p for p in root_dir.rglob("pyproject.toml") if p != main_toml and ".venv" not in str(p)]
if not sub_tomls:
print("没有找到子项目的 pyproject.toml")
return
print(f"找到 {len(sub_tomls)} 个子项目配置文件")
for sub_toml in sub_tomls:
subprocess.run(["ruff", "format", str(sub_toml)], check=False)
print("配置同步完成")
def format_all(root_dir: Path) -> None:
"""格式化所有 Python 文件.
Parameters
----------
root_dir : Path
根目录
"""
subprocess.run(["ruff", "format", str(root_dir)], check=True)
subprocess.run(["ruff", "check", "--fix", "--unsafe-fixes", str(root_dir)], check=True)
print(f"格式化完成: {root_dir}")
+232
View File
@@ -0,0 +1,232 @@
"""版本号管理模块.
提供单文件版本号更新 (``bump_file_version``) 与项目级批量版本号同步
(``bump_project_version``) 能力. ``bump_project_version`` 通过 ``@px.tool``
注册为 CLI 工具, 可通过 ``pf bumpversion`` 调用.
设计要点
--------
``bump_project_version`` 采用 "先读取基准、再统一写入" 的两阶段策略:
先扫描所有 ``__init__.py`` / ``pyproject.toml`` 文件, 读取各自的版本号,
取最大值作为基准版本计算新版本号, 然后把新版本号统一写入所有文件,
避免文件间版本号不同步导致的跳号问题.
"""
from __future__ import annotations
import re
import subprocess
from pathlib import Path
from typing import Literal
import pyflowx as px
__all__ = [
"BumpVersionType",
"bump_file_version",
"bump_project_version",
]
# ============================================================================
# 配置
# ============================================================================
BumpVersionType = Literal["patch", "minor", "major"]
_PYPROJECT_VERSION_PATTERN = re.compile(
r'(?:^|\n)\s*version\s*=\s*["\']'
r"(?P<major>0|[1-9]\d*)\.(?P<minor>0|[1-9]\d*)\.(?P<patch>0|[1-9]\d*)"
r"(?:-(?P<prerelease>(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)(?:\.(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*))?"
r"(?:\+(?P<buildmetadata>[0-9a-zA-Z-]+(?:\.[0-9a-zA-Z-]+)*))?"
r'["\']',
re.MULTILINE,
)
_INIT_VERSION_PATTERN = re.compile(
r'(?:^|\n)\s*__version__\s*=\s*["\']'
r"(?P<major>0|[1-9]\d*)\.(?P<minor>0|[1-9]\d*)\.(?P<patch>0|[1-9]\d*)"
r"(?:-(?P<prerelease>(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)(?:\.(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*))?"
r"(?:\+(?P<buildmetadata>[0-9a-zA-Z-]+(?:\.[0-9a-zA-Z-]+)*))?"
r'["\']',
re.MULTILINE,
)
_IGNORE_DIRS = frozenset({".venv", "venv", ".git", "__pycache__", ".tox", "node_modules", "build", "dist", ".eggs"})
# ============================================================================
# 私有辅助函数
# ============================================================================
def _get_pattern_for_file(file_name: str) -> re.Pattern[str] | None:
"""根据文件类型获取对应的正则表达式."""
if file_name == "pyproject.toml":
return _PYPROJECT_VERSION_PATTERN
if file_name == "__init__.py":
return _INIT_VERSION_PATTERN
return None
def _calculate_new_version(major: int, minor: int, patch: int, part: BumpVersionType) -> str:
"""计算新版本号."""
if part == "major":
return f"{major + 1}.0.0"
if part == "minor":
return f"{major}.{minor + 1}.0"
return f"{major}.{minor}.{patch + 1}"
def _build_replacement_string(original_match: str, new_version: str, file_name: str) -> str:
"""构建替换字符串, 保留原始格式."""
quote_char = '"' if '"' in original_match else "'"
key = "__version__" if file_name == "__init__.py" else "version"
prefix_match = re.match(rf"(\s*{key}\s*=\s*)[\"']", original_match)
prefix = prefix_match.group(1) if prefix_match else f"{key} = "
return f"{prefix}{quote_char}{new_version}{quote_char}"
def _read_version_tuple(file_path: Path) -> tuple[int, int, int] | None:
"""从文件中读取版本号, 返回 (major, minor, patch) 元组; 未找到返回 None.
读取失败时抛出 ``OSError`` / ``UnicodeDecodeError`` 由调用方处理.
"""
pattern = _get_pattern_for_file(file_path.name)
if pattern is None:
return None
content = file_path.read_text(encoding="utf-8")
match = pattern.search(content)
if not match:
return None
return int(match.group("major")), int(match.group("minor")), int(match.group("patch"))
def _write_version_to_file(file_path: Path, new_version: str) -> bool:
"""把新版本号写入指定文件; 成功返回 True, 未匹配到版本号返回 False."""
pattern = _get_pattern_for_file(file_path.name)
if pattern is None: # pragma: no cover - 调用方已保证 pattern 不为 None
return False
content = file_path.read_text(encoding="utf-8")
match = pattern.search(content)
if not match: # pragma: no cover - 调用方已通过 _read_version_tuple 验证
return False
replacement = _build_replacement_string(match.group(0), new_version, file_path.name)
content = content.replace(match.group(0), replacement)
try:
file_path.write_text(content, encoding="utf-8")
except OSError as e:
print(f"更新文件 {file_path} 版本号时出错: {e}")
raise
return True
# ============================================================================
# 公共函数
# ============================================================================
def bump_file_version(file_path: Path, part: BumpVersionType = "patch") -> str | None:
"""更新单个文件中的版本号.
读取文件当前版本号, 按 ``part`` 指定的部分递增, 写回文件.
Parameters
----------
file_path : Path
要更新的文件路径 (``pyproject.toml`` 或 ``__init__.py``)
part : BumpVersionType
版本部分: patch, minor, major
Returns
-------
str | None
更新后的新版本号; 文件中未找到版本号或读取失败时返回 None
"""
version_tuple = _read_version_tuple(file_path)
if version_tuple is None:
print(f"文件 {file_path} 中未找到版本号模式")
return None
major, minor, patch = version_tuple
new_version = _calculate_new_version(major, minor, patch, part)
if not _write_version_to_file(file_path, new_version): # pragma: no cover - _read_version_tuple 已验证
return None
return new_version
@px.tool("bumpversion", help="版本号自动管理")
def bump_project_version(part: BumpVersionType = "patch", no_tag: bool = False) -> str | None:
"""批量同步项目所有版本号文件并提交.
扫描当前目录下所有 ``__init__.py`` 和 ``pyproject.toml`` 文件
(排除虚拟环境和缓存目录), 先读取每个文件的当前版本号取最大值作为基准,
计算新版本号后统一写入所有文件, 最后执行 git add (按文件名) + commit + tag.
采用 "先读取基准、再统一写入" 的两阶段策略, 即使某些文件版本号不同步,
也能在一次 bump 后重新对齐, 避免跳号.
Parameters
----------
part : BumpVersionType
版本部分: patch, minor, major
no_tag : bool
提交后不创建 git tag
Returns
-------
str | None
更新后的新版本号; 未找到版本号文件时返回 None
"""
all_files: set[Path] = set()
for pattern in ("__init__.py", "pyproject.toml"):
for file in Path.cwd().rglob(pattern):
if not any(ignore_dir in file.parts for ignore_dir in _IGNORE_DIRS):
all_files.add(file)
if not all_files:
print("未找到包含版本号的文件")
return None
print(f"找到 {len(all_files)} 个文件需要更新版本号")
cwd = Path.cwd()
for file in sorted(all_files):
print(f" - {file.relative_to(cwd)}")
# 阶段 1: 读取所有文件版本号, 取最大值作为基准
versions: list[tuple[int, int, int]] = []
for file in sorted(all_files):
v = _read_version_tuple(file)
if v is not None:
versions.append(v)
if not versions:
print("未能从任何文件读取版本号")
return None
major, minor, patch = max(versions)
new_version = _calculate_new_version(major, minor, patch, part)
print(f"基准版本: {major}.{minor}.{patch} -> 新版本: {new_version}")
# 阶段 2: 统一写入新版本号到所有文件
for file in sorted(all_files):
_write_version_to_file(file, new_version)
# 阶段 3: git add (按文件名) + commit + tag
relative_files = [str(file.relative_to(cwd)) for file in sorted(all_files)]
subprocess.run(["git", "add", *relative_files], check=True)
subprocess.run(["git", "commit", "-m", f"bump version to {new_version}"], check=True)
if not no_tag:
tag_name = f"v{new_version}"
subprocess.run(["git", "tag", "-a", tag_name, "-m", f"Release {tag_name}"], check=True)
print(f"已创建标签: {tag_name}")
return new_version
+20
View File
@@ -0,0 +1,20 @@
"""clr - 清屏工具.
跨平台清屏: Windows 调用 ``cls``, Linux/macOS 调用 ``clear``.
"""
from __future__ import annotations
import subprocess
import pyflowx as px
from pyflowx.ops._common import platform_command
@px.tool("clr", help="清屏 (跨平台)")
def clear_screen_run() -> None:
"""清屏 (跨平台).
Windows 调用 ``cls``, Linux/macOS 调用 ``clear``.
"""
subprocess.run(platform_command(["cls"], ["clear"]), check=False)
-618
View File
@@ -1,618 +0,0 @@
"""开发工具类函数模块.
聚合自动格式化 (autofmt)、版本号管理 (bumpversion)、pip 包管理 (piptool)、
git 工具 (gittool) 的可复用函数. 所有公共函数通过 ``@px.register_fn`` 注册,
供 YAML 任务编排引用.
"""
from __future__ import annotations
import ast
import fnmatch
import re
import subprocess
from pathlib import Path
from typing import Literal
import pyflowx as px
__all__ = [
"IGNORE_PATTERNS",
"PACKAGE_DIR",
"REQUIREMENTS_FILE",
"_PROTECTED_PACKAGES",
"BumpVersionType",
"add_docstring",
"auto_add_docstrings",
"bump_file_version",
"bump_project_version",
"format_all",
"format_with_ruff",
"generate_module_docstring",
"git_add_commit",
"git_init_add_commit",
"has_files",
"init_sub_dirs",
"lint_with_ruff",
"not_has_git_repo",
"pip_download",
"pip_freeze",
"pip_reinstall",
"pip_uninstall",
"sync_pyproject_config",
]
# ============================================================================
# autofmt 配置
# ============================================================================
IGNORE_PATTERNS = [
"__pycache__",
"*.pyc",
"*.pyo",
".git",
".venv",
".idea",
".vscode",
"*.egg-info",
"dist",
"build",
".pytest_cache",
".tox",
".mypy_cache",
]
# ============================================================================
# bumpversion 配置
# ============================================================================
BumpVersionType = Literal["patch", "minor", "major"]
_PYPROJECT_VERSION_PATTERN = re.compile(
r'(?:^|\n)\s*version\s*=\s*["\']'
r"(?P<major>0|[1-9]\d*)\.(?P<minor>0|[1-9]\d*)\.(?P<patch>0|[1-9]\d*)"
r"(?:-(?P<prerelease>(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)(?:\.(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*))?"
r"(?:\+(?P<buildmetadata>[0-9a-zA-Z-]+(?:\.[0-9a-zA-Z-]+)*))?"
r'["\']',
re.MULTILINE,
)
_INIT_VERSION_PATTERN = re.compile(
r'(?:^|\n)\s*__version__\s*=\s*["\']'
r"(?P<major>0|[1-9]\d*)\.(?P<minor>0|[1-9]\d*)\.(?P<patch>0|[1-9]\d*)"
r"(?:-(?P<prerelease>(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)(?:\.(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*))?"
r"(?:\+(?P<buildmetadata>[0-9a-zA-Z-]+(?:\.[0-9a-zA-Z-]+)*))?"
r'["\']',
re.MULTILINE,
)
# ============================================================================
# piptool 配置
# ============================================================================
PACKAGE_DIR = "packages"
REQUIREMENTS_FILE = "requirements.txt"
_PROTECTED_PACKAGES: frozenset[str] = frozenset(
{
"pyflowx",
"bitool",
}
)
# ============================================================================
# autofmt 私有辅助函数
# ============================================================================
# ============================================================================
# autofmt 函数
# ============================================================================
@px.register_fn
def format_with_ruff(target: Path, fix: bool = True) -> None:
"""使用 ruff 格式化代码.
Parameters
----------
target : Path
目标路径
fix : bool
是否自动修复
"""
cmd = ["ruff", "format", str(target)]
if fix:
cmd.append("--fix")
subprocess.run(cmd, check=True)
print(f"ruff format 完成: {target}")
@px.register_fn
def lint_with_ruff(target: Path, fix: bool = True) -> None:
"""使用 ruff 检查代码.
Parameters
----------
target : Path
目标路径
fix : bool
是否自动修复
"""
cmd = ["ruff", "check", str(target)]
if fix:
cmd.extend(["--fix", "--unsafe-fixes"])
subprocess.run(cmd, check=True)
print(f"ruff check 完成: {target}")
@px.register_fn
def add_docstring(file_path: Path, docstring: str) -> bool:
"""为文件添加 docstring.
Parameters
----------
file_path : Path
文件路径
docstring : str
docstring 内容
Returns
-------
bool
是否成功添加
"""
try:
content = file_path.read_text(encoding="utf-8")
tree = ast.parse(content)
first_node = tree.body[0] if tree.body else None
if first_node and isinstance(first_node, ast.Expr) and isinstance(first_node.value, ast.Constant):
return False
lines = content.splitlines()
doc_lines = docstring.splitlines()
doc_lines.append("")
new_content = "\n".join(doc_lines + lines)
file_path.write_text(new_content, encoding="utf-8")
print(f"添加 docstring: {file_path}")
return True
except (OSError, UnicodeDecodeError, SyntaxError) as e:
print(f"处理失败: {file_path} - {e}")
return False
@px.register_fn
def generate_module_docstring(file_path: Path) -> str:
"""生成模块 docstring.
Parameters
----------
file_path : Path
文件路径
Returns
-------
str
生成的 docstring
"""
stem = file_path.stem
parent = file_path.parent.name
keywords = {
"cli": f"Command-line interface for {parent}",
"gui": f"Graphical user interface for {parent}",
"core": f"Core functionality for {parent}",
"util": f"Utility functions for {parent}",
"model": f"Data models for {parent}",
"test": f"Tests for {parent}",
}
for key, desc in keywords.items():
if key in stem.lower():
return f'"""{desc}."""'
return f'"""{stem.replace("_", " ").title()} module."""'
@px.register_fn
def auto_add_docstrings(root_dir: Path) -> int:
"""自动为所有 Python 文件添加 docstring.
Parameters
----------
root_dir : Path
根目录
Returns
-------
int
添加的 docstring 数量
"""
count = 0
for py_file in root_dir.rglob("*.py"):
if any(pattern in str(py_file) for pattern in IGNORE_PATTERNS):
continue
docstring = generate_module_docstring(py_file)
if add_docstring(py_file, docstring):
count += 1
print(f"共添加 {count} 个 docstring")
return count
@px.register_fn
def sync_pyproject_config(root_dir: Path) -> None:
"""同步 pyproject.toml 配置到子项目.
Parameters
----------
root_dir : Path
根目录
"""
main_toml = root_dir / "pyproject.toml"
if not main_toml.exists():
print(f"主项目配置文件不存在: {main_toml}")
return
sub_tomls = [p for p in root_dir.rglob("pyproject.toml") if p != main_toml and ".venv" not in str(p)]
if not sub_tomls:
print("没有找到子项目的 pyproject.toml")
return
print(f"找到 {len(sub_tomls)} 个子项目配置文件")
for sub_toml in sub_tomls:
subprocess.run(["ruff", "format", str(sub_toml)], check=False)
print("配置同步完成")
@px.register_fn
def format_all(root_dir: Path) -> None:
"""格式化所有 Python 文件.
Parameters
----------
root_dir : Path
根目录
"""
subprocess.run(["ruff", "format", str(root_dir)], check=True)
subprocess.run(["ruff", "check", "--fix", "--unsafe-fixes", str(root_dir)], check=True)
print(f"格式化完成: {root_dir}")
# ============================================================================
# bumpversion 私有辅助函数
# ============================================================================
def _get_pattern_for_file(file_name: str) -> re.Pattern[str] | None:
"""根据文件类型获取对应的正则表达式."""
if file_name == "pyproject.toml":
return _PYPROJECT_VERSION_PATTERN
if file_name == "__init__.py":
return _INIT_VERSION_PATTERN
return None
def _calculate_new_version(major: int, minor: int, patch: int, part: BumpVersionType) -> str:
"""计算新版本号."""
if part == "major":
return f"{major + 1}.0.0"
if part == "minor":
return f"{major}.{minor + 1}.0"
return f"{major}.{minor}.{patch + 1}"
def _build_replacement_string(original_match: str, new_version: str, file_name: str) -> str:
"""构建替换字符串, 保留原始格式."""
quote_char = '"' if '"' in original_match else "'"
if file_name == "pyproject.toml":
prefix_match = re.match(r'(\s*version\s*=\s*)["\']', original_match)
prefix = prefix_match.group(1) if prefix_match else "version = "
return f"{prefix}{quote_char}{new_version}{quote_char}"
if file_name == "__init__.py":
prefix_match = re.match(r'(\s*__version__\s*=\s*)["\']', original_match)
prefix = prefix_match.group(1) if prefix_match else "__version__ = "
return f"{prefix}{quote_char}{new_version}{quote_char}"
return new_version
# ============================================================================
# bumpversion 函数
# ============================================================================
@px.register_fn
def bump_file_version(file_path: Path, part: BumpVersionType = "patch") -> str | None:
"""更新文件中的版本号.
Parameters
----------
file_path : Path
要更新的文件路径
part : BumpVersionType
版本部分: patch, minor, major
Returns
-------
str | None
更新后的新版本号, 如果文件中未找到版本号则返回 None
"""
try:
content = file_path.read_text(encoding="utf-8")
except Exception as e:
print(f"读取文件 {file_path} 时出错: {e}")
raise
pattern = _get_pattern_for_file(file_path.name)
if pattern:
match = pattern.search(content)
else:
match = _PYPROJECT_VERSION_PATTERN.search(content) or _INIT_VERSION_PATTERN.search(content)
if not match:
print(f"文件 {file_path} 中未找到版本号模式")
return None
major = int(match.group("major"))
minor = int(match.group("minor"))
patch = int(match.group("patch"))
new_version = _calculate_new_version(major, minor, patch, part)
original_match = match.group(0)
replacement = _build_replacement_string(original_match, new_version, file_path.name)
content = content.replace(original_match, replacement)
try:
file_path.write_text(content, encoding="utf-8")
except Exception as e:
print(f"更新文件 {file_path} 版本号时出错: {e}")
raise
return new_version
@px.register_fn
def bump_project_version(part: BumpVersionType = "patch", no_tag: bool = False) -> str | None:
"""批量更新项目所有文件的版本号并提交.
搜索当前目录下所有 ``__init__.py`` 和 ``pyproject.toml`` 文件
(排除虚拟环境和缓存目录), 更新版本号, 然后执行 git add + commit + tag.
Parameters
----------
part : BumpVersionType
版本部分: patch, minor, major
no_tag : bool
提交后不创建 git tag
Returns
-------
str | None
更新后的新版本号, 如果未找到版本号文件则返回 None
"""
ignore_dirs = {".venv", "venv", ".git", "__pycache__", ".tox", "node_modules", "build", "dist", ".eggs"}
all_files: set[Path] = set()
for pattern in ["__init__.py", "pyproject.toml"]:
for file in Path.cwd().rglob(pattern):
if not any(ignore_dir in file.parts for ignore_dir in ignore_dirs):
all_files.add(file)
if not all_files:
print("未找到包含版本号的文件")
return None
print(f"找到 {len(all_files)} 个文件需要更新版本号")
for file in sorted(all_files):
print(f" - {file.relative_to(Path.cwd())}")
new_version: str | None = None
for file in sorted(all_files):
version = bump_file_version(file, part)
if version is not None and new_version is None:
new_version = version
if not new_version:
print("未能获取新版本号")
return None
print(f"版本号已更新为: {new_version}")
subprocess.run(["git", "add", "."], check=False)
subprocess.run(["git", "commit", "-m", f"bump version to {new_version}"], check=False)
if not no_tag:
tag_name = f"v{new_version}"
subprocess.run(["git", "tag", "-a", tag_name, "-m", f"Release {tag_name}"], check=False)
print(f"已创建标签: {tag_name}")
return new_version
# ============================================================================
# piptool 私有辅助函数
# ============================================================================
def _get_installed_packages() -> list[str]:
"""获取当前环境中所有已安装的包名."""
try:
result = subprocess.run(
["pip", "list", "--format=freeze"],
capture_output=True,
text=True,
check=True,
)
packages: list[str] = []
for line in result.stdout.strip().split("\n"):
if line and "==" in line:
pkg_name = line.split("==")[0].strip()
packages.append(pkg_name)
except (subprocess.SubprocessError, OSError):
return []
return packages
def _expand_wildcard_packages(pattern: str) -> list[str]:
"""展开通配符模式为实际的包名列表."""
if not any(char in pattern for char in ["*", "?", "[", "]"]):
return [pattern]
installed_packages = _get_installed_packages()
matched = [pkg for pkg in installed_packages if fnmatch.fnmatchcase(pkg.lower(), pattern.lower())]
return matched
def _filter_protected_packages(packages: list[str]) -> list[str]:
"""过滤掉受保护的包名."""
safe = [p for p in packages if p.lower() not in {p.lower() for p in _PROTECTED_PACKAGES}]
filtered = [p for p in packages if p.lower() in {p.lower() for p in _PROTECTED_PACKAGES}]
if filtered:
print(f"跳过受保护的包: {', '.join(filtered)}")
return safe
# ============================================================================
# piptool 函数
# ============================================================================
@px.register_fn
def pip_uninstall(pkg_names: list[str]) -> None:
"""卸载包."""
packages_to_uninstall: list[str] = []
for pattern in pkg_names:
packages_to_uninstall.extend(_expand_wildcard_packages(pattern))
packages_to_uninstall = _filter_protected_packages(packages_to_uninstall)
if not packages_to_uninstall:
return
subprocess.run(["pip", "uninstall", "-y", *packages_to_uninstall], check=True)
@px.register_fn
def pip_reinstall(pkg_names: list[str], offline: bool = False) -> None:
"""重新安装包."""
safe_ps = _filter_protected_packages(pkg_names)
if not safe_ps:
print("所有指定的包均为受保护包, 跳过重装")
return
subprocess.run(["pip", "uninstall", "-y", *safe_ps], check=True)
options = ["--no-index", "--find-links", "."] if offline else []
subprocess.run(["pip", "install", *options, *safe_ps], check=True)
@px.register_fn
def pip_download(pkg_names: list[str], offline: bool = False) -> None:
"""下载包."""
options = ["--no-index", "--find-links", "."] if offline else []
subprocess.run(
["pip", "download", *pkg_names, *options, "-d", PACKAGE_DIR],
check=True,
)
@px.register_fn
def pip_freeze() -> None:
"""冻结依赖."""
result = subprocess.run(
["pip", "freeze", "--exclude-editable"],
capture_output=True,
text=True,
check=True,
)
Path(REQUIREMENTS_FILE).write_text(result.stdout)
# ============================================================================
# gittool 函数
# ============================================================================
@px.register_fn
def init_sub_dirs() -> None:
"""初始化子目录的 Git 仓库."""
sub_dirs = [subdir for subdir in Path.cwd().iterdir() if subdir.is_dir()]
for subdir in sub_dirs:
px.run(
px.Graph().chain(
px.cmd(["git", "init"], conditions=(lambda _: not_has_git_repo(),), cwd=subdir),
px.cmd(["git", "add", "."]),
px.cmd(["git", "commit", "-m", "init commit"]),
),
)
@px.register_fn
def not_has_git_repo() -> bool:
"""检查当前目录没有 Git 仓库."""
return not Path.cwd().exists() or not (Path.cwd() / ".git").is_dir()
@px.register_fn
def has_files() -> bool:
"""检查当前 Git 仓库是否有未提交的更改."""
try:
result = subprocess.run(
["git", "status", "--porcelain"],
capture_output=True,
text=True,
check=False,
)
return bool(result.stdout.strip())
except (subprocess.SubprocessError, OSError):
return False
@px.register_fn
def git_add_commit(message: str = "chore: update") -> None:
"""执行 git add + git commit (仅当有未提交更改时).
Parameters
----------
message : str
提交信息
"""
if not has_files():
print("没有文件需要提交")
return
subprocess.run(["git", "add", "."], check=True)
subprocess.run(["git", "commit", "-m", message], check=True)
@px.register_fn
def git_init_add_commit(message: str = "init commit") -> None:
"""执行 git init (若需) + git add + git commit (若有更改).
Parameters
----------
message : str
提交信息
"""
if not_has_git_repo():
subprocess.run(["git", "init"], check=True)
if has_files():
subprocess.run(["git", "add", "."], check=True)
subprocess.run(["git", "commit", "-m", message], check=True)
else:
print("没有文件需要提交")
+27
View File
@@ -0,0 +1,27 @@
"""dockercmd - Docker 镜像登录工具.
登录腾讯云 Docker 镜像仓库.
"""
from __future__ import annotations
import getpass
import subprocess
import pyflowx as px
_DOCKER_MIRROR_TENCENT: str = "ccr.ccs.tencentyun.com"
@px.tool("dockercmd", subcommand="login", help="登录腾讯云 Docker 镜像仓库")
def docker_login_tencent(username: str = "") -> None:
"""登录腾讯云 Docker 镜像仓库.
Parameters
----------
username : str
Docker 用户名 (为空时由 docker 交互式提示输入)
"""
user = username or getpass.getuser()
subprocess.run(["docker", "login", "--username", user, _DOCKER_MIRROR_TENCENT], check=False)
print(f"已尝试登录腾讯云镜像仓库 (用户: {user})")
+418
View File
@@ -0,0 +1,418 @@
"""envdev - 开发环境镜像源配置工具.
配置 Python / Conda / Rust 镜像源 (Linux 还会安装 Qt 库、中文字体、Docker).
所有镜像源参数互不影响, 可单独使用.
Linux 专用操作 (系统镜像/Qt/字体/Docker) 在非 Linux 平台上由函数内部跳过.
"""
from __future__ import annotations
import getpass
import os
import shutil
import subprocess
from pathlib import Path
from typing import Literal
import pyflowx as px
from pyflowx.conditions import Constants
from pyflowx.ops._common import ensure_platform
__all__ = [
"download_rustup_script",
"install_linux_docker",
"install_linux_fonts",
"install_linux_qt_libs",
"install_rust_toolchain",
"setup_conda_mirror",
"setup_linux_system_mirror",
"setup_python_mirror",
"setup_rust_mirror",
]
# ============================================================================
# 配置常量
# ============================================================================
PyMirrorType = Literal["tsinghua", "aliyun", "huaweicloud", "ustc", "zju"]
CondaMirrorType = Literal["tsinghua", "ustc", "bsfu", "aliyun"]
RustMirrorType = Literal["tsinghua", "ustc", "aliyun"]
_PIP_INDEX_URLS: dict[str, str] = {
"tsinghua": "https://pypi.tuna.tsinghua.edu.cn/simple",
"aliyun": "https://mirrors.aliyun.com/pypi/simple/",
"huaweicloud": "https://mirrors.huaweicloud.com/repository/pypi/simple/",
"ustc": "https://pypi.mirrors.ustc.edu.cn/simple/",
"zju": "https://mirrors.zju.edu.cn/pypi/simple/",
}
_PIP_TRUSTED_HOSTS: dict[str, str] = {
"tsinghua": "pypi.tuna.tsinghua.edu.cn",
"aliyun": "mirrors.aliyun.com",
"huaweicloud": "mirrors.huaweicloud.com",
"ustc": "pypi.mirrors.ustc.edu.cn",
"zju": "mirrors.zju.edu.cn",
}
_UV_PYTHON_INSTALL_MIRROR: str = "https://registry.npmmirror.com/-/binary/python-build-standalone"
_CONDA_MIRROR_URLS: dict[str, list[str]] = {
"tsinghua": [
"https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/",
"https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free/",
"https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/r/",
"https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/msys2/",
"https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/pro/",
"https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/",
"https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/bioconda/",
"https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/menpo/",
"https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/pytorch/",
],
"ustc": [
"https://mirrors.ustc.edu.cn/anaconda/pkgs/main/",
"https://mirrors.ustc.edu.cn/anaconda/pkgs/free/",
"https://mirrors.ustc.edu.cn/anaconda/pkgs/r/",
"https://mirrors.ustc.edu.cn/anaconda/pkgs/msys2/",
"https://mirrors.ustc.edu.cn/anaconda/pkgs/pro/",
"https://mirrors.ustc.edu.cn/anaconda/pkgs/dev/",
"https://mirrors.ustc.edu.cn/anaconda/cloud/conda-forge/",
"https://mirrors.ustc.edu.cn/anaconda/cloud/bioconda/",
"https://mirrors.ustc.edu.cn/anaconda/cloud/menpo/",
"https://mirrors.ustc.edu.cn/anaconda/cloud/pytorch/",
],
"bsfu": [
"https://mirrors.bsfu.edu.cn/anaconda/pkgs/main/",
"https://mirrors.bsfu.edu.cn/anaconda/pkgs/free/",
"https://mirrors.bsfu.edu.cn/anaconda/pkgs/r/",
"https://mirrors.bsfu.edu.cn/anaconda/pkgs/msys2/",
"https://mirrors.bsfu.edu.cn/anaconda/pkgs/pro/",
"https://mirrors.bsfu.edu.cn/anaconda/pkgs/dev/",
"https://mirrors.bsfu.edu.cn/anaconda/cloud/conda-forge/",
"https://mirrors.bsfu.edu.cn/anaconda/cloud/bioconda/",
"https://mirrors.bsfu.edu.cn/anaconda/cloud/menpo/",
"https://mirrors.bsfu.edu.cn/anaconda/cloud/pytorch/",
],
"aliyun": [
"https://mirrors.aliyun.com/anaconda/pkgs/main/",
"https://mirrors.aliyun.com/anaconda/pkgs/free/",
"https://mirrors.aliyun.com/anaconda/pkgs/r/",
"https://mirrors.aliyun.com/anaconda/pkgs/msys2/",
"https://mirrors.aliyun.com/anaconda/pkgs/pro/",
"https://mirrors.aliyun.com/anaconda/pkgs/dev/",
"https://mirrors.aliyun.com/anaconda/cloud/conda-forge/",
"https://mirrors.aliyun.com/anaconda/cloud/bioconda/",
"https://mirrors.aliyun.com/anaconda/cloud/menpo/",
"https://mirrors.aliyun.com/anaconda/cloud/pytorch/",
],
}
_RUSTUP_MIRRORS: dict[str, dict[str, str]] = {
"tsinghua": {
"RUSTUP_DIST_SERVER": "https://mirrors.tuna.tsinghua.edu.cn/rustup",
"RUSTUP_UPDATE_ROOT": "https://mirrors.tuna.tsinghua.edu.cn/rustup/rustup",
"TOML_REGISTRY": "https://mirrors.tuna.tsinghua.edu.cn/crates.io-index/",
},
"aliyun": {
"RUSTUP_DIST_SERVER": "https://mirrors.aliyun.com/rustup",
"RUSTUP_UPDATE_ROOT": "https://mirrors.aliyun.com/rustup/rustup",
"TOML_REGISTRY": "https://mirrors.aliyun.com/crates.io-index/",
},
"ustc": {
"RUSTUP_DIST_SERVER": "https://mirrors.ustc.edu.cn/rust-static",
"RUSTUP_UPDATE_ROOT": "https://mirrors.ustc.edu.cn/rust-static/rustup",
"TOML_REGISTRY": "https://mirrors.ustc.edu.cn/crates.io-index/",
},
}
_RUST_SCCACHE_DIR: Path = Path.home() / ".cargo" / "sccache"
_RUST_SCCACHE_CACHE_SIZE: str = "20G"
_QT_LIBS: list[str] = [
"build-essential",
"libgl1",
"libegl1",
"libglib2.0-0",
"libfontconfig1",
"libfreetype6",
"libxkbcommon0",
"libdbus-1-3",
"libxcb-xinerama0",
"libxcb-icccm4",
"libxcb-image0",
"libxcb-keysyms1",
"libxcb-randr0",
"libxcb-render-util0",
"libxcb-shape0",
"libxcb-xfixes0",
"libxcb-cursor0",
]
_CHINESE_FONTS: list[str] = [
"fonts-noto-cjk",
"fonts-wqy-microhei",
"fonts-wqy-zenhei",
"fonts-noto-color-emoji",
]
_DOWNLOAD_MIRROR_SCRIPT: str = "curl -sSL https://linuxmirrors.cn/main.sh -o /tmp/linuxmirrors.sh"
_INSTALL_MIRROR_SCRIPT: str = "sudo bash /tmp/linuxmirrors.sh"
_RUSTUP_DOWNLOAD_URL_LINUX: str = "https://mirrors.aliyun.com/repo/rust/rustup-init.sh"
_RUSTUP_DOWNLOAD_URL_WINDOWS: str = "https://static.rust-lang.org/rustup/dist/x86_64-pc-windows-msvc/rustup-init.exe"
# ============================================================================
# 私有辅助
# ============================================================================
def _pip_config_path() -> Path:
"""返回当前平台的 pip 配置文件路径."""
if Constants.IS_LINUX:
return Path.home() / ".pip" / "pip.conf"
return Path.home() / "pip" / "pip.ini"
# ============================================================================
# 镜像源配置函数
# ============================================================================
@px.tool("envdev", subcommand="setup-python", help="配置 Python 镜像源")
def setup_python_mirror(mirror: str = "tsinghua") -> None:
"""配置 Python 镜像源 (设置环境变量 + 写入 pip 配置文件).
设置 ``PIP_INDEX_URL`` / ``PIP_TRUSTED_HOSTS`` / ``UV_INDEX_URL`` /
``UV_PYTHON_INSTALL_MIRROR`` 等环境变量, 并写入 pip 配置文件.
Parameters
----------
mirror : str
镜像源名称: tsinghua/aliyun/huaweicloud/ustc/zju (默认: tsinghua)
"""
if mirror not in _PIP_INDEX_URLS:
print(f"未知 Python 镜像源: {mirror}")
return
index_url = _PIP_INDEX_URLS[mirror]
trusted_host = _PIP_TRUSTED_HOSTS[mirror]
os.environ["PIP_INDEX_URL"] = index_url
os.environ["PIP_TRUSTED_HOSTS"] = trusted_host
os.environ["UV_INDEX_URL"] = index_url
os.environ["UV_PYTHON_INSTALL_MIRROR"] = _UV_PYTHON_INSTALL_MIRROR
os.environ["UV_HTTP_TIMEOUT"] = "600"
os.environ["UV_LINK_MODE"] = "copy"
config_path = _pip_config_path()
config_path.parent.mkdir(parents=True, exist_ok=True)
content = f"[global]\nindex-url = {index_url}\ntrusted-host = {trusted_host}\n"
config_path.write_text(content, encoding="utf-8")
print(f"Python 镜像源已配置: {mirror} -> {config_path}")
@px.tool("envdev", subcommand="setup-conda", help="配置 Conda 镜像源")
def setup_conda_mirror(mirror: str = "tsinghua") -> None:
"""配置 Conda 镜像源 (写入 ~/.condarc).
Parameters
----------
mirror : str
镜像源名称: tsinghua/ustc/bsfu/aliyun (默认: tsinghua)
"""
if mirror not in _CONDA_MIRROR_URLS:
print(f"未知 Conda 镜像源: {mirror}")
return
urls = _CONDA_MIRROR_URLS[mirror]
config_path = Path.home() / ".condarc"
config_path.parent.mkdir(parents=True, exist_ok=True)
content = "show_channel_urls: true\nchannels:\n - " + "\n - ".join(urls) + "\n - defaults\n"
config_path.write_text(content, encoding="utf-8")
print(f"Conda 镜像源已配置: {mirror} -> {config_path}")
@px.tool("envdev", subcommand="setup-rust", help="配置 Rust 镜像源")
def setup_rust_mirror(mirror: str = "tsinghua", version: str = "stable") -> None:
"""配置 Rust 镜像源 (设置环境变量 + 写入 cargo config + 创建 sccache 目录).
设置 ``RUSTUP_DIST_SERVER`` / ``RUSTUP_UPDATE_ROOT`` / ``RUST_SCCACHE_DIR``
等环境变量, 写入 ``~/.cargo/config.toml``, 并创建 sccache 缓存目录.
Parameters
----------
mirror : str
镜像源名称: tsinghua/ustc/aliyun (默认: tsinghua)
version : str
Rust 版本 (未使用, 保留以与原 envdev 参数对齐)
"""
del version # 兼容旧参数, 实际安装由独立 job 处理
if mirror not in _RUSTUP_MIRRORS:
print(f"未知 Rust 镜像源: {mirror}")
return
mirrors = _RUSTUP_MIRRORS[mirror]
os.environ["RUSTUP_DIST_SERVER"] = mirrors["RUSTUP_DIST_SERVER"]
os.environ["RUSTUP_UPDATE_ROOT"] = mirrors["RUSTUP_UPDATE_ROOT"]
os.environ["RUST_SCCACHE_DIR"] = str(_RUST_SCCACHE_DIR)
os.environ["RUST_SCCACHE_CACHE_SIZE"] = _RUST_SCCACHE_CACHE_SIZE
_RUST_SCCACHE_DIR.mkdir(parents=True, exist_ok=True)
config_path = Path.home() / ".cargo" / "config.toml"
config_path.parent.mkdir(parents=True, exist_ok=True)
registry = mirrors["TOML_REGISTRY"]
content = (
f"\n[source.crates-io]\nreplace-with = '{mirror}'\n\n"
f'[source.{mirror}]\nregistry = "sparse+{registry}"\n\n'
f'[registries.{mirror}]\nindex = "sparse+{registry}"\n'
)
config_path.write_text(content, encoding="utf-8")
print(f"Rust 镜像源已配置: {mirror} -> {config_path}")
# ============================================================================
# Rust 工具链安装
# ============================================================================
@px.tool("envdev", subcommand="download-rustup", help="下载 Rustup 安装脚本")
def download_rustup_script() -> None:
"""下载 Rustup 安装脚本 (跨平台, 已安装 rustup 时跳过).
Linux 下载 ``rustup-init.sh``, Windows 下载 ``rustup-init.exe``.
"""
if shutil.which("rustup") is not None:
print("rustup 已安装, 跳过下载")
return
if Constants.IS_WINDOWS:
print("下载 rustup-init.exe...")
subprocess.run(
[
"powershell",
"-Command",
"Invoke-WebRequest",
"-Uri",
_RUSTUP_DOWNLOAD_URL_WINDOWS,
"-OutFile",
"rustup-init.exe",
],
check=False,
)
else:
print("下载 rustup-init.sh...")
subprocess.run(
["curl", "-fsSL", _RUSTUP_DOWNLOAD_URL_LINUX, "-o", "rustup-init.sh"],
check=False,
)
@px.tool(
"envdev",
subcommand="install-rust",
help="安装 Rust 工具链",
needs=["setup-rust", "download-rustup"],
allow_upstream_skip=True,
)
def install_rust_toolchain(version: str = "stable") -> None:
"""安装 Rust 工具链 (rustup 未安装时跳过).
Parameters
----------
version : str
Rust 版本: ``stable`` / ``nightly`` / ``beta`` (默认: ``stable``)
"""
if shutil.which("rustup") is None:
print("rustup 未安装, 跳过工具链安装")
return
subprocess.run(["rustup", "toolchain", "install", version], check=False)
print(f"Rust 工具链 {version} 安装完成")
# ============================================================================
# Linux 专用函数
# ============================================================================
@px.tool("envdev", subcommand="setup-linux-mirror", help="配置 Linux 系统镜像源")
def setup_linux_system_mirror() -> None:
"""下载并安装 Linux 系统镜像源 (仅 Linux, 已配置国内镜像时跳过).
检查 ``/etc/apt/sources.list`` 与 ``/etc/apt/sources.list.d/ubuntu.sources``
是否已配置国内镜像, 已配置则跳过; 未配置则下载并执行 linuxmirrors 脚本.
"""
if not ensure_platform(Constants.IS_LINUX, "setup_linux_system_mirror", "Linux"):
return
apt_files = ["/etc/apt/sources.list", "/etc/apt/sources.list.d/ubuntu.sources"]
mirror_keys = list(_PIP_INDEX_URLS.keys())
already_configured = False
for apt_file in apt_files:
try:
content = Path(apt_file).read_text(encoding="utf-8")
except (OSError, UnicodeDecodeError):
continue
if any(mirror in content for mirror in mirror_keys):
already_configured = True
break
if already_configured:
print("已配置国内镜像源, 跳过系统镜像配置")
return
print("下载 linuxmirrors 脚本...")
subprocess.run(_DOWNLOAD_MIRROR_SCRIPT, shell=True, check=False)
print("安装 linuxmirrors...")
subprocess.run(_INSTALL_MIRROR_SCRIPT, shell=True, check=False)
@px.tool(
"envdev",
subcommand="install-qt-libs",
help="安装 Qt 依赖库",
needs=["setup-linux-mirror"],
allow_upstream_skip=True,
)
def install_linux_qt_libs() -> None:
"""安装 Qt 依赖库 (仅 Linux)."""
if not ensure_platform(Constants.IS_LINUX, "install_linux_qt_libs", "Linux"):
return
subprocess.run(["sudo", "apt", "install", "-y", *_QT_LIBS], check=False)
print("Qt 依赖库安装完成")
@px.tool(
"envdev",
subcommand="install-fonts",
help="安装中文字体",
needs=["setup-linux-mirror"],
allow_upstream_skip=True,
)
def install_linux_fonts() -> None:
"""安装中文字体 (仅 Linux)."""
if not ensure_platform(Constants.IS_LINUX, "install_linux_fonts", "Linux"):
return
subprocess.run(["sudo", "apt", "install", "-y", *_CHINESE_FONTS], check=False)
print("中文字体安装完成")
@px.tool(
"envdev",
subcommand="install-docker",
help="安装 Docker",
needs=["setup-linux-mirror"],
allow_upstream_skip=True,
)
def install_linux_docker() -> None:
"""安装 Docker (仅 Linux)."""
if not ensure_platform(Constants.IS_LINUX, "install_linux_docker", "Linux"):
return
subprocess.run(["sudo", "apt", "install", "-y", "docker-compose-v2"], check=False)
subprocess.run(["sudo", "usermod", "-aG", "docker", getpass.getuser()], check=False)
print("Docker 安装完成 (需重新登录以生效 docker 用户组)")
+120
View File
@@ -0,0 +1,120 @@
"""filedate - 文件日期处理工具.
提供添加日期前缀/清除日期前缀 子命令.
"""
from __future__ import annotations
import re
import time
from pathlib import Path
import pyflowx as px
__all__ = [
"DATE_PATTERN",
"SEP",
"add_date_prefix",
"get_file_timestamp",
"process_file_date",
"process_files_date",
"remove_date_prefix",
]
# ============================================================================
# 配置
# ============================================================================
DATE_PATTERN = re.compile(r"(20|19)\d{2}[-_#.~]?((0[1-9])|(1[012]))[-_#.~]?((0[1-9])|([12]\d)|(3[01]))[-_#.~]?")
SEP = "_"
# ============================================================================
# 公共函数
# ============================================================================
def get_file_timestamp(filepath: Path) -> str:
"""获取文件时间戳."""
modified_time = filepath.stat().st_mtime
created_time = filepath.stat().st_ctime
return time.strftime("%Y%m%d", time.localtime(max((modified_time, created_time))))
def remove_date_prefix(filepath: Path) -> Path:
"""移除文件日期前缀."""
stem = filepath.stem
new_stem = DATE_PATTERN.sub("", stem)
if new_stem != stem:
new_path = filepath.with_name(new_stem + filepath.suffix)
filepath.rename(new_path)
return new_path
return filepath
def add_date_prefix(filepath: Path) -> Path:
"""添加文件日期前缀."""
timestamp = get_file_timestamp(filepath)
stem = filepath.stem
new_stem = f"{timestamp}{SEP}{stem}"
new_path = filepath.with_name(new_stem + filepath.suffix)
if new_path != filepath:
filepath.rename(new_path)
return new_path
return filepath
def process_file_date(filepath: Path, clear: bool = False) -> None:
"""处理单个文件的日期前缀.
Parameters
----------
filepath : Path
文件路径
clear : bool
是否清除日期前缀
"""
if clear:
remove_date_prefix(filepath)
else:
new_path = remove_date_prefix(filepath)
add_date_prefix(new_path)
@px.tool("filedate", subcommand="add", help="添加日期前缀")
def process_files_date_add(files: list[Path]) -> None:
"""添加日期前缀.
Parameters
----------
files : list[Path]
文件路径列表
"""
process_files_date(files, clear=False)
@px.tool("filedate", subcommand="clear", help="清除日期前缀")
def process_files_date_clear(files: list[Path]) -> None:
"""清除日期前缀.
Parameters
----------
files : list[Path]
文件路径列表
"""
process_files_date(files, clear=True)
def process_files_date(targets: list[Path], clear: bool = False) -> None:
"""批量处理文件日期前缀.
Parameters
----------
targets : list[Path]
文件路径列表
clear : bool
是否清除日期前缀
"""
for target in targets:
if target.exists() and not target.name.startswith("."):
process_file_date(target, clear)
+108
View File
@@ -0,0 +1,108 @@
"""filelevel - 文件等级重命名工具.
提供设置文件等级 子命令.
"""
from __future__ import annotations
from pathlib import Path
import pyflowx as px
__all__ = [
"BRACKETS",
"LEVELS",
"process_file_level",
"process_files_level",
"remove_marks",
]
# ============================================================================
# 配置
# ============================================================================
LEVELS: dict[str, str] = {
"0": "",
"1": "PUB,NOR",
"2": "INT",
"3": "CON",
"4": "CLA",
}
BRACKETS: tuple[str, str] = (" ([_(【-", " )]_)】")
# ============================================================================
# 公共函数
# ============================================================================
def remove_marks(stem: str, marks: list[str]) -> str:
"""从文件名主干中移除所有标记."""
left_brackets, right_brackets = BRACKETS
for mark in marks:
pos = 0
while True:
pos = stem.find(mark, pos)
if pos == -1:
break
b, e = pos - 1, pos + len(mark)
if b >= 0 and e < len(stem) and stem[b] in left_brackets and stem[e] in right_brackets:
stem = stem[:b] + stem[e + 1 :]
else:
pos = e
return stem
def process_file_level(filepath: Path, level: int = 0) -> None:
"""处理单个文件的等级标记.
Parameters
----------
filepath : Path
文件路径
level : int
文件等级 (0-4), 0 用于清除等级
"""
if not (0 <= level < len(LEVELS)):
print(f"无效的等级 {level}, 必须在 0 和 {len(LEVELS) - 1} 之间")
return
if not filepath.exists():
print(f"文件不存在: {filepath}")
return
filestem = filepath.stem
original_stem = filestem
for level_names in LEVELS.values():
if level_names:
filestem = remove_marks(filestem, level_names.split(","))
for digit in map(str, range(1, 10)):
filestem = remove_marks(filestem, [digit])
if level > 0:
levelstr = LEVELS.get(str(level), "").split(",")[0]
if levelstr:
filestem = f"{filestem}({levelstr})"
if filestem != original_stem:
new_path = filepath.with_name(filestem + filepath.suffix)
filepath.rename(new_path)
print(f"重命名: {filepath} -> {new_path}")
@px.tool("filelevel", subcommand="set", help="设置文件等级")
def process_files_level(files: list[Path], level: int = 0) -> None:
"""批量处理文件等级标记.
Parameters
----------
files : list[Path]
文件路径列表
level : int
文件等级 (0-4)
"""
for target in files:
process_file_level(target, level)
-327
View File
@@ -1,327 +0,0 @@
"""文件类函数模块.
聚合文件日期处理、文件等级重命名、文件夹备份、文件夹压缩工具的可复用函数.
所有公共函数通过 ``@px.register_fn`` 注册, 供 YAML 任务编排引用.
"""
from __future__ import annotations
import re
import shutil
import time
import zipfile
from pathlib import Path
import pyflowx as px
__all__ = [
"BRACKETS",
"DATE_PATTERN",
"IGNORE_DIRS",
"IGNORE_EXT",
"IGNORE_FILES",
"LEVELS",
"SEP",
"add_date_prefix",
"archive_folder",
"backup_folder",
"folderback_default",
"folderzip_default",
"get_file_timestamp",
"process_file_date",
"process_file_level",
"process_files_date",
"process_files_level",
"remove_date_prefix",
"remove_dump",
"remove_marks",
"zip_folders",
"zip_target",
]
# ============================================================================
# filedate 配置
# ============================================================================
DATE_PATTERN = re.compile(r"(20|19)\d{2}[-_#.~]?((0[1-9])|(1[012]))[-_#.~]?((0[1-9])|([12]\d)|(3[01]))[-_#.~]?")
SEP = "_"
# ============================================================================
# filelevel 配置
# ============================================================================
LEVELS: dict[str, str] = {
"0": "",
"1": "PUB,NOR",
"2": "INT",
"3": "CON",
"4": "CLA",
}
BRACKETS: tuple[str, str] = (" ([_(【-", " )]_)】")
# ============================================================================
# folderzip 配置
# ============================================================================
IGNORE_DIRS: list[str] = [".git", ".idea", ".vscode", "__pycache__"]
IGNORE_FILES: list[str] = [".gitignore"]
IGNORE: list[str] = [*IGNORE_DIRS, *IGNORE_FILES]
IGNORE_EXT: list[str] = [".zip", ".rar", ".7z", ".tar", ".gz"]
# ============================================================================
# filedate 函数
# ============================================================================
@px.register_fn
def get_file_timestamp(filepath: Path) -> str:
"""获取文件时间戳."""
modified_time = filepath.stat().st_mtime
created_time = filepath.stat().st_ctime
return time.strftime("%Y%m%d", time.localtime(max((modified_time, created_time))))
@px.register_fn
def remove_date_prefix(filepath: Path) -> Path:
"""移除文件日期前缀."""
stem = filepath.stem
new_stem = DATE_PATTERN.sub("", stem)
if new_stem != stem:
new_path = filepath.with_name(new_stem + filepath.suffix)
filepath.rename(new_path)
return new_path
return filepath
@px.register_fn
def add_date_prefix(filepath: Path) -> Path:
"""添加文件日期前缀."""
timestamp = get_file_timestamp(filepath)
stem = filepath.stem
new_stem = f"{timestamp}{SEP}{stem}"
new_path = filepath.with_name(new_stem + filepath.suffix)
if new_path != filepath:
filepath.rename(new_path)
return new_path
return filepath
@px.register_fn
def process_file_date(filepath: Path, clear: bool = False) -> None:
"""处理单个文件的日期前缀.
Parameters
----------
filepath : Path
文件路径
clear : bool
是否清除日期前缀
"""
if clear:
remove_date_prefix(filepath)
else:
new_path = remove_date_prefix(filepath)
add_date_prefix(new_path)
@px.register_fn
def process_files_date(targets: list[Path], clear: bool = False) -> None:
"""批量处理文件日期前缀.
Parameters
----------
targets : list[Path]
文件路径列表
clear : bool
是否清除日期前缀
"""
for target in targets:
if target.exists() and not target.name.startswith("."):
process_file_date(target, clear)
# ============================================================================
# filelevel 函数
# ============================================================================
@px.register_fn
def remove_marks(stem: str, marks: list[str]) -> str:
"""从文件名主干中移除所有标记."""
left_brackets, right_brackets = BRACKETS
for mark in marks:
pos = 0
while True:
pos = stem.find(mark, pos)
if pos == -1:
break
b, e = pos - 1, pos + len(mark)
if b >= 0 and e < len(stem) and stem[b] in left_brackets and stem[e] in right_brackets:
stem = stem[:b] + stem[e + 1 :]
else:
pos = e
return stem
@px.register_fn
def process_file_level(filepath: Path, level: int = 0) -> None:
"""处理单个文件的等级标记.
Parameters
----------
filepath : Path
文件路径
level : int
文件等级 (0-4), 0 用于清除等级
"""
if not (0 <= level < len(LEVELS)):
print(f"无效的等级 {level}, 必须在 0 和 {len(LEVELS) - 1} 之间")
return
if not filepath.exists():
print(f"文件不存在: {filepath}")
return
filestem = filepath.stem
original_stem = filestem
for level_names in LEVELS.values():
if level_names:
filestem = remove_marks(filestem, level_names.split(","))
for digit in map(str, range(1, 10)):
filestem = remove_marks(filestem, [digit])
if level > 0:
levelstr = LEVELS.get(str(level), "").split(",")[0]
if levelstr:
filestem = f"{filestem}({levelstr})"
if filestem != original_stem:
new_path = filepath.with_name(filestem + filepath.suffix)
filepath.rename(new_path)
print(f"重命名: {filepath} -> {new_path}")
@px.register_fn
def process_files_level(targets: list[Path], level: int = 0) -> None:
"""批量处理文件等级标记.
Parameters
----------
targets : list[Path]
文件路径列表
level : int
文件等级 (0-4)
"""
for target in targets:
process_file_level(target, level)
# ============================================================================
# folderback 函数
# ============================================================================
@px.register_fn
def remove_dump(src: Path, dst: Path, max_zip: int) -> None:
"""递归删除旧的备份 zip 文件."""
zip_paths = [filepath for filepath in dst.rglob("*.zip") if src.stem in str(filepath)]
zip_files = sorted(zip_paths, key=lambda fn: str(fn)[-19:-4])
if len(zip_files) > max_zip:
zip_files[0].unlink()
remove_dump(src, dst, max_zip)
@px.register_fn
def zip_target(src: Path, dst: Path, max_zip: int) -> None:
"""将单个文件或文件夹压缩为 zip 文件."""
files = [str(_) for _ in src.rglob("*")]
timestamp = time.strftime("_%Y%m%d_%H%M%S")
target_path = dst / (src.stem + timestamp + ".zip")
with zipfile.ZipFile(target_path, "w") as zip_file:
for file in files:
zip_file.write(file, arcname=file.replace(str(src.parent), ""))
remove_dump(src, dst, max_zip)
print(f"备份完成: {target_path}")
@px.register_fn
def backup_folder(src: str, dst: str, max_zip: int = 5) -> None:
"""备份文件夹.
Parameters
----------
src : str
源文件夹路径
dst : str
目标文件夹路径
max_zip : int
最大备份数量
"""
src_path = Path(src)
dst_path = Path(dst)
if not src_path.exists():
print(f"源文件夹不存在: {src_path}")
return
if not dst_path.exists():
dst_path.mkdir(parents=True, exist_ok=True)
print(f"创建目标文件夹: {dst_path}")
zip_target(src_path, dst_path, max_zip)
@px.register_fn("folderback_default")
def folderback_default() -> None:
"""备份当前目录到 ./backup."""
backup_folder(".", "./backup", 5)
# ============================================================================
# folderzip 函数
# ============================================================================
@px.register_fn
def archive_folder(folder: Path) -> None:
"""压缩单个文件夹."""
shutil.make_archive(
str(folder.with_name(folder.name)),
format="zip",
base_dir=folder,
)
print(f"压缩完成: {folder.name}.zip")
@px.register_fn
def zip_folders(cwd: str = ".") -> None:
"""压缩目录下的所有文件夹.
Parameters
----------
cwd : str
工作目录
"""
cwd_path = Path(cwd)
if not cwd_path.exists():
print(f"目录不存在: {cwd_path}")
return
dirs: list[Path] = [
e for e in cwd_path.iterdir() if e.is_dir() and e.name not in IGNORE_DIRS and e.suffix not in IGNORE_EXT
]
for dir_path in dirs:
archive_folder(dir_path)
@px.register_fn("folderzip_default")
def folderzip_default() -> None:
"""压缩当前目录下的所有文件夹."""
zip_folders(".")
+73
View File
@@ -0,0 +1,73 @@
"""folderback - 文件夹备份工具.
备份当前文件夹到指定目录, 自动清理旧备份.
"""
from __future__ import annotations
import time
import zipfile
from pathlib import Path
import pyflowx as px
__all__ = [
"backup_folder",
"remove_dump",
"zip_target",
]
# ============================================================================
# 公共函数
# ============================================================================
def remove_dump(src: Path, dst: Path, max_zip: int) -> None:
"""递归删除旧的备份 zip 文件."""
zip_paths = [filepath for filepath in dst.rglob("*.zip") if src.stem in str(filepath)]
zip_files = sorted(zip_paths, key=lambda fn: str(fn)[-19:-4])
if len(zip_files) > max_zip:
zip_files[0].unlink()
remove_dump(src, dst, max_zip)
def zip_target(src: Path, dst: Path, max_zip: int) -> None:
"""将单个文件或文件夹压缩为 zip 文件."""
files = [str(_) for _ in src.rglob("*")]
timestamp = time.strftime("_%Y%m%d_%H%M%S")
target_path = dst / (src.stem + timestamp + ".zip")
with zipfile.ZipFile(target_path, "w") as zip_file:
for file in files:
zip_file.write(file, arcname=file.replace(str(src.parent), ""))
remove_dump(src, dst, max_zip)
print(f"备份完成: {target_path}")
@px.tool("folderback", help="备份文件夹")
def backup_folder(src: str = ".", dst: str = "./backup", max_zip: int = 5) -> None:
"""备份文件夹.
Parameters
----------
src : str
源文件夹路径 (默认: 当前目录)
dst : str
目标文件夹路径 (默认: ./backup)
max_zip : int
最大备份数量 (默认: 5)
"""
src_path = Path(src)
dst_path = Path(dst)
if not src_path.exists():
print(f"源文件夹不存在: {src_path}")
return
if not dst_path.exists():
dst_path.mkdir(parents=True, exist_ok=True)
print(f"创建目标文件夹: {dst_path}")
zip_target(src_path, dst_path, max_zip)
+64
View File
@@ -0,0 +1,64 @@
"""folderzip - 文件夹压缩工具.
压缩当前目录下的所有子文件夹为 zip 文件.
"""
from __future__ import annotations
import shutil
from pathlib import Path
import pyflowx as px
__all__ = [
"IGNORE_DIRS",
"IGNORE_EXT",
"IGNORE_FILES",
"archive_folder",
"zip_folders",
]
# ============================================================================
# 配置
# ============================================================================
IGNORE_DIRS: list[str] = [".git", ".idea", ".vscode", "__pycache__"]
IGNORE_FILES: list[str] = [".gitignore"]
IGNORE_EXT: list[str] = [".zip", ".rar", ".7z", ".tar", ".gz"]
# ============================================================================
# 公共函数
# ============================================================================
def archive_folder(folder: Path) -> None:
"""压缩单个文件夹."""
shutil.make_archive(
str(folder.with_name(folder.name)),
format="zip",
base_dir=folder,
)
print(f"压缩完成: {folder.name}.zip")
@px.tool("folderzip", help="压缩文件夹")
def zip_folders(cwd: str = ".") -> None:
"""压缩目录下的所有文件夹.
Parameters
----------
cwd : str
工作目录 (默认: 当前目录)
"""
cwd_path = Path(cwd)
if not cwd_path.exists():
print(f"目录不存在: {cwd_path}")
return
dirs: list[Path] = [
e for e in cwd_path.iterdir() if e.is_dir() and e.name not in IGNORE_DIRS and e.suffix not in IGNORE_EXT
]
for dir_path in dirs:
archive_folder(dir_path)
+142
View File
@@ -0,0 +1,142 @@
"""gittool - Git 执行工具.
提供添加提交/清理/初始化/推送/拉取 子命令.
"""
from __future__ import annotations
import subprocess
from pathlib import Path
import pyflowx as px
EXCLUDE_DIRS = [
# 编辑器相关目录
".vscode",
".idea",
".editorconfig",
".trae",
".qoder",
# 项目相关目录
".venv",
".git",
".ruff_cache",
".tox",
"node_modules",
]
EXCLUDE_CMDS = [arg for d in EXCLUDE_DIRS for arg in ["-e", d]]
# ============================================================================
# 私有辅助函数
# ============================================================================
def not_has_git_repo() -> bool:
"""检查当前目录没有 Git 仓库."""
return not Path.cwd().exists() or not (Path.cwd() / ".git").is_dir()
def has_files() -> bool:
"""检查当前 Git 仓库是否有未提交的更改."""
try:
result = subprocess.run(
["git", "status", "--porcelain"],
capture_output=True,
text=True,
check=False,
)
return bool(result.stdout.strip())
except (subprocess.SubprocessError, OSError):
return False
# ============================================================================
# fn 子命令
# ============================================================================
@px.tool("gittool", subcommand="a", help="添加并提交")
def git_add_commit(message: str = "chore: update") -> None:
"""执行 git add + git commit (仅当有未提交更改时).
Parameters
----------
message : str
提交信息 (默认: ``chore: update``)
"""
if not has_files():
print("没有文件需要提交")
return
subprocess.run(["git", "add", "."], check=True)
subprocess.run(["git", "commit", "-m", message], check=True)
@px.tool("gittool", subcommand="i", help="初始化并提交")
def git_init_add_commit(message: str = "init commit") -> None:
"""执行 git init (若需) + git add + git commit (若有更改).
Parameters
----------
message : str
提交信息 (默认: ``init commit``)
"""
if not_has_git_repo():
subprocess.run(["git", "init"], check=True)
if has_files():
subprocess.run(["git", "add", "."], check=True)
subprocess.run(["git", "commit", "-m", message], check=True)
else:
print("没有文件需要提交")
@px.tool("gittool", subcommand="isub", help="初始化子目录")
def init_sub_dirs() -> None:
"""初始化子目录的 Git 仓库."""
sub_dirs = [subdir for subdir in Path.cwd().iterdir() if subdir.is_dir()]
for subdir in sub_dirs:
px.run(
px.Graph().chain(
px.cmd(["git", "init"], conditions=(lambda _: not_has_git_repo(),), cwd=subdir),
px.cmd(["git", "add", "."]),
px.cmd(["git", "commit", "-m", "init commit"]),
),
)
# ============================================================================
# cmd 子命令
# ============================================================================
@px.tool(
"gittool",
subcommand="clean",
help="清理未跟踪文件",
cmd=["git", "clean", "-xfd", *EXCLUDE_CMDS],
hidden=True,
)
def clean() -> None:
"""清理 Git 未跟踪文件 (隐藏命令, 被 c 依赖)."""
@px.tool(
"gittool",
subcommand="c",
help="清理并查看状态",
cmd=["git", "status", "--porcelain"],
needs=["clean"],
)
def c() -> None:
"""清理未跟踪文件并查看 Git 状态."""
@px.tool("gittool", subcommand="p", help="推送", cmd=["git", "push"])
def p() -> None:
"""推送代码到远程仓库."""
@px.tool("gittool", subcommand="pl", help="拉取", cmd=["git", "pull"])
def pl() -> None:
"""从远程仓库拉取代码."""
+111
View File
@@ -0,0 +1,111 @@
"""lscalc - LS-DYNA 计算工具.
运行 LS-DYNA 计算 (单机/MPI) 与进程状态检查.
"""
from __future__ import annotations
import subprocess
from pathlib import Path
import pyflowx as px
from pyflowx.conditions import Constants
_DEFAULT_NCPU: int = 4
def get_ls_dyna_command(input_file: str, ncpu: int) -> list[str]:
"""获取 LS-DYNA 命令.
Parameters
----------
input_file : str
输入文件路径
ncpu : int
CPU 核心数
Returns
-------
list[str]
LS-DYNA 命令列表
"""
return ["ls-dyna_mpp", f"i={input_file}", f"ncpu={ncpu}"]
@px.tool("lscalc", subcommand="run", help="运行 LS-DYNA 计算")
def run_ls_dyna(input_file: str, ncpu: int = _DEFAULT_NCPU) -> None:
"""运行 LS-DYNA 计算.
Parameters
----------
input_file : str
输入文件路径
ncpu : int
CPU 核心数
"""
input_path = Path(input_file)
if not input_path.exists():
print(f"输入文件不存在: {input_path}")
return
cmd = get_ls_dyna_command(input_file, ncpu)
try:
subprocess.run(cmd, check=True)
print(f"LS-DYNA 计算完成: {input_file}")
except FileNotFoundError:
print("未找到 ls-dyna_mpp 命令")
except subprocess.CalledProcessError as e:
print(f"LS-DYNA 计算失败: {e}")
@px.tool("lscalc", subcommand="mpi", help="运行 LS-DYNA MPI 计算")
def run_ls_dyna_mpi(input_file: str, ncpu: int = _DEFAULT_NCPU) -> None:
"""运行 LS-DYNA MPI 计算.
Parameters
----------
input_file : str
输入文件路径
ncpu : int
CPU 核心数
"""
input_path = Path(input_file)
if not input_path.exists():
print(f"输入文件不存在: {input_path}")
return
cmd = ["mpirun", "-np", str(ncpu), "ls-dyna_mpp", f"i={input_file}"]
try:
subprocess.run(cmd, check=True)
print(f"LS-DYNA MPI 计算完成: {input_file}")
except FileNotFoundError:
print("未找到 mpirun 或 ls-dyna_mpp 命令")
except subprocess.CalledProcessError as e:
print(f"LS-DYNA MPI 计算失败: {e}")
@px.tool("lscalc", subcommand="status", help="检查 LS-DYNA 进程状态")
def check_ls_dyna_status() -> None:
"""检查 LS-DYNA 进程状态."""
try:
if Constants.IS_WINDOWS:
result = subprocess.run(
["tasklist", "/fi", "imagename eq ls-dyna_mpp.exe"],
capture_output=True,
text=True,
check=True,
)
print(result.stdout)
else:
result = subprocess.run(
["pgrep", "-f", "ls-dyna"],
capture_output=True,
text=True,
check=False,
)
if result.stdout.strip():
print(f"运行中的 LS-DYNA 进程 PID: {result.stdout.strip()}")
else:
print("没有运行中的 LS-DYNA 进程")
except subprocess.CalledProcessError as e:
print(f"检查进程状态失败: {e}")
+41
View File
@@ -0,0 +1,41 @@
"""msdownload - ModelScope 模型/数据集下载工具.
从 ModelScope 下载模型、数据集或空间.
"""
from __future__ import annotations
import subprocess
from pathlib import Path
import pyflowx as px
__all__ = ["msdownload_run"]
@px.tool("msdownload", help="ModelScope 模型/数据集下载工具")
def msdownload_run(name: str, target_type: str = "model", download_dir: str | None = None) -> None:
"""从 ModelScope 下载模型/数据集/空间.
Parameters
----------
name : str
目标名称 (如: ``Qwen/Qwen2.5-Coder-32B-Instruct``)
target_type : str
目标类型: ``model`` / ``dataset`` / ``space`` (默认: ``model``)
download_dir : str | None
下载目录; 为 None 时默认 ``~/.models/<name 最后一段>``
"""
if not name:
print("msdownload: name 不能为空")
return
if download_dir:
out_dir = Path(download_dir)
else:
out_dir = Path.home() / ".models" / name.rsplit("/", 1)[-1]
out_dir.mkdir(parents=True, exist_ok=True)
cmd = ["uvx", "modelscope", "download", f"--{target_type}", name, "--local_dir", str(out_dir)]
print(f"下载 {target_type}: {name} -> {out_dir}")
subprocess.run(cmd, check=False)
+236
View File
@@ -0,0 +1,236 @@
"""packtool - Python 打包工具.
提供源码打包/依赖打包/wheel 构建/嵌入式 Python 安装/zip 包创建/清理 子命令.
"""
from __future__ import annotations
import platform
import shutil
import subprocess
import urllib.request
import zipfile
from pathlib import Path
import pyflowx as px
from pyflowx.ops._common import IGNORE_PATTERNS
__all__ = [
"clean_build_dir",
"create_zip_package",
"install_embed_python",
"pack_dependencies",
"pack_source",
"pack_wheel",
]
# ============================================================================
# 配置
# ============================================================================
DEFAULT_BUILD_DIR = ".pypack"
DEFAULT_DIST_DIR = "dist"
DEFAULT_LIB_DIR = "libs"
DEFAULT_CACHE_DIR = ".cache/pypack"
# ============================================================================
# 公共函数
# ============================================================================
@px.tool("packtool", subcommand="src", help="打包源码")
def pack_source(project_dir: Path = Path(), output_dir: Path = Path(".pypack")) -> None:
"""打包项目源码.
Parameters
----------
project_dir : Path
项目目录 (默认: 当前目录)
output_dir : Path
输出目录 (默认: .pypack)
"""
output_dir.mkdir(parents=True, exist_ok=True)
pyproject_file = project_dir / "pyproject.toml"
project_name = project_dir.name
if pyproject_file.exists():
try:
import tomllib
content = pyproject_file.read_text(encoding="utf-8")
data = tomllib.loads(content)
project_name = data.get("project", {}).get("name", project_name)
except ImportError:
pass
source_dir = output_dir / "src" / project_name
source_dir.mkdir(parents=True, exist_ok=True)
src_subdir = project_dir / "src"
if src_subdir.exists():
shutil.copytree(
src_subdir,
source_dir / "src",
ignore=shutil.ignore_patterns(*IGNORE_PATTERNS),
dirs_exist_ok=True,
)
else:
for item in project_dir.iterdir():
if item.name in IGNORE_PATTERNS or item.name.startswith("."):
continue
dst_item = source_dir / item.name
if item.is_dir():
shutil.copytree(
item,
dst_item,
ignore=shutil.ignore_patterns(*IGNORE_PATTERNS),
dirs_exist_ok=True,
)
else:
shutil.copy2(item, dst_item)
print(f"源码打包完成: {source_dir}")
@px.tool("packtool", subcommand="deps", help="打包依赖")
def pack_dependencies(lib_dir: Path = Path("libs"), dependencies: list[str] | None = None) -> None:
"""打包项目依赖.
Parameters
----------
lib_dir : Path
依赖库目录 (默认: libs)
dependencies : list[str] | None
依赖列表
"""
if dependencies is None:
dependencies = []
lib_dir.mkdir(parents=True, exist_ok=True)
if not dependencies:
print("没有依赖需要打包")
return
cmd = [
"pip",
"install",
"--target",
str(lib_dir),
"--no-compile",
"--no-warn-script-location",
]
cmd.extend(dependencies)
subprocess.run(cmd, check=True)
print(f"依赖打包完成: {lib_dir}")
@px.tool("packtool", subcommand="wheel", help="构建 wheel")
def pack_wheel(project_dir: Path = Path(), output_dir: Path = Path("dist")) -> None:
"""打包项目为 wheel 文件.
Parameters
----------
project_dir : Path
项目目录 (默认: 当前目录)
output_dir : Path
输出目录 (默认: dist)
"""
output_dir.mkdir(parents=True, exist_ok=True)
cmd = [
"pip",
"wheel",
"--no-deps",
"--wheel-dir",
str(output_dir),
str(project_dir),
]
subprocess.run(cmd, check=True)
print(f"Wheel 打包完成: {output_dir}")
@px.tool("packtool", subcommand="embed", help="安装嵌入式 Python")
def install_embed_python(version: str = "3.10", output_dir: Path = Path("python")) -> None:
"""安装嵌入式 Python.
Parameters
----------
version : str
Python 版本 (如: 3.10, 3.11), 默认 3.10
output_dir : Path
输出目录 (默认: python)
"""
output_dir.mkdir(parents=True, exist_ok=True)
arch = platform.machine().lower()
if arch in ["x86_64", "amd64"]:
arch = "amd64"
elif arch in ["arm64", "aarch64"]:
arch = "arm64"
version_map = {
"3.8": "3.8.10",
"3.9": "3.9.13",
"3.10": "3.10.11",
"3.11": "3.11.9",
"3.12": "3.12.4",
}
full_version = version_map.get(version, f"{version}.0")
url = f"https://www.python.org/ftp/python/{full_version}/python-{full_version}-embed-{arch}.zip"
cache_file = Path(DEFAULT_CACHE_DIR) / f"python-{full_version}-embed-{arch}.zip"
cache_file.parent.mkdir(parents=True, exist_ok=True)
if not cache_file.exists():
print(f"正在下载嵌入式 Python {full_version}...")
urllib.request.urlretrieve(url, cache_file)
print(f"下载完成: {cache_file}")
with zipfile.ZipFile(cache_file, "r") as zf:
zf.extractall(output_dir)
print(f"嵌入式 Python 安装完成: {output_dir}")
@px.tool("packtool", subcommand="zip", help="创建 zip 包")
def create_zip_package(source_dir: Path = Path(), output_file: Path = Path("package.zip")) -> None:
"""创建 ZIP 打包文件.
Parameters
----------
source_dir : Path
源目录 (默认: 当前目录)
output_file : Path
输出文件 (默认: package.zip)
"""
output_file.parent.mkdir(parents=True, exist_ok=True)
with zipfile.ZipFile(output_file, "w", zipfile.ZIP_DEFLATED) as zf:
for file in source_dir.rglob("*"):
if file.is_file():
arcname = file.relative_to(source_dir)
zf.write(file, arcname)
print(f"ZIP 打包完成: {output_file}")
@px.tool("packtool", subcommand="clean", help="清理构建目录")
def clean_build_dir(build_dir: Path = Path(".pypack")) -> None:
"""清理构建目录.
Parameters
----------
build_dir : Path
构建目录 (默认: .pypack)
"""
if build_dir.exists():
shutil.rmtree(build_dir)
print(f"清理完成: {build_dir}")
else:
print(f"目录不存在: {build_dir}")
@@ -1,23 +1,16 @@
"""媒体类函数模块.
"""pdftool - PDF 文件工具集.
聚合 PDF 工具 (pdftool) 和截图工具 (screenshot) 的可复用函数.
所有公共函数通过 ``@px.register_fn`` 注册, YAML 任务编排引用.
提供 PDF 合并/拆分/压缩/加密/解密/提取文本/提取图片/水印/旋转/裁剪/
信息/OCR/转图片/修复 等子命令.
"""
from __future__ import annotations
import subprocess
from datetime import datetime
from pathlib import Path
import pyflowx as px
from pyflowx.conditions import Constants
__all__ = [
"DEFAULT_PASSWORD",
"DEFAULT_QUALITY",
"PDF_SUFFIX",
"get_screenshot_path",
"pdf_add_watermark",
"pdf_compress",
"pdf_crop",
@@ -33,8 +26,6 @@ __all__ = [
"pdf_rotate",
"pdf_split",
"pdf_to_images",
"take_screenshot_area",
"take_screenshot_full",
]
try:
@@ -52,25 +43,34 @@ except ImportError:
HAS_PYPDF = False
# ============================================================================
# 配置
# ============================================================================
PDF_SUFFIX = ".pdf"
DEFAULT_QUALITY = 75
DEFAULT_PASSWORD = ""
def _require_pymupdf() -> bool:
"""PyMuPDF 未安装时打印提示, 返回是否可用."""
if not HAS_PYMUPDF:
print("未安装 PyMuPDF 库, 请安装: pip install PyMuPDF")
return False
return True
# ============================================================================
# PDF 函数
# ============================================================================
@px.register_fn
def pdf_merge(input_paths: list[Path], output_path: Path) -> None:
"""合并多个 PDF 文件."""
def _require_pypdf() -> bool:
"""pypdf 未安装时打印提示, 返回是否可用."""
if not HAS_PYPDF:
print("未安装 pypdf 库, 请安装: pip install pypdf")
return False
return True
@px.tool("pdftool", subcommand="m", help="合并 PDF")
def pdf_merge(input_paths: list[Path], output_path: Path = Path("merged.pdf")) -> None:
"""合并多个 PDF 文件.
Parameters
----------
input_paths : list[Path]
输入 PDF 文件列表
output_path : Path
输出文件 (默认: merged.pdf)
"""
if not _require_pypdf():
return
writer = pypdf.PdfWriter()
@@ -87,11 +87,18 @@ def pdf_merge(input_paths: list[Path], output_path: Path) -> None:
print(f"合并完成: {output_path}")
@px.register_fn
def pdf_split(input_path: Path, output_dir: Path) -> None:
"""拆分 PDF 文件为单页."""
if not HAS_PYPDF:
print("未安装 pypdf 库, 请安装: pip install pypdf")
@px.tool("pdftool", subcommand="s", help="拆分 PDF")
def pdf_split(input_path: Path, output_dir: Path = Path("split")) -> None:
"""拆分 PDF 文件为单页.
Parameters
----------
input_path : Path
输入 PDF 文件
output_dir : Path
输出目录 (默认: split)
"""
if not _require_pypdf():
return
reader = pypdf.PdfReader(str(input_path))
@@ -107,11 +114,18 @@ def pdf_split(input_path: Path, output_dir: Path) -> None:
print(f"拆分完成: {output_dir}")
@px.register_fn
def pdf_compress(input_path: Path, output_path: Path) -> None:
"""压缩 PDF 文件."""
if not HAS_PYMUPDF:
print("未安装 PyMuPDF 库, 请安装: pip install PyMuPDF")
@px.tool("pdftool", subcommand="c", help="压缩 PDF")
def pdf_compress(input_path: Path, output_path: Path = Path("compressed.pdf")) -> None:
"""压缩 PDF 文件.
Parameters
----------
input_path : Path
输入 PDF 文件
output_path : Path
输出文件 (默认: compressed.pdf)
"""
if not _require_pymupdf():
return
doc = fitz.open(str(input_path))
@@ -125,11 +139,23 @@ def pdf_compress(input_path: Path, output_path: Path) -> None:
print(f"压缩完成: {output_path} (缩小 {ratio:.1f}%)")
@px.register_fn
def pdf_encrypt(input_path: Path, output_path: Path, password: str) -> None:
"""加密 PDF 文件."""
if not HAS_PYPDF:
print("未安装 pypdf 库, 请安装: pip install pypdf")
@px.tool("pdftool", subcommand="e", help="加密 PDF")
def pdf_encrypt(input_path: Path, output_path: Path = Path("encrypted.pdf"), password: str = "") -> None:
"""加密 PDF 文件.
Parameters
----------
input_path : Path
输入 PDF 文件
output_path : Path
输出文件 (默认: encrypted.pdf)
password : str
密码 (必填)
"""
if not password:
print("错误: --password 为必填参数")
return
if not _require_pypdf():
return
reader = pypdf.PdfReader(str(input_path))
@@ -146,11 +172,23 @@ def pdf_encrypt(input_path: Path, output_path: Path, password: str) -> None:
print(f"加密完成: {output_path}")
@px.register_fn
def pdf_decrypt(input_path: Path, output_path: Path, password: str) -> None:
"""解密 PDF 文件."""
if not HAS_PYPDF:
print("未安装 pypdf 库, 请安装: pip install pypdf")
@px.tool("pdftool", subcommand="d", help="解密 PDF")
def pdf_decrypt(input_path: Path, output_path: Path = Path("decrypted.pdf"), password: str = "") -> None:
"""解密 PDF 文件.
Parameters
----------
input_path : Path
输入 PDF 文件
output_path : Path
输出文件 (默认: decrypted.pdf)
password : str
密码 (必填)
"""
if not password:
print("错误: --password 为必填参数")
return
if not _require_pypdf():
return
reader = pypdf.PdfReader(str(input_path))
@@ -168,11 +206,18 @@ def pdf_decrypt(input_path: Path, output_path: Path, password: str) -> None:
print(f"解密完成: {output_path}")
@px.register_fn
def pdf_extract_text(input_path: Path, output_path: Path) -> None:
"""提取 PDF 文本."""
if not HAS_PYMUPDF:
print("未安装 PyMuPDF 库, 请安装: pip install PyMuPDF")
@px.tool("pdftool", subcommand="xt", help="提取文本")
def pdf_extract_text(input_path: Path, output_path: Path = Path("output.txt")) -> None:
"""提取 PDF 文本.
Parameters
----------
input_path : Path
输入 PDF 文件
output_path : Path
输出文件 (默认: output.txt)
"""
if not _require_pymupdf():
return
doc = fitz.open(str(input_path))
@@ -186,11 +231,18 @@ def pdf_extract_text(input_path: Path, output_path: Path) -> None:
print(f"文本提取完成: {output_path}")
@px.register_fn
def pdf_extract_images(input_path: Path, output_dir: Path) -> None:
"""提取 PDF 图片."""
if not HAS_PYMUPDF:
print("未安装 PyMuPDF 库, 请安装: pip install PyMuPDF")
@px.tool("pdftool", subcommand="xi", help="提取图片")
def pdf_extract_images(input_path: Path, output_dir: Path = Path("images")) -> None:
"""提取 PDF 图片.
Parameters
----------
input_path : Path
输入 PDF 文件
output_dir : Path
输出目录 (默认: images)
"""
if not _require_pymupdf():
return
doc = fitz.open(str(input_path))
@@ -213,11 +265,22 @@ def pdf_extract_images(input_path: Path, output_dir: Path) -> None:
print(f"图片提取完成: {output_dir} (共 {image_count} 张)")
@px.register_fn
def pdf_add_watermark(input_path: Path, output_path: Path, text: str = "CONFIDENTIAL") -> None:
"""添加 PDF 水印."""
if not HAS_PYMUPDF:
print("未安装 PyMuPDF 库, 请安装: pip install PyMuPDF")
@px.tool("pdftool", subcommand="w", help="添加水印")
def pdf_add_watermark(
input_path: Path, output_path: Path = Path("watermarked.pdf"), text: str = "CONFIDENTIAL"
) -> None:
"""添加 PDF 水印.
Parameters
----------
input_path : Path
输入 PDF 文件
output_path : Path
输出文件 (默认: watermarked.pdf)
text : str
水印文字 (默认: CONFIDENTIAL)
"""
if not _require_pymupdf():
return
doc = fitz.open(str(input_path))
@@ -234,11 +297,20 @@ def pdf_add_watermark(input_path: Path, output_path: Path, text: str = "CONFIDEN
print(f"水印添加完成: {output_path}")
@px.register_fn
def pdf_rotate(input_path: Path, output_path: Path, rotation: int = 90) -> None:
"""旋转 PDF 页面."""
if not HAS_PYMUPDF:
print("未安装 PyMuPDF 库, 请安装: pip install PyMuPDF")
@px.tool("pdftool", subcommand="r", help="旋转 PDF")
def pdf_rotate(input_path: Path, output_path: Path = Path("rotated.pdf"), rotation: int = 90) -> None:
"""旋转 PDF 页面.
Parameters
----------
input_path : Path
输入 PDF 文件
output_path : Path
输出文件 (默认: rotated.pdf)
rotation : int
旋转角度 (默认: 90)
"""
if not _require_pymupdf():
return
doc = fitz.open(str(input_path))
@@ -251,11 +323,24 @@ def pdf_rotate(input_path: Path, output_path: Path, rotation: int = 90) -> None:
print(f"旋转完成: {output_path}")
@px.register_fn
def pdf_crop(input_path: Path, output_path: Path, margins: tuple[int, int, int, int]) -> None:
"""裁剪 PDF 页面."""
if not HAS_PYMUPDF:
print("未安装 PyMuPDF 库, 请安装: pip install PyMuPDF")
@px.tool("pdftool", subcommand="crop", help="裁剪 PDF")
def pdf_crop(
input_path: Path,
output_path: Path = Path("cropped.pdf"),
margins: tuple[int, int, int, int] = (10, 10, 10, 10),
) -> None:
"""裁剪 PDF 页面.
Parameters
----------
input_path : Path
输入 PDF 文件
output_path : Path
输出文件 (默认: cropped.pdf)
margins : tuple[int, int, int, int]
边距 (, , , ), 默认 (10, 10, 10, 10)
"""
if not _require_pymupdf():
return
doc = fitz.open(str(input_path))
@@ -277,11 +362,16 @@ def pdf_crop(input_path: Path, output_path: Path, margins: tuple[int, int, int,
print(f"裁剪完成: {output_path}")
@px.register_fn
@px.tool("pdftool", subcommand="i", help="查看 PDF 信息")
def pdf_info(input_path: Path) -> None:
"""显示 PDF 信息."""
if not HAS_PYMUPDF:
print("未安装 PyMuPDF 库, 请安装: pip install PyMuPDF")
"""显示 PDF 信息.
Parameters
----------
input_path : Path
输入 PDF 文件
"""
if not _require_pymupdf():
return
doc = fitz.open(str(input_path))
@@ -299,9 +389,19 @@ def pdf_info(input_path: Path) -> None:
doc.close()
@px.register_fn
def pdf_ocr(input_path: Path, output_path: Path, lang: str = "chi_sim+eng") -> None:
"""PDF OCR 识别."""
@px.tool("pdftool", subcommand="ocr", help="PDF OCR 识别")
def pdf_ocr(input_path: Path, output_path: Path = Path("ocr.pdf"), lang: str = "chi_sim+eng") -> None:
"""PDF OCR 识别.
Parameters
----------
input_path : Path
输入 PDF 文件
output_path : Path
输出文件 (默认: ocr.pdf)
lang : str
识别语言 (默认: chi_sim+eng)
"""
try:
import pytesseract
from PIL import Image
@@ -309,8 +409,7 @@ def pdf_ocr(input_path: Path, output_path: Path, lang: str = "chi_sim+eng") -> N
print("未安装 OCR 相关库, 请安装: pip install pytesseract pillow")
return
if not HAS_PYMUPDF:
print("未安装 PyMuPDF 库, 请安装: pip install PyMuPDF")
if not _require_pymupdf():
return
doc = fitz.open(str(input_path))
@@ -334,11 +433,19 @@ def pdf_ocr(input_path: Path, output_path: Path, lang: str = "chi_sim+eng") -> N
print(f"OCR 识别完成: {output_path}")
@px.register_fn
def pdf_reorder(input_path: Path, output_path: Path, order: list[int]) -> None:
"""重排 PDF 页面顺序."""
if not HAS_PYPDF:
print("未安装 pypdf 库, 请安装: pip install pypdf")
"""重排 PDF 页面顺序.
Parameters
----------
input_path : Path
输入 PDF 文件
output_path : Path
输出文件
order : list[int]
页面顺序列表 (0-based)
"""
if not _require_pypdf():
return
reader = pypdf.PdfReader(str(input_path))
@@ -355,11 +462,20 @@ def pdf_reorder(input_path: Path, output_path: Path, order: list[int]) -> None:
print(f"重排完成: {output_path}")
@px.register_fn
def pdf_to_images(input_path: Path, output_dir: Path, dpi: int = 300) -> None:
"""PDF 转图片."""
if not HAS_PYMUPDF:
print("未安装 PyMuPDF 库, 请安装: pip install PyMuPDF")
@px.tool("pdftool", subcommand="img", help="PDF 转图片")
def pdf_to_images(input_path: Path, output_dir: Path = Path("images"), dpi: int = 300) -> None:
"""PDF 转图片.
Parameters
----------
input_path : Path
输入 PDF 文件
output_dir : Path
输出目录 (默认: images)
dpi : int
DPI (默认: 300)
"""
if not _require_pymupdf():
return
doc = fitz.open(str(input_path))
@@ -375,11 +491,18 @@ def pdf_to_images(input_path: Path, output_dir: Path, dpi: int = 300) -> None:
print(f"转换完成: {output_dir}")
@px.register_fn
def pdf_repair(input_path: Path, output_path: Path) -> None:
"""修复 PDF 文件."""
if not HAS_PYMUPDF:
print("未安装 PyMuPDF 库, 请安装: pip install PyMuPDF")
@px.tool("pdftool", subcommand="repair", help="修复 PDF")
def pdf_repair(input_path: Path, output_path: Path = Path("repaired.pdf")) -> None:
"""修复 PDF 文件.
Parameters
----------
input_path : Path
输入 PDF 文件
output_path : Path
输出文件 (默认: repaired.pdf)
"""
if not _require_pymupdf():
return
doc = fitz.open(str(input_path))
@@ -387,112 +510,3 @@ def pdf_repair(input_path: Path, output_path: Path) -> None:
doc.save(str(output_path), garbage=4, deflate=True, clean=True)
doc.close()
print(f"修复完成: {output_path}")
# ============================================================================
# screenshot 函数
# ============================================================================
@px.register_fn
def get_screenshot_path(filename: str | None = None) -> Path:
"""获取截图保存路径.
Parameters
----------
filename : str | None
文件名, 如果为 None 则自动生成
Returns
-------
Path
截图保存路径
"""
if filename is None:
timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
filename = f"screenshot_{timestamp}.png"
screenshots_dir = Path.home() / "Pictures" / "screenshots"
screenshots_dir.mkdir(parents=True, exist_ok=True)
return screenshots_dir / filename
@px.register_fn
def take_screenshot_full(filename: str | None = None) -> None:
"""全屏截图.
Parameters
----------
filename : str | None
文件名
"""
output_path = get_screenshot_path(filename)
if Constants.IS_WINDOWS:
ps_script = f"""
Add-Type -AssemblyName System.Windows.Forms
Add-Type -AssemblyName System.Drawing
$screen = [System.Windows.Forms.Screen]::PrimaryScreen
$bounds = $screen.Bounds
$bitmap = New-Object System.Drawing.Bitmap $bounds.Width, $bounds.Height
$graphics = [System.Drawing.Graphics]::FromImage($bitmap)
$graphics.CopyFromScreen($bounds.Location, [System.Drawing.Point]::Empty, $bounds.Size)
$bitmap.Save('{output_path.as_posix()}')
$graphics.Dispose()
$bitmap.Dispose()
"""
subprocess.run(["powershell", "-Command", ps_script], check=True)
elif Constants.IS_MACOS:
subprocess.run(["screencapture", "-x", str(output_path)], check=True)
else:
try:
subprocess.run(["gnome-screenshot", "-f", str(output_path)], check=True)
except FileNotFoundError:
subprocess.run(["scrot", str(output_path)], check=True)
print(f"截图已保存: {output_path}")
@px.register_fn
def take_screenshot_area(filename: str | None = None) -> None:
"""区域截图.
Parameters
----------
filename : str | None
文件名
"""
output_path = get_screenshot_path(filename)
if Constants.IS_WINDOWS:
ps_script = f"""
Add-Type -AssemblyName System.Windows.Forms
Add-Type -AssemblyName System.Drawing
$form = New-Object System.Windows.Forms.Form
$form.WindowState = 'Maximized'
$form.FormBorderStyle = 'None'
$form.BackColor = [System.Drawing.Color]::FromArgb(1, 0, 0)
$form.Opacity = 0.5
$form.TopMost = $true
$form.Show()
Start-Sleep -Milliseconds 100
$screen = [System.Windows.Forms.Screen]::PrimaryScreen
$bounds = $screen.Bounds
$bitmap = New-Object System.Drawing.Bitmap $bounds.Width, $bounds.Height
$graphics = [System.Drawing.Graphics]::FromImage($bitmap)
$graphics.CopyFromScreen($bounds.Location, [System.Drawing.Point]::Empty, $bounds.Size)
$form.Close()
$bitmap.Save('{output_path.as_posix()}')
$graphics.Dispose()
$bitmap.Dispose()
"""
subprocess.run(["powershell", "-Command", ps_script], check=True)
elif Constants.IS_MACOS:
subprocess.run(["screencapture", "-i", str(output_path)], check=True)
else:
try:
subprocess.run(["gnome-screenshot", "-a", "-f", str(output_path)], check=True)
except FileNotFoundError:
subprocess.run(["scrot", "-s", str(output_path)], check=True)
print(f"截图已保存: {output_path}")
+177
View File
@@ -0,0 +1,177 @@
"""piptool - pip 包管理工具.
提供安装/卸载/重装/下载/升级 pip/冻结依赖 子命令.
"""
from __future__ import annotations
import fnmatch
import subprocess
from pathlib import Path
import pyflowx as px
__all__ = [
"pip_download",
"pip_freeze",
"pip_install",
"pip_reinstall",
"pip_uninstall",
"pip_upgrade",
]
# ============================================================================
# 配置
# ============================================================================
PACKAGE_DIR = "packages"
REQUIREMENTS_FILE = "requirements.txt"
_PROTECTED_PACKAGES: frozenset[str] = frozenset(
{
"pyflowx",
"bitool",
}
)
# ============================================================================
# 私有辅助函数
# ============================================================================
def _get_installed_packages() -> list[str]:
"""获取当前环境中所有已安装的包名."""
try:
result = subprocess.run(
["pip", "list", "--format=freeze"],
capture_output=True,
text=True,
check=True,
)
packages: list[str] = []
for line in result.stdout.strip().split("\n"):
if line and "==" in line:
pkg_name = line.split("==")[0].strip()
packages.append(pkg_name)
except (subprocess.SubprocessError, OSError):
return []
return packages
def _expand_wildcard_packages(pattern: str) -> list[str]:
"""展开通配符模式为实际的包名列表."""
if not any(char in pattern for char in ["*", "?", "[", "]"]):
return [pattern]
installed_packages = _get_installed_packages()
matched = [pkg for pkg in installed_packages if fnmatch.fnmatchcase(pkg.lower(), pattern.lower())]
return matched
def _filter_protected_packages(packages: list[str]) -> list[str]:
"""过滤掉受保护的包名."""
safe = [p for p in packages if p.lower() not in {p.lower() for p in _PROTECTED_PACKAGES}]
filtered = [p for p in packages if p.lower() in {p.lower() for p in _PROTECTED_PACKAGES}]
if filtered:
print(f"跳过受保护的包: {', '.join(filtered)}")
return safe
# ============================================================================
# 公共函数
# ============================================================================
@px.tool("piptool", subcommand="i", help="安装包")
def pip_install(packages: list[str]) -> None:
"""安装包.
Parameters
----------
packages : list[str]
包名列表
"""
subprocess.run(["pip", "install", *packages], check=True)
print(f"安装完成: {', '.join(packages)}")
@px.tool("piptool", subcommand="u", help="卸载包")
def pip_uninstall(packages: list[str]) -> None:
"""卸载包.
Parameters
----------
packages : list[str]
包名列表
"""
packages_to_uninstall: list[str] = []
for pattern in packages:
packages_to_uninstall.extend(_expand_wildcard_packages(pattern))
packages_to_uninstall = _filter_protected_packages(packages_to_uninstall)
if not packages_to_uninstall:
return
subprocess.run(["pip", "uninstall", "-y", *packages_to_uninstall], check=True)
@px.tool("piptool", subcommand="r", help="重装包")
def pip_reinstall(packages: list[str], offline: bool = False) -> None:
"""重新安装包.
Parameters
----------
packages : list[str]
包名列表
offline : bool
离线模式
"""
safe_ps = _filter_protected_packages(packages)
if not safe_ps:
print("所有指定的包均为受保护包, 跳过重装")
return
subprocess.run(["pip", "uninstall", "-y", *safe_ps], check=True)
options = ["--no-index", "--find-links", "."] if offline else []
subprocess.run(["pip", "install", *options, *safe_ps], check=True)
@px.tool("piptool", subcommand="d", help="下载包")
def pip_download(packages: list[str], offline: bool = False) -> None:
"""下载包.
Parameters
----------
packages : list[str]
包名列表
offline : bool
离线模式
"""
options = ["--no-index", "--find-links", "."] if offline else []
subprocess.run(
["pip", "download", *packages, *options, "-d", PACKAGE_DIR],
check=True,
)
@px.tool("piptool", subcommand="up", help="升级 pip")
def pip_upgrade() -> None:
"""升级 pip 到最新版本."""
subprocess.run(["python", "-m", "pip", "install", "--upgrade", "pip"], check=True)
print("pip 升级完成")
@px.tool("piptool", subcommand="f", help="导出依赖")
def pip_freeze() -> None:
"""冻结依赖到 requirements.txt."""
result = subprocess.run(
["pip", "freeze", "--exclude-editable"],
capture_output=True,
text=True,
check=True,
)
Path(REQUIREMENTS_FILE).write_text(result.stdout)
print(f"依赖已导出到 {REQUIREMENTS_FILE}")
+243
View File
@@ -0,0 +1,243 @@
"""pymake - 项目构建工具.
提供构建/测试/清理/文档/格式化/发布等子命令.
"""
from __future__ import annotations
__all__ = [
"b",
"ba",
"bc",
"bump",
"bumpmi",
"bumpversion",
"c",
"cov",
"doc",
"git_add_all",
"git_push",
"git_push_tags",
"lint",
"p",
"pb",
"publish_python",
"pyrefly_check",
"sync",
"t",
"tc",
"test_coverage",
"tf",
"tox",
"twine_publish",
]
from pathlib import Path
import pyflowx as px
# ============================================================================
# 单任务别名 (cmd 任务)
# ============================================================================
@px.tool("pymake", subcommand="b", help="构建 Python 主包 (uv build)", cmd=["uv", "build"])
def b(cwd: Path = Path()) -> None:
"""构建 Python 主包."""
@px.tool("pymake", subcommand="bc", help="构建 Rust 核心模块 (maturin build)", cmd=["maturin", "build", "-r"])
def bc(cwd: Path = Path()) -> None:
"""构建 Rust 核心模块."""
@px.tool("pymake", subcommand="sync", help="同步依赖 (uv sync)", cmd=["uv", "sync"])
def sync(cwd: Path = Path()) -> None:
"""同步依赖."""
@px.tool("pymake", subcommand="c", help="清理构建产物 (调用 gitt c)", cmd=["pf", "gitt", "c"])
def c(cwd: Path = Path()) -> None:
"""清理构建产物."""
@px.tool(
"pymake",
subcommand="t",
help="运行测试",
cmd=["pytest", "-m", "not slow", "-n", "8", "--dist", "loadfile", "--color=yes", "--durations=10"],
)
def t(cwd: Path = Path()) -> None:
"""运行测试 (含并行)."""
@px.tool(
"pymake",
subcommand="tf",
help="快速测试 (无 slow)",
cmd=["pytest", "-m", "not slow", "--dist", "loadfile", "--color=yes", "--durations=10"],
)
def tf(cwd: Path = Path()) -> None:
"""快速测试 (无 slow, 无并行)."""
@px.tool("pymake", subcommand="bumpmi", help="升级次版本号 (bumpversion minor)", cmd=["pf", "bumpversion", "minor"])
def bumpmi(cwd: Path = Path()) -> None:
"""升级次版本号."""
@px.tool(
"pymake",
subcommand="doc",
help="构建 Sphinx 文档",
cmd=["sphinx-build", "-b", "html", "docs", "docs/_build"],
)
def doc(cwd: Path = Path()) -> None:
"""构建 Sphinx 文档."""
@px.tool("pymake", subcommand="lint", help="代码格式化与检查 (ruff)", cmd=["ruff", "check", "--fix", "--unsafe-fixes"])
def lint(cwd: Path = Path()) -> None:
"""代码格式化与检查."""
@px.tool("pymake", subcommand="tox", help="多版本测试 (tox)", cmd=["tox", "-p", "auto"])
def tox(cwd: Path = Path()) -> None:
"""多版本测试."""
# ============================================================================
# 内部 job (hidden, 不暴露为 subcommand)
# ============================================================================
@px.tool(
"pymake",
subcommand="bumpversion",
help="升级版本号 (patch)",
cmd=["pf", "bumpversion", "patch"],
needs=["git_add_all"],
hidden=True,
)
def bumpversion(cwd: Path = Path()) -> None:
"""升级版本号 (patch, 内部 job)."""
@px.tool(
"pymake",
subcommand="test_coverage",
help="测试并生成覆盖率",
cmd=["pytest", "--cov", "-n", "8", "--dist", "loadfile", "--tb=short", "-v", "--color=yes", "--durations=10"],
needs=["c"],
hidden=True,
)
def test_coverage(cwd: Path = Path()) -> None:
"""测试并生成覆盖率 (内部 job)."""
@px.tool(
"pymake",
subcommand="pyrefly_check",
help="pyrefly 类型检查",
cmd=["pyrefly", "check", "."],
hidden=True,
)
def pyrefly_check(cwd: Path = Path()) -> None:
"""pyrefly 类型检查 (内部 job)."""
@px.tool(
"pymake",
subcommand="git_add_all",
help="git add -A",
cmd=["git", "add", "-A"],
needs=["tc"],
hidden=True,
)
def git_add_all(cwd: Path = Path()) -> None:
"""git add -A (内部 job)."""
@px.tool("pymake", subcommand="git_push", help="git push", cmd=["git", "push"], hidden=True)
def git_push(cwd: Path = Path()) -> None:
"""git push (内部 job)."""
@px.tool(
"pymake",
subcommand="git_push_tags",
help="git push --tags",
cmd=["git", "push", "--tags"],
hidden=True,
)
def git_push_tags(cwd: Path = Path()) -> None:
"""git push --tags (内部 job)."""
@px.tool(
"pymake",
subcommand="twine_publish",
help="twine upload",
cmd=["twine", "upload", "--disable-progress-bar"],
hidden=True,
)
def twine_publish(cwd: Path = Path()) -> None:
"""twine upload (内部 job)."""
@px.tool("pymake", subcommand="publish_python", help="hatch publish", cmd=["hatch", "publish"], hidden=True)
def publish_python(cwd: Path = Path()) -> None:
"""hatch publish (内部 job)."""
# ============================================================================
# 聚合 job (有 needs 无 cmd)
# ============================================================================
@px.tool("pymake", subcommand="ba", help="构建所有包 (Python + Rust)", needs=["b", "bc"], strategy="thread")
def ba(cwd: Path = Path()) -> None:
"""构建所有包 (Python + Rust)."""
@px.tool("pymake", subcommand="bump", help="升级版本号 (清理 + 检查 + add + bumpversion)", needs=["bumpversion"])
def bump(cwd: Path = Path()) -> None:
"""升级版本号 (聚合)."""
@px.tool("pymake", subcommand="cov", help="测试并生成覆盖率", needs=["test_coverage"])
def cov(cwd: Path = Path()) -> None:
"""测试并生成覆盖率 (聚合)."""
@px.tool(
"pymake",
subcommand="tc",
help="类型检查 (pyrefly + ruff)",
needs=["c", "pyrefly_check", "lint"],
strategy="thread",
)
def tc(cwd: Path = Path()) -> None:
"""类型检查 (聚合)."""
@px.tool(
"pymake",
subcommand="p",
help="推送代码 (清理 + push + push tags)",
needs=["c", "git_push", "git_push_tags"],
strategy="thread",
)
def p(cwd: Path = Path()) -> None:
"""推送代码 (聚合)."""
@px.tool(
"pymake",
subcommand="pb",
help="发布到 PyPI (twine + hatch)",
needs=["twine_publish", "publish_python"],
strategy="thread",
)
def pb(cwd: Path = Path()) -> None:
"""发布到 PyPI (聚合)."""
+52
View File
@@ -0,0 +1,52 @@
"""reseticoncache - 重置 Windows 图标缓存工具.
执行流程: 杀掉 explorer → 删除 IconCache.db → 删除 iconcache* → 重启 explorer.
仅在 Windows 上执行, 非 Windows 平台打印提示并跳过.
"""
from __future__ import annotations
import os
import subprocess
from pathlib import Path
import pyflowx as px
from pyflowx.conditions import Constants
from pyflowx.ops._common import ensure_platform
@px.tool("reseticoncache", help="重置 Windows 图标缓存")
def reset_icon_cache_run() -> None:
"""重置 Windows 图标缓存.
执行流程: 杀掉 explorer → 删除 IconCache.db → 删除 iconcache* → 重启 explorer.
仅在 Windows 上执行, 非 Windows 平台打印提示并跳过.
"""
if not ensure_platform(Constants.IS_WINDOWS, "reset_icon_cache", "Windows"):
return
local_app_data = os.environ.get("LOCALAPPDATA", "")
if not local_app_data:
print("reset_icon_cache: LOCALAPPDATA 环境变量未设置")
return
icon_cache_db = Path(local_app_data) / "IconCache.db"
explorer_cache_dir = Path(local_app_data) / "Microsoft" / "Windows" / "Explorer"
print("正在终止 explorer 进程...")
subprocess.run(["taskkill", "/f", "/im", "explorer.exe"], check=False)
if icon_cache_db.exists():
print(f"删除图标缓存: {icon_cache_db}")
subprocess.run(["cmd", "/c", "del", "/a", "/q", str(icon_cache_db)], check=False)
if explorer_cache_dir.exists():
print(f"清理 Explorer 缓存: {explorer_cache_dir}")
subprocess.run(
["cmd", "/c", "del", "/a", "/q", str(explorer_cache_dir / "iconcache*")],
check=False,
)
print("重启 explorer...")
subprocess.run(["cmd", "/c", "start", "explorer.exe"], check=False)
print("图标缓存已重置")
+116
View File
@@ -0,0 +1,116 @@
"""screenshot - 截图工具.
跨平台截图: Windows 用 PowerShell, macOS 用 screencapture, Linux 用 gnome-screenshot/scrot.
"""
from __future__ import annotations
import subprocess
from datetime import datetime
from pathlib import Path
import pyflowx as px
from pyflowx.conditions import Constants
def get_screenshot_path(filename: str | None = None) -> Path:
"""获取截图保存路径.
Parameters
----------
filename : str | None
文件名, 如果为 None 则自动生成
Returns
-------
Path
截图保存路径
"""
if filename is None:
timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
filename = f"screenshot_{timestamp}.png"
screenshots_dir = Path.home() / "Pictures" / "screenshots"
screenshots_dir.mkdir(parents=True, exist_ok=True)
return screenshots_dir / filename
@px.tool("screenshot", subcommand="full", help="全屏截图")
def take_screenshot_full(filename: str | None = None) -> None:
"""全屏截图.
Parameters
----------
filename : str | None
文件名
"""
output_path = get_screenshot_path(filename)
if Constants.IS_WINDOWS:
ps_script = f"""
Add-Type -AssemblyName System.Windows.Forms
Add-Type -AssemblyName System.Drawing
$screen = [System.Windows.Forms.Screen]::PrimaryScreen
$bounds = $screen.Bounds
$bitmap = New-Object System.Drawing.Bitmap $bounds.Width, $bounds.Height
$graphics = [System.Drawing.Graphics]::FromImage($bitmap)
$graphics.CopyFromScreen($bounds.Location, [System.Drawing.Point]::Empty, $bounds.Size)
$bitmap.Save('{output_path.as_posix()}')
$graphics.Dispose()
$bitmap.Dispose()
"""
subprocess.run(["powershell", "-Command", ps_script], check=True)
elif Constants.IS_MACOS:
subprocess.run(["screencapture", "-x", str(output_path)], check=True)
else:
try:
subprocess.run(["gnome-screenshot", "-f", str(output_path)], check=True)
except FileNotFoundError:
subprocess.run(["scrot", str(output_path)], check=True)
print(f"截图已保存: {output_path}")
@px.tool("screenshot", subcommand="area", help="区域截图")
def take_screenshot_area(filename: str | None = None) -> None:
"""区域截图.
Parameters
----------
filename : str | None
文件名
"""
output_path = get_screenshot_path(filename)
if Constants.IS_WINDOWS:
ps_script = f"""
Add-Type -AssemblyName System.Windows.Forms
Add-Type -AssemblyName System.Drawing
$form = New-Object System.Windows.Forms.Form
$form.WindowState = 'Maximized'
$form.FormBorderStyle = 'None'
$form.BackColor = [System.Drawing.Color]::FromArgb(1, 0, 0)
$form.Opacity = 0.5
$form.TopMost = $true
$form.Show()
Start-Sleep -Milliseconds 100
$screen = [System.Windows.Forms.Screen]::PrimaryScreen
$bounds = $screen.Bounds
$bitmap = New-Object System.Drawing.Bitmap $bounds.Width, $bounds.Height
$graphics = [System.Drawing.Graphics]::FromImage($bitmap)
$graphics.CopyFromScreen($bounds.Location, [System.Drawing.Point]::Empty, $bounds.Size)
$form.Close()
$bitmap.Save('{output_path.as_posix()}')
$graphics.Dispose()
$bitmap.Dispose()
"""
subprocess.run(["powershell", "-Command", ps_script], check=True)
elif Constants.IS_MACOS:
subprocess.run(["screencapture", "-i", str(output_path)], check=True)
else:
try:
subprocess.run(["gnome-screenshot", "-a", "-f", str(output_path)], check=True)
except FileNotFoundError:
subprocess.run(["scrot", "-s", str(output_path)], check=True)
print(f"截图已保存: {output_path}")
+87
View File
@@ -0,0 +1,87 @@
"""sglang - SGLang 本地模型服务工具.
提供安装与启动 SGLang 服务的子命令.
"""
from __future__ import annotations
import shutil
import subprocess
from pathlib import Path
import pyflowx as px
from pyflowx.ops._common import platform_command
__all__ = [
"install_sglang",
"run_sglang",
]
@px.tool("sglang", subcommand="install", help="安装 sglang")
def install_sglang() -> None:
"""安装 sglang (若未安装).
通过 ``shutil.which`` 检测 sglang 是否已安装, 未安装时执行 ``uv install sglang[all]``.
"""
if shutil.which("sglang") is not None:
print("sglang 已安装, 跳过安装步骤")
return
print("正在安装 sglang[all]...")
subprocess.run(["uv", "install", "sglang[all]"], check=False)
@px.tool("sglang", subcommand="run", help="启动 SGLang 服务", needs=["install"])
def run_sglang(
model: str = "~/.models/Qwen2.5-Coder-32B-Instruct-AWQ",
port: int = 8000,
ctx_len: int = 32768,
mem_fraction: float = 0.75,
host: str = "0.0.0.0",
log_level: str = "info",
) -> None:
"""启动 SGLang 本地模型服务.
Parameters
----------
model : str
模型路径 (默认: ``~/.models/Qwen2.5-Coder-32B-Instruct-AWQ``)
port : int
服务端口 (默认: 8000)
ctx_len : int
最大上下文长度 (默认: 32768)
mem_fraction : float
显存占比 0-1 (默认: 0.75)
host : str
主机地址 (默认: 0.0.0.0)
log_level : str
日志级别 (默认: info)
"""
model_dir = Path(model).expanduser()
if not model_dir.exists():
print(f"模型目录不存在: {model_dir}")
return
python_bin = platform_command(["python"], ["python3"])[0]
cmd = [
python_bin,
"-m",
"sglang.launch_server",
"--model-path",
str(model_dir),
"--host",
host,
"--port",
str(port),
"--mem-fraction-static",
str(mem_fraction),
"--context-length",
str(ctx_len),
"--tool-call-parser",
"qwen",
"--log-level",
log_level,
]
print(f"启动 SGLang: {model_dir} (port={port}, ctx={ctx_len}, mem={mem_fraction})")
subprocess.run(cmd, check=False)
+84
View File
@@ -0,0 +1,84 @@
"""sshcopyid - SSH 密钥部署工具.
将本地 SSH 公钥部署到远程服务器.
"""
from __future__ import annotations
import subprocess
import sys
from pathlib import Path
import pyflowx as px
__all__ = ["ssh_copy_id"]
@px.tool("sshcopyid", help="部署 SSH 公钥到远程服务器")
def ssh_copy_id(
hostname: str,
username: str,
password: str,
port: int = 22,
keypath: str = "~/.ssh/id_rsa.pub",
timeout: int = 30,
) -> None:
"""将 SSH 公钥部署到远程服务器.
Parameters
----------
hostname : str
远程服务器主机名或 IP 地址
username : str
远程服务器用户名
password : str
远程服务器密码
port : int
SSH 端口, 默认 22
keypath : str
公钥文件路径, 默认 ~/.ssh/id_rsa.pub
timeout : int
SSH 操作超时秒数, 默认 30
"""
pub_key_path = Path(keypath).expanduser()
if not pub_key_path.exists():
print(f"公钥文件不存在: {pub_key_path}")
sys.exit(1)
pub_key = pub_key_path.read_text().strip()
script = f"""mkdir -p ~/.ssh && chmod 700 ~/.ssh
cd ~/.ssh && touch authorized_keys && chmod 600 authorized_keys
grep -qF '{pub_key.split()[1]}' authorized_keys 2>/dev/null || echo '{pub_key}' >> authorized_keys"""
try:
subprocess.run(
[
"sshpass",
"-p",
password,
"ssh",
"-p",
str(port),
"-o",
"StrictHostKeyChecking=no",
"-o",
"UserKnownHostsFile=/dev/null",
"-o",
f"ConnectTimeout={timeout}",
f"{username}@{hostname}",
script,
],
check=True,
timeout=timeout,
)
print(f"SSH 密钥已部署到 {username}@{hostname}:{port}")
except FileNotFoundError:
print(f"未找到 sshpass 工具, 请手动执行: ssh-copy-id -p {port} {username}@{hostname}")
sys.exit(1)
except subprocess.TimeoutExpired:
print("SSH 连接超时")
sys.exit(1)
except subprocess.CalledProcessError as e:
print(f"SSH 执行失败: {e}")
sys.exit(1)
-504
View File
@@ -1,504 +0,0 @@
"""系统类函数模块.
聚合 LS-DYNA 计算 (lscalc)、SSH 密钥部署 (sshcopyid)、Python 打包 (packtool)、
重置图标缓存 (reset_icon_cache) 的可复用函数. 所有公共函数通过 ``@px.register_fn``
注册, 供 YAML 任务编排引用.
"""
from __future__ import annotations
import os
import platform
import shutil
import subprocess
import sys
import urllib.request
import zipfile
from pathlib import Path
import pyflowx as px
from pyflowx.conditions import Constants
__all__ = [
"DEFAULT_BUILD_DIR",
"DEFAULT_CACHE_DIR",
"DEFAULT_DIST_DIR",
"DEFAULT_INPUT_FILE",
"DEFAULT_LIB_DIR",
"DEFAULT_NCPU",
"IGNORE_PATTERNS",
"LS_DYNA_COMMANDS",
"check_ls_dyna_status",
"clean_build_dir",
"create_zip_package",
"get_ls_dyna_command",
"install_embed_python",
"pack_dependencies",
"pack_source",
"pack_wheel",
"reset_icon_cache_run",
"run_ls_dyna",
"run_ls_dyna_mpi",
"ssh_copy_id",
]
# ============================================================================
# lscalc 配置
# ============================================================================
LS_DYNA_COMMANDS: dict[str, list[str]] = {
"windows": ["ls-dyna_mpp", "i=input.k", "ncpu=4"],
"linux": ["ls-dyna_mpp", "i=input.k", "ncpu=8"],
"macos": ["ls-dyna_mpp", "i=input.k", "ncpu=4"],
}
DEFAULT_INPUT_FILE: str = "input.k"
DEFAULT_NCPU: int = 4
# ============================================================================
# packtool 配置
# ============================================================================
DEFAULT_BUILD_DIR = ".pypack"
DEFAULT_DIST_DIR = "dist"
DEFAULT_LIB_DIR = "libs"
DEFAULT_CACHE_DIR = ".cache/pypack"
IGNORE_PATTERNS = [
"__pycache__",
"*.pyc",
"*.pyo",
".git",
".venv",
".idea",
".vscode",
"*.egg-info",
"dist",
"build",
".pytest_cache",
".tox",
".mypy_cache",
]
# ============================================================================
# lscalc 函数
# ============================================================================
@px.register_fn
def get_ls_dyna_command(input_file: str, ncpu: int) -> list[str]:
"""获取 LS-DYNA 命令.
Parameters
----------
input_file : str
输入文件路径
ncpu : int
CPU 核心数
Returns
-------
list[str]
LS-DYNA 命令列表
"""
if Constants.IS_WINDOWS or Constants.IS_MACOS:
return ["ls-dyna_mpp", f"i={input_file}", f"ncpu={ncpu}"]
else:
return ["ls-dyna_mpp", f"i={input_file}", f"ncpu={ncpu}"]
@px.register_fn
def run_ls_dyna(input_file: str, ncpu: int = DEFAULT_NCPU) -> None:
"""运行 LS-DYNA 计算.
Parameters
----------
input_file : str
输入文件路径
ncpu : int
CPU 核心数
"""
input_path = Path(input_file)
if not input_path.exists():
print(f"输入文件不存在: {input_path}")
return
cmd = get_ls_dyna_command(input_file, ncpu)
try:
subprocess.run(cmd, check=True)
print(f"LS-DYNA 计算完成: {input_file}")
except FileNotFoundError:
print("未找到 ls-dyna_mpp 命令")
except subprocess.CalledProcessError as e:
print(f"LS-DYNA 计算失败: {e}")
@px.register_fn
def run_ls_dyna_mpi(input_file: str, ncpu: int = DEFAULT_NCPU) -> None:
"""运行 LS-DYNA MPI 计算.
Parameters
----------
input_file : str
输入文件路径
ncpu : int
CPU 核心数
"""
input_path = Path(input_file)
if not input_path.exists():
print(f"输入文件不存在: {input_path}")
return
cmd = ["mpirun", "-np", str(ncpu), "ls-dyna_mpp", f"i={input_file}"]
try:
subprocess.run(cmd, check=True)
print(f"LS-DYNA MPI 计算完成: {input_file}")
except FileNotFoundError:
print("未找到 mpirun 或 ls-dyna_mpp 命令")
except subprocess.CalledProcessError as e:
print(f"LS-DYNA MPI 计算失败: {e}")
@px.register_fn
def check_ls_dyna_status() -> None:
"""检查 LS-DYNA 进程状态."""
try:
if Constants.IS_WINDOWS:
result = subprocess.run(
["tasklist", "/fi", "imagename eq ls-dyna_mpp.exe"],
capture_output=True,
text=True,
check=True,
)
print(result.stdout)
else:
result = subprocess.run(
["pgrep", "-f", "ls-dyna"],
capture_output=True,
text=True,
check=False,
)
if result.stdout.strip():
print(f"运行中的 LS-DYNA 进程 PID: {result.stdout.strip()}")
else:
print("没有运行中的 LS-DYNA 进程")
except subprocess.CalledProcessError as e:
print(f"检查进程状态失败: {e}")
# ============================================================================
# sshcopyid 函数
# ============================================================================
@px.register_fn
def ssh_copy_id(
hostname: str,
username: str,
password: str,
port: int = 22,
keypath: str = "~/.ssh/id_rsa.pub",
timeout: int = 30,
) -> None:
"""将 SSH 公钥部署到远程服务器.
Parameters
----------
hostname : str
远程服务器主机名或 IP 地址
username : str
远程服务器用户名
password : str
远程服务器密码
port : int
SSH 端口, 默认 22
keypath : str
公钥文件路径, 默认 ~/.ssh/id_rsa.pub
timeout : int
SSH 操作超时秒数, 默认 30
"""
pub_key_path = Path(keypath).expanduser()
if not pub_key_path.exists():
print(f"公钥文件不存在: {pub_key_path}")
sys.exit(1)
pub_key = pub_key_path.read_text().strip()
script = f"""mkdir -p ~/.ssh && chmod 700 ~/.ssh
cd ~/.ssh && touch authorized_keys && chmod 600 authorized_keys
grep -qF '{pub_key.split()[1]}' authorized_keys 2>/dev/null || echo '{pub_key}' >> authorized_keys"""
try:
subprocess.run(
[
"sshpass",
"-p",
password,
"ssh",
"-p",
str(port),
"-o",
"StrictHostKeyChecking=no",
"-o",
"UserKnownHostsFile=/dev/null",
"-o",
f"ConnectTimeout={timeout}",
f"{username}@{hostname}",
script,
],
check=True,
timeout=timeout,
)
print(f"SSH 密钥已部署到 {username}@{hostname}:{port}")
except FileNotFoundError:
print(f"未找到 sshpass 工具, 请手动执行: ssh-copy-id -p {port} {username}@{hostname}")
sys.exit(1)
except subprocess.TimeoutExpired:
print("SSH 连接超时")
sys.exit(1)
except subprocess.CalledProcessError as e:
print(f"SSH 执行失败: {e}")
sys.exit(1)
# ============================================================================
# packtool 函数
# ============================================================================
@px.register_fn
def pack_source(project_dir: Path, output_dir: Path) -> None:
"""打包项目源码.
Parameters
----------
project_dir : Path
项目目录
output_dir : Path
输出目录
"""
output_dir.mkdir(parents=True, exist_ok=True)
pyproject_file = project_dir / "pyproject.toml"
project_name = project_dir.name
if pyproject_file.exists():
try:
import tomllib
content = pyproject_file.read_text(encoding="utf-8")
data = tomllib.loads(content)
project_name = data.get("project", {}).get("name", project_name)
except ImportError:
pass
source_dir = output_dir / "src" / project_name
source_dir.mkdir(parents=True, exist_ok=True)
src_subdir = project_dir / "src"
if src_subdir.exists():
shutil.copytree(
src_subdir,
source_dir / "src",
ignore=shutil.ignore_patterns(*IGNORE_PATTERNS),
dirs_exist_ok=True,
)
else:
for item in project_dir.iterdir():
if item.name in IGNORE_PATTERNS or item.name.startswith("."):
continue
dst_item = source_dir / item.name
if item.is_dir():
shutil.copytree(
item,
dst_item,
ignore=shutil.ignore_patterns(*IGNORE_PATTERNS),
dirs_exist_ok=True,
)
else:
shutil.copy2(item, dst_item)
print(f"源码打包完成: {source_dir}")
@px.register_fn
def pack_dependencies(lib_dir: Path, dependencies: list[str]) -> None:
"""打包项目依赖.
Parameters
----------
lib_dir : Path
依赖库目录
dependencies : list[str]
依赖列表
"""
lib_dir.mkdir(parents=True, exist_ok=True)
if not dependencies:
print("没有依赖需要打包")
return
cmd = [
"pip",
"install",
"--target",
str(lib_dir),
"--no-compile",
"--no-warn-script-location",
]
cmd.extend(dependencies)
subprocess.run(cmd, check=True)
print(f"依赖打包完成: {lib_dir}")
@px.register_fn
def pack_wheel(project_dir: Path, output_dir: Path) -> None:
"""打包项目为 wheel 文件.
Parameters
----------
project_dir : Path
项目目录
output_dir : Path
输出目录
"""
output_dir.mkdir(parents=True, exist_ok=True)
cmd = [
"pip",
"wheel",
"--no-deps",
"--wheel-dir",
str(output_dir),
str(project_dir),
]
subprocess.run(cmd, check=True)
print(f"Wheel 打包完成: {output_dir}")
@px.register_fn
def install_embed_python(version: str, output_dir: Path) -> None:
"""安装嵌入式 Python.
Parameters
----------
version : str
Python 版本 (如: 3.10, 3.11)
output_dir : Path
输出目录
"""
output_dir.mkdir(parents=True, exist_ok=True)
arch = platform.machine().lower()
if arch in ["x86_64", "amd64"]:
arch = "amd64"
elif arch in ["arm64", "aarch64"]:
arch = "arm64"
version_map = {
"3.8": "3.8.10",
"3.9": "3.9.13",
"3.10": "3.10.11",
"3.11": "3.11.9",
"3.12": "3.12.4",
}
full_version = version_map.get(version, f"{version}.0")
url = f"https://www.python.org/ftp/python/{full_version}/python-{full_version}-embed-{arch}.zip"
cache_file = Path(DEFAULT_CACHE_DIR) / f"python-{full_version}-embed-{arch}.zip"
cache_file.parent.mkdir(parents=True, exist_ok=True)
if not cache_file.exists():
print(f"正在下载嵌入式 Python {full_version}...")
urllib.request.urlretrieve(url, cache_file)
print(f"下载完成: {cache_file}")
with zipfile.ZipFile(cache_file, "r") as zf:
zf.extractall(output_dir)
print(f"嵌入式 Python 安装完成: {output_dir}")
@px.register_fn
def create_zip_package(source_dir: Path, output_file: Path) -> None:
"""创建 ZIP 打包文件.
Parameters
----------
source_dir : Path
源目录
output_file : Path
输出文件
"""
output_file.parent.mkdir(parents=True, exist_ok=True)
with zipfile.ZipFile(output_file, "w", zipfile.ZIP_DEFLATED) as zf:
for file in source_dir.rglob("*"):
if file.is_file():
arcname = file.relative_to(source_dir)
zf.write(file, arcname)
print(f"ZIP 打包完成: {output_file}")
@px.register_fn
def clean_build_dir(build_dir: Path) -> None:
"""清理构建目录.
Parameters
----------
build_dir : Path
构建目录
"""
if build_dir.exists():
shutil.rmtree(build_dir)
print(f"清理完成: {build_dir}")
else:
print(f"目录不存在: {build_dir}")
# ============================================================================
# reseticoncache 函数
# ============================================================================
@px.register_fn
def reset_icon_cache_run() -> None:
"""重置 Windows 图标缓存.
执行流程: 杀掉 explorer → 删除 IconCache.db → 删除 iconcache* → 重启 explorer.
仅在 Windows 上执行, 非 Windows 平台打印提示并跳过.
"""
if not Constants.IS_WINDOWS:
print("reset_icon_cache: 仅在 Windows 上支持")
return
local_app_data = os.environ.get("LOCALAPPDATA", "")
if not local_app_data:
print("reset_icon_cache: LOCALAPPDATA 环境变量未设置")
return
icon_cache_db = Path(local_app_data) / "IconCache.db"
explorer_cache_dir = Path(local_app_data) / "Microsoft" / "Windows" / "Explorer"
print("正在终止 explorer 进程...")
subprocess.run(["taskkill", "/f", "/im", "explorer.exe"], check=False)
if icon_cache_db.exists():
print(f"删除图标缓存: {icon_cache_db}")
subprocess.run(["cmd", "/c", "del", "/a", "/q", str(icon_cache_db)], check=False)
if explorer_cache_dir.exists():
print(f"清理 Explorer 缓存: {explorer_cache_dir}")
subprocess.run(
["cmd", "/c", "del", "/a", "/q", str(explorer_cache_dir / "iconcache*")],
check=False,
)
print("重启 explorer...")
subprocess.run(["cmd", "/c", "start", "explorer.exe"], check=False)
print("图标缓存已重置")
+30
View File
@@ -0,0 +1,30 @@
"""taskkill - 进程终止工具.
跨平台按名称终止进程: Windows 用 ``taskkill``, Linux/macOS 用 ``pkill``.
"""
from __future__ import annotations
import subprocess
import pyflowx as px
from pyflowx.ops._common import platform_command
@px.tool("taskkill", help="按名称终止进程 (跨平台)")
def taskkill_run(process_names: list[str]) -> None:
"""按名称终止进程 (跨平台).
Windows 使用 ``taskkill /f /im <name>*``,
Linux/macOS 使用 ``pkill -f <name>*``.
Parameters
----------
process_names : list[str]
进程名称列表 (如: ``["chrome.exe", "python"]``)
"""
cmd_prefix = platform_command(["taskkill", "/f", "/im"], ["pkill", "-f"])
for name in process_names:
print(f"终止进程: {name}")
subprocess.run([*cmd_prefix, f"{name}*"], check=False)
+35
View File
@@ -0,0 +1,35 @@
"""which - 命令查找工具.
跨平台查找可执行命令路径: Windows 用 ``where``, Linux/macOS 用 ``which``.
"""
from __future__ import annotations
import subprocess
import pyflowx as px
from pyflowx.ops._common import platform_command
@px.tool("which", help="查找可执行命令路径 (跨平台)")
def which_run(commands: list[str]) -> None:
"""查找可执行命令路径 (跨平台).
Windows 使用 ``where``, Linux/macOS 使用 ``which``.
对每个命令打印 ``<cmd> -> <path>`` 或 ``<cmd> -> 未找到``.
Parameters
----------
commands : list[str]
要查找的命令名称列表
"""
which_cmd = platform_command(["where"], ["which"])[0]
for cmd in commands:
result = subprocess.run([which_cmd, cmd], capture_output=True, text=True, check=False)
if result.returncode == 0:
# Windows 的 where 可能返回多行, 取第一个
path = result.stdout.strip().split("\n")[0].strip()
print(f"{cmd} -> {path}")
else:
print(f"{cmd} -> 未找到")
-159
View File
@@ -1,159 +0,0 @@
"""函数注册表.
提供全局函数注册机制, 供 YAML 任务编排通过 ``fn`` 字段引用 Python 函数.
使用方式
--------
import pyflowx as px
@px.register_fn("pack_source")
def pack_source(project_dir, output_dir):
...
# YAML 中引用:
# jobs:
# pack:
# fn: pack_source
# args: ["./project", "./dist"]
"""
from __future__ import annotations
import sys
from typing import Any, Callable, TypeVar, overload
if sys.version_info >= (3, 10):
from typing import ParamSpec
else:
from typing_extensions import ParamSpec # pragma: no cover
__all__ = ["FnRegistry", "get_fn", "has_fn", "register_fn"]
P = ParamSpec("P")
T = TypeVar("T")
_REGISTRY: dict[str, Callable[..., Any]] = {}
@overload
def register_fn(name: Callable[P, T]) -> Callable[P, T]: ...
@overload
def register_fn(name: str | None = None) -> Callable[[Callable[P, T]], Callable[P, T]]: ...
def register_fn(name: str | Callable[..., Any] | None = None) -> Callable[..., Any]:
"""装饰器:将函数注册到全局 registry.
支持两种用法::
@register_fn # 使用函数 __name__ 作为注册名
def my_func(): ...
@register_fn("custom") # 显式指定注册名
def my_func(): ...
Parameters
----------
name : str | Callable | None
注册名或被装饰函数; 为 None 时使用函数 ``__name__``
Returns
-------
Callable
装饰器函数或被装饰函数
Raises
------
ValueError
名称已注册或无法推断函数名
"""
if callable(name):
fn = name
key = getattr(fn, "__name__", None)
if key is None:
raise ValueError("无法推断函数名, 请显式提供 name 参数")
if key in _REGISTRY:
raise ValueError(f"函数 {key!r} 已注册")
_REGISTRY[key] = fn
return fn
def decorator(fn: Callable[P, T]) -> Callable[P, T]:
key = name if name is not None else getattr(fn, "__name__", None)
if key is None:
raise ValueError("无法推断函数名, 请显式提供 name 参数")
if key in _REGISTRY:
raise ValueError(f"函数 {key!r} 已注册")
_REGISTRY[key] = fn
return fn
return decorator
def get_fn(name: str) -> Callable[..., Any]:
"""按名称获取已注册的函数.
Parameters
----------
name : str
函数名
Returns
-------
Callable
已注册的函数
Raises
------
KeyError
函数未注册
"""
if name not in _REGISTRY:
raise KeyError(f"函数 {name!r} 未注册")
return _REGISTRY[name]
def has_fn(name: str) -> bool:
"""检查函数是否已注册.
Parameters
----------
name : str
函数名
Returns
-------
bool
是否已注册
"""
return name in _REGISTRY
class FnRegistry:
"""函数注册表的面向对象访问接口."""
@staticmethod
def register(name: str | None = None) -> Callable[[Callable[..., T]], Callable[..., T]]:
"""注册装饰器, 等价于 :func:`register_fn`."""
return register_fn(name)
@staticmethod
def get(name: str) -> Callable[..., Any]:
"""获取已注册函数, 等价于 :func:`get_fn`."""
return get_fn(name)
@staticmethod
def has(name: str) -> bool:
"""检查是否已注册, 等价于 :func:`has_fn`."""
return has_fn(name)
@staticmethod
def clear() -> None:
"""清空注册表."""
_REGISTRY.clear()
@staticmethod
def names() -> list[str]:
"""返回所有已注册函数名."""
return list(_REGISTRY.keys())
+2 -1
View File
@@ -6,8 +6,9 @@
from __future__ import annotations
from collections.abc import Iterator
from dataclasses import dataclass, field
from typing import Any, Iterator
from typing import Any
from .task import TaskResult, TaskStatus
+2 -1
View File
@@ -14,9 +14,10 @@ from __future__ import annotations
import argparse
import enum
import sys
from collections.abc import Sequence
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any, Sequence, get_args
from typing import Any, get_args
from .compose import GraphComposer
from .errors import PyFlowXError
+4 -4
View File
@@ -17,10 +17,10 @@ import json
import sys
import time
from abc import ABC, abstractmethod
from collections.abc import Iterator
from contextlib import contextmanager, nullcontext
from collections.abc import Iterator, Mapping
from contextlib import AbstractContextManager, contextmanager, nullcontext
from pathlib import Path
from typing import Any, ContextManager, Mapping
from typing import Any
if sys.version_info >= (3, 12):
from typing import override
@@ -63,7 +63,7 @@ class StateBackend(ABC):
:class:`JSONBackend` 在 :meth:`batch` 期间会延迟落盘,需在退出时调用。
"""
def batch(self) -> ContextManager[None]:
def batch(self) -> AbstractContextManager[None]:
"""返回一个上下文管理器,期间 :meth:`save` 可延迟 :meth:`flush`。
默认实现为 no-op(如 :class:`MemoryBackend`)。:class:`JSONBackend`
+6 -19
View File
@@ -22,7 +22,8 @@ import os
import shutil
import sys
import threading
from contextlib import contextmanager
from collections.abc import Callable, Coroutine, Generator, Mapping
from contextlib import AbstractContextManager, contextmanager
from dataclasses import dataclass, field
from datetime import datetime
from enum import Enum
@@ -30,14 +31,7 @@ from functools import cached_property
from pathlib import Path
from typing import (
Any,
Callable,
ContextManager,
Coroutine,
Generator,
Generic,
List,
Mapping,
Union,
cast,
)
@@ -49,25 +43,18 @@ else:
T = TypeVar("T", default=Any)
# 任务可调用对象可以是同步或异步的。显式保留联合类型,让 mypy 理解两种形态。
TaskFn = Union[
Callable[..., T],
Callable[..., Coroutine[Any, Any, T]],
]
TaskFn = Callable[..., T] | Callable[..., Coroutine[Any, Any, T]]
# 跨任务结果映射。值刻意使用 ``Any``,因为不同任务返回不同类型;
# 单任务类型由函数签名本身保留。
Context = Mapping[str, Any]
# 命令类型支持
TaskCmd = Union[
List[str], # 命令列表, 如 ["ls", "-la"]
str, # shell 命令字符串
Callable[..., Any], # Python 函数
]
TaskCmd = list[str] | str | Callable[..., Any]
# 执行策略:sequential/thread/async 为层屏障模型,dependency 为依赖驱动模型。
Strategy = Union[str, "StrategyKind"]
StrategyKind = Any # 占位,避免循环;executors 模块用 Literal 约束
Strategy = str | StrategyKind
logger = logging.getLogger(__name__)
@@ -376,7 +363,7 @@ class TaskSpec(Generic[T]):
return shutil.which(cmd[0]) is not None
return True
def env_context(self) -> ContextManager[None]:
def env_context(self) -> AbstractContextManager[None]:
"""返回临时应用 ``env`` 与 ``cwd`` 的上下文管理器。
对 ``fn`` 任务生效。``cmd`` 任务在 :func:`_run_command` 中直接
+572
View File
@@ -0,0 +1,572 @@
"""@px.tool 装饰器: typer 驱动 CLI + DAG 编排.
替代 YAML 配置模式, .py 装饰器统一描述工具.
函数签名 typer 自动生成 CLI, 函数体即任务逻辑, needs/strategy 表达 DAG.
示例
----
::
@px.tool("pdftool", subcommand="m", help="合并 PDF")
def pdf_merge(
inputs: list[Path], # positional, nargs="+"
output: Path = Path("merged.pdf"), # --output 选项
) -> None:
... # 函数体即逻辑
# CLI: pf pdftool m a.pdf b.pdf --output merged.pdf
cmd 任务 (执行外部命令, 函数体不执行)::
@px.tool("pymake", subcommand="b", help="构建", cmd=["uv", "build"])
def b(cwd: Path = Path(".")) -> None:
pass # 签名仅驱动 CLI, cwd 注入到 cmd 工作目录
聚合任务 ( needs cmd 无函数逻辑)::
@px.tool("pymake", subcommand="tc", help="类型检查",
needs=["c", "pyrefly_check", "lint"], strategy="thread")
def tc(cwd: Path = Path(".")) -> None:
pass # 仅作依赖聚合点
"""
from __future__ import annotations
__all__ = [
"ToolExitCode",
"ToolSpec",
"clear_tool_registry",
"get_tool",
"list_subcommands",
"list_tools",
"run_tool",
"tool",
]
import ast
import enum
import inspect
import sys
import textwrap
from collections.abc import Callable, Mapping, Sequence
from dataclasses import dataclass
from pathlib import Path
from typing import Any, Literal, cast
import typer
from rich.console import Console
from rich.table import Table
# typer 内置 click 异常 (typer._click 与公共 click 是两套独立类层次)
from typer._click.exceptions import ClickException as _TyperClickException
from typer._click.exceptions import NoArgsIsHelpError as _NoArgsIsHelpError
from pyflowx.errors import PyFlowXError
from pyflowx.executors import run
from pyflowx.graph import Graph
from pyflowx.task import RetryPolicy, TaskSpec
class ToolExitCode(enum.IntEnum):
"""工具执行退出码."""
SUCCESS = 0
FAILURE = 1
INTERRUPTED = 130 # 与 POSIX 信号中断一致
def _noop() -> None:
"""聚合任务的占位函数."""
return None
# ---------------------------------------------------------------------- #
# ToolSpec: 工具描述符
# ---------------------------------------------------------------------- #
@dataclass(frozen=True)
class ToolSpec:
"""工具描述符: 由 ``@px.tool`` 装饰器注册.
封装函数 + CLI 元数据 + DAG 编排参数, 运行时映射为 :class:`TaskSpec`.
"""
name: str # 工具名 (如 "pdftool")
subcommand: str | None # 子命令名 (None=单命令工具)
func: Callable[..., Any] # 函数 (签名驱动 CLI, 体即逻辑)
help: str = "" # 帮助文本
description: str = "" # 工具描述 (pf 列表用)
cmd: tuple[str, ...] | str | None = None # 命令 (有则执行 cmd, 无则执行函数体)
needs: tuple[str, ...] = () # 依赖任务 (引用同 tool 的 subcommand)
strategy: Literal["sequential", "thread", "async", "dependency"] | None = None # 执行策略
cwd: str | Path | None = None # 工作目录 (cmd 任务, 装饰器级默认)
allow_upstream_skip: bool = False
hidden: bool = False # 不暴露为 subcommand (内部 job)
env: Mapping[str, str] | None = None
retry: RetryPolicy | None = None
timeout: float | None = None
# 全局工具注册表: {tool_name: {subcommand: ToolSpec}}
_TOOL_REGISTRY: dict[str, dict[str | None, ToolSpec]] = {}
# ---------------------------------------------------------------------- #
# @tool 装饰器 + 注册表
# ---------------------------------------------------------------------- #
def tool(
name: str,
*,
subcommand: str | None = None,
help: str = "",
description: str = "",
cmd: Sequence[str] | str | None = None,
needs: Sequence[str] | None = None,
strategy: Literal["sequential", "thread", "async", "dependency"] | None = None,
cwd: str | Path | None = None,
allow_upstream_skip: bool = False,
hidden: bool = False,
env: Mapping[str, str] | None = None,
retry: RetryPolicy | None = None,
timeout: float | None = None,
) -> Callable[[Callable[..., Any]], Callable[..., Any]]:
"""装饰器: 将函数注册为 ``@px.tool`` 工具.
Parameters
----------
name : str
工具名 ( "pdftool"); 多个 ``@px.tool`` 共用同名即多 subcommand 工具
subcommand : str | None
子命令名; ``None`` 表示单命令工具 (整个工具仅一个函数)
help : str
子命令帮助文本; 默认用函数 docstring
description : str
工具描述, 用于 pf 工具列表
cmd : Sequence[str] | str | None
命令列表或 shell 字符串; ``cmd`` 执行命令, 函数体不执行 (签名仅驱动 CLI)
needs : Sequence[str] | None
依赖任务名 (引用同 tool 的其他 subcommand)
strategy : str | None
执行策略: ``sequential`` / ``thread`` / ``async`` / ``dependency``
cwd : str | Path | None
工作目录 (cmd 任务装饰器级默认); 若函数签名有 ``cwd`` 参数则被 CLI 值覆盖
allow_upstream_skip : bool
上游 SKIPPED 时本任务仍执行
hidden : bool
不暴露为 subcommand (内部 job, 仅被 needs 引用)
env / retry / timeout
透传 :class:`TaskSpec` 对应字段
"""
def decorator(func: Callable[..., Any]) -> Callable[..., Any]:
spec = ToolSpec(
name=name,
subcommand=subcommand,
func=func,
help=help or inspect.getdoc(func) or "",
description=description,
cmd=cast("tuple[str, ...] | str | None", tuple(cmd) if isinstance(cmd, (list, tuple)) else cmd),
needs=tuple(needs) if needs else (),
strategy=strategy,
cwd=cwd,
allow_upstream_skip=allow_upstream_skip,
hidden=hidden,
env=dict(env) if env else None,
retry=retry,
timeout=timeout,
)
_register_tool(spec)
return func
return decorator
def _register_tool(spec: ToolSpec) -> None:
"""注册工具到全局注册表, 校验重复."""
if spec.name not in _TOOL_REGISTRY:
_TOOL_REGISTRY[spec.name] = {}
if spec.subcommand in _TOOL_REGISTRY[spec.name]:
raise ValueError(f"工具 {spec.name!r} 的子命令 {spec.subcommand!r} 已注册")
_TOOL_REGISTRY[spec.name][spec.subcommand] = spec
def get_tool(name: str, subcommand: str | None = None) -> ToolSpec:
"""获取已注册工具.
Raises
------
KeyError
工具或子命令未注册
"""
if name not in _TOOL_REGISTRY:
raise KeyError(f"工具 {name!r} 未注册")
subs = _TOOL_REGISTRY[name]
if subcommand not in subs:
raise KeyError(f"工具 {name!r} 没有子命令 {subcommand!r}")
return subs[subcommand]
def list_tools() -> list[str]:
"""列出所有已注册工具名."""
return sorted(_TOOL_REGISTRY.keys())
def list_subcommands(name: str, include_hidden: bool = False) -> list[str]:
"""列出工具的子命令 (hidden 默认排除).
单命令工具 (subcommand=None) 返回空列表.
"""
if name not in _TOOL_REGISTRY:
return []
return sorted(
sc for sc, spec in _TOOL_REGISTRY[name].items() if sc is not None and (include_hidden or not spec.hidden)
)
def clear_tool_registry() -> None:
"""清空注册表 (测试用)."""
_TOOL_REGISTRY.clear()
# ---------------------------------------------------------------------- #
# 依赖收集 + TaskSpec 构建
# ---------------------------------------------------------------------- #
def _collect_with_deps(name: str, target: str | None) -> list[str | None]:
"""BFS 收集 target 及其传递依赖 (subcommand 名).
返回顺序: 依赖在前, target 在后 (符合 DAG 拓扑).
"""
if name not in _TOOL_REGISTRY:
return [target]
subs = _TOOL_REGISTRY[name]
result: list[str | None] = []
seen: set[str | None] = set()
queue: list[str | None] = [target]
while queue:
sc = queue.pop(0)
if sc in seen:
continue
seen.add(sc)
result.append(sc)
if sc in subs:
queue.extend(subs[sc].needs)
# 反转: 依赖在前, target 在后
result.reverse()
return result
def _has_function_logic(func: Callable[..., Any]) -> bool:
"""判断函数体是否有实际逻辑 (非 pass/.../docstring).
ast 分析, 避免 exec 函数.
"""
try:
src = inspect.getsource(func)
src = textwrap.dedent(src)
tree = ast.parse(src)
except (OSError, TypeError, SyntaxError): # pragma: no cover
return True
func_def = next((n for n in ast.walk(tree) if isinstance(n, (ast.FunctionDef, ast.AsyncFunctionDef))), None)
if func_def is None: # pragma: no cover
return True
for stmt in func_def.body:
if isinstance(stmt, ast.Expr) and isinstance(stmt.value, ast.Constant):
continue # docstring
if isinstance(stmt, ast.Pass):
continue
return True
return False
def _is_aggregate(spec: ToolSpec) -> bool:
"""判断是否为聚合任务 (有 needs 无 cmd 无函数逻辑)."""
if spec.cmd is not None or not spec.needs:
return False
return not _has_function_logic(spec.func)
def _build_task_spec(spec: ToolSpec, variables: Mapping[str, Any]) -> TaskSpec:
"""将 ToolSpec + 解析后的变量转为 TaskSpec.
- cmd 任务: 执行命令, cwd variables["cwd"] 或装饰器 cwd
- 聚合任务 ( needs cmd 无函数逻辑): fn=noop
- fn 任务: 执行函数, kwargs 按签名从 variables
"""
task_name = spec.subcommand if spec.subcommand is not None else spec.name
# cmd 任务
if spec.cmd is not None:
cwd_value = variables.get("cwd", spec.cwd)
cwd = Path(cwd_value) if cwd_value is not None else None
cmd_value: Any = list(spec.cmd) if isinstance(spec.cmd, tuple) else spec.cmd
return TaskSpec(
name=task_name,
cmd=cmd_value,
depends_on=spec.needs,
cwd=cwd,
env=spec.env,
retry=spec.retry or RetryPolicy(),
timeout=spec.timeout,
allow_upstream_skip=spec.allow_upstream_skip,
strategy=spec.strategy,
)
# 聚合任务
if _is_aggregate(spec):
return TaskSpec(
name=task_name,
fn=_noop,
depends_on=spec.needs,
allow_upstream_skip=spec.allow_upstream_skip,
strategy=spec.strategy,
)
# fn 任务
sig = inspect.signature(spec.func)
kwargs: dict[str, Any] = {}
for pname in sig.parameters:
if pname in variables:
kwargs[pname] = variables[pname]
cwd_value = variables.get("cwd")
cwd = Path(cwd_value) if cwd_value is not None else None
return TaskSpec(
name=task_name,
fn=spec.func,
kwargs=kwargs,
depends_on=spec.needs,
cwd=cwd,
env=spec.env,
retry=spec.retry or RetryPolicy(),
timeout=spec.timeout,
allow_upstream_skip=spec.allow_upstream_skip,
strategy=spec.strategy,
)
# ---------------------------------------------------------------------- #
# Typer app 构建 (函数签名 → typer 命令)
# ---------------------------------------------------------------------- #
def _global_option_params() -> list[inspect.Parameter]:
"""构建全局选项的 inspect.Parameter 列表.
各命令 dispatcher 追加这些参数到用户函数签名末尾, 使 ``--dry-run`` / ``--quiet`` /
``--strategy`` 出现在每个子命令上. ``--list`` :func:`run_tool` 预处理.
"""
return [
inspect.Parameter(
"dry_run",
inspect.Parameter.KEYWORD_ONLY,
default=typer.Option(False, "--dry-run", help="仅打印执行计划, 不执行"),
annotation=bool,
),
inspect.Parameter(
"quiet",
inspect.Parameter.KEYWORD_ONLY,
default=typer.Option(False, "--quiet", "-q", help="减少输出"),
annotation=bool,
),
inspect.Parameter(
"strategy",
inspect.Parameter.KEYWORD_ONLY,
default=typer.Option(None, "--strategy", help="执行策略 (sequential/thread/dependency)"),
annotation=str | None,
),
]
def _make_dispatcher(tool_name: str, spec: ToolSpec) -> Callable[..., None]:
"""为 spec 创建 typer 命令 dispatcher.
dispatcher 签名 = 用户函数签名 (过滤 ``*args``/``**kwargs``) + 全局选项;
body kwargs 分离全局选项后调用 :func:`_execute_dag`.
"""
func = spec.func
subcommand = spec.subcommand
user_sig = inspect.signature(func)
# 过滤 *args / **kwargs, 追加全局选项
params = [
p
for p in user_sig.parameters.values()
if p.kind not in (inspect.Parameter.VAR_POSITIONAL, inspect.Parameter.VAR_KEYWORD)
]
params.extend(_global_option_params())
combined_sig = user_sig.replace(parameters=params)
def dispatcher(**kwargs: Any) -> None:
global_args = {
"dry_run": kwargs.pop("dry_run", False),
"verbose": not kwargs.pop("quiet", False),
"strategy": kwargs.pop("strategy", None),
}
_execute_dag(tool_name, subcommand, kwargs, global_args)
dispatcher.__signature__ = combined_sig # type: ignore[attr-defined]
dispatcher.__doc__ = spec.help or func.__doc__ or ""
dispatcher.__name__ = func.__name__
dispatcher.__annotations__ = {p.name: p.annotation for p in params}
return dispatcher
def _build_typer_app(name: str, all_subs: dict[str | None, ToolSpec]) -> typer.Typer:
"""构建 typer app.
单命令工具 ( subcommand=None) callback(invoke_without_command=True);
subcommand 工具 各可见 subcommand 注册为命令 (``--list`` run_tool 预处理).
"""
first_spec = next(iter(all_subs.values()))
description = first_spec.description or first_spec.help or name
is_single = None in all_subs and len(all_subs) == 1
app = typer.Typer(
name=f"pf {name}",
help=description,
# 单命令工具: 无参数时直接执行 callback; 多命令工具: 无 subcommand 时打印 help
no_args_is_help=not is_single,
add_completion=False,
rich_markup_mode="rich",
)
if is_single:
spec = all_subs[None]
dispatcher = _make_dispatcher(name, spec)
app.callback(invoke_without_command=True)(dispatcher)
return app
@app.callback()
def _callback() -> None:
"""工具入口; ``--list`` 由 :func:`run_tool` 预处理."""
for sc, spec in sorted(all_subs.items(), key=lambda x: x[0] or ""):
if sc is None or spec.hidden:
continue
dispatcher = _make_dispatcher(name, spec)
app.command(name=sc, help=spec.help)(dispatcher)
return app
# ---------------------------------------------------------------------- #
# 执行入口
# ---------------------------------------------------------------------- #
def _execute_dag(
tool_name: str,
subcommand: str | None,
variables: dict[str, Any],
global_args: dict[str, Any],
) -> None:
"""收集依赖, 构建 Graph, 执行 DAG.
成功正常返回; 失败时打印 rich 错误并 ``raise typer.Exit``.
"""
all_subs = _TOOL_REGISTRY[tool_name]
if subcommand not in all_subs:
Console(stderr=True).print(f"[red]错误:[/red] 未知子命令 [yellow]{subcommand!r}[/yellow]")
raise typer.Exit(ToolExitCode.FAILURE)
target_spec = all_subs[subcommand]
# 收集依赖 + 构建子图
all_names = _collect_with_deps(tool_name, subcommand)
specs: list[TaskSpec] = []
for sc in all_names:
dep_spec = all_subs[sc]
specs.append(_build_task_spec(dep_spec, variables))
graph = Graph.from_specs(specs)
console = Console()
console.print(f"[bold cyan][{tool_name}][/bold cyan] 执行: [yellow]{subcommand or '全部任务'}[/yellow]")
try:
run(
graph,
strategy=global_args["strategy"] or target_spec.strategy or "dependency",
dry_run=global_args["dry_run"],
verbose=global_args["verbose"],
)
except PyFlowXError as e:
Console(stderr=True).print(f"[red]错误:[/red] {e}")
raise typer.Exit(ToolExitCode.FAILURE) from e
except KeyboardInterrupt:
Console(stderr=True).print("\n[yellow]已中断[/yellow]")
raise typer.Exit(ToolExitCode.INTERRUPTED) from None
def _handle_list_rich(name: str, all_subs: dict[str | None, ToolSpec]) -> None:
"""rich 表格打印任务列表."""
console = Console()
console.print(f"[bold]工具 {name} 的任务列表[/bold]")
table = Table(show_header=True, header_style="bold")
table.add_column("任务")
table.add_column("依赖")
table.add_column("类型")
table.add_column("状态")
for sc, spec in sorted(all_subs.items(), key=lambda x: (x[0] is not None, x[0] or "")):
display = sc if sc is not None else name
deps = ", ".join(spec.needs) if spec.needs else ""
if spec.cmd is not None:
task_type = "cmd"
elif _is_aggregate(spec):
task_type = "聚合"
else:
task_type = "fn"
hidden = "[dim]隐藏[/dim]" if spec.hidden else ""
table.add_row(display, deps, task_type, hidden)
console.print(table)
def run_tool(name: str, argv: Sequence[str] | None = None) -> int:
"""执行工具, 返回退出码 (0 成功 / 1 失败 / 130 中断).
Parameters
----------
name : str
工具名 (已通过 ``@px.tool`` 注册)
argv : Sequence[str] | None
命令行参数 (不含工具名); None ``sys.argv[1:]``
Returns
-------
int
退出码
"""
if name not in _TOOL_REGISTRY:
Console(stderr=True).print(f"[red]错误:[/red] 未注册工具 [yellow]{name!r}[/yellow]")
return ToolExitCode.FAILURE
all_subs = _TOOL_REGISTRY[name]
argv = list(argv) if argv is not None else sys.argv[1:]
# --list 预处理: 出现在任意位置即列出任务后退出
if "--list" in argv:
_handle_list_rich(name, all_subs)
return ToolExitCode.SUCCESS
app = _build_typer_app(name, all_subs)
try:
# standalone_mode=False: click 捕获 typer.Exit 后返回 exit_code (而非抛出);
# ClickException/Abort 则向上抛出, 由下方 except 分支处理
result = app(args=argv, standalone_mode=False)
# 正常完成返回 None; typer.Exit 返回 int exit_code
if isinstance(result, int) and result != 0:
return (
ToolExitCode(result)
if result in (ToolExitCode.SUCCESS, ToolExitCode.FAILURE, ToolExitCode.INTERRUPTED)
else ToolExitCode.FAILURE
)
return ToolExitCode.SUCCESS
except _NoArgsIsHelpError:
# 多命令工具无 subcommand → 打印 help, 视为成功
return ToolExitCode.SUCCESS
except _TyperClickException as e:
return ToolExitCode.SUCCESS if e.exit_code == 0 else ToolExitCode.FAILURE
except typer.Abort:
return ToolExitCode.INTERRUPTED
except SystemExit as e:
code = e.code
if isinstance(code, int):
return code
return ToolExitCode.SUCCESS if code in (None, "0") else ToolExitCode.FAILURE
except KeyboardInterrupt:
return ToolExitCode.INTERRUPTED
File diff suppressed because it is too large Load Diff
+1 -1
View File
@@ -5,7 +5,7 @@ from __future__ import annotations
from pathlib import Path
from unittest.mock import MagicMock, patch
from pyflowx.ops import dev
from pyflowx.ops import autofmt as dev
# ---------------------------------------------------------------------- #
+136 -216
View File
@@ -1,13 +1,13 @@
"""Tests for cli.bumpversion module."""
"""Tests for ops.bumpversion module."""
from __future__ import annotations
import os
import subprocess
from pathlib import Path
import pytest
from pyflowx.ops import dev
from pyflowx.ops import bumpversion
@pytest.fixture(autouse=True)
@@ -22,251 +22,171 @@ def auto_use_tmp_path(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
class TestBumpFileVersion:
"""Test bump_file_version function."""
def test_bump_patch_version(self, tmp_path: Path) -> None:
"""Should bump patch version correctly."""
test_file = tmp_path / "pyproject.toml"
test_file.write_text('version = "1.2.3"', encoding="utf-8")
@pytest.mark.parametrize(
("part", "expected"),
[("patch", "1.2.4"), ("minor", "1.3.0"), ("major", "2.0.0")],
)
def test_bump_pyproject(self, tmp_path: Path, part: str, expected: str) -> None:
"""pyproject.toml 三种 part 递增."""
f = tmp_path / "pyproject.toml"
f.write_text('version = "1.2.3"', encoding="utf-8")
assert bumpversion.bump_file_version(f, part) == expected # type: ignore[arg-type]
assert f'version = "{expected}"' in f.read_text(encoding="utf-8")
result = dev.bump_file_version(test_file, "patch")
@pytest.mark.parametrize(
("part", "expected"),
[("patch", "1.2.4"), ("minor", "1.3.0"), ("major", "2.0.0")],
)
def test_bump_init_py(self, tmp_path: Path, part: str, expected: str) -> None:
"""__init__.py 三种 part 递增."""
f = tmp_path / "__init__.py"
f.write_text('__version__ = "1.2.3"', encoding="utf-8")
assert bumpversion.bump_file_version(f, part) == expected # type: ignore[arg-type]
assert f'__version__ = "{expected}"' in f.read_text(encoding="utf-8")
assert result == "1.2.4"
assert test_file.read_text(encoding="utf-8") == 'version = "1.2.4"'
def test_bump_minor_version(self, tmp_path: Path) -> None:
"""Should bump minor version correctly."""
test_file = tmp_path / "pyproject.toml"
test_file.write_text('version = "1.2.3"', encoding="utf-8")
result = dev.bump_file_version(test_file, "minor")
assert result == "1.3.0"
assert test_file.read_text(encoding="utf-8") == 'version = "1.3.0"'
def test_bump_major_version(self, tmp_path: Path) -> None:
"""Should bump major version correctly."""
test_file = tmp_path / "pyproject.toml"
test_file.write_text('version = "1.2.3"', encoding="utf-8")
result = dev.bump_file_version(test_file, "major")
assert result == "2.0.0"
assert test_file.read_text(encoding="utf-8") == 'version = "2.0.0"'
def test_version_pattern_with_prerelease(self, tmp_path: Path) -> None:
"""Should handle version with prerelease suffix."""
test_file = tmp_path / "pyproject.toml"
test_file.write_text('version = "1.2.3-alpha.1"', encoding="utf-8")
result = dev.bump_file_version(test_file, "patch")
assert result == "1.2.4"
# 预发布版本应该被清除
content = test_file.read_text(encoding="utf-8")
def test_prerelease_and_build_metadata_stripped(self, tmp_path: Path) -> None:
"""prerelease 和 build metadata 应被清除."""
f = tmp_path / "pyproject.toml"
f.write_text('version = "1.2.3-alpha.1+build.123"', encoding="utf-8")
assert bumpversion.bump_file_version(f, "patch") == "1.2.4"
content = f.read_text(encoding="utf-8")
assert "alpha" not in content
def test_version_pattern_with_build_metadata(self, tmp_path: Path) -> None:
"""Should handle version with build metadata."""
test_file = tmp_path / "pyproject.toml"
test_file.write_text('version = "1.2.3+build.123"', encoding="utf-8")
result = dev.bump_file_version(test_file, "patch")
assert result == "1.2.4"
# 构建元数据应该被清除
content = test_file.read_text(encoding="utf-8")
assert "build" not in content
def test_no_version_found(self, tmp_path: Path, capsys: pytest.CaptureFixture[str]) -> None:
"""Should return None when no version pattern found."""
test_file = tmp_path / "test.txt"
test_file.write_text("no version here", encoding="utf-8")
def test_dependencies_not_modified(self, tmp_path: Path) -> None:
"""只更新 project version, 不动 dependencies 中的版本号."""
f = tmp_path / "pyproject.toml"
f.write_text(
'[project]\nversion = "1.0.0"\ndependencies = ["lib >= 2.0.0", "other >= 3.0.0"]\n',
encoding="utf-8",
)
assert bumpversion.bump_file_version(f, "patch") == "1.0.1"
content = f.read_text(encoding="utf-8")
assert 'version = "1.0.1"' in content
assert "lib >= 2.0.0" in content
assert "other >= 3.0.0" in content
result = dev.bump_file_version(test_file, "patch")
def test_no_version_pattern_returns_none(self, tmp_path: Path, capsys: pytest.CaptureFixture[str]) -> None:
"""未匹配到版本号模式返回 None (支持类型但无版本 / 不支持的文件类型)."""
f1 = tmp_path / "__init__.py"
f1.write_text("# no version here", encoding="utf-8")
assert bumpversion.bump_file_version(f1, "patch") is None
assert "未找到版本号模式" in capsys.readouterr().out
assert result is None
captured = capsys.readouterr()
assert "未找到版本号模式" in captured.out
def test_utf8_encoding(self, tmp_path: Path) -> None:
"""Should handle UTF-8 encoded files correctly."""
test_file = tmp_path / "__init__.py"
test_file.write_text('__version__ = "1.2.3"', encoding="utf-8")
result = dev.bump_file_version(test_file, "patch")
assert result == "1.2.4"
assert test_file.read_text(encoding="utf-8") == '__version__ = "1.2.4"'
def test_pyproject_toml_format(self, tmp_path: Path) -> None:
"""Should handle pyproject.toml format correctly."""
test_file = tmp_path / "pyproject.toml"
content = """
[project]
name = "test"
version = "0.1.0"
description = "Test project"
"""
test_file.write_text(content, encoding="utf-8")
result = dev.bump_file_version(test_file, "minor")
assert result == "0.2.0"
updated = test_file.read_text(encoding="utf-8")
assert 'version = "0.2.0"' in updated
assert 'name = "test"' in updated
def test_init_py_format(self, tmp_path: Path) -> None:
"""Should handle __init__.py format correctly."""
test_file = tmp_path / "__init__.py"
content = '''"""Package info."""
__version__ = "1.0.0"
'''
test_file.write_text(content, encoding="utf-8")
result = dev.bump_file_version(test_file, "major")
assert result == "2.0.0"
updated = test_file.read_text(encoding="utf-8")
assert '__version__ = "2.0.0"' in updated
def test_multiple_versions_in_file(self, tmp_path: Path) -> None:
"""Should only bump the project version, not dependencies."""
test_file = tmp_path / "pyproject.toml"
content = """
[project]
version = "1.0.0"
dependencies = ["lib >= 2.0.0", "other >= 3.0.0"]
"""
test_file.write_text(content, encoding="utf-8")
result = dev.bump_file_version(test_file, "patch")
assert result == "1.0.1"
updated = test_file.read_text(encoding="utf-8")
assert 'version = "1.0.1"' in updated
# 确保 dependencies 中的版本没有被更新
assert "lib >= 2.0.0" in updated
assert "other >= 3.0.0" in updated
def test_file_read_error(self, tmp_path: Path, capsys: pytest.CaptureFixture[str]) -> None:
"""Should handle file read errors."""
# 创建一个目录而不是文件
test_file = tmp_path / "test_dir"
test_file.mkdir()
f2 = tmp_path / "test.txt"
f2.write_text("no version here", encoding="utf-8")
assert bumpversion.bump_file_version(f2, "patch") is None
def test_read_directory_raises(self, tmp_path: Path) -> None:
"""读取目录 (名为 __init__.py) 应抛异常."""
f = tmp_path / "__init__.py"
f.mkdir()
with pytest.raises(Exception): # noqa: B017
dev.bump_file_version(test_file, "patch")
bumpversion.bump_file_version(f, "patch")
def test_file_write_error(self, tmp_path: Path, capsys: pytest.CaptureFixture[str]) -> None:
"""Should handle file write errors."""
if os.geteuid() == 0:
pytest.skip("测试在 root 权限下不适用")
def test_write_failure_raises(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""写入失败应抛 OSError."""
f = tmp_path / "__init__.py"
f.write_text('__version__ = "1.0.0"', encoding="utf-8")
# 在只读目录中创建文件(这个测试在某些系统上可能不适用)
test_file = tmp_path / "readonly.toml"
test_file.write_text('version = "1.0.0"', encoding="utf-8")
# 设置为只读
test_file.chmod(0o444)
def raise_oserror(*_args: object, **_kwargs: object) -> None:
raise OSError("write failed")
try:
with pytest.raises(Exception): # noqa: B017
dev.bump_file_version(test_file, "patch")
finally:
# 恢复权限以便清理
test_file.chmod(0o644)
monkeypatch.setattr(Path, "write_text", raise_oserror)
with pytest.raises(OSError, match="write failed"):
bumpversion.bump_file_version(f, "patch")
# ---------------------------------------------------------------------- #
# Version pattern tests
# bump_project_version (核心 bug 修复: 不同步文件统一同步)
# ---------------------------------------------------------------------- #
class TestVersionPattern:
"""Test version pattern matching."""
class TestBumpProjectVersion:
"""Test bump_project_version function."""
def test_simple_version(self, tmp_path: Path) -> None:
"""Should match simple version."""
test_file = tmp_path / "__init__.py"
test_file.write_text('__version__ = "1.0.0"', encoding="utf-8")
@staticmethod
def _mock_subprocess(monkeypatch: pytest.MonkeyPatch) -> list[list[str]]:
"""Mock subprocess.run, 返回调用记录列表."""
calls: list[list[str]] = []
result = dev.bump_file_version(test_file, "patch")
def fake_run(cmd: list[str], **_kwargs: object) -> subprocess.CompletedProcess[bytes]:
calls.append(cmd)
return subprocess.CompletedProcess(cmd, 0, b"", b"")
assert result == "1.0.1"
monkeypatch.setattr(subprocess, "run", fake_run)
return calls
def test_version_with_zeros(self, tmp_path: Path) -> None:
"""Should handle versions with zeros correctly."""
test_file = tmp_path / "__init__.py"
test_file.write_text('__version__ = "0.0.0"', encoding="utf-8")
def test_unsynced_files_synchronized(
self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
) -> None:
"""核心 bug 修复: 不同步的文件应统一同步到同一新版本号.
result = dev.bump_file_version(test_file, "patch")
场景: __init__.py = 0.4.5, pyproject.toml = 0.3.5 (历史不同步)
期望: bump patch 后两者都变为 0.4.6 (取最大值 0.4.5 作为基准 +1)
"""
init_file = tmp_path / "src" / "pkg" / "__init__.py"
init_file.parent.mkdir(parents=True)
init_file.write_text('__version__ = "0.4.5"', encoding="utf-8")
pyproj = tmp_path / "pyproject.toml"
pyproj.write_text('version = "0.3.5"', encoding="utf-8")
assert result == "0.0.1"
calls = self._mock_subprocess(monkeypatch)
def test_large_version_numbers(self, tmp_path: Path) -> None:
"""Should handle large version numbers."""
test_file = tmp_path / "__init__.py"
test_file.write_text('__version__ = "10.20.30"', encoding="utf-8")
result = bumpversion.bump_project_version("patch")
result = dev.bump_file_version(test_file, "minor")
assert result == "0.4.6"
assert '__version__ = "0.4.6"' in init_file.read_text(encoding="utf-8")
assert 'version = "0.4.6"' in pyproj.read_text(encoding="utf-8")
out = capsys.readouterr().out
assert "基准版本: 0.4.5" in out
assert "新版本: 0.4.6" in out
assert result == "10.21.0"
add_calls = [c for c in calls if c[:2] == ["git", "add"]]
assert len(add_calls) == 1
# 跨平台: Windows 上 Path 转换为反斜杠, 统一用正斜杠比较
init_path = str(init_file.relative_to(tmp_path)).replace("\\", "/")
assert init_path in [arg.replace("\\", "/") for arg in add_calls[0]]
assert "pyproject.toml" in add_calls[0]
assert "." not in add_calls[0][2:]
def test_version_in_url(self, tmp_path: Path) -> None:
"""Should not match version in URL or other contexts."""
test_file = tmp_path / "test.txt"
test_file.write_text("https://example.com/v1.2.3/download", encoding="utf-8")
tag_calls = [c for c in calls if c[:2] == ["git", "tag"]]
assert len(tag_calls) == 1
assert "v0.4.6" in tag_calls[0]
result = dev.bump_file_version(test_file, "patch")
def test_no_files_returns_none(self, capsys: pytest.CaptureFixture[str]) -> None:
"""无版本号文件返回 None."""
assert bumpversion.bump_project_version("patch") is None
assert "未找到包含版本号的文件" in capsys.readouterr().out
# 不应该匹配 URL 中的版本号
assert result is None
def test_files_without_version_returns_none(
self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
) -> None:
"""有文件但所有文件都无版本号返回 None."""
f = tmp_path / "__init__.py"
f.write_text("# no version here", encoding="utf-8")
self._mock_subprocess(monkeypatch)
assert bumpversion.bump_project_version("patch") is None
assert "未能从任何文件读取版本号" in capsys.readouterr().out
# ---------------------------------------------------------------------- #
# Edge cases
# ---------------------------------------------------------------------- #
class TestEdgeCases:
"""Test edge cases and error handling."""
def test_no_tag_skips_tag_creation(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""no_tag=True 跳过 tag 创建."""
pyproj = tmp_path / "pyproject.toml"
pyproj.write_text('version = "1.0.0"', encoding="utf-8")
def test_empty_file(self, tmp_path: Path, capsys: pytest.CaptureFixture[str]) -> None:
"""Should handle empty file."""
test_file = tmp_path / "empty.txt"
test_file.write_text("", encoding="utf-8")
calls = self._mock_subprocess(monkeypatch)
result = dev.bump_file_version(test_file, "patch")
assert bumpversion.bump_project_version("patch", no_tag=True) == "1.0.1"
assert not any(c[:2] == ["git", "tag"] for c in calls)
assert result is None
captured = capsys.readouterr()
assert "未找到版本号模式" in captured.out
def test_ignored_dirs_excluded(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
""".venv 等忽略目录中的版本号文件不被处理."""
venv_init = tmp_path / ".venv" / "lib" / "pkg" / "__init__.py"
venv_init.parent.mkdir(parents=True)
venv_init.write_text('__version__ = "0.1.0"', encoding="utf-8")
pyproj = tmp_path / "pyproject.toml"
pyproj.write_text('version = "1.0.0"', encoding="utf-8")
def test_file_with_special_chars(self, tmp_path: Path) -> None:
"""Should handle file with special characters."""
test_file = tmp_path / "__init__.py"
content = '# 中文注释\n__version__ = "1.0.0"\n# 特殊字符: @#$%'
test_file.write_text(content, encoding="utf-8")
self._mock_subprocess(monkeypatch)
result = dev.bump_file_version(test_file, "patch")
assert result == "1.0.1"
updated = test_file.read_text(encoding="utf-8")
assert "# 中文注释" in updated
assert "# 特殊字符: @#$%" in updated
def test_consecutive_bumps(self, tmp_path: Path) -> None:
"""Should handle consecutive version bumps correctly."""
test_file = tmp_path / "__init__.py"
test_file.write_text('__version__ = "1.0.0"', encoding="utf-8")
# 第一次 bump
result1 = dev.bump_file_version(test_file, "patch")
assert result1 == "1.0.1"
# 第二次 bump
result2 = dev.bump_file_version(test_file, "minor")
assert result2 == "1.1.0"
# 第三次 bump
result3 = dev.bump_file_version(test_file, "major")
assert result3 == "2.0.0"
# 验证最终结果
assert test_file.read_text(encoding="utf-8") == '__version__ = "2.0.0"'
assert bumpversion.bump_project_version("patch") == "1.0.1"
assert venv_init.read_text(encoding="utf-8") == '__version__ = "0.1.0"'
-21
View File
@@ -1,21 +0,0 @@
"""Tests for cli.clearscreen module."""
from __future__ import annotations
from unittest.mock import patch
import pyflowx as px
from pyflowx.cli.system import clearscreen
# ---------------------------------------------------------------------- #
# main function
# ---------------------------------------------------------------------- #
class TestMain:
"""Test main function."""
def test_main_creates_graph_and_runs(self) -> None:
"""main() should create a Graph and run it."""
with patch.object(px, "run") as mock_run:
clearscreen.main()
assert mock_run.called
+5 -3
View File
@@ -884,9 +884,11 @@ Date: Mon, {i + 1} Jan 2024 12:00:00 +0000
Body {i}
""")
with patch("sys.argv", ["emlmanager", "--dir", str(tmp_path), "--port", "8080"]), patch.object(
emlmanager, "ThreadingHTTPServer"
) as mock_server, patch("threading.Thread"):
with (
patch("sys.argv", ["emlmanager", "--dir", str(tmp_path), "--port", "8080"]),
patch.object(emlmanager, "ThreadingHTTPServer") as mock_server,
patch("threading.Thread"),
):
# Don't actually start the server
mock_server_instance = Mock()
mock_server.return_value = mock_server_instance
+389
View File
@@ -0,0 +1,389 @@
"""Tests for ops.envdev / ops.dockercmd 模块 (镜像源配置/Docker 登录)."""
from __future__ import annotations
import subprocess
from pathlib import Path
from unittest.mock import MagicMock
import pytest
from pyflowx.conditions import Constants
from pyflowx.ops.dockercmd import docker_login_tencent
from pyflowx.ops.envdev import (
download_rustup_script,
install_linux_docker,
install_linux_fonts,
install_linux_qt_libs,
install_rust_toolchain,
setup_conda_mirror,
setup_linux_system_mirror,
setup_python_mirror,
setup_rust_mirror,
)
# ---------------------------------------------------------------------- #
# setup_python_mirror
# ---------------------------------------------------------------------- #
class TestSetupPythonMirror:
"""``setup_python_mirror`` 函数测试."""
def test_unknown_mirror_skips(self, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]) -> None:
"""未知镜像源应打印提示并跳过."""
monkeypatch.setattr(subprocess, "run", lambda *_, **__: MagicMock())
setup_python_mirror("unknown_mirror")
captured = capsys.readouterr()
assert "未知 Python 镜像源" in captured.out
def test_known_mirror_writes_config(self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None:
"""已知镜像源应写入配置文件."""
monkeypatch.setattr(Path, "home", lambda: tmp_path)
setup_python_mirror("tsinghua")
# Linux 平台默认配置路径
config_path = tmp_path / ".pip" / "pip.conf"
if not config_path.exists():
config_path = tmp_path / "pip" / "pip.ini"
assert config_path.exists()
content = config_path.read_text(encoding="utf-8")
assert "pypi.tuna.tsinghua.edu.cn" in content
def test_linux_uses_pip_conf(self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None:
"""Linux 平台应写入 ~/.pip/pip.conf."""
monkeypatch.setattr(Constants, "IS_LINUX", True)
monkeypatch.setattr(Path, "home", lambda: tmp_path)
setup_python_mirror("tsinghua")
config_path = tmp_path / ".pip" / "pip.conf"
assert config_path.exists()
def test_non_linux_uses_pip_ini(self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None:
"""非 Linux 平台应写入 ~/pip/pip.ini."""
monkeypatch.setattr(Constants, "IS_LINUX", False)
monkeypatch.setattr(Path, "home", lambda: tmp_path)
setup_python_mirror("tsinghua")
config_path = tmp_path / "pip" / "pip.ini"
assert config_path.exists()
def test_sets_env_vars(self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None:
"""应设置 PIP_INDEX_URL 等环境变量."""
import os
monkeypatch.setattr(Path, "home", lambda: tmp_path)
monkeypatch.setattr(os, "environ", {})
setup_python_mirror("aliyun")
assert "PIP_INDEX_URL" in os.environ
assert "mirrors.aliyun.com" in os.environ["PIP_INDEX_URL"]
# ---------------------------------------------------------------------- #
# setup_conda_mirror
# ---------------------------------------------------------------------- #
class TestSetupCondaMirror:
"""``setup_conda_mirror`` 函数测试."""
def test_unknown_mirror_skips(self, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]) -> None:
"""未知镜像源应跳过."""
setup_conda_mirror("unknown")
captured = capsys.readouterr()
assert "未知 Conda 镜像源" in captured.out
def test_known_mirror_writes_condarc(self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None:
"""已知镜像源应写入 ~/.condarc."""
monkeypatch.setattr(Path, "home", lambda: tmp_path)
setup_conda_mirror("tsinghua")
condarc = tmp_path / ".condarc"
assert condarc.exists()
content = condarc.read_text(encoding="utf-8")
assert "tsinghua" in content
assert "channels:" in content
# ---------------------------------------------------------------------- #
# setup_rust_mirror
# ---------------------------------------------------------------------- #
class TestSetupRustMirror:
"""``setup_rust_mirror`` 函数测试."""
def test_unknown_mirror_skips(self, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]) -> None:
"""未知镜像源应跳过."""
setup_rust_mirror("unknown")
captured = capsys.readouterr()
assert "未知 Rust 镜像源" in captured.out
def test_known_mirror_writes_config(self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None:
"""已知镜像源应写入 ~/.cargo/config.toml."""
monkeypatch.setattr(Path, "home", lambda: tmp_path)
setup_rust_mirror("ustc", "nightly")
config = tmp_path / ".cargo" / "config.toml"
assert config.exists()
content = config.read_text(encoding="utf-8")
assert "ustc" in content
def test_creates_sccache_dir(self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None:
"""应创建 sccache 缓存目录."""
from pyflowx.ops import envdev as envdev_module
fake_sccache = tmp_path / ".cargo" / "sccache"
monkeypatch.setattr(envdev_module, "_RUST_SCCACHE_DIR", fake_sccache)
monkeypatch.setattr(Path, "home", lambda: tmp_path)
setup_rust_mirror("tsinghua")
assert fake_sccache.exists()
# ---------------------------------------------------------------------- #
# docker_login_tencent
# ---------------------------------------------------------------------- #
class TestDockerLoginTencent:
"""``docker_login_tencent`` 函数测试."""
def test_default_username_uses_getpass(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""未提供 username 时应使用 getpass.getuser."""
monkeypatch.setattr("getpass.getuser", lambda: "testuser")
ran_cmds: list[list[str]] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: ran_cmds.append(cmd) or MagicMock())
docker_login_tencent()
assert ran_cmds[0][0] == "docker"
assert ran_cmds[0][1] == "login"
assert "testuser" in ran_cmds[0]
assert "ccr.ccs.tencentyun.com" in ran_cmds[0]
def test_custom_username(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""提供 username 时应使用自定义用户名."""
ran_cmds: list[list[str]] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: ran_cmds.append(cmd) or MagicMock())
docker_login_tencent("myuser")
assert "myuser" in ran_cmds[0]
# ---------------------------------------------------------------------- #
# setup_linux_system_mirror
# ---------------------------------------------------------------------- #
class TestSetupLinuxSystemMirror:
"""``setup_linux_system_mirror`` 函数测试."""
def test_non_linux_skips(self, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]) -> None:
"""非 Linux 平台应跳过."""
monkeypatch.setattr(Constants, "IS_LINUX", False)
called: list[str] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: called.append(cmd))
setup_linux_system_mirror()
captured = capsys.readouterr()
assert "仅在 Linux" in captured.out
assert called == []
def test_linux_already_configured_skips(
self, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
) -> None:
"""Linux 上已配置国内镜像时应跳过."""
monkeypatch.setattr(Constants, "IS_LINUX", True)
def fake_read_text(self: Path, encoding: str = "utf-8") -> str:
return "tsinghua mirror configured"
monkeypatch.setattr(Path, "read_text", fake_read_text)
called: list[str] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: called.append(cmd))
setup_linux_system_mirror()
captured = capsys.readouterr()
assert "已配置" in captured.out
assert called == []
def test_linux_not_configured_runs_script(
self, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
) -> None:
"""Linux 上未配置镜像时应执行下载与安装脚本."""
monkeypatch.setattr(Constants, "IS_LINUX", True)
def fake_read_text(self: Path, encoding: str = "utf-8") -> str:
raise OSError("file not found")
monkeypatch.setattr(Path, "read_text", fake_read_text)
ran_cmds: list[str] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: ran_cmds.append(cmd) or MagicMock())
setup_linux_system_mirror()
captured = capsys.readouterr()
assert "下载" in captured.out
assert "安装" in captured.out
assert len(ran_cmds) == 2
def test_linux_content_without_mirror_runs_script(
self, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
) -> None:
"""Linux 上文件存在但不包含镜像关键词时应执行脚本."""
monkeypatch.setattr(Constants, "IS_LINUX", True)
def fake_read_text(self: Path, encoding: str = "utf-8") -> str:
return "deb http://archive.ubuntu.com/ubuntu/ jammy main"
monkeypatch.setattr(Path, "read_text", fake_read_text)
ran_cmds: list[str] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: ran_cmds.append(cmd) or MagicMock())
setup_linux_system_mirror()
captured = capsys.readouterr()
assert "下载" in captured.out
assert len(ran_cmds) == 2
# ---------------------------------------------------------------------- #
# install_linux_qt_libs / install_linux_fonts / install_linux_docker
# ---------------------------------------------------------------------- #
class TestLinuxInstallers:
"""Linux 专用安装函数测试."""
def test_qt_libs_non_linux_skips(self, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]) -> None:
"""非 Linux 上 install_linux_qt_libs 应跳过."""
monkeypatch.setattr(Constants, "IS_LINUX", False)
called: list[list[str]] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: called.append(cmd))
install_linux_qt_libs()
captured = capsys.readouterr()
assert "仅在 Linux" in captured.out
assert called == []
def test_qt_libs_linux_runs_apt(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""Linux 上应执行 apt install."""
monkeypatch.setattr(Constants, "IS_LINUX", True)
ran_cmds: list[list[str]] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: ran_cmds.append(cmd) or MagicMock())
install_linux_qt_libs()
assert ran_cmds[0][0] == "sudo"
assert "apt" in ran_cmds[0]
assert "install" in ran_cmds[0]
def test_fonts_non_linux_skips(self, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]) -> None:
"""非 Linux 上 install_linux_fonts 应跳过."""
monkeypatch.setattr(Constants, "IS_LINUX", False)
called: list[list[str]] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: called.append(cmd))
install_linux_fonts()
captured = capsys.readouterr()
assert "仅在 Linux" in captured.out
assert called == []
def test_fonts_linux_runs_apt(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""Linux 上应执行 apt install 字体包."""
monkeypatch.setattr(Constants, "IS_LINUX", True)
ran_cmds: list[list[str]] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: ran_cmds.append(cmd) or MagicMock())
install_linux_fonts()
assert "fonts-noto-cjk" in ran_cmds[0]
def test_docker_non_linux_skips(self, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]) -> None:
"""非 Linux 上 install_linux_docker 应跳过."""
monkeypatch.setattr(Constants, "IS_LINUX", False)
called: list[list[str]] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: called.append(cmd))
install_linux_docker()
captured = capsys.readouterr()
assert "仅在 Linux" in captured.out
assert called == []
def test_docker_linux_runs_install_and_usermod(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""Linux 上应执行 apt install docker-compose-v2 和 usermod."""
monkeypatch.setattr(Constants, "IS_LINUX", True)
monkeypatch.setattr("getpass.getuser", lambda: "testuser")
ran_cmds: list[list[str]] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: ran_cmds.append(cmd) or MagicMock())
install_linux_docker()
assert any("docker-compose-v2" in cmd for cmd in ran_cmds)
assert any("usermod" in cmd and "docker" in cmd for cmd in ran_cmds)
# ---------------------------------------------------------------------- #
# download_rustup_script
# ---------------------------------------------------------------------- #
class TestDownloadRustupScript:
"""``download_rustup_script`` 函数测试."""
def test_rustup_installed_skips(self, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]) -> None:
"""rustup 已安装时应跳过."""
monkeypatch.setattr("shutil.which", lambda _cmd: "/usr/bin/rustup")
called: list[list[str]] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: called.append(cmd))
download_rustup_script()
captured = capsys.readouterr()
assert "已安装" in captured.out
assert called == []
def test_windows_downloads_exe(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""Windows 上应下载 rustup-init.exe."""
monkeypatch.setattr("shutil.which", lambda _cmd: None)
monkeypatch.setattr(Constants, "IS_WINDOWS", True)
ran_cmds: list[list[str]] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: ran_cmds.append(cmd) or MagicMock())
download_rustup_script()
assert "powershell" in ran_cmds[0]
assert "rustup-init.exe" in ran_cmds[0]
def test_non_windows_downloads_sh(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""非 Windows 上应下载 rustup-init.sh."""
monkeypatch.setattr("shutil.which", lambda _cmd: None)
monkeypatch.setattr(Constants, "IS_WINDOWS", False)
ran_cmds: list[list[str]] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: ran_cmds.append(cmd) or MagicMock())
download_rustup_script()
assert "curl" in ran_cmds[0]
assert "rustup-init.sh" in ran_cmds[0]
# ---------------------------------------------------------------------- #
# install_rust_toolchain
# ---------------------------------------------------------------------- #
class TestInstallRustToolchain:
"""``install_rust_toolchain`` 函数测试."""
def test_rustup_not_installed_skips(
self, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
) -> None:
"""rustup 未安装时应跳过."""
monkeypatch.setattr("shutil.which", lambda _cmd: None)
called: list[list[str]] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: called.append(cmd))
install_rust_toolchain()
captured = capsys.readouterr()
assert "未安装" in captured.out
assert called == []
def test_rustup_installed_runs_toolchain_install(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""rustup 已安装时应执行 toolchain install."""
monkeypatch.setattr("shutil.which", lambda _cmd: "/usr/bin/rustup")
ran_cmds: list[list[str]] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: ran_cmds.append(cmd) or MagicMock())
install_rust_toolchain("nightly")
assert ran_cmds == [["rustup", "toolchain", "install", "nightly"]]
def test_default_version_stable(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""默认版本应为 stable."""
monkeypatch.setattr("shutil.which", lambda _cmd: "/usr/bin/rustup")
ran_cmds: list[list[str]] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: ran_cmds.append(cmd) or MagicMock())
install_rust_toolchain()
assert "stable" in ran_cmds[0]
+99
View File
@@ -0,0 +1,99 @@
"""Tests for ops.envdev / sglang / msdownload 模块 (@px.tool 注册验证)."""
from __future__ import annotations
import pyflowx as px
from pyflowx.ops import envdev, msdownload, sglang # noqa: F401 -- 触发 @px.tool 注册
from pyflowx.tools import _TOOL_REGISTRY, get_tool
# ---------------------------------------------------------------------- #
# envdev 注册验证
# ---------------------------------------------------------------------- #
class TestEnvdevRegistration:
"""``envdev`` 模块 ``@px.tool`` 注册验证."""
def test_envdev_registered(self) -> None:
"""envdev 应在 _TOOL_REGISTRY 中注册."""
assert "envdev" in _TOOL_REGISTRY
def test_visible_subcommands(self) -> None:
"""可见子命令应包含核心镜像源配置命令."""
subs = px.list_subcommands("envdev")
assert "setup-python" in subs
assert "setup-conda" in subs
assert "setup-rust" in subs
assert "setup-linux-mirror" in subs
def test_install_subcommands_visible(self) -> None:
"""安装类子命令应可见."""
subs = px.list_subcommands("envdev")
assert "download-rustup" in subs
assert "install-rust" in subs
assert "install-qt-libs" in subs
assert "install-fonts" in subs
assert "install-docker" in subs
def test_setup_python_is_fn(self) -> None:
"""setup-python 应为 fn 任务 (非 cmd)."""
spec = get_tool("envdev", "setup-python")
assert spec.func is not None
assert spec.cmd is None
def test_setup_conda_is_fn(self) -> None:
"""setup-conda 应为 fn 任务."""
spec = get_tool("envdev", "setup-conda")
assert spec.func is not None
def test_setup_rust_is_fn(self) -> None:
"""setup-rust 应为 fn 任务."""
spec = get_tool("envdev", "setup-rust")
assert spec.func is not None
def test_setup_linux_mirror_is_fn(self) -> None:
"""setup-linux-mirror 应为 fn 任务."""
spec = get_tool("envdev", "setup-linux-mirror")
assert spec.func is not None
# ---------------------------------------------------------------------- #
# sglang 注册验证
# ---------------------------------------------------------------------- #
class TestSglangRegistration:
"""``sglang`` 模块 ``@px.tool`` 注册验证."""
def test_sglang_registered(self) -> None:
"""sglang 应在 _TOOL_REGISTRY 中注册."""
assert "sglang" in _TOOL_REGISTRY
def test_visible_subcommands(self) -> None:
"""可见子命令: install, run."""
subs = px.list_subcommands("sglang")
assert "install" in subs
assert "run" in subs
def test_run_needs_install(self) -> None:
"""run 应依赖 install."""
spec = get_tool("sglang", "run")
assert "install" in spec.needs
# ---------------------------------------------------------------------- #
# msdownload 注册验证
# ---------------------------------------------------------------------- #
class TestMsdownloadRegistration:
"""``msdownload`` 模块 ``@px.tool`` 注册验证."""
def test_msdownload_registered(self) -> None:
"""msdownload 应在 _TOOL_REGISTRY 中注册."""
assert "msdownload" in _TOOL_REGISTRY
def test_msdownload_default_spec(self) -> None:
"""msdownload 默认 spec 应可获取且为 fn 任务."""
spec = get_tool("msdownload")
assert spec.func is not None
assert spec.cmd is None
def test_msdownload_run_callable(self) -> None:
"""msdownload_run 应为可调用函数."""
assert callable(msdownload.msdownload_run)
+1 -1
View File
@@ -4,7 +4,7 @@ from __future__ import annotations
from pathlib import Path
from pyflowx.ops import files
from pyflowx.ops import filedate as files
# ---------------------------------------------------------------------- #
+1 -1
View File
@@ -4,7 +4,7 @@ from __future__ import annotations
from pathlib import Path
from pyflowx.ops import files
from pyflowx.ops import filelevel as files
# ---------------------------------------------------------------------- #
+1 -13
View File
@@ -5,7 +5,7 @@ from __future__ import annotations
from pathlib import Path
from unittest.mock import patch
from pyflowx.ops import files
from pyflowx.ops import folderback as files
# ---------------------------------------------------------------------- #
@@ -145,15 +145,3 @@ class TestBackupFolder:
files.backup_folder(str(source_dir), str(backup_dir), 5)
assert backup_dir.exists()
assert mock_zip.called
# ---------------------------------------------------------------------- #
# 函数注册
# ---------------------------------------------------------------------- #
class TestRegisteredFunctions:
"""Test that folderback functions are registered."""
def test_folderback_default_spec(self) -> None:
"""folderback_default should be a registered callable."""
# folderback_default 现在是通过 @px.register_fn 注册的普通函数, 不是 TaskSpec
assert callable(files.folderback_default)
+1 -13
View File
@@ -5,7 +5,7 @@ from __future__ import annotations
from pathlib import Path
from unittest.mock import patch
from pyflowx.ops import files
from pyflowx.ops import folderzip as files
# ---------------------------------------------------------------------- #
@@ -47,15 +47,3 @@ class TestZipFolders:
"""Should handle nonexistent cwd."""
files.zip_folders(str(tmp_path / "nonexistent"))
# Should print error message and return
# ---------------------------------------------------------------------- #
# 函数注册
# ---------------------------------------------------------------------- #
class TestRegisteredFunctions:
"""Test that folderzip functions are registered."""
def test_folderzip_default_spec(self) -> None:
"""folderzip_default should be a registered callable."""
# folderzip_default 现在是通过 @px.register_fn 注册的普通函数, 不是 TaskSpec
assert callable(files.folderzip_default)
+1 -1
View File
@@ -8,7 +8,7 @@ from unittest.mock import patch
import pytest
import pyflowx as px
from pyflowx.ops import dev
from pyflowx.ops import gittool as dev
# ---------------------------------------------------------------------- #
+267
View File
@@ -0,0 +1,267 @@
"""Tests for ops.gittool 模块 (@px.tool 注册验证)."""
from __future__ import annotations
import subprocess
from pathlib import Path
from unittest.mock import patch
import pytest
import pyflowx as px
import pyflowx.ops.gittool
from pyflowx.ops import gittool
from pyflowx.tools import _TOOL_REGISTRY, get_tool
# ---------------------------------------------------------------------- #
# @px.tool 注册验证
# ---------------------------------------------------------------------- #
class TestGittoolRegistration:
"""``gittool`` 模块 ``@px.tool`` 注册验证."""
def test_gittool_registered(self) -> None:
"""gittool 应在 _TOOL_REGISTRY 中注册."""
assert "gittool" in _TOOL_REGISTRY
def test_visible_subcommands(self) -> None:
"""可见子命令: a, c, i, isub, p, pl."""
subs = px.list_subcommands("gittool")
assert "a" in subs
assert "c" in subs
assert "i" in subs
assert "isub" in subs
assert "p" in subs
assert "pl" in subs
def test_clean_is_hidden(self) -> None:
"""clean 应为 hidden (不在可见子命令中)."""
subs = px.list_subcommands("gittool")
assert "clean" not in subs
def test_clean_hidden_included_with_flag(self) -> None:
"""include_hidden=True 时 clean 应出现."""
subs = px.list_subcommands("gittool", include_hidden=True)
assert "clean" in subs
def test_c_needs_clean(self) -> None:
"""c 应依赖 clean."""
spec = get_tool("gittool", "c")
assert "clean" in spec.needs
def test_clean_has_cmd(self) -> None:
"""clean 应有 cmd."""
spec = get_tool("gittool", "clean")
assert spec.cmd is not None
assert "git" in spec.cmd
assert "clean" in spec.cmd
assert "-xfd" in spec.cmd
def test_clean_cmd_includes_excludes(self) -> None:
"""clean cmd 应包含排除项."""
spec = get_tool("gittool", "clean")
assert spec.cmd is not None
assert "-e" in spec.cmd
assert ".venv" in spec.cmd
def test_c_has_cmd(self) -> None:
"""c 应有 cmd."""
spec = get_tool("gittool", "c")
assert spec.cmd == ("git", "status", "--porcelain")
def test_p_has_cmd(self) -> None:
"""p 应有 cmd."""
spec = get_tool("gittool", "p")
assert spec.cmd == ("git", "push")
def test_pl_has_cmd(self) -> None:
"""pl 应有 cmd."""
spec = get_tool("gittool", "pl")
assert spec.cmd == ("git", "pull")
# ---------------------------------------------------------------------- #
# 辅助函数验证 (not_has_git_repo / has_files)
# ---------------------------------------------------------------------- #
class TestGittoolHelpers:
"""``gittool`` 模块辅助函数验证."""
def test_not_has_git_repo_true(self, tmp_path: Path) -> None:
"""无 .git 目录时 not_has_git_repo 应返回 True."""
with patch.object(Path, "cwd", return_value=tmp_path):
assert gittool.not_has_git_repo() is True
def test_not_has_git_repo_false(self, tmp_path: Path) -> None:
"""有 .git 目录时 not_has_git_repo 应返回 False."""
(tmp_path / ".git").mkdir()
with patch.object(Path, "cwd", return_value=tmp_path):
assert gittool.not_has_git_repo() is False
def test_has_files_false_when_no_changes(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""无未提交更改时 has_files 应返回 False."""
class _FakeResult:
stdout = ""
monkeypatch.setattr("subprocess.run", lambda *_, **__: _FakeResult())
assert gittool.has_files() is False
def test_has_files_true_when_changes(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""有未提交更改时 has_files 应返回 True."""
class _FakeResult:
stdout = " M test.txt\n"
monkeypatch.setattr("subprocess.run", lambda *_, **__: _FakeResult())
assert gittool.has_files() is True
def test_has_files_false_on_subprocess_error(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""subprocess.SubprocessError 时 has_files 应返回 False."""
def _raise(*args: object, **kwargs: object) -> None:
raise subprocess.SubprocessError("fail")
monkeypatch.setattr("subprocess.run", _raise)
assert gittool.has_files() is False
def test_has_files_false_on_oserror(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""OSError 时 has_files 应返回 False."""
def _raise(*args: object, **kwargs: object) -> None:
raise OSError("fail")
monkeypatch.setattr("subprocess.run", _raise)
assert gittool.has_files() is False
# ---------------------------------------------------------------------- #
# git_add_commit / git_init_add_commit 函数体验证
# ---------------------------------------------------------------------- #
class TestGittoolCommit:
"""``gittool`` 模块提交函数验证."""
def test_git_add_commit_no_changes(
self,
monkeypatch: pytest.MonkeyPatch,
capsys: pytest.CaptureFixture[str],
) -> None:
"""无更改时 git_add_commit 应打印提示且不执行 git 命令."""
monkeypatch.setattr(gittool, "has_files", lambda: False)
calls: list[list[str]] = []
def _capture(*args: object, **kwargs: object) -> None:
calls.append(args[0]) # type: ignore[arg-type]
monkeypatch.setattr("subprocess.run", _capture)
gittool.git_add_commit()
assert calls == []
assert "没有文件需要提交" in capsys.readouterr().out
def test_git_add_commit_with_changes(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""有更改时 git_add_commit 应执行 git add + git commit."""
monkeypatch.setattr(gittool, "has_files", lambda: True)
calls: list[list[str]] = []
def _capture(*args: object, **kwargs: object) -> None:
calls.append(args[0]) # type: ignore[arg-type]
monkeypatch.setattr("subprocess.run", _capture)
gittool.git_add_commit(message="custom msg")
assert calls == [["git", "add", "."], ["git", "commit", "-m", "custom msg"]]
def test_git_init_add_commit_init_and_commit(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""无仓库且有更改时 git_init_add_commit 应 git init + add + commit."""
monkeypatch.setattr(gittool, "not_has_git_repo", lambda: True)
monkeypatch.setattr(gittool, "has_files", lambda: True)
calls: list[list[str]] = []
def _capture(*args: object, **kwargs: object) -> None:
calls.append(args[0]) # type: ignore[arg-type]
monkeypatch.setattr("subprocess.run", _capture)
gittool.git_init_add_commit()
assert calls == [["git", "init"], ["git", "add", "."], ["git", "commit", "-m", "init commit"]]
def test_git_init_add_commit_init_no_changes(
self,
monkeypatch: pytest.MonkeyPatch,
capsys: pytest.CaptureFixture[str],
) -> None:
"""无仓库且无更改时 git_init_add_commit 应 git init + 打印提示."""
monkeypatch.setattr(gittool, "not_has_git_repo", lambda: True)
monkeypatch.setattr(gittool, "has_files", lambda: False)
calls: list[list[str]] = []
def _capture(*args: object, **kwargs: object) -> None:
calls.append(args[0]) # type: ignore[arg-type]
monkeypatch.setattr("subprocess.run", _capture)
gittool.git_init_add_commit()
assert calls == [["git", "init"]]
assert "没有文件需要提交" in capsys.readouterr().out
def test_git_init_add_commit_no_init_with_changes(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""有仓库且有更改时 git_init_add_commit 应 add + commit (不 init)."""
monkeypatch.setattr(gittool, "not_has_git_repo", lambda: False)
monkeypatch.setattr(gittool, "has_files", lambda: True)
calls: list[list[str]] = []
def _capture(*args: object, **kwargs: object) -> None:
calls.append(args[0]) # type: ignore[arg-type]
monkeypatch.setattr("subprocess.run", _capture)
gittool.git_init_add_commit()
assert calls == [["git", "add", "."], ["git", "commit", "-m", "init commit"]]
def test_git_init_add_commit_no_init_no_changes(
self,
monkeypatch: pytest.MonkeyPatch,
capsys: pytest.CaptureFixture[str],
) -> None:
"""有仓库且无更改时 git_init_add_commit 应仅打印提示 (不 init, 不 add/commit)."""
monkeypatch.setattr(gittool, "not_has_git_repo", lambda: False)
monkeypatch.setattr(gittool, "has_files", lambda: False)
calls: list[list[str]] = []
def _capture(*args: object, **kwargs: object) -> None:
calls.append(args[0]) # type: ignore[arg-type]
monkeypatch.setattr("subprocess.run", _capture)
gittool.git_init_add_commit()
assert calls == []
assert "没有文件需要提交" in capsys.readouterr().out
# ---------------------------------------------------------------------- #
# init_sub_dirs 函数验证
# ---------------------------------------------------------------------- #
class TestGittoolInitSubDirs:
"""``init_sub_dirs`` 函数验证."""
def test_init_sub_dirs_runs_graph_for_each_subdir(
self,
monkeypatch: pytest.MonkeyPatch,
tmp_path: Path,
) -> None:
"""init_sub_dirs 应为每个子目录构建并运行图 (文件被跳过)."""
(tmp_path / "subdir1").mkdir()
(tmp_path / "subdir2").mkdir()
(tmp_path / "file.txt").write_text("test")
monkeypatch.setattr(Path, "cwd", lambda: tmp_path)
runs: list[object] = []
monkeypatch.setattr(gittool.px, "run", runs.append)
gittool.init_sub_dirs()
assert len(runs) == 2
def test_init_sub_dirs_no_subdirs(
self,
monkeypatch: pytest.MonkeyPatch,
tmp_path: Path,
) -> None:
"""无子目录时 init_sub_dirs 不应运行任何图."""
(tmp_path / "file.txt").write_text("test")
monkeypatch.setattr(Path, "cwd", lambda: tmp_path)
runs: list[object] = []
monkeypatch.setattr(gittool.px, "run", runs.append)
gittool.init_sub_dirs()
assert runs == []
+169
View File
@@ -0,0 +1,169 @@
"""Tests for ops.msdownload / ops.sglang 模块 (ModelScope 下载/SGLang 服务)."""
from __future__ import annotations
import subprocess
from pathlib import Path
from unittest.mock import MagicMock
import pytest
from pyflowx.conditions import Constants
from pyflowx.ops.msdownload import msdownload_run
from pyflowx.ops.sglang import install_sglang, run_sglang
# ---------------------------------------------------------------------- #
# msdownload_run
# ---------------------------------------------------------------------- #
class TestMsdownloadRun:
"""``msdownload_run`` 函数测试."""
def test_empty_name_does_nothing(self, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]) -> None:
"""name 为空时应直接返回."""
called: list[list[str]] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: called.append(cmd))
msdownload_run("")
captured = capsys.readouterr()
assert "name 不能为空" in captured.out
assert called == []
def test_default_download_dir(self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None:
"""未提供 download_dir 时默认使用 ~/.models/<name 最后一段>."""
monkeypatch.setattr(Path, "home", lambda: tmp_path)
ran_cmds: list[list[str]] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: ran_cmds.append(cmd) or MagicMock())
msdownload_run("Qwen/Qwen2.5-Coder")
expected_dir = tmp_path / ".models" / "Qwen2.5-Coder"
assert expected_dir.exists()
assert ran_cmds[0] == [
"uvx",
"modelscope",
"download",
"--model",
"Qwen/Qwen2.5-Coder",
"--local_dir",
str(expected_dir),
]
def test_custom_download_dir(self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None:
"""提供 download_dir 时应使用指定目录."""
custom_dir = tmp_path / "custom"
ran_cmds: list[list[str]] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: ran_cmds.append(cmd) or MagicMock())
msdownload_run("Qwen/Qwen2.5", "dataset", str(custom_dir))
assert custom_dir.exists()
assert ran_cmds[0][3] == "--dataset"
assert str(custom_dir) in ran_cmds[0]
def test_dataset_type(self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None:
"""target_type=dataset 时应传递 --dataset."""
monkeypatch.setattr(Path, "home", lambda: tmp_path)
ran_cmds: list[list[str]] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: ran_cmds.append(cmd) or MagicMock())
msdownload_run("foo/bar", "dataset")
assert "--dataset" in ran_cmds[0]
# ---------------------------------------------------------------------- #
# install_sglang
# ---------------------------------------------------------------------- #
class TestInstallSglang:
"""``install_sglang`` 函数测试."""
def test_already_installed_skips(self, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]) -> None:
"""sglang 已安装时应跳过."""
monkeypatch.setattr("shutil.which", lambda _cmd: "/usr/bin/sglang")
called: list[list[str]] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: called.append(cmd))
install_sglang()
captured = capsys.readouterr()
assert "已安装" in captured.out
assert called == []
def test_not_installed_runs_uv_install(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""sglang 未安装时应执行 uv install sglang[all]."""
monkeypatch.setattr("shutil.which", lambda _cmd: None)
ran_cmds: list[list[str]] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: ran_cmds.append(cmd) or MagicMock())
install_sglang()
assert ran_cmds == [["uv", "install", "sglang[all]"]]
# ---------------------------------------------------------------------- #
# run_sglang
# ---------------------------------------------------------------------- #
class TestRunSglang:
"""``run_sglang`` 函数测试."""
def test_model_dir_not_exist_skips(
self, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
) -> None:
"""模型目录不存在时应跳过."""
monkeypatch.setattr(Path, "exists", lambda _self: False)
called: list[list[str]] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: called.append(cmd))
run_sglang(model="/nonexistent/path")
captured = capsys.readouterr()
assert "模型目录不存在" in captured.out
assert called == []
def test_windows_uses_python(self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None:
"""Windows 上应使用 python."""
monkeypatch.setattr(Constants, "IS_WINDOWS", True)
monkeypatch.setattr(Path, "expanduser", lambda _self: tmp_path)
monkeypatch.setattr(Path, "exists", lambda _self: True)
ran_cmds: list[list[str]] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: ran_cmds.append(cmd) or MagicMock())
run_sglang(model=str(tmp_path))
assert ran_cmds[0][0] == "python"
assert "-m" in ran_cmds[0]
assert "sglang.launch_server" in ran_cmds[0]
def test_non_windows_uses_python3(self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None:
"""非 Windows 上应使用 python3."""
monkeypatch.setattr(Constants, "IS_WINDOWS", False)
monkeypatch.setattr(Path, "expanduser", lambda _self: tmp_path)
monkeypatch.setattr(Path, "exists", lambda _self: True)
ran_cmds: list[list[str]] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: ran_cmds.append(cmd) or MagicMock())
run_sglang(model=str(tmp_path), port=9000, ctx_len=4096, mem_fraction=0.5, host="127.0.0.1", log_level="debug")
cmd = ran_cmds[0]
assert cmd[0] == "python3"
assert "--port" in cmd
assert "9000" in cmd
assert "4096" in cmd
assert "0.5" in cmd
assert "127.0.0.1" in cmd
assert "debug" in cmd
def test_command_includes_qwen_parser(self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None:
"""命令应包含 --tool-call-parser qwen."""
monkeypatch.setattr(Constants, "IS_WINDOWS", True)
monkeypatch.setattr(Path, "expanduser", lambda _self: tmp_path)
monkeypatch.setattr(Path, "exists", lambda _self: True)
ran_cmds: list[list[str]] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: ran_cmds.append(cmd) or MagicMock())
run_sglang(model=str(tmp_path))
cmd = ran_cmds[0]
assert "--tool-call-parser" in cmd
qwen_idx = cmd.index("--tool-call-parser")
assert cmd[qwen_idx + 1] == "qwen"
+1 -1
View File
@@ -6,7 +6,7 @@ from pathlib import Path
from unittest.mock import MagicMock, patch
from pyflowx.conditions import Constants
from pyflowx.ops import system
from pyflowx.ops import lscalc as system
# ---------------------------------------------------------------------- #
+113
View File
@@ -0,0 +1,113 @@
"""Tests for ops._common 共享辅助模块."""
from __future__ import annotations
import pytest
from pyflowx.conditions import Constants
from pyflowx.ops._common import IGNORE_PATTERNS, ensure_platform, platform_command
# ---------------------------------------------------------------------- #
# IGNORE_PATTERNS
# ---------------------------------------------------------------------- #
class TestIgnorePatterns:
"""``IGNORE_PATTERNS`` 常量测试."""
def test_is_list_of_str(self) -> None:
"""应为 list[str]."""
assert isinstance(IGNORE_PATTERNS, list)
assert all(isinstance(p, str) for p in IGNORE_PATTERNS)
def test_contains_key_patterns(self) -> None:
"""应包含常见忽略目录/文件模式."""
for key in ["__pycache__", "*.pyc", ".git", ".venv", "dist", "build"]:
assert key in IGNORE_PATTERNS
def test_contains_ide_and_cache_dirs(self) -> None:
"""应包含 IDE 与工具缓存目录."""
for key in [".idea", ".vscode", ".pytest_cache", ".mypy_cache", ".tox"]:
assert key in IGNORE_PATTERNS
# ---------------------------------------------------------------------- #
# platform_command
# ---------------------------------------------------------------------- #
class TestPlatformCommand:
"""``platform_command`` 函数测试."""
def test_windows_returns_windows_cmd(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""Windows 上应返回 windows 命令."""
monkeypatch.setattr(Constants, "IS_WINDOWS", True)
result = platform_command(["cls"], ["clear"])
assert result == ["cls"]
def test_non_windows_returns_posix_cmd(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""非 Windows 上应返回 posix 命令."""
monkeypatch.setattr(Constants, "IS_WINDOWS", False)
result = platform_command(["cls"], ["clear"])
assert result == ["clear"]
def test_returns_copy_not_reference(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""返回值应为副本, 修改不影响后续调用."""
monkeypatch.setattr(Constants, "IS_WINDOWS", True)
result = platform_command(["where"], ["which"])
result.append("extra")
result2 = platform_command(["where"], ["which"])
assert result2 == ["where"]
assert "extra" not in result2
def test_multi_arg_command(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""多参数命令应完整保留."""
monkeypatch.setattr(Constants, "IS_WINDOWS", True)
result = platform_command(["taskkill", "/f", "/im"], ["pkill", "-f"])
assert result == ["taskkill", "/f", "/im"]
def test_index_access(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""应支持 [0] 索引取首元素."""
monkeypatch.setattr(Constants, "IS_WINDOWS", False)
cmd = platform_command(["python"], ["python3"])[0]
assert cmd == "python3"
# ---------------------------------------------------------------------- #
# ensure_platform
# ---------------------------------------------------------------------- #
class TestEnsurePlatform:
"""``ensure_platform`` 函数测试."""
def test_predicate_true_returns_true(
self, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
) -> None:
"""predicate 为 True 时应返回 True 且不打印."""
result = ensure_platform(True, "tool", "Windows")
assert result is True
captured = capsys.readouterr()
assert captured.out == ""
def test_predicate_false_returns_false(
self, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
) -> None:
"""predicate 为 False 时应返回 False 并打印提示."""
result = ensure_platform(False, "reset_icon_cache", "Windows")
assert result is False
captured = capsys.readouterr()
assert "仅在 Windows 上支持" in captured.out
assert "reset_icon_cache" in captured.out
def test_message_includes_tool_and_platform(self, capsys: pytest.CaptureFixture[str]) -> None:
"""提示信息应包含工具名与平台名."""
ensure_platform(False, "setup_linux_system_mirror", "Linux")
captured = capsys.readouterr()
assert "setup_linux_system_mirror" in captured.out
assert "Linux" in captured.out
def test_with_constants_predicate(
self, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
) -> None:
"""应能与 Constants.IS_* 谓词配合使用."""
monkeypatch.setattr(Constants, "IS_LINUX", False)
result = ensure_platform(Constants.IS_LINUX, "install_linux_docker", "Linux")
assert result is False
captured = capsys.readouterr()
assert "仅在 Linux 上支持" in captured.out
+16 -10
View File
@@ -7,7 +7,7 @@ from unittest.mock import MagicMock, patch
import pytest
from pyflowx.ops import system
from pyflowx.ops import packtool as system
@pytest.fixture(autouse=True)
@@ -103,9 +103,11 @@ class TestInstallEmbedPython:
output_dir = tmp_path / "python"
# Create a mock cache file that doesn't exist (force download)
with patch("platform.machine", return_value="x86_64"), patch(
"urllib.request.urlretrieve"
) as mock_urlretrieve, patch("zipfile.ZipFile") as mock_zipfile:
with (
patch("platform.machine", return_value="x86_64"),
patch("urllib.request.urlretrieve") as mock_urlretrieve,
patch("zipfile.ZipFile") as mock_zipfile,
):
# Mock successful download
mock_urlretrieve.return_value = None
mock_zip_instance = MagicMock()
@@ -190,9 +192,11 @@ class TestInstallEmbedPython:
"""Should handle different Python versions."""
output_dir = tmp_path / "python"
with patch("platform.machine", return_value="x86_64"), patch(
"urllib.request.urlretrieve"
) as mock_urlretrieve, patch("zipfile.ZipFile") as mock_zipfile:
with (
patch("platform.machine", return_value="x86_64"),
patch("urllib.request.urlretrieve") as mock_urlretrieve,
patch("zipfile.ZipFile") as mock_zipfile,
):
mock_zip_instance = MagicMock()
mock_zipfile.return_value.__enter__.return_value = mock_zip_instance
@@ -205,9 +209,11 @@ class TestInstallEmbedPython:
"""Should create cache directory and file."""
output_dir = tmp_path / "python"
with patch("platform.machine", return_value="x86_64"), patch(
"urllib.request.urlretrieve"
) as mock_urlretrieve, patch("zipfile.ZipFile") as mock_zipfile:
with (
patch("platform.machine", return_value="x86_64"),
patch("urllib.request.urlretrieve") as mock_urlretrieve,
patch("zipfile.ZipFile") as mock_zipfile,
):
mock_urlretrieve.return_value = None
mock_zip_instance = MagicMock()
mock_zipfile.return_value.__enter__.return_value = mock_zip_instance
+213 -4
View File
@@ -8,7 +8,7 @@ from unittest.mock import MagicMock, patch
import pytest
from pyflowx.ops import media
from pyflowx.ops import pdftool as media
# ---------------------------------------------------------------------- #
@@ -247,9 +247,11 @@ class TestPdfOcr:
input_file.write_bytes(b"PDF content")
output_file = tmp_path / "ocr.pdf"
with patch("fitz.open") as mock_fitz_open, patch("PIL.Image.frombytes"), patch(
"pytesseract.image_to_string"
) as mock_ocr:
with (
patch("fitz.open") as mock_fitz_open,
patch("PIL.Image.frombytes"),
patch("pytesseract.image_to_string") as mock_ocr,
):
mock_doc = MagicMock()
mock_page = MagicMock()
mock_page.rect = MagicMock(width=800, height=600)
@@ -278,3 +280,210 @@ class TestPdfRepair:
mock_fitz_open.return_value = mock_doc
media.pdf_repair(input_file, output_file)
assert mock_doc.save.called
# ---------------------------------------------------------------------- #
# pdf_encrypt
# ---------------------------------------------------------------------- #
class TestPdfEncrypt:
"""Test pdf_encrypt function."""
def test_pdf_encrypt_file(self, tmp_path: Path) -> None:
"""Should encrypt PDF."""
pytest.importorskip("pypdf")
input_file = tmp_path / "input.pdf"
input_file.write_bytes(b"PDF content")
output_file = tmp_path / "encrypted.pdf"
with patch("pypdf.PdfReader") as mock_reader, patch("pypdf.PdfWriter") as mock_writer:
mock_reader_instance = MagicMock()
mock_reader.return_value = mock_reader_instance
mock_reader_instance.pages = [MagicMock()]
mock_writer_instance = MagicMock()
mock_writer.return_value = mock_writer_instance
media.pdf_encrypt(input_file, output_file, "secret")
assert mock_writer_instance.encrypt.called
assert mock_writer_instance.write.called
def test_pdf_encrypt_not_installed(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""Should print message when pypdf not installed."""
monkeypatch.setattr(media, "HAS_PYPDF", False)
media.pdf_encrypt(tmp_path / "in.pdf", tmp_path / "out.pdf", "pw")
# ---------------------------------------------------------------------- #
# pdf_decrypt
# ---------------------------------------------------------------------- #
class TestPdfDecrypt:
"""Test pdf_decrypt function."""
def test_pdf_decrypt_encrypted(self, tmp_path: Path) -> None:
"""Should decrypt encrypted PDF."""
pytest.importorskip("pypdf")
input_file = tmp_path / "input.pdf"
input_file.write_bytes(b"PDF content")
output_file = tmp_path / "decrypted.pdf"
with patch("pypdf.PdfReader") as mock_reader, patch("pypdf.PdfWriter") as mock_writer:
mock_reader_instance = MagicMock()
mock_reader.return_value = mock_reader_instance
mock_reader_instance.is_encrypted = True
mock_reader_instance.pages = [MagicMock()]
mock_writer_instance = MagicMock()
mock_writer.return_value = mock_writer_instance
media.pdf_decrypt(input_file, output_file, "secret")
assert mock_reader_instance.decrypt.called
assert mock_writer_instance.write.called
def test_pdf_decrypt_not_encrypted(self, tmp_path: Path) -> None:
"""Should handle non-encrypted PDF."""
pytest.importorskip("pypdf")
input_file = tmp_path / "input.pdf"
input_file.write_bytes(b"PDF content")
output_file = tmp_path / "decrypted.pdf"
with patch("pypdf.PdfReader") as mock_reader, patch("pypdf.PdfWriter") as mock_writer:
mock_reader_instance = MagicMock()
mock_reader.return_value = mock_reader_instance
mock_reader_instance.is_encrypted = False
mock_reader_instance.pages = [MagicMock()]
mock_writer_instance = MagicMock()
mock_writer.return_value = mock_writer_instance
media.pdf_decrypt(input_file, output_file, "secret")
assert not mock_reader_instance.decrypt.called
def test_pdf_decrypt_not_installed(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""Should print message when pypdf not installed."""
monkeypatch.setattr(media, "HAS_PYPDF", False)
media.pdf_decrypt(tmp_path / "in.pdf", tmp_path / "out.pdf", "pw")
# ---------------------------------------------------------------------- #
# pdf_reorder
# ---------------------------------------------------------------------- #
class TestPdfReorder:
"""Test pdf_reorder function."""
def test_pdf_reorder_pages(self, tmp_path: Path) -> None:
"""Should reorder PDF pages."""
pytest.importorskip("pypdf")
input_file = tmp_path / "input.pdf"
input_file.write_bytes(b"PDF content")
output_file = tmp_path / "reordered.pdf"
with patch("pypdf.PdfReader") as mock_reader, patch("pypdf.PdfWriter") as mock_writer:
mock_reader_instance = MagicMock()
mock_reader.return_value = mock_reader_instance
mock_reader_instance.pages = [MagicMock(), MagicMock(), MagicMock()]
mock_writer_instance = MagicMock()
mock_writer.return_value = mock_writer_instance
media.pdf_reorder(input_file, output_file, [2, 0, 1])
assert mock_writer_instance.add_page.call_count == 3
def test_pdf_reorder_out_of_range(self, tmp_path: Path) -> None:
"""Should skip out-of-range page numbers."""
pytest.importorskip("pypdf")
input_file = tmp_path / "input.pdf"
input_file.write_bytes(b"PDF content")
output_file = tmp_path / "reordered.pdf"
with patch("pypdf.PdfReader") as mock_reader, patch("pypdf.PdfWriter") as mock_writer:
mock_reader_instance = MagicMock()
mock_reader.return_value = mock_reader_instance
mock_reader_instance.pages = [MagicMock()]
mock_writer_instance = MagicMock()
mock_writer.return_value = mock_writer_instance
media.pdf_reorder(input_file, output_file, [0, 5, -1])
assert mock_writer_instance.add_page.call_count == 1
def test_pdf_reorder_not_installed(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""Should print message when pypdf not installed."""
monkeypatch.setattr(media, "HAS_PYPDF", False)
media.pdf_reorder(tmp_path / "in.pdf", tmp_path / "out.pdf", [0])
# ---------------------------------------------------------------------- #
# pdf_to_images
# ---------------------------------------------------------------------- #
class TestPdfToImages:
"""Test pdf_to_images function."""
def test_pdf_to_images_convert(self, tmp_path: Path) -> None:
"""Should convert PDF to images."""
pytest.importorskip("fitz")
input_file = tmp_path / "input.pdf"
input_file.write_bytes(b"PDF content")
output_dir = tmp_path / "images"
with patch("fitz.open") as mock_fitz_open:
mock_doc = MagicMock()
mock_page = MagicMock()
mock_pixmap = MagicMock()
mock_page.get_pixmap.return_value = mock_pixmap
mock_doc.__iter__ = MagicMock(return_value=iter([mock_page]))
mock_fitz_open.return_value = mock_doc
media.pdf_to_images(input_file, output_dir, dpi=150)
assert output_dir.exists()
assert mock_pixmap.save.called
def test_pdf_to_images_not_installed(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""Should print message when PyMuPDF not installed."""
monkeypatch.setattr(media, "HAS_PYMUPDF", False)
media.pdf_to_images(tmp_path / "in.pdf", tmp_path / "out")
# ---------------------------------------------------------------------- #
# Not-installed branches for already-tested functions
# ---------------------------------------------------------------------- #
class TestNotInstalledBranches:
"""Test 'not installed' branches for PDF functions."""
def test_pdf_merge_not_installed(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""Should print message when pypdf not installed."""
monkeypatch.setattr(media, "HAS_PYPDF", False)
media.pdf_merge([tmp_path / "a.pdf"], tmp_path / "out.pdf")
def test_pdf_split_not_installed(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""Should print message when pypdf not installed."""
monkeypatch.setattr(media, "HAS_PYPDF", False)
media.pdf_split(tmp_path / "in.pdf", tmp_path / "out")
def test_pdf_compress_not_installed(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""Should print message when PyMuPDF not installed."""
monkeypatch.setattr(media, "HAS_PYMUPDF", False)
media.pdf_compress(tmp_path / "in.pdf", tmp_path / "out.pdf")
def test_pdf_extract_text_not_installed(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""Should print message when PyMuPDF not installed."""
monkeypatch.setattr(media, "HAS_PYMUPDF", False)
media.pdf_extract_text(tmp_path / "in.pdf", tmp_path / "out.txt")
def test_pdf_extract_images_not_installed(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""Should print message when PyMuPDF not installed."""
monkeypatch.setattr(media, "HAS_PYMUPDF", False)
media.pdf_extract_images(tmp_path / "in.pdf", tmp_path / "out")
def test_pdf_add_watermark_not_installed(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""Should print message when PyMuPDF not installed."""
monkeypatch.setattr(media, "HAS_PYMUPDF", False)
media.pdf_add_watermark(tmp_path / "in.pdf", tmp_path / "out.pdf")
def test_pdf_rotate_not_installed(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""Should print message when PyMuPDF not installed."""
monkeypatch.setattr(media, "HAS_PYMUPDF", False)
media.pdf_rotate(tmp_path / "in.pdf", tmp_path / "out.pdf")
def test_pdf_crop_not_installed(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""Should print message when PyMuPDF not installed."""
monkeypatch.setattr(media, "HAS_PYMUPDF", False)
media.pdf_crop(tmp_path / "in.pdf", tmp_path / "out.pdf", (1, 1, 1, 1))
def test_pdf_info_not_installed(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""Should print message when PyMuPDF not installed."""
monkeypatch.setattr(media, "HAS_PYMUPDF", False)
media.pdf_info(tmp_path / "in.pdf")
def test_pdf_repair_not_installed(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""Should print message when PyMuPDF not installed."""
monkeypatch.setattr(media, "HAS_PYMUPDF", False)
media.pdf_repair(tmp_path / "in.pdf", tmp_path / "out.pdf")
+5 -4
View File
@@ -6,7 +6,7 @@ import subprocess
from pathlib import Path
from unittest.mock import MagicMock, patch
from pyflowx.ops import dev
from pyflowx.ops import piptool as dev
# ---------------------------------------------------------------------- #
@@ -121,9 +121,10 @@ class TestPipUninstall:
def test_pip_uninstall_with_wildcard(self) -> None:
"""Should handle wildcard in package name."""
with patch.object(dev, "_expand_wildcard_packages", return_value=["numpy", "numpy-core"]), patch(
"subprocess.run"
) as mock_run:
with (
patch.object(dev, "_expand_wildcard_packages", return_value=["numpy", "numpy-core"]),
patch("subprocess.run") as mock_run,
):
mock_run.return_value = MagicMock(returncode=0)
dev.pip_uninstall(["numpy*"])
assert mock_run.called

Some files were not shown because too many files have changed in this diff Show More