Compare commits

..

33 Commits

Author SHA1 Message Date
zhou c9c7529c58 bump version to 0.4.6
CI / Lint, Typecheck & Test (push) Successful in 2m4s
Release / Build, Publish & Release (push) Successful in 46s
2026-07-05 19:09:10 +08:00
zhou c498d9b1c9 feat: 实现方向 B 聚合 job 并消除 pymake/reseticoncache CLI 入口
CI / Lint, Typecheck & Test (push) Successful in 2m3s
放宽 yaml_loader 校验允许有 needs 无 cmd/fn 的聚合 job, 完善 pymake.yaml 覆盖原 pymake.py 所有别名, 新增 reset_icon_cache_run fn 与 reseticoncache.yaml, 删除 pymake.py/reseticoncache.py 及对应 scripts 入口, 修复 --list 在 subcommands 模式下的可达性 bug.
2026-07-05 18:12:59 +08:00
zhou b36e279f92 feat(cli): add pymake project build tool support
add pymake command alias to CLI, create pymake config yaml and move its legacy tool config out of _LEGACY_TOOLS dict
2026-07-05 17:45:08 +08:00
zhou 58d6f1faad refactor: 迁移 cli/_ops/ 到 ops/, 按类别保持 dev/files/media/system 分类
CI / Lint, Typecheck & Test (push) Successful in 1m20s
将 src/pyflowx/cli/_ops/ 整体迁移至 src/pyflowx/ops/, 与 cli/ 平级
(工具函数非 CLI 专属, 可被 YAML 任务编排通用引用). 分类保持不变:
dev (git/pip/bump/autofmt), files (date/level/back/zip),
media (pdf/screenshot), system (ls/pack/ssh).

同步更新 15 个引用文件 (yaml_loader + 14 个测试) 的 import 路径,
README 模块结构表与 test_registry docstring.
2026-07-05 17:32:22 +08:00
zhou d93da0d8b4 refactor: 迁移 cli/_ops/ 到 ops/, 按类别保持 dev/files/media/system 分类
CI / Lint, Typecheck & Test (push) Successful in 1m35s
将 src/pyflowx/cli/_ops/ 整体迁移至 src/pyflowx/ops/, 与 cli/ 平级
(工具函数非 CLI 专属, 可被 YAML 任务编排通用引用). 分类保持不变:
dev (git/pip/bump/autofmt), files (date/level/back/zip),
media (pdf/screenshot), system (ls/pack/ssh).

同步更新 15 个引用文件 (yaml_loader + 14 个测试) 的 import 路径,
README 模块结构表与 test_registry docstring.
2026-07-05 17:30:35 +08:00
zhou 701c455c42 refactor: 用 YamlCliRunner/PfApp class 封装 CLI 入口逻辑
CI / Lint & Typecheck (push) Failing after 11m31s
CI / Test (Python 3.11) (push) Failing after 6m1s
CI / Test (Python 3.13) (push) Failing after 12m44s
CI / Test (Python 3.8) (push) Failing after 12m2s
CI / Docs Build (push) Failing after 6m56s
yaml_loader.py 新增 YamlCliRunner class, 将 run_cli 的 90 行
流程拆为 run/_load_config/_add_global_options/_extract_variables/
_handle_list 等单一职责方法; run_cli 保留为薄包装 (公共 API).
pf.py 新增 PfApp class, 将 main 的路由逻辑拆为 run/_list_tools/
_resolve_tool/_run_legacy/_run_yaml 等方法; main 仅调用
PfApp().run().

附带修复 run_yaml 的 jobs=None 误报 bug: 原代码 jobs is None
时 raise ValueError, 但 None 本意是执行全部任务, 导致
pf folderzip 报 "jobs 不能为空".
2026-07-05 15:59:05 +08:00
zhou e174b64495 refactor(cli): 重构配置文件路径并新增多工具配置
1. 移除旧的cli/configs/__init__.py占位文件
2. 修正pf.py中配置目录的路径指向
3. 新增folderzip、bumpversion等十余种工具的配置文件
4. 更新uv.lock中的依赖版本匹配规则
2026-07-05 15:50:18 +08:00
zhou 3afb25bb5e fix: 修正 typing-extensions 依赖条件为 python_version < '3.13'
CI / Lint & Typecheck (push) Failing after 15m3s
CI / Test (Python 3.11) (push) Failing after 24m15s
CI / Test (Python 3.13) (push) Failing after 27m17s
CI / Test (Python 3.8) (push) Failing after 5m56s
CI / Docs Build (push) Failing after 31s
task.py 在 Python < 3.13 时需要 typing_extensions 的 TypeVar
(PEP 696 default= 参数), 此前条件 < '3.10' 导致 3.10-3.12
环境 import 失败, ReadTheDocs (Python 3.11) 构建报
ModuleNotFoundError: No module named 'typing_extensions'.
2026-07-05 13:16:24 +08:00
zhou fbd17536fd ci: 重写 CI/Release 为 GitHub 兼容版本并加文档构建
CI / Lint & Typecheck (push) Failing after 8m55s
CI / Test (Python 3.11) (push) Failing after 31s
CI / Test (Python 3.13) (push) Failing after 31s
CI / Test (Python 3.8) (push) Failing after 31s
CI / Docs Build (push) Failing after 31s
ci.yml 改用标准 actions (checkout/setup-uv/setup-python), 新增
pyrefly 类型检查、coverage 阈值检查 (>=95%)、Sphinx 文档构建三个
job, 多版本矩阵测试 (py38/py311/py313)。release.yml 改用标准
actions, 发布到 PyPI + GitHub Release (替代原 Gitea Release)。
2026-07-05 12:32:00 +08:00
zhou 32ca8c1208 docs: 搭建 Sphinx 文档站并清理死代码
CI / Lint, Typecheck & Test (push) Successful in 1m57s
1. 新建 docs/ Sphinx 文档结构 (conf.py + 8 个 rst 章节),
   napoleon 支持 Google/NumPy docstring, rtd 主题,
   自动生成 API 参考与错误家族文档
2. 新建 .readthedocs.yaml 配置, pyproject.toml 加 docs 依赖
3. 删除 runner.py 的 _apply_verbose_to_graph 死代码及对应测试
   (功能已移入 executors.run 统一处理)
4. 更新 README: CLI 示例改为 pf 统一入口, 模块结构表补全
   cli/pf.py/cli/configs/cli/_ops 等模块
5. 修复版本不一致 (pyproject.toml 0.3.5 → 0.4.5)
6. 加文档徽章链接到 ReadTheDocs
2026-07-05 12:17:10 +08:00
zhou a7ff68d279 feat: pf 默认显示 verbose 执行过程, --quiet 关闭
CI / Lint, Typecheck & Test (push) Successful in 1m17s
run() 在 verbose=True 时自动把 verbose 标记应用到所有 spec,
使 execute_command 打印执行命令与返回码 (此前只 callback 打印
任务生命周期)。全局选项 --verbose 改为 --quiet (默认 verbose=True,
传 --quiet 关闭)。gittool CLEAN_EXCLUDES 补全 .pytest_cache/
.ruff_cache/.vscode/.trae/.qoder/.editorconfig 等目录。
2026-07-05 08:46:15 +08:00
zhou de368ea810 refactor: 删除冗余 cli 入口脚本, gittool 用数组配置 clean excludes
CI / Lint, Typecheck & Test (push) Successful in 1m11s
1. 删除 13 个已有 YAML 配置的 cli .py 入口脚本, 统一通过 pf 调用
2. gittool.yaml 用 CLEAN_EXCLUDES 数组变量配置 git clean 的 -e 参数,
   保留 .venv/.tox/node_modules/.idea 等目录避免误删
3. run_cli 执行前打印调用信息: [gittool] 执行: c
4. 更新 pyproject.toml 移除 13 个冗余 entry points, 仅保留 pf
5. 清理测试文件中的 TestMain 类 (测 _ops 模块的测试保留)
2026-07-05 08:39:20 +08:00
zhou 6a3e3a57cd fix: cmd 任务成功时打印 stdout
CI / Lint, Typecheck & Test (push) Failing after 50s
execute_command 在非 verbose 模式下捕获 stdout 后直接 return None,
导致 git status --porcelain 等命令的输出被丢弃。
现在成功时若有 stdout 则打印到终端, 保留失败时的 stderr 信息。
2026-07-05 00:52:38 +08:00
zhou 7089944306 build: 调整pyproject.toml中pf命令的脚本位置
将pf命令的脚本配置移至文件末尾,修正脚本条目排序
2026-07-05 00:50:16 +08:00
zhou ec5e348694 feat: 新增 pf 统一入口, YAML 配置自带 CLI 参数定义
CI / Lint, Typecheck & Test (push) Failing after 47s
新增 pf 统一 CLI 入口, 通过 YAML 的 cli: 段定义参数解析规则,
逐步消除工具 .py 入口文件。yaml_loader 新增 build_cli_parser
和 run_cli 函数, 支持 subcommands/positional/options 三级 schema,
内置 --dry-run/--verbose/--strategy/--list 全局选项。
13 个工具 YAML 配置全部添加 cli: 段。
2026-07-04 20:31:40 +08:00
zhou 12d9f2f647 fix: 恢复 gittool 条件逻辑,修复 has_files 检查 git status
CI / Lint, Typecheck & Test (push) Successful in 1m56s
将 gitt a/i 命令改用 fn job 包装(git_add_commit/git_init_add_commit),
内部检查 has_files() 和 not_has_git_repo() 条件,避免无更改时 git commit
报错。修正 has_files() 实现为检查 git status --porcelain 而非目录文件。
2026-07-04 20:00:25 +08:00
zhou 6ffcbecade chore: update 2026-07-04 19:57:20 +08:00
zhou e76d93187b chore: update 2026-07-04 19:55:09 +08:00
zhou 52e20e3f93 style: 统一调整代码格式,将单行列表展开为多行缩进格式 2026-07-04 19:49:10 +08:00
zhou 3f966a230e refactor: 简化 CLI 工具入口为 YAML 加载器
CI / Lint, Typecheck & Test (push) Successful in 2m5s
将 13 个工具入口文件重构为通过 px.run_yaml 调用 YAML 配置,
辅助函数移至 _ops 模块。新增 run_yaml 便捷函数支持 job 选择
和传递依赖收集,修复 _build_cmd 列表变量展开,新增 bump_project_version
高层函数封装版本号更新+git 提交流程。
2026-07-04 19:35:08 +08:00
zhou 5d0b211a44 feat: 新增 13 个 CLI 工具的 YAML 配置并修复 _ops 函数注册
CI / Lint, Typecheck & Test (push) Successful in 1m41s
- 在 cli/configs/ 下创建 13 个 YAML 工作流配置, 覆盖 filedate/filelevel/folderback/
  folderzip/autofmt/bumpversion/piptool/gittool/pdftool/screenshot/lscalc/
  sshcopyid/packtool 工具, 共 51 个 job (cmd 与 fn 混合)
- yaml_loader 模块级导入 _ops 子模块, 使 YAML fn 字段可引用注册函数,
  try/except 守卫避免最小安装场景下的 ImportError
- 修复 test_registry 的 clear_registry fixture: 保存/恢复 _REGISTRY 原始状态,
  避免 teardown 清空 _ops 自动注册的函数导致 TestOpsModules 失败
2026-07-04 18:35:20 +08:00
zhou 6931f36fd1 feat: 新增函数注册机制与 CLI 工具函数模块
CI / Lint, Typecheck & Test (push) Successful in 2m27s
- 新增 registry.py 提供 register_fn/get_fn/has_fn 函数注册机制, 支持 @register_fn 和 @register_fn("name") 两种用法
- 新增 cli/_ops 包 (files/dev/media/system 四个子模块), 聚合 59 个可复用函数供 YAML fn 字段引用
- 扩展 yaml_loader 支持 fn 字段、args/kwargs 传参、${VAR} 变量占位符
- 新增 test_registry.py (20 个测试) 和扩展 test_yaml_loader.py
- 更新自驱动规则: 自动 commit+push, 删除需要用户明确指示的步骤
2026-07-04 18:24:52 +08:00
zhou db02443463 feat: 新增 YAML 任务编排功能
1. 新增 yaml_loader 模块,支持加载 GitHub Actions 风格的 YAML 任务图
2. 新增 Graph.from_yaml 静态方法,支持从 YAML 文件构建任务图
3. 新增 yamlrun CLI 工具,支持执行、预览 YAML 任务流水线
4. 添加 pyyaml 运行时依赖与 types-PyYAML 开发依赖
5. 更新 README 文档与对外暴露的 API 接口
2026-07-04 16:00:04 +08:00
zhou eb8e1402bc docs: 更新自驱动规则文档,补充决策判据与细节
补充自主决策的具体范围、收尾规则,新增决策判据章节,细化暂停条件与沟通要求
2026-07-04 15:29:47 +08:00
zhou c93f45dcb8 refactor: 统一使用px.task/px.cmd替代旧版TaskSpec创建任务
本次提交将项目内所有使用px.TaskSpec创建任务的代码,替换为新的px.task和px.cmd快捷API,简化了任务定义写法,同时更新了版本号到0.3.5。重构过程中保持了原有功能逻辑不变,仅调整了代码书写格式,提升了代码可读性和编写效率。
2026-07-04 15:22:27 +08:00
zhou a0b1814024 style: 格式化sshcopyid.py的列表代码,提升可读性
调整了px.Graph.from_specs的参数列表排版,将多行列表缩进优化为更简洁的单行展开格式,不改变代码实际功能。
2026-07-04 13:43:33 +08:00
zhou 3a2826d3f9 bump version to 0.3.5
CI / Lint, Typecheck & Test (push) Successful in 1m47s
Release / Build, Publish & Release (push) Successful in 1m2s
2026-07-04 11:36:07 +08:00
zhou dbd30689ab chore(ci): 更新release工作流的gitea服务地址
将GITEA_URL从10.0.16.16:3000调整为172.17.0.1:3000,适配新的内网部署地址
2026-07-04 11:36:04 +08:00
zhou 5eb59b8a66 bump version to 0.3.4
CI / Lint, Typecheck & Test (push) Successful in 1m8s
Release / Build, Publish & Release (push) Failing after 30s
2026-07-04 11:24:11 +08:00
zhou 8e7b866de2 更新 .github/workflows/release.yml
CI / Lint, Typecheck & Test (push) Has been cancelled
2026-07-04 03:23:16 +00:00
zhou 1b4f9bfa6a bump version to 0.3.3
CI / Lint, Typecheck & Test (push) Successful in 1m8s
Release / Build, Publish & Release (push) Has been cancelled
2026-07-04 11:16:31 +08:00
zhou 2d39272330 ci(github workflows): update pypi api token secret name
将PyPI发布步骤中的密钥变量名从PYPI_API_TOKEN改为PYPI_TOKEN,保持配置一致性
2026-07-04 11:16:24 +08:00
zhou f699bb9167 chore: 升级pyflowx版本到0.3.2
CI / Lint, Typecheck & Test (push) Successful in 1m19s
2026-07-04 10:50:50 +08:00
101 changed files with 10599 additions and 4495 deletions
+2 -2
View File
@@ -27,14 +27,14 @@ jobs:
run: uv build
- name: Publish to pypi
run: uv publish --token '${{ secrets.PYPI_API_TOKEN }}'
run: uv publish --token '${{ secrets.PYPI_TOKEN }}'
- name: Create Gitea Release & Upload Assets
env:
GITEA_TOKEN: ${{ secrets.GITHUB_TOKEN }}
TAG_NAME: ${{ github.ref_name }}
REPO: ${{ github.repository }}
GITEA_URL: http://gitea:3000
GITEA_URL: http://172.17.0.1:3000
run: |
set -e
# 1. 创建 Release
+3
View File
@@ -11,3 +11,6 @@ wheels/
.coverage
.idea
*_profile.html
# Sphinx 文档构建输出
docs/_build/
+23
View File
@@ -0,0 +1,23 @@
# ReadTheDocs 配置
# https://docs.readthedocs.io/en/stable/config-file/v2.html
version: 2
# 构建配置
build:
os: ubuntu-24.04
tools:
python: "3.11"
# Python 依赖与构建命令
python:
install:
- method: pip
path: .
extra_requirements:
- docs
# Sphinx 构建
sphinx:
configuration: docs/conf.py
builder: html
fail_on_warning: false
+108
View File
@@ -0,0 +1,108 @@
# 文档整理与 Sphinx 文档搭建计划
## Context
最近完成 CLI 重构:新增 `pf` 统一入口,13 个工具迁移到 YAML 配置并删除了对应 .py 入口脚本,`run()` 的 verbose 统一应用到 spec。但文档未同步:README 仍引用旧命令(`yamlrun``python build.py`),模块结构表缺漏;`runner.py``_apply_verbose_to_graph` 成为死代码;项目缺少可发布的 Sphinx 文档。本次任务整理这些遗留,并搭建 ReadTheDocs 文档站。
## 任务范围
### 1. 清理死代码
- 删除 `src/pyflowx/runner.py``_apply_verbose_to_graph` 函数(line 38-68),功能已移入 `executors.py``run()`
- 删除 `tests/test_runner.py` 中对应测试(line 610-636`TestApplyVerboseToGraph` 类)。
- 清理 `runner.py` 顶部 `from dataclasses import replace` 若变为未使用。
### 2. 修复版本不一致
- `src/pyflowx/__init__.py:105` 硬编码 `__version__ = "0.4.5"``pyproject.toml:25``0.3.5`
- 统一为 `0.4.5``__init__.py` 为准,pyproject.toml 是源但 bumpversion 工具应同时更新两者)。
### 3. 更新 README.md
- L304-308`python build.py clean/build/test``pf pymake clean/build/test`
- L335-351、L435`yamlrun pipeline.yaml ...``pf yamlrun pipeline.yaml ...`6 处)。
- L311`verbose=True(默认)` 描述保留,但 CLI 示例改为 `pf`
- L558-574 模块结构表:补充 `cli/pf.py`(统一入口)、`cli/configs/`YAML 工具配置)、`cli/_ops/`(工具函数)、`profiling.py``registry.py`
- 顶部增加「文档」徽章链接到 ReadTheDocs。
### 4. 搭建 Sphinx 文档结构
新建 `docs/` 目录:
```
docs/
├── conf.py # Sphinx 配置
├── index.rst # 首页与目录
├── installation.rst # 安装
├── quickstart.rst # 快速上手(从 README 提炼)
├── guide/
│ ├── task.rst # TaskSpec 任务描述
│ ├── graph.rst # Graph DAG 构建
│ ├── execution.rst # 执行策略与 run()
│ ├── yaml.rst # YAML 任务编排
│ └── cli.rst # pf 统一入口与工具列表
├── api.rst # API 参考(automodule 自动生成)
└── changelog.rst # 变更日志摘要
```
**conf.py 要点**
- 扩展:`sphinx.ext.autodoc``sphinx.ext.napoleon`(支持 Google/NumPy docstring)、`sphinx.ext.viewcode``myst_parser`(支持 Markdown
- 主题:`sphinx_rtd_theme`
- 项目版本从 `pyflowx.__version__` 动态读取
- `autodoc_default_options``members: True, undoc-members: True, show-inheritance: True`
**api.rst**:用 `automodule:: pyflowx` 抓取 `__all__` 的 56 个公共符号。
### 5. ReadTheDocs 配置
- 新建 `.readthedocs.yaml`Python 3.11`pip install -e .[docs]``sphinx -b html docs/ docs/_build/`
- `.gitignore` 增加 `docs/_build/`
### 6. pyproject.toml 补充 docs 依赖
```toml
docs = [
"sphinx>=7.0",
"sphinx-rtd-theme>=2.0",
"myst-parser>=3.0",
]
```
并在 `[dependency-groups]` 的 dev 中加入 `pyflowx[docs]`
## 关键文件
| 文件 | 操作 |
|------|------|
| `src/pyflowx/runner.py` | 删除 `_apply_verbose_to_graph` |
| `tests/test_runner.py` | 删除 `TestApplyVerboseToGraph` |
| `src/pyflowx/__init__.py` | 版本统一(已 0.4.5,确认) |
| `pyproject.toml` | 版本 → 0.4.5;加 docs 依赖 |
| `README.md` | 更新 CLI 示例与模块结构表 |
| `docs/conf.py` | 新建 |
| `docs/*.rst` | 新建 |
| `.readthedocs.yaml` | 新建 |
| `.gitignore` | 加 docs/_build/ |
## 验证
1. **测试与 lint**
```bash
uv run pytest tests/ -q
uv run ruff check src/ tests/ docs/conf.py
uv run pyrefly check src/pyflowx/runner.py
```
2. **Sphinx 构建本地验证**
```bash
uv sync --extra docs
uv run sphinx-build -b html docs/ docs/_build/
```
确认无 warning,打开 `docs/_build/index.html` 检查页面。
3. **pf 功能回归**
```bash
pf gitt c
pf pymake b --dry-run
```
4. **RTD 配置校验**`.readthedocs.yaml` 语法正确,`docs/conf.py` 能独立构建。
## 不在范围
- 不统一各模块 docstring 风格(napoleon 兼容 Google/NumPy,够用)。
- 不重构现有 CLI 工具 YAML。
- 不新增中文文档翻译(文档用中文撰写,与项目既有风格一致)。
+1 -1
View File
@@ -150,7 +150,7 @@ uvx --from pyflowx pymake cov
## Git 与提交
- **自动提交/push**:除非用户明确要求
- **自动提交**:任务完成后自动 `git add`(按文件名)+ `git commit` + `git push`(仅当分支已跟踪远程时执行 push;新分支跳过 push 并在总结中说明)
- **不修改 git config**。
- **不运行破坏性命令**`push --force`/`reset --hard`/`clean -f`)除非用户明确要求。
- **staging**:按文件名添加,不用 `git add -A`/`git add .`,避免误加敏感文件。
+134
View File
@@ -0,0 +1,134 @@
---
alwaysApply: true
---
# 自驱动开发规则
本规则定义一种"目标驱动、闭环执行"的工作模式:仅在任务开始时与用户确认一次目标与边界,后续由 Agent 自主完成"计划 → 编码 → 测试 → 文档 → 验证"的迭代循环,直到用户目标达成。
## 核心原则
- **目标导向**:始终以用户最终目标为准绳,所有阶段产出都应服务于该目标。
- **闭环执行**:每个子任务必须走完"计划 → 实现 → 测试 → 文档 → 验证"五步;禁止跳步留半成品。
- **自主决策**:初始确认之后,实现路径、API 形态、重构范围、文件命名、测试组织、错误修复策略等由 Agent 自行决断,不再逐项请示。**可逆操作(编辑文件、运行测试、修复 lint、调整实现)直接执行,不询问**;只有不可逆/高风险操作才暂停。
- **透明沟通**:每个阶段开始前用一句话说明意图;关键节点(完成、阻塞、转向)给简短更新;不复述内部思考,**不在收尾时停下询问"是否继续"或"是否提交"**——直接输出总结并结束。
- **安全边界**:仅在高风险、不可逆操作或真正阻塞时才暂停找用户。
## 初始确认(一次性,仅在最开始)
任务启动时,用 `AskUserQuestion` 一次性确认以下信息(已由项目规范覆盖的不必重复确认):
1. **目标与范围**:要解决什么问题?交付物是什么?显式列出不在范围内的内容。
2. **验收标准**:怎样算"完成"?可观测的判定条件(功能、性能、覆盖率阈值)。
3. **特殊约束**:除 `python-standards.md` 之外的约束(兼容性、依赖限制、API 兼容策略等)。
4. **测试要求**:覆盖率门槛(项目默认 ≥95%,branch);是否需要新增 `slow` 标记。
**git commit/push 不在确认范围内**:任务完成后自动 commit + push(仅当分支已跟踪远程时执行 push;新分支跳过 push 并在总结中说明),遵循 `.trae/rules/git-commit-message.md` 风格。仅 force-push、reset --hard、clean -f、修改 git config 等真正破坏性操作才需暂停确认。
确认后,将目标与验收标准固化进 `TaskCreate` 任务列表,后续不再就同一信息反复询问。
## 迭代循环
下列五个阶段构成一个完整闭环。未达验收标准时,回到「计划」开启下一轮;达标准时,进入「收尾」。
### 1. 计划(Plan
- 用 Explore/Glob/Grep 研究相关代码与既有模式,避免凭空设计。
-`TaskCreate` 把目标拆为可独立验证的子任务;每完成一项立即 `TaskUpdate` 为 completed。
- 优先复用现有抽象;不为本轮假想需求设计接口。
- 不过早抽象:三处相似才考虑提取,否则就地写。
### 2. 实现(Code
- 严格遵守 `.trae/rules/python-standards.md` 与既有代码风格。
- 优先 Edit 现有文件;新增文件需有明确职责边界。
- 不引入运行时依赖(项目零依赖原则);确需引入须在计划阶段说明。
- 公共 API 必须有完整类型注解与中文 docstring。
- 不写未被要求的功能、不为未来场景预留扩展点。
### 3. 测试(Test
- 新增/修改的公共 API 必须配套测试;优先通过公共接口测试,故障注入可访问私有属性并在 docstring 注明。
- Mock 优先级:`monkeypatch` > 内联 stub > `unittest.mock` > `pytest-mock`;禁用 `@patch` 装饰器。
- 必跑校验(每次修改后):
```bash
uvx --from pyflowx pymake tc
uvx --from pyflowx pymake cov
```
- 测试失败时定位根因再修复,不通过放宽断言或 `# pragma: no cover` 绕过。
- 覆盖率不得低于上一次的值(项目门槛 95%,branch)。
### 4. 文档(Docs
- 同步更新 docstring、README、模块结构说明。
- 行为变更须同步更新 `.agents/skills/pyflowx-development/SKILL.md` 中的对应章节。
- 跨会话有价值的设计决策、约束、陷阱,追加到 memory(`project_memory.md` 或对应 `topics.md`)。
- 不主动新建 `*.md` 文档;除非用户明确要求。
### 5. 验证(Verify
- 逐条对照初始确认的「验收标准」核验;未满足则回到「计划」继续下一轮。
- 全套门禁通过:ruff、pyrefly、pytest、coverage。
- 给出本轮变更清单(改了哪些文件、为什么)。
## 暂停条件(仅在以下情况中断自驱动找用户)
1. **歧义无法自决**:需求存在多种合理解读且无既有约定可循。
2. **高风险/不可逆操作**:删除非临时文件、`git push --force`、`reset --hard`、删表、修改 CI 配置、修改 git config、卸载依赖等。**普通 `git commit`/`push` 不属于此类**(任务完成后自动执行)。
3. **不可恢复的失败**:根因不在本仓库、需外部环境/权限配合、或经两轮尝试仍无法定位。
4. **超出初始确认范围**:用户目标在执行中发现需要显著扩大范围或改变方向。
5. **用户主动询问**:用户在对话中提出新问题或要求澄清。
**注意**"目标已达成"**不是**暂停条件——验收标准全部满足后直接进入收尾并结束任务,不询问"是否扩展范围"或"是否提交"。
非以上情况,一律继续自驱动,不要为"求确认"而暂停。
## 决策判据:该问还是自决
遇到不确定时,按以下顺序判断:
1. **是否不可逆/高风险?** 是 → 暂停确认(如删除文件、`push --force`、修改 CI 配置、卸载依赖)。否 → 继续。
2. **是否在初始确认范围内?** 是 → 按确认执行,不询问。否 → 视为"超出初始确认范围",暂停。
3. **是否有既有约定可循?** 是 → 按约定执行(参考 `python-standards.md`、`project_memory.md`)。否 → 视为"歧义无法自决",暂停。
4. **是否可逆?** 是 → 直接执行,即使结果可能不完美(可在后续迭代修正)。否 → 暂停。
**可直接自决(不询问)的典型情况**:
- 测试失败、覆盖率不达标、lint/类型检查报错 → 定位根因并修复。
- 代码风格选择(命名、模块划分、参数顺序)→ 自决。
- 文件编辑、运行测试、运行校验命令 → 直接执行。
- 任务完成后输出收尾总结 → 直接输出,不询问下一步。
- 显式指定 `name` 参数以保持测试兼容性 → 自决。
- 重命名局部变量以避免遮蔽 → 自决。
**必须暂停询问的典型情况**
- 删除非临时文件、重命名公共模块/包。
- `git push --force`、`reset --hard`、`clean -f`、修改 git config(普通 commit/push 自动执行,无需询问)。
- 引入新的运行时依赖(违反项目零依赖原则)。
- 修改 CI 配置、pre-commit 钩子、pyproject.toml 的工具链配置。
- 卸载或降级既有依赖。
## 沟通风格
- 阶段切换时一句话说明即可;不要把内部推理写给用户看。
- 完成子任务后用一两句总结改了什么、下一步做什么。
- 遇到阻塞时直接说明:卡在哪、试了什么、需要用户做什么。
- **不在收尾时询问"是否需要提交"或"是否扩展范围"**——直接输出总结并结束。用户后续若有新需求,由用户主动提出。
- 不使用 emoji,除非用户明确要求。
## 工具使用
- 独立操作尽量并行调用(多个 Read/Grep/Glob 一批发出)。
- 用 `TaskCreate`/`TaskUpdate` 维护进度,不批量推迟标记。
- 长命令用后台运行(`run_in_background`),完成会自动通知。
- 文件操作一律用专用工具:Read/Edit/Write/Glob/Grep,不用 `cat`/`sed`/`grep`/`find`。
## 收尾
- 验收标准全部满足后,**直接输出最终总结并结束任务**:交付物、关键决策、遗留事项。
- **自动提交**:收尾时自动 `git add`(按文件名)+ `git commit`(遵循 `.trae/rules/git-commit-message.md` 风格)+ `git push`(仅当分支已跟踪远程时执行;新分支跳过 push 并在总结中说明);**不询问**"是否需要提交"或"是否扩展范围"。
- 若验收标准未全部满足,回到「计划」继续下一轮,不停下询问。
- 将本次会话的关键产出与决策更新到 memory,便于后续会话续接。
+140 -7
View File
@@ -5,6 +5,7 @@
[![CI](https://github.com/gookeryoung/pyflowx/actions/workflows/ci.yml/badge.svg)](https://github.com/gookeryoung/pyflowx/actions/workflows/ci.yml)
[![PyPI](https://img.shields.io/pypi/v/pyflowx.svg)](https://pypi.org/project/pyflowx/)
[![Python](https://img.shields.io/pypi/pyversions/pyflowx.svg)](https://pypi.org/project/pyflowx/)
[![Documentation Status](https://readthedocs.org/projects/pyflowx/badge/?version=latest)](https://pyflowx.readthedocs.io/zh/latest/)
[![Coverage](https://img.shields.io/badge/coverage-100%25-brightgreen.svg)](https://github.com/gookeryoung/pyflowx)
[![License](https://img.shields.io/pypi/l/pyflowx.svg)](https://github.com/gookeryoung/pyflowx/blob/main/LICENSE)
@@ -31,7 +32,8 @@ PyFlowX 把"任务依赖"这件事做到极致简单:**参数名就是依赖
- **图级默认值** —— `GraphDefaults` 统一配置 retry/timeout/concurrency 等
- **CLI 运行器** —— `CliRunner` 把多个图映射为命令行子命令,替代 Makefile
- **可观测** —— `on_event` 回调(RUNNING/SUCCESS/FAILED/SKIPPED)、`dry_run` 预览、`verbose` 生命周期日志、Mermaid 可视化
- **零运行时依赖** —— 仅依赖标准库(3.8 需 `graphlib_backport`
- **YAML 任务编排** —— GitHub Actions 风格的声明式任务图,支持 `jobs`/`needs`/`strategy.matrix`/`if` 等 CI/CD 概念,从 YAML 文件直接加载执行
- **最小依赖** —— 仅依赖标准库 + PyYAML3.8 需 `graphlib_backport``typing-extensions`
- **97% 测试覆盖** —— 分支覆盖率 >= 95%
## 安装
@@ -300,15 +302,123 @@ runner.run_cli() # 解析 sys.argv 并执行
命令行用法:
```bash
python build.py clean # 执行 clean 图
python build.py build --strategy thread # 覆盖执行策略
python build.py test --dry-run # 仅打印执行计划
python build.py --list # 列出所有命令
python build.py --quiet # 静默模式
pf pymake clean # 执行 clean 图
pf pymake build --strategy thread # 覆盖执行策略
pf pymake test --dry-run # 仅打印执行计划
pf pymake --list # 列出所有命令
pf pymake --quiet # 静默模式
```
`verbose=True`(默认)时打印任务生命周期(开始/成功/失败/跳过)与命令输出;`--quiet` 关闭。
## YAML 任务编排
PyFlowX 支持 GitHub Actions 风格的声明式 YAML 任务编排,从 YAML 文件直接加载任务图。
### 编程式 API
```python
import pyflowx as px
# 从 YAML 文件加载任务图
graph = px.Graph.from_yaml("pipeline.yaml")
report = px.run(graph, strategy="thread")
# 或用函数式 API
graph = px.load_yaml("pipeline.yaml")
graph = px.parse_yaml_string("""
jobs:
hello:
cmd: ["echo", "hello"]
""")
```
### CLI 入口
通过 `pf` 统一入口调用(详见 [pf 工具](#cli-工具) 章节):
```bash
# 执行 YAML 任务图
pf yamlrun pipeline.yaml
# 指定执行策略
pf yamlrun pipeline.yaml --strategy thread
# 仅打印任务分层,不执行
pf yamlrun pipeline.yaml --dry-run
# 列出所有任务名
pf yamlrun pipeline.yaml --list
# 静默模式
pf yamlrun pipeline.yaml --quiet
```
### YAML SchemaGitHub Actions 风格)
```yaml
strategy: thread # 图级默认策略
defaults: # 图级默认值
retry: {max_attempts: 3}
verbose: true
env: {CI: "true"}
jobs:
setup:
cmd: ["git", "clone", "..."]
runs-on: linux
build:
needs: [setup] # 依赖列表
cmd: ["python", "-m", "build"]
timeout: 300
retry: {max_attempts: 2, delay: 1.0}
test:
needs: [build]
cmd: ["python${{ matrix.version }}", "-m", "pytest"] # 矩阵占位符
strategy:
matrix: # 笛卡尔积展开为 6 个任务
version: ["3.8", "3.9", "3.10"]
os: ["linux", "macos"]
if: "env.CI" # 条件: 环境变量存在
lint:
needs: [build]
cmd: ["ruff", "check"]
if: "env.CI == 'true'" # 条件: 环境变量等于
deploy:
needs: [test, lint] # 矩阵依赖自动展开
cmd: ["twine", "upload"]
if: "env.DEPLOY_TOKEN != ''"
allow-upstream-skip: true
concurrency-key: deploy_lock
```
### 字段映射
| YAML 字段 | TaskSpec 字段 | 说明 |
|-----------|---------------|------|
| `jobs.<id>` | `name` | job ID 作为任务名 |
| `cmd` / `run` | `cmd` | `cmd` 为列表形式,`run` 为 shell 字符串 |
| `needs` | `depends_on` | 依赖列表(矩阵任务自动展开) |
| `if` | `conditions` | `success()` / `always()` / `env.VAR` / `env.VAR == 'x'` |
| `strategy.matrix` | 矩阵扇出 | 笛卡尔积展开为多个任务 |
| `${{ matrix.key }}` | 占位符 | 在 cmd/run/cwd/env 中替换 |
| `timeout` | `timeout` | 超时秒数 |
| `retry` | `retry` | `{max_attempts, delay, backoff, jitter}` |
| `cwd` | `cwd` | 工作目录 |
| `env` | `env` | 环境变量 |
| `verbose` | `verbose` | 详细输出 |
| `continue-on-error` | `continue_on_error` | 失败不中止整图 |
| `skip-if-missing` | `skip_if_missing` | 命令不存在时跳过 |
| `allow-upstream-skip` | `allow_upstream_skip` | 上游跳过时仍执行 |
| `priority` | `priority` | 同层优先级 |
| `concurrency-key` | `concurrency_key` | 并发限制键 |
| `tags` | `tags` | 自由标签 |
| `runs-on` | `tags`(追加) | 运行环境标签 |
## 示例
仓库 `examples/` 目录包含完整示例:
@@ -316,6 +426,7 @@ python build.py --quiet # 静默模式
- [`etl_pipeline.py`](examples/etl_pipeline.py) —— ETL 流水线(sequential
- [`parallel_run.py`](examples/parallel_run.py) —— 并行执行对比(thread vs sequential
- [`async_aggregation.py`](examples/async_aggregation.py) —— 异步聚合 + Context 注入
- [`yaml_pipeline.yaml`](examples/yaml_pipeline.yaml) + [`yaml_pipeline.py`](examples/yaml_pipeline.py) —— YAML 声明式 CI/CD 流水线(矩阵扇出 + 条件执行)
运行:
@@ -323,6 +434,8 @@ python build.py --quiet # 静默模式
python examples/etl_pipeline.py
python examples/parallel_run.py
python examples/async_aggregation.py
python -m pyflowx.examples.yaml_pipeline
pf yamlrun src/pyflowx/examples/yaml_pipeline.yaml --dry-run
```
## 断点续跑
@@ -447,6 +560,8 @@ uv run ruff format --check src tests examples
## 模块结构
### 核心
| 模块 | 职责 |
|------|------|
| `task.py` | 纯数据结构:`TaskSpec``RetryPolicy``TaskHooks``TaskStatus` |
@@ -455,11 +570,29 @@ uv run ruff format --check src tests examples
| `context.py` | 上下文注入:参数名→依赖解析 |
| `command.py` | 命令执行:`run_command`list/shell/Callable |
| `conditions.py` | 条件执行:内置条件与组合器 |
| `executors.py` | 执行器与 `run` 入口:四种策略共享模块级辅助 |
| `executors.py` | 执行器与 `run` 入口:四种策略共享模块级辅助verbose 统一应用到 spec |
| `storage.py` | 状态后端:`MemoryBackend` / `JSONBackend`batch flush |
| `runner.py` | CLI 运行器:`CliRunner` |
| `report.py` | 运行结果:`RunReport` / `TaskResult` |
| `yaml_loader.py` | YAML 任务编排:GitHub Actions 风格 schema 解析(`load_yaml` / `parse_yaml_string` / `run_cli` |
| `registry.py` | 函数注册中心:`register_fn` / `get_fn` / `has_fn`YAML 的 `fn:` 引用) |
| `profiling.py` | 性能分析:`Profiler` 任务耗时统计 |
| `errors.py` | 错误家族:`PyFlowXError` 子类 |
| `ops/` | 工具函数(dev/files/media/system),被 YAML 的 `fn:` 引用 |
### CLI 工具
| 模块 | 职责 |
|------|------|
| `cli/pf.py` | 统一入口:`pf <tool> [command]`,自动发现 `configs/*.yaml` 并路由 |
| `cli/configs/` | YAML 工具配置(gittool/filedate/pdftool 等 13 个工具的 `cli:` schema |
| `cli/pymake.py` | 构建工具(替代 Makefile),`pf pymake b` 调用 |
| `cli/yamlrun.py` | YAML pipeline 执行器,`pf yamlrun pipeline.yaml` 调用 |
| `cli/profiler.py` | 性能分析 CLI |
| `cli/emlmanager.py` | 邮件管理 CLI |
| `cli/dev/` | 开发工具(dockercmd/envdev |
| `cli/llm/` | LLM 工具(msdownload/sglang |
| `cli/system/` | 系统工具(clearscreen/taskkill/which |
## 许可证
View File
+106
View File
@@ -0,0 +1,106 @@
API 参考
========
任务描述
--------
.. autoclass:: pyflowx.TaskSpec
:members:
:undoc-members:
:show-inheritance:
:exclude-members: args, kwargs
.. autoclass:: pyflowx.RetryPolicy
:members:
:undoc-members:
.. autoclass:: pyflowx.TaskHooks
:members:
:undoc-members:
.. autoclass:: pyflowx.TaskStatus
:members:
:undoc-members:
图构建
------
.. autoclass:: pyflowx.Graph
:members:
:undoc-members:
:exclude-members: from_specs, from_yaml
.. autoclass:: pyflowx.GraphDefaults
:members:
:undoc-members:
.. autofunction:: pyflowx.compose
.. autofunction:: pyflowx.task_template
执行
----
.. autofunction:: pyflowx.run
.. autoclass:: pyflowx.RunReport
:members:
:undoc-members:
.. autoclass:: pyflowx.TaskResult
:members:
:undoc-members:
YAML 编排
---------
.. autofunction:: pyflowx.load_yaml
.. autofunction:: pyflowx.parse_yaml_string
.. autofunction:: pyflowx.run_yaml
.. autofunction:: pyflowx.run_cli
.. autofunction:: pyflowx.build_cli_parser
函数注册
--------
.. autofunction:: pyflowx.register_fn
.. autofunction:: pyflowx.get_fn
.. autofunction:: pyflowx.has_fn
命令执行
--------
.. autofunction:: pyflowx.run_command
CLI 运行器
----------
.. autoclass:: pyflowx.CliRunner
:members:
:undoc-members:
状态后端
--------
.. autoclass:: pyflowx.StateBackend
:members:
:undoc-members:
.. autoclass:: pyflowx.MemoryBackend
:members:
:undoc-members:
.. autoclass:: pyflowx.JSONBackend
:members:
:undoc-members:
错误家族
--------
.. autoexception:: pyflowx.PyFlowXError
.. autoexception:: pyflowx.DuplicateTaskError
.. autoexception:: pyflowx.MissingDependencyError
.. autoexception:: pyflowx.CycleError
.. autoexception:: pyflowx.TaskFailedError
.. autoexception:: pyflowx.TaskTimeoutError
.. autoexception:: pyflowx.InjectionError
.. autoexception:: pyflowx.StorageError
+45
View File
@@ -0,0 +1,45 @@
变更日志
========
0.4.5
-----
CLI 重构
~~~~~~~~
- 新增 ``pf`` 统一入口:通过 ``pf <tool> [command] [options]`` 调用所有工具
- 13 个工具迁移到 YAML 配置(filedate/filelevel/folderback/folderzip/screenshot/sshcopyid/lscalc/bumpversion/autofmt/piptool/packtool/pdftool/gittool
- YAML 配置支持 ``cli:`` 段声明命令行参数 schema,由 ``build_cli_parser`` 自动生成 argparse
- 删除 13 个冗余 ``.py`` 入口脚本,统一通过 ``pf`` 调用
- ``run()````verbose=True`` 时自动把 verbose 标记应用到所有 spec
- 全局选项 ``--verbose`` 改为 ``--quiet``(默认显示执行过程)
- ``cmd`` 任务成功时打印 stdout(此前被静默丢弃)
- ``gittool````CLEAN_EXCLUDES`` 数组变量配置 ``git clean -e`` 参数
YAML 任务编排
~~~~~~~~~~~~~
- 支持 ``variables`` 变量定义,``${VAR}`` 在 cmd/env/cwd 中替换
- 列表变量展开为 cmd 数组多个元素
- ``cli:`` 段支持 subcommands/positional/options 三级 schema
- 支持 ``type: path`` 自动转为 ``pathlib.Path``
文档
~~~~
- 搭建 Sphinx 文档,发布到 ReadTheDocs
- 更新 READMECLI 示例改为 ``pf`` 统一入口,模块结构表补全
0.3.x
-----
- 新增 YAML 任务编排(GitHub Actions 风格 schema
- 新增 ``fn:`` 函数引用与 ``register_fn`` / ``get_fn`` 注册中心
- 新增 ``compose`` / ``GraphComposer`` 多图组合
- 新增 ``task_template`` 任务模板工厂
- 新增 ``concurrency_key`` + ``concurrency_limits`` 并发限制
- 新增 ``JSONBackend`` 断点续跑与 ``batch()`` 批量落盘
- 新增 ``cache_key`` 缓存键函数
- 新增条件执行(``IS_WINDOWS`` / ``HAS_INSTALLED`` / ``ENV_VAR_EQUALS`` 等)
- 四种执行策略:``sequential`` / ``thread`` / ``async`` / ``dependency``
- 参数名即依赖的上下文注入机制
+65
View File
@@ -0,0 +1,65 @@
"""Sphinx 配置.
ReadTheDocs 构建 PyFlowX 文档站。
"""
from __future__ import annotations
import sys
from pathlib import Path
# 确保 src/ 在 sys.path 中, autodoc 能导入 pyflowx
sys.path.insert(0, str(Path(__file__).resolve().parent.parent / "src"))
from pyflowx import __version__
# -- 项目信息 --------------------------------------------------------------
project = "PyFlowX"
author = "pyflowx"
copyright = "2024, pyflowx"
release = __version__
version = __version__
# -- Sphinx 配置 -----------------------------------------------------------
extensions = [
"sphinx.ext.autodoc",
"sphinx.ext.napoleon",
"sphinx.ext.viewcode",
"sphinx.ext.intersphinx",
"myst_parser",
]
# -- 主题 ------------------------------------------------------------------
html_theme = "sphinx_rtd_theme"
html_static_path = ["_static"]
# -- autodoc 配置 ----------------------------------------------------------
autodoc_default_options = {
"members": True,
"undoc-members": True,
"show-inheritance": True,
"member-order": "bysource",
}
autodoc_type_hints = "description"
autodoc_typehints_format = "short"
# -- napoleon 配置 (Google/NumPy docstring 兼容) --------------------------
napoleon_google_docstring = True
napoleon_numpy_docstring = True
napoleon_include_init_with_doc = False
napoleon_include_private_with_doc = False
napoleon_include_special_with_doc = True
# -- intersphinx -----------------------------------------------------------
intersphinx_mapping = {
"python": ("https://docs.python.org/3", None),
}
# -- 全局选项 ---------------------------------------------------------------
language = "zh_CN"
master_doc = "index"
exclude_patterns = ["_build", "Thumbs.db", ".DS_Store"]
source_suffix = {
".rst": "restructuredtext",
".md": "markdown",
}
+158
View File
@@ -0,0 +1,158 @@
pf 统一 CLI 入口
================
所有工具通过 ``pf <tool> [command] [options]`` 调用。工具定义在 ``cli/configs/`` 目录下的 YAML 文件中。
基本用法
--------
.. code-block:: bash
pf # 列出所有可用工具
pf filedate # 查看 filedate 工具帮助
pf filedate add a.txt # 调用 filedate 的 add 子命令
pf gitt c # 调用 gittool 的 c 子命令
pf pymake b # 调用 pymake 的 b 别名
全局选项
--------
所有 YAML 工具支持以下全局选项:
.. list-table::
:header-rows: 1
:widths: 25 75
* - 选项
- 说明
* - ``--dry-run``
- 仅打印执行计划,不执行
* - ``--quiet`` / ``-q``
- 减少输出,不显示执行过程
* - ``--strategy``
- 执行策略(``sequential`` / ``thread`` / ``async`` / ``dependency``
* - ``--list``
- 列出所有任务名后退出
默认 ``verbose`` 开启,显示执行过程(任务开始/命令/返回码/任务成功)。``--quiet`` 关闭。
YAML 配置工具
--------------
.. list-table::
:header-rows: 1
:widths: 20 15 65
* - 工具
- 别名
- 说明
* - ``filedate``
- ``fd``
- 文件日期处理
* - ``filelevel``
- ``fl``
- 文件等级重命名
* - ``folderback``
- ``fb``
- 文件夹备份
* - ``folderzip``
- ``fz``
- 文件夹压缩
* - ``gittool``
- ``gitt``
- Git 执行工具
* - ``lscalc``
- ``ls``
- LS-DYNA 计算工具
* - ``packtool``
- ``pack``
- Python 打包工具
* - ``pdftool``
- ``pdf``
- PDF 文件工具集
* - ``piptool``
- ``pip``
- pip 包管理工具
* - ``screenshot``
- ``ss``
- 截图工具
* - ``sshcopyid``
- ``ssh``
- SSH 密钥部署工具
* - ``autofmt``
- ``af``
- 自动格式化工具
* - ``bumpversion``
- ``bump``
- 版本号自动管理工具
传统工具
--------
.. list-table::
:header-rows: 1
:widths: 20 80
* - 工具
- 说明
* - ``pymake``
- 构建工具(替代 Makefile),如 ``pf pymake b`` 构建
* - ``yamlrun``
- YAML pipeline 执行器,``pf yamlrun pipeline.yaml``
* - ``profiler``
- 性能分析
* - ``emlman``
- 邮件管理
* - ``reseticon``
- 重置图标缓存
自定义工具
----------
``cli/configs/`` 目录新建 ``<tool>.yaml`` 即可被 ``pf`` 自动发现:
.. code-block:: yaml
# cli/configs/mytool.yaml
strategy: sequential
variables:
MSG: "hello"
cli:
description: "我的工具"
usage: "pf mytool [command]"
subcommands:
greet:
help: "打招呼"
jobs:
greet:
cmd: ["echo", "${MSG}"]
执行::
pf mytool greet
CliRunner(编程式)
-------------------
``CliRunner`` 把多个 Graph 映射为命令行子命令,适合构建项目专属构建工具:
.. code-block:: python
runner = px.CliRunner(
strategy="sequential",
description="My Build Tool",
graphs={
"clean": clean_graph,
"build": build_graph,
"test": test_graph,
},
)
runner.run_cli() # 解析 sys.argv 并执行
命令行::
pf pymake clean
pf pymake build --strategy thread
pf pymake test --dry-run
pf pymake --list
pf pymake --quiet
+93
View File
@@ -0,0 +1,93 @@
执行策略与 run()
=================
``run()`` 是执行入口,支持四种策略:
.. code-block:: python
report = px.run(
graph,
strategy="async", # sequential | thread | async | dependency
max_workers=8, # thread 策略的线程池大小
concurrency_limits={"db": 2}, # 按 concurrency_key 限流
dry_run=False, # True = 仅打印计划
verbose=True, # True = 打印执行过程
on_event=callback, # 状态转换回调
state=px.JSONBackend("state.json"), # 断点续跑后端
continue_on_error=False, # True = 单任务失败不中断整体
)
策略对比
--------
.. list-table::
:header-rows: 1
:widths: 18 18 30 16 18
* - 策略
- 并发模型
- 适用场景
- 同步任务
- 异步任务
* - ``sequential``
- 串行
- 调试、CPU 密集
- 直接调用
- 事件循环
* - ``thread``
- 线程池
- I/O 密集同步
- 线程池
- 不支持
* - ``async``
- 事件循环
- I/O 密集异步
- 卸载到线程池
- 事件循环
* - ``dependency``
- 依赖驱动
- 最大化并行度
- 卸载到线程池
- 事件循环
所有策略都遵循 ``RetryPolicy````timeout``、上下文注入、状态后端、``concurrency_limits``
并发出 ``TaskEvent``RUNNING/SUCCESS/FAILED/SKIPPED)。``dependency`` 策略无层屏障:
任务在其所有硬依赖完成后立即启动。
上下文注入规则
--------------
按顺序求值:
1. **标注为 ``Context``** 的参数 → 接收完整上游结果映射
2. **名称匹配依赖** 的参数 → 接收该依赖的结果(含软依赖,缺失时注入默认值)
3. **``**kwargs``** 参数 → 接收所有依赖结果(dict)
4. **``TaskSpec.args`` / ``kwargs``** → 为非依赖参数提供静态值
.. code-block:: python
from typing import Any, Dict
def aggregate(ctx: px.Context) -> Dict[str, Any]:
"""ctx 包含所有 depends_on 任务的返回值。"""
return dict(ctx)
def merge(fetch_a: str, fetch_b: str) -> str:
"""fetch_a / fetch_b 自动注入。"""
return fetch_a + fetch_b
断点续跑
--------
.. code-block:: python
from pyflowx import JSONBackend
backend = JSONBackend("state.json", ttl=3600)
report = px.run(graph, strategy="sequential", state=backend)
``run()`` 内部以 ``backend.batch()`` 包裹整个执行:所有 ``save`` 延迟到运行结束时统一落盘一次。
缓存键:默认存储键为任务名。配置 ``cache_key`` 函数后,键为 ``"name:cache_key_value"``
完整 API 说明详见 :doc:`/api`
+50
View File
@@ -0,0 +1,50 @@
Graph —— DAG 构建
=================
``Graph`` 管理任务集合,提供建构建、校验、分层、可视化能力。
构建方式
--------
.. code-block:: python
# 图级默认值:TaskSpec 字段为 None 时回退
defaults = px.GraphDefaults(retry=px.RetryPolicy(max_attempts=2), timeout=60.0)
graph = px.Graph.from_specs([...], defaults=defaults) # 整批校验(推荐)
# 或增量构建
graph = px.Graph(defaults=defaults)
graph.add(px.TaskSpec("a", fn_a))
graph.add(px.TaskSpec("b", fn_b, ("a",)))
常用方法
--------
.. code-block:: python
graph.validate() # 显式校验(环检测)
graph.layers() # 拓扑分层(Kahn 算法)
graph.to_mermaid() # Mermaid 可视化
graph.describe() # 人类可读摘要
graph.subgraph(("api",)) # 按标签切片
graph.subgraph_by_names(("a", "b")) # 按名称切片
graph.map("fetch", [1, 2, 3], lambda i: TaskSpec(f"fetch_{i}", ...)) # 批量 fan-out
图组合
------
``compose`` / ``GraphComposer`` 把带字符串引用的多个图展开为纯 ``Graph``
.. code-block:: python
graphs = {
"build": px.Graph.from_specs([px.TaskSpec("b", cmd=["echo", "b"])]),
"all": px.Graph.from_specs(["build", px.TaskSpec("t", cmd=["echo", "t"])]),
}
resolved = px.compose(graphs) # "all" 图中的 "build" 引用被展开
引用格式:``"command_name"``(整个图)或 ``"command_name.task_name"``(特定任务)。
``CliRunner`` 内部自动调用 ``compose``
完整方法说明详见 :doc:`/api`
+89
View File
@@ -0,0 +1,89 @@
TaskSpec —— 任务描述
=====================
``TaskSpec`` 是不可变的任务描述符(``Generic[T]``,返回类型一路传到 ``RunReport``),是唯一需要配置的东西。
主要参数说明:
.. code-block:: python
px.TaskSpec(
name="fetch_user", # 唯一标识
fn=fetch_user, # 同步或异步函数
cmd=["curl", "..."], # 或: 执行命令(覆盖 fn)
depends_on=("auth",), # 硬依赖(参与拓扑分层)
soft_depends_on=("cache",), # 软依赖(仅注入,不参与分层)
args=(uid,), # 静态位置参数(追加在注入参数后)
kwargs={"timeout": 30}, # 静态关键字参数
retry=px.RetryPolicy(max_attempts=3, delay=1.0, backoff=2.0),
timeout=30.0, # 超时秒数(None = 不限制)
tags=("api", "user"), # 自由标签,用于子图过滤
conditions=(is_prod,), # 条件函数列表(全部为 True 才执行)
priority=10, # 同层内优先级(高优先执行,默认 0)
concurrency_key="db", # 并发分组键(配合 concurrency_limits 限流)
cache_key=lambda ctx: str(ctx.get("uid")), # 缓存键函数
hooks=px.TaskHooks(pre_run=..., post_run=..., on_failure=...),
cwd=Path("/tmp"), # 命令工作目录(仅 cmd 模式)
env={"DEBUG": "1"}, # 环境变量覆盖
verbose=True, # 打印命令输出(仅 cmd 模式)
skip_if_missing=True, # 命令不存在时自动跳过(仅 list[str] cmd
allow_upstream_skip=False, # 上游 SKIPPED/FAILED 时是否仍执行
continue_on_error=False, # 本任务失败是否不中断整体
)
两种任务形态
------------
- **函数任务**``fn``):普通 Python 函数,参数名驱动自动注入
- **命令任务**``cmd``):执行外部命令,支持 ``list[str]````str``shell)、``Callable`` 三种形态
``skip_if_missing=True`` 时,``list[str]`` 类型的 ``cmd`` 会通过 ``shutil.which`` 检查命令是否存在,不存在则跳过任务(标记为 ``SKIPPED``)而非失败。
重试策略
--------
``RetryPolicy`` 配置重试次数、延迟、退避:
.. code-block:: python
retry = px.RetryPolicy(
max_attempts=3, # 最大尝试次数
delay=1.0, # 初始延迟秒数
backoff=2.0, # 退避倍数
jitter=0.1, # 随机抖动(避免惊群)
retry_on=(ConnectionError,), # 仅对这些异常重试
)
任务钩子
--------
``TaskHooks`` 在任务生命周期触发(异常仅记录,不影响任务状态):
.. code-block:: python
hooks = px.TaskHooks(
pre_run=lambda spec: print(f"start {spec.name}"),
post_run=lambda spec, value: print(f"done {spec.name}"),
on_failure=lambda spec, exc: alert(spec.name, exc),
)
px.TaskSpec("task", fn=work, hooks=hooks)
任务模板
--------
``task_template`` 工厂批量生成相似 TaskSpec
.. code-block:: python
fetch = px.task_template(
fn=fetch_url,
retry=px.RetryPolicy(max_attempts=5),
timeout=30.0,
tags=("api",),
)
graph = px.Graph.from_specs([
fetch("users", url="https://api.example.com/users"),
fetch("posts", url="https://api.example.com/posts"),
])
完整字段说明详见 :doc:`/api`
+164
View File
@@ -0,0 +1,164 @@
YAML 任务编排
=============
PyFlowX 支持 GitHub Actions 风格的声明式 YAML 任务编排,从 YAML 文件直接加载任务图。
编程式 API
----------
.. code-block:: python
import pyflowx as px
# 从 YAML 文件加载任务图
graph = px.Graph.from_yaml("pipeline.yaml")
report = px.run(graph, strategy="thread")
# 或用函数式 API
graph = px.load_yaml("pipeline.yaml")
# 从字符串解析
graph = px.parse_yaml_string("""
jobs:
hello:
cmd: ["echo", "hello"]
""")
YAML Schema
-----------
.. code-block:: yaml
strategy: thread # 图级默认策略
defaults: # 图级默认值
retry: {max_attempts: 3}
verbose: true
env: {CI: "true"}
variables: # 变量定义 (可在 cmd/env 中 ${VAR} 引用)
OUTPUT: "dist"
jobs:
setup:
cmd: ["git", "clone", "..."]
runs-on: linux
build:
needs: [setup] # 依赖列表
cmd: ["python", "-m", "build"]
timeout: 300
retry: {max_attempts: 2, delay: 1.0}
test:
needs: [build]
cmd: ["python${{ matrix.version }}", "-m", "pytest"]
strategy:
matrix: # 笛卡尔积展开为 6 个任务
version: ["3.8", "3.9", "3.10"]
os: ["linux", "macos"]
if: "env.CI" # 条件: 环境变量存在
lint:
needs: [build]
cmd: ["ruff", "check"]
if: "env.CI == 'true'"
deploy:
needs: [test, lint] # 矩阵依赖自动展开
cmd: ["twine", "upload"]
if: "env.DEPLOY_TOKEN != ''"
allow-upstream-skip: true
concurrency-key: deploy_lock
字段映射
--------
.. list-table::
:header-rows: 1
:widths: 30 30 40
* - YAML 字段
- TaskSpec 字段
- 说明
* - ``jobs.<id>``
- ``name``
- job ID 作为任务名
* - ``cmd`` / ``run``
- ``cmd``
- ``cmd`` 为列表形式,``run`` 为 shell 字符串
* - ``needs``
- ``depends_on``
- 依赖列表(矩阵任务自动展开)
* - ``if``
- ``conditions``
- ``success()`` / ``always()`` / ``env.VAR`` / ``env.VAR == 'x'``
* - ``strategy.matrix``
- 矩阵扇出
- 笛卡尔积展开为多个任务
* - ``${{ matrix.key }}``
- 占位符
- 在 cmd/run/cwd/env 中替换
* - ``timeout``
- ``timeout``
- 超时秒数
* - ``retry``
- ``retry``
- ``{max_attempts, delay, backoff, jitter}``
* - ``cwd``
- ``cwd``
- 工作目录
* - ``env``
- ``env``
- 环境变量
* - ``verbose``
- ``verbose``
- 详细输出
* - ``continue-on-error``
- ``continue_on_error``
- 失败不中止整图
* - ``skip-if-missing``
- ``skip_if_missing``
- 命令不存在时跳过
* - ``allow-upstream-skip``
- ``allow_upstream_skip``
- 上游跳过时仍执行
* - ``priority``
- ``priority``
- 同层优先级
* - ``concurrency-key``
- ``concurrency_key``
- 并发限制键
* - ``tags``
- ``tags``
- 自由标签
* - ``runs-on``
- ``tags``(追加)
- 运行环境标签
CLI 配置段(``cli:``
----------------------
工具 YAML 还可定义 ``cli:`` 段,声明命令行参数 schema,由 ``pf`` 自动解析:
.. code-block:: yaml
cli:
description: "FileDate - 文件日期处理工具"
usage: "pf filedate <command> [files...]"
subcommands:
add:
help: "添加日期前缀"
positional:
- name: FILES
nargs: "+"
type: path
help: "文件路径"
options:
- name: CLEAR
flag: "--clear"
action: store_true
help: "清除已有日期前缀"
支持的 ``type````str`` / ``int`` / ``float`` / ``path``
完整 API 说明详见 :doc:`/api`
+56
View File
@@ -0,0 +1,56 @@
PyFlowX 文档
============
PyFlowX 是一个轻量、类型安全的 DAG 任务调度器:**参数名就是依赖声明**
无需装饰器、无需样板包装器,写一个普通函数,框架按参数名自动注入上游结果。
特性
----
- **零样板** —— 参数名即依赖,框架自动注入上游结果
- **四种执行策略** —— sequential(串行)、thread(线程池)、async(事件循环)、dependency(依赖驱动,最大化并行)
- **类型安全** —— ``TaskSpec[T]`` 把返回类型一路传到 ``RunReport``
- **DAG 校验** —— 构建时即时校验重名、缺失依赖、环
- **自动分层** —— Kahn 算法分组,同层任务可并行
- **重试与超时** —— 每个任务独立配置 ``RetryPolicy````timeout``
- **并发限制** —— ``concurrency_key`` + ``concurrency_limits`` 按组限流
- **断点续跑** —— ``MemoryBackend`` / ``JSONBackend``,成功结果可缓存复用
- **命令任务** —— ``cmd`` 参数直接执行外部命令
- **条件执行** —— ``conditions`` 按平台、环境变量等条件跳过任务
- **YAML 任务编排** —— GitHub Actions 风格声明式任务图
- **pf 统一 CLI** —— ``pf <tool> [command]`` 调用所有工具
- **最小依赖** —— 仅依赖标准库 + PyYAML
文档导航
--------
.. toctree::
:maxdepth: 2
:caption: 入门
installation
quickstart
.. toctree::
:maxdepth: 2
:caption: 用户指南
guide/task
guide/graph
guide/execution
guide/yaml
guide/cli
.. toctree::
:maxdepth: 2
:caption: 参考
api
changelog
索引
----
* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`
+51
View File
@@ -0,0 +1,51 @@
安装
====
PyFlowX 支持 Python 3.8+,仅依赖标准库与 PyYAML(3.8 额外需要 ``graphlib_backport````typing-extensions``)。
pip 安装
--------
.. code-block:: bash
pip install pyflowx
uv 安装
-------
推荐使用 `uv <https://docs.astral.sh/uv/>`_
.. code-block:: bash
uv add pyflowx
可选依赖
--------
``office`` —— PDF/图片处理(pdftool、screenshot 等工具需要):
.. code-block:: bash
pip install pyflowx[office]
``dev`` —— 开发工具链(ruff、pyrefly、pytest、tox 等):
.. code-block:: bash
pip install pyflowx[dev]
验证安装
--------
.. code-block:: bash
pf --version
输出示例::
PyFlowX 0.4.5
下一步
------
前往 :doc:`quickstart` 开始使用。
+87
View File
@@ -0,0 +1,87 @@
快速上手
========
核心思想:**参数名即依赖**。写一个普通函数,参数名匹配上游任务名,框架自动注入结果。
最小示例
--------
.. code-block:: python
import pyflowx as px
def extract() -> list[int]:
return [1, 2, 3]
# 参数名 extract 自动匹配上游任务名 → 自动注入
def double(extract: list[int]) -> list[int]:
return [x * 2 for x in extract]
graph = px.Graph.from_specs([
px.TaskSpec("extract", extract),
px.TaskSpec("double", double, ("extract",)),
])
report = px.run(graph, strategy="sequential")
print(report["double"]) # [2, 4, 6]
三种任务形态
------------
1. **函数任务**``fn``):普通 Python 函数,参数名驱动自动注入
2. **命令任务**``cmd``):执行外部命令,支持 ``list[str]`` / ``str``shell/ ``Callable``
3. **YAML 声明式**:从 YAML 文件加载任务图
.. code-block:: python
graph = px.Graph.from_specs([
px.TaskSpec("list", cmd=["ls", "-la"]),
px.TaskSpec("greet", fn=lambda: "hello"),
])
执行策略
--------
PyFlowX 提供四种执行策略:
.. list-table::
:header-rows: 1
:widths: 20 20 60
* - 策略
- 并发模型
- 适用场景
* - ``sequential``
- 串行
- 调试、CPU 密集
* - ``thread``
- 线程池
- I/O 密集同步
* - ``async``
- 事件循环
- I/O 密集异步
* - ``dependency``
- 依赖驱动
- 最大化并行度(默认推荐)
.. code-block:: python
report = px.run(graph, strategy="dependency")
结果访问
--------
.. code-block:: python
report["task_name"] # 任务返回值
report.result_of("task_name") # 完整 TaskResult
report.success # 整体是否成功
report.summary() # 统计字典
report.failed_tasks() # 失败任务名列表
下一步
------
- :doc:`guide/task` —— TaskSpec 详细配置
- :doc:`guide/yaml` —— YAML 声明式任务编排
- :doc:`guide/cli` —— ``pf`` 统一 CLI 入口
+13 -23
View File
@@ -13,7 +13,8 @@ classifiers = [
]
dependencies = [
"graphlib_backport >= 1.0.0; python_version < '3.9'",
"typing-extensions>=4.13.2; python_version < '3.10'",
"pyyaml>=6.0.1",
"typing-extensions>=4.13.2; python_version < '3.13'",
]
description = "Lightweight, type-safe DAG task scheduler with multi-strategy execution."
keywords = ["async", "dag", "scheduler", "task", "workflow"]
@@ -21,29 +22,16 @@ license = { text = "MIT" }
name = "pyflowx"
readme = "README.md"
requires-python = ">=3.8"
version = "0.3.2"
version = "0.4.6"
[project.scripts]
autofmt = "pyflowx.cli.autofmt:main"
bumpversion = "pyflowx.cli.bumpversion:main"
dockercmd = "pyflowx.cli.dev.dockercmd:main"
emlman = "pyflowx.cli.emlmanager:main"
filedate = "pyflowx.cli.filedate:main"
filelvl = "pyflowx.cli.filelevel:main"
foldback = "pyflowx.cli.folderback:main"
foldzip = "pyflowx.cli.folderzip:main"
gitt = "pyflowx.cli.gittool:main"
lscalc = "pyflowx.cli.lscalc:main"
msdown = "pyflowx.cli.llm.msdownload:main"
packtool = "pyflowx.cli.packtool:main"
pdftool = "pyflowx.cli.pdftool:main"
piptool = "pyflowx.cli.piptool:main"
pxp = "pyflowx.cli.profiler:main"
pymake = "pyflowx.cli.pymake:main"
reseticon = "pyflowx.cli.reseticoncache:main"
scrcap = "pyflowx.cli.screenshot:main"
sglang = "pyflowx.cli.llm.sglang:main"
sshcopy = "pyflowx.cli.sshcopyid:main"
dockercmd = "pyflowx.cli.dev.dockercmd:main"
emlman = "pyflowx.cli.emlmanager:main"
msdown = "pyflowx.cli.llm.msdownload:main"
pf = "pyflowx.cli.pf:main"
pxp = "pyflowx.cli.profiler:main"
sglang = "pyflowx.cli.llm.sglang:main"
yamlrun = "pyflowx.cli.yamlrun:main"
# dev
envdev = "pyflowx.cli.dev.envdev:main"
# system
@@ -66,7 +54,9 @@ dev = [
"ruff>=0.8.0",
"tox-uv>=1.13.1",
"tox>=4.25.0",
"types-PyYAML>=6.0.12",
]
docs = ["myst-parser>=3.0", "sphinx-rtd-theme>=2.0", "sphinx>=7.0"]
office = [
"pillow>=10.4.0",
"pymupdf>=1.24.11",
@@ -95,7 +85,7 @@ packages = ["src/pyflowx"]
pyflowx = { workspace = true }
[dependency-groups]
dev = ["pyflowx[dev,office]"]
dev = ["pyflowx[dev,docs,office]"]
[tool.coverage.run]
branch = true
+13 -1
View File
@@ -83,6 +83,7 @@ from .errors import (
from .executors import Strategy, run
from .graph import Graph, GraphDefaults
from .profiling import ProfileReport, TaskProfile
from .registry import FnRegistry, get_fn, has_fn, register_fn
from .report import RunReport
from .runner import CliExitCode, CliRunner
from .storage import JSONBackend, MemoryBackend, StateBackend
@@ -99,8 +100,9 @@ from .task import (
task,
task_template,
)
from .yaml_loader import YamlLoadError, build_cli_parser, load_yaml, parse_yaml_string, run_cli, run_yaml
__version__ = "0.4.2"
__version__ = "0.4.6"
__all__ = [
"IS_LINUX",
@@ -116,6 +118,7 @@ __all__ = [
"Context",
"CycleError",
"DuplicateTaskError",
"FnRegistry",
"Graph",
"GraphComposer",
"GraphDefaults",
@@ -139,12 +142,21 @@ __all__ = [
"TaskSpec",
"TaskStatus",
"TaskTimeoutError",
"YamlLoadError",
"build_call_args",
"build_cli_parser",
"cmd",
"compose",
"describe_injection",
"get_fn",
"has_fn",
"load_yaml",
"parse_yaml_string",
"register_fn",
"run",
"run_cli",
"run_command",
"run_yaml",
"task",
"task_template",
]
-282
View File
@@ -1,282 +0,0 @@
"""自动格式化工具模块.
提供 Python 代码自动格式化的常用功能封装,
支持 docstring 自动生成、pyproject.toml 配置同步等功能.
"""
from __future__ import annotations
import argparse
import ast
import subprocess
from pathlib import Path
import pyflowx as px
try:
import tomllib # noqa: F401
HAS_TOMLLIB = True
except ImportError:
HAS_TOMLLIB = False
# ============================================================================
# 配置
# ============================================================================
IGNORE_PATTERNS = [
"__pycache__",
"*.pyc",
"*.pyo",
".git",
".venv",
".idea",
".vscode",
"*.egg-info",
"dist",
"build",
".pytest_cache",
".tox",
".mypy_cache",
]
# ============================================================================
# 辅助函数
# ============================================================================
def format_with_ruff(target: Path, fix: bool = True) -> None:
"""使用 ruff 格式化代码.
Parameters
----------
target : Path
目标路径
fix : bool
是否自动修复
"""
cmd = ["ruff", "format", str(target)]
if fix:
cmd.append("--fix")
subprocess.run(cmd, check=True)
print(f"ruff format 完成: {target}")
def lint_with_ruff(target: Path, fix: bool = True) -> None:
"""使用 ruff 检查代码.
Parameters
----------
target : Path
目标路径
fix : bool
是否自动修复
"""
cmd = ["ruff", "check", str(target)]
if fix:
cmd.extend(["--fix", "--unsafe-fixes"])
subprocess.run(cmd, check=True)
print(f"ruff check 完成: {target}")
def add_docstring(file_path: Path, docstring: str) -> bool:
"""为文件添加 docstring.
Parameters
----------
file_path : Path
文件路径
docstring : str
docstring 内容
Returns
-------
bool
是否成功添加
"""
try:
content = file_path.read_text(encoding="utf-8")
tree = ast.parse(content)
# 检查是否已有 docstring
first_node = tree.body[0] if tree.body else None
if first_node and isinstance(first_node, ast.Expr) and isinstance(first_node.value, ast.Constant):
return False
# 添加 docstring
lines = content.splitlines()
doc_lines = docstring.splitlines()
doc_lines.append("")
new_content = "\n".join(doc_lines + lines)
file_path.write_text(new_content, encoding="utf-8")
print(f"添加 docstring: {file_path}")
return True
except (OSError, UnicodeDecodeError, SyntaxError) as e:
print(f"处理失败: {file_path} - {e}")
return False
def generate_module_docstring(file_path: Path) -> str:
"""生成模块 docstring.
Parameters
----------
file_path : Path
文件路径
Returns
-------
str
生成的 docstring
"""
stem = file_path.stem
parent = file_path.parent.name
# 关键词匹配
keywords = {
"cli": f"Command-line interface for {parent}",
"gui": f"Graphical user interface for {parent}",
"core": f"Core functionality for {parent}",
"util": f"Utility functions for {parent}",
"model": f"Data models for {parent}",
"test": f"Tests for {parent}",
}
for key, desc in keywords.items():
if key in stem.lower():
return f'"""{desc}."""'
return f'"""{stem.replace("_", " ").title()} module."""'
def auto_add_docstrings(root_dir: Path) -> int:
"""自动为所有 Python 文件添加 docstring.
Parameters
----------
root_dir : Path
根目录
Returns
-------
int
添加的 docstring 数量
"""
count = 0
for py_file in root_dir.rglob("*.py"):
# 跳过忽略的文件
if any(pattern in str(py_file) for pattern in IGNORE_PATTERNS):
continue
docstring = generate_module_docstring(py_file)
if add_docstring(py_file, docstring):
count += 1
print(f"共添加 {count} 个 docstring")
return count
def sync_pyproject_config(root_dir: Path) -> None:
"""同步 pyproject.toml 配置到子项目.
Parameters
----------
root_dir : Path
根目录
"""
main_toml = root_dir / "pyproject.toml"
if not main_toml.exists():
print(f"主项目配置文件不存在: {main_toml}")
return
# 查找所有子项目的 pyproject.toml
sub_tomls = [p for p in root_dir.rglob("pyproject.toml") if p != main_toml and ".venv" not in str(p)]
if not sub_tomls:
print("没有找到子项目的 pyproject.toml")
return
print(f"找到 {len(sub_tomls)} 个子项目配置文件")
# 对每个子项目调用 ruff format
for sub_toml in sub_tomls:
subprocess.run(["ruff", "format", str(sub_toml)], check=False)
print("配置同步完成")
def format_all(root_dir: Path) -> None:
"""格式化所有 Python 文件.
Parameters
----------
root_dir : Path
根目录
"""
# 使用 ruff format
subprocess.run(["ruff", "format", str(root_dir)], check=True)
# 使用 ruff check
subprocess.run(["ruff", "check", "--fix", "--unsafe-fixes", str(root_dir)], check=True)
print(f"格式化完成: {root_dir}")
# ============================================================================
# CLI Runner
# ============================================================================
def main() -> None:
"""自动格式化工具主函数."""
parser = argparse.ArgumentParser(
description="AutoFmt - 自动格式化工具",
usage="autofmt <command> [options]",
)
subparsers = parser.add_subparsers(dest="command", help="可用命令")
# ruff format 命令
format_parser = subparsers.add_parser("fmt", help="使用 ruff 格式化代码")
format_parser.add_argument("--target", type=str, default=".", help="目标路径")
# ruff check 命令
lint_parser = subparsers.add_parser("lint", help="使用 ruff 检查代码")
lint_parser.add_argument("--target", type=str, default=".", help="目标路径")
lint_parser.add_argument("--fix", action="store_true", help="自动修复")
# 自动添加 docstring 命令
doc_parser = subparsers.add_parser("doc", help="自动添加 docstring")
doc_parser.add_argument("--root-dir", type=str, default=".", help="根目录")
# 同步配置命令
sync_parser = subparsers.add_parser("sync", help="同步 pyproject.toml 配置")
sync_parser.add_argument("--root-dir", type=str, default=".", help="根目录")
args = parser.parse_args()
if args.command == "fmt":
graph = px.Graph.from_specs([px.TaskSpec("ruff_format", cmd=["ruff", "format", args.target], verbose=True)])
elif args.command == "lint":
cmd = ["ruff", "check", args.target]
if args.fix:
cmd.extend(["--fix", "--unsafe-fixes"])
graph = px.Graph.from_specs([px.TaskSpec("ruff_check", cmd=cmd, verbose=True)])
elif args.command == "doc":
graph = px.Graph.from_specs([
px.TaskSpec("auto_docstring", fn=auto_add_docstrings, args=(Path(args.root_dir),), verbose=True)
])
elif args.command == "sync":
graph = px.Graph.from_specs([
px.TaskSpec("sync_config", fn=sync_pyproject_config, args=(Path(args.root_dir),), verbose=True)
])
else:
parser.print_help()
return
px.run(graph, strategy="thread")
-263
View File
@@ -1,263 +0,0 @@
"""版本号自动管理工具.
使用 TaskSpec 模式实现, 支持语义化版本管理和多文件格式的版本号更新.
"""
from __future__ import annotations
import argparse
import re
from pathlib import Path
from typing import Literal, get_args
import pyflowx as px
BumpVersionType = Literal["patch", "minor", "major"]
# 针对不同文件类型的版本号匹配模式
# pyproject.toml: version = "X.Y.Z" 或 version = 'X.Y.Z'
_PYPROJECT_VERSION_PATTERN = re.compile(
r'(?:^|\n)\s*version\s*=\s*["\']'
r"(?P<major>0|[1-9]\d*)\.(?P<minor>0|[1-9]\d*)\.(?P<patch>0|[1-9]\d*)"
r"(?:-(?P<prerelease>(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)(?:\.(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*))?"
r"(?:\+(?P<buildmetadata>[0-9a-zA-Z-]+(?:\.[0-9a-zA-Z-]+)*))?"
r'["\']',
re.MULTILINE,
)
# __init__.py: __version__ = "X.Y.Z" 或 __version__ = 'X.Y.Z'
_INIT_VERSION_PATTERN = re.compile(
r'(?:^|\n)\s*__version__\s*=\s*["\']'
r"(?P<major>0|[1-9]\d*)\.(?P<minor>0|[1-9]\d*)\.(?P<patch>0|[1-9]\d*)"
r"(?:-(?P<prerelease>(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)(?:\.(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*))?"
r"(?:\+(?P<buildmetadata>[0-9a-zA-Z-]+(?:\.[0-9a-zA-Z-]+)*))?"
r'["\']',
re.MULTILINE,
)
def _get_pattern_for_file(file_name: str) -> re.Pattern[str] | None:
"""根据文件类型获取对应的正则表达式.
Parameters
----------
file_name : str
文件名
Returns
-------
re.Pattern[str] | None
对应的正则表达式,如果无法确定则返回 None
"""
if file_name == "pyproject.toml":
return _PYPROJECT_VERSION_PATTERN
if file_name == "__init__.py":
return _INIT_VERSION_PATTERN
return None
def _calculate_new_version(major: int, minor: int, patch: int, part: BumpVersionType) -> str:
"""计算新版本号.
Parameters
----------
major : int
当前主版本号
minor : int
当前次版本号
patch : int
当前补丁版本号
part : BumpVersionType
要更新的部分
Returns
-------
str
新版本号
"""
if part == "major":
return f"{major + 1}.0.0"
if part == "minor":
return f"{major}.{minor + 1}.0"
return f"{major}.{minor}.{patch + 1}"
def _build_replacement_string(original_match: str, new_version: str, file_name: str) -> str:
"""构建替换字符串,保留原始格式.
Parameters
----------
original_match : str
原始匹配的字符串
new_version : str
新版本号
file_name : str
文件名
Returns
-------
str
替换字符串
"""
quote_char = '"' if '"' in original_match else "'"
if file_name == "pyproject.toml":
prefix_match = re.match(r'(\s*version\s*=\s*)["\']', original_match)
prefix = prefix_match.group(1) if prefix_match else "version = "
return f"{prefix}{quote_char}{new_version}{quote_char}"
if file_name == "__init__.py":
prefix_match = re.match(r'(\s*__version__\s*=\s*)["\']', original_match)
prefix = prefix_match.group(1) if prefix_match else "__version__ = "
return f"{prefix}{quote_char}{new_version}{quote_char}"
return new_version
def bump_file_version(file_path: Path, part: BumpVersionType = "patch") -> str | None:
"""更新文件中的版本号.
Parameters
----------
file_path : Path
要更新的文件路径
part : BumpVersionType
版本部分: patch, minor, major
Returns
-------
str | None
更新后的新版本号,如果文件中未找到版本号则返回 None
"""
try:
content = file_path.read_text(encoding="utf-8")
except Exception as e:
print(f"读取文件 {file_path} 时出错: {e}")
raise
# 获取文件对应的正则表达式
pattern = _get_pattern_for_file(file_path.name)
# 对于未知文件类型,尝试两种模式
if pattern:
match = pattern.search(content)
else:
match = _PYPROJECT_VERSION_PATTERN.search(content) or _INIT_VERSION_PATTERN.search(content)
if not match:
print(f"文件 {file_path} 中未找到版本号模式")
return None
# 提取当前版本号
major = int(match.group("major"))
minor = int(match.group("minor"))
patch = int(match.group("patch"))
# 计算新版本号
new_version = _calculate_new_version(major, minor, patch, part)
# 构建替换字符串
original_match = match.group(0)
replacement = _build_replacement_string(original_match, new_version, file_path.name)
# 更新文件内容
content = content.replace(original_match, replacement)
try:
file_path.write_text(content, encoding="utf-8")
except Exception as e:
print(f"更新文件 {file_path} 版本号时出错: {e}")
raise
return new_version
def main() -> None:
"""版本号管理工具主函数."""
parser = argparse.ArgumentParser(description="BumpVersion - 版本号自动管理工具")
parser.add_argument(
"part",
type=str,
nargs="?",
default="patch",
choices=get_args(BumpVersionType),
help=f"版本部分: {get_args(BumpVersionType)}",
)
parser.add_argument(
"--no-tag",
action="store_true",
help="提交后不创建 git tag",
)
args = parser.parse_args()
part = args.part
# 搜索文件,排除常见的虚拟环境和缓存目录
ignore_dirs = {".venv", "venv", ".git", "__pycache__", ".tox", "node_modules", "build", "dist", ".eggs"}
all_files = set()
for pattern in ["__init__.py", "pyproject.toml"]:
for file in Path.cwd().rglob(pattern):
# 检查路径中是否包含需要忽略的目录
if not any(ignore_dir in file.parts for ignore_dir in ignore_dirs):
all_files.add(file)
if not all_files:
print("未找到包含版本号的文件")
return
print(f"找到 {len(all_files)} 个文件需要更新版本号")
for file in sorted(all_files):
print(f" - {file.relative_to(Path.cwd())}")
# 更新所有文件的版本号(使用顺序执行避免竞争条件)
# 使用相对于 cwd 的路径作为任务名,确保唯一性
graph = px.Graph.from_specs([
px.TaskSpec(
f"bump_{file.relative_to(Path.cwd())}".replace("\\", "_").replace("/", "_").replace(".", "_"),
fn=bump_file_version,
args=(file, part),
)
for file in all_files
])
report = px.run(graph, strategy="sequential")
# 收集新版本号(取第一个成功的结果)
new_version = None
for task_name in report:
result = report[task_name]
if result is not None:
new_version = result
break
if not new_version:
print("未能获取新版本号")
return
print(f"版本号已更新为: {new_version}")
# 提交修改并创建标签
tasks = [
px.TaskSpec("git_add", cmd=["git", "add", "."]),
px.TaskSpec(
"git_commit",
cmd=["git", "commit", "-m", f"bump version to {new_version}"],
depends_on=("git_add",),
),
]
if not args.no_tag:
tag_name = f"v{new_version}"
tasks.append(
px.TaskSpec(
"git_tag",
cmd=["git", "tag", "-a", tag_name, "-m", f"Release {tag_name}"],
depends_on=("git_commit",),
)
)
graph = px.Graph.from_specs(tasks)
px.run(graph, strategy="sequential")
if not args.no_tag:
print(f"已创建标签: v{new_version}")
+142 -133
View File
@@ -209,107 +209,112 @@ def main() -> None:
RUST_SCCACHE_DIR.mkdir(parents=True, exist_ok=True)
# 使用 conditions 自动控制任务执行
graph = px.Graph.from_specs([
# 系统镜像配置(仅 Linux 且未配置国内镜像)
px.TaskSpec(
"download_mirror",
cmd=DOWNLOAD_MIRROR_SCRIPT,
conditions=(
BuiltinConditions.IS_LINUX(),
BuiltinConditions.NOT(
BuiltinConditions.OR(
*[
BuiltinConditions.FILE_CONTENT_EXISTS(f, m)
for f in [
"/etc/apt/sources.list",
"/etc/apt/sources.list.d/ubuntu.sources",
]
for m in get_args(PyMirrorType)
],
)
graph = px.Graph.from_specs(
[
# 系统镜像配置(仅 Linux 且未配置国内镜像)
px.TaskSpec(
"download_mirror",
cmd=DOWNLOAD_MIRROR_SCRIPT,
conditions=(
BuiltinConditions.IS_LINUX(),
BuiltinConditions.NOT(
BuiltinConditions.OR(
*[
BuiltinConditions.FILE_CONTENT_EXISTS(f, m)
for f in [
"/etc/apt/sources.list",
"/etc/apt/sources.list.d/ubuntu.sources",
]
for m in get_args(PyMirrorType)
],
)
),
),
verbose=True,
),
verbose=True,
),
px.TaskSpec(
"install_mirror",
cmd=INSTALL_MIRROR_SCRIPT,
depends_on=("download_mirror",),
verbose=True,
),
# 安装 Qt 依赖(仅 Linux
px.TaskSpec(
"install_qt_libs",
cmd=["sudo", "apt", "install", "-y", *QT_LIBS],
conditions=(BuiltinConditions.IS_LINUX(),),
depends_on=("install_mirror",),
allow_upstream_skip=True,
verbose=True,
),
# 安装中文字体(仅 Linux
px.TaskSpec(
"install_fonts",
cmd=["sudo", "apt", "install", "-y", *CHINESE_FONTS],
conditions=(BuiltinConditions.IS_LINUX(),),
depends_on=("install_mirror",),
allow_upstream_skip=True,
verbose=True,
),
# 安装 Docker
px.TaskSpec(
"install_docker",
cmd=["sudo", "apt", "install", "-y", "docker-compose-v2"],
conditions=(BuiltinConditions.IS_LINUX(),),
depends_on=("install_mirror",),
allow_upstream_skip=True,
verbose=True,
),
px.TaskSpec(
"add_docker_group",
cmd=["sudo", "usermod", "-aG", "docker", getpass.getuser()],
conditions=(BuiltinConditions.IS_LINUX(),),
depends_on=("install_docker",),
allow_upstream_skip=True,
verbose=True,
),
px.TaskSpec(
"refresh_docker_group",
cmd=["newgrp", "docker"],
conditions=(BuiltinConditions.IS_LINUX(),),
depends_on=("add_docker_group",),
allow_upstream_skip=True,
verbose=True,
),
# 设置 Python 环境变量
*setenv_group({
"PIP_INDEX_URL": PIP_INDEX_URLS[python_mirror],
"PIP_TRUSTED_HOSTS": PIP_TRUSTED_HOSTS[python_mirror],
"UV_INDEX_URL": UV_INDEX_URLS[python_mirror],
"UV_PYTHON_INSTALL_MIRROR": UV_PYTHON_INSTALL_MIRROR,
"UV_HTTP_TIMEOUT": "600",
"UV_LINK_MODE": "copy",
}),
# 写入 Python 配置(仅当未配置)
write_file(
str(PIP_CONFIG_PATH),
f"[global]\nindex-url = {PIP_INDEX_URLS[python_mirror]}\ntrusted-host = {PIP_TRUSTED_HOSTS[python_mirror]}",
),
# 写入 Conda 配置(仅当未配置)
write_file(
str(CONDA_CONFIG_PATH),
"show_channel_urls: true\nchannels:\n - " + "\n - ".join(conda_mirror_urls) + "\n - defaults",
),
# 设置 Rust 镜像源
*setenv_group({
"RUSTUP_DIST_SERVER": RUSTUP_MIRRORS[rust_mirror]["RUSTUP_DIST_SERVER"],
"RUSTUP_UPDATE_ROOT": RUSTUP_MIRRORS[rust_mirror]["RUSTUP_UPDATE_ROOT"],
"RUST_SCCACHE_DIR": str(RUST_SCCACHE_DIR),
"RUST_SCCACHE_CACHE_SIZE": RUST_SCCACHE_CACHE_SIZE,
}),
# 写入 Rust 配置(仅当未配置)
write_file(
str(RUST_CONFIG_PATH),
f"""
px.TaskSpec(
"install_mirror",
cmd=INSTALL_MIRROR_SCRIPT,
depends_on=("download_mirror",),
verbose=True,
),
# 安装 Qt 依赖(仅 Linux
px.TaskSpec(
"install_qt_libs",
cmd=["sudo", "apt", "install", "-y", *QT_LIBS],
conditions=(BuiltinConditions.IS_LINUX(),),
depends_on=("install_mirror",),
allow_upstream_skip=True,
verbose=True,
),
# 安装中文字体(仅 Linux
px.TaskSpec(
"install_fonts",
cmd=["sudo", "apt", "install", "-y", *CHINESE_FONTS],
conditions=(BuiltinConditions.IS_LINUX(),),
depends_on=("install_mirror",),
allow_upstream_skip=True,
verbose=True,
),
# 安装 Docker
px.TaskSpec(
"install_docker",
cmd=["sudo", "apt", "install", "-y", "docker-compose-v2"],
conditions=(BuiltinConditions.IS_LINUX(),),
depends_on=("install_mirror",),
allow_upstream_skip=True,
verbose=True,
),
px.TaskSpec(
"add_docker_group",
cmd=["sudo", "usermod", "-aG", "docker", getpass.getuser()],
conditions=(BuiltinConditions.IS_LINUX(),),
depends_on=("install_docker",),
allow_upstream_skip=True,
verbose=True,
),
px.TaskSpec(
"refresh_docker_group",
cmd=["newgrp", "docker"],
conditions=(BuiltinConditions.IS_LINUX(),),
depends_on=("add_docker_group",),
allow_upstream_skip=True,
verbose=True,
),
# 设置 Python 环境变量
*setenv_group(
{
"PIP_INDEX_URL": PIP_INDEX_URLS[python_mirror],
"PIP_TRUSTED_HOSTS": PIP_TRUSTED_HOSTS[python_mirror],
"UV_INDEX_URL": UV_INDEX_URLS[python_mirror],
"UV_PYTHON_INSTALL_MIRROR": UV_PYTHON_INSTALL_MIRROR,
"UV_HTTP_TIMEOUT": "600",
"UV_LINK_MODE": "copy",
}
),
# 写入 Python 配置(仅当未配置)
write_file(
str(PIP_CONFIG_PATH),
f"[global]\nindex-url = {PIP_INDEX_URLS[python_mirror]}\ntrusted-host = {PIP_TRUSTED_HOSTS[python_mirror]}",
),
# 写入 Conda 配置(仅当未配置)
write_file(
str(CONDA_CONFIG_PATH),
"show_channel_urls: true\nchannels:\n - " + "\n - ".join(conda_mirror_urls) + "\n - defaults",
),
# 设置 Rust 镜像源
*setenv_group(
{
"RUSTUP_DIST_SERVER": RUSTUP_MIRRORS[rust_mirror]["RUSTUP_DIST_SERVER"],
"RUSTUP_UPDATE_ROOT": RUSTUP_MIRRORS[rust_mirror]["RUSTUP_UPDATE_ROOT"],
"RUST_SCCACHE_DIR": str(RUST_SCCACHE_DIR),
"RUST_SCCACHE_CACHE_SIZE": RUST_SCCACHE_CACHE_SIZE,
}
),
# 写入 Rust 配置(仅当未配置)
write_file(
str(RUST_CONFIG_PATH),
f"""
[source.crates-io]
replace-with = '{rust_mirror}'
@@ -319,39 +324,43 @@ registry = "sparse+{RUSTUP_MIRRORS[rust_mirror]["TOML_REGISTRY"]}"
[registries.{rust_mirror}]
index = "sparse+{RUSTUP_MIRRORS[rust_mirror]["TOML_REGISTRY"]}"
""",
),
# 下载 Rustup 安装脚本
px.TaskSpec(
"download_rustup",
cmd=["curl", "-fsSL", RUSTUP_DOWNLOAD_URL_LINUX, "-o", "rustup-init.sh"],
conditions=(BuiltinConditions.IS_LINUX(), BuiltinConditions.NOT(BuiltinConditions.HAS_INSTALLED("rustup"))),
verbose=True,
),
px.TaskSpec(
"download_rustup_win",
cmd=[
"powershell",
"-Command",
"Invoke-WebRequest",
"-Uri",
RUSTUP_DOWNLOAD_URL_WINDOWS,
"-OutFile",
"rustup-init.exe",
],
conditions=(
BuiltinConditions.IS_WINDOWS(),
BuiltinConditions.NOT(BuiltinConditions.HAS_INSTALLED("rustup")),
),
verbose=True,
),
# 安装 Rust 工具链
px.TaskSpec(
"install_rust",
cmd=["rustup", "toolchain", "install", rust_version],
conditions=(BuiltinConditions.HAS_INSTALLED("rustup"),),
depends_on=("setenv_rustup_dist_server",),
allow_upstream_skip=True,
verbose=True,
),
])
# 下载 Rustup 安装脚本
px.TaskSpec(
"download_rustup",
cmd=["curl", "-fsSL", RUSTUP_DOWNLOAD_URL_LINUX, "-o", "rustup-init.sh"],
conditions=(
BuiltinConditions.IS_LINUX(),
BuiltinConditions.NOT(BuiltinConditions.HAS_INSTALLED("rustup")),
),
verbose=True,
),
px.TaskSpec(
"download_rustup_win",
cmd=[
"powershell",
"-Command",
"Invoke-WebRequest",
"-Uri",
RUSTUP_DOWNLOAD_URL_WINDOWS,
"-OutFile",
"rustup-init.exe",
],
conditions=(
BuiltinConditions.IS_WINDOWS(),
BuiltinConditions.NOT(BuiltinConditions.HAS_INSTALLED("rustup")),
),
verbose=True,
),
# 安装 Rust 工具链
px.TaskSpec(
"install_rust",
cmd=["rustup", "toolchain", "install", rust_version],
conditions=(BuiltinConditions.HAS_INSTALLED("rustup"),),
depends_on=("setenv_rustup_dist_server",),
allow_upstream_skip=True,
verbose=True,
),
]
)
px.run(graph, strategy="thread", verbose=True)
+9 -7
View File
@@ -567,13 +567,15 @@ class EmlManagerHandler(BaseHTTPRequestHandler):
emails = self.db.search_emails(keyword, field, limit, offset)
total_count = self.db.get_email_count()
self._send_json_response({
"emails": emails,
"count": len(emails),
"total": total_count,
"limit": limit,
"offset": offset,
})
self._send_json_response(
{
"emails": emails,
"count": len(emails),
"total": total_count,
"limit": limit,
"offset": offset,
}
)
def _api_get_email(self, query_params: dict[str, list[str]]) -> None:
"""API: 获取单个邮件详情."""
-137
View File
@@ -1,137 +0,0 @@
"""文件日期处理工具.
自动检测文件名的日期前缀,
并根据文件的实际创建或修改时间重命名文件.
"""
from __future__ import annotations
import argparse
import re
import time
from pathlib import Path
import pyflowx as px
# ============================================================================
# 配置
# ============================================================================
DATE_PATTERN = re.compile(r"(20|19)\d{2}[-_#.~]?((0[1-9])|(1[012]))[-_#.~]?((0[1-9])|([12]\d)|(3[01]))[-_#.~]?")
SEP = "_"
# ============================================================================
# 辅助函数
# ============================================================================
def get_file_timestamp(filepath: Path) -> str:
"""获取文件时间戳."""
modified_time = filepath.stat().st_mtime
created_time = filepath.stat().st_ctime
return time.strftime("%Y%m%d", time.localtime(max((modified_time, created_time))))
def remove_date_prefix(filepath: Path) -> Path:
"""移除文件日期前缀."""
stem = filepath.stem
new_stem = DATE_PATTERN.sub("", stem)
if new_stem != stem:
new_path = filepath.with_name(new_stem + filepath.suffix)
filepath.rename(new_path)
return new_path
return filepath
def add_date_prefix(filepath: Path) -> Path:
"""添加文件日期前缀."""
timestamp = get_file_timestamp(filepath)
stem = filepath.stem
new_stem = f"{timestamp}{SEP}{stem}"
new_path = filepath.with_name(new_stem + filepath.suffix)
if new_path != filepath:
filepath.rename(new_path)
return new_path
return filepath
def process_file_date(filepath: Path, clear: bool = False) -> None:
"""处理单个文件的日期前缀.
Parameters
----------
filepath : Path
文件路径
clear : bool
是否清除日期前缀
"""
if clear:
remove_date_prefix(filepath)
else:
# 先移除旧日期前缀,再添加新日期前缀
new_path = remove_date_prefix(filepath)
add_date_prefix(new_path)
def process_files_date(targets: list[Path], clear: bool = False) -> None:
"""批量处理文件日期前缀.
Parameters
----------
targets : list[Path]
文件路径列表
clear : bool
是否清除日期前缀
"""
for target in targets:
if target.exists() and not target.name.startswith("."):
process_file_date(target, clear)
# ============================================================================
# CLI Runner
# ============================================================================
def main() -> None:
"""文件日期处理工具主函数."""
parser = argparse.ArgumentParser(
description="FileDate - 文件日期处理工具",
usage="filedate <command> [options]",
)
subparsers = parser.add_subparsers(dest="command", help="可用命令")
# 添加日期前缀命令
add_parser = subparsers.add_parser("add", help="添加日期前缀")
add_parser.add_argument("files", nargs="+", help="文件路径")
# 清除日期前缀命令
clear_parser = subparsers.add_parser("clear", help="清除日期前缀")
clear_parser.add_argument("files", nargs="+", help="文件路径")
args = parser.parse_args()
if args.command == "add":
graph = px.Graph.from_specs([
px.TaskSpec(
"process_files_date",
fn=process_files_date,
args=([Path(f) for f in args.files],),
kwargs={"clear": False},
)
])
elif args.command == "clear":
graph = px.Graph.from_specs([
px.TaskSpec(
"process_files_date",
fn=process_files_date,
args=([Path(f) for f in args.files],),
kwargs={"clear": True},
)
])
else:
parser.print_help()
return
px.run(graph, strategy="thread")
-140
View File
@@ -1,140 +0,0 @@
"""文件等级重命名工具.
根据文件等级配置自动重命名文件,
支持多种等级标识和括号格式.
"""
from __future__ import annotations
import argparse
from pathlib import Path
import pyflowx as px
# ============================================================================
# 配置
# ============================================================================
LEVELS: dict[str, str] = {
"0": "",
"1": "PUB,NOR",
"2": "INT",
"3": "CON",
"4": "CLA",
}
BRACKETS: tuple[str, str] = (" ([_(【-", " )]_)】")
# ============================================================================
# 辅助函数
# ============================================================================
def remove_marks(stem: str, marks: list[str]) -> str:
"""从文件名主干中移除所有标记."""
left_brackets, right_brackets = BRACKETS
for mark in marks:
pos = 0
while True:
pos = stem.find(mark, pos)
if pos == -1:
break
b, e = pos - 1, pos + len(mark)
if b >= 0 and e < len(stem) and stem[b] in left_brackets and stem[e] in right_brackets:
stem = stem[:b] + stem[e + 1 :]
else:
pos = e
return stem
def process_file_level(filepath: Path, level: int = 0) -> None:
"""处理单个文件的等级标记.
Parameters
----------
filepath : Path
文件路径
level : int
文件等级 (0-4), 0 用于清除等级
"""
if not (0 <= level < len(LEVELS)):
print(f"无效的等级 {level}, 必须在 0 和 {len(LEVELS) - 1} 之间")
return
if not filepath.exists():
print(f"文件不存在: {filepath}")
return
filestem = filepath.stem
original_stem = filestem
# 移除所有等级标记
for level_names in LEVELS.values():
if level_names:
filestem = remove_marks(filestem, level_names.split(","))
# 移除数字标记
for digit in map(str, range(1, 10)):
filestem = remove_marks(filestem, [digit])
# 添加等级标记
if level > 0:
levelstr = LEVELS.get(str(level), "").split(",")[0]
if levelstr:
filestem = f"{filestem}({levelstr})"
# 重命名文件
if filestem != original_stem:
new_path = filepath.with_name(filestem + filepath.suffix)
filepath.rename(new_path)
print(f"重命名: {filepath} -> {new_path}")
def process_files_level(targets: list[Path], level: int = 0) -> None:
"""批量处理文件等级标记.
Parameters
----------
targets : list[Path]
文件路径列表
level : int
文件等级 (0-4)
"""
for target in targets:
process_file_level(target, level)
# ============================================================================
# CLI Runner
# ============================================================================
def main() -> None:
"""文件等级重命名工具主函数."""
parser = argparse.ArgumentParser(
description="FileLevel - 文件等级重命名工具",
usage="filelevel <command> [options]",
)
subparsers = parser.add_subparsers(dest="command", help="可用命令")
# 设置等级命令
level_parser = subparsers.add_parser("set", help="设置文件等级")
level_parser.add_argument("files", nargs="+", help="文件路径")
level_parser.add_argument("--level", type=int, choices=[0, 1, 2, 3, 4], required=True, help="文件等级 (0-4)")
args = parser.parse_args()
if args.command == "set":
graph = px.Graph.from_specs(
[
px.TaskSpec(
"process_files_level", fn=process_files_level, args=([Path(f) for f in args.files], args.level)
)
]
)
else:
parser.print_help()
return
px.run(graph, strategy="thread")
-85
View File
@@ -1,85 +0,0 @@
"""文件夹备份工具.
备份文件和文件夹为 zip 文件,
自动删除超过最大数量的旧备份文件.
"""
from __future__ import annotations
import time
import zipfile
from pathlib import Path
import pyflowx as px
# ============================================================================
# 辅助函数
# ============================================================================
def remove_dump(src: Path, dst: Path, max_zip: int) -> None:
"""递归删除旧的备份 zip 文件."""
zip_paths = [filepath for filepath in dst.rglob("*.zip") if src.stem in str(filepath)]
zip_files = sorted(zip_paths, key=lambda fn: str(fn)[-19:-4])
if len(zip_files) > max_zip:
zip_files[0].unlink()
remove_dump(src, dst, max_zip)
def zip_target(src: Path, dst: Path, max_zip: int) -> None:
"""将单个文件或文件夹压缩为 zip 文件."""
files = [str(_) for _ in src.rglob("*")]
timestamp = time.strftime("_%Y%m%d_%H%M%S")
target_path = dst / (src.stem + timestamp + ".zip")
with zipfile.ZipFile(target_path, "w") as zip_file:
for file in files:
zip_file.write(file, arcname=file.replace(str(src.parent), ""))
remove_dump(src, dst, max_zip)
print(f"备份完成: {target_path}")
def backup_folder(src: str, dst: str, max_zip: int = 5) -> None:
"""备份文件夹.
Parameters
----------
src : str
源文件夹路径
dst : str
目标文件夹路径
max_zip : int
最大备份数量
"""
src_path = Path(src)
dst_path = Path(dst)
if not src_path.exists():
print(f"源文件夹不存在: {src_path}")
return
if not dst_path.exists():
dst_path.mkdir(parents=True, exist_ok=True)
print(f"创建目标文件夹: {dst_path}")
zip_target(src_path, dst_path, max_zip)
@px.task
def folderback_default() -> None:
"""备份当前目录到 ./backup."""
backup_folder(".", "./backup", 5)
def main() -> None:
"""文件夹备份工具主函数."""
runner = px.CliRunner(
strategy="thread",
description="FolderBack - 文件夹备份工具",
aliases={
# 备份当前目录到 ./backup
"b": folderback_default,
},
)
runner.run_cli()
-76
View File
@@ -1,76 +0,0 @@
"""文件夹压缩工具.
压缩目录下的所有文件/文件夹为 zip 文件,
默认压缩当前目录下的所有子文件夹.
"""
from __future__ import annotations
import shutil
from pathlib import Path
import pyflowx as px
# ============================================================================
# 配置
# ============================================================================
IGNORE_DIRS: list[str] = [".git", ".idea", ".vscode", "__pycache__"]
IGNORE_FILES: list[str] = [".gitignore"]
IGNORE: list[str] = [*IGNORE_DIRS, *IGNORE_FILES]
IGNORE_EXT: list[str] = [".zip", ".rar", ".7z", ".tar", ".gz"]
# ============================================================================
# 辅助函数
# ============================================================================
def archive_folder(folder: Path) -> None:
"""压缩单个文件夹."""
shutil.make_archive(
str(folder.with_name(folder.name)),
format="zip",
base_dir=folder,
)
print(f"压缩完成: {folder.name}.zip")
def zip_folders(cwd: str = ".") -> None:
"""压缩目录下的所有文件夹.
Parameters
----------
cwd : str
工作目录
"""
cwd_path = Path(cwd)
if not cwd_path.exists():
print(f"目录不存在: {cwd_path}")
return
dirs: list[Path] = [
e for e in cwd_path.iterdir() if e.is_dir() and e.name not in IGNORE_DIRS and e.suffix not in IGNORE_EXT
]
for dir_path in dirs:
archive_folder(dir_path)
@px.task
def folderzip_default() -> None:
"""压缩当前目录下的所有文件夹."""
zip_folders(".")
def main() -> None:
"""文件夹压缩工具主函数."""
runner = px.CliRunner(
strategy="thread",
description="FolderZip - 文件夹压缩工具",
aliases={
# 压缩当前目录下的所有文件夹
"z": folderzip_default,
},
)
runner.run_cli()
-107
View File
@@ -1,107 +0,0 @@
"""Git 工具模块.
提供 Git 仓库管理的常用操作封装,
支持初始化、提交、清理、推送等功能.
"""
from __future__ import annotations
from pathlib import Path
import pyflowx as px
EXCLUDE_DIRS = [
# 编辑器相关目录
".vscode",
".idea",
".editorconfig",
".trae",
".qoder",
# 项目相关目录
".venv",
".git",
".tox",
".pytest_cache",
"node_modules",
".ruff_cache",
]
EXCLUDE_CMDS = [arg for d in EXCLUDE_DIRS for arg in ["-e", d]]
def init_sub_dirs() -> None:
"""初始化子目录的Git仓库."""
sub_dirs = [subdir for subdir in Path.cwd().iterdir() if subdir.is_dir()]
for subdir in sub_dirs:
px.run(
px.Graph.from_specs([
px.TaskSpec(
"init",
cmd=["git", "init"],
conditions=(lambda _: not_has_git_repo(),),
cwd=subdir,
),
px.TaskSpec("add", cmd=["git", "add", "."], depends_on=("init",)),
px.TaskSpec("commit", cmd=["git", "commit", "-m", "init commit"], depends_on=("add",)),
]),
)
@px.task(name="isub")
def isub() -> None:
"""初始化子目录的Git仓库."""
init_sub_dirs()
push: px.TaskSpec = px.TaskSpec("push", cmd=["git", "push"])
pull: px.TaskSpec = px.TaskSpec("pull", cmd=["git", "pull"])
kill_tgit: px.TaskSpec = px.TaskSpec("task_kill", cmd=["taskkill", "/f", "/t", "/im", "tgitcache.exe"])
def not_has_git_repo() -> bool:
"""检查当前目录没有Git仓库."""
return not Path.cwd().exists() or not (Path.cwd() / ".git").is_dir()
def has_files() -> bool:
"""检查当前目录是否有文件."""
return bool(list(Path.cwd().glob("*")))
def main() -> None:
"""Git工具主函数."""
runner = px.CliRunner(
strategy="thread",
description="Gittool - Git 执行工具.",
aliases={
# 添加并提交
"a": px.Graph.from_specs([
px.TaskSpec("add", cmd=["git", "add", "."], conditions=(lambda _: has_files(),)),
px.TaskSpec("commit", cmd=["git", "commit", "-m", "chore: update"], depends_on=("add",)),
]),
# 清理(chain: clean → status
"c": px.Graph().chain(
px.TaskSpec("clean", cmd=["git", "clean", "-xfd", *EXCLUDE_CMDS]),
px.TaskSpec("status", cmd=["git", "status", "--porcelain"]),
),
# 初始化、添加并提交
"i": px.Graph.from_specs([
px.TaskSpec("init", cmd=["git", "init"], conditions=(lambda _: not_has_git_repo(),)),
px.TaskSpec("add", cmd=["git", "add", "."], depends_on=("init",), conditions=(lambda _: has_files(),)),
px.TaskSpec(
"commit",
cmd=["git", "commit", "-m", "init commit"],
depends_on=("add",),
conditions=(lambda _: has_files(),),
),
]),
# 初始化子目录
"isub": isub,
# 推送
"p": push,
# 拉取
"pl": pull,
# 重启TGit缓存
"r": kill_tgit,
},
)
runner.run_cli()
+17 -15
View File
@@ -22,20 +22,22 @@ def main():
download_dir: Path = Path(args.dir) if args.dir else Path.home() / ".models" / args.name.split("/")[-1]
download_dir.mkdir(parents=True, exist_ok=True)
graph = px.Graph.from_specs([
px.TaskSpec(
name="download",
cmd=[
"uvx",
"modelscope",
"download",
f"--{args.type}",
args.name,
"--local_dir",
str(download_dir),
],
verbose=True,
),
])
graph = px.Graph.from_specs(
[
px.TaskSpec(
name="download",
cmd=[
"uvx",
"modelscope",
"download",
f"--{args.type}",
args.name,
"--local_dir",
str(download_dir),
],
verbose=True,
),
]
)
px.run(graph, strategy="thread", verbose=True)
+37 -35
View File
@@ -24,40 +24,42 @@ def main():
if not model_dir.exists():
parser.error(f"Model directory {model_dir} does not exist.")
graph = px.Graph.from_specs([
px.TaskSpec(
name="download",
cmd=[
"uv",
"install",
"sglang[all]",
],
conditions=(BuiltinConditions.NOT(BuiltinConditions.HAS_INSTALLED("sglang")),),
verbose=True,
),
px.TaskSpec(
name="run",
cmd=[
"python" if Constants.IS_WINDOWS else "python3",
"-m",
"sglang.launch_server",
"--model-path",
str(model_dir),
"--host",
str(args.host),
"--port",
"8000",
"--mem-fraction-static",
str(args.mem),
"--context-length",
"32768",
"--tool-call-parser",
"qwen",
"--log-level",
str(args.log_level),
],
verbose=True,
),
])
graph = px.Graph.from_specs(
[
px.TaskSpec(
name="download",
cmd=[
"uv",
"install",
"sglang[all]",
],
conditions=(BuiltinConditions.NOT(BuiltinConditions.HAS_INSTALLED("sglang")),),
verbose=True,
),
px.TaskSpec(
name="run",
cmd=[
"python" if Constants.IS_WINDOWS else "python3",
"-m",
"sglang.launch_server",
"--model-path",
str(model_dir),
"--host",
str(args.host),
"--port",
"8000",
"--mem-fraction-static",
str(args.mem),
"--context-length",
"32768",
"--tool-call-parser",
"qwen",
"--log-level",
str(args.log_level),
],
verbose=True,
),
]
)
px.run(graph, strategy="sequential", verbose=True)
-174
View File
@@ -1,174 +0,0 @@
"""LS-DYNA 计算工具.
用于管理 LS-DYNA 仿真计算任务,
支持启动、监控和管理计算进程.
"""
from __future__ import annotations
import argparse
import subprocess
from pathlib import Path
import pyflowx as px
from pyflowx.conditions import Constants
# ============================================================================
# 配置
# ============================================================================
LS_DYNA_COMMANDS: dict[str, list[str]] = {
"windows": ["ls-dyna_mpp", "i=input.k", "ncpu=4"],
"linux": ["ls-dyna_mpp", "i=input.k", "ncpu=8"],
"macos": ["ls-dyna_mpp", "i=input.k", "ncpu=4"],
}
DEFAULT_INPUT_FILE: str = "input.k"
DEFAULT_NCPU: int = 4
# ============================================================================
# 辅助函数
# ============================================================================
def get_ls_dyna_command(input_file: str, ncpu: int) -> list[str]:
"""获取 LS-DYNA 命令.
Parameters
----------
input_file : str
输入文件路径
ncpu : int
CPU 核心数
Returns
-------
list[str]
LS-DYNA 命令列表
"""
if Constants.IS_WINDOWS or Constants.IS_MACOS:
return ["ls-dyna_mpp", f"i={input_file}", f"ncpu={ncpu}"]
else:
return ["ls-dyna_mpp", f"i={input_file}", f"ncpu={ncpu}"]
def run_ls_dyna(input_file: str, ncpu: int = DEFAULT_NCPU) -> None:
"""运行 LS-DYNA 计算.
Parameters
----------
input_file : str
输入文件路径
ncpu : int
CPU 核心数
"""
input_path = Path(input_file)
if not input_path.exists():
print(f"输入文件不存在: {input_path}")
return
cmd = get_ls_dyna_command(input_file, ncpu)
try:
subprocess.run(cmd, check=True)
print(f"LS-DYNA 计算完成: {input_file}")
except FileNotFoundError:
print("未找到 ls-dyna_mpp 命令")
except subprocess.CalledProcessError as e:
print(f"LS-DYNA 计算失败: {e}")
def run_ls_dyna_mpi(input_file: str, ncpu: int = DEFAULT_NCPU) -> None:
"""运行 LS-DYNA MPI 计算.
Parameters
----------
input_file : str
输入文件路径
ncpu : int
CPU 核心数
"""
input_path = Path(input_file)
if not input_path.exists():
print(f"输入文件不存在: {input_path}")
return
cmd = ["mpirun", "-np", str(ncpu), "ls-dyna_mpp", f"i={input_file}"]
try:
subprocess.run(cmd, check=True)
print(f"LS-DYNA MPI 计算完成: {input_file}")
except FileNotFoundError:
print("未找到 mpirun 或 ls-dyna_mpp 命令")
except subprocess.CalledProcessError as e:
print(f"LS-DYNA MPI 计算失败: {e}")
def check_ls_dyna_status() -> None:
"""检查 LS-DYNA 进程状态."""
try:
if Constants.IS_WINDOWS:
result = subprocess.run(
["tasklist", "/fi", "imagename eq ls-dyna_mpp.exe"],
capture_output=True,
text=True,
check=True,
)
print(result.stdout)
else:
result = subprocess.run(
["pgrep", "-f", "ls-dyna"],
capture_output=True,
text=True,
check=False,
)
if result.stdout.strip():
print(f"运行中的 LS-DYNA 进程 PID: {result.stdout.strip()}")
else:
print("没有运行中的 LS-DYNA 进程")
except subprocess.CalledProcessError as e:
print(f"检查进程状态失败: {e}")
# ============================================================================
# CLI Runner
# ============================================================================
def main() -> None:
"""LS-DYNA 计算工具主函数."""
parser = argparse.ArgumentParser(
description="LSCalc - LS-DYNA 计算工具",
usage="lscalc <command> [options]",
)
subparsers = parser.add_subparsers(dest="command", help="可用命令")
# 运行计算命令
run_parser = subparsers.add_parser("run", help="运行 LS-DYNA 计算")
run_parser.add_argument("input_file", help="输入文件路径")
run_parser.add_argument("--ncpu", type=int, default=DEFAULT_NCPU, help="CPU 核心数")
# 运行 MPI 计算命令
mpi_parser = subparsers.add_parser("mpi", help="运行 LS-DYNA MPI 计算")
mpi_parser.add_argument("input_file", help="输入文件路径")
mpi_parser.add_argument("--ncpu", type=int, default=DEFAULT_NCPU, help="CPU 核心数")
# 检查进程状态命令
subparsers.add_parser("status", help="检查 LS-DYNA 进程状态")
args = parser.parse_args()
if args.command == "run":
graph = px.Graph.from_specs(
[px.TaskSpec("run_ls_dyna", fn=run_ls_dyna, args=(args.input_file,), kwargs={"ncpu": args.ncpu})]
)
elif args.command == "mpi":
graph = px.Graph.from_specs(
[px.TaskSpec("run_ls_dyna_mpi", fn=run_ls_dyna_mpi, args=(args.input_file,), kwargs={"ncpu": args.ncpu})]
)
elif args.command == "status":
graph = px.Graph.from_specs([px.TaskSpec("check_ls_dyna_status", fn=check_ls_dyna_status)])
else:
parser.print_help()
return
px.run(graph, strategy="thread")
-349
View File
@@ -1,349 +0,0 @@
"""Python 打包工具模块.
提供 Python 项目打包的常用功能封装,
支持源码打包、依赖打包、嵌入式 Python 安装等功能.
"""
from __future__ import annotations
import argparse
import shutil
import subprocess
import zipfile
from pathlib import Path
import pyflowx as px
# ============================================================================
# 配置
# ============================================================================
DEFAULT_BUILD_DIR = ".pypack"
DEFAULT_DIST_DIR = "dist"
DEFAULT_LIB_DIR = "libs"
DEFAULT_CACHE_DIR = ".cache/pypack"
IGNORE_PATTERNS = [
"__pycache__",
"*.pyc",
"*.pyo",
".git",
".venv",
".idea",
".vscode",
"*.egg-info",
"dist",
"build",
".pytest_cache",
".tox",
".mypy_cache",
]
# ============================================================================
# 辅助函数
# ============================================================================
def pack_source(project_dir: Path, output_dir: Path) -> None:
"""打包项目源码.
Parameters
----------
project_dir : Path
项目目录
output_dir : Path
输出目录
"""
output_dir.mkdir(parents=True, exist_ok=True)
# 检测项目名称
pyproject_file = project_dir / "pyproject.toml"
project_name = project_dir.name
if pyproject_file.exists():
try:
import tomllib
content = pyproject_file.read_text(encoding="utf-8")
data = tomllib.loads(content)
project_name = data.get("project", {}).get("name", project_name)
except ImportError:
pass
# 打包源码
source_dir = output_dir / "src" / project_name
source_dir.mkdir(parents=True, exist_ok=True)
# 复制文件
src_subdir = project_dir / "src"
if src_subdir.exists():
shutil.copytree(
src_subdir,
source_dir / "src",
ignore=shutil.ignore_patterns(*IGNORE_PATTERNS),
dirs_exist_ok=True,
)
else:
for item in project_dir.iterdir():
if item.name in IGNORE_PATTERNS or item.name.startswith("."):
continue
dst_item = source_dir / item.name
if item.is_dir():
shutil.copytree(
item,
dst_item,
ignore=shutil.ignore_patterns(*IGNORE_PATTERNS),
dirs_exist_ok=True,
)
else:
shutil.copy2(item, dst_item)
print(f"源码打包完成: {source_dir}")
def pack_dependencies(lib_dir: Path, dependencies: list[str]) -> None:
"""打包项目依赖.
Parameters
----------
lib_dir : Path
依赖库目录
dependencies : list[str]
依赖列表
"""
lib_dir.mkdir(parents=True, exist_ok=True)
if not dependencies:
print("没有依赖需要打包")
return
# 使用 pip 安装依赖到目标目录
cmd = [
"pip",
"install",
"--target",
str(lib_dir),
"--no-compile",
"--no-warn-script-location",
]
cmd.extend(dependencies)
subprocess.run(cmd, check=True)
print(f"依赖打包完成: {lib_dir}")
def pack_wheel(project_dir: Path, output_dir: Path) -> None:
"""打包项目为 wheel 文件.
Parameters
----------
project_dir : Path
项目目录
output_dir : Path
输出目录
"""
output_dir.mkdir(parents=True, exist_ok=True)
# 使用 pip wheel 打包
cmd = [
"pip",
"wheel",
"--no-deps",
"--wheel-dir",
str(output_dir),
str(project_dir),
]
subprocess.run(cmd, check=True)
print(f"Wheel 打包完成: {output_dir}")
def install_embed_python(version: str, output_dir: Path) -> None:
"""安装嵌入式 Python.
Parameters
----------
version : str
Python 版本 (如: 3.10, 3.11)
output_dir : Path
输出目录
"""
import platform
output_dir.mkdir(parents=True, exist_ok=True)
# 构建下载 URL
arch = platform.machine().lower()
if arch in ["x86_64", "amd64"]:
arch = "amd64"
elif arch in ["arm64", "aarch64"]:
arch = "arm64"
# 解析完整版本号
version_map = {
"3.8": "3.8.10",
"3.9": "3.9.13",
"3.10": "3.10.11",
"3.11": "3.11.9",
"3.12": "3.12.4",
}
full_version = version_map.get(version, f"{version}.0")
# Windows 嵌入式 Python 下载 URL
url = f"https://www.python.org/ftp/python/{full_version}/python-{full_version}-embed-{arch}.zip"
# 下载并解压
cache_file = Path(DEFAULT_CACHE_DIR) / f"python-{full_version}-embed-{arch}.zip"
cache_file.parent.mkdir(parents=True, exist_ok=True)
if not cache_file.exists():
print(f"正在下载嵌入式 Python {full_version}...")
import urllib.request
urllib.request.urlretrieve(url, cache_file)
print(f"下载完成: {cache_file}")
# 解压
with zipfile.ZipFile(cache_file, "r") as zf:
zf.extractall(output_dir)
print(f"嵌入式 Python 安装完成: {output_dir}")
def create_zip_package(source_dir: Path, output_file: Path) -> None:
"""创建 ZIP 打包文件.
Parameters
----------
source_dir : Path
源目录
output_file : Path
输出文件
"""
output_file.parent.mkdir(parents=True, exist_ok=True)
with zipfile.ZipFile(output_file, "w", zipfile.ZIP_DEFLATED) as zf:
for file in source_dir.rglob("*"):
if file.is_file():
arcname = file.relative_to(source_dir)
zf.write(file, arcname)
print(f"ZIP 打包完成: {output_file}")
def clean_build_dir(build_dir: Path) -> None:
"""清理构建目录.
Parameters
----------
build_dir : Path
构建目录
"""
if build_dir.exists():
shutil.rmtree(build_dir)
print(f"清理完成: {build_dir}")
else:
print(f"目录不存在: {build_dir}")
# ============================================================================
# CLI Runner
# ============================================================================
def main() -> None:
"""Python 打包工具主函数."""
parser = argparse.ArgumentParser(
description="PackTool - Python 打包工具",
usage="packtool <command> [options]",
)
subparsers = parser.add_subparsers(dest="command", help="可用命令")
# 源码打包命令
src_parser = subparsers.add_parser("src", help="打包项目源码")
src_parser.add_argument("--project-dir", type=str, default=".", help="项目目录")
src_parser.add_argument("--output-dir", type=str, default=DEFAULT_BUILD_DIR, help="输出目录")
# 依赖打包命令
deps_parser = subparsers.add_parser("deps", help="打包项目依赖")
deps_parser.add_argument("--lib-dir", type=str, default=DEFAULT_LIB_DIR, help="依赖库目录")
deps_parser.add_argument("dependencies", nargs="*", help="依赖列表")
# Wheel 打包命令
wheel_parser = subparsers.add_parser("wheel", help="打包项目为 wheel 文件")
wheel_parser.add_argument("--project-dir", type=str, default=".", help="项目目录")
wheel_parser.add_argument("--output-dir", type=str, default=DEFAULT_DIST_DIR, help="输出目录")
# 嵌入式 Python 安装命令
embed_parser = subparsers.add_parser("embed", help="安装嵌入式 Python")
embed_parser.add_argument("--version", type=str, default="3.10", help="Python 版本")
embed_parser.add_argument("--output-dir", type=str, default="python", help="输出目录")
# ZIP 打包命令
zip_parser = subparsers.add_parser("zip", help="创建 ZIP 打包文件")
zip_parser.add_argument("--source-dir", type=str, default=".", help="源目录")
zip_parser.add_argument("--output-file", type=str, default="package.zip", help="输出文件")
# 清理命令
subparsers.add_parser("clean", help="清理构建目录")
args = parser.parse_args()
if args.command == "src":
graph = px.Graph.from_specs(
[
px.TaskSpec(
"pack_source",
fn=pack_source,
args=(Path(args.project_dir), Path(args.output_dir)),
)
]
)
elif args.command == "deps":
graph = px.Graph.from_specs(
[
px.TaskSpec(
"pack_deps",
fn=pack_dependencies,
args=(Path(args.lib_dir), args.dependencies),
)
]
)
elif args.command == "wheel":
graph = px.Graph.from_specs(
[
px.TaskSpec(
"pack_wheel",
fn=pack_wheel,
args=(Path(args.project_dir), Path(args.output_dir)),
)
]
)
elif args.command == "embed":
graph = px.Graph.from_specs(
[
px.TaskSpec(
"install_embed",
fn=install_embed_python,
args=(args.version, Path(args.output_dir)),
)
]
)
elif args.command == "zip":
graph = px.Graph.from_specs(
[
px.TaskSpec(
"create_zip",
fn=create_zip_package,
args=(Path(args.source_dir), Path(args.output_file)),
)
]
)
elif args.command == "clean":
graph = px.Graph.from_specs([px.TaskSpec("clean_build", fn=clean_build_dir, args=(Path(DEFAULT_BUILD_DIR),))])
else:
parser.print_help()
return
px.run(graph, strategy="thread")
+186
View File
@@ -0,0 +1,186 @@
"""PyFlowX 统一 CLI 入口.
通过 ``pf <tool> [command] [options]`` 调用所有工具,
工具定义在 ``configs/`` 目录下的 YAML 文件中.
用法
----
pf # 列出所有可用工具
pf filedate # 查看 filedate 工具帮助
pf filedate add a.txt # 调用 filedate 的 add 子命令
pf pymake b # 调用 pymake 的 b 别名
"""
from __future__ import annotations
import importlib
import sys
from pathlib import Path
from typing import Sequence
import pyflowx as px
class PfApp:
"""pf 统一入口应用.
路由 ``pf <tool> [command]`` 到 YAML 配置工具或传统 Python 工具.
"""
_CONFIGS_DIR = Path(__file__).parent.parent / "configs"
# 工具名到 YAML 配置文件的映射 (支持短别名)
_TOOL_ALIASES: dict[str, str] = {
"autofmt": "autofmt",
"af": "autofmt",
"bump": "bumpversion",
"bumpversion": "bumpversion",
"bv": "bumpversion",
"filedate": "filedate",
"fd": "filedate",
"filelevel": "filelevel",
"fl": "filelevel",
"folderback": "folderback",
"foldback": "folderback",
"fb": "folderback",
"folderzip": "folderzip",
"foldzip": "folderzip",
"fz": "folderzip",
"git": "gittool",
"gitt": "gittool",
"gittool": "gittool",
"gt": "gittool",
"ls": "lscalc",
"lscalc": "lscalc",
"pack": "packtool",
"packtool": "packtool",
"pk": "packtool",
"pdf": "pdftool",
"pdftool": "pdftool",
"pt": "pdftool",
"pip": "piptool",
"pymake": "pymake",
"piptool": "piptool",
"pp": "piptool",
"reseticon": "reseticoncache",
"reseticoncache": "reseticoncache",
"ric": "reseticoncache",
"screenshot": "screenshot",
"scrcap": "screenshot",
"ss": "screenshot",
"ssh": "sshcopyid",
"sshcopy": "sshcopyid",
"sshcopyid": "sshcopyid",
"sc": "sshcopyid",
}
# 传统工具: 有自己的 main() 函数 (无法 YAML 化的复杂逻辑)
_LEGACY_TOOLS: dict[str, str] = {
"emlman": "pyflowx.cli.emlmanager:main",
"profiler": "pyflowx.cli.profiler:main",
"pxp": "pyflowx.cli.profiler:main",
"yamlrun": "pyflowx.cli.yamlrun:main",
}
def __init__(self, argv: Sequence[str] | None = None) -> None:
self._argv = list(argv) if argv is not None else sys.argv[1:]
def run(self) -> int:
"""主入口, 返回退出码."""
if not self._argv:
self._list_tools()
return 0
tool_name = self._argv[0]
rest_argv = self._argv[1:]
resolved = self._resolve_tool(tool_name)
if resolved is None:
print(f"错误: 未知工具 '{tool_name}'", file=sys.stderr)
print("运行 'pf' 查看可用工具列表", file=sys.stderr)
return 1
tool_type, target = resolved
if tool_type == "legacy":
return self._run_legacy(target, rest_argv)
return self._run_yaml(target, rest_argv)
def _list_tools(self) -> None:
"""列出所有可用工具."""
print("PyFlowX 工具列表:")
print()
print("YAML 配置工具:")
yaml_tools = sorted(set(self._TOOL_ALIASES.values()))
for tool in yaml_tools:
print(f" pf {tool:<15} - {self._tool_description(tool)}")
print()
print("传统工具:")
for tool in sorted(self._LEGACY_TOOLS):
print(f" pf {tool:<15}")
print()
print("示例:")
print(" pf filedate add a.txt")
print(" pf pymake b")
def _tool_description(self, tool_name: str) -> str:
"""获取工具描述 (从 YAML cli.description)."""
config_path = self._CONFIGS_DIR / f"{tool_name}.yaml"
if not config_path.exists():
return ""
try:
import yaml
data = yaml.safe_load(config_path.read_text(encoding="utf-8"))
if isinstance(data, dict) and isinstance(data.get("cli"), dict):
return str(data["cli"].get("description", ""))
except Exception:
pass
return ""
def _resolve_tool(self, name: str) -> tuple[str, str] | None:
"""解析工具名, 返回 (类型, 目标).
类型: "yaml""legacy"
目标: YAML 文件名 (不含 .yaml) 或 legacy 模块路径
"""
if name in self._TOOL_ALIASES:
return ("yaml", self._TOOL_ALIASES[name])
if name in self._LEGACY_TOOLS:
return ("legacy", self._LEGACY_TOOLS[name])
return None
def _run_legacy(self, module_path: str, argv: list[str]) -> int:
"""运行传统工具的 main() 函数."""
module_name, func_name = module_path.split(":", 1)
module = importlib.import_module(module_name)
func = getattr(module, func_name)
original_argv = sys.argv
sys.argv = [f"pf {module_name.split('.')[-1]}", *argv]
try:
func()
return 0
except SystemExit as e:
return int(e.code) if e.code is not None else 0
finally:
sys.argv = original_argv
def _run_yaml(self, target: str, argv: list[str]) -> int:
"""运行 YAML 配置工具."""
config_path = self._CONFIGS_DIR / f"{target}.yaml"
if not config_path.exists():
print(f"错误: 未找到配置文件 '{config_path}'", file=sys.stderr)
print("运行 'pf' 查看可用工具列表", file=sys.stderr)
return 1
print(f"运行配置文件 '{config_path}'")
return px.run_cli(config_path, argv)
def main() -> None:
"""pf 统一入口主函数."""
sys.exit(PfApp().run())
if __name__ == "__main__":
main()
-195
View File
@@ -1,195 +0,0 @@
"""pip 包管理工具模块.
提供 pip 包管理操作的封装,
支持安装、卸载、下载等功能.
"""
from __future__ import annotations
import argparse
import fnmatch
import subprocess
from pathlib import Path
import pyflowx as px
# ============================================================================
# 配置
# ============================================================================
PACKAGE_DIR = "packages"
REQUIREMENTS_FILE = "requirements.txt"
# 受保护的包名集合
_PROTECTED_PACKAGES: frozenset[str] = frozenset({
"pyflowx",
"bitool",
})
# ============================================================================
# 辅助函数
# ============================================================================
def _get_installed_packages() -> list[str]:
"""获取当前环境中所有已安装的包名."""
try:
result = subprocess.run(
["pip", "list", "--format=freeze"],
capture_output=True,
text=True,
check=True,
)
packages: list[str] = []
for line in result.stdout.strip().split("\n"):
if line and "==" in line:
pkg_name = line.split("==")[0].strip()
packages.append(pkg_name)
except (subprocess.SubprocessError, OSError):
return []
return packages
def _expand_wildcard_packages(pattern: str) -> list[str]:
"""展开通配符模式为实际的包名列表."""
if not any(char in pattern for char in ["*", "?", "[", "]"]):
return [pattern]
installed_packages = _get_installed_packages()
matched = [pkg for pkg in installed_packages if fnmatch.fnmatchcase(pkg.lower(), pattern.lower())]
return matched
def _filter_protected_packages(packages: list[str]) -> list[str]:
"""过滤掉受保护的包名."""
safe = [p for p in packages if p.lower() not in {p.lower() for p in _PROTECTED_PACKAGES}]
filtered = [p for p in packages if p.lower() in {p.lower() for p in _PROTECTED_PACKAGES}]
if filtered:
print(f"跳过受保护的包: {', '.join(filtered)}")
return safe
def pip_uninstall(pkg_names: list[str]) -> None:
"""卸载包."""
packages_to_uninstall: list[str] = []
for pattern in pkg_names:
packages_to_uninstall.extend(_expand_wildcard_packages(pattern))
packages_to_uninstall = _filter_protected_packages(packages_to_uninstall)
if not packages_to_uninstall:
return
subprocess.run(["pip", "uninstall", "-y", *packages_to_uninstall], check=True)
def pip_reinstall(pkg_names: list[str], offline: bool = False) -> None:
"""重新安装包."""
safe_pkgs = _filter_protected_packages(pkg_names)
if not safe_pkgs:
print("所有指定的包均为受保护包, 跳过重装")
return
subprocess.run(["pip", "uninstall", "-y", *safe_pkgs], check=True)
options = ["--no-index", "--find-links", "."] if offline else []
subprocess.run(["pip", "install", *options, *safe_pkgs], check=True)
def pip_download(pkg_names: list[str], offline: bool = False) -> None:
"""下载包."""
options = ["--no-index", "--find-links", "."] if offline else []
subprocess.run(
["pip", "download", *pkg_names, *options, "-d", PACKAGE_DIR],
check=True,
)
def pip_freeze() -> None:
"""冻结依赖."""
result = subprocess.run(
["pip", "freeze", "--exclude-editable"],
capture_output=True,
text=True,
check=True,
)
Path(REQUIREMENTS_FILE).write_text(result.stdout)
# ============================================================================
# CLI Runner
# ============================================================================
def main() -> None:
"""pip 工具主函数."""
parser = argparse.ArgumentParser(
description="PipTool - pip 包管理工具",
usage="piptool <command> [options]",
)
subparsers = parser.add_subparsers(dest="command", help="可用命令")
# 安装命令
install_parser = subparsers.add_parser("i", help="安装包")
install_parser.add_argument("packages", nargs="+", help="要安装的包名")
# 卸载命令
uninstall_parser = subparsers.add_parser("u", help="卸载包")
uninstall_parser.add_argument("packages", nargs="+", help="要卸载的包名 (支持通配符)")
# 重装命令
reinstall_parser = subparsers.add_parser("r", help="重新安装包")
reinstall_parser.add_argument("packages", nargs="+", help="要重装的包名")
reinstall_parser.add_argument("--offline", action="store_true", help="使用离线模式")
# 下载命令
download_parser = subparsers.add_parser("d", help="下载包")
download_parser.add_argument("packages", nargs="+", help="要下载的包名")
download_parser.add_argument("--offline", action="store_true", help="使用离线模式")
# 升级 pip 命令
subparsers.add_parser("up", help="升级 pip")
# 冻结依赖命令
subparsers.add_parser("f", help="冻结依赖到 requirements.txt")
args = parser.parse_args()
if args.command == "i":
graph = px.Graph.from_specs([px.TaskSpec("pip_install", cmd=["pip", "install", *args.packages], verbose=True)])
elif args.command == "u":
graph = px.Graph.from_specs([
px.TaskSpec("pip_uninstall", fn=pip_uninstall, args=(args.packages,), verbose=True)
])
elif args.command == "r":
graph = px.Graph.from_specs([
px.TaskSpec(
"pip_reinstall",
fn=pip_reinstall,
args=(args.packages,),
kwargs={"offline": args.offline},
verbose=True,
)
])
elif args.command == "d":
graph = px.Graph.from_specs([
px.TaskSpec(
"pip_download",
fn=pip_download,
args=(args.packages,),
kwargs={"offline": args.offline},
verbose=True,
)
])
elif args.command == "up":
graph = px.Graph.from_specs([
px.TaskSpec("pip_upgrade", cmd=["python", "-m", "pip", "install", "--upgrade", "pip"], verbose=True)
])
elif args.command == "f":
graph = px.Graph.from_specs([px.TaskSpec("pip_freeze", fn=pip_freeze, verbose=True)])
else:
parser.print_help()
return
px.run(graph, strategy="thread")
-125
View File
@@ -1,125 +0,0 @@
"""Python 构建工具模块.
完全替代传统的 Makefile,
提供更好的跨平台兼容性和 Python 生态集成.
"""
from __future__ import annotations
from pathlib import Path
import pyflowx as px
from pyflowx.conditions import Constants
# 项目根目录(pymake.py 在 src/pyflowx/cli,向上四层到达根目录)
ROOT_DIR = Path(__file__).parent.parent.parent.parent
MATURIN_BUILD_COMMAND = ["maturin", "build", "-r"]
if Constants.IS_WINDOWS:
MATURIN_BUILD_COMMAND.extend(["--target", "x86_64-win7-windows-msvc", "-Zbuild-std", "-i", "python3.8"])
# 扁平注册所有任务(px.cmd 自动从命令前两段推导 name)
# 所有任务指定 cwd=ROOT_DIR,确保在项目根目录执行
tasks: list[px.TaskSpec] = [
px.cmd(["uv", "build"], cwd=ROOT_DIR),
px.cmd(MATURIN_BUILD_COMMAND, cwd=ROOT_DIR),
px.cmd(["uv", "sync"], cwd=ROOT_DIR),
px.cmd(["gitt", "c"], name="git_clean", cwd=ROOT_DIR),
px.cmd(
["pytest", "-m", "not slow", "-n", "8", "--dist", "loadfile", "--color=yes", "--durations=10"],
name="test",
cwd=ROOT_DIR,
),
px.cmd(
["pytest", "-m", "not slow", "--dist", "loadfile", "--color=yes", "--durations=10"],
name="test_fast",
cwd=ROOT_DIR,
),
px.cmd(
["pytest", "--cov", "-n", "8", "--dist", "loadfile", "--tb=short", "-v", "--color=yes", "--durations=10"],
name="test_coverage",
cwd=ROOT_DIR,
),
px.cmd(["pyrefly", "check", "."], cwd=ROOT_DIR),
px.cmd(["git", "add", "-A"], name="git_add_all", cwd=ROOT_DIR),
px.cmd(["bumpversion"], cwd=ROOT_DIR),
px.cmd(["bumpversion", "minor"], cwd=ROOT_DIR),
px.cmd(["git", "push"], cwd=ROOT_DIR),
px.cmd(["git", "push", "--tags"], name="git_push_tags", cwd=ROOT_DIR),
px.cmd(["hatch", "publish"], name="publish_python", cwd=ROOT_DIR),
px.cmd(["twine", "upload", "--disable-progress-bar"], name="twine_publish", cwd=ROOT_DIR),
]
# 单任务别名(alias 名与任务名相同):直接内联 TaskSpec,避免 str 自引用
aliases: dict[str, str | list[str | px.TaskSpec] | px.TaskSpec | px.Graph] = {
# 构建命令
"b": "uv_build",
"bc": "maturin_build",
"ba": ["b", "bc"],
# 安装命令
"sync": "uv_sync",
# 清理命令
"c": "git_clean",
# 开发工具
"bump": ["c", "tc", "git_add_all", "bumpversion"],
"bumpmi": "bumpversion_minor",
"cov": ["git_clean", "test_coverage"],
"doc": px.cmd(["sphinx-build", "-b", "html", "docs", "docs/_build"], name="doc", cwd=ROOT_DIR),
"lint": px.cmd(["ruff", "check", "--fix", "--unsafe-fixes"], name="lint", cwd=ROOT_DIR),
"pb": ["twine_publish", "publish_python"],
"t": "test",
"tf": "test_fast",
"tc": ["pyrefly_check", "lint"],
"tox": px.cmd(["tox", "-p", "auto"], name="tox", cwd=ROOT_DIR),
# 发布命令
"p": ["git_clean", "git_push", "git_push_tags"],
}
def main() -> None:
"""pymake 构建工具.
🔨 构建命令:
pymake b - 构建 Python 主包 (uv build)
pymake bc - 构建 Rust 核心模块 (maturin build)
pymake ba - 构建所有包 (先 Python 后 Rust)
📦 安装命令 (开发模式):
pymake sync - 安装依赖包 (uv sync)
🧹 清理命令:
pymake c - 清理所有构建产物 (gitt c)
🛠️ 开发工具:
pymake t - 运行测试 (pytest)
pymake tc - 运行测试并生成覆盖率报告
pymake tf - 运行快速测试 (pytest -m not slow)
pymake lint - 代码格式化与检查 (ruff)
pymake type - 类型检查 (mypy, ty)
pymake doc - 构建文档 (sphinx)
🔬 多版本测试:
pymake tox - 多版本 Python 测试 (tox -p auto)
📦 发布命令:
pymake pb - 发布到 PyPI (twine + hatch)
🔖 版本管理:
pymake bump - 自动升级版本号并提交修改 (清理 + 检查 + 格式化 + git add + bumpversion)
💡 常用工作流:
1. 日常开发: pymake lint && pymake t
2. 构建发布包: pymake ba
3. 多版本兼容性测试: pymake tox
4. 发布到 PyPI: pymake pb
📝 示例:
pymake ba # 构建所有包
pymake sync # 安装依赖
pymake t # 运行测试
pymake tox # 多版本兼容性测试
pymake lint # 格式化代码
pymake type # 类型检查
"""
runner = px.CliRunner(strategy="sequential", description="PyMake - Python 构建工具", tasks=tasks, aliases=aliases)
runner.run_cli()
-10
View File
@@ -1,10 +0,0 @@
from __future__ import annotations
import pyflowx as px
from pyflowx.tasks.system import reset_icon_cache
def main() -> None:
"""重启图标缓存工具主函数."""
graph = px.Graph.from_specs(reset_icon_cache())
px.run(graph, strategy="thread")
-163
View File
@@ -1,163 +0,0 @@
"""截图工具.
跨平台截图工具, 支持全屏截图和区域截图.
"""
from __future__ import annotations
import argparse
import subprocess
from datetime import datetime
from pathlib import Path
import pyflowx as px
from pyflowx.conditions import Constants
# ============================================================================
# 辅助函数
# ============================================================================
def get_screenshot_path(filename: str | None = None) -> Path:
"""获取截图保存路径.
Parameters
----------
filename : str | None
文件名, 如果为 None 则自动生成
Returns
-------
Path
截图保存路径
"""
if filename is None:
timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
filename = f"screenshot_{timestamp}.png"
screenshots_dir = Path.home() / "Pictures" / "screenshots"
screenshots_dir.mkdir(parents=True, exist_ok=True)
return screenshots_dir / filename
def take_screenshot_full(filename: str | None = None) -> None:
"""全屏截图.
Parameters
----------
filename : str | None
文件名
"""
output_path = get_screenshot_path(filename)
if Constants.IS_WINDOWS:
# Windows: 使用 PowerShell 截图
ps_script = f"""
Add-Type -AssemblyName System.Windows.Forms
Add-Type -AssemblyName System.Drawing
$screen = [System.Windows.Forms.Screen]::PrimaryScreen
$bounds = $screen.Bounds
$bitmap = New-Object System.Drawing.Bitmap $bounds.Width, $bounds.Height
$graphics = [System.Drawing.Graphics]::FromImage($bitmap)
$graphics.CopyFromScreen($bounds.Location, [System.Drawing.Point]::Empty, $bounds.Size)
$bitmap.Save('{output_path.as_posix()}')
$graphics.Dispose()
$bitmap.Dispose()
"""
subprocess.run(["powershell", "-Command", ps_script], check=True)
elif Constants.IS_MACOS:
# macOS: 使用 screencapture
subprocess.run(["screencapture", "-x", str(output_path)], check=True)
else:
# Linux: 使用 gnome-screenshot 或 scrot
try:
subprocess.run(["gnome-screenshot", "-f", str(output_path)], check=True)
except FileNotFoundError:
subprocess.run(["scrot", str(output_path)], check=True)
print(f"截图已保存: {output_path}")
def take_screenshot_area(filename: str | None = None) -> None:
"""区域截图.
Parameters
----------
filename : str | None
文件名
"""
output_path = get_screenshot_path(filename)
if Constants.IS_WINDOWS:
# Windows: 使用 PowerShell 截图 (需要用户选择区域)
ps_script = f"""
Add-Type -AssemblyName System.Windows.Forms
Add-Type -AssemblyName System.Drawing
$form = New-Object System.Windows.Forms.Form
$form.WindowState = 'Maximized'
$form.FormBorderStyle = 'None'
$form.BackColor = [System.Drawing.Color]::FromArgb(1, 0, 0)
$form.Opacity = 0.5
$form.TopMost = $true
$form.Show()
Start-Sleep -Milliseconds 100
$screen = [System.Windows.Forms.Screen]::PrimaryScreen
$bounds = $screen.Bounds
$bitmap = New-Object System.Drawing.Bitmap $bounds.Width, $bounds.Height
$graphics = [System.Drawing.Graphics]::FromImage($bitmap)
$graphics.CopyFromScreen($bounds.Location, [System.Drawing.Point]::Empty, $bounds.Size)
$form.Close()
$bitmap.Save('{output_path.as_posix()}')
$graphics.Dispose()
$bitmap.Dispose()
"""
subprocess.run(["powershell", "-Command", ps_script], check=True)
elif Constants.IS_MACOS:
# macOS: 使用 screencapture 交互模式
subprocess.run(["screencapture", "-i", str(output_path)], check=True)
else:
# Linux: 使用 gnome-screenshot 交互模式
try:
subprocess.run(["gnome-screenshot", "-a", "-f", str(output_path)], check=True)
except FileNotFoundError:
subprocess.run(["scrot", "-s", str(output_path)], check=True)
print(f"截图已保存: {output_path}")
# ============================================================================
# CLI Runner
# ============================================================================
def main() -> None:
"""截图工具主函数."""
parser = argparse.ArgumentParser(
description="Screenshot - 截图工具",
usage="screenshot <command> [options]",
)
subparsers = parser.add_subparsers(dest="command", help="可用命令")
# 全屏截图命令
full_parser = subparsers.add_parser("full", help="全屏截图")
full_parser.add_argument("--filename", type=str, help="文件名")
# 区域截图命令
area_parser = subparsers.add_parser("area", help="区域截图")
area_parser.add_argument("--filename", type=str, help="文件名")
args = parser.parse_args()
if args.command == "full":
graph = px.Graph.from_specs(
[px.TaskSpec("screenshot_full", fn=take_screenshot_full, kwargs={"filename": args.filename})]
)
elif args.command == "area":
graph = px.Graph.from_specs(
[px.TaskSpec("screenshot_area", fn=take_screenshot_area, kwargs={"filename": args.filename})]
)
else:
parser.print_help()
return
px.run(graph, strategy="thread")
-122
View File
@@ -1,122 +0,0 @@
"""SSH 密钥部署工具.
类似 ssh-copy-id, 自动将 SSH 公钥部署到远程服务器,
支持密码认证和密钥认证两种方式.
"""
from __future__ import annotations
import argparse
import subprocess
import sys
from pathlib import Path
import pyflowx as px
# ============================================================================
# 辅助函数
# ============================================================================
def ssh_copy_id(
hostname: str,
username: str,
password: str,
port: int = 22,
keypath: str = "~/.ssh/id_rsa.pub",
timeout: int = 30,
) -> None:
"""将 SSH 公钥部署到远程服务器.
Parameters
----------
hostname : str
远程服务器主机名或 IP 地址
username : str
远程服务器用户名
password : str
远程服务器密码
port : int
SSH 端口, 默认 22
keypath : str
公钥文件路径, 默认 ~/.ssh/id_rsa.pub
timeout : int
SSH 操作超时秒数, 默认 30
"""
# 读取公钥
pub_key_path = Path(keypath).expanduser()
if not pub_key_path.exists():
print(f"公钥文件不存在: {pub_key_path}")
sys.exit(1)
pub_key = pub_key_path.read_text().strip()
# 构建部署脚本
script = f"""mkdir -p ~/.ssh && chmod 700 ~/.ssh
cd ~/.ssh && touch authorized_keys && chmod 600 authorized_keys
grep -qF '{pub_key.split()[1]}' authorized_keys 2>/dev/null || echo '{pub_key}' >> authorized_keys"""
# 使用 sshpass 执行
try:
subprocess.run(
[
"sshpass",
"-p",
password,
"ssh",
"-p",
str(port),
"-o",
"StrictHostKeyChecking=no",
"-o",
"UserKnownHostsFile=/dev/null",
"-o",
f"ConnectTimeout={timeout}",
f"{username}@{hostname}",
script,
],
check=True,
timeout=timeout,
)
print(f"SSH 密钥已部署到 {username}@{hostname}:{port}")
except FileNotFoundError:
print(f"未找到 sshpass 工具,请手动执行: ssh-copy-id -p {port} {username}@{hostname}")
sys.exit(1)
except subprocess.TimeoutExpired:
print("SSH 连接超时")
sys.exit(1)
except subprocess.CalledProcessError as e:
print(f"SSH 执行失败: {e}")
sys.exit(1)
# ============================================================================
# CLI Runner
# ============================================================================
def main() -> None:
"""SSH 密钥部署工具主函数."""
parser = argparse.ArgumentParser(
description="SSHCopyID - SSH 密钥部署工具",
usage="sshcopyid <hostname> <username> <password> [--port PORT] [--keypath KEYPATH]",
)
parser.add_argument("hostname", type=str, help="远程服务器主机名或 IP 地址")
parser.add_argument("username", type=str, help="远程服务器用户名")
parser.add_argument("password", type=str, help="远程服务器密码")
parser.add_argument("--port", type=int, default=22, help="SSH 端口 (默认: 22)")
parser.add_argument("--keypath", type=str, default="~/.ssh/id_rsa.pub", help="公钥文件路径")
parser.add_argument("--timeout", type=int, default=30, help="SSH 操作超时秒数 (默认: 30)")
args = parser.parse_args()
graph = px.Graph.from_specs(
[
px.TaskSpec(
"ssh_deploy",
fn=ssh_copy_id,
args=(args.hostname, args.username, args.password),
kwargs={"port": args.port, "keypath": args.keypath, "timeout": args.timeout},
)
]
)
px.run(graph, strategy="thread")
+109
View File
@@ -0,0 +1,109 @@
"""YAML 任务编排执行工具.
从 YAML 文件加载 GitHub Actions 风格的任务图并执行.
支持串并行编排、矩阵扇出、条件执行等 CI/CD 核心概念.
用法
----
yamlrun pipeline.yaml # 执行 YAML 任务图
yamlrun pipeline.yaml --strategy thread # 指定执行策略
yamlrun pipeline.yaml --dry-run # 仅打印任务分层, 不执行
yamlrun pipeline.yaml --list # 列出所有任务名
yamlrun pipeline.yaml --quiet # 静默模式
示例 YAML
----------
::
strategy: thread
jobs:
setup:
cmd: ["git", "clone", "https://github.com/foo/bar"]
build:
needs: [setup]
cmd: ["python", "-m", "build"]
test:
needs: [build]
cmd: ["pytest"]
strategy:
matrix:
python: ["3.8", "3.9"]
"""
from __future__ import annotations
import argparse
import sys
from pathlib import Path
from typing import cast
import pyflowx as px
from pyflowx.executors import Strategy
def main() -> None:
"""YAML 任务编排执行工具主函数."""
parser = argparse.ArgumentParser(
description="YamlRun - 从 YAML 文件加载并执行任务图",
usage="yamlrun <file.yaml> [--strategy STRATEGY] [--dry-run] [--list] [--quiet]",
)
parser.add_argument("file", type=str, help="YAML 任务图文件路径")
parser.add_argument(
"--strategy",
type=str,
default=None,
help="执行策略: sequential/thread/async/dependency (默认: YAML 中指定的策略或 dependency)",
)
parser.add_argument("--dry-run", action="store_true", help="仅打印任务分层, 不执行")
parser.add_argument("--list", action="store_true", help="列出所有任务名后退出")
parser.add_argument("--quiet", action="store_true", help="静默模式, 不打印详细输出")
args = parser.parse_args()
file_path = Path(args.file)
if not file_path.exists():
print(f"错误: 文件不存在: {file_path}", file=sys.stderr)
sys.exit(1)
try:
graph = px.Graph.from_yaml(file_path)
except px.YamlLoadError as e:
print(f"错误: YAML 加载失败: {e}", file=sys.stderr)
sys.exit(1)
if args.list:
print("任务列表:")
for name in graph.names:
spec = graph.spec(name)
deps = ", ".join(spec.depends_on) if spec.depends_on else "(无依赖)"
print(f" - {name} (依赖: {deps})")
sys.exit(0)
layers = graph.layers()
print(f"任务分层 ({len(layers)} 层):")
for i, layer in enumerate(layers):
print(f"{i + 1}: {layer}")
if args.dry_run:
print("\n[dry-run] 跳过执行")
sys.exit(0)
strategy = args.strategy or graph.defaults.strategy or "dependency"
print(f"\n执行策略: {strategy}")
print(f"任务总数: {len(graph.names)}")
print("-" * 40)
report = px.run(graph, strategy=cast(Strategy, strategy), verbose=not args.quiet)
print("-" * 40)
succeeded = report.succeeded_tasks()
failed = report.failed_tasks()
skipped = report.skipped_tasks()
print(f"完成: {len(succeeded)} 成功 / {len(failed)} 失败 / {len(skipped)} 跳过 (共 {len(graph.names)})")
if failed:
print(f"失败任务: {failed}")
sys.exit(1)
if __name__ == "__main__":
main()
+2
View File
@@ -90,6 +90,8 @@ def run_command(spec: TaskSpec[Any]) -> Any: # noqa: PLR0912
print(f"[verbose] 返回码: {result.returncode}", flush=True)
if result.returncode == 0:
if not verbose and result.stdout:
print(result.stdout, end="", flush=True)
return None
err_msg = f"{label}执行失败: `{cmd_str}`, 返回码: {result.returncode}"
+65
View File
@@ -0,0 +1,65 @@
# autofmt - 自动格式化工具
# 用法:
# pf autofmt fmt --target .
# pf autofmt lint --target .
# pf autofmt lint --target . --fix
# pf autofmt doc --root-dir .
# pf autofmt sync --root-dir .
strategy: thread
variables:
TARGET: "."
ROOT_DIR: "."
FIX: false
cli:
description: "AutoFmt - 自动格式化工具"
usage: "pf autofmt <command> [options]"
subcommands:
fmt:
help: "格式化代码"
options:
- name: TARGET
flag: "--target"
type: str
default: "."
help: "目标路径 (默认: .)"
lint:
help: "代码检查"
options:
- name: TARGET
flag: "--target"
type: str
default: "."
help: "目标路径 (默认: .)"
- name: FIX
flag: "--fix"
action: "store_true"
help: "自动修复问题"
doc:
help: "自动添加文档字符串"
options:
- name: ROOT_DIR
flag: "--root-dir"
type: str
default: "."
help: "根目录 (默认: .)"
sync:
help: "同步 pyproject 配置"
options:
- name: ROOT_DIR
flag: "--root-dir"
type: str
default: "."
help: "根目录 (默认: .)"
jobs:
fmt:
cmd: ["ruff", "format", "${TARGET}"]
lint:
cmd: ["ruff", "check", "${TARGET}"]
lint_fix:
cmd: ["ruff", "check", "--fix", "--unsafe-fixes", "${TARGET}"]
doc:
fn: auto_add_docstrings
args: ["${ROOT_DIR}"]
sync:
fn: sync_pyproject_config
args: ["${ROOT_DIR}"]
+27
View File
@@ -0,0 +1,27 @@
# bumpversion - 版本号自动管理工具
# 用法:
# pf bumpversion
# pf bumpversion minor --no-tag
strategy: sequential
variables:
PART: patch
NO_TAG: false
cli:
description: "BumpVersion - 版本号自动管理工具"
usage: "pf bumpversion [part] [options]"
positional:
- name: PART
type: str
default: patch
help: "版本部分: patch, minor, major"
options:
- name: NO_TAG
flag: "--no-tag"
action: "store_true"
help: "提交后不创建 git tag"
jobs:
bump:
fn: bump_project_version
args: ["${PART}"]
kwargs:
no_tag: ${NO_TAG}
+36
View File
@@ -0,0 +1,36 @@
# filedate - 文件日期处理工具
# 用法:
# pf filedate add file1.txt file2.txt
# pf filedate clear file1.txt file2.txt
strategy: thread
variables:
FILES: []
cli:
description: "FileDate - 文件日期处理工具"
usage: "pf filedate <command> [files...]"
subcommands:
add:
help: "添加日期前缀"
positional:
- name: FILES
nargs: "+"
type: path
help: "文件路径"
clear:
help: "清除日期前缀"
positional:
- name: FILES
nargs: "+"
type: path
help: "文件路径"
jobs:
add:
fn: process_files_date
args: ["${FILES}"]
kwargs:
clear: false
clear:
fn: process_files_date
args: ["${FILES}"]
kwargs:
clear: true
+28
View File
@@ -0,0 +1,28 @@
# filelevel - 文件等级重命名工具
# 用法:
# pf filelevel set file.txt --level 2
strategy: thread
variables:
FILES: []
LEVEL: 0
cli:
description: "FileLevel - 文件等级重命名工具"
usage: "pf filelevel <command> [files...] [options]"
subcommands:
set:
help: "设置文件等级"
positional:
- name: FILES
nargs: "+"
type: path
help: "文件路径"
options:
- name: LEVEL
flag: "--level"
type: int
required: true
help: "文件等级 (0-4)"
jobs:
set:
fn: process_files_level
args: ["${FILES}", "${LEVEL}"]
+34
View File
@@ -0,0 +1,34 @@
# folderback - 文件夹备份工具
# 用法:
# pf folderback
# pf folderback --src ./project --dst ./backup --max-zip 10
strategy: thread
variables:
SRC: "."
DST: "./backup"
MAX_ZIP: 5
cli:
description: "FolderBack - 文件夹备份工具"
usage: "pf folderback [options]"
options:
- name: SRC
flag: "--src"
type: str
default: "."
help: "源文件夹路径 (默认: 当前目录)"
- name: DST
flag: "--dst"
type: str
default: "./backup"
help: "目标文件夹路径 (默认: ./backup)"
- name: MAX_ZIP
flag: "--max-zip"
type: int
default: 5
help: "最大备份数量 (默认: 5)"
jobs:
backup:
fn: backup_folder
args: ["${SRC}", "${DST}"]
kwargs:
max_zip: ${MAX_ZIP}
+21
View File
@@ -0,0 +1,21 @@
# folderzip - 文件夹压缩工具
# 用法:
# pf folderzip
# pf folderzip --cwd ./project
strategy: thread
variables:
CWD: "."
cli:
description: "FolderZip - 文件夹压缩工具"
usage: "pf folderzip [options]"
options:
- name: CWD
flag: "--cwd"
type: str
required: false
default: "."
help: "工作目录 (默认: 当前目录)"
jobs:
zip:
fn: zip_folders
args: ["${CWD}"]
+51
View File
@@ -0,0 +1,51 @@
# gittool - Git 执行工具
# 用法:
# pf gittool a
# pf gittool c
# pf gittool i
# pf gittool isub
# pf gittool p
# pf gittool pl
strategy: thread
variables:
# git clean -e 参数列表 (展开为 cmd 数组元素)
CLEAN_EXCLUDES: ["-e", ".venv", "-e", ".tox", "-e", ".pytest_cache",
"-e", ".ruff_cache", "-e", "node_modules",
"-e", ".idea", "-e", ".vscode",
"-e", ".trae", "-e", ".qoder",
"-e", ".editorconfig", "-e", "idea.config",
"-e", "idea_modules.xml", "-e", "vcs.xml"]
cli:
description: "GitTool - Git 执行工具"
usage: "pf gittool <command>"
subcommands:
a:
help: "添加并提交"
c:
help: "清理并查看状态"
i:
help: "初始化并提交"
isub:
help: "初始化子目录"
p:
help: "推送"
pl:
help: "拉取"
jobs:
a:
fn: git_add_commit
args: ["chore: update"]
clean:
cmd: ["git", "clean", "-xfd", "${CLEAN_EXCLUDES}"]
c:
needs: [clean]
cmd: ["git", "status", "--porcelain"]
i:
fn: git_init_add_commit
args: ["init commit"]
isub:
fn: init_sub_dirs
p:
cmd: ["git", "push"]
pl:
cmd: ["git", "pull"]
+51
View File
@@ -0,0 +1,51 @@
# lscalc - LS-DYNA 计算工具
# 用法:
# pf lscalc run input.k --ncpu 4
# pf lscalc status
strategy: thread
variables:
INPUT_FILE: input.k
NCPU: 4
cli:
description: "LSCalc - LS-DYNA 计算工具"
usage: "pf lscalc <command> [options]"
subcommands:
run:
help: "运行 LS-DYNA 计算"
positional:
- name: INPUT_FILE
type: str
help: "输入文件路径"
options:
- name: NCPU
flag: "--ncpu"
type: int
default: 4
help: "CPU 核心数 (默认: 4)"
mpi:
help: "运行 LS-DYNA MPI 计算"
positional:
- name: INPUT_FILE
type: str
help: "输入文件路径"
options:
- name: NCPU
flag: "--ncpu"
type: int
default: 4
help: "CPU 核心数 (默认: 4)"
status:
help: "检查 LS-DYNA 进程状态"
jobs:
run:
fn: run_ls_dyna
args: ["${INPUT_FILE}"]
kwargs:
ncpu: ${NCPU}
mpi:
fn: run_ls_dyna_mpi
args: ["${INPUT_FILE}"]
kwargs:
ncpu: ${NCPU}
status:
fn: check_ls_dyna_status
+107
View File
@@ -0,0 +1,107 @@
# packtool - Python 打包工具
# 用法:
# pf packtool src --project-dir . --output-dir .pypack
# pf packtool deps requests numpy --lib-dir libs
# pf packtool wheel --project-dir . --output-dir dist
# pf packtool embed --version 3.10 --output-dir python
# pf packtool zip --source-dir . --output-file package.zip
# pf packtool clean
strategy: thread
variables:
PROJECT_DIR: "."
OUTPUT_DIR: ".pypack"
LIB_DIR: "libs"
DEPENDENCIES: []
VERSION: "3.10"
OUTPUT_FILE: "package.zip"
SOURCE_DIR: "."
cli:
description: "PackTool - Python 打包工具"
usage: "pf packtool <command> [options]"
subcommands:
src:
help: "打包源码"
options:
- name: PROJECT_DIR
flag: "--project-dir"
type: path
default: "."
help: "项目目录 (默认: .)"
- name: OUTPUT_DIR
flag: "--output-dir"
type: str
default: ".pypack"
help: "输出目录 (默认: .pypack)"
deps:
help: "打包依赖"
positional:
- name: DEPENDENCIES
nargs: "*"
type: str
help: "依赖包列表"
options:
- name: LIB_DIR
flag: "--lib-dir"
type: path
default: "libs"
help: "依赖库目录 (默认: libs)"
wheel:
help: "构建 wheel"
options:
- name: PROJECT_DIR
flag: "--project-dir"
type: path
default: "."
help: "项目目录 (默认: .)"
- name: OUTPUT_DIR
flag: "--output-dir"
type: path
default: "dist"
help: "输出目录 (默认: dist)"
embed:
help: "安装嵌入式 Python"
options:
- name: VERSION
flag: "--version"
type: str
default: "3.10"
help: "Python 版本 (默认: 3.10)"
- name: OUTPUT_DIR
flag: "--output-dir"
type: path
default: "python"
help: "输出目录 (默认: python)"
zip:
help: "创建 zip 包"
options:
- name: SOURCE_DIR
flag: "--source-dir"
type: path
default: "."
help: "源目录 (默认: .)"
- name: OUTPUT_FILE
flag: "--output-file"
type: path
default: "package.zip"
help: "输出文件 (默认: package.zip)"
clean:
help: "清理构建目录"
jobs:
src:
fn: pack_source
args: ["${PROJECT_DIR}", "${OUTPUT_DIR}"]
deps:
fn: pack_dependencies
args: ["${LIB_DIR}", "${DEPENDENCIES}"]
wheel:
fn: pack_wheel
args: ["${PROJECT_DIR}", "${OUTPUT_DIR}"]
embed:
fn: install_embed_python
args: ["${VERSION}", "${OUTPUT_DIR}"]
zip:
fn: create_zip_package
args: ["${SOURCE_DIR}", "${OUTPUT_FILE}"]
clean:
fn: clean_build_dir
args: ["${OUTPUT_DIR}"]
+303
View File
@@ -0,0 +1,303 @@
# pdftool - PDF 文件工具集
# 用法:
# pf pdftool m a.pdf b.pdf --output merged.pdf
# pf pdftool s input.pdf --output-dir split
# pf pdftool c input.pdf --output compressed.pdf
# pf pdftool e input.pdf --output encrypted.pdf --password 123456
# pf pdftool d input.pdf --output decrypted.pdf --password 123456
# pf pdftool xt input.pdf --output output.txt
# pf pdftool xi input.pdf --output-dir images
# pf pdftool w input.pdf --output watermarked.pdf --text CONFIDENTIAL
# pf pdftool r input.pdf --output rotated.pdf --rotation 90
# pf pdftool crop input.pdf --output cropped.pdf --left 10 --top 10 --right 10 --bottom 10
# pf pdftool i input.pdf
# pf pdftool ocr input.pdf --output ocr.pdf --lang chi_sim+eng
# pf pdftool img input.pdf --output-dir images --dpi 300
# pf pdftool repair input.pdf --output repaired.pdf
strategy: thread
variables:
INPUT: input.pdf
INPUTS: []
OUTPUT: output.pdf
OUTPUT_DIR: output
PASSWORD: ""
TEXT: CONFIDENTIAL
ROTATION: 90
MARGINS: [10, 10, 10, 10]
DPI: 300
LANG: chi_sim+eng
ORDER: []
LEFT: 10
TOP: 10
RIGHT: 10
BOTTOM: 10
cli:
description: "PdfTool - PDF 文件工具集"
usage: "pf pdftool <command> [options]"
subcommands:
m:
help: "合并 PDF"
positional:
- name: INPUTS
nargs: "+"
type: path
help: "输入 PDF 文件列表"
options:
- name: OUTPUT
flag: "--output"
type: path
default: "merged.pdf"
help: "输出文件 (默认: merged.pdf)"
s:
help: "拆分 PDF"
positional:
- name: INPUT
type: path
help: "输入 PDF 文件"
options:
- name: OUTPUT_DIR
flag: "--output-dir"
type: path
default: "split"
help: "输出目录 (默认: split)"
c:
help: "压缩 PDF"
positional:
- name: INPUT
type: path
help: "输入 PDF 文件"
options:
- name: OUTPUT
flag: "--output"
type: path
default: "compressed.pdf"
help: "输出文件 (默认: compressed.pdf)"
e:
help: "加密 PDF"
positional:
- name: INPUT
type: path
help: "输入 PDF 文件"
options:
- name: OUTPUT
flag: "--output"
type: path
default: "encrypted.pdf"
help: "输出文件 (默认: encrypted.pdf)"
- name: PASSWORD
flag: "--password"
type: str
required: true
help: "密码 (必填)"
d:
help: "解密 PDF"
positional:
- name: INPUT
type: path
help: "输入 PDF 文件"
options:
- name: OUTPUT
flag: "--output"
type: path
default: "decrypted.pdf"
help: "输出文件 (默认: decrypted.pdf)"
- name: PASSWORD
flag: "--password"
type: str
required: true
help: "密码 (必填)"
xt:
help: "提取文本"
positional:
- name: INPUT
type: path
help: "输入 PDF 文件"
options:
- name: OUTPUT
flag: "--output"
type: path
default: "output.txt"
help: "输出文件 (默认: output.txt)"
xi:
help: "提取图片"
positional:
- name: INPUT
type: path
help: "输入 PDF 文件"
options:
- name: OUTPUT_DIR
flag: "--output-dir"
type: path
default: "images"
help: "输出目录 (默认: images)"
w:
help: "添加水印"
positional:
- name: INPUT
type: path
help: "输入 PDF 文件"
options:
- name: OUTPUT
flag: "--output"
type: path
default: "watermarked.pdf"
help: "输出文件 (默认: watermarked.pdf)"
- name: TEXT
flag: "--text"
type: str
default: "CONFIDENTIAL"
help: "水印文字 (默认: CONFIDENTIAL)"
r:
help: "旋转 PDF"
positional:
- name: INPUT
type: path
help: "输入 PDF 文件"
options:
- name: OUTPUT
flag: "--output"
type: path
default: "rotated.pdf"
help: "输出文件 (默认: rotated.pdf)"
- name: ROTATION
flag: "--rotation"
type: int
default: 90
help: "旋转角度 (默认: 90)"
crop:
help: "裁剪 PDF"
positional:
- name: INPUT
type: path
help: "输入 PDF 文件"
options:
- name: OUTPUT
flag: "--output"
type: path
default: "cropped.pdf"
help: "输出文件 (默认: cropped.pdf)"
- name: LEFT
flag: "--left"
type: int
default: 10
help: "左边距 (默认: 10)"
- name: TOP
flag: "--top"
type: int
default: 10
help: "上边距 (默认: 10)"
- name: RIGHT
flag: "--right"
type: int
default: 10
help: "右边距 (默认: 10)"
- name: BOTTOM
flag: "--bottom"
type: int
default: 10
help: "下边距 (默认: 10)"
i:
help: "查看 PDF 信息"
positional:
- name: INPUT
type: path
help: "输入 PDF 文件"
ocr:
help: "PDF OCR 识别"
positional:
- name: INPUT
type: path
help: "输入 PDF 文件"
options:
- name: OUTPUT
flag: "--output"
type: path
default: "ocr.pdf"
help: "输出文件 (默认: ocr.pdf)"
- name: LANG
flag: "--lang"
type: str
default: "chi_sim+eng"
help: "识别语言 (默认: chi_sim+eng)"
img:
help: "PDF 转图片"
positional:
- name: INPUT
type: path
help: "输入 PDF 文件"
options:
- name: OUTPUT_DIR
flag: "--output-dir"
type: path
default: "images"
help: "输出目录 (默认: images)"
- name: DPI
flag: "--dpi"
type: int
default: 300
help: "DPI (默认: 300)"
repair:
help: "修复 PDF"
positional:
- name: INPUT
type: path
help: "输入 PDF 文件"
options:
- name: OUTPUT
flag: "--output"
type: path
default: "repaired.pdf"
help: "输出文件 (默认: repaired.pdf)"
jobs:
m:
fn: pdf_merge
args: ["${INPUTS}", "${OUTPUT}"]
s:
fn: pdf_split
args: ["${INPUT}", "${OUTPUT_DIR}"]
c:
fn: pdf_compress
args: ["${INPUT}", "${OUTPUT}"]
e:
fn: pdf_encrypt
args: ["${INPUT}", "${OUTPUT}", "${PASSWORD}"]
d:
fn: pdf_decrypt
args: ["${INPUT}", "${OUTPUT}", "${PASSWORD}"]
xt:
fn: pdf_extract_text
args: ["${INPUT}", "${OUTPUT}"]
xi:
fn: pdf_extract_images
args: ["${INPUT}", "${OUTPUT_DIR}"]
w:
fn: pdf_add_watermark
args: ["${INPUT}", "${OUTPUT}"]
kwargs:
text: "${TEXT}"
r:
fn: pdf_rotate
args: ["${INPUT}", "${OUTPUT}"]
kwargs:
rotation: ${ROTATION}
crop:
fn: pdf_crop
args: ["${INPUT}", "${OUTPUT}"]
kwargs:
margins: "${MARGINS}"
i:
fn: pdf_info
args: ["${INPUT}"]
ocr:
fn: pdf_ocr
args: ["${INPUT}", "${OUTPUT}"]
kwargs:
lang: "${LANG}"
img:
fn: pdf_to_images
args: ["${INPUT}", "${OUTPUT_DIR}"]
kwargs:
dpi: ${DPI}
repair:
fn: pdf_repair
args: ["${INPUT}", "${OUTPUT}"]
+78
View File
@@ -0,0 +1,78 @@
# piptool - pip 包管理工具
# 用法:
# pf piptool i requests
# pf piptool u requests
# pf piptool r requests
# pf piptool d requests
# pf piptool up
# pf piptool f
strategy: thread
variables:
PACKAGES: []
OFFLINE: false
cli:
description: "PipTool - pip 包管理工具"
usage: "pf piptool <command> [packages...] [options]"
subcommands:
i:
help: "安装包"
positional:
- name: PACKAGES
nargs: "+"
type: str
help: "包名列表"
u:
help: "卸载包"
positional:
- name: PACKAGES
nargs: "+"
type: str
help: "包名列表"
r:
help: "重装包"
positional:
- name: PACKAGES
nargs: "+"
type: str
help: "包名列表"
options:
- name: OFFLINE
flag: "--offline"
action: "store_true"
help: "离线模式"
d:
help: "下载包"
positional:
- name: PACKAGES
nargs: "+"
type: str
help: "包名列表"
options:
- name: OFFLINE
flag: "--offline"
action: "store_true"
help: "离线模式"
up:
help: "升级 pip"
f:
help: "导出依赖"
jobs:
i:
cmd: ["pip", "install", "${PACKAGES}"]
u:
fn: pip_uninstall
args: ["${PACKAGES}"]
r:
fn: pip_reinstall
args: ["${PACKAGES}"]
kwargs:
offline: ${OFFLINE}
d:
fn: pip_download
args: ["${PACKAGES}"]
kwargs:
offline: ${OFFLINE}
up:
cmd: ["python", "-m", "pip", "install", "--upgrade", "pip"]
f:
fn: pip_freeze
+125
View File
@@ -0,0 +1,125 @@
# pymake - 项目构建工具
# 用法
# pf pymake <command>
# 命令
# b: 构建 Python 主包 (uv build)
# ba: 构建所有包 (Python + Rust)
# bc: 构建 Rust 核心模块 (maturin build)
# bump: 升级版本号 (清理 + 检查 + add + bumpversion)
# bumpmi: 升级次版本号 (bumpversion minor)
# c: 清理构建产物 (调用 gitt c)
# cov: 测试并生成覆盖率
# doc: 构建 Sphinx 文档
# lint: 代码格式化与检查 (ruff)
# p: 推送代码 (清理 + push + push tags)
# pb: 发布到 PyPI (twine + hatch)
# sync: 同步依赖 (uv sync)
# t: 运行测试
# tc: 类型检查 (pyrefly + ruff)
# tf: 快速测试 (无 slow)
# tox: 多版本测试 (tox)
strategy: thread
variables:
CWD: "."
cli:
description: "PyMake - 项目构建工具"
usage: "pf pymake <command>"
options:
- name: CWD
flag: "--cwd"
type: path
required: false
default: "."
help: "工作目录 (默认: 当前目录)"
subcommands:
b: {help: "构建 Python 主包 (uv build)"}
ba: {help: "构建所有包 (Python + Rust)"}
bc: {help: "构建 Rust 核心模块 (maturin build)"}
bump: {help: "升级版本号 (清理 + 检查 + add + bumpversion)"}
bumpmi: {help: "升级次版本号 (bumpversion minor)"}
c: {help: "清理构建产物 (调用 gitt c)"}
cov: {help: "测试并生成覆盖率"}
doc: {help: "构建 Sphinx 文档"}
lint: {help: "代码格式化与检查 (ruff)"}
p: {help: "推送代码 (清理 + push + push tags)"}
pb: {help: "发布到 PyPI (twine + hatch)"}
sync: {help: "同步依赖 (uv sync)"}
t: {help: "运行测试"}
tc: {help: "类型检查 (pyrefly + ruff)"}
tf: {help: "快速测试 (无 slow)"}
tox: {help: "多版本测试 (tox)"}
jobs:
# 单任务别名
b:
cmd: ["uv", "build"]
cwd: ${CWD}
bc:
cmd: ["maturin", "build", "-r"]
cwd: ${CWD}
sync:
cmd: ["uv", "sync"]
cwd: ${CWD}
c:
cmd: ["pf", "gitt", "c"]
cwd: ${CWD}
t:
cmd: ["pytest", "-m", "not slow", "-n", "8", "--dist", "loadfile", "--color=yes", "--durations=10"]
cwd: ${CWD}
tf:
cmd: ["pytest", "-m", "not slow", "--dist", "loadfile", "--color=yes", "--durations=10"]
cwd: ${CWD}
bumpversion:
cmd: ["bumpversion", "patch"]
needs: [git_add_all]
cwd: ${CWD}
bumpmi:
cmd: ["bumpversion", "minor"]
cwd: ${CWD}
doc:
cmd: ["sphinx-build", "-b", "html", "docs", "docs/_build"]
cwd: ${CWD}
lint:
cmd: ["ruff", "check", "--fix", "--unsafe-fixes"]
cwd: ${CWD}
tox:
cmd: ["tox", "-p", "auto"]
cwd: ${CWD}
# 内部 job (不暴露为 subcommand)
test_coverage:
cmd: ["pytest", "--cov", "-n", "8", "--dist", "loadfile", "--tb=short", "-v", "--color=yes", "--durations=10"]
needs: [c]
cwd: ${CWD}
pyrefly_check:
cmd: ["pyrefly", "check", "."]
cwd: ${CWD}
git_add_all:
cmd: ["git", "add", "-A"]
needs: [tc]
cwd: ${CWD}
git_push:
cmd: ["git", "push"]
cwd: ${CWD}
git_push_tags:
cmd: ["git", "push", "--tags"]
cwd: ${CWD}
twine_publish:
cmd: ["twine", "upload", "--disable-progress-bar"]
cwd: ${CWD}
publish_python:
cmd: ["hatch", "publish"]
cwd: ${CWD}
# 聚合 job (方向 B: 有 needs 无 cmd/fn)
ba:
needs: [b, bc]
bump:
needs: [bumpversion]
cov:
needs: [test_coverage]
tc:
needs: [c, pyrefly_check, lint]
p:
needs: [c, git_push, git_push_tags]
pb:
needs: [twine_publish, publish_python]
+13
View File
@@ -0,0 +1,13 @@
# reseticoncache - 重置 Windows 图标缓存
# 用法
# pf reseticon
# 说明
# 杀掉 explorer → 删除 IconCache.db → 删除 iconcache* → 重启 explorer
# 仅在 Windows 上有效, 非 Windows 平台打印提示并跳过
strategy: sequential
cli:
description: "重置 Windows 图标缓存"
usage: "pf reseticon"
jobs:
reset:
fn: reset_icon_cache_run
+34
View File
@@ -0,0 +1,34 @@
# screenshot - 截图工具
# 用法:
# pf screenshot full
# pf screenshot area --filename custom.png
strategy: thread
variables:
FILENAME: null
cli:
description: "Screenshot - 截图工具"
usage: "pf screenshot <command> [options]"
subcommands:
full:
help: "全屏截图"
options:
- name: FILENAME
flag: "--filename"
type: str
help: "文件名"
area:
help: "区域截图"
options:
- name: FILENAME
flag: "--filename"
type: str
help: "文件名"
jobs:
full:
fn: take_screenshot_full
kwargs:
filename: "${FILENAME}"
area:
fn: take_screenshot_area
kwargs:
filename: "${FILENAME}"
+49
View File
@@ -0,0 +1,49 @@
# sshcopyid - SSH 密钥部署工具
# 用法:
# pf sshcopyid hostname username password
# pf sshcopyid server user pass --port 2222
strategy: thread
variables:
HOSTNAME: ""
USERNAME: ""
PASSWORD: ""
PORT: 22
KEYPATH: "~/.ssh/id_rsa.pub"
TIMEOUT: 30
cli:
description: "SSHCopyID - SSH 密钥部署工具"
usage: "pf sshcopyid <hostname> <username> <password> [options]"
positional:
- name: HOSTNAME
type: str
help: "远程服务器主机名或 IP 地址"
- name: USERNAME
type: str
help: "远程服务器用户名"
- name: PASSWORD
type: str
help: "远程服务器密码"
options:
- name: PORT
flag: "--port"
type: int
default: 22
help: "SSH 端口 (默认: 22)"
- name: KEYPATH
flag: "--keypath"
type: str
default: "~/.ssh/id_rsa.pub"
help: "公钥文件路径"
- name: TIMEOUT
flag: "--timeout"
type: int
default: 30
help: "SSH 操作超时秒数 (默认: 30)"
jobs:
deploy:
fn: ssh_copy_id
args: ["${HOSTNAME}", "${USERNAME}", "${PASSWORD}"]
kwargs:
port: ${PORT}
keypath: "${KEYPATH}"
timeout: ${TIMEOUT}
+8 -6
View File
@@ -31,12 +31,14 @@ def aggregate(ctx: px.Context) -> dict[str, Any]:
def main() -> None:
graph = px.Graph.from_specs([
# Static positional args parameterise the same function twice.
px.TaskSpec("fetch_user", fetch_user, args=(1,)),
px.TaskSpec("fetch_posts", fetch_posts, args=(1,)),
px.TaskSpec("aggregate", aggregate, depends_on=("fetch_user", "fetch_posts")),
])
graph = px.Graph.from_specs(
[
# Static positional args parameterise the same function twice.
px.TaskSpec("fetch_user", fetch_user, args=(1,)),
px.TaskSpec("fetch_posts", fetch_posts, args=(1,)),
px.TaskSpec("aggregate", aggregate, depends_on=("fetch_user", "fetch_posts")),
]
)
print("=== Dry run ===")
_ = px.run(graph, strategy="async", dry_run=True)
+15 -13
View File
@@ -46,19 +46,21 @@ def load(transform: list[dict[str, Any]]) -> int:
def main() -> None:
graph = px.Graph.from_specs([
px.TaskSpec("extract_customers", extract_customers, tags=("extract",)),
px.TaskSpec("extract_orders", extract_orders, tags=("extract",)),
px.TaskSpec(
"transform",
transform,
depends_on=("extract_customers", "extract_orders"),
tags=("transform",),
),
px.TaskSpec(
"load", load, depends_on=("transform",), retry=px.RetryPolicy(max_attempts=1, delay=1.0), tags=("load",)
),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("extract_customers", extract_customers, tags=("extract",)),
px.TaskSpec("extract_orders", extract_orders, tags=("extract",)),
px.TaskSpec(
"transform",
transform,
depends_on=("extract_customers", "extract_orders"),
tags=("transform",),
),
px.TaskSpec(
"load", load, depends_on=("transform",), retry=px.RetryPolicy(max_attempts=1, delay=1.0), tags=("load",)
),
]
)
print("=== Execution plan ===")
print(graph.describe())
+7 -5
View File
@@ -29,11 +29,13 @@ def merge(fetch_a: str, fetch_b: str) -> str:
def main() -> None:
graph = px.Graph.from_specs([
px.TaskSpec("fetch_a", fetch_a),
px.TaskSpec("fetch_b", fetch_b),
px.TaskSpec("merge", merge, depends_on=("fetch_a", "fetch_b")),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("fetch_a", fetch_a),
px.TaskSpec("fetch_b", fetch_b),
px.TaskSpec("merge", merge, depends_on=("fetch_a", "fetch_b")),
]
)
print("=== Mermaid diagram ===")
print(graph.to_mermaid("LR"))
+7
View File
@@ -792,6 +792,13 @@ def run(
_print_dry_run(graph, layers)
return RunReport(success=True)
# verbose 模式下, 把所有 spec 的 verbose 标记设为 True,
# 使 execute_command 打印执行命令与返回码 (任务生命周期由 callback 打印)
if verbose:
from dataclasses import replace
graph = Graph.from_specs([replace(s, verbose=True) if not s.verbose else s for s in graph.all_specs().values()])
# 入口统一校验一次:所有策略共用,避免 layers() / dependency 路径
# 各自重复调用 validate()。
graph.validate()
+33 -1
View File
@@ -20,6 +20,7 @@ __all__ = [
import inspect
import sys
from dataclasses import dataclass, field, replace
from pathlib import Path
from typing import Any, Callable, Iterable, Mapping, Sequence
from .errors import CycleError, DuplicateTaskError, MissingDependencyError
@@ -219,7 +220,6 @@ class Graph:
"""
graph = cls(defaults=defaults or GraphDefaults(), namespace=namespace)
pending_refs: list[str] = []
for spec in specs:
if isinstance(spec, str):
pending_refs.append(spec)
@@ -235,6 +235,38 @@ class Graph:
graph.validate()
return graph
@classmethod
def from_yaml(
cls,
path: str | Path,
variables: Mapping[str, Any] | None = None,
) -> Graph:
"""从 YAML 文件构建任务图。
参考 GitHub Actions 风格 schema, 支持 jobs/needs/strategy.matrix/if
等 CI/CD 概念。详见 :mod:`pyflowx.yaml_loader`。
Parameters
----------
path : str | Path
YAML 文件路径
variables : Mapping[str, Any] | None
运行时变量, 用于替换 ``${VAR}`` 占位符
Returns
-------
Graph
构建好的任务图
Raises
------
YamlLoadError
文件不存在、YAML 格式错误、schema 校验失败、循环依赖等
"""
from .yaml_loader import load_yaml
return load_yaml(path, variables=variables)
def add_subgraph(self, sub: Graph, *, namespace: str | None = None) -> Graph:
"""将子图合并到当前图,任务名加命名空间前缀避免冲突。
+18
View File
@@ -0,0 +1,18 @@
"""工具函数模块.
按类别组织 CLI 工具中可复用的函数, 每个子模块使用 ``@px.register_fn`` 注册函数,
供 YAML 任务编排通过 ``fn`` 字段引用.
子模块
------
- :mod:`files` —— 文件日期/等级/备份/压缩相关函数
- :mod:`dev` —— 开发工具 (ruff/版本号/pip/git) 相关函数
- :mod:`media` —— PDF/截图相关函数
- :mod:`system` —— LS-DYNA/SSH/打包相关函数
"""
from __future__ import annotations
from . import dev, files, media, system
__all__ = ["dev", "files", "media", "system"]
+618
View File
@@ -0,0 +1,618 @@
"""开发工具类函数模块.
聚合自动格式化 (autofmt)、版本号管理 (bumpversion)、pip 包管理 (piptool)、
git 工具 (gittool) 的可复用函数. 所有公共函数通过 ``@px.register_fn`` 注册,
供 YAML 任务编排引用.
"""
from __future__ import annotations
import ast
import fnmatch
import re
import subprocess
from pathlib import Path
from typing import Literal
import pyflowx as px
__all__ = [
"IGNORE_PATTERNS",
"PACKAGE_DIR",
"REQUIREMENTS_FILE",
"_PROTECTED_PACKAGES",
"BumpVersionType",
"add_docstring",
"auto_add_docstrings",
"bump_file_version",
"bump_project_version",
"format_all",
"format_with_ruff",
"generate_module_docstring",
"git_add_commit",
"git_init_add_commit",
"has_files",
"init_sub_dirs",
"lint_with_ruff",
"not_has_git_repo",
"pip_download",
"pip_freeze",
"pip_reinstall",
"pip_uninstall",
"sync_pyproject_config",
]
# ============================================================================
# autofmt 配置
# ============================================================================
IGNORE_PATTERNS = [
"__pycache__",
"*.pyc",
"*.pyo",
".git",
".venv",
".idea",
".vscode",
"*.egg-info",
"dist",
"build",
".pytest_cache",
".tox",
".mypy_cache",
]
# ============================================================================
# bumpversion 配置
# ============================================================================
BumpVersionType = Literal["patch", "minor", "major"]
_PYPROJECT_VERSION_PATTERN = re.compile(
r'(?:^|\n)\s*version\s*=\s*["\']'
r"(?P<major>0|[1-9]\d*)\.(?P<minor>0|[1-9]\d*)\.(?P<patch>0|[1-9]\d*)"
r"(?:-(?P<prerelease>(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)(?:\.(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*))?"
r"(?:\+(?P<buildmetadata>[0-9a-zA-Z-]+(?:\.[0-9a-zA-Z-]+)*))?"
r'["\']',
re.MULTILINE,
)
_INIT_VERSION_PATTERN = re.compile(
r'(?:^|\n)\s*__version__\s*=\s*["\']'
r"(?P<major>0|[1-9]\d*)\.(?P<minor>0|[1-9]\d*)\.(?P<patch>0|[1-9]\d*)"
r"(?:-(?P<prerelease>(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)(?:\.(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*))?"
r"(?:\+(?P<buildmetadata>[0-9a-zA-Z-]+(?:\.[0-9a-zA-Z-]+)*))?"
r'["\']',
re.MULTILINE,
)
# ============================================================================
# piptool 配置
# ============================================================================
PACKAGE_DIR = "packages"
REQUIREMENTS_FILE = "requirements.txt"
_PROTECTED_PACKAGES: frozenset[str] = frozenset(
{
"pyflowx",
"bitool",
}
)
# ============================================================================
# autofmt 私有辅助函数
# ============================================================================
# ============================================================================
# autofmt 函数
# ============================================================================
@px.register_fn
def format_with_ruff(target: Path, fix: bool = True) -> None:
"""使用 ruff 格式化代码.
Parameters
----------
target : Path
目标路径
fix : bool
是否自动修复
"""
cmd = ["ruff", "format", str(target)]
if fix:
cmd.append("--fix")
subprocess.run(cmd, check=True)
print(f"ruff format 完成: {target}")
@px.register_fn
def lint_with_ruff(target: Path, fix: bool = True) -> None:
"""使用 ruff 检查代码.
Parameters
----------
target : Path
目标路径
fix : bool
是否自动修复
"""
cmd = ["ruff", "check", str(target)]
if fix:
cmd.extend(["--fix", "--unsafe-fixes"])
subprocess.run(cmd, check=True)
print(f"ruff check 完成: {target}")
@px.register_fn
def add_docstring(file_path: Path, docstring: str) -> bool:
"""为文件添加 docstring.
Parameters
----------
file_path : Path
文件路径
docstring : str
docstring 内容
Returns
-------
bool
是否成功添加
"""
try:
content = file_path.read_text(encoding="utf-8")
tree = ast.parse(content)
first_node = tree.body[0] if tree.body else None
if first_node and isinstance(first_node, ast.Expr) and isinstance(first_node.value, ast.Constant):
return False
lines = content.splitlines()
doc_lines = docstring.splitlines()
doc_lines.append("")
new_content = "\n".join(doc_lines + lines)
file_path.write_text(new_content, encoding="utf-8")
print(f"添加 docstring: {file_path}")
return True
except (OSError, UnicodeDecodeError, SyntaxError) as e:
print(f"处理失败: {file_path} - {e}")
return False
@px.register_fn
def generate_module_docstring(file_path: Path) -> str:
"""生成模块 docstring.
Parameters
----------
file_path : Path
文件路径
Returns
-------
str
生成的 docstring
"""
stem = file_path.stem
parent = file_path.parent.name
keywords = {
"cli": f"Command-line interface for {parent}",
"gui": f"Graphical user interface for {parent}",
"core": f"Core functionality for {parent}",
"util": f"Utility functions for {parent}",
"model": f"Data models for {parent}",
"test": f"Tests for {parent}",
}
for key, desc in keywords.items():
if key in stem.lower():
return f'"""{desc}."""'
return f'"""{stem.replace("_", " ").title()} module."""'
@px.register_fn
def auto_add_docstrings(root_dir: Path) -> int:
"""自动为所有 Python 文件添加 docstring.
Parameters
----------
root_dir : Path
根目录
Returns
-------
int
添加的 docstring 数量
"""
count = 0
for py_file in root_dir.rglob("*.py"):
if any(pattern in str(py_file) for pattern in IGNORE_PATTERNS):
continue
docstring = generate_module_docstring(py_file)
if add_docstring(py_file, docstring):
count += 1
print(f"共添加 {count} 个 docstring")
return count
@px.register_fn
def sync_pyproject_config(root_dir: Path) -> None:
"""同步 pyproject.toml 配置到子项目.
Parameters
----------
root_dir : Path
根目录
"""
main_toml = root_dir / "pyproject.toml"
if not main_toml.exists():
print(f"主项目配置文件不存在: {main_toml}")
return
sub_tomls = [p for p in root_dir.rglob("pyproject.toml") if p != main_toml and ".venv" not in str(p)]
if not sub_tomls:
print("没有找到子项目的 pyproject.toml")
return
print(f"找到 {len(sub_tomls)} 个子项目配置文件")
for sub_toml in sub_tomls:
subprocess.run(["ruff", "format", str(sub_toml)], check=False)
print("配置同步完成")
@px.register_fn
def format_all(root_dir: Path) -> None:
"""格式化所有 Python 文件.
Parameters
----------
root_dir : Path
根目录
"""
subprocess.run(["ruff", "format", str(root_dir)], check=True)
subprocess.run(["ruff", "check", "--fix", "--unsafe-fixes", str(root_dir)], check=True)
print(f"格式化完成: {root_dir}")
# ============================================================================
# bumpversion 私有辅助函数
# ============================================================================
def _get_pattern_for_file(file_name: str) -> re.Pattern[str] | None:
"""根据文件类型获取对应的正则表达式."""
if file_name == "pyproject.toml":
return _PYPROJECT_VERSION_PATTERN
if file_name == "__init__.py":
return _INIT_VERSION_PATTERN
return None
def _calculate_new_version(major: int, minor: int, patch: int, part: BumpVersionType) -> str:
"""计算新版本号."""
if part == "major":
return f"{major + 1}.0.0"
if part == "minor":
return f"{major}.{minor + 1}.0"
return f"{major}.{minor}.{patch + 1}"
def _build_replacement_string(original_match: str, new_version: str, file_name: str) -> str:
"""构建替换字符串, 保留原始格式."""
quote_char = '"' if '"' in original_match else "'"
if file_name == "pyproject.toml":
prefix_match = re.match(r'(\s*version\s*=\s*)["\']', original_match)
prefix = prefix_match.group(1) if prefix_match else "version = "
return f"{prefix}{quote_char}{new_version}{quote_char}"
if file_name == "__init__.py":
prefix_match = re.match(r'(\s*__version__\s*=\s*)["\']', original_match)
prefix = prefix_match.group(1) if prefix_match else "__version__ = "
return f"{prefix}{quote_char}{new_version}{quote_char}"
return new_version
# ============================================================================
# bumpversion 函数
# ============================================================================
@px.register_fn
def bump_file_version(file_path: Path, part: BumpVersionType = "patch") -> str | None:
"""更新文件中的版本号.
Parameters
----------
file_path : Path
要更新的文件路径
part : BumpVersionType
版本部分: patch, minor, major
Returns
-------
str | None
更新后的新版本号, 如果文件中未找到版本号则返回 None
"""
try:
content = file_path.read_text(encoding="utf-8")
except Exception as e:
print(f"读取文件 {file_path} 时出错: {e}")
raise
pattern = _get_pattern_for_file(file_path.name)
if pattern:
match = pattern.search(content)
else:
match = _PYPROJECT_VERSION_PATTERN.search(content) or _INIT_VERSION_PATTERN.search(content)
if not match:
print(f"文件 {file_path} 中未找到版本号模式")
return None
major = int(match.group("major"))
minor = int(match.group("minor"))
patch = int(match.group("patch"))
new_version = _calculate_new_version(major, minor, patch, part)
original_match = match.group(0)
replacement = _build_replacement_string(original_match, new_version, file_path.name)
content = content.replace(original_match, replacement)
try:
file_path.write_text(content, encoding="utf-8")
except Exception as e:
print(f"更新文件 {file_path} 版本号时出错: {e}")
raise
return new_version
@px.register_fn
def bump_project_version(part: BumpVersionType = "patch", no_tag: bool = False) -> str | None:
"""批量更新项目所有文件的版本号并提交.
搜索当前目录下所有 ``__init__.py`` 和 ``pyproject.toml`` 文件
(排除虚拟环境和缓存目录), 更新版本号, 然后执行 git add + commit + tag.
Parameters
----------
part : BumpVersionType
版本部分: patch, minor, major
no_tag : bool
提交后不创建 git tag
Returns
-------
str | None
更新后的新版本号, 如果未找到版本号文件则返回 None
"""
ignore_dirs = {".venv", "venv", ".git", "__pycache__", ".tox", "node_modules", "build", "dist", ".eggs"}
all_files: set[Path] = set()
for pattern in ["__init__.py", "pyproject.toml"]:
for file in Path.cwd().rglob(pattern):
if not any(ignore_dir in file.parts for ignore_dir in ignore_dirs):
all_files.add(file)
if not all_files:
print("未找到包含版本号的文件")
return None
print(f"找到 {len(all_files)} 个文件需要更新版本号")
for file in sorted(all_files):
print(f" - {file.relative_to(Path.cwd())}")
new_version: str | None = None
for file in sorted(all_files):
version = bump_file_version(file, part)
if version is not None and new_version is None:
new_version = version
if not new_version:
print("未能获取新版本号")
return None
print(f"版本号已更新为: {new_version}")
subprocess.run(["git", "add", "."], check=False)
subprocess.run(["git", "commit", "-m", f"bump version to {new_version}"], check=False)
if not no_tag:
tag_name = f"v{new_version}"
subprocess.run(["git", "tag", "-a", tag_name, "-m", f"Release {tag_name}"], check=False)
print(f"已创建标签: {tag_name}")
return new_version
# ============================================================================
# piptool 私有辅助函数
# ============================================================================
def _get_installed_packages() -> list[str]:
"""获取当前环境中所有已安装的包名."""
try:
result = subprocess.run(
["pip", "list", "--format=freeze"],
capture_output=True,
text=True,
check=True,
)
packages: list[str] = []
for line in result.stdout.strip().split("\n"):
if line and "==" in line:
pkg_name = line.split("==")[0].strip()
packages.append(pkg_name)
except (subprocess.SubprocessError, OSError):
return []
return packages
def _expand_wildcard_packages(pattern: str) -> list[str]:
"""展开通配符模式为实际的包名列表."""
if not any(char in pattern for char in ["*", "?", "[", "]"]):
return [pattern]
installed_packages = _get_installed_packages()
matched = [pkg for pkg in installed_packages if fnmatch.fnmatchcase(pkg.lower(), pattern.lower())]
return matched
def _filter_protected_packages(packages: list[str]) -> list[str]:
"""过滤掉受保护的包名."""
safe = [p for p in packages if p.lower() not in {p.lower() for p in _PROTECTED_PACKAGES}]
filtered = [p for p in packages if p.lower() in {p.lower() for p in _PROTECTED_PACKAGES}]
if filtered:
print(f"跳过受保护的包: {', '.join(filtered)}")
return safe
# ============================================================================
# piptool 函数
# ============================================================================
@px.register_fn
def pip_uninstall(pkg_names: list[str]) -> None:
"""卸载包."""
packages_to_uninstall: list[str] = []
for pattern in pkg_names:
packages_to_uninstall.extend(_expand_wildcard_packages(pattern))
packages_to_uninstall = _filter_protected_packages(packages_to_uninstall)
if not packages_to_uninstall:
return
subprocess.run(["pip", "uninstall", "-y", *packages_to_uninstall], check=True)
@px.register_fn
def pip_reinstall(pkg_names: list[str], offline: bool = False) -> None:
"""重新安装包."""
safe_ps = _filter_protected_packages(pkg_names)
if not safe_ps:
print("所有指定的包均为受保护包, 跳过重装")
return
subprocess.run(["pip", "uninstall", "-y", *safe_ps], check=True)
options = ["--no-index", "--find-links", "."] if offline else []
subprocess.run(["pip", "install", *options, *safe_ps], check=True)
@px.register_fn
def pip_download(pkg_names: list[str], offline: bool = False) -> None:
"""下载包."""
options = ["--no-index", "--find-links", "."] if offline else []
subprocess.run(
["pip", "download", *pkg_names, *options, "-d", PACKAGE_DIR],
check=True,
)
@px.register_fn
def pip_freeze() -> None:
"""冻结依赖."""
result = subprocess.run(
["pip", "freeze", "--exclude-editable"],
capture_output=True,
text=True,
check=True,
)
Path(REQUIREMENTS_FILE).write_text(result.stdout)
# ============================================================================
# gittool 函数
# ============================================================================
@px.register_fn
def init_sub_dirs() -> None:
"""初始化子目录的 Git 仓库."""
sub_dirs = [subdir for subdir in Path.cwd().iterdir() if subdir.is_dir()]
for subdir in sub_dirs:
px.run(
px.Graph().chain(
px.cmd(["git", "init"], conditions=(lambda _: not_has_git_repo(),), cwd=subdir),
px.cmd(["git", "add", "."]),
px.cmd(["git", "commit", "-m", "init commit"]),
),
)
@px.register_fn
def not_has_git_repo() -> bool:
"""检查当前目录没有 Git 仓库."""
return not Path.cwd().exists() or not (Path.cwd() / ".git").is_dir()
@px.register_fn
def has_files() -> bool:
"""检查当前 Git 仓库是否有未提交的更改."""
try:
result = subprocess.run(
["git", "status", "--porcelain"],
capture_output=True,
text=True,
check=False,
)
return bool(result.stdout.strip())
except (subprocess.SubprocessError, OSError):
return False
@px.register_fn
def git_add_commit(message: str = "chore: update") -> None:
"""执行 git add + git commit (仅当有未提交更改时).
Parameters
----------
message : str
提交信息
"""
if not has_files():
print("没有文件需要提交")
return
subprocess.run(["git", "add", "."], check=True)
subprocess.run(["git", "commit", "-m", message], check=True)
@px.register_fn
def git_init_add_commit(message: str = "init commit") -> None:
"""执行 git init (若需) + git add + git commit (若有更改).
Parameters
----------
message : str
提交信息
"""
if not_has_git_repo():
subprocess.run(["git", "init"], check=True)
if has_files():
subprocess.run(["git", "add", "."], check=True)
subprocess.run(["git", "commit", "-m", message], check=True)
else:
print("没有文件需要提交")
+327
View File
@@ -0,0 +1,327 @@
"""文件类函数模块.
聚合文件日期处理、文件等级重命名、文件夹备份、文件夹压缩工具的可复用函数.
所有公共函数通过 ``@px.register_fn`` 注册, 供 YAML 任务编排引用.
"""
from __future__ import annotations
import re
import shutil
import time
import zipfile
from pathlib import Path
import pyflowx as px
__all__ = [
"BRACKETS",
"DATE_PATTERN",
"IGNORE_DIRS",
"IGNORE_EXT",
"IGNORE_FILES",
"LEVELS",
"SEP",
"add_date_prefix",
"archive_folder",
"backup_folder",
"folderback_default",
"folderzip_default",
"get_file_timestamp",
"process_file_date",
"process_file_level",
"process_files_date",
"process_files_level",
"remove_date_prefix",
"remove_dump",
"remove_marks",
"zip_folders",
"zip_target",
]
# ============================================================================
# filedate 配置
# ============================================================================
DATE_PATTERN = re.compile(r"(20|19)\d{2}[-_#.~]?((0[1-9])|(1[012]))[-_#.~]?((0[1-9])|([12]\d)|(3[01]))[-_#.~]?")
SEP = "_"
# ============================================================================
# filelevel 配置
# ============================================================================
LEVELS: dict[str, str] = {
"0": "",
"1": "PUB,NOR",
"2": "INT",
"3": "CON",
"4": "CLA",
}
BRACKETS: tuple[str, str] = (" ([_(【-", " )]_)】")
# ============================================================================
# folderzip 配置
# ============================================================================
IGNORE_DIRS: list[str] = [".git", ".idea", ".vscode", "__pycache__"]
IGNORE_FILES: list[str] = [".gitignore"]
IGNORE: list[str] = [*IGNORE_DIRS, *IGNORE_FILES]
IGNORE_EXT: list[str] = [".zip", ".rar", ".7z", ".tar", ".gz"]
# ============================================================================
# filedate 函数
# ============================================================================
@px.register_fn
def get_file_timestamp(filepath: Path) -> str:
"""获取文件时间戳."""
modified_time = filepath.stat().st_mtime
created_time = filepath.stat().st_ctime
return time.strftime("%Y%m%d", time.localtime(max((modified_time, created_time))))
@px.register_fn
def remove_date_prefix(filepath: Path) -> Path:
"""移除文件日期前缀."""
stem = filepath.stem
new_stem = DATE_PATTERN.sub("", stem)
if new_stem != stem:
new_path = filepath.with_name(new_stem + filepath.suffix)
filepath.rename(new_path)
return new_path
return filepath
@px.register_fn
def add_date_prefix(filepath: Path) -> Path:
"""添加文件日期前缀."""
timestamp = get_file_timestamp(filepath)
stem = filepath.stem
new_stem = f"{timestamp}{SEP}{stem}"
new_path = filepath.with_name(new_stem + filepath.suffix)
if new_path != filepath:
filepath.rename(new_path)
return new_path
return filepath
@px.register_fn
def process_file_date(filepath: Path, clear: bool = False) -> None:
"""处理单个文件的日期前缀.
Parameters
----------
filepath : Path
文件路径
clear : bool
是否清除日期前缀
"""
if clear:
remove_date_prefix(filepath)
else:
new_path = remove_date_prefix(filepath)
add_date_prefix(new_path)
@px.register_fn
def process_files_date(targets: list[Path], clear: bool = False) -> None:
"""批量处理文件日期前缀.
Parameters
----------
targets : list[Path]
文件路径列表
clear : bool
是否清除日期前缀
"""
for target in targets:
if target.exists() and not target.name.startswith("."):
process_file_date(target, clear)
# ============================================================================
# filelevel 函数
# ============================================================================
@px.register_fn
def remove_marks(stem: str, marks: list[str]) -> str:
"""从文件名主干中移除所有标记."""
left_brackets, right_brackets = BRACKETS
for mark in marks:
pos = 0
while True:
pos = stem.find(mark, pos)
if pos == -1:
break
b, e = pos - 1, pos + len(mark)
if b >= 0 and e < len(stem) and stem[b] in left_brackets and stem[e] in right_brackets:
stem = stem[:b] + stem[e + 1 :]
else:
pos = e
return stem
@px.register_fn
def process_file_level(filepath: Path, level: int = 0) -> None:
"""处理单个文件的等级标记.
Parameters
----------
filepath : Path
文件路径
level : int
文件等级 (0-4), 0 用于清除等级
"""
if not (0 <= level < len(LEVELS)):
print(f"无效的等级 {level}, 必须在 0 和 {len(LEVELS) - 1} 之间")
return
if not filepath.exists():
print(f"文件不存在: {filepath}")
return
filestem = filepath.stem
original_stem = filestem
for level_names in LEVELS.values():
if level_names:
filestem = remove_marks(filestem, level_names.split(","))
for digit in map(str, range(1, 10)):
filestem = remove_marks(filestem, [digit])
if level > 0:
levelstr = LEVELS.get(str(level), "").split(",")[0]
if levelstr:
filestem = f"{filestem}({levelstr})"
if filestem != original_stem:
new_path = filepath.with_name(filestem + filepath.suffix)
filepath.rename(new_path)
print(f"重命名: {filepath} -> {new_path}")
@px.register_fn
def process_files_level(targets: list[Path], level: int = 0) -> None:
"""批量处理文件等级标记.
Parameters
----------
targets : list[Path]
文件路径列表
level : int
文件等级 (0-4)
"""
for target in targets:
process_file_level(target, level)
# ============================================================================
# folderback 函数
# ============================================================================
@px.register_fn
def remove_dump(src: Path, dst: Path, max_zip: int) -> None:
"""递归删除旧的备份 zip 文件."""
zip_paths = [filepath for filepath in dst.rglob("*.zip") if src.stem in str(filepath)]
zip_files = sorted(zip_paths, key=lambda fn: str(fn)[-19:-4])
if len(zip_files) > max_zip:
zip_files[0].unlink()
remove_dump(src, dst, max_zip)
@px.register_fn
def zip_target(src: Path, dst: Path, max_zip: int) -> None:
"""将单个文件或文件夹压缩为 zip 文件."""
files = [str(_) for _ in src.rglob("*")]
timestamp = time.strftime("_%Y%m%d_%H%M%S")
target_path = dst / (src.stem + timestamp + ".zip")
with zipfile.ZipFile(target_path, "w") as zip_file:
for file in files:
zip_file.write(file, arcname=file.replace(str(src.parent), ""))
remove_dump(src, dst, max_zip)
print(f"备份完成: {target_path}")
@px.register_fn
def backup_folder(src: str, dst: str, max_zip: int = 5) -> None:
"""备份文件夹.
Parameters
----------
src : str
源文件夹路径
dst : str
目标文件夹路径
max_zip : int
最大备份数量
"""
src_path = Path(src)
dst_path = Path(dst)
if not src_path.exists():
print(f"源文件夹不存在: {src_path}")
return
if not dst_path.exists():
dst_path.mkdir(parents=True, exist_ok=True)
print(f"创建目标文件夹: {dst_path}")
zip_target(src_path, dst_path, max_zip)
@px.register_fn("folderback_default")
def folderback_default() -> None:
"""备份当前目录到 ./backup."""
backup_folder(".", "./backup", 5)
# ============================================================================
# folderzip 函数
# ============================================================================
@px.register_fn
def archive_folder(folder: Path) -> None:
"""压缩单个文件夹."""
shutil.make_archive(
str(folder.with_name(folder.name)),
format="zip",
base_dir=folder,
)
print(f"压缩完成: {folder.name}.zip")
@px.register_fn
def zip_folders(cwd: str = ".") -> None:
"""压缩目录下的所有文件夹.
Parameters
----------
cwd : str
工作目录
"""
cwd_path = Path(cwd)
if not cwd_path.exists():
print(f"目录不存在: {cwd_path}")
return
dirs: list[Path] = [
e for e in cwd_path.iterdir() if e.is_dir() and e.name not in IGNORE_DIRS and e.suffix not in IGNORE_EXT
]
for dir_path in dirs:
archive_folder(dir_path)
@px.register_fn("folderzip_default")
def folderzip_default() -> None:
"""压缩当前目录下的所有文件夹."""
zip_folders(".")
@@ -1,15 +1,41 @@
"""PDF 工具模块.
"""媒体类函数模块.
提供 PDF 文件操作的常用功能封装,
支持合并拆分压缩加密水印OCR等功能.
聚合 PDF 工具 (pdftool) 和截图工具 (screenshot) 的可复用函数.
所有公共函数通过 ``@px.register_fn`` 注册, YAML 任务编排引用.
"""
from __future__ import annotations
import argparse
import subprocess
from datetime import datetime
from pathlib import Path
import pyflowx as px
from pyflowx.conditions import Constants
__all__ = [
"DEFAULT_PASSWORD",
"DEFAULT_QUALITY",
"PDF_SUFFIX",
"get_screenshot_path",
"pdf_add_watermark",
"pdf_compress",
"pdf_crop",
"pdf_decrypt",
"pdf_encrypt",
"pdf_extract_images",
"pdf_extract_text",
"pdf_info",
"pdf_merge",
"pdf_ocr",
"pdf_reorder",
"pdf_repair",
"pdf_rotate",
"pdf_split",
"pdf_to_images",
"take_screenshot_area",
"take_screenshot_full",
]
try:
import fitz # PyMuPDF
@@ -36,14 +62,15 @@ DEFAULT_PASSWORD = ""
# ============================================================================
# 辅助函数
# PDF 函数
# ============================================================================
@px.register_fn
def pdf_merge(input_paths: list[Path], output_path: Path) -> None:
"""合并多个 PDF 文件."""
if not HAS_PYPDF:
print("未安装 pypdf 库请安装: pip install pypdf")
print("未安装 pypdf 库, 请安装: pip install pypdf")
return
writer = pypdf.PdfWriter()
@@ -60,10 +87,11 @@ def pdf_merge(input_paths: list[Path], output_path: Path) -> None:
print(f"合并完成: {output_path}")
@px.register_fn
def pdf_split(input_path: Path, output_dir: Path) -> None:
"""拆分 PDF 文件为单页."""
if not HAS_PYPDF:
print("未安装 pypdf 库请安装: pip install pypdf")
print("未安装 pypdf 库, 请安装: pip install pypdf")
return
reader = pypdf.PdfReader(str(input_path))
@@ -79,10 +107,11 @@ def pdf_split(input_path: Path, output_dir: Path) -> None:
print(f"拆分完成: {output_dir}")
@px.register_fn
def pdf_compress(input_path: Path, output_path: Path) -> None:
"""压缩 PDF 文件."""
if not HAS_PYMUPDF:
print("未安装 PyMuPDF 库请安装: pip install PyMuPDF")
print("未安装 PyMuPDF 库, 请安装: pip install PyMuPDF")
return
doc = fitz.open(str(input_path))
@@ -96,10 +125,11 @@ def pdf_compress(input_path: Path, output_path: Path) -> None:
print(f"压缩完成: {output_path} (缩小 {ratio:.1f}%)")
@px.register_fn
def pdf_encrypt(input_path: Path, output_path: Path, password: str) -> None:
"""加密 PDF 文件."""
if not HAS_PYPDF:
print("未安装 pypdf 库请安装: pip install pypdf")
print("未安装 pypdf 库, 请安装: pip install pypdf")
return
reader = pypdf.PdfReader(str(input_path))
@@ -116,10 +146,11 @@ def pdf_encrypt(input_path: Path, output_path: Path, password: str) -> None:
print(f"加密完成: {output_path}")
@px.register_fn
def pdf_decrypt(input_path: Path, output_path: Path, password: str) -> None:
"""解密 PDF 文件."""
if not HAS_PYPDF:
print("未安装 pypdf 库请安装: pip install pypdf")
print("未安装 pypdf 库, 请安装: pip install pypdf")
return
reader = pypdf.PdfReader(str(input_path))
@@ -137,10 +168,11 @@ def pdf_decrypt(input_path: Path, output_path: Path, password: str) -> None:
print(f"解密完成: {output_path}")
@px.register_fn
def pdf_extract_text(input_path: Path, output_path: Path) -> None:
"""提取 PDF 文本."""
if not HAS_PYMUPDF:
print("未安装 PyMuPDF 库请安装: pip install PyMuPDF")
print("未安装 PyMuPDF 库, 请安装: pip install PyMuPDF")
return
doc = fitz.open(str(input_path))
@@ -154,10 +186,11 @@ def pdf_extract_text(input_path: Path, output_path: Path) -> None:
print(f"文本提取完成: {output_path}")
@px.register_fn
def pdf_extract_images(input_path: Path, output_dir: Path) -> None:
"""提取 PDF 图片."""
if not HAS_PYMUPDF:
print("未安装 PyMuPDF 库请安装: pip install PyMuPDF")
print("未安装 PyMuPDF 库, 请安装: pip install PyMuPDF")
return
doc = fitz.open(str(input_path))
@@ -180,10 +213,11 @@ def pdf_extract_images(input_path: Path, output_dir: Path) -> None:
print(f"图片提取完成: {output_dir} (共 {image_count} 张)")
@px.register_fn
def pdf_add_watermark(input_path: Path, output_path: Path, text: str = "CONFIDENTIAL") -> None:
"""添加 PDF 水印."""
if not HAS_PYMUPDF:
print("未安装 PyMuPDF 库请安装: pip install PyMuPDF")
print("未安装 PyMuPDF 库, 请安装: pip install PyMuPDF")
return
doc = fitz.open(str(input_path))
@@ -200,10 +234,11 @@ def pdf_add_watermark(input_path: Path, output_path: Path, text: str = "CONFIDEN
print(f"水印添加完成: {output_path}")
@px.register_fn
def pdf_rotate(input_path: Path, output_path: Path, rotation: int = 90) -> None:
"""旋转 PDF 页面."""
if not HAS_PYMUPDF:
print("未安装 PyMuPDF 库请安装: pip install PyMuPDF")
print("未安装 PyMuPDF 库, 请安装: pip install PyMuPDF")
return
doc = fitz.open(str(input_path))
@@ -216,10 +251,11 @@ def pdf_rotate(input_path: Path, output_path: Path, rotation: int = 90) -> None:
print(f"旋转完成: {output_path}")
@px.register_fn
def pdf_crop(input_path: Path, output_path: Path, margins: tuple[int, int, int, int]) -> None:
"""裁剪 PDF 页面."""
if not HAS_PYMUPDF:
print("未安装 PyMuPDF 库请安装: pip install PyMuPDF")
print("未安装 PyMuPDF 库, 请安装: pip install PyMuPDF")
return
doc = fitz.open(str(input_path))
@@ -241,10 +277,11 @@ def pdf_crop(input_path: Path, output_path: Path, margins: tuple[int, int, int,
print(f"裁剪完成: {output_path}")
@px.register_fn
def pdf_info(input_path: Path) -> None:
"""显示 PDF 信息."""
if not HAS_PYMUPDF:
print("未安装 PyMuPDF 库请安装: pip install PyMuPDF")
print("未安装 PyMuPDF 库, 请安装: pip install PyMuPDF")
return
doc = fitz.open(str(input_path))
@@ -262,17 +299,18 @@ def pdf_info(input_path: Path) -> None:
doc.close()
@px.register_fn
def pdf_ocr(input_path: Path, output_path: Path, lang: str = "chi_sim+eng") -> None:
"""PDF OCR 识别."""
try:
import pytesseract
from PIL import Image
except ImportError:
print("未安装 OCR 相关库请安装: pip install pytesseract pillow")
print("未安装 OCR 相关库, 请安装: pip install pytesseract pillow")
return
if not HAS_PYMUPDF:
print("未安装 PyMuPDF 库请安装: pip install PyMuPDF")
print("未安装 PyMuPDF 库, 请安装: pip install PyMuPDF")
return
doc = fitz.open(str(input_path))
@@ -296,10 +334,11 @@ def pdf_ocr(input_path: Path, output_path: Path, lang: str = "chi_sim+eng") -> N
print(f"OCR 识别完成: {output_path}")
@px.register_fn
def pdf_reorder(input_path: Path, output_path: Path, order: list[int]) -> None:
"""重排 PDF 页面顺序."""
if not HAS_PYPDF:
print("未安装 pypdf 库请安装: pip install pypdf")
print("未安装 pypdf 库, 请安装: pip install pypdf")
return
reader = pypdf.PdfReader(str(input_path))
@@ -316,10 +355,11 @@ def pdf_reorder(input_path: Path, output_path: Path, order: list[int]) -> None:
print(f"重排完成: {output_path}")
@px.register_fn
def pdf_to_images(input_path: Path, output_dir: Path, dpi: int = 300) -> None:
"""PDF 转图片."""
if not HAS_PYMUPDF:
print("未安装 PyMuPDF 库请安装: pip install PyMuPDF")
print("未安装 PyMuPDF 库, 请安装: pip install PyMuPDF")
return
doc = fitz.open(str(input_path))
@@ -335,10 +375,11 @@ def pdf_to_images(input_path: Path, output_dir: Path, dpi: int = 300) -> None:
print(f"转换完成: {output_dir}")
@px.register_fn
def pdf_repair(input_path: Path, output_path: Path) -> None:
"""修复 PDF 文件."""
if not HAS_PYMUPDF:
print("未安装 PyMuPDF 库请安装: pip install PyMuPDF")
print("未安装 PyMuPDF 库, 请安装: pip install PyMuPDF")
return
doc = fitz.open(str(input_path))
@@ -349,175 +390,109 @@ def pdf_repair(input_path: Path, output_path: Path) -> None:
# ============================================================================
# CLI Runner
# screenshot 函数
# ============================================================================
def main() -> None: # noqa: PLR0912
"""PDF 工具主函数."""
parser = argparse.ArgumentParser(
description="PDFTool - PDF 文件工具集",
usage="pdftool <command> [options]",
)
subparsers = parser.add_subparsers(dest="command", help="可用命令")
@px.register_fn
def get_screenshot_path(filename: str | None = None) -> Path:
"""获取截图保存路径.
# 合并 PDF 命令
merge_parser = subparsers.add_parser("m", help="合并 PDF 文件")
merge_parser.add_argument("inputs", nargs="+", help="输入 PDF 文件路径")
merge_parser.add_argument("--output", type=str, default="merged.pdf", help="输出文件路径")
Parameters
----------
filename : str | None
文件名, 如果为 None 则自动生成
# 拆分 PDF 命令
split_parser = subparsers.add_parser("s", help="拆分 PDF 文件为单页")
split_parser.add_argument("input", help="输入 PDF 文件路径")
split_parser.add_argument("--output-dir", type=str, default="split", help="输出目录")
Returns
-------
Path
截图保存路径
"""
if filename is None:
timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
filename = f"screenshot_{timestamp}.png"
# 压缩 PDF 命令
compress_parser = subparsers.add_parser("c", help="压缩 PDF 文件")
compress_parser.add_argument("input", help="输入 PDF 文件路径")
compress_parser.add_argument("--output", type=str, default="compressed.pdf", help="输出文件路径")
screenshots_dir = Path.home() / "Pictures" / "screenshots"
screenshots_dir.mkdir(parents=True, exist_ok=True)
return screenshots_dir / filename
# 加密 PDF 命令
encrypt_parser = subparsers.add_parser("e", help="加密 PDF 文件")
encrypt_parser.add_argument("input", help="输入 PDF 文件路径")
encrypt_parser.add_argument("--output", type=str, default="encrypted.pdf", help="输出文件路径")
encrypt_parser.add_argument("--password", type=str, required=True, help="密码")
# 解密 PDF 命令
decrypt_parser = subparsers.add_parser("d", help="解密 PDF 文件")
decrypt_parser.add_argument("input", help="输入 PDF 文件路径")
decrypt_parser.add_argument("--output", type=str, default="decrypted.pdf", help="输出文件路径")
decrypt_parser.add_argument("--password", type=str, required=True, help="密码")
@px.register_fn
def take_screenshot_full(filename: str | None = None) -> None:
"""全屏截图.
# 提取文本命令
extract_text_parser = subparsers.add_parser("xt", help="提取 PDF 文本")
extract_text_parser.add_argument("input", help="输入 PDF 文件路径")
extract_text_parser.add_argument("--output", type=str, default="output.txt", help="输出文件路径")
Parameters
----------
filename : str | None
文件名
"""
output_path = get_screenshot_path(filename)
# 提取图片命令
extract_images_parser = subparsers.add_parser("xi", help="提取 PDF 图片")
extract_images_parser.add_argument("input", help="输入 PDF 文件路径")
extract_images_parser.add_argument("--output-dir", type=str, default="images", help="输出目录")
# 添加水印命令
watermark_parser = subparsers.add_parser("w", help="添加 PDF 水印")
watermark_parser.add_argument("input", help="输入 PDF 文件路径")
watermark_parser.add_argument("--output", type=str, default="watermarked.pdf", help="输出文件路径")
watermark_parser.add_argument("--text", type=str, default="CONFIDENTIAL", help="水印文本")
# 旋转 PDF 命令
rotate_parser = subparsers.add_parser("r", help="旋转 PDF 页面")
rotate_parser.add_argument("input", help="输入 PDF 文件路径")
rotate_parser.add_argument("--output", type=str, default="rotated.pdf", help="输出文件路径")
rotate_parser.add_argument("--rotation", type=int, default=90, help="旋转角度 (90, 180, 270)")
# 裁剪 PDF 命令
crop_parser = subparsers.add_parser("crop", help="裁剪 PDF 页面")
crop_parser.add_argument("input", help="输入 PDF 文件路径")
crop_parser.add_argument("--output", type=str, default="cropped.pdf", help="输出文件路径")
crop_parser.add_argument("--left", type=int, default=10, help="左边裁剪")
crop_parser.add_argument("--top", type=int, default=10, help="顶部裁剪")
crop_parser.add_argument("--right", type=int, default=10, help="右边裁剪")
crop_parser.add_argument("--bottom", type=int, default=10, help="底部裁剪")
# 显示信息命令
info_parser = subparsers.add_parser("i", help="显示 PDF 信息")
info_parser.add_argument("input", help="输入 PDF 文件路径")
# OCR 识别命令
ocr_parser = subparsers.add_parser("ocr", help="PDF OCR 识别")
ocr_parser.add_argument("input", help="输入 PDF 文件路径")
ocr_parser.add_argument("--output", type=str, default="ocr.pdf", help="输出文件路径")
ocr_parser.add_argument("--lang", type=str, default="chi_sim+eng", help="OCR 语言")
# 转换图片命令
to_images_parser = subparsers.add_parser("img", help="PDF 转图片")
to_images_parser.add_argument("input", help="输入 PDF 文件路径")
to_images_parser.add_argument("--output-dir", type=str, default="images", help="输出目录")
to_images_parser.add_argument("--dpi", type=int, default=300, help="图片 DPI")
# 修复 PDF 命令
repair_parser = subparsers.add_parser("repair", help="修复 PDF 文件")
repair_parser.add_argument("input", help="输入 PDF 文件路径")
repair_parser.add_argument("--output", type=str, default="repaired.pdf", help="输出文件路径")
args = parser.parse_args()
if args.command == "m":
graph = px.Graph.from_specs([
px.TaskSpec("pdf_merge", fn=pdf_merge, args=([Path(p) for p in args.inputs], Path(args.output)))
])
elif args.command == "s":
graph = px.Graph.from_specs([
px.TaskSpec("pdf_split", fn=pdf_split, args=(Path(args.input), Path(args.output_dir)))
])
elif args.command == "c":
graph = px.Graph.from_specs([
px.TaskSpec("pdf_compress", fn=pdf_compress, args=(Path(args.input), Path(args.output)))
])
elif args.command == "e":
graph = px.Graph.from_specs([
px.TaskSpec("pdf_encrypt", fn=pdf_encrypt, args=(Path(args.input), Path(args.output), args.password))
])
elif args.command == "d":
graph = px.Graph.from_specs([
px.TaskSpec("pdf_decrypt", fn=pdf_decrypt, args=(Path(args.input), Path(args.output), args.password))
])
elif args.command == "xt":
graph = px.Graph.from_specs([
px.TaskSpec("pdf_extract_text", fn=pdf_extract_text, args=(Path(args.input), Path(args.output)))
])
elif args.command == "xi":
graph = px.Graph.from_specs([
px.TaskSpec("pdf_extract_images", fn=pdf_extract_images, args=(Path(args.input), Path(args.output_dir)))
])
elif args.command == "w":
graph = px.Graph.from_specs([
px.TaskSpec(
"pdf_watermark",
fn=pdf_add_watermark,
args=(Path(args.input), Path(args.output)),
kwargs={"text": args.text},
)
])
elif args.command == "r":
graph = px.Graph.from_specs([
px.TaskSpec(
"pdf_rotate",
fn=pdf_rotate,
args=(Path(args.input), Path(args.output)),
kwargs={"rotation": args.rotation},
)
])
elif args.command == "crop":
graph = px.Graph.from_specs([
px.TaskSpec(
"pdf_crop",
fn=pdf_crop,
args=(Path(args.input), Path(args.output)),
kwargs={"margins": (args.left, args.top, args.right, args.bottom)},
)
])
elif args.command == "i":
graph = px.Graph.from_specs([px.TaskSpec("pdf_info", fn=pdf_info, args=(Path(args.input),))])
elif args.command == "ocr":
graph = px.Graph.from_specs([
px.TaskSpec("pdf_ocr", fn=pdf_ocr, args=(Path(args.input), Path(args.output)), kwargs={"lang": args.lang})
])
elif args.command == "img":
graph = px.Graph.from_specs([
px.TaskSpec(
"pdf_to_images",
fn=pdf_to_images,
args=(Path(args.input), Path(args.output_dir)),
kwargs={"dpi": args.dpi},
)
])
elif args.command == "repair":
graph = px.Graph.from_specs([
px.TaskSpec("pdf_repair", fn=pdf_repair, args=(Path(args.input), Path(args.output)))
])
if Constants.IS_WINDOWS:
ps_script = f"""
Add-Type -AssemblyName System.Windows.Forms
Add-Type -AssemblyName System.Drawing
$screen = [System.Windows.Forms.Screen]::PrimaryScreen
$bounds = $screen.Bounds
$bitmap = New-Object System.Drawing.Bitmap $bounds.Width, $bounds.Height
$graphics = [System.Drawing.Graphics]::FromImage($bitmap)
$graphics.CopyFromScreen($bounds.Location, [System.Drawing.Point]::Empty, $bounds.Size)
$bitmap.Save('{output_path.as_posix()}')
$graphics.Dispose()
$bitmap.Dispose()
"""
subprocess.run(["powershell", "-Command", ps_script], check=True)
elif Constants.IS_MACOS:
subprocess.run(["screencapture", "-x", str(output_path)], check=True)
else:
parser.print_help()
return
try:
subprocess.run(["gnome-screenshot", "-f", str(output_path)], check=True)
except FileNotFoundError:
subprocess.run(["scrot", str(output_path)], check=True)
px.run(graph, strategy="thread")
print(f"截图已保存: {output_path}")
@px.register_fn
def take_screenshot_area(filename: str | None = None) -> None:
"""区域截图.
Parameters
----------
filename : str | None
文件名
"""
output_path = get_screenshot_path(filename)
if Constants.IS_WINDOWS:
ps_script = f"""
Add-Type -AssemblyName System.Windows.Forms
Add-Type -AssemblyName System.Drawing
$form = New-Object System.Windows.Forms.Form
$form.WindowState = 'Maximized'
$form.FormBorderStyle = 'None'
$form.BackColor = [System.Drawing.Color]::FromArgb(1, 0, 0)
$form.Opacity = 0.5
$form.TopMost = $true
$form.Show()
Start-Sleep -Milliseconds 100
$screen = [System.Windows.Forms.Screen]::PrimaryScreen
$bounds = $screen.Bounds
$bitmap = New-Object System.Drawing.Bitmap $bounds.Width, $bounds.Height
$graphics = [System.Drawing.Graphics]::FromImage($bitmap)
$graphics.CopyFromScreen($bounds.Location, [System.Drawing.Point]::Empty, $bounds.Size)
$form.Close()
$bitmap.Save('{output_path.as_posix()}')
$graphics.Dispose()
$bitmap.Dispose()
"""
subprocess.run(["powershell", "-Command", ps_script], check=True)
elif Constants.IS_MACOS:
subprocess.run(["screencapture", "-i", str(output_path)], check=True)
else:
try:
subprocess.run(["gnome-screenshot", "-a", "-f", str(output_path)], check=True)
except FileNotFoundError:
subprocess.run(["scrot", "-s", str(output_path)], check=True)
print(f"截图已保存: {output_path}")
+504
View File
@@ -0,0 +1,504 @@
"""系统类函数模块.
聚合 LS-DYNA 计算 (lscalc)、SSH 密钥部署 (sshcopyid)、Python 打包 (packtool)、
重置图标缓存 (reset_icon_cache) 的可复用函数. 所有公共函数通过 ``@px.register_fn``
注册, 供 YAML 任务编排引用.
"""
from __future__ import annotations
import os
import platform
import shutil
import subprocess
import sys
import urllib.request
import zipfile
from pathlib import Path
import pyflowx as px
from pyflowx.conditions import Constants
__all__ = [
"DEFAULT_BUILD_DIR",
"DEFAULT_CACHE_DIR",
"DEFAULT_DIST_DIR",
"DEFAULT_INPUT_FILE",
"DEFAULT_LIB_DIR",
"DEFAULT_NCPU",
"IGNORE_PATTERNS",
"LS_DYNA_COMMANDS",
"check_ls_dyna_status",
"clean_build_dir",
"create_zip_package",
"get_ls_dyna_command",
"install_embed_python",
"pack_dependencies",
"pack_source",
"pack_wheel",
"reset_icon_cache_run",
"run_ls_dyna",
"run_ls_dyna_mpi",
"ssh_copy_id",
]
# ============================================================================
# lscalc 配置
# ============================================================================
LS_DYNA_COMMANDS: dict[str, list[str]] = {
"windows": ["ls-dyna_mpp", "i=input.k", "ncpu=4"],
"linux": ["ls-dyna_mpp", "i=input.k", "ncpu=8"],
"macos": ["ls-dyna_mpp", "i=input.k", "ncpu=4"],
}
DEFAULT_INPUT_FILE: str = "input.k"
DEFAULT_NCPU: int = 4
# ============================================================================
# packtool 配置
# ============================================================================
DEFAULT_BUILD_DIR = ".pypack"
DEFAULT_DIST_DIR = "dist"
DEFAULT_LIB_DIR = "libs"
DEFAULT_CACHE_DIR = ".cache/pypack"
IGNORE_PATTERNS = [
"__pycache__",
"*.pyc",
"*.pyo",
".git",
".venv",
".idea",
".vscode",
"*.egg-info",
"dist",
"build",
".pytest_cache",
".tox",
".mypy_cache",
]
# ============================================================================
# lscalc 函数
# ============================================================================
@px.register_fn
def get_ls_dyna_command(input_file: str, ncpu: int) -> list[str]:
"""获取 LS-DYNA 命令.
Parameters
----------
input_file : str
输入文件路径
ncpu : int
CPU 核心数
Returns
-------
list[str]
LS-DYNA 命令列表
"""
if Constants.IS_WINDOWS or Constants.IS_MACOS:
return ["ls-dyna_mpp", f"i={input_file}", f"ncpu={ncpu}"]
else:
return ["ls-dyna_mpp", f"i={input_file}", f"ncpu={ncpu}"]
@px.register_fn
def run_ls_dyna(input_file: str, ncpu: int = DEFAULT_NCPU) -> None:
"""运行 LS-DYNA 计算.
Parameters
----------
input_file : str
输入文件路径
ncpu : int
CPU 核心数
"""
input_path = Path(input_file)
if not input_path.exists():
print(f"输入文件不存在: {input_path}")
return
cmd = get_ls_dyna_command(input_file, ncpu)
try:
subprocess.run(cmd, check=True)
print(f"LS-DYNA 计算完成: {input_file}")
except FileNotFoundError:
print("未找到 ls-dyna_mpp 命令")
except subprocess.CalledProcessError as e:
print(f"LS-DYNA 计算失败: {e}")
@px.register_fn
def run_ls_dyna_mpi(input_file: str, ncpu: int = DEFAULT_NCPU) -> None:
"""运行 LS-DYNA MPI 计算.
Parameters
----------
input_file : str
输入文件路径
ncpu : int
CPU 核心数
"""
input_path = Path(input_file)
if not input_path.exists():
print(f"输入文件不存在: {input_path}")
return
cmd = ["mpirun", "-np", str(ncpu), "ls-dyna_mpp", f"i={input_file}"]
try:
subprocess.run(cmd, check=True)
print(f"LS-DYNA MPI 计算完成: {input_file}")
except FileNotFoundError:
print("未找到 mpirun 或 ls-dyna_mpp 命令")
except subprocess.CalledProcessError as e:
print(f"LS-DYNA MPI 计算失败: {e}")
@px.register_fn
def check_ls_dyna_status() -> None:
"""检查 LS-DYNA 进程状态."""
try:
if Constants.IS_WINDOWS:
result = subprocess.run(
["tasklist", "/fi", "imagename eq ls-dyna_mpp.exe"],
capture_output=True,
text=True,
check=True,
)
print(result.stdout)
else:
result = subprocess.run(
["pgrep", "-f", "ls-dyna"],
capture_output=True,
text=True,
check=False,
)
if result.stdout.strip():
print(f"运行中的 LS-DYNA 进程 PID: {result.stdout.strip()}")
else:
print("没有运行中的 LS-DYNA 进程")
except subprocess.CalledProcessError as e:
print(f"检查进程状态失败: {e}")
# ============================================================================
# sshcopyid 函数
# ============================================================================
@px.register_fn
def ssh_copy_id(
hostname: str,
username: str,
password: str,
port: int = 22,
keypath: str = "~/.ssh/id_rsa.pub",
timeout: int = 30,
) -> None:
"""将 SSH 公钥部署到远程服务器.
Parameters
----------
hostname : str
远程服务器主机名或 IP 地址
username : str
远程服务器用户名
password : str
远程服务器密码
port : int
SSH 端口, 默认 22
keypath : str
公钥文件路径, 默认 ~/.ssh/id_rsa.pub
timeout : int
SSH 操作超时秒数, 默认 30
"""
pub_key_path = Path(keypath).expanduser()
if not pub_key_path.exists():
print(f"公钥文件不存在: {pub_key_path}")
sys.exit(1)
pub_key = pub_key_path.read_text().strip()
script = f"""mkdir -p ~/.ssh && chmod 700 ~/.ssh
cd ~/.ssh && touch authorized_keys && chmod 600 authorized_keys
grep -qF '{pub_key.split()[1]}' authorized_keys 2>/dev/null || echo '{pub_key}' >> authorized_keys"""
try:
subprocess.run(
[
"sshpass",
"-p",
password,
"ssh",
"-p",
str(port),
"-o",
"StrictHostKeyChecking=no",
"-o",
"UserKnownHostsFile=/dev/null",
"-o",
f"ConnectTimeout={timeout}",
f"{username}@{hostname}",
script,
],
check=True,
timeout=timeout,
)
print(f"SSH 密钥已部署到 {username}@{hostname}:{port}")
except FileNotFoundError:
print(f"未找到 sshpass 工具, 请手动执行: ssh-copy-id -p {port} {username}@{hostname}")
sys.exit(1)
except subprocess.TimeoutExpired:
print("SSH 连接超时")
sys.exit(1)
except subprocess.CalledProcessError as e:
print(f"SSH 执行失败: {e}")
sys.exit(1)
# ============================================================================
# packtool 函数
# ============================================================================
@px.register_fn
def pack_source(project_dir: Path, output_dir: Path) -> None:
"""打包项目源码.
Parameters
----------
project_dir : Path
项目目录
output_dir : Path
输出目录
"""
output_dir.mkdir(parents=True, exist_ok=True)
pyproject_file = project_dir / "pyproject.toml"
project_name = project_dir.name
if pyproject_file.exists():
try:
import tomllib
content = pyproject_file.read_text(encoding="utf-8")
data = tomllib.loads(content)
project_name = data.get("project", {}).get("name", project_name)
except ImportError:
pass
source_dir = output_dir / "src" / project_name
source_dir.mkdir(parents=True, exist_ok=True)
src_subdir = project_dir / "src"
if src_subdir.exists():
shutil.copytree(
src_subdir,
source_dir / "src",
ignore=shutil.ignore_patterns(*IGNORE_PATTERNS),
dirs_exist_ok=True,
)
else:
for item in project_dir.iterdir():
if item.name in IGNORE_PATTERNS or item.name.startswith("."):
continue
dst_item = source_dir / item.name
if item.is_dir():
shutil.copytree(
item,
dst_item,
ignore=shutil.ignore_patterns(*IGNORE_PATTERNS),
dirs_exist_ok=True,
)
else:
shutil.copy2(item, dst_item)
print(f"源码打包完成: {source_dir}")
@px.register_fn
def pack_dependencies(lib_dir: Path, dependencies: list[str]) -> None:
"""打包项目依赖.
Parameters
----------
lib_dir : Path
依赖库目录
dependencies : list[str]
依赖列表
"""
lib_dir.mkdir(parents=True, exist_ok=True)
if not dependencies:
print("没有依赖需要打包")
return
cmd = [
"pip",
"install",
"--target",
str(lib_dir),
"--no-compile",
"--no-warn-script-location",
]
cmd.extend(dependencies)
subprocess.run(cmd, check=True)
print(f"依赖打包完成: {lib_dir}")
@px.register_fn
def pack_wheel(project_dir: Path, output_dir: Path) -> None:
"""打包项目为 wheel 文件.
Parameters
----------
project_dir : Path
项目目录
output_dir : Path
输出目录
"""
output_dir.mkdir(parents=True, exist_ok=True)
cmd = [
"pip",
"wheel",
"--no-deps",
"--wheel-dir",
str(output_dir),
str(project_dir),
]
subprocess.run(cmd, check=True)
print(f"Wheel 打包完成: {output_dir}")
@px.register_fn
def install_embed_python(version: str, output_dir: Path) -> None:
"""安装嵌入式 Python.
Parameters
----------
version : str
Python 版本 (如: 3.10, 3.11)
output_dir : Path
输出目录
"""
output_dir.mkdir(parents=True, exist_ok=True)
arch = platform.machine().lower()
if arch in ["x86_64", "amd64"]:
arch = "amd64"
elif arch in ["arm64", "aarch64"]:
arch = "arm64"
version_map = {
"3.8": "3.8.10",
"3.9": "3.9.13",
"3.10": "3.10.11",
"3.11": "3.11.9",
"3.12": "3.12.4",
}
full_version = version_map.get(version, f"{version}.0")
url = f"https://www.python.org/ftp/python/{full_version}/python-{full_version}-embed-{arch}.zip"
cache_file = Path(DEFAULT_CACHE_DIR) / f"python-{full_version}-embed-{arch}.zip"
cache_file.parent.mkdir(parents=True, exist_ok=True)
if not cache_file.exists():
print(f"正在下载嵌入式 Python {full_version}...")
urllib.request.urlretrieve(url, cache_file)
print(f"下载完成: {cache_file}")
with zipfile.ZipFile(cache_file, "r") as zf:
zf.extractall(output_dir)
print(f"嵌入式 Python 安装完成: {output_dir}")
@px.register_fn
def create_zip_package(source_dir: Path, output_file: Path) -> None:
"""创建 ZIP 打包文件.
Parameters
----------
source_dir : Path
源目录
output_file : Path
输出文件
"""
output_file.parent.mkdir(parents=True, exist_ok=True)
with zipfile.ZipFile(output_file, "w", zipfile.ZIP_DEFLATED) as zf:
for file in source_dir.rglob("*"):
if file.is_file():
arcname = file.relative_to(source_dir)
zf.write(file, arcname)
print(f"ZIP 打包完成: {output_file}")
@px.register_fn
def clean_build_dir(build_dir: Path) -> None:
"""清理构建目录.
Parameters
----------
build_dir : Path
构建目录
"""
if build_dir.exists():
shutil.rmtree(build_dir)
print(f"清理完成: {build_dir}")
else:
print(f"目录不存在: {build_dir}")
# ============================================================================
# reseticoncache 函数
# ============================================================================
@px.register_fn
def reset_icon_cache_run() -> None:
"""重置 Windows 图标缓存.
执行流程: 杀掉 explorer → 删除 IconCache.db → 删除 iconcache* → 重启 explorer.
仅在 Windows 上执行, 非 Windows 平台打印提示并跳过.
"""
if not Constants.IS_WINDOWS:
print("reset_icon_cache: 仅在 Windows 上支持")
return
local_app_data = os.environ.get("LOCALAPPDATA", "")
if not local_app_data:
print("reset_icon_cache: LOCALAPPDATA 环境变量未设置")
return
icon_cache_db = Path(local_app_data) / "IconCache.db"
explorer_cache_dir = Path(local_app_data) / "Microsoft" / "Windows" / "Explorer"
print("正在终止 explorer 进程...")
subprocess.run(["taskkill", "/f", "/im", "explorer.exe"], check=False)
if icon_cache_db.exists():
print(f"删除图标缓存: {icon_cache_db}")
subprocess.run(["cmd", "/c", "del", "/a", "/q", str(icon_cache_db)], check=False)
if explorer_cache_dir.exists():
print(f"清理 Explorer 缓存: {explorer_cache_dir}")
subprocess.run(
["cmd", "/c", "del", "/a", "/q", str(explorer_cache_dir / "iconcache*")],
check=False,
)
print("重启 explorer...")
subprocess.run(["cmd", "/c", "start", "explorer.exe"], check=False)
print("图标缓存已重置")
+159
View File
@@ -0,0 +1,159 @@
"""函数注册表.
提供全局函数注册机制, 供 YAML 任务编排通过 ``fn`` 字段引用 Python 函数.
使用方式
--------
import pyflowx as px
@px.register_fn("pack_source")
def pack_source(project_dir, output_dir):
...
# YAML 中引用:
# jobs:
# pack:
# fn: pack_source
# args: ["./project", "./dist"]
"""
from __future__ import annotations
import sys
from typing import Any, Callable, TypeVar, overload
if sys.version_info >= (3, 10):
from typing import ParamSpec
else:
from typing_extensions import ParamSpec # pragma: no cover
__all__ = ["FnRegistry", "get_fn", "has_fn", "register_fn"]
P = ParamSpec("P")
T = TypeVar("T")
_REGISTRY: dict[str, Callable[..., Any]] = {}
@overload
def register_fn(name: Callable[P, T]) -> Callable[P, T]: ...
@overload
def register_fn(name: str | None = None) -> Callable[[Callable[P, T]], Callable[P, T]]: ...
def register_fn(name: str | Callable[..., Any] | None = None) -> Callable[..., Any]:
"""装饰器:将函数注册到全局 registry.
支持两种用法::
@register_fn # 使用函数 __name__ 作为注册名
def my_func(): ...
@register_fn("custom") # 显式指定注册名
def my_func(): ...
Parameters
----------
name : str | Callable | None
注册名或被装饰函数; 为 None 时使用函数 ``__name__``
Returns
-------
Callable
装饰器函数或被装饰函数
Raises
------
ValueError
名称已注册或无法推断函数名
"""
if callable(name):
fn = name
key = getattr(fn, "__name__", None)
if key is None:
raise ValueError("无法推断函数名, 请显式提供 name 参数")
if key in _REGISTRY:
raise ValueError(f"函数 {key!r} 已注册")
_REGISTRY[key] = fn
return fn
def decorator(fn: Callable[P, T]) -> Callable[P, T]:
key = name if name is not None else getattr(fn, "__name__", None)
if key is None:
raise ValueError("无法推断函数名, 请显式提供 name 参数")
if key in _REGISTRY:
raise ValueError(f"函数 {key!r} 已注册")
_REGISTRY[key] = fn
return fn
return decorator
def get_fn(name: str) -> Callable[..., Any]:
"""按名称获取已注册的函数.
Parameters
----------
name : str
函数名
Returns
-------
Callable
已注册的函数
Raises
------
KeyError
函数未注册
"""
if name not in _REGISTRY:
raise KeyError(f"函数 {name!r} 未注册")
return _REGISTRY[name]
def has_fn(name: str) -> bool:
"""检查函数是否已注册.
Parameters
----------
name : str
函数名
Returns
-------
bool
是否已注册
"""
return name in _REGISTRY
class FnRegistry:
"""函数注册表的面向对象访问接口."""
@staticmethod
def register(name: str | None = None) -> Callable[[Callable[..., T]], Callable[..., T]]:
"""注册装饰器, 等价于 :func:`register_fn`."""
return register_fn(name)
@staticmethod
def get(name: str) -> Callable[..., Any]:
"""获取已注册函数, 等价于 :func:`get_fn`."""
return get_fn(name)
@staticmethod
def has(name: str) -> bool:
"""检查是否已注册, 等价于 :func:`has_fn`."""
return has_fn(name)
@staticmethod
def clear() -> None:
"""清空注册表."""
_REGISTRY.clear()
@staticmethod
def names() -> list[str]:
"""返回所有已注册函数名."""
return list(_REGISTRY.keys())
+2 -39
View File
@@ -14,7 +14,7 @@ from __future__ import annotations
import argparse
import enum
import sys
from dataclasses import dataclass, field, replace
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any, Sequence, get_args
@@ -35,39 +35,6 @@ class CliExitCode(enum.IntEnum):
INTERRUPTED = 130 # 与 POSIX 信号中断一致
def _apply_verbose_to_graph(graph: Graph, verbose: bool) -> Graph:
"""创建新图, 其中所有 TaskSpec 的 verbose 字段被设置为指定值.
使用 ``dataclasses.replace`` 在不可变的 TaskSpec 上创建带 verbose 标记的副本.
依赖关系、标签等元数据全部保留.
Note
-----
自 ``_wrap_cmd`` 不再闭包捕获 ``verbose`` 后,此函数不再是必需的——
直接翻转 ``spec.verbose`` 即可生效。保留是为了向后兼容现有调用与测试。
TaskSpec 仍是 frozen dataclass,故仍用 ``replace`` 创建副本。
Parameters
----------
graph : Graph
原始图.
verbose : bool
要设置的 verbose 值.
Returns
-------
Graph
所有 spec 的 verbose 字段已更新的新图.
"""
new_specs: list[TaskSpec[Any]] = []
for spec in graph.all_specs().values():
if spec.verbose == verbose:
new_specs.append(spec)
else:
new_specs.append(replace(spec, verbose=verbose))
return Graph.from_specs(new_specs)
@dataclass
class CliRunner:
"""命令行运行器: 根据用户输入执行对应的任务流图.
@@ -296,12 +263,8 @@ class CliRunner:
# 确定是否 verbose: --quiet 覆盖默认值
verbose = self.verbose and not parsed.quiet
# 对图应用 verbose 设置 (重建带 verbose 标记的 spec)
# 执行对应的图 (verbose 标记由 run() 统一应用到各 spec)
graph = self.graphs[parsed.command]
if verbose:
graph = _apply_verbose_to_graph(graph, verbose=True)
# 执行对应的图
try:
report = run(
graph,
File diff suppressed because it is too large Load Diff
+20 -103
View File
@@ -5,8 +5,7 @@ from __future__ import annotations
from pathlib import Path
from unittest.mock import MagicMock, patch
import pyflowx as px
from pyflowx.cli import autofmt
from pyflowx.ops import dev
# ---------------------------------------------------------------------- #
@@ -19,14 +18,14 @@ class TestFormatWithRuff:
"""Should format with ruff."""
with patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(returncode=0)
autofmt.format_with_ruff(tmp_path, fix=True)
dev.format_with_ruff(tmp_path, fix=True)
assert mock_run.called
def test_format_with_ruff_no_fix(self, tmp_path: Path) -> None:
"""Should format with ruff without fix."""
with patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(returncode=0)
autofmt.format_with_ruff(tmp_path, fix=False)
dev.format_with_ruff(tmp_path, fix=False)
# Should not include --fix flag
call_args = mock_run.call_args[0][0]
assert "--fix" not in call_args
@@ -42,14 +41,14 @@ class TestLintWithRuff:
"""Should lint with ruff."""
with patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(returncode=0)
autofmt.lint_with_ruff(tmp_path, fix=True)
dev.lint_with_ruff(tmp_path, fix=True)
assert mock_run.called
def test_lint_with_ruff_no_fix(self, tmp_path: Path) -> None:
"""Should lint with ruff without fix."""
with patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(returncode=0)
autofmt.lint_with_ruff(tmp_path, fix=False)
dev.lint_with_ruff(tmp_path, fix=False)
# Should not include --fix flag
call_args = mock_run.call_args[0][0]
assert "--fix" not in call_args
@@ -66,7 +65,7 @@ class TestAddDocstring:
py_file = tmp_path / "test.py"
py_file.write_text("def test():\n pass\n")
result = autofmt.add_docstring(py_file, '"""Test module."""')
result = dev.add_docstring(py_file, '"""Test module."""')
assert result is True
def test_add_docstring_skips_files_with_docstring(self, tmp_path: Path) -> None:
@@ -74,7 +73,7 @@ class TestAddDocstring:
py_file = tmp_path / "test.py"
py_file.write_text('"""Existing docstring."""\ndef test():\n pass\n')
result = autofmt.add_docstring(py_file, '"""New docstring."""')
result = dev.add_docstring(py_file, '"""New docstring."""')
assert result is False
def test_add_docstring_empty_file(self, tmp_path: Path) -> None:
@@ -82,7 +81,7 @@ class TestAddDocstring:
py_file = tmp_path / "test.py"
py_file.write_text("")
result = autofmt.add_docstring(py_file, '"""Test module."""')
result = dev.add_docstring(py_file, '"""Test module."""')
# Should handle empty file
assert result is True
@@ -98,7 +97,7 @@ class TestGenerateModuleDocstring:
py_file = tmp_path / "test.py"
py_file.write_text("def test():\n pass\n")
result = autofmt.generate_module_docstring(py_file)
result = dev.generate_module_docstring(py_file)
# Should contain "Tests for" since stem contains "test"
assert "Tests for" in result
@@ -108,7 +107,7 @@ class TestGenerateModuleDocstring:
py_file.parent.mkdir(parents=True)
py_file.write_text("def test():\n pass\n")
result = autofmt.generate_module_docstring(py_file)
result = dev.generate_module_docstring(py_file)
assert "mypackage" in result
def test_generate_module_docstring_cli(self, tmp_path: Path) -> None:
@@ -116,7 +115,7 @@ class TestGenerateModuleDocstring:
py_file = tmp_path / "cli.py"
py_file.write_text("def test():\n pass\n")
result = autofmt.generate_module_docstring(py_file)
result = dev.generate_module_docstring(py_file)
assert "Command-line interface" in result
def test_generate_module_docstring_util(self, tmp_path: Path) -> None:
@@ -124,7 +123,7 @@ class TestGenerateModuleDocstring:
py_file = tmp_path / "utils.py"
py_file.write_text("def test():\n pass\n")
result = autofmt.generate_module_docstring(py_file)
result = dev.generate_module_docstring(py_file)
assert "Utility functions" in result
@@ -139,8 +138,8 @@ class TestAutoAddDocstrings:
py_file = tmp_path / "test.py"
py_file.write_text("def test():\n pass\n")
with patch.object(autofmt, "add_docstring", return_value=True):
count = autofmt.auto_add_docstrings(tmp_path)
with patch.object(dev, "add_docstring", return_value=True):
count = dev.auto_add_docstrings(tmp_path)
assert count >= 0
def test_auto_add_docstrings_skips_ignored(self, tmp_path: Path) -> None:
@@ -149,7 +148,7 @@ class TestAutoAddDocstrings:
py_file.parent.mkdir()
py_file.write_text("def test():\n pass\n")
count = autofmt.auto_add_docstrings(tmp_path)
count = dev.auto_add_docstrings(tmp_path)
# Should skip __pycache__
assert count == 0
@@ -158,7 +157,7 @@ class TestAutoAddDocstrings:
txt_file = tmp_path / "test.txt"
txt_file.write_text("test content")
count = autofmt.auto_add_docstrings(tmp_path)
count = dev.auto_add_docstrings(tmp_path)
assert count == 0
@@ -179,7 +178,7 @@ class TestSyncPyprojectConfig:
with patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(returncode=0)
autofmt.sync_pyproject_config(tmp_path)
dev.sync_pyproject_config(tmp_path)
assert mock_run.called
def test_sync_pyproject_config_updates_file(self, tmp_path: Path) -> None:
@@ -193,7 +192,7 @@ class TestSyncPyprojectConfig:
with patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(returncode=0)
autofmt.sync_pyproject_config(tmp_path)
dev.sync_pyproject_config(tmp_path)
assert mock_run.called
@@ -207,95 +206,13 @@ class TestFormatAll:
"""Should run ruff format."""
with patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(returncode=0)
autofmt.format_all(tmp_path)
dev.format_all(tmp_path)
assert mock_run.called
def test_format_all_runs_ruff_check(self, tmp_path: Path) -> None:
"""Should run ruff check."""
with patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(returncode=0)
autofmt.format_all(tmp_path)
dev.format_all(tmp_path)
# Should call ruff format and ruff check
assert mock_run.call_count == 2
# ---------------------------------------------------------------------- #
# main function
# ---------------------------------------------------------------------- #
class TestMain:
"""Test main function."""
def test_main_fmt_default_target(self) -> None:
"""main() should handle fmt command with default target."""
with patch("sys.argv", ["autofmt", "fmt"]), patch.object(px, "run") as mock_run:
autofmt.main()
assert mock_run.called
def test_main_fmt_custom_target(self) -> None:
"""main() should handle fmt command with custom target."""
with patch("sys.argv", ["autofmt", "fmt", "--target", "src"]), patch.object(px, "run") as mock_run:
autofmt.main()
assert mock_run.called
def test_main_lint_default_target(self) -> None:
"""main() should handle lint command with default target."""
with patch("sys.argv", ["autofmt", "lint"]), patch.object(px, "run") as mock_run:
autofmt.main()
assert mock_run.called
def test_main_lint_with_fix(self) -> None:
"""main() should handle lint command with fix."""
with patch("sys.argv", ["autofmt", "lint", "--fix"]), patch.object(px, "run") as mock_run:
autofmt.main()
assert mock_run.called
def test_main_lint_custom_target(self) -> None:
"""main() should handle lint command with custom target."""
with patch("sys.argv", ["autofmt", "lint", "--target", "src"]), patch.object(px, "run") as mock_run:
autofmt.main()
assert mock_run.called
def test_main_doc_default_root(self) -> None:
"""main() should handle doc command with default root."""
with patch("sys.argv", ["autofmt", "doc"]), patch.object(px, "run") as mock_run:
autofmt.main()
assert mock_run.called
def test_main_doc_custom_root(self) -> None:
"""main() should handle doc command with custom root."""
with patch("sys.argv", ["autofmt", "doc", "--root-dir", "src"]), patch.object(px, "run") as mock_run:
autofmt.main()
assert mock_run.called
def test_main_sync_default_root(self) -> None:
"""main() should handle sync command with default root."""
with patch("sys.argv", ["autofmt", "sync"]), patch.object(px, "run") as mock_run:
autofmt.main()
assert mock_run.called
def test_main_sync_custom_root(self) -> None:
"""main() should handle sync command with custom root."""
with patch("sys.argv", ["autofmt", "sync", "--root-dir", "."]), patch.object(px, "run") as mock_run:
autofmt.main()
assert mock_run.called
def test_main_with_no_args_shows_help(self) -> None:
"""main() with no args should show help."""
with patch("sys.argv", ["autofmt"]), patch.object(autofmt, "main"):
# Just call main, it should show help and return
autofmt.main()
# main() should return without calling px.run
assert True
def test_main_creates_task_specs_with_verbose(self) -> None:
"""main() should create TaskSpecs with verbose=True."""
with patch("sys.argv", ["autofmt", "fmt"]), patch.object(px, "run") as mock_run:
autofmt.main()
assert mock_run.called
def test_main_uses_thread_strategy(self) -> None:
"""main() should use thread strategy."""
with patch("sys.argv", ["autofmt", "fmt"]), patch.object(px, "run") as mock_run:
autofmt.main()
# Check that strategy="thread" was used
assert mock_run.called
+22 -72
View File
@@ -4,12 +4,10 @@ from __future__ import annotations
import os
from pathlib import Path
from unittest.mock import patch
import pytest
import pyflowx as px
from pyflowx.cli import bumpversion
from pyflowx.ops import dev
@pytest.fixture(autouse=True)
@@ -29,7 +27,7 @@ class TestBumpFileVersion:
test_file = tmp_path / "pyproject.toml"
test_file.write_text('version = "1.2.3"', encoding="utf-8")
result = bumpversion.bump_file_version(test_file, "patch")
result = dev.bump_file_version(test_file, "patch")
assert result == "1.2.4"
assert test_file.read_text(encoding="utf-8") == 'version = "1.2.4"'
@@ -39,7 +37,7 @@ class TestBumpFileVersion:
test_file = tmp_path / "pyproject.toml"
test_file.write_text('version = "1.2.3"', encoding="utf-8")
result = bumpversion.bump_file_version(test_file, "minor")
result = dev.bump_file_version(test_file, "minor")
assert result == "1.3.0"
assert test_file.read_text(encoding="utf-8") == 'version = "1.3.0"'
@@ -49,7 +47,7 @@ class TestBumpFileVersion:
test_file = tmp_path / "pyproject.toml"
test_file.write_text('version = "1.2.3"', encoding="utf-8")
result = bumpversion.bump_file_version(test_file, "major")
result = dev.bump_file_version(test_file, "major")
assert result == "2.0.0"
assert test_file.read_text(encoding="utf-8") == 'version = "2.0.0"'
@@ -59,7 +57,7 @@ class TestBumpFileVersion:
test_file = tmp_path / "pyproject.toml"
test_file.write_text('version = "1.2.3-alpha.1"', encoding="utf-8")
result = bumpversion.bump_file_version(test_file, "patch")
result = dev.bump_file_version(test_file, "patch")
assert result == "1.2.4"
# 预发布版本应该被清除
@@ -71,7 +69,7 @@ class TestBumpFileVersion:
test_file = tmp_path / "pyproject.toml"
test_file.write_text('version = "1.2.3+build.123"', encoding="utf-8")
result = bumpversion.bump_file_version(test_file, "patch")
result = dev.bump_file_version(test_file, "patch")
assert result == "1.2.4"
# 构建元数据应该被清除
@@ -83,7 +81,7 @@ class TestBumpFileVersion:
test_file = tmp_path / "test.txt"
test_file.write_text("no version here", encoding="utf-8")
result = bumpversion.bump_file_version(test_file, "patch")
result = dev.bump_file_version(test_file, "patch")
assert result is None
captured = capsys.readouterr()
@@ -94,7 +92,7 @@ class TestBumpFileVersion:
test_file = tmp_path / "__init__.py"
test_file.write_text('__version__ = "1.2.3"', encoding="utf-8")
result = bumpversion.bump_file_version(test_file, "patch")
result = dev.bump_file_version(test_file, "patch")
assert result == "1.2.4"
assert test_file.read_text(encoding="utf-8") == '__version__ = "1.2.4"'
@@ -110,7 +108,7 @@ description = "Test project"
"""
test_file.write_text(content, encoding="utf-8")
result = bumpversion.bump_file_version(test_file, "minor")
result = dev.bump_file_version(test_file, "minor")
assert result == "0.2.0"
updated = test_file.read_text(encoding="utf-8")
@@ -126,7 +124,7 @@ __version__ = "1.0.0"
'''
test_file.write_text(content, encoding="utf-8")
result = bumpversion.bump_file_version(test_file, "major")
result = dev.bump_file_version(test_file, "major")
assert result == "2.0.0"
updated = test_file.read_text(encoding="utf-8")
@@ -142,7 +140,7 @@ dependencies = ["lib >= 2.0.0", "other >= 3.0.0"]
"""
test_file.write_text(content, encoding="utf-8")
result = bumpversion.bump_file_version(test_file, "patch")
result = dev.bump_file_version(test_file, "patch")
assert result == "1.0.1"
updated = test_file.read_text(encoding="utf-8")
@@ -158,7 +156,7 @@ dependencies = ["lib >= 2.0.0", "other >= 3.0.0"]
test_file.mkdir()
with pytest.raises(Exception): # noqa: B017
bumpversion.bump_file_version(test_file, "patch")
dev.bump_file_version(test_file, "patch")
def test_file_write_error(self, tmp_path: Path, capsys: pytest.CaptureFixture[str]) -> None:
"""Should handle file write errors."""
@@ -173,7 +171,7 @@ dependencies = ["lib >= 2.0.0", "other >= 3.0.0"]
try:
with pytest.raises(Exception): # noqa: B017
bumpversion.bump_file_version(test_file, "patch")
dev.bump_file_version(test_file, "patch")
finally:
# 恢复权限以便清理
test_file.chmod(0o644)
@@ -190,7 +188,7 @@ class TestVersionPattern:
test_file = tmp_path / "__init__.py"
test_file.write_text('__version__ = "1.0.0"', encoding="utf-8")
result = bumpversion.bump_file_version(test_file, "patch")
result = dev.bump_file_version(test_file, "patch")
assert result == "1.0.1"
@@ -199,7 +197,7 @@ class TestVersionPattern:
test_file = tmp_path / "__init__.py"
test_file.write_text('__version__ = "0.0.0"', encoding="utf-8")
result = bumpversion.bump_file_version(test_file, "patch")
result = dev.bump_file_version(test_file, "patch")
assert result == "0.0.1"
@@ -208,7 +206,7 @@ class TestVersionPattern:
test_file = tmp_path / "__init__.py"
test_file.write_text('__version__ = "10.20.30"', encoding="utf-8")
result = bumpversion.bump_file_version(test_file, "minor")
result = dev.bump_file_version(test_file, "minor")
assert result == "10.21.0"
@@ -217,7 +215,7 @@ class TestVersionPattern:
test_file = tmp_path / "test.txt"
test_file.write_text("https://example.com/v1.2.3/download", encoding="utf-8")
result = bumpversion.bump_file_version(test_file, "patch")
result = dev.bump_file_version(test_file, "patch")
# 不应该匹配 URL 中的版本号
assert result is None
@@ -234,7 +232,7 @@ class TestEdgeCases:
test_file = tmp_path / "empty.txt"
test_file.write_text("", encoding="utf-8")
result = bumpversion.bump_file_version(test_file, "patch")
result = dev.bump_file_version(test_file, "patch")
assert result is None
captured = capsys.readouterr()
@@ -246,7 +244,7 @@ class TestEdgeCases:
content = '# 中文注释\n__version__ = "1.0.0"\n# 特殊字符: @#$%'
test_file.write_text(content, encoding="utf-8")
result = bumpversion.bump_file_version(test_file, "patch")
result = dev.bump_file_version(test_file, "patch")
assert result == "1.0.1"
updated = test_file.read_text(encoding="utf-8")
@@ -259,64 +257,16 @@ class TestEdgeCases:
test_file.write_text('__version__ = "1.0.0"', encoding="utf-8")
# 第一次 bump
result1 = bumpversion.bump_file_version(test_file, "patch")
result1 = dev.bump_file_version(test_file, "patch")
assert result1 == "1.0.1"
# 第二次 bump
result2 = bumpversion.bump_file_version(test_file, "minor")
result2 = dev.bump_file_version(test_file, "minor")
assert result2 == "1.1.0"
# 第三次 bump
result3 = bumpversion.bump_file_version(test_file, "major")
result3 = dev.bump_file_version(test_file, "major")
assert result3 == "2.0.0"
# 验证最终结果
assert test_file.read_text(encoding="utf-8") == '__version__ = "2.0.0"'
class TestBumpVersionCli:
"""Test bumpversion CLI."""
def test_minor(self, tmp_path: Path) -> None:
"""Should handle minor version bump."""
test_file = tmp_path / "__init__.py"
test_file.write_text('__version__ = "1.0.0"', encoding="utf-8")
# Mock px.run: 只真正执行第一次调用(版本更新),其余返回空 dict
with patch("sys.argv", ["bumpversion", "minor", "--no-tag"]), patch("pyflowx.run") as mock_run:
def run_side_effect(graph: px.Graph, strategy: str | None = None):
# 执行实际版本更新任务
results = {}
for spec in graph.specs.values():
if spec.fn is not None and spec.args:
results[spec.name] = spec.fn(*spec.args)
return results
mock_run.side_effect = run_side_effect
bumpversion.main()
# 验证版本号已更新
assert test_file.read_text(encoding="utf-8") == '__version__ = "1.1.0"'
def test_no_valid_files(self, tmp_path: Path, capsys: pytest.CaptureFixture[str]) -> None:
"""Should handle no valid files."""
test_file = tmp_path / "test.txt"
test_file.write_text("这是一个测试文件", encoding="utf-8")
with patch("sys.argv", ["bumpversion", "minor", "--no-tag"]), patch("pyflowx.run") as mock_run:
def run_side_effect(graph: px.Graph, strategy: str | None = None):
# 执行实际版本更新任务
results = {}
for spec in graph.specs.values():
if spec.fn is not None and spec.args:
results[spec.name] = spec.fn(*spec.args)
return results
mock_run.side_effect = run_side_effect
bumpversion.main()
# 验证未更新任何文件
assert test_file.read_text(encoding="utf-8") == "这是一个测试文件"
assert "未找到包含版本号的文件" in capsys.readouterr().out
+210 -182
View File
@@ -125,19 +125,21 @@ class TestEmailDatabase:
# Insert test emails
for i in range(5):
db.insert_email({
"file_path": f"/test/path{i}.eml",
"file_hash": f"hash{i}",
"subject": f"Subject {i}",
"sender": f"sender{i}@example.com",
"recipients": "recipient@example.com",
"date": f"Mon, {i + 1} Jan 2024 12:00:00 +0000",
"date_parsed": f"2024-01-0{i + 1}T12:00:00",
"body_text": f"Body {i}",
"body_html": f"<p>Body {i}</p>",
"has_attachments": 0,
"file_size": 1024,
})
db.insert_email(
{
"file_path": f"/test/path{i}.eml",
"file_hash": f"hash{i}",
"subject": f"Subject {i}",
"sender": f"sender{i}@example.com",
"recipients": "recipient@example.com",
"date": f"Mon, {i + 1} Jan 2024 12:00:00 +0000",
"date_parsed": f"2024-01-0{i + 1}T12:00:00",
"body_text": f"Body {i}",
"body_html": f"<p>Body {i}</p>",
"has_attachments": 0,
"file_size": 1024,
}
)
results = db.search_emails(limit=3)
assert len(results) == 3
@@ -148,33 +150,37 @@ class TestEmailDatabase:
db_path = tmp_path / "test.db"
db = emlmanager.EmailDatabase(db_path)
db.insert_email({
"file_path": "/test/path1.eml",
"file_hash": "hash1",
"subject": "Important Meeting",
"sender": "sender1@example.com",
"recipients": "recipient@example.com",
"date": "Mon, 1 Jan 2024 12:00:00 +0000",
"date_parsed": "2024-01-01T12:00:00",
"body_text": "Meeting body",
"body_html": "<p>Meeting body</p>",
"has_attachments": 0,
"file_size": 1024,
})
db.insert_email(
{
"file_path": "/test/path1.eml",
"file_hash": "hash1",
"subject": "Important Meeting",
"sender": "sender1@example.com",
"recipients": "recipient@example.com",
"date": "Mon, 1 Jan 2024 12:00:00 +0000",
"date_parsed": "2024-01-01T12:00:00",
"body_text": "Meeting body",
"body_html": "<p>Meeting body</p>",
"has_attachments": 0,
"file_size": 1024,
}
)
db.insert_email({
"file_path": "/test/path2.eml",
"file_hash": "hash2",
"subject": "Casual Chat",
"sender": "sender2@example.com",
"recipients": "recipient@example.com",
"date": "Tue, 2 Jan 2024 12:00:00 +0000",
"date_parsed": "2024-01-02T12:00:00",
"body_text": "Chat body",
"body_html": "<p>Chat body</p>",
"has_attachments": 0,
"file_size": 1024,
})
db.insert_email(
{
"file_path": "/test/path2.eml",
"file_hash": "hash2",
"subject": "Casual Chat",
"sender": "sender2@example.com",
"recipients": "recipient@example.com",
"date": "Tue, 2 Jan 2024 12:00:00 +0000",
"date_parsed": "2024-01-02T12:00:00",
"body_text": "Chat body",
"body_html": "<p>Chat body</p>",
"has_attachments": 0,
"file_size": 1024,
}
)
results = db.search_emails(keyword="Meeting", field="subject")
assert len(results) == 1
@@ -186,33 +192,37 @@ class TestEmailDatabase:
db_path = tmp_path / "test.db"
db = emlmanager.EmailDatabase(db_path)
db.insert_email({
"file_path": "/test/path1.eml",
"file_hash": "hash1",
"subject": "Test",
"sender": "alice@example.com",
"recipients": "recipient@example.com",
"date": "Mon, 1 Jan 2024 12:00:00 +0000",
"date_parsed": "2024-01-01T12:00:00",
"body_text": "Body",
"body_html": "<p>Body</p>",
"has_attachments": 0,
"file_size": 1024,
})
db.insert_email(
{
"file_path": "/test/path1.eml",
"file_hash": "hash1",
"subject": "Test",
"sender": "alice@example.com",
"recipients": "recipient@example.com",
"date": "Mon, 1 Jan 2024 12:00:00 +0000",
"date_parsed": "2024-01-01T12:00:00",
"body_text": "Body",
"body_html": "<p>Body</p>",
"has_attachments": 0,
"file_size": 1024,
}
)
db.insert_email({
"file_path": "/test/path2.eml",
"file_hash": "hash2",
"subject": "Test",
"sender": "bob@example.com",
"recipients": "recipient@example.com",
"date": "Tue, 2 Jan 2024 12:00:00 +0000",
"date_parsed": "2024-01-02T12:00:00",
"body_text": "Body",
"body_html": "<p>Body</p>",
"has_attachments": 0,
"file_size": 1024,
})
db.insert_email(
{
"file_path": "/test/path2.eml",
"file_hash": "hash2",
"subject": "Test",
"sender": "bob@example.com",
"recipients": "recipient@example.com",
"date": "Tue, 2 Jan 2024 12:00:00 +0000",
"date_parsed": "2024-01-02T12:00:00",
"body_text": "Body",
"body_html": "<p>Body</p>",
"has_attachments": 0,
"file_size": 1024,
}
)
results = db.search_emails(keyword="alice", field="sender")
assert len(results) == 1
@@ -224,19 +234,21 @@ class TestEmailDatabase:
db_path = tmp_path / "test.db"
db = emlmanager.EmailDatabase(db_path)
db.insert_email({
"file_path": "/test/path1.eml",
"file_hash": "hash1",
"subject": "Project Update",
"sender": "manager@example.com",
"recipients": "team@example.com",
"date": "Mon, 1 Jan 2024 12:00:00 +0000",
"date_parsed": "2024-01-01T12:00:00",
"body_text": "Please review the quarterly report",
"body_html": "<p>Please review the quarterly report</p>",
"has_attachments": 0,
"file_size": 1024,
})
db.insert_email(
{
"file_path": "/test/path1.eml",
"file_hash": "hash1",
"subject": "Project Update",
"sender": "manager@example.com",
"recipients": "team@example.com",
"date": "Mon, 1 Jan 2024 12:00:00 +0000",
"date_parsed": "2024-01-01T12:00:00",
"body_text": "Please review the quarterly report",
"body_html": "<p>Please review the quarterly report</p>",
"has_attachments": 0,
"file_size": 1024,
}
)
# Search for keyword in subject
results = db.search_emails(keyword="Project", field="all")
@@ -253,47 +265,53 @@ class TestEmailDatabase:
db = emlmanager.EmailDatabase(db_path)
# Insert emails with same subject (different prefixes)
db.insert_email({
"file_path": "/test/path1.eml",
"file_hash": "hash1",
"subject": "Meeting Tomorrow",
"sender": "sender1@example.com",
"recipients": "recipient@example.com",
"date": "Mon, 1 Jan 2024 12:00:00 +0000",
"date_parsed": "2024-01-01T12:00:00",
"body_text": "Body 1",
"body_html": "<p>Body 1</p>",
"has_attachments": 0,
"file_size": 1024,
})
db.insert_email(
{
"file_path": "/test/path1.eml",
"file_hash": "hash1",
"subject": "Meeting Tomorrow",
"sender": "sender1@example.com",
"recipients": "recipient@example.com",
"date": "Mon, 1 Jan 2024 12:00:00 +0000",
"date_parsed": "2024-01-01T12:00:00",
"body_text": "Body 1",
"body_html": "<p>Body 1</p>",
"has_attachments": 0,
"file_size": 1024,
}
)
db.insert_email({
"file_path": "/test/path2.eml",
"file_hash": "hash2",
"subject": "Re: Meeting Tomorrow",
"sender": "sender2@example.com",
"recipients": "recipient@example.com",
"date": "Tue, 2 Jan 2024 12:00:00 +0000",
"date_parsed": "2024-01-02T12:00:00",
"body_text": "Body 2",
"body_html": "<p>Body 2</p>",
"has_attachments": 0,
"file_size": 1024,
})
db.insert_email(
{
"file_path": "/test/path2.eml",
"file_hash": "hash2",
"subject": "Re: Meeting Tomorrow",
"sender": "sender2@example.com",
"recipients": "recipient@example.com",
"date": "Tue, 2 Jan 2024 12:00:00 +0000",
"date_parsed": "2024-01-02T12:00:00",
"body_text": "Body 2",
"body_html": "<p>Body 2</p>",
"has_attachments": 0,
"file_size": 1024,
}
)
db.insert_email({
"file_path": "/test/path3.eml",
"file_hash": "hash3",
"subject": "Different Topic",
"sender": "sender3@example.com",
"recipients": "recipient@example.com",
"date": "Wed, 3 Jan 2024 12:00:00 +0000",
"date_parsed": "2024-01-03T12:00:00",
"body_text": "Body 3",
"body_html": "<p>Body 3</p>",
"has_attachments": 0,
"file_size": 1024,
})
db.insert_email(
{
"file_path": "/test/path3.eml",
"file_hash": "hash3",
"subject": "Different Topic",
"sender": "sender3@example.com",
"recipients": "recipient@example.com",
"date": "Wed, 3 Jan 2024 12:00:00 +0000",
"date_parsed": "2024-01-03T12:00:00",
"body_text": "Body 3",
"body_html": "<p>Body 3</p>",
"has_attachments": 0,
"file_size": 1024,
}
)
grouped = db.get_grouped_emails()
# Should have 2 groups: "Meeting Tomorrow" and "Different Topic"
@@ -322,19 +340,21 @@ class TestEmailDatabase:
assert db.get_email_count() == 0
for i in range(3):
db.insert_email({
"file_path": f"/test/path{i}.eml",
"file_hash": f"hash{i}",
"subject": f"Subject {i}",
"sender": f"sender{i}@example.com",
"recipients": "recipient@example.com",
"date": f"Mon, {i + 1} Jan 2024 12:00:00 +0000",
"date_parsed": f"2024-01-0{i + 1}T12:00:00",
"body_text": f"Body {i}",
"body_html": f"<p>Body {i}</p>",
"has_attachments": 0,
"file_size": 1024,
})
db.insert_email(
{
"file_path": f"/test/path{i}.eml",
"file_hash": f"hash{i}",
"subject": f"Subject {i}",
"sender": f"sender{i}@example.com",
"recipients": "recipient@example.com",
"date": f"Mon, {i + 1} Jan 2024 12:00:00 +0000",
"date_parsed": f"2024-01-0{i + 1}T12:00:00",
"body_text": f"Body {i}",
"body_html": f"<p>Body {i}</p>",
"has_attachments": 0,
"file_size": 1024,
}
)
assert db.get_email_count() == 3
db.close()
@@ -346,19 +366,21 @@ class TestEmailDatabase:
# Insert some emails
for i in range(3):
db.insert_email({
"file_path": f"/test/path{i}.eml",
"file_hash": f"hash{i}",
"subject": f"Subject {i}",
"sender": f"sender{i}@example.com",
"recipients": "recipient@example.com",
"date": f"Mon, {i + 1} Jan 2024 12:00:00 +0000",
"date_parsed": f"2024-01-0{i + 1}T12:00:00",
"body_text": f"Body {i}",
"body_html": f"<p>Body {i}</p>",
"has_attachments": 0,
"file_size": 1024,
})
db.insert_email(
{
"file_path": f"/test/path{i}.eml",
"file_hash": f"hash{i}",
"subject": f"Subject {i}",
"sender": f"sender{i}@example.com",
"recipients": "recipient@example.com",
"date": f"Mon, {i + 1} Jan 2024 12:00:00 +0000",
"date_parsed": f"2024-01-0{i + 1}T12:00:00",
"body_text": f"Body {i}",
"body_html": f"<p>Body {i}</p>",
"has_attachments": 0,
"file_size": 1024,
}
)
assert db.get_email_count() == 3
@@ -687,19 +709,21 @@ class TestEmlManagerHandler:
# Insert some emails
for i in range(3):
db.insert_email({
"file_path": f"/test/path{i}.eml",
"file_hash": f"hash{i}",
"subject": f"Subject {i}",
"sender": f"sender{i}@example.com",
"recipients": "recipient@example.com",
"date": f"Mon, {i + 1} Jan 2024 12:00:00 +0000",
"date_parsed": f"2024-01-0{i + 1}T12:00:00",
"body_text": f"Body {i}",
"body_html": f"<p>Body {i}</p>",
"has_attachments": 0,
"file_size": 1024,
})
db.insert_email(
{
"file_path": f"/test/path{i}.eml",
"file_hash": f"hash{i}",
"subject": f"Subject {i}",
"sender": f"sender{i}@example.com",
"recipients": "recipient@example.com",
"date": f"Mon, {i + 1} Jan 2024 12:00:00 +0000",
"date_parsed": f"2024-01-0{i + 1}T12:00:00",
"body_text": f"Body {i}",
"body_html": f"<p>Body {i}</p>",
"has_attachments": 0,
"file_size": 1024,
}
)
# Create a mock handler instance without calling __init__
handler = Mock(spec=emlmanager.EmlManagerHandler)
@@ -721,19 +745,21 @@ class TestEmlManagerHandler:
db = emlmanager.EmailDatabase(db_path)
# Insert test email
db.insert_email({
"file_path": "/test/path.eml",
"file_hash": "hash",
"subject": "Test Subject",
"sender": "sender@example.com",
"recipients": "recipient@example.com",
"date": "Mon, 1 Jan 2024 12:00:00 +0000",
"date_parsed": "2024-01-01T12:00:00",
"body_text": "Test body",
"body_html": "<p>Test body</p>",
"has_attachments": 0,
"file_size": 1024,
})
db.insert_email(
{
"file_path": "/test/path.eml",
"file_hash": "hash",
"subject": "Test Subject",
"sender": "sender@example.com",
"recipients": "recipient@example.com",
"date": "Mon, 1 Jan 2024 12:00:00 +0000",
"date_parsed": "2024-01-01T12:00:00",
"body_text": "Test body",
"body_html": "<p>Test body</p>",
"has_attachments": 0,
"file_size": 1024,
}
)
# Create a mock handler instance without calling __init__
handler = Mock(spec=emlmanager.EmlManagerHandler)
@@ -756,19 +782,21 @@ class TestEmlManagerHandler:
db = emlmanager.EmailDatabase(db_path)
# Insert test email
db.insert_email({
"file_path": "/test/path.eml",
"file_hash": "hash",
"subject": "Test Subject",
"sender": "sender@example.com",
"recipients": "recipient@example.com",
"date": "Mon, 1 Jan 2024 12:00:00 +0000",
"date_parsed": "2024-01-01T12:00:00",
"body_text": "Test body",
"body_html": "<p>Test body</p>",
"has_attachments": 0,
"file_size": 1024,
})
db.insert_email(
{
"file_path": "/test/path.eml",
"file_hash": "hash",
"subject": "Test Subject",
"sender": "sender@example.com",
"recipients": "recipient@example.com",
"date": "Mon, 1 Jan 2024 12:00:00 +0000",
"date_parsed": "2024-01-01T12:00:00",
"body_text": "Test body",
"body_html": "<p>Test body</p>",
"has_attachments": 0,
"file_size": 1024,
}
)
assert db.get_email_count() == 1
+10 -43
View File
@@ -3,10 +3,8 @@
from __future__ import annotations
from pathlib import Path
from unittest.mock import patch
import pyflowx as px
from pyflowx.cli import filedate
from pyflowx.ops import files
# ---------------------------------------------------------------------- #
@@ -20,7 +18,7 @@ class TestGetFileTimestamp:
test_file = tmp_path / "test.txt"
test_file.write_text("test content")
timestamp = filedate.get_file_timestamp(test_file)
timestamp = files.get_file_timestamp(test_file)
assert len(timestamp) == 8 # YYYYMMDD format
assert timestamp.isdigit()
@@ -36,7 +34,7 @@ class TestRemoveDatePrefix:
test_file = tmp_path / "20240101_test.txt"
test_file.write_text("test content")
new_path = filedate.remove_date_prefix(test_file)
new_path = files.remove_date_prefix(test_file)
assert new_path.name == "test.txt"
def test_remove_date_prefix_without_date(self, tmp_path: Path) -> None:
@@ -44,7 +42,7 @@ class TestRemoveDatePrefix:
test_file = tmp_path / "test.txt"
test_file.write_text("test content")
new_path = filedate.remove_date_prefix(test_file)
new_path = files.remove_date_prefix(test_file)
assert new_path == test_file
@@ -59,7 +57,7 @@ class TestAddDatePrefix:
test_file = tmp_path / "test.txt"
test_file.write_text("test content")
new_path = filedate.add_date_prefix(test_file)
new_path = files.add_date_prefix(test_file)
assert new_path.name.startswith("20") # Starts with year
assert "_test.txt" in new_path.name
@@ -75,7 +73,7 @@ class TestProcessFileDate:
test_file = tmp_path / "test.txt"
test_file.write_text("test content")
filedate.process_file_date(test_file, clear=False)
files.process_file_date(test_file, clear=False)
# File should be renamed with date prefix
def test_process_file_date_clear(self, tmp_path: Path) -> None:
@@ -83,7 +81,7 @@ class TestProcessFileDate:
test_file = tmp_path / "20240101_test.txt"
test_file.write_text("test content")
filedate.process_file_date(test_file, clear=True)
files.process_file_date(test_file, clear=True)
# File should be renamed without date prefix
@@ -95,42 +93,11 @@ class TestProcessFilesDate:
def test_process_files_date_batch(self, tmp_path: Path) -> None:
"""Should process multiple files."""
files = []
file_list = []
for i in range(3):
test_file = tmp_path / f"test{i}.txt"
test_file.write_text(f"content{i}")
files.append(test_file)
file_list.append(test_file)
filedate.process_files_date(files, clear=False)
files.process_files_date(file_list, clear=False)
# All files should be processed
# ---------------------------------------------------------------------- #
# main function
# ---------------------------------------------------------------------- #
class TestMain:
"""Test main function."""
def test_main_add_command(self, tmp_path: Path) -> None:
"""main() should handle add command."""
test_file = tmp_path / "test.txt"
test_file.write_text("test content")
with patch("sys.argv", ["filedate", "add", str(test_file)]), patch.object(px, "run") as mock_run:
filedate.main()
assert mock_run.called
def test_main_clear_command(self, tmp_path: Path) -> None:
"""main() should handle clear command."""
test_file = tmp_path / "20240101_test.txt"
test_file.write_text("test content")
with patch("sys.argv", ["filedate", "clear", str(test_file)]), patch.object(px, "run") as mock_run:
filedate.main()
assert mock_run.called
def test_main_with_no_args_shows_help(self) -> None:
"""main() with no args should show help."""
with patch("sys.argv", ["filedate"]):
filedate.main()
# Should print help and return
+12 -49
View File
@@ -3,10 +3,8 @@
from __future__ import annotations
from pathlib import Path
from unittest.mock import patch
import pyflowx as px
from pyflowx.cli import filelevel
from pyflowx.ops import files
# ---------------------------------------------------------------------- #
@@ -18,19 +16,19 @@ class TestRemoveMarks:
def test_remove_marks_single_mark(self) -> None:
"""Should remove single mark."""
stem = "filename(PUB)"
result = filelevel.remove_marks(stem, ["PUB"])
result = files.remove_marks(stem, ["PUB"])
assert result == "filename"
def test_remove_marks_multiple_marks(self) -> None:
"""Should remove multiple marks."""
stem = "filename(PUB)(NOR)"
result = filelevel.remove_marks(stem, ["PUB", "NOR"])
result = files.remove_marks(stem, ["PUB", "NOR"])
assert result == "filename"
def test_remove_marks_no_marks(self) -> None:
"""Should not change stem without marks."""
stem = "filename"
result = filelevel.remove_marks(stem, ["PUB"])
result = files.remove_marks(stem, ["PUB"])
assert result == "filename"
@@ -45,7 +43,7 @@ class TestProcessFileLevel:
test_file = tmp_path / "test.txt"
test_file.write_text("test content")
filelevel.process_file_level(test_file, level=1)
files.process_file_level(test_file, level=1)
# File should be renamed with PUB level
def test_process_file_level_set_int(self, tmp_path: Path) -> None:
@@ -53,7 +51,7 @@ class TestProcessFileLevel:
test_file = tmp_path / "test.txt"
test_file.write_text("test content")
filelevel.process_file_level(test_file, level=2)
files.process_file_level(test_file, level=2)
# File should be renamed with INT level
def test_process_file_level_clear(self, tmp_path: Path) -> None:
@@ -61,7 +59,7 @@ class TestProcessFileLevel:
test_file = tmp_path / "test(PUB).txt"
test_file.write_text("test content")
filelevel.process_file_level(test_file, level=0)
files.process_file_level(test_file, level=0)
# File should be renamed without level
def test_process_file_level_invalid_level(self, tmp_path: Path) -> None:
@@ -69,14 +67,14 @@ class TestProcessFileLevel:
test_file = tmp_path / "test.txt"
test_file.write_text("test content")
filelevel.process_file_level(test_file, level=5)
files.process_file_level(test_file, level=5)
# Should print error message
def test_process_file_level_nonexistent_file(self, tmp_path: Path) -> None:
"""Should handle nonexistent file."""
test_file = tmp_path / "nonexistent.txt"
filelevel.process_file_level(test_file, level=1)
files.process_file_level(test_file, level=1)
# Should print error message
@@ -88,46 +86,11 @@ class TestProcessFilesLevel:
def test_process_files_level_batch(self, tmp_path: Path) -> None:
"""Should process multiple files."""
files = []
file_list = []
for i in range(3):
test_file = tmp_path / f"test{i}.txt"
test_file.write_text(f"content{i}")
files.append(test_file)
file_list.append(test_file)
filelevel.process_files_level(files, level=1)
files.process_files_level(file_list, level=1)
# All files should be processed
# ---------------------------------------------------------------------- #
# main function
# ---------------------------------------------------------------------- #
class TestMain:
"""Test main function."""
def test_main_set_command(self, tmp_path: Path) -> None:
"""main() should handle set command."""
test_file = tmp_path / "test.txt"
test_file.write_text("test content")
with patch("sys.argv", ["filelevel", "set", str(test_file), "--level", "1"]), patch.object(
px, "run"
) as mock_run:
filelevel.main()
assert mock_run.called
def test_main_set_command_level_2(self, tmp_path: Path) -> None:
"""main() should handle set command with level 2."""
test_file = tmp_path / "test.txt"
test_file.write_text("test content")
with patch("sys.argv", ["filelevel", "set", str(test_file), "--level", "2"]), patch.object(
px, "run"
) as mock_run:
filelevel.main()
assert mock_run.called
def test_main_with_no_args_shows_help(self) -> None:
"""main() with no args should show help."""
with patch("sys.argv", ["filelevel"]):
filelevel.main()
# Should print help and return
+19 -33
View File
@@ -5,8 +5,7 @@ from __future__ import annotations
from pathlib import Path
from unittest.mock import patch
import pyflowx as px
from pyflowx.cli import folderback
from pyflowx.ops import files
# ---------------------------------------------------------------------- #
@@ -22,7 +21,7 @@ class TestRemoveDump:
dst = tmp_path / "backup"
dst.mkdir()
folderback.remove_dump(src, dst, 5)
files.remove_dump(src, dst, 5)
# Should not raise error
def test_remove_dump_within_limit(self, tmp_path: Path) -> None:
@@ -37,7 +36,7 @@ class TestRemoveDump:
zip_file = dst / f"source_20240101_12000{i}.zip"
zip_file.write_bytes(b"ZIP content")
folderback.remove_dump(src, dst, 5)
files.remove_dump(src, dst, 5)
# All files should remain
assert len(list(dst.glob("*.zip"))) == 3
@@ -53,7 +52,7 @@ class TestRemoveDump:
zip_file = dst / f"source_20240101_12000{i}.zip"
zip_file.write_bytes(b"ZIP content")
folderback.remove_dump(src, dst, 5)
files.remove_dump(src, dst, 5)
# Should have only 5 files
assert len(list(dst.glob("*.zip"))) == 5
@@ -73,7 +72,7 @@ class TestZipTarget:
dst.mkdir()
with patch("time.strftime", return_value="_20240101_120000"):
folderback.zip_target(src, dst, 5)
files.zip_target(src, dst, 5)
# Should create zip file
zip_files = list(dst.glob("*.zip"))
@@ -91,7 +90,7 @@ class TestZipTarget:
dst.mkdir()
with patch("time.strftime", return_value="_20240101_120000"):
folderback.zip_target(src, dst, 5)
files.zip_target(src, dst, 5)
# Should create zip file
zip_files = list(dst.glob("*.zip"))
@@ -111,8 +110,8 @@ class TestBackupFolder:
(source_dir / "test.txt").write_text("test content")
backup_dir = tmp_path / "backup"
with patch.object(folderback, "zip_target") as mock_zip:
folderback.backup_folder(str(source_dir), str(backup_dir), 5)
with patch.object(files, "zip_target") as mock_zip:
files.backup_folder(str(source_dir), str(backup_dir), 5)
assert mock_zip.called
def test_backup_folder_with_max_backups(self, tmp_path: Path) -> None:
@@ -122,8 +121,8 @@ class TestBackupFolder:
(source_dir / "test.txt").write_text("test content")
backup_dir = tmp_path / "backup"
with patch.object(folderback, "zip_target") as mock_zip:
folderback.backup_folder(str(source_dir), str(backup_dir), 10)
with patch.object(files, "zip_target") as mock_zip:
files.backup_folder(str(source_dir), str(backup_dir), 10)
assert mock_zip.called
def test_backup_folder_source_not_exists(self, tmp_path: Path) -> None:
@@ -132,7 +131,7 @@ class TestBackupFolder:
backup_dir = tmp_path / "backup"
backup_dir.mkdir()
folderback.backup_folder(str(source_dir), str(backup_dir), 5)
files.backup_folder(str(source_dir), str(backup_dir), 5)
# Should print error message and return
def test_backup_folder_creates_dst(self, tmp_path: Path) -> None:
@@ -142,32 +141,19 @@ class TestBackupFolder:
(source_dir / "test.txt").write_text("test content")
backup_dir = tmp_path / "backup"
with patch.object(folderback, "zip_target") as mock_zip:
folderback.backup_folder(str(source_dir), str(backup_dir), 5)
with patch.object(files, "zip_target") as mock_zip:
files.backup_folder(str(source_dir), str(backup_dir), 5)
assert backup_dir.exists()
assert mock_zip.called
# ---------------------------------------------------------------------- #
# TaskSpec definitions
# 函数注册
# ---------------------------------------------------------------------- #
class TestTaskSpecDefinitions:
"""Test that all TaskSpec definitions are valid."""
class TestRegisteredFunctions:
"""Test that folderback functions are registered."""
def test_folderback_default_spec(self) -> None:
"""folderback_default spec should be properly defined."""
assert folderback.folderback_default.name == "folderback_default"
assert folderback.folderback_default.fn is not None
# ---------------------------------------------------------------------- #
# main function
# ---------------------------------------------------------------------- #
class TestMain:
"""Test main function."""
def test_main_calls_run_cli(self) -> None:
"""main() should create a CliRunner and call run_cli()."""
with patch.object(px.CliRunner, "run_cli") as mock_run_cli:
folderback.main()
assert mock_run_cli.called
"""folderback_default should be a registered callable."""
# folderback_default 现在是通过 @px.register_fn 注册的普通函数, 不是 TaskSpec
assert callable(files.folderback_default)
+11 -25
View File
@@ -5,8 +5,7 @@ from __future__ import annotations
from pathlib import Path
from unittest.mock import patch
import pyflowx as px
from pyflowx.cli import folderzip
from pyflowx.ops import files
# ---------------------------------------------------------------------- #
@@ -22,7 +21,7 @@ class TestArchiveFolder:
(folder / "test.txt").write_text("test content")
with patch("shutil.make_archive") as mock_archive:
folderzip.archive_folder(folder)
files.archive_folder(folder)
assert mock_archive.called
@@ -39,37 +38,24 @@ class TestZipFolders:
(tmp_path / "folder2").mkdir()
(tmp_path / ".git").mkdir() # Should be ignored
with patch.object(folderzip, "archive_folder") as mock_archive:
folderzip.zip_folders(str(tmp_path))
with patch.object(files, "archive_folder") as mock_archive:
files.zip_folders(str(tmp_path))
# Should archive folder1 and folder2, but not .git
assert mock_archive.call_count == 2
def test_zip_folders_nonexistent_cwd(self, tmp_path: Path) -> None:
"""Should handle nonexistent cwd."""
folderzip.zip_folders(str(tmp_path / "nonexistent"))
files.zip_folders(str(tmp_path / "nonexistent"))
# Should print error message and return
# ---------------------------------------------------------------------- #
# TaskSpec definitions
# 函数注册
# ---------------------------------------------------------------------- #
class TestTaskSpecDefinitions:
"""Test that all TaskSpec definitions are valid."""
class TestRegisteredFunctions:
"""Test that folderzip functions are registered."""
def test_folderzip_default_spec(self) -> None:
"""folderzip_default spec should be properly defined."""
assert folderzip.folderzip_default.name == "folderzip_default"
assert folderzip.folderzip_default.fn is not None
# ---------------------------------------------------------------------- #
# main function
# ---------------------------------------------------------------------- #
class TestMain:
"""Test main function."""
def test_main_calls_run_cli(self) -> None:
"""main() should create a CliRunner and call run_cli()."""
with patch.object(px.CliRunner, "run_cli") as mock_run_cli:
folderzip.main()
assert mock_run_cli.called
"""folderzip_default should be a registered callable."""
# folderzip_default 现在是通过 @px.register_fn 注册的普通函数, 不是 TaskSpec
assert callable(files.folderzip_default)
+23 -66
View File
@@ -8,7 +8,7 @@ from unittest.mock import patch
import pytest
import pyflowx as px
from pyflowx.cli import gittool
from pyflowx.ops import dev
# ---------------------------------------------------------------------- #
@@ -20,7 +20,7 @@ class TestNotHasGitRepo:
def test_not_has_git_repo_true(self, tmp_path: Path) -> None:
"""Should return True when no .git directory."""
with patch.object(Path, "cwd", return_value=tmp_path):
result = gittool.not_has_git_repo()
result = dev.not_has_git_repo()
assert result is True
def test_not_has_git_repo_false(self, tmp_path: Path) -> None:
@@ -29,7 +29,7 @@ class TestNotHasGitRepo:
git_dir.mkdir()
with patch.object(Path, "cwd", return_value=tmp_path):
result = gittool.not_has_git_repo()
result = dev.not_has_git_repo()
assert result is False
def test_not_has_git_repo_cwd_not_exists(self, tmp_path: Path) -> None:
@@ -37,7 +37,7 @@ class TestNotHasGitRepo:
nonexistent = tmp_path / "nonexistent"
with patch.object(Path, "cwd", return_value=nonexistent):
result = gittool.not_has_git_repo()
result = dev.not_has_git_repo()
assert result is True
@@ -47,19 +47,25 @@ class TestNotHasGitRepo:
class TestHasFiles:
"""Test has_files function."""
def test_has_files_true(self, tmp_path: Path) -> None:
"""Should return True when files exist."""
(tmp_path / "test.txt").write_text("test")
def test_has_files_true(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""Should return True when there are uncommitted changes."""
with patch.object(Path, "cwd", return_value=tmp_path):
result = gittool.has_files()
assert result is True
class _FakeResult:
stdout = " M test.txt\n"
def test_has_files_false(self, tmp_path: Path) -> None:
"""Should return False when no files."""
with patch.object(Path, "cwd", return_value=tmp_path):
result = gittool.has_files()
assert result is False
monkeypatch.setattr("subprocess.run", lambda *_, **__: _FakeResult())
result = dev.has_files()
assert result is True
def test_has_files_false(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""Should return False when no uncommitted changes."""
class _FakeResult:
stdout = ""
monkeypatch.setattr("subprocess.run", lambda *_, **__: _FakeResult())
result = dev.has_files()
assert result is False
# ---------------------------------------------------------------------- #
@@ -76,62 +82,13 @@ class TestInitSubDirs:
subdir2.mkdir()
with patch.object(Path, "cwd", return_value=tmp_path), patch.object(px, "run") as mock_run:
gittool.init_sub_dirs()
dev.init_sub_dirs()
# Should call px.run for each subdirectory
assert mock_run.call_count == 2
def test_init_sub_dirs_no_subdirectories(self, tmp_path: Path) -> None:
"""Should handle no subdirectories."""
with patch.object(Path, "cwd", return_value=tmp_path), patch.object(px, "run") as mock_run:
gittool.init_sub_dirs()
dev.init_sub_dirs()
# Should not call px.run
assert mock_run.call_count == 0
# ---------------------------------------------------------------------- #
# TaskSpec definitions
# ---------------------------------------------------------------------- #
class TestTaskSpecDefinitions:
"""Test that all TaskSpec definitions are valid."""
def test_push_spec(self) -> None:
"""push spec should be properly defined."""
assert gittool.push.name == "push"
assert gittool.push.cmd == ["git", "push"]
def test_pull_spec(self) -> None:
"""pull spec should be properly defined."""
assert gittool.pull.name == "pull"
assert gittool.pull.cmd == ["git", "pull"]
def test_kill_tgit_spec(self) -> None:
"""kill_tgit spec should be properly defined."""
assert gittool.kill_tgit.name == "task_kill"
assert isinstance(gittool.kill_tgit.cmd, list)
assert "taskkill" in gittool.kill_tgit.cmd
# ---------------------------------------------------------------------- #
# main function
# ---------------------------------------------------------------------- #
class TestMain:
"""Test main function."""
def test_main_calls_run_cli(self) -> None:
"""main() should create a CliRunner and call run_cli()."""
with pytest.raises(SystemExit) as exc_info:
gittool.main()
# run_cli() calls sys.exit(), so we should get SystemExit
assert exc_info.value.code in (0, 1, 2)
def test_main_with_list_argument(self) -> None:
"""main() should handle --list argument."""
with patch("sys.argv", ["gittool", "--list"]), pytest.raises(SystemExit) as exc_info:
gittool.main()
assert exc_info.value.code == 0
def test_main_with_no_args_shows_help(self) -> None:
"""main() with no args should show help and exit."""
with patch("sys.argv", ["gittool"]), pytest.raises(SystemExit) as exc_info:
gittool.main()
assert exc_info.value.code == 1
+10 -57
View File
@@ -5,9 +5,8 @@ from __future__ import annotations
from pathlib import Path
from unittest.mock import MagicMock, patch
import pyflowx as px
from pyflowx.cli import lscalc
from pyflowx.conditions import Constants
from pyflowx.ops import system
# ---------------------------------------------------------------------- #
@@ -19,7 +18,7 @@ class TestGetLsDynaCommand:
def test_get_ls_dyna_command_windows(self) -> None:
"""Should get LS-DYNA command for Windows."""
with patch.object(Constants, "IS_WINDOWS", True), patch.object(Constants, "IS_MACOS", False):
cmd = lscalc.get_ls_dyna_command("input.k", 4)
cmd = system.get_ls_dyna_command("input.k", 4)
assert "ls-dyna_mpp" in cmd
assert "i=input.k" in cmd
assert "ncpu=4" in cmd
@@ -27,7 +26,7 @@ class TestGetLsDynaCommand:
def test_get_ls_dyna_command_linux(self) -> None:
"""Should get LS-DYNA command for Linux."""
with patch.object(Constants, "IS_WINDOWS", False), patch.object(Constants, "IS_MACOS", False):
cmd = lscalc.get_ls_dyna_command("input.k", 8)
cmd = system.get_ls_dyna_command("input.k", 8)
assert "ls-dyna_mpp" in cmd
assert "i=input.k" in cmd
assert "ncpu=8" in cmd
@@ -46,14 +45,14 @@ class TestRunLsDyna:
with patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(returncode=0)
lscalc.run_ls_dyna(str(input_file), ncpu=4)
system.run_ls_dyna(str(input_file), ncpu=4)
assert mock_run.called
def test_run_ls_dyna_file_not_found(self, tmp_path: Path) -> None:
"""Should handle nonexistent input file."""
input_file = tmp_path / "nonexistent.k"
lscalc.run_ls_dyna(str(input_file), ncpu=4)
system.run_ls_dyna(str(input_file), ncpu=4)
# Should print error message
def test_run_ls_dyna_command_not_found(self, tmp_path: Path) -> None:
@@ -62,7 +61,7 @@ class TestRunLsDyna:
input_file.write_text("LS-DYNA input")
with patch("subprocess.run", side_effect=FileNotFoundError):
lscalc.run_ls_dyna(str(input_file), ncpu=4)
system.run_ls_dyna(str(input_file), ncpu=4)
# Should print error message
@@ -79,14 +78,14 @@ class TestRunLsDynaMpi:
with patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(returncode=0)
lscalc.run_ls_dyna_mpi(str(input_file), ncpu=8)
system.run_ls_dyna_mpi(str(input_file), ncpu=8)
assert mock_run.called
def test_run_ls_dyna_mpi_file_not_found(self, tmp_path: Path) -> None:
"""Should handle nonexistent input file."""
input_file = tmp_path / "nonexistent.k"
lscalc.run_ls_dyna_mpi(str(input_file), ncpu=8)
system.run_ls_dyna_mpi(str(input_file), ncpu=8)
# Should print error message
@@ -100,58 +99,12 @@ class TestCheckLsDynaStatus:
"""Should check LS-DYNA status on Windows."""
with patch.object(Constants, "IS_WINDOWS", True), patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(stdout="ls-dyna_mpp.exe", returncode=0)
lscalc.check_ls_dyna_status()
system.check_ls_dyna_status()
assert mock_run.called
def test_check_ls_dyna_status_linux(self) -> None:
"""Should check LS-DYNA status on Linux."""
with patch.object(Constants, "IS_WINDOWS", False), patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(stdout="1234", returncode=0)
lscalc.check_ls_dyna_status()
system.check_ls_dyna_status()
assert mock_run.called
# ---------------------------------------------------------------------- #
# main function
# ---------------------------------------------------------------------- #
class TestMain:
"""Test main function."""
def test_main_run_command(self, tmp_path: Path) -> None:
"""main() should handle run command."""
input_file = tmp_path / "input.k"
input_file.write_text("LS-DYNA input")
with patch("sys.argv", ["lscalc", "run", str(input_file)]), patch.object(px, "run") as mock_run:
lscalc.main()
assert mock_run.called
def test_main_run_command_with_ncpu(self, tmp_path: Path) -> None:
"""main() should handle run command with ncpu."""
input_file = tmp_path / "input.k"
input_file.write_text("LS-DYNA input")
with patch("sys.argv", ["lscalc", "run", str(input_file), "--ncpu", "8"]), patch.object(px, "run") as mock_run:
lscalc.main()
assert mock_run.called
def test_main_mpi_command(self, tmp_path: Path) -> None:
"""main() should handle mpi command."""
input_file = tmp_path / "input.k"
input_file.write_text("LS-DYNA input")
with patch("sys.argv", ["lscalc", "mpi", str(input_file)]), patch.object(px, "run") as mock_run:
lscalc.main()
assert mock_run.called
def test_main_status_command(self) -> None:
"""main() should handle status command."""
with patch("sys.argv", ["lscalc", "status"]), patch.object(px, "run") as mock_run:
lscalc.main()
assert mock_run.called
def test_main_with_no_args_shows_help(self) -> None:
"""main() with no args should show help."""
with patch("sys.argv", ["lscalc"]):
lscalc.main()
# Should print help and return
+16 -81
View File
@@ -7,8 +7,7 @@ from unittest.mock import MagicMock, patch
import pytest
import pyflowx as px
from pyflowx.cli import packtool
from pyflowx.ops import system
@pytest.fixture(autouse=True)
@@ -20,7 +19,7 @@ def packtool_tmp_workdir(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Non
monkeypatch: pytest monkeypatch 工具
"""
# Mock DEFAULT_CACHE_DIR 到临时目录
monkeypatch.setattr(packtool, "DEFAULT_CACHE_DIR", str(tmp_path / ".cache" / "pypack"))
monkeypatch.setattr(system, "DEFAULT_CACHE_DIR", str(tmp_path / ".cache" / "pypack"))
# ---------------------------------------------------------------------- #
@@ -36,7 +35,7 @@ class TestPackSource:
(project_dir / "main.py").write_text("print('hello')")
output_dir = tmp_path / "output"
packtool.pack_source(project_dir, output_dir)
system.pack_source(project_dir, output_dir)
assert output_dir.exists()
def test_pack_source_with_pyproject(self, tmp_path: Path) -> None:
@@ -47,7 +46,7 @@ class TestPackSource:
(project_dir / "main.py").write_text("print('hello')")
output_dir = tmp_path / "output"
packtool.pack_source(project_dir, output_dir)
system.pack_source(project_dir, output_dir)
assert output_dir.exists()
@@ -61,7 +60,7 @@ class TestPackDependencies:
"""Should handle empty dependencies."""
lib_dir = tmp_path / "libs"
packtool.pack_dependencies(lib_dir, [])
system.pack_dependencies(lib_dir, [])
# Should print message and return
def test_pack_dependencies_with_deps(self, tmp_path: Path) -> None:
@@ -70,7 +69,7 @@ class TestPackDependencies:
with patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(returncode=0)
packtool.pack_dependencies(lib_dir, ["numpy", "pandas"])
system.pack_dependencies(lib_dir, ["numpy", "pandas"])
assert mock_run.called
@@ -89,7 +88,7 @@ class TestPackWheel:
with patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(returncode=0)
packtool.pack_wheel(project_dir, output_dir)
system.pack_wheel(project_dir, output_dir)
assert mock_run.called
@@ -112,7 +111,7 @@ class TestInstallEmbedPython:
mock_zip_instance = MagicMock()
mock_zipfile.return_value.__enter__.return_value = mock_zip_instance
packtool.install_embed_python("3.10", output_dir)
system.install_embed_python("3.10", output_dir)
# Verify download was called
assert mock_urlretrieve.called
@@ -135,7 +134,7 @@ class TestInstallEmbedPython:
mock_zip_instance = MagicMock()
mock_zipfile.return_value.__enter__.return_value = mock_zip_instance
packtool.install_embed_python("3.10", output_dir)
system.install_embed_python("3.10", output_dir)
# Verify extraction was called (using cache)
assert mock_zip_instance.extractall.called
@@ -158,7 +157,7 @@ class TestInstallEmbedPython:
return
# Perform real installation
packtool.install_embed_python("3.10", output_dir)
system.install_embed_python("3.10", output_dir)
# Verify installation succeeded
assert output_dir.exists()
@@ -199,7 +198,7 @@ class TestInstallEmbedPython:
# Test different versions
for version in ["3.8", "3.9", "3.10", "3.11", "3.12"]:
packtool.install_embed_python(version, output_dir)
system.install_embed_python(version, output_dir)
assert mock_urlretrieve.called
def test_install_embed_python_creates_cache(self, tmp_path: Path) -> None:
@@ -213,10 +212,10 @@ class TestInstallEmbedPython:
mock_zip_instance = MagicMock()
mock_zipfile.return_value.__enter__.return_value = mock_zip_instance
packtool.install_embed_python("3.10", output_dir)
system.install_embed_python("3.10", output_dir)
# Verify cache directory was created (now in tmp_path)
Path(packtool.DEFAULT_CACHE_DIR)
Path(system.DEFAULT_CACHE_DIR)
# Note: In test environment, cache might not persist due to mocking
@@ -233,7 +232,7 @@ class TestCreateZipPackage:
(source_dir / "test.txt").write_text("test content")
output_file = tmp_path / "package.zip"
packtool.create_zip_package(source_dir, output_file)
system.create_zip_package(source_dir, output_file)
assert output_file.exists()
@@ -249,76 +248,12 @@ class TestCleanBuildDir:
build_dir.mkdir()
(build_dir / "test.txt").write_text("test")
packtool.clean_build_dir(build_dir)
system.clean_build_dir(build_dir)
assert not build_dir.exists()
def test_clean_build_dir_not_exists(self, tmp_path: Path) -> None:
"""Should handle nonexistent build directory."""
build_dir = tmp_path / "nonexistent"
packtool.clean_build_dir(build_dir)
system.clean_build_dir(build_dir)
# Should print message
# ---------------------------------------------------------------------- #
# main function
# ---------------------------------------------------------------------- #
class TestMain:
"""Test main function."""
def test_main_src_command(self, tmp_path: Path) -> None:
"""main() should handle src command."""
project_dir = tmp_path / "project"
project_dir.mkdir()
with patch("sys.argv", ["packtool", "src", "--project-dir", str(project_dir)]), patch.object(
px, "run"
) as mock_run:
packtool.main()
assert mock_run.called
def test_main_deps_command(self, tmp_path: Path) -> None:
"""main() should handle deps command."""
with patch("sys.argv", ["packtool", "deps", "numpy", "pandas"]), patch.object(px, "run") as mock_run:
packtool.main()
assert mock_run.called
def test_main_wheel_command(self, tmp_path: Path) -> None:
"""main() should handle wheel command."""
project_dir = tmp_path / "project"
project_dir.mkdir()
with patch("sys.argv", ["packtool", "wheel", "--project-dir", str(project_dir)]), patch.object(
px, "run"
) as mock_run:
packtool.main()
assert mock_run.called
def test_main_embed_command(self, tmp_path: Path) -> None:
"""main() should handle embed command."""
with patch("sys.argv", ["packtool", "embed", "--version", "3.10"]), patch.object(px, "run") as mock_run:
packtool.main()
assert mock_run.called
def test_main_zip_command(self, tmp_path: Path) -> None:
"""main() should handle zip command."""
source_dir = tmp_path / "source"
source_dir.mkdir()
with patch("sys.argv", ["packtool", "zip", "--source-dir", str(source_dir)]), patch.object(
px, "run"
) as mock_run:
packtool.main()
assert mock_run.called
def test_main_clean_command(self) -> None:
"""main() should handle clean command."""
with patch("sys.argv", ["packtool", "clean"]), patch.object(px, "run") as mock_run:
packtool.main()
assert mock_run.called
def test_main_with_no_args_shows_help(self) -> None:
"""main() with no args should show help."""
with patch("sys.argv", ["packtool"]):
packtool.main()
# Should print help and return
+13 -57
View File
@@ -8,8 +8,7 @@ from unittest.mock import MagicMock, patch
import pytest
import pyflowx as px
from pyflowx.cli import pdftool
from pyflowx.ops import media
# ---------------------------------------------------------------------- #
@@ -29,7 +28,7 @@ class TestPdfMerge:
with patch("pypdf.PdfReader"), patch("pypdf.PdfWriter") as mock_writer:
mock_writer_instance = MagicMock()
mock_writer.return_value = mock_writer_instance
pdftool.pdf_merge(input_files, output_file)
media.pdf_merge(input_files, output_file)
assert mock_writer_instance.write.called
@@ -50,7 +49,7 @@ class TestPdfSplit:
mock_reader_instance = MagicMock()
mock_reader.return_value = mock_reader_instance
mock_reader_instance.pages = [MagicMock()]
pdftool.pdf_split(input_file, output_dir)
media.pdf_split(input_file, output_dir)
assert output_dir.exists()
@@ -76,7 +75,7 @@ class TestPdfCompress:
output_file.write_bytes(b"Compressed PDF")
mock_doc.save = mock_save
pdftool.pdf_compress(input_file, output_file)
media.pdf_compress(input_file, output_file)
assert output_file.exists()
@@ -99,7 +98,7 @@ class TestPdfExtractText:
mock_page.get_text.return_value = "Test text"
mock_doc.__iter__ = MagicMock(return_value=iter([mock_page]))
mock_fitz_open.return_value = mock_doc
pdftool.pdf_extract_text(input_file, output_file)
media.pdf_extract_text(input_file, output_file)
assert output_file.exists()
@@ -123,7 +122,7 @@ class TestPdfExtractImages:
mock_doc.__iter__ = MagicMock(return_value=iter([mock_page]))
mock_doc.extract_image.return_value = {"image": b"image data", "ext": "png"}
mock_fitz_open.return_value = mock_doc
pdftool.pdf_extract_images(input_file, output_dir)
media.pdf_extract_images(input_file, output_dir)
assert output_dir.exists()
@@ -147,7 +146,7 @@ class TestPdfAddWatermark:
mock_doc.__iter__ = MagicMock(return_value=iter([mock_page]))
mock_fitz_open.return_value = mock_doc
mock_text_length.return_value = 100
pdftool.pdf_add_watermark(input_file, output_file)
media.pdf_add_watermark(input_file, output_file)
assert mock_doc.save.called
@@ -169,7 +168,7 @@ class TestPdfRotate:
mock_page = MagicMock()
mock_doc.__iter__ = MagicMock(return_value=iter([mock_page]))
mock_fitz_open.return_value = mock_doc
pdftool.pdf_rotate(input_file, output_file, rotation=90)
media.pdf_rotate(input_file, output_file, rotation=90)
assert mock_doc.save.called
def test_pdf_rotate_file_180(self, tmp_path: Path) -> None:
@@ -184,7 +183,7 @@ class TestPdfRotate:
mock_page = MagicMock()
mock_doc.__iter__ = MagicMock(return_value=iter([mock_page]))
mock_fitz_open.return_value = mock_doc
pdftool.pdf_rotate(input_file, output_file, rotation=180)
media.pdf_rotate(input_file, output_file, rotation=180)
assert mock_doc.save.called
@@ -207,7 +206,7 @@ class TestPdfCrop:
mock_page.rect = MagicMock(x0=0, y0=0, x1=800, y1=600)
mock_doc.__iter__ = MagicMock(return_value=iter([mock_page]))
mock_fitz_open.return_value = mock_doc
pdftool.pdf_crop(input_file, output_file, margins=(10, 10, 10, 10))
media.pdf_crop(input_file, output_file, margins=(10, 10, 10, 10))
assert mock_doc.save.called
@@ -228,7 +227,7 @@ class TestPdfInfo:
mock_doc.page_count = 10
mock_doc.metadata = {"title": "Test", "author": "Author"}
mock_fitz_open.return_value = mock_doc
pdftool.pdf_info(input_file)
media.pdf_info(input_file)
assert mock_fitz_open.called
@@ -257,7 +256,7 @@ class TestPdfOcr:
mock_doc.__iter__ = MagicMock(return_value=iter([mock_page]))
mock_fitz_open.return_value = mock_doc
mock_ocr.return_value = "OCR text"
pdftool.pdf_ocr(input_file, output_file)
media.pdf_ocr(input_file, output_file)
# Should complete OCR
@@ -277,48 +276,5 @@ class TestPdfRepair:
with patch("fitz.open") as mock_fitz_open:
mock_doc = MagicMock()
mock_fitz_open.return_value = mock_doc
pdftool.pdf_repair(input_file, output_file)
media.pdf_repair(input_file, output_file)
assert mock_doc.save.called
# ---------------------------------------------------------------------- #
# main function
# ---------------------------------------------------------------------- #
class TestMain:
"""Test main function."""
def test_main_merge_command(self, tmp_path: Path) -> None:
"""main() should handle merge command."""
input_files = [tmp_path / "input1.pdf", tmp_path / "input2.pdf"]
for f in input_files:
f.write_bytes(b"PDF content")
with patch("sys.argv", ["pdftool", "m", str(input_files[0]), str(input_files[1])]), patch.object(
px, "run"
) as mock_run:
pdftool.main()
assert mock_run.called
def test_main_split_command(self, tmp_path: Path) -> None:
"""main() should handle split command."""
input_file = tmp_path / "input.pdf"
input_file.write_bytes(b"PDF content")
with patch("sys.argv", ["pdftool", "s", str(input_file)]), patch.object(px, "run") as mock_run:
pdftool.main()
assert mock_run.called
def test_main_compress_command(self, tmp_path: Path) -> None:
"""main() should handle compress command."""
input_file = tmp_path / "input.pdf"
input_file.write_bytes(b"PDF content")
with patch("sys.argv", ["pdftool", "c", str(input_file)]), patch.object(px, "run") as mock_run:
pdftool.main()
assert mock_run.called
def test_main_with_no_args_shows_help(self) -> None:
"""main() with no args should show help."""
with patch("sys.argv", ["pdftool"]):
pdftool.main()
# Should print help and return
+28 -78
View File
@@ -6,8 +6,7 @@ import subprocess
from pathlib import Path
from unittest.mock import MagicMock, patch
import pyflowx as px
from pyflowx.cli import piptool
from pyflowx.ops import dev
# ---------------------------------------------------------------------- #
@@ -20,7 +19,7 @@ class TestGetInstalledPackages:
"""Should get installed packages."""
with patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(stdout="numpy==1.0.0\npandas==2.0.0\n", returncode=0)
result = piptool._get_installed_packages()
result = dev._get_installed_packages()
assert "numpy" in result
assert "pandas" in result
@@ -28,19 +27,19 @@ class TestGetInstalledPackages:
"""Should handle empty output."""
with patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(stdout="", returncode=0)
result = piptool._get_installed_packages()
result = dev._get_installed_packages()
assert result == []
def test_get_installed_packages_error(self) -> None:
"""Should handle subprocess error."""
with patch("subprocess.run", side_effect=subprocess.SubprocessError):
result = piptool._get_installed_packages()
result = dev._get_installed_packages()
assert result == []
def test_get_installed_packages_oserror(self) -> None:
"""Should handle OSError."""
with patch("subprocess.run", side_effect=OSError):
result = piptool._get_installed_packages()
result = dev._get_installed_packages()
assert result == []
@@ -52,26 +51,26 @@ class TestExpandWildcardPackages:
def test_expand_wildcard_no_pattern(self) -> None:
"""Should return package name when no wildcard."""
result = piptool._expand_wildcard_packages("numpy")
result = dev._expand_wildcard_packages("numpy")
assert result == ["numpy"]
def test_expand_wildcard_with_star(self) -> None:
"""Should expand wildcard with star."""
with patch.object(piptool, "_get_installed_packages", return_value=["numpy", "numpy-core", "pandas"]):
result = piptool._expand_wildcard_packages("numpy*")
with patch.object(dev, "_get_installed_packages", return_value=["numpy", "numpy-core", "pandas"]):
result = dev._expand_wildcard_packages("numpy*")
assert "numpy" in result
assert "numpy-core" in result
def test_expand_wildcard_with_question(self) -> None:
"""Should expand wildcard with question mark."""
with patch.object(piptool, "_get_installed_packages", return_value=["numpy", "numba"]):
result = piptool._expand_wildcard_packages("num??")
with patch.object(dev, "_get_installed_packages", return_value=["numpy", "numba"]):
result = dev._expand_wildcard_packages("num??")
assert len(result) > 0
def test_expand_wildcard_no_match(self) -> None:
"""Should return empty list when no match."""
with patch.object(piptool, "_get_installed_packages", return_value=["pandas", "scipy"]):
result = piptool._expand_wildcard_packages("numpy*")
with patch.object(dev, "_get_installed_packages", return_value=["pandas", "scipy"]):
result = dev._expand_wildcard_packages("numpy*")
assert result == []
@@ -83,19 +82,19 @@ class TestFilterProtectedPackages:
def test_filter_protected_packages_normal(self) -> None:
"""Should filter protected packages."""
result = piptool._filter_protected_packages(["numpy", "pandas", "pyflowx"])
result = dev._filter_protected_packages(["numpy", "pandas", "pyflowx"])
assert "numpy" in result
assert "pandas" in result
assert "pyflowx" not in result
def test_filter_protected_packages_all_protected(self) -> None:
"""Should filter all protected packages."""
result = piptool._filter_protected_packages(["pyflowx", "bitool"])
result = dev._filter_protected_packages(["pyflowx", "bitool"])
assert result == []
def test_filter_protected_packages_case_insensitive(self) -> None:
"""Should filter case insensitive."""
result = piptool._filter_protected_packages(["PyFlowX", "BITOOL"])
result = dev._filter_protected_packages(["PyFlowX", "BITOOL"])
assert result == []
@@ -109,35 +108,35 @@ class TestPipUninstall:
"""Should uninstall single package."""
with patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(returncode=0)
piptool.pip_uninstall(["numpy"])
dev.pip_uninstall(["numpy"])
assert mock_run.called
def test_pip_uninstall_multiple_packages(self) -> None:
"""Should uninstall multiple packages."""
with patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(returncode=0)
piptool.pip_uninstall(["numpy", "pandas", "scipy"])
dev.pip_uninstall(["numpy", "pandas", "scipy"])
# Should call pip uninstall
assert mock_run.called
def test_pip_uninstall_with_wildcard(self) -> None:
"""Should handle wildcard in package name."""
with patch.object(piptool, "_expand_wildcard_packages", return_value=["numpy", "numpy-core"]), patch(
with patch.object(dev, "_expand_wildcard_packages", return_value=["numpy", "numpy-core"]), patch(
"subprocess.run"
) as mock_run:
mock_run.return_value = MagicMock(returncode=0)
piptool.pip_uninstall(["numpy*"])
dev.pip_uninstall(["numpy*"])
assert mock_run.called
def test_pip_uninstall_empty_packages(self) -> None:
"""Should handle empty packages list."""
with patch.object(piptool, "_expand_wildcard_packages", return_value=[]):
piptool.pip_uninstall(["nonexistent*"])
with patch.object(dev, "_expand_wildcard_packages", return_value=[]):
dev.pip_uninstall(["nonexistent*"])
# Should not call subprocess.run
def test_pip_uninstall_all_protected(self) -> None:
"""Should handle all protected packages."""
piptool.pip_uninstall(["pyflowx"])
dev.pip_uninstall(["pyflowx"])
# Should not call subprocess.run
@@ -151,7 +150,7 @@ class TestPipReinstall:
"""Should reinstall single package."""
with patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(returncode=0)
piptool.pip_reinstall(["numpy"])
dev.pip_reinstall(["numpy"])
# Should call pip uninstall and pip install
assert mock_run.call_count == 2
@@ -159,13 +158,13 @@ class TestPipReinstall:
"""Should reinstall packages offline."""
with patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(returncode=0)
piptool.pip_reinstall(["numpy"], offline=True)
dev.pip_reinstall(["numpy"], offline=True)
# Should call pip install with offline flags
assert mock_run.called
def test_pip_reinstall_all_protected(self) -> None:
"""Should handle all protected packages."""
piptool.pip_reinstall(["pyflowx"])
dev.pip_reinstall(["pyflowx"])
# Should not call subprocess.run
@@ -179,14 +178,14 @@ class TestPipDownload:
"""Should download single package."""
with patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(returncode=0)
piptool.pip_download(["numpy"])
dev.pip_download(["numpy"])
assert mock_run.called
def test_pip_download_offline(self) -> None:
"""Should download packages offline."""
with patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(returncode=0)
piptool.pip_download(["numpy"], offline=True)
dev.pip_download(["numpy"], offline=True)
# Should call pip download with offline flags
assert mock_run.called
@@ -201,54 +200,5 @@ class TestPipFreeze:
"""Should freeze dependencies."""
with patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(stdout="numpy==1.0.0\npandas==2.0.0", returncode=0)
piptool.pip_freeze()
dev.pip_freeze()
assert mock_run.called
# ---------------------------------------------------------------------- #
# main function
# ---------------------------------------------------------------------- #
class TestMain:
"""Test main function."""
def test_main_install_command(self) -> None:
"""main() should handle install command."""
with patch("sys.argv", ["piptool", "i", "numpy", "pandas"]), patch.object(px, "run") as mock_run:
piptool.main()
assert mock_run.called
def test_main_uninstall_command(self) -> None:
"""main() should handle uninstall command."""
with patch("sys.argv", ["piptool", "u", "numpy"]), patch.object(px, "run") as mock_run:
piptool.main()
assert mock_run.called
def test_main_reinstall_command(self) -> None:
"""main() should handle reinstall command."""
with patch("sys.argv", ["piptool", "r", "numpy"]), patch.object(px, "run") as mock_run:
piptool.main()
assert mock_run.called
def test_main_download_command(self) -> None:
"""main() should handle download command."""
with patch("sys.argv", ["piptool", "d", "numpy"]), patch.object(px, "run") as mock_run:
piptool.main()
assert mock_run.called
def test_main_upgrade_command(self) -> None:
"""main() should handle upgrade command."""
with patch("sys.argv", ["piptool", "up"]), patch.object(px, "run") as mock_run:
piptool.main()
assert mock_run.called
def test_main_freeze_command(self) -> None:
"""main() should handle freeze command."""
with patch("sys.argv", ["piptool", "f"]), patch.object(px, "run") as mock_run:
piptool.main()
assert mock_run.called
def test_main_with_no_args_shows_help(self) -> None:
"""main() with no args should show help."""
with patch("sys.argv", ["piptool"]):
piptool.main()
# Should print help and return
+6 -4
View File
@@ -56,10 +56,12 @@ def _build_simple_profile() -> ProfileReport:
report = px.RunReport()
report.results["a"] = _result("a", start, 1.0)
report.results["b"] = _result("b", start + timedelta(seconds=1), 2.0)
graph = px.Graph.from_specs([
_spec("a"),
_spec("b", deps=("a",)),
])
graph = px.Graph.from_specs(
[
_spec("a"),
_spec("b", deps=("a",)),
]
)
return ProfileReport.from_report(report, graph)
-175
View File
@@ -1,175 +0,0 @@
"""Tests for cli.pymake module."""
from __future__ import annotations
from pathlib import Path
from unittest.mock import patch
import pytest
from pyflowx.cli import pymake
# ---------------------------------------------------------------------- #
# TaskSpec definitions
# ---------------------------------------------------------------------- #
def _find_task(name: str) -> pymake.px.TaskSpec:
"""从 pymake.tasks 或 aliases 中查找指定名称的 TaskSpec."""
for spec in pymake.tasks:
if spec.name == name:
return spec
# 单任务别名(doc/lint/tox)内联在 aliases dict 中
value = pymake.aliases.get(name)
if isinstance(value, pymake.px.TaskSpec):
return value
raise KeyError(f"任务 {name!r} 未找到")
class TestTaskSpecDefinitions:
"""Test that all TaskSpec definitions are valid."""
def test_uv_build_spec(self) -> None:
"""uv_build spec should be properly defined."""
spec = _find_task("uv_build")
assert spec.name == "uv_build"
assert spec.cmd == ["uv", "build"]
assert spec.skip_if_missing is False
def test_maturin_build_spec(self) -> None:
"""maturin_build spec should be properly defined."""
spec = _find_task("maturin_build")
assert spec.name == "maturin_build"
assert isinstance(spec.cmd, list)
assert spec.skip_if_missing is False
def test_uv_sync_spec(self) -> None:
"""uv_sync spec should be properly defined."""
spec = _find_task("uv_sync")
assert spec.name == "uv_sync"
assert spec.cmd == ["uv", "sync"]
assert spec.skip_if_missing is False
def test_git_clean_spec(self) -> None:
"""git_clean spec should be properly defined."""
spec = _find_task("git_clean")
assert spec.name == "git_clean"
assert spec.cmd == ["gitt", "c"]
assert spec.skip_if_missing is False
def test_test_spec(self) -> None:
"""test spec should be properly defined."""
spec = _find_task("test")
assert spec.name == "test"
assert isinstance(spec.cmd, list)
assert "pytest" in spec.cmd
assert "-m" in spec.cmd
assert "not slow" in spec.cmd
assert spec.skip_if_missing is False
def test_test_fast_spec(self) -> None:
"""test_fast spec should be properly defined."""
spec = _find_task("test_fast")
assert spec.name == "test_fast"
assert isinstance(spec.cmd, list)
assert "pytest" in spec.cmd
assert "-n" not in spec.cmd # test_fast doesn't use parallel
assert spec.skip_if_missing is False
def test_test_coverage_spec(self) -> None:
"""test_coverage spec should be properly defined."""
spec = _find_task("test_coverage")
assert spec.name == "test_coverage"
assert isinstance(spec.cmd, list)
assert "pytest" in spec.cmd
assert "--cov" in spec.cmd
assert spec.skip_if_missing is False
def test_ruff_lint_spec(self) -> None:
"""lint spec should be properly defined."""
spec = _find_task("lint")
assert spec.name == "lint"
assert isinstance(spec.cmd, list)
assert "ruff" in spec.cmd
assert "check" in spec.cmd
assert spec.skip_if_missing is False
def test_doc_spec(self) -> None:
"""doc spec should be properly defined."""
spec = _find_task("doc")
assert spec.name == "doc"
assert isinstance(spec.cmd, list)
assert "sphinx-build" in spec.cmd
assert spec.skip_if_missing is False
def test_hatch_publish_spec(self) -> None:
"""publish_python spec should be properly defined."""
spec = _find_task("publish_python")
assert spec.name == "publish_python"
assert spec.cmd == ["hatch", "publish"]
assert spec.skip_if_missing is False
def test_twine_publish_spec(self) -> None:
"""twine_publish spec should be properly defined."""
spec = _find_task("twine_publish")
assert spec.name == "twine_publish"
assert isinstance(spec.cmd, list)
assert "twine" in spec.cmd
assert "upload" in spec.cmd
assert spec.skip_if_missing is False
def test_tox_spec(self) -> None:
"""tox spec should be properly defined."""
spec = _find_task("tox")
assert spec.name == "tox"
assert spec.cmd == ["tox", "-p", "auto"]
assert spec.skip_if_missing is False
def test_all_tasks_have_correct_cwd(self) -> None:
"""所有任务应该有正确的 cwd 设置(指向项目根目录)."""
# 验证 ROOT_DIR 定义正确(向上三层到达项目根目录)
expected_root = Path(__file__).parent.parent.parent
assert expected_root == pymake.ROOT_DIR
# 验证 tasks 中的所有命令任务都有正确的 cwd
for spec in pymake.tasks:
if spec.cmd is not None:
assert spec.cwd == pymake.ROOT_DIR, f"任务 {spec.name} 的 cwd 应为 {pymake.ROOT_DIR}"
# 验证 aliases 中的内联任务(doc/lint/tox)也有正确的 cwd
for name in ("doc", "lint", "tox"):
spec = _find_task(name)
assert spec.cwd == pymake.ROOT_DIR, f"任务 {name} 的 cwd 应为 {pymake.ROOT_DIR}"
# ---------------------------------------------------------------------- #
# main function
# ---------------------------------------------------------------------- #
class TestMain:
"""Test main function."""
def test_main_calls_run_cli(self) -> None:
"""main() should create a CliRunner and call run_cli()."""
with pytest.raises(SystemExit) as exc_info:
pymake.main()
# run_cli() calls sys.exit(), so we should get SystemExit
# The exit code depends on whether any commands are available
assert exc_info.value.code in (0, 1, 2)
def test_main_with_list_argument(self) -> None:
"""main() should handle --list argument."""
with patch("sys.argv", ["pymake", "--list"]), pytest.raises(SystemExit) as exc_info:
pymake.main()
assert exc_info.value.code == 0
def test_main_creates_runner_with_multiple_commands(self) -> None:
"""main() should create a CliRunner with multiple commands."""
# We can't easily test the runner creation without mocking,
# but we can verify that main() doesn't raise an error for --list
with patch("sys.argv", ["pymake", "--list"]), pytest.raises(SystemExit):
pymake.main()
def test_main_with_no_args_shows_help(self) -> None:
"""main() with no args should show help and exit with failure."""
with patch("sys.argv", ["pymake"]), pytest.raises(SystemExit) as exc_info:
pymake.main()
assert exc_info.value.code == 1
+58
View File
@@ -0,0 +1,58 @@
"""Tests for reseticoncache tool (reset_icon_cache_run 函数)."""
from __future__ import annotations
import subprocess
import pytest
from pyflowx.conditions import Constants
from pyflowx.ops.system import reset_icon_cache_run
class TestResetIconCacheRun:
"""``reset_icon_cache_run`` 函数测试."""
def test_non_windows_skips(self, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]) -> None:
"""非 Windows 平台应打印提示并跳过."""
monkeypatch.setattr(Constants, "IS_WINDOWS", False)
reset_icon_cache_run()
captured = capsys.readouterr()
assert "仅在 Windows 上支持" in captured.out
def test_windows_no_localappdata_skips(
self, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
) -> None:
"""Windows 上 LOCALAPPDATA 未设置时应跳过."""
monkeypatch.setattr(Constants, "IS_WINDOWS", True)
monkeypatch.setattr("os.environ.get", lambda key, default="": "" if key == "LOCALAPPDATA" else default)
reset_icon_cache_run()
captured = capsys.readouterr()
assert "LOCALAPPDATA" in captured.out
def test_windows_executes_all_steps(
self, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
) -> None:
"""Windows 上应依次执行 kill/delete/delete_all/restart 四步."""
monkeypatch.setattr(Constants, "IS_WINDOWS", True)
local_app_data = "/tmp/fake_local_app_data"
monkeypatch.setattr(
"os.environ.get", lambda key, default="": local_app_data if key == "LOCALAPPDATA" else default
)
# 让 icon_cache_db 和 explorer_cache_dir 都不存在, 跳过删除分支
from pathlib import Path
monkeypatch.setattr(Path, "exists", lambda _self: False)
ran_cmds: list[list[str]] = []
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: ran_cmds.append(cmd))
reset_icon_cache_run()
captured = capsys.readouterr()
# 验证执行了 kill_explorer 和 restart_explorer (delete 分支因 exists=False 被跳过)
assert any("taskkill" in str(cmd) for cmd in ran_cmds)
assert any("explorer.exe" in str(cmd) for cmd in ran_cmds)
assert any("start" in str(cmd) and "explorer.exe" in str(cmd) for cmd in ran_cmds)
assert "图标缓存已重置" in captured.out
+9 -35
View File
@@ -5,9 +5,8 @@ from __future__ import annotations
from pathlib import Path
from unittest.mock import MagicMock, patch
import pyflowx as px
from pyflowx.cli import screenshot
from pyflowx.conditions import Constants
from pyflowx.ops import media
# ---------------------------------------------------------------------- #
@@ -19,13 +18,13 @@ class TestGetScreenshotPath:
def test_get_screenshot_path_with_filename(self, tmp_path: Path) -> None:
"""Should get screenshot path with filename."""
with patch.object(Path, "home", return_value=tmp_path):
result = screenshot.get_screenshot_path("test.png")
result = media.get_screenshot_path("test.png")
assert result.name == "test.png"
def test_get_screenshot_path_without_filename(self, tmp_path: Path) -> None:
"""Should get screenshot path without filename."""
with patch.object(Path, "home", return_value=tmp_path):
result = screenshot.get_screenshot_path()
result = media.get_screenshot_path()
assert "screenshot_" in result.name
assert result.suffix == ".png"
@@ -42,7 +41,7 @@ class TestTakeScreenshotFull:
Path, "home", return_value=tmp_path
), patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(returncode=0)
screenshot.take_screenshot_full()
media.take_screenshot_full()
assert mock_run.called
def test_take_screenshot_full_macos(self, tmp_path: Path) -> None:
@@ -51,7 +50,7 @@ class TestTakeScreenshotFull:
Path, "home", return_value=tmp_path
), patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(returncode=0)
screenshot.take_screenshot_full()
media.take_screenshot_full()
assert mock_run.called
def test_take_screenshot_full_linux(self, tmp_path: Path) -> None:
@@ -60,7 +59,7 @@ class TestTakeScreenshotFull:
Path, "home", return_value=tmp_path
), patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(returncode=0)
screenshot.take_screenshot_full()
media.take_screenshot_full()
assert mock_run.called
@@ -76,7 +75,7 @@ class TestTakeScreenshotArea:
Path, "home", return_value=tmp_path
), patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(returncode=0)
screenshot.take_screenshot_area()
media.take_screenshot_area()
assert mock_run.called
def test_take_screenshot_area_macos(self, tmp_path: Path) -> None:
@@ -85,7 +84,7 @@ class TestTakeScreenshotArea:
Path, "home", return_value=tmp_path
), patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(returncode=0)
screenshot.take_screenshot_area()
media.take_screenshot_area()
assert mock_run.called
def test_take_screenshot_area_linux(self, tmp_path: Path) -> None:
@@ -94,30 +93,5 @@ class TestTakeScreenshotArea:
Path, "home", return_value=tmp_path
), patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(returncode=0)
screenshot.take_screenshot_area()
media.take_screenshot_area()
assert mock_run.called
# ---------------------------------------------------------------------- #
# main function
# ---------------------------------------------------------------------- #
class TestMain:
"""Test main function."""
def test_main_full_command(self, tmp_path: Path) -> None:
"""main() should handle full command."""
with patch("sys.argv", ["screenshot", "full"]), patch.object(px, "run") as mock_run:
screenshot.main()
assert mock_run.called
def test_main_area_command(self, tmp_path: Path) -> None:
"""main() should handle area command."""
with patch("sys.argv", ["screenshot", "area"]), patch.object(px, "run") as mock_run:
screenshot.main()
assert mock_run.called
def test_main_with_no_args_shows_help(self) -> None:
"""main() with no args should show help."""
with patch("sys.argv", ["screenshot"]):
screenshot.main()
# Should print help and return
+9 -75
View File
@@ -8,8 +8,7 @@ from unittest.mock import MagicMock, patch
import pytest
import pyflowx as px
from pyflowx.cli import sshcopyid
from pyflowx.ops import system
# ---------------------------------------------------------------------- #
@@ -21,7 +20,7 @@ class TestSshCopyId:
def test_ssh_copy_id_pub_key_not_exists(self, tmp_path: Path) -> None:
"""Should handle nonexistent public key."""
with patch.object(Path, "expanduser", return_value=tmp_path / "nonexistent.pub"), pytest.raises(SystemExit):
sshcopyid.ssh_copy_id("localhost", "user", "password")
system.ssh_copy_id("localhost", "user", "password")
def test_ssh_copy_id_sshpass_not_found(self, tmp_path: Path) -> None:
"""Should handle sshpass not found."""
@@ -31,7 +30,7 @@ class TestSshCopyId:
with patch.object(Path, "expanduser", return_value=pub_key), patch(
"subprocess.run", side_effect=FileNotFoundError
), pytest.raises(SystemExit):
sshcopyid.ssh_copy_id("localhost", "user", "password")
system.ssh_copy_id("localhost", "user", "password")
def test_ssh_copy_id_timeout(self, tmp_path: Path) -> None:
"""Should handle SSH timeout."""
@@ -41,7 +40,7 @@ class TestSshCopyId:
with patch.object(Path, "expanduser", return_value=pub_key), patch(
"subprocess.run", side_effect=subprocess.TimeoutExpired("cmd", 30)
), pytest.raises(SystemExit):
sshcopyid.ssh_copy_id("localhost", "user", "password")
system.ssh_copy_id("localhost", "user", "password")
def test_ssh_copy_id_process_error(self, tmp_path: Path) -> None:
"""Should handle SSH process error."""
@@ -51,7 +50,7 @@ class TestSshCopyId:
with patch.object(Path, "expanduser", return_value=pub_key), patch(
"subprocess.run", side_effect=subprocess.CalledProcessError(1, "cmd")
), pytest.raises(SystemExit):
sshcopyid.ssh_copy_id("localhost", "user", "password")
system.ssh_copy_id("localhost", "user", "password")
def test_ssh_copy_id_success(self, tmp_path: Path) -> None:
"""Should deploy SSH key successfully."""
@@ -60,7 +59,7 @@ class TestSshCopyId:
with patch.object(Path, "expanduser", return_value=pub_key), patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(returncode=0)
sshcopyid.ssh_copy_id("localhost", "user", "password")
system.ssh_copy_id("localhost", "user", "password")
assert mock_run.called
def test_ssh_copy_id_with_custom_port(self, tmp_path: Path) -> None:
@@ -70,7 +69,7 @@ class TestSshCopyId:
with patch.object(Path, "expanduser", return_value=pub_key), patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(returncode=0)
sshcopyid.ssh_copy_id("localhost", "user", "password", port=2222)
system.ssh_copy_id("localhost", "user", "password", port=2222)
# Verify port is used
call_args = mock_run.call_args[0][0]
assert "2222" in call_args
@@ -82,7 +81,7 @@ class TestSshCopyId:
with patch.object(Path, "expanduser", return_value=custom_key), patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(returncode=0)
sshcopyid.ssh_copy_id("localhost", "user", "password", keypath=str(custom_key))
system.ssh_copy_id("localhost", "user", "password", keypath=str(custom_key))
assert mock_run.called
def test_ssh_copy_id_with_custom_timeout(self, tmp_path: Path) -> None:
@@ -92,72 +91,7 @@ class TestSshCopyId:
with patch.object(Path, "expanduser", return_value=pub_key), patch("subprocess.run") as mock_run:
mock_run.return_value = MagicMock(returncode=0)
sshcopyid.ssh_copy_id("localhost", "user", "password", timeout=60)
system.ssh_copy_id("localhost", "user", "password", timeout=60)
# Verify timeout is used in ConnectTimeout option
call_args = mock_run.call_args[0][0]
assert "ConnectTimeout=60" in call_args
# ---------------------------------------------------------------------- #
# main function
# ---------------------------------------------------------------------- #
class TestMain:
"""Test main function."""
def test_main_with_required_args(self) -> None:
"""main() should handle required arguments."""
with patch("sys.argv", ["sshcopyid", "localhost", "user", "password"]), patch.object(
px, "run"
) as mock_run, patch.object(sshcopyid, "ssh_copy_id"):
sshcopyid.main()
assert mock_run.called
graph = mock_run.call_args[0][0]
assert isinstance(graph, px.Graph)
def test_main_with_custom_port(self) -> None:
"""main() should handle custom port argument."""
with patch("sys.argv", ["sshcopyid", "localhost", "user", "password", "--port", "2222"]), patch.object(
px, "run"
) as mock_run, patch.object(sshcopyid, "ssh_copy_id"):
sshcopyid.main()
assert mock_run.called
def test_main_with_custom_keypath(self) -> None:
"""main() should handle custom keypath argument."""
with patch(
"sys.argv", ["sshcopyid", "localhost", "user", "password", "--keypath", "/custom/key.pub"]
), patch.object(px, "run") as mock_run, patch.object(sshcopyid, "ssh_copy_id"):
sshcopyid.main()
assert mock_run.called
def test_main_with_custom_timeout(self) -> None:
"""main() should handle custom timeout argument."""
with patch("sys.argv", ["sshcopyid", "localhost", "user", "password", "--timeout", "60"]), patch.object(
px, "run"
) as mock_run, patch.object(sshcopyid, "ssh_copy_id"):
sshcopyid.main()
assert mock_run.called
def test_main_with_no_args_shows_help(self) -> None:
"""main() with no args should show help and exit."""
with patch("sys.argv", ["sshcopyid"]), pytest.raises(SystemExit) as exc_info:
sshcopyid.main()
assert exc_info.value.code == 2
def test_main_creates_task_spec_with_correct_name(self) -> None:
"""main() should create TaskSpec with correct name."""
with patch("sys.argv", ["sshcopyid", "localhost", "user", "password"]), patch.object(
px, "run"
) as mock_run, patch.object(sshcopyid, "ssh_copy_id"):
sshcopyid.main()
graph = mock_run.call_args[0][0]
task_names = list(graph.all_specs().keys())
assert "ssh_deploy" in task_names
def test_main_uses_thread_strategy(self) -> None:
"""main() should use thread strategy."""
with patch("sys.argv", ["sshcopyid", "localhost", "user", "password"]), patch.object(
px, "run"
) as mock_run, patch.object(sshcopyid, "ssh_copy_id"):
sshcopyid.main()
assert mock_run.call_args[1]["strategy"] == "thread"
+273 -197
View File
@@ -82,9 +82,11 @@ class TestRetryPolicy:
raise RuntimeError("not yet")
return "ok"
graph = px.Graph.from_specs([
px.TaskSpec("flaky", flaky, retry=RetryPolicy(max_attempts=3)),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("flaky", flaky, retry=RetryPolicy(max_attempts=3)),
]
)
report = px.run(graph, strategy="sequential")
assert report.success
assert report["flaky"] == "ok"
@@ -95,9 +97,11 @@ class TestRetryPolicy:
def always_fail() -> None:
raise RuntimeError("nope")
graph = px.Graph.from_specs([
px.TaskSpec("f", always_fail, retry=RetryPolicy(max_attempts=3)),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("f", always_fail, retry=RetryPolicy(max_attempts=3)),
]
)
with pytest.raises(px.TaskFailedError) as exc_info:
px.run(graph, strategy="sequential")
assert exc_info.value.attempts == 3
@@ -111,13 +115,15 @@ class TestRetryPolicy:
raise KeyError("not retried")
# retry_on=(ValueError,) -> KeyError 不应被重试
graph = px.Graph.from_specs([
px.TaskSpec(
"f",
fail_with_keyerror,
retry=RetryPolicy(max_attempts=3, retry_on=(ValueError,)),
),
])
graph = px.Graph.from_specs(
[
px.TaskSpec(
"f",
fail_with_keyerror,
retry=RetryPolicy(max_attempts=3, retry_on=(ValueError,)),
),
]
)
with pytest.raises(px.TaskFailedError) as exc_info:
px.run(graph, strategy="sequential")
# KeyError 不在 retry_on 中,应只尝试 1 次
@@ -136,13 +142,15 @@ class TestRetryPolicy:
raise RuntimeError("not yet")
return "ok"
graph = px.Graph.from_specs([
px.TaskSpec(
"flaky",
flaky,
retry=RetryPolicy(max_attempts=3, delay=0.05, backoff=2.0),
),
])
graph = px.Graph.from_specs(
[
px.TaskSpec(
"flaky",
flaky,
retry=RetryPolicy(max_attempts=3, delay=0.05, backoff=2.0),
),
]
)
report = px.run(graph, strategy="sequential")
assert report.success
# 第 2 次重试应在 delay=0.05 后,第 3 次应在 0.05*2=0.10 后
@@ -161,9 +169,11 @@ class TestRetryPolicy:
raise RuntimeError("not yet")
return "ok"
graph = px.Graph.from_specs([
px.TaskSpec("flaky", flaky, retry=RetryPolicy(max_attempts=3)),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("flaky", flaky, retry=RetryPolicy(max_attempts=3)),
]
)
report = px.run(graph, strategy="async")
assert report.success
assert report["flaky"] == "ok"
@@ -187,9 +197,11 @@ class TestTaskHooks:
return "ok"
hooks = TaskHooks(pre_run=pre_run)
graph = px.Graph.from_specs([
px.TaskSpec("t", fn, hooks=hooks),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("t", fn, hooks=hooks),
]
)
report = px.run(graph, strategy="sequential")
assert report.success
assert events == ["pre:t", "run"]
@@ -205,9 +217,11 @@ class TestTaskHooks:
return 42
hooks = TaskHooks(post_run=post_run)
graph = px.Graph.from_specs([
px.TaskSpec("t", fn, hooks=hooks),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("t", fn, hooks=hooks),
]
)
report = px.run(graph, strategy="sequential")
assert report.success
assert captured == {"name": "t", "result": 42}
@@ -223,9 +237,11 @@ class TestTaskHooks:
raise ValueError("boom")
hooks = TaskHooks(on_failure=on_failure)
graph = px.Graph.from_specs([
px.TaskSpec("t", fn, hooks=hooks, continue_on_error=True),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("t", fn, hooks=hooks, continue_on_error=True),
]
)
report = px.run(graph, strategy="sequential")
# continue_on_error=True -> 报告成功但任务失败
assert report.success
@@ -242,14 +258,16 @@ class TestTaskHooks:
events.append("post")
hooks = TaskHooks(pre_run=pre_run, post_run=post_run)
graph = px.Graph.from_specs([
px.TaskSpec(
"t",
fn=lambda: "ok",
hooks=hooks,
conditions=(lambda _ctx: False,),
),
])
graph = px.Graph.from_specs(
[
px.TaskSpec(
"t",
fn=lambda: "ok",
hooks=hooks,
conditions=(lambda _ctx: False,),
),
]
)
report = px.run(graph, strategy="sequential")
assert report.success
assert report.result_of("t").status == TaskStatus.SKIPPED
@@ -266,9 +284,11 @@ class TestTaskHooks:
return "ok"
hooks = TaskHooks(pre_run=pre_run)
graph = px.Graph.from_specs([
px.TaskSpec("t", fn, hooks=hooks),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("t", fn, hooks=hooks),
]
)
report = px.run(graph, strategy="async")
assert report.success
assert events == ["pre:t", "run"]
@@ -414,10 +434,12 @@ class TestSoftDependencies:
order.append("fast")
return f"after-{slow}"
graph = px.Graph.from_specs([
px.TaskSpec("slow", slow),
px.TaskSpec("fast", fast, soft_depends_on=("slow",)),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("slow", slow),
px.TaskSpec("fast", fast, soft_depends_on=("slow",)),
]
)
report = px.run(graph, strategy="dependency")
assert report.success
# soft 依赖应等待 slow 完成后再执行 fast
@@ -433,15 +455,17 @@ class TestSoftDependencies:
def downstream(fail: str = "default") -> str:
return f"got:{fail}"
graph = px.Graph.from_specs([
px.TaskSpec("fail", fail, continue_on_error=True),
px.TaskSpec(
"downstream",
downstream,
soft_depends_on=("fail",),
continue_on_error=True,
),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("fail", fail, continue_on_error=True),
px.TaskSpec(
"downstream",
downstream,
soft_depends_on=("fail",),
continue_on_error=True,
),
]
)
report = px.run(graph, strategy="dependency")
assert report.success
# fail 失败但下游仍执行(使用默认值)
@@ -450,9 +474,11 @@ class TestSoftDependencies:
def test_soft_dependency_validation_unknown_dep(self) -> None:
with pytest.raises(px.MissingDependencyError):
px.Graph.from_specs([
px.TaskSpec("a", lambda: "ok", soft_depends_on=("missing",)),
])
px.Graph.from_specs(
[
px.TaskSpec("a", lambda: "ok", soft_depends_on=("missing",)),
]
)
def test_soft_and_hard_dependency_combined(self) -> None:
order: list[str] = []
@@ -469,11 +495,13 @@ class TestSoftDependencies:
order.append("c")
return f"c-{b}"
graph = px.Graph.from_specs([
px.TaskSpec("a", a),
px.TaskSpec("b", b, depends_on=("a",)),
px.TaskSpec("c", c, depends_on=("b",), soft_depends_on=("a",)),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("a", a),
px.TaskSpec("b", b, depends_on=("a",)),
px.TaskSpec("c", c, depends_on=("b",), soft_depends_on=("a",)),
]
)
report = px.run(graph, strategy="dependency")
assert report.success
assert order == ["a", "b", "c"]
@@ -495,11 +523,13 @@ class TestDependencyDrivenScheduling:
def c(b: int) -> int:
return b + 1
graph = px.Graph.from_specs([
px.TaskSpec("a", a),
px.TaskSpec("b", b, depends_on=("a",)),
px.TaskSpec("c", c, depends_on=("b",)),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("a", a),
px.TaskSpec("b", b, depends_on=("a",)),
px.TaskSpec("c", c, depends_on=("b",)),
]
)
report = px.run(graph, strategy="dependency")
assert report.success
assert report["a"] == 1
@@ -514,10 +544,12 @@ class TestDependencyDrivenScheduling:
async def b(a: str) -> str:
return f"b-{a}"
graph = px.Graph.from_specs([
px.TaskSpec("a", a),
px.TaskSpec("b", b, depends_on=("a",)),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("a", a),
px.TaskSpec("b", b, depends_on=("a",)),
]
)
report = px.run(graph, strategy="dependency")
assert report.success
assert report["b"] == "b-a"
@@ -537,12 +569,14 @@ class TestDependencyDrivenScheduling:
def d(b: int, c: int) -> int:
return b + c
graph = px.Graph.from_specs([
px.TaskSpec("a", a),
px.TaskSpec("b", b, depends_on=("a",)),
px.TaskSpec("c", c, depends_on=("a",)),
px.TaskSpec("d", d, depends_on=("b", "c")),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("a", a),
px.TaskSpec("b", b, depends_on=("a",)),
px.TaskSpec("c", c, depends_on=("a",)),
px.TaskSpec("d", d, depends_on=("b", "c")),
]
)
report = px.run(graph, strategy="dependency")
assert report.success
assert report["a"] == 10
@@ -574,11 +608,13 @@ class TestConcurrencyLimits:
return fn
graph = px.Graph.from_specs([
px.TaskSpec("a", make_fn(1), concurrency_key="db"),
px.TaskSpec("b", make_fn(2), concurrency_key="db"),
px.TaskSpec("c", make_fn(3), concurrency_key="db"),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("a", make_fn(1), concurrency_key="db"),
px.TaskSpec("b", make_fn(2), concurrency_key="db"),
px.TaskSpec("c", make_fn(3), concurrency_key="db"),
]
)
report = px.run(
graph,
strategy="dependency",
@@ -604,10 +640,12 @@ class TestConcurrencyLimits:
return fn
graph = px.Graph.from_specs([
px.TaskSpec("a", make_fn("a"), concurrency_key="db1"),
px.TaskSpec("b", make_fn("b"), concurrency_key="db2"),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("a", make_fn("a"), concurrency_key="db1"),
px.TaskSpec("b", make_fn("b"), concurrency_key="db2"),
]
)
report = px.run(
graph,
strategy="dependency",
@@ -633,12 +671,14 @@ class TestConcurrencyLimits:
return fn
graph = px.Graph.from_specs([
px.TaskSpec("a", make_fn(1), concurrency_key="pool"),
px.TaskSpec("b", make_fn(2), concurrency_key="pool"),
px.TaskSpec("c", make_fn(3), concurrency_key="pool"),
px.TaskSpec("d", make_fn(4), concurrency_key="pool"),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("a", make_fn(1), concurrency_key="pool"),
px.TaskSpec("b", make_fn(2), concurrency_key="pool"),
px.TaskSpec("c", make_fn(3), concurrency_key="pool"),
px.TaskSpec("d", make_fn(4), concurrency_key="pool"),
]
)
report = px.run(
graph,
strategy="dependency",
@@ -666,11 +706,13 @@ class TestPriority:
return fn
graph = px.Graph.from_specs([
px.TaskSpec("low", make_fn("low"), priority=1),
px.TaskSpec("high", make_fn("high"), priority=10),
px.TaskSpec("mid", make_fn("mid"), priority=5),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("low", make_fn("low"), priority=1),
px.TaskSpec("high", make_fn("high"), priority=10),
px.TaskSpec("mid", make_fn("mid"), priority=5),
]
)
report = px.run(graph, strategy="sequential")
assert report.success
# 高优先级先执行
@@ -696,10 +738,12 @@ class TestContinueOnError:
def downstream() -> str:
return "ran"
graph = px.Graph.from_specs([
px.TaskSpec("fail", fail, continue_on_error=True),
px.TaskSpec("downstream", downstream, depends_on=("fail",)),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("fail", fail, continue_on_error=True),
px.TaskSpec("downstream", downstream, depends_on=("fail",)),
]
)
report = px.run(graph, strategy="sequential")
# continue_on_error 使整体报告成功(不抛异常)
assert report.success
@@ -716,10 +760,12 @@ class TestContinueOnError:
def downstream() -> str:
return "ran"
graph = px.Graph.from_specs([
px.TaskSpec("fail", fail, continue_on_error=True),
px.TaskSpec("downstream", downstream, soft_depends_on=("fail",)),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("fail", fail, continue_on_error=True),
px.TaskSpec("downstream", downstream, soft_depends_on=("fail",)),
]
)
report = px.run(graph, strategy="dependency")
assert report.success
assert report.result_of("fail").status == TaskStatus.FAILED
@@ -734,10 +780,12 @@ class TestContinueOnError:
def other() -> str:
return "ok"
graph = px.Graph.from_specs([
px.TaskSpec("fail", fail, continue_on_error=True),
px.TaskSpec("other", other),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("fail", fail, continue_on_error=True),
px.TaskSpec("other", other),
]
)
report = px.run(graph, strategy="dependency")
assert report.success
assert report.result_of("fail").status == TaskStatus.FAILED
@@ -750,10 +798,12 @@ class TestContinueOnError:
def other() -> str:
return "ok"
graph = px.Graph.from_specs([
px.TaskSpec("fail", fail),
px.TaskSpec("other", other),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("fail", fail),
px.TaskSpec("other", other),
]
)
with pytest.raises(px.TaskFailedError):
px.run(graph, strategy="sequential")
@@ -854,9 +904,11 @@ class TestCompose:
g_extract = px.Graph.from_specs([px.TaskSpec("extract", extract)])
# transform 图:通过 _pending_refs 引用 "extract" 命令
# transform 自身不声明 depends_on,由 compose 展开时自动连接
g_transform = px.Graph.from_specs([
px.TaskSpec("transform", transform),
])
g_transform = px.Graph.from_specs(
[
px.TaskSpec("transform", transform),
]
)
g_transform._pending_refs = ["extract"]
resolved = px.compose({"extract": g_extract, "transform": g_transform})
@@ -943,18 +995,22 @@ class TestCacheKey:
return key
graph1 = px.Graph.from_specs([
px.TaskSpec("t", expensive, args=(5,), cache_key=make_cache_key(5)),
])
graph1 = px.Graph.from_specs(
[
px.TaskSpec("t", expensive, args=(5,), cache_key=make_cache_key(5)),
]
)
report1 = px.run(graph1, strategy="sequential", state=backend)
assert report1.success
assert report1["t"] == 10
assert calls["n"] == 1
# 第二次运行相同输入应命中缓存
graph2 = px.Graph.from_specs([
px.TaskSpec("t", expensive, args=(5,), cache_key=make_cache_key(5)),
])
graph2 = px.Graph.from_specs(
[
px.TaskSpec("t", expensive, args=(5,), cache_key=make_cache_key(5)),
]
)
report2 = px.run(graph2, strategy="sequential", state=backend)
assert report2.success
assert report2["t"] == 10
@@ -976,16 +1032,20 @@ class TestCacheKey:
return key
graph1 = px.Graph.from_specs([
px.TaskSpec("t", expensive, args=(5,), cache_key=make_cache_key(5)),
])
graph1 = px.Graph.from_specs(
[
px.TaskSpec("t", expensive, args=(5,), cache_key=make_cache_key(5)),
]
)
px.run(graph1, strategy="sequential", state=backend)
assert calls["n"] == 1
# 不同输入应 miss
graph2 = px.Graph.from_specs([
px.TaskSpec("t", expensive, args=(7,), cache_key=make_cache_key(7)),
])
graph2 = px.Graph.from_specs(
[
px.TaskSpec("t", expensive, args=(7,), cache_key=make_cache_key(7)),
]
)
px.run(graph2, strategy="sequential", state=backend)
assert calls["n"] == 2
@@ -997,13 +1057,15 @@ class TestEnvAndCwd:
"""测试环境变量与工作目录隔离。"""
def test_env_override_for_cmd(self) -> None:
graph = px.Graph.from_specs([
px.TaskSpec(
"print_var",
cmd=[sys.executable, "-c", "import os; print(os.environ.get('PYFLOWX_TEST_VAR', 'unset'))"],
env={"PYFLOWX_TEST_VAR": "isolated"},
),
])
graph = px.Graph.from_specs(
[
px.TaskSpec(
"print_var",
cmd=[sys.executable, "-c", "import os; print(os.environ.get('PYFLOWX_TEST_VAR', 'unset'))"],
env={"PYFLOWX_TEST_VAR": "isolated"},
),
]
)
report = px.run(graph, strategy="sequential")
assert report.success
@@ -1011,13 +1073,15 @@ class TestEnvAndCwd:
# 在 tmp_path 下创建标记文件
marker = tmp_path / "marker.txt"
marker.write_text("found")
graph = px.Graph.from_specs([
px.TaskSpec(
"check_cwd",
cmd=["ls", "marker.txt"],
cwd=tmp_path,
),
])
graph = px.Graph.from_specs(
[
px.TaskSpec(
"check_cwd",
cmd=["ls", "marker.txt"],
cwd=tmp_path,
),
]
)
report = px.run(graph, strategy="sequential")
assert report.success
@@ -1027,13 +1091,15 @@ class TestEnvAndCwd:
def check_env() -> str:
return os.environ.get("PYFLOWX_LEAK_TEST", "not-set")
graph = px.Graph.from_specs([
px.TaskSpec(
"t",
check_env,
env={"PYFLOWX_LEAK_TEST": "leaked"},
),
])
graph = px.Graph.from_specs(
[
px.TaskSpec(
"t",
check_env,
env={"PYFLOWX_LEAK_TEST": "leaked"},
),
]
)
# fn 任务的环境变量隔离仅在 cmd 任务生效,fn 共享进程环境
# 这里验证 fn 任务不修改外层环境
report = px.run(graph, strategy="sequential")
@@ -1059,21 +1125,23 @@ class TestContextAwareConditions:
def path_b(decide: str = "") -> str:
return f"ran-b:{decide}"
graph = px.Graph.from_specs([
px.TaskSpec("decide", decide),
px.TaskSpec(
"path_a",
path_a,
depends_on=("decide",),
conditions=(BuiltinConditions.DEP_EQUALS("decide", "path_a"),),
),
px.TaskSpec(
"path_b",
path_b,
depends_on=("decide",),
conditions=(BuiltinConditions.DEP_EQUALS("decide", "path_b"),),
),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("decide", decide),
px.TaskSpec(
"path_a",
path_a,
depends_on=("decide",),
conditions=(BuiltinConditions.DEP_EQUALS("decide", "path_a"),),
),
px.TaskSpec(
"path_b",
path_b,
depends_on=("decide",),
conditions=(BuiltinConditions.DEP_EQUALS("decide", "path_b"),),
),
]
)
report = px.run(graph, strategy="dependency")
assert report.success
assert report.result_of("path_a").status == TaskStatus.SKIPPED
@@ -1087,15 +1155,17 @@ class TestContextAwareConditions:
def only_if_nonempty(source: list[int]) -> str:
return f"has-{len(source)}"
graph = px.Graph.from_specs([
px.TaskSpec("source", source),
px.TaskSpec(
"only_if_nonempty",
only_if_nonempty,
depends_on=("source",),
conditions=(BuiltinConditions.DEP_TRUTHY("source"),),
),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("source", source),
px.TaskSpec(
"only_if_nonempty",
only_if_nonempty,
depends_on=("source",),
conditions=(BuiltinConditions.DEP_TRUTHY("source"),),
),
]
)
report = px.run(graph, strategy="dependency")
assert report.success
assert report["only_if_nonempty"] == "has-3"
@@ -1107,15 +1177,17 @@ class TestContextAwareConditions:
def only_if_nonempty(source: list[int]) -> str:
return "should-not-run"
graph = px.Graph.from_specs([
px.TaskSpec("source", source),
px.TaskSpec(
"only_if_nonempty",
only_if_nonempty,
depends_on=("source",),
conditions=(BuiltinConditions.DEP_TRUTHY("source"),),
),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("source", source),
px.TaskSpec(
"only_if_nonempty",
only_if_nonempty,
depends_on=("source",),
conditions=(BuiltinConditions.DEP_TRUTHY("source"),),
),
]
)
report = px.run(graph, strategy="dependency")
assert report.success
assert report.result_of("only_if_nonempty").status == TaskStatus.SKIPPED
@@ -1127,15 +1199,17 @@ class TestContextAwareConditions:
def downstream(source: int) -> str:
return f"got-{source}"
graph = px.Graph.from_specs([
px.TaskSpec("source", source),
px.TaskSpec(
"downstream",
downstream,
depends_on=("source",),
conditions=(BuiltinConditions.DEP_MATCHES("source", lambda v: v > 10),),
),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("source", source),
px.TaskSpec(
"downstream",
downstream,
depends_on=("source",),
conditions=(BuiltinConditions.DEP_MATCHES("source", lambda v: v > 10),),
),
]
)
report = px.run(graph, strategy="dependency")
assert report.success
assert report["downstream"] == "got-42"
@@ -1165,10 +1239,12 @@ class TestPerTaskStrategy:
await asyncio.sleep(0.01)
return f"async-{sync}"
graph = px.Graph.from_specs([
px.TaskSpec("sync", sync_fn),
px.TaskSpec("async", async_fn, depends_on=("sync",)),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("sync", sync_fn),
px.TaskSpec("async", async_fn, depends_on=("sync",)),
]
)
report = px.run(graph, strategy="dependency")
assert report.success
assert report["async"] == "async-sync"
+1
View File
@@ -68,6 +68,7 @@ def test_chain_execution_order() -> None:
def fn() -> str:
order.append(name)
return name
return fn
a = TaskSpec("a", make("a"))
+3 -3
View File
@@ -93,7 +93,7 @@ class TestCommandReferences:
with pytest.raises(ValueError, match="循环引用"):
px.CliRunner(
strategy="sequential",
aliases={
aliases={
"cmd1": px.Graph.from_specs(["cmd1", task1]),
},
)
@@ -105,7 +105,7 @@ class TestCommandReferences:
with pytest.raises(ValueError, match="引用的命令 'invalid' 不存在"):
px.CliRunner(
strategy="sequential",
aliases={
aliases={
"cmd1": px.Graph.from_specs(["invalid", task1]),
},
)
@@ -117,7 +117,7 @@ class TestCommandReferences:
with pytest.raises(ValueError, match="任务 'invalid' 不存在于命令 'cmd1'"):
px.CliRunner(
strategy="sequential",
aliases={
aliases={
"cmd1": px.Graph.from_specs([task1]),
"cmd2": px.Graph.from_specs(["cmd1.invalid"]),
},
+118 -78
View File
@@ -27,10 +27,12 @@ def test_sequential_basic() -> None:
def double(extract: list[int]) -> list[int]:
return [x * 2 for x in extract]
graph = px.Graph.from_specs([
px.TaskSpec("extract", extract),
px.TaskSpec("double", double, depends_on=("extract",)),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("extract", extract),
px.TaskSpec("double", double, depends_on=("extract",)),
]
)
report = px.run(graph, strategy="sequential")
assert report.success
assert report["extract"] == [1, 2, 3]
@@ -47,12 +49,14 @@ def test_sequential_diamond() -> None:
return fn
graph = px.Graph.from_specs([
px.TaskSpec("a", make("a")),
px.TaskSpec("b", make("b"), depends_on=("a",)),
px.TaskSpec("c", make("c"), depends_on=("a",)),
px.TaskSpec("d", make("d"), depends_on=("b", "c")),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("a", make("a")),
px.TaskSpec("b", make("b"), depends_on=("a",)),
px.TaskSpec("c", make("c"), depends_on=("a",)),
px.TaskSpec("d", make("d"), depends_on=("b", "c")),
]
)
report = px.run(graph, strategy="sequential")
assert report.success
assert report["d"] == "d"
@@ -66,10 +70,12 @@ def test_failure_propagates() -> None:
def downstream(_boom: None) -> int:
return 1
graph = px.Graph.from_specs([
px.TaskSpec("boom", boom),
px.TaskSpec("downstream", downstream, depends_on=("boom",)),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("boom", boom),
px.TaskSpec("downstream", downstream, depends_on=("boom",)),
]
)
with pytest.raises(TaskFailedError) as exc_info:
_ = px.run(graph, strategy="sequential")
assert exc_info.value.task == "boom"
@@ -85,9 +91,11 @@ def test_retries_then_succeeds() -> None:
raise RuntimeError("not yet")
return "ok"
graph = px.Graph.from_specs([
px.TaskSpec("flaky", flaky, retry=px.RetryPolicy(max_attempts=3)),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("flaky", flaky, retry=px.RetryPolicy(max_attempts=3)),
]
)
report = px.run(graph, strategy="sequential")
assert report.success
assert report["flaky"] == "ok"
@@ -105,9 +113,11 @@ def test_retries_with_delay() -> None:
raise RuntimeError("not yet")
return "ok"
graph = px.Graph.from_specs([
px.TaskSpec("flaky", flaky, retry=px.RetryPolicy(max_attempts=2, delay=0.1)),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("flaky", flaky, retry=px.RetryPolicy(max_attempts=2, delay=0.1)),
]
)
report = px.run(graph, strategy="sequential")
elapsed = time.time() - start_time
assert report.success
@@ -122,9 +132,11 @@ def test_timeout_then_retry_async(caplog: pytest.LogCaptureFixture) -> None:
await asyncio.sleep(10) # 会触发超时
return "ok"
graph = px.Graph.from_specs([
px.TaskSpec("slow", slow_task, timeout=0.2, retry=px.RetryPolicy(max_attempts=2)),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("slow", slow_task, timeout=0.2, retry=px.RetryPolicy(max_attempts=2)),
]
)
with caplog.at_level(logging.WARNING, logger="pyflowx"):
with pytest.raises(px.TaskFailedError) as exc_info:
_ = px.run(graph, strategy="async")
@@ -138,9 +150,11 @@ def test_retries_exhausted() -> None:
def always_fail() -> None:
raise RuntimeError("nope")
graph = px.Graph.from_specs([
px.TaskSpec("f", always_fail, retry=px.RetryPolicy(max_attempts=3)),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("f", always_fail, retry=px.RetryPolicy(max_attempts=3)),
]
)
with pytest.raises(TaskFailedError) as exc_info:
_ = px.run(graph, strategy="sequential")
assert exc_info.value.attempts == 3
@@ -155,11 +169,13 @@ def test_threaded_parallelism() -> None:
time.sleep(0.3)
return "done"
graph = px.Graph.from_specs([
px.TaskSpec("a", slow),
px.TaskSpec("b", slow),
px.TaskSpec("c", slow),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("a", slow),
px.TaskSpec("b", slow),
px.TaskSpec("c", slow),
]
)
start = time.time()
report = px.run(graph, strategy="thread", max_workers=3)
elapsed = time.time() - start
@@ -182,11 +198,13 @@ def test_threaded_layer_barrier() -> None:
return fn
graph = px.Graph.from_specs([
px.TaskSpec("a", make("a")),
px.TaskSpec("b", make("b")),
px.TaskSpec("c", make("c"), depends_on=("a", "b")),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("a", make("a")),
px.TaskSpec("b", make("b")),
px.TaskSpec("c", make("c"), depends_on=("a", "b")),
]
)
report = px.run(graph, strategy="thread", max_workers=2)
assert report.success
# c must finish after both a and b.
@@ -205,10 +223,12 @@ def test_async_basic() -> None:
async def transform(fetch: int) -> int:
return fetch * 2
graph = px.Graph.from_specs([
px.TaskSpec("fetch", fetch),
px.TaskSpec("transform", transform, depends_on=("fetch",)),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("fetch", fetch),
px.TaskSpec("transform", transform, depends_on=("fetch",)),
]
)
report = px.run(graph, strategy="async")
assert report.success
assert report["transform"] == 84
@@ -237,10 +257,12 @@ def test_async_mixed_sync_and_async() -> None:
await asyncio.sleep(0.01)
return sync_task + 5
graph = px.Graph.from_specs([
px.TaskSpec("sync_task", sync_task),
px.TaskSpec("async_task", async_task, depends_on=("sync_task",)),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("sync_task", sync_task),
px.TaskSpec("async_task", async_task, depends_on=("sync_task",)),
]
)
report = px.run(graph, strategy="async")
assert report.success
assert report["async_task"] == 15
@@ -288,10 +310,12 @@ def test_memory_backend_resume() -> None:
return fn
graph = px.Graph.from_specs([
px.TaskSpec("a", make("a")),
px.TaskSpec("b", make("b"), depends_on=("a",)),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("a", make("a")),
px.TaskSpec("b", make("b"), depends_on=("a",)),
]
)
backend = MemoryBackend()
_ = px.run(graph, strategy="sequential", state=backend)
assert runs == ["a", "b"]
@@ -377,9 +401,11 @@ def test_async_timeout_retry_then_succeed() -> None:
await asyncio.sleep(10) # 触发超时
return "ok"
graph = px.Graph.from_specs([
px.TaskSpec("a", flaky, retry=px.RetryPolicy(max_attempts=3), timeout=0.05),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("a", flaky, retry=px.RetryPolicy(max_attempts=3), timeout=0.05),
]
)
report = px.run(graph, strategy="async")
assert report.success
assert report["a"] == "ok"
@@ -396,9 +422,11 @@ def test_async_failure_retry_branch(caplog: pytest.LogCaptureFixture) -> None:
raise RuntimeError("not yet")
return "ok"
graph = px.Graph.from_specs([
px.TaskSpec("a", flaky, retry=px.RetryPolicy(max_attempts=3)),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("a", flaky, retry=px.RetryPolicy(max_attempts=3)),
]
)
with caplog.at_level("WARNING", logger="pyflowx"):
report = px.run(graph, strategy="async")
assert report.success
@@ -421,10 +449,12 @@ def test_threaded_skips_cached_tasks() -> None:
return fn
graph = px.Graph.from_specs([
px.TaskSpec("a", make("a")),
px.TaskSpec("b", make("b"), depends_on=("a",)),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("a", make("a")),
px.TaskSpec("b", make("b"), depends_on=("a",)),
]
)
backend = px.MemoryBackend()
# 第一次运行填充缓存
_ = px.run(graph, strategy="thread", max_workers=2, state=backend)
@@ -464,10 +494,12 @@ def test_async_skips_cached_tasks() -> None:
runs.append("b")
return a + "b"
graph = px.Graph.from_specs([
px.TaskSpec("a", a),
px.TaskSpec("b", b, depends_on=("a",)),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("a", a),
px.TaskSpec("b", b, depends_on=("a",)),
]
)
backend = px.MemoryBackend()
_ = px.run(graph, strategy="async", state=backend)
assert runs == ["a", "b"]
@@ -543,10 +575,12 @@ def test_downstream_skipped_when_upstream_skipped_sequential() -> None:
def downstream(upstream: str) -> str:
return upstream + "_processed"
graph = px.Graph.from_specs([
px.TaskSpec("upstream", cmd=["echo", "hello"], conditions=(never_true,)),
px.TaskSpec("downstream", downstream, depends_on=("upstream",)),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("upstream", cmd=["echo", "hello"], conditions=(never_true,)),
px.TaskSpec("downstream", downstream, depends_on=("upstream",)),
]
)
report = px.run(graph, strategy="sequential")
assert report.success
assert report.result_of("upstream").status == px.TaskStatus.SKIPPED
@@ -560,10 +594,12 @@ def test_downstream_skipped_when_upstream_skipped_thread() -> None:
def downstream(upstream: str) -> str:
return upstream + "_processed"
graph = px.Graph.from_specs([
px.TaskSpec("upstream", cmd=["echo", "hello"], conditions=(never_true,)),
px.TaskSpec("downstream", downstream, depends_on=("upstream",)),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("upstream", cmd=["echo", "hello"], conditions=(never_true,)),
px.TaskSpec("downstream", downstream, depends_on=("upstream",)),
]
)
report = px.run(graph, strategy="thread", max_workers=2)
assert report.success
assert report.result_of("upstream").status == px.TaskStatus.SKIPPED
@@ -581,10 +617,12 @@ def test_downstream_skipped_when_upstream_skipped_async() -> None:
never_true = lambda _ctx: False # noqa: E731
graph = px.Graph.from_specs([
px.TaskSpec("upstream", upstream, conditions=(never_true,)),
px.TaskSpec("downstream", downstream, depends_on=("upstream",)),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("upstream", upstream, conditions=(never_true,)),
px.TaskSpec("downstream", downstream, depends_on=("upstream",)),
]
)
report = px.run(graph, strategy="async")
assert report.success
assert report.result_of("upstream").status == px.TaskStatus.SKIPPED
@@ -601,10 +639,12 @@ def test_downstream_executes_when_upstream_succeeds() -> None:
def downstream(upstream: str) -> str:
return upstream + "_processed"
graph = px.Graph.from_specs([
px.TaskSpec("upstream", upstream, conditions=(always_true,)),
px.TaskSpec("downstream", downstream, depends_on=("upstream",)),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("upstream", upstream, conditions=(always_true,)),
px.TaskSpec("downstream", downstream, depends_on=("upstream",)),
]
)
report = px.run(graph, strategy="sequential")
assert report.success
assert report.result_of("upstream").status == px.TaskStatus.SUCCESS
+63 -43
View File
@@ -271,10 +271,12 @@ def test_allow_upstream_skip_allows_execution_after_skipped() -> None:
def downstream_task() -> str:
return "ran despite upstream skipped"
graph = px.Graph.from_specs([
px.TaskSpec("upstream", fn=lambda: "up", conditions=(never_true,)),
px.TaskSpec("downstream", fn=downstream_task, depends_on=("upstream",), allow_upstream_skip=True),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("upstream", fn=lambda: "up", conditions=(never_true,)),
px.TaskSpec("downstream", fn=downstream_task, depends_on=("upstream",), allow_upstream_skip=True),
]
)
report = px.run(graph, strategy="sequential")
assert report.success
assert report.results["upstream"].status == TaskStatus.SKIPPED
@@ -291,10 +293,12 @@ def test_upstream_failed_skips_downstream() -> None:
def downstream():
return "should not run"
graph = px.Graph.from_specs([
px.TaskSpec("upstream", fn=boom),
px.TaskSpec("downstream", fn=downstream, depends_on=("upstream",)),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("upstream", fn=boom),
px.TaskSpec("downstream", fn=downstream, depends_on=("upstream",)),
]
)
with pytest.raises(px.TaskFailedError):
px.run(graph, strategy="sequential")
@@ -342,11 +346,13 @@ def test_concurrency_key_thread() -> None:
return fn
graph = px.Graph.from_specs([
px.TaskSpec("a", fn=make("a"), concurrency_key="group1"),
px.TaskSpec("b", fn=make("b"), concurrency_key="group1"),
px.TaskSpec("c", fn=make("c"), concurrency_key="group1"),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("a", fn=make("a"), concurrency_key="group1"),
px.TaskSpec("b", fn=make("b"), concurrency_key="group1"),
px.TaskSpec("c", fn=make("c"), concurrency_key="group1"),
]
)
report = px.run(graph, strategy="thread", max_workers=10, concurrency_limits={"group1": 1})
assert report.success
# 由于 concurrency_key 限制为 1,任务应串行执行
@@ -366,10 +372,12 @@ def test_concurrency_key_async() -> None:
await asyncio.sleep(0.01)
return "b"
graph = px.Graph.from_specs([
px.TaskSpec("a", fn=task_a, concurrency_key="group1"),
px.TaskSpec("b", fn=task_b, concurrency_key="group1"),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("a", fn=task_a, concurrency_key="group1"),
px.TaskSpec("b", fn=task_b, concurrency_key="group1"),
]
)
report = px.run(graph, strategy="async", concurrency_limits={"group1": 1})
assert report.success
@@ -388,12 +396,14 @@ def test_dependency_strategy_basic() -> None:
return fn
graph = px.Graph.from_specs([
px.TaskSpec("a", fn=make("a")),
px.TaskSpec("b", fn=make("b"), depends_on=("a",)),
px.TaskSpec("c", fn=make("c"), depends_on=("a",)),
px.TaskSpec("d", fn=make("d"), depends_on=("b", "c")),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("a", fn=make("a")),
px.TaskSpec("b", fn=make("b"), depends_on=("a",)),
px.TaskSpec("c", fn=make("c"), depends_on=("a",)),
px.TaskSpec("d", fn=make("d"), depends_on=("b", "c")),
]
)
report = px.run(graph, strategy="dependency")
assert report.success
assert "a" in order
@@ -409,10 +419,12 @@ def test_dependency_strategy_async() -> None:
async def b(a: str):
return a + "b"
graph = px.Graph.from_specs([
px.TaskSpec("a", fn=a),
px.TaskSpec("b", fn=b, depends_on=("a",)),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("a", fn=a),
px.TaskSpec("b", fn=b, depends_on=("a",)),
]
)
report = px.run(graph, strategy="dependency")
assert report.success
assert report["b"] == "ab"
@@ -427,10 +439,12 @@ def test_continue_on_error_marks_failed_but_continues() -> None:
def boom():
raise ValueError("boom")
graph = px.Graph.from_specs([
px.TaskSpec("fail", fn=boom, continue_on_error=True),
px.TaskSpec("other", fn=lambda: "ok"), # 无依赖,应继续
])
graph = px.Graph.from_specs(
[
px.TaskSpec("fail", fn=boom, continue_on_error=True),
px.TaskSpec("other", fn=lambda: "ok"), # 无依赖,应继续
]
)
# continue_on_error=True 时 run 不抛异常,report.success 为 True
report = px.run(graph, strategy="sequential")
# report.success 为 True 因为没有抛 TaskFailedError
@@ -448,10 +462,12 @@ def test_continue_on_error_downstream_skipped() -> None:
def downstream():
return "should not run"
graph = px.Graph.from_specs([
px.TaskSpec("fail", fn=boom, continue_on_error=True),
px.TaskSpec("dep", fn=downstream, depends_on=("fail",), allow_upstream_skip=False),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("fail", fn=boom, continue_on_error=True),
px.TaskSpec("dep", fn=downstream, depends_on=("fail",), allow_upstream_skip=False),
]
)
report = px.run(graph, strategy="sequential")
# report.success 为 True 因为 continue_on_error 阻止了 TaskFailedError
assert report.success
@@ -468,10 +484,12 @@ def test_soft_depends_on_default_value_injection() -> None:
def task_with_soft_dep(a: str | None = None) -> str:
return f"a={a}"
graph = px.Graph.from_specs([
px.TaskSpec("a", fn=lambda: "value"),
px.TaskSpec("b", fn=task_with_soft_dep, soft_depends_on=("a",)),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("a", fn=lambda: "value"),
px.TaskSpec("b", fn=task_with_soft_dep, soft_depends_on=("a",)),
]
)
report = px.run(graph, strategy="sequential")
assert report.success
assert report["b"] == "a=value"
@@ -484,10 +502,12 @@ def test_soft_depends_on_skipped_injects_none() -> None:
def task_with_soft_dep(skipped: str | None = None) -> str:
return f"skipped={skipped}"
graph = px.Graph.from_specs([
px.TaskSpec("skipped", fn=lambda: "value", conditions=(never_true,)),
px.TaskSpec("b", fn=task_with_soft_dep, soft_depends_on=("skipped",)),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("skipped", fn=lambda: "value", conditions=(never_true,)),
px.TaskSpec("b", fn=task_with_soft_dep, soft_depends_on=("skipped",)),
]
)
report = px.run(graph, strategy="sequential")
assert report.success
# 软依赖被 skipped 时注入 None(因为 global_context 中有 skipped,值为 None
+98 -70
View File
@@ -14,11 +14,13 @@ def _fn() -> None:
def test_from_specs_builds_graph() -> None:
graph = px.Graph.from_specs([
px.TaskSpec("a", _fn),
px.TaskSpec("b", _fn, depends_on=("a",)),
px.TaskSpec("c", _fn, depends_on=("a", "b")),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("a", _fn),
px.TaskSpec("b", _fn, depends_on=("a",)),
px.TaskSpec("c", _fn, depends_on=("a", "b")),
]
)
assert set(graph.names) == {"a", "b", "c"}
assert graph.dependencies("c") == ("a", "b")
assert len(graph) == 3
@@ -27,19 +29,23 @@ def test_from_specs_builds_graph() -> None:
def test_from_specs_allows_forward_references() -> None:
# b depends on a, but a is declared after b — order should not matter.
graph = px.Graph.from_specs([
px.TaskSpec("b", _fn, depends_on=("a",)),
px.TaskSpec("a", _fn),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("b", _fn, depends_on=("a",)),
px.TaskSpec("a", _fn),
]
)
assert graph.layers() == [["a"], ["b"]]
def test_duplicate_task_raises() -> None:
with pytest.raises(DuplicateTaskError):
_ = px.Graph.from_specs([
px.TaskSpec("a", _fn),
px.TaskSpec("a", _fn),
])
_ = px.Graph.from_specs(
[
px.TaskSpec("a", _fn),
px.TaskSpec("a", _fn),
]
)
def test_missing_dependency_raises() -> None:
@@ -52,20 +58,24 @@ def test_missing_dependency_raises() -> None:
def test_cycle_detection() -> None:
with pytest.raises(CycleError):
_ = px.Graph.from_specs([
px.TaskSpec("a", _fn, depends_on=("c",)),
px.TaskSpec("b", _fn, depends_on=("a",)),
px.TaskSpec("c", _fn, depends_on=("b",)),
])
_ = px.Graph.from_specs(
[
px.TaskSpec("a", _fn, depends_on=("c",)),
px.TaskSpec("b", _fn, depends_on=("a",)),
px.TaskSpec("c", _fn, depends_on=("b",)),
]
)
def test_layers_grouping() -> None:
graph = px.Graph.from_specs([
px.TaskSpec("a", _fn),
px.TaskSpec("b", _fn),
px.TaskSpec("c", _fn, depends_on=("a", "b")),
px.TaskSpec("d", _fn, depends_on=("c",)),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("a", _fn),
px.TaskSpec("b", _fn),
px.TaskSpec("c", _fn, depends_on=("a", "b")),
px.TaskSpec("d", _fn, depends_on=("c",)),
]
)
layers = graph.layers()
assert layers == [["a", "b"], ["c"], ["d"]]
@@ -76,10 +86,12 @@ def test_self_dependency_rejected() -> None:
def test_to_mermaid() -> None:
graph = px.Graph.from_specs([
px.TaskSpec("a", _fn),
px.TaskSpec("b", _fn, depends_on=("a",)),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("a", _fn),
px.TaskSpec("b", _fn, depends_on=("a",)),
]
)
mermaid = graph.to_mermaid()
assert mermaid.startswith("graph TD")
assert 'a["a"]' in mermaid
@@ -93,11 +105,13 @@ def test_to_mermaid_invalid_orientation() -> None:
def test_subgraph_by_tags() -> None:
graph = px.Graph.from_specs([
px.TaskSpec("a", _fn, tags=("ingest",)),
px.TaskSpec("b", _fn, depends_on=("a",), tags=("ingest",)),
px.TaskSpec("c", _fn, depends_on=("b",), tags=("report",)),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("a", _fn, tags=("ingest",)),
px.TaskSpec("b", _fn, depends_on=("a",), tags=("ingest",)),
px.TaskSpec("c", _fn, depends_on=("b",), tags=("report",)),
]
)
sub = graph.subgraph(["ingest"])
assert set(sub.names) == {"a", "b"}
# Edge to dropped task c is removed; b no longer waits for anything
@@ -106,11 +120,13 @@ def test_subgraph_by_tags() -> None:
def test_subgraph_by_names() -> None:
graph = px.Graph.from_specs([
px.TaskSpec("a", _fn),
px.TaskSpec("b", _fn, depends_on=("a",)),
px.TaskSpec("c", _fn, depends_on=("b",)),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("a", _fn),
px.TaskSpec("b", _fn, depends_on=("a",)),
px.TaskSpec("c", _fn, depends_on=("b",)),
]
)
sub = graph.subgraph_by_names(["a", "b"])
assert set(sub.names) == {"a", "b"}
# c is dropped, so b's dep on c (none here) — but a->b edge preserved.
@@ -124,10 +140,12 @@ def test_subgraph_by_names_unknown() -> None:
def test_describe() -> None:
graph = px.Graph.from_specs([
px.TaskSpec("a", _fn),
px.TaskSpec("b", _fn, depends_on=("a",)),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("a", _fn),
px.TaskSpec("b", _fn, depends_on=("a",)),
]
)
desc = graph.describe()
assert "Layer 1" in desc
assert "Layer 2" in desc
@@ -164,11 +182,13 @@ def test_all_specs_returns_view() -> None:
def test_all_deps_combines_hard_and_soft() -> None:
"""all_deps 应返回硬依赖 + 软依赖的组合。"""
graph = px.Graph.from_specs([
px.TaskSpec("a", _fn),
px.TaskSpec("b", _fn),
px.TaskSpec("c", _fn, depends_on=("a",), soft_depends_on=("b",)),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("a", _fn),
px.TaskSpec("b", _fn),
px.TaskSpec("c", _fn, depends_on=("a",), soft_depends_on=("b",)),
]
)
all_deps = graph.all_deps("c")
assert set(all_deps) == {"a", "b"}
# 硬依赖在前,软依赖在后
@@ -183,10 +203,12 @@ def test_spec_accessor() -> None:
def test_dependencies_accessor() -> None:
graph = px.Graph.from_specs([
px.TaskSpec("a", _fn),
px.TaskSpec("b", _fn, depends_on=("a",)),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("a", _fn),
px.TaskSpec("b", _fn, depends_on=("a",)),
]
)
assert graph.dependencies("a") == ()
assert graph.dependencies("b") == ("a",)
@@ -205,16 +227,18 @@ def test_empty_graph_layers() -> None:
def test_subgraph_preserves_metadata() -> None:
"""子图应保留原任务的 retry/timeout/tags 等元数据。"""
graph = px.Graph.from_specs([
px.TaskSpec(
"a",
_fn,
tags=("x",),
retry=px.RetryPolicy(max_attempts=3),
timeout=5.0,
),
px.TaskSpec("b", _fn, depends_on=("a",), tags=("y",)),
])
graph = px.Graph.from_specs(
[
px.TaskSpec(
"a",
_fn,
tags=("x",),
retry=px.RetryPolicy(max_attempts=3),
timeout=5.0,
),
px.TaskSpec("b", _fn, depends_on=("a",), tags=("y",)),
]
)
sub = graph.subgraph(["x"])
spec = sub.spec("a")
assert spec.retry.max_attempts == 3
@@ -250,10 +274,12 @@ def test_from_specs_with_invalid_type() -> None:
# ---------------------------------------------------------------------- #
def test_to_mermaid_soft_depends_on() -> None:
"""to_mermaid 应正确绘制软依赖为虚线."""
graph = px.Graph.from_specs([
px.TaskSpec("a", _fn),
px.TaskSpec("b", _fn, soft_depends_on=("a",)),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("a", _fn),
px.TaskSpec("b", _fn, soft_depends_on=("a",)),
]
)
mermaid = graph.to_mermaid()
assert "a -.-> b" in mermaid # 软依赖用虚线
@@ -355,11 +381,13 @@ def test_graph_composer_expand_refs_ref_returns_empty() -> None:
def test_graph_composer_expand_refs_multiple_original_specs_serialized() -> None:
"""expand_refs 多个 original_specs 应串行依赖,且首个依赖 ref 末任务."""
graph_a = px.Graph.from_specs([px.TaskSpec("a1", _fn)])
graph_b = px.Graph.from_specs([
px.TaskSpec("b1", _fn),
px.TaskSpec("b2", _fn),
px.TaskSpec("b3", _fn),
])
graph_b = px.Graph.from_specs(
[
px.TaskSpec("b1", _fn),
px.TaskSpec("b2", _fn),
px.TaskSpec("b3", _fn),
]
)
graph_b._pending_refs = ["cmd_a"]
composer = GraphComposer({"cmd_a": graph_a, "cmd_b": graph_b})
+62 -44
View File
@@ -95,11 +95,13 @@ class TestProfileReportConstruction:
report.results["a"] = _result("a", start, 1.0)
report.results["b"] = _result("b", start + timedelta(seconds=1), 2.0)
report.results["c"] = _result("c", start + timedelta(seconds=3), 1.5)
graph = px.Graph.from_specs([
_spec("a"),
_spec("b", deps=("a",)),
_spec("c", deps=("b",)),
])
graph = px.Graph.from_specs(
[
_spec("a"),
_spec("b", deps=("a",)),
_spec("c", deps=("b",)),
]
)
profile = ProfileReport.from_report(report, graph)
@@ -136,11 +138,13 @@ class TestProfileReportConstruction:
report.results["a"] = _result("a", start, 1.0)
report.results["b"] = _result("b", start, 3.0)
report.results["c"] = _result("c", start + timedelta(seconds=3), 1.0)
graph = px.Graph.from_specs([
_spec("a"),
_spec("b"),
_spec("c", deps=("a", "b")),
])
graph = px.Graph.from_specs(
[
_spec("a"),
_spec("b"),
_spec("c", deps=("a", "b")),
]
)
profile = ProfileReport.from_report(report, graph)
@@ -188,10 +192,12 @@ class TestWaitTime:
report = px.RunReport()
report.results["a"] = _result("a", start, 1.0)
report.results["b"] = _result("b", start + timedelta(seconds=1.5), 1.0)
graph = px.Graph.from_specs([
_spec("a"),
_spec("b", deps=("a",)),
])
graph = px.Graph.from_specs(
[
_spec("a"),
_spec("b", deps=("a",)),
]
)
profile = ProfileReport.from_report(report, graph)
@@ -204,10 +210,12 @@ class TestWaitTime:
report.results["a"] = _result("a", start, 2.0)
# b 在 a 还没完成时就开始(不应该但可能发生)
report.results["b"] = _result("b", start + timedelta(seconds=1), 1.0)
graph = px.Graph.from_specs([
_spec("a"),
_spec("b", deps=("a",)),
])
graph = px.Graph.from_specs(
[
_spec("a"),
_spec("b", deps=("a",)),
]
)
profile = ProfileReport.from_report(report, graph)
@@ -220,10 +228,12 @@ class TestWaitTime:
report = px.RunReport()
report.results["a"] = _result("a", start, 1.0)
report.results["b"] = _skipped_result("b")
graph = px.Graph.from_specs([
_spec("a"),
_spec("b", deps=("a",)),
])
graph = px.Graph.from_specs(
[
_spec("a"),
_spec("b", deps=("a",)),
]
)
profile = ProfileReport.from_report(report, graph)
@@ -241,12 +251,14 @@ class TestCriticalPath:
report.results["b"] = _result("b", start + timedelta(seconds=1), 3.0)
report.results["c"] = _result("c", start + timedelta(seconds=1), 1.0)
report.results["d"] = _result("d", start + timedelta(seconds=4), 1.0)
graph = px.Graph.from_specs([
_spec("a"),
_spec("b", deps=("a",)),
_spec("c", deps=("a",)),
_spec("d", deps=("b", "c")),
])
graph = px.Graph.from_specs(
[
_spec("a"),
_spec("b", deps=("a",)),
_spec("c", deps=("a",)),
_spec("d", deps=("b", "c")),
]
)
profile = ProfileReport.from_report(report, graph)
@@ -393,12 +405,14 @@ class TestQueries:
report.results["b"] = _result("b", start + timedelta(seconds=1), 3.0)
report.results["c"] = _result("c", start + timedelta(seconds=1), 1.0)
report.results["d"] = _result("d", start + timedelta(seconds=4), 1.0)
graph = px.Graph.from_specs([
_spec("a"),
_spec("b", deps=("a",)),
_spec("c", deps=("a",)),
_spec("d", deps=("b", "c")),
])
graph = px.Graph.from_specs(
[
_spec("a"),
_spec("b", deps=("a",)),
_spec("c", deps=("a",)),
_spec("d", deps=("b", "c")),
]
)
profile = ProfileReport.from_report(report, graph)
@@ -537,11 +551,13 @@ class TestIntegrationWithRun:
time.sleep(0.01) # 确保任务有实际耗时,避免 duration 极小导致并行度计算为 0
return 1
graph = px.Graph.from_specs([
px.TaskSpec("a", slow),
px.TaskSpec("b", slow, depends_on=("a",)),
px.TaskSpec("c", slow, depends_on=("a",)),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("a", slow),
px.TaskSpec("b", slow, depends_on=("a",)),
px.TaskSpec("c", slow, depends_on=("a",)),
]
)
report = px.run(graph, strategy="sequential")
profile = ProfileReport.from_report(report, graph)
@@ -560,11 +576,13 @@ class TestIntegrationWithRun:
time.sleep(0.05)
return 1
graph = px.Graph.from_specs([
px.TaskSpec("a", slow),
px.TaskSpec("b", slow),
px.TaskSpec("c", slow),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("a", slow),
px.TaskSpec("b", slow),
px.TaskSpec("c", slow),
]
)
report = px.run(graph, strategy="thread", max_workers=3)
profile = ProfileReport.from_report(report, graph)
+253
View File
@@ -0,0 +1,253 @@
"""``pyflowx.registry`` 模块的单元测试."""
from __future__ import annotations
from typing import Any
import pytest
import pyflowx as px
from pyflowx.registry import _REGISTRY, FnRegistry, get_fn, has_fn, register_fn
@pytest.fixture
def clear_registry() -> Any:
"""每个测试前后清空 registry, 但保留 ops 模块自动注册的函数.
ops 模块在首次导入时注册函数, 模块不会重新执行, 因此清空后无法重新注册.
保存原始状态并在 teardown 时恢复, 确保 TestOpsModules 等不使用此 fixture
测试仍能看到 ops 的注册.
"""
saved = dict(_REGISTRY)
_REGISTRY.clear()
yield
_REGISTRY.clear()
_REGISTRY.update(saved)
class TestRegisterFn:
"""``register_fn`` 装饰器测试."""
@pytest.fixture(autouse=True)
def _clear(self, clear_registry: Any) -> None:
"""每个测试前后清空 registry."""
def test_register_with_explicit_name(self) -> None:
"""使用显式名称注册函数."""
@register_fn("custom_name")
def _func(x: int) -> int:
return x * 2
assert has_fn("custom_name")
assert get_fn("custom_name")(5) == 10
def test_register_without_parentheses(self) -> None:
"""无括号使用 ``@register_fn`` 时以 ``__name__`` 注册."""
@register_fn
def my_func(x: int) -> int:
return x + 1
assert has_fn("my_func")
assert get_fn("my_func")(10) == 11
def test_register_with_none_uses_func_name(self) -> None:
"""``@register_fn(None)`` 等价于使用 ``__name__``."""
@register_fn(None)
def auto_named(x: int) -> int:
return x
assert has_fn("auto_named")
def test_register_preserves_function(self) -> None:
"""装饰后函数仍可直接调用."""
@register_fn("calc")
def _calc(a: int, b: int) -> int:
return a + b
assert _calc(2, 3) == 5
def test_duplicate_registration_raises(self) -> None:
"""重复注册同名函数抛出 ValueError."""
with pytest.raises(ValueError, match="已注册"):
@register_fn("dup")
def _first() -> None:
pass
@register_fn("dup")
def _second() -> None:
pass
def test_duplicate_registration_without_parentheses(self) -> None:
"""无括号模式下重复注册也抛出 ValueError."""
with pytest.raises(ValueError, match="已注册"):
@register_fn
def shared_name() -> None:
pass
@register_fn
def shared_name() -> None: # noqa: F811
pass
class TestGetFn:
"""``get_fn`` 函数测试."""
@pytest.fixture(autouse=True)
def _clear(self, clear_registry: Any) -> None:
"""每个测试前后清空 registry."""
def test_get_registered_function(self) -> None:
"""获取已注册的函数."""
@register_fn("target")
def _target(x: int) -> int:
return x * 3
retrieved = get_fn("target")
assert retrieved is _target
def test_get_unregistered_raises_keyerror(self) -> None:
"""获取未注册的函数抛出 KeyError."""
with pytest.raises(KeyError, match="not_found"):
get_fn("not_found")
class TestHasFn:
"""``has_fn`` 函数测试."""
@pytest.fixture(autouse=True)
def _clear(self, clear_registry: Any) -> None:
"""每个测试前后清空 registry."""
def test_has_registered(self) -> None:
"""已注册返回 True."""
@register_fn("exists")
def _exists() -> None:
pass
assert has_fn("exists") is True
def test_has_not_registered(self) -> None:
"""未注册返回 False."""
assert has_fn("nope") is False
class TestFnRegistry:
"""``FnRegistry`` 类测试."""
@pytest.fixture(autouse=True)
def _clear(self, clear_registry: Any) -> None:
"""每个测试前后清空 registry."""
def test_register_method(self) -> None:
"""``FnRegistry.register`` 等价于 ``register_fn``."""
@FnRegistry.register("via_class")
def _func(x: int) -> int:
return x
assert FnRegistry.has("via_class")
assert FnRegistry.get("via_class")(7) == 7
def test_register_method_without_name(self) -> None:
"""``FnRegistry.register(None)`` 使用函数名."""
@FnRegistry.register()
def class_auto(x: int) -> int:
return x
assert FnRegistry.has("class_auto")
def test_get_method(self) -> None:
"""``FnRegistry.get`` 获取已注册函数."""
@register_fn("klass")
def _klass() -> str:
return "hello"
assert FnRegistry.get("klass")() == "hello"
def test_has_method_false(self) -> None:
"""``FnRegistry.has`` 未注册返回 False."""
assert FnRegistry.has("missing") is False
def test_clear_method(self) -> None:
"""``FnRegistry.clear`` 清空注册表."""
@register_fn("temp")
def _temp() -> None:
pass
assert FnRegistry.has("temp")
FnRegistry.clear()
assert not FnRegistry.has("temp")
def test_names_method(self) -> None:
"""``FnRegistry.names`` 返回所有注册名."""
@register_fn("alpha")
def _alpha() -> None:
pass
@register_fn("beta")
def _beta() -> None:
pass
names = FnRegistry.names()
assert "alpha" in names
assert "beta" in names
def test_names_returns_copy(self) -> None:
"""``names`` 返回列表副本, 修改不影响内部状态."""
@register_fn("orig")
def _orig() -> None:
pass
names = FnRegistry.names()
names.append("fake")
assert not FnRegistry.has("fake")
class TestOpsModules:
"""``ops`` 子模块导入后函数自动注册的集成测试.
这些测试不使用 ``_clear_registry`` fixture, 因为 ``ops`` 模块在首次导入时
注册函数, 清空后将无法重新注册 (模块不会再次执行).
"""
def test_all_ops_functions_registered(self) -> None:
"""导入 ``ops`` 后所有函数应已注册."""
import inspect
from pyflowx.ops import dev, files, media, system
for module in (files, dev, media, system):
for name in module.__all__:
obj = getattr(module, name)
if not inspect.isfunction(obj):
continue
assert px.has_fn(name), f"{module.__name__}.{name} 未注册"
def test_total_function_count(self) -> None:
"""注册函数总数 = 18+15+18+12 = 63."""
from pyflowx.ops import dev, files, media, system # noqa: F401
all_names = px.FnRegistry.names()
assert len(all_names) == 63
def test_specific_functions_callable(self) -> None:
"""关键注册函数可调用."""
from pyflowx.ops import dev, files, media, system
assert px.get_fn("get_file_timestamp") is files.get_file_timestamp
assert px.get_fn("bump_file_version") is dev.bump_file_version
assert px.get_fn("pdf_merge") is media.pdf_merge
assert px.get_fn("ssh_copy_id") is system.ssh_copy_id
-1
View File
@@ -172,4 +172,3 @@ class TestRunReportQueries:
report.results["a"] = TaskResult[Any](spec=spec, status=TaskStatus.PENDING)
durs = report.durations()
assert durs["a"] == 0.0
+55 -70
View File
@@ -29,20 +29,24 @@ def _echo_graph(name: str = "echo_task", msg: str = "hello") -> px.Graph:
def _failing_graph() -> px.Graph:
"""构造一个必定失败的单任务图."""
return px.Graph.from_specs([
px.TaskSpec(
"fail",
cmd=["python", "-c", "import sys; sys.exit(1)"],
)
])
return px.Graph.from_specs(
[
px.TaskSpec(
"fail",
cmd=["python", "-c", "import sys; sys.exit(1)"],
)
]
)
def _multi_task_graph() -> px.Graph:
"""构造一个带依赖的多任务图."""
return px.Graph.from_specs([
px.TaskSpec("a", cmd=[*ECHO_CMD, "a"]),
px.TaskSpec("b", cmd=[*ECHO_CMD, "b"], depends_on=("a",)),
])
return px.Graph.from_specs(
[
px.TaskSpec("a", cmd=[*ECHO_CMD, "a"]),
px.TaskSpec("b", cmd=[*ECHO_CMD, "b"], depends_on=("a",)),
]
)
# ---------------------------------------------------------------------- #
@@ -314,13 +318,15 @@ class TestCliRunnerVerbose:
def test_verbose_prints_skip_lifecycle(self, capsys: pytest.CaptureFixture[str]) -> None:
"""verbose 模式下跳过的任务应打印跳过信息."""
graph = px.Graph.from_specs([
px.TaskSpec(
"skip_me",
cmd=[*ECHO_CMD, "skip"],
conditions=(lambda _ctx: False,),
),
])
graph = px.Graph.from_specs(
[
px.TaskSpec(
"skip_me",
cmd=[*ECHO_CMD, "skip"],
conditions=(lambda _ctx: False,),
),
]
)
runner = px.CliRunner(aliases={"skip": graph})
_ = runner.run(["skip"])
captured = capsys.readouterr()
@@ -517,26 +523,30 @@ class TestCliRunnerIntegration:
def test_condition_skipped_command_succeeds(self) -> None:
"""条件不满足时任务跳过, 整体仍成功."""
graph = px.Graph.from_specs([
px.TaskSpec(
"skip_me",
cmd=[*ECHO_CMD, "should not run"],
conditions=(lambda _ctx: False,),
),
])
graph = px.Graph.from_specs(
[
px.TaskSpec(
"skip_me",
cmd=[*ECHO_CMD, "should not run"],
conditions=(lambda _ctx: False,),
),
]
)
runner = px.CliRunner(aliases={"skip": graph})
exit_code = runner.run(["skip"])
assert exit_code == CliExitCode.SUCCESS.value
def test_condition_met_command_succeeds(self) -> None:
"""条件满足时任务执行, 整体成功."""
graph = px.Graph.from_specs([
px.TaskSpec(
"run_me",
cmd=[*ECHO_CMD, "should run"],
conditions=(lambda _ctx: True,),
),
])
graph = px.Graph.from_specs(
[
px.TaskSpec(
"run_me",
cmd=[*ECHO_CMD, "should run"],
conditions=(lambda _ctx: True,),
),
]
)
runner = px.CliRunner(aliases={"run": graph})
exit_code = runner.run(["run"])
assert exit_code == CliExitCode.SUCCESS.value
@@ -552,12 +562,14 @@ class TestCliRunnerIntegration:
return fn
graph = px.Graph.from_specs([
px.TaskSpec("a", make("a")),
px.TaskSpec("b", make("b"), depends_on=("a",)),
px.TaskSpec("c", make("c"), depends_on=("a",)),
px.TaskSpec("d", make("d"), depends_on=("b", "c")),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("a", make("a")),
px.TaskSpec("b", make("b"), depends_on=("a",)),
px.TaskSpec("c", make("c"), depends_on=("a",)),
px.TaskSpec("d", make("d"), depends_on=("b", "c")),
]
)
runner = px.CliRunner(aliases={"diamond": graph})
exit_code = runner.run(["diamond"])
assert exit_code == CliExitCode.SUCCESS.value
@@ -596,35 +608,6 @@ class TestCliRunnerIntegration:
assert exit_code == CliExitCode.SUCCESS.value
# ---------------------------------------------------------------------- #
# _apply_verbose_to_graph (补充覆盖)
# ---------------------------------------------------------------------- #
class TestApplyVerboseToGraph:
"""测试 _apply_verbose_to_graph 函数 (补充覆盖)."""
def test_specs_with_matching_verbose_are_kept(self) -> None:
"""spec.verbose 已与目标值匹配时应保留原 spec (覆盖 runner.py line 57)."""
from pyflowx.runner import _apply_verbose_to_graph
# 创建 verbose=True 的 spec
graph = px.Graph.from_specs([px.TaskSpec("a", cmd=[*ECHO_CMD, "a"], verbose=True)])
# 应用 verbose=True, spec.verbose 已匹配, 应保留原 spec
new_graph = _apply_verbose_to_graph(graph, verbose=True)
new_spec = new_graph.spec("a")
assert new_spec.verbose is True
def test_specs_with_non_matching_verbose_are_replaced(self) -> None:
"""spec.verbose 与目标值不匹配时应替换 (覆盖 else 分支)."""
from pyflowx.runner import _apply_verbose_to_graph
# 创建 verbose=False 的 spec
graph = px.Graph.from_specs([px.TaskSpec("a", cmd=[*ECHO_CMD, "a"], verbose=False)])
# 应用 verbose=True, spec.verbose 不匹配, 应替换
new_graph = _apply_verbose_to_graph(graph, verbose=True)
new_spec = new_graph.spec("a")
assert new_spec.verbose is True
# ---------------------------------------------------------------------- #
# 新 API: tasks + aliases
# ---------------------------------------------------------------------- #
@@ -661,10 +644,12 @@ class TestCliRunnerNewApi:
def test_aliases_graph_value(self) -> None:
"""aliases 值为 Graph 时原样使用(复杂场景:conditions 等)."""
graph = px.Graph.from_specs([
px.TaskSpec("a", cmd=[*ECHO_CMD, "a"]),
px.TaskSpec("b", cmd=[*ECHO_CMD, "b"], depends_on=("a",)),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("a", cmd=[*ECHO_CMD, "a"]),
px.TaskSpec("b", cmd=[*ECHO_CMD, "b"], depends_on=("a",)),
]
)
runner = px.CliRunner(aliases={"g": graph})
assert set(runner.graphs["g"].all_specs().keys()) == {"a", "b"}
+4 -2
View File
@@ -193,8 +193,10 @@ def test_should_execute_skip_if_missing_cmd_not_found() -> None:
def test_should_execute_skip_if_missing_cmd_found() -> None:
"""skip_if_missing 但命令存在时应执行."""
# 使用 Python 作为已安装的命令(Windows 上 echo 是 shell 内置,shutil.which 找不到)
spec = TaskSpec("a", cmd=["python"], skip_if_missing=True) # python 应存在
# 使用 sys.executable 确保命令一定存在 (跨平台兼容)
import sys
spec = TaskSpec("a", cmd=[sys.executable], skip_if_missing=True)
should_run, reason = spec.should_execute({})
assert should_run is True
assert reason is None
+178 -142
View File
@@ -21,9 +21,11 @@ else:
def test_taskspec_with_cmd_list():
"""测试使用命令列表的 TaskSpec."""
graph = px.Graph.from_specs([
px.TaskSpec("echo_test", cmd=[*ECHO_CMD, "hello"]),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("echo_test", cmd=[*ECHO_CMD, "hello"]),
]
)
report = px.run(graph, strategy="sequential")
assert report.success
@@ -38,9 +40,11 @@ def test_taskspec_with_cmd_string():
else:
shell_cmd = "echo 'hello from shell'"
graph = px.Graph.from_specs([
px.TaskSpec("shell_test", cmd=shell_cmd),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("shell_test", cmd=shell_cmd),
]
)
report = px.run(graph, strategy="sequential")
assert report.success
@@ -55,13 +59,15 @@ def test_taskspec_with_conditions_skip():
def never_true(_ctx):
return False
graph = px.Graph.from_specs([
px.TaskSpec(
"should_skip",
cmd=[*ECHO_CMD, "this should not run"],
conditions=(never_true,),
),
])
graph = px.Graph.from_specs(
[
px.TaskSpec(
"should_skip",
cmd=[*ECHO_CMD, "this should not run"],
conditions=(never_true,),
),
]
)
report = px.run(graph, strategy="sequential")
assert report.success
@@ -76,13 +82,15 @@ def test_taskspec_with_conditions_execute():
def always_true(_ctx):
return True
graph = px.Graph.from_specs([
px.TaskSpec(
"should_run",
cmd=[*ECHO_CMD, "this should run"],
conditions=(always_true,),
),
])
graph = px.Graph.from_specs(
[
px.TaskSpec(
"should_run",
cmd=[*ECHO_CMD, "this should run"],
conditions=(always_true,),
),
]
)
report = px.run(graph, strategy="sequential")
assert report.success
@@ -99,23 +107,25 @@ def test_platform_conditions():
win_cmd = ["echo", "Windows"]
posix_cmd = ["echo", "POSIX"]
graph = px.Graph.from_specs([
px.TaskSpec(
"win_task",
cmd=win_cmd,
conditions=(lambda _ctx: Constants.IS_WINDOWS,),
),
px.TaskSpec(
"linux_task",
cmd=posix_cmd,
conditions=(lambda _ctx: Constants.IS_LINUX,),
),
px.TaskSpec(
"macos_task",
cmd=posix_cmd,
conditions=(lambda _ctx: Constants.IS_MACOS,),
),
])
graph = px.Graph.from_specs(
[
px.TaskSpec(
"win_task",
cmd=win_cmd,
conditions=(lambda _ctx: Constants.IS_WINDOWS,),
),
px.TaskSpec(
"linux_task",
cmd=posix_cmd,
conditions=(lambda _ctx: Constants.IS_LINUX,),
),
px.TaskSpec(
"macos_task",
cmd=posix_cmd,
conditions=(lambda _ctx: Constants.IS_MACOS,),
),
]
)
report = px.run(graph, strategy="sequential")
assert report.success
@@ -141,13 +151,15 @@ def test_app_installed_conditions():
python_cmd = [sys.executable, "--version"]
py_name = "python" if sys.platform == "win32" else "python3"
graph = px.Graph.from_specs([
px.TaskSpec(
"python_check",
cmd=python_cmd,
conditions=(BuiltinConditions.HAS_INSTALLED(py_name),),
),
])
graph = px.Graph.from_specs(
[
px.TaskSpec(
"python_check",
cmd=python_cmd,
conditions=(BuiltinConditions.HAS_INSTALLED(py_name),),
),
]
)
report = px.run(graph, strategy="sequential")
assert report.success
@@ -173,23 +185,25 @@ def test_combined_conditions():
# NOT 条件
not_condition = BuiltinConditions.NOT(lambda _ctx: False)
graph = px.Graph.from_specs([
px.TaskSpec(
"and_test",
cmd=[*ECHO_CMD, "AND"],
conditions=(and_condition,),
),
px.TaskSpec(
"or_test",
cmd=[*ECHO_CMD, "OR"],
conditions=(or_condition,),
),
px.TaskSpec(
"not_test",
cmd=[*ECHO_CMD, "NOT"],
conditions=(not_condition,),
),
])
graph = px.Graph.from_specs(
[
px.TaskSpec(
"and_test",
cmd=[*ECHO_CMD, "AND"],
conditions=(and_condition,),
),
px.TaskSpec(
"or_test",
cmd=[*ECHO_CMD, "OR"],
conditions=(or_condition,),
),
px.TaskSpec(
"not_test",
cmd=[*ECHO_CMD, "NOT"],
conditions=(not_condition,),
),
]
)
report = px.run(graph, strategy="sequential")
assert report.success
@@ -205,13 +219,15 @@ def test_taskspec_with_cwd():
else:
ls_cmd = ["ls", "-la"]
graph = px.Graph.from_specs([
px.TaskSpec(
"list_current",
cmd=ls_cmd,
cwd=Path.cwd(),
),
])
graph = px.Graph.from_specs(
[
px.TaskSpec(
"list_current",
cmd=ls_cmd,
cwd=Path.cwd(),
),
]
)
report = px.run(graph, strategy="sequential")
assert report.success
@@ -222,14 +238,16 @@ def test_taskspec_with_cwd():
@pytest.mark.slow
def test_taskspec_with_timeout():
"""测试超时设置."""
graph = px.Graph.from_specs([
# 短时间任务应该成功
px.TaskSpec(
"short_task",
cmd=[sys.executable, "-c", "import time; time.sleep(0.1)"],
timeout=1.0,
),
])
graph = px.Graph.from_specs(
[
# 短时间任务应该成功
px.TaskSpec(
"short_task",
cmd=[sys.executable, "-c", "import time; time.sleep(0.1)"],
timeout=1.0,
),
]
)
report = px.run(graph, strategy="sequential")
assert report.success
@@ -239,24 +257,26 @@ def test_taskspec_with_timeout():
def test_taskspec_dependency_with_conditions():
"""测试依赖和条件的组合."""
graph = px.Graph.from_specs([
px.TaskSpec(
"first",
cmd=[*ECHO_CMD, "first"],
conditions=(lambda _ctx: True,),
),
px.TaskSpec(
"second",
cmd=[*ECHO_CMD, "second"],
depends_on=("first",),
conditions=(lambda _ctx: True,),
),
px.TaskSpec(
"third",
cmd=[*ECHO_CMD, "third"],
depends_on=("second",),
),
])
graph = px.Graph.from_specs(
[
px.TaskSpec(
"first",
cmd=[*ECHO_CMD, "first"],
conditions=(lambda _ctx: True,),
),
px.TaskSpec(
"second",
cmd=[*ECHO_CMD, "second"],
depends_on=("first",),
conditions=(lambda _ctx: True,),
),
px.TaskSpec(
"third",
cmd=[*ECHO_CMD, "third"],
depends_on=("second",),
),
]
)
report = px.run(graph, strategy="sequential")
assert report.success
@@ -271,10 +291,12 @@ def test_taskspec_mixed_fn_and_cmd():
def my_function():
return "result from function"
graph = px.Graph.from_specs([
px.TaskSpec("fn_task", fn=my_function),
px.TaskSpec("cmd_task", cmd=[*ECHO_CMD, "from command"]),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("fn_task", fn=my_function),
px.TaskSpec("cmd_task", cmd=[*ECHO_CMD, "from command"]),
]
)
report = px.run(graph, strategy="sequential")
assert report.success
@@ -289,13 +311,15 @@ def test_taskspec_cmd_overrides_fn():
def my_function():
return "should not run"
graph = px.Graph.from_specs([
px.TaskSpec(
"cmd_priority",
fn=my_function,
cmd=[*ECHO_CMD, "cmd takes priority"],
),
])
graph = px.Graph.from_specs(
[
px.TaskSpec(
"cmd_priority",
fn=my_function,
cmd=[*ECHO_CMD, "cmd takes priority"],
),
]
)
report = px.run(graph, strategy="sequential")
assert report.success
@@ -310,9 +334,11 @@ def test_taskspec_callable_cmd():
def my_callable():
return "callable result"
graph = px.Graph.from_specs([
px.TaskSpec("callable_cmd", cmd=my_callable),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("callable_cmd", cmd=my_callable),
]
)
report = px.run(graph, strategy="sequential")
assert report.success
@@ -373,13 +399,15 @@ class TestTaskSpecVerbose:
"""verbose=True 时失败也应打印返回码."""
from pyflowx.errors import TaskFailedError
graph = px.Graph.from_specs([
px.TaskSpec(
"fail",
cmd=[sys.executable, "-c", "import sys; sys.exit(1)"],
verbose=True,
)
])
graph = px.Graph.from_specs(
[
px.TaskSpec(
"fail",
cmd=[sys.executable, "-c", "import sys; sys.exit(1)"],
verbose=True,
)
]
)
with pytest.raises(TaskFailedError):
_ = px.run(graph, strategy="sequential")
captured = capsys.readouterr()
@@ -408,16 +436,18 @@ class TestTaskSpecCmdErrors:
"""命令失败时错误信息应包含 stderr."""
from pyflowx.errors import TaskFailedError
graph = px.Graph.from_specs([
px.TaskSpec(
"fail",
cmd=[
sys.executable,
"-c",
"import sys; sys.stderr.write('error-msg'); sys.exit(1)",
],
)
])
graph = px.Graph.from_specs(
[
px.TaskSpec(
"fail",
cmd=[
sys.executable,
"-c",
"import sys; sys.stderr.write('error-msg'); sys.exit(1)",
],
)
]
)
with pytest.raises(TaskFailedError) as exc_info:
_ = px.run(graph, strategy="sequential")
# 非 verbose 模式下, stderr 应包含在错误信息中
@@ -435,9 +465,11 @@ class TestTaskSpecCmdErrors:
"""shell 命令失败时应抛出 RuntimeError."""
from pyflowx.errors import TaskFailedError
graph = px.Graph.from_specs([
px.TaskSpec("fail", cmd=f'{sys.executable} -c "import sys; sys.exit(1)"'),
])
graph = px.Graph.from_specs(
[
px.TaskSpec("fail", cmd=f'{sys.executable} -c "import sys; sys.exit(1)"'),
]
)
with pytest.raises(TaskFailedError) as exc_info:
_ = px.run(graph, strategy="sequential")
assert "Shell 命令执行失败" in str(exc_info.value.cause)
@@ -447,13 +479,15 @@ class TestTaskSpecCmdErrors:
"""命令超时应抛出 RuntimeError."""
from pyflowx.errors import TaskFailedError
graph = px.Graph.from_specs([
px.TaskSpec(
"slow",
cmd=[sys.executable, "-c", "import time; time.sleep(5)"],
timeout=0.1,
)
])
graph = px.Graph.from_specs(
[
px.TaskSpec(
"slow",
cmd=[sys.executable, "-c", "import time; time.sleep(5)"],
timeout=0.1,
)
]
)
with pytest.raises(TaskFailedError) as exc_info:
_ = px.run(graph, strategy="sequential")
assert "超时" in str(exc_info.value.cause)
@@ -463,13 +497,15 @@ class TestTaskSpecCmdErrors:
"""shell 命令超时应抛出 RuntimeError."""
from pyflowx.errors import TaskFailedError
graph = px.Graph.from_specs([
px.TaskSpec(
"slow",
cmd=f'{sys.executable} -c "import time; time.sleep(5)"',
timeout=0.1,
),
])
graph = px.Graph.from_specs(
[
px.TaskSpec(
"slow",
cmd=f'{sys.executable} -c "import time; time.sleep(5)"',
timeout=0.1,
),
]
)
with pytest.raises(TaskFailedError) as exc_info:
_ = px.run(graph, strategy="sequential")
assert "超时" in str(exc_info.value.cause)
File diff suppressed because it is too large Load Diff

Some files were not shown because too many files have changed in this diff Show More