Compare commits
33 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| c9c7529c58 | |||
| c498d9b1c9 | |||
| b36e279f92 | |||
| 58d6f1faad | |||
| d93da0d8b4 | |||
| 701c455c42 | |||
| e174b64495 | |||
| 3afb25bb5e | |||
| fbd17536fd | |||
| 32ca8c1208 | |||
| a7ff68d279 | |||
| de368ea810 | |||
| 6a3e3a57cd | |||
| 7089944306 | |||
| ec5e348694 | |||
| 12d9f2f647 | |||
| 6ffcbecade | |||
| e76d93187b | |||
| 52e20e3f93 | |||
| 3f966a230e | |||
| 5d0b211a44 | |||
| 6931f36fd1 | |||
| db02443463 | |||
| eb8e1402bc | |||
| c93f45dcb8 | |||
| a0b1814024 | |||
| 3a2826d3f9 | |||
| dbd30689ab | |||
| 5eb59b8a66 | |||
| 8e7b866de2 | |||
| 1b4f9bfa6a | |||
| 2d39272330 | |||
| f699bb9167 |
@@ -27,14 +27,14 @@ jobs:
|
||||
run: uv build
|
||||
|
||||
- name: Publish to pypi
|
||||
run: uv publish --token '${{ secrets.PYPI_API_TOKEN }}'
|
||||
run: uv publish --token '${{ secrets.PYPI_TOKEN }}'
|
||||
|
||||
- name: Create Gitea Release & Upload Assets
|
||||
env:
|
||||
GITEA_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
TAG_NAME: ${{ github.ref_name }}
|
||||
REPO: ${{ github.repository }}
|
||||
GITEA_URL: http://gitea:3000
|
||||
GITEA_URL: http://172.17.0.1:3000
|
||||
run: |
|
||||
set -e
|
||||
# 1. 创建 Release
|
||||
|
||||
@@ -11,3 +11,6 @@ wheels/
|
||||
.coverage
|
||||
.idea
|
||||
*_profile.html
|
||||
|
||||
# Sphinx 文档构建输出
|
||||
docs/_build/
|
||||
|
||||
@@ -0,0 +1,23 @@
|
||||
# ReadTheDocs 配置
|
||||
# https://docs.readthedocs.io/en/stable/config-file/v2.html
|
||||
version: 2
|
||||
|
||||
# 构建配置
|
||||
build:
|
||||
os: ubuntu-24.04
|
||||
tools:
|
||||
python: "3.11"
|
||||
|
||||
# Python 依赖与构建命令
|
||||
python:
|
||||
install:
|
||||
- method: pip
|
||||
path: .
|
||||
extra_requirements:
|
||||
- docs
|
||||
|
||||
# Sphinx 构建
|
||||
sphinx:
|
||||
configuration: docs/conf.py
|
||||
builder: html
|
||||
fail_on_warning: false
|
||||
@@ -0,0 +1,108 @@
|
||||
# 文档整理与 Sphinx 文档搭建计划
|
||||
|
||||
## Context
|
||||
|
||||
最近完成 CLI 重构:新增 `pf` 统一入口,13 个工具迁移到 YAML 配置并删除了对应 .py 入口脚本,`run()` 的 verbose 统一应用到 spec。但文档未同步:README 仍引用旧命令(`yamlrun`、`python build.py`),模块结构表缺漏;`runner.py` 的 `_apply_verbose_to_graph` 成为死代码;项目缺少可发布的 Sphinx 文档。本次任务整理这些遗留,并搭建 ReadTheDocs 文档站。
|
||||
|
||||
## 任务范围
|
||||
|
||||
### 1. 清理死代码
|
||||
- 删除 `src/pyflowx/runner.py` 的 `_apply_verbose_to_graph` 函数(line 38-68),功能已移入 `executors.py` 的 `run()`。
|
||||
- 删除 `tests/test_runner.py` 中对应测试(line 610-636,`TestApplyVerboseToGraph` 类)。
|
||||
- 清理 `runner.py` 顶部 `from dataclasses import replace` 若变为未使用。
|
||||
|
||||
### 2. 修复版本不一致
|
||||
- `src/pyflowx/__init__.py:105` 硬编码 `__version__ = "0.4.5"`,`pyproject.toml:25` 为 `0.3.5`。
|
||||
- 统一为 `0.4.5`(`__init__.py` 为准,pyproject.toml 是源但 bumpversion 工具应同时更新两者)。
|
||||
|
||||
### 3. 更新 README.md
|
||||
- L304-308:`python build.py clean/build/test` → `pf pymake clean/build/test`。
|
||||
- L335-351、L435:`yamlrun pipeline.yaml ...` → `pf yamlrun pipeline.yaml ...`(6 处)。
|
||||
- L311:`verbose=True(默认)` 描述保留,但 CLI 示例改为 `pf`。
|
||||
- L558-574 模块结构表:补充 `cli/pf.py`(统一入口)、`cli/configs/`(YAML 工具配置)、`cli/_ops/`(工具函数)、`profiling.py`、`registry.py`。
|
||||
- 顶部增加「文档」徽章链接到 ReadTheDocs。
|
||||
|
||||
### 4. 搭建 Sphinx 文档结构
|
||||
新建 `docs/` 目录:
|
||||
|
||||
```
|
||||
docs/
|
||||
├── conf.py # Sphinx 配置
|
||||
├── index.rst # 首页与目录
|
||||
├── installation.rst # 安装
|
||||
├── quickstart.rst # 快速上手(从 README 提炼)
|
||||
├── guide/
|
||||
│ ├── task.rst # TaskSpec 任务描述
|
||||
│ ├── graph.rst # Graph DAG 构建
|
||||
│ ├── execution.rst # 执行策略与 run()
|
||||
│ ├── yaml.rst # YAML 任务编排
|
||||
│ └── cli.rst # pf 统一入口与工具列表
|
||||
├── api.rst # API 参考(automodule 自动生成)
|
||||
└── changelog.rst # 变更日志摘要
|
||||
```
|
||||
|
||||
**conf.py 要点**:
|
||||
- 扩展:`sphinx.ext.autodoc`、`sphinx.ext.napoleon`(支持 Google/NumPy docstring)、`sphinx.ext.viewcode`、`myst_parser`(支持 Markdown)
|
||||
- 主题:`sphinx_rtd_theme`
|
||||
- 项目版本从 `pyflowx.__version__` 动态读取
|
||||
- `autodoc_default_options`:`members: True, undoc-members: True, show-inheritance: True`
|
||||
|
||||
**api.rst**:用 `automodule:: pyflowx` 抓取 `__all__` 的 56 个公共符号。
|
||||
|
||||
### 5. ReadTheDocs 配置
|
||||
- 新建 `.readthedocs.yaml`:Python 3.11,`pip install -e .[docs]`,`sphinx -b html docs/ docs/_build/`。
|
||||
- `.gitignore` 增加 `docs/_build/`。
|
||||
|
||||
### 6. pyproject.toml 补充 docs 依赖
|
||||
```toml
|
||||
docs = [
|
||||
"sphinx>=7.0",
|
||||
"sphinx-rtd-theme>=2.0",
|
||||
"myst-parser>=3.0",
|
||||
]
|
||||
```
|
||||
并在 `[dependency-groups]` 的 dev 中加入 `pyflowx[docs]`。
|
||||
|
||||
## 关键文件
|
||||
|
||||
| 文件 | 操作 |
|
||||
|------|------|
|
||||
| `src/pyflowx/runner.py` | 删除 `_apply_verbose_to_graph` |
|
||||
| `tests/test_runner.py` | 删除 `TestApplyVerboseToGraph` |
|
||||
| `src/pyflowx/__init__.py` | 版本统一(已 0.4.5,确认) |
|
||||
| `pyproject.toml` | 版本 → 0.4.5;加 docs 依赖 |
|
||||
| `README.md` | 更新 CLI 示例与模块结构表 |
|
||||
| `docs/conf.py` | 新建 |
|
||||
| `docs/*.rst` | 新建 |
|
||||
| `.readthedocs.yaml` | 新建 |
|
||||
| `.gitignore` | 加 docs/_build/ |
|
||||
|
||||
## 验证
|
||||
|
||||
1. **测试与 lint**:
|
||||
```bash
|
||||
uv run pytest tests/ -q
|
||||
uv run ruff check src/ tests/ docs/conf.py
|
||||
uv run pyrefly check src/pyflowx/runner.py
|
||||
```
|
||||
|
||||
2. **Sphinx 构建本地验证**:
|
||||
```bash
|
||||
uv sync --extra docs
|
||||
uv run sphinx-build -b html docs/ docs/_build/
|
||||
```
|
||||
确认无 warning,打开 `docs/_build/index.html` 检查页面。
|
||||
|
||||
3. **pf 功能回归**:
|
||||
```bash
|
||||
pf gitt c
|
||||
pf pymake b --dry-run
|
||||
```
|
||||
|
||||
4. **RTD 配置校验**:`.readthedocs.yaml` 语法正确,`docs/conf.py` 能独立构建。
|
||||
|
||||
## 不在范围
|
||||
|
||||
- 不统一各模块 docstring 风格(napoleon 兼容 Google/NumPy,够用)。
|
||||
- 不重构现有 CLI 工具 YAML。
|
||||
- 不新增中文文档翻译(文档用中文撰写,与项目既有风格一致)。
|
||||
@@ -150,7 +150,7 @@ uvx --from pyflowx pymake cov
|
||||
|
||||
## Git 与提交
|
||||
|
||||
- **不自动提交/push**:除非用户明确要求。
|
||||
- **自动提交**:任务完成后自动 `git add`(按文件名)+ `git commit` + `git push`(仅当分支已跟踪远程时执行 push;新分支跳过 push 并在总结中说明)。
|
||||
- **不修改 git config**。
|
||||
- **不运行破坏性命令**(`push --force`/`reset --hard`/`clean -f`)除非用户明确要求。
|
||||
- **staging**:按文件名添加,不用 `git add -A`/`git add .`,避免误加敏感文件。
|
||||
|
||||
@@ -0,0 +1,134 @@
|
||||
---
|
||||
alwaysApply: true
|
||||
---
|
||||
|
||||
# 自驱动开发规则
|
||||
|
||||
本规则定义一种"目标驱动、闭环执行"的工作模式:仅在任务开始时与用户确认一次目标与边界,后续由 Agent 自主完成"计划 → 编码 → 测试 → 文档 → 验证"的迭代循环,直到用户目标达成。
|
||||
|
||||
## 核心原则
|
||||
|
||||
- **目标导向**:始终以用户最终目标为准绳,所有阶段产出都应服务于该目标。
|
||||
- **闭环执行**:每个子任务必须走完"计划 → 实现 → 测试 → 文档 → 验证"五步;禁止跳步留半成品。
|
||||
- **自主决策**:初始确认之后,实现路径、API 形态、重构范围、文件命名、测试组织、错误修复策略等由 Agent 自行决断,不再逐项请示。**可逆操作(编辑文件、运行测试、修复 lint、调整实现)直接执行,不询问**;只有不可逆/高风险操作才暂停。
|
||||
- **透明沟通**:每个阶段开始前用一句话说明意图;关键节点(完成、阻塞、转向)给简短更新;不复述内部思考,**不在收尾时停下询问"是否继续"或"是否提交"**——直接输出总结并结束。
|
||||
- **安全边界**:仅在高风险、不可逆操作或真正阻塞时才暂停找用户。
|
||||
|
||||
## 初始确认(一次性,仅在最开始)
|
||||
|
||||
任务启动时,用 `AskUserQuestion` 一次性确认以下信息(已由项目规范覆盖的不必重复确认):
|
||||
|
||||
1. **目标与范围**:要解决什么问题?交付物是什么?显式列出不在范围内的内容。
|
||||
2. **验收标准**:怎样算"完成"?可观测的判定条件(功能、性能、覆盖率阈值)。
|
||||
3. **特殊约束**:除 `python-standards.md` 之外的约束(兼容性、依赖限制、API 兼容策略等)。
|
||||
4. **测试要求**:覆盖率门槛(项目默认 ≥95%,branch);是否需要新增 `slow` 标记。
|
||||
|
||||
**git commit/push 不在确认范围内**:任务完成后自动 commit + push(仅当分支已跟踪远程时执行 push;新分支跳过 push 并在总结中说明),遵循 `.trae/rules/git-commit-message.md` 风格。仅 force-push、reset --hard、clean -f、修改 git config 等真正破坏性操作才需暂停确认。
|
||||
|
||||
确认后,将目标与验收标准固化进 `TaskCreate` 任务列表,后续不再就同一信息反复询问。
|
||||
|
||||
## 迭代循环
|
||||
|
||||
下列五个阶段构成一个完整闭环。未达验收标准时,回到「计划」开启下一轮;达标准时,进入「收尾」。
|
||||
|
||||
### 1. 计划(Plan)
|
||||
|
||||
- 用 Explore/Glob/Grep 研究相关代码与既有模式,避免凭空设计。
|
||||
- 用 `TaskCreate` 把目标拆为可独立验证的子任务;每完成一项立即 `TaskUpdate` 为 completed。
|
||||
- 优先复用现有抽象;不为本轮假想需求设计接口。
|
||||
- 不过早抽象:三处相似才考虑提取,否则就地写。
|
||||
|
||||
### 2. 实现(Code)
|
||||
|
||||
- 严格遵守 `.trae/rules/python-standards.md` 与既有代码风格。
|
||||
- 优先 Edit 现有文件;新增文件需有明确职责边界。
|
||||
- 不引入运行时依赖(项目零依赖原则);确需引入须在计划阶段说明。
|
||||
- 公共 API 必须有完整类型注解与中文 docstring。
|
||||
- 不写未被要求的功能、不为未来场景预留扩展点。
|
||||
|
||||
### 3. 测试(Test)
|
||||
|
||||
- 新增/修改的公共 API 必须配套测试;优先通过公共接口测试,故障注入可访问私有属性并在 docstring 注明。
|
||||
- Mock 优先级:`monkeypatch` > 内联 stub > `unittest.mock` > `pytest-mock`;禁用 `@patch` 装饰器。
|
||||
- 必跑校验(每次修改后):
|
||||
|
||||
```bash
|
||||
uvx --from pyflowx pymake tc
|
||||
uvx --from pyflowx pymake cov
|
||||
```
|
||||
|
||||
- 测试失败时定位根因再修复,不通过放宽断言或 `# pragma: no cover` 绕过。
|
||||
- 覆盖率不得低于上一次的值(项目门槛 95%,branch)。
|
||||
|
||||
### 4. 文档(Docs)
|
||||
|
||||
- 同步更新 docstring、README、模块结构说明。
|
||||
- 行为变更须同步更新 `.agents/skills/pyflowx-development/SKILL.md` 中的对应章节。
|
||||
- 跨会话有价值的设计决策、约束、陷阱,追加到 memory(`project_memory.md` 或对应 `topics.md`)。
|
||||
- 不主动新建 `*.md` 文档;除非用户明确要求。
|
||||
|
||||
### 5. 验证(Verify)
|
||||
|
||||
- 逐条对照初始确认的「验收标准」核验;未满足则回到「计划」继续下一轮。
|
||||
- 全套门禁通过:ruff、pyrefly、pytest、coverage。
|
||||
- 给出本轮变更清单(改了哪些文件、为什么)。
|
||||
|
||||
## 暂停条件(仅在以下情况中断自驱动找用户)
|
||||
|
||||
1. **歧义无法自决**:需求存在多种合理解读且无既有约定可循。
|
||||
2. **高风险/不可逆操作**:删除非临时文件、`git push --force`、`reset --hard`、删表、修改 CI 配置、修改 git config、卸载依赖等。**普通 `git commit`/`push` 不属于此类**(任务完成后自动执行)。
|
||||
3. **不可恢复的失败**:根因不在本仓库、需外部环境/权限配合、或经两轮尝试仍无法定位。
|
||||
4. **超出初始确认范围**:用户目标在执行中发现需要显著扩大范围或改变方向。
|
||||
5. **用户主动询问**:用户在对话中提出新问题或要求澄清。
|
||||
|
||||
**注意**:"目标已达成"**不是**暂停条件——验收标准全部满足后直接进入收尾并结束任务,不询问"是否扩展范围"或"是否提交"。
|
||||
|
||||
非以上情况,一律继续自驱动,不要为"求确认"而暂停。
|
||||
|
||||
## 决策判据:该问还是自决
|
||||
|
||||
遇到不确定时,按以下顺序判断:
|
||||
|
||||
1. **是否不可逆/高风险?** 是 → 暂停确认(如删除文件、`push --force`、修改 CI 配置、卸载依赖)。否 → 继续。
|
||||
2. **是否在初始确认范围内?** 是 → 按确认执行,不询问。否 → 视为"超出初始确认范围",暂停。
|
||||
3. **是否有既有约定可循?** 是 → 按约定执行(参考 `python-standards.md`、`project_memory.md`)。否 → 视为"歧义无法自决",暂停。
|
||||
4. **是否可逆?** 是 → 直接执行,即使结果可能不完美(可在后续迭代修正)。否 → 暂停。
|
||||
|
||||
**可直接自决(不询问)的典型情况**:
|
||||
|
||||
- 测试失败、覆盖率不达标、lint/类型检查报错 → 定位根因并修复。
|
||||
- 代码风格选择(命名、模块划分、参数顺序)→ 自决。
|
||||
- 文件编辑、运行测试、运行校验命令 → 直接执行。
|
||||
- 任务完成后输出收尾总结 → 直接输出,不询问下一步。
|
||||
- 显式指定 `name` 参数以保持测试兼容性 → 自决。
|
||||
- 重命名局部变量以避免遮蔽 → 自决。
|
||||
|
||||
**必须暂停询问的典型情况**:
|
||||
|
||||
- 删除非临时文件、重命名公共模块/包。
|
||||
- `git push --force`、`reset --hard`、`clean -f`、修改 git config(普通 commit/push 自动执行,无需询问)。
|
||||
- 引入新的运行时依赖(违反项目零依赖原则)。
|
||||
- 修改 CI 配置、pre-commit 钩子、pyproject.toml 的工具链配置。
|
||||
- 卸载或降级既有依赖。
|
||||
|
||||
## 沟通风格
|
||||
|
||||
- 阶段切换时一句话说明即可;不要把内部推理写给用户看。
|
||||
- 完成子任务后用一两句总结改了什么、下一步做什么。
|
||||
- 遇到阻塞时直接说明:卡在哪、试了什么、需要用户做什么。
|
||||
- **不在收尾时询问"是否需要提交"或"是否扩展范围"**——直接输出总结并结束。用户后续若有新需求,由用户主动提出。
|
||||
- 不使用 emoji,除非用户明确要求。
|
||||
|
||||
## 工具使用
|
||||
|
||||
- 独立操作尽量并行调用(多个 Read/Grep/Glob 一批发出)。
|
||||
- 用 `TaskCreate`/`TaskUpdate` 维护进度,不批量推迟标记。
|
||||
- 长命令用后台运行(`run_in_background`),完成会自动通知。
|
||||
- 文件操作一律用专用工具:Read/Edit/Write/Glob/Grep,不用 `cat`/`sed`/`grep`/`find`。
|
||||
|
||||
## 收尾
|
||||
|
||||
- 验收标准全部满足后,**直接输出最终总结并结束任务**:交付物、关键决策、遗留事项。
|
||||
- **自动提交**:收尾时自动 `git add`(按文件名)+ `git commit`(遵循 `.trae/rules/git-commit-message.md` 风格)+ `git push`(仅当分支已跟踪远程时执行;新分支跳过 push 并在总结中说明);**不询问**"是否需要提交"或"是否扩展范围"。
|
||||
- 若验收标准未全部满足,回到「计划」继续下一轮,不停下询问。
|
||||
- 将本次会话的关键产出与决策更新到 memory,便于后续会话续接。
|
||||
@@ -5,6 +5,7 @@
|
||||
[](https://github.com/gookeryoung/pyflowx/actions/workflows/ci.yml)
|
||||
[](https://pypi.org/project/pyflowx/)
|
||||
[](https://pypi.org/project/pyflowx/)
|
||||
[](https://pyflowx.readthedocs.io/zh/latest/)
|
||||
[](https://github.com/gookeryoung/pyflowx)
|
||||
[](https://github.com/gookeryoung/pyflowx/blob/main/LICENSE)
|
||||
|
||||
@@ -31,7 +32,8 @@ PyFlowX 把"任务依赖"这件事做到极致简单:**参数名就是依赖
|
||||
- **图级默认值** —— `GraphDefaults` 统一配置 retry/timeout/concurrency 等
|
||||
- **CLI 运行器** —— `CliRunner` 把多个图映射为命令行子命令,替代 Makefile
|
||||
- **可观测** —— `on_event` 回调(RUNNING/SUCCESS/FAILED/SKIPPED)、`dry_run` 预览、`verbose` 生命周期日志、Mermaid 可视化
|
||||
- **零运行时依赖** —— 仅依赖标准库(3.8 需 `graphlib_backport`)
|
||||
- **YAML 任务编排** —— GitHub Actions 风格的声明式任务图,支持 `jobs`/`needs`/`strategy.matrix`/`if` 等 CI/CD 概念,从 YAML 文件直接加载执行
|
||||
- **最小依赖** —— 仅依赖标准库 + PyYAML(3.8 需 `graphlib_backport`、`typing-extensions`)
|
||||
- **97% 测试覆盖** —— 分支覆盖率 >= 95%
|
||||
|
||||
## 安装
|
||||
@@ -300,15 +302,123 @@ runner.run_cli() # 解析 sys.argv 并执行
|
||||
命令行用法:
|
||||
|
||||
```bash
|
||||
python build.py clean # 执行 clean 图
|
||||
python build.py build --strategy thread # 覆盖执行策略
|
||||
python build.py test --dry-run # 仅打印执行计划
|
||||
python build.py --list # 列出所有命令
|
||||
python build.py --quiet # 静默模式
|
||||
pf pymake clean # 执行 clean 图
|
||||
pf pymake build --strategy thread # 覆盖执行策略
|
||||
pf pymake test --dry-run # 仅打印执行计划
|
||||
pf pymake --list # 列出所有命令
|
||||
pf pymake --quiet # 静默模式
|
||||
```
|
||||
|
||||
`verbose=True`(默认)时打印任务生命周期(开始/成功/失败/跳过)与命令输出;`--quiet` 关闭。
|
||||
|
||||
## YAML 任务编排
|
||||
|
||||
PyFlowX 支持 GitHub Actions 风格的声明式 YAML 任务编排,从 YAML 文件直接加载任务图。
|
||||
|
||||
### 编程式 API
|
||||
|
||||
```python
|
||||
import pyflowx as px
|
||||
|
||||
# 从 YAML 文件加载任务图
|
||||
graph = px.Graph.from_yaml("pipeline.yaml")
|
||||
report = px.run(graph, strategy="thread")
|
||||
|
||||
# 或用函数式 API
|
||||
graph = px.load_yaml("pipeline.yaml")
|
||||
graph = px.parse_yaml_string("""
|
||||
jobs:
|
||||
hello:
|
||||
cmd: ["echo", "hello"]
|
||||
""")
|
||||
```
|
||||
|
||||
### CLI 入口
|
||||
|
||||
通过 `pf` 统一入口调用(详见 [pf 工具](#cli-工具) 章节):
|
||||
|
||||
```bash
|
||||
# 执行 YAML 任务图
|
||||
pf yamlrun pipeline.yaml
|
||||
|
||||
# 指定执行策略
|
||||
pf yamlrun pipeline.yaml --strategy thread
|
||||
|
||||
# 仅打印任务分层,不执行
|
||||
pf yamlrun pipeline.yaml --dry-run
|
||||
|
||||
# 列出所有任务名
|
||||
pf yamlrun pipeline.yaml --list
|
||||
|
||||
# 静默模式
|
||||
pf yamlrun pipeline.yaml --quiet
|
||||
```
|
||||
|
||||
### YAML Schema(GitHub Actions 风格)
|
||||
|
||||
```yaml
|
||||
strategy: thread # 图级默认策略
|
||||
defaults: # 图级默认值
|
||||
retry: {max_attempts: 3}
|
||||
verbose: true
|
||||
env: {CI: "true"}
|
||||
|
||||
jobs:
|
||||
setup:
|
||||
cmd: ["git", "clone", "..."]
|
||||
runs-on: linux
|
||||
|
||||
build:
|
||||
needs: [setup] # 依赖列表
|
||||
cmd: ["python", "-m", "build"]
|
||||
timeout: 300
|
||||
retry: {max_attempts: 2, delay: 1.0}
|
||||
|
||||
test:
|
||||
needs: [build]
|
||||
cmd: ["python${{ matrix.version }}", "-m", "pytest"] # 矩阵占位符
|
||||
strategy:
|
||||
matrix: # 笛卡尔积展开为 6 个任务
|
||||
version: ["3.8", "3.9", "3.10"]
|
||||
os: ["linux", "macos"]
|
||||
if: "env.CI" # 条件: 环境变量存在
|
||||
|
||||
lint:
|
||||
needs: [build]
|
||||
cmd: ["ruff", "check"]
|
||||
if: "env.CI == 'true'" # 条件: 环境变量等于
|
||||
|
||||
deploy:
|
||||
needs: [test, lint] # 矩阵依赖自动展开
|
||||
cmd: ["twine", "upload"]
|
||||
if: "env.DEPLOY_TOKEN != ''"
|
||||
allow-upstream-skip: true
|
||||
concurrency-key: deploy_lock
|
||||
```
|
||||
|
||||
### 字段映射
|
||||
|
||||
| YAML 字段 | TaskSpec 字段 | 说明 |
|
||||
|-----------|---------------|------|
|
||||
| `jobs.<id>` | `name` | job ID 作为任务名 |
|
||||
| `cmd` / `run` | `cmd` | `cmd` 为列表形式,`run` 为 shell 字符串 |
|
||||
| `needs` | `depends_on` | 依赖列表(矩阵任务自动展开) |
|
||||
| `if` | `conditions` | `success()` / `always()` / `env.VAR` / `env.VAR == 'x'` |
|
||||
| `strategy.matrix` | 矩阵扇出 | 笛卡尔积展开为多个任务 |
|
||||
| `${{ matrix.key }}` | 占位符 | 在 cmd/run/cwd/env 中替换 |
|
||||
| `timeout` | `timeout` | 超时秒数 |
|
||||
| `retry` | `retry` | `{max_attempts, delay, backoff, jitter}` |
|
||||
| `cwd` | `cwd` | 工作目录 |
|
||||
| `env` | `env` | 环境变量 |
|
||||
| `verbose` | `verbose` | 详细输出 |
|
||||
| `continue-on-error` | `continue_on_error` | 失败不中止整图 |
|
||||
| `skip-if-missing` | `skip_if_missing` | 命令不存在时跳过 |
|
||||
| `allow-upstream-skip` | `allow_upstream_skip` | 上游跳过时仍执行 |
|
||||
| `priority` | `priority` | 同层优先级 |
|
||||
| `concurrency-key` | `concurrency_key` | 并发限制键 |
|
||||
| `tags` | `tags` | 自由标签 |
|
||||
| `runs-on` | `tags`(追加) | 运行环境标签 |
|
||||
|
||||
## 示例
|
||||
|
||||
仓库 `examples/` 目录包含完整示例:
|
||||
@@ -316,6 +426,7 @@ python build.py --quiet # 静默模式
|
||||
- [`etl_pipeline.py`](examples/etl_pipeline.py) —— ETL 流水线(sequential)
|
||||
- [`parallel_run.py`](examples/parallel_run.py) —— 并行执行对比(thread vs sequential)
|
||||
- [`async_aggregation.py`](examples/async_aggregation.py) —— 异步聚合 + Context 注入
|
||||
- [`yaml_pipeline.yaml`](examples/yaml_pipeline.yaml) + [`yaml_pipeline.py`](examples/yaml_pipeline.py) —— YAML 声明式 CI/CD 流水线(矩阵扇出 + 条件执行)
|
||||
|
||||
运行:
|
||||
|
||||
@@ -323,6 +434,8 @@ python build.py --quiet # 静默模式
|
||||
python examples/etl_pipeline.py
|
||||
python examples/parallel_run.py
|
||||
python examples/async_aggregation.py
|
||||
python -m pyflowx.examples.yaml_pipeline
|
||||
pf yamlrun src/pyflowx/examples/yaml_pipeline.yaml --dry-run
|
||||
```
|
||||
|
||||
## 断点续跑
|
||||
@@ -447,6 +560,8 @@ uv run ruff format --check src tests examples
|
||||
|
||||
## 模块结构
|
||||
|
||||
### 核心
|
||||
|
||||
| 模块 | 职责 |
|
||||
|------|------|
|
||||
| `task.py` | 纯数据结构:`TaskSpec`、`RetryPolicy`、`TaskHooks`、`TaskStatus` |
|
||||
@@ -455,11 +570,29 @@ uv run ruff format --check src tests examples
|
||||
| `context.py` | 上下文注入:参数名→依赖解析 |
|
||||
| `command.py` | 命令执行:`run_command`(list/shell/Callable) |
|
||||
| `conditions.py` | 条件执行:内置条件与组合器 |
|
||||
| `executors.py` | 执行器与 `run` 入口:四种策略共享模块级辅助 |
|
||||
| `executors.py` | 执行器与 `run` 入口:四种策略共享模块级辅助;verbose 统一应用到 spec |
|
||||
| `storage.py` | 状态后端:`MemoryBackend` / `JSONBackend`(batch flush) |
|
||||
| `runner.py` | CLI 运行器:`CliRunner` |
|
||||
| `report.py` | 运行结果:`RunReport` / `TaskResult` |
|
||||
| `yaml_loader.py` | YAML 任务编排:GitHub Actions 风格 schema 解析(`load_yaml` / `parse_yaml_string` / `run_cli`) |
|
||||
| `registry.py` | 函数注册中心:`register_fn` / `get_fn` / `has_fn`(YAML 的 `fn:` 引用) |
|
||||
| `profiling.py` | 性能分析:`Profiler` 任务耗时统计 |
|
||||
| `errors.py` | 错误家族:`PyFlowXError` 子类 |
|
||||
| `ops/` | 工具函数(dev/files/media/system),被 YAML 的 `fn:` 引用 |
|
||||
|
||||
### CLI 工具
|
||||
|
||||
| 模块 | 职责 |
|
||||
|------|------|
|
||||
| `cli/pf.py` | 统一入口:`pf <tool> [command]`,自动发现 `configs/*.yaml` 并路由 |
|
||||
| `cli/configs/` | YAML 工具配置(gittool/filedate/pdftool 等 13 个工具的 `cli:` schema) |
|
||||
| `cli/pymake.py` | 构建工具(替代 Makefile),`pf pymake b` 调用 |
|
||||
| `cli/yamlrun.py` | YAML pipeline 执行器,`pf yamlrun pipeline.yaml` 调用 |
|
||||
| `cli/profiler.py` | 性能分析 CLI |
|
||||
| `cli/emlmanager.py` | 邮件管理 CLI |
|
||||
| `cli/dev/` | 开发工具(dockercmd/envdev) |
|
||||
| `cli/llm/` | LLM 工具(msdownload/sglang) |
|
||||
| `cli/system/` | 系统工具(clearscreen/taskkill/which) |
|
||||
|
||||
## 许可证
|
||||
|
||||
|
||||
Vendored
+106
@@ -0,0 +1,106 @@
|
||||
API 参考
|
||||
========
|
||||
|
||||
任务描述
|
||||
--------
|
||||
|
||||
.. autoclass:: pyflowx.TaskSpec
|
||||
:members:
|
||||
:undoc-members:
|
||||
:show-inheritance:
|
||||
:exclude-members: args, kwargs
|
||||
|
||||
.. autoclass:: pyflowx.RetryPolicy
|
||||
:members:
|
||||
:undoc-members:
|
||||
|
||||
.. autoclass:: pyflowx.TaskHooks
|
||||
:members:
|
||||
:undoc-members:
|
||||
|
||||
.. autoclass:: pyflowx.TaskStatus
|
||||
:members:
|
||||
:undoc-members:
|
||||
|
||||
图构建
|
||||
------
|
||||
|
||||
.. autoclass:: pyflowx.Graph
|
||||
:members:
|
||||
:undoc-members:
|
||||
:exclude-members: from_specs, from_yaml
|
||||
|
||||
.. autoclass:: pyflowx.GraphDefaults
|
||||
:members:
|
||||
:undoc-members:
|
||||
|
||||
.. autofunction:: pyflowx.compose
|
||||
.. autofunction:: pyflowx.task_template
|
||||
|
||||
执行
|
||||
----
|
||||
|
||||
.. autofunction:: pyflowx.run
|
||||
|
||||
.. autoclass:: pyflowx.RunReport
|
||||
:members:
|
||||
:undoc-members:
|
||||
|
||||
.. autoclass:: pyflowx.TaskResult
|
||||
:members:
|
||||
:undoc-members:
|
||||
|
||||
YAML 编排
|
||||
---------
|
||||
|
||||
.. autofunction:: pyflowx.load_yaml
|
||||
.. autofunction:: pyflowx.parse_yaml_string
|
||||
.. autofunction:: pyflowx.run_yaml
|
||||
.. autofunction:: pyflowx.run_cli
|
||||
.. autofunction:: pyflowx.build_cli_parser
|
||||
|
||||
函数注册
|
||||
--------
|
||||
|
||||
.. autofunction:: pyflowx.register_fn
|
||||
.. autofunction:: pyflowx.get_fn
|
||||
.. autofunction:: pyflowx.has_fn
|
||||
|
||||
命令执行
|
||||
--------
|
||||
|
||||
.. autofunction:: pyflowx.run_command
|
||||
|
||||
CLI 运行器
|
||||
----------
|
||||
|
||||
.. autoclass:: pyflowx.CliRunner
|
||||
:members:
|
||||
:undoc-members:
|
||||
|
||||
状态后端
|
||||
--------
|
||||
|
||||
.. autoclass:: pyflowx.StateBackend
|
||||
:members:
|
||||
:undoc-members:
|
||||
|
||||
.. autoclass:: pyflowx.MemoryBackend
|
||||
:members:
|
||||
:undoc-members:
|
||||
|
||||
.. autoclass:: pyflowx.JSONBackend
|
||||
:members:
|
||||
:undoc-members:
|
||||
|
||||
错误家族
|
||||
--------
|
||||
|
||||
.. autoexception:: pyflowx.PyFlowXError
|
||||
.. autoexception:: pyflowx.DuplicateTaskError
|
||||
.. autoexception:: pyflowx.MissingDependencyError
|
||||
.. autoexception:: pyflowx.CycleError
|
||||
.. autoexception:: pyflowx.TaskFailedError
|
||||
.. autoexception:: pyflowx.TaskTimeoutError
|
||||
.. autoexception:: pyflowx.InjectionError
|
||||
.. autoexception:: pyflowx.StorageError
|
||||
@@ -0,0 +1,45 @@
|
||||
变更日志
|
||||
========
|
||||
|
||||
0.4.5
|
||||
-----
|
||||
|
||||
CLI 重构
|
||||
~~~~~~~~
|
||||
|
||||
- 新增 ``pf`` 统一入口:通过 ``pf <tool> [command] [options]`` 调用所有工具
|
||||
- 13 个工具迁移到 YAML 配置(filedate/filelevel/folderback/folderzip/screenshot/sshcopyid/lscalc/bumpversion/autofmt/piptool/packtool/pdftool/gittool)
|
||||
- YAML 配置支持 ``cli:`` 段声明命令行参数 schema,由 ``build_cli_parser`` 自动生成 argparse
|
||||
- 删除 13 个冗余 ``.py`` 入口脚本,统一通过 ``pf`` 调用
|
||||
- ``run()`` 在 ``verbose=True`` 时自动把 verbose 标记应用到所有 spec
|
||||
- 全局选项 ``--verbose`` 改为 ``--quiet``(默认显示执行过程)
|
||||
- ``cmd`` 任务成功时打印 stdout(此前被静默丢弃)
|
||||
- ``gittool`` 用 ``CLEAN_EXCLUDES`` 数组变量配置 ``git clean -e`` 参数
|
||||
|
||||
YAML 任务编排
|
||||
~~~~~~~~~~~~~
|
||||
|
||||
- 支持 ``variables`` 变量定义,``${VAR}`` 在 cmd/env/cwd 中替换
|
||||
- 列表变量展开为 cmd 数组多个元素
|
||||
- ``cli:`` 段支持 subcommands/positional/options 三级 schema
|
||||
- 支持 ``type: path`` 自动转为 ``pathlib.Path``
|
||||
|
||||
文档
|
||||
~~~~
|
||||
|
||||
- 搭建 Sphinx 文档,发布到 ReadTheDocs
|
||||
- 更新 README:CLI 示例改为 ``pf`` 统一入口,模块结构表补全
|
||||
|
||||
0.3.x
|
||||
-----
|
||||
|
||||
- 新增 YAML 任务编排(GitHub Actions 风格 schema)
|
||||
- 新增 ``fn:`` 函数引用与 ``register_fn`` / ``get_fn`` 注册中心
|
||||
- 新增 ``compose`` / ``GraphComposer`` 多图组合
|
||||
- 新增 ``task_template`` 任务模板工厂
|
||||
- 新增 ``concurrency_key`` + ``concurrency_limits`` 并发限制
|
||||
- 新增 ``JSONBackend`` 断点续跑与 ``batch()`` 批量落盘
|
||||
- 新增 ``cache_key`` 缓存键函数
|
||||
- 新增条件执行(``IS_WINDOWS`` / ``HAS_INSTALLED`` / ``ENV_VAR_EQUALS`` 等)
|
||||
- 四种执行策略:``sequential`` / ``thread`` / ``async`` / ``dependency``
|
||||
- 参数名即依赖的上下文注入机制
|
||||
@@ -0,0 +1,65 @@
|
||||
"""Sphinx 配置.
|
||||
|
||||
ReadTheDocs 构建 PyFlowX 文档站。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
# 确保 src/ 在 sys.path 中, autodoc 能导入 pyflowx
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent.parent / "src"))
|
||||
|
||||
from pyflowx import __version__
|
||||
|
||||
# -- 项目信息 --------------------------------------------------------------
|
||||
project = "PyFlowX"
|
||||
author = "pyflowx"
|
||||
copyright = "2024, pyflowx"
|
||||
release = __version__
|
||||
version = __version__
|
||||
|
||||
# -- Sphinx 配置 -----------------------------------------------------------
|
||||
extensions = [
|
||||
"sphinx.ext.autodoc",
|
||||
"sphinx.ext.napoleon",
|
||||
"sphinx.ext.viewcode",
|
||||
"sphinx.ext.intersphinx",
|
||||
"myst_parser",
|
||||
]
|
||||
|
||||
# -- 主题 ------------------------------------------------------------------
|
||||
html_theme = "sphinx_rtd_theme"
|
||||
html_static_path = ["_static"]
|
||||
|
||||
# -- autodoc 配置 ----------------------------------------------------------
|
||||
autodoc_default_options = {
|
||||
"members": True,
|
||||
"undoc-members": True,
|
||||
"show-inheritance": True,
|
||||
"member-order": "bysource",
|
||||
}
|
||||
autodoc_type_hints = "description"
|
||||
autodoc_typehints_format = "short"
|
||||
|
||||
# -- napoleon 配置 (Google/NumPy docstring 兼容) --------------------------
|
||||
napoleon_google_docstring = True
|
||||
napoleon_numpy_docstring = True
|
||||
napoleon_include_init_with_doc = False
|
||||
napoleon_include_private_with_doc = False
|
||||
napoleon_include_special_with_doc = True
|
||||
|
||||
# -- intersphinx -----------------------------------------------------------
|
||||
intersphinx_mapping = {
|
||||
"python": ("https://docs.python.org/3", None),
|
||||
}
|
||||
|
||||
# -- 全局选项 ---------------------------------------------------------------
|
||||
language = "zh_CN"
|
||||
master_doc = "index"
|
||||
exclude_patterns = ["_build", "Thumbs.db", ".DS_Store"]
|
||||
source_suffix = {
|
||||
".rst": "restructuredtext",
|
||||
".md": "markdown",
|
||||
}
|
||||
@@ -0,0 +1,158 @@
|
||||
pf 统一 CLI 入口
|
||||
================
|
||||
|
||||
所有工具通过 ``pf <tool> [command] [options]`` 调用。工具定义在 ``cli/configs/`` 目录下的 YAML 文件中。
|
||||
|
||||
基本用法
|
||||
--------
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
pf # 列出所有可用工具
|
||||
pf filedate # 查看 filedate 工具帮助
|
||||
pf filedate add a.txt # 调用 filedate 的 add 子命令
|
||||
pf gitt c # 调用 gittool 的 c 子命令
|
||||
pf pymake b # 调用 pymake 的 b 别名
|
||||
|
||||
全局选项
|
||||
--------
|
||||
|
||||
所有 YAML 工具支持以下全局选项:
|
||||
|
||||
.. list-table::
|
||||
:header-rows: 1
|
||||
:widths: 25 75
|
||||
|
||||
* - 选项
|
||||
- 说明
|
||||
* - ``--dry-run``
|
||||
- 仅打印执行计划,不执行
|
||||
* - ``--quiet`` / ``-q``
|
||||
- 减少输出,不显示执行过程
|
||||
* - ``--strategy``
|
||||
- 执行策略(``sequential`` / ``thread`` / ``async`` / ``dependency``)
|
||||
* - ``--list``
|
||||
- 列出所有任务名后退出
|
||||
|
||||
默认 ``verbose`` 开启,显示执行过程(任务开始/命令/返回码/任务成功)。``--quiet`` 关闭。
|
||||
|
||||
YAML 配置工具
|
||||
--------------
|
||||
|
||||
.. list-table::
|
||||
:header-rows: 1
|
||||
:widths: 20 15 65
|
||||
|
||||
* - 工具
|
||||
- 别名
|
||||
- 说明
|
||||
* - ``filedate``
|
||||
- ``fd``
|
||||
- 文件日期处理
|
||||
* - ``filelevel``
|
||||
- ``fl``
|
||||
- 文件等级重命名
|
||||
* - ``folderback``
|
||||
- ``fb``
|
||||
- 文件夹备份
|
||||
* - ``folderzip``
|
||||
- ``fz``
|
||||
- 文件夹压缩
|
||||
* - ``gittool``
|
||||
- ``gitt``
|
||||
- Git 执行工具
|
||||
* - ``lscalc``
|
||||
- ``ls``
|
||||
- LS-DYNA 计算工具
|
||||
* - ``packtool``
|
||||
- ``pack``
|
||||
- Python 打包工具
|
||||
* - ``pdftool``
|
||||
- ``pdf``
|
||||
- PDF 文件工具集
|
||||
* - ``piptool``
|
||||
- ``pip``
|
||||
- pip 包管理工具
|
||||
* - ``screenshot``
|
||||
- ``ss``
|
||||
- 截图工具
|
||||
* - ``sshcopyid``
|
||||
- ``ssh``
|
||||
- SSH 密钥部署工具
|
||||
* - ``autofmt``
|
||||
- ``af``
|
||||
- 自动格式化工具
|
||||
* - ``bumpversion``
|
||||
- ``bump``
|
||||
- 版本号自动管理工具
|
||||
|
||||
传统工具
|
||||
--------
|
||||
|
||||
.. list-table::
|
||||
:header-rows: 1
|
||||
:widths: 20 80
|
||||
|
||||
* - 工具
|
||||
- 说明
|
||||
* - ``pymake``
|
||||
- 构建工具(替代 Makefile),如 ``pf pymake b`` 构建
|
||||
* - ``yamlrun``
|
||||
- YAML pipeline 执行器,``pf yamlrun pipeline.yaml``
|
||||
* - ``profiler``
|
||||
- 性能分析
|
||||
* - ``emlman``
|
||||
- 邮件管理
|
||||
* - ``reseticon``
|
||||
- 重置图标缓存
|
||||
|
||||
自定义工具
|
||||
----------
|
||||
|
||||
在 ``cli/configs/`` 目录新建 ``<tool>.yaml`` 即可被 ``pf`` 自动发现:
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
# cli/configs/mytool.yaml
|
||||
strategy: sequential
|
||||
variables:
|
||||
MSG: "hello"
|
||||
cli:
|
||||
description: "我的工具"
|
||||
usage: "pf mytool [command]"
|
||||
subcommands:
|
||||
greet:
|
||||
help: "打招呼"
|
||||
jobs:
|
||||
greet:
|
||||
cmd: ["echo", "${MSG}"]
|
||||
|
||||
执行::
|
||||
|
||||
pf mytool greet
|
||||
|
||||
CliRunner(编程式)
|
||||
-------------------
|
||||
|
||||
``CliRunner`` 把多个 Graph 映射为命令行子命令,适合构建项目专属构建工具:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
runner = px.CliRunner(
|
||||
strategy="sequential",
|
||||
description="My Build Tool",
|
||||
graphs={
|
||||
"clean": clean_graph,
|
||||
"build": build_graph,
|
||||
"test": test_graph,
|
||||
},
|
||||
)
|
||||
runner.run_cli() # 解析 sys.argv 并执行
|
||||
|
||||
命令行::
|
||||
|
||||
pf pymake clean
|
||||
pf pymake build --strategy thread
|
||||
pf pymake test --dry-run
|
||||
pf pymake --list
|
||||
pf pymake --quiet
|
||||
@@ -0,0 +1,93 @@
|
||||
执行策略与 run()
|
||||
=================
|
||||
|
||||
``run()`` 是执行入口,支持四种策略:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
report = px.run(
|
||||
graph,
|
||||
strategy="async", # sequential | thread | async | dependency
|
||||
max_workers=8, # thread 策略的线程池大小
|
||||
concurrency_limits={"db": 2}, # 按 concurrency_key 限流
|
||||
dry_run=False, # True = 仅打印计划
|
||||
verbose=True, # True = 打印执行过程
|
||||
on_event=callback, # 状态转换回调
|
||||
state=px.JSONBackend("state.json"), # 断点续跑后端
|
||||
continue_on_error=False, # True = 单任务失败不中断整体
|
||||
)
|
||||
|
||||
策略对比
|
||||
--------
|
||||
|
||||
.. list-table::
|
||||
:header-rows: 1
|
||||
:widths: 18 18 30 16 18
|
||||
|
||||
* - 策略
|
||||
- 并发模型
|
||||
- 适用场景
|
||||
- 同步任务
|
||||
- 异步任务
|
||||
* - ``sequential``
|
||||
- 串行
|
||||
- 调试、CPU 密集
|
||||
- 直接调用
|
||||
- 事件循环
|
||||
* - ``thread``
|
||||
- 线程池
|
||||
- I/O 密集同步
|
||||
- 线程池
|
||||
- 不支持
|
||||
* - ``async``
|
||||
- 事件循环
|
||||
- I/O 密集异步
|
||||
- 卸载到线程池
|
||||
- 事件循环
|
||||
* - ``dependency``
|
||||
- 依赖驱动
|
||||
- 最大化并行度
|
||||
- 卸载到线程池
|
||||
- 事件循环
|
||||
|
||||
所有策略都遵循 ``RetryPolicy``、``timeout``、上下文注入、状态后端、``concurrency_limits``,
|
||||
并发出 ``TaskEvent``(RUNNING/SUCCESS/FAILED/SKIPPED)。``dependency`` 策略无层屏障:
|
||||
任务在其所有硬依赖完成后立即启动。
|
||||
|
||||
上下文注入规则
|
||||
--------------
|
||||
|
||||
按顺序求值:
|
||||
|
||||
1. **标注为 ``Context``** 的参数 → 接收完整上游结果映射
|
||||
2. **名称匹配依赖** 的参数 → 接收该依赖的结果(含软依赖,缺失时注入默认值)
|
||||
3. **``**kwargs``** 参数 → 接收所有依赖结果(dict)
|
||||
4. **``TaskSpec.args`` / ``kwargs``** → 为非依赖参数提供静态值
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
from typing import Any, Dict
|
||||
|
||||
def aggregate(ctx: px.Context) -> Dict[str, Any]:
|
||||
"""ctx 包含所有 depends_on 任务的返回值。"""
|
||||
return dict(ctx)
|
||||
|
||||
def merge(fetch_a: str, fetch_b: str) -> str:
|
||||
"""fetch_a / fetch_b 自动注入。"""
|
||||
return fetch_a + fetch_b
|
||||
|
||||
断点续跑
|
||||
--------
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
from pyflowx import JSONBackend
|
||||
|
||||
backend = JSONBackend("state.json", ttl=3600)
|
||||
report = px.run(graph, strategy="sequential", state=backend)
|
||||
|
||||
``run()`` 内部以 ``backend.batch()`` 包裹整个执行:所有 ``save`` 延迟到运行结束时统一落盘一次。
|
||||
|
||||
缓存键:默认存储键为任务名。配置 ``cache_key`` 函数后,键为 ``"name:cache_key_value"``。
|
||||
|
||||
完整 API 说明详见 :doc:`/api`。
|
||||
@@ -0,0 +1,50 @@
|
||||
Graph —— DAG 构建
|
||||
=================
|
||||
|
||||
``Graph`` 管理任务集合,提供建构建、校验、分层、可视化能力。
|
||||
|
||||
构建方式
|
||||
--------
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
# 图级默认值:TaskSpec 字段为 None 时回退
|
||||
defaults = px.GraphDefaults(retry=px.RetryPolicy(max_attempts=2), timeout=60.0)
|
||||
|
||||
graph = px.Graph.from_specs([...], defaults=defaults) # 整批校验(推荐)
|
||||
|
||||
# 或增量构建
|
||||
graph = px.Graph(defaults=defaults)
|
||||
graph.add(px.TaskSpec("a", fn_a))
|
||||
graph.add(px.TaskSpec("b", fn_b, ("a",)))
|
||||
|
||||
常用方法
|
||||
--------
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
graph.validate() # 显式校验(环检测)
|
||||
graph.layers() # 拓扑分层(Kahn 算法)
|
||||
graph.to_mermaid() # Mermaid 可视化
|
||||
graph.describe() # 人类可读摘要
|
||||
graph.subgraph(("api",)) # 按标签切片
|
||||
graph.subgraph_by_names(("a", "b")) # 按名称切片
|
||||
graph.map("fetch", [1, 2, 3], lambda i: TaskSpec(f"fetch_{i}", ...)) # 批量 fan-out
|
||||
|
||||
图组合
|
||||
------
|
||||
|
||||
``compose`` / ``GraphComposer`` 把带字符串引用的多个图展开为纯 ``Graph``:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
graphs = {
|
||||
"build": px.Graph.from_specs([px.TaskSpec("b", cmd=["echo", "b"])]),
|
||||
"all": px.Graph.from_specs(["build", px.TaskSpec("t", cmd=["echo", "t"])]),
|
||||
}
|
||||
resolved = px.compose(graphs) # "all" 图中的 "build" 引用被展开
|
||||
|
||||
引用格式:``"command_name"``(整个图)或 ``"command_name.task_name"``(特定任务)。
|
||||
``CliRunner`` 内部自动调用 ``compose``。
|
||||
|
||||
完整方法说明详见 :doc:`/api`。
|
||||
@@ -0,0 +1,89 @@
|
||||
TaskSpec —— 任务描述
|
||||
=====================
|
||||
|
||||
``TaskSpec`` 是不可变的任务描述符(``Generic[T]``,返回类型一路传到 ``RunReport``),是唯一需要配置的东西。
|
||||
|
||||
主要参数说明:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
px.TaskSpec(
|
||||
name="fetch_user", # 唯一标识
|
||||
fn=fetch_user, # 同步或异步函数
|
||||
cmd=["curl", "..."], # 或: 执行命令(覆盖 fn)
|
||||
depends_on=("auth",), # 硬依赖(参与拓扑分层)
|
||||
soft_depends_on=("cache",), # 软依赖(仅注入,不参与分层)
|
||||
args=(uid,), # 静态位置参数(追加在注入参数后)
|
||||
kwargs={"timeout": 30}, # 静态关键字参数
|
||||
retry=px.RetryPolicy(max_attempts=3, delay=1.0, backoff=2.0),
|
||||
timeout=30.0, # 超时秒数(None = 不限制)
|
||||
tags=("api", "user"), # 自由标签,用于子图过滤
|
||||
conditions=(is_prod,), # 条件函数列表(全部为 True 才执行)
|
||||
priority=10, # 同层内优先级(高优先执行,默认 0)
|
||||
concurrency_key="db", # 并发分组键(配合 concurrency_limits 限流)
|
||||
cache_key=lambda ctx: str(ctx.get("uid")), # 缓存键函数
|
||||
hooks=px.TaskHooks(pre_run=..., post_run=..., on_failure=...),
|
||||
cwd=Path("/tmp"), # 命令工作目录(仅 cmd 模式)
|
||||
env={"DEBUG": "1"}, # 环境变量覆盖
|
||||
verbose=True, # 打印命令输出(仅 cmd 模式)
|
||||
skip_if_missing=True, # 命令不存在时自动跳过(仅 list[str] cmd)
|
||||
allow_upstream_skip=False, # 上游 SKIPPED/FAILED 时是否仍执行
|
||||
continue_on_error=False, # 本任务失败是否不中断整体
|
||||
)
|
||||
|
||||
两种任务形态
|
||||
------------
|
||||
|
||||
- **函数任务**(``fn``):普通 Python 函数,参数名驱动自动注入
|
||||
- **命令任务**(``cmd``):执行外部命令,支持 ``list[str]``、``str``(shell)、``Callable`` 三种形态
|
||||
|
||||
``skip_if_missing=True`` 时,``list[str]`` 类型的 ``cmd`` 会通过 ``shutil.which`` 检查命令是否存在,不存在则跳过任务(标记为 ``SKIPPED``)而非失败。
|
||||
|
||||
重试策略
|
||||
--------
|
||||
|
||||
``RetryPolicy`` 配置重试次数、延迟、退避:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
retry = px.RetryPolicy(
|
||||
max_attempts=3, # 最大尝试次数
|
||||
delay=1.0, # 初始延迟秒数
|
||||
backoff=2.0, # 退避倍数
|
||||
jitter=0.1, # 随机抖动(避免惊群)
|
||||
retry_on=(ConnectionError,), # 仅对这些异常重试
|
||||
)
|
||||
|
||||
任务钩子
|
||||
--------
|
||||
|
||||
``TaskHooks`` 在任务生命周期触发(异常仅记录,不影响任务状态):
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
hooks = px.TaskHooks(
|
||||
pre_run=lambda spec: print(f"start {spec.name}"),
|
||||
post_run=lambda spec, value: print(f"done {spec.name}"),
|
||||
on_failure=lambda spec, exc: alert(spec.name, exc),
|
||||
)
|
||||
px.TaskSpec("task", fn=work, hooks=hooks)
|
||||
|
||||
任务模板
|
||||
--------
|
||||
|
||||
``task_template`` 工厂批量生成相似 TaskSpec:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
fetch = px.task_template(
|
||||
fn=fetch_url,
|
||||
retry=px.RetryPolicy(max_attempts=5),
|
||||
timeout=30.0,
|
||||
tags=("api",),
|
||||
)
|
||||
graph = px.Graph.from_specs([
|
||||
fetch("users", url="https://api.example.com/users"),
|
||||
fetch("posts", url="https://api.example.com/posts"),
|
||||
])
|
||||
|
||||
完整字段说明详见 :doc:`/api`。
|
||||
@@ -0,0 +1,164 @@
|
||||
YAML 任务编排
|
||||
=============
|
||||
|
||||
PyFlowX 支持 GitHub Actions 风格的声明式 YAML 任务编排,从 YAML 文件直接加载任务图。
|
||||
|
||||
编程式 API
|
||||
----------
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
import pyflowx as px
|
||||
|
||||
# 从 YAML 文件加载任务图
|
||||
graph = px.Graph.from_yaml("pipeline.yaml")
|
||||
report = px.run(graph, strategy="thread")
|
||||
|
||||
# 或用函数式 API
|
||||
graph = px.load_yaml("pipeline.yaml")
|
||||
|
||||
# 从字符串解析
|
||||
graph = px.parse_yaml_string("""
|
||||
jobs:
|
||||
hello:
|
||||
cmd: ["echo", "hello"]
|
||||
""")
|
||||
|
||||
YAML Schema
|
||||
-----------
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
strategy: thread # 图级默认策略
|
||||
defaults: # 图级默认值
|
||||
retry: {max_attempts: 3}
|
||||
verbose: true
|
||||
env: {CI: "true"}
|
||||
|
||||
variables: # 变量定义 (可在 cmd/env 中 ${VAR} 引用)
|
||||
OUTPUT: "dist"
|
||||
|
||||
jobs:
|
||||
setup:
|
||||
cmd: ["git", "clone", "..."]
|
||||
runs-on: linux
|
||||
|
||||
build:
|
||||
needs: [setup] # 依赖列表
|
||||
cmd: ["python", "-m", "build"]
|
||||
timeout: 300
|
||||
retry: {max_attempts: 2, delay: 1.0}
|
||||
|
||||
test:
|
||||
needs: [build]
|
||||
cmd: ["python${{ matrix.version }}", "-m", "pytest"]
|
||||
strategy:
|
||||
matrix: # 笛卡尔积展开为 6 个任务
|
||||
version: ["3.8", "3.9", "3.10"]
|
||||
os: ["linux", "macos"]
|
||||
if: "env.CI" # 条件: 环境变量存在
|
||||
|
||||
lint:
|
||||
needs: [build]
|
||||
cmd: ["ruff", "check"]
|
||||
if: "env.CI == 'true'"
|
||||
|
||||
deploy:
|
||||
needs: [test, lint] # 矩阵依赖自动展开
|
||||
cmd: ["twine", "upload"]
|
||||
if: "env.DEPLOY_TOKEN != ''"
|
||||
allow-upstream-skip: true
|
||||
concurrency-key: deploy_lock
|
||||
|
||||
字段映射
|
||||
--------
|
||||
|
||||
.. list-table::
|
||||
:header-rows: 1
|
||||
:widths: 30 30 40
|
||||
|
||||
* - YAML 字段
|
||||
- TaskSpec 字段
|
||||
- 说明
|
||||
* - ``jobs.<id>``
|
||||
- ``name``
|
||||
- job ID 作为任务名
|
||||
* - ``cmd`` / ``run``
|
||||
- ``cmd``
|
||||
- ``cmd`` 为列表形式,``run`` 为 shell 字符串
|
||||
* - ``needs``
|
||||
- ``depends_on``
|
||||
- 依赖列表(矩阵任务自动展开)
|
||||
* - ``if``
|
||||
- ``conditions``
|
||||
- ``success()`` / ``always()`` / ``env.VAR`` / ``env.VAR == 'x'``
|
||||
* - ``strategy.matrix``
|
||||
- 矩阵扇出
|
||||
- 笛卡尔积展开为多个任务
|
||||
* - ``${{ matrix.key }}``
|
||||
- 占位符
|
||||
- 在 cmd/run/cwd/env 中替换
|
||||
* - ``timeout``
|
||||
- ``timeout``
|
||||
- 超时秒数
|
||||
* - ``retry``
|
||||
- ``retry``
|
||||
- ``{max_attempts, delay, backoff, jitter}``
|
||||
* - ``cwd``
|
||||
- ``cwd``
|
||||
- 工作目录
|
||||
* - ``env``
|
||||
- ``env``
|
||||
- 环境变量
|
||||
* - ``verbose``
|
||||
- ``verbose``
|
||||
- 详细输出
|
||||
* - ``continue-on-error``
|
||||
- ``continue_on_error``
|
||||
- 失败不中止整图
|
||||
* - ``skip-if-missing``
|
||||
- ``skip_if_missing``
|
||||
- 命令不存在时跳过
|
||||
* - ``allow-upstream-skip``
|
||||
- ``allow_upstream_skip``
|
||||
- 上游跳过时仍执行
|
||||
* - ``priority``
|
||||
- ``priority``
|
||||
- 同层优先级
|
||||
* - ``concurrency-key``
|
||||
- ``concurrency_key``
|
||||
- 并发限制键
|
||||
* - ``tags``
|
||||
- ``tags``
|
||||
- 自由标签
|
||||
* - ``runs-on``
|
||||
- ``tags``(追加)
|
||||
- 运行环境标签
|
||||
|
||||
CLI 配置段(``cli:``)
|
||||
----------------------
|
||||
|
||||
工具 YAML 还可定义 ``cli:`` 段,声明命令行参数 schema,由 ``pf`` 自动解析:
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
cli:
|
||||
description: "FileDate - 文件日期处理工具"
|
||||
usage: "pf filedate <command> [files...]"
|
||||
subcommands:
|
||||
add:
|
||||
help: "添加日期前缀"
|
||||
positional:
|
||||
- name: FILES
|
||||
nargs: "+"
|
||||
type: path
|
||||
help: "文件路径"
|
||||
options:
|
||||
- name: CLEAR
|
||||
flag: "--clear"
|
||||
action: store_true
|
||||
help: "清除已有日期前缀"
|
||||
|
||||
支持的 ``type``:``str`` / ``int`` / ``float`` / ``path``。
|
||||
|
||||
完整 API 说明详见 :doc:`/api`。
|
||||
@@ -0,0 +1,56 @@
|
||||
PyFlowX 文档
|
||||
============
|
||||
|
||||
PyFlowX 是一个轻量、类型安全的 DAG 任务调度器:**参数名就是依赖声明**。
|
||||
无需装饰器、无需样板包装器,写一个普通函数,框架按参数名自动注入上游结果。
|
||||
|
||||
特性
|
||||
----
|
||||
|
||||
- **零样板** —— 参数名即依赖,框架自动注入上游结果
|
||||
- **四种执行策略** —— sequential(串行)、thread(线程池)、async(事件循环)、dependency(依赖驱动,最大化并行)
|
||||
- **类型安全** —— ``TaskSpec[T]`` 把返回类型一路传到 ``RunReport``
|
||||
- **DAG 校验** —— 构建时即时校验重名、缺失依赖、环
|
||||
- **自动分层** —— Kahn 算法分组,同层任务可并行
|
||||
- **重试与超时** —— 每个任务独立配置 ``RetryPolicy`` 与 ``timeout``
|
||||
- **并发限制** —— ``concurrency_key`` + ``concurrency_limits`` 按组限流
|
||||
- **断点续跑** —— ``MemoryBackend`` / ``JSONBackend``,成功结果可缓存复用
|
||||
- **命令任务** —— ``cmd`` 参数直接执行外部命令
|
||||
- **条件执行** —— ``conditions`` 按平台、环境变量等条件跳过任务
|
||||
- **YAML 任务编排** —— GitHub Actions 风格声明式任务图
|
||||
- **pf 统一 CLI** —— ``pf <tool> [command]`` 调用所有工具
|
||||
- **最小依赖** —— 仅依赖标准库 + PyYAML
|
||||
|
||||
文档导航
|
||||
--------
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
:caption: 入门
|
||||
|
||||
installation
|
||||
quickstart
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
:caption: 用户指南
|
||||
|
||||
guide/task
|
||||
guide/graph
|
||||
guide/execution
|
||||
guide/yaml
|
||||
guide/cli
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
:caption: 参考
|
||||
|
||||
api
|
||||
changelog
|
||||
|
||||
索引
|
||||
----
|
||||
|
||||
* :ref:`genindex`
|
||||
* :ref:`modindex`
|
||||
* :ref:`search`
|
||||
@@ -0,0 +1,51 @@
|
||||
安装
|
||||
====
|
||||
|
||||
PyFlowX 支持 Python 3.8+,仅依赖标准库与 PyYAML(3.8 额外需要 ``graphlib_backport`` 和 ``typing-extensions``)。
|
||||
|
||||
pip 安装
|
||||
--------
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
pip install pyflowx
|
||||
|
||||
uv 安装
|
||||
-------
|
||||
|
||||
推荐使用 `uv <https://docs.astral.sh/uv/>`_:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
uv add pyflowx
|
||||
|
||||
可选依赖
|
||||
--------
|
||||
|
||||
``office`` —— PDF/图片处理(pdftool、screenshot 等工具需要):
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
pip install pyflowx[office]
|
||||
|
||||
``dev`` —— 开发工具链(ruff、pyrefly、pytest、tox 等):
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
pip install pyflowx[dev]
|
||||
|
||||
验证安装
|
||||
--------
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
pf --version
|
||||
|
||||
输出示例::
|
||||
|
||||
PyFlowX 0.4.5
|
||||
|
||||
下一步
|
||||
------
|
||||
|
||||
前往 :doc:`quickstart` 开始使用。
|
||||
@@ -0,0 +1,87 @@
|
||||
快速上手
|
||||
========
|
||||
|
||||
核心思想:**参数名即依赖**。写一个普通函数,参数名匹配上游任务名,框架自动注入结果。
|
||||
|
||||
最小示例
|
||||
--------
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
import pyflowx as px
|
||||
|
||||
def extract() -> list[int]:
|
||||
return [1, 2, 3]
|
||||
|
||||
# 参数名 extract 自动匹配上游任务名 → 自动注入
|
||||
def double(extract: list[int]) -> list[int]:
|
||||
return [x * 2 for x in extract]
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("extract", extract),
|
||||
px.TaskSpec("double", double, ("extract",)),
|
||||
])
|
||||
|
||||
report = px.run(graph, strategy="sequential")
|
||||
print(report["double"]) # [2, 4, 6]
|
||||
|
||||
三种任务形态
|
||||
------------
|
||||
|
||||
1. **函数任务**(``fn``):普通 Python 函数,参数名驱动自动注入
|
||||
2. **命令任务**(``cmd``):执行外部命令,支持 ``list[str]`` / ``str``(shell)/ ``Callable``
|
||||
3. **YAML 声明式**:从 YAML 文件加载任务图
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("list", cmd=["ls", "-la"]),
|
||||
px.TaskSpec("greet", fn=lambda: "hello"),
|
||||
])
|
||||
|
||||
执行策略
|
||||
--------
|
||||
|
||||
PyFlowX 提供四种执行策略:
|
||||
|
||||
.. list-table::
|
||||
:header-rows: 1
|
||||
:widths: 20 20 60
|
||||
|
||||
* - 策略
|
||||
- 并发模型
|
||||
- 适用场景
|
||||
* - ``sequential``
|
||||
- 串行
|
||||
- 调试、CPU 密集
|
||||
* - ``thread``
|
||||
- 线程池
|
||||
- I/O 密集同步
|
||||
* - ``async``
|
||||
- 事件循环
|
||||
- I/O 密集异步
|
||||
* - ``dependency``
|
||||
- 依赖驱动
|
||||
- 最大化并行度(默认推荐)
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
report = px.run(graph, strategy="dependency")
|
||||
|
||||
结果访问
|
||||
--------
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
report["task_name"] # 任务返回值
|
||||
report.result_of("task_name") # 完整 TaskResult
|
||||
report.success # 整体是否成功
|
||||
report.summary() # 统计字典
|
||||
report.failed_tasks() # 失败任务名列表
|
||||
|
||||
下一步
|
||||
------
|
||||
|
||||
- :doc:`guide/task` —— TaskSpec 详细配置
|
||||
- :doc:`guide/yaml` —— YAML 声明式任务编排
|
||||
- :doc:`guide/cli` —— ``pf`` 统一 CLI 入口
|
||||
+13
-23
@@ -13,7 +13,8 @@ classifiers = [
|
||||
]
|
||||
dependencies = [
|
||||
"graphlib_backport >= 1.0.0; python_version < '3.9'",
|
||||
"typing-extensions>=4.13.2; python_version < '3.10'",
|
||||
"pyyaml>=6.0.1",
|
||||
"typing-extensions>=4.13.2; python_version < '3.13'",
|
||||
]
|
||||
description = "Lightweight, type-safe DAG task scheduler with multi-strategy execution."
|
||||
keywords = ["async", "dag", "scheduler", "task", "workflow"]
|
||||
@@ -21,29 +22,16 @@ license = { text = "MIT" }
|
||||
name = "pyflowx"
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.8"
|
||||
version = "0.3.2"
|
||||
version = "0.4.6"
|
||||
|
||||
[project.scripts]
|
||||
autofmt = "pyflowx.cli.autofmt:main"
|
||||
bumpversion = "pyflowx.cli.bumpversion:main"
|
||||
dockercmd = "pyflowx.cli.dev.dockercmd:main"
|
||||
emlman = "pyflowx.cli.emlmanager:main"
|
||||
filedate = "pyflowx.cli.filedate:main"
|
||||
filelvl = "pyflowx.cli.filelevel:main"
|
||||
foldback = "pyflowx.cli.folderback:main"
|
||||
foldzip = "pyflowx.cli.folderzip:main"
|
||||
gitt = "pyflowx.cli.gittool:main"
|
||||
lscalc = "pyflowx.cli.lscalc:main"
|
||||
msdown = "pyflowx.cli.llm.msdownload:main"
|
||||
packtool = "pyflowx.cli.packtool:main"
|
||||
pdftool = "pyflowx.cli.pdftool:main"
|
||||
piptool = "pyflowx.cli.piptool:main"
|
||||
pxp = "pyflowx.cli.profiler:main"
|
||||
pymake = "pyflowx.cli.pymake:main"
|
||||
reseticon = "pyflowx.cli.reseticoncache:main"
|
||||
scrcap = "pyflowx.cli.screenshot:main"
|
||||
sglang = "pyflowx.cli.llm.sglang:main"
|
||||
sshcopy = "pyflowx.cli.sshcopyid:main"
|
||||
dockercmd = "pyflowx.cli.dev.dockercmd:main"
|
||||
emlman = "pyflowx.cli.emlmanager:main"
|
||||
msdown = "pyflowx.cli.llm.msdownload:main"
|
||||
pf = "pyflowx.cli.pf:main"
|
||||
pxp = "pyflowx.cli.profiler:main"
|
||||
sglang = "pyflowx.cli.llm.sglang:main"
|
||||
yamlrun = "pyflowx.cli.yamlrun:main"
|
||||
# dev
|
||||
envdev = "pyflowx.cli.dev.envdev:main"
|
||||
# system
|
||||
@@ -66,7 +54,9 @@ dev = [
|
||||
"ruff>=0.8.0",
|
||||
"tox-uv>=1.13.1",
|
||||
"tox>=4.25.0",
|
||||
"types-PyYAML>=6.0.12",
|
||||
]
|
||||
docs = ["myst-parser>=3.0", "sphinx-rtd-theme>=2.0", "sphinx>=7.0"]
|
||||
office = [
|
||||
"pillow>=10.4.0",
|
||||
"pymupdf>=1.24.11",
|
||||
@@ -95,7 +85,7 @@ packages = ["src/pyflowx"]
|
||||
pyflowx = { workspace = true }
|
||||
|
||||
[dependency-groups]
|
||||
dev = ["pyflowx[dev,office]"]
|
||||
dev = ["pyflowx[dev,docs,office]"]
|
||||
|
||||
[tool.coverage.run]
|
||||
branch = true
|
||||
|
||||
+13
-1
@@ -83,6 +83,7 @@ from .errors import (
|
||||
from .executors import Strategy, run
|
||||
from .graph import Graph, GraphDefaults
|
||||
from .profiling import ProfileReport, TaskProfile
|
||||
from .registry import FnRegistry, get_fn, has_fn, register_fn
|
||||
from .report import RunReport
|
||||
from .runner import CliExitCode, CliRunner
|
||||
from .storage import JSONBackend, MemoryBackend, StateBackend
|
||||
@@ -99,8 +100,9 @@ from .task import (
|
||||
task,
|
||||
task_template,
|
||||
)
|
||||
from .yaml_loader import YamlLoadError, build_cli_parser, load_yaml, parse_yaml_string, run_cli, run_yaml
|
||||
|
||||
__version__ = "0.4.2"
|
||||
__version__ = "0.4.6"
|
||||
|
||||
__all__ = [
|
||||
"IS_LINUX",
|
||||
@@ -116,6 +118,7 @@ __all__ = [
|
||||
"Context",
|
||||
"CycleError",
|
||||
"DuplicateTaskError",
|
||||
"FnRegistry",
|
||||
"Graph",
|
||||
"GraphComposer",
|
||||
"GraphDefaults",
|
||||
@@ -139,12 +142,21 @@ __all__ = [
|
||||
"TaskSpec",
|
||||
"TaskStatus",
|
||||
"TaskTimeoutError",
|
||||
"YamlLoadError",
|
||||
"build_call_args",
|
||||
"build_cli_parser",
|
||||
"cmd",
|
||||
"compose",
|
||||
"describe_injection",
|
||||
"get_fn",
|
||||
"has_fn",
|
||||
"load_yaml",
|
||||
"parse_yaml_string",
|
||||
"register_fn",
|
||||
"run",
|
||||
"run_cli",
|
||||
"run_command",
|
||||
"run_yaml",
|
||||
"task",
|
||||
"task_template",
|
||||
]
|
||||
|
||||
@@ -1,282 +0,0 @@
|
||||
"""自动格式化工具模块.
|
||||
|
||||
提供 Python 代码自动格式化的常用功能封装,
|
||||
支持 docstring 自动生成、pyproject.toml 配置同步等功能.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import ast
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
import pyflowx as px
|
||||
|
||||
try:
|
||||
import tomllib # noqa: F401
|
||||
|
||||
HAS_TOMLLIB = True
|
||||
except ImportError:
|
||||
HAS_TOMLLIB = False
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# 配置
|
||||
# ============================================================================
|
||||
|
||||
IGNORE_PATTERNS = [
|
||||
"__pycache__",
|
||||
"*.pyc",
|
||||
"*.pyo",
|
||||
".git",
|
||||
".venv",
|
||||
".idea",
|
||||
".vscode",
|
||||
"*.egg-info",
|
||||
"dist",
|
||||
"build",
|
||||
".pytest_cache",
|
||||
".tox",
|
||||
".mypy_cache",
|
||||
]
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# 辅助函数
|
||||
# ============================================================================
|
||||
|
||||
|
||||
def format_with_ruff(target: Path, fix: bool = True) -> None:
|
||||
"""使用 ruff 格式化代码.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
target : Path
|
||||
目标路径
|
||||
fix : bool
|
||||
是否自动修复
|
||||
"""
|
||||
cmd = ["ruff", "format", str(target)]
|
||||
if fix:
|
||||
cmd.append("--fix")
|
||||
|
||||
subprocess.run(cmd, check=True)
|
||||
print(f"ruff format 完成: {target}")
|
||||
|
||||
|
||||
def lint_with_ruff(target: Path, fix: bool = True) -> None:
|
||||
"""使用 ruff 检查代码.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
target : Path
|
||||
目标路径
|
||||
fix : bool
|
||||
是否自动修复
|
||||
"""
|
||||
cmd = ["ruff", "check", str(target)]
|
||||
if fix:
|
||||
cmd.extend(["--fix", "--unsafe-fixes"])
|
||||
|
||||
subprocess.run(cmd, check=True)
|
||||
print(f"ruff check 完成: {target}")
|
||||
|
||||
|
||||
def add_docstring(file_path: Path, docstring: str) -> bool:
|
||||
"""为文件添加 docstring.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
file_path : Path
|
||||
文件路径
|
||||
docstring : str
|
||||
docstring 内容
|
||||
|
||||
Returns
|
||||
-------
|
||||
bool
|
||||
是否成功添加
|
||||
"""
|
||||
try:
|
||||
content = file_path.read_text(encoding="utf-8")
|
||||
tree = ast.parse(content)
|
||||
|
||||
# 检查是否已有 docstring
|
||||
first_node = tree.body[0] if tree.body else None
|
||||
if first_node and isinstance(first_node, ast.Expr) and isinstance(first_node.value, ast.Constant):
|
||||
return False
|
||||
|
||||
# 添加 docstring
|
||||
lines = content.splitlines()
|
||||
doc_lines = docstring.splitlines()
|
||||
doc_lines.append("")
|
||||
new_content = "\n".join(doc_lines + lines)
|
||||
|
||||
file_path.write_text(new_content, encoding="utf-8")
|
||||
print(f"添加 docstring: {file_path}")
|
||||
return True
|
||||
|
||||
except (OSError, UnicodeDecodeError, SyntaxError) as e:
|
||||
print(f"处理失败: {file_path} - {e}")
|
||||
return False
|
||||
|
||||
|
||||
def generate_module_docstring(file_path: Path) -> str:
|
||||
"""生成模块 docstring.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
file_path : Path
|
||||
文件路径
|
||||
|
||||
Returns
|
||||
-------
|
||||
str
|
||||
生成的 docstring
|
||||
"""
|
||||
stem = file_path.stem
|
||||
parent = file_path.parent.name
|
||||
|
||||
# 关键词匹配
|
||||
keywords = {
|
||||
"cli": f"Command-line interface for {parent}",
|
||||
"gui": f"Graphical user interface for {parent}",
|
||||
"core": f"Core functionality for {parent}",
|
||||
"util": f"Utility functions for {parent}",
|
||||
"model": f"Data models for {parent}",
|
||||
"test": f"Tests for {parent}",
|
||||
}
|
||||
|
||||
for key, desc in keywords.items():
|
||||
if key in stem.lower():
|
||||
return f'"""{desc}."""'
|
||||
|
||||
return f'"""{stem.replace("_", " ").title()} module."""'
|
||||
|
||||
|
||||
def auto_add_docstrings(root_dir: Path) -> int:
|
||||
"""自动为所有 Python 文件添加 docstring.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
root_dir : Path
|
||||
根目录
|
||||
|
||||
Returns
|
||||
-------
|
||||
int
|
||||
添加的 docstring 数量
|
||||
"""
|
||||
count = 0
|
||||
for py_file in root_dir.rglob("*.py"):
|
||||
# 跳过忽略的文件
|
||||
if any(pattern in str(py_file) for pattern in IGNORE_PATTERNS):
|
||||
continue
|
||||
|
||||
docstring = generate_module_docstring(py_file)
|
||||
if add_docstring(py_file, docstring):
|
||||
count += 1
|
||||
|
||||
print(f"共添加 {count} 个 docstring")
|
||||
return count
|
||||
|
||||
|
||||
def sync_pyproject_config(root_dir: Path) -> None:
|
||||
"""同步 pyproject.toml 配置到子项目.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
root_dir : Path
|
||||
根目录
|
||||
"""
|
||||
main_toml = root_dir / "pyproject.toml"
|
||||
if not main_toml.exists():
|
||||
print(f"主项目配置文件不存在: {main_toml}")
|
||||
return
|
||||
|
||||
# 查找所有子项目的 pyproject.toml
|
||||
sub_tomls = [p for p in root_dir.rglob("pyproject.toml") if p != main_toml and ".venv" not in str(p)]
|
||||
|
||||
if not sub_tomls:
|
||||
print("没有找到子项目的 pyproject.toml")
|
||||
return
|
||||
|
||||
print(f"找到 {len(sub_tomls)} 个子项目配置文件")
|
||||
|
||||
# 对每个子项目调用 ruff format
|
||||
for sub_toml in sub_tomls:
|
||||
subprocess.run(["ruff", "format", str(sub_toml)], check=False)
|
||||
|
||||
print("配置同步完成")
|
||||
|
||||
|
||||
def format_all(root_dir: Path) -> None:
|
||||
"""格式化所有 Python 文件.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
root_dir : Path
|
||||
根目录
|
||||
"""
|
||||
# 使用 ruff format
|
||||
subprocess.run(["ruff", "format", str(root_dir)], check=True)
|
||||
|
||||
# 使用 ruff check
|
||||
subprocess.run(["ruff", "check", "--fix", "--unsafe-fixes", str(root_dir)], check=True)
|
||||
|
||||
print(f"格式化完成: {root_dir}")
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# CLI Runner
|
||||
# ============================================================================
|
||||
|
||||
|
||||
def main() -> None:
|
||||
"""自动格式化工具主函数."""
|
||||
parser = argparse.ArgumentParser(
|
||||
description="AutoFmt - 自动格式化工具",
|
||||
usage="autofmt <command> [options]",
|
||||
)
|
||||
subparsers = parser.add_subparsers(dest="command", help="可用命令")
|
||||
|
||||
# ruff format 命令
|
||||
format_parser = subparsers.add_parser("fmt", help="使用 ruff 格式化代码")
|
||||
format_parser.add_argument("--target", type=str, default=".", help="目标路径")
|
||||
|
||||
# ruff check 命令
|
||||
lint_parser = subparsers.add_parser("lint", help="使用 ruff 检查代码")
|
||||
lint_parser.add_argument("--target", type=str, default=".", help="目标路径")
|
||||
lint_parser.add_argument("--fix", action="store_true", help="自动修复")
|
||||
|
||||
# 自动添加 docstring 命令
|
||||
doc_parser = subparsers.add_parser("doc", help="自动添加 docstring")
|
||||
doc_parser.add_argument("--root-dir", type=str, default=".", help="根目录")
|
||||
|
||||
# 同步配置命令
|
||||
sync_parser = subparsers.add_parser("sync", help="同步 pyproject.toml 配置")
|
||||
sync_parser.add_argument("--root-dir", type=str, default=".", help="根目录")
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
if args.command == "fmt":
|
||||
graph = px.Graph.from_specs([px.TaskSpec("ruff_format", cmd=["ruff", "format", args.target], verbose=True)])
|
||||
elif args.command == "lint":
|
||||
cmd = ["ruff", "check", args.target]
|
||||
if args.fix:
|
||||
cmd.extend(["--fix", "--unsafe-fixes"])
|
||||
graph = px.Graph.from_specs([px.TaskSpec("ruff_check", cmd=cmd, verbose=True)])
|
||||
elif args.command == "doc":
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("auto_docstring", fn=auto_add_docstrings, args=(Path(args.root_dir),), verbose=True)
|
||||
])
|
||||
elif args.command == "sync":
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("sync_config", fn=sync_pyproject_config, args=(Path(args.root_dir),), verbose=True)
|
||||
])
|
||||
else:
|
||||
parser.print_help()
|
||||
return
|
||||
|
||||
px.run(graph, strategy="thread")
|
||||
@@ -1,263 +0,0 @@
|
||||
"""版本号自动管理工具.
|
||||
|
||||
使用 TaskSpec 模式实现, 支持语义化版本管理和多文件格式的版本号更新.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import re
|
||||
from pathlib import Path
|
||||
from typing import Literal, get_args
|
||||
|
||||
import pyflowx as px
|
||||
|
||||
BumpVersionType = Literal["patch", "minor", "major"]
|
||||
|
||||
# 针对不同文件类型的版本号匹配模式
|
||||
# pyproject.toml: version = "X.Y.Z" 或 version = 'X.Y.Z'
|
||||
_PYPROJECT_VERSION_PATTERN = re.compile(
|
||||
r'(?:^|\n)\s*version\s*=\s*["\']'
|
||||
r"(?P<major>0|[1-9]\d*)\.(?P<minor>0|[1-9]\d*)\.(?P<patch>0|[1-9]\d*)"
|
||||
r"(?:-(?P<prerelease>(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)(?:\.(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*))?"
|
||||
r"(?:\+(?P<buildmetadata>[0-9a-zA-Z-]+(?:\.[0-9a-zA-Z-]+)*))?"
|
||||
r'["\']',
|
||||
re.MULTILINE,
|
||||
)
|
||||
|
||||
# __init__.py: __version__ = "X.Y.Z" 或 __version__ = 'X.Y.Z'
|
||||
_INIT_VERSION_PATTERN = re.compile(
|
||||
r'(?:^|\n)\s*__version__\s*=\s*["\']'
|
||||
r"(?P<major>0|[1-9]\d*)\.(?P<minor>0|[1-9]\d*)\.(?P<patch>0|[1-9]\d*)"
|
||||
r"(?:-(?P<prerelease>(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)(?:\.(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*))?"
|
||||
r"(?:\+(?P<buildmetadata>[0-9a-zA-Z-]+(?:\.[0-9a-zA-Z-]+)*))?"
|
||||
r'["\']',
|
||||
re.MULTILINE,
|
||||
)
|
||||
|
||||
|
||||
def _get_pattern_for_file(file_name: str) -> re.Pattern[str] | None:
|
||||
"""根据文件类型获取对应的正则表达式.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
file_name : str
|
||||
文件名
|
||||
|
||||
Returns
|
||||
-------
|
||||
re.Pattern[str] | None
|
||||
对应的正则表达式,如果无法确定则返回 None
|
||||
"""
|
||||
if file_name == "pyproject.toml":
|
||||
return _PYPROJECT_VERSION_PATTERN
|
||||
if file_name == "__init__.py":
|
||||
return _INIT_VERSION_PATTERN
|
||||
return None
|
||||
|
||||
|
||||
def _calculate_new_version(major: int, minor: int, patch: int, part: BumpVersionType) -> str:
|
||||
"""计算新版本号.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
major : int
|
||||
当前主版本号
|
||||
minor : int
|
||||
当前次版本号
|
||||
patch : int
|
||||
当前补丁版本号
|
||||
part : BumpVersionType
|
||||
要更新的部分
|
||||
|
||||
Returns
|
||||
-------
|
||||
str
|
||||
新版本号
|
||||
"""
|
||||
if part == "major":
|
||||
return f"{major + 1}.0.0"
|
||||
if part == "minor":
|
||||
return f"{major}.{minor + 1}.0"
|
||||
return f"{major}.{minor}.{patch + 1}"
|
||||
|
||||
|
||||
def _build_replacement_string(original_match: str, new_version: str, file_name: str) -> str:
|
||||
"""构建替换字符串,保留原始格式.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
original_match : str
|
||||
原始匹配的字符串
|
||||
new_version : str
|
||||
新版本号
|
||||
file_name : str
|
||||
文件名
|
||||
|
||||
Returns
|
||||
-------
|
||||
str
|
||||
替换字符串
|
||||
"""
|
||||
quote_char = '"' if '"' in original_match else "'"
|
||||
|
||||
if file_name == "pyproject.toml":
|
||||
prefix_match = re.match(r'(\s*version\s*=\s*)["\']', original_match)
|
||||
prefix = prefix_match.group(1) if prefix_match else "version = "
|
||||
return f"{prefix}{quote_char}{new_version}{quote_char}"
|
||||
|
||||
if file_name == "__init__.py":
|
||||
prefix_match = re.match(r'(\s*__version__\s*=\s*)["\']', original_match)
|
||||
prefix = prefix_match.group(1) if prefix_match else "__version__ = "
|
||||
return f"{prefix}{quote_char}{new_version}{quote_char}"
|
||||
|
||||
return new_version
|
||||
|
||||
|
||||
def bump_file_version(file_path: Path, part: BumpVersionType = "patch") -> str | None:
|
||||
"""更新文件中的版本号.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
file_path : Path
|
||||
要更新的文件路径
|
||||
part : BumpVersionType
|
||||
版本部分: patch, minor, major
|
||||
|
||||
Returns
|
||||
-------
|
||||
str | None
|
||||
更新后的新版本号,如果文件中未找到版本号则返回 None
|
||||
"""
|
||||
try:
|
||||
content = file_path.read_text(encoding="utf-8")
|
||||
except Exception as e:
|
||||
print(f"读取文件 {file_path} 时出错: {e}")
|
||||
raise
|
||||
|
||||
# 获取文件对应的正则表达式
|
||||
pattern = _get_pattern_for_file(file_path.name)
|
||||
|
||||
# 对于未知文件类型,尝试两种模式
|
||||
if pattern:
|
||||
match = pattern.search(content)
|
||||
else:
|
||||
match = _PYPROJECT_VERSION_PATTERN.search(content) or _INIT_VERSION_PATTERN.search(content)
|
||||
|
||||
if not match:
|
||||
print(f"文件 {file_path} 中未找到版本号模式")
|
||||
return None
|
||||
|
||||
# 提取当前版本号
|
||||
major = int(match.group("major"))
|
||||
minor = int(match.group("minor"))
|
||||
patch = int(match.group("patch"))
|
||||
|
||||
# 计算新版本号
|
||||
new_version = _calculate_new_version(major, minor, patch, part)
|
||||
|
||||
# 构建替换字符串
|
||||
original_match = match.group(0)
|
||||
replacement = _build_replacement_string(original_match, new_version, file_path.name)
|
||||
|
||||
# 更新文件内容
|
||||
content = content.replace(original_match, replacement)
|
||||
|
||||
try:
|
||||
file_path.write_text(content, encoding="utf-8")
|
||||
except Exception as e:
|
||||
print(f"更新文件 {file_path} 版本号时出错: {e}")
|
||||
raise
|
||||
|
||||
return new_version
|
||||
|
||||
|
||||
def main() -> None:
|
||||
"""版本号管理工具主函数."""
|
||||
parser = argparse.ArgumentParser(description="BumpVersion - 版本号自动管理工具")
|
||||
parser.add_argument(
|
||||
"part",
|
||||
type=str,
|
||||
nargs="?",
|
||||
default="patch",
|
||||
choices=get_args(BumpVersionType),
|
||||
help=f"版本部分: {get_args(BumpVersionType)}",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--no-tag",
|
||||
action="store_true",
|
||||
help="提交后不创建 git tag",
|
||||
)
|
||||
|
||||
args = parser.parse_args()
|
||||
part = args.part
|
||||
|
||||
# 搜索文件,排除常见的虚拟环境和缓存目录
|
||||
ignore_dirs = {".venv", "venv", ".git", "__pycache__", ".tox", "node_modules", "build", "dist", ".eggs"}
|
||||
all_files = set()
|
||||
|
||||
for pattern in ["__init__.py", "pyproject.toml"]:
|
||||
for file in Path.cwd().rglob(pattern):
|
||||
# 检查路径中是否包含需要忽略的目录
|
||||
if not any(ignore_dir in file.parts for ignore_dir in ignore_dirs):
|
||||
all_files.add(file)
|
||||
|
||||
if not all_files:
|
||||
print("未找到包含版本号的文件")
|
||||
return
|
||||
|
||||
print(f"找到 {len(all_files)} 个文件需要更新版本号")
|
||||
for file in sorted(all_files):
|
||||
print(f" - {file.relative_to(Path.cwd())}")
|
||||
|
||||
# 更新所有文件的版本号(使用顺序执行避免竞争条件)
|
||||
# 使用相对于 cwd 的路径作为任务名,确保唯一性
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec(
|
||||
f"bump_{file.relative_to(Path.cwd())}".replace("\\", "_").replace("/", "_").replace(".", "_"),
|
||||
fn=bump_file_version,
|
||||
args=(file, part),
|
||||
)
|
||||
for file in all_files
|
||||
])
|
||||
report = px.run(graph, strategy="sequential")
|
||||
|
||||
# 收集新版本号(取第一个成功的结果)
|
||||
new_version = None
|
||||
for task_name in report:
|
||||
result = report[task_name]
|
||||
if result is not None:
|
||||
new_version = result
|
||||
break
|
||||
|
||||
if not new_version:
|
||||
print("未能获取新版本号")
|
||||
return
|
||||
|
||||
print(f"版本号已更新为: {new_version}")
|
||||
|
||||
# 提交修改并创建标签
|
||||
tasks = [
|
||||
px.TaskSpec("git_add", cmd=["git", "add", "."]),
|
||||
px.TaskSpec(
|
||||
"git_commit",
|
||||
cmd=["git", "commit", "-m", f"bump version to {new_version}"],
|
||||
depends_on=("git_add",),
|
||||
),
|
||||
]
|
||||
|
||||
if not args.no_tag:
|
||||
tag_name = f"v{new_version}"
|
||||
tasks.append(
|
||||
px.TaskSpec(
|
||||
"git_tag",
|
||||
cmd=["git", "tag", "-a", tag_name, "-m", f"Release {tag_name}"],
|
||||
depends_on=("git_commit",),
|
||||
)
|
||||
)
|
||||
|
||||
graph = px.Graph.from_specs(tasks)
|
||||
px.run(graph, strategy="sequential")
|
||||
|
||||
if not args.no_tag:
|
||||
print(f"已创建标签: v{new_version}")
|
||||
+142
-133
@@ -209,107 +209,112 @@ def main() -> None:
|
||||
RUST_SCCACHE_DIR.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# 使用 conditions 自动控制任务执行
|
||||
graph = px.Graph.from_specs([
|
||||
# 系统镜像配置(仅 Linux 且未配置国内镜像)
|
||||
px.TaskSpec(
|
||||
"download_mirror",
|
||||
cmd=DOWNLOAD_MIRROR_SCRIPT,
|
||||
conditions=(
|
||||
BuiltinConditions.IS_LINUX(),
|
||||
BuiltinConditions.NOT(
|
||||
BuiltinConditions.OR(
|
||||
*[
|
||||
BuiltinConditions.FILE_CONTENT_EXISTS(f, m)
|
||||
for f in [
|
||||
"/etc/apt/sources.list",
|
||||
"/etc/apt/sources.list.d/ubuntu.sources",
|
||||
]
|
||||
for m in get_args(PyMirrorType)
|
||||
],
|
||||
)
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
# 系统镜像配置(仅 Linux 且未配置国内镜像)
|
||||
px.TaskSpec(
|
||||
"download_mirror",
|
||||
cmd=DOWNLOAD_MIRROR_SCRIPT,
|
||||
conditions=(
|
||||
BuiltinConditions.IS_LINUX(),
|
||||
BuiltinConditions.NOT(
|
||||
BuiltinConditions.OR(
|
||||
*[
|
||||
BuiltinConditions.FILE_CONTENT_EXISTS(f, m)
|
||||
for f in [
|
||||
"/etc/apt/sources.list",
|
||||
"/etc/apt/sources.list.d/ubuntu.sources",
|
||||
]
|
||||
for m in get_args(PyMirrorType)
|
||||
],
|
||||
)
|
||||
),
|
||||
),
|
||||
verbose=True,
|
||||
),
|
||||
verbose=True,
|
||||
),
|
||||
px.TaskSpec(
|
||||
"install_mirror",
|
||||
cmd=INSTALL_MIRROR_SCRIPT,
|
||||
depends_on=("download_mirror",),
|
||||
verbose=True,
|
||||
),
|
||||
# 安装 Qt 依赖(仅 Linux)
|
||||
px.TaskSpec(
|
||||
"install_qt_libs",
|
||||
cmd=["sudo", "apt", "install", "-y", *QT_LIBS],
|
||||
conditions=(BuiltinConditions.IS_LINUX(),),
|
||||
depends_on=("install_mirror",),
|
||||
allow_upstream_skip=True,
|
||||
verbose=True,
|
||||
),
|
||||
# 安装中文字体(仅 Linux)
|
||||
px.TaskSpec(
|
||||
"install_fonts",
|
||||
cmd=["sudo", "apt", "install", "-y", *CHINESE_FONTS],
|
||||
conditions=(BuiltinConditions.IS_LINUX(),),
|
||||
depends_on=("install_mirror",),
|
||||
allow_upstream_skip=True,
|
||||
verbose=True,
|
||||
),
|
||||
# 安装 Docker
|
||||
px.TaskSpec(
|
||||
"install_docker",
|
||||
cmd=["sudo", "apt", "install", "-y", "docker-compose-v2"],
|
||||
conditions=(BuiltinConditions.IS_LINUX(),),
|
||||
depends_on=("install_mirror",),
|
||||
allow_upstream_skip=True,
|
||||
verbose=True,
|
||||
),
|
||||
px.TaskSpec(
|
||||
"add_docker_group",
|
||||
cmd=["sudo", "usermod", "-aG", "docker", getpass.getuser()],
|
||||
conditions=(BuiltinConditions.IS_LINUX(),),
|
||||
depends_on=("install_docker",),
|
||||
allow_upstream_skip=True,
|
||||
verbose=True,
|
||||
),
|
||||
px.TaskSpec(
|
||||
"refresh_docker_group",
|
||||
cmd=["newgrp", "docker"],
|
||||
conditions=(BuiltinConditions.IS_LINUX(),),
|
||||
depends_on=("add_docker_group",),
|
||||
allow_upstream_skip=True,
|
||||
verbose=True,
|
||||
),
|
||||
# 设置 Python 环境变量
|
||||
*setenv_group({
|
||||
"PIP_INDEX_URL": PIP_INDEX_URLS[python_mirror],
|
||||
"PIP_TRUSTED_HOSTS": PIP_TRUSTED_HOSTS[python_mirror],
|
||||
"UV_INDEX_URL": UV_INDEX_URLS[python_mirror],
|
||||
"UV_PYTHON_INSTALL_MIRROR": UV_PYTHON_INSTALL_MIRROR,
|
||||
"UV_HTTP_TIMEOUT": "600",
|
||||
"UV_LINK_MODE": "copy",
|
||||
}),
|
||||
# 写入 Python 配置(仅当未配置)
|
||||
write_file(
|
||||
str(PIP_CONFIG_PATH),
|
||||
f"[global]\nindex-url = {PIP_INDEX_URLS[python_mirror]}\ntrusted-host = {PIP_TRUSTED_HOSTS[python_mirror]}",
|
||||
),
|
||||
# 写入 Conda 配置(仅当未配置)
|
||||
write_file(
|
||||
str(CONDA_CONFIG_PATH),
|
||||
"show_channel_urls: true\nchannels:\n - " + "\n - ".join(conda_mirror_urls) + "\n - defaults",
|
||||
),
|
||||
# 设置 Rust 镜像源
|
||||
*setenv_group({
|
||||
"RUSTUP_DIST_SERVER": RUSTUP_MIRRORS[rust_mirror]["RUSTUP_DIST_SERVER"],
|
||||
"RUSTUP_UPDATE_ROOT": RUSTUP_MIRRORS[rust_mirror]["RUSTUP_UPDATE_ROOT"],
|
||||
"RUST_SCCACHE_DIR": str(RUST_SCCACHE_DIR),
|
||||
"RUST_SCCACHE_CACHE_SIZE": RUST_SCCACHE_CACHE_SIZE,
|
||||
}),
|
||||
# 写入 Rust 配置(仅当未配置)
|
||||
write_file(
|
||||
str(RUST_CONFIG_PATH),
|
||||
f"""
|
||||
px.TaskSpec(
|
||||
"install_mirror",
|
||||
cmd=INSTALL_MIRROR_SCRIPT,
|
||||
depends_on=("download_mirror",),
|
||||
verbose=True,
|
||||
),
|
||||
# 安装 Qt 依赖(仅 Linux)
|
||||
px.TaskSpec(
|
||||
"install_qt_libs",
|
||||
cmd=["sudo", "apt", "install", "-y", *QT_LIBS],
|
||||
conditions=(BuiltinConditions.IS_LINUX(),),
|
||||
depends_on=("install_mirror",),
|
||||
allow_upstream_skip=True,
|
||||
verbose=True,
|
||||
),
|
||||
# 安装中文字体(仅 Linux)
|
||||
px.TaskSpec(
|
||||
"install_fonts",
|
||||
cmd=["sudo", "apt", "install", "-y", *CHINESE_FONTS],
|
||||
conditions=(BuiltinConditions.IS_LINUX(),),
|
||||
depends_on=("install_mirror",),
|
||||
allow_upstream_skip=True,
|
||||
verbose=True,
|
||||
),
|
||||
# 安装 Docker
|
||||
px.TaskSpec(
|
||||
"install_docker",
|
||||
cmd=["sudo", "apt", "install", "-y", "docker-compose-v2"],
|
||||
conditions=(BuiltinConditions.IS_LINUX(),),
|
||||
depends_on=("install_mirror",),
|
||||
allow_upstream_skip=True,
|
||||
verbose=True,
|
||||
),
|
||||
px.TaskSpec(
|
||||
"add_docker_group",
|
||||
cmd=["sudo", "usermod", "-aG", "docker", getpass.getuser()],
|
||||
conditions=(BuiltinConditions.IS_LINUX(),),
|
||||
depends_on=("install_docker",),
|
||||
allow_upstream_skip=True,
|
||||
verbose=True,
|
||||
),
|
||||
px.TaskSpec(
|
||||
"refresh_docker_group",
|
||||
cmd=["newgrp", "docker"],
|
||||
conditions=(BuiltinConditions.IS_LINUX(),),
|
||||
depends_on=("add_docker_group",),
|
||||
allow_upstream_skip=True,
|
||||
verbose=True,
|
||||
),
|
||||
# 设置 Python 环境变量
|
||||
*setenv_group(
|
||||
{
|
||||
"PIP_INDEX_URL": PIP_INDEX_URLS[python_mirror],
|
||||
"PIP_TRUSTED_HOSTS": PIP_TRUSTED_HOSTS[python_mirror],
|
||||
"UV_INDEX_URL": UV_INDEX_URLS[python_mirror],
|
||||
"UV_PYTHON_INSTALL_MIRROR": UV_PYTHON_INSTALL_MIRROR,
|
||||
"UV_HTTP_TIMEOUT": "600",
|
||||
"UV_LINK_MODE": "copy",
|
||||
}
|
||||
),
|
||||
# 写入 Python 配置(仅当未配置)
|
||||
write_file(
|
||||
str(PIP_CONFIG_PATH),
|
||||
f"[global]\nindex-url = {PIP_INDEX_URLS[python_mirror]}\ntrusted-host = {PIP_TRUSTED_HOSTS[python_mirror]}",
|
||||
),
|
||||
# 写入 Conda 配置(仅当未配置)
|
||||
write_file(
|
||||
str(CONDA_CONFIG_PATH),
|
||||
"show_channel_urls: true\nchannels:\n - " + "\n - ".join(conda_mirror_urls) + "\n - defaults",
|
||||
),
|
||||
# 设置 Rust 镜像源
|
||||
*setenv_group(
|
||||
{
|
||||
"RUSTUP_DIST_SERVER": RUSTUP_MIRRORS[rust_mirror]["RUSTUP_DIST_SERVER"],
|
||||
"RUSTUP_UPDATE_ROOT": RUSTUP_MIRRORS[rust_mirror]["RUSTUP_UPDATE_ROOT"],
|
||||
"RUST_SCCACHE_DIR": str(RUST_SCCACHE_DIR),
|
||||
"RUST_SCCACHE_CACHE_SIZE": RUST_SCCACHE_CACHE_SIZE,
|
||||
}
|
||||
),
|
||||
# 写入 Rust 配置(仅当未配置)
|
||||
write_file(
|
||||
str(RUST_CONFIG_PATH),
|
||||
f"""
|
||||
[source.crates-io]
|
||||
replace-with = '{rust_mirror}'
|
||||
|
||||
@@ -319,39 +324,43 @@ registry = "sparse+{RUSTUP_MIRRORS[rust_mirror]["TOML_REGISTRY"]}"
|
||||
[registries.{rust_mirror}]
|
||||
index = "sparse+{RUSTUP_MIRRORS[rust_mirror]["TOML_REGISTRY"]}"
|
||||
""",
|
||||
),
|
||||
# 下载 Rustup 安装脚本
|
||||
px.TaskSpec(
|
||||
"download_rustup",
|
||||
cmd=["curl", "-fsSL", RUSTUP_DOWNLOAD_URL_LINUX, "-o", "rustup-init.sh"],
|
||||
conditions=(BuiltinConditions.IS_LINUX(), BuiltinConditions.NOT(BuiltinConditions.HAS_INSTALLED("rustup"))),
|
||||
verbose=True,
|
||||
),
|
||||
px.TaskSpec(
|
||||
"download_rustup_win",
|
||||
cmd=[
|
||||
"powershell",
|
||||
"-Command",
|
||||
"Invoke-WebRequest",
|
||||
"-Uri",
|
||||
RUSTUP_DOWNLOAD_URL_WINDOWS,
|
||||
"-OutFile",
|
||||
"rustup-init.exe",
|
||||
],
|
||||
conditions=(
|
||||
BuiltinConditions.IS_WINDOWS(),
|
||||
BuiltinConditions.NOT(BuiltinConditions.HAS_INSTALLED("rustup")),
|
||||
),
|
||||
verbose=True,
|
||||
),
|
||||
# 安装 Rust 工具链
|
||||
px.TaskSpec(
|
||||
"install_rust",
|
||||
cmd=["rustup", "toolchain", "install", rust_version],
|
||||
conditions=(BuiltinConditions.HAS_INSTALLED("rustup"),),
|
||||
depends_on=("setenv_rustup_dist_server",),
|
||||
allow_upstream_skip=True,
|
||||
verbose=True,
|
||||
),
|
||||
])
|
||||
# 下载 Rustup 安装脚本
|
||||
px.TaskSpec(
|
||||
"download_rustup",
|
||||
cmd=["curl", "-fsSL", RUSTUP_DOWNLOAD_URL_LINUX, "-o", "rustup-init.sh"],
|
||||
conditions=(
|
||||
BuiltinConditions.IS_LINUX(),
|
||||
BuiltinConditions.NOT(BuiltinConditions.HAS_INSTALLED("rustup")),
|
||||
),
|
||||
verbose=True,
|
||||
),
|
||||
px.TaskSpec(
|
||||
"download_rustup_win",
|
||||
cmd=[
|
||||
"powershell",
|
||||
"-Command",
|
||||
"Invoke-WebRequest",
|
||||
"-Uri",
|
||||
RUSTUP_DOWNLOAD_URL_WINDOWS,
|
||||
"-OutFile",
|
||||
"rustup-init.exe",
|
||||
],
|
||||
conditions=(
|
||||
BuiltinConditions.IS_WINDOWS(),
|
||||
BuiltinConditions.NOT(BuiltinConditions.HAS_INSTALLED("rustup")),
|
||||
),
|
||||
verbose=True,
|
||||
),
|
||||
# 安装 Rust 工具链
|
||||
px.TaskSpec(
|
||||
"install_rust",
|
||||
cmd=["rustup", "toolchain", "install", rust_version],
|
||||
conditions=(BuiltinConditions.HAS_INSTALLED("rustup"),),
|
||||
depends_on=("setenv_rustup_dist_server",),
|
||||
allow_upstream_skip=True,
|
||||
verbose=True,
|
||||
),
|
||||
]
|
||||
)
|
||||
px.run(graph, strategy="thread", verbose=True)
|
||||
|
||||
@@ -567,13 +567,15 @@ class EmlManagerHandler(BaseHTTPRequestHandler):
|
||||
|
||||
emails = self.db.search_emails(keyword, field, limit, offset)
|
||||
total_count = self.db.get_email_count()
|
||||
self._send_json_response({
|
||||
"emails": emails,
|
||||
"count": len(emails),
|
||||
"total": total_count,
|
||||
"limit": limit,
|
||||
"offset": offset,
|
||||
})
|
||||
self._send_json_response(
|
||||
{
|
||||
"emails": emails,
|
||||
"count": len(emails),
|
||||
"total": total_count,
|
||||
"limit": limit,
|
||||
"offset": offset,
|
||||
}
|
||||
)
|
||||
|
||||
def _api_get_email(self, query_params: dict[str, list[str]]) -> None:
|
||||
"""API: 获取单个邮件详情."""
|
||||
|
||||
@@ -1,137 +0,0 @@
|
||||
"""文件日期处理工具.
|
||||
|
||||
自动检测文件名的日期前缀,
|
||||
并根据文件的实际创建或修改时间重命名文件.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import re
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
import pyflowx as px
|
||||
|
||||
# ============================================================================
|
||||
# 配置
|
||||
# ============================================================================
|
||||
|
||||
DATE_PATTERN = re.compile(r"(20|19)\d{2}[-_#.~]?((0[1-9])|(1[012]))[-_#.~]?((0[1-9])|([12]\d)|(3[01]))[-_#.~]?")
|
||||
SEP = "_"
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# 辅助函数
|
||||
# ============================================================================
|
||||
|
||||
|
||||
def get_file_timestamp(filepath: Path) -> str:
|
||||
"""获取文件时间戳."""
|
||||
modified_time = filepath.stat().st_mtime
|
||||
created_time = filepath.stat().st_ctime
|
||||
return time.strftime("%Y%m%d", time.localtime(max((modified_time, created_time))))
|
||||
|
||||
|
||||
def remove_date_prefix(filepath: Path) -> Path:
|
||||
"""移除文件日期前缀."""
|
||||
stem = filepath.stem
|
||||
new_stem = DATE_PATTERN.sub("", stem)
|
||||
if new_stem != stem:
|
||||
new_path = filepath.with_name(new_stem + filepath.suffix)
|
||||
filepath.rename(new_path)
|
||||
return new_path
|
||||
return filepath
|
||||
|
||||
|
||||
def add_date_prefix(filepath: Path) -> Path:
|
||||
"""添加文件日期前缀."""
|
||||
timestamp = get_file_timestamp(filepath)
|
||||
stem = filepath.stem
|
||||
new_stem = f"{timestamp}{SEP}{stem}"
|
||||
new_path = filepath.with_name(new_stem + filepath.suffix)
|
||||
if new_path != filepath:
|
||||
filepath.rename(new_path)
|
||||
return new_path
|
||||
return filepath
|
||||
|
||||
|
||||
def process_file_date(filepath: Path, clear: bool = False) -> None:
|
||||
"""处理单个文件的日期前缀.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
filepath : Path
|
||||
文件路径
|
||||
clear : bool
|
||||
是否清除日期前缀
|
||||
"""
|
||||
if clear:
|
||||
remove_date_prefix(filepath)
|
||||
else:
|
||||
# 先移除旧日期前缀,再添加新日期前缀
|
||||
new_path = remove_date_prefix(filepath)
|
||||
add_date_prefix(new_path)
|
||||
|
||||
|
||||
def process_files_date(targets: list[Path], clear: bool = False) -> None:
|
||||
"""批量处理文件日期前缀.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
targets : list[Path]
|
||||
文件路径列表
|
||||
clear : bool
|
||||
是否清除日期前缀
|
||||
"""
|
||||
for target in targets:
|
||||
if target.exists() and not target.name.startswith("."):
|
||||
process_file_date(target, clear)
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# CLI Runner
|
||||
# ============================================================================
|
||||
|
||||
|
||||
def main() -> None:
|
||||
"""文件日期处理工具主函数."""
|
||||
parser = argparse.ArgumentParser(
|
||||
description="FileDate - 文件日期处理工具",
|
||||
usage="filedate <command> [options]",
|
||||
)
|
||||
subparsers = parser.add_subparsers(dest="command", help="可用命令")
|
||||
|
||||
# 添加日期前缀命令
|
||||
add_parser = subparsers.add_parser("add", help="添加日期前缀")
|
||||
add_parser.add_argument("files", nargs="+", help="文件路径")
|
||||
|
||||
# 清除日期前缀命令
|
||||
clear_parser = subparsers.add_parser("clear", help="清除日期前缀")
|
||||
clear_parser.add_argument("files", nargs="+", help="文件路径")
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
if args.command == "add":
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec(
|
||||
"process_files_date",
|
||||
fn=process_files_date,
|
||||
args=([Path(f) for f in args.files],),
|
||||
kwargs={"clear": False},
|
||||
)
|
||||
])
|
||||
elif args.command == "clear":
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec(
|
||||
"process_files_date",
|
||||
fn=process_files_date,
|
||||
args=([Path(f) for f in args.files],),
|
||||
kwargs={"clear": True},
|
||||
)
|
||||
])
|
||||
else:
|
||||
parser.print_help()
|
||||
return
|
||||
|
||||
px.run(graph, strategy="thread")
|
||||
@@ -1,140 +0,0 @@
|
||||
"""文件等级重命名工具.
|
||||
|
||||
根据文件等级配置自动重命名文件,
|
||||
支持多种等级标识和括号格式.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
from pathlib import Path
|
||||
|
||||
import pyflowx as px
|
||||
|
||||
# ============================================================================
|
||||
# 配置
|
||||
# ============================================================================
|
||||
|
||||
LEVELS: dict[str, str] = {
|
||||
"0": "",
|
||||
"1": "PUB,NOR",
|
||||
"2": "INT",
|
||||
"3": "CON",
|
||||
"4": "CLA",
|
||||
}
|
||||
|
||||
BRACKETS: tuple[str, str] = (" ([_(【-", " )]_)】")
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# 辅助函数
|
||||
# ============================================================================
|
||||
|
||||
|
||||
def remove_marks(stem: str, marks: list[str]) -> str:
|
||||
"""从文件名主干中移除所有标记."""
|
||||
left_brackets, right_brackets = BRACKETS
|
||||
for mark in marks:
|
||||
pos = 0
|
||||
while True:
|
||||
pos = stem.find(mark, pos)
|
||||
if pos == -1:
|
||||
break
|
||||
b, e = pos - 1, pos + len(mark)
|
||||
if b >= 0 and e < len(stem) and stem[b] in left_brackets and stem[e] in right_brackets:
|
||||
stem = stem[:b] + stem[e + 1 :]
|
||||
else:
|
||||
pos = e
|
||||
return stem
|
||||
|
||||
|
||||
def process_file_level(filepath: Path, level: int = 0) -> None:
|
||||
"""处理单个文件的等级标记.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
filepath : Path
|
||||
文件路径
|
||||
level : int
|
||||
文件等级 (0-4), 0 用于清除等级
|
||||
"""
|
||||
if not (0 <= level < len(LEVELS)):
|
||||
print(f"无效的等级 {level}, 必须在 0 和 {len(LEVELS) - 1} 之间")
|
||||
return
|
||||
|
||||
if not filepath.exists():
|
||||
print(f"文件不存在: {filepath}")
|
||||
return
|
||||
|
||||
filestem = filepath.stem
|
||||
original_stem = filestem
|
||||
|
||||
# 移除所有等级标记
|
||||
for level_names in LEVELS.values():
|
||||
if level_names:
|
||||
filestem = remove_marks(filestem, level_names.split(","))
|
||||
|
||||
# 移除数字标记
|
||||
for digit in map(str, range(1, 10)):
|
||||
filestem = remove_marks(filestem, [digit])
|
||||
|
||||
# 添加等级标记
|
||||
if level > 0:
|
||||
levelstr = LEVELS.get(str(level), "").split(",")[0]
|
||||
if levelstr:
|
||||
filestem = f"{filestem}({levelstr})"
|
||||
|
||||
# 重命名文件
|
||||
if filestem != original_stem:
|
||||
new_path = filepath.with_name(filestem + filepath.suffix)
|
||||
filepath.rename(new_path)
|
||||
print(f"重命名: {filepath} -> {new_path}")
|
||||
|
||||
|
||||
def process_files_level(targets: list[Path], level: int = 0) -> None:
|
||||
"""批量处理文件等级标记.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
targets : list[Path]
|
||||
文件路径列表
|
||||
level : int
|
||||
文件等级 (0-4)
|
||||
"""
|
||||
for target in targets:
|
||||
process_file_level(target, level)
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# CLI Runner
|
||||
# ============================================================================
|
||||
|
||||
|
||||
def main() -> None:
|
||||
"""文件等级重命名工具主函数."""
|
||||
parser = argparse.ArgumentParser(
|
||||
description="FileLevel - 文件等级重命名工具",
|
||||
usage="filelevel <command> [options]",
|
||||
)
|
||||
subparsers = parser.add_subparsers(dest="command", help="可用命令")
|
||||
|
||||
# 设置等级命令
|
||||
level_parser = subparsers.add_parser("set", help="设置文件等级")
|
||||
level_parser.add_argument("files", nargs="+", help="文件路径")
|
||||
level_parser.add_argument("--level", type=int, choices=[0, 1, 2, 3, 4], required=True, help="文件等级 (0-4)")
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
if args.command == "set":
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec(
|
||||
"process_files_level", fn=process_files_level, args=([Path(f) for f in args.files], args.level)
|
||||
)
|
||||
]
|
||||
)
|
||||
else:
|
||||
parser.print_help()
|
||||
return
|
||||
|
||||
px.run(graph, strategy="thread")
|
||||
@@ -1,85 +0,0 @@
|
||||
"""文件夹备份工具.
|
||||
|
||||
备份文件和文件夹为 zip 文件,
|
||||
自动删除超过最大数量的旧备份文件.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import time
|
||||
import zipfile
|
||||
from pathlib import Path
|
||||
|
||||
import pyflowx as px
|
||||
|
||||
# ============================================================================
|
||||
# 辅助函数
|
||||
# ============================================================================
|
||||
|
||||
|
||||
def remove_dump(src: Path, dst: Path, max_zip: int) -> None:
|
||||
"""递归删除旧的备份 zip 文件."""
|
||||
zip_paths = [filepath for filepath in dst.rglob("*.zip") if src.stem in str(filepath)]
|
||||
zip_files = sorted(zip_paths, key=lambda fn: str(fn)[-19:-4])
|
||||
if len(zip_files) > max_zip:
|
||||
zip_files[0].unlink()
|
||||
remove_dump(src, dst, max_zip)
|
||||
|
||||
|
||||
def zip_target(src: Path, dst: Path, max_zip: int) -> None:
|
||||
"""将单个文件或文件夹压缩为 zip 文件."""
|
||||
files = [str(_) for _ in src.rglob("*")]
|
||||
timestamp = time.strftime("_%Y%m%d_%H%M%S")
|
||||
target_path = dst / (src.stem + timestamp + ".zip")
|
||||
|
||||
with zipfile.ZipFile(target_path, "w") as zip_file:
|
||||
for file in files:
|
||||
zip_file.write(file, arcname=file.replace(str(src.parent), ""))
|
||||
|
||||
remove_dump(src, dst, max_zip)
|
||||
print(f"备份完成: {target_path}")
|
||||
|
||||
|
||||
def backup_folder(src: str, dst: str, max_zip: int = 5) -> None:
|
||||
"""备份文件夹.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
src : str
|
||||
源文件夹路径
|
||||
dst : str
|
||||
目标文件夹路径
|
||||
max_zip : int
|
||||
最大备份数量
|
||||
"""
|
||||
src_path = Path(src)
|
||||
dst_path = Path(dst)
|
||||
|
||||
if not src_path.exists():
|
||||
print(f"源文件夹不存在: {src_path}")
|
||||
return
|
||||
|
||||
if not dst_path.exists():
|
||||
dst_path.mkdir(parents=True, exist_ok=True)
|
||||
print(f"创建目标文件夹: {dst_path}")
|
||||
|
||||
zip_target(src_path, dst_path, max_zip)
|
||||
|
||||
|
||||
@px.task
|
||||
def folderback_default() -> None:
|
||||
"""备份当前目录到 ./backup."""
|
||||
backup_folder(".", "./backup", 5)
|
||||
|
||||
|
||||
def main() -> None:
|
||||
"""文件夹备份工具主函数."""
|
||||
runner = px.CliRunner(
|
||||
strategy="thread",
|
||||
description="FolderBack - 文件夹备份工具",
|
||||
aliases={
|
||||
# 备份当前目录到 ./backup
|
||||
"b": folderback_default,
|
||||
},
|
||||
)
|
||||
runner.run_cli()
|
||||
@@ -1,76 +0,0 @@
|
||||
"""文件夹压缩工具.
|
||||
|
||||
压缩目录下的所有文件/文件夹为 zip 文件,
|
||||
默认压缩当前目录下的所有子文件夹.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import shutil
|
||||
from pathlib import Path
|
||||
|
||||
import pyflowx as px
|
||||
|
||||
# ============================================================================
|
||||
# 配置
|
||||
# ============================================================================
|
||||
|
||||
IGNORE_DIRS: list[str] = [".git", ".idea", ".vscode", "__pycache__"]
|
||||
IGNORE_FILES: list[str] = [".gitignore"]
|
||||
IGNORE: list[str] = [*IGNORE_DIRS, *IGNORE_FILES]
|
||||
IGNORE_EXT: list[str] = [".zip", ".rar", ".7z", ".tar", ".gz"]
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# 辅助函数
|
||||
# ============================================================================
|
||||
|
||||
|
||||
def archive_folder(folder: Path) -> None:
|
||||
"""压缩单个文件夹."""
|
||||
shutil.make_archive(
|
||||
str(folder.with_name(folder.name)),
|
||||
format="zip",
|
||||
base_dir=folder,
|
||||
)
|
||||
print(f"压缩完成: {folder.name}.zip")
|
||||
|
||||
|
||||
def zip_folders(cwd: str = ".") -> None:
|
||||
"""压缩目录下的所有文件夹.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
cwd : str
|
||||
工作目录
|
||||
"""
|
||||
cwd_path = Path(cwd)
|
||||
if not cwd_path.exists():
|
||||
print(f"目录不存在: {cwd_path}")
|
||||
return
|
||||
|
||||
dirs: list[Path] = [
|
||||
e for e in cwd_path.iterdir() if e.is_dir() and e.name not in IGNORE_DIRS and e.suffix not in IGNORE_EXT
|
||||
]
|
||||
|
||||
for dir_path in dirs:
|
||||
archive_folder(dir_path)
|
||||
|
||||
|
||||
@px.task
|
||||
def folderzip_default() -> None:
|
||||
"""压缩当前目录下的所有文件夹."""
|
||||
zip_folders(".")
|
||||
|
||||
|
||||
def main() -> None:
|
||||
"""文件夹压缩工具主函数."""
|
||||
runner = px.CliRunner(
|
||||
strategy="thread",
|
||||
description="FolderZip - 文件夹压缩工具",
|
||||
aliases={
|
||||
# 压缩当前目录下的所有文件夹
|
||||
"z": folderzip_default,
|
||||
},
|
||||
)
|
||||
runner.run_cli()
|
||||
@@ -1,107 +0,0 @@
|
||||
"""Git 工具模块.
|
||||
|
||||
提供 Git 仓库管理的常用操作封装,
|
||||
支持初始化、提交、清理、推送等功能.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
import pyflowx as px
|
||||
|
||||
EXCLUDE_DIRS = [
|
||||
# 编辑器相关目录
|
||||
".vscode",
|
||||
".idea",
|
||||
".editorconfig",
|
||||
".trae",
|
||||
".qoder",
|
||||
# 项目相关目录
|
||||
".venv",
|
||||
".git",
|
||||
".tox",
|
||||
".pytest_cache",
|
||||
"node_modules",
|
||||
".ruff_cache",
|
||||
]
|
||||
EXCLUDE_CMDS = [arg for d in EXCLUDE_DIRS for arg in ["-e", d]]
|
||||
|
||||
|
||||
def init_sub_dirs() -> None:
|
||||
"""初始化子目录的Git仓库."""
|
||||
sub_dirs = [subdir for subdir in Path.cwd().iterdir() if subdir.is_dir()]
|
||||
for subdir in sub_dirs:
|
||||
px.run(
|
||||
px.Graph.from_specs([
|
||||
px.TaskSpec(
|
||||
"init",
|
||||
cmd=["git", "init"],
|
||||
conditions=(lambda _: not_has_git_repo(),),
|
||||
cwd=subdir,
|
||||
),
|
||||
px.TaskSpec("add", cmd=["git", "add", "."], depends_on=("init",)),
|
||||
px.TaskSpec("commit", cmd=["git", "commit", "-m", "init commit"], depends_on=("add",)),
|
||||
]),
|
||||
)
|
||||
|
||||
|
||||
@px.task(name="isub")
|
||||
def isub() -> None:
|
||||
"""初始化子目录的Git仓库."""
|
||||
init_sub_dirs()
|
||||
|
||||
|
||||
push: px.TaskSpec = px.TaskSpec("push", cmd=["git", "push"])
|
||||
pull: px.TaskSpec = px.TaskSpec("pull", cmd=["git", "pull"])
|
||||
kill_tgit: px.TaskSpec = px.TaskSpec("task_kill", cmd=["taskkill", "/f", "/t", "/im", "tgitcache.exe"])
|
||||
|
||||
|
||||
def not_has_git_repo() -> bool:
|
||||
"""检查当前目录没有Git仓库."""
|
||||
return not Path.cwd().exists() or not (Path.cwd() / ".git").is_dir()
|
||||
|
||||
|
||||
def has_files() -> bool:
|
||||
"""检查当前目录是否有文件."""
|
||||
return bool(list(Path.cwd().glob("*")))
|
||||
|
||||
|
||||
def main() -> None:
|
||||
"""Git工具主函数."""
|
||||
runner = px.CliRunner(
|
||||
strategy="thread",
|
||||
description="Gittool - Git 执行工具.",
|
||||
aliases={
|
||||
# 添加并提交
|
||||
"a": px.Graph.from_specs([
|
||||
px.TaskSpec("add", cmd=["git", "add", "."], conditions=(lambda _: has_files(),)),
|
||||
px.TaskSpec("commit", cmd=["git", "commit", "-m", "chore: update"], depends_on=("add",)),
|
||||
]),
|
||||
# 清理(chain: clean → status)
|
||||
"c": px.Graph().chain(
|
||||
px.TaskSpec("clean", cmd=["git", "clean", "-xfd", *EXCLUDE_CMDS]),
|
||||
px.TaskSpec("status", cmd=["git", "status", "--porcelain"]),
|
||||
),
|
||||
# 初始化、添加并提交
|
||||
"i": px.Graph.from_specs([
|
||||
px.TaskSpec("init", cmd=["git", "init"], conditions=(lambda _: not_has_git_repo(),)),
|
||||
px.TaskSpec("add", cmd=["git", "add", "."], depends_on=("init",), conditions=(lambda _: has_files(),)),
|
||||
px.TaskSpec(
|
||||
"commit",
|
||||
cmd=["git", "commit", "-m", "init commit"],
|
||||
depends_on=("add",),
|
||||
conditions=(lambda _: has_files(),),
|
||||
),
|
||||
]),
|
||||
# 初始化子目录
|
||||
"isub": isub,
|
||||
# 推送
|
||||
"p": push,
|
||||
# 拉取
|
||||
"pl": pull,
|
||||
# 重启TGit缓存
|
||||
"r": kill_tgit,
|
||||
},
|
||||
)
|
||||
runner.run_cli()
|
||||
@@ -22,20 +22,22 @@ def main():
|
||||
download_dir: Path = Path(args.dir) if args.dir else Path.home() / ".models" / args.name.split("/")[-1]
|
||||
download_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec(
|
||||
name="download",
|
||||
cmd=[
|
||||
"uvx",
|
||||
"modelscope",
|
||||
"download",
|
||||
f"--{args.type}",
|
||||
args.name,
|
||||
"--local_dir",
|
||||
str(download_dir),
|
||||
],
|
||||
verbose=True,
|
||||
),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec(
|
||||
name="download",
|
||||
cmd=[
|
||||
"uvx",
|
||||
"modelscope",
|
||||
"download",
|
||||
f"--{args.type}",
|
||||
args.name,
|
||||
"--local_dir",
|
||||
str(download_dir),
|
||||
],
|
||||
verbose=True,
|
||||
),
|
||||
]
|
||||
)
|
||||
|
||||
px.run(graph, strategy="thread", verbose=True)
|
||||
|
||||
@@ -24,40 +24,42 @@ def main():
|
||||
if not model_dir.exists():
|
||||
parser.error(f"Model directory {model_dir} does not exist.")
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec(
|
||||
name="download",
|
||||
cmd=[
|
||||
"uv",
|
||||
"install",
|
||||
"sglang[all]",
|
||||
],
|
||||
conditions=(BuiltinConditions.NOT(BuiltinConditions.HAS_INSTALLED("sglang")),),
|
||||
verbose=True,
|
||||
),
|
||||
px.TaskSpec(
|
||||
name="run",
|
||||
cmd=[
|
||||
"python" if Constants.IS_WINDOWS else "python3",
|
||||
"-m",
|
||||
"sglang.launch_server",
|
||||
"--model-path",
|
||||
str(model_dir),
|
||||
"--host",
|
||||
str(args.host),
|
||||
"--port",
|
||||
"8000",
|
||||
"--mem-fraction-static",
|
||||
str(args.mem),
|
||||
"--context-length",
|
||||
"32768",
|
||||
"--tool-call-parser",
|
||||
"qwen",
|
||||
"--log-level",
|
||||
str(args.log_level),
|
||||
],
|
||||
verbose=True,
|
||||
),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec(
|
||||
name="download",
|
||||
cmd=[
|
||||
"uv",
|
||||
"install",
|
||||
"sglang[all]",
|
||||
],
|
||||
conditions=(BuiltinConditions.NOT(BuiltinConditions.HAS_INSTALLED("sglang")),),
|
||||
verbose=True,
|
||||
),
|
||||
px.TaskSpec(
|
||||
name="run",
|
||||
cmd=[
|
||||
"python" if Constants.IS_WINDOWS else "python3",
|
||||
"-m",
|
||||
"sglang.launch_server",
|
||||
"--model-path",
|
||||
str(model_dir),
|
||||
"--host",
|
||||
str(args.host),
|
||||
"--port",
|
||||
"8000",
|
||||
"--mem-fraction-static",
|
||||
str(args.mem),
|
||||
"--context-length",
|
||||
"32768",
|
||||
"--tool-call-parser",
|
||||
"qwen",
|
||||
"--log-level",
|
||||
str(args.log_level),
|
||||
],
|
||||
verbose=True,
|
||||
),
|
||||
]
|
||||
)
|
||||
|
||||
px.run(graph, strategy="sequential", verbose=True)
|
||||
|
||||
@@ -1,174 +0,0 @@
|
||||
"""LS-DYNA 计算工具.
|
||||
|
||||
用于管理 LS-DYNA 仿真计算任务,
|
||||
支持启动、监控和管理计算进程.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
import pyflowx as px
|
||||
from pyflowx.conditions import Constants
|
||||
|
||||
# ============================================================================
|
||||
# 配置
|
||||
# ============================================================================
|
||||
|
||||
LS_DYNA_COMMANDS: dict[str, list[str]] = {
|
||||
"windows": ["ls-dyna_mpp", "i=input.k", "ncpu=4"],
|
||||
"linux": ["ls-dyna_mpp", "i=input.k", "ncpu=8"],
|
||||
"macos": ["ls-dyna_mpp", "i=input.k", "ncpu=4"],
|
||||
}
|
||||
|
||||
DEFAULT_INPUT_FILE: str = "input.k"
|
||||
DEFAULT_NCPU: int = 4
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# 辅助函数
|
||||
# ============================================================================
|
||||
|
||||
|
||||
def get_ls_dyna_command(input_file: str, ncpu: int) -> list[str]:
|
||||
"""获取 LS-DYNA 命令.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
input_file : str
|
||||
输入文件路径
|
||||
ncpu : int
|
||||
CPU 核心数
|
||||
|
||||
Returns
|
||||
-------
|
||||
list[str]
|
||||
LS-DYNA 命令列表
|
||||
"""
|
||||
if Constants.IS_WINDOWS or Constants.IS_MACOS:
|
||||
return ["ls-dyna_mpp", f"i={input_file}", f"ncpu={ncpu}"]
|
||||
else:
|
||||
return ["ls-dyna_mpp", f"i={input_file}", f"ncpu={ncpu}"]
|
||||
|
||||
|
||||
def run_ls_dyna(input_file: str, ncpu: int = DEFAULT_NCPU) -> None:
|
||||
"""运行 LS-DYNA 计算.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
input_file : str
|
||||
输入文件路径
|
||||
ncpu : int
|
||||
CPU 核心数
|
||||
"""
|
||||
input_path = Path(input_file)
|
||||
if not input_path.exists():
|
||||
print(f"输入文件不存在: {input_path}")
|
||||
return
|
||||
|
||||
cmd = get_ls_dyna_command(input_file, ncpu)
|
||||
try:
|
||||
subprocess.run(cmd, check=True)
|
||||
print(f"LS-DYNA 计算完成: {input_file}")
|
||||
except FileNotFoundError:
|
||||
print("未找到 ls-dyna_mpp 命令")
|
||||
except subprocess.CalledProcessError as e:
|
||||
print(f"LS-DYNA 计算失败: {e}")
|
||||
|
||||
|
||||
def run_ls_dyna_mpi(input_file: str, ncpu: int = DEFAULT_NCPU) -> None:
|
||||
"""运行 LS-DYNA MPI 计算.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
input_file : str
|
||||
输入文件路径
|
||||
ncpu : int
|
||||
CPU 核心数
|
||||
"""
|
||||
input_path = Path(input_file)
|
||||
if not input_path.exists():
|
||||
print(f"输入文件不存在: {input_path}")
|
||||
return
|
||||
|
||||
cmd = ["mpirun", "-np", str(ncpu), "ls-dyna_mpp", f"i={input_file}"]
|
||||
try:
|
||||
subprocess.run(cmd, check=True)
|
||||
print(f"LS-DYNA MPI 计算完成: {input_file}")
|
||||
except FileNotFoundError:
|
||||
print("未找到 mpirun 或 ls-dyna_mpp 命令")
|
||||
except subprocess.CalledProcessError as e:
|
||||
print(f"LS-DYNA MPI 计算失败: {e}")
|
||||
|
||||
|
||||
def check_ls_dyna_status() -> None:
|
||||
"""检查 LS-DYNA 进程状态."""
|
||||
try:
|
||||
if Constants.IS_WINDOWS:
|
||||
result = subprocess.run(
|
||||
["tasklist", "/fi", "imagename eq ls-dyna_mpp.exe"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=True,
|
||||
)
|
||||
print(result.stdout)
|
||||
else:
|
||||
result = subprocess.run(
|
||||
["pgrep", "-f", "ls-dyna"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
if result.stdout.strip():
|
||||
print(f"运行中的 LS-DYNA 进程 PID: {result.stdout.strip()}")
|
||||
else:
|
||||
print("没有运行中的 LS-DYNA 进程")
|
||||
except subprocess.CalledProcessError as e:
|
||||
print(f"检查进程状态失败: {e}")
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# CLI Runner
|
||||
# ============================================================================
|
||||
|
||||
|
||||
def main() -> None:
|
||||
"""LS-DYNA 计算工具主函数."""
|
||||
parser = argparse.ArgumentParser(
|
||||
description="LSCalc - LS-DYNA 计算工具",
|
||||
usage="lscalc <command> [options]",
|
||||
)
|
||||
subparsers = parser.add_subparsers(dest="command", help="可用命令")
|
||||
|
||||
# 运行计算命令
|
||||
run_parser = subparsers.add_parser("run", help="运行 LS-DYNA 计算")
|
||||
run_parser.add_argument("input_file", help="输入文件路径")
|
||||
run_parser.add_argument("--ncpu", type=int, default=DEFAULT_NCPU, help="CPU 核心数")
|
||||
|
||||
# 运行 MPI 计算命令
|
||||
mpi_parser = subparsers.add_parser("mpi", help="运行 LS-DYNA MPI 计算")
|
||||
mpi_parser.add_argument("input_file", help="输入文件路径")
|
||||
mpi_parser.add_argument("--ncpu", type=int, default=DEFAULT_NCPU, help="CPU 核心数")
|
||||
|
||||
# 检查进程状态命令
|
||||
subparsers.add_parser("status", help="检查 LS-DYNA 进程状态")
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
if args.command == "run":
|
||||
graph = px.Graph.from_specs(
|
||||
[px.TaskSpec("run_ls_dyna", fn=run_ls_dyna, args=(args.input_file,), kwargs={"ncpu": args.ncpu})]
|
||||
)
|
||||
elif args.command == "mpi":
|
||||
graph = px.Graph.from_specs(
|
||||
[px.TaskSpec("run_ls_dyna_mpi", fn=run_ls_dyna_mpi, args=(args.input_file,), kwargs={"ncpu": args.ncpu})]
|
||||
)
|
||||
elif args.command == "status":
|
||||
graph = px.Graph.from_specs([px.TaskSpec("check_ls_dyna_status", fn=check_ls_dyna_status)])
|
||||
else:
|
||||
parser.print_help()
|
||||
return
|
||||
|
||||
px.run(graph, strategy="thread")
|
||||
@@ -1,349 +0,0 @@
|
||||
"""Python 打包工具模块.
|
||||
|
||||
提供 Python 项目打包的常用功能封装,
|
||||
支持源码打包、依赖打包、嵌入式 Python 安装等功能.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import shutil
|
||||
import subprocess
|
||||
import zipfile
|
||||
from pathlib import Path
|
||||
|
||||
import pyflowx as px
|
||||
|
||||
# ============================================================================
|
||||
# 配置
|
||||
# ============================================================================
|
||||
|
||||
DEFAULT_BUILD_DIR = ".pypack"
|
||||
DEFAULT_DIST_DIR = "dist"
|
||||
DEFAULT_LIB_DIR = "libs"
|
||||
DEFAULT_CACHE_DIR = ".cache/pypack"
|
||||
|
||||
IGNORE_PATTERNS = [
|
||||
"__pycache__",
|
||||
"*.pyc",
|
||||
"*.pyo",
|
||||
".git",
|
||||
".venv",
|
||||
".idea",
|
||||
".vscode",
|
||||
"*.egg-info",
|
||||
"dist",
|
||||
"build",
|
||||
".pytest_cache",
|
||||
".tox",
|
||||
".mypy_cache",
|
||||
]
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# 辅助函数
|
||||
# ============================================================================
|
||||
|
||||
|
||||
def pack_source(project_dir: Path, output_dir: Path) -> None:
|
||||
"""打包项目源码.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
project_dir : Path
|
||||
项目目录
|
||||
output_dir : Path
|
||||
输出目录
|
||||
"""
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# 检测项目名称
|
||||
pyproject_file = project_dir / "pyproject.toml"
|
||||
project_name = project_dir.name
|
||||
|
||||
if pyproject_file.exists():
|
||||
try:
|
||||
import tomllib
|
||||
|
||||
content = pyproject_file.read_text(encoding="utf-8")
|
||||
data = tomllib.loads(content)
|
||||
project_name = data.get("project", {}).get("name", project_name)
|
||||
except ImportError:
|
||||
pass
|
||||
|
||||
# 打包源码
|
||||
source_dir = output_dir / "src" / project_name
|
||||
source_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# 复制文件
|
||||
src_subdir = project_dir / "src"
|
||||
if src_subdir.exists():
|
||||
shutil.copytree(
|
||||
src_subdir,
|
||||
source_dir / "src",
|
||||
ignore=shutil.ignore_patterns(*IGNORE_PATTERNS),
|
||||
dirs_exist_ok=True,
|
||||
)
|
||||
else:
|
||||
for item in project_dir.iterdir():
|
||||
if item.name in IGNORE_PATTERNS or item.name.startswith("."):
|
||||
continue
|
||||
dst_item = source_dir / item.name
|
||||
if item.is_dir():
|
||||
shutil.copytree(
|
||||
item,
|
||||
dst_item,
|
||||
ignore=shutil.ignore_patterns(*IGNORE_PATTERNS),
|
||||
dirs_exist_ok=True,
|
||||
)
|
||||
else:
|
||||
shutil.copy2(item, dst_item)
|
||||
|
||||
print(f"源码打包完成: {source_dir}")
|
||||
|
||||
|
||||
def pack_dependencies(lib_dir: Path, dependencies: list[str]) -> None:
|
||||
"""打包项目依赖.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
lib_dir : Path
|
||||
依赖库目录
|
||||
dependencies : list[str]
|
||||
依赖列表
|
||||
"""
|
||||
lib_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
if not dependencies:
|
||||
print("没有依赖需要打包")
|
||||
return
|
||||
|
||||
# 使用 pip 安装依赖到目标目录
|
||||
cmd = [
|
||||
"pip",
|
||||
"install",
|
||||
"--target",
|
||||
str(lib_dir),
|
||||
"--no-compile",
|
||||
"--no-warn-script-location",
|
||||
]
|
||||
cmd.extend(dependencies)
|
||||
|
||||
subprocess.run(cmd, check=True)
|
||||
print(f"依赖打包完成: {lib_dir}")
|
||||
|
||||
|
||||
def pack_wheel(project_dir: Path, output_dir: Path) -> None:
|
||||
"""打包项目为 wheel 文件.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
project_dir : Path
|
||||
项目目录
|
||||
output_dir : Path
|
||||
输出目录
|
||||
"""
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# 使用 pip wheel 打包
|
||||
cmd = [
|
||||
"pip",
|
||||
"wheel",
|
||||
"--no-deps",
|
||||
"--wheel-dir",
|
||||
str(output_dir),
|
||||
str(project_dir),
|
||||
]
|
||||
|
||||
subprocess.run(cmd, check=True)
|
||||
print(f"Wheel 打包完成: {output_dir}")
|
||||
|
||||
|
||||
def install_embed_python(version: str, output_dir: Path) -> None:
|
||||
"""安装嵌入式 Python.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
version : str
|
||||
Python 版本 (如: 3.10, 3.11)
|
||||
output_dir : Path
|
||||
输出目录
|
||||
"""
|
||||
import platform
|
||||
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# 构建下载 URL
|
||||
arch = platform.machine().lower()
|
||||
if arch in ["x86_64", "amd64"]:
|
||||
arch = "amd64"
|
||||
elif arch in ["arm64", "aarch64"]:
|
||||
arch = "arm64"
|
||||
|
||||
# 解析完整版本号
|
||||
version_map = {
|
||||
"3.8": "3.8.10",
|
||||
"3.9": "3.9.13",
|
||||
"3.10": "3.10.11",
|
||||
"3.11": "3.11.9",
|
||||
"3.12": "3.12.4",
|
||||
}
|
||||
full_version = version_map.get(version, f"{version}.0")
|
||||
|
||||
# Windows 嵌入式 Python 下载 URL
|
||||
url = f"https://www.python.org/ftp/python/{full_version}/python-{full_version}-embed-{arch}.zip"
|
||||
|
||||
# 下载并解压
|
||||
cache_file = Path(DEFAULT_CACHE_DIR) / f"python-{full_version}-embed-{arch}.zip"
|
||||
cache_file.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
if not cache_file.exists():
|
||||
print(f"正在下载嵌入式 Python {full_version}...")
|
||||
import urllib.request
|
||||
|
||||
urllib.request.urlretrieve(url, cache_file)
|
||||
print(f"下载完成: {cache_file}")
|
||||
|
||||
# 解压
|
||||
with zipfile.ZipFile(cache_file, "r") as zf:
|
||||
zf.extractall(output_dir)
|
||||
|
||||
print(f"嵌入式 Python 安装完成: {output_dir}")
|
||||
|
||||
|
||||
def create_zip_package(source_dir: Path, output_file: Path) -> None:
|
||||
"""创建 ZIP 打包文件.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
source_dir : Path
|
||||
源目录
|
||||
output_file : Path
|
||||
输出文件
|
||||
"""
|
||||
output_file.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
with zipfile.ZipFile(output_file, "w", zipfile.ZIP_DEFLATED) as zf:
|
||||
for file in source_dir.rglob("*"):
|
||||
if file.is_file():
|
||||
arcname = file.relative_to(source_dir)
|
||||
zf.write(file, arcname)
|
||||
|
||||
print(f"ZIP 打包完成: {output_file}")
|
||||
|
||||
|
||||
def clean_build_dir(build_dir: Path) -> None:
|
||||
"""清理构建目录.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
build_dir : Path
|
||||
构建目录
|
||||
"""
|
||||
if build_dir.exists():
|
||||
shutil.rmtree(build_dir)
|
||||
print(f"清理完成: {build_dir}")
|
||||
else:
|
||||
print(f"目录不存在: {build_dir}")
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# CLI Runner
|
||||
# ============================================================================
|
||||
|
||||
|
||||
def main() -> None:
|
||||
"""Python 打包工具主函数."""
|
||||
parser = argparse.ArgumentParser(
|
||||
description="PackTool - Python 打包工具",
|
||||
usage="packtool <command> [options]",
|
||||
)
|
||||
subparsers = parser.add_subparsers(dest="command", help="可用命令")
|
||||
|
||||
# 源码打包命令
|
||||
src_parser = subparsers.add_parser("src", help="打包项目源码")
|
||||
src_parser.add_argument("--project-dir", type=str, default=".", help="项目目录")
|
||||
src_parser.add_argument("--output-dir", type=str, default=DEFAULT_BUILD_DIR, help="输出目录")
|
||||
|
||||
# 依赖打包命令
|
||||
deps_parser = subparsers.add_parser("deps", help="打包项目依赖")
|
||||
deps_parser.add_argument("--lib-dir", type=str, default=DEFAULT_LIB_DIR, help="依赖库目录")
|
||||
deps_parser.add_argument("dependencies", nargs="*", help="依赖列表")
|
||||
|
||||
# Wheel 打包命令
|
||||
wheel_parser = subparsers.add_parser("wheel", help="打包项目为 wheel 文件")
|
||||
wheel_parser.add_argument("--project-dir", type=str, default=".", help="项目目录")
|
||||
wheel_parser.add_argument("--output-dir", type=str, default=DEFAULT_DIST_DIR, help="输出目录")
|
||||
|
||||
# 嵌入式 Python 安装命令
|
||||
embed_parser = subparsers.add_parser("embed", help="安装嵌入式 Python")
|
||||
embed_parser.add_argument("--version", type=str, default="3.10", help="Python 版本")
|
||||
embed_parser.add_argument("--output-dir", type=str, default="python", help="输出目录")
|
||||
|
||||
# ZIP 打包命令
|
||||
zip_parser = subparsers.add_parser("zip", help="创建 ZIP 打包文件")
|
||||
zip_parser.add_argument("--source-dir", type=str, default=".", help="源目录")
|
||||
zip_parser.add_argument("--output-file", type=str, default="package.zip", help="输出文件")
|
||||
|
||||
# 清理命令
|
||||
subparsers.add_parser("clean", help="清理构建目录")
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
if args.command == "src":
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec(
|
||||
"pack_source",
|
||||
fn=pack_source,
|
||||
args=(Path(args.project_dir), Path(args.output_dir)),
|
||||
)
|
||||
]
|
||||
)
|
||||
elif args.command == "deps":
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec(
|
||||
"pack_deps",
|
||||
fn=pack_dependencies,
|
||||
args=(Path(args.lib_dir), args.dependencies),
|
||||
)
|
||||
]
|
||||
)
|
||||
elif args.command == "wheel":
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec(
|
||||
"pack_wheel",
|
||||
fn=pack_wheel,
|
||||
args=(Path(args.project_dir), Path(args.output_dir)),
|
||||
)
|
||||
]
|
||||
)
|
||||
elif args.command == "embed":
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec(
|
||||
"install_embed",
|
||||
fn=install_embed_python,
|
||||
args=(args.version, Path(args.output_dir)),
|
||||
)
|
||||
]
|
||||
)
|
||||
elif args.command == "zip":
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec(
|
||||
"create_zip",
|
||||
fn=create_zip_package,
|
||||
args=(Path(args.source_dir), Path(args.output_file)),
|
||||
)
|
||||
]
|
||||
)
|
||||
elif args.command == "clean":
|
||||
graph = px.Graph.from_specs([px.TaskSpec("clean_build", fn=clean_build_dir, args=(Path(DEFAULT_BUILD_DIR),))])
|
||||
else:
|
||||
parser.print_help()
|
||||
return
|
||||
|
||||
px.run(graph, strategy="thread")
|
||||
@@ -0,0 +1,186 @@
|
||||
"""PyFlowX 统一 CLI 入口.
|
||||
|
||||
通过 ``pf <tool> [command] [options]`` 调用所有工具,
|
||||
工具定义在 ``configs/`` 目录下的 YAML 文件中.
|
||||
|
||||
用法
|
||||
----
|
||||
pf # 列出所有可用工具
|
||||
pf filedate # 查看 filedate 工具帮助
|
||||
pf filedate add a.txt # 调用 filedate 的 add 子命令
|
||||
pf pymake b # 调用 pymake 的 b 别名
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import importlib
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from typing import Sequence
|
||||
|
||||
import pyflowx as px
|
||||
|
||||
|
||||
class PfApp:
|
||||
"""pf 统一入口应用.
|
||||
|
||||
路由 ``pf <tool> [command]`` 到 YAML 配置工具或传统 Python 工具.
|
||||
"""
|
||||
|
||||
_CONFIGS_DIR = Path(__file__).parent.parent / "configs"
|
||||
|
||||
# 工具名到 YAML 配置文件的映射 (支持短别名)
|
||||
_TOOL_ALIASES: dict[str, str] = {
|
||||
"autofmt": "autofmt",
|
||||
"af": "autofmt",
|
||||
"bump": "bumpversion",
|
||||
"bumpversion": "bumpversion",
|
||||
"bv": "bumpversion",
|
||||
"filedate": "filedate",
|
||||
"fd": "filedate",
|
||||
"filelevel": "filelevel",
|
||||
"fl": "filelevel",
|
||||
"folderback": "folderback",
|
||||
"foldback": "folderback",
|
||||
"fb": "folderback",
|
||||
"folderzip": "folderzip",
|
||||
"foldzip": "folderzip",
|
||||
"fz": "folderzip",
|
||||
"git": "gittool",
|
||||
"gitt": "gittool",
|
||||
"gittool": "gittool",
|
||||
"gt": "gittool",
|
||||
"ls": "lscalc",
|
||||
"lscalc": "lscalc",
|
||||
"pack": "packtool",
|
||||
"packtool": "packtool",
|
||||
"pk": "packtool",
|
||||
"pdf": "pdftool",
|
||||
"pdftool": "pdftool",
|
||||
"pt": "pdftool",
|
||||
"pip": "piptool",
|
||||
"pymake": "pymake",
|
||||
"piptool": "piptool",
|
||||
"pp": "piptool",
|
||||
"reseticon": "reseticoncache",
|
||||
"reseticoncache": "reseticoncache",
|
||||
"ric": "reseticoncache",
|
||||
"screenshot": "screenshot",
|
||||
"scrcap": "screenshot",
|
||||
"ss": "screenshot",
|
||||
"ssh": "sshcopyid",
|
||||
"sshcopy": "sshcopyid",
|
||||
"sshcopyid": "sshcopyid",
|
||||
"sc": "sshcopyid",
|
||||
}
|
||||
|
||||
# 传统工具: 有自己的 main() 函数 (无法 YAML 化的复杂逻辑)
|
||||
_LEGACY_TOOLS: dict[str, str] = {
|
||||
"emlman": "pyflowx.cli.emlmanager:main",
|
||||
"profiler": "pyflowx.cli.profiler:main",
|
||||
"pxp": "pyflowx.cli.profiler:main",
|
||||
"yamlrun": "pyflowx.cli.yamlrun:main",
|
||||
}
|
||||
|
||||
def __init__(self, argv: Sequence[str] | None = None) -> None:
|
||||
self._argv = list(argv) if argv is not None else sys.argv[1:]
|
||||
|
||||
def run(self) -> int:
|
||||
"""主入口, 返回退出码."""
|
||||
if not self._argv:
|
||||
self._list_tools()
|
||||
return 0
|
||||
|
||||
tool_name = self._argv[0]
|
||||
rest_argv = self._argv[1:]
|
||||
|
||||
resolved = self._resolve_tool(tool_name)
|
||||
if resolved is None:
|
||||
print(f"错误: 未知工具 '{tool_name}'", file=sys.stderr)
|
||||
print("运行 'pf' 查看可用工具列表", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
tool_type, target = resolved
|
||||
if tool_type == "legacy":
|
||||
return self._run_legacy(target, rest_argv)
|
||||
return self._run_yaml(target, rest_argv)
|
||||
|
||||
def _list_tools(self) -> None:
|
||||
"""列出所有可用工具."""
|
||||
print("PyFlowX 工具列表:")
|
||||
print()
|
||||
print("YAML 配置工具:")
|
||||
yaml_tools = sorted(set(self._TOOL_ALIASES.values()))
|
||||
for tool in yaml_tools:
|
||||
print(f" pf {tool:<15} - {self._tool_description(tool)}")
|
||||
print()
|
||||
print("传统工具:")
|
||||
for tool in sorted(self._LEGACY_TOOLS):
|
||||
print(f" pf {tool:<15}")
|
||||
print()
|
||||
print("示例:")
|
||||
print(" pf filedate add a.txt")
|
||||
print(" pf pymake b")
|
||||
|
||||
def _tool_description(self, tool_name: str) -> str:
|
||||
"""获取工具描述 (从 YAML cli.description)."""
|
||||
config_path = self._CONFIGS_DIR / f"{tool_name}.yaml"
|
||||
if not config_path.exists():
|
||||
return ""
|
||||
try:
|
||||
import yaml
|
||||
|
||||
data = yaml.safe_load(config_path.read_text(encoding="utf-8"))
|
||||
if isinstance(data, dict) and isinstance(data.get("cli"), dict):
|
||||
return str(data["cli"].get("description", ""))
|
||||
except Exception:
|
||||
pass
|
||||
return ""
|
||||
|
||||
def _resolve_tool(self, name: str) -> tuple[str, str] | None:
|
||||
"""解析工具名, 返回 (类型, 目标).
|
||||
|
||||
类型: "yaml" 或 "legacy"
|
||||
目标: YAML 文件名 (不含 .yaml) 或 legacy 模块路径
|
||||
"""
|
||||
if name in self._TOOL_ALIASES:
|
||||
return ("yaml", self._TOOL_ALIASES[name])
|
||||
if name in self._LEGACY_TOOLS:
|
||||
return ("legacy", self._LEGACY_TOOLS[name])
|
||||
return None
|
||||
|
||||
def _run_legacy(self, module_path: str, argv: list[str]) -> int:
|
||||
"""运行传统工具的 main() 函数."""
|
||||
module_name, func_name = module_path.split(":", 1)
|
||||
module = importlib.import_module(module_name)
|
||||
func = getattr(module, func_name)
|
||||
|
||||
original_argv = sys.argv
|
||||
sys.argv = [f"pf {module_name.split('.')[-1]}", *argv]
|
||||
try:
|
||||
func()
|
||||
return 0
|
||||
except SystemExit as e:
|
||||
return int(e.code) if e.code is not None else 0
|
||||
finally:
|
||||
sys.argv = original_argv
|
||||
|
||||
def _run_yaml(self, target: str, argv: list[str]) -> int:
|
||||
"""运行 YAML 配置工具."""
|
||||
config_path = self._CONFIGS_DIR / f"{target}.yaml"
|
||||
if not config_path.exists():
|
||||
print(f"错误: 未找到配置文件 '{config_path}'", file=sys.stderr)
|
||||
print("运行 'pf' 查看可用工具列表", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
print(f"运行配置文件 '{config_path}'")
|
||||
return px.run_cli(config_path, argv)
|
||||
|
||||
|
||||
def main() -> None:
|
||||
"""pf 统一入口主函数."""
|
||||
sys.exit(PfApp().run())
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,195 +0,0 @@
|
||||
"""pip 包管理工具模块.
|
||||
|
||||
提供 pip 包管理操作的封装,
|
||||
支持安装、卸载、下载等功能.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import fnmatch
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
import pyflowx as px
|
||||
|
||||
# ============================================================================
|
||||
# 配置
|
||||
# ============================================================================
|
||||
|
||||
PACKAGE_DIR = "packages"
|
||||
REQUIREMENTS_FILE = "requirements.txt"
|
||||
|
||||
# 受保护的包名集合
|
||||
_PROTECTED_PACKAGES: frozenset[str] = frozenset({
|
||||
"pyflowx",
|
||||
"bitool",
|
||||
})
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# 辅助函数
|
||||
# ============================================================================
|
||||
|
||||
|
||||
def _get_installed_packages() -> list[str]:
|
||||
"""获取当前环境中所有已安装的包名."""
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["pip", "list", "--format=freeze"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=True,
|
||||
)
|
||||
packages: list[str] = []
|
||||
for line in result.stdout.strip().split("\n"):
|
||||
if line and "==" in line:
|
||||
pkg_name = line.split("==")[0].strip()
|
||||
packages.append(pkg_name)
|
||||
except (subprocess.SubprocessError, OSError):
|
||||
return []
|
||||
return packages
|
||||
|
||||
|
||||
def _expand_wildcard_packages(pattern: str) -> list[str]:
|
||||
"""展开通配符模式为实际的包名列表."""
|
||||
if not any(char in pattern for char in ["*", "?", "[", "]"]):
|
||||
return [pattern]
|
||||
|
||||
installed_packages = _get_installed_packages()
|
||||
matched = [pkg for pkg in installed_packages if fnmatch.fnmatchcase(pkg.lower(), pattern.lower())]
|
||||
return matched
|
||||
|
||||
|
||||
def _filter_protected_packages(packages: list[str]) -> list[str]:
|
||||
"""过滤掉受保护的包名."""
|
||||
safe = [p for p in packages if p.lower() not in {p.lower() for p in _PROTECTED_PACKAGES}]
|
||||
filtered = [p for p in packages if p.lower() in {p.lower() for p in _PROTECTED_PACKAGES}]
|
||||
if filtered:
|
||||
print(f"跳过受保护的包: {', '.join(filtered)}")
|
||||
return safe
|
||||
|
||||
|
||||
def pip_uninstall(pkg_names: list[str]) -> None:
|
||||
"""卸载包."""
|
||||
packages_to_uninstall: list[str] = []
|
||||
for pattern in pkg_names:
|
||||
packages_to_uninstall.extend(_expand_wildcard_packages(pattern))
|
||||
|
||||
packages_to_uninstall = _filter_protected_packages(packages_to_uninstall)
|
||||
|
||||
if not packages_to_uninstall:
|
||||
return
|
||||
|
||||
subprocess.run(["pip", "uninstall", "-y", *packages_to_uninstall], check=True)
|
||||
|
||||
|
||||
def pip_reinstall(pkg_names: list[str], offline: bool = False) -> None:
|
||||
"""重新安装包."""
|
||||
safe_pkgs = _filter_protected_packages(pkg_names)
|
||||
if not safe_pkgs:
|
||||
print("所有指定的包均为受保护包, 跳过重装")
|
||||
return
|
||||
|
||||
subprocess.run(["pip", "uninstall", "-y", *safe_pkgs], check=True)
|
||||
|
||||
options = ["--no-index", "--find-links", "."] if offline else []
|
||||
subprocess.run(["pip", "install", *options, *safe_pkgs], check=True)
|
||||
|
||||
|
||||
def pip_download(pkg_names: list[str], offline: bool = False) -> None:
|
||||
"""下载包."""
|
||||
options = ["--no-index", "--find-links", "."] if offline else []
|
||||
subprocess.run(
|
||||
["pip", "download", *pkg_names, *options, "-d", PACKAGE_DIR],
|
||||
check=True,
|
||||
)
|
||||
|
||||
|
||||
def pip_freeze() -> None:
|
||||
"""冻结依赖."""
|
||||
result = subprocess.run(
|
||||
["pip", "freeze", "--exclude-editable"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=True,
|
||||
)
|
||||
Path(REQUIREMENTS_FILE).write_text(result.stdout)
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# CLI Runner
|
||||
# ============================================================================
|
||||
|
||||
|
||||
def main() -> None:
|
||||
"""pip 工具主函数."""
|
||||
parser = argparse.ArgumentParser(
|
||||
description="PipTool - pip 包管理工具",
|
||||
usage="piptool <command> [options]",
|
||||
)
|
||||
subparsers = parser.add_subparsers(dest="command", help="可用命令")
|
||||
|
||||
# 安装命令
|
||||
install_parser = subparsers.add_parser("i", help="安装包")
|
||||
install_parser.add_argument("packages", nargs="+", help="要安装的包名")
|
||||
|
||||
# 卸载命令
|
||||
uninstall_parser = subparsers.add_parser("u", help="卸载包")
|
||||
uninstall_parser.add_argument("packages", nargs="+", help="要卸载的包名 (支持通配符)")
|
||||
|
||||
# 重装命令
|
||||
reinstall_parser = subparsers.add_parser("r", help="重新安装包")
|
||||
reinstall_parser.add_argument("packages", nargs="+", help="要重装的包名")
|
||||
reinstall_parser.add_argument("--offline", action="store_true", help="使用离线模式")
|
||||
|
||||
# 下载命令
|
||||
download_parser = subparsers.add_parser("d", help="下载包")
|
||||
download_parser.add_argument("packages", nargs="+", help="要下载的包名")
|
||||
download_parser.add_argument("--offline", action="store_true", help="使用离线模式")
|
||||
|
||||
# 升级 pip 命令
|
||||
subparsers.add_parser("up", help="升级 pip")
|
||||
|
||||
# 冻结依赖命令
|
||||
subparsers.add_parser("f", help="冻结依赖到 requirements.txt")
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
if args.command == "i":
|
||||
graph = px.Graph.from_specs([px.TaskSpec("pip_install", cmd=["pip", "install", *args.packages], verbose=True)])
|
||||
elif args.command == "u":
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("pip_uninstall", fn=pip_uninstall, args=(args.packages,), verbose=True)
|
||||
])
|
||||
elif args.command == "r":
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec(
|
||||
"pip_reinstall",
|
||||
fn=pip_reinstall,
|
||||
args=(args.packages,),
|
||||
kwargs={"offline": args.offline},
|
||||
verbose=True,
|
||||
)
|
||||
])
|
||||
elif args.command == "d":
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec(
|
||||
"pip_download",
|
||||
fn=pip_download,
|
||||
args=(args.packages,),
|
||||
kwargs={"offline": args.offline},
|
||||
verbose=True,
|
||||
)
|
||||
])
|
||||
elif args.command == "up":
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("pip_upgrade", cmd=["python", "-m", "pip", "install", "--upgrade", "pip"], verbose=True)
|
||||
])
|
||||
elif args.command == "f":
|
||||
graph = px.Graph.from_specs([px.TaskSpec("pip_freeze", fn=pip_freeze, verbose=True)])
|
||||
else:
|
||||
parser.print_help()
|
||||
return
|
||||
|
||||
px.run(graph, strategy="thread")
|
||||
@@ -1,125 +0,0 @@
|
||||
"""Python 构建工具模块.
|
||||
|
||||
完全替代传统的 Makefile,
|
||||
提供更好的跨平台兼容性和 Python 生态集成.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
import pyflowx as px
|
||||
from pyflowx.conditions import Constants
|
||||
|
||||
# 项目根目录(pymake.py 在 src/pyflowx/cli,向上四层到达根目录)
|
||||
ROOT_DIR = Path(__file__).parent.parent.parent.parent
|
||||
|
||||
MATURIN_BUILD_COMMAND = ["maturin", "build", "-r"]
|
||||
if Constants.IS_WINDOWS:
|
||||
MATURIN_BUILD_COMMAND.extend(["--target", "x86_64-win7-windows-msvc", "-Zbuild-std", "-i", "python3.8"])
|
||||
|
||||
# 扁平注册所有任务(px.cmd 自动从命令前两段推导 name)
|
||||
# 所有任务指定 cwd=ROOT_DIR,确保在项目根目录执行
|
||||
tasks: list[px.TaskSpec] = [
|
||||
px.cmd(["uv", "build"], cwd=ROOT_DIR),
|
||||
px.cmd(MATURIN_BUILD_COMMAND, cwd=ROOT_DIR),
|
||||
px.cmd(["uv", "sync"], cwd=ROOT_DIR),
|
||||
px.cmd(["gitt", "c"], name="git_clean", cwd=ROOT_DIR),
|
||||
px.cmd(
|
||||
["pytest", "-m", "not slow", "-n", "8", "--dist", "loadfile", "--color=yes", "--durations=10"],
|
||||
name="test",
|
||||
cwd=ROOT_DIR,
|
||||
),
|
||||
px.cmd(
|
||||
["pytest", "-m", "not slow", "--dist", "loadfile", "--color=yes", "--durations=10"],
|
||||
name="test_fast",
|
||||
cwd=ROOT_DIR,
|
||||
),
|
||||
px.cmd(
|
||||
["pytest", "--cov", "-n", "8", "--dist", "loadfile", "--tb=short", "-v", "--color=yes", "--durations=10"],
|
||||
name="test_coverage",
|
||||
cwd=ROOT_DIR,
|
||||
),
|
||||
px.cmd(["pyrefly", "check", "."], cwd=ROOT_DIR),
|
||||
px.cmd(["git", "add", "-A"], name="git_add_all", cwd=ROOT_DIR),
|
||||
px.cmd(["bumpversion"], cwd=ROOT_DIR),
|
||||
px.cmd(["bumpversion", "minor"], cwd=ROOT_DIR),
|
||||
px.cmd(["git", "push"], cwd=ROOT_DIR),
|
||||
px.cmd(["git", "push", "--tags"], name="git_push_tags", cwd=ROOT_DIR),
|
||||
px.cmd(["hatch", "publish"], name="publish_python", cwd=ROOT_DIR),
|
||||
px.cmd(["twine", "upload", "--disable-progress-bar"], name="twine_publish", cwd=ROOT_DIR),
|
||||
]
|
||||
|
||||
# 单任务别名(alias 名与任务名相同):直接内联 TaskSpec,避免 str 自引用
|
||||
aliases: dict[str, str | list[str | px.TaskSpec] | px.TaskSpec | px.Graph] = {
|
||||
# 构建命令
|
||||
"b": "uv_build",
|
||||
"bc": "maturin_build",
|
||||
"ba": ["b", "bc"],
|
||||
# 安装命令
|
||||
"sync": "uv_sync",
|
||||
# 清理命令
|
||||
"c": "git_clean",
|
||||
# 开发工具
|
||||
"bump": ["c", "tc", "git_add_all", "bumpversion"],
|
||||
"bumpmi": "bumpversion_minor",
|
||||
"cov": ["git_clean", "test_coverage"],
|
||||
"doc": px.cmd(["sphinx-build", "-b", "html", "docs", "docs/_build"], name="doc", cwd=ROOT_DIR),
|
||||
"lint": px.cmd(["ruff", "check", "--fix", "--unsafe-fixes"], name="lint", cwd=ROOT_DIR),
|
||||
"pb": ["twine_publish", "publish_python"],
|
||||
"t": "test",
|
||||
"tf": "test_fast",
|
||||
"tc": ["pyrefly_check", "lint"],
|
||||
"tox": px.cmd(["tox", "-p", "auto"], name="tox", cwd=ROOT_DIR),
|
||||
# 发布命令
|
||||
"p": ["git_clean", "git_push", "git_push_tags"],
|
||||
}
|
||||
|
||||
|
||||
def main() -> None:
|
||||
"""pymake 构建工具.
|
||||
|
||||
🔨 构建命令:
|
||||
pymake b - 构建 Python 主包 (uv build)
|
||||
pymake bc - 构建 Rust 核心模块 (maturin build)
|
||||
pymake ba - 构建所有包 (先 Python 后 Rust)
|
||||
|
||||
📦 安装命令 (开发模式):
|
||||
pymake sync - 安装依赖包 (uv sync)
|
||||
|
||||
🧹 清理命令:
|
||||
pymake c - 清理所有构建产物 (gitt c)
|
||||
|
||||
🛠️ 开发工具:
|
||||
pymake t - 运行测试 (pytest)
|
||||
pymake tc - 运行测试并生成覆盖率报告
|
||||
pymake tf - 运行快速测试 (pytest -m not slow)
|
||||
pymake lint - 代码格式化与检查 (ruff)
|
||||
pymake type - 类型检查 (mypy, ty)
|
||||
pymake doc - 构建文档 (sphinx)
|
||||
|
||||
🔬 多版本测试:
|
||||
pymake tox - 多版本 Python 测试 (tox -p auto)
|
||||
|
||||
📦 发布命令:
|
||||
pymake pb - 发布到 PyPI (twine + hatch)
|
||||
|
||||
🔖 版本管理:
|
||||
pymake bump - 自动升级版本号并提交修改 (清理 + 检查 + 格式化 + git add + bumpversion)
|
||||
|
||||
💡 常用工作流:
|
||||
1. 日常开发: pymake lint && pymake t
|
||||
2. 构建发布包: pymake ba
|
||||
3. 多版本兼容性测试: pymake tox
|
||||
4. 发布到 PyPI: pymake pb
|
||||
|
||||
📝 示例:
|
||||
pymake ba # 构建所有包
|
||||
pymake sync # 安装依赖
|
||||
pymake t # 运行测试
|
||||
pymake tox # 多版本兼容性测试
|
||||
pymake lint # 格式化代码
|
||||
pymake type # 类型检查
|
||||
"""
|
||||
runner = px.CliRunner(strategy="sequential", description="PyMake - Python 构建工具", tasks=tasks, aliases=aliases)
|
||||
runner.run_cli()
|
||||
@@ -1,10 +0,0 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import pyflowx as px
|
||||
from pyflowx.tasks.system import reset_icon_cache
|
||||
|
||||
|
||||
def main() -> None:
|
||||
"""重启图标缓存工具主函数."""
|
||||
graph = px.Graph.from_specs(reset_icon_cache())
|
||||
px.run(graph, strategy="thread")
|
||||
@@ -1,163 +0,0 @@
|
||||
"""截图工具.
|
||||
|
||||
跨平台截图工具, 支持全屏截图和区域截图.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import subprocess
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
|
||||
import pyflowx as px
|
||||
from pyflowx.conditions import Constants
|
||||
|
||||
# ============================================================================
|
||||
# 辅助函数
|
||||
# ============================================================================
|
||||
|
||||
|
||||
def get_screenshot_path(filename: str | None = None) -> Path:
|
||||
"""获取截图保存路径.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
filename : str | None
|
||||
文件名, 如果为 None 则自动生成
|
||||
|
||||
Returns
|
||||
-------
|
||||
Path
|
||||
截图保存路径
|
||||
"""
|
||||
if filename is None:
|
||||
timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
|
||||
filename = f"screenshot_{timestamp}.png"
|
||||
|
||||
screenshots_dir = Path.home() / "Pictures" / "screenshots"
|
||||
screenshots_dir.mkdir(parents=True, exist_ok=True)
|
||||
return screenshots_dir / filename
|
||||
|
||||
|
||||
def take_screenshot_full(filename: str | None = None) -> None:
|
||||
"""全屏截图.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
filename : str | None
|
||||
文件名
|
||||
"""
|
||||
output_path = get_screenshot_path(filename)
|
||||
|
||||
if Constants.IS_WINDOWS:
|
||||
# Windows: 使用 PowerShell 截图
|
||||
ps_script = f"""
|
||||
Add-Type -AssemblyName System.Windows.Forms
|
||||
Add-Type -AssemblyName System.Drawing
|
||||
$screen = [System.Windows.Forms.Screen]::PrimaryScreen
|
||||
$bounds = $screen.Bounds
|
||||
$bitmap = New-Object System.Drawing.Bitmap $bounds.Width, $bounds.Height
|
||||
$graphics = [System.Drawing.Graphics]::FromImage($bitmap)
|
||||
$graphics.CopyFromScreen($bounds.Location, [System.Drawing.Point]::Empty, $bounds.Size)
|
||||
$bitmap.Save('{output_path.as_posix()}')
|
||||
$graphics.Dispose()
|
||||
$bitmap.Dispose()
|
||||
"""
|
||||
subprocess.run(["powershell", "-Command", ps_script], check=True)
|
||||
elif Constants.IS_MACOS:
|
||||
# macOS: 使用 screencapture
|
||||
subprocess.run(["screencapture", "-x", str(output_path)], check=True)
|
||||
else:
|
||||
# Linux: 使用 gnome-screenshot 或 scrot
|
||||
try:
|
||||
subprocess.run(["gnome-screenshot", "-f", str(output_path)], check=True)
|
||||
except FileNotFoundError:
|
||||
subprocess.run(["scrot", str(output_path)], check=True)
|
||||
|
||||
print(f"截图已保存: {output_path}")
|
||||
|
||||
|
||||
def take_screenshot_area(filename: str | None = None) -> None:
|
||||
"""区域截图.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
filename : str | None
|
||||
文件名
|
||||
"""
|
||||
output_path = get_screenshot_path(filename)
|
||||
|
||||
if Constants.IS_WINDOWS:
|
||||
# Windows: 使用 PowerShell 截图 (需要用户选择区域)
|
||||
ps_script = f"""
|
||||
Add-Type -AssemblyName System.Windows.Forms
|
||||
Add-Type -AssemblyName System.Drawing
|
||||
$form = New-Object System.Windows.Forms.Form
|
||||
$form.WindowState = 'Maximized'
|
||||
$form.FormBorderStyle = 'None'
|
||||
$form.BackColor = [System.Drawing.Color]::FromArgb(1, 0, 0)
|
||||
$form.Opacity = 0.5
|
||||
$form.TopMost = $true
|
||||
$form.Show()
|
||||
Start-Sleep -Milliseconds 100
|
||||
$screen = [System.Windows.Forms.Screen]::PrimaryScreen
|
||||
$bounds = $screen.Bounds
|
||||
$bitmap = New-Object System.Drawing.Bitmap $bounds.Width, $bounds.Height
|
||||
$graphics = [System.Drawing.Graphics]::FromImage($bitmap)
|
||||
$graphics.CopyFromScreen($bounds.Location, [System.Drawing.Point]::Empty, $bounds.Size)
|
||||
$form.Close()
|
||||
$bitmap.Save('{output_path.as_posix()}')
|
||||
$graphics.Dispose()
|
||||
$bitmap.Dispose()
|
||||
"""
|
||||
subprocess.run(["powershell", "-Command", ps_script], check=True)
|
||||
elif Constants.IS_MACOS:
|
||||
# macOS: 使用 screencapture 交互模式
|
||||
subprocess.run(["screencapture", "-i", str(output_path)], check=True)
|
||||
else:
|
||||
# Linux: 使用 gnome-screenshot 交互模式
|
||||
try:
|
||||
subprocess.run(["gnome-screenshot", "-a", "-f", str(output_path)], check=True)
|
||||
except FileNotFoundError:
|
||||
subprocess.run(["scrot", "-s", str(output_path)], check=True)
|
||||
|
||||
print(f"截图已保存: {output_path}")
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# CLI Runner
|
||||
# ============================================================================
|
||||
|
||||
|
||||
def main() -> None:
|
||||
"""截图工具主函数."""
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Screenshot - 截图工具",
|
||||
usage="screenshot <command> [options]",
|
||||
)
|
||||
subparsers = parser.add_subparsers(dest="command", help="可用命令")
|
||||
|
||||
# 全屏截图命令
|
||||
full_parser = subparsers.add_parser("full", help="全屏截图")
|
||||
full_parser.add_argument("--filename", type=str, help="文件名")
|
||||
|
||||
# 区域截图命令
|
||||
area_parser = subparsers.add_parser("area", help="区域截图")
|
||||
area_parser.add_argument("--filename", type=str, help="文件名")
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
if args.command == "full":
|
||||
graph = px.Graph.from_specs(
|
||||
[px.TaskSpec("screenshot_full", fn=take_screenshot_full, kwargs={"filename": args.filename})]
|
||||
)
|
||||
elif args.command == "area":
|
||||
graph = px.Graph.from_specs(
|
||||
[px.TaskSpec("screenshot_area", fn=take_screenshot_area, kwargs={"filename": args.filename})]
|
||||
)
|
||||
else:
|
||||
parser.print_help()
|
||||
return
|
||||
|
||||
px.run(graph, strategy="thread")
|
||||
@@ -1,122 +0,0 @@
|
||||
"""SSH 密钥部署工具.
|
||||
|
||||
类似 ssh-copy-id, 自动将 SSH 公钥部署到远程服务器,
|
||||
支持密码认证和密钥认证两种方式.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import pyflowx as px
|
||||
|
||||
# ============================================================================
|
||||
# 辅助函数
|
||||
# ============================================================================
|
||||
|
||||
|
||||
def ssh_copy_id(
|
||||
hostname: str,
|
||||
username: str,
|
||||
password: str,
|
||||
port: int = 22,
|
||||
keypath: str = "~/.ssh/id_rsa.pub",
|
||||
timeout: int = 30,
|
||||
) -> None:
|
||||
"""将 SSH 公钥部署到远程服务器.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
hostname : str
|
||||
远程服务器主机名或 IP 地址
|
||||
username : str
|
||||
远程服务器用户名
|
||||
password : str
|
||||
远程服务器密码
|
||||
port : int
|
||||
SSH 端口, 默认 22
|
||||
keypath : str
|
||||
公钥文件路径, 默认 ~/.ssh/id_rsa.pub
|
||||
timeout : int
|
||||
SSH 操作超时秒数, 默认 30
|
||||
"""
|
||||
# 读取公钥
|
||||
pub_key_path = Path(keypath).expanduser()
|
||||
if not pub_key_path.exists():
|
||||
print(f"公钥文件不存在: {pub_key_path}")
|
||||
sys.exit(1)
|
||||
|
||||
pub_key = pub_key_path.read_text().strip()
|
||||
|
||||
# 构建部署脚本
|
||||
script = f"""mkdir -p ~/.ssh && chmod 700 ~/.ssh
|
||||
cd ~/.ssh && touch authorized_keys && chmod 600 authorized_keys
|
||||
grep -qF '{pub_key.split()[1]}' authorized_keys 2>/dev/null || echo '{pub_key}' >> authorized_keys"""
|
||||
|
||||
# 使用 sshpass 执行
|
||||
try:
|
||||
subprocess.run(
|
||||
[
|
||||
"sshpass",
|
||||
"-p",
|
||||
password,
|
||||
"ssh",
|
||||
"-p",
|
||||
str(port),
|
||||
"-o",
|
||||
"StrictHostKeyChecking=no",
|
||||
"-o",
|
||||
"UserKnownHostsFile=/dev/null",
|
||||
"-o",
|
||||
f"ConnectTimeout={timeout}",
|
||||
f"{username}@{hostname}",
|
||||
script,
|
||||
],
|
||||
check=True,
|
||||
timeout=timeout,
|
||||
)
|
||||
print(f"SSH 密钥已部署到 {username}@{hostname}:{port}")
|
||||
except FileNotFoundError:
|
||||
print(f"未找到 sshpass 工具,请手动执行: ssh-copy-id -p {port} {username}@{hostname}")
|
||||
sys.exit(1)
|
||||
except subprocess.TimeoutExpired:
|
||||
print("SSH 连接超时")
|
||||
sys.exit(1)
|
||||
except subprocess.CalledProcessError as e:
|
||||
print(f"SSH 执行失败: {e}")
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# CLI Runner
|
||||
# ============================================================================
|
||||
|
||||
|
||||
def main() -> None:
|
||||
"""SSH 密钥部署工具主函数."""
|
||||
parser = argparse.ArgumentParser(
|
||||
description="SSHCopyID - SSH 密钥部署工具",
|
||||
usage="sshcopyid <hostname> <username> <password> [--port PORT] [--keypath KEYPATH]",
|
||||
)
|
||||
parser.add_argument("hostname", type=str, help="远程服务器主机名或 IP 地址")
|
||||
parser.add_argument("username", type=str, help="远程服务器用户名")
|
||||
parser.add_argument("password", type=str, help="远程服务器密码")
|
||||
parser.add_argument("--port", type=int, default=22, help="SSH 端口 (默认: 22)")
|
||||
parser.add_argument("--keypath", type=str, default="~/.ssh/id_rsa.pub", help="公钥文件路径")
|
||||
parser.add_argument("--timeout", type=int, default=30, help="SSH 操作超时秒数 (默认: 30)")
|
||||
args = parser.parse_args()
|
||||
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec(
|
||||
"ssh_deploy",
|
||||
fn=ssh_copy_id,
|
||||
args=(args.hostname, args.username, args.password),
|
||||
kwargs={"port": args.port, "keypath": args.keypath, "timeout": args.timeout},
|
||||
)
|
||||
]
|
||||
)
|
||||
px.run(graph, strategy="thread")
|
||||
@@ -0,0 +1,109 @@
|
||||
"""YAML 任务编排执行工具.
|
||||
|
||||
从 YAML 文件加载 GitHub Actions 风格的任务图并执行.
|
||||
支持串并行编排、矩阵扇出、条件执行等 CI/CD 核心概念.
|
||||
|
||||
用法
|
||||
----
|
||||
yamlrun pipeline.yaml # 执行 YAML 任务图
|
||||
yamlrun pipeline.yaml --strategy thread # 指定执行策略
|
||||
yamlrun pipeline.yaml --dry-run # 仅打印任务分层, 不执行
|
||||
yamlrun pipeline.yaml --list # 列出所有任务名
|
||||
yamlrun pipeline.yaml --quiet # 静默模式
|
||||
|
||||
示例 YAML
|
||||
----------
|
||||
::
|
||||
|
||||
strategy: thread
|
||||
jobs:
|
||||
setup:
|
||||
cmd: ["git", "clone", "https://github.com/foo/bar"]
|
||||
build:
|
||||
needs: [setup]
|
||||
cmd: ["python", "-m", "build"]
|
||||
test:
|
||||
needs: [build]
|
||||
cmd: ["pytest"]
|
||||
strategy:
|
||||
matrix:
|
||||
python: ["3.8", "3.9"]
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from typing import cast
|
||||
|
||||
import pyflowx as px
|
||||
from pyflowx.executors import Strategy
|
||||
|
||||
|
||||
def main() -> None:
|
||||
"""YAML 任务编排执行工具主函数."""
|
||||
parser = argparse.ArgumentParser(
|
||||
description="YamlRun - 从 YAML 文件加载并执行任务图",
|
||||
usage="yamlrun <file.yaml> [--strategy STRATEGY] [--dry-run] [--list] [--quiet]",
|
||||
)
|
||||
parser.add_argument("file", type=str, help="YAML 任务图文件路径")
|
||||
parser.add_argument(
|
||||
"--strategy",
|
||||
type=str,
|
||||
default=None,
|
||||
help="执行策略: sequential/thread/async/dependency (默认: YAML 中指定的策略或 dependency)",
|
||||
)
|
||||
parser.add_argument("--dry-run", action="store_true", help="仅打印任务分层, 不执行")
|
||||
parser.add_argument("--list", action="store_true", help="列出所有任务名后退出")
|
||||
parser.add_argument("--quiet", action="store_true", help="静默模式, 不打印详细输出")
|
||||
args = parser.parse_args()
|
||||
|
||||
file_path = Path(args.file)
|
||||
if not file_path.exists():
|
||||
print(f"错误: 文件不存在: {file_path}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
try:
|
||||
graph = px.Graph.from_yaml(file_path)
|
||||
except px.YamlLoadError as e:
|
||||
print(f"错误: YAML 加载失败: {e}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
if args.list:
|
||||
print("任务列表:")
|
||||
for name in graph.names:
|
||||
spec = graph.spec(name)
|
||||
deps = ", ".join(spec.depends_on) if spec.depends_on else "(无依赖)"
|
||||
print(f" - {name} (依赖: {deps})")
|
||||
sys.exit(0)
|
||||
|
||||
layers = graph.layers()
|
||||
print(f"任务分层 ({len(layers)} 层):")
|
||||
for i, layer in enumerate(layers):
|
||||
print(f" 层 {i + 1}: {layer}")
|
||||
|
||||
if args.dry_run:
|
||||
print("\n[dry-run] 跳过执行")
|
||||
sys.exit(0)
|
||||
|
||||
strategy = args.strategy or graph.defaults.strategy or "dependency"
|
||||
print(f"\n执行策略: {strategy}")
|
||||
print(f"任务总数: {len(graph.names)}")
|
||||
print("-" * 40)
|
||||
|
||||
report = px.run(graph, strategy=cast(Strategy, strategy), verbose=not args.quiet)
|
||||
|
||||
print("-" * 40)
|
||||
succeeded = report.succeeded_tasks()
|
||||
failed = report.failed_tasks()
|
||||
skipped = report.skipped_tasks()
|
||||
print(f"完成: {len(succeeded)} 成功 / {len(failed)} 失败 / {len(skipped)} 跳过 (共 {len(graph.names)})")
|
||||
|
||||
if failed:
|
||||
print(f"失败任务: {failed}")
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -90,6 +90,8 @@ def run_command(spec: TaskSpec[Any]) -> Any: # noqa: PLR0912
|
||||
print(f"[verbose] 返回码: {result.returncode}", flush=True)
|
||||
|
||||
if result.returncode == 0:
|
||||
if not verbose and result.stdout:
|
||||
print(result.stdout, end="", flush=True)
|
||||
return None
|
||||
|
||||
err_msg = f"{label}执行失败: `{cmd_str}`, 返回码: {result.returncode}"
|
||||
|
||||
@@ -0,0 +1,65 @@
|
||||
# autofmt - 自动格式化工具
|
||||
# 用法:
|
||||
# pf autofmt fmt --target .
|
||||
# pf autofmt lint --target .
|
||||
# pf autofmt lint --target . --fix
|
||||
# pf autofmt doc --root-dir .
|
||||
# pf autofmt sync --root-dir .
|
||||
strategy: thread
|
||||
variables:
|
||||
TARGET: "."
|
||||
ROOT_DIR: "."
|
||||
FIX: false
|
||||
cli:
|
||||
description: "AutoFmt - 自动格式化工具"
|
||||
usage: "pf autofmt <command> [options]"
|
||||
subcommands:
|
||||
fmt:
|
||||
help: "格式化代码"
|
||||
options:
|
||||
- name: TARGET
|
||||
flag: "--target"
|
||||
type: str
|
||||
default: "."
|
||||
help: "目标路径 (默认: .)"
|
||||
lint:
|
||||
help: "代码检查"
|
||||
options:
|
||||
- name: TARGET
|
||||
flag: "--target"
|
||||
type: str
|
||||
default: "."
|
||||
help: "目标路径 (默认: .)"
|
||||
- name: FIX
|
||||
flag: "--fix"
|
||||
action: "store_true"
|
||||
help: "自动修复问题"
|
||||
doc:
|
||||
help: "自动添加文档字符串"
|
||||
options:
|
||||
- name: ROOT_DIR
|
||||
flag: "--root-dir"
|
||||
type: str
|
||||
default: "."
|
||||
help: "根目录 (默认: .)"
|
||||
sync:
|
||||
help: "同步 pyproject 配置"
|
||||
options:
|
||||
- name: ROOT_DIR
|
||||
flag: "--root-dir"
|
||||
type: str
|
||||
default: "."
|
||||
help: "根目录 (默认: .)"
|
||||
jobs:
|
||||
fmt:
|
||||
cmd: ["ruff", "format", "${TARGET}"]
|
||||
lint:
|
||||
cmd: ["ruff", "check", "${TARGET}"]
|
||||
lint_fix:
|
||||
cmd: ["ruff", "check", "--fix", "--unsafe-fixes", "${TARGET}"]
|
||||
doc:
|
||||
fn: auto_add_docstrings
|
||||
args: ["${ROOT_DIR}"]
|
||||
sync:
|
||||
fn: sync_pyproject_config
|
||||
args: ["${ROOT_DIR}"]
|
||||
@@ -0,0 +1,27 @@
|
||||
# bumpversion - 版本号自动管理工具
|
||||
# 用法:
|
||||
# pf bumpversion
|
||||
# pf bumpversion minor --no-tag
|
||||
strategy: sequential
|
||||
variables:
|
||||
PART: patch
|
||||
NO_TAG: false
|
||||
cli:
|
||||
description: "BumpVersion - 版本号自动管理工具"
|
||||
usage: "pf bumpversion [part] [options]"
|
||||
positional:
|
||||
- name: PART
|
||||
type: str
|
||||
default: patch
|
||||
help: "版本部分: patch, minor, major"
|
||||
options:
|
||||
- name: NO_TAG
|
||||
flag: "--no-tag"
|
||||
action: "store_true"
|
||||
help: "提交后不创建 git tag"
|
||||
jobs:
|
||||
bump:
|
||||
fn: bump_project_version
|
||||
args: ["${PART}"]
|
||||
kwargs:
|
||||
no_tag: ${NO_TAG}
|
||||
@@ -0,0 +1,36 @@
|
||||
# filedate - 文件日期处理工具
|
||||
# 用法:
|
||||
# pf filedate add file1.txt file2.txt
|
||||
# pf filedate clear file1.txt file2.txt
|
||||
strategy: thread
|
||||
variables:
|
||||
FILES: []
|
||||
cli:
|
||||
description: "FileDate - 文件日期处理工具"
|
||||
usage: "pf filedate <command> [files...]"
|
||||
subcommands:
|
||||
add:
|
||||
help: "添加日期前缀"
|
||||
positional:
|
||||
- name: FILES
|
||||
nargs: "+"
|
||||
type: path
|
||||
help: "文件路径"
|
||||
clear:
|
||||
help: "清除日期前缀"
|
||||
positional:
|
||||
- name: FILES
|
||||
nargs: "+"
|
||||
type: path
|
||||
help: "文件路径"
|
||||
jobs:
|
||||
add:
|
||||
fn: process_files_date
|
||||
args: ["${FILES}"]
|
||||
kwargs:
|
||||
clear: false
|
||||
clear:
|
||||
fn: process_files_date
|
||||
args: ["${FILES}"]
|
||||
kwargs:
|
||||
clear: true
|
||||
@@ -0,0 +1,28 @@
|
||||
# filelevel - 文件等级重命名工具
|
||||
# 用法:
|
||||
# pf filelevel set file.txt --level 2
|
||||
strategy: thread
|
||||
variables:
|
||||
FILES: []
|
||||
LEVEL: 0
|
||||
cli:
|
||||
description: "FileLevel - 文件等级重命名工具"
|
||||
usage: "pf filelevel <command> [files...] [options]"
|
||||
subcommands:
|
||||
set:
|
||||
help: "设置文件等级"
|
||||
positional:
|
||||
- name: FILES
|
||||
nargs: "+"
|
||||
type: path
|
||||
help: "文件路径"
|
||||
options:
|
||||
- name: LEVEL
|
||||
flag: "--level"
|
||||
type: int
|
||||
required: true
|
||||
help: "文件等级 (0-4)"
|
||||
jobs:
|
||||
set:
|
||||
fn: process_files_level
|
||||
args: ["${FILES}", "${LEVEL}"]
|
||||
@@ -0,0 +1,34 @@
|
||||
# folderback - 文件夹备份工具
|
||||
# 用法:
|
||||
# pf folderback
|
||||
# pf folderback --src ./project --dst ./backup --max-zip 10
|
||||
strategy: thread
|
||||
variables:
|
||||
SRC: "."
|
||||
DST: "./backup"
|
||||
MAX_ZIP: 5
|
||||
cli:
|
||||
description: "FolderBack - 文件夹备份工具"
|
||||
usage: "pf folderback [options]"
|
||||
options:
|
||||
- name: SRC
|
||||
flag: "--src"
|
||||
type: str
|
||||
default: "."
|
||||
help: "源文件夹路径 (默认: 当前目录)"
|
||||
- name: DST
|
||||
flag: "--dst"
|
||||
type: str
|
||||
default: "./backup"
|
||||
help: "目标文件夹路径 (默认: ./backup)"
|
||||
- name: MAX_ZIP
|
||||
flag: "--max-zip"
|
||||
type: int
|
||||
default: 5
|
||||
help: "最大备份数量 (默认: 5)"
|
||||
jobs:
|
||||
backup:
|
||||
fn: backup_folder
|
||||
args: ["${SRC}", "${DST}"]
|
||||
kwargs:
|
||||
max_zip: ${MAX_ZIP}
|
||||
@@ -0,0 +1,21 @@
|
||||
# folderzip - 文件夹压缩工具
|
||||
# 用法:
|
||||
# pf folderzip
|
||||
# pf folderzip --cwd ./project
|
||||
strategy: thread
|
||||
variables:
|
||||
CWD: "."
|
||||
cli:
|
||||
description: "FolderZip - 文件夹压缩工具"
|
||||
usage: "pf folderzip [options]"
|
||||
options:
|
||||
- name: CWD
|
||||
flag: "--cwd"
|
||||
type: str
|
||||
required: false
|
||||
default: "."
|
||||
help: "工作目录 (默认: 当前目录)"
|
||||
jobs:
|
||||
zip:
|
||||
fn: zip_folders
|
||||
args: ["${CWD}"]
|
||||
@@ -0,0 +1,51 @@
|
||||
# gittool - Git 执行工具
|
||||
# 用法:
|
||||
# pf gittool a
|
||||
# pf gittool c
|
||||
# pf gittool i
|
||||
# pf gittool isub
|
||||
# pf gittool p
|
||||
# pf gittool pl
|
||||
strategy: thread
|
||||
variables:
|
||||
# git clean -e 参数列表 (展开为 cmd 数组元素)
|
||||
CLEAN_EXCLUDES: ["-e", ".venv", "-e", ".tox", "-e", ".pytest_cache",
|
||||
"-e", ".ruff_cache", "-e", "node_modules",
|
||||
"-e", ".idea", "-e", ".vscode",
|
||||
"-e", ".trae", "-e", ".qoder",
|
||||
"-e", ".editorconfig", "-e", "idea.config",
|
||||
"-e", "idea_modules.xml", "-e", "vcs.xml"]
|
||||
cli:
|
||||
description: "GitTool - Git 执行工具"
|
||||
usage: "pf gittool <command>"
|
||||
subcommands:
|
||||
a:
|
||||
help: "添加并提交"
|
||||
c:
|
||||
help: "清理并查看状态"
|
||||
i:
|
||||
help: "初始化并提交"
|
||||
isub:
|
||||
help: "初始化子目录"
|
||||
p:
|
||||
help: "推送"
|
||||
pl:
|
||||
help: "拉取"
|
||||
jobs:
|
||||
a:
|
||||
fn: git_add_commit
|
||||
args: ["chore: update"]
|
||||
clean:
|
||||
cmd: ["git", "clean", "-xfd", "${CLEAN_EXCLUDES}"]
|
||||
c:
|
||||
needs: [clean]
|
||||
cmd: ["git", "status", "--porcelain"]
|
||||
i:
|
||||
fn: git_init_add_commit
|
||||
args: ["init commit"]
|
||||
isub:
|
||||
fn: init_sub_dirs
|
||||
p:
|
||||
cmd: ["git", "push"]
|
||||
pl:
|
||||
cmd: ["git", "pull"]
|
||||
@@ -0,0 +1,51 @@
|
||||
# lscalc - LS-DYNA 计算工具
|
||||
# 用法:
|
||||
# pf lscalc run input.k --ncpu 4
|
||||
# pf lscalc status
|
||||
strategy: thread
|
||||
variables:
|
||||
INPUT_FILE: input.k
|
||||
NCPU: 4
|
||||
cli:
|
||||
description: "LSCalc - LS-DYNA 计算工具"
|
||||
usage: "pf lscalc <command> [options]"
|
||||
subcommands:
|
||||
run:
|
||||
help: "运行 LS-DYNA 计算"
|
||||
positional:
|
||||
- name: INPUT_FILE
|
||||
type: str
|
||||
help: "输入文件路径"
|
||||
options:
|
||||
- name: NCPU
|
||||
flag: "--ncpu"
|
||||
type: int
|
||||
default: 4
|
||||
help: "CPU 核心数 (默认: 4)"
|
||||
mpi:
|
||||
help: "运行 LS-DYNA MPI 计算"
|
||||
positional:
|
||||
- name: INPUT_FILE
|
||||
type: str
|
||||
help: "输入文件路径"
|
||||
options:
|
||||
- name: NCPU
|
||||
flag: "--ncpu"
|
||||
type: int
|
||||
default: 4
|
||||
help: "CPU 核心数 (默认: 4)"
|
||||
status:
|
||||
help: "检查 LS-DYNA 进程状态"
|
||||
jobs:
|
||||
run:
|
||||
fn: run_ls_dyna
|
||||
args: ["${INPUT_FILE}"]
|
||||
kwargs:
|
||||
ncpu: ${NCPU}
|
||||
mpi:
|
||||
fn: run_ls_dyna_mpi
|
||||
args: ["${INPUT_FILE}"]
|
||||
kwargs:
|
||||
ncpu: ${NCPU}
|
||||
status:
|
||||
fn: check_ls_dyna_status
|
||||
@@ -0,0 +1,107 @@
|
||||
# packtool - Python 打包工具
|
||||
# 用法:
|
||||
# pf packtool src --project-dir . --output-dir .pypack
|
||||
# pf packtool deps requests numpy --lib-dir libs
|
||||
# pf packtool wheel --project-dir . --output-dir dist
|
||||
# pf packtool embed --version 3.10 --output-dir python
|
||||
# pf packtool zip --source-dir . --output-file package.zip
|
||||
# pf packtool clean
|
||||
strategy: thread
|
||||
variables:
|
||||
PROJECT_DIR: "."
|
||||
OUTPUT_DIR: ".pypack"
|
||||
LIB_DIR: "libs"
|
||||
DEPENDENCIES: []
|
||||
VERSION: "3.10"
|
||||
OUTPUT_FILE: "package.zip"
|
||||
SOURCE_DIR: "."
|
||||
cli:
|
||||
description: "PackTool - Python 打包工具"
|
||||
usage: "pf packtool <command> [options]"
|
||||
subcommands:
|
||||
src:
|
||||
help: "打包源码"
|
||||
options:
|
||||
- name: PROJECT_DIR
|
||||
flag: "--project-dir"
|
||||
type: path
|
||||
default: "."
|
||||
help: "项目目录 (默认: .)"
|
||||
- name: OUTPUT_DIR
|
||||
flag: "--output-dir"
|
||||
type: str
|
||||
default: ".pypack"
|
||||
help: "输出目录 (默认: .pypack)"
|
||||
deps:
|
||||
help: "打包依赖"
|
||||
positional:
|
||||
- name: DEPENDENCIES
|
||||
nargs: "*"
|
||||
type: str
|
||||
help: "依赖包列表"
|
||||
options:
|
||||
- name: LIB_DIR
|
||||
flag: "--lib-dir"
|
||||
type: path
|
||||
default: "libs"
|
||||
help: "依赖库目录 (默认: libs)"
|
||||
wheel:
|
||||
help: "构建 wheel"
|
||||
options:
|
||||
- name: PROJECT_DIR
|
||||
flag: "--project-dir"
|
||||
type: path
|
||||
default: "."
|
||||
help: "项目目录 (默认: .)"
|
||||
- name: OUTPUT_DIR
|
||||
flag: "--output-dir"
|
||||
type: path
|
||||
default: "dist"
|
||||
help: "输出目录 (默认: dist)"
|
||||
embed:
|
||||
help: "安装嵌入式 Python"
|
||||
options:
|
||||
- name: VERSION
|
||||
flag: "--version"
|
||||
type: str
|
||||
default: "3.10"
|
||||
help: "Python 版本 (默认: 3.10)"
|
||||
- name: OUTPUT_DIR
|
||||
flag: "--output-dir"
|
||||
type: path
|
||||
default: "python"
|
||||
help: "输出目录 (默认: python)"
|
||||
zip:
|
||||
help: "创建 zip 包"
|
||||
options:
|
||||
- name: SOURCE_DIR
|
||||
flag: "--source-dir"
|
||||
type: path
|
||||
default: "."
|
||||
help: "源目录 (默认: .)"
|
||||
- name: OUTPUT_FILE
|
||||
flag: "--output-file"
|
||||
type: path
|
||||
default: "package.zip"
|
||||
help: "输出文件 (默认: package.zip)"
|
||||
clean:
|
||||
help: "清理构建目录"
|
||||
jobs:
|
||||
src:
|
||||
fn: pack_source
|
||||
args: ["${PROJECT_DIR}", "${OUTPUT_DIR}"]
|
||||
deps:
|
||||
fn: pack_dependencies
|
||||
args: ["${LIB_DIR}", "${DEPENDENCIES}"]
|
||||
wheel:
|
||||
fn: pack_wheel
|
||||
args: ["${PROJECT_DIR}", "${OUTPUT_DIR}"]
|
||||
embed:
|
||||
fn: install_embed_python
|
||||
args: ["${VERSION}", "${OUTPUT_DIR}"]
|
||||
zip:
|
||||
fn: create_zip_package
|
||||
args: ["${SOURCE_DIR}", "${OUTPUT_FILE}"]
|
||||
clean:
|
||||
fn: clean_build_dir
|
||||
args: ["${OUTPUT_DIR}"]
|
||||
@@ -0,0 +1,303 @@
|
||||
# pdftool - PDF 文件工具集
|
||||
# 用法:
|
||||
# pf pdftool m a.pdf b.pdf --output merged.pdf
|
||||
# pf pdftool s input.pdf --output-dir split
|
||||
# pf pdftool c input.pdf --output compressed.pdf
|
||||
# pf pdftool e input.pdf --output encrypted.pdf --password 123456
|
||||
# pf pdftool d input.pdf --output decrypted.pdf --password 123456
|
||||
# pf pdftool xt input.pdf --output output.txt
|
||||
# pf pdftool xi input.pdf --output-dir images
|
||||
# pf pdftool w input.pdf --output watermarked.pdf --text CONFIDENTIAL
|
||||
# pf pdftool r input.pdf --output rotated.pdf --rotation 90
|
||||
# pf pdftool crop input.pdf --output cropped.pdf --left 10 --top 10 --right 10 --bottom 10
|
||||
# pf pdftool i input.pdf
|
||||
# pf pdftool ocr input.pdf --output ocr.pdf --lang chi_sim+eng
|
||||
# pf pdftool img input.pdf --output-dir images --dpi 300
|
||||
# pf pdftool repair input.pdf --output repaired.pdf
|
||||
strategy: thread
|
||||
variables:
|
||||
INPUT: input.pdf
|
||||
INPUTS: []
|
||||
OUTPUT: output.pdf
|
||||
OUTPUT_DIR: output
|
||||
PASSWORD: ""
|
||||
TEXT: CONFIDENTIAL
|
||||
ROTATION: 90
|
||||
MARGINS: [10, 10, 10, 10]
|
||||
DPI: 300
|
||||
LANG: chi_sim+eng
|
||||
ORDER: []
|
||||
LEFT: 10
|
||||
TOP: 10
|
||||
RIGHT: 10
|
||||
BOTTOM: 10
|
||||
cli:
|
||||
description: "PdfTool - PDF 文件工具集"
|
||||
usage: "pf pdftool <command> [options]"
|
||||
subcommands:
|
||||
m:
|
||||
help: "合并 PDF"
|
||||
positional:
|
||||
- name: INPUTS
|
||||
nargs: "+"
|
||||
type: path
|
||||
help: "输入 PDF 文件列表"
|
||||
options:
|
||||
- name: OUTPUT
|
||||
flag: "--output"
|
||||
type: path
|
||||
default: "merged.pdf"
|
||||
help: "输出文件 (默认: merged.pdf)"
|
||||
s:
|
||||
help: "拆分 PDF"
|
||||
positional:
|
||||
- name: INPUT
|
||||
type: path
|
||||
help: "输入 PDF 文件"
|
||||
options:
|
||||
- name: OUTPUT_DIR
|
||||
flag: "--output-dir"
|
||||
type: path
|
||||
default: "split"
|
||||
help: "输出目录 (默认: split)"
|
||||
c:
|
||||
help: "压缩 PDF"
|
||||
positional:
|
||||
- name: INPUT
|
||||
type: path
|
||||
help: "输入 PDF 文件"
|
||||
options:
|
||||
- name: OUTPUT
|
||||
flag: "--output"
|
||||
type: path
|
||||
default: "compressed.pdf"
|
||||
help: "输出文件 (默认: compressed.pdf)"
|
||||
e:
|
||||
help: "加密 PDF"
|
||||
positional:
|
||||
- name: INPUT
|
||||
type: path
|
||||
help: "输入 PDF 文件"
|
||||
options:
|
||||
- name: OUTPUT
|
||||
flag: "--output"
|
||||
type: path
|
||||
default: "encrypted.pdf"
|
||||
help: "输出文件 (默认: encrypted.pdf)"
|
||||
- name: PASSWORD
|
||||
flag: "--password"
|
||||
type: str
|
||||
required: true
|
||||
help: "密码 (必填)"
|
||||
d:
|
||||
help: "解密 PDF"
|
||||
positional:
|
||||
- name: INPUT
|
||||
type: path
|
||||
help: "输入 PDF 文件"
|
||||
options:
|
||||
- name: OUTPUT
|
||||
flag: "--output"
|
||||
type: path
|
||||
default: "decrypted.pdf"
|
||||
help: "输出文件 (默认: decrypted.pdf)"
|
||||
- name: PASSWORD
|
||||
flag: "--password"
|
||||
type: str
|
||||
required: true
|
||||
help: "密码 (必填)"
|
||||
xt:
|
||||
help: "提取文本"
|
||||
positional:
|
||||
- name: INPUT
|
||||
type: path
|
||||
help: "输入 PDF 文件"
|
||||
options:
|
||||
- name: OUTPUT
|
||||
flag: "--output"
|
||||
type: path
|
||||
default: "output.txt"
|
||||
help: "输出文件 (默认: output.txt)"
|
||||
xi:
|
||||
help: "提取图片"
|
||||
positional:
|
||||
- name: INPUT
|
||||
type: path
|
||||
help: "输入 PDF 文件"
|
||||
options:
|
||||
- name: OUTPUT_DIR
|
||||
flag: "--output-dir"
|
||||
type: path
|
||||
default: "images"
|
||||
help: "输出目录 (默认: images)"
|
||||
w:
|
||||
help: "添加水印"
|
||||
positional:
|
||||
- name: INPUT
|
||||
type: path
|
||||
help: "输入 PDF 文件"
|
||||
options:
|
||||
- name: OUTPUT
|
||||
flag: "--output"
|
||||
type: path
|
||||
default: "watermarked.pdf"
|
||||
help: "输出文件 (默认: watermarked.pdf)"
|
||||
- name: TEXT
|
||||
flag: "--text"
|
||||
type: str
|
||||
default: "CONFIDENTIAL"
|
||||
help: "水印文字 (默认: CONFIDENTIAL)"
|
||||
r:
|
||||
help: "旋转 PDF"
|
||||
positional:
|
||||
- name: INPUT
|
||||
type: path
|
||||
help: "输入 PDF 文件"
|
||||
options:
|
||||
- name: OUTPUT
|
||||
flag: "--output"
|
||||
type: path
|
||||
default: "rotated.pdf"
|
||||
help: "输出文件 (默认: rotated.pdf)"
|
||||
- name: ROTATION
|
||||
flag: "--rotation"
|
||||
type: int
|
||||
default: 90
|
||||
help: "旋转角度 (默认: 90)"
|
||||
crop:
|
||||
help: "裁剪 PDF"
|
||||
positional:
|
||||
- name: INPUT
|
||||
type: path
|
||||
help: "输入 PDF 文件"
|
||||
options:
|
||||
- name: OUTPUT
|
||||
flag: "--output"
|
||||
type: path
|
||||
default: "cropped.pdf"
|
||||
help: "输出文件 (默认: cropped.pdf)"
|
||||
- name: LEFT
|
||||
flag: "--left"
|
||||
type: int
|
||||
default: 10
|
||||
help: "左边距 (默认: 10)"
|
||||
- name: TOP
|
||||
flag: "--top"
|
||||
type: int
|
||||
default: 10
|
||||
help: "上边距 (默认: 10)"
|
||||
- name: RIGHT
|
||||
flag: "--right"
|
||||
type: int
|
||||
default: 10
|
||||
help: "右边距 (默认: 10)"
|
||||
- name: BOTTOM
|
||||
flag: "--bottom"
|
||||
type: int
|
||||
default: 10
|
||||
help: "下边距 (默认: 10)"
|
||||
i:
|
||||
help: "查看 PDF 信息"
|
||||
positional:
|
||||
- name: INPUT
|
||||
type: path
|
||||
help: "输入 PDF 文件"
|
||||
ocr:
|
||||
help: "PDF OCR 识别"
|
||||
positional:
|
||||
- name: INPUT
|
||||
type: path
|
||||
help: "输入 PDF 文件"
|
||||
options:
|
||||
- name: OUTPUT
|
||||
flag: "--output"
|
||||
type: path
|
||||
default: "ocr.pdf"
|
||||
help: "输出文件 (默认: ocr.pdf)"
|
||||
- name: LANG
|
||||
flag: "--lang"
|
||||
type: str
|
||||
default: "chi_sim+eng"
|
||||
help: "识别语言 (默认: chi_sim+eng)"
|
||||
img:
|
||||
help: "PDF 转图片"
|
||||
positional:
|
||||
- name: INPUT
|
||||
type: path
|
||||
help: "输入 PDF 文件"
|
||||
options:
|
||||
- name: OUTPUT_DIR
|
||||
flag: "--output-dir"
|
||||
type: path
|
||||
default: "images"
|
||||
help: "输出目录 (默认: images)"
|
||||
- name: DPI
|
||||
flag: "--dpi"
|
||||
type: int
|
||||
default: 300
|
||||
help: "DPI (默认: 300)"
|
||||
repair:
|
||||
help: "修复 PDF"
|
||||
positional:
|
||||
- name: INPUT
|
||||
type: path
|
||||
help: "输入 PDF 文件"
|
||||
options:
|
||||
- name: OUTPUT
|
||||
flag: "--output"
|
||||
type: path
|
||||
default: "repaired.pdf"
|
||||
help: "输出文件 (默认: repaired.pdf)"
|
||||
jobs:
|
||||
m:
|
||||
fn: pdf_merge
|
||||
args: ["${INPUTS}", "${OUTPUT}"]
|
||||
s:
|
||||
fn: pdf_split
|
||||
args: ["${INPUT}", "${OUTPUT_DIR}"]
|
||||
c:
|
||||
fn: pdf_compress
|
||||
args: ["${INPUT}", "${OUTPUT}"]
|
||||
e:
|
||||
fn: pdf_encrypt
|
||||
args: ["${INPUT}", "${OUTPUT}", "${PASSWORD}"]
|
||||
d:
|
||||
fn: pdf_decrypt
|
||||
args: ["${INPUT}", "${OUTPUT}", "${PASSWORD}"]
|
||||
xt:
|
||||
fn: pdf_extract_text
|
||||
args: ["${INPUT}", "${OUTPUT}"]
|
||||
xi:
|
||||
fn: pdf_extract_images
|
||||
args: ["${INPUT}", "${OUTPUT_DIR}"]
|
||||
w:
|
||||
fn: pdf_add_watermark
|
||||
args: ["${INPUT}", "${OUTPUT}"]
|
||||
kwargs:
|
||||
text: "${TEXT}"
|
||||
r:
|
||||
fn: pdf_rotate
|
||||
args: ["${INPUT}", "${OUTPUT}"]
|
||||
kwargs:
|
||||
rotation: ${ROTATION}
|
||||
crop:
|
||||
fn: pdf_crop
|
||||
args: ["${INPUT}", "${OUTPUT}"]
|
||||
kwargs:
|
||||
margins: "${MARGINS}"
|
||||
i:
|
||||
fn: pdf_info
|
||||
args: ["${INPUT}"]
|
||||
ocr:
|
||||
fn: pdf_ocr
|
||||
args: ["${INPUT}", "${OUTPUT}"]
|
||||
kwargs:
|
||||
lang: "${LANG}"
|
||||
img:
|
||||
fn: pdf_to_images
|
||||
args: ["${INPUT}", "${OUTPUT_DIR}"]
|
||||
kwargs:
|
||||
dpi: ${DPI}
|
||||
repair:
|
||||
fn: pdf_repair
|
||||
args: ["${INPUT}", "${OUTPUT}"]
|
||||
@@ -0,0 +1,78 @@
|
||||
# piptool - pip 包管理工具
|
||||
# 用法:
|
||||
# pf piptool i requests
|
||||
# pf piptool u requests
|
||||
# pf piptool r requests
|
||||
# pf piptool d requests
|
||||
# pf piptool up
|
||||
# pf piptool f
|
||||
strategy: thread
|
||||
variables:
|
||||
PACKAGES: []
|
||||
OFFLINE: false
|
||||
cli:
|
||||
description: "PipTool - pip 包管理工具"
|
||||
usage: "pf piptool <command> [packages...] [options]"
|
||||
subcommands:
|
||||
i:
|
||||
help: "安装包"
|
||||
positional:
|
||||
- name: PACKAGES
|
||||
nargs: "+"
|
||||
type: str
|
||||
help: "包名列表"
|
||||
u:
|
||||
help: "卸载包"
|
||||
positional:
|
||||
- name: PACKAGES
|
||||
nargs: "+"
|
||||
type: str
|
||||
help: "包名列表"
|
||||
r:
|
||||
help: "重装包"
|
||||
positional:
|
||||
- name: PACKAGES
|
||||
nargs: "+"
|
||||
type: str
|
||||
help: "包名列表"
|
||||
options:
|
||||
- name: OFFLINE
|
||||
flag: "--offline"
|
||||
action: "store_true"
|
||||
help: "离线模式"
|
||||
d:
|
||||
help: "下载包"
|
||||
positional:
|
||||
- name: PACKAGES
|
||||
nargs: "+"
|
||||
type: str
|
||||
help: "包名列表"
|
||||
options:
|
||||
- name: OFFLINE
|
||||
flag: "--offline"
|
||||
action: "store_true"
|
||||
help: "离线模式"
|
||||
up:
|
||||
help: "升级 pip"
|
||||
f:
|
||||
help: "导出依赖"
|
||||
jobs:
|
||||
i:
|
||||
cmd: ["pip", "install", "${PACKAGES}"]
|
||||
u:
|
||||
fn: pip_uninstall
|
||||
args: ["${PACKAGES}"]
|
||||
r:
|
||||
fn: pip_reinstall
|
||||
args: ["${PACKAGES}"]
|
||||
kwargs:
|
||||
offline: ${OFFLINE}
|
||||
d:
|
||||
fn: pip_download
|
||||
args: ["${PACKAGES}"]
|
||||
kwargs:
|
||||
offline: ${OFFLINE}
|
||||
up:
|
||||
cmd: ["python", "-m", "pip", "install", "--upgrade", "pip"]
|
||||
f:
|
||||
fn: pip_freeze
|
||||
@@ -0,0 +1,125 @@
|
||||
# pymake - 项目构建工具
|
||||
# 用法
|
||||
# pf pymake <command>
|
||||
# 命令
|
||||
# b: 构建 Python 主包 (uv build)
|
||||
# ba: 构建所有包 (Python + Rust)
|
||||
# bc: 构建 Rust 核心模块 (maturin build)
|
||||
# bump: 升级版本号 (清理 + 检查 + add + bumpversion)
|
||||
# bumpmi: 升级次版本号 (bumpversion minor)
|
||||
# c: 清理构建产物 (调用 gitt c)
|
||||
# cov: 测试并生成覆盖率
|
||||
# doc: 构建 Sphinx 文档
|
||||
# lint: 代码格式化与检查 (ruff)
|
||||
# p: 推送代码 (清理 + push + push tags)
|
||||
# pb: 发布到 PyPI (twine + hatch)
|
||||
# sync: 同步依赖 (uv sync)
|
||||
# t: 运行测试
|
||||
# tc: 类型检查 (pyrefly + ruff)
|
||||
# tf: 快速测试 (无 slow)
|
||||
# tox: 多版本测试 (tox)
|
||||
strategy: thread
|
||||
variables:
|
||||
CWD: "."
|
||||
cli:
|
||||
description: "PyMake - 项目构建工具"
|
||||
usage: "pf pymake <command>"
|
||||
options:
|
||||
- name: CWD
|
||||
flag: "--cwd"
|
||||
type: path
|
||||
required: false
|
||||
default: "."
|
||||
help: "工作目录 (默认: 当前目录)"
|
||||
subcommands:
|
||||
b: {help: "构建 Python 主包 (uv build)"}
|
||||
ba: {help: "构建所有包 (Python + Rust)"}
|
||||
bc: {help: "构建 Rust 核心模块 (maturin build)"}
|
||||
bump: {help: "升级版本号 (清理 + 检查 + add + bumpversion)"}
|
||||
bumpmi: {help: "升级次版本号 (bumpversion minor)"}
|
||||
c: {help: "清理构建产物 (调用 gitt c)"}
|
||||
cov: {help: "测试并生成覆盖率"}
|
||||
doc: {help: "构建 Sphinx 文档"}
|
||||
lint: {help: "代码格式化与检查 (ruff)"}
|
||||
p: {help: "推送代码 (清理 + push + push tags)"}
|
||||
pb: {help: "发布到 PyPI (twine + hatch)"}
|
||||
sync: {help: "同步依赖 (uv sync)"}
|
||||
t: {help: "运行测试"}
|
||||
tc: {help: "类型检查 (pyrefly + ruff)"}
|
||||
tf: {help: "快速测试 (无 slow)"}
|
||||
tox: {help: "多版本测试 (tox)"}
|
||||
jobs:
|
||||
# 单任务别名
|
||||
b:
|
||||
cmd: ["uv", "build"]
|
||||
cwd: ${CWD}
|
||||
bc:
|
||||
cmd: ["maturin", "build", "-r"]
|
||||
cwd: ${CWD}
|
||||
sync:
|
||||
cmd: ["uv", "sync"]
|
||||
cwd: ${CWD}
|
||||
c:
|
||||
cmd: ["pf", "gitt", "c"]
|
||||
cwd: ${CWD}
|
||||
t:
|
||||
cmd: ["pytest", "-m", "not slow", "-n", "8", "--dist", "loadfile", "--color=yes", "--durations=10"]
|
||||
cwd: ${CWD}
|
||||
tf:
|
||||
cmd: ["pytest", "-m", "not slow", "--dist", "loadfile", "--color=yes", "--durations=10"]
|
||||
cwd: ${CWD}
|
||||
bumpversion:
|
||||
cmd: ["bumpversion", "patch"]
|
||||
needs: [git_add_all]
|
||||
cwd: ${CWD}
|
||||
bumpmi:
|
||||
cmd: ["bumpversion", "minor"]
|
||||
cwd: ${CWD}
|
||||
doc:
|
||||
cmd: ["sphinx-build", "-b", "html", "docs", "docs/_build"]
|
||||
cwd: ${CWD}
|
||||
lint:
|
||||
cmd: ["ruff", "check", "--fix", "--unsafe-fixes"]
|
||||
cwd: ${CWD}
|
||||
tox:
|
||||
cmd: ["tox", "-p", "auto"]
|
||||
cwd: ${CWD}
|
||||
|
||||
# 内部 job (不暴露为 subcommand)
|
||||
test_coverage:
|
||||
cmd: ["pytest", "--cov", "-n", "8", "--dist", "loadfile", "--tb=short", "-v", "--color=yes", "--durations=10"]
|
||||
needs: [c]
|
||||
cwd: ${CWD}
|
||||
pyrefly_check:
|
||||
cmd: ["pyrefly", "check", "."]
|
||||
cwd: ${CWD}
|
||||
git_add_all:
|
||||
cmd: ["git", "add", "-A"]
|
||||
needs: [tc]
|
||||
cwd: ${CWD}
|
||||
git_push:
|
||||
cmd: ["git", "push"]
|
||||
cwd: ${CWD}
|
||||
git_push_tags:
|
||||
cmd: ["git", "push", "--tags"]
|
||||
cwd: ${CWD}
|
||||
twine_publish:
|
||||
cmd: ["twine", "upload", "--disable-progress-bar"]
|
||||
cwd: ${CWD}
|
||||
publish_python:
|
||||
cmd: ["hatch", "publish"]
|
||||
cwd: ${CWD}
|
||||
|
||||
# 聚合 job (方向 B: 有 needs 无 cmd/fn)
|
||||
ba:
|
||||
needs: [b, bc]
|
||||
bump:
|
||||
needs: [bumpversion]
|
||||
cov:
|
||||
needs: [test_coverage]
|
||||
tc:
|
||||
needs: [c, pyrefly_check, lint]
|
||||
p:
|
||||
needs: [c, git_push, git_push_tags]
|
||||
pb:
|
||||
needs: [twine_publish, publish_python]
|
||||
@@ -0,0 +1,13 @@
|
||||
# reseticoncache - 重置 Windows 图标缓存
|
||||
# 用法
|
||||
# pf reseticon
|
||||
# 说明
|
||||
# 杀掉 explorer → 删除 IconCache.db → 删除 iconcache* → 重启 explorer
|
||||
# 仅在 Windows 上有效, 非 Windows 平台打印提示并跳过
|
||||
strategy: sequential
|
||||
cli:
|
||||
description: "重置 Windows 图标缓存"
|
||||
usage: "pf reseticon"
|
||||
jobs:
|
||||
reset:
|
||||
fn: reset_icon_cache_run
|
||||
@@ -0,0 +1,34 @@
|
||||
# screenshot - 截图工具
|
||||
# 用法:
|
||||
# pf screenshot full
|
||||
# pf screenshot area --filename custom.png
|
||||
strategy: thread
|
||||
variables:
|
||||
FILENAME: null
|
||||
cli:
|
||||
description: "Screenshot - 截图工具"
|
||||
usage: "pf screenshot <command> [options]"
|
||||
subcommands:
|
||||
full:
|
||||
help: "全屏截图"
|
||||
options:
|
||||
- name: FILENAME
|
||||
flag: "--filename"
|
||||
type: str
|
||||
help: "文件名"
|
||||
area:
|
||||
help: "区域截图"
|
||||
options:
|
||||
- name: FILENAME
|
||||
flag: "--filename"
|
||||
type: str
|
||||
help: "文件名"
|
||||
jobs:
|
||||
full:
|
||||
fn: take_screenshot_full
|
||||
kwargs:
|
||||
filename: "${FILENAME}"
|
||||
area:
|
||||
fn: take_screenshot_area
|
||||
kwargs:
|
||||
filename: "${FILENAME}"
|
||||
@@ -0,0 +1,49 @@
|
||||
# sshcopyid - SSH 密钥部署工具
|
||||
# 用法:
|
||||
# pf sshcopyid hostname username password
|
||||
# pf sshcopyid server user pass --port 2222
|
||||
strategy: thread
|
||||
variables:
|
||||
HOSTNAME: ""
|
||||
USERNAME: ""
|
||||
PASSWORD: ""
|
||||
PORT: 22
|
||||
KEYPATH: "~/.ssh/id_rsa.pub"
|
||||
TIMEOUT: 30
|
||||
cli:
|
||||
description: "SSHCopyID - SSH 密钥部署工具"
|
||||
usage: "pf sshcopyid <hostname> <username> <password> [options]"
|
||||
positional:
|
||||
- name: HOSTNAME
|
||||
type: str
|
||||
help: "远程服务器主机名或 IP 地址"
|
||||
- name: USERNAME
|
||||
type: str
|
||||
help: "远程服务器用户名"
|
||||
- name: PASSWORD
|
||||
type: str
|
||||
help: "远程服务器密码"
|
||||
options:
|
||||
- name: PORT
|
||||
flag: "--port"
|
||||
type: int
|
||||
default: 22
|
||||
help: "SSH 端口 (默认: 22)"
|
||||
- name: KEYPATH
|
||||
flag: "--keypath"
|
||||
type: str
|
||||
default: "~/.ssh/id_rsa.pub"
|
||||
help: "公钥文件路径"
|
||||
- name: TIMEOUT
|
||||
flag: "--timeout"
|
||||
type: int
|
||||
default: 30
|
||||
help: "SSH 操作超时秒数 (默认: 30)"
|
||||
jobs:
|
||||
deploy:
|
||||
fn: ssh_copy_id
|
||||
args: ["${HOSTNAME}", "${USERNAME}", "${PASSWORD}"]
|
||||
kwargs:
|
||||
port: ${PORT}
|
||||
keypath: "${KEYPATH}"
|
||||
timeout: ${TIMEOUT}
|
||||
@@ -31,12 +31,14 @@ def aggregate(ctx: px.Context) -> dict[str, Any]:
|
||||
|
||||
|
||||
def main() -> None:
|
||||
graph = px.Graph.from_specs([
|
||||
# Static positional args parameterise the same function twice.
|
||||
px.TaskSpec("fetch_user", fetch_user, args=(1,)),
|
||||
px.TaskSpec("fetch_posts", fetch_posts, args=(1,)),
|
||||
px.TaskSpec("aggregate", aggregate, depends_on=("fetch_user", "fetch_posts")),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
# Static positional args parameterise the same function twice.
|
||||
px.TaskSpec("fetch_user", fetch_user, args=(1,)),
|
||||
px.TaskSpec("fetch_posts", fetch_posts, args=(1,)),
|
||||
px.TaskSpec("aggregate", aggregate, depends_on=("fetch_user", "fetch_posts")),
|
||||
]
|
||||
)
|
||||
|
||||
print("=== Dry run ===")
|
||||
_ = px.run(graph, strategy="async", dry_run=True)
|
||||
|
||||
@@ -46,19 +46,21 @@ def load(transform: list[dict[str, Any]]) -> int:
|
||||
|
||||
|
||||
def main() -> None:
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("extract_customers", extract_customers, tags=("extract",)),
|
||||
px.TaskSpec("extract_orders", extract_orders, tags=("extract",)),
|
||||
px.TaskSpec(
|
||||
"transform",
|
||||
transform,
|
||||
depends_on=("extract_customers", "extract_orders"),
|
||||
tags=("transform",),
|
||||
),
|
||||
px.TaskSpec(
|
||||
"load", load, depends_on=("transform",), retry=px.RetryPolicy(max_attempts=1, delay=1.0), tags=("load",)
|
||||
),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("extract_customers", extract_customers, tags=("extract",)),
|
||||
px.TaskSpec("extract_orders", extract_orders, tags=("extract",)),
|
||||
px.TaskSpec(
|
||||
"transform",
|
||||
transform,
|
||||
depends_on=("extract_customers", "extract_orders"),
|
||||
tags=("transform",),
|
||||
),
|
||||
px.TaskSpec(
|
||||
"load", load, depends_on=("transform",), retry=px.RetryPolicy(max_attempts=1, delay=1.0), tags=("load",)
|
||||
),
|
||||
]
|
||||
)
|
||||
|
||||
print("=== Execution plan ===")
|
||||
print(graph.describe())
|
||||
|
||||
@@ -29,11 +29,13 @@ def merge(fetch_a: str, fetch_b: str) -> str:
|
||||
|
||||
|
||||
def main() -> None:
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("fetch_a", fetch_a),
|
||||
px.TaskSpec("fetch_b", fetch_b),
|
||||
px.TaskSpec("merge", merge, depends_on=("fetch_a", "fetch_b")),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("fetch_a", fetch_a),
|
||||
px.TaskSpec("fetch_b", fetch_b),
|
||||
px.TaskSpec("merge", merge, depends_on=("fetch_a", "fetch_b")),
|
||||
]
|
||||
)
|
||||
|
||||
print("=== Mermaid diagram ===")
|
||||
print(graph.to_mermaid("LR"))
|
||||
|
||||
@@ -792,6 +792,13 @@ def run(
|
||||
_print_dry_run(graph, layers)
|
||||
return RunReport(success=True)
|
||||
|
||||
# verbose 模式下, 把所有 spec 的 verbose 标记设为 True,
|
||||
# 使 execute_command 打印执行命令与返回码 (任务生命周期由 callback 打印)
|
||||
if verbose:
|
||||
from dataclasses import replace
|
||||
|
||||
graph = Graph.from_specs([replace(s, verbose=True) if not s.verbose else s for s in graph.all_specs().values()])
|
||||
|
||||
# 入口统一校验一次:所有策略共用,避免 layers() / dependency 路径
|
||||
# 各自重复调用 validate()。
|
||||
graph.validate()
|
||||
|
||||
+33
-1
@@ -20,6 +20,7 @@ __all__ = [
|
||||
import inspect
|
||||
import sys
|
||||
from dataclasses import dataclass, field, replace
|
||||
from pathlib import Path
|
||||
from typing import Any, Callable, Iterable, Mapping, Sequence
|
||||
|
||||
from .errors import CycleError, DuplicateTaskError, MissingDependencyError
|
||||
@@ -219,7 +220,6 @@ class Graph:
|
||||
"""
|
||||
graph = cls(defaults=defaults or GraphDefaults(), namespace=namespace)
|
||||
pending_refs: list[str] = []
|
||||
|
||||
for spec in specs:
|
||||
if isinstance(spec, str):
|
||||
pending_refs.append(spec)
|
||||
@@ -235,6 +235,38 @@ class Graph:
|
||||
graph.validate()
|
||||
return graph
|
||||
|
||||
@classmethod
|
||||
def from_yaml(
|
||||
cls,
|
||||
path: str | Path,
|
||||
variables: Mapping[str, Any] | None = None,
|
||||
) -> Graph:
|
||||
"""从 YAML 文件构建任务图。
|
||||
|
||||
参考 GitHub Actions 风格 schema, 支持 jobs/needs/strategy.matrix/if
|
||||
等 CI/CD 概念。详见 :mod:`pyflowx.yaml_loader`。
|
||||
|
||||
Parameters
|
||||
----------
|
||||
path : str | Path
|
||||
YAML 文件路径
|
||||
variables : Mapping[str, Any] | None
|
||||
运行时变量, 用于替换 ``${VAR}`` 占位符
|
||||
|
||||
Returns
|
||||
-------
|
||||
Graph
|
||||
构建好的任务图
|
||||
|
||||
Raises
|
||||
------
|
||||
YamlLoadError
|
||||
文件不存在、YAML 格式错误、schema 校验失败、循环依赖等
|
||||
"""
|
||||
from .yaml_loader import load_yaml
|
||||
|
||||
return load_yaml(path, variables=variables)
|
||||
|
||||
def add_subgraph(self, sub: Graph, *, namespace: str | None = None) -> Graph:
|
||||
"""将子图合并到当前图,任务名加命名空间前缀避免冲突。
|
||||
|
||||
|
||||
@@ -0,0 +1,18 @@
|
||||
"""工具函数模块.
|
||||
|
||||
按类别组织 CLI 工具中可复用的函数, 每个子模块使用 ``@px.register_fn`` 注册函数,
|
||||
供 YAML 任务编排通过 ``fn`` 字段引用.
|
||||
|
||||
子模块
|
||||
------
|
||||
- :mod:`files` —— 文件日期/等级/备份/压缩相关函数
|
||||
- :mod:`dev` —— 开发工具 (ruff/版本号/pip/git) 相关函数
|
||||
- :mod:`media` —— PDF/截图相关函数
|
||||
- :mod:`system` —— LS-DYNA/SSH/打包相关函数
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from . import dev, files, media, system
|
||||
|
||||
__all__ = ["dev", "files", "media", "system"]
|
||||
@@ -0,0 +1,618 @@
|
||||
"""开发工具类函数模块.
|
||||
|
||||
聚合自动格式化 (autofmt)、版本号管理 (bumpversion)、pip 包管理 (piptool)、
|
||||
git 工具 (gittool) 的可复用函数. 所有公共函数通过 ``@px.register_fn`` 注册,
|
||||
供 YAML 任务编排引用.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import ast
|
||||
import fnmatch
|
||||
import re
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
from typing import Literal
|
||||
|
||||
import pyflowx as px
|
||||
|
||||
__all__ = [
|
||||
"IGNORE_PATTERNS",
|
||||
"PACKAGE_DIR",
|
||||
"REQUIREMENTS_FILE",
|
||||
"_PROTECTED_PACKAGES",
|
||||
"BumpVersionType",
|
||||
"add_docstring",
|
||||
"auto_add_docstrings",
|
||||
"bump_file_version",
|
||||
"bump_project_version",
|
||||
"format_all",
|
||||
"format_with_ruff",
|
||||
"generate_module_docstring",
|
||||
"git_add_commit",
|
||||
"git_init_add_commit",
|
||||
"has_files",
|
||||
"init_sub_dirs",
|
||||
"lint_with_ruff",
|
||||
"not_has_git_repo",
|
||||
"pip_download",
|
||||
"pip_freeze",
|
||||
"pip_reinstall",
|
||||
"pip_uninstall",
|
||||
"sync_pyproject_config",
|
||||
]
|
||||
|
||||
# ============================================================================
|
||||
# autofmt 配置
|
||||
# ============================================================================
|
||||
|
||||
IGNORE_PATTERNS = [
|
||||
"__pycache__",
|
||||
"*.pyc",
|
||||
"*.pyo",
|
||||
".git",
|
||||
".venv",
|
||||
".idea",
|
||||
".vscode",
|
||||
"*.egg-info",
|
||||
"dist",
|
||||
"build",
|
||||
".pytest_cache",
|
||||
".tox",
|
||||
".mypy_cache",
|
||||
]
|
||||
|
||||
# ============================================================================
|
||||
# bumpversion 配置
|
||||
# ============================================================================
|
||||
|
||||
BumpVersionType = Literal["patch", "minor", "major"]
|
||||
|
||||
_PYPROJECT_VERSION_PATTERN = re.compile(
|
||||
r'(?:^|\n)\s*version\s*=\s*["\']'
|
||||
r"(?P<major>0|[1-9]\d*)\.(?P<minor>0|[1-9]\d*)\.(?P<patch>0|[1-9]\d*)"
|
||||
r"(?:-(?P<prerelease>(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)(?:\.(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*))?"
|
||||
r"(?:\+(?P<buildmetadata>[0-9a-zA-Z-]+(?:\.[0-9a-zA-Z-]+)*))?"
|
||||
r'["\']',
|
||||
re.MULTILINE,
|
||||
)
|
||||
|
||||
_INIT_VERSION_PATTERN = re.compile(
|
||||
r'(?:^|\n)\s*__version__\s*=\s*["\']'
|
||||
r"(?P<major>0|[1-9]\d*)\.(?P<minor>0|[1-9]\d*)\.(?P<patch>0|[1-9]\d*)"
|
||||
r"(?:-(?P<prerelease>(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)(?:\.(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*))?"
|
||||
r"(?:\+(?P<buildmetadata>[0-9a-zA-Z-]+(?:\.[0-9a-zA-Z-]+)*))?"
|
||||
r'["\']',
|
||||
re.MULTILINE,
|
||||
)
|
||||
|
||||
# ============================================================================
|
||||
# piptool 配置
|
||||
# ============================================================================
|
||||
|
||||
PACKAGE_DIR = "packages"
|
||||
REQUIREMENTS_FILE = "requirements.txt"
|
||||
|
||||
_PROTECTED_PACKAGES: frozenset[str] = frozenset(
|
||||
{
|
||||
"pyflowx",
|
||||
"bitool",
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# autofmt 私有辅助函数
|
||||
# ============================================================================
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# autofmt 函数
|
||||
# ============================================================================
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def format_with_ruff(target: Path, fix: bool = True) -> None:
|
||||
"""使用 ruff 格式化代码.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
target : Path
|
||||
目标路径
|
||||
fix : bool
|
||||
是否自动修复
|
||||
"""
|
||||
cmd = ["ruff", "format", str(target)]
|
||||
if fix:
|
||||
cmd.append("--fix")
|
||||
|
||||
subprocess.run(cmd, check=True)
|
||||
print(f"ruff format 完成: {target}")
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def lint_with_ruff(target: Path, fix: bool = True) -> None:
|
||||
"""使用 ruff 检查代码.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
target : Path
|
||||
目标路径
|
||||
fix : bool
|
||||
是否自动修复
|
||||
"""
|
||||
cmd = ["ruff", "check", str(target)]
|
||||
if fix:
|
||||
cmd.extend(["--fix", "--unsafe-fixes"])
|
||||
|
||||
subprocess.run(cmd, check=True)
|
||||
print(f"ruff check 完成: {target}")
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def add_docstring(file_path: Path, docstring: str) -> bool:
|
||||
"""为文件添加 docstring.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
file_path : Path
|
||||
文件路径
|
||||
docstring : str
|
||||
docstring 内容
|
||||
|
||||
Returns
|
||||
-------
|
||||
bool
|
||||
是否成功添加
|
||||
"""
|
||||
try:
|
||||
content = file_path.read_text(encoding="utf-8")
|
||||
tree = ast.parse(content)
|
||||
|
||||
first_node = tree.body[0] if tree.body else None
|
||||
if first_node and isinstance(first_node, ast.Expr) and isinstance(first_node.value, ast.Constant):
|
||||
return False
|
||||
|
||||
lines = content.splitlines()
|
||||
doc_lines = docstring.splitlines()
|
||||
doc_lines.append("")
|
||||
new_content = "\n".join(doc_lines + lines)
|
||||
|
||||
file_path.write_text(new_content, encoding="utf-8")
|
||||
print(f"添加 docstring: {file_path}")
|
||||
return True
|
||||
|
||||
except (OSError, UnicodeDecodeError, SyntaxError) as e:
|
||||
print(f"处理失败: {file_path} - {e}")
|
||||
return False
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def generate_module_docstring(file_path: Path) -> str:
|
||||
"""生成模块 docstring.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
file_path : Path
|
||||
文件路径
|
||||
|
||||
Returns
|
||||
-------
|
||||
str
|
||||
生成的 docstring
|
||||
"""
|
||||
stem = file_path.stem
|
||||
parent = file_path.parent.name
|
||||
|
||||
keywords = {
|
||||
"cli": f"Command-line interface for {parent}",
|
||||
"gui": f"Graphical user interface for {parent}",
|
||||
"core": f"Core functionality for {parent}",
|
||||
"util": f"Utility functions for {parent}",
|
||||
"model": f"Data models for {parent}",
|
||||
"test": f"Tests for {parent}",
|
||||
}
|
||||
|
||||
for key, desc in keywords.items():
|
||||
if key in stem.lower():
|
||||
return f'"""{desc}."""'
|
||||
|
||||
return f'"""{stem.replace("_", " ").title()} module."""'
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def auto_add_docstrings(root_dir: Path) -> int:
|
||||
"""自动为所有 Python 文件添加 docstring.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
root_dir : Path
|
||||
根目录
|
||||
|
||||
Returns
|
||||
-------
|
||||
int
|
||||
添加的 docstring 数量
|
||||
"""
|
||||
count = 0
|
||||
for py_file in root_dir.rglob("*.py"):
|
||||
if any(pattern in str(py_file) for pattern in IGNORE_PATTERNS):
|
||||
continue
|
||||
|
||||
docstring = generate_module_docstring(py_file)
|
||||
if add_docstring(py_file, docstring):
|
||||
count += 1
|
||||
|
||||
print(f"共添加 {count} 个 docstring")
|
||||
return count
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def sync_pyproject_config(root_dir: Path) -> None:
|
||||
"""同步 pyproject.toml 配置到子项目.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
root_dir : Path
|
||||
根目录
|
||||
"""
|
||||
main_toml = root_dir / "pyproject.toml"
|
||||
if not main_toml.exists():
|
||||
print(f"主项目配置文件不存在: {main_toml}")
|
||||
return
|
||||
|
||||
sub_tomls = [p for p in root_dir.rglob("pyproject.toml") if p != main_toml and ".venv" not in str(p)]
|
||||
|
||||
if not sub_tomls:
|
||||
print("没有找到子项目的 pyproject.toml")
|
||||
return
|
||||
|
||||
print(f"找到 {len(sub_tomls)} 个子项目配置文件")
|
||||
|
||||
for sub_toml in sub_tomls:
|
||||
subprocess.run(["ruff", "format", str(sub_toml)], check=False)
|
||||
|
||||
print("配置同步完成")
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def format_all(root_dir: Path) -> None:
|
||||
"""格式化所有 Python 文件.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
root_dir : Path
|
||||
根目录
|
||||
"""
|
||||
subprocess.run(["ruff", "format", str(root_dir)], check=True)
|
||||
subprocess.run(["ruff", "check", "--fix", "--unsafe-fixes", str(root_dir)], check=True)
|
||||
print(f"格式化完成: {root_dir}")
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# bumpversion 私有辅助函数
|
||||
# ============================================================================
|
||||
|
||||
|
||||
def _get_pattern_for_file(file_name: str) -> re.Pattern[str] | None:
|
||||
"""根据文件类型获取对应的正则表达式."""
|
||||
if file_name == "pyproject.toml":
|
||||
return _PYPROJECT_VERSION_PATTERN
|
||||
if file_name == "__init__.py":
|
||||
return _INIT_VERSION_PATTERN
|
||||
return None
|
||||
|
||||
|
||||
def _calculate_new_version(major: int, minor: int, patch: int, part: BumpVersionType) -> str:
|
||||
"""计算新版本号."""
|
||||
if part == "major":
|
||||
return f"{major + 1}.0.0"
|
||||
if part == "minor":
|
||||
return f"{major}.{minor + 1}.0"
|
||||
return f"{major}.{minor}.{patch + 1}"
|
||||
|
||||
|
||||
def _build_replacement_string(original_match: str, new_version: str, file_name: str) -> str:
|
||||
"""构建替换字符串, 保留原始格式."""
|
||||
quote_char = '"' if '"' in original_match else "'"
|
||||
|
||||
if file_name == "pyproject.toml":
|
||||
prefix_match = re.match(r'(\s*version\s*=\s*)["\']', original_match)
|
||||
prefix = prefix_match.group(1) if prefix_match else "version = "
|
||||
return f"{prefix}{quote_char}{new_version}{quote_char}"
|
||||
|
||||
if file_name == "__init__.py":
|
||||
prefix_match = re.match(r'(\s*__version__\s*=\s*)["\']', original_match)
|
||||
prefix = prefix_match.group(1) if prefix_match else "__version__ = "
|
||||
return f"{prefix}{quote_char}{new_version}{quote_char}"
|
||||
|
||||
return new_version
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# bumpversion 函数
|
||||
# ============================================================================
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def bump_file_version(file_path: Path, part: BumpVersionType = "patch") -> str | None:
|
||||
"""更新文件中的版本号.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
file_path : Path
|
||||
要更新的文件路径
|
||||
part : BumpVersionType
|
||||
版本部分: patch, minor, major
|
||||
|
||||
Returns
|
||||
-------
|
||||
str | None
|
||||
更新后的新版本号, 如果文件中未找到版本号则返回 None
|
||||
"""
|
||||
try:
|
||||
content = file_path.read_text(encoding="utf-8")
|
||||
except Exception as e:
|
||||
print(f"读取文件 {file_path} 时出错: {e}")
|
||||
raise
|
||||
|
||||
pattern = _get_pattern_for_file(file_path.name)
|
||||
|
||||
if pattern:
|
||||
match = pattern.search(content)
|
||||
else:
|
||||
match = _PYPROJECT_VERSION_PATTERN.search(content) or _INIT_VERSION_PATTERN.search(content)
|
||||
|
||||
if not match:
|
||||
print(f"文件 {file_path} 中未找到版本号模式")
|
||||
return None
|
||||
|
||||
major = int(match.group("major"))
|
||||
minor = int(match.group("minor"))
|
||||
patch = int(match.group("patch"))
|
||||
|
||||
new_version = _calculate_new_version(major, minor, patch, part)
|
||||
|
||||
original_match = match.group(0)
|
||||
replacement = _build_replacement_string(original_match, new_version, file_path.name)
|
||||
|
||||
content = content.replace(original_match, replacement)
|
||||
|
||||
try:
|
||||
file_path.write_text(content, encoding="utf-8")
|
||||
except Exception as e:
|
||||
print(f"更新文件 {file_path} 版本号时出错: {e}")
|
||||
raise
|
||||
|
||||
return new_version
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def bump_project_version(part: BumpVersionType = "patch", no_tag: bool = False) -> str | None:
|
||||
"""批量更新项目所有文件的版本号并提交.
|
||||
|
||||
搜索当前目录下所有 ``__init__.py`` 和 ``pyproject.toml`` 文件
|
||||
(排除虚拟环境和缓存目录), 更新版本号, 然后执行 git add + commit + tag.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
part : BumpVersionType
|
||||
版本部分: patch, minor, major
|
||||
no_tag : bool
|
||||
提交后不创建 git tag
|
||||
|
||||
Returns
|
||||
-------
|
||||
str | None
|
||||
更新后的新版本号, 如果未找到版本号文件则返回 None
|
||||
"""
|
||||
ignore_dirs = {".venv", "venv", ".git", "__pycache__", ".tox", "node_modules", "build", "dist", ".eggs"}
|
||||
all_files: set[Path] = set()
|
||||
|
||||
for pattern in ["__init__.py", "pyproject.toml"]:
|
||||
for file in Path.cwd().rglob(pattern):
|
||||
if not any(ignore_dir in file.parts for ignore_dir in ignore_dirs):
|
||||
all_files.add(file)
|
||||
|
||||
if not all_files:
|
||||
print("未找到包含版本号的文件")
|
||||
return None
|
||||
|
||||
print(f"找到 {len(all_files)} 个文件需要更新版本号")
|
||||
for file in sorted(all_files):
|
||||
print(f" - {file.relative_to(Path.cwd())}")
|
||||
|
||||
new_version: str | None = None
|
||||
for file in sorted(all_files):
|
||||
version = bump_file_version(file, part)
|
||||
if version is not None and new_version is None:
|
||||
new_version = version
|
||||
|
||||
if not new_version:
|
||||
print("未能获取新版本号")
|
||||
return None
|
||||
|
||||
print(f"版本号已更新为: {new_version}")
|
||||
|
||||
subprocess.run(["git", "add", "."], check=False)
|
||||
subprocess.run(["git", "commit", "-m", f"bump version to {new_version}"], check=False)
|
||||
|
||||
if not no_tag:
|
||||
tag_name = f"v{new_version}"
|
||||
subprocess.run(["git", "tag", "-a", tag_name, "-m", f"Release {tag_name}"], check=False)
|
||||
print(f"已创建标签: {tag_name}")
|
||||
|
||||
return new_version
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# piptool 私有辅助函数
|
||||
# ============================================================================
|
||||
|
||||
|
||||
def _get_installed_packages() -> list[str]:
|
||||
"""获取当前环境中所有已安装的包名."""
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["pip", "list", "--format=freeze"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=True,
|
||||
)
|
||||
packages: list[str] = []
|
||||
for line in result.stdout.strip().split("\n"):
|
||||
if line and "==" in line:
|
||||
pkg_name = line.split("==")[0].strip()
|
||||
packages.append(pkg_name)
|
||||
except (subprocess.SubprocessError, OSError):
|
||||
return []
|
||||
return packages
|
||||
|
||||
|
||||
def _expand_wildcard_packages(pattern: str) -> list[str]:
|
||||
"""展开通配符模式为实际的包名列表."""
|
||||
if not any(char in pattern for char in ["*", "?", "[", "]"]):
|
||||
return [pattern]
|
||||
|
||||
installed_packages = _get_installed_packages()
|
||||
matched = [pkg for pkg in installed_packages if fnmatch.fnmatchcase(pkg.lower(), pattern.lower())]
|
||||
return matched
|
||||
|
||||
|
||||
def _filter_protected_packages(packages: list[str]) -> list[str]:
|
||||
"""过滤掉受保护的包名."""
|
||||
safe = [p for p in packages if p.lower() not in {p.lower() for p in _PROTECTED_PACKAGES}]
|
||||
filtered = [p for p in packages if p.lower() in {p.lower() for p in _PROTECTED_PACKAGES}]
|
||||
if filtered:
|
||||
print(f"跳过受保护的包: {', '.join(filtered)}")
|
||||
return safe
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# piptool 函数
|
||||
# ============================================================================
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def pip_uninstall(pkg_names: list[str]) -> None:
|
||||
"""卸载包."""
|
||||
packages_to_uninstall: list[str] = []
|
||||
for pattern in pkg_names:
|
||||
packages_to_uninstall.extend(_expand_wildcard_packages(pattern))
|
||||
|
||||
packages_to_uninstall = _filter_protected_packages(packages_to_uninstall)
|
||||
|
||||
if not packages_to_uninstall:
|
||||
return
|
||||
|
||||
subprocess.run(["pip", "uninstall", "-y", *packages_to_uninstall], check=True)
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def pip_reinstall(pkg_names: list[str], offline: bool = False) -> None:
|
||||
"""重新安装包."""
|
||||
safe_ps = _filter_protected_packages(pkg_names)
|
||||
if not safe_ps:
|
||||
print("所有指定的包均为受保护包, 跳过重装")
|
||||
return
|
||||
|
||||
subprocess.run(["pip", "uninstall", "-y", *safe_ps], check=True)
|
||||
|
||||
options = ["--no-index", "--find-links", "."] if offline else []
|
||||
subprocess.run(["pip", "install", *options, *safe_ps], check=True)
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def pip_download(pkg_names: list[str], offline: bool = False) -> None:
|
||||
"""下载包."""
|
||||
options = ["--no-index", "--find-links", "."] if offline else []
|
||||
subprocess.run(
|
||||
["pip", "download", *pkg_names, *options, "-d", PACKAGE_DIR],
|
||||
check=True,
|
||||
)
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def pip_freeze() -> None:
|
||||
"""冻结依赖."""
|
||||
result = subprocess.run(
|
||||
["pip", "freeze", "--exclude-editable"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=True,
|
||||
)
|
||||
Path(REQUIREMENTS_FILE).write_text(result.stdout)
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# gittool 函数
|
||||
# ============================================================================
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def init_sub_dirs() -> None:
|
||||
"""初始化子目录的 Git 仓库."""
|
||||
sub_dirs = [subdir for subdir in Path.cwd().iterdir() if subdir.is_dir()]
|
||||
for subdir in sub_dirs:
|
||||
px.run(
|
||||
px.Graph().chain(
|
||||
px.cmd(["git", "init"], conditions=(lambda _: not_has_git_repo(),), cwd=subdir),
|
||||
px.cmd(["git", "add", "."]),
|
||||
px.cmd(["git", "commit", "-m", "init commit"]),
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def not_has_git_repo() -> bool:
|
||||
"""检查当前目录没有 Git 仓库."""
|
||||
return not Path.cwd().exists() or not (Path.cwd() / ".git").is_dir()
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def has_files() -> bool:
|
||||
"""检查当前 Git 仓库是否有未提交的更改."""
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["git", "status", "--porcelain"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
return bool(result.stdout.strip())
|
||||
except (subprocess.SubprocessError, OSError):
|
||||
return False
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def git_add_commit(message: str = "chore: update") -> None:
|
||||
"""执行 git add + git commit (仅当有未提交更改时).
|
||||
|
||||
Parameters
|
||||
----------
|
||||
message : str
|
||||
提交信息
|
||||
"""
|
||||
if not has_files():
|
||||
print("没有文件需要提交")
|
||||
return
|
||||
subprocess.run(["git", "add", "."], check=True)
|
||||
subprocess.run(["git", "commit", "-m", message], check=True)
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def git_init_add_commit(message: str = "init commit") -> None:
|
||||
"""执行 git init (若需) + git add + git commit (若有更改).
|
||||
|
||||
Parameters
|
||||
----------
|
||||
message : str
|
||||
提交信息
|
||||
"""
|
||||
if not_has_git_repo():
|
||||
subprocess.run(["git", "init"], check=True)
|
||||
if has_files():
|
||||
subprocess.run(["git", "add", "."], check=True)
|
||||
subprocess.run(["git", "commit", "-m", message], check=True)
|
||||
else:
|
||||
print("没有文件需要提交")
|
||||
@@ -0,0 +1,327 @@
|
||||
"""文件类函数模块.
|
||||
|
||||
聚合文件日期处理、文件等级重命名、文件夹备份、文件夹压缩工具的可复用函数.
|
||||
所有公共函数通过 ``@px.register_fn`` 注册, 供 YAML 任务编排引用.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
import shutil
|
||||
import time
|
||||
import zipfile
|
||||
from pathlib import Path
|
||||
|
||||
import pyflowx as px
|
||||
|
||||
__all__ = [
|
||||
"BRACKETS",
|
||||
"DATE_PATTERN",
|
||||
"IGNORE_DIRS",
|
||||
"IGNORE_EXT",
|
||||
"IGNORE_FILES",
|
||||
"LEVELS",
|
||||
"SEP",
|
||||
"add_date_prefix",
|
||||
"archive_folder",
|
||||
"backup_folder",
|
||||
"folderback_default",
|
||||
"folderzip_default",
|
||||
"get_file_timestamp",
|
||||
"process_file_date",
|
||||
"process_file_level",
|
||||
"process_files_date",
|
||||
"process_files_level",
|
||||
"remove_date_prefix",
|
||||
"remove_dump",
|
||||
"remove_marks",
|
||||
"zip_folders",
|
||||
"zip_target",
|
||||
]
|
||||
|
||||
# ============================================================================
|
||||
# filedate 配置
|
||||
# ============================================================================
|
||||
|
||||
DATE_PATTERN = re.compile(r"(20|19)\d{2}[-_#.~]?((0[1-9])|(1[012]))[-_#.~]?((0[1-9])|([12]\d)|(3[01]))[-_#.~]?")
|
||||
SEP = "_"
|
||||
|
||||
# ============================================================================
|
||||
# filelevel 配置
|
||||
# ============================================================================
|
||||
|
||||
LEVELS: dict[str, str] = {
|
||||
"0": "",
|
||||
"1": "PUB,NOR",
|
||||
"2": "INT",
|
||||
"3": "CON",
|
||||
"4": "CLA",
|
||||
}
|
||||
|
||||
BRACKETS: tuple[str, str] = (" ([_(【-", " )]_)】")
|
||||
|
||||
# ============================================================================
|
||||
# folderzip 配置
|
||||
# ============================================================================
|
||||
|
||||
IGNORE_DIRS: list[str] = [".git", ".idea", ".vscode", "__pycache__"]
|
||||
IGNORE_FILES: list[str] = [".gitignore"]
|
||||
IGNORE: list[str] = [*IGNORE_DIRS, *IGNORE_FILES]
|
||||
IGNORE_EXT: list[str] = [".zip", ".rar", ".7z", ".tar", ".gz"]
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# filedate 函数
|
||||
# ============================================================================
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def get_file_timestamp(filepath: Path) -> str:
|
||||
"""获取文件时间戳."""
|
||||
modified_time = filepath.stat().st_mtime
|
||||
created_time = filepath.stat().st_ctime
|
||||
return time.strftime("%Y%m%d", time.localtime(max((modified_time, created_time))))
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def remove_date_prefix(filepath: Path) -> Path:
|
||||
"""移除文件日期前缀."""
|
||||
stem = filepath.stem
|
||||
new_stem = DATE_PATTERN.sub("", stem)
|
||||
if new_stem != stem:
|
||||
new_path = filepath.with_name(new_stem + filepath.suffix)
|
||||
filepath.rename(new_path)
|
||||
return new_path
|
||||
return filepath
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def add_date_prefix(filepath: Path) -> Path:
|
||||
"""添加文件日期前缀."""
|
||||
timestamp = get_file_timestamp(filepath)
|
||||
stem = filepath.stem
|
||||
new_stem = f"{timestamp}{SEP}{stem}"
|
||||
new_path = filepath.with_name(new_stem + filepath.suffix)
|
||||
if new_path != filepath:
|
||||
filepath.rename(new_path)
|
||||
return new_path
|
||||
return filepath
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def process_file_date(filepath: Path, clear: bool = False) -> None:
|
||||
"""处理单个文件的日期前缀.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
filepath : Path
|
||||
文件路径
|
||||
clear : bool
|
||||
是否清除日期前缀
|
||||
"""
|
||||
if clear:
|
||||
remove_date_prefix(filepath)
|
||||
else:
|
||||
new_path = remove_date_prefix(filepath)
|
||||
add_date_prefix(new_path)
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def process_files_date(targets: list[Path], clear: bool = False) -> None:
|
||||
"""批量处理文件日期前缀.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
targets : list[Path]
|
||||
文件路径列表
|
||||
clear : bool
|
||||
是否清除日期前缀
|
||||
"""
|
||||
for target in targets:
|
||||
if target.exists() and not target.name.startswith("."):
|
||||
process_file_date(target, clear)
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# filelevel 函数
|
||||
# ============================================================================
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def remove_marks(stem: str, marks: list[str]) -> str:
|
||||
"""从文件名主干中移除所有标记."""
|
||||
left_brackets, right_brackets = BRACKETS
|
||||
for mark in marks:
|
||||
pos = 0
|
||||
while True:
|
||||
pos = stem.find(mark, pos)
|
||||
if pos == -1:
|
||||
break
|
||||
b, e = pos - 1, pos + len(mark)
|
||||
if b >= 0 and e < len(stem) and stem[b] in left_brackets and stem[e] in right_brackets:
|
||||
stem = stem[:b] + stem[e + 1 :]
|
||||
else:
|
||||
pos = e
|
||||
return stem
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def process_file_level(filepath: Path, level: int = 0) -> None:
|
||||
"""处理单个文件的等级标记.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
filepath : Path
|
||||
文件路径
|
||||
level : int
|
||||
文件等级 (0-4), 0 用于清除等级
|
||||
"""
|
||||
if not (0 <= level < len(LEVELS)):
|
||||
print(f"无效的等级 {level}, 必须在 0 和 {len(LEVELS) - 1} 之间")
|
||||
return
|
||||
|
||||
if not filepath.exists():
|
||||
print(f"文件不存在: {filepath}")
|
||||
return
|
||||
|
||||
filestem = filepath.stem
|
||||
original_stem = filestem
|
||||
|
||||
for level_names in LEVELS.values():
|
||||
if level_names:
|
||||
filestem = remove_marks(filestem, level_names.split(","))
|
||||
|
||||
for digit in map(str, range(1, 10)):
|
||||
filestem = remove_marks(filestem, [digit])
|
||||
|
||||
if level > 0:
|
||||
levelstr = LEVELS.get(str(level), "").split(",")[0]
|
||||
if levelstr:
|
||||
filestem = f"{filestem}({levelstr})"
|
||||
|
||||
if filestem != original_stem:
|
||||
new_path = filepath.with_name(filestem + filepath.suffix)
|
||||
filepath.rename(new_path)
|
||||
print(f"重命名: {filepath} -> {new_path}")
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def process_files_level(targets: list[Path], level: int = 0) -> None:
|
||||
"""批量处理文件等级标记.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
targets : list[Path]
|
||||
文件路径列表
|
||||
level : int
|
||||
文件等级 (0-4)
|
||||
"""
|
||||
for target in targets:
|
||||
process_file_level(target, level)
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# folderback 函数
|
||||
# ============================================================================
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def remove_dump(src: Path, dst: Path, max_zip: int) -> None:
|
||||
"""递归删除旧的备份 zip 文件."""
|
||||
zip_paths = [filepath for filepath in dst.rglob("*.zip") if src.stem in str(filepath)]
|
||||
zip_files = sorted(zip_paths, key=lambda fn: str(fn)[-19:-4])
|
||||
if len(zip_files) > max_zip:
|
||||
zip_files[0].unlink()
|
||||
remove_dump(src, dst, max_zip)
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def zip_target(src: Path, dst: Path, max_zip: int) -> None:
|
||||
"""将单个文件或文件夹压缩为 zip 文件."""
|
||||
files = [str(_) for _ in src.rglob("*")]
|
||||
timestamp = time.strftime("_%Y%m%d_%H%M%S")
|
||||
target_path = dst / (src.stem + timestamp + ".zip")
|
||||
|
||||
with zipfile.ZipFile(target_path, "w") as zip_file:
|
||||
for file in files:
|
||||
zip_file.write(file, arcname=file.replace(str(src.parent), ""))
|
||||
|
||||
remove_dump(src, dst, max_zip)
|
||||
print(f"备份完成: {target_path}")
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def backup_folder(src: str, dst: str, max_zip: int = 5) -> None:
|
||||
"""备份文件夹.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
src : str
|
||||
源文件夹路径
|
||||
dst : str
|
||||
目标文件夹路径
|
||||
max_zip : int
|
||||
最大备份数量
|
||||
"""
|
||||
src_path = Path(src)
|
||||
dst_path = Path(dst)
|
||||
|
||||
if not src_path.exists():
|
||||
print(f"源文件夹不存在: {src_path}")
|
||||
return
|
||||
|
||||
if not dst_path.exists():
|
||||
dst_path.mkdir(parents=True, exist_ok=True)
|
||||
print(f"创建目标文件夹: {dst_path}")
|
||||
|
||||
zip_target(src_path, dst_path, max_zip)
|
||||
|
||||
|
||||
@px.register_fn("folderback_default")
|
||||
def folderback_default() -> None:
|
||||
"""备份当前目录到 ./backup."""
|
||||
backup_folder(".", "./backup", 5)
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# folderzip 函数
|
||||
# ============================================================================
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def archive_folder(folder: Path) -> None:
|
||||
"""压缩单个文件夹."""
|
||||
shutil.make_archive(
|
||||
str(folder.with_name(folder.name)),
|
||||
format="zip",
|
||||
base_dir=folder,
|
||||
)
|
||||
print(f"压缩完成: {folder.name}.zip")
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def zip_folders(cwd: str = ".") -> None:
|
||||
"""压缩目录下的所有文件夹.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
cwd : str
|
||||
工作目录
|
||||
"""
|
||||
cwd_path = Path(cwd)
|
||||
if not cwd_path.exists():
|
||||
print(f"目录不存在: {cwd_path}")
|
||||
return
|
||||
|
||||
dirs: list[Path] = [
|
||||
e for e in cwd_path.iterdir() if e.is_dir() and e.name not in IGNORE_DIRS and e.suffix not in IGNORE_EXT
|
||||
]
|
||||
|
||||
for dir_path in dirs:
|
||||
archive_folder(dir_path)
|
||||
|
||||
|
||||
@px.register_fn("folderzip_default")
|
||||
def folderzip_default() -> None:
|
||||
"""压缩当前目录下的所有文件夹."""
|
||||
zip_folders(".")
|
||||
@@ -1,15 +1,41 @@
|
||||
"""PDF 工具模块.
|
||||
"""媒体类函数模块.
|
||||
|
||||
提供 PDF 文件操作的常用功能封装,
|
||||
支持合并、拆分、压缩、加密、水印、OCR等功能.
|
||||
聚合 PDF 工具 (pdftool) 和截图工具 (screenshot) 的可复用函数.
|
||||
所有公共函数通过 ``@px.register_fn`` 注册, 供 YAML 任务编排引用.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import subprocess
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
|
||||
import pyflowx as px
|
||||
from pyflowx.conditions import Constants
|
||||
|
||||
__all__ = [
|
||||
"DEFAULT_PASSWORD",
|
||||
"DEFAULT_QUALITY",
|
||||
"PDF_SUFFIX",
|
||||
"get_screenshot_path",
|
||||
"pdf_add_watermark",
|
||||
"pdf_compress",
|
||||
"pdf_crop",
|
||||
"pdf_decrypt",
|
||||
"pdf_encrypt",
|
||||
"pdf_extract_images",
|
||||
"pdf_extract_text",
|
||||
"pdf_info",
|
||||
"pdf_merge",
|
||||
"pdf_ocr",
|
||||
"pdf_reorder",
|
||||
"pdf_repair",
|
||||
"pdf_rotate",
|
||||
"pdf_split",
|
||||
"pdf_to_images",
|
||||
"take_screenshot_area",
|
||||
"take_screenshot_full",
|
||||
]
|
||||
|
||||
try:
|
||||
import fitz # PyMuPDF
|
||||
@@ -36,14 +62,15 @@ DEFAULT_PASSWORD = ""
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# 辅助函数
|
||||
# PDF 函数
|
||||
# ============================================================================
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def pdf_merge(input_paths: list[Path], output_path: Path) -> None:
|
||||
"""合并多个 PDF 文件."""
|
||||
if not HAS_PYPDF:
|
||||
print("未安装 pypdf 库,请安装: pip install pypdf")
|
||||
print("未安装 pypdf 库, 请安装: pip install pypdf")
|
||||
return
|
||||
|
||||
writer = pypdf.PdfWriter()
|
||||
@@ -60,10 +87,11 @@ def pdf_merge(input_paths: list[Path], output_path: Path) -> None:
|
||||
print(f"合并完成: {output_path}")
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def pdf_split(input_path: Path, output_dir: Path) -> None:
|
||||
"""拆分 PDF 文件为单页."""
|
||||
if not HAS_PYPDF:
|
||||
print("未安装 pypdf 库,请安装: pip install pypdf")
|
||||
print("未安装 pypdf 库, 请安装: pip install pypdf")
|
||||
return
|
||||
|
||||
reader = pypdf.PdfReader(str(input_path))
|
||||
@@ -79,10 +107,11 @@ def pdf_split(input_path: Path, output_dir: Path) -> None:
|
||||
print(f"拆分完成: {output_dir}")
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def pdf_compress(input_path: Path, output_path: Path) -> None:
|
||||
"""压缩 PDF 文件."""
|
||||
if not HAS_PYMUPDF:
|
||||
print("未安装 PyMuPDF 库,请安装: pip install PyMuPDF")
|
||||
print("未安装 PyMuPDF 库, 请安装: pip install PyMuPDF")
|
||||
return
|
||||
|
||||
doc = fitz.open(str(input_path))
|
||||
@@ -96,10 +125,11 @@ def pdf_compress(input_path: Path, output_path: Path) -> None:
|
||||
print(f"压缩完成: {output_path} (缩小 {ratio:.1f}%)")
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def pdf_encrypt(input_path: Path, output_path: Path, password: str) -> None:
|
||||
"""加密 PDF 文件."""
|
||||
if not HAS_PYPDF:
|
||||
print("未安装 pypdf 库,请安装: pip install pypdf")
|
||||
print("未安装 pypdf 库, 请安装: pip install pypdf")
|
||||
return
|
||||
|
||||
reader = pypdf.PdfReader(str(input_path))
|
||||
@@ -116,10 +146,11 @@ def pdf_encrypt(input_path: Path, output_path: Path, password: str) -> None:
|
||||
print(f"加密完成: {output_path}")
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def pdf_decrypt(input_path: Path, output_path: Path, password: str) -> None:
|
||||
"""解密 PDF 文件."""
|
||||
if not HAS_PYPDF:
|
||||
print("未安装 pypdf 库,请安装: pip install pypdf")
|
||||
print("未安装 pypdf 库, 请安装: pip install pypdf")
|
||||
return
|
||||
|
||||
reader = pypdf.PdfReader(str(input_path))
|
||||
@@ -137,10 +168,11 @@ def pdf_decrypt(input_path: Path, output_path: Path, password: str) -> None:
|
||||
print(f"解密完成: {output_path}")
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def pdf_extract_text(input_path: Path, output_path: Path) -> None:
|
||||
"""提取 PDF 文本."""
|
||||
if not HAS_PYMUPDF:
|
||||
print("未安装 PyMuPDF 库,请安装: pip install PyMuPDF")
|
||||
print("未安装 PyMuPDF 库, 请安装: pip install PyMuPDF")
|
||||
return
|
||||
|
||||
doc = fitz.open(str(input_path))
|
||||
@@ -154,10 +186,11 @@ def pdf_extract_text(input_path: Path, output_path: Path) -> None:
|
||||
print(f"文本提取完成: {output_path}")
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def pdf_extract_images(input_path: Path, output_dir: Path) -> None:
|
||||
"""提取 PDF 图片."""
|
||||
if not HAS_PYMUPDF:
|
||||
print("未安装 PyMuPDF 库,请安装: pip install PyMuPDF")
|
||||
print("未安装 PyMuPDF 库, 请安装: pip install PyMuPDF")
|
||||
return
|
||||
|
||||
doc = fitz.open(str(input_path))
|
||||
@@ -180,10 +213,11 @@ def pdf_extract_images(input_path: Path, output_dir: Path) -> None:
|
||||
print(f"图片提取完成: {output_dir} (共 {image_count} 张)")
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def pdf_add_watermark(input_path: Path, output_path: Path, text: str = "CONFIDENTIAL") -> None:
|
||||
"""添加 PDF 水印."""
|
||||
if not HAS_PYMUPDF:
|
||||
print("未安装 PyMuPDF 库,请安装: pip install PyMuPDF")
|
||||
print("未安装 PyMuPDF 库, 请安装: pip install PyMuPDF")
|
||||
return
|
||||
|
||||
doc = fitz.open(str(input_path))
|
||||
@@ -200,10 +234,11 @@ def pdf_add_watermark(input_path: Path, output_path: Path, text: str = "CONFIDEN
|
||||
print(f"水印添加完成: {output_path}")
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def pdf_rotate(input_path: Path, output_path: Path, rotation: int = 90) -> None:
|
||||
"""旋转 PDF 页面."""
|
||||
if not HAS_PYMUPDF:
|
||||
print("未安装 PyMuPDF 库,请安装: pip install PyMuPDF")
|
||||
print("未安装 PyMuPDF 库, 请安装: pip install PyMuPDF")
|
||||
return
|
||||
|
||||
doc = fitz.open(str(input_path))
|
||||
@@ -216,10 +251,11 @@ def pdf_rotate(input_path: Path, output_path: Path, rotation: int = 90) -> None:
|
||||
print(f"旋转完成: {output_path}")
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def pdf_crop(input_path: Path, output_path: Path, margins: tuple[int, int, int, int]) -> None:
|
||||
"""裁剪 PDF 页面."""
|
||||
if not HAS_PYMUPDF:
|
||||
print("未安装 PyMuPDF 库,请安装: pip install PyMuPDF")
|
||||
print("未安装 PyMuPDF 库, 请安装: pip install PyMuPDF")
|
||||
return
|
||||
|
||||
doc = fitz.open(str(input_path))
|
||||
@@ -241,10 +277,11 @@ def pdf_crop(input_path: Path, output_path: Path, margins: tuple[int, int, int,
|
||||
print(f"裁剪完成: {output_path}")
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def pdf_info(input_path: Path) -> None:
|
||||
"""显示 PDF 信息."""
|
||||
if not HAS_PYMUPDF:
|
||||
print("未安装 PyMuPDF 库,请安装: pip install PyMuPDF")
|
||||
print("未安装 PyMuPDF 库, 请安装: pip install PyMuPDF")
|
||||
return
|
||||
|
||||
doc = fitz.open(str(input_path))
|
||||
@@ -262,17 +299,18 @@ def pdf_info(input_path: Path) -> None:
|
||||
doc.close()
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def pdf_ocr(input_path: Path, output_path: Path, lang: str = "chi_sim+eng") -> None:
|
||||
"""PDF OCR 识别."""
|
||||
try:
|
||||
import pytesseract
|
||||
from PIL import Image
|
||||
except ImportError:
|
||||
print("未安装 OCR 相关库,请安装: pip install pytesseract pillow")
|
||||
print("未安装 OCR 相关库, 请安装: pip install pytesseract pillow")
|
||||
return
|
||||
|
||||
if not HAS_PYMUPDF:
|
||||
print("未安装 PyMuPDF 库,请安装: pip install PyMuPDF")
|
||||
print("未安装 PyMuPDF 库, 请安装: pip install PyMuPDF")
|
||||
return
|
||||
|
||||
doc = fitz.open(str(input_path))
|
||||
@@ -296,10 +334,11 @@ def pdf_ocr(input_path: Path, output_path: Path, lang: str = "chi_sim+eng") -> N
|
||||
print(f"OCR 识别完成: {output_path}")
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def pdf_reorder(input_path: Path, output_path: Path, order: list[int]) -> None:
|
||||
"""重排 PDF 页面顺序."""
|
||||
if not HAS_PYPDF:
|
||||
print("未安装 pypdf 库,请安装: pip install pypdf")
|
||||
print("未安装 pypdf 库, 请安装: pip install pypdf")
|
||||
return
|
||||
|
||||
reader = pypdf.PdfReader(str(input_path))
|
||||
@@ -316,10 +355,11 @@ def pdf_reorder(input_path: Path, output_path: Path, order: list[int]) -> None:
|
||||
print(f"重排完成: {output_path}")
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def pdf_to_images(input_path: Path, output_dir: Path, dpi: int = 300) -> None:
|
||||
"""PDF 转图片."""
|
||||
if not HAS_PYMUPDF:
|
||||
print("未安装 PyMuPDF 库,请安装: pip install PyMuPDF")
|
||||
print("未安装 PyMuPDF 库, 请安装: pip install PyMuPDF")
|
||||
return
|
||||
|
||||
doc = fitz.open(str(input_path))
|
||||
@@ -335,10 +375,11 @@ def pdf_to_images(input_path: Path, output_dir: Path, dpi: int = 300) -> None:
|
||||
print(f"转换完成: {output_dir}")
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def pdf_repair(input_path: Path, output_path: Path) -> None:
|
||||
"""修复 PDF 文件."""
|
||||
if not HAS_PYMUPDF:
|
||||
print("未安装 PyMuPDF 库,请安装: pip install PyMuPDF")
|
||||
print("未安装 PyMuPDF 库, 请安装: pip install PyMuPDF")
|
||||
return
|
||||
|
||||
doc = fitz.open(str(input_path))
|
||||
@@ -349,175 +390,109 @@ def pdf_repair(input_path: Path, output_path: Path) -> None:
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# CLI Runner
|
||||
# screenshot 函数
|
||||
# ============================================================================
|
||||
|
||||
|
||||
def main() -> None: # noqa: PLR0912
|
||||
"""PDF 工具主函数."""
|
||||
parser = argparse.ArgumentParser(
|
||||
description="PDFTool - PDF 文件工具集",
|
||||
usage="pdftool <command> [options]",
|
||||
)
|
||||
subparsers = parser.add_subparsers(dest="command", help="可用命令")
|
||||
@px.register_fn
|
||||
def get_screenshot_path(filename: str | None = None) -> Path:
|
||||
"""获取截图保存路径.
|
||||
|
||||
# 合并 PDF 命令
|
||||
merge_parser = subparsers.add_parser("m", help="合并 PDF 文件")
|
||||
merge_parser.add_argument("inputs", nargs="+", help="输入 PDF 文件路径")
|
||||
merge_parser.add_argument("--output", type=str, default="merged.pdf", help="输出文件路径")
|
||||
Parameters
|
||||
----------
|
||||
filename : str | None
|
||||
文件名, 如果为 None 则自动生成
|
||||
|
||||
# 拆分 PDF 命令
|
||||
split_parser = subparsers.add_parser("s", help="拆分 PDF 文件为单页")
|
||||
split_parser.add_argument("input", help="输入 PDF 文件路径")
|
||||
split_parser.add_argument("--output-dir", type=str, default="split", help="输出目录")
|
||||
Returns
|
||||
-------
|
||||
Path
|
||||
截图保存路径
|
||||
"""
|
||||
if filename is None:
|
||||
timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
|
||||
filename = f"screenshot_{timestamp}.png"
|
||||
|
||||
# 压缩 PDF 命令
|
||||
compress_parser = subparsers.add_parser("c", help="压缩 PDF 文件")
|
||||
compress_parser.add_argument("input", help="输入 PDF 文件路径")
|
||||
compress_parser.add_argument("--output", type=str, default="compressed.pdf", help="输出文件路径")
|
||||
screenshots_dir = Path.home() / "Pictures" / "screenshots"
|
||||
screenshots_dir.mkdir(parents=True, exist_ok=True)
|
||||
return screenshots_dir / filename
|
||||
|
||||
# 加密 PDF 命令
|
||||
encrypt_parser = subparsers.add_parser("e", help="加密 PDF 文件")
|
||||
encrypt_parser.add_argument("input", help="输入 PDF 文件路径")
|
||||
encrypt_parser.add_argument("--output", type=str, default="encrypted.pdf", help="输出文件路径")
|
||||
encrypt_parser.add_argument("--password", type=str, required=True, help="密码")
|
||||
|
||||
# 解密 PDF 命令
|
||||
decrypt_parser = subparsers.add_parser("d", help="解密 PDF 文件")
|
||||
decrypt_parser.add_argument("input", help="输入 PDF 文件路径")
|
||||
decrypt_parser.add_argument("--output", type=str, default="decrypted.pdf", help="输出文件路径")
|
||||
decrypt_parser.add_argument("--password", type=str, required=True, help="密码")
|
||||
@px.register_fn
|
||||
def take_screenshot_full(filename: str | None = None) -> None:
|
||||
"""全屏截图.
|
||||
|
||||
# 提取文本命令
|
||||
extract_text_parser = subparsers.add_parser("xt", help="提取 PDF 文本")
|
||||
extract_text_parser.add_argument("input", help="输入 PDF 文件路径")
|
||||
extract_text_parser.add_argument("--output", type=str, default="output.txt", help="输出文件路径")
|
||||
Parameters
|
||||
----------
|
||||
filename : str | None
|
||||
文件名
|
||||
"""
|
||||
output_path = get_screenshot_path(filename)
|
||||
|
||||
# 提取图片命令
|
||||
extract_images_parser = subparsers.add_parser("xi", help="提取 PDF 图片")
|
||||
extract_images_parser.add_argument("input", help="输入 PDF 文件路径")
|
||||
extract_images_parser.add_argument("--output-dir", type=str, default="images", help="输出目录")
|
||||
|
||||
# 添加水印命令
|
||||
watermark_parser = subparsers.add_parser("w", help="添加 PDF 水印")
|
||||
watermark_parser.add_argument("input", help="输入 PDF 文件路径")
|
||||
watermark_parser.add_argument("--output", type=str, default="watermarked.pdf", help="输出文件路径")
|
||||
watermark_parser.add_argument("--text", type=str, default="CONFIDENTIAL", help="水印文本")
|
||||
|
||||
# 旋转 PDF 命令
|
||||
rotate_parser = subparsers.add_parser("r", help="旋转 PDF 页面")
|
||||
rotate_parser.add_argument("input", help="输入 PDF 文件路径")
|
||||
rotate_parser.add_argument("--output", type=str, default="rotated.pdf", help="输出文件路径")
|
||||
rotate_parser.add_argument("--rotation", type=int, default=90, help="旋转角度 (90, 180, 270)")
|
||||
|
||||
# 裁剪 PDF 命令
|
||||
crop_parser = subparsers.add_parser("crop", help="裁剪 PDF 页面")
|
||||
crop_parser.add_argument("input", help="输入 PDF 文件路径")
|
||||
crop_parser.add_argument("--output", type=str, default="cropped.pdf", help="输出文件路径")
|
||||
crop_parser.add_argument("--left", type=int, default=10, help="左边裁剪")
|
||||
crop_parser.add_argument("--top", type=int, default=10, help="顶部裁剪")
|
||||
crop_parser.add_argument("--right", type=int, default=10, help="右边裁剪")
|
||||
crop_parser.add_argument("--bottom", type=int, default=10, help="底部裁剪")
|
||||
|
||||
# 显示信息命令
|
||||
info_parser = subparsers.add_parser("i", help="显示 PDF 信息")
|
||||
info_parser.add_argument("input", help="输入 PDF 文件路径")
|
||||
|
||||
# OCR 识别命令
|
||||
ocr_parser = subparsers.add_parser("ocr", help="PDF OCR 识别")
|
||||
ocr_parser.add_argument("input", help="输入 PDF 文件路径")
|
||||
ocr_parser.add_argument("--output", type=str, default="ocr.pdf", help="输出文件路径")
|
||||
ocr_parser.add_argument("--lang", type=str, default="chi_sim+eng", help="OCR 语言")
|
||||
|
||||
# 转换图片命令
|
||||
to_images_parser = subparsers.add_parser("img", help="PDF 转图片")
|
||||
to_images_parser.add_argument("input", help="输入 PDF 文件路径")
|
||||
to_images_parser.add_argument("--output-dir", type=str, default="images", help="输出目录")
|
||||
to_images_parser.add_argument("--dpi", type=int, default=300, help="图片 DPI")
|
||||
|
||||
# 修复 PDF 命令
|
||||
repair_parser = subparsers.add_parser("repair", help="修复 PDF 文件")
|
||||
repair_parser.add_argument("input", help="输入 PDF 文件路径")
|
||||
repair_parser.add_argument("--output", type=str, default="repaired.pdf", help="输出文件路径")
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
if args.command == "m":
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("pdf_merge", fn=pdf_merge, args=([Path(p) for p in args.inputs], Path(args.output)))
|
||||
])
|
||||
elif args.command == "s":
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("pdf_split", fn=pdf_split, args=(Path(args.input), Path(args.output_dir)))
|
||||
])
|
||||
elif args.command == "c":
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("pdf_compress", fn=pdf_compress, args=(Path(args.input), Path(args.output)))
|
||||
])
|
||||
elif args.command == "e":
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("pdf_encrypt", fn=pdf_encrypt, args=(Path(args.input), Path(args.output), args.password))
|
||||
])
|
||||
elif args.command == "d":
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("pdf_decrypt", fn=pdf_decrypt, args=(Path(args.input), Path(args.output), args.password))
|
||||
])
|
||||
elif args.command == "xt":
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("pdf_extract_text", fn=pdf_extract_text, args=(Path(args.input), Path(args.output)))
|
||||
])
|
||||
elif args.command == "xi":
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("pdf_extract_images", fn=pdf_extract_images, args=(Path(args.input), Path(args.output_dir)))
|
||||
])
|
||||
elif args.command == "w":
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec(
|
||||
"pdf_watermark",
|
||||
fn=pdf_add_watermark,
|
||||
args=(Path(args.input), Path(args.output)),
|
||||
kwargs={"text": args.text},
|
||||
)
|
||||
])
|
||||
elif args.command == "r":
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec(
|
||||
"pdf_rotate",
|
||||
fn=pdf_rotate,
|
||||
args=(Path(args.input), Path(args.output)),
|
||||
kwargs={"rotation": args.rotation},
|
||||
)
|
||||
])
|
||||
elif args.command == "crop":
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec(
|
||||
"pdf_crop",
|
||||
fn=pdf_crop,
|
||||
args=(Path(args.input), Path(args.output)),
|
||||
kwargs={"margins": (args.left, args.top, args.right, args.bottom)},
|
||||
)
|
||||
])
|
||||
elif args.command == "i":
|
||||
graph = px.Graph.from_specs([px.TaskSpec("pdf_info", fn=pdf_info, args=(Path(args.input),))])
|
||||
elif args.command == "ocr":
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("pdf_ocr", fn=pdf_ocr, args=(Path(args.input), Path(args.output)), kwargs={"lang": args.lang})
|
||||
])
|
||||
elif args.command == "img":
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec(
|
||||
"pdf_to_images",
|
||||
fn=pdf_to_images,
|
||||
args=(Path(args.input), Path(args.output_dir)),
|
||||
kwargs={"dpi": args.dpi},
|
||||
)
|
||||
])
|
||||
elif args.command == "repair":
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("pdf_repair", fn=pdf_repair, args=(Path(args.input), Path(args.output)))
|
||||
])
|
||||
if Constants.IS_WINDOWS:
|
||||
ps_script = f"""
|
||||
Add-Type -AssemblyName System.Windows.Forms
|
||||
Add-Type -AssemblyName System.Drawing
|
||||
$screen = [System.Windows.Forms.Screen]::PrimaryScreen
|
||||
$bounds = $screen.Bounds
|
||||
$bitmap = New-Object System.Drawing.Bitmap $bounds.Width, $bounds.Height
|
||||
$graphics = [System.Drawing.Graphics]::FromImage($bitmap)
|
||||
$graphics.CopyFromScreen($bounds.Location, [System.Drawing.Point]::Empty, $bounds.Size)
|
||||
$bitmap.Save('{output_path.as_posix()}')
|
||||
$graphics.Dispose()
|
||||
$bitmap.Dispose()
|
||||
"""
|
||||
subprocess.run(["powershell", "-Command", ps_script], check=True)
|
||||
elif Constants.IS_MACOS:
|
||||
subprocess.run(["screencapture", "-x", str(output_path)], check=True)
|
||||
else:
|
||||
parser.print_help()
|
||||
return
|
||||
try:
|
||||
subprocess.run(["gnome-screenshot", "-f", str(output_path)], check=True)
|
||||
except FileNotFoundError:
|
||||
subprocess.run(["scrot", str(output_path)], check=True)
|
||||
|
||||
px.run(graph, strategy="thread")
|
||||
print(f"截图已保存: {output_path}")
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def take_screenshot_area(filename: str | None = None) -> None:
|
||||
"""区域截图.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
filename : str | None
|
||||
文件名
|
||||
"""
|
||||
output_path = get_screenshot_path(filename)
|
||||
|
||||
if Constants.IS_WINDOWS:
|
||||
ps_script = f"""
|
||||
Add-Type -AssemblyName System.Windows.Forms
|
||||
Add-Type -AssemblyName System.Drawing
|
||||
$form = New-Object System.Windows.Forms.Form
|
||||
$form.WindowState = 'Maximized'
|
||||
$form.FormBorderStyle = 'None'
|
||||
$form.BackColor = [System.Drawing.Color]::FromArgb(1, 0, 0)
|
||||
$form.Opacity = 0.5
|
||||
$form.TopMost = $true
|
||||
$form.Show()
|
||||
Start-Sleep -Milliseconds 100
|
||||
$screen = [System.Windows.Forms.Screen]::PrimaryScreen
|
||||
$bounds = $screen.Bounds
|
||||
$bitmap = New-Object System.Drawing.Bitmap $bounds.Width, $bounds.Height
|
||||
$graphics = [System.Drawing.Graphics]::FromImage($bitmap)
|
||||
$graphics.CopyFromScreen($bounds.Location, [System.Drawing.Point]::Empty, $bounds.Size)
|
||||
$form.Close()
|
||||
$bitmap.Save('{output_path.as_posix()}')
|
||||
$graphics.Dispose()
|
||||
$bitmap.Dispose()
|
||||
"""
|
||||
subprocess.run(["powershell", "-Command", ps_script], check=True)
|
||||
elif Constants.IS_MACOS:
|
||||
subprocess.run(["screencapture", "-i", str(output_path)], check=True)
|
||||
else:
|
||||
try:
|
||||
subprocess.run(["gnome-screenshot", "-a", "-f", str(output_path)], check=True)
|
||||
except FileNotFoundError:
|
||||
subprocess.run(["scrot", "-s", str(output_path)], check=True)
|
||||
|
||||
print(f"截图已保存: {output_path}")
|
||||
@@ -0,0 +1,504 @@
|
||||
"""系统类函数模块.
|
||||
|
||||
聚合 LS-DYNA 计算 (lscalc)、SSH 密钥部署 (sshcopyid)、Python 打包 (packtool)、
|
||||
重置图标缓存 (reset_icon_cache) 的可复用函数. 所有公共函数通过 ``@px.register_fn``
|
||||
注册, 供 YAML 任务编排引用.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import platform
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import urllib.request
|
||||
import zipfile
|
||||
from pathlib import Path
|
||||
|
||||
import pyflowx as px
|
||||
from pyflowx.conditions import Constants
|
||||
|
||||
__all__ = [
|
||||
"DEFAULT_BUILD_DIR",
|
||||
"DEFAULT_CACHE_DIR",
|
||||
"DEFAULT_DIST_DIR",
|
||||
"DEFAULT_INPUT_FILE",
|
||||
"DEFAULT_LIB_DIR",
|
||||
"DEFAULT_NCPU",
|
||||
"IGNORE_PATTERNS",
|
||||
"LS_DYNA_COMMANDS",
|
||||
"check_ls_dyna_status",
|
||||
"clean_build_dir",
|
||||
"create_zip_package",
|
||||
"get_ls_dyna_command",
|
||||
"install_embed_python",
|
||||
"pack_dependencies",
|
||||
"pack_source",
|
||||
"pack_wheel",
|
||||
"reset_icon_cache_run",
|
||||
"run_ls_dyna",
|
||||
"run_ls_dyna_mpi",
|
||||
"ssh_copy_id",
|
||||
]
|
||||
|
||||
# ============================================================================
|
||||
# lscalc 配置
|
||||
# ============================================================================
|
||||
|
||||
LS_DYNA_COMMANDS: dict[str, list[str]] = {
|
||||
"windows": ["ls-dyna_mpp", "i=input.k", "ncpu=4"],
|
||||
"linux": ["ls-dyna_mpp", "i=input.k", "ncpu=8"],
|
||||
"macos": ["ls-dyna_mpp", "i=input.k", "ncpu=4"],
|
||||
}
|
||||
|
||||
DEFAULT_INPUT_FILE: str = "input.k"
|
||||
DEFAULT_NCPU: int = 4
|
||||
|
||||
# ============================================================================
|
||||
# packtool 配置
|
||||
# ============================================================================
|
||||
|
||||
DEFAULT_BUILD_DIR = ".pypack"
|
||||
DEFAULT_DIST_DIR = "dist"
|
||||
DEFAULT_LIB_DIR = "libs"
|
||||
DEFAULT_CACHE_DIR = ".cache/pypack"
|
||||
|
||||
IGNORE_PATTERNS = [
|
||||
"__pycache__",
|
||||
"*.pyc",
|
||||
"*.pyo",
|
||||
".git",
|
||||
".venv",
|
||||
".idea",
|
||||
".vscode",
|
||||
"*.egg-info",
|
||||
"dist",
|
||||
"build",
|
||||
".pytest_cache",
|
||||
".tox",
|
||||
".mypy_cache",
|
||||
]
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# lscalc 函数
|
||||
# ============================================================================
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def get_ls_dyna_command(input_file: str, ncpu: int) -> list[str]:
|
||||
"""获取 LS-DYNA 命令.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
input_file : str
|
||||
输入文件路径
|
||||
ncpu : int
|
||||
CPU 核心数
|
||||
|
||||
Returns
|
||||
-------
|
||||
list[str]
|
||||
LS-DYNA 命令列表
|
||||
"""
|
||||
if Constants.IS_WINDOWS or Constants.IS_MACOS:
|
||||
return ["ls-dyna_mpp", f"i={input_file}", f"ncpu={ncpu}"]
|
||||
else:
|
||||
return ["ls-dyna_mpp", f"i={input_file}", f"ncpu={ncpu}"]
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def run_ls_dyna(input_file: str, ncpu: int = DEFAULT_NCPU) -> None:
|
||||
"""运行 LS-DYNA 计算.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
input_file : str
|
||||
输入文件路径
|
||||
ncpu : int
|
||||
CPU 核心数
|
||||
"""
|
||||
input_path = Path(input_file)
|
||||
if not input_path.exists():
|
||||
print(f"输入文件不存在: {input_path}")
|
||||
return
|
||||
|
||||
cmd = get_ls_dyna_command(input_file, ncpu)
|
||||
try:
|
||||
subprocess.run(cmd, check=True)
|
||||
print(f"LS-DYNA 计算完成: {input_file}")
|
||||
except FileNotFoundError:
|
||||
print("未找到 ls-dyna_mpp 命令")
|
||||
except subprocess.CalledProcessError as e:
|
||||
print(f"LS-DYNA 计算失败: {e}")
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def run_ls_dyna_mpi(input_file: str, ncpu: int = DEFAULT_NCPU) -> None:
|
||||
"""运行 LS-DYNA MPI 计算.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
input_file : str
|
||||
输入文件路径
|
||||
ncpu : int
|
||||
CPU 核心数
|
||||
"""
|
||||
input_path = Path(input_file)
|
||||
if not input_path.exists():
|
||||
print(f"输入文件不存在: {input_path}")
|
||||
return
|
||||
|
||||
cmd = ["mpirun", "-np", str(ncpu), "ls-dyna_mpp", f"i={input_file}"]
|
||||
try:
|
||||
subprocess.run(cmd, check=True)
|
||||
print(f"LS-DYNA MPI 计算完成: {input_file}")
|
||||
except FileNotFoundError:
|
||||
print("未找到 mpirun 或 ls-dyna_mpp 命令")
|
||||
except subprocess.CalledProcessError as e:
|
||||
print(f"LS-DYNA MPI 计算失败: {e}")
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def check_ls_dyna_status() -> None:
|
||||
"""检查 LS-DYNA 进程状态."""
|
||||
try:
|
||||
if Constants.IS_WINDOWS:
|
||||
result = subprocess.run(
|
||||
["tasklist", "/fi", "imagename eq ls-dyna_mpp.exe"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=True,
|
||||
)
|
||||
print(result.stdout)
|
||||
else:
|
||||
result = subprocess.run(
|
||||
["pgrep", "-f", "ls-dyna"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
if result.stdout.strip():
|
||||
print(f"运行中的 LS-DYNA 进程 PID: {result.stdout.strip()}")
|
||||
else:
|
||||
print("没有运行中的 LS-DYNA 进程")
|
||||
except subprocess.CalledProcessError as e:
|
||||
print(f"检查进程状态失败: {e}")
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# sshcopyid 函数
|
||||
# ============================================================================
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def ssh_copy_id(
|
||||
hostname: str,
|
||||
username: str,
|
||||
password: str,
|
||||
port: int = 22,
|
||||
keypath: str = "~/.ssh/id_rsa.pub",
|
||||
timeout: int = 30,
|
||||
) -> None:
|
||||
"""将 SSH 公钥部署到远程服务器.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
hostname : str
|
||||
远程服务器主机名或 IP 地址
|
||||
username : str
|
||||
远程服务器用户名
|
||||
password : str
|
||||
远程服务器密码
|
||||
port : int
|
||||
SSH 端口, 默认 22
|
||||
keypath : str
|
||||
公钥文件路径, 默认 ~/.ssh/id_rsa.pub
|
||||
timeout : int
|
||||
SSH 操作超时秒数, 默认 30
|
||||
"""
|
||||
pub_key_path = Path(keypath).expanduser()
|
||||
if not pub_key_path.exists():
|
||||
print(f"公钥文件不存在: {pub_key_path}")
|
||||
sys.exit(1)
|
||||
|
||||
pub_key = pub_key_path.read_text().strip()
|
||||
|
||||
script = f"""mkdir -p ~/.ssh && chmod 700 ~/.ssh
|
||||
cd ~/.ssh && touch authorized_keys && chmod 600 authorized_keys
|
||||
grep -qF '{pub_key.split()[1]}' authorized_keys 2>/dev/null || echo '{pub_key}' >> authorized_keys"""
|
||||
|
||||
try:
|
||||
subprocess.run(
|
||||
[
|
||||
"sshpass",
|
||||
"-p",
|
||||
password,
|
||||
"ssh",
|
||||
"-p",
|
||||
str(port),
|
||||
"-o",
|
||||
"StrictHostKeyChecking=no",
|
||||
"-o",
|
||||
"UserKnownHostsFile=/dev/null",
|
||||
"-o",
|
||||
f"ConnectTimeout={timeout}",
|
||||
f"{username}@{hostname}",
|
||||
script,
|
||||
],
|
||||
check=True,
|
||||
timeout=timeout,
|
||||
)
|
||||
print(f"SSH 密钥已部署到 {username}@{hostname}:{port}")
|
||||
except FileNotFoundError:
|
||||
print(f"未找到 sshpass 工具, 请手动执行: ssh-copy-id -p {port} {username}@{hostname}")
|
||||
sys.exit(1)
|
||||
except subprocess.TimeoutExpired:
|
||||
print("SSH 连接超时")
|
||||
sys.exit(1)
|
||||
except subprocess.CalledProcessError as e:
|
||||
print(f"SSH 执行失败: {e}")
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# packtool 函数
|
||||
# ============================================================================
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def pack_source(project_dir: Path, output_dir: Path) -> None:
|
||||
"""打包项目源码.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
project_dir : Path
|
||||
项目目录
|
||||
output_dir : Path
|
||||
输出目录
|
||||
"""
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
pyproject_file = project_dir / "pyproject.toml"
|
||||
project_name = project_dir.name
|
||||
|
||||
if pyproject_file.exists():
|
||||
try:
|
||||
import tomllib
|
||||
|
||||
content = pyproject_file.read_text(encoding="utf-8")
|
||||
data = tomllib.loads(content)
|
||||
project_name = data.get("project", {}).get("name", project_name)
|
||||
except ImportError:
|
||||
pass
|
||||
|
||||
source_dir = output_dir / "src" / project_name
|
||||
source_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
src_subdir = project_dir / "src"
|
||||
if src_subdir.exists():
|
||||
shutil.copytree(
|
||||
src_subdir,
|
||||
source_dir / "src",
|
||||
ignore=shutil.ignore_patterns(*IGNORE_PATTERNS),
|
||||
dirs_exist_ok=True,
|
||||
)
|
||||
else:
|
||||
for item in project_dir.iterdir():
|
||||
if item.name in IGNORE_PATTERNS or item.name.startswith("."):
|
||||
continue
|
||||
dst_item = source_dir / item.name
|
||||
if item.is_dir():
|
||||
shutil.copytree(
|
||||
item,
|
||||
dst_item,
|
||||
ignore=shutil.ignore_patterns(*IGNORE_PATTERNS),
|
||||
dirs_exist_ok=True,
|
||||
)
|
||||
else:
|
||||
shutil.copy2(item, dst_item)
|
||||
|
||||
print(f"源码打包完成: {source_dir}")
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def pack_dependencies(lib_dir: Path, dependencies: list[str]) -> None:
|
||||
"""打包项目依赖.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
lib_dir : Path
|
||||
依赖库目录
|
||||
dependencies : list[str]
|
||||
依赖列表
|
||||
"""
|
||||
lib_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
if not dependencies:
|
||||
print("没有依赖需要打包")
|
||||
return
|
||||
|
||||
cmd = [
|
||||
"pip",
|
||||
"install",
|
||||
"--target",
|
||||
str(lib_dir),
|
||||
"--no-compile",
|
||||
"--no-warn-script-location",
|
||||
]
|
||||
cmd.extend(dependencies)
|
||||
|
||||
subprocess.run(cmd, check=True)
|
||||
print(f"依赖打包完成: {lib_dir}")
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def pack_wheel(project_dir: Path, output_dir: Path) -> None:
|
||||
"""打包项目为 wheel 文件.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
project_dir : Path
|
||||
项目目录
|
||||
output_dir : Path
|
||||
输出目录
|
||||
"""
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
cmd = [
|
||||
"pip",
|
||||
"wheel",
|
||||
"--no-deps",
|
||||
"--wheel-dir",
|
||||
str(output_dir),
|
||||
str(project_dir),
|
||||
]
|
||||
|
||||
subprocess.run(cmd, check=True)
|
||||
print(f"Wheel 打包完成: {output_dir}")
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def install_embed_python(version: str, output_dir: Path) -> None:
|
||||
"""安装嵌入式 Python.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
version : str
|
||||
Python 版本 (如: 3.10, 3.11)
|
||||
output_dir : Path
|
||||
输出目录
|
||||
"""
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
arch = platform.machine().lower()
|
||||
if arch in ["x86_64", "amd64"]:
|
||||
arch = "amd64"
|
||||
elif arch in ["arm64", "aarch64"]:
|
||||
arch = "arm64"
|
||||
|
||||
version_map = {
|
||||
"3.8": "3.8.10",
|
||||
"3.9": "3.9.13",
|
||||
"3.10": "3.10.11",
|
||||
"3.11": "3.11.9",
|
||||
"3.12": "3.12.4",
|
||||
}
|
||||
full_version = version_map.get(version, f"{version}.0")
|
||||
|
||||
url = f"https://www.python.org/ftp/python/{full_version}/python-{full_version}-embed-{arch}.zip"
|
||||
|
||||
cache_file = Path(DEFAULT_CACHE_DIR) / f"python-{full_version}-embed-{arch}.zip"
|
||||
cache_file.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
if not cache_file.exists():
|
||||
print(f"正在下载嵌入式 Python {full_version}...")
|
||||
urllib.request.urlretrieve(url, cache_file)
|
||||
print(f"下载完成: {cache_file}")
|
||||
|
||||
with zipfile.ZipFile(cache_file, "r") as zf:
|
||||
zf.extractall(output_dir)
|
||||
|
||||
print(f"嵌入式 Python 安装完成: {output_dir}")
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def create_zip_package(source_dir: Path, output_file: Path) -> None:
|
||||
"""创建 ZIP 打包文件.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
source_dir : Path
|
||||
源目录
|
||||
output_file : Path
|
||||
输出文件
|
||||
"""
|
||||
output_file.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
with zipfile.ZipFile(output_file, "w", zipfile.ZIP_DEFLATED) as zf:
|
||||
for file in source_dir.rglob("*"):
|
||||
if file.is_file():
|
||||
arcname = file.relative_to(source_dir)
|
||||
zf.write(file, arcname)
|
||||
|
||||
print(f"ZIP 打包完成: {output_file}")
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def clean_build_dir(build_dir: Path) -> None:
|
||||
"""清理构建目录.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
build_dir : Path
|
||||
构建目录
|
||||
"""
|
||||
if build_dir.exists():
|
||||
shutil.rmtree(build_dir)
|
||||
print(f"清理完成: {build_dir}")
|
||||
else:
|
||||
print(f"目录不存在: {build_dir}")
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# reseticoncache 函数
|
||||
# ============================================================================
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def reset_icon_cache_run() -> None:
|
||||
"""重置 Windows 图标缓存.
|
||||
|
||||
执行流程: 杀掉 explorer → 删除 IconCache.db → 删除 iconcache* → 重启 explorer.
|
||||
仅在 Windows 上执行, 非 Windows 平台打印提示并跳过.
|
||||
"""
|
||||
if not Constants.IS_WINDOWS:
|
||||
print("reset_icon_cache: 仅在 Windows 上支持")
|
||||
return
|
||||
|
||||
local_app_data = os.environ.get("LOCALAPPDATA", "")
|
||||
if not local_app_data:
|
||||
print("reset_icon_cache: LOCALAPPDATA 环境变量未设置")
|
||||
return
|
||||
|
||||
icon_cache_db = Path(local_app_data) / "IconCache.db"
|
||||
explorer_cache_dir = Path(local_app_data) / "Microsoft" / "Windows" / "Explorer"
|
||||
|
||||
print("正在终止 explorer 进程...")
|
||||
subprocess.run(["taskkill", "/f", "/im", "explorer.exe"], check=False)
|
||||
|
||||
if icon_cache_db.exists():
|
||||
print(f"删除图标缓存: {icon_cache_db}")
|
||||
subprocess.run(["cmd", "/c", "del", "/a", "/q", str(icon_cache_db)], check=False)
|
||||
|
||||
if explorer_cache_dir.exists():
|
||||
print(f"清理 Explorer 缓存: {explorer_cache_dir}")
|
||||
subprocess.run(
|
||||
["cmd", "/c", "del", "/a", "/q", str(explorer_cache_dir / "iconcache*")],
|
||||
check=False,
|
||||
)
|
||||
|
||||
print("重启 explorer...")
|
||||
subprocess.run(["cmd", "/c", "start", "explorer.exe"], check=False)
|
||||
print("图标缓存已重置")
|
||||
@@ -0,0 +1,159 @@
|
||||
"""函数注册表.
|
||||
|
||||
提供全局函数注册机制, 供 YAML 任务编排通过 ``fn`` 字段引用 Python 函数.
|
||||
|
||||
使用方式
|
||||
--------
|
||||
import pyflowx as px
|
||||
|
||||
@px.register_fn("pack_source")
|
||||
def pack_source(project_dir, output_dir):
|
||||
...
|
||||
|
||||
# YAML 中引用:
|
||||
# jobs:
|
||||
# pack:
|
||||
# fn: pack_source
|
||||
# args: ["./project", "./dist"]
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
from typing import Any, Callable, TypeVar, overload
|
||||
|
||||
if sys.version_info >= (3, 10):
|
||||
from typing import ParamSpec
|
||||
else:
|
||||
from typing_extensions import ParamSpec # pragma: no cover
|
||||
|
||||
__all__ = ["FnRegistry", "get_fn", "has_fn", "register_fn"]
|
||||
|
||||
P = ParamSpec("P")
|
||||
T = TypeVar("T")
|
||||
|
||||
_REGISTRY: dict[str, Callable[..., Any]] = {}
|
||||
|
||||
|
||||
@overload
|
||||
def register_fn(name: Callable[P, T]) -> Callable[P, T]: ...
|
||||
|
||||
|
||||
@overload
|
||||
def register_fn(name: str | None = None) -> Callable[[Callable[P, T]], Callable[P, T]]: ...
|
||||
|
||||
|
||||
def register_fn(name: str | Callable[..., Any] | None = None) -> Callable[..., Any]:
|
||||
"""装饰器:将函数注册到全局 registry.
|
||||
|
||||
支持两种用法::
|
||||
|
||||
@register_fn # 使用函数 __name__ 作为注册名
|
||||
def my_func(): ...
|
||||
|
||||
@register_fn("custom") # 显式指定注册名
|
||||
def my_func(): ...
|
||||
|
||||
Parameters
|
||||
----------
|
||||
name : str | Callable | None
|
||||
注册名或被装饰函数; 为 None 时使用函数 ``__name__``
|
||||
|
||||
Returns
|
||||
-------
|
||||
Callable
|
||||
装饰器函数或被装饰函数
|
||||
|
||||
Raises
|
||||
------
|
||||
ValueError
|
||||
名称已注册或无法推断函数名
|
||||
"""
|
||||
if callable(name):
|
||||
fn = name
|
||||
key = getattr(fn, "__name__", None)
|
||||
if key is None:
|
||||
raise ValueError("无法推断函数名, 请显式提供 name 参数")
|
||||
if key in _REGISTRY:
|
||||
raise ValueError(f"函数 {key!r} 已注册")
|
||||
_REGISTRY[key] = fn
|
||||
return fn
|
||||
|
||||
def decorator(fn: Callable[P, T]) -> Callable[P, T]:
|
||||
key = name if name is not None else getattr(fn, "__name__", None)
|
||||
if key is None:
|
||||
raise ValueError("无法推断函数名, 请显式提供 name 参数")
|
||||
if key in _REGISTRY:
|
||||
raise ValueError(f"函数 {key!r} 已注册")
|
||||
_REGISTRY[key] = fn
|
||||
return fn
|
||||
|
||||
return decorator
|
||||
|
||||
|
||||
def get_fn(name: str) -> Callable[..., Any]:
|
||||
"""按名称获取已注册的函数.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
name : str
|
||||
函数名
|
||||
|
||||
Returns
|
||||
-------
|
||||
Callable
|
||||
已注册的函数
|
||||
|
||||
Raises
|
||||
------
|
||||
KeyError
|
||||
函数未注册
|
||||
"""
|
||||
if name not in _REGISTRY:
|
||||
raise KeyError(f"函数 {name!r} 未注册")
|
||||
return _REGISTRY[name]
|
||||
|
||||
|
||||
def has_fn(name: str) -> bool:
|
||||
"""检查函数是否已注册.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
name : str
|
||||
函数名
|
||||
|
||||
Returns
|
||||
-------
|
||||
bool
|
||||
是否已注册
|
||||
"""
|
||||
return name in _REGISTRY
|
||||
|
||||
|
||||
class FnRegistry:
|
||||
"""函数注册表的面向对象访问接口."""
|
||||
|
||||
@staticmethod
|
||||
def register(name: str | None = None) -> Callable[[Callable[..., T]], Callable[..., T]]:
|
||||
"""注册装饰器, 等价于 :func:`register_fn`."""
|
||||
return register_fn(name)
|
||||
|
||||
@staticmethod
|
||||
def get(name: str) -> Callable[..., Any]:
|
||||
"""获取已注册函数, 等价于 :func:`get_fn`."""
|
||||
return get_fn(name)
|
||||
|
||||
@staticmethod
|
||||
def has(name: str) -> bool:
|
||||
"""检查是否已注册, 等价于 :func:`has_fn`."""
|
||||
return has_fn(name)
|
||||
|
||||
@staticmethod
|
||||
def clear() -> None:
|
||||
"""清空注册表."""
|
||||
_REGISTRY.clear()
|
||||
|
||||
@staticmethod
|
||||
def names() -> list[str]:
|
||||
"""返回所有已注册函数名."""
|
||||
return list(_REGISTRY.keys())
|
||||
+2
-39
@@ -14,7 +14,7 @@ from __future__ import annotations
|
||||
import argparse
|
||||
import enum
|
||||
import sys
|
||||
from dataclasses import dataclass, field, replace
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import Any, Sequence, get_args
|
||||
|
||||
@@ -35,39 +35,6 @@ class CliExitCode(enum.IntEnum):
|
||||
INTERRUPTED = 130 # 与 POSIX 信号中断一致
|
||||
|
||||
|
||||
def _apply_verbose_to_graph(graph: Graph, verbose: bool) -> Graph:
|
||||
"""创建新图, 其中所有 TaskSpec 的 verbose 字段被设置为指定值.
|
||||
|
||||
使用 ``dataclasses.replace`` 在不可变的 TaskSpec 上创建带 verbose 标记的副本.
|
||||
依赖关系、标签等元数据全部保留.
|
||||
|
||||
Note
|
||||
-----
|
||||
自 ``_wrap_cmd`` 不再闭包捕获 ``verbose`` 后,此函数不再是必需的——
|
||||
直接翻转 ``spec.verbose`` 即可生效。保留是为了向后兼容现有调用与测试。
|
||||
TaskSpec 仍是 frozen dataclass,故仍用 ``replace`` 创建副本。
|
||||
|
||||
Parameters
|
||||
----------
|
||||
graph : Graph
|
||||
原始图.
|
||||
verbose : bool
|
||||
要设置的 verbose 值.
|
||||
|
||||
Returns
|
||||
-------
|
||||
Graph
|
||||
所有 spec 的 verbose 字段已更新的新图.
|
||||
"""
|
||||
new_specs: list[TaskSpec[Any]] = []
|
||||
for spec in graph.all_specs().values():
|
||||
if spec.verbose == verbose:
|
||||
new_specs.append(spec)
|
||||
else:
|
||||
new_specs.append(replace(spec, verbose=verbose))
|
||||
return Graph.from_specs(new_specs)
|
||||
|
||||
|
||||
@dataclass
|
||||
class CliRunner:
|
||||
"""命令行运行器: 根据用户输入执行对应的任务流图.
|
||||
@@ -296,12 +263,8 @@ class CliRunner:
|
||||
# 确定是否 verbose: --quiet 覆盖默认值
|
||||
verbose = self.verbose and not parsed.quiet
|
||||
|
||||
# 对图应用 verbose 设置 (重建带 verbose 标记的 spec)
|
||||
# 执行对应的图 (verbose 标记由 run() 统一应用到各 spec)
|
||||
graph = self.graphs[parsed.command]
|
||||
if verbose:
|
||||
graph = _apply_verbose_to_graph(graph, verbose=True)
|
||||
|
||||
# 执行对应的图
|
||||
try:
|
||||
report = run(
|
||||
graph,
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
+20
-103
@@ -5,8 +5,7 @@ from __future__ import annotations
|
||||
from pathlib import Path
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
import pyflowx as px
|
||||
from pyflowx.cli import autofmt
|
||||
from pyflowx.ops import dev
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
@@ -19,14 +18,14 @@ class TestFormatWithRuff:
|
||||
"""Should format with ruff."""
|
||||
with patch("subprocess.run") as mock_run:
|
||||
mock_run.return_value = MagicMock(returncode=0)
|
||||
autofmt.format_with_ruff(tmp_path, fix=True)
|
||||
dev.format_with_ruff(tmp_path, fix=True)
|
||||
assert mock_run.called
|
||||
|
||||
def test_format_with_ruff_no_fix(self, tmp_path: Path) -> None:
|
||||
"""Should format with ruff without fix."""
|
||||
with patch("subprocess.run") as mock_run:
|
||||
mock_run.return_value = MagicMock(returncode=0)
|
||||
autofmt.format_with_ruff(tmp_path, fix=False)
|
||||
dev.format_with_ruff(tmp_path, fix=False)
|
||||
# Should not include --fix flag
|
||||
call_args = mock_run.call_args[0][0]
|
||||
assert "--fix" not in call_args
|
||||
@@ -42,14 +41,14 @@ class TestLintWithRuff:
|
||||
"""Should lint with ruff."""
|
||||
with patch("subprocess.run") as mock_run:
|
||||
mock_run.return_value = MagicMock(returncode=0)
|
||||
autofmt.lint_with_ruff(tmp_path, fix=True)
|
||||
dev.lint_with_ruff(tmp_path, fix=True)
|
||||
assert mock_run.called
|
||||
|
||||
def test_lint_with_ruff_no_fix(self, tmp_path: Path) -> None:
|
||||
"""Should lint with ruff without fix."""
|
||||
with patch("subprocess.run") as mock_run:
|
||||
mock_run.return_value = MagicMock(returncode=0)
|
||||
autofmt.lint_with_ruff(tmp_path, fix=False)
|
||||
dev.lint_with_ruff(tmp_path, fix=False)
|
||||
# Should not include --fix flag
|
||||
call_args = mock_run.call_args[0][0]
|
||||
assert "--fix" not in call_args
|
||||
@@ -66,7 +65,7 @@ class TestAddDocstring:
|
||||
py_file = tmp_path / "test.py"
|
||||
py_file.write_text("def test():\n pass\n")
|
||||
|
||||
result = autofmt.add_docstring(py_file, '"""Test module."""')
|
||||
result = dev.add_docstring(py_file, '"""Test module."""')
|
||||
assert result is True
|
||||
|
||||
def test_add_docstring_skips_files_with_docstring(self, tmp_path: Path) -> None:
|
||||
@@ -74,7 +73,7 @@ class TestAddDocstring:
|
||||
py_file = tmp_path / "test.py"
|
||||
py_file.write_text('"""Existing docstring."""\ndef test():\n pass\n')
|
||||
|
||||
result = autofmt.add_docstring(py_file, '"""New docstring."""')
|
||||
result = dev.add_docstring(py_file, '"""New docstring."""')
|
||||
assert result is False
|
||||
|
||||
def test_add_docstring_empty_file(self, tmp_path: Path) -> None:
|
||||
@@ -82,7 +81,7 @@ class TestAddDocstring:
|
||||
py_file = tmp_path / "test.py"
|
||||
py_file.write_text("")
|
||||
|
||||
result = autofmt.add_docstring(py_file, '"""Test module."""')
|
||||
result = dev.add_docstring(py_file, '"""Test module."""')
|
||||
# Should handle empty file
|
||||
assert result is True
|
||||
|
||||
@@ -98,7 +97,7 @@ class TestGenerateModuleDocstring:
|
||||
py_file = tmp_path / "test.py"
|
||||
py_file.write_text("def test():\n pass\n")
|
||||
|
||||
result = autofmt.generate_module_docstring(py_file)
|
||||
result = dev.generate_module_docstring(py_file)
|
||||
# Should contain "Tests for" since stem contains "test"
|
||||
assert "Tests for" in result
|
||||
|
||||
@@ -108,7 +107,7 @@ class TestGenerateModuleDocstring:
|
||||
py_file.parent.mkdir(parents=True)
|
||||
py_file.write_text("def test():\n pass\n")
|
||||
|
||||
result = autofmt.generate_module_docstring(py_file)
|
||||
result = dev.generate_module_docstring(py_file)
|
||||
assert "mypackage" in result
|
||||
|
||||
def test_generate_module_docstring_cli(self, tmp_path: Path) -> None:
|
||||
@@ -116,7 +115,7 @@ class TestGenerateModuleDocstring:
|
||||
py_file = tmp_path / "cli.py"
|
||||
py_file.write_text("def test():\n pass\n")
|
||||
|
||||
result = autofmt.generate_module_docstring(py_file)
|
||||
result = dev.generate_module_docstring(py_file)
|
||||
assert "Command-line interface" in result
|
||||
|
||||
def test_generate_module_docstring_util(self, tmp_path: Path) -> None:
|
||||
@@ -124,7 +123,7 @@ class TestGenerateModuleDocstring:
|
||||
py_file = tmp_path / "utils.py"
|
||||
py_file.write_text("def test():\n pass\n")
|
||||
|
||||
result = autofmt.generate_module_docstring(py_file)
|
||||
result = dev.generate_module_docstring(py_file)
|
||||
assert "Utility functions" in result
|
||||
|
||||
|
||||
@@ -139,8 +138,8 @@ class TestAutoAddDocstrings:
|
||||
py_file = tmp_path / "test.py"
|
||||
py_file.write_text("def test():\n pass\n")
|
||||
|
||||
with patch.object(autofmt, "add_docstring", return_value=True):
|
||||
count = autofmt.auto_add_docstrings(tmp_path)
|
||||
with patch.object(dev, "add_docstring", return_value=True):
|
||||
count = dev.auto_add_docstrings(tmp_path)
|
||||
assert count >= 0
|
||||
|
||||
def test_auto_add_docstrings_skips_ignored(self, tmp_path: Path) -> None:
|
||||
@@ -149,7 +148,7 @@ class TestAutoAddDocstrings:
|
||||
py_file.parent.mkdir()
|
||||
py_file.write_text("def test():\n pass\n")
|
||||
|
||||
count = autofmt.auto_add_docstrings(tmp_path)
|
||||
count = dev.auto_add_docstrings(tmp_path)
|
||||
# Should skip __pycache__
|
||||
assert count == 0
|
||||
|
||||
@@ -158,7 +157,7 @@ class TestAutoAddDocstrings:
|
||||
txt_file = tmp_path / "test.txt"
|
||||
txt_file.write_text("test content")
|
||||
|
||||
count = autofmt.auto_add_docstrings(tmp_path)
|
||||
count = dev.auto_add_docstrings(tmp_path)
|
||||
assert count == 0
|
||||
|
||||
|
||||
@@ -179,7 +178,7 @@ class TestSyncPyprojectConfig:
|
||||
|
||||
with patch("subprocess.run") as mock_run:
|
||||
mock_run.return_value = MagicMock(returncode=0)
|
||||
autofmt.sync_pyproject_config(tmp_path)
|
||||
dev.sync_pyproject_config(tmp_path)
|
||||
assert mock_run.called
|
||||
|
||||
def test_sync_pyproject_config_updates_file(self, tmp_path: Path) -> None:
|
||||
@@ -193,7 +192,7 @@ class TestSyncPyprojectConfig:
|
||||
|
||||
with patch("subprocess.run") as mock_run:
|
||||
mock_run.return_value = MagicMock(returncode=0)
|
||||
autofmt.sync_pyproject_config(tmp_path)
|
||||
dev.sync_pyproject_config(tmp_path)
|
||||
assert mock_run.called
|
||||
|
||||
|
||||
@@ -207,95 +206,13 @@ class TestFormatAll:
|
||||
"""Should run ruff format."""
|
||||
with patch("subprocess.run") as mock_run:
|
||||
mock_run.return_value = MagicMock(returncode=0)
|
||||
autofmt.format_all(tmp_path)
|
||||
dev.format_all(tmp_path)
|
||||
assert mock_run.called
|
||||
|
||||
def test_format_all_runs_ruff_check(self, tmp_path: Path) -> None:
|
||||
"""Should run ruff check."""
|
||||
with patch("subprocess.run") as mock_run:
|
||||
mock_run.return_value = MagicMock(returncode=0)
|
||||
autofmt.format_all(tmp_path)
|
||||
dev.format_all(tmp_path)
|
||||
# Should call ruff format and ruff check
|
||||
assert mock_run.call_count == 2
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
# main function
|
||||
# ---------------------------------------------------------------------- #
|
||||
class TestMain:
|
||||
"""Test main function."""
|
||||
|
||||
def test_main_fmt_default_target(self) -> None:
|
||||
"""main() should handle fmt command with default target."""
|
||||
with patch("sys.argv", ["autofmt", "fmt"]), patch.object(px, "run") as mock_run:
|
||||
autofmt.main()
|
||||
assert mock_run.called
|
||||
|
||||
def test_main_fmt_custom_target(self) -> None:
|
||||
"""main() should handle fmt command with custom target."""
|
||||
with patch("sys.argv", ["autofmt", "fmt", "--target", "src"]), patch.object(px, "run") as mock_run:
|
||||
autofmt.main()
|
||||
assert mock_run.called
|
||||
|
||||
def test_main_lint_default_target(self) -> None:
|
||||
"""main() should handle lint command with default target."""
|
||||
with patch("sys.argv", ["autofmt", "lint"]), patch.object(px, "run") as mock_run:
|
||||
autofmt.main()
|
||||
assert mock_run.called
|
||||
|
||||
def test_main_lint_with_fix(self) -> None:
|
||||
"""main() should handle lint command with fix."""
|
||||
with patch("sys.argv", ["autofmt", "lint", "--fix"]), patch.object(px, "run") as mock_run:
|
||||
autofmt.main()
|
||||
assert mock_run.called
|
||||
|
||||
def test_main_lint_custom_target(self) -> None:
|
||||
"""main() should handle lint command with custom target."""
|
||||
with patch("sys.argv", ["autofmt", "lint", "--target", "src"]), patch.object(px, "run") as mock_run:
|
||||
autofmt.main()
|
||||
assert mock_run.called
|
||||
|
||||
def test_main_doc_default_root(self) -> None:
|
||||
"""main() should handle doc command with default root."""
|
||||
with patch("sys.argv", ["autofmt", "doc"]), patch.object(px, "run") as mock_run:
|
||||
autofmt.main()
|
||||
assert mock_run.called
|
||||
|
||||
def test_main_doc_custom_root(self) -> None:
|
||||
"""main() should handle doc command with custom root."""
|
||||
with patch("sys.argv", ["autofmt", "doc", "--root-dir", "src"]), patch.object(px, "run") as mock_run:
|
||||
autofmt.main()
|
||||
assert mock_run.called
|
||||
|
||||
def test_main_sync_default_root(self) -> None:
|
||||
"""main() should handle sync command with default root."""
|
||||
with patch("sys.argv", ["autofmt", "sync"]), patch.object(px, "run") as mock_run:
|
||||
autofmt.main()
|
||||
assert mock_run.called
|
||||
|
||||
def test_main_sync_custom_root(self) -> None:
|
||||
"""main() should handle sync command with custom root."""
|
||||
with patch("sys.argv", ["autofmt", "sync", "--root-dir", "."]), patch.object(px, "run") as mock_run:
|
||||
autofmt.main()
|
||||
assert mock_run.called
|
||||
|
||||
def test_main_with_no_args_shows_help(self) -> None:
|
||||
"""main() with no args should show help."""
|
||||
with patch("sys.argv", ["autofmt"]), patch.object(autofmt, "main"):
|
||||
# Just call main, it should show help and return
|
||||
autofmt.main()
|
||||
# main() should return without calling px.run
|
||||
assert True
|
||||
|
||||
def test_main_creates_task_specs_with_verbose(self) -> None:
|
||||
"""main() should create TaskSpecs with verbose=True."""
|
||||
with patch("sys.argv", ["autofmt", "fmt"]), patch.object(px, "run") as mock_run:
|
||||
autofmt.main()
|
||||
assert mock_run.called
|
||||
|
||||
def test_main_uses_thread_strategy(self) -> None:
|
||||
"""main() should use thread strategy."""
|
||||
with patch("sys.argv", ["autofmt", "fmt"]), patch.object(px, "run") as mock_run:
|
||||
autofmt.main()
|
||||
# Check that strategy="thread" was used
|
||||
assert mock_run.called
|
||||
|
||||
@@ -4,12 +4,10 @@ from __future__ import annotations
|
||||
|
||||
import os
|
||||
from pathlib import Path
|
||||
from unittest.mock import patch
|
||||
|
||||
import pytest
|
||||
|
||||
import pyflowx as px
|
||||
from pyflowx.cli import bumpversion
|
||||
from pyflowx.ops import dev
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
@@ -29,7 +27,7 @@ class TestBumpFileVersion:
|
||||
test_file = tmp_path / "pyproject.toml"
|
||||
test_file.write_text('version = "1.2.3"', encoding="utf-8")
|
||||
|
||||
result = bumpversion.bump_file_version(test_file, "patch")
|
||||
result = dev.bump_file_version(test_file, "patch")
|
||||
|
||||
assert result == "1.2.4"
|
||||
assert test_file.read_text(encoding="utf-8") == 'version = "1.2.4"'
|
||||
@@ -39,7 +37,7 @@ class TestBumpFileVersion:
|
||||
test_file = tmp_path / "pyproject.toml"
|
||||
test_file.write_text('version = "1.2.3"', encoding="utf-8")
|
||||
|
||||
result = bumpversion.bump_file_version(test_file, "minor")
|
||||
result = dev.bump_file_version(test_file, "minor")
|
||||
|
||||
assert result == "1.3.0"
|
||||
assert test_file.read_text(encoding="utf-8") == 'version = "1.3.0"'
|
||||
@@ -49,7 +47,7 @@ class TestBumpFileVersion:
|
||||
test_file = tmp_path / "pyproject.toml"
|
||||
test_file.write_text('version = "1.2.3"', encoding="utf-8")
|
||||
|
||||
result = bumpversion.bump_file_version(test_file, "major")
|
||||
result = dev.bump_file_version(test_file, "major")
|
||||
|
||||
assert result == "2.0.0"
|
||||
assert test_file.read_text(encoding="utf-8") == 'version = "2.0.0"'
|
||||
@@ -59,7 +57,7 @@ class TestBumpFileVersion:
|
||||
test_file = tmp_path / "pyproject.toml"
|
||||
test_file.write_text('version = "1.2.3-alpha.1"', encoding="utf-8")
|
||||
|
||||
result = bumpversion.bump_file_version(test_file, "patch")
|
||||
result = dev.bump_file_version(test_file, "patch")
|
||||
|
||||
assert result == "1.2.4"
|
||||
# 预发布版本应该被清除
|
||||
@@ -71,7 +69,7 @@ class TestBumpFileVersion:
|
||||
test_file = tmp_path / "pyproject.toml"
|
||||
test_file.write_text('version = "1.2.3+build.123"', encoding="utf-8")
|
||||
|
||||
result = bumpversion.bump_file_version(test_file, "patch")
|
||||
result = dev.bump_file_version(test_file, "patch")
|
||||
|
||||
assert result == "1.2.4"
|
||||
# 构建元数据应该被清除
|
||||
@@ -83,7 +81,7 @@ class TestBumpFileVersion:
|
||||
test_file = tmp_path / "test.txt"
|
||||
test_file.write_text("no version here", encoding="utf-8")
|
||||
|
||||
result = bumpversion.bump_file_version(test_file, "patch")
|
||||
result = dev.bump_file_version(test_file, "patch")
|
||||
|
||||
assert result is None
|
||||
captured = capsys.readouterr()
|
||||
@@ -94,7 +92,7 @@ class TestBumpFileVersion:
|
||||
test_file = tmp_path / "__init__.py"
|
||||
test_file.write_text('__version__ = "1.2.3"', encoding="utf-8")
|
||||
|
||||
result = bumpversion.bump_file_version(test_file, "patch")
|
||||
result = dev.bump_file_version(test_file, "patch")
|
||||
|
||||
assert result == "1.2.4"
|
||||
assert test_file.read_text(encoding="utf-8") == '__version__ = "1.2.4"'
|
||||
@@ -110,7 +108,7 @@ description = "Test project"
|
||||
"""
|
||||
test_file.write_text(content, encoding="utf-8")
|
||||
|
||||
result = bumpversion.bump_file_version(test_file, "minor")
|
||||
result = dev.bump_file_version(test_file, "minor")
|
||||
|
||||
assert result == "0.2.0"
|
||||
updated = test_file.read_text(encoding="utf-8")
|
||||
@@ -126,7 +124,7 @@ __version__ = "1.0.0"
|
||||
'''
|
||||
test_file.write_text(content, encoding="utf-8")
|
||||
|
||||
result = bumpversion.bump_file_version(test_file, "major")
|
||||
result = dev.bump_file_version(test_file, "major")
|
||||
|
||||
assert result == "2.0.0"
|
||||
updated = test_file.read_text(encoding="utf-8")
|
||||
@@ -142,7 +140,7 @@ dependencies = ["lib >= 2.0.0", "other >= 3.0.0"]
|
||||
"""
|
||||
test_file.write_text(content, encoding="utf-8")
|
||||
|
||||
result = bumpversion.bump_file_version(test_file, "patch")
|
||||
result = dev.bump_file_version(test_file, "patch")
|
||||
|
||||
assert result == "1.0.1"
|
||||
updated = test_file.read_text(encoding="utf-8")
|
||||
@@ -158,7 +156,7 @@ dependencies = ["lib >= 2.0.0", "other >= 3.0.0"]
|
||||
test_file.mkdir()
|
||||
|
||||
with pytest.raises(Exception): # noqa: B017
|
||||
bumpversion.bump_file_version(test_file, "patch")
|
||||
dev.bump_file_version(test_file, "patch")
|
||||
|
||||
def test_file_write_error(self, tmp_path: Path, capsys: pytest.CaptureFixture[str]) -> None:
|
||||
"""Should handle file write errors."""
|
||||
@@ -173,7 +171,7 @@ dependencies = ["lib >= 2.0.0", "other >= 3.0.0"]
|
||||
|
||||
try:
|
||||
with pytest.raises(Exception): # noqa: B017
|
||||
bumpversion.bump_file_version(test_file, "patch")
|
||||
dev.bump_file_version(test_file, "patch")
|
||||
finally:
|
||||
# 恢复权限以便清理
|
||||
test_file.chmod(0o644)
|
||||
@@ -190,7 +188,7 @@ class TestVersionPattern:
|
||||
test_file = tmp_path / "__init__.py"
|
||||
test_file.write_text('__version__ = "1.0.0"', encoding="utf-8")
|
||||
|
||||
result = bumpversion.bump_file_version(test_file, "patch")
|
||||
result = dev.bump_file_version(test_file, "patch")
|
||||
|
||||
assert result == "1.0.1"
|
||||
|
||||
@@ -199,7 +197,7 @@ class TestVersionPattern:
|
||||
test_file = tmp_path / "__init__.py"
|
||||
test_file.write_text('__version__ = "0.0.0"', encoding="utf-8")
|
||||
|
||||
result = bumpversion.bump_file_version(test_file, "patch")
|
||||
result = dev.bump_file_version(test_file, "patch")
|
||||
|
||||
assert result == "0.0.1"
|
||||
|
||||
@@ -208,7 +206,7 @@ class TestVersionPattern:
|
||||
test_file = tmp_path / "__init__.py"
|
||||
test_file.write_text('__version__ = "10.20.30"', encoding="utf-8")
|
||||
|
||||
result = bumpversion.bump_file_version(test_file, "minor")
|
||||
result = dev.bump_file_version(test_file, "minor")
|
||||
|
||||
assert result == "10.21.0"
|
||||
|
||||
@@ -217,7 +215,7 @@ class TestVersionPattern:
|
||||
test_file = tmp_path / "test.txt"
|
||||
test_file.write_text("https://example.com/v1.2.3/download", encoding="utf-8")
|
||||
|
||||
result = bumpversion.bump_file_version(test_file, "patch")
|
||||
result = dev.bump_file_version(test_file, "patch")
|
||||
|
||||
# 不应该匹配 URL 中的版本号
|
||||
assert result is None
|
||||
@@ -234,7 +232,7 @@ class TestEdgeCases:
|
||||
test_file = tmp_path / "empty.txt"
|
||||
test_file.write_text("", encoding="utf-8")
|
||||
|
||||
result = bumpversion.bump_file_version(test_file, "patch")
|
||||
result = dev.bump_file_version(test_file, "patch")
|
||||
|
||||
assert result is None
|
||||
captured = capsys.readouterr()
|
||||
@@ -246,7 +244,7 @@ class TestEdgeCases:
|
||||
content = '# 中文注释\n__version__ = "1.0.0"\n# 特殊字符: @#$%'
|
||||
test_file.write_text(content, encoding="utf-8")
|
||||
|
||||
result = bumpversion.bump_file_version(test_file, "patch")
|
||||
result = dev.bump_file_version(test_file, "patch")
|
||||
|
||||
assert result == "1.0.1"
|
||||
updated = test_file.read_text(encoding="utf-8")
|
||||
@@ -259,64 +257,16 @@ class TestEdgeCases:
|
||||
test_file.write_text('__version__ = "1.0.0"', encoding="utf-8")
|
||||
|
||||
# 第一次 bump
|
||||
result1 = bumpversion.bump_file_version(test_file, "patch")
|
||||
result1 = dev.bump_file_version(test_file, "patch")
|
||||
assert result1 == "1.0.1"
|
||||
|
||||
# 第二次 bump
|
||||
result2 = bumpversion.bump_file_version(test_file, "minor")
|
||||
result2 = dev.bump_file_version(test_file, "minor")
|
||||
assert result2 == "1.1.0"
|
||||
|
||||
# 第三次 bump
|
||||
result3 = bumpversion.bump_file_version(test_file, "major")
|
||||
result3 = dev.bump_file_version(test_file, "major")
|
||||
assert result3 == "2.0.0"
|
||||
|
||||
# 验证最终结果
|
||||
assert test_file.read_text(encoding="utf-8") == '__version__ = "2.0.0"'
|
||||
|
||||
|
||||
class TestBumpVersionCli:
|
||||
"""Test bumpversion CLI."""
|
||||
|
||||
def test_minor(self, tmp_path: Path) -> None:
|
||||
"""Should handle minor version bump."""
|
||||
test_file = tmp_path / "__init__.py"
|
||||
test_file.write_text('__version__ = "1.0.0"', encoding="utf-8")
|
||||
|
||||
# Mock px.run: 只真正执行第一次调用(版本更新),其余返回空 dict
|
||||
with patch("sys.argv", ["bumpversion", "minor", "--no-tag"]), patch("pyflowx.run") as mock_run:
|
||||
|
||||
def run_side_effect(graph: px.Graph, strategy: str | None = None):
|
||||
# 执行实际版本更新任务
|
||||
results = {}
|
||||
for spec in graph.specs.values():
|
||||
if spec.fn is not None and spec.args:
|
||||
results[spec.name] = spec.fn(*spec.args)
|
||||
return results
|
||||
|
||||
mock_run.side_effect = run_side_effect
|
||||
bumpversion.main()
|
||||
|
||||
# 验证版本号已更新
|
||||
assert test_file.read_text(encoding="utf-8") == '__version__ = "1.1.0"'
|
||||
|
||||
def test_no_valid_files(self, tmp_path: Path, capsys: pytest.CaptureFixture[str]) -> None:
|
||||
"""Should handle no valid files."""
|
||||
test_file = tmp_path / "test.txt"
|
||||
test_file.write_text("这是一个测试文件", encoding="utf-8")
|
||||
|
||||
with patch("sys.argv", ["bumpversion", "minor", "--no-tag"]), patch("pyflowx.run") as mock_run:
|
||||
|
||||
def run_side_effect(graph: px.Graph, strategy: str | None = None):
|
||||
# 执行实际版本更新任务
|
||||
results = {}
|
||||
for spec in graph.specs.values():
|
||||
if spec.fn is not None and spec.args:
|
||||
results[spec.name] = spec.fn(*spec.args)
|
||||
return results
|
||||
|
||||
mock_run.side_effect = run_side_effect
|
||||
bumpversion.main()
|
||||
|
||||
# 验证未更新任何文件
|
||||
assert test_file.read_text(encoding="utf-8") == "这是一个测试文件"
|
||||
assert "未找到包含版本号的文件" in capsys.readouterr().out
|
||||
|
||||
+210
-182
@@ -125,19 +125,21 @@ class TestEmailDatabase:
|
||||
|
||||
# Insert test emails
|
||||
for i in range(5):
|
||||
db.insert_email({
|
||||
"file_path": f"/test/path{i}.eml",
|
||||
"file_hash": f"hash{i}",
|
||||
"subject": f"Subject {i}",
|
||||
"sender": f"sender{i}@example.com",
|
||||
"recipients": "recipient@example.com",
|
||||
"date": f"Mon, {i + 1} Jan 2024 12:00:00 +0000",
|
||||
"date_parsed": f"2024-01-0{i + 1}T12:00:00",
|
||||
"body_text": f"Body {i}",
|
||||
"body_html": f"<p>Body {i}</p>",
|
||||
"has_attachments": 0,
|
||||
"file_size": 1024,
|
||||
})
|
||||
db.insert_email(
|
||||
{
|
||||
"file_path": f"/test/path{i}.eml",
|
||||
"file_hash": f"hash{i}",
|
||||
"subject": f"Subject {i}",
|
||||
"sender": f"sender{i}@example.com",
|
||||
"recipients": "recipient@example.com",
|
||||
"date": f"Mon, {i + 1} Jan 2024 12:00:00 +0000",
|
||||
"date_parsed": f"2024-01-0{i + 1}T12:00:00",
|
||||
"body_text": f"Body {i}",
|
||||
"body_html": f"<p>Body {i}</p>",
|
||||
"has_attachments": 0,
|
||||
"file_size": 1024,
|
||||
}
|
||||
)
|
||||
|
||||
results = db.search_emails(limit=3)
|
||||
assert len(results) == 3
|
||||
@@ -148,33 +150,37 @@ class TestEmailDatabase:
|
||||
db_path = tmp_path / "test.db"
|
||||
db = emlmanager.EmailDatabase(db_path)
|
||||
|
||||
db.insert_email({
|
||||
"file_path": "/test/path1.eml",
|
||||
"file_hash": "hash1",
|
||||
"subject": "Important Meeting",
|
||||
"sender": "sender1@example.com",
|
||||
"recipients": "recipient@example.com",
|
||||
"date": "Mon, 1 Jan 2024 12:00:00 +0000",
|
||||
"date_parsed": "2024-01-01T12:00:00",
|
||||
"body_text": "Meeting body",
|
||||
"body_html": "<p>Meeting body</p>",
|
||||
"has_attachments": 0,
|
||||
"file_size": 1024,
|
||||
})
|
||||
db.insert_email(
|
||||
{
|
||||
"file_path": "/test/path1.eml",
|
||||
"file_hash": "hash1",
|
||||
"subject": "Important Meeting",
|
||||
"sender": "sender1@example.com",
|
||||
"recipients": "recipient@example.com",
|
||||
"date": "Mon, 1 Jan 2024 12:00:00 +0000",
|
||||
"date_parsed": "2024-01-01T12:00:00",
|
||||
"body_text": "Meeting body",
|
||||
"body_html": "<p>Meeting body</p>",
|
||||
"has_attachments": 0,
|
||||
"file_size": 1024,
|
||||
}
|
||||
)
|
||||
|
||||
db.insert_email({
|
||||
"file_path": "/test/path2.eml",
|
||||
"file_hash": "hash2",
|
||||
"subject": "Casual Chat",
|
||||
"sender": "sender2@example.com",
|
||||
"recipients": "recipient@example.com",
|
||||
"date": "Tue, 2 Jan 2024 12:00:00 +0000",
|
||||
"date_parsed": "2024-01-02T12:00:00",
|
||||
"body_text": "Chat body",
|
||||
"body_html": "<p>Chat body</p>",
|
||||
"has_attachments": 0,
|
||||
"file_size": 1024,
|
||||
})
|
||||
db.insert_email(
|
||||
{
|
||||
"file_path": "/test/path2.eml",
|
||||
"file_hash": "hash2",
|
||||
"subject": "Casual Chat",
|
||||
"sender": "sender2@example.com",
|
||||
"recipients": "recipient@example.com",
|
||||
"date": "Tue, 2 Jan 2024 12:00:00 +0000",
|
||||
"date_parsed": "2024-01-02T12:00:00",
|
||||
"body_text": "Chat body",
|
||||
"body_html": "<p>Chat body</p>",
|
||||
"has_attachments": 0,
|
||||
"file_size": 1024,
|
||||
}
|
||||
)
|
||||
|
||||
results = db.search_emails(keyword="Meeting", field="subject")
|
||||
assert len(results) == 1
|
||||
@@ -186,33 +192,37 @@ class TestEmailDatabase:
|
||||
db_path = tmp_path / "test.db"
|
||||
db = emlmanager.EmailDatabase(db_path)
|
||||
|
||||
db.insert_email({
|
||||
"file_path": "/test/path1.eml",
|
||||
"file_hash": "hash1",
|
||||
"subject": "Test",
|
||||
"sender": "alice@example.com",
|
||||
"recipients": "recipient@example.com",
|
||||
"date": "Mon, 1 Jan 2024 12:00:00 +0000",
|
||||
"date_parsed": "2024-01-01T12:00:00",
|
||||
"body_text": "Body",
|
||||
"body_html": "<p>Body</p>",
|
||||
"has_attachments": 0,
|
||||
"file_size": 1024,
|
||||
})
|
||||
db.insert_email(
|
||||
{
|
||||
"file_path": "/test/path1.eml",
|
||||
"file_hash": "hash1",
|
||||
"subject": "Test",
|
||||
"sender": "alice@example.com",
|
||||
"recipients": "recipient@example.com",
|
||||
"date": "Mon, 1 Jan 2024 12:00:00 +0000",
|
||||
"date_parsed": "2024-01-01T12:00:00",
|
||||
"body_text": "Body",
|
||||
"body_html": "<p>Body</p>",
|
||||
"has_attachments": 0,
|
||||
"file_size": 1024,
|
||||
}
|
||||
)
|
||||
|
||||
db.insert_email({
|
||||
"file_path": "/test/path2.eml",
|
||||
"file_hash": "hash2",
|
||||
"subject": "Test",
|
||||
"sender": "bob@example.com",
|
||||
"recipients": "recipient@example.com",
|
||||
"date": "Tue, 2 Jan 2024 12:00:00 +0000",
|
||||
"date_parsed": "2024-01-02T12:00:00",
|
||||
"body_text": "Body",
|
||||
"body_html": "<p>Body</p>",
|
||||
"has_attachments": 0,
|
||||
"file_size": 1024,
|
||||
})
|
||||
db.insert_email(
|
||||
{
|
||||
"file_path": "/test/path2.eml",
|
||||
"file_hash": "hash2",
|
||||
"subject": "Test",
|
||||
"sender": "bob@example.com",
|
||||
"recipients": "recipient@example.com",
|
||||
"date": "Tue, 2 Jan 2024 12:00:00 +0000",
|
||||
"date_parsed": "2024-01-02T12:00:00",
|
||||
"body_text": "Body",
|
||||
"body_html": "<p>Body</p>",
|
||||
"has_attachments": 0,
|
||||
"file_size": 1024,
|
||||
}
|
||||
)
|
||||
|
||||
results = db.search_emails(keyword="alice", field="sender")
|
||||
assert len(results) == 1
|
||||
@@ -224,19 +234,21 @@ class TestEmailDatabase:
|
||||
db_path = tmp_path / "test.db"
|
||||
db = emlmanager.EmailDatabase(db_path)
|
||||
|
||||
db.insert_email({
|
||||
"file_path": "/test/path1.eml",
|
||||
"file_hash": "hash1",
|
||||
"subject": "Project Update",
|
||||
"sender": "manager@example.com",
|
||||
"recipients": "team@example.com",
|
||||
"date": "Mon, 1 Jan 2024 12:00:00 +0000",
|
||||
"date_parsed": "2024-01-01T12:00:00",
|
||||
"body_text": "Please review the quarterly report",
|
||||
"body_html": "<p>Please review the quarterly report</p>",
|
||||
"has_attachments": 0,
|
||||
"file_size": 1024,
|
||||
})
|
||||
db.insert_email(
|
||||
{
|
||||
"file_path": "/test/path1.eml",
|
||||
"file_hash": "hash1",
|
||||
"subject": "Project Update",
|
||||
"sender": "manager@example.com",
|
||||
"recipients": "team@example.com",
|
||||
"date": "Mon, 1 Jan 2024 12:00:00 +0000",
|
||||
"date_parsed": "2024-01-01T12:00:00",
|
||||
"body_text": "Please review the quarterly report",
|
||||
"body_html": "<p>Please review the quarterly report</p>",
|
||||
"has_attachments": 0,
|
||||
"file_size": 1024,
|
||||
}
|
||||
)
|
||||
|
||||
# Search for keyword in subject
|
||||
results = db.search_emails(keyword="Project", field="all")
|
||||
@@ -253,47 +265,53 @@ class TestEmailDatabase:
|
||||
db = emlmanager.EmailDatabase(db_path)
|
||||
|
||||
# Insert emails with same subject (different prefixes)
|
||||
db.insert_email({
|
||||
"file_path": "/test/path1.eml",
|
||||
"file_hash": "hash1",
|
||||
"subject": "Meeting Tomorrow",
|
||||
"sender": "sender1@example.com",
|
||||
"recipients": "recipient@example.com",
|
||||
"date": "Mon, 1 Jan 2024 12:00:00 +0000",
|
||||
"date_parsed": "2024-01-01T12:00:00",
|
||||
"body_text": "Body 1",
|
||||
"body_html": "<p>Body 1</p>",
|
||||
"has_attachments": 0,
|
||||
"file_size": 1024,
|
||||
})
|
||||
db.insert_email(
|
||||
{
|
||||
"file_path": "/test/path1.eml",
|
||||
"file_hash": "hash1",
|
||||
"subject": "Meeting Tomorrow",
|
||||
"sender": "sender1@example.com",
|
||||
"recipients": "recipient@example.com",
|
||||
"date": "Mon, 1 Jan 2024 12:00:00 +0000",
|
||||
"date_parsed": "2024-01-01T12:00:00",
|
||||
"body_text": "Body 1",
|
||||
"body_html": "<p>Body 1</p>",
|
||||
"has_attachments": 0,
|
||||
"file_size": 1024,
|
||||
}
|
||||
)
|
||||
|
||||
db.insert_email({
|
||||
"file_path": "/test/path2.eml",
|
||||
"file_hash": "hash2",
|
||||
"subject": "Re: Meeting Tomorrow",
|
||||
"sender": "sender2@example.com",
|
||||
"recipients": "recipient@example.com",
|
||||
"date": "Tue, 2 Jan 2024 12:00:00 +0000",
|
||||
"date_parsed": "2024-01-02T12:00:00",
|
||||
"body_text": "Body 2",
|
||||
"body_html": "<p>Body 2</p>",
|
||||
"has_attachments": 0,
|
||||
"file_size": 1024,
|
||||
})
|
||||
db.insert_email(
|
||||
{
|
||||
"file_path": "/test/path2.eml",
|
||||
"file_hash": "hash2",
|
||||
"subject": "Re: Meeting Tomorrow",
|
||||
"sender": "sender2@example.com",
|
||||
"recipients": "recipient@example.com",
|
||||
"date": "Tue, 2 Jan 2024 12:00:00 +0000",
|
||||
"date_parsed": "2024-01-02T12:00:00",
|
||||
"body_text": "Body 2",
|
||||
"body_html": "<p>Body 2</p>",
|
||||
"has_attachments": 0,
|
||||
"file_size": 1024,
|
||||
}
|
||||
)
|
||||
|
||||
db.insert_email({
|
||||
"file_path": "/test/path3.eml",
|
||||
"file_hash": "hash3",
|
||||
"subject": "Different Topic",
|
||||
"sender": "sender3@example.com",
|
||||
"recipients": "recipient@example.com",
|
||||
"date": "Wed, 3 Jan 2024 12:00:00 +0000",
|
||||
"date_parsed": "2024-01-03T12:00:00",
|
||||
"body_text": "Body 3",
|
||||
"body_html": "<p>Body 3</p>",
|
||||
"has_attachments": 0,
|
||||
"file_size": 1024,
|
||||
})
|
||||
db.insert_email(
|
||||
{
|
||||
"file_path": "/test/path3.eml",
|
||||
"file_hash": "hash3",
|
||||
"subject": "Different Topic",
|
||||
"sender": "sender3@example.com",
|
||||
"recipients": "recipient@example.com",
|
||||
"date": "Wed, 3 Jan 2024 12:00:00 +0000",
|
||||
"date_parsed": "2024-01-03T12:00:00",
|
||||
"body_text": "Body 3",
|
||||
"body_html": "<p>Body 3</p>",
|
||||
"has_attachments": 0,
|
||||
"file_size": 1024,
|
||||
}
|
||||
)
|
||||
|
||||
grouped = db.get_grouped_emails()
|
||||
# Should have 2 groups: "Meeting Tomorrow" and "Different Topic"
|
||||
@@ -322,19 +340,21 @@ class TestEmailDatabase:
|
||||
assert db.get_email_count() == 0
|
||||
|
||||
for i in range(3):
|
||||
db.insert_email({
|
||||
"file_path": f"/test/path{i}.eml",
|
||||
"file_hash": f"hash{i}",
|
||||
"subject": f"Subject {i}",
|
||||
"sender": f"sender{i}@example.com",
|
||||
"recipients": "recipient@example.com",
|
||||
"date": f"Mon, {i + 1} Jan 2024 12:00:00 +0000",
|
||||
"date_parsed": f"2024-01-0{i + 1}T12:00:00",
|
||||
"body_text": f"Body {i}",
|
||||
"body_html": f"<p>Body {i}</p>",
|
||||
"has_attachments": 0,
|
||||
"file_size": 1024,
|
||||
})
|
||||
db.insert_email(
|
||||
{
|
||||
"file_path": f"/test/path{i}.eml",
|
||||
"file_hash": f"hash{i}",
|
||||
"subject": f"Subject {i}",
|
||||
"sender": f"sender{i}@example.com",
|
||||
"recipients": "recipient@example.com",
|
||||
"date": f"Mon, {i + 1} Jan 2024 12:00:00 +0000",
|
||||
"date_parsed": f"2024-01-0{i + 1}T12:00:00",
|
||||
"body_text": f"Body {i}",
|
||||
"body_html": f"<p>Body {i}</p>",
|
||||
"has_attachments": 0,
|
||||
"file_size": 1024,
|
||||
}
|
||||
)
|
||||
|
||||
assert db.get_email_count() == 3
|
||||
db.close()
|
||||
@@ -346,19 +366,21 @@ class TestEmailDatabase:
|
||||
|
||||
# Insert some emails
|
||||
for i in range(3):
|
||||
db.insert_email({
|
||||
"file_path": f"/test/path{i}.eml",
|
||||
"file_hash": f"hash{i}",
|
||||
"subject": f"Subject {i}",
|
||||
"sender": f"sender{i}@example.com",
|
||||
"recipients": "recipient@example.com",
|
||||
"date": f"Mon, {i + 1} Jan 2024 12:00:00 +0000",
|
||||
"date_parsed": f"2024-01-0{i + 1}T12:00:00",
|
||||
"body_text": f"Body {i}",
|
||||
"body_html": f"<p>Body {i}</p>",
|
||||
"has_attachments": 0,
|
||||
"file_size": 1024,
|
||||
})
|
||||
db.insert_email(
|
||||
{
|
||||
"file_path": f"/test/path{i}.eml",
|
||||
"file_hash": f"hash{i}",
|
||||
"subject": f"Subject {i}",
|
||||
"sender": f"sender{i}@example.com",
|
||||
"recipients": "recipient@example.com",
|
||||
"date": f"Mon, {i + 1} Jan 2024 12:00:00 +0000",
|
||||
"date_parsed": f"2024-01-0{i + 1}T12:00:00",
|
||||
"body_text": f"Body {i}",
|
||||
"body_html": f"<p>Body {i}</p>",
|
||||
"has_attachments": 0,
|
||||
"file_size": 1024,
|
||||
}
|
||||
)
|
||||
|
||||
assert db.get_email_count() == 3
|
||||
|
||||
@@ -687,19 +709,21 @@ class TestEmlManagerHandler:
|
||||
|
||||
# Insert some emails
|
||||
for i in range(3):
|
||||
db.insert_email({
|
||||
"file_path": f"/test/path{i}.eml",
|
||||
"file_hash": f"hash{i}",
|
||||
"subject": f"Subject {i}",
|
||||
"sender": f"sender{i}@example.com",
|
||||
"recipients": "recipient@example.com",
|
||||
"date": f"Mon, {i + 1} Jan 2024 12:00:00 +0000",
|
||||
"date_parsed": f"2024-01-0{i + 1}T12:00:00",
|
||||
"body_text": f"Body {i}",
|
||||
"body_html": f"<p>Body {i}</p>",
|
||||
"has_attachments": 0,
|
||||
"file_size": 1024,
|
||||
})
|
||||
db.insert_email(
|
||||
{
|
||||
"file_path": f"/test/path{i}.eml",
|
||||
"file_hash": f"hash{i}",
|
||||
"subject": f"Subject {i}",
|
||||
"sender": f"sender{i}@example.com",
|
||||
"recipients": "recipient@example.com",
|
||||
"date": f"Mon, {i + 1} Jan 2024 12:00:00 +0000",
|
||||
"date_parsed": f"2024-01-0{i + 1}T12:00:00",
|
||||
"body_text": f"Body {i}",
|
||||
"body_html": f"<p>Body {i}</p>",
|
||||
"has_attachments": 0,
|
||||
"file_size": 1024,
|
||||
}
|
||||
)
|
||||
|
||||
# Create a mock handler instance without calling __init__
|
||||
handler = Mock(spec=emlmanager.EmlManagerHandler)
|
||||
@@ -721,19 +745,21 @@ class TestEmlManagerHandler:
|
||||
db = emlmanager.EmailDatabase(db_path)
|
||||
|
||||
# Insert test email
|
||||
db.insert_email({
|
||||
"file_path": "/test/path.eml",
|
||||
"file_hash": "hash",
|
||||
"subject": "Test Subject",
|
||||
"sender": "sender@example.com",
|
||||
"recipients": "recipient@example.com",
|
||||
"date": "Mon, 1 Jan 2024 12:00:00 +0000",
|
||||
"date_parsed": "2024-01-01T12:00:00",
|
||||
"body_text": "Test body",
|
||||
"body_html": "<p>Test body</p>",
|
||||
"has_attachments": 0,
|
||||
"file_size": 1024,
|
||||
})
|
||||
db.insert_email(
|
||||
{
|
||||
"file_path": "/test/path.eml",
|
||||
"file_hash": "hash",
|
||||
"subject": "Test Subject",
|
||||
"sender": "sender@example.com",
|
||||
"recipients": "recipient@example.com",
|
||||
"date": "Mon, 1 Jan 2024 12:00:00 +0000",
|
||||
"date_parsed": "2024-01-01T12:00:00",
|
||||
"body_text": "Test body",
|
||||
"body_html": "<p>Test body</p>",
|
||||
"has_attachments": 0,
|
||||
"file_size": 1024,
|
||||
}
|
||||
)
|
||||
|
||||
# Create a mock handler instance without calling __init__
|
||||
handler = Mock(spec=emlmanager.EmlManagerHandler)
|
||||
@@ -756,19 +782,21 @@ class TestEmlManagerHandler:
|
||||
db = emlmanager.EmailDatabase(db_path)
|
||||
|
||||
# Insert test email
|
||||
db.insert_email({
|
||||
"file_path": "/test/path.eml",
|
||||
"file_hash": "hash",
|
||||
"subject": "Test Subject",
|
||||
"sender": "sender@example.com",
|
||||
"recipients": "recipient@example.com",
|
||||
"date": "Mon, 1 Jan 2024 12:00:00 +0000",
|
||||
"date_parsed": "2024-01-01T12:00:00",
|
||||
"body_text": "Test body",
|
||||
"body_html": "<p>Test body</p>",
|
||||
"has_attachments": 0,
|
||||
"file_size": 1024,
|
||||
})
|
||||
db.insert_email(
|
||||
{
|
||||
"file_path": "/test/path.eml",
|
||||
"file_hash": "hash",
|
||||
"subject": "Test Subject",
|
||||
"sender": "sender@example.com",
|
||||
"recipients": "recipient@example.com",
|
||||
"date": "Mon, 1 Jan 2024 12:00:00 +0000",
|
||||
"date_parsed": "2024-01-01T12:00:00",
|
||||
"body_text": "Test body",
|
||||
"body_html": "<p>Test body</p>",
|
||||
"has_attachments": 0,
|
||||
"file_size": 1024,
|
||||
}
|
||||
)
|
||||
|
||||
assert db.get_email_count() == 1
|
||||
|
||||
|
||||
+10
-43
@@ -3,10 +3,8 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
from unittest.mock import patch
|
||||
|
||||
import pyflowx as px
|
||||
from pyflowx.cli import filedate
|
||||
from pyflowx.ops import files
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
@@ -20,7 +18,7 @@ class TestGetFileTimestamp:
|
||||
test_file = tmp_path / "test.txt"
|
||||
test_file.write_text("test content")
|
||||
|
||||
timestamp = filedate.get_file_timestamp(test_file)
|
||||
timestamp = files.get_file_timestamp(test_file)
|
||||
assert len(timestamp) == 8 # YYYYMMDD format
|
||||
assert timestamp.isdigit()
|
||||
|
||||
@@ -36,7 +34,7 @@ class TestRemoveDatePrefix:
|
||||
test_file = tmp_path / "20240101_test.txt"
|
||||
test_file.write_text("test content")
|
||||
|
||||
new_path = filedate.remove_date_prefix(test_file)
|
||||
new_path = files.remove_date_prefix(test_file)
|
||||
assert new_path.name == "test.txt"
|
||||
|
||||
def test_remove_date_prefix_without_date(self, tmp_path: Path) -> None:
|
||||
@@ -44,7 +42,7 @@ class TestRemoveDatePrefix:
|
||||
test_file = tmp_path / "test.txt"
|
||||
test_file.write_text("test content")
|
||||
|
||||
new_path = filedate.remove_date_prefix(test_file)
|
||||
new_path = files.remove_date_prefix(test_file)
|
||||
assert new_path == test_file
|
||||
|
||||
|
||||
@@ -59,7 +57,7 @@ class TestAddDatePrefix:
|
||||
test_file = tmp_path / "test.txt"
|
||||
test_file.write_text("test content")
|
||||
|
||||
new_path = filedate.add_date_prefix(test_file)
|
||||
new_path = files.add_date_prefix(test_file)
|
||||
assert new_path.name.startswith("20") # Starts with year
|
||||
assert "_test.txt" in new_path.name
|
||||
|
||||
@@ -75,7 +73,7 @@ class TestProcessFileDate:
|
||||
test_file = tmp_path / "test.txt"
|
||||
test_file.write_text("test content")
|
||||
|
||||
filedate.process_file_date(test_file, clear=False)
|
||||
files.process_file_date(test_file, clear=False)
|
||||
# File should be renamed with date prefix
|
||||
|
||||
def test_process_file_date_clear(self, tmp_path: Path) -> None:
|
||||
@@ -83,7 +81,7 @@ class TestProcessFileDate:
|
||||
test_file = tmp_path / "20240101_test.txt"
|
||||
test_file.write_text("test content")
|
||||
|
||||
filedate.process_file_date(test_file, clear=True)
|
||||
files.process_file_date(test_file, clear=True)
|
||||
# File should be renamed without date prefix
|
||||
|
||||
|
||||
@@ -95,42 +93,11 @@ class TestProcessFilesDate:
|
||||
|
||||
def test_process_files_date_batch(self, tmp_path: Path) -> None:
|
||||
"""Should process multiple files."""
|
||||
files = []
|
||||
file_list = []
|
||||
for i in range(3):
|
||||
test_file = tmp_path / f"test{i}.txt"
|
||||
test_file.write_text(f"content{i}")
|
||||
files.append(test_file)
|
||||
file_list.append(test_file)
|
||||
|
||||
filedate.process_files_date(files, clear=False)
|
||||
files.process_files_date(file_list, clear=False)
|
||||
# All files should be processed
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
# main function
|
||||
# ---------------------------------------------------------------------- #
|
||||
class TestMain:
|
||||
"""Test main function."""
|
||||
|
||||
def test_main_add_command(self, tmp_path: Path) -> None:
|
||||
"""main() should handle add command."""
|
||||
test_file = tmp_path / "test.txt"
|
||||
test_file.write_text("test content")
|
||||
|
||||
with patch("sys.argv", ["filedate", "add", str(test_file)]), patch.object(px, "run") as mock_run:
|
||||
filedate.main()
|
||||
assert mock_run.called
|
||||
|
||||
def test_main_clear_command(self, tmp_path: Path) -> None:
|
||||
"""main() should handle clear command."""
|
||||
test_file = tmp_path / "20240101_test.txt"
|
||||
test_file.write_text("test content")
|
||||
|
||||
with patch("sys.argv", ["filedate", "clear", str(test_file)]), patch.object(px, "run") as mock_run:
|
||||
filedate.main()
|
||||
assert mock_run.called
|
||||
|
||||
def test_main_with_no_args_shows_help(self) -> None:
|
||||
"""main() with no args should show help."""
|
||||
with patch("sys.argv", ["filedate"]):
|
||||
filedate.main()
|
||||
# Should print help and return
|
||||
|
||||
+12
-49
@@ -3,10 +3,8 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
from unittest.mock import patch
|
||||
|
||||
import pyflowx as px
|
||||
from pyflowx.cli import filelevel
|
||||
from pyflowx.ops import files
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
@@ -18,19 +16,19 @@ class TestRemoveMarks:
|
||||
def test_remove_marks_single_mark(self) -> None:
|
||||
"""Should remove single mark."""
|
||||
stem = "filename(PUB)"
|
||||
result = filelevel.remove_marks(stem, ["PUB"])
|
||||
result = files.remove_marks(stem, ["PUB"])
|
||||
assert result == "filename"
|
||||
|
||||
def test_remove_marks_multiple_marks(self) -> None:
|
||||
"""Should remove multiple marks."""
|
||||
stem = "filename(PUB)(NOR)"
|
||||
result = filelevel.remove_marks(stem, ["PUB", "NOR"])
|
||||
result = files.remove_marks(stem, ["PUB", "NOR"])
|
||||
assert result == "filename"
|
||||
|
||||
def test_remove_marks_no_marks(self) -> None:
|
||||
"""Should not change stem without marks."""
|
||||
stem = "filename"
|
||||
result = filelevel.remove_marks(stem, ["PUB"])
|
||||
result = files.remove_marks(stem, ["PUB"])
|
||||
assert result == "filename"
|
||||
|
||||
|
||||
@@ -45,7 +43,7 @@ class TestProcessFileLevel:
|
||||
test_file = tmp_path / "test.txt"
|
||||
test_file.write_text("test content")
|
||||
|
||||
filelevel.process_file_level(test_file, level=1)
|
||||
files.process_file_level(test_file, level=1)
|
||||
# File should be renamed with PUB level
|
||||
|
||||
def test_process_file_level_set_int(self, tmp_path: Path) -> None:
|
||||
@@ -53,7 +51,7 @@ class TestProcessFileLevel:
|
||||
test_file = tmp_path / "test.txt"
|
||||
test_file.write_text("test content")
|
||||
|
||||
filelevel.process_file_level(test_file, level=2)
|
||||
files.process_file_level(test_file, level=2)
|
||||
# File should be renamed with INT level
|
||||
|
||||
def test_process_file_level_clear(self, tmp_path: Path) -> None:
|
||||
@@ -61,7 +59,7 @@ class TestProcessFileLevel:
|
||||
test_file = tmp_path / "test(PUB).txt"
|
||||
test_file.write_text("test content")
|
||||
|
||||
filelevel.process_file_level(test_file, level=0)
|
||||
files.process_file_level(test_file, level=0)
|
||||
# File should be renamed without level
|
||||
|
||||
def test_process_file_level_invalid_level(self, tmp_path: Path) -> None:
|
||||
@@ -69,14 +67,14 @@ class TestProcessFileLevel:
|
||||
test_file = tmp_path / "test.txt"
|
||||
test_file.write_text("test content")
|
||||
|
||||
filelevel.process_file_level(test_file, level=5)
|
||||
files.process_file_level(test_file, level=5)
|
||||
# Should print error message
|
||||
|
||||
def test_process_file_level_nonexistent_file(self, tmp_path: Path) -> None:
|
||||
"""Should handle nonexistent file."""
|
||||
test_file = tmp_path / "nonexistent.txt"
|
||||
|
||||
filelevel.process_file_level(test_file, level=1)
|
||||
files.process_file_level(test_file, level=1)
|
||||
# Should print error message
|
||||
|
||||
|
||||
@@ -88,46 +86,11 @@ class TestProcessFilesLevel:
|
||||
|
||||
def test_process_files_level_batch(self, tmp_path: Path) -> None:
|
||||
"""Should process multiple files."""
|
||||
files = []
|
||||
file_list = []
|
||||
for i in range(3):
|
||||
test_file = tmp_path / f"test{i}.txt"
|
||||
test_file.write_text(f"content{i}")
|
||||
files.append(test_file)
|
||||
file_list.append(test_file)
|
||||
|
||||
filelevel.process_files_level(files, level=1)
|
||||
files.process_files_level(file_list, level=1)
|
||||
# All files should be processed
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
# main function
|
||||
# ---------------------------------------------------------------------- #
|
||||
class TestMain:
|
||||
"""Test main function."""
|
||||
|
||||
def test_main_set_command(self, tmp_path: Path) -> None:
|
||||
"""main() should handle set command."""
|
||||
test_file = tmp_path / "test.txt"
|
||||
test_file.write_text("test content")
|
||||
|
||||
with patch("sys.argv", ["filelevel", "set", str(test_file), "--level", "1"]), patch.object(
|
||||
px, "run"
|
||||
) as mock_run:
|
||||
filelevel.main()
|
||||
assert mock_run.called
|
||||
|
||||
def test_main_set_command_level_2(self, tmp_path: Path) -> None:
|
||||
"""main() should handle set command with level 2."""
|
||||
test_file = tmp_path / "test.txt"
|
||||
test_file.write_text("test content")
|
||||
|
||||
with patch("sys.argv", ["filelevel", "set", str(test_file), "--level", "2"]), patch.object(
|
||||
px, "run"
|
||||
) as mock_run:
|
||||
filelevel.main()
|
||||
assert mock_run.called
|
||||
|
||||
def test_main_with_no_args_shows_help(self) -> None:
|
||||
"""main() with no args should show help."""
|
||||
with patch("sys.argv", ["filelevel"]):
|
||||
filelevel.main()
|
||||
# Should print help and return
|
||||
|
||||
@@ -5,8 +5,7 @@ from __future__ import annotations
|
||||
from pathlib import Path
|
||||
from unittest.mock import patch
|
||||
|
||||
import pyflowx as px
|
||||
from pyflowx.cli import folderback
|
||||
from pyflowx.ops import files
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
@@ -22,7 +21,7 @@ class TestRemoveDump:
|
||||
dst = tmp_path / "backup"
|
||||
dst.mkdir()
|
||||
|
||||
folderback.remove_dump(src, dst, 5)
|
||||
files.remove_dump(src, dst, 5)
|
||||
# Should not raise error
|
||||
|
||||
def test_remove_dump_within_limit(self, tmp_path: Path) -> None:
|
||||
@@ -37,7 +36,7 @@ class TestRemoveDump:
|
||||
zip_file = dst / f"source_20240101_12000{i}.zip"
|
||||
zip_file.write_bytes(b"ZIP content")
|
||||
|
||||
folderback.remove_dump(src, dst, 5)
|
||||
files.remove_dump(src, dst, 5)
|
||||
# All files should remain
|
||||
assert len(list(dst.glob("*.zip"))) == 3
|
||||
|
||||
@@ -53,7 +52,7 @@ class TestRemoveDump:
|
||||
zip_file = dst / f"source_20240101_12000{i}.zip"
|
||||
zip_file.write_bytes(b"ZIP content")
|
||||
|
||||
folderback.remove_dump(src, dst, 5)
|
||||
files.remove_dump(src, dst, 5)
|
||||
# Should have only 5 files
|
||||
assert len(list(dst.glob("*.zip"))) == 5
|
||||
|
||||
@@ -73,7 +72,7 @@ class TestZipTarget:
|
||||
dst.mkdir()
|
||||
|
||||
with patch("time.strftime", return_value="_20240101_120000"):
|
||||
folderback.zip_target(src, dst, 5)
|
||||
files.zip_target(src, dst, 5)
|
||||
|
||||
# Should create zip file
|
||||
zip_files = list(dst.glob("*.zip"))
|
||||
@@ -91,7 +90,7 @@ class TestZipTarget:
|
||||
dst.mkdir()
|
||||
|
||||
with patch("time.strftime", return_value="_20240101_120000"):
|
||||
folderback.zip_target(src, dst, 5)
|
||||
files.zip_target(src, dst, 5)
|
||||
|
||||
# Should create zip file
|
||||
zip_files = list(dst.glob("*.zip"))
|
||||
@@ -111,8 +110,8 @@ class TestBackupFolder:
|
||||
(source_dir / "test.txt").write_text("test content")
|
||||
backup_dir = tmp_path / "backup"
|
||||
|
||||
with patch.object(folderback, "zip_target") as mock_zip:
|
||||
folderback.backup_folder(str(source_dir), str(backup_dir), 5)
|
||||
with patch.object(files, "zip_target") as mock_zip:
|
||||
files.backup_folder(str(source_dir), str(backup_dir), 5)
|
||||
assert mock_zip.called
|
||||
|
||||
def test_backup_folder_with_max_backups(self, tmp_path: Path) -> None:
|
||||
@@ -122,8 +121,8 @@ class TestBackupFolder:
|
||||
(source_dir / "test.txt").write_text("test content")
|
||||
backup_dir = tmp_path / "backup"
|
||||
|
||||
with patch.object(folderback, "zip_target") as mock_zip:
|
||||
folderback.backup_folder(str(source_dir), str(backup_dir), 10)
|
||||
with patch.object(files, "zip_target") as mock_zip:
|
||||
files.backup_folder(str(source_dir), str(backup_dir), 10)
|
||||
assert mock_zip.called
|
||||
|
||||
def test_backup_folder_source_not_exists(self, tmp_path: Path) -> None:
|
||||
@@ -132,7 +131,7 @@ class TestBackupFolder:
|
||||
backup_dir = tmp_path / "backup"
|
||||
backup_dir.mkdir()
|
||||
|
||||
folderback.backup_folder(str(source_dir), str(backup_dir), 5)
|
||||
files.backup_folder(str(source_dir), str(backup_dir), 5)
|
||||
# Should print error message and return
|
||||
|
||||
def test_backup_folder_creates_dst(self, tmp_path: Path) -> None:
|
||||
@@ -142,32 +141,19 @@ class TestBackupFolder:
|
||||
(source_dir / "test.txt").write_text("test content")
|
||||
backup_dir = tmp_path / "backup"
|
||||
|
||||
with patch.object(folderback, "zip_target") as mock_zip:
|
||||
folderback.backup_folder(str(source_dir), str(backup_dir), 5)
|
||||
with patch.object(files, "zip_target") as mock_zip:
|
||||
files.backup_folder(str(source_dir), str(backup_dir), 5)
|
||||
assert backup_dir.exists()
|
||||
assert mock_zip.called
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
# TaskSpec definitions
|
||||
# 函数注册
|
||||
# ---------------------------------------------------------------------- #
|
||||
class TestTaskSpecDefinitions:
|
||||
"""Test that all TaskSpec definitions are valid."""
|
||||
class TestRegisteredFunctions:
|
||||
"""Test that folderback functions are registered."""
|
||||
|
||||
def test_folderback_default_spec(self) -> None:
|
||||
"""folderback_default spec should be properly defined."""
|
||||
assert folderback.folderback_default.name == "folderback_default"
|
||||
assert folderback.folderback_default.fn is not None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
# main function
|
||||
# ---------------------------------------------------------------------- #
|
||||
class TestMain:
|
||||
"""Test main function."""
|
||||
|
||||
def test_main_calls_run_cli(self) -> None:
|
||||
"""main() should create a CliRunner and call run_cli()."""
|
||||
with patch.object(px.CliRunner, "run_cli") as mock_run_cli:
|
||||
folderback.main()
|
||||
assert mock_run_cli.called
|
||||
"""folderback_default should be a registered callable."""
|
||||
# folderback_default 现在是通过 @px.register_fn 注册的普通函数, 不是 TaskSpec
|
||||
assert callable(files.folderback_default)
|
||||
|
||||
+11
-25
@@ -5,8 +5,7 @@ from __future__ import annotations
|
||||
from pathlib import Path
|
||||
from unittest.mock import patch
|
||||
|
||||
import pyflowx as px
|
||||
from pyflowx.cli import folderzip
|
||||
from pyflowx.ops import files
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
@@ -22,7 +21,7 @@ class TestArchiveFolder:
|
||||
(folder / "test.txt").write_text("test content")
|
||||
|
||||
with patch("shutil.make_archive") as mock_archive:
|
||||
folderzip.archive_folder(folder)
|
||||
files.archive_folder(folder)
|
||||
assert mock_archive.called
|
||||
|
||||
|
||||
@@ -39,37 +38,24 @@ class TestZipFolders:
|
||||
(tmp_path / "folder2").mkdir()
|
||||
(tmp_path / ".git").mkdir() # Should be ignored
|
||||
|
||||
with patch.object(folderzip, "archive_folder") as mock_archive:
|
||||
folderzip.zip_folders(str(tmp_path))
|
||||
with patch.object(files, "archive_folder") as mock_archive:
|
||||
files.zip_folders(str(tmp_path))
|
||||
# Should archive folder1 and folder2, but not .git
|
||||
assert mock_archive.call_count == 2
|
||||
|
||||
def test_zip_folders_nonexistent_cwd(self, tmp_path: Path) -> None:
|
||||
"""Should handle nonexistent cwd."""
|
||||
folderzip.zip_folders(str(tmp_path / "nonexistent"))
|
||||
files.zip_folders(str(tmp_path / "nonexistent"))
|
||||
# Should print error message and return
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
# TaskSpec definitions
|
||||
# 函数注册
|
||||
# ---------------------------------------------------------------------- #
|
||||
class TestTaskSpecDefinitions:
|
||||
"""Test that all TaskSpec definitions are valid."""
|
||||
class TestRegisteredFunctions:
|
||||
"""Test that folderzip functions are registered."""
|
||||
|
||||
def test_folderzip_default_spec(self) -> None:
|
||||
"""folderzip_default spec should be properly defined."""
|
||||
assert folderzip.folderzip_default.name == "folderzip_default"
|
||||
assert folderzip.folderzip_default.fn is not None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
# main function
|
||||
# ---------------------------------------------------------------------- #
|
||||
class TestMain:
|
||||
"""Test main function."""
|
||||
|
||||
def test_main_calls_run_cli(self) -> None:
|
||||
"""main() should create a CliRunner and call run_cli()."""
|
||||
with patch.object(px.CliRunner, "run_cli") as mock_run_cli:
|
||||
folderzip.main()
|
||||
assert mock_run_cli.called
|
||||
"""folderzip_default should be a registered callable."""
|
||||
# folderzip_default 现在是通过 @px.register_fn 注册的普通函数, 不是 TaskSpec
|
||||
assert callable(files.folderzip_default)
|
||||
|
||||
+23
-66
@@ -8,7 +8,7 @@ from unittest.mock import patch
|
||||
import pytest
|
||||
|
||||
import pyflowx as px
|
||||
from pyflowx.cli import gittool
|
||||
from pyflowx.ops import dev
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
@@ -20,7 +20,7 @@ class TestNotHasGitRepo:
|
||||
def test_not_has_git_repo_true(self, tmp_path: Path) -> None:
|
||||
"""Should return True when no .git directory."""
|
||||
with patch.object(Path, "cwd", return_value=tmp_path):
|
||||
result = gittool.not_has_git_repo()
|
||||
result = dev.not_has_git_repo()
|
||||
assert result is True
|
||||
|
||||
def test_not_has_git_repo_false(self, tmp_path: Path) -> None:
|
||||
@@ -29,7 +29,7 @@ class TestNotHasGitRepo:
|
||||
git_dir.mkdir()
|
||||
|
||||
with patch.object(Path, "cwd", return_value=tmp_path):
|
||||
result = gittool.not_has_git_repo()
|
||||
result = dev.not_has_git_repo()
|
||||
assert result is False
|
||||
|
||||
def test_not_has_git_repo_cwd_not_exists(self, tmp_path: Path) -> None:
|
||||
@@ -37,7 +37,7 @@ class TestNotHasGitRepo:
|
||||
nonexistent = tmp_path / "nonexistent"
|
||||
|
||||
with patch.object(Path, "cwd", return_value=nonexistent):
|
||||
result = gittool.not_has_git_repo()
|
||||
result = dev.not_has_git_repo()
|
||||
assert result is True
|
||||
|
||||
|
||||
@@ -47,19 +47,25 @@ class TestNotHasGitRepo:
|
||||
class TestHasFiles:
|
||||
"""Test has_files function."""
|
||||
|
||||
def test_has_files_true(self, tmp_path: Path) -> None:
|
||||
"""Should return True when files exist."""
|
||||
(tmp_path / "test.txt").write_text("test")
|
||||
def test_has_files_true(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""Should return True when there are uncommitted changes."""
|
||||
|
||||
with patch.object(Path, "cwd", return_value=tmp_path):
|
||||
result = gittool.has_files()
|
||||
assert result is True
|
||||
class _FakeResult:
|
||||
stdout = " M test.txt\n"
|
||||
|
||||
def test_has_files_false(self, tmp_path: Path) -> None:
|
||||
"""Should return False when no files."""
|
||||
with patch.object(Path, "cwd", return_value=tmp_path):
|
||||
result = gittool.has_files()
|
||||
assert result is False
|
||||
monkeypatch.setattr("subprocess.run", lambda *_, **__: _FakeResult())
|
||||
result = dev.has_files()
|
||||
assert result is True
|
||||
|
||||
def test_has_files_false(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""Should return False when no uncommitted changes."""
|
||||
|
||||
class _FakeResult:
|
||||
stdout = ""
|
||||
|
||||
monkeypatch.setattr("subprocess.run", lambda *_, **__: _FakeResult())
|
||||
result = dev.has_files()
|
||||
assert result is False
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
@@ -76,62 +82,13 @@ class TestInitSubDirs:
|
||||
subdir2.mkdir()
|
||||
|
||||
with patch.object(Path, "cwd", return_value=tmp_path), patch.object(px, "run") as mock_run:
|
||||
gittool.init_sub_dirs()
|
||||
dev.init_sub_dirs()
|
||||
# Should call px.run for each subdirectory
|
||||
assert mock_run.call_count == 2
|
||||
|
||||
def test_init_sub_dirs_no_subdirectories(self, tmp_path: Path) -> None:
|
||||
"""Should handle no subdirectories."""
|
||||
with patch.object(Path, "cwd", return_value=tmp_path), patch.object(px, "run") as mock_run:
|
||||
gittool.init_sub_dirs()
|
||||
dev.init_sub_dirs()
|
||||
# Should not call px.run
|
||||
assert mock_run.call_count == 0
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
# TaskSpec definitions
|
||||
# ---------------------------------------------------------------------- #
|
||||
class TestTaskSpecDefinitions:
|
||||
"""Test that all TaskSpec definitions are valid."""
|
||||
|
||||
def test_push_spec(self) -> None:
|
||||
"""push spec should be properly defined."""
|
||||
assert gittool.push.name == "push"
|
||||
assert gittool.push.cmd == ["git", "push"]
|
||||
|
||||
def test_pull_spec(self) -> None:
|
||||
"""pull spec should be properly defined."""
|
||||
assert gittool.pull.name == "pull"
|
||||
assert gittool.pull.cmd == ["git", "pull"]
|
||||
|
||||
def test_kill_tgit_spec(self) -> None:
|
||||
"""kill_tgit spec should be properly defined."""
|
||||
assert gittool.kill_tgit.name == "task_kill"
|
||||
assert isinstance(gittool.kill_tgit.cmd, list)
|
||||
assert "taskkill" in gittool.kill_tgit.cmd
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
# main function
|
||||
# ---------------------------------------------------------------------- #
|
||||
class TestMain:
|
||||
"""Test main function."""
|
||||
|
||||
def test_main_calls_run_cli(self) -> None:
|
||||
"""main() should create a CliRunner and call run_cli()."""
|
||||
with pytest.raises(SystemExit) as exc_info:
|
||||
gittool.main()
|
||||
# run_cli() calls sys.exit(), so we should get SystemExit
|
||||
assert exc_info.value.code in (0, 1, 2)
|
||||
|
||||
def test_main_with_list_argument(self) -> None:
|
||||
"""main() should handle --list argument."""
|
||||
with patch("sys.argv", ["gittool", "--list"]), pytest.raises(SystemExit) as exc_info:
|
||||
gittool.main()
|
||||
assert exc_info.value.code == 0
|
||||
|
||||
def test_main_with_no_args_shows_help(self) -> None:
|
||||
"""main() with no args should show help and exit."""
|
||||
with patch("sys.argv", ["gittool"]), pytest.raises(SystemExit) as exc_info:
|
||||
gittool.main()
|
||||
assert exc_info.value.code == 1
|
||||
|
||||
+10
-57
@@ -5,9 +5,8 @@ from __future__ import annotations
|
||||
from pathlib import Path
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
import pyflowx as px
|
||||
from pyflowx.cli import lscalc
|
||||
from pyflowx.conditions import Constants
|
||||
from pyflowx.ops import system
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
@@ -19,7 +18,7 @@ class TestGetLsDynaCommand:
|
||||
def test_get_ls_dyna_command_windows(self) -> None:
|
||||
"""Should get LS-DYNA command for Windows."""
|
||||
with patch.object(Constants, "IS_WINDOWS", True), patch.object(Constants, "IS_MACOS", False):
|
||||
cmd = lscalc.get_ls_dyna_command("input.k", 4)
|
||||
cmd = system.get_ls_dyna_command("input.k", 4)
|
||||
assert "ls-dyna_mpp" in cmd
|
||||
assert "i=input.k" in cmd
|
||||
assert "ncpu=4" in cmd
|
||||
@@ -27,7 +26,7 @@ class TestGetLsDynaCommand:
|
||||
def test_get_ls_dyna_command_linux(self) -> None:
|
||||
"""Should get LS-DYNA command for Linux."""
|
||||
with patch.object(Constants, "IS_WINDOWS", False), patch.object(Constants, "IS_MACOS", False):
|
||||
cmd = lscalc.get_ls_dyna_command("input.k", 8)
|
||||
cmd = system.get_ls_dyna_command("input.k", 8)
|
||||
assert "ls-dyna_mpp" in cmd
|
||||
assert "i=input.k" in cmd
|
||||
assert "ncpu=8" in cmd
|
||||
@@ -46,14 +45,14 @@ class TestRunLsDyna:
|
||||
|
||||
with patch("subprocess.run") as mock_run:
|
||||
mock_run.return_value = MagicMock(returncode=0)
|
||||
lscalc.run_ls_dyna(str(input_file), ncpu=4)
|
||||
system.run_ls_dyna(str(input_file), ncpu=4)
|
||||
assert mock_run.called
|
||||
|
||||
def test_run_ls_dyna_file_not_found(self, tmp_path: Path) -> None:
|
||||
"""Should handle nonexistent input file."""
|
||||
input_file = tmp_path / "nonexistent.k"
|
||||
|
||||
lscalc.run_ls_dyna(str(input_file), ncpu=4)
|
||||
system.run_ls_dyna(str(input_file), ncpu=4)
|
||||
# Should print error message
|
||||
|
||||
def test_run_ls_dyna_command_not_found(self, tmp_path: Path) -> None:
|
||||
@@ -62,7 +61,7 @@ class TestRunLsDyna:
|
||||
input_file.write_text("LS-DYNA input")
|
||||
|
||||
with patch("subprocess.run", side_effect=FileNotFoundError):
|
||||
lscalc.run_ls_dyna(str(input_file), ncpu=4)
|
||||
system.run_ls_dyna(str(input_file), ncpu=4)
|
||||
# Should print error message
|
||||
|
||||
|
||||
@@ -79,14 +78,14 @@ class TestRunLsDynaMpi:
|
||||
|
||||
with patch("subprocess.run") as mock_run:
|
||||
mock_run.return_value = MagicMock(returncode=0)
|
||||
lscalc.run_ls_dyna_mpi(str(input_file), ncpu=8)
|
||||
system.run_ls_dyna_mpi(str(input_file), ncpu=8)
|
||||
assert mock_run.called
|
||||
|
||||
def test_run_ls_dyna_mpi_file_not_found(self, tmp_path: Path) -> None:
|
||||
"""Should handle nonexistent input file."""
|
||||
input_file = tmp_path / "nonexistent.k"
|
||||
|
||||
lscalc.run_ls_dyna_mpi(str(input_file), ncpu=8)
|
||||
system.run_ls_dyna_mpi(str(input_file), ncpu=8)
|
||||
# Should print error message
|
||||
|
||||
|
||||
@@ -100,58 +99,12 @@ class TestCheckLsDynaStatus:
|
||||
"""Should check LS-DYNA status on Windows."""
|
||||
with patch.object(Constants, "IS_WINDOWS", True), patch("subprocess.run") as mock_run:
|
||||
mock_run.return_value = MagicMock(stdout="ls-dyna_mpp.exe", returncode=0)
|
||||
lscalc.check_ls_dyna_status()
|
||||
system.check_ls_dyna_status()
|
||||
assert mock_run.called
|
||||
|
||||
def test_check_ls_dyna_status_linux(self) -> None:
|
||||
"""Should check LS-DYNA status on Linux."""
|
||||
with patch.object(Constants, "IS_WINDOWS", False), patch("subprocess.run") as mock_run:
|
||||
mock_run.return_value = MagicMock(stdout="1234", returncode=0)
|
||||
lscalc.check_ls_dyna_status()
|
||||
system.check_ls_dyna_status()
|
||||
assert mock_run.called
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
# main function
|
||||
# ---------------------------------------------------------------------- #
|
||||
class TestMain:
|
||||
"""Test main function."""
|
||||
|
||||
def test_main_run_command(self, tmp_path: Path) -> None:
|
||||
"""main() should handle run command."""
|
||||
input_file = tmp_path / "input.k"
|
||||
input_file.write_text("LS-DYNA input")
|
||||
|
||||
with patch("sys.argv", ["lscalc", "run", str(input_file)]), patch.object(px, "run") as mock_run:
|
||||
lscalc.main()
|
||||
assert mock_run.called
|
||||
|
||||
def test_main_run_command_with_ncpu(self, tmp_path: Path) -> None:
|
||||
"""main() should handle run command with ncpu."""
|
||||
input_file = tmp_path / "input.k"
|
||||
input_file.write_text("LS-DYNA input")
|
||||
|
||||
with patch("sys.argv", ["lscalc", "run", str(input_file), "--ncpu", "8"]), patch.object(px, "run") as mock_run:
|
||||
lscalc.main()
|
||||
assert mock_run.called
|
||||
|
||||
def test_main_mpi_command(self, tmp_path: Path) -> None:
|
||||
"""main() should handle mpi command."""
|
||||
input_file = tmp_path / "input.k"
|
||||
input_file.write_text("LS-DYNA input")
|
||||
|
||||
with patch("sys.argv", ["lscalc", "mpi", str(input_file)]), patch.object(px, "run") as mock_run:
|
||||
lscalc.main()
|
||||
assert mock_run.called
|
||||
|
||||
def test_main_status_command(self) -> None:
|
||||
"""main() should handle status command."""
|
||||
with patch("sys.argv", ["lscalc", "status"]), patch.object(px, "run") as mock_run:
|
||||
lscalc.main()
|
||||
assert mock_run.called
|
||||
|
||||
def test_main_with_no_args_shows_help(self) -> None:
|
||||
"""main() with no args should show help."""
|
||||
with patch("sys.argv", ["lscalc"]):
|
||||
lscalc.main()
|
||||
# Should print help and return
|
||||
|
||||
+16
-81
@@ -7,8 +7,7 @@ from unittest.mock import MagicMock, patch
|
||||
|
||||
import pytest
|
||||
|
||||
import pyflowx as px
|
||||
from pyflowx.cli import packtool
|
||||
from pyflowx.ops import system
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
@@ -20,7 +19,7 @@ def packtool_tmp_workdir(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Non
|
||||
monkeypatch: pytest 的 monkeypatch 工具
|
||||
"""
|
||||
# Mock DEFAULT_CACHE_DIR 到临时目录
|
||||
monkeypatch.setattr(packtool, "DEFAULT_CACHE_DIR", str(tmp_path / ".cache" / "pypack"))
|
||||
monkeypatch.setattr(system, "DEFAULT_CACHE_DIR", str(tmp_path / ".cache" / "pypack"))
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
@@ -36,7 +35,7 @@ class TestPackSource:
|
||||
(project_dir / "main.py").write_text("print('hello')")
|
||||
output_dir = tmp_path / "output"
|
||||
|
||||
packtool.pack_source(project_dir, output_dir)
|
||||
system.pack_source(project_dir, output_dir)
|
||||
assert output_dir.exists()
|
||||
|
||||
def test_pack_source_with_pyproject(self, tmp_path: Path) -> None:
|
||||
@@ -47,7 +46,7 @@ class TestPackSource:
|
||||
(project_dir / "main.py").write_text("print('hello')")
|
||||
output_dir = tmp_path / "output"
|
||||
|
||||
packtool.pack_source(project_dir, output_dir)
|
||||
system.pack_source(project_dir, output_dir)
|
||||
assert output_dir.exists()
|
||||
|
||||
|
||||
@@ -61,7 +60,7 @@ class TestPackDependencies:
|
||||
"""Should handle empty dependencies."""
|
||||
lib_dir = tmp_path / "libs"
|
||||
|
||||
packtool.pack_dependencies(lib_dir, [])
|
||||
system.pack_dependencies(lib_dir, [])
|
||||
# Should print message and return
|
||||
|
||||
def test_pack_dependencies_with_deps(self, tmp_path: Path) -> None:
|
||||
@@ -70,7 +69,7 @@ class TestPackDependencies:
|
||||
|
||||
with patch("subprocess.run") as mock_run:
|
||||
mock_run.return_value = MagicMock(returncode=0)
|
||||
packtool.pack_dependencies(lib_dir, ["numpy", "pandas"])
|
||||
system.pack_dependencies(lib_dir, ["numpy", "pandas"])
|
||||
assert mock_run.called
|
||||
|
||||
|
||||
@@ -89,7 +88,7 @@ class TestPackWheel:
|
||||
|
||||
with patch("subprocess.run") as mock_run:
|
||||
mock_run.return_value = MagicMock(returncode=0)
|
||||
packtool.pack_wheel(project_dir, output_dir)
|
||||
system.pack_wheel(project_dir, output_dir)
|
||||
assert mock_run.called
|
||||
|
||||
|
||||
@@ -112,7 +111,7 @@ class TestInstallEmbedPython:
|
||||
mock_zip_instance = MagicMock()
|
||||
mock_zipfile.return_value.__enter__.return_value = mock_zip_instance
|
||||
|
||||
packtool.install_embed_python("3.10", output_dir)
|
||||
system.install_embed_python("3.10", output_dir)
|
||||
|
||||
# Verify download was called
|
||||
assert mock_urlretrieve.called
|
||||
@@ -135,7 +134,7 @@ class TestInstallEmbedPython:
|
||||
mock_zip_instance = MagicMock()
|
||||
mock_zipfile.return_value.__enter__.return_value = mock_zip_instance
|
||||
|
||||
packtool.install_embed_python("3.10", output_dir)
|
||||
system.install_embed_python("3.10", output_dir)
|
||||
|
||||
# Verify extraction was called (using cache)
|
||||
assert mock_zip_instance.extractall.called
|
||||
@@ -158,7 +157,7 @@ class TestInstallEmbedPython:
|
||||
return
|
||||
|
||||
# Perform real installation
|
||||
packtool.install_embed_python("3.10", output_dir)
|
||||
system.install_embed_python("3.10", output_dir)
|
||||
|
||||
# Verify installation succeeded
|
||||
assert output_dir.exists()
|
||||
@@ -199,7 +198,7 @@ class TestInstallEmbedPython:
|
||||
|
||||
# Test different versions
|
||||
for version in ["3.8", "3.9", "3.10", "3.11", "3.12"]:
|
||||
packtool.install_embed_python(version, output_dir)
|
||||
system.install_embed_python(version, output_dir)
|
||||
assert mock_urlretrieve.called
|
||||
|
||||
def test_install_embed_python_creates_cache(self, tmp_path: Path) -> None:
|
||||
@@ -213,10 +212,10 @@ class TestInstallEmbedPython:
|
||||
mock_zip_instance = MagicMock()
|
||||
mock_zipfile.return_value.__enter__.return_value = mock_zip_instance
|
||||
|
||||
packtool.install_embed_python("3.10", output_dir)
|
||||
system.install_embed_python("3.10", output_dir)
|
||||
|
||||
# Verify cache directory was created (now in tmp_path)
|
||||
Path(packtool.DEFAULT_CACHE_DIR)
|
||||
Path(system.DEFAULT_CACHE_DIR)
|
||||
# Note: In test environment, cache might not persist due to mocking
|
||||
|
||||
|
||||
@@ -233,7 +232,7 @@ class TestCreateZipPackage:
|
||||
(source_dir / "test.txt").write_text("test content")
|
||||
output_file = tmp_path / "package.zip"
|
||||
|
||||
packtool.create_zip_package(source_dir, output_file)
|
||||
system.create_zip_package(source_dir, output_file)
|
||||
assert output_file.exists()
|
||||
|
||||
|
||||
@@ -249,76 +248,12 @@ class TestCleanBuildDir:
|
||||
build_dir.mkdir()
|
||||
(build_dir / "test.txt").write_text("test")
|
||||
|
||||
packtool.clean_build_dir(build_dir)
|
||||
system.clean_build_dir(build_dir)
|
||||
assert not build_dir.exists()
|
||||
|
||||
def test_clean_build_dir_not_exists(self, tmp_path: Path) -> None:
|
||||
"""Should handle nonexistent build directory."""
|
||||
build_dir = tmp_path / "nonexistent"
|
||||
|
||||
packtool.clean_build_dir(build_dir)
|
||||
system.clean_build_dir(build_dir)
|
||||
# Should print message
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
# main function
|
||||
# ---------------------------------------------------------------------- #
|
||||
class TestMain:
|
||||
"""Test main function."""
|
||||
|
||||
def test_main_src_command(self, tmp_path: Path) -> None:
|
||||
"""main() should handle src command."""
|
||||
project_dir = tmp_path / "project"
|
||||
project_dir.mkdir()
|
||||
|
||||
with patch("sys.argv", ["packtool", "src", "--project-dir", str(project_dir)]), patch.object(
|
||||
px, "run"
|
||||
) as mock_run:
|
||||
packtool.main()
|
||||
assert mock_run.called
|
||||
|
||||
def test_main_deps_command(self, tmp_path: Path) -> None:
|
||||
"""main() should handle deps command."""
|
||||
with patch("sys.argv", ["packtool", "deps", "numpy", "pandas"]), patch.object(px, "run") as mock_run:
|
||||
packtool.main()
|
||||
assert mock_run.called
|
||||
|
||||
def test_main_wheel_command(self, tmp_path: Path) -> None:
|
||||
"""main() should handle wheel command."""
|
||||
project_dir = tmp_path / "project"
|
||||
project_dir.mkdir()
|
||||
|
||||
with patch("sys.argv", ["packtool", "wheel", "--project-dir", str(project_dir)]), patch.object(
|
||||
px, "run"
|
||||
) as mock_run:
|
||||
packtool.main()
|
||||
assert mock_run.called
|
||||
|
||||
def test_main_embed_command(self, tmp_path: Path) -> None:
|
||||
"""main() should handle embed command."""
|
||||
with patch("sys.argv", ["packtool", "embed", "--version", "3.10"]), patch.object(px, "run") as mock_run:
|
||||
packtool.main()
|
||||
assert mock_run.called
|
||||
|
||||
def test_main_zip_command(self, tmp_path: Path) -> None:
|
||||
"""main() should handle zip command."""
|
||||
source_dir = tmp_path / "source"
|
||||
source_dir.mkdir()
|
||||
|
||||
with patch("sys.argv", ["packtool", "zip", "--source-dir", str(source_dir)]), patch.object(
|
||||
px, "run"
|
||||
) as mock_run:
|
||||
packtool.main()
|
||||
assert mock_run.called
|
||||
|
||||
def test_main_clean_command(self) -> None:
|
||||
"""main() should handle clean command."""
|
||||
with patch("sys.argv", ["packtool", "clean"]), patch.object(px, "run") as mock_run:
|
||||
packtool.main()
|
||||
assert mock_run.called
|
||||
|
||||
def test_main_with_no_args_shows_help(self) -> None:
|
||||
"""main() with no args should show help."""
|
||||
with patch("sys.argv", ["packtool"]):
|
||||
packtool.main()
|
||||
# Should print help and return
|
||||
|
||||
+13
-57
@@ -8,8 +8,7 @@ from unittest.mock import MagicMock, patch
|
||||
|
||||
import pytest
|
||||
|
||||
import pyflowx as px
|
||||
from pyflowx.cli import pdftool
|
||||
from pyflowx.ops import media
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
@@ -29,7 +28,7 @@ class TestPdfMerge:
|
||||
with patch("pypdf.PdfReader"), patch("pypdf.PdfWriter") as mock_writer:
|
||||
mock_writer_instance = MagicMock()
|
||||
mock_writer.return_value = mock_writer_instance
|
||||
pdftool.pdf_merge(input_files, output_file)
|
||||
media.pdf_merge(input_files, output_file)
|
||||
assert mock_writer_instance.write.called
|
||||
|
||||
|
||||
@@ -50,7 +49,7 @@ class TestPdfSplit:
|
||||
mock_reader_instance = MagicMock()
|
||||
mock_reader.return_value = mock_reader_instance
|
||||
mock_reader_instance.pages = [MagicMock()]
|
||||
pdftool.pdf_split(input_file, output_dir)
|
||||
media.pdf_split(input_file, output_dir)
|
||||
assert output_dir.exists()
|
||||
|
||||
|
||||
@@ -76,7 +75,7 @@ class TestPdfCompress:
|
||||
output_file.write_bytes(b"Compressed PDF")
|
||||
|
||||
mock_doc.save = mock_save
|
||||
pdftool.pdf_compress(input_file, output_file)
|
||||
media.pdf_compress(input_file, output_file)
|
||||
assert output_file.exists()
|
||||
|
||||
|
||||
@@ -99,7 +98,7 @@ class TestPdfExtractText:
|
||||
mock_page.get_text.return_value = "Test text"
|
||||
mock_doc.__iter__ = MagicMock(return_value=iter([mock_page]))
|
||||
mock_fitz_open.return_value = mock_doc
|
||||
pdftool.pdf_extract_text(input_file, output_file)
|
||||
media.pdf_extract_text(input_file, output_file)
|
||||
assert output_file.exists()
|
||||
|
||||
|
||||
@@ -123,7 +122,7 @@ class TestPdfExtractImages:
|
||||
mock_doc.__iter__ = MagicMock(return_value=iter([mock_page]))
|
||||
mock_doc.extract_image.return_value = {"image": b"image data", "ext": "png"}
|
||||
mock_fitz_open.return_value = mock_doc
|
||||
pdftool.pdf_extract_images(input_file, output_dir)
|
||||
media.pdf_extract_images(input_file, output_dir)
|
||||
assert output_dir.exists()
|
||||
|
||||
|
||||
@@ -147,7 +146,7 @@ class TestPdfAddWatermark:
|
||||
mock_doc.__iter__ = MagicMock(return_value=iter([mock_page]))
|
||||
mock_fitz_open.return_value = mock_doc
|
||||
mock_text_length.return_value = 100
|
||||
pdftool.pdf_add_watermark(input_file, output_file)
|
||||
media.pdf_add_watermark(input_file, output_file)
|
||||
assert mock_doc.save.called
|
||||
|
||||
|
||||
@@ -169,7 +168,7 @@ class TestPdfRotate:
|
||||
mock_page = MagicMock()
|
||||
mock_doc.__iter__ = MagicMock(return_value=iter([mock_page]))
|
||||
mock_fitz_open.return_value = mock_doc
|
||||
pdftool.pdf_rotate(input_file, output_file, rotation=90)
|
||||
media.pdf_rotate(input_file, output_file, rotation=90)
|
||||
assert mock_doc.save.called
|
||||
|
||||
def test_pdf_rotate_file_180(self, tmp_path: Path) -> None:
|
||||
@@ -184,7 +183,7 @@ class TestPdfRotate:
|
||||
mock_page = MagicMock()
|
||||
mock_doc.__iter__ = MagicMock(return_value=iter([mock_page]))
|
||||
mock_fitz_open.return_value = mock_doc
|
||||
pdftool.pdf_rotate(input_file, output_file, rotation=180)
|
||||
media.pdf_rotate(input_file, output_file, rotation=180)
|
||||
assert mock_doc.save.called
|
||||
|
||||
|
||||
@@ -207,7 +206,7 @@ class TestPdfCrop:
|
||||
mock_page.rect = MagicMock(x0=0, y0=0, x1=800, y1=600)
|
||||
mock_doc.__iter__ = MagicMock(return_value=iter([mock_page]))
|
||||
mock_fitz_open.return_value = mock_doc
|
||||
pdftool.pdf_crop(input_file, output_file, margins=(10, 10, 10, 10))
|
||||
media.pdf_crop(input_file, output_file, margins=(10, 10, 10, 10))
|
||||
assert mock_doc.save.called
|
||||
|
||||
|
||||
@@ -228,7 +227,7 @@ class TestPdfInfo:
|
||||
mock_doc.page_count = 10
|
||||
mock_doc.metadata = {"title": "Test", "author": "Author"}
|
||||
mock_fitz_open.return_value = mock_doc
|
||||
pdftool.pdf_info(input_file)
|
||||
media.pdf_info(input_file)
|
||||
assert mock_fitz_open.called
|
||||
|
||||
|
||||
@@ -257,7 +256,7 @@ class TestPdfOcr:
|
||||
mock_doc.__iter__ = MagicMock(return_value=iter([mock_page]))
|
||||
mock_fitz_open.return_value = mock_doc
|
||||
mock_ocr.return_value = "OCR text"
|
||||
pdftool.pdf_ocr(input_file, output_file)
|
||||
media.pdf_ocr(input_file, output_file)
|
||||
# Should complete OCR
|
||||
|
||||
|
||||
@@ -277,48 +276,5 @@ class TestPdfRepair:
|
||||
with patch("fitz.open") as mock_fitz_open:
|
||||
mock_doc = MagicMock()
|
||||
mock_fitz_open.return_value = mock_doc
|
||||
pdftool.pdf_repair(input_file, output_file)
|
||||
media.pdf_repair(input_file, output_file)
|
||||
assert mock_doc.save.called
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
# main function
|
||||
# ---------------------------------------------------------------------- #
|
||||
class TestMain:
|
||||
"""Test main function."""
|
||||
|
||||
def test_main_merge_command(self, tmp_path: Path) -> None:
|
||||
"""main() should handle merge command."""
|
||||
input_files = [tmp_path / "input1.pdf", tmp_path / "input2.pdf"]
|
||||
for f in input_files:
|
||||
f.write_bytes(b"PDF content")
|
||||
|
||||
with patch("sys.argv", ["pdftool", "m", str(input_files[0]), str(input_files[1])]), patch.object(
|
||||
px, "run"
|
||||
) as mock_run:
|
||||
pdftool.main()
|
||||
assert mock_run.called
|
||||
|
||||
def test_main_split_command(self, tmp_path: Path) -> None:
|
||||
"""main() should handle split command."""
|
||||
input_file = tmp_path / "input.pdf"
|
||||
input_file.write_bytes(b"PDF content")
|
||||
|
||||
with patch("sys.argv", ["pdftool", "s", str(input_file)]), patch.object(px, "run") as mock_run:
|
||||
pdftool.main()
|
||||
assert mock_run.called
|
||||
|
||||
def test_main_compress_command(self, tmp_path: Path) -> None:
|
||||
"""main() should handle compress command."""
|
||||
input_file = tmp_path / "input.pdf"
|
||||
input_file.write_bytes(b"PDF content")
|
||||
|
||||
with patch("sys.argv", ["pdftool", "c", str(input_file)]), patch.object(px, "run") as mock_run:
|
||||
pdftool.main()
|
||||
assert mock_run.called
|
||||
|
||||
def test_main_with_no_args_shows_help(self) -> None:
|
||||
"""main() with no args should show help."""
|
||||
with patch("sys.argv", ["pdftool"]):
|
||||
pdftool.main()
|
||||
# Should print help and return
|
||||
|
||||
+28
-78
@@ -6,8 +6,7 @@ import subprocess
|
||||
from pathlib import Path
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
import pyflowx as px
|
||||
from pyflowx.cli import piptool
|
||||
from pyflowx.ops import dev
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
@@ -20,7 +19,7 @@ class TestGetInstalledPackages:
|
||||
"""Should get installed packages."""
|
||||
with patch("subprocess.run") as mock_run:
|
||||
mock_run.return_value = MagicMock(stdout="numpy==1.0.0\npandas==2.0.0\n", returncode=0)
|
||||
result = piptool._get_installed_packages()
|
||||
result = dev._get_installed_packages()
|
||||
assert "numpy" in result
|
||||
assert "pandas" in result
|
||||
|
||||
@@ -28,19 +27,19 @@ class TestGetInstalledPackages:
|
||||
"""Should handle empty output."""
|
||||
with patch("subprocess.run") as mock_run:
|
||||
mock_run.return_value = MagicMock(stdout="", returncode=0)
|
||||
result = piptool._get_installed_packages()
|
||||
result = dev._get_installed_packages()
|
||||
assert result == []
|
||||
|
||||
def test_get_installed_packages_error(self) -> None:
|
||||
"""Should handle subprocess error."""
|
||||
with patch("subprocess.run", side_effect=subprocess.SubprocessError):
|
||||
result = piptool._get_installed_packages()
|
||||
result = dev._get_installed_packages()
|
||||
assert result == []
|
||||
|
||||
def test_get_installed_packages_oserror(self) -> None:
|
||||
"""Should handle OSError."""
|
||||
with patch("subprocess.run", side_effect=OSError):
|
||||
result = piptool._get_installed_packages()
|
||||
result = dev._get_installed_packages()
|
||||
assert result == []
|
||||
|
||||
|
||||
@@ -52,26 +51,26 @@ class TestExpandWildcardPackages:
|
||||
|
||||
def test_expand_wildcard_no_pattern(self) -> None:
|
||||
"""Should return package name when no wildcard."""
|
||||
result = piptool._expand_wildcard_packages("numpy")
|
||||
result = dev._expand_wildcard_packages("numpy")
|
||||
assert result == ["numpy"]
|
||||
|
||||
def test_expand_wildcard_with_star(self) -> None:
|
||||
"""Should expand wildcard with star."""
|
||||
with patch.object(piptool, "_get_installed_packages", return_value=["numpy", "numpy-core", "pandas"]):
|
||||
result = piptool._expand_wildcard_packages("numpy*")
|
||||
with patch.object(dev, "_get_installed_packages", return_value=["numpy", "numpy-core", "pandas"]):
|
||||
result = dev._expand_wildcard_packages("numpy*")
|
||||
assert "numpy" in result
|
||||
assert "numpy-core" in result
|
||||
|
||||
def test_expand_wildcard_with_question(self) -> None:
|
||||
"""Should expand wildcard with question mark."""
|
||||
with patch.object(piptool, "_get_installed_packages", return_value=["numpy", "numba"]):
|
||||
result = piptool._expand_wildcard_packages("num??")
|
||||
with patch.object(dev, "_get_installed_packages", return_value=["numpy", "numba"]):
|
||||
result = dev._expand_wildcard_packages("num??")
|
||||
assert len(result) > 0
|
||||
|
||||
def test_expand_wildcard_no_match(self) -> None:
|
||||
"""Should return empty list when no match."""
|
||||
with patch.object(piptool, "_get_installed_packages", return_value=["pandas", "scipy"]):
|
||||
result = piptool._expand_wildcard_packages("numpy*")
|
||||
with patch.object(dev, "_get_installed_packages", return_value=["pandas", "scipy"]):
|
||||
result = dev._expand_wildcard_packages("numpy*")
|
||||
assert result == []
|
||||
|
||||
|
||||
@@ -83,19 +82,19 @@ class TestFilterProtectedPackages:
|
||||
|
||||
def test_filter_protected_packages_normal(self) -> None:
|
||||
"""Should filter protected packages."""
|
||||
result = piptool._filter_protected_packages(["numpy", "pandas", "pyflowx"])
|
||||
result = dev._filter_protected_packages(["numpy", "pandas", "pyflowx"])
|
||||
assert "numpy" in result
|
||||
assert "pandas" in result
|
||||
assert "pyflowx" not in result
|
||||
|
||||
def test_filter_protected_packages_all_protected(self) -> None:
|
||||
"""Should filter all protected packages."""
|
||||
result = piptool._filter_protected_packages(["pyflowx", "bitool"])
|
||||
result = dev._filter_protected_packages(["pyflowx", "bitool"])
|
||||
assert result == []
|
||||
|
||||
def test_filter_protected_packages_case_insensitive(self) -> None:
|
||||
"""Should filter case insensitive."""
|
||||
result = piptool._filter_protected_packages(["PyFlowX", "BITOOL"])
|
||||
result = dev._filter_protected_packages(["PyFlowX", "BITOOL"])
|
||||
assert result == []
|
||||
|
||||
|
||||
@@ -109,35 +108,35 @@ class TestPipUninstall:
|
||||
"""Should uninstall single package."""
|
||||
with patch("subprocess.run") as mock_run:
|
||||
mock_run.return_value = MagicMock(returncode=0)
|
||||
piptool.pip_uninstall(["numpy"])
|
||||
dev.pip_uninstall(["numpy"])
|
||||
assert mock_run.called
|
||||
|
||||
def test_pip_uninstall_multiple_packages(self) -> None:
|
||||
"""Should uninstall multiple packages."""
|
||||
with patch("subprocess.run") as mock_run:
|
||||
mock_run.return_value = MagicMock(returncode=0)
|
||||
piptool.pip_uninstall(["numpy", "pandas", "scipy"])
|
||||
dev.pip_uninstall(["numpy", "pandas", "scipy"])
|
||||
# Should call pip uninstall
|
||||
assert mock_run.called
|
||||
|
||||
def test_pip_uninstall_with_wildcard(self) -> None:
|
||||
"""Should handle wildcard in package name."""
|
||||
with patch.object(piptool, "_expand_wildcard_packages", return_value=["numpy", "numpy-core"]), patch(
|
||||
with patch.object(dev, "_expand_wildcard_packages", return_value=["numpy", "numpy-core"]), patch(
|
||||
"subprocess.run"
|
||||
) as mock_run:
|
||||
mock_run.return_value = MagicMock(returncode=0)
|
||||
piptool.pip_uninstall(["numpy*"])
|
||||
dev.pip_uninstall(["numpy*"])
|
||||
assert mock_run.called
|
||||
|
||||
def test_pip_uninstall_empty_packages(self) -> None:
|
||||
"""Should handle empty packages list."""
|
||||
with patch.object(piptool, "_expand_wildcard_packages", return_value=[]):
|
||||
piptool.pip_uninstall(["nonexistent*"])
|
||||
with patch.object(dev, "_expand_wildcard_packages", return_value=[]):
|
||||
dev.pip_uninstall(["nonexistent*"])
|
||||
# Should not call subprocess.run
|
||||
|
||||
def test_pip_uninstall_all_protected(self) -> None:
|
||||
"""Should handle all protected packages."""
|
||||
piptool.pip_uninstall(["pyflowx"])
|
||||
dev.pip_uninstall(["pyflowx"])
|
||||
# Should not call subprocess.run
|
||||
|
||||
|
||||
@@ -151,7 +150,7 @@ class TestPipReinstall:
|
||||
"""Should reinstall single package."""
|
||||
with patch("subprocess.run") as mock_run:
|
||||
mock_run.return_value = MagicMock(returncode=0)
|
||||
piptool.pip_reinstall(["numpy"])
|
||||
dev.pip_reinstall(["numpy"])
|
||||
# Should call pip uninstall and pip install
|
||||
assert mock_run.call_count == 2
|
||||
|
||||
@@ -159,13 +158,13 @@ class TestPipReinstall:
|
||||
"""Should reinstall packages offline."""
|
||||
with patch("subprocess.run") as mock_run:
|
||||
mock_run.return_value = MagicMock(returncode=0)
|
||||
piptool.pip_reinstall(["numpy"], offline=True)
|
||||
dev.pip_reinstall(["numpy"], offline=True)
|
||||
# Should call pip install with offline flags
|
||||
assert mock_run.called
|
||||
|
||||
def test_pip_reinstall_all_protected(self) -> None:
|
||||
"""Should handle all protected packages."""
|
||||
piptool.pip_reinstall(["pyflowx"])
|
||||
dev.pip_reinstall(["pyflowx"])
|
||||
# Should not call subprocess.run
|
||||
|
||||
|
||||
@@ -179,14 +178,14 @@ class TestPipDownload:
|
||||
"""Should download single package."""
|
||||
with patch("subprocess.run") as mock_run:
|
||||
mock_run.return_value = MagicMock(returncode=0)
|
||||
piptool.pip_download(["numpy"])
|
||||
dev.pip_download(["numpy"])
|
||||
assert mock_run.called
|
||||
|
||||
def test_pip_download_offline(self) -> None:
|
||||
"""Should download packages offline."""
|
||||
with patch("subprocess.run") as mock_run:
|
||||
mock_run.return_value = MagicMock(returncode=0)
|
||||
piptool.pip_download(["numpy"], offline=True)
|
||||
dev.pip_download(["numpy"], offline=True)
|
||||
# Should call pip download with offline flags
|
||||
assert mock_run.called
|
||||
|
||||
@@ -201,54 +200,5 @@ class TestPipFreeze:
|
||||
"""Should freeze dependencies."""
|
||||
with patch("subprocess.run") as mock_run:
|
||||
mock_run.return_value = MagicMock(stdout="numpy==1.0.0\npandas==2.0.0", returncode=0)
|
||||
piptool.pip_freeze()
|
||||
dev.pip_freeze()
|
||||
assert mock_run.called
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
# main function
|
||||
# ---------------------------------------------------------------------- #
|
||||
class TestMain:
|
||||
"""Test main function."""
|
||||
|
||||
def test_main_install_command(self) -> None:
|
||||
"""main() should handle install command."""
|
||||
with patch("sys.argv", ["piptool", "i", "numpy", "pandas"]), patch.object(px, "run") as mock_run:
|
||||
piptool.main()
|
||||
assert mock_run.called
|
||||
|
||||
def test_main_uninstall_command(self) -> None:
|
||||
"""main() should handle uninstall command."""
|
||||
with patch("sys.argv", ["piptool", "u", "numpy"]), patch.object(px, "run") as mock_run:
|
||||
piptool.main()
|
||||
assert mock_run.called
|
||||
|
||||
def test_main_reinstall_command(self) -> None:
|
||||
"""main() should handle reinstall command."""
|
||||
with patch("sys.argv", ["piptool", "r", "numpy"]), patch.object(px, "run") as mock_run:
|
||||
piptool.main()
|
||||
assert mock_run.called
|
||||
|
||||
def test_main_download_command(self) -> None:
|
||||
"""main() should handle download command."""
|
||||
with patch("sys.argv", ["piptool", "d", "numpy"]), patch.object(px, "run") as mock_run:
|
||||
piptool.main()
|
||||
assert mock_run.called
|
||||
|
||||
def test_main_upgrade_command(self) -> None:
|
||||
"""main() should handle upgrade command."""
|
||||
with patch("sys.argv", ["piptool", "up"]), patch.object(px, "run") as mock_run:
|
||||
piptool.main()
|
||||
assert mock_run.called
|
||||
|
||||
def test_main_freeze_command(self) -> None:
|
||||
"""main() should handle freeze command."""
|
||||
with patch("sys.argv", ["piptool", "f"]), patch.object(px, "run") as mock_run:
|
||||
piptool.main()
|
||||
assert mock_run.called
|
||||
|
||||
def test_main_with_no_args_shows_help(self) -> None:
|
||||
"""main() with no args should show help."""
|
||||
with patch("sys.argv", ["piptool"]):
|
||||
piptool.main()
|
||||
# Should print help and return
|
||||
|
||||
@@ -56,10 +56,12 @@ def _build_simple_profile() -> ProfileReport:
|
||||
report = px.RunReport()
|
||||
report.results["a"] = _result("a", start, 1.0)
|
||||
report.results["b"] = _result("b", start + timedelta(seconds=1), 2.0)
|
||||
graph = px.Graph.from_specs([
|
||||
_spec("a"),
|
||||
_spec("b", deps=("a",)),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
_spec("a"),
|
||||
_spec("b", deps=("a",)),
|
||||
]
|
||||
)
|
||||
return ProfileReport.from_report(report, graph)
|
||||
|
||||
|
||||
|
||||
@@ -1,175 +0,0 @@
|
||||
"""Tests for cli.pymake module."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
from unittest.mock import patch
|
||||
|
||||
import pytest
|
||||
|
||||
from pyflowx.cli import pymake
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
# TaskSpec definitions
|
||||
# ---------------------------------------------------------------------- #
|
||||
def _find_task(name: str) -> pymake.px.TaskSpec:
|
||||
"""从 pymake.tasks 或 aliases 中查找指定名称的 TaskSpec."""
|
||||
for spec in pymake.tasks:
|
||||
if spec.name == name:
|
||||
return spec
|
||||
# 单任务别名(doc/lint/tox)内联在 aliases dict 中
|
||||
value = pymake.aliases.get(name)
|
||||
if isinstance(value, pymake.px.TaskSpec):
|
||||
return value
|
||||
raise KeyError(f"任务 {name!r} 未找到")
|
||||
|
||||
|
||||
class TestTaskSpecDefinitions:
|
||||
"""Test that all TaskSpec definitions are valid."""
|
||||
|
||||
def test_uv_build_spec(self) -> None:
|
||||
"""uv_build spec should be properly defined."""
|
||||
spec = _find_task("uv_build")
|
||||
assert spec.name == "uv_build"
|
||||
assert spec.cmd == ["uv", "build"]
|
||||
assert spec.skip_if_missing is False
|
||||
|
||||
def test_maturin_build_spec(self) -> None:
|
||||
"""maturin_build spec should be properly defined."""
|
||||
spec = _find_task("maturin_build")
|
||||
assert spec.name == "maturin_build"
|
||||
assert isinstance(spec.cmd, list)
|
||||
assert spec.skip_if_missing is False
|
||||
|
||||
def test_uv_sync_spec(self) -> None:
|
||||
"""uv_sync spec should be properly defined."""
|
||||
spec = _find_task("uv_sync")
|
||||
assert spec.name == "uv_sync"
|
||||
assert spec.cmd == ["uv", "sync"]
|
||||
assert spec.skip_if_missing is False
|
||||
|
||||
def test_git_clean_spec(self) -> None:
|
||||
"""git_clean spec should be properly defined."""
|
||||
spec = _find_task("git_clean")
|
||||
assert spec.name == "git_clean"
|
||||
assert spec.cmd == ["gitt", "c"]
|
||||
assert spec.skip_if_missing is False
|
||||
|
||||
def test_test_spec(self) -> None:
|
||||
"""test spec should be properly defined."""
|
||||
spec = _find_task("test")
|
||||
assert spec.name == "test"
|
||||
assert isinstance(spec.cmd, list)
|
||||
assert "pytest" in spec.cmd
|
||||
assert "-m" in spec.cmd
|
||||
assert "not slow" in spec.cmd
|
||||
assert spec.skip_if_missing is False
|
||||
|
||||
def test_test_fast_spec(self) -> None:
|
||||
"""test_fast spec should be properly defined."""
|
||||
spec = _find_task("test_fast")
|
||||
assert spec.name == "test_fast"
|
||||
assert isinstance(spec.cmd, list)
|
||||
assert "pytest" in spec.cmd
|
||||
assert "-n" not in spec.cmd # test_fast doesn't use parallel
|
||||
assert spec.skip_if_missing is False
|
||||
|
||||
def test_test_coverage_spec(self) -> None:
|
||||
"""test_coverage spec should be properly defined."""
|
||||
spec = _find_task("test_coverage")
|
||||
assert spec.name == "test_coverage"
|
||||
assert isinstance(spec.cmd, list)
|
||||
assert "pytest" in spec.cmd
|
||||
assert "--cov" in spec.cmd
|
||||
assert spec.skip_if_missing is False
|
||||
|
||||
def test_ruff_lint_spec(self) -> None:
|
||||
"""lint spec should be properly defined."""
|
||||
spec = _find_task("lint")
|
||||
assert spec.name == "lint"
|
||||
assert isinstance(spec.cmd, list)
|
||||
assert "ruff" in spec.cmd
|
||||
assert "check" in spec.cmd
|
||||
assert spec.skip_if_missing is False
|
||||
|
||||
def test_doc_spec(self) -> None:
|
||||
"""doc spec should be properly defined."""
|
||||
spec = _find_task("doc")
|
||||
assert spec.name == "doc"
|
||||
assert isinstance(spec.cmd, list)
|
||||
assert "sphinx-build" in spec.cmd
|
||||
assert spec.skip_if_missing is False
|
||||
|
||||
def test_hatch_publish_spec(self) -> None:
|
||||
"""publish_python spec should be properly defined."""
|
||||
spec = _find_task("publish_python")
|
||||
assert spec.name == "publish_python"
|
||||
assert spec.cmd == ["hatch", "publish"]
|
||||
assert spec.skip_if_missing is False
|
||||
|
||||
def test_twine_publish_spec(self) -> None:
|
||||
"""twine_publish spec should be properly defined."""
|
||||
spec = _find_task("twine_publish")
|
||||
assert spec.name == "twine_publish"
|
||||
assert isinstance(spec.cmd, list)
|
||||
assert "twine" in spec.cmd
|
||||
assert "upload" in spec.cmd
|
||||
assert spec.skip_if_missing is False
|
||||
|
||||
def test_tox_spec(self) -> None:
|
||||
"""tox spec should be properly defined."""
|
||||
spec = _find_task("tox")
|
||||
assert spec.name == "tox"
|
||||
assert spec.cmd == ["tox", "-p", "auto"]
|
||||
assert spec.skip_if_missing is False
|
||||
|
||||
def test_all_tasks_have_correct_cwd(self) -> None:
|
||||
"""所有任务应该有正确的 cwd 设置(指向项目根目录)."""
|
||||
# 验证 ROOT_DIR 定义正确(向上三层到达项目根目录)
|
||||
expected_root = Path(__file__).parent.parent.parent
|
||||
assert expected_root == pymake.ROOT_DIR
|
||||
|
||||
# 验证 tasks 中的所有命令任务都有正确的 cwd
|
||||
for spec in pymake.tasks:
|
||||
if spec.cmd is not None:
|
||||
assert spec.cwd == pymake.ROOT_DIR, f"任务 {spec.name} 的 cwd 应为 {pymake.ROOT_DIR}"
|
||||
|
||||
# 验证 aliases 中的内联任务(doc/lint/tox)也有正确的 cwd
|
||||
for name in ("doc", "lint", "tox"):
|
||||
spec = _find_task(name)
|
||||
assert spec.cwd == pymake.ROOT_DIR, f"任务 {name} 的 cwd 应为 {pymake.ROOT_DIR}"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
# main function
|
||||
# ---------------------------------------------------------------------- #
|
||||
class TestMain:
|
||||
"""Test main function."""
|
||||
|
||||
def test_main_calls_run_cli(self) -> None:
|
||||
"""main() should create a CliRunner and call run_cli()."""
|
||||
with pytest.raises(SystemExit) as exc_info:
|
||||
pymake.main()
|
||||
# run_cli() calls sys.exit(), so we should get SystemExit
|
||||
# The exit code depends on whether any commands are available
|
||||
assert exc_info.value.code in (0, 1, 2)
|
||||
|
||||
def test_main_with_list_argument(self) -> None:
|
||||
"""main() should handle --list argument."""
|
||||
with patch("sys.argv", ["pymake", "--list"]), pytest.raises(SystemExit) as exc_info:
|
||||
pymake.main()
|
||||
assert exc_info.value.code == 0
|
||||
|
||||
def test_main_creates_runner_with_multiple_commands(self) -> None:
|
||||
"""main() should create a CliRunner with multiple commands."""
|
||||
# We can't easily test the runner creation without mocking,
|
||||
# but we can verify that main() doesn't raise an error for --list
|
||||
with patch("sys.argv", ["pymake", "--list"]), pytest.raises(SystemExit):
|
||||
pymake.main()
|
||||
|
||||
def test_main_with_no_args_shows_help(self) -> None:
|
||||
"""main() with no args should show help and exit with failure."""
|
||||
with patch("sys.argv", ["pymake"]), pytest.raises(SystemExit) as exc_info:
|
||||
pymake.main()
|
||||
assert exc_info.value.code == 1
|
||||
@@ -0,0 +1,58 @@
|
||||
"""Tests for reseticoncache tool (reset_icon_cache_run 函数)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import subprocess
|
||||
|
||||
import pytest
|
||||
|
||||
from pyflowx.conditions import Constants
|
||||
from pyflowx.ops.system import reset_icon_cache_run
|
||||
|
||||
|
||||
class TestResetIconCacheRun:
|
||||
"""``reset_icon_cache_run`` 函数测试."""
|
||||
|
||||
def test_non_windows_skips(self, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]) -> None:
|
||||
"""非 Windows 平台应打印提示并跳过."""
|
||||
monkeypatch.setattr(Constants, "IS_WINDOWS", False)
|
||||
reset_icon_cache_run()
|
||||
captured = capsys.readouterr()
|
||||
assert "仅在 Windows 上支持" in captured.out
|
||||
|
||||
def test_windows_no_localappdata_skips(
|
||||
self, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
|
||||
) -> None:
|
||||
"""Windows 上 LOCALAPPDATA 未设置时应跳过."""
|
||||
monkeypatch.setattr(Constants, "IS_WINDOWS", True)
|
||||
monkeypatch.setattr("os.environ.get", lambda key, default="": "" if key == "LOCALAPPDATA" else default)
|
||||
reset_icon_cache_run()
|
||||
captured = capsys.readouterr()
|
||||
assert "LOCALAPPDATA" in captured.out
|
||||
|
||||
def test_windows_executes_all_steps(
|
||||
self, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
|
||||
) -> None:
|
||||
"""Windows 上应依次执行 kill/delete/delete_all/restart 四步."""
|
||||
monkeypatch.setattr(Constants, "IS_WINDOWS", True)
|
||||
local_app_data = "/tmp/fake_local_app_data"
|
||||
monkeypatch.setattr(
|
||||
"os.environ.get", lambda key, default="": local_app_data if key == "LOCALAPPDATA" else default
|
||||
)
|
||||
|
||||
# 让 icon_cache_db 和 explorer_cache_dir 都不存在, 跳过删除分支
|
||||
from pathlib import Path
|
||||
|
||||
monkeypatch.setattr(Path, "exists", lambda _self: False)
|
||||
|
||||
ran_cmds: list[list[str]] = []
|
||||
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: ran_cmds.append(cmd))
|
||||
|
||||
reset_icon_cache_run()
|
||||
captured = capsys.readouterr()
|
||||
|
||||
# 验证执行了 kill_explorer 和 restart_explorer (delete 分支因 exists=False 被跳过)
|
||||
assert any("taskkill" in str(cmd) for cmd in ran_cmds)
|
||||
assert any("explorer.exe" in str(cmd) for cmd in ran_cmds)
|
||||
assert any("start" in str(cmd) and "explorer.exe" in str(cmd) for cmd in ran_cmds)
|
||||
assert "图标缓存已重置" in captured.out
|
||||
@@ -5,9 +5,8 @@ from __future__ import annotations
|
||||
from pathlib import Path
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
import pyflowx as px
|
||||
from pyflowx.cli import screenshot
|
||||
from pyflowx.conditions import Constants
|
||||
from pyflowx.ops import media
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
@@ -19,13 +18,13 @@ class TestGetScreenshotPath:
|
||||
def test_get_screenshot_path_with_filename(self, tmp_path: Path) -> None:
|
||||
"""Should get screenshot path with filename."""
|
||||
with patch.object(Path, "home", return_value=tmp_path):
|
||||
result = screenshot.get_screenshot_path("test.png")
|
||||
result = media.get_screenshot_path("test.png")
|
||||
assert result.name == "test.png"
|
||||
|
||||
def test_get_screenshot_path_without_filename(self, tmp_path: Path) -> None:
|
||||
"""Should get screenshot path without filename."""
|
||||
with patch.object(Path, "home", return_value=tmp_path):
|
||||
result = screenshot.get_screenshot_path()
|
||||
result = media.get_screenshot_path()
|
||||
assert "screenshot_" in result.name
|
||||
assert result.suffix == ".png"
|
||||
|
||||
@@ -42,7 +41,7 @@ class TestTakeScreenshotFull:
|
||||
Path, "home", return_value=tmp_path
|
||||
), patch("subprocess.run") as mock_run:
|
||||
mock_run.return_value = MagicMock(returncode=0)
|
||||
screenshot.take_screenshot_full()
|
||||
media.take_screenshot_full()
|
||||
assert mock_run.called
|
||||
|
||||
def test_take_screenshot_full_macos(self, tmp_path: Path) -> None:
|
||||
@@ -51,7 +50,7 @@ class TestTakeScreenshotFull:
|
||||
Path, "home", return_value=tmp_path
|
||||
), patch("subprocess.run") as mock_run:
|
||||
mock_run.return_value = MagicMock(returncode=0)
|
||||
screenshot.take_screenshot_full()
|
||||
media.take_screenshot_full()
|
||||
assert mock_run.called
|
||||
|
||||
def test_take_screenshot_full_linux(self, tmp_path: Path) -> None:
|
||||
@@ -60,7 +59,7 @@ class TestTakeScreenshotFull:
|
||||
Path, "home", return_value=tmp_path
|
||||
), patch("subprocess.run") as mock_run:
|
||||
mock_run.return_value = MagicMock(returncode=0)
|
||||
screenshot.take_screenshot_full()
|
||||
media.take_screenshot_full()
|
||||
assert mock_run.called
|
||||
|
||||
|
||||
@@ -76,7 +75,7 @@ class TestTakeScreenshotArea:
|
||||
Path, "home", return_value=tmp_path
|
||||
), patch("subprocess.run") as mock_run:
|
||||
mock_run.return_value = MagicMock(returncode=0)
|
||||
screenshot.take_screenshot_area()
|
||||
media.take_screenshot_area()
|
||||
assert mock_run.called
|
||||
|
||||
def test_take_screenshot_area_macos(self, tmp_path: Path) -> None:
|
||||
@@ -85,7 +84,7 @@ class TestTakeScreenshotArea:
|
||||
Path, "home", return_value=tmp_path
|
||||
), patch("subprocess.run") as mock_run:
|
||||
mock_run.return_value = MagicMock(returncode=0)
|
||||
screenshot.take_screenshot_area()
|
||||
media.take_screenshot_area()
|
||||
assert mock_run.called
|
||||
|
||||
def test_take_screenshot_area_linux(self, tmp_path: Path) -> None:
|
||||
@@ -94,30 +93,5 @@ class TestTakeScreenshotArea:
|
||||
Path, "home", return_value=tmp_path
|
||||
), patch("subprocess.run") as mock_run:
|
||||
mock_run.return_value = MagicMock(returncode=0)
|
||||
screenshot.take_screenshot_area()
|
||||
media.take_screenshot_area()
|
||||
assert mock_run.called
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
# main function
|
||||
# ---------------------------------------------------------------------- #
|
||||
class TestMain:
|
||||
"""Test main function."""
|
||||
|
||||
def test_main_full_command(self, tmp_path: Path) -> None:
|
||||
"""main() should handle full command."""
|
||||
with patch("sys.argv", ["screenshot", "full"]), patch.object(px, "run") as mock_run:
|
||||
screenshot.main()
|
||||
assert mock_run.called
|
||||
|
||||
def test_main_area_command(self, tmp_path: Path) -> None:
|
||||
"""main() should handle area command."""
|
||||
with patch("sys.argv", ["screenshot", "area"]), patch.object(px, "run") as mock_run:
|
||||
screenshot.main()
|
||||
assert mock_run.called
|
||||
|
||||
def test_main_with_no_args_shows_help(self) -> None:
|
||||
"""main() with no args should show help."""
|
||||
with patch("sys.argv", ["screenshot"]):
|
||||
screenshot.main()
|
||||
# Should print help and return
|
||||
|
||||
@@ -8,8 +8,7 @@ from unittest.mock import MagicMock, patch
|
||||
|
||||
import pytest
|
||||
|
||||
import pyflowx as px
|
||||
from pyflowx.cli import sshcopyid
|
||||
from pyflowx.ops import system
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
@@ -21,7 +20,7 @@ class TestSshCopyId:
|
||||
def test_ssh_copy_id_pub_key_not_exists(self, tmp_path: Path) -> None:
|
||||
"""Should handle nonexistent public key."""
|
||||
with patch.object(Path, "expanduser", return_value=tmp_path / "nonexistent.pub"), pytest.raises(SystemExit):
|
||||
sshcopyid.ssh_copy_id("localhost", "user", "password")
|
||||
system.ssh_copy_id("localhost", "user", "password")
|
||||
|
||||
def test_ssh_copy_id_sshpass_not_found(self, tmp_path: Path) -> None:
|
||||
"""Should handle sshpass not found."""
|
||||
@@ -31,7 +30,7 @@ class TestSshCopyId:
|
||||
with patch.object(Path, "expanduser", return_value=pub_key), patch(
|
||||
"subprocess.run", side_effect=FileNotFoundError
|
||||
), pytest.raises(SystemExit):
|
||||
sshcopyid.ssh_copy_id("localhost", "user", "password")
|
||||
system.ssh_copy_id("localhost", "user", "password")
|
||||
|
||||
def test_ssh_copy_id_timeout(self, tmp_path: Path) -> None:
|
||||
"""Should handle SSH timeout."""
|
||||
@@ -41,7 +40,7 @@ class TestSshCopyId:
|
||||
with patch.object(Path, "expanduser", return_value=pub_key), patch(
|
||||
"subprocess.run", side_effect=subprocess.TimeoutExpired("cmd", 30)
|
||||
), pytest.raises(SystemExit):
|
||||
sshcopyid.ssh_copy_id("localhost", "user", "password")
|
||||
system.ssh_copy_id("localhost", "user", "password")
|
||||
|
||||
def test_ssh_copy_id_process_error(self, tmp_path: Path) -> None:
|
||||
"""Should handle SSH process error."""
|
||||
@@ -51,7 +50,7 @@ class TestSshCopyId:
|
||||
with patch.object(Path, "expanduser", return_value=pub_key), patch(
|
||||
"subprocess.run", side_effect=subprocess.CalledProcessError(1, "cmd")
|
||||
), pytest.raises(SystemExit):
|
||||
sshcopyid.ssh_copy_id("localhost", "user", "password")
|
||||
system.ssh_copy_id("localhost", "user", "password")
|
||||
|
||||
def test_ssh_copy_id_success(self, tmp_path: Path) -> None:
|
||||
"""Should deploy SSH key successfully."""
|
||||
@@ -60,7 +59,7 @@ class TestSshCopyId:
|
||||
|
||||
with patch.object(Path, "expanduser", return_value=pub_key), patch("subprocess.run") as mock_run:
|
||||
mock_run.return_value = MagicMock(returncode=0)
|
||||
sshcopyid.ssh_copy_id("localhost", "user", "password")
|
||||
system.ssh_copy_id("localhost", "user", "password")
|
||||
assert mock_run.called
|
||||
|
||||
def test_ssh_copy_id_with_custom_port(self, tmp_path: Path) -> None:
|
||||
@@ -70,7 +69,7 @@ class TestSshCopyId:
|
||||
|
||||
with patch.object(Path, "expanduser", return_value=pub_key), patch("subprocess.run") as mock_run:
|
||||
mock_run.return_value = MagicMock(returncode=0)
|
||||
sshcopyid.ssh_copy_id("localhost", "user", "password", port=2222)
|
||||
system.ssh_copy_id("localhost", "user", "password", port=2222)
|
||||
# Verify port is used
|
||||
call_args = mock_run.call_args[0][0]
|
||||
assert "2222" in call_args
|
||||
@@ -82,7 +81,7 @@ class TestSshCopyId:
|
||||
|
||||
with patch.object(Path, "expanduser", return_value=custom_key), patch("subprocess.run") as mock_run:
|
||||
mock_run.return_value = MagicMock(returncode=0)
|
||||
sshcopyid.ssh_copy_id("localhost", "user", "password", keypath=str(custom_key))
|
||||
system.ssh_copy_id("localhost", "user", "password", keypath=str(custom_key))
|
||||
assert mock_run.called
|
||||
|
||||
def test_ssh_copy_id_with_custom_timeout(self, tmp_path: Path) -> None:
|
||||
@@ -92,72 +91,7 @@ class TestSshCopyId:
|
||||
|
||||
with patch.object(Path, "expanduser", return_value=pub_key), patch("subprocess.run") as mock_run:
|
||||
mock_run.return_value = MagicMock(returncode=0)
|
||||
sshcopyid.ssh_copy_id("localhost", "user", "password", timeout=60)
|
||||
system.ssh_copy_id("localhost", "user", "password", timeout=60)
|
||||
# Verify timeout is used in ConnectTimeout option
|
||||
call_args = mock_run.call_args[0][0]
|
||||
assert "ConnectTimeout=60" in call_args
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
# main function
|
||||
# ---------------------------------------------------------------------- #
|
||||
class TestMain:
|
||||
"""Test main function."""
|
||||
|
||||
def test_main_with_required_args(self) -> None:
|
||||
"""main() should handle required arguments."""
|
||||
with patch("sys.argv", ["sshcopyid", "localhost", "user", "password"]), patch.object(
|
||||
px, "run"
|
||||
) as mock_run, patch.object(sshcopyid, "ssh_copy_id"):
|
||||
sshcopyid.main()
|
||||
assert mock_run.called
|
||||
graph = mock_run.call_args[0][0]
|
||||
assert isinstance(graph, px.Graph)
|
||||
|
||||
def test_main_with_custom_port(self) -> None:
|
||||
"""main() should handle custom port argument."""
|
||||
with patch("sys.argv", ["sshcopyid", "localhost", "user", "password", "--port", "2222"]), patch.object(
|
||||
px, "run"
|
||||
) as mock_run, patch.object(sshcopyid, "ssh_copy_id"):
|
||||
sshcopyid.main()
|
||||
assert mock_run.called
|
||||
|
||||
def test_main_with_custom_keypath(self) -> None:
|
||||
"""main() should handle custom keypath argument."""
|
||||
with patch(
|
||||
"sys.argv", ["sshcopyid", "localhost", "user", "password", "--keypath", "/custom/key.pub"]
|
||||
), patch.object(px, "run") as mock_run, patch.object(sshcopyid, "ssh_copy_id"):
|
||||
sshcopyid.main()
|
||||
assert mock_run.called
|
||||
|
||||
def test_main_with_custom_timeout(self) -> None:
|
||||
"""main() should handle custom timeout argument."""
|
||||
with patch("sys.argv", ["sshcopyid", "localhost", "user", "password", "--timeout", "60"]), patch.object(
|
||||
px, "run"
|
||||
) as mock_run, patch.object(sshcopyid, "ssh_copy_id"):
|
||||
sshcopyid.main()
|
||||
assert mock_run.called
|
||||
|
||||
def test_main_with_no_args_shows_help(self) -> None:
|
||||
"""main() with no args should show help and exit."""
|
||||
with patch("sys.argv", ["sshcopyid"]), pytest.raises(SystemExit) as exc_info:
|
||||
sshcopyid.main()
|
||||
assert exc_info.value.code == 2
|
||||
|
||||
def test_main_creates_task_spec_with_correct_name(self) -> None:
|
||||
"""main() should create TaskSpec with correct name."""
|
||||
with patch("sys.argv", ["sshcopyid", "localhost", "user", "password"]), patch.object(
|
||||
px, "run"
|
||||
) as mock_run, patch.object(sshcopyid, "ssh_copy_id"):
|
||||
sshcopyid.main()
|
||||
graph = mock_run.call_args[0][0]
|
||||
task_names = list(graph.all_specs().keys())
|
||||
assert "ssh_deploy" in task_names
|
||||
|
||||
def test_main_uses_thread_strategy(self) -> None:
|
||||
"""main() should use thread strategy."""
|
||||
with patch("sys.argv", ["sshcopyid", "localhost", "user", "password"]), patch.object(
|
||||
px, "run"
|
||||
) as mock_run, patch.object(sshcopyid, "ssh_copy_id"):
|
||||
sshcopyid.main()
|
||||
assert mock_run.call_args[1]["strategy"] == "thread"
|
||||
|
||||
+273
-197
@@ -82,9 +82,11 @@ class TestRetryPolicy:
|
||||
raise RuntimeError("not yet")
|
||||
return "ok"
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("flaky", flaky, retry=RetryPolicy(max_attempts=3)),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("flaky", flaky, retry=RetryPolicy(max_attempts=3)),
|
||||
]
|
||||
)
|
||||
report = px.run(graph, strategy="sequential")
|
||||
assert report.success
|
||||
assert report["flaky"] == "ok"
|
||||
@@ -95,9 +97,11 @@ class TestRetryPolicy:
|
||||
def always_fail() -> None:
|
||||
raise RuntimeError("nope")
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("f", always_fail, retry=RetryPolicy(max_attempts=3)),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("f", always_fail, retry=RetryPolicy(max_attempts=3)),
|
||||
]
|
||||
)
|
||||
with pytest.raises(px.TaskFailedError) as exc_info:
|
||||
px.run(graph, strategy="sequential")
|
||||
assert exc_info.value.attempts == 3
|
||||
@@ -111,13 +115,15 @@ class TestRetryPolicy:
|
||||
raise KeyError("not retried")
|
||||
|
||||
# retry_on=(ValueError,) -> KeyError 不应被重试
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec(
|
||||
"f",
|
||||
fail_with_keyerror,
|
||||
retry=RetryPolicy(max_attempts=3, retry_on=(ValueError,)),
|
||||
),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec(
|
||||
"f",
|
||||
fail_with_keyerror,
|
||||
retry=RetryPolicy(max_attempts=3, retry_on=(ValueError,)),
|
||||
),
|
||||
]
|
||||
)
|
||||
with pytest.raises(px.TaskFailedError) as exc_info:
|
||||
px.run(graph, strategy="sequential")
|
||||
# KeyError 不在 retry_on 中,应只尝试 1 次
|
||||
@@ -136,13 +142,15 @@ class TestRetryPolicy:
|
||||
raise RuntimeError("not yet")
|
||||
return "ok"
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec(
|
||||
"flaky",
|
||||
flaky,
|
||||
retry=RetryPolicy(max_attempts=3, delay=0.05, backoff=2.0),
|
||||
),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec(
|
||||
"flaky",
|
||||
flaky,
|
||||
retry=RetryPolicy(max_attempts=3, delay=0.05, backoff=2.0),
|
||||
),
|
||||
]
|
||||
)
|
||||
report = px.run(graph, strategy="sequential")
|
||||
assert report.success
|
||||
# 第 2 次重试应在 delay=0.05 后,第 3 次应在 0.05*2=0.10 后
|
||||
@@ -161,9 +169,11 @@ class TestRetryPolicy:
|
||||
raise RuntimeError("not yet")
|
||||
return "ok"
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("flaky", flaky, retry=RetryPolicy(max_attempts=3)),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("flaky", flaky, retry=RetryPolicy(max_attempts=3)),
|
||||
]
|
||||
)
|
||||
report = px.run(graph, strategy="async")
|
||||
assert report.success
|
||||
assert report["flaky"] == "ok"
|
||||
@@ -187,9 +197,11 @@ class TestTaskHooks:
|
||||
return "ok"
|
||||
|
||||
hooks = TaskHooks(pre_run=pre_run)
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("t", fn, hooks=hooks),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("t", fn, hooks=hooks),
|
||||
]
|
||||
)
|
||||
report = px.run(graph, strategy="sequential")
|
||||
assert report.success
|
||||
assert events == ["pre:t", "run"]
|
||||
@@ -205,9 +217,11 @@ class TestTaskHooks:
|
||||
return 42
|
||||
|
||||
hooks = TaskHooks(post_run=post_run)
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("t", fn, hooks=hooks),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("t", fn, hooks=hooks),
|
||||
]
|
||||
)
|
||||
report = px.run(graph, strategy="sequential")
|
||||
assert report.success
|
||||
assert captured == {"name": "t", "result": 42}
|
||||
@@ -223,9 +237,11 @@ class TestTaskHooks:
|
||||
raise ValueError("boom")
|
||||
|
||||
hooks = TaskHooks(on_failure=on_failure)
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("t", fn, hooks=hooks, continue_on_error=True),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("t", fn, hooks=hooks, continue_on_error=True),
|
||||
]
|
||||
)
|
||||
report = px.run(graph, strategy="sequential")
|
||||
# continue_on_error=True -> 报告成功但任务失败
|
||||
assert report.success
|
||||
@@ -242,14 +258,16 @@ class TestTaskHooks:
|
||||
events.append("post")
|
||||
|
||||
hooks = TaskHooks(pre_run=pre_run, post_run=post_run)
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec(
|
||||
"t",
|
||||
fn=lambda: "ok",
|
||||
hooks=hooks,
|
||||
conditions=(lambda _ctx: False,),
|
||||
),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec(
|
||||
"t",
|
||||
fn=lambda: "ok",
|
||||
hooks=hooks,
|
||||
conditions=(lambda _ctx: False,),
|
||||
),
|
||||
]
|
||||
)
|
||||
report = px.run(graph, strategy="sequential")
|
||||
assert report.success
|
||||
assert report.result_of("t").status == TaskStatus.SKIPPED
|
||||
@@ -266,9 +284,11 @@ class TestTaskHooks:
|
||||
return "ok"
|
||||
|
||||
hooks = TaskHooks(pre_run=pre_run)
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("t", fn, hooks=hooks),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("t", fn, hooks=hooks),
|
||||
]
|
||||
)
|
||||
report = px.run(graph, strategy="async")
|
||||
assert report.success
|
||||
assert events == ["pre:t", "run"]
|
||||
@@ -414,10 +434,12 @@ class TestSoftDependencies:
|
||||
order.append("fast")
|
||||
return f"after-{slow}"
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("slow", slow),
|
||||
px.TaskSpec("fast", fast, soft_depends_on=("slow",)),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("slow", slow),
|
||||
px.TaskSpec("fast", fast, soft_depends_on=("slow",)),
|
||||
]
|
||||
)
|
||||
report = px.run(graph, strategy="dependency")
|
||||
assert report.success
|
||||
# soft 依赖应等待 slow 完成后再执行 fast
|
||||
@@ -433,15 +455,17 @@ class TestSoftDependencies:
|
||||
def downstream(fail: str = "default") -> str:
|
||||
return f"got:{fail}"
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("fail", fail, continue_on_error=True),
|
||||
px.TaskSpec(
|
||||
"downstream",
|
||||
downstream,
|
||||
soft_depends_on=("fail",),
|
||||
continue_on_error=True,
|
||||
),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("fail", fail, continue_on_error=True),
|
||||
px.TaskSpec(
|
||||
"downstream",
|
||||
downstream,
|
||||
soft_depends_on=("fail",),
|
||||
continue_on_error=True,
|
||||
),
|
||||
]
|
||||
)
|
||||
report = px.run(graph, strategy="dependency")
|
||||
assert report.success
|
||||
# fail 失败但下游仍执行(使用默认值)
|
||||
@@ -450,9 +474,11 @@ class TestSoftDependencies:
|
||||
|
||||
def test_soft_dependency_validation_unknown_dep(self) -> None:
|
||||
with pytest.raises(px.MissingDependencyError):
|
||||
px.Graph.from_specs([
|
||||
px.TaskSpec("a", lambda: "ok", soft_depends_on=("missing",)),
|
||||
])
|
||||
px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("a", lambda: "ok", soft_depends_on=("missing",)),
|
||||
]
|
||||
)
|
||||
|
||||
def test_soft_and_hard_dependency_combined(self) -> None:
|
||||
order: list[str] = []
|
||||
@@ -469,11 +495,13 @@ class TestSoftDependencies:
|
||||
order.append("c")
|
||||
return f"c-{b}"
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("a", a),
|
||||
px.TaskSpec("b", b, depends_on=("a",)),
|
||||
px.TaskSpec("c", c, depends_on=("b",), soft_depends_on=("a",)),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("a", a),
|
||||
px.TaskSpec("b", b, depends_on=("a",)),
|
||||
px.TaskSpec("c", c, depends_on=("b",), soft_depends_on=("a",)),
|
||||
]
|
||||
)
|
||||
report = px.run(graph, strategy="dependency")
|
||||
assert report.success
|
||||
assert order == ["a", "b", "c"]
|
||||
@@ -495,11 +523,13 @@ class TestDependencyDrivenScheduling:
|
||||
def c(b: int) -> int:
|
||||
return b + 1
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("a", a),
|
||||
px.TaskSpec("b", b, depends_on=("a",)),
|
||||
px.TaskSpec("c", c, depends_on=("b",)),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("a", a),
|
||||
px.TaskSpec("b", b, depends_on=("a",)),
|
||||
px.TaskSpec("c", c, depends_on=("b",)),
|
||||
]
|
||||
)
|
||||
report = px.run(graph, strategy="dependency")
|
||||
assert report.success
|
||||
assert report["a"] == 1
|
||||
@@ -514,10 +544,12 @@ class TestDependencyDrivenScheduling:
|
||||
async def b(a: str) -> str:
|
||||
return f"b-{a}"
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("a", a),
|
||||
px.TaskSpec("b", b, depends_on=("a",)),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("a", a),
|
||||
px.TaskSpec("b", b, depends_on=("a",)),
|
||||
]
|
||||
)
|
||||
report = px.run(graph, strategy="dependency")
|
||||
assert report.success
|
||||
assert report["b"] == "b-a"
|
||||
@@ -537,12 +569,14 @@ class TestDependencyDrivenScheduling:
|
||||
def d(b: int, c: int) -> int:
|
||||
return b + c
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("a", a),
|
||||
px.TaskSpec("b", b, depends_on=("a",)),
|
||||
px.TaskSpec("c", c, depends_on=("a",)),
|
||||
px.TaskSpec("d", d, depends_on=("b", "c")),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("a", a),
|
||||
px.TaskSpec("b", b, depends_on=("a",)),
|
||||
px.TaskSpec("c", c, depends_on=("a",)),
|
||||
px.TaskSpec("d", d, depends_on=("b", "c")),
|
||||
]
|
||||
)
|
||||
report = px.run(graph, strategy="dependency")
|
||||
assert report.success
|
||||
assert report["a"] == 10
|
||||
@@ -574,11 +608,13 @@ class TestConcurrencyLimits:
|
||||
|
||||
return fn
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("a", make_fn(1), concurrency_key="db"),
|
||||
px.TaskSpec("b", make_fn(2), concurrency_key="db"),
|
||||
px.TaskSpec("c", make_fn(3), concurrency_key="db"),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("a", make_fn(1), concurrency_key="db"),
|
||||
px.TaskSpec("b", make_fn(2), concurrency_key="db"),
|
||||
px.TaskSpec("c", make_fn(3), concurrency_key="db"),
|
||||
]
|
||||
)
|
||||
report = px.run(
|
||||
graph,
|
||||
strategy="dependency",
|
||||
@@ -604,10 +640,12 @@ class TestConcurrencyLimits:
|
||||
|
||||
return fn
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("a", make_fn("a"), concurrency_key="db1"),
|
||||
px.TaskSpec("b", make_fn("b"), concurrency_key="db2"),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("a", make_fn("a"), concurrency_key="db1"),
|
||||
px.TaskSpec("b", make_fn("b"), concurrency_key="db2"),
|
||||
]
|
||||
)
|
||||
report = px.run(
|
||||
graph,
|
||||
strategy="dependency",
|
||||
@@ -633,12 +671,14 @@ class TestConcurrencyLimits:
|
||||
|
||||
return fn
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("a", make_fn(1), concurrency_key="pool"),
|
||||
px.TaskSpec("b", make_fn(2), concurrency_key="pool"),
|
||||
px.TaskSpec("c", make_fn(3), concurrency_key="pool"),
|
||||
px.TaskSpec("d", make_fn(4), concurrency_key="pool"),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("a", make_fn(1), concurrency_key="pool"),
|
||||
px.TaskSpec("b", make_fn(2), concurrency_key="pool"),
|
||||
px.TaskSpec("c", make_fn(3), concurrency_key="pool"),
|
||||
px.TaskSpec("d", make_fn(4), concurrency_key="pool"),
|
||||
]
|
||||
)
|
||||
report = px.run(
|
||||
graph,
|
||||
strategy="dependency",
|
||||
@@ -666,11 +706,13 @@ class TestPriority:
|
||||
|
||||
return fn
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("low", make_fn("low"), priority=1),
|
||||
px.TaskSpec("high", make_fn("high"), priority=10),
|
||||
px.TaskSpec("mid", make_fn("mid"), priority=5),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("low", make_fn("low"), priority=1),
|
||||
px.TaskSpec("high", make_fn("high"), priority=10),
|
||||
px.TaskSpec("mid", make_fn("mid"), priority=5),
|
||||
]
|
||||
)
|
||||
report = px.run(graph, strategy="sequential")
|
||||
assert report.success
|
||||
# 高优先级先执行
|
||||
@@ -696,10 +738,12 @@ class TestContinueOnError:
|
||||
def downstream() -> str:
|
||||
return "ran"
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("fail", fail, continue_on_error=True),
|
||||
px.TaskSpec("downstream", downstream, depends_on=("fail",)),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("fail", fail, continue_on_error=True),
|
||||
px.TaskSpec("downstream", downstream, depends_on=("fail",)),
|
||||
]
|
||||
)
|
||||
report = px.run(graph, strategy="sequential")
|
||||
# continue_on_error 使整体报告成功(不抛异常)
|
||||
assert report.success
|
||||
@@ -716,10 +760,12 @@ class TestContinueOnError:
|
||||
def downstream() -> str:
|
||||
return "ran"
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("fail", fail, continue_on_error=True),
|
||||
px.TaskSpec("downstream", downstream, soft_depends_on=("fail",)),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("fail", fail, continue_on_error=True),
|
||||
px.TaskSpec("downstream", downstream, soft_depends_on=("fail",)),
|
||||
]
|
||||
)
|
||||
report = px.run(graph, strategy="dependency")
|
||||
assert report.success
|
||||
assert report.result_of("fail").status == TaskStatus.FAILED
|
||||
@@ -734,10 +780,12 @@ class TestContinueOnError:
|
||||
def other() -> str:
|
||||
return "ok"
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("fail", fail, continue_on_error=True),
|
||||
px.TaskSpec("other", other),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("fail", fail, continue_on_error=True),
|
||||
px.TaskSpec("other", other),
|
||||
]
|
||||
)
|
||||
report = px.run(graph, strategy="dependency")
|
||||
assert report.success
|
||||
assert report.result_of("fail").status == TaskStatus.FAILED
|
||||
@@ -750,10 +798,12 @@ class TestContinueOnError:
|
||||
def other() -> str:
|
||||
return "ok"
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("fail", fail),
|
||||
px.TaskSpec("other", other),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("fail", fail),
|
||||
px.TaskSpec("other", other),
|
||||
]
|
||||
)
|
||||
with pytest.raises(px.TaskFailedError):
|
||||
px.run(graph, strategy="sequential")
|
||||
|
||||
@@ -854,9 +904,11 @@ class TestCompose:
|
||||
g_extract = px.Graph.from_specs([px.TaskSpec("extract", extract)])
|
||||
# transform 图:通过 _pending_refs 引用 "extract" 命令
|
||||
# transform 自身不声明 depends_on,由 compose 展开时自动连接
|
||||
g_transform = px.Graph.from_specs([
|
||||
px.TaskSpec("transform", transform),
|
||||
])
|
||||
g_transform = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("transform", transform),
|
||||
]
|
||||
)
|
||||
g_transform._pending_refs = ["extract"]
|
||||
|
||||
resolved = px.compose({"extract": g_extract, "transform": g_transform})
|
||||
@@ -943,18 +995,22 @@ class TestCacheKey:
|
||||
|
||||
return key
|
||||
|
||||
graph1 = px.Graph.from_specs([
|
||||
px.TaskSpec("t", expensive, args=(5,), cache_key=make_cache_key(5)),
|
||||
])
|
||||
graph1 = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("t", expensive, args=(5,), cache_key=make_cache_key(5)),
|
||||
]
|
||||
)
|
||||
report1 = px.run(graph1, strategy="sequential", state=backend)
|
||||
assert report1.success
|
||||
assert report1["t"] == 10
|
||||
assert calls["n"] == 1
|
||||
|
||||
# 第二次运行相同输入应命中缓存
|
||||
graph2 = px.Graph.from_specs([
|
||||
px.TaskSpec("t", expensive, args=(5,), cache_key=make_cache_key(5)),
|
||||
])
|
||||
graph2 = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("t", expensive, args=(5,), cache_key=make_cache_key(5)),
|
||||
]
|
||||
)
|
||||
report2 = px.run(graph2, strategy="sequential", state=backend)
|
||||
assert report2.success
|
||||
assert report2["t"] == 10
|
||||
@@ -976,16 +1032,20 @@ class TestCacheKey:
|
||||
|
||||
return key
|
||||
|
||||
graph1 = px.Graph.from_specs([
|
||||
px.TaskSpec("t", expensive, args=(5,), cache_key=make_cache_key(5)),
|
||||
])
|
||||
graph1 = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("t", expensive, args=(5,), cache_key=make_cache_key(5)),
|
||||
]
|
||||
)
|
||||
px.run(graph1, strategy="sequential", state=backend)
|
||||
assert calls["n"] == 1
|
||||
|
||||
# 不同输入应 miss
|
||||
graph2 = px.Graph.from_specs([
|
||||
px.TaskSpec("t", expensive, args=(7,), cache_key=make_cache_key(7)),
|
||||
])
|
||||
graph2 = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("t", expensive, args=(7,), cache_key=make_cache_key(7)),
|
||||
]
|
||||
)
|
||||
px.run(graph2, strategy="sequential", state=backend)
|
||||
assert calls["n"] == 2
|
||||
|
||||
@@ -997,13 +1057,15 @@ class TestEnvAndCwd:
|
||||
"""测试环境变量与工作目录隔离。"""
|
||||
|
||||
def test_env_override_for_cmd(self) -> None:
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec(
|
||||
"print_var",
|
||||
cmd=[sys.executable, "-c", "import os; print(os.environ.get('PYFLOWX_TEST_VAR', 'unset'))"],
|
||||
env={"PYFLOWX_TEST_VAR": "isolated"},
|
||||
),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec(
|
||||
"print_var",
|
||||
cmd=[sys.executable, "-c", "import os; print(os.environ.get('PYFLOWX_TEST_VAR', 'unset'))"],
|
||||
env={"PYFLOWX_TEST_VAR": "isolated"},
|
||||
),
|
||||
]
|
||||
)
|
||||
report = px.run(graph, strategy="sequential")
|
||||
assert report.success
|
||||
|
||||
@@ -1011,13 +1073,15 @@ class TestEnvAndCwd:
|
||||
# 在 tmp_path 下创建标记文件
|
||||
marker = tmp_path / "marker.txt"
|
||||
marker.write_text("found")
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec(
|
||||
"check_cwd",
|
||||
cmd=["ls", "marker.txt"],
|
||||
cwd=tmp_path,
|
||||
),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec(
|
||||
"check_cwd",
|
||||
cmd=["ls", "marker.txt"],
|
||||
cwd=tmp_path,
|
||||
),
|
||||
]
|
||||
)
|
||||
report = px.run(graph, strategy="sequential")
|
||||
assert report.success
|
||||
|
||||
@@ -1027,13 +1091,15 @@ class TestEnvAndCwd:
|
||||
def check_env() -> str:
|
||||
return os.environ.get("PYFLOWX_LEAK_TEST", "not-set")
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec(
|
||||
"t",
|
||||
check_env,
|
||||
env={"PYFLOWX_LEAK_TEST": "leaked"},
|
||||
),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec(
|
||||
"t",
|
||||
check_env,
|
||||
env={"PYFLOWX_LEAK_TEST": "leaked"},
|
||||
),
|
||||
]
|
||||
)
|
||||
# fn 任务的环境变量隔离仅在 cmd 任务生效,fn 共享进程环境
|
||||
# 这里验证 fn 任务不修改外层环境
|
||||
report = px.run(graph, strategy="sequential")
|
||||
@@ -1059,21 +1125,23 @@ class TestContextAwareConditions:
|
||||
def path_b(decide: str = "") -> str:
|
||||
return f"ran-b:{decide}"
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("decide", decide),
|
||||
px.TaskSpec(
|
||||
"path_a",
|
||||
path_a,
|
||||
depends_on=("decide",),
|
||||
conditions=(BuiltinConditions.DEP_EQUALS("decide", "path_a"),),
|
||||
),
|
||||
px.TaskSpec(
|
||||
"path_b",
|
||||
path_b,
|
||||
depends_on=("decide",),
|
||||
conditions=(BuiltinConditions.DEP_EQUALS("decide", "path_b"),),
|
||||
),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("decide", decide),
|
||||
px.TaskSpec(
|
||||
"path_a",
|
||||
path_a,
|
||||
depends_on=("decide",),
|
||||
conditions=(BuiltinConditions.DEP_EQUALS("decide", "path_a"),),
|
||||
),
|
||||
px.TaskSpec(
|
||||
"path_b",
|
||||
path_b,
|
||||
depends_on=("decide",),
|
||||
conditions=(BuiltinConditions.DEP_EQUALS("decide", "path_b"),),
|
||||
),
|
||||
]
|
||||
)
|
||||
report = px.run(graph, strategy="dependency")
|
||||
assert report.success
|
||||
assert report.result_of("path_a").status == TaskStatus.SKIPPED
|
||||
@@ -1087,15 +1155,17 @@ class TestContextAwareConditions:
|
||||
def only_if_nonempty(source: list[int]) -> str:
|
||||
return f"has-{len(source)}"
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("source", source),
|
||||
px.TaskSpec(
|
||||
"only_if_nonempty",
|
||||
only_if_nonempty,
|
||||
depends_on=("source",),
|
||||
conditions=(BuiltinConditions.DEP_TRUTHY("source"),),
|
||||
),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("source", source),
|
||||
px.TaskSpec(
|
||||
"only_if_nonempty",
|
||||
only_if_nonempty,
|
||||
depends_on=("source",),
|
||||
conditions=(BuiltinConditions.DEP_TRUTHY("source"),),
|
||||
),
|
||||
]
|
||||
)
|
||||
report = px.run(graph, strategy="dependency")
|
||||
assert report.success
|
||||
assert report["only_if_nonempty"] == "has-3"
|
||||
@@ -1107,15 +1177,17 @@ class TestContextAwareConditions:
|
||||
def only_if_nonempty(source: list[int]) -> str:
|
||||
return "should-not-run"
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("source", source),
|
||||
px.TaskSpec(
|
||||
"only_if_nonempty",
|
||||
only_if_nonempty,
|
||||
depends_on=("source",),
|
||||
conditions=(BuiltinConditions.DEP_TRUTHY("source"),),
|
||||
),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("source", source),
|
||||
px.TaskSpec(
|
||||
"only_if_nonempty",
|
||||
only_if_nonempty,
|
||||
depends_on=("source",),
|
||||
conditions=(BuiltinConditions.DEP_TRUTHY("source"),),
|
||||
),
|
||||
]
|
||||
)
|
||||
report = px.run(graph, strategy="dependency")
|
||||
assert report.success
|
||||
assert report.result_of("only_if_nonempty").status == TaskStatus.SKIPPED
|
||||
@@ -1127,15 +1199,17 @@ class TestContextAwareConditions:
|
||||
def downstream(source: int) -> str:
|
||||
return f"got-{source}"
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("source", source),
|
||||
px.TaskSpec(
|
||||
"downstream",
|
||||
downstream,
|
||||
depends_on=("source",),
|
||||
conditions=(BuiltinConditions.DEP_MATCHES("source", lambda v: v > 10),),
|
||||
),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("source", source),
|
||||
px.TaskSpec(
|
||||
"downstream",
|
||||
downstream,
|
||||
depends_on=("source",),
|
||||
conditions=(BuiltinConditions.DEP_MATCHES("source", lambda v: v > 10),),
|
||||
),
|
||||
]
|
||||
)
|
||||
report = px.run(graph, strategy="dependency")
|
||||
assert report.success
|
||||
assert report["downstream"] == "got-42"
|
||||
@@ -1165,10 +1239,12 @@ class TestPerTaskStrategy:
|
||||
await asyncio.sleep(0.01)
|
||||
return f"async-{sync}"
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("sync", sync_fn),
|
||||
px.TaskSpec("async", async_fn, depends_on=("sync",)),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("sync", sync_fn),
|
||||
px.TaskSpec("async", async_fn, depends_on=("sync",)),
|
||||
]
|
||||
)
|
||||
report = px.run(graph, strategy="dependency")
|
||||
assert report.success
|
||||
assert report["async"] == "async-sync"
|
||||
|
||||
@@ -68,6 +68,7 @@ def test_chain_execution_order() -> None:
|
||||
def fn() -> str:
|
||||
order.append(name)
|
||||
return name
|
||||
|
||||
return fn
|
||||
|
||||
a = TaskSpec("a", make("a"))
|
||||
|
||||
@@ -93,7 +93,7 @@ class TestCommandReferences:
|
||||
with pytest.raises(ValueError, match="循环引用"):
|
||||
px.CliRunner(
|
||||
strategy="sequential",
|
||||
aliases={
|
||||
aliases={
|
||||
"cmd1": px.Graph.from_specs(["cmd1", task1]),
|
||||
},
|
||||
)
|
||||
@@ -105,7 +105,7 @@ class TestCommandReferences:
|
||||
with pytest.raises(ValueError, match="引用的命令 'invalid' 不存在"):
|
||||
px.CliRunner(
|
||||
strategy="sequential",
|
||||
aliases={
|
||||
aliases={
|
||||
"cmd1": px.Graph.from_specs(["invalid", task1]),
|
||||
},
|
||||
)
|
||||
@@ -117,7 +117,7 @@ class TestCommandReferences:
|
||||
with pytest.raises(ValueError, match="任务 'invalid' 不存在于命令 'cmd1' 中"):
|
||||
px.CliRunner(
|
||||
strategy="sequential",
|
||||
aliases={
|
||||
aliases={
|
||||
"cmd1": px.Graph.from_specs([task1]),
|
||||
"cmd2": px.Graph.from_specs(["cmd1.invalid"]),
|
||||
},
|
||||
|
||||
+118
-78
@@ -27,10 +27,12 @@ def test_sequential_basic() -> None:
|
||||
def double(extract: list[int]) -> list[int]:
|
||||
return [x * 2 for x in extract]
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("extract", extract),
|
||||
px.TaskSpec("double", double, depends_on=("extract",)),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("extract", extract),
|
||||
px.TaskSpec("double", double, depends_on=("extract",)),
|
||||
]
|
||||
)
|
||||
report = px.run(graph, strategy="sequential")
|
||||
assert report.success
|
||||
assert report["extract"] == [1, 2, 3]
|
||||
@@ -47,12 +49,14 @@ def test_sequential_diamond() -> None:
|
||||
|
||||
return fn
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("a", make("a")),
|
||||
px.TaskSpec("b", make("b"), depends_on=("a",)),
|
||||
px.TaskSpec("c", make("c"), depends_on=("a",)),
|
||||
px.TaskSpec("d", make("d"), depends_on=("b", "c")),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("a", make("a")),
|
||||
px.TaskSpec("b", make("b"), depends_on=("a",)),
|
||||
px.TaskSpec("c", make("c"), depends_on=("a",)),
|
||||
px.TaskSpec("d", make("d"), depends_on=("b", "c")),
|
||||
]
|
||||
)
|
||||
report = px.run(graph, strategy="sequential")
|
||||
assert report.success
|
||||
assert report["d"] == "d"
|
||||
@@ -66,10 +70,12 @@ def test_failure_propagates() -> None:
|
||||
def downstream(_boom: None) -> int:
|
||||
return 1
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("boom", boom),
|
||||
px.TaskSpec("downstream", downstream, depends_on=("boom",)),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("boom", boom),
|
||||
px.TaskSpec("downstream", downstream, depends_on=("boom",)),
|
||||
]
|
||||
)
|
||||
with pytest.raises(TaskFailedError) as exc_info:
|
||||
_ = px.run(graph, strategy="sequential")
|
||||
assert exc_info.value.task == "boom"
|
||||
@@ -85,9 +91,11 @@ def test_retries_then_succeeds() -> None:
|
||||
raise RuntimeError("not yet")
|
||||
return "ok"
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("flaky", flaky, retry=px.RetryPolicy(max_attempts=3)),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("flaky", flaky, retry=px.RetryPolicy(max_attempts=3)),
|
||||
]
|
||||
)
|
||||
report = px.run(graph, strategy="sequential")
|
||||
assert report.success
|
||||
assert report["flaky"] == "ok"
|
||||
@@ -105,9 +113,11 @@ def test_retries_with_delay() -> None:
|
||||
raise RuntimeError("not yet")
|
||||
return "ok"
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("flaky", flaky, retry=px.RetryPolicy(max_attempts=2, delay=0.1)),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("flaky", flaky, retry=px.RetryPolicy(max_attempts=2, delay=0.1)),
|
||||
]
|
||||
)
|
||||
report = px.run(graph, strategy="sequential")
|
||||
elapsed = time.time() - start_time
|
||||
assert report.success
|
||||
@@ -122,9 +132,11 @@ def test_timeout_then_retry_async(caplog: pytest.LogCaptureFixture) -> None:
|
||||
await asyncio.sleep(10) # 会触发超时
|
||||
return "ok"
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("slow", slow_task, timeout=0.2, retry=px.RetryPolicy(max_attempts=2)),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("slow", slow_task, timeout=0.2, retry=px.RetryPolicy(max_attempts=2)),
|
||||
]
|
||||
)
|
||||
with caplog.at_level(logging.WARNING, logger="pyflowx"):
|
||||
with pytest.raises(px.TaskFailedError) as exc_info:
|
||||
_ = px.run(graph, strategy="async")
|
||||
@@ -138,9 +150,11 @@ def test_retries_exhausted() -> None:
|
||||
def always_fail() -> None:
|
||||
raise RuntimeError("nope")
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("f", always_fail, retry=px.RetryPolicy(max_attempts=3)),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("f", always_fail, retry=px.RetryPolicy(max_attempts=3)),
|
||||
]
|
||||
)
|
||||
with pytest.raises(TaskFailedError) as exc_info:
|
||||
_ = px.run(graph, strategy="sequential")
|
||||
assert exc_info.value.attempts == 3
|
||||
@@ -155,11 +169,13 @@ def test_threaded_parallelism() -> None:
|
||||
time.sleep(0.3)
|
||||
return "done"
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("a", slow),
|
||||
px.TaskSpec("b", slow),
|
||||
px.TaskSpec("c", slow),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("a", slow),
|
||||
px.TaskSpec("b", slow),
|
||||
px.TaskSpec("c", slow),
|
||||
]
|
||||
)
|
||||
start = time.time()
|
||||
report = px.run(graph, strategy="thread", max_workers=3)
|
||||
elapsed = time.time() - start
|
||||
@@ -182,11 +198,13 @@ def test_threaded_layer_barrier() -> None:
|
||||
|
||||
return fn
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("a", make("a")),
|
||||
px.TaskSpec("b", make("b")),
|
||||
px.TaskSpec("c", make("c"), depends_on=("a", "b")),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("a", make("a")),
|
||||
px.TaskSpec("b", make("b")),
|
||||
px.TaskSpec("c", make("c"), depends_on=("a", "b")),
|
||||
]
|
||||
)
|
||||
report = px.run(graph, strategy="thread", max_workers=2)
|
||||
assert report.success
|
||||
# c must finish after both a and b.
|
||||
@@ -205,10 +223,12 @@ def test_async_basic() -> None:
|
||||
async def transform(fetch: int) -> int:
|
||||
return fetch * 2
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("fetch", fetch),
|
||||
px.TaskSpec("transform", transform, depends_on=("fetch",)),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("fetch", fetch),
|
||||
px.TaskSpec("transform", transform, depends_on=("fetch",)),
|
||||
]
|
||||
)
|
||||
report = px.run(graph, strategy="async")
|
||||
assert report.success
|
||||
assert report["transform"] == 84
|
||||
@@ -237,10 +257,12 @@ def test_async_mixed_sync_and_async() -> None:
|
||||
await asyncio.sleep(0.01)
|
||||
return sync_task + 5
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("sync_task", sync_task),
|
||||
px.TaskSpec("async_task", async_task, depends_on=("sync_task",)),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("sync_task", sync_task),
|
||||
px.TaskSpec("async_task", async_task, depends_on=("sync_task",)),
|
||||
]
|
||||
)
|
||||
report = px.run(graph, strategy="async")
|
||||
assert report.success
|
||||
assert report["async_task"] == 15
|
||||
@@ -288,10 +310,12 @@ def test_memory_backend_resume() -> None:
|
||||
|
||||
return fn
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("a", make("a")),
|
||||
px.TaskSpec("b", make("b"), depends_on=("a",)),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("a", make("a")),
|
||||
px.TaskSpec("b", make("b"), depends_on=("a",)),
|
||||
]
|
||||
)
|
||||
backend = MemoryBackend()
|
||||
_ = px.run(graph, strategy="sequential", state=backend)
|
||||
assert runs == ["a", "b"]
|
||||
@@ -377,9 +401,11 @@ def test_async_timeout_retry_then_succeed() -> None:
|
||||
await asyncio.sleep(10) # 触发超时
|
||||
return "ok"
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("a", flaky, retry=px.RetryPolicy(max_attempts=3), timeout=0.05),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("a", flaky, retry=px.RetryPolicy(max_attempts=3), timeout=0.05),
|
||||
]
|
||||
)
|
||||
report = px.run(graph, strategy="async")
|
||||
assert report.success
|
||||
assert report["a"] == "ok"
|
||||
@@ -396,9 +422,11 @@ def test_async_failure_retry_branch(caplog: pytest.LogCaptureFixture) -> None:
|
||||
raise RuntimeError("not yet")
|
||||
return "ok"
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("a", flaky, retry=px.RetryPolicy(max_attempts=3)),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("a", flaky, retry=px.RetryPolicy(max_attempts=3)),
|
||||
]
|
||||
)
|
||||
with caplog.at_level("WARNING", logger="pyflowx"):
|
||||
report = px.run(graph, strategy="async")
|
||||
assert report.success
|
||||
@@ -421,10 +449,12 @@ def test_threaded_skips_cached_tasks() -> None:
|
||||
|
||||
return fn
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("a", make("a")),
|
||||
px.TaskSpec("b", make("b"), depends_on=("a",)),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("a", make("a")),
|
||||
px.TaskSpec("b", make("b"), depends_on=("a",)),
|
||||
]
|
||||
)
|
||||
backend = px.MemoryBackend()
|
||||
# 第一次运行填充缓存
|
||||
_ = px.run(graph, strategy="thread", max_workers=2, state=backend)
|
||||
@@ -464,10 +494,12 @@ def test_async_skips_cached_tasks() -> None:
|
||||
runs.append("b")
|
||||
return a + "b"
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("a", a),
|
||||
px.TaskSpec("b", b, depends_on=("a",)),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("a", a),
|
||||
px.TaskSpec("b", b, depends_on=("a",)),
|
||||
]
|
||||
)
|
||||
backend = px.MemoryBackend()
|
||||
_ = px.run(graph, strategy="async", state=backend)
|
||||
assert runs == ["a", "b"]
|
||||
@@ -543,10 +575,12 @@ def test_downstream_skipped_when_upstream_skipped_sequential() -> None:
|
||||
def downstream(upstream: str) -> str:
|
||||
return upstream + "_processed"
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("upstream", cmd=["echo", "hello"], conditions=(never_true,)),
|
||||
px.TaskSpec("downstream", downstream, depends_on=("upstream",)),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("upstream", cmd=["echo", "hello"], conditions=(never_true,)),
|
||||
px.TaskSpec("downstream", downstream, depends_on=("upstream",)),
|
||||
]
|
||||
)
|
||||
report = px.run(graph, strategy="sequential")
|
||||
assert report.success
|
||||
assert report.result_of("upstream").status == px.TaskStatus.SKIPPED
|
||||
@@ -560,10 +594,12 @@ def test_downstream_skipped_when_upstream_skipped_thread() -> None:
|
||||
def downstream(upstream: str) -> str:
|
||||
return upstream + "_processed"
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("upstream", cmd=["echo", "hello"], conditions=(never_true,)),
|
||||
px.TaskSpec("downstream", downstream, depends_on=("upstream",)),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("upstream", cmd=["echo", "hello"], conditions=(never_true,)),
|
||||
px.TaskSpec("downstream", downstream, depends_on=("upstream",)),
|
||||
]
|
||||
)
|
||||
report = px.run(graph, strategy="thread", max_workers=2)
|
||||
assert report.success
|
||||
assert report.result_of("upstream").status == px.TaskStatus.SKIPPED
|
||||
@@ -581,10 +617,12 @@ def test_downstream_skipped_when_upstream_skipped_async() -> None:
|
||||
|
||||
never_true = lambda _ctx: False # noqa: E731
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("upstream", upstream, conditions=(never_true,)),
|
||||
px.TaskSpec("downstream", downstream, depends_on=("upstream",)),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("upstream", upstream, conditions=(never_true,)),
|
||||
px.TaskSpec("downstream", downstream, depends_on=("upstream",)),
|
||||
]
|
||||
)
|
||||
report = px.run(graph, strategy="async")
|
||||
assert report.success
|
||||
assert report.result_of("upstream").status == px.TaskStatus.SKIPPED
|
||||
@@ -601,10 +639,12 @@ def test_downstream_executes_when_upstream_succeeds() -> None:
|
||||
def downstream(upstream: str) -> str:
|
||||
return upstream + "_processed"
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("upstream", upstream, conditions=(always_true,)),
|
||||
px.TaskSpec("downstream", downstream, depends_on=("upstream",)),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("upstream", upstream, conditions=(always_true,)),
|
||||
px.TaskSpec("downstream", downstream, depends_on=("upstream",)),
|
||||
]
|
||||
)
|
||||
report = px.run(graph, strategy="sequential")
|
||||
assert report.success
|
||||
assert report.result_of("upstream").status == px.TaskStatus.SUCCESS
|
||||
|
||||
@@ -271,10 +271,12 @@ def test_allow_upstream_skip_allows_execution_after_skipped() -> None:
|
||||
def downstream_task() -> str:
|
||||
return "ran despite upstream skipped"
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("upstream", fn=lambda: "up", conditions=(never_true,)),
|
||||
px.TaskSpec("downstream", fn=downstream_task, depends_on=("upstream",), allow_upstream_skip=True),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("upstream", fn=lambda: "up", conditions=(never_true,)),
|
||||
px.TaskSpec("downstream", fn=downstream_task, depends_on=("upstream",), allow_upstream_skip=True),
|
||||
]
|
||||
)
|
||||
report = px.run(graph, strategy="sequential")
|
||||
assert report.success
|
||||
assert report.results["upstream"].status == TaskStatus.SKIPPED
|
||||
@@ -291,10 +293,12 @@ def test_upstream_failed_skips_downstream() -> None:
|
||||
def downstream():
|
||||
return "should not run"
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("upstream", fn=boom),
|
||||
px.TaskSpec("downstream", fn=downstream, depends_on=("upstream",)),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("upstream", fn=boom),
|
||||
px.TaskSpec("downstream", fn=downstream, depends_on=("upstream",)),
|
||||
]
|
||||
)
|
||||
with pytest.raises(px.TaskFailedError):
|
||||
px.run(graph, strategy="sequential")
|
||||
|
||||
@@ -342,11 +346,13 @@ def test_concurrency_key_thread() -> None:
|
||||
|
||||
return fn
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("a", fn=make("a"), concurrency_key="group1"),
|
||||
px.TaskSpec("b", fn=make("b"), concurrency_key="group1"),
|
||||
px.TaskSpec("c", fn=make("c"), concurrency_key="group1"),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("a", fn=make("a"), concurrency_key="group1"),
|
||||
px.TaskSpec("b", fn=make("b"), concurrency_key="group1"),
|
||||
px.TaskSpec("c", fn=make("c"), concurrency_key="group1"),
|
||||
]
|
||||
)
|
||||
report = px.run(graph, strategy="thread", max_workers=10, concurrency_limits={"group1": 1})
|
||||
assert report.success
|
||||
# 由于 concurrency_key 限制为 1,任务应串行执行
|
||||
@@ -366,10 +372,12 @@ def test_concurrency_key_async() -> None:
|
||||
await asyncio.sleep(0.01)
|
||||
return "b"
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("a", fn=task_a, concurrency_key="group1"),
|
||||
px.TaskSpec("b", fn=task_b, concurrency_key="group1"),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("a", fn=task_a, concurrency_key="group1"),
|
||||
px.TaskSpec("b", fn=task_b, concurrency_key="group1"),
|
||||
]
|
||||
)
|
||||
report = px.run(graph, strategy="async", concurrency_limits={"group1": 1})
|
||||
assert report.success
|
||||
|
||||
@@ -388,12 +396,14 @@ def test_dependency_strategy_basic() -> None:
|
||||
|
||||
return fn
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("a", fn=make("a")),
|
||||
px.TaskSpec("b", fn=make("b"), depends_on=("a",)),
|
||||
px.TaskSpec("c", fn=make("c"), depends_on=("a",)),
|
||||
px.TaskSpec("d", fn=make("d"), depends_on=("b", "c")),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("a", fn=make("a")),
|
||||
px.TaskSpec("b", fn=make("b"), depends_on=("a",)),
|
||||
px.TaskSpec("c", fn=make("c"), depends_on=("a",)),
|
||||
px.TaskSpec("d", fn=make("d"), depends_on=("b", "c")),
|
||||
]
|
||||
)
|
||||
report = px.run(graph, strategy="dependency")
|
||||
assert report.success
|
||||
assert "a" in order
|
||||
@@ -409,10 +419,12 @@ def test_dependency_strategy_async() -> None:
|
||||
async def b(a: str):
|
||||
return a + "b"
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("a", fn=a),
|
||||
px.TaskSpec("b", fn=b, depends_on=("a",)),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("a", fn=a),
|
||||
px.TaskSpec("b", fn=b, depends_on=("a",)),
|
||||
]
|
||||
)
|
||||
report = px.run(graph, strategy="dependency")
|
||||
assert report.success
|
||||
assert report["b"] == "ab"
|
||||
@@ -427,10 +439,12 @@ def test_continue_on_error_marks_failed_but_continues() -> None:
|
||||
def boom():
|
||||
raise ValueError("boom")
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("fail", fn=boom, continue_on_error=True),
|
||||
px.TaskSpec("other", fn=lambda: "ok"), # 无依赖,应继续
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("fail", fn=boom, continue_on_error=True),
|
||||
px.TaskSpec("other", fn=lambda: "ok"), # 无依赖,应继续
|
||||
]
|
||||
)
|
||||
# continue_on_error=True 时 run 不抛异常,report.success 为 True
|
||||
report = px.run(graph, strategy="sequential")
|
||||
# report.success 为 True 因为没有抛 TaskFailedError
|
||||
@@ -448,10 +462,12 @@ def test_continue_on_error_downstream_skipped() -> None:
|
||||
def downstream():
|
||||
return "should not run"
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("fail", fn=boom, continue_on_error=True),
|
||||
px.TaskSpec("dep", fn=downstream, depends_on=("fail",), allow_upstream_skip=False),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("fail", fn=boom, continue_on_error=True),
|
||||
px.TaskSpec("dep", fn=downstream, depends_on=("fail",), allow_upstream_skip=False),
|
||||
]
|
||||
)
|
||||
report = px.run(graph, strategy="sequential")
|
||||
# report.success 为 True 因为 continue_on_error 阻止了 TaskFailedError
|
||||
assert report.success
|
||||
@@ -468,10 +484,12 @@ def test_soft_depends_on_default_value_injection() -> None:
|
||||
def task_with_soft_dep(a: str | None = None) -> str:
|
||||
return f"a={a}"
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("a", fn=lambda: "value"),
|
||||
px.TaskSpec("b", fn=task_with_soft_dep, soft_depends_on=("a",)),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("a", fn=lambda: "value"),
|
||||
px.TaskSpec("b", fn=task_with_soft_dep, soft_depends_on=("a",)),
|
||||
]
|
||||
)
|
||||
report = px.run(graph, strategy="sequential")
|
||||
assert report.success
|
||||
assert report["b"] == "a=value"
|
||||
@@ -484,10 +502,12 @@ def test_soft_depends_on_skipped_injects_none() -> None:
|
||||
def task_with_soft_dep(skipped: str | None = None) -> str:
|
||||
return f"skipped={skipped}"
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("skipped", fn=lambda: "value", conditions=(never_true,)),
|
||||
px.TaskSpec("b", fn=task_with_soft_dep, soft_depends_on=("skipped",)),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("skipped", fn=lambda: "value", conditions=(never_true,)),
|
||||
px.TaskSpec("b", fn=task_with_soft_dep, soft_depends_on=("skipped",)),
|
||||
]
|
||||
)
|
||||
report = px.run(graph, strategy="sequential")
|
||||
assert report.success
|
||||
# 软依赖被 skipped 时注入 None(因为 global_context 中有 skipped,值为 None)
|
||||
|
||||
+98
-70
@@ -14,11 +14,13 @@ def _fn() -> None:
|
||||
|
||||
|
||||
def test_from_specs_builds_graph() -> None:
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("a", _fn),
|
||||
px.TaskSpec("b", _fn, depends_on=("a",)),
|
||||
px.TaskSpec("c", _fn, depends_on=("a", "b")),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("a", _fn),
|
||||
px.TaskSpec("b", _fn, depends_on=("a",)),
|
||||
px.TaskSpec("c", _fn, depends_on=("a", "b")),
|
||||
]
|
||||
)
|
||||
assert set(graph.names) == {"a", "b", "c"}
|
||||
assert graph.dependencies("c") == ("a", "b")
|
||||
assert len(graph) == 3
|
||||
@@ -27,19 +29,23 @@ def test_from_specs_builds_graph() -> None:
|
||||
|
||||
def test_from_specs_allows_forward_references() -> None:
|
||||
# b depends on a, but a is declared after b — order should not matter.
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("b", _fn, depends_on=("a",)),
|
||||
px.TaskSpec("a", _fn),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("b", _fn, depends_on=("a",)),
|
||||
px.TaskSpec("a", _fn),
|
||||
]
|
||||
)
|
||||
assert graph.layers() == [["a"], ["b"]]
|
||||
|
||||
|
||||
def test_duplicate_task_raises() -> None:
|
||||
with pytest.raises(DuplicateTaskError):
|
||||
_ = px.Graph.from_specs([
|
||||
px.TaskSpec("a", _fn),
|
||||
px.TaskSpec("a", _fn),
|
||||
])
|
||||
_ = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("a", _fn),
|
||||
px.TaskSpec("a", _fn),
|
||||
]
|
||||
)
|
||||
|
||||
|
||||
def test_missing_dependency_raises() -> None:
|
||||
@@ -52,20 +58,24 @@ def test_missing_dependency_raises() -> None:
|
||||
|
||||
def test_cycle_detection() -> None:
|
||||
with pytest.raises(CycleError):
|
||||
_ = px.Graph.from_specs([
|
||||
px.TaskSpec("a", _fn, depends_on=("c",)),
|
||||
px.TaskSpec("b", _fn, depends_on=("a",)),
|
||||
px.TaskSpec("c", _fn, depends_on=("b",)),
|
||||
])
|
||||
_ = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("a", _fn, depends_on=("c",)),
|
||||
px.TaskSpec("b", _fn, depends_on=("a",)),
|
||||
px.TaskSpec("c", _fn, depends_on=("b",)),
|
||||
]
|
||||
)
|
||||
|
||||
|
||||
def test_layers_grouping() -> None:
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("a", _fn),
|
||||
px.TaskSpec("b", _fn),
|
||||
px.TaskSpec("c", _fn, depends_on=("a", "b")),
|
||||
px.TaskSpec("d", _fn, depends_on=("c",)),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("a", _fn),
|
||||
px.TaskSpec("b", _fn),
|
||||
px.TaskSpec("c", _fn, depends_on=("a", "b")),
|
||||
px.TaskSpec("d", _fn, depends_on=("c",)),
|
||||
]
|
||||
)
|
||||
layers = graph.layers()
|
||||
assert layers == [["a", "b"], ["c"], ["d"]]
|
||||
|
||||
@@ -76,10 +86,12 @@ def test_self_dependency_rejected() -> None:
|
||||
|
||||
|
||||
def test_to_mermaid() -> None:
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("a", _fn),
|
||||
px.TaskSpec("b", _fn, depends_on=("a",)),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("a", _fn),
|
||||
px.TaskSpec("b", _fn, depends_on=("a",)),
|
||||
]
|
||||
)
|
||||
mermaid = graph.to_mermaid()
|
||||
assert mermaid.startswith("graph TD")
|
||||
assert 'a["a"]' in mermaid
|
||||
@@ -93,11 +105,13 @@ def test_to_mermaid_invalid_orientation() -> None:
|
||||
|
||||
|
||||
def test_subgraph_by_tags() -> None:
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("a", _fn, tags=("ingest",)),
|
||||
px.TaskSpec("b", _fn, depends_on=("a",), tags=("ingest",)),
|
||||
px.TaskSpec("c", _fn, depends_on=("b",), tags=("report",)),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("a", _fn, tags=("ingest",)),
|
||||
px.TaskSpec("b", _fn, depends_on=("a",), tags=("ingest",)),
|
||||
px.TaskSpec("c", _fn, depends_on=("b",), tags=("report",)),
|
||||
]
|
||||
)
|
||||
sub = graph.subgraph(["ingest"])
|
||||
assert set(sub.names) == {"a", "b"}
|
||||
# Edge to dropped task c is removed; b no longer waits for anything
|
||||
@@ -106,11 +120,13 @@ def test_subgraph_by_tags() -> None:
|
||||
|
||||
|
||||
def test_subgraph_by_names() -> None:
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("a", _fn),
|
||||
px.TaskSpec("b", _fn, depends_on=("a",)),
|
||||
px.TaskSpec("c", _fn, depends_on=("b",)),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("a", _fn),
|
||||
px.TaskSpec("b", _fn, depends_on=("a",)),
|
||||
px.TaskSpec("c", _fn, depends_on=("b",)),
|
||||
]
|
||||
)
|
||||
sub = graph.subgraph_by_names(["a", "b"])
|
||||
assert set(sub.names) == {"a", "b"}
|
||||
# c is dropped, so b's dep on c (none here) — but a->b edge preserved.
|
||||
@@ -124,10 +140,12 @@ def test_subgraph_by_names_unknown() -> None:
|
||||
|
||||
|
||||
def test_describe() -> None:
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("a", _fn),
|
||||
px.TaskSpec("b", _fn, depends_on=("a",)),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("a", _fn),
|
||||
px.TaskSpec("b", _fn, depends_on=("a",)),
|
||||
]
|
||||
)
|
||||
desc = graph.describe()
|
||||
assert "Layer 1" in desc
|
||||
assert "Layer 2" in desc
|
||||
@@ -164,11 +182,13 @@ def test_all_specs_returns_view() -> None:
|
||||
|
||||
def test_all_deps_combines_hard_and_soft() -> None:
|
||||
"""all_deps 应返回硬依赖 + 软依赖的组合。"""
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("a", _fn),
|
||||
px.TaskSpec("b", _fn),
|
||||
px.TaskSpec("c", _fn, depends_on=("a",), soft_depends_on=("b",)),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("a", _fn),
|
||||
px.TaskSpec("b", _fn),
|
||||
px.TaskSpec("c", _fn, depends_on=("a",), soft_depends_on=("b",)),
|
||||
]
|
||||
)
|
||||
all_deps = graph.all_deps("c")
|
||||
assert set(all_deps) == {"a", "b"}
|
||||
# 硬依赖在前,软依赖在后
|
||||
@@ -183,10 +203,12 @@ def test_spec_accessor() -> None:
|
||||
|
||||
|
||||
def test_dependencies_accessor() -> None:
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("a", _fn),
|
||||
px.TaskSpec("b", _fn, depends_on=("a",)),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("a", _fn),
|
||||
px.TaskSpec("b", _fn, depends_on=("a",)),
|
||||
]
|
||||
)
|
||||
assert graph.dependencies("a") == ()
|
||||
assert graph.dependencies("b") == ("a",)
|
||||
|
||||
@@ -205,16 +227,18 @@ def test_empty_graph_layers() -> None:
|
||||
|
||||
def test_subgraph_preserves_metadata() -> None:
|
||||
"""子图应保留原任务的 retry/timeout/tags 等元数据。"""
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec(
|
||||
"a",
|
||||
_fn,
|
||||
tags=("x",),
|
||||
retry=px.RetryPolicy(max_attempts=3),
|
||||
timeout=5.0,
|
||||
),
|
||||
px.TaskSpec("b", _fn, depends_on=("a",), tags=("y",)),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec(
|
||||
"a",
|
||||
_fn,
|
||||
tags=("x",),
|
||||
retry=px.RetryPolicy(max_attempts=3),
|
||||
timeout=5.0,
|
||||
),
|
||||
px.TaskSpec("b", _fn, depends_on=("a",), tags=("y",)),
|
||||
]
|
||||
)
|
||||
sub = graph.subgraph(["x"])
|
||||
spec = sub.spec("a")
|
||||
assert spec.retry.max_attempts == 3
|
||||
@@ -250,10 +274,12 @@ def test_from_specs_with_invalid_type() -> None:
|
||||
# ---------------------------------------------------------------------- #
|
||||
def test_to_mermaid_soft_depends_on() -> None:
|
||||
"""to_mermaid 应正确绘制软依赖为虚线."""
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("a", _fn),
|
||||
px.TaskSpec("b", _fn, soft_depends_on=("a",)),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("a", _fn),
|
||||
px.TaskSpec("b", _fn, soft_depends_on=("a",)),
|
||||
]
|
||||
)
|
||||
mermaid = graph.to_mermaid()
|
||||
assert "a -.-> b" in mermaid # 软依赖用虚线
|
||||
|
||||
@@ -355,11 +381,13 @@ def test_graph_composer_expand_refs_ref_returns_empty() -> None:
|
||||
def test_graph_composer_expand_refs_multiple_original_specs_serialized() -> None:
|
||||
"""expand_refs 多个 original_specs 应串行依赖,且首个依赖 ref 末任务."""
|
||||
graph_a = px.Graph.from_specs([px.TaskSpec("a1", _fn)])
|
||||
graph_b = px.Graph.from_specs([
|
||||
px.TaskSpec("b1", _fn),
|
||||
px.TaskSpec("b2", _fn),
|
||||
px.TaskSpec("b3", _fn),
|
||||
])
|
||||
graph_b = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("b1", _fn),
|
||||
px.TaskSpec("b2", _fn),
|
||||
px.TaskSpec("b3", _fn),
|
||||
]
|
||||
)
|
||||
graph_b._pending_refs = ["cmd_a"]
|
||||
|
||||
composer = GraphComposer({"cmd_a": graph_a, "cmd_b": graph_b})
|
||||
|
||||
+62
-44
@@ -95,11 +95,13 @@ class TestProfileReportConstruction:
|
||||
report.results["a"] = _result("a", start, 1.0)
|
||||
report.results["b"] = _result("b", start + timedelta(seconds=1), 2.0)
|
||||
report.results["c"] = _result("c", start + timedelta(seconds=3), 1.5)
|
||||
graph = px.Graph.from_specs([
|
||||
_spec("a"),
|
||||
_spec("b", deps=("a",)),
|
||||
_spec("c", deps=("b",)),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
_spec("a"),
|
||||
_spec("b", deps=("a",)),
|
||||
_spec("c", deps=("b",)),
|
||||
]
|
||||
)
|
||||
|
||||
profile = ProfileReport.from_report(report, graph)
|
||||
|
||||
@@ -136,11 +138,13 @@ class TestProfileReportConstruction:
|
||||
report.results["a"] = _result("a", start, 1.0)
|
||||
report.results["b"] = _result("b", start, 3.0)
|
||||
report.results["c"] = _result("c", start + timedelta(seconds=3), 1.0)
|
||||
graph = px.Graph.from_specs([
|
||||
_spec("a"),
|
||||
_spec("b"),
|
||||
_spec("c", deps=("a", "b")),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
_spec("a"),
|
||||
_spec("b"),
|
||||
_spec("c", deps=("a", "b")),
|
||||
]
|
||||
)
|
||||
|
||||
profile = ProfileReport.from_report(report, graph)
|
||||
|
||||
@@ -188,10 +192,12 @@ class TestWaitTime:
|
||||
report = px.RunReport()
|
||||
report.results["a"] = _result("a", start, 1.0)
|
||||
report.results["b"] = _result("b", start + timedelta(seconds=1.5), 1.0)
|
||||
graph = px.Graph.from_specs([
|
||||
_spec("a"),
|
||||
_spec("b", deps=("a",)),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
_spec("a"),
|
||||
_spec("b", deps=("a",)),
|
||||
]
|
||||
)
|
||||
|
||||
profile = ProfileReport.from_report(report, graph)
|
||||
|
||||
@@ -204,10 +210,12 @@ class TestWaitTime:
|
||||
report.results["a"] = _result("a", start, 2.0)
|
||||
# b 在 a 还没完成时就开始(不应该但可能发生)
|
||||
report.results["b"] = _result("b", start + timedelta(seconds=1), 1.0)
|
||||
graph = px.Graph.from_specs([
|
||||
_spec("a"),
|
||||
_spec("b", deps=("a",)),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
_spec("a"),
|
||||
_spec("b", deps=("a",)),
|
||||
]
|
||||
)
|
||||
|
||||
profile = ProfileReport.from_report(report, graph)
|
||||
|
||||
@@ -220,10 +228,12 @@ class TestWaitTime:
|
||||
report = px.RunReport()
|
||||
report.results["a"] = _result("a", start, 1.0)
|
||||
report.results["b"] = _skipped_result("b")
|
||||
graph = px.Graph.from_specs([
|
||||
_spec("a"),
|
||||
_spec("b", deps=("a",)),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
_spec("a"),
|
||||
_spec("b", deps=("a",)),
|
||||
]
|
||||
)
|
||||
|
||||
profile = ProfileReport.from_report(report, graph)
|
||||
|
||||
@@ -241,12 +251,14 @@ class TestCriticalPath:
|
||||
report.results["b"] = _result("b", start + timedelta(seconds=1), 3.0)
|
||||
report.results["c"] = _result("c", start + timedelta(seconds=1), 1.0)
|
||||
report.results["d"] = _result("d", start + timedelta(seconds=4), 1.0)
|
||||
graph = px.Graph.from_specs([
|
||||
_spec("a"),
|
||||
_spec("b", deps=("a",)),
|
||||
_spec("c", deps=("a",)),
|
||||
_spec("d", deps=("b", "c")),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
_spec("a"),
|
||||
_spec("b", deps=("a",)),
|
||||
_spec("c", deps=("a",)),
|
||||
_spec("d", deps=("b", "c")),
|
||||
]
|
||||
)
|
||||
|
||||
profile = ProfileReport.from_report(report, graph)
|
||||
|
||||
@@ -393,12 +405,14 @@ class TestQueries:
|
||||
report.results["b"] = _result("b", start + timedelta(seconds=1), 3.0)
|
||||
report.results["c"] = _result("c", start + timedelta(seconds=1), 1.0)
|
||||
report.results["d"] = _result("d", start + timedelta(seconds=4), 1.0)
|
||||
graph = px.Graph.from_specs([
|
||||
_spec("a"),
|
||||
_spec("b", deps=("a",)),
|
||||
_spec("c", deps=("a",)),
|
||||
_spec("d", deps=("b", "c")),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
_spec("a"),
|
||||
_spec("b", deps=("a",)),
|
||||
_spec("c", deps=("a",)),
|
||||
_spec("d", deps=("b", "c")),
|
||||
]
|
||||
)
|
||||
|
||||
profile = ProfileReport.from_report(report, graph)
|
||||
|
||||
@@ -537,11 +551,13 @@ class TestIntegrationWithRun:
|
||||
time.sleep(0.01) # 确保任务有实际耗时,避免 duration 极小导致并行度计算为 0
|
||||
return 1
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("a", slow),
|
||||
px.TaskSpec("b", slow, depends_on=("a",)),
|
||||
px.TaskSpec("c", slow, depends_on=("a",)),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("a", slow),
|
||||
px.TaskSpec("b", slow, depends_on=("a",)),
|
||||
px.TaskSpec("c", slow, depends_on=("a",)),
|
||||
]
|
||||
)
|
||||
report = px.run(graph, strategy="sequential")
|
||||
|
||||
profile = ProfileReport.from_report(report, graph)
|
||||
@@ -560,11 +576,13 @@ class TestIntegrationWithRun:
|
||||
time.sleep(0.05)
|
||||
return 1
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("a", slow),
|
||||
px.TaskSpec("b", slow),
|
||||
px.TaskSpec("c", slow),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("a", slow),
|
||||
px.TaskSpec("b", slow),
|
||||
px.TaskSpec("c", slow),
|
||||
]
|
||||
)
|
||||
report = px.run(graph, strategy="thread", max_workers=3)
|
||||
|
||||
profile = ProfileReport.from_report(report, graph)
|
||||
|
||||
@@ -0,0 +1,253 @@
|
||||
"""``pyflowx.registry`` 模块的单元测试."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
|
||||
import pytest
|
||||
|
||||
import pyflowx as px
|
||||
from pyflowx.registry import _REGISTRY, FnRegistry, get_fn, has_fn, register_fn
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def clear_registry() -> Any:
|
||||
"""每个测试前后清空 registry, 但保留 ops 模块自动注册的函数.
|
||||
|
||||
ops 模块在首次导入时注册函数, 模块不会重新执行, 因此清空后无法重新注册.
|
||||
保存原始状态并在 teardown 时恢复, 确保 TestOpsModules 等不使用此 fixture 的
|
||||
测试仍能看到 ops 的注册.
|
||||
"""
|
||||
saved = dict(_REGISTRY)
|
||||
_REGISTRY.clear()
|
||||
yield
|
||||
_REGISTRY.clear()
|
||||
_REGISTRY.update(saved)
|
||||
|
||||
|
||||
class TestRegisterFn:
|
||||
"""``register_fn`` 装饰器测试."""
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _clear(self, clear_registry: Any) -> None:
|
||||
"""每个测试前后清空 registry."""
|
||||
|
||||
def test_register_with_explicit_name(self) -> None:
|
||||
"""使用显式名称注册函数."""
|
||||
|
||||
@register_fn("custom_name")
|
||||
def _func(x: int) -> int:
|
||||
return x * 2
|
||||
|
||||
assert has_fn("custom_name")
|
||||
assert get_fn("custom_name")(5) == 10
|
||||
|
||||
def test_register_without_parentheses(self) -> None:
|
||||
"""无括号使用 ``@register_fn`` 时以 ``__name__`` 注册."""
|
||||
|
||||
@register_fn
|
||||
def my_func(x: int) -> int:
|
||||
return x + 1
|
||||
|
||||
assert has_fn("my_func")
|
||||
assert get_fn("my_func")(10) == 11
|
||||
|
||||
def test_register_with_none_uses_func_name(self) -> None:
|
||||
"""``@register_fn(None)`` 等价于使用 ``__name__``."""
|
||||
|
||||
@register_fn(None)
|
||||
def auto_named(x: int) -> int:
|
||||
return x
|
||||
|
||||
assert has_fn("auto_named")
|
||||
|
||||
def test_register_preserves_function(self) -> None:
|
||||
"""装饰后函数仍可直接调用."""
|
||||
|
||||
@register_fn("calc")
|
||||
def _calc(a: int, b: int) -> int:
|
||||
return a + b
|
||||
|
||||
assert _calc(2, 3) == 5
|
||||
|
||||
def test_duplicate_registration_raises(self) -> None:
|
||||
"""重复注册同名函数抛出 ValueError."""
|
||||
with pytest.raises(ValueError, match="已注册"):
|
||||
|
||||
@register_fn("dup")
|
||||
def _first() -> None:
|
||||
pass
|
||||
|
||||
@register_fn("dup")
|
||||
def _second() -> None:
|
||||
pass
|
||||
|
||||
def test_duplicate_registration_without_parentheses(self) -> None:
|
||||
"""无括号模式下重复注册也抛出 ValueError."""
|
||||
with pytest.raises(ValueError, match="已注册"):
|
||||
|
||||
@register_fn
|
||||
def shared_name() -> None:
|
||||
pass
|
||||
|
||||
@register_fn
|
||||
def shared_name() -> None: # noqa: F811
|
||||
pass
|
||||
|
||||
|
||||
class TestGetFn:
|
||||
"""``get_fn`` 函数测试."""
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _clear(self, clear_registry: Any) -> None:
|
||||
"""每个测试前后清空 registry."""
|
||||
|
||||
def test_get_registered_function(self) -> None:
|
||||
"""获取已注册的函数."""
|
||||
|
||||
@register_fn("target")
|
||||
def _target(x: int) -> int:
|
||||
return x * 3
|
||||
|
||||
retrieved = get_fn("target")
|
||||
assert retrieved is _target
|
||||
|
||||
def test_get_unregistered_raises_keyerror(self) -> None:
|
||||
"""获取未注册的函数抛出 KeyError."""
|
||||
with pytest.raises(KeyError, match="not_found"):
|
||||
get_fn("not_found")
|
||||
|
||||
|
||||
class TestHasFn:
|
||||
"""``has_fn`` 函数测试."""
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _clear(self, clear_registry: Any) -> None:
|
||||
"""每个测试前后清空 registry."""
|
||||
|
||||
def test_has_registered(self) -> None:
|
||||
"""已注册返回 True."""
|
||||
|
||||
@register_fn("exists")
|
||||
def _exists() -> None:
|
||||
pass
|
||||
|
||||
assert has_fn("exists") is True
|
||||
|
||||
def test_has_not_registered(self) -> None:
|
||||
"""未注册返回 False."""
|
||||
assert has_fn("nope") is False
|
||||
|
||||
|
||||
class TestFnRegistry:
|
||||
"""``FnRegistry`` 类测试."""
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _clear(self, clear_registry: Any) -> None:
|
||||
"""每个测试前后清空 registry."""
|
||||
|
||||
def test_register_method(self) -> None:
|
||||
"""``FnRegistry.register`` 等价于 ``register_fn``."""
|
||||
|
||||
@FnRegistry.register("via_class")
|
||||
def _func(x: int) -> int:
|
||||
return x
|
||||
|
||||
assert FnRegistry.has("via_class")
|
||||
assert FnRegistry.get("via_class")(7) == 7
|
||||
|
||||
def test_register_method_without_name(self) -> None:
|
||||
"""``FnRegistry.register(None)`` 使用函数名."""
|
||||
|
||||
@FnRegistry.register()
|
||||
def class_auto(x: int) -> int:
|
||||
return x
|
||||
|
||||
assert FnRegistry.has("class_auto")
|
||||
|
||||
def test_get_method(self) -> None:
|
||||
"""``FnRegistry.get`` 获取已注册函数."""
|
||||
|
||||
@register_fn("klass")
|
||||
def _klass() -> str:
|
||||
return "hello"
|
||||
|
||||
assert FnRegistry.get("klass")() == "hello"
|
||||
|
||||
def test_has_method_false(self) -> None:
|
||||
"""``FnRegistry.has`` 未注册返回 False."""
|
||||
assert FnRegistry.has("missing") is False
|
||||
|
||||
def test_clear_method(self) -> None:
|
||||
"""``FnRegistry.clear`` 清空注册表."""
|
||||
|
||||
@register_fn("temp")
|
||||
def _temp() -> None:
|
||||
pass
|
||||
|
||||
assert FnRegistry.has("temp")
|
||||
FnRegistry.clear()
|
||||
assert not FnRegistry.has("temp")
|
||||
|
||||
def test_names_method(self) -> None:
|
||||
"""``FnRegistry.names`` 返回所有注册名."""
|
||||
|
||||
@register_fn("alpha")
|
||||
def _alpha() -> None:
|
||||
pass
|
||||
|
||||
@register_fn("beta")
|
||||
def _beta() -> None:
|
||||
pass
|
||||
|
||||
names = FnRegistry.names()
|
||||
assert "alpha" in names
|
||||
assert "beta" in names
|
||||
|
||||
def test_names_returns_copy(self) -> None:
|
||||
"""``names`` 返回列表副本, 修改不影响内部状态."""
|
||||
|
||||
@register_fn("orig")
|
||||
def _orig() -> None:
|
||||
pass
|
||||
|
||||
names = FnRegistry.names()
|
||||
names.append("fake")
|
||||
assert not FnRegistry.has("fake")
|
||||
|
||||
|
||||
class TestOpsModules:
|
||||
"""``ops`` 子模块导入后函数自动注册的集成测试.
|
||||
|
||||
这些测试不使用 ``_clear_registry`` fixture, 因为 ``ops`` 模块在首次导入时
|
||||
注册函数, 清空后将无法重新注册 (模块不会再次执行).
|
||||
"""
|
||||
|
||||
def test_all_ops_functions_registered(self) -> None:
|
||||
"""导入 ``ops`` 后所有函数应已注册."""
|
||||
import inspect
|
||||
|
||||
from pyflowx.ops import dev, files, media, system
|
||||
|
||||
for module in (files, dev, media, system):
|
||||
for name in module.__all__:
|
||||
obj = getattr(module, name)
|
||||
if not inspect.isfunction(obj):
|
||||
continue
|
||||
assert px.has_fn(name), f"{module.__name__}.{name} 未注册"
|
||||
|
||||
def test_total_function_count(self) -> None:
|
||||
"""注册函数总数 = 18+15+18+12 = 63."""
|
||||
from pyflowx.ops import dev, files, media, system # noqa: F401
|
||||
|
||||
all_names = px.FnRegistry.names()
|
||||
assert len(all_names) == 63
|
||||
|
||||
def test_specific_functions_callable(self) -> None:
|
||||
"""关键注册函数可调用."""
|
||||
from pyflowx.ops import dev, files, media, system
|
||||
|
||||
assert px.get_fn("get_file_timestamp") is files.get_file_timestamp
|
||||
assert px.get_fn("bump_file_version") is dev.bump_file_version
|
||||
assert px.get_fn("pdf_merge") is media.pdf_merge
|
||||
assert px.get_fn("ssh_copy_id") is system.ssh_copy_id
|
||||
@@ -172,4 +172,3 @@ class TestRunReportQueries:
|
||||
report.results["a"] = TaskResult[Any](spec=spec, status=TaskStatus.PENDING)
|
||||
durs = report.durations()
|
||||
assert durs["a"] == 0.0
|
||||
|
||||
|
||||
+55
-70
@@ -29,20 +29,24 @@ def _echo_graph(name: str = "echo_task", msg: str = "hello") -> px.Graph:
|
||||
|
||||
def _failing_graph() -> px.Graph:
|
||||
"""构造一个必定失败的单任务图."""
|
||||
return px.Graph.from_specs([
|
||||
px.TaskSpec(
|
||||
"fail",
|
||||
cmd=["python", "-c", "import sys; sys.exit(1)"],
|
||||
)
|
||||
])
|
||||
return px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec(
|
||||
"fail",
|
||||
cmd=["python", "-c", "import sys; sys.exit(1)"],
|
||||
)
|
||||
]
|
||||
)
|
||||
|
||||
|
||||
def _multi_task_graph() -> px.Graph:
|
||||
"""构造一个带依赖的多任务图."""
|
||||
return px.Graph.from_specs([
|
||||
px.TaskSpec("a", cmd=[*ECHO_CMD, "a"]),
|
||||
px.TaskSpec("b", cmd=[*ECHO_CMD, "b"], depends_on=("a",)),
|
||||
])
|
||||
return px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("a", cmd=[*ECHO_CMD, "a"]),
|
||||
px.TaskSpec("b", cmd=[*ECHO_CMD, "b"], depends_on=("a",)),
|
||||
]
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
@@ -314,13 +318,15 @@ class TestCliRunnerVerbose:
|
||||
|
||||
def test_verbose_prints_skip_lifecycle(self, capsys: pytest.CaptureFixture[str]) -> None:
|
||||
"""verbose 模式下跳过的任务应打印跳过信息."""
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec(
|
||||
"skip_me",
|
||||
cmd=[*ECHO_CMD, "skip"],
|
||||
conditions=(lambda _ctx: False,),
|
||||
),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec(
|
||||
"skip_me",
|
||||
cmd=[*ECHO_CMD, "skip"],
|
||||
conditions=(lambda _ctx: False,),
|
||||
),
|
||||
]
|
||||
)
|
||||
runner = px.CliRunner(aliases={"skip": graph})
|
||||
_ = runner.run(["skip"])
|
||||
captured = capsys.readouterr()
|
||||
@@ -517,26 +523,30 @@ class TestCliRunnerIntegration:
|
||||
|
||||
def test_condition_skipped_command_succeeds(self) -> None:
|
||||
"""条件不满足时任务跳过, 整体仍成功."""
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec(
|
||||
"skip_me",
|
||||
cmd=[*ECHO_CMD, "should not run"],
|
||||
conditions=(lambda _ctx: False,),
|
||||
),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec(
|
||||
"skip_me",
|
||||
cmd=[*ECHO_CMD, "should not run"],
|
||||
conditions=(lambda _ctx: False,),
|
||||
),
|
||||
]
|
||||
)
|
||||
runner = px.CliRunner(aliases={"skip": graph})
|
||||
exit_code = runner.run(["skip"])
|
||||
assert exit_code == CliExitCode.SUCCESS.value
|
||||
|
||||
def test_condition_met_command_succeeds(self) -> None:
|
||||
"""条件满足时任务执行, 整体成功."""
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec(
|
||||
"run_me",
|
||||
cmd=[*ECHO_CMD, "should run"],
|
||||
conditions=(lambda _ctx: True,),
|
||||
),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec(
|
||||
"run_me",
|
||||
cmd=[*ECHO_CMD, "should run"],
|
||||
conditions=(lambda _ctx: True,),
|
||||
),
|
||||
]
|
||||
)
|
||||
runner = px.CliRunner(aliases={"run": graph})
|
||||
exit_code = runner.run(["run"])
|
||||
assert exit_code == CliExitCode.SUCCESS.value
|
||||
@@ -552,12 +562,14 @@ class TestCliRunnerIntegration:
|
||||
|
||||
return fn
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("a", make("a")),
|
||||
px.TaskSpec("b", make("b"), depends_on=("a",)),
|
||||
px.TaskSpec("c", make("c"), depends_on=("a",)),
|
||||
px.TaskSpec("d", make("d"), depends_on=("b", "c")),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("a", make("a")),
|
||||
px.TaskSpec("b", make("b"), depends_on=("a",)),
|
||||
px.TaskSpec("c", make("c"), depends_on=("a",)),
|
||||
px.TaskSpec("d", make("d"), depends_on=("b", "c")),
|
||||
]
|
||||
)
|
||||
runner = px.CliRunner(aliases={"diamond": graph})
|
||||
exit_code = runner.run(["diamond"])
|
||||
assert exit_code == CliExitCode.SUCCESS.value
|
||||
@@ -596,35 +608,6 @@ class TestCliRunnerIntegration:
|
||||
assert exit_code == CliExitCode.SUCCESS.value
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
# _apply_verbose_to_graph (补充覆盖)
|
||||
# ---------------------------------------------------------------------- #
|
||||
class TestApplyVerboseToGraph:
|
||||
"""测试 _apply_verbose_to_graph 函数 (补充覆盖)."""
|
||||
|
||||
def test_specs_with_matching_verbose_are_kept(self) -> None:
|
||||
"""spec.verbose 已与目标值匹配时应保留原 spec (覆盖 runner.py line 57)."""
|
||||
from pyflowx.runner import _apply_verbose_to_graph
|
||||
|
||||
# 创建 verbose=True 的 spec
|
||||
graph = px.Graph.from_specs([px.TaskSpec("a", cmd=[*ECHO_CMD, "a"], verbose=True)])
|
||||
# 应用 verbose=True, spec.verbose 已匹配, 应保留原 spec
|
||||
new_graph = _apply_verbose_to_graph(graph, verbose=True)
|
||||
new_spec = new_graph.spec("a")
|
||||
assert new_spec.verbose is True
|
||||
|
||||
def test_specs_with_non_matching_verbose_are_replaced(self) -> None:
|
||||
"""spec.verbose 与目标值不匹配时应替换 (覆盖 else 分支)."""
|
||||
from pyflowx.runner import _apply_verbose_to_graph
|
||||
|
||||
# 创建 verbose=False 的 spec
|
||||
graph = px.Graph.from_specs([px.TaskSpec("a", cmd=[*ECHO_CMD, "a"], verbose=False)])
|
||||
# 应用 verbose=True, spec.verbose 不匹配, 应替换
|
||||
new_graph = _apply_verbose_to_graph(graph, verbose=True)
|
||||
new_spec = new_graph.spec("a")
|
||||
assert new_spec.verbose is True
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
# 新 API: tasks + aliases
|
||||
# ---------------------------------------------------------------------- #
|
||||
@@ -661,10 +644,12 @@ class TestCliRunnerNewApi:
|
||||
|
||||
def test_aliases_graph_value(self) -> None:
|
||||
"""aliases 值为 Graph 时原样使用(复杂场景:conditions 等)."""
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("a", cmd=[*ECHO_CMD, "a"]),
|
||||
px.TaskSpec("b", cmd=[*ECHO_CMD, "b"], depends_on=("a",)),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("a", cmd=[*ECHO_CMD, "a"]),
|
||||
px.TaskSpec("b", cmd=[*ECHO_CMD, "b"], depends_on=("a",)),
|
||||
]
|
||||
)
|
||||
runner = px.CliRunner(aliases={"g": graph})
|
||||
assert set(runner.graphs["g"].all_specs().keys()) == {"a", "b"}
|
||||
|
||||
|
||||
+4
-2
@@ -193,8 +193,10 @@ def test_should_execute_skip_if_missing_cmd_not_found() -> None:
|
||||
|
||||
def test_should_execute_skip_if_missing_cmd_found() -> None:
|
||||
"""skip_if_missing 但命令存在时应执行."""
|
||||
# 使用 Python 作为已安装的命令(Windows 上 echo 是 shell 内置,shutil.which 找不到)
|
||||
spec = TaskSpec("a", cmd=["python"], skip_if_missing=True) # python 应存在
|
||||
# 使用 sys.executable 确保命令一定存在 (跨平台兼容)
|
||||
import sys
|
||||
|
||||
spec = TaskSpec("a", cmd=[sys.executable], skip_if_missing=True)
|
||||
should_run, reason = spec.should_execute({})
|
||||
assert should_run is True
|
||||
assert reason is None
|
||||
|
||||
+178
-142
@@ -21,9 +21,11 @@ else:
|
||||
|
||||
def test_taskspec_with_cmd_list():
|
||||
"""测试使用命令列表的 TaskSpec."""
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("echo_test", cmd=[*ECHO_CMD, "hello"]),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("echo_test", cmd=[*ECHO_CMD, "hello"]),
|
||||
]
|
||||
)
|
||||
|
||||
report = px.run(graph, strategy="sequential")
|
||||
assert report.success
|
||||
@@ -38,9 +40,11 @@ def test_taskspec_with_cmd_string():
|
||||
else:
|
||||
shell_cmd = "echo 'hello from shell'"
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("shell_test", cmd=shell_cmd),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("shell_test", cmd=shell_cmd),
|
||||
]
|
||||
)
|
||||
|
||||
report = px.run(graph, strategy="sequential")
|
||||
assert report.success
|
||||
@@ -55,13 +59,15 @@ def test_taskspec_with_conditions_skip():
|
||||
def never_true(_ctx):
|
||||
return False
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec(
|
||||
"should_skip",
|
||||
cmd=[*ECHO_CMD, "this should not run"],
|
||||
conditions=(never_true,),
|
||||
),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec(
|
||||
"should_skip",
|
||||
cmd=[*ECHO_CMD, "this should not run"],
|
||||
conditions=(never_true,),
|
||||
),
|
||||
]
|
||||
)
|
||||
|
||||
report = px.run(graph, strategy="sequential")
|
||||
assert report.success
|
||||
@@ -76,13 +82,15 @@ def test_taskspec_with_conditions_execute():
|
||||
def always_true(_ctx):
|
||||
return True
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec(
|
||||
"should_run",
|
||||
cmd=[*ECHO_CMD, "this should run"],
|
||||
conditions=(always_true,),
|
||||
),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec(
|
||||
"should_run",
|
||||
cmd=[*ECHO_CMD, "this should run"],
|
||||
conditions=(always_true,),
|
||||
),
|
||||
]
|
||||
)
|
||||
|
||||
report = px.run(graph, strategy="sequential")
|
||||
assert report.success
|
||||
@@ -99,23 +107,25 @@ def test_platform_conditions():
|
||||
win_cmd = ["echo", "Windows"]
|
||||
posix_cmd = ["echo", "POSIX"]
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec(
|
||||
"win_task",
|
||||
cmd=win_cmd,
|
||||
conditions=(lambda _ctx: Constants.IS_WINDOWS,),
|
||||
),
|
||||
px.TaskSpec(
|
||||
"linux_task",
|
||||
cmd=posix_cmd,
|
||||
conditions=(lambda _ctx: Constants.IS_LINUX,),
|
||||
),
|
||||
px.TaskSpec(
|
||||
"macos_task",
|
||||
cmd=posix_cmd,
|
||||
conditions=(lambda _ctx: Constants.IS_MACOS,),
|
||||
),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec(
|
||||
"win_task",
|
||||
cmd=win_cmd,
|
||||
conditions=(lambda _ctx: Constants.IS_WINDOWS,),
|
||||
),
|
||||
px.TaskSpec(
|
||||
"linux_task",
|
||||
cmd=posix_cmd,
|
||||
conditions=(lambda _ctx: Constants.IS_LINUX,),
|
||||
),
|
||||
px.TaskSpec(
|
||||
"macos_task",
|
||||
cmd=posix_cmd,
|
||||
conditions=(lambda _ctx: Constants.IS_MACOS,),
|
||||
),
|
||||
]
|
||||
)
|
||||
|
||||
report = px.run(graph, strategy="sequential")
|
||||
assert report.success
|
||||
@@ -141,13 +151,15 @@ def test_app_installed_conditions():
|
||||
python_cmd = [sys.executable, "--version"]
|
||||
py_name = "python" if sys.platform == "win32" else "python3"
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec(
|
||||
"python_check",
|
||||
cmd=python_cmd,
|
||||
conditions=(BuiltinConditions.HAS_INSTALLED(py_name),),
|
||||
),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec(
|
||||
"python_check",
|
||||
cmd=python_cmd,
|
||||
conditions=(BuiltinConditions.HAS_INSTALLED(py_name),),
|
||||
),
|
||||
]
|
||||
)
|
||||
|
||||
report = px.run(graph, strategy="sequential")
|
||||
assert report.success
|
||||
@@ -173,23 +185,25 @@ def test_combined_conditions():
|
||||
# NOT 条件
|
||||
not_condition = BuiltinConditions.NOT(lambda _ctx: False)
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec(
|
||||
"and_test",
|
||||
cmd=[*ECHO_CMD, "AND"],
|
||||
conditions=(and_condition,),
|
||||
),
|
||||
px.TaskSpec(
|
||||
"or_test",
|
||||
cmd=[*ECHO_CMD, "OR"],
|
||||
conditions=(or_condition,),
|
||||
),
|
||||
px.TaskSpec(
|
||||
"not_test",
|
||||
cmd=[*ECHO_CMD, "NOT"],
|
||||
conditions=(not_condition,),
|
||||
),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec(
|
||||
"and_test",
|
||||
cmd=[*ECHO_CMD, "AND"],
|
||||
conditions=(and_condition,),
|
||||
),
|
||||
px.TaskSpec(
|
||||
"or_test",
|
||||
cmd=[*ECHO_CMD, "OR"],
|
||||
conditions=(or_condition,),
|
||||
),
|
||||
px.TaskSpec(
|
||||
"not_test",
|
||||
cmd=[*ECHO_CMD, "NOT"],
|
||||
conditions=(not_condition,),
|
||||
),
|
||||
]
|
||||
)
|
||||
|
||||
report = px.run(graph, strategy="sequential")
|
||||
assert report.success
|
||||
@@ -205,13 +219,15 @@ def test_taskspec_with_cwd():
|
||||
else:
|
||||
ls_cmd = ["ls", "-la"]
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec(
|
||||
"list_current",
|
||||
cmd=ls_cmd,
|
||||
cwd=Path.cwd(),
|
||||
),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec(
|
||||
"list_current",
|
||||
cmd=ls_cmd,
|
||||
cwd=Path.cwd(),
|
||||
),
|
||||
]
|
||||
)
|
||||
|
||||
report = px.run(graph, strategy="sequential")
|
||||
assert report.success
|
||||
@@ -222,14 +238,16 @@ def test_taskspec_with_cwd():
|
||||
@pytest.mark.slow
|
||||
def test_taskspec_with_timeout():
|
||||
"""测试超时设置."""
|
||||
graph = px.Graph.from_specs([
|
||||
# 短时间任务应该成功
|
||||
px.TaskSpec(
|
||||
"short_task",
|
||||
cmd=[sys.executable, "-c", "import time; time.sleep(0.1)"],
|
||||
timeout=1.0,
|
||||
),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
# 短时间任务应该成功
|
||||
px.TaskSpec(
|
||||
"short_task",
|
||||
cmd=[sys.executable, "-c", "import time; time.sleep(0.1)"],
|
||||
timeout=1.0,
|
||||
),
|
||||
]
|
||||
)
|
||||
|
||||
report = px.run(graph, strategy="sequential")
|
||||
assert report.success
|
||||
@@ -239,24 +257,26 @@ def test_taskspec_with_timeout():
|
||||
|
||||
def test_taskspec_dependency_with_conditions():
|
||||
"""测试依赖和条件的组合."""
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec(
|
||||
"first",
|
||||
cmd=[*ECHO_CMD, "first"],
|
||||
conditions=(lambda _ctx: True,),
|
||||
),
|
||||
px.TaskSpec(
|
||||
"second",
|
||||
cmd=[*ECHO_CMD, "second"],
|
||||
depends_on=("first",),
|
||||
conditions=(lambda _ctx: True,),
|
||||
),
|
||||
px.TaskSpec(
|
||||
"third",
|
||||
cmd=[*ECHO_CMD, "third"],
|
||||
depends_on=("second",),
|
||||
),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec(
|
||||
"first",
|
||||
cmd=[*ECHO_CMD, "first"],
|
||||
conditions=(lambda _ctx: True,),
|
||||
),
|
||||
px.TaskSpec(
|
||||
"second",
|
||||
cmd=[*ECHO_CMD, "second"],
|
||||
depends_on=("first",),
|
||||
conditions=(lambda _ctx: True,),
|
||||
),
|
||||
px.TaskSpec(
|
||||
"third",
|
||||
cmd=[*ECHO_CMD, "third"],
|
||||
depends_on=("second",),
|
||||
),
|
||||
]
|
||||
)
|
||||
|
||||
report = px.run(graph, strategy="sequential")
|
||||
assert report.success
|
||||
@@ -271,10 +291,12 @@ def test_taskspec_mixed_fn_and_cmd():
|
||||
def my_function():
|
||||
return "result from function"
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("fn_task", fn=my_function),
|
||||
px.TaskSpec("cmd_task", cmd=[*ECHO_CMD, "from command"]),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("fn_task", fn=my_function),
|
||||
px.TaskSpec("cmd_task", cmd=[*ECHO_CMD, "from command"]),
|
||||
]
|
||||
)
|
||||
|
||||
report = px.run(graph, strategy="sequential")
|
||||
assert report.success
|
||||
@@ -289,13 +311,15 @@ def test_taskspec_cmd_overrides_fn():
|
||||
def my_function():
|
||||
return "should not run"
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec(
|
||||
"cmd_priority",
|
||||
fn=my_function,
|
||||
cmd=[*ECHO_CMD, "cmd takes priority"],
|
||||
),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec(
|
||||
"cmd_priority",
|
||||
fn=my_function,
|
||||
cmd=[*ECHO_CMD, "cmd takes priority"],
|
||||
),
|
||||
]
|
||||
)
|
||||
|
||||
report = px.run(graph, strategy="sequential")
|
||||
assert report.success
|
||||
@@ -310,9 +334,11 @@ def test_taskspec_callable_cmd():
|
||||
def my_callable():
|
||||
return "callable result"
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("callable_cmd", cmd=my_callable),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("callable_cmd", cmd=my_callable),
|
||||
]
|
||||
)
|
||||
|
||||
report = px.run(graph, strategy="sequential")
|
||||
assert report.success
|
||||
@@ -373,13 +399,15 @@ class TestTaskSpecVerbose:
|
||||
"""verbose=True 时失败也应打印返回码."""
|
||||
from pyflowx.errors import TaskFailedError
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec(
|
||||
"fail",
|
||||
cmd=[sys.executable, "-c", "import sys; sys.exit(1)"],
|
||||
verbose=True,
|
||||
)
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec(
|
||||
"fail",
|
||||
cmd=[sys.executable, "-c", "import sys; sys.exit(1)"],
|
||||
verbose=True,
|
||||
)
|
||||
]
|
||||
)
|
||||
with pytest.raises(TaskFailedError):
|
||||
_ = px.run(graph, strategy="sequential")
|
||||
captured = capsys.readouterr()
|
||||
@@ -408,16 +436,18 @@ class TestTaskSpecCmdErrors:
|
||||
"""命令失败时错误信息应包含 stderr."""
|
||||
from pyflowx.errors import TaskFailedError
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec(
|
||||
"fail",
|
||||
cmd=[
|
||||
sys.executable,
|
||||
"-c",
|
||||
"import sys; sys.stderr.write('error-msg'); sys.exit(1)",
|
||||
],
|
||||
)
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec(
|
||||
"fail",
|
||||
cmd=[
|
||||
sys.executable,
|
||||
"-c",
|
||||
"import sys; sys.stderr.write('error-msg'); sys.exit(1)",
|
||||
],
|
||||
)
|
||||
]
|
||||
)
|
||||
with pytest.raises(TaskFailedError) as exc_info:
|
||||
_ = px.run(graph, strategy="sequential")
|
||||
# 非 verbose 模式下, stderr 应包含在错误信息中
|
||||
@@ -435,9 +465,11 @@ class TestTaskSpecCmdErrors:
|
||||
"""shell 命令失败时应抛出 RuntimeError."""
|
||||
from pyflowx.errors import TaskFailedError
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec("fail", cmd=f'{sys.executable} -c "import sys; sys.exit(1)"'),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("fail", cmd=f'{sys.executable} -c "import sys; sys.exit(1)"'),
|
||||
]
|
||||
)
|
||||
with pytest.raises(TaskFailedError) as exc_info:
|
||||
_ = px.run(graph, strategy="sequential")
|
||||
assert "Shell 命令执行失败" in str(exc_info.value.cause)
|
||||
@@ -447,13 +479,15 @@ class TestTaskSpecCmdErrors:
|
||||
"""命令超时应抛出 RuntimeError."""
|
||||
from pyflowx.errors import TaskFailedError
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec(
|
||||
"slow",
|
||||
cmd=[sys.executable, "-c", "import time; time.sleep(5)"],
|
||||
timeout=0.1,
|
||||
)
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec(
|
||||
"slow",
|
||||
cmd=[sys.executable, "-c", "import time; time.sleep(5)"],
|
||||
timeout=0.1,
|
||||
)
|
||||
]
|
||||
)
|
||||
with pytest.raises(TaskFailedError) as exc_info:
|
||||
_ = px.run(graph, strategy="sequential")
|
||||
assert "超时" in str(exc_info.value.cause)
|
||||
@@ -463,13 +497,15 @@ class TestTaskSpecCmdErrors:
|
||||
"""shell 命令超时应抛出 RuntimeError."""
|
||||
from pyflowx.errors import TaskFailedError
|
||||
|
||||
graph = px.Graph.from_specs([
|
||||
px.TaskSpec(
|
||||
"slow",
|
||||
cmd=f'{sys.executable} -c "import time; time.sleep(5)"',
|
||||
timeout=0.1,
|
||||
),
|
||||
])
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec(
|
||||
"slow",
|
||||
cmd=f'{sys.executable} -c "import time; time.sleep(5)"',
|
||||
timeout=0.1,
|
||||
),
|
||||
]
|
||||
)
|
||||
with pytest.raises(TaskFailedError) as exc_info:
|
||||
_ = px.run(graph, strategy="sequential")
|
||||
assert "超时" in str(exc_info.value.cause)
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user