Compare commits
20 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| d733452c8a | |||
| c502556db9 | |||
| e1824b9f5a | |||
| 5e93a75987 | |||
| 529a024ea1 | |||
| 4721b59f40 | |||
| fca6f17a5c | |||
| d493c6233e | |||
| 7a87a4177e | |||
| 04d6871e8d | |||
| 08149d990d | |||
| 54aef98f3a | |||
| e31646e281 | |||
| f7fb95af83 | |||
| 6da42ec5ff | |||
| c55a37173a | |||
| 960b8672f4 | |||
| 4fd1d70b58 | |||
| 6fb9223066 | |||
| 1f7127357e |
+1
-2
@@ -38,9 +38,8 @@ env
|
||||
*.swp
|
||||
*.swo
|
||||
|
||||
# 文档与示例(按需保留)
|
||||
# 文档(按需保留)
|
||||
docs
|
||||
examples
|
||||
|
||||
# 系统文件
|
||||
.DS_Store
|
||||
|
||||
@@ -14,3 +14,4 @@ wheels/
|
||||
|
||||
# Sphinx 文档构建输出
|
||||
docs/_build/
|
||||
.trae/refs
|
||||
|
||||
@@ -0,0 +1,62 @@
|
||||
# 迭代 01: typer/rich 迁移
|
||||
|
||||
## 本轮目标
|
||||
|
||||
采用 typer + rich 简化 CLI 设计, 提高显示效果:
|
||||
1. 修复 Python 3.8 兼容性测试失败
|
||||
2. 全量重写 tools.py 为 typer 内核 (保留 @px.tool 装饰器 API)
|
||||
3. 重写 pf.py 为 rich 驱动主入口
|
||||
4. rich 深度集成 (执行生命周期/错误/工具列表/dry-run)
|
||||
5. 放弃 Python 3.8/3.9 支持 (requires-python >=3.10)
|
||||
|
||||
## 改动文件清单
|
||||
|
||||
### 依赖与配置
|
||||
- `pyproject.toml`: requires-python >=3.10, 添加 typer>=0.24.0 + rich>=13.7.0 依赖, ruff target-version=py310, pyrefly python-version=3.10
|
||||
- `tox.ini`: envlist 改为 py310-py314
|
||||
|
||||
### 核心重写
|
||||
- `src/pyflowx/tools.py`: 全量重写 — argparse → typer.Typer; _make_dispatcher 构建命令 dispatcher; _build_typer_app 单命令/多命令路由; _execute_dag DAG 编排; standalone_mode=False + 捕获 typer._click 异常处理退出码; _handle_list_rich rich 表格 --list
|
||||
- `src/pyflowx/cli/pf.py`: 重写 — rich Panel+Table 工具列表 (含别名/说明列), --help/-h/--version/-V 全局选项, 未知工具模糊匹配建议 (difflib), rich Console stderr 错误
|
||||
|
||||
### rich 深度集成
|
||||
- `src/pyflowx/executors.py`: _make_verbose_callback 用 ✓/✗/▸/○ 符号 + 颜色替代 [verbose] 前缀; _print_dry_run rich styled
|
||||
- `src/pyflowx/command.py`: 执行命令/返回码用 rich Console + 颜色 (green=0/red=非0)
|
||||
|
||||
### Python 3.10+ 现代化
|
||||
- `src/pyflowx/graph.py`: 移除 graphlib_backport, 直接 import graphlib
|
||||
- `src/pyflowx/task.py`: Union → X|Y, typing.ContextManager → contextlib.AbstractContextManager
|
||||
- `src/pyflowx/storage.py`: 同上
|
||||
- `src/pyflowx/command.py`: typing.List → list, Union → X|Y
|
||||
- `src/pyflowx/cli/emlmanager.py`: zip() 加 strict= 参数
|
||||
- `src/pyflowx/executors.py`: zip() 加 strict=True
|
||||
|
||||
### 测试
|
||||
- `tests/test_tools.py`: 全量重写 (84 测试) — 覆盖 typer 内核 (_make_dispatcher/_build_typer_app/_execute_dag/退出码三态)
|
||||
- `tests/test_runner.py`: verbose 断言更新 (✓ 替代 [verbose])
|
||||
|
||||
## 关键决策与依据
|
||||
|
||||
1. **保留 @px.tool 装饰器 API**: 用户既有 ops/ 模块全部使用 @px.tool, 改 API 会破坏所有工具. 内部替换 argparse→typer, 外部 API 不变.
|
||||
|
||||
2. **standalone_mode=False**: typer 默认 standalone_mode=True 会 sys.exit() 退出进程, 测试中无法捕获. 用 False + 手动捕获 typer._click 异常 (NoArgsIsHelpError/ClickException/Abort) 处理退出码.
|
||||
|
||||
3. **typer._click 私有导入**: typer 内置 click (typer._click) 与公共 click 是两套独立类层次, 必须从 typer._click.exceptions 导入 NoArgsIsHelpError/ClickException 才能捕获.
|
||||
|
||||
4. **--list 预处理而非 typer callback**: typer 的 no_args_is_help=True 在多命令工具中会拦截 --list (要求先给 subcommand). 在 run_tool 中预处理 --list (任意位置出现即列出任务) 绕过此限制.
|
||||
|
||||
5. **单命令 vs 多命令**: 单命令工具 (仅 subcommand=None) 用 callback(invoke_without_command=True), no_args_is_help=False; 多命令工具用各 subcommand 注册为 command, no_args_is_help=True.
|
||||
|
||||
6. **legacy 工具不迁移**: emlmanager.py 是 Web 应用 (sqlite3+threading+HTTP), profiler.py 用 runpy.run_path, 均不适配 @px.tool DAG 模型. 保留 pf._run_legacy() 路由是更简方案.
|
||||
|
||||
## 验证结果
|
||||
|
||||
- ruff check: 0 errors
|
||||
- ruff format: 96 files already formatted
|
||||
- pyrefly: 0 errors (12 suppressed)
|
||||
- pytest: 1003 passed
|
||||
- coverage: 97.00% (门槛 95%)
|
||||
|
||||
## 遗留事项
|
||||
|
||||
- 无
|
||||
@@ -1,108 +0,0 @@
|
||||
# 文档整理与 Sphinx 文档搭建计划
|
||||
|
||||
## Context
|
||||
|
||||
最近完成 CLI 重构:新增 `pf` 统一入口,13 个工具迁移到 YAML 配置并删除了对应 .py 入口脚本,`run()` 的 verbose 统一应用到 spec。但文档未同步:README 仍引用旧命令(`yamlrun`、`python build.py`),模块结构表缺漏;`runner.py` 的 `_apply_verbose_to_graph` 成为死代码;项目缺少可发布的 Sphinx 文档。本次任务整理这些遗留,并搭建 ReadTheDocs 文档站。
|
||||
|
||||
## 任务范围
|
||||
|
||||
### 1. 清理死代码
|
||||
- 删除 `src/pyflowx/runner.py` 的 `_apply_verbose_to_graph` 函数(line 38-68),功能已移入 `executors.py` 的 `run()`。
|
||||
- 删除 `tests/test_runner.py` 中对应测试(line 610-636,`TestApplyVerboseToGraph` 类)。
|
||||
- 清理 `runner.py` 顶部 `from dataclasses import replace` 若变为未使用。
|
||||
|
||||
### 2. 修复版本不一致
|
||||
- `src/pyflowx/__init__.py:105` 硬编码 `__version__ = "0.4.5"`,`pyproject.toml:25` 为 `0.3.5`。
|
||||
- 统一为 `0.4.5`(`__init__.py` 为准,pyproject.toml 是源但 bumpversion 工具应同时更新两者)。
|
||||
|
||||
### 3. 更新 README.md
|
||||
- L304-308:`python build.py clean/build/test` → `pf pymake clean/build/test`。
|
||||
- L335-351、L435:`yamlrun pipeline.yaml ...` → `pf yamlrun pipeline.yaml ...`(6 处)。
|
||||
- L311:`verbose=True(默认)` 描述保留,但 CLI 示例改为 `pf`。
|
||||
- L558-574 模块结构表:补充 `cli/pf.py`(统一入口)、`cli/configs/`(YAML 工具配置)、`cli/_ops/`(工具函数)、`profiling.py`、`registry.py`。
|
||||
- 顶部增加「文档」徽章链接到 ReadTheDocs。
|
||||
|
||||
### 4. 搭建 Sphinx 文档结构
|
||||
新建 `docs/` 目录:
|
||||
|
||||
```
|
||||
docs/
|
||||
├── conf.py # Sphinx 配置
|
||||
├── index.rst # 首页与目录
|
||||
├── installation.rst # 安装
|
||||
├── quickstart.rst # 快速上手(从 README 提炼)
|
||||
├── guide/
|
||||
│ ├── task.rst # TaskSpec 任务描述
|
||||
│ ├── graph.rst # Graph DAG 构建
|
||||
│ ├── execution.rst # 执行策略与 run()
|
||||
│ ├── yaml.rst # YAML 任务编排
|
||||
│ └── cli.rst # pf 统一入口与工具列表
|
||||
├── api.rst # API 参考(automodule 自动生成)
|
||||
└── changelog.rst # 变更日志摘要
|
||||
```
|
||||
|
||||
**conf.py 要点**:
|
||||
- 扩展:`sphinx.ext.autodoc`、`sphinx.ext.napoleon`(支持 Google/NumPy docstring)、`sphinx.ext.viewcode`、`myst_parser`(支持 Markdown)
|
||||
- 主题:`sphinx_rtd_theme`
|
||||
- 项目版本从 `pyflowx.__version__` 动态读取
|
||||
- `autodoc_default_options`:`members: True, undoc-members: True, show-inheritance: True`
|
||||
|
||||
**api.rst**:用 `automodule:: pyflowx` 抓取 `__all__` 的 56 个公共符号。
|
||||
|
||||
### 5. ReadTheDocs 配置
|
||||
- 新建 `.readthedocs.yaml`:Python 3.11,`pip install -e .[docs]`,`sphinx -b html docs/ docs/_build/`。
|
||||
- `.gitignore` 增加 `docs/_build/`。
|
||||
|
||||
### 6. pyproject.toml 补充 docs 依赖
|
||||
```toml
|
||||
docs = [
|
||||
"sphinx>=7.0",
|
||||
"sphinx-rtd-theme>=2.0",
|
||||
"myst-parser>=3.0",
|
||||
]
|
||||
```
|
||||
并在 `[dependency-groups]` 的 dev 中加入 `pyflowx[docs]`。
|
||||
|
||||
## 关键文件
|
||||
|
||||
| 文件 | 操作 |
|
||||
|------|------|
|
||||
| `src/pyflowx/runner.py` | 删除 `_apply_verbose_to_graph` |
|
||||
| `tests/test_runner.py` | 删除 `TestApplyVerboseToGraph` |
|
||||
| `src/pyflowx/__init__.py` | 版本统一(已 0.4.5,确认) |
|
||||
| `pyproject.toml` | 版本 → 0.4.5;加 docs 依赖 |
|
||||
| `README.md` | 更新 CLI 示例与模块结构表 |
|
||||
| `docs/conf.py` | 新建 |
|
||||
| `docs/*.rst` | 新建 |
|
||||
| `.readthedocs.yaml` | 新建 |
|
||||
| `.gitignore` | 加 docs/_build/ |
|
||||
|
||||
## 验证
|
||||
|
||||
1. **测试与 lint**:
|
||||
```bash
|
||||
uv run pytest tests/ -q
|
||||
uv run ruff check src/ tests/ docs/conf.py
|
||||
uv run pyrefly check src/pyflowx/runner.py
|
||||
```
|
||||
|
||||
2. **Sphinx 构建本地验证**:
|
||||
```bash
|
||||
uv sync --extra docs
|
||||
uv run sphinx-build -b html docs/ docs/_build/
|
||||
```
|
||||
确认无 warning,打开 `docs/_build/index.html` 检查页面。
|
||||
|
||||
3. **pf 功能回归**:
|
||||
```bash
|
||||
pf gitt c
|
||||
pf pymake b --dry-run
|
||||
```
|
||||
|
||||
4. **RTD 配置校验**:`.readthedocs.yaml` 语法正确,`docs/conf.py` 能独立构建。
|
||||
|
||||
## 不在范围
|
||||
|
||||
- 不统一各模块 docstring 风格(napoleon 兼容 Google/NumPy,够用)。
|
||||
- 不重构现有 CLI 工具 YAML。
|
||||
- 不新增中文文档翻译(文档用中文撰写,与项目既有风格一致)。
|
||||
@@ -0,0 +1,194 @@
|
||||
# PyFlowX 项目结构重构 — Phase 2 收尾计划
|
||||
|
||||
## Summary
|
||||
|
||||
承接上一会话已批准的 [项目结构重构计划](file:///home/zhou/pyflowx/.trae/documents/项目结构重构计划.md)。Phase 1(cli/legacy/ 归位)已完成并验证;Phase 2(ops/ 按功能分组)的文件移动、`_TOOL_MODULES` 注册表、`pf.py` 动态导入改造、`ops/__init__.py` docstring 更新均已落地,**仅剩 3 项收尾工作**:测试导入路径更新、注册表完整性测试、全量验证 + 提交。
|
||||
|
||||
本计划仅覆盖 Phase 2 剩余收尾,不引入新重构范围。
|
||||
|
||||
## Current State Analysis
|
||||
|
||||
### 已完成(无需再动)
|
||||
|
||||
| 项 | 状态 | 证据 |
|
||||
|----|------|------|
|
||||
| `cli/legacy/` 子包 | ✅ | [cli/legacy/__init__.py](file:///home/zhou/pyflowx/src/pyflowx/cli/legacy/__init__.py)、emlmanager.py、profiler.py 均在位 |
|
||||
| `pyproject.toml` 入口点 | ✅ | L26-28 已指向 `pyflowx.cli.legacy.*` |
|
||||
| `pf.py` `_LEGACY_TOOLS` | ✅ | L99-103 已更新 |
|
||||
| ops/ 4 个子目录 + `__init__.py` | ✅ | files/ dev/ system/ infra/ 均存在 |
|
||||
| 22 个工具文件已 `git mv` 到子目录 | ✅ | Glob 确认所有 .py 均在子目录下 |
|
||||
| `_common.py` 保留 ops 根 | ✅ | 8 处 `from pyflowx.ops._common import` 绝对路径仍有效 |
|
||||
| `pf.py` `_TOOL_MODULES` 注册表(22 项)| ✅ | L106-129 |
|
||||
| `pf.py` `_run_tool` / `_tool_description` 动态导入 | ✅ | L203-204、L258-259 改用 `self._TOOL_MODULES[target]` |
|
||||
| `ops/__init__.py` docstring | ✅ | 已文档化 4 个子目录 |
|
||||
| ops 模块间无交叉引用需改 | ✅ | Grep 确认仅 `_common` 被引用,路径不变 |
|
||||
|
||||
### 未完成(本计划处理)
|
||||
|
||||
1. **20 个测试文件的导入路径未更新**(26 行,含 2 行 `import pyflowx.ops.X` 形式)
|
||||
2. **`_TOOL_MODULES` 完整性测试未新增**
|
||||
3. **未跑验证 + 未提交**
|
||||
|
||||
## Proposed Changes
|
||||
|
||||
### 改动 1:更新 20 个测试文件的导入路径
|
||||
|
||||
所有改动均为机械替换,按子目录分组如下。每处仅替换 `pyflowx.ops` → `pyflowx.ops.<group>`,不动其他内容。
|
||||
|
||||
**files/ 组(6 文件,6 行):**
|
||||
|
||||
| 文件 | 行号 | 当前 | 修改后 |
|
||||
|------|------|------|--------|
|
||||
| [tests/cli/test_filedate.py](file:///home/zhou/pyflowx/tests/cli/test_filedate.py#L7) | 7 | `from pyflowx.ops import filedate as files` | `from pyflowx.ops.files import filedate as files` |
|
||||
| [tests/cli/test_filelevel.py](file:///home/zhou/pyflowx/tests/cli/test_filelevel.py#L7) | 7 | `from pyflowx.ops import filelevel as files` | `from pyflowx.ops.files import filelevel as files` |
|
||||
| [tests/cli/test_folderback.py](file:///home/zhou/pyflowx/tests/cli/test_folderback.py#L8) | 8 | `from pyflowx.ops import folderback as files` | `from pyflowx.ops.files import folderback as files` |
|
||||
| [tests/cli/test_folderzip.py](file:///home/zhou/pyflowx/tests/cli/test_folderzip.py#L8) | 8 | `from pyflowx.ops import folderzip as files` | `from pyflowx.ops.files import folderzip as files` |
|
||||
| [tests/cli/test_pdftool.py](file:///home/zhou/pyflowx/tests/cli/test_pdftool.py#L11) | 11 | `from pyflowx.ops import pdftool as media` | `from pyflowx.ops.files import pdftool as media` |
|
||||
| [tests/cli/test_screenshot.py](file:///home/zhou/pyflowx/tests/cli/test_screenshot.py#L9) | 9 | `from pyflowx.ops import screenshot as media` | `from pyflowx.ops.files import screenshot as media` |
|
||||
|
||||
**dev/ 组(8 文件,9 行):**
|
||||
|
||||
| 文件 | 行号 | 当前 | 修改后 |
|
||||
|------|------|------|--------|
|
||||
| [tests/cli/test_autofmt.py](file:///home/zhou/pyflowx/tests/cli/test_autofmt.py#L8) | 8 | `from pyflowx.ops import autofmt as dev` | `from pyflowx.ops.dev import autofmt as dev` |
|
||||
| [tests/cli/test_bumpversion.py](file:///home/zhou/pyflowx/tests/cli/test_bumpversion.py#L10) | 10 | `from pyflowx.ops import bumpversion` | `from pyflowx.ops.dev import bumpversion` |
|
||||
| [tests/cli/test_gittool.py](file:///home/zhou/pyflowx/tests/cli/test_gittool.py#L11) | 11 | `from pyflowx.ops import gittool as dev` | `from pyflowx.ops.dev import gittool as dev` |
|
||||
| [tests/cli/test_gittool_tool.py](file:///home/zhou/pyflowx/tests/cli/test_gittool_tool.py#L12-L13) | 12, 13 | `import pyflowx.ops.gittool` + `from pyflowx.ops import gittool` | `import pyflowx.ops.dev.gittool` + `from pyflowx.ops.dev import gittool` |
|
||||
| [tests/cli/test_lscalc.py](file:///home/zhou/pyflowx/tests/cli/test_lscalc.py#L9) | 9 | `from pyflowx.ops import lscalc as system` | `from pyflowx.ops.dev import lscalc as system` |
|
||||
| [tests/cli/test_packtool.py](file:///home/zhou/pyflowx/tests/cli/test_packtool.py#L10) | 10 | `from pyflowx.ops import packtool as system` | `from pyflowx.ops.dev import packtool as system` |
|
||||
| [tests/cli/test_piptool.py](file:///home/zhou/pyflowx/tests/cli/test_piptool.py#L9) | 9 | `from pyflowx.ops import piptool as dev` | `from pyflowx.ops.dev import piptool as dev` |
|
||||
| [tests/cli/test_pymake_tool.py](file:///home/zhou/pyflowx/tests/cli/test_pymake_tool.py#L6) | 6 | `import pyflowx.ops.pymake` | `import pyflowx.ops.dev.pymake` |
|
||||
|
||||
**system/ 组(2 文件,4 行):**
|
||||
|
||||
| 文件 | 行号 | 当前 | 修改后 |
|
||||
|------|------|------|--------|
|
||||
| [tests/cli/test_reseticoncache.py](file:///home/zhou/pyflowx/tests/cli/test_reseticoncache.py#L10) | 10 | `from pyflowx.ops.reseticoncache import reset_icon_cache_run` | `from pyflowx.ops.system.reseticoncache import reset_icon_cache_run` |
|
||||
| [tests/cli/test_system_run.py](file:///home/zhou/pyflowx/tests/cli/test_system_run.py#L11-L13) | 11, 12, 13 | `from pyflowx.ops.clr import` / `from pyflowx.ops.taskkill import` / `from pyflowx.ops.which import` | `from pyflowx.ops.system.clr import` / `from pyflowx.ops.system.taskkill import` / `from pyflowx.ops.system.which import` |
|
||||
|
||||
**infra/ 组(4 文件,7 行):**
|
||||
|
||||
| 文件 | 行号 | 当前 | 修改后 |
|
||||
|------|------|------|--------|
|
||||
| [tests/cli/test_envdev.py](file:///home/zhou/pyflowx/tests/cli/test_envdev.py#L12-L13) | 12, 13 | `from pyflowx.ops.dockercmd import` / `from pyflowx.ops.envdev import (` | `from pyflowx.ops.infra.dockercmd import` / `from pyflowx.ops.infra.envdev import (` |
|
||||
| [tests/cli/test_envdev_tool.py](file:///home/zhou/pyflowx/tests/cli/test_envdev_tool.py#L6) | 6 | `from pyflowx.ops import envdev, msdownload, sglang` | `from pyflowx.ops.infra import envdev, msdownload, sglang` |
|
||||
| [tests/cli/test_llm.py](file:///home/zhou/pyflowx/tests/cli/test_llm.py#L12-L13) | 12, 13 | `from pyflowx.ops.msdownload import` / `from pyflowx.ops.sglang import` | `from pyflowx.ops.infra.msdownload import` / `from pyflowx.ops.infra.sglang import` |
|
||||
| [tests/cli/test_sshcopyid.py](file:///home/zhou/pyflowx/tests/cli/test_sshcopyid.py#L11) | 11 | `from pyflowx.ops import sshcopyid as system` | `from pyflowx.ops.infra import sshcopyid as system` |
|
||||
|
||||
**不动:** [tests/cli/test_ops_common.py](file:///home/zhou/pyflowx/tests/cli/test_ops_common.py#L8) — 引用 `_common`,位置不变。
|
||||
|
||||
### 改动 2:新增 `_TOOL_MODULES` 完整性测试
|
||||
|
||||
在 [tests/cli/](file:///home/zhou/pyflowx/tests/cli/) 下新建 `test_tool_modules.py`,断言:
|
||||
- `_TOOL_ALIASES` 中每个规范名(值集合)都在 `_TOOL_MODULES` 中有对应模块路径
|
||||
- 每个注册的模块路径可被 `importlib.import_module` 成功导入
|
||||
|
||||
目的:防止未来新增工具时遗漏注册表项,导致 `pf <新工具>` 静默失败。
|
||||
|
||||
测试骨架:
|
||||
|
||||
```python
|
||||
"""Tests for pf._TOOL_MODULES 注册表完整性."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import importlib
|
||||
|
||||
import pytest
|
||||
|
||||
from pyflowx.cli.pf import PfApp
|
||||
|
||||
|
||||
class TestToolModulesComplete:
|
||||
"""_TOOL_MODULES 必须覆盖 _TOOL_ALIASES 的所有规范名."""
|
||||
|
||||
def test_all_canonical_tools_have_module_path(self) -> None:
|
||||
canonical_names = set(PfApp._TOOL_ALIASES.values())
|
||||
module_names = set(PfApp._TOOL_MODULES)
|
||||
missing = canonical_names - module_names
|
||||
assert not missing, f"未在 _TOOL_MODULES 注册的工具: {missing}"
|
||||
|
||||
@pytest.mark.parametrize("tool_name, module_path", sorted(PfApp._TOOL_MODULES.items()))
|
||||
def test_module_importable(self, tool_name: str, module_path: str) -> None:
|
||||
"""每个注册的模块路径必须可导入."""
|
||||
importlib.import_module(module_path)
|
||||
```
|
||||
|
||||
### 改动 3:验证 + 提交
|
||||
|
||||
```bash
|
||||
# 1. lint + 类型检查
|
||||
uv run ruff check src/ tests/
|
||||
uv run ruff format --check src/ tests/
|
||||
uv run pyrefly check src/
|
||||
|
||||
# 2. 全量测试(项目门槛 95% branch coverage)
|
||||
uv run pytest -m "not slow" -q
|
||||
|
||||
# 3. CLI 烟囱测试
|
||||
uv run pf # 工具列表
|
||||
uv run pf pdftool # 多命令工具 (files 组)
|
||||
uv run pf wch echo # 单命令工具 (system 组)
|
||||
uv run pf gitt c # gittool (dev 组)
|
||||
uv run pf emlman --help # legacy 路由
|
||||
uv run pxp --help # legacy 入口点
|
||||
|
||||
# 4. 提交(按文件名 add,遵循 git-commit-message.md 中文风格)
|
||||
git add src/pyflowx/cli/pf.py \
|
||||
src/pyflowx/cli/legacy/__init__.py \
|
||||
src/pyflowx/cli/legacy/emlmanager.py \
|
||||
src/pyflowx/cli/legacy/profiler.py \
|
||||
src/pyflowx/ops/__init__.py \
|
||||
src/pyflowx/ops/files/ \
|
||||
src/pyflowx/ops/dev/ \
|
||||
src/pyflowx/ops/system/ \
|
||||
src/pyflowx/ops/infra/ \
|
||||
tests/cli/test_tool_modules.py \
|
||||
tests/cli/test_emlmanager.py \
|
||||
tests/cli/test_profiler.py \
|
||||
tests/cli/test_autofmt.py \
|
||||
tests/cli/test_bumpversion.py \
|
||||
tests/cli/test_envdev.py \
|
||||
tests/cli/test_envdev_tool.py \
|
||||
tests/cli/test_filedate.py \
|
||||
tests/cli/test_filelevel.py \
|
||||
tests/cli/test_folderback.py \
|
||||
tests/cli/test_folderzip.py \
|
||||
tests/cli/test_gittool.py \
|
||||
tests/cli/test_gittool_tool.py \
|
||||
tests/cli/test_llm.py \
|
||||
tests/cli/test_lscalc.py \
|
||||
tests/cli/test_packtool.py \
|
||||
tests/cli/test_pdftool.py \
|
||||
tests/cli/test_piptool.py \
|
||||
tests/cli/test_pymake_tool.py \
|
||||
tests/cli/test_reseticoncache.py \
|
||||
tests/cli/test_screenshot.py \
|
||||
tests/cli/test_sshcopyid.py \
|
||||
tests/cli/test_system_run.py \
|
||||
pyproject.toml
|
||||
git commit -m "refactor: 按功能划分重构项目结构,ops 工具归入 files/dev/system/infra 子包,legacy 工具归入 cli/legacy"
|
||||
git push # 仅当分支已跟踪远程
|
||||
```
|
||||
|
||||
## Assumptions & Decisions
|
||||
|
||||
1. **不动 ops 模块内部代码**:已核查 22 个工具模块均用 `import pyflowx as px`(绝对导入)和 `from pyflowx.ops._common import`(_common 位置不变),无相对导入需调整。
|
||||
2. **`_TOOL_MODULES` 用类属性而非模块级常量**:与既有 `_TOOL_ALIASES`/`_LEGACY_TOOLS` 保持一致风格。
|
||||
3. **完整性测试放 `tests/cli/`**:与既有 `test_pf.py` 等同目录,聚焦 CLI 路由层。
|
||||
4. **不修改覆盖率配置**:`omit = ["src/pyflowx/cli/*"]` 递归匹配 `cli/legacy/*`,`src/pyflowx/ops/*` 仍在覆盖范围内(ops 模块本就有测试)。
|
||||
5. **不重命名 ops/ 为 tools/**:与顶层 `tools.py` 冲突,收益不匹配风险(沿用原计划决策)。
|
||||
6. **git add 按文件名**:遵循项目硬约束,不用 `git add .`/`-A`。移动文件用 `git mv` 已在前序步骤完成,本次 add 仅暂存修改和新增文件。
|
||||
|
||||
## Verification
|
||||
|
||||
完成标志(逐条核验):
|
||||
- [ ] `ruff check` 无错误
|
||||
- [ ] `ruff format --check` 无需改动
|
||||
- [ ] `pyrefly check` 无错误
|
||||
- [ ] `pytest -m "not slow"` 全绿,覆盖率 ≥ 95%
|
||||
- [ ] `pf` 列出全部工具(22 个 @px.tool + 3 个 legacy)
|
||||
- [ ] `pf pdftool` / `pf wch echo` / `pf gitt c` 三组工具均可调用
|
||||
- [ ] `pf emlman --help` / `pxp --help` legacy 路由正常
|
||||
- [ ] `test_tool_modules.py` 全绿
|
||||
- [ ] commit 已创建,远程已推送(若分支跟踪远程)
|
||||
@@ -0,0 +1,129 @@
|
||||
# PyFlowX 项目结构重构计划
|
||||
|
||||
## Context
|
||||
|
||||
当前项目结构存在三个问题:
|
||||
1. **cli/ 职责混杂**:`pf.py`(入口)、`emlmanager.py`(1337 行 legacy web 应用)、`profiler.py`(268 行 legacy 性能分析器)混在同一层级,入口与遗留工具不分
|
||||
2. **ops/ 下 22 个文件平铺**:从 filedate 到 sglang 全部堆在一级目录,无功能分组,难以浏览和维护
|
||||
3. **ops/ 命名歧义**:与顶层 `tools.py`(@px.tool 装饰器框架)容易混淆
|
||||
|
||||
本次重构通过两个阶段解决:Phase 1 将 legacy 工具归入 `cli/legacy/`,Phase 2 将 ops/ 下工具按功能分入 4 个子目录。**不移动** runner.py 和 tools.py(保留顶层),因为它们是公共 API 库代码且移动涉及覆盖率配置变更和 profiler monkey-patch 链路调整,风险不匹配收益。
|
||||
|
||||
## 目标结构
|
||||
|
||||
```
|
||||
src/pyflowx/
|
||||
├── __init__.py # 公共 API 导出(不变)
|
||||
├── task.py / graph.py / executors.py / ... # 核心库(不变)
|
||||
├── runner.py / tools.py # CLI 框架库代码(保留顶层)
|
||||
├── cli/
|
||||
│ ├── pf.py # pf 统一入口
|
||||
│ └── legacy/ # ← Phase 1: legacy 工具归位
|
||||
│ ├── __init__.py
|
||||
│ ├── emlmanager.py
|
||||
│ └── profiler.py
|
||||
└── ops/
|
||||
├── __init__.py # 更新 docstring
|
||||
├── _common.py # 保留根(所有子目录共用)
|
||||
├── files/ # ← Phase 2: 文件与文档操作 (6)
|
||||
│ ├── filedate.py, filelevel.py, folderback.py, folderzip.py
|
||||
│ ├── pdftool.py, screenshot.py
|
||||
├── dev/ # 开发与构建工具 (7)
|
||||
│ ├── autofmt.py, bumpversion.py, gittool.py, packtool.py
|
||||
│ ├── piptool.py, pymake.py, lscalc.py
|
||||
├── system/ # 系统工具 (4)
|
||||
│ ├── clr.py, reseticoncache.py, taskkill.py, which.py
|
||||
└── infra/ # 基础设施与服务部署 (5)
|
||||
├── dockercmd.py, envdev.py, sshcopyid.py
|
||||
├── msdownload.py, sglang.py
|
||||
```
|
||||
|
||||
## Phase 1: cli/ 清理 — legacy 工具归位(低风险)
|
||||
|
||||
**文件移动:**
|
||||
- `cli/emlmanager.py` → `cli/legacy/emlmanager.py`
|
||||
- `cli/profiler.py` → `cli/legacy/profiler.py`
|
||||
- 新建 `cli/legacy/__init__.py`
|
||||
|
||||
**导入修改:**
|
||||
|
||||
| 文件 | 当前 | 修改后 |
|
||||
|------|------|--------|
|
||||
| `cli/legacy/profiler.py` L43-46 | `from .. import executors` 等 4 处(`..`=pyflowx) | `from ... import executors` 等(`...`=pyflowx,多一层 legacy/) |
|
||||
|
||||
**配置修改:**
|
||||
|
||||
| 文件 | 修改 |
|
||||
|------|------|
|
||||
| `pyproject.toml` L26 | `emlman = "pyflowx.cli.legacy.emlmanager:main"` |
|
||||
| `pyproject.toml` L28 | `pxp = "pyflowx.cli.legacy.profiler:main"` |
|
||||
| `cli/pf.py` L100-102 | `_LEGACY_TOOLS` 3 个值改为 `pyflowx.cli.legacy.xxx:main` |
|
||||
|
||||
**测试修改(2 文件):**
|
||||
- `tests/cli/test_emlmanager.py`:`from pyflowx.cli import emlmanager` → `from pyflowx.cli.legacy import emlmanager`
|
||||
- `tests/cli/test_profiler.py`:`from pyflowx.cli import profiler` → `from pyflowx.cli.legacy import profiler`
|
||||
|
||||
**覆盖率:** `omit = ["src/pyflowx/cli/*"]` 递归匹配 `cli/legacy/*`,无需改动。
|
||||
|
||||
## Phase 2: ops/ 按功能分组(中等风险)
|
||||
|
||||
**文件移动(22 个工具文件 + 4 个新建 `__init__.py`):**
|
||||
|
||||
| 子目录 | 移入文件 |
|
||||
|--------|---------|
|
||||
| `ops/files/` | filedate, filelevel, folderback, folderzip, pdftool, screenshot |
|
||||
| `ops/dev/` | autofmt, bumpversion, gittool, packtool, piptool, pymake, lscalc |
|
||||
| `ops/system/` | clr, reseticoncache, taskkill, which |
|
||||
| `ops/infra/` | dockercmd, envdev, sshcopyid, msdownload, sglang |
|
||||
|
||||
`_common.py` 保留 `ops/` 根——子目录工具通过绝对路径 `from pyflowx.ops._common import ...` 引用,路径不变。
|
||||
|
||||
**ops 模块自身导入:** 无需修改。所有 ops 模块用 `import pyflowx as px`(绝对导入)和 `from pyflowx.ops._common import ...`(_common 位置不变)。已核查 ops 模块间无交叉引用。
|
||||
|
||||
**pf.py 修改 — 新增 `_TOOL_MODULES` 注册表:**
|
||||
|
||||
```python
|
||||
_TOOL_MODULES: dict[str, str] = {
|
||||
"autofmt": "pyflowx.ops.dev.autofmt",
|
||||
"bumpversion": "pyflowx.ops.dev.bumpversion",
|
||||
# ... 22 个规范工具名 → 完整模块路径
|
||||
}
|
||||
```
|
||||
|
||||
修改 `_run_tool` 和 `_tool_description` 两处动态导入:
|
||||
- `importlib.import_module(f"pyflowx.ops.{target}")` → `importlib.import_module(self._TOOL_MODULES[target])`
|
||||
|
||||
**ops/__init__.py 修改:** 更新 docstring,补充 lscalc(当前遗漏),标注分组结构。`__all__` 保持空列表。
|
||||
|
||||
**测试修改(20 文件,机械替换导入路径):**
|
||||
|
||||
代表性修改:
|
||||
- `from pyflowx.ops import filedate` → `from pyflowx.ops.files import filedate`
|
||||
- `from pyflowx.ops.gittool import ...` → `from pyflowx.ops.dev.gittool import ...`
|
||||
- `from pyflowx.ops.clr import ...` → `from pyflowx.ops.system.clr import ...`
|
||||
- `from pyflowx.ops.dockercmd import ...` → `from pyflowx.ops.infra.dockercmd import ...`
|
||||
|
||||
`test_ops_common.py` 无需修改(`_common` 位置不变)。
|
||||
|
||||
**新增测试:** 遍历 `_TOOL_ALIASES` 所有规范名,断言每个都在 `_TOOL_MODULES` 中且模块可导入,防止注册表遗漏。
|
||||
|
||||
## 不做的改动
|
||||
|
||||
- **runner.py / tools.py 保留顶层**:它们是公共 API(`px.CliRunner`、`px.tool`),有完整测试。移入 cli/ 涉及覆盖率配置变更(`omit` 从 `cli/*` 改为精确排除)和 profiler monkey-patch 链路调整,风险不匹配收益。
|
||||
- **ops/ 不重命名**:重命名为 `tools/` 与顶层 `tools.py` 冲突;其他名字(`cli_tools/`、`commands/`)收益不明显。通过 docstring 明确"ops = CLI 工具实现集合"即可。
|
||||
- **核心库模块不动**:task.py、graph.py 等是公共 API 的一部分,用户通过 `px.TaskSpec`、`px.Graph` 访问,移动增加风险且价值不大。
|
||||
- **ops/__init__.py 不做 re-export**:保持惰性加载,避免 CLI 启动时加载全部 22 个工具模块。
|
||||
|
||||
## 验证
|
||||
|
||||
每个 Phase 完成后:
|
||||
```bash
|
||||
uv run ruff check src/ tests/ # lint
|
||||
uv run pyrefly check src/ # 类型检查
|
||||
uv run pytest -m "not slow" -q # 全套测试 (993 个)
|
||||
uv run pytest tests/cli/ -q # CLI 测试子集(快速反馈)
|
||||
uv run pf pdftool # 验证多命令工具
|
||||
uv run pf wch echo # 验证单命令工具
|
||||
uv run pf emlman --help # 验证 legacy 工具路由
|
||||
uv run pxp --help # 验证 legacy 工具入口点
|
||||
```
|
||||
@@ -0,0 +1,23 @@
|
||||
# 开发流程约束
|
||||
|
||||
本规则补充 `self-driven.md`,定义执行过程记录与规则变更的两条硬约束。
|
||||
|
||||
## 执行过程记录
|
||||
|
||||
- **每次迭代**(一个完整的"计划 → 实现 → 测试 → 文档 → 验证"闭环)必须记录到
|
||||
`.trae/docs/` 目录,文件名形如 `iter-NN-<主题>.md`,`NN` 为零填充序号。
|
||||
- 记录内容至少包括:本轮目标、改动文件清单、关键决策与依据、验证结果、遗留事项。
|
||||
- **每 5 次迭代后**清理 `.trae/docs/`:
|
||||
- 把需要长期保留的整合性内容(可复用模式、踩坑总结、设计决策)
|
||||
归档到 `.trae/skills/` 对应技能文档中;
|
||||
- 其余过程性记录删除;
|
||||
- docs 目录只保留最近未归档的迭代记录(即第 6 次迭代开始前,清理 1–5 号记录)。
|
||||
- 归档动作需在迭代记录中标注"已归档至 skills/<文件>"。
|
||||
|
||||
## 规则变更约束
|
||||
|
||||
- 任何对 `.trae/rules/` 下文件的修改(新增、编辑、删除)**必须先询问用户**,
|
||||
获得明确授权后方可变动;**未授权前不得擅自修改 rules**。
|
||||
- 用户在对话中显式要求写入的规则内容(如"把 X 写入 rules")视为已授权,
|
||||
可直接写入,写入后在回复中说明改了哪个文件、加了什么。
|
||||
- 规则变更后需同步更新 `project_memory.md` 中的"开发约定"小节。
|
||||
@@ -73,6 +73,22 @@ alwaysApply: true
|
||||
- 全套门禁通过:ruff、pyrefly、pytest、coverage。
|
||||
- 给出本轮变更清单(改了哪些文件、为什么)。
|
||||
|
||||
## 多阶段项目
|
||||
|
||||
当项目按阶段(P0/P1/P2/...)划分时,初始确认的目标是**整个项目**,而非单个阶段。
|
||||
|
||||
- **阶段是里程碑,不是边界**:确认"推进 P1"只是指定下一步重点,不缩小整体目标范围;
|
||||
也不意味着 P1 完成后就可以停下。整体目标 = 所有阶段交付完毕。
|
||||
- **阶段完成不是停止条件**:某阶段验收标准全部满足后,**自动进入下一阶段的「计划」步骤**,
|
||||
不停下来输出"是否继续"或"是否扩展范围"的询问。仅在阶段切换时用一句话说明
|
||||
"进入 Pn,重点:…",然后继续自驱动迭代。
|
||||
- **「收尾」仅适用于整个项目完成**:只有所有阶段(含最后一个)都交付后,才执行
|
||||
「收尾」输出最终总结。单阶段完成 → 进入下一阶段的「计划」,不触发收尾。
|
||||
- **跨阶段依赖**:若下一阶段依赖外部资源(如新依赖审批、环境配置),属"不可恢复的失败"
|
||||
暂停条件;否则一律连续推进。
|
||||
- **阶段范围超出预期**:执行中发现某阶段需要显著扩大范围或改变方向(如 P2 本来只做
|
||||
表格抽取,发现必须先重写 IR),属"超出初始确认范围"暂停条件,需找用户确认。
|
||||
|
||||
## 暂停条件(仅在以下情况中断自驱动找用户)
|
||||
|
||||
1. **歧义无法自决**:需求存在多种合理解读且无既有约定可循。
|
||||
@@ -81,7 +97,7 @@ alwaysApply: true
|
||||
4. **超出初始确认范围**:用户目标在执行中发现需要显著扩大范围或改变方向。
|
||||
5. **用户主动询问**:用户在对话中提出新问题或要求澄清。
|
||||
|
||||
**注意**:"目标已达成"**不是**暂停条件——验收标准全部满足后直接进入收尾并结束任务,不询问"是否扩展范围"或"是否提交"。
|
||||
**注意**:"目标已达成"**不是**暂停条件——但"目标已达成"指**整个项目目标**(所有阶段交付完毕),而非单个阶段验收满足。单阶段验收满足后应自动进入下一阶段(见「多阶段项目」),不触发收尾。
|
||||
|
||||
非以上情况,一律继续自驱动,不要为"求确认"而暂停。
|
||||
|
||||
@@ -128,7 +144,8 @@ alwaysApply: true
|
||||
|
||||
## 收尾
|
||||
|
||||
- 验收标准全部满足后,**直接输出最终总结并结束任务**:交付物、关键决策、遗留事项。
|
||||
- **仅当整个项目所有阶段都交付后**,才执行收尾:直接输出最终总结并结束任务(交付物、关键决策、遗留事项)。
|
||||
- 单阶段完成**不属于收尾**——见「多阶段项目」,应自动进入下一阶段的「计划」。
|
||||
- **自动提交**:收尾时自动 `git add`(按文件名)+ `git commit`(遵循 `.trae/rules/git-commit-message.md` 风格)+ `git push`(仅当分支已跟踪远程时执行;新分支跳过 push 并在总结中说明);**不询问**"是否需要提交"或"是否扩展范围"。
|
||||
- 若验收标准未全部满足,回到「计划」继续下一轮,不停下询问。
|
||||
- 将本次会话的关键产出与决策更新到 memory,便于后续会话续接。
|
||||
|
||||
@@ -419,25 +419,6 @@ jobs:
|
||||
| `tags` | `tags` | 自由标签 |
|
||||
| `runs-on` | `tags`(追加) | 运行环境标签 |
|
||||
|
||||
## 示例
|
||||
|
||||
仓库 `examples/` 目录包含完整示例:
|
||||
|
||||
- [`etl_pipeline.py`](examples/etl_pipeline.py) —— ETL 流水线(sequential)
|
||||
- [`parallel_run.py`](examples/parallel_run.py) —— 并行执行对比(thread vs sequential)
|
||||
- [`async_aggregation.py`](examples/async_aggregation.py) —— 异步聚合 + Context 注入
|
||||
- [`yaml_pipeline.yaml`](examples/yaml_pipeline.yaml) + [`yaml_pipeline.py`](examples/yaml_pipeline.py) —— YAML 声明式 CI/CD 流水线(矩阵扇出 + 条件执行)
|
||||
|
||||
运行:
|
||||
|
||||
```bash
|
||||
python examples/etl_pipeline.py
|
||||
python examples/parallel_run.py
|
||||
python examples/async_aggregation.py
|
||||
python -m pyflowx.examples.yaml_pipeline
|
||||
pf yamlrun src/pyflowx/examples/yaml_pipeline.yaml --dry-run
|
||||
```
|
||||
|
||||
## 断点续跑
|
||||
|
||||
```python
|
||||
@@ -554,8 +535,8 @@ uv run pytest --cov=pyflowx --cov-fail-under=95
|
||||
uv run mypy
|
||||
|
||||
# 代码风格
|
||||
uv run ruff check src tests examples
|
||||
uv run ruff format --check src tests examples
|
||||
uv run ruff check src tests
|
||||
uv run ruff format --check src tests
|
||||
```
|
||||
|
||||
## 模块结构
|
||||
@@ -574,25 +555,18 @@ uv run ruff format --check src tests examples
|
||||
| `storage.py` | 状态后端:`MemoryBackend` / `JSONBackend`(batch flush) |
|
||||
| `runner.py` | CLI 运行器:`CliRunner` |
|
||||
| `report.py` | 运行结果:`RunReport` / `TaskResult` |
|
||||
| `yaml_loader.py` | YAML 任务编排:GitHub Actions 风格 schema 解析(`load_yaml` / `parse_yaml_string` / `run_cli`) |
|
||||
| `registry.py` | 函数注册中心:`register_fn` / `get_fn` / `has_fn`(YAML 的 `fn:` 引用) |
|
||||
| `tools.py` | `@px.tool` 装饰器:`ToolSpec` / `run_tool` / `list_tools` / `list_subcommands` |
|
||||
| `profiling.py` | 性能分析:`Profiler` 任务耗时统计 |
|
||||
| `errors.py` | 错误家族:`PyFlowXError` 子类 |
|
||||
| `ops/` | 工具函数(dev/files/media/system),被 YAML 的 `fn:` 引用 |
|
||||
| `ops/` | CLI 工具模块集合(每个子模块用 `@px.tool` 注册一个工具) |
|
||||
|
||||
### CLI 工具
|
||||
|
||||
| 模块 | 职责 |
|
||||
|------|------|
|
||||
| `cli/pf.py` | 统一入口:`pf <tool> [command]`,自动发现 `configs/*.yaml` 并路由 |
|
||||
| `cli/configs/` | YAML 工具配置(gittool/filedate/pdftool 等 13 个工具的 `cli:` schema) |
|
||||
| `cli/pymake.py` | 构建工具(替代 Makefile),`pf pymake b` 调用 |
|
||||
| `cli/yamlrun.py` | YAML pipeline 执行器,`pf yamlrun pipeline.yaml` 调用 |
|
||||
| `cli/pf.py` | 统一入口:`pf <tool> [command]`,按需导入 `ops/<tool>.py` 触发 `@px.tool` 注册 |
|
||||
| `cli/profiler.py` | 性能分析 CLI |
|
||||
| `cli/emlmanager.py` | 邮件管理 CLI |
|
||||
| `cli/dev/` | 开发工具(dockercmd/envdev) |
|
||||
| `cli/llm/` | LLM 工具(msdownload/sglang) |
|
||||
| `cli/system/` | 系统工具(clearscreen/taskkill/which) |
|
||||
|
||||
## 许可证
|
||||
|
||||
|
||||
+9
-13
@@ -50,21 +50,17 @@ API 参考
|
||||
:members:
|
||||
:undoc-members:
|
||||
|
||||
YAML 编排
|
||||
---------
|
||||
@px.tool 工具
|
||||
-------------
|
||||
|
||||
.. autofunction:: pyflowx.load_yaml
|
||||
.. autofunction:: pyflowx.parse_yaml_string
|
||||
.. autofunction:: pyflowx.run_yaml
|
||||
.. autofunction:: pyflowx.run_cli
|
||||
.. autofunction:: pyflowx.build_cli_parser
|
||||
.. autoclass:: pyflowx.ToolSpec
|
||||
:members:
|
||||
:undoc-members:
|
||||
|
||||
函数注册
|
||||
--------
|
||||
|
||||
.. autofunction:: pyflowx.register_fn
|
||||
.. autofunction:: pyflowx.get_fn
|
||||
.. autofunction:: pyflowx.has_fn
|
||||
.. autofunction:: pyflowx.tool
|
||||
.. autofunction:: pyflowx.run_tool
|
||||
.. autofunction:: pyflowx.list_tools
|
||||
.. autofunction:: pyflowx.list_subcommands
|
||||
|
||||
命令执行
|
||||
--------
|
||||
|
||||
+12
-24
@@ -7,13 +7,11 @@ classifiers = [
|
||||
"Programming Language :: Python :: 3.12",
|
||||
"Programming Language :: Python :: 3.13",
|
||||
"Programming Language :: Python :: 3.14",
|
||||
"Programming Language :: Python :: 3.8",
|
||||
"Programming Language :: Python :: 3.9",
|
||||
"Topic :: Software Development :: Libraries :: Application Frameworks",
|
||||
]
|
||||
dependencies = [
|
||||
"graphlib_backport >= 1.0.0; python_version < '3.9'",
|
||||
"pyyaml>=6.0.1",
|
||||
"rich>=13.7.0",
|
||||
"typer>=0.24.0",
|
||||
"typing-extensions>=4.13.2; python_version < '3.13'",
|
||||
]
|
||||
description = "Lightweight, type-safe DAG task scheduler with multi-strategy execution."
|
||||
@@ -21,23 +19,13 @@ keywords = ["async", "dag", "scheduler", "task", "workflow"]
|
||||
license = { text = "MIT" }
|
||||
name = "pyflowx"
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.8"
|
||||
version = "0.4.7"
|
||||
requires-python = ">=3.10"
|
||||
version = "0.4.8"
|
||||
|
||||
[project.scripts]
|
||||
dockercmd = "pyflowx.cli.dev.dockercmd:main"
|
||||
emlman = "pyflowx.cli.emlmanager:main"
|
||||
msdown = "pyflowx.cli.llm.msdownload:main"
|
||||
pf = "pyflowx.cli.pf:main"
|
||||
pxp = "pyflowx.cli.profiler:main"
|
||||
sglang = "pyflowx.cli.llm.sglang:main"
|
||||
yamlrun = "pyflowx.cli.yamlrun:main"
|
||||
# dev
|
||||
envdev = "pyflowx.cli.dev.envdev:main"
|
||||
# system
|
||||
clr = "pyflowx.cli.system.clearscreen:main"
|
||||
taskk = "pyflowx.cli.system.taskkill:main"
|
||||
wch = "pyflowx.cli.system.which:main"
|
||||
emlman = "pyflowx.cli.legacy.emlmanager:main"
|
||||
pf = "pyflowx.cli.pf:main"
|
||||
pxp = "pyflowx.cli.legacy.profiler:main"
|
||||
|
||||
[project.optional-dependencies]
|
||||
dev = [
|
||||
@@ -54,7 +42,6 @@ dev = [
|
||||
"ruff>=0.8.0",
|
||||
"tox-uv>=1.13.1",
|
||||
"tox>=4.25.0",
|
||||
"types-PyYAML>=6.0.12",
|
||||
]
|
||||
docs = ["myst-parser>=3.0", "sphinx-rtd-theme>=2.0", "sphinx>=7.0"]
|
||||
office = [
|
||||
@@ -90,7 +77,7 @@ dev = ["pyflowx[dev,docs,office]"]
|
||||
[tool.coverage.run]
|
||||
branch = true
|
||||
concurrency = ["thread"]
|
||||
omit = ["src/pyflowx/cli/*", "src/pyflowx/examples/*", "tests/*"]
|
||||
omit = ["src/pyflowx/cli/*", "tests/*"]
|
||||
source = ["pyflowx"]
|
||||
|
||||
[tool.coverage.report]
|
||||
@@ -110,7 +97,7 @@ markers = ["slow: marks tests as slow (deselect with
|
||||
# Ruff 配置 - 与 .pre-commit-config.yaml 保持一致
|
||||
[tool.ruff]
|
||||
line-length = 120
|
||||
target-version = "py38"
|
||||
target-version = "py310"
|
||||
|
||||
[tool.ruff.lint]
|
||||
ignore = [
|
||||
@@ -143,9 +130,10 @@ select = [
|
||||
]
|
||||
|
||||
[tool.ruff.lint.per-file-ignores]
|
||||
"**/tests/**" = ["ARG001", "ARG002"]
|
||||
"**/tests/**" = ["ARG001", "ARG002"]
|
||||
"src/pyflowx/tools.py" = ["PLR0911"]
|
||||
|
||||
[tool.pyrefly]
|
||||
preset = "strict"
|
||||
project-includes = ["**/*.ipynb", "**/*.py*"]
|
||||
python-version = "3.8"
|
||||
python-version = "3.10"
|
||||
|
||||
+8
-22
@@ -60,15 +60,7 @@ from __future__ import annotations
|
||||
|
||||
from .command import run_command
|
||||
from .compose import GraphComposer, compose
|
||||
from .conditions import (
|
||||
IS_LINUX,
|
||||
IS_MACOS,
|
||||
IS_POSIX,
|
||||
IS_WINDOWS,
|
||||
BuiltinConditions,
|
||||
Condition,
|
||||
Constants,
|
||||
)
|
||||
from .conditions import IS_LINUX, IS_MACOS, IS_POSIX, IS_WINDOWS, BuiltinConditions, Condition, Constants
|
||||
from .context import Context, build_call_args, describe_injection
|
||||
from .errors import (
|
||||
CycleError,
|
||||
@@ -83,7 +75,6 @@ from .errors import (
|
||||
from .executors import Strategy, run
|
||||
from .graph import Graph, GraphDefaults
|
||||
from .profiling import ProfileReport, TaskProfile
|
||||
from .registry import FnRegistry, get_fn, has_fn, register_fn
|
||||
from .report import RunReport
|
||||
from .runner import CliExitCode, CliRunner
|
||||
from .storage import JSONBackend, MemoryBackend, StateBackend
|
||||
@@ -100,9 +91,9 @@ from .task import (
|
||||
task,
|
||||
task_template,
|
||||
)
|
||||
from .yaml_loader import YamlLoadError, build_cli_parser, load_yaml, parse_yaml_string, run_cli, run_yaml
|
||||
from .tools import ToolSpec, list_subcommands, list_tools, run_tool, tool
|
||||
|
||||
__version__ = "0.4.7"
|
||||
__version__ = "0.4.8"
|
||||
|
||||
__all__ = [
|
||||
"IS_LINUX",
|
||||
@@ -118,7 +109,6 @@ __all__ = [
|
||||
"Context",
|
||||
"CycleError",
|
||||
"DuplicateTaskError",
|
||||
"FnRegistry",
|
||||
"Graph",
|
||||
"GraphComposer",
|
||||
"GraphDefaults",
|
||||
@@ -142,21 +132,17 @@ __all__ = [
|
||||
"TaskSpec",
|
||||
"TaskStatus",
|
||||
"TaskTimeoutError",
|
||||
"YamlLoadError",
|
||||
"ToolSpec",
|
||||
"build_call_args",
|
||||
"build_cli_parser",
|
||||
"cmd",
|
||||
"compose",
|
||||
"describe_injection",
|
||||
"get_fn",
|
||||
"has_fn",
|
||||
"load_yaml",
|
||||
"parse_yaml_string",
|
||||
"register_fn",
|
||||
"list_subcommands",
|
||||
"list_tools",
|
||||
"run",
|
||||
"run_cli",
|
||||
"run_command",
|
||||
"run_yaml",
|
||||
"run_tool",
|
||||
"task",
|
||||
"task_template",
|
||||
"tool",
|
||||
]
|
||||
|
||||
@@ -1,26 +0,0 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Literal
|
||||
|
||||
import pyflowx as px
|
||||
|
||||
DockerMirrorType = Literal["tencent"]
|
||||
|
||||
DOCKER_MIRROR_URLS: dict[DockerMirrorType, str] = {"tencent": "ccr.ccs.tencentyun.com"}
|
||||
|
||||
|
||||
def main():
|
||||
# parser = argparse.ArgumentParser(description="Docker 命令行工具")
|
||||
# parser.add_argument("--username", nargs="?", default="", type=str, help="Docker 用户名")
|
||||
# args = parser.parse_args()
|
||||
|
||||
tasks: list[px.TaskSpec] = [
|
||||
px.cmd(["docker", "login", "--username", "xxx", DOCKER_MIRROR_URLS["tencent"]], name="docker_login_tencent"),
|
||||
]
|
||||
|
||||
alias: dict[str, str | list[str | px.TaskSpec] | px.TaskSpec | px.Graph] = {
|
||||
"login": "docker_login_tencent",
|
||||
}
|
||||
|
||||
runner = px.CliRunner(strategy="sequential", tasks=tasks, aliases=alias)
|
||||
runner.run_cli()
|
||||
@@ -1,366 +0,0 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import getpass
|
||||
from pathlib import Path
|
||||
from typing import Literal, get_args
|
||||
|
||||
import pyflowx as px
|
||||
from pyflowx.conditions import BuiltinConditions
|
||||
from pyflowx.tasks.system import setenv_group, write_file
|
||||
|
||||
# ============================================================================
|
||||
# Mirror 配置
|
||||
# ============================================================================
|
||||
DOWNLOAD_MIRROR_SCRIPT: str = "curl -sSL https://linuxmirrors.cn/main.sh -o /tmp/linuxmirrors.sh"
|
||||
INSTALL_MIRROR_SCRIPT: str = "sudo bash /tmp/linuxmirrors.sh"
|
||||
|
||||
# ============================================================================
|
||||
# Python 配置
|
||||
# ============================================================================
|
||||
PyMirrorType = Literal["tsinghua", "aliyun", "huaweicloud", "ustc", "zju"]
|
||||
|
||||
PIP_INDEX_URLS: dict[PyMirrorType, str] = {
|
||||
"tsinghua": "https://pypi.tuna.tsinghua.edu.cn/simple",
|
||||
"aliyun": "https://mirrors.aliyun.com/pypi/simple/",
|
||||
"huaweicloud": "https://mirrors.huaweicloud.com/repository/pypi/simple/",
|
||||
"ustc": "https://pypi.mirrors.ustc.edu.cn/simple/",
|
||||
"zju": "https://mirrors.zju.edu.cn/pypi/simple/",
|
||||
}
|
||||
|
||||
PIP_TRUSTED_HOSTS: dict[PyMirrorType, str] = {
|
||||
"tsinghua": "pypi.tuna.tsinghua.edu.cn",
|
||||
"aliyun": "mirrors.aliyun.com",
|
||||
"huaweicloud": "mirrors.huaweicloud.com",
|
||||
"ustc": "pypi.mirrors.ustc.edu.cn",
|
||||
"zju": "mirrors.zju.edu.cn",
|
||||
}
|
||||
PIP_CONFIG_PATH = Path.home() / ".pip" / "pip.conf" if BuiltinConditions.IS_LINUX() else Path.home() / "pip" / "pip.ini"
|
||||
|
||||
UV_INDEX_URLS = PIP_INDEX_URLS
|
||||
UV_PYTHON_INSTALL_MIRROR: str = "https://registry.npmmirror.com/-/binary/python-build-standalone"
|
||||
|
||||
# ============================================================================
|
||||
# Conda 配置
|
||||
# ============================================================================
|
||||
CondaMirrorType = Literal["tsinghua", "ustc", "bsfu", "aliyun"]
|
||||
|
||||
CONDA_MIRROR_URLS: dict[CondaMirrorType, list[str]] = {
|
||||
"tsinghua": [
|
||||
"https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/",
|
||||
"https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free/",
|
||||
"https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/r/",
|
||||
"https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/msys2/",
|
||||
"https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/pro/",
|
||||
"https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/",
|
||||
"https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/bioconda/",
|
||||
"https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/menpo/",
|
||||
"https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/pytorch/",
|
||||
],
|
||||
"ustc": [
|
||||
"https://mirrors.ustc.edu.cn/anaconda/pkgs/main/",
|
||||
"https://mirrors.ustc.edu.cn/anaconda/pkgs/free/",
|
||||
"https://mirrors.ustc.edu.cn/anaconda/pkgs/r/",
|
||||
"https://mirrors.ustc.edu.cn/anaconda/pkgs/msys2/",
|
||||
"https://mirrors.ustc.edu.cn/anaconda/pkgs/pro/",
|
||||
"https://mirrors.ustc.edu.cn/anaconda/pkgs/dev/",
|
||||
"https://mirrors.ustc.edu.cn/anaconda/cloud/conda-forge/",
|
||||
"https://mirrors.ustc.edu.cn/anaconda/cloud/bioconda/",
|
||||
"https://mirrors.ustc.edu.cn/anaconda/cloud/menpo/",
|
||||
"https://mirrors.ustc.edu.cn/anaconda/cloud/pytorch/",
|
||||
],
|
||||
"bsfu": [
|
||||
"https://mirrors.bsfu.edu.cn/anaconda/pkgs/main/",
|
||||
"https://mirrors.bsfu.edu.cn/anaconda/pkgs/free/",
|
||||
"https://mirrors.bsfu.edu.cn/anaconda/pkgs/r/",
|
||||
"https://mirrors.bsfu.edu.cn/anaconda/pkgs/msys2/",
|
||||
"https://mirrors.bsfu.edu.cn/anaconda/pkgs/pro/",
|
||||
"https://mirrors.bsfu.edu.cn/anaconda/pkgs/dev/",
|
||||
"https://mirrors.bsfu.edu.cn/anaconda/cloud/conda-forge/",
|
||||
"https://mirrors.bsfu.edu.cn/anaconda/cloud/bioconda/",
|
||||
"https://mirrors.bsfu.edu.cn/anaconda/cloud/menpo/",
|
||||
"https://mirrors.bsfu.edu.cn/anaconda/cloud/pytorch/",
|
||||
],
|
||||
"aliyun": [
|
||||
"https://mirrors.aliyun.com/anaconda/pkgs/main/",
|
||||
"https://mirrors.aliyun.com/anaconda/pkgs/free/",
|
||||
"https://mirrors.aliyun.com/anaconda/pkgs/r/",
|
||||
"https://mirrors.aliyun.com/anaconda/pkgs/msys2/",
|
||||
"https://mirrors.aliyun.com/anaconda/pkgs/pro/",
|
||||
"https://mirrors.aliyun.com/anaconda/pkgs/dev/",
|
||||
"https://mirrors.aliyun.com/anaconda/cloud/conda-forge/",
|
||||
"https://mirrors.aliyun.com/anaconda/cloud/bioconda/",
|
||||
"https://mirrors.aliyun.com/anaconda/cloud/menpo/",
|
||||
"https://mirrors.aliyun.com/anaconda/cloud/pytorch/",
|
||||
],
|
||||
}
|
||||
CONDA_CONFIG_PATH = Path.home() / ".condarc"
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# Qt 配置
|
||||
# ============================================================================
|
||||
|
||||
QT_LIBS: list[str] = [
|
||||
"build-essential",
|
||||
"libgl1",
|
||||
"libegl1",
|
||||
"libglib2.0-0",
|
||||
"libfontconfig1",
|
||||
"libfreetype6",
|
||||
"libxkbcommon0",
|
||||
"libdbus-1-3",
|
||||
"libxcb-xinerama0",
|
||||
"libxcb-icccm4",
|
||||
"libxcb-image0",
|
||||
"libxcb-keysyms1",
|
||||
"libxcb-randr0",
|
||||
"libxcb-render-util0",
|
||||
"libxcb-shape0",
|
||||
"libxcb-xfixes0",
|
||||
"libxcb-cursor0",
|
||||
]
|
||||
|
||||
CHINESE_FONTS: list[str] = [
|
||||
"fonts-noto-cjk",
|
||||
"fonts-wqy-microhei",
|
||||
"fonts-wqy-zenhei",
|
||||
"fonts-noto-color-emoji",
|
||||
]
|
||||
|
||||
# ============================================================================
|
||||
# Rust 配置
|
||||
# ============================================================================
|
||||
RustMirrorType = Literal["tsinghua", "ustc", "aliyun"]
|
||||
RustVersionType = Literal["stable", "nightly", "beta"]
|
||||
DEFAULT_RUST_VERSION: RustVersionType = "stable"
|
||||
DEFAULT_MIRROR: RustMirrorType = "tsinghua"
|
||||
|
||||
RUSTUP_MIRRORS: dict[RustMirrorType, dict[str, str]] = {
|
||||
"tsinghua": {
|
||||
"RUSTUP_DIST_SERVER": "https://mirrors.tuna.tsinghua.edu.cn/rustup",
|
||||
"RUSTUP_UPDATE_ROOT": "https://mirrors.tuna.tsinghua.edu.cn/rustup/rustup",
|
||||
"TOML_REGISTRY": "https://mirrors.tuna.tsinghua.edu.cn/crates.io-index/",
|
||||
},
|
||||
"aliyun": {
|
||||
"RUSTUP_DIST_SERVER": "https://mirrors.aliyun.com/rustup",
|
||||
"RUSTUP_UPDATE_ROOT": "https://mirrors.aliyun.com/rustup/rustup",
|
||||
"TOML_REGISTRY": "https://mirrors.aliyun.com/crates.io-index/",
|
||||
},
|
||||
"ustc": {
|
||||
"RUSTUP_DIST_SERVER": "https://mirrors.ustc.edu.cn/rust-static",
|
||||
"RUSTUP_UPDATE_ROOT": "https://mirrors.ustc.edu.cn/rust-static/rustup",
|
||||
"TOML_REGISTRY": "https://mirrors.ustc.edu.cn/crates.io-index/",
|
||||
},
|
||||
}
|
||||
RUSTUP_DOWNLOAD_URL_LINUX = "https://mirrors.aliyun.com/repo/rust/rustup-init.sh"
|
||||
RUSTUP_DOWNLOAD_URL_WINDOWS = "https://static.rust-lang.org/rustup/dist/x86_64-pc-windows-msvc/rustup-init.exe"
|
||||
RUST_CONFIG_PATH = Path.home() / ".cargo" / "config.toml"
|
||||
RUST_SCCACHE_DIR: Path = Path.home() / ".cargo" / "sccache"
|
||||
RUST_SCCACHE_CACHE_SIZE: str = "20G"
|
||||
|
||||
|
||||
def main() -> None:
|
||||
"""主函数."""
|
||||
parser = argparse.ArgumentParser(description="环境开发工具")
|
||||
parser.add_argument(
|
||||
"--python-mirror",
|
||||
nargs="?",
|
||||
type=str,
|
||||
default="tsinghua",
|
||||
choices=get_args(PyMirrorType),
|
||||
help="Python 镜像源",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--conda-mirror",
|
||||
nargs="?",
|
||||
type=str,
|
||||
default="tsinghua",
|
||||
choices=get_args(CondaMirrorType),
|
||||
help="Conda 镜镜像源",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--rust-mirror",
|
||||
nargs="?",
|
||||
type=str,
|
||||
default=DEFAULT_MIRROR,
|
||||
choices=get_args(RustMirrorType),
|
||||
help="Rust 镜像源",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--rust-version",
|
||||
nargs="?",
|
||||
type=str,
|
||||
default=DEFAULT_RUST_VERSION,
|
||||
choices=get_args(RustVersionType),
|
||||
help=f"Rust 版本, 推荐: {get_args(RustVersionType)}",
|
||||
)
|
||||
args = parser.parse_args()
|
||||
|
||||
python_mirror = args.python_mirror
|
||||
conda_mirror_urls = CONDA_MIRROR_URLS[args.conda_mirror]
|
||||
rust_mirror = args.rust_mirror
|
||||
rust_version = args.rust_version
|
||||
|
||||
# 确保配置文件目录存在
|
||||
PIP_CONFIG_PATH.parent.mkdir(parents=True, exist_ok=True)
|
||||
CONDA_CONFIG_PATH.parent.mkdir(parents=True, exist_ok=True)
|
||||
RUST_CONFIG_PATH.parent.mkdir(parents=True, exist_ok=True)
|
||||
RUST_SCCACHE_DIR.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# 使用 conditions 自动控制任务执行
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
# 系统镜像配置(仅 Linux 且未配置国内镜像)
|
||||
px.TaskSpec(
|
||||
"download_mirror",
|
||||
cmd=DOWNLOAD_MIRROR_SCRIPT,
|
||||
conditions=(
|
||||
BuiltinConditions.IS_LINUX(),
|
||||
BuiltinConditions.NOT(
|
||||
BuiltinConditions.OR(
|
||||
*[
|
||||
BuiltinConditions.FILE_CONTENT_EXISTS(f, m)
|
||||
for f in [
|
||||
"/etc/apt/sources.list",
|
||||
"/etc/apt/sources.list.d/ubuntu.sources",
|
||||
]
|
||||
for m in get_args(PyMirrorType)
|
||||
],
|
||||
)
|
||||
),
|
||||
),
|
||||
verbose=True,
|
||||
),
|
||||
px.TaskSpec(
|
||||
"install_mirror",
|
||||
cmd=INSTALL_MIRROR_SCRIPT,
|
||||
depends_on=("download_mirror",),
|
||||
verbose=True,
|
||||
),
|
||||
# 安装 Qt 依赖(仅 Linux)
|
||||
px.TaskSpec(
|
||||
"install_qt_libs",
|
||||
cmd=["sudo", "apt", "install", "-y", *QT_LIBS],
|
||||
conditions=(BuiltinConditions.IS_LINUX(),),
|
||||
depends_on=("install_mirror",),
|
||||
allow_upstream_skip=True,
|
||||
verbose=True,
|
||||
),
|
||||
# 安装中文字体(仅 Linux)
|
||||
px.TaskSpec(
|
||||
"install_fonts",
|
||||
cmd=["sudo", "apt", "install", "-y", *CHINESE_FONTS],
|
||||
conditions=(BuiltinConditions.IS_LINUX(),),
|
||||
depends_on=("install_mirror",),
|
||||
allow_upstream_skip=True,
|
||||
verbose=True,
|
||||
),
|
||||
# 安装 Docker
|
||||
px.TaskSpec(
|
||||
"install_docker",
|
||||
cmd=["sudo", "apt", "install", "-y", "docker-compose-v2"],
|
||||
conditions=(BuiltinConditions.IS_LINUX(),),
|
||||
depends_on=("install_mirror",),
|
||||
allow_upstream_skip=True,
|
||||
verbose=True,
|
||||
),
|
||||
px.TaskSpec(
|
||||
"add_docker_group",
|
||||
cmd=["sudo", "usermod", "-aG", "docker", getpass.getuser()],
|
||||
conditions=(BuiltinConditions.IS_LINUX(),),
|
||||
depends_on=("install_docker",),
|
||||
allow_upstream_skip=True,
|
||||
verbose=True,
|
||||
),
|
||||
px.TaskSpec(
|
||||
"refresh_docker_group",
|
||||
cmd=["newgrp", "docker"],
|
||||
conditions=(BuiltinConditions.IS_LINUX(),),
|
||||
depends_on=("add_docker_group",),
|
||||
allow_upstream_skip=True,
|
||||
verbose=True,
|
||||
),
|
||||
# 设置 Python 环境变量
|
||||
*setenv_group(
|
||||
{
|
||||
"PIP_INDEX_URL": PIP_INDEX_URLS[python_mirror],
|
||||
"PIP_TRUSTED_HOSTS": PIP_TRUSTED_HOSTS[python_mirror],
|
||||
"UV_INDEX_URL": UV_INDEX_URLS[python_mirror],
|
||||
"UV_PYTHON_INSTALL_MIRROR": UV_PYTHON_INSTALL_MIRROR,
|
||||
"UV_HTTP_TIMEOUT": "600",
|
||||
"UV_LINK_MODE": "copy",
|
||||
}
|
||||
),
|
||||
# 写入 Python 配置(仅当未配置)
|
||||
write_file(
|
||||
str(PIP_CONFIG_PATH),
|
||||
f"[global]\nindex-url = {PIP_INDEX_URLS[python_mirror]}\ntrusted-host = {PIP_TRUSTED_HOSTS[python_mirror]}",
|
||||
),
|
||||
# 写入 Conda 配置(仅当未配置)
|
||||
write_file(
|
||||
str(CONDA_CONFIG_PATH),
|
||||
"show_channel_urls: true\nchannels:\n - " + "\n - ".join(conda_mirror_urls) + "\n - defaults",
|
||||
),
|
||||
# 设置 Rust 镜像源
|
||||
*setenv_group(
|
||||
{
|
||||
"RUSTUP_DIST_SERVER": RUSTUP_MIRRORS[rust_mirror]["RUSTUP_DIST_SERVER"],
|
||||
"RUSTUP_UPDATE_ROOT": RUSTUP_MIRRORS[rust_mirror]["RUSTUP_UPDATE_ROOT"],
|
||||
"RUST_SCCACHE_DIR": str(RUST_SCCACHE_DIR),
|
||||
"RUST_SCCACHE_CACHE_SIZE": RUST_SCCACHE_CACHE_SIZE,
|
||||
}
|
||||
),
|
||||
# 写入 Rust 配置(仅当未配置)
|
||||
write_file(
|
||||
str(RUST_CONFIG_PATH),
|
||||
f"""
|
||||
[source.crates-io]
|
||||
replace-with = '{rust_mirror}'
|
||||
|
||||
[source.{rust_mirror}]
|
||||
registry = "sparse+{RUSTUP_MIRRORS[rust_mirror]["TOML_REGISTRY"]}"
|
||||
|
||||
[registries.{rust_mirror}]
|
||||
index = "sparse+{RUSTUP_MIRRORS[rust_mirror]["TOML_REGISTRY"]}"
|
||||
""",
|
||||
),
|
||||
# 下载 Rustup 安装脚本
|
||||
px.TaskSpec(
|
||||
"download_rustup",
|
||||
cmd=["curl", "-fsSL", RUSTUP_DOWNLOAD_URL_LINUX, "-o", "rustup-init.sh"],
|
||||
conditions=(
|
||||
BuiltinConditions.IS_LINUX(),
|
||||
BuiltinConditions.NOT(BuiltinConditions.HAS_INSTALLED("rustup")),
|
||||
),
|
||||
verbose=True,
|
||||
),
|
||||
px.TaskSpec(
|
||||
"download_rustup_win",
|
||||
cmd=[
|
||||
"powershell",
|
||||
"-Command",
|
||||
"Invoke-WebRequest",
|
||||
"-Uri",
|
||||
RUSTUP_DOWNLOAD_URL_WINDOWS,
|
||||
"-OutFile",
|
||||
"rustup-init.exe",
|
||||
],
|
||||
conditions=(
|
||||
BuiltinConditions.IS_WINDOWS(),
|
||||
BuiltinConditions.NOT(BuiltinConditions.HAS_INSTALLED("rustup")),
|
||||
),
|
||||
verbose=True,
|
||||
),
|
||||
# 安装 Rust 工具链
|
||||
px.TaskSpec(
|
||||
"install_rust",
|
||||
cmd=["rustup", "toolchain", "install", rust_version],
|
||||
conditions=(BuiltinConditions.HAS_INSTALLED("rustup"),),
|
||||
depends_on=("setenv_rustup_dist_server",),
|
||||
allow_upstream_skip=True,
|
||||
verbose=True,
|
||||
),
|
||||
]
|
||||
)
|
||||
px.run(graph, strategy="thread", verbose=True)
|
||||
@@ -0,0 +1,15 @@
|
||||
"""legacy 子包 — 未迁移到 @px.tool 的独立工具.
|
||||
|
||||
每个子模块有自己的 ``main()`` 函数, 由 ``pf`` 入口通过
|
||||
``_LEGACY_TOOLS`` 路由调用. 这些工具因逻辑复杂 (web 应用、runpy 注入等)
|
||||
暂未用 ``@px.tool`` 装饰器重写.
|
||||
|
||||
子模块
|
||||
------
|
||||
- :mod:`emlmanager` —— EML 邮件管理 Web 应用
|
||||
- :mod:`profiler` —— pxp 性能分析器 (monkey-patch + runpy)
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
__all__: list[str] = []
|
||||
@@ -154,7 +154,7 @@ class EmailDatabase:
|
||||
cursor.execute(query, (f"%{keyword}%", f"%{keyword}%", f"%{keyword}%", f"%{keyword}%", limit, offset))
|
||||
|
||||
columns = [description[0] for description in cursor.description]
|
||||
return [dict(zip(columns, row)) for row in cursor.fetchall()]
|
||||
return [dict(zip(columns, row, strict=False)) for row in cursor.fetchall()]
|
||||
|
||||
def get_grouped_emails(self) -> dict[str, list[dict[str, Any]]]:
|
||||
"""获取按主题分组的邮件."""
|
||||
@@ -165,7 +165,7 @@ class EmailDatabase:
|
||||
cursor.execute(f"SELECT * FROM {TABLE_NAME} ORDER BY subject, date_parsed DESC")
|
||||
|
||||
columns = [description[0] for description in cursor.description]
|
||||
emails = [dict(zip(columns, row)) for row in cursor.fetchall()]
|
||||
emails = [dict(zip(columns, row, strict=False)) for row in cursor.fetchall()]
|
||||
|
||||
# 按主题分组
|
||||
grouped: dict[str, list[dict[str, Any]]] = {}
|
||||
@@ -602,7 +602,7 @@ class EmlManagerHandler(BaseHTTPRequestHandler):
|
||||
self._send_json_response({"error": "邮件不存在"}, 404)
|
||||
return
|
||||
|
||||
email_data = dict(zip(columns, row))
|
||||
email_data = dict(zip(columns, row, strict=False))
|
||||
self._send_json_response({"email": email_data})
|
||||
|
||||
def _api_get_grouped_emails(self) -> None:
|
||||
@@ -40,10 +40,10 @@ import webbrowser
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from .. import executors as _executors
|
||||
from .. import runner as _runner
|
||||
from ..profiling import ProfileReport
|
||||
from ..report import RunReport
|
||||
from ... import executors as _executors
|
||||
from ... import runner as _runner
|
||||
from ...profiling import ProfileReport
|
||||
from ...report import RunReport
|
||||
|
||||
|
||||
def _build_parser() -> argparse.ArgumentParser:
|
||||
@@ -1,43 +0,0 @@
|
||||
"""Download from ModelScopeHub."""
|
||||
|
||||
import argparse
|
||||
from pathlib import Path
|
||||
from typing import Literal, get_args
|
||||
|
||||
import pyflowx as px
|
||||
|
||||
DownloadType = Literal["model", "dataset", "space"]
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description="Download a model from ModelScopeHub.")
|
||||
parser.add_argument("name", help="Target name.")
|
||||
parser.add_argument("--type", "-t", nargs="?", default="model", choices=get_args(DownloadType), help="Target type.")
|
||||
parser.add_argument("--dir", default=None, help="Download directory.")
|
||||
args = parser.parse_args()
|
||||
|
||||
if not args.name:
|
||||
parser.error("name is required")
|
||||
|
||||
download_dir: Path = Path(args.dir) if args.dir else Path.home() / ".models" / args.name.split("/")[-1]
|
||||
download_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec(
|
||||
name="download",
|
||||
cmd=[
|
||||
"uvx",
|
||||
"modelscope",
|
||||
"download",
|
||||
f"--{args.type}",
|
||||
args.name,
|
||||
"--local_dir",
|
||||
str(download_dir),
|
||||
],
|
||||
verbose=True,
|
||||
),
|
||||
]
|
||||
)
|
||||
|
||||
px.run(graph, strategy="thread", verbose=True)
|
||||
@@ -1,65 +0,0 @@
|
||||
"""使用 SGLang 运行本地模型."""
|
||||
|
||||
import argparse
|
||||
from pathlib import Path
|
||||
|
||||
import pyflowx as px
|
||||
from pyflowx.conditions import BuiltinConditions, Constants
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description="启动 SGLang 服务")
|
||||
parser.add_argument("--model", default="~/.models/Qwen2.5-Coder-32B-Instruct-AWQ", help="模型路径")
|
||||
parser.add_argument("--port", type=int, default=8000, help="服务端口")
|
||||
parser.add_argument("--ctx-len", type=int, default=28672, help="最大上下文长度")
|
||||
parser.add_argument("--mem", type=float, default=0.75, help="显存占比 (0-1)")
|
||||
parser.add_argument("--host", default="0.0.0.0", help="主机地址")
|
||||
parser.add_argument("--log-level", default="info", help="日志级别")
|
||||
args = parser.parse_args()
|
||||
|
||||
if not args.model:
|
||||
parser.error("model is required")
|
||||
|
||||
model_dir = Path(args.model).expanduser()
|
||||
if not model_dir.exists():
|
||||
parser.error(f"Model directory {model_dir} does not exist.")
|
||||
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec(
|
||||
name="download",
|
||||
cmd=[
|
||||
"uv",
|
||||
"install",
|
||||
"sglang[all]",
|
||||
],
|
||||
conditions=(BuiltinConditions.NOT(BuiltinConditions.HAS_INSTALLED("sglang")),),
|
||||
verbose=True,
|
||||
),
|
||||
px.TaskSpec(
|
||||
name="run",
|
||||
cmd=[
|
||||
"python" if Constants.IS_WINDOWS else "python3",
|
||||
"-m",
|
||||
"sglang.launch_server",
|
||||
"--model-path",
|
||||
str(model_dir),
|
||||
"--host",
|
||||
str(args.host),
|
||||
"--port",
|
||||
"8000",
|
||||
"--mem-fraction-static",
|
||||
str(args.mem),
|
||||
"--context-length",
|
||||
"32768",
|
||||
"--tool-call-parser",
|
||||
"qwen",
|
||||
"--log-level",
|
||||
str(args.log_level),
|
||||
],
|
||||
verbose=True,
|
||||
),
|
||||
]
|
||||
)
|
||||
|
||||
px.run(graph, strategy="sequential", verbose=True)
|
||||
+151
-62
@@ -1,7 +1,7 @@
|
||||
"""PyFlowX 统一 CLI 入口.
|
||||
|
||||
通过 ``pf <tool> [command] [options]`` 调用所有工具,
|
||||
工具定义在 ``configs/`` 目录下的 YAML 文件中.
|
||||
工具定义在 ``pyflowx.ops`` 子包中, 每个模块用 ``@px.tool`` 装饰器注册.
|
||||
|
||||
用法
|
||||
----
|
||||
@@ -13,29 +13,40 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import contextlib
|
||||
import difflib
|
||||
import importlib
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from typing import Sequence
|
||||
from collections.abc import Sequence
|
||||
|
||||
import pyflowx as px
|
||||
from rich.console import Console
|
||||
from rich.panel import Panel
|
||||
from rich.table import Table
|
||||
from rich.text import Text
|
||||
|
||||
from pyflowx import __version__
|
||||
from pyflowx.tools import _TOOL_REGISTRY, run_tool
|
||||
|
||||
|
||||
class PfApp:
|
||||
"""pf 统一入口应用.
|
||||
|
||||
路由 ``pf <tool> [command]`` 到 YAML 配置工具或传统 Python 工具.
|
||||
路由 ``pf <tool> [command]`` 到 ``@px.tool`` 注册的工具或传统 Python 工具.
|
||||
"""
|
||||
|
||||
_CONFIGS_DIR = Path(__file__).parent.parent / "configs"
|
||||
|
||||
# 工具名到 YAML 配置文件的映射 (支持短别名)
|
||||
# 工具名到 ops 模块名的映射 (支持短别名)
|
||||
_TOOL_ALIASES: dict[str, str] = {
|
||||
"autofmt": "autofmt",
|
||||
"af": "autofmt",
|
||||
"bump": "bumpversion",
|
||||
"bumpversion": "bumpversion",
|
||||
"bv": "bumpversion",
|
||||
"clr": "clr",
|
||||
"clearscreen": "clr",
|
||||
"dockercmd": "dockercmd",
|
||||
"docker": "dockercmd",
|
||||
"envdev": "envdev",
|
||||
"env": "envdev",
|
||||
"filedate": "filedate",
|
||||
"fd": "filedate",
|
||||
"filelevel": "filelevel",
|
||||
@@ -52,6 +63,9 @@ class PfApp:
|
||||
"gt": "gittool",
|
||||
"ls": "lscalc",
|
||||
"lscalc": "lscalc",
|
||||
"msdown": "msdownload",
|
||||
"msdownload": "msdownload",
|
||||
"msd": "msdownload",
|
||||
"pack": "packtool",
|
||||
"packtool": "packtool",
|
||||
"pk": "packtool",
|
||||
@@ -68,22 +82,56 @@ class PfApp:
|
||||
"screenshot": "screenshot",
|
||||
"scrcap": "screenshot",
|
||||
"ss": "screenshot",
|
||||
"sglang": "sglang",
|
||||
"sg": "sglang",
|
||||
"ssh": "sshcopyid",
|
||||
"sshcopy": "sshcopyid",
|
||||
"sshcopyid": "sshcopyid",
|
||||
"sc": "sshcopyid",
|
||||
"taskk": "taskkill",
|
||||
"taskkill": "taskkill",
|
||||
"tk": "taskkill",
|
||||
"wch": "which",
|
||||
"which": "which",
|
||||
}
|
||||
|
||||
# 传统工具: 有自己的 main() 函数 (无法 YAML 化的复杂逻辑)
|
||||
# 传统工具: 有自己的 main() 函数 (无法 @px.tool 化的复杂逻辑)
|
||||
_LEGACY_TOOLS: dict[str, str] = {
|
||||
"emlman": "pyflowx.cli.emlmanager:main",
|
||||
"profiler": "pyflowx.cli.profiler:main",
|
||||
"pxp": "pyflowx.cli.profiler:main",
|
||||
"yamlrun": "pyflowx.cli.yamlrun:main",
|
||||
"emlman": "pyflowx.cli.legacy.emlmanager:main",
|
||||
"profiler": "pyflowx.cli.legacy.profiler:main",
|
||||
"pxp": "pyflowx.cli.legacy.profiler:main",
|
||||
}
|
||||
|
||||
# 规范工具名 → 完整模块路径 (ops/ 按功能分组后的动态导入映射)
|
||||
_TOOL_MODULES: dict[str, str] = {
|
||||
"autofmt": "pyflowx.ops.dev.autofmt",
|
||||
"bumpversion": "pyflowx.ops.dev.bumpversion",
|
||||
"clr": "pyflowx.ops.system.clr",
|
||||
"dockercmd": "pyflowx.ops.infra.dockercmd",
|
||||
"envdev": "pyflowx.ops.infra.envdev",
|
||||
"filedate": "pyflowx.ops.files.filedate",
|
||||
"filelevel": "pyflowx.ops.files.filelevel",
|
||||
"folderback": "pyflowx.ops.files.folderback",
|
||||
"folderzip": "pyflowx.ops.files.folderzip",
|
||||
"gittool": "pyflowx.ops.dev.gittool",
|
||||
"lscalc": "pyflowx.ops.dev.lscalc",
|
||||
"msdownload": "pyflowx.ops.infra.msdownload",
|
||||
"packtool": "pyflowx.ops.dev.packtool",
|
||||
"pdftool": "pyflowx.ops.files.pdftool",
|
||||
"piptool": "pyflowx.ops.dev.piptool",
|
||||
"pymake": "pyflowx.ops.dev.pymake",
|
||||
"reseticoncache": "pyflowx.ops.system.reseticoncache",
|
||||
"screenshot": "pyflowx.ops.files.screenshot",
|
||||
"sglang": "pyflowx.ops.infra.sglang",
|
||||
"sshcopyid": "pyflowx.ops.infra.sshcopyid",
|
||||
"taskkill": "pyflowx.ops.system.taskkill",
|
||||
"which": "pyflowx.ops.system.which",
|
||||
}
|
||||
|
||||
def __init__(self, argv: Sequence[str] | None = None) -> None:
|
||||
self._argv = list(argv) if argv is not None else sys.argv[1:]
|
||||
self._console = Console()
|
||||
self._err = Console(stderr=True)
|
||||
|
||||
def run(self) -> int:
|
||||
"""主入口, 返回退出码."""
|
||||
@@ -91,64 +139,101 @@ class PfApp:
|
||||
self._list_tools()
|
||||
return 0
|
||||
|
||||
tool_name = self._argv[0]
|
||||
rest_argv = self._argv[1:]
|
||||
first = self._argv[0]
|
||||
|
||||
resolved = self._resolve_tool(tool_name)
|
||||
if first in ("--help", "-h"):
|
||||
self._list_tools()
|
||||
return 0
|
||||
if first in ("--version", "-V"):
|
||||
self._console.print(f"PyFlowX [bold cyan]{__version__}[/bold cyan]")
|
||||
return 0
|
||||
|
||||
rest = self._argv[1:]
|
||||
resolved = self._resolve_tool(first)
|
||||
if resolved is None:
|
||||
print(f"错误: 未知工具 '{tool_name}'", file=sys.stderr)
|
||||
print("运行 'pf' 查看可用工具列表", file=sys.stderr)
|
||||
self._print_unknown_tool(first)
|
||||
return 1
|
||||
|
||||
tool_type, target = resolved
|
||||
if tool_type == "legacy":
|
||||
return self._run_legacy(target, rest_argv)
|
||||
return self._run_yaml(target, rest_argv)
|
||||
kind, target = resolved
|
||||
if kind == "legacy":
|
||||
return self._run_legacy(target, rest)
|
||||
return self._run_tool(target, rest)
|
||||
|
||||
# ------------------------------------------------------------------ #
|
||||
# 工具列表 (rich)
|
||||
# ------------------------------------------------------------------ #
|
||||
def _list_tools(self) -> None:
|
||||
"""列出所有可用工具."""
|
||||
print("PyFlowX 工具列表:")
|
||||
print()
|
||||
print("YAML 配置工具:")
|
||||
yaml_tools = sorted(set(self._TOOL_ALIASES.values()))
|
||||
for tool in yaml_tools:
|
||||
print(f" pf {tool:<15} - {self._tool_description(tool)}")
|
||||
print()
|
||||
print("传统工具:")
|
||||
for tool in sorted(self._LEGACY_TOOLS):
|
||||
print(f" pf {tool:<15}")
|
||||
print()
|
||||
print("示例:")
|
||||
print(" pf filedate add a.txt")
|
||||
print(" pf pymake b")
|
||||
"""rich 表格列出所有可用工具."""
|
||||
self._console.print(
|
||||
Panel(
|
||||
Text(f"PyFlowX v{__version__}", style="bold cyan", justify="center"),
|
||||
subtitle="[dim]pf <tool> [command] [options][/dim]",
|
||||
)
|
||||
)
|
||||
|
||||
table = Table(title="@px.tool 工具", show_header=True, header_style="bold", show_lines=False)
|
||||
table.add_column("命令", style="cyan", no_wrap=True)
|
||||
table.add_column("别名", style="dim", no_wrap=True)
|
||||
table.add_column("说明")
|
||||
|
||||
for tool in sorted(set(self._TOOL_ALIASES.values())):
|
||||
aliases = self._aliases_for(tool)
|
||||
table.add_row(f"pf {tool}", ", ".join(aliases), self._tool_description(tool))
|
||||
|
||||
self._console.print(table)
|
||||
|
||||
if self._LEGACY_TOOLS:
|
||||
legacy = Table(title="传统工具", show_header=True, header_style="bold", show_lines=False)
|
||||
legacy.add_column("命令", style="cyan", no_wrap=True)
|
||||
for tool in sorted(self._LEGACY_TOOLS):
|
||||
legacy.add_row(f"pf {tool}")
|
||||
self._console.print(legacy)
|
||||
|
||||
self._console.print("\n[bold]示例:[/bold]")
|
||||
self._console.print(" [cyan]pf filedate add a.txt[/cyan] # 给文件添加日期前缀")
|
||||
self._console.print(" [cyan]pf pymake b[/cyan] # 构建 Python 包")
|
||||
self._console.print(" [cyan]pf gitt c[/cyan] # 清理并查看 git 状态")
|
||||
|
||||
def _aliases_for(self, canonical: str) -> list[str]:
|
||||
"""获取工具的别名 (不含规范名本身)."""
|
||||
return sorted(a for a, t in self._TOOL_ALIASES.items() if t == canonical and a != canonical)
|
||||
|
||||
def _tool_description(self, tool_name: str) -> str:
|
||||
"""获取工具描述 (从 YAML cli.description)."""
|
||||
config_path = self._CONFIGS_DIR / f"{tool_name}.yaml"
|
||||
if not config_path.exists():
|
||||
return ""
|
||||
try:
|
||||
import yaml
|
||||
"""获取工具描述 (从 _TOOL_REGISTRY 中已注册 ToolSpec 的 description/help)."""
|
||||
with contextlib.suppress(ImportError, KeyError):
|
||||
importlib.import_module(self._TOOL_MODULES[tool_name])
|
||||
|
||||
data = yaml.safe_load(config_path.read_text(encoding="utf-8"))
|
||||
if isinstance(data, dict) and isinstance(data.get("cli"), dict):
|
||||
return str(data["cli"].get("description", ""))
|
||||
except Exception:
|
||||
pass
|
||||
if tool_name not in _TOOL_REGISTRY:
|
||||
return ""
|
||||
|
||||
subs = _TOOL_REGISTRY[tool_name]
|
||||
for spec in subs.values():
|
||||
if spec.description:
|
||||
return spec.description
|
||||
for spec in subs.values():
|
||||
if not spec.hidden and spec.help:
|
||||
return spec.help
|
||||
return ""
|
||||
|
||||
# ------------------------------------------------------------------ #
|
||||
# 路由
|
||||
# ------------------------------------------------------------------ #
|
||||
def _resolve_tool(self, name: str) -> tuple[str, str] | None:
|
||||
"""解析工具名, 返回 (类型, 目标).
|
||||
|
||||
类型: "yaml" 或 "legacy"
|
||||
目标: YAML 文件名 (不含 .yaml) 或 legacy 模块路径
|
||||
"""
|
||||
"""解析工具名, 返回 (类型, 目标)."""
|
||||
if name in self._TOOL_ALIASES:
|
||||
return ("yaml", self._TOOL_ALIASES[name])
|
||||
return ("tool", self._TOOL_ALIASES[name])
|
||||
if name in self._LEGACY_TOOLS:
|
||||
return ("legacy", self._LEGACY_TOOLS[name])
|
||||
return None
|
||||
|
||||
def _print_unknown_tool(self, name: str) -> None:
|
||||
"""打印未知工具错误 + 模糊匹配建议."""
|
||||
self._err.print(f"[red]错误:[/red] 未知工具 [yellow]{name!r}[/yellow]")
|
||||
suggestions = difflib.get_close_matches(name, list(self._TOOL_ALIASES), n=3, cutoff=0.5)
|
||||
if suggestions:
|
||||
self._err.print(f"[dim]是否想用: {', '.join(suggestions)}[/dim]")
|
||||
self._err.print("[dim]运行 'pf' 查看可用工具列表[/dim]")
|
||||
|
||||
def _run_legacy(self, module_path: str, argv: list[str]) -> int:
|
||||
"""运行传统工具的 main() 函数."""
|
||||
module_name, func_name = module_path.split(":", 1)
|
||||
@@ -165,16 +250,20 @@ class PfApp:
|
||||
finally:
|
||||
sys.argv = original_argv
|
||||
|
||||
def _run_yaml(self, target: str, argv: list[str]) -> int:
|
||||
"""运行 YAML 配置工具."""
|
||||
config_path = self._CONFIGS_DIR / f"{target}.yaml"
|
||||
if not config_path.exists():
|
||||
print(f"错误: 未找到配置文件 '{config_path}'", file=sys.stderr)
|
||||
print("运行 'pf' 查看可用工具列表", file=sys.stderr)
|
||||
def _run_tool(self, target: str, argv: list[str]) -> int:
|
||||
"""运行 @px.tool 工具.
|
||||
|
||||
导入 ``pyflowx.ops.<group>.<target>`` 触发 ``@px.tool`` 注册, 然后调用 ``run_tool``.
|
||||
"""
|
||||
with contextlib.suppress(ImportError, KeyError):
|
||||
importlib.import_module(self._TOOL_MODULES[target])
|
||||
|
||||
if target not in _TOOL_REGISTRY:
|
||||
self._err.print(f"[red]错误:[/red] 未找到工具 [yellow]{target!r}[/yellow]")
|
||||
self._err.print("[dim]运行 'pf' 查看可用工具列表[/dim]")
|
||||
return 1
|
||||
|
||||
print(f"运行配置文件 '{config_path}'")
|
||||
return px.run_cli(config_path, argv)
|
||||
return run_tool(target, argv)
|
||||
|
||||
|
||||
def main() -> None:
|
||||
|
||||
@@ -1,15 +0,0 @@
|
||||
"""清屏工具.
|
||||
|
||||
跨平台清屏工具, 支持终端和控制台清屏.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import pyflowx as px
|
||||
from pyflowx.tasks.system import clr
|
||||
|
||||
|
||||
def main() -> None:
|
||||
"""清屏工具主函数."""
|
||||
graph = px.Graph.from_specs([clr()])
|
||||
px.run(graph, strategy="thread")
|
||||
@@ -1,40 +0,0 @@
|
||||
"""进程终止工具.
|
||||
|
||||
跨平台进程终止工具, 支持按名称终止进程.
|
||||
用法: taskkill proc_name [proc_name ...]
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
|
||||
import pyflowx as px
|
||||
from pyflowx.conditions import Constants
|
||||
|
||||
|
||||
def main() -> None:
|
||||
"""进程终止工具主函数."""
|
||||
parser = argparse.ArgumentParser(
|
||||
description="TaskKill - 进程终止工具",
|
||||
usage="taskkill <process_name> [process_name ...]",
|
||||
)
|
||||
parser.add_argument(
|
||||
"process_names",
|
||||
type=str,
|
||||
nargs="+",
|
||||
help="进程名称 (如: chrome.exe python node)",
|
||||
)
|
||||
args = parser.parse_args()
|
||||
|
||||
if Constants.IS_WINDOWS:
|
||||
cmd = ["taskkill", "/f", "/im"]
|
||||
else:
|
||||
cmd = ["pkill", "-f"]
|
||||
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec(f"kill_{proc_name}", cmd=[*cmd, f"{proc_name}*"], verbose=True)
|
||||
for proc_name in args.process_names
|
||||
],
|
||||
)
|
||||
px.run(graph, strategy="thread")
|
||||
@@ -1,21 +0,0 @@
|
||||
"""命令查找工具.
|
||||
|
||||
跨平台查找可执行命令路径, 类似 Unix 的 which 命令.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
|
||||
import pyflowx as px
|
||||
from pyflowx.tasks.system import which
|
||||
|
||||
|
||||
def main() -> None:
|
||||
"""命令查找工具主函数."""
|
||||
parser = argparse.ArgumentParser(description="Which - 命令查找工具")
|
||||
parser.add_argument("commands", nargs="+", help="要查找的命令名称, 如: python ls ps gcc...")
|
||||
args = parser.parse_args()
|
||||
|
||||
graph = px.Graph.from_specs([which(cmd) for cmd in args.commands])
|
||||
px.run(graph, strategy="thread")
|
||||
@@ -1,109 +0,0 @@
|
||||
"""YAML 任务编排执行工具.
|
||||
|
||||
从 YAML 文件加载 GitHub Actions 风格的任务图并执行.
|
||||
支持串并行编排、矩阵扇出、条件执行等 CI/CD 核心概念.
|
||||
|
||||
用法
|
||||
----
|
||||
yamlrun pipeline.yaml # 执行 YAML 任务图
|
||||
yamlrun pipeline.yaml --strategy thread # 指定执行策略
|
||||
yamlrun pipeline.yaml --dry-run # 仅打印任务分层, 不执行
|
||||
yamlrun pipeline.yaml --list # 列出所有任务名
|
||||
yamlrun pipeline.yaml --quiet # 静默模式
|
||||
|
||||
示例 YAML
|
||||
----------
|
||||
::
|
||||
|
||||
strategy: thread
|
||||
jobs:
|
||||
setup:
|
||||
cmd: ["git", "clone", "https://github.com/foo/bar"]
|
||||
build:
|
||||
needs: [setup]
|
||||
cmd: ["python", "-m", "build"]
|
||||
test:
|
||||
needs: [build]
|
||||
cmd: ["pytest"]
|
||||
strategy:
|
||||
matrix:
|
||||
python: ["3.8", "3.9"]
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from typing import cast
|
||||
|
||||
import pyflowx as px
|
||||
from pyflowx.executors import Strategy
|
||||
|
||||
|
||||
def main() -> None:
|
||||
"""YAML 任务编排执行工具主函数."""
|
||||
parser = argparse.ArgumentParser(
|
||||
description="YamlRun - 从 YAML 文件加载并执行任务图",
|
||||
usage="yamlrun <file.yaml> [--strategy STRATEGY] [--dry-run] [--list] [--quiet]",
|
||||
)
|
||||
parser.add_argument("file", type=str, help="YAML 任务图文件路径")
|
||||
parser.add_argument(
|
||||
"--strategy",
|
||||
type=str,
|
||||
default=None,
|
||||
help="执行策略: sequential/thread/async/dependency (默认: YAML 中指定的策略或 dependency)",
|
||||
)
|
||||
parser.add_argument("--dry-run", action="store_true", help="仅打印任务分层, 不执行")
|
||||
parser.add_argument("--list", action="store_true", help="列出所有任务名后退出")
|
||||
parser.add_argument("--quiet", action="store_true", help="静默模式, 不打印详细输出")
|
||||
args = parser.parse_args()
|
||||
|
||||
file_path = Path(args.file)
|
||||
if not file_path.exists():
|
||||
print(f"错误: 文件不存在: {file_path}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
try:
|
||||
graph = px.Graph.from_yaml(file_path)
|
||||
except px.YamlLoadError as e:
|
||||
print(f"错误: YAML 加载失败: {e}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
if args.list:
|
||||
print("任务列表:")
|
||||
for name in graph.names:
|
||||
spec = graph.spec(name)
|
||||
deps = ", ".join(spec.depends_on) if spec.depends_on else "(无依赖)"
|
||||
print(f" - {name} (依赖: {deps})")
|
||||
sys.exit(0)
|
||||
|
||||
layers = graph.layers()
|
||||
print(f"任务分层 ({len(layers)} 层):")
|
||||
for i, layer in enumerate(layers):
|
||||
print(f" 层 {i + 1}: {layer}")
|
||||
|
||||
if args.dry_run:
|
||||
print("\n[dry-run] 跳过执行")
|
||||
sys.exit(0)
|
||||
|
||||
strategy = args.strategy or graph.defaults.strategy or "dependency"
|
||||
print(f"\n执行策略: {strategy}")
|
||||
print(f"任务总数: {len(graph.names)}")
|
||||
print("-" * 40)
|
||||
|
||||
report = px.run(graph, strategy=cast(Strategy, strategy), verbose=not args.quiet)
|
||||
|
||||
print("-" * 40)
|
||||
succeeded = report.succeeded_tasks()
|
||||
failed = report.failed_tasks()
|
||||
skipped = report.skipped_tasks()
|
||||
print(f"完成: {len(succeeded)} 成功 / {len(failed)} 失败 / {len(skipped)} 跳过 (共 {len(graph.names)})")
|
||||
|
||||
if failed:
|
||||
print(f"失败任务: {failed}")
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
+12
-7
@@ -10,12 +10,16 @@ from __future__ import annotations
|
||||
|
||||
import os
|
||||
import subprocess
|
||||
from typing import Any, List, Union, cast
|
||||
from typing import Any, cast
|
||||
|
||||
from rich.console import Console
|
||||
|
||||
from .task import TaskSpec
|
||||
|
||||
__all__ = ["run_command"]
|
||||
|
||||
_cmd_console = Console()
|
||||
|
||||
|
||||
def run_command(spec: TaskSpec[Any]) -> Any: # noqa: PLR0912
|
||||
"""执行 ``spec.cmd`` 指定的命令(list / shell 字符串 / 可调用对象)。
|
||||
@@ -39,9 +43,9 @@ def run_command(spec: TaskSpec[Any]) -> Any: # noqa: PLR0912
|
||||
if callable(cmd) and not isinstance(cmd, (list, str)):
|
||||
name = getattr(cmd, "__name__", "callable")
|
||||
if verbose:
|
||||
print(f"[verbose] 执行可调用命令: {name}", flush=True)
|
||||
_cmd_console.print(f"[cyan]▸[/cyan] 执行可调用命令: [bold]{name}[/bold]")
|
||||
if cwd is not None:
|
||||
print(f"[verbose] 工作目录: {cwd}", flush=True)
|
||||
_cmd_console.print(f" [dim]工作目录: {cwd}[/dim]")
|
||||
try:
|
||||
return cmd()
|
||||
except Exception as e:
|
||||
@@ -58,9 +62,9 @@ def run_command(spec: TaskSpec[Any]) -> Any: # noqa: PLR0912
|
||||
label = "Shell 命令"
|
||||
|
||||
if verbose:
|
||||
print(f"[verbose] {verb}: {cmd_str}", flush=True)
|
||||
_cmd_console.print(f"[cyan]▸[/cyan] {verb}: [bold]{cmd_str}[/bold]")
|
||||
if cwd is not None:
|
||||
print(f"[verbose] 工作目录: {cwd}", flush=True)
|
||||
_cmd_console.print(f" [dim]工作目录: {cwd}[/dim]")
|
||||
|
||||
# 合并环境变量
|
||||
run_env: dict[str, str] | None = None
|
||||
@@ -70,7 +74,7 @@ def run_command(spec: TaskSpec[Any]) -> Any: # noqa: PLR0912
|
||||
|
||||
try:
|
||||
result = subprocess.run(
|
||||
cast(Union[str, List[str]], cmd),
|
||||
cast(str | list[str], cmd),
|
||||
shell=not is_list,
|
||||
cwd=cwd,
|
||||
env=run_env,
|
||||
@@ -87,7 +91,8 @@ def run_command(spec: TaskSpec[Any]) -> Any: # noqa: PLR0912
|
||||
raise RuntimeError(f"{label}执行异常: {cmd_str}: {e}") from e
|
||||
|
||||
if verbose:
|
||||
print(f"[verbose] 返回码: {result.returncode}", flush=True)
|
||||
style = "green" if result.returncode == 0 else "red"
|
||||
_cmd_console.print(f"[{style}]返回码: {result.returncode}[/{style}]")
|
||||
|
||||
if result.returncode == 0:
|
||||
if not verbose and result.stdout:
|
||||
|
||||
@@ -16,8 +16,9 @@ import os
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
from collections.abc import Callable
|
||||
from pathlib import Path
|
||||
from typing import Any, Callable
|
||||
from typing import Any
|
||||
|
||||
from .task import Condition, Context
|
||||
|
||||
|
||||
@@ -1,65 +0,0 @@
|
||||
# autofmt - 自动格式化工具
|
||||
# 用法:
|
||||
# pf autofmt fmt --target .
|
||||
# pf autofmt lint --target .
|
||||
# pf autofmt lint --target . --fix
|
||||
# pf autofmt doc --root-dir .
|
||||
# pf autofmt sync --root-dir .
|
||||
strategy: thread
|
||||
variables:
|
||||
TARGET: "."
|
||||
ROOT_DIR: "."
|
||||
FIX: false
|
||||
cli:
|
||||
description: "AutoFmt - 自动格式化工具"
|
||||
usage: "pf autofmt <command> [options]"
|
||||
subcommands:
|
||||
fmt:
|
||||
help: "格式化代码"
|
||||
options:
|
||||
- name: TARGET
|
||||
flag: "--target"
|
||||
type: str
|
||||
default: "."
|
||||
help: "目标路径 (默认: .)"
|
||||
lint:
|
||||
help: "代码检查"
|
||||
options:
|
||||
- name: TARGET
|
||||
flag: "--target"
|
||||
type: str
|
||||
default: "."
|
||||
help: "目标路径 (默认: .)"
|
||||
- name: FIX
|
||||
flag: "--fix"
|
||||
action: "store_true"
|
||||
help: "自动修复问题"
|
||||
doc:
|
||||
help: "自动添加文档字符串"
|
||||
options:
|
||||
- name: ROOT_DIR
|
||||
flag: "--root-dir"
|
||||
type: str
|
||||
default: "."
|
||||
help: "根目录 (默认: .)"
|
||||
sync:
|
||||
help: "同步 pyproject 配置"
|
||||
options:
|
||||
- name: ROOT_DIR
|
||||
flag: "--root-dir"
|
||||
type: str
|
||||
default: "."
|
||||
help: "根目录 (默认: .)"
|
||||
jobs:
|
||||
fmt:
|
||||
cmd: ["ruff", "format", "${TARGET}"]
|
||||
lint:
|
||||
cmd: ["ruff", "check", "${TARGET}"]
|
||||
lint_fix:
|
||||
cmd: ["ruff", "check", "--fix", "--unsafe-fixes", "${TARGET}"]
|
||||
doc:
|
||||
fn: auto_add_docstrings
|
||||
args: ["${ROOT_DIR}"]
|
||||
sync:
|
||||
fn: sync_pyproject_config
|
||||
args: ["${ROOT_DIR}"]
|
||||
@@ -1,27 +0,0 @@
|
||||
# bumpversion - 版本号自动管理工具
|
||||
# 用法:
|
||||
# pf bumpversion
|
||||
# pf bumpversion minor --no-tag
|
||||
strategy: sequential
|
||||
variables:
|
||||
PART: patch
|
||||
NO_TAG: false
|
||||
cli:
|
||||
description: "BumpVersion - 版本号自动管理工具"
|
||||
usage: "pf bumpversion [part] [options]"
|
||||
positional:
|
||||
- name: PART
|
||||
type: str
|
||||
default: patch
|
||||
help: "版本部分: patch, minor, major"
|
||||
options:
|
||||
- name: NO_TAG
|
||||
flag: "--no-tag"
|
||||
action: "store_true"
|
||||
help: "提交后不创建 git tag"
|
||||
jobs:
|
||||
bump:
|
||||
fn: bump_project_version
|
||||
args: ["${PART}"]
|
||||
kwargs:
|
||||
no_tag: ${NO_TAG}
|
||||
@@ -1,36 +0,0 @@
|
||||
# filedate - 文件日期处理工具
|
||||
# 用法:
|
||||
# pf filedate add file1.txt file2.txt
|
||||
# pf filedate clear file1.txt file2.txt
|
||||
strategy: thread
|
||||
variables:
|
||||
FILES: []
|
||||
cli:
|
||||
description: "FileDate - 文件日期处理工具"
|
||||
usage: "pf filedate <command> [files...]"
|
||||
subcommands:
|
||||
add:
|
||||
help: "添加日期前缀"
|
||||
positional:
|
||||
- name: FILES
|
||||
nargs: "+"
|
||||
type: path
|
||||
help: "文件路径"
|
||||
clear:
|
||||
help: "清除日期前缀"
|
||||
positional:
|
||||
- name: FILES
|
||||
nargs: "+"
|
||||
type: path
|
||||
help: "文件路径"
|
||||
jobs:
|
||||
add:
|
||||
fn: process_files_date
|
||||
args: ["${FILES}"]
|
||||
kwargs:
|
||||
clear: false
|
||||
clear:
|
||||
fn: process_files_date
|
||||
args: ["${FILES}"]
|
||||
kwargs:
|
||||
clear: true
|
||||
@@ -1,28 +0,0 @@
|
||||
# filelevel - 文件等级重命名工具
|
||||
# 用法:
|
||||
# pf filelevel set file.txt --level 2
|
||||
strategy: thread
|
||||
variables:
|
||||
FILES: []
|
||||
LEVEL: 0
|
||||
cli:
|
||||
description: "FileLevel - 文件等级重命名工具"
|
||||
usage: "pf filelevel <command> [files...] [options]"
|
||||
subcommands:
|
||||
set:
|
||||
help: "设置文件等级"
|
||||
positional:
|
||||
- name: FILES
|
||||
nargs: "+"
|
||||
type: path
|
||||
help: "文件路径"
|
||||
options:
|
||||
- name: LEVEL
|
||||
flag: "--level"
|
||||
type: int
|
||||
required: true
|
||||
help: "文件等级 (0-4)"
|
||||
jobs:
|
||||
set:
|
||||
fn: process_files_level
|
||||
args: ["${FILES}", "${LEVEL}"]
|
||||
@@ -1,34 +0,0 @@
|
||||
# folderback - 文件夹备份工具
|
||||
# 用法:
|
||||
# pf folderback
|
||||
# pf folderback --src ./project --dst ./backup --max-zip 10
|
||||
strategy: thread
|
||||
variables:
|
||||
SRC: "."
|
||||
DST: "./backup"
|
||||
MAX_ZIP: 5
|
||||
cli:
|
||||
description: "FolderBack - 文件夹备份工具"
|
||||
usage: "pf folderback [options]"
|
||||
options:
|
||||
- name: SRC
|
||||
flag: "--src"
|
||||
type: str
|
||||
default: "."
|
||||
help: "源文件夹路径 (默认: 当前目录)"
|
||||
- name: DST
|
||||
flag: "--dst"
|
||||
type: str
|
||||
default: "./backup"
|
||||
help: "目标文件夹路径 (默认: ./backup)"
|
||||
- name: MAX_ZIP
|
||||
flag: "--max-zip"
|
||||
type: int
|
||||
default: 5
|
||||
help: "最大备份数量 (默认: 5)"
|
||||
jobs:
|
||||
backup:
|
||||
fn: backup_folder
|
||||
args: ["${SRC}", "${DST}"]
|
||||
kwargs:
|
||||
max_zip: ${MAX_ZIP}
|
||||
@@ -1,21 +0,0 @@
|
||||
# folderzip - 文件夹压缩工具
|
||||
# 用法:
|
||||
# pf folderzip
|
||||
# pf folderzip --cwd ./project
|
||||
strategy: thread
|
||||
variables:
|
||||
CWD: "."
|
||||
cli:
|
||||
description: "FolderZip - 文件夹压缩工具"
|
||||
usage: "pf folderzip [options]"
|
||||
options:
|
||||
- name: CWD
|
||||
flag: "--cwd"
|
||||
type: str
|
||||
required: false
|
||||
default: "."
|
||||
help: "工作目录 (默认: 当前目录)"
|
||||
jobs:
|
||||
zip:
|
||||
fn: zip_folders
|
||||
args: ["${CWD}"]
|
||||
@@ -1,51 +0,0 @@
|
||||
# gittool - Git 执行工具
|
||||
# 用法:
|
||||
# pf gittool a
|
||||
# pf gittool c
|
||||
# pf gittool i
|
||||
# pf gittool isub
|
||||
# pf gittool p
|
||||
# pf gittool pl
|
||||
strategy: thread
|
||||
variables:
|
||||
# git clean -e 参数列表 (展开为 cmd 数组元素)
|
||||
CLEAN_EXCLUDES: ["-e", ".venv", "-e", ".tox", "-e", ".pytest_cache",
|
||||
"-e", ".ruff_cache", "-e", "node_modules",
|
||||
"-e", ".idea", "-e", ".vscode",
|
||||
"-e", ".trae", "-e", ".qoder",
|
||||
"-e", ".editorconfig", "-e", "idea.config",
|
||||
"-e", "idea_modules.xml", "-e", "vcs.xml"]
|
||||
cli:
|
||||
description: "GitTool - Git 执行工具"
|
||||
usage: "pf gittool <command>"
|
||||
subcommands:
|
||||
a:
|
||||
help: "添加并提交"
|
||||
c:
|
||||
help: "清理并查看状态"
|
||||
i:
|
||||
help: "初始化并提交"
|
||||
isub:
|
||||
help: "初始化子目录"
|
||||
p:
|
||||
help: "推送"
|
||||
pl:
|
||||
help: "拉取"
|
||||
jobs:
|
||||
a:
|
||||
fn: git_add_commit
|
||||
args: ["chore: update"]
|
||||
clean:
|
||||
cmd: ["git", "clean", "-xfd", "${CLEAN_EXCLUDES}"]
|
||||
c:
|
||||
needs: [clean]
|
||||
cmd: ["git", "status", "--porcelain"]
|
||||
i:
|
||||
fn: git_init_add_commit
|
||||
args: ["init commit"]
|
||||
isub:
|
||||
fn: init_sub_dirs
|
||||
p:
|
||||
cmd: ["git", "push"]
|
||||
pl:
|
||||
cmd: ["git", "pull"]
|
||||
@@ -1,51 +0,0 @@
|
||||
# lscalc - LS-DYNA 计算工具
|
||||
# 用法:
|
||||
# pf lscalc run input.k --ncpu 4
|
||||
# pf lscalc status
|
||||
strategy: thread
|
||||
variables:
|
||||
INPUT_FILE: input.k
|
||||
NCPU: 4
|
||||
cli:
|
||||
description: "LSCalc - LS-DYNA 计算工具"
|
||||
usage: "pf lscalc <command> [options]"
|
||||
subcommands:
|
||||
run:
|
||||
help: "运行 LS-DYNA 计算"
|
||||
positional:
|
||||
- name: INPUT_FILE
|
||||
type: str
|
||||
help: "输入文件路径"
|
||||
options:
|
||||
- name: NCPU
|
||||
flag: "--ncpu"
|
||||
type: int
|
||||
default: 4
|
||||
help: "CPU 核心数 (默认: 4)"
|
||||
mpi:
|
||||
help: "运行 LS-DYNA MPI 计算"
|
||||
positional:
|
||||
- name: INPUT_FILE
|
||||
type: str
|
||||
help: "输入文件路径"
|
||||
options:
|
||||
- name: NCPU
|
||||
flag: "--ncpu"
|
||||
type: int
|
||||
default: 4
|
||||
help: "CPU 核心数 (默认: 4)"
|
||||
status:
|
||||
help: "检查 LS-DYNA 进程状态"
|
||||
jobs:
|
||||
run:
|
||||
fn: run_ls_dyna
|
||||
args: ["${INPUT_FILE}"]
|
||||
kwargs:
|
||||
ncpu: ${NCPU}
|
||||
mpi:
|
||||
fn: run_ls_dyna_mpi
|
||||
args: ["${INPUT_FILE}"]
|
||||
kwargs:
|
||||
ncpu: ${NCPU}
|
||||
status:
|
||||
fn: check_ls_dyna_status
|
||||
@@ -1,107 +0,0 @@
|
||||
# packtool - Python 打包工具
|
||||
# 用法:
|
||||
# pf packtool src --project-dir . --output-dir .pypack
|
||||
# pf packtool deps requests numpy --lib-dir libs
|
||||
# pf packtool wheel --project-dir . --output-dir dist
|
||||
# pf packtool embed --version 3.10 --output-dir python
|
||||
# pf packtool zip --source-dir . --output-file package.zip
|
||||
# pf packtool clean
|
||||
strategy: thread
|
||||
variables:
|
||||
PROJECT_DIR: "."
|
||||
OUTPUT_DIR: ".pypack"
|
||||
LIB_DIR: "libs"
|
||||
DEPENDENCIES: []
|
||||
VERSION: "3.10"
|
||||
OUTPUT_FILE: "package.zip"
|
||||
SOURCE_DIR: "."
|
||||
cli:
|
||||
description: "PackTool - Python 打包工具"
|
||||
usage: "pf packtool <command> [options]"
|
||||
subcommands:
|
||||
src:
|
||||
help: "打包源码"
|
||||
options:
|
||||
- name: PROJECT_DIR
|
||||
flag: "--project-dir"
|
||||
type: path
|
||||
default: "."
|
||||
help: "项目目录 (默认: .)"
|
||||
- name: OUTPUT_DIR
|
||||
flag: "--output-dir"
|
||||
type: str
|
||||
default: ".pypack"
|
||||
help: "输出目录 (默认: .pypack)"
|
||||
deps:
|
||||
help: "打包依赖"
|
||||
positional:
|
||||
- name: DEPENDENCIES
|
||||
nargs: "*"
|
||||
type: str
|
||||
help: "依赖包列表"
|
||||
options:
|
||||
- name: LIB_DIR
|
||||
flag: "--lib-dir"
|
||||
type: path
|
||||
default: "libs"
|
||||
help: "依赖库目录 (默认: libs)"
|
||||
wheel:
|
||||
help: "构建 wheel"
|
||||
options:
|
||||
- name: PROJECT_DIR
|
||||
flag: "--project-dir"
|
||||
type: path
|
||||
default: "."
|
||||
help: "项目目录 (默认: .)"
|
||||
- name: OUTPUT_DIR
|
||||
flag: "--output-dir"
|
||||
type: path
|
||||
default: "dist"
|
||||
help: "输出目录 (默认: dist)"
|
||||
embed:
|
||||
help: "安装嵌入式 Python"
|
||||
options:
|
||||
- name: VERSION
|
||||
flag: "--version"
|
||||
type: str
|
||||
default: "3.10"
|
||||
help: "Python 版本 (默认: 3.10)"
|
||||
- name: OUTPUT_DIR
|
||||
flag: "--output-dir"
|
||||
type: path
|
||||
default: "python"
|
||||
help: "输出目录 (默认: python)"
|
||||
zip:
|
||||
help: "创建 zip 包"
|
||||
options:
|
||||
- name: SOURCE_DIR
|
||||
flag: "--source-dir"
|
||||
type: path
|
||||
default: "."
|
||||
help: "源目录 (默认: .)"
|
||||
- name: OUTPUT_FILE
|
||||
flag: "--output-file"
|
||||
type: path
|
||||
default: "package.zip"
|
||||
help: "输出文件 (默认: package.zip)"
|
||||
clean:
|
||||
help: "清理构建目录"
|
||||
jobs:
|
||||
src:
|
||||
fn: pack_source
|
||||
args: ["${PROJECT_DIR}", "${OUTPUT_DIR}"]
|
||||
deps:
|
||||
fn: pack_dependencies
|
||||
args: ["${LIB_DIR}", "${DEPENDENCIES}"]
|
||||
wheel:
|
||||
fn: pack_wheel
|
||||
args: ["${PROJECT_DIR}", "${OUTPUT_DIR}"]
|
||||
embed:
|
||||
fn: install_embed_python
|
||||
args: ["${VERSION}", "${OUTPUT_DIR}"]
|
||||
zip:
|
||||
fn: create_zip_package
|
||||
args: ["${SOURCE_DIR}", "${OUTPUT_FILE}"]
|
||||
clean:
|
||||
fn: clean_build_dir
|
||||
args: ["${OUTPUT_DIR}"]
|
||||
@@ -1,303 +0,0 @@
|
||||
# pdftool - PDF 文件工具集
|
||||
# 用法:
|
||||
# pf pdftool m a.pdf b.pdf --output merged.pdf
|
||||
# pf pdftool s input.pdf --output-dir split
|
||||
# pf pdftool c input.pdf --output compressed.pdf
|
||||
# pf pdftool e input.pdf --output encrypted.pdf --password 123456
|
||||
# pf pdftool d input.pdf --output decrypted.pdf --password 123456
|
||||
# pf pdftool xt input.pdf --output output.txt
|
||||
# pf pdftool xi input.pdf --output-dir images
|
||||
# pf pdftool w input.pdf --output watermarked.pdf --text CONFIDENTIAL
|
||||
# pf pdftool r input.pdf --output rotated.pdf --rotation 90
|
||||
# pf pdftool crop input.pdf --output cropped.pdf --left 10 --top 10 --right 10 --bottom 10
|
||||
# pf pdftool i input.pdf
|
||||
# pf pdftool ocr input.pdf --output ocr.pdf --lang chi_sim+eng
|
||||
# pf pdftool img input.pdf --output-dir images --dpi 300
|
||||
# pf pdftool repair input.pdf --output repaired.pdf
|
||||
strategy: thread
|
||||
variables:
|
||||
INPUT: input.pdf
|
||||
INPUTS: []
|
||||
OUTPUT: output.pdf
|
||||
OUTPUT_DIR: output
|
||||
PASSWORD: ""
|
||||
TEXT: CONFIDENTIAL
|
||||
ROTATION: 90
|
||||
MARGINS: [10, 10, 10, 10]
|
||||
DPI: 300
|
||||
LANG: chi_sim+eng
|
||||
ORDER: []
|
||||
LEFT: 10
|
||||
TOP: 10
|
||||
RIGHT: 10
|
||||
BOTTOM: 10
|
||||
cli:
|
||||
description: "PdfTool - PDF 文件工具集"
|
||||
usage: "pf pdftool <command> [options]"
|
||||
subcommands:
|
||||
m:
|
||||
help: "合并 PDF"
|
||||
positional:
|
||||
- name: INPUTS
|
||||
nargs: "+"
|
||||
type: path
|
||||
help: "输入 PDF 文件列表"
|
||||
options:
|
||||
- name: OUTPUT
|
||||
flag: "--output"
|
||||
type: path
|
||||
default: "merged.pdf"
|
||||
help: "输出文件 (默认: merged.pdf)"
|
||||
s:
|
||||
help: "拆分 PDF"
|
||||
positional:
|
||||
- name: INPUT
|
||||
type: path
|
||||
help: "输入 PDF 文件"
|
||||
options:
|
||||
- name: OUTPUT_DIR
|
||||
flag: "--output-dir"
|
||||
type: path
|
||||
default: "split"
|
||||
help: "输出目录 (默认: split)"
|
||||
c:
|
||||
help: "压缩 PDF"
|
||||
positional:
|
||||
- name: INPUT
|
||||
type: path
|
||||
help: "输入 PDF 文件"
|
||||
options:
|
||||
- name: OUTPUT
|
||||
flag: "--output"
|
||||
type: path
|
||||
default: "compressed.pdf"
|
||||
help: "输出文件 (默认: compressed.pdf)"
|
||||
e:
|
||||
help: "加密 PDF"
|
||||
positional:
|
||||
- name: INPUT
|
||||
type: path
|
||||
help: "输入 PDF 文件"
|
||||
options:
|
||||
- name: OUTPUT
|
||||
flag: "--output"
|
||||
type: path
|
||||
default: "encrypted.pdf"
|
||||
help: "输出文件 (默认: encrypted.pdf)"
|
||||
- name: PASSWORD
|
||||
flag: "--password"
|
||||
type: str
|
||||
required: true
|
||||
help: "密码 (必填)"
|
||||
d:
|
||||
help: "解密 PDF"
|
||||
positional:
|
||||
- name: INPUT
|
||||
type: path
|
||||
help: "输入 PDF 文件"
|
||||
options:
|
||||
- name: OUTPUT
|
||||
flag: "--output"
|
||||
type: path
|
||||
default: "decrypted.pdf"
|
||||
help: "输出文件 (默认: decrypted.pdf)"
|
||||
- name: PASSWORD
|
||||
flag: "--password"
|
||||
type: str
|
||||
required: true
|
||||
help: "密码 (必填)"
|
||||
xt:
|
||||
help: "提取文本"
|
||||
positional:
|
||||
- name: INPUT
|
||||
type: path
|
||||
help: "输入 PDF 文件"
|
||||
options:
|
||||
- name: OUTPUT
|
||||
flag: "--output"
|
||||
type: path
|
||||
default: "output.txt"
|
||||
help: "输出文件 (默认: output.txt)"
|
||||
xi:
|
||||
help: "提取图片"
|
||||
positional:
|
||||
- name: INPUT
|
||||
type: path
|
||||
help: "输入 PDF 文件"
|
||||
options:
|
||||
- name: OUTPUT_DIR
|
||||
flag: "--output-dir"
|
||||
type: path
|
||||
default: "images"
|
||||
help: "输出目录 (默认: images)"
|
||||
w:
|
||||
help: "添加水印"
|
||||
positional:
|
||||
- name: INPUT
|
||||
type: path
|
||||
help: "输入 PDF 文件"
|
||||
options:
|
||||
- name: OUTPUT
|
||||
flag: "--output"
|
||||
type: path
|
||||
default: "watermarked.pdf"
|
||||
help: "输出文件 (默认: watermarked.pdf)"
|
||||
- name: TEXT
|
||||
flag: "--text"
|
||||
type: str
|
||||
default: "CONFIDENTIAL"
|
||||
help: "水印文字 (默认: CONFIDENTIAL)"
|
||||
r:
|
||||
help: "旋转 PDF"
|
||||
positional:
|
||||
- name: INPUT
|
||||
type: path
|
||||
help: "输入 PDF 文件"
|
||||
options:
|
||||
- name: OUTPUT
|
||||
flag: "--output"
|
||||
type: path
|
||||
default: "rotated.pdf"
|
||||
help: "输出文件 (默认: rotated.pdf)"
|
||||
- name: ROTATION
|
||||
flag: "--rotation"
|
||||
type: int
|
||||
default: 90
|
||||
help: "旋转角度 (默认: 90)"
|
||||
crop:
|
||||
help: "裁剪 PDF"
|
||||
positional:
|
||||
- name: INPUT
|
||||
type: path
|
||||
help: "输入 PDF 文件"
|
||||
options:
|
||||
- name: OUTPUT
|
||||
flag: "--output"
|
||||
type: path
|
||||
default: "cropped.pdf"
|
||||
help: "输出文件 (默认: cropped.pdf)"
|
||||
- name: LEFT
|
||||
flag: "--left"
|
||||
type: int
|
||||
default: 10
|
||||
help: "左边距 (默认: 10)"
|
||||
- name: TOP
|
||||
flag: "--top"
|
||||
type: int
|
||||
default: 10
|
||||
help: "上边距 (默认: 10)"
|
||||
- name: RIGHT
|
||||
flag: "--right"
|
||||
type: int
|
||||
default: 10
|
||||
help: "右边距 (默认: 10)"
|
||||
- name: BOTTOM
|
||||
flag: "--bottom"
|
||||
type: int
|
||||
default: 10
|
||||
help: "下边距 (默认: 10)"
|
||||
i:
|
||||
help: "查看 PDF 信息"
|
||||
positional:
|
||||
- name: INPUT
|
||||
type: path
|
||||
help: "输入 PDF 文件"
|
||||
ocr:
|
||||
help: "PDF OCR 识别"
|
||||
positional:
|
||||
- name: INPUT
|
||||
type: path
|
||||
help: "输入 PDF 文件"
|
||||
options:
|
||||
- name: OUTPUT
|
||||
flag: "--output"
|
||||
type: path
|
||||
default: "ocr.pdf"
|
||||
help: "输出文件 (默认: ocr.pdf)"
|
||||
- name: LANG
|
||||
flag: "--lang"
|
||||
type: str
|
||||
default: "chi_sim+eng"
|
||||
help: "识别语言 (默认: chi_sim+eng)"
|
||||
img:
|
||||
help: "PDF 转图片"
|
||||
positional:
|
||||
- name: INPUT
|
||||
type: path
|
||||
help: "输入 PDF 文件"
|
||||
options:
|
||||
- name: OUTPUT_DIR
|
||||
flag: "--output-dir"
|
||||
type: path
|
||||
default: "images"
|
||||
help: "输出目录 (默认: images)"
|
||||
- name: DPI
|
||||
flag: "--dpi"
|
||||
type: int
|
||||
default: 300
|
||||
help: "DPI (默认: 300)"
|
||||
repair:
|
||||
help: "修复 PDF"
|
||||
positional:
|
||||
- name: INPUT
|
||||
type: path
|
||||
help: "输入 PDF 文件"
|
||||
options:
|
||||
- name: OUTPUT
|
||||
flag: "--output"
|
||||
type: path
|
||||
default: "repaired.pdf"
|
||||
help: "输出文件 (默认: repaired.pdf)"
|
||||
jobs:
|
||||
m:
|
||||
fn: pdf_merge
|
||||
args: ["${INPUTS}", "${OUTPUT}"]
|
||||
s:
|
||||
fn: pdf_split
|
||||
args: ["${INPUT}", "${OUTPUT_DIR}"]
|
||||
c:
|
||||
fn: pdf_compress
|
||||
args: ["${INPUT}", "${OUTPUT}"]
|
||||
e:
|
||||
fn: pdf_encrypt
|
||||
args: ["${INPUT}", "${OUTPUT}", "${PASSWORD}"]
|
||||
d:
|
||||
fn: pdf_decrypt
|
||||
args: ["${INPUT}", "${OUTPUT}", "${PASSWORD}"]
|
||||
xt:
|
||||
fn: pdf_extract_text
|
||||
args: ["${INPUT}", "${OUTPUT}"]
|
||||
xi:
|
||||
fn: pdf_extract_images
|
||||
args: ["${INPUT}", "${OUTPUT_DIR}"]
|
||||
w:
|
||||
fn: pdf_add_watermark
|
||||
args: ["${INPUT}", "${OUTPUT}"]
|
||||
kwargs:
|
||||
text: "${TEXT}"
|
||||
r:
|
||||
fn: pdf_rotate
|
||||
args: ["${INPUT}", "${OUTPUT}"]
|
||||
kwargs:
|
||||
rotation: ${ROTATION}
|
||||
crop:
|
||||
fn: pdf_crop
|
||||
args: ["${INPUT}", "${OUTPUT}"]
|
||||
kwargs:
|
||||
margins: "${MARGINS}"
|
||||
i:
|
||||
fn: pdf_info
|
||||
args: ["${INPUT}"]
|
||||
ocr:
|
||||
fn: pdf_ocr
|
||||
args: ["${INPUT}", "${OUTPUT}"]
|
||||
kwargs:
|
||||
lang: "${LANG}"
|
||||
img:
|
||||
fn: pdf_to_images
|
||||
args: ["${INPUT}", "${OUTPUT_DIR}"]
|
||||
kwargs:
|
||||
dpi: ${DPI}
|
||||
repair:
|
||||
fn: pdf_repair
|
||||
args: ["${INPUT}", "${OUTPUT}"]
|
||||
@@ -1,78 +0,0 @@
|
||||
# piptool - pip 包管理工具
|
||||
# 用法:
|
||||
# pf piptool i requests
|
||||
# pf piptool u requests
|
||||
# pf piptool r requests
|
||||
# pf piptool d requests
|
||||
# pf piptool up
|
||||
# pf piptool f
|
||||
strategy: thread
|
||||
variables:
|
||||
PACKAGES: []
|
||||
OFFLINE: false
|
||||
cli:
|
||||
description: "PipTool - pip 包管理工具"
|
||||
usage: "pf piptool <command> [packages...] [options]"
|
||||
subcommands:
|
||||
i:
|
||||
help: "安装包"
|
||||
positional:
|
||||
- name: PACKAGES
|
||||
nargs: "+"
|
||||
type: str
|
||||
help: "包名列表"
|
||||
u:
|
||||
help: "卸载包"
|
||||
positional:
|
||||
- name: PACKAGES
|
||||
nargs: "+"
|
||||
type: str
|
||||
help: "包名列表"
|
||||
r:
|
||||
help: "重装包"
|
||||
positional:
|
||||
- name: PACKAGES
|
||||
nargs: "+"
|
||||
type: str
|
||||
help: "包名列表"
|
||||
options:
|
||||
- name: OFFLINE
|
||||
flag: "--offline"
|
||||
action: "store_true"
|
||||
help: "离线模式"
|
||||
d:
|
||||
help: "下载包"
|
||||
positional:
|
||||
- name: PACKAGES
|
||||
nargs: "+"
|
||||
type: str
|
||||
help: "包名列表"
|
||||
options:
|
||||
- name: OFFLINE
|
||||
flag: "--offline"
|
||||
action: "store_true"
|
||||
help: "离线模式"
|
||||
up:
|
||||
help: "升级 pip"
|
||||
f:
|
||||
help: "导出依赖"
|
||||
jobs:
|
||||
i:
|
||||
cmd: ["pip", "install", "${PACKAGES}"]
|
||||
u:
|
||||
fn: pip_uninstall
|
||||
args: ["${PACKAGES}"]
|
||||
r:
|
||||
fn: pip_reinstall
|
||||
args: ["${PACKAGES}"]
|
||||
kwargs:
|
||||
offline: ${OFFLINE}
|
||||
d:
|
||||
fn: pip_download
|
||||
args: ["${PACKAGES}"]
|
||||
kwargs:
|
||||
offline: ${OFFLINE}
|
||||
up:
|
||||
cmd: ["python", "-m", "pip", "install", "--upgrade", "pip"]
|
||||
f:
|
||||
fn: pip_freeze
|
||||
@@ -1,125 +0,0 @@
|
||||
# pymake - 项目构建工具
|
||||
# 用法
|
||||
# pf pymake <command>
|
||||
# 命令
|
||||
# b: 构建 Python 主包 (uv build)
|
||||
# ba: 构建所有包 (Python + Rust)
|
||||
# bc: 构建 Rust 核心模块 (maturin build)
|
||||
# bump: 升级版本号 (清理 + 检查 + add + bumpversion)
|
||||
# bumpmi: 升级次版本号 (bumpversion minor)
|
||||
# c: 清理构建产物 (调用 gitt c)
|
||||
# cov: 测试并生成覆盖率
|
||||
# doc: 构建 Sphinx 文档
|
||||
# lint: 代码格式化与检查 (ruff)
|
||||
# p: 推送代码 (清理 + push + push tags)
|
||||
# pb: 发布到 PyPI (twine + hatch)
|
||||
# sync: 同步依赖 (uv sync)
|
||||
# t: 运行测试
|
||||
# tc: 类型检查 (pyrefly + ruff)
|
||||
# tf: 快速测试 (无 slow)
|
||||
# tox: 多版本测试 (tox)
|
||||
strategy: thread
|
||||
variables:
|
||||
CWD: "."
|
||||
cli:
|
||||
description: "PyMake - 项目构建工具"
|
||||
usage: "pf pymake <command>"
|
||||
options:
|
||||
- name: CWD
|
||||
flag: "--cwd"
|
||||
type: path
|
||||
required: false
|
||||
default: "."
|
||||
help: "工作目录 (默认: 当前目录)"
|
||||
subcommands:
|
||||
b: {help: "构建 Python 主包 (uv build)"}
|
||||
ba: {help: "构建所有包 (Python + Rust)"}
|
||||
bc: {help: "构建 Rust 核心模块 (maturin build)"}
|
||||
bump: {help: "升级版本号 (清理 + 检查 + add + bumpversion)"}
|
||||
bumpmi: {help: "升级次版本号 (bumpversion minor)"}
|
||||
c: {help: "清理构建产物 (调用 gitt c)"}
|
||||
cov: {help: "测试并生成覆盖率"}
|
||||
doc: {help: "构建 Sphinx 文档"}
|
||||
lint: {help: "代码格式化与检查 (ruff)"}
|
||||
p: {help: "推送代码 (清理 + push + push tags)"}
|
||||
pb: {help: "发布到 PyPI (twine + hatch)"}
|
||||
sync: {help: "同步依赖 (uv sync)"}
|
||||
t: {help: "运行测试"}
|
||||
tc: {help: "类型检查 (pyrefly + ruff)"}
|
||||
tf: {help: "快速测试 (无 slow)"}
|
||||
tox: {help: "多版本测试 (tox)"}
|
||||
jobs:
|
||||
# 单任务别名
|
||||
b:
|
||||
cmd: ["uv", "build"]
|
||||
cwd: ${CWD}
|
||||
bc:
|
||||
cmd: ["maturin", "build", "-r"]
|
||||
cwd: ${CWD}
|
||||
sync:
|
||||
cmd: ["uv", "sync"]
|
||||
cwd: ${CWD}
|
||||
c:
|
||||
cmd: ["pf", "gitt", "c"]
|
||||
cwd: ${CWD}
|
||||
t:
|
||||
cmd: ["pytest", "-m", "not slow", "-n", "8", "--dist", "loadfile", "--color=yes", "--durations=10"]
|
||||
cwd: ${CWD}
|
||||
tf:
|
||||
cmd: ["pytest", "-m", "not slow", "--dist", "loadfile", "--color=yes", "--durations=10"]
|
||||
cwd: ${CWD}
|
||||
bumpversion:
|
||||
cmd: ["bumpversion", "patch"]
|
||||
needs: [git_add_all]
|
||||
cwd: ${CWD}
|
||||
bumpmi:
|
||||
cmd: ["bumpversion", "minor"]
|
||||
cwd: ${CWD}
|
||||
doc:
|
||||
cmd: ["sphinx-build", "-b", "html", "docs", "docs/_build"]
|
||||
cwd: ${CWD}
|
||||
lint:
|
||||
cmd: ["ruff", "check", "--fix", "--unsafe-fixes"]
|
||||
cwd: ${CWD}
|
||||
tox:
|
||||
cmd: ["tox", "-p", "auto"]
|
||||
cwd: ${CWD}
|
||||
|
||||
# 内部 job (不暴露为 subcommand)
|
||||
test_coverage:
|
||||
cmd: ["pytest", "--cov", "-n", "8", "--dist", "loadfile", "--tb=short", "-v", "--color=yes", "--durations=10"]
|
||||
needs: [c]
|
||||
cwd: ${CWD}
|
||||
pyrefly_check:
|
||||
cmd: ["pyrefly", "check", "."]
|
||||
cwd: ${CWD}
|
||||
git_add_all:
|
||||
cmd: ["git", "add", "-A"]
|
||||
needs: [tc]
|
||||
cwd: ${CWD}
|
||||
git_push:
|
||||
cmd: ["git", "push"]
|
||||
cwd: ${CWD}
|
||||
git_push_tags:
|
||||
cmd: ["git", "push", "--tags"]
|
||||
cwd: ${CWD}
|
||||
twine_publish:
|
||||
cmd: ["twine", "upload", "--disable-progress-bar"]
|
||||
cwd: ${CWD}
|
||||
publish_python:
|
||||
cmd: ["hatch", "publish"]
|
||||
cwd: ${CWD}
|
||||
|
||||
# 聚合 job (方向 B: 有 needs 无 cmd/fn)
|
||||
ba:
|
||||
needs: [b, bc]
|
||||
bump:
|
||||
needs: [bumpversion]
|
||||
cov:
|
||||
needs: [test_coverage]
|
||||
tc:
|
||||
needs: [c, pyrefly_check, lint]
|
||||
p:
|
||||
needs: [c, git_push, git_push_tags]
|
||||
pb:
|
||||
needs: [twine_publish, publish_python]
|
||||
@@ -1,13 +0,0 @@
|
||||
# reseticoncache - 重置 Windows 图标缓存
|
||||
# 用法
|
||||
# pf reseticon
|
||||
# 说明
|
||||
# 杀掉 explorer → 删除 IconCache.db → 删除 iconcache* → 重启 explorer
|
||||
# 仅在 Windows 上有效, 非 Windows 平台打印提示并跳过
|
||||
strategy: sequential
|
||||
cli:
|
||||
description: "重置 Windows 图标缓存"
|
||||
usage: "pf reseticon"
|
||||
jobs:
|
||||
reset:
|
||||
fn: reset_icon_cache_run
|
||||
@@ -1,34 +0,0 @@
|
||||
# screenshot - 截图工具
|
||||
# 用法:
|
||||
# pf screenshot full
|
||||
# pf screenshot area --filename custom.png
|
||||
strategy: thread
|
||||
variables:
|
||||
FILENAME: null
|
||||
cli:
|
||||
description: "Screenshot - 截图工具"
|
||||
usage: "pf screenshot <command> [options]"
|
||||
subcommands:
|
||||
full:
|
||||
help: "全屏截图"
|
||||
options:
|
||||
- name: FILENAME
|
||||
flag: "--filename"
|
||||
type: str
|
||||
help: "文件名"
|
||||
area:
|
||||
help: "区域截图"
|
||||
options:
|
||||
- name: FILENAME
|
||||
flag: "--filename"
|
||||
type: str
|
||||
help: "文件名"
|
||||
jobs:
|
||||
full:
|
||||
fn: take_screenshot_full
|
||||
kwargs:
|
||||
filename: "${FILENAME}"
|
||||
area:
|
||||
fn: take_screenshot_area
|
||||
kwargs:
|
||||
filename: "${FILENAME}"
|
||||
@@ -1,49 +0,0 @@
|
||||
# sshcopyid - SSH 密钥部署工具
|
||||
# 用法:
|
||||
# pf sshcopyid hostname username password
|
||||
# pf sshcopyid server user pass --port 2222
|
||||
strategy: thread
|
||||
variables:
|
||||
HOSTNAME: ""
|
||||
USERNAME: ""
|
||||
PASSWORD: ""
|
||||
PORT: 22
|
||||
KEYPATH: "~/.ssh/id_rsa.pub"
|
||||
TIMEOUT: 30
|
||||
cli:
|
||||
description: "SSHCopyID - SSH 密钥部署工具"
|
||||
usage: "pf sshcopyid <hostname> <username> <password> [options]"
|
||||
positional:
|
||||
- name: HOSTNAME
|
||||
type: str
|
||||
help: "远程服务器主机名或 IP 地址"
|
||||
- name: USERNAME
|
||||
type: str
|
||||
help: "远程服务器用户名"
|
||||
- name: PASSWORD
|
||||
type: str
|
||||
help: "远程服务器密码"
|
||||
options:
|
||||
- name: PORT
|
||||
flag: "--port"
|
||||
type: int
|
||||
default: 22
|
||||
help: "SSH 端口 (默认: 22)"
|
||||
- name: KEYPATH
|
||||
flag: "--keypath"
|
||||
type: str
|
||||
default: "~/.ssh/id_rsa.pub"
|
||||
help: "公钥文件路径"
|
||||
- name: TIMEOUT
|
||||
flag: "--timeout"
|
||||
type: int
|
||||
default: 30
|
||||
help: "SSH 操作超时秒数 (默认: 30)"
|
||||
jobs:
|
||||
deploy:
|
||||
fn: ssh_copy_id
|
||||
args: ["${HOSTNAME}", "${USERNAME}", "${PASSWORD}"]
|
||||
kwargs:
|
||||
port: ${PORT}
|
||||
keypath: "${KEYPATH}"
|
||||
timeout: ${TIMEOUT}
|
||||
@@ -16,8 +16,9 @@ DAG 库中泛滥的样板包装器。
|
||||
from __future__ import annotations
|
||||
|
||||
import inspect
|
||||
from collections.abc import Mapping
|
||||
from functools import lru_cache
|
||||
from typing import Any, Mapping
|
||||
from typing import Any
|
||||
|
||||
from .errors import InjectionError
|
||||
from .task import Context, TaskSpec
|
||||
|
||||
@@ -6,7 +6,8 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any, Iterable
|
||||
from collections.abc import Iterable
|
||||
from typing import Any
|
||||
|
||||
|
||||
class PyFlowXError(Exception):
|
||||
|
||||
@@ -1,58 +0,0 @@
|
||||
"""Example 3: async aggregation with static args and Context injection.
|
||||
|
||||
Shows:
|
||||
* async task functions executed with strategy="async".
|
||||
* static positional args (TaskSpec.args) for parameterised tasks.
|
||||
* Context annotation to receive the full upstream result mapping.
|
||||
* on_event callback for real-time progress.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
from typing import Any
|
||||
|
||||
import pyflowx as px
|
||||
|
||||
|
||||
async def fetch_user(uid: int) -> dict[str, Any]:
|
||||
await asyncio.sleep(0.2)
|
||||
return {"id": uid, "name": f"User{uid}"}
|
||||
|
||||
|
||||
async def fetch_posts(uid: int) -> list[int]:
|
||||
await asyncio.sleep(0.2)
|
||||
return [uid, uid + 1]
|
||||
|
||||
|
||||
# Context annotation → receives the full mapping of upstream results.
|
||||
def aggregate(ctx: px.Context) -> dict[str, Any]:
|
||||
return dict(ctx)
|
||||
|
||||
|
||||
def main() -> None:
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
# Static positional args parameterise the same function twice.
|
||||
px.TaskSpec("fetch_user", fetch_user, args=(1,)),
|
||||
px.TaskSpec("fetch_posts", fetch_posts, args=(1,)),
|
||||
px.TaskSpec("aggregate", aggregate, depends_on=("fetch_user", "fetch_posts")),
|
||||
]
|
||||
)
|
||||
|
||||
print("=== Dry run ===")
|
||||
_ = px.run(graph, strategy="async", dry_run=True)
|
||||
|
||||
events: list[px.TaskEvent] = []
|
||||
print("\n=== Async execution ===")
|
||||
report = px.run(graph, strategy="async", on_event=events.append)
|
||||
|
||||
for ev in events:
|
||||
print(f" event: {ev.task} -> {ev.status.value}")
|
||||
|
||||
print(f"\naggregate = {report['aggregate']}")
|
||||
print(report.describe())
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,79 +0,0 @@
|
||||
"""Example 1: ETL pipeline (sequential strategy).
|
||||
|
||||
Demonstrates the core PyFlowX workflow:
|
||||
* Define tasks as plain functions.
|
||||
* Declare the DAG with a list of TaskSpec.
|
||||
* Parameter names == dependency names → automatic context injection,
|
||||
no wrappers needed (contrast with flowweaver's get_task_result boilerplate).
|
||||
* dry_run to preview, then execute and read typed results from RunReport.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
|
||||
import pyflowx as px
|
||||
|
||||
# --- task functions: pure, testable, no framework coupling ------------- #
|
||||
|
||||
|
||||
def extract_customers() -> list[dict[str, Any]]:
|
||||
return [
|
||||
{"id": "C001", "name": "Alice"},
|
||||
{"id": "C002", "name": "Bob"},
|
||||
]
|
||||
|
||||
|
||||
def extract_orders() -> list[dict[str, Any]]:
|
||||
return [
|
||||
{"id": "O001", "customer_id": "C001", "amount": 150.0},
|
||||
{"id": "O002", "customer_id": "C002", "amount": 200.5},
|
||||
]
|
||||
|
||||
|
||||
# Parameter names match dependency names → automatic injection.
|
||||
def transform(
|
||||
extract_customers: list[dict[str, Any]],
|
||||
extract_orders: list[dict[str, Any]],
|
||||
) -> list[dict[str, Any]]:
|
||||
cmap = {c["id"]: c for c in extract_customers}
|
||||
return [{**o, "customer_name": cmap[o["customer_id"]]["name"]} for o in extract_orders if o["customer_id"] in cmap]
|
||||
|
||||
|
||||
def load(transform: list[dict[str, Any]]) -> int:
|
||||
print(f" loaded {len(transform)} records")
|
||||
return len(transform)
|
||||
|
||||
|
||||
def main() -> None:
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("extract_customers", extract_customers, tags=("extract",)),
|
||||
px.TaskSpec("extract_orders", extract_orders, tags=("extract",)),
|
||||
px.TaskSpec(
|
||||
"transform",
|
||||
transform,
|
||||
depends_on=("extract_customers", "extract_orders"),
|
||||
tags=("transform",),
|
||||
),
|
||||
px.TaskSpec(
|
||||
"load", load, depends_on=("transform",), retry=px.RetryPolicy(max_attempts=1, delay=1.0), tags=("load",)
|
||||
),
|
||||
]
|
||||
)
|
||||
|
||||
print("=== Execution plan ===")
|
||||
print(graph.describe())
|
||||
|
||||
print("\n=== Dry run (no execution) ===")
|
||||
_ = px.run(graph, strategy="sequential", dry_run=True)
|
||||
|
||||
print("\n=== Sequential execution ===")
|
||||
report = px.run(graph, strategy="sequential")
|
||||
print(report.describe())
|
||||
print(f"\nload result = {report['load']}")
|
||||
print(f"summary = {report.summary()}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,59 +0,0 @@
|
||||
"""Example 2: parallel execution (thread strategy).
|
||||
|
||||
Same DAG run with sequential vs. thread strategy to show layer-internal
|
||||
parallelism. Tasks within a layer run concurrently; layers are barriers.
|
||||
|
||||
Layer 1: [fetch_a, fetch_b] (parallel)
|
||||
Layer 2: [merge] (waits for both)
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import time
|
||||
|
||||
import pyflowx as px
|
||||
|
||||
|
||||
def fetch_a() -> str:
|
||||
time.sleep(0.5)
|
||||
return "a"
|
||||
|
||||
|
||||
def fetch_b() -> str:
|
||||
time.sleep(0.5)
|
||||
return "b"
|
||||
|
||||
|
||||
def merge(fetch_a: str, fetch_b: str) -> str:
|
||||
return fetch_a + fetch_b
|
||||
|
||||
|
||||
def main() -> None:
|
||||
graph = px.Graph.from_specs(
|
||||
[
|
||||
px.TaskSpec("fetch_a", fetch_a),
|
||||
px.TaskSpec("fetch_b", fetch_b),
|
||||
px.TaskSpec("merge", merge, depends_on=("fetch_a", "fetch_b")),
|
||||
]
|
||||
)
|
||||
|
||||
print("=== Mermaid diagram ===")
|
||||
print(graph.to_mermaid("LR"))
|
||||
|
||||
print("\n=== Sequential (expect ~1.0s) ===")
|
||||
start = time.time()
|
||||
report_seq = px.run(graph, strategy="sequential")
|
||||
t_seq = time.time() - start
|
||||
print(f" result={report_seq['merge']} time={t_seq:.2f}s")
|
||||
|
||||
print("\n=== Threaded (expect ~0.5s) ===")
|
||||
start = time.time()
|
||||
report_thr = px.run(graph, strategy="thread", max_workers=2)
|
||||
t_thr = time.time() - start
|
||||
print(f" result={report_thr['merge']} time={t_thr:.2f}s")
|
||||
|
||||
print(f"\nspeedup = {t_seq / t_thr:.2f}x")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
+17
-12
@@ -48,8 +48,11 @@ import inspect
|
||||
import logging
|
||||
import threading
|
||||
import time
|
||||
from collections.abc import Awaitable, Callable, Mapping
|
||||
from datetime import datetime
|
||||
from typing import Any, Awaitable, Callable, Literal, Mapping, cast
|
||||
from typing import Any, Literal, cast
|
||||
|
||||
from rich.console import Console
|
||||
|
||||
from .context import build_call_args, describe_injection
|
||||
from .errors import TaskFailedError, TaskTimeoutError
|
||||
@@ -60,6 +63,9 @@ from .task import TaskEvent, TaskHooks, TaskResult, TaskSpec, TaskStatus
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# verbose 模式输出用的 Console (不抢占用户 stdout, 仅格式化)
|
||||
_verbose_console = Console()
|
||||
|
||||
# 进程池复用:同一次 run() 内的 process 任务共享一个 ProcessPoolExecutor。
|
||||
# 模块级缓存避免每次任务都创建/销毁进程池的开销。
|
||||
# run() 结束后通过 _shutdown_process_pool() 关闭(shutdown(wait=False) +
|
||||
@@ -666,7 +672,7 @@ class AsyncLayerRunner:
|
||||
return task_ctx, result
|
||||
|
||||
results = await asyncio.gather(*[_run_async_task(name) for name in to_run])
|
||||
for name, (task_ctx, result) in zip(to_run, results):
|
||||
for name, (task_ctx, result) in zip(to_run, results, strict=True):
|
||||
_store_result(name, result, graph.resolved_spec(name), task_ctx, context, report, backend, on_event)
|
||||
|
||||
|
||||
@@ -723,23 +729,22 @@ class DependencyRunner:
|
||||
# 公共 API
|
||||
# ---------------------------------------------------------------------- #
|
||||
def _make_verbose_callback(on_event: EventCallback | None) -> EventCallback:
|
||||
"""包装 on_event 回调, 在 verbose 模式下打印任务生命周期。"""
|
||||
"""包装 on_event 回调, 在 verbose 模式下用 rich 打印任务生命周期。"""
|
||||
|
||||
def _verbose_callback(event: TaskEvent) -> None:
|
||||
dur = f" ({event.duration:.3f}s)" if event.duration is not None else ""
|
||||
if event.status == TaskStatus.RUNNING:
|
||||
print(f"[verbose] 任务 {event.task!r} 开始执行...", flush=True)
|
||||
_verbose_console.print(f"[cyan]▸[/cyan] [bold]{event.task!r}[/bold] 开始执行...")
|
||||
elif event.status == TaskStatus.SUCCESS:
|
||||
print(f"[verbose] 任务 {event.task!r} 成功{dur}", flush=True)
|
||||
_verbose_console.print(f"[green]✓[/green] [bold]{event.task!r}[/bold] 成功[dim]{dur}[/dim]")
|
||||
elif event.status == TaskStatus.FAILED:
|
||||
err = f": {event.error}" if event.error else ""
|
||||
print(
|
||||
f"[verbose] 任务 {event.task!r} 失败{dur} (尝试 {event.attempts} 次){err}",
|
||||
flush=True,
|
||||
_verbose_console.print(
|
||||
f"[red]✗[/red] [bold]{event.task!r}[/bold] 失败[dim]{dur} (尝试 {event.attempts} 次)[/dim][red]{err}[/red]"
|
||||
)
|
||||
elif event.status == TaskStatus.SKIPPED:
|
||||
reason = f" ({event.reason})" if event.reason else ""
|
||||
print(f"[verbose] 任务 {event.task!r} 跳过{reason}", flush=True)
|
||||
_verbose_console.print(f"[yellow]○[/yellow] [bold]{event.task!r}[/bold] 跳过[dim]{reason}[/dim]")
|
||||
if on_event is not None:
|
||||
on_event(event)
|
||||
|
||||
@@ -840,11 +845,11 @@ def run(
|
||||
|
||||
def _print_dry_run(graph: Graph, layers: list[list[str]]) -> None:
|
||||
"""打印执行计划但不运行任何任务。"""
|
||||
print(f"Dry run: {len(graph)} tasks, {len(layers)} layers")
|
||||
_verbose_console.print(f"[bold]Dry run:[/bold] [cyan]{len(graph)}[/cyan] tasks, [cyan]{len(layers)}[/cyan] layers")
|
||||
for idx, layer in enumerate(layers, 1):
|
||||
print(f" Layer {idx}: {layer}")
|
||||
_verbose_console.print(f" [dim]Layer {idx}:[/dim] {layer}")
|
||||
for name in layer:
|
||||
print(f" - {describe_injection(graph.resolved_spec(name))}")
|
||||
_verbose_console.print(f" [cyan]-[/cyan] {describe_injection(graph.resolved_spec(name))}")
|
||||
|
||||
|
||||
def _drive_sequential(
|
||||
|
||||
+7
-46
@@ -1,7 +1,7 @@
|
||||
"""DAG 构建、校验、分层与可视化。
|
||||
|
||||
使用标准库的 :mod:`graphlib`(3.9+)或 :mod:`graphlib_backport`(3.8)
|
||||
进行拓扑排序。图以增量方式构建并即时校验,使配置错误在构建时(而非执行时)快速失败。
|
||||
使用标准库的 :mod:`graphlib` 进行拓扑排序。图以增量方式构建并即时校验,
|
||||
使配置错误在构建时(而非执行时)快速失败。
|
||||
|
||||
支持:
|
||||
* 图级默认值 :class:`GraphDefaults`,TaskSpec 字段为 ``None`` 时回退。
|
||||
@@ -17,23 +17,16 @@ __all__ = [
|
||||
"GraphDefaults",
|
||||
]
|
||||
|
||||
import graphlib
|
||||
import inspect
|
||||
import sys
|
||||
from collections.abc import Callable, Iterable, Mapping, Sequence
|
||||
from dataclasses import dataclass, field, replace
|
||||
from pathlib import Path
|
||||
from typing import Any, Callable, Iterable, Mapping, Sequence
|
||||
from typing import Any
|
||||
|
||||
from .errors import CycleError, DuplicateTaskError, MissingDependencyError
|
||||
from .task import Context, RetryPolicy, TaskSpec
|
||||
|
||||
if sys.version_info >= (3, 9): # pragma: no cover
|
||||
import graphlib # pyright: ignore[reportUnreachable]
|
||||
|
||||
_TopologicalSorter = graphlib.TopologicalSorter
|
||||
else: # pragma: no cover
|
||||
import graphlib # type: ignore[import-untyped]
|
||||
|
||||
_TopologicalSorter = graphlib.TopologicalSorter # pragma: no cover
|
||||
_TopologicalSorter = graphlib.TopologicalSorter
|
||||
|
||||
|
||||
@dataclass
|
||||
@@ -235,38 +228,6 @@ class Graph:
|
||||
graph.validate()
|
||||
return graph
|
||||
|
||||
@classmethod
|
||||
def from_yaml(
|
||||
cls,
|
||||
path: str | Path,
|
||||
variables: Mapping[str, Any] | None = None,
|
||||
) -> Graph:
|
||||
"""从 YAML 文件构建任务图。
|
||||
|
||||
参考 GitHub Actions 风格 schema, 支持 jobs/needs/strategy.matrix/if
|
||||
等 CI/CD 概念。详见 :mod:`pyflowx.yaml_loader`。
|
||||
|
||||
Parameters
|
||||
----------
|
||||
path : str | Path
|
||||
YAML 文件路径
|
||||
variables : Mapping[str, Any] | None
|
||||
运行时变量, 用于替换 ``${VAR}`` 占位符
|
||||
|
||||
Returns
|
||||
-------
|
||||
Graph
|
||||
构建好的任务图
|
||||
|
||||
Raises
|
||||
------
|
||||
YamlLoadError
|
||||
文件不存在、YAML 格式错误、schema 校验失败、循环依赖等
|
||||
"""
|
||||
from .yaml_loader import load_yaml
|
||||
|
||||
return load_yaml(path, variables=variables)
|
||||
|
||||
def add_subgraph(self, sub: Graph, *, namespace: str | None = None) -> Graph:
|
||||
"""将子图合并到当前图,任务名加命名空间前缀避免冲突。
|
||||
|
||||
@@ -326,7 +287,7 @@ class Graph:
|
||||
sorter = _TopologicalSorter(self.deps)
|
||||
try:
|
||||
sorter.prepare()
|
||||
except graphlib.CycleError as exc: # type: ignore[name-defined]
|
||||
except graphlib.CycleError as exc:
|
||||
cycle: Sequence[str] = exc.args[1] if len(exc.args) > 1 else []
|
||||
raise CycleError(list(cycle)) from exc
|
||||
|
||||
|
||||
+11
-12
@@ -1,19 +1,18 @@
|
||||
"""工具函数模块.
|
||||
"""ops 子包 — CLI 工具模块集合 (按功能分组).
|
||||
|
||||
按类别组织 CLI 工具中可复用的函数, 每个子模块使用 ``@px.register_fn`` 注册函数,
|
||||
供 YAML 任务编排通过 ``fn`` 字段引用.
|
||||
每个子模块使用 ``@px.tool`` 装饰器注册 CLI 工具, 由 ``pf`` 入口按需
|
||||
``importlib.import_module`` 触发注册. 子模块按功能分组到子目录.
|
||||
|
||||
子模块
|
||||
子目录
|
||||
------
|
||||
- :mod:`files` —— 文件日期/等级/备份/压缩相关函数
|
||||
- :mod:`dev` —— 开发工具 (ruff/pip/git) 相关函数
|
||||
- :mod:`bumpversion` —— 版本号管理相关函数
|
||||
- :mod:`media` —— PDF/截图相关函数
|
||||
- :mod:`system` —— LS-DYNA/SSH/打包相关函数
|
||||
- :mod:`files` —— 文件与文档操作 (filedate/filelevel/folderback/folderzip/pdftool/screenshot)
|
||||
- :mod:`dev` —— 开发与构建工具 (autofmt/bumpversion/gittool/packtool/piptool/pymake/lscalc)
|
||||
- :mod:`system` —— 系统工具 (clr/reseticoncache/taskkill/which)
|
||||
- :mod:`infra` —— 基础设施与服务部署 (dockercmd/envdev/sshcopyid/msdownload/sglang)
|
||||
|
||||
:mod:`_common` 保留在根目录, 提供跨平台命令选择、平台守卫等共享辅助.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from . import bumpversion, dev, files, media, system
|
||||
|
||||
__all__ = ["bumpversion", "dev", "files", "media", "system"]
|
||||
__all__: list[str] = []
|
||||
|
||||
@@ -0,0 +1,66 @@
|
||||
"""ops 共享辅助: 跨平台命令选择、平台守卫、共享忽略模式."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Sequence
|
||||
|
||||
from pyflowx.conditions import Constants
|
||||
|
||||
__all__ = ["IGNORE_PATTERNS", "ensure_platform", "platform_command"]
|
||||
|
||||
IGNORE_PATTERNS: list[str] = [
|
||||
"__pycache__",
|
||||
"*.pyc",
|
||||
"*.pyo",
|
||||
".git",
|
||||
".venv",
|
||||
".idea",
|
||||
".vscode",
|
||||
"*.egg-info",
|
||||
"dist",
|
||||
"build",
|
||||
".pytest_cache",
|
||||
".tox",
|
||||
".mypy_cache",
|
||||
]
|
||||
|
||||
|
||||
def platform_command(windows: Sequence[str], posix: Sequence[str]) -> list[str]:
|
||||
"""按当前平台返回对应命令.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
windows : Sequence[str]
|
||||
Windows 平台命令 (含参数).
|
||||
posix : Sequence[str]
|
||||
Linux/macOS 平台命令 (含参数).
|
||||
|
||||
Returns
|
||||
-------
|
||||
list[str]
|
||||
当前平台对应的命令列表 (副本, 可安全修改).
|
||||
"""
|
||||
return list(windows) if Constants.IS_WINDOWS else list(posix)
|
||||
|
||||
|
||||
def ensure_platform(predicate: bool, tool_name: str, platform_name: str) -> bool:
|
||||
"""平台不满足时打印提示, 返回是否满足.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
predicate : bool
|
||||
平台匹配谓词 (如 ``Constants.IS_WINDOWS``).
|
||||
tool_name : str
|
||||
工具名称, 用于提示信息前缀.
|
||||
platform_name : str
|
||||
目标平台名称, 用于提示信息.
|
||||
|
||||
Returns
|
||||
-------
|
||||
bool
|
||||
平台满足时返回 True, 否则打印 ``{tool_name}: 仅在 {platform_name} 上支持`` 并返回 False.
|
||||
"""
|
||||
if not predicate:
|
||||
print(f"{tool_name}: 仅在 {platform_name} 上支持")
|
||||
return False
|
||||
return True
|
||||
@@ -1,433 +0,0 @@
|
||||
"""开发工具类函数模块.
|
||||
|
||||
聚合自动格式化 (autofmt)、pip 包管理 (piptool)、git 工具 (gittool) 的可复用函数.
|
||||
版本号管理已抽离到 :mod:`pyflowx.ops.bumpversion`. 所有公共函数通过
|
||||
``@px.register_fn`` 注册, 供 YAML 任务编排引用.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import ast
|
||||
import fnmatch
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
import pyflowx as px
|
||||
|
||||
__all__ = [
|
||||
"IGNORE_PATTERNS",
|
||||
"PACKAGE_DIR",
|
||||
"REQUIREMENTS_FILE",
|
||||
"_PROTECTED_PACKAGES",
|
||||
"add_docstring",
|
||||
"auto_add_docstrings",
|
||||
"format_all",
|
||||
"format_with_ruff",
|
||||
"generate_module_docstring",
|
||||
"git_add_commit",
|
||||
"git_init_add_commit",
|
||||
"has_files",
|
||||
"init_sub_dirs",
|
||||
"lint_with_ruff",
|
||||
"not_has_git_repo",
|
||||
"pip_download",
|
||||
"pip_freeze",
|
||||
"pip_reinstall",
|
||||
"pip_uninstall",
|
||||
"sync_pyproject_config",
|
||||
]
|
||||
|
||||
# ============================================================================
|
||||
# autofmt 配置
|
||||
# ============================================================================
|
||||
|
||||
IGNORE_PATTERNS = [
|
||||
"__pycache__",
|
||||
"*.pyc",
|
||||
"*.pyo",
|
||||
".git",
|
||||
".venv",
|
||||
".idea",
|
||||
".vscode",
|
||||
"*.egg-info",
|
||||
"dist",
|
||||
"build",
|
||||
".pytest_cache",
|
||||
".tox",
|
||||
".mypy_cache",
|
||||
]
|
||||
|
||||
# ============================================================================
|
||||
# piptool 配置
|
||||
# ============================================================================
|
||||
|
||||
PACKAGE_DIR = "packages"
|
||||
REQUIREMENTS_FILE = "requirements.txt"
|
||||
|
||||
_PROTECTED_PACKAGES: frozenset[str] = frozenset(
|
||||
{
|
||||
"pyflowx",
|
||||
"bitool",
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# autofmt 私有辅助函数
|
||||
# ============================================================================
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# autofmt 函数
|
||||
# ============================================================================
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def format_with_ruff(target: Path, fix: bool = True) -> None:
|
||||
"""使用 ruff 格式化代码.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
target : Path
|
||||
目标路径
|
||||
fix : bool
|
||||
是否自动修复
|
||||
"""
|
||||
cmd = ["ruff", "format", str(target)]
|
||||
if fix:
|
||||
cmd.append("--fix")
|
||||
|
||||
subprocess.run(cmd, check=True)
|
||||
print(f"ruff format 完成: {target}")
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def lint_with_ruff(target: Path, fix: bool = True) -> None:
|
||||
"""使用 ruff 检查代码.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
target : Path
|
||||
目标路径
|
||||
fix : bool
|
||||
是否自动修复
|
||||
"""
|
||||
cmd = ["ruff", "check", str(target)]
|
||||
if fix:
|
||||
cmd.extend(["--fix", "--unsafe-fixes"])
|
||||
|
||||
subprocess.run(cmd, check=True)
|
||||
print(f"ruff check 完成: {target}")
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def add_docstring(file_path: Path, docstring: str) -> bool:
|
||||
"""为文件添加 docstring.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
file_path : Path
|
||||
文件路径
|
||||
docstring : str
|
||||
docstring 内容
|
||||
|
||||
Returns
|
||||
-------
|
||||
bool
|
||||
是否成功添加
|
||||
"""
|
||||
try:
|
||||
content = file_path.read_text(encoding="utf-8")
|
||||
tree = ast.parse(content)
|
||||
|
||||
first_node = tree.body[0] if tree.body else None
|
||||
if first_node and isinstance(first_node, ast.Expr) and isinstance(first_node.value, ast.Constant):
|
||||
return False
|
||||
|
||||
lines = content.splitlines()
|
||||
doc_lines = docstring.splitlines()
|
||||
doc_lines.append("")
|
||||
new_content = "\n".join(doc_lines + lines)
|
||||
|
||||
file_path.write_text(new_content, encoding="utf-8")
|
||||
print(f"添加 docstring: {file_path}")
|
||||
return True
|
||||
|
||||
except (OSError, UnicodeDecodeError, SyntaxError) as e:
|
||||
print(f"处理失败: {file_path} - {e}")
|
||||
return False
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def generate_module_docstring(file_path: Path) -> str:
|
||||
"""生成模块 docstring.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
file_path : Path
|
||||
文件路径
|
||||
|
||||
Returns
|
||||
-------
|
||||
str
|
||||
生成的 docstring
|
||||
"""
|
||||
stem = file_path.stem
|
||||
parent = file_path.parent.name
|
||||
|
||||
keywords = {
|
||||
"cli": f"Command-line interface for {parent}",
|
||||
"gui": f"Graphical user interface for {parent}",
|
||||
"core": f"Core functionality for {parent}",
|
||||
"util": f"Utility functions for {parent}",
|
||||
"model": f"Data models for {parent}",
|
||||
"test": f"Tests for {parent}",
|
||||
}
|
||||
|
||||
for key, desc in keywords.items():
|
||||
if key in stem.lower():
|
||||
return f'"""{desc}."""'
|
||||
|
||||
return f'"""{stem.replace("_", " ").title()} module."""'
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def auto_add_docstrings(root_dir: Path) -> int:
|
||||
"""自动为所有 Python 文件添加 docstring.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
root_dir : Path
|
||||
根目录
|
||||
|
||||
Returns
|
||||
-------
|
||||
int
|
||||
添加的 docstring 数量
|
||||
"""
|
||||
count = 0
|
||||
for py_file in root_dir.rglob("*.py"):
|
||||
if any(pattern in str(py_file) for pattern in IGNORE_PATTERNS):
|
||||
continue
|
||||
|
||||
docstring = generate_module_docstring(py_file)
|
||||
if add_docstring(py_file, docstring):
|
||||
count += 1
|
||||
|
||||
print(f"共添加 {count} 个 docstring")
|
||||
return count
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def sync_pyproject_config(root_dir: Path) -> None:
|
||||
"""同步 pyproject.toml 配置到子项目.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
root_dir : Path
|
||||
根目录
|
||||
"""
|
||||
main_toml = root_dir / "pyproject.toml"
|
||||
if not main_toml.exists():
|
||||
print(f"主项目配置文件不存在: {main_toml}")
|
||||
return
|
||||
|
||||
sub_tomls = [p for p in root_dir.rglob("pyproject.toml") if p != main_toml and ".venv" not in str(p)]
|
||||
|
||||
if not sub_tomls:
|
||||
print("没有找到子项目的 pyproject.toml")
|
||||
return
|
||||
|
||||
print(f"找到 {len(sub_tomls)} 个子项目配置文件")
|
||||
|
||||
for sub_toml in sub_tomls:
|
||||
subprocess.run(["ruff", "format", str(sub_toml)], check=False)
|
||||
|
||||
print("配置同步完成")
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def format_all(root_dir: Path) -> None:
|
||||
"""格式化所有 Python 文件.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
root_dir : Path
|
||||
根目录
|
||||
"""
|
||||
subprocess.run(["ruff", "format", str(root_dir)], check=True)
|
||||
subprocess.run(["ruff", "check", "--fix", "--unsafe-fixes", str(root_dir)], check=True)
|
||||
print(f"格式化完成: {root_dir}")
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# piptool 私有辅助函数
|
||||
# ============================================================================
|
||||
|
||||
|
||||
def _get_installed_packages() -> list[str]:
|
||||
"""获取当前环境中所有已安装的包名."""
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["pip", "list", "--format=freeze"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=True,
|
||||
)
|
||||
packages: list[str] = []
|
||||
for line in result.stdout.strip().split("\n"):
|
||||
if line and "==" in line:
|
||||
pkg_name = line.split("==")[0].strip()
|
||||
packages.append(pkg_name)
|
||||
except (subprocess.SubprocessError, OSError):
|
||||
return []
|
||||
return packages
|
||||
|
||||
|
||||
def _expand_wildcard_packages(pattern: str) -> list[str]:
|
||||
"""展开通配符模式为实际的包名列表."""
|
||||
if not any(char in pattern for char in ["*", "?", "[", "]"]):
|
||||
return [pattern]
|
||||
|
||||
installed_packages = _get_installed_packages()
|
||||
matched = [pkg for pkg in installed_packages if fnmatch.fnmatchcase(pkg.lower(), pattern.lower())]
|
||||
return matched
|
||||
|
||||
|
||||
def _filter_protected_packages(packages: list[str]) -> list[str]:
|
||||
"""过滤掉受保护的包名."""
|
||||
safe = [p for p in packages if p.lower() not in {p.lower() for p in _PROTECTED_PACKAGES}]
|
||||
filtered = [p for p in packages if p.lower() in {p.lower() for p in _PROTECTED_PACKAGES}]
|
||||
if filtered:
|
||||
print(f"跳过受保护的包: {', '.join(filtered)}")
|
||||
return safe
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# piptool 函数
|
||||
# ============================================================================
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def pip_uninstall(pkg_names: list[str]) -> None:
|
||||
"""卸载包."""
|
||||
packages_to_uninstall: list[str] = []
|
||||
for pattern in pkg_names:
|
||||
packages_to_uninstall.extend(_expand_wildcard_packages(pattern))
|
||||
|
||||
packages_to_uninstall = _filter_protected_packages(packages_to_uninstall)
|
||||
|
||||
if not packages_to_uninstall:
|
||||
return
|
||||
|
||||
subprocess.run(["pip", "uninstall", "-y", *packages_to_uninstall], check=True)
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def pip_reinstall(pkg_names: list[str], offline: bool = False) -> None:
|
||||
"""重新安装包."""
|
||||
safe_ps = _filter_protected_packages(pkg_names)
|
||||
if not safe_ps:
|
||||
print("所有指定的包均为受保护包, 跳过重装")
|
||||
return
|
||||
|
||||
subprocess.run(["pip", "uninstall", "-y", *safe_ps], check=True)
|
||||
|
||||
options = ["--no-index", "--find-links", "."] if offline else []
|
||||
subprocess.run(["pip", "install", *options, *safe_ps], check=True)
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def pip_download(pkg_names: list[str], offline: bool = False) -> None:
|
||||
"""下载包."""
|
||||
options = ["--no-index", "--find-links", "."] if offline else []
|
||||
subprocess.run(
|
||||
["pip", "download", *pkg_names, *options, "-d", PACKAGE_DIR],
|
||||
check=True,
|
||||
)
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def pip_freeze() -> None:
|
||||
"""冻结依赖."""
|
||||
result = subprocess.run(
|
||||
["pip", "freeze", "--exclude-editable"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=True,
|
||||
)
|
||||
Path(REQUIREMENTS_FILE).write_text(result.stdout)
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# gittool 函数
|
||||
# ============================================================================
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def init_sub_dirs() -> None:
|
||||
"""初始化子目录的 Git 仓库."""
|
||||
sub_dirs = [subdir for subdir in Path.cwd().iterdir() if subdir.is_dir()]
|
||||
for subdir in sub_dirs:
|
||||
px.run(
|
||||
px.Graph().chain(
|
||||
px.cmd(["git", "init"], conditions=(lambda _: not_has_git_repo(),), cwd=subdir),
|
||||
px.cmd(["git", "add", "."]),
|
||||
px.cmd(["git", "commit", "-m", "init commit"]),
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def not_has_git_repo() -> bool:
|
||||
"""检查当前目录没有 Git 仓库."""
|
||||
return not Path.cwd().exists() or not (Path.cwd() / ".git").is_dir()
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def has_files() -> bool:
|
||||
"""检查当前 Git 仓库是否有未提交的更改."""
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["git", "status", "--porcelain"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
return bool(result.stdout.strip())
|
||||
except (subprocess.SubprocessError, OSError):
|
||||
return False
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def git_add_commit(message: str = "chore: update") -> None:
|
||||
"""执行 git add + git commit (仅当有未提交更改时).
|
||||
|
||||
Parameters
|
||||
----------
|
||||
message : str
|
||||
提交信息
|
||||
"""
|
||||
if not has_files():
|
||||
print("没有文件需要提交")
|
||||
return
|
||||
subprocess.run(["git", "add", "."], check=True)
|
||||
subprocess.run(["git", "commit", "-m", message], check=True)
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def git_init_add_commit(message: str = "init commit") -> None:
|
||||
"""执行 git init (若需) + git add + git commit (若有更改).
|
||||
|
||||
Parameters
|
||||
----------
|
||||
message : str
|
||||
提交信息
|
||||
"""
|
||||
if not_has_git_repo():
|
||||
subprocess.run(["git", "init"], check=True)
|
||||
if has_files():
|
||||
subprocess.run(["git", "add", "."], check=True)
|
||||
subprocess.run(["git", "commit", "-m", message], check=True)
|
||||
else:
|
||||
print("没有文件需要提交")
|
||||
@@ -0,0 +1,5 @@
|
||||
"""dev 子包 — 开发与构建工具."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
__all__: list[str] = []
|
||||
@@ -0,0 +1,228 @@
|
||||
"""autofmt - 自动格式化工具.
|
||||
|
||||
提供格式化代码/代码检查/自动添加文档/同步配置 子命令.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import ast
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
import pyflowx as px
|
||||
from pyflowx.ops._common import IGNORE_PATTERNS
|
||||
|
||||
__all__ = [
|
||||
"add_docstring",
|
||||
"auto_add_docstrings",
|
||||
"fmt",
|
||||
"format_all",
|
||||
"format_with_ruff",
|
||||
"generate_module_docstring",
|
||||
"lint",
|
||||
"lint_with_ruff",
|
||||
"sync_pyproject_config",
|
||||
]
|
||||
|
||||
# ============================================================================
|
||||
# 辅助函数
|
||||
# ============================================================================
|
||||
|
||||
|
||||
@px.tool("autofmt", subcommand="fmt", help="格式化代码")
|
||||
def fmt(target: str = ".") -> None:
|
||||
"""格式化代码 (ruff format).
|
||||
|
||||
Parameters
|
||||
----------
|
||||
target : str
|
||||
目标路径 (默认: .)
|
||||
"""
|
||||
format_with_ruff(Path(target), fix=False)
|
||||
|
||||
|
||||
@px.tool("autofmt", subcommand="lint", help="代码检查")
|
||||
def lint(target: str = ".", fix: bool = False) -> None:
|
||||
"""代码检查 (ruff check).
|
||||
|
||||
Parameters
|
||||
----------
|
||||
target : str
|
||||
目标路径 (默认: .)
|
||||
fix : bool
|
||||
自动修复问题
|
||||
"""
|
||||
lint_with_ruff(Path(target), fix=fix)
|
||||
|
||||
|
||||
def format_with_ruff(target: Path, fix: bool = True) -> None:
|
||||
"""使用 ruff 格式化代码.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
target : Path
|
||||
目标路径
|
||||
fix : bool
|
||||
是否自动修复
|
||||
"""
|
||||
cmd = ["ruff", "format", str(target)]
|
||||
if fix:
|
||||
cmd.append("--fix")
|
||||
|
||||
subprocess.run(cmd, check=True)
|
||||
print(f"ruff format 完成: {target}")
|
||||
|
||||
|
||||
def lint_with_ruff(target: Path, fix: bool = True) -> None:
|
||||
"""使用 ruff 检查代码.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
target : Path
|
||||
目标路径
|
||||
fix : bool
|
||||
是否自动修复
|
||||
"""
|
||||
cmd = ["ruff", "check", str(target)]
|
||||
if fix:
|
||||
cmd.extend(["--fix", "--unsafe-fixes"])
|
||||
|
||||
subprocess.run(cmd, check=True)
|
||||
print(f"ruff check 完成: {target}")
|
||||
|
||||
|
||||
def add_docstring(file_path: Path, docstring: str) -> bool:
|
||||
"""为文件添加 docstring.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
file_path : Path
|
||||
文件路径
|
||||
docstring : str
|
||||
docstring 内容
|
||||
|
||||
Returns
|
||||
-------
|
||||
bool
|
||||
是否成功添加
|
||||
"""
|
||||
try:
|
||||
content = file_path.read_text(encoding="utf-8")
|
||||
tree = ast.parse(content)
|
||||
|
||||
first_node = tree.body[0] if tree.body else None
|
||||
if first_node and isinstance(first_node, ast.Expr) and isinstance(first_node.value, ast.Constant):
|
||||
return False
|
||||
|
||||
lines = content.splitlines()
|
||||
doc_lines = docstring.splitlines()
|
||||
doc_lines.append("")
|
||||
new_content = "\n".join(doc_lines + lines)
|
||||
|
||||
file_path.write_text(new_content, encoding="utf-8")
|
||||
print(f"添加 docstring: {file_path}")
|
||||
return True
|
||||
|
||||
except (OSError, UnicodeDecodeError, SyntaxError) as e:
|
||||
print(f"处理失败: {file_path} - {e}")
|
||||
return False
|
||||
|
||||
|
||||
def generate_module_docstring(file_path: Path) -> str:
|
||||
"""生成模块 docstring.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
file_path : Path
|
||||
文件路径
|
||||
|
||||
Returns
|
||||
-------
|
||||
str
|
||||
生成的 docstring
|
||||
"""
|
||||
stem = file_path.stem
|
||||
parent = file_path.parent.name
|
||||
|
||||
keywords = {
|
||||
"cli": f"Command-line interface for {parent}",
|
||||
"gui": f"Graphical user interface for {parent}",
|
||||
"core": f"Core functionality for {parent}",
|
||||
"util": f"Utility functions for {parent}",
|
||||
"model": f"Data models for {parent}",
|
||||
"test": f"Tests for {parent}",
|
||||
}
|
||||
|
||||
for key, desc in keywords.items():
|
||||
if key in stem.lower():
|
||||
return f'"""{desc}."""'
|
||||
|
||||
return f'"""{stem.replace("_", " ").title()} module."""'
|
||||
|
||||
|
||||
@px.tool("autofmt", subcommand="doc", help="自动添加文档字符串")
|
||||
def auto_add_docstrings(root_dir: Path = Path()) -> int:
|
||||
"""自动为所有 Python 文件添加 docstring.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
root_dir : Path
|
||||
根目录 (默认: 当前目录)
|
||||
|
||||
Returns
|
||||
-------
|
||||
int
|
||||
添加的 docstring 数量
|
||||
"""
|
||||
count = 0
|
||||
for py_file in root_dir.rglob("*.py"):
|
||||
if any(pattern in str(py_file) for pattern in IGNORE_PATTERNS):
|
||||
continue
|
||||
|
||||
docstring = generate_module_docstring(py_file)
|
||||
if add_docstring(py_file, docstring):
|
||||
count += 1
|
||||
|
||||
print(f"共添加 {count} 个 docstring")
|
||||
return count
|
||||
|
||||
|
||||
@px.tool("autofmt", subcommand="sync", help="同步 pyproject 配置")
|
||||
def sync_pyproject_config(root_dir: Path = Path()) -> None:
|
||||
"""同步 pyproject.toml 配置到子项目.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
root_dir : Path
|
||||
根目录 (默认: 当前目录)
|
||||
"""
|
||||
main_toml = root_dir / "pyproject.toml"
|
||||
if not main_toml.exists():
|
||||
print(f"主项目配置文件不存在: {main_toml}")
|
||||
return
|
||||
|
||||
sub_tomls = [p for p in root_dir.rglob("pyproject.toml") if p != main_toml and ".venv" not in str(p)]
|
||||
|
||||
if not sub_tomls:
|
||||
print("没有找到子项目的 pyproject.toml")
|
||||
return
|
||||
|
||||
print(f"找到 {len(sub_tomls)} 个子项目配置文件")
|
||||
|
||||
for sub_toml in sub_tomls:
|
||||
subprocess.run(["ruff", "format", str(sub_toml)], check=False)
|
||||
|
||||
print("配置同步完成")
|
||||
|
||||
|
||||
def format_all(root_dir: Path) -> None:
|
||||
"""格式化所有 Python 文件.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
root_dir : Path
|
||||
根目录
|
||||
"""
|
||||
subprocess.run(["ruff", "format", str(root_dir)], check=True)
|
||||
subprocess.run(["ruff", "check", "--fix", "--unsafe-fixes", str(root_dir)], check=True)
|
||||
print(f"格式化完成: {root_dir}")
|
||||
@@ -1,8 +1,8 @@
|
||||
"""版本号管理模块.
|
||||
|
||||
提供单文件版本号更新 (``bump_file_version``) 与项目级批量版本号同步
|
||||
(``bump_project_version``) 能力. 所有公共函数通过 ``@px.register_fn`` 注册,
|
||||
供 YAML 任务编排引用.
|
||||
(``bump_project_version``) 能力. ``bump_project_version`` 通过 ``@px.tool``
|
||||
注册为 CLI 工具, 可通过 ``pf bumpversion`` 调用.
|
||||
|
||||
设计要点
|
||||
--------
|
||||
@@ -131,7 +131,6 @@ def _write_version_to_file(file_path: Path, new_version: str) -> bool:
|
||||
# ============================================================================
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def bump_file_version(file_path: Path, part: BumpVersionType = "patch") -> str | None:
|
||||
"""更新单个文件中的版本号.
|
||||
|
||||
@@ -163,7 +162,7 @@ def bump_file_version(file_path: Path, part: BumpVersionType = "patch") -> str |
|
||||
return new_version
|
||||
|
||||
|
||||
@px.register_fn
|
||||
@px.tool("bumpversion", help="版本号自动管理")
|
||||
def bump_project_version(part: BumpVersionType = "patch", no_tag: bool = False) -> str | None:
|
||||
"""批量同步项目所有版本号文件并提交.
|
||||
|
||||
@@ -0,0 +1,142 @@
|
||||
"""gittool - Git 执行工具.
|
||||
|
||||
提供添加提交/清理/初始化/推送/拉取 子命令.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
import pyflowx as px
|
||||
|
||||
EXCLUDE_DIRS = [
|
||||
# 编辑器相关目录
|
||||
".vscode",
|
||||
".idea",
|
||||
".editorconfig",
|
||||
".trae",
|
||||
".qoder",
|
||||
# 项目相关目录
|
||||
".venv",
|
||||
".git",
|
||||
".ruff_cache",
|
||||
".tox",
|
||||
"node_modules",
|
||||
]
|
||||
|
||||
EXCLUDE_CMDS = [arg for d in EXCLUDE_DIRS for arg in ["-e", d]]
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# 私有辅助函数
|
||||
# ============================================================================
|
||||
|
||||
|
||||
def not_has_git_repo() -> bool:
|
||||
"""检查当前目录没有 Git 仓库."""
|
||||
return not Path.cwd().exists() or not (Path.cwd() / ".git").is_dir()
|
||||
|
||||
|
||||
def has_files() -> bool:
|
||||
"""检查当前 Git 仓库是否有未提交的更改."""
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["git", "status", "--porcelain"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
return bool(result.stdout.strip())
|
||||
except (subprocess.SubprocessError, OSError):
|
||||
return False
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# fn 子命令
|
||||
# ============================================================================
|
||||
|
||||
|
||||
@px.tool("gittool", subcommand="a", help="添加并提交")
|
||||
def git_add_commit(message: str = "chore: update") -> None:
|
||||
"""执行 git add + git commit (仅当有未提交更改时).
|
||||
|
||||
Parameters
|
||||
----------
|
||||
message : str
|
||||
提交信息 (默认: ``chore: update``)
|
||||
"""
|
||||
if not has_files():
|
||||
print("没有文件需要提交")
|
||||
return
|
||||
subprocess.run(["git", "add", "."], check=True)
|
||||
subprocess.run(["git", "commit", "-m", message], check=True)
|
||||
|
||||
|
||||
@px.tool("gittool", subcommand="i", help="初始化并提交")
|
||||
def git_init_add_commit(message: str = "init commit") -> None:
|
||||
"""执行 git init (若需) + git add + git commit (若有更改).
|
||||
|
||||
Parameters
|
||||
----------
|
||||
message : str
|
||||
提交信息 (默认: ``init commit``)
|
||||
"""
|
||||
if not_has_git_repo():
|
||||
subprocess.run(["git", "init"], check=True)
|
||||
if has_files():
|
||||
subprocess.run(["git", "add", "."], check=True)
|
||||
subprocess.run(["git", "commit", "-m", message], check=True)
|
||||
else:
|
||||
print("没有文件需要提交")
|
||||
|
||||
|
||||
@px.tool("gittool", subcommand="isub", help="初始化子目录")
|
||||
def init_sub_dirs() -> None:
|
||||
"""初始化子目录的 Git 仓库."""
|
||||
sub_dirs = [subdir for subdir in Path.cwd().iterdir() if subdir.is_dir()]
|
||||
for subdir in sub_dirs:
|
||||
px.run(
|
||||
px.Graph().chain(
|
||||
px.cmd(["git", "init"], conditions=(lambda _: not_has_git_repo(),), cwd=subdir),
|
||||
px.cmd(["git", "add", "."]),
|
||||
px.cmd(["git", "commit", "-m", "init commit"]),
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# cmd 子命令
|
||||
# ============================================================================
|
||||
|
||||
|
||||
@px.tool(
|
||||
"gittool",
|
||||
subcommand="clean",
|
||||
help="清理未跟踪文件",
|
||||
cmd=["git", "clean", "-xfd", *EXCLUDE_CMDS],
|
||||
hidden=True,
|
||||
)
|
||||
def clean() -> None:
|
||||
"""清理 Git 未跟踪文件 (隐藏命令, 被 c 依赖)."""
|
||||
|
||||
|
||||
@px.tool(
|
||||
"gittool",
|
||||
subcommand="c",
|
||||
help="清理并查看状态",
|
||||
cmd=["git", "status", "--porcelain"],
|
||||
needs=["clean"],
|
||||
)
|
||||
def c() -> None:
|
||||
"""清理未跟踪文件并查看 Git 状态."""
|
||||
|
||||
|
||||
@px.tool("gittool", subcommand="p", help="推送", cmd=["git", "push"])
|
||||
def p() -> None:
|
||||
"""推送代码到远程仓库."""
|
||||
|
||||
|
||||
@px.tool("gittool", subcommand="pl", help="拉取", cmd=["git", "pull"])
|
||||
def pl() -> None:
|
||||
"""从远程仓库拉取代码."""
|
||||
@@ -0,0 +1,111 @@
|
||||
"""lscalc - LS-DYNA 计算工具.
|
||||
|
||||
运行 LS-DYNA 计算 (单机/MPI) 与进程状态检查.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
import pyflowx as px
|
||||
from pyflowx.conditions import Constants
|
||||
|
||||
_DEFAULT_NCPU: int = 4
|
||||
|
||||
|
||||
def get_ls_dyna_command(input_file: str, ncpu: int) -> list[str]:
|
||||
"""获取 LS-DYNA 命令.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
input_file : str
|
||||
输入文件路径
|
||||
ncpu : int
|
||||
CPU 核心数
|
||||
|
||||
Returns
|
||||
-------
|
||||
list[str]
|
||||
LS-DYNA 命令列表
|
||||
"""
|
||||
return ["ls-dyna_mpp", f"i={input_file}", f"ncpu={ncpu}"]
|
||||
|
||||
|
||||
@px.tool("lscalc", subcommand="run", help="运行 LS-DYNA 计算")
|
||||
def run_ls_dyna(input_file: str, ncpu: int = _DEFAULT_NCPU) -> None:
|
||||
"""运行 LS-DYNA 计算.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
input_file : str
|
||||
输入文件路径
|
||||
ncpu : int
|
||||
CPU 核心数
|
||||
"""
|
||||
input_path = Path(input_file)
|
||||
if not input_path.exists():
|
||||
print(f"输入文件不存在: {input_path}")
|
||||
return
|
||||
|
||||
cmd = get_ls_dyna_command(input_file, ncpu)
|
||||
try:
|
||||
subprocess.run(cmd, check=True)
|
||||
print(f"LS-DYNA 计算完成: {input_file}")
|
||||
except FileNotFoundError:
|
||||
print("未找到 ls-dyna_mpp 命令")
|
||||
except subprocess.CalledProcessError as e:
|
||||
print(f"LS-DYNA 计算失败: {e}")
|
||||
|
||||
|
||||
@px.tool("lscalc", subcommand="mpi", help="运行 LS-DYNA MPI 计算")
|
||||
def run_ls_dyna_mpi(input_file: str, ncpu: int = _DEFAULT_NCPU) -> None:
|
||||
"""运行 LS-DYNA MPI 计算.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
input_file : str
|
||||
输入文件路径
|
||||
ncpu : int
|
||||
CPU 核心数
|
||||
"""
|
||||
input_path = Path(input_file)
|
||||
if not input_path.exists():
|
||||
print(f"输入文件不存在: {input_path}")
|
||||
return
|
||||
|
||||
cmd = ["mpirun", "-np", str(ncpu), "ls-dyna_mpp", f"i={input_file}"]
|
||||
try:
|
||||
subprocess.run(cmd, check=True)
|
||||
print(f"LS-DYNA MPI 计算完成: {input_file}")
|
||||
except FileNotFoundError:
|
||||
print("未找到 mpirun 或 ls-dyna_mpp 命令")
|
||||
except subprocess.CalledProcessError as e:
|
||||
print(f"LS-DYNA MPI 计算失败: {e}")
|
||||
|
||||
|
||||
@px.tool("lscalc", subcommand="status", help="检查 LS-DYNA 进程状态")
|
||||
def check_ls_dyna_status() -> None:
|
||||
"""检查 LS-DYNA 进程状态."""
|
||||
try:
|
||||
if Constants.IS_WINDOWS:
|
||||
result = subprocess.run(
|
||||
["tasklist", "/fi", "imagename eq ls-dyna_mpp.exe"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=True,
|
||||
)
|
||||
print(result.stdout)
|
||||
else:
|
||||
result = subprocess.run(
|
||||
["pgrep", "-f", "ls-dyna"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
if result.stdout.strip():
|
||||
print(f"运行中的 LS-DYNA 进程 PID: {result.stdout.strip()}")
|
||||
else:
|
||||
print("没有运行中的 LS-DYNA 进程")
|
||||
except subprocess.CalledProcessError as e:
|
||||
print(f"检查进程状态失败: {e}")
|
||||
@@ -0,0 +1,236 @@
|
||||
"""packtool - Python 打包工具.
|
||||
|
||||
提供源码打包/依赖打包/wheel 构建/嵌入式 Python 安装/zip 包创建/清理 子命令.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import platform
|
||||
import shutil
|
||||
import subprocess
|
||||
import urllib.request
|
||||
import zipfile
|
||||
from pathlib import Path
|
||||
|
||||
import pyflowx as px
|
||||
from pyflowx.ops._common import IGNORE_PATTERNS
|
||||
|
||||
__all__ = [
|
||||
"clean_build_dir",
|
||||
"create_zip_package",
|
||||
"install_embed_python",
|
||||
"pack_dependencies",
|
||||
"pack_source",
|
||||
"pack_wheel",
|
||||
]
|
||||
|
||||
# ============================================================================
|
||||
# 配置
|
||||
# ============================================================================
|
||||
|
||||
DEFAULT_BUILD_DIR = ".pypack"
|
||||
DEFAULT_DIST_DIR = "dist"
|
||||
DEFAULT_LIB_DIR = "libs"
|
||||
DEFAULT_CACHE_DIR = ".cache/pypack"
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# 公共函数
|
||||
# ============================================================================
|
||||
|
||||
|
||||
@px.tool("packtool", subcommand="src", help="打包源码")
|
||||
def pack_source(project_dir: Path = Path(), output_dir: Path = Path(".pypack")) -> None:
|
||||
"""打包项目源码.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
project_dir : Path
|
||||
项目目录 (默认: 当前目录)
|
||||
output_dir : Path
|
||||
输出目录 (默认: .pypack)
|
||||
"""
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
pyproject_file = project_dir / "pyproject.toml"
|
||||
project_name = project_dir.name
|
||||
|
||||
if pyproject_file.exists():
|
||||
try:
|
||||
import tomllib
|
||||
|
||||
content = pyproject_file.read_text(encoding="utf-8")
|
||||
data = tomllib.loads(content)
|
||||
project_name = data.get("project", {}).get("name", project_name)
|
||||
except ImportError:
|
||||
pass
|
||||
|
||||
source_dir = output_dir / "src" / project_name
|
||||
source_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
src_subdir = project_dir / "src"
|
||||
if src_subdir.exists():
|
||||
shutil.copytree(
|
||||
src_subdir,
|
||||
source_dir / "src",
|
||||
ignore=shutil.ignore_patterns(*IGNORE_PATTERNS),
|
||||
dirs_exist_ok=True,
|
||||
)
|
||||
else:
|
||||
for item in project_dir.iterdir():
|
||||
if item.name in IGNORE_PATTERNS or item.name.startswith("."):
|
||||
continue
|
||||
dst_item = source_dir / item.name
|
||||
if item.is_dir():
|
||||
shutil.copytree(
|
||||
item,
|
||||
dst_item,
|
||||
ignore=shutil.ignore_patterns(*IGNORE_PATTERNS),
|
||||
dirs_exist_ok=True,
|
||||
)
|
||||
else:
|
||||
shutil.copy2(item, dst_item)
|
||||
|
||||
print(f"源码打包完成: {source_dir}")
|
||||
|
||||
|
||||
@px.tool("packtool", subcommand="deps", help="打包依赖")
|
||||
def pack_dependencies(lib_dir: Path = Path("libs"), dependencies: list[str] | None = None) -> None:
|
||||
"""打包项目依赖.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
lib_dir : Path
|
||||
依赖库目录 (默认: libs)
|
||||
dependencies : list[str] | None
|
||||
依赖列表
|
||||
"""
|
||||
if dependencies is None:
|
||||
dependencies = []
|
||||
|
||||
lib_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
if not dependencies:
|
||||
print("没有依赖需要打包")
|
||||
return
|
||||
|
||||
cmd = [
|
||||
"pip",
|
||||
"install",
|
||||
"--target",
|
||||
str(lib_dir),
|
||||
"--no-compile",
|
||||
"--no-warn-script-location",
|
||||
]
|
||||
cmd.extend(dependencies)
|
||||
|
||||
subprocess.run(cmd, check=True)
|
||||
print(f"依赖打包完成: {lib_dir}")
|
||||
|
||||
|
||||
@px.tool("packtool", subcommand="wheel", help="构建 wheel")
|
||||
def pack_wheel(project_dir: Path = Path(), output_dir: Path = Path("dist")) -> None:
|
||||
"""打包项目为 wheel 文件.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
project_dir : Path
|
||||
项目目录 (默认: 当前目录)
|
||||
output_dir : Path
|
||||
输出目录 (默认: dist)
|
||||
"""
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
cmd = [
|
||||
"pip",
|
||||
"wheel",
|
||||
"--no-deps",
|
||||
"--wheel-dir",
|
||||
str(output_dir),
|
||||
str(project_dir),
|
||||
]
|
||||
|
||||
subprocess.run(cmd, check=True)
|
||||
print(f"Wheel 打包完成: {output_dir}")
|
||||
|
||||
|
||||
@px.tool("packtool", subcommand="embed", help="安装嵌入式 Python")
|
||||
def install_embed_python(version: str = "3.10", output_dir: Path = Path("python")) -> None:
|
||||
"""安装嵌入式 Python.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
version : str
|
||||
Python 版本 (如: 3.10, 3.11), 默认 3.10
|
||||
output_dir : Path
|
||||
输出目录 (默认: python)
|
||||
"""
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
arch = platform.machine().lower()
|
||||
if arch in ["x86_64", "amd64"]:
|
||||
arch = "amd64"
|
||||
elif arch in ["arm64", "aarch64"]:
|
||||
arch = "arm64"
|
||||
|
||||
version_map = {
|
||||
"3.8": "3.8.10",
|
||||
"3.9": "3.9.13",
|
||||
"3.10": "3.10.11",
|
||||
"3.11": "3.11.9",
|
||||
"3.12": "3.12.4",
|
||||
}
|
||||
full_version = version_map.get(version, f"{version}.0")
|
||||
|
||||
url = f"https://www.python.org/ftp/python/{full_version}/python-{full_version}-embed-{arch}.zip"
|
||||
|
||||
cache_file = Path(DEFAULT_CACHE_DIR) / f"python-{full_version}-embed-{arch}.zip"
|
||||
cache_file.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
if not cache_file.exists():
|
||||
print(f"正在下载嵌入式 Python {full_version}...")
|
||||
urllib.request.urlretrieve(url, cache_file)
|
||||
print(f"下载完成: {cache_file}")
|
||||
|
||||
with zipfile.ZipFile(cache_file, "r") as zf:
|
||||
zf.extractall(output_dir)
|
||||
|
||||
print(f"嵌入式 Python 安装完成: {output_dir}")
|
||||
|
||||
|
||||
@px.tool("packtool", subcommand="zip", help="创建 zip 包")
|
||||
def create_zip_package(source_dir: Path = Path(), output_file: Path = Path("package.zip")) -> None:
|
||||
"""创建 ZIP 打包文件.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
source_dir : Path
|
||||
源目录 (默认: 当前目录)
|
||||
output_file : Path
|
||||
输出文件 (默认: package.zip)
|
||||
"""
|
||||
output_file.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
with zipfile.ZipFile(output_file, "w", zipfile.ZIP_DEFLATED) as zf:
|
||||
for file in source_dir.rglob("*"):
|
||||
if file.is_file():
|
||||
arcname = file.relative_to(source_dir)
|
||||
zf.write(file, arcname)
|
||||
|
||||
print(f"ZIP 打包完成: {output_file}")
|
||||
|
||||
|
||||
@px.tool("packtool", subcommand="clean", help="清理构建目录")
|
||||
def clean_build_dir(build_dir: Path = Path(".pypack")) -> None:
|
||||
"""清理构建目录.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
build_dir : Path
|
||||
构建目录 (默认: .pypack)
|
||||
"""
|
||||
if build_dir.exists():
|
||||
shutil.rmtree(build_dir)
|
||||
print(f"清理完成: {build_dir}")
|
||||
else:
|
||||
print(f"目录不存在: {build_dir}")
|
||||
@@ -0,0 +1,177 @@
|
||||
"""piptool - pip 包管理工具.
|
||||
|
||||
提供安装/卸载/重装/下载/升级 pip/冻结依赖 子命令.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import fnmatch
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
import pyflowx as px
|
||||
|
||||
__all__ = [
|
||||
"pip_download",
|
||||
"pip_freeze",
|
||||
"pip_install",
|
||||
"pip_reinstall",
|
||||
"pip_uninstall",
|
||||
"pip_upgrade",
|
||||
]
|
||||
|
||||
# ============================================================================
|
||||
# 配置
|
||||
# ============================================================================
|
||||
|
||||
PACKAGE_DIR = "packages"
|
||||
REQUIREMENTS_FILE = "requirements.txt"
|
||||
|
||||
_PROTECTED_PACKAGES: frozenset[str] = frozenset(
|
||||
{
|
||||
"pyflowx",
|
||||
"bitool",
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# 私有辅助函数
|
||||
# ============================================================================
|
||||
|
||||
|
||||
def _get_installed_packages() -> list[str]:
|
||||
"""获取当前环境中所有已安装的包名."""
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["pip", "list", "--format=freeze"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=True,
|
||||
)
|
||||
packages: list[str] = []
|
||||
for line in result.stdout.strip().split("\n"):
|
||||
if line and "==" in line:
|
||||
pkg_name = line.split("==")[0].strip()
|
||||
packages.append(pkg_name)
|
||||
except (subprocess.SubprocessError, OSError):
|
||||
return []
|
||||
return packages
|
||||
|
||||
|
||||
def _expand_wildcard_packages(pattern: str) -> list[str]:
|
||||
"""展开通配符模式为实际的包名列表."""
|
||||
if not any(char in pattern for char in ["*", "?", "[", "]"]):
|
||||
return [pattern]
|
||||
|
||||
installed_packages = _get_installed_packages()
|
||||
matched = [pkg for pkg in installed_packages if fnmatch.fnmatchcase(pkg.lower(), pattern.lower())]
|
||||
return matched
|
||||
|
||||
|
||||
def _filter_protected_packages(packages: list[str]) -> list[str]:
|
||||
"""过滤掉受保护的包名."""
|
||||
safe = [p for p in packages if p.lower() not in {p.lower() for p in _PROTECTED_PACKAGES}]
|
||||
filtered = [p for p in packages if p.lower() in {p.lower() for p in _PROTECTED_PACKAGES}]
|
||||
if filtered:
|
||||
print(f"跳过受保护的包: {', '.join(filtered)}")
|
||||
return safe
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# 公共函数
|
||||
# ============================================================================
|
||||
|
||||
|
||||
@px.tool("piptool", subcommand="i", help="安装包")
|
||||
def pip_install(packages: list[str]) -> None:
|
||||
"""安装包.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
packages : list[str]
|
||||
包名列表
|
||||
"""
|
||||
subprocess.run(["pip", "install", *packages], check=True)
|
||||
print(f"安装完成: {', '.join(packages)}")
|
||||
|
||||
|
||||
@px.tool("piptool", subcommand="u", help="卸载包")
|
||||
def pip_uninstall(packages: list[str]) -> None:
|
||||
"""卸载包.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
packages : list[str]
|
||||
包名列表
|
||||
"""
|
||||
packages_to_uninstall: list[str] = []
|
||||
for pattern in packages:
|
||||
packages_to_uninstall.extend(_expand_wildcard_packages(pattern))
|
||||
|
||||
packages_to_uninstall = _filter_protected_packages(packages_to_uninstall)
|
||||
|
||||
if not packages_to_uninstall:
|
||||
return
|
||||
|
||||
subprocess.run(["pip", "uninstall", "-y", *packages_to_uninstall], check=True)
|
||||
|
||||
|
||||
@px.tool("piptool", subcommand="r", help="重装包")
|
||||
def pip_reinstall(packages: list[str], offline: bool = False) -> None:
|
||||
"""重新安装包.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
packages : list[str]
|
||||
包名列表
|
||||
offline : bool
|
||||
离线模式
|
||||
"""
|
||||
safe_ps = _filter_protected_packages(packages)
|
||||
if not safe_ps:
|
||||
print("所有指定的包均为受保护包, 跳过重装")
|
||||
return
|
||||
|
||||
subprocess.run(["pip", "uninstall", "-y", *safe_ps], check=True)
|
||||
|
||||
options = ["--no-index", "--find-links", "."] if offline else []
|
||||
subprocess.run(["pip", "install", *options, *safe_ps], check=True)
|
||||
|
||||
|
||||
@px.tool("piptool", subcommand="d", help="下载包")
|
||||
def pip_download(packages: list[str], offline: bool = False) -> None:
|
||||
"""下载包.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
packages : list[str]
|
||||
包名列表
|
||||
offline : bool
|
||||
离线模式
|
||||
"""
|
||||
options = ["--no-index", "--find-links", "."] if offline else []
|
||||
subprocess.run(
|
||||
["pip", "download", *packages, *options, "-d", PACKAGE_DIR],
|
||||
check=True,
|
||||
)
|
||||
|
||||
|
||||
@px.tool("piptool", subcommand="up", help="升级 pip")
|
||||
def pip_upgrade() -> None:
|
||||
"""升级 pip 到最新版本."""
|
||||
subprocess.run(["python", "-m", "pip", "install", "--upgrade", "pip"], check=True)
|
||||
print("pip 升级完成")
|
||||
|
||||
|
||||
@px.tool("piptool", subcommand="f", help="导出依赖")
|
||||
def pip_freeze() -> None:
|
||||
"""冻结依赖到 requirements.txt."""
|
||||
result = subprocess.run(
|
||||
["pip", "freeze", "--exclude-editable"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=True,
|
||||
)
|
||||
Path(REQUIREMENTS_FILE).write_text(result.stdout)
|
||||
print(f"依赖已导出到 {REQUIREMENTS_FILE}")
|
||||
@@ -0,0 +1,243 @@
|
||||
"""pymake - 项目构建工具.
|
||||
|
||||
提供构建/测试/清理/文档/格式化/发布等子命令.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
__all__ = [
|
||||
"b",
|
||||
"ba",
|
||||
"bc",
|
||||
"bump",
|
||||
"bumpmi",
|
||||
"bumpversion",
|
||||
"c",
|
||||
"cov",
|
||||
"doc",
|
||||
"git_add_all",
|
||||
"git_push",
|
||||
"git_push_tags",
|
||||
"lint",
|
||||
"p",
|
||||
"pb",
|
||||
"publish_python",
|
||||
"pyrefly_check",
|
||||
"sync",
|
||||
"t",
|
||||
"tc",
|
||||
"test_coverage",
|
||||
"tf",
|
||||
"tox",
|
||||
"twine_publish",
|
||||
]
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
import pyflowx as px
|
||||
|
||||
# ============================================================================
|
||||
# 单任务别名 (cmd 任务)
|
||||
# ============================================================================
|
||||
|
||||
|
||||
@px.tool("pymake", subcommand="b", help="构建 Python 主包 (uv build)", cmd=["uv", "build"])
|
||||
def b(cwd: Path = Path()) -> None:
|
||||
"""构建 Python 主包."""
|
||||
|
||||
|
||||
@px.tool("pymake", subcommand="bc", help="构建 Rust 核心模块 (maturin build)", cmd=["maturin", "build", "-r"])
|
||||
def bc(cwd: Path = Path()) -> None:
|
||||
"""构建 Rust 核心模块."""
|
||||
|
||||
|
||||
@px.tool("pymake", subcommand="sync", help="同步依赖 (uv sync)", cmd=["uv", "sync"])
|
||||
def sync(cwd: Path = Path()) -> None:
|
||||
"""同步依赖."""
|
||||
|
||||
|
||||
@px.tool("pymake", subcommand="c", help="清理构建产物 (调用 gitt c)", cmd=["pf", "gitt", "c"])
|
||||
def c(cwd: Path = Path()) -> None:
|
||||
"""清理构建产物."""
|
||||
|
||||
|
||||
@px.tool(
|
||||
"pymake",
|
||||
subcommand="t",
|
||||
help="运行测试",
|
||||
cmd=["pytest", "-m", "not slow", "-n", "8", "--dist", "loadfile", "--color=yes", "--durations=10"],
|
||||
)
|
||||
def t(cwd: Path = Path()) -> None:
|
||||
"""运行测试 (含并行)."""
|
||||
|
||||
|
||||
@px.tool(
|
||||
"pymake",
|
||||
subcommand="tf",
|
||||
help="快速测试 (无 slow)",
|
||||
cmd=["pytest", "-m", "not slow", "--dist", "loadfile", "--color=yes", "--durations=10"],
|
||||
)
|
||||
def tf(cwd: Path = Path()) -> None:
|
||||
"""快速测试 (无 slow, 无并行)."""
|
||||
|
||||
|
||||
@px.tool("pymake", subcommand="bumpmi", help="升级次版本号 (bumpversion minor)", cmd=["pf", "bumpversion", "minor"])
|
||||
def bumpmi(cwd: Path = Path()) -> None:
|
||||
"""升级次版本号."""
|
||||
|
||||
|
||||
@px.tool(
|
||||
"pymake",
|
||||
subcommand="doc",
|
||||
help="构建 Sphinx 文档",
|
||||
cmd=["sphinx-build", "-b", "html", "docs", "docs/_build"],
|
||||
)
|
||||
def doc(cwd: Path = Path()) -> None:
|
||||
"""构建 Sphinx 文档."""
|
||||
|
||||
|
||||
@px.tool("pymake", subcommand="lint", help="代码格式化与检查 (ruff)", cmd=["ruff", "check", "--fix", "--unsafe-fixes"])
|
||||
def lint(cwd: Path = Path()) -> None:
|
||||
"""代码格式化与检查."""
|
||||
|
||||
|
||||
@px.tool("pymake", subcommand="tox", help="多版本测试 (tox)", cmd=["tox", "-p", "auto"])
|
||||
def tox(cwd: Path = Path()) -> None:
|
||||
"""多版本测试."""
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# 内部 job (hidden, 不暴露为 subcommand)
|
||||
# ============================================================================
|
||||
|
||||
|
||||
@px.tool(
|
||||
"pymake",
|
||||
subcommand="bumpversion",
|
||||
help="升级版本号 (patch)",
|
||||
cmd=["pf", "bumpversion", "patch"],
|
||||
needs=["git_add_all"],
|
||||
hidden=True,
|
||||
)
|
||||
def bumpversion(cwd: Path = Path()) -> None:
|
||||
"""升级版本号 (patch, 内部 job)."""
|
||||
|
||||
|
||||
@px.tool(
|
||||
"pymake",
|
||||
subcommand="test_coverage",
|
||||
help="测试并生成覆盖率",
|
||||
cmd=["pytest", "--cov", "-n", "8", "--dist", "loadfile", "--tb=short", "-v", "--color=yes", "--durations=10"],
|
||||
needs=["c"],
|
||||
hidden=True,
|
||||
)
|
||||
def test_coverage(cwd: Path = Path()) -> None:
|
||||
"""测试并生成覆盖率 (内部 job)."""
|
||||
|
||||
|
||||
@px.tool(
|
||||
"pymake",
|
||||
subcommand="pyrefly_check",
|
||||
help="pyrefly 类型检查",
|
||||
cmd=["pyrefly", "check", "."],
|
||||
hidden=True,
|
||||
)
|
||||
def pyrefly_check(cwd: Path = Path()) -> None:
|
||||
"""pyrefly 类型检查 (内部 job)."""
|
||||
|
||||
|
||||
@px.tool(
|
||||
"pymake",
|
||||
subcommand="git_add_all",
|
||||
help="git add -A",
|
||||
cmd=["git", "add", "-A"],
|
||||
needs=["tc"],
|
||||
hidden=True,
|
||||
)
|
||||
def git_add_all(cwd: Path = Path()) -> None:
|
||||
"""git add -A (内部 job)."""
|
||||
|
||||
|
||||
@px.tool("pymake", subcommand="git_push", help="git push", cmd=["git", "push"], hidden=True)
|
||||
def git_push(cwd: Path = Path()) -> None:
|
||||
"""git push (内部 job)."""
|
||||
|
||||
|
||||
@px.tool(
|
||||
"pymake",
|
||||
subcommand="git_push_tags",
|
||||
help="git push --tags",
|
||||
cmd=["git", "push", "--tags"],
|
||||
hidden=True,
|
||||
)
|
||||
def git_push_tags(cwd: Path = Path()) -> None:
|
||||
"""git push --tags (内部 job)."""
|
||||
|
||||
|
||||
@px.tool(
|
||||
"pymake",
|
||||
subcommand="twine_publish",
|
||||
help="twine upload",
|
||||
cmd=["twine", "upload", "--disable-progress-bar"],
|
||||
hidden=True,
|
||||
)
|
||||
def twine_publish(cwd: Path = Path()) -> None:
|
||||
"""twine upload (内部 job)."""
|
||||
|
||||
|
||||
@px.tool("pymake", subcommand="publish_python", help="hatch publish", cmd=["hatch", "publish"], hidden=True)
|
||||
def publish_python(cwd: Path = Path()) -> None:
|
||||
"""hatch publish (内部 job)."""
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# 聚合 job (有 needs 无 cmd)
|
||||
# ============================================================================
|
||||
|
||||
|
||||
@px.tool("pymake", subcommand="ba", help="构建所有包 (Python + Rust)", needs=["b", "bc"], strategy="thread")
|
||||
def ba(cwd: Path = Path()) -> None:
|
||||
"""构建所有包 (Python + Rust)."""
|
||||
|
||||
|
||||
@px.tool("pymake", subcommand="bump", help="升级版本号 (清理 + 检查 + add + bumpversion)", needs=["bumpversion"])
|
||||
def bump(cwd: Path = Path()) -> None:
|
||||
"""升级版本号 (聚合)."""
|
||||
|
||||
|
||||
@px.tool("pymake", subcommand="cov", help="测试并生成覆盖率", needs=["test_coverage"])
|
||||
def cov(cwd: Path = Path()) -> None:
|
||||
"""测试并生成覆盖率 (聚合)."""
|
||||
|
||||
|
||||
@px.tool(
|
||||
"pymake",
|
||||
subcommand="tc",
|
||||
help="类型检查 (pyrefly + ruff)",
|
||||
needs=["c", "pyrefly_check", "lint"],
|
||||
strategy="thread",
|
||||
)
|
||||
def tc(cwd: Path = Path()) -> None:
|
||||
"""类型检查 (聚合)."""
|
||||
|
||||
|
||||
@px.tool(
|
||||
"pymake",
|
||||
subcommand="p",
|
||||
help="推送代码 (清理 + push + push tags)",
|
||||
needs=["c", "git_push", "git_push_tags"],
|
||||
strategy="thread",
|
||||
)
|
||||
def p(cwd: Path = Path()) -> None:
|
||||
"""推送代码 (聚合)."""
|
||||
|
||||
|
||||
@px.tool(
|
||||
"pymake",
|
||||
subcommand="pb",
|
||||
help="发布到 PyPI (twine + hatch)",
|
||||
needs=["twine_publish", "publish_python"],
|
||||
strategy="thread",
|
||||
)
|
||||
def pb(cwd: Path = Path()) -> None:
|
||||
"""发布到 PyPI (聚合)."""
|
||||
@@ -1,327 +0,0 @@
|
||||
"""文件类函数模块.
|
||||
|
||||
聚合文件日期处理、文件等级重命名、文件夹备份、文件夹压缩工具的可复用函数.
|
||||
所有公共函数通过 ``@px.register_fn`` 注册, 供 YAML 任务编排引用.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
import shutil
|
||||
import time
|
||||
import zipfile
|
||||
from pathlib import Path
|
||||
|
||||
import pyflowx as px
|
||||
|
||||
__all__ = [
|
||||
"BRACKETS",
|
||||
"DATE_PATTERN",
|
||||
"IGNORE_DIRS",
|
||||
"IGNORE_EXT",
|
||||
"IGNORE_FILES",
|
||||
"LEVELS",
|
||||
"SEP",
|
||||
"add_date_prefix",
|
||||
"archive_folder",
|
||||
"backup_folder",
|
||||
"folderback_default",
|
||||
"folderzip_default",
|
||||
"get_file_timestamp",
|
||||
"process_file_date",
|
||||
"process_file_level",
|
||||
"process_files_date",
|
||||
"process_files_level",
|
||||
"remove_date_prefix",
|
||||
"remove_dump",
|
||||
"remove_marks",
|
||||
"zip_folders",
|
||||
"zip_target",
|
||||
]
|
||||
|
||||
# ============================================================================
|
||||
# filedate 配置
|
||||
# ============================================================================
|
||||
|
||||
DATE_PATTERN = re.compile(r"(20|19)\d{2}[-_#.~]?((0[1-9])|(1[012]))[-_#.~]?((0[1-9])|([12]\d)|(3[01]))[-_#.~]?")
|
||||
SEP = "_"
|
||||
|
||||
# ============================================================================
|
||||
# filelevel 配置
|
||||
# ============================================================================
|
||||
|
||||
LEVELS: dict[str, str] = {
|
||||
"0": "",
|
||||
"1": "PUB,NOR",
|
||||
"2": "INT",
|
||||
"3": "CON",
|
||||
"4": "CLA",
|
||||
}
|
||||
|
||||
BRACKETS: tuple[str, str] = (" ([_(【-", " )]_)】")
|
||||
|
||||
# ============================================================================
|
||||
# folderzip 配置
|
||||
# ============================================================================
|
||||
|
||||
IGNORE_DIRS: list[str] = [".git", ".idea", ".vscode", "__pycache__"]
|
||||
IGNORE_FILES: list[str] = [".gitignore"]
|
||||
IGNORE: list[str] = [*IGNORE_DIRS, *IGNORE_FILES]
|
||||
IGNORE_EXT: list[str] = [".zip", ".rar", ".7z", ".tar", ".gz"]
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# filedate 函数
|
||||
# ============================================================================
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def get_file_timestamp(filepath: Path) -> str:
|
||||
"""获取文件时间戳."""
|
||||
modified_time = filepath.stat().st_mtime
|
||||
created_time = filepath.stat().st_ctime
|
||||
return time.strftime("%Y%m%d", time.localtime(max((modified_time, created_time))))
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def remove_date_prefix(filepath: Path) -> Path:
|
||||
"""移除文件日期前缀."""
|
||||
stem = filepath.stem
|
||||
new_stem = DATE_PATTERN.sub("", stem)
|
||||
if new_stem != stem:
|
||||
new_path = filepath.with_name(new_stem + filepath.suffix)
|
||||
filepath.rename(new_path)
|
||||
return new_path
|
||||
return filepath
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def add_date_prefix(filepath: Path) -> Path:
|
||||
"""添加文件日期前缀."""
|
||||
timestamp = get_file_timestamp(filepath)
|
||||
stem = filepath.stem
|
||||
new_stem = f"{timestamp}{SEP}{stem}"
|
||||
new_path = filepath.with_name(new_stem + filepath.suffix)
|
||||
if new_path != filepath:
|
||||
filepath.rename(new_path)
|
||||
return new_path
|
||||
return filepath
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def process_file_date(filepath: Path, clear: bool = False) -> None:
|
||||
"""处理单个文件的日期前缀.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
filepath : Path
|
||||
文件路径
|
||||
clear : bool
|
||||
是否清除日期前缀
|
||||
"""
|
||||
if clear:
|
||||
remove_date_prefix(filepath)
|
||||
else:
|
||||
new_path = remove_date_prefix(filepath)
|
||||
add_date_prefix(new_path)
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def process_files_date(targets: list[Path], clear: bool = False) -> None:
|
||||
"""批量处理文件日期前缀.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
targets : list[Path]
|
||||
文件路径列表
|
||||
clear : bool
|
||||
是否清除日期前缀
|
||||
"""
|
||||
for target in targets:
|
||||
if target.exists() and not target.name.startswith("."):
|
||||
process_file_date(target, clear)
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# filelevel 函数
|
||||
# ============================================================================
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def remove_marks(stem: str, marks: list[str]) -> str:
|
||||
"""从文件名主干中移除所有标记."""
|
||||
left_brackets, right_brackets = BRACKETS
|
||||
for mark in marks:
|
||||
pos = 0
|
||||
while True:
|
||||
pos = stem.find(mark, pos)
|
||||
if pos == -1:
|
||||
break
|
||||
b, e = pos - 1, pos + len(mark)
|
||||
if b >= 0 and e < len(stem) and stem[b] in left_brackets and stem[e] in right_brackets:
|
||||
stem = stem[:b] + stem[e + 1 :]
|
||||
else:
|
||||
pos = e
|
||||
return stem
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def process_file_level(filepath: Path, level: int = 0) -> None:
|
||||
"""处理单个文件的等级标记.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
filepath : Path
|
||||
文件路径
|
||||
level : int
|
||||
文件等级 (0-4), 0 用于清除等级
|
||||
"""
|
||||
if not (0 <= level < len(LEVELS)):
|
||||
print(f"无效的等级 {level}, 必须在 0 和 {len(LEVELS) - 1} 之间")
|
||||
return
|
||||
|
||||
if not filepath.exists():
|
||||
print(f"文件不存在: {filepath}")
|
||||
return
|
||||
|
||||
filestem = filepath.stem
|
||||
original_stem = filestem
|
||||
|
||||
for level_names in LEVELS.values():
|
||||
if level_names:
|
||||
filestem = remove_marks(filestem, level_names.split(","))
|
||||
|
||||
for digit in map(str, range(1, 10)):
|
||||
filestem = remove_marks(filestem, [digit])
|
||||
|
||||
if level > 0:
|
||||
levelstr = LEVELS.get(str(level), "").split(",")[0]
|
||||
if levelstr:
|
||||
filestem = f"{filestem}({levelstr})"
|
||||
|
||||
if filestem != original_stem:
|
||||
new_path = filepath.with_name(filestem + filepath.suffix)
|
||||
filepath.rename(new_path)
|
||||
print(f"重命名: {filepath} -> {new_path}")
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def process_files_level(targets: list[Path], level: int = 0) -> None:
|
||||
"""批量处理文件等级标记.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
targets : list[Path]
|
||||
文件路径列表
|
||||
level : int
|
||||
文件等级 (0-4)
|
||||
"""
|
||||
for target in targets:
|
||||
process_file_level(target, level)
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# folderback 函数
|
||||
# ============================================================================
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def remove_dump(src: Path, dst: Path, max_zip: int) -> None:
|
||||
"""递归删除旧的备份 zip 文件."""
|
||||
zip_paths = [filepath for filepath in dst.rglob("*.zip") if src.stem in str(filepath)]
|
||||
zip_files = sorted(zip_paths, key=lambda fn: str(fn)[-19:-4])
|
||||
if len(zip_files) > max_zip:
|
||||
zip_files[0].unlink()
|
||||
remove_dump(src, dst, max_zip)
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def zip_target(src: Path, dst: Path, max_zip: int) -> None:
|
||||
"""将单个文件或文件夹压缩为 zip 文件."""
|
||||
files = [str(_) for _ in src.rglob("*")]
|
||||
timestamp = time.strftime("_%Y%m%d_%H%M%S")
|
||||
target_path = dst / (src.stem + timestamp + ".zip")
|
||||
|
||||
with zipfile.ZipFile(target_path, "w") as zip_file:
|
||||
for file in files:
|
||||
zip_file.write(file, arcname=file.replace(str(src.parent), ""))
|
||||
|
||||
remove_dump(src, dst, max_zip)
|
||||
print(f"备份完成: {target_path}")
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def backup_folder(src: str, dst: str, max_zip: int = 5) -> None:
|
||||
"""备份文件夹.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
src : str
|
||||
源文件夹路径
|
||||
dst : str
|
||||
目标文件夹路径
|
||||
max_zip : int
|
||||
最大备份数量
|
||||
"""
|
||||
src_path = Path(src)
|
||||
dst_path = Path(dst)
|
||||
|
||||
if not src_path.exists():
|
||||
print(f"源文件夹不存在: {src_path}")
|
||||
return
|
||||
|
||||
if not dst_path.exists():
|
||||
dst_path.mkdir(parents=True, exist_ok=True)
|
||||
print(f"创建目标文件夹: {dst_path}")
|
||||
|
||||
zip_target(src_path, dst_path, max_zip)
|
||||
|
||||
|
||||
@px.register_fn("folderback_default")
|
||||
def folderback_default() -> None:
|
||||
"""备份当前目录到 ./backup."""
|
||||
backup_folder(".", "./backup", 5)
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# folderzip 函数
|
||||
# ============================================================================
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def archive_folder(folder: Path) -> None:
|
||||
"""压缩单个文件夹."""
|
||||
shutil.make_archive(
|
||||
str(folder.with_name(folder.name)),
|
||||
format="zip",
|
||||
base_dir=folder,
|
||||
)
|
||||
print(f"压缩完成: {folder.name}.zip")
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def zip_folders(cwd: str = ".") -> None:
|
||||
"""压缩目录下的所有文件夹.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
cwd : str
|
||||
工作目录
|
||||
"""
|
||||
cwd_path = Path(cwd)
|
||||
if not cwd_path.exists():
|
||||
print(f"目录不存在: {cwd_path}")
|
||||
return
|
||||
|
||||
dirs: list[Path] = [
|
||||
e for e in cwd_path.iterdir() if e.is_dir() and e.name not in IGNORE_DIRS and e.suffix not in IGNORE_EXT
|
||||
]
|
||||
|
||||
for dir_path in dirs:
|
||||
archive_folder(dir_path)
|
||||
|
||||
|
||||
@px.register_fn("folderzip_default")
|
||||
def folderzip_default() -> None:
|
||||
"""压缩当前目录下的所有文件夹."""
|
||||
zip_folders(".")
|
||||
@@ -0,0 +1,5 @@
|
||||
"""files 子包 — 文件与文档操作工具."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
__all__: list[str] = []
|
||||
@@ -0,0 +1,120 @@
|
||||
"""filedate - 文件日期处理工具.
|
||||
|
||||
提供添加日期前缀/清除日期前缀 子命令.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
import pyflowx as px
|
||||
|
||||
__all__ = [
|
||||
"DATE_PATTERN",
|
||||
"SEP",
|
||||
"add_date_prefix",
|
||||
"get_file_timestamp",
|
||||
"process_file_date",
|
||||
"process_files_date",
|
||||
"remove_date_prefix",
|
||||
]
|
||||
|
||||
# ============================================================================
|
||||
# 配置
|
||||
# ============================================================================
|
||||
|
||||
DATE_PATTERN = re.compile(r"(20|19)\d{2}[-_#.~]?((0[1-9])|(1[012]))[-_#.~]?((0[1-9])|([12]\d)|(3[01]))[-_#.~]?")
|
||||
SEP = "_"
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# 公共函数
|
||||
# ============================================================================
|
||||
|
||||
|
||||
def get_file_timestamp(filepath: Path) -> str:
|
||||
"""获取文件时间戳."""
|
||||
modified_time = filepath.stat().st_mtime
|
||||
created_time = filepath.stat().st_ctime
|
||||
return time.strftime("%Y%m%d", time.localtime(max((modified_time, created_time))))
|
||||
|
||||
|
||||
def remove_date_prefix(filepath: Path) -> Path:
|
||||
"""移除文件日期前缀."""
|
||||
stem = filepath.stem
|
||||
new_stem = DATE_PATTERN.sub("", stem)
|
||||
if new_stem != stem:
|
||||
new_path = filepath.with_name(new_stem + filepath.suffix)
|
||||
filepath.rename(new_path)
|
||||
return new_path
|
||||
return filepath
|
||||
|
||||
|
||||
def add_date_prefix(filepath: Path) -> Path:
|
||||
"""添加文件日期前缀."""
|
||||
timestamp = get_file_timestamp(filepath)
|
||||
stem = filepath.stem
|
||||
new_stem = f"{timestamp}{SEP}{stem}"
|
||||
new_path = filepath.with_name(new_stem + filepath.suffix)
|
||||
if new_path != filepath:
|
||||
filepath.rename(new_path)
|
||||
return new_path
|
||||
return filepath
|
||||
|
||||
|
||||
def process_file_date(filepath: Path, clear: bool = False) -> None:
|
||||
"""处理单个文件的日期前缀.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
filepath : Path
|
||||
文件路径
|
||||
clear : bool
|
||||
是否清除日期前缀
|
||||
"""
|
||||
if clear:
|
||||
remove_date_prefix(filepath)
|
||||
else:
|
||||
new_path = remove_date_prefix(filepath)
|
||||
add_date_prefix(new_path)
|
||||
|
||||
|
||||
@px.tool("filedate", subcommand="add", help="添加日期前缀")
|
||||
def process_files_date_add(files: list[Path]) -> None:
|
||||
"""添加日期前缀.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
files : list[Path]
|
||||
文件路径列表
|
||||
"""
|
||||
process_files_date(files, clear=False)
|
||||
|
||||
|
||||
@px.tool("filedate", subcommand="clear", help="清除日期前缀")
|
||||
def process_files_date_clear(files: list[Path]) -> None:
|
||||
"""清除日期前缀.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
files : list[Path]
|
||||
文件路径列表
|
||||
"""
|
||||
process_files_date(files, clear=True)
|
||||
|
||||
|
||||
def process_files_date(targets: list[Path], clear: bool = False) -> None:
|
||||
"""批量处理文件日期前缀.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
targets : list[Path]
|
||||
文件路径列表
|
||||
clear : bool
|
||||
是否清除日期前缀
|
||||
"""
|
||||
for target in targets:
|
||||
if target.exists() and not target.name.startswith("."):
|
||||
process_file_date(target, clear)
|
||||
@@ -0,0 +1,108 @@
|
||||
"""filelevel - 文件等级重命名工具.
|
||||
|
||||
提供设置文件等级 子命令.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
import pyflowx as px
|
||||
|
||||
__all__ = [
|
||||
"BRACKETS",
|
||||
"LEVELS",
|
||||
"process_file_level",
|
||||
"process_files_level",
|
||||
"remove_marks",
|
||||
]
|
||||
|
||||
# ============================================================================
|
||||
# 配置
|
||||
# ============================================================================
|
||||
|
||||
LEVELS: dict[str, str] = {
|
||||
"0": "",
|
||||
"1": "PUB,NOR",
|
||||
"2": "INT",
|
||||
"3": "CON",
|
||||
"4": "CLA",
|
||||
}
|
||||
|
||||
BRACKETS: tuple[str, str] = (" ([_(【-", " )]_)】")
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# 公共函数
|
||||
# ============================================================================
|
||||
|
||||
|
||||
def remove_marks(stem: str, marks: list[str]) -> str:
|
||||
"""从文件名主干中移除所有标记."""
|
||||
left_brackets, right_brackets = BRACKETS
|
||||
for mark in marks:
|
||||
pos = 0
|
||||
while True:
|
||||
pos = stem.find(mark, pos)
|
||||
if pos == -1:
|
||||
break
|
||||
b, e = pos - 1, pos + len(mark)
|
||||
if b >= 0 and e < len(stem) and stem[b] in left_brackets and stem[e] in right_brackets:
|
||||
stem = stem[:b] + stem[e + 1 :]
|
||||
else:
|
||||
pos = e
|
||||
return stem
|
||||
|
||||
|
||||
def process_file_level(filepath: Path, level: int = 0) -> None:
|
||||
"""处理单个文件的等级标记.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
filepath : Path
|
||||
文件路径
|
||||
level : int
|
||||
文件等级 (0-4), 0 用于清除等级
|
||||
"""
|
||||
if not (0 <= level < len(LEVELS)):
|
||||
print(f"无效的等级 {level}, 必须在 0 和 {len(LEVELS) - 1} 之间")
|
||||
return
|
||||
|
||||
if not filepath.exists():
|
||||
print(f"文件不存在: {filepath}")
|
||||
return
|
||||
|
||||
filestem = filepath.stem
|
||||
original_stem = filestem
|
||||
|
||||
for level_names in LEVELS.values():
|
||||
if level_names:
|
||||
filestem = remove_marks(filestem, level_names.split(","))
|
||||
|
||||
for digit in map(str, range(1, 10)):
|
||||
filestem = remove_marks(filestem, [digit])
|
||||
|
||||
if level > 0:
|
||||
levelstr = LEVELS.get(str(level), "").split(",")[0]
|
||||
if levelstr:
|
||||
filestem = f"{filestem}({levelstr})"
|
||||
|
||||
if filestem != original_stem:
|
||||
new_path = filepath.with_name(filestem + filepath.suffix)
|
||||
filepath.rename(new_path)
|
||||
print(f"重命名: {filepath} -> {new_path}")
|
||||
|
||||
|
||||
@px.tool("filelevel", subcommand="set", help="设置文件等级")
|
||||
def process_files_level(files: list[Path], level: int = 0) -> None:
|
||||
"""批量处理文件等级标记.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
files : list[Path]
|
||||
文件路径列表
|
||||
level : int
|
||||
文件等级 (0-4)
|
||||
"""
|
||||
for target in files:
|
||||
process_file_level(target, level)
|
||||
@@ -0,0 +1,73 @@
|
||||
"""folderback - 文件夹备份工具.
|
||||
|
||||
备份当前文件夹到指定目录, 自动清理旧备份.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import time
|
||||
import zipfile
|
||||
from pathlib import Path
|
||||
|
||||
import pyflowx as px
|
||||
|
||||
__all__ = [
|
||||
"backup_folder",
|
||||
"remove_dump",
|
||||
"zip_target",
|
||||
]
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# 公共函数
|
||||
# ============================================================================
|
||||
|
||||
|
||||
def remove_dump(src: Path, dst: Path, max_zip: int) -> None:
|
||||
"""递归删除旧的备份 zip 文件."""
|
||||
zip_paths = [filepath for filepath in dst.rglob("*.zip") if src.stem in str(filepath)]
|
||||
zip_files = sorted(zip_paths, key=lambda fn: str(fn)[-19:-4])
|
||||
if len(zip_files) > max_zip:
|
||||
zip_files[0].unlink()
|
||||
remove_dump(src, dst, max_zip)
|
||||
|
||||
|
||||
def zip_target(src: Path, dst: Path, max_zip: int) -> None:
|
||||
"""将单个文件或文件夹压缩为 zip 文件."""
|
||||
files = [str(_) for _ in src.rglob("*")]
|
||||
timestamp = time.strftime("_%Y%m%d_%H%M%S")
|
||||
target_path = dst / (src.stem + timestamp + ".zip")
|
||||
|
||||
with zipfile.ZipFile(target_path, "w") as zip_file:
|
||||
for file in files:
|
||||
zip_file.write(file, arcname=file.replace(str(src.parent), ""))
|
||||
|
||||
remove_dump(src, dst, max_zip)
|
||||
print(f"备份完成: {target_path}")
|
||||
|
||||
|
||||
@px.tool("folderback", help="备份文件夹")
|
||||
def backup_folder(src: str = ".", dst: str = "./backup", max_zip: int = 5) -> None:
|
||||
"""备份文件夹.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
src : str
|
||||
源文件夹路径 (默认: 当前目录)
|
||||
dst : str
|
||||
目标文件夹路径 (默认: ./backup)
|
||||
max_zip : int
|
||||
最大备份数量 (默认: 5)
|
||||
"""
|
||||
src_path = Path(src)
|
||||
dst_path = Path(dst)
|
||||
|
||||
if not src_path.exists():
|
||||
print(f"源文件夹不存在: {src_path}")
|
||||
return
|
||||
|
||||
if not dst_path.exists():
|
||||
dst_path.mkdir(parents=True, exist_ok=True)
|
||||
print(f"创建目标文件夹: {dst_path}")
|
||||
|
||||
zip_target(src_path, dst_path, max_zip)
|
||||
@@ -0,0 +1,64 @@
|
||||
"""folderzip - 文件夹压缩工具.
|
||||
|
||||
压缩当前目录下的所有子文件夹为 zip 文件.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import shutil
|
||||
from pathlib import Path
|
||||
|
||||
import pyflowx as px
|
||||
|
||||
__all__ = [
|
||||
"IGNORE_DIRS",
|
||||
"IGNORE_EXT",
|
||||
"IGNORE_FILES",
|
||||
"archive_folder",
|
||||
"zip_folders",
|
||||
]
|
||||
|
||||
# ============================================================================
|
||||
# 配置
|
||||
# ============================================================================
|
||||
|
||||
IGNORE_DIRS: list[str] = [".git", ".idea", ".vscode", "__pycache__"]
|
||||
IGNORE_FILES: list[str] = [".gitignore"]
|
||||
IGNORE_EXT: list[str] = [".zip", ".rar", ".7z", ".tar", ".gz"]
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# 公共函数
|
||||
# ============================================================================
|
||||
|
||||
|
||||
def archive_folder(folder: Path) -> None:
|
||||
"""压缩单个文件夹."""
|
||||
shutil.make_archive(
|
||||
str(folder.with_name(folder.name)),
|
||||
format="zip",
|
||||
base_dir=folder,
|
||||
)
|
||||
print(f"压缩完成: {folder.name}.zip")
|
||||
|
||||
|
||||
@px.tool("folderzip", help="压缩文件夹")
|
||||
def zip_folders(cwd: str = ".") -> None:
|
||||
"""压缩目录下的所有文件夹.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
cwd : str
|
||||
工作目录 (默认: 当前目录)
|
||||
"""
|
||||
cwd_path = Path(cwd)
|
||||
if not cwd_path.exists():
|
||||
print(f"目录不存在: {cwd_path}")
|
||||
return
|
||||
|
||||
dirs: list[Path] = [
|
||||
e for e in cwd_path.iterdir() if e.is_dir() and e.name not in IGNORE_DIRS and e.suffix not in IGNORE_EXT
|
||||
]
|
||||
|
||||
for dir_path in dirs:
|
||||
archive_folder(dir_path)
|
||||
@@ -1,23 +1,16 @@
|
||||
"""媒体类函数模块.
|
||||
"""pdftool - PDF 文件工具集.
|
||||
|
||||
聚合 PDF 工具 (pdftool) 和截图工具 (screenshot) 的可复用函数.
|
||||
所有公共函数通过 ``@px.register_fn`` 注册, 供 YAML 任务编排引用.
|
||||
提供 PDF 合并/拆分/压缩/加密/解密/提取文本/提取图片/水印/旋转/裁剪/
|
||||
信息/OCR/转图片/修复 等子命令.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import subprocess
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
|
||||
import pyflowx as px
|
||||
from pyflowx.conditions import Constants
|
||||
|
||||
__all__ = [
|
||||
"DEFAULT_PASSWORD",
|
||||
"DEFAULT_QUALITY",
|
||||
"PDF_SUFFIX",
|
||||
"get_screenshot_path",
|
||||
"pdf_add_watermark",
|
||||
"pdf_compress",
|
||||
"pdf_crop",
|
||||
@@ -33,8 +26,6 @@ __all__ = [
|
||||
"pdf_rotate",
|
||||
"pdf_split",
|
||||
"pdf_to_images",
|
||||
"take_screenshot_area",
|
||||
"take_screenshot_full",
|
||||
]
|
||||
|
||||
try:
|
||||
@@ -52,25 +43,34 @@ except ImportError:
|
||||
HAS_PYPDF = False
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# 配置
|
||||
# ============================================================================
|
||||
|
||||
PDF_SUFFIX = ".pdf"
|
||||
DEFAULT_QUALITY = 75
|
||||
DEFAULT_PASSWORD = ""
|
||||
def _require_pymupdf() -> bool:
|
||||
"""PyMuPDF 未安装时打印提示, 返回是否可用."""
|
||||
if not HAS_PYMUPDF:
|
||||
print("未安装 PyMuPDF 库, 请安装: pip install PyMuPDF")
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# PDF 函数
|
||||
# ============================================================================
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def pdf_merge(input_paths: list[Path], output_path: Path) -> None:
|
||||
"""合并多个 PDF 文件."""
|
||||
def _require_pypdf() -> bool:
|
||||
"""pypdf 未安装时打印提示, 返回是否可用."""
|
||||
if not HAS_PYPDF:
|
||||
print("未安装 pypdf 库, 请安装: pip install pypdf")
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
@px.tool("pdftool", subcommand="m", help="合并 PDF")
|
||||
def pdf_merge(input_paths: list[Path], output_path: Path = Path("merged.pdf")) -> None:
|
||||
"""合并多个 PDF 文件.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
input_paths : list[Path]
|
||||
输入 PDF 文件列表
|
||||
output_path : Path
|
||||
输出文件 (默认: merged.pdf)
|
||||
"""
|
||||
if not _require_pypdf():
|
||||
return
|
||||
|
||||
writer = pypdf.PdfWriter()
|
||||
@@ -87,11 +87,18 @@ def pdf_merge(input_paths: list[Path], output_path: Path) -> None:
|
||||
print(f"合并完成: {output_path}")
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def pdf_split(input_path: Path, output_dir: Path) -> None:
|
||||
"""拆分 PDF 文件为单页."""
|
||||
if not HAS_PYPDF:
|
||||
print("未安装 pypdf 库, 请安装: pip install pypdf")
|
||||
@px.tool("pdftool", subcommand="s", help="拆分 PDF")
|
||||
def pdf_split(input_path: Path, output_dir: Path = Path("split")) -> None:
|
||||
"""拆分 PDF 文件为单页.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
input_path : Path
|
||||
输入 PDF 文件
|
||||
output_dir : Path
|
||||
输出目录 (默认: split)
|
||||
"""
|
||||
if not _require_pypdf():
|
||||
return
|
||||
|
||||
reader = pypdf.PdfReader(str(input_path))
|
||||
@@ -107,11 +114,18 @@ def pdf_split(input_path: Path, output_dir: Path) -> None:
|
||||
print(f"拆分完成: {output_dir}")
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def pdf_compress(input_path: Path, output_path: Path) -> None:
|
||||
"""压缩 PDF 文件."""
|
||||
if not HAS_PYMUPDF:
|
||||
print("未安装 PyMuPDF 库, 请安装: pip install PyMuPDF")
|
||||
@px.tool("pdftool", subcommand="c", help="压缩 PDF")
|
||||
def pdf_compress(input_path: Path, output_path: Path = Path("compressed.pdf")) -> None:
|
||||
"""压缩 PDF 文件.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
input_path : Path
|
||||
输入 PDF 文件
|
||||
output_path : Path
|
||||
输出文件 (默认: compressed.pdf)
|
||||
"""
|
||||
if not _require_pymupdf():
|
||||
return
|
||||
|
||||
doc = fitz.open(str(input_path))
|
||||
@@ -125,11 +139,23 @@ def pdf_compress(input_path: Path, output_path: Path) -> None:
|
||||
print(f"压缩完成: {output_path} (缩小 {ratio:.1f}%)")
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def pdf_encrypt(input_path: Path, output_path: Path, password: str) -> None:
|
||||
"""加密 PDF 文件."""
|
||||
if not HAS_PYPDF:
|
||||
print("未安装 pypdf 库, 请安装: pip install pypdf")
|
||||
@px.tool("pdftool", subcommand="e", help="加密 PDF")
|
||||
def pdf_encrypt(input_path: Path, output_path: Path = Path("encrypted.pdf"), password: str = "") -> None:
|
||||
"""加密 PDF 文件.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
input_path : Path
|
||||
输入 PDF 文件
|
||||
output_path : Path
|
||||
输出文件 (默认: encrypted.pdf)
|
||||
password : str
|
||||
密码 (必填)
|
||||
"""
|
||||
if not password:
|
||||
print("错误: --password 为必填参数")
|
||||
return
|
||||
if not _require_pypdf():
|
||||
return
|
||||
|
||||
reader = pypdf.PdfReader(str(input_path))
|
||||
@@ -146,11 +172,23 @@ def pdf_encrypt(input_path: Path, output_path: Path, password: str) -> None:
|
||||
print(f"加密完成: {output_path}")
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def pdf_decrypt(input_path: Path, output_path: Path, password: str) -> None:
|
||||
"""解密 PDF 文件."""
|
||||
if not HAS_PYPDF:
|
||||
print("未安装 pypdf 库, 请安装: pip install pypdf")
|
||||
@px.tool("pdftool", subcommand="d", help="解密 PDF")
|
||||
def pdf_decrypt(input_path: Path, output_path: Path = Path("decrypted.pdf"), password: str = "") -> None:
|
||||
"""解密 PDF 文件.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
input_path : Path
|
||||
输入 PDF 文件
|
||||
output_path : Path
|
||||
输出文件 (默认: decrypted.pdf)
|
||||
password : str
|
||||
密码 (必填)
|
||||
"""
|
||||
if not password:
|
||||
print("错误: --password 为必填参数")
|
||||
return
|
||||
if not _require_pypdf():
|
||||
return
|
||||
|
||||
reader = pypdf.PdfReader(str(input_path))
|
||||
@@ -168,11 +206,18 @@ def pdf_decrypt(input_path: Path, output_path: Path, password: str) -> None:
|
||||
print(f"解密完成: {output_path}")
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def pdf_extract_text(input_path: Path, output_path: Path) -> None:
|
||||
"""提取 PDF 文本."""
|
||||
if not HAS_PYMUPDF:
|
||||
print("未安装 PyMuPDF 库, 请安装: pip install PyMuPDF")
|
||||
@px.tool("pdftool", subcommand="xt", help="提取文本")
|
||||
def pdf_extract_text(input_path: Path, output_path: Path = Path("output.txt")) -> None:
|
||||
"""提取 PDF 文本.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
input_path : Path
|
||||
输入 PDF 文件
|
||||
output_path : Path
|
||||
输出文件 (默认: output.txt)
|
||||
"""
|
||||
if not _require_pymupdf():
|
||||
return
|
||||
|
||||
doc = fitz.open(str(input_path))
|
||||
@@ -186,11 +231,18 @@ def pdf_extract_text(input_path: Path, output_path: Path) -> None:
|
||||
print(f"文本提取完成: {output_path}")
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def pdf_extract_images(input_path: Path, output_dir: Path) -> None:
|
||||
"""提取 PDF 图片."""
|
||||
if not HAS_PYMUPDF:
|
||||
print("未安装 PyMuPDF 库, 请安装: pip install PyMuPDF")
|
||||
@px.tool("pdftool", subcommand="xi", help="提取图片")
|
||||
def pdf_extract_images(input_path: Path, output_dir: Path = Path("images")) -> None:
|
||||
"""提取 PDF 图片.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
input_path : Path
|
||||
输入 PDF 文件
|
||||
output_dir : Path
|
||||
输出目录 (默认: images)
|
||||
"""
|
||||
if not _require_pymupdf():
|
||||
return
|
||||
|
||||
doc = fitz.open(str(input_path))
|
||||
@@ -213,11 +265,22 @@ def pdf_extract_images(input_path: Path, output_dir: Path) -> None:
|
||||
print(f"图片提取完成: {output_dir} (共 {image_count} 张)")
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def pdf_add_watermark(input_path: Path, output_path: Path, text: str = "CONFIDENTIAL") -> None:
|
||||
"""添加 PDF 水印."""
|
||||
if not HAS_PYMUPDF:
|
||||
print("未安装 PyMuPDF 库, 请安装: pip install PyMuPDF")
|
||||
@px.tool("pdftool", subcommand="w", help="添加水印")
|
||||
def pdf_add_watermark(
|
||||
input_path: Path, output_path: Path = Path("watermarked.pdf"), text: str = "CONFIDENTIAL"
|
||||
) -> None:
|
||||
"""添加 PDF 水印.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
input_path : Path
|
||||
输入 PDF 文件
|
||||
output_path : Path
|
||||
输出文件 (默认: watermarked.pdf)
|
||||
text : str
|
||||
水印文字 (默认: CONFIDENTIAL)
|
||||
"""
|
||||
if not _require_pymupdf():
|
||||
return
|
||||
|
||||
doc = fitz.open(str(input_path))
|
||||
@@ -234,11 +297,20 @@ def pdf_add_watermark(input_path: Path, output_path: Path, text: str = "CONFIDEN
|
||||
print(f"水印添加完成: {output_path}")
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def pdf_rotate(input_path: Path, output_path: Path, rotation: int = 90) -> None:
|
||||
"""旋转 PDF 页面."""
|
||||
if not HAS_PYMUPDF:
|
||||
print("未安装 PyMuPDF 库, 请安装: pip install PyMuPDF")
|
||||
@px.tool("pdftool", subcommand="r", help="旋转 PDF")
|
||||
def pdf_rotate(input_path: Path, output_path: Path = Path("rotated.pdf"), rotation: int = 90) -> None:
|
||||
"""旋转 PDF 页面.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
input_path : Path
|
||||
输入 PDF 文件
|
||||
output_path : Path
|
||||
输出文件 (默认: rotated.pdf)
|
||||
rotation : int
|
||||
旋转角度 (默认: 90)
|
||||
"""
|
||||
if not _require_pymupdf():
|
||||
return
|
||||
|
||||
doc = fitz.open(str(input_path))
|
||||
@@ -251,11 +323,24 @@ def pdf_rotate(input_path: Path, output_path: Path, rotation: int = 90) -> None:
|
||||
print(f"旋转完成: {output_path}")
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def pdf_crop(input_path: Path, output_path: Path, margins: tuple[int, int, int, int]) -> None:
|
||||
"""裁剪 PDF 页面."""
|
||||
if not HAS_PYMUPDF:
|
||||
print("未安装 PyMuPDF 库, 请安装: pip install PyMuPDF")
|
||||
@px.tool("pdftool", subcommand="crop", help="裁剪 PDF")
|
||||
def pdf_crop(
|
||||
input_path: Path,
|
||||
output_path: Path = Path("cropped.pdf"),
|
||||
margins: tuple[int, int, int, int] = (10, 10, 10, 10),
|
||||
) -> None:
|
||||
"""裁剪 PDF 页面.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
input_path : Path
|
||||
输入 PDF 文件
|
||||
output_path : Path
|
||||
输出文件 (默认: cropped.pdf)
|
||||
margins : tuple[int, int, int, int]
|
||||
边距 (左, 上, 右, 下), 默认 (10, 10, 10, 10)
|
||||
"""
|
||||
if not _require_pymupdf():
|
||||
return
|
||||
|
||||
doc = fitz.open(str(input_path))
|
||||
@@ -277,11 +362,16 @@ def pdf_crop(input_path: Path, output_path: Path, margins: tuple[int, int, int,
|
||||
print(f"裁剪完成: {output_path}")
|
||||
|
||||
|
||||
@px.register_fn
|
||||
@px.tool("pdftool", subcommand="i", help="查看 PDF 信息")
|
||||
def pdf_info(input_path: Path) -> None:
|
||||
"""显示 PDF 信息."""
|
||||
if not HAS_PYMUPDF:
|
||||
print("未安装 PyMuPDF 库, 请安装: pip install PyMuPDF")
|
||||
"""显示 PDF 信息.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
input_path : Path
|
||||
输入 PDF 文件
|
||||
"""
|
||||
if not _require_pymupdf():
|
||||
return
|
||||
|
||||
doc = fitz.open(str(input_path))
|
||||
@@ -299,9 +389,19 @@ def pdf_info(input_path: Path) -> None:
|
||||
doc.close()
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def pdf_ocr(input_path: Path, output_path: Path, lang: str = "chi_sim+eng") -> None:
|
||||
"""PDF OCR 识别."""
|
||||
@px.tool("pdftool", subcommand="ocr", help="PDF OCR 识别")
|
||||
def pdf_ocr(input_path: Path, output_path: Path = Path("ocr.pdf"), lang: str = "chi_sim+eng") -> None:
|
||||
"""PDF OCR 识别.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
input_path : Path
|
||||
输入 PDF 文件
|
||||
output_path : Path
|
||||
输出文件 (默认: ocr.pdf)
|
||||
lang : str
|
||||
识别语言 (默认: chi_sim+eng)
|
||||
"""
|
||||
try:
|
||||
import pytesseract
|
||||
from PIL import Image
|
||||
@@ -309,8 +409,7 @@ def pdf_ocr(input_path: Path, output_path: Path, lang: str = "chi_sim+eng") -> N
|
||||
print("未安装 OCR 相关库, 请安装: pip install pytesseract pillow")
|
||||
return
|
||||
|
||||
if not HAS_PYMUPDF:
|
||||
print("未安装 PyMuPDF 库, 请安装: pip install PyMuPDF")
|
||||
if not _require_pymupdf():
|
||||
return
|
||||
|
||||
doc = fitz.open(str(input_path))
|
||||
@@ -334,11 +433,19 @@ def pdf_ocr(input_path: Path, output_path: Path, lang: str = "chi_sim+eng") -> N
|
||||
print(f"OCR 识别完成: {output_path}")
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def pdf_reorder(input_path: Path, output_path: Path, order: list[int]) -> None:
|
||||
"""重排 PDF 页面顺序."""
|
||||
if not HAS_PYPDF:
|
||||
print("未安装 pypdf 库, 请安装: pip install pypdf")
|
||||
"""重排 PDF 页面顺序.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
input_path : Path
|
||||
输入 PDF 文件
|
||||
output_path : Path
|
||||
输出文件
|
||||
order : list[int]
|
||||
页面顺序列表 (0-based)
|
||||
"""
|
||||
if not _require_pypdf():
|
||||
return
|
||||
|
||||
reader = pypdf.PdfReader(str(input_path))
|
||||
@@ -355,11 +462,20 @@ def pdf_reorder(input_path: Path, output_path: Path, order: list[int]) -> None:
|
||||
print(f"重排完成: {output_path}")
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def pdf_to_images(input_path: Path, output_dir: Path, dpi: int = 300) -> None:
|
||||
"""PDF 转图片."""
|
||||
if not HAS_PYMUPDF:
|
||||
print("未安装 PyMuPDF 库, 请安装: pip install PyMuPDF")
|
||||
@px.tool("pdftool", subcommand="img", help="PDF 转图片")
|
||||
def pdf_to_images(input_path: Path, output_dir: Path = Path("images"), dpi: int = 300) -> None:
|
||||
"""PDF 转图片.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
input_path : Path
|
||||
输入 PDF 文件
|
||||
output_dir : Path
|
||||
输出目录 (默认: images)
|
||||
dpi : int
|
||||
DPI (默认: 300)
|
||||
"""
|
||||
if not _require_pymupdf():
|
||||
return
|
||||
|
||||
doc = fitz.open(str(input_path))
|
||||
@@ -375,11 +491,18 @@ def pdf_to_images(input_path: Path, output_dir: Path, dpi: int = 300) -> None:
|
||||
print(f"转换完成: {output_dir}")
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def pdf_repair(input_path: Path, output_path: Path) -> None:
|
||||
"""修复 PDF 文件."""
|
||||
if not HAS_PYMUPDF:
|
||||
print("未安装 PyMuPDF 库, 请安装: pip install PyMuPDF")
|
||||
@px.tool("pdftool", subcommand="repair", help="修复 PDF")
|
||||
def pdf_repair(input_path: Path, output_path: Path = Path("repaired.pdf")) -> None:
|
||||
"""修复 PDF 文件.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
input_path : Path
|
||||
输入 PDF 文件
|
||||
output_path : Path
|
||||
输出文件 (默认: repaired.pdf)
|
||||
"""
|
||||
if not _require_pymupdf():
|
||||
return
|
||||
|
||||
doc = fitz.open(str(input_path))
|
||||
@@ -387,112 +510,3 @@ def pdf_repair(input_path: Path, output_path: Path) -> None:
|
||||
doc.save(str(output_path), garbage=4, deflate=True, clean=True)
|
||||
doc.close()
|
||||
print(f"修复完成: {output_path}")
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# screenshot 函数
|
||||
# ============================================================================
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def get_screenshot_path(filename: str | None = None) -> Path:
|
||||
"""获取截图保存路径.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
filename : str | None
|
||||
文件名, 如果为 None 则自动生成
|
||||
|
||||
Returns
|
||||
-------
|
||||
Path
|
||||
截图保存路径
|
||||
"""
|
||||
if filename is None:
|
||||
timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
|
||||
filename = f"screenshot_{timestamp}.png"
|
||||
|
||||
screenshots_dir = Path.home() / "Pictures" / "screenshots"
|
||||
screenshots_dir.mkdir(parents=True, exist_ok=True)
|
||||
return screenshots_dir / filename
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def take_screenshot_full(filename: str | None = None) -> None:
|
||||
"""全屏截图.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
filename : str | None
|
||||
文件名
|
||||
"""
|
||||
output_path = get_screenshot_path(filename)
|
||||
|
||||
if Constants.IS_WINDOWS:
|
||||
ps_script = f"""
|
||||
Add-Type -AssemblyName System.Windows.Forms
|
||||
Add-Type -AssemblyName System.Drawing
|
||||
$screen = [System.Windows.Forms.Screen]::PrimaryScreen
|
||||
$bounds = $screen.Bounds
|
||||
$bitmap = New-Object System.Drawing.Bitmap $bounds.Width, $bounds.Height
|
||||
$graphics = [System.Drawing.Graphics]::FromImage($bitmap)
|
||||
$graphics.CopyFromScreen($bounds.Location, [System.Drawing.Point]::Empty, $bounds.Size)
|
||||
$bitmap.Save('{output_path.as_posix()}')
|
||||
$graphics.Dispose()
|
||||
$bitmap.Dispose()
|
||||
"""
|
||||
subprocess.run(["powershell", "-Command", ps_script], check=True)
|
||||
elif Constants.IS_MACOS:
|
||||
subprocess.run(["screencapture", "-x", str(output_path)], check=True)
|
||||
else:
|
||||
try:
|
||||
subprocess.run(["gnome-screenshot", "-f", str(output_path)], check=True)
|
||||
except FileNotFoundError:
|
||||
subprocess.run(["scrot", str(output_path)], check=True)
|
||||
|
||||
print(f"截图已保存: {output_path}")
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def take_screenshot_area(filename: str | None = None) -> None:
|
||||
"""区域截图.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
filename : str | None
|
||||
文件名
|
||||
"""
|
||||
output_path = get_screenshot_path(filename)
|
||||
|
||||
if Constants.IS_WINDOWS:
|
||||
ps_script = f"""
|
||||
Add-Type -AssemblyName System.Windows.Forms
|
||||
Add-Type -AssemblyName System.Drawing
|
||||
$form = New-Object System.Windows.Forms.Form
|
||||
$form.WindowState = 'Maximized'
|
||||
$form.FormBorderStyle = 'None'
|
||||
$form.BackColor = [System.Drawing.Color]::FromArgb(1, 0, 0)
|
||||
$form.Opacity = 0.5
|
||||
$form.TopMost = $true
|
||||
$form.Show()
|
||||
Start-Sleep -Milliseconds 100
|
||||
$screen = [System.Windows.Forms.Screen]::PrimaryScreen
|
||||
$bounds = $screen.Bounds
|
||||
$bitmap = New-Object System.Drawing.Bitmap $bounds.Width, $bounds.Height
|
||||
$graphics = [System.Drawing.Graphics]::FromImage($bitmap)
|
||||
$graphics.CopyFromScreen($bounds.Location, [System.Drawing.Point]::Empty, $bounds.Size)
|
||||
$form.Close()
|
||||
$bitmap.Save('{output_path.as_posix()}')
|
||||
$graphics.Dispose()
|
||||
$bitmap.Dispose()
|
||||
"""
|
||||
subprocess.run(["powershell", "-Command", ps_script], check=True)
|
||||
elif Constants.IS_MACOS:
|
||||
subprocess.run(["screencapture", "-i", str(output_path)], check=True)
|
||||
else:
|
||||
try:
|
||||
subprocess.run(["gnome-screenshot", "-a", "-f", str(output_path)], check=True)
|
||||
except FileNotFoundError:
|
||||
subprocess.run(["scrot", "-s", str(output_path)], check=True)
|
||||
|
||||
print(f"截图已保存: {output_path}")
|
||||
@@ -0,0 +1,116 @@
|
||||
"""screenshot - 截图工具.
|
||||
|
||||
跨平台截图: Windows 用 PowerShell, macOS 用 screencapture, Linux 用 gnome-screenshot/scrot.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import subprocess
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
|
||||
import pyflowx as px
|
||||
from pyflowx.conditions import Constants
|
||||
|
||||
|
||||
def get_screenshot_path(filename: str | None = None) -> Path:
|
||||
"""获取截图保存路径.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
filename : str | None
|
||||
文件名, 如果为 None 则自动生成
|
||||
|
||||
Returns
|
||||
-------
|
||||
Path
|
||||
截图保存路径
|
||||
"""
|
||||
if filename is None:
|
||||
timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
|
||||
filename = f"screenshot_{timestamp}.png"
|
||||
|
||||
screenshots_dir = Path.home() / "Pictures" / "screenshots"
|
||||
screenshots_dir.mkdir(parents=True, exist_ok=True)
|
||||
return screenshots_dir / filename
|
||||
|
||||
|
||||
@px.tool("screenshot", subcommand="full", help="全屏截图")
|
||||
def take_screenshot_full(filename: str | None = None) -> None:
|
||||
"""全屏截图.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
filename : str | None
|
||||
文件名
|
||||
"""
|
||||
output_path = get_screenshot_path(filename)
|
||||
|
||||
if Constants.IS_WINDOWS:
|
||||
ps_script = f"""
|
||||
Add-Type -AssemblyName System.Windows.Forms
|
||||
Add-Type -AssemblyName System.Drawing
|
||||
$screen = [System.Windows.Forms.Screen]::PrimaryScreen
|
||||
$bounds = $screen.Bounds
|
||||
$bitmap = New-Object System.Drawing.Bitmap $bounds.Width, $bounds.Height
|
||||
$graphics = [System.Drawing.Graphics]::FromImage($bitmap)
|
||||
$graphics.CopyFromScreen($bounds.Location, [System.Drawing.Point]::Empty, $bounds.Size)
|
||||
$bitmap.Save('{output_path.as_posix()}')
|
||||
$graphics.Dispose()
|
||||
$bitmap.Dispose()
|
||||
"""
|
||||
subprocess.run(["powershell", "-Command", ps_script], check=True)
|
||||
elif Constants.IS_MACOS:
|
||||
subprocess.run(["screencapture", "-x", str(output_path)], check=True)
|
||||
else:
|
||||
try:
|
||||
subprocess.run(["gnome-screenshot", "-f", str(output_path)], check=True)
|
||||
except FileNotFoundError:
|
||||
subprocess.run(["scrot", str(output_path)], check=True)
|
||||
|
||||
print(f"截图已保存: {output_path}")
|
||||
|
||||
|
||||
@px.tool("screenshot", subcommand="area", help="区域截图")
|
||||
def take_screenshot_area(filename: str | None = None) -> None:
|
||||
"""区域截图.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
filename : str | None
|
||||
文件名
|
||||
"""
|
||||
output_path = get_screenshot_path(filename)
|
||||
|
||||
if Constants.IS_WINDOWS:
|
||||
ps_script = f"""
|
||||
Add-Type -AssemblyName System.Windows.Forms
|
||||
Add-Type -AssemblyName System.Drawing
|
||||
$form = New-Object System.Windows.Forms.Form
|
||||
$form.WindowState = 'Maximized'
|
||||
$form.FormBorderStyle = 'None'
|
||||
$form.BackColor = [System.Drawing.Color]::FromArgb(1, 0, 0)
|
||||
$form.Opacity = 0.5
|
||||
$form.TopMost = $true
|
||||
$form.Show()
|
||||
Start-Sleep -Milliseconds 100
|
||||
$screen = [System.Windows.Forms.Screen]::PrimaryScreen
|
||||
$bounds = $screen.Bounds
|
||||
$bitmap = New-Object System.Drawing.Bitmap $bounds.Width, $bounds.Height
|
||||
$graphics = [System.Drawing.Graphics]::FromImage($bitmap)
|
||||
$graphics.CopyFromScreen($bounds.Location, [System.Drawing.Point]::Empty, $bounds.Size)
|
||||
$form.Close()
|
||||
$bitmap.Save('{output_path.as_posix()}')
|
||||
$graphics.Dispose()
|
||||
$bitmap.Dispose()
|
||||
"""
|
||||
subprocess.run(["powershell", "-Command", ps_script], check=True)
|
||||
elif Constants.IS_MACOS:
|
||||
subprocess.run(["screencapture", "-i", str(output_path)], check=True)
|
||||
else:
|
||||
try:
|
||||
subprocess.run(["gnome-screenshot", "-a", "-f", str(output_path)], check=True)
|
||||
except FileNotFoundError:
|
||||
subprocess.run(["scrot", "-s", str(output_path)], check=True)
|
||||
|
||||
print(f"截图已保存: {output_path}")
|
||||
@@ -0,0 +1,5 @@
|
||||
"""infra 子包 — 基础设施与服务部署工具."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
__all__: list[str] = []
|
||||
@@ -0,0 +1,27 @@
|
||||
"""dockercmd - Docker 镜像登录工具.
|
||||
|
||||
登录腾讯云 Docker 镜像仓库.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import getpass
|
||||
import subprocess
|
||||
|
||||
import pyflowx as px
|
||||
|
||||
_DOCKER_MIRROR_TENCENT: str = "ccr.ccs.tencentyun.com"
|
||||
|
||||
|
||||
@px.tool("dockercmd", subcommand="login", help="登录腾讯云 Docker 镜像仓库")
|
||||
def docker_login_tencent(username: str = "") -> None:
|
||||
"""登录腾讯云 Docker 镜像仓库.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
username : str
|
||||
Docker 用户名 (为空时由 docker 交互式提示输入)
|
||||
"""
|
||||
user = username or getpass.getuser()
|
||||
subprocess.run(["docker", "login", "--username", user, _DOCKER_MIRROR_TENCENT], check=False)
|
||||
print(f"已尝试登录腾讯云镜像仓库 (用户: {user})")
|
||||
@@ -0,0 +1,418 @@
|
||||
"""envdev - 开发环境镜像源配置工具.
|
||||
|
||||
配置 Python / Conda / Rust 镜像源 (Linux 还会安装 Qt 库、中文字体、Docker).
|
||||
所有镜像源参数互不影响, 可单独使用.
|
||||
Linux 专用操作 (系统镜像/Qt/字体/Docker) 在非 Linux 平台上由函数内部跳过.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import getpass
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
from typing import Literal
|
||||
|
||||
import pyflowx as px
|
||||
from pyflowx.conditions import Constants
|
||||
from pyflowx.ops._common import ensure_platform
|
||||
|
||||
__all__ = [
|
||||
"download_rustup_script",
|
||||
"install_linux_docker",
|
||||
"install_linux_fonts",
|
||||
"install_linux_qt_libs",
|
||||
"install_rust_toolchain",
|
||||
"setup_conda_mirror",
|
||||
"setup_linux_system_mirror",
|
||||
"setup_python_mirror",
|
||||
"setup_rust_mirror",
|
||||
]
|
||||
|
||||
# ============================================================================
|
||||
# 配置常量
|
||||
# ============================================================================
|
||||
|
||||
PyMirrorType = Literal["tsinghua", "aliyun", "huaweicloud", "ustc", "zju"]
|
||||
CondaMirrorType = Literal["tsinghua", "ustc", "bsfu", "aliyun"]
|
||||
RustMirrorType = Literal["tsinghua", "ustc", "aliyun"]
|
||||
|
||||
_PIP_INDEX_URLS: dict[str, str] = {
|
||||
"tsinghua": "https://pypi.tuna.tsinghua.edu.cn/simple",
|
||||
"aliyun": "https://mirrors.aliyun.com/pypi/simple/",
|
||||
"huaweicloud": "https://mirrors.huaweicloud.com/repository/pypi/simple/",
|
||||
"ustc": "https://pypi.mirrors.ustc.edu.cn/simple/",
|
||||
"zju": "https://mirrors.zju.edu.cn/pypi/simple/",
|
||||
}
|
||||
|
||||
_PIP_TRUSTED_HOSTS: dict[str, str] = {
|
||||
"tsinghua": "pypi.tuna.tsinghua.edu.cn",
|
||||
"aliyun": "mirrors.aliyun.com",
|
||||
"huaweicloud": "mirrors.huaweicloud.com",
|
||||
"ustc": "pypi.mirrors.ustc.edu.cn",
|
||||
"zju": "mirrors.zju.edu.cn",
|
||||
}
|
||||
|
||||
_UV_PYTHON_INSTALL_MIRROR: str = "https://registry.npmmirror.com/-/binary/python-build-standalone"
|
||||
|
||||
_CONDA_MIRROR_URLS: dict[str, list[str]] = {
|
||||
"tsinghua": [
|
||||
"https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/",
|
||||
"https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free/",
|
||||
"https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/r/",
|
||||
"https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/msys2/",
|
||||
"https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/pro/",
|
||||
"https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/",
|
||||
"https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/bioconda/",
|
||||
"https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/menpo/",
|
||||
"https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/pytorch/",
|
||||
],
|
||||
"ustc": [
|
||||
"https://mirrors.ustc.edu.cn/anaconda/pkgs/main/",
|
||||
"https://mirrors.ustc.edu.cn/anaconda/pkgs/free/",
|
||||
"https://mirrors.ustc.edu.cn/anaconda/pkgs/r/",
|
||||
"https://mirrors.ustc.edu.cn/anaconda/pkgs/msys2/",
|
||||
"https://mirrors.ustc.edu.cn/anaconda/pkgs/pro/",
|
||||
"https://mirrors.ustc.edu.cn/anaconda/pkgs/dev/",
|
||||
"https://mirrors.ustc.edu.cn/anaconda/cloud/conda-forge/",
|
||||
"https://mirrors.ustc.edu.cn/anaconda/cloud/bioconda/",
|
||||
"https://mirrors.ustc.edu.cn/anaconda/cloud/menpo/",
|
||||
"https://mirrors.ustc.edu.cn/anaconda/cloud/pytorch/",
|
||||
],
|
||||
"bsfu": [
|
||||
"https://mirrors.bsfu.edu.cn/anaconda/pkgs/main/",
|
||||
"https://mirrors.bsfu.edu.cn/anaconda/pkgs/free/",
|
||||
"https://mirrors.bsfu.edu.cn/anaconda/pkgs/r/",
|
||||
"https://mirrors.bsfu.edu.cn/anaconda/pkgs/msys2/",
|
||||
"https://mirrors.bsfu.edu.cn/anaconda/pkgs/pro/",
|
||||
"https://mirrors.bsfu.edu.cn/anaconda/pkgs/dev/",
|
||||
"https://mirrors.bsfu.edu.cn/anaconda/cloud/conda-forge/",
|
||||
"https://mirrors.bsfu.edu.cn/anaconda/cloud/bioconda/",
|
||||
"https://mirrors.bsfu.edu.cn/anaconda/cloud/menpo/",
|
||||
"https://mirrors.bsfu.edu.cn/anaconda/cloud/pytorch/",
|
||||
],
|
||||
"aliyun": [
|
||||
"https://mirrors.aliyun.com/anaconda/pkgs/main/",
|
||||
"https://mirrors.aliyun.com/anaconda/pkgs/free/",
|
||||
"https://mirrors.aliyun.com/anaconda/pkgs/r/",
|
||||
"https://mirrors.aliyun.com/anaconda/pkgs/msys2/",
|
||||
"https://mirrors.aliyun.com/anaconda/pkgs/pro/",
|
||||
"https://mirrors.aliyun.com/anaconda/pkgs/dev/",
|
||||
"https://mirrors.aliyun.com/anaconda/cloud/conda-forge/",
|
||||
"https://mirrors.aliyun.com/anaconda/cloud/bioconda/",
|
||||
"https://mirrors.aliyun.com/anaconda/cloud/menpo/",
|
||||
"https://mirrors.aliyun.com/anaconda/cloud/pytorch/",
|
||||
],
|
||||
}
|
||||
|
||||
_RUSTUP_MIRRORS: dict[str, dict[str, str]] = {
|
||||
"tsinghua": {
|
||||
"RUSTUP_DIST_SERVER": "https://mirrors.tuna.tsinghua.edu.cn/rustup",
|
||||
"RUSTUP_UPDATE_ROOT": "https://mirrors.tuna.tsinghua.edu.cn/rustup/rustup",
|
||||
"TOML_REGISTRY": "https://mirrors.tuna.tsinghua.edu.cn/crates.io-index/",
|
||||
},
|
||||
"aliyun": {
|
||||
"RUSTUP_DIST_SERVER": "https://mirrors.aliyun.com/rustup",
|
||||
"RUSTUP_UPDATE_ROOT": "https://mirrors.aliyun.com/rustup/rustup",
|
||||
"TOML_REGISTRY": "https://mirrors.aliyun.com/crates.io-index/",
|
||||
},
|
||||
"ustc": {
|
||||
"RUSTUP_DIST_SERVER": "https://mirrors.ustc.edu.cn/rust-static",
|
||||
"RUSTUP_UPDATE_ROOT": "https://mirrors.ustc.edu.cn/rust-static/rustup",
|
||||
"TOML_REGISTRY": "https://mirrors.ustc.edu.cn/crates.io-index/",
|
||||
},
|
||||
}
|
||||
|
||||
_RUST_SCCACHE_DIR: Path = Path.home() / ".cargo" / "sccache"
|
||||
_RUST_SCCACHE_CACHE_SIZE: str = "20G"
|
||||
|
||||
_QT_LIBS: list[str] = [
|
||||
"build-essential",
|
||||
"libgl1",
|
||||
"libegl1",
|
||||
"libglib2.0-0",
|
||||
"libfontconfig1",
|
||||
"libfreetype6",
|
||||
"libxkbcommon0",
|
||||
"libdbus-1-3",
|
||||
"libxcb-xinerama0",
|
||||
"libxcb-icccm4",
|
||||
"libxcb-image0",
|
||||
"libxcb-keysyms1",
|
||||
"libxcb-randr0",
|
||||
"libxcb-render-util0",
|
||||
"libxcb-shape0",
|
||||
"libxcb-xfixes0",
|
||||
"libxcb-cursor0",
|
||||
]
|
||||
|
||||
_CHINESE_FONTS: list[str] = [
|
||||
"fonts-noto-cjk",
|
||||
"fonts-wqy-microhei",
|
||||
"fonts-wqy-zenhei",
|
||||
"fonts-noto-color-emoji",
|
||||
]
|
||||
|
||||
_DOWNLOAD_MIRROR_SCRIPT: str = "curl -sSL https://linuxmirrors.cn/main.sh -o /tmp/linuxmirrors.sh"
|
||||
_INSTALL_MIRROR_SCRIPT: str = "sudo bash /tmp/linuxmirrors.sh"
|
||||
|
||||
_RUSTUP_DOWNLOAD_URL_LINUX: str = "https://mirrors.aliyun.com/repo/rust/rustup-init.sh"
|
||||
_RUSTUP_DOWNLOAD_URL_WINDOWS: str = "https://static.rust-lang.org/rustup/dist/x86_64-pc-windows-msvc/rustup-init.exe"
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# 私有辅助
|
||||
# ============================================================================
|
||||
|
||||
|
||||
def _pip_config_path() -> Path:
|
||||
"""返回当前平台的 pip 配置文件路径."""
|
||||
if Constants.IS_LINUX:
|
||||
return Path.home() / ".pip" / "pip.conf"
|
||||
return Path.home() / "pip" / "pip.ini"
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# 镜像源配置函数
|
||||
# ============================================================================
|
||||
|
||||
|
||||
@px.tool("envdev", subcommand="setup-python", help="配置 Python 镜像源")
|
||||
def setup_python_mirror(mirror: str = "tsinghua") -> None:
|
||||
"""配置 Python 镜像源 (设置环境变量 + 写入 pip 配置文件).
|
||||
|
||||
设置 ``PIP_INDEX_URL`` / ``PIP_TRUSTED_HOSTS`` / ``UV_INDEX_URL`` /
|
||||
``UV_PYTHON_INSTALL_MIRROR`` 等环境变量, 并写入 pip 配置文件.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
mirror : str
|
||||
镜像源名称: tsinghua/aliyun/huaweicloud/ustc/zju (默认: tsinghua)
|
||||
"""
|
||||
if mirror not in _PIP_INDEX_URLS:
|
||||
print(f"未知 Python 镜像源: {mirror}")
|
||||
return
|
||||
|
||||
index_url = _PIP_INDEX_URLS[mirror]
|
||||
trusted_host = _PIP_TRUSTED_HOSTS[mirror]
|
||||
|
||||
os.environ["PIP_INDEX_URL"] = index_url
|
||||
os.environ["PIP_TRUSTED_HOSTS"] = trusted_host
|
||||
os.environ["UV_INDEX_URL"] = index_url
|
||||
os.environ["UV_PYTHON_INSTALL_MIRROR"] = _UV_PYTHON_INSTALL_MIRROR
|
||||
os.environ["UV_HTTP_TIMEOUT"] = "600"
|
||||
os.environ["UV_LINK_MODE"] = "copy"
|
||||
|
||||
config_path = _pip_config_path()
|
||||
config_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
content = f"[global]\nindex-url = {index_url}\ntrusted-host = {trusted_host}\n"
|
||||
config_path.write_text(content, encoding="utf-8")
|
||||
print(f"Python 镜像源已配置: {mirror} -> {config_path}")
|
||||
|
||||
|
||||
@px.tool("envdev", subcommand="setup-conda", help="配置 Conda 镜像源")
|
||||
def setup_conda_mirror(mirror: str = "tsinghua") -> None:
|
||||
"""配置 Conda 镜像源 (写入 ~/.condarc).
|
||||
|
||||
Parameters
|
||||
----------
|
||||
mirror : str
|
||||
镜像源名称: tsinghua/ustc/bsfu/aliyun (默认: tsinghua)
|
||||
"""
|
||||
if mirror not in _CONDA_MIRROR_URLS:
|
||||
print(f"未知 Conda 镜像源: {mirror}")
|
||||
return
|
||||
|
||||
urls = _CONDA_MIRROR_URLS[mirror]
|
||||
config_path = Path.home() / ".condarc"
|
||||
config_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
content = "show_channel_urls: true\nchannels:\n - " + "\n - ".join(urls) + "\n - defaults\n"
|
||||
config_path.write_text(content, encoding="utf-8")
|
||||
print(f"Conda 镜像源已配置: {mirror} -> {config_path}")
|
||||
|
||||
|
||||
@px.tool("envdev", subcommand="setup-rust", help="配置 Rust 镜像源")
|
||||
def setup_rust_mirror(mirror: str = "tsinghua", version: str = "stable") -> None:
|
||||
"""配置 Rust 镜像源 (设置环境变量 + 写入 cargo config + 创建 sccache 目录).
|
||||
|
||||
设置 ``RUSTUP_DIST_SERVER`` / ``RUSTUP_UPDATE_ROOT`` / ``RUST_SCCACHE_DIR``
|
||||
等环境变量, 写入 ``~/.cargo/config.toml``, 并创建 sccache 缓存目录.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
mirror : str
|
||||
镜像源名称: tsinghua/ustc/aliyun (默认: tsinghua)
|
||||
version : str
|
||||
Rust 版本 (未使用, 保留以与原 envdev 参数对齐)
|
||||
"""
|
||||
del version # 兼容旧参数, 实际安装由独立 job 处理
|
||||
|
||||
if mirror not in _RUSTUP_MIRRORS:
|
||||
print(f"未知 Rust 镜像源: {mirror}")
|
||||
return
|
||||
|
||||
mirrors = _RUSTUP_MIRRORS[mirror]
|
||||
os.environ["RUSTUP_DIST_SERVER"] = mirrors["RUSTUP_DIST_SERVER"]
|
||||
os.environ["RUSTUP_UPDATE_ROOT"] = mirrors["RUSTUP_UPDATE_ROOT"]
|
||||
os.environ["RUST_SCCACHE_DIR"] = str(_RUST_SCCACHE_DIR)
|
||||
os.environ["RUST_SCCACHE_CACHE_SIZE"] = _RUST_SCCACHE_CACHE_SIZE
|
||||
|
||||
_RUST_SCCACHE_DIR.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
config_path = Path.home() / ".cargo" / "config.toml"
|
||||
config_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
registry = mirrors["TOML_REGISTRY"]
|
||||
content = (
|
||||
f"\n[source.crates-io]\nreplace-with = '{mirror}'\n\n"
|
||||
f'[source.{mirror}]\nregistry = "sparse+{registry}"\n\n'
|
||||
f'[registries.{mirror}]\nindex = "sparse+{registry}"\n'
|
||||
)
|
||||
config_path.write_text(content, encoding="utf-8")
|
||||
print(f"Rust 镜像源已配置: {mirror} -> {config_path}")
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# Rust 工具链安装
|
||||
# ============================================================================
|
||||
|
||||
|
||||
@px.tool("envdev", subcommand="download-rustup", help="下载 Rustup 安装脚本")
|
||||
def download_rustup_script() -> None:
|
||||
"""下载 Rustup 安装脚本 (跨平台, 已安装 rustup 时跳过).
|
||||
|
||||
Linux 下载 ``rustup-init.sh``, Windows 下载 ``rustup-init.exe``.
|
||||
"""
|
||||
if shutil.which("rustup") is not None:
|
||||
print("rustup 已安装, 跳过下载")
|
||||
return
|
||||
|
||||
if Constants.IS_WINDOWS:
|
||||
print("下载 rustup-init.exe...")
|
||||
subprocess.run(
|
||||
[
|
||||
"powershell",
|
||||
"-Command",
|
||||
"Invoke-WebRequest",
|
||||
"-Uri",
|
||||
_RUSTUP_DOWNLOAD_URL_WINDOWS,
|
||||
"-OutFile",
|
||||
"rustup-init.exe",
|
||||
],
|
||||
check=False,
|
||||
)
|
||||
else:
|
||||
print("下载 rustup-init.sh...")
|
||||
subprocess.run(
|
||||
["curl", "-fsSL", _RUSTUP_DOWNLOAD_URL_LINUX, "-o", "rustup-init.sh"],
|
||||
check=False,
|
||||
)
|
||||
|
||||
|
||||
@px.tool(
|
||||
"envdev",
|
||||
subcommand="install-rust",
|
||||
help="安装 Rust 工具链",
|
||||
needs=["setup-rust", "download-rustup"],
|
||||
allow_upstream_skip=True,
|
||||
)
|
||||
def install_rust_toolchain(version: str = "stable") -> None:
|
||||
"""安装 Rust 工具链 (rustup 未安装时跳过).
|
||||
|
||||
Parameters
|
||||
----------
|
||||
version : str
|
||||
Rust 版本: ``stable`` / ``nightly`` / ``beta`` (默认: ``stable``)
|
||||
"""
|
||||
if shutil.which("rustup") is None:
|
||||
print("rustup 未安装, 跳过工具链安装")
|
||||
return
|
||||
|
||||
subprocess.run(["rustup", "toolchain", "install", version], check=False)
|
||||
print(f"Rust 工具链 {version} 安装完成")
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# Linux 专用函数
|
||||
# ============================================================================
|
||||
|
||||
|
||||
@px.tool("envdev", subcommand="setup-linux-mirror", help="配置 Linux 系统镜像源")
|
||||
def setup_linux_system_mirror() -> None:
|
||||
"""下载并安装 Linux 系统镜像源 (仅 Linux, 已配置国内镜像时跳过).
|
||||
|
||||
检查 ``/etc/apt/sources.list`` 与 ``/etc/apt/sources.list.d/ubuntu.sources``
|
||||
是否已配置国内镜像, 已配置则跳过; 未配置则下载并执行 linuxmirrors 脚本.
|
||||
"""
|
||||
if not ensure_platform(Constants.IS_LINUX, "setup_linux_system_mirror", "Linux"):
|
||||
return
|
||||
|
||||
apt_files = ["/etc/apt/sources.list", "/etc/apt/sources.list.d/ubuntu.sources"]
|
||||
mirror_keys = list(_PIP_INDEX_URLS.keys())
|
||||
already_configured = False
|
||||
for apt_file in apt_files:
|
||||
try:
|
||||
content = Path(apt_file).read_text(encoding="utf-8")
|
||||
except (OSError, UnicodeDecodeError):
|
||||
continue
|
||||
if any(mirror in content for mirror in mirror_keys):
|
||||
already_configured = True
|
||||
break
|
||||
|
||||
if already_configured:
|
||||
print("已配置国内镜像源, 跳过系统镜像配置")
|
||||
return
|
||||
|
||||
print("下载 linuxmirrors 脚本...")
|
||||
subprocess.run(_DOWNLOAD_MIRROR_SCRIPT, shell=True, check=False)
|
||||
print("安装 linuxmirrors...")
|
||||
subprocess.run(_INSTALL_MIRROR_SCRIPT, shell=True, check=False)
|
||||
|
||||
|
||||
@px.tool(
|
||||
"envdev",
|
||||
subcommand="install-qt-libs",
|
||||
help="安装 Qt 依赖库",
|
||||
needs=["setup-linux-mirror"],
|
||||
allow_upstream_skip=True,
|
||||
)
|
||||
def install_linux_qt_libs() -> None:
|
||||
"""安装 Qt 依赖库 (仅 Linux)."""
|
||||
if not ensure_platform(Constants.IS_LINUX, "install_linux_qt_libs", "Linux"):
|
||||
return
|
||||
|
||||
subprocess.run(["sudo", "apt", "install", "-y", *_QT_LIBS], check=False)
|
||||
print("Qt 依赖库安装完成")
|
||||
|
||||
|
||||
@px.tool(
|
||||
"envdev",
|
||||
subcommand="install-fonts",
|
||||
help="安装中文字体",
|
||||
needs=["setup-linux-mirror"],
|
||||
allow_upstream_skip=True,
|
||||
)
|
||||
def install_linux_fonts() -> None:
|
||||
"""安装中文字体 (仅 Linux)."""
|
||||
if not ensure_platform(Constants.IS_LINUX, "install_linux_fonts", "Linux"):
|
||||
return
|
||||
|
||||
subprocess.run(["sudo", "apt", "install", "-y", *_CHINESE_FONTS], check=False)
|
||||
print("中文字体安装完成")
|
||||
|
||||
|
||||
@px.tool(
|
||||
"envdev",
|
||||
subcommand="install-docker",
|
||||
help="安装 Docker",
|
||||
needs=["setup-linux-mirror"],
|
||||
allow_upstream_skip=True,
|
||||
)
|
||||
def install_linux_docker() -> None:
|
||||
"""安装 Docker (仅 Linux)."""
|
||||
if not ensure_platform(Constants.IS_LINUX, "install_linux_docker", "Linux"):
|
||||
return
|
||||
|
||||
subprocess.run(["sudo", "apt", "install", "-y", "docker-compose-v2"], check=False)
|
||||
subprocess.run(["sudo", "usermod", "-aG", "docker", getpass.getuser()], check=False)
|
||||
print("Docker 安装完成 (需重新登录以生效 docker 用户组)")
|
||||
@@ -0,0 +1,41 @@
|
||||
"""msdownload - ModelScope 模型/数据集下载工具.
|
||||
|
||||
从 ModelScope 下载模型、数据集或空间.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
import pyflowx as px
|
||||
|
||||
__all__ = ["msdownload_run"]
|
||||
|
||||
|
||||
@px.tool("msdownload", help="ModelScope 模型/数据集下载工具")
|
||||
def msdownload_run(name: str, target_type: str = "model", download_dir: str | None = None) -> None:
|
||||
"""从 ModelScope 下载模型/数据集/空间.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
name : str
|
||||
目标名称 (如: ``Qwen/Qwen2.5-Coder-32B-Instruct``)
|
||||
target_type : str
|
||||
目标类型: ``model`` / ``dataset`` / ``space`` (默认: ``model``)
|
||||
download_dir : str | None
|
||||
下载目录; 为 None 时默认 ``~/.models/<name 最后一段>``
|
||||
"""
|
||||
if not name:
|
||||
print("msdownload: name 不能为空")
|
||||
return
|
||||
|
||||
if download_dir:
|
||||
out_dir = Path(download_dir)
|
||||
else:
|
||||
out_dir = Path.home() / ".models" / name.rsplit("/", 1)[-1]
|
||||
out_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
cmd = ["uvx", "modelscope", "download", f"--{target_type}", name, "--local_dir", str(out_dir)]
|
||||
print(f"下载 {target_type}: {name} -> {out_dir}")
|
||||
subprocess.run(cmd, check=False)
|
||||
@@ -0,0 +1,87 @@
|
||||
"""sglang - SGLang 本地模型服务工具.
|
||||
|
||||
提供安装与启动 SGLang 服务的子命令.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import shutil
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
import pyflowx as px
|
||||
from pyflowx.ops._common import platform_command
|
||||
|
||||
__all__ = [
|
||||
"install_sglang",
|
||||
"run_sglang",
|
||||
]
|
||||
|
||||
|
||||
@px.tool("sglang", subcommand="install", help="安装 sglang")
|
||||
def install_sglang() -> None:
|
||||
"""安装 sglang (若未安装).
|
||||
|
||||
通过 ``shutil.which`` 检测 sglang 是否已安装, 未安装时执行 ``uv install sglang[all]``.
|
||||
"""
|
||||
if shutil.which("sglang") is not None:
|
||||
print("sglang 已安装, 跳过安装步骤")
|
||||
return
|
||||
|
||||
print("正在安装 sglang[all]...")
|
||||
subprocess.run(["uv", "install", "sglang[all]"], check=False)
|
||||
|
||||
|
||||
@px.tool("sglang", subcommand="run", help="启动 SGLang 服务", needs=["install"])
|
||||
def run_sglang(
|
||||
model: str = "~/.models/Qwen2.5-Coder-32B-Instruct-AWQ",
|
||||
port: int = 8000,
|
||||
ctx_len: int = 32768,
|
||||
mem_fraction: float = 0.75,
|
||||
host: str = "0.0.0.0",
|
||||
log_level: str = "info",
|
||||
) -> None:
|
||||
"""启动 SGLang 本地模型服务.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
model : str
|
||||
模型路径 (默认: ``~/.models/Qwen2.5-Coder-32B-Instruct-AWQ``)
|
||||
port : int
|
||||
服务端口 (默认: 8000)
|
||||
ctx_len : int
|
||||
最大上下文长度 (默认: 32768)
|
||||
mem_fraction : float
|
||||
显存占比 0-1 (默认: 0.75)
|
||||
host : str
|
||||
主机地址 (默认: 0.0.0.0)
|
||||
log_level : str
|
||||
日志级别 (默认: info)
|
||||
"""
|
||||
model_dir = Path(model).expanduser()
|
||||
if not model_dir.exists():
|
||||
print(f"模型目录不存在: {model_dir}")
|
||||
return
|
||||
|
||||
python_bin = platform_command(["python"], ["python3"])[0]
|
||||
cmd = [
|
||||
python_bin,
|
||||
"-m",
|
||||
"sglang.launch_server",
|
||||
"--model-path",
|
||||
str(model_dir),
|
||||
"--host",
|
||||
host,
|
||||
"--port",
|
||||
str(port),
|
||||
"--mem-fraction-static",
|
||||
str(mem_fraction),
|
||||
"--context-length",
|
||||
str(ctx_len),
|
||||
"--tool-call-parser",
|
||||
"qwen",
|
||||
"--log-level",
|
||||
log_level,
|
||||
]
|
||||
print(f"启动 SGLang: {model_dir} (port={port}, ctx={ctx_len}, mem={mem_fraction})")
|
||||
subprocess.run(cmd, check=False)
|
||||
@@ -0,0 +1,84 @@
|
||||
"""sshcopyid - SSH 密钥部署工具.
|
||||
|
||||
将本地 SSH 公钥部署到远程服务器.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import pyflowx as px
|
||||
|
||||
__all__ = ["ssh_copy_id"]
|
||||
|
||||
|
||||
@px.tool("sshcopyid", help="部署 SSH 公钥到远程服务器")
|
||||
def ssh_copy_id(
|
||||
hostname: str,
|
||||
username: str,
|
||||
password: str,
|
||||
port: int = 22,
|
||||
keypath: str = "~/.ssh/id_rsa.pub",
|
||||
timeout: int = 30,
|
||||
) -> None:
|
||||
"""将 SSH 公钥部署到远程服务器.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
hostname : str
|
||||
远程服务器主机名或 IP 地址
|
||||
username : str
|
||||
远程服务器用户名
|
||||
password : str
|
||||
远程服务器密码
|
||||
port : int
|
||||
SSH 端口, 默认 22
|
||||
keypath : str
|
||||
公钥文件路径, 默认 ~/.ssh/id_rsa.pub
|
||||
timeout : int
|
||||
SSH 操作超时秒数, 默认 30
|
||||
"""
|
||||
pub_key_path = Path(keypath).expanduser()
|
||||
if not pub_key_path.exists():
|
||||
print(f"公钥文件不存在: {pub_key_path}")
|
||||
sys.exit(1)
|
||||
|
||||
pub_key = pub_key_path.read_text().strip()
|
||||
|
||||
script = f"""mkdir -p ~/.ssh && chmod 700 ~/.ssh
|
||||
cd ~/.ssh && touch authorized_keys && chmod 600 authorized_keys
|
||||
grep -qF '{pub_key.split()[1]}' authorized_keys 2>/dev/null || echo '{pub_key}' >> authorized_keys"""
|
||||
|
||||
try:
|
||||
subprocess.run(
|
||||
[
|
||||
"sshpass",
|
||||
"-p",
|
||||
password,
|
||||
"ssh",
|
||||
"-p",
|
||||
str(port),
|
||||
"-o",
|
||||
"StrictHostKeyChecking=no",
|
||||
"-o",
|
||||
"UserKnownHostsFile=/dev/null",
|
||||
"-o",
|
||||
f"ConnectTimeout={timeout}",
|
||||
f"{username}@{hostname}",
|
||||
script,
|
||||
],
|
||||
check=True,
|
||||
timeout=timeout,
|
||||
)
|
||||
print(f"SSH 密钥已部署到 {username}@{hostname}:{port}")
|
||||
except FileNotFoundError:
|
||||
print(f"未找到 sshpass 工具, 请手动执行: ssh-copy-id -p {port} {username}@{hostname}")
|
||||
sys.exit(1)
|
||||
except subprocess.TimeoutExpired:
|
||||
print("SSH 连接超时")
|
||||
sys.exit(1)
|
||||
except subprocess.CalledProcessError as e:
|
||||
print(f"SSH 执行失败: {e}")
|
||||
sys.exit(1)
|
||||
@@ -1,504 +0,0 @@
|
||||
"""系统类函数模块.
|
||||
|
||||
聚合 LS-DYNA 计算 (lscalc)、SSH 密钥部署 (sshcopyid)、Python 打包 (packtool)、
|
||||
重置图标缓存 (reset_icon_cache) 的可复用函数. 所有公共函数通过 ``@px.register_fn``
|
||||
注册, 供 YAML 任务编排引用.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import platform
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import urllib.request
|
||||
import zipfile
|
||||
from pathlib import Path
|
||||
|
||||
import pyflowx as px
|
||||
from pyflowx.conditions import Constants
|
||||
|
||||
__all__ = [
|
||||
"DEFAULT_BUILD_DIR",
|
||||
"DEFAULT_CACHE_DIR",
|
||||
"DEFAULT_DIST_DIR",
|
||||
"DEFAULT_INPUT_FILE",
|
||||
"DEFAULT_LIB_DIR",
|
||||
"DEFAULT_NCPU",
|
||||
"IGNORE_PATTERNS",
|
||||
"LS_DYNA_COMMANDS",
|
||||
"check_ls_dyna_status",
|
||||
"clean_build_dir",
|
||||
"create_zip_package",
|
||||
"get_ls_dyna_command",
|
||||
"install_embed_python",
|
||||
"pack_dependencies",
|
||||
"pack_source",
|
||||
"pack_wheel",
|
||||
"reset_icon_cache_run",
|
||||
"run_ls_dyna",
|
||||
"run_ls_dyna_mpi",
|
||||
"ssh_copy_id",
|
||||
]
|
||||
|
||||
# ============================================================================
|
||||
# lscalc 配置
|
||||
# ============================================================================
|
||||
|
||||
LS_DYNA_COMMANDS: dict[str, list[str]] = {
|
||||
"windows": ["ls-dyna_mpp", "i=input.k", "ncpu=4"],
|
||||
"linux": ["ls-dyna_mpp", "i=input.k", "ncpu=8"],
|
||||
"macos": ["ls-dyna_mpp", "i=input.k", "ncpu=4"],
|
||||
}
|
||||
|
||||
DEFAULT_INPUT_FILE: str = "input.k"
|
||||
DEFAULT_NCPU: int = 4
|
||||
|
||||
# ============================================================================
|
||||
# packtool 配置
|
||||
# ============================================================================
|
||||
|
||||
DEFAULT_BUILD_DIR = ".pypack"
|
||||
DEFAULT_DIST_DIR = "dist"
|
||||
DEFAULT_LIB_DIR = "libs"
|
||||
DEFAULT_CACHE_DIR = ".cache/pypack"
|
||||
|
||||
IGNORE_PATTERNS = [
|
||||
"__pycache__",
|
||||
"*.pyc",
|
||||
"*.pyo",
|
||||
".git",
|
||||
".venv",
|
||||
".idea",
|
||||
".vscode",
|
||||
"*.egg-info",
|
||||
"dist",
|
||||
"build",
|
||||
".pytest_cache",
|
||||
".tox",
|
||||
".mypy_cache",
|
||||
]
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# lscalc 函数
|
||||
# ============================================================================
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def get_ls_dyna_command(input_file: str, ncpu: int) -> list[str]:
|
||||
"""获取 LS-DYNA 命令.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
input_file : str
|
||||
输入文件路径
|
||||
ncpu : int
|
||||
CPU 核心数
|
||||
|
||||
Returns
|
||||
-------
|
||||
list[str]
|
||||
LS-DYNA 命令列表
|
||||
"""
|
||||
if Constants.IS_WINDOWS or Constants.IS_MACOS:
|
||||
return ["ls-dyna_mpp", f"i={input_file}", f"ncpu={ncpu}"]
|
||||
else:
|
||||
return ["ls-dyna_mpp", f"i={input_file}", f"ncpu={ncpu}"]
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def run_ls_dyna(input_file: str, ncpu: int = DEFAULT_NCPU) -> None:
|
||||
"""运行 LS-DYNA 计算.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
input_file : str
|
||||
输入文件路径
|
||||
ncpu : int
|
||||
CPU 核心数
|
||||
"""
|
||||
input_path = Path(input_file)
|
||||
if not input_path.exists():
|
||||
print(f"输入文件不存在: {input_path}")
|
||||
return
|
||||
|
||||
cmd = get_ls_dyna_command(input_file, ncpu)
|
||||
try:
|
||||
subprocess.run(cmd, check=True)
|
||||
print(f"LS-DYNA 计算完成: {input_file}")
|
||||
except FileNotFoundError:
|
||||
print("未找到 ls-dyna_mpp 命令")
|
||||
except subprocess.CalledProcessError as e:
|
||||
print(f"LS-DYNA 计算失败: {e}")
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def run_ls_dyna_mpi(input_file: str, ncpu: int = DEFAULT_NCPU) -> None:
|
||||
"""运行 LS-DYNA MPI 计算.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
input_file : str
|
||||
输入文件路径
|
||||
ncpu : int
|
||||
CPU 核心数
|
||||
"""
|
||||
input_path = Path(input_file)
|
||||
if not input_path.exists():
|
||||
print(f"输入文件不存在: {input_path}")
|
||||
return
|
||||
|
||||
cmd = ["mpirun", "-np", str(ncpu), "ls-dyna_mpp", f"i={input_file}"]
|
||||
try:
|
||||
subprocess.run(cmd, check=True)
|
||||
print(f"LS-DYNA MPI 计算完成: {input_file}")
|
||||
except FileNotFoundError:
|
||||
print("未找到 mpirun 或 ls-dyna_mpp 命令")
|
||||
except subprocess.CalledProcessError as e:
|
||||
print(f"LS-DYNA MPI 计算失败: {e}")
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def check_ls_dyna_status() -> None:
|
||||
"""检查 LS-DYNA 进程状态."""
|
||||
try:
|
||||
if Constants.IS_WINDOWS:
|
||||
result = subprocess.run(
|
||||
["tasklist", "/fi", "imagename eq ls-dyna_mpp.exe"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=True,
|
||||
)
|
||||
print(result.stdout)
|
||||
else:
|
||||
result = subprocess.run(
|
||||
["pgrep", "-f", "ls-dyna"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
if result.stdout.strip():
|
||||
print(f"运行中的 LS-DYNA 进程 PID: {result.stdout.strip()}")
|
||||
else:
|
||||
print("没有运行中的 LS-DYNA 进程")
|
||||
except subprocess.CalledProcessError as e:
|
||||
print(f"检查进程状态失败: {e}")
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# sshcopyid 函数
|
||||
# ============================================================================
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def ssh_copy_id(
|
||||
hostname: str,
|
||||
username: str,
|
||||
password: str,
|
||||
port: int = 22,
|
||||
keypath: str = "~/.ssh/id_rsa.pub",
|
||||
timeout: int = 30,
|
||||
) -> None:
|
||||
"""将 SSH 公钥部署到远程服务器.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
hostname : str
|
||||
远程服务器主机名或 IP 地址
|
||||
username : str
|
||||
远程服务器用户名
|
||||
password : str
|
||||
远程服务器密码
|
||||
port : int
|
||||
SSH 端口, 默认 22
|
||||
keypath : str
|
||||
公钥文件路径, 默认 ~/.ssh/id_rsa.pub
|
||||
timeout : int
|
||||
SSH 操作超时秒数, 默认 30
|
||||
"""
|
||||
pub_key_path = Path(keypath).expanduser()
|
||||
if not pub_key_path.exists():
|
||||
print(f"公钥文件不存在: {pub_key_path}")
|
||||
sys.exit(1)
|
||||
|
||||
pub_key = pub_key_path.read_text().strip()
|
||||
|
||||
script = f"""mkdir -p ~/.ssh && chmod 700 ~/.ssh
|
||||
cd ~/.ssh && touch authorized_keys && chmod 600 authorized_keys
|
||||
grep -qF '{pub_key.split()[1]}' authorized_keys 2>/dev/null || echo '{pub_key}' >> authorized_keys"""
|
||||
|
||||
try:
|
||||
subprocess.run(
|
||||
[
|
||||
"sshpass",
|
||||
"-p",
|
||||
password,
|
||||
"ssh",
|
||||
"-p",
|
||||
str(port),
|
||||
"-o",
|
||||
"StrictHostKeyChecking=no",
|
||||
"-o",
|
||||
"UserKnownHostsFile=/dev/null",
|
||||
"-o",
|
||||
f"ConnectTimeout={timeout}",
|
||||
f"{username}@{hostname}",
|
||||
script,
|
||||
],
|
||||
check=True,
|
||||
timeout=timeout,
|
||||
)
|
||||
print(f"SSH 密钥已部署到 {username}@{hostname}:{port}")
|
||||
except FileNotFoundError:
|
||||
print(f"未找到 sshpass 工具, 请手动执行: ssh-copy-id -p {port} {username}@{hostname}")
|
||||
sys.exit(1)
|
||||
except subprocess.TimeoutExpired:
|
||||
print("SSH 连接超时")
|
||||
sys.exit(1)
|
||||
except subprocess.CalledProcessError as e:
|
||||
print(f"SSH 执行失败: {e}")
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# packtool 函数
|
||||
# ============================================================================
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def pack_source(project_dir: Path, output_dir: Path) -> None:
|
||||
"""打包项目源码.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
project_dir : Path
|
||||
项目目录
|
||||
output_dir : Path
|
||||
输出目录
|
||||
"""
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
pyproject_file = project_dir / "pyproject.toml"
|
||||
project_name = project_dir.name
|
||||
|
||||
if pyproject_file.exists():
|
||||
try:
|
||||
import tomllib
|
||||
|
||||
content = pyproject_file.read_text(encoding="utf-8")
|
||||
data = tomllib.loads(content)
|
||||
project_name = data.get("project", {}).get("name", project_name)
|
||||
except ImportError:
|
||||
pass
|
||||
|
||||
source_dir = output_dir / "src" / project_name
|
||||
source_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
src_subdir = project_dir / "src"
|
||||
if src_subdir.exists():
|
||||
shutil.copytree(
|
||||
src_subdir,
|
||||
source_dir / "src",
|
||||
ignore=shutil.ignore_patterns(*IGNORE_PATTERNS),
|
||||
dirs_exist_ok=True,
|
||||
)
|
||||
else:
|
||||
for item in project_dir.iterdir():
|
||||
if item.name in IGNORE_PATTERNS or item.name.startswith("."):
|
||||
continue
|
||||
dst_item = source_dir / item.name
|
||||
if item.is_dir():
|
||||
shutil.copytree(
|
||||
item,
|
||||
dst_item,
|
||||
ignore=shutil.ignore_patterns(*IGNORE_PATTERNS),
|
||||
dirs_exist_ok=True,
|
||||
)
|
||||
else:
|
||||
shutil.copy2(item, dst_item)
|
||||
|
||||
print(f"源码打包完成: {source_dir}")
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def pack_dependencies(lib_dir: Path, dependencies: list[str]) -> None:
|
||||
"""打包项目依赖.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
lib_dir : Path
|
||||
依赖库目录
|
||||
dependencies : list[str]
|
||||
依赖列表
|
||||
"""
|
||||
lib_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
if not dependencies:
|
||||
print("没有依赖需要打包")
|
||||
return
|
||||
|
||||
cmd = [
|
||||
"pip",
|
||||
"install",
|
||||
"--target",
|
||||
str(lib_dir),
|
||||
"--no-compile",
|
||||
"--no-warn-script-location",
|
||||
]
|
||||
cmd.extend(dependencies)
|
||||
|
||||
subprocess.run(cmd, check=True)
|
||||
print(f"依赖打包完成: {lib_dir}")
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def pack_wheel(project_dir: Path, output_dir: Path) -> None:
|
||||
"""打包项目为 wheel 文件.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
project_dir : Path
|
||||
项目目录
|
||||
output_dir : Path
|
||||
输出目录
|
||||
"""
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
cmd = [
|
||||
"pip",
|
||||
"wheel",
|
||||
"--no-deps",
|
||||
"--wheel-dir",
|
||||
str(output_dir),
|
||||
str(project_dir),
|
||||
]
|
||||
|
||||
subprocess.run(cmd, check=True)
|
||||
print(f"Wheel 打包完成: {output_dir}")
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def install_embed_python(version: str, output_dir: Path) -> None:
|
||||
"""安装嵌入式 Python.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
version : str
|
||||
Python 版本 (如: 3.10, 3.11)
|
||||
output_dir : Path
|
||||
输出目录
|
||||
"""
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
arch = platform.machine().lower()
|
||||
if arch in ["x86_64", "amd64"]:
|
||||
arch = "amd64"
|
||||
elif arch in ["arm64", "aarch64"]:
|
||||
arch = "arm64"
|
||||
|
||||
version_map = {
|
||||
"3.8": "3.8.10",
|
||||
"3.9": "3.9.13",
|
||||
"3.10": "3.10.11",
|
||||
"3.11": "3.11.9",
|
||||
"3.12": "3.12.4",
|
||||
}
|
||||
full_version = version_map.get(version, f"{version}.0")
|
||||
|
||||
url = f"https://www.python.org/ftp/python/{full_version}/python-{full_version}-embed-{arch}.zip"
|
||||
|
||||
cache_file = Path(DEFAULT_CACHE_DIR) / f"python-{full_version}-embed-{arch}.zip"
|
||||
cache_file.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
if not cache_file.exists():
|
||||
print(f"正在下载嵌入式 Python {full_version}...")
|
||||
urllib.request.urlretrieve(url, cache_file)
|
||||
print(f"下载完成: {cache_file}")
|
||||
|
||||
with zipfile.ZipFile(cache_file, "r") as zf:
|
||||
zf.extractall(output_dir)
|
||||
|
||||
print(f"嵌入式 Python 安装完成: {output_dir}")
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def create_zip_package(source_dir: Path, output_file: Path) -> None:
|
||||
"""创建 ZIP 打包文件.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
source_dir : Path
|
||||
源目录
|
||||
output_file : Path
|
||||
输出文件
|
||||
"""
|
||||
output_file.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
with zipfile.ZipFile(output_file, "w", zipfile.ZIP_DEFLATED) as zf:
|
||||
for file in source_dir.rglob("*"):
|
||||
if file.is_file():
|
||||
arcname = file.relative_to(source_dir)
|
||||
zf.write(file, arcname)
|
||||
|
||||
print(f"ZIP 打包完成: {output_file}")
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def clean_build_dir(build_dir: Path) -> None:
|
||||
"""清理构建目录.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
build_dir : Path
|
||||
构建目录
|
||||
"""
|
||||
if build_dir.exists():
|
||||
shutil.rmtree(build_dir)
|
||||
print(f"清理完成: {build_dir}")
|
||||
else:
|
||||
print(f"目录不存在: {build_dir}")
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# reseticoncache 函数
|
||||
# ============================================================================
|
||||
|
||||
|
||||
@px.register_fn
|
||||
def reset_icon_cache_run() -> None:
|
||||
"""重置 Windows 图标缓存.
|
||||
|
||||
执行流程: 杀掉 explorer → 删除 IconCache.db → 删除 iconcache* → 重启 explorer.
|
||||
仅在 Windows 上执行, 非 Windows 平台打印提示并跳过.
|
||||
"""
|
||||
if not Constants.IS_WINDOWS:
|
||||
print("reset_icon_cache: 仅在 Windows 上支持")
|
||||
return
|
||||
|
||||
local_app_data = os.environ.get("LOCALAPPDATA", "")
|
||||
if not local_app_data:
|
||||
print("reset_icon_cache: LOCALAPPDATA 环境变量未设置")
|
||||
return
|
||||
|
||||
icon_cache_db = Path(local_app_data) / "IconCache.db"
|
||||
explorer_cache_dir = Path(local_app_data) / "Microsoft" / "Windows" / "Explorer"
|
||||
|
||||
print("正在终止 explorer 进程...")
|
||||
subprocess.run(["taskkill", "/f", "/im", "explorer.exe"], check=False)
|
||||
|
||||
if icon_cache_db.exists():
|
||||
print(f"删除图标缓存: {icon_cache_db}")
|
||||
subprocess.run(["cmd", "/c", "del", "/a", "/q", str(icon_cache_db)], check=False)
|
||||
|
||||
if explorer_cache_dir.exists():
|
||||
print(f"清理 Explorer 缓存: {explorer_cache_dir}")
|
||||
subprocess.run(
|
||||
["cmd", "/c", "del", "/a", "/q", str(explorer_cache_dir / "iconcache*")],
|
||||
check=False,
|
||||
)
|
||||
|
||||
print("重启 explorer...")
|
||||
subprocess.run(["cmd", "/c", "start", "explorer.exe"], check=False)
|
||||
print("图标缓存已重置")
|
||||
@@ -0,0 +1,5 @@
|
||||
"""system 子包 — 系统工具."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
__all__: list[str] = []
|
||||
@@ -0,0 +1,20 @@
|
||||
"""clr - 清屏工具.
|
||||
|
||||
跨平台清屏: Windows 调用 ``cls``, Linux/macOS 调用 ``clear``.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import subprocess
|
||||
|
||||
import pyflowx as px
|
||||
from pyflowx.ops._common import platform_command
|
||||
|
||||
|
||||
@px.tool("clr", help="清屏 (跨平台)")
|
||||
def clear_screen_run() -> None:
|
||||
"""清屏 (跨平台).
|
||||
|
||||
Windows 调用 ``cls``, Linux/macOS 调用 ``clear``.
|
||||
"""
|
||||
subprocess.run(platform_command(["cls"], ["clear"]), check=False)
|
||||
@@ -0,0 +1,52 @@
|
||||
"""reseticoncache - 重置 Windows 图标缓存工具.
|
||||
|
||||
执行流程: 杀掉 explorer → 删除 IconCache.db → 删除 iconcache* → 重启 explorer.
|
||||
仅在 Windows 上执行, 非 Windows 平台打印提示并跳过.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
import pyflowx as px
|
||||
from pyflowx.conditions import Constants
|
||||
from pyflowx.ops._common import ensure_platform
|
||||
|
||||
|
||||
@px.tool("reseticoncache", help="重置 Windows 图标缓存")
|
||||
def reset_icon_cache_run() -> None:
|
||||
"""重置 Windows 图标缓存.
|
||||
|
||||
执行流程: 杀掉 explorer → 删除 IconCache.db → 删除 iconcache* → 重启 explorer.
|
||||
仅在 Windows 上执行, 非 Windows 平台打印提示并跳过.
|
||||
"""
|
||||
if not ensure_platform(Constants.IS_WINDOWS, "reset_icon_cache", "Windows"):
|
||||
return
|
||||
|
||||
local_app_data = os.environ.get("LOCALAPPDATA", "")
|
||||
if not local_app_data:
|
||||
print("reset_icon_cache: LOCALAPPDATA 环境变量未设置")
|
||||
return
|
||||
|
||||
icon_cache_db = Path(local_app_data) / "IconCache.db"
|
||||
explorer_cache_dir = Path(local_app_data) / "Microsoft" / "Windows" / "Explorer"
|
||||
|
||||
print("正在终止 explorer 进程...")
|
||||
subprocess.run(["taskkill", "/f", "/im", "explorer.exe"], check=False)
|
||||
|
||||
if icon_cache_db.exists():
|
||||
print(f"删除图标缓存: {icon_cache_db}")
|
||||
subprocess.run(["cmd", "/c", "del", "/a", "/q", str(icon_cache_db)], check=False)
|
||||
|
||||
if explorer_cache_dir.exists():
|
||||
print(f"清理 Explorer 缓存: {explorer_cache_dir}")
|
||||
subprocess.run(
|
||||
["cmd", "/c", "del", "/a", "/q", str(explorer_cache_dir / "iconcache*")],
|
||||
check=False,
|
||||
)
|
||||
|
||||
print("重启 explorer...")
|
||||
subprocess.run(["cmd", "/c", "start", "explorer.exe"], check=False)
|
||||
print("图标缓存已重置")
|
||||
@@ -0,0 +1,30 @@
|
||||
"""taskkill - 进程终止工具.
|
||||
|
||||
跨平台按名称终止进程: Windows 用 ``taskkill``, Linux/macOS 用 ``pkill``.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import subprocess
|
||||
|
||||
import pyflowx as px
|
||||
from pyflowx.ops._common import platform_command
|
||||
|
||||
|
||||
@px.tool("taskkill", help="按名称终止进程 (跨平台)")
|
||||
def taskkill_run(process_names: list[str]) -> None:
|
||||
"""按名称终止进程 (跨平台).
|
||||
|
||||
Windows 使用 ``taskkill /f /im <name>*``,
|
||||
Linux/macOS 使用 ``pkill -f <name>*``.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
process_names : list[str]
|
||||
进程名称列表 (如: ``["chrome.exe", "python"]``)
|
||||
"""
|
||||
cmd_prefix = platform_command(["taskkill", "/f", "/im"], ["pkill", "-f"])
|
||||
|
||||
for name in process_names:
|
||||
print(f"终止进程: {name}")
|
||||
subprocess.run([*cmd_prefix, f"{name}*"], check=False)
|
||||
@@ -0,0 +1,35 @@
|
||||
"""which - 命令查找工具.
|
||||
|
||||
跨平台查找可执行命令路径: Windows 用 ``where``, Linux/macOS 用 ``which``.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import subprocess
|
||||
|
||||
import pyflowx as px
|
||||
from pyflowx.ops._common import platform_command
|
||||
|
||||
|
||||
@px.tool("which", help="查找可执行命令路径 (跨平台)")
|
||||
def which_run(commands: list[str]) -> None:
|
||||
"""查找可执行命令路径 (跨平台).
|
||||
|
||||
Windows 使用 ``where``, Linux/macOS 使用 ``which``.
|
||||
对每个命令打印 ``<cmd> -> <path>`` 或 ``<cmd> -> 未找到``.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
commands : list[str]
|
||||
要查找的命令名称列表
|
||||
"""
|
||||
which_cmd = platform_command(["where"], ["which"])[0]
|
||||
|
||||
for cmd in commands:
|
||||
result = subprocess.run([which_cmd, cmd], capture_output=True, text=True, check=False)
|
||||
if result.returncode == 0:
|
||||
# Windows 的 where 可能返回多行, 取第一个
|
||||
path = result.stdout.strip().split("\n")[0].strip()
|
||||
print(f"{cmd} -> {path}")
|
||||
else:
|
||||
print(f"{cmd} -> 未找到")
|
||||
@@ -1,159 +0,0 @@
|
||||
"""函数注册表.
|
||||
|
||||
提供全局函数注册机制, 供 YAML 任务编排通过 ``fn`` 字段引用 Python 函数.
|
||||
|
||||
使用方式
|
||||
--------
|
||||
import pyflowx as px
|
||||
|
||||
@px.register_fn("pack_source")
|
||||
def pack_source(project_dir, output_dir):
|
||||
...
|
||||
|
||||
# YAML 中引用:
|
||||
# jobs:
|
||||
# pack:
|
||||
# fn: pack_source
|
||||
# args: ["./project", "./dist"]
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
from typing import Any, Callable, TypeVar, overload
|
||||
|
||||
if sys.version_info >= (3, 10):
|
||||
from typing import ParamSpec
|
||||
else:
|
||||
from typing_extensions import ParamSpec # pragma: no cover
|
||||
|
||||
__all__ = ["FnRegistry", "get_fn", "has_fn", "register_fn"]
|
||||
|
||||
P = ParamSpec("P")
|
||||
T = TypeVar("T")
|
||||
|
||||
_REGISTRY: dict[str, Callable[..., Any]] = {}
|
||||
|
||||
|
||||
@overload
|
||||
def register_fn(name: Callable[P, T]) -> Callable[P, T]: ...
|
||||
|
||||
|
||||
@overload
|
||||
def register_fn(name: str | None = None) -> Callable[[Callable[P, T]], Callable[P, T]]: ...
|
||||
|
||||
|
||||
def register_fn(name: str | Callable[..., Any] | None = None) -> Callable[..., Any]:
|
||||
"""装饰器:将函数注册到全局 registry.
|
||||
|
||||
支持两种用法::
|
||||
|
||||
@register_fn # 使用函数 __name__ 作为注册名
|
||||
def my_func(): ...
|
||||
|
||||
@register_fn("custom") # 显式指定注册名
|
||||
def my_func(): ...
|
||||
|
||||
Parameters
|
||||
----------
|
||||
name : str | Callable | None
|
||||
注册名或被装饰函数; 为 None 时使用函数 ``__name__``
|
||||
|
||||
Returns
|
||||
-------
|
||||
Callable
|
||||
装饰器函数或被装饰函数
|
||||
|
||||
Raises
|
||||
------
|
||||
ValueError
|
||||
名称已注册或无法推断函数名
|
||||
"""
|
||||
if callable(name):
|
||||
fn = name
|
||||
key = getattr(fn, "__name__", None)
|
||||
if key is None:
|
||||
raise ValueError("无法推断函数名, 请显式提供 name 参数")
|
||||
if key in _REGISTRY:
|
||||
raise ValueError(f"函数 {key!r} 已注册")
|
||||
_REGISTRY[key] = fn
|
||||
return fn
|
||||
|
||||
def decorator(fn: Callable[P, T]) -> Callable[P, T]:
|
||||
key = name if name is not None else getattr(fn, "__name__", None)
|
||||
if key is None:
|
||||
raise ValueError("无法推断函数名, 请显式提供 name 参数")
|
||||
if key in _REGISTRY:
|
||||
raise ValueError(f"函数 {key!r} 已注册")
|
||||
_REGISTRY[key] = fn
|
||||
return fn
|
||||
|
||||
return decorator
|
||||
|
||||
|
||||
def get_fn(name: str) -> Callable[..., Any]:
|
||||
"""按名称获取已注册的函数.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
name : str
|
||||
函数名
|
||||
|
||||
Returns
|
||||
-------
|
||||
Callable
|
||||
已注册的函数
|
||||
|
||||
Raises
|
||||
------
|
||||
KeyError
|
||||
函数未注册
|
||||
"""
|
||||
if name not in _REGISTRY:
|
||||
raise KeyError(f"函数 {name!r} 未注册")
|
||||
return _REGISTRY[name]
|
||||
|
||||
|
||||
def has_fn(name: str) -> bool:
|
||||
"""检查函数是否已注册.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
name : str
|
||||
函数名
|
||||
|
||||
Returns
|
||||
-------
|
||||
bool
|
||||
是否已注册
|
||||
"""
|
||||
return name in _REGISTRY
|
||||
|
||||
|
||||
class FnRegistry:
|
||||
"""函数注册表的面向对象访问接口."""
|
||||
|
||||
@staticmethod
|
||||
def register(name: str | None = None) -> Callable[[Callable[..., T]], Callable[..., T]]:
|
||||
"""注册装饰器, 等价于 :func:`register_fn`."""
|
||||
return register_fn(name)
|
||||
|
||||
@staticmethod
|
||||
def get(name: str) -> Callable[..., Any]:
|
||||
"""获取已注册函数, 等价于 :func:`get_fn`."""
|
||||
return get_fn(name)
|
||||
|
||||
@staticmethod
|
||||
def has(name: str) -> bool:
|
||||
"""检查是否已注册, 等价于 :func:`has_fn`."""
|
||||
return has_fn(name)
|
||||
|
||||
@staticmethod
|
||||
def clear() -> None:
|
||||
"""清空注册表."""
|
||||
_REGISTRY.clear()
|
||||
|
||||
@staticmethod
|
||||
def names() -> list[str]:
|
||||
"""返回所有已注册函数名."""
|
||||
return list(_REGISTRY.keys())
|
||||
@@ -6,8 +6,9 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Iterator
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any, Iterator
|
||||
from typing import Any
|
||||
|
||||
from .task import TaskResult, TaskStatus
|
||||
|
||||
|
||||
@@ -14,9 +14,10 @@ from __future__ import annotations
|
||||
import argparse
|
||||
import enum
|
||||
import sys
|
||||
from collections.abc import Sequence
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import Any, Sequence, get_args
|
||||
from typing import Any, get_args
|
||||
|
||||
from .compose import GraphComposer
|
||||
from .errors import PyFlowXError
|
||||
|
||||
@@ -17,10 +17,10 @@ import json
|
||||
import sys
|
||||
import time
|
||||
from abc import ABC, abstractmethod
|
||||
from collections.abc import Iterator
|
||||
from contextlib import contextmanager, nullcontext
|
||||
from collections.abc import Iterator, Mapping
|
||||
from contextlib import AbstractContextManager, contextmanager, nullcontext
|
||||
from pathlib import Path
|
||||
from typing import Any, ContextManager, Mapping
|
||||
from typing import Any
|
||||
|
||||
if sys.version_info >= (3, 12):
|
||||
from typing import override
|
||||
@@ -63,7 +63,7 @@ class StateBackend(ABC):
|
||||
:class:`JSONBackend` 在 :meth:`batch` 期间会延迟落盘,需在退出时调用。
|
||||
"""
|
||||
|
||||
def batch(self) -> ContextManager[None]:
|
||||
def batch(self) -> AbstractContextManager[None]:
|
||||
"""返回一个上下文管理器,期间 :meth:`save` 可延迟 :meth:`flush`。
|
||||
|
||||
默认实现为 no-op(如 :class:`MemoryBackend`)。:class:`JSONBackend`
|
||||
|
||||
+6
-19
@@ -22,7 +22,8 @@ import os
|
||||
import shutil
|
||||
import sys
|
||||
import threading
|
||||
from contextlib import contextmanager
|
||||
from collections.abc import Callable, Coroutine, Generator, Mapping
|
||||
from contextlib import AbstractContextManager, contextmanager
|
||||
from dataclasses import dataclass, field
|
||||
from datetime import datetime
|
||||
from enum import Enum
|
||||
@@ -30,14 +31,7 @@ from functools import cached_property
|
||||
from pathlib import Path
|
||||
from typing import (
|
||||
Any,
|
||||
Callable,
|
||||
ContextManager,
|
||||
Coroutine,
|
||||
Generator,
|
||||
Generic,
|
||||
List,
|
||||
Mapping,
|
||||
Union,
|
||||
cast,
|
||||
)
|
||||
|
||||
@@ -49,25 +43,18 @@ else:
|
||||
T = TypeVar("T", default=Any)
|
||||
|
||||
# 任务可调用对象可以是同步或异步的。显式保留联合类型,让 mypy 理解两种形态。
|
||||
TaskFn = Union[
|
||||
Callable[..., T],
|
||||
Callable[..., Coroutine[Any, Any, T]],
|
||||
]
|
||||
TaskFn = Callable[..., T] | Callable[..., Coroutine[Any, Any, T]]
|
||||
|
||||
# 跨任务结果映射。值刻意使用 ``Any``,因为不同任务返回不同类型;
|
||||
# 单任务类型由函数签名本身保留。
|
||||
Context = Mapping[str, Any]
|
||||
|
||||
# 命令类型支持
|
||||
TaskCmd = Union[
|
||||
List[str], # 命令列表, 如 ["ls", "-la"]
|
||||
str, # shell 命令字符串
|
||||
Callable[..., Any], # Python 函数
|
||||
]
|
||||
TaskCmd = list[str] | str | Callable[..., Any]
|
||||
|
||||
# 执行策略:sequential/thread/async 为层屏障模型,dependency 为依赖驱动模型。
|
||||
Strategy = Union[str, "StrategyKind"]
|
||||
StrategyKind = Any # 占位,避免循环;executors 模块用 Literal 约束
|
||||
Strategy = str | StrategyKind
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -376,7 +363,7 @@ class TaskSpec(Generic[T]):
|
||||
return shutil.which(cmd[0]) is not None
|
||||
return True
|
||||
|
||||
def env_context(self) -> ContextManager[None]:
|
||||
def env_context(self) -> AbstractContextManager[None]:
|
||||
"""返回临时应用 ``env`` 与 ``cwd`` 的上下文管理器。
|
||||
|
||||
对 ``fn`` 任务生效。``cmd`` 任务在 :func:`_run_command` 中直接
|
||||
|
||||
@@ -0,0 +1,578 @@
|
||||
"""@px.tool 装饰器: typer 驱动 CLI + DAG 编排.
|
||||
|
||||
替代 YAML 配置模式, 用 .py 装饰器统一描述工具.
|
||||
函数签名 → typer 自动生成 CLI, 函数体即任务逻辑, needs/strategy 表达 DAG.
|
||||
|
||||
示例
|
||||
----
|
||||
::
|
||||
@px.tool("pdftool", subcommand="m", help="合并 PDF")
|
||||
def pdf_merge(
|
||||
inputs: list[Path], # positional, nargs="+"
|
||||
output: Path = Path("merged.pdf"), # --output 选项
|
||||
) -> None:
|
||||
... # 函数体即逻辑
|
||||
|
||||
# CLI: pf pdftool m a.pdf b.pdf --output merged.pdf
|
||||
|
||||
cmd 任务 (执行外部命令, 函数体不执行)::
|
||||
|
||||
@px.tool("pymake", subcommand="b", help="构建", cmd=["uv", "build"])
|
||||
def b(cwd: Path = Path(".")) -> None:
|
||||
pass # 签名仅驱动 CLI, cwd 注入到 cmd 工作目录
|
||||
|
||||
聚合任务 (有 needs 无 cmd 无函数逻辑)::
|
||||
|
||||
@px.tool("pymake", subcommand="tc", help="类型检查",
|
||||
needs=["c", "pyrefly_check", "lint"], strategy="thread")
|
||||
def tc(cwd: Path = Path(".")) -> None:
|
||||
pass # 仅作依赖聚合点
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
__all__ = [
|
||||
"ToolExitCode",
|
||||
"ToolSpec",
|
||||
"clear_tool_registry",
|
||||
"get_tool",
|
||||
"list_subcommands",
|
||||
"list_tools",
|
||||
"run_tool",
|
||||
"tool",
|
||||
]
|
||||
|
||||
import ast
|
||||
import enum
|
||||
import inspect
|
||||
import sys
|
||||
import textwrap
|
||||
from collections.abc import Callable, Mapping, Sequence
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Any, Literal, cast
|
||||
|
||||
import typer
|
||||
from rich.console import Console
|
||||
from rich.table import Table
|
||||
|
||||
# typer 内置 click 异常 (typer._click 与公共 click 是两套独立类层次)
|
||||
from typer._click.exceptions import ClickException as _TyperClickException
|
||||
from typer._click.exceptions import NoArgsIsHelpError as _NoArgsIsHelpError
|
||||
from typer.rich_utils import rich_format_error as _rich_format_error
|
||||
|
||||
from pyflowx.errors import PyFlowXError
|
||||
from pyflowx.executors import run
|
||||
from pyflowx.graph import Graph
|
||||
from pyflowx.task import RetryPolicy, TaskSpec
|
||||
|
||||
|
||||
class ToolExitCode(enum.IntEnum):
|
||||
"""工具执行退出码."""
|
||||
|
||||
SUCCESS = 0
|
||||
FAILURE = 1
|
||||
INTERRUPTED = 130 # 与 POSIX 信号中断一致
|
||||
|
||||
|
||||
def _noop() -> None:
|
||||
"""聚合任务的占位函数."""
|
||||
return None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
# ToolSpec: 工具描述符
|
||||
# ---------------------------------------------------------------------- #
|
||||
@dataclass(frozen=True)
|
||||
class ToolSpec:
|
||||
"""工具描述符: 由 ``@px.tool`` 装饰器注册.
|
||||
|
||||
封装函数 + CLI 元数据 + DAG 编排参数, 运行时映射为 :class:`TaskSpec`.
|
||||
"""
|
||||
|
||||
name: str # 工具名 (如 "pdftool")
|
||||
subcommand: str | None # 子命令名 (None=单命令工具)
|
||||
func: Callable[..., Any] # 函数 (签名驱动 CLI, 体即逻辑)
|
||||
help: str = "" # 帮助文本
|
||||
description: str = "" # 工具描述 (pf 列表用)
|
||||
cmd: tuple[str, ...] | str | None = None # 命令 (有则执行 cmd, 无则执行函数体)
|
||||
needs: tuple[str, ...] = () # 依赖任务 (引用同 tool 的 subcommand)
|
||||
strategy: Literal["sequential", "thread", "async", "dependency"] | None = None # 执行策略
|
||||
cwd: str | Path | None = None # 工作目录 (cmd 任务, 装饰器级默认)
|
||||
allow_upstream_skip: bool = False
|
||||
hidden: bool = False # 不暴露为 subcommand (内部 job)
|
||||
env: Mapping[str, str] | None = None
|
||||
retry: RetryPolicy | None = None
|
||||
timeout: float | None = None
|
||||
|
||||
|
||||
# 全局工具注册表: {tool_name: {subcommand: ToolSpec}}
|
||||
_TOOL_REGISTRY: dict[str, dict[str | None, ToolSpec]] = {}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
# @tool 装饰器 + 注册表
|
||||
# ---------------------------------------------------------------------- #
|
||||
def tool(
|
||||
name: str,
|
||||
*,
|
||||
subcommand: str | None = None,
|
||||
help: str = "",
|
||||
description: str = "",
|
||||
cmd: Sequence[str] | str | None = None,
|
||||
needs: Sequence[str] | None = None,
|
||||
strategy: Literal["sequential", "thread", "async", "dependency"] | None = None,
|
||||
cwd: str | Path | None = None,
|
||||
allow_upstream_skip: bool = False,
|
||||
hidden: bool = False,
|
||||
env: Mapping[str, str] | None = None,
|
||||
retry: RetryPolicy | None = None,
|
||||
timeout: float | None = None,
|
||||
) -> Callable[[Callable[..., Any]], Callable[..., Any]]:
|
||||
"""装饰器: 将函数注册为 ``@px.tool`` 工具.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
name : str
|
||||
工具名 (如 "pdftool"); 多个 ``@px.tool`` 共用同名即多 subcommand 工具
|
||||
subcommand : str | None
|
||||
子命令名; ``None`` 表示单命令工具 (整个工具仅一个函数)
|
||||
help : str
|
||||
子命令帮助文本; 默认用函数 docstring
|
||||
description : str
|
||||
工具描述, 用于 pf 工具列表
|
||||
cmd : Sequence[str] | str | None
|
||||
命令列表或 shell 字符串; 有 ``cmd`` 执行命令, 函数体不执行 (签名仅驱动 CLI)
|
||||
needs : Sequence[str] | None
|
||||
依赖任务名 (引用同 tool 的其他 subcommand)
|
||||
strategy : str | None
|
||||
执行策略: ``sequential`` / ``thread`` / ``async`` / ``dependency``
|
||||
cwd : str | Path | None
|
||||
工作目录 (cmd 任务装饰器级默认); 若函数签名有 ``cwd`` 参数则被 CLI 值覆盖
|
||||
allow_upstream_skip : bool
|
||||
上游 SKIPPED 时本任务仍执行
|
||||
hidden : bool
|
||||
不暴露为 subcommand (内部 job, 仅被 needs 引用)
|
||||
env / retry / timeout
|
||||
透传 :class:`TaskSpec` 对应字段
|
||||
"""
|
||||
|
||||
def decorator(func: Callable[..., Any]) -> Callable[..., Any]:
|
||||
spec = ToolSpec(
|
||||
name=name,
|
||||
subcommand=subcommand,
|
||||
func=func,
|
||||
help=help or inspect.getdoc(func) or "",
|
||||
description=description,
|
||||
cmd=cast("tuple[str, ...] | str | None", tuple(cmd) if isinstance(cmd, (list, tuple)) else cmd),
|
||||
needs=tuple(needs) if needs else (),
|
||||
strategy=strategy,
|
||||
cwd=cwd,
|
||||
allow_upstream_skip=allow_upstream_skip,
|
||||
hidden=hidden,
|
||||
env=dict(env) if env else None,
|
||||
retry=retry,
|
||||
timeout=timeout,
|
||||
)
|
||||
_register_tool(spec)
|
||||
return func
|
||||
|
||||
return decorator
|
||||
|
||||
|
||||
def _register_tool(spec: ToolSpec) -> None:
|
||||
"""注册工具到全局注册表, 校验重复."""
|
||||
if spec.name not in _TOOL_REGISTRY:
|
||||
_TOOL_REGISTRY[spec.name] = {}
|
||||
if spec.subcommand in _TOOL_REGISTRY[spec.name]:
|
||||
raise ValueError(f"工具 {spec.name!r} 的子命令 {spec.subcommand!r} 已注册")
|
||||
_TOOL_REGISTRY[spec.name][spec.subcommand] = spec
|
||||
|
||||
|
||||
def get_tool(name: str, subcommand: str | None = None) -> ToolSpec:
|
||||
"""获取已注册工具.
|
||||
|
||||
Raises
|
||||
------
|
||||
KeyError
|
||||
工具或子命令未注册
|
||||
"""
|
||||
if name not in _TOOL_REGISTRY:
|
||||
raise KeyError(f"工具 {name!r} 未注册")
|
||||
subs = _TOOL_REGISTRY[name]
|
||||
if subcommand not in subs:
|
||||
raise KeyError(f"工具 {name!r} 没有子命令 {subcommand!r}")
|
||||
return subs[subcommand]
|
||||
|
||||
|
||||
def list_tools() -> list[str]:
|
||||
"""列出所有已注册工具名."""
|
||||
return sorted(_TOOL_REGISTRY.keys())
|
||||
|
||||
|
||||
def list_subcommands(name: str, include_hidden: bool = False) -> list[str]:
|
||||
"""列出工具的子命令 (hidden 默认排除).
|
||||
|
||||
单命令工具 (subcommand=None) 返回空列表.
|
||||
"""
|
||||
if name not in _TOOL_REGISTRY:
|
||||
return []
|
||||
return sorted(
|
||||
sc for sc, spec in _TOOL_REGISTRY[name].items() if sc is not None and (include_hidden or not spec.hidden)
|
||||
)
|
||||
|
||||
|
||||
def clear_tool_registry() -> None:
|
||||
"""清空注册表 (测试用)."""
|
||||
_TOOL_REGISTRY.clear()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
# 依赖收集 + TaskSpec 构建
|
||||
# ---------------------------------------------------------------------- #
|
||||
def _collect_with_deps(name: str, target: str | None) -> list[str | None]:
|
||||
"""BFS 收集 target 及其传递依赖 (subcommand 名).
|
||||
|
||||
返回顺序: 依赖在前, target 在后 (符合 DAG 拓扑).
|
||||
"""
|
||||
if name not in _TOOL_REGISTRY:
|
||||
return [target]
|
||||
subs = _TOOL_REGISTRY[name]
|
||||
result: list[str | None] = []
|
||||
seen: set[str | None] = set()
|
||||
queue: list[str | None] = [target]
|
||||
while queue:
|
||||
sc = queue.pop(0)
|
||||
if sc in seen:
|
||||
continue
|
||||
seen.add(sc)
|
||||
result.append(sc)
|
||||
if sc in subs:
|
||||
queue.extend(subs[sc].needs)
|
||||
# 反转: 依赖在前, target 在后
|
||||
result.reverse()
|
||||
return result
|
||||
|
||||
|
||||
def _has_function_logic(func: Callable[..., Any]) -> bool:
|
||||
"""判断函数体是否有实际逻辑 (非 pass/.../docstring).
|
||||
|
||||
用 ast 分析, 避免 exec 函数.
|
||||
"""
|
||||
try:
|
||||
src = inspect.getsource(func)
|
||||
src = textwrap.dedent(src)
|
||||
tree = ast.parse(src)
|
||||
except (OSError, TypeError, SyntaxError): # pragma: no cover
|
||||
return True
|
||||
func_def = next((n for n in ast.walk(tree) if isinstance(n, (ast.FunctionDef, ast.AsyncFunctionDef))), None)
|
||||
if func_def is None: # pragma: no cover
|
||||
return True
|
||||
for stmt in func_def.body:
|
||||
if isinstance(stmt, ast.Expr) and isinstance(stmt.value, ast.Constant):
|
||||
continue # docstring
|
||||
if isinstance(stmt, ast.Pass):
|
||||
continue
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def _is_aggregate(spec: ToolSpec) -> bool:
|
||||
"""判断是否为聚合任务 (有 needs 无 cmd 无函数逻辑)."""
|
||||
if spec.cmd is not None or not spec.needs:
|
||||
return False
|
||||
return not _has_function_logic(spec.func)
|
||||
|
||||
|
||||
def _build_task_spec(spec: ToolSpec, variables: Mapping[str, Any]) -> TaskSpec:
|
||||
"""将 ToolSpec + 解析后的变量转为 TaskSpec.
|
||||
|
||||
- cmd 任务: 执行命令, cwd 从 variables["cwd"] 或装饰器 cwd 取
|
||||
- 聚合任务 (有 needs 无 cmd 无函数逻辑): fn=noop
|
||||
- fn 任务: 执行函数, kwargs 按签名从 variables 取
|
||||
"""
|
||||
task_name = spec.subcommand if spec.subcommand is not None else spec.name
|
||||
|
||||
# cmd 任务
|
||||
if spec.cmd is not None:
|
||||
cwd_value = variables.get("cwd", spec.cwd)
|
||||
cwd = Path(cwd_value) if cwd_value is not None else None
|
||||
cmd_value: Any = list(spec.cmd) if isinstance(spec.cmd, tuple) else spec.cmd
|
||||
return TaskSpec(
|
||||
name=task_name,
|
||||
cmd=cmd_value,
|
||||
depends_on=spec.needs,
|
||||
cwd=cwd,
|
||||
env=spec.env,
|
||||
retry=spec.retry or RetryPolicy(),
|
||||
timeout=spec.timeout,
|
||||
allow_upstream_skip=spec.allow_upstream_skip,
|
||||
strategy=spec.strategy,
|
||||
)
|
||||
|
||||
# 聚合任务
|
||||
if _is_aggregate(spec):
|
||||
return TaskSpec(
|
||||
name=task_name,
|
||||
fn=_noop,
|
||||
depends_on=spec.needs,
|
||||
allow_upstream_skip=spec.allow_upstream_skip,
|
||||
strategy=spec.strategy,
|
||||
)
|
||||
|
||||
# fn 任务
|
||||
sig = inspect.signature(spec.func)
|
||||
kwargs: dict[str, Any] = {}
|
||||
for pname in sig.parameters:
|
||||
if pname in variables:
|
||||
kwargs[pname] = variables[pname]
|
||||
cwd_value = variables.get("cwd")
|
||||
cwd = Path(cwd_value) if cwd_value is not None else None
|
||||
return TaskSpec(
|
||||
name=task_name,
|
||||
fn=spec.func,
|
||||
kwargs=kwargs,
|
||||
depends_on=spec.needs,
|
||||
cwd=cwd,
|
||||
env=spec.env,
|
||||
retry=spec.retry or RetryPolicy(),
|
||||
timeout=spec.timeout,
|
||||
allow_upstream_skip=spec.allow_upstream_skip,
|
||||
strategy=spec.strategy,
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
# Typer app 构建 (函数签名 → typer 命令)
|
||||
# ---------------------------------------------------------------------- #
|
||||
def _global_option_params() -> list[inspect.Parameter]:
|
||||
"""构建全局选项的 inspect.Parameter 列表.
|
||||
|
||||
各命令 dispatcher 追加这些参数到用户函数签名末尾, 使 ``--dry-run`` / ``--quiet`` /
|
||||
``--strategy`` 出现在每个子命令上. ``--list`` 由 :func:`run_tool` 预处理.
|
||||
"""
|
||||
return [
|
||||
inspect.Parameter(
|
||||
"dry_run",
|
||||
inspect.Parameter.KEYWORD_ONLY,
|
||||
default=typer.Option(False, "--dry-run", help="仅打印执行计划, 不执行"),
|
||||
annotation=bool,
|
||||
),
|
||||
inspect.Parameter(
|
||||
"quiet",
|
||||
inspect.Parameter.KEYWORD_ONLY,
|
||||
default=typer.Option(False, "--quiet", "-q", help="减少输出"),
|
||||
annotation=bool,
|
||||
),
|
||||
inspect.Parameter(
|
||||
"strategy",
|
||||
inspect.Parameter.KEYWORD_ONLY,
|
||||
default=typer.Option(None, "--strategy", help="执行策略 (sequential/thread/dependency)"),
|
||||
annotation=str | None,
|
||||
),
|
||||
]
|
||||
|
||||
|
||||
def _make_dispatcher(tool_name: str, spec: ToolSpec) -> Callable[..., None]:
|
||||
"""为 spec 创建 typer 命令 dispatcher.
|
||||
|
||||
dispatcher 签名 = 用户函数签名 (过滤 ``*args``/``**kwargs``) + 全局选项;
|
||||
body 从 kwargs 分离全局选项后调用 :func:`_execute_dag`.
|
||||
"""
|
||||
func = spec.func
|
||||
subcommand = spec.subcommand
|
||||
user_sig = inspect.signature(func)
|
||||
|
||||
# 过滤 *args / **kwargs, 追加全局选项
|
||||
params = [
|
||||
p
|
||||
for p in user_sig.parameters.values()
|
||||
if p.kind not in (inspect.Parameter.VAR_POSITIONAL, inspect.Parameter.VAR_KEYWORD)
|
||||
]
|
||||
params.extend(_global_option_params())
|
||||
combined_sig = user_sig.replace(parameters=params)
|
||||
|
||||
def dispatcher(**kwargs: Any) -> None:
|
||||
global_args = {
|
||||
"dry_run": kwargs.pop("dry_run", False),
|
||||
"verbose": not kwargs.pop("quiet", False),
|
||||
"strategy": kwargs.pop("strategy", None),
|
||||
}
|
||||
_execute_dag(tool_name, subcommand, kwargs, global_args)
|
||||
|
||||
dispatcher.__signature__ = combined_sig # type: ignore[attr-defined]
|
||||
dispatcher.__doc__ = spec.help or func.__doc__ or ""
|
||||
dispatcher.__name__ = func.__name__
|
||||
dispatcher.__annotations__ = {p.name: p.annotation for p in params}
|
||||
return dispatcher
|
||||
|
||||
|
||||
def _build_typer_app(name: str, all_subs: dict[str | None, ToolSpec]) -> typer.Typer:
|
||||
"""构建 typer app.
|
||||
|
||||
单命令工具 (仅 subcommand=None) → callback(invoke_without_command=True);
|
||||
多 subcommand 工具 → 各可见 subcommand 注册为命令 (``--list`` 由 run_tool 预处理).
|
||||
"""
|
||||
first_spec = next(iter(all_subs.values()))
|
||||
description = first_spec.description or first_spec.help or name
|
||||
is_single = None in all_subs and len(all_subs) == 1
|
||||
|
||||
app = typer.Typer(
|
||||
name=f"pf {name}",
|
||||
help=description,
|
||||
# 单命令工具: 无参数时直接执行 callback; 多命令工具: 无 subcommand 时打印 help
|
||||
no_args_is_help=not is_single,
|
||||
add_completion=False,
|
||||
rich_markup_mode="rich",
|
||||
)
|
||||
|
||||
if is_single:
|
||||
spec = all_subs[None]
|
||||
dispatcher = _make_dispatcher(name, spec)
|
||||
app.callback(invoke_without_command=True)(dispatcher)
|
||||
return app
|
||||
|
||||
@app.callback()
|
||||
def _callback() -> None:
|
||||
"""工具入口; ``--list`` 由 :func:`run_tool` 预处理."""
|
||||
|
||||
for sc, spec in sorted(all_subs.items(), key=lambda x: x[0] or ""):
|
||||
if sc is None or spec.hidden:
|
||||
continue
|
||||
dispatcher = _make_dispatcher(name, spec)
|
||||
app.command(name=sc, help=spec.help)(dispatcher)
|
||||
|
||||
return app
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
# 执行入口
|
||||
# ---------------------------------------------------------------------- #
|
||||
def _execute_dag(
|
||||
tool_name: str,
|
||||
subcommand: str | None,
|
||||
variables: dict[str, Any],
|
||||
global_args: dict[str, Any],
|
||||
) -> None:
|
||||
"""收集依赖, 构建 Graph, 执行 DAG.
|
||||
|
||||
成功正常返回; 失败时打印 rich 错误并 ``raise typer.Exit``.
|
||||
"""
|
||||
all_subs = _TOOL_REGISTRY[tool_name]
|
||||
|
||||
if subcommand not in all_subs:
|
||||
Console(stderr=True).print(f"[red]错误:[/red] 未知子命令 [yellow]{subcommand!r}[/yellow]")
|
||||
raise typer.Exit(ToolExitCode.FAILURE)
|
||||
|
||||
target_spec = all_subs[subcommand]
|
||||
|
||||
# 收集依赖 + 构建子图
|
||||
all_names = _collect_with_deps(tool_name, subcommand)
|
||||
specs: list[TaskSpec] = []
|
||||
for sc in all_names:
|
||||
dep_spec = all_subs[sc]
|
||||
specs.append(_build_task_spec(dep_spec, variables))
|
||||
|
||||
graph = Graph.from_specs(specs)
|
||||
|
||||
console = Console()
|
||||
console.print(f"[bold cyan][{tool_name}][/bold cyan] 执行: [yellow]{subcommand or '全部任务'}[/yellow]")
|
||||
|
||||
try:
|
||||
run(
|
||||
graph,
|
||||
strategy=global_args["strategy"] or target_spec.strategy or "dependency",
|
||||
dry_run=global_args["dry_run"],
|
||||
verbose=global_args["verbose"],
|
||||
)
|
||||
except PyFlowXError as e:
|
||||
Console(stderr=True).print(f"[red]错误:[/red] {e}")
|
||||
raise typer.Exit(ToolExitCode.FAILURE) from e
|
||||
except KeyboardInterrupt:
|
||||
Console(stderr=True).print("\n[yellow]已中断[/yellow]")
|
||||
raise typer.Exit(ToolExitCode.INTERRUPTED) from None
|
||||
|
||||
|
||||
def _handle_list_rich(name: str, all_subs: dict[str | None, ToolSpec]) -> None:
|
||||
"""rich 表格打印任务列表."""
|
||||
console = Console()
|
||||
console.print(f"[bold]工具 {name} 的任务列表[/bold]")
|
||||
table = Table(show_header=True, header_style="bold")
|
||||
table.add_column("任务")
|
||||
table.add_column("依赖")
|
||||
table.add_column("类型")
|
||||
table.add_column("状态")
|
||||
for sc, spec in sorted(all_subs.items(), key=lambda x: (x[0] is not None, x[0] or "")):
|
||||
display = sc if sc is not None else name
|
||||
deps = ", ".join(spec.needs) if spec.needs else "—"
|
||||
if spec.cmd is not None:
|
||||
task_type = "cmd"
|
||||
elif _is_aggregate(spec):
|
||||
task_type = "聚合"
|
||||
else:
|
||||
task_type = "fn"
|
||||
hidden = "[dim]隐藏[/dim]" if spec.hidden else ""
|
||||
table.add_row(display, deps, task_type, hidden)
|
||||
console.print(table)
|
||||
|
||||
|
||||
def run_tool(name: str, argv: Sequence[str] | None = None) -> int:
|
||||
"""执行工具, 返回退出码 (0 成功 / 1 失败 / 130 中断).
|
||||
|
||||
Parameters
|
||||
----------
|
||||
name : str
|
||||
工具名 (已通过 ``@px.tool`` 注册)
|
||||
argv : Sequence[str] | None
|
||||
命令行参数 (不含工具名); None 用 ``sys.argv[1:]``
|
||||
|
||||
Returns
|
||||
-------
|
||||
int
|
||||
退出码
|
||||
"""
|
||||
if name not in _TOOL_REGISTRY:
|
||||
Console(stderr=True).print(f"[red]错误:[/red] 未注册工具 [yellow]{name!r}[/yellow]")
|
||||
return ToolExitCode.FAILURE
|
||||
|
||||
all_subs = _TOOL_REGISTRY[name]
|
||||
argv = list(argv) if argv is not None else sys.argv[1:]
|
||||
|
||||
# --list 预处理: 出现在任意位置即列出任务后退出
|
||||
if "--list" in argv:
|
||||
_handle_list_rich(name, all_subs)
|
||||
return ToolExitCode.SUCCESS
|
||||
|
||||
app = _build_typer_app(name, all_subs)
|
||||
|
||||
try:
|
||||
# standalone_mode=False: click 捕获 typer.Exit 后返回 exit_code (而非抛出);
|
||||
# ClickException/Abort 则向上抛出, 由下方 except 分支处理
|
||||
result = app(args=argv, standalone_mode=False)
|
||||
# 正常完成返回 None; typer.Exit 返回 int exit_code
|
||||
if isinstance(result, int) and result != 0:
|
||||
return (
|
||||
ToolExitCode(result)
|
||||
if result in (ToolExitCode.SUCCESS, ToolExitCode.FAILURE, ToolExitCode.INTERRUPTED)
|
||||
else ToolExitCode.FAILURE
|
||||
)
|
||||
return ToolExitCode.SUCCESS
|
||||
except _NoArgsIsHelpError as e:
|
||||
# 多命令工具无 subcommand → 打印 help, 视为成功
|
||||
# rich_format_error 对 NoArgsIsHelpError 直接 return, 需手动打印 help
|
||||
if e.message:
|
||||
Console().print(e.message, highlight=False, markup=False)
|
||||
return ToolExitCode.SUCCESS
|
||||
except _TyperClickException as e:
|
||||
# standalone_mode=False 时不自动打印错误, 需用 rich 格式打印
|
||||
_rich_format_error(e)
|
||||
return ToolExitCode.SUCCESS if e.exit_code == 0 else ToolExitCode.FAILURE
|
||||
except typer.Abort:
|
||||
return ToolExitCode.INTERRUPTED
|
||||
except SystemExit as e:
|
||||
code = e.code
|
||||
if isinstance(code, int):
|
||||
return code
|
||||
return ToolExitCode.SUCCESS if code in (None, "0") else ToolExitCode.FAILURE
|
||||
except KeyboardInterrupt:
|
||||
return ToolExitCode.INTERRUPTED
|
||||
File diff suppressed because it is too large
Load Diff
@@ -5,7 +5,7 @@ from __future__ import annotations
|
||||
from pathlib import Path
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
from pyflowx.ops import dev
|
||||
from pyflowx.ops.dev import autofmt as dev
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
|
||||
@@ -7,7 +7,7 @@ from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from pyflowx.ops import bumpversion
|
||||
from pyflowx.ops.dev import bumpversion
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
@@ -142,7 +142,9 @@ class TestBumpProjectVersion:
|
||||
|
||||
add_calls = [c for c in calls if c[:2] == ["git", "add"]]
|
||||
assert len(add_calls) == 1
|
||||
assert "src/pkg/__init__.py" in add_calls[0]
|
||||
# 跨平台: Windows 上 Path 转换为反斜杠, 统一用正斜杠比较
|
||||
init_path = str(init_file.relative_to(tmp_path)).replace("\\", "/")
|
||||
assert init_path in [arg.replace("\\", "/") for arg in add_calls[0]]
|
||||
assert "pyproject.toml" in add_calls[0]
|
||||
assert "." not in add_calls[0][2:]
|
||||
|
||||
|
||||
@@ -1,21 +0,0 @@
|
||||
"""Tests for cli.clearscreen module."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from unittest.mock import patch
|
||||
|
||||
import pyflowx as px
|
||||
from pyflowx.cli.system import clearscreen
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
# main function
|
||||
# ---------------------------------------------------------------------- #
|
||||
class TestMain:
|
||||
"""Test main function."""
|
||||
|
||||
def test_main_creates_graph_and_runs(self) -> None:
|
||||
"""main() should create a Graph and run it."""
|
||||
with patch.object(px, "run") as mock_run:
|
||||
clearscreen.main()
|
||||
assert mock_run.called
|
||||
@@ -7,7 +7,7 @@ from io import BytesIO
|
||||
from pathlib import Path
|
||||
from unittest.mock import Mock, patch
|
||||
|
||||
from pyflowx.cli import emlmanager
|
||||
from pyflowx.cli.legacy import emlmanager
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
@@ -884,9 +884,11 @@ Date: Mon, {i + 1} Jan 2024 12:00:00 +0000
|
||||
Body {i}
|
||||
""")
|
||||
|
||||
with patch("sys.argv", ["emlmanager", "--dir", str(tmp_path), "--port", "8080"]), patch.object(
|
||||
emlmanager, "ThreadingHTTPServer"
|
||||
) as mock_server, patch("threading.Thread"):
|
||||
with (
|
||||
patch("sys.argv", ["emlmanager", "--dir", str(tmp_path), "--port", "8080"]),
|
||||
patch.object(emlmanager, "ThreadingHTTPServer") as mock_server,
|
||||
patch("threading.Thread"),
|
||||
):
|
||||
# Don't actually start the server
|
||||
mock_server_instance = Mock()
|
||||
mock_server.return_value = mock_server_instance
|
||||
|
||||
@@ -0,0 +1,389 @@
|
||||
"""Tests for ops.envdev / ops.dockercmd 模块 (镜像源配置/Docker 登录)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
import pytest
|
||||
|
||||
from pyflowx.conditions import Constants
|
||||
from pyflowx.ops.infra.dockercmd import docker_login_tencent
|
||||
from pyflowx.ops.infra.envdev import (
|
||||
download_rustup_script,
|
||||
install_linux_docker,
|
||||
install_linux_fonts,
|
||||
install_linux_qt_libs,
|
||||
install_rust_toolchain,
|
||||
setup_conda_mirror,
|
||||
setup_linux_system_mirror,
|
||||
setup_python_mirror,
|
||||
setup_rust_mirror,
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
# setup_python_mirror
|
||||
# ---------------------------------------------------------------------- #
|
||||
class TestSetupPythonMirror:
|
||||
"""``setup_python_mirror`` 函数测试."""
|
||||
|
||||
def test_unknown_mirror_skips(self, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]) -> None:
|
||||
"""未知镜像源应打印提示并跳过."""
|
||||
monkeypatch.setattr(subprocess, "run", lambda *_, **__: MagicMock())
|
||||
setup_python_mirror("unknown_mirror")
|
||||
captured = capsys.readouterr()
|
||||
assert "未知 Python 镜像源" in captured.out
|
||||
|
||||
def test_known_mirror_writes_config(self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None:
|
||||
"""已知镜像源应写入配置文件."""
|
||||
monkeypatch.setattr(Path, "home", lambda: tmp_path)
|
||||
setup_python_mirror("tsinghua")
|
||||
# Linux 平台默认配置路径
|
||||
config_path = tmp_path / ".pip" / "pip.conf"
|
||||
if not config_path.exists():
|
||||
config_path = tmp_path / "pip" / "pip.ini"
|
||||
assert config_path.exists()
|
||||
content = config_path.read_text(encoding="utf-8")
|
||||
assert "pypi.tuna.tsinghua.edu.cn" in content
|
||||
|
||||
def test_linux_uses_pip_conf(self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None:
|
||||
"""Linux 平台应写入 ~/.pip/pip.conf."""
|
||||
monkeypatch.setattr(Constants, "IS_LINUX", True)
|
||||
monkeypatch.setattr(Path, "home", lambda: tmp_path)
|
||||
setup_python_mirror("tsinghua")
|
||||
config_path = tmp_path / ".pip" / "pip.conf"
|
||||
assert config_path.exists()
|
||||
|
||||
def test_non_linux_uses_pip_ini(self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None:
|
||||
"""非 Linux 平台应写入 ~/pip/pip.ini."""
|
||||
monkeypatch.setattr(Constants, "IS_LINUX", False)
|
||||
monkeypatch.setattr(Path, "home", lambda: tmp_path)
|
||||
setup_python_mirror("tsinghua")
|
||||
config_path = tmp_path / "pip" / "pip.ini"
|
||||
assert config_path.exists()
|
||||
|
||||
def test_sets_env_vars(self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None:
|
||||
"""应设置 PIP_INDEX_URL 等环境变量."""
|
||||
import os
|
||||
|
||||
monkeypatch.setattr(Path, "home", lambda: tmp_path)
|
||||
monkeypatch.setattr(os, "environ", {})
|
||||
|
||||
setup_python_mirror("aliyun")
|
||||
|
||||
assert "PIP_INDEX_URL" in os.environ
|
||||
assert "mirrors.aliyun.com" in os.environ["PIP_INDEX_URL"]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
# setup_conda_mirror
|
||||
# ---------------------------------------------------------------------- #
|
||||
class TestSetupCondaMirror:
|
||||
"""``setup_conda_mirror`` 函数测试."""
|
||||
|
||||
def test_unknown_mirror_skips(self, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]) -> None:
|
||||
"""未知镜像源应跳过."""
|
||||
setup_conda_mirror("unknown")
|
||||
captured = capsys.readouterr()
|
||||
assert "未知 Conda 镜像源" in captured.out
|
||||
|
||||
def test_known_mirror_writes_condarc(self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None:
|
||||
"""已知镜像源应写入 ~/.condarc."""
|
||||
monkeypatch.setattr(Path, "home", lambda: tmp_path)
|
||||
setup_conda_mirror("tsinghua")
|
||||
condarc = tmp_path / ".condarc"
|
||||
assert condarc.exists()
|
||||
content = condarc.read_text(encoding="utf-8")
|
||||
assert "tsinghua" in content
|
||||
assert "channels:" in content
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
# setup_rust_mirror
|
||||
# ---------------------------------------------------------------------- #
|
||||
class TestSetupRustMirror:
|
||||
"""``setup_rust_mirror`` 函数测试."""
|
||||
|
||||
def test_unknown_mirror_skips(self, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]) -> None:
|
||||
"""未知镜像源应跳过."""
|
||||
setup_rust_mirror("unknown")
|
||||
captured = capsys.readouterr()
|
||||
assert "未知 Rust 镜像源" in captured.out
|
||||
|
||||
def test_known_mirror_writes_config(self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None:
|
||||
"""已知镜像源应写入 ~/.cargo/config.toml."""
|
||||
monkeypatch.setattr(Path, "home", lambda: tmp_path)
|
||||
setup_rust_mirror("ustc", "nightly")
|
||||
config = tmp_path / ".cargo" / "config.toml"
|
||||
assert config.exists()
|
||||
content = config.read_text(encoding="utf-8")
|
||||
assert "ustc" in content
|
||||
|
||||
def test_creates_sccache_dir(self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None:
|
||||
"""应创建 sccache 缓存目录."""
|
||||
from pyflowx.ops.infra import envdev as envdev_module
|
||||
|
||||
fake_sccache = tmp_path / ".cargo" / "sccache"
|
||||
monkeypatch.setattr(envdev_module, "_RUST_SCCACHE_DIR", fake_sccache)
|
||||
monkeypatch.setattr(Path, "home", lambda: tmp_path)
|
||||
setup_rust_mirror("tsinghua")
|
||||
assert fake_sccache.exists()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
# docker_login_tencent
|
||||
# ---------------------------------------------------------------------- #
|
||||
class TestDockerLoginTencent:
|
||||
"""``docker_login_tencent`` 函数测试."""
|
||||
|
||||
def test_default_username_uses_getpass(self, monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""未提供 username 时应使用 getpass.getuser."""
|
||||
monkeypatch.setattr("getpass.getuser", lambda: "testuser")
|
||||
ran_cmds: list[list[str]] = []
|
||||
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: ran_cmds.append(cmd) or MagicMock())
|
||||
|
||||
docker_login_tencent()
|
||||
|
||||
assert ran_cmds[0][0] == "docker"
|
||||
assert ran_cmds[0][1] == "login"
|
||||
assert "testuser" in ran_cmds[0]
|
||||
assert "ccr.ccs.tencentyun.com" in ran_cmds[0]
|
||||
|
||||
def test_custom_username(self, monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""提供 username 时应使用自定义用户名."""
|
||||
ran_cmds: list[list[str]] = []
|
||||
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: ran_cmds.append(cmd) or MagicMock())
|
||||
|
||||
docker_login_tencent("myuser")
|
||||
|
||||
assert "myuser" in ran_cmds[0]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
# setup_linux_system_mirror
|
||||
# ---------------------------------------------------------------------- #
|
||||
class TestSetupLinuxSystemMirror:
|
||||
"""``setup_linux_system_mirror`` 函数测试."""
|
||||
|
||||
def test_non_linux_skips(self, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]) -> None:
|
||||
"""非 Linux 平台应跳过."""
|
||||
monkeypatch.setattr(Constants, "IS_LINUX", False)
|
||||
called: list[str] = []
|
||||
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: called.append(cmd))
|
||||
|
||||
setup_linux_system_mirror()
|
||||
|
||||
captured = capsys.readouterr()
|
||||
assert "仅在 Linux" in captured.out
|
||||
assert called == []
|
||||
|
||||
def test_linux_already_configured_skips(
|
||||
self, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
|
||||
) -> None:
|
||||
"""Linux 上已配置国内镜像时应跳过."""
|
||||
monkeypatch.setattr(Constants, "IS_LINUX", True)
|
||||
|
||||
def fake_read_text(self: Path, encoding: str = "utf-8") -> str:
|
||||
return "tsinghua mirror configured"
|
||||
|
||||
monkeypatch.setattr(Path, "read_text", fake_read_text)
|
||||
called: list[str] = []
|
||||
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: called.append(cmd))
|
||||
|
||||
setup_linux_system_mirror()
|
||||
|
||||
captured = capsys.readouterr()
|
||||
assert "已配置" in captured.out
|
||||
assert called == []
|
||||
|
||||
def test_linux_not_configured_runs_script(
|
||||
self, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
|
||||
) -> None:
|
||||
"""Linux 上未配置镜像时应执行下载与安装脚本."""
|
||||
monkeypatch.setattr(Constants, "IS_LINUX", True)
|
||||
|
||||
def fake_read_text(self: Path, encoding: str = "utf-8") -> str:
|
||||
raise OSError("file not found")
|
||||
|
||||
monkeypatch.setattr(Path, "read_text", fake_read_text)
|
||||
ran_cmds: list[str] = []
|
||||
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: ran_cmds.append(cmd) or MagicMock())
|
||||
|
||||
setup_linux_system_mirror()
|
||||
|
||||
captured = capsys.readouterr()
|
||||
assert "下载" in captured.out
|
||||
assert "安装" in captured.out
|
||||
assert len(ran_cmds) == 2
|
||||
|
||||
def test_linux_content_without_mirror_runs_script(
|
||||
self, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
|
||||
) -> None:
|
||||
"""Linux 上文件存在但不包含镜像关键词时应执行脚本."""
|
||||
monkeypatch.setattr(Constants, "IS_LINUX", True)
|
||||
|
||||
def fake_read_text(self: Path, encoding: str = "utf-8") -> str:
|
||||
return "deb http://archive.ubuntu.com/ubuntu/ jammy main"
|
||||
|
||||
monkeypatch.setattr(Path, "read_text", fake_read_text)
|
||||
ran_cmds: list[str] = []
|
||||
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: ran_cmds.append(cmd) or MagicMock())
|
||||
|
||||
setup_linux_system_mirror()
|
||||
|
||||
captured = capsys.readouterr()
|
||||
assert "下载" in captured.out
|
||||
assert len(ran_cmds) == 2
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
# install_linux_qt_libs / install_linux_fonts / install_linux_docker
|
||||
# ---------------------------------------------------------------------- #
|
||||
class TestLinuxInstallers:
|
||||
"""Linux 专用安装函数测试."""
|
||||
|
||||
def test_qt_libs_non_linux_skips(self, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]) -> None:
|
||||
"""非 Linux 上 install_linux_qt_libs 应跳过."""
|
||||
monkeypatch.setattr(Constants, "IS_LINUX", False)
|
||||
called: list[list[str]] = []
|
||||
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: called.append(cmd))
|
||||
install_linux_qt_libs()
|
||||
captured = capsys.readouterr()
|
||||
assert "仅在 Linux" in captured.out
|
||||
assert called == []
|
||||
|
||||
def test_qt_libs_linux_runs_apt(self, monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""Linux 上应执行 apt install."""
|
||||
monkeypatch.setattr(Constants, "IS_LINUX", True)
|
||||
ran_cmds: list[list[str]] = []
|
||||
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: ran_cmds.append(cmd) or MagicMock())
|
||||
install_linux_qt_libs()
|
||||
assert ran_cmds[0][0] == "sudo"
|
||||
assert "apt" in ran_cmds[0]
|
||||
assert "install" in ran_cmds[0]
|
||||
|
||||
def test_fonts_non_linux_skips(self, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]) -> None:
|
||||
"""非 Linux 上 install_linux_fonts 应跳过."""
|
||||
monkeypatch.setattr(Constants, "IS_LINUX", False)
|
||||
called: list[list[str]] = []
|
||||
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: called.append(cmd))
|
||||
install_linux_fonts()
|
||||
captured = capsys.readouterr()
|
||||
assert "仅在 Linux" in captured.out
|
||||
assert called == []
|
||||
|
||||
def test_fonts_linux_runs_apt(self, monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""Linux 上应执行 apt install 字体包."""
|
||||
monkeypatch.setattr(Constants, "IS_LINUX", True)
|
||||
ran_cmds: list[list[str]] = []
|
||||
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: ran_cmds.append(cmd) or MagicMock())
|
||||
install_linux_fonts()
|
||||
assert "fonts-noto-cjk" in ran_cmds[0]
|
||||
|
||||
def test_docker_non_linux_skips(self, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]) -> None:
|
||||
"""非 Linux 上 install_linux_docker 应跳过."""
|
||||
monkeypatch.setattr(Constants, "IS_LINUX", False)
|
||||
called: list[list[str]] = []
|
||||
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: called.append(cmd))
|
||||
install_linux_docker()
|
||||
captured = capsys.readouterr()
|
||||
assert "仅在 Linux" in captured.out
|
||||
assert called == []
|
||||
|
||||
def test_docker_linux_runs_install_and_usermod(self, monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""Linux 上应执行 apt install docker-compose-v2 和 usermod."""
|
||||
monkeypatch.setattr(Constants, "IS_LINUX", True)
|
||||
monkeypatch.setattr("getpass.getuser", lambda: "testuser")
|
||||
ran_cmds: list[list[str]] = []
|
||||
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: ran_cmds.append(cmd) or MagicMock())
|
||||
|
||||
install_linux_docker()
|
||||
|
||||
assert any("docker-compose-v2" in cmd for cmd in ran_cmds)
|
||||
assert any("usermod" in cmd and "docker" in cmd for cmd in ran_cmds)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
# download_rustup_script
|
||||
# ---------------------------------------------------------------------- #
|
||||
class TestDownloadRustupScript:
|
||||
"""``download_rustup_script`` 函数测试."""
|
||||
|
||||
def test_rustup_installed_skips(self, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]) -> None:
|
||||
"""rustup 已安装时应跳过."""
|
||||
monkeypatch.setattr("shutil.which", lambda _cmd: "/usr/bin/rustup")
|
||||
called: list[list[str]] = []
|
||||
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: called.append(cmd))
|
||||
|
||||
download_rustup_script()
|
||||
|
||||
captured = capsys.readouterr()
|
||||
assert "已安装" in captured.out
|
||||
assert called == []
|
||||
|
||||
def test_windows_downloads_exe(self, monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""Windows 上应下载 rustup-init.exe."""
|
||||
monkeypatch.setattr("shutil.which", lambda _cmd: None)
|
||||
monkeypatch.setattr(Constants, "IS_WINDOWS", True)
|
||||
ran_cmds: list[list[str]] = []
|
||||
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: ran_cmds.append(cmd) or MagicMock())
|
||||
|
||||
download_rustup_script()
|
||||
|
||||
assert "powershell" in ran_cmds[0]
|
||||
assert "rustup-init.exe" in ran_cmds[0]
|
||||
|
||||
def test_non_windows_downloads_sh(self, monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""非 Windows 上应下载 rustup-init.sh."""
|
||||
monkeypatch.setattr("shutil.which", lambda _cmd: None)
|
||||
monkeypatch.setattr(Constants, "IS_WINDOWS", False)
|
||||
ran_cmds: list[list[str]] = []
|
||||
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: ran_cmds.append(cmd) or MagicMock())
|
||||
|
||||
download_rustup_script()
|
||||
|
||||
assert "curl" in ran_cmds[0]
|
||||
assert "rustup-init.sh" in ran_cmds[0]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
# install_rust_toolchain
|
||||
# ---------------------------------------------------------------------- #
|
||||
class TestInstallRustToolchain:
|
||||
"""``install_rust_toolchain`` 函数测试."""
|
||||
|
||||
def test_rustup_not_installed_skips(
|
||||
self, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
|
||||
) -> None:
|
||||
"""rustup 未安装时应跳过."""
|
||||
monkeypatch.setattr("shutil.which", lambda _cmd: None)
|
||||
called: list[list[str]] = []
|
||||
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: called.append(cmd))
|
||||
|
||||
install_rust_toolchain()
|
||||
|
||||
captured = capsys.readouterr()
|
||||
assert "未安装" in captured.out
|
||||
assert called == []
|
||||
|
||||
def test_rustup_installed_runs_toolchain_install(self, monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""rustup 已安装时应执行 toolchain install."""
|
||||
monkeypatch.setattr("shutil.which", lambda _cmd: "/usr/bin/rustup")
|
||||
ran_cmds: list[list[str]] = []
|
||||
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: ran_cmds.append(cmd) or MagicMock())
|
||||
|
||||
install_rust_toolchain("nightly")
|
||||
|
||||
assert ran_cmds == [["rustup", "toolchain", "install", "nightly"]]
|
||||
|
||||
def test_default_version_stable(self, monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""默认版本应为 stable."""
|
||||
monkeypatch.setattr("shutil.which", lambda _cmd: "/usr/bin/rustup")
|
||||
ran_cmds: list[list[str]] = []
|
||||
monkeypatch.setattr(subprocess, "run", lambda cmd, **_: ran_cmds.append(cmd) or MagicMock())
|
||||
|
||||
install_rust_toolchain()
|
||||
|
||||
assert "stable" in ran_cmds[0]
|
||||
@@ -0,0 +1,99 @@
|
||||
"""Tests for ops.envdev / sglang / msdownload 模块 (@px.tool 注册验证)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import pyflowx as px
|
||||
from pyflowx.ops.infra import envdev, msdownload, sglang # noqa: F401 -- 触发 @px.tool 注册
|
||||
from pyflowx.tools import _TOOL_REGISTRY, get_tool
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
# envdev 注册验证
|
||||
# ---------------------------------------------------------------------- #
|
||||
class TestEnvdevRegistration:
|
||||
"""``envdev`` 模块 ``@px.tool`` 注册验证."""
|
||||
|
||||
def test_envdev_registered(self) -> None:
|
||||
"""envdev 应在 _TOOL_REGISTRY 中注册."""
|
||||
assert "envdev" in _TOOL_REGISTRY
|
||||
|
||||
def test_visible_subcommands(self) -> None:
|
||||
"""可见子命令应包含核心镜像源配置命令."""
|
||||
subs = px.list_subcommands("envdev")
|
||||
assert "setup-python" in subs
|
||||
assert "setup-conda" in subs
|
||||
assert "setup-rust" in subs
|
||||
assert "setup-linux-mirror" in subs
|
||||
|
||||
def test_install_subcommands_visible(self) -> None:
|
||||
"""安装类子命令应可见."""
|
||||
subs = px.list_subcommands("envdev")
|
||||
assert "download-rustup" in subs
|
||||
assert "install-rust" in subs
|
||||
assert "install-qt-libs" in subs
|
||||
assert "install-fonts" in subs
|
||||
assert "install-docker" in subs
|
||||
|
||||
def test_setup_python_is_fn(self) -> None:
|
||||
"""setup-python 应为 fn 任务 (非 cmd)."""
|
||||
spec = get_tool("envdev", "setup-python")
|
||||
assert spec.func is not None
|
||||
assert spec.cmd is None
|
||||
|
||||
def test_setup_conda_is_fn(self) -> None:
|
||||
"""setup-conda 应为 fn 任务."""
|
||||
spec = get_tool("envdev", "setup-conda")
|
||||
assert spec.func is not None
|
||||
|
||||
def test_setup_rust_is_fn(self) -> None:
|
||||
"""setup-rust 应为 fn 任务."""
|
||||
spec = get_tool("envdev", "setup-rust")
|
||||
assert spec.func is not None
|
||||
|
||||
def test_setup_linux_mirror_is_fn(self) -> None:
|
||||
"""setup-linux-mirror 应为 fn 任务."""
|
||||
spec = get_tool("envdev", "setup-linux-mirror")
|
||||
assert spec.func is not None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
# sglang 注册验证
|
||||
# ---------------------------------------------------------------------- #
|
||||
class TestSglangRegistration:
|
||||
"""``sglang`` 模块 ``@px.tool`` 注册验证."""
|
||||
|
||||
def test_sglang_registered(self) -> None:
|
||||
"""sglang 应在 _TOOL_REGISTRY 中注册."""
|
||||
assert "sglang" in _TOOL_REGISTRY
|
||||
|
||||
def test_visible_subcommands(self) -> None:
|
||||
"""可见子命令: install, run."""
|
||||
subs = px.list_subcommands("sglang")
|
||||
assert "install" in subs
|
||||
assert "run" in subs
|
||||
|
||||
def test_run_needs_install(self) -> None:
|
||||
"""run 应依赖 install."""
|
||||
spec = get_tool("sglang", "run")
|
||||
assert "install" in spec.needs
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
# msdownload 注册验证
|
||||
# ---------------------------------------------------------------------- #
|
||||
class TestMsdownloadRegistration:
|
||||
"""``msdownload`` 模块 ``@px.tool`` 注册验证."""
|
||||
|
||||
def test_msdownload_registered(self) -> None:
|
||||
"""msdownload 应在 _TOOL_REGISTRY 中注册."""
|
||||
assert "msdownload" in _TOOL_REGISTRY
|
||||
|
||||
def test_msdownload_default_spec(self) -> None:
|
||||
"""msdownload 默认 spec 应可获取且为 fn 任务."""
|
||||
spec = get_tool("msdownload")
|
||||
assert spec.func is not None
|
||||
assert spec.cmd is None
|
||||
|
||||
def test_msdownload_run_callable(self) -> None:
|
||||
"""msdownload_run 应为可调用函数."""
|
||||
assert callable(msdownload.msdownload_run)
|
||||
@@ -4,7 +4,7 @@ from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
from pyflowx.ops import files
|
||||
from pyflowx.ops.files import filedate as files
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
|
||||
@@ -4,7 +4,7 @@ from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
from pyflowx.ops import files
|
||||
from pyflowx.ops.files import filelevel as files
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
|
||||
@@ -5,7 +5,7 @@ from __future__ import annotations
|
||||
from pathlib import Path
|
||||
from unittest.mock import patch
|
||||
|
||||
from pyflowx.ops import files
|
||||
from pyflowx.ops.files import folderback as files
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
@@ -145,15 +145,3 @@ class TestBackupFolder:
|
||||
files.backup_folder(str(source_dir), str(backup_dir), 5)
|
||||
assert backup_dir.exists()
|
||||
assert mock_zip.called
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
# 函数注册
|
||||
# ---------------------------------------------------------------------- #
|
||||
class TestRegisteredFunctions:
|
||||
"""Test that folderback functions are registered."""
|
||||
|
||||
def test_folderback_default_spec(self) -> None:
|
||||
"""folderback_default should be a registered callable."""
|
||||
# folderback_default 现在是通过 @px.register_fn 注册的普通函数, 不是 TaskSpec
|
||||
assert callable(files.folderback_default)
|
||||
|
||||
@@ -5,7 +5,7 @@ from __future__ import annotations
|
||||
from pathlib import Path
|
||||
from unittest.mock import patch
|
||||
|
||||
from pyflowx.ops import files
|
||||
from pyflowx.ops.files import folderzip as files
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
@@ -47,15 +47,3 @@ class TestZipFolders:
|
||||
"""Should handle nonexistent cwd."""
|
||||
files.zip_folders(str(tmp_path / "nonexistent"))
|
||||
# Should print error message and return
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
# 函数注册
|
||||
# ---------------------------------------------------------------------- #
|
||||
class TestRegisteredFunctions:
|
||||
"""Test that folderzip functions are registered."""
|
||||
|
||||
def test_folderzip_default_spec(self) -> None:
|
||||
"""folderzip_default should be a registered callable."""
|
||||
# folderzip_default 现在是通过 @px.register_fn 注册的普通函数, 不是 TaskSpec
|
||||
assert callable(files.folderzip_default)
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user