zhou
32ca8c1208
CI / Lint, Typecheck & Test (push) Successful in 1m57s
1. 新建 docs/ Sphinx 文档结构 (conf.py + 8 个 rst 章节), napoleon 支持 Google/NumPy docstring, rtd 主题, 自动生成 API 参考与错误家族文档 2. 新建 .readthedocs.yaml 配置, pyproject.toml 加 docs 依赖 3. 删除 runner.py 的 _apply_verbose_to_graph 死代码及对应测试 (功能已移入 executors.run 统一处理) 4. 更新 README: CLI 示例改为 pf 统一入口, 模块结构表补全 cli/pf.py/cli/configs/cli/_ops 等模块 5. 修复版本不一致 (pyproject.toml 0.3.5 → 0.4.5) 6. 加文档徽章链接到 ReadTheDocs
165 lines
3.9 KiB
ReStructuredText
165 lines
3.9 KiB
ReStructuredText
YAML 任务编排
|
||
=============
|
||
|
||
PyFlowX 支持 GitHub Actions 风格的声明式 YAML 任务编排，从 YAML 文件直接加载任务图。
|
||
|
||
编程式 API
|
||
----------
|
||
|
||
.. code-block:: python
|
||
|
||
import pyflowx as px
|
||
|
||
# 从 YAML 文件加载任务图
|
||
graph = px.Graph.from_yaml("pipeline.yaml")
|
||
report = px.run(graph, strategy="thread")
|
||
|
||
# 或用函数式 API
|
||
graph = px.load_yaml("pipeline.yaml")
|
||
|
||
# 从字符串解析
|
||
graph = px.parse_yaml_string("""
|
||
jobs:
|
||
hello:
|
||
cmd: ["echo", "hello"]
|
||
""")
|
||
|
||
YAML Schema
|
||
-----------
|
||
|
||
.. code-block:: yaml
|
||
|
||
strategy: thread # 图级默认策略
|
||
defaults: # 图级默认值
|
||
retry: {max_attempts: 3}
|
||
verbose: true
|
||
env: {CI: "true"}
|
||
|
||
variables: # 变量定义 (可在 cmd/env 中 ${VAR} 引用)
|
||
OUTPUT: "dist"
|
||
|
||
jobs:
|
||
setup:
|
||
cmd: ["git", "clone", "..."]
|
||
runs-on: linux
|
||
|
||
build:
|
||
needs: [setup] # 依赖列表
|
||
cmd: ["python", "-m", "build"]
|
||
timeout: 300
|
||
retry: {max_attempts: 2, delay: 1.0}
|
||
|
||
test:
|
||
needs: [build]
|
||
cmd: ["python${{ matrix.version }}", "-m", "pytest"]
|
||
strategy:
|
||
matrix: # 笛卡尔积展开为 6 个任务
|
||
version: ["3.8", "3.9", "3.10"]
|
||
os: ["linux", "macos"]
|
||
if: "env.CI" # 条件: 环境变量存在
|
||
|
||
lint:
|
||
needs: [build]
|
||
cmd: ["ruff", "check"]
|
||
if: "env.CI == 'true'"
|
||
|
||
deploy:
|
||
needs: [test, lint] # 矩阵依赖自动展开
|
||
cmd: ["twine", "upload"]
|
||
if: "env.DEPLOY_TOKEN != ''"
|
||
allow-upstream-skip: true
|
||
concurrency-key: deploy_lock
|
||
|
||
字段映射
|
||
--------
|
||
|
||
.. list-table::
|
||
:header-rows: 1
|
||
:widths: 30 30 40
|
||
|
||
* - YAML 字段
|
||
- TaskSpec 字段
|
||
- 说明
|
||
* - ``jobs.<id>``
|
||
- ``name``
|
||
- job ID 作为任务名
|
||
* - ``cmd`` / ``run``
|
||
- ``cmd``
|
||
- ``cmd`` 为列表形式，``run`` 为 shell 字符串
|
||
* - ``needs``
|
||
- ``depends_on``
|
||
- 依赖列表（矩阵任务自动展开）
|
||
* - ``if``
|
||
- ``conditions``
|
||
- ``success()`` / ``always()`` / ``env.VAR`` / ``env.VAR == 'x'``
|
||
* - ``strategy.matrix``
|
||
- 矩阵扇出
|
||
- 笛卡尔积展开为多个任务
|
||
* - ``${{ matrix.key }}``
|
||
- 占位符
|
||
- 在 cmd/run/cwd/env 中替换
|
||
* - ``timeout``
|
||
- ``timeout``
|
||
- 超时秒数
|
||
* - ``retry``
|
||
- ``retry``
|
||
- ``{max_attempts, delay, backoff, jitter}``
|
||
* - ``cwd``
|
||
- ``cwd``
|
||
- 工作目录
|
||
* - ``env``
|
||
- ``env``
|
||
- 环境变量
|
||
* - ``verbose``
|
||
- ``verbose``
|
||
- 详细输出
|
||
* - ``continue-on-error``
|
||
- ``continue_on_error``
|
||
- 失败不中止整图
|
||
* - ``skip-if-missing``
|
||
- ``skip_if_missing``
|
||
- 命令不存在时跳过
|
||
* - ``allow-upstream-skip``
|
||
- ``allow_upstream_skip``
|
||
- 上游跳过时仍执行
|
||
* - ``priority``
|
||
- ``priority``
|
||
- 同层优先级
|
||
* - ``concurrency-key``
|
||
- ``concurrency_key``
|
||
- 并发限制键
|
||
* - ``tags``
|
||
- ``tags``
|
||
- 自由标签
|
||
* - ``runs-on``
|
||
- ``tags``（追加）
|
||
- 运行环境标签
|
||
|
||
CLI 配置段（``cli:``）
|
||
----------------------
|
||
|
||
工具 YAML 还可定义 ``cli:`` 段，声明命令行参数 schema，由 ``pf`` 自动解析：
|
||
|
||
.. code-block:: yaml
|
||
|
||
cli:
|
||
description: "FileDate - 文件日期处理工具"
|
||
usage: "pf filedate <command> [files...]"
|
||
subcommands:
|
||
add:
|
||
help: "添加日期前缀"
|
||
positional:
|
||
- name: FILES
|
||
nargs: "+"
|
||
type: path
|
||
help: "文件路径"
|
||
options:
|
||
- name: CLEAR
|
||
flag: "--clear"
|
||
action: store_true
|
||
help: "清除已有日期前缀"
|
||
|
||
支持的 ``type``：``str`` / ``int`` / ``float`` / ``path``。
|
||
|
||
完整 API 说明详见 :doc:`/api`。
|