feat(compat): seed-compat workflow + emitter (Phase 4)
All checks were successful
addon-qualify / qualify (push) Successful in 10s

Wires the nightly cold-start seeder. The Gitea Action runs
qualify-addon.py against every addon on each version branch (18.0 +
19.0), emits a canonical JSON snapshot to compat-bootstrap/seeded-ci.json,
and commits only when content changed. Tower's CompatSeedLoader fetches
this file at startup + every 24h, replays unseen stampIds into the
matrix.

Decisions: Git-as-bus over HTTP endpoint, static lint over real install,
content-hash stampId for byte-stability across runs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
compat-seeder
2026-05-10 00:14:14 +03:00
parent 906e5ebd6d
commit 820ee83c09
3 changed files with 359 additions and 0 deletions

View File

@@ -0,0 +1,126 @@
# Pillar 2 / Phase 4 — Cold-start compat-matrix seed.
#
# This file is the SOURCE-OF-TRUTH copy. The deployed copy lives in
# the Gitea repo at:
#
# odoo-tower/odoo-addons (branch `compat-bootstrap`)
# └── .gitea/workflows/seed-compat.yml
#
# Why a dedicated branch instead of `main`/`18.0`/`19.0`: the addon
# code lives on per-Odoo-major branches. The cold-start snapshot is
# orthogonal data — keeping it on its own branch lets Tower fetch
# from one stable place and lets the cron commit without touching
# addon source.
#
# Schedule: daily 03:00 UTC. The emitter computes a content-hash
# stampId, so identical results across nights produce no commit
# (git diff is empty → push is skipped). When the catalog drifts
# (a new addon, a fix, a regression), one commit lands and Tower's
# next 24h tick replays it.
#
# Deployment (one-shot, manual until we mass-publish workflows):
#
# git -C /path/to/odoo-addons checkout -b compat-bootstrap origin/18.0
# mkdir -p .gitea/workflows compat-bootstrap
# cp infrastructure/gitea-actions/workflows/seed-compat.yml \
# /path/to/odoo-addons/.gitea/workflows/
# cp scripts/qualify-addon.py scripts/emit-compat-bootstrap.py \
# /path/to/odoo-addons/scripts/
# git -C /path/to/odoo-addons add .gitea scripts compat-bootstrap
# git -C /path/to/odoo-addons commit -m "feat(compat): seed bootstrap workflow"
# git -C /path/to/odoo-addons push -u origin compat-bootstrap
name: Cold-start compat seed
on:
schedule:
- cron: '0 3 * * *' # nightly 03:00 UTC
workflow_dispatch: {} # manual trigger from Gitea UI
jobs:
seed:
runs-on: ubuntu-latest
steps:
- name: Checkout compat-bootstrap branch
uses: actions/checkout@v4
with:
ref: compat-bootstrap
fetch-depth: 1
- name: Materialise per-major addon trees
run: |
set -euo pipefail
# Detached worktrees keep each version branch's tree independent
# so the qualifier sees a clean addon root with no cross-branch
# contamination. Cleaned up in the final step.
for major in 18.0 19.0; do
git fetch --depth=1 origin "$major"
git worktree add --detach "addons-$major" "origin/$major"
ls -1 "addons-$major/addons" 2>/dev/null | head -5 || true
done
- name: Run qualifier + emit bootstrap snapshot per major
run: |
set -euo pipefail
mkdir -p compat-bootstrap/per-major
for major in 18.0 19.0; do
# qualify-addon.py is vendored into each version branch under
# .gitea/qualify-addon.py by Pillar 1; reuse that copy so static-
# lint logic stays in lockstep with what addon-qualify.yml runs
# on push.
python3 scripts/emit-compat-bootstrap.py \
--addons-root "addons-$major/addons" \
--qualifier "addons-$major/.gitea/qualify-addon.py" \
--output "compat-bootstrap/per-major/$major.json" \
--pg-by-major '{"18.0":"16","19.0":"17"}' \
--source "qualify-addon-v1"
done
- name: Merge per-major snapshots into seeded-ci.json
run: |
set -euo pipefail
python3 - <<'PY'
import hashlib, json, pathlib
rows = []
for p in sorted(pathlib.Path('compat-bootstrap/per-major').glob('*.json')):
rows.extend(json.loads(p.read_text())['rows'])
rows.sort(key=lambda r: (r['addonCode'], r['addonVersion'],
r['odooMajor'], r['postgresMajor']))
canonical = json.dumps(rows, sort_keys=True, separators=(',', ':'))
stamp = 'sha256:' + hashlib.sha256(canonical.encode('utf-8')).hexdigest()
out = {
'stampId': stamp,
'schemaVersion': 1,
'source': 'qualify-addon-v1',
'rows': rows,
}
pathlib.Path('compat-bootstrap/seeded-ci.json').write_text(
json.dumps(out, indent=2) + '\n')
print(f'merged {len(rows)} rows; stamp={stamp}')
PY
- name: Commit + push if content changed
env:
GIT_USER_TOKEN: ${{ secrets.GITEA_PUSH_TOKEN }}
run: |
set -euo pipefail
# Discard the per-major scratch files + addon worktrees; only
# the consolidated snapshot is part of the canonical state.
for major in 18.0 19.0; do git worktree remove --force "addons-$major" || true; done
rm -rf compat-bootstrap/per-major
git config user.email "compat-seeder@odoosky.cloud"
git config user.name "compat-seeder"
if git diff --quiet -- compat-bootstrap/seeded-ci.json; then
echo "no content change in seeded-ci.json; nothing to commit"
exit 0
fi
git add compat-bootstrap/seeded-ci.json
stamp=$(python3 -c "import json,sys; print(json.load(open('compat-bootstrap/seeded-ci.json'))['stampId'])")
git commit -m "chore(compat): refresh cold-start seed (${stamp})"
# Push via token. GITEA_PUSH_TOKEN is a repo-scoped PAT with
# write access to odoo-tower/odoo-addons:compat-bootstrap.
remote="https://compat-seeder:${GIT_USER_TOKEN}@git.odoosky.org/odoo-tower/odoo-addons.git"
git push "$remote" HEAD:compat-bootstrap

View File

@@ -0,0 +1,5 @@
# compat-bootstrap
Tower's CompatSeedLoader fetches `seeded-ci.json` from this directory.
Generated by `.gitea/workflows/seed-compat.yml` (cron 03:00 UTC daily).
Don't edit by hand — the next workflow run will overwrite.

228
scripts/emit-compat-bootstrap.py Executable file
View File

@@ -0,0 +1,228 @@
#!/usr/bin/env python3
"""
emit-compat-bootstrap.py — Pillar 2 / Phase 4 emitter.
Walks every addon directory under --addons-root, runs qualify-addon.py
against each, and emits a consolidated JSON snapshot Tower's
CompatSeedLoader replays into the matrix on startup.
Wire shape (matches CompatSeedBootstrap in compat_seed_loader.go):
{
"stampId": "sha256:<canonical-row-hash>",
"schemaVersion": 1,
"source": "qualify-addon-v1",
"rows": [
{"addonCode": "report_carbone", "addonVersion": "18.0.1.0.9",
"odooMajor": "18.0", "postgresMajor": "16",
"outcome": "success"},
...
]
}
Usage (typical Gitea Actions invocation):
python3 scripts/emit-compat-bootstrap.py \\
--addons-root . \\
--output compat-bootstrap/seeded-ci.json \\
--pg-by-major '{"18.0":"16","19.0":"17"}'
Exit codes:
0 snapshot written successfully (file may or may not have changed)
2 bad usage / I/O error
"""
from __future__ import annotations
import argparse
import ast
import hashlib
import json
import subprocess
import sys
from pathlib import Path
# Static-lint check name → matrix error_class taxonomy. Aligns with
# the keys in backend/cmd/api/error_translations.yaml so Phase 2
# aggregation can group static (seeded_ci) and runtime (real_install)
# evidence under the same bucket.
ERROR_CLASS_MAP = {
'manifest': 'manifest-malformed',
'pip-deps': 'pip-deps-missing',
'app-name': 'settings-page-broken',
'menu-icon': 'menu-icon-missing',
'hoot-import': 'bundle-module-missing',
'webpack-name': 'bundle-module-missing',
'webpack-chunk': 'bundle-module-missing',
}
def find_addon_dirs(root: Path) -> list[Path]:
"""Return every direct child of `root` that ships a __manifest__.py.
Skips dotfiles, .git, and the bootstrap output directory itself so
re-running over a previously emitted tree doesn't try to lint
JSON files.
"""
out: list[Path] = []
for child in sorted(root.iterdir()):
if not child.is_dir():
continue
if child.name.startswith('.') or child.name == 'compat-bootstrap':
continue
if (child / '__manifest__.py').is_file():
out.append(child)
return out
def parse_manifest_version(addon_dir: Path) -> tuple[str, str]:
"""Return (addonVersion, odooMajor). Best-effort: empty strings on parse failure.
Odoo's convention is `<odoo-major>.<addon-x>.<addon-y>.<addon-z>`,
where odoo-major itself is `X.0` (e.g. `18.0`, `19.0`). We split
on the first two dots → odoo_major = "18.0", addon_version = full.
"""
try:
src = (addon_dir / '__manifest__.py').read_text(encoding='utf-8')
manifest = ast.literal_eval(src)
except Exception:
return '', ''
version = str(manifest.get('version', '')).strip()
if not version:
return '', ''
parts = version.split('.')
if len(parts) >= 2 and parts[0].isdigit() and parts[1] == '0':
odoo_major = f'{parts[0]}.0'
else:
odoo_major = ''
return version, odoo_major
def run_qualifier(qualifier: Path, addon_dirs: list[Path]) -> list[dict]:
"""Invoke qualify-addon.py --json over every addon dir at once."""
if not addon_dirs:
return []
cmd = ['python3', str(qualifier), '--json'] + [str(p) for p in addon_dirs]
proc = subprocess.run(cmd, capture_output=True, text=True)
if proc.returncode == 2:
# Bad usage / I/O — surface stderr and abort.
sys.stderr.write(proc.stderr)
sys.exit(2)
# returncode 0 (all qualified) or 1 (some failed) both produce the
# JSON array on stdout. Parse regardless.
try:
return json.loads(proc.stdout)
except json.JSONDecodeError as e:
sys.stderr.write(f'qualifier JSON parse failed: {e}\nstdout was:\n{proc.stdout}\n')
sys.exit(2)
def first_error_class(findings: list[dict]) -> str:
"""Pick the first ERROR-severity finding's check name and translate
it through ERROR_CLASS_MAP. Returns '' if no ERROR-level finding."""
for f in findings:
if f.get('severity') == 'ERROR':
check = f.get('check', '')
return ERROR_CLASS_MAP.get(check, check or 'unmatched')
return ''
def build_rows(qualifier_results: list[dict],
manifests: dict[str, tuple[str, str]],
pg_by_major: dict[str, str]) -> list[dict]:
"""Map qualifier output → matrix-shaped rows.
Skips addons whose manifest didn't parse (no addonVersion) — the
matrix needs both code and version to be useful, and we'd rather
drop a row than seed it with empty strings the aggregator would
later have to special-case.
"""
rows: list[dict] = []
for r in qualifier_results:
code = r['addon']
version, odoo_major = manifests.get(code, ('', ''))
if not version or not odoo_major:
continue
pg_major = pg_by_major.get(odoo_major, '')
if r.get('qualified'):
outcome = 'success'
error_class = ''
else:
outcome = 'failed_install'
error_class = first_error_class(r.get('findings', []))
row = {
'addonCode': code,
'addonVersion': version,
'odooMajor': odoo_major,
'postgresMajor': pg_major,
'outcome': outcome,
}
if error_class:
row['errorClass'] = error_class
rows.append(row)
rows.sort(key=lambda r: (r['addonCode'], r['addonVersion'],
r['odooMajor'], r['postgresMajor']))
return rows
def stamp_id(rows: list[dict]) -> str:
"""Content hash of the canonical row set. Same rows → same stamp,
so Tower's idempotency check works across emitter invocations."""
canonical = json.dumps(rows, sort_keys=True, separators=(',', ':'))
h = hashlib.sha256(canonical.encode('utf-8')).hexdigest()
return f'sha256:{h}'
def main(argv: list[str]) -> int:
p = argparse.ArgumentParser()
p.add_argument('--addons-root', required=True, type=Path,
help='directory whose immediate children are addon dirs')
p.add_argument('--qualifier', type=Path,
default=Path(__file__).parent / 'qualify-addon.py',
help='path to qualify-addon.py (defaults to sibling script)')
p.add_argument('--output', required=True, type=Path,
help='where to write the bootstrap JSON')
p.add_argument('--pg-by-major', type=str,
default='{"18.0":"16","19.0":"17"}',
help='JSON map: odoo_major → recommended postgres_major')
p.add_argument('--source', type=str, default='qualify-addon-v1',
help='source identifier persisted in compat_seed_stamps.source')
args = p.parse_args(argv[1:])
if not args.addons_root.is_dir():
sys.stderr.write(f'addons-root not a directory: {args.addons_root}\n')
return 2
if not args.qualifier.is_file():
sys.stderr.write(f'qualifier not found: {args.qualifier}\n')
return 2
try:
pg_by_major = json.loads(args.pg_by_major)
except json.JSONDecodeError as e:
sys.stderr.write(f'--pg-by-major must be JSON: {e}\n')
return 2
addon_dirs = find_addon_dirs(args.addons_root)
manifests = {d.name: parse_manifest_version(d) for d in addon_dirs}
qualifier_results = run_qualifier(args.qualifier, addon_dirs)
rows = build_rows(qualifier_results, manifests, pg_by_major)
# No generatedAt field — it would change every run and defeat the
# workflow's "git diff --quiet → no commit" idempotency. The git
# commit timestamp + stampId in commit message carry the same info.
bootstrap = {
'stampId': stamp_id(rows),
'schemaVersion': 1,
'source': args.source,
'rows': rows,
}
args.output.parent.mkdir(parents=True, exist_ok=True)
args.output.write_text(json.dumps(bootstrap, indent=2) + '\n', encoding='utf-8')
print(f'wrote {len(rows)} rows to {args.output} (stamp {bootstrap["stampId"]})',
file=sys.stderr)
return 0
if __name__ == '__main__':
sys.exit(main(sys.argv))