phoenix-sqlean

phoenix-sqlean

Vendored fork of nalgeon/sqlean.py.

Bumping a bundled version

Pins live in packages/phoenix-sqlean/Makefile. Each has a partner that must move with it:

Pin	Moves with
SQLITE_VERSION	SQLITE_RELEASE_YEAR (the URL embeds both, so a mismatch 404s) and SQLITE_SHA256
SQLEAN_COMMIT	SQLEAN_VERSION in the Makefile and in setup.py — the commit is what gets fetched, the strings only describe it
SQLEAN_VERSION	setup.py's copy, compiled in as what sqlean_version() returns. Suffix +<short sha> iff SQLEAN_COMMIT sits ahead of the named tag
XXHASH_TAG	XXHASH_COMMIT + XXHASH_SHA256, and the tag sqlean's own Makefile fetches

check-sqlean-pin, a prerequisite of download-sqlean, enforces the SQLEAN_* rows for one ls-remote: suffix prefixes the commit, base is a real tag, suffix present iff ahead, setup.py agrees. It cannot tell you the base names the wrong release — the single-commit fetch carries no ancestry. SQLITE_* and XXHASH_* get no equivalent cross-check — only their hashes, at fetch.

sqlean is fetched over git by commit: GitHub's source archives are generated on demand and not byte-stable, so they cannot be checksum-pinned, while git hashes every object it receives. Hence download-sqlean needs git on PATH and no checksum. The other two arrive over plain HTTP, where curl verifies nothing — XXHASH_COMMIT fixes which bytes are correct and XXHASH_SHA256 proves they are those. src/test_windirent.h needs neither; it is committed to this repo.

With the script

scripts/check-upstream.sh resolves and rewrites every pin, hashes included:

cd packages/phoenix-sqlean
./scripts/check-upstream.sh sqlite            # detection only, no downloads
./scripts/check-upstream.sh sqlite --apply    # rewrite pins, print a Markdown summary
./scripts/check-upstream.sh sqlean [--apply]  # also moves XXHASH_* if sqlean moved its own

By hand

Resolve a tag to a commit with git ls-remote <repo> 'refs/tags/<version>*', taking the peeled ^{} line when the tag is annotated. sqlean also carries a non-version tag (incubator), so filter by shape before picking a latest.

For SQLITE_SHA256, confirm the download against the SHA3-256 in the PRODUCT line on https://sqlite.org/download.html before recording the SHA-256 (shasum cannot do SHA3). Skipping that makes the pin trust-on-first-use.

Verify

cd packages/phoenix-sqlean
make prepare-src download-sqlite download-sqlean   # always the full chain: download-sqlean
                                                   # APPENDS init.c, so alone it double-appends
python setup.py build_ext -i
python -m tests

Staying current

Dependabot cannot see these pins — no custom-regex manager — and there is no Renovate here. phoenix-sqlean-upstream.yml fills the gap: weekly, one draft PR per component, running check-upstream.sh --apply, no model involved.

It builds and tests nothing. CI covers that on the PR; verifying inside the bump job would re-run a subset of it, on the one platform where the likeliest failure — patch(1) on the MSVC patch — cannot reproduce.

A failure of the script itself is different: upstream changed shape, so it exits before writing and the job goes red with no PR, because there is no computed bump to review.

XXHASH_* is deliberately unwatched. sqlean drives it, and download-sqlean already fails when sqlean moves its own pin.

Gotchas

• src/test_windirent.h is pinned at 3.50.4 — do not re-sync it. SQLite removed it in 3.51.0 and no newer tag has it. Vendored deliberately, not downloaded.
• sqlean >= 0.28 needs crypto/xxhash.impl.h, which sqlean does not ship. Any new third-party source needs a THIRD_PARTY_LICENSES section and its SPDX identifier in license=.
• patch(1) failing on src/sqlean-time-msvc.patch is the intended tripwire, not a breakage. Regenerate it against the new sqlean sources.
• "sqlean fetches xxHash X, but XXHASH_TAG is Y" is the other one. Move all three XXHASH_* pins, and the version named in THIRD_PARTY_LICENSES.
• The sqlean_version() test is a tautology on its own — it compares the compiled macro to setup.py's own constant, proving only that the binary is not stale. check-sqlean-pin is what ties that constant back to SQLEAN_COMMIT.
• Changing SQLEAN_VERSION alone does not trigger a recompile. It only moves a -D flag, and build_ext keys on source mtime. touch src/sqlean.c after editing, or the version test fails against a merely stale build.
• Keep the package version plain SemVer. release-please's version regex is unanchored, so 1.2.3.1 and 1.2.3.post1 truncate to 1.2.3 and collide with an existing tag, silently. For the same reason VERSION must stay above SQLEAN_VERSION in setup.py. The package version is intentionally independent of the bundled SQLite version.

Wheels and publishing

• Settings are in cibuildwheel.toml; runners and steps are in phoenix-sqlean-build.yml, a workflow_call workflow that build-sqlean in publish.yaml and CI's wheels job both invoke with no overrides. A PR runs the exact build a release runs — keep it that way rather than narrowing CI's copy.
• The wheel matrix is every CPython python_requires admits × the toml's build platforms × the build workflow's runners, minus the toml's skip, plus an sdist of the C sources. Read it there rather than restating it.
• Every wheel is tested by cibuildwheel, which needs one runner per Linux/Windows arch. Those archs are native; cibuildwheel skips testing a cross-compiled win_arm64, and auto on Windows AMD64 adds 32-bit x86. macOS builds both arches on one arm64 runner and tests x86_64 under Rosetta. Free-threaded builds are excluded by the toml's skip; cibuildwheel would otherwise build them by default.
• scripts/check-wheelhouse.py gates the merged set in the build workflow's check job, so both CI and publish run it: exactly one wheel per CPython that python_requires admits on each of its PLATFORMS, plus one sdist. Adding or dropping a platform means editing PLATFORMS alongside the runners and archs.
• The supported Python range is python_requires in setup.py, and nothing else. cibuildwheel reads it to pick CPythons, and the wheelhouse check expects exactly its versions. It must stay a string literal keyword in the setuptools.setup(...) call under if __name__ == "__main__" — cibuildwheel parses that statically, and silently builds every CPython it knows if it cannot find one (the wheelhouse check would then fail).
• Admitting a new CPython can fail two ways, both loudly. cibuildwheel may still gate it as a prerelease, so its wheels are missing and the wheelhouse check fails — bump cibuildwheel. Or it may have removed private C API the driver uses, which fails that version's wheel builds.
• Source preparation is the build workflow's first job, on Ubuntu because Windows runners lack make/curl/unzip. Its ref input selects what to build: CI leaves it empty (the PR's commit), publish passes the release tag. Callers pass nothing else.
• test-sources resolves against cibuildwheel's working directory, not package-dir — hence the build legs unpack the sources at the workspace root.
• Publishing is gated on the tag arize-phoenix-sqlean-v<manifest version>. Until release-please creates it, sqlean-release skips and nothing builds.
• sqlean publishes until PyPI holds every distribution, not just the version. The generic "version on PyPI" gate in list-python-packages skips sqlean; sqlean-release runs on every release and runs check-wheelhouse.py --missing-only on PyPI's file list for that version. A release interrupted mid-upload is finished by re-running publish, which rebuilds from the tag and uploads with skip-existing.
• check-wheelhouse.py always comes from the commit running the workflow; python_requires comes from the sources being built. PLATFORMS must match the runners, which also come from that commit, and older release tags predate the script.
• PyPI uses trusted publishing with no GitHub environment; the publisher's Environment field must stay blank to match.