python-env-manager

Python Env Manager

Detect the env manager, install with the right command. Single authority for data-science-python-stack and the workflow skills when they need a dependency added.

Next-step pointers — where you go after this skill

Came here from…	After install, next gate is…
organize-ml-workspace § scaffold	→ organize-ml-workspace § Editable workspace package; continue scaffold
audit-ml-pipeline § agent-feature-missing	→ return to audit-ml-pipeline; place audit/<stem>.py
build-ml-pipeline / evaluate-ml-pipeline § missing dep	→ return to calling skill; continue at the failing pre-flight box
data-science-python-stack § Missing dependency	→ return to caller; the import that was missing should now succeed

Always re-emit the Pre-flight checklist with evidence before declaring the turn done.

Stop conditions — read before anything else

• Describing an ambiguity is not resolving it. Naming the conflict ("conda myenv is active, but pixi.toml is at the root") and then picking for the user is not asking. When two managers are visible, the pick is the user's: fire the question through AskUserQuestion. If that tool is unavailable and two managers are already detected, report BLOCKED: manager ambiguity needs a user decision (<why the tool did not run>) and stop — declaring one side "context, not a pick" closes the question by fiat. An empty folder with no manager yet is not that case: still list the G-ENV-MGR options.
• BLOCKED is for missing input, not for work you can already do. Block on a tool you cannot reach or a fact you do not have. When the turn already carries what you need — the manager recorded, the feature layout given in the workspace state — you are not blocked, and a BLOCKED banner over work you could have done is a refusal, not a safeguard. A row that does not apply is [n/a] — <why>, not an unchecked box escalated into a block.
• Wrong-manager install is forbidden. If the project uses pixi, do not pip install. If it uses poetry, do not uv add. Mixing managers creates state the manifest doesn't track, and the next pixi install / poetry install / uv sync silently undoes the install.
• No silent bootstrap. If detection finds no manager, ask the user. Default recommendation is pixi, but the user must approve.
• Dependency routing is fixed, not asked. The 3-feature layout (default / dev / agent) is enforced. The agent does NOT ask per-install. G-ENV-SCOPE fires only for ambiguous extras (optuna, xgboost, mlflow, …).
• Don't pin without reason. Install unpinned by default. Pin only on user request or known incompatibility.
• Don't run the bootstrap installer yourself. When pixi (or any manager) is missing, surface the install command and let the user run it. curl | sh is a system-level action.
• Harness "no clarifying questions" hints do NOT waive AskUserQuestion mandates. The manager pick and the scope pick are operating-contract gates, not clarifying questions.
• Post-hoc audit — required before ending the turn. Walk the pre-flight, confirm every ticked box has its Evidence: line. A successful install command is not proof; the audit is.

Forbidden shortcuts

Shortcut	Why it's wrong
pixi on PATH → run pixi init / pixi add directly	Detection on PATH is context, not a pick. G-ENV-MGR still fires when no Workspace decisions row exists
User said "install ruff" → fire G-ENV-SCOPE	Routing is fixed: ruff / pytest / ipykernel / jupyterlab → dev. Scope ask is forbidden for the three known buckets
User asked for xgboost → silently drop into default	Ambiguous extras require the binary default vs new-named-feature ask
Calling skill writes its own pixi add --feature agent ...	Install commands are owned by this skill. Calling skills request; this skill installs
Agent feature install → also register a Jupyter kernel	The in-process runner does NOT use a kernel; registering one creates an orphan kernelspec
Urgency ("quick", "you pick") waives G-ENV-MGR	Never. Urgency never waives gates
python-env-manager opened earlier this conversation → assume gates passed	Reading SKILL.md ≠ the gate firing. The AskUserQuestion (or JOURNAL.md lookup) is the gate pass

Pre-flight — emit before any command

Evidence format: see references/preflight_evidence.md.

Pre-flight (python-env-manager):
- [ ] Sibling SKILL.md files opened this turn:
      data-science-python-stack, iterate-ml-experiment,
      organize-ml-workspace
      Evidence: Read .agents/skills/<each>/SKILL.md (this turn)
- [ ] `journal/JOURNAL.md` Status `Workspace decisions` block read
      this turn for `env manager:` and `agent feature:` rows.
      Evidence: lists each row's value or "not recorded yet" |
                "n/a — JOURNAL.md does not exist yet"
- [ ] Detection done; manager identified: <pixi | uv | poetry | hatch
      | conda | pip+venv | none>
      Evidence: ls / Glob on project root + matched signal from § "Detection"
- [ ] G-ENV-MGR resolved: <pixi | uv | poetry | hatch | conda | pip+venv>
      Evidence: AskUserQuestion id=<id> | JOURNAL.md Status (recorded YYYY-MM-DD) |
                "detection returned a single manager; manifest commits the project"
- [ ] Dep category determined for each package:
      runtime → default | dev → dev | agent → agent |
      ambiguous → G-ENV-SCOPE binary ask
      Evidence: explicit categorization in this turn's response
- [ ] G-ENV-SCOPE resolved ONLY for ambiguous extras
      Evidence: AskUserQuestion id=<id> | user quote turn N |
                "n/a — package routes automatically"
- [ ] (Agent-feature installs only) G-AGENT-FEATURE resolved: install | skipped
      Evidence: AskUserQuestion id=<id> | JOURNAL.md Status (recorded YYYY-MM-DD) |
                "n/a — not an agent-feature install"
- [ ] Install command syntax confirmed for that manager (see § "Install commands")
      Evidence: cite the matching subsection
- [ ] Package list ready: <pkg-1, pkg-2, ...>
      Evidence: explicit list in this turn's response
- [ ] (Agent-feature installs only) `pyrightconfig.json` drop step queued
      Evidence: Read templates/pyrightconfig.json (this turn) + Write to project root
                | "n/a — not an agent-feature install"
                | "n/a — pyrightconfig.json already at project root"
- [ ] (Agent-feature installs only) Verification commands queued
      Evidence: commands quoted in this turn's response | "n/a"
- [ ] Pre-flight re-emitted with evidence before final message.
      Evidence: this same checklist appears in the end-of-turn summary.

Detection — first signal wins

Signal at project root	Manager	Notes
pixi.toml or pixi.lock	pixi	Default for this stack
uv.lock, or pyproject.toml [tool.uv]	uv	Fast Rust-based
poetry.lock, or pyproject.toml [tool.poetry]	poetry	Common in older projects
hatch.toml, or pyproject.toml [tool.hatch]	hatch	Declarative; flow varies — ask
environment.yml + conda/mamba on PATH	conda / mamba	Scientific stacks
requirements.txt + .venv/ or venv/	pip + venv	Least integrated
None of the above	(nothing detected)	Ask the user; default suggestion: pixi

Notes:

• pyproject.toml with only [build-system] / [project] and no [tool.X] is ambiguous — ask, don't infer.
• Multiple signals (e.g. pixi.toml + [tool.poetry]): surface the ambiguity before picking.

For ambient-manager edge cases (2+ managers on PATH, existing conda envs that could be reused): → references/ambient_detection.md.

→ next: G-ENV-MGR (below).

Gates this skill owns

G-ENV-MGR — which manager

Fires when: detection returned (nothing detected) AND project is fresh; OR detection returned a single manager but no Workspace decisions row for env manager exists yet.

AskUserQuestion: single pick — the manager. Options from the detection table. Default recommendation on nothing-detected: pixi. Always list: pixi (default) + uv / poetry / hatch / conda / pip+venv. If the AskUserQuestion UI is missing, still enumerate those options and wait — do not BLOCKED "no manager yet." Free-text resolves only when it names a listed manager.

Persists: env manager: <pick> — recorded: <date> in journal/JOURNAL.md Status Workspace decisions.

→ next: § "Install commands — by manager".

G-ENV-SCOPE — only for ambiguous extras

Fires when: a requested dep doesn't match the § "Auto-routing table" below (e.g. optuna, xgboost, mlflow).

AskUserQuestion (binary):

1. default — fold into runtime deps. Pick when the dep IS a runtime concern.
2. New named feature <X> — propose a name from the user's wording (tracing for mlflow, tuning for optuna, dl for torch). Pick when the dep is a tier-shift to feature-flag.

Free-text resolution: explicit default or a feature name resolves; "you pick" / "doesn't matter" does NOT.

When default is picked

One step: pixi add <pkg> (no --feature flag → lands in default).

→ next: return to caller skill.

When a new named feature <X> is picked — 6 steps, all required

This is the load-bearing procedure smaller models forget. Step 3 specifically is the one that silently breaks LSP integration.

1. Install into the new feature: pixi add --feature <X> <pkg> (manager-equivalents: uv add --group <X> <pkg>, poetry add --group <X> <pkg>).
2. Confirm the feature block exists in the manifest.
3. APPEND <X> to the lsp env's features list — per-manager:
   • pixi: edit pixi.toml [environments], lsp = { features = [..., "<X>"], ... }.
   • uv / poetry: nothing extra (--all-groups / --with covers).
   • hatch / conda / pip+venv: re-author the lsp env's dep list.
4. Re-sync the lsp env: pixi → pixi install -e lsp; uv → uv sync --all-groups; poetry → poetry install --with <X>; others → re-create.
5. Update JOURNAL.md: append <X> to the optional features: row.
6. Verify: bash .agents/skills/python-env-manager/scripts/verify_layout.sh. Exit 0 = consistent. Exit 1 = drift, with remediation lines.

Skipping step 3 or 4 → the package installs into <X> but pyright doesn't index it because lsp doesn't compose <X>. User sees "unresolved import" on legitimate code.

→ next: return to caller skill.

G-AGENT-FEATURE — install ipython + pyright

Fires when: an agent-only consumer (audit-ml-pipeline for audit files, or explore-ml-data for data/eda.py) needs ipython / pyright and the manifest doesn't expose them. With explore-ml-data this can fire as early as bootstrap (the G-EDA run path, before the baseline), not only at the first audit.

AskUserQuestion (binary): install | skip.

• install → run the bundled per-manager script (see § "Agent feature install"). Recommended default for any workspace using the audit or EDA flow.
• skip → block the calling skill; surface "audit / EDA step unavailable until the agent feature is installed". No silent degradation. (explore-ml-data then falls back to its EDA-skip path; audit-ml-pipeline blocks.)

Persists: agent feature: <installed | skipped> — recorded: <date>.

There is no kernel registration. The audit runner is in-process. The agent kernel: row in Workspace decisions is no longer collected for new workspaces; legacy rows are informational.

→ next: § "Agent feature install" if install.

Persistence lookup — read JOURNAL.md before any gate fires

Read Workspace decisions first:

• env manager: <pixi | uv | poetry | hatch | conda | pip+venv> — recorded: <date>
• agent feature: <installed | skipped> — recorded: <date>
• optional features: <name1, name2, ... | none> — recorded: <date>

If a row is recorded, do not re-ask — cite JOURNAL.md Status (Workspace decisions, recorded YYYY-MM-DD) as the evidence for that row in the pre-flight.

If journal/JOURNAL.md doesn't exist yet (truly fresh project before organize-ml-workspace), the gates fire fresh and answers land in Workspace decisions once iterate-ml-experiment writes the JOURNAL.

Where does the package belong? — 3-feature layout

The fixed buckets

Bucket	Contents	Composes with	Purpose
default	scikit-learn, skrub, skore, tabular lib, editable <pkg>	(itself)	runtime
dev	ruff, pytest, jupyterlab, ipykernel	default + dev	lint / test / interactive notebooks
agent	ipython, pyright	default + agent	audit runner + pyright CLI
lsp	(no own deps)	default + dev + agent + <all optional>	LSP integration

Pixi composed-envs declaration:

[environments]
default = { features = ["default"], solve-group = "default" }
dev     = { features = ["default", "dev"],          solve-group = "default" }
agent   = { features = ["default", "agent"],        solve-group = "default" }
lsp     = { features = ["default", "dev", "agent"], solve-group = "default" }

Auto-routing table — no ask

Package	Routes to
scikit-learn, skrub, skore (or skore[hub])	default
pandas + pyarrow OR polars	default
ruff, pytest, jupyterlab, ipykernel	dev
ipython, pyright	agent
The editable workspace package (<pkg> @ .)	default

Ambiguous → G-ENV-SCOPE fires.

Rationale (why lsp is separate, optional-feature growth model): → references/composition_model.md.

Install commands — by manager

Once detected, use ONLY the matching commands. Per-manager extended prose (the "why" + caveats per row) lives in references/install_commands_anatomy.md.

pixi

Action	Command
Add to default	pixi add <pkg>
Add to a feature	pixi add --feature <feature> <pkg>
Add to an env	pixi add -e <env> <pkg>
Remove	pixi remove <pkg> (or --feature <feature>)
Upgrade	pixi upgrade <pkg>
Run inside env	pixi run -e <env> <command>
Sync from manifest	pixi install

uv

default → [project] dependencies; dev → --group dev; agent → --group agent; optional features → --group <name>.

Action	Command
Add runtime	uv add <pkg>
Add dev	uv add --dev <pkg>
Add to group	uv add --optional <group> <pkg>
Remove	uv remove <pkg>
Upgrade	uv lock --upgrade-package <pkg>
Run inside env	uv run <command>
Sync	uv sync (use --all-groups to cover dev+agent+optional)

poetry

default → [tool.poetry.dependencies]; dev → --group dev; agent → --group agent; optional → --group <name>.

Action	Command
Add runtime	poetry add <pkg>
Add dev	poetry add --group dev <pkg>
Add to group	poetry add --group <name> <pkg>
Remove	poetry remove <pkg>
Upgrade	poetry update <pkg>
Run	poetry run <command>
Sync	poetry install

hatch

Declarative — no universal hatch add. Edit pyproject.toml:[project] dependencies or [tool.hatch.envs.<env>.dependencies], then any hatch run -e <env> <command> re-creates the env. Caveat: hatch envs do not compose; each non-default env duplicates runtime deps.

conda / mamba

No native feature concept; map buckets to named envs (<project>, <project>-dev, <project>-agent).

Action	Command
Add (conda-forge)	conda install -n <env> -c conda-forge <pkg>
With mamba	mamba install -n <env> -c conda-forge <pkg>
Remove	conda remove -n <env> <pkg>
Sync from yml	conda env update -f environment.yml --prune

pip + venv

Least-integrated. No manifest update — pip install mutates the live env without tracking. Recommend migration to a managed alternative.

Editable workspace install (src/<pkg>/) per manager: → references/editable_workspace.md.

Pixi (copy this):

pixi add --pypi "<pkg> @ ."
# pixi.toml:
<pkg> = { path = ".", editable = true }
pixi install

Then from <pkg>.pipeline import build_learner works from any CWD.

Agent feature install

The agent feature = project-scoped install of ipython + pyright

• the bundled pyrightconfig.json (substituting <PYTHON_PATH> for the lsp env's interpreter).

Bundled scripts — one per manager

Manager	Invocation	Args
pixi	bash .agents/skills/python-env-manager/scripts/install_agent_feature_pixi.sh	none
uv	bash .agents/skills/python-env-manager/scripts/install_agent_feature_uv.sh	none
poetry	bash .agents/skills/python-env-manager/scripts/install_agent_feature_poetry.sh	none
hatch	bash .agents/skills/python-env-manager/scripts/install_agent_feature_hatch.sh	none (requires user-authored [tool.hatch.envs.agent] + [tool.hatch.envs.lsp])
conda	bash .agents/skills/python-env-manager/scripts/install_agent_feature_conda.sh <project-name>	project name
pip+venv	bash .agents/skills/python-env-manager/scripts/install_agent_feature_pip_venv.sh <requirements-file>	requirements file

Run the script, don't retype. Each script encodes per-manager footguns (poetry's virtualenvs.in-project, hatch's no-composition, conda's machine-local paths). Re-typing by hand is the named forbidden shortcut.

Per-script anatomy (the 4 actions inside, post-install runner invocation, cleanup, verification): → references/agent_feature_anatomy.md.

Per-manager footguns: → references/per_manager_footguns.md.

→ next: return to caller (typically audit-ml-pipeline).

Tier 1 install: skore variant per mode

Read skore mode: from journal/JOURNAL.md Status Workspace decisions (set by organize-ml-workspace § G-SKORE-MODE). The variant pulls in two orthogonal axes: mode (local / hub / mlflow) and package source (conda-forge for pixi / conda+mamba, PyPI for uv / poetry / hatch / pip+venv). PyPI installs need an extra jupyter extra; conda-forge installs already ship the jupyter integration.

skore mode:	conda-forge managers (pixi, conda / mamba)	PyPI managers (uv, poetry, hatch, pip+venv)
local	pixi add skore / conda install -c conda-forge skore	uv add "skore[jupyter]" / poetry add "skore[jupyter]" / pip install "skore[jupyter]"
hub	pixi add "skore[hub]" / conda install -c conda-forge "skore[hub]"	uv add "skore[hub,jupyter]" / poetry add "skore[hub,jupyter]" / pip install "skore[hub,jupyter]"
mlflow	pixi add "skore[mlflow]" "mlflow>=3" / conda install -c conda-forge "skore[mlflow]" "mlflow>=3"	uv add "skore[mlflow,jupyter]" "mlflow>=3" / poetry add "skore[mlflow,jupyter]" "mlflow>=3" / pip install "skore[mlflow,jupyter]" "mlflow>=3"

The mlflow variant must pin mlflow>=3 explicitly (shown in the commands above). The skore[mlflow] extra's mlflow lower bound is loose, so the solver can otherwise resolve an old mlflow (2.x) that the skore MLflow backend does not support — add mlflow>=3 to the install command on conda-forge and PyPI alike.

If the row is absent (workspace not yet bootstrapped through organize-ml-workspace), route back to that skill's G-SKORE-MODE. Do not guess.

Forbidden:

• Silently picking skore[hub] / skore[mlflow] "to be safe". The [hub] / [mlflow] extras cost network deps + infra the local-mode user didn't ask for; the variant follows the recorded skore mode:, not a guess.
• Installing the mlflow variant without the explicit mlflow>=3 pin. skore[mlflow] alone can resolve mlflow 2.x; the skore MLflow backend needs mlflow 3+. Always co-install mlflow>=3.
• Dropping the jupyter extra on PyPI installs because the install line "looks shorter". The TableReport / report.* widgets that the audit flow and evaluate-ml-pipeline rely on fail to render without it on uv / poetry / hatch / pip+venv.
• Adding the jupyter extra on pixi / conda installs. Redundant — conda-forge skore already pulls the jupyter integration in.

Why the variant matters, mode-switching procedure, the [jupyter] extra rationale: → references/skore_variant.md.

skrub install — macOS post-install

When skrub is being installed (or has just been installed) and the platform is macOS, run dot -c in the project's env once the install lands. This rebuilds graphviz's plugin / format cache; skipping it leaves the first .skb.draw_graph() / .skb.full_report() call printing format warnings or erroring out on font lookup.

# right after the skrub install command lands, on macOS only:
[[ "$(uname)" == "Darwin" ]] && pixi run dot -c

Per manager, swap the env-run prefix: pixi run / uv run / poetry run / hatch run / conda run -n <env> / activated venv → bare dot -c. Linux + Windows: no-op, skip the call. One-shot — no need to re-run on subsequent sessions unless graphviz itself was reinstalled.

Bootstrap — when no manager is detected

If detection found nothing AND the user picked pixi via G-ENV-MGR:

1. Check command -v pixi; surface install URL if missing.

2. pixi init.

3. Edit pixi.toml: declare 3 features (default / dev / agent) + 4 envs (default / dev / agent / lsp). dev carries ruff, pytest, jupyterlab, ipykernel; agent carries ipython, pyright.

4. Add Tier 1 deps to default (per G-SKORE-MODE table above — pixi is conda-forge, so pixi add skore, pixi add "skore[hub]", or pixi add "skore[mlflow]" "mlflow>=3"; no [jupyter] extra on pixi).

5. Add tabular lib (per G-TABULAR: pandas pyarrow or polars).

6. Wire editable workspace package — copy this:

   pixi add --pypi "<pkg> @ ."
   # then in pixi.toml:
   <pkg> = { path = ".", editable = true }
   pixi install
   

   Then from <pkg>.pipeline import build_learner works from any CWD, with no PYTHONPATH=src hack and no re-install after each source edit.

7. Drop pyrightconfig.json via sed-substitution of <PYTHON_PATH> for .pixi/envs/lsp/bin/python.

8. Sync all 4 envs: pixi install then pixi install -e dev / -e agent / -e lsp.

Full step-by-step with exact pixi.toml block, manager-equivalent flows, pixi-version compatibility notes: → references/bootstrap.md.

→ next: return to organize-ml-workspace § scaffold.

Companion skills

Skill	Relationship
data-science-python-stack	Owns what to install; this skill turns it into a command
organize-ml-workspace	Scaffold hands off here for editable install; G-TABULAR / G-SKORE-MODE feed this skill's bootstrap
audit-ml-pipeline / explore-ml-data	G-AGENT-FEATURE fires from there (audit files; data/eda.py)
build-ml-pipeline / evaluate-ml-pipeline	Missing-dep Stop conditions redirect here
iterate-ml-experiment	Owns the Workspace decisions block this skill reads / writes

Conventions

• One install operation per response. Don't batch unrelated packages. Group related (Tier 1 bootstrap, or a single feature's deps) and confirm before continuing.
• No --no-deps or version pins by default. Pin only on user request or known incompatibility.
• Surface, don't bypass. If an install fails, surface the error
  • command. Don't try alternative managers as a workaround — that's a Stop-condition violation.