tinybird-deploy

Tinybird Deployment Skill

Deploy observability pipes and datasources to Tinybird Cloud.

Requirements

• Preflight: which -a tb && tb --version. If tb is missing or resolves to a pyenv/pip shim instead of ~/.local/bin/tb, PATH is wrong or the Forward CLI isn't installed — fix PATH or point the user at the install docs; don't curl-pipe an installer.
• Use the Tinybird Forward CLI as tb. On this machine it should resolve to ~/.local/bin/tb and support tb --cloud deployment create --check.
• Do not install or update this workflow with pip install tinybird-cli; that can put the Classic CLI first on PATH.
• If tb --cloud is missing, or Tinybird says this is a Forward workspace but the CLI is Classic, fix PATH so ~/.local/bin wins. The Classic CLI is kept only as tb-classic.
• Run commands from enter.pollinations.ai/observability.
• Set TB_TOKEN explicitly to a token with WORKSPACE:DEPLOY for the target workspace, and pass --host on every deploy command. Do not rely on .tinyb or tb workspace use for workspace selection.
• SELECT 1 succeeding doesn't mean a token can read datasources (PIPES:READ vs DATASOURCES:READ vs WORKSPACE:DEPLOY). Probe with a SELECT on a known datasource before relying on a token for a new path.

Workspaces

Two workspaces, same region (gcp-europe-west2). Pipes and datasources must be kept in sync across both.

Workspace	Receives traffic from	UI
pollinations_enter	production worker only	https://cloud.tinybird.co/gcp/europe-west2/pollinations_enter
pollinations_enter_staging	staging worker + dev worker + local npm run dev	https://cloud.tinybird.co/gcp/europe-west2/pollinations_enter_staging

Workspace routing is token-scoped: the same regional host serves both workspaces, and the token selects the workspace.

The local .tinyb is gitignored and must not be trusted for prod/staging selection. Set TB_TOKEN from the macOS Keychain, not from Enter runtime SOPS files (those only hold read/ingest tokens for the Worker). Use a staging-workspace deploy token for staging and a prod-workspace deploy token for prod. Keep tokens out of logs.

Operator deploy tokens live in the macOS Keychain (same pattern as the SOPS age key):

# staging (pollinations_enter_staging)
TB_TOKEN="$(security find-generic-password -a "$USER" -s tinybird-staging-deploy -w)"
# prod (pollinations_enter)
TB_TOKEN="$(security find-generic-password -a "$USER" -s tinybird-prod-deploy -w)"

Always sanity-check which workspace a token targets before deploying: tb --cloud --host "$TB_HOST" info prints workspace_name.

Directory Structure

enter.pollinations.ai/observability/
├── datasources/       # Data source definitions (.datasource)
│   ├── generation_event_v2.datasource
│   ├── polar_event.datasource
│   ├── stripe_event.datasource
│   └── ...
└── endpoints/         # Pipe definitions (.pipe)
    ├── weekly_usage_stats.pipe
    ├── weekly_active_users.pipe
    ├── daily_stripe_revenue.pipe
    └── ...

---

Commands

Set the region once per shell session and pull the staging deploy token from the Keychain:

TB_HOST="https://api.europe-west2.gcp.tinybird.co"
TB_TOKEN="$(security find-generic-password -a "$USER" -s tinybird-staging-deploy -w)"
export TB_HOST TB_TOKEN

Step 1: Validate (Dry Run)

Always validate before deploying. This is safe to run against staging.

tb --cloud --host "$TB_HOST" deployment create --check --no-allow-destructive-operations

Example output:

| status   | name                  | type     | path                                 |
|----------|-----------------------|----------|--------------------------------------|
| modified | weekly_usage_stats    | endpoint | endpoints/weekly_usage_stats.pipe    |

Step 2: Deploy

If validation passes, deploy to staging first. deployment create --wait creates a staging deployment and waits for it to be ready. It does not promote unless --auto is passed.

tb --cloud --host "$TB_HOST" deployment create --wait --no-allow-destructive-operations

Never pass --auto or run deployment promote unless the user explicitly asks for promotion.

Prod deploys use the same command shape after explicitly replacing TB_TOKEN with the tinybird-prod-deploy Keychain token, but only after staging validation and verification. Deploying to both workspaces is still manual until #11127 is resolved.

Staging and prod are separate approvals — don't batch them into one command, since the user may want to look at staging first. But if the instruction already names both ("deploy to staging and prod"), that is both approvals; don't re-ask in between.

Step 3: Verify

Verify the staging deployment and test the deployed endpoints with the read token for the same workspace.

set -o pipefail
tb --staging --cloud --host "$TB_HOST" endpoint ls
tb --cloud --host "$TB_HOST" deployment ls

TINYBIRD_TOKEN="$(SOPS_AGE_KEY=$(security find-generic-password -a "$USER" -s sops-age-key -w 2>/dev/null || true) \
  sops -d ../secrets/staging.vars.json | jq -r '.TINYBIRD_READ_TOKEN')"
curl -fsS "https://api.europe-west2.gcp.tinybird.co/v0/pipes/weekly_usage_stats.json?weeks_back=12" \
  -H "Authorization: Bearer $TINYBIRD_TOKEN" | jq -e '
    if has("error") then error(.error)
    elif (.data | type) != "array" then error("Missing data array")
    else .data | length end'

For prod verification, swap staging.vars.json to prod.vars.json and do not use --staging.

---

Safety Features

Flag	Description
--check	Validates without making changes (dry run)
--wait	Waits for deployment to complete
--no-allow-destructive-operations	Prevents removing datasources (default)
--allow-destructive-operations	Required to delete datasources
--auto	Auto-promotes a ready deployment; do not use unless explicitly requested

---

Common Tasks

Add a New Pipe

1. Create .pipe file in endpoints/
2. Validate against staging
3. Deploy to staging, verify, then prod only when requested

Modify Existing Pipe

Same as above: edit, validate staging, deploy staging, verify, then prod only when requested.

View Pipe in UI

• Prod: https://cloud.tinybird.co/gcp/europe-west2/pollinations_enter/pipes
• Staging: https://cloud.tinybird.co/gcp/europe-west2/pollinations_enter_staging/pipes

---

Troubleshooting

"tb: command not found"

Ensure ~/.local/bin is on PATH, then open a new shell. Do not install the Classic CLI with pip install tinybird-cli for this workflow.

"This is a Tinybird Forward workspace" / Classic CLI errors

which -a tb
tb --version
tb --help | rg "deployment"

tb should be the Forward CLI. If the first tb path is under /Library/Frameworks/Python.framework/..., PATH is wrong; use ~/.local/bin/tb or fix PATH.

Validation Reports Datasource or Pipe Deletion

Investigate the deletion. Restore accidentally missing definitions from a staging pull in temp/. For an intentional staging migration/reset within the task, verify the workspace and rerun with --allow-destructive-operations under root AGENTS.md's disposable-staging rule. Production deletions require explicit permission.

Pipe Timeout Issues

If a pipe times out with large weeks_back:

• Use uniq() instead of uniqExact() for user counts (~10x faster)
• Avoid CTE + JOIN patterns - use single-pass queries
• Consider materialized views for expensive aggregations

Materialized View Validation Issues

• Forward materialized views cannot use UNION; split sources into separate materialized pipes that write to the same datasource.
• Validate with deployment create --check; older local checks are not enough.
• Query-time dedup can fail validation in some endpoint pipes. Prefer dedup in the materialization when possible.

---

Important Notes

• Always use --cloud: Without it, CLI tries to use Tinybird Local.
• Do NOT use tb push: It is deprecated for this workflow.
• Avoid tb deploy: Use explicit deployment create commands so promotion is never accidental.
• Destructive operations: Follow root AGENTS.md's staging and production rules; the commands above default to non-destructive validation and deployment.
• Run from observability directory: Not from repo root.