Kebivo

Kebivo: self-hosted knowledge gap feedback for AI agents

Collect the questions your AI assistant could not answer.

Kebivo collects unanswered questions and proposed knowledge corrections from AI agents over REST or MCP. It groups repeat reports and routes them to the people who maintain the knowledge, through an admin inbox, email or signed webhooks. Run it yourself in one container with no cloud account or AI provider required.

Kebivo works alongside observability and tracing tools such as Langfuse and Phoenix. Those tools show engineers what happened during a run. Kebivo collects gaps reported by the agent and sends them to the people who maintain the knowledge. It is not a RAG engine: it does not score traces, retrieve documents or answer questions. See What it does not do.

In the demo, an assistant can't answer a question and the agent calls report_knowledge_gap over MCP. A routing rule delivers the report to the docs owner by webhook. Repeat questions join the same group, so the count goes up without a second alert. The owner adds a note and marks the group resolved. The product frames are screenshots of a real Kebivo container with synthetic data; the chat and end card are drawn. The source is in docs/assets/demo-src.

Status: 0.1.0, the first release. Run the published image ghcr.io/ocnai/kebivo:0.1.0 (linux/amd64, linux/arm64) or build it from this repository. Licensed under Apache-2.0.

Start with the Quickstart. For framework tutorials, see the Kebivo Guides. For deployment, API behavior and limits, use the technical documentation.

Why

When an assistant cannot answer, the documentation owner may never see the question. The same question can fail again, and corrections offered in chat may never reach the people who maintain the content. Kebivo collects those reports in one place, where owners can triage and route them to the tools they use.

When AI assistants cannot answer questions, Kebivo sends those knowledge gaps to a documentation or support owner for review. It groups repeated questions so the owner can see what keeps coming up. It can sit alongside a RAG application or agent framework; it does not retrieve content or update a knowledge base for you.

Features

• Send reports from application code through REST, or give agents the Streamable HTTP MCP tools report_knowledge_gap and submit_knowledge_correction.
• Create one project per assistant or team, with default tags, a corrections policy, and bearer tokens that are shown once, can expire and can be revoked.
• Route reports with ordered rules, ANY/ALL tag matching, several destinations per rule, a fallback, and a dry-run simulator.
• Deliver to signed webhooks with a versioned JSON payload, SMTP email, or the built-in inbox. Deliveries have durable retries with backoff, restart recovery, a delivery log and manual retry.
• Identical questions are grouped automatically. You can also configure an LLM provider for semantic grouping.
• Triage reports in the inbox with search, filters, statuses, notes, tags, merge, split and deletion. Each group has a full history.
• Enable OpenAI-compatible, Azure OpenAI or local (Ollama, LM Studio) providers for semantic grouping on a per-project basis. This is off by default.
• Run one container with one SQLite file. Startup migrations, online backup and checked restore, retention (on by default), JSON Lines export, and health checks are included.

Screenshots

All screenshots use synthetic data.

Dashboard	Inbox
Feedback group	Connect a client
Routing rules	Failed delivery

More: destinations, settings, LLM settings, sign-in. Regenerate them with make e2e-ui SCREENSHOTS=1.

What it does not do

• It does not answer questions, edit your knowledge base or apply corrections. People review everything.
• It is not a chatbot, a ticketing system or an analytics product. It hands feedback to the tools you use.
• No multi-user accounts, SSO or roles: there is one local administrator.
• No hosted service, no telemetry and no account. The default install makes no outbound connection and needs no AI provider.
• No high availability: one process, one SQLite file on local storage.

Quickstart

You need Docker with Compose, OpenSSL and a free port 8000.

1. Start Kebivo.

git clone <REPOSITORY_URL> kebivo
cd kebivo
cp .env.example .env
printf 'SECRET_KEY=%s\n' "$(openssl rand -hex 32)" >> .env
printf 'ENCRYPTION_KEY=%s\n' "$(openssl rand -hex 32)" >> .env
printf 'ALLOWED_ORIGINS=http://127.0.0.1:8000\n' >> .env
docker compose up --build -d
curl -fsS http://127.0.0.1:8000/health/ready

<REPOSITORY_URL> is the address you cloned from. The first build takes a few minutes. Readiness returns {"status":"ok",...} once migrations are applied.

Or run the published image instead of cloning and building. It is the same container; keep kebivo.env private and back it up apart from the data volume.

(umask 077; printf 'SECRET_KEY=%s\nENCRYPTION_KEY=%s\nALLOWED_ORIGINS=http://127.0.0.1:8000\n' \
  "$(openssl rand -hex 32)" "$(openssl rand -hex 32)" > kebivo.env)
docker run -d --name kebivo --restart unless-stopped -p 127.0.0.1:8000:8000 \
  -v kebivo-data:/data --env-file kebivo.env ghcr.io/ocnai/kebivo:0.1.0
curl -fsS --retry 30 --retry-all-errors --retry-delay 1 http://127.0.0.1:8000/health/ready

curl may print a few Connection reset errors while migrations run, then readiness returns {"status":"ok",...}. With this container, run the step 2 command as docker exec -it kebivo python -m kebivo.cli bootstrap --email [email protected], and stop it with docker stop kebivo; data stays in the kebivo-data volume.

2. Create the administrator. The command asks for a password (at least 12 characters).

docker compose exec app python -m kebivo.cli bootstrap --email [email protected]

3. Create a project and a token. Open http://127.0.0.1:8000/ (exactly this address: localhost is a different origin and sign-in would fail) and sign in. Create a project, then under Project tokens issue a token with the feedback:gap scope and copy it. It is shown only once.

4. Send your first event.

export KEBIVO_TOKEN='<PROJECT_TOKEN>'
curl -i http://127.0.0.1:8000/api/v1/feedback \
  -H 'Content-Type: application/json' \
  -H "Authorization: Bearer $KEBIVO_TOKEN" \
  -H 'Idempotency-Key: first-event-001' \
  --data '{"event_type":"knowledge_gap","question":"How do I reset my corporate VPN?","tags":["it"]}'

The answer is 202 Accepted with an event_id. 5. Open the Inbox in the UI: the question is there in its own group. Add a destination and a routing rule to send the next ones to email or a webhook.

To stop: docker compose down (data is kept). docker compose down --volumes deletes all data. For production (TLS, backups, upgrades, every setting), read the self-hosting guide.

Collect AI agent feedback with REST

Send POST /api/v1/feedback with a project token. A 202 means the report is stored durably; routing and delivery happen in the background. Send an Idempotency-Key and reuse it on retries. Retry 429 and 5xx responses with backoff; do not retry other 4xx errors.

Runnable clients for cURL, Python (standard library) and TypeScript (Node.js 24, no build step) are in examples/, and the project page in the UI shows the same snippets with your endpoint. The REST integration guide covers fields, authentication, idempotency, errors, retries and rate limits.

Report knowledge gaps with MCP

Point an MCP client at http://127.0.0.1:8000/mcp with the header Authorization: Bearer <PROJECT_TOKEN>. For Claude Code:

claude mcp add --transport http kebivo http://127.0.0.1:8000/mcp --header "Authorization: Bearer <PROJECT_TOKEN>"

Tokens with feedback:gap see report_knowledge_gap; submit_knowledge_correction also needs feedback:correction and a project that accepts corrections. A model may not call a tool when it should, so use REST where every report matters. See the MCP guide, a client configuration, a Python client and copyable agent instructions.

Destinations and routing

Destination	What it sends
Built-in inbox	Always available; every event is in the admin inbox.
Webhook	HTTPS POST with a versioned JSON payload, Standard Webhooks signatures, optional bearer or custom headers. A receiver example verifies both.
SMTP email	A message (plain text with an escaped HTML alternative) to 1–20 recipients over STARTTLS, implicit TLS, or an unauthenticated local relay.

Each project has ordered rules: a rule matches ANY or ALL of its tags and sends to one or more destinations, optionally stopping evaluation. Unmatched events go to the fallback destinations. For example, it → IT email, customer-service → the support webhook. Each destination delivers either for new groups only or for every occurrence. See routing and destinations.

Optional AI

No model is called by default. To group questions that are worded differently, configure an LLM provider (LLM settings in the UI) and switch a project to semantic deduplication after acknowledging that question text will be sent to that provider. If the provider fails, the event keeps its own group and is still routed and delivered. See deduplication and LLM providers.

Security

• Run behind a TLS reverse proxy and keep the app port on loopback (guide).
• Keep .env private. Back up SECRET_KEY and ENCRYPTION_KEY separately from database backups.
• Feedback is stored unencrypted in SQLite. Protect the volume and backups, and do not ask assistants to send secrets or unnecessary personal data.
• Give each client its own token, give customer-facing agents only feedback:gap, and revoke tokens you no longer use.
• Webhooks and LLM providers cannot reach private or loopback addresses unless you opt in per destination or provider; cloud metadata addresses are always blocked, also for SMTP. Admin writes need the configured Origin and a CSRF token.
• Logs never contain feedback content or credentials.

Report vulnerabilities as described in SECURITY.md.

Data, retention and export

Retention is on by default: feedback older than 180 days, finished delivery attempt details older than 30 days, and audit entries older than 90 days are deleted daily. Change or disable each with an environment variable (0 turns it off). Retention does not touch backups you already made. An administrator can export a project's feedback as JSON Lines. See retention and export.

Known limits

• At-least-once delivery. A receiver may see a delivery twice; deduplicate by the webhook ID or delivery ID.
• One administrator, one instance, one SQLite file on local storage. No replicas, no network filesystem.
• MCP needs a manually configured bearer token; OAuth-only clients cannot connect. Responses are stateless JSON without streaming notifications.
• Rate limits are counted in memory and reset when the container restarts.
• Deleted feedback cannot be recalled from destinations that already received it or from existing backups.
• No encryption at rest for feedback content.

Documentation

Framework tutorials are in the website Guides. Runnable clients and framework examples are in examples/. The pages below cover the technical details.

Guide	Contents
Self-hosting	Requirements, every setting, TLS proxy, secrets, backup and restore, upgrades, credential rotation, troubleshooting
REST API	Authentication, fields, idempotency, errors, retries, examples
MCP	Connecting clients, tools and permissions, testing, limitations
Routing and destinations	Rules, fallback, webhooks, signatures, SMTP
Deduplication and LLM providers	Exact and semantic grouping, providers, privacy
Feedback management	Groups, search syntax, statuses, merge, split, deletion
Retention and export	Automatic deletion, JSON Lines export
Releasing	Versioning, image labels, release and upgrade checks
Third-party licenses	Inventory of shipped dependencies
Changelog	User-visible changes per version

Contributing

Development setup, make targets, style, tests and the documentation rule are in CONTRIBUTING.md. In short: make install, then make check before every commit, and update the docs in the same change. Sign off every commit (git commit -s) under the Developer Certificate of Origin; there is no CLA.

Everyone taking part follows the Code of Conduct. Ideas under consideration, none of them committed, are in the roadmap.

License

Kebivo is licensed under the Apache License 2.0. Contributions are accepted under the same license with a Developer Certificate of Origin sign-off; there is no CLA. The third-party license inventory lists the licenses of everything the image ships.