mcpField
C
chromadb-integration-skills

chromadb-integration-skills

github.com/kimasplund数据与数据库549 次安装2 周前更新
代码仓库

关于 chromadb-integration-skills

chromadb-integration-skills 教授各类 Agent 使用 ChromaDB 进行语义检索和持久化数据管理,涵盖集合设计、数据导入、搜索、模式匹配及运维实践。

主要功能

  1. 01集合与元数据设计
  2. 02文档导入与去重
  3. 03带筛选条件的语义搜索
  4. 04跨集合模式匹配
  5. 05错误处理与测试模式

适用场景

  1. 01搜索大量研究论文
  2. 02查找重复出现的代码缺陷模式
  3. 03比较历史交易回测结果
  4. 04整理文档与测试用例

MCP 相关

介绍使用 ChromaDB MCP 工具的工作流,包括集合管理、文档导入和查询。

安装

复制下面的命令,在终端中运行。

$npx skills add https://github.com/kimasplund/claude_cognitive_reasoning --skill chromadb-integration-skills

安装后在 Claude Code 中输入/chromadb-integration-skills确认已加载。

包含的文件

除 SKILL.md 外,安装时一起下载的参考资料和脚本。

chromadb-error-handling.md
chromadb-lifecycle-management.md
chromadb-security-patterns.md
chromadb-testing-patterns.md

详细介绍

ChromaDB 集成技能

用途:本技能教 Agent 如何集成 ChromaDB,以便在任何领域(研究、代码、交易、法律、文档等)实现语义搜索、持久化存储和模式匹配。

关键用例:当 Agent 需要处理大型数据集(1000+ 项)、执行语义搜索、维护持久化知识或从历史模式中学习时,ChromaDB 可以突破令牌限制,并支持强大的基于向量的检索。

使用者:所有类型的 Agent——研究人员、开发人员、交易员、法律分析师、文档撰写人员、QA 测试人员等。


何时使用 ChromaDB 集成

在以下情况下使用 ChromaDB:

  • 大型数据集:处理 1000+ 项(文档、代码文件、Bug、交易、合同等)
  • 语义搜索:按含义而非仅按关键词查找项目
  • 持久化记忆:知识需要跨会话、数天或数月保留
  • 模式匹配:识别相似的历史案例或模式,以辅助决策
  • 跨会话学习:随着时间推移积累机构知识
  • 令牌限制:数据量过大,无法放入上下文窗口(100K+ 个令牌)
  • 聚合:合并来自多个查询或来源的结果

ChromaDB 核心概念

集合

定义:存储带有嵌入和元数据的文档的命名向量数据库

命名策略:

  • 按领域命名:{domain}_{purpose}_{identifier}
  • 示例:
    • 研究:research_prior_art_blockchain_2024、research_literature_ml_transformers
    • 代码:codebase_api_endpoints、codebase_bug_patterns_auth
    • 交易:backtest_results_sma_strategy、market_conditions_spy_2024
    • 法律:case_law_patent_eligibility、contracts_saas_clauses
    • 文档:api_docs_v2、architecture_decisions_2024
文档

定义:用于语义搜索的文本内容

最佳实践:

  • 分块大小:200-500 个词为最佳(过小会丢失上下文,过大会降低粒度)
  • 内容格式:标题 + 摘要 + 关键细节(例如,"Patent US10123456 - Blockchain Authentication. Abstract: A method for..."))
  • 去重:使用唯一 ID,防止重复存储
元数据

定义:用于筛选而非语义搜索的结构化数据

策略:

{
  // Temporal filters
  "date": "2024-11-14",
  "year": 2024,
  "month": 11,

  // Categorical filters
  "type": "bug_report",
  "category": "authentication",
  "severity": "high",

  // Numeric filters
  "citations": 42,
  "price": 150.25,
  "performance_score": 0.87,

  // Source tracking
  "source": "github_issue",
  "author": "kim-asplund",
  "url": "https://..."
}
嵌入

定义:用于实现语义相似度的向量表示

工作原理:

  • ChromaDB 会根据文档文本自动生成嵌入
  • 含义相似 → 向量相似 → 在向量空间中距离接近
  • 距离度量(余弦、欧几里得)用于衡量相似度

通用 ChromaDB 工作流

阶段 1:集合设计
// Step 1: Design collection strategy based on agent type
const collectionStrategy = {
  research_agent: "One collection per research topic/question",
  code_agent: "Collections by codebase module/feature",
  trading_agent: "Collections by strategy/timeframe/symbol",
  legal_agent: "Collections by practice area/jurisdiction",
  documentation_agent: "Collections by project/version"
};

// Step 2: Create collection with descriptive metadata
mcp__chroma__create_collection({
  collection_name: "{domain}_{purpose}_{identifier}",
  embedding_function_name: "default",  // Uses sentence transformers
  metadata: {
    created_date: "2024-11-14",
    domain: "research|code|trading|legal|docs",
    purpose: "Descriptive purpose",
    total_items: 0,  // Will update
    last_updated: "2024-11-14"
  }
});
阶段 2:数据摄取
// Step 1: Batch data collection (minimize API calls)
const items = collectAllItems();  // From API, files, database, etc.

// Step 2: Transform to ChromaDB format
const documents = items.map(item => formatDocument(item));
const ids = items.map(item => item.id || generateUniqueId());
const metadatas = items.map(item => extractMetadata(item));

// Step 3: Batch insert (ChromaDB handles chunking automatically)
mcp__chroma__add_documents({
  collection_name: collectionName,
  documents: documents,
  ids: ids,
  metadatas: metadatas
});

// Step 4: Update collection metadata
mcp__chroma__modify_collection({
  collection_name: collectionName,
  new_metadata: {
    ...existingMetadata,
    total_items: items.length,
    last_updated: new Date().toISOString()
  }
});
阶段 3:语义搜索
// Step 1: Formulate semantic query (natural language works!)
const query = "authentication failures in production environment";

// Step 2: Execute semantic search with filters
const results = mcp__chroma__query_documents({
  collection_name: collectionName,
  query_texts: [query],
  n_results: 20,
  where: {
    "$and": [
      { "environment": "production" },
      { "severity": { "$in": ["high", "critical"] } },
      { "date": { "$gte": "2024-01-01" } }
    ]
  },
  include: ["documents", "metadatas", "distances"]
});

// Step 3: Filter by semantic similarity (distance threshold)
const highlyRelevant = results.ids[0].filter((id, idx) =>
  results.distances[0][idx] < 0.3  // Adjust threshold based on use case
);

// Step 4: Retrieve full details if needed
const fullDetails = mcp__chroma__get_documents({
  collection_name: collectionName,
  ids: highlyRelevant,
  include: ["documents", "metadatas"]
});
阶段 4:模式匹配
// Cross-collection pattern detection
const allCollections = mcp__chroma__list_collections();
const relevantCollections = allCollections.filter(c =>
  c.startsWith(collectionPrefix)
);

const patterns = [];
for (const collection of relevantCollections) {
  const matches = mcp__chroma__query_documents({
    collection_name: collection,
    query_texts: [patternQuery],
    n_results: 10,
    where: { "outcome": "success" }  // Only successful cases
  });

  if (matches.ids[0].length > 0) {
    patterns.push({
      collection: collection,
      matches: matches,
      success_rate: calculateSuccessRate(matches)
    });
  }
}

// Identify best pattern
const bestPattern = patterns.sort((a, b) =>
  b.success_rate - a.success_rate
)[0];

用例模板

模板 1:研究 Agent——文献综述

问题:存储 1000+ 篇研究论文,并查找语义相似的研究

// Collection: research_literature_{topic}
const papers = fetchPapersFromAPI("machine learning transformers");

mcp__chroma__create_collection({
  collection_name: "research_literature_ml_transformers",
  metadata: { topic: "ML Transformers", papers_count: 0 }
});

// Store papers with rich metadata
papers.forEach(paper => {
  mcp__chroma__add_documents({
    collection_name: "research_literature_ml_transformers",
    documents: [`${paper.title}. ${paper.abstract}`],
    ids: [paper.doi || paper.id],
    metadatas: [{
      title: paper.title,
      authors: paper.authors.join(", "),
      year: paper.year,
      citations: paper.citation_count,
      venue: paper.venue,
      url: paper.url
    }]
  });
});

// Semantic search: "Find papers about attention mechanisms for vision"
const relevant = mcp__chroma__query_documents({
  collection_name: "research_literature_ml_transformers",
  query_texts: ["attention mechanisms computer vision"],
  n_results: 20,
  where: { "year": { "$gte": 2020 }, "citations": { "$gte": 50 } }
});

优势:无令牌限制、语义发现、引用筛选、持久化文献库


模板 2:代码 Agent——Bug 模式识别

问题:存储 Bug 报告、识别相似问题并提出解决方案

// Collection: codebase_bug_patterns_{module}
const bugs = fetchAllGitHubIssues("is:issue label:bug");

mcp__chroma__create_collection({
  collection_name: "codebase_bug_patterns_auth",
  metadata: { module: "authentication", total_bugs: 0 }
});

// Store bugs with solutions
bugs.forEach(bug => {
  mcp__chroma__add_documents({
    collection_name: "codebase_bug_patterns_auth",
    documents: [`Bug #${bug.number}: ${bug.title}. ${bug.body}`],
    ids: [`bug_${bug.number}`],
    metadatas: [{
      number: bug.number,
      title: bug.title,
      severity: bug.labels.find(l => l.startsWith("severity:"))?.split(":")[1],
      status: bug.state,
      solution: bug.resolution || "No solution yet",
      created_at: bug.created_at,
      resolved_at: bug.closed_at,
      url: bug.html_url
    }]
  });
});

// New bug arrives - find similar historical bugs
const newBugDescription = "User login fails with 401 error after password reset";

const similarBugs = mcp__chroma__query_documents({
  collection_name: "codebase_bug_patterns_auth",
  query_texts: [newBugDescription],
  n_results: 10,
  where: { "status": "closed", "solution": { "$ne": "No solution yet" } }
});

// Extract solution from most similar resolved bug
const suggestedSolution = similarBugs.metadatas[0][0].solution;

优势:即时 Bug 模式匹配、复用解决方案、检测相似问题


模板 3:交易 Agent——回测结果数据库

问题:存储 10,000+ 条回测结果,并识别最佳参数模式

// Collection: backtest_results_{strategy_name}
const backtests = runParameterSweep(strategyCode, parameterRanges);

mcp__chroma__create_collection({
  collection_name: "backtest_results_sma_crossover",
  metadata: { strategy: "SMA Crossover", total_backtests: 0 }
});

// Store each backtest with parameters + results
backtests.forEach(backtest => {
  const description = `
    SMA Crossover strategy with fast=${backtest.params.fast_period},
    slow=${backtest.params.slow_period}, stop_loss=${backtest.params.stop_loss}.
    Market conditions: ${backtest.market_regime}, volatility=${backtest.avg_volatility}.
  `;

  mcp__chroma__add_documents({
    collection_name: "backtest_results_sma_crossover",
    documents: [description],
    ids: [`backtest_${backtest.id}`],
    metadatas: [{
      fast_period: backtest.params.fast_period,
      slow_period: backtest.params.slow_period,
      stop_loss: backtest.params.stop_loss,
      sharpe_ratio: backtest.sharpe_ratio,
      max_drawdown: backtest.max_drawdown,
      win_rate: backtest.win_rate,
      total_return: backtest.total_return,
      market_regime: backtest.market_regime,
      symbol: backtest.symbol,
      timeframe: backtest.timeframe,
      start_date: backtest.start_date,
      end_date: backtest.end_date
    }]
  });
});

// Find optimal parameters for current market conditions
const currentMarket = analyzeCurrentMarket();
const marketDescription = `
  Market regime: ${currentMarket.regime}, volatility: ${currentMarket.volatility},
  trend strength: ${currentMarket.trend_strength}
`;

const optimalBacktests = mcp__chroma__query_documents({
  collection_name: "backtest_results_sma_crossover",
  query_texts: [marketDescription],
  n_results: 20,
  where: {
    "$and": [
      { "sharpe_ratio": { "$gte": 1.5 } },
      { "max_drawdown": { "$lte": -0.15 } },
      { "symbol": currentMarket.symbol }
    ]
  }
});

// Extract best parameter set
const bestParams = optimalBacktests.metadatas[0][0];

优势:参数优化、市场状态匹配、发现表现模式


模板 4:文档 Agent——样式指南执行

问题:存储 API 文档示例,确保样式一致

// Collection: api_docs_{project_version}
const existingDocs = parseAllApiDocs("./docs/api/");

mcp__chroma__create_collection({
  collection_name: "api_docs_v2",
  metadata: { version: "2.0", total_endpoints: 0 }
});

// Store documentation with style metadata
existingDocs.forEach(doc => {
  mcp__chroma__add_documents({
    collection_name: "api_docs_v2",
    documents: [doc.fullContent],
    ids: [doc.endpoint],
    metadatas: [{
      endpoint: doc.endpoint,
      method: doc.method,
      category: doc.category,
      style_score: doc.styleScore,  // Computed during ingestion
      has_examples: doc.examples.length > 0,
      has_error_codes: doc.errorCodes.length > 0,
      last_updated: doc.lastModified
    }]
  });
});

// New endpoint documented - find similar endpoints for style consistency
const newEndpoint = "POST /api/v2/users/{id}/preferences";

const similarEndpoints = mcp__chroma__query_documents({
  collection_name: "api_docs_v2",
  query_texts: [`${newEndpoint} user preferences update`],
  n_results: 5,
  where: {
    "$and": [
      { "method": "POST" },
      { "style_score": { "$gte": 0.9 } },
      { "has_examples": true }
    ]
  }
});

// Use similar endpoint as template
const template = similarEndpoints.documents[0][0];

优势:样式一致性、模板发现、自动化质量检查


模板 5:QA 测试 Agent——测试模式库

问题:存储测试用例、识别覆盖缺口并提出新测试

// Collection: test_cases_{module}
const existingTests = parseTestFiles("./tests/");

mcp__chroma__create_collection({
  collection_name: "test_cases_authentication",
  metadata: { module: "authentication", total_tests: 0 }
});

// Store test cases with coverage metadata
existingTests.forEach(test => {
  mcp__chroma__add_documents({
    collection_name: "test_cases_authentication",
    documents: [`${test.description}. Covers: ${test.coveredScenarios.join(", ")}`],
    ids: [test.id],
    metadatas: [{
      test_type: test.type,  // "unit", "integration", "e2e"
      file_path: test.filePath,
      line_number: test.lineNumber,
      last_run: test.lastRun,
      status: test.lastStatus,
      execution_time_ms: test.executionTime,
      assertions: test.assertionCount
    }]
  });
});

// New feature added - identify missing test coverage
const newFeature = "Password reset with 2FA verification";

const existingCoverage = mcp__chroma__query_documents({
  collection_name: "test_cases_authentication",
  query_texts: [newFeature],
  n_results: 10
});

// If distance > 0.5, probably not covered
const isCovered = existingCoverage.distances[0][0] < 0.5;

if (!isCovered) {
  // Suggest test cases based on similar features
  const similarFeatures = mcp__chroma__query_documents({
    collection_name: "test_cases_authentication",
    query_texts: ["password reset", "2FA verification"],
    n_results: 5
  });

  // Use similar tests as templates
  const testTemplates = similarFeatures.documents[0];
}

优势:检测覆盖缺口、发现测试模板、基于模式生成测试


高级模式

模式 1:多集合聚合

使用场景:同时搜索多个相关集合

// Example: Search all research topics for a cross-cutting concept
const researchCollections = mcp__chroma__list_collections();
const topicCollections = researchCollections.filter(c =>
  c.startsWith("research_literature_")
);

const crossTopicResults = [];
for (const collection of topicCollections) {
  const results = mcp__chroma__query_documents({
    collection_name: collection,
    query_texts: ["transfer learning"],
    n_results: 10
  });

  crossTopicResults.push({
    topic: collection.replace("research_literature_", ""),
    papers: results
  });
}

// Aggregate and rank by relevance across topics
const allPapers = crossTopicResults.flatMap(r =>
  r.papers.ids[0].map((id, idx) => ({
    id: id,
    topic: r.topic,
    distance: r.papers.distances[0][idx],
    metadata: r.papers.metadatas[0][idx]
  }))
);

const rankedPapers = allPapers.sort((a, b) => a.distance - b.distance);
模式 2:层级集合

使用场景:集合之间的父子关系

// Parent: codebase_architecture_decisions
// Children: codebase_architecture_decisions_{year}

// Create parent collection with aggregated data
mcp__chroma__create_collection({
  collection_name: "codebase_architecture_decisions",
  metadata: { type: "parent", child_collections: [] }
});

// Create child collections by year
[2022, 2023, 2024].forEach(year => {
  mcp__chroma__create_collection({
    collection_name: `codebase_architecture_decisions_${year}`,
    metadata: { type: "child", parent: "codebase_architecture_decisions", year }
  });
});

// Query strategy: Try child first (faster), fallback to parent
const queryYear = 2024;
let results = mcp__chroma__query_documents({
  collection_name: `codebase_architecture_decisions_${queryYear}`,
  query_texts: [query],
  n_results: 10
});

if (results.ids[0].length < 5) {
  // Not enough results in child, query parent
  results = mcp__chroma__query_documents({
    collection_name: "codebase_architecture_decisions",
    query_texts: [query],
    n_results: 10
  });
}
模式 3:时间衰减

使用场景:优先处理近期条目,同时保留历史上下文

// Store items with temporal metadata
mcp__chroma__add_documents({
  collection_name: collectionName,
  documents: documents,
  ids: ids,
  metadatas: metadatas.map(m => ({
    ...m,
    timestamp: Date.now(),
    age_days: 0  // Will be updated
  }))
});

// Query with temporal boost
const results = mcp__chroma__query_documents({
  collection_name: collectionName,
  query_texts: [query],
  n_results: 50  // Get more results for re-ranking
});

// Re-rank with temporal decay
const now = Date.now();
const rankedResults = results.ids[0].map((id, idx) => {
  const ageDays = (now - results.metadatas[0][idx].timestamp) / (1000 * 60 * 60 * 24);
  const decayFactor = Math.exp(-ageDays / 30);  // Half-life ~30 days
  const semanticScore = 1 - results.distances[0][idx];
  const combinedScore = semanticScore * 0.7 + decayFactor * 0.3;

  return {
    id: id,
    semantic_score: semanticScore,
    decay_factor: decayFactor,
    combined_score: combinedScore,
    metadata: results.metadatas[0][idx]
  };
}).sort((a, b) => b.combined_score - a.combined_score);

性能优化

批处理策略
// BAD: One document at a time (slow)
for (const item of items) {
  mcp__chroma__add_documents({
    collection_name: collectionName,
    documents: [item.document],
    ids: [item.id],
    metadatas: [item.metadata]
  });
}

// GOOD: Batch insert (100x faster)
const BATCH_SIZE = 100;
for (let i = 0; i < items.length; i += BATCH_SIZE) {
  const batch = items.slice(i, i + BATCH_SIZE);
  mcp__chroma__add_documents({
    collection_name: collectionName,
    documents: batch.map(item => item.document),
    ids: batch.map(item => item.id),
    metadatas: batch.map(item => item.metadata)
  });
}
缓存策略
// Check collection exists before creating
const existingCollections = mcp__chroma__list_collections();
if (!existingCollections.includes(collectionName)) {
  mcp__chroma__create_collection({ collection_name: collectionName });
}

// Check document exists before adding
const existing = mcp__chroma__get_documents({
  collection_name: collectionName,
  ids: [documentId]
});

if (!existing.ids || existing.ids.length === 0) {
  // Document doesn't exist, add it
  mcp__chroma__add_documents({ ... });
} else {
  // Document exists, update instead
  mcp__chroma__update_documents({ ... });
}
查询优化
// Use metadata filters to reduce search space
const results = mcp__chroma__query_documents({
  collection_name: collectionName,
  query_texts: [query],
  n_results: 20,
  where: {
    // Pre-filter with metadata (faster than post-filtering semantic results)
    "date": { "$gte": "2024-01-01" },
    "category": { "$in": ["high_priority", "critical"] }
  }
});

// Only include what you need
const minimalResults = mcp__chroma__query_documents({
  collection_name: collectionName,
  query_texts: [query],
  n_results: 10,
  include: ["metadatas", "distances"]  // Exclude documents if not needed
});

成功标准

ChromaDB 集成在满足以下条件时即为成功:

  • ✅ 已创建集合:命名有意义,元数据恰当
  • ✅ 已摄取数据:高效批处理,已去重
  • ✅ 语义搜索正常:返回相关结果(距离 < 0.4)
  • ✅ 已应用元数据筛选器:正确限定搜索范围
  • ✅ 性能已优化:批处理、缓存、尽量减少查询
  • ✅ 跨集合查询:在适合使用场景时执行
  • ✅ 知识持久化:数据在不同会话之间保留
  • ✅ 模式匹配:识别相似的历史案例
  • ✅ 消除令牌限制:无需担心上下文溢出,即可处理 1000+ 个条目

技能版本:1.0 创建日期:2025-11-14 用途:教授适用于所有 Agent 类型的通用 ChromaDB 集成模式 目标质量:65/70 依赖项:ChromaDB MCP (mcp__chroma__*) 通用性:适用于研究、代码、交易、法律、文档、QA 及所有其他领域


🔴 错误处理与弹性(优先级 1)

生产环境至关重要:防止数据丢失,并妥善处理故障

指数退避重试
async function retryWithBackoff(operation, maxRetries = 3) {
  for (let attempt = 0; attempt < maxRetries; attempt++) {
    try {
      return await operation();
    } catch (error) {
      if (attempt === maxRetries - 1) throw error;
      const delay = Math.pow(2, attempt) * 1000;  // 1s, 2s, 4s
      await sleep(delay);
    }
  }
}
获取或创建模式
function getOrCreateCollection(collectionName, metadata = {}) {
  const collections = mcp__chroma__list_collections();

  if (collections.includes(collectionName)) {
    return { created: false, collection_name: collectionName };
  }

  mcp__chroma__create_collection({
    collection_name: collectionName,
    embedding_function_name: "default",
    metadata: metadata
  });

  return { created: true, collection_name: collectionName };
}
文档验证
function validateDocument(document, id, metadata) {
  if (!document || typeof document !== 'string') {
    throw new Error(`Document must be non-empty string`);
  }
  if (!id || id.includes(' ')) {
    throw new Error(`ID must be non-empty string without spaces`);
  }
  if (metadata && typeof metadata !== 'object') {
    throw new Error(`Metadata must be object`);
  }
}

查看完整模式:加载 chromadb-error-handling 子技能


🧪 测试模式(优先级 1)

质量保障必不可少:确保 ChromaDB 集成正常运行

单元测试(模拟 ChromaDB)
// Mock ChromaDB for fast unit tests
class MockChromaDB {
  constructor() {
    this.collections = {};
  }

  create_collection({ collection_name, metadata }) {
    this.collections[collection_name] = {
      documents: [], ids: [], metadatas: [], metadata
    };
  }

  query_documents({ collection_name, query_texts, n_results }) {
    const collection = this.collections[collection_name];
    return {
      ids: [collection.ids.slice(0, n_results)],
      distances: [collection.ids.slice(0, n_results).map(() => 0.2)]
    };
  }
}
集成测试(真实 ChromaDB)
describe('Semantic Search Integration', () => {
  test('returns relevant documents', async () => {
    await chromaClient.add({
      collection_name: testCollection,
      documents: [
        'Machine learning uses neural networks',
        'Python is a programming language'
      ],
      ids: ['doc1', 'doc2']
    });

    const results = await chromaClient.query({
      collection_name: testCollection,
      query_texts: ['neural networks deep learning'],
      n_results: 2
    });

    expect(results.ids[0]).toContain('doc1');
    expect(results.distances[0][0]).toBeLessThan(0.4);
  });
});

查看完整模式:加载 chromadb-testing-patterns 子技能


🔐 安全与隐私(优先级 2)

合规至关重要:保护 PII,清理数据

PII 脱敏
function redactPII(text) {
  let redacted = text;

  // Email redaction
  redacted = redacted.replace(
    /\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b/g,
    '[EMAIL_REDACTED]'
  );

  // Phone redaction
  redacted = redacted.replace(/\b\d{3}[-.]?\d{3}[-.]?\d{4}\b/g, '[PHONE_REDACTED]');

  // SSN redaction
  redacted = redacted.replace(/\b\d{3}-\d{2}-\d{4}\b/g, '[SSN_REDACTED]');

  return redacted;
}
秘密信息清理
function redactSecrets(text) {
  let redacted = text;

  // GitHub tokens
  redacted = redacted.replace(/ghp_[a-zA-Z0-9]{36}/g, '[GITHUB_TOKEN]');

  // AWS keys
  redacted = redacted.replace(/AKIA[0-9A-Z]{16}/g, '[AWS_KEY]');

  // API keys
  redacted = redacted.replace(
    /api[_-]?key['\"]?\s*[:=]\s*['\"]?([a-zA-Z0-9_-]{20,})/gi,
    'api_key: [REDACTED]'
  );

  return redacted;
}
访问控制
const collectionPermissions = {
  'research_confidential': ['research_team', 'admin'],
  'customer_pii': ['support_team', 'admin']
};

function checkAccess(collectionName, userRole) {
  const allowedRoles = collectionPermissions[collectionName] || ['admin'];

  if (!allowedRoles.includes(userRole)) {
    throw new Error(`Access denied for role '${userRole}'`);
  }
}

查看完整模式:加载 chromadb-security-patterns 子技能


🗄️ 数据生命周期管理(优先级 2)

保障生产环境持续运行:为集合设置版本,归档旧数据

架构版本控制
// Semantic versioning for collections
function createVersionedCollection(domain, purpose, version = 'v1') {
  const collectionName = `${domain}_${purpose}_${version}`;

  mcp__chroma__create_collection({
    collection_name: collectionName,
    metadata: {
      version: version,
      created_at: new Date().toISOString(),
      schema_version: '1.0',
      retention_days: 730,  // 2 years
      lifecycle_stage: 'active'
    }
  });

  return collectionName;
}
架构迁移
async function migrateCollectionSchema(oldCollection, newVersion) {
  const newCollection = `${oldCollection}_v${newVersion}`;

  // Create new collection
  await mcp__chroma__create_collection({
    collection_name: newCollection,
    metadata: { migrated_from: oldCollection, version: newVersion }
  });

  // Copy all documents with transformed metadata
  const allDocs = await mcp__chroma__get_documents({
    collection_name: oldCollection,
    limit: 100000
  });

  const transformedMetadatas = allDocs.metadatas.map(transformMetadata);

  await mcp__chroma__add_documents({
    collection_name: newCollection,
    documents: allDocs.documents,
    ids: allDocs.ids,
    metadatas: transformedMetadatas
  });

  // Mark old as deprecated
  await mcp__chroma__modify_collection({
    collection_name: oldCollection,
    new_metadata: { lifecycle_stage: 'deprecated', replacement: newCollection }
  });
}
保留策略执行
async function enforceRetentionPolicies() {
  const collections = await mcp__chroma__list_collections();

  for (const collectionName of collections) {
    const info = await mcp__chroma__get_collection_info({ collection_name });
    const retentionDays = info.metadata.retention_days || 730;
    const ageDays = calculateAgeDays(info.metadata.created_at);

    if (ageDays > retentionDays) {
      await archiveCollection(collectionName);  // Backup first
      await mcp__chroma__delete_collection({ collection_name: collectionName });
    }
  }
}

查看完整模式:加载 chromadb-lifecycle-management 子技能


🐛 调试与故障排除(优先级 1)

常见问题

问题:未返回结果

  • 原因:距离阈值过于严格,或集合有误
  • 修复:提高阈值(0.3 → 0.5),验证集合名称
  • 调试:检查 results.distances[0] 的值

问题:语义匹配效果差

  • 原因:文档分块过大或过小
  • 修复:最佳分块大小为 200-500 个词
  • 调试:检查文档长度,拆分较长的文档

问题:查询缓慢

  • 原因:集合较大且未使用元数据筛选器
  • 修复:添加元数据预筛选器(where 子句)
  • 调试:检查集合大小,添加筛选器
距离阈值指南
距离相似度使用场景
< 0.2几乎完全相同重复项检测
0.2-0.3非常相似高精度搜索
0.3-0.5中度相似平衡搜索
0.5-0.7相似度较低广泛探索
> 0.7主题不同不相关
应避免的反模式

❌ 将整个文件存储为单个文档

  • 丢失细粒度信息,搜索相关性差
  • ✅ 修复方法:拆分为 200-500 字的段落

❌ 大型集合未使用元数据筛选器

  • 查询缓慢,延迟高
  • ✅ 修复方法:始终按日期、类别和类型进行筛选

❌ 未对文档去重

  • 浪费存储空间,产生重复结果
  • ✅ 修复方法:添加前检查是否已存在

❌ 忽略连接失败

  • 数据丢失,故障无提示
  • ✅ 修复方法:实现重试逻辑和回退机制

📚 子技能参考

加载目标子技能以深入了解:

  • chromadb-error-handling:重试模式、验证、熔断器(约 150 行)
  • chromadb-testing-patterns:单元/集成测试、模拟、测试夹具(约 120 行)
  • chromadb-security-patterns:PII 脱敏、访问控制、GDPR 合规(约 90 行)
  • chromadb-lifecycle-management:版本控制、迁移、归档、保留(约 100 行)

用法:Skill({ skill: "chromadb-error-handling" })


更新后的成功标准

满足以下条件时,ChromaDB 集成即为可用于生产环境:

核心功能(原有):

  • ✅ 使用有意义的名称创建集合
  • ✅ 高效摄取数据(批处理)
  • ✅ 语义搜索返回相关结果
  • ✅ 正确应用元数据筛选器
  • ✅ 优化性能

生产就绪(新增):

  • ✅ 错误处理:重试逻辑、验证、平稳降级
  • ✅ 测试:单元测试(使用模拟)、集成测试(使用真实 ChromaDB)
  • ✅ 安全:PII 已脱敏,机密信息已清理,访问控制已启用
  • ✅ 生命周期:版本控制策略、保留政策、归档

质量评分:65/70 → 85/100(包含所有增强项)


技能版本:2.0 更新日期:2025-11-14 增强项:错误处理、测试、安全、生命周期管理 质量评分:85/100(可用于生产环境) 依赖项:ChromaDB MCP (mcp__chroma__*) 子技能:4 个用于按需加载的模块化子技能

评论

支持 Markdown