mcpField
L
list-dedup

list-dedup

github.com/cold-iqData & databases347 installsUpdated 2w ago
Repository

About list-dedup

A data-processing skill for cleaning and deduplicating contact and company lists before ColdIQ enrichment. It uses normalized LinkedIn URLs as the primary match key and makes no API calls.

Key features

  1. 01Normalize LinkedIn URLs
  2. 02Deduplicate people by URL or email
  3. 03Deduplicate companies by URL or name
  4. 04Combine sources into a standard schema
  5. 05Keep highest-scored duplicate rows
  6. 06Cap contacts per company

Use cases

  1. 01Clean lists before enrichment
  2. 02Combine contacts from several sources
  3. 03Retain the best-scored duplicate record
  4. 04Limit contacts assigned to each company

Install

Copy the command and run it in a terminal.

$npx skills add https://github.com/cold-iq/coldiq-marketplace-skills --skill list-dedup

After installing, type/list-dedupin Claude Code to confirm it loaded.

Bundled filesresources/credit-optimization.md

Details

From the project README

List Dedup

A pure data-processing utility: dedup and clean BEFORE any ColdIQ enrichment. This is the single biggest credit saver — every duplicate you remove is a paid find/enrich call you don't make. See resources/credit-optimization.md. This skill makes no API calls.

When to dedup

  • Before enrichment (never pay to enrich the same person twice)
  • After combining multiple sources (database search + enrichment + ad scrape)
  • After scoring (collapse duplicates, keep the highest-scored row)

Primary key: LinkedIn URL (normalize first)

import re
def normalize_linkedin(url):
    if not url: return ""
    return re.sub(r'\?.*$', '', url).rstrip('/').lower().replace('https://www.', 'https://')

def dedup_people(rows):
    seen, out = set(), []
    for r in rows:
        key = normalize_linkedin(r.get('linkedin_url', '')) or (r.get('email','').lower())
        if key and key not in seen:
            seen.add(key); out.append(r)
    return out

Secondary key: company name / company LinkedIn URL

Normalize company names (strip Inc/LLC/Ltd suffixes, lowercase) or, better, dedup on the company LinkedIn URL when present.

Multi-source combine pattern

  1. Normalize columns — map each source's columns to one standard schema (full_name, title, linkedin_url, email, company_name, company_domain, company_linkedin, source, score).
  2. Load all source CSVs.
  3. Dedup people by normalized LinkedIn URL (fallback email); companies by company LinkedIn / name.
  4. Sort & export — Tier 1 first, then by persona, then company name.

Per-company contact caps

Cap 5–10 contacts per company so you don't blast one account. Rank by title priority (C-level/VP/Director > Manager > IC), keep the top N per company_domain.

Cross-reference against TAM

Left-join the contact list against the scored TAM on company_domain to attach Tier, and drop contacts at DQ'd companies before enrichment.

Checklist

  • Normalize LinkedIn URLs before comparing
  • Dedup people by LinkedIn URL (fallback email)
  • Dedup companies by LinkedIn URL / normalized name
  • Map all sources to one schema before combining
  • Keep highest-scored row on duplicate
  • Apply per-company contact cap (5–10)
  • Cross-reference against TAM, drop DQ companies
  • THEN enrich (see contact-enrichment)

Comments

Markdown supported