SetuHQ transit search

• Physics checked by tests against closed forms and published values; the four below are drawn with the same functions.
• Coronagraphs: closing the Lyot stop from 150% to 80% cuts the starlight at 8 λ/D from 6.7×10⁻⁵ to 1.8×10⁻⁷ of the peak.
• Molecules, mode by mode. Ten molecules, each vibration linked to its infrared band, infrared-silent modes marked, the amplitude exaggeration stated per mode.
• Radial velocity: a star and planet orbiting their barycentre, solved with Kepler's equation; the star's motion is drawn exaggerated by a stated factor.

1. Ingests TESS SPOC light curves from MAST, CHEOPS light-curve FITS files, or CSV.
2. Cleans and detrends: spike clipping, a masked local-quadratic baseline, and binning.
3. Cuts blind-coded subjects: fixed 13-day segments with opaque IDs. The target name never reaches a reviewer.
4. Runs the panel: every reviewer marks and labels every subject independently. The task is the Planet Hunters task: mark the start and end of each dip, then label potential_planet or noise.
5. Aggregates marks by single-linkage clustering, scores each dip by the share of skill-weighted reviewers who marked it, and requires the panel to call the subject a potential planet.
6. Vets each consensus event: SNR, duration, edge effects, odd/even depth, secondary eclipse, and fit to the periodic ephemeris. Extended tests after the Kepler/SPOC Data Validation suite add flags beside them (dvtests.py, pixel.py): SPOC crowding (CROWDSAP below 0.8 is crowded), a weak-secondary search at every phase with the fitted duration (threshold from a white-noise simulation, scripts/calibrate_secondary.py), odd/even depths from per-epoch fits, per-epoch transit times (linear-ephemeris and odd/even timing tests), all on the light curve re-detrended in two passes with the fitted transits masked, as DV masks transits (the second pass also masks phase 0.5, and the first pass's strongest other dip only when that pass already finds it significant), and, from the SPOC target pixel file, a difference-image centroid in pixels and arcseconds with its significance, the centroid motion test, a core/halo ghost check, momentum-dump coincidence, and a table of catalogued neighbours with the eclipse depth each would need. By default these only add flags: pass/fail is decided as before, so runs and benchmarks stay comparable with earlier ones. [vetting] decide = true makes secondary_any_phase, odd_even_depth, odd_even_timing and centroid_offset disqualifying, and lets the all-phase secondary search replace the phase-0.5 check. The pixel tests download TPFs and are off unless [vetting] pixel = true.
7. Weighs false-positive scenarios for every periodic candidate (validate.py): SetuHQ's own reduced FPP (planet, eclipsing binary at P or 2P, eclipse on a resolved neighbour), built from the S2 measurements with stated priors. It is not TRICERATOPS and never says "validated". "Validated (statistically)" is reserved for TRICERATOPS with FPP < 0.015 and NFPP < 10⁻³ and a high-resolution imaging contrast curve; without a curve every candidate is "not validated: no contrast curve". SetuHQ never calls its results confirmed.
8. Writes a candidate catalogue (CSV and JSON), a Zooniverse-shaped classification export, a SQLite store, a provenance manifest, and a static report site.
9. Exports for follow-up (setuhq export RUN --out DIR, exofop.py): an ExoFOP CTOI-style table (CSV and JSON) of vetted candidates that pass the S2 and S3 gates and duplicate no TOI on the same TIC, plus predicted transit windows for the next 12 months with the ephemeris uncertainty cone, σ(Tₙ)² = σ_T0² + n²σ_P² (+ 2n cov), the same propagation match_toi uses (ephemeris.py). The CTOI field list is unverified against ExoFOP's current form (ExoFOP was unreachable); it lives in one place, exofop.CTOI_FIELDS. A person submits CTOIs under their own ExoFOP account; SetuHQ does not. Without the archive's TOI table the export refuses unless --allow-unchecked, and then every row says the check was not done.
10. Scales with setuhq batch CONFIG --targets FILE (batch.py): the discover flow over any target list, with light curves cached in cache/lightkurve and a state file that makes a killed or time-limited session resumable (done targets are skipped, failures retried, a changed config starts afresh).

The pages show display names; reviewer ids stay unchanged in every data file, CSV, JSON and URL. The default panel is 2 Sentinels and 3 Probes: Sentinel · Volunteer (claude-volunteer), Sentinel · Skeptic (claude-skeptic), and Probe · Sensitive, Probe · Typical and Probe · Cautious (volunteer-sensitive, volunteer-typical, volunteer-cautious).

Sentinels receive the same task as volunteers, the blind-coded plot, and no target name. They return strict JSON, which is validated and clipped to the subject's time range. A reviewer that is not available is skipped and listed in the manifest. Responses are cached by a hash of model, prompt and image, so re-runs are free and reproducible.

The two Sentinels use two personas: volunteer (careful, marks dips) and skeptic (rejects binary and systematic signatures unless they are clear). Sentinels can also run on OpenAI, Gemini or any OpenAI-compatible endpoint, labelled as such (see AI providers).

Real data, TESS SPOC sector 1 of pi Men (TIC 261136679), the three Probes plus the two Sentinels (Skeptic and Volunteer, on Claude via the CLI), as stored in runs/tess-pi-men-s1, run on the current code (the Sentinels' stored answers reused: the subject images and prompts are byte-identical):

• 7 consensus dips: 6 passed vetting, 1 of them flagged off_ephemeris, and 1 failed it (low_snr, off_ephemeris).
• 5 of them lie on the published pi Men c ephemeris. 5 of the 5 predicted epochs in the 26-day window are recovered, with a mean timing offset of −8 minutes against the NASA Exoplanet Archive value (docs/results/pi_men_crossmatch.json).
• Transit shape, limb-darkened fit (see docs/results/pi_men_validation.md). A joint fit of the 5 epochs, with the period free near the BLS period (6.2680 ± 0.0005 d; Huang: 6.26790 d), gives the like-for-like quantities first: Rp/R★ = 0.0179 ± 0.0017 (1.05 against Huang et al. 2018) and T14 = 3.07 ± 0.17 h (1.04). The mid-transit depth of the limb-darkened model is 292 ± 8 ppm (0.97 against Huang's published 300 ppm, which is defined by that paper, not by this model). The box measure it replaces gave 0.81 in depth and 0.64 in duration. Errors are conservative: the covariance plus the spread along the impact-parameter profile.
• The impact parameter needs the star. Free, 5 TESS transits give only b < 0.87 (1σ), so a/R★ and ρ★ are ranges (ρ★ 0.28 to 2.2 g/cm³ at 1σ, consistent with the TIC's 1.02). With the TIC density as a Gaussian prior (1.02 ± 0.20 g/cm³), the fit gives b = 0.64 ± 0.12, a/R★ = 12.85 (1σ 11.5 to 13.6) and T23/T14 = 0.95, against Huang's 0.59, 13.38 and 0.95. The fit is U-shaped, not V-shaped.
• The periodic signal's false-alarm probability is below 10⁻⁴ in both subjects. No one of 10,000 block-bootstrap trials reached the observed SNR, so 10⁻⁴ is a bound, not a value. The null maxima are committed in runs/tess-pi-men-s1/fap_null/; a Gumbel fit to them, stored as fap_tail, puts the tail below 10⁻⁶ (indicative only, see below).
• The passed dip at BTJD 1348.9 (0.87 h, SNR 9.5) does not fit the ephemeris and is a likely false positive. It is kept and flagged, not hidden.
• The dip at BTJD 1348.4 (SNR 6.7) failed vetting and stays in the stored candidates with its flags.

The stored run was re-run from the start on the current code on 2026-10-21 (scripts/rerun_pi_men.py). Every dip, vetting result, fit and FAP is the same as in the first run, made with pre-fix code (the run's manifest records the field-by-field comparison); the Sentinels were not asked again: their stored answers were reused because each subject image and prompt is byte-identical. Extended vetting, the reduced FPP and the export on the same run:

• Extended and pixel-level vetting (scripts/vet_pi_men_s2.py, real SPOC sector-1 files; see docs/results/pi_men_s2_vetting.md). On the joint fit of all five epochs (P = 6.26795 ± 0.00049 d, T14 = 3.09 ± 0.16 h), on the light curve re-detrended in two passes with the transits and phase 0.5 masked (window 1.55 d, 12 T14; the first pass's strongest dip is not significant, so it is not masked): CROWDSAP 0.9997, FLFRCSAP 0.980 (not crowded); the strongest dip at any phase away from the transit is 42 ± 7 ppm at phase 0.622 (5.6σ nominal), against a calibrated threshold of 6.9σ (5.16 times the scan's robust spread of 1.33), with 86% of it from one epoch; at phase 0.5, −12 ± 7 ppm. Odd/even depths from per-epoch fits agree (301 ± 14 against 311 ± 12 ppm, 0.5σ); the five mid-times fit a linear ephemeris (χ² = 1.9 for 3 dof, p = 0.59, O−C within 1.8 min) with no odd/even timing offset (0.5σ). No flag at the run's own setting, but the secondary is at the threshold and its flag depends on the setting: of 12 nearby run settings (detrend window 1 or 2 d × bins of 2, 6, 10, 15, 20 and 30 minutes, each through the full two-pass preparation) 3 flag secondary_any_phase: 15- and 30-minute bins with the 1.55 d vetting window, and 30-minute bins with 2 d; there the dip, at phase 0.64, reaches 1.05–1.2 times the threshold, and in the first two the first pass already finds it significant. The other 9 do not flag it: 0.68–0.93 of the threshold, or not periodic (with 20-minute bins and the 1.55 d window a dip at phase 0.74 reaches 1.05 times the threshold, 90% of it from one epoch). The previous code, which masked the strongest first-pass dip whether or not it was significant, raised the default setting's dip from 5.6σ to 7.0σ and flagged 8 of 15 settings, including every 6- and 15-minute bin with a window up to 1.5 d (the reviewer's measurement). These are not independent trials: every run window up to 12 T14 collapses to the same 1.55 d vetting window, so the 12 settings are 2 distinct vetting windows (1.55 and 2 d) × 6 bin sizes, all on the same data (the reviewer's 15-setting grid, windows 0.75–2 d × 3 bin sizes, is 6 distinct settings for the same reason).
• The reduced FPP of pi Men c, from the stored run (metrics.validation in candidates.json; the scheme's priors are a stated choice, see Honest limits). Per 13-day subject: 0.019 on the first (0.0186) and 0.015 on the second (0.0148). Over the pipeline's five detrending settings (the default, and windows ×0.75 and ×1.5 with masks of 1.0 and 2.0 T14) under two noise models the range is 0.0111–0.0216 (second subject 0.0123–0.0190), a log-odds spread of 0.29 dex (0.19), so the stability gate passes. The class is "undetermined": the range straddles the 0.015 threshold. NFPP is unknown, because the run has no neighbour table, and the label is "not validated: no contrast curve". pi Men c is a published planet with a measured mass; this shows the scheme's limits on real data. History: on 2026-10-20 the joint fit of all five epochs over the whole sector (scripts/vet_pi_men_s2.py, docs/results/pi_men_s2_vetting.json) gave 0.0079–0.093 across the same settings and was refused as unstable (odds spread 12.9×; 10× allowed); on a wider grid its secondary scan was white on 6 of 15 settings, and its largest brightening ranged from 2.6 to 6.2σ.
• Ephemeris cone. Exporting the stored run (setuhq export runs/tess-pi-men-s1 --allow-unchecked; the archive's TOI table timed out, HTTP 504) gives one signal, P = 6.2681 ± 0.0012 d from the first subject's joint fit, and its predicted transit in October 2026 (epoch 479) carries σ = 13.8 h, a 3σ window of 86 h. A one-subject ephemeris is stale within a few years, as ATLAS G5 warns. With the TOI table, this signal is TOI 144.01 and is excluded as a duplicate (tested with the published ephemeris).
• The difference-image centroid of pi Men is not usable, and SetuHQ says so. The PSF fit puts the difference image 2.05 px (40.1″) from the catalogue position, mostly in the row coordinate (row −1.91 px, column −0.74 px), which is the direction charge bleeds along a CCD column. SPOC's own DV report for TOI 144.01 finds the same size and direction: 40.5″, row −2.07 px, column 0.02 px, with a DV quality metric of 0.58, below DV's 0.7 reliability cut. pi Men (Tmag 5.1) is saturated: five pixels of one column sit on a plateau within 5% of the peak, a bleed trail. A transit shortens the trail instead of dimming a PSF, so the "offset" is the trail's end. The fit says so too: the difference-image PSF collapses to its 0.3 px width bound, 3.4 times narrower than the out-of-transit one. SetuHQ flags saturated and centroid_unreliable and raises no centroid_offset. The per-cadence centroid motion test sees no shift (0.1σ), but on a saturated star that carries no information either: the flux-weighted centroid of a clipped bleed trail need not move when the star dims. Of 56 TIC stars within 150″, all but the target would need an eclipse deeper than 100%; the nearest, TIC 261139071 (Tmag 14.0 ± 0.75, 27″), needs 111%, which is marginal: within 3σ of its Tmag uncertainty.

What the Sentinels did in this run, read from the stored classifications:

• On the first subject (BTJD 1325.3 to 1338.3), Sentinel · Volunteer marked the dips at 1325.50, 1331.77 and 1338.05 and labelled the subject potential_planet (confidence 0.8); Sentinel · Skeptic marked the dips at 1325.50, 1331.77 and 1338.05 and labelled the subject potential_planet (confidence 0.7).
• On the second subject (BTJD 1338.3 to 1351.3), Sentinel · Volunteer marked the dips at 1344.27 and 1350.57 and labelled the subject potential_planet (confidence 0.4); Sentinel · Skeptic marked the dips at 1344.27 and 1350.57 and labelled the subject potential_planet (confidence 0.7). Neither marked the dip at 1348.38 (rejected by vetting) or the dip at 1348.94 (passed but off the ephemeris).
• In the consensus each Sentinel carries weight 0.5, against 0.4 to 0.6 for the Probes: the Sentinels are voices in a calibrated panel, not the whole panel.

Blind benchmark on real TOIs (re-run on the current code on 2026-10-21; measured on a stated benchmark, not discoveries; see docs/results/benchmark_real_tois_clipfix.md, and for the first run, on the pre-fix code, docs/results/benchmark_real_tois.md). 662 TOIs from the NASA Exoplanet Archive TOI table (fetched 2026-10-19, seed 20261019): 300 CP/KP, 300 FP and 62 FA, one per star, split into halves by star. Each star's first SPOC 2-minute sector goes through the Probes panel, along with a flux-inverted and a block-scrambled copy as false-alarm controls. Held-out half, with Wilson 95% intervals:

• Completeness: 60% (90 of 150 CP/KP on searchable stars; 52 to 67%). It is 84% above 8 R⊕, 45% at 4 to 8 R⊕, 23% at 2 to 4 R⊕ and 10% below 2 R⊕. In the bins the published references use: all radii: 60% (90/150; 52 to 67%) against Kepler Robovetter DR25's > 85% (vetting only, on detected TCEs, P < 100 d); 3 to 4 R⊕: 44% (4/9; 19 to 73%) against Planet Hunters TESS's 51% (a citizen-science search); above 4 R⊕: 79% (81/102; 71 to 86%) against Planet Hunters TESS's 85% (a citizen-science search). SetuHQ is below the Kepler Robovetter's completeness (the interval excludes it); not distinguishable from Planet Hunters TESS's 51% at 3 to 4 R⊕ (44%, 19%–73% over 9 stars; a 7-point lower estimate the interval cannot resolve); not distinguishable from Planet Hunters TESS's 85% above 4 R⊕ (79%, 71%–86% over 102 stars; a 6-point lower estimate the interval cannot resolve). Planet Hunters TESS's rates carry their own uncertainty and come from sectors 1 to 26.

• FP TOIs: the panel found the dip of 77% and rejected or flagged 68% of those as binary-like. This applies to FP stars with 2-minute data only (124 of 158). FP TOIs are far more often FFI-only, so this is not reliability on the whole FP list.

• The live feed's "new" signals are consistent with noise. On the controls, the feed's own admission rule passes 58 (inverted) and 61 (scrambled) false signals per 1,000 star-sectors. That is about 12 per 100 targets reviewed, against 15.8 per 100 on the live feed under that rule (3 over 19 targets).

• The fix, pre-registered and measured on the held-out half. The feed now requires the signal in both reviewed sectors (periods within 0.5%) and a FAP below 0.001 over 2,000 trials. The rule was chosen on the tuning half and committed (8909cda) before its held-out evaluation; that evaluation was re-run unchanged on the current code (docs/results/feed_rule_validation_clipfix.json). Held-out results, measured with the discovery config, old rule to new rule:

  • noise from the controls: 27.8 (95% CI 23.1–33.1) to 6.1 (4.0–9.4) false signals per 100 targets (first evaluation, on the pre-fix code: 23.0 to 6.1);
  • feed-level completeness: 39% to 17% (the same as before);
  • FP TOIs kept out: 70% to 94%.

  Noise drops about 4.5-fold, but 6 per 100 targets is still above the 1.6 new signals per 100 that real light curves now give (0.3 on the pre-fix code). The feed is still dominated by false alarms, and an admission is a lead for human or follow-up review, not a candidate of any confirmed status. The live feed's 19 targets, re-run on the current code on their previous sectors, admit 0 new signals and 2 known TOIs (24 entries), as on the pre-fix code.

• The Sentinels on the stated subsample of 40 held-out stars were not asked again for the re-run. Their stored answers carry over to the 37 stars whose subject images and prompts are byte-identical on the current code; the other 3 (deep transits the fix now keeps) are left out. On those 37, each persona marked 47% (9/19) of the CP/KP dips and 22% (4/18) of the FP dips as potential planets. They agree on 35 of 37 stars (φ = 0.88): one model, so one correlated voter.

Injection-recovery benchmark for the Probes (re-run on the current code): docs/results/injection_recovery_simulated_panel.md. It and the threshold sweep were first made on the code of 2026-10-16, before the injected transits became limb-darkened (2026-10-19), so their changes on the re-run are not the two fixes' alone. The sweep's held-out score fell from 0.875 to 0.833 because 2 of its 48 injections at 1000 ppm are no longer recovered (6 to 5 of 12 at 2 h, 12 to 11 at 4 h); that change appears with the limb-darkened injections and is already there on the pre-fix code of 5a03087. The Sentinels are not in the benchmark. Each injected subject would need its own CLI call, so the Sentinels are calibrated separately on a small sample (10 injected and 10 empty subjects: weight 0.5 each, against 0.4 to 0.6 for the Probes).

The search, setuhq discover, runs one batch of TESS targets. It rotates through the TOI host list so coverage advances each day, reviews the two latest sectors of each target, and keeps every new dip in a deduplicated catalogue. A dip is known_toi when it matches a published TOI period, and new_candidate otherwise.

• .github/workflows/discover.yml runs one batch when started by hand (its schedule is off), commits the catalogue as ashfordeOU, and deploys the site. Claude's API reviewer runs when the ANTHROPIC_API_KEY secret is set; without it the batch runs on the Probes only.
• scripts/improve.py sweeps panel thresholds on injection benchmarks. It promotes a setting only if it wins on a held-out seed. Every trial is in the threshold sweep, docs/threshold_sweep.md, including losers. This synthetic sweep is not the real-TOI leaderboard (see Leaderboard and methods paper).

Measured false-alarm rate. Under the earlier rule (FAP < 0.01 over 200 trials, one sector), light curves with every signal destroyed passed about 12 noise signals per 100 targets. The live feed showed 15.8 new_candidate signals per 100 targets, so its "new" signals were consistent with noise (see the benchmark).

The feed now requires the signal in both reviewed sectors and a FAP below 0.001 over 2,000 trials. That rule was chosen on the tuning half, pre-registered, and evaluated on the held-out half, first on the pre-fix code and again, unchanged, on the current code (validation). It lowers noise about 4.5-fold, from 27.8 (95% CI 23.1–33.1) to 6.1 (4.0–9.4) per 100 targets, and more than halves completeness (feed level, 39% to 17%). Noise still outweighs the real new signals (about 6 per 100 targets against 1.6), so the feed's admissions are leads for human or follow-up review.

Re-run under the new rule on the current code (on the sectors of its previous run, with the TOI table cached on 2026-10-19), the 19 live targets admit 0 new signals and the same 24 entries on 2 known TOIs. The catalogue built under the old rule is kept in docs/discoveries/catalog_v1_fap200.json.

What this can and cannot claim. Every entry is a candidate. A detection here means the pipeline saw a vetted dip that matches no published ephemeris. It does not mean a planet has been found. Confirmation needs radial velocities, ground-based photometry, or the community's TESS follow-up. The pixel-level centroid test is new, off by default and unusable on saturated stars, so blended eclipsing binaries can still survive vetting. The claim this project can defend is a reproducible, calibrated search with open provenance. It is not a claim of new planets or of beating published pipelines.

• The real-TOI leaderboard (docs/leaderboard/, and /evidence/leaderboard/ on the site) takes a method's detected events per star of the held-out half of the frozen 662-TOI benchmark, and scores them itself: completeness, FP rejection, control false alarms and reliability, with Wilson intervals. It ranks by Youden's J on the benchmark's searchable set with one predicate (a clean candidate) on both sides, so calling every TOI a planet scores zero, flagged or not, and ranks only entries measured on both controls; the Sentinel subsample (37 of the stated 40 stars, those the re-run left byte-identical) is listed, never ranked. A metadata-only floor that never reads a light curve is ranked too, and the benchmark cannot show that the Probes beat it: a paired comparison over the same stars puts the floor ahead by a ΔJ whose interval includes zero (on the re-run, J = 0.35 for the Probes against 0.39 for the floor, paired ΔJ +0.04 with a 95% interval of −0.10 to +0.17; docs/leaderboard/leaderboard.md). setuhq leaderboard my.json scores a submission and refuses one that states anything but the hash of the tuning-half-only inputs or carries a field outside the whitelist. It is seeded with SetuHQ's own configurations, built from the benchmark's stored outputs (scripts/leaderboard_seed.py).
• The methods paper draft (docs/paper/paper.md, AAS/AJ style, Markdown) reports the benchmark with its negative results. Every number in it comes from the committed JSON through scripts/paper_numbers.py; results that later work will change are marked pending.

The planet search used to throw away every dip that was not a planet candidate. The sweep keeps them. setuhq sweep works through a TESS sector's SPOC 2-minute targets and, for each star-sector, keeps every consensus dip of the panel (whatever the panel's label), every brightening and the star's variability, and classifies each by first-match rules in src/setuhq/variables.py:

Every entry keeps its confidence and the cues that decided it, its marks, its weights and the Sentinels' own words, the light curve's sha256, the code fingerprint and config hash, and the name of whoever searched it. Nothing is ever called confirmed. Every class is set against its own null on the same stars: the dip classes against the flux-inverted copy (a real dip becomes a bump, though a real flare or pulsation maximum becomes a dip, so this null is not empty of real signals), flares against downward events with the real dips masked, and variability against a red-noise copy of the same star, an AR(1) process fitted to its own bins (a periodogram is unchanged by flipping the flux, so the inverted copy is no null for it, and shuffling removes the red noise that makes false peaks). The variability rule itself asks for a coherent period: the same best peak, within 10%, in each half of the light curve. Each kind is compared with the same kind, and the star-sector is the unit: dips cluster on a few stars, so the excess's 95% interval comes from a paired bootstrap over star-sectors, beside McNemar's exact test on the star-sectors with events on one side only. A class exceeds its null only when both agree; otherwise the site calls it "not distinguishable from noise in this slice".

setuhq sweep configs/sweep.toml --sector 1 --limit 200 --workers 3   # resumable; cached; polite to MAST
setuhq bundle my-sweep/sector_0005.json.gz --contributor "Your Name" --out bundle/
setuhq ingest bundle/ --attested-by <pr-author>   # sealed, not signed: re-searches every finding + max(3, 5%) by a nonce

The site reads docs/sweep/catalogue.json: the home page's counter, /catalogue/everything-that-dims/ (Explore › Everything that dims, with filters, per-TIC pages and CSV/JSON export), the sweep layer on the Sky map, Mission Control's telemetry and the search. Anyone can add a sector with their own Claude subscription or API key (kept in their environment, never in a file) and be credited: see CONTRIBUTING.md.

First real slice (measured, Probes only). TESS sector 1: the first 200 of its 15,898 SPOC 2-minute targets in sweep order, all 200 searched (no download failures) by the search code of 0e87409 and classified by the rules of this commit (each record's search_code and classify_code). The panel reviewed 936 consensus dips (2,662 marks; 60 mark clusters fell below consensus), and the catalogue holds 1,011 entries. Each comparison against its null on the same 200 star-sectors, kind for kind; the interval is the paired bootstrap over star-sectors, and "real / null" star-sectors count those with at least one event:

No class exceeds its null in this slice. Two earlier versions of this table claimed one did, and both claims were wrong:

• The eclipsing-binary claim counted dips as independent events. Two stars supply about a third of them, and star by star the excess is not distinguishable from zero.
• The variable-star claim used a periodogram rule that red noise also passes, against a shuffled null that had lost the red noise. Of the 36 peaks that pass the FAP and amplitude rule, 8 have a coherent period, the same best peak in each half of the light curve (1.0 to 2.7 d). 20 are longer than a half can test, 5 sit at the edge of the period grid, and 3 wander between halves; those 28 are now unclassified, with the reason. The null is now a red-noise copy of each star, through the same rule.

Six eclipsing-binary entries of the variability kind have no null of their kind and are not compared. The 3 entries that pass the planet-candidate rule (none matching a known TOI) are about what noise alone gives, so the site does not present them as candidates. Of the 11 flares, 8 rise faster than they decay and 2 the other way (1 equal). The slice finds no planet, and every class needs many more stars before an excess could show. FAP trials are off in the sweep (50 trials cost about two minutes a star), and the stellar parameters come from each file's header. Records: docs/sweep/sector_0001.json.gz and docs/sweep/catalogue.json.

Provenance of this slice. The records were searched again at 0e87409, then reclassified once on this commit, from the cached files and stored runs, for the coherence rule and the red-noise null. Each record's reclassifications lists that step and its rule changes. The earlier records are in the history of docs/sweep/: searched at b8647ad, reclassified at 3c5862e for two bugs (a flare whose equal rise and decay read as a fast rise, and two brightenings within 0.01 d that shared an id), and at 1eb9a6f for the short-period density rule and the per-class nulls.

One fix after the re-search: the stored run directories still held the earlier search's classifications, and the first count of marks included them (5,358). The consensus, entries and nulls used only this search's. n_marks was recounted from this search's own classifications (each record that changed says so in n_marks_note). Eight star-sectors searched again in clean directories, including every one with a planet candidate, reproduce their records exactly. The sweep now clears a star's run directory before searching it. A resume reuses a stored record only when its config hash, search code and classification code all match. The catalogue counts one search per star-sector, the latest, and keeps earlier searches as provenance only.

Data size. Records are one gzip file per sector (sector_NNNN.json.gz, 0.47 MB for these 200 stars; the catalogue the site fetches is 1.7 MB), so a full sector would be about 37 MB of records. Past a few sectors, the records would live outside the repository as release assets, with a committed manifest of their sha256 that the site build checks. The catalogue would then hold only the entries and coverage, not the per-star events and nulls. That second step is not built yet.

• The Sentinels are calibrated on a small sample only. The full benchmark needs many CLI or API calls. See the calibration file for the sample size used.
• Synthetic results are not sky results. The benchmark uses white plus red noise. It measures the pipeline's logic, not real completeness.
• The CHEOPS column mapping is unverified. The loader accepts several candidate names and records the ones used. Check them against a real SCI_COR_Lightcurve file.
• Transit fits are light-curve fits. Every consensus dip gets a limb-darkened fit (Claret 2017 TESS coefficients from the TIC's Teff and log g) with depth, T14, impact parameter and mid-time with errors, a ρ★ consistency check and a V-shape test. On a periodic signal the period is fitted too, near the BLS period, so an approximate search period does not smear the epochs. With a few TESS transits the impact parameter, and so a/R★ and ρ★, is poorly constrained: b is then printed as an upper limit and ρ★ as a range, and a second fit with the TIC density as a prior is stored beside the free one. The density band (0.3 to 3 times the catalogue value, at 3σ) is illustrative, not a published threshold.
• False-alarm probabilities are bounded by the trials. A FAP is (k + 1)/(N + 1) over N block-bootstrap trials, and is never printed below 1/(N + 1). The stored pi Men run uses 10⁴ trials, the search batches 2,000 (stopped once 2 trials reach the signal's SNR), and the library default 20. The feed's FAP < 0.001 rule passes at k ≤ 1 of 2,000, which by itself still admits about N/1,000 noise signals across N targets. The two-sector part of the rule is what removes most of the rest. A Gumbel fit to the null maxima is stored with each candidate (fap_tail, with fap_exceed and fap_method). It is indicative, extrapolated beyond the trials and conservative (BLS maxima are lighter-tailed than Gumbel), and it is never given below 10⁻⁶.
• Pixel-level vetting is new and limited. The difference-image centroid uses a circular Gaussian in place of the SPOC pixel response function, and its flag is tested against the catalogue position with a systematic floor equal to the out-of-transit fit's own offset from that position, which makes it insensitive below a few tenths of a pixel. On synthetic TPFs whose PSF is the same Gaussian (the ideal case) it localises a neighbour's eclipse 1.5 to 3 px away to within 0.002 px (15 cases), and a planet on the target stays below 0.2σ (10 cases). With an elongated PSF (8 geometries 1.5–3 px away, 2 seeds each) the model error dominates: the neighbour is placed up to 1.10 px off at 1.6:1 (median 0.36), 2.15 px at 2:1 (median 0.72) and 2.30 px at 2.5:1 (median 0.95). The binary is still flagged in every case at 1.6:1 and 2:1; at 2.5:1, 2 of the 8 geometries are marked unreliable instead (both seeds), and the rest flagged. The planet on the target is never flagged (3 seeds per ratio). It has not yet been benchmarked on real false positives (ATLAS S2 asks for Kepler koi_fpflag_co KOIs and TESS FP TOIs). Saturated stars, like pi Men, get no usable centroid. The neighbour table's flux shares use the same Gaussian; an exclusion within 3σ of the star's Tmag uncertainty is reported as marginal.
• The secondary search is calibrated on white noise. Its threshold is the white-noise 99.9% quantile of (largest dip)/(robust spread) for the scan's number of positions, times max(1, the scan's robust spread); on 3000 clean simulated planets it flagged none (95% CI 0–0.12%; the 2 once quoted predate the max(1, ·) floor), against 87 (2.9%) for the rule it replaced and 681 (23%) for a plain 3 × robust spread (scripts/calibrate_secondary.py, docs/results/secondary_calibration.json). On red-ish noise (robust spread 1.7) it flagged 2 in 1000 with seeds 700000–700999 (the reviewer's run: 0 in 1000). Through the full two-pass preparation, on 600 clean planets at 10-minute cadence with 270 ppm white plus 100 ppm red noise (the reviewer's seeds), 2 were flagged, against 1 with one pass; the second pass lifts the strongest dip by +0.03σ at the median. The previous code, which masked the strongest first-pass dip every time, protected the largest noise dip from the trend: 8 of 600 flagged and +0.57σ (the "1 of 300" quoted for it was wrong). The second pass now masks phase 0.5 always and that dip only when the first pass already finds it significant. The detrending used to cost sensitivity, not the threshold: with only the transits masked, the trend absorbed about half of a secondary (134 of ~180 ppm kept), and a 1000/200 ppm binary at 600 ppm per 2-minute point was flagged 10 times in 20; with phase 0.5 masked, 161 ppm is kept and 19 of 20 are flagged (1500/300 ppm at 1000 ppm noise: 5 → 18 of 20). An eccentric secondary below the threshold in the first pass is not protected: at phase 0.62, 1000/200 ppm is flagged 13 times in 20 (12 with one pass; 18 when every first-pass dip was masked). Not built, future work: recovering those with a lower first-pass threshold for the mask together with a second-pass threshold calibrated by simulating that exact two-pass procedure. Long transits used to break the masked detrending (a 1-day quadratic across a 3 T14 mask extrapolated by 10⁴–10⁶ ppm at 7–10 hours); the trend is now interpolated across the mask and the vetting window is at least 12 T14, so in vetting detrend_window_days has no effect below that (1.55 d for pi Men c).
• The reduced FPP is checked for stability. It is computed on the default detrending and four variants (window ×0.75 and ×1.5, masks 1 and 2 T14; a variant whose masks leave less than half the points beside a masked span is left out) under two noise models (the scan's robust spread and its std without the strongest dip, never below the nominal σ/√n). Above 0.5 on every one, the class is "likely false positive"; otherwise the odds may vary by at most a factor of 10 or no FPP is given, and a range across 0.015 or 0.5 is "undetermined". Its no-secondary likelihood is the simulated null distribution (the earlier max-of-N model charged a clean planet for an unusually small strongest dip: FPP up to 0.97), with an exponential tail fitted above its 99th percentile (fewer than 40 of 40,000 simulated scans lie beyond z = 5.1). Measured through the vetting path: clean planets with white noise 0 of 150 refused (10-minute cadence; most are "undetermined", their range straddling 0.015), and 0 of 36 with 6–12 hour transits (the 10-hour dozen re-run on the final code), against 7% and about 1 in 6 in the review; with red noise refusals remain, about 3% (the reviewer measured 5 of 150 on 2d268cb); simulated binaries "likely false positive" 19 of 20 (1000/200 ppm), 17 of 20 (1500/300 ppm) and 13 of 20 (1000/200 ppm at phase 0.62), where 15–16 of 20 used to be refused as unstable.
• Validation is not done. TRICERATOPS 1.1.0 installs with pip (extra validate), but its target() queries MAST, Gaia and TESSCut on construction, so it cannot run offline, and MAST's API was failing when this was built: the adapter (validate.run_triceratops) is tested against a stub only and has not been run on a real target. The reduced FPP that runs instead is a documented approximation: its scenario priors are a choice (a 25% false-positive total after Guerrero et al. 2021), it ignores unresolved companions and background stars, and without a reliable centroid it cannot tell a target from a neighbour that could host the depth. No contrast curves are ingested yet.
• Batch mode is tested offline only. Resumability, retries and caching are tested with synthetic light curves; no real batch was run because MAST's search API was failing (2026-10-19), so no real throughput is reported. The CTOI export has not been compared with ExoFOP's live form.
• Two fixes change default results (pass/fail included). (1) Until commit 5a03087, preprocess.clean removed every point more than 10 times the point-to-point scatter below a 0.5-day running median, so an eclipse or transit deeper than that (a 3% binary, a 1% hot Jupiter on a quiet star, 5000 ppm at 300 ppm noise) vanished before the panel saw it. A low outlier is now removed only when it is not part of a dip: a run of at least max(3, 20 minutes / cadence) cadences more than 3σ below the median (hysteresis, so a 10–12σ eclipse whose points straddle the clip keeps them all). The upward clip is unchanged. (2) With deep dips now reaching vetting, vet.py's odd_even_mismatch failed 6 of 10 clean 1% hot Jupiters: box depths of a deep transit scatter far beyond white noise. Its errors are now inflated by the per-epoch depths' scatter about their parity means and carry a 2% systematic floor. On 2026-10-21 every result made before these fixes that can be re-run without calling a Sentinel was re-run on the current code: the stored pi Men run (the same dips, vetting, fit and FAP; pi Men c, at 300 ppm about 1.5 times the 2-minute scatter, is affected by neither fix), the live discovery catalogue, the injection-recovery benchmark, the threshold sweep, the real-TOI benchmark (and so the real-TOI leaderboard and the methods paper's numbers) and the feed-rule evaluation. The first runs are kept as the historical record, each with the pre-fix note in its own file, naming its re-run: docs/results/benchmark_real_tois.{md,json}, feed_rule_validation.json, feed_rule_live_rerun.json and docs/discoveries/catalog_v1_fap200.json. The calibration record (reviewer_weights_full_panel.calibration.json) is not re-run: its Sentinel rates cannot be measured again without calling the Sentinels. Its Probe rates (TPR 0.6, 0.6 and 0.4) were measured on the code of 2026-10-16; since the injected transits became limb-darkened (2026-10-19) the same seed gives 0.9, 0.8 and 0.6. The weights stay frozen, as a parameter of the benchmark protocol: with three Probes, a minimum score of 0.5 and a minimum planet fraction of 0.5, either weight set accepts exactly the dips at least two of the three Probes mark, so every consensus decision is the same (tests/test_calibration_equivalence.py). A consequence of (2): a simulated eccentric binary (0.8% primary, 0.4% secondary at phase 0.37) now passes default vetting, because its earlier odd_even_mismatch failures were spurious; each of its 8 events on the primary ephemeris is caught by secondary_any_phase (a flag, or a failure with [vetting] decide = true; its secondary eclipses, found as events of their own, are flagged off_ephemeris 5 times and not at all once), and each of those 8 is classed "likely false positive" by the reduced FPP (above 0.5 on every detrending setting; before the class-aware gate such binaries mostly read "FPP not computed: unstable", not "about 1"). The trade-off of (1): a real event deeper than 10σ that lasts under 20 minutes (an 18-minute eclipse at 2-minute cadence) is still removed as a spike.
• Edge events are the weak spot for both the statistics and the reviewers.
• The coherent-variable rule is weak in red noise and blind to long periods. It finds a 300 ppm sinusoid in strong red noise (AR(1), 600 ppm, 1-day correlation) only 3 times in 20 (and 20 in 20 in white noise; scripts/variability_rule_check.py, docs/sweep/variability_rule_check.json), and it cannot test periods longer than about 3.8 d in one sector, since each half of the light curve must show it (20 of the sector-1 slice's peaks are "coherence not tested" and stay unclassified).
• Feed admissions are leads, not candidates of any confirmed status, and the feed is still dominated by false alarms. Nothing on the search feed is a confirmed planet. Under the pre-registered two-sector rule, noise alone still gives about 6 false signals per 100 targets (held-out, 95% CI 4.0–9.4), above the 1.6 per 100 that real light curves give. The rule also costs completeness (feed level, held-out: 39% to 17%).

• Real-TOI completeness is 60% on searchable stars (held-out; 52 to 67%), below the Kepler Robovetter's completeness, and only 10% for planets below 2 R⊕. The FP measures cover only FP stars with 2-minute data, a population skewed away from the full FP list.

• The clipping bug, fixed, and the benchmark re-run. The code of the benchmark's first run removed every point more than 10× the point-to-point scatter below the running median, with no run-length test, so transits deeper than about 10× the per-cadence scatter lost their in-transit points. In its held-out half 27 CP/KP and 11 FP stars were exposed (depth more than 8× the scatter), and 3 CP/KP misses were explained by it. Re-run on the fixed code with the protocol unchanged, 23 of the 27 exposed CP/KP stars are recovered (21 before), all 3 of those misses among them, and held-out completeness is 60% (90/150) against 57% (86/150): 5 stars gained and 1 lost, +3 points, not shown to exceed resampling noise (paired bootstrap 95% CI 0 to +6 points; McNemar p = 0.22). Most of the gains are deep planets the old odd/even test rejected. The same fix explains most of the other moves: on the inverted control, 16 of the 24 dips that pass vetting now and did not before were rejected by the old odd/even test and 5 are deep inverted features the old cleaning deleted (3 others; 3 no longer pass), which takes it from 573 to 641 vetted dips per 1,000 star-sectors; and 4 of the 5 FP TOIs whose dip now passes were rejected by the old odd/even test (one no longer passes), which takes FP dips passing vetting from 31% to 34%. The one loss, TOI 1905.01, is a stale published ephemeris: its sector holds two full transits, about 25 ppt and 2.5 h, but the ephemeris predicts them 7.2 h later (7σ of its propagated error), so even a perfect detection would not match. The first run's "recovery" was a residual dip left at the predicted time after the clipping had cut the transit out; the re-run's dip is the real transit, measured on a window the Probes marked over the ingress only (0.79 h, 2.9 ppt; the limb-darkened fit gives 25.3 ppt and 2.75 h) and on an alias period (3P/2). An audit finds it is the only held-out CP/KP miss of this kind (no dip on the published ephemeris, but a vetted dip of comparable depth within 12 h of a predicted epoch and at least 3σ of the propagated ephemeris error off it); the frozen matching rule is unchanged, so it stays a miss (the re-run's report, "Known limits"). The first run is kept, with its pre-fix note, in docs/results/benchmark_real_tois.md.

src/setuhq/
  lightcurve.py   container with provenance      loaders.py   TESS (MAST), CHEOPS FITS, CSV
  preprocess.py   clean, detrend, bin            subjects.py  blind-coded 13-day segments
  search.py       BLS and single-event search    vet.py       automatic vetting
  dvtests.py      DV-style light-curve tests      pixel.py     TPF difference images, centroids, neighbours
  validate.py     reduced FPP, TRICERATOPS adapter, validation labels
  ephemeris.py    propagation, uncertainty cone   exofop.py    CTOI-style export, predicted windows
  batch.py        resumable multi-target runs     leaderboard.py the real-TOI leaderboard
  injection.py    synthetic transits              calibrate.py skill-based reviewer weights
  aggregate.py    clustering and consensus       pipeline.py  end-to-end run
  benchmark.py    injection recovery             report.py    static HTML site
  store.py        SQLite and CSV export          config.py    TOML configs
  variables.py    everything-that-dims rules     starsearch.py one star-sector of the sweep
  sweep.py        resumable sector sweep         contrib.py   contribution bundles and ingest
  everything.py   #/variables, counter, Sky layer
  tools.py        the MCP tools' functions       mcp_server.py setuhq-mcp
  agreement.py    Sentinel agreement, effective number of voters
  reviewers/      simulated; Sentinels: sentinel.py (shared), claude (API), claude_code (CLI),
                  openai_compat (OpenAI and compatible endpoints), gemini
configs/          run configs                    scripts/     known-planet validation
docs/             research notes and results     site/        generated report
tests/            offline tests, plus network- and slow-marked tests
tools/            README figures (gen_readme_assets.py) and app screenshots (capture_site_shots.mjs)
skills/, commands/, .claude-plugin/   the Claude Code plugin    examples/  mixed-provider panel, example agent