Skip to content

Catalog import: fix the types, names, positions, magnitudes and sizes the 2026-10 audit found #670

Closed

Description

@brickbots

The Paper Chart atlas joins every symbol it draws to pifinder_objects.db, and an audit of the catalog against OpenNGC and Hipparcos found 1,374 disagreements. Each was decided against SIMBAD, NED, Corwin, Steinicke, Bica et al. 2008, RC3, Harris, WDS, the BSC and IAU WGSN. About 414 are PiFinder's to fix: 326 values the catalog has wrong, 29 that both sources have wrong, and 59 names the catalog lacks. The per-object evidence (reports, verdicts and the findings.py script) is in the Paper Chart repository's docs/software/catalog-audit/, which is private for now. Each PR quotes the verdicts it acts on.

The fixes follow ADR 0046 and docs/ax/catalog.md §9, both added by PR 1: every wrong value is fixed in exactly one place.

  • Loader: a loader misreads its source.
  • Supplementary source: a published source is wrong, and one reference covers many objects by a stable key.
  • Curated source: a list PiFinder compiles itself is wrong or incomplete, so it's edited directly.
  • Correction: a published source is wrong for one object. A cited, checked TOML entry under astro_data/corrections/ fixes it, applied last.

Pull requests, in order

PR contents DB rebuilt?
A #671 (mrosseel's, extracted from #379 with his agreement) parse_designation and its prefix allowlist; names kept in source order (TAAS, WDS); "Cr 42" links M 45; Sharpless J2000 Dec; M 45's values on the shared Collinder object; two Abell cross-identification fixes. As a side effect, Arp's zero-padded "NGC 0070" akas join for the first time (33 duplicate sky objects gone; Arp 127, 276 and 282 move from their IC galaxy to the NGC galaxy the source lists first). Lyngå's "Collinder nnn" names were expected to join too, but the Lyngå loader stores them as common names, so they never reach the matcher (found while building PR 2). Not the Perek-Kohoutek catalog, which stays in #379. no
1 #672 The correction mechanism: TOML loading, the final pass, was checks, the hip hook. ADR 0046, catalog.md §9, and the catalog glossary. astro_data/corrections/README.md. tomli added as a dev dependency. Unit tests on a fixture DB. No corrections yet. no
2 #675 Matching and precedence: the match guard; later loaders fill but never replace (exceptions: Harris on globulars, EGC → Gb); Sharpless and Collinder load before Caldwell; the Caldwell, Arp, TAAS, Sharpless and Barnard loader fixes no
3 #676 Steinicke: the status field, duplicates, not-found objects, the Trumpler "n" flag, type mappings no
4 Stars: HIP cross-match and merge, the D* rule, and the SAC, RASC, TLK, bright-star and WDS loader fixes no
5 Supplementary sources: Hipparcos values, IAU WGSN names, Bica 2008 sizes, the RC3 extract no
6 #677 Names and curated sources: the Messier names reader and file, title-casing NGC 2000 names, deleting extra_names.dat, EGC additions no
7 The corrections files, one rebuilt DB, and data assertions in test_catalog_data.py yes, once

The DB is committed only in PR 7. Every committed DB is about 23 MB that every fielded unit downloads, and the device never runs the import, so loader code may sit ahead of the committed DB in the meantime. Each of PRs 2–6 shows its effect by running the audit's findings.py against a locally built DB and listing which findings disappear.

The items

catalog_import_utils.py: matching

  1. Herschel double-star numbers match Herschel 400 objects (16 objects, reported in Catalog import: 17 SaM listings, SaR 186 and Abl 51 are attached to the wrong objects #668). Loader: PR A's allowlist (bare h, a and b are never catalog prefixes), plus PR 2's match guard. An aka joins an existing sky object only if the types are compatible (stars only with stars) and the listing lies within 30′, or within the object's own size if larger. A rejected match is logged and leaves the listing as its own sky object. PR 2 lists every match the guard rejects on today's sources. A known case: Arp 65 joins NGC 91, which Steinicke types *. (The Lyngå → Collinder joins once expected here don't occur; see PR A's row.) The guard also closes the exact-key lookup, which still matches an unspaced alias ("B53", "h99") without consulting the allowlist; no current source triggers it.
  2. Stars are never merged across Str, SaM, SaR, TLK, RDS and WDS. Loader, PR 4: each star listing is cross-matched to hip_main.dat: the nearest Hipparcos star within about 3′ whose V is within about 1 mag. Listings with one HIP are one sky object, WDS included. A merged star is D* if SaM (less its "rej." rows), RDS, or a WDS pair not flagged optical (S/U/Y/L) or dubious (X) lists it, and * otherwise.
  3. A later loader's magnitude is discarded. Loader, PR 2: later loaders fill an empty magnitude or size and never replace a value. Harris replaces the magnitude on globulars, which also fixes the five globulars off against Harris (NGC 6401 7.4 → 9.45…) without corrections.
  4. Other objects' numbers stored as names (NGC 884 "NGC 869", NGC 1097 "NGC 1097A", NGC 6960 "NGC 6992"). Loader, PR 2: only akas that resolve to the matched object are stored as names.

steinicke_loader.py

  1. Status field S unread. Loader, PR 3: type by status first. Status 6 (part of a galaxy) is typed by its remark: HII → Nb, a cluster or association → OC. Status 5 → Gb. Status 4 *Grp → OC (NGC 6885, NGC 5120).
  2. Trumpler "n" dropped. Loader, PR 3: C+N for the ten listed (IC 1805 … M 16), with IC 4685 as Nb. Check M 18 and IC 1985 before applying the rule broadly.
  3. Type mappings. Loader, PR 3: *Cloud → Ast; PN+OCL → PN. NGC 5299 and 6682 → OC are corrections (PR 7).
  4. Duplicates (status 7 and 8, 713 rows) become objects of their own. Loader, PR 3: attach each number to its twin's sky object. Where Corwin names a different twin (NGC 4362 is NGC 4364), a same_as correction in PR 7.
  5. Not-found objects (status 10, 301 rows) look like faint objects. Loader, PR 3: keep the listing, the nominal position and type ?, and lead the description with "Not found: nothing at Dreyer's position (Steinicke)." Corrections, PR 7: Corwin's identifications, each setting position, type, magnitude and size where known, plus the description (with was.description): IC 346, IC 3056, IC 4991, NGC 4323, NGC 6847, and IC 2477 same_as IC 2480. NGC 3355 stays unresolved.
  6. Magnitudes.
    • Supplementary source, PR 5: an RC3 extract (VizieR VII/155, only the rows whose BT is V-coded, with the query in its ReadMe). Where Steinicke's Bmag equals RC3's V-coded BT within 0.05, V becomes that value. This fixes the 48 galaxies about 1 mag too bright.
    • No Bmag fallback. Item 3's fill covers the Galactic clusters.
    • Corrections, PR 7: M 43 9.0, M 78 8.3, M 8 6.0, NGC 2070 8.0, IC 1266 11.4.

Steinicke's own errors: corrections (PR 7, steinicke.toml)

  1. Every value with its citation is in the audit's reports/ and verdicts/.
    • Identifications (Corwin decides): IC 919, IC 997 and 998, IC 4494, IC 5281 (own_object, a double star, not NGC 7466), IC 1795, NGC 4230, NGC 6174, NGC 6625, NGC 4362, and NGC 2244's position.
      • NGC 553 is on the audit's list but its sizes verdict is unresolved: apply it only if Corwin's identification is firm.
      • NGC 6002 is unresolved, so no change.
    • Types:
      • NGC 2244, 2579, 3247 and IC 2128 → C+N;
      • NGC 1839, 361, 2034 and 1943 → OC;
      • NGC 1921 → C+N.
      • The old LMC globulars (NGC 1835, 1916, 2210, 1754, 1898, 2257, 1786) and the intermediate-age clusters are not corrections: see item 11a.
    • Magnitudes:
      • NGC 4872 14.4 and NGC 4874 11.7 (swapped, positions too);
      • about 15 galaxies with B off by 1–1.8;
      • the LMC "9.p" values (NGC 1814, 1816, 1820, 1974).
    • Sizes:
      • 13 galaxies carrying MCG inner diameters;
      • 7 compact planetaries;
      • the Magellanic decimal slips (NGC 602 3.2′, NGC 395 1.5′, NGC 1869 2′, IC 2145 0.2′);
      • the Magellanic cores (NGC 1986, 2016, 2027, 2037, 2041);
      • IC 424 2.5′;
      • IC 1995 about 60′×30′;
      • NGC 6526 about 40′;
      • NGC 2455, 2579, 6605 and 6625.
    • Missing sizes:
      • the 49 LMC/SMC objects are filled by the Bica 2008 supplementary source (PR 5, fill only);
      • IC 1590, IC 1871, NGC 1931, NGC 2246 and NGC 6847 are corrections.
    • Boundary: NGC 1404's const → Eri.

11a. Magellanic globulars: curated source, PR 6. Add NGC 1754, 1786, 1898, 2210 and 2257 (old), and NGC 1783, 1846, 1978, 2154, 2173 and 419 (intermediate age, 1.4–2.1 Gyr), to EGC.tsv. EGC sets Gb when it joins an object. The 30 young populous clusters (13–560 Myr: NGC 1866, 1850, 1818…) stay OC. The rule is age: 1 Gyr or older is a globular, citing Baumgardt et al. 2013.

specialized_loaders.py

  1. Barnard hard-codes Nb. Loader, PR 2: DN. The Horsehead moves to B 33 by correction (item 24).
  2. Arp doesn't match "NGC 0070" or "NGC 4676A" (42 sizeless duplicates). Loader: PR A's parse_designation reads the zero-padded numbers, so 33 listings join their galaxy and 3 move to the NGC galaxy listed first; the lettered components ("NGC 4676A") are PR 2.
  3. TLK hard-codes *. Loader, PR 4: the D* rule from item 2 types Propus. La Superba's V range 4.86–7.32 is a correction (TLK's range is photographic).
  4. RASC loses the sign of −00° declinations. Loader, PR 4.

sac_loaders.py and the SAC files

  1. All 333 red stars typed D*. Loader, PR 4: *, with the item 2 rule deciding doubles (π¹ Gru becomes D* from its WDS pair).

  2. int("-00") drops the sign (23 rows, Mintaka and δ Ori among them). Loader, PR 4.

  3. load_sac_multistars. Loader, PR 4: skip the five "rej." rows (σ Per); keep a companion's magnitude off its primary (ν Sco); parse "2 v", "var." and "Var" as variable ranges rather than strings (Mira).

    • Corrections, PR 7: Fomalhaut and ζ Scl obj_type = "*", because the companion is a field star and the only pair is dubious respectively.
  4. SAC transcription errors. Corrections, PR 7, sac_multistars.toml (also the list to send SAC):

    • names: SaM 307, 1709, 1348, 2003, 853, 860, 835, 1165, 1771/1772 (ν Sco is Jabbah), 1440, 587, 1252;
    • hip identifications for SaM 1297 (α Mic) and 1211 (ξ Lup), whose declinations are whole degrees off.

    The magnitude table (Vega 0.6 → 0.03, Zaurak, 42 Ori…) is fixed by Hipparcos replacing each matched primary's V (PR 5), not by corrections.

bright_stars_loader.py

  1. Hard-codes *. Loader, PR 4: the merge and the D* rule type Mizar, Mintaka, Acrux and the rest.
  2. Star names predate the IAU's. Supplementary source, PR 5: the IAU WGSN Catalog of Star Names, joined by HIP, adds each IAU name as the preferred name (placed first). The old forms (Graffias, Azmidiske, Auva, Metallah, Tegmen) stay as ordinary names so text search still finds them.

caldwell_loader.py and caldwell.dat

  1. Unresolvable other-names become orphan objects (C 14, C 33, C 49, C 41, C 9). Loader, PR 2:
    • split "a/b" and "a-b";
    • skip "none";
    • keep "Sh2-155" whole;
    • load Sharpless and Collinder before Caldwell;
    • Caldwell fills a missing magnitude (C 37 5.9).

Names: post_processing.py and the name files

  1. No Messier names load (parts[2] of a two-field file). Curated source and loader, PR 6: clean messier_names.dat to M<n><TAB>Name; Name:

    • drop the NGC prefix;
    • "Triangulum Galaxy" only, since "Pinwheel" is M 101;
    • "Beehive Cluster; Praesepe";
    • no name for M 32.

    The fixed reader makes each object's first name its preferred name, which also demotes "Great Nebula in Andromeda" and "Great Nebula in Orion". Then delete extra_names.dat: M 24 gets "IC 4715" as an aka in add_missing_messier_objects, and the other two lines are covered.

  2. NGC 2000's names.dat misplaces names.

    • Loader, PR 6: title-case on read, with small words after the first in lower case ("Crab Nebula", "Ring Nebula in Lyra").
    • Corrections, PR 7, ngc2000_names.toml:
      • M 82 remove "Bode's nebulae", preferred "Cigar Galaxy";
      • IC 434 remove "Horsehead nebula", B 33 preferred "Horsehead Nebula";
      • NGC 5195 remove "M51" and "Whirlpool galaxy";
      • NGC 651 remove "M76" and "Little Dumbbell";
      • NGC 2261's truncated "Hubble's variable neb" → "Hubble's Variable Nebula".
    • "Double cluster" names the pair, so it stays on both NGC 869 and NGC 884 and is made preferred on neither.
  3. Two names in one string, or a description as a name. Corrections, PR 7 (five, so no splitting rule): "Owl or E.T. Cluster", "Theta Carinae Cluster, Southern Pleiades", "Al Rischa (Alrescha)", "Part of Kemble's Cascade", "Burnham pm".

wds_loader.py

  1. wds_precise.txt columns misread (Vega AB "PA=80.0, Sep=0.0" where it should be 184.8° and 84.9″). Loader, PR 4.

Decisions taken with the audit's judgement calls

  • Young LMC clusters: by age. 1 Gyr or older is Gb (via EGC, item 11a); younger stays OC.
  • Open-cluster diameters: Archinal & Hynes (the catalog's), not Sky Catalogue 2000.0. NGC 1966 and NGC 2077 keep their 13–15′ complex sizes, not Bica's cluster-only 1.1–1.3′. No corrections.
  • NGC/IC identifications: Corwin's decides when the sources disagree. Where Corwin is undecided ("?", "??"), the sources' identification stands. IC 4606 therefore becomes NGC 6144 (Corwin and Steinicke), not the Antares Nebula.
  • Unresolved rows stay as the sources give them: NGC 6002, NGC 2319, IC 337, IC 1831, NGC 3355, NGC 7287, NGC 553 and IC 374, plus the magnitudes of IC 132, 466, 2067, 4670 and 5154 and NGC 1711, 1910, 5120 and 6236.

Out of scope here

  • Ordering every object's names (common names ahead of cross-designations everywhere, which would settle feat(chart): name the center object on an optional readout #633's open call) is a device-UX change of its own. It needs a separate issue, probably after a cleanup of names.dat.
  • A NF type code for not-found objects, so the type filter can exclude them. That would be a device feature of its own.
  • OpenNGC's and starplot's own errors (cluster sizes that are MWSC radii, galaxy V that isn't a total, two misplaced common names) go upstream separately. The audit's reports list them.

Activity

  1. 14 remaining items

  2. brickbots commented on Nov 19, 2026

    @brickbots
    OwnerAuthor

    Done. The fixes merged in #671–#696, and #697 rebuilt the catalog database with all of them (squash-merged as 4d6e6d6, sha256 474d0331…). It ships in 2.6.4. The on-device checklist in #697 passed in full on a rev4 unit. A summary for users is in release_notes/2.6.4-catalog.md. Follow-ups: #665 (Barnard's six "a" objects), planned after 2.6.4.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions