Skip to content

fix(catalogs): join akas by allowlisted designation, and four import fixes - #671

Merged
brickbots merged 1 commit into
mainfrom
fix/catalog-designation-allowlist
Oct 24, 2026
Merged

brickbots merged 1 commit into
mainfrom
fix/catalog-designation-allowlist

Conversation

@brickbots

Copy link
Copy Markdown
Owner

Part of #670 (PR A in its plan). Refs #668.

These are mrosseel's catalog-import fixes, extracted from #379 (squash commit 954a83c46 on nixos) with his agreement. The Perek-Kohoutek catalog stays in #379. Every changed line is identical to his, so these hunks drop out when #379 rebases onto main.

This PR is code only. The committed pifinder_objects.db is unchanged, so nothing below reaches a device until the database is rebuilt. That happens once, with the rest of the catalog-audit fixes.

What changes

  • Aliases join only by an allowlisted designation. ObjectFinder fell back to ui_utils.normalize(), which strips spaces and hyphens. A double star's Herschel number ("H 24", "h 289") became a Herschel 400 key, and a compound designation ("M 2-9") became another catalog's sequence ("M 29"). parse_designation() now splits an alias into a catalog code and a sequence, and accepts only the prefixes in CATALOG_CODE_ALIASES. The allowlist covers every catalog code in the database except WDS, whose designations aren't sequences. Bare "h", "a" and "b" are left out because double-star sources use them for discoverers.
  • M 45: "Cr" now maps to Col, so M 45's "Cr 42" alias joins the Collinder 42 object. add_missing_messier_objects() sets the curated Pleiades position, magnitude and size on that shared object.
  • Sharpless: load_sharpless passed the B1950 declination. It now passes j_dec_deg.
  • abell.tsv: the ngc6742 alias moves from row 51 to row 50, and row 47 gets back its minus sign.
  • Reproducible name order: load_taas200 and load_wds keep their names in source order instead of a set(), so two imports give the same names and the same links.
  • Tests: tests/test_designation_parsing.py.

Data effect at the next rebuild

These numbers come from simulating the old and new lookups against the committed database, in load order.

  • Catalog import: 17 SaM listings, SaR 186 and Abl 51 are attached to the wrong objects #668: all 19 listings detach.
    • 13 SaM listings become objects of their own.
    • SaM 1257/1258 and SaM 1521/1522 stay together as pairs, off the Herschel 400 object. Each pair is two components of one star.
    • SaR 186 becomes its own object.
    • Abl 51 is fixed by the abell.tsv row move.
  • Lyngå: 67 listings now join their Collinder object. The long form "Collinder 463" never matched before. 55 of them are within 10′.
  • Arp:
    • 33 listings now join their NGC or IC galaxy, because int() reads zero-padded names like "NGC 0497". All 33 are within 1.9′.
    • Arp 127, 276 and 282 move from their IC galaxy to the NGC galaxy the source lists first.
  • M 45 and Col 42 become one object with the Pleiades values, not two Pleiades.
  • Sharpless: all 313 objects move 0.26° to 0.37° in declination, to their J2000 positions.
  • Ta2 52 always joins NGC 2237, which TAAS lists before NGC 2244. Today it depends on the hash seed. This settles one of Catalog import: 17 SaM listings, SaR 186 and Abl 51 are attached to the wrong objects #668's "less clear-cut" items.

Each listing that now joins used to be a duplicate sky object.

Left for the match guard

Two kinds of join are left for the match guard, which comes in PR 2 of #670 and checks type and separation before a join. They're recorded there as test cases.

  • Twelve Lyngå joins more than 10′ apart.
    • Lyn 639 → Col 285, 20° apart: the UMa moving group.
    • Lyn 91 → Col 33 (59′) and Lyn 92 → Col 34 (71′). Both Collinder entries already sit on IC 1848's object, so Lyn 90, 91 and 92 all end up on it.
    • Lyn 829 → Col 394, 39′.
    • Lyn 163 → Col 65, 26′.
    • Lyn 780 → Col 367, 19′.
    • Lyn 155 → Col 464, 17′.
    • Lyn 529 → Col 223, 16′.
    • Lyn 505 → Col 213, 13′.
    • Lyn 350 → Col 467, 12′.
    • Lyn 735 → Col 338, 10.3′.
    • Lyn 55 → Col 21, 10.1′.
  • Arp 65 → NGC 91, which Steinicke types as a star. This is Catalog import: 17 SaM listings, SaR 186 and Abl 51 are attached to the wrong objects #668's other "less clear-cut" item, and the nixos behaviour.

Testing

All sessions ran on Python 3.9, against the committed database.

  • nox -s lint: all checks passed, and ruff format --check finds nothing to reformat.
  • nox -s type_hints: no issues in 153 source files.
  • nox -s smoke_tests: 5 passed.
  • nox -s unit_tests: 1565 passed, including the new test_designation_parsing.py and the unchanged test_catalog_data.py.

🤖 Generated with Claude Code

…fixes

Extracted from mrosseel's #379 (squash commit 954a83c on nixos) with
his agreement. The Perek-Kohoutek catalog stays in #379. The changed
lines are identical to his, so they drop out when #379 rebases.

- ObjectFinder fell back to ui_utils.normalize(), which strips spaces
  and hyphens. That turned a double star's Herschel number ("H 24",
  "h 289") into a Herschel 400 key, so SAC multiple-star and red-star
  listings joined H400 galaxies and clusters (#668). It also turned a
  compound designation ("M 2-9") into another catalog's sequence (M 29).
  parse_designation() now splits an aka into a catalog code and a
  sequence, and accepts only the prefixes in CATALOG_CODE_ALIASES.
- That allowlist maps "Cr" to Col, so M 45's "Cr 42" aka now joins the
  Collinder 42 sky object, where the import used to build a second
  Pleiades. add_missing_messier_objects() sets the curated Pleiades
  position, magnitude and size on that shared object, because insert()
  leaves a joined sky object unchanged.
- load_sharpless computed j_dec_deg but passed the B1950 declination,
  so all 313 Sharpless objects sat 0.26 to 0.37 degrees off in
  declination.
- abell.tsv: the ngc6742 alias belongs on row 50, not row 51 (#668).
  Row 47 had lost its declination sign (-00 13 51 per SIMBAD).
- load_taas200 and load_wds kept names in a set(), whose order follows
  Python's per-process hash seed. They keep the source order now, so
  two imports give the same names and the same TAAS links. Ta2 52 joins
  NGC 2237, which TAAS lists before NGC 2244.

parse_designation also reads long-form and zero-padded designations
("Collinder 463", "NGC 0497"). So 67 Lyngå listings now join their
Collinder objects, and 33 Arp listings their NGC or IC galaxies, where
each used to be a duplicate sky object. Arp 127, 276 and 282 move from
their IC galaxy to the NGC galaxy the source lists first.

This is code only: the committed database is unchanged. The rebuilt
database lands later, in one rebuild with the other catalog-audit
fixes.

Co-authored-by: mrosseel <mike.rosseel@gmail.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@brickbots
brickbots merged commit 08bbf60 into main Oct 24, 2026
4 checks passed
Zhuohang-Shen pushed a commit to Zhuohang-Shen/PiFinder that referenced this pull request Nov 15, 2026
Brings in 24 commits of brickbots main: the catalog import fixes
(brickbots#671 to brickbots#694), the Named Stars catalog (brickbots#695), the names view of the
object list (brickbots#696), the Pi OS side of the migration (brickbots#657), the release
workflow (brickbots#658), the OLED SPI clock (brickbots#701) and CI changes.

Conflicts and their resolution:
- Pi OS migration files (migration_gate.json, migration_progress,
  migration_progress.c, nixos_migration_calc.py): nixos keeps them
  deleted. They belong to the Pi OS line.
- noxfile.py, requirements_dev.txt: nixos keeps them deleted. The flake
  dev shell and uv2nix replace them.
- ui/software.py, tests/test_software.py, sys_utils.py, main.py: nixos
  keeps its own software screen. Its keep_awake and on_battery are the
  same feature as upstream's. The migration UI stays out of nixos.
- catalog_imports: upstream's code. The parser moved to designations.py,
  so the local copy of parse_designation goes. The Perek-Kohoutek loader
  stays and runs before the Named Stars loader.
- catalogs.py: upstream's NAME_MODE_CATALOGS. The T9 tables stay in
  catalog_arrays.py.
- object_details.py: upstream's listing index (brickbots#678), with a search on
  the listing_key column for an ObjectSequence.
- obslist.py and its tests: the name index of nixos (resolvers) with
  upstream's ambiguous designation check.
- tests/test_ui_modules.py: upstream's comet retry stub, the catalog
  build of nixos (no background loader).
- docs/ax/catalog.md, CONTEXT.md: text of both sides.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
brickbots added a commit that referenced this pull request Nov 19, 2026
…xes (#670) — squash-merge only (#697)

* fix(catalog-corrections): NGC 771 is the star 50 Cas, Named Stars' Gang

Corwin's NGC notes: "NGC 771 = 50 Cassiopeiae is a star" (an A1 V star,
no nebulosity on the POSS1). 50 Cas is HIP 9598, IAU name Gang, which
#695 lists as Nam 9598. The listing joins that star; the match guard
accepts the join (both type *, 0.04' apart).

Refs #670

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* data(catalog): rebuild pifinder_objects.db with the catalog-audit fixes

The one rebuild for the catalog-audit campaign. It carries the import,
loader, star-identity and correction changes of #671 to #696, and the
NGC 771 correction before it. Built from a fresh file with
`python -m PiFinder.catalog_imports.main`; two builds with different
PYTHONHASHSEED values are byte-identical:

sha256 03b6bcb3288b1a2909e4fedb49559f46517c7bf4e3f13ad097f848f2c5ad670e

All 225 corrections apply (70 of them hip identifications).

Refs #670

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(catalog-data): assert the rebuilt database and the campaign's main fixes

test_catalog_data.py reads the committed database, so its assertions
land with the rebuild: 22 catalogs, EGC 102, SaM 2157 (rejected pairs
skipped), Nam 578, and IC 1805 typed C+N (Trumpler "n", #676).

New assertions, each read from the rebuilt file: the M 82 and M 57
preferred names; a not-found NGC object's description; Mizar as one D*
sky object with its Str, SaM, WDS and Nam listings and the double-star
filter magnitude; Vega's V 0.03 and Hipparcos position; SaM 1019 on
HIP 44804; IC 2145 joined to NGC 2086, NGC 771 to Nam 9598, and Gumala
one object (WDS 99456 + Nam 94645); Dhanuśarāgra with its diacritics.

docs/ax/catalog.md §9: the status note goes, since the committed
database now matches the import §9 describes. Its code map and the
ADR 0046 link stay, as a plain paragraph. The compressed size is
about 24 MB.

Refs #670

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(catalog-import): read Collinder's two name columns, not their raw text

load_collinder added the raw text of collinder.txt's "NGC/Other Cat." column
and of collinder2.txt's "Other names" column as names. "581 (M103)" stayed a
plain name, so Col 14 was a second sky object 0.1' from M 103, and Col 399
carried "4-5 Vul." and the one name "Al Sufi's Cluster, Brocchi's Cluster,
Coathanger".

The loader now reads both columns (collinder_names):

- a designation is written out: "581 (M103)" is "NGC 581" and "M 103",
  "IC1848" is "IC 1848", "Tr.9" is "Trumpler 9". 29 Collinder listings now
  join the Messier, NGC or IC object that Collinder names (26 Messier
  clusters, Col 334 = NGC 6374, Col 382 = IC 4725, Col 26 = IC 1805);
- "(incorrectly) 3247" gives no name, so Col 220 can never join NGC 3247;
- "nl", the star numbers "4-5 Vul." and "9-12 Gem", and notes in
  parentheses ("(PMC)", "(Mel 72 per A/H)") are no names;
- "2384a" is the name "NGC 2384a", which joins nothing;
- a comma list is separate names, and a name is listed once.

The IC 2602 correction removed "Theta Carinae Cluster, Southern Pleiades".
The loader now splits that list, so the correction is deleted.

Refs #670

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

* data(catalog): rebuild pifinder_objects.db with the Collinder name fix

A fresh-file build of this branch (the database was deleted first), after the
Collinder loader fix. Two builds with PYTHONHASHSEED=1 and 4242 are
byte-identical:

sha256 c3cdb493338f57a4c01d8d69b4170039b8dffedf991b4b93ce73ed961dad861d

The import loads and applies all 224 corrections.

Against the first rebuild of this branch (ead7120, sha256 03b6bcb3...):
146,631 -> 146,602 sky objects, because 29 Collinder listings now share the
Messier, NGC or IC object that Collinder names. The 151,754 catalog listings
are the same. No other join changed.

This replaces ead7120's database. Squash-merge the pull request, so that
only this version of the file lands on main.

Refs #670

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

* test(catalog-data): assert the Collinder name fix in the rebuilt database

Col 399 has "Al Sufi's Cluster", "Brocchi's Cluster" and "Coathanger" as
three names, and no "4-5 Vul.". Col 14, 189, 382, 26 and 334 share the
Messier, NGC or IC object that Collinder names; M 44 keeps "Beehive Cluster"
first, and IC 1805 keeps its type C+N. Col 220 is not NGC 3247, and Col 142
("NGC 2384a") is not NGC 2384. No Collinder name holds raw column text.

Refs #670

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

* docs(release-notes): the 2.6.4 catalog changes, as an appendix the release notes link to

Refs #670

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(catalog-import): every WDS pair is a double star unless WDS marks it dubious

The D* rule's WDS clause now types a WDS listing D* if any of its pairs
has a companion magnitude and is not noted X (dubious). Optical pairs
(S, U, Y, L), pairs without the primary (BC), and pairs of any
separation or magnitude difference count. The type says what the
object is; visibility is for the filters (a double filters by its
brighter star) and for the separation and magnitudes the entry shows.
This follows mrosseel's review of #697. The old limits (0.5-60",
5 mag, primary only, no optical flags) typed 40,788 of 131,303 WDS
listings single; the new rule types 10,597 single.

The Vega (SaM 1241) and Fomalhaut (SaM 1633) obj_type corrections are
deleted: they undid the old rule for optical/wide companions, and both
stars are now D* by WDS, as SAC lists them. zeta Scl's (SaM 1745, only
pair X-flagged) agrees with the new rule and stays. 222 corrections.

The type still affects no match: the match guard treats * and D* alike
(is_star_type), and the star identity reads the type only to type the
merged star.

Refs #670

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* data(catalog): rebuild pifinder_objects.db with the WDS double-star rule

Every WDS pair is a double star unless WDS marks it dubious (X) or gives
no companion magnitude. Objects typed D*: 91,444 -> 121,232; WDS
listings D*: 90,916 -> 120,706. Only types change against the previous
build (c3cdb493...): 30,049 listings go from * to D*, no join, value,
name or image changes. 222 corrections apply clean.

Two fresh builds (PYTHONHASHSEED=1 and 4242) are byte-identical:
sha256 474d0331a5bfacebb90b2f2cfd08c1e1db1ca24ac18efc92459aedecf35a52c7

Refs #670

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(catalog-data): assert the WDS double-star rule in the rebuilt database

Atria (optical pair WFC 179) and Suhail (SEE 109, 12.6 mag fainter) are
D*; Arcturus (only pair HDS 2003, flagged X) is *; Vega is D*. WDS
listings by type: 120,706 D*, 10,597 *. The 2.6.4 catalog release note
gets a "Double stars follow WDS" bullet.

Refs #670

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant