Repository navigation
fix(catalog-import): match guard, fill-only precedence, and names that resolve - #675
Merged
Merged
Conversation
…t resolve A listing's aka joins an existing sky object only if the match guard passes: compatible types (a star only with a star; an unknown type or a star group, "***", with either) and within 30' of the sky object, or within its major axis where that is larger. A rejected join is logged and counted, and the listing stays a sky object of its own. Every lookup now goes through the designation allowlist: listing_named is the one question ObjectFinder and the correction pass's resolves_to ask, so an unspaced "B53" or "h99" no longer slips past it. A listing that joins an earlier loader's sky object fills an empty magnitude or size and replaces nothing, but for the two exceptions in catalog.md 9.1: Harris replaces a globular's magnitude, and EGC sets Gb. A listing whose akas resolve to more than one sky object describes the group and fills nothing. Only akas that resolve to the listing's own sky object are stored as its names, so NGC 884's number no longer sits on NGC 869. The Arp loader opts in to lettered components: "NGC 4676A" resolves to NGC 4676 when the listing doesn't also name NGC 4676, and any other letter resolves to nothing. Collinder and Sharpless load before Caldwell. Caldwell splits "a/b" and "a-b", skips "none" and keeps "Sh2-155" whole. Barnard is typed DN. parse_arcmin_size keeps a two-axis size major first, since the fill can now write Collinder's "25x30". Refs #670 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…egrees Five loaders applied a declination's sign by negating its whole degrees, which loses it at "-00": Harris (NGC 6535, M 2, Pal 5, Pal 15), TAAS (NGC 3521, NGC 4666), Barnard (B 326), Collinder (Col 401) and Lyngå (Lyn 229, 267, 306 and 834) put those objects north of the equator. A joined listing hid it behind the first loader's position, until the match guard measured the distance: Har 102, Har 143 and Ta2 129 were refused, and would have become sky objects at their mirrored positions. Each loader now reads the sign on its own through calc_utils.dms_to_dec; Caldwell's identical code has no such row today and gets the same fix. RASC and SAC have the same bug and are #670 items 15 and 17. Refs #670 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #670 (PR 2 in its plan).
How a catalog listing joins a sky object, and what it may change when it does: #670 items 1, 3, 4, 12, 13 (the lettered part) and 22. A second commit fixes a declination-sign bug in five loaders that the match guard brought to light. Every rule below is documented in
docs/ax/catalog.md§9.1 (precedence) and §9.3 (matching).What and why
Match guard (item 1). An aka joins an existing sky object only if
match_guardpasses. Two tests:*,D*,**, maybe with a trailing?) joins only a star, and anything else joins only a non-star. An unknown type (?,-, empty) and a star group (***, Steinicke's*3/*4) are compatible with either kind. The guard reads the loaders' own type strings, beforefix_object_typesnormalises them.The akas are tried in source order, and the first one that resolves and passes decides the join. A rejection is logged as a warning, kept in
MATCH_REJECTIONS, and counted at the end of the import ("Match guard rejected N joins"). It never fails the import, and the listing becomes a sky object of its own.One lookup, through the allowlist.
ObjectFinderused to trymappings.get(name.lower())beforeparse_designation, so an unspaced "B53" or "h99" skipped the allowlist. It now keys on(catalog_code, sequence)and resolves only throughlisting_named(which isparse_designation).corrections.resolves_toasks the same helper, so the correction pass's checks can't drift from the loaders.astro_data/corrections/README.mdand theH24case intest_resolves_to_asks_the_loaders_lookup_question(now False, withB53added) follow.parse_designationandCATALOG_CODE_ALIASESare unchanged.Precedence (item 3). When a listing joins a sky object an earlier loader created,
NewCatalogObject.fill_joined_objectapplies the §9.1 table:filter_mag == 99) or an empty size, and replaces nothing;MAGNITUDE_REPLACED_BY: every Harris listing is a Milky Way globular);Gb(TYPE_SET_BY);The exceptions live in those two constants and in the table, not in a loader.
Names (item 4). A PiFinder-catalog designation is stored as a name only if it resolves to the listing's own sky object. If it resolves to nothing, it is dropped from a joined sky object and kept on one the listing mints. Common names and other catalogs' numbers (UGC, SIMEIS) are always stored. An aka the object already has under another origin is stored again: origin is per-loader provenance, and
get_object_id_to_namesdrops repeats when the device loads names.Load order. Collinder, then Sharpless, then Caldwell. See the cycle below for what that order decides.
Caldwell (item 22).
caldwell_akashandles the other-name field:add_space_after_prefix's "Sh 2".Caldwell fills a missing magnitude through the general rule. Its only fill today is C 100's 4.5 on IC 2944: Collinder fills C 37 first (see the cycle).
Barnard (item 12). Typed
DN, notNb.Arp lettered components (item 13), opt-in. Agreed with the coordinator: the shared default stays strict, and only the Arp loader sets
lettered_components=True.Two-axis sizes major-first.
parse_arcmin_sizesorts a size with exactly two extents, because SizeObject keeps[major, minor]. The fill now writes Collinder's "25x30" for IC 1590, and the audit'sfindings.pyrefuses a minor axis larger than the major. Sizes with one extent, or three or more, keep their order.Declinations between −1° and 0° (second commit). This bug wasn't on #670's list; it is the same class as items 15 and 17, and the guard found it. Harris, TAAS, Barnard, Collinder and Lyngå applied the sign by negating the whole degrees, so "−00" came out north. On
maina joined listing hid this behind the first loader's position. With the guard, Har 102, Har 143 and Ta2 129 were rejected and would have become sky objects at mirrored positions. Each loader now usescalc_utils.dms_to_dec. Caldwell's identical code gets the same fix (no −00 rows today). RASC and SAC are untouched (PR 4).Docs. In §9, the verified rebuild command and the reproducibility caveat below. In §9.1, the precedence table with the group rule, and the load-order cycle. In §9.3, the compatibility rule, the akas-to-names rule and lettered components.
Data effect
Built from
python/withpython -m PiFinder.catalog_imports.main, starting from the committed DB. Compared with the baseline worker'sbaseline.db(main at cfcf4e0), andfindings.pyrun on both.PYTHONHASHSEED0, 1 and 7 are byte-identical, and the committed tree reproduces the measured DB. The import rewrites the existing file in place, so SQLite's header counters (change counter, schema cookie) carry over from whatever file was there. A build on top of a previous build differs in those bytes alone. This holds onmaintoo: back-to-back builds go 601 → 615 → 629. §9 now says so.Findings: 1,366 → 1,329 (52 removed, 15 added)
Removed (52). Each has a verdict, and each new value matches the verdict's correct value.
Fills (13
magnitude_missing):Size fills (3
size_missing):Harris replaces (5
magnitude):EGC sets
Gb(2kind): NGC 1835, NGC 1916.Barnard
DN:kindB 33;nameB 33 also drops out, but only because B 33 now draws as a dark nebula: "Horsehead Nebula" is still PR 7's (ngc2000_names.toml).Caldwell (
nameC 14): C 14 now sits on NGC 869.Names (item 4) and Arp letters (15). The companions' numbers no longer sit on the main galaxies, so OpenNGC's companion rows stop joining them:
position: NGC 1097, 1232, 1253;size: NGC 1097, 1232, 1253, 2735, 3769, 5679, and NGC 7317 twice;magnitude: NGC 1097, 1253;name_catalog_only: NGC 7317 twice.Arp lettered joins (11
size_missing): Arp 170, 176, 236, 242 and 278 now take their NGC/IC object's size.Added (15). None has a verdict yet; my reading of each:
magnitude,kind,position. It now has a magnitude (Collinder's 8.1, see the cycle), so the atlas draws it and its oldAst/position problems show:position(3.4′). Drawn now that Lyngå fills its magnitude (10.0). Steinicke's position, no verdict yet.size. Collinder's 30′×25′ against Corwin's 6′×4′: a PR 7 correction (see below).size. 15′ is inside the verdict's 8–16′, so OpenNGC's 4.2′ is the one that's wrong.size. Lyngå's 15′ against OpenNGC's 4.5′. Unverified: worth a look.name_catalog_onlyC 9. The Cave Nebula object now carries Sharpless's "CAVE", which OpenNGC lacks. Harmless.name_catalog_onlyNGC 4676. "NGC 4676A" from Arp 242, stored by the lettered rule.The other 62 changed findings keep their identity. 20 are Harris magnitudes that now equal the verdicts' correct values (NGC 6401 7.4 → 9.45, NGC 6235 8.9 → 9.97, NGC 5946 8.4 → 9.61, IC 4499 10.1 → 9.76…). The rest are name and listing-order changes from items 4 and 22 (NGC 884 off NGC 869; NGC 4039, 4438 and 4568 off their pair; IC 63 off IC 59; NGC 2175 off NGC 2174; the Copeland's Septet and Stephan's Quintet members off NGC 3746 and NGC 7317; "none" and "Sh 2" gone), or catalog_entries order.
Match guard: every rejection on today's sources
The first build rejected 16. The −00 commit removes 3 of them (they were the loader bug, not mismatches), and the star-group rule removes 1. 12 remain:
*)*same_as = "NGC 90".same_as.same_as = "NGC 6997".harris_loaderfiles "Arp 2" as a catalog designation, so item 4 now drops it from Har 133's names. It should be a common name.D*)D*Removed by the −00 commit: Har 102 → NGC 6535 (35.6′ = 2 × 17.8′), Har 143 → M 2 (98.8′ = 2 × 49.4′), Ta2 129 → NGC 4666 (55.8′ = 2 × 28′). All three now join at the right position.
Removed by the star-group rule: Lyn 921 (OC) → NGC 6994 = M 73 (
***, 2.4′) now joins.Accepted joins nearest the limit. 1,364 guard evaluations in all. Those more than 15′ apart:
Sh2 54 → NGC 6604 passes by 0.4′.
The Lyngå → Collinder cases in the brief don't occur on today's sources.
lynga_loader.identify_catalog_typeknows "Col" but not "Collinder", so Lyngå's 67 "Collinder nnn" names are stored as common names and never reach the matcher. #671's description expected them to join. If that is ever fixed, the guard would reject 3 of them:It would accept the rest, among them Lyn 163 → Col 65 (26′, Col 65's 220′ size), Lyn 780 → Col 367 (18.7′), Lyn 155 → Col 464 (17.1′), Lyn 529 → Col 223 (16.0′), Lyn 505 → Col 213 (13.2′), Lyn 350 → Col 467 (12.1′), Lyn 735 → Col 338 (10.3′) and Lyn 55 → Col 21 (10.1′). Lyn 91 and 92 would now find Col 33 and Col 34 as objects of their own.
Arp: joins that change with the lettered rule
Only the A name is stored on the base object (NGC 4676B, NGC 4933B/C, M51B… are not).
Where Arp also lists the plain number, the join is unchanged and the lettered companion is no longer a name of the main galaxy:
Item 4 drops 129 Arp names in all.
Precedence
The load order is a cycle, and no order of Collinder, Sharpless and Caldwell gets every value right:
The brief's order (Col, Sh2, C) gets C 9, NGC 2467 and NGC 1931 right. C 37's 5.9 becomes a PR 7 correction of Collinder's 8.1. Measured by diffing two scratch builds: nothing else depends on the relative order of the three. The same table is in §9.1.
Names. By origin, item 4 drops:
Caldwell adds 4: "Sh2-155", "NGC 869", "NGC 6992" and "NGC 2237" replace "Sh 2", "NGC 869/884", "NGC 6992/5", "NGC 2237-9" and "none". The four Abell names are the listing's own "Abl n", which Abell's aka list repeats: insert() adds that name anyway, so nothing is lost.
Declinations between −1° and 0°: position before → after
Col 401 is now read faithfully, but its source is probably wrong. Lyngå has the same cluster (Lyn 855, which it names "Collinder 401") at +00°20′, and its B1950 +00°13′ precesses to that. The old sign bug happened to agree with Lyngå. Worth a PR 7 check against SIMBAD/WEBDA.
Left for PR 7
Corrections:
was8.1: a correction of Collinder's value for Col 417.same_as = "Col 50". caldwell.dat names no designation for it ("none"), so no loader order can join it.was.size = "30'x25'". Collinder now fills it.same_as = "NGC 6997".same_as = "NGC 90".Redundant now: NGC 1931's missing-size correction, since Collinder fills 4′×4′ = the verdict.
Check, and correct if confirmed:
same_asNGC 6846);Left for later
lynga_loader.identify_catalog_type). A follow-up PR, stacked on this one, teaches the Lyngå loader "Collinder" and makes Harris file "Arp 2" as a common name (Har 133 above).extra_aliases.Testing
tests/test_catalog_matching.py(new, 79 tests, on a fixture DB, never the committed one):MATCH_REJECTIONS, trying the next aka;parse_component, Arp 242/85/77/79, the strict default elsewhere;load_caldwell,load_barnard,load_arp,load_taas200andload_collinderloaders on fixture sources;test_size_parsing.py: two-axis sizes major-first, three-extent order kept.test_catalog_corrections.py:resolves_tofollows the shared lookup.nox -s lint,format,type_hints,smoke_tests(5 passed) andunit_tests(1,775 passed): all green.findings.pyagainst the baseline, and three reproducibility builds, as above. The DB is not committed (PR 7).🤖 Generated with Claude Code