Register the FAIR transforms by their PURLs instead of keeping copies

.github/workflows/build.yml

@@ -6,7 +6,7 @@ on:
         paths:
             - 'mapping-server.yml'
             - 'scripts/cli.py'
-            - 'transforms/**'
+            - 'registry.yml'
     schedule:
         - cron: "0 0 1 * *"
     workflow_dispatch:

CONTRIBUTING.md

@@ -54,38 +54,29 @@ To add a new mapping registry to the Mapping Commons:
 
 ## Add a new transform specification
 
-This repository is the source of truth for linkml-map transforms that convert mapping formats into the FAIR Mappings Schema. Currently supported:
+The linkml-map transforms that convert mapping formats into the FAIR Mappings Schema, and the slot-level SSSOM sets that document them, live in [fair-mappings-schema](https://github.com/mapping-commons/fair-mappings-schema) (`src/fair_mappings_schema/transformations/` and `src/fair_mappings_schema/mappings/`). That repository is their source of truth, and this one only registers them. Currently supported:
 
-| Format | Transform file | Source type |
-|--------|---------------|-------------|
-| SSSOM | `transforms/sssom-to-fair.transformation.yaml` | `mapping set` |
-| LinkML-Map | `transforms/linkmlmap-to-fair.transformation.yaml` | `TransformationSpecification` |
+| Format | Transform | Source type |
+|--------|-----------|-------------|
+| SSSOM | <https://w3id.org/fair-map/transforms/sssom-to-fair> | `mapping set` |
+| LinkML-Map | <https://w3id.org/fair-map/transforms/linkmlmap-to-fair> | `TransformationSpecification` |
 
 To add support for a new mapping format:
 
-1. **Create a transform file** in `transforms/` following the [linkml-map specification](https://linkml.io/linkml-map/). The transform should map your format's metadata to `MappingSpecification` fields.
+1. **Add the transform** (and its slot-level SSSOM set) to fair-mappings-schema, following the [linkml-map specification](https://linkml.io/linkml-map/). The transform maps your format's metadata to `MappingSpecification` fields.
 
-2. **Register it** in `registry.yml`:
+2. **Give it a PURL** under `https://w3id.org/fair-map/transforms/` in the [w3id `fair-map` config](https://github.com/perma-id/w3id.org/tree/master/ids/fair-map).
+
+3. **Register it** in `registry.yml` by that PURL:
 
    ```yaml
-   transform_references:
-     - id: https://w3id.org/linkml/transformer/myformat-to-fair-mappings
-       title: MyFormat to FAIR Mappings Schema
-       description: Transforms MyFormat metadata to FAIR Mappings Schema
+   mapping_set_references:
+     - mapping_set_id: https://w3id.org/fair-map/transforms/myformat-to-fair
+       mapping_set_group: fair-mappings-transforms
        local_name: myformat-to-fair.transformation.yaml
-       source_format: myformat
-       source_schema_url: https://example.org/myformat-schema.yaml
-       source_type: MyRootClass
    ```
 
-3. **Update `scripts/cli.py`** to handle the new format if it requires different fetching/parsing logic.
-
-4. **Test locally**:
-
-   ```bash
-   just transform-sssom my-input.yaml my-output.yaml   # for SSSOM
-   just transform-linkmlmap my-input.yaml my-output.yaml  # for LinkML-Map
-   ```
+4. **Update `scripts/cli.py`** to handle the new format if it requires different fetching/parsing logic.
 
 ## Local development
 

README.md

@@ -12,7 +12,7 @@ This repository:
 - Transforms SSSOM and LinkML-Map metadata into the FAIR Mappings Schema using [linkml-map](https://github.com/linkml/linkml-map)
 - Serves a static search/browse interface on GitHub Pages
 
-The transform specifications in [`transforms/`](transforms/) are the source of truth for translating existing mapping formats into the FAIR Mappings Schema. See [`registry.yml`](registry.yml) for the full list.
+The transform specifications that translate existing mapping formats into the FAIR Mappings Schema, and the slot-level SSSOM mapping sets that document them, live in [fair-mappings-schema](https://github.com/mapping-commons/fair-mappings-schema), which is their source of truth. [`registry.yml`](registry.yml) lists them by their PURLs, and the ETL fetches them from there.
 
 ## Local development
 

registry.yml

@@ -6,19 +6,23 @@ mapping_registry_description: >-
 homepage: https://github.com/mapping-commons/mapping-commons.github.io
 documentation: https://github.com/mapping-commons/mapping-commons.github.io/blob/main/README.md
 
+# The source of truth for these four files is
+# https://github.com/mapping-commons/fair-mappings-schema. Each mapping_set_id
+# is a PURL that resolves to the file there; the ETL fetches it from that id.
+# local_name is the file name, which tells the ETL how to read it.
 mapping_set_references:
-  - mapping_set_id: https://w3id.org/mapping-commons/fair/mappingspecification/sssom-to-fair-mappings
+  - mapping_set_id: https://w3id.org/fair-map/transforms/sssom-to-fair
     mapping_set_group: fair-mappings-transforms
     local_name: sssom-to-fair.transformation.yaml
 
-  - mapping_set_id: https://w3id.org/mapping-commons/fair/mappingspecification/linkml-map-to-fair-mappings
+  - mapping_set_id: https://w3id.org/fair-map/transforms/linkmlmap-to-fair
     mapping_set_group: fair-mappings-transforms
     local_name: linkmlmap-to-fair.transformation.yaml
 
-  - mapping_set_id: https://w3id.org/mapping-commons/fair/mappingspecification/sssom-to-fair-mappings/slot-mappings
+  - mapping_set_id: https://w3id.org/fair-map/mappings/sssom-to-fair
     mapping_set_group: fair-mappings-slot-mappings
     local_name: sssom-to-fair.sssom.tsv
 
-  - mapping_set_id: https://w3id.org/mapping-commons/fair/mappingspecification/linkml-map-to-fair-mappings/slot-mappings
+  - mapping_set_id: https://w3id.org/fair-map/mappings/linkmlmap-to-fair
     mapping_set_group: fair-mappings-slot-mappings
     local_name: linkmlmap-to-fair.sssom.tsv

scripts/cli.py

@@ -137,47 +137,41 @@ def _process_sssom_mapping_set(mapping_set_id: str, mapping_set_uri: str) -> dic
 
 
 def _process_transform_registry(registry_path: str) -> list[dict]:
-    """Process local transform specs in registry.yml as browsable entries."""
+    """Process the transform specs and slot mappings in registry.yml as browsable entries.
+
+    The files live in mapping-commons/fair-mappings-schema; each one is fetched
+    from its mapping_set_id (a PURL) or mirror_from, not read from this repository.
+    """
     with open(registry_path) as f:
         registry = yaml.safe_load(f)
 
     info = _registry_info(registry)
-    base_dir = os.path.dirname(registry_path)
-    search_dirs = ["transforms", "mappings"]
     specs = []
 
     for ref in registry.get("mapping_set_references", []):
         local_name = ref.get("local_name")
-        if not local_name:
-            continue
-
-        # Find the file in one of the search directories
-        input_file = None
-        for d in search_dirs:
-            candidate = os.path.join(base_dir, d, local_name)
-            if os.path.exists(candidate):
-                input_file = candidate
-                break
-        if not input_file:
-            click.echo(f"File not found: {local_name} (searched {search_dirs})", err=True)
+        url = ref.get("mirror_from", ref.get("mapping_set_id"))
+        if not local_name or not url:
             continue
 
         # Detect mapping type from file extension
         mapping_type = "sssom" if local_name.endswith(".sssom.tsv") else "linkml_map"
 
         try:
+            response = _session.get(url)
+            if response.status_code != 200:
+                raise RuntimeError(f"HTTP {response.status_code} fetching {url}")
             if mapping_type == "sssom":
-                with open(input_file) as f:
-                    captured, columns = _capture_sssom_head(f)
+                captured, columns = _capture_sssom_head(response.text.splitlines())
                 meta = parse_sssom_tsv(iter(captured))
                 _inject_per_mapping_source(meta, columns)
                 spec = transform_to_fair(meta, "sssom")
             else:
-                spec = load_mapping(input_file, mapping_type)
+                spec = transform_to_fair(yaml.safe_load(response.text), mapping_type)
             spec["registries"] = [info]
             specs.append(spec)
         except Exception as e:
-            click.echo(f"Transform failed for {local_name}: {e}", err=True)
+            click.echo(f"Transform failed for {local_name} ({url}): {e}", err=True)
 
     return specs