Repository navigation
Processing Parameters and Asset Roles - #46
fmigneault wants to merge 6 commits into
Conversation
gadomski
left a comment
gadomski
left a comment
There was a problem hiding this comment.
Thanks, this makes more sense to me, but there's a few bits that I find confusing ... I might not be the target audience for these changes, though.
| | ------------------ | ------ | ----------- | ------- | | ||
| | `gdal-calc` | string | A [`gdal_calc.py`](https://gdal.org/programs/gdal_calc.html) expression based on numpy syntax. | `A*logical_or(A<=177,A>=185)` | | ||
| | `openeo` | object | An [openEO process](https://openeo.org/documentation/1.0/developers/api/reference.html#section/Processes) | [Example](https://raw.githubusercontent.com/Open-EO/openeo-processes/1.2.0/normalized_difference.json) | | ||
| | `ogcapi-processes` | object | An [OGC ProcessDescription](https://docs.ogc.org/is/18-062r2/18-062r2.html#ogc_process_description) | [Example](https://schemas.opengis.net/ogcapi/processes/part1/1.0/examples/json/ProcessDescription.json) | |
There was a problem hiding this comment.
This change (adding ogcapi-processes) might be easier to review in its own PR, since it's technically independent of processing parameters.
There was a problem hiding this comment.
I've added it because the document already mentioned "OGC Process API" in the relation types, but didn't provide any corresponding value here.
There was a problem hiding this comment.
I guess my point is, "Add ogciapi-processes to expression formats" could be its own, light pull request that could be easily reviewed and merged?
There was a problem hiding this comment.
I can cherry-pick it if desired.
| > [!NOTE] | ||
| > To ensure portability, `format` may employ a specific profile URI from a registered naming authority | ||
| > that explicitly describes the represented `expression` without ambiguity. | ||
|
|
There was a problem hiding this comment.
I'm not familiar with a "profile URI" — it could be helpful to link out to an explanation to help the reader understand how to use a profile?
There was a problem hiding this comment.
I agree about adding an example for clarity.
Following is an example: https://www.opengis.net/def/profile/OGC/0/ogc-process-description
It can be any URI. Ideally it resolves to a concept/ontology, but it can technically be even just an well-defined string in a standard document. It can also be a JSON schema URI or some other identifiable resource that strictly characterizes the resource.
The idea of using profile URIs is mainly such that what they refer to is more interpretable and removes potential ambiguity in otherwise generic strings.
| | processing-input | An Asset that represents an individual input (e.g., a configuration file required by the processing expression). Note that an input STAC object should instead use the `derived_from` link [relation type](#relation-types). | | ||
| | processing-parameters | An Asset that embeds the entire processing parametrization contents that were submitted. This is similar to [Parameters](#parameters) but is not limited to `application/json` and can contain much more data. | |
There was a problem hiding this comment.
I don't intuitively understand the different between the input and the parameters? In programming terms, aren't they both "function arguments"?
There was a problem hiding this comment.
The difference would be as follows.
Given a process that takes {"a": <some-geotiff>, "b": <some-parquet>, "c": <some-float>}.
You would have multiple processing-input, each respectively pointing at the href of <some-geotiff> and <some-parquet>. The <some-float> can be embedded in an Asset file as text/plain, JSON, or similar if one really feels the need for it.
The processing-parameters instead represent the structure itself, as submitted to the process, i.e.:
{
"a": {"href": <some-geotiff>},
"b": {"href": <some-parquet>},
"c": <some-float>
}This is dependant of the specific encoding the process takes. Ideally it is also indicated as processing:expression to clarify, though not strictly enforced.
The processing-parameters also allows other formats, so if a multipart/related was used for examples instead of the JSON, it could also be provided as a whole.
I purposely used processing-parameters to align with the suggested (as discussed the meeting):
"processing:expression": {
"format": "https://www.opengis.net/def/profile/OGC/0/ogc-process-description",
"expression": { ... the big process description ... },
"parameters": { ... same as above 'processing-parameters' content ... }
}I think it is also a good idea to have distinct inputs/outputs and parameters/results names because they represent differ document contents. From experience with OGC Processes, "inputs" term as been used to refer to different concepts (description vs execution values), and people always get very confused about the structures. Different terms ensure a clear distinction.
There was a problem hiding this comment.
The processing-parameters instead represent the structure itself, as submitted to the process, i.e.:
If I'm understanding (I'm probably not), the asset hrefs are duplicated in processing-input and processing-parameters? If so, this feels redundant.
Different terms ensure a clear distinction.
To me, it's less clear (inputs are inputs, that seems simplest to me). But again, I'm probably not the target audience for this extension, so I won't block.
There was a problem hiding this comment.
Optionally duplicated. It gives the flexibility. I would say it depends how specific the provenance wants to illustrate the concepts.
I can see at least 2 reasons why processing-parameters remains relevant:
- If a process only used a few floating point values, it is overkill to have it generate multiple distinct asset files with just a few bytes of data. Those files might not even exist also.
- If a provenance chain provides the original submission (in the context of how the openEO, OGC API, etc. produced it), it easy to re-execute it as is. Using
processing-inputswould force a client to manually recompose it (maybe incorrectly) from the Assets.
| | processing-output | An Asset that represents an invidivudal output (possibily amongst multiple) that was generated by the processing expression. These Assets should typically include `data` role as well, although not strictly required. | | ||
| | processing-results | An Asset that embeds the entire processing results contents generated by the processing expression (e.g., an _OGC API - Processes_ [Results](https://schemas.opengis.net/ogcapi/processes/part1/1.0/examples/json/Result.json) document. | |
There was a problem hiding this comment.
Similarly, what's the different between an "output" and a "result"? Aren't these both akin to "the return value of the function"?
There was a problem hiding this comment.
Output/Results happens to align with the same OGC concepts of the big "Results" document containing all of the individual "Outputs".
Beside that, same reason as Inputs/Parameters for distinct naming for different concepts.
Note
As discussed at the STAC Community Meeting 2026-10-19, and as replacement of:
processing:expressionwithformat: results#44Changes
parametersfield underprocessing:expressionto provide a mapping of named input values employed.processing-input,processing-output,processing-parametersandprocessing-results.profilein Asset using the above roles and examples for common applications.