Repository navigation
[core-spec] Add Relational Query Interface specification - #354
justint-db wants to merge 8 commits into
Conversation
Adds core-spec/relational_semantics.md, a draft specification for querying Ossie semantic models through SQL extended with measure semantics -- Layer 2 of the layered query interface (the Relational Interface).
Syncs the relational_semantics.md spec with the upstream draft, adding Appendix D relating the design to Hyde and Fremlin's Measures in SQL.
christianeu-db
left a comment
christianeu-db
left a comment
There was a problem hiding this comment.
For this spec, it would be good to state how this fits in with OSSIE_SQL_2026 (e.g. the query complement to the expression language) and an initial subset of SQL that MUST be supported by vendors to be compliant with Layer 2 OSSIE_SQL_2026
| MEASURE(s.ticket_count), | ||
| MEASURE(o.total_revenue) / MEASURE(s.ticket_count) AS revenue_per_ticket | ||
| FROM model.orders o | ||
| JOIN model.support s ON o.customer_id = s.customer_id |
There was a problem hiding this comment.
If all the underlying datasets were in a a single model, no joins would be necessary. This example seems to be about two different models, is that correct?
Does the spec describe valid interactions between two different models?
There was a problem hiding this comment.
No, this example is a single model (model) with multiple datasets (orders and support). This spec does not consider multiple model interactions.
To support multi-dataset models, we have to provide a mechanism to join the datasets together. There are three ways I see to do this:
- Hard code the exact joins that must always be used (the join conditions, the join types, and the join order). This approach has no expressivity, the model is equivalent to a model with just a single dataset defined by the join of all the underlying tables.
- (This spec) Allow the model consumer to pick the joins at query time through SQL. This is flexible and expressive, but potentially more complicated for consumers.
- Use heuristics to automatically pick joins at query time. This leads to a simpler interface (consumers can pretend the model is a single table), but limits expressivity, does not compose well with general SQL, and there are no well-defined semantics for this approach. There is an Ossie proposal to add this type of query interface as well (Initial Foundational Semantics Spec #246).
|
|
||
| ### 7.6 JOIN | ||
|
|
||
| A measure relation MAY be joined to any relation using any join type (INNER, LEFT, RIGHT, FULL |
There was a problem hiding this comment.
consider the case where the model, as well as the (perhaps the same) underlying physical tables are exposed to the BI tool. Model provides a safe way to join the tables. This proposal says that model can be joined with regular tables as well. Would that produce correct results in all cases?
There was a problem hiding this comment.
Under our correctness definition, yes. Our correctness definition basically says: "when measure aggregates are computed, never include duplicates or NULLs introduced by joins" and "all SQL queries must return the same results for the same query text (modulo normal SQL dialect differences)".
This guarantees consistent results and no fan or chasm traps.
Generalize the argument rule from "a measure column" to "an expression of type T measure". In this version the only such expression is a measure column reference, so behavior is unchanged, but later extensions (e.g. a measure with a modified evaluation context, or an anonymous inline measure) can be added without changing the rule. Co-authored-by: Isaac <no-reply@databricks.com>
…ce rows Rewrite Section 8.1 to define, for each output group, the set of a measure's source rows it aggregates, via derivation rules for every SQL operator between the measure's definition and its evaluation. No duplication (G2), filtering through intermediate queries, and outer-join null-padding all follow from the definition. Results are unchanged; the Section 9 reference strategy is one realization. Remove normative uses of "evaluation context" so the spec does not commit to a model for AT-style extensions. Co-authored-by: Isaac <no-reply@databricks.com>
"Grain" was used for three different concepts: a measure's row identity, the level a query's grouping reports at, and AT-style manipulation of which rows a measure aggregates. Use "row identity" (and "source rows") for the first, "grouping"/"level" for the second, and "evaluation controls" for the third. Rename G2 to "No duplication" and Section 5.3 to "Evaluation controls". Add "input set" to the glossary. Co-authored-by: Isaac <no-reply@databricks.com>
…AGG() Use AGG() as the single normative spelling throughout, so evaluating a measure is visually distinct from defining one (TO MEASURE). Engines MAY keep MEASURE() or other spellings as non-normative synonyms; portable queries and conformance tests use AGG(). Update Appendix D to explain the choice over the paper's AGGREGATE(). Co-authored-by: Isaac <no-reply@databricks.com>
Phrase the aggregated-SELECT rule and the Section 7 type summary in terms of expressions of type T measure rather than measure columns, matching the generalized AGG() argument rule. Rules that are genuinely about columns (relations, propagation, discovery) are unchanged. Co-authored-by: Isaac <no-reply@databricks.com>
AGG() corresponds to the paper's AGGREGATE() (evaluation AT VISIBLE); the difference is that bare measure references are rejected. Add a section on choices this specification makes where the paper is open (intermediate filters and joins, joins in the evaluating query, outer joins). Rewrite the extension considerations: where controls are written and how they compose, which filters a control may lift, and identifying dimensions. Co-authored-by: Isaac <no-reply@databricks.com>
Follow the updates to the Relational Query Interface spec (apache#354): the evaluation function is AGG(), and "grain-safe" becomes "duplication-free". Co-authored-by: Isaac <no-reply@databricks.com>
3 participants
Summary
Adds the Relational Query Interface specification to
core-spec/asrelational_semantics.md. It defines Layer 2, the Relational Interface of the layered Ossie query interface: how a semantic layer exposes an Ossie model to SQL-native BI and AI tools as a set of related SQL relations, and the semantics and correctness guarantees of querying the model's measures through ordinary SQL.Grounded in Hyde and Fremlin's Measures in SQL (https://arxiv.org/abs/2406.00251), a measure is a column evaluated by a
MEASURE()aggregate. A tool writes the SQL it always would and substitutesMEASURE(name)for a raw aggregate; the engine evaluates each measure at its own grain, so results stay correct across joins, filters, and grouping.It realizes two Ossie design proposals: the BI-Semantic Layer Interface and the Layered Query Interface. It is the "semantics of a layer" companion foreshadowed in #344, sits alongside the foundational semantics in #246, and builds on the dataset-scoped metrics proposed in #343.
Related Issues
Related: #246, #343, #344.
Checklist
Specification
core-spec/and follow the existing structuredev@ossie.apache.org; on-list discussion of its contents is still to comeOntology
ontology/are consistent with spec changes — N/A, no ontology changeConverters
converters/is updated to reflect spec or ontology changes — N/A, no converter changeValidation
validation/are updated if the spec changed — N/A, no schema/validation changeDocumentation
docs/is updated to reflect any user-facing changes — N/A, the change is itself a spec documentCONTRIBUTING.mdis updated if the contribution process changed — N/AExamples
examples/are added or updated for any new spec constructs or converter support — N/A, no new constructs requiring example modelsTests
pytest/ CI green) — N/A, documentation only, no codeCompliance
AI disclosure
Per the ASF Generative Tooling Guidance, this contribution was prepared with AI assistance. All specification decisions and design choices are mine. I have reviewed and verified every change.