Repository navigation
Correct quality-reporting guidance and advance the work plan - #912
Merged
Merged
Conversation
Separate measurement agreement, model quality and refinement benefit; remove unsupported thresholds and capability claims. Align schema descriptions and generated models with actual reporting support, and reconcile the backlog without changing retained scientific evidence.
Correct residual reporting guidance found by adversarial review (#913), preventing authors from expecting unrouted confidence fields or prediction summaries for NMR-only evidence. Refresh the complete backlog snapshot and retain the review audit.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The quality-reporting guide prescribed unsupported cutoffs and refinement verdicts, used an incorrect pLDDT conversion, and overstated tool independence and retained 1SAR coverage. It now separates measurement agreement, final-model quality and improvement over the actual starting model, with matched-input requirements and links to applicable registry criteria.
Correct directly repeated handbook, oracle and schema descriptions, and regenerate Python models without changing field requirements or runtime grading. Document current emitter limits instead of promising automatic QDS content. Refresh NEXT_TASKS.md from complete queues (13 issues and this PR; October 17 local / October 18 UTC) and record that the existing reconcile-next-tasks skill provides the requested update workflow.
Independent adversarial reviews checked scientific interpretation against primary sources and retained evidence, and reporting instructions against current contract-4 behavior. The fresh contract review found #913: residual unrouted-field examples and a prediction-ensemble rule incorrectly applied to T17 NMR. The finding was independently verified, filed before editing, corrected, and re-reviewed with no remaining material findings. An earlier review also corrected a residual ligand-block overstatement under #910.
Validation: locked Python 3.12 benchmark-extra full gate, including backlog guards, retained evidence replay and generated-model consistency; whitespace check; separate read-only review of the final three-file fix. Detailed audit: ref/research/quality_reporting_review_2026-10-17.md. No new scientific runs or changes to adopted thresholds, canonical recommendations/assumptions, or retained EVAL/QDS evidence.
Closes #907
Closes #908
Closes #909
Closes #910
Closes #913
Remaining work: #911 audits the ProSMART Rama-Z recommendation and requires a separate dated registry correction. Scientific issues #760, #765, #790, #799, #872, #873 and #876 retain their dispositions. The next proposed deliverable is a reusable before/after refinement assessment; relying on the disputed ProSMART capability first requires resolving #911.