Skip to content

[SYSTEMDS-3971] Refactor redundant Scuro tests - #2614

Open
shieru1214 wants to merge 3 commits into
apache:mainfrom
shieru1214:scuro-v3
Open

shieru1214 wants to merge 3 commits into
apache:mainfrom
shieru1214:scuro-v3

Conversation

@shieru1214

@shieru1214 shieru1214 commented Sep 14, 2026

Copy link
Copy Markdown

1. Purpose

This PR only changes test files and does not change the Scuro source code. The main purpose is to identify redundant or overly large component tests and refactor them into smaller tests, while also reducing the runtime if possible.

2. Overview

This PR focuses on the representation operator, window operation, and fusion parts.

Stage File Main change
Representation Operator test_unimodal_optimizer.py Combines five modality optimizer tests into one test
Representation Operator test_unimodal_representations.py Reuses the existing audio modality helper
Window Operation test_window_operations.py Combines repeated 1D modality tests and the separate 2D and 3D shape tests
Fusion test_fusion_orders.py Combines four fusion tests

3. Changes

3.1 Representation Operator

In test_unimodal_optimizer.py, the text, image, audio, video, and text-image tests previously created their modalities separately before calling the same optimizer helper. I moved the shared codes and loader setup into create_modality, while the input modalities are stored in MODALITY_SETS.

This keeps the differences between the five cases visible in one place. Each case is executed as a subtest, so a failure directly shows which modality combination caused it.

In test_unimodal_representations.py, I reused the existing create_audio_modality helper in test_audio_representations. This removes a repeated block that created the audio data, loader, and modality. The audio setup can be maintained in one place.

3.2 Window Operation

In test_window_operations.py, the audio, video, and text cases all applied the same four window aggregations to 1D data. These inputs have the same data layout (single level), and the aggregation path is selected by DataLayout rather than ModalityType.

I combined the three methods into one test that runs three modalities and four aggregation methods, keeping all 12 combinations. A failure now reports both the modality and aggregation method, instead of only the name of the original test method.

I also combined the separate 2D and 3D shape tests. Both tests used the same three window operators, and their expected output shapes follow the same rule. The new test keeps all six combinations of two dimensions and three operators. Each case includes dims and operator labels, which makes it easier to find the failing combination.

3.3 Fusion

In test_fusion_orders.py, the separate tests for Average, Concatenation, RowMax, and Hadamard followed the same fusion steps. I combined them into FUSION_PROPERTIES and one shared test method with four subtests.

FUSION_PROPERTIES stores whether switching the first two inputs changes the chained result, and whether chained fusion gives the same result as combining all three inputs in one call. And commutativity is checked by comparing the actual result with the operator's commutative attribute. This checks whether the declared property agrees with the implementation. Another compatible fusion operator can be covered by adding one entry to the table.

This PR also changes the previous Concatenation comparison. The old test compared a two-input result with a three-input result, which were already different in shape. The new test compares chained three-input fusion with direct three-input fusion, so all four operators now use the same comparison.

Finally, the fusion input size was reduced. The test checks input order and fusion calling style rather than data size, so the smaller input follows the same test path while processing less data.

The five optimizer tests built one modality each and then called the same
helper, which holds all the assertions. They become a table of modality
sets and a small factory, driven by one subTest per set, so the number of
combinations stays the same. The keyword arguments keep the inputs as the
separate tests had them, including the text case that used one sentence
instead of ten.

test_audio_representations built its audio modality inline although
_create_audio_modality already exists and the other two audio tests use
it. It now calls the helper as well.

Assisted-by: AI
The three per-modality window aggregation tests ran the same helper over
data of the same shape and dtype, because window_aggregation dispatches on
the data layout and not on the modality type. They become one test with
the modality and the aggregation as subTest dimensions, which also splits
the inner aggregation loop into separately reported cases.

The 3d and 2d shape tests differed only in the input dimensions and in the
hard coded expected shape. Window aggregation only changes the first axis,
so one expression covers both.

Assisted-by: AI
The four tests ran the same eleven line sequence and differed only in the
operator and in the expected booleans. The booleans are the actual
content, so they move into a table with one subTest per operator.

The commutativity check now compares the measured result with the
operator's own "commutative" attribute instead of a second hand written
copy of it. The concatenation test also compared the wrong operand, so its
chain against n-ary property was never covered.

Assisted-by: AI
@codecov

codecov Bot commented Sep 16, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 74.90%. Comparing base (0a245ab) to head (929d9ff).
⚠️ Report is 19 commits behind head on main.

Additional details and impacted files
@@              Coverage Diff              @@
##               main    #2614       +/-   ##
=============================================
+ Coverage     71.39%   74.90%    +3.50%     
=============================================
  Files          1632      436     -1196     
  Lines        196237    25189   -171048     
  Branches      38201        0    -38201     
=============================================
- Hits         140113    18869   -121244     
+ Misses        45099     6320    -38779     
+ Partials      11025        0    -11025     
Flag Coverage Δ
python 74.90% <100.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@shieru1214 shieru1214 changed the title [SYSTEMDS-3971] Improve Scuro test suite [SYSTEMDS-3971] Refactor redundant Scuro tests Sep 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: In Progress

Development

Successfully merging this pull request may close these issues.

1 participant