You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Follow-ups on the gap-merge pass shipped in #1020 (merge_gapped_clones in rust/crates/cpd-core/src/detect.rs), from the review of that PR. None of them affect default runs (the pass is skipped at --max-gap-lines 0).
Chain search for repeated blocks. The pass only tries to extend the last clone of a file pair. When a block is copied twice with an inserted line each, the sorted list interleaves the pairs, the chain breaks, and the option silently does nothing for that pair. Search all open chains of a pair (or retry the previous clone) and add a fixture with a block copied twice.
Bound the gap in tokens and add a similarity floor. The gap is measured in lines only, so a single 3000-token line bridges two unrelated clones into a similar ~0.05 result. Add a token bound and/or a minimum similarity for merged clones.
--max-gap-lines N tolerates N+2 inserted lines. Exact windows already reach into the inserted region on both sides, so a 3-line insertion measures as a 1-line gap. Measure from the matched lines, or document the real rule.
Deep overlaps.continuation() accepts any overlap as long as start and end increase; A B C vs A B junk B C merges into one ~0.70 clone that reuses A's tokens for two B regions. Cap the tolerated overlap (one boundary token) or subtract it correctly on both sides.
Cross-format pairs never merge. Primary and secondary passes stamp different formats on the two halves of a js/ts pair; drop format from the pair key or normalize it in both passes.
Clone order changes when the flag is on even if nothing merges (the pass sorts the whole list). Emit in the original order of each chain's first member.
Gap lines count as duplicated lines in statistics, so --threshold gets stricter when merging is enabled. Count matched lines only, or exclude gaps from the percentage.
Merged renamed halves lose kind: renamed. Decide precedence (similar subsumes renamed, or keep renamed when every part was renamed) and document it.
Rename the SARIF rules so the vocabulary lines up: renamed → jscpd/renamed-code, similar → jscpd/similar-code (currently jscpd/similar-code is the renamed rule and jscpd/near-miss-code the similar one). Unreleased, so cheap to change now.
Fixture README inaccuracies in fixtures/type3-demo/README.md: the shared boundary is 3 tokens (not one ;), the wide-gap second half starts on line 9's const payment, the JSON similarity for wide-gap is 0.905.
Follow-ups on the gap-merge pass shipped in #1020 (
merge_gapped_clonesinrust/crates/cpd-core/src/detect.rs), from the review of that PR. None of them affect default runs (the pass is skipped at--max-gap-lines 0).similar ~0.05result. Add a token bound and/or a minimum similarity for merged clones.--max-gap-lines Ntolerates N+2 inserted lines. Exact windows already reach into the inserted region on both sides, so a 3-line insertion measures as a 1-line gap. Measure from the matched lines, or document the real rule.continuation()accepts any overlap as long as start and end increase;A B CvsA B junk B Cmerges into one~0.70clone that reuses A's tokens for two B regions. Cap the tolerated overlap (one boundary token) or subtract it correctly on both sides.js/tspair; dropformatfrom the pair key or normalize it in both passes.--thresholdgets stricter when merging is enabled. Count matched lines only, or exclude gaps from the percentage.kind: renamed. Decide precedence (similarsubsumesrenamed, or keeprenamedwhen every part was renamed) and document it.renamed→jscpd/renamed-code,similar→jscpd/similar-code(currentlyjscpd/similar-codeis the renamed rule andjscpd/near-miss-codethe similar one). Unreleased, so cheap to change now.fixtures/type3-demo/README.md: the shared boundary is 3 tokens (not one;), the wide-gap second half starts on line 9'sconst payment, the JSON similarity for wide-gap is 0.905.