Skip to content

ci-and-hooks.md

Latest commit

 

History

History
369 lines (276 loc) · 10.7 KB

ci-and-hooks.md

File metadata and controls

369 lines (276 loc) · 10.7 KB
Raw

CI & Pre-Commit Hooks

jscpd can enforce duplication thresholds in CI pipelines and as a local pre-commit hook — catching copy/pasted code before it reaches the main branch.

GitHub Action

The jscpd-copy-paste-detector GitHub Action runs jscpd in your CI workflow. It installs the Rust engine, runs detection, uploads SARIF to GitHub Code Scanning, and optionally uploads the report as an artifact.

Basic Usage

name: Duplication Check

on: [push, pull_request]

jobs:
  jscpd:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: kucherenko/jscpd@v5

This scans the entire repository with default settings and uploads SARIF results to GitHub Code Scanning.

@v5 is a moving tag that follows the latest 5.x release of the action. Pin to an exact tag (e.g. kucherenko/jscpd@v5.1.2) or a commit SHA if you need reproducible runs.

Fail on Threshold

Set threshold to fail the build when duplication exceeds a percentage:

- uses: kucherenko/jscpd@v5
  with:
    threshold: 5

The workflow fails if more than 5% of the code is duplicated.

Action Inputs

Input Description Default
path Paths to scan (space-separated) .
config Path to .jscpd.json config file —
min-tokens Minimum tokens for a clone 50
min-lines Minimum lines for a clone 5
max-lines Maximum lines per block —
mode Detection mode: mild, weak, strict mild
format Comma-separated formats to check —
ignore Comma-separated glob patterns to ignore —
ignore-pattern Comma-separated regex patterns to skip —
reporters Comma-separated reporters console
output Output directory for file reporters report
threshold Max duplication % before exit 1 —
baseline Path to a clone baseline file; clones absent from it are reported as new —
update-baseline Rewrite the baseline file from the current run (requires baseline) false
fail-on-new-clones Exit 1 on new clones (true, or an integer N to allow up to N) —
baseline-from-ref Compare against an ephemeral baseline built from a git ref (needs fetch-depth: 0) —
blame Enrich clones with git blame data false
exit-code Exit with code when duplicates found (true or integer) —
pattern Glob pattern for file search —
max-size Skip files larger than SIZE —
skip-local Skip clones in same directory false
ignore-case Ignore case of symbols (experimental) false
ignore-identifiers Treat all identifiers as equal (Type-2 clones) false
ignore-literals Treat all string and numeric literals as equal false
ignore-annotations Skip annotations and decorators (@Name, @Name(...)) false
max-gap-lines Merge clones separated by at most N unmatched lines into one near-miss clone (Type-3); 0 disables 0
similarity Report JS/TS function pairs whose syntax-tree similarity reaches this ratio, a number in (0, 1]; 1 means exact matches only 1
follow-symlinks Follow symbolic links false
no-gitignore Don't respect .gitignore files false
absolute Use absolute paths in reports false
formats-exts Custom format-to-extension mappings —
formats-names Custom format-to-filename mappings —
version jscpd version to install latest
install-prefix Installation directory for the binary —
skip-install Skip installation (binary already present) false
extra-args Additional arguments passed to jscpd —
upload-report Upload report directory as artifact false
upload-sarif Upload SARIF to GitHub Code Scanning true

Action Outputs

Output Description
duplication-percentage Percentage of duplicated code found
clones-found Number of clone pairs found
duplicated-lines Number of duplicated lines
total-lines Total lines scanned
files-count Number of source files scanned
report-path Path to the output directory
sarif-path Path to the SARIF report file
exit-code Exit code from jscpd

Examples

Scan specific directories with threshold

- uses: kucherenko/jscpd@v5
  with:
    path: src/lib src/utils
    threshold: 3
    ignore: "**/*.test.*,**/*.spec.*"

Use a config file

- uses: kucherenko/jscpd@v5
  with:
    config: .jscpd.json
    upload-report: true

Multi-reporter with artifact upload

- uses: kucherenko/jscpd@v5
  with:
    reporters: console,json,html,sarif
    output: jscpd-report
    upload-report: true

Pin a specific version

- uses: kucherenko/jscpd@v5
  with:
    version: "5.1.2"

Skip install (binary already in image)

- uses: kucherenko/jscpd@v5
  with:
    skip-install: true

Fail only on new duplication (pull requests)

- uses: actions/checkout@v4
  with:
    fetch-depth: 0
- uses: kucherenko/jscpd@v5
  with:
    baseline-from-ref: origin/${{ github.base_ref }}
    fail-on-new-clones: true

Existing clones are tolerated; the job fails only when the pull request introduces clones that are absent from the base branch.

Use outputs in subsequent steps

- uses: kucherenko/jscpd@v5
  id: jscpd

- name: Check results
  if: steps.jscpd.outputs.duplication-percentage > 5
  run: |
    echo "Duplication is ${{ steps.jscpd.outputs.duplication-percentage }}%"
    echo "Found ${{ steps.jscpd.outputs.clones-found }} clones"

Docker

A multi-arch image (linux/amd64, linux/arm64) built from the release binaries is published to GitHub Container Registry on every release. It contains only the static jscpd binary, so it is suitable for any CI system that can run containers:

# Scan the current directory
docker run --rm -v "$PWD:/src" ghcr.io/kucherenko/jscpd .

# Pin a version and fail over a threshold
docker run --rm -v "$PWD:/src" ghcr.io/kucherenko/jscpd:5 --threshold 5 --reporters console,silent .

# Write reports to ./report on the host
docker run --rm -v "$PWD:/src" ghcr.io/kucherenko/jscpd --reporters json,html --output report .

The working directory inside the container is /src. Tags: latest, 5, 5.1, 5.1.2 (major, minor, and exact version). The image contains only the static binary — --blame and --baseline-from-ref need git, which is not included. On Linux hosts, add --user "$(id -u):$(id -g)" so report files are owned by you rather than root.

GitLab CI example, using the Code Quality and metrics reporters:

duplication:
  image:
    name: ghcr.io/kucherenko/jscpd:5
    entrypoint: [""]
  script:
    - jscpd --reporters console,codeclimate,openmetrics --output report .
  artifacts:
    reports:
      codequality: report/gl-code-quality-report.json
      metrics: report/jscpd-metrics.txt

Pre-Commit Hook

Run jscpd before every commit to prevent duplicated code from entering the repository.

Using pre-commit framework

The pre-commit framework manages git hooks for you. After configuring the hook, it runs automatically on every git commit.

1. Install pre-commit (one time, any of these):

# pip
pip install pre-commit

# brew
brew install pre-commit

# npm (wrapper around the Python tool)
npm install -g pre-commit

2. Add the hook config to .pre-commit-config.yaml in your repo:

Option A: language: node — pre-commit installs jscpd automatically:

repos:
  - repo: local
    hooks:
      - id: jscpd
        name: jscpd - copy/paste detector
        entry: jscpd
        language: node
        additional_dependencies: ['jscpd']
        args: [--threshold, "5", --reporters, console,silent]
        pass_filenames: false
        always_run: true

Option B: language: system — jscpd must be pre-installed globally:

repos:
  - repo: local
    hooks:
      - id: jscpd
        name: jscpd - copy/paste detector
        entry: jscpd
        language: system
        args: [--threshold, "5", --reporters, console,silent]
        pass_filenames: false
        always_run: true

If using Option B, install jscpd globally first: npm install -g jscpd or cargo install jscpd.

3. Install the hook into git:

pre-commit install

That's it — jscpd now runs on every git commit. If duplication exceeds the threshold, the commit is blocked.

To run manually without committing:

pre-commit run jscpd --all-files

Using Husky

npm install -D husky
npx husky init

Add the hook:

echo 'npx jscpd --threshold 5 --reporters console,silent .' > .husky/pre-commit

Manual git hook

No extra tools required — just a shell script in .git/hooks/.

  1. Create .git/hooks/pre-commit:
#!/bin/sh
jscpd --threshold 5 --reporters console,silent .
  1. Make it executable:
chmod +x .git/hooks/pre-commit

Hooks in .git/hooks/ are not version-controlled. To share the hook with your team, store it in the repo and symlink or copy it:

Option A: Symlink from a versioned script

Store the hook logic in the repo (e.g. scripts/pre-commit), then symlink:

ln -s ../../scripts/pre-commit .git/hooks/pre-commit

Each developer runs the symlink command once after cloning.

Option B: core.hooksPath (Git 2.9+)

Point Git at a versioned hooks directory:

git config core.hooksPath .githooks

Create .githooks/pre-commit:

#!/bin/sh
jscpd --threshold 5 --reporters console,silent .
chmod +x .githooks/pre-commit

Commit .githooks/ to the repo. New contributors run the git config command once after cloning. Add it to your onboarding docs or a scripts/setup.sh:

#!/bin/sh
git config core.hooksPath .githooks

Option C: npm prepare script

Add to package.json:

{
  "scripts": {
    "prepare": "git config core.hooksPath .githooks"
  }
}

npm install (and npm ci) automatically run prepare, so the hooks path is set with no manual steps.

Option D: Makefile

.PHONY: hooks
hooks:
	git config core.hooksPath .githooks

Contributors run make hooks after cloning.

Tips

  • Use --reporters console,silent to show clone details without writing report files on every commit
  • Use --threshold to set a failure threshold — the hook exits with code 1 if exceeded
  • Use --ignore to exclude generated files, test fixtures, or vendor directories
  • Consider --format to limit detection to specific languages during the hook, with a full scan in CI