Rust pipeline

[chinyeungli]

Issues

• Closes: t-rust: Design and implement Scancode.io pipeline for a single Rust package #1767

Changes

• Added the dedicated pipeline for Rust
  • It takes a single PURL as an input (cargo type)
  • Downloads the crate sources
  • Run scans and determine if there's any license mismatch between "declared licenses" and "detected licenses"
  • Builds the crate source in a container and determines which files are actually used based on the generated .d files.
    • If the build succeeds, the source files are placed under codebase/from/ and the built artifacts under codebase/to/.
  • Identifies the project's source repo and compare if there's any differences between the source repo and source crates

Samples:

I used pkg:cargo/rand@0.7.2 as a sample.

The source crate files are put under the codebase/from

and the built files are put under codebase/to

If there is a license mismatch issue, a small red alert sign will be put next to the PURL

A comparison summary and issue info are put under the extra_data tab

The detected_codebase_license is built by collecting every detected license from the codebase and classifying each one as either a license text or not.
License text detections are grouped with OR, because a project that ships multiple license text files (COPYING, COPYING3, COPYING.LESSER) is offering those licenses as alternatives. All other detections (notices, tags, references) are grouped with AND.
The two groups are then combined with AND.

Checklist

• I have read the contributing guidelines
• I have linked an existing issue above
• I have added unit tests covering the new code
• I have reviewed and understood every line of this PR