[core] Prune manifest blocks with sidecars during scans

[leaves12138]

Purpose

Wire manifest sidecars into Java scans, following the format (#9845), lifecycle (#9886), and writer (#9889) changes. Extracted from #9743 after rebasing the tracking PR onto current master.

• Reuse manifest.sidecar.enabled; no additional read option is introduced.
• Select physical Avro blocks using partition, row-ID and conservative bucket predicates before reading manifest entries. Empty selections skip the manifest body entirely.
• Preserve normal entry filters, statistics filtering, conversion and ADD/DELETE reconciliation.
• Follow only explicit sidecar references. Missing, corrupt or unsupported sidecars fall back to ordinary manifest reads. Disabled sidecars and scans without relevant filters skip sidecar access.
• Share the manifest byte cache with sidecar and selected-block reads, keeping block keys separate from whole-manifest entry keys. Partial results cannot populate the whole-manifest entry cache.
• Preserve the existing ManifestFile.read overloads for callers that do not supply a block selection.

No file-format, writer, lifecycle or Python changes are included.

Tests

Java 8: 195 targeted tests passed with Checkstyle and Spotless enabled.

Tests cover row-ID, partition-only and bucket-only pruning; nullable coverage; real seek/byte-read accounting; repeated partial/all-block cache reads; explicit/custom references; disabled and unfiltered reads; missing/corrupt sidecar fallback while preserving deletes and overlapping row-ID groups; and existing data-evolution, bucket-filter, primary-key and global-index scans.

mvn -B -ntp -nsu -pl paimon-core -am -DwildcardSuites=none -DfailIfNoTests=false -Dflink.forkCount=2 \
  -Dtest=ManifestFileTest,ManifestSidecarTest,ManifestSidecarWriteTest,ManifestBlockIndexTest,DataEvolutionFileStoreScanTest,BucketFilterScanTest,KeyValueFileStoreScanTest,DataEvolutionBatchScanTest test

[JingsongLi]

[P2] Reuse an existing whole-manifest cache hit before reading selected blocks

With manifest.sidecar.enabled=true and sink.writer-coordinator.prefetch-manifests=true, TableWriteCoordinator.refresh() warms the whole-manifest entry cache through an unfiltered scan. Its subsequent restore requests use withPartitionBucket(...), so sidecar selection becomes non-null and this condition bypasses those already-cached entries. The selected-block reader only checks BlockCacheKey, whereas prefetch populated the Path key, so the first access to each uncached block reads the manifest again—even when every block is selected. This defeats the existing prefetch behavior and adds storage I/O and Avro decoding during writer recovery.

I reproduced this with a recording FileIO: an unfiltered scan populated ManifestEntrySegments, then a bucket-filtered scan reopened both the sidecar and the manifest despite the complete entry cache still being present.

Could we reuse an existing whole-manifest cache hit first and use selected-block reads only on a miss? Partial results should still never populate the whole-manifest cache.