diff --git a/ChangeLog.md b/ChangeLog.md index 900ec37a2..c5b25221b 100644 --- a/ChangeLog.md +++ b/ChangeLog.md @@ -19,6 +19,8 @@ Blob: - Fixed Stage Block responses for API version `2019-02-02` and later to return `Content-MD5` when the request supplies `Content-MD5`, matching Azure Storage while preserving CRC64 responses for requests without MD5. (issue #2394) - Bumped the default Blob service API version to `2026-10-06`, and added `2026-10-06` to the supported Blob service API versions. - Fixed Blob GET and HEAD requests for nonexistent blobs with `If-Match` so they now return `BlobNotFound` instead of `ConditionNotMet`, matching Azure Storage. (issue #2751) +- Accept a service SAS or user delegation SAS scoped to a blob version (signed resource `bv`), which Azurite previously refused, so a version can be read, or copied from with Copy Blob and Put Blob From URL, by a SAS URL for it. Such a SAS reaches only the version it names: it reads the version, its properties and its tags, sets its tier and tags, and deletes it, which takes the Delete version permission (`x`), and anything that would act on the base blob is refused with `AuthorizationResourceTypeMismatch`. A user delegation SAS for a blob snapshot (`bs`) is also accepted now: it signs the snapshot time, which Azurite left out of the string to sign. A snapshot SAS is likewise confined to the snapshot it names -- its content, properties, tags, block list and page ranges -- which a writable one could get past by setting the base blob's metadata, and a snapshot or version SAS is refused with a signed version before `2018-11-09`, whose signature does not bind the snapshot or version. +- Fixed Get Block List on a blob snapshot returning the base blob's uncommitted blocks with `blocklisttype=all` or `uncommitted`. A snapshot has no uncommitted blocks. - Added opt-in, per-account Blob Versioning configured with `--accountConfigFilePath` or `--accountConfigAsJson`; Azurite preserves previous versions on supported blob writes and supports listing, reading, restoring, and deleting specific versions. - List Blobs continuation tokens are now opaque, matching Azure Storage. Tokens are base64url encoded, versioned JSON carrying the `[name, timestamp, recordId]` tuple used to sort records, filter records after the marker, and build the next marker, which makes paging deterministic when several records share a blob name. Tokens issued by previous versions of Azurite (plain blob names) are still accepted. - Reject cross-type Put Blob and Copy Blob replacements while a versioned blob retains a current blob or previous versions. diff --git a/README.md b/README.md index 39a5121fa..6405e117d 100644 --- a/README.md +++ b/README.md @@ -646,7 +646,7 @@ azurite --accountConfigAsJson "{\"isBlobVersioningEnabled\":true}" #### Limitations - Soft delete and blob expiration are not integrated with versioning. -- SAS URIs for specific versions and version-level immutability policies are not supported. +- Version-level immutability policies are not supported. - `include=deletedwithversions` on List Blobs is accepted but ignored. Azurite does not return `HasVersionsOnly` entries for deleted blobs that still have previous versions; use `include=versions` to list them. - `Put Page` and `Append Block` do not create versions. Other supported writes, including `Put Blob`, `Put Block List`, `Set Blob Metadata`, and `Copy Blob`, do. - Blob batch sub-requests use the versioning setting of the account the batch request was sent to. Azure always runs a batch's sub-requests against that account. With path-style URLs, Azurite also accepts a sub-request whose path names a different account, and runs it there with versioning treated as disabled. For example, a batch sent to `devstoreaccount1` that deletes a blob in `devstoreaccount2` permanently deletes a blob that has no version ID yet, even if `devstoreaccount2` has versioning enabled. Send each batch to the account that owns its blobs. @@ -1207,7 +1207,7 @@ Detailed support matrix: - Shared Access Signature Service Level (Not support response header override in service SAS) - Container Public Access - Blob Tags (preview) - - Blob versioning (Only in LokiDb instances of Azurite, which is the default. Does not support SAS URIs targeting specific blob versions) + - Blob versioning (Only in LokiDb instances of Azurite, which is the default) - CRC-64/NVME transactional checksums (`x-ms-content-crc64`) - Supported REST APIs @@ -1257,7 +1257,6 @@ Detailed support matrix: - Encryption Scope - Get Page Ranges Continuation Token - Blob Immutability Policy and Legal Hold - - SAS URIs for Blob Versions - List Blobs `include=deletedwithversions` (accepted but ignored) Latest version supports **2026-06-06** API version **queue** service. diff --git a/docs/designs/2025-12-blob-versioning.md b/docs/designs/2025-12-blob-versioning.md index 3f82f8008..d586e63f1 100644 --- a/docs/designs/2025-12-blob-versioning.md +++ b/docs/designs/2025-12-blob-versioning.md @@ -508,7 +508,7 @@ flowchart TD Versioning does not bypass or replace Azurite authentication. Requests still pass through the existing public access, Shared Key, account SAS, blob SAS, and optional OAuth authenticators before reaching handlers. -The current limitation is that Azurite does not generate or validate SAS URIs scoped to a specific blob version. Normal authenticated version operations continue through the existing authorization pipeline. +A service SAS or user delegation SAS scoped to one blob version (signed resource `bv`) signs the version ID in the field that a snapshot SAS signs the snapshot time in. Azurite reads that value from the request's `versionid` query parameter, so such a SAS authorizes requests for the version it names and no other. It is accepted only for the operations that act on that version -- Get Blob, Get Blob Properties, Get and Set Blob Tags, Set Blob Tier, and Delete Blob -- because every other operation would act on the base blob, which a version SAS does not grant; those answer `AuthorizationResourceTypeMismatch`. Deleting the version takes the Delete version permission (`x`), as Azure requires, rather than Delete (`d`). A version SAS signed in a format older than `2018-11-09` is refused, since those formats sign neither the resource nor the version ID. ## Error and validation behavior @@ -555,7 +555,6 @@ The implementation preserves existing lease and conditional-request checks and a - Versioning works only with LokiJS metadata, not `AZURITE_DB` SQL metadata. - Soft delete and blob expiration are not integrated with versioning. -- Version-specific SAS URIs are not supported. - Version-level immutability policies are not supported. - List Blobs `include=deletedwithversions` is accepted but ignored. Azure returns one `HasVersionsOnly=true` entry per deleted blob that still has versions; Azurite returns nothing for such blobs unless `include=versions` is also requested. - Azurite's timestamps use JavaScript time plus a local per-blob counter. They reproduce the external seven-digit shape, but not Azure's internal clock implementation. @@ -619,6 +618,5 @@ The SQL metadata schema and query paths require separate migrations and parity v - Add SQL metadata schema and query support for versions. - Integrate versions with soft delete and lifecycle expiration. -- Add version-scoped SAS support. - Add version-level immutability policies. - Add automated live-Azure parity infrastructure for account-level setting transitions. diff --git a/src/blob/authentication/BlobSASAuthenticator.ts b/src/blob/authentication/BlobSASAuthenticator.ts index db58186e5..16c110324 100644 --- a/src/blob/authentication/BlobSASAuthenticator.ts +++ b/src/blob/authentication/BlobSASAuthenticator.ts @@ -27,6 +27,8 @@ import { } from "./IBlobSASSignatureValues"; import { OPERATION_BLOB_SAS_BLOB_PERMISSIONS, + OPERATION_BLOB_SAS_BLOB_SNAPSHOT_PERMISSIONS, + OPERATION_BLOB_SAS_BLOB_VERSION_PERMISSIONS, OPERATION_BLOB_SAS_CONTAINER_PERMISSIONS } from "./OperationBlobSASPermission"; @@ -105,7 +107,8 @@ export default class BlobSASAuthenticator implements IAuthenticator { if ( resource !== BlobSASResourceType.Container && resource !== BlobSASResourceType.Blob && - resource !== BlobSASResourceType.BlobSnapshot + resource !== BlobSASResourceType.BlobSnapshot && + resource !== BlobSASResourceType.BlobVersion ) { this.logger.debug( // tslint:disable-next-line:max-line-length @@ -134,6 +137,22 @@ export default class BlobSASAuthenticator implements IAuthenticator { return undefined; } + // The bs and bv resources date from 2018-11-09, the first signing format + // to sign the resource and the snapshot time or version ID. An older + // format signs neither, so it could not bind the SAS to the snapshot or + // version it names. + if ( + (resource === BlobSASResourceType.BlobSnapshot || + resource === BlobSASResourceType.BlobVersion) && + values.version < "2018-11-09" + ) { + this.logger.info( + `BlobSASAuthenticator:validate() Signed resource type ${resource} needs version 2018-11-09 or later, not ${values.version}. Skip blob service SAS validation.`, + context.contextId + ); + return undefined; + } + this.logger.debug( `BlobSASAuthenticator:validate() Successfully got valid blob service SAS values from request. ${JSON.stringify( values @@ -388,10 +407,17 @@ export default class BlobSASAuthenticator implements IAuthenticator { AUTHENTICATION_BEARERTOKEN_REQUIRED); } + const selectsBlob = + resource === BlobSASResourceType.BlobSnapshot || + resource === BlobSASResourceType.BlobVersion; const blobSASPermission = - resource === BlobSASResourceType.Blob - ? OPERATION_BLOB_SAS_BLOB_PERMISSIONS.get(operation) - : OPERATION_BLOB_SAS_CONTAINER_PERMISSIONS.get(operation); + resource === BlobSASResourceType.BlobVersion + ? OPERATION_BLOB_SAS_BLOB_VERSION_PERMISSIONS.get(operation) + : resource === BlobSASResourceType.BlobSnapshot + ? OPERATION_BLOB_SAS_BLOB_SNAPSHOT_PERMISSIONS.get(operation) + : resource === BlobSASResourceType.Blob + ? OPERATION_BLOB_SAS_BLOB_PERMISSIONS.get(operation) + : OPERATION_BLOB_SAS_CONTAINER_PERMISSIONS.get(operation); this.logger.debug( `BlobSASAuthenticator:validate() Got permission requirements for operation ${ @@ -399,6 +425,13 @@ export default class BlobSASAuthenticator implements IAuthenticator { } - ${JSON.stringify(blobSASPermission)}`, context.contextId ); + if (blobSASPermission === undefined && selectsBlob) { + // An operation that does not act on the snapshot or version the SAS + // names would act on the base blob, which such a SAS does not grant. + throw StorageErrorFactory.getAuthorizationResourceTypeMismatch( + context.contextId! + ); + } if (blobSASPermission === undefined) { throw new Error( // tslint:disable-next-line:max-line-length @@ -499,7 +532,15 @@ export default class BlobSASAuthenticator implements IAuthenticator { const contentLanguage = req.getQuery("rscl"); const contentType = req.getQuery("rsct"); const signedResource = this.decodeIfExist(req.getQuery("sr")); - const snapshot = this.decodeIfExist(req.getQuery("snapshot")); + // A SAS for a blob version signs its version ID in the field that a + // SAS for a snapshot signs its snapshot time in. + const snapshot = this.decodeIfExist( + req.getQuery( + signedResource === BlobSASResourceType.BlobVersion + ? "versionid" + : "snapshot" + ) + ); const encryptionScope = this.decodeIfExist(req.getQuery("ses")); const signedObjectId = this.decodeIfExist(req.getQuery("skoid")); const signedTenantId = this.decodeIfExist(req.getQuery("sktid")); diff --git a/src/blob/authentication/BlobSASResourceType.ts b/src/blob/authentication/BlobSASResourceType.ts index 1fb709d1e..548c0d4d6 100644 --- a/src/blob/authentication/BlobSASResourceType.ts +++ b/src/blob/authentication/BlobSASResourceType.ts @@ -1,5 +1,6 @@ export enum BlobSASResourceType { Container = "c", Blob = "b", - BlobSnapshot = "bs" + BlobSnapshot = "bs", + BlobVersion = "bv" } diff --git a/src/blob/authentication/IBlobSASSignatureValues.ts b/src/blob/authentication/IBlobSASSignatureValues.ts index abedd0018..bec19e121 100644 --- a/src/blob/authentication/IBlobSASSignatureValues.ts +++ b/src/blob/authentication/IBlobSASSignatureValues.ts @@ -150,7 +150,9 @@ export interface IBlobSASSignatureValues { signedResource?: string; /** - * Optional. Specifies the snapshot time. Required for version 2018-11-09 and later. + * Optional. Specifies the snapshot time, or for a SAS for a blob version + * (signed resource bv) the version ID. Required for version 2018-11-09 and + * later. * * @type {string} * @memberof IBlobSASSignatureValues @@ -365,8 +367,7 @@ function generateBlobSASSignature20201206( getCanonicalName( accountName, blobSASSignatureValues.containerName, - resource === BlobSASResourceType.Blob || - resource === BlobSASResourceType.BlobSnapshot + isBlobScoped(resource) ? blobSASSignatureValues.blobName : "" ), @@ -438,8 +439,7 @@ function generateBlobSASSignature20181109( getCanonicalName( accountName, blobSASSignatureValues.containerName, - resource === BlobSASResourceType.Blob || - resource === BlobSASResourceType.BlobSnapshot + isBlobScoped(resource) ? blobSASSignatureValues.blobName : "" ), @@ -507,7 +507,7 @@ function generateBlobSASSignature20150405( getCanonicalName( accountName, blobSASSignatureValues.containerName, - resource === BlobSASResourceType.Blob + isBlobScoped(resource) ? blobSASSignatureValues.blobName : "" ), @@ -587,7 +587,7 @@ function generateBlobSASSignatureUDK20181109( blobSASSignatureValues.protocol ? blobSASSignatureValues.protocol : "", blobSASSignatureValues.version, resource, - undefined, // blob version timestamp, + signedSnapshotTime(blobSASSignatureValues, resource), blobSASSignatureValues.cacheControl, blobSASSignatureValues.contentDisposition, blobSASSignatureValues.contentEncoding, @@ -651,7 +651,7 @@ function generateBlobSASSignatureWithUDK20200210( blobSASSignatureValues.protocol ? blobSASSignatureValues.protocol : "", blobSASSignatureValues.version, resource, - undefined, // blob versiontimestamp, + signedSnapshotTime(blobSASSignatureValues, resource), blobSASSignatureValues.cacheControl, blobSASSignatureValues.contentDisposition, blobSASSignatureValues.contentEncoding, @@ -696,7 +696,7 @@ function generateBlobSASBlobSASSignatureWithUDK20201206( getCanonicalName( accountName, blobSASSignatureValues.containerName, - resource === BlobSASResourceType.Blob + isBlobScoped(resource) ? blobSASSignatureValues.blobName : "" ), @@ -717,7 +717,7 @@ function generateBlobSASBlobSASSignatureWithUDK20201206( blobSASSignatureValues.protocol ? blobSASSignatureValues.protocol : "", blobSASSignatureValues.version, resource, - undefined, // blob version timestamp, + signedSnapshotTime(blobSASSignatureValues, resource), blobSASSignatureValues.encryptionScope, blobSASSignatureValues.cacheControl, blobSASSignatureValues.contentDisposition, @@ -765,8 +765,7 @@ function generateBlobSASBlobSASSignatureWithUDK20260406( getCanonicalName( accountName, blobSASSignatureValues.containerName, - resource === BlobSASResourceType.Blob || - resource === BlobSASResourceType.BlobSnapshot + isBlobScoped(resource) ? blobSASSignatureValues.blobName : "" ), @@ -789,7 +788,7 @@ function generateBlobSASBlobSASSignatureWithUDK20260406( blobSASSignatureValues.protocol ? blobSASSignatureValues.protocol : "", blobSASSignatureValues.version, resource, - undefined, // blob version timestamp, + signedSnapshotTime(blobSASSignatureValues, resource), blobSASSignatureValues.encryptionScope, undefined, // requestHeaders (srh), added in 2026-04-06 undefined, // requestQueryParameters (srq), added in 2026-04-06 @@ -837,7 +836,7 @@ function generateBlobSASBlobSASSignatureWithUDK20250705( getCanonicalName( accountName, blobSASSignatureValues.containerName, - resource === BlobSASResourceType.Blob + isBlobScoped(resource) ? blobSASSignatureValues.blobName : "" ), @@ -860,7 +859,7 @@ function generateBlobSASBlobSASSignatureWithUDK20250705( blobSASSignatureValues.protocol ? blobSASSignatureValues.protocol : "", blobSASSignatureValues.version, resource, - undefined, // blob version timestamp, + signedSnapshotTime(blobSASSignatureValues, resource), blobSASSignatureValues.encryptionScope, blobSASSignatureValues.cacheControl, blobSASSignatureValues.contentDisposition, @@ -873,6 +872,34 @@ function generateBlobSASBlobSASSignatureWithUDK20250705( return [signature, stringToSign]; } +/** + * Whether a SAS for this resource names a blob, and so signs the blob's name + * in its canonicalized resource: a blob, a snapshot of one or a version of + * one, as against a container. + */ +function isBlobScoped(resource: BlobSASResourceType): boolean { + return ( + resource === BlobSASResourceType.Blob || + resource === BlobSASResourceType.BlobSnapshot || + resource === BlobSASResourceType.BlobVersion + ); +} + +/** + * The signedSnapshotTime field of a user delegation SAS: the snapshot time of + * a SAS for a snapshot, the version ID of a SAS for a version, and empty + * otherwise. + */ +function signedSnapshotTime( + blobSASSignatureValues: IBlobSASSignatureValues, + resource: BlobSASResourceType +): string | undefined { + return resource === BlobSASResourceType.BlobSnapshot || + resource === BlobSASResourceType.BlobVersion + ? blobSASSignatureValues.snapshot + : undefined; +} + function getCanonicalName( accountName: string, containerName: string, diff --git a/src/blob/authentication/OperationBlobSASPermission.ts b/src/blob/authentication/OperationBlobSASPermission.ts index 3a27e5e19..149fefc7e 100644 --- a/src/blob/authentication/OperationBlobSASPermission.ts +++ b/src/blob/authentication/OperationBlobSASPermission.ts @@ -291,6 +291,80 @@ OPERATION_BLOB_SAS_BLOB_PERMISSIONS.set( new OperationBlobSASPermission(BlobSASPermission.Tag) ); +// Blob Service SAS Permissions for a blob version (signed resource bv). +// Only the operations whose handlers act on the version that the request's +// versionid names are listed: a version SAS grants access to that version and +// not to the base blob, and every other operation would act on the base blob. +// Deleting a version takes the Delete version permission, x, rather than d. +// See https://learn.microsoft.com/en-us/azure/storage/blobs/versioning-overview#shared-access-signature-sas-parameters +export const OPERATION_BLOB_SAS_BLOB_VERSION_PERMISSIONS = new Map< + Operation, + OperationBlobSASPermission +>(); + +OPERATION_BLOB_SAS_BLOB_VERSION_PERMISSIONS.set( + Operation.Blob_Download, + new OperationBlobSASPermission(BlobSASPermission.Read) +); +OPERATION_BLOB_SAS_BLOB_VERSION_PERMISSIONS.set( + Operation.Blob_GetProperties, + new OperationBlobSASPermission(BlobSASPermission.Read) +); +OPERATION_BLOB_SAS_BLOB_VERSION_PERMISSIONS.set( + Operation.Blob_Delete, + new OperationBlobSASPermission(BlobSASPermission.DeleteVersion) +); +OPERATION_BLOB_SAS_BLOB_VERSION_PERMISSIONS.set( + Operation.Blob_SetTier, + new OperationBlobSASPermission(BlobSASPermission.Write) +); +OPERATION_BLOB_SAS_BLOB_VERSION_PERMISSIONS.set( + Operation.Blob_SetTags, + new OperationBlobSASPermission(BlobSASPermission.Tag) +); +OPERATION_BLOB_SAS_BLOB_VERSION_PERMISSIONS.set( + Operation.Blob_GetTags, + new OperationBlobSASPermission(BlobSASPermission.Tag) +); + +// Blob Service SAS Permissions for a blob snapshot (signed resource bs), on +// the same reasoning: only the operations whose handlers act on the snapshot +// that the request's snapshot parameter names. Set Blob Tier is left out, as +// its handler acts on the base blob whatever snapshot is named. +export const OPERATION_BLOB_SAS_BLOB_SNAPSHOT_PERMISSIONS = new Map< + Operation, + OperationBlobSASPermission +>(); + +OPERATION_BLOB_SAS_BLOB_SNAPSHOT_PERMISSIONS.set( + Operation.Blob_Download, + new OperationBlobSASPermission(BlobSASPermission.Read) +); +OPERATION_BLOB_SAS_BLOB_SNAPSHOT_PERMISSIONS.set( + Operation.Blob_GetProperties, + new OperationBlobSASPermission(BlobSASPermission.Read) +); +OPERATION_BLOB_SAS_BLOB_SNAPSHOT_PERMISSIONS.set( + Operation.Blob_Delete, + new OperationBlobSASPermission(BlobSASPermission.Delete) +); +OPERATION_BLOB_SAS_BLOB_SNAPSHOT_PERMISSIONS.set( + Operation.BlockBlob_GetBlockList, + new OperationBlobSASPermission(BlobSASPermission.Read) +); +OPERATION_BLOB_SAS_BLOB_SNAPSHOT_PERMISSIONS.set( + Operation.PageBlob_GetPageRanges, + new OperationBlobSASPermission(BlobSASPermission.Read) +); +OPERATION_BLOB_SAS_BLOB_SNAPSHOT_PERMISSIONS.set( + Operation.Blob_SetTags, + new OperationBlobSASPermission(BlobSASPermission.Tag) +); +OPERATION_BLOB_SAS_BLOB_SNAPSHOT_PERMISSIONS.set( + Operation.Blob_GetTags, + new OperationBlobSASPermission(BlobSASPermission.Tag) +); + // Blob Service SAS Permissions for container level export const OPERATION_BLOB_SAS_CONTAINER_PERMISSIONS = new Map< Operation, diff --git a/src/blob/persistence/LokiBlobMetadataStore.ts b/src/blob/persistence/LokiBlobMetadataStore.ts index f6d0eb540..c6d8713bf 100644 --- a/src/blob/persistence/LokiBlobMetadataStore.ts +++ b/src/blob/persistence/LokiBlobMetadataStore.ts @@ -3147,7 +3147,9 @@ export default class LokiBlobMetadataStore res.committedBlocks = doc.committedBlocksInOrder; } - if (isCommitted !== true) { + // Uncommitted blocks are staged on the base blob, so a snapshot or a + // version has none, though they are stored under the same blob name. + if (isCommitted !== true && snapshot === "" && versionId === "") { const blockColl = this.db.getCollection(this.BLOCKS_COLLECTION); const blocks = await blockColl .chain() diff --git a/src/blob/persistence/SqlBlobMetadataStore.ts b/src/blob/persistence/SqlBlobMetadataStore.ts index a6cfc5b00..6bb9aa394 100644 --- a/src/blob/persistence/SqlBlobMetadataStore.ts +++ b/src/blob/persistence/SqlBlobMetadataStore.ts @@ -1649,7 +1649,9 @@ export default class SqlBlobMetadataStore implements IBlobMetadataStore { res.committedBlocks = blobModel.committedBlocksInOrder || []; } - if (isCommitted !== true) { + // Uncommitted blocks are staged on the base blob, so a snapshot has + // none, though they are stored under the same blob name. + if (isCommitted !== true && snapshot === "") { const blocks = await BlocksModel.findAll({ attributes: ["blockName", "size"], where: { diff --git a/tests/blob/apis/blockblob.test.ts b/tests/blob/apis/blockblob.test.ts index 348e568d8..9ce56bf7f 100644 --- a/tests/blob/apis/blockblob.test.ts +++ b/tests/blob/apis/blockblob.test.ts @@ -758,6 +758,29 @@ describe("BlockBlobAPIs", () => { assert.equal(await bodyToString(result, content.length), content); }); + // Blocks staged on the base blob are not the snapshot's: a snapshot is a + // committed blob and has no uncommitted blocks. + it("getBlockList on a snapshot lists no uncommitted blocks @loki @sql", async () => { + await blockBlobClient.stageBlock(base64encode("1"), "a", 1); + await blockBlobClient.commitBlockList([base64encode("1")]); + const snapshot = (await blockBlobClient.createSnapshot()).snapshot!; + await blockBlobClient.stageBlock(base64encode("2"), "b", 1); + + const base = await blockBlobClient.getBlockList("all"); + assert.deepStrictEqual( + base.uncommittedBlocks!.map((block) => block.name), + [base64encode("2")] + ); + const snapshotList = await blockBlobClient + .withSnapshot(snapshot) + .getBlockList("all"); + assert.deepStrictEqual( + snapshotList.committedBlocks!.map((block) => block.name), + [base64encode("1")] + ); + assert.deepStrictEqual(snapshotList.uncommittedBlocks, []); + }); + it("stageBlockFromURL with matching sourceContentMD5 @loki @sql", async () => { const content = "HelloWorldFromSourceBlob"; const sourceClient = containerClient.getBlockBlobClient( diff --git a/tests/blob/apis/blockblob.versioning.test.ts b/tests/blob/apis/blockblob.versioning.test.ts index 3122f7425..ef5a32cb4 100644 --- a/tests/blob/apis/blockblob.versioning.test.ts +++ b/tests/blob/apis/blockblob.versioning.test.ts @@ -1,8 +1,14 @@ import { + AnonymousCredential, + BlobClient, BlobSASPermissions, + BlockBlobClient, + PageBlobClient, StorageSharedKeyCredential, BlobServiceClient, + generateBlobSASQueryParameters, newPipeline, + RestError, Tags } from "@azure/storage-blob"; import assert = require("assert"); @@ -14,11 +20,15 @@ import { bodyToString, EMULATOR_ACCOUNT_KEY, EMULATOR_ACCOUNT_NAME, + generateJWTToken, getUniqueName, listBlobVersions, sleep } from "../../testutils"; +import { SimpleTokenCredential } from "../../simpleTokenCredential"; import { parseDateFromAssumedString } from "../../../src/blob/utils/utils"; +import { BlobSASResourceType } from "../../../src/blob/authentication/BlobSASResourceType"; +import { generateBlobSASSignature } from "../../../src/blob/authentication/IBlobSASSignatureValues"; import { AccountModel } from "../../../src/common/account/AccountModel"; import LokiAccountModelStore from "../../../src/common/account/LokiAccountModelStore"; @@ -786,6 +796,287 @@ describe("BlockBlobVersioningAPIs @loki", () => { assert.strictEqual(properties.isCurrentVersion, true); }); + // A SAS for a version (signed resource bv) signs the version ID where a SAS + // for a snapshot signs the snapshot time. + it("a SAS for a version reads that version @loki", async () => { + const first = await blockBlobClient.upload("v1", 2); + await blockBlobClient.upload("v2", 2); + const sasUrl = await blockBlobClient + .withVersion(first.versionId!) + .generateSasUrl({ + permissions: BlobSASPermissions.parse("r"), + expiresOn: new Date(Date.now() + 60 * 60 * 1000) + }); + assert.strictEqual(new URL(sasUrl).searchParams.get("sr"), "bv"); + + const sasClient = new BlobClient( + sasUrl, + newPipeline(new AnonymousCredential(), { retryOptions: { maxTries: 1 } }) + ); + const properties = await sasClient.getProperties(); + assert.strictEqual(properties.versionId, first.versionId); + assert.strictEqual(await bodyToString(await sasClient.download(), 2), "v1"); + }); + + it("a SAS for a version reads no other version @loki", async () => { + const first = await blockBlobClient.upload("v1", 2); + const second = await blockBlobClient.upload("v2", 2); + const sasUrl = await blockBlobClient + .withVersion(first.versionId!) + .generateSasUrl({ + permissions: BlobSASPermissions.parse("r"), + expiresOn: new Date(Date.now() + 60 * 60 * 1000) + }); + + const otherVersion = new URL(sasUrl); + otherVersion.searchParams.set("versionid", second.versionId!); + const currentBlob = new URL(sasUrl); + currentBlob.searchParams.delete("versionid"); + for (const url of [otherVersion, currentBlob]) { + const sasClient = new BlobClient( + url.toString(), + newPipeline(new AnonymousCredential(), { retryOptions: { maxTries: 1 } }) + ); + await assert.rejects( + sasClient.download(), + (err: RestError) => err.statusCode === 403 + ); + } + }); + + // A version SAS grants the version it names and not the base blob, so an + // operation that ignores versionid, and would act on the base blob, is + // refused whatever the permissions. + it("a SAS for a version does not reach the base blob @loki", async () => { + const first = await blockBlobClient.upload("v1", 2); + await blockBlobClient.upload("v2", 2); + const sasUrl = await blockBlobClient + .withVersion(first.versionId!) + .generateSasUrl({ + permissions: BlobSASPermissions.parse("racwt"), + expiresOn: new Date(Date.now() + 60 * 60 * 1000) + }); + const sasClient = new BlockBlobClient( + sasUrl, + newPipeline(new AnonymousCredential(), { retryOptions: { maxTries: 1 } }) + ); + + for (const request of [ + () => sasClient.setMetadata({ key: "value" }), + () => sasClient.upload("v3", 2), + () => sasClient.getBlockList("committed") + ]) { + await assert.rejects( + request(), + (err: RestError) => + err.statusCode === 403 && + err.code === "AuthorizationResourceTypeMismatch" + ); + } + + const current = await blockBlobClient.getProperties(); + assert.deepStrictEqual(current.metadata, {}); + assert.strictEqual( + await bodyToString(await blockBlobClient.download(), 2), + "v2" + ); + }); + + // Deleting a version takes the Delete version permission, x; d alone, which + // deletes a base blob, does not. + it("a SAS for a version deletes it with x and not d @loki", async () => { + const first = await blockBlobClient.upload("v1", 2); + const second = await blockBlobClient.upload("v2", 2); + const sasClientWith = async (permissions: string) => + new BlobClient( + await blockBlobClient.withVersion(first.versionId!).generateSasUrl({ + permissions: BlobSASPermissions.parse(permissions), + expiresOn: new Date(Date.now() + 60 * 60 * 1000) + }), + newPipeline(new AnonymousCredential(), { retryOptions: { maxTries: 1 } }) + ); + + await assert.rejects( + (await sasClientWith("d")).delete(), + (err: RestError) => + err.statusCode === 403 && + err.code === "AuthorizationPermissionMismatch" + ); + assert.deepStrictEqual( + (await listBlobVersions(containerClient, blobName)).map( + (v) => v.versionId + ), + [first.versionId, second.versionId] + ); + + await (await sasClientWith("x")).delete(); + assert.deepStrictEqual( + (await listBlobVersions(containerClient, blobName)).map( + (v) => v.versionId + ), + [second.versionId] + ); + assert.strictEqual( + await bodyToString(await blockBlobClient.download(), 2), + "v2" + ); + }); + + // Signing formats before 2018-11-09 sign neither the resource nor the + // snapshot time or version ID, so they could not bind a snapshot or version + // SAS to what it names. The same token for the blob itself is accepted, + // which shows it is otherwise well formed. + it("a SAS for a snapshot or version needs a 2018-11-09 signature @loki", async () => { + const first = await blockBlobClient.upload("v1", 2); + const snapshot = (await blockBlobClient.createSnapshot()).snapshot!; + await blockBlobClient.upload("v2", 2); + const expiryTime = + new Date(Date.now() + 60 * 60 * 1000).toISOString().split(".")[0] + "Z"; + const legacyClient = (resource: BlobSASResourceType, url: string) => { + const [sig] = generateBlobSASSignature( + { + version: "2017-11-09", + permissions: "r", + expiryTime, + containerName, + blobName, + signedResource: resource + }, + resource, + EMULATOR_ACCOUNT_NAME, + Buffer.from(EMULATOR_ACCOUNT_KEY, "base64") + ); + const sas = new URLSearchParams({ + sv: "2017-11-09", + sr: resource, + sp: "r", + se: expiryTime, + sig + }); + return new BlobClient( + `${url}${url.includes("?") ? "&" : "?"}${sas}`, + newPipeline(new AnonymousCredential(), { retryOptions: { maxTries: 1 } }) + ); + }; + + assert.strictEqual( + await bodyToString( + await legacyClient(BlobSASResourceType.Blob, blockBlobClient.url) + .download(), + 2 + ), + "v2" + ); + for (const client of [ + legacyClient( + BlobSASResourceType.BlobVersion, + blockBlobClient.withVersion(first.versionId!).url + ), + legacyClient( + BlobSASResourceType.BlobSnapshot, + blockBlobClient.withSnapshot(snapshot).url + ) + ]) { + await assert.rejects( + client.download(), + (err: RestError) => err.statusCode === 403 + ); + } + }); + + // A snapshot SAS likewise grants the snapshot and not the base blob. + it("a SAS for a snapshot does not reach the base blob @loki", async () => { + await blockBlobClient.upload("v1", 2); + const snapshot = (await blockBlobClient.createSnapshot()).snapshot!; + await blockBlobClient.upload("v2", 2); + const sasClient = new BlockBlobClient( + await blockBlobClient.withSnapshot(snapshot).generateSasUrl({ + permissions: BlobSASPermissions.parse("racwt"), + expiresOn: new Date(Date.now() + 60 * 60 * 1000) + }), + newPipeline(new AnonymousCredential(), { retryOptions: { maxTries: 1 } }) + ); + + assert.strictEqual(await bodyToString(await sasClient.download(), 2), "v1"); + await assert.rejects( + sasClient.setMetadata({ key: "value" }), + (err: RestError) => + err.statusCode === 403 && + err.code === "AuthorizationResourceTypeMismatch" + ); + assert.deepStrictEqual( + (await blockBlobClient.getProperties()).metadata, + {} + ); + }); + + // What reads a snapshot through a snapshot SAS reads the snapshot: its + // committed blocks, none of the blocks staged on the base blob, and the + // page ranges of a page blob snapshot. + it("a SAS for a snapshot reads its block list and page ranges @loki", async () => { + await blockBlobClient.stageBlock(base64encode("1"), "a", 1); + await blockBlobClient.commitBlockList([base64encode("1")]); + const blockSnapshot = (await blockBlobClient.createSnapshot()).snapshot!; + await blockBlobClient.stageBlock(base64encode("2"), "b", 1); + const blockSasClient = new BlockBlobClient( + await blockBlobClient.withSnapshot(blockSnapshot).generateSasUrl({ + permissions: BlobSASPermissions.parse("r"), + expiresOn: new Date(Date.now() + 60 * 60 * 1000) + }), + newPipeline(new AnonymousCredential(), { retryOptions: { maxTries: 1 } }) + ); + for (const listType of ["committed", "all"] as const) { + const list = await blockSasClient.getBlockList(listType); + assert.deepStrictEqual( + list.committedBlocks!.map((block) => block.name), + [base64encode("1")] + ); + assert.deepStrictEqual(list.uncommittedBlocks ?? [], []); + } + + const pageBlobClient = containerClient.getPageBlobClient( + getUniqueName("page") + ); + await pageBlobClient.create(1024); + await pageBlobClient.uploadPages("a".repeat(512), 0, 512); + const pageSnapshot = (await pageBlobClient.createSnapshot()).snapshot!; + await pageBlobClient.uploadPages("b".repeat(512), 512, 512); + const pageSasClient = new PageBlobClient( + await pageBlobClient.withSnapshot(pageSnapshot).generateSasUrl({ + permissions: BlobSASPermissions.parse("r"), + expiresOn: new Date(Date.now() + 60 * 60 * 1000) + }), + newPipeline(new AnonymousCredential(), { retryOptions: { maxTries: 1 } }) + ); + // Only the first page, which the snapshot holds; the second was written + // to the base blob after it. The SDK reports a range's last byte as + // count. + const ranges = await pageSasClient.getPageRanges(); + assert.deepStrictEqual(ranges.pageRange, [{ offset: 0, count: 511 }]); + }); + + it("copies from a SAS URL for a version @loki", async () => { + const first = await blockBlobClient.upload("v1", 2); + await blockBlobClient.upload("v2", 2); + const sasUrl = await blockBlobClient + .withVersion(first.versionId!) + .generateSasUrl({ + permissions: BlobSASPermissions.parse("r"), + expiresOn: new Date(Date.now() + 60 * 60 * 1000) + }); + + // Put Blob From URL + const putTarget = containerClient.getBlockBlobClient(getUniqueName("put")); + await putTarget.syncUploadFromURL(sasUrl); + assert.strictEqual(await bodyToString(await putTarget.download(), 2), "v1"); + + // Copy Blob + const copyTarget = containerClient.getBlobClient(getUniqueName("copy")); + const poller = await copyTarget.beginCopyFromURL(sasUrl); + await poller.pollUntilDone(); + assert.strictEqual(await bodyToString(await copyTarget.download(), 2), "v1"); + }); + // Batch sub-requests must inherit the versioning setting of the batch request's account. // deleteSnapshots=include is the delete path whose outcome depends on that setting. it("should keep previous versions when deleting through a blob batch @loki", async () => { @@ -946,3 +1237,140 @@ describe("BlockBlobVersioningAPIs - concurrent blob batches on accounts with dif assert.strictEqual(remaining[1].length, 0); }); }); + +describe("BlockBlobVersioningAPIs - user delegation SAS @loki", () => { + const factory = new BlobTestServerFactory(); + const accountModelStore = createAccountModelStore( + { key: EMULATOR_ACCOUNT_NAME, isBlobVersioningEnabled: true }, + true + ); + const server = factory.createServer( + false, + false, + true, + "basic", + accountModelStore + ); + const baseURL = `https://${server.config.host}:${server.config.port}/${EMULATOR_ACCOUNT_NAME}`; + const token = generateJWTToken( + new Date("2019/01/01"), + new Date("2019/01/01"), + new Date("2100/01/01"), + "https://sts.windows-ppe.net/ab1f708d-50f6-404c-a006-d71b2ac7a606/", + "https://storage.azure.com", + "user_impersonation", + "23657296-5cd5-45b0-a809-d972a7f4dfe1", + "dd0d0df1-06c3-436c-8034-4b9a153097ce" + ); + const serviceClient = new BlobServiceClient( + baseURL, + newPipeline(new SimpleTokenCredential(token), { + retryOptions: { maxTries: 1 }, + keepAliveOptions: { enable: false } + }) + ); + + before(async () => { + await server.start(); + }); + + after(async () => { + await server.close(); + await server.clean(); + }); + + // A user delegation SAS signs the version ID, or the snapshot time, in its + // signedSnapshotTime field just as a service SAS does. + for (const target of ["version", "snapshot"]) { + it(`a user delegation SAS for a ${target} reads it @loki`, async () => { + const containerClient = serviceClient.getContainerClient( + getUniqueName("container") + ); + await containerClient.create(); + const blobName = getUniqueName("blob"); + const blockBlobClient = containerClient.getBlockBlobClient(blobName); + const first = await blockBlobClient.upload("v1", 2); + const snapshot = + target === "snapshot" + ? (await blockBlobClient.createSnapshot()).snapshot + : undefined; + await blockBlobClient.upload("v2", 2); + + const startsOn = new Date(Date.now() - 60 * 60 * 1000); + const expiresOn = new Date(Date.now() + 60 * 60 * 1000); + const userDelegationKey = await serviceClient.getUserDelegationKey( + startsOn, + expiresOn + ); + const sas = generateBlobSASQueryParameters( + { + containerName: containerClient.containerName, + blobName, + versionId: snapshot === undefined ? first.versionId : undefined, + snapshotTime: snapshot, + permissions: BlobSASPermissions.parse("r"), + expiresOn + }, + userDelegationKey, + EMULATOR_ACCOUNT_NAME + ); + assert.strictEqual(sas.resource, snapshot === undefined ? "bv" : "bs"); + const targetClient = + snapshot === undefined + ? blockBlobClient.withVersion(first.versionId!) + : blockBlobClient.withSnapshot(snapshot); + + const sasClient = new BlobClient( + `${targetClient.url}&${sas}`, + newPipeline(new AnonymousCredential(), { retryOptions: { maxTries: 1 } }) + ); + assert.strictEqual( + await bodyToString(await sasClient.download(), 2), + "v1" + ); + }); + } + + it("a writable user delegation SAS for a snapshot does not reach the base blob @loki", async () => { + const containerClient = serviceClient.getContainerClient( + getUniqueName("container") + ); + await containerClient.create(); + const blobName = getUniqueName("blob"); + const blockBlobClient = containerClient.getBlockBlobClient(blobName); + await blockBlobClient.upload("v1", 2); + const snapshot = (await blockBlobClient.createSnapshot()).snapshot!; + + const expiresOn = new Date(Date.now() + 60 * 60 * 1000); + const userDelegationKey = await serviceClient.getUserDelegationKey( + new Date(Date.now() - 60 * 60 * 1000), + expiresOn + ); + const sas = generateBlobSASQueryParameters( + { + containerName: containerClient.containerName, + blobName, + snapshotTime: snapshot, + permissions: BlobSASPermissions.parse("rw"), + expiresOn + }, + userDelegationKey, + EMULATOR_ACCOUNT_NAME + ); + const sasClient = new BlobClient( + `${blockBlobClient.withSnapshot(snapshot).url}&${sas}`, + newPipeline(new AnonymousCredential(), { retryOptions: { maxTries: 1 } }) + ); + + await assert.rejects( + sasClient.setMetadata({ key: "value" }), + (err: RestError) => + err.statusCode === 403 && + err.code === "AuthorizationResourceTypeMismatch" + ); + assert.deepStrictEqual( + (await blockBlobClient.getProperties()).metadata, + {} + ); + }); +});