From 4adf8466a531bd850581ebf1640608ea2c271259 Mon Sep 17 00:00:00 2001 From: Denise Schlesinger Date: Sun, 25 Oct 2026 23:28:27 +0200 Subject: [PATCH] Doc denisa changes (#7614) * test Added introductory content to the add-cluster-connection article. * mv changes * links * mv changes * corrections * suggested changes applied * chnages * sagiv asks * changes by Shiri * Update data-explorer/kusto/management/materialized-views/materialized-views-optimization.md Co-authored-by: learn-build-service-prod-10[bot] <274431553+learn-build-service-prod-10[bot]@users.noreply.github.com> * new pr * shiri comments fixed pr7585 * shiri take 3 * last changes * export and cached query results * export sales tbl * export * mv ishealthy changes * changes * chg * chgs * date changed --------- Co-authored-by: learn-build-service-prod-10[bot] <274431553+learn-build-service-prod-10[bot]@users.noreply.github.com> --- data-explorer/kusto/concepts/sandboxes.md | 36 +++++++++++++++++-- ...ialized-view-show-command-output-schema.md | 2 +- .../data-export/export-data-to-storage.md | 14 ++++++++ .../kusto/management/data-export/index.md | 23 ++++++------ .../materialized-views-monitoring.md | 30 +++++++++++++--- 5 files changed, 86 insertions(+), 19 deletions(-) diff --git a/data-explorer/kusto/concepts/sandboxes.md b/data-explorer/kusto/concepts/sandboxes.md index 7c444843ad..c12629567d 100644 --- a/data-explorer/kusto/concepts/sandboxes.md +++ b/data-explorer/kusto/concepts/sandboxes.md @@ -1,9 +1,9 @@ --- title: Sandboxes description: This article describes Sandboxes. -ms.reviewer: orspodek +ms.reviewer: kareem-ismail ms.topic: reference -ms.date: 08/11/2024 +ms.date: 10/13/2026 monikerRange: "azure-data-explorer" --- # Sandboxes @@ -71,18 +71,38 @@ The following table lists all modern VM sizes that support Hyper-V sandbox techn |---------------------------------------|-------------------| | Standard_L8s_v3 | storage-optimized | | Standard_L16s_v3 | storage-optimized | +| Standard_L32s_v3 | storage-optimized | +| Standard_L8s_v4 | storage-optimized | +| Standard_L16s_v4 | storage-optimized | +| Standard_L32s_v4 | storage-optimized | | Standard_L8as_v3 | storage-optimized | | Standard_L16as_v3 | storage-optimized | +| Standard_L32as_v3 | storage-optimized | +| Standard_L8as_v4 | storage-optimized | +| Standard_L16as_v4 | storage-optimized | +| Standard_L32as_v4 | storage-optimized | +| Standard_L8aos_v4 | storage-optimized | +| Standard_L16aos_v4 | storage-optimized | +| Standard_L32aos_v4 | storage-optimized | | Standard_E8as_v5 | storage-optimized | | Standard_E16as_v5 | storage-optimized | +| Standard_E8as_v6 | storage-optimized | +| Standard_E16as_v6 | storage-optimized | | Standard_E8s_v4 | storage-optimized | | Standard_E16s_v4 | storage-optimized | | Standard_E8s_v5 | storage-optimized | | Standard_E16s_v5 | storage-optimized | +| Standard_E8s_v6 | storage-optimized | +| Standard_E16s_v6 | storage-optimized | | Standard_E2ads_v5 | compute-optimized | | Standard_E4ads_v5 | compute-optimized | | Standard_E8ads_v5 | compute-optimized | | Standard_E16ads_v5 | compute-optimized | +| Standard_E2ads_v6 | compute-optimized | +| Standard_E4ads_v6 | compute-optimized | +| Standard_E8ads_v6 | compute-optimized | +| Standard_E16ads_v6 | compute-optimized | +| Standard_E32ads_v6 | compute-optimized | | Standard_E2d_v4 | compute-optimized | | Standard_E4d_v4 | compute-optimized | | Standard_E8d_v4 | compute-optimized | @@ -92,3 +112,15 @@ The following table lists all modern VM sizes that support Hyper-V sandbox techn | Standard_E8d_v5 | compute-optimized | | Standard_E16d_v5 | compute-optimized | | Standard_D32d_v4 | compute-optimized | +| Standard_D16d_v5 | compute-optimized | +| Standard_D32d_v5 | compute-optimized | +| Standard_D2ads_v6 | compute-optimized | +| Standard_D4ads_v6 | compute-optimized | +| Standard_D8ads_v6 | compute-optimized | +| Standard_D16ads_v6 | compute-optimized | +| Standard_D32ads_v6 | compute-optimized | +| Standard_D2ds_v6 | compute-optimized | +| Standard_D4ds_v6 | compute-optimized | +| Standard_D8ds_v6 | compute-optimized | +| Standard_D16ds_v6 | compute-optimized | +| Standard_D32ds_v6 | compute-optimized | diff --git a/data-explorer/kusto/includes/materialized-view-show-command-output-schema.md b/data-explorer/kusto/includes/materialized-view-show-command-output-schema.md index 2872b5a9bd..2339dd586d 100644 --- a/data-explorer/kusto/includes/materialized-view-show-command-output-schema.md +++ b/data-explorer/kusto/includes/materialized-view-show-command-output-schema.md @@ -13,7 +13,7 @@ ms.date: 02/03/2025 | MaterializedTo | `datetime` | Maximum materialized ingestion_time() timestamp in source table. For more information, see [how materialized views work](../management/materialized-views/materialized-view-overview.md#how-materialized-views-work). | | LastRun | `datetime` | Last time materialization was run. | | LastRunResult | `string` | Result of last run. Returns `Completed` for successful runs, otherwise `Failed`. | -| IsHealthy | `bool` | `true` when view is considered healthy, `false` otherwise. View is considered healthy if it was successfully materialized up to the last hour (`MaterializedTo` is greater than `ago(1h)`). | +| IsHealthy | `bool` | `false` only when the view is stuck:
• At least five consecutive materialization attempts failed, and `MaterializedTo` is more than one hour old and hasn't caught up to the database cursor.
• The view status is `FAILED`.
• A view that's catching up, handling an ingestion burst, or waiting for a concurrency slot is `true`, even if its age exceeds one hour.
• A disabled view can also appear as `true`. Check `IsEnabled` as well. | | IsEnabled | `bool` | `true` when view is enabled (see [Disable or enable materialized view](../management/materialized-views/materialized-view-enable-disable.md)). | | Folder | `string` | Folder under which the materialized view is created. | | DocString | `string` | Description assigned to the materialized view. | diff --git a/data-explorer/kusto/management/data-export/export-data-to-storage.md b/data-explorer/kusto/management/data-export/export-data-to-storage.md index c8c7fa47bb..25a5a3f3e0 100644 --- a/data-explorer/kusto/management/data-export/export-data-to-storage.md +++ b/data-explorer/kusto/management/data-export/export-data-to-storage.md @@ -51,6 +51,20 @@ You must have at least [Database Viewer](../../access-control/role-based-access- | `distributed` | `bool` | Disable or enable distributed export. Setting to false is equivalent to `single` distribution hint. Default is true. | | `parquetDatetimePrecision` | `string` | The precision to use when exporting `datetime` values to Parquet. Possible values are millisecond and microsecond. Default is millisecond. | +## Example + +The following example exports all records from `MyTable` to compressed CSV files in Azure Blob Storage. Each file includes column headers and uses `my-export` as its name prefix. + +```kusto +.export compressed + to csv (h@"https://storageaccount.blob.core.windows.net/container;impersonate") + with ( + includeHeaders="all", + namePrefix="my-export" + ) + <| MyTable +``` + ## Authentication and authorization The authentication method is based on the connection string provided, and the permissions required vary depending on the authentication method. diff --git a/data-explorer/kusto/management/data-export/index.md b/data-explorer/kusto/management/data-export/index.md index c99354ec7d..563949581b 100644 --- a/data-explorer/kusto/management/data-export/index.md +++ b/data-explorer/kusto/management/data-export/index.md @@ -11,25 +11,24 @@ ms.date: 11/27/2024 Data export involves executing a Kusto query and saving its results. This process can be carried out either on the client side or the service side. -For examples on data export, see [Related content](#related-content). +## Export types -## Client-side export +For scalable data export, the service offers various `.export` management commands to export query results. +This approach enhances scalability by avoiding the bottleneck of streaming through a single network connection. -Client-side export gives you control over saving query results either to the local file system or pushing them to a preferred storage location. This flexibility is facilitated by using [Kusto client libraries](../../api/client-libraries.md). You can [create an app to run queries](../../api/get-started/app-basic-query.md), read the desired data, and implement an export process tailored to your requirements. +**One-time export** -::: moniker range="azure-data-explorer" -Alternatively, you can use a client tool like the Azure Data Explorer web UI to export data from your Kusto cluster. For more information, see [Share queries](/azure/data-explorer/web-share-queries). -::: moniker-end +Use a one-time `.export` command to run a query and export its current results to a supported destination. This option is useful for ad hoc exports or exports managed by an external scheduler. -## Service-side export (pull) - -Use the [ingest from query](../../management/data-ingestion/ingest-from-query.md) commands to pull query results into a table in the same or different database. See the [performance tips](../../management/data-ingestion/ingest-from-query.md#performance-tips) before using these commands. +* [Export to cloud storage](export-data-to-storage.md) +* [Export to an external table](export-data-to-an-external-table.md) +* [Export to a SQL table](export-data-to-sql.md) -## Service-side export (push) +**Continuous data export** -For scalable data export, the service offers various `.export` management commands to push query results to [cloud storage](export-data-to-storage.md), an [external table](export-data-to-an-external-table.md), or an [SQL table](export-data-to-sql.md). This approach enhances scalability by avoiding the bottleneck of streaming through a single network connection. +Continuous export automatically exports newly ingested data to an external table by using a recurring query. This option keeps exported data up to date without requiring you to run each export manually. -[Continuous data export](continuous-data-export.md) is supported for export to external tables. +* [Continuous data export overview](continuous-data-export.md) > [!NOTE] > The `.export` management commands are limited by the available data export capacity of your database. Run the [.show capacity command](../show-capacity-command.md) to view the total, consumed, and remaining data export capacity. diff --git a/data-explorer/kusto/management/materialized-views/materialized-views-monitoring.md b/data-explorer/kusto/management/materialized-views/materialized-views-monitoring.md index ff7a1840be..e1ec8348d0 100644 --- a/data-explorer/kusto/management/materialized-views/materialized-views-monitoring.md +++ b/data-explorer/kusto/management/materialized-views/materialized-views-monitoring.md @@ -18,7 +18,7 @@ Monitor the health of materialized views in the following ways: * Monitor [materialized view metrics](/fabric/real-time-intelligence/monitor-metrics#metric-specific-dimension-column) in your Microsoft Fabric workspace. Use the materialized view age metric, `MaterializedViewAgeSeconds`, as the primary metric to monitor the freshness of the view. For more information, see [Enable monitoring in your workspace](/fabric/get-started/enable-workspace-monitoring). ::: moniker-end -* Monitor the `IsHealthy` property by using [`.show materialized-view`](materialized-view-show-command.md#show-materialized-views). +* Monitor the `IsHealthy` property by using [`.show materialized-view`](materialized-view-show-command.md#show-materialized-views). `IsHealthy` is `false` only when the view is stuck. For more information, see [When a materialized view is unhealthy](#when-a-materialized-view-is-unhealthy). * Inspect extent, hot-cache, size, and effective policy details by using [`.show materialized-view details`](materialized-view-show-details-command.md). @@ -28,9 +28,31 @@ Monitor the health of materialized views in the following ways: > > Materialization never skips any data, even if there are constant failures. The view always returns the most up-to-date snapshot of the query, based on all records in the source table. Constant failures significantly degrade query performance, but don't cause incorrect results in view queries. -## Troubleshooting unhealthy materialized views +## When a materialized view is unhealthy -If the `MaterializedViewAgeSeconds` metric constantly increases and the `MaterializedViewHealth` metric shows that the view is unhealthy, use the following symptoms to identify the root cause. +`IsHealthy` is `false` only when both of the following conditions are true: + +* At least five consecutive materialization attempts failed. +* `MaterializedTo` is more than one hour old and hasn't caught up to the database cursor. + +The view status is then `FAILED`. The view might recover on its own, such as when an overloaded cluster frees up resources, or it might need user intervention. + +If either condition isn't met, `IsHealthy` is `true`, regardless of the other. The following table shows examples: + +| Condition | `IsHealthy` | +|---|---| +| `MaterializedTo` is more than one hour old, but the view is catching up or processing an ingestion burst. | `true` | +| `MaterializedTo` is more than one hour old because the view is queued due to concurrency limits. | `true` | +| Fewer than five consecutive failures, regardless of lag. | `true` | +| Five or more consecutive failures, but `MaterializedTo` is less than one hour old. | `true` | +| Five or more consecutive failures, and `MaterializedTo` is more than one hour old and hasn't caught up to the database cursor (for example, repeated out-of-memory failures with no successful batch). | `false` | +| The source table was dropped or the query is permanently invalid. | The view is automatically disabled and appears as `true`. Check `IsEnabled`. | + +A healthy view can still have an increasing age, and a disabled view doesn't materialize. Use the `MaterializedViewAgeSeconds` metric to monitor freshness, and check `IsHealthy` together with `IsEnabled`. + +## Troubleshooting materialized views + +If the `MaterializedViewAgeSeconds` metric constantly increases, or the view is unhealthy (`IsHealthy` is `false`), use the following symptoms to identify the root cause. :::moniker range="azure-data-explorer" @@ -114,7 +136,7 @@ Check whether the [`MaterializedViewResult` metric](#materializedviewresult-metr Recurring `InsufficientCapacity` values indicate that available ingestion capacity is limiting materialization. Increase the resources available for materialization. For more information, see [Increase available resources](materialized-views-optimization.md#increase-available-resources). -### Materialized view remains unhealthy +### Materialized view remains unhealthy or keeps falling behind If the preceding diagnostics don't identify a specific cause, the service might not have sufficient capacity or resources to materialize all data on time. Increase available resources or split a memory-intensive view only after applying less disruptive optimizations. For more information, see [Optimize materialized views](materialized-views-optimization.md).