Skip to content

[AZINTS-3391] add azure storage monitoring setup #28861

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
89 changes: 85 additions & 4 deletions content/en/integrations/guide/storage-monitoring-setup.md
Original file line number Diff line number Diff line change
@@ -1,17 +1,17 @@
---
title: Storage Monitoring for Amazon S3 & Google Cloud Storage
title: Storage Monitoring for Amazon S3, Google Cloud Storage, and Azure Blob Storage
private: true
---

<div class="alert alert-info">Storage Monitoring is in Preview.</div>

## Overview

Storage Monitoring for Amazon S3 & Google Cloud Storage provides deep, prefix-level analytics to help you understand exactly how your storage is being used, detect potential issues before they impact operations, and make data-driven decisions about storage optimization. Use these insights to help you track storage growth, investigate access patterns, and optimize costs.
Storage Monitoring for Amazon S3, Google Cloud Storage, and Azure Blob Storage provides deep, prefix-level analytics to help you understand exactly how your storage is being used. It also detect potential issues before they impact operations, and make data-driven decisions about storage optimization. Use these insights to help you track storage growth, investigate access patterns, and optimize costs.

Check notice on line 10 in content/en/integrations/guide/storage-monitoring-setup.md

View workflow job for this annotation

GitHub Actions / vale

Datadog.sentencelength

Suggestion: Try to keep your sentence length to 25 words or fewer.

This guide explains how to configure Storage Monitoring in Datadog for your S3 & GCS buckets.
This guide explains how to configure Storage Monitoring in Datadog for your S3 buckets, GCS buckets, and Azure Storage Accounts.

You can set this up either manually or using the provided CloudFormation templates. Access your Storage Monitoring data by navigating to **Infrastructure -> Storage Monitoring**.
Access your Storage Monitoring data by navigating to **Infrastructure > Storage Monitoring**.

## Setup for Amazon S3

Expand Down Expand Up @@ -364,3 +364,84 @@
[2]: https://cloud.google.com/storage/docs/insights/using-inventory-reports#enable_the_api
[3]: https://forms.gle/c7b8JiLENDaUEqGk8



## Setup for Azure Blob Storage

### Installation

To set up Storage Monitoring for Azure Blob Storage, follow these steps:

{{< tabs >}}
{{% tab "Azure CLI" %}}

To enable inventories for the selected storage accounts in each subscription, run the following script in your [Azure Cloud Shell][301]:

```shell
curl https://datadogstoragemonitoring.blob.core.windows.net/scripts/install.sh \
| bash -s -- <client_id> <subscription_id> <comma_separated_storage_account_names>
```

Before running the script, set your [shell environment][302] to Bash and replace the various placeholder inputs with the correct values:
- `<client_id>`: The client ID of an App Registration already set up using the [Datadog Azure integration][302]
- `<subscription_id>`: The subscription ID of the Azure subscription containing the storage accounts
- `<comma_separated_storage_account_names>`: A comma-separated list of the storage accounts you want to monitor. For example, `storageaccount1,storageaccount2`


[301]: https://shell.azure.com
[302]: /integrations/azure/#setup
[303]: https://learn.microsoft.com/en-us/azure/cloud-shell/get-started/classic?tabs=azurecli#select-your-shell-environment
{{% /tab %}}

{{% tab "Azure Portal" %}}

For Each Storage Account you wish to monitor, follow all of the steps here:


#### Create a blob inventory policy
1. In the Azure portal, navigate to your Storage Account.
2. Go to **Data management** > **Blob inventory**.
3. Click **Add**.
4. Configure the policy:
- Name: **datadog-storage-monitoring**
- Destination container:
- Click **Create new**, and enter the name `datadog-storage-monitoring`.
- Object type to inventory: **Blob**
- Schedule: **Daily**
- Blob types: Select **Block blobs**, **Append blobs**, and **Page blobs**.
- Subtypes: Select **Include blob versions**
- Schema fields: Select All, or ensure that at least the following are selected:
- **Name**
- **Access tier**
- **Last modified**
- **Content length**
- **Server encrypted**
- **Current version status**
- **Version ID**
- Exclude prefix: datadog-storage-monitoring
5. Click **Add**.

#### Add the role assignment
1. In the Azure portal, navigate to your Storage Account.
2. Go to **Data storage** > **Containers**.
3. Click on the **datadog-storage-monitoring** container.
4. Click on **Access control (IAM)** in the left-hand menu.
5. Click **Add** > **Add role assignment**.
6. Fill out the role assignment:
- Role: Select **Storage Blob Data Reader**. Click **Next**.
- Assign access to: **User, group, or service principal**.
- Members: Click **+ Select members** and search for your App Registration by its name and select it.
- **Note**: This should be an App Registration set up in the Datadog Azure integration. Keep in mind the Client ID for later.
7. Click **Review + assign**.

{{% /tab %}}
{{< /tabs >}}

### Post-Installation

After you finish with the above steps, fill out the [post-setup form][310].

[310]: https://forms.gle/WXFbGyBwWfEo3gbM7



Loading