Skip to content

Conversation

@joshcoughlan
Copy link

@joshcoughlan joshcoughlan commented Nov 17, 2025

Summary

This PR adds per-sink proxy support to the azure_blob sink, matching the behavior already available for AWS S3 and GCP sinks.

Highlights:

  • Each Azure Blob sink instance now uses its own proxy settings from SinkContext (ProxyConfig), not environment variables.
  • The Azure SDK transport is configured via azure_core::http::ClientOptions and a dedicated, aliased reqwest 0.12 client (reqwest_0_12_24), so it does not impact the project-wide reqwest usage.
  • Supports HTTP and HTTPS proxies, including credentials embedded in the proxy URL (handled by reqwest::Proxy).
  • Honors no_proxy patterns at the sink level to bypass the proxy for matching Azure endpoints.

Vector configuration

Example of per-sink proxy settings for Azure Blob:

[sinks.my_azure_blob]
type = "azure_blob"
connection_string = "DefaultEndpointsProtocol=https;AccountName=<account>;AccountKey=<key>;EndpointSuffix=core.windows.net"
container_name = "my-logs"
healthcheck.enabled = true

[sinks.my_azure_blob.proxy]
enabled = true
http = "http://user:[email protected]:3128"
https = "http://user:[email protected]:3128"
# Bypass proxy for these hosts/domains
no_proxy = "localhost,.internal.example.com"

Notes:

  • Credentials in the proxy URL are supported by reqwest.
  • If the Azure endpoint matches no_proxy or the proxy is disabled, the sink makes direct connections and disables any system proxy auto-detection for this client.

How did you test this PR?

  • Build and lint:
    • Ensured the updated call sites compile (e.g., integration helpers that construct the client).
  • Functional checks:
    • Verified that the Azure client is constructed with a reqwest 0.12 transport and that proxies are applied only for this sink.
    • Sanity-tested no_proxy handling by targeting a host in the exclusion list to ensure direct connections are used.
    • Am running a compiled binary of this PR in a real-world environment.
  • Suggested reviewer test plan (optional):
    • Run Azurite locally and point an azure_blob sink at it with:
      • No proxy (baseline).
      • HTTP proxy set.
      • HTTPS proxy set.
      • A no_proxy entry matching the Azurite host to confirm bypass.
    • Confirm healthcheck behavior with valid/invalid credentials and unknown containers.

Change Type

  • Bug fix
  • New feature
  • Non-functional (chore, refactoring, docs)
  • Performance

Is this a breaking change?

  • Yes
  • No

Does this PR include user facing changes?

  • Yes. Please add a changelog fragment based on our guidelines.
    • Suggested changelog: enhancement(azure_blob sink): Add per-sink proxy support via reqwest + azure_core::http::ClientOptions (HTTP/HTTPS proxies, honors no_proxy, supports proxy credentials).
  • No. A maintainer will apply the no-changelog label to this PR.

References

Notes

  • Implementation uses an aliased reqwest 0.12 client (reqwest_0_12_24) to avoid affecting other components that depend on a different reqwest version.
  • No environment variables are used; proxy configuration is per-sink via ProxyConfig.
  • If this PR modifies dependencies (Cargo.lock), I’ll run make build-licenses to refresh the license inventory.
  • This (or the parent SDK PR) fixes a bug introduced by Azure SDK version bump introduced regression in token refresh #23036 in which the documented SAS connection_string ("BlobEndpoint=https://mylogstorage.blob.core.windows.net/;SharedAccessSignature=generatedsastoken") doesn't work and fails with an error indicating it requires AccountName:
    ERROR vector::topology::builder: Configuration error. error=Sink "testing-azure_blob": Account name missing in connection string

@github-actions github-actions bot added the domain: sinks Anything related to the Vector's sinks label Nov 17, 2025
@joshcoughlan joshcoughlan marked this pull request as ready for review November 17, 2025 18:28
@joshcoughlan joshcoughlan requested a review from a team as a code owner November 17, 2025 18:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

domain: sinks Anything related to the Vector's sinks

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant