Refactor http, grpc senders and promote to public API #7782

jack-berg · 2025-10-22T20:33:51Z

Resolves #6718.

Sketching out all the changes I think would be required to promote the HttpSender, GrpcSender interfaces to the public API. There's a lot. Summary:

We need to hide Marshaler and related APIs, since they balloon the API surface area and would take lot of work to get ready for public. Introducing narrow focused GrpcMessageWriter, HttpRequestBodyWriter to serve the function currently performed by Marshaler.
Need to get rid of MarshalerServiceStub and related APIs from the gRPC senders stuff. It drags in a bunch of unnecessary cruft and io.grpc.stub dependency.
Need to hide the HttpSenderConfig#getExportAsJson() option. Senders don't need to be burdened with understanding whether the request is binary or JSON. They just need to be told the content type and a way to write the request body.
Compressor and related APIs need to get promoted as well.
Need to refine GrpcSenderConfig, HttpSenderConfig
- Need to be interfaces instead of autovalue classes
- Need javadoc
- Make endpoint URI instead of string
- Provide grpc senders the fully qualified service name and method name
- Hide the Object GrpcSenderConfig#getManagedChannel(), which is required for backwards compatibility and for UpstreamGrpcSender, behind an internal-only ExtendedGrpcSenderConfig

Leaving as a draft because:

there are still some things to clean up
now that i've laid out what's necessary, I want to confirm that we want to do this
if we do indeed want this, I want to discuss the best way to land it. 1500 lines changes across 94 files is obviously a lot to review. I can break it into smaller parts, but we'd want to get those parts merged in one minor release to minimize churn.

brunobat · 2025-11-06T17:30:06Z

@jack-berg, I didn't forgot this issue. I'm planning to give you a response in the week of Nov. 17.

brunobat

@jack-berg, thanks very much for this.
The end user changes are fair enough and after updating the Quarkus code, it worked out of the box and the IT tests with a real OTel Collector pass.
I'm adding a couple of comments.

brunobat · 2025-11-20T15:15:50Z

exporters/common/src/main/java/io/opentelemetry/exporter/grpc/GrpcResponse.java

+
+  /** The gRPC response message bytes. */
+  @SuppressWarnings("mutable")
+  byte[] getResponseMessage();


I'm not sure what to do here because our gRPC API is reactive and I only have access to a Future. I wouldn't like to block here.
What can be done with this response?

I also think the response is empty and that will return null during normal usage. Should we expect an Optional?

Originally OTLP exporter responses were empty, but they now contain some optional information. See the proto definition for details.

Right now, the java implementation ignores anything in the response. I was trying to get ahead of future requirements to do something useful with the response by forcing sender implementations to provide a byte representation of the response body.

Thinking about it more, I think its possible to add this later, or to add a default implementation that just returns an empty byte array. The underlying exporter implementation could then try to do something useful if the response body is available, and skip if not.

Sounds good to me.

brunobat · 2025-11-20T15:25:00Z

exporters/common/src/main/java/io/opentelemetry/exporter/internal/http/HttpExporter.java

  private final String type;
  private final HttpSender httpSender;
  private final ExporterInstrumentation exporterMetrics;
+  private final boolean exportAsJson;


I wonder if the JSON flag could be part of the HttpRequestBodyWriter

Its embedded in there. The HttpRequestBodyWriter passed to HttpSender#send serializes the body in the appropriately based on the whether json is selected or not. All a sender has to do is ensure the correct Content-Type header is set by adhering to HttpSenderConfig#getContentType().

I was just mentioning that because we could probably get rid of this parameter.

trask · 2025-11-20T17:48:01Z

exporters/common/src/main/java/io/opentelemetry/exporter/http/HttpSenderConfig.java

+   * Additional headers that must be appended to every request. The resulting {@link Supplier} must
+   * be invoked for each request.
+   */
+  Supplier<Map<String, List<String>>> getHeadersSupplier();


note to me: handles auth

brunobat

I've taken a look at the declarative config being proposed. We currently don't use it but I have a few comments.

brunobat · 2025-11-24T09:58:40Z

exporters/common/src/main/java/io/opentelemetry/exporter/grpc/GrpcSenderConfig.java

+   * #getTrustManager()} will also be non-null.
+   */
+  @Nullable
+  SSLContext getSslContext();


This should probably be a configuration on the sender side because each sender will have it's own security wiring. In Quarkus we don't use these properties because the client is integrated in the framework and has it's own configs.

This will be non-null if a user explicitly calls the various setters which change the security context, e.g.:

setTrustedCertificates

setClientTls

setSsslContext

I understand that quarkus has its own security context integrated into the framework, but its a confusing experience for a user to try to explicitly set the security context and have those options ignored conditionally based on the sender.

Perhaps the quarkus sender can default to security context integrated into the framework, and only use this SSLContext when non-null.

That's fair, as long as these properties are not used outside the sender. This could cause a split config problem.
I don't know how to ensure that.

brunobat · 2025-11-24T09:59:31Z

exporters/common/src/main/java/io/opentelemetry/exporter/grpc/GrpcSenderConfig.java

+   * #getSslContext()} will also be non-null.
+   */
+  @Nullable
+  X509TrustManager getTrustManager();


Same as above.

brunobat · 2025-11-24T10:01:34Z

exporters/common/src/main/java/io/opentelemetry/exporter/grpc/GrpcSenderConfig.java

+   * Additional headers that must be appended to every request. The resulting {@link Supplier} must
+   * be invoked for each request.
+   */
+  Supplier<Map<String, List<String>>> getHeadersSupplier();


I'm not sure a supplier should live in config objects, but probably that is a larger discussion related with declarative config.

See comment here: #7782 (comment)

brunobat · 2025-11-24T10:02:23Z

exporters/common/src/main/java/io/opentelemetry/exporter/http/HttpSenderConfig.java

+   * #getTrustManager()} will also be non-null.
+   */
+  @Nullable
+  SSLContext getSslContext();


Same as gRPC

brunobat · 2025-11-24T10:02:35Z

exporters/common/src/main/java/io/opentelemetry/exporter/http/HttpSenderConfig.java

+   * #getSslContext()} will also be non-null.
+   */
+  @Nullable
+  X509TrustManager getTrustManager();


Same as gRPC

brunobat · 2025-11-24T10:06:12Z

exporters/common/src/main/java/io/opentelemetry/exporter/grpc/GrpcSenderConfig.java

+   * grpc-encoding}.
+   */
+  @Nullable
+  Compressor getCompressor();


At some point I'll need to create Compressor objects but the existing implementations are in internal packages.

This PR promotes them to the stable package. The idea is that this config option gives you a ref to the resolved compressor, or null if none is configured. Hopefully the quarkus sender can integrate with the compressor, adding the appropriate headers and using it to compress the payload.

zeitlinger · 2025-11-24T12:42:29Z

The end user changes are fair enough and after updating the Quarkus code

Why is this relavant for Quarkus? I'm asking because the spring starter doesn't care - and I'm trying to understand what the difference is

jack-berg · 2025-11-24T22:27:39Z

I've taken a look at the declarative config being proposed.

There's actually nothing in here related to declarative config. The getters in HttpSenderConfig, GrpcSenderConfig reflect the standard configuration options we expose via programmatic config for OTLP exporters (e.g. OtlpHttpSpanExporterBuilder, OtlpGrpcSpanExporterBuilder).

As the set of configuration options we expose via the Otlp{Signal}{Protocol}ExporterBuilders evolves, HttpSenderConfig and GrpcSenderConfig will evolve along side.

I would say that if a sender ignores any of those options, its technically not compliant. Non-compliant doesn't mean its not useful, but it does mean that the sender isn't able to behave like the user expects based on how they configured the builder.

brunobat · 2025-11-25T11:14:50Z

The end user changes are fair enough and after updating the Quarkus code

Why is this relavant for Quarkus? I'm asking because the spring starter doesn't care - and I'm trying to understand what the difference is

Because Quarkus has it's own senders based on a Vert.x client. It does not use OkHttp. See #6718 and its comments for more details.

brunobat · 2025-11-25T11:23:55Z

I've taken a look at the declarative config being proposed.

There's actually nothing in here related to declarative config. The getters in HttpSenderConfig, GrpcSenderConfig reflect the standard configuration options we expose via programmatic config for OTLP exporters (e.g. OtlpHttpSpanExporterBuilder, OtlpGrpcSpanExporterBuilder).

As the set of configuration options we expose via the Otlp{Signal}{Protocol}ExporterBuilders evolves, HttpSenderConfig and GrpcSenderConfig will evolve along side.

I would say that if a sender ignores any of those options, its technically not compliant. Non-compliant doesn't mean its not useful, but it does mean that the sender isn't able to behave like the user expects based on how they configured the builder.

Thanks for the explanation @jack-berg.
If those properties are not used, the compliance is left to the implementator.
As in the other properties, we always try to match the property behavior, even if properties are handled by Quarkus and some behavior is not implemented directly on the SDK. We have other examples with samplers and several customizations we provide for the SDK.
The Idea is to melt the OTel experience into Quarkus and have minimal friction for people with knowledge of both frameworks.

jack-berg · 2025-11-25T14:47:13Z

Ok so it seems like something close to this will work for quarkus. There are still some small tweaks (example), but its mostly correct.

The next step will be to coordinate with @jkwatson and @open-telemetry/java-approvers to choose a strategy and schedule to get this merged. I opened this as one big PR to make the whole picture clear, but I'm happy to break it up in smaller more reviewable pieces as long as we can commit to work quickly to get all those pieces merged for a single release. This will be important to minimize churn.

Refactor http, grpc senders and promote to public API

b4ef764

brunobat reviewed Nov 20, 2025

View reviewed changes

brunobat mentioned this pull request Nov 20, 2025

Use HttpSender and GrpcSender from upstream OpenTelemetry API quarkusio/quarkus#51143

Draft

tylerbenson mentioned this pull request Nov 20, 2025

Make the disk buffered exporting more efficient open-telemetry/opentelemetry-java-contrib#2190

Closed

trask reviewed Nov 20, 2025

View reviewed changes

brunobat reviewed Nov 24, 2025

View reviewed changes

Refactor http, grpc senders and promote to public API #7782

Are you sure you want to change the base?

Refactor http, grpc senders and promote to public API #7782

Uh oh!

Conversation

jack-berg commented Oct 22, 2025

Uh oh!

brunobat commented Nov 6, 2025

Uh oh!

brunobat left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

brunobat Nov 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

brunobat left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

zeitlinger commented Nov 24, 2025

Uh oh!

jack-berg commented Nov 24, 2025

Uh oh!

brunobat commented Nov 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

brunobat commented Nov 25, 2025

Uh oh!

jack-berg commented Nov 25, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

brunobat Nov 20, 2025 •

edited

Loading

brunobat commented Nov 25, 2025 •

edited

Loading