Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Dec 3, 2025

Why make this change?

Closes #2711

DAB does not currently support response compression (gzip/brotli), which can reduce payload sizes by ~79% for large JSON responses. This is especially beneficial for REST and GraphQL endpoints over slow or high-latency networks.

What is this change?

Adds response compression middleware with configurable compression levels.

Configuration Model:

  • CompressionLevel enum: optimal, fastest, none
  • CompressionOptions record with JSON converter and validation
  • Default: optimal

Runtime Behavior:

  • Gzip and Brotli providers enabled for HTTPS
  • Applies to REST, GraphQL, and MCP endpoints
  • Logs compression level on startup (silent when none)

Schema & CLI:

{
  "runtime": {
    "compression": {
      "level": "optimal"
    }
  }
}

How was this tested?

  • Integration Tests
  • Unit Tests

Added 3 tests in ConfigureOptionsTests.cs covering all compression levels.

Sample Request(s)

CLI Configuration:

dab configure --runtime.compression.level fastest

Sample Request/Response:

GET /api/employees HTTP/1.1
Accept-Encoding: gzip, br

HTTP/1.1 200 OK
Content-Encoding: gzip
Content-Type: application/json

Startup Output (when enabled):

Response compression enabled with level 'Optimal' for REST, GraphQL, and MCP endpoints.
Original prompt

This section details on the original issue you should resolve

<issue_title>⭐ [Enhancement]: Add Compression</issue_title>
<issue_description>Data API builder does not currently support response compression (e.g., gzip or brotli), which can significantly reduce payload size and improve performance over slow or high-latency networks. Compression is especially beneficial for large JSON responses, common in REST and GraphQL endpoints. Adding support at the server level—either via built-in middleware or integration guidance—would help DAB serve clients more efficiently. This is a common feature in modern web APIs and would be a strong addition to the roadmap.

Sample request

GET /api/employees HTTP/1.1  
Host: api.example.com  
Accept-Encoding: gzip, deflate, br  

Sample response

HTTP/1.1 200 OK  
Content-Encoding: gzip  
Content-Type: application/json  
Content-Length: 920  

Payload impact

Payload Type Uncompressed (bytes) Gzipped (bytes) Compression Savings
Large JSON array 4,356 920 ~79%

Configuration Update

{
    "runtime": {
        "compression": {
            "level": "fastest" | "optimal" | "none" <default: optimal>
        }
    }
}

CLI update

dab configure --runtime.compression.level fastest

Starter JSON Schema section

{
  "runtime": {
    "type": "object",
    "properties": {
      "compression": {
        "type": "object",
        "description": "Configures HTTP response compression settings.",
        "properties": {
          "level": {
            "type": "string",
            "enum": ["fastest", "optimal", "none"],
            "default": "optimal",
            "description": "Specifies the response compression level."
          }
        },
        "additionalProperties": false
      }
    }
  }
}

Basic ASPNET implementation

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddResponseCompression(options =>
{
    options.EnableForHttps = true;
    options.Providers.Add<GzipCompressionProvider>();
});
builder.Services.Configure<GzipCompressionProviderOptions>(options =>
{
    options.Level = CompressionLevel.Fastest;
});

var app = builder.Build();

app.UseResponseCompression();

// map endpoints and controllers

app.Run();

FAQ

  1. Is this backwards compatible?
    Yes. Clients that don’t request compression (via Accept-Encoding) receive uncompressed responses exactly as before.

  2. Does this require a special client?
    No. Most modern browsers and HTTP clients (like curl, Postman, fetch, axios) already support compressed responses out-of-the-box.

  3. Does this impact controller logic?
    No. Compression is handled at the middleware level, completely separate from your controller logic or data access code.

  4. What is the compute cost compared to bandwidth savings?
    Compression uses CPU but reduces payload size dramatically. In most APIs, the tradeoff favors compression—especially for large or repetitive JSON responses. The CompressionLevel.Fastest option balances speed and size.

  5. Does this work over HTTPS?
    Yes. Response compression is fully supported over HTTPS and can be explicitly enabled with EnableForHttps = true.

  6. Can it be customized for specific MIME types?
    Yes. You can add custom MIME types (e.g., application/graphql+json) if needed.

  7. What compression formats are supported?
    Gzip is most common, but Brotli and Deflate are also supported in ASP.NET Core with the appropriate providers.

  8. Does it work with REST and GraphQL?
    Yes. Compression applies to all HTTP responses regardless of the endpoint type, including both REST and GraphQL APIs.

  9. Does Hot Chocolate support it?
    Hot Chocolate runs on ASP.NET Core and inherits its middleware pipeline, so response compression works with it automatically—no special configuration needed in Hot Chocolate itself.</issue_description>

<agent_instructions>You are a senior microsoft engineer dedicated to terse code and changing as little code as possible in order to complete the task.</agent_instructions>

Comments on the Issue (you are @copilot in this section)


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Copilot AI changed the title [WIP] Add support for response compression in Data API Add HTTP Response Compression Support Dec 3, 2025
Copilot AI requested a review from JerryNixon December 3, 2025 21:45
@JerryNixon JerryNixon marked this pull request as ready for review December 4, 2025 03:05
Copilot AI review requested due to automatic review settings December 4, 2025 03:05
@JerryNixon JerryNixon enabled auto-merge (squash) December 4, 2025 03:05
@JerryNixon JerryNixon moved this from Todo to Review In Progress in Data API builder Dec 4, 2025
@JerryNixon JerryNixon added this to the Dec 2025 milestone Dec 4, 2025
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds HTTP response compression support to Data API Builder, enabling Gzip and Brotli compression for REST, GraphQL, and MCP endpoints to reduce payload sizes and improve performance over slow networks.

Key Changes:

  • Added configurable compression middleware with three levels: optimal (default), fastest, and none
  • Implemented JSON schema, CLI commands, and configuration model for compression settings
  • Added unit tests for CLI configuration updates

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
src/Service/Startup.cs Adds ConfigureResponseCompression method to register Gzip and Brotli providers, and calls UseResponseCompression middleware when compression is enabled
src/Config/ObjectModel/CompressionOptions.cs Defines the CompressionOptions record with Level property and UserProvidedLevel flag for JSON serialization control
src/Config/ObjectModel/CompressionLevel.cs Defines the CompressionLevel enum with Optimal, Fastest, and None values
src/Config/Converters/CompressionOptionsConverterFactory.cs Implements custom JSON converter for reading and writing compression configuration with validation
src/Config/RuntimeConfigLoader.cs Registers the CompressionOptionsConverterFactory in the serialization options
src/Config/ObjectModel/RuntimeOptions.cs Adds Compression property to runtime configuration options
src/Cli/ConfigGenerator.cs Implements CLI configuration update logic for compression level settings
src/Cli/Commands/ConfigureOptions.cs Adds RuntimeCompressionLevel command-line option for the dab configure command
src/Cli.Tests/ConfigureOptionsTests.cs Adds test cases for all three compression levels (Optimal, Fastest, None)
schemas/dab.draft.schema.json Adds JSON schema definition for the compression configuration section

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

}
}
}
}
Copy link

Copilot AI Dec 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The converter doesn't properly handle unknown properties with complex values (objects or arrays). After reading a property name on line 58, if the property is not "level", the code continues the loop without skipping the property value. This works for simple values but could cause parsing issues if an unknown property contains an object or array.

Add a default case to properly skip unknown properties:

if (string.Equals(propertyName, "level", StringComparison.OrdinalIgnoreCase))
{
    string? levelStr = reader.GetString();
    if (levelStr is not null)
    {
        if (Enum.TryParse<CompressionLevel>(levelStr, ignoreCase: true, out CompressionLevel parsedLevel))
        {
            level = parsedLevel;
            userProvidedLevel = true;
        }
        else
        {
            throw new JsonException($"Invalid compression level: '{levelStr}'. Valid values are: optimal, fastest, none.");
        }
    }
}
else
{
    // Skip unknown properties
    reader.Skip();
}
Suggested change
}
else
{
// Skip unknown properties and their values (including objects/arrays)
reader.Skip();
}

Copilot uses AI. Check for mistakes.
/// <summary>
/// Tests that running "dab configure --runtime.compression.level {value}" on a config with various values results
/// in runtime config update. Takes in updated value for compression.level and
/// validates whether the runtime config reflects those updated values
Copy link

Copilot AI Dec 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing closing period in the documentation comment. The summary should end with a period for consistency with other test documentation in the file.

Suggested change
/// validates whether the runtime config reflects those updated values
/// validates whether the runtime config reflects those updated values.

Copilot uses AI. Check for mistakes.
Comment on lines +470 to +505
private void ConfigureResponseCompression(IServiceCollection services, RuntimeConfig? runtimeConfig)
{
CompressionLevel compressionLevel = runtimeConfig?.Runtime?.Compression?.Level ?? CompressionOptions.DEFAULT_LEVEL;

// Only configure compression if level is not None
if (compressionLevel == CompressionLevel.None)
{
return;
}

System.IO.Compression.CompressionLevel systemCompressionLevel = compressionLevel switch
{
CompressionLevel.Fastest => System.IO.Compression.CompressionLevel.Fastest,
CompressionLevel.Optimal => System.IO.Compression.CompressionLevel.Optimal,
_ => System.IO.Compression.CompressionLevel.Optimal
};

services.AddResponseCompression(options =>
{
options.EnableForHttps = true;
options.Providers.Add<Microsoft.AspNetCore.ResponseCompression.GzipCompressionProvider>();
options.Providers.Add<Microsoft.AspNetCore.ResponseCompression.BrotliCompressionProvider>();
});

services.Configure<Microsoft.AspNetCore.ResponseCompression.GzipCompressionProviderOptions>(options =>
{
options.Level = systemCompressionLevel;
});

services.Configure<Microsoft.AspNetCore.ResponseCompression.BrotliCompressionProviderOptions>(options =>
{
options.Level = systemCompressionLevel;
});

_logger.LogInformation("Response compression enabled with level '{compressionLevel}' for REST, GraphQL, and MCP endpoints.", compressionLevel);
}
Copy link

Copilot AI Dec 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The response compression middleware lacks integration test coverage. While unit tests verify the CLI configuration, there are no tests validating that:

  1. The compression middleware is actually registered when configured
  2. Responses are compressed when clients send Accept-Encoding headers
  3. Compression is disabled when the level is set to None
  4. Different compression levels (Optimal vs Fastest) behave as expected

Consider adding integration tests similar to CorrelationIdTests.cs that verify the middleware behavior with actual HTTP requests and responses.

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: Review In Progress

Development

Successfully merging this pull request may close these issues.

Test REST over Fabric SQL Databases ⭐ [Enhancement]: Add Compression

2 participants