[Opentelemetry] Enable async-graphql opentelemetry auto instrumentation

[meskill]

Description

async-graphql crate provides integration with opentelemetry out of the box through extension.

But the provided integration has the issue with number of data that is generated and how it's affect the performance.

On the image you can see that the spans are generated for every field in tailcall, even if they don't have resolvers and therefore don't provide any valuable info about.

The performance impact is following:

no opentelemetry

Running 30s test @ http://localhost:8000/graphql
  8 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     3.04ms    1.72ms 108.91ms   74.95%
    Req/Sec     4.01k   478.71     8.76k    76.95%
  963599 requests in 32.98s, 4.83GB read
  Socket errors: connect 0, read 0, write 0, timeout 96
Requests/sec:  29221.23
Transfer/sec:    149.98MB

opentelemetry enabled with no async_graphql extension

Running 30s test @ http://localhost:8000/graphql
  8 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     3.13ms    1.60ms  34.92ms   70.23%
    Req/Sec     3.88k   416.51    12.10k    90.92%
  933190 requests in 33.00s, 4.68GB read
  Socket errors: connect 0, read 0, write 0, timeout 96
Requests/sec:  28277.76
Transfer/sec:    145.14MB

opentelemetry enabled + async_graphql extension

Running 30s test @ http://localhost:8000/graphql
  8 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     4.72ms    2.39ms  49.07ms   68.68%
    Req/Sec     2.57k   264.10     7.91k    93.11%
  617038 requests in 32.99s, 3.09GB read
  Socket errors: connect 0, read 0, write 0, timeout 96
Requests/sec:  18705.06
Transfer/sec:     96.01MB

Solution

Write our own implementation of async_graphql's extension for opentelemetry that will generate spans only for valuable data (i.e. for fields with resolvers).

As references explore suggestions from async-graphql/async-graphql#1395 (comment) async-graphql/examples#59 (comment) in order to implement this

[meskill]

#924 provided tracing data that is more related to tailcall logic.

Closing this in favour of current implementation until we require something additional from the plugin like this