Improve examples using Async::HTTP and Falcon

[trevorturk]

Issues hit when upgrading my existing Rails app:

• Protocol::HTTP1::BadRequest with old VCR cassettes Unexpected Protocol::HTTP1::BadRequest exception protocol-http1#11
• Problematic Hop-by-hop Headers in VCR Consider ignoring Hop-by-hop Headers such as Transfer-Encoding vcr/vcr#850 [discussion created]
• VCR appending to existing cassettes when record none set: Async http adapter: only consider real requests to be real bblimke/webmock#931

Need examples and/or documentation for the proof of concept sample apps in this repo:

• Attempting to use async-await but running into trouble: async-await (wip) #3
• Attempting to use lazy_resolve but not sure about the semaphore(s) graphql-ruby lazy_resolve, Async::Semaphore #4
• How to enable persistent http connections to multiple hosts, if possible/automatic, do we need internet.close (?)
• Can we use Ruby 3.0 or 3.1 when available? (IIRC Heroku doesn't support 3.0, so I'm working on 2.7 now)
• Review rails-async-http-falcon-graphql-batch -- is Sync the appropriate here? Do we need Async::Barrier?
• Example to replace Typhoeus#total_time, should use Async::Clock.measure or something manual (?)
• Example of params in a hash like https://github.com/typhoeus/typhoeus#introduction
• Docs PR re async/http/internet/instance Example using async/http/internet/instance async-http#72
• Docs PR re automatic gzip handling Demo using Protocol::HTTP::AcceptEncoding async-http#73 (needs help)
• How to manage preloading (I was able to use the Bundler group :preload, don't understand preload.rb)
• How to configure for Heroku, see rack-async-http-falcon-graphql-batch and falcon hosting on heroku using falcon.rb falcon#121 (comment)

Feature requests:

• Merge wait_all re: Async::Task#wait_all async#89 if/when acceptable (don't need this, can use barriers)
• Automatic support for gzip rack-async-http-falcon-gzip #7 -> Consider Async::HTTP::Internet should use Protocol::HTTP::AcceptEncoding by default. async-http#68
• Consider adding debug logging re automatic gzip handling etc Potentially add verbose logging to Async::HTTP::Internet async-http#74
• Monitor Support for non-blocking fibers puma/puma#2601 re: rack-async-http-puma-graphql (not expected to work currently?)
• Investigate rack-async-http-falcon-graphql-dataloader (should work in Ruby 3.1? Can we make it work now?)
• Improved handling of connection pools Performance issue with version 0.3.5 of async-pool async-http#67
• Add Redis support to https://github.com/socketry/async-http-cache instead of just in-memory

For the examples/documentation issues, I'd like to work in the open and/or contribute back to the relevant projects' documentation wherever possible. I'm finding the async suite of tools to be lovely, but the documentation sparse, and I'm hoping to contribute back in an effort to increase adoption.

For the issues found while upgrading my existing Rails app, I'll need to keep that source code private, but happy to work in the open wherever possible, as illustrated by the VCR cassette issue: socketry/protocol-http1#11 since obviously most of these concerns can be generalized.

[josh]

Other approaches more than welcome, favorite so far is rails-async-http-falcon-graphql-batch

Another idea is to return Async::Task objects from resolvers. Then register that class with the GraphQL::Schema.

class Schema < GraphQL::Schema
  query Query
  lazy_resolve Async::Task, :wait
end

Resolver methods can now just return an Async::Task:

def one
  Async {
    response = internet.get("https://httpbin.org/delay/1").read
    JSON.parse(response)["url"]
  }
end

You could probably take things further by injecting something that wraps all resolvers with Async {} to remove that step.

This approach may mean you can avoid the graphql-batch dependency all together.

[trevorturk]

Thanks for the note, @josh -- I'll try to work up an example app using this style for reference today and report back.

I'm also trying working up an example suggested by @ioquatix #3 using async-await but I ran into some trouble there. I'm sure I'm just misunderstanding how this all works, but perhaps this can be a good strategy to avoid graphql batch loaders, which really shouldn't be necessary in my app's use case. (Still, I think the more examples we can provide people the better.)

[trevorturk]

@josh see #4 for my attempt to implement your suggestion. The second commit doesn't work, but I'm not understanding why, but I think this seems like a good approach to avoid the batch loader which isn't really necessary in this use case with the Async::HTTP stuff.

[ioquatix]

How to enable Accept-Encoding: gzip #7 (?) multiple headers?

We have discussed this and I think this is appropriate for default inclusion into Async::HTTP::Internet. See socketry/async-http#68

[ioquatix]

We also discussed improving this: socketry/async-http#67

[ioquatix]

Investigate rack-async-http-falcon-graphql-dataloader (should work in Ruby 3.1?)

Investigate how the fiber interface is supposed to be used.

[trevorturk]

I think we covered everything! I have some notes to follow up with docs re preloading and Heroku falcon.rb config once I sort those things out for myself. Thank you again for all of your help so far!