Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Upgrading from aws-sdk-s3 (1.176.1) to aws-sdk-s3 (1.178.0) breaks uploads to Cloudflare R2 #3174

Closed
1 task done
lylo opened this issue Jan 26, 2025 · 16 comments
Closed
1 task done

Upgrading from aws-sdk-s3 (1.176.1) to aws-sdk-s3 (1.178.0) breaks uploads to Cloudflare R2 #3174

lylo opened this issue Jan 26, 2025 · 16 comments
Labels
guidance Question that needs advice or information. third-party This issue is related to third-party libraries or applications. wontfix We have determined that we will not resolve the issue.

Comments

@lylo
Copy link

lylo commented Jan 26, 2025

Describe the bug

I'm running Rails 8 and using aws-sdk-s3 to manage ActiveStorage uploads to Cloudflare R2. This has been working fine but if I upgrade to v1.178.0 I am seeing the following error:

12:56:01 web.1    | Completed 500 Internal Server Error in 523ms (ActiveRecord: 9.6ms (7 queries, 0 cached) | GC: 11.2ms)
12:56:01 web.1    |
12:56:01 web.1    |
12:56:01 web.1    |
12:56:01 web.1    | Aws::S3::Errors::InvalidRequest (You can only specify one checksum at a time.):
12:56:01 web.1    |
12:56:01 web.1    | aws-sdk-core (3.217.0) lib/seahorse/client/plugins/raise_response_errors.rb:17:in 'Seahorse::Client::Plugins::RaiseResponseErrors::Handler#call'
12:56:01 web.1    | aws-sdk-s3 (1.178.0) lib/aws-sdk-s3/plugins/sse_cpk.rb:24:in 'Aws::S3::Plugins::SseCpk::Handler#call'
12:56:01 web.1    | aws-sdk-s3 (1.178.0) lib/aws-sdk-s3/plugins/dualstack.rb:21:in 'Aws::S3::Plugins::Dualstack::OptionHandler#call'
12:56:01 web.1    | aws-sdk-s3 (1.178.0) lib/aws-sdk-s3/plugins/accelerate.rb:43:in 'Aws::S3::Plugins::Accelerate::OptionHandler#call'
12:56:01 web.1    | aws-sdk-core (3.217.0) lib/aws-sdk-core/plugins/checksum_algorithm.rb:169:in 'Aws::Plugins::ChecksumAlgorithm::OptionHandler#call'
12:56:01 web.1    | aws-sdk-core (3.217.0) lib/aws-sdk-core/plugins/jsonvalue_converter.rb:16:in 'Aws::Plugins::JsonvalueConverter::Handler#call'
12:56:01 web.1    | aws-sdk-core (3.217.0) lib/aws-sdk-core/plugins/invocation_id.rb:16:in 'Aws::Plugins::InvocationId::Handler#call'
12:56:01 web.1    | aws-sdk-core (3.217.0) lib/aws-sdk-core/plugins/idempotency_token.rb:19:in 'Aws::Plugins::IdempotencyToken::Handler#call'
12:56:01 web.1    | aws-sdk-core (3.217.0) lib/aws-sdk-core/plugins/param_converter.rb:26:in 'Aws::Plugins::ParamConverter::Handler#call'
12:56:01 web.1    | aws-sdk-core (3.217.0) lib/seahorse/client/plugins/request_callback.rb:89:in 'Seahorse::Client::Plugins::RequestCallback::OptionHandler#call'
12:56:01 web.1    | aws-sdk-core (3.217.0) lib/aws-sdk-core/plugins/response_paging.rb:12:in 'Aws::Plugins::ResponsePaging::Handler#call'
12:56:01 web.1    | aws-sdk-core (3.217.0) lib/seahorse/client/plugins/response_target.rb:24:in 'Seahorse::Client::Plugins::ResponseTarget::Handler#call'
12:56:01 web.1    | aws-sdk-core (3.217.0) lib/aws-sdk-core/plugins/telemetry.rb:39:in 'block in Aws::Plugins::Telemetry::Handler#call'
12:56:01 web.1    | aws-sdk-core (3.217.0) lib/aws-sdk-core/telemetry/no_op.rb:29:in 'Aws::Telemetry::NoOpTracer#in_span'
12:56:01 web.1    | aws-sdk-core (3.217.0) lib/aws-sdk-core/plugins/telemetry.rb:53:in 'Aws::Plugins::Telemetry::Handler#span_wrapper'
12:56:01 web.1    | aws-sdk-core (3.217.0) lib/aws-sdk-core/plugins/telemetry.rb:39:in 'Aws::Plugins::Telemetry::Handler#call'
12:56:01 web.1    | aws-sdk-core (3.217.0) lib/seahorse/client/request.rb:72:in 'Seahorse::Client::Request#send_request'
12:56:01 web.1    | aws-sdk-s3 (1.178.0) lib/aws-sdk-s3/client.rb:17300:in 'Aws::S3::Client#put_object'
12:56:01 web.1    | aws-sdk-s3 (1.178.0) lib/aws-sdk-s3/object.rb:3005:in 'block in Aws::S3::Object#put'
12:56:01 web.1    | aws-sdk-core (3.217.0) lib/aws-sdk-core/plugins/user_agent.rb:69:in 'Aws::Plugins::UserAgent.metric'
12:56:01 web.1    | aws-sdk-s3 (1.178.0) lib/aws-sdk-s3/object.rb:3004:in 'Aws::S3::Object#put'
12:56:01 web.1    | rails (f1e610ec2053) activestorage/lib/active_storage/service/s3_service.rb:134:in 'ActiveStorage::Service::S3Service#upload_with_single_part'
12:56:01 web.1    | rails (f1e610ec2053) activestorage/lib/active_storage/service/s3_service.rb:33:in 'block in ActiveStorage::Service::S3Service#upload'
12:56:01 web.1    | rails (f1e610ec2053) activesupport/lib/active_support/notifications.rb:210:in 'block in ActiveSupport::Notifications.instrument'
12:56:01 web.1    | rails (f1e610ec2053) activesupport/lib/active_support/notifications/instrumenter.rb:58:in 'ActiveSupport::Notifications::Instrumenter#instrument'
12:56:01 web.1    | rails (f1e610ec2053) activesupport/lib/active_support/notifications.rb:210:in 'ActiveSupport::Notifications.instrument'
12:56:01 web.1    | rails (f1e610ec2053) activestorage/lib/active_storage/service.rb:165:in 'ActiveStorage::Service#instrument'
12:56:01 web.1    | rails (f1e610ec2053) activestorage/lib/active_storage/service/s3_service.rb:29:in 'ActiveStorage::Service::S3Service#upload'
12:56:01 web.1    | rails (f1e610ec2053) activestorage/app/models/active_storage/blob.rb:256:in 'ActiveStorage::Blob#upload_without_unfurling'
12:56:01 web.1    | rails (f1e610ec2053) activestorage/lib/active_storage/attached/changes/create_one.rb:32:in 'ActiveStorage::Attached::Changes::CreateOne#upload'
12:56:01 web.1    | rails (f1e610ec2053) activesupport/lib/active_support/core_ext/object/try.rb:15:in 'Kernel#public_send'
12:56:01 web.1    | rails (f1e610ec2053) activesupport/lib/active_support/core_ext/object/try.rb:15:in 'ActiveSupport::Tryable#try'
12:56:01 web.1    | rails (f1e610ec2053) activestorage/lib/active_storage/attached/model.rb:144:in 'block in ActiveStorage::Attached::Model::ClassMethods#has_one_attached'
12:56:01 web.1    | rails (f1e610ec2053) activesupport/lib/active_support/callbacks.rb:406:in 'BasicObject#instance_exec'
12:56:01 web.1    | rails (f1e610ec2053) activesupport/lib/active_support/callbacks.rb:406:in 'block in ActiveSupport::Callbacks::CallTemplate::InstanceExec0#make_lambda'
12:56:01 web.1    | rails (f1e610ec2053) activesupport/lib/active_support/callbacks.rb:207:in 'ActiveSupport::Callbacks::Filters::After#call'
12:56:01 web.1    | rails (f1e610ec2053) activesupport/lib/active_support/callbacks.rb:563:in 'block in ActiveSupport::Callbacks::CallbackSequence#invoke_after'
12:56:01 web.1    | rails (f1e610ec2053) activesupport/lib/active_support/callbacks.rb:563:in 'Array#each'
12:56:01 web.1    | rails (f1e610ec2053) activesupport/lib/active_support/callbacks.rb:563:in 'ActiveSupport::Callbacks::CallbackSequence#invoke_after'
12:56:01 web.1    | rails (f1e610ec2053) activesupport/lib/active_support/callbacks.rb:110:in 'ActiveSupport::Callbacks#run_callbacks'
12:56:01 web.1    | rails (f1e610ec2053) activesupport/lib/active_support/callbacks.rb:913:in 'ActiveRecord::Base#_run_commit_callbacks'
12:56:01 web.1    | rails (f1e610ec2053) activerecord/lib/active_record/transactions.rb:384:in 'ActiveRecord::Transactions#committed!'
12:56:01 web.1    | rails (f1e610ec2053) activerecord/lib/active_record/connection_adapters/abstract/transaction.rb:307:in 'block in ActiveRecord::ConnectionAdapters::Transaction#commit_records'
12:56:01 web.1    | rails (f1e610ec2053) activerecord/lib/active_record/connection_adapters/abstract/transaction.rb:344:in 'ActiveRecord::ConnectionAdapters::Transaction#run_action_on_records'
12:56:01 web.1    | rails (f1e610ec2053) activerecord/lib/active_record/connection_adapters/abstract/transaction.rb:306:in 'ActiveRecord::ConnectionAdapters::Transaction#commit_records'
12:56:01 web.1    | rails (f1e610ec2053) activerecord/lib/active_record/connection_adapters/abstract/transaction.rb:606:in 'block in ActiveRecord::ConnectionAdapters::TransactionManager#commit_transaction'
12:56:01 web.1    | rails (f1e610ec2053) activesupport/lib/active_support/concurrency/null_lock.rb:9:in 'ActiveSupport::Concurrency::NullLock#synchronize'
12:56:01 web.1    | rails (f1e610ec2053) activerecord/lib/active_record/connection_adapters/abstract/transaction.rb:594:in 'ActiveRecord::ConnectionAdapters::TransactionManager#commit_transaction'
12:56:01 web.1    | rails (f1e610ec2053) activerecord/lib/active_record/connection_adapters/abstract/transaction.rb:638:in 'block in ActiveRecord::ConnectionAdapters::TransactionManager#within_new_transaction'
12:56:01 web.1    | rails (f1e610ec2053) activesupport/lib/active_support/concurrency/null_lock.rb:9:in 'ActiveSupport::Concurrency::NullLock#synchronize'
12:56:01 web.1    | rails (f1e610ec2053) activerecord/lib/active_record/connection_adapters/abstract/transaction.rb:623:in 'ActiveRecord::ConnectionAdapters::TransactionManager#within_new_transaction'
12:56:01 web.1    | rails (f1e610ec2053) activerecord/lib/active_record/connection_adapters/abstract/database_statements.rb:367:in 'ActiveRecord::ConnectionAdapters::DatabaseStatements#within_new_transaction'
12:56:01 web.1    | rails (f1e610ec2053) activerecord/lib/active_record/connection_adapters/abstract/database_statements.rb:359:in 'ActiveRecord::ConnectionAdapters::DatabaseStatements#transaction'
12:56:01 web.1    | rails (f1e610ec2053) activerecord/lib/active_record/transactions.rb:233:in 'block in ActiveRecord::Transactions::ClassMethods#transaction'
12:56:01 web.1    | rails (f1e610ec2053) activerecord/lib/active_record/connection_adapters/abstract/connection_pool.rb:412:in 'ActiveRecord::ConnectionAdapters::ConnectionPool#with_connection'
12:56:01 web.1    | rails (f1e610ec2053) activerecord/lib/active_record/connection_handling.rb:310:in 'ActiveRecord::ConnectionHandling#with_connection'
12:56:01 web.1    | rails (f1e610ec2053) activerecord/lib/active_record/transactions.rb:232:in 'ActiveRecord::Transactions::ClassMethods#transaction'
12:56:01 web.1    | rails (f1e610ec2053) activerecord/lib/active_record/relation/delegation.rb:105:in 'ActiveRecord::Delegation#transaction'
12:56:01 web.1    | rails (f1e610ec2053) activerecord/lib/active_record/relation.rb:290:in 'block in ActiveRecord::Relation#create_or_find_by!'
12:56:01 web.1    | rails (f1e610ec2053) activerecord/lib/active_record/connection_adapters/abstract/connection_pool.rb:418:in 'ActiveRecord::ConnectionAdapters::ConnectionPool#with_connection'
12:56:01 web.1    | rails (f1e610ec2053) activerecord/lib/active_record/connection_handling.rb:310:in 'ActiveRecord::ConnectionHandling#with_connection'
12:56:01 web.1    | rails (f1e610ec2053) activerecord/lib/active_record/relation/delegation.rb:105:in 'ActiveRecord::Delegation#with_connection'
12:56:01 web.1    | rails (f1e610ec2053) activerecord/lib/active_record/relation.rb:289:in 'ActiveRecord::Relation#create_or_find_by!'
12:56:01 web.1    | rails (f1e610ec2053) activestorage/app/models/active_storage/variant_with_record.rb:59:in 'block in ActiveStorage::VariantWithRecord#create_or_find_record'
12:56:01 web.1    | rails (f1e610ec2053) activerecord/lib/active_record/connection_handling.rb:398:in 'ActiveRecord::ConnectionHandling#with_role_and_shard'
12:56:01 web.1    | rails (f1e610ec2053) activerecord/lib/active_record/connection_handling.rb:149:in 'ActiveRecord::ConnectionHandling#connected_to'
12:56:01 web.1    | rails (f1e610ec2053) activestorage/app/models/active_storage/variant_with_record.rb:58:in 'ActiveStorage::VariantWithRecord#create_or_find_record'
12:56:01 web.1    | rails (f1e610ec2053) activestorage/app/models/active_storage/variant_with_record.rb:44:in 'block in ActiveStorage::VariantWithRecord#process'
12:56:01 web.1    | rails (f1e610ec2053) activestorage/app/models/active_storage/variant_with_record.rb:50:in 'block (2 levels) in ActiveStorage::VariantWithRecord#transform_blob'
12:56:01 web.1    | rails (f1e610ec2053) activestorage/lib/active_storage/transformers/transformer.rb:27:in 'ActiveStorage::Transformers::Transformer#transform'
12:56:01 web.1    | rails (f1e610ec2053) activestorage/app/models/active_storage/variation.rb:58:in 'block in ActiveStorage::Variation#transform'
12:56:01 web.1    | rails (f1e610ec2053) activesupport/lib/active_support/notifications.rb:210:in 'block in ActiveSupport::Notifications.instrument'
12:56:01 web.1    | rails (f1e610ec2053) activesupport/lib/active_support/notifications/instrumenter.rb:58:in 'ActiveSupport::Notifications::Instrumenter#instrument'
12:56:01 web.1    | rails (f1e610ec2053) activesupport/lib/active_support/notifications.rb:210:in 'ActiveSupport::Notifications.instrument'
12:56:01 web.1    | rails (f1e610ec2053) activestorage/app/models/active_storage/variation.rb:57:in 'ActiveStorage::Variation#transform'
12:56:01 web.1    | rails (f1e610ec2053) activestorage/app/models/active_storage/variant_with_record.rb:49:in 'block in ActiveStorage::VariantWithRecord#transform_blob'
12:56:01 web.1    | rails (f1e610ec2053) activestorage/lib/active_storage/downloader.rb:15:in 'block in ActiveStorage::Downloader#open'
12:56:01 web.1    | rails (f1e610ec2053) activestorage/lib/active_storage/downloader.rb:24:in 'ActiveStorage::Downloader#open_tempfile'
12:56:01 web.1    | rails (f1e610ec2053) activestorage/lib/active_storage/downloader.rb:12:in 'ActiveStorage::Downloader#open'
12:56:01 web.1    | rails (f1e610ec2053) activestorage/lib/active_storage/service.rb:92:in 'ActiveStorage::Service#open'
12:56:01 web.1    | rails (f1e610ec2053) activestorage/app/models/active_storage/blob.rb:289:in 'ActiveStorage::Blob#open'
12:56:01 web.1    | rails (f1e610ec2053) activestorage/app/models/active_storage/variant_with_record.rb:48:in 'ActiveStorage::VariantWithRecord#transform_blob'
12:56:01 web.1    | rails (f1e610ec2053) activestorage/app/models/active_storage/variant_with_record.rb:44:in 'ActiveStorage::VariantWithRecord#process'
12:56:01 web.1    | rails (f1e610ec2053) activestorage/app/models/active_storage/variant_with_record.rb:19:in 'ActiveStorage::VariantWithRecord#processed'
12:56:01 web.1    | rails (f1e610ec2053) activestorage/app/controllers/active_storage/representations/base_controller.rb:14:in 'ActiveStorage::Representations::BaseController#set_representation'
12:56:01 web.1    | rails (f1e610ec2053) activesupport/lib/active_support/callbacks.rb:361:in 'block in ActiveSupport::Callbacks::CallTemplate::MethodCall#make_lambda'
12:56:01 web.1    | rails (f1e610ec2053) activesupport/lib/active_support/callbacks.rb:178:in 'block in ActiveSupport::Callbacks::Filters::Before#call'
12:56:01 web.1    | rails (f1e610ec2053) actionpack/lib/abstract_controller/callbacks.rb:34:in 'block (2 levels) in <module:Callbacks>'
12:56:01 web.1    | rails (f1e610ec2053) activesupport/lib/active_support/callbacks.rb:179:in 'ActiveSupport::Callbacks::Filters::Before#call'
12:56:01 web.1    | rails (f1e610ec2053) activesupport/lib/active_support/callbacks.rb:559:in 'block in ActiveSupport::Callbacks::CallbackSequence#invoke_before'
12:56:01 web.1    | rails (f1e610ec2053) activesupport/lib/active_support/callbacks.rb:559:in 'Array#each'
12:56:01 web.1    | rails (f1e610ec2053) activesupport/lib/active_support/callbacks.rb:559:in 'ActiveSupport::Callbacks::CallbackSequence#invoke_before'
12:56:01 web.1    | rails (f1e610ec2053) activesupport/lib/active_support/callbacks.rb:118:in 'block in ActiveSupport::Callbacks#run_callbacks'
12:56:01 web.1    | turbo-rails (2.0.11) lib/turbo-rails.rb:24:in 'Turbo.with_request_id'
12:56:01 web.1    | turbo-rails (2.0.11) app/controllers/concerns/turbo/request_id_tracking.rb:10:in 'Turbo::RequestIdTracking#turbo_tracking_request_id'
12:56:01 web.1    | rails (f1e610ec2053) activesupport/lib/active_support/callbacks.rb:129:in 'block in ActiveSupport::Callbacks#run_callbacks'
12:56:01 web.1    | rails (f1e610ec2053) actiontext/lib/action_text/rendering.rb:24:in 'ActionText::Rendering::ClassMethods#with_renderer'
12:56:01 web.1    | rails (f1e610ec2053) actiontext/lib/action_text/engine.rb:71:in 'block (4 levels) in <class:Engine>'
12:56:01 web.1    | rails (f1e610ec2053) activesupport/lib/active_support/callbacks.rb:129:in 'BasicObject#instance_exec'
12:56:01 web.1    | rails (f1e610ec2053) activesupport/lib/active_support/callbacks.rb:129:in 'block in ActiveSupport::Callbacks#run_callbacks'
12:56:01 web.1    | rails (f1e610ec2053) activesupport/lib/active_support/callbacks.rb:140:in 'ActiveSupport::Callbacks#run_callbacks'
12:56:01 web.1    | rails (f1e610ec2053) actionpack/lib/abstract_controller/callbacks.rb:260:in 'AbstractController::Callbacks#process_action'
12:56:01 web.1    | rails (f1e610ec2053) actionpack/lib/action_controller/metal/rescue.rb:27:in 'ActionController::Rescue#process_action'
12:56:01 web.1    | rails (f1e610ec2053) actionpack/lib/action_controller/metal/instrumentation.rb:76:in 'block in ActionController::Instrumentation#process_action'
12:56:01 web.1    | rails (f1e610ec2053) activesupport/lib/active_support/notifications.rb:210:in 'block in ActiveSupport::Notifications.instrument'
12:56:01 web.1    | rails (f1e610ec2053) activesupport/lib/active_support/notifications/instrumenter.rb:58:in 'ActiveSupport::Notifications::Instrumenter#instrument'
12:56:01 web.1    | rails (f1e610ec2053) activesupport/lib/active_support/notifications.rb:210:in 'ActiveSupport::Notifications.instrument'
12:56:01 web.1    | rails (f1e610ec2053) actionpack/lib/action_controller/metal/instrumentation.rb:75:in 'ActionController::Instrumentation#process_action'
12:56:01 web.1    | rails (f1e610ec2053) actionpack/lib/action_controller/metal/params_wrapper.rb:259:in 'ActionController::ParamsWrapper#process_action'
12:56:01 web.1    | rails (f1e610ec2053) activerecord/lib/active_record/railties/controller_runtime.rb:39:in 'ActiveRecord::Railties::ControllerRuntime#process_action'
12:56:01 web.1    | rails (f1e610ec2053) actionpack/lib/abstract_controller/base.rb:167:in 'AbstractController::Base#process'
12:56:01 web.1    | rails (f1e610ec2053) actionview/lib/action_view/rendering.rb:40:in 'ActionView::Rendering#process'
12:56:01 web.1    | rails (f1e610ec2053) actionpack/lib/action_controller/metal.rb:252:in 'ActionController::Metal#dispatch'
12:56:01 web.1    | rails (f1e610ec2053) actionpack/lib/action_controller/metal.rb:335:in 'ActionController::Metal.dispatch'
12:56:01 web.1    | rails (f1e610ec2053) actionpack/lib/action_dispatch/routing/route_set.rb:67:in 'ActionDispatch::Routing::RouteSet::Dispatcher#dispatch'
12:56:01 web.1    | rails (f1e610ec2053) actionpack/lib/action_dispatch/routing/route_set.rb:50:in 'ActionDispatch::Routing::RouteSet::Dispatcher#serve'
12:56:01 web.1    | rails (f1e610ec2053) actionpack/lib/action_dispatch/journey/router.rb:48:in 'block in ActionDispatch::Journey::Router#serve'
12:56:01 web.1    | rails (f1e610ec2053) actionpack/lib/action_dispatch/journey/router.rb:128:in 'block in ActionDispatch::Journey::Router#find_routes'
12:56:01 web.1    | rails (f1e610ec2053) actionpack/lib/action_dispatch/journey/router.rb:121:in 'Array#each'
12:56:01 web.1    | rails (f1e610ec2053) actionpack/lib/action_dispatch/journey/router.rb:121:in 'ActionDispatch::Journey::Router#find_routes'
12:56:01 web.1    | rails (f1e610ec2053) actionpack/lib/action_dispatch/journey/router.rb:29:in 'ActionDispatch::Journey::Router#serve'
12:56:01 web.1    | rails (f1e610ec2053) actionpack/lib/action_dispatch/routing/route_set.rb:908:in 'ActionDispatch::Routing::RouteSet#call'
12:56:01 web.1    | rack (3.1.8) lib/rack/tempfile_reaper.rb:20:in 'Rack::TempfileReaper#call'
12:56:01 web.1    | rack (3.1.8) lib/rack/etag.rb:29:in 'Rack::ETag#call'
12:56:01 web.1    | rack (3.1.8) lib/rack/conditional_get.rb:31:in 'Rack::ConditionalGet#call'
12:56:01 web.1    | rack (3.1.8) lib/rack/head.rb:15:in 'Rack::Head#call'
12:56:01 web.1    | rails (f1e610ec2053) actionpack/lib/action_dispatch/http/content_security_policy.rb:38:in 'ActionDispatch::ContentSecurityPolicy::Middleware#call'
12:56:01 web.1    | rack-session (2.1.0) lib/rack/session/abstract/id.rb:274:in 'Rack::Session::Abstract::Persisted#context'
12:56:01 web.1    | rack-session (2.1.0) lib/rack/session/abstract/id.rb:268:in 'Rack::Session::Abstract::Persisted#call'
12:56:01 web.1    | rails (f1e610ec2053) actionpack/lib/action_dispatch/middleware/cookies.rb:706:in 'ActionDispatch::Cookies#call'
12:56:01 web.1    | rails (f1e610ec2053) activerecord/lib/active_record/migration.rb:672:in 'ActiveRecord::Migration::CheckPending#call'
12:56:01 web.1    | rails (f1e610ec2053) actionpack/lib/action_dispatch/middleware/callbacks.rb:31:in 'block in ActionDispatch::Callbacks#call'
12:56:01 web.1    | rails (f1e610ec2053) activesupport/lib/active_support/callbacks.rb:100:in 'ActiveSupport::Callbacks#run_callbacks'
12:56:01 web.1    | rails (f1e610ec2053) actionpack/lib/action_dispatch/middleware/callbacks.rb:30:in 'ActionDispatch::Callbacks#call'
12:56:01 web.1    | rails (f1e610ec2053) actionpack/lib/action_dispatch/middleware/executor.rb:16:in 'ActionDispatch::Executor#call'
12:56:01 web.1    | rails (f1e610ec2053) actionpack/lib/action_dispatch/middleware/actionable_exceptions.rb:18:in 'ActionDispatch::ActionableExceptions#call'
12:56:01 web.1    | sentry-rails (5.22.1) lib/sentry/rails/rescued_exception_interceptor.rb:11:in 'Sentry::Rails::RescuedExceptionInterceptor#call'
12:56:01 web.1    | rails (f1e610ec2053) actionpack/lib/action_dispatch/middleware/debug_exceptions.rb:31:in 'ActionDispatch::DebugExceptions#call'
12:56:01 web.1    | web-console (4.2.1) lib/web_console/middleware.rb:132:in 'WebConsole::Middleware#call_app'
12:56:01 web.1    | web-console (4.2.1) lib/web_console/middleware.rb:28:in 'block in WebConsole::Middleware#call'
12:56:01 web.1    | web-console (4.2.1) lib/web_console/middleware.rb:17:in 'Kernel#catch'
12:56:01 web.1    | web-console (4.2.1) lib/web_console/middleware.rb:17:in 'WebConsole::Middleware#call'
12:56:01 web.1    | sentry-ruby (5.22.1) lib/sentry/rack/capture_exceptions.rb:15:in 'Sentry::Rack::CaptureExceptions#call'
12:56:01 web.1    | rails (f1e610ec2053) actionpack/lib/action_dispatch/middleware/show_exceptions.rb:32:in 'ActionDispatch::ShowExceptions#call'
12:56:01 web.1    | rails (f1e610ec2053) railties/lib/rails/rack/logger.rb:41:in 'Rails::Rack::Logger#call_app'
12:56:01 web.1    | rails (f1e610ec2053) railties/lib/rails/rack/logger.rb:29:in 'Rails::Rack::Logger#call'
12:56:01 web.1    | sprockets-rails (3.5.2) lib/sprockets/rails/quiet_assets.rb:17:in 'Sprockets::Rails::QuietAssets#call'
12:56:01 web.1    | rails (f1e610ec2053) actionpack/lib/action_dispatch/middleware/remote_ip.rb:96:in 'ActionDispatch::RemoteIp#call'
12:56:01 web.1    | rails (f1e610ec2053) actionpack/lib/action_dispatch/middleware/request_id.rb:34:in 'ActionDispatch::RequestId#call'
12:56:01 web.1    | rack (3.1.8) lib/rack/method_override.rb:28:in 'Rack::MethodOverride#call'
12:56:01 web.1    | rack (3.1.8) lib/rack/runtime.rb:24:in 'Rack::Runtime#call'
12:56:01 web.1    | rails (f1e610ec2053) activesupport/lib/active_support/cache/strategy/local_cache_middleware.rb:29:in 'ActiveSupport::Cache::Strategy::LocalCache::Middleware#call'
12:56:01 web.1    | rails (f1e610ec2053) actionpack/lib/action_dispatch/middleware/server_timing.rb:61:in 'block in ActionDispatch::ServerTiming#call'
12:56:01 web.1    | rails (f1e610ec2053) actionpack/lib/action_dispatch/middleware/server_timing.rb:26:in 'ActionDispatch::ServerTiming::Subscriber#collect_events'
12:56:01 web.1    | rails (f1e610ec2053) actionpack/lib/action_dispatch/middleware/server_timing.rb:60:in 'ActionDispatch::ServerTiming#call'
12:56:01 web.1    | rails (f1e610ec2053) actionpack/lib/action_dispatch/middleware/executor.rb:16:in 'ActionDispatch::Executor#call'
12:56:01 web.1    | rails (f1e610ec2053) actionpack/lib/action_dispatch/middleware/static.rb:27:in 'ActionDispatch::Static#call'
12:56:01 web.1    | rack (3.1.8) lib/rack/sendfile.rb:114:in 'Rack::Sendfile#call'
12:56:01 web.1    | rails (f1e610ec2053) actionpack/lib/action_dispatch/middleware/host_authorization.rb:143:in 'ActionDispatch::HostAuthorization#call'
12:56:01 web.1    | rails (f1e610ec2053) railties/lib/rails/engine.rb:534:in 'Rails::Engine#call'
12:56:01 web.1    | puma (6.5.0) lib/puma/configuration.rb:279:in 'Puma::Configuration::ConfigMiddleware#call'
12:56:01 web.1    | puma (6.5.0) lib/puma/request.rb:99:in 'block in Puma::Request#handle_request'
12:56:01 web.1    | puma (6.5.0) lib/puma/thread_pool.rb:389:in 'Puma::ThreadPool#with_force_shutdown'
12:56:01 web.1    | puma (6.5.0) lib/puma/request.rb:98:in 'Puma::Request#handle_request'
12:56:01 web.1    | puma (6.5.0) lib/puma/server.rb:468:in 'Puma::Server#process_client'
12:56:01 web.1    | puma (6.5.0) lib/puma/server.rb:249:in 'block in Puma::Server#run'
12:56:01 web.1    | puma (6.5.0) lib/puma/thread_pool.rb:166:in 'block in Puma::ThreadPool#spawn_thread'

Regression Issue

  • Select this option if this issue appears to be a regression.

Expected Behavior

I expect the file to be uploaded to R2 without errors.

Current Behavior

The file is not uploaded to R2 and the aforementioned error is raised.

Reproduction Steps

I am using standard ActiveStorage configured to use Cloudflare R2.

cloudflare:
  service: s3
  endpoint: https://[ENDPOINT-ID].r2.cloudflarestorage.com
  access_key_id: <%= ENV["CLOUDFLARE_R2_ACCESS_KEY_ID"] %>
  secret_access_key: <%= ENV["CLOUDFLARE_R2_SECRET_ACCESS_KEY"] %>
  region: auto
  bucket: my-bucket

Possible Solution

No response

Additional Information/Context

No response

Gem name ('aws-sdk', 'aws-sdk-resources' or service gems like 'aws-sdk-s3') and its version

aws-sdk-s3

Environment details (Version of Ruby, OS environment)

Ruby 3.4.1. macOS 15, Ubuntu 22.
@lylo lylo added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Jan 26, 2025
@github-actions github-actions bot added the potential-regression Marking this issue as a potential regression to be checked by team member label Jan 26, 2025
@lylo
Copy link
Author

lylo commented Jan 26, 2025

I noticed the following recent commit regarding checksums: df86be6

@mullermp
Copy link
Contributor

Thanks for opening an issue. The AWS SDKs are intended to be used with AWS services and not 3rd party services. I am not sure of the internals of R2. The error seems to indicate you are calculating more than one checksum. Perhaps you should open an issue in ActiveStorage or R2 instead. You can try setting the client :request_checksum_calculation value to WHEN_REQUIRED to not calculate a checksum and if you need to calculate MD5 which is no longer calculated, you can insert a plugin like this: #3167 (comment)

@mullermp mullermp added guidance Question that needs advice or information. response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days. third-party This issue is related to third-party libraries or applications. wontfix We have determined that we will not resolve the issue. and removed bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. potential-regression Marking this issue as a potential regression to be checked by team member labels Jan 26, 2025
@lylo
Copy link
Author

lylo commented Jan 26, 2025

Thanks @mullermp, it was always a long shot but I thought I'd file the issue just in case there was an obvious answer. I appreciate the library is intended for S3, but there are many "S3 compatible" object stores which people are using with this gem.

I'll see what Cloudflare say but for now I'll have to pin to the previous version which works completely fine.

@mullermp
Copy link
Contributor

I understand. This was a deliberate decision from S3 (see #3166) and I was simply the monkey that wrote the code.

@lylo
Copy link
Author

lylo commented Jan 26, 2025

I understand. This was a deliberate decision from S3 (see #3166) and I was simply the monkey that wrote the code.

Understood! Thank you for all you do!! 🙏

@lylo
Copy link
Author

lylo commented Jan 26, 2025

https://www.cloudflarestatus.com/incidents/t5nrjmpxc1cj

@github-actions github-actions bot removed the response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days. label Jan 27, 2025
@danielfriis
Copy link

For anyone else struggling with this, pinning both aws-sdk-s3 and aws-sdk-core to a lower version, fixes it for now.

@richjdsmith richjdsmith mentioned this issue Jan 28, 2025
Closed
@mullermp
Copy link
Contributor

I'm going to close this as there's no action from us. I believe the work around would be to set request_checksum_calculation to WHEN_REQUIRED if you are able to configure an instance of the S3 client. You can also try export AWS_REQUEST_CHECKSUM_CALCULATION=WHEN_REQUIRED environment variable. These should turn off optional checksum calculation scenarios.

@github-actions GitHub Actions
Copy link

This issue is now closed. Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.

@collimarco
Copy link

Has anyone found a solution for this?

@lylo
Copy link
Author

lylo commented Feb 4, 2025

Has anyone found a solution for this?

Looks like it got fixed permanently yesterday @collimarco

https://www.cloudflarestatus.com/incidents/t5nrjmpxc1cj

@collimarco
Copy link

@lylo Unfortunately no, it's not fixed for R2, I got the error today (a few hours ago) after upgrading this gem

@lylo
Copy link
Author

lylo commented Feb 4, 2025

@collimarco I believe you can set these values

Image

@phil-6
Copy link

phil-6 commented Feb 5, 2025

Cloudflare have added a compatibility section at the top of their aws-sdk-ruby documentation.

https://developers.cloudflare.com/r2/examples/aws/aws-sdk-ruby/

If you're using active storage, these can be set in the storage.yml configuration file for each bucket.

@a-chris
Copy link

a-chris commented Feb 11, 2025

@phil-6 what was the suggested compatibility fixes?

I don't see anything like that in https://developers.cloudflare.com/r2/examples/aws/aws-sdk-ruby/ which seems to have been edited today Last updated: Feb 11, 2025

@phil-6
Copy link

phil-6 commented Feb 11, 2025

Yeah, looks like they removed it, maybe it's not needed with their latest updates. 🤷

We added

  request_checksum_calculation: "when_required"
  response_checksum_validation: "when_required"

To the service configuration (in config/storage.rb in our rails app)

thibaudgg added a commit to csa-admin-org/csa-admin that referenced this issue Mar 1, 2025
@thibaudgg
Fix aws-sdk-s3 issue with Cloudflare R2
3ba5dc8 
aws/aws-sdk-ruby#3174
amree added a commit to petakopi/petakopi.my that referenced this issue Mar 8, 2025
@amree
Fix issue with ActiveStorage and Object Storage (R2)
606b625 
See:
- rails/rails#54374
- aws/aws-sdk-ruby#3174
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
guidance Question that needs advice or information. third-party This issue is related to third-party libraries or applications. wontfix We have determined that we will not resolve the issue.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@lylo @collimarco @danielfriis @phil-6 @a-chris @mullermp