Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow passing extra connection kwargs with host address #337

Closed

Conversation

marcinowski
Copy link

This should fix some of the issues with #331 by allowing to pass extra connection arguments with the host configuration.
The specific case that this helps me with is passing the ssl kwargs into the connection as advised by Heroku redis docs

@marcinowski marcinowski changed the title Allow passing extra connection kwargs with address Allow passing extra connection kwargs with host address Nov 2, 2022
@nasir733
Copy link

nasir733 commented Nov 7, 2022

@carltongibson what do you think about this pull request?

@carltongibson
Copy link
Member

@nasir733 I haven't had a chance to review it yes, otherwise I'd have commented. (@mentioning me doesn't help. I feel we've discussed this before... 😅)

If you want to give it a run, verify it works (or not) for you and comment along those lines, then that would be a useful thing to do. (If it does work you're then able to use this branch whilst waiting for a release.)

@bbrowning918
Copy link
Contributor

Again with test hangs, I helped fight these not too long ago. I think with #342 also fixing pubsub's connection the two can be combined into one.

@jackkinsella
Copy link

jackkinsella commented Nov 29, 2022

The approach in my PR seemed to just replace this issue with another: TypeError: Connection.__init__() got an unexpected keyword argument 'ssl' - however this might be because I am also using rq`

File "/app/.heroku/python/lib/python3.11/site-packages/channels/routing.py", line 116, in __call__
app[web.1]: return await application(
File "/app/.heroku/python/lib/python3.11/site-packages/channels/consumer.py", line 94, in app
return await consumer(scope, receive, send)
File "/app/.heroku/python/lib/python3.11/site-packages/channels/consumer.py", line 58, in __call__
await await_many_dispatch(
File "/app/.heroku/python/lib/python3.11/site-packages/channels/utils.py", line 57, in await_many_dispatch
await task
File "/app/.heroku/src/channels-redis/channels_redis/core.py", line 368, in receive
message_channel, message = await self.receive_single(
File "/app/.heroku/src/channels-redis/channels_redis/core.py", line 423, in receive_single
content = await self._brpop_with_clean(
File "/app/.heroku/src/channels-redis/channels_redis/core.py", line 259, in _brpop_with_clean
await connection.eval(cleanup_script, 0, channel, backup_queue)
File "/app/.heroku/python/lib/python3.11/site-packages/redis/asyncio/client.py", line 484, in execute_command
conn = self.connection or await pool.get_connection(command_name, **options)
File "/app/.heroku/python/lib/python3.11/site-packages/redis/asyncio/connection.py", line 1520, in get_connection
connection = self.make_connection()
File "/app/.heroku/python/lib/python3.11/site-packages/redis/asyncio/connection.py", line 1560, in make_connection
return self.connection_class(**self.connection_kwargs)
File "/app/.heroku/python/lib/python3.11/site-packages/redis/asyncio/connection.py", line 1088, in __init__
super().__init__(**kwargs)
TypeError: Connection.__init__() got an unexpected keyword argument 'ssl'

Perhaps a parallel with rq/rq#1383

@@ -130,7 +130,8 @@ def create_pool(self, index):
host = self.hosts[index]

if "address" in host:
return aioredis.ConnectionPool.from_url(host["address"])
address = host.pop("address")
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For this to work, we shouldn't mutate the original dict. We need to deepcopy(host), and mutate that here.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also see #341 which addresses the same mutation issue for the branch below.

It might make sense to just copy.deepcopy() on line 130 to avoid the risk of mutation in the downstream redis library too.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should also be addressed in the pubsub layer (note that pubsub already performs a copy of host dict in RedisSingleShardConnection init):

pool = aioredis.ConnectionPool.from_url(self.host["address"])

@carltongibson
Copy link
Member

Resolved in #352. Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

8 participants