Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Read timeout with Autogen Bedrock implementation in v0.2.40 #5505

Open
akshay20t opened this issue Feb 12, 2025 · 4 comments · May be fixed by #5529
Open

Read timeout with Autogen Bedrock implementation in v0.2.40 #5505

akshay20t opened this issue Feb 12, 2025 · 4 comments · May be fixed by #5529
Labels
0.2 Issues which are related to the pre 0.4 codebase

Comments

@akshay20t
Copy link

We are facing read timeout error while using the autogen bedrock implementation with large input tokens. The default read timeout in boto3 configuration is 60 seconds and autogen implementation doesn't allow any modification to this value while initializing the BedrockClient class.

Here is the error message:

RuntimeError: Failed to get response from Bedrock: Read timeout on endpoint URL: "https://bedrock-runtime.us-west-2.amazonaws.com/model/us.anthropic.claude-3-5-sonnet-20241022-v2%3A0/converse"

We can see that the __init__ method for class BedrockClient does accept variable keyword arguments however the same is not being used while defining the variable bedrock_config within this method.

def __init__(self, **kwargs: Any):
        """
        Initialises BedrockClient for Amazon's Bedrock Converse API
        """
        self._aws_access_key = kwargs.get("aws_access_key", None)
        self._aws_secret_key = kwargs.get("aws_secret_key", None)
        self._aws_session_token = kwargs.get("aws_session_token", None)
        self._aws_region = kwargs.get("aws_region", None)
        self._aws_profile_name = kwargs.get("aws_profile_name", None)

        if not self._aws_access_key:
            self._aws_access_key = os.getenv("AWS_ACCESS_KEY_ID")

        if not self._aws_secret_key:
            self._aws_secret_key = os.getenv("AWS_SECRET_ACCESS_KEY")

        if not self._aws_session_token:
            self._aws_session_token = os.getenv("AWS_SESSION_TOKEN")

        if not self._aws_region:
            self._aws_region = os.getenv("AWS_DEFAULT_REGION")

        if self._aws_region is None:
            raise ValueError("Region is required to use the Amazon Bedrock API.")

        # Initialize Bedrock client, session, and runtime
        bedrock_config = Config(
            region_name=self._aws_region,
            signature_version="v4",
            retries={"max_attempts": self._retries, "mode": "standard"},
        )

        session = boto3.Session(
            aws_access_key_id=self._aws_access_key,
            aws_secret_access_key=self._aws_secret_key,
            aws_session_token=self._aws_session_token,
            profile_name=self._aws_profile_name,
        )

        self.bedrock_runtime = session.client(service_name="bedrock-runtime", config=bedrock_config)
@akshay20t akshay20t changed the title Read timeout with Autogen Bedrock implementation Read timeout with Autogen Bedrock implementation in v0.2.40 Feb 12, 2025
@ekzhu
Copy link
Collaborator

ekzhu commented Feb 12, 2025

Thanks for reporting the issue. Would you like to submit a PR to fix it?

@akshay20t
Copy link
Author

akshay20t commented Feb 13, 2025

@ekzhu - I have raised a PR #5529 to implement fix for this issue. Kindly let me know if any further details are required from my side.

If everything looks good based on the review of this PR, can you please help us understand on what's the plan to make this fix available via release for autogen 0.2.x version.

@ekzhu
Copy link
Collaborator

ekzhu commented Feb 13, 2025

Once the fix gets merged we will create a new v0.2 release.

@akshay20t
Copy link
Author

Thanks @ekzhu for the confirmation!

@jackgerrits jackgerrits added 0.2 Issues which are related to the pre 0.4 codebase and removed needs-triage labels Feb 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
0.2 Issues which are related to the pre 0.4 codebase
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants