Deepgram Voice Agent on Amazon Bedrock AgentCore with Daily

This example demonstrates a Pipecat voice agent deployed to Amazon Bedrock AgentCore Runtime using Daily as the WebRTC transport. The pipeline uses Deepgram for both speech-to-text and text-to-speech, Amazon Nova for the LLM, Silero for voice activity detection, and AgentCore Memory for persistent conversation memory across sessions.

All infrastructure is defined as CDK in an AWS Amplify Gen 2 project. A single npx ampx sandbox command deploys everything: VPC, AgentCore Memory, AgentCore Runtime, Lambda function, AppSync GraphQL API, authentication, and a hosted React frontend.

Users visit the website, sign in with email OTP (passwordless), click "Start Chat," and get connected to a voice agent via a Daily WebRTC room.

Note: This example focuses on illustrating how to get a Pipecat bot running as an agent in AgentCore Runtime with Deepgram and persistent memory. It does not address various production-readiness concerns, including but not limited to: sanitized logging, rate limiting, CORS tightening, and input validation. Be sure to address these before deploying to production.

Architecture

flowchart LR
    User([User])

    subgraph Frontend
        App[Amplify React Frontend]
        Auth[Cognito Auth<br/>Email OTP / Passkey]
    end

    subgraph AWS[AWS Backend]
        AppSync[AppSync<br/>GraphQL API]
        Lambda[Lambda<br/>Function]
        DailyAPI[Daily API]
    end

    DailyRoom[Daily<br/>WebRTC Room]

    subgraph AgentLayer[" "]
        direction TB
        subgraph VPC[VPC + NAT Gateway]
            subgraph AC[AgentCore Runtime]
                subgraph Pipecat[Pipecat Framework]
                    SileroVAD[Silero VAD]
                end
            end
            Memory[AgentCore<br/>Memory]
            Strategies[Summaries<br/>Preferences<br/>Semantic Facts]
        end
        subgraph Deepgram[Deepgram Hosted API]
            STT[Deepgram STT]
            TTS[Deepgram TTS]
        end
        subgraph Bedrock[Amazon Bedrock]
            Nova[Nova LLM]
        end
        Deepgram ~~~ Bedrock
    end

    User --> App
    App <--> Auth
    App -->|startSession| AppSync
    AppSync --> Lambda
    Lambda -->|Creates room| DailyAPI
    DailyAPI -.->|room_url| Lambda
    Lambda -->|room_url + user_id| AC
    Lambda -.->|room_url| App

    App -->|Opens room| DailyRoom
    User <-->|WebRTC audio| DailyRoom
    Pipecat <-->|Daily transport| DailyRoom

    Pipecat <--> STT
    Pipecat <--> TTS
    Pipecat <--> Nova

    Pipecat -->|Load memories| Memory
    Pipecat -->|Save conversation| Memory
    Memory --> Strategies

Loading

Prerequisites

• Accounts with:
  • AWS (with permissions for Amplify, Bedrock AgentCore, VPC, Lambda)
  • Daily (API key)
  • Deepgram (API key)
• Node.js 18+ and npm
• AWS CLI configured (aws configure or environment variables)

Project Structure

agent/                                # AgentCore bot (Python)
  pipecat-agent.py                    # Voice pipeline: Deepgram STT -> Silero VAD -> Nova LLM -> Deepgram TTS
  daily_agentcore_prep.py             # IPv6 UDP relay workaround for AgentCore networking
  Dockerfile                          # Container image for AgentCore
  pyproject.toml                      # Python dependencies
  env.example                         # Environment variable template for local dev
web/                                  # Amplify Gen 2 project
  amplify/
    backend.ts                        # All CDK infrastructure (VPC, AgentCore, Memory, Lambda)
    auth/
      resource.ts                     # Passwordless authentication config
    data/
      resource.ts                     # GraphQL schema (AppSync)
    functions/
      start-session/
        resource.ts                   # Lambda definition
        handler.ts                    # Room creation + AgentCore invocation
  src/
    App.tsx                           # React frontend
    main.tsx                          # React entry point with Authenticator
  package.json

Deployment

1. Install dependencies

cd web
npm install

2. Set secrets

npx ampx sandbox secret set DAILY_API_KEY
npx ampx sandbox secret set DEEPGRAM_API_KEY

Enter each API key when prompted. Both are stored as SecureString parameters in SSM Parameter Store. The Daily key is used by the Lambda function, and the Deepgram key is fetched by the AgentCore container at startup via its SSM path.

3. Deploy everything

npx ampx sandbox

This single command deploys:

• VPC with NAT Gateway
• AgentCore Memory (with summarization, preference, and fact extraction strategies)
• AgentCore Runtime (builds and pushes the Docker image from agent/)
• AppSync GraphQL API + Lambda function
• Authentication (Cognito with passwordless email OTP + WebAuthn)
• React frontend (hosted by Amplify)

The Amplify website URL is printed in the output.

Note: The VPC NAT Gateway costs ~$32/month while the sandbox is active.

Usage

1. Open the Amplify-hosted URL (printed after deploy)
2. Sign in with your email (you'll receive a one-time passcode)
3. Click "Start Chat"
4. A Daily room opens in a new tab — allow microphone permissions
5. Speak to the agent; you should hear a voice response

Test persistent memory

• Disconnect from the call
• Click "Start Chat" again
• The bot welcomes you back, referencing your previous conversation (preferences, facts, and summaries are recalled from AgentCore Memory)

Test multi-user isolation

• Sign out and sign in with a different email — the new user gets a separate room, bot instance, and conversation history

Teardown

npx ampx sandbox delete

This removes everything: VPC, NAT Gateway, AgentCore Memory, AgentCore Runtime, Lambda, AppSync, authentication, and the frontend.

Manual Frontend Deployment (S3)

If you prefer to deploy the frontend manually instead of through CI/CD, you can build locally and deploy to Amplify Hosting via an S3 bucket.

One-time setup

1. Create an S3 bucket for hosting artifacts:

   aws s3 mb s3://YOUR-BUCKET-NAME --region us-east-1

2. Create an Amplify Hosting app and production branch:

   aws amplify create-app --name pipecat-deepgram-agentcore --platform WEB --region us-east-1
   # Note the appId from the output
   
   aws amplify create-branch --app-id <APP_ID> --branch-name main --stage PRODUCTION --region us-east-1

3. Add an S3 bucket policy granting Amplify read access. Create a file bucket-policy.json (replace <ACCOUNT_ID>, <APP_ID>, and <BUCKET_NAME>):

   {
     "Version": "2012-10-17",
     "Statement": [
       {
         "Sid": "AllowAmplifyToListPrefix",
         "Effect": "Allow",
         "Principal": { "Service": "amplify.amazonaws.com" },
         "Action": "s3:ListBucket",
         "Resource": "arn:aws:s3:::<BUCKET_NAME>",
         "Condition": {
           "StringEquals": {
             "aws:SourceAccount": "<ACCOUNT_ID>",
             "aws:SourceArn": "arn%3Aaws%3Aamplify%3Aus-east-1%3A<ACCOUNT_ID>%3Aapps%2F<APP_ID>%2Fbranches%2Fmain",
             "s3:prefix": ""
           }
         }
       },
       {
         "Sid": "AllowAmplifyToRead",
         "Effect": "Allow",
         "Principal": { "Service": "amplify.amazonaws.com" },
         "Action": "s3:GetObject",
         "Resource": "arn:aws:s3:::<BUCKET_NAME>/*",
         "Condition": {
           "StringEquals": {
             "aws:SourceAccount": "<ACCOUNT_ID>",
             "aws:SourceArn": "arn%3Aaws%3Aamplify%3Aus-east-1%3A<ACCOUNT_ID>%3Aapps%2F<APP_ID>%2Fbranches%2Fmain"
           }
         }
       },
       {
         "Sid": "DenyInsecureTransport",
         "Effect": "Deny",
         "Principal": "*",
         "Action": "s3:*",
         "Resource": "arn:aws:s3:::<BUCKET_NAME>/*",
         "Condition": { "Bool": { "aws:SecureTransport": "false" } }
       }
     ]
   }

   Apply the policy:

   aws s3api put-bucket-policy --bucket <BUCKET_NAME> --policy file://bucket-policy.json

Deploy

From the web/ directory, run the deploy script:

AMPLIFY_APP_ID=<APP_ID> ./deploy.sh

This builds the frontend, syncs dist/ to S3, and triggers an Amplify deployment. The site is available at https://main.<APP_ID>.amplifyapp.com.

The script accepts these environment variables:

Variable	Default	Description
AMPLIFY_APP_ID	(required)	Amplify app ID
AMPLIFY_BRANCH_NAME	main	Branch name
AMPLIFY_S3_BUCKET	pipecat-deepgram-agentcore-hosting	S3 bucket name
AWS_PROFILE	default	AWS CLI profile
AWS_REGION	us-east-1	AWS region

Local Development

For testing the agent locally without deploying to AgentCore:

1. Set up the agent .env file:

   cd agent
   cp env.example .env

   Add: DEEPGRAM_API_KEY, DAILY_ROOM_URL (create a room via the Daily dashboard), and AWS credentials for AgentCore Memory access.

2. Run the bot:

   PIPECAT_LOCAL_DEV=1 uv run pipecat-agent.py -t daily -d

Additional Resources

• Amazon Bedrock AgentCore Developer Guide
• AWS Amplify Gen 2 Documentation
• Daily Documentation
• Deepgram Documentation
• Pipecat Documentation