"sam build" feedback: Support private PyPi repositories

Description:
Some companies host their own internal Pypi servers. They install requirements thru something like:

pip install -i http://some-private-server.com/pypi \
    --extra-index-url https://pypi.python.org/simple/ \
    --trusted-host some-private-server.com
    -t build

sam build needs to support passing extra options to PIP in this case

In addition to passing the options directly in the sam build command line, pip has a couple of ways of storing the information so you don't have to explicitly pass it in via each command line:

Config file

The pypi user and password can be stored in a config file in some specific locations (e.g. $HOME/.config/pip/pip.conf). You could look to see if any of the supported pip config files are present, and mount them into the docker container.

Environment Variables

You can now use environment variables within a requirements.txt with pip 10+.

This could allow you to do something like...

$ cat requirements.txt
--extra-index-url https://${PYPI_USER}:${PYPI_PASS}@mypypi.com/simple/
privatetool==1.0.0

With this I can now run sam build and it will use my private pypi as long as I have those environment variables available. This doesn't work when building using a container as you can not pass in environment variables.

@sanathkr

@sanathkr you reckon we could add an additional argument to the build command to pass in environment variables to the build container?

sam build --use-container --docker-env PYPI_USER=123,PYPI_PASS=xxx

Should be fairly easy to hook up as the BuildContainer uses Container class which already allows env vars to be passed in.

Thanks!

@billyshambrook

@billyshambrook I wonder if we just pass system env vars into the container instead? I have had this conversation with @sthulb about the Go builders some time back. I am not sure why we don't pass the env vars into the container already (maybe some security reason?) but I think it makes sense for build to pass the system env vars into the container by default.

Here is the code where we made the decision right now: https://github.com/awslabs/aws-sam-cli/blob/develop/samcli/local/docker/lambda_build_container.py#L87

@sanathkr Thoughts?

@billyshambrook

We did not pass environment variables to provide isolation. When you build inside the container, the environment is not tainted by configuration you have set in your terminal. However, I think it is a good idea to pass through standard environment variables used by package managers.

@billyshambrook Do you know which env vars are standard to Python Packaging (Twine, PyPi) etc?

We could make a change to automatically pass those through, if available.

So we would have to keep a mapping of what each package manager to "standard" environment variables? That sounds like more configuration we have to keep track of and another point for customers to get confused by ("I can build locally but not in the container"). I don't see why we can't pass the system env vars. I don't agree that it taints the configuration. I should be able to define env vars to configure things in the container or locally.

Yes, it is another set of configs to maintain, but in reasonable time we will get good coverage of the standard envvars.

I think we should set clear expectations of building locally vs container. This where the confusion is. Based on how we define we could go one way or the other

It’s too constraining. If we don’t want to mount system env vars, we should at least provide a way for customers to configure.

Keeping this mapping does not scale. Provided runtimes, plugin builders, etc. I am strongly against having SAM CLI maintain and updating what env vars map to what container. It’s just another “X introduced Y” that we have to keep track of.

@jfuss

@jfuss @sanathkr

I had the same issue with env vars and setting specific build parameters for the pip install (CFLAGS, etc)

would not make sense something like it was suggested above by @billyshambrook --docker-env MYVAR=myval?

We'd like to pass --ignore-installed pip flag to sam build --use-container and it doesn't work.

docker-env

$ sam build --use-container --docker-env PYP_IGNORE_INSTALLED=true
Usage: sam build [OPTIONS] [FUNCTION_IDENTIFIER]
Try "sam build --help" for help.

Error: no such option: --docker-env

requirements.txt file

file:

--ignore-installed
pyathena==1.7.1

Output:

$ sam build --use-container
Starting Build inside a container
Unable to process properties of PredictFunction.AWS::Serverless::Function
Building resource 'InitPredictFunction'

Fetching lambci/lambda:build-python3.6 Docker container image......
Mounting /Users/rinat/Developer/Projects/Basking/bs-data/aws_cfn/ds_model/application as /tmp/samcli/source:ro,delegated inside runtime container

Build Failed
Running PythonPipBuilder:ResolveDependencies
Error: PythonPipBuilder:ResolveDependencies - Usage: -c [options]

ERROR: Invalid requirement: --ignore-installed
-c: error: no such option: --ignore-installed

Is there any workaround or another approach to pass these settings to pip in sam?

Is there any more work on this? We are facing a similar problem where we want to include private repos into our requirements.txt file but would need to hardcode user and passwords into the file to get them.

@naxxfish

We're facing currently the same issue as @naxxfish at the company: trying to build an application, with python depenencies in CodeArtifact with sam build --use-container. Index configuration within the container is not supported, which blocks developers (using windows) to build apps.
We can't find any cleanish resolution for this issue yet.
@sanathkr do you have any plan to implement this feature in the near future?

@liszi

👍 to @liszi - I'm exploring CodeArtifact for private packages and trying to reconcile local builds using containers with this.

A way to leverage pip.conf would be preferable. Requiring the addition of --extra-index-url in every requirements.txt file less so, but I will test it.

@liszi

👍 to @liszi - I'm exploring CodeArtifact for private packages and trying to reconcile local builds using containers with this.

Be warned that it can be a pain in the butt to keep regenerating your token for local development.

@timbedard the auth tokens can go up to 12 hours so not too concerned there. More an issue of guidance to my devs on reusing tokens and not grabbing new ones every time.

This. Please. Not sure why this isn't fully supported, especially with CodeArtifact.

@sanathkr

Hi @sanathkr and team,

After thorough analysis of this issue, code review, and practical testing, I'd like to propose closing this issue as resolved. Here's why:

✅ The Core Request Has Been Fully Implemented

Original Request (November 2018):

"sam build needs to support passing extra options to PIP to enable the use of private PyPi repositories"

Solution Implemented (2020):

--container-env-var parameter for passing individual environment variables
--container-env-var-file parameter for loading multiple variables from a JSON file

Both features are fully functional and tested as of December 2024.

Current Working Solution

Method 1: Inline Environment Variables

Pass environment variables directly via command line:

sam build --use-container \
  --container-env-var PYPI_USER=myuser \
  --container-env-var PYPI_PASS=mypassword

Or reference from shell environment:

export PYPI_USER="myuser"
export PYPI_PASS="mypassword"

sam build --use-container \
  --container-env-var PYPI_USER \
  --container-env-var PYPI_PASS

Method 2: Environment Variables File

For multiple variables, use a JSON file:

Create build-env.json:

{
  "Parameters": {
    "PYPI_USER": "myuser",
    "PYPI_PASS": "mypassword",
    "PYPI_INDEX_URL": "https://pypi.example.com/simple"
  }
}

⚠️ Important: The JSON file must use the {"Parameters": {...}} structure shown above.

Use in build:

sam build --use-container --container-env-var-file build-env.json

Method 3: Combining Both Methods

You can use both inline and file-based env vars together:

sam build --use-container \
  --container-env-var-file build-env.json \
  --container-env-var ADDITIONAL_VAR=value

Practical Usage Examples

Example 1: Private PyPi with Basic Authentication

requirements.txt:

--index-url https://${PYPI_USER}:${PYPI_PASS}@pypi.example.com/simple
--extra-index-url https://pypi.python.org/simple/
--trusted-host pypi.example.com
my-private-package==1.0.0
requests==2.31.0

Build command:

sam build --use-container \
  --container-env-var PYPI_USER=myuser \
  --container-env-var PYPI_PASS=mypassword

Example 2: AWS CodeArtifact

Fetch token and build:

export CODEARTIFACT_TOKEN=$(aws codeartifact get-authorization-token \
  --domain my-domain \
  --domain-owner 123456789012 \
  --region us-east-1 \
  --query authorizationToken \
  --output text)

sam build --use-container \
  --container-env-var CODEARTIFACT_TOKEN

requirements.txt:

--index-url https://aws:${CODEARTIFACT_TOKEN}@my-domain-123456789012.d.codeartifact.us-east-1.amazonaws.com/pypi/my-repo/simple/
my-private-package==1.0.0

Example 3: Multiple Private Repositories

build-env.json:

{
  "Parameters": {
    "INTERNAL_PYPI_USER": "user1",
    "INTERNAL_PYPI_PASS": "pass1",
    "VENDOR_PYPI_TOKEN": "token2"
  }
}

requirements.txt:

--extra-index-url https://${INTERNAL_PYPI_USER}:${INTERNAL_PYPI_PASS}@internal-pypi.company.com/simple
--extra-index-url https://token:${VENDOR_PYPI_TOKEN}@vendor-pypi.example.com/simple
--extra-index-url https://pypi.python.org/simple/
internal-lib==1.0.0
vendor-package==2.0.0

This should be possible for some use cases with image functions and secret repositories after #8913. An example template would be like:

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31

Resources:
  SecretDemoFunction:
    Type: AWS::Serverless::Function
    Properties:
      PackageType: Image
      Architectures: [x86_64]
      Timeout: 10
    Metadata:
      Dockerfile: Dockerfile
      DockerContext: ./src
      DockerTag: latest
      DockerBuildExtraParams:
        - "--secret"
        - "id=demo,src=./secret.txt"

and a Dockerfile like

# syntax=docker/dockerfile:1.4
FROM public.ecr.aws/lambda/python:3.12

RUN --mount=type=secret,id=demo \
    echo "you shouldn't print secrets $(cat run/secrets/demo)"
COPY app.py ${LAMBDA_TASK_ROOT}
CMD ["app.handler"]

At runtime, we would call the docker (or finch) CLI, like docker buildx --secret id=demo,src=./secret.txt .... At this point it works the same as any other docker build secret.

Edit: Sorry for closing the issue, I did not mean to close it because this doesn't solve it for ZIP functions.

Navigation Menu

Repository navigation

"sam build" feedback: Support private PyPi repositories #794

Config file

Environment Variables

11 remaining items

✅ The Core Request Has Been Fully Implemented

Current Working Solution

Method 1: Inline Environment Variables

Method 2: Environment Variables File

Method 3: Combining Both Methods

Practical Usage Examples

Example 1: Private PyPi with Basic Authentication

Example 2: AWS CodeArtifact

Example 3: Multiple Private Repositories

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

"sam build" feedback: Support private PyPi repositories #794

Description

Activity

billyshambrook commented on Dec 29, 2018

Config file

Environment Variables

billyshambrook commented on Apr 12, 2019

jfuss commented on Apr 12, 2019

sanathkr commented on Apr 15, 2019

jfuss commented on Apr 15, 2019

sanathkr commented on Apr 15, 2019

jfuss commented on Apr 15, 2019

ltrojana3d commented on May 28, 2019

rinatio commented on Sep 13, 2019

benDavidson6 commented on May 11, 2020

11 remaining items

liszi commented on Jun 1, 2021

brysontyrrell commented on Dec 14, 2021

qbedard commented on Dec 16, 2021

brysontyrrell commented on Dec 16, 2021

mbtx2 commented on May 24, 2023

dcabib commented on Oct 6, 2025

✅ The Core Request Has Been Fully Implemented

Current Working Solution

Method 1: Inline Environment Variables

Method 2: Environment Variables File

Method 3: Combining Both Methods

Practical Usage Examples

Example 1: Private PyPi with Basic Authentication

Example 2: AWS CodeArtifact

Example 3: Multiple Private Repositories

reedham-aws commented on May 13, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions