[Bug] Too many open files (on OSX aarch64) when setting up test environment

[iamthebot]

Starting with rattler-build 0.22.0 we can no longer build packages containing a test environment on OSX (at least on apple silicon). We experience failures with linking packages due to too many open files. I binary searched all the released versions from 0.21.0 (which works fine) to 0.38.0. Looks like the breakage occurred in 0.22.0.

 │ │ Installing test environment
│ │ │ ⚠ warning validation for "/Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/typer-0.15.2-pyhff008b6_0" failed: failed to read 'index.json'
│ │ │ ⚠ warning validation for "/Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/shellingham-1.5.4-pyhd8ed1ab_1" failed: failed to read 'index.json'
│ │ │ ⚠ warning validation for "/Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/requests-oauthlib-2.0.0-pyhd8ed1ab_1" failed: failed to read 'index.json'
│ │ │ ⚠ warning validation for "/Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/mistletoe-1.4.0-pyhd8ed1ab_1" failed: failed to read 'index.json'
│ │ │ ⚠ warning validation for "/Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/starlette-0.46.0-pyha770c72_0" failed: failed to read 'index.json'
│ │ │ ⚠ warning validation for "/Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/jinja2-3.1.6-pyhd8ed1ab_0" failed: failed to read 'index.json'
│ │ │ ⚠ warning validation for "/Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/ruamel.yaml-0.18.10-py313h90d716c_0" failed: failed to read 'index.json'
│ │ │ ⚠ warning validation for "/Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/xmod-1.8.1-pyhd8ed1ab_1" failed: failed to read 'index.json'
│ │ │ ⚠ warning validation for "/Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/propcache-0.2.1-py313ha9b7d5b_1" failed: failed to read 'index.json'
│ │ │ ⚠ warning validation for "/Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/simdjson-3.11.6-ha393de7_0" failed: failed to read 'index.json'
│ │ │ ⚠ warning validation for "/Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/pyyaml-6.0.2-py313ha9b7d5b_2" failed: failed to read 'index.json'
│ │ │
│ │ ╰─────────────────── (took 1 second)
│ │ ⚠ warning validation for "/Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/c-ares-1.34.4-h5505292_0" failed: failed to read 'index.json'
│ │
│ ╰─────────────────── (took 2 seconds)
│
╰─────────────────── (took 19 seconds)
× error Error building package: failed to setup test environment: failed to link tk-8.6.13-h5083fa2_1.conda
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/noarch/pyu2f-0.1.5-pyhd8ed1ab_1.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/pyu2f-0.1.5-pyhd8ed1ab_1: Request error: error sending request for url
⚠ warning  (https://conda.anaconda.org/conda-forge/noarch/pyu2f-0.1.5-pyhd8ed1ab_1.conda). Retry #1, Sleeping 1.18711036s until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/noarch/truststore-0.10.1-pyh29332c3_0.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/truststore-0.10.1-pyh29332c3_0: Request error: error sending req
⚠ warning uest for url (https://conda.anaconda.org/conda-forge/noarch/truststore-0.10.1-pyh29332c3_0.conda). Retry #1, Sleeping 24.165805ms until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/noarch/tqdm-4.67.1-pyhd8ed1ab_1.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/tqdm-4.67.1-pyhd8ed1ab_1: Request error: error sending request for url
⚠ warning  (https://conda.anaconda.org/conda-forge/noarch/tqdm-4.67.1-pyhd8ed1ab_1.conda). Retry #1, Sleeping 359.415573ms until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/noarch/attrs-25.1.0-pyh71513ae_0.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/attrs-25.1.0-pyh71513ae_0: Request error: error sending request for u
⚠ warning rl (https://conda.anaconda.org/conda-forge/noarch/attrs-25.1.0-pyh71513ae_0.conda). Retry #1, Sleeping 1.300461892s until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/noarch/smart_open-7.1.0-pyhd8ed1ab_0.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/smart_open-7.1.0-pyhd8ed1ab_0: Request error: error sending reque
⚠ warning st for url (https://conda.anaconda.org/conda-forge/noarch/smart_open-7.1.0-pyhd8ed1ab_0.conda). Retry #1, Sleeping 457.552561ms until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/osx-arm64/lzo-2.10-h93a5062_1001.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/lzo-2.10-h93a5062_1001: Request error: error sending request for url
⚠ warning (https://conda.anaconda.org/conda-forge/osx-arm64/lzo-2.10-h93a5062_1001.conda). Retry #1, Sleeping 1.135503333s until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/noarch/google-auth-2.38.0-pyhd8ed1ab_0.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/google-auth-2.38.0-pyhd8ed1ab_0: Request error: error sending r
⚠ warning equest for url (https://conda.anaconda.org/conda-forge/noarch/google-auth-2.38.0-pyhd8ed1ab_0.conda). Retry #1, Sleeping 1.218463303s until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/noarch/mypy_boto3_dynamodb-1.37.0-pyhd8ed1ab_0.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/mypy_boto3_dynamodb-1.37.0-pyhd8ed1ab_0: Request error:
⚠ warning  error sending request for url (https://conda.anaconda.org/conda-forge/noarch/mypy_boto3_dynamodb-1.37.0-pyhd8ed1ab_0.conda). Retry #1, Sleeping 923.405871ms until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/noarch/conda-package-streaming-0.11.0-pyhd8ed1ab_0.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/conda-package-streaming-0.11.0-pyhd8ed1ab_0: Reques
⚠ warning t error: error sending request for url (https://conda.anaconda.org/conda-forge/noarch/conda-package-streaming-0.11.0-pyhd8ed1ab_0.conda). Retry #1, Sleeping 1.544767782s until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/noarch/anyio-4.8.0-pyhd8ed1ab_0.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/anyio-4.8.0-pyhd8ed1ab_0: Request error: error sending request for url
⚠ warning  (https://conda.anaconda.org/conda-forge/noarch/anyio-4.8.0-pyhd8ed1ab_0.conda). Retry #1, Sleeping 1.692350331s until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/noarch/mypy_boto3_cloudformation-1.37.0-pyhd8ed1ab_0.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/mypy_boto3_cloudformation-1.37.0-pyhd8ed1ab_0: Re
⚠ warning quest error: error sending request for url (https://conda.anaconda.org/conda-forge/noarch/mypy_boto3_cloudformation-1.37.0-pyhd8ed1ab_0.conda). Retry #1, Sleeping 1.348007015s until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/noarch/python-dateutil-2.9.0.post0-pyhff2d567_1.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/python-dateutil-2.9.0.post0-pyhff2d567_1: Request erro
⚠ warning r: error sending request for url (https://conda.anaconda.org/conda-forge/noarch/python-dateutil-2.9.0.post0-pyhff2d567_1.conda). Retry #1, Sleeping 1.099889176s until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/osx-arm64/markupsafe-3.0.2-py313ha9b7d5b_1.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/markupsafe-3.0.2-py313ha9b7d5b_1: Request error: error send
⚠ warning ing request for url (https://conda.anaconda.org/conda-forge/osx-arm64/markupsafe-3.0.2-py313ha9b7d5b_1.conda). Retry #1, Sleeping 1.732789029s until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/noarch/rsa-4.9-pyhd8ed1ab_1.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/rsa-4.9-pyhd8ed1ab_1: Request error: error sending request for url (https:
⚠ warning //conda.anaconda.org/conda-forge/noarch/rsa-4.9-pyhd8ed1ab_1.conda). Retry #1, Sleeping 748.513119ms until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/osx-arm64/zstandard-0.23.0-py313hf2da073_1.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/zstandard-0.23.0-py313hf2da073_1: Request error: error send
⚠ warning ing request for url (https://conda.anaconda.org/conda-forge/osx-arm64/zstandard-0.23.0-py313hf2da073_1.conda). Retry #1, Sleeping 269.892908ms until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/noarch/pyopenssl-25.0.0-pyhd8ed1ab_0.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/pyopenssl-25.0.0-pyhd8ed1ab_0: Request error: error sending reque
⚠ warning st for url (https://conda.anaconda.org/conda-forge/noarch/pyopenssl-25.0.0-pyhd8ed1ab_0.conda). Retry #1, Sleeping 1.057146958s until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/osx-arm64/propcache-0.2.1-py313ha9b7d5b_1.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/propcache-0.2.1-py313ha9b7d5b_1: Request error: error sendin
⚠ warning g request for url (https://conda.anaconda.org/conda-forge/osx-arm64/propcache-0.2.1-py313ha9b7d5b_1.conda). Retry #1, Sleeping 110.317974ms until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/osx-arm64/lz4-4.3.3-py313h28882b1_2.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/lz4-4.3.3-py313h28882b1_2: Request error: error sending request fo
⚠ warning r url (https://conda.anaconda.org/conda-forge/osx-arm64/lz4-4.3.3-py313h28882b1_2.conda). Retry #1, Sleeping 1.696926316s until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/noarch/jsonpatch-1.33-pyhd8ed1ab_1.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/jsonpatch-1.33-pyhd8ed1ab_1: Request error: error sending request f
⚠ warning or url (https://conda.anaconda.org/conda-forge/noarch/jsonpatch-1.33-pyhd8ed1ab_1.conda). Retry #1, Sleeping 584.538455ms until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/osx-arm64/libmambapy-2.0.5-py313h33995f7_1.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/libmambapy-2.0.5-py313h33995f7_1: Request error: error send
⚠ warning ing request for url (https://conda.anaconda.org/conda-forge/osx-arm64/libmambapy-2.0.5-py313h33995f7_1.conda). Retry #1, Sleeping 1.116721347s until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/osx-arm64/watchfiles-1.0.4-py313hdde674f_0.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/watchfiles-1.0.4-py313hdde674f_0: Request error: error send
⚠ warning ing request for url (https://conda.anaconda.org/conda-forge/osx-arm64/watchfiles-1.0.4-py313hdde674f_0.conda). Retry #1, Sleeping 419.767264ms until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/osx-arm64/websockets-15.0.1-py313h90d716c_0.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/websockets-15.0.1-py313h90d716c_0: Request error: error se
⚠ warning nding request for url (https://conda.anaconda.org/conda-forge/osx-arm64/websockets-15.0.1-py313h90d716c_0.conda). Retry #1, Sleeping 1.792140524s until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/osx-arm64/conda-25.1.1-py313h8f79df9_1.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/conda-25.1.1-py313h8f79df9_1: Request error: error sending requ
⚠ warning est for url (https://conda.anaconda.org/conda-forge/osx-arm64/conda-25.1.1-py313h8f79df9_1.conda). Retry #1, Sleeping 463.953376ms until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/noarch/email_validator-2.2.0-hd8ed1ab_1.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/email_validator-2.2.0-hd8ed1ab_1: Request error: error sending
⚠ warning  request for url (https://conda.anaconda.org/conda-forge/noarch/email_validator-2.2.0-hd8ed1ab_1.conda). Retry #1, Sleeping 424.769844ms until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/noarch/h11-0.14.0-pyhd8ed1ab_1.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/h11-0.14.0-pyhd8ed1ab_1: Request error: error sending request for url (
⚠ warning https://conda.anaconda.org/conda-forge/noarch/h11-0.14.0-pyhd8ed1ab_1.conda). Retry #1, Sleeping 232.792634ms until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/noarch/blessed-1.20.0-pyh29332c3_0.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/blessed-1.20.0-pyh29332c3_0: Request error: error sending request f
⚠ warning or url (https://conda.anaconda.org/conda-forge/noarch/blessed-1.20.0-pyh29332c3_0.conda). Retry #1, Sleeping 636.26118ms until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/noarch/networkx-3.4.2-pyh267e887_2.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/networkx-3.4.2-pyh267e887_2: Request error: error sending request f
⚠ warning or url (https://conda.anaconda.org/conda-forge/noarch/networkx-3.4.2-pyh267e887_2.conda). Retry #1, Sleeping 287.586731ms until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/osx-arm64/yarl-1.18.3-py313ha9b7d5b_1.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/yarl-1.18.3-py313ha9b7d5b_1: Request error: error sending reques
⚠ warning t for url (https://conda.anaconda.org/conda-forge/osx-arm64/yarl-1.18.3-py313ha9b7d5b_1.conda). Retry #1, Sleeping 117.508142ms until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/noarch/mypy-boto3-s3-1.37.0-pyhd8ed1ab_0.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/mypy-boto3-s3-1.37.0-pyhd8ed1ab_0: Request error: error sendi
⚠ warning ng request for url (https://conda.anaconda.org/conda-forge/noarch/mypy-boto3-s3-1.37.0-pyhd8ed1ab_0.conda). Retry #1, Sleeping 382.742202ms until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/osx-arm64/simdjson-3.11.6-ha393de7_0.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/simdjson-3.11.6-ha393de7_0: Request error: error sending request
⚠ warning for url (https://conda.anaconda.org/conda-forge/osx-arm64/simdjson-3.11.6-ha393de7_0.conda). Retry #1, Sleeping 811.146087ms until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/noarch/natsort-8.4.0-pyh29332c3_1.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/natsort-8.4.0-pyh29332c3_1: Request error: error sending request for
⚠ warning  url (https://conda.anaconda.org/conda-forge/noarch/natsort-8.4.0-pyh29332c3_1.conda). Retry #1, Sleeping 1.673429ms until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/noarch/mypy_boto3_sqs-1.37.0-pyhd8ed1ab_0.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/mypy_boto3_sqs-1.37.0-pyhd8ed1ab_0: Request error: error sen
⚠ warning ding request for url (https://conda.anaconda.org/conda-forge/noarch/mypy_boto3_sqs-1.37.0-pyhd8ed1ab_0.conda). Retry #1, Sleeping 1.755764866s until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/osx-arm64/frozenlist-1.5.0-py313ha9b7d5b_1.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/frozenlist-1.5.0-py313ha9b7d5b_1: Request error: error send
⚠ warning ing request for url (https://conda.anaconda.org/conda-forge/osx-arm64/frozenlist-1.5.0-py313ha9b7d5b_1.conda). Retry #1, Sleeping 1.233191873s until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/osx-arm64/pycosat-0.6.6-py313h90d716c_2.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/pycosat-0.6.6-py313h90d716c_2: Request error: error sending re
⚠ warning quest for url (https://conda.anaconda.org/conda-forge/osx-arm64/pycosat-0.6.6-py313h90d716c_2.conda). Retry #1, Sleeping 231.279722ms until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/osx-arm64/ruamel.yaml.clib-0.2.8-py313h63a2874_1.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/ruamel.yaml.clib-0.2.8-py313h63a2874_1: Request error
⚠ warning : error sending request for url (https://conda.anaconda.org/conda-forge/osx-arm64/ruamel.yaml.clib-0.2.8-py313h63a2874_1.conda). Retry #1, Sleeping 622.93385ms until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/noarch/pyjwt-2.10.1-pyhd8ed1ab_0.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/pyjwt-2.10.1-pyhd8ed1ab_0: Request error: error sending request for u
⚠ warning rl (https://conda.anaconda.org/conda-forge/noarch/pyjwt-2.10.1-pyhd8ed1ab_0.conda). Retry #1, Sleeping 1.844161581s until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/osx-arm64/fmt-11.0.2-h420ef59_0.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/fmt-11.0.2-h420ef59_0: Request error: error sending request for url (h
⚠ warning ttps://conda.anaconda.org/conda-forge/osx-arm64/fmt-11.0.2-h420ef59_0.conda). Retry #1, Sleeping 1.664634247s until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/noarch/rich-toolkit-0.11.3-pyh29332c3_0.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/rich-toolkit-0.11.3-pyh29332c3_0: Request error: error sending
⚠ warning  request for url (https://conda.anaconda.org/conda-forge/noarch/rich-toolkit-0.11.3-pyh29332c3_0.conda). Retry #1, Sleeping 153.470704ms until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/noarch/wcwidth-0.2.13-pyhd8ed1ab_1.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/wcwidth-0.2.13-pyhd8ed1ab_1: Request error: error sending request f
⚠ warning or url (https://conda.anaconda.org/conda-forge/noarch/wcwidth-0.2.13-pyhd8ed1ab_1.conda). Retry #1, Sleeping 421.000668ms until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/noarch/consolekit-1.7.2-pyhd8ed1ab_1.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/consolekit-1.7.2-pyhd8ed1ab_1: Request error: error sending reque
⚠ warning st for url (https://conda.anaconda.org/conda-forge/noarch/consolekit-1.7.2-pyhd8ed1ab_1.conda). Retry #1, Sleeping 243.663368ms until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/noarch/s3transfer-0.11.4-pyhd8ed1ab_0.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/s3transfer-0.11.4-pyhd8ed1ab_0: Request error: error sending req
⚠ warning uest for url (https://conda.anaconda.org/conda-forge/noarch/s3transfer-0.11.4-pyhd8ed1ab_0.conda). Retry #1, Sleeping 373.242945ms until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/osx-arm64/menuinst-2.2.0-py313h8f79df9_0.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/menuinst-2.2.0-py313h8f79df9_0: Request error: error sending
⚠ warning request for url (https://conda.anaconda.org/conda-forge/osx-arm64/menuinst-2.2.0-py313h8f79df9_0.conda). Retry #1, Sleeping 1.498390341s until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/osx-arm64/libiconv-1.18-hfe07756_1.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/libiconv-1.18-hfe07756_1: Request error: error sending request for
⚠ warning url (https://conda.anaconda.org/conda-forge/osx-arm64/libiconv-1.18-hfe07756_1.conda). Retry #1, Sleeping 1.009233672s until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/osx-arm64/yaml-0.2.5-h3422bc3_2.tar.bz2 to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/yaml-0.2.5-h3422bc3_2: Request error: error sending request for url
⚠ warning (https://conda.anaconda.org/conda-forge/osx-arm64/yaml-0.2.5-h3422bc3_2.tar.bz2). Retry #1, Sleeping 404.495249ms until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/noarch/jmespath-1.0.1-pyhd8ed1ab_1.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/jmespath-1.0.1-pyhd8ed1ab_1: Request error: error sending request f
⚠ warning or url (https://conda.anaconda.org/conda-forge/noarch/jmespath-1.0.1-pyhd8ed1ab_1.conda). Retry #1, Sleeping 1.575632884s until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/noarch/hurry.filesize-0.9-pyhd8ed1ab_2.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/hurry.filesize-0.9-pyhd8ed1ab_2: Request error: error sending r
⚠ warning equest for url (https://conda.anaconda.org/conda-forge/noarch/hurry.filesize-0.9-pyhd8ed1ab_2.conda). Retry #1, Sleeping 1.936315697s until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/noarch/mypy_boto3_rds-1.37.6-pyhd8ed1ab_0.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/mypy_boto3_rds-1.37.6-pyhd8ed1ab_0: Request error: error sen
⚠ warning ding request for url (https://conda.anaconda.org/conda-forge/noarch/mypy_boto3_rds-1.37.6-pyhd8ed1ab_0.conda). Retry #1, Sleeping 293.721109ms until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/osx-arm64/wrapt-1.17.2-py313h90d716c_0.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/wrapt-1.17.2-py313h90d716c_0: Request error: error sending requ
⚠ warning est for url (https://conda.anaconda.org/conda-forge/osx-arm64/wrapt-1.17.2-py313h90d716c_0.conda). Retry #1, Sleeping 544.222381ms until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/noarch/apeye-core-1.1.5-pyhd8ed1ab_1.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/apeye-core-1.1.5-pyhd8ed1ab_1: Request error: error sending reque
⚠ warning st for url (https://conda.anaconda.org/conda-forge/noarch/apeye-core-1.1.5-pyhd8ed1ab_1.conda). Retry #1, Sleeping 1.349422786s until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/noarch/urllib3-2.3.0-pyhd8ed1ab_0.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/urllib3-2.3.0-pyhd8ed1ab_0: Request error: error sending request for
⚠ warning  url (https://conda.anaconda.org/conda-forge/noarch/urllib3-2.3.0-pyhd8ed1ab_0.conda). Retry #1, Sleeping 756.445557ms until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/noarch/types-awscrt-0.23.10-pyhd8ed1ab_0.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/types-awscrt-0.23.10-pyhd8ed1ab_0: Request error: error sendi
⚠ warning ng request for url (https://conda.anaconda.org/conda-forge/noarch/types-awscrt-0.23.10-pyhd8ed1ab_0.conda). Retry #1, Sleeping 1.119948648s until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/osx-arm64/libnghttp2-1.64.0-h6d7220d_0.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/libnghttp2-1.64.0-h6d7220d_0: Request error: error sending requ
⚠ warning est for url (https://conda.anaconda.org/conda-forge/osx-arm64/libnghttp2-1.64.0-h6d7220d_0.conda). Retry #1, Sleeping 787.598673ms until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/noarch/readchar-4.2.0-pyhd8ed1ab_1.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/readchar-4.2.0-pyhd8ed1ab_1: Request error: error sending request f
⚠ warning or url (https://conda.anaconda.org/conda-forge/noarch/readchar-4.2.0-pyhd8ed1ab_1.conda). Retry #1, Sleeping 1.205492113s until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/noarch/conda-package-handling-2.4.0-pyh7900ff3_2.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/conda-package-handling-2.4.0-pyh7900ff3_2: Request er
⚠ warning ror: error sending request for url (https://conda.anaconda.org/conda-forge/noarch/conda-package-handling-2.4.0-pyh7900ff3_2.conda). Retry #1, Sleeping 837.444279ms until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/osx-arm64/reproc-cpp-14.2.5.post0-h286801f_0.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/reproc-cpp-14.2.5.post0-h286801f_0: Request error: error
⚠ warning sending request for url (https://conda.anaconda.org/conda-forge/osx-arm64/reproc-cpp-14.2.5.post0-h286801f_0.conda). Retry #1, Sleeping 780.810997ms until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/noarch/mypy_boto3_ec2-1.37.5-pyhd8ed1ab_0.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/mypy_boto3_ec2-1.37.5-pyhd8ed1ab_0: Request error: error sen
⚠ warning ding request for url (https://conda.anaconda.org/conda-forge/noarch/mypy_boto3_ec2-1.37.5-pyhd8ed1ab_0.conda). Retry #1, Sleeping 630.179964ms until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/osx-arm64/c-ares-1.34.4-h5505292_0.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/c-ares-1.34.4-h5505292_0: Request error: error sending request for
⚠ warning url (https://conda.anaconda.org/conda-forge/osx-arm64/c-ares-1.34.4-h5505292_0.conda). Retry #1, Sleeping 1.717256702s until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/osx-arm64/cryptography-44.0.2-py313h54e0d97_0.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/cryptography-44.0.2-py313h54e0d97_0: Request error: erro
⚠ warning r sending request for url (https://conda.anaconda.org/conda-forge/osx-arm64/cryptography-44.0.2-py313h54e0d97_0.conda). Retry #1, Sleeping 563.908527ms until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/osx-arm64/libmamba-2.0.5-hdf44a08_1.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/libmamba-2.0.5-hdf44a08_1: Request error: error sending request fo
⚠ warning r url (https://conda.anaconda.org/conda-forge/osx-arm64/libmamba-2.0.5-hdf44a08_1.conda). Retry #1, Sleeping 485.707044ms until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/noarch/deprecation-2.1.0-pyh9f0ad1d_0.tar.bz2 to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/deprecation-2.1.0-pyh9f0ad1d_0: Request error: error sending r
⚠ warning equest for url (https://conda.anaconda.org/conda-forge/noarch/deprecation-2.1.0-pyh9f0ad1d_0.tar.bz2). Retry #1, Sleeping 407.987516ms until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/noarch/python-multipart-0.0.20-pyhff2d567_0.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/python-multipart-0.0.20-pyhff2d567_0: Request error: error
⚠ warning  sending request for url (https://conda.anaconda.org/conda-forge/noarch/python-multipart-0.0.20-pyhff2d567_0.conda). Retry #1, Sleeping 1.201254289s until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/noarch/conda-libmamba-solver-25.1.1-pyhd8ed1ab_0.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/conda-libmamba-solver-25.1.1-pyhd8ed1ab_0: Request er
⚠ warning ror: error sending request for url (https://conda.anaconda.org/conda-forge/noarch/conda-libmamba-solver-25.1.1-pyhd8ed1ab_0.conda). Retry #1, Sleeping 201.993364ms until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/noarch/archspec-0.2.5-pyhd8ed1ab_0.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/archspec-0.2.5-pyhd8ed1ab_0: Request error: error sending request f
⚠ warning or url (https://conda.anaconda.org/conda-forge/noarch/archspec-0.2.5-pyhd8ed1ab_0.conda). Retry #1, Sleeping 560.994427ms until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/noarch/types-s3transfer-0.6.0.post4-pyhd8ed1ab_1.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/types-s3transfer-0.6.0.post4-pyhd8ed1ab_1: Request er
⚠ warning ror: error sending request for url (https://conda.anaconda.org/conda-forge/noarch/types-s3transfer-0.6.0.post4-pyhd8ed1ab_1.conda). Retry #1, Sleeping 284.652184ms until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/osx-arm64/libuv-1.50.0-h5505292_0.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/libuv-1.50.0-h5505292_0: Request error: error sending request for ur
⚠ warning l (https://conda.anaconda.org/conda-forge/osx-arm64/libuv-1.50.0-h5505292_0.conda). Retry #1, Sleeping 692.243429ms until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/noarch/requests-oauthlib-2.0.0-pyhd8ed1ab_1.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/requests-oauthlib-2.0.0-pyhd8ed1ab_1: Request error: error
⚠ warning  sending request for url (https://conda.anaconda.org/conda-forge/noarch/requests-oauthlib-2.0.0-pyhd8ed1ab_1.conda). Retry #1, Sleeping 1.904842789s until the next attempt...
⚠ warning validation for "/Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/distro-1.9.0-pyhd8ed1ab_1" failed: neither a 'paths.json' or a deprecated 'files' file was found
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/noarch/aiosignal-1.3.2-pyhd8ed1ab_0.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/aiosignal-1.3.2-pyhd8ed1ab_0: Request error: error sending request
⚠ warning  for url (https://conda.anaconda.org/conda-forge/noarch/aiosignal-1.3.2-pyhd8ed1ab_0.conda). Retry #1, Sleeping 1.19476532s until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/noarch/editor-1.6.6-pyhd8ed1ab_2.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/editor-1.6.6-pyhd8ed1ab_2: Request error: error sending request for u
⚠ warning rl (https://conda.anaconda.org/conda-forge/noarch/editor-1.6.6-pyhd8ed1ab_2.conda). Retry #1, Sleeping 442.970016ms until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/osx-arm64/pydantic-core-2.27.2-py313hdde674f_0.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/pydantic-core-2.27.2-py313hdde674f_0: Request error: er
⚠ warning ror sending request for url (https://conda.anaconda.org/conda-forge/osx-arm64/pydantic-core-2.27.2-py313hdde674f_0.conda). Retry #1, Sleeping 554.099273ms until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/noarch/shellingham-1.5.4-pyhd8ed1ab_1.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/shellingham-1.5.4-pyhd8ed1ab_1: Request error: error sending req
⚠ warning uest for url (https://conda.anaconda.org/conda-forge/noarch/shellingham-1.5.4-pyhd8ed1ab_1.conda). Retry #1, Sleeping 1.816272727s until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/noarch/boltons-24.0.0-pyhd8ed1ab_1.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/boltons-24.0.0-pyhd8ed1ab_1: Request error: error sending request f
⚠ warning or url (https://conda.anaconda.org/conda-forge/noarch/boltons-24.0.0-pyhd8ed1ab_1.conda). Retry #1, Sleeping 138.774923ms until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/noarch/annotated-types-0.7.0-pyhd8ed1ab_1.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/annotated-types-0.7.0-pyhd8ed1ab_1: Request error: error sen
⚠ warning ding request for url (https://conda.anaconda.org/conda-forge/noarch/annotated-types-0.7.0-pyhd8ed1ab_1.conda). Retry #1, Sleeping 1.038180447s until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/noarch/pyasn1-0.6.1-pyhd8ed1ab_2.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/pyasn1-0.6.1-pyhd8ed1ab_2: Request error: error sending request for u
⚠ warning rl (https://conda.anaconda.org/conda-forge/noarch/pyasn1-0.6.1-pyhd8ed1ab_2.conda). Retry #1, Sleeping 1.867793106s until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/osx-arm64/orjson-3.10.15-py313hdde674f_0.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/orjson-3.10.15-py313hdde674f_0: Request error: error sending
⚠ warning request for url (https://conda.anaconda.org/conda-forge/osx-arm64/orjson-3.10.15-py313hdde674f_0.conda). Retry #1, Sleeping 1.69503719s until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/osx-arm64/multidict-6.1.0-py313h6347b5a_1.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/multidict-6.1.0-py313h6347b5a_1: Request error: error sendin
⚠ warning g request for url (https://conda.anaconda.org/conda-forge/osx-arm64/multidict-6.1.0-py313h6347b5a_1.conda). Retry #1, Sleeping 1.78242816s until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/noarch/platformdirs-4.3.6-pyhd8ed1ab_1.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/platformdirs-4.3.6-pyhd8ed1ab_1: Request error: error sending r
⚠ warning equest for url (https://conda.anaconda.org/conda-forge/noarch/platformdirs-4.3.6-pyhd8ed1ab_1.conda). Retry #1, Sleeping 1.564179735s until the next attempt...
⚠ warning failed to download and extract https://conda.anaconda.org/conda-forge/noarch/six-1.17.0-pyhd8ed1ab_0.conda to /Users/alfredo_luque/Library/Caches/rattler/cache/pkgs/six-1.17.0-pyhd8ed1ab_0: Request error: error sending request for url (
⚠ warning https://conda.anaconda.org/conda-forge/noarch/six-1.17.0-pyhd8ed1ab_0.conda). Retry #1, Sleeping 261.124746ms until the next attempt...
Error:   × failed to setup test environment: failed to link tk-8.6.13-h5083fa2_1.conda
├─▶ failed to link tk-8.6.13-h5083fa2_1.conda
├─▶ failed to read 'index.json'
╰─▶ Too many open files (os error 24)

Note that this issue does not occur on linux (tested on Linux x86_64 and Linux aarch64 w/ rattler-build 0.38.0).

My guess is maybe some kind of concurrency limit might be needed on OSX that respects the current ulimit.

[iamthebot]

cc @snapbug

[beckermr]

If you set ulimit to something like 1024, this goes away. It'd be nice for rattler to query the limits and then not blow through them.

[iamthebot]

@beckermr unfortunately ulimit doesn't persist due to OSX's system integrity protection. So you either have to set it immediately before invoking rattler-build or something like sudo launchctl limit maxfiles <softlimit> <hardlimit> to get it to persist.

That does work as a workaround.

[wolfv]

We have a way of limiting the concurrency when linking files and we should use it. I think the installer struct has a flag for it.

[iamthebot]

@wolfv is this the correct place to fix it? If so, will throw up a PR.

[wolfv]

Yeah there should be a .with_io_concurrency_semaphore(io_concurrency_limit) option that might help.

.with_io_concurrency_semaphore(io_concurrency_limit)