Skip to content

Conversation

@matteomorlack
Copy link

This PR proposes the addition of methods to allow users to clear the /vsicurl/ internal GDAL cache.

Implemented methods support both a full cleanup and a cleanup by prefix, using the provided methods VSICurlClearCache and VSICurlPartialClearCache

Basic tests are included to verify that the methods do not raise exceptions. The behavior should be tested more extensively against different GDAL builds; if you have a better testing approach, let me know and I will add it.

To test the functionaly one can:

import os
from pyogrio import read_dataframe, vsi_curl_clear_cache, set_gdal_config_options

os.environ["AWS_NO_SIGN_REQUEST"] = "YES"

set_gdal_config_options({"CPL_CURL_VERBOSE": True})

# observe from debug that a curl request is issued
read_dataframe("s3://mcsc-geotide-geojson-files/geojson_files/bayarea.geojson")

# observe from debug that NO curl request is issued
read_dataframe("s3://mcsc-geotide-geojson-files/geojson_files/bayarea.geojson")

vsi_curl_clear_cache("s3://") 

# observe from debug that a new curl request is issued, given cache has been cleared
read_dataframe("s3://mcsc-geotide-geojson-files/geojson_files/bayarea.geojson")

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant