SyntaxWarning on invalid escape sequences in verdored third-party libraries (boto, pyasn1)

[ttencate]

Using Arch Linux, google-cloud-cli from AUR, Python 3.12.3, gsutil emits the following warnings every time it is run:

$ gsutil --version
/opt/google-cloud-cli/platform/gsutil/gslib/vendored/boto/boto/__init__.py:1105: SyntaxWarning: invalid escape sequence '\c'
  """
/opt/google-cloud-cli/platform/gsutil/gslib/vendored/boto/boto/pyami/config.py:98: SyntaxWarning: invalid escape sequence '\s'
  match = re.match("^#import[\s\t]*([^\s^\t]*)[\s\t]*$", line)
/opt/google-cloud-cli/platform/gsutil/third_party/pyasn1/pyasn1/type/constraint.py:100: SyntaxWarning: invalid escape sequence '\*'
  """Create a SingleValueConstraint object.
/opt/google-cloud-cli/platform/gsutil/third_party/pyasn1/pyasn1/type/constraint.py:141: SyntaxWarning: invalid escape sequence '\*'
  """Create a ContainedSubtypeConstraint object.
/opt/google-cloud-cli/platform/gsutil/third_party/pyasn1/pyasn1/type/constraint.py:301: SyntaxWarning: invalid escape sequence '\*'
  """Create a PermittedAlphabetConstraint object.
/opt/google-cloud-cli/platform/gsutil/third_party/pyasn1/pyasn1/type/constraint.py:457: SyntaxWarning: invalid escape sequence '\*'
  """Create a ConstraintsIntersection logic operator object.
/opt/google-cloud-cli/platform/gsutil/third_party/pyasn1/pyasn1/type/constraint.py:501: SyntaxWarning: invalid escape sequence '\*'
  """Create a ConstraintsUnion logic operator object.
/opt/google-cloud-cli/platform/gsutil/third_party/pyasn1/pyasn1/type/namedval.py:15: SyntaxWarning: invalid escape sequence '\*'
  """Create named values object.
/opt/google-cloud-cli/platform/gsutil/gslib/vendored/boto/boto/connection.py:672: SyntaxWarning: invalid escape sequence '\w'
  '(?:(?P<user>[\w\-\.]+):(?P<pass>.*)@)?'
/opt/google-cloud-cli/platform/gsutil/gslib/vendored/boto/boto/connection.py:673: SyntaxWarning: invalid escape sequence '\w'
  '(?P<host>[\w\-\.]+)'
/opt/google-cloud-cli/platform/gsutil/gslib/vendored/boto/boto/connection.py:674: SyntaxWarning: invalid escape sequence '\d'
  '(?::(?P<port>\d+))?'
/opt/google-cloud-cli/platform/gsutil/gslib/vendored/boto/boto/connection.py:1121: SyntaxWarning: invalid escape sequence '\.'
  'specific endpoint this request was sent to\.'
/opt/google-cloud-cli/platform/gsutil/gslib/vendored/boto/boto/https_connection.py:80: SyntaxWarning: invalid escape sequence '\.'
  host_re = host.replace('.', '\.').replace('*', '[^.]*')
gsutil version: 5.27

This is probably happening because in Python 3.12 (2013-10-12, release notes), invalid escape sequences turned from a DeprecationWarning into a SyntaxWarning.

[tomsykes]

This SyntaxWarning is showing in the logs for us when using gsutil as part of the cloud-sdk:alpine image during a cloud build step

[daichunqing]

Is there a workaround for that?

[ttencate]

Seems like this is no longer happening in gsutil 5.29.

[daichunqing]

@ttencate Thank you. Confirmed with gsutil 5.30.

[muffl0n]

This does not seem to be the case for Alpine. Building the image with Dockerfile https://github.com/GoogleCloudPlatform/cloud-sdk-docker/blob/master/alpine/Dockerfile with Alpine 3.20 show this error:

/ # cat /etc/os-release 
NAME="Alpine Linux"
ID=alpine
VERSION_ID=3.20.2
PRETTY_NAME="Alpine Linux v3.20"
HOME_URL="https://alpinelinux.org/"
BUG_REPORT_URL="https://gitlab.alpinelinux.org/alpine/aports/-/issues"
/ # gsutil version
/google-cloud-sdk/platform/gsutil/gslib/vendored/boto/boto/__init__.py:1105: SyntaxWarning: invalid escape sequence '\c'
  """
/google-cloud-sdk/platform/gsutil/gslib/vendored/boto/boto/pyami/config.py:98: SyntaxWarning: invalid escape sequence '\s'
  match = re.match("^#import[\s\t]*([^\s^\t]*)[\s\t]*$", line)
/google-cloud-sdk/platform/gsutil/third_party/pyasn1/pyasn1/type/constraint.py:100: SyntaxWarning: invalid escape sequence '\*'
...
gsutil version: 5.30

/ # python --version
Python 3.12.3

[pavandagagoogle]

gsutil has issues with Python 3.12 which is packaged with Alpine 3.20 so it is a compatibility issue

[muffl0n]

Thanks for the feedback! I interpreted the closed issue, along with the positive feedback from @daichunqing and @ttencate, as an indication that this should have already been fixed. The original poster describes the exact incompatibility with Python 3.12.

I added a comment to the issue in the official bug tracker cause the documentation states

The Google Cloud CLI requires Python 3.8 to 3.12.

[SimonPrague]

I'm sorry, but I'm still having these warnings even with 5.30
Python 3.12.5 (main, Aug 7 2024, 00:00:00) [GCC 14.2.1 20240801 (Red Hat 14.2.1-1)] on linux

gsutil version                                                                                                                          
/usr/lib64/google-cloud-sdk/platform/gsutil/gslib/vendored/boto/boto/__init__.py:1105: SyntaxWarning: invalid escape sequence '\c'        
  """                                                                                                                                     
/usr/lib64/google-cloud-sdk/platform/gsutil/gslib/vendored/boto/boto/pyami/config.py:98: SyntaxWarning: invalid escape sequence '\s'      
  match = re.match("^#import[\s\t]*([^\s^\t]*)[\s\t]*$", line)                                                                            
/usr/lib64/google-cloud-sdk/platform/gsutil/third_party/pyasn1/pyasn1/type/constraint.py:100: SyntaxWarning: invalid escape sequence '\*' 
  """Create a SingleValueConstraint object.                                                                                               
/usr/lib64/google-cloud-sdk/platform/gsutil/third_party/pyasn1/pyasn1/type/constraint.py:141: SyntaxWarning: invalid escape sequence '\*' 
  """Create a ContainedSubtypeConstraint object.                                                                                          
/usr/lib64/google-cloud-sdk/platform/gsutil/third_party/pyasn1/pyasn1/type/constraint.py:301: SyntaxWarning: invalid escape sequence '\*' 
  """Create a PermittedAlphabetConstraint object.                                                                                         
/usr/lib64/google-cloud-sdk/platform/gsutil/third_party/pyasn1/pyasn1/type/constraint.py:457: SyntaxWarning: invalid escape sequence '\*' 
  """Create a ConstraintsIntersection logic operator object.                                                                              
/usr/lib64/google-cloud-sdk/platform/gsutil/third_party/pyasn1/pyasn1/type/constraint.py:501: SyntaxWarning: invalid escape sequence '\*' 
  """Create a ConstraintsUnion logic operator object.                                                                                     
/usr/lib64/google-cloud-sdk/platform/gsutil/third_party/pyasn1/pyasn1/type/namedval.py:15: SyntaxWarning: invalid escape sequence '\*'    
  """Create named values object.                                                                                                          
/usr/lib64/google-cloud-sdk/platform/gsutil/gslib/vendored/boto/boto/connection.py:672: SyntaxWarning: invalid escape sequence '\w'       
  '(?:(?P<user>[\w\-\.]+):(?P<pass>.*)@)?'                                                                                                
/usr/lib64/google-cloud-sdk/platform/gsutil/gslib/vendored/boto/boto/connection.py:673: SyntaxWarning: invalid escape sequence '\w'       
  '(?P<host>[\w\-\.]+)'                                                                                                                   
/usr/lib64/google-cloud-sdk/platform/gsutil/gslib/vendored/boto/boto/connection.py:674: SyntaxWarning: invalid escape sequence '\d'       
  '(?::(?P<port>\d+))?'                                                                                                                   
/usr/lib64/google-cloud-sdk/platform/gsutil/gslib/vendored/boto/boto/connection.py:1121: SyntaxWarning: invalid escape sequence '\.'      
  'specific endpoint this request was sent to\.'                                                                                          
/usr/lib64/google-cloud-sdk/platform/gsutil/gslib/vendored/boto/boto/https_connection.py:80: SyntaxWarning: invalid escape sequence '\.'  
  host_re = host.replace('.', '\.').replace('*', '[^.]*')                                                                                 
gsutil version: 5.30                                                                                                                      

[ttencate]

Maybe Arch distributors patched it or suppressed the warnings? Anyway, reopening.

[maxdanilov]

It's been over a year (!) since Python 3.12 was released, and still Google hasn't tackled this, smh.

[phlax]

its a super easy fix - the problem strings need to be marked as raw - ie r"..." - so the escapes are not interpreted