diff --git a/README.md b/README.md index 7468701de..98d60175e 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # Sprint Challenge: Data Structures -In this week's Sprint you implemented some classic and fundamental data structures and learned about how to go about evaluating their respective runtimes and performance. This Sprint Challenge aims to assess your comfort with these topics through exercises that build on the data structures you implemented and the algorithmic intuition you've started to build up. +In this week's Sprint you implemented some classic and fundamental data structures and learned about how to go about evaluating their respective runtimes and performance. This Sprint Challenge aims to assess your comfort with these topics through exercises that build on the data structures you implemented and the algorithmic intuition you've started to build up.cc ## Instructions @@ -58,7 +58,7 @@ buffer.get() # should return ['d', 'e', 'f'] #### Task 2. Runtime Optimization -***!Important!*** If you are running this using PowerShell by clicking on the green play button, you will get an error that `names1.txt` is not found. To resolve this, run it, get the error, then `cd` into the `names` directory in the `python` terminal that opens in VSCode. +**_!Important!_** If you are running this using PowerShell by clicking on the green play button, you will get an error that `names1.txt` is not found. To resolve this, run it, get the error, then `cd` into the `names` directory in the `python` terminal that opens in VSCode. Navigate into the `names` directory. Here you will find two text files containing 10,000 names each, along with a program `names.py` that compares the two files and prints out duplicate name entries. Try running the code with `python3 names.py`. Be patient because it might take a while: approximately six seconds on my laptop. What is the runtime complexity of this code? @@ -71,28 +71,31 @@ A follow-up question to think about: _*once you've used one of the data structur Inside of the `reverse` directory, you'll find a basic implementation of a Singly Linked List. _Without_ making it a Doubly Linked List (adding a tail attribute), complete the `reverse_list()` function within `reverse/reverse.py`. For example, + ``` 1->2->3->None ``` + would become... + ``` 3->2->1->None ``` -While credit will be given for a functional solution, only optimal solutions will earn a ***3*** on this task. +While credit will be given for a functional solution, only optimal solutions will earn a **_3_** on this task. -#### Stretch - -* Say your code from `names.py` is to run on an embedded computer with very limited RAM. Because of this, memory is extremely constrained and you are only allowed to store names in arrays (i.e. Python lists). How would you go about optimizing the code under these conditions? Try it out and compare your solution to the original runtime. (If this solution is less efficient than your original solution, include both and label the strech solution with a comment) +#### Stretch +- Say your code from `names.py` is to run on an embedded computer with very limited RAM. Because of this, memory is extremely constrained and you are only allowed to store names in arrays (i.e. Python lists). How would you go about optimizing the code under these conditions? Try it out and compare your solution to the original runtime. (If this solution is less efficient than your original solution, include both and label the strech solution with a comment) ### Rubric -| TASK | 1 - DOES NOT MEET Expectations | 2 - MEETS Expectations | 3 - EXCEEDS Expectations | SCORE | -| ----- | ------- | ------- | ------- | -- | -| Task 1. Implement a Ring Buffer Data Structure | Solution in `ring_buffer.py` DOES NOT run OR it runs but has multiple logical errors, failing 2 or more tests. | Solution in `ring_buffer.py` runs, but may have one or two logical errors; passes at least 5/6 tests (Note that each function in the test file that begins with `test` is a test). | Solution in `ring_buffer.py` has no syntax or logical errors and passes all tests (Note that each function in the test file that begins with `test` is a test). | | -| Task 2. Runtime Optimization | Student does NOT correctly identify the runtime of the starter code in `name.py` and is not able to optimize it to run in under 6 seconds using a data structure that was implemented during the week. | Student does not identify the runtime of the starter code in `name.py`, but optimizes it to run in under 6 seconds, with a solution that exhibits the appropriate runtime, using a data structure that was implemented during the week | Student does BOTH correctly identify the runtime of the starter code in `name.py` and optimizes it to run in under 6 seconds, with an appropriate runtime using a data structure that was implemented during the week. | | -| Task 3. Reverse the contents of a Singly Linked List | Student's solution in `reverse.py` is failing one or more tests. | Student's solution in `reverse.py` is able to correctly print out the contents of the Linked List in reverse order, passing all tests, BUT, the runtime of their solution is not optimal (requires looping through the list more than once). | Student's solution in `reverse.py` is able to correctly print out the contents of the Linked List in reverse order, passing all tests AND exhibits an appropriate runtime. | | +| TASK | 1 - DOES NOT MEET Expectations | 2 - MEETS Expectations | 3 - EXCEEDS Expectations | SCORE | +| ---------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----- | +| Task 1. Implement a Ring Buffer Data Structure | Solution in `ring_buffer.py` DOES NOT run OR it runs but has multiple logical errors, failing 2 or more tests. | Solution in `ring_buffer.py` runs, but may have one or two logical errors; passes at least 5/6 tests (Note that each function in the test file that begins with `test` is a test). | Solution in `ring_buffer.py` has no syntax or logical errors and passes all tests (Note that each function in the test file that begins with `test` is a test). | | +| Task 2. Runtime Optimization | Student does NOT correctly identify the runtime of the starter code in `name.py` and is not able to optimize it to run in under 6 seconds using a data structure that was implemented during the week. | Student does not identify the runtime of the starter code in `name.py`, but optimizes it to run in under 6 seconds, with a solution that exhibits the appropriate runtime, using a data structure that was implemented during the week | Student does BOTH correctly identify the runtime of the starter code in `name.py` and optimizes it to run in under 6 seconds, with an appropriate runtime using a data structure that was implemented during the week. | | +| Task 3. Reverse the contents of a Singly Linked List | Student's solution in `reverse.py` is failing one or more tests. | Student's solution in `reverse.py` is able to correctly print out the contents of the Linked List in reverse order, passing all tests, BUT, the runtime of their solution is not optimal (requires looping through the list more than once). | Student's solution in `reverse.py` is able to correctly print out the contents of the Linked List in reverse order, passing all tests AND exhibits an appropriate runtime. | | #### Passing the Sprint + Score ranges for a 1, 2, and 3 are shown in the rubric above. For a student to have _passed_ a sprint challenge, they need to earn an **at least 2** for all items on the rubric. diff --git a/env/bin/Activate.ps1 b/env/bin/Activate.ps1 new file mode 100644 index 000000000..900f50e88 --- /dev/null +++ b/env/bin/Activate.ps1 @@ -0,0 +1,230 @@ +<# +.Synopsis +Activate a Python virtual environment for the current Powershell session. + +.Description +Pushes the python executable for a virtual environment to the front of the +$Env:PATH environment variable and sets the prompt to signify that you are +in a Python virtual environment. Makes use of the command line switches as +well as the `pyvenv.cfg` file values present in the virtual environment. + +.Parameter VenvDir +Path to the directory that contains the virtual environment to activate. The +default value for this is the parent of the directory that the Activate.ps1 +script is located within. + +.Parameter Prompt +The prompt prefix to display when this virtual environment is activated. By +default, this prompt is the name of the virtual environment folder (VenvDir) +surrounded by parentheses and followed by a single space (ie. '(.venv) '). + +.Example +Activate.ps1 +Activates the Python virtual environment that contains the Activate.ps1 script. + +.Example +Activate.ps1 -Verbose +Activates the Python virtual environment that contains the Activate.ps1 script, +and shows extra information about the activation as it executes. + +.Example +Activate.ps1 -VenvDir C:\Users\MyUser\Common\.venv +Activates the Python virtual environment located in the specified location. + +.Example +Activate.ps1 -Prompt "MyPython" +Activates the Python virtual environment that contains the Activate.ps1 script, +and prefixes the current prompt with the specified string (surrounded in +parentheses) while the virtual environment is active. + + +#> +Param( + [Parameter(Mandatory = $false)] + [String] + $VenvDir, + [Parameter(Mandatory = $false)] + [String] + $Prompt +) + +<# Function declarations --------------------------------------------------- #> + +<# +.Synopsis +Remove all shell session elements added by the Activate script, including the +addition of the virtual environment's Python executable from the beginning of +the PATH variable. + +.Parameter NonDestructive +If present, do not remove this function from the global namespace for the +session. + +#> +function global:deactivate ([switch]$NonDestructive) { + # Revert to original values + + # The prior prompt: + if (Test-Path -Path Function:_OLD_VIRTUAL_PROMPT) { + Copy-Item -Path Function:_OLD_VIRTUAL_PROMPT -Destination Function:prompt + Remove-Item -Path Function:_OLD_VIRTUAL_PROMPT + } + + # The prior PYTHONHOME: + if (Test-Path -Path Env:_OLD_VIRTUAL_PYTHONHOME) { + Copy-Item -Path Env:_OLD_VIRTUAL_PYTHONHOME -Destination Env:PYTHONHOME + Remove-Item -Path Env:_OLD_VIRTUAL_PYTHONHOME + } + + # The prior PATH: + if (Test-Path -Path Env:_OLD_VIRTUAL_PATH) { + Copy-Item -Path Env:_OLD_VIRTUAL_PATH -Destination Env:PATH + Remove-Item -Path Env:_OLD_VIRTUAL_PATH + } + + # Just remove the VIRTUAL_ENV altogether: + if (Test-Path -Path Env:VIRTUAL_ENV) { + Remove-Item -Path env:VIRTUAL_ENV + } + + # Just remove the _PYTHON_VENV_PROMPT_PREFIX altogether: + if (Get-Variable -Name "_PYTHON_VENV_PROMPT_PREFIX" -ErrorAction SilentlyContinue) { + Remove-Variable -Name _PYTHON_VENV_PROMPT_PREFIX -Scope Global -Force + } + + # Leave deactivate function in the global namespace if requested: + if (-not $NonDestructive) { + Remove-Item -Path function:deactivate + } +} + +<# +.Description +Get-PyVenvConfig parses the values from the pyvenv.cfg file located in the +given folder, and returns them in a map. + +For each line in the pyvenv.cfg file, if that line can be parsed into exactly +two strings separated by `=` (with any amount of whitespace surrounding the =) +then it is considered a `key = value` line. The left hand string is the key, +the right hand is the value. + +If the value starts with a `'` or a `"` then the first and last character is +stripped from the value before being captured. + +.Parameter ConfigDir +Path to the directory that contains the `pyvenv.cfg` file. +#> +function Get-PyVenvConfig( + [String] + $ConfigDir +) { + Write-Verbose "Given ConfigDir=$ConfigDir, obtain values in pyvenv.cfg" + + # Ensure the file exists, and issue a warning if it doesn't (but still allow the function to continue). + $pyvenvConfigPath = Join-Path -Resolve -Path $ConfigDir -ChildPath 'pyvenv.cfg' -ErrorAction Continue + + # An empty map will be returned if no config file is found. + $pyvenvConfig = @{ } + + if ($pyvenvConfigPath) { + + Write-Verbose "File exists, parse `key = value` lines" + $pyvenvConfigContent = Get-Content -Path $pyvenvConfigPath + + $pyvenvConfigContent | ForEach-Object { + $keyval = $PSItem -split "\s*=\s*", 2 + if ($keyval[0] -and $keyval[1]) { + $val = $keyval[1] + + # Remove extraneous quotations around a string value. + if ("'""".Contains($val.Substring(0,1))) { + $val = $val.Substring(1, $val.Length - 2) + } + + $pyvenvConfig[$keyval[0]] = $val + Write-Verbose "Adding Key: '$($keyval[0])'='$val'" + } + } + } + return $pyvenvConfig +} + + +<# Begin Activate script --------------------------------------------------- #> + +# Determine the containing directory of this script +$VenvExecPath = Split-Path -Parent $MyInvocation.MyCommand.Definition +$VenvExecDir = Get-Item -Path $VenvExecPath + +Write-Verbose "Activation script is located in path: '$VenvExecPath'" +Write-Verbose "VenvExecDir Fullname: '$($VenvExecDir.FullName)" +Write-Verbose "VenvExecDir Name: '$($VenvExecDir.Name)" + +# Set values required in priority: CmdLine, ConfigFile, Default +# First, get the location of the virtual environment, it might not be +# VenvExecDir if specified on the command line. +if ($VenvDir) { + Write-Verbose "VenvDir given as parameter, using '$VenvDir' to determine values" +} else { + Write-Verbose "VenvDir not given as a parameter, using parent directory name as VenvDir." + $VenvDir = $VenvExecDir.Parent.FullName.TrimEnd("\\/") + Write-Verbose "VenvDir=$VenvDir" +} + +# Next, read the `pyvenv.cfg` file to determine any required value such +# as `prompt`. +$pyvenvCfg = Get-PyVenvConfig -ConfigDir $VenvDir + +# Next, set the prompt from the command line, or the config file, or +# just use the name of the virtual environment folder. +if ($Prompt) { + Write-Verbose "Prompt specified as argument, using '$Prompt'" +} else { + Write-Verbose "Prompt not specified as argument to script, checking pyvenv.cfg value" + if ($pyvenvCfg -and $pyvenvCfg['prompt']) { + Write-Verbose " Setting based on value in pyvenv.cfg='$($pyvenvCfg['prompt'])'" + $Prompt = $pyvenvCfg['prompt']; + } + else { + Write-Verbose " Setting prompt based on parent's directory's name. (Is the directory name passed to venv module when creating the virutal environment)" + Write-Verbose " Got leaf-name of $VenvDir='$(Split-Path -Path $venvDir -Leaf)'" + $Prompt = Split-Path -Path $venvDir -Leaf + } +} + +Write-Verbose "Prompt = '$Prompt'" +Write-Verbose "VenvDir='$VenvDir'" + +# Deactivate any currently active virtual environment, but leave the +# deactivate function in place. +deactivate -nondestructive + +# Now set the environment variable VIRTUAL_ENV, used by many tools to determine +# that there is an activated venv. +$env:VIRTUAL_ENV = $VenvDir + +if (-not $Env:VIRTUAL_ENV_DISABLE_PROMPT) { + + Write-Verbose "Setting prompt to '$Prompt'" + + # Set the prompt to include the env name + # Make sure _OLD_VIRTUAL_PROMPT is global + function global:_OLD_VIRTUAL_PROMPT { "" } + Copy-Item -Path function:prompt -Destination function:_OLD_VIRTUAL_PROMPT + New-Variable -Name _PYTHON_VENV_PROMPT_PREFIX -Description "Python virtual environment prompt prefix" -Scope Global -Option ReadOnly -Visibility Public -Value $Prompt + + function global:prompt { + Write-Host -NoNewline -ForegroundColor Green "($_PYTHON_VENV_PROMPT_PREFIX) " + _OLD_VIRTUAL_PROMPT + } +} + +# Clear PYTHONHOME +if (Test-Path -Path Env:PYTHONHOME) { + Copy-Item -Path Env:PYTHONHOME -Destination Env:_OLD_VIRTUAL_PYTHONHOME + Remove-Item -Path Env:PYTHONHOME +} + +# Add the venv to the PATH +Copy-Item -Path Env:PATH -Destination Env:_OLD_VIRTUAL_PATH +$Env:PATH = "$VenvExecDir$([System.IO.Path]::PathSeparator)$Env:PATH" diff --git a/env/bin/activate b/env/bin/activate new file mode 100644 index 000000000..75c9ef3e0 --- /dev/null +++ b/env/bin/activate @@ -0,0 +1,76 @@ +# This file must be used with "source bin/activate" *from bash* +# you cannot run it directly + +deactivate () { + # reset old environment variables + if [ -n "${_OLD_VIRTUAL_PATH:-}" ] ; then + PATH="${_OLD_VIRTUAL_PATH:-}" + export PATH + unset _OLD_VIRTUAL_PATH + fi + if [ -n "${_OLD_VIRTUAL_PYTHONHOME:-}" ] ; then + PYTHONHOME="${_OLD_VIRTUAL_PYTHONHOME:-}" + export PYTHONHOME + unset _OLD_VIRTUAL_PYTHONHOME + fi + + # This should detect bash and zsh, which have a hash command that must + # be called to get it to forget past commands. Without forgetting + # past commands the $PATH changes we made may not be respected + if [ -n "${BASH:-}" -o -n "${ZSH_VERSION:-}" ] ; then + hash -r + fi + + if [ -n "${_OLD_VIRTUAL_PS1:-}" ] ; then + PS1="${_OLD_VIRTUAL_PS1:-}" + export PS1 + unset _OLD_VIRTUAL_PS1 + fi + + unset VIRTUAL_ENV + if [ ! "${1:-}" = "nondestructive" ] ; then + # Self destruct! + unset -f deactivate + fi +} + +# unset irrelevant variables +deactivate nondestructive + +VIRTUAL_ENV="/home/nthansen/linux-hanse/Lambdaschool/Sprint-Challenge--Data-Structures-Python/env" +export VIRTUAL_ENV + +_OLD_VIRTUAL_PATH="$PATH" +PATH="$VIRTUAL_ENV/bin:$PATH" +export PATH + +# unset PYTHONHOME if set +# this will fail if PYTHONHOME is set to the empty string (which is bad anyway) +# could use `if (set -u; : $PYTHONHOME) ;` in bash +if [ -n "${PYTHONHOME:-}" ] ; then + _OLD_VIRTUAL_PYTHONHOME="${PYTHONHOME:-}" + unset PYTHONHOME +fi + +if [ -z "${VIRTUAL_ENV_DISABLE_PROMPT:-}" ] ; then + _OLD_VIRTUAL_PS1="${PS1:-}" + if [ "x(env) " != x ] ; then + PS1="(env) ${PS1:-}" + else + if [ "`basename \"$VIRTUAL_ENV\"`" = "__" ] ; then + # special case for Aspen magic directories + # see http://www.zetadev.com/software/aspen/ + PS1="[`basename \`dirname \"$VIRTUAL_ENV\"\``] $PS1" + else + PS1="(`basename \"$VIRTUAL_ENV\"`)$PS1" + fi + fi + export PS1 +fi + +# This should detect bash and zsh, which have a hash command that must +# be called to get it to forget past commands. Without forgetting +# past commands the $PATH changes we made may not be respected +if [ -n "${BASH:-}" -o -n "${ZSH_VERSION:-}" ] ; then + hash -r +fi diff --git a/env/bin/activate.csh b/env/bin/activate.csh new file mode 100644 index 000000000..f6be95798 --- /dev/null +++ b/env/bin/activate.csh @@ -0,0 +1,37 @@ +# This file must be used with "source bin/activate.csh" *from csh*. +# You cannot run it directly. +# Created by Davide Di Blasi . +# Ported to Python 3.3 venv by Andrew Svetlov + +alias deactivate 'test $?_OLD_VIRTUAL_PATH != 0 && setenv PATH "$_OLD_VIRTUAL_PATH" && unset _OLD_VIRTUAL_PATH; rehash; test $?_OLD_VIRTUAL_PROMPT != 0 && set prompt="$_OLD_VIRTUAL_PROMPT" && unset _OLD_VIRTUAL_PROMPT; unsetenv VIRTUAL_ENV; test "\!:*" != "nondestructive" && unalias deactivate' + +# Unset irrelevant variables. +deactivate nondestructive + +setenv VIRTUAL_ENV "/home/nthansen/linux-hanse/Lambdaschool/Sprint-Challenge--Data-Structures-Python/env" + +set _OLD_VIRTUAL_PATH="$PATH" +setenv PATH "$VIRTUAL_ENV/bin:$PATH" + + +set _OLD_VIRTUAL_PROMPT="$prompt" + +if (! "$?VIRTUAL_ENV_DISABLE_PROMPT") then + if ("env" != "") then + set env_name = "env" + else + if (`basename "VIRTUAL_ENV"` == "__") then + # special case for Aspen magic directories + # see http://www.zetadev.com/software/aspen/ + set env_name = `basename \`dirname "$VIRTUAL_ENV"\`` + else + set env_name = `basename "$VIRTUAL_ENV"` + endif + endif + set prompt = "[$env_name] $prompt" + unset env_name +endif + +alias pydoc python -m pydoc + +rehash diff --git a/env/bin/activate.fish b/env/bin/activate.fish new file mode 100644 index 000000000..03a29a2b8 --- /dev/null +++ b/env/bin/activate.fish @@ -0,0 +1,75 @@ +# This file must be used with ". bin/activate.fish" *from fish* (http://fishshell.org) +# you cannot run it directly + +function deactivate -d "Exit virtualenv and return to normal shell environment" + # reset old environment variables + if test -n "$_OLD_VIRTUAL_PATH" + set -gx PATH $_OLD_VIRTUAL_PATH + set -e _OLD_VIRTUAL_PATH + end + if test -n "$_OLD_VIRTUAL_PYTHONHOME" + set -gx PYTHONHOME $_OLD_VIRTUAL_PYTHONHOME + set -e _OLD_VIRTUAL_PYTHONHOME + end + + if test -n "$_OLD_FISH_PROMPT_OVERRIDE" + functions -e fish_prompt + set -e _OLD_FISH_PROMPT_OVERRIDE + functions -c _old_fish_prompt fish_prompt + functions -e _old_fish_prompt + end + + set -e VIRTUAL_ENV + if test "$argv[1]" != "nondestructive" + # Self destruct! + functions -e deactivate + end +end + +# unset irrelevant variables +deactivate nondestructive + +set -gx VIRTUAL_ENV "/home/nthansen/linux-hanse/Lambdaschool/Sprint-Challenge--Data-Structures-Python/env" + +set -gx _OLD_VIRTUAL_PATH $PATH +set -gx PATH "$VIRTUAL_ENV/bin" $PATH + +# unset PYTHONHOME if set +if set -q PYTHONHOME + set -gx _OLD_VIRTUAL_PYTHONHOME $PYTHONHOME + set -e PYTHONHOME +end + +if test -z "$VIRTUAL_ENV_DISABLE_PROMPT" + # fish uses a function instead of an env var to generate the prompt. + + # save the current fish_prompt function as the function _old_fish_prompt + functions -c fish_prompt _old_fish_prompt + + # with the original prompt function renamed, we can override with our own. + function fish_prompt + # Save the return status of the last command + set -l old_status $status + + # Prompt override? + if test -n "(env) " + printf "%s%s" "(env) " (set_color normal) + else + # ...Otherwise, prepend env + set -l _checkbase (basename "$VIRTUAL_ENV") + if test $_checkbase = "__" + # special case for Aspen magic directories + # see http://www.zetadev.com/software/aspen/ + printf "%s[%s]%s " (set_color -b blue white) (basename (dirname "$VIRTUAL_ENV")) (set_color normal) + else + printf "%s(%s)%s" (set_color -b blue white) (basename "$VIRTUAL_ENV") (set_color normal) + end + end + + # Restore the return status of the previous command. + echo "exit $old_status" | . + _old_fish_prompt + end + + set -gx _OLD_FISH_PROMPT_OVERRIDE "$VIRTUAL_ENV" +end diff --git a/env/bin/autopep8 b/env/bin/autopep8 new file mode 100755 index 000000000..9671d9805 --- /dev/null +++ b/env/bin/autopep8 @@ -0,0 +1,8 @@ +#!/home/nthansen/linux-hanse/Lambdaschool/Sprint-Challenge--Data-Structures-Python/env/bin/python3 +# -*- coding: utf-8 -*- +import re +import sys +from autopep8 import main +if __name__ == '__main__': + sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0]) + sys.exit(main()) diff --git a/env/bin/easy_install b/env/bin/easy_install new file mode 100755 index 000000000..126154e70 --- /dev/null +++ b/env/bin/easy_install @@ -0,0 +1,8 @@ +#!/home/nthansen/linux-hanse/Lambdaschool/Sprint-Challenge--Data-Structures-Python/env/bin/python3 +# -*- coding: utf-8 -*- +import re +import sys +from setuptools.command.easy_install import main +if __name__ == '__main__': + sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0]) + sys.exit(main()) diff --git a/env/bin/easy_install-3.8 b/env/bin/easy_install-3.8 new file mode 100755 index 000000000..126154e70 --- /dev/null +++ b/env/bin/easy_install-3.8 @@ -0,0 +1,8 @@ +#!/home/nthansen/linux-hanse/Lambdaschool/Sprint-Challenge--Data-Structures-Python/env/bin/python3 +# -*- coding: utf-8 -*- +import re +import sys +from setuptools.command.easy_install import main +if __name__ == '__main__': + sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0]) + sys.exit(main()) diff --git a/env/bin/epylint b/env/bin/epylint new file mode 100755 index 000000000..fb37f0964 --- /dev/null +++ b/env/bin/epylint @@ -0,0 +1,8 @@ +#!/home/nthansen/linux-hanse/Lambdaschool/Sprint-Challenge--Data-Structures-Python/env/bin/python3 +# -*- coding: utf-8 -*- +import re +import sys +from pylint import run_epylint +if __name__ == '__main__': + sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0]) + sys.exit(run_epylint()) diff --git a/env/bin/isort b/env/bin/isort new file mode 100755 index 000000000..a76f4cdc2 --- /dev/null +++ b/env/bin/isort @@ -0,0 +1,8 @@ +#!/home/nthansen/linux-hanse/Lambdaschool/Sprint-Challenge--Data-Structures-Python/env/bin/python3 +# -*- coding: utf-8 -*- +import re +import sys +from isort.main import main +if __name__ == '__main__': + sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0]) + sys.exit(main()) diff --git a/env/bin/pip b/env/bin/pip new file mode 100755 index 000000000..6b2709b49 --- /dev/null +++ b/env/bin/pip @@ -0,0 +1,8 @@ +#!/home/nthansen/linux-hanse/Lambdaschool/Sprint-Challenge--Data-Structures-Python/env/bin/python3 +# -*- coding: utf-8 -*- +import re +import sys +from pip._internal.cli.main import main +if __name__ == '__main__': + sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0]) + sys.exit(main()) diff --git a/env/bin/pip3 b/env/bin/pip3 new file mode 100755 index 000000000..6b2709b49 --- /dev/null +++ b/env/bin/pip3 @@ -0,0 +1,8 @@ +#!/home/nthansen/linux-hanse/Lambdaschool/Sprint-Challenge--Data-Structures-Python/env/bin/python3 +# -*- coding: utf-8 -*- +import re +import sys +from pip._internal.cli.main import main +if __name__ == '__main__': + sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0]) + sys.exit(main()) diff --git a/env/bin/pip3.8 b/env/bin/pip3.8 new file mode 100755 index 000000000..6b2709b49 --- /dev/null +++ b/env/bin/pip3.8 @@ -0,0 +1,8 @@ +#!/home/nthansen/linux-hanse/Lambdaschool/Sprint-Challenge--Data-Structures-Python/env/bin/python3 +# -*- coding: utf-8 -*- +import re +import sys +from pip._internal.cli.main import main +if __name__ == '__main__': + sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0]) + sys.exit(main()) diff --git a/env/bin/pycodestyle b/env/bin/pycodestyle new file mode 100755 index 000000000..d3067e63f --- /dev/null +++ b/env/bin/pycodestyle @@ -0,0 +1,8 @@ +#!/home/nthansen/linux-hanse/Lambdaschool/Sprint-Challenge--Data-Structures-Python/env/bin/python3 +# -*- coding: utf-8 -*- +import re +import sys +from pycodestyle import _main +if __name__ == '__main__': + sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0]) + sys.exit(_main()) diff --git a/env/bin/pylint b/env/bin/pylint new file mode 100755 index 000000000..1d7276118 --- /dev/null +++ b/env/bin/pylint @@ -0,0 +1,8 @@ +#!/home/nthansen/linux-hanse/Lambdaschool/Sprint-Challenge--Data-Structures-Python/env/bin/python3 +# -*- coding: utf-8 -*- +import re +import sys +from pylint import run_pylint +if __name__ == '__main__': + sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0]) + sys.exit(run_pylint()) diff --git a/env/bin/pyreverse b/env/bin/pyreverse new file mode 100755 index 000000000..abcafb8dd --- /dev/null +++ b/env/bin/pyreverse @@ -0,0 +1,8 @@ +#!/home/nthansen/linux-hanse/Lambdaschool/Sprint-Challenge--Data-Structures-Python/env/bin/python3 +# -*- coding: utf-8 -*- +import re +import sys +from pylint import run_pyreverse +if __name__ == '__main__': + sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0]) + sys.exit(run_pyreverse()) diff --git a/env/bin/python b/env/bin/python new file mode 120000 index 000000000..b8a0adbbb --- /dev/null +++ b/env/bin/python @@ -0,0 +1 @@ +python3 \ No newline at end of file diff --git a/env/bin/python3 b/env/bin/python3 new file mode 120000 index 000000000..ae65fdaa1 --- /dev/null +++ b/env/bin/python3 @@ -0,0 +1 @@ +/usr/bin/python3 \ No newline at end of file diff --git a/env/bin/symilar b/env/bin/symilar new file mode 100755 index 000000000..83fa6b678 --- /dev/null +++ b/env/bin/symilar @@ -0,0 +1,8 @@ +#!/home/nthansen/linux-hanse/Lambdaschool/Sprint-Challenge--Data-Structures-Python/env/bin/python3 +# -*- coding: utf-8 -*- +import re +import sys +from pylint import run_symilar +if __name__ == '__main__': + sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0]) + sys.exit(run_symilar()) diff --git a/env/lib/python3.8/site-packages/astroid-2.4.2.dist-info/COPYING b/env/lib/python3.8/site-packages/astroid-2.4.2.dist-info/COPYING new file mode 100644 index 000000000..d511905c1 --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid-2.4.2.dist-info/COPYING @@ -0,0 +1,339 @@ + GNU GENERAL PUBLIC LICENSE + Version 2, June 1991 + + Copyright (C) 1989, 1991 Free Software Foundation, Inc., + 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + + Preamble + + The licenses for most software are designed to take away your +freedom to share and change it. By contrast, the GNU General Public +License is intended to guarantee your freedom to share and change free +software--to make sure the software is free for all its users. This +General Public License applies to most of the Free Software +Foundation's software and to any other program whose authors commit to +using it. (Some other Free Software Foundation software is covered by +the GNU Lesser General Public License instead.) You can apply it to +your programs, too. + + When we speak of free software, we are referring to freedom, not +price. Our General Public Licenses are designed to make sure that you +have the freedom to distribute copies of free software (and charge for +this service if you wish), that you receive source code or can get it +if you want it, that you can change the software or use pieces of it +in new free programs; and that you know you can do these things. + + To protect your rights, we need to make restrictions that forbid +anyone to deny you these rights or to ask you to surrender the rights. +These restrictions translate to certain responsibilities for you if you +distribute copies of the software, or if you modify it. + + For example, if you distribute copies of such a program, whether +gratis or for a fee, you must give the recipients all the rights that +you have. You must make sure that they, too, receive or can get the +source code. And you must show them these terms so they know their +rights. + + We protect your rights with two steps: (1) copyright the software, and +(2) offer you this license which gives you legal permission to copy, +distribute and/or modify the software. + + Also, for each author's protection and ours, we want to make certain +that everyone understands that there is no warranty for this free +software. If the software is modified by someone else and passed on, we +want its recipients to know that what they have is not the original, so +that any problems introduced by others will not reflect on the original +authors' reputations. + + Finally, any free program is threatened constantly by software +patents. We wish to avoid the danger that redistributors of a free +program will individually obtain patent licenses, in effect making the +program proprietary. To prevent this, we have made it clear that any +patent must be licensed for everyone's free use or not licensed at all. + + The precise terms and conditions for copying, distribution and +modification follow. + + GNU GENERAL PUBLIC LICENSE + TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION + + 0. This License applies to any program or other work which contains +a notice placed by the copyright holder saying it may be distributed +under the terms of this General Public License. The "Program", below, +refers to any such program or work, and a "work based on the Program" +means either the Program or any derivative work under copyright law: +that is to say, a work containing the Program or a portion of it, +either verbatim or with modifications and/or translated into another +language. (Hereinafter, translation is included without limitation in +the term "modification".) Each licensee is addressed as "you". + +Activities other than copying, distribution and modification are not +covered by this License; they are outside its scope. The act of +running the Program is not restricted, and the output from the Program +is covered only if its contents constitute a work based on the +Program (independent of having been made by running the Program). +Whether that is true depends on what the Program does. + + 1. You may copy and distribute verbatim copies of the Program's +source code as you receive it, in any medium, provided that you +conspicuously and appropriately publish on each copy an appropriate +copyright notice and disclaimer of warranty; keep intact all the +notices that refer to this License and to the absence of any warranty; +and give any other recipients of the Program a copy of this License +along with the Program. + +You may charge a fee for the physical act of transferring a copy, and +you may at your option offer warranty protection in exchange for a fee. + + 2. You may modify your copy or copies of the Program or any portion +of it, thus forming a work based on the Program, and copy and +distribute such modifications or work under the terms of Section 1 +above, provided that you also meet all of these conditions: + + a) You must cause the modified files to carry prominent notices + stating that you changed the files and the date of any change. + + b) You must cause any work that you distribute or publish, that in + whole or in part contains or is derived from the Program or any + part thereof, to be licensed as a whole at no charge to all third + parties under the terms of this License. + + c) If the modified program normally reads commands interactively + when run, you must cause it, when started running for such + interactive use in the most ordinary way, to print or display an + announcement including an appropriate copyright notice and a + notice that there is no warranty (or else, saying that you provide + a warranty) and that users may redistribute the program under + these conditions, and telling the user how to view a copy of this + License. (Exception: if the Program itself is interactive but + does not normally print such an announcement, your work based on + the Program is not required to print an announcement.) + +These requirements apply to the modified work as a whole. If +identifiable sections of that work are not derived from the Program, +and can be reasonably considered independent and separate works in +themselves, then this License, and its terms, do not apply to those +sections when you distribute them as separate works. But when you +distribute the same sections as part of a whole which is a work based +on the Program, the distribution of the whole must be on the terms of +this License, whose permissions for other licensees extend to the +entire whole, and thus to each and every part regardless of who wrote it. + +Thus, it is not the intent of this section to claim rights or contest +your rights to work written entirely by you; rather, the intent is to +exercise the right to control the distribution of derivative or +collective works based on the Program. + +In addition, mere aggregation of another work not based on the Program +with the Program (or with a work based on the Program) on a volume of +a storage or distribution medium does not bring the other work under +the scope of this License. + + 3. You may copy and distribute the Program (or a work based on it, +under Section 2) in object code or executable form under the terms of +Sections 1 and 2 above provided that you also do one of the following: + + a) Accompany it with the complete corresponding machine-readable + source code, which must be distributed under the terms of Sections + 1 and 2 above on a medium customarily used for software interchange; or, + + b) Accompany it with a written offer, valid for at least three + years, to give any third party, for a charge no more than your + cost of physically performing source distribution, a complete + machine-readable copy of the corresponding source code, to be + distributed under the terms of Sections 1 and 2 above on a medium + customarily used for software interchange; or, + + c) Accompany it with the information you received as to the offer + to distribute corresponding source code. (This alternative is + allowed only for noncommercial distribution and only if you + received the program in object code or executable form with such + an offer, in accord with Subsection b above.) + +The source code for a work means the preferred form of the work for +making modifications to it. For an executable work, complete source +code means all the source code for all modules it contains, plus any +associated interface definition files, plus the scripts used to +control compilation and installation of the executable. However, as a +special exception, the source code distributed need not include +anything that is normally distributed (in either source or binary +form) with the major components (compiler, kernel, and so on) of the +operating system on which the executable runs, unless that component +itself accompanies the executable. + +If distribution of executable or object code is made by offering +access to copy from a designated place, then offering equivalent +access to copy the source code from the same place counts as +distribution of the source code, even though third parties are not +compelled to copy the source along with the object code. + + 4. You may not copy, modify, sublicense, or distribute the Program +except as expressly provided under this License. Any attempt +otherwise to copy, modify, sublicense or distribute the Program is +void, and will automatically terminate your rights under this License. +However, parties who have received copies, or rights, from you under +this License will not have their licenses terminated so long as such +parties remain in full compliance. + + 5. You are not required to accept this License, since you have not +signed it. However, nothing else grants you permission to modify or +distribute the Program or its derivative works. These actions are +prohibited by law if you do not accept this License. Therefore, by +modifying or distributing the Program (or any work based on the +Program), you indicate your acceptance of this License to do so, and +all its terms and conditions for copying, distributing or modifying +the Program or works based on it. + + 6. Each time you redistribute the Program (or any work based on the +Program), the recipient automatically receives a license from the +original licensor to copy, distribute or modify the Program subject to +these terms and conditions. You may not impose any further +restrictions on the recipients' exercise of the rights granted herein. +You are not responsible for enforcing compliance by third parties to +this License. + + 7. If, as a consequence of a court judgment or allegation of patent +infringement or for any other reason (not limited to patent issues), +conditions are imposed on you (whether by court order, agreement or +otherwise) that contradict the conditions of this License, they do not +excuse you from the conditions of this License. If you cannot +distribute so as to satisfy simultaneously your obligations under this +License and any other pertinent obligations, then as a consequence you +may not distribute the Program at all. For example, if a patent +license would not permit royalty-free redistribution of the Program by +all those who receive copies directly or indirectly through you, then +the only way you could satisfy both it and this License would be to +refrain entirely from distribution of the Program. + +If any portion of this section is held invalid or unenforceable under +any particular circumstance, the balance of the section is intended to +apply and the section as a whole is intended to apply in other +circumstances. + +It is not the purpose of this section to induce you to infringe any +patents or other property right claims or to contest validity of any +such claims; this section has the sole purpose of protecting the +integrity of the free software distribution system, which is +implemented by public license practices. Many people have made +generous contributions to the wide range of software distributed +through that system in reliance on consistent application of that +system; it is up to the author/donor to decide if he or she is willing +to distribute software through any other system and a licensee cannot +impose that choice. + +This section is intended to make thoroughly clear what is believed to +be a consequence of the rest of this License. + + 8. If the distribution and/or use of the Program is restricted in +certain countries either by patents or by copyrighted interfaces, the +original copyright holder who places the Program under this License +may add an explicit geographical distribution limitation excluding +those countries, so that distribution is permitted only in or among +countries not thus excluded. In such case, this License incorporates +the limitation as if written in the body of this License. + + 9. The Free Software Foundation may publish revised and/or new versions +of the General Public License from time to time. Such new versions will +be similar in spirit to the present version, but may differ in detail to +address new problems or concerns. + +Each version is given a distinguishing version number. If the Program +specifies a version number of this License which applies to it and "any +later version", you have the option of following the terms and conditions +either of that version or of any later version published by the Free +Software Foundation. If the Program does not specify a version number of +this License, you may choose any version ever published by the Free Software +Foundation. + + 10. If you wish to incorporate parts of the Program into other free +programs whose distribution conditions are different, write to the author +to ask for permission. For software which is copyrighted by the Free +Software Foundation, write to the Free Software Foundation; we sometimes +make exceptions for this. Our decision will be guided by the two goals +of preserving the free status of all derivatives of our free software and +of promoting the sharing and reuse of software generally. + + NO WARRANTY + + 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY +FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN +OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES +PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED +OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS +TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE +PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, +REPAIR OR CORRECTION. + + 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING +WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR +REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, +INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING +OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED +TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY +YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER +PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE +POSSIBILITY OF SUCH DAMAGES. + + END OF TERMS AND CONDITIONS + + How to Apply These Terms to Your New Programs + + If you develop a new program, and you want it to be of the greatest +possible use to the public, the best way to achieve this is to make it +free software which everyone can redistribute and change under these terms. + + To do so, attach the following notices to the program. It is safest +to attach them to the start of each source file to most effectively +convey the exclusion of warranty; and each file should have at least +the "copyright" line and a pointer to where the full notice is found. + + + Copyright (C) + + This program is free software; you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation; either version 2 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License along + with this program; if not, write to the Free Software Foundation, Inc., + 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. + +Also add information on how to contact you by electronic and paper mail. + +If the program is interactive, make it output a short notice like this +when it starts in an interactive mode: + + Gnomovision version 69, Copyright (C) year name of author + Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'. + This is free software, and you are welcome to redistribute it + under certain conditions; type `show c' for details. + +The hypothetical commands `show w' and `show c' should show the appropriate +parts of the General Public License. Of course, the commands you use may +be called something other than `show w' and `show c'; they could even be +mouse-clicks or menu items--whatever suits your program. + +You should also get your employer (if you work as a programmer) or your +school, if any, to sign a "copyright disclaimer" for the program, if +necessary. Here is a sample; alter the names: + + Yoyodyne, Inc., hereby disclaims all copyright interest in the program + `Gnomovision' (which makes passes at compilers) written by James Hacker. + + , 1 April 1989 + Ty Coon, President of Vice + +This General Public License does not permit incorporating your program into +proprietary programs. If your program is a subroutine library, you may +consider it more useful to permit linking proprietary applications with the +library. If this is what you want to do, use the GNU Lesser General +Public License instead of this License. diff --git a/env/lib/python3.8/site-packages/astroid-2.4.2.dist-info/COPYING.LESSER b/env/lib/python3.8/site-packages/astroid-2.4.2.dist-info/COPYING.LESSER new file mode 100644 index 000000000..2d2d780e6 --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid-2.4.2.dist-info/COPYING.LESSER @@ -0,0 +1,510 @@ + + GNU LESSER GENERAL PUBLIC LICENSE + Version 2.1, February 1999 + + Copyright (C) 1991, 1999 Free Software Foundation, Inc. + 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + +[This is the first released version of the Lesser GPL. It also counts + as the successor of the GNU Library Public License, version 2, hence + the version number 2.1.] + + Preamble + + The licenses for most software are designed to take away your +freedom to share and change it. By contrast, the GNU General Public +Licenses are intended to guarantee your freedom to share and change +free software--to make sure the software is free for all its users. + + This license, the Lesser General Public License, applies to some +specially designated software packages--typically libraries--of the +Free Software Foundation and other authors who decide to use it. You +can use it too, but we suggest you first think carefully about whether +this license or the ordinary General Public License is the better +strategy to use in any particular case, based on the explanations +below. + + When we speak of free software, we are referring to freedom of use, +not price. Our General Public Licenses are designed to make sure that +you have the freedom to distribute copies of free software (and charge +for this service if you wish); that you receive source code or can get +it if you want it; that you can change the software and use pieces of +it in new free programs; and that you are informed that you can do +these things. + + To protect your rights, we need to make restrictions that forbid +distributors to deny you these rights or to ask you to surrender these +rights. These restrictions translate to certain responsibilities for +you if you distribute copies of the library or if you modify it. + + For example, if you distribute copies of the library, whether gratis +or for a fee, you must give the recipients all the rights that we gave +you. You must make sure that they, too, receive or can get the source +code. If you link other code with the library, you must provide +complete object files to the recipients, so that they can relink them +with the library after making changes to the library and recompiling +it. And you must show them these terms so they know their rights. + + We protect your rights with a two-step method: (1) we copyright the +library, and (2) we offer you this license, which gives you legal +permission to copy, distribute and/or modify the library. + + To protect each distributor, we want to make it very clear that +there is no warranty for the free library. Also, if the library is +modified by someone else and passed on, the recipients should know +that what they have is not the original version, so that the original +author's reputation will not be affected by problems that might be +introduced by others. + + Finally, software patents pose a constant threat to the existence of +any free program. We wish to make sure that a company cannot +effectively restrict the users of a free program by obtaining a +restrictive license from a patent holder. Therefore, we insist that +any patent license obtained for a version of the library must be +consistent with the full freedom of use specified in this license. + + Most GNU software, including some libraries, is covered by the +ordinary GNU General Public License. This license, the GNU Lesser +General Public License, applies to certain designated libraries, and +is quite different from the ordinary General Public License. We use +this license for certain libraries in order to permit linking those +libraries into non-free programs. + + When a program is linked with a library, whether statically or using +a shared library, the combination of the two is legally speaking a +combined work, a derivative of the original library. The ordinary +General Public License therefore permits such linking only if the +entire combination fits its criteria of freedom. The Lesser General +Public License permits more lax criteria for linking other code with +the library. + + We call this license the "Lesser" General Public License because it +does Less to protect the user's freedom than the ordinary General +Public License. It also provides other free software developers Less +of an advantage over competing non-free programs. These disadvantages +are the reason we use the ordinary General Public License for many +libraries. However, the Lesser license provides advantages in certain +special circumstances. + + For example, on rare occasions, there may be a special need to +encourage the widest possible use of a certain library, so that it +becomes a de-facto standard. To achieve this, non-free programs must +be allowed to use the library. A more frequent case is that a free +library does the same job as widely used non-free libraries. In this +case, there is little to gain by limiting the free library to free +software only, so we use the Lesser General Public License. + + In other cases, permission to use a particular library in non-free +programs enables a greater number of people to use a large body of +free software. For example, permission to use the GNU C Library in +non-free programs enables many more people to use the whole GNU +operating system, as well as its variant, the GNU/Linux operating +system. + + Although the Lesser General Public License is Less protective of the +users' freedom, it does ensure that the user of a program that is +linked with the Library has the freedom and the wherewithal to run +that program using a modified version of the Library. + + The precise terms and conditions for copying, distribution and +modification follow. Pay close attention to the difference between a +"work based on the library" and a "work that uses the library". The +former contains code derived from the library, whereas the latter must +be combined with the library in order to run. + + GNU LESSER GENERAL PUBLIC LICENSE + TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION + + 0. This License Agreement applies to any software library or other +program which contains a notice placed by the copyright holder or +other authorized party saying it may be distributed under the terms of +this Lesser General Public License (also called "this License"). +Each licensee is addressed as "you". + + A "library" means a collection of software functions and/or data +prepared so as to be conveniently linked with application programs +(which use some of those functions and data) to form executables. + + The "Library", below, refers to any such software library or work +which has been distributed under these terms. A "work based on the +Library" means either the Library or any derivative work under +copyright law: that is to say, a work containing the Library or a +portion of it, either verbatim or with modifications and/or translated +straightforwardly into another language. (Hereinafter, translation is +included without limitation in the term "modification".) + + "Source code" for a work means the preferred form of the work for +making modifications to it. For a library, complete source code means +all the source code for all modules it contains, plus any associated +interface definition files, plus the scripts used to control +compilation and installation of the library. + + Activities other than copying, distribution and modification are not +covered by this License; they are outside its scope. The act of +running a program using the Library is not restricted, and output from +such a program is covered only if its contents constitute a work based +on the Library (independent of the use of the Library in a tool for +writing it). Whether that is true depends on what the Library does +and what the program that uses the Library does. + + 1. You may copy and distribute verbatim copies of the Library's +complete source code as you receive it, in any medium, provided that +you conspicuously and appropriately publish on each copy an +appropriate copyright notice and disclaimer of warranty; keep intact +all the notices that refer to this License and to the absence of any +warranty; and distribute a copy of this License along with the +Library. + + You may charge a fee for the physical act of transferring a copy, +and you may at your option offer warranty protection in exchange for a +fee. + + 2. You may modify your copy or copies of the Library or any portion +of it, thus forming a work based on the Library, and copy and +distribute such modifications or work under the terms of Section 1 +above, provided that you also meet all of these conditions: + + a) The modified work must itself be a software library. + + b) You must cause the files modified to carry prominent notices + stating that you changed the files and the date of any change. + + c) You must cause the whole of the work to be licensed at no + charge to all third parties under the terms of this License. + + d) If a facility in the modified Library refers to a function or a + table of data to be supplied by an application program that uses + the facility, other than as an argument passed when the facility + is invoked, then you must make a good faith effort to ensure that, + in the event an application does not supply such function or + table, the facility still operates, and performs whatever part of + its purpose remains meaningful. + + (For example, a function in a library to compute square roots has + a purpose that is entirely well-defined independent of the + application. Therefore, Subsection 2d requires that any + application-supplied function or table used by this function must + be optional: if the application does not supply it, the square + root function must still compute square roots.) + +These requirements apply to the modified work as a whole. If +identifiable sections of that work are not derived from the Library, +and can be reasonably considered independent and separate works in +themselves, then this License, and its terms, do not apply to those +sections when you distribute them as separate works. But when you +distribute the same sections as part of a whole which is a work based +on the Library, the distribution of the whole must be on the terms of +this License, whose permissions for other licensees extend to the +entire whole, and thus to each and every part regardless of who wrote +it. + +Thus, it is not the intent of this section to claim rights or contest +your rights to work written entirely by you; rather, the intent is to +exercise the right to control the distribution of derivative or +collective works based on the Library. + +In addition, mere aggregation of another work not based on the Library +with the Library (or with a work based on the Library) on a volume of +a storage or distribution medium does not bring the other work under +the scope of this License. + + 3. You may opt to apply the terms of the ordinary GNU General Public +License instead of this License to a given copy of the Library. To do +this, you must alter all the notices that refer to this License, so +that they refer to the ordinary GNU General Public License, version 2, +instead of to this License. (If a newer version than version 2 of the +ordinary GNU General Public License has appeared, then you can specify +that version instead if you wish.) Do not make any other change in +these notices. + + Once this change is made in a given copy, it is irreversible for +that copy, so the ordinary GNU General Public License applies to all +subsequent copies and derivative works made from that copy. + + This option is useful when you wish to copy part of the code of +the Library into a program that is not a library. + + 4. You may copy and distribute the Library (or a portion or +derivative of it, under Section 2) in object code or executable form +under the terms of Sections 1 and 2 above provided that you accompany +it with the complete corresponding machine-readable source code, which +must be distributed under the terms of Sections 1 and 2 above on a +medium customarily used for software interchange. + + If distribution of object code is made by offering access to copy +from a designated place, then offering equivalent access to copy the +source code from the same place satisfies the requirement to +distribute the source code, even though third parties are not +compelled to copy the source along with the object code. + + 5. A program that contains no derivative of any portion of the +Library, but is designed to work with the Library by being compiled or +linked with it, is called a "work that uses the Library". Such a +work, in isolation, is not a derivative work of the Library, and +therefore falls outside the scope of this License. + + However, linking a "work that uses the Library" with the Library +creates an executable that is a derivative of the Library (because it +contains portions of the Library), rather than a "work that uses the +library". The executable is therefore covered by this License. +Section 6 states terms for distribution of such executables. + + When a "work that uses the Library" uses material from a header file +that is part of the Library, the object code for the work may be a +derivative work of the Library even though the source code is not. +Whether this is true is especially significant if the work can be +linked without the Library, or if the work is itself a library. The +threshold for this to be true is not precisely defined by law. + + If such an object file uses only numerical parameters, data +structure layouts and accessors, and small macros and small inline +functions (ten lines or less in length), then the use of the object +file is unrestricted, regardless of whether it is legally a derivative +work. (Executables containing this object code plus portions of the +Library will still fall under Section 6.) + + Otherwise, if the work is a derivative of the Library, you may +distribute the object code for the work under the terms of Section 6. +Any executables containing that work also fall under Section 6, +whether or not they are linked directly with the Library itself. + + 6. As an exception to the Sections above, you may also combine or +link a "work that uses the Library" with the Library to produce a +work containing portions of the Library, and distribute that work +under terms of your choice, provided that the terms permit +modification of the work for the customer's own use and reverse +engineering for debugging such modifications. + + You must give prominent notice with each copy of the work that the +Library is used in it and that the Library and its use are covered by +this License. You must supply a copy of this License. If the work +during execution displays copyright notices, you must include the +copyright notice for the Library among them, as well as a reference +directing the user to the copy of this License. Also, you must do one +of these things: + + a) Accompany the work with the complete corresponding + machine-readable source code for the Library including whatever + changes were used in the work (which must be distributed under + Sections 1 and 2 above); and, if the work is an executable linked + with the Library, with the complete machine-readable "work that + uses the Library", as object code and/or source code, so that the + user can modify the Library and then relink to produce a modified + executable containing the modified Library. (It is understood + that the user who changes the contents of definitions files in the + Library will not necessarily be able to recompile the application + to use the modified definitions.) + + b) Use a suitable shared library mechanism for linking with the + Library. A suitable mechanism is one that (1) uses at run time a + copy of the library already present on the user's computer system, + rather than copying library functions into the executable, and (2) + will operate properly with a modified version of the library, if + the user installs one, as long as the modified version is + interface-compatible with the version that the work was made with. + + c) Accompany the work with a written offer, valid for at least + three years, to give the same user the materials specified in + Subsection 6a, above, for a charge no more than the cost of + performing this distribution. + + d) If distribution of the work is made by offering access to copy + from a designated place, offer equivalent access to copy the above + specified materials from the same place. + + e) Verify that the user has already received a copy of these + materials or that you have already sent this user a copy. + + For an executable, the required form of the "work that uses the +Library" must include any data and utility programs needed for +reproducing the executable from it. However, as a special exception, +the materials to be distributed need not include anything that is +normally distributed (in either source or binary form) with the major +components (compiler, kernel, and so on) of the operating system on +which the executable runs, unless that component itself accompanies +the executable. + + It may happen that this requirement contradicts the license +restrictions of other proprietary libraries that do not normally +accompany the operating system. Such a contradiction means you cannot +use both them and the Library together in an executable that you +distribute. + + 7. You may place library facilities that are a work based on the +Library side-by-side in a single library together with other library +facilities not covered by this License, and distribute such a combined +library, provided that the separate distribution of the work based on +the Library and of the other library facilities is otherwise +permitted, and provided that you do these two things: + + a) Accompany the combined library with a copy of the same work + based on the Library, uncombined with any other library + facilities. This must be distributed under the terms of the + Sections above. + + b) Give prominent notice with the combined library of the fact + that part of it is a work based on the Library, and explaining + where to find the accompanying uncombined form of the same work. + + 8. You may not copy, modify, sublicense, link with, or distribute +the Library except as expressly provided under this License. Any +attempt otherwise to copy, modify, sublicense, link with, or +distribute the Library is void, and will automatically terminate your +rights under this License. However, parties who have received copies, +or rights, from you under this License will not have their licenses +terminated so long as such parties remain in full compliance. + + 9. You are not required to accept this License, since you have not +signed it. However, nothing else grants you permission to modify or +distribute the Library or its derivative works. These actions are +prohibited by law if you do not accept this License. Therefore, by +modifying or distributing the Library (or any work based on the +Library), you indicate your acceptance of this License to do so, and +all its terms and conditions for copying, distributing or modifying +the Library or works based on it. + + 10. Each time you redistribute the Library (or any work based on the +Library), the recipient automatically receives a license from the +original licensor to copy, distribute, link with or modify the Library +subject to these terms and conditions. You may not impose any further +restrictions on the recipients' exercise of the rights granted herein. +You are not responsible for enforcing compliance by third parties with +this License. + + 11. If, as a consequence of a court judgment or allegation of patent +infringement or for any other reason (not limited to patent issues), +conditions are imposed on you (whether by court order, agreement or +otherwise) that contradict the conditions of this License, they do not +excuse you from the conditions of this License. If you cannot +distribute so as to satisfy simultaneously your obligations under this +License and any other pertinent obligations, then as a consequence you +may not distribute the Library at all. For example, if a patent +license would not permit royalty-free redistribution of the Library by +all those who receive copies directly or indirectly through you, then +the only way you could satisfy both it and this License would be to +refrain entirely from distribution of the Library. + +If any portion of this section is held invalid or unenforceable under +any particular circumstance, the balance of the section is intended to +apply, and the section as a whole is intended to apply in other +circumstances. + +It is not the purpose of this section to induce you to infringe any +patents or other property right claims or to contest validity of any +such claims; this section has the sole purpose of protecting the +integrity of the free software distribution system which is +implemented by public license practices. Many people have made +generous contributions to the wide range of software distributed +through that system in reliance on consistent application of that +system; it is up to the author/donor to decide if he or she is willing +to distribute software through any other system and a licensee cannot +impose that choice. + +This section is intended to make thoroughly clear what is believed to +be a consequence of the rest of this License. + + 12. If the distribution and/or use of the Library is restricted in +certain countries either by patents or by copyrighted interfaces, the +original copyright holder who places the Library under this License +may add an explicit geographical distribution limitation excluding those +countries, so that distribution is permitted only in or among +countries not thus excluded. In such case, this License incorporates +the limitation as if written in the body of this License. + + 13. The Free Software Foundation may publish revised and/or new +versions of the Lesser General Public License from time to time. +Such new versions will be similar in spirit to the present version, +but may differ in detail to address new problems or concerns. + +Each version is given a distinguishing version number. If the Library +specifies a version number of this License which applies to it and +"any later version", you have the option of following the terms and +conditions either of that version or of any later version published by +the Free Software Foundation. If the Library does not specify a +license version number, you may choose any version ever published by +the Free Software Foundation. + + 14. If you wish to incorporate parts of the Library into other free +programs whose distribution conditions are incompatible with these, +write to the author to ask for permission. For software which is +copyrighted by the Free Software Foundation, write to the Free +Software Foundation; we sometimes make exceptions for this. Our +decision will be guided by the two goals of preserving the free status +of all derivatives of our free software and of promoting the sharing +and reuse of software generally. + + NO WARRANTY + + 15. BECAUSE THE LIBRARY IS LICENSED FREE OF CHARGE, THERE IS NO +WARRANTY FOR THE LIBRARY, TO THE EXTENT PERMITTED BY APPLICABLE LAW. +EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR +OTHER PARTIES PROVIDE THE LIBRARY "AS IS" WITHOUT WARRANTY OF ANY +KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR +PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE +LIBRARY IS WITH YOU. SHOULD THE LIBRARY PROVE DEFECTIVE, YOU ASSUME +THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION. + + 16. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN +WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY +AND/OR REDISTRIBUTE THE LIBRARY AS PERMITTED ABOVE, BE LIABLE TO YOU +FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR +CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE +LIBRARY (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING +RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A +FAILURE OF THE LIBRARY TO OPERATE WITH ANY OTHER SOFTWARE), EVEN IF +SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH +DAMAGES. + + END OF TERMS AND CONDITIONS + + How to Apply These Terms to Your New Libraries + + If you develop a new library, and you want it to be of the greatest +possible use to the public, we recommend making it free software that +everyone can redistribute and change. You can do so by permitting +redistribution under these terms (or, alternatively, under the terms +of the ordinary General Public License). + + To apply these terms, attach the following notices to the library. +It is safest to attach them to the start of each source file to most +effectively convey the exclusion of warranty; and each file should +have at least the "copyright" line and a pointer to where the full +notice is found. + + + + Copyright (C) + + This library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + This library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with this library; if not, write to the Free Software + Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA + +Also add information on how to contact you by electronic and paper mail. + +You should also get your employer (if you work as a programmer) or +your school, if any, to sign a "copyright disclaimer" for the library, +if necessary. Here is a sample; alter the names: + + Yoyodyne, Inc., hereby disclaims all copyright interest in the + library `Frob' (a library for tweaking knobs) written by James + Random Hacker. + + , 1 April 1990 + Ty Coon, President of Vice + +That's all there is to it! + + diff --git a/env/lib/python3.8/site-packages/astroid-2.4.2.dist-info/INSTALLER b/env/lib/python3.8/site-packages/astroid-2.4.2.dist-info/INSTALLER new file mode 100644 index 000000000..a1b589e38 --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid-2.4.2.dist-info/INSTALLER @@ -0,0 +1 @@ +pip diff --git a/env/lib/python3.8/site-packages/astroid-2.4.2.dist-info/METADATA b/env/lib/python3.8/site-packages/astroid-2.4.2.dist-info/METADATA new file mode 100644 index 000000000..47dad9426 --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid-2.4.2.dist-info/METADATA @@ -0,0 +1,118 @@ +Metadata-Version: 2.1 +Name: astroid +Version: 2.4.2 +Summary: An abstract syntax tree for Python with inference support. +Home-page: https://github.com/PyCQA/astroid +Author: Python Code Quality Authority +Author-email: code-quality@python.org +License: LGPL +Platform: UNKNOWN +Classifier: Topic :: Software Development :: Libraries :: Python Modules +Classifier: Topic :: Software Development :: Quality Assurance +Classifier: Programming Language :: Python +Classifier: Programming Language :: Python :: 3 +Classifier: Programming Language :: Python :: 3.5 +Classifier: Programming Language :: Python :: 3.6 +Classifier: Programming Language :: Python :: 3.7 +Classifier: Programming Language :: Python :: 3.8 +Classifier: Programming Language :: Python :: Implementation :: CPython +Classifier: Programming Language :: Python :: Implementation :: PyPy +Requires-Python: >=3.5 +Requires-Dist: lazy-object-proxy (==1.4.*) +Requires-Dist: six (~=1.12) +Requires-Dist: wrapt (~=1.11) +Requires-Dist: typed-ast (<1.5,>=1.4.0) ; implementation_name == "cpython" and python_version < "3.8" + +Astroid +======= + +.. image:: https://travis-ci.org/PyCQA/astroid.svg?branch=master + :target: https://travis-ci.org/PyCQA/astroid + +.. image:: https://ci.appveyor.com/api/projects/status/co3u42kunguhbh6l/branch/master?svg=true + :alt: AppVeyor Build Status + :target: https://ci.appveyor.com/project/PCManticore/astroid + +.. image:: https://coveralls.io/repos/github/PyCQA/astroid/badge.svg?branch=master + :target: https://coveralls.io/github/PyCQA/astroid?branch=master + +.. image:: https://readthedocs.org/projects/astroid/badge/?version=latest + :target: http://astroid.readthedocs.io/en/latest/?badge=latest + :alt: Documentation Status + +.. image:: https://img.shields.io/badge/code%20style-black-000000.svg + :target: https://github.com/ambv/black + +.. |tideliftlogo| image:: doc/media/Tidelift_Logos_RGB_Tidelift_Shorthand_On-White_small.png + :width: 75 + :height: 60 + :alt: Tidelift + +.. list-table:: + :widths: 10 100 + + * - |tideliftlogo| + - Professional support for astroid is available as part of the `Tidelift + Subscription`_. Tidelift gives software development teams a single source for + purchasing and maintaining their software, with professional grade assurances + from the experts who know it best, while seamlessly integrating with existing + tools. + +.. _Tidelift Subscription: https://tidelift.com/subscription/pkg/pypi-astroid?utm_source=pypi-astroid&utm_medium=referral&utm_campaign=readme + + + +What's this? +------------ + +The aim of this module is to provide a common base representation of +python source code. It is currently the library powering pylint's capabilities. + +It provides a compatible representation which comes from the `_ast` +module. It rebuilds the tree generated by the builtin _ast module by +recursively walking down the AST and building an extended ast. The new +node classes have additional methods and attributes for different +usages. They include some support for static inference and local name +scopes. Furthermore, astroid can also build partial trees by inspecting living +objects. + + +Installation +------------ + +Extract the tarball, jump into the created directory and run:: + + pip install . + + +If you want to do an editable installation, you can run:: + + pip install -e . + + +If you have any questions, please mail the code-quality@python.org +mailing list for support. See +http://mail.python.org/mailman/listinfo/code-quality for subscription +information and archives. + +Documentation +------------- +http://astroid.readthedocs.io/en/latest/ + + +Python Versions +--------------- + +astroid 2.0 is currently available for Python 3 only. If you want Python 2 +support, older versions of astroid will still supported until 2020. + +Test +---- + +Tests are in the 'test' subdirectory. To launch the whole tests suite, you can use +either `tox` or `pytest`:: + + tox + pytest astroid + + diff --git a/env/lib/python3.8/site-packages/astroid-2.4.2.dist-info/RECORD b/env/lib/python3.8/site-packages/astroid-2.4.2.dist-info/RECORD new file mode 100644 index 000000000..74774657b --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid-2.4.2.dist-info/RECORD @@ -0,0 +1,151 @@ +astroid-2.4.2.dist-info/COPYING,sha256=qxX9UmvY3Rip5368E5ZWv00z6X_HI4zRG_YOK5uGZsY,17987 +astroid-2.4.2.dist-info/COPYING.LESSER,sha256=qb3eVhbs3R6YC0TzYGAO6Hg7H5m4zIOivrFjoKOQ6GE,26527 +astroid-2.4.2.dist-info/INSTALLER,sha256=zuuue4knoyJ-UwPPXg8fezS7VCrXJQrAP7zeNuwvFQg,4 +astroid-2.4.2.dist-info/METADATA,sha256=mWlOM8vl6thD_vA7Lzn2yKmQuaX8NukwS72gEpsGM0U,3915 +astroid-2.4.2.dist-info/RECORD,, +astroid-2.4.2.dist-info/WHEEL,sha256=g4nMs7d-Xl9-xC9XovUrsDHGXt-FT0E17Yqo92DEfvY,92 +astroid-2.4.2.dist-info/top_level.txt,sha256=HsdW4O2x7ZXRj6k-agi3RaQybGLobI3VSE-jt4vQUXM,8 +astroid/__init__.py,sha256=zZF5EWBTfOtOcFd62WMbhLAtS2b_Fm-3JJW2AVUUMUI,5655 +astroid/__pkginfo__.py,sha256=G6QlkatrWucF445cQW7Kb2ZSgBzf7TiKLPwqkP0CoQU,2329 +astroid/__pycache__/__init__.cpython-38.pyc,, +astroid/__pycache__/__pkginfo__.cpython-38.pyc,, +astroid/__pycache__/_ast.cpython-38.pyc,, +astroid/__pycache__/arguments.cpython-38.pyc,, +astroid/__pycache__/as_string.cpython-38.pyc,, +astroid/__pycache__/bases.cpython-38.pyc,, +astroid/__pycache__/builder.cpython-38.pyc,, +astroid/__pycache__/context.cpython-38.pyc,, +astroid/__pycache__/decorators.cpython-38.pyc,, +astroid/__pycache__/exceptions.cpython-38.pyc,, +astroid/__pycache__/helpers.cpython-38.pyc,, +astroid/__pycache__/inference.cpython-38.pyc,, +astroid/__pycache__/manager.cpython-38.pyc,, +astroid/__pycache__/mixins.cpython-38.pyc,, +astroid/__pycache__/modutils.cpython-38.pyc,, +astroid/__pycache__/node_classes.cpython-38.pyc,, +astroid/__pycache__/nodes.cpython-38.pyc,, +astroid/__pycache__/objects.cpython-38.pyc,, +astroid/__pycache__/protocols.cpython-38.pyc,, +astroid/__pycache__/raw_building.cpython-38.pyc,, +astroid/__pycache__/rebuilder.cpython-38.pyc,, +astroid/__pycache__/scoped_nodes.cpython-38.pyc,, +astroid/__pycache__/test_utils.cpython-38.pyc,, +astroid/__pycache__/transforms.cpython-38.pyc,, +astroid/__pycache__/util.cpython-38.pyc,, +astroid/_ast.py,sha256=n1U2HblBNrMhsmrjF84HUfIgEZ3PyHyiJJOrOkgiUFk,3385 +astroid/arguments.py,sha256=KJcZn7HeEhj2fZICr-9Pi7iueCNDZRQWh0T7O-qb-AE,12558 +astroid/as_string.py,sha256=8bOZWsGG79TLmlzRzPtmHdanIAqQUFTKiYH873iMhOw,22821 +astroid/bases.py,sha256=wL9C23mHFaj7vhMqM83DdsU6kfP488aEjoFWaO8p6Cg,19175 +astroid/brain/__pycache__/brain_argparse.cpython-38.pyc,, +astroid/brain/__pycache__/brain_attrs.cpython-38.pyc,, +astroid/brain/__pycache__/brain_boto3.cpython-38.pyc,, +astroid/brain/__pycache__/brain_builtin_inference.cpython-38.pyc,, +astroid/brain/__pycache__/brain_collections.cpython-38.pyc,, +astroid/brain/__pycache__/brain_crypt.cpython-38.pyc,, +astroid/brain/__pycache__/brain_curses.cpython-38.pyc,, +astroid/brain/__pycache__/brain_dataclasses.cpython-38.pyc,, +astroid/brain/__pycache__/brain_dateutil.cpython-38.pyc,, +astroid/brain/__pycache__/brain_fstrings.cpython-38.pyc,, +astroid/brain/__pycache__/brain_functools.cpython-38.pyc,, +astroid/brain/__pycache__/brain_gi.cpython-38.pyc,, +astroid/brain/__pycache__/brain_hashlib.cpython-38.pyc,, +astroid/brain/__pycache__/brain_http.cpython-38.pyc,, +astroid/brain/__pycache__/brain_io.cpython-38.pyc,, +astroid/brain/__pycache__/brain_mechanize.cpython-38.pyc,, +astroid/brain/__pycache__/brain_multiprocessing.cpython-38.pyc,, +astroid/brain/__pycache__/brain_namedtuple_enum.cpython-38.pyc,, +astroid/brain/__pycache__/brain_nose.cpython-38.pyc,, +astroid/brain/__pycache__/brain_numpy_core_fromnumeric.cpython-38.pyc,, +astroid/brain/__pycache__/brain_numpy_core_function_base.cpython-38.pyc,, +astroid/brain/__pycache__/brain_numpy_core_multiarray.cpython-38.pyc,, +astroid/brain/__pycache__/brain_numpy_core_numeric.cpython-38.pyc,, +astroid/brain/__pycache__/brain_numpy_core_numerictypes.cpython-38.pyc,, +astroid/brain/__pycache__/brain_numpy_core_umath.cpython-38.pyc,, +astroid/brain/__pycache__/brain_numpy_ndarray.cpython-38.pyc,, +astroid/brain/__pycache__/brain_numpy_random_mtrand.cpython-38.pyc,, +astroid/brain/__pycache__/brain_numpy_utils.cpython-38.pyc,, +astroid/brain/__pycache__/brain_pkg_resources.cpython-38.pyc,, +astroid/brain/__pycache__/brain_pytest.cpython-38.pyc,, +astroid/brain/__pycache__/brain_qt.cpython-38.pyc,, +astroid/brain/__pycache__/brain_random.cpython-38.pyc,, +astroid/brain/__pycache__/brain_re.cpython-38.pyc,, +astroid/brain/__pycache__/brain_responses.cpython-38.pyc,, +astroid/brain/__pycache__/brain_scipy_signal.cpython-38.pyc,, +astroid/brain/__pycache__/brain_six.cpython-38.pyc,, +astroid/brain/__pycache__/brain_ssl.cpython-38.pyc,, +astroid/brain/__pycache__/brain_subprocess.cpython-38.pyc,, +astroid/brain/__pycache__/brain_threading.cpython-38.pyc,, +astroid/brain/__pycache__/brain_typing.cpython-38.pyc,, +astroid/brain/__pycache__/brain_uuid.cpython-38.pyc,, +astroid/brain/brain_argparse.py,sha256=5XqcThekktCIWRlWAMs-R47wkbsOUSnQlsEbLEnRpsY,1041 +astroid/brain/brain_attrs.py,sha256=k8zJqIXsIbQrncthrzyB5NtdPTktgVi9wG7nyl8xMzs,2208 +astroid/brain/brain_boto3.py,sha256=nE8Cw_S_Gp___IszRDRkhEGS6WGrKcF_gTOs3GVxH9k,862 +astroid/brain/brain_builtin_inference.py,sha256=F6_09yM2mmu_u3ad_f9Uumc-q0lDA2CY8v9-PYtUMmA,28793 +astroid/brain/brain_collections.py,sha256=Uqi4VmpLDl0ndQa9x-5tIXRtVdtI6TlwR9KNHmOqLyw,2724 +astroid/brain/brain_crypt.py,sha256=gA7Q4GVuAM4viuTGWM6SNTPQXv5Gr_mFapyKMTRcsJ0,875 +astroid/brain/brain_curses.py,sha256=tDnlCP1bEvleqCMz856yua9mM5um1p_JendFhT4rBFk,3303 +astroid/brain/brain_dataclasses.py,sha256=5WndOYSY0oi2v-Od6KdPte-FKt00LoNRH2riSB4S1os,1647 +astroid/brain/brain_dateutil.py,sha256=GwDrgbaUkKbp3ImLAvuUtPuDBeK3lPh0bNI_Wphm2_8,801 +astroid/brain/brain_fstrings.py,sha256=Jaf-G-wkLecwG4jfjjfR8MDKzgAjjn2mgrrWZQLOAd8,2126 +astroid/brain/brain_functools.py,sha256=Nljy7o2vu16S2amFot4wdTI0U76Yafq-ujVPHOdGgCE,5481 +astroid/brain/brain_gi.py,sha256=oraXhBWyCCxmPEAEvRboeTIho0--ORObvckni00_1wY,7554 +astroid/brain/brain_hashlib.py,sha256=W2cS6-rixdBcre6lPWIuSOqPLCcVji5JBlImdPbV6Qo,2292 +astroid/brain/brain_http.py,sha256=a_8eIACvZetnR2xI4ZARVMuB1WSjyyqM3rCl2DVltMo,10524 +astroid/brain/brain_io.py,sha256=wEY3vvTeV23tYxFn8HHQeyjhb7-jqzwgwNI-kl2Yu-c,1476 +astroid/brain/brain_mechanize.py,sha256=boZxoCxPGSpHC_RccO5U026hyXyhunXR55M9WM1lYTo,895 +astroid/brain/brain_multiprocessing.py,sha256=9OswtRuVBj-KemJ0yp4prz8mrkJDzOzN6n13u2MgmwM,3096 +astroid/brain/brain_namedtuple_enum.py,sha256=eZ8IaHPLLHBakywJ3q4Ogtd2Tk0gtNcAgYAloMAKTjM,15966 +astroid/brain/brain_nose.py,sha256=PSPmme611h7fC8MKuVXbiGw0HZhdmIuoxM6yieZ1OOc,2217 +astroid/brain/brain_numpy_core_fromnumeric.py,sha256=NxiHbcMyQQ3Gpx4KEA5eAmGrpjlPN5rfXjRLjHPOVkw,621 +astroid/brain/brain_numpy_core_function_base.py,sha256=7i6Kge_PviqoWXhbeRgB581xwLo0dxvO54_5UB3ppig,1168 +astroid/brain/brain_numpy_core_multiarray.py,sha256=vr-nBt3EF_z-UmHmcAlJhGorgDXPpQaYX8g4TWu_2Vw,4015 +astroid/brain/brain_numpy_core_numeric.py,sha256=M0RXOym2YC-DaZfRqGNIpr6s-Oh1KHn1okGr3PtPl0k,1384 +astroid/brain/brain_numpy_core_numerictypes.py,sha256=ltlZyQprEbVdUJpC_ERGyXyJsYwzFHx7zGYDLheIonA,8013 +astroid/brain/brain_numpy_core_umath.py,sha256=je6K3ILMen2mgkZseHCOjOgnFV6SaNgAihEi6irhbRk,4436 +astroid/brain/brain_numpy_ndarray.py,sha256=WZbRG7GgOWZ68sEr4JjZUnJHLfUc91xp2mucHeLMrnw,8463 +astroid/brain/brain_numpy_random_mtrand.py,sha256=WRy_CStllgZN2B3Dw2qxXbY4SBrCzu-cdaK230VQHaE,3278 +astroid/brain/brain_numpy_utils.py,sha256=ZYtCVmn1x6P7iwYFohdW3CCrvVT5BCNYe-7jKcz_25s,1875 +astroid/brain/brain_pkg_resources.py,sha256=S_5UED1Zg8ObEJumRdpYGnjxZzemh_G_NFj3p5NGPfc,2262 +astroid/brain/brain_pytest.py,sha256=RYp7StKnC22n4vFJkHH5h7DAFWtouIcg-ZCXQ8Oz3xk,2390 +astroid/brain/brain_qt.py,sha256=m2s4YXFrnOynWDf9omHOBLVqFriBdGJAmvarJCxiffM,2536 +astroid/brain/brain_random.py,sha256=2RZY-QEXMNWp7E6h0l0-ke-DtjKTOFlTdjiQZi3XdQc,2432 +astroid/brain/brain_re.py,sha256=le7VJHUAf80HyE_aQCh7_8FyDVK6JwNWA--c9RaMVQ8,1128 +astroid/brain/brain_responses.py,sha256=GIAGkwMEGEZ9bJHI9uBGs_UmUy3DRBtjI_j_8dLBVsc,1534 +astroid/brain/brain_scipy_signal.py,sha256=IKNnGXzEH5FKhPHM2hv57pNdo1emThWwZ0ksckdsOeE,2255 +astroid/brain/brain_six.py,sha256=79aws4au6ZQJuf-YhBT4R-9wuVOwg-u0T42aUhCswK0,6187 +astroid/brain/brain_ssl.py,sha256=9dMCTQ0JsX2gpdmBkYO7BByw3zN23oXRL8Svo2S1V28,3722 +astroid/brain/brain_subprocess.py,sha256=yp-HsE69uh7T7C1kZYciAdRMQ48aNT51lI032oDppDk,4688 +astroid/brain/brain_threading.py,sha256=dY8YwZ-zrB30_CQFYWh4AK3qUVssakgwLVwunQ6yYkU,837 +astroid/brain/brain_typing.py,sha256=iFw33beNCitCJjJNvccIY6SsFJcdKVDdl-56DxDioh0,2780 +astroid/brain/brain_uuid.py,sha256=YzfiOXavu515cS1prsKGfY4-12g_KWT0Yr4-5ti0v74,569 +astroid/builder.py,sha256=72JlHrXRF9sS5zh1mqXlRHGFg88Efc2rm9yRJKDIShA,16891 +astroid/context.py,sha256=MpwiEzWZ39a4WyqYgbSmePL2nZLqALt1gjrvi3TP35U,5169 +astroid/decorators.py,sha256=z_wTjsiMlu_ElHtYUUuxaB2F_s4G0Ks7bmtCZS3IQ_Q,4283 +astroid/exceptions.py,sha256=GLiZo9BdWILJShO-il8ra-tPZqaODMAX987F--LWv2w,6891 +astroid/helpers.py,sha256=3YoJeVLoS-66T_abDorUNHiP8m2R7-faNnpc5yPJhrc,9448 +astroid/inference.py,sha256=UiKKPRYqb5JINQIw1lSdBniHhXjCohCGAqycp2GkVI4,34686 +astroid/interpreter/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0 +astroid/interpreter/__pycache__/__init__.cpython-38.pyc,, +astroid/interpreter/__pycache__/dunder_lookup.cpython-38.pyc,, +astroid/interpreter/__pycache__/objectmodel.cpython-38.pyc,, +astroid/interpreter/_import/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0 +astroid/interpreter/_import/__pycache__/__init__.cpython-38.pyc,, +astroid/interpreter/_import/__pycache__/spec.cpython-38.pyc,, +astroid/interpreter/_import/__pycache__/util.cpython-38.pyc,, +astroid/interpreter/_import/spec.py,sha256=rZ9kX3I0xPo_pFAYWeOk2Xbi4ZIg_5bRsouZgxOXegA,11436 +astroid/interpreter/_import/util.py,sha256=inubUz6F3_kaMFaeleKUW6E6wCMIPrhU882zvwEZ02I,255 +astroid/interpreter/dunder_lookup.py,sha256=dP-AZU_aGPNt03b1ttrMglxzeU3NtgnG0MfpSLPH6sg,2155 +astroid/interpreter/objectmodel.py,sha256=bbPIaamrqrx7WHtG5YNs9dbrlDC00GrJPxPGoSTsnqg,25792 +astroid/manager.py,sha256=jmEm9uH00mPA2Y68s10xrPNbZaadv_2c-CWluB3SYoE,13701 +astroid/mixins.py,sha256=F2rv2Ow7AU3YT_2jitVJik95ZWRVK6hpf8BrkkspzUY,5571 +astroid/modutils.py,sha256=GBW5Z691eqf6VAnPsZzeQ0WYzrl-0GGTHkkZNb_9XRQ,23242 +astroid/node_classes.py,sha256=cBRkZ_u8ZoRkiXznYWq1L3I7cO-P9nGowNCy8T7Qpzk,142740 +astroid/nodes.py,sha256=WoyRe22GNVRc2TRHWOUlqdxCdOVD8GKOq9v1LpPhkr8,2978 +astroid/objects.py,sha256=caKeQPBtrrfqa1q844vkehXwpdMzvph5YJdJJdbD_m8,11035 +astroid/protocols.py,sha256=m1ZHvKvFQZSZp993na4f9s8V17kqNPRfH-XpQc8gJ7c,27396 +astroid/raw_building.py,sha256=PI70y2mPQ7JURDVyNZ6deeBFkvjNNaOQBkuo0VPbvQ4,17340 +astroid/rebuilder.py,sha256=xn82eWlxzGcDd2VACUnNNWCxArfV9HI8g67fUip0XFk,39411 +astroid/scoped_nodes.py,sha256=fUrmTyltaLCr9-i8RdC2e5dPA9O8w946Mvvft8mgXdM,98203 +astroid/test_utils.py,sha256=axGB3j6ZEaVspL1ZgPizKgWehNEREb58Z97U7mQ-GA8,2319 +astroid/transforms.py,sha256=1npwJWcQUSIjcpcWd1pc-dJhtHOyiboQHsETAIQd5co,3377 +astroid/util.py,sha256=jg5LnqbWSZTZP1KgpxGBuC6Lfwhn9Jb2T2TohXghmC0,4785 diff --git a/env/lib/python3.8/site-packages/astroid-2.4.2.dist-info/WHEEL b/env/lib/python3.8/site-packages/astroid-2.4.2.dist-info/WHEEL new file mode 100644 index 000000000..b552003ff --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid-2.4.2.dist-info/WHEEL @@ -0,0 +1,5 @@ +Wheel-Version: 1.0 +Generator: bdist_wheel (0.34.2) +Root-Is-Purelib: true +Tag: py3-none-any + diff --git a/env/lib/python3.8/site-packages/astroid-2.4.2.dist-info/top_level.txt b/env/lib/python3.8/site-packages/astroid-2.4.2.dist-info/top_level.txt new file mode 100644 index 000000000..450d4fe9e --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid-2.4.2.dist-info/top_level.txt @@ -0,0 +1 @@ +astroid diff --git a/env/lib/python3.8/site-packages/astroid/__init__.py b/env/lib/python3.8/site-packages/astroid/__init__.py new file mode 100644 index 000000000..e869274e4 --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/__init__.py @@ -0,0 +1,168 @@ +# Copyright (c) 2006-2013, 2015 LOGILAB S.A. (Paris, FRANCE) +# Copyright (c) 2014 Google, Inc. +# Copyright (c) 2014 Eevee (Alex Munroe) +# Copyright (c) 2015-2016, 2018 Claudiu Popa +# Copyright (c) 2015-2016 Ceridwen +# Copyright (c) 2016 Derek Gustafson +# Copyright (c) 2016 Moises Lopez +# Copyright (c) 2018 Bryce Guinta +# Copyright (c) 2019 Nick Drozd + +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER + +"""Python Abstract Syntax Tree New Generation + +The aim of this module is to provide a common base representation of +python source code for projects such as pychecker, pyreverse, +pylint... Well, actually the development of this library is essentially +governed by pylint's needs. + +It extends class defined in the python's _ast module with some +additional methods and attributes. Instance attributes are added by a +builder object, which can either generate extended ast (let's call +them astroid ;) by visiting an existent ast tree or by inspecting living +object. Methods are added by monkey patching ast classes. + +Main modules are: + +* nodes and scoped_nodes for more information about methods and + attributes added to different node classes + +* the manager contains a high level object to get astroid trees from + source files and living objects. It maintains a cache of previously + constructed tree for quick access + +* builder contains the class responsible to build astroid trees +""" + +import enum +import itertools +import os +import sys + +import wrapt + + +_Context = enum.Enum("Context", "Load Store Del") +Load = _Context.Load +Store = _Context.Store +Del = _Context.Del +del _Context + + +# pylint: disable=wrong-import-order,wrong-import-position +from .__pkginfo__ import version as __version__ + +# WARNING: internal imports order matters ! + +# pylint: disable=redefined-builtin + +# make all exception classes accessible from astroid package +from astroid.exceptions import * + +# make all node classes accessible from astroid package +from astroid.nodes import * + +# trigger extra monkey-patching +from astroid import inference + +# more stuff available +from astroid import raw_building +from astroid.bases import BaseInstance, Instance, BoundMethod, UnboundMethod +from astroid.node_classes import are_exclusive, unpack_infer +from astroid.scoped_nodes import builtin_lookup +from astroid.builder import parse, extract_node +from astroid.util import Uninferable + +# make a manager instance (borg) accessible from astroid package +from astroid.manager import AstroidManager + +MANAGER = AstroidManager() +del AstroidManager + +# transform utilities (filters and decorator) + + +# pylint: disable=dangerous-default-value +@wrapt.decorator +def _inference_tip_cached(func, instance, args, kwargs, _cache={}): + """Cache decorator used for inference tips""" + node = args[0] + try: + return iter(_cache[func, node]) + except KeyError: + result = func(*args, **kwargs) + # Need to keep an iterator around + original, copy = itertools.tee(result) + _cache[func, node] = list(copy) + return original + + +# pylint: enable=dangerous-default-value + + +def inference_tip(infer_function, raise_on_overwrite=False): + """Given an instance specific inference function, return a function to be + given to MANAGER.register_transform to set this inference function. + + :param bool raise_on_overwrite: Raise an `InferenceOverwriteError` + if the inference tip will overwrite another. Used for debugging + + Typical usage + + .. sourcecode:: python + + MANAGER.register_transform(Call, inference_tip(infer_named_tuple), + predicate) + + .. Note:: + + Using an inference tip will override + any previously set inference tip for the given + node. Use a predicate in the transform to prevent + excess overwrites. + """ + + def transform(node, infer_function=infer_function): + if ( + raise_on_overwrite + and node._explicit_inference is not None + and node._explicit_inference is not infer_function + ): + raise InferenceOverwriteError( + "Inference already set to {existing_inference}. " + "Trying to overwrite with {new_inference} for {node}".format( + existing_inference=infer_function, + new_inference=node._explicit_inference, + node=node, + ) + ) + # pylint: disable=no-value-for-parameter + node._explicit_inference = _inference_tip_cached(infer_function) + return node + + return transform + + +def register_module_extender(manager, module_name, get_extension_mod): + def transform(node): + extension_module = get_extension_mod() + for name, objs in extension_module.locals.items(): + node.locals[name] = objs + for obj in objs: + if obj.parent is extension_module: + obj.parent = node + + manager.register_transform(Module, transform, lambda n: n.name == module_name) + + +# load brain plugins +BRAIN_MODULES_DIR = os.path.join(os.path.dirname(__file__), "brain") +if BRAIN_MODULES_DIR not in sys.path: + # add it to the end of the list so user path take precedence + sys.path.append(BRAIN_MODULES_DIR) +# load modules in this directory +for module in os.listdir(BRAIN_MODULES_DIR): + if module.endswith(".py"): + __import__(module[:-3]) diff --git a/env/lib/python3.8/site-packages/astroid/__pkginfo__.py b/env/lib/python3.8/site-packages/astroid/__pkginfo__.py new file mode 100644 index 000000000..fd8e1323c --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/__pkginfo__.py @@ -0,0 +1,56 @@ +# -*- coding: utf-8 -*- +# Copyright (c) 2006-2014 LOGILAB S.A. (Paris, FRANCE) +# Copyright (c) 2014-2019 Claudiu Popa +# Copyright (c) 2014 Google, Inc. +# Copyright (c) 2015-2017 Ceridwen +# Copyright (c) 2015 Florian Bruhin +# Copyright (c) 2015 Radosław Ganczarek +# Copyright (c) 2016 Moises Lopez +# Copyright (c) 2017 Hugo +# Copyright (c) 2017 Łukasz Rogalski +# Copyright (c) 2017 Calen Pennington +# Copyright (c) 2018 Ville Skyttä +# Copyright (c) 2018 Ashley Whetter +# Copyright (c) 2018 Bryce Guinta +# Copyright (c) 2019 Uilian Ries +# Copyright (c) 2019 Thomas Hisch +# Copyright (c) 2020 Michael + +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER + +"""astroid packaging information""" + +version = "2.4.2" +numversion = tuple(int(elem) for elem in version.split(".") if elem.isdigit()) + +extras_require = {} +install_requires = [ + "lazy_object_proxy==1.4.*", + "six~=1.12", + "wrapt~=1.11", + 'typed-ast>=1.4.0,<1.5;implementation_name== "cpython" and python_version<"3.8"', +] + +# pylint: disable=redefined-builtin; why license is a builtin anyway? +license = "LGPL" + +author = "Python Code Quality Authority" +author_email = "code-quality@python.org" +mailinglist = "mailto://%s" % author_email +web = "https://github.com/PyCQA/astroid" + +description = "An abstract syntax tree for Python with inference support." + +classifiers = [ + "Topic :: Software Development :: Libraries :: Python Modules", + "Topic :: Software Development :: Quality Assurance", + "Programming Language :: Python", + "Programming Language :: Python :: 3", + "Programming Language :: Python :: 3.5", + "Programming Language :: Python :: 3.6", + "Programming Language :: Python :: 3.7", + "Programming Language :: Python :: 3.8", + "Programming Language :: Python :: Implementation :: CPython", + "Programming Language :: Python :: Implementation :: PyPy", +] diff --git a/env/lib/python3.8/site-packages/astroid/_ast.py b/env/lib/python3.8/site-packages/astroid/_ast.py new file mode 100644 index 000000000..34b74c5f2 --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/_ast.py @@ -0,0 +1,131 @@ +import ast +from collections import namedtuple +from functools import partial +from typing import Optional +import sys + +import astroid + +_ast_py3 = None +try: + import typed_ast.ast3 as _ast_py3 +except ImportError: + pass + + +PY38 = sys.version_info[:2] >= (3, 8) +if PY38: + # On Python 3.8, typed_ast was merged back into `ast` + _ast_py3 = ast + + +FunctionType = namedtuple("FunctionType", ["argtypes", "returns"]) + + +class ParserModule( + namedtuple( + "ParserModule", + [ + "module", + "unary_op_classes", + "cmp_op_classes", + "bool_op_classes", + "bin_op_classes", + "context_classes", + ], + ) +): + def parse(self, string: str, type_comments=True): + if self.module is _ast_py3: + if PY38: + parse_func = partial(self.module.parse, type_comments=type_comments) + else: + parse_func = partial( + self.module.parse, feature_version=sys.version_info.minor + ) + else: + parse_func = self.module.parse + return parse_func(string) + + +def parse_function_type_comment(type_comment: str) -> Optional[FunctionType]: + """Given a correct type comment, obtain a FunctionType object""" + if _ast_py3 is None: + return None + + func_type = _ast_py3.parse(type_comment, "", "func_type") + return FunctionType(argtypes=func_type.argtypes, returns=func_type.returns) + + +def get_parser_module(type_comments=True) -> ParserModule: + if not type_comments: + parser_module = ast + else: + parser_module = _ast_py3 + parser_module = parser_module or ast + + unary_op_classes = _unary_operators_from_module(parser_module) + cmp_op_classes = _compare_operators_from_module(parser_module) + bool_op_classes = _bool_operators_from_module(parser_module) + bin_op_classes = _binary_operators_from_module(parser_module) + context_classes = _contexts_from_module(parser_module) + + return ParserModule( + parser_module, + unary_op_classes, + cmp_op_classes, + bool_op_classes, + bin_op_classes, + context_classes, + ) + + +def _unary_operators_from_module(module): + return {module.UAdd: "+", module.USub: "-", module.Not: "not", module.Invert: "~"} + + +def _binary_operators_from_module(module): + binary_operators = { + module.Add: "+", + module.BitAnd: "&", + module.BitOr: "|", + module.BitXor: "^", + module.Div: "/", + module.FloorDiv: "//", + module.MatMult: "@", + module.Mod: "%", + module.Mult: "*", + module.Pow: "**", + module.Sub: "-", + module.LShift: "<<", + module.RShift: ">>", + } + return binary_operators + + +def _bool_operators_from_module(module): + return {module.And: "and", module.Or: "or"} + + +def _compare_operators_from_module(module): + return { + module.Eq: "==", + module.Gt: ">", + module.GtE: ">=", + module.In: "in", + module.Is: "is", + module.IsNot: "is not", + module.Lt: "<", + module.LtE: "<=", + module.NotEq: "!=", + module.NotIn: "not in", + } + + +def _contexts_from_module(module): + return { + module.Load: astroid.Load, + module.Store: astroid.Store, + module.Del: astroid.Del, + module.Param: astroid.Store, + } diff --git a/env/lib/python3.8/site-packages/astroid/arguments.py b/env/lib/python3.8/site-packages/astroid/arguments.py new file mode 100644 index 000000000..5f4d90924 --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/arguments.py @@ -0,0 +1,300 @@ +# Copyright (c) 2015-2016, 2018-2020 Claudiu Popa +# Copyright (c) 2015-2016 Ceridwen +# Copyright (c) 2018 Bryce Guinta +# Copyright (c) 2018 Nick Drozd +# Copyright (c) 2018 Anthony Sottile + +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER + + +from astroid import bases +from astroid import context as contextmod +from astroid import exceptions +from astroid import nodes +from astroid import util + + +class CallSite: + """Class for understanding arguments passed into a call site + + It needs a call context, which contains the arguments and the + keyword arguments that were passed into a given call site. + In order to infer what an argument represents, call :meth:`infer_argument` + with the corresponding function node and the argument name. + + :param callcontext: + An instance of :class:`astroid.context.CallContext`, that holds + the arguments for the call site. + :param argument_context_map: + Additional contexts per node, passed in from :attr:`astroid.context.Context.extra_context` + :param context: + An instance of :class:`astroid.context.Context`. + """ + + def __init__(self, callcontext, argument_context_map=None, context=None): + if argument_context_map is None: + argument_context_map = {} + self.argument_context_map = argument_context_map + args = callcontext.args + keywords = callcontext.keywords + self.duplicated_keywords = set() + self._unpacked_args = self._unpack_args(args, context=context) + self._unpacked_kwargs = self._unpack_keywords(keywords, context=context) + + self.positional_arguments = [ + arg for arg in self._unpacked_args if arg is not util.Uninferable + ] + self.keyword_arguments = { + key: value + for key, value in self._unpacked_kwargs.items() + if value is not util.Uninferable + } + + @classmethod + def from_call(cls, call_node, context=None): + """Get a CallSite object from the given Call node. + + :param context: + An instance of :class:`astroid.context.Context` that will be used + to force a single inference path. + """ + + # Determine the callcontext from the given `context` object if any. + context = context or contextmod.InferenceContext() + callcontext = contextmod.CallContext(call_node.args, call_node.keywords) + return cls(callcontext, context=context) + + def has_invalid_arguments(self): + """Check if in the current CallSite were passed *invalid* arguments + + This can mean multiple things. For instance, if an unpacking + of an invalid object was passed, then this method will return True. + Other cases can be when the arguments can't be inferred by astroid, + for example, by passing objects which aren't known statically. + """ + return len(self.positional_arguments) != len(self._unpacked_args) + + def has_invalid_keywords(self): + """Check if in the current CallSite were passed *invalid* keyword arguments + + For instance, unpacking a dictionary with integer keys is invalid + (**{1:2}), because the keys must be strings, which will make this + method to return True. Other cases where this might return True if + objects which can't be inferred were passed. + """ + return len(self.keyword_arguments) != len(self._unpacked_kwargs) + + def _unpack_keywords(self, keywords, context=None): + values = {} + context = context or contextmod.InferenceContext() + context.extra_context = self.argument_context_map + for name, value in keywords: + if name is None: + # Then it's an unpacking operation (**) + try: + inferred = next(value.infer(context=context)) + except exceptions.InferenceError: + values[name] = util.Uninferable + continue + + if not isinstance(inferred, nodes.Dict): + # Not something we can work with. + values[name] = util.Uninferable + continue + + for dict_key, dict_value in inferred.items: + try: + dict_key = next(dict_key.infer(context=context)) + except exceptions.InferenceError: + values[name] = util.Uninferable + continue + if not isinstance(dict_key, nodes.Const): + values[name] = util.Uninferable + continue + if not isinstance(dict_key.value, str): + values[name] = util.Uninferable + continue + if dict_key.value in values: + # The name is already in the dictionary + values[dict_key.value] = util.Uninferable + self.duplicated_keywords.add(dict_key.value) + continue + values[dict_key.value] = dict_value + else: + values[name] = value + return values + + def _unpack_args(self, args, context=None): + values = [] + context = context or contextmod.InferenceContext() + context.extra_context = self.argument_context_map + for arg in args: + if isinstance(arg, nodes.Starred): + try: + inferred = next(arg.value.infer(context=context)) + except exceptions.InferenceError: + values.append(util.Uninferable) + continue + + if inferred is util.Uninferable: + values.append(util.Uninferable) + continue + if not hasattr(inferred, "elts"): + values.append(util.Uninferable) + continue + values.extend(inferred.elts) + else: + values.append(arg) + return values + + def infer_argument(self, funcnode, name, context): + """infer a function argument value according to the call context + + Arguments: + funcnode: The function being called. + name: The name of the argument whose value is being inferred. + context: Inference context object + """ + if name in self.duplicated_keywords: + raise exceptions.InferenceError( + "The arguments passed to {func!r} " " have duplicate keywords.", + call_site=self, + func=funcnode, + arg=name, + context=context, + ) + + # Look into the keywords first, maybe it's already there. + try: + return self.keyword_arguments[name].infer(context) + except KeyError: + pass + + # Too many arguments given and no variable arguments. + if len(self.positional_arguments) > len(funcnode.args.args): + if not funcnode.args.vararg: + raise exceptions.InferenceError( + "Too many positional arguments " + "passed to {func!r} that does " + "not have *args.", + call_site=self, + func=funcnode, + arg=name, + context=context, + ) + + positional = self.positional_arguments[: len(funcnode.args.args)] + vararg = self.positional_arguments[len(funcnode.args.args) :] + argindex = funcnode.args.find_argname(name)[0] + kwonlyargs = {arg.name for arg in funcnode.args.kwonlyargs} + kwargs = { + key: value + for key, value in self.keyword_arguments.items() + if key not in kwonlyargs + } + # If there are too few positionals compared to + # what the function expects to receive, check to see + # if the missing positional arguments were passed + # as keyword arguments and if so, place them into the + # positional args list. + if len(positional) < len(funcnode.args.args): + for func_arg in funcnode.args.args: + if func_arg.name in kwargs: + arg = kwargs.pop(func_arg.name) + positional.append(arg) + + if argindex is not None: + # 2. first argument of instance/class method + if argindex == 0 and funcnode.type in ("method", "classmethod"): + if context.boundnode is not None: + boundnode = context.boundnode + else: + # XXX can do better ? + boundnode = funcnode.parent.frame() + + if isinstance(boundnode, nodes.ClassDef): + # Verify that we're accessing a method + # of the metaclass through a class, as in + # `cls.metaclass_method`. In this case, the + # first argument is always the class. + method_scope = funcnode.parent.scope() + if method_scope is boundnode.metaclass(): + return iter((boundnode,)) + + if funcnode.type == "method": + if not isinstance(boundnode, bases.Instance): + boundnode = boundnode.instantiate_class() + return iter((boundnode,)) + if funcnode.type == "classmethod": + return iter((boundnode,)) + # if we have a method, extract one position + # from the index, so we'll take in account + # the extra parameter represented by `self` or `cls` + if funcnode.type in ("method", "classmethod"): + argindex -= 1 + # 2. search arg index + try: + return self.positional_arguments[argindex].infer(context) + except IndexError: + pass + + if funcnode.args.kwarg == name: + # It wants all the keywords that were passed into + # the call site. + if self.has_invalid_keywords(): + raise exceptions.InferenceError( + "Inference failed to find values for all keyword arguments " + "to {func!r}: {unpacked_kwargs!r} doesn't correspond to " + "{keyword_arguments!r}.", + keyword_arguments=self.keyword_arguments, + unpacked_kwargs=self._unpacked_kwargs, + call_site=self, + func=funcnode, + arg=name, + context=context, + ) + kwarg = nodes.Dict( + lineno=funcnode.args.lineno, + col_offset=funcnode.args.col_offset, + parent=funcnode.args, + ) + kwarg.postinit( + [(nodes.const_factory(key), value) for key, value in kwargs.items()] + ) + return iter((kwarg,)) + if funcnode.args.vararg == name: + # It wants all the args that were passed into + # the call site. + if self.has_invalid_arguments(): + raise exceptions.InferenceError( + "Inference failed to find values for all positional " + "arguments to {func!r}: {unpacked_args!r} doesn't " + "correspond to {positional_arguments!r}.", + positional_arguments=self.positional_arguments, + unpacked_args=self._unpacked_args, + call_site=self, + func=funcnode, + arg=name, + context=context, + ) + args = nodes.Tuple( + lineno=funcnode.args.lineno, + col_offset=funcnode.args.col_offset, + parent=funcnode.args, + ) + args.postinit(vararg) + return iter((args,)) + + # Check if it's a default parameter. + try: + return funcnode.args.default_value(name).infer(context) + except exceptions.NoDefault: + pass + raise exceptions.InferenceError( + "No value found for argument {name} to " "{func!r}", + call_site=self, + func=funcnode, + arg=name, + context=context, + ) diff --git a/env/lib/python3.8/site-packages/astroid/as_string.py b/env/lib/python3.8/site-packages/astroid/as_string.py new file mode 100644 index 000000000..653411f4d --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/as_string.py @@ -0,0 +1,631 @@ +# -*- coding: utf-8 -*- +# Copyright (c) 2009-2011, 2013-2014 LOGILAB S.A. (Paris, FRANCE) +# Copyright (c) 2010 Daniel Harding +# Copyright (c) 2013-2016, 2018-2020 Claudiu Popa +# Copyright (c) 2013-2014 Google, Inc. +# Copyright (c) 2015-2016 Ceridwen +# Copyright (c) 2016 Jared Garst +# Copyright (c) 2016 Jakub Wilk +# Copyright (c) 2017, 2019 Łukasz Rogalski +# Copyright (c) 2017 rr- +# Copyright (c) 2018 Serhiy Storchaka +# Copyright (c) 2018 Ville Skyttä +# Copyright (c) 2018 brendanator +# Copyright (c) 2018 Nick Drozd +# Copyright (c) 2019 Alex Hall +# Copyright (c) 2019 Hugo van Kemenade + +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER + +"""This module renders Astroid nodes as string: + +* :func:`to_code` function return equivalent (hopefully valid) python string + +* :func:`dump` function return an internal representation of nodes found + in the tree, useful for debugging or understanding the tree structure +""" + +# pylint: disable=unused-argument + +DOC_NEWLINE = "\0" + + +class AsStringVisitor: + """Visitor to render an Astroid node as a valid python code string""" + + def __init__(self, indent): + self.indent = indent + + def __call__(self, node): + """Makes this visitor behave as a simple function""" + return node.accept(self).replace(DOC_NEWLINE, "\n") + + def _docs_dedent(self, doc): + """Stop newlines in docs being indented by self._stmt_list""" + return '\n%s"""%s"""' % (self.indent, doc.replace("\n", DOC_NEWLINE)) + + def _stmt_list(self, stmts, indent=True): + """return a list of nodes to string""" + stmts = "\n".join(nstr for nstr in [n.accept(self) for n in stmts] if nstr) + if indent: + return self.indent + stmts.replace("\n", "\n" + self.indent) + + return stmts + + def _precedence_parens(self, node, child, is_left=True): + """Wrap child in parens only if required to keep same semantics""" + if self._should_wrap(node, child, is_left): + return "(%s)" % child.accept(self) + + return child.accept(self) + + def _should_wrap(self, node, child, is_left): + """Wrap child if: + - it has lower precedence + - same precedence with position opposite to associativity direction + """ + node_precedence = node.op_precedence() + child_precedence = child.op_precedence() + + if node_precedence > child_precedence: + # 3 * (4 + 5) + return True + + if ( + node_precedence == child_precedence + and is_left != node.op_left_associative() + ): + # 3 - (4 - 5) + # (2**3)**4 + return True + + return False + + ## visit_ methods ########################################### + + def visit_await(self, node): + return "await %s" % node.value.accept(self) + + def visit_asyncwith(self, node): + return "async %s" % self.visit_with(node) + + def visit_asyncfor(self, node): + return "async %s" % self.visit_for(node) + + def visit_arguments(self, node): + """return an astroid.Function node as string""" + return node.format_args() + + def visit_assignattr(self, node): + """return an astroid.AssAttr node as string""" + return self.visit_attribute(node) + + def visit_assert(self, node): + """return an astroid.Assert node as string""" + if node.fail: + return "assert %s, %s" % (node.test.accept(self), node.fail.accept(self)) + return "assert %s" % node.test.accept(self) + + def visit_assignname(self, node): + """return an astroid.AssName node as string""" + return node.name + + def visit_assign(self, node): + """return an astroid.Assign node as string""" + lhs = " = ".join(n.accept(self) for n in node.targets) + return "%s = %s" % (lhs, node.value.accept(self)) + + def visit_augassign(self, node): + """return an astroid.AugAssign node as string""" + return "%s %s %s" % (node.target.accept(self), node.op, node.value.accept(self)) + + def visit_annassign(self, node): + """Return an astroid.AugAssign node as string""" + + target = node.target.accept(self) + annotation = node.annotation.accept(self) + if node.value is None: + return "%s: %s" % (target, annotation) + return "%s: %s = %s" % (target, annotation, node.value.accept(self)) + + def visit_repr(self, node): + """return an astroid.Repr node as string""" + return "`%s`" % node.value.accept(self) + + def visit_binop(self, node): + """return an astroid.BinOp node as string""" + left = self._precedence_parens(node, node.left) + right = self._precedence_parens(node, node.right, is_left=False) + if node.op == "**": + return "%s%s%s" % (left, node.op, right) + + return "%s %s %s" % (left, node.op, right) + + def visit_boolop(self, node): + """return an astroid.BoolOp node as string""" + values = ["%s" % self._precedence_parens(node, n) for n in node.values] + return (" %s " % node.op).join(values) + + def visit_break(self, node): + """return an astroid.Break node as string""" + return "break" + + def visit_call(self, node): + """return an astroid.Call node as string""" + expr_str = self._precedence_parens(node, node.func) + args = [arg.accept(self) for arg in node.args] + if node.keywords: + keywords = [kwarg.accept(self) for kwarg in node.keywords] + else: + keywords = [] + + args.extend(keywords) + return "%s(%s)" % (expr_str, ", ".join(args)) + + def visit_classdef(self, node): + """return an astroid.ClassDef node as string""" + decorate = node.decorators.accept(self) if node.decorators else "" + args = [n.accept(self) for n in node.bases] + if node._metaclass and not node.has_metaclass_hack(): + args.append("metaclass=" + node._metaclass.accept(self)) + args += [n.accept(self) for n in node.keywords] + args = "(%s)" % ", ".join(args) if args else "" + docs = self._docs_dedent(node.doc) if node.doc else "" + return "\n\n%sclass %s%s:%s\n%s\n" % ( + decorate, + node.name, + args, + docs, + self._stmt_list(node.body), + ) + + def visit_compare(self, node): + """return an astroid.Compare node as string""" + rhs_str = " ".join( + [ + "%s %s" % (op, self._precedence_parens(node, expr, is_left=False)) + for op, expr in node.ops + ] + ) + return "%s %s" % (self._precedence_parens(node, node.left), rhs_str) + + def visit_comprehension(self, node): + """return an astroid.Comprehension node as string""" + ifs = "".join(" if %s" % n.accept(self) for n in node.ifs) + generated = "for %s in %s%s" % ( + node.target.accept(self), + node.iter.accept(self), + ifs, + ) + return "%s%s" % ("async " if node.is_async else "", generated) + + def visit_const(self, node): + """return an astroid.Const node as string""" + if node.value is Ellipsis: + return "..." + return repr(node.value) + + def visit_continue(self, node): + """return an astroid.Continue node as string""" + return "continue" + + def visit_delete(self, node): # XXX check if correct + """return an astroid.Delete node as string""" + return "del %s" % ", ".join(child.accept(self) for child in node.targets) + + def visit_delattr(self, node): + """return an astroid.DelAttr node as string""" + return self.visit_attribute(node) + + def visit_delname(self, node): + """return an astroid.DelName node as string""" + return node.name + + def visit_decorators(self, node): + """return an astroid.Decorators node as string""" + return "@%s\n" % "\n@".join(item.accept(self) for item in node.nodes) + + def visit_dict(self, node): + """return an astroid.Dict node as string""" + return "{%s}" % ", ".join(self._visit_dict(node)) + + def _visit_dict(self, node): + for key, value in node.items: + key = key.accept(self) + value = value.accept(self) + if key == "**": + # It can only be a DictUnpack node. + yield key + value + else: + yield "%s: %s" % (key, value) + + def visit_dictunpack(self, node): + return "**" + + def visit_dictcomp(self, node): + """return an astroid.DictComp node as string""" + return "{%s: %s %s}" % ( + node.key.accept(self), + node.value.accept(self), + " ".join(n.accept(self) for n in node.generators), + ) + + def visit_expr(self, node): + """return an astroid.Discard node as string""" + return node.value.accept(self) + + def visit_emptynode(self, node): + """dummy method for visiting an Empty node""" + return "" + + def visit_excepthandler(self, node): + if node.type: + if node.name: + excs = "except %s as %s" % ( + node.type.accept(self), + node.name.accept(self), + ) + else: + excs = "except %s" % node.type.accept(self) + else: + excs = "except" + return "%s:\n%s" % (excs, self._stmt_list(node.body)) + + def visit_ellipsis(self, node): + """return an astroid.Ellipsis node as string""" + return "..." + + def visit_empty(self, node): + """return an Empty node as string""" + return "" + + def visit_exec(self, node): + """return an astroid.Exec node as string""" + if node.locals: + return "exec %s in %s, %s" % ( + node.expr.accept(self), + node.locals.accept(self), + node.globals.accept(self), + ) + if node.globals: + return "exec %s in %s" % (node.expr.accept(self), node.globals.accept(self)) + return "exec %s" % node.expr.accept(self) + + def visit_extslice(self, node): + """return an astroid.ExtSlice node as string""" + return ", ".join(dim.accept(self) for dim in node.dims) + + def visit_for(self, node): + """return an astroid.For node as string""" + fors = "for %s in %s:\n%s" % ( + node.target.accept(self), + node.iter.accept(self), + self._stmt_list(node.body), + ) + if node.orelse: + fors = "%s\nelse:\n%s" % (fors, self._stmt_list(node.orelse)) + return fors + + def visit_importfrom(self, node): + """return an astroid.ImportFrom node as string""" + return "from %s import %s" % ( + "." * (node.level or 0) + node.modname, + _import_string(node.names), + ) + + def visit_joinedstr(self, node): + string = "".join( + # Use repr on the string literal parts + # to get proper escapes, e.g. \n, \\, \" + # But strip the quotes off the ends + # (they will always be one character: ' or ") + repr(value.value)[1:-1] + # Literal braces must be doubled to escape them + .replace("{", "{{").replace("}", "}}") + # Each value in values is either a string literal (Const) + # or a FormattedValue + if type(value).__name__ == "Const" else value.accept(self) + for value in node.values + ) + + # Try to find surrounding quotes that don't appear at all in the string. + # Because the formatted values inside {} can't contain backslash (\) + # using a triple quote is sometimes necessary + for quote in ["'", '"', '"""', "'''"]: + if quote not in string: + break + + return "f" + quote + string + quote + + def visit_formattedvalue(self, node): + result = node.value.accept(self) + if node.conversion and node.conversion >= 0: + # e.g. if node.conversion == 114: result += "!r" + result += "!" + chr(node.conversion) + if node.format_spec: + # The format spec is itself a JoinedString, i.e. an f-string + # We strip the f and quotes of the ends + result += ":" + node.format_spec.accept(self)[2:-1] + return "{%s}" % result + + def handle_functiondef(self, node, keyword): + """return a (possibly async) function definition node as string""" + decorate = node.decorators.accept(self) if node.decorators else "" + docs = self._docs_dedent(node.doc) if node.doc else "" + trailer = ":" + if node.returns: + return_annotation = " -> " + node.returns.as_string() + trailer = return_annotation + ":" + def_format = "\n%s%s %s(%s)%s%s\n%s" + return def_format % ( + decorate, + keyword, + node.name, + node.args.accept(self), + trailer, + docs, + self._stmt_list(node.body), + ) + + def visit_functiondef(self, node): + """return an astroid.FunctionDef node as string""" + return self.handle_functiondef(node, "def") + + def visit_asyncfunctiondef(self, node): + """return an astroid.AsyncFunction node as string""" + return self.handle_functiondef(node, "async def") + + def visit_generatorexp(self, node): + """return an astroid.GeneratorExp node as string""" + return "(%s %s)" % ( + node.elt.accept(self), + " ".join(n.accept(self) for n in node.generators), + ) + + def visit_attribute(self, node): + """return an astroid.Getattr node as string""" + left = self._precedence_parens(node, node.expr) + if left.isdigit(): + left = "(%s)" % left + return "%s.%s" % (left, node.attrname) + + def visit_global(self, node): + """return an astroid.Global node as string""" + return "global %s" % ", ".join(node.names) + + def visit_if(self, node): + """return an astroid.If node as string""" + ifs = ["if %s:\n%s" % (node.test.accept(self), self._stmt_list(node.body))] + if node.has_elif_block(): + ifs.append("el%s" % self._stmt_list(node.orelse, indent=False)) + elif node.orelse: + ifs.append("else:\n%s" % self._stmt_list(node.orelse)) + return "\n".join(ifs) + + def visit_ifexp(self, node): + """return an astroid.IfExp node as string""" + return "%s if %s else %s" % ( + self._precedence_parens(node, node.body, is_left=True), + self._precedence_parens(node, node.test, is_left=True), + self._precedence_parens(node, node.orelse, is_left=False), + ) + + def visit_import(self, node): + """return an astroid.Import node as string""" + return "import %s" % _import_string(node.names) + + def visit_keyword(self, node): + """return an astroid.Keyword node as string""" + if node.arg is None: + return "**%s" % node.value.accept(self) + return "%s=%s" % (node.arg, node.value.accept(self)) + + def visit_lambda(self, node): + """return an astroid.Lambda node as string""" + args = node.args.accept(self) + body = node.body.accept(self) + if args: + return "lambda %s: %s" % (args, body) + + return "lambda: %s" % body + + def visit_list(self, node): + """return an astroid.List node as string""" + return "[%s]" % ", ".join(child.accept(self) for child in node.elts) + + def visit_listcomp(self, node): + """return an astroid.ListComp node as string""" + return "[%s %s]" % ( + node.elt.accept(self), + " ".join(n.accept(self) for n in node.generators), + ) + + def visit_module(self, node): + """return an astroid.Module node as string""" + docs = '"""%s"""\n\n' % node.doc if node.doc else "" + return docs + "\n".join(n.accept(self) for n in node.body) + "\n\n" + + def visit_name(self, node): + """return an astroid.Name node as string""" + return node.name + + def visit_namedexpr(self, node): + """Return an assignment expression node as string""" + target = node.target.accept(self) + value = node.value.accept(self) + return "%s := %s" % (target, value) + + def visit_nonlocal(self, node): + """return an astroid.Nonlocal node as string""" + return "nonlocal %s" % ", ".join(node.names) + + def visit_pass(self, node): + """return an astroid.Pass node as string""" + return "pass" + + def visit_print(self, node): + """return an astroid.Print node as string""" + nodes = ", ".join(n.accept(self) for n in node.values) + if not node.nl: + nodes = "%s," % nodes + if node.dest: + return "print >> %s, %s" % (node.dest.accept(self), nodes) + return "print %s" % nodes + + def visit_raise(self, node): + """return an astroid.Raise node as string""" + if node.exc: + if node.cause: + return "raise %s from %s" % ( + node.exc.accept(self), + node.cause.accept(self), + ) + return "raise %s" % node.exc.accept(self) + return "raise" + + def visit_return(self, node): + """return an astroid.Return node as string""" + if node.is_tuple_return() and len(node.value.elts) > 1: + elts = [child.accept(self) for child in node.value.elts] + return "return %s" % ", ".join(elts) + + if node.value: + return "return %s" % node.value.accept(self) + + return "return" + + def visit_index(self, node): + """return an astroid.Index node as string""" + return node.value.accept(self) + + def visit_set(self, node): + """return an astroid.Set node as string""" + return "{%s}" % ", ".join(child.accept(self) for child in node.elts) + + def visit_setcomp(self, node): + """return an astroid.SetComp node as string""" + return "{%s %s}" % ( + node.elt.accept(self), + " ".join(n.accept(self) for n in node.generators), + ) + + def visit_slice(self, node): + """return an astroid.Slice node as string""" + lower = node.lower.accept(self) if node.lower else "" + upper = node.upper.accept(self) if node.upper else "" + step = node.step.accept(self) if node.step else "" + if step: + return "%s:%s:%s" % (lower, upper, step) + return "%s:%s" % (lower, upper) + + def visit_subscript(self, node): + """return an astroid.Subscript node as string""" + idx = node.slice + if idx.__class__.__name__.lower() == "index": + idx = idx.value + idxstr = idx.accept(self) + if idx.__class__.__name__.lower() == "tuple" and idx.elts: + # Remove parenthesis in tuple and extended slice. + # a[(::1, 1:)] is not valid syntax. + idxstr = idxstr[1:-1] + return "%s[%s]" % (self._precedence_parens(node, node.value), idxstr) + + def visit_tryexcept(self, node): + """return an astroid.TryExcept node as string""" + trys = ["try:\n%s" % self._stmt_list(node.body)] + for handler in node.handlers: + trys.append(handler.accept(self)) + if node.orelse: + trys.append("else:\n%s" % self._stmt_list(node.orelse)) + return "\n".join(trys) + + def visit_tryfinally(self, node): + """return an astroid.TryFinally node as string""" + return "try:\n%s\nfinally:\n%s" % ( + self._stmt_list(node.body), + self._stmt_list(node.finalbody), + ) + + def visit_tuple(self, node): + """return an astroid.Tuple node as string""" + if len(node.elts) == 1: + return "(%s, )" % node.elts[0].accept(self) + return "(%s)" % ", ".join(child.accept(self) for child in node.elts) + + def visit_unaryop(self, node): + """return an astroid.UnaryOp node as string""" + if node.op == "not": + operator = "not " + else: + operator = node.op + return "%s%s" % (operator, self._precedence_parens(node, node.operand)) + + def visit_while(self, node): + """return an astroid.While node as string""" + whiles = "while %s:\n%s" % (node.test.accept(self), self._stmt_list(node.body)) + if node.orelse: + whiles = "%s\nelse:\n%s" % (whiles, self._stmt_list(node.orelse)) + return whiles + + def visit_with(self, node): # 'with' without 'as' is possible + """return an astroid.With node as string""" + items = ", ".join( + ("%s" % expr.accept(self)) + (vars and " as %s" % (vars.accept(self)) or "") + for expr, vars in node.items + ) + return "with %s:\n%s" % (items, self._stmt_list(node.body)) + + def visit_yield(self, node): + """yield an ast.Yield node as string""" + yi_val = (" " + node.value.accept(self)) if node.value else "" + expr = "yield" + yi_val + if node.parent.is_statement: + return expr + + return "(%s)" % (expr,) + + def visit_yieldfrom(self, node): + """ Return an astroid.YieldFrom node as string. """ + yi_val = (" " + node.value.accept(self)) if node.value else "" + expr = "yield from" + yi_val + if node.parent.is_statement: + return expr + + return "(%s)" % (expr,) + + def visit_starred(self, node): + """return Starred node as string""" + return "*" + node.value.accept(self) + + # These aren't for real AST nodes, but for inference objects. + + def visit_frozenset(self, node): + return node.parent.accept(self) + + def visit_super(self, node): + return node.parent.accept(self) + + def visit_uninferable(self, node): + return str(node) + + def visit_property(self, node): + return node.function.accept(self) + + def visit_evaluatedobject(self, node): + return node.original.accept(self) + + +def _import_string(names): + """return a list of (name, asname) formatted as a string""" + _names = [] + for name, asname in names: + if asname is not None: + _names.append("%s as %s" % (name, asname)) + else: + _names.append(name) + return ", ".join(_names) + + +# This sets the default indent to 4 spaces. +to_code = AsStringVisitor(" ") diff --git a/env/lib/python3.8/site-packages/astroid/bases.py b/env/lib/python3.8/site-packages/astroid/bases.py new file mode 100644 index 000000000..9c743031d --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/bases.py @@ -0,0 +1,548 @@ +# -*- coding: utf-8 -*- +# Copyright (c) 2009-2011, 2013-2014 LOGILAB S.A. (Paris, FRANCE) +# Copyright (c) 2012 FELD Boris +# Copyright (c) 2014-2020 Claudiu Popa +# Copyright (c) 2014 Google, Inc. +# Copyright (c) 2014 Eevee (Alex Munroe) +# Copyright (c) 2015-2016 Ceridwen +# Copyright (c) 2015 Florian Bruhin +# Copyright (c) 2016-2017 Derek Gustafson +# Copyright (c) 2017 Calen Pennington +# Copyright (c) 2018-2019 hippo91 +# Copyright (c) 2018 Ville Skyttä +# Copyright (c) 2018 Bryce Guinta +# Copyright (c) 2018 Nick Drozd +# Copyright (c) 2018 Daniel Colascione +# Copyright (c) 2019 Hugo van Kemenade + +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER + +"""This module contains base classes and functions for the nodes and some +inference utils. +""" + +import builtins +import collections + +from astroid import context as contextmod +from astroid import exceptions +from astroid import util + +objectmodel = util.lazy_import("interpreter.objectmodel") +helpers = util.lazy_import("helpers") +BUILTINS = builtins.__name__ +manager = util.lazy_import("manager") +MANAGER = manager.AstroidManager() + +# TODO: check if needs special treatment +BUILTINS = "builtins" +BOOL_SPECIAL_METHOD = "__bool__" + +PROPERTIES = {BUILTINS + ".property", "abc.abstractproperty"} +# List of possible property names. We use this list in order +# to see if a method is a property or not. This should be +# pretty reliable and fast, the alternative being to check each +# decorator to see if its a real property-like descriptor, which +# can be too complicated. +# Also, these aren't qualified, because each project can +# define them, we shouldn't expect to know every possible +# property-like decorator! +POSSIBLE_PROPERTIES = { + "cached_property", + "cachedproperty", + "lazyproperty", + "lazy_property", + "reify", + "lazyattribute", + "lazy_attribute", + "LazyProperty", + "lazy", + "cache_readonly", +} + + +def _is_property(meth, context=None): + decoratornames = meth.decoratornames(context=context) + if PROPERTIES.intersection(decoratornames): + return True + stripped = { + name.split(".")[-1] for name in decoratornames if name is not util.Uninferable + } + if any(name in stripped for name in POSSIBLE_PROPERTIES): + return True + + # Lookup for subclasses of *property* + if not meth.decorators: + return False + for decorator in meth.decorators.nodes or (): + inferred = helpers.safe_infer(decorator, context=context) + if inferred is None or inferred is util.Uninferable: + continue + if inferred.__class__.__name__ == "ClassDef": + for base_class in inferred.bases: + if base_class.__class__.__name__ != "Name": + continue + module, _ = base_class.lookup(base_class.name) + if module.name == BUILTINS and base_class.name == "property": + return True + + return False + + +class Proxy: + """a simple proxy object + + Note: + + Subclasses of this object will need a custom __getattr__ + if new instance attributes are created. See the Const class + """ + + _proxied = None # proxied object may be set by class or by instance + + def __init__(self, proxied=None): + if proxied is not None: + self._proxied = proxied + + def __getattr__(self, name): + if name == "_proxied": + return getattr(self.__class__, "_proxied") + if name in self.__dict__: + return self.__dict__[name] + return getattr(self._proxied, name) + + def infer(self, context=None): + yield self + + +def _infer_stmts(stmts, context, frame=None): + """Return an iterator on statements inferred by each statement in *stmts*.""" + inferred = False + if context is not None: + name = context.lookupname + context = context.clone() + else: + name = None + context = contextmod.InferenceContext() + + for stmt in stmts: + if stmt is util.Uninferable: + yield stmt + inferred = True + continue + context.lookupname = stmt._infer_name(frame, name) + try: + for inferred in stmt.infer(context=context): + yield inferred + inferred = True + except exceptions.NameInferenceError: + continue + except exceptions.InferenceError: + yield util.Uninferable + inferred = True + if not inferred: + raise exceptions.InferenceError( + "Inference failed for all members of {stmts!r}.", + stmts=stmts, + frame=frame, + context=context, + ) + + +def _infer_method_result_truth(instance, method_name, context): + # Get the method from the instance and try to infer + # its return's truth value. + meth = next(instance.igetattr(method_name, context=context), None) + if meth and hasattr(meth, "infer_call_result"): + if not meth.callable(): + return util.Uninferable + try: + for value in meth.infer_call_result(instance, context=context): + if value is util.Uninferable: + return value + + inferred = next(value.infer(context=context)) + return inferred.bool_value() + except exceptions.InferenceError: + pass + return util.Uninferable + + +class BaseInstance(Proxy): + """An instance base class, which provides lookup methods for potential instances.""" + + special_attributes = None + + def display_type(self): + return "Instance of" + + def getattr(self, name, context=None, lookupclass=True): + try: + values = self._proxied.instance_attr(name, context) + except exceptions.AttributeInferenceError as exc: + if self.special_attributes and name in self.special_attributes: + return [self.special_attributes.lookup(name)] + + if lookupclass: + # Class attributes not available through the instance + # unless they are explicitly defined. + return self._proxied.getattr(name, context, class_context=False) + + raise exceptions.AttributeInferenceError( + target=self, attribute=name, context=context + ) from exc + # since we've no context information, return matching class members as + # well + if lookupclass: + try: + return values + self._proxied.getattr( + name, context, class_context=False + ) + except exceptions.AttributeInferenceError: + pass + return values + + def igetattr(self, name, context=None): + """inferred getattr""" + if not context: + context = contextmod.InferenceContext() + try: + # avoid recursively inferring the same attr on the same class + if context.push((self._proxied, name)): + raise exceptions.InferenceError( + message="Cannot infer the same attribute again", + node=self, + context=context, + ) + + # XXX frame should be self._proxied, or not ? + get_attr = self.getattr(name, context, lookupclass=False) + yield from _infer_stmts( + self._wrap_attr(get_attr, context), context, frame=self + ) + except exceptions.AttributeInferenceError as error: + try: + # fallback to class.igetattr since it has some logic to handle + # descriptors + # But only if the _proxied is the Class. + if self._proxied.__class__.__name__ != "ClassDef": + raise + attrs = self._proxied.igetattr(name, context, class_context=False) + yield from self._wrap_attr(attrs, context) + except exceptions.AttributeInferenceError as error: + raise exceptions.InferenceError(**vars(error)) from error + + def _wrap_attr(self, attrs, context=None): + """wrap bound methods of attrs in a InstanceMethod proxies""" + for attr in attrs: + if isinstance(attr, UnboundMethod): + if _is_property(attr): + yield from attr.infer_call_result(self, context) + else: + yield BoundMethod(attr, self) + elif hasattr(attr, "name") and attr.name == "": + if attr.args.arguments and attr.args.arguments[0].name == "self": + yield BoundMethod(attr, self) + continue + yield attr + else: + yield attr + + def infer_call_result(self, caller, context=None): + """infer what a class instance is returning when called""" + context = contextmod.bind_context_to_node(context, self) + inferred = False + for node in self._proxied.igetattr("__call__", context): + if node is util.Uninferable or not node.callable(): + continue + for res in node.infer_call_result(caller, context): + inferred = True + yield res + if not inferred: + raise exceptions.InferenceError(node=self, caller=caller, context=context) + + +class Instance(BaseInstance): + """A special node representing a class instance.""" + + # pylint: disable=unnecessary-lambda + special_attributes = util.lazy_descriptor(lambda: objectmodel.InstanceModel()) + + def __repr__(self): + return "" % ( + self._proxied.root().name, + self._proxied.name, + id(self), + ) + + def __str__(self): + return "Instance of %s.%s" % (self._proxied.root().name, self._proxied.name) + + def callable(self): + try: + self._proxied.getattr("__call__", class_context=False) + return True + except exceptions.AttributeInferenceError: + return False + + def pytype(self): + return self._proxied.qname() + + def display_type(self): + return "Instance of" + + def bool_value(self, context=None): + """Infer the truth value for an Instance + + The truth value of an instance is determined by these conditions: + + * if it implements __bool__ on Python 3 or __nonzero__ + on Python 2, then its bool value will be determined by + calling this special method and checking its result. + * when this method is not defined, __len__() is called, if it + is defined, and the object is considered true if its result is + nonzero. If a class defines neither __len__() nor __bool__(), + all its instances are considered true. + """ + context = context or contextmod.InferenceContext() + context.callcontext = contextmod.CallContext(args=[]) + context.boundnode = self + + try: + result = _infer_method_result_truth(self, BOOL_SPECIAL_METHOD, context) + except (exceptions.InferenceError, exceptions.AttributeInferenceError): + # Fallback to __len__. + try: + result = _infer_method_result_truth(self, "__len__", context) + except (exceptions.AttributeInferenceError, exceptions.InferenceError): + return True + return result + + # This is set in inference.py. + def getitem(self, index, context=None): + pass + + +class UnboundMethod(Proxy): + """a special node representing a method not bound to an instance""" + + # pylint: disable=unnecessary-lambda + special_attributes = util.lazy_descriptor(lambda: objectmodel.UnboundMethodModel()) + + def __repr__(self): + frame = self._proxied.parent.frame() + return "<%s %s of %s at 0x%s" % ( + self.__class__.__name__, + self._proxied.name, + frame.qname(), + id(self), + ) + + def implicit_parameters(self): + return 0 + + def is_bound(self): + return False + + def getattr(self, name, context=None): + if name in self.special_attributes: + return [self.special_attributes.lookup(name)] + return self._proxied.getattr(name, context) + + def igetattr(self, name, context=None): + if name in self.special_attributes: + return iter((self.special_attributes.lookup(name),)) + return self._proxied.igetattr(name, context) + + def infer_call_result(self, caller, context): + """ + The boundnode of the regular context with a function called + on ``object.__new__`` will be of type ``object``, + which is incorrect for the argument in general. + If no context is given the ``object.__new__`` call argument will + correctly inferred except when inside a call that requires + the additional context (such as a classmethod) of the boundnode + to determine which class the method was called from + """ + + # If we're unbound method __new__ of builtin object, the result is an + # instance of the class given as first argument. + if ( + self._proxied.name == "__new__" + and self._proxied.parent.frame().qname() == "%s.object" % BUILTINS + ): + if caller.args: + node_context = context.extra_context.get(caller.args[0]) + infer = caller.args[0].infer(context=node_context) + else: + infer = [] + return (Instance(x) if x is not util.Uninferable else x for x in infer) + return self._proxied.infer_call_result(caller, context) + + def bool_value(self, context=None): + return True + + +class BoundMethod(UnboundMethod): + """a special node representing a method bound to an instance""" + + # pylint: disable=unnecessary-lambda + special_attributes = util.lazy_descriptor(lambda: objectmodel.BoundMethodModel()) + + def __init__(self, proxy, bound): + UnboundMethod.__init__(self, proxy) + self.bound = bound + + def implicit_parameters(self): + if self.name == "__new__": + # __new__ acts as a classmethod but the class argument is not implicit. + return 0 + return 1 + + def is_bound(self): + return True + + def _infer_type_new_call(self, caller, context): + """Try to infer what type.__new__(mcs, name, bases, attrs) returns. + + In order for such call to be valid, the metaclass needs to be + a subtype of ``type``, the name needs to be a string, the bases + needs to be a tuple of classes + """ + # pylint: disable=import-outside-toplevel; circular import + from astroid import node_classes + + # Verify the metaclass + mcs = next(caller.args[0].infer(context=context)) + if mcs.__class__.__name__ != "ClassDef": + # Not a valid first argument. + return None + if not mcs.is_subtype_of("%s.type" % BUILTINS): + # Not a valid metaclass. + return None + + # Verify the name + name = next(caller.args[1].infer(context=context)) + if name.__class__.__name__ != "Const": + # Not a valid name, needs to be a const. + return None + if not isinstance(name.value, str): + # Needs to be a string. + return None + + # Verify the bases + bases = next(caller.args[2].infer(context=context)) + if bases.__class__.__name__ != "Tuple": + # Needs to be a tuple. + return None + inferred_bases = [next(elt.infer(context=context)) for elt in bases.elts] + if any(base.__class__.__name__ != "ClassDef" for base in inferred_bases): + # All the bases needs to be Classes + return None + + # Verify the attributes. + attrs = next(caller.args[3].infer(context=context)) + if attrs.__class__.__name__ != "Dict": + # Needs to be a dictionary. + return None + cls_locals = collections.defaultdict(list) + for key, value in attrs.items: + key = next(key.infer(context=context)) + value = next(value.infer(context=context)) + # Ignore non string keys + if key.__class__.__name__ == "Const" and isinstance(key.value, str): + cls_locals[key.value].append(value) + + # Build the class from now. + cls = mcs.__class__( + name=name.value, + lineno=caller.lineno, + col_offset=caller.col_offset, + parent=caller, + ) + empty = node_classes.Pass() + cls.postinit( + bases=bases.elts, + body=[empty], + decorators=[], + newstyle=True, + metaclass=mcs, + keywords=[], + ) + cls.locals = cls_locals + return cls + + def infer_call_result(self, caller, context=None): + context = contextmod.bind_context_to_node(context, self.bound) + if ( + self.bound.__class__.__name__ == "ClassDef" + and self.bound.name == "type" + and self.name == "__new__" + and len(caller.args) == 4 + ): + # Check if we have a ``type.__new__(mcs, name, bases, attrs)`` call. + new_cls = self._infer_type_new_call(caller, context) + if new_cls: + return iter((new_cls,)) + + return super().infer_call_result(caller, context) + + def bool_value(self, context=None): + return True + + +class Generator(BaseInstance): + """a special node representing a generator. + + Proxied class is set once for all in raw_building. + """ + + # pylint: disable=unnecessary-lambda + special_attributes = util.lazy_descriptor(lambda: objectmodel.GeneratorModel()) + + # pylint: disable=super-init-not-called + def __init__(self, parent=None): + self.parent = parent + + def callable(self): + return False + + def pytype(self): + return "%s.generator" % BUILTINS + + def display_type(self): + return "Generator" + + def bool_value(self, context=None): + return True + + def __repr__(self): + return "" % ( + self._proxied.name, + self.lineno, + id(self), + ) + + def __str__(self): + return "Generator(%s)" % (self._proxied.name) + + +class AsyncGenerator(Generator): + """Special node representing an async generator""" + + def pytype(self): + return "%s.async_generator" % BUILTINS + + def display_type(self): + return "AsyncGenerator" + + def __repr__(self): + return "" % ( + self._proxied.name, + self.lineno, + id(self), + ) + + def __str__(self): + return "AsyncGenerator(%s)" % (self._proxied.name) diff --git a/env/lib/python3.8/site-packages/astroid/brain/brain_argparse.py b/env/lib/python3.8/site-packages/astroid/brain/brain_argparse.py new file mode 100644 index 000000000..6a7556f61 --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/brain/brain_argparse.py @@ -0,0 +1,33 @@ +from astroid import MANAGER, arguments, nodes, inference_tip, UseInferenceDefault + + +def infer_namespace(node, context=None): + callsite = arguments.CallSite.from_call(node, context=context) + if not callsite.keyword_arguments: + # Cannot make sense of it. + raise UseInferenceDefault() + + class_node = nodes.ClassDef("Namespace", "docstring") + class_node.parent = node.parent + for attr in set(callsite.keyword_arguments): + fake_node = nodes.EmptyNode() + fake_node.parent = class_node + fake_node.attrname = attr + class_node.instance_attrs[attr] = [fake_node] + return iter((class_node.instantiate_class(),)) + + +def _looks_like_namespace(node): + func = node.func + if isinstance(func, nodes.Attribute): + return ( + func.attrname == "Namespace" + and isinstance(func.expr, nodes.Name) + and func.expr.name == "argparse" + ) + return False + + +MANAGER.register_transform( + nodes.Call, inference_tip(infer_namespace), _looks_like_namespace +) diff --git a/env/lib/python3.8/site-packages/astroid/brain/brain_attrs.py b/env/lib/python3.8/site-packages/astroid/brain/brain_attrs.py new file mode 100644 index 000000000..670736fe4 --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/brain/brain_attrs.py @@ -0,0 +1,65 @@ +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER +""" +Astroid hook for the attrs library + +Without this hook pylint reports unsupported-assignment-operation +for attrs classes +""" + +import astroid +from astroid import MANAGER + + +ATTRIB_NAMES = frozenset(("attr.ib", "attrib", "attr.attrib")) +ATTRS_NAMES = frozenset(("attr.s", "attrs", "attr.attrs", "attr.attributes")) + + +def is_decorated_with_attrs(node, decorator_names=ATTRS_NAMES): + """Return True if a decorated node has + an attr decorator applied.""" + if not node.decorators: + return False + for decorator_attribute in node.decorators.nodes: + if isinstance(decorator_attribute, astroid.Call): # decorator with arguments + decorator_attribute = decorator_attribute.func + if decorator_attribute.as_string() in decorator_names: + return True + return False + + +def attr_attributes_transform(node): + """Given that the ClassNode has an attr decorator, + rewrite class attributes as instance attributes + """ + # Astroid can't infer this attribute properly + # Prevents https://github.com/PyCQA/pylint/issues/1884 + node.locals["__attrs_attrs__"] = [astroid.Unknown(parent=node)] + + for cdefbodynode in node.body: + if not isinstance(cdefbodynode, (astroid.Assign, astroid.AnnAssign)): + continue + if isinstance(cdefbodynode.value, astroid.Call): + if cdefbodynode.value.func.as_string() not in ATTRIB_NAMES: + continue + else: + continue + targets = ( + cdefbodynode.targets + if hasattr(cdefbodynode, "targets") + else [cdefbodynode.target] + ) + for target in targets: + + rhs_node = astroid.Unknown( + lineno=cdefbodynode.lineno, + col_offset=cdefbodynode.col_offset, + parent=cdefbodynode, + ) + node.locals[target.name] = [rhs_node] + node.instance_attrs[target.name] = [rhs_node] + + +MANAGER.register_transform( + astroid.ClassDef, attr_attributes_transform, is_decorated_with_attrs +) diff --git a/env/lib/python3.8/site-packages/astroid/brain/brain_boto3.py b/env/lib/python3.8/site-packages/astroid/brain/brain_boto3.py new file mode 100644 index 000000000..342ca5719 --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/brain/brain_boto3.py @@ -0,0 +1,28 @@ +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER + +"""Astroid hooks for understanding boto3.ServiceRequest()""" +import astroid +from astroid import MANAGER, extract_node + +BOTO_SERVICE_FACTORY_QUALIFIED_NAME = "boto3.resources.base.ServiceResource" + + +def service_request_transform(node): + """Transform ServiceResource to look like dynamic classes""" + code = """ + def __getattr__(self, attr): + return 0 + """ + func_getattr = extract_node(code) + node.locals["__getattr__"] = [func_getattr] + return node + + +def _looks_like_boto3_service_request(node): + return node.qname() == BOTO_SERVICE_FACTORY_QUALIFIED_NAME + + +MANAGER.register_transform( + astroid.ClassDef, service_request_transform, _looks_like_boto3_service_request +) diff --git a/env/lib/python3.8/site-packages/astroid/brain/brain_builtin_inference.py b/env/lib/python3.8/site-packages/astroid/brain/brain_builtin_inference.py new file mode 100644 index 000000000..4b07ac5c0 --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/brain/brain_builtin_inference.py @@ -0,0 +1,873 @@ +# -*- coding: utf-8 -*- +# Copyright (c) 2014-2020 Claudiu Popa +# Copyright (c) 2014-2015 LOGILAB S.A. (Paris, FRANCE) +# Copyright (c) 2015-2016 Ceridwen +# Copyright (c) 2015 Rene Zhang +# Copyright (c) 2018 Bryce Guinta +# Copyright (c) 2018 Ville Skyttä +# Copyright (c) 2019 Stanislav Levin +# Copyright (c) 2019 David Liu +# Copyright (c) 2019 Bryce Guinta +# Copyright (c) 2019 Frédéric Chapoton + +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER + +"""Astroid hooks for various builtins.""" + +from functools import partial +from textwrap import dedent + +import six +from astroid import ( + MANAGER, + UseInferenceDefault, + AttributeInferenceError, + inference_tip, + InferenceError, + NameInferenceError, + AstroidTypeError, + MroError, +) +from astroid import arguments +from astroid.builder import AstroidBuilder +from astroid import helpers +from astroid import nodes +from astroid import objects +from astroid import scoped_nodes +from astroid import util + + +OBJECT_DUNDER_NEW = "object.__new__" + + +def _extend_str(class_node, rvalue): + """function to extend builtin str/unicode class""" + code = dedent( + """ + class whatever(object): + def join(self, iterable): + return {rvalue} + def replace(self, old, new, count=None): + return {rvalue} + def format(self, *args, **kwargs): + return {rvalue} + def encode(self, encoding='ascii', errors=None): + return '' + def decode(self, encoding='ascii', errors=None): + return u'' + def capitalize(self): + return {rvalue} + def title(self): + return {rvalue} + def lower(self): + return {rvalue} + def upper(self): + return {rvalue} + def swapcase(self): + return {rvalue} + def index(self, sub, start=None, end=None): + return 0 + def find(self, sub, start=None, end=None): + return 0 + def count(self, sub, start=None, end=None): + return 0 + def strip(self, chars=None): + return {rvalue} + def lstrip(self, chars=None): + return {rvalue} + def rstrip(self, chars=None): + return {rvalue} + def rjust(self, width, fillchar=None): + return {rvalue} + def center(self, width, fillchar=None): + return {rvalue} + def ljust(self, width, fillchar=None): + return {rvalue} + """ + ) + code = code.format(rvalue=rvalue) + fake = AstroidBuilder(MANAGER).string_build(code)["whatever"] + for method in fake.mymethods(): + method.parent = class_node + method.lineno = None + method.col_offset = None + if "__class__" in method.locals: + method.locals["__class__"] = [class_node] + class_node.locals[method.name] = [method] + method.parent = class_node + + +def _extend_builtins(class_transforms): + builtin_ast = MANAGER.builtins_module + for class_name, transform in class_transforms.items(): + transform(builtin_ast[class_name]) + + +_extend_builtins( + { + "bytes": partial(_extend_str, rvalue="b''"), + "str": partial(_extend_str, rvalue="''"), + } +) + + +def _builtin_filter_predicate(node, builtin_name): + if isinstance(node.func, nodes.Name) and node.func.name == builtin_name: + return True + if isinstance(node.func, nodes.Attribute): + return ( + node.func.attrname == "fromkeys" + and isinstance(node.func.expr, nodes.Name) + and node.func.expr.name == "dict" + ) + return False + + +def register_builtin_transform(transform, builtin_name): + """Register a new transform function for the given *builtin_name*. + + The transform function must accept two parameters, a node and + an optional context. + """ + + def _transform_wrapper(node, context=None): + result = transform(node, context=context) + if result: + if not result.parent: + # Let the transformation function determine + # the parent for its result. Otherwise, + # we set it to be the node we transformed from. + result.parent = node + + if result.lineno is None: + result.lineno = node.lineno + if result.col_offset is None: + result.col_offset = node.col_offset + return iter([result]) + + MANAGER.register_transform( + nodes.Call, + inference_tip(_transform_wrapper), + partial(_builtin_filter_predicate, builtin_name=builtin_name), + ) + + +def _container_generic_inference(node, context, node_type, transform): + args = node.args + if not args: + return node_type() + if len(node.args) > 1: + raise UseInferenceDefault() + + (arg,) = args + transformed = transform(arg) + if not transformed: + try: + inferred = next(arg.infer(context=context)) + except (InferenceError, StopIteration): + raise UseInferenceDefault() + if inferred is util.Uninferable: + raise UseInferenceDefault() + transformed = transform(inferred) + if not transformed or transformed is util.Uninferable: + raise UseInferenceDefault() + return transformed + + +def _container_generic_transform(arg, context, klass, iterables, build_elts): + if isinstance(arg, klass): + return arg + elif isinstance(arg, iterables): + if all(isinstance(elt, nodes.Const) for elt in arg.elts): + elts = [elt.value for elt in arg.elts] + else: + # TODO: Does not handle deduplication for sets. + elts = [] + for element in arg.elts: + inferred = helpers.safe_infer(element, context=context) + if inferred: + evaluated_object = nodes.EvaluatedObject( + original=element, value=inferred + ) + elts.append(evaluated_object) + elif isinstance(arg, nodes.Dict): + # Dicts need to have consts as strings already. + if not all(isinstance(elt[0], nodes.Const) for elt in arg.items): + raise UseInferenceDefault() + elts = [item[0].value for item in arg.items] + elif isinstance(arg, nodes.Const) and isinstance( + arg.value, (six.string_types, six.binary_type) + ): + elts = arg.value + else: + return + return klass.from_elements(elts=build_elts(elts)) + + +def _infer_builtin_container( + node, context, klass=None, iterables=None, build_elts=None +): + transform_func = partial( + _container_generic_transform, + context=context, + klass=klass, + iterables=iterables, + build_elts=build_elts, + ) + + return _container_generic_inference(node, context, klass, transform_func) + + +# pylint: disable=invalid-name +infer_tuple = partial( + _infer_builtin_container, + klass=nodes.Tuple, + iterables=( + nodes.List, + nodes.Set, + objects.FrozenSet, + objects.DictItems, + objects.DictKeys, + objects.DictValues, + ), + build_elts=tuple, +) + +infer_list = partial( + _infer_builtin_container, + klass=nodes.List, + iterables=( + nodes.Tuple, + nodes.Set, + objects.FrozenSet, + objects.DictItems, + objects.DictKeys, + objects.DictValues, + ), + build_elts=list, +) + +infer_set = partial( + _infer_builtin_container, + klass=nodes.Set, + iterables=(nodes.List, nodes.Tuple, objects.FrozenSet, objects.DictKeys), + build_elts=set, +) + +infer_frozenset = partial( + _infer_builtin_container, + klass=objects.FrozenSet, + iterables=(nodes.List, nodes.Tuple, nodes.Set, objects.FrozenSet, objects.DictKeys), + build_elts=frozenset, +) + + +def _get_elts(arg, context): + is_iterable = lambda n: isinstance(n, (nodes.List, nodes.Tuple, nodes.Set)) + try: + inferred = next(arg.infer(context)) + except (InferenceError, NameInferenceError): + raise UseInferenceDefault() + if isinstance(inferred, nodes.Dict): + items = inferred.items + elif is_iterable(inferred): + items = [] + for elt in inferred.elts: + # If an item is not a pair of two items, + # then fallback to the default inference. + # Also, take in consideration only hashable items, + # tuples and consts. We are choosing Names as well. + if not is_iterable(elt): + raise UseInferenceDefault() + if len(elt.elts) != 2: + raise UseInferenceDefault() + if not isinstance(elt.elts[0], (nodes.Tuple, nodes.Const, nodes.Name)): + raise UseInferenceDefault() + items.append(tuple(elt.elts)) + else: + raise UseInferenceDefault() + return items + + +def infer_dict(node, context=None): + """Try to infer a dict call to a Dict node. + + The function treats the following cases: + + * dict() + * dict(mapping) + * dict(iterable) + * dict(iterable, **kwargs) + * dict(mapping, **kwargs) + * dict(**kwargs) + + If a case can't be inferred, we'll fallback to default inference. + """ + call = arguments.CallSite.from_call(node, context=context) + if call.has_invalid_arguments() or call.has_invalid_keywords(): + raise UseInferenceDefault + + args = call.positional_arguments + kwargs = list(call.keyword_arguments.items()) + + if not args and not kwargs: + # dict() + return nodes.Dict() + elif kwargs and not args: + # dict(a=1, b=2, c=4) + items = [(nodes.Const(key), value) for key, value in kwargs] + elif len(args) == 1 and kwargs: + # dict(some_iterable, b=2, c=4) + elts = _get_elts(args[0], context) + keys = [(nodes.Const(key), value) for key, value in kwargs] + items = elts + keys + elif len(args) == 1: + items = _get_elts(args[0], context) + else: + raise UseInferenceDefault() + + value = nodes.Dict( + col_offset=node.col_offset, lineno=node.lineno, parent=node.parent + ) + value.postinit(items) + return value + + +def infer_super(node, context=None): + """Understand super calls. + + There are some restrictions for what can be understood: + + * unbounded super (one argument form) is not understood. + + * if the super call is not inside a function (classmethod or method), + then the default inference will be used. + + * if the super arguments can't be inferred, the default inference + will be used. + """ + if len(node.args) == 1: + # Ignore unbounded super. + raise UseInferenceDefault + + scope = node.scope() + if not isinstance(scope, nodes.FunctionDef): + # Ignore non-method uses of super. + raise UseInferenceDefault + if scope.type not in ("classmethod", "method"): + # Not interested in staticmethods. + raise UseInferenceDefault + + cls = scoped_nodes.get_wrapping_class(scope) + if not len(node.args): + mro_pointer = cls + # In we are in a classmethod, the interpreter will fill + # automatically the class as the second argument, not an instance. + if scope.type == "classmethod": + mro_type = cls + else: + mro_type = cls.instantiate_class() + else: + try: + mro_pointer = next(node.args[0].infer(context=context)) + except InferenceError: + raise UseInferenceDefault + try: + mro_type = next(node.args[1].infer(context=context)) + except InferenceError: + raise UseInferenceDefault + + if mro_pointer is util.Uninferable or mro_type is util.Uninferable: + # No way we could understand this. + raise UseInferenceDefault + + super_obj = objects.Super( + mro_pointer=mro_pointer, mro_type=mro_type, self_class=cls, scope=scope + ) + super_obj.parent = node + return super_obj + + +def _infer_getattr_args(node, context): + if len(node.args) not in (2, 3): + # Not a valid getattr call. + raise UseInferenceDefault + + try: + obj = next(node.args[0].infer(context=context)) + attr = next(node.args[1].infer(context=context)) + except InferenceError: + raise UseInferenceDefault + + if obj is util.Uninferable or attr is util.Uninferable: + # If one of the arguments is something we can't infer, + # then also make the result of the getattr call something + # which is unknown. + return util.Uninferable, util.Uninferable + + is_string = isinstance(attr, nodes.Const) and isinstance( + attr.value, six.string_types + ) + if not is_string: + raise UseInferenceDefault + + return obj, attr.value + + +def infer_getattr(node, context=None): + """Understand getattr calls + + If one of the arguments is an Uninferable object, then the + result will be an Uninferable object. Otherwise, the normal attribute + lookup will be done. + """ + obj, attr = _infer_getattr_args(node, context) + if ( + obj is util.Uninferable + or attr is util.Uninferable + or not hasattr(obj, "igetattr") + ): + return util.Uninferable + + try: + return next(obj.igetattr(attr, context=context)) + except (StopIteration, InferenceError, AttributeInferenceError): + if len(node.args) == 3: + # Try to infer the default and return it instead. + try: + return next(node.args[2].infer(context=context)) + except InferenceError: + raise UseInferenceDefault + + raise UseInferenceDefault + + +def infer_hasattr(node, context=None): + """Understand hasattr calls + + This always guarantees three possible outcomes for calling + hasattr: Const(False) when we are sure that the object + doesn't have the intended attribute, Const(True) when + we know that the object has the attribute and Uninferable + when we are unsure of the outcome of the function call. + """ + try: + obj, attr = _infer_getattr_args(node, context) + if ( + obj is util.Uninferable + or attr is util.Uninferable + or not hasattr(obj, "getattr") + ): + return util.Uninferable + obj.getattr(attr, context=context) + except UseInferenceDefault: + # Can't infer something from this function call. + return util.Uninferable + except AttributeInferenceError: + # Doesn't have it. + return nodes.Const(False) + return nodes.Const(True) + + +def infer_callable(node, context=None): + """Understand callable calls + + This follows Python's semantics, where an object + is callable if it provides an attribute __call__, + even though that attribute is something which can't be + called. + """ + if len(node.args) != 1: + # Invalid callable call. + raise UseInferenceDefault + + argument = node.args[0] + try: + inferred = next(argument.infer(context=context)) + except InferenceError: + return util.Uninferable + if inferred is util.Uninferable: + return util.Uninferable + return nodes.Const(inferred.callable()) + + +def infer_property(node, context=None): + """Understand `property` class + + This only infers the output of `property` + call, not the arguments themselves. + """ + if len(node.args) < 1: + # Invalid property call. + raise UseInferenceDefault + + getter = node.args[0] + try: + inferred = next(getter.infer(context=context)) + except InferenceError: + raise UseInferenceDefault + + if not isinstance(inferred, (nodes.FunctionDef, nodes.Lambda)): + raise UseInferenceDefault + + return objects.Property( + function=inferred, + name=inferred.name, + doc=getattr(inferred, "doc", None), + lineno=node.lineno, + parent=node, + col_offset=node.col_offset, + ) + + +def infer_bool(node, context=None): + """Understand bool calls.""" + if len(node.args) > 1: + # Invalid bool call. + raise UseInferenceDefault + + if not node.args: + return nodes.Const(False) + + argument = node.args[0] + try: + inferred = next(argument.infer(context=context)) + except InferenceError: + return util.Uninferable + if inferred is util.Uninferable: + return util.Uninferable + + bool_value = inferred.bool_value(context=context) + if bool_value is util.Uninferable: + return util.Uninferable + return nodes.Const(bool_value) + + +def infer_type(node, context=None): + """Understand the one-argument form of *type*.""" + if len(node.args) != 1: + raise UseInferenceDefault + + return helpers.object_type(node.args[0], context) + + +def infer_slice(node, context=None): + """Understand `slice` calls.""" + args = node.args + if not 0 < len(args) <= 3: + raise UseInferenceDefault + + infer_func = partial(helpers.safe_infer, context=context) + args = [infer_func(arg) for arg in args] + for arg in args: + if not arg or arg is util.Uninferable: + raise UseInferenceDefault + if not isinstance(arg, nodes.Const): + raise UseInferenceDefault + if not isinstance(arg.value, (type(None), int)): + raise UseInferenceDefault + + if len(args) < 3: + # Make sure we have 3 arguments. + args.extend([None] * (3 - len(args))) + + slice_node = nodes.Slice( + lineno=node.lineno, col_offset=node.col_offset, parent=node.parent + ) + slice_node.postinit(*args) + return slice_node + + +def _infer_object__new__decorator(node, context=None): + # Instantiate class immediately + # since that's what @object.__new__ does + return iter((node.instantiate_class(),)) + + +def _infer_object__new__decorator_check(node): + """Predicate before inference_tip + + Check if the given ClassDef has an @object.__new__ decorator + """ + if not node.decorators: + return False + + for decorator in node.decorators.nodes: + if isinstance(decorator, nodes.Attribute): + if decorator.as_string() == OBJECT_DUNDER_NEW: + return True + return False + + +def infer_issubclass(callnode, context=None): + """Infer issubclass() calls + + :param nodes.Call callnode: an `issubclass` call + :param InferenceContext: the context for the inference + :rtype nodes.Const: Boolean Const value of the `issubclass` call + :raises UseInferenceDefault: If the node cannot be inferred + """ + call = arguments.CallSite.from_call(callnode, context=context) + if call.keyword_arguments: + # issubclass doesn't support keyword arguments + raise UseInferenceDefault("TypeError: issubclass() takes no keyword arguments") + if len(call.positional_arguments) != 2: + raise UseInferenceDefault( + "Expected two arguments, got {count}".format( + count=len(call.positional_arguments) + ) + ) + # The left hand argument is the obj to be checked + obj_node, class_or_tuple_node = call.positional_arguments + + try: + obj_type = next(obj_node.infer(context=context)) + except InferenceError as exc: + raise UseInferenceDefault from exc + if not isinstance(obj_type, nodes.ClassDef): + raise UseInferenceDefault("TypeError: arg 1 must be class") + + # The right hand argument is the class(es) that the given + # object is to be checked against. + try: + class_container = _class_or_tuple_to_container( + class_or_tuple_node, context=context + ) + except InferenceError as exc: + raise UseInferenceDefault from exc + try: + issubclass_bool = helpers.object_issubclass(obj_type, class_container, context) + except AstroidTypeError as exc: + raise UseInferenceDefault("TypeError: " + str(exc)) from exc + except MroError as exc: + raise UseInferenceDefault from exc + return nodes.Const(issubclass_bool) + + +def infer_isinstance(callnode, context=None): + """Infer isinstance calls + + :param nodes.Call callnode: an isinstance call + :param InferenceContext: context for call + (currently unused but is a common interface for inference) + :rtype nodes.Const: Boolean Const value of isinstance call + + :raises UseInferenceDefault: If the node cannot be inferred + """ + call = arguments.CallSite.from_call(callnode, context=context) + if call.keyword_arguments: + # isinstance doesn't support keyword arguments + raise UseInferenceDefault("TypeError: isinstance() takes no keyword arguments") + if len(call.positional_arguments) != 2: + raise UseInferenceDefault( + "Expected two arguments, got {count}".format( + count=len(call.positional_arguments) + ) + ) + # The left hand argument is the obj to be checked + obj_node, class_or_tuple_node = call.positional_arguments + # The right hand argument is the class(es) that the given + # obj is to be check is an instance of + try: + class_container = _class_or_tuple_to_container( + class_or_tuple_node, context=context + ) + except InferenceError: + raise UseInferenceDefault + try: + isinstance_bool = helpers.object_isinstance(obj_node, class_container, context) + except AstroidTypeError as exc: + raise UseInferenceDefault("TypeError: " + str(exc)) + except MroError as exc: + raise UseInferenceDefault from exc + if isinstance_bool is util.Uninferable: + raise UseInferenceDefault + return nodes.Const(isinstance_bool) + + +def _class_or_tuple_to_container(node, context=None): + # Move inferences results into container + # to simplify later logic + # raises InferenceError if any of the inferences fall through + node_infer = next(node.infer(context=context)) + # arg2 MUST be a type or a TUPLE of types + # for isinstance + if isinstance(node_infer, nodes.Tuple): + class_container = [ + next(node.infer(context=context)) for node in node_infer.elts + ] + class_container = [ + klass_node for klass_node in class_container if klass_node is not None + ] + else: + class_container = [node_infer] + return class_container + + +def infer_len(node, context=None): + """Infer length calls + + :param nodes.Call node: len call to infer + :param context.InferenceContext: node context + :rtype nodes.Const: a Const node with the inferred length, if possible + """ + call = arguments.CallSite.from_call(node, context=context) + if call.keyword_arguments: + raise UseInferenceDefault("TypeError: len() must take no keyword arguments") + if len(call.positional_arguments) != 1: + raise UseInferenceDefault( + "TypeError: len() must take exactly one argument " + "({len}) given".format(len=len(call.positional_arguments)) + ) + [argument_node] = call.positional_arguments + try: + return nodes.Const(helpers.object_len(argument_node, context=context)) + except (AstroidTypeError, InferenceError) as exc: + raise UseInferenceDefault(str(exc)) from exc + + +def infer_str(node, context=None): + """Infer str() calls + + :param nodes.Call node: str() call to infer + :param context.InferenceContext: node context + :rtype nodes.Const: a Const containing an empty string + """ + call = arguments.CallSite.from_call(node, context=context) + if call.keyword_arguments: + raise UseInferenceDefault("TypeError: str() must take no keyword arguments") + try: + return nodes.Const("") + except (AstroidTypeError, InferenceError) as exc: + raise UseInferenceDefault(str(exc)) from exc + + +def infer_int(node, context=None): + """Infer int() calls + + :param nodes.Call node: int() call to infer + :param context.InferenceContext: node context + :rtype nodes.Const: a Const containing the integer value of the int() call + """ + call = arguments.CallSite.from_call(node, context=context) + if call.keyword_arguments: + raise UseInferenceDefault("TypeError: int() must take no keyword arguments") + + if call.positional_arguments: + try: + first_value = next(call.positional_arguments[0].infer(context=context)) + except (InferenceError, StopIteration) as exc: + raise UseInferenceDefault(str(exc)) from exc + + if first_value is util.Uninferable: + raise UseInferenceDefault + + if isinstance(first_value, nodes.Const) and isinstance( + first_value.value, (int, str) + ): + try: + actual_value = int(first_value.value) + except ValueError: + return nodes.Const(0) + return nodes.Const(actual_value) + + return nodes.Const(0) + + +def infer_dict_fromkeys(node, context=None): + """Infer dict.fromkeys + + :param nodes.Call node: dict.fromkeys() call to infer + :param context.InferenceContext: node context + :rtype nodes.Dict: + a Dictionary containing the values that astroid was able to infer. + In case the inference failed for any reason, an empty dictionary + will be inferred instead. + """ + + def _build_dict_with_elements(elements): + new_node = nodes.Dict( + col_offset=node.col_offset, lineno=node.lineno, parent=node.parent + ) + new_node.postinit(elements) + return new_node + + call = arguments.CallSite.from_call(node, context=context) + if call.keyword_arguments: + raise UseInferenceDefault("TypeError: int() must take no keyword arguments") + if len(call.positional_arguments) not in {1, 2}: + raise UseInferenceDefault( + "TypeError: Needs between 1 and 2 positional arguments" + ) + + default = nodes.Const(None) + values = call.positional_arguments[0] + try: + inferred_values = next(values.infer(context=context)) + except InferenceError: + return _build_dict_with_elements([]) + if inferred_values is util.Uninferable: + return _build_dict_with_elements([]) + + # Limit to a couple of potential values, as this can become pretty complicated + accepted_iterable_elements = (nodes.Const,) + if isinstance(inferred_values, (nodes.List, nodes.Set, nodes.Tuple)): + elements = inferred_values.elts + for element in elements: + if not isinstance(element, accepted_iterable_elements): + # Fallback to an empty dict + return _build_dict_with_elements([]) + + elements_with_value = [(element, default) for element in elements] + return _build_dict_with_elements(elements_with_value) + + elif isinstance(inferred_values, nodes.Const) and isinstance( + inferred_values.value, (str, bytes) + ): + elements = [ + (nodes.Const(element), default) for element in inferred_values.value + ] + return _build_dict_with_elements(elements) + elif isinstance(inferred_values, nodes.Dict): + keys = inferred_values.itered() + for key in keys: + if not isinstance(key, accepted_iterable_elements): + # Fallback to an empty dict + return _build_dict_with_elements([]) + + elements_with_value = [(element, default) for element in keys] + return _build_dict_with_elements(elements_with_value) + + # Fallback to an empty dictionary + return _build_dict_with_elements([]) + + +# Builtins inference +register_builtin_transform(infer_bool, "bool") +register_builtin_transform(infer_super, "super") +register_builtin_transform(infer_callable, "callable") +register_builtin_transform(infer_property, "property") +register_builtin_transform(infer_getattr, "getattr") +register_builtin_transform(infer_hasattr, "hasattr") +register_builtin_transform(infer_tuple, "tuple") +register_builtin_transform(infer_set, "set") +register_builtin_transform(infer_list, "list") +register_builtin_transform(infer_dict, "dict") +register_builtin_transform(infer_frozenset, "frozenset") +register_builtin_transform(infer_type, "type") +register_builtin_transform(infer_slice, "slice") +register_builtin_transform(infer_isinstance, "isinstance") +register_builtin_transform(infer_issubclass, "issubclass") +register_builtin_transform(infer_len, "len") +register_builtin_transform(infer_str, "str") +register_builtin_transform(infer_int, "int") +register_builtin_transform(infer_dict_fromkeys, "dict.fromkeys") + + +# Infer object.__new__ calls +MANAGER.register_transform( + nodes.ClassDef, + inference_tip(_infer_object__new__decorator), + _infer_object__new__decorator_check, +) diff --git a/env/lib/python3.8/site-packages/astroid/brain/brain_collections.py b/env/lib/python3.8/site-packages/astroid/brain/brain_collections.py new file mode 100644 index 000000000..669c6ca41 --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/brain/brain_collections.py @@ -0,0 +1,75 @@ +# -*- coding: utf-8 -*- +# Copyright (c) 2016, 2018 Claudiu Popa +# Copyright (c) 2016-2017 Łukasz Rogalski +# Copyright (c) 2017 Derek Gustafson +# Copyright (c) 2018 Ioana Tagirta +# Copyright (c) 2019 Hugo van Kemenade + +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER +import sys + +import astroid + + +def _collections_transform(): + return astroid.parse( + """ + class defaultdict(dict): + default_factory = None + def __missing__(self, key): pass + def __getitem__(self, key): return default_factory + + """ + + _deque_mock() + + _ordered_dict_mock() + ) + + +def _deque_mock(): + base_deque_class = """ + class deque(object): + maxlen = 0 + def __init__(self, iterable=None, maxlen=None): + self.iterable = iterable or [] + def append(self, x): pass + def appendleft(self, x): pass + def clear(self): pass + def count(self, x): return 0 + def extend(self, iterable): pass + def extendleft(self, iterable): pass + def pop(self): return self.iterable[0] + def popleft(self): return self.iterable[0] + def remove(self, value): pass + def reverse(self): return reversed(self.iterable) + def rotate(self, n=1): return self + def __iter__(self): return self + def __reversed__(self): return self.iterable[::-1] + def __getitem__(self, index): return self.iterable[index] + def __setitem__(self, index, value): pass + def __delitem__(self, index): pass + def __bool__(self): return bool(self.iterable) + def __nonzero__(self): return bool(self.iterable) + def __contains__(self, o): return o in self.iterable + def __len__(self): return len(self.iterable) + def __copy__(self): return deque(self.iterable) + def copy(self): return deque(self.iterable) + def index(self, x, start=0, end=0): return 0 + def insert(self, x, i): pass + def __add__(self, other): pass + def __iadd__(self, other): pass + def __mul__(self, other): pass + def __imul__(self, other): pass + def __rmul__(self, other): pass""" + return base_deque_class + + +def _ordered_dict_mock(): + base_ordered_dict_class = """ + class OrderedDict(dict): + def __reversed__(self): return self[::-1] + def move_to_end(self, key, last=False): pass""" + return base_ordered_dict_class + + +astroid.register_module_extender(astroid.MANAGER, "collections", _collections_transform) diff --git a/env/lib/python3.8/site-packages/astroid/brain/brain_crypt.py b/env/lib/python3.8/site-packages/astroid/brain/brain_crypt.py new file mode 100644 index 000000000..491ee23c6 --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/brain/brain_crypt.py @@ -0,0 +1,26 @@ +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER +import sys +import astroid + +PY37 = sys.version_info >= (3, 7) + +if PY37: + # Since Python 3.7 Hashing Methods are added + # dynamically to globals() + + def _re_transform(): + return astroid.parse( + """ + from collections import namedtuple + _Method = namedtuple('_Method', 'name ident salt_chars total_size') + + METHOD_SHA512 = _Method('SHA512', '6', 16, 106) + METHOD_SHA256 = _Method('SHA256', '5', 16, 63) + METHOD_BLOWFISH = _Method('BLOWFISH', 2, 'b', 22) + METHOD_MD5 = _Method('MD5', '1', 8, 34) + METHOD_CRYPT = _Method('CRYPT', None, 2, 13) + """ + ) + + astroid.register_module_extender(astroid.MANAGER, "crypt", _re_transform) diff --git a/env/lib/python3.8/site-packages/astroid/brain/brain_curses.py b/env/lib/python3.8/site-packages/astroid/brain/brain_curses.py new file mode 100644 index 000000000..68e88b90a --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/brain/brain_curses.py @@ -0,0 +1,179 @@ +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER +import astroid + + +def _curses_transform(): + return astroid.parse( + """ + A_ALTCHARSET = 1 + A_BLINK = 1 + A_BOLD = 1 + A_DIM = 1 + A_INVIS = 1 + A_ITALIC = 1 + A_NORMAL = 1 + A_PROTECT = 1 + A_REVERSE = 1 + A_STANDOUT = 1 + A_UNDERLINE = 1 + A_HORIZONTAL = 1 + A_LEFT = 1 + A_LOW = 1 + A_RIGHT = 1 + A_TOP = 1 + A_VERTICAL = 1 + A_CHARTEXT = 1 + A_ATTRIBUTES = 1 + A_CHARTEXT = 1 + A_COLOR = 1 + KEY_MIN = 1 + KEY_BREAK = 1 + KEY_DOWN = 1 + KEY_UP = 1 + KEY_LEFT = 1 + KEY_RIGHT = 1 + KEY_HOME = 1 + KEY_BACKSPACE = 1 + KEY_F0 = 1 + KEY_Fn = 1 + KEY_DL = 1 + KEY_IL = 1 + KEY_DC = 1 + KEY_IC = 1 + KEY_EIC = 1 + KEY_CLEAR = 1 + KEY_EOS = 1 + KEY_EOL = 1 + KEY_SF = 1 + KEY_SR = 1 + KEY_NPAGE = 1 + KEY_PPAGE = 1 + KEY_STAB = 1 + KEY_CTAB = 1 + KEY_CATAB = 1 + KEY_ENTER = 1 + KEY_SRESET = 1 + KEY_RESET = 1 + KEY_PRINT = 1 + KEY_LL = 1 + KEY_A1 = 1 + KEY_A3 = 1 + KEY_B2 = 1 + KEY_C1 = 1 + KEY_C3 = 1 + KEY_BTAB = 1 + KEY_BEG = 1 + KEY_CANCEL = 1 + KEY_CLOSE = 1 + KEY_COMMAND = 1 + KEY_COPY = 1 + KEY_CREATE = 1 + KEY_END = 1 + KEY_EXIT = 1 + KEY_FIND = 1 + KEY_HELP = 1 + KEY_MARK = 1 + KEY_MESSAGE = 1 + KEY_MOVE = 1 + KEY_NEXT = 1 + KEY_OPEN = 1 + KEY_OPTIONS = 1 + KEY_PREVIOUS = 1 + KEY_REDO = 1 + KEY_REFERENCE = 1 + KEY_REFRESH = 1 + KEY_REPLACE = 1 + KEY_RESTART = 1 + KEY_RESUME = 1 + KEY_SAVE = 1 + KEY_SBEG = 1 + KEY_SCANCEL = 1 + KEY_SCOMMAND = 1 + KEY_SCOPY = 1 + KEY_SCREATE = 1 + KEY_SDC = 1 + KEY_SDL = 1 + KEY_SELECT = 1 + KEY_SEND = 1 + KEY_SEOL = 1 + KEY_SEXIT = 1 + KEY_SFIND = 1 + KEY_SHELP = 1 + KEY_SHOME = 1 + KEY_SIC = 1 + KEY_SLEFT = 1 + KEY_SMESSAGE = 1 + KEY_SMOVE = 1 + KEY_SNEXT = 1 + KEY_SOPTIONS = 1 + KEY_SPREVIOUS = 1 + KEY_SPRINT = 1 + KEY_SREDO = 1 + KEY_SREPLACE = 1 + KEY_SRIGHT = 1 + KEY_SRSUME = 1 + KEY_SSAVE = 1 + KEY_SSUSPEND = 1 + KEY_SUNDO = 1 + KEY_SUSPEND = 1 + KEY_UNDO = 1 + KEY_MOUSE = 1 + KEY_RESIZE = 1 + KEY_MAX = 1 + ACS_BBSS = 1 + ACS_BLOCK = 1 + ACS_BOARD = 1 + ACS_BSBS = 1 + ACS_BSSB = 1 + ACS_BSSS = 1 + ACS_BTEE = 1 + ACS_BULLET = 1 + ACS_CKBOARD = 1 + ACS_DARROW = 1 + ACS_DEGREE = 1 + ACS_DIAMOND = 1 + ACS_GEQUAL = 1 + ACS_HLINE = 1 + ACS_LANTERN = 1 + ACS_LARROW = 1 + ACS_LEQUAL = 1 + ACS_LLCORNER = 1 + ACS_LRCORNER = 1 + ACS_LTEE = 1 + ACS_NEQUAL = 1 + ACS_PI = 1 + ACS_PLMINUS = 1 + ACS_PLUS = 1 + ACS_RARROW = 1 + ACS_RTEE = 1 + ACS_S1 = 1 + ACS_S3 = 1 + ACS_S7 = 1 + ACS_S9 = 1 + ACS_SBBS = 1 + ACS_SBSB = 1 + ACS_SBSS = 1 + ACS_SSBB = 1 + ACS_SSBS = 1 + ACS_SSSB = 1 + ACS_SSSS = 1 + ACS_STERLING = 1 + ACS_TTEE = 1 + ACS_UARROW = 1 + ACS_ULCORNER = 1 + ACS_URCORNER = 1 + ACS_VLINE = 1 + COLOR_BLACK = 1 + COLOR_BLUE = 1 + COLOR_CYAN = 1 + COLOR_GREEN = 1 + COLOR_MAGENTA = 1 + COLOR_RED = 1 + COLOR_WHITE = 1 + COLOR_YELLOW = 1 + """ + ) + + +astroid.register_module_extender(astroid.MANAGER, "curses", _curses_transform) diff --git a/env/lib/python3.8/site-packages/astroid/brain/brain_dataclasses.py b/env/lib/python3.8/site-packages/astroid/brain/brain_dataclasses.py new file mode 100644 index 000000000..7a25e0c63 --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/brain/brain_dataclasses.py @@ -0,0 +1,50 @@ +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER +""" +Astroid hook for the dataclasses library +""" + +import astroid +from astroid import MANAGER + + +DATACLASSES_DECORATORS = frozenset(("dataclasses.dataclass", "dataclass")) + + +def is_decorated_with_dataclass(node, decorator_names=DATACLASSES_DECORATORS): + """Return True if a decorated node has a `dataclass` decorator applied.""" + if not node.decorators: + return False + for decorator_attribute in node.decorators.nodes: + if isinstance(decorator_attribute, astroid.Call): # decorator with arguments + decorator_attribute = decorator_attribute.func + if decorator_attribute.as_string() in decorator_names: + return True + return False + + +def dataclass_transform(node): + """Rewrite a dataclass to be easily understood by pylint""" + + for assign_node in node.body: + if not isinstance(assign_node, (astroid.AnnAssign, astroid.Assign)): + continue + + targets = ( + assign_node.targets + if hasattr(assign_node, "targets") + else [assign_node.target] + ) + for target in targets: + rhs_node = astroid.Unknown( + lineno=assign_node.lineno, + col_offset=assign_node.col_offset, + parent=assign_node, + ) + node.instance_attrs[target.name] = [rhs_node] + node.locals[target.name] = [rhs_node] + + +MANAGER.register_transform( + astroid.ClassDef, dataclass_transform, is_decorated_with_dataclass +) diff --git a/env/lib/python3.8/site-packages/astroid/brain/brain_dateutil.py b/env/lib/python3.8/site-packages/astroid/brain/brain_dateutil.py new file mode 100644 index 000000000..9fdb9fde0 --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/brain/brain_dateutil.py @@ -0,0 +1,28 @@ +# Copyright (c) 2015-2016, 2018 Claudiu Popa +# Copyright (c) 2015 raylu +# Copyright (c) 2016 Ceridwen + +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER + +"""Astroid hooks for dateutil""" + +import textwrap + +from astroid import MANAGER, register_module_extender +from astroid.builder import AstroidBuilder + + +def dateutil_transform(): + return AstroidBuilder(MANAGER).string_build( + textwrap.dedent( + """ + import datetime + def parse(timestr, parserinfo=None, **kwargs): + return datetime.datetime() + """ + ) + ) + + +register_module_extender(MANAGER, "dateutil.parser", dateutil_transform) diff --git a/env/lib/python3.8/site-packages/astroid/brain/brain_fstrings.py b/env/lib/python3.8/site-packages/astroid/brain/brain_fstrings.py new file mode 100644 index 000000000..298d58afa --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/brain/brain_fstrings.py @@ -0,0 +1,51 @@ +# Copyright (c) 2017-2018 Claudiu Popa + +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER +import collections +import sys + +import astroid + + +def _clone_node_with_lineno(node, parent, lineno): + cls = node.__class__ + other_fields = node._other_fields + _astroid_fields = node._astroid_fields + init_params = {"lineno": lineno, "col_offset": node.col_offset, "parent": parent} + postinit_params = {param: getattr(node, param) for param in _astroid_fields} + if other_fields: + init_params.update({param: getattr(node, param) for param in other_fields}) + new_node = cls(**init_params) + if hasattr(node, "postinit") and _astroid_fields: + for param, child in postinit_params.items(): + if child and not isinstance(child, collections.Sequence): + cloned_child = _clone_node_with_lineno( + node=child, lineno=new_node.lineno, parent=new_node + ) + postinit_params[param] = cloned_child + new_node.postinit(**postinit_params) + return new_node + + +def _transform_formatted_value(node): + if node.value and node.value.lineno == 1: + if node.lineno != node.value.lineno: + new_node = astroid.FormattedValue( + lineno=node.lineno, col_offset=node.col_offset, parent=node.parent + ) + new_value = _clone_node_with_lineno( + node=node.value, lineno=node.lineno, parent=new_node + ) + new_node.postinit(value=new_value, format_spec=node.format_spec) + return new_node + + +if sys.version_info[:2] >= (3, 6): + # TODO: this fix tries to *patch* http://bugs.python.org/issue29051 + # The problem is that FormattedValue.value, which is a Name node, + # has wrong line numbers, usually 1. This creates problems for pylint, + # which expects correct line numbers for things such as message control. + astroid.MANAGER.register_transform( + astroid.FormattedValue, _transform_formatted_value + ) diff --git a/env/lib/python3.8/site-packages/astroid/brain/brain_functools.py b/env/lib/python3.8/site-packages/astroid/brain/brain_functools.py new file mode 100644 index 000000000..d6c60691d --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/brain/brain_functools.py @@ -0,0 +1,159 @@ +# Copyright (c) 2016, 2018-2020 Claudiu Popa +# Copyright (c) 2018 hippo91 +# Copyright (c) 2018 Bryce Guinta + +"""Astroid hooks for understanding functools library module.""" +from functools import partial +from itertools import chain + +import astroid +from astroid import arguments +from astroid import BoundMethod +from astroid import extract_node +from astroid import helpers +from astroid.interpreter import objectmodel +from astroid import MANAGER +from astroid import objects + + +LRU_CACHE = "functools.lru_cache" + + +class LruWrappedModel(objectmodel.FunctionModel): + """Special attribute model for functions decorated with functools.lru_cache. + + The said decorators patches at decoration time some functions onto + the decorated function. + """ + + @property + def attr___wrapped__(self): + return self._instance + + @property + def attr_cache_info(self): + cache_info = extract_node( + """ + from functools import _CacheInfo + _CacheInfo(0, 0, 0, 0) + """ + ) + + class CacheInfoBoundMethod(BoundMethod): + def infer_call_result(self, caller, context=None): + yield helpers.safe_infer(cache_info) + + return CacheInfoBoundMethod(proxy=self._instance, bound=self._instance) + + @property + def attr_cache_clear(self): + node = extract_node("""def cache_clear(self): pass""") + return BoundMethod(proxy=node, bound=self._instance.parent.scope()) + + +def _transform_lru_cache(node, context=None): + # TODO: this is not ideal, since the node should be immutable, + # but due to https://github.com/PyCQA/astroid/issues/354, + # there's not much we can do now. + # Replacing the node would work partially, because, + # in pylint, the old node would still be available, leading + # to spurious false positives. + node.special_attributes = LruWrappedModel()(node) + return + + +def _functools_partial_inference(node, context=None): + call = arguments.CallSite.from_call(node, context=context) + number_of_positional = len(call.positional_arguments) + if number_of_positional < 1: + raise astroid.UseInferenceDefault( + "functools.partial takes at least one argument" + ) + if number_of_positional == 1 and not call.keyword_arguments: + raise astroid.UseInferenceDefault( + "functools.partial needs at least to have some filled arguments" + ) + + partial_function = call.positional_arguments[0] + try: + inferred_wrapped_function = next(partial_function.infer(context=context)) + except astroid.InferenceError as exc: + raise astroid.UseInferenceDefault from exc + if inferred_wrapped_function is astroid.Uninferable: + raise astroid.UseInferenceDefault("Cannot infer the wrapped function") + if not isinstance(inferred_wrapped_function, astroid.FunctionDef): + raise astroid.UseInferenceDefault("The wrapped function is not a function") + + # Determine if the passed keywords into the callsite are supported + # by the wrapped function. + function_parameters = chain( + inferred_wrapped_function.args.args or (), + inferred_wrapped_function.args.posonlyargs or (), + inferred_wrapped_function.args.kwonlyargs or (), + ) + parameter_names = set( + param.name + for param in function_parameters + if isinstance(param, astroid.AssignName) + ) + if set(call.keyword_arguments) - parameter_names: + raise astroid.UseInferenceDefault( + "wrapped function received unknown parameters" + ) + + partial_function = objects.PartialFunction( + call, + name=inferred_wrapped_function.name, + doc=inferred_wrapped_function.doc, + lineno=inferred_wrapped_function.lineno, + col_offset=inferred_wrapped_function.col_offset, + parent=inferred_wrapped_function.parent, + ) + partial_function.postinit( + args=inferred_wrapped_function.args, + body=inferred_wrapped_function.body, + decorators=inferred_wrapped_function.decorators, + returns=inferred_wrapped_function.returns, + type_comment_returns=inferred_wrapped_function.type_comment_returns, + type_comment_args=inferred_wrapped_function.type_comment_args, + ) + return iter((partial_function,)) + + +def _looks_like_lru_cache(node): + """Check if the given function node is decorated with lru_cache.""" + if not node.decorators: + return False + for decorator in node.decorators.nodes: + if not isinstance(decorator, astroid.Call): + continue + if _looks_like_functools_member(decorator, "lru_cache"): + return True + return False + + +def _looks_like_functools_member(node, member): + """Check if the given Call node is a functools.partial call""" + if isinstance(node.func, astroid.Name): + return node.func.name == member + elif isinstance(node.func, astroid.Attribute): + return ( + node.func.attrname == member + and isinstance(node.func.expr, astroid.Name) + and node.func.expr.name == "functools" + ) + + +_looks_like_partial = partial(_looks_like_functools_member, member="partial") + + +MANAGER.register_transform( + astroid.FunctionDef, _transform_lru_cache, _looks_like_lru_cache +) + + +MANAGER.register_transform( + astroid.Call, + astroid.inference_tip(_functools_partial_inference), + _looks_like_partial, +) diff --git a/env/lib/python3.8/site-packages/astroid/brain/brain_gi.py b/env/lib/python3.8/site-packages/astroid/brain/brain_gi.py new file mode 100644 index 000000000..e49f3a22e --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/brain/brain_gi.py @@ -0,0 +1,253 @@ +# -*- coding: utf-8 -*- +# Copyright (c) 2013-2014 LOGILAB S.A. (Paris, FRANCE) +# Copyright (c) 2014 Google, Inc. +# Copyright (c) 2014 Cole Robinson +# Copyright (c) 2015-2016, 2018 Claudiu Popa +# Copyright (c) 2015-2016 Ceridwen +# Copyright (c) 2015 David Shea +# Copyright (c) 2016 Jakub Wilk +# Copyright (c) 2016 Giuseppe Scrivano +# Copyright (c) 2018 Christoph Reiter +# Copyright (c) 2019 Philipp Hörist + +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER + +"""Astroid hooks for the Python 2 GObject introspection bindings. + +Helps with understanding everything imported from 'gi.repository' +""" + +import inspect +import itertools +import sys +import re +import warnings + +from astroid import MANAGER, AstroidBuildingError, nodes +from astroid.builder import AstroidBuilder + + +_inspected_modules = {} + +_identifier_re = r"^[A-Za-z_]\w*$" + +_special_methods = frozenset( + { + "__lt__", + "__le__", + "__eq__", + "__ne__", + "__ge__", + "__gt__", + "__iter__", + "__getitem__", + "__setitem__", + "__delitem__", + "__len__", + "__bool__", + "__nonzero__", + "__next__", + "__str__", + "__len__", + "__contains__", + "__enter__", + "__exit__", + "__repr__", + "__getattr__", + "__setattr__", + "__delattr__", + "__del__", + "__hash__", + } +) + + +def _gi_build_stub(parent): + """ + Inspect the passed module recursively and build stubs for functions, + classes, etc. + """ + classes = {} + functions = {} + constants = {} + methods = {} + for name in dir(parent): + if name.startswith("__") and name not in _special_methods: + continue + + # Check if this is a valid name in python + if not re.match(_identifier_re, name): + continue + + try: + obj = getattr(parent, name) + except: + continue + + if inspect.isclass(obj): + classes[name] = obj + elif inspect.isfunction(obj) or inspect.isbuiltin(obj): + functions[name] = obj + elif inspect.ismethod(obj) or inspect.ismethoddescriptor(obj): + methods[name] = obj + elif ( + str(obj).startswith(", ) + # Only accept function calls with two constant arguments + if len(node.args) != 2: + return False + + if not all(isinstance(arg, nodes.Const) for arg in node.args): + return False + + func = node.func + if isinstance(func, nodes.Attribute): + if func.attrname != "require_version": + return False + if isinstance(func.expr, nodes.Name) and func.expr.name == "gi": + return True + + return False + + if isinstance(func, nodes.Name): + return func.name == "require_version" + + return False + + +def _register_require_version(node): + # Load the gi.require_version locally + try: + import gi + + gi.require_version(node.args[0].value, node.args[1].value) + except Exception: + pass + + return node + + +MANAGER.register_failed_import_hook(_import_gi_module) +MANAGER.register_transform( + nodes.Call, _register_require_version, _looks_like_require_version +) diff --git a/env/lib/python3.8/site-packages/astroid/brain/brain_hashlib.py b/env/lib/python3.8/site-packages/astroid/brain/brain_hashlib.py new file mode 100644 index 000000000..eb34e1590 --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/brain/brain_hashlib.py @@ -0,0 +1,69 @@ +# Copyright (c) 2016, 2018 Claudiu Popa +# Copyright (c) 2018 David Poirier +# Copyright (c) 2018 wgehalo +# Copyright (c) 2018 Ioana Tagirta + +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER +import sys + +import six + +import astroid + +PY36 = sys.version_info >= (3, 6) + + +def _hashlib_transform(): + signature = "value=''" + template = """ + class %(name)s(object): + def __init__(self, %(signature)s): pass + def digest(self): + return %(digest)s + def copy(self): + return self + def update(self, value): pass + def hexdigest(self): + return '' + @property + def name(self): + return %(name)r + @property + def block_size(self): + return 1 + @property + def digest_size(self): + return 1 + """ + algorithms_with_signature = dict.fromkeys( + ["md5", "sha1", "sha224", "sha256", "sha384", "sha512"], signature + ) + if PY36: + blake2b_signature = "data=b'', *, digest_size=64, key=b'', salt=b'', \ + person=b'', fanout=1, depth=1, leaf_size=0, node_offset=0, \ + node_depth=0, inner_size=0, last_node=False" + blake2s_signature = "data=b'', *, digest_size=32, key=b'', salt=b'', \ + person=b'', fanout=1, depth=1, leaf_size=0, node_offset=0, \ + node_depth=0, inner_size=0, last_node=False" + new_algorithms = dict.fromkeys( + ["sha3_224", "sha3_256", "sha3_384", "sha3_512", "shake_128", "shake_256"], + signature, + ) + algorithms_with_signature.update(new_algorithms) + algorithms_with_signature.update( + {"blake2b": blake2b_signature, "blake2s": blake2s_signature} + ) + classes = "".join( + template + % { + "name": hashfunc, + "digest": 'b""' if six.PY3 else '""', + "signature": signature, + } + for hashfunc, signature in algorithms_with_signature.items() + ) + return astroid.parse(classes) + + +astroid.register_module_extender(astroid.MANAGER, "hashlib", _hashlib_transform) diff --git a/env/lib/python3.8/site-packages/astroid/brain/brain_http.py b/env/lib/python3.8/site-packages/astroid/brain/brain_http.py new file mode 100644 index 000000000..b16464e8e --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/brain/brain_http.py @@ -0,0 +1,211 @@ +# Copyright (c) 2018-2019 Claudiu Popa + +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER + +"""Astroid brain hints for some of the `http` module.""" +import textwrap + +import astroid +from astroid.builder import AstroidBuilder + + +def _http_transform(): + code = textwrap.dedent( + """ + from collections import namedtuple + _HTTPStatus = namedtuple('_HTTPStatus', 'value phrase description') + + class HTTPStatus: + + @property + def phrase(self): + return "" + @property + def value(self): + return 0 + @property + def description(self): + return "" + + # informational + CONTINUE = _HTTPStatus(100, 'Continue', 'Request received, please continue') + SWITCHING_PROTOCOLS = _HTTPStatus(101, 'Switching Protocols', + 'Switching to new protocol; obey Upgrade header') + PROCESSING = _HTTPStatus(102, 'Processing', '') + OK = _HTTPStatus(200, 'OK', 'Request fulfilled, document follows') + CREATED = _HTTPStatus(201, 'Created', 'Document created, URL follows') + ACCEPTED = _HTTPStatus(202, 'Accepted', + 'Request accepted, processing continues off-line') + NON_AUTHORITATIVE_INFORMATION = _HTTPStatus(203, + 'Non-Authoritative Information', 'Request fulfilled from cache') + NO_CONTENT = _HTTPStatus(204, 'No Content', 'Request fulfilled, nothing follows') + RESET_CONTENT =_HTTPStatus(205, 'Reset Content', 'Clear input form for further input') + PARTIAL_CONTENT = _HTTPStatus(206, 'Partial Content', 'Partial content follows') + MULTI_STATUS = _HTTPStatus(207, 'Multi-Status', '') + ALREADY_REPORTED = _HTTPStatus(208, 'Already Reported', '') + IM_USED = _HTTPStatus(226, 'IM Used', '') + MULTIPLE_CHOICES = _HTTPStatus(300, 'Multiple Choices', + 'Object has several resources -- see URI list') + MOVED_PERMANENTLY = _HTTPStatus(301, 'Moved Permanently', + 'Object moved permanently -- see URI list') + FOUND = _HTTPStatus(302, 'Found', 'Object moved temporarily -- see URI list') + SEE_OTHER = _HTTPStatus(303, 'See Other', 'Object moved -- see Method and URL list') + NOT_MODIFIED = _HTTPStatus(304, 'Not Modified', + 'Document has not changed since given time') + USE_PROXY = _HTTPStatus(305, 'Use Proxy', + 'You must use proxy specified in Location to access this resource') + TEMPORARY_REDIRECT = _HTTPStatus(307, 'Temporary Redirect', + 'Object moved temporarily -- see URI list') + PERMANENT_REDIRECT = _HTTPStatus(308, 'Permanent Redirect', + 'Object moved permanently -- see URI list') + BAD_REQUEST = _HTTPStatus(400, 'Bad Request', + 'Bad request syntax or unsupported method') + UNAUTHORIZED = _HTTPStatus(401, 'Unauthorized', + 'No permission -- see authorization schemes') + PAYMENT_REQUIRED = _HTTPStatus(402, 'Payment Required', + 'No payment -- see charging schemes') + FORBIDDEN = _HTTPStatus(403, 'Forbidden', + 'Request forbidden -- authorization will not help') + NOT_FOUND = _HTTPStatus(404, 'Not Found', + 'Nothing matches the given URI') + METHOD_NOT_ALLOWED = _HTTPStatus(405, 'Method Not Allowed', + 'Specified method is invalid for this resource') + NOT_ACCEPTABLE = _HTTPStatus(406, 'Not Acceptable', + 'URI not available in preferred format') + PROXY_AUTHENTICATION_REQUIRED = _HTTPStatus(407, + 'Proxy Authentication Required', + 'You must authenticate with this proxy before proceeding') + REQUEST_TIMEOUT = _HTTPStatus(408, 'Request Timeout', + 'Request timed out; try again later') + CONFLICT = _HTTPStatus(409, 'Conflict', 'Request conflict') + GONE = _HTTPStatus(410, 'Gone', + 'URI no longer exists and has been permanently removed') + LENGTH_REQUIRED = _HTTPStatus(411, 'Length Required', + 'Client must specify Content-Length') + PRECONDITION_FAILED = _HTTPStatus(412, 'Precondition Failed', + 'Precondition in headers is false') + REQUEST_ENTITY_TOO_LARGE = _HTTPStatus(413, 'Request Entity Too Large', + 'Entity is too large') + REQUEST_URI_TOO_LONG = _HTTPStatus(414, 'Request-URI Too Long', + 'URI is too long') + UNSUPPORTED_MEDIA_TYPE = _HTTPStatus(415, 'Unsupported Media Type', + 'Entity body in unsupported format') + REQUESTED_RANGE_NOT_SATISFIABLE = _HTTPStatus(416, + 'Requested Range Not Satisfiable', + 'Cannot satisfy request range') + EXPECTATION_FAILED = _HTTPStatus(417, 'Expectation Failed', + 'Expect condition could not be satisfied') + MISDIRECTED_REQUEST = _HTTPStatus(421, 'Misdirected Request', + 'Server is not able to produce a response') + UNPROCESSABLE_ENTITY = _HTTPStatus(422, 'Unprocessable Entity') + LOCKED = _HTTPStatus(423, 'Locked') + FAILED_DEPENDENCY = _HTTPStatus(424, 'Failed Dependency') + UPGRADE_REQUIRED = _HTTPStatus(426, 'Upgrade Required') + PRECONDITION_REQUIRED = _HTTPStatus(428, 'Precondition Required', + 'The origin server requires the request to be conditional') + TOO_MANY_REQUESTS = _HTTPStatus(429, 'Too Many Requests', + 'The user has sent too many requests in ' + 'a given amount of time ("rate limiting")') + REQUEST_HEADER_FIELDS_TOO_LARGE = _HTTPStatus(431, + 'Request Header Fields Too Large', + 'The server is unwilling to process the request because its header ' + 'fields are too large') + UNAVAILABLE_FOR_LEGAL_REASONS = _HTTPStatus(451, + 'Unavailable For Legal Reasons', + 'The server is denying access to the ' + 'resource as a consequence of a legal demand') + INTERNAL_SERVER_ERROR = _HTTPStatus(500, 'Internal Server Error', + 'Server got itself in trouble') + NOT_IMPLEMENTED = _HTTPStatus(501, 'Not Implemented', + 'Server does not support this operation') + BAD_GATEWAY = _HTTPStatus(502, 'Bad Gateway', + 'Invalid responses from another server/proxy') + SERVICE_UNAVAILABLE = _HTTPStatus(503, 'Service Unavailable', + 'The server cannot process the request due to a high load') + GATEWAY_TIMEOUT = _HTTPStatus(504, 'Gateway Timeout', + 'The gateway server did not receive a timely response') + HTTP_VERSION_NOT_SUPPORTED = _HTTPStatus(505, 'HTTP Version Not Supported', + 'Cannot fulfill request') + VARIANT_ALSO_NEGOTIATES = _HTTPStatus(506, 'Variant Also Negotiates') + INSUFFICIENT_STORAGE = _HTTPStatus(507, 'Insufficient Storage') + LOOP_DETECTED = _HTTPStatus(508, 'Loop Detected') + NOT_EXTENDED = _HTTPStatus(510, 'Not Extended') + NETWORK_AUTHENTICATION_REQUIRED = _HTTPStatus(511, + 'Network Authentication Required', + 'The client needs to authenticate to gain network access') + """ + ) + return AstroidBuilder(astroid.MANAGER).string_build(code) + + +def _http_client_transform(): + return AstroidBuilder(astroid.MANAGER).string_build( + textwrap.dedent( + """ + from http import HTTPStatus + + CONTINUE = HTTPStatus.CONTINUE + SWITCHING_PROTOCOLS = HTTPStatus.SWITCHING_PROTOCOLS + PROCESSING = HTTPStatus.PROCESSING + OK = HTTPStatus.OK + CREATED = HTTPStatus.CREATED + ACCEPTED = HTTPStatus.ACCEPTED + NON_AUTHORITATIVE_INFORMATION = HTTPStatus.NON_AUTHORITATIVE_INFORMATION + NO_CONTENT = HTTPStatus.NO_CONTENT + RESET_CONTENT = HTTPStatus.RESET_CONTENT + PARTIAL_CONTENT = HTTPStatus.PARTIAL_CONTENT + MULTI_STATUS = HTTPStatus.MULTI_STATUS + ALREADY_REPORTED = HTTPStatus.ALREADY_REPORTED + IM_USED = HTTPStatus.IM_USED + MULTIPLE_CHOICES = HTTPStatus.MULTIPLE_CHOICES + MOVED_PERMANENTLY = HTTPStatus.MOVED_PERMANENTLY + FOUND = HTTPStatus.FOUND + SEE_OTHER = HTTPStatus.SEE_OTHER + NOT_MODIFIED = HTTPStatus.NOT_MODIFIED + USE_PROXY = HTTPStatus.USE_PROXY + TEMPORARY_REDIRECT = HTTPStatus.TEMPORARY_REDIRECT + PERMANENT_REDIRECT = HTTPStatus.PERMANENT_REDIRECT + BAD_REQUEST = HTTPStatus.BAD_REQUEST + UNAUTHORIZED = HTTPStatus.UNAUTHORIZED + PAYMENT_REQUIRED = HTTPStatus.PAYMENT_REQUIRED + FORBIDDEN = HTTPStatus.FORBIDDEN + NOT_FOUND = HTTPStatus.NOT_FOUND + METHOD_NOT_ALLOWED = HTTPStatus.METHOD_NOT_ALLOWED + NOT_ACCEPTABLE = HTTPStatus.NOT_ACCEPTABLE + PROXY_AUTHENTICATION_REQUIRED = HTTPStatus.PROXY_AUTHENTICATION_REQUIRED + REQUEST_TIMEOUT = HTTPStatus.REQUEST_TIMEOUT + CONFLICT = HTTPStatus.CONFLICT + GONE = HTTPStatus.GONE + LENGTH_REQUIRED = HTTPStatus.LENGTH_REQUIRED + PRECONDITION_FAILED = HTTPStatus.PRECONDITION_FAILED + REQUEST_ENTITY_TOO_LARGE = HTTPStatus.REQUEST_ENTITY_TOO_LARGE + REQUEST_URI_TOO_LONG = HTTPStatus.REQUEST_URI_TOO_LONG + UNSUPPORTED_MEDIA_TYPE = HTTPStatus.UNSUPPORTED_MEDIA_TYPE + REQUESTED_RANGE_NOT_SATISFIABLE = HTTPStatus.REQUESTED_RANGE_NOT_SATISFIABLE + EXPECTATION_FAILED = HTTPStatus.EXPECTATION_FAILED + UNPROCESSABLE_ENTITY = HTTPStatus.UNPROCESSABLE_ENTITY + LOCKED = HTTPStatus.LOCKED + FAILED_DEPENDENCY = HTTPStatus.FAILED_DEPENDENCY + UPGRADE_REQUIRED = HTTPStatus.UPGRADE_REQUIRED + PRECONDITION_REQUIRED = HTTPStatus.PRECONDITION_REQUIRED + TOO_MANY_REQUESTS = HTTPStatus.TOO_MANY_REQUESTS + REQUEST_HEADER_FIELDS_TOO_LARGE = HTTPStatus.REQUEST_HEADER_FIELDS_TOO_LARGE + INTERNAL_SERVER_ERROR = HTTPStatus.INTERNAL_SERVER_ERROR + NOT_IMPLEMENTED = HTTPStatus.NOT_IMPLEMENTED + BAD_GATEWAY = HTTPStatus.BAD_GATEWAY + SERVICE_UNAVAILABLE = HTTPStatus.SERVICE_UNAVAILABLE + GATEWAY_TIMEOUT = HTTPStatus.GATEWAY_TIMEOUT + HTTP_VERSION_NOT_SUPPORTED = HTTPStatus.HTTP_VERSION_NOT_SUPPORTED + VARIANT_ALSO_NEGOTIATES = HTTPStatus.VARIANT_ALSO_NEGOTIATES + INSUFFICIENT_STORAGE = HTTPStatus.INSUFFICIENT_STORAGE + LOOP_DETECTED = HTTPStatus.LOOP_DETECTED + NOT_EXTENDED = HTTPStatus.NOT_EXTENDED + NETWORK_AUTHENTICATION_REQUIRED = HTTPStatus.NETWORK_AUTHENTICATION_REQUIRED + """ + ) + ) + + +astroid.register_module_extender(astroid.MANAGER, "http", _http_transform) +astroid.register_module_extender(astroid.MANAGER, "http.client", _http_client_transform) diff --git a/env/lib/python3.8/site-packages/astroid/brain/brain_io.py b/env/lib/python3.8/site-packages/astroid/brain/brain_io.py new file mode 100644 index 000000000..c74531129 --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/brain/brain_io.py @@ -0,0 +1,45 @@ +# Copyright (c) 2016, 2018 Claudiu Popa + +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER + +"""Astroid brain hints for some of the _io C objects.""" + +import astroid + + +BUFFERED = {"BufferedWriter", "BufferedReader"} +TextIOWrapper = "TextIOWrapper" +FileIO = "FileIO" +BufferedWriter = "BufferedWriter" + + +def _generic_io_transform(node, name, cls): + """Transform the given name, by adding the given *class* as a member of the node.""" + + io_module = astroid.MANAGER.ast_from_module_name("_io") + attribute_object = io_module[cls] + instance = attribute_object.instantiate_class() + node.locals[name] = [instance] + + +def _transform_text_io_wrapper(node): + # This is not always correct, since it can vary with the type of the descriptor, + # being stdout, stderr or stdin. But we cannot get access to the name of the + # stream, which is why we are using the BufferedWriter class as a default + # value + return _generic_io_transform(node, name="buffer", cls=BufferedWriter) + + +def _transform_buffered(node): + return _generic_io_transform(node, name="raw", cls=FileIO) + + +astroid.MANAGER.register_transform( + astroid.ClassDef, _transform_buffered, lambda node: node.name in BUFFERED +) +astroid.MANAGER.register_transform( + astroid.ClassDef, + _transform_text_io_wrapper, + lambda node: node.name == TextIOWrapper, +) diff --git a/env/lib/python3.8/site-packages/astroid/brain/brain_mechanize.py b/env/lib/python3.8/site-packages/astroid/brain/brain_mechanize.py new file mode 100644 index 000000000..ef62c53bc --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/brain/brain_mechanize.py @@ -0,0 +1,29 @@ +# Copyright (c) 2012-2013 LOGILAB S.A. (Paris, FRANCE) +# Copyright (c) 2014 Google, Inc. +# Copyright (c) 2015-2016, 2018 Claudiu Popa +# Copyright (c) 2016 Ceridwen + +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER + +from astroid import MANAGER, register_module_extender +from astroid.builder import AstroidBuilder + + +def mechanize_transform(): + return AstroidBuilder(MANAGER).string_build( + """ + +class Browser(object): + def open(self, url, data=None, timeout=None): + return None + def open_novisit(self, url, data=None, timeout=None): + return None + def open_local_file(self, filename): + return None + +""" + ) + + +register_module_extender(MANAGER, "mechanize", mechanize_transform) diff --git a/env/lib/python3.8/site-packages/astroid/brain/brain_multiprocessing.py b/env/lib/python3.8/site-packages/astroid/brain/brain_multiprocessing.py new file mode 100644 index 000000000..3629b0321 --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/brain/brain_multiprocessing.py @@ -0,0 +1,107 @@ +# Copyright (c) 2016, 2018 Claudiu Popa +# Copyright (c) 2019 Hugo van Kemenade + +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER + +import sys + +import astroid +from astroid import exceptions + + +def _multiprocessing_transform(): + module = astroid.parse( + """ + from multiprocessing.managers import SyncManager + def Manager(): + return SyncManager() + """ + ) + # Multiprocessing uses a getattr lookup inside contexts, + # in order to get the attributes they need. Since it's extremely + # dynamic, we use this approach to fake it. + node = astroid.parse( + """ + from multiprocessing.context import DefaultContext, BaseContext + default = DefaultContext() + base = BaseContext() + """ + ) + try: + context = next(node["default"].infer()) + base = next(node["base"].infer()) + except exceptions.InferenceError: + return module + + for node in (context, base): + for key, value in node.locals.items(): + if key.startswith("_"): + continue + + value = value[0] + if isinstance(value, astroid.FunctionDef): + # We need to rebound this, since otherwise + # it will have an extra argument (self). + value = astroid.BoundMethod(value, node) + module[key] = value + return module + + +def _multiprocessing_managers_transform(): + return astroid.parse( + """ + import array + import threading + import multiprocessing.pool as pool + + import six + + class Namespace(object): + pass + + class Value(object): + def __init__(self, typecode, value, lock=True): + self._typecode = typecode + self._value = value + def get(self): + return self._value + def set(self, value): + self._value = value + def __repr__(self): + return '%s(%r, %r)'%(type(self).__name__, self._typecode, self._value) + value = property(get, set) + + def Array(typecode, sequence, lock=True): + return array.array(typecode, sequence) + + class SyncManager(object): + Queue = JoinableQueue = six.moves.queue.Queue + Event = threading.Event + RLock = threading.RLock + BoundedSemaphore = threading.BoundedSemaphore + Condition = threading.Condition + Barrier = threading.Barrier + Pool = pool.Pool + list = list + dict = dict + Value = Value + Array = Array + Namespace = Namespace + __enter__ = lambda self: self + __exit__ = lambda *args: args + + def start(self, initializer=None, initargs=None): + pass + def shutdown(self): + pass + """ + ) + + +astroid.register_module_extender( + astroid.MANAGER, "multiprocessing.managers", _multiprocessing_managers_transform +) +astroid.register_module_extender( + astroid.MANAGER, "multiprocessing", _multiprocessing_transform +) diff --git a/env/lib/python3.8/site-packages/astroid/brain/brain_namedtuple_enum.py b/env/lib/python3.8/site-packages/astroid/brain/brain_namedtuple_enum.py new file mode 100644 index 000000000..13fcf793f --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/brain/brain_namedtuple_enum.py @@ -0,0 +1,455 @@ +# -*- coding: utf-8 -*- +# Copyright (c) 2012-2015 LOGILAB S.A. (Paris, FRANCE) +# Copyright (c) 2013-2014 Google, Inc. +# Copyright (c) 2014-2020 Claudiu Popa +# Copyright (c) 2014 Eevee (Alex Munroe) +# Copyright (c) 2015-2016 Ceridwen +# Copyright (c) 2015 Dmitry Pribysh +# Copyright (c) 2015 David Shea +# Copyright (c) 2015 Philip Lorenz +# Copyright (c) 2016 Jakub Wilk +# Copyright (c) 2016 Mateusz Bysiek +# Copyright (c) 2017 Hugo +# Copyright (c) 2017 Łukasz Rogalski +# Copyright (c) 2018 Ville Skyttä +# Copyright (c) 2019 Ashley Whetter + +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER + +"""Astroid hooks for the Python standard library.""" + +import functools +import keyword +from textwrap import dedent + +from astroid import MANAGER, UseInferenceDefault, inference_tip, InferenceError +from astroid import arguments +from astroid import exceptions +from astroid import nodes +from astroid.builder import AstroidBuilder, extract_node +from astroid import util + + +TYPING_NAMEDTUPLE_BASENAMES = {"NamedTuple", "typing.NamedTuple"} +ENUM_BASE_NAMES = { + "Enum", + "IntEnum", + "enum.Enum", + "enum.IntEnum", + "IntFlag", + "enum.IntFlag", +} + + +def _infer_first(node, context): + if node is util.Uninferable: + raise UseInferenceDefault + try: + value = next(node.infer(context=context)) + if value is util.Uninferable: + raise UseInferenceDefault() + else: + return value + except StopIteration: + raise InferenceError() + + +def _find_func_form_arguments(node, context): + def _extract_namedtuple_arg_or_keyword(position, key_name=None): + + if len(args) > position: + return _infer_first(args[position], context) + if key_name and key_name in found_keywords: + return _infer_first(found_keywords[key_name], context) + + args = node.args + keywords = node.keywords + found_keywords = ( + {keyword.arg: keyword.value for keyword in keywords} if keywords else {} + ) + + name = _extract_namedtuple_arg_or_keyword(position=0, key_name="typename") + names = _extract_namedtuple_arg_or_keyword(position=1, key_name="field_names") + if name and names: + return name.value, names + + raise UseInferenceDefault() + + +def infer_func_form(node, base_type, context=None, enum=False): + """Specific inference function for namedtuple or Python 3 enum. """ + # node is a Call node, class name as first argument and generated class + # attributes as second argument + + # namedtuple or enums list of attributes can be a list of strings or a + # whitespace-separate string + try: + name, names = _find_func_form_arguments(node, context) + try: + attributes = names.value.replace(",", " ").split() + except AttributeError: + if not enum: + attributes = [ + _infer_first(const, context).value for const in names.elts + ] + else: + # Enums supports either iterator of (name, value) pairs + # or mappings. + if hasattr(names, "items") and isinstance(names.items, list): + attributes = [ + _infer_first(const[0], context).value + for const in names.items + if isinstance(const[0], nodes.Const) + ] + elif hasattr(names, "elts"): + # Enums can support either ["a", "b", "c"] + # or [("a", 1), ("b", 2), ...], but they can't + # be mixed. + if all(isinstance(const, nodes.Tuple) for const in names.elts): + attributes = [ + _infer_first(const.elts[0], context).value + for const in names.elts + if isinstance(const, nodes.Tuple) + ] + else: + attributes = [ + _infer_first(const, context).value for const in names.elts + ] + else: + raise AttributeError + if not attributes: + raise AttributeError + except (AttributeError, exceptions.InferenceError): + raise UseInferenceDefault() + + attributes = [attr for attr in attributes if " " not in attr] + + # If we can't infer the name of the class, don't crash, up to this point + # we know it is a namedtuple anyway. + name = name or "Uninferable" + # we want to return a Class node instance with proper attributes set + class_node = nodes.ClassDef(name, "docstring") + class_node.parent = node.parent + # set base class=tuple + class_node.bases.append(base_type) + # XXX add __init__(*attributes) method + for attr in attributes: + fake_node = nodes.EmptyNode() + fake_node.parent = class_node + fake_node.attrname = attr + class_node.instance_attrs[attr] = [fake_node] + return class_node, name, attributes + + +def _has_namedtuple_base(node): + """Predicate for class inference tip + + :type node: ClassDef + :rtype: bool + """ + return set(node.basenames) & TYPING_NAMEDTUPLE_BASENAMES + + +def _looks_like(node, name): + func = node.func + if isinstance(func, nodes.Attribute): + return func.attrname == name + if isinstance(func, nodes.Name): + return func.name == name + return False + + +_looks_like_namedtuple = functools.partial(_looks_like, name="namedtuple") +_looks_like_enum = functools.partial(_looks_like, name="Enum") +_looks_like_typing_namedtuple = functools.partial(_looks_like, name="NamedTuple") + + +def infer_named_tuple(node, context=None): + """Specific inference function for namedtuple Call node""" + tuple_base_name = nodes.Name(name="tuple", parent=node.root()) + class_node, name, attributes = infer_func_form( + node, tuple_base_name, context=context + ) + call_site = arguments.CallSite.from_call(node, context=context) + func = next(extract_node("import collections; collections.namedtuple").infer()) + try: + rename = next(call_site.infer_argument(func, "rename", context)).bool_value() + except InferenceError: + rename = False + + if rename: + attributes = _get_renamed_namedtuple_attributes(attributes) + + replace_args = ", ".join("{arg}=None".format(arg=arg) for arg in attributes) + field_def = ( + " {name} = property(lambda self: self[{index:d}], " + "doc='Alias for field number {index:d}')" + ) + field_defs = "\n".join( + field_def.format(name=name, index=index) + for index, name in enumerate(attributes) + ) + fake = AstroidBuilder(MANAGER).string_build( + """ +class %(name)s(tuple): + __slots__ = () + _fields = %(fields)r + def _asdict(self): + return self.__dict__ + @classmethod + def _make(cls, iterable, new=tuple.__new__, len=len): + return new(cls, iterable) + def _replace(self, %(replace_args)s): + return self + def __getnewargs__(self): + return tuple(self) +%(field_defs)s + """ + % { + "name": name, + "fields": attributes, + "field_defs": field_defs, + "replace_args": replace_args, + } + ) + class_node.locals["_asdict"] = fake.body[0].locals["_asdict"] + class_node.locals["_make"] = fake.body[0].locals["_make"] + class_node.locals["_replace"] = fake.body[0].locals["_replace"] + class_node.locals["_fields"] = fake.body[0].locals["_fields"] + for attr in attributes: + class_node.locals[attr] = fake.body[0].locals[attr] + # we use UseInferenceDefault, we can't be a generator so return an iterator + return iter([class_node]) + + +def _get_renamed_namedtuple_attributes(field_names): + names = list(field_names) + seen = set() + for i, name in enumerate(field_names): + if ( + not all(c.isalnum() or c == "_" for c in name) + or keyword.iskeyword(name) + or not name + or name[0].isdigit() + or name.startswith("_") + or name in seen + ): + names[i] = "_%d" % i + seen.add(name) + return tuple(names) + + +def infer_enum(node, context=None): + """ Specific inference function for enum Call node. """ + enum_meta = extract_node( + """ + class EnumMeta(object): + 'docstring' + def __call__(self, node): + class EnumAttribute(object): + name = '' + value = 0 + return EnumAttribute() + def __iter__(self): + class EnumAttribute(object): + name = '' + value = 0 + return [EnumAttribute()] + def __reversed__(self): + class EnumAttribute(object): + name = '' + value = 0 + return (EnumAttribute, ) + def __next__(self): + return next(iter(self)) + def __getitem__(self, attr): + class Value(object): + @property + def name(self): + return '' + @property + def value(self): + return attr + + return Value() + __members__ = [''] + """ + ) + class_node = infer_func_form(node, enum_meta, context=context, enum=True)[0] + return iter([class_node.instantiate_class()]) + + +INT_FLAG_ADDITION_METHODS = """ + def __or__(self, other): + return {name}(self.value | other.value) + def __and__(self, other): + return {name}(self.value & other.value) + def __xor__(self, other): + return {name}(self.value ^ other.value) + def __add__(self, other): + return {name}(self.value + other.value) + def __div__(self, other): + return {name}(self.value / other.value) + def __invert__(self): + return {name}(~self.value) + def __mul__(self, other): + return {name}(self.value * other.value) +""" + + +def infer_enum_class(node): + """ Specific inference for enums. """ + for basename in node.basenames: + # TODO: doesn't handle subclasses yet. This implementation + # is a hack to support enums. + if basename not in ENUM_BASE_NAMES: + continue + if node.root().name == "enum": + # Skip if the class is directly from enum module. + break + for local, values in node.locals.items(): + if any(not isinstance(value, nodes.AssignName) for value in values): + continue + + targets = [] + stmt = values[0].statement() + if isinstance(stmt, nodes.Assign): + if isinstance(stmt.targets[0], nodes.Tuple): + targets = stmt.targets[0].itered() + else: + targets = stmt.targets + elif isinstance(stmt, nodes.AnnAssign): + targets = [stmt.target] + else: + continue + + inferred_return_value = None + if isinstance(stmt, nodes.Assign): + if isinstance(stmt.value, nodes.Const): + if isinstance(stmt.value.value, str): + inferred_return_value = repr(stmt.value.value) + else: + inferred_return_value = stmt.value.value + else: + inferred_return_value = stmt.value.as_string() + + new_targets = [] + for target in targets: + # Replace all the assignments with our mocked class. + classdef = dedent( + """ + class {name}({types}): + @property + def value(self): + return {return_value} + @property + def name(self): + return "{name}" + """.format( + name=target.name, + types=", ".join(node.basenames), + return_value=inferred_return_value, + ) + ) + if "IntFlag" in basename: + # Alright, we need to add some additional methods. + # Unfortunately we still can't infer the resulting objects as + # Enum members, but once we'll be able to do that, the following + # should result in some nice symbolic execution + classdef += INT_FLAG_ADDITION_METHODS.format(name=target.name) + + fake = AstroidBuilder(MANAGER).string_build(classdef)[target.name] + fake.parent = target.parent + for method in node.mymethods(): + fake.locals[method.name] = [method] + new_targets.append(fake.instantiate_class()) + node.locals[local] = new_targets + break + return node + + +def infer_typing_namedtuple_class(class_node, context=None): + """Infer a subclass of typing.NamedTuple""" + # Check if it has the corresponding bases + annassigns_fields = [ + annassign.target.name + for annassign in class_node.body + if isinstance(annassign, nodes.AnnAssign) + ] + code = dedent( + """ + from collections import namedtuple + namedtuple({typename!r}, {fields!r}) + """ + ).format(typename=class_node.name, fields=",".join(annassigns_fields)) + node = extract_node(code) + generated_class_node = next(infer_named_tuple(node, context)) + for method in class_node.mymethods(): + generated_class_node.locals[method.name] = [method] + + for assign in class_node.body: + if not isinstance(assign, nodes.Assign): + continue + + for target in assign.targets: + attr = target.name + generated_class_node.locals[attr] = class_node.locals[attr] + + return iter((generated_class_node,)) + + +def infer_typing_namedtuple(node, context=None): + """Infer a typing.NamedTuple(...) call.""" + # This is essentially a namedtuple with different arguments + # so we extract the args and infer a named tuple. + try: + func = next(node.func.infer()) + except InferenceError: + raise UseInferenceDefault + + if func.qname() != "typing.NamedTuple": + raise UseInferenceDefault + + if len(node.args) != 2: + raise UseInferenceDefault + + if not isinstance(node.args[1], (nodes.List, nodes.Tuple)): + raise UseInferenceDefault + + names = [] + for elt in node.args[1].elts: + if not isinstance(elt, (nodes.List, nodes.Tuple)): + raise UseInferenceDefault + if len(elt.elts) != 2: + raise UseInferenceDefault + names.append(elt.elts[0].as_string()) + + typename = node.args[0].as_string() + if names: + field_names = "({},)".format(",".join(names)) + else: + field_names = "''" + node = extract_node( + "namedtuple({typename}, {fields})".format(typename=typename, fields=field_names) + ) + return infer_named_tuple(node, context) + + +MANAGER.register_transform( + nodes.Call, inference_tip(infer_named_tuple), _looks_like_namedtuple +) +MANAGER.register_transform(nodes.Call, inference_tip(infer_enum), _looks_like_enum) +MANAGER.register_transform( + nodes.ClassDef, + infer_enum_class, + predicate=lambda cls: any( + basename for basename in cls.basenames if basename in ENUM_BASE_NAMES + ), +) +MANAGER.register_transform( + nodes.ClassDef, inference_tip(infer_typing_namedtuple_class), _has_namedtuple_base +) +MANAGER.register_transform( + nodes.Call, inference_tip(infer_typing_namedtuple), _looks_like_typing_namedtuple +) diff --git a/env/lib/python3.8/site-packages/astroid/brain/brain_nose.py b/env/lib/python3.8/site-packages/astroid/brain/brain_nose.py new file mode 100644 index 000000000..d0280a3ff --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/brain/brain_nose.py @@ -0,0 +1,77 @@ +# Copyright (c) 2015-2016, 2018 Claudiu Popa +# Copyright (c) 2016 Ceridwen + +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER + + +"""Hooks for nose library.""" + +import re +import textwrap + +import astroid +import astroid.builder + +_BUILDER = astroid.builder.AstroidBuilder(astroid.MANAGER) + + +def _pep8(name, caps=re.compile("([A-Z])")): + return caps.sub(lambda m: "_" + m.groups()[0].lower(), name) + + +def _nose_tools_functions(): + """Get an iterator of names and bound methods.""" + module = _BUILDER.string_build( + textwrap.dedent( + """ + import unittest + + class Test(unittest.TestCase): + pass + a = Test() + """ + ) + ) + try: + case = next(module["a"].infer()) + except astroid.InferenceError: + return + for method in case.methods(): + if method.name.startswith("assert") and "_" not in method.name: + pep8_name = _pep8(method.name) + yield pep8_name, astroid.BoundMethod(method, case) + if method.name == "assertEqual": + # nose also exports assert_equals. + yield "assert_equals", astroid.BoundMethod(method, case) + + +def _nose_tools_transform(node): + for method_name, method in _nose_tools_functions(): + node.locals[method_name] = [method] + + +def _nose_tools_trivial_transform(): + """Custom transform for the nose.tools module.""" + stub = _BUILDER.string_build("""__all__ = []""") + all_entries = ["ok_", "eq_"] + + for pep8_name, method in _nose_tools_functions(): + all_entries.append(pep8_name) + stub[pep8_name] = method + + # Update the __all__ variable, since nose.tools + # does this manually with .append. + all_assign = stub["__all__"].parent + all_object = astroid.List(all_entries) + all_object.parent = all_assign + all_assign.value = all_object + return stub + + +astroid.register_module_extender( + astroid.MANAGER, "nose.tools.trivial", _nose_tools_trivial_transform +) +astroid.MANAGER.register_transform( + astroid.Module, _nose_tools_transform, lambda n: n.name == "nose.tools" +) diff --git a/env/lib/python3.8/site-packages/astroid/brain/brain_numpy_core_fromnumeric.py b/env/lib/python3.8/site-packages/astroid/brain/brain_numpy_core_fromnumeric.py new file mode 100644 index 000000000..62dfe991c --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/brain/brain_numpy_core_fromnumeric.py @@ -0,0 +1,23 @@ +# Copyright (c) 2019 hippo91 + +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER + + +"""Astroid hooks for numpy.core.fromnumeric module.""" + +import astroid + + +def numpy_core_fromnumeric_transform(): + return astroid.parse( + """ + def sum(a, axis=None, dtype=None, out=None, keepdims=None, initial=None): + return numpy.ndarray([0, 0]) + """ + ) + + +astroid.register_module_extender( + astroid.MANAGER, "numpy.core.fromnumeric", numpy_core_fromnumeric_transform +) diff --git a/env/lib/python3.8/site-packages/astroid/brain/brain_numpy_core_function_base.py b/env/lib/python3.8/site-packages/astroid/brain/brain_numpy_core_function_base.py new file mode 100644 index 000000000..58aa0a985 --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/brain/brain_numpy_core_function_base.py @@ -0,0 +1,29 @@ +# Copyright (c) 2019 hippo91 + +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER + + +"""Astroid hooks for numpy.core.function_base module.""" + +import functools +import astroid +from brain_numpy_utils import looks_like_numpy_member, infer_numpy_member + + +METHODS_TO_BE_INFERRED = { + "linspace": """def linspace(start, stop, num=50, endpoint=True, retstep=False, dtype=None, axis=0): + return numpy.ndarray([0, 0])""", + "logspace": """def logspace(start, stop, num=50, endpoint=True, base=10.0, dtype=None, axis=0): + return numpy.ndarray([0, 0])""", + "geomspace": """def geomspace(start, stop, num=50, endpoint=True, dtype=None, axis=0): + return numpy.ndarray([0, 0])""", +} + +for func_name, func_src in METHODS_TO_BE_INFERRED.items(): + inference_function = functools.partial(infer_numpy_member, func_src) + astroid.MANAGER.register_transform( + astroid.Attribute, + astroid.inference_tip(inference_function), + functools.partial(looks_like_numpy_member, func_name), + ) diff --git a/env/lib/python3.8/site-packages/astroid/brain/brain_numpy_core_multiarray.py b/env/lib/python3.8/site-packages/astroid/brain/brain_numpy_core_multiarray.py new file mode 100644 index 000000000..b2e32bc85 --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/brain/brain_numpy_core_multiarray.py @@ -0,0 +1,92 @@ +# Copyright (c) 2019-2020 hippo91 + +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER + + +"""Astroid hooks for numpy.core.multiarray module.""" + +import functools +import astroid +from brain_numpy_utils import looks_like_numpy_member, infer_numpy_member + + +def numpy_core_multiarray_transform(): + return astroid.parse( + """ + # different functions defined in multiarray.py + def inner(a, b): + return numpy.ndarray([0, 0]) + + def vdot(a, b): + return numpy.ndarray([0, 0]) + """ + ) + + +astroid.register_module_extender( + astroid.MANAGER, "numpy.core.multiarray", numpy_core_multiarray_transform +) + + +METHODS_TO_BE_INFERRED = { + "array": """def array(object, dtype=None, copy=True, order='K', subok=False, ndmin=0): + return numpy.ndarray([0, 0])""", + "dot": """def dot(a, b, out=None): + return numpy.ndarray([0, 0])""", + "empty_like": """def empty_like(a, dtype=None, order='K', subok=True): + return numpy.ndarray((0, 0))""", + "concatenate": """def concatenate(arrays, axis=None, out=None): + return numpy.ndarray((0, 0))""", + "where": """def where(condition, x=None, y=None): + return numpy.ndarray([0, 0])""", + "empty": """def empty(shape, dtype=float, order='C'): + return numpy.ndarray([0, 0])""", + "bincount": """def bincount(x, weights=None, minlength=0): + return numpy.ndarray([0, 0])""", + "busday_count": """def busday_count(begindates, enddates, weekmask='1111100', holidays=[], busdaycal=None, out=None): + return numpy.ndarray([0, 0])""", + "busday_offset": """def busday_offset(dates, offsets, roll='raise', weekmask='1111100', holidays=None, busdaycal=None, out=None): + return numpy.ndarray([0, 0])""", + "can_cast": """def can_cast(from_, to, casting='safe'): + return True""", + "copyto": """def copyto(dst, src, casting='same_kind', where=True): + return None""", + "datetime_as_string": """def datetime_as_string(arr, unit=None, timezone='naive', casting='same_kind'): + return numpy.ndarray([0, 0])""", + "is_busday": """def is_busday(dates, weekmask='1111100', holidays=None, busdaycal=None, out=None): + return numpy.ndarray([0, 0])""", + "lexsort": """def lexsort(keys, axis=-1): + return numpy.ndarray([0, 0])""", + "may_share_memory": """def may_share_memory(a, b, max_work=None): + return True""", + # Not yet available because dtype is not yet present in those brains + # "min_scalar_type": """def min_scalar_type(a): + # return numpy.dtype('int16')""", + "packbits": """def packbits(a, axis=None, bitorder='big'): + return numpy.ndarray([0, 0])""", + # Not yet available because dtype is not yet present in those brains + # "result_type": """def result_type(*arrays_and_dtypes): + # return numpy.dtype('int16')""", + "shares_memory": """def shares_memory(a, b, max_work=None): + return True""", + "unpackbits": """def unpackbits(a, axis=None, count=None, bitorder='big'): + return numpy.ndarray([0, 0])""", + "unravel_index": """def unravel_index(indices, shape, order='C'): + return (numpy.ndarray([0, 0]),)""", + "zeros": """def zeros(shape, dtype=float, order='C'): + return numpy.ndarray([0, 0])""", +} + +for method_name, function_src in METHODS_TO_BE_INFERRED.items(): + inference_function = functools.partial(infer_numpy_member, function_src) + astroid.MANAGER.register_transform( + astroid.Attribute, + astroid.inference_tip(inference_function), + functools.partial(looks_like_numpy_member, method_name), + ) + astroid.MANAGER.register_transform( + astroid.Name, + astroid.inference_tip(inference_function), + functools.partial(looks_like_numpy_member, method_name), + ) diff --git a/env/lib/python3.8/site-packages/astroid/brain/brain_numpy_core_numeric.py b/env/lib/python3.8/site-packages/astroid/brain/brain_numpy_core_numeric.py new file mode 100644 index 000000000..2a6f37e39 --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/brain/brain_numpy_core_numeric.py @@ -0,0 +1,43 @@ +# Copyright (c) 2019 hippo91 + +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER + + +"""Astroid hooks for numpy.core.numeric module.""" + +import functools +import astroid +from brain_numpy_utils import looks_like_numpy_member, infer_numpy_member + + +def numpy_core_numeric_transform(): + return astroid.parse( + """ + # different functions defined in numeric.py + import numpy + def zeros_like(a, dtype=None, order='K', subok=True): return numpy.ndarray((0, 0)) + def ones_like(a, dtype=None, order='K', subok=True): return numpy.ndarray((0, 0)) + def full_like(a, fill_value, dtype=None, order='K', subok=True): return numpy.ndarray((0, 0)) + """ + ) + + +astroid.register_module_extender( + astroid.MANAGER, "numpy.core.numeric", numpy_core_numeric_transform +) + + +METHODS_TO_BE_INFERRED = { + "ones": """def ones(shape, dtype=None, order='C'): + return numpy.ndarray([0, 0])""" +} + + +for method_name, function_src in METHODS_TO_BE_INFERRED.items(): + inference_function = functools.partial(infer_numpy_member, function_src) + astroid.MANAGER.register_transform( + astroid.Attribute, + astroid.inference_tip(inference_function), + functools.partial(looks_like_numpy_member, method_name), + ) diff --git a/env/lib/python3.8/site-packages/astroid/brain/brain_numpy_core_numerictypes.py b/env/lib/python3.8/site-packages/astroid/brain/brain_numpy_core_numerictypes.py new file mode 100644 index 000000000..6ac4a1463 --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/brain/brain_numpy_core_numerictypes.py @@ -0,0 +1,254 @@ +# Copyright (c) 2019-2020 hippo91 + +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER + +# TODO(hippo91) : correct the methods signature. + +"""Astroid hooks for numpy.core.numerictypes module.""" + +import astroid + + +def numpy_core_numerictypes_transform(): + # TODO: Uniformize the generic API with the ndarray one. + # According to numpy doc the generic object should expose + # the same API than ndarray. This has been done here partially + # through the astype method. + return astroid.parse( + """ + # different types defined in numerictypes.py + class generic(object): + def __init__(self, value): + self.T = None + self.base = None + self.data = None + self.dtype = None + self.flags = None + self.flat = None + self.imag = None + self.itemsize = None + self.nbytes = None + self.ndim = None + self.real = None + self.size = None + self.strides = None + + def all(self): return uninferable + def any(self): return uninferable + def argmax(self): return uninferable + def argmin(self): return uninferable + def argsort(self): return uninferable + def astype(self, dtype, order='K', casting='unsafe', subok=True, copy=True): return np.ndarray([0, 0]) + def base(self): return uninferable + def byteswap(self): return uninferable + def choose(self): return uninferable + def clip(self): return uninferable + def compress(self): return uninferable + def conj(self): return uninferable + def conjugate(self): return uninferable + def copy(self): return uninferable + def cumprod(self): return uninferable + def cumsum(self): return uninferable + def data(self): return uninferable + def diagonal(self): return uninferable + def dtype(self): return uninferable + def dump(self): return uninferable + def dumps(self): return uninferable + def fill(self): return uninferable + def flags(self): return uninferable + def flat(self): return uninferable + def flatten(self): return uninferable + def getfield(self): return uninferable + def imag(self): return uninferable + def item(self): return uninferable + def itemset(self): return uninferable + def itemsize(self): return uninferable + def max(self): return uninferable + def mean(self): return uninferable + def min(self): return uninferable + def nbytes(self): return uninferable + def ndim(self): return uninferable + def newbyteorder(self): return uninferable + def nonzero(self): return uninferable + def prod(self): return uninferable + def ptp(self): return uninferable + def put(self): return uninferable + def ravel(self): return uninferable + def real(self): return uninferable + def repeat(self): return uninferable + def reshape(self): return uninferable + def resize(self): return uninferable + def round(self): return uninferable + def searchsorted(self): return uninferable + def setfield(self): return uninferable + def setflags(self): return uninferable + def shape(self): return uninferable + def size(self): return uninferable + def sort(self): return uninferable + def squeeze(self): return uninferable + def std(self): return uninferable + def strides(self): return uninferable + def sum(self): return uninferable + def swapaxes(self): return uninferable + def take(self): return uninferable + def tobytes(self): return uninferable + def tofile(self): return uninferable + def tolist(self): return uninferable + def tostring(self): return uninferable + def trace(self): return uninferable + def transpose(self): return uninferable + def var(self): return uninferable + def view(self): return uninferable + + + class dtype(object): + def __init__(self, obj, align=False, copy=False): + self.alignment = None + self.base = None + self.byteorder = None + self.char = None + self.descr = None + self.fields = None + self.flags = None + self.hasobject = None + self.isalignedstruct = None + self.isbuiltin = None + self.isnative = None + self.itemsize = None + self.kind = None + self.metadata = None + self.name = None + self.names = None + self.num = None + self.shape = None + self.str = None + self.subdtype = None + self.type = None + + def newbyteorder(self, new_order='S'): return uninferable + def __neg__(self): return uninferable + + class busdaycalendar(object): + def __init__(self, weekmask='1111100', holidays=None): + self.holidays = None + self.weekmask = None + + class flexible(generic): pass + class bool_(generic): pass + class number(generic): + def __neg__(self): return uninferable + class datetime64(generic): + def __init__(self, nb, unit=None): pass + + + class void(flexible): + def __init__(self, *args, **kwargs): + self.base = None + self.dtype = None + self.flags = None + def getfield(self): return uninferable + def setfield(self): return uninferable + + + class character(flexible): pass + + + class integer(number): + def __init__(self, value): + self.denominator = None + self.numerator = None + + + class inexact(number): pass + + + class str_(str, character): + def maketrans(self, x, y=None, z=None): return uninferable + + + class bytes_(bytes, character): + def fromhex(self, string): return uninferable + def maketrans(self, frm, to): return uninferable + + + class signedinteger(integer): pass + + + class unsignedinteger(integer): pass + + + class complexfloating(inexact): pass + + + class floating(inexact): pass + + + class float64(floating, float): + def fromhex(self, string): return uninferable + + + class uint64(unsignedinteger): pass + class complex64(complexfloating): pass + class int16(signedinteger): pass + class float96(floating): pass + class int8(signedinteger): pass + class uint32(unsignedinteger): pass + class uint8(unsignedinteger): pass + class _typedict(dict): pass + class complex192(complexfloating): pass + class timedelta64(signedinteger): + def __init__(self, nb, unit=None): pass + class int32(signedinteger): pass + class uint16(unsignedinteger): pass + class float32(floating): pass + class complex128(complexfloating, complex): pass + class float16(floating): pass + class int64(signedinteger): pass + + buffer_type = memoryview + bool8 = bool_ + byte = int8 + bytes0 = bytes_ + cdouble = complex128 + cfloat = complex128 + clongdouble = complex192 + clongfloat = complex192 + complex_ = complex128 + csingle = complex64 + double = float64 + float_ = float64 + half = float16 + int0 = int32 + int_ = int32 + intc = int32 + intp = int32 + long = int32 + longcomplex = complex192 + longdouble = float96 + longfloat = float96 + longlong = int64 + object0 = object_ + object_ = object_ + short = int16 + single = float32 + singlecomplex = complex64 + str0 = str_ + string_ = bytes_ + ubyte = uint8 + uint = uint32 + uint0 = uint32 + uintc = uint32 + uintp = uint32 + ulonglong = uint64 + unicode = str_ + unicode_ = str_ + ushort = uint16 + void0 = void + """ + ) + + +astroid.register_module_extender( + astroid.MANAGER, "numpy.core.numerictypes", numpy_core_numerictypes_transform +) diff --git a/env/lib/python3.8/site-packages/astroid/brain/brain_numpy_core_umath.py b/env/lib/python3.8/site-packages/astroid/brain/brain_numpy_core_umath.py new file mode 100644 index 000000000..897961eab --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/brain/brain_numpy_core_umath.py @@ -0,0 +1,147 @@ +# Copyright (c) 2019 hippo91 + +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER + + +"""Astroid hooks for numpy.core.umath module.""" + +import astroid + + +def numpy_core_umath_transform(): + ufunc_optional_keyword_arguments = ( + """out=None, where=True, casting='same_kind', order='K', """ + """dtype=None, subok=True""" + ) + return astroid.parse( + """ + class FakeUfunc: + def __init__(self): + self.__doc__ = str() + self.__name__ = str() + self.nin = 0 + self.nout = 0 + self.nargs = 0 + self.ntypes = 0 + self.types = None + self.identity = None + self.signature = None + + @classmethod + def reduce(cls, a, axis=None, dtype=None, out=None): + return numpy.ndarray([0, 0]) + + @classmethod + def accumulate(cls, array, axis=None, dtype=None, out=None): + return numpy.ndarray([0, 0]) + + @classmethod + def reduceat(cls, a, indices, axis=None, dtype=None, out=None): + return numpy.ndarray([0, 0]) + + @classmethod + def outer(cls, A, B, **kwargs): + return numpy.ndarray([0, 0]) + + @classmethod + def at(cls, a, indices, b=None): + return numpy.ndarray([0, 0]) + + class FakeUfuncOneArg(FakeUfunc): + def __call__(self, x, {opt_args:s}): + return numpy.ndarray([0, 0]) + + class FakeUfuncOneArgBis(FakeUfunc): + def __call__(self, x, {opt_args:s}): + return numpy.ndarray([0, 0]), numpy.ndarray([0, 0]) + + class FakeUfuncTwoArgs(FakeUfunc): + def __call__(self, x1, x2, {opt_args:s}): + return numpy.ndarray([0, 0]) + + # Constants + e = 2.718281828459045 + euler_gamma = 0.5772156649015329 + + # One arg functions with optional kwargs + arccos = FakeUfuncOneArg() + arccosh = FakeUfuncOneArg() + arcsin = FakeUfuncOneArg() + arcsinh = FakeUfuncOneArg() + arctan = FakeUfuncOneArg() + arctanh = FakeUfuncOneArg() + cbrt = FakeUfuncOneArg() + conj = FakeUfuncOneArg() + conjugate = FakeUfuncOneArg() + cosh = FakeUfuncOneArg() + deg2rad = FakeUfuncOneArg() + exp2 = FakeUfuncOneArg() + expm1 = FakeUfuncOneArg() + fabs = FakeUfuncOneArg() + frexp = FakeUfuncOneArgBis() + isfinite = FakeUfuncOneArg() + isinf = FakeUfuncOneArg() + log = FakeUfuncOneArg() + log1p = FakeUfuncOneArg() + log2 = FakeUfuncOneArg() + logical_not = FakeUfuncOneArg() + modf = FakeUfuncOneArgBis() + negative = FakeUfuncOneArg() + positive = FakeUfuncOneArg() + rad2deg = FakeUfuncOneArg() + reciprocal = FakeUfuncOneArg() + rint = FakeUfuncOneArg() + sign = FakeUfuncOneArg() + signbit = FakeUfuncOneArg() + sinh = FakeUfuncOneArg() + spacing = FakeUfuncOneArg() + square = FakeUfuncOneArg() + tan = FakeUfuncOneArg() + tanh = FakeUfuncOneArg() + trunc = FakeUfuncOneArg() + + # Two args functions with optional kwargs + bitwise_and = FakeUfuncTwoArgs() + bitwise_or = FakeUfuncTwoArgs() + bitwise_xor = FakeUfuncTwoArgs() + copysign = FakeUfuncTwoArgs() + divide = FakeUfuncTwoArgs() + divmod = FakeUfuncTwoArgs() + equal = FakeUfuncTwoArgs() + float_power = FakeUfuncTwoArgs() + floor_divide = FakeUfuncTwoArgs() + fmax = FakeUfuncTwoArgs() + fmin = FakeUfuncTwoArgs() + fmod = FakeUfuncTwoArgs() + greater = FakeUfuncTwoArgs() + gcd = FakeUfuncTwoArgs() + hypot = FakeUfuncTwoArgs() + heaviside = FakeUfuncTwoArgs() + lcm = FakeUfuncTwoArgs() + ldexp = FakeUfuncTwoArgs() + left_shift = FakeUfuncTwoArgs() + less = FakeUfuncTwoArgs() + logaddexp = FakeUfuncTwoArgs() + logaddexp2 = FakeUfuncTwoArgs() + logical_and = FakeUfuncTwoArgs() + logical_or = FakeUfuncTwoArgs() + logical_xor = FakeUfuncTwoArgs() + maximum = FakeUfuncTwoArgs() + minimum = FakeUfuncTwoArgs() + nextafter = FakeUfuncTwoArgs() + not_equal = FakeUfuncTwoArgs() + power = FakeUfuncTwoArgs() + remainder = FakeUfuncTwoArgs() + right_shift = FakeUfuncTwoArgs() + subtract = FakeUfuncTwoArgs() + true_divide = FakeUfuncTwoArgs() + """.format( + opt_args=ufunc_optional_keyword_arguments + ) + ) + + +astroid.register_module_extender( + astroid.MANAGER, "numpy.core.umath", numpy_core_umath_transform +) diff --git a/env/lib/python3.8/site-packages/astroid/brain/brain_numpy_ndarray.py b/env/lib/python3.8/site-packages/astroid/brain/brain_numpy_ndarray.py new file mode 100644 index 000000000..d40a7dd0b --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/brain/brain_numpy_ndarray.py @@ -0,0 +1,153 @@ +# Copyright (c) 2015-2016, 2018-2019 Claudiu Popa +# Copyright (c) 2016 Ceridwen +# Copyright (c) 2017-2020 hippo91 + +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER + + +"""Astroid hooks for numpy ndarray class.""" + +import functools +import astroid + + +def infer_numpy_ndarray(node, context=None): + ndarray = """ + class ndarray(object): + def __init__(self, shape, dtype=float, buffer=None, offset=0, + strides=None, order=None): + self.T = None + self.base = None + self.ctypes = None + self.data = None + self.dtype = None + self.flags = None + self.flat = None + self.imag = np.ndarray([0, 0]) + self.itemsize = None + self.nbytes = None + self.ndim = None + self.real = np.ndarray([0, 0]) + self.shape = numpy.ndarray([0, 0]) + self.size = None + self.strides = None + + def __abs__(self): return numpy.ndarray([0, 0]) + def __add__(self, value): return numpy.ndarray([0, 0]) + def __and__(self, value): return numpy.ndarray([0, 0]) + def __array__(self, dtype=None): return numpy.ndarray([0, 0]) + def __array_wrap__(self, obj): return numpy.ndarray([0, 0]) + def __contains__(self, key): return True + def __copy__(self): return numpy.ndarray([0, 0]) + def __deepcopy__(self, memo): return numpy.ndarray([0, 0]) + def __divmod__(self, value): return (numpy.ndarray([0, 0]), numpy.ndarray([0, 0])) + def __eq__(self, value): return numpy.ndarray([0, 0]) + def __float__(self): return 0. + def __floordiv__(self): return numpy.ndarray([0, 0]) + def __ge__(self, value): return numpy.ndarray([0, 0]) + def __getitem__(self, key): return uninferable + def __gt__(self, value): return numpy.ndarray([0, 0]) + def __iadd__(self, value): return numpy.ndarray([0, 0]) + def __iand__(self, value): return numpy.ndarray([0, 0]) + def __ifloordiv__(self, value): return numpy.ndarray([0, 0]) + def __ilshift__(self, value): return numpy.ndarray([0, 0]) + def __imod__(self, value): return numpy.ndarray([0, 0]) + def __imul__(self, value): return numpy.ndarray([0, 0]) + def __int__(self): return 0 + def __invert__(self): return numpy.ndarray([0, 0]) + def __ior__(self, value): return numpy.ndarray([0, 0]) + def __ipow__(self, value): return numpy.ndarray([0, 0]) + def __irshift__(self, value): return numpy.ndarray([0, 0]) + def __isub__(self, value): return numpy.ndarray([0, 0]) + def __itruediv__(self, value): return numpy.ndarray([0, 0]) + def __ixor__(self, value): return numpy.ndarray([0, 0]) + def __le__(self, value): return numpy.ndarray([0, 0]) + def __len__(self): return 1 + def __lshift__(self, value): return numpy.ndarray([0, 0]) + def __lt__(self, value): return numpy.ndarray([0, 0]) + def __matmul__(self, value): return numpy.ndarray([0, 0]) + def __mod__(self, value): return numpy.ndarray([0, 0]) + def __mul__(self, value): return numpy.ndarray([0, 0]) + def __ne__(self, value): return numpy.ndarray([0, 0]) + def __neg__(self): return numpy.ndarray([0, 0]) + def __or__(self): return numpy.ndarray([0, 0]) + def __pos__(self): return numpy.ndarray([0, 0]) + def __pow__(self): return numpy.ndarray([0, 0]) + def __repr__(self): return str() + def __rshift__(self): return numpy.ndarray([0, 0]) + def __setitem__(self, key, value): return uninferable + def __str__(self): return str() + def __sub__(self, value): return numpy.ndarray([0, 0]) + def __truediv__(self, value): return numpy.ndarray([0, 0]) + def __xor__(self, value): return numpy.ndarray([0, 0]) + def all(self, axis=None, out=None, keepdims=False): return np.ndarray([0, 0]) + def any(self, axis=None, out=None, keepdims=False): return np.ndarray([0, 0]) + def argmax(self, axis=None, out=None): return np.ndarray([0, 0]) + def argmin(self, axis=None, out=None): return np.ndarray([0, 0]) + def argpartition(self, kth, axis=-1, kind='introselect', order=None): return np.ndarray([0, 0]) + def argsort(self, axis=-1, kind='quicksort', order=None): return np.ndarray([0, 0]) + def astype(self, dtype, order='K', casting='unsafe', subok=True, copy=True): return np.ndarray([0, 0]) + def byteswap(self, inplace=False): return np.ndarray([0, 0]) + def choose(self, choices, out=None, mode='raise'): return np.ndarray([0, 0]) + def clip(self, min=None, max=None, out=None): return np.ndarray([0, 0]) + def compress(self, condition, axis=None, out=None): return np.ndarray([0, 0]) + def conj(self): return np.ndarray([0, 0]) + def conjugate(self): return np.ndarray([0, 0]) + def copy(self, order='C'): return np.ndarray([0, 0]) + def cumprod(self, axis=None, dtype=None, out=None): return np.ndarray([0, 0]) + def cumsum(self, axis=None, dtype=None, out=None): return np.ndarray([0, 0]) + def diagonal(self, offset=0, axis1=0, axis2=1): return np.ndarray([0, 0]) + def dot(self, b, out=None): return np.ndarray([0, 0]) + def dump(self, file): return None + def dumps(self): return str() + def fill(self, value): return None + def flatten(self, order='C'): return np.ndarray([0, 0]) + def getfield(self, dtype, offset=0): return np.ndarray([0, 0]) + def item(self, *args): return uninferable + def itemset(self, *args): return None + def max(self, axis=None, out=None): return np.ndarray([0, 0]) + def mean(self, axis=None, dtype=None, out=None, keepdims=False): return np.ndarray([0, 0]) + def min(self, axis=None, out=None, keepdims=False): return np.ndarray([0, 0]) + def newbyteorder(self, new_order='S'): return np.ndarray([0, 0]) + def nonzero(self): return (1,) + def partition(self, kth, axis=-1, kind='introselect', order=None): return None + def prod(self, axis=None, dtype=None, out=None, keepdims=False): return np.ndarray([0, 0]) + def ptp(self, axis=None, out=None): return np.ndarray([0, 0]) + def put(self, indices, values, mode='raise'): return None + def ravel(self, order='C'): return np.ndarray([0, 0]) + def repeat(self, repeats, axis=None): return np.ndarray([0, 0]) + def reshape(self, shape, order='C'): return np.ndarray([0, 0]) + def resize(self, new_shape, refcheck=True): return None + def round(self, decimals=0, out=None): return np.ndarray([0, 0]) + def searchsorted(self, v, side='left', sorter=None): return np.ndarray([0, 0]) + def setfield(self, val, dtype, offset=0): return None + def setflags(self, write=None, align=None, uic=None): return None + def sort(self, axis=-1, kind='quicksort', order=None): return None + def squeeze(self, axis=None): return np.ndarray([0, 0]) + def std(self, axis=None, dtype=None, out=None, ddof=0, keepdims=False): return np.ndarray([0, 0]) + def sum(self, axis=None, dtype=None, out=None, keepdims=False): return np.ndarray([0, 0]) + def swapaxes(self, axis1, axis2): return np.ndarray([0, 0]) + def take(self, indices, axis=None, out=None, mode='raise'): return np.ndarray([0, 0]) + def tobytes(self, order='C'): return b'' + def tofile(self, fid, sep="", format="%s"): return None + def tolist(self, ): return [] + def tostring(self, order='C'): return b'' + def trace(self, offset=0, axis1=0, axis2=1, dtype=None, out=None): return np.ndarray([0, 0]) + def transpose(self, *axes): return np.ndarray([0, 0]) + def var(self, axis=None, dtype=None, out=None, ddof=0, keepdims=False): return np.ndarray([0, 0]) + def view(self, dtype=None, type=None): return np.ndarray([0, 0]) + """ + node = astroid.extract_node(ndarray) + return node.infer(context=context) + + +def _looks_like_numpy_ndarray(node): + return isinstance(node, astroid.Attribute) and node.attrname == "ndarray" + + +astroid.MANAGER.register_transform( + astroid.Attribute, + astroid.inference_tip(infer_numpy_ndarray), + _looks_like_numpy_ndarray, +) diff --git a/env/lib/python3.8/site-packages/astroid/brain/brain_numpy_random_mtrand.py b/env/lib/python3.8/site-packages/astroid/brain/brain_numpy_random_mtrand.py new file mode 100644 index 000000000..cffdceeff --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/brain/brain_numpy_random_mtrand.py @@ -0,0 +1,70 @@ +# Copyright (c) 2019 hippo91 + +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER + +# TODO(hippo91) : correct the functions return types +"""Astroid hooks for numpy.random.mtrand module.""" + +import astroid + + +def numpy_random_mtrand_transform(): + return astroid.parse( + """ + def beta(a, b, size=None): return uninferable + def binomial(n, p, size=None): return uninferable + def bytes(length): return uninferable + def chisquare(df, size=None): return uninferable + def choice(a, size=None, replace=True, p=None): return uninferable + def dirichlet(alpha, size=None): return uninferable + def exponential(scale=1.0, size=None): return uninferable + def f(dfnum, dfden, size=None): return uninferable + def gamma(shape, scale=1.0, size=None): return uninferable + def geometric(p, size=None): return uninferable + def get_state(): return uninferable + def gumbel(loc=0.0, scale=1.0, size=None): return uninferable + def hypergeometric(ngood, nbad, nsample, size=None): return uninferable + def laplace(loc=0.0, scale=1.0, size=None): return uninferable + def logistic(loc=0.0, scale=1.0, size=None): return uninferable + def lognormal(mean=0.0, sigma=1.0, size=None): return uninferable + def logseries(p, size=None): return uninferable + def multinomial(n, pvals, size=None): return uninferable + def multivariate_normal(mean, cov, size=None): return uninferable + def negative_binomial(n, p, size=None): return uninferable + def noncentral_chisquare(df, nonc, size=None): return uninferable + def noncentral_f(dfnum, dfden, nonc, size=None): return uninferable + def normal(loc=0.0, scale=1.0, size=None): return uninferable + def pareto(a, size=None): return uninferable + def permutation(x): return uninferable + def poisson(lam=1.0, size=None): return uninferable + def power(a, size=None): return uninferable + def rand(*args): return uninferable + def randint(low, high=None, size=None, dtype='l'): + import numpy + return numpy.ndarray((1,1)) + def randn(*args): return uninferable + def random_integers(low, high=None, size=None): return uninferable + def random_sample(size=None): return uninferable + def rayleigh(scale=1.0, size=None): return uninferable + def seed(seed=None): return uninferable + def set_state(state): return uninferable + def shuffle(x): return uninferable + def standard_cauchy(size=None): return uninferable + def standard_exponential(size=None): return uninferable + def standard_gamma(shape, size=None): return uninferable + def standard_normal(size=None): return uninferable + def standard_t(df, size=None): return uninferable + def triangular(left, mode, right, size=None): return uninferable + def uniform(low=0.0, high=1.0, size=None): return uninferable + def vonmises(mu, kappa, size=None): return uninferable + def wald(mean, scale, size=None): return uninferable + def weibull(a, size=None): return uninferable + def zipf(a, size=None): return uninferable + """ + ) + + +astroid.register_module_extender( + astroid.MANAGER, "numpy.random.mtrand", numpy_random_mtrand_transform +) diff --git a/env/lib/python3.8/site-packages/astroid/brain/brain_numpy_utils.py b/env/lib/python3.8/site-packages/astroid/brain/brain_numpy_utils.py new file mode 100644 index 000000000..b29d2719c --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/brain/brain_numpy_utils.py @@ -0,0 +1,65 @@ +# Copyright (c) 2019-2020 hippo91 +# Copyright (c) 2019 Claudiu Popa + +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER + + +"""Different utilities for the numpy brains""" + + +import astroid + + +def infer_numpy_member(src, node, context=None): + node = astroid.extract_node(src) + return node.infer(context=context) + + +def _is_a_numpy_module(node: astroid.node_classes.Name) -> bool: + """ + Returns True if the node is a representation of a numpy module. + + For example in : + import numpy as np + x = np.linspace(1, 2) + The node is a representation of the numpy module. + + :param node: node to test + :return: True if the node is a representation of the numpy module. + """ + module_nickname = node.name + potential_import_target = [ + x for x in node.lookup(module_nickname)[1] if isinstance(x, astroid.Import) + ] + for target in potential_import_target: + if ("numpy", module_nickname) in target.names: + return True + return False + + +def looks_like_numpy_member( + member_name: str, node: astroid.node_classes.NodeNG +) -> bool: + """ + Returns True if the node is a member of numpy whose + name is member_name. + + :param member_name: name of the member + :param node: node to test + :return: True if the node is a member of numpy + """ + if ( + isinstance(node, astroid.Attribute) + and node.attrname == member_name + and isinstance(node.expr, astroid.Name) + and _is_a_numpy_module(node.expr) + ): + return True + if ( + isinstance(node, astroid.Name) + and node.name == member_name + and node.root().name.startswith("numpy") + ): + return True + return False diff --git a/env/lib/python3.8/site-packages/astroid/brain/brain_pkg_resources.py b/env/lib/python3.8/site-packages/astroid/brain/brain_pkg_resources.py new file mode 100644 index 000000000..25e764959 --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/brain/brain_pkg_resources.py @@ -0,0 +1,75 @@ +# Copyright (c) 2016, 2018 Claudiu Popa +# Copyright (c) 2016 Ceridwen + +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER + + +import astroid +from astroid import parse +from astroid import inference_tip +from astroid import register_module_extender +from astroid import MANAGER + + +def pkg_resources_transform(): + return parse( + """ +def require(*requirements): + return pkg_resources.working_set.require(*requirements) + +def run_script(requires, script_name): + return pkg_resources.working_set.run_script(requires, script_name) + +def iter_entry_points(group, name=None): + return pkg_resources.working_set.iter_entry_points(group, name) + +def resource_exists(package_or_requirement, resource_name): + return get_provider(package_or_requirement).has_resource(resource_name) + +def resource_isdir(package_or_requirement, resource_name): + return get_provider(package_or_requirement).resource_isdir( + resource_name) + +def resource_filename(package_or_requirement, resource_name): + return get_provider(package_or_requirement).get_resource_filename( + self, resource_name) + +def resource_stream(package_or_requirement, resource_name): + return get_provider(package_or_requirement).get_resource_stream( + self, resource_name) + +def resource_string(package_or_requirement, resource_name): + return get_provider(package_or_requirement).get_resource_string( + self, resource_name) + +def resource_listdir(package_or_requirement, resource_name): + return get_provider(package_or_requirement).resource_listdir( + resource_name) + +def extraction_error(): + pass + +def get_cache_path(archive_name, names=()): + extract_path = self.extraction_path or get_default_cache() + target_path = os.path.join(extract_path, archive_name+'-tmp', *names) + return target_path + +def postprocess(tempname, filename): + pass + +def set_extraction_path(path): + pass + +def cleanup_resources(force=False): + pass + +def get_distribution(dist): + return Distribution(dist) + +_namespace_packages = {} +""" + ) + + +register_module_extender(MANAGER, "pkg_resources", pkg_resources_transform) diff --git a/env/lib/python3.8/site-packages/astroid/brain/brain_pytest.py b/env/lib/python3.8/site-packages/astroid/brain/brain_pytest.py new file mode 100644 index 000000000..56202ab85 --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/brain/brain_pytest.py @@ -0,0 +1,88 @@ +# Copyright (c) 2014-2016, 2018 Claudiu Popa +# Copyright (c) 2014 Jeff Quast +# Copyright (c) 2014 Google, Inc. +# Copyright (c) 2016 Florian Bruhin +# Copyright (c) 2016 Ceridwen + +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER + +"""Astroid hooks for pytest.""" +from __future__ import absolute_import +from astroid import MANAGER, register_module_extender +from astroid.builder import AstroidBuilder + + +def pytest_transform(): + return AstroidBuilder(MANAGER).string_build( + """ + +try: + import _pytest.mark + import _pytest.recwarn + import _pytest.runner + import _pytest.python + import _pytest.skipping + import _pytest.assertion +except ImportError: + pass +else: + deprecated_call = _pytest.recwarn.deprecated_call + warns = _pytest.recwarn.warns + + exit = _pytest.runner.exit + fail = _pytest.runner.fail + skip = _pytest.runner.skip + importorskip = _pytest.runner.importorskip + + xfail = _pytest.skipping.xfail + mark = _pytest.mark.MarkGenerator() + raises = _pytest.python.raises + + # New in pytest 3.0 + try: + approx = _pytest.python.approx + register_assert_rewrite = _pytest.assertion.register_assert_rewrite + except AttributeError: + pass + + +# Moved in pytest 3.0 + +try: + import _pytest.freeze_support + freeze_includes = _pytest.freeze_support.freeze_includes +except ImportError: + try: + import _pytest.genscript + freeze_includes = _pytest.genscript.freeze_includes + except ImportError: + pass + +try: + import _pytest.debugging + set_trace = _pytest.debugging.pytestPDB().set_trace +except ImportError: + try: + import _pytest.pdb + set_trace = _pytest.pdb.pytestPDB().set_trace + except ImportError: + pass + +try: + import _pytest.fixtures + fixture = _pytest.fixtures.fixture + yield_fixture = _pytest.fixtures.yield_fixture +except ImportError: + try: + import _pytest.python + fixture = _pytest.python.fixture + yield_fixture = _pytest.python.yield_fixture + except ImportError: + pass +""" + ) + + +register_module_extender(MANAGER, "pytest", pytest_transform) +register_module_extender(MANAGER, "py.test", pytest_transform) diff --git a/env/lib/python3.8/site-packages/astroid/brain/brain_qt.py b/env/lib/python3.8/site-packages/astroid/brain/brain_qt.py new file mode 100644 index 000000000..b703b373e --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/brain/brain_qt.py @@ -0,0 +1,83 @@ +# Copyright (c) 2015-2016, 2018 Claudiu Popa +# Copyright (c) 2016 Ceridwen +# Copyright (c) 2017 Roy Wright +# Copyright (c) 2018 Ashley Whetter +# Copyright (c) 2019 Antoine Boellinger + +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER + +"""Astroid hooks for the PyQT library.""" + +from astroid import MANAGER, register_module_extender +from astroid.builder import AstroidBuilder +from astroid import nodes +from astroid import parse + + +def _looks_like_signal(node, signal_name="pyqtSignal"): + if "__class__" in node.instance_attrs: + try: + cls = node.instance_attrs["__class__"][0] + return cls.name == signal_name + except AttributeError: + # return False if the cls does not have a name attribute + pass + return False + + +def transform_pyqt_signal(node): + module = parse( + """ + class pyqtSignal(object): + def connect(self, slot, type=None, no_receiver_check=False): + pass + def disconnect(self, slot): + pass + def emit(self, *args): + pass + """ + ) + signal_cls = module["pyqtSignal"] + node.instance_attrs["emit"] = signal_cls["emit"] + node.instance_attrs["disconnect"] = signal_cls["disconnect"] + node.instance_attrs["connect"] = signal_cls["connect"] + + +def transform_pyside_signal(node): + module = parse( + """ + class NotPySideSignal(object): + def connect(self, receiver, type=None): + pass + def disconnect(self, receiver): + pass + def emit(self, *args): + pass + """ + ) + signal_cls = module["NotPySideSignal"] + node.instance_attrs["connect"] = signal_cls["connect"] + node.instance_attrs["disconnect"] = signal_cls["disconnect"] + node.instance_attrs["emit"] = signal_cls["emit"] + + +def pyqt4_qtcore_transform(): + return AstroidBuilder(MANAGER).string_build( + """ + +def SIGNAL(signal_name): pass + +class QObject(object): + def emit(self, signal): pass +""" + ) + + +register_module_extender(MANAGER, "PyQt4.QtCore", pyqt4_qtcore_transform) +MANAGER.register_transform(nodes.FunctionDef, transform_pyqt_signal, _looks_like_signal) +MANAGER.register_transform( + nodes.ClassDef, + transform_pyside_signal, + lambda node: node.qname() in ("PySide.QtCore.Signal", "PySide2.QtCore.Signal"), +) diff --git a/env/lib/python3.8/site-packages/astroid/brain/brain_random.py b/env/lib/python3.8/site-packages/astroid/brain/brain_random.py new file mode 100644 index 000000000..5ec858a1c --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/brain/brain_random.py @@ -0,0 +1,75 @@ +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER +import random + +import astroid +from astroid import helpers +from astroid import MANAGER + + +ACCEPTED_ITERABLES_FOR_SAMPLE = (astroid.List, astroid.Set, astroid.Tuple) + + +def _clone_node_with_lineno(node, parent, lineno): + cls = node.__class__ + other_fields = node._other_fields + _astroid_fields = node._astroid_fields + init_params = {"lineno": lineno, "col_offset": node.col_offset, "parent": parent} + postinit_params = {param: getattr(node, param) for param in _astroid_fields} + if other_fields: + init_params.update({param: getattr(node, param) for param in other_fields}) + new_node = cls(**init_params) + if hasattr(node, "postinit") and _astroid_fields: + new_node.postinit(**postinit_params) + return new_node + + +def infer_random_sample(node, context=None): + if len(node.args) != 2: + raise astroid.UseInferenceDefault + + length = node.args[1] + if not isinstance(length, astroid.Const): + raise astroid.UseInferenceDefault + if not isinstance(length.value, int): + raise astroid.UseInferenceDefault + + inferred_sequence = helpers.safe_infer(node.args[0], context=context) + if not inferred_sequence: + raise astroid.UseInferenceDefault + + if not isinstance(inferred_sequence, ACCEPTED_ITERABLES_FOR_SAMPLE): + raise astroid.UseInferenceDefault + + if length.value > len(inferred_sequence.elts): + # In this case, this will raise a ValueError + raise astroid.UseInferenceDefault + + try: + elts = random.sample(inferred_sequence.elts, length.value) + except ValueError: + raise astroid.UseInferenceDefault + + new_node = astroid.List( + lineno=node.lineno, col_offset=node.col_offset, parent=node.scope() + ) + new_elts = [ + _clone_node_with_lineno(elt, parent=new_node, lineno=new_node.lineno) + for elt in elts + ] + new_node.postinit(new_elts) + return iter((new_node,)) + + +def _looks_like_random_sample(node): + func = node.func + if isinstance(func, astroid.Attribute): + return func.attrname == "sample" + if isinstance(func, astroid.Name): + return func.name == "sample" + return False + + +MANAGER.register_transform( + astroid.Call, astroid.inference_tip(infer_random_sample), _looks_like_random_sample +) diff --git a/env/lib/python3.8/site-packages/astroid/brain/brain_re.py b/env/lib/python3.8/site-packages/astroid/brain/brain_re.py new file mode 100644 index 000000000..c7ee51a5a --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/brain/brain_re.py @@ -0,0 +1,36 @@ +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER +import sys +import astroid + +PY36 = sys.version_info >= (3, 6) + +if PY36: + # Since Python 3.6 there is the RegexFlag enum + # where every entry will be exposed via updating globals() + + def _re_transform(): + return astroid.parse( + """ + import sre_compile + ASCII = sre_compile.SRE_FLAG_ASCII + IGNORECASE = sre_compile.SRE_FLAG_IGNORECASE + LOCALE = sre_compile.SRE_FLAG_LOCALE + UNICODE = sre_compile.SRE_FLAG_UNICODE + MULTILINE = sre_compile.SRE_FLAG_MULTILINE + DOTALL = sre_compile.SRE_FLAG_DOTALL + VERBOSE = sre_compile.SRE_FLAG_VERBOSE + A = ASCII + I = IGNORECASE + L = LOCALE + U = UNICODE + M = MULTILINE + S = DOTALL + X = VERBOSE + TEMPLATE = sre_compile.SRE_FLAG_TEMPLATE + T = TEMPLATE + DEBUG = sre_compile.SRE_FLAG_DEBUG + """ + ) + + astroid.register_module_extender(astroid.MANAGER, "re", _re_transform) diff --git a/env/lib/python3.8/site-packages/astroid/brain/brain_responses.py b/env/lib/python3.8/site-packages/astroid/brain/brain_responses.py new file mode 100644 index 000000000..3a4412980 --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/brain/brain_responses.py @@ -0,0 +1,73 @@ +""" +Astroid hooks for responses. + +It might need to be manually updated from the public methods of +:class:`responses.RequestsMock`. + +See: https://github.com/getsentry/responses/blob/master/responses.py + +""" +import astroid + + +def responses_funcs(): + return astroid.parse( + """ + DELETE = "DELETE" + GET = "GET" + HEAD = "HEAD" + OPTIONS = "OPTIONS" + PATCH = "PATCH" + POST = "POST" + PUT = "PUT" + response_callback = None + + def reset(): + return + + def add( + method=None, # method or ``Response`` + url=None, + body="", + adding_headers=None, + *args, + **kwargs + ): + return + + def add_passthru(prefix): + return + + def remove(method_or_response=None, url=None): + return + + def replace(method_or_response=None, url=None, body="", *args, **kwargs): + return + + def add_callback( + method, url, callback, match_querystring=False, content_type="text/plain" + ): + return + + calls = [] + + def __enter__(): + return + + def __exit__(type, value, traceback): + success = type is None + return success + + def activate(func): + return func + + def start(): + return + + def stop(allow_assert=True): + return + """ + ) + + +astroid.register_module_extender(astroid.MANAGER, "responses", responses_funcs) diff --git a/env/lib/python3.8/site-packages/astroid/brain/brain_scipy_signal.py b/env/lib/python3.8/site-packages/astroid/brain/brain_scipy_signal.py new file mode 100644 index 000000000..996300d48 --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/brain/brain_scipy_signal.py @@ -0,0 +1,89 @@ +# Copyright (c) 2019 Valentin Valls + +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER + + +"""Astroid hooks for scipy.signal module.""" + +import astroid + + +def scipy_signal(): + return astroid.parse( + """ + # different functions defined in scipy.signals + + def barthann(M, sym=True): + return numpy.ndarray([0]) + + def bartlett(M, sym=True): + return numpy.ndarray([0]) + + def blackman(M, sym=True): + return numpy.ndarray([0]) + + def blackmanharris(M, sym=True): + return numpy.ndarray([0]) + + def bohman(M, sym=True): + return numpy.ndarray([0]) + + def boxcar(M, sym=True): + return numpy.ndarray([0]) + + def chebwin(M, at, sym=True): + return numpy.ndarray([0]) + + def cosine(M, sym=True): + return numpy.ndarray([0]) + + def exponential(M, center=None, tau=1.0, sym=True): + return numpy.ndarray([0]) + + def flattop(M, sym=True): + return numpy.ndarray([0]) + + def gaussian(M, std, sym=True): + return numpy.ndarray([0]) + + def general_gaussian(M, p, sig, sym=True): + return numpy.ndarray([0]) + + def hamming(M, sym=True): + return numpy.ndarray([0]) + + def hann(M, sym=True): + return numpy.ndarray([0]) + + def hanning(M, sym=True): + return numpy.ndarray([0]) + + def impulse2(system, X0=None, T=None, N=None, **kwargs): + return numpy.ndarray([0]), numpy.ndarray([0]) + + def kaiser(M, beta, sym=True): + return numpy.ndarray([0]) + + def nuttall(M, sym=True): + return numpy.ndarray([0]) + + def parzen(M, sym=True): + return numpy.ndarray([0]) + + def slepian(M, width, sym=True): + return numpy.ndarray([0]) + + def step2(system, X0=None, T=None, N=None, **kwargs): + return numpy.ndarray([0]), numpy.ndarray([0]) + + def triang(M, sym=True): + return numpy.ndarray([0]) + + def tukey(M, alpha=0.5, sym=True): + return numpy.ndarray([0]) + """ + ) + + +astroid.register_module_extender(astroid.MANAGER, "scipy.signal", scipy_signal) diff --git a/env/lib/python3.8/site-packages/astroid/brain/brain_six.py b/env/lib/python3.8/site-packages/astroid/brain/brain_six.py new file mode 100644 index 000000000..46d9fa329 --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/brain/brain_six.py @@ -0,0 +1,201 @@ +# Copyright (c) 2014-2016, 2018, 2020 Claudiu Popa +# Copyright (c) 2015-2016 Ceridwen +# Copyright (c) 2018 Bryce Guinta + +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER + + +"""Astroid hooks for six module.""" + +from textwrap import dedent + +from astroid import MANAGER, register_module_extender +from astroid.builder import AstroidBuilder +from astroid.exceptions import ( + AstroidBuildingError, + InferenceError, + AttributeInferenceError, +) +from astroid import nodes + + +SIX_ADD_METACLASS = "six.add_metaclass" + + +def _indent(text, prefix, predicate=None): + """Adds 'prefix' to the beginning of selected lines in 'text'. + + If 'predicate' is provided, 'prefix' will only be added to the lines + where 'predicate(line)' is True. If 'predicate' is not provided, + it will default to adding 'prefix' to all non-empty lines that do not + consist solely of whitespace characters. + """ + if predicate is None: + predicate = lambda line: line.strip() + + def prefixed_lines(): + for line in text.splitlines(True): + yield prefix + line if predicate(line) else line + + return "".join(prefixed_lines()) + + +_IMPORTS = """ +import _io +cStringIO = _io.StringIO +filter = filter +from itertools import filterfalse +input = input +from sys import intern +map = map +range = range +from importlib import reload +reload_module = lambda module: reload(module) +from functools import reduce +from shlex import quote as shlex_quote +from io import StringIO +from collections import UserDict, UserList, UserString +xrange = range +zip = zip +from itertools import zip_longest +import builtins +import configparser +import copyreg +import _dummy_thread +import http.cookiejar as http_cookiejar +import http.cookies as http_cookies +import html.entities as html_entities +import html.parser as html_parser +import http.client as http_client +import http.server as http_server +BaseHTTPServer = CGIHTTPServer = SimpleHTTPServer = http.server +import pickle as cPickle +import queue +import reprlib +import socketserver +import _thread +import winreg +import xmlrpc.server as xmlrpc_server +import xmlrpc.client as xmlrpc_client +import urllib.robotparser as urllib_robotparser +import email.mime.multipart as email_mime_multipart +import email.mime.nonmultipart as email_mime_nonmultipart +import email.mime.text as email_mime_text +import email.mime.base as email_mime_base +import urllib.parse as urllib_parse +import urllib.error as urllib_error +import tkinter +import tkinter.dialog as tkinter_dialog +import tkinter.filedialog as tkinter_filedialog +import tkinter.scrolledtext as tkinter_scrolledtext +import tkinter.simpledialog as tkinder_simpledialog +import tkinter.tix as tkinter_tix +import tkinter.ttk as tkinter_ttk +import tkinter.constants as tkinter_constants +import tkinter.dnd as tkinter_dnd +import tkinter.colorchooser as tkinter_colorchooser +import tkinter.commondialog as tkinter_commondialog +import tkinter.filedialog as tkinter_tkfiledialog +import tkinter.font as tkinter_font +import tkinter.messagebox as tkinter_messagebox +import urllib +import urllib.request as urllib_request +import urllib.robotparser as urllib_robotparser +import urllib.parse as urllib_parse +import urllib.error as urllib_error +""" + + +def six_moves_transform(): + code = dedent( + """ + class Moves(object): + {} + moves = Moves() + """ + ).format(_indent(_IMPORTS, " ")) + module = AstroidBuilder(MANAGER).string_build(code) + module.name = "six.moves" + return module + + +def _six_fail_hook(modname): + """Fix six.moves imports due to the dynamic nature of this + class. + + Construct a pseudo-module which contains all the necessary imports + for six + + :param modname: Name of failed module + :type modname: str + + :return: An astroid module + :rtype: nodes.Module + """ + + attribute_of = modname != "six.moves" and modname.startswith("six.moves") + if modname != "six.moves" and not attribute_of: + raise AstroidBuildingError(modname=modname) + module = AstroidBuilder(MANAGER).string_build(_IMPORTS) + module.name = "six.moves" + if attribute_of: + # Facilitate import of submodules in Moves + start_index = len(module.name) + attribute = modname[start_index:].lstrip(".").replace(".", "_") + try: + import_attr = module.getattr(attribute)[0] + except AttributeInferenceError: + raise AstroidBuildingError(modname=modname) + if isinstance(import_attr, nodes.Import): + submodule = MANAGER.ast_from_module_name(import_attr.names[0][0]) + return submodule + # Let dummy submodule imports pass through + # This will cause an Uninferable result, which is okay + return module + + +def _looks_like_decorated_with_six_add_metaclass(node): + if not node.decorators: + return False + + for decorator in node.decorators.nodes: + if not isinstance(decorator, nodes.Call): + continue + if decorator.func.as_string() == SIX_ADD_METACLASS: + return True + return False + + +def transform_six_add_metaclass(node): + """Check if the given class node is decorated with *six.add_metaclass* + + If so, inject its argument as the metaclass of the underlying class. + """ + if not node.decorators: + return + + for decorator in node.decorators.nodes: + if not isinstance(decorator, nodes.Call): + continue + + try: + func = next(decorator.func.infer()) + except InferenceError: + continue + if func.qname() == SIX_ADD_METACLASS and decorator.args: + metaclass = decorator.args[0] + node._metaclass = metaclass + return node + + +register_module_extender(MANAGER, "six", six_moves_transform) +register_module_extender( + MANAGER, "requests.packages.urllib3.packages.six", six_moves_transform +) +MANAGER.register_failed_import_hook(_six_fail_hook) +MANAGER.register_transform( + nodes.ClassDef, + transform_six_add_metaclass, + _looks_like_decorated_with_six_add_metaclass, +) diff --git a/env/lib/python3.8/site-packages/astroid/brain/brain_ssl.py b/env/lib/python3.8/site-packages/astroid/brain/brain_ssl.py new file mode 100644 index 000000000..2ae21c353 --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/brain/brain_ssl.py @@ -0,0 +1,75 @@ +# Copyright (c) 2016, 2018 Claudiu Popa +# Copyright (c) 2016 Ceridwen +# Copyright (c) 2019 Benjamin Elven <25181435+S3ntinelX@users.noreply.github.com> + +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER + +"""Astroid hooks for the ssl library.""" + +from astroid import MANAGER, register_module_extender +from astroid.builder import AstroidBuilder +from astroid import nodes +from astroid import parse + + +def ssl_transform(): + return parse( + """ + from _ssl import OPENSSL_VERSION_NUMBER, OPENSSL_VERSION_INFO, OPENSSL_VERSION + from _ssl import _SSLContext, MemoryBIO + from _ssl import ( + SSLError, SSLZeroReturnError, SSLWantReadError, SSLWantWriteError, + SSLSyscallError, SSLEOFError, + ) + from _ssl import CERT_NONE, CERT_OPTIONAL, CERT_REQUIRED + from _ssl import txt2obj as _txt2obj, nid2obj as _nid2obj + from _ssl import RAND_status, RAND_add, RAND_bytes, RAND_pseudo_bytes + try: + from _ssl import RAND_egd + except ImportError: + # LibreSSL does not provide RAND_egd + pass + from _ssl import (OP_ALL, OP_CIPHER_SERVER_PREFERENCE, + OP_NO_COMPRESSION, OP_NO_SSLv2, OP_NO_SSLv3, + OP_NO_TLSv1, OP_NO_TLSv1_1, OP_NO_TLSv1_2, + OP_SINGLE_DH_USE, OP_SINGLE_ECDH_USE) + + from _ssl import (ALERT_DESCRIPTION_ACCESS_DENIED, ALERT_DESCRIPTION_BAD_CERTIFICATE, + ALERT_DESCRIPTION_BAD_CERTIFICATE_HASH_VALUE, + ALERT_DESCRIPTION_BAD_CERTIFICATE_STATUS_RESPONSE, + ALERT_DESCRIPTION_BAD_RECORD_MAC, + ALERT_DESCRIPTION_CERTIFICATE_EXPIRED, + ALERT_DESCRIPTION_CERTIFICATE_REVOKED, + ALERT_DESCRIPTION_CERTIFICATE_UNKNOWN, + ALERT_DESCRIPTION_CERTIFICATE_UNOBTAINABLE, + ALERT_DESCRIPTION_CLOSE_NOTIFY, ALERT_DESCRIPTION_DECODE_ERROR, + ALERT_DESCRIPTION_DECOMPRESSION_FAILURE, + ALERT_DESCRIPTION_DECRYPT_ERROR, + ALERT_DESCRIPTION_HANDSHAKE_FAILURE, + ALERT_DESCRIPTION_ILLEGAL_PARAMETER, + ALERT_DESCRIPTION_INSUFFICIENT_SECURITY, + ALERT_DESCRIPTION_INTERNAL_ERROR, + ALERT_DESCRIPTION_NO_RENEGOTIATION, + ALERT_DESCRIPTION_PROTOCOL_VERSION, + ALERT_DESCRIPTION_RECORD_OVERFLOW, + ALERT_DESCRIPTION_UNEXPECTED_MESSAGE, + ALERT_DESCRIPTION_UNKNOWN_CA, + ALERT_DESCRIPTION_UNKNOWN_PSK_IDENTITY, + ALERT_DESCRIPTION_UNRECOGNIZED_NAME, + ALERT_DESCRIPTION_UNSUPPORTED_CERTIFICATE, + ALERT_DESCRIPTION_UNSUPPORTED_EXTENSION, + ALERT_DESCRIPTION_USER_CANCELLED) + from _ssl import (SSL_ERROR_EOF, SSL_ERROR_INVALID_ERROR_CODE, SSL_ERROR_SSL, + SSL_ERROR_SYSCALL, SSL_ERROR_WANT_CONNECT, SSL_ERROR_WANT_READ, + SSL_ERROR_WANT_WRITE, SSL_ERROR_WANT_X509_LOOKUP, SSL_ERROR_ZERO_RETURN) + from _ssl import VERIFY_CRL_CHECK_CHAIN, VERIFY_CRL_CHECK_LEAF, VERIFY_DEFAULT, VERIFY_X509_STRICT + from _ssl import HAS_SNI, HAS_ECDH, HAS_NPN, HAS_ALPN + from _ssl import _OPENSSL_API_VERSION + from _ssl import PROTOCOL_SSLv23, PROTOCOL_TLSv1, PROTOCOL_TLSv1_1, PROTOCOL_TLSv1_2 + from _ssl import PROTOCOL_TLS, PROTOCOL_TLS_CLIENT, PROTOCOL_TLS_SERVER + """ + ) + + +register_module_extender(MANAGER, "ssl", ssl_transform) diff --git a/env/lib/python3.8/site-packages/astroid/brain/brain_subprocess.py b/env/lib/python3.8/site-packages/astroid/brain/brain_subprocess.py new file mode 100644 index 000000000..ab7d5d7e4 --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/brain/brain_subprocess.py @@ -0,0 +1,146 @@ +# Copyright (c) 2016-2020 Claudiu Popa +# Copyright (c) 2017 Hugo +# Copyright (c) 2018 Peter Talley +# Copyright (c) 2018 Bryce Guinta +# Copyright (c) 2019 Hugo van Kemenade + +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER + +import sys +import textwrap + +import astroid + + +PY37 = sys.version_info >= (3, 7) +PY36 = sys.version_info >= (3, 6) + + +def _subprocess_transform(): + communicate = (bytes("string", "ascii"), bytes("string", "ascii")) + communicate_signature = "def communicate(self, input=None, timeout=None)" + if PY37: + init = """ + def __init__(self, args, bufsize=0, executable=None, + stdin=None, stdout=None, stderr=None, + preexec_fn=None, close_fds=False, shell=False, + cwd=None, env=None, universal_newlines=False, + startupinfo=None, creationflags=0, restore_signals=True, + start_new_session=False, pass_fds=(), *, + encoding=None, errors=None, text=None): + pass + """ + elif PY36: + init = """ + def __init__(self, args, bufsize=0, executable=None, + stdin=None, stdout=None, stderr=None, + preexec_fn=None, close_fds=False, shell=False, + cwd=None, env=None, universal_newlines=False, + startupinfo=None, creationflags=0, restore_signals=True, + start_new_session=False, pass_fds=(), *, + encoding=None, errors=None): + pass + """ + else: + init = """ + def __init__(self, args, bufsize=0, executable=None, + stdin=None, stdout=None, stderr=None, + preexec_fn=None, close_fds=False, shell=False, + cwd=None, env=None, universal_newlines=False, + startupinfo=None, creationflags=0, restore_signals=True, + start_new_session=False, pass_fds=()): + pass + """ + wait_signature = "def wait(self, timeout=None)" + ctx_manager = """ + def __enter__(self): return self + def __exit__(self, *args): pass + """ + py3_args = "args = []" + + if PY37: + check_output_signature = """ + check_output( + args, *, + stdin=None, + stderr=None, + shell=False, + cwd=None, + encoding=None, + errors=None, + universal_newlines=False, + timeout=None, + env=None, + text=None, + restore_signals=True, + preexec_fn=None, + pass_fds=(), + input=None, + start_new_session=False + ): + """.strip() + else: + check_output_signature = """ + check_output( + args, *, + stdin=None, + stderr=None, + shell=False, + cwd=None, + encoding=None, + errors=None, + universal_newlines=False, + timeout=None, + env=None, + restore_signals=True, + preexec_fn=None, + pass_fds=(), + input=None, + start_new_session=False + ): + """.strip() + + code = textwrap.dedent( + """ + def %(check_output_signature)s + if universal_newlines: + return "" + return b"" + + class Popen(object): + returncode = pid = 0 + stdin = stdout = stderr = file() + %(py3_args)s + + %(communicate_signature)s: + return %(communicate)r + %(wait_signature)s: + return self.returncode + def poll(self): + return self.returncode + def send_signal(self, signal): + pass + def terminate(self): + pass + def kill(self): + pass + %(ctx_manager)s + """ + % { + "check_output_signature": check_output_signature, + "communicate": communicate, + "communicate_signature": communicate_signature, + "wait_signature": wait_signature, + "ctx_manager": ctx_manager, + "py3_args": py3_args, + } + ) + + init_lines = textwrap.dedent(init).splitlines() + indented_init = "\n".join(" " * 4 + line for line in init_lines) + code += indented_init + return astroid.parse(code) + + +astroid.register_module_extender(astroid.MANAGER, "subprocess", _subprocess_transform) diff --git a/env/lib/python3.8/site-packages/astroid/brain/brain_threading.py b/env/lib/python3.8/site-packages/astroid/brain/brain_threading.py new file mode 100644 index 000000000..ba3085b5e --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/brain/brain_threading.py @@ -0,0 +1,31 @@ +# -*- coding: utf-8 -*- +# Copyright (c) 2016, 2018-2019 Claudiu Popa +# Copyright (c) 2017 Łukasz Rogalski + +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER +import astroid + + +def _thread_transform(): + return astroid.parse( + """ + class lock(object): + def acquire(self, blocking=True, timeout=-1): + return False + def release(self): + pass + def __enter__(self): + return True + def __exit__(self, *args): + pass + def locked(self): + return False + + def Lock(): + return lock() + """ + ) + + +astroid.register_module_extender(astroid.MANAGER, "threading", _thread_transform) diff --git a/env/lib/python3.8/site-packages/astroid/brain/brain_typing.py b/env/lib/python3.8/site-packages/astroid/brain/brain_typing.py new file mode 100644 index 000000000..9ff72274d --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/brain/brain_typing.py @@ -0,0 +1,96 @@ +# -*- coding: utf-8 -*- +# Copyright (c) 2017-2018 Claudiu Popa +# Copyright (c) 2017 Łukasz Rogalski +# Copyright (c) 2017 David Euresti +# Copyright (c) 2018 Bryce Guinta + +"""Astroid hooks for typing.py support.""" +import typing + +from astroid import ( + MANAGER, + UseInferenceDefault, + extract_node, + inference_tip, + nodes, + InferenceError, +) + + +TYPING_NAMEDTUPLE_BASENAMES = {"NamedTuple", "typing.NamedTuple"} +TYPING_TYPEVARS = {"TypeVar", "NewType"} +TYPING_TYPEVARS_QUALIFIED = {"typing.TypeVar", "typing.NewType"} +TYPING_TYPE_TEMPLATE = """ +class Meta(type): + def __getitem__(self, item): + return self + + @property + def __args__(self): + return () + +class {0}(metaclass=Meta): + pass +""" +TYPING_MEMBERS = set(typing.__all__) + + +def looks_like_typing_typevar_or_newtype(node): + func = node.func + if isinstance(func, nodes.Attribute): + return func.attrname in TYPING_TYPEVARS + if isinstance(func, nodes.Name): + return func.name in TYPING_TYPEVARS + return False + + +def infer_typing_typevar_or_newtype(node, context=None): + """Infer a typing.TypeVar(...) or typing.NewType(...) call""" + try: + func = next(node.func.infer(context=context)) + except InferenceError as exc: + raise UseInferenceDefault from exc + + if func.qname() not in TYPING_TYPEVARS_QUALIFIED: + raise UseInferenceDefault + if not node.args: + raise UseInferenceDefault + + typename = node.args[0].as_string().strip("'") + node = extract_node(TYPING_TYPE_TEMPLATE.format(typename)) + return node.infer(context=context) + + +def _looks_like_typing_subscript(node): + """Try to figure out if a Subscript node *might* be a typing-related subscript""" + if isinstance(node, nodes.Name): + return node.name in TYPING_MEMBERS + elif isinstance(node, nodes.Attribute): + return node.attrname in TYPING_MEMBERS + elif isinstance(node, nodes.Subscript): + return _looks_like_typing_subscript(node.value) + return False + + +def infer_typing_attr(node, context=None): + """Infer a typing.X[...] subscript""" + try: + value = next(node.value.infer()) + except InferenceError as exc: + raise UseInferenceDefault from exc + + if not value.qname().startswith("typing."): + raise UseInferenceDefault + + node = extract_node(TYPING_TYPE_TEMPLATE.format(value.qname().split(".")[-1])) + return node.infer(context=context) + + +MANAGER.register_transform( + nodes.Call, + inference_tip(infer_typing_typevar_or_newtype), + looks_like_typing_typevar_or_newtype, +) +MANAGER.register_transform( + nodes.Subscript, inference_tip(infer_typing_attr), _looks_like_typing_subscript +) diff --git a/env/lib/python3.8/site-packages/astroid/brain/brain_uuid.py b/env/lib/python3.8/site-packages/astroid/brain/brain_uuid.py new file mode 100644 index 000000000..5a33fc251 --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/brain/brain_uuid.py @@ -0,0 +1,20 @@ +# Copyright (c) 2017-2018 Claudiu Popa + +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER + +"""Astroid hooks for the UUID module.""" + + +from astroid import MANAGER +from astroid import nodes + + +def _patch_uuid_class(node): + # The .int member is patched using __dict__ + node.locals["int"] = [nodes.Const(0, parent=node)] + + +MANAGER.register_transform( + nodes.ClassDef, _patch_uuid_class, lambda node: node.qname() == "uuid.UUID" +) diff --git a/env/lib/python3.8/site-packages/astroid/builder.py b/env/lib/python3.8/site-packages/astroid/builder.py new file mode 100644 index 000000000..142764b14 --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/builder.py @@ -0,0 +1,455 @@ +# -*- coding: utf-8 -*- +# Copyright (c) 2006-2011, 2013-2014 LOGILAB S.A. (Paris, FRANCE) +# Copyright (c) 2013 Phil Schaf +# Copyright (c) 2014-2019 Claudiu Popa +# Copyright (c) 2014-2015 Google, Inc. +# Copyright (c) 2014 Alexander Presnyakov +# Copyright (c) 2015-2016 Ceridwen +# Copyright (c) 2016 Derek Gustafson +# Copyright (c) 2017 Łukasz Rogalski +# Copyright (c) 2018 Anthony Sottile + +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER + +"""The AstroidBuilder makes astroid from living object and / or from _ast + +The builder is not thread safe and can't be used to parse different sources +at the same time. +""" + +import os +import textwrap +from tokenize import detect_encoding + +from astroid._ast import get_parser_module +from astroid import bases +from astroid import exceptions +from astroid import manager +from astroid import modutils +from astroid import raw_building +from astroid import rebuilder +from astroid import nodes +from astroid import util + +objects = util.lazy_import("objects") + +# The name of the transient function that is used to +# wrap expressions to be extracted when calling +# extract_node. +_TRANSIENT_FUNCTION = "__" + +# The comment used to select a statement to be extracted +# when calling extract_node. +_STATEMENT_SELECTOR = "#@" +MISPLACED_TYPE_ANNOTATION_ERROR = "misplaced type annotation" +MANAGER = manager.AstroidManager() + + +def open_source_file(filename): + with open(filename, "rb") as byte_stream: + encoding = detect_encoding(byte_stream.readline)[0] + stream = open(filename, "r", newline=None, encoding=encoding) + data = stream.read() + return stream, encoding, data + + +def _can_assign_attr(node, attrname): + try: + slots = node.slots() + except NotImplementedError: + pass + else: + if slots and attrname not in {slot.value for slot in slots}: + return False + return True + + +class AstroidBuilder(raw_building.InspectBuilder): + """Class for building an astroid tree from source code or from a live module. + + The param *manager* specifies the manager class which should be used. + If no manager is given, then the default one will be used. The + param *apply_transforms* determines if the transforms should be + applied after the tree was built from source or from a live object, + by default being True. + """ + + # pylint: disable=redefined-outer-name + def __init__(self, manager=None, apply_transforms=True): + super().__init__() + self._manager = manager or MANAGER + self._apply_transforms = apply_transforms + + def module_build(self, module, modname=None): + """Build an astroid from a living module instance.""" + node = None + path = getattr(module, "__file__", None) + if path is not None: + path_, ext = os.path.splitext(modutils._path_from_filename(path)) + if ext in (".py", ".pyc", ".pyo") and os.path.exists(path_ + ".py"): + node = self.file_build(path_ + ".py", modname) + if node is None: + # this is a built-in module + # get a partial representation by introspection + node = self.inspect_build(module, modname=modname, path=path) + if self._apply_transforms: + # We have to handle transformation by ourselves since the + # rebuilder isn't called for builtin nodes + node = self._manager.visit_transforms(node) + return node + + def file_build(self, path, modname=None): + """Build astroid from a source code file (i.e. from an ast) + + *path* is expected to be a python source file + """ + try: + stream, encoding, data = open_source_file(path) + except IOError as exc: + raise exceptions.AstroidBuildingError( + "Unable to load file {path}:\n{error}", + modname=modname, + path=path, + error=exc, + ) from exc + except (SyntaxError, LookupError) as exc: + raise exceptions.AstroidSyntaxError( + "Python 3 encoding specification error or unknown encoding:\n" + "{error}", + modname=modname, + path=path, + error=exc, + ) from exc + except UnicodeError as exc: # wrong encoding + # detect_encoding returns utf-8 if no encoding specified + raise exceptions.AstroidBuildingError( + "Wrong or no encoding specified for {filename}.", filename=path + ) from exc + with stream: + # get module name if necessary + if modname is None: + try: + modname = ".".join(modutils.modpath_from_file(path)) + except ImportError: + modname = os.path.splitext(os.path.basename(path))[0] + # build astroid representation + module = self._data_build(data, modname, path) + return self._post_build(module, encoding) + + def string_build(self, data, modname="", path=None): + """Build astroid from source code string.""" + module = self._data_build(data, modname, path) + module.file_bytes = data.encode("utf-8") + return self._post_build(module, "utf-8") + + def _post_build(self, module, encoding): + """Handles encoding and delayed nodes after a module has been built""" + module.file_encoding = encoding + self._manager.cache_module(module) + # post tree building steps after we stored the module in the cache: + for from_node in module._import_from_nodes: + if from_node.modname == "__future__": + for symbol, _ in from_node.names: + module.future_imports.add(symbol) + self.add_from_names_to_locals(from_node) + # handle delayed assattr nodes + for delayed in module._delayed_assattr: + self.delayed_assattr(delayed) + + # Visit the transforms + if self._apply_transforms: + module = self._manager.visit_transforms(module) + return module + + def _data_build(self, data, modname, path): + """Build tree node from data and add some informations""" + try: + node, parser_module = _parse_string(data, type_comments=True) + except (TypeError, ValueError, SyntaxError) as exc: + raise exceptions.AstroidSyntaxError( + "Parsing Python code failed:\n{error}", + source=data, + modname=modname, + path=path, + error=exc, + ) from exc + + if path is not None: + node_file = os.path.abspath(path) + else: + node_file = "" + if modname.endswith(".__init__"): + modname = modname[:-9] + package = True + else: + package = ( + path is not None + and os.path.splitext(os.path.basename(path))[0] == "__init__" + ) + builder = rebuilder.TreeRebuilder(self._manager, parser_module) + module = builder.visit_module(node, modname, node_file, package) + module._import_from_nodes = builder._import_from_nodes + module._delayed_assattr = builder._delayed_assattr + return module + + def add_from_names_to_locals(self, node): + """Store imported names to the locals + + Resort the locals if coming from a delayed node + """ + _key_func = lambda node: node.fromlineno + + def sort_locals(my_list): + my_list.sort(key=_key_func) + + for (name, asname) in node.names: + if name == "*": + try: + imported = node.do_import_module() + except exceptions.AstroidBuildingError: + continue + for name in imported.public_names(): + node.parent.set_local(name, node) + sort_locals(node.parent.scope().locals[name]) + else: + node.parent.set_local(asname or name, node) + sort_locals(node.parent.scope().locals[asname or name]) + + def delayed_assattr(self, node): + """Visit a AssAttr node + + This adds name to locals and handle members definition. + """ + try: + frame = node.frame() + for inferred in node.expr.infer(): + if inferred is util.Uninferable: + continue + try: + cls = inferred.__class__ + if cls is bases.Instance or cls is objects.ExceptionInstance: + inferred = inferred._proxied + iattrs = inferred.instance_attrs + if not _can_assign_attr(inferred, node.attrname): + continue + elif isinstance(inferred, bases.Instance): + # Const, Tuple or other containers that inherit from + # `Instance` + continue + elif inferred.is_function: + iattrs = inferred.instance_attrs + else: + iattrs = inferred.locals + except AttributeError: + # XXX log error + continue + values = iattrs.setdefault(node.attrname, []) + if node in values: + continue + # get assign in __init__ first XXX useful ? + if ( + frame.name == "__init__" + and values + and values[0].frame().name != "__init__" + ): + values.insert(0, node) + else: + values.append(node) + except exceptions.InferenceError: + pass + + +def build_namespace_package_module(name, path): + return nodes.Module(name, doc="", path=path, package=True) + + +def parse(code, module_name="", path=None, apply_transforms=True): + """Parses a source string in order to obtain an astroid AST from it + + :param str code: The code for the module. + :param str module_name: The name for the module, if any + :param str path: The path for the module + :param bool apply_transforms: + Apply the transforms for the give code. Use it if you + don't want the default transforms to be applied. + """ + code = textwrap.dedent(code) + builder = AstroidBuilder(manager=MANAGER, apply_transforms=apply_transforms) + return builder.string_build(code, modname=module_name, path=path) + + +def _extract_expressions(node): + """Find expressions in a call to _TRANSIENT_FUNCTION and extract them. + + The function walks the AST recursively to search for expressions that + are wrapped into a call to _TRANSIENT_FUNCTION. If it finds such an + expression, it completely removes the function call node from the tree, + replacing it by the wrapped expression inside the parent. + + :param node: An astroid node. + :type node: astroid.bases.NodeNG + :yields: The sequence of wrapped expressions on the modified tree + expression can be found. + """ + if ( + isinstance(node, nodes.Call) + and isinstance(node.func, nodes.Name) + and node.func.name == _TRANSIENT_FUNCTION + ): + real_expr = node.args[0] + real_expr.parent = node.parent + # Search for node in all _astng_fields (the fields checked when + # get_children is called) of its parent. Some of those fields may + # be lists or tuples, in which case the elements need to be checked. + # When we find it, replace it by real_expr, so that the AST looks + # like no call to _TRANSIENT_FUNCTION ever took place. + for name in node.parent._astroid_fields: + child = getattr(node.parent, name) + if isinstance(child, (list, tuple)): + for idx, compound_child in enumerate(child): + if compound_child is node: + child[idx] = real_expr + elif child is node: + setattr(node.parent, name, real_expr) + yield real_expr + else: + for child in node.get_children(): + yield from _extract_expressions(child) + + +def _find_statement_by_line(node, line): + """Extracts the statement on a specific line from an AST. + + If the line number of node matches line, it will be returned; + otherwise its children are iterated and the function is called + recursively. + + :param node: An astroid node. + :type node: astroid.bases.NodeNG + :param line: The line number of the statement to extract. + :type line: int + :returns: The statement on the line, or None if no statement for the line + can be found. + :rtype: astroid.bases.NodeNG or None + """ + if isinstance(node, (nodes.ClassDef, nodes.FunctionDef)): + # This is an inaccuracy in the AST: the nodes that can be + # decorated do not carry explicit information on which line + # the actual definition (class/def), but .fromline seems to + # be close enough. + node_line = node.fromlineno + else: + node_line = node.lineno + + if node_line == line: + return node + + for child in node.get_children(): + result = _find_statement_by_line(child, line) + if result: + return result + + return None + + +def extract_node(code, module_name=""): + """Parses some Python code as a module and extracts a designated AST node. + + Statements: + To extract one or more statement nodes, append #@ to the end of the line + + Examples: + >>> def x(): + >>> def y(): + >>> return 1 #@ + + The return statement will be extracted. + + >>> class X(object): + >>> def meth(self): #@ + >>> pass + + The function object 'meth' will be extracted. + + Expressions: + To extract arbitrary expressions, surround them with the fake + function call __(...). After parsing, the surrounded expression + will be returned and the whole AST (accessible via the returned + node's parent attribute) will look like the function call was + never there in the first place. + + Examples: + >>> a = __(1) + + The const node will be extracted. + + >>> def x(d=__(foo.bar)): pass + + The node containing the default argument will be extracted. + + >>> def foo(a, b): + >>> return 0 < __(len(a)) < b + + The node containing the function call 'len' will be extracted. + + If no statements or expressions are selected, the last toplevel + statement will be returned. + + If the selected statement is a discard statement, (i.e. an expression + turned into a statement), the wrapped expression is returned instead. + + For convenience, singleton lists are unpacked. + + :param str code: A piece of Python code that is parsed as + a module. Will be passed through textwrap.dedent first. + :param str module_name: The name of the module. + :returns: The designated node from the parse tree, or a list of nodes. + :rtype: astroid.bases.NodeNG, or a list of nodes. + """ + + def _extract(node): + if isinstance(node, nodes.Expr): + return node.value + + return node + + requested_lines = [] + for idx, line in enumerate(code.splitlines()): + if line.strip().endswith(_STATEMENT_SELECTOR): + requested_lines.append(idx + 1) + + tree = parse(code, module_name=module_name) + if not tree.body: + raise ValueError("Empty tree, cannot extract from it") + + extracted = [] + if requested_lines: + extracted = [_find_statement_by_line(tree, line) for line in requested_lines] + + # Modifies the tree. + extracted.extend(_extract_expressions(tree)) + + if not extracted: + extracted.append(tree.body[-1]) + + extracted = [_extract(node) for node in extracted] + if len(extracted) == 1: + return extracted[0] + return extracted + + +def _parse_string(data, type_comments=True): + parser_module = get_parser_module(type_comments=type_comments) + try: + parsed = parser_module.parse(data + "\n", type_comments=type_comments) + except SyntaxError as exc: + # If the type annotations are misplaced for some reason, we do not want + # to fail the entire parsing of the file, so we need to retry the parsing without + # type comment support. + if exc.args[0] != MISPLACED_TYPE_ANNOTATION_ERROR or not type_comments: + raise + + parser_module = get_parser_module(type_comments=False) + parsed = parser_module.parse(data + "\n", type_comments=False) + return parsed, parser_module diff --git a/env/lib/python3.8/site-packages/astroid/context.py b/env/lib/python3.8/site-packages/astroid/context.py new file mode 100644 index 000000000..40cebf222 --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/context.py @@ -0,0 +1,179 @@ +# Copyright (c) 2015-2016, 2018-2019 Claudiu Popa +# Copyright (c) 2015-2016 Ceridwen +# Copyright (c) 2018 Bryce Guinta +# Copyright (c) 2018 Nick Drozd + +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER + +"""Various context related utilities, including inference and call contexts.""" +import contextlib +import pprint +from typing import Optional + + +class InferenceContext: + """Provide context for inference + + Store already inferred nodes to save time + Account for already visited nodes to infinite stop infinite recursion + """ + + __slots__ = ( + "path", + "lookupname", + "callcontext", + "boundnode", + "inferred", + "extra_context", + ) + + def __init__(self, path=None, inferred=None): + self.path = path or set() + """ + :type: set(tuple(NodeNG, optional(str))) + + Path of visited nodes and their lookupname + + Currently this key is ``(node, context.lookupname)`` + """ + self.lookupname = None + """ + :type: optional[str] + + The original name of the node + + e.g. + foo = 1 + The inference of 'foo' is nodes.Const(1) but the lookup name is 'foo' + """ + self.callcontext = None + """ + :type: optional[CallContext] + + The call arguments and keywords for the given context + """ + self.boundnode = None + """ + :type: optional[NodeNG] + + The bound node of the given context + + e.g. the bound node of object.__new__(cls) is the object node + """ + self.inferred = inferred or {} + """ + :type: dict(seq, seq) + + Inferred node contexts to their mapped results + Currently the key is ``(node, lookupname, callcontext, boundnode)`` + and the value is tuple of the inferred results + """ + self.extra_context = {} + """ + :type: dict(NodeNG, Context) + + Context that needs to be passed down through call stacks + for call arguments + """ + + def push(self, node): + """Push node into inference path + + :return: True if node is already in context path else False + :rtype: bool + + Allows one to see if the given node has already + been looked at for this inference context""" + name = self.lookupname + if (node, name) in self.path: + return True + + self.path.add((node, name)) + return False + + def clone(self): + """Clone inference path + + For example, each side of a binary operation (BinOp) + starts with the same context but diverge as each side is inferred + so the InferenceContext will need be cloned""" + # XXX copy lookupname/callcontext ? + clone = InferenceContext(self.path, inferred=self.inferred) + clone.callcontext = self.callcontext + clone.boundnode = self.boundnode + clone.extra_context = self.extra_context + return clone + + def cache_generator(self, key, generator): + """Cache result of generator into dictionary + + Used to cache inference results""" + results = [] + for result in generator: + results.append(result) + yield result + + self.inferred[key] = tuple(results) + + @contextlib.contextmanager + def restore_path(self): + path = set(self.path) + yield + self.path = path + + def __str__(self): + state = ( + "%s=%s" + % (field, pprint.pformat(getattr(self, field), width=80 - len(field))) + for field in self.__slots__ + ) + return "%s(%s)" % (type(self).__name__, ",\n ".join(state)) + + +class CallContext: + """Holds information for a call site.""" + + __slots__ = ("args", "keywords") + + def __init__(self, args, keywords=None): + """ + :param List[NodeNG] args: Call positional arguments + :param Union[List[nodes.Keyword], None] keywords: Call keywords + """ + self.args = args + if keywords: + keywords = [(arg.arg, arg.value) for arg in keywords] + else: + keywords = [] + self.keywords = keywords + + +def copy_context(context: Optional[InferenceContext]) -> InferenceContext: + """Clone a context if given, or return a fresh contexxt""" + if context is not None: + return context.clone() + + return InferenceContext() + + +def bind_context_to_node(context, node): + """Give a context a boundnode + to retrieve the correct function name or attribute value + with from further inference. + + Do not use an existing context since the boundnode could then + be incorrectly propagated higher up in the call stack. + + :param context: Context to use + :type context: Optional(context) + + :param node: Node to do name lookups from + :type node NodeNG: + + :returns: A new context + :rtype: InferenceContext + """ + context = copy_context(context) + context.boundnode = node + return context diff --git a/env/lib/python3.8/site-packages/astroid/decorators.py b/env/lib/python3.8/site-packages/astroid/decorators.py new file mode 100644 index 000000000..0f3632c48 --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/decorators.py @@ -0,0 +1,142 @@ +# Copyright (c) 2015-2016, 2018 Claudiu Popa +# Copyright (c) 2015-2016 Ceridwen +# Copyright (c) 2015 Florian Bruhin +# Copyright (c) 2016 Derek Gustafson +# Copyright (c) 2018 Nick Drozd +# Copyright (c) 2018 Tomas Gavenciak +# Copyright (c) 2018 Ashley Whetter +# Copyright (c) 2018 HoverHell +# Copyright (c) 2018 Bryce Guinta + +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER + +""" A few useful function/method decorators.""" + +import functools + +import wrapt + +from astroid import context as contextmod +from astroid import exceptions +from astroid import util + + +@wrapt.decorator +def cached(func, instance, args, kwargs): + """Simple decorator to cache result of method calls without args.""" + cache = getattr(instance, "__cache", None) + if cache is None: + instance.__cache = cache = {} + try: + return cache[func] + except KeyError: + cache[func] = result = func(*args, **kwargs) + return result + + +class cachedproperty: + """ Provides a cached property equivalent to the stacking of + @cached and @property, but more efficient. + + After first usage, the becomes part of the object's + __dict__. Doing: + + del obj. empties the cache. + + Idea taken from the pyramid_ framework and the mercurial_ project. + + .. _pyramid: http://pypi.python.org/pypi/pyramid + .. _mercurial: http://pypi.python.org/pypi/Mercurial + """ + + __slots__ = ("wrapped",) + + def __init__(self, wrapped): + try: + wrapped.__name__ + except AttributeError as exc: + raise TypeError("%s must have a __name__ attribute" % wrapped) from exc + self.wrapped = wrapped + + @property + def __doc__(self): + doc = getattr(self.wrapped, "__doc__", None) + return "%s" % ( + "\n%s" % doc if doc else "" + ) + + def __get__(self, inst, objtype=None): + if inst is None: + return self + val = self.wrapped(inst) + setattr(inst, self.wrapped.__name__, val) + return val + + +def path_wrapper(func): + """return the given infer function wrapped to handle the path + + Used to stop inference if the node has already been looked + at for a given `InferenceContext` to prevent infinite recursion + """ + + @functools.wraps(func) + def wrapped(node, context=None, _func=func, **kwargs): + """wrapper function handling context""" + if context is None: + context = contextmod.InferenceContext() + if context.push(node): + return None + + yielded = set() + generator = _func(node, context, **kwargs) + try: + while True: + res = next(generator) + # unproxy only true instance, not const, tuple, dict... + if res.__class__.__name__ == "Instance": + ares = res._proxied + else: + ares = res + if ares not in yielded: + yield res + yielded.add(ares) + except StopIteration as error: + if error.args: + return error.args[0] + return None + + return wrapped + + +@wrapt.decorator +def yes_if_nothing_inferred(func, instance, args, kwargs): + generator = func(*args, **kwargs) + + try: + yield next(generator) + except StopIteration: + # generator is empty + yield util.Uninferable + return + + yield from generator + + +@wrapt.decorator +def raise_if_nothing_inferred(func, instance, args, kwargs): + generator = func(*args, **kwargs) + + try: + yield next(generator) + except StopIteration as error: + # generator is empty + if error.args: + # pylint: disable=not-a-mapping + raise exceptions.InferenceError(**error.args[0]) + raise exceptions.InferenceError( + "StopIteration raised without any error information." + ) + + yield from generator diff --git a/env/lib/python3.8/site-packages/astroid/exceptions.py b/env/lib/python3.8/site-packages/astroid/exceptions.py new file mode 100644 index 000000000..08e72c137 --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/exceptions.py @@ -0,0 +1,230 @@ +# Copyright (c) 2007, 2009-2010, 2013 LOGILAB S.A. (Paris, FRANCE) +# Copyright (c) 2014 Google, Inc. +# Copyright (c) 2015-2018 Claudiu Popa +# Copyright (c) 2015-2016 Ceridwen +# Copyright (c) 2016 Derek Gustafson +# Copyright (c) 2018 Bryce Guinta + +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER + +"""this module contains exceptions used in the astroid library +""" +from astroid import util + + +class AstroidError(Exception): + """base exception class for all astroid related exceptions + + AstroidError and its subclasses are structured, intended to hold + objects representing state when the exception is thrown. Field + values are passed to the constructor as keyword-only arguments. + Each subclass has its own set of standard fields, but use your + best judgment to decide whether a specific exception instance + needs more or fewer fields for debugging. Field values may be + used to lazily generate the error message: self.message.format() + will be called with the field names and values supplied as keyword + arguments. + """ + + def __init__(self, message="", **kws): + super().__init__(message) + self.message = message + for key, value in kws.items(): + setattr(self, key, value) + + def __str__(self): + return self.message.format(**vars(self)) + + +class AstroidBuildingError(AstroidError): + """exception class when we are unable to build an astroid representation + + Standard attributes: + modname: Name of the module that AST construction failed for. + error: Exception raised during construction. + """ + + def __init__(self, message="Failed to import module {modname}.", **kws): + super().__init__(message, **kws) + + +class AstroidImportError(AstroidBuildingError): + """Exception class used when a module can't be imported by astroid.""" + + +class TooManyLevelsError(AstroidImportError): + """Exception class which is raised when a relative import was beyond the top-level. + + Standard attributes: + level: The level which was attempted. + name: the name of the module on which the relative import was attempted. + """ + + level = None + name = None + + def __init__( + self, + message="Relative import with too many levels " "({level}) for module {name!r}", + **kws + ): + super().__init__(message, **kws) + + +class AstroidSyntaxError(AstroidBuildingError): + """Exception class used when a module can't be parsed.""" + + +class NoDefault(AstroidError): + """raised by function's `default_value` method when an argument has + no default value + + Standard attributes: + func: Function node. + name: Name of argument without a default. + """ + + func = None + name = None + + def __init__(self, message="{func!r} has no default for {name!r}.", **kws): + super().__init__(message, **kws) + + +class ResolveError(AstroidError): + """Base class of astroid resolution/inference error. + + ResolveError is not intended to be raised. + + Standard attributes: + context: InferenceContext object. + """ + + context = None + + +class MroError(ResolveError): + """Error raised when there is a problem with method resolution of a class. + + Standard attributes: + mros: A sequence of sequences containing ClassDef nodes. + cls: ClassDef node whose MRO resolution failed. + context: InferenceContext object. + """ + + mros = () + cls = None + + def __str__(self): + mro_names = ", ".join( + "({})".format(", ".join(b.name for b in m)) for m in self.mros + ) + return self.message.format(mros=mro_names, cls=self.cls) + + +class DuplicateBasesError(MroError): + """Error raised when there are duplicate bases in the same class bases.""" + + +class InconsistentMroError(MroError): + """Error raised when a class's MRO is inconsistent.""" + + +class SuperError(ResolveError): + """Error raised when there is a problem with a *super* call. + + Standard attributes: + *super_*: The Super instance that raised the exception. + context: InferenceContext object. + """ + + super_ = None + + def __str__(self): + return self.message.format(**vars(self.super_)) + + +class InferenceError(ResolveError): + """raised when we are unable to infer a node + + Standard attributes: + node: The node inference was called on. + context: InferenceContext object. + """ + + node = None + context = None + + def __init__(self, message="Inference failed for {node!r}.", **kws): + super().__init__(message, **kws) + + +# Why does this inherit from InferenceError rather than ResolveError? +# Changing it causes some inference tests to fail. +class NameInferenceError(InferenceError): + """Raised when a name lookup fails, corresponds to NameError. + + Standard attributes: + name: The name for which lookup failed, as a string. + scope: The node representing the scope in which the lookup occurred. + context: InferenceContext object. + """ + + name = None + scope = None + + def __init__(self, message="{name!r} not found in {scope!r}.", **kws): + super().__init__(message, **kws) + + +class AttributeInferenceError(ResolveError): + """Raised when an attribute lookup fails, corresponds to AttributeError. + + Standard attributes: + target: The node for which lookup failed. + attribute: The attribute for which lookup failed, as a string. + context: InferenceContext object. + """ + + target = None + attribute = None + + def __init__(self, message="{attribute!r} not found on {target!r}.", **kws): + super().__init__(message, **kws) + + +class UseInferenceDefault(Exception): + """exception to be raised in custom inference function to indicate that it + should go back to the default behaviour + """ + + +class _NonDeducibleTypeHierarchy(Exception): + """Raised when is_subtype / is_supertype can't deduce the relation between two types.""" + + +class AstroidIndexError(AstroidError): + """Raised when an Indexable / Mapping does not have an index / key.""" + + +class AstroidTypeError(AstroidError): + """Raised when a TypeError would be expected in Python code.""" + + +class InferenceOverwriteError(AstroidError): + """Raised when an inference tip is overwritten + + Currently only used for debugging. + """ + + +# Backwards-compatibility aliases +OperationError = util.BadOperationMessage +UnaryOperationError = util.BadUnaryOperationMessage +BinaryOperationError = util.BadBinaryOperationMessage + +SuperArgumentTypeError = SuperError +UnresolvableName = NameInferenceError +NotFoundError = AttributeInferenceError +AstroidBuildingException = AstroidBuildingError diff --git a/env/lib/python3.8/site-packages/astroid/helpers.py b/env/lib/python3.8/site-packages/astroid/helpers.py new file mode 100644 index 000000000..1c84651d0 --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/helpers.py @@ -0,0 +1,282 @@ +# Copyright (c) 2015-2020 Claudiu Popa +# Copyright (c) 2015-2016 Ceridwen +# Copyright (c) 2018 Bryce Guinta + +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER + + +""" +Various helper utilities. +""" + +import builtins as builtins_mod + +from astroid import bases +from astroid import context as contextmod +from astroid import exceptions +from astroid import manager +from astroid import nodes +from astroid import raw_building +from astroid import scoped_nodes +from astroid import util + + +BUILTINS = builtins_mod.__name__ + + +def _build_proxy_class(cls_name, builtins): + proxy = raw_building.build_class(cls_name) + proxy.parent = builtins + return proxy + + +def _function_type(function, builtins): + if isinstance(function, scoped_nodes.Lambda): + if function.root().name == BUILTINS: + cls_name = "builtin_function_or_method" + else: + cls_name = "function" + elif isinstance(function, bases.BoundMethod): + cls_name = "method" + elif isinstance(function, bases.UnboundMethod): + cls_name = "function" + return _build_proxy_class(cls_name, builtins) + + +def _object_type(node, context=None): + astroid_manager = manager.AstroidManager() + builtins = astroid_manager.builtins_module + context = context or contextmod.InferenceContext() + + for inferred in node.infer(context=context): + if isinstance(inferred, scoped_nodes.ClassDef): + if inferred.newstyle: + metaclass = inferred.metaclass(context=context) + if metaclass: + yield metaclass + continue + yield builtins.getattr("type")[0] + elif isinstance(inferred, (scoped_nodes.Lambda, bases.UnboundMethod)): + yield _function_type(inferred, builtins) + elif isinstance(inferred, scoped_nodes.Module): + yield _build_proxy_class("module", builtins) + else: + yield inferred._proxied + + +def object_type(node, context=None): + """Obtain the type of the given node + + This is used to implement the ``type`` builtin, which means that it's + used for inferring type calls, as well as used in a couple of other places + in the inference. + The node will be inferred first, so this function can support all + sorts of objects, as long as they support inference. + """ + + try: + types = set(_object_type(node, context)) + except exceptions.InferenceError: + return util.Uninferable + if len(types) > 1 or not types: + return util.Uninferable + return list(types)[0] + + +def _object_type_is_subclass(obj_type, class_or_seq, context=None): + if not isinstance(class_or_seq, (tuple, list)): + class_seq = (class_or_seq,) + else: + class_seq = class_or_seq + + if obj_type is util.Uninferable: + return util.Uninferable + + # Instances are not types + class_seq = [ + item if not isinstance(item, bases.Instance) else util.Uninferable + for item in class_seq + ] + # strict compatibility with issubclass + # issubclass(type, (object, 1)) evaluates to true + # issubclass(object, (1, type)) raises TypeError + for klass in class_seq: + if klass is util.Uninferable: + raise exceptions.AstroidTypeError("arg 2 must be a type or tuple of types") + + for obj_subclass in obj_type.mro(): + if obj_subclass == klass: + return True + return False + + +def object_isinstance(node, class_or_seq, context=None): + """Check if a node 'isinstance' any node in class_or_seq + + :param node: A given node + :param class_or_seq: Union[nodes.NodeNG, Sequence[nodes.NodeNG]] + :rtype: bool + + :raises AstroidTypeError: if the given ``classes_or_seq`` are not types + """ + obj_type = object_type(node, context) + if obj_type is util.Uninferable: + return util.Uninferable + return _object_type_is_subclass(obj_type, class_or_seq, context=context) + + +def object_issubclass(node, class_or_seq, context=None): + """Check if a type is a subclass of any node in class_or_seq + + :param node: A given node + :param class_or_seq: Union[Nodes.NodeNG, Sequence[nodes.NodeNG]] + :rtype: bool + + :raises AstroidTypeError: if the given ``classes_or_seq`` are not types + :raises AstroidError: if the type of the given node cannot be inferred + or its type's mro doesn't work + """ + if not isinstance(node, nodes.ClassDef): + raise TypeError("{node} needs to be a ClassDef node".format(node=node)) + return _object_type_is_subclass(node, class_or_seq, context=context) + + +def safe_infer(node, context=None): + """Return the inferred value for the given node. + + Return None if inference failed or if there is some ambiguity (more than + one node has been inferred). + """ + try: + inferit = node.infer(context=context) + value = next(inferit) + except exceptions.InferenceError: + return None + try: + next(inferit) + return None # None if there is ambiguity on the inferred node + except exceptions.InferenceError: + return None # there is some kind of ambiguity + except StopIteration: + return value + + +def has_known_bases(klass, context=None): + """Return true if all base classes of a class could be inferred.""" + try: + return klass._all_bases_known + except AttributeError: + pass + for base in klass.bases: + result = safe_infer(base, context=context) + # TODO: check for A->B->A->B pattern in class structure too? + if ( + not isinstance(result, scoped_nodes.ClassDef) + or result is klass + or not has_known_bases(result, context=context) + ): + klass._all_bases_known = False + return False + klass._all_bases_known = True + return True + + +def _type_check(type1, type2): + if not all(map(has_known_bases, (type1, type2))): + raise exceptions._NonDeducibleTypeHierarchy + + if not all([type1.newstyle, type2.newstyle]): + return False + try: + return type1 in type2.mro()[:-1] + except exceptions.MroError: + # The MRO is invalid. + raise exceptions._NonDeducibleTypeHierarchy + + +def is_subtype(type1, type2): + """Check if *type1* is a subtype of *type2*.""" + return _type_check(type1=type2, type2=type1) + + +def is_supertype(type1, type2): + """Check if *type2* is a supertype of *type1*.""" + return _type_check(type1, type2) + + +def class_instance_as_index(node): + """Get the value as an index for the given instance. + + If an instance provides an __index__ method, then it can + be used in some scenarios where an integer is expected, + for instance when multiplying or subscripting a list. + """ + context = contextmod.InferenceContext() + context.callcontext = contextmod.CallContext(args=[node]) + + try: + for inferred in node.igetattr("__index__", context=context): + if not isinstance(inferred, bases.BoundMethod): + continue + + for result in inferred.infer_call_result(node, context=context): + if isinstance(result, nodes.Const) and isinstance(result.value, int): + return result + except exceptions.InferenceError: + pass + return None + + +def object_len(node, context=None): + """Infer length of given node object + + :param Union[nodes.ClassDef, nodes.Instance] node: + :param node: Node to infer length of + + :raises AstroidTypeError: If an invalid node is returned + from __len__ method or no __len__ method exists + :raises InferenceError: If the given node cannot be inferred + or if multiple nodes are inferred + :rtype int: Integer length of node + """ + # pylint: disable=import-outside-toplevel; circular import + from astroid.objects import FrozenSet + + inferred_node = safe_infer(node, context=context) + if inferred_node is None or inferred_node is util.Uninferable: + raise exceptions.InferenceError(node=node) + if isinstance(inferred_node, nodes.Const) and isinstance( + inferred_node.value, (bytes, str) + ): + return len(inferred_node.value) + if isinstance(inferred_node, (nodes.List, nodes.Set, nodes.Tuple, FrozenSet)): + return len(inferred_node.elts) + if isinstance(inferred_node, nodes.Dict): + return len(inferred_node.items) + + node_type = object_type(inferred_node, context=context) + if not node_type: + raise exceptions.InferenceError(node=node) + + try: + len_call = next(node_type.igetattr("__len__", context=context)) + except exceptions.AttributeInferenceError: + raise exceptions.AstroidTypeError( + "object of type '{}' has no len()".format(node_type.pytype()) + ) + + result_of_len = next(len_call.infer_call_result(node, context)) + if ( + isinstance(result_of_len, nodes.Const) + and result_of_len.pytype() == "builtins.int" + ): + return result_of_len.value + if isinstance(result_of_len, bases.Instance) and result_of_len.is_subtype_of( + "builtins.int" + ): + # Fake a result as we don't know the arguments of the instance call. + return 0 + raise exceptions.AstroidTypeError( + "'{}' object cannot be interpreted as an integer".format(result_of_len) + ) diff --git a/env/lib/python3.8/site-packages/astroid/inference.py b/env/lib/python3.8/site-packages/astroid/inference.py new file mode 100644 index 000000000..bc3e1f970 --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/inference.py @@ -0,0 +1,994 @@ +# -*- coding: utf-8 -*- +# Copyright (c) 2006-2011, 2013-2014 LOGILAB S.A. (Paris, FRANCE) +# Copyright (c) 2012 FELD Boris +# Copyright (c) 2013-2014 Google, Inc. +# Copyright (c) 2014-2020 Claudiu Popa +# Copyright (c) 2014 Eevee (Alex Munroe) +# Copyright (c) 2015-2016 Ceridwen +# Copyright (c) 2015 Dmitry Pribysh +# Copyright (c) 2016 Jakub Wilk +# Copyright (c) 2017 Michał Masłowski +# Copyright (c) 2017 Calen Pennington +# Copyright (c) 2017 Łukasz Rogalski +# Copyright (c) 2018-2019 Nick Drozd +# Copyright (c) 2018 Daniel Martin +# Copyright (c) 2018 Ville Skyttä +# Copyright (c) 2018 Bryce Guinta +# Copyright (c) 2018 Ashley Whetter +# Copyright (c) 2018 HoverHell +# Copyright (c) 2020 Leandro T. C. Melo + +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER + +"""this module contains a set of functions to handle inference on astroid trees +""" + +import functools +import itertools +import operator + +import wrapt +from astroid import bases +from astroid import context as contextmod +from astroid import exceptions +from astroid import decorators +from astroid import helpers +from astroid import manager +from astroid import nodes +from astroid.interpreter import dunder_lookup +from astroid import protocols +from astroid import util + + +MANAGER = manager.AstroidManager() +# Prevents circular imports +objects = util.lazy_import("objects") + + +# .infer method ############################################################### + + +def infer_end(self, context=None): + """Inference's end for nodes that yield themselves on inference + + These are objects for which inference does not have any semantic, + such as Module or Consts. + """ + yield self + + +nodes.Module._infer = infer_end +nodes.ClassDef._infer = infer_end +nodes.Lambda._infer = infer_end +nodes.Const._infer = infer_end +nodes.Slice._infer = infer_end + + +def _infer_sequence_helper(node, context=None): + """Infer all values based on _BaseContainer.elts""" + values = [] + + for elt in node.elts: + if isinstance(elt, nodes.Starred): + starred = helpers.safe_infer(elt.value, context) + if not starred: + raise exceptions.InferenceError(node=node, context=context) + if not hasattr(starred, "elts"): + raise exceptions.InferenceError(node=node, context=context) + values.extend(_infer_sequence_helper(starred)) + elif isinstance(elt, nodes.NamedExpr): + value = helpers.safe_infer(elt.value, context) + if not value: + raise exceptions.InferenceError(node=node, context=context) + values.append(value) + else: + values.append(elt) + return values + + +@decorators.raise_if_nothing_inferred +def infer_sequence(self, context=None): + has_starred_named_expr = any( + isinstance(e, (nodes.Starred, nodes.NamedExpr)) for e in self.elts + ) + if has_starred_named_expr: + values = _infer_sequence_helper(self, context) + new_seq = type(self)( + lineno=self.lineno, col_offset=self.col_offset, parent=self.parent + ) + new_seq.postinit(values) + + yield new_seq + else: + yield self + + +nodes.List._infer = infer_sequence +nodes.Tuple._infer = infer_sequence +nodes.Set._infer = infer_sequence + + +def infer_map(self, context=None): + if not any(isinstance(k, nodes.DictUnpack) for k, _ in self.items): + yield self + else: + items = _infer_map(self, context) + new_seq = type(self)(self.lineno, self.col_offset, self.parent) + new_seq.postinit(list(items.items())) + yield new_seq + + +def _update_with_replacement(lhs_dict, rhs_dict): + """Delete nodes that equate to duplicate keys + + Since an astroid node doesn't 'equal' another node with the same value, + this function uses the as_string method to make sure duplicate keys + don't get through + + Note that both the key and the value are astroid nodes + + Fixes issue with DictUnpack causing duplicte keys + in inferred Dict items + + :param dict(nodes.NodeNG, nodes.NodeNG) lhs_dict: Dictionary to 'merge' nodes into + :param dict(nodes.NodeNG, nodes.NodeNG) rhs_dict: Dictionary with nodes to pull from + :return dict(nodes.NodeNG, nodes.NodeNG): merged dictionary of nodes + """ + combined_dict = itertools.chain(lhs_dict.items(), rhs_dict.items()) + # Overwrite keys which have the same string values + string_map = {key.as_string(): (key, value) for key, value in combined_dict} + # Return to dictionary + return dict(string_map.values()) + + +def _infer_map(node, context): + """Infer all values based on Dict.items""" + values = {} + for name, value in node.items: + if isinstance(name, nodes.DictUnpack): + double_starred = helpers.safe_infer(value, context) + if not double_starred: + raise exceptions.InferenceError + if not isinstance(double_starred, nodes.Dict): + raise exceptions.InferenceError(node=node, context=context) + unpack_items = _infer_map(double_starred, context) + values = _update_with_replacement(values, unpack_items) + else: + key = helpers.safe_infer(name, context=context) + value = helpers.safe_infer(value, context=context) + if any(not elem for elem in (key, value)): + raise exceptions.InferenceError(node=node, context=context) + values = _update_with_replacement(values, {key: value}) + return values + + +nodes.Dict._infer = infer_map + + +def _higher_function_scope(node): + """ Search for the first function which encloses the given + scope. This can be used for looking up in that function's + scope, in case looking up in a lower scope for a particular + name fails. + + :param node: A scope node. + :returns: + ``None``, if no parent function scope was found, + otherwise an instance of :class:`astroid.scoped_nodes.Function`, + which encloses the given node. + """ + current = node + while current.parent and not isinstance(current.parent, nodes.FunctionDef): + current = current.parent + if current and current.parent: + return current.parent + return None + + +def infer_name(self, context=None): + """infer a Name: use name lookup rules""" + frame, stmts = self.lookup(self.name) + if not stmts: + # Try to see if the name is enclosed in a nested function + # and use the higher (first function) scope for searching. + parent_function = _higher_function_scope(self.scope()) + if parent_function: + _, stmts = parent_function.lookup(self.name) + + if not stmts: + raise exceptions.NameInferenceError( + name=self.name, scope=self.scope(), context=context + ) + context = contextmod.copy_context(context) + context.lookupname = self.name + return bases._infer_stmts(stmts, context, frame) + + +# pylint: disable=no-value-for-parameter +nodes.Name._infer = decorators.raise_if_nothing_inferred( + decorators.path_wrapper(infer_name) +) +nodes.AssignName.infer_lhs = infer_name # won't work with a path wrapper + + +@decorators.raise_if_nothing_inferred +@decorators.path_wrapper +def infer_call(self, context=None): + """infer a Call node by trying to guess what the function returns""" + callcontext = contextmod.copy_context(context) + callcontext.callcontext = contextmod.CallContext( + args=self.args, keywords=self.keywords + ) + callcontext.boundnode = None + if context is not None: + callcontext.extra_context = _populate_context_lookup(self, context.clone()) + + for callee in self.func.infer(context): + if callee is util.Uninferable: + yield callee + continue + try: + if hasattr(callee, "infer_call_result"): + yield from callee.infer_call_result(caller=self, context=callcontext) + except exceptions.InferenceError: + continue + return dict(node=self, context=context) + + +nodes.Call._infer = infer_call + + +@decorators.raise_if_nothing_inferred +@decorators.path_wrapper +def infer_import(self, context=None, asname=True): + """infer an Import node: return the imported module/object""" + name = context.lookupname + if name is None: + raise exceptions.InferenceError(node=self, context=context) + + try: + if asname: + yield self.do_import_module(self.real_name(name)) + else: + yield self.do_import_module(name) + except exceptions.AstroidBuildingError as exc: + raise exceptions.InferenceError(node=self, context=context) from exc + + +nodes.Import._infer = infer_import + + +@decorators.raise_if_nothing_inferred +@decorators.path_wrapper +def infer_import_from(self, context=None, asname=True): + """infer a ImportFrom node: return the imported module/object""" + name = context.lookupname + if name is None: + raise exceptions.InferenceError(node=self, context=context) + if asname: + name = self.real_name(name) + + try: + module = self.do_import_module() + except exceptions.AstroidBuildingError as exc: + raise exceptions.InferenceError(node=self, context=context) from exc + + try: + context = contextmod.copy_context(context) + context.lookupname = name + stmts = module.getattr(name, ignore_locals=module is self.root()) + return bases._infer_stmts(stmts, context) + except exceptions.AttributeInferenceError as error: + raise exceptions.InferenceError( + error.message, target=self, attribute=name, context=context + ) from error + + +nodes.ImportFrom._infer = infer_import_from + + +def infer_attribute(self, context=None): + """infer an Attribute node by using getattr on the associated object""" + for owner in self.expr.infer(context): + if owner is util.Uninferable: + yield owner + continue + + if context and context.boundnode: + # This handles the situation where the attribute is accessed through a subclass + # of a base class and the attribute is defined at the base class's level, + # by taking in consideration a redefinition in the subclass. + if isinstance(owner, bases.Instance) and isinstance( + context.boundnode, bases.Instance + ): + try: + if helpers.is_subtype( + helpers.object_type(context.boundnode), + helpers.object_type(owner), + ): + owner = context.boundnode + except exceptions._NonDeducibleTypeHierarchy: + # Can't determine anything useful. + pass + elif not context: + context = contextmod.InferenceContext() + + try: + context.boundnode = owner + yield from owner.igetattr(self.attrname, context) + except ( + exceptions.AttributeInferenceError, + exceptions.InferenceError, + AttributeError, + ): + pass + finally: + context.boundnode = None + return dict(node=self, context=context) + + +nodes.Attribute._infer = decorators.raise_if_nothing_inferred( + decorators.path_wrapper(infer_attribute) +) +# won't work with a path wrapper +nodes.AssignAttr.infer_lhs = decorators.raise_if_nothing_inferred(infer_attribute) + + +@decorators.raise_if_nothing_inferred +@decorators.path_wrapper +def infer_global(self, context=None): + if context.lookupname is None: + raise exceptions.InferenceError(node=self, context=context) + try: + return bases._infer_stmts(self.root().getattr(context.lookupname), context) + except exceptions.AttributeInferenceError as error: + raise exceptions.InferenceError( + error.message, target=self, attribute=context.lookupname, context=context + ) from error + + +nodes.Global._infer = infer_global + + +_SUBSCRIPT_SENTINEL = object() + + +def infer_subscript(self, context=None): + """Inference for subscripts + + We're understanding if the index is a Const + or a slice, passing the result of inference + to the value's `getitem` method, which should + handle each supported index type accordingly. + """ + + found_one = False + for value in self.value.infer(context): + if value is util.Uninferable: + yield util.Uninferable + return None + for index in self.slice.infer(context): + if index is util.Uninferable: + yield util.Uninferable + return None + + # Try to deduce the index value. + index_value = _SUBSCRIPT_SENTINEL + if value.__class__ == bases.Instance: + index_value = index + elif index.__class__ == bases.Instance: + instance_as_index = helpers.class_instance_as_index(index) + if instance_as_index: + index_value = instance_as_index + else: + index_value = index + + if index_value is _SUBSCRIPT_SENTINEL: + raise exceptions.InferenceError(node=self, context=context) + + try: + assigned = value.getitem(index_value, context) + except ( + exceptions.AstroidTypeError, + exceptions.AstroidIndexError, + exceptions.AttributeInferenceError, + AttributeError, + ) as exc: + raise exceptions.InferenceError(node=self, context=context) from exc + + # Prevent inferring if the inferred subscript + # is the same as the original subscripted object. + if self is assigned or assigned is util.Uninferable: + yield util.Uninferable + return None + yield from assigned.infer(context) + found_one = True + + if found_one: + return dict(node=self, context=context) + return None + + +nodes.Subscript._infer = decorators.raise_if_nothing_inferred( + decorators.path_wrapper(infer_subscript) +) +nodes.Subscript.infer_lhs = decorators.raise_if_nothing_inferred(infer_subscript) + + +@decorators.raise_if_nothing_inferred +@decorators.path_wrapper +def _infer_boolop(self, context=None): + """Infer a boolean operation (and / or / not). + + The function will calculate the boolean operation + for all pairs generated through inference for each component + node. + """ + values = self.values + if self.op == "or": + predicate = operator.truth + else: + predicate = operator.not_ + + try: + values = [value.infer(context=context) for value in values] + except exceptions.InferenceError: + yield util.Uninferable + return None + + for pair in itertools.product(*values): + if any(item is util.Uninferable for item in pair): + # Can't infer the final result, just yield Uninferable. + yield util.Uninferable + continue + + bool_values = [item.bool_value() for item in pair] + if any(item is util.Uninferable for item in bool_values): + # Can't infer the final result, just yield Uninferable. + yield util.Uninferable + continue + + # Since the boolean operations are short circuited operations, + # this code yields the first value for which the predicate is True + # and if no value respected the predicate, then the last value will + # be returned (or Uninferable if there was no last value). + # This is conforming to the semantics of `and` and `or`: + # 1 and 0 -> 1 + # 0 and 1 -> 0 + # 1 or 0 -> 1 + # 0 or 1 -> 1 + value = util.Uninferable + for value, bool_value in zip(pair, bool_values): + if predicate(bool_value): + yield value + break + else: + yield value + + return dict(node=self, context=context) + + +nodes.BoolOp._infer = _infer_boolop + + +# UnaryOp, BinOp and AugAssign inferences + + +def _filter_operation_errors(self, infer_callable, context, error): + for result in infer_callable(self, context): + if isinstance(result, error): + # For the sake of .infer(), we don't care about operation + # errors, which is the job of pylint. So return something + # which shows that we can't infer the result. + yield util.Uninferable + else: + yield result + + +def _infer_unaryop(self, context=None): + """Infer what an UnaryOp should return when evaluated.""" + for operand in self.operand.infer(context): + try: + yield operand.infer_unary_op(self.op) + except TypeError as exc: + # The operand doesn't support this operation. + yield util.BadUnaryOperationMessage(operand, self.op, exc) + except AttributeError as exc: + meth = protocols.UNARY_OP_METHOD[self.op] + if meth is None: + # `not node`. Determine node's boolean + # value and negate its result, unless it is + # Uninferable, which will be returned as is. + bool_value = operand.bool_value() + if bool_value is not util.Uninferable: + yield nodes.const_factory(not bool_value) + else: + yield util.Uninferable + else: + if not isinstance(operand, (bases.Instance, nodes.ClassDef)): + # The operation was used on something which + # doesn't support it. + yield util.BadUnaryOperationMessage(operand, self.op, exc) + continue + + try: + try: + methods = dunder_lookup.lookup(operand, meth) + except exceptions.AttributeInferenceError: + yield util.BadUnaryOperationMessage(operand, self.op, exc) + continue + + meth = methods[0] + inferred = next(meth.infer(context=context)) + if inferred is util.Uninferable or not inferred.callable(): + continue + + context = contextmod.copy_context(context) + context.callcontext = contextmod.CallContext(args=[operand]) + call_results = inferred.infer_call_result(self, context=context) + result = next(call_results, None) + if result is None: + # Failed to infer, return the same type. + yield operand + else: + yield result + except exceptions.AttributeInferenceError as exc: + # The unary operation special method was not found. + yield util.BadUnaryOperationMessage(operand, self.op, exc) + except exceptions.InferenceError: + yield util.Uninferable + + +@decorators.raise_if_nothing_inferred +@decorators.path_wrapper +def infer_unaryop(self, context=None): + """Infer what an UnaryOp should return when evaluated.""" + yield from _filter_operation_errors( + self, _infer_unaryop, context, util.BadUnaryOperationMessage + ) + return dict(node=self, context=context) + + +nodes.UnaryOp._infer_unaryop = _infer_unaryop +nodes.UnaryOp._infer = infer_unaryop + + +def _is_not_implemented(const): + """Check if the given const node is NotImplemented.""" + return isinstance(const, nodes.Const) and const.value is NotImplemented + + +def _invoke_binop_inference(instance, opnode, op, other, context, method_name): + """Invoke binary operation inference on the given instance.""" + methods = dunder_lookup.lookup(instance, method_name) + context = contextmod.bind_context_to_node(context, instance) + method = methods[0] + inferred = next(method.infer(context=context)) + if inferred is util.Uninferable: + raise exceptions.InferenceError + return instance.infer_binary_op(opnode, op, other, context, inferred) + + +def _aug_op(instance, opnode, op, other, context, reverse=False): + """Get an inference callable for an augmented binary operation.""" + method_name = protocols.AUGMENTED_OP_METHOD[op] + return functools.partial( + _invoke_binop_inference, + instance=instance, + op=op, + opnode=opnode, + other=other, + context=context, + method_name=method_name, + ) + + +def _bin_op(instance, opnode, op, other, context, reverse=False): + """Get an inference callable for a normal binary operation. + + If *reverse* is True, then the reflected method will be used instead. + """ + if reverse: + method_name = protocols.REFLECTED_BIN_OP_METHOD[op] + else: + method_name = protocols.BIN_OP_METHOD[op] + return functools.partial( + _invoke_binop_inference, + instance=instance, + op=op, + opnode=opnode, + other=other, + context=context, + method_name=method_name, + ) + + +def _get_binop_contexts(context, left, right): + """Get contexts for binary operations. + + This will return two inference contexts, the first one + for x.__op__(y), the other one for y.__rop__(x), where + only the arguments are inversed. + """ + # The order is important, since the first one should be + # left.__op__(right). + for arg in (right, left): + new_context = context.clone() + new_context.callcontext = contextmod.CallContext(args=[arg]) + new_context.boundnode = None + yield new_context + + +def _same_type(type1, type2): + """Check if type1 is the same as type2.""" + return type1.qname() == type2.qname() + + +def _get_binop_flow( + left, left_type, binary_opnode, right, right_type, context, reverse_context +): + """Get the flow for binary operations. + + The rules are a bit messy: + + * if left and right have the same type, then only one + method will be called, left.__op__(right) + * if left and right are unrelated typewise, then first + left.__op__(right) is tried and if this does not exist + or returns NotImplemented, then right.__rop__(left) is tried. + * if left is a subtype of right, then only left.__op__(right) + is tried. + * if left is a supertype of right, then right.__rop__(left) + is first tried and then left.__op__(right) + """ + op = binary_opnode.op + if _same_type(left_type, right_type): + methods = [_bin_op(left, binary_opnode, op, right, context)] + elif helpers.is_subtype(left_type, right_type): + methods = [_bin_op(left, binary_opnode, op, right, context)] + elif helpers.is_supertype(left_type, right_type): + methods = [ + _bin_op(right, binary_opnode, op, left, reverse_context, reverse=True), + _bin_op(left, binary_opnode, op, right, context), + ] + else: + methods = [ + _bin_op(left, binary_opnode, op, right, context), + _bin_op(right, binary_opnode, op, left, reverse_context, reverse=True), + ] + return methods + + +def _get_aug_flow( + left, left_type, aug_opnode, right, right_type, context, reverse_context +): + """Get the flow for augmented binary operations. + + The rules are a bit messy: + + * if left and right have the same type, then left.__augop__(right) + is first tried and then left.__op__(right). + * if left and right are unrelated typewise, then + left.__augop__(right) is tried, then left.__op__(right) + is tried and then right.__rop__(left) is tried. + * if left is a subtype of right, then left.__augop__(right) + is tried and then left.__op__(right). + * if left is a supertype of right, then left.__augop__(right) + is tried, then right.__rop__(left) and then + left.__op__(right) + """ + bin_op = aug_opnode.op.strip("=") + aug_op = aug_opnode.op + if _same_type(left_type, right_type): + methods = [ + _aug_op(left, aug_opnode, aug_op, right, context), + _bin_op(left, aug_opnode, bin_op, right, context), + ] + elif helpers.is_subtype(left_type, right_type): + methods = [ + _aug_op(left, aug_opnode, aug_op, right, context), + _bin_op(left, aug_opnode, bin_op, right, context), + ] + elif helpers.is_supertype(left_type, right_type): + methods = [ + _aug_op(left, aug_opnode, aug_op, right, context), + _bin_op(right, aug_opnode, bin_op, left, reverse_context, reverse=True), + _bin_op(left, aug_opnode, bin_op, right, context), + ] + else: + methods = [ + _aug_op(left, aug_opnode, aug_op, right, context), + _bin_op(left, aug_opnode, bin_op, right, context), + _bin_op(right, aug_opnode, bin_op, left, reverse_context, reverse=True), + ] + return methods + + +def _infer_binary_operation(left, right, binary_opnode, context, flow_factory): + """Infer a binary operation between a left operand and a right operand + + This is used by both normal binary operations and augmented binary + operations, the only difference is the flow factory used. + """ + + context, reverse_context = _get_binop_contexts(context, left, right) + left_type = helpers.object_type(left) + right_type = helpers.object_type(right) + methods = flow_factory( + left, left_type, binary_opnode, right, right_type, context, reverse_context + ) + for method in methods: + try: + results = list(method()) + except AttributeError: + continue + except exceptions.AttributeInferenceError: + continue + except exceptions.InferenceError: + yield util.Uninferable + return + else: + if any(result is util.Uninferable for result in results): + yield util.Uninferable + return + + if all(map(_is_not_implemented, results)): + continue + not_implemented = sum( + 1 for result in results if _is_not_implemented(result) + ) + if not_implemented and not_implemented != len(results): + # Can't infer yet what this is. + yield util.Uninferable + return + + yield from results + return + # The operation doesn't seem to be supported so let the caller know about it + yield util.BadBinaryOperationMessage(left_type, binary_opnode.op, right_type) + + +def _infer_binop(self, context): + """Binary operation inference logic.""" + left = self.left + right = self.right + + # we use two separate contexts for evaluating lhs and rhs because + # 1. evaluating lhs may leave some undesired entries in context.path + # which may not let us infer right value of rhs + context = context or contextmod.InferenceContext() + lhs_context = contextmod.copy_context(context) + rhs_context = contextmod.copy_context(context) + lhs_iter = left.infer(context=lhs_context) + rhs_iter = right.infer(context=rhs_context) + for lhs, rhs in itertools.product(lhs_iter, rhs_iter): + if any(value is util.Uninferable for value in (rhs, lhs)): + # Don't know how to process this. + yield util.Uninferable + return + + try: + yield from _infer_binary_operation(lhs, rhs, self, context, _get_binop_flow) + except exceptions._NonDeducibleTypeHierarchy: + yield util.Uninferable + + +@decorators.yes_if_nothing_inferred +@decorators.path_wrapper +def infer_binop(self, context=None): + return _filter_operation_errors( + self, _infer_binop, context, util.BadBinaryOperationMessage + ) + + +nodes.BinOp._infer_binop = _infer_binop +nodes.BinOp._infer = infer_binop + + +def _infer_augassign(self, context=None): + """Inference logic for augmented binary operations.""" + if context is None: + context = contextmod.InferenceContext() + + rhs_context = context.clone() + + lhs_iter = self.target.infer_lhs(context=context) + rhs_iter = self.value.infer(context=rhs_context) + for lhs, rhs in itertools.product(lhs_iter, rhs_iter): + if any(value is util.Uninferable for value in (rhs, lhs)): + # Don't know how to process this. + yield util.Uninferable + return + + try: + yield from _infer_binary_operation( + left=lhs, + right=rhs, + binary_opnode=self, + context=context, + flow_factory=_get_aug_flow, + ) + except exceptions._NonDeducibleTypeHierarchy: + yield util.Uninferable + + +@decorators.raise_if_nothing_inferred +@decorators.path_wrapper +def infer_augassign(self, context=None): + return _filter_operation_errors( + self, _infer_augassign, context, util.BadBinaryOperationMessage + ) + + +nodes.AugAssign._infer_augassign = _infer_augassign +nodes.AugAssign._infer = infer_augassign + +# End of binary operation inference. + + +@decorators.raise_if_nothing_inferred +def infer_arguments(self, context=None): + name = context.lookupname + if name is None: + raise exceptions.InferenceError(node=self, context=context) + return protocols._arguments_infer_argname(self, name, context) + + +nodes.Arguments._infer = infer_arguments + + +@decorators.raise_if_nothing_inferred +@decorators.path_wrapper +def infer_assign(self, context=None): + """infer a AssignName/AssignAttr: need to inspect the RHS part of the + assign node + """ + if isinstance(self.parent, nodes.AugAssign): + return self.parent.infer(context) + + stmts = list(self.assigned_stmts(context=context)) + return bases._infer_stmts(stmts, context) + + +nodes.AssignName._infer = infer_assign +nodes.AssignAttr._infer = infer_assign + + +@decorators.raise_if_nothing_inferred +@decorators.path_wrapper +def infer_empty_node(self, context=None): + if not self.has_underlying_object(): + yield util.Uninferable + else: + try: + yield from MANAGER.infer_ast_from_something(self.object, context=context) + except exceptions.AstroidError: + yield util.Uninferable + + +nodes.EmptyNode._infer = infer_empty_node + + +@decorators.raise_if_nothing_inferred +def infer_index(self, context=None): + return self.value.infer(context) + + +nodes.Index._infer = infer_index + +# TODO: move directly into bases.Instance when the dependency hell +# will be solved. +def instance_getitem(self, index, context=None): + # Rewrap index to Const for this case + new_context = contextmod.bind_context_to_node(context, self) + if not context: + context = new_context + + # Create a new callcontext for providing index as an argument. + new_context.callcontext = contextmod.CallContext(args=[index]) + + method = next(self.igetattr("__getitem__", context=context), None) + if not isinstance(method, bases.BoundMethod): + raise exceptions.InferenceError( + "Could not find __getitem__ for {node!r}.", node=self, context=context + ) + + return next(method.infer_call_result(self, new_context)) + + +bases.Instance.getitem = instance_getitem + + +def _populate_context_lookup(call, context): + # Allows context to be saved for later + # for inference inside a function + context_lookup = {} + if context is None: + return context_lookup + for arg in call.args: + if isinstance(arg, nodes.Starred): + context_lookup[arg.value] = context + else: + context_lookup[arg] = context + keywords = call.keywords if call.keywords is not None else [] + for keyword in keywords: + context_lookup[keyword.value] = context + return context_lookup + + +@decorators.raise_if_nothing_inferred +def infer_ifexp(self, context=None): + """Support IfExp inference + + If we can't infer the truthiness of the condition, we default + to inferring both branches. Otherwise, we infer either branch + depending on the condition. + """ + both_branches = False + # We use two separate contexts for evaluating lhs and rhs because + # evaluating lhs may leave some undesired entries in context.path + # which may not let us infer right value of rhs. + + context = context or contextmod.InferenceContext() + lhs_context = contextmod.copy_context(context) + rhs_context = contextmod.copy_context(context) + try: + test = next(self.test.infer(context=context.clone())) + except exceptions.InferenceError: + both_branches = True + else: + if test is not util.Uninferable: + if test.bool_value(): + yield from self.body.infer(context=lhs_context) + else: + yield from self.orelse.infer(context=rhs_context) + else: + both_branches = True + if both_branches: + yield from self.body.infer(context=lhs_context) + yield from self.orelse.infer(context=rhs_context) + + +nodes.IfExp._infer = infer_ifexp + + +# pylint: disable=dangerous-default-value +@wrapt.decorator +def _cached_generator(func, instance, args, kwargs, _cache={}): + node = args[0] + try: + return iter(_cache[func, id(node)]) + except KeyError: + result = func(*args, **kwargs) + # Need to keep an iterator around + original, copy = itertools.tee(result) + _cache[func, id(node)] = list(copy) + return original + + +# When inferring a property, we instantiate a new `objects.Property` object, +# which in turn, because it inherits from `FunctionDef`, sets itself in the locals +# of the wrapping frame. This means that everytime we infer a property, the locals +# are mutated with a new instance of the property. This is why we cache the result +# of the function's inference. +@_cached_generator +def infer_functiondef(self, context=None): + if not self.decorators or not bases._is_property(self): + yield self + return dict(node=self, context=context) + + prop_func = objects.Property( + function=self, + name=self.name, + doc=self.doc, + lineno=self.lineno, + parent=self.parent, + col_offset=self.col_offset, + ) + prop_func.postinit(body=[], args=self.args) + yield prop_func + return dict(node=self, context=context) + + +nodes.FunctionDef._infer = infer_functiondef diff --git a/env/lib/python3.8/site-packages/astroid/interpreter/__init__.py b/env/lib/python3.8/site-packages/astroid/interpreter/__init__.py new file mode 100644 index 000000000..e69de29bb diff --git a/env/lib/python3.8/site-packages/astroid/interpreter/_import/__init__.py b/env/lib/python3.8/site-packages/astroid/interpreter/_import/__init__.py new file mode 100644 index 000000000..e69de29bb diff --git a/env/lib/python3.8/site-packages/astroid/interpreter/_import/spec.py b/env/lib/python3.8/site-packages/astroid/interpreter/_import/spec.py new file mode 100644 index 000000000..3cf5fea57 --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/interpreter/_import/spec.py @@ -0,0 +1,346 @@ +# Copyright (c) 2016-2018 Claudiu Popa +# Copyright (c) 2016 Derek Gustafson +# Copyright (c) 2017 Chris Philip +# Copyright (c) 2017 Hugo +# Copyright (c) 2017 ioanatia +# Copyright (c) 2017 Calen Pennington +# Copyright (c) 2018 Nick Drozd +# Copyright (c) 2019 Hugo van Kemenade +# Copyright (c) 2019 Ashley Whetter + +import abc +import collections +import distutils +import enum +import imp +import os +import sys +import zipimport + +try: + import importlib.machinery + + _HAS_MACHINERY = True +except ImportError: + _HAS_MACHINERY = False + +try: + from functools import lru_cache +except ImportError: + from backports.functools_lru_cache import lru_cache + +from . import util + +ModuleType = enum.Enum( + "ModuleType", + "C_BUILTIN C_EXTENSION PKG_DIRECTORY " + "PY_CODERESOURCE PY_COMPILED PY_FROZEN PY_RESOURCE " + "PY_SOURCE PY_ZIPMODULE PY_NAMESPACE", +) +_ImpTypes = { + imp.C_BUILTIN: ModuleType.C_BUILTIN, + imp.C_EXTENSION: ModuleType.C_EXTENSION, + imp.PKG_DIRECTORY: ModuleType.PKG_DIRECTORY, + imp.PY_COMPILED: ModuleType.PY_COMPILED, + imp.PY_FROZEN: ModuleType.PY_FROZEN, + imp.PY_SOURCE: ModuleType.PY_SOURCE, +} +if hasattr(imp, "PY_RESOURCE"): + _ImpTypes[imp.PY_RESOURCE] = ModuleType.PY_RESOURCE +if hasattr(imp, "PY_CODERESOURCE"): + _ImpTypes[imp.PY_CODERESOURCE] = ModuleType.PY_CODERESOURCE + + +def _imp_type_to_module_type(imp_type): + return _ImpTypes[imp_type] + + +_ModuleSpec = collections.namedtuple( + "_ModuleSpec", "name type location " "origin submodule_search_locations" +) + + +class ModuleSpec(_ModuleSpec): + """Defines a class similar to PEP 420's ModuleSpec + + A module spec defines a name of a module, its type, location + and where submodules can be found, if the module is a package. + """ + + def __new__( + cls, + name, + module_type, + location=None, + origin=None, + submodule_search_locations=None, + ): + return _ModuleSpec.__new__( + cls, + name=name, + type=module_type, + location=location, + origin=origin, + submodule_search_locations=submodule_search_locations, + ) + + +class Finder: + """A finder is a class which knows how to find a particular module.""" + + def __init__(self, path=None): + self._path = path or sys.path + + @abc.abstractmethod + def find_module(self, modname, module_parts, processed, submodule_path): + """Find the given module + + Each finder is responsible for each protocol of finding, as long as + they all return a ModuleSpec. + + :param str modname: The module which needs to be searched. + :param list module_parts: It should be a list of strings, + where each part contributes to the module's + namespace. + :param list processed: What parts from the module parts were processed + so far. + :param list submodule_path: A list of paths where the module + can be looked into. + :returns: A ModuleSpec, describing how and where the module was found, + None, otherwise. + """ + + def contribute_to_path(self, spec, processed): + """Get a list of extra paths where this finder can search.""" + + +class ImpFinder(Finder): + """A finder based on the imp module.""" + + def find_module(self, modname, module_parts, processed, submodule_path): + if submodule_path is not None: + submodule_path = list(submodule_path) + try: + stream, mp_filename, mp_desc = imp.find_module(modname, submodule_path) + except ImportError: + return None + + # Close resources. + if stream: + stream.close() + + return ModuleSpec( + name=modname, + location=mp_filename, + module_type=_imp_type_to_module_type(mp_desc[2]), + ) + + def contribute_to_path(self, spec, processed): + if spec.location is None: + # Builtin. + return None + + if _is_setuptools_namespace(spec.location): + # extend_path is called, search sys.path for module/packages + # of this name see pkgutil.extend_path documentation + path = [ + os.path.join(p, *processed) + for p in sys.path + if os.path.isdir(os.path.join(p, *processed)) + ] + # We already import distutils elsewhere in astroid, + # so if it is the same module, we can use it directly. + elif spec.name == "distutils" and spec.location in distutils.__path__: + # distutils is patched inside virtualenvs to pick up submodules + # from the original Python, not from the virtualenv itself. + path = list(distutils.__path__) + else: + path = [spec.location] + return path + + +class ExplicitNamespacePackageFinder(ImpFinder): + """A finder for the explicit namespace packages, generated through pkg_resources.""" + + def find_module(self, modname, module_parts, processed, submodule_path): + if processed: + modname = ".".join(processed + [modname]) + if util.is_namespace(modname) and modname in sys.modules: + submodule_path = sys.modules[modname].__path__ + return ModuleSpec( + name=modname, + location="", + origin="namespace", + module_type=ModuleType.PY_NAMESPACE, + submodule_search_locations=submodule_path, + ) + return None + + def contribute_to_path(self, spec, processed): + return spec.submodule_search_locations + + +class ZipFinder(Finder): + """Finder that knows how to find a module inside zip files.""" + + def __init__(self, path): + super().__init__(path) + self._zipimporters = _precache_zipimporters(path) + + def find_module(self, modname, module_parts, processed, submodule_path): + try: + file_type, filename, path = _search_zip(module_parts, self._zipimporters) + except ImportError: + return None + + return ModuleSpec( + name=modname, + location=filename, + origin="egg", + module_type=file_type, + submodule_search_locations=path, + ) + + +class PathSpecFinder(Finder): + """Finder based on importlib.machinery.PathFinder.""" + + def find_module(self, modname, module_parts, processed, submodule_path): + spec = importlib.machinery.PathFinder.find_spec(modname, path=submodule_path) + if spec: + # origin can be either a string on older Python versions + # or None in case it is a namespace package: + # https://github.com/python/cpython/pull/5481 + is_namespace_pkg = spec.origin in ("namespace", None) + location = spec.origin if not is_namespace_pkg else None + module_type = ModuleType.PY_NAMESPACE if is_namespace_pkg else None + spec = ModuleSpec( + name=spec.name, + location=location, + origin=spec.origin, + module_type=module_type, + submodule_search_locations=list(spec.submodule_search_locations or []), + ) + return spec + + def contribute_to_path(self, spec, processed): + if spec.type == ModuleType.PY_NAMESPACE: + return spec.submodule_search_locations + return None + + +_SPEC_FINDERS = (ImpFinder, ZipFinder) +if _HAS_MACHINERY: + _SPEC_FINDERS += (PathSpecFinder,) +_SPEC_FINDERS += (ExplicitNamespacePackageFinder,) + + +def _is_setuptools_namespace(location): + try: + with open(os.path.join(location, "__init__.py"), "rb") as stream: + data = stream.read(4096) + except IOError: + pass + else: + extend_path = b"pkgutil" in data and b"extend_path" in data + declare_namespace = ( + b"pkg_resources" in data and b"declare_namespace(__name__)" in data + ) + return extend_path or declare_namespace + + +@lru_cache() +def _cached_set_diff(left, right): + result = set(left) + result.difference_update(right) + return result + + +def _precache_zipimporters(path=None): + pic = sys.path_importer_cache + + # When measured, despite having the same complexity (O(n)), + # converting to tuples and then caching the conversion to sets + # and the set difference is faster than converting to sets + # and then only caching the set difference. + + req_paths = tuple(path or sys.path) + cached_paths = tuple(pic) + new_paths = _cached_set_diff(req_paths, cached_paths) + for entry_path in new_paths: + try: + pic[entry_path] = zipimport.zipimporter(entry_path) + except zipimport.ZipImportError: + continue + return pic + + +def _search_zip(modpath, pic): + for filepath, importer in list(pic.items()): + if importer is not None: + found = importer.find_module(modpath[0]) + if found: + if not importer.find_module(os.path.sep.join(modpath)): + raise ImportError( + "No module named %s in %s/%s" + % (".".join(modpath[1:]), filepath, modpath) + ) + # import code; code.interact(local=locals()) + return ( + ModuleType.PY_ZIPMODULE, + os.path.abspath(filepath) + os.path.sep + os.path.sep.join(modpath), + filepath, + ) + raise ImportError("No module named %s" % ".".join(modpath)) + + +def _find_spec_with_path(search_path, modname, module_parts, processed, submodule_path): + finders = [finder(search_path) for finder in _SPEC_FINDERS] + for finder in finders: + spec = finder.find_module(modname, module_parts, processed, submodule_path) + if spec is None: + continue + return finder, spec + + raise ImportError("No module named %s" % ".".join(module_parts)) + + +def find_spec(modpath, path=None): + """Find a spec for the given module. + + :type modpath: list or tuple + :param modpath: + split module's name (i.e name of a module or package split + on '.'), with leading empty strings for explicit relative import + + :type path: list or None + :param path: + optional list of path where the module or package should be + searched (use sys.path if nothing or None is given) + + :rtype: ModuleSpec + :return: A module spec, which describes how the module was + found and where. + """ + _path = path or sys.path + + # Need a copy for not mutating the argument. + modpath = modpath[:] + + submodule_path = None + module_parts = modpath[:] + processed = [] + + while modpath: + modname = modpath.pop(0) + finder, spec = _find_spec_with_path( + _path, modname, module_parts, processed, submodule_path or path + ) + processed.append(modname) + if modpath: + submodule_path = finder.contribute_to_path(spec, processed) + + if spec.type == ModuleType.PKG_DIRECTORY: + spec = spec._replace(submodule_search_locations=submodule_path) + + return spec diff --git a/env/lib/python3.8/site-packages/astroid/interpreter/_import/util.py b/env/lib/python3.8/site-packages/astroid/interpreter/_import/util.py new file mode 100644 index 000000000..a917bd3d1 --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/interpreter/_import/util.py @@ -0,0 +1,10 @@ +# Copyright (c) 2016, 2018 Claudiu Popa + +try: + import pkg_resources +except ImportError: + pkg_resources = None + + +def is_namespace(modname): + return pkg_resources is not None and modname in pkg_resources._namespace_packages diff --git a/env/lib/python3.8/site-packages/astroid/interpreter/dunder_lookup.py b/env/lib/python3.8/site-packages/astroid/interpreter/dunder_lookup.py new file mode 100644 index 000000000..0ae9bc90d --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/interpreter/dunder_lookup.py @@ -0,0 +1,66 @@ +# Copyright (c) 2016-2018 Claudiu Popa +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER + +"""Contains logic for retrieving special methods. + +This implementation does not rely on the dot attribute access +logic, found in ``.getattr()``. The difference between these two +is that the dunder methods are looked with the type slots +(you can find more about these here +http://lucumr.pocoo.org/2014/8/16/the-python-i-would-like-to-see/) +As such, the lookup for the special methods is actually simpler than +the dot attribute access. +""" +import itertools + +import astroid +from astroid import exceptions + + +def _lookup_in_mro(node, name): + attrs = node.locals.get(name, []) + + nodes = itertools.chain.from_iterable( + ancestor.locals.get(name, []) for ancestor in node.ancestors(recurs=True) + ) + values = list(itertools.chain(attrs, nodes)) + if not values: + raise exceptions.AttributeInferenceError(attribute=name, target=node) + + return values + + +def lookup(node, name): + """Lookup the given special method name in the given *node* + + If the special method was found, then a list of attributes + will be returned. Otherwise, `astroid.AttributeInferenceError` + is going to be raised. + """ + if isinstance( + node, (astroid.List, astroid.Tuple, astroid.Const, astroid.Dict, astroid.Set) + ): + return _builtin_lookup(node, name) + if isinstance(node, astroid.Instance): + return _lookup_in_mro(node, name) + if isinstance(node, astroid.ClassDef): + return _class_lookup(node, name) + + raise exceptions.AttributeInferenceError(attribute=name, target=node) + + +def _class_lookup(node, name): + metaclass = node.metaclass() + if metaclass is None: + raise exceptions.AttributeInferenceError(attribute=name, target=node) + + return _lookup_in_mro(metaclass, name) + + +def _builtin_lookup(node, name): + values = node.locals.get(name, []) + if not values: + raise exceptions.AttributeInferenceError(attribute=name, target=node) + + return values diff --git a/env/lib/python3.8/site-packages/astroid/interpreter/objectmodel.py b/env/lib/python3.8/site-packages/astroid/interpreter/objectmodel.py new file mode 100644 index 000000000..277c8250b --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/interpreter/objectmodel.py @@ -0,0 +1,801 @@ +# -*- coding: utf-8 -*- +# Copyright (c) 2016-2019 Claudiu Popa +# Copyright (c) 2016 Derek Gustafson +# Copyright (c) 2017-2018 Bryce Guinta +# Copyright (c) 2017 Ceridwen +# Copyright (c) 2017 Calen Pennington +# Copyright (c) 2018 Ville Skyttä +# Copyright (c) 2018 Nick Drozd +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER +""" +Data object model, as per https://docs.python.org/3/reference/datamodel.html. + +This module describes, at least partially, a data object model for some +of astroid's nodes. The model contains special attributes that nodes such +as functions, classes, modules etc have, such as __doc__, __class__, +__module__ etc, being used when doing attribute lookups over nodes. + +For instance, inferring `obj.__class__` will first trigger an inference +of the `obj` variable. If it was successfully inferred, then an attribute +`__class__ will be looked for in the inferred object. This is the part +where the data model occurs. The model is attached to those nodes +and the lookup mechanism will try to see if attributes such as +`__class__` are defined by the model or not. If they are defined, +the model will be requested to return the corresponding value of that +attribute. Thus the model can be viewed as a special part of the lookup +mechanism. +""" + +import itertools +import pprint +import os +import types +from functools import lru_cache + +import astroid +from astroid import context as contextmod +from astroid import exceptions +from astroid import node_classes + + +IMPL_PREFIX = "attr_" + + +def _dunder_dict(instance, attributes): + obj = node_classes.Dict(parent=instance) + + # Convert the keys to node strings + keys = [ + node_classes.Const(value=value, parent=obj) for value in list(attributes.keys()) + ] + + # The original attribute has a list of elements for each key, + # but that is not useful for retrieving the special attribute's value. + # In this case, we're picking the last value from each list. + values = [elem[-1] for elem in attributes.values()] + + obj.postinit(list(zip(keys, values))) + return obj + + +class ObjectModel: + def __init__(self): + self._instance = None + + def __repr__(self): + result = [] + cname = type(self).__name__ + string = "%(cname)s(%(fields)s)" + alignment = len(cname) + 1 + for field in sorted(self.attributes()): + width = 80 - len(field) - alignment + lines = pprint.pformat(field, indent=2, width=width).splitlines(True) + + inner = [lines[0]] + for line in lines[1:]: + inner.append(" " * alignment + line) + result.append(field) + + return string % { + "cname": cname, + "fields": (",\n" + " " * alignment).join(result), + } + + def __call__(self, instance): + self._instance = instance + return self + + def __get__(self, instance, cls=None): + # ObjectModel needs to be a descriptor so that just doing + # `special_attributes = SomeObjectModel` should be enough in the body of a node. + # But at the same time, node.special_attributes should return an object + # which can be used for manipulating the special attributes. That's the reason + # we pass the instance through which it got accessed to ObjectModel.__call__, + # returning itself afterwards, so we can still have access to the + # underlying data model and to the instance for which it got accessed. + return self(instance) + + def __contains__(self, name): + return name in self.attributes() + + @lru_cache(maxsize=None) + def attributes(self): + """Get the attributes which are exported by this object model.""" + return [ + obj[len(IMPL_PREFIX) :] for obj in dir(self) if obj.startswith(IMPL_PREFIX) + ] + + def lookup(self, name): + """Look up the given *name* in the current model + + It should return an AST or an interpreter object, + but if the name is not found, then an AttributeInferenceError will be raised. + """ + + if name in self.attributes(): + return getattr(self, IMPL_PREFIX + name) + raise exceptions.AttributeInferenceError(target=self._instance, attribute=name) + + +class ModuleModel(ObjectModel): + def _builtins(self): + builtins_ast_module = astroid.MANAGER.builtins_module + return builtins_ast_module.special_attributes.lookup("__dict__") + + @property + def attr_builtins(self): + return self._builtins() + + @property + def attr___path__(self): + if not self._instance.package: + raise exceptions.AttributeInferenceError( + target=self._instance, attribute="__path__" + ) + + path_objs = [ + node_classes.Const( + value=path + if not path.endswith("__init__.py") + else os.path.dirname(path), + parent=self._instance, + ) + for path in self._instance.path + ] + + container = node_classes.List(parent=self._instance) + container.postinit(path_objs) + + return container + + @property + def attr___name__(self): + return node_classes.Const(value=self._instance.name, parent=self._instance) + + @property + def attr___doc__(self): + return node_classes.Const(value=self._instance.doc, parent=self._instance) + + @property + def attr___file__(self): + return node_classes.Const(value=self._instance.file, parent=self._instance) + + @property + def attr___dict__(self): + return _dunder_dict(self._instance, self._instance.globals) + + @property + def attr___package__(self): + if not self._instance.package: + value = "" + else: + value = self._instance.name + + return node_classes.Const(value=value, parent=self._instance) + + # These are related to the Python 3 implementation of the + # import system, + # https://docs.python.org/3/reference/import.html#import-related-module-attributes + + @property + def attr___spec__(self): + # No handling for now. + return node_classes.Unknown() + + @property + def attr___loader__(self): + # No handling for now. + return node_classes.Unknown() + + @property + def attr___cached__(self): + # No handling for now. + return node_classes.Unknown() + + +class FunctionModel(ObjectModel): + @property + def attr___name__(self): + return node_classes.Const(value=self._instance.name, parent=self._instance) + + @property + def attr___doc__(self): + return node_classes.Const(value=self._instance.doc, parent=self._instance) + + @property + def attr___qualname__(self): + return node_classes.Const(value=self._instance.qname(), parent=self._instance) + + @property + def attr___defaults__(self): + func = self._instance + if not func.args.defaults: + return node_classes.Const(value=None, parent=func) + + defaults_obj = node_classes.Tuple(parent=func) + defaults_obj.postinit(func.args.defaults) + return defaults_obj + + @property + def attr___annotations__(self): + obj = node_classes.Dict(parent=self._instance) + + if not self._instance.returns: + returns = None + else: + returns = self._instance.returns + + args = self._instance.args + pair_annotations = itertools.chain( + zip(args.args or [], args.annotations), + zip(args.kwonlyargs, args.kwonlyargs_annotations), + zip(args.posonlyargs or [], args.posonlyargs_annotations), + ) + + annotations = { + arg.name: annotation for (arg, annotation) in pair_annotations if annotation + } + if args.varargannotation: + annotations[args.vararg] = args.varargannotation + if args.kwargannotation: + annotations[args.kwarg] = args.kwargannotation + if returns: + annotations["return"] = returns + + items = [ + (node_classes.Const(key, parent=obj), value) + for (key, value) in annotations.items() + ] + + obj.postinit(items) + return obj + + @property + def attr___dict__(self): + return node_classes.Dict(parent=self._instance) + + attr___globals__ = attr___dict__ + + @property + def attr___kwdefaults__(self): + def _default_args(args, parent): + for arg in args.kwonlyargs: + try: + default = args.default_value(arg.name) + except exceptions.NoDefault: + continue + + name = node_classes.Const(arg.name, parent=parent) + yield name, default + + args = self._instance.args + obj = node_classes.Dict(parent=self._instance) + defaults = dict(_default_args(args, obj)) + + obj.postinit(list(defaults.items())) + return obj + + @property + def attr___module__(self): + return node_classes.Const(self._instance.root().qname()) + + @property + def attr___get__(self): + # pylint: disable=import-outside-toplevel; circular import + from astroid import bases + + func = self._instance + + class DescriptorBoundMethod(bases.BoundMethod): + """Bound method which knows how to understand calling descriptor binding.""" + + def implicit_parameters(self): + # Different than BoundMethod since the signature + # is different. + return 0 + + def infer_call_result(self, caller, context=None): + if len(caller.args) > 2 or len(caller.args) < 1: + raise exceptions.InferenceError( + "Invalid arguments for descriptor binding", + target=self, + context=context, + ) + + context = contextmod.copy_context(context) + cls = next(caller.args[0].infer(context=context)) + + if cls is astroid.Uninferable: + raise exceptions.InferenceError( + "Invalid class inferred", target=self, context=context + ) + + # For some reason func is a Node that the below + # code is not expecting + if isinstance(func, bases.BoundMethod): + yield func + return + + # Rebuild the original value, but with the parent set as the + # class where it will be bound. + new_func = func.__class__( + name=func.name, + doc=func.doc, + lineno=func.lineno, + col_offset=func.col_offset, + ) + # pylint: disable=no-member + new_func.postinit(func.args, func.body, func.decorators, func.returns) + + # Build a proper bound method that points to our newly built function. + proxy = bases.UnboundMethod(new_func) + yield bases.BoundMethod(proxy=proxy, bound=cls) + + @property + def args(self): + """Overwrite the underlying args to match those of the underlying func + + Usually the underlying *func* is a function/method, as in: + + def test(self): + pass + + This has only the *self* parameter but when we access test.__get__ + we get a new object which has two parameters, *self* and *type*. + """ + nonlocal func + positional_or_keyword_params = func.args.args.copy() + positional_or_keyword_params.append(astroid.AssignName(name="type")) + + positional_only_params = func.args.posonlyargs.copy() + + arguments = astroid.Arguments(parent=func.args.parent) + arguments.postinit( + args=positional_or_keyword_params, + posonlyargs=positional_only_params, + defaults=[], + kwonlyargs=[], + kw_defaults=[], + annotations=[], + ) + return arguments + + return DescriptorBoundMethod(proxy=self._instance, bound=self._instance) + + # These are here just for completion. + @property + def attr___ne__(self): + return node_classes.Unknown() + + attr___subclasshook__ = attr___ne__ + attr___str__ = attr___ne__ + attr___sizeof__ = attr___ne__ + attr___setattr___ = attr___ne__ + attr___repr__ = attr___ne__ + attr___reduce__ = attr___ne__ + attr___reduce_ex__ = attr___ne__ + attr___new__ = attr___ne__ + attr___lt__ = attr___ne__ + attr___eq__ = attr___ne__ + attr___gt__ = attr___ne__ + attr___format__ = attr___ne__ + attr___delattr___ = attr___ne__ + attr___getattribute__ = attr___ne__ + attr___hash__ = attr___ne__ + attr___init__ = attr___ne__ + attr___dir__ = attr___ne__ + attr___call__ = attr___ne__ + attr___class__ = attr___ne__ + attr___closure__ = attr___ne__ + attr___code__ = attr___ne__ + + +class ClassModel(ObjectModel): + @property + def attr___module__(self): + return node_classes.Const(self._instance.root().qname()) + + @property + def attr___name__(self): + return node_classes.Const(self._instance.name) + + @property + def attr___qualname__(self): + return node_classes.Const(self._instance.qname()) + + @property + def attr___doc__(self): + return node_classes.Const(self._instance.doc) + + @property + def attr___mro__(self): + if not self._instance.newstyle: + raise exceptions.AttributeInferenceError( + target=self._instance, attribute="__mro__" + ) + + mro = self._instance.mro() + obj = node_classes.Tuple(parent=self._instance) + obj.postinit(mro) + return obj + + @property + def attr_mro(self): + if not self._instance.newstyle: + raise exceptions.AttributeInferenceError( + target=self._instance, attribute="mro" + ) + + # pylint: disable=import-outside-toplevel; circular import + from astroid import bases + + other_self = self + + # Cls.mro is a method and we need to return one in order to have a proper inference. + # The method we're returning is capable of inferring the underlying MRO though. + class MroBoundMethod(bases.BoundMethod): + def infer_call_result(self, caller, context=None): + yield other_self.attr___mro__ + + implicit_metaclass = self._instance.implicit_metaclass() + mro_method = implicit_metaclass.locals["mro"][0] + return MroBoundMethod(proxy=mro_method, bound=implicit_metaclass) + + @property + def attr___bases__(self): + obj = node_classes.Tuple() + context = contextmod.InferenceContext() + elts = list(self._instance._inferred_bases(context)) + obj.postinit(elts=elts) + return obj + + @property + def attr___class__(self): + # pylint: disable=import-outside-toplevel; circular import + from astroid import helpers + + return helpers.object_type(self._instance) + + @property + def attr___subclasses__(self): + """Get the subclasses of the underlying class + + This looks only in the current module for retrieving the subclasses, + thus it might miss a couple of them. + """ + # pylint: disable=import-outside-toplevel; circular import + from astroid import bases + from astroid import scoped_nodes + + if not self._instance.newstyle: + raise exceptions.AttributeInferenceError( + target=self._instance, attribute="__subclasses__" + ) + + qname = self._instance.qname() + root = self._instance.root() + classes = [ + cls + for cls in root.nodes_of_class(scoped_nodes.ClassDef) + if cls != self._instance and cls.is_subtype_of(qname) + ] + + obj = node_classes.List(parent=self._instance) + obj.postinit(classes) + + class SubclassesBoundMethod(bases.BoundMethod): + def infer_call_result(self, caller, context=None): + yield obj + + implicit_metaclass = self._instance.implicit_metaclass() + subclasses_method = implicit_metaclass.locals["__subclasses__"][0] + return SubclassesBoundMethod(proxy=subclasses_method, bound=implicit_metaclass) + + @property + def attr___dict__(self): + return node_classes.Dict(parent=self._instance) + + +class SuperModel(ObjectModel): + @property + def attr___thisclass__(self): + return self._instance.mro_pointer + + @property + def attr___self_class__(self): + return self._instance._self_class + + @property + def attr___self__(self): + return self._instance.type + + @property + def attr___class__(self): + return self._instance._proxied + + +class UnboundMethodModel(ObjectModel): + @property + def attr___class__(self): + # pylint: disable=import-outside-toplevel; circular import + from astroid import helpers + + return helpers.object_type(self._instance) + + @property + def attr___func__(self): + return self._instance._proxied + + @property + def attr___self__(self): + return node_classes.Const(value=None, parent=self._instance) + + attr_im_func = attr___func__ + attr_im_class = attr___class__ + attr_im_self = attr___self__ + + +class BoundMethodModel(FunctionModel): + @property + def attr___func__(self): + return self._instance._proxied._proxied + + @property + def attr___self__(self): + return self._instance.bound + + +class GeneratorModel(FunctionModel): + def __new__(cls, *args, **kwargs): + # Append the values from the GeneratorType unto this object. + ret = super(GeneratorModel, cls).__new__(cls, *args, **kwargs) + generator = astroid.MANAGER.builtins_module["generator"] + for name, values in generator.locals.items(): + method = values[0] + patched = lambda cls, meth=method: meth + + setattr(type(ret), IMPL_PREFIX + name, property(patched)) + + return ret + + @property + def attr___name__(self): + return node_classes.Const( + value=self._instance.parent.name, parent=self._instance + ) + + @property + def attr___doc__(self): + return node_classes.Const( + value=self._instance.parent.doc, parent=self._instance + ) + + +class AsyncGeneratorModel(GeneratorModel): + def __new__(cls, *args, **kwargs): + # Append the values from the AGeneratorType unto this object. + ret = super().__new__(cls, *args, **kwargs) + astroid_builtins = astroid.MANAGER.builtins_module + generator = astroid_builtins.get("async_generator") + if generator is None: + # Make it backward compatible. + generator = astroid_builtins.get("generator") + + for name, values in generator.locals.items(): + method = values[0] + patched = lambda cls, meth=method: meth + + setattr(type(ret), IMPL_PREFIX + name, property(patched)) + + return ret + + +class InstanceModel(ObjectModel): + @property + def attr___class__(self): + return self._instance._proxied + + @property + def attr___module__(self): + return node_classes.Const(self._instance.root().qname()) + + @property + def attr___doc__(self): + return node_classes.Const(self._instance.doc) + + @property + def attr___dict__(self): + return _dunder_dict(self._instance, self._instance.instance_attrs) + + +# Exception instances + + +class ExceptionInstanceModel(InstanceModel): + @property + def attr_args(self): + message = node_classes.Const("") + args = node_classes.Tuple(parent=self._instance) + args.postinit((message,)) + return args + + @property + def attr___traceback__(self): + builtins_ast_module = astroid.MANAGER.builtins_module + traceback_type = builtins_ast_module[types.TracebackType.__name__] + return traceback_type.instantiate_class() + + +class SyntaxErrorInstanceModel(ExceptionInstanceModel): + @property + def attr_text(self): + return node_classes.Const("") + + +class OSErrorInstanceModel(ExceptionInstanceModel): + @property + def attr_filename(self): + return node_classes.Const("") + + @property + def attr_errno(self): + return node_classes.Const(0) + + @property + def attr_strerror(self): + return node_classes.Const("") + + attr_filename2 = attr_filename + + +class ImportErrorInstanceModel(ExceptionInstanceModel): + @property + def attr_name(self): + return node_classes.Const("") + + @property + def attr_path(self): + return node_classes.Const("") + + +BUILTIN_EXCEPTIONS = { + "builtins.SyntaxError": SyntaxErrorInstanceModel, + "builtins.ImportError": ImportErrorInstanceModel, + # These are all similar to OSError in terms of attributes + "builtins.OSError": OSErrorInstanceModel, + "builtins.BlockingIOError": OSErrorInstanceModel, + "builtins.BrokenPipeError": OSErrorInstanceModel, + "builtins.ChildProcessError": OSErrorInstanceModel, + "builtins.ConnectionAbortedError": OSErrorInstanceModel, + "builtins.ConnectionError": OSErrorInstanceModel, + "builtins.ConnectionRefusedError": OSErrorInstanceModel, + "builtins.ConnectionResetError": OSErrorInstanceModel, + "builtins.FileExistsError": OSErrorInstanceModel, + "builtins.FileNotFoundError": OSErrorInstanceModel, + "builtins.InterruptedError": OSErrorInstanceModel, + "builtins.IsADirectoryError": OSErrorInstanceModel, + "builtins.NotADirectoryError": OSErrorInstanceModel, + "builtins.PermissionError": OSErrorInstanceModel, + "builtins.ProcessLookupError": OSErrorInstanceModel, + "builtins.TimeoutError": OSErrorInstanceModel, +} + + +class DictModel(ObjectModel): + @property + def attr___class__(self): + return self._instance._proxied + + def _generic_dict_attribute(self, obj, name): + """Generate a bound method that can infer the given *obj*.""" + + class DictMethodBoundMethod(astroid.BoundMethod): + def infer_call_result(self, caller, context=None): + yield obj + + meth = next(self._instance._proxied.igetattr(name)) + return DictMethodBoundMethod(proxy=meth, bound=self._instance) + + @property + def attr_items(self): + elems = [] + obj = node_classes.List(parent=self._instance) + for key, value in self._instance.items: + elem = node_classes.Tuple(parent=obj) + elem.postinit((key, value)) + elems.append(elem) + obj.postinit(elts=elems) + + # pylint: disable=import-outside-toplevel; circular import + from astroid import objects + + obj = objects.DictItems(obj) + return self._generic_dict_attribute(obj, "items") + + @property + def attr_keys(self): + keys = [key for (key, _) in self._instance.items] + obj = node_classes.List(parent=self._instance) + obj.postinit(elts=keys) + + # pylint: disable=import-outside-toplevel; circular import + from astroid import objects + + obj = objects.DictKeys(obj) + return self._generic_dict_attribute(obj, "keys") + + @property + def attr_values(self): + + values = [value for (_, value) in self._instance.items] + obj = node_classes.List(parent=self._instance) + obj.postinit(values) + + # pylint: disable=import-outside-toplevel; circular import + from astroid import objects + + obj = objects.DictValues(obj) + return self._generic_dict_attribute(obj, "values") + + +class PropertyModel(ObjectModel): + """Model for a builtin property""" + + # pylint: disable=import-outside-toplevel + def _init_function(self, name): + from astroid.node_classes import Arguments + from astroid.scoped_nodes import FunctionDef + + args = Arguments() + args.postinit( + args=[], + defaults=[], + kwonlyargs=[], + kw_defaults=[], + annotations=[], + posonlyargs=[], + posonlyargs_annotations=[], + kwonlyargs_annotations=[], + ) + + function = FunctionDef(name=name, parent=self._instance) + + function.postinit(args=args, body=[]) + return function + + @property + def attr_fget(self): + from astroid.scoped_nodes import FunctionDef + + func = self._instance + + class PropertyFuncAccessor(FunctionDef): + def infer_call_result(self, caller=None, context=None): + nonlocal func + if caller and len(caller.args) != 1: + raise exceptions.InferenceError( + "fget() needs a single argument", target=self, context=context + ) + + yield from func.function.infer_call_result( + caller=caller, context=context + ) + + property_accessor = PropertyFuncAccessor(name="fget", parent=self._instance) + property_accessor.postinit(args=func.args, body=func.body) + return property_accessor + + @property + def attr_setter(self): + return self._init_function("setter") + + @property + def attr_deleter(self): + return self._init_function("deleter") + + @property + def attr_getter(self): + return self._init_function("getter") + + # pylint: enable=import-outside-toplevel diff --git a/env/lib/python3.8/site-packages/astroid/manager.py b/env/lib/python3.8/site-packages/astroid/manager.py new file mode 100644 index 000000000..82208adf7 --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/manager.py @@ -0,0 +1,350 @@ +# Copyright (c) 2006-2011, 2013-2014 LOGILAB S.A. (Paris, FRANCE) +# Copyright (c) 2014-2019 Claudiu Popa +# Copyright (c) 2014 BioGeek +# Copyright (c) 2014 Google, Inc. +# Copyright (c) 2014 Eevee (Alex Munroe) +# Copyright (c) 2015-2016 Ceridwen +# Copyright (c) 2016 Derek Gustafson +# Copyright (c) 2017 Iva Miholic +# Copyright (c) 2018 Bryce Guinta +# Copyright (c) 2018 Nick Drozd +# Copyright (c) 2019 Raphael Gaschignard +# Copyright (c) 2020 Anubhav <35621759+anubh-v@users.noreply.github.com> +# Copyright (c) 2020 Ashley Whetter + +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER + +"""astroid manager: avoid multiple astroid build of a same module when +possible by providing a class responsible to get astroid representation +from various source and using a cache of built modules) +""" + +import os +import zipimport + +from astroid import exceptions +from astroid.interpreter._import import spec +from astroid import modutils +from astroid import transforms + + +ZIP_IMPORT_EXTS = (".zip", ".egg", ".whl") + + +def safe_repr(obj): + try: + return repr(obj) + except Exception: # pylint: disable=broad-except + return "???" + + +class AstroidManager: + """the astroid manager, responsible to build astroid from files + or modules. + + Use the Borg pattern. + """ + + name = "astroid loader" + brain = {} + + def __init__(self): + self.__dict__ = AstroidManager.brain + if not self.__dict__: + # NOTE: cache entries are added by the [re]builder + self.astroid_cache = {} + self._mod_file_cache = {} + self._failed_import_hooks = [] + self.always_load_extensions = False + self.optimize_ast = False + self.extension_package_whitelist = set() + self._transform = transforms.TransformVisitor() + + # Export these APIs for convenience + self.register_transform = self._transform.register_transform + self.unregister_transform = self._transform.unregister_transform + self.max_inferable_values = 100 + + @property + def builtins_module(self): + return self.astroid_cache["builtins"] + + def visit_transforms(self, node): + """Visit the transforms and apply them to the given *node*.""" + return self._transform.visit(node) + + def ast_from_file(self, filepath, modname=None, fallback=True, source=False): + """given a module name, return the astroid object""" + try: + filepath = modutils.get_source_file(filepath, include_no_ext=True) + source = True + except modutils.NoSourceFile: + pass + if modname is None: + try: + modname = ".".join(modutils.modpath_from_file(filepath)) + except ImportError: + modname = filepath + if ( + modname in self.astroid_cache + and self.astroid_cache[modname].file == filepath + ): + return self.astroid_cache[modname] + if source: + # pylint: disable=import-outside-toplevel; circular import + from astroid.builder import AstroidBuilder + + return AstroidBuilder(self).file_build(filepath, modname) + if fallback and modname: + return self.ast_from_module_name(modname) + raise exceptions.AstroidBuildingError( + "Unable to build an AST for {path}.", path=filepath + ) + + def ast_from_string(self, data, modname="", filepath=None): + """ Given some source code as a string, return its corresponding astroid object""" + # pylint: disable=import-outside-toplevel; circular import + from astroid.builder import AstroidBuilder + + return AstroidBuilder(self).string_build(data, modname, filepath) + + def _build_stub_module(self, modname): + # pylint: disable=import-outside-toplevel; circular import + from astroid.builder import AstroidBuilder + + return AstroidBuilder(self).string_build("", modname) + + def _build_namespace_module(self, modname, path): + # pylint: disable=import-outside-toplevel; circular import + from astroid.builder import build_namespace_package_module + + return build_namespace_package_module(modname, path) + + def _can_load_extension(self, modname): + if self.always_load_extensions: + return True + if modutils.is_standard_module(modname): + return True + parts = modname.split(".") + return any( + ".".join(parts[:x]) in self.extension_package_whitelist + for x in range(1, len(parts) + 1) + ) + + def ast_from_module_name(self, modname, context_file=None): + """given a module name, return the astroid object""" + if modname in self.astroid_cache: + return self.astroid_cache[modname] + if modname == "__main__": + return self._build_stub_module(modname) + if context_file: + old_cwd = os.getcwd() + os.chdir(os.path.dirname(context_file)) + try: + found_spec = self.file_from_module_name(modname, context_file) + if found_spec.type == spec.ModuleType.PY_ZIPMODULE: + module = self.zip_import_data(found_spec.location) + if module is not None: + return module + + elif found_spec.type in ( + spec.ModuleType.C_BUILTIN, + spec.ModuleType.C_EXTENSION, + ): + if ( + found_spec.type == spec.ModuleType.C_EXTENSION + and not self._can_load_extension(modname) + ): + return self._build_stub_module(modname) + try: + module = modutils.load_module_from_name(modname) + except Exception as ex: + raise exceptions.AstroidImportError( + "Loading {modname} failed with:\n{error}", + modname=modname, + path=found_spec.location, + ) from ex + return self.ast_from_module(module, modname) + + elif found_spec.type == spec.ModuleType.PY_COMPILED: + raise exceptions.AstroidImportError( + "Unable to load compiled module {modname}.", + modname=modname, + path=found_spec.location, + ) + + elif found_spec.type == spec.ModuleType.PY_NAMESPACE: + return self._build_namespace_module( + modname, found_spec.submodule_search_locations + ) + elif found_spec.type == spec.ModuleType.PY_FROZEN: + return self._build_stub_module(modname) + + if found_spec.location is None: + raise exceptions.AstroidImportError( + "Can't find a file for module {modname}.", modname=modname + ) + + return self.ast_from_file(found_spec.location, modname, fallback=False) + except exceptions.AstroidBuildingError as e: + for hook in self._failed_import_hooks: + try: + return hook(modname) + except exceptions.AstroidBuildingError: + pass + raise e + finally: + if context_file: + os.chdir(old_cwd) + + def zip_import_data(self, filepath): + if zipimport is None: + return None + + # pylint: disable=import-outside-toplevel; circular import + from astroid.builder import AstroidBuilder + + builder = AstroidBuilder(self) + for ext in ZIP_IMPORT_EXTS: + try: + eggpath, resource = filepath.rsplit(ext + os.path.sep, 1) + except ValueError: + continue + try: + importer = zipimport.zipimporter(eggpath + ext) + zmodname = resource.replace(os.path.sep, ".") + if importer.is_package(resource): + zmodname = zmodname + ".__init__" + module = builder.string_build( + importer.get_source(resource), zmodname, filepath + ) + return module + except Exception: # pylint: disable=broad-except + continue + return None + + def file_from_module_name(self, modname, contextfile): + try: + value = self._mod_file_cache[(modname, contextfile)] + except KeyError: + try: + value = modutils.file_info_from_modpath( + modname.split("."), context_file=contextfile + ) + except ImportError as ex: + value = exceptions.AstroidImportError( + "Failed to import module {modname} with error:\n{error}.", + modname=modname, + error=ex, + ) + self._mod_file_cache[(modname, contextfile)] = value + if isinstance(value, exceptions.AstroidBuildingError): + raise value + return value + + def ast_from_module(self, module, modname=None): + """given an imported module, return the astroid object""" + modname = modname or module.__name__ + if modname in self.astroid_cache: + return self.astroid_cache[modname] + try: + # some builtin modules don't have __file__ attribute + filepath = module.__file__ + if modutils.is_python_source(filepath): + return self.ast_from_file(filepath, modname) + except AttributeError: + pass + + # pylint: disable=import-outside-toplevel; circular import + from astroid.builder import AstroidBuilder + + return AstroidBuilder(self).module_build(module, modname) + + def ast_from_class(self, klass, modname=None): + """get astroid for the given class""" + if modname is None: + try: + modname = klass.__module__ + except AttributeError as exc: + raise exceptions.AstroidBuildingError( + "Unable to get module for class {class_name}.", + cls=klass, + class_repr=safe_repr(klass), + modname=modname, + ) from exc + modastroid = self.ast_from_module_name(modname) + return modastroid.getattr(klass.__name__)[0] # XXX + + def infer_ast_from_something(self, obj, context=None): + """infer astroid for the given class""" + if hasattr(obj, "__class__") and not isinstance(obj, type): + klass = obj.__class__ + else: + klass = obj + try: + modname = klass.__module__ + except AttributeError as exc: + raise exceptions.AstroidBuildingError( + "Unable to get module for {class_repr}.", + cls=klass, + class_repr=safe_repr(klass), + ) from exc + except Exception as exc: + raise exceptions.AstroidImportError( + "Unexpected error while retrieving module for {class_repr}:\n" + "{error}", + cls=klass, + class_repr=safe_repr(klass), + ) from exc + try: + name = klass.__name__ + except AttributeError as exc: + raise exceptions.AstroidBuildingError( + "Unable to get name for {class_repr}:\n", + cls=klass, + class_repr=safe_repr(klass), + ) from exc + except Exception as exc: + raise exceptions.AstroidImportError( + "Unexpected error while retrieving name for {class_repr}:\n" "{error}", + cls=klass, + class_repr=safe_repr(klass), + ) from exc + # take care, on living object __module__ is regularly wrong :( + modastroid = self.ast_from_module_name(modname) + if klass is obj: + for inferred in modastroid.igetattr(name, context): + yield inferred + else: + for inferred in modastroid.igetattr(name, context): + yield inferred.instantiate_class() + + def register_failed_import_hook(self, hook): + """Registers a hook to resolve imports that cannot be found otherwise. + + `hook` must be a function that accepts a single argument `modname` which + contains the name of the module or package that could not be imported. + If `hook` can resolve the import, must return a node of type `astroid.Module`, + otherwise, it must raise `AstroidBuildingError`. + """ + self._failed_import_hooks.append(hook) + + def cache_module(self, module): + """Cache a module if no module with the same name is known yet.""" + self.astroid_cache.setdefault(module.name, module) + + def bootstrap(self): + """Bootstrap the required AST modules needed for the manager to work + + The bootstrap usually involves building the AST for the builtins + module, which is required by the rest of astroid to work correctly. + """ + from astroid import raw_building # pylint: disable=import-outside-toplevel + + raw_building._astroid_bootstrapping() + + def clear_cache(self): + """Clear the underlying cache. Also bootstraps the builtins module.""" + self.astroid_cache.clear() + self.bootstrap() diff --git a/env/lib/python3.8/site-packages/astroid/mixins.py b/env/lib/python3.8/site-packages/astroid/mixins.py new file mode 100644 index 000000000..497a8400b --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/mixins.py @@ -0,0 +1,160 @@ +# Copyright (c) 2010-2011, 2013-2014 LOGILAB S.A. (Paris, FRANCE) +# Copyright (c) 2014-2016, 2018 Claudiu Popa +# Copyright (c) 2014 Google, Inc. +# Copyright (c) 2014 Eevee (Alex Munroe) +# Copyright (c) 2015-2016 Ceridwen +# Copyright (c) 2015 Florian Bruhin +# Copyright (c) 2016 Jakub Wilk +# Copyright (c) 2018 Nick Drozd + +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER + +"""This module contains some mixins for the different nodes. +""" +import itertools + +from astroid import decorators +from astroid import exceptions + + +class BlockRangeMixIn: + """override block range """ + + @decorators.cachedproperty + def blockstart_tolineno(self): + return self.lineno + + def _elsed_block_range(self, lineno, orelse, last=None): + """handle block line numbers range for try/finally, for, if and while + statements + """ + if lineno == self.fromlineno: + return lineno, lineno + if orelse: + if lineno >= orelse[0].fromlineno: + return lineno, orelse[-1].tolineno + return lineno, orelse[0].fromlineno - 1 + return lineno, last or self.tolineno + + +class FilterStmtsMixin: + """Mixin for statement filtering and assignment type""" + + def _get_filtered_stmts(self, _, node, _stmts, mystmt): + """method used in _filter_stmts to get statements and trigger break""" + if self.statement() is mystmt: + # original node's statement is the assignment, only keep + # current node (gen exp, list comp) + return [node], True + return _stmts, False + + def assign_type(self): + return self + + +class AssignTypeMixin: + def assign_type(self): + return self + + def _get_filtered_stmts(self, lookup_node, node, _stmts, mystmt): + """method used in filter_stmts""" + if self is mystmt: + return _stmts, True + if self.statement() is mystmt: + # original node's statement is the assignment, only keep + # current node (gen exp, list comp) + return [node], True + return _stmts, False + + +class ParentAssignTypeMixin(AssignTypeMixin): + def assign_type(self): + return self.parent.assign_type() + + +class ImportFromMixin(FilterStmtsMixin): + """MixIn for From and Import Nodes""" + + def _infer_name(self, frame, name): + return name + + def do_import_module(self, modname=None): + """return the ast for a module whose name is imported by + """ + # handle special case where we are on a package node importing a module + # using the same name as the package, which may end in an infinite loop + # on relative imports + # XXX: no more needed ? + mymodule = self.root() + level = getattr(self, "level", None) # Import as no level + if modname is None: + modname = self.modname + # XXX we should investigate deeper if we really want to check + # importing itself: modname and mymodule.name be relative or absolute + if mymodule.relative_to_absolute_name(modname, level) == mymodule.name: + # FIXME: we used to raise InferenceError here, but why ? + return mymodule + + return mymodule.import_module( + modname, level=level, relative_only=level and level >= 1 + ) + + def real_name(self, asname): + """get name from 'as' name""" + for name, _asname in self.names: + if name == "*": + return asname + if not _asname: + name = name.split(".", 1)[0] + _asname = name + if asname == _asname: + return name + raise exceptions.AttributeInferenceError( + "Could not find original name for {attribute} in {target!r}", + target=self, + attribute=asname, + ) + + +class MultiLineBlockMixin: + """Mixin for nodes with multi-line blocks, e.g. For and FunctionDef. + Note that this does not apply to every node with a `body` field. + For instance, an If node has a multi-line body, but the body of an + IfExpr is not multi-line, and hence cannot contain Return nodes, + Assign nodes, etc. + """ + + @decorators.cachedproperty + def _multi_line_blocks(self): + return tuple(getattr(self, field) for field in self._multi_line_block_fields) + + def _get_return_nodes_skip_functions(self): + for block in self._multi_line_blocks: + for child_node in block: + if child_node.is_function: + continue + yield from child_node._get_return_nodes_skip_functions() + + def _get_yield_nodes_skip_lambdas(self): + for block in self._multi_line_blocks: + for child_node in block: + if child_node.is_lambda: + continue + yield from child_node._get_yield_nodes_skip_lambdas() + + @decorators.cached + def _get_assign_nodes(self): + children_assign_nodes = ( + child_node._get_assign_nodes() + for block in self._multi_line_blocks + for child_node in block + ) + return list(itertools.chain.from_iterable(children_assign_nodes)) + + +class NoChildrenMixin: + """Mixin for nodes with no children, e.g. Pass.""" + + def get_children(self): + yield from () diff --git a/env/lib/python3.8/site-packages/astroid/modutils.py b/env/lib/python3.8/site-packages/astroid/modutils.py new file mode 100644 index 000000000..4e6ed86bf --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/modutils.py @@ -0,0 +1,690 @@ +# -*- coding: utf-8 -*- +# Copyright (c) 2014-2018, 2020 Claudiu Popa +# Copyright (c) 2014 Google, Inc. +# Copyright (c) 2014 Denis Laxalde +# Copyright (c) 2014 LOGILAB S.A. (Paris, FRANCE) +# Copyright (c) 2014 Eevee (Alex Munroe) +# Copyright (c) 2015 Florian Bruhin +# Copyright (c) 2015 Radosław Ganczarek +# Copyright (c) 2016 Derek Gustafson +# Copyright (c) 2016 Jakub Wilk +# Copyright (c) 2016 Ceridwen +# Copyright (c) 2018 Ville Skyttä +# Copyright (c) 2018 Mario Corchero +# Copyright (c) 2018 Mario Corchero +# Copyright (c) 2018 Anthony Sottile +# Copyright (c) 2019 Hugo van Kemenade +# Copyright (c) 2019 markmcclain +# Copyright (c) 2019 BasPH + +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER + +"""Python modules manipulation utility functions. + +:type PY_SOURCE_EXTS: tuple(str) +:var PY_SOURCE_EXTS: list of possible python source file extension + +:type STD_LIB_DIRS: set of str +:var STD_LIB_DIRS: directories where standard modules are located + +:type BUILTIN_MODULES: dict +:var BUILTIN_MODULES: dictionary with builtin module names has key +""" +import imp +import os +import platform +import sys +import itertools +from distutils.sysconfig import get_python_lib # pylint: disable=import-error + +# pylint: disable=import-error, no-name-in-module +from distutils.errors import DistutilsPlatformError + +# distutils is replaced by virtualenv with a module that does +# weird path manipulations in order to get to the +# real distutils module. +from typing import Optional, List + +from .interpreter._import import spec +from .interpreter._import import util + +if sys.platform.startswith("win"): + PY_SOURCE_EXTS = ("py", "pyw") + PY_COMPILED_EXTS = ("dll", "pyd") +else: + PY_SOURCE_EXTS = ("py",) + PY_COMPILED_EXTS = ("so",) + + +try: + # The explicit sys.prefix is to work around a patch in virtualenv that + # replaces the 'real' sys.prefix (i.e. the location of the binary) + # with the prefix from which the virtualenv was created. This throws + # off the detection logic for standard library modules, thus the + # workaround. + STD_LIB_DIRS = { + get_python_lib(standard_lib=True, prefix=sys.prefix), + # Take care of installations where exec_prefix != prefix. + get_python_lib(standard_lib=True, prefix=sys.exec_prefix), + get_python_lib(standard_lib=True), + } +# get_python_lib(standard_lib=1) is not available on pypy, set STD_LIB_DIR to +# non-valid path, see https://bugs.pypy.org/issue1164 +except DistutilsPlatformError: + STD_LIB_DIRS = set() + +if os.name == "nt": + STD_LIB_DIRS.add(os.path.join(sys.prefix, "dlls")) + try: + # real_prefix is defined when running inside virtual environments, + # created with the **virtualenv** library. + STD_LIB_DIRS.add(os.path.join(sys.real_prefix, "dlls")) + except AttributeError: + # sys.base_exec_prefix is always defined, but in a virtual environment + # created with the stdlib **venv** module, it points to the original + # installation, if the virtual env is activated. + try: + STD_LIB_DIRS.add(os.path.join(sys.base_exec_prefix, "dlls")) + except AttributeError: + pass + +if platform.python_implementation() == "PyPy": + _root = os.path.join(sys.prefix, "lib_pypy") + STD_LIB_DIRS.add(_root) + try: + # real_prefix is defined when running inside virtualenv. + STD_LIB_DIRS.add(os.path.join(sys.real_prefix, "lib_pypy")) + except AttributeError: + pass + del _root +if os.name == "posix": + # Need the real prefix is we're under a virtualenv, otherwise + # the usual one will do. + try: + prefix = sys.real_prefix + except AttributeError: + prefix = sys.prefix + + def _posix_path(path): + base_python = "python%d.%d" % sys.version_info[:2] + return os.path.join(prefix, path, base_python) + + STD_LIB_DIRS.add(_posix_path("lib")) + if sys.maxsize > 2 ** 32: + # This tries to fix a problem with /usr/lib64 builds, + # where systems are running both 32-bit and 64-bit code + # on the same machine, which reflects into the places where + # standard library could be found. More details can be found + # here http://bugs.python.org/issue1294959. + # An easy reproducing case would be + # https://github.com/PyCQA/pylint/issues/712#issuecomment-163178753 + STD_LIB_DIRS.add(_posix_path("lib64")) + +EXT_LIB_DIRS = {get_python_lib(), get_python_lib(True)} +IS_JYTHON = platform.python_implementation() == "Jython" +BUILTIN_MODULES = dict.fromkeys(sys.builtin_module_names, True) + + +class NoSourceFile(Exception): + """exception raised when we are not able to get a python + source file for a precompiled file + """ + + +def _normalize_path(path): + return os.path.normcase(os.path.abspath(path)) + + +def _canonicalize_path(path): + return os.path.realpath(os.path.expanduser(path)) + + +def _path_from_filename(filename, is_jython=IS_JYTHON): + if not is_jython: + return filename + head, has_pyclass, _ = filename.partition("$py.class") + if has_pyclass: + return head + ".py" + return filename + + +def _handle_blacklist(blacklist, dirnames, filenames): + """remove files/directories in the black list + + dirnames/filenames are usually from os.walk + """ + for norecurs in blacklist: + if norecurs in dirnames: + dirnames.remove(norecurs) + elif norecurs in filenames: + filenames.remove(norecurs) + + +_NORM_PATH_CACHE = {} + + +def _cache_normalize_path(path): + """abspath with caching""" + # _module_file calls abspath on every path in sys.path every time it's + # called; on a larger codebase this easily adds up to half a second just + # assembling path components. This cache alleviates that. + try: + return _NORM_PATH_CACHE[path] + except KeyError: + if not path: # don't cache result for '' + return _normalize_path(path) + result = _NORM_PATH_CACHE[path] = _normalize_path(path) + return result + + +def load_module_from_name(dotted_name, path=None, use_sys=True): + """Load a Python module from its name. + + :type dotted_name: str + :param dotted_name: python name of a module or package + + :type path: list or None + :param path: + optional list of path where the module or package should be + searched (use sys.path if nothing or None is given) + + :type use_sys: bool + :param use_sys: + boolean indicating whether the sys.modules dictionary should be + used or not + + + :raise ImportError: if the module or package is not found + + :rtype: module + :return: the loaded module + """ + return load_module_from_modpath(dotted_name.split("."), path, use_sys) + + +def load_module_from_modpath(parts, path: Optional[List[str]] = None, use_sys=1): + """Load a python module from its split name. + + :type parts: list(str) or tuple(str) + :param parts: + python name of a module or package split on '.' + + :param path: + Optional list of path where the module or package should be + searched (use sys.path if nothing or None is given) + + :type use_sys: bool + :param use_sys: + boolean indicating whether the sys.modules dictionary should be used or not + + :raise ImportError: if the module or package is not found + + :rtype: module + :return: the loaded module + """ + if use_sys: + try: + return sys.modules[".".join(parts)] + except KeyError: + pass + modpath = [] + prevmodule = None + for part in parts: + modpath.append(part) + curname = ".".join(modpath) + module = None + if len(modpath) != len(parts): + # even with use_sys=False, should try to get outer packages from sys.modules + module = sys.modules.get(curname) + elif use_sys: + # because it may have been indirectly loaded through a parent + module = sys.modules.get(curname) + if module is None: + mp_file, mp_filename, mp_desc = imp.find_module(part, path) + module = imp.load_module(curname, mp_file, mp_filename, mp_desc) + # mp_file still needs to be closed. + if mp_file: + mp_file.close() + if prevmodule: + setattr(prevmodule, part, module) + _file = getattr(module, "__file__", "") + prevmodule = module + if not _file and util.is_namespace(curname): + continue + if not _file and len(modpath) != len(parts): + raise ImportError("no module in %s" % ".".join(parts[len(modpath) :])) + path = [os.path.dirname(_file)] + return module + + +def load_module_from_file( + filepath: str, path: Optional[List[str]] = None, use_sys=True +): + """Load a Python module from it's path. + + :type filepath: str + :param filepath: path to the python module or package + + :param Optional[List[str]] path: + Optional list of path where the module or package should be + searched (use sys.path if nothing or None is given) + + :type use_sys: bool + :param use_sys: + boolean indicating whether the sys.modules dictionary should be + used or not + + :raise ImportError: if the module or package is not found + + :rtype: module + :return: the loaded module + """ + modpath = modpath_from_file(filepath) + return load_module_from_modpath(modpath, path, use_sys) + + +def check_modpath_has_init(path, mod_path): + """check there are some __init__.py all along the way""" + modpath = [] + for part in mod_path: + modpath.append(part) + path = os.path.join(path, part) + if not _has_init(path): + old_namespace = util.is_namespace(".".join(modpath)) + if not old_namespace: + return False + return True + + +def _get_relative_base_path(filename, path_to_check): + """Extracts the relative mod path of the file to import from + + Check if a file is within the passed in path and if so, returns the + relative mod path from the one passed in. + + If the filename is no in path_to_check, returns None + + Note this function will look for both abs and realpath of the file, + this allows to find the relative base path even if the file is a + symlink of a file in the passed in path + + Examples: + _get_relative_base_path("/a/b/c/d.py", "/a/b") -> ["c","d"] + _get_relative_base_path("/a/b/c/d.py", "/dev") -> None + """ + importable_path = None + path_to_check = os.path.normcase(path_to_check) + abs_filename = os.path.abspath(filename) + if os.path.normcase(abs_filename).startswith(path_to_check): + importable_path = abs_filename + + real_filename = os.path.realpath(filename) + if os.path.normcase(real_filename).startswith(path_to_check): + importable_path = real_filename + + if importable_path: + base_path = os.path.splitext(importable_path)[0] + relative_base_path = base_path[len(path_to_check) :] + return [pkg for pkg in relative_base_path.split(os.sep) if pkg] + + return None + + +def modpath_from_file_with_callback(filename, path=None, is_package_cb=None): + filename = os.path.expanduser(_path_from_filename(filename)) + for pathname in itertools.chain( + path or [], map(_canonicalize_path, sys.path), sys.path + ): + pathname = _cache_normalize_path(pathname) + if not pathname: + continue + modpath = _get_relative_base_path(filename, pathname) + if not modpath: + continue + if is_package_cb(pathname, modpath[:-1]): + return modpath + + raise ImportError( + "Unable to find module for %s in %s" % (filename, ", \n".join(sys.path)) + ) + + +def modpath_from_file(filename, path=None): + """Get the corresponding split module's name from a filename + + This function will return the name of a module or package split on `.`. + + :type filename: str + :param filename: file's path for which we want the module's name + + :type Optional[List[str]] path: + Optional list of path where the module or package should be + searched (use sys.path if nothing or None is given) + + :raise ImportError: + if the corresponding module's name has not been found + + :rtype: list(str) + :return: the corresponding split module's name + """ + return modpath_from_file_with_callback(filename, path, check_modpath_has_init) + + +def file_from_modpath(modpath, path=None, context_file=None): + return file_info_from_modpath(modpath, path, context_file).location + + +def file_info_from_modpath(modpath, path=None, context_file=None): + """given a mod path (i.e. split module / package name), return the + corresponding file, giving priority to source file over precompiled + file if it exists + + :type modpath: list or tuple + :param modpath: + split module's name (i.e name of a module or package split + on '.') + (this means explicit relative imports that start with dots have + empty strings in this list!) + + :type path: list or None + :param path: + optional list of path where the module or package should be + searched (use sys.path if nothing or None is given) + + :type context_file: str or None + :param context_file: + context file to consider, necessary if the identifier has been + introduced using a relative import unresolvable in the actual + context (i.e. modutils) + + :raise ImportError: if there is no such module in the directory + + :rtype: (str or None, import type) + :return: + the path to the module's file or None if it's an integrated + builtin module such as 'sys' + """ + if context_file is not None: + context = os.path.dirname(context_file) + else: + context = context_file + if modpath[0] == "xml": + # handle _xmlplus + try: + return _spec_from_modpath(["_xmlplus"] + modpath[1:], path, context) + except ImportError: + return _spec_from_modpath(modpath, path, context) + elif modpath == ["os", "path"]: + # FIXME: currently ignoring search_path... + return spec.ModuleSpec( + name="os.path", location=os.path.__file__, module_type=imp.PY_SOURCE + ) + return _spec_from_modpath(modpath, path, context) + + +def get_module_part(dotted_name, context_file=None): + """given a dotted name return the module part of the name : + + >>> get_module_part('astroid.as_string.dump') + 'astroid.as_string' + + :type dotted_name: str + :param dotted_name: full name of the identifier we are interested in + + :type context_file: str or None + :param context_file: + context file to consider, necessary if the identifier has been + introduced using a relative import unresolvable in the actual + context (i.e. modutils) + + + :raise ImportError: if there is no such module in the directory + + :rtype: str or None + :return: + the module part of the name or None if we have not been able at + all to import the given name + + XXX: deprecated, since it doesn't handle package precedence over module + (see #10066) + """ + # os.path trick + if dotted_name.startswith("os.path"): + return "os.path" + parts = dotted_name.split(".") + if context_file is not None: + # first check for builtin module which won't be considered latter + # in that case (path != None) + if parts[0] in BUILTIN_MODULES: + if len(parts) > 2: + raise ImportError(dotted_name) + return parts[0] + # don't use += or insert, we want a new list to be created ! + path = None + starti = 0 + if parts[0] == "": + assert ( + context_file is not None + ), "explicit relative import, but no context_file?" + path = [] # prevent resolving the import non-relatively + starti = 1 + while parts[starti] == "": # for all further dots: change context + starti += 1 + context_file = os.path.dirname(context_file) + for i in range(starti, len(parts)): + try: + file_from_modpath( + parts[starti : i + 1], path=path, context_file=context_file + ) + except ImportError: + if i < max(1, len(parts) - 2): + raise + return ".".join(parts[:i]) + return dotted_name + + +def get_module_files(src_directory, blacklist, list_all=False): + """given a package directory return a list of all available python + module's files in the package and its subpackages + + :type src_directory: str + :param src_directory: + path of the directory corresponding to the package + + :type blacklist: list or tuple + :param blacklist: iterable + list of files or directories to ignore. + + :type list_all: bool + :param list_all: + get files from all paths, including ones without __init__.py + + :rtype: list + :return: + the list of all available python module's files in the package and + its subpackages + """ + files = [] + for directory, dirnames, filenames in os.walk(src_directory): + if directory in blacklist: + continue + _handle_blacklist(blacklist, dirnames, filenames) + # check for __init__.py + if not list_all and "__init__.py" not in filenames: + dirnames[:] = () + continue + for filename in filenames: + if _is_python_file(filename): + src = os.path.join(directory, filename) + files.append(src) + return files + + +def get_source_file(filename, include_no_ext=False): + """given a python module's file name return the matching source file + name (the filename will be returned identically if it's already an + absolute path to a python source file...) + + :type filename: str + :param filename: python module's file name + + + :raise NoSourceFile: if no source file exists on the file system + + :rtype: str + :return: the absolute path of the source file if it exists + """ + filename = os.path.abspath(_path_from_filename(filename)) + base, orig_ext = os.path.splitext(filename) + for ext in PY_SOURCE_EXTS: + source_path = "%s.%s" % (base, ext) + if os.path.exists(source_path): + return source_path + if include_no_ext and not orig_ext and os.path.exists(base): + return base + raise NoSourceFile(filename) + + +def is_python_source(filename): + """ + rtype: bool + return: True if the filename is a python source file + """ + return os.path.splitext(filename)[1][1:] in PY_SOURCE_EXTS + + +def is_standard_module(modname, std_path=None): + """try to guess if a module is a standard python module (by default, + see `std_path` parameter's description) + + :type modname: str + :param modname: name of the module we are interested in + + :type std_path: list(str) or tuple(str) + :param std_path: list of path considered has standard + + + :rtype: bool + :return: + true if the module: + - is located on the path listed in one of the directory in `std_path` + - is a built-in module + """ + modname = modname.split(".")[0] + try: + filename = file_from_modpath([modname]) + except ImportError: + # import failed, i'm probably not so wrong by supposing it's + # not standard... + return False + # modules which are not living in a file are considered standard + # (sys and __builtin__ for instance) + if filename is None: + # we assume there are no namespaces in stdlib + return not util.is_namespace(modname) + filename = _normalize_path(filename) + for path in EXT_LIB_DIRS: + if filename.startswith(_cache_normalize_path(path)): + return False + if std_path is None: + std_path = STD_LIB_DIRS + for path in std_path: + if filename.startswith(_cache_normalize_path(path)): + return True + return False + + +def is_relative(modname, from_file): + """return true if the given module name is relative to the given + file name + + :type modname: str + :param modname: name of the module we are interested in + + :type from_file: str + :param from_file: + path of the module from which modname has been imported + + :rtype: bool + :return: + true if the module has been imported relatively to `from_file` + """ + if not os.path.isdir(from_file): + from_file = os.path.dirname(from_file) + if from_file in sys.path: + return False + try: + stream, _, _ = imp.find_module(modname.split(".")[0], [from_file]) + + # Close the stream to avoid ResourceWarnings. + if stream: + stream.close() + return True + except ImportError: + return False + + +# internal only functions ##################################################### + + +def _spec_from_modpath(modpath, path=None, context=None): + """given a mod path (i.e. split module / package name), return the + corresponding spec + + this function is used internally, see `file_from_modpath`'s + documentation for more information + """ + assert modpath + location = None + if context is not None: + try: + found_spec = spec.find_spec(modpath, [context]) + location = found_spec.location + except ImportError: + found_spec = spec.find_spec(modpath, path) + location = found_spec.location + else: + found_spec = spec.find_spec(modpath, path) + if found_spec.type == spec.ModuleType.PY_COMPILED: + try: + location = get_source_file(found_spec.location) + return found_spec._replace( + location=location, type=spec.ModuleType.PY_SOURCE + ) + except NoSourceFile: + return found_spec._replace(location=location) + elif found_spec.type == spec.ModuleType.C_BUILTIN: + # integrated builtin module + return found_spec._replace(location=None) + elif found_spec.type == spec.ModuleType.PKG_DIRECTORY: + location = _has_init(found_spec.location) + return found_spec._replace(location=location, type=spec.ModuleType.PY_SOURCE) + return found_spec + + +def _is_python_file(filename): + """return true if the given filename should be considered as a python file + + .pyc and .pyo are ignored + """ + return filename.endswith((".py", ".so", ".pyd", ".pyw")) + + +def _has_init(directory): + """if the given directory has a valid __init__ file, return its path, + else return None + """ + mod_or_pack = os.path.join(directory, "__init__") + for ext in PY_SOURCE_EXTS + ("pyc", "pyo"): + if os.path.exists(mod_or_pack + "." + ext): + return mod_or_pack + "." + ext + return None + + +def is_namespace(specobj): + return specobj.type == spec.ModuleType.PY_NAMESPACE + + +def is_directory(specobj): + return specobj.type == spec.ModuleType.PKG_DIRECTORY diff --git a/env/lib/python3.8/site-packages/astroid/node_classes.py b/env/lib/python3.8/site-packages/astroid/node_classes.py new file mode 100644 index 000000000..621dc5f21 --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/node_classes.py @@ -0,0 +1,4862 @@ +# -*- coding: utf-8 -*- +# Copyright (c) 2009-2011, 2013-2014 LOGILAB S.A. (Paris, FRANCE) +# Copyright (c) 2010 Daniel Harding +# Copyright (c) 2012 FELD Boris +# Copyright (c) 2013-2014 Google, Inc. +# Copyright (c) 2014-2020 Claudiu Popa +# Copyright (c) 2014 Eevee (Alex Munroe) +# Copyright (c) 2015-2016 Ceridwen +# Copyright (c) 2015 Florian Bruhin +# Copyright (c) 2016-2017 Derek Gustafson +# Copyright (c) 2016 Jared Garst +# Copyright (c) 2016 Jakub Wilk +# Copyright (c) 2016 Dave Baum +# Copyright (c) 2017-2020 Ashley Whetter +# Copyright (c) 2017, 2019 Łukasz Rogalski +# Copyright (c) 2017 rr- +# Copyright (c) 2018-2019 hippo91 +# Copyright (c) 2018 Nick Drozd +# Copyright (c) 2018 Ville Skyttä +# Copyright (c) 2018 Bryce Guinta +# Copyright (c) 2018 brendanator +# Copyright (c) 2018 HoverHell +# Copyright (c) 2019 kavins14 +# Copyright (c) 2019 kavins14 + +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER + +# pylint: disable=too-many-lines; https://github.com/PyCQA/astroid/issues/465 + +"""Module for some node classes. More nodes in scoped_nodes.py +""" + +import abc +import builtins as builtins_mod +import itertools +import pprint +import sys +from functools import lru_cache, singledispatch as _singledispatch + +from astroid import as_string +from astroid import bases +from astroid import context as contextmod +from astroid import decorators +from astroid import exceptions +from astroid import manager +from astroid import mixins +from astroid import util + + +BUILTINS = builtins_mod.__name__ +MANAGER = manager.AstroidManager() +PY38 = sys.version_info[:2] >= (3, 8) + + +def _is_const(value): + return isinstance(value, tuple(CONST_CLS)) + + +@decorators.raise_if_nothing_inferred +def unpack_infer(stmt, context=None): + """recursively generate nodes inferred by the given statement. + If the inferred value is a list or a tuple, recurse on the elements + """ + if isinstance(stmt, (List, Tuple)): + for elt in stmt.elts: + if elt is util.Uninferable: + yield elt + continue + yield from unpack_infer(elt, context) + return dict(node=stmt, context=context) + # if inferred is a final node, return it and stop + inferred = next(stmt.infer(context)) + if inferred is stmt: + yield inferred + return dict(node=stmt, context=context) + # else, infer recursively, except Uninferable object that should be returned as is + for inferred in stmt.infer(context): + if inferred is util.Uninferable: + yield inferred + else: + yield from unpack_infer(inferred, context) + + return dict(node=stmt, context=context) + + +def are_exclusive( + stmt1, stmt2, exceptions=None +): # pylint: disable=redefined-outer-name + """return true if the two given statements are mutually exclusive + + `exceptions` may be a list of exception names. If specified, discard If + branches and check one of the statement is in an exception handler catching + one of the given exceptions. + + algorithm : + 1) index stmt1's parents + 2) climb among stmt2's parents until we find a common parent + 3) if the common parent is a If or TryExcept statement, look if nodes are + in exclusive branches + """ + # index stmt1's parents + stmt1_parents = {} + children = {} + node = stmt1.parent + previous = stmt1 + while node: + stmt1_parents[node] = 1 + children[node] = previous + previous = node + node = node.parent + # climb among stmt2's parents until we find a common parent + node = stmt2.parent + previous = stmt2 + while node: + if node in stmt1_parents: + # if the common parent is a If or TryExcept statement, look if + # nodes are in exclusive branches + if isinstance(node, If) and exceptions is None: + if ( + node.locate_child(previous)[1] + is not node.locate_child(children[node])[1] + ): + return True + elif isinstance(node, TryExcept): + c2attr, c2node = node.locate_child(previous) + c1attr, c1node = node.locate_child(children[node]) + if c1node is not c2node: + first_in_body_caught_by_handlers = ( + c2attr == "handlers" + and c1attr == "body" + and previous.catch(exceptions) + ) + second_in_body_caught_by_handlers = ( + c2attr == "body" + and c1attr == "handlers" + and children[node].catch(exceptions) + ) + first_in_else_other_in_handlers = ( + c2attr == "handlers" and c1attr == "orelse" + ) + second_in_else_other_in_handlers = ( + c2attr == "orelse" and c1attr == "handlers" + ) + if any( + ( + first_in_body_caught_by_handlers, + second_in_body_caught_by_handlers, + first_in_else_other_in_handlers, + second_in_else_other_in_handlers, + ) + ): + return True + elif c2attr == "handlers" and c1attr == "handlers": + return previous is not children[node] + return False + previous = node + node = node.parent + return False + + +# getitem() helpers. + +_SLICE_SENTINEL = object() + + +def _slice_value(index, context=None): + """Get the value of the given slice index.""" + + if isinstance(index, Const): + if isinstance(index.value, (int, type(None))): + return index.value + elif index is None: + return None + else: + # Try to infer what the index actually is. + # Since we can't return all the possible values, + # we'll stop at the first possible value. + try: + inferred = next(index.infer(context=context)) + except exceptions.InferenceError: + pass + else: + if isinstance(inferred, Const): + if isinstance(inferred.value, (int, type(None))): + return inferred.value + + # Use a sentinel, because None can be a valid + # value that this function can return, + # as it is the case for unspecified bounds. + return _SLICE_SENTINEL + + +def _infer_slice(node, context=None): + lower = _slice_value(node.lower, context) + upper = _slice_value(node.upper, context) + step = _slice_value(node.step, context) + if all(elem is not _SLICE_SENTINEL for elem in (lower, upper, step)): + return slice(lower, upper, step) + + raise exceptions.AstroidTypeError( + message="Could not infer slice used in subscript", + node=node, + index=node.parent, + context=context, + ) + + +def _container_getitem(instance, elts, index, context=None): + """Get a slice or an item, using the given *index*, for the given sequence.""" + try: + if isinstance(index, Slice): + index_slice = _infer_slice(index, context=context) + new_cls = instance.__class__() + new_cls.elts = elts[index_slice] + new_cls.parent = instance.parent + return new_cls + if isinstance(index, Const): + return elts[index.value] + except IndexError as exc: + raise exceptions.AstroidIndexError( + message="Index {index!s} out of range", + node=instance, + index=index, + context=context, + ) from exc + except TypeError as exc: + raise exceptions.AstroidTypeError( + message="Type error {error!r}", node=instance, index=index, context=context + ) from exc + + raise exceptions.AstroidTypeError("Could not use %s as subscript index" % index) + + +OP_PRECEDENCE = { + op: precedence + for precedence, ops in enumerate( + [ + ["Lambda"], # lambda x: x + 1 + ["IfExp"], # 1 if True else 2 + ["or"], + ["and"], + ["not"], + ["Compare"], # in, not in, is, is not, <, <=, >, >=, !=, == + ["|"], + ["^"], + ["&"], + ["<<", ">>"], + ["+", "-"], + ["*", "@", "/", "//", "%"], + ["UnaryOp"], # +, -, ~ + ["**"], + ["Await"], + ] + ) + for op in ops +} + + +class NodeNG: + """ A node of the new Abstract Syntax Tree (AST). + + This is the base class for all Astroid node classes. + """ + + is_statement = False + """Whether this node indicates a statement. + + :type: bool + """ + optional_assign = False # True for For (and for Comprehension if py <3.0) + """Whether this node optionally assigns a variable. + + This is for loop assignments because loop won't necessarily perform an + assignment if the loop has no iterations. + This is also the case from comprehensions in Python 2. + + :type: bool + """ + is_function = False # True for FunctionDef nodes + """Whether this node indicates a function. + + :type: bool + """ + is_lambda = False + # Attributes below are set by the builder module or by raw factories + lineno = None + """The line that this node appears on in the source code. + + :type: int or None + """ + col_offset = None + """The column that this node appears on in the source code. + + :type: int or None + """ + parent = None + """The parent node in the syntax tree. + + :type: NodeNG or None + """ + _astroid_fields = () + """Node attributes that contain child nodes. + + This is redefined in most concrete classes. + + :type: tuple(str) + """ + _other_fields = () + """Node attributes that do not contain child nodes. + + :type: tuple(str) + """ + _other_other_fields = () + """Attributes that contain AST-dependent fields. + + :type: tuple(str) + """ + # instance specific inference function infer(node, context) + _explicit_inference = None + + def __init__(self, lineno=None, col_offset=None, parent=None): + """ + :param lineno: The line that this node appears on in the source code. + :type lineno: int or None + + :param col_offset: The column that this node appears on in the + source code. + :type col_offset: int or None + + :param parent: The parent node in the syntax tree. + :type parent: NodeNG or None + """ + self.lineno = lineno + self.col_offset = col_offset + self.parent = parent + + def infer(self, context=None, **kwargs): + """Get a generator of the inferred values. + + This is the main entry point to the inference system. + + .. seealso:: :ref:`inference` + + If the instance has some explicit inference function set, it will be + called instead of the default interface. + + :returns: The inferred values. + :rtype: iterable + """ + if context is not None: + context = context.extra_context.get(self, context) + if self._explicit_inference is not None: + # explicit_inference is not bound, give it self explicitly + try: + # pylint: disable=not-callable + return self._explicit_inference(self, context, **kwargs) + except exceptions.UseInferenceDefault: + pass + + if not context: + return self._infer(context, **kwargs) + + key = (self, context.lookupname, context.callcontext, context.boundnode) + if key in context.inferred: + return iter(context.inferred[key]) + + gen = context.cache_generator(key, self._infer(context, **kwargs)) + return util.limit_inference(gen, MANAGER.max_inferable_values) + + def _repr_name(self): + """Get a name for nice representation. + + This is either :attr:`name`, :attr:`attrname`, or the empty string. + + :returns: The nice name. + :rtype: str + """ + names = {"name", "attrname"} + if all(name not in self._astroid_fields for name in names): + return getattr(self, "name", getattr(self, "attrname", "")) + return "" + + def __str__(self): + rname = self._repr_name() + cname = type(self).__name__ + if rname: + string = "%(cname)s.%(rname)s(%(fields)s)" + alignment = len(cname) + len(rname) + 2 + else: + string = "%(cname)s(%(fields)s)" + alignment = len(cname) + 1 + result = [] + for field in self._other_fields + self._astroid_fields: + value = getattr(self, field) + width = 80 - len(field) - alignment + lines = pprint.pformat(value, indent=2, width=width).splitlines(True) + + inner = [lines[0]] + for line in lines[1:]: + inner.append(" " * alignment + line) + result.append("%s=%s" % (field, "".join(inner))) + + return string % { + "cname": cname, + "rname": rname, + "fields": (",\n" + " " * alignment).join(result), + } + + def __repr__(self): + rname = self._repr_name() + if rname: + string = "<%(cname)s.%(rname)s l.%(lineno)s at 0x%(id)x>" + else: + string = "<%(cname)s l.%(lineno)s at 0x%(id)x>" + return string % { + "cname": type(self).__name__, + "rname": rname, + "lineno": self.fromlineno, + "id": id(self), + } + + def accept(self, visitor): + """Visit this node using the given visitor.""" + func = getattr(visitor, "visit_" + self.__class__.__name__.lower()) + return func(self) + + def get_children(self): + """Get the child nodes below this node. + + :returns: The children. + :rtype: iterable(NodeNG) + """ + for field in self._astroid_fields: + attr = getattr(self, field) + if attr is None: + continue + if isinstance(attr, (list, tuple)): + yield from attr + else: + yield attr + + def last_child(self): + """An optimized version of list(get_children())[-1] + + :returns: The last child, or None if no children exist. + :rtype: NodeNG or None + """ + for field in self._astroid_fields[::-1]: + attr = getattr(self, field) + if not attr: # None or empty listy / tuple + continue + if isinstance(attr, (list, tuple)): + return attr[-1] + + return attr + return None + + def parent_of(self, node): + """Check if this node is the parent of the given node. + + :param node: The node to check if it is the child. + :type node: NodeNG + + :returns: True if this node is the parent of the given node, + False otherwise. + :rtype: bool + """ + parent = node.parent + while parent is not None: + if self is parent: + return True + parent = parent.parent + return False + + def statement(self): + """The first parent node, including self, marked as statement node. + + :returns: The first parent statement. + :rtype: NodeNG + """ + if self.is_statement: + return self + return self.parent.statement() + + def frame(self): + """The first parent frame node. + + A frame node is a :class:`Module`, :class:`FunctionDef`, + or :class:`ClassDef`. + + :returns: The first parent frame node. + :rtype: Module or FunctionDef or ClassDef + """ + return self.parent.frame() + + def scope(self): + """The first parent node defining a new scope. + + :returns: The first parent scope node. + :rtype: Module or FunctionDef or ClassDef or Lambda or GenExpr + """ + if self.parent: + return self.parent.scope() + return None + + def root(self): + """Return the root node of the syntax tree. + + :returns: The root node. + :rtype: Module + """ + if self.parent: + return self.parent.root() + return self + + def child_sequence(self, child): + """Search for the sequence that contains this child. + + :param child: The child node to search sequences for. + :type child: NodeNG + + :returns: The sequence containing the given child node. + :rtype: iterable(NodeNG) + + :raises AstroidError: If no sequence could be found that contains + the given child. + """ + for field in self._astroid_fields: + node_or_sequence = getattr(self, field) + if node_or_sequence is child: + return [node_or_sequence] + # /!\ compiler.ast Nodes have an __iter__ walking over child nodes + if ( + isinstance(node_or_sequence, (tuple, list)) + and child in node_or_sequence + ): + return node_or_sequence + + msg = "Could not find %s in %s's children" + raise exceptions.AstroidError(msg % (repr(child), repr(self))) + + def locate_child(self, child): + """Find the field of this node that contains the given child. + + :param child: The child node to search fields for. + :type child: NodeNG + + :returns: A tuple of the name of the field that contains the child, + and the sequence or node that contains the child node. + :rtype: tuple(str, iterable(NodeNG) or NodeNG) + + :raises AstroidError: If no field could be found that contains + the given child. + """ + for field in self._astroid_fields: + node_or_sequence = getattr(self, field) + # /!\ compiler.ast Nodes have an __iter__ walking over child nodes + if child is node_or_sequence: + return field, child + if ( + isinstance(node_or_sequence, (tuple, list)) + and child in node_or_sequence + ): + return field, node_or_sequence + msg = "Could not find %s in %s's children" + raise exceptions.AstroidError(msg % (repr(child), repr(self))) + + # FIXME : should we merge child_sequence and locate_child ? locate_child + # is only used in are_exclusive, child_sequence one time in pylint. + + def next_sibling(self): + """The next sibling statement node. + + :returns: The next sibling statement node. + :rtype: NodeNG or None + """ + return self.parent.next_sibling() + + def previous_sibling(self): + """The previous sibling statement. + + :returns: The previous sibling statement node. + :rtype: NodeNG or None + """ + return self.parent.previous_sibling() + + # these are lazy because they're relatively expensive to compute for every + # single node, and they rarely get looked at + + @decorators.cachedproperty + def fromlineno(self): + """The first line that this node appears on in the source code. + + :type: int or None + """ + if self.lineno is None: + return self._fixed_source_line() + + return self.lineno + + @decorators.cachedproperty + def tolineno(self): + """The last line that this node appears on in the source code. + + :type: int or None + """ + if not self._astroid_fields: + # can't have children + lastchild = None + else: + lastchild = self.last_child() + if lastchild is None: + return self.fromlineno + + return lastchild.tolineno + + def _fixed_source_line(self): + """Attempt to find the line that this node appears on. + + We need this method since not all nodes have :attr:`lineno` set. + + :returns: The line number of this node, + or None if this could not be determined. + :rtype: int or None + """ + line = self.lineno + _node = self + try: + while line is None: + _node = next(_node.get_children()) + line = _node.lineno + except StopIteration: + _node = self.parent + while _node and line is None: + line = _node.lineno + _node = _node.parent + return line + + def block_range(self, lineno): + """Get a range from the given line number to where this node ends. + + :param lineno: The line number to start the range at. + :type lineno: int + + :returns: The range of line numbers that this node belongs to, + starting at the given line number. + :rtype: tuple(int, int or None) + """ + return lineno, self.tolineno + + def set_local(self, name, stmt): + """Define that the given name is declared in the given statement node. + + This definition is stored on the parent scope node. + + .. seealso:: :meth:`scope` + + :param name: The name that is being defined. + :type name: str + + :param stmt: The statement that defines the given name. + :type stmt: NodeNG + """ + self.parent.set_local(name, stmt) + + def nodes_of_class(self, klass, skip_klass=None): + """Get the nodes (including this one or below) of the given types. + + :param klass: The types of node to search for. + :type klass: builtins.type or tuple(builtins.type) + + :param skip_klass: The types of node to ignore. This is useful to ignore + subclasses of :attr:`klass`. + :type skip_klass: builtins.type or tuple(builtins.type) + + :returns: The node of the given types. + :rtype: iterable(NodeNG) + """ + if isinstance(self, klass): + yield self + + if skip_klass is None: + for child_node in self.get_children(): + yield from child_node.nodes_of_class(klass, skip_klass) + + return + + for child_node in self.get_children(): + if isinstance(child_node, skip_klass): + continue + yield from child_node.nodes_of_class(klass, skip_klass) + + @decorators.cached + def _get_assign_nodes(self): + return [] + + def _get_name_nodes(self): + for child_node in self.get_children(): + yield from child_node._get_name_nodes() + + def _get_return_nodes_skip_functions(self): + yield from () + + def _get_yield_nodes_skip_lambdas(self): + yield from () + + def _infer_name(self, frame, name): + # overridden for ImportFrom, Import, Global, TryExcept and Arguments + pass + + def _infer(self, context=None): + """we don't know how to resolve a statement by default""" + # this method is overridden by most concrete classes + raise exceptions.InferenceError( + "No inference function for {node!r}.", node=self, context=context + ) + + def inferred(self): + """Get a list of the inferred values. + + .. seealso:: :ref:`inference` + + :returns: The inferred values. + :rtype: list + """ + return list(self.infer()) + + def instantiate_class(self): + """Instantiate an instance of the defined class. + + .. note:: + + On anything other than a :class:`ClassDef` this will return self. + + :returns: An instance of the defined class. + :rtype: object + """ + return self + + def has_base(self, node): + """Check if this node inherits from the given type. + + :param node: The node defining the base to look for. + Usually this is a :class:`Name` node. + :type node: NodeNG + """ + return False + + def callable(self): + """Whether this node defines something that is callable. + + :returns: True if this defines something that is callable, + False otherwise. + :rtype: bool + """ + return False + + def eq(self, value): + return False + + def as_string(self): + """Get the source code that this node represents. + + :returns: The source code. + :rtype: str + """ + return as_string.to_code(self) + + def repr_tree( + self, + ids=False, + include_linenos=False, + ast_state=False, + indent=" ", + max_depth=0, + max_width=80, + ): + """Get a string representation of the AST from this node. + + :param ids: If true, includes the ids with the node type names. + :type ids: bool + + :param include_linenos: If true, includes the line numbers and + column offsets. + :type include_linenos: bool + + :param ast_state: If true, includes information derived from + the whole AST like local and global variables. + :type ast_state: bool + + :param indent: A string to use to indent the output string. + :type indent: str + + :param max_depth: If set to a positive integer, won't return + nodes deeper than max_depth in the string. + :type max_depth: int + + :param max_width: Attempt to format the output string to stay + within this number of characters, but can exceed it under some + circumstances. Only positive integer values are valid, the default is 80. + :type max_width: int + + :returns: The string representation of the AST. + :rtype: str + """ + # pylint: disable=too-many-statements + @_singledispatch + def _repr_tree(node, result, done, cur_indent="", depth=1): + """Outputs a representation of a non-tuple/list, non-node that's + contained within an AST, including strings. + """ + lines = pprint.pformat( + node, width=max(max_width - len(cur_indent), 1) + ).splitlines(True) + result.append(lines[0]) + result.extend([cur_indent + line for line in lines[1:]]) + return len(lines) != 1 + + # pylint: disable=unused-variable; doesn't understand singledispatch + @_repr_tree.register(tuple) + @_repr_tree.register(list) + def _repr_seq(node, result, done, cur_indent="", depth=1): + """Outputs a representation of a sequence that's contained within an AST.""" + cur_indent += indent + result.append("[") + if not node: + broken = False + elif len(node) == 1: + broken = _repr_tree(node[0], result, done, cur_indent, depth) + elif len(node) == 2: + broken = _repr_tree(node[0], result, done, cur_indent, depth) + if not broken: + result.append(", ") + else: + result.append(",\n") + result.append(cur_indent) + broken = _repr_tree(node[1], result, done, cur_indent, depth) or broken + else: + result.append("\n") + result.append(cur_indent) + for child in node[:-1]: + _repr_tree(child, result, done, cur_indent, depth) + result.append(",\n") + result.append(cur_indent) + _repr_tree(node[-1], result, done, cur_indent, depth) + broken = True + result.append("]") + return broken + + # pylint: disable=unused-variable; doesn't understand singledispatch + @_repr_tree.register(NodeNG) + def _repr_node(node, result, done, cur_indent="", depth=1): + """Outputs a strings representation of an astroid node.""" + if node in done: + result.append( + indent + + " max_depth: + result.append("...") + return False + depth += 1 + cur_indent += indent + if ids: + result.append("%s<0x%x>(\n" % (type(node).__name__, id(node))) + else: + result.append("%s(" % type(node).__name__) + fields = [] + if include_linenos: + fields.extend(("lineno", "col_offset")) + fields.extend(node._other_fields) + fields.extend(node._astroid_fields) + if ast_state: + fields.extend(node._other_other_fields) + if not fields: + broken = False + elif len(fields) == 1: + result.append("%s=" % fields[0]) + broken = _repr_tree( + getattr(node, fields[0]), result, done, cur_indent, depth + ) + else: + result.append("\n") + result.append(cur_indent) + for field in fields[:-1]: + result.append("%s=" % field) + _repr_tree(getattr(node, field), result, done, cur_indent, depth) + result.append(",\n") + result.append(cur_indent) + result.append("%s=" % fields[-1]) + _repr_tree(getattr(node, fields[-1]), result, done, cur_indent, depth) + broken = True + result.append(")") + return broken + + result = [] + _repr_tree(self, result, set()) + return "".join(result) + + def bool_value(self, context=None): + """Determine the boolean value of this node. + + The boolean value of a node can have three + possible values: + + * False: For instance, empty data structures, + False, empty strings, instances which return + explicitly False from the __nonzero__ / __bool__ + method. + * True: Most of constructs are True by default: + classes, functions, modules etc + * Uninferable: The inference engine is uncertain of the + node's value. + + :returns: The boolean value of this node. + :rtype: bool or Uninferable + """ + return util.Uninferable + + def op_precedence(self): + # Look up by class name or default to highest precedence + return OP_PRECEDENCE.get(self.__class__.__name__, len(OP_PRECEDENCE)) + + def op_left_associative(self): + # Everything is left associative except `**` and IfExp + return True + + +class Statement(NodeNG): + """Statement node adding a few attributes""" + + is_statement = True + """Whether this node indicates a statement. + + :type: bool + """ + + def next_sibling(self): + """The next sibling statement node. + + :returns: The next sibling statement node. + :rtype: NodeNG or None + """ + stmts = self.parent.child_sequence(self) + index = stmts.index(self) + try: + return stmts[index + 1] + except IndexError: + pass + + def previous_sibling(self): + """The previous sibling statement. + + :returns: The previous sibling statement node. + :rtype: NodeNG or None + """ + stmts = self.parent.child_sequence(self) + index = stmts.index(self) + if index >= 1: + return stmts[index - 1] + return None + + +class _BaseContainer( + mixins.ParentAssignTypeMixin, NodeNG, bases.Instance, metaclass=abc.ABCMeta +): + """Base class for Set, FrozenSet, Tuple and List.""" + + _astroid_fields = ("elts",) + + def __init__(self, lineno=None, col_offset=None, parent=None): + """ + :param lineno: The line that this node appears on in the source code. + :type lineno: int or None + + :param col_offset: The column that this node appears on in the + source code. + :type col_offset: int or None + + :param parent: The parent node in the syntax tree. + :type parent: NodeNG or None + """ + self.elts = [] + """The elements in the node. + + :type: list(NodeNG) + """ + + super().__init__(lineno, col_offset, parent) + + def postinit(self, elts): + """Do some setup after initialisation. + + :param elts: The list of elements the that node contains. + :type elts: list(NodeNG) + """ + self.elts = elts + + @classmethod + def from_elements(cls, elts=None): + """Create a node of this type from the given list of elements. + + :param elts: The list of elements that the node should contain. + :type elts: list(NodeNG) + + :returns: A new node containing the given elements. + :rtype: NodeNG + """ + node = cls() + if elts is None: + node.elts = [] + else: + node.elts = [const_factory(e) if _is_const(e) else e for e in elts] + return node + + def itered(self): + """An iterator over the elements this node contains. + + :returns: The contents of this node. + :rtype: iterable(NodeNG) + """ + return self.elts + + def bool_value(self, context=None): + """Determine the boolean value of this node. + + :returns: The boolean value of this node. + :rtype: bool or Uninferable + """ + return bool(self.elts) + + @abc.abstractmethod + def pytype(self): + """Get the name of the type that this node represents. + + :returns: The name of the type. + :rtype: str + """ + + def get_children(self): + yield from self.elts + + +class LookupMixIn: + """Mixin to look up a name in the right scope.""" + + @lru_cache(maxsize=None) + def lookup(self, name): + """Lookup where the given variable is assigned. + + The lookup starts from self's scope. If self is not a frame itself + and the name is found in the inner frame locals, statements will be + filtered to remove ignorable statements according to self's location. + + :param name: The name of the variable to find assignments for. + :type name: str + + :returns: The scope node and the list of assignments associated to the + given name according to the scope where it has been found (locals, + globals or builtin). + :rtype: tuple(str, list(NodeNG)) + """ + return self.scope().scope_lookup(self, name) + + def ilookup(self, name): + """Lookup the inferred values of the given variable. + + :param name: The variable name to find values for. + :type name: str + + :returns: The inferred values of the statements returned from + :meth:`lookup`. + :rtype: iterable + """ + frame, stmts = self.lookup(name) + context = contextmod.InferenceContext() + return bases._infer_stmts(stmts, context, frame) + + def _get_filtered_node_statements(self, nodes): + statements = [(node, node.statement()) for node in nodes] + # Next we check if we have ExceptHandlers that are parent + # of the underlying variable, in which case the last one survives + if len(statements) > 1 and all( + isinstance(stmt, ExceptHandler) for _, stmt in statements + ): + statements = [ + (node, stmt) for node, stmt in statements if stmt.parent_of(self) + ] + return statements + + def _filter_stmts(self, stmts, frame, offset): + """Filter the given list of statements to remove ignorable statements. + + If self is not a frame itself and the name is found in the inner + frame locals, statements will be filtered to remove ignorable + statements according to self's location. + + :param stmts: The statements to filter. + :type stmts: list(NodeNG) + + :param frame: The frame that all of the given statements belong to. + :type frame: NodeNG + + :param offset: The line offset to filter statements up to. + :type offset: int + + :returns: The filtered statements. + :rtype: list(NodeNG) + """ + # if offset == -1, my actual frame is not the inner frame but its parent + # + # class A(B): pass + # + # we need this to resolve B correctly + if offset == -1: + myframe = self.frame().parent.frame() + else: + myframe = self.frame() + # If the frame of this node is the same as the statement + # of this node, then the node is part of a class or + # a function definition and the frame of this node should be the + # the upper frame, not the frame of the definition. + # For more information why this is important, + # see Pylint issue #295. + # For example, for 'b', the statement is the same + # as the frame / scope: + # + # def test(b=1): + # ... + + if self.statement() is myframe and myframe.parent: + myframe = myframe.parent.frame() + mystmt = self.statement() + # line filtering if we are in the same frame + # + # take care node may be missing lineno information (this is the case for + # nodes inserted for living objects) + if myframe is frame and mystmt.fromlineno is not None: + assert mystmt.fromlineno is not None, mystmt + mylineno = mystmt.fromlineno + offset + else: + # disabling lineno filtering + mylineno = 0 + + _stmts = [] + _stmt_parents = [] + statements = self._get_filtered_node_statements(stmts) + + for node, stmt in statements: + # line filtering is on and we have reached our location, break + if stmt.fromlineno > mylineno > 0: + break + # Ignore decorators with the same name as the + # decorated function + # Fixes issue #375 + if mystmt is stmt and is_from_decorator(self): + continue + assert hasattr(node, "assign_type"), ( + node, + node.scope(), + node.scope().locals, + ) + assign_type = node.assign_type() + if node.has_base(self): + break + + _stmts, done = assign_type._get_filtered_stmts(self, node, _stmts, mystmt) + if done: + break + + optional_assign = assign_type.optional_assign + if optional_assign and assign_type.parent_of(self): + # we are inside a loop, loop var assignment is hiding previous + # assignment + _stmts = [node] + _stmt_parents = [stmt.parent] + continue + + if isinstance(assign_type, NamedExpr): + _stmts = [node] + continue + + # XXX comment various branches below!!! + try: + pindex = _stmt_parents.index(stmt.parent) + except ValueError: + pass + else: + # we got a parent index, this means the currently visited node + # is at the same block level as a previously visited node + if _stmts[pindex].assign_type().parent_of(assign_type): + # both statements are not at the same block level + continue + # if currently visited node is following previously considered + # assignment and both are not exclusive, we can drop the + # previous one. For instance in the following code :: + # + # if a: + # x = 1 + # else: + # x = 2 + # print x + # + # we can't remove neither x = 1 nor x = 2 when looking for 'x' + # of 'print x'; while in the following :: + # + # x = 1 + # x = 2 + # print x + # + # we can remove x = 1 when we see x = 2 + # + # moreover, on loop assignment types, assignment won't + # necessarily be done if the loop has no iteration, so we don't + # want to clear previous assignments if any (hence the test on + # optional_assign) + if not (optional_assign or are_exclusive(_stmts[pindex], node)): + if ( + # In case of partial function node, if the statement is different + # from the origin function then it can be deleted otherwise it should + # remain to be able to correctly infer the call to origin function. + not node.is_function + or node.qname() != "PartialFunction" + or node.name != _stmts[pindex].name + ): + del _stmt_parents[pindex] + del _stmts[pindex] + if isinstance(node, AssignName): + if not optional_assign and stmt.parent is mystmt.parent: + _stmts = [] + _stmt_parents = [] + elif isinstance(node, DelName): + _stmts = [] + _stmt_parents = [] + continue + if not are_exclusive(self, node): + _stmts.append(node) + _stmt_parents.append(stmt.parent) + return _stmts + + +# Name classes + + +class AssignName( + mixins.NoChildrenMixin, LookupMixIn, mixins.ParentAssignTypeMixin, NodeNG +): + """Variation of :class:`ast.Assign` representing assignment to a name. + + An :class:`AssignName` is the name of something that is assigned to. + This includes variables defined in a function signature or in a loop. + + >>> node = astroid.extract_node('variable = range(10)') + >>> node + + >>> list(node.get_children()) + [, ] + >>> list(node.get_children())[0].as_string() + 'variable' + """ + + _other_fields = ("name",) + + def __init__(self, name=None, lineno=None, col_offset=None, parent=None): + """ + :param name: The name that is assigned to. + :type name: str or None + + :param lineno: The line that this node appears on in the source code. + :type lineno: int or None + + :param col_offset: The column that this node appears on in the + source code. + :type col_offset: int or None + + :param parent: The parent node in the syntax tree. + :type parent: NodeNG or None + """ + self.name = name + """The name that is assigned to. + + :type: str or None + """ + + super().__init__(lineno, col_offset, parent) + + +class DelName( + mixins.NoChildrenMixin, LookupMixIn, mixins.ParentAssignTypeMixin, NodeNG +): + """Variation of :class:`ast.Delete` representing deletion of a name. + + A :class:`DelName` is the name of something that is deleted. + + >>> node = astroid.extract_node("del variable #@") + >>> list(node.get_children()) + [] + >>> list(node.get_children())[0].as_string() + 'variable' + """ + + _other_fields = ("name",) + + def __init__(self, name=None, lineno=None, col_offset=None, parent=None): + """ + :param name: The name that is being deleted. + :type name: str or None + + :param lineno: The line that this node appears on in the source code. + :type lineno: int or None + + :param col_offset: The column that this node appears on in the + source code. + :type col_offset: int or None + + :param parent: The parent node in the syntax tree. + :type parent: NodeNG or None + """ + self.name = name + """The name that is being deleted. + + :type: str or None + """ + + super().__init__(lineno, col_offset, parent) + + +class Name(mixins.NoChildrenMixin, LookupMixIn, NodeNG): + """Class representing an :class:`ast.Name` node. + + A :class:`Name` node is something that is named, but not covered by + :class:`AssignName` or :class:`DelName`. + + >>> node = astroid.extract_node('range(10)') + >>> node + + >>> list(node.get_children()) + [, ] + >>> list(node.get_children())[0].as_string() + 'range' + """ + + _other_fields = ("name",) + + def __init__(self, name=None, lineno=None, col_offset=None, parent=None): + """ + :param name: The name that this node refers to. + :type name: str or None + + :param lineno: The line that this node appears on in the source code. + :type lineno: int or None + + :param col_offset: The column that this node appears on in the + source code. + :type col_offset: int or None + + :param parent: The parent node in the syntax tree. + :type parent: NodeNG or None + """ + self.name = name + """The name that this node refers to. + + :type: str or None + """ + + super().__init__(lineno, col_offset, parent) + + def _get_name_nodes(self): + yield self + + for child_node in self.get_children(): + yield from child_node._get_name_nodes() + + +class Arguments(mixins.AssignTypeMixin, NodeNG): + """Class representing an :class:`ast.arguments` node. + + An :class:`Arguments` node represents that arguments in a + function definition. + + >>> node = astroid.extract_node('def foo(bar): pass') + >>> node + + >>> node.args + + """ + + # Python 3.4+ uses a different approach regarding annotations, + # each argument is a new class, _ast.arg, which exposes an + # 'annotation' attribute. In astroid though, arguments are exposed + # as is in the Arguments node and the only way to expose annotations + # is by using something similar with Python 3.3: + # - we expose 'varargannotation' and 'kwargannotation' of annotations + # of varargs and kwargs. + # - we expose 'annotation', a list with annotations for + # for each normal argument. If an argument doesn't have an + # annotation, its value will be None. + # pylint: disable=too-many-instance-attributes + _astroid_fields = ( + "args", + "defaults", + "kwonlyargs", + "posonlyargs", + "posonlyargs_annotations", + "kw_defaults", + "annotations", + "varargannotation", + "kwargannotation", + "kwonlyargs_annotations", + "type_comment_args", + "type_comment_kwonlyargs", + "type_comment_posonlyargs", + ) + varargannotation = None + """The type annotation for the variable length arguments. + + :type: NodeNG + """ + kwargannotation = None + """The type annotation for the variable length keyword arguments. + + :type: NodeNG + """ + + _other_fields = ("vararg", "kwarg") + + def __init__(self, vararg=None, kwarg=None, parent=None): + """ + :param vararg: The name of the variable length arguments. + :type vararg: str or None + + :param kwarg: The name of the variable length keyword arguments. + :type kwarg: str or None + + :param parent: The parent node in the syntax tree. + :type parent: NodeNG or None + """ + super().__init__(parent=parent) + self.vararg = vararg + """The name of the variable length arguments. + + :type: str or None + """ + + self.kwarg = kwarg + """The name of the variable length keyword arguments. + + :type: str or None + """ + + self.args = [] + """The names of the required arguments. + + :type: list(AssignName) + """ + + self.defaults = [] + """The default values for arguments that can be passed positionally. + + :type: list(NodeNG) + """ + + self.kwonlyargs = [] + """The keyword arguments that cannot be passed positionally. + + :type: list(AssignName) + """ + + self.posonlyargs = [] + """The arguments that can only be passed positionally. + + :type: list(AssignName) + """ + + self.kw_defaults = [] + """The default values for keyword arguments that cannot be passed positionally. + + :type: list(NodeNG) + """ + + self.annotations = [] + """The type annotations of arguments that can be passed positionally. + + :type: list(NodeNG) + """ + + self.posonlyargs_annotations = [] + """The type annotations of arguments that can only be passed positionally. + + :type: list(NodeNG) + """ + + self.kwonlyargs_annotations = [] + """The type annotations of arguments that cannot be passed positionally. + + :type: list(NodeNG) + """ + + self.type_comment_args = [] + """The type annotation, passed by a type comment, of each argument. + + If an argument does not have a type comment, + the value for that argument will be None. + + :type: list(NodeNG or None) + """ + + self.type_comment_kwonlyargs = [] + """The type annotation, passed by a type comment, of each keyword only argument. + + If an argument does not have a type comment, + the value for that argument will be None. + + :type: list(NodeNG or None) + """ + + self.type_comment_posonlyargs = [] + """The type annotation, passed by a type comment, of each positional argument. + + If an argument does not have a type comment, + the value for that argument will be None. + + :type: list(NodeNG or None) + """ + + # pylint: disable=too-many-arguments + def postinit( + self, + args, + defaults, + kwonlyargs, + kw_defaults, + annotations, + posonlyargs=None, + kwonlyargs_annotations=None, + posonlyargs_annotations=None, + varargannotation=None, + kwargannotation=None, + type_comment_args=None, + type_comment_kwonlyargs=None, + type_comment_posonlyargs=None, + ): + """Do some setup after initialisation. + + :param args: The names of the required arguments. + :type args: list(AssignName) + + :param defaults: The default values for arguments that can be passed + positionally. + :type defaults: list(NodeNG) + + :param kwonlyargs: The keyword arguments that cannot be passed + positionally. + :type kwonlyargs: list(AssignName) + + :param posonlyargs: The arguments that can only be passed + positionally. + :type kwonlyargs: list(AssignName) + + :param kw_defaults: The default values for keyword arguments that + cannot be passed positionally. + :type kw_defaults: list(NodeNG) + + :param annotations: The type annotations of arguments that can be + passed positionally. + :type annotations: list(NodeNG) + + :param kwonlyargs_annotations: The type annotations of arguments that + cannot be passed positionally. This should always be passed in + Python 3. + :type kwonlyargs_annotations: list(NodeNG) + + :param posonlyargs_annotations: The type annotations of arguments that + can only be passed positionally. This should always be passed in + Python 3. + :type posonlyargs_annotations: list(NodeNG) + + :param varargannotation: The type annotation for the variable length + arguments. + :type varargannotation: NodeNG + + :param kwargannotation: The type annotation for the variable length + keyword arguments. + :type kwargannotation: NodeNG + + :param type_comment_args: The type annotation, + passed by a type comment, of each argument. + :type type_comment_args: list(NodeNG or None) + + :param type_comment_args: The type annotation, + passed by a type comment, of each keyword only argument. + :type type_comment_args: list(NodeNG or None) + + :param type_comment_args: The type annotation, + passed by a type comment, of each positional argument. + :type type_comment_args: list(NodeNG or None) + """ + self.args = args + self.defaults = defaults + self.kwonlyargs = kwonlyargs + self.posonlyargs = posonlyargs + self.kw_defaults = kw_defaults + self.annotations = annotations + self.kwonlyargs_annotations = kwonlyargs_annotations + self.posonlyargs_annotations = posonlyargs_annotations + self.varargannotation = varargannotation + self.kwargannotation = kwargannotation + self.type_comment_args = type_comment_args + self.type_comment_kwonlyargs = type_comment_kwonlyargs + self.type_comment_posonlyargs = type_comment_posonlyargs + + # pylint: disable=too-many-arguments + + def _infer_name(self, frame, name): + if self.parent is frame: + return name + return None + + @decorators.cachedproperty + def fromlineno(self): + """The first line that this node appears on in the source code. + + :type: int or None + """ + lineno = super().fromlineno + return max(lineno, self.parent.fromlineno or 0) + + @decorators.cachedproperty + def arguments(self): + """Get all the arguments for this node, including positional only and positional and keyword""" + return list(itertools.chain((self.posonlyargs or ()), self.args or ())) + + def format_args(self): + """Get the arguments formatted as string. + + :returns: The formatted arguments. + :rtype: str + """ + result = [] + positional_only_defaults = [] + positional_or_keyword_defaults = self.defaults + if self.defaults: + args = self.args or [] + positional_or_keyword_defaults = self.defaults[-len(args) :] + positional_only_defaults = self.defaults[: len(self.defaults) - len(args)] + + if self.posonlyargs: + result.append( + _format_args( + self.posonlyargs, + positional_only_defaults, + self.posonlyargs_annotations, + ) + ) + result.append("/") + if self.args: + result.append( + _format_args( + self.args, + positional_or_keyword_defaults, + getattr(self, "annotations", None), + ) + ) + if self.vararg: + result.append("*%s" % self.vararg) + if self.kwonlyargs: + if not self.vararg: + result.append("*") + result.append( + _format_args( + self.kwonlyargs, self.kw_defaults, self.kwonlyargs_annotations + ) + ) + if self.kwarg: + result.append("**%s" % self.kwarg) + return ", ".join(result) + + def default_value(self, argname): + """Get the default value for an argument. + + :param argname: The name of the argument to get the default value for. + :type argname: str + + :raises NoDefault: If there is no default value defined for the + given argument. + """ + args = self.arguments + index = _find_arg(argname, args)[0] + if index is not None: + idx = index - (len(args) - len(self.defaults)) + if idx >= 0: + return self.defaults[idx] + index = _find_arg(argname, self.kwonlyargs)[0] + if index is not None and self.kw_defaults[index] is not None: + return self.kw_defaults[index] + raise exceptions.NoDefault(func=self.parent, name=argname) + + def is_argument(self, name): + """Check if the given name is defined in the arguments. + + :param name: The name to check for. + :type name: str + + :returns: True if the given name is defined in the arguments, + False otherwise. + :rtype: bool + """ + if name == self.vararg: + return True + if name == self.kwarg: + return True + return ( + self.find_argname(name, rec=True)[1] is not None + or self.kwonlyargs + and _find_arg(name, self.kwonlyargs, rec=True)[1] is not None + ) + + def find_argname(self, argname, rec=False): + """Get the index and :class:`AssignName` node for given name. + + :param argname: The name of the argument to search for. + :type argname: str + + :param rec: Whether or not to include arguments in unpacked tuples + in the search. + :type rec: bool + + :returns: The index and node for the argument. + :rtype: tuple(str or None, AssignName or None) + """ + if self.arguments: + return _find_arg(argname, self.arguments, rec) + return None, None + + def get_children(self): + yield from self.posonlyargs or () + + for elt in self.posonlyargs_annotations: + if elt is not None: + yield elt + + yield from self.args or () + + yield from self.defaults + yield from self.kwonlyargs + + for elt in self.kw_defaults: + if elt is not None: + yield elt + + for elt in self.annotations: + if elt is not None: + yield elt + + if self.varargannotation is not None: + yield self.varargannotation + + if self.kwargannotation is not None: + yield self.kwargannotation + + for elt in self.kwonlyargs_annotations: + if elt is not None: + yield elt + + +def _find_arg(argname, args, rec=False): + for i, arg in enumerate(args): + if isinstance(arg, Tuple): + if rec: + found = _find_arg(argname, arg.elts) + if found[0] is not None: + return found + elif arg.name == argname: + return i, arg + return None, None + + +def _format_args(args, defaults=None, annotations=None): + values = [] + if args is None: + return "" + if annotations is None: + annotations = [] + if defaults is not None: + default_offset = len(args) - len(defaults) + packed = itertools.zip_longest(args, annotations) + for i, (arg, annotation) in enumerate(packed): + if isinstance(arg, Tuple): + values.append("(%s)" % _format_args(arg.elts)) + else: + argname = arg.name + default_sep = "=" + if annotation is not None: + argname += ": " + annotation.as_string() + default_sep = " = " + values.append(argname) + + if defaults is not None and i >= default_offset: + if defaults[i - default_offset] is not None: + values[-1] += default_sep + defaults[i - default_offset].as_string() + return ", ".join(values) + + +class AssignAttr(mixins.ParentAssignTypeMixin, NodeNG): + """Variation of :class:`ast.Assign` representing assignment to an attribute. + + >>> node = astroid.extract_node('self.attribute = range(10)') + >>> node + + >>> list(node.get_children()) + [, ] + >>> list(node.get_children())[0].as_string() + 'self.attribute' + """ + + _astroid_fields = ("expr",) + _other_fields = ("attrname",) + expr = None + """What has the attribute that is being assigned to. + + :type: NodeNG or None + """ + + def __init__(self, attrname=None, lineno=None, col_offset=None, parent=None): + """ + :param attrname: The name of the attribute being assigned to. + :type attrname: str or None + + :param lineno: The line that this node appears on in the source code. + :type lineno: int or None + + :param col_offset: The column that this node appears on in the + source code. + :type col_offset: int or None + + :param parent: The parent node in the syntax tree. + :type parent: NodeNG or None + """ + self.attrname = attrname + """The name of the attribute being assigned to. + + :type: str or None + """ + + super().__init__(lineno, col_offset, parent) + + def postinit(self, expr=None): + """Do some setup after initialisation. + + :param expr: What has the attribute that is being assigned to. + :type expr: NodeNG or None + """ + self.expr = expr + + def get_children(self): + yield self.expr + + +class Assert(Statement): + """Class representing an :class:`ast.Assert` node. + + An :class:`Assert` node represents an assert statement. + + >>> node = astroid.extract_node('assert len(things) == 10, "Not enough things"') + >>> node + + """ + + _astroid_fields = ("test", "fail") + test = None + """The test that passes or fails the assertion. + + :type: NodeNG or None + """ + fail = None + """The message shown when the assertion fails. + + :type: NodeNG or None + """ + + def postinit(self, test=None, fail=None): + """Do some setup after initialisation. + + :param test: The test that passes or fails the assertion. + :type test: NodeNG or None + + :param fail: The message shown when the assertion fails. + :type fail: NodeNG or None + """ + self.fail = fail + self.test = test + + def get_children(self): + yield self.test + + if self.fail is not None: + yield self.fail + + +class Assign(mixins.AssignTypeMixin, Statement): + """Class representing an :class:`ast.Assign` node. + + An :class:`Assign` is a statement where something is explicitly + asssigned to. + + >>> node = astroid.extract_node('variable = range(10)') + >>> node + + """ + + _astroid_fields = ("targets", "value") + _other_other_fields = ("type_annotation",) + targets = None + """What is being assigned to. + + :type: list(NodeNG) or None + """ + value = None + """The value being assigned to the variables. + + :type: NodeNG or None + """ + type_annotation = None + """If present, this will contain the type annotation passed by a type comment + + :type: NodeNG or None + """ + + def postinit(self, targets=None, value=None, type_annotation=None): + """Do some setup after initialisation. + + :param targets: What is being assigned to. + :type targets: list(NodeNG) or None + + :param value: The value being assigned to the variables. + :type: NodeNG or None + """ + self.targets = targets + self.value = value + self.type_annotation = type_annotation + + def get_children(self): + yield from self.targets + + yield self.value + + @decorators.cached + def _get_assign_nodes(self): + return [self] + list(self.value._get_assign_nodes()) + + def _get_yield_nodes_skip_lambdas(self): + yield from self.value._get_yield_nodes_skip_lambdas() + + +class AnnAssign(mixins.AssignTypeMixin, Statement): + """Class representing an :class:`ast.AnnAssign` node. + + An :class:`AnnAssign` is an assignment with a type annotation. + + >>> node = astroid.extract_node('variable: List[int] = range(10)') + >>> node + + """ + + _astroid_fields = ("target", "annotation", "value") + _other_fields = ("simple",) + target = None + """What is being assigned to. + + :type: NodeNG or None + """ + annotation = None + """The type annotation of what is being assigned to. + + :type: NodeNG + """ + value = None + """The value being assigned to the variables. + + :type: NodeNG or None + """ + simple = None + """Whether :attr:`target` is a pure name or a complex statement. + + :type: int + """ + + def postinit(self, target, annotation, simple, value=None): + """Do some setup after initialisation. + + :param target: What is being assigned to. + :type target: NodeNG + + :param annotation: The type annotation of what is being assigned to. + :type: NodeNG + + :param simple: Whether :attr:`target` is a pure name + or a complex statement. + :type simple: int + + :param value: The value being assigned to the variables. + :type: NodeNG or None + """ + self.target = target + self.annotation = annotation + self.value = value + self.simple = simple + + def get_children(self): + yield self.target + yield self.annotation + + if self.value is not None: + yield self.value + + +class AugAssign(mixins.AssignTypeMixin, Statement): + """Class representing an :class:`ast.AugAssign` node. + + An :class:`AugAssign` is an assignment paired with an operator. + + >>> node = astroid.extract_node('variable += 1') + >>> node + + """ + + _astroid_fields = ("target", "value") + _other_fields = ("op",) + target = None + """What is being assigned to. + + :type: NodeNG or None + """ + value = None + """The value being assigned to the variable. + + :type: NodeNG or None + """ + + def __init__(self, op=None, lineno=None, col_offset=None, parent=None): + """ + :param op: The operator that is being combined with the assignment. + This includes the equals sign. + :type op: str or None + + :param lineno: The line that this node appears on in the source code. + :type lineno: int or None + + :param col_offset: The column that this node appears on in the + source code. + :type col_offset: int or None + + :param parent: The parent node in the syntax tree. + :type parent: NodeNG or None + """ + self.op = op + """The operator that is being combined with the assignment. + + This includes the equals sign. + + :type: str or None + """ + + super().__init__(lineno, col_offset, parent) + + def postinit(self, target=None, value=None): + """Do some setup after initialisation. + + :param target: What is being assigned to. + :type target: NodeNG or None + + :param value: The value being assigned to the variable. + :type: NodeNG or None + """ + self.target = target + self.value = value + + # This is set by inference.py + def _infer_augassign(self, context=None): + raise NotImplementedError + + def type_errors(self, context=None): + """Get a list of type errors which can occur during inference. + + Each TypeError is represented by a :class:`BadBinaryOperationMessage` , + which holds the original exception. + + :returns: The list of possible type errors. + :rtype: list(BadBinaryOperationMessage) + """ + try: + results = self._infer_augassign(context=context) + return [ + result + for result in results + if isinstance(result, util.BadBinaryOperationMessage) + ] + except exceptions.InferenceError: + return [] + + def get_children(self): + yield self.target + yield self.value + + +class Repr(NodeNG): + """Class representing an :class:`ast.Repr` node. + + A :class:`Repr` node represents the backtick syntax, + which is a deprecated alias for :func:`repr` removed in Python 3. + + >>> node = astroid.extract_node('`variable`') + >>> node + + """ + + _astroid_fields = ("value",) + value = None + """What is having :func:`repr` called on it. + + :type: NodeNG or None + """ + + def postinit(self, value=None): + """Do some setup after initialisation. + + :param value: What is having :func:`repr` called on it. + :type value: NodeNG or None + """ + self.value = value + + +class BinOp(NodeNG): + """Class representing an :class:`ast.BinOp` node. + + A :class:`BinOp` node is an application of a binary operator. + + >>> node = astroid.extract_node('a + b') + >>> node + + """ + + _astroid_fields = ("left", "right") + _other_fields = ("op",) + left = None + """What is being applied to the operator on the left side. + + :type: NodeNG or None + """ + right = None + """What is being applied to the operator on the right side. + + :type: NodeNG or None + """ + + def __init__(self, op=None, lineno=None, col_offset=None, parent=None): + """ + :param op: The operator. + :type: str or None + + :param lineno: The line that this node appears on in the source code. + :type lineno: int or None + + :param col_offset: The column that this node appears on in the + source code. + :type col_offset: int or None + + :param parent: The parent node in the syntax tree. + :type parent: NodeNG or None + """ + self.op = op + """The operator. + + :type: str or None + """ + + super().__init__(lineno, col_offset, parent) + + def postinit(self, left=None, right=None): + """Do some setup after initialisation. + + :param left: What is being applied to the operator on the left side. + :type left: NodeNG or None + + :param right: What is being applied to the operator on the right side. + :type right: NodeNG or None + """ + self.left = left + self.right = right + + # This is set by inference.py + def _infer_binop(self, context=None): + raise NotImplementedError + + def type_errors(self, context=None): + """Get a list of type errors which can occur during inference. + + Each TypeError is represented by a :class:`BadBinaryOperationMessage`, + which holds the original exception. + + :returns: The list of possible type errors. + :rtype: list(BadBinaryOperationMessage) + """ + try: + results = self._infer_binop(context=context) + return [ + result + for result in results + if isinstance(result, util.BadBinaryOperationMessage) + ] + except exceptions.InferenceError: + return [] + + def get_children(self): + yield self.left + yield self.right + + def op_precedence(self): + return OP_PRECEDENCE[self.op] + + def op_left_associative(self): + # 2**3**4 == 2**(3**4) + return self.op != "**" + + +class BoolOp(NodeNG): + """Class representing an :class:`ast.BoolOp` node. + + A :class:`BoolOp` is an application of a boolean operator. + + >>> node = astroid.extract_node('a and b') + >>> node + + """ + + _astroid_fields = ("values",) + _other_fields = ("op",) + values = None + """The values being applied to the operator. + + :type: list(NodeNG) or None + """ + + def __init__(self, op=None, lineno=None, col_offset=None, parent=None): + """ + :param op: The operator. + :type: str or None + + :param lineno: The line that this node appears on in the source code. + :type lineno: int or None + + :param col_offset: The column that this node appears on in the + source code. + :type col_offset: int or None + + :param parent: The parent node in the syntax tree. + :type parent: NodeNG or None + """ + self.op = op + """The operator. + + :type: str or None + """ + + super().__init__(lineno, col_offset, parent) + + def postinit(self, values=None): + """Do some setup after initialisation. + + :param values: The values being applied to the operator. + :type values: list(NodeNG) or None + """ + self.values = values + + def get_children(self): + yield from self.values + + def op_precedence(self): + return OP_PRECEDENCE[self.op] + + +class Break(mixins.NoChildrenMixin, Statement): + """Class representing an :class:`ast.Break` node. + + >>> node = astroid.extract_node('break') + >>> node + + """ + + +class Call(NodeNG): + """Class representing an :class:`ast.Call` node. + + A :class:`Call` node is a call to a function, method, etc. + + >>> node = astroid.extract_node('function()') + >>> node + + """ + + _astroid_fields = ("func", "args", "keywords") + func = None + """What is being called. + + :type: NodeNG or None + """ + args = None + """The positional arguments being given to the call. + + :type: list(NodeNG) or None + """ + keywords = None + """The keyword arguments being given to the call. + + :type: list(NodeNG) or None + """ + + def postinit(self, func=None, args=None, keywords=None): + """Do some setup after initialisation. + + :param func: What is being called. + :type func: NodeNG or None + + :param args: The positional arguments being given to the call. + :type args: list(NodeNG) or None + + :param keywords: The keyword arguments being given to the call. + :type keywords: list(NodeNG) or None + """ + self.func = func + self.args = args + self.keywords = keywords + + @property + def starargs(self): + """The positional arguments that unpack something. + + :type: list(Starred) + """ + args = self.args or [] + return [arg for arg in args if isinstance(arg, Starred)] + + @property + def kwargs(self): + """The keyword arguments that unpack something. + + :type: list(Keyword) + """ + keywords = self.keywords or [] + return [keyword for keyword in keywords if keyword.arg is None] + + def get_children(self): + yield self.func + + yield from self.args + + yield from self.keywords or () + + +class Compare(NodeNG): + """Class representing an :class:`ast.Compare` node. + + A :class:`Compare` node indicates a comparison. + + >>> node = astroid.extract_node('a <= b <= c') + >>> node + + >>> node.ops + [('<=', ), ('<=', )] + """ + + _astroid_fields = ("left", "ops") + left = None + """The value at the left being applied to a comparison operator. + + :type: NodeNG or None + """ + ops = None + """The remainder of the operators and their relevant right hand value. + + :type: list(tuple(str, NodeNG)) or None + """ + + def postinit(self, left=None, ops=None): + """Do some setup after initialisation. + + :param left: The value at the left being applied to a comparison + operator. + :type left: NodeNG or None + + :param ops: The remainder of the operators + and their relevant right hand value. + :type ops: list(tuple(str, NodeNG)) or None + """ + self.left = left + self.ops = ops + + def get_children(self): + """Get the child nodes below this node. + + Overridden to handle the tuple fields and skip returning the operator + strings. + + :returns: The children. + :rtype: iterable(NodeNG) + """ + yield self.left + for _, comparator in self.ops: + yield comparator # we don't want the 'op' + + def last_child(self): + """An optimized version of list(get_children())[-1] + + :returns: The last child. + :rtype: NodeNG + """ + # XXX maybe if self.ops: + return self.ops[-1][1] + # return self.left + + +class Comprehension(NodeNG): + """Class representing an :class:`ast.comprehension` node. + + A :class:`Comprehension` indicates the loop inside any type of + comprehension including generator expressions. + + >>> node = astroid.extract_node('[x for x in some_values]') + >>> list(node.get_children()) + [, ] + >>> list(node.get_children())[1].as_string() + 'for x in some_values' + """ + + _astroid_fields = ("target", "iter", "ifs") + _other_fields = ("is_async",) + target = None + """What is assigned to by the comprehension. + + :type: NodeNG or None + """ + iter = None + """What is iterated over by the comprehension. + + :type: NodeNG or None + """ + ifs = None + """The contents of any if statements that filter the comprehension. + + :type: list(NodeNG) or None + """ + is_async = None + """Whether this is an asynchronous comprehension or not. + + :type: bool or None + """ + + def __init__(self, parent=None): + """ + :param parent: The parent node in the syntax tree. + :type parent: NodeNG or None + """ + super().__init__() + self.parent = parent + + # pylint: disable=redefined-builtin; same name as builtin ast module. + def postinit(self, target=None, iter=None, ifs=None, is_async=None): + """Do some setup after initialisation. + + :param target: What is assigned to by the comprehension. + :type target: NodeNG or None + + :param iter: What is iterated over by the comprehension. + :type iter: NodeNG or None + + :param ifs: The contents of any if statements that filter + the comprehension. + :type ifs: list(NodeNG) or None + + :param is_async: Whether this is an asynchronous comprehension or not. + :type: bool or None + """ + self.target = target + self.iter = iter + self.ifs = ifs + self.is_async = is_async + + optional_assign = True + """Whether this node optionally assigns a variable. + + :type: bool + """ + + def assign_type(self): + """The type of assignment that this node performs. + + :returns: The assignment type. + :rtype: NodeNG + """ + return self + + def _get_filtered_stmts(self, lookup_node, node, stmts, mystmt): + """method used in filter_stmts""" + if self is mystmt: + if isinstance(lookup_node, (Const, Name)): + return [lookup_node], True + + elif self.statement() is mystmt: + # original node's statement is the assignment, only keeps + # current node (gen exp, list comp) + + return [node], True + + return stmts, False + + def get_children(self): + yield self.target + yield self.iter + + yield from self.ifs + + +class Const(mixins.NoChildrenMixin, NodeNG, bases.Instance): + """Class representing any constant including num, str, bool, None, bytes. + + >>> node = astroid.extract_node('(5, "This is a string.", True, None, b"bytes")') + >>> node + + >>> list(node.get_children()) + [, + , + , + , + ] + """ + + _other_fields = ("value",) + + def __init__(self, value, lineno=None, col_offset=None, parent=None): + """ + :param value: The value that the constant represents. + :type value: object + + :param lineno: The line that this node appears on in the source code. + :type lineno: int or None + + :param col_offset: The column that this node appears on in the + source code. + :type col_offset: int or None + + :param parent: The parent node in the syntax tree. + :type parent: NodeNG or None + """ + self.value = value + """The value that the constant represents. + + :type: object + """ + + super().__init__(lineno, col_offset, parent) + + def __getattr__(self, name): + # This is needed because of Proxy's __getattr__ method. + # Calling object.__new__ on this class without calling + # __init__ would result in an infinite loop otherwise + # since __getattr__ is called when an attribute doesn't + # exist and self._proxied indirectly calls self.value + # and Proxy __getattr__ calls self.value + if name == "value": + raise AttributeError + return super().__getattr__(name) + + def getitem(self, index, context=None): + """Get an item from this node if subscriptable. + + :param index: The node to use as a subscript index. + :type index: Const or Slice + + :raises AstroidTypeError: When the given index cannot be used as a + subscript index, or if this node is not subscriptable. + """ + if isinstance(index, Const): + index_value = index.value + elif isinstance(index, Slice): + index_value = _infer_slice(index, context=context) + + else: + raise exceptions.AstroidTypeError( + "Could not use type {} as subscript index".format(type(index)) + ) + + try: + if isinstance(self.value, (str, bytes)): + return Const(self.value[index_value]) + except IndexError as exc: + raise exceptions.AstroidIndexError( + message="Index {index!r} out of range", + node=self, + index=index, + context=context, + ) from exc + except TypeError as exc: + raise exceptions.AstroidTypeError( + message="Type error {error!r}", node=self, index=index, context=context + ) from exc + + raise exceptions.AstroidTypeError("%r (value=%s)" % (self, self.value)) + + def has_dynamic_getattr(self): + """Check if the node has a custom __getattr__ or __getattribute__. + + :returns: True if the class has a custom + __getattr__ or __getattribute__, False otherwise. + For a :class:`Const` this is always ``False``. + :rtype: bool + """ + return False + + def itered(self): + """An iterator over the elements this node contains. + + :returns: The contents of this node. + :rtype: iterable(Const) + + :raises TypeError: If this node does not represent something that is iterable. + """ + if isinstance(self.value, str): + return [const_factory(elem) for elem in self.value] + raise TypeError("Cannot iterate over type {!r}".format(type(self.value))) + + def pytype(self): + """Get the name of the type that this node represents. + + :returns: The name of the type. + :rtype: str + """ + return self._proxied.qname() + + def bool_value(self, context=None): + """Determine the boolean value of this node. + + :returns: The boolean value of this node. + :rtype: bool + """ + return bool(self.value) + + +class Continue(mixins.NoChildrenMixin, Statement): + """Class representing an :class:`ast.Continue` node. + + >>> node = astroid.extract_node('continue') + >>> node + + """ + + +class Decorators(NodeNG): + """A node representing a list of decorators. + + A :class:`Decorators` is the decorators that are applied to + a method or function. + + >>> node = astroid.extract_node(''' + @property + def my_property(self): + return 3 + ''') + >>> node + + >>> list(node.get_children())[0] + + """ + + _astroid_fields = ("nodes",) + nodes = None + """The decorators that this node contains. + + :type: list(Name or Call) or None + """ + + def postinit(self, nodes): + """Do some setup after initialisation. + + :param nodes: The decorators that this node contains. + :type nodes: list(Name or Call) + """ + self.nodes = nodes + + def scope(self): + """The first parent node defining a new scope. + + :returns: The first parent scope node. + :rtype: Module or FunctionDef or ClassDef or Lambda or GenExpr + """ + # skip the function node to go directly to the upper level scope + return self.parent.parent.scope() + + def get_children(self): + yield from self.nodes + + +class DelAttr(mixins.ParentAssignTypeMixin, NodeNG): + """Variation of :class:`ast.Delete` representing deletion of an attribute. + + >>> node = astroid.extract_node('del self.attr') + >>> node + + >>> list(node.get_children())[0] + + """ + + _astroid_fields = ("expr",) + _other_fields = ("attrname",) + expr = None + """The name that this node represents. + + :type: Name or None + """ + + def __init__(self, attrname=None, lineno=None, col_offset=None, parent=None): + """ + :param attrname: The name of the attribute that is being deleted. + :type attrname: str or None + + :param lineno: The line that this node appears on in the source code. + :type lineno: int or None + + :param col_offset: The column that this node appears on in the + source code. + :type col_offset: int or None + + :param parent: The parent node in the syntax tree. + :type parent: NodeNG or None + """ + self.attrname = attrname + """The name of the attribute that is being deleted. + + :type: str or None + """ + + super().__init__(lineno, col_offset, parent) + + def postinit(self, expr=None): + """Do some setup after initialisation. + + :param expr: The name that this node represents. + :type expr: Name or None + """ + self.expr = expr + + def get_children(self): + yield self.expr + + +class Delete(mixins.AssignTypeMixin, Statement): + """Class representing an :class:`ast.Delete` node. + + A :class:`Delete` is a ``del`` statement this is deleting something. + + >>> node = astroid.extract_node('del self.attr') + >>> node + + """ + + _astroid_fields = ("targets",) + targets = None + """What is being deleted. + + :type: list(NodeNG) or None + """ + + def postinit(self, targets=None): + """Do some setup after initialisation. + + :param targets: What is being deleted. + :type targets: list(NodeNG) or None + """ + self.targets = targets + + def get_children(self): + yield from self.targets + + +class Dict(NodeNG, bases.Instance): + """Class representing an :class:`ast.Dict` node. + + A :class:`Dict` is a dictionary that is created with ``{}`` syntax. + + >>> node = astroid.extract_node('{1: "1"}') + >>> node + + """ + + _astroid_fields = ("items",) + + def __init__(self, lineno=None, col_offset=None, parent=None): + """ + :param lineno: The line that this node appears on in the source code. + :type lineno: int or None + + :param col_offset: The column that this node appears on in the + source code. + :type col_offset: int or None + + :param parent: The parent node in the syntax tree. + :type parent: NodeNG or None + """ + self.items = [] + """The key-value pairs contained in the dictionary. + + :type: list(tuple(NodeNG, NodeNG)) + """ + + super().__init__(lineno, col_offset, parent) + + def postinit(self, items): + """Do some setup after initialisation. + + :param items: The key-value pairs contained in the dictionary. + :type items: list(tuple(NodeNG, NodeNG)) + """ + self.items = items + + @classmethod + def from_elements(cls, items=None): + """Create a :class:`Dict` of constants from a live dictionary. + + :param items: The items to store in the node. + :type items: dict + + :returns: The created dictionary node. + :rtype: Dict + """ + node = cls() + if items is None: + node.items = [] + else: + node.items = [ + (const_factory(k), const_factory(v) if _is_const(v) else v) + for k, v in items.items() + # The keys need to be constants + if _is_const(k) + ] + return node + + def pytype(self): + """Get the name of the type that this node represents. + + :returns: The name of the type. + :rtype: str + """ + return "%s.dict" % BUILTINS + + def get_children(self): + """Get the key and value nodes below this node. + + Children are returned in the order that they are defined in the source + code, key first then the value. + + :returns: The children. + :rtype: iterable(NodeNG) + """ + for key, value in self.items: + yield key + yield value + + def last_child(self): + """An optimized version of list(get_children())[-1] + + :returns: The last child, or None if no children exist. + :rtype: NodeNG or None + """ + if self.items: + return self.items[-1][1] + return None + + def itered(self): + """An iterator over the keys this node contains. + + :returns: The keys of this node. + :rtype: iterable(NodeNG) + """ + return [key for (key, _) in self.items] + + def getitem(self, index, context=None): + """Get an item from this node. + + :param index: The node to use as a subscript index. + :type index: Const or Slice + + :raises AstroidTypeError: When the given index cannot be used as a + subscript index, or if this node is not subscriptable. + :raises AstroidIndexError: If the given index does not exist in the + dictionary. + """ + for key, value in self.items: + # TODO(cpopa): no support for overriding yet, {1:2, **{1: 3}}. + if isinstance(key, DictUnpack): + try: + return value.getitem(index, context) + except (exceptions.AstroidTypeError, exceptions.AstroidIndexError): + continue + for inferredkey in key.infer(context): + if inferredkey is util.Uninferable: + continue + if isinstance(inferredkey, Const) and isinstance(index, Const): + if inferredkey.value == index.value: + return value + + raise exceptions.AstroidIndexError(index) + + def bool_value(self, context=None): + """Determine the boolean value of this node. + + :returns: The boolean value of this node. + :rtype: bool + """ + return bool(self.items) + + +class Expr(Statement): + """Class representing an :class:`ast.Expr` node. + + An :class:`Expr` is any expression that does not have its value used or + stored. + + >>> node = astroid.extract_node('method()') + >>> node + + >>> node.parent + + """ + + _astroid_fields = ("value",) + value = None + """What the expression does. + + :type: NodeNG or None + """ + + def postinit(self, value=None): + """Do some setup after initialisation. + + :param value: What the expression does. + :type value: NodeNG or None + """ + self.value = value + + def get_children(self): + yield self.value + + def _get_yield_nodes_skip_lambdas(self): + if not self.value.is_lambda: + yield from self.value._get_yield_nodes_skip_lambdas() + + +class Ellipsis(mixins.NoChildrenMixin, NodeNG): # pylint: disable=redefined-builtin + """Class representing an :class:`ast.Ellipsis` node. + + An :class:`Ellipsis` is the ``...`` syntax. + + >>> node = astroid.extract_node('...') + >>> node + + """ + + def bool_value(self, context=None): + """Determine the boolean value of this node. + + :returns: The boolean value of this node. + For an :class:`Ellipsis` this is always ``True``. + :rtype: bool + """ + return True + + +class EmptyNode(mixins.NoChildrenMixin, NodeNG): + """Holds an arbitrary object in the :attr:`LocalsDictNodeNG.locals`.""" + + object = None + + +class ExceptHandler(mixins.MultiLineBlockMixin, mixins.AssignTypeMixin, Statement): + """Class representing an :class:`ast.ExceptHandler`. node. + + An :class:`ExceptHandler` is an ``except`` block on a try-except. + + >>> node = astroid.extract_node(''' + try: + do_something() + except Exception as error: + print("Error!") + ''') + >>> node + + >>> >>> node.handlers + [] + """ + + _astroid_fields = ("type", "name", "body") + _multi_line_block_fields = ("body",) + type = None + """The types that the block handles. + + :type: Tuple or NodeNG or None + """ + name = None + """The name that the caught exception is assigned to. + + :type: AssignName or None + """ + body = None + """The contents of the block. + + :type: list(NodeNG) or None + """ + + def get_children(self): + if self.type is not None: + yield self.type + + if self.name is not None: + yield self.name + + yield from self.body + + # pylint: disable=redefined-builtin; had to use the same name as builtin ast module. + def postinit(self, type=None, name=None, body=None): + """Do some setup after initialisation. + + :param type: The types that the block handles. + :type type: Tuple or NodeNG or None + + :param name: The name that the caught exception is assigned to. + :type name: AssignName or None + + :param body:The contents of the block. + :type body: list(NodeNG) or None + """ + self.type = type + self.name = name + self.body = body + + @decorators.cachedproperty + def blockstart_tolineno(self): + """The line on which the beginning of this block ends. + + :type: int + """ + if self.name: + return self.name.tolineno + if self.type: + return self.type.tolineno + return self.lineno + + def catch(self, exceptions): # pylint: disable=redefined-outer-name + """Check if this node handles any of the given exceptions. + + If ``exceptions`` is empty, this will default to ``True``. + + :param exceptions: The name of the exceptions to check for. + :type exceptions: list(str) + """ + if self.type is None or exceptions is None: + return True + for node in self.type._get_name_nodes(): + if node.name in exceptions: + return True + return False + + +class Exec(Statement): + """Class representing the ``exec`` statement. + + >>> node = astroid.extract_node('exec "True"') + >>> node + + """ + + _astroid_fields = ("expr", "globals", "locals") + expr = None + """The expression to be executed. + + :type: NodeNG or None + """ + globals = None + """The globals dictionary to execute with. + + :type: NodeNG or None + """ + locals = None + """The locals dictionary to execute with. + + :type: NodeNG or None + """ + + # pylint: disable=redefined-builtin; had to use the same name as builtin ast module. + def postinit(self, expr=None, globals=None, locals=None): + """Do some setup after initialisation. + + :param expr: The expression to be executed. + :type expr: NodeNG or None + + :param globals:The globals dictionary to execute with. + :type globals: NodeNG or None + + :param locals: The locals dictionary to execute with. + :type locals: NodeNG or None + """ + self.expr = expr + self.globals = globals + self.locals = locals + + +class ExtSlice(NodeNG): + """Class representing an :class:`ast.ExtSlice` node. + + An :class:`ExtSlice` is a complex slice expression. + + >>> node = astroid.extract_node('l[1:3, 5]') + >>> node + + >>> node.slice + + """ + + _astroid_fields = ("dims",) + dims = None + """The simple dimensions that form the complete slice. + + :type: list(NodeNG) or None + """ + + def postinit(self, dims=None): + """Do some setup after initialisation. + + :param dims: The simple dimensions that form the complete slice. + :type dims: list(NodeNG) or None + """ + self.dims = dims + + +class For( + mixins.MultiLineBlockMixin, + mixins.BlockRangeMixIn, + mixins.AssignTypeMixin, + Statement, +): + """Class representing an :class:`ast.For` node. + + >>> node = astroid.extract_node('for thing in things: print(thing)') + >>> node + + """ + + _astroid_fields = ("target", "iter", "body", "orelse") + _other_other_fields = ("type_annotation",) + _multi_line_block_fields = ("body", "orelse") + target = None + """What the loop assigns to. + + :type: NodeNG or None + """ + iter = None + """What the loop iterates over. + + :type: NodeNG or None + """ + body = None + """The contents of the body of the loop. + + :type: list(NodeNG) or None + """ + orelse = None + """The contents of the ``else`` block of the loop. + + :type: list(NodeNG) or None + """ + type_annotation = None + """If present, this will contain the type annotation passed by a type comment + + :type: NodeNG or None + """ + + # pylint: disable=redefined-builtin; had to use the same name as builtin ast module. + def postinit( + self, target=None, iter=None, body=None, orelse=None, type_annotation=None + ): + """Do some setup after initialisation. + + :param target: What the loop assigns to. + :type target: NodeNG or None + + :param iter: What the loop iterates over. + :type iter: NodeNG or None + + :param body: The contents of the body of the loop. + :type body: list(NodeNG) or None + + :param orelse: The contents of the ``else`` block of the loop. + :type orelse: list(NodeNG) or None + """ + self.target = target + self.iter = iter + self.body = body + self.orelse = orelse + self.type_annotation = type_annotation + + optional_assign = True + """Whether this node optionally assigns a variable. + + This is always ``True`` for :class:`For` nodes. + + :type: bool + """ + + @decorators.cachedproperty + def blockstart_tolineno(self): + """The line on which the beginning of this block ends. + + :type: int + """ + return self.iter.tolineno + + def get_children(self): + yield self.target + yield self.iter + + yield from self.body + yield from self.orelse + + +class AsyncFor(For): + """Class representing an :class:`ast.AsyncFor` node. + + An :class:`AsyncFor` is an asynchronous :class:`For` built with + the ``async`` keyword. + + >>> node = astroid.extract_node(''' + async def func(things): + async for thing in things: + print(thing) + ''') + >>> node + + >>> node.body[0] + + """ + + +class Await(NodeNG): + """Class representing an :class:`ast.Await` node. + + An :class:`Await` is the ``await`` keyword. + + >>> node = astroid.extract_node(''' + async def func(things): + await other_func() + ''') + >>> node + + >>> node.body[0] + + >>> list(node.body[0].get_children())[0] + + """ + + _astroid_fields = ("value",) + value = None + """What to wait for. + + :type: NodeNG or None + """ + + def postinit(self, value=None): + """Do some setup after initialisation. + + :param value: What to wait for. + :type value: NodeNG or None + """ + self.value = value + + def get_children(self): + yield self.value + + +class ImportFrom(mixins.NoChildrenMixin, mixins.ImportFromMixin, Statement): + """Class representing an :class:`ast.ImportFrom` node. + + >>> node = astroid.extract_node('from my_package import my_module') + >>> node + + """ + + _other_fields = ("modname", "names", "level") + + def __init__( + self, fromname, names, level=0, lineno=None, col_offset=None, parent=None + ): + """ + :param fromname: The module that is being imported from. + :type fromname: str or None + + :param names: What is being imported from the module. + :type names: list(tuple(str, str or None)) + + :param level: The level of relative import. + :type level: int + + :param lineno: The line that this node appears on in the source code. + :type lineno: int or None + + :param col_offset: The column that this node appears on in the + source code. + :type col_offset: int or None + + :param parent: The parent node in the syntax tree. + :type parent: NodeNG or None + """ + self.modname = fromname + """The module that is being imported from. + + This is ``None`` for relative imports. + + :type: str or None + """ + + self.names = names + """What is being imported from the module. + + Each entry is a :class:`tuple` of the name being imported, + and the alias that the name is assigned to (if any). + + :type: list(tuple(str, str or None)) + """ + + self.level = level + """The level of relative import. + + Essentially this is the number of dots in the import. + This is always 0 for absolute imports. + + :type: int + """ + + super().__init__(lineno, col_offset, parent) + + +class Attribute(NodeNG): + """Class representing an :class:`ast.Attribute` node.""" + + _astroid_fields = ("expr",) + _other_fields = ("attrname",) + expr = None + """The name that this node represents. + + :type: Name or None + """ + + def __init__(self, attrname=None, lineno=None, col_offset=None, parent=None): + """ + :param attrname: The name of the attribute. + :type attrname: str or None + + :param lineno: The line that this node appears on in the source code. + :type lineno: int or None + + :param col_offset: The column that this node appears on in the + source code. + :type col_offset: int or None + + :param parent: The parent node in the syntax tree. + :type parent: NodeNG or None + """ + self.attrname = attrname + """The name of the attribute. + + :type: str or None + """ + + super().__init__(lineno, col_offset, parent) + + def postinit(self, expr=None): + """Do some setup after initialisation. + + :param expr: The name that this node represents. + :type expr: Name or None + """ + self.expr = expr + + def get_children(self): + yield self.expr + + +class Global(mixins.NoChildrenMixin, Statement): + """Class representing an :class:`ast.Global` node. + + >>> node = astroid.extract_node('global a_global') + >>> node + + """ + + _other_fields = ("names",) + + def __init__(self, names, lineno=None, col_offset=None, parent=None): + """ + :param names: The names being declared as global. + :type names: list(str) + + :param lineno: The line that this node appears on in the source code. + :type lineno: int or None + + :param col_offset: The column that this node appears on in the + source code. + :type col_offset: int or None + + :param parent: The parent node in the syntax tree. + :type parent: NodeNG or None + """ + self.names = names + """The names being declared as global. + + :type: list(str) + """ + + super().__init__(lineno, col_offset, parent) + + def _infer_name(self, frame, name): + return name + + +class If(mixins.MultiLineBlockMixin, mixins.BlockRangeMixIn, Statement): + """Class representing an :class:`ast.If` node. + + >>> node = astroid.extract_node('if condition: print(True)') + >>> node + + """ + + _astroid_fields = ("test", "body", "orelse") + _multi_line_block_fields = ("body", "orelse") + test = None + """The condition that the statement tests. + + :type: NodeNG or None + """ + body = None + """The contents of the block. + + :type: list(NodeNG) or None + """ + orelse = None + """The contents of the ``else`` block. + + :type: list(NodeNG) or None + """ + + def postinit(self, test=None, body=None, orelse=None): + """Do some setup after initialisation. + + :param test: The condition that the statement tests. + :type test: NodeNG or None + + :param body: The contents of the block. + :type body: list(NodeNG) or None + + :param orelse: The contents of the ``else`` block. + :type orelse: list(NodeNG) or None + """ + self.test = test + self.body = body + self.orelse = orelse + + @decorators.cachedproperty + def blockstart_tolineno(self): + """The line on which the beginning of this block ends. + + :type: int + """ + return self.test.tolineno + + def block_range(self, lineno): + """Get a range from the given line number to where this node ends. + + :param lineno: The line number to start the range at. + :type lineno: int + + :returns: The range of line numbers that this node belongs to, + starting at the given line number. + :rtype: tuple(int, int) + """ + if lineno == self.body[0].fromlineno: + return lineno, lineno + if lineno <= self.body[-1].tolineno: + return lineno, self.body[-1].tolineno + return self._elsed_block_range(lineno, self.orelse, self.body[0].fromlineno - 1) + + def get_children(self): + yield self.test + + yield from self.body + yield from self.orelse + + def has_elif_block(self): + return len(self.orelse) == 1 and isinstance(self.orelse[0], If) + + +class IfExp(NodeNG): + """Class representing an :class:`ast.IfExp` node. + + >>> node = astroid.extract_node('value if condition else other') + >>> node + + """ + + _astroid_fields = ("test", "body", "orelse") + test = None + """The condition that the statement tests. + + :type: NodeNG or None + """ + body = None + """The contents of the block. + + :type: list(NodeNG) or None + """ + orelse = None + """The contents of the ``else`` block. + + :type: list(NodeNG) or None + """ + + def postinit(self, test=None, body=None, orelse=None): + """Do some setup after initialisation. + + :param test: The condition that the statement tests. + :type test: NodeNG or None + + :param body: The contents of the block. + :type body: list(NodeNG) or None + + :param orelse: The contents of the ``else`` block. + :type orelse: list(NodeNG) or None + """ + self.test = test + self.body = body + self.orelse = orelse + + def get_children(self): + yield self.test + yield self.body + yield self.orelse + + def op_left_associative(self): + # `1 if True else 2 if False else 3` is parsed as + # `1 if True else (2 if False else 3)` + return False + + +class Import(mixins.NoChildrenMixin, mixins.ImportFromMixin, Statement): + """Class representing an :class:`ast.Import` node. + + >>> node = astroid.extract_node('import astroid') + >>> node + + """ + + _other_fields = ("names",) + + def __init__(self, names=None, lineno=None, col_offset=None, parent=None): + """ + :param names: The names being imported. + :type names: list(tuple(str, str or None)) or None + + :param lineno: The line that this node appears on in the source code. + :type lineno: int or None + + :param col_offset: The column that this node appears on in the + source code. + :type col_offset: int or None + + :param parent: The parent node in the syntax tree. + :type parent: NodeNG or None + """ + self.names = names + """The names being imported. + + Each entry is a :class:`tuple` of the name being imported, + and the alias that the name is assigned to (if any). + + :type: list(tuple(str, str or None)) or None + """ + + super().__init__(lineno, col_offset, parent) + + +class Index(NodeNG): + """Class representing an :class:`ast.Index` node. + + An :class:`Index` is a simple subscript. + + >>> node = astroid.extract_node('things[1]') + >>> node + + >>> node.slice + + """ + + _astroid_fields = ("value",) + value = None + """The value to subscript with. + + :type: NodeNG or None + """ + + def postinit(self, value=None): + """Do some setup after initialisation. + + :param value: The value to subscript with. + :type value: NodeNG or None + """ + self.value = value + + def get_children(self): + yield self.value + + +class Keyword(NodeNG): + """Class representing an :class:`ast.keyword` node. + + >>> node = astroid.extract_node('function(a_kwarg=True)') + >>> node + + >>> node.keywords + [] + """ + + _astroid_fields = ("value",) + _other_fields = ("arg",) + value = None + """The value being assigned to the keyword argument. + + :type: NodeNG or None + """ + + def __init__(self, arg=None, lineno=None, col_offset=None, parent=None): + """ + :param arg: The argument being assigned to. + :type arg: Name or None + + :param lineno: The line that this node appears on in the source code. + :type lineno: int or None + + :param col_offset: The column that this node appears on in the + source code. + :type col_offset: int or None + + :param parent: The parent node in the syntax tree. + :type parent: NodeNG or None + """ + self.arg = arg + """The argument being assigned to. + + :type: Name or None + """ + + super().__init__(lineno, col_offset, parent) + + def postinit(self, value=None): + """Do some setup after initialisation. + + :param value: The value being assigned to the ketword argument. + :type value: NodeNG or None + """ + self.value = value + + def get_children(self): + yield self.value + + +class List(_BaseContainer): + """Class representing an :class:`ast.List` node. + + >>> node = astroid.extract_node('[1, 2, 3]') + >>> node + + """ + + _other_fields = ("ctx",) + + def __init__(self, ctx=None, lineno=None, col_offset=None, parent=None): + """ + :param ctx: Whether the list is assigned to or loaded from. + :type ctx: Context or None + + :param lineno: The line that this node appears on in the source code. + :type lineno: int or None + + :param col_offset: The column that this node appears on in the + source code. + :type col_offset: int or None + + :param parent: The parent node in the syntax tree. + :type parent: NodeNG or None + """ + self.ctx = ctx + """Whether the list is assigned to or loaded from. + + :type: Context or None + """ + + super().__init__(lineno, col_offset, parent) + + def pytype(self): + """Get the name of the type that this node represents. + + :returns: The name of the type. + :rtype: str + """ + return "%s.list" % BUILTINS + + def getitem(self, index, context=None): + """Get an item from this node. + + :param index: The node to use as a subscript index. + :type index: Const or Slice + """ + return _container_getitem(self, self.elts, index, context=context) + + +class Nonlocal(mixins.NoChildrenMixin, Statement): + """Class representing an :class:`ast.Nonlocal` node. + + >>> node = astroid.extract_node(''' + def function(): + nonlocal var + ''') + >>> node + + >>> node.body[0] + + """ + + _other_fields = ("names",) + + def __init__(self, names, lineno=None, col_offset=None, parent=None): + """ + :param names: The names being declared as not local. + :type names: list(str) + + :param lineno: The line that this node appears on in the source code. + :type lineno: int or None + + :param col_offset: The column that this node appears on in the + source code. + :type col_offset: int or None + + :param parent: The parent node in the syntax tree. + :type parent: NodeNG or None + """ + self.names = names + """The names being declared as not local. + + :type: list(str) + """ + + super().__init__(lineno, col_offset, parent) + + def _infer_name(self, frame, name): + return name + + +class Pass(mixins.NoChildrenMixin, Statement): + """Class representing an :class:`ast.Pass` node. + + >>> node = astroid.extract_node('pass') + >>> node + + """ + + +class Print(Statement): + """Class representing an :class:`ast.Print` node. + + >>> node = astroid.extract_node('print "A message"') + >>> node + + """ + + _astroid_fields = ("dest", "values") + dest = None + """Where to print to. + + :type: NodeNG or None + """ + values = None + """What to print. + + :type: list(NodeNG) or None + """ + + def __init__(self, nl=None, lineno=None, col_offset=None, parent=None): + """ + :param nl: Whether to print a new line. + :type nl: bool or None + + :param lineno: The line that this node appears on in the source code. + :type lineno: int or None + + :param col_offset: The column that this node appears on in the + source code. + :type col_offset: int or None + + :param parent: The parent node in the syntax tree. + :type parent: NodeNG or None + """ + self.nl = nl + """Whether to print a new line. + + :type: bool or None + """ + + super().__init__(lineno, col_offset, parent) + + def postinit(self, dest=None, values=None): + """Do some setup after initialisation. + + :param dest: Where to print to. + :type dest: NodeNG or None + + :param values: What to print. + :type values: list(NodeNG) or None + """ + self.dest = dest + self.values = values + + +class Raise(Statement): + """Class representing an :class:`ast.Raise` node. + + >>> node = astroid.extract_node('raise RuntimeError("Something bad happened!")') + >>> node + + """ + + exc = None + """What is being raised. + + :type: NodeNG or None + """ + _astroid_fields = ("exc", "cause") + cause = None + """The exception being used to raise this one. + + :type: NodeNG or None + """ + + def postinit(self, exc=None, cause=None): + """Do some setup after initialisation. + + :param exc: What is being raised. + :type exc: NodeNG or None + + :param cause: The exception being used to raise this one. + :type cause: NodeNG or None + """ + self.exc = exc + self.cause = cause + + def raises_not_implemented(self): + """Check if this node raises a :class:`NotImplementedError`. + + :returns: True if this node raises a :class:`NotImplementedError`, + False otherwise. + :rtype: bool + """ + if not self.exc: + return False + for name in self.exc._get_name_nodes(): + if name.name == "NotImplementedError": + return True + return False + + def get_children(self): + if self.exc is not None: + yield self.exc + + if self.cause is not None: + yield self.cause + + +class Return(Statement): + """Class representing an :class:`ast.Return` node. + + >>> node = astroid.extract_node('return True') + >>> node + + """ + + _astroid_fields = ("value",) + value = None + """The value being returned. + + :type: NodeNG or None + """ + + def postinit(self, value=None): + """Do some setup after initialisation. + + :param value: The value being returned. + :type value: NodeNG or None + """ + self.value = value + + def get_children(self): + if self.value is not None: + yield self.value + + def is_tuple_return(self): + return isinstance(self.value, Tuple) + + def _get_return_nodes_skip_functions(self): + yield self + + +class Set(_BaseContainer): + """Class representing an :class:`ast.Set` node. + + >>> node = astroid.extract_node('{1, 2, 3}') + >>> node + + """ + + def pytype(self): + """Get the name of the type that this node represents. + + :returns: The name of the type. + :rtype: str + """ + return "%s.set" % BUILTINS + + +class Slice(NodeNG): + """Class representing an :class:`ast.Slice` node. + + >>> node = astroid.extract_node('things[1:3]') + >>> node + + >>> node.slice + + """ + + _astroid_fields = ("lower", "upper", "step") + lower = None + """The lower index in the slice. + + :type: NodeNG or None + """ + upper = None + """The upper index in the slice. + + :type: NodeNG or None + """ + step = None + """The step to take between indexes. + + :type: NodeNG or None + """ + + def postinit(self, lower=None, upper=None, step=None): + """Do some setup after initialisation. + + :param lower: The lower index in the slice. + :value lower: NodeNG or None + + :param upper: The upper index in the slice. + :value upper: NodeNG or None + + :param step: The step to take between index. + :param step: NodeNG or None + """ + self.lower = lower + self.upper = upper + self.step = step + + def _wrap_attribute(self, attr): + """Wrap the empty attributes of the Slice in a Const node.""" + if not attr: + const = const_factory(attr) + const.parent = self + return const + return attr + + @decorators.cachedproperty + def _proxied(self): + builtins = MANAGER.builtins_module + return builtins.getattr("slice")[0] + + def pytype(self): + """Get the name of the type that this node represents. + + :returns: The name of the type. + :rtype: str + """ + return "%s.slice" % BUILTINS + + def igetattr(self, attrname, context=None): + """Infer the possible values of the given attribute on the slice. + + :param attrname: The name of the attribute to infer. + :type attrname: str + + :returns: The inferred possible values. + :rtype: iterable(NodeNG) + """ + if attrname == "start": + yield self._wrap_attribute(self.lower) + elif attrname == "stop": + yield self._wrap_attribute(self.upper) + elif attrname == "step": + yield self._wrap_attribute(self.step) + else: + yield from self.getattr(attrname, context=context) + + def getattr(self, attrname, context=None): + return self._proxied.getattr(attrname, context) + + def get_children(self): + if self.lower is not None: + yield self.lower + + if self.upper is not None: + yield self.upper + + if self.step is not None: + yield self.step + + +class Starred(mixins.ParentAssignTypeMixin, NodeNG): + """Class representing an :class:`ast.Starred` node. + + >>> node = astroid.extract_node('*args') + >>> node + + """ + + _astroid_fields = ("value",) + _other_fields = ("ctx",) + value = None + """What is being unpacked. + + :type: NodeNG or None + """ + + def __init__(self, ctx=None, lineno=None, col_offset=None, parent=None): + """ + :param ctx: Whether the list is assigned to or loaded from. + :type ctx: Context or None + + :param lineno: The line that this node appears on in the source code. + :type lineno: int or None + + :param col_offset: The column that this node appears on in the + source code. + :type col_offset: int or None + + :param parent: The parent node in the syntax tree. + :type parent: NodeNG or None + """ + self.ctx = ctx + """Whether the starred item is assigned to or loaded from. + + :type: Context or None + """ + + super().__init__(lineno=lineno, col_offset=col_offset, parent=parent) + + def postinit(self, value=None): + """Do some setup after initialisation. + + :param value: What is being unpacked. + :type value: NodeNG or None + """ + self.value = value + + def get_children(self): + yield self.value + + +class Subscript(NodeNG): + """Class representing an :class:`ast.Subscript` node. + + >>> node = astroid.extract_node('things[1:3]') + >>> node + + """ + + _astroid_fields = ("value", "slice") + _other_fields = ("ctx",) + value = None + """What is being indexed. + + :type: NodeNG or None + """ + slice = None + """The slice being used to lookup. + + :type: NodeNG or None + """ + + def __init__(self, ctx=None, lineno=None, col_offset=None, parent=None): + """ + :param ctx: Whether the subscripted item is assigned to or loaded from. + :type ctx: Context or None + + :param lineno: The line that this node appears on in the source code. + :type lineno: int or None + + :param col_offset: The column that this node appears on in the + source code. + :type col_offset: int or None + + :param parent: The parent node in the syntax tree. + :type parent: NodeNG or None + """ + self.ctx = ctx + """Whether the subscripted item is assigned to or loaded from. + + :type: Context or None + """ + + super().__init__(lineno=lineno, col_offset=col_offset, parent=parent) + + # pylint: disable=redefined-builtin; had to use the same name as builtin ast module. + def postinit(self, value=None, slice=None): + """Do some setup after initialisation. + + :param value: What is being indexed. + :type value: NodeNG or None + + :param slice: The slice being used to lookup. + :type slice: NodeNG or None + """ + self.value = value + self.slice = slice + + def get_children(self): + yield self.value + yield self.slice + + +class TryExcept(mixins.MultiLineBlockMixin, mixins.BlockRangeMixIn, Statement): + """Class representing an :class:`ast.TryExcept` node. + + >>> node = astroid.extract_node(''' + try: + do_something() + except Exception as error: + print("Error!") + ''') + >>> node + + """ + + _astroid_fields = ("body", "handlers", "orelse") + _multi_line_block_fields = ("body", "handlers", "orelse") + body = None + """The contents of the block to catch exceptions from. + + :type: list(NodeNG) or None + """ + handlers = None + """The exception handlers. + + :type: list(ExceptHandler) or None + """ + orelse = None + """The contents of the ``else`` block. + + :type: list(NodeNG) or None + """ + + def postinit(self, body=None, handlers=None, orelse=None): + """Do some setup after initialisation. + + :param body: The contents of the block to catch exceptions from. + :type body: list(NodeNG) or None + + :param handlers: The exception handlers. + :type handlers: list(ExceptHandler) or None + + :param orelse: The contents of the ``else`` block. + :type orelse: list(NodeNG) or None + """ + self.body = body + self.handlers = handlers + self.orelse = orelse + + def _infer_name(self, frame, name): + return name + + def block_range(self, lineno): + """Get a range from the given line number to where this node ends. + + :param lineno: The line number to start the range at. + :type lineno: int + + :returns: The range of line numbers that this node belongs to, + starting at the given line number. + :rtype: tuple(int, int) + """ + last = None + for exhandler in self.handlers: + if exhandler.type and lineno == exhandler.type.fromlineno: + return lineno, lineno + if exhandler.body[0].fromlineno <= lineno <= exhandler.body[-1].tolineno: + return lineno, exhandler.body[-1].tolineno + if last is None: + last = exhandler.body[0].fromlineno - 1 + return self._elsed_block_range(lineno, self.orelse, last) + + def get_children(self): + yield from self.body + + yield from self.handlers or () + yield from self.orelse or () + + +class TryFinally(mixins.MultiLineBlockMixin, mixins.BlockRangeMixIn, Statement): + """Class representing an :class:`ast.TryFinally` node. + + >>> node = astroid.extract_node(''' + try: + do_something() + except Exception as error: + print("Error!") + finally: + print("Cleanup!") + ''') + >>> node + + """ + + _astroid_fields = ("body", "finalbody") + _multi_line_block_fields = ("body", "finalbody") + body = None + """The try-except that the finally is attached to. + + :type: list(TryExcept) or None + """ + finalbody = None + """The contents of the ``finally`` block. + + :type: list(NodeNG) or None + """ + + def postinit(self, body=None, finalbody=None): + """Do some setup after initialisation. + + :param body: The try-except that the finally is attached to. + :type body: list(TryExcept) or None + + :param finalbody: The contents of the ``finally`` block. + :type finalbody: list(NodeNG) or None + """ + self.body = body + self.finalbody = finalbody + + def block_range(self, lineno): + """Get a range from the given line number to where this node ends. + + :param lineno: The line number to start the range at. + :type lineno: int + + :returns: The range of line numbers that this node belongs to, + starting at the given line number. + :rtype: tuple(int, int) + """ + child = self.body[0] + # py2.5 try: except: finally: + if ( + isinstance(child, TryExcept) + and child.fromlineno == self.fromlineno + and child.tolineno >= lineno > self.fromlineno + ): + return child.block_range(lineno) + return self._elsed_block_range(lineno, self.finalbody) + + def get_children(self): + yield from self.body + yield from self.finalbody + + +class Tuple(_BaseContainer): + """Class representing an :class:`ast.Tuple` node. + + >>> node = astroid.extract_node('(1, 2, 3)') + >>> node + + """ + + _other_fields = ("ctx",) + + def __init__(self, ctx=None, lineno=None, col_offset=None, parent=None): + """ + :param ctx: Whether the tuple is assigned to or loaded from. + :type ctx: Context or None + + :param lineno: The line that this node appears on in the source code. + :type lineno: int or None + + :param col_offset: The column that this node appears on in the + source code. + :type col_offset: int or None + + :param parent: The parent node in the syntax tree. + :type parent: NodeNG or None + """ + self.ctx = ctx + """Whether the tuple is assigned to or loaded from. + + :type: Context or None + """ + + super().__init__(lineno, col_offset, parent) + + def pytype(self): + """Get the name of the type that this node represents. + + :returns: The name of the type. + :rtype: str + """ + return "%s.tuple" % BUILTINS + + def getitem(self, index, context=None): + """Get an item from this node. + + :param index: The node to use as a subscript index. + :type index: Const or Slice + """ + return _container_getitem(self, self.elts, index, context=context) + + +class UnaryOp(NodeNG): + """Class representing an :class:`ast.UnaryOp` node. + + >>> node = astroid.extract_node('-5') + >>> node + + """ + + _astroid_fields = ("operand",) + _other_fields = ("op",) + operand = None + """What the unary operator is applied to. + + :type: NodeNG or None + """ + + def __init__(self, op=None, lineno=None, col_offset=None, parent=None): + """ + :param op: The operator. + :type: str or None + + :param lineno: The line that this node appears on in the source code. + :type lineno: int or None + + :param col_offset: The column that this node appears on in the + source code. + :type col_offset: int or None + + :param parent: The parent node in the syntax tree. + :type parent: NodeNG or None + """ + self.op = op + """The operator. + + :type: str or None + """ + + super().__init__(lineno, col_offset, parent) + + def postinit(self, operand=None): + """Do some setup after initialisation. + + :param operand: What the unary operator is applied to. + :type operand: NodeNG or None + """ + self.operand = operand + + # This is set by inference.py + def _infer_unaryop(self, context=None): + raise NotImplementedError + + def type_errors(self, context=None): + """Get a list of type errors which can occur during inference. + + Each TypeError is represented by a :class:`BadBinaryOperationMessage`, + which holds the original exception. + + :returns: The list of possible type errors. + :rtype: list(BadBinaryOperationMessage) + """ + try: + results = self._infer_unaryop(context=context) + return [ + result + for result in results + if isinstance(result, util.BadUnaryOperationMessage) + ] + except exceptions.InferenceError: + return [] + + def get_children(self): + yield self.operand + + def op_precedence(self): + if self.op == "not": + return OP_PRECEDENCE[self.op] + + return super().op_precedence() + + +class While(mixins.MultiLineBlockMixin, mixins.BlockRangeMixIn, Statement): + """Class representing an :class:`ast.While` node. + + >>> node = astroid.extract_node(''' + while condition(): + print("True") + ''') + >>> node + + """ + + _astroid_fields = ("test", "body", "orelse") + _multi_line_block_fields = ("body", "orelse") + test = None + """The condition that the loop tests. + + :type: NodeNG or None + """ + body = None + """The contents of the loop. + + :type: list(NodeNG) or None + """ + orelse = None + """The contents of the ``else`` block. + + :type: list(NodeNG) or None + """ + + def postinit(self, test=None, body=None, orelse=None): + """Do some setup after initialisation. + + :param test: The condition that the loop tests. + :type test: NodeNG or None + + :param body: The contents of the loop. + :type body: list(NodeNG) or None + + :param orelse: The contents of the ``else`` block. + :type orelse: list(NodeNG) or None + """ + self.test = test + self.body = body + self.orelse = orelse + + @decorators.cachedproperty + def blockstart_tolineno(self): + """The line on which the beginning of this block ends. + + :type: int + """ + return self.test.tolineno + + def block_range(self, lineno): + """Get a range from the given line number to where this node ends. + + :param lineno: The line number to start the range at. + :type lineno: int + + :returns: The range of line numbers that this node belongs to, + starting at the given line number. + :rtype: tuple(int, int) + """ + return self._elsed_block_range(lineno, self.orelse) + + def get_children(self): + yield self.test + + yield from self.body + yield from self.orelse + + def _get_yield_nodes_skip_lambdas(self): + """A While node can contain a Yield node in the test""" + yield from self.test._get_yield_nodes_skip_lambdas() + yield from super()._get_yield_nodes_skip_lambdas() + + +class With( + mixins.MultiLineBlockMixin, + mixins.BlockRangeMixIn, + mixins.AssignTypeMixin, + Statement, +): + """Class representing an :class:`ast.With` node. + + >>> node = astroid.extract_node(''' + with open(file_path) as file_: + print(file_.read()) + ''') + >>> node + + """ + + _astroid_fields = ("items", "body") + _other_other_fields = ("type_annotation",) + _multi_line_block_fields = ("body",) + items = None + """The pairs of context managers and the names they are assigned to. + + :type: list(tuple(NodeNG, AssignName or None)) or None + """ + body = None + """The contents of the ``with`` block. + + :type: list(NodeNG) or None + """ + type_annotation = None + """If present, this will contain the type annotation passed by a type comment + + :type: NodeNG or None + """ + + def postinit(self, items=None, body=None, type_annotation=None): + """Do some setup after initialisation. + + :param items: The pairs of context managers and the names + they are assigned to. + :type items: list(tuple(NodeNG, AssignName or None)) or None + + :param body: The contents of the ``with`` block. + :type body: list(NodeNG) or None + """ + self.items = items + self.body = body + self.type_annotation = type_annotation + + @decorators.cachedproperty + def blockstart_tolineno(self): + """The line on which the beginning of this block ends. + + :type: int + """ + return self.items[-1][0].tolineno + + def get_children(self): + """Get the child nodes below this node. + + :returns: The children. + :rtype: iterable(NodeNG) + """ + for expr, var in self.items: + yield expr + if var: + yield var + yield from self.body + + +class AsyncWith(With): + """Asynchronous ``with`` built with the ``async`` keyword.""" + + +class Yield(NodeNG): + """Class representing an :class:`ast.Yield` node. + + >>> node = astroid.extract_node('yield True') + >>> node + + """ + + _astroid_fields = ("value",) + value = None + """The value to yield. + + :type: NodeNG or None + """ + + def postinit(self, value=None): + """Do some setup after initialisation. + + :param value: The value to yield. + :type value: NodeNG or None + """ + self.value = value + + def get_children(self): + if self.value is not None: + yield self.value + + def _get_yield_nodes_skip_lambdas(self): + yield self + + +class YieldFrom(Yield): + """Class representing an :class:`ast.YieldFrom` node.""" + + +class DictUnpack(mixins.NoChildrenMixin, NodeNG): + """Represents the unpacking of dicts into dicts using :pep:`448`.""" + + +class FormattedValue(NodeNG): + """Class representing an :class:`ast.FormattedValue` node. + + Represents a :pep:`498` format string. + + >>> node = astroid.extract_node('f"Format {type_}"') + >>> node + + >>> node.values + [, ] + """ + + _astroid_fields = ("value", "format_spec") + value = None + """The value to be formatted into the string. + + :type: NodeNG or None + """ + conversion = None + """The type of formatting to be applied to the value. + + .. seealso:: + :class:`ast.FormattedValue` + + :type: int or None + """ + format_spec = None + """The formatting to be applied to the value. + + .. seealso:: + :class:`ast.FormattedValue` + + :type: JoinedStr or None + """ + + def postinit(self, value, conversion=None, format_spec=None): + """Do some setup after initialisation. + + :param value: The value to be formatted into the string. + :type value: NodeNG + + :param conversion: The type of formatting to be applied to the value. + :type conversion: int or None + + :param format_spec: The formatting to be applied to the value. + :type format_spec: JoinedStr or None + """ + self.value = value + self.conversion = conversion + self.format_spec = format_spec + + def get_children(self): + yield self.value + + if self.format_spec is not None: + yield self.format_spec + + +class JoinedStr(NodeNG): + """Represents a list of string expressions to be joined. + + >>> node = astroid.extract_node('f"Format {type_}"') + >>> node + + """ + + _astroid_fields = ("values",) + values = None + """The string expressions to be joined. + + :type: list(FormattedValue or Const) or None + """ + + def postinit(self, values=None): + """Do some setup after initialisation. + + :param value: The string expressions to be joined. + + :type: list(FormattedValue or Const) or None + """ + self.values = values + + def get_children(self): + yield from self.values + + +class NamedExpr(mixins.AssignTypeMixin, NodeNG): + """Represents the assignment from the assignment expression + + >>> module = astroid.parse('if a := 1: pass') + >>> module.body[0].test + + """ + + _astroid_fields = ("target", "value") + target = None + """The assignment target + + :type: Name + """ + value = None + """The value that gets assigned in the expression + + :type: NodeNG + """ + + def postinit(self, target, value): + self.target = target + self.value = value + + +class Unknown(mixins.AssignTypeMixin, NodeNG): + """This node represents a node in a constructed AST where + introspection is not possible. At the moment, it's only used in + the args attribute of FunctionDef nodes where function signature + introspection failed. + """ + + name = "Unknown" + + def qname(self): + return "Unknown" + + def infer(self, context=None, **kwargs): + """Inference on an Unknown node immediately terminates.""" + yield util.Uninferable + + +class EvaluatedObject(NodeNG): + """Contains an object that has already been inferred + + This class is useful to pre-evaluate a particular node, + with the resulting class acting as the non-evaluated node. + """ + + name = "EvaluatedObject" + _astroid_fields = ("original",) + _other_fields = ("value",) + + original = None + """The original node that has already been evaluated + + :type: NodeNG + """ + + value = None + """The inferred value + + :type: Union[Uninferable, NodeNG] + """ + + def __init__(self, original, value): + self.original = original + self.value = value + super().__init__( + lineno=self.original.lineno, + col_offset=self.original.col_offset, + parent=self.original.parent, + ) + + def infer(self, context=None, **kwargs): + yield self.value + + +# constants ############################################################## + +CONST_CLS = { + list: List, + tuple: Tuple, + dict: Dict, + set: Set, + type(None): Const, + type(NotImplemented): Const, +} +if PY38: + CONST_CLS[type(...)] = Const + + +def _update_const_classes(): + """update constant classes, so the keys of CONST_CLS can be reused""" + klasses = (bool, int, float, complex, str, bytes) + for kls in klasses: + CONST_CLS[kls] = Const + + +_update_const_classes() + + +def _two_step_initialization(cls, value): + instance = cls() + instance.postinit(value) + return instance + + +def _dict_initialization(cls, value): + if isinstance(value, dict): + value = tuple(value.items()) + return _two_step_initialization(cls, value) + + +_CONST_CLS_CONSTRUCTORS = { + List: _two_step_initialization, + Tuple: _two_step_initialization, + Dict: _dict_initialization, + Set: _two_step_initialization, + Const: lambda cls, value: cls(value), +} + + +def const_factory(value): + """return an astroid node for a python value""" + # XXX we should probably be stricter here and only consider stuff in + # CONST_CLS or do better treatment: in case where value is not in CONST_CLS, + # we should rather recall the builder on this value than returning an empty + # node (another option being that const_factory shouldn't be called with something + # not in CONST_CLS) + assert not isinstance(value, NodeNG) + + # Hack for ignoring elements of a sequence + # or a mapping, in order to avoid transforming + # each element to an AST. This is fixed in 2.0 + # and this approach is a temporary hack. + if isinstance(value, (list, set, tuple, dict)): + elts = [] + else: + elts = value + + try: + initializer_cls = CONST_CLS[value.__class__] + initializer = _CONST_CLS_CONSTRUCTORS[initializer_cls] + return initializer(initializer_cls, elts) + except (KeyError, AttributeError): + node = EmptyNode() + node.object = value + return node + + +def is_from_decorator(node): + """Return True if the given node is the child of a decorator""" + parent = node.parent + while parent is not None: + if isinstance(parent, Decorators): + return True + parent = parent.parent + return False diff --git a/env/lib/python3.8/site-packages/astroid/nodes.py b/env/lib/python3.8/site-packages/astroid/nodes.py new file mode 100644 index 000000000..4ce4ebe23 --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/nodes.py @@ -0,0 +1,176 @@ +# Copyright (c) 2006-2011, 2013 LOGILAB S.A. (Paris, FRANCE) +# Copyright (c) 2010 Daniel Harding +# Copyright (c) 2014-2020 Claudiu Popa +# Copyright (c) 2014 Google, Inc. +# Copyright (c) 2015-2016 Ceridwen +# Copyright (c) 2016 Jared Garst +# Copyright (c) 2017 Ashley Whetter +# Copyright (c) 2017 rr- +# Copyright (c) 2018 Bryce Guinta + +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER + +"""Every available node class. + +.. seealso:: + :doc:`ast documentation ` + +All nodes inherit from :class:`~astroid.node_classes.NodeNG`. +""" +# pylint: disable=unused-import,redefined-builtin + +from astroid.node_classes import ( + Arguments, + AssignAttr, + Assert, + Assign, + AnnAssign, + AssignName, + AugAssign, + Repr, + BinOp, + BoolOp, + Break, + Call, + Compare, + Comprehension, + Const, + Continue, + Decorators, + DelAttr, + DelName, + Delete, + Dict, + Expr, + Ellipsis, + EmptyNode, + ExceptHandler, + Exec, + ExtSlice, + For, + ImportFrom, + Attribute, + Global, + If, + IfExp, + Import, + Index, + Keyword, + List, + Name, + NamedExpr, + Nonlocal, + Pass, + Print, + Raise, + Return, + Set, + Slice, + Starred, + Subscript, + TryExcept, + TryFinally, + Tuple, + UnaryOp, + While, + With, + Yield, + YieldFrom, + const_factory, + AsyncFor, + Await, + AsyncWith, + FormattedValue, + JoinedStr, + # Node not present in the builtin ast module. + DictUnpack, + Unknown, + EvaluatedObject, +) +from astroid.scoped_nodes import ( + Module, + GeneratorExp, + Lambda, + DictComp, + ListComp, + SetComp, + FunctionDef, + ClassDef, + AsyncFunctionDef, +) + + +ALL_NODE_CLASSES = ( + AsyncFunctionDef, + AsyncFor, + AsyncWith, + Await, + Arguments, + AssignAttr, + Assert, + Assign, + AnnAssign, + AssignName, + AugAssign, + Repr, + BinOp, + BoolOp, + Break, + Call, + ClassDef, + Compare, + Comprehension, + Const, + Continue, + Decorators, + DelAttr, + DelName, + Delete, + Dict, + DictComp, + DictUnpack, + Expr, + Ellipsis, + EmptyNode, + ExceptHandler, + Exec, + ExtSlice, + For, + ImportFrom, + FunctionDef, + Attribute, + GeneratorExp, + Global, + If, + IfExp, + Import, + Index, + Keyword, + Lambda, + List, + ListComp, + Name, + NamedExpr, + Nonlocal, + Module, + Pass, + Print, + Raise, + Return, + Set, + SetComp, + Slice, + Starred, + Subscript, + TryExcept, + TryFinally, + Tuple, + UnaryOp, + While, + With, + Yield, + YieldFrom, + FormattedValue, + JoinedStr, +) diff --git a/env/lib/python3.8/site-packages/astroid/objects.py b/env/lib/python3.8/site-packages/astroid/objects.py new file mode 100644 index 000000000..fb782e6d7 --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/objects.py @@ -0,0 +1,314 @@ +# Copyright (c) 2015-2016, 2018-2020 Claudiu Popa +# Copyright (c) 2015-2016 Ceridwen +# Copyright (c) 2015 Florian Bruhin +# Copyright (c) 2016 Derek Gustafson +# Copyright (c) 2018 hippo91 +# Copyright (c) 2018 Bryce Guinta + +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER + + +""" +Inference objects are a way to represent composite AST nodes, +which are used only as inference results, so they can't be found in the +original AST tree. For instance, inferring the following frozenset use, +leads to an inferred FrozenSet: + + Call(func=Name('frozenset'), args=Tuple(...)) +""" + +import builtins + +from astroid import bases +from astroid import decorators +from astroid import exceptions +from astroid import MANAGER +from astroid import node_classes +from astroid import scoped_nodes +from astroid import util + + +BUILTINS = builtins.__name__ +objectmodel = util.lazy_import("interpreter.objectmodel") + + +class FrozenSet(node_classes._BaseContainer): + """class representing a FrozenSet composite node""" + + def pytype(self): + return "%s.frozenset" % BUILTINS + + def _infer(self, context=None): + yield self + + @decorators.cachedproperty + def _proxied(self): # pylint: disable=method-hidden + ast_builtins = MANAGER.builtins_module + return ast_builtins.getattr("frozenset")[0] + + +class Super(node_classes.NodeNG): + """Proxy class over a super call. + + This class offers almost the same behaviour as Python's super, + which is MRO lookups for retrieving attributes from the parents. + + The *mro_pointer* is the place in the MRO from where we should + start looking, not counting it. *mro_type* is the object which + provides the MRO, it can be both a type or an instance. + *self_class* is the class where the super call is, while + *scope* is the function where the super call is. + """ + + # pylint: disable=unnecessary-lambda + special_attributes = util.lazy_descriptor(lambda: objectmodel.SuperModel()) + + # pylint: disable=super-init-not-called + def __init__(self, mro_pointer, mro_type, self_class, scope): + self.type = mro_type + self.mro_pointer = mro_pointer + self._class_based = False + self._self_class = self_class + self._scope = scope + + def _infer(self, context=None): + yield self + + def super_mro(self): + """Get the MRO which will be used to lookup attributes in this super.""" + if not isinstance(self.mro_pointer, scoped_nodes.ClassDef): + raise exceptions.SuperError( + "The first argument to super must be a subtype of " + "type, not {mro_pointer}.", + super_=self, + ) + + if isinstance(self.type, scoped_nodes.ClassDef): + # `super(type, type)`, most likely in a class method. + self._class_based = True + mro_type = self.type + else: + mro_type = getattr(self.type, "_proxied", None) + if not isinstance(mro_type, (bases.Instance, scoped_nodes.ClassDef)): + raise exceptions.SuperError( + "The second argument to super must be an " + "instance or subtype of type, not {type}.", + super_=self, + ) + + if not mro_type.newstyle: + raise exceptions.SuperError( + "Unable to call super on old-style classes.", super_=self + ) + + mro = mro_type.mro() + if self.mro_pointer not in mro: + raise exceptions.SuperError( + "The second argument to super must be an " + "instance or subtype of type, not {type}.", + super_=self, + ) + + index = mro.index(self.mro_pointer) + return mro[index + 1 :] + + @decorators.cachedproperty + def _proxied(self): + ast_builtins = MANAGER.builtins_module + return ast_builtins.getattr("super")[0] + + def pytype(self): + return "%s.super" % BUILTINS + + def display_type(self): + return "Super of" + + @property + def name(self): + """Get the name of the MRO pointer.""" + return self.mro_pointer.name + + def qname(self): + return "super" + + def igetattr(self, name, context=None): + """Retrieve the inferred values of the given attribute name.""" + + if name in self.special_attributes: + yield self.special_attributes.lookup(name) + return + + try: + mro = self.super_mro() + # Don't let invalid MROs or invalid super calls + # leak out as is from this function. + except exceptions.SuperError as exc: + raise exceptions.AttributeInferenceError( + ( + "Lookup for {name} on {target!r} because super call {super!r} " + "is invalid." + ), + target=self, + attribute=name, + context=context, + super_=exc.super_, + ) from exc + except exceptions.MroError as exc: + raise exceptions.AttributeInferenceError( + ( + "Lookup for {name} on {target!r} failed because {cls!r} has an " + "invalid MRO." + ), + target=self, + attribute=name, + context=context, + mros=exc.mros, + cls=exc.cls, + ) from exc + found = False + for cls in mro: + if name not in cls.locals: + continue + + found = True + for inferred in bases._infer_stmts([cls[name]], context, frame=self): + if not isinstance(inferred, scoped_nodes.FunctionDef): + yield inferred + continue + + # We can obtain different descriptors from a super depending + # on what we are accessing and where the super call is. + if inferred.type == "classmethod": + yield bases.BoundMethod(inferred, cls) + elif self._scope.type == "classmethod" and inferred.type == "method": + yield inferred + elif self._class_based or inferred.type == "staticmethod": + yield inferred + elif isinstance(inferred, Property): + function = inferred.function + try: + yield from function.infer_call_result( + caller=self, context=context + ) + except exceptions.InferenceError: + yield util.Uninferable + elif bases._is_property(inferred): + # TODO: support other descriptors as well. + try: + yield from inferred.infer_call_result(self, context) + except exceptions.InferenceError: + yield util.Uninferable + else: + yield bases.BoundMethod(inferred, cls) + + if not found: + raise exceptions.AttributeInferenceError( + target=self, attribute=name, context=context + ) + + def getattr(self, name, context=None): + return list(self.igetattr(name, context=context)) + + +class ExceptionInstance(bases.Instance): + """Class for instances of exceptions + + It has special treatment for some of the exceptions's attributes, + which are transformed at runtime into certain concrete objects, such as + the case of .args. + """ + + @decorators.cachedproperty + def special_attributes(self): + qname = self.qname() + instance = objectmodel.BUILTIN_EXCEPTIONS.get( + qname, objectmodel.ExceptionInstanceModel + ) + return instance()(self) + + +class DictInstance(bases.Instance): + """Special kind of instances for dictionaries + + This instance knows the underlying object model of the dictionaries, which means + that methods such as .values or .items can be properly inferred. + """ + + # pylint: disable=unnecessary-lambda + special_attributes = util.lazy_descriptor(lambda: objectmodel.DictModel()) + + +# Custom objects tailored for dictionaries, which are used to +# disambiguate between the types of Python 2 dict's method returns +# and Python 3 (where they return set like objects). +class DictItems(bases.Proxy): + __str__ = node_classes.NodeNG.__str__ + __repr__ = node_classes.NodeNG.__repr__ + + +class DictKeys(bases.Proxy): + __str__ = node_classes.NodeNG.__str__ + __repr__ = node_classes.NodeNG.__repr__ + + +class DictValues(bases.Proxy): + __str__ = node_classes.NodeNG.__str__ + __repr__ = node_classes.NodeNG.__repr__ + + +class PartialFunction(scoped_nodes.FunctionDef): + """A class representing partial function obtained via functools.partial""" + + def __init__( + self, call, name=None, doc=None, lineno=None, col_offset=None, parent=None + ): + super().__init__(name, doc, lineno, col_offset, parent) + self.filled_positionals = len(call.positional_arguments[1:]) + self.filled_args = call.positional_arguments[1:] + self.filled_keywords = call.keyword_arguments + + def infer_call_result(self, caller=None, context=None): + if context: + current_passed_keywords = { + keyword for (keyword, _) in context.callcontext.keywords + } + for keyword, value in self.filled_keywords.items(): + if keyword not in current_passed_keywords: + context.callcontext.keywords.append((keyword, value)) + + call_context_args = context.callcontext.args or [] + context.callcontext.args = self.filled_args + call_context_args + + return super().infer_call_result(caller=caller, context=context) + + def qname(self): + return self.__class__.__name__ + + +# TODO: Hack to solve the circular import problem between node_classes and objects +# This is not needed in 2.0, which has a cleaner design overall +node_classes.Dict.__bases__ = (node_classes.NodeNG, DictInstance) + + +class Property(scoped_nodes.FunctionDef): + """Class representing a Python property""" + + def __init__( + self, function, name=None, doc=None, lineno=None, col_offset=None, parent=None + ): + self.function = function + super().__init__(name, doc, lineno, col_offset, parent) + + # pylint: disable=unnecessary-lambda + special_attributes = util.lazy_descriptor(lambda: objectmodel.PropertyModel()) + type = "property" + + def pytype(self): + return "%s.property" % BUILTINS + + def infer_call_result(self, caller=None, context=None): + raise exceptions.InferenceError("Properties are not callable") + + def infer(self, context=None, **kwargs): + return iter((self,)) diff --git a/env/lib/python3.8/site-packages/astroid/protocols.py b/env/lib/python3.8/site-packages/astroid/protocols.py new file mode 100644 index 000000000..2cdf5548b --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/protocols.py @@ -0,0 +1,780 @@ +# -*- coding: utf-8 -*- +# Copyright (c) 2009-2011, 2013-2014 LOGILAB S.A. (Paris, FRANCE) +# Copyright (c) 2014-2020 Claudiu Popa +# Copyright (c) 2014 Google, Inc. +# Copyright (c) 2014 Eevee (Alex Munroe) +# Copyright (c) 2015-2016 Ceridwen +# Copyright (c) 2015 Dmitry Pribysh +# Copyright (c) 2016 Derek Gustafson +# Copyright (c) 2017-2018 Ashley Whetter +# Copyright (c) 2017 Łukasz Rogalski +# Copyright (c) 2017 rr- +# Copyright (c) 2018 Nick Drozd +# Copyright (c) 2018 Ville Skyttä +# Copyright (c) 2018 Bryce Guinta +# Copyright (c) 2018 HoverHell +# Copyright (c) 2019 Hugo van Kemenade + +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER + +"""this module contains a set of functions to handle python protocols for nodes +where it makes sense. +""" + +import collections +import operator as operator_mod + +import itertools + +from astroid import Store +from astroid import arguments +from astroid import bases +from astroid import context as contextmod +from astroid import exceptions +from astroid import decorators +from astroid import node_classes +from astroid import helpers +from astroid import nodes +from astroid import util + +raw_building = util.lazy_import("raw_building") +objects = util.lazy_import("objects") + + +def _reflected_name(name): + return "__r" + name[2:] + + +def _augmented_name(name): + return "__i" + name[2:] + + +_CONTEXTLIB_MGR = "contextlib.contextmanager" +BIN_OP_METHOD = { + "+": "__add__", + "-": "__sub__", + "/": "__truediv__", + "//": "__floordiv__", + "*": "__mul__", + "**": "__pow__", + "%": "__mod__", + "&": "__and__", + "|": "__or__", + "^": "__xor__", + "<<": "__lshift__", + ">>": "__rshift__", + "@": "__matmul__", +} + +REFLECTED_BIN_OP_METHOD = { + key: _reflected_name(value) for (key, value) in BIN_OP_METHOD.items() +} +AUGMENTED_OP_METHOD = { + key + "=": _augmented_name(value) for (key, value) in BIN_OP_METHOD.items() +} + +UNARY_OP_METHOD = { + "+": "__pos__", + "-": "__neg__", + "~": "__invert__", + "not": None, # XXX not '__nonzero__' +} +_UNARY_OPERATORS = { + "+": operator_mod.pos, + "-": operator_mod.neg, + "~": operator_mod.invert, + "not": operator_mod.not_, +} + + +def _infer_unary_op(obj, op): + func = _UNARY_OPERATORS[op] + value = func(obj) + return nodes.const_factory(value) + + +nodes.Tuple.infer_unary_op = lambda self, op: _infer_unary_op(tuple(self.elts), op) +nodes.List.infer_unary_op = lambda self, op: _infer_unary_op(self.elts, op) +nodes.Set.infer_unary_op = lambda self, op: _infer_unary_op(set(self.elts), op) +nodes.Const.infer_unary_op = lambda self, op: _infer_unary_op(self.value, op) +nodes.Dict.infer_unary_op = lambda self, op: _infer_unary_op(dict(self.items), op) + +# Binary operations + +BIN_OP_IMPL = { + "+": lambda a, b: a + b, + "-": lambda a, b: a - b, + "/": lambda a, b: a / b, + "//": lambda a, b: a // b, + "*": lambda a, b: a * b, + "**": lambda a, b: a ** b, + "%": lambda a, b: a % b, + "&": lambda a, b: a & b, + "|": lambda a, b: a | b, + "^": lambda a, b: a ^ b, + "<<": lambda a, b: a << b, + ">>": lambda a, b: a >> b, + "@": operator_mod.matmul, +} +for _KEY, _IMPL in list(BIN_OP_IMPL.items()): + BIN_OP_IMPL[_KEY + "="] = _IMPL + + +@decorators.yes_if_nothing_inferred +def const_infer_binary_op(self, opnode, operator, other, context, _): + not_implemented = nodes.Const(NotImplemented) + if isinstance(other, nodes.Const): + try: + impl = BIN_OP_IMPL[operator] + try: + yield nodes.const_factory(impl(self.value, other.value)) + except TypeError: + # ArithmeticError is not enough: float >> float is a TypeError + yield not_implemented + except Exception: # pylint: disable=broad-except + yield util.Uninferable + except TypeError: + yield not_implemented + elif isinstance(self.value, str) and operator == "%": + # TODO(cpopa): implement string interpolation later on. + yield util.Uninferable + else: + yield not_implemented + + +nodes.Const.infer_binary_op = const_infer_binary_op + + +def _multiply_seq_by_int(self, opnode, other, context): + node = self.__class__(parent=opnode) + filtered_elts = ( + helpers.safe_infer(elt, context) or util.Uninferable + for elt in self.elts + if elt is not util.Uninferable + ) + node.elts = list(filtered_elts) * other.value + return node + + +def _filter_uninferable_nodes(elts, context): + for elt in elts: + if elt is util.Uninferable: + yield nodes.Unknown() + else: + for inferred in elt.infer(context): + if inferred is not util.Uninferable: + yield inferred + else: + yield nodes.Unknown() + + +@decorators.yes_if_nothing_inferred +def tl_infer_binary_op(self, opnode, operator, other, context, method): + not_implemented = nodes.Const(NotImplemented) + if isinstance(other, self.__class__) and operator == "+": + node = self.__class__(parent=opnode) + node.elts = list( + itertools.chain( + _filter_uninferable_nodes(self.elts, context), + _filter_uninferable_nodes(other.elts, context), + ) + ) + yield node + elif isinstance(other, nodes.Const) and operator == "*": + if not isinstance(other.value, int): + yield not_implemented + return + yield _multiply_seq_by_int(self, opnode, other, context) + elif isinstance(other, bases.Instance) and operator == "*": + # Verify if the instance supports __index__. + as_index = helpers.class_instance_as_index(other) + if not as_index: + yield util.Uninferable + else: + yield _multiply_seq_by_int(self, opnode, as_index, context) + else: + yield not_implemented + + +nodes.Tuple.infer_binary_op = tl_infer_binary_op +nodes.List.infer_binary_op = tl_infer_binary_op + + +@decorators.yes_if_nothing_inferred +def instance_class_infer_binary_op(self, opnode, operator, other, context, method): + return method.infer_call_result(self, context) + + +bases.Instance.infer_binary_op = instance_class_infer_binary_op +nodes.ClassDef.infer_binary_op = instance_class_infer_binary_op + + +# assignment ################################################################## + +"""the assigned_stmts method is responsible to return the assigned statement +(e.g. not inferred) according to the assignment type. + +The `assign_path` argument is used to record the lhs path of the original node. +For instance if we want assigned statements for 'c' in 'a, (b,c)', assign_path +will be [1, 1] once arrived to the Assign node. + +The `context` argument is the current inference context which should be given +to any intermediary inference necessary. +""" + + +def _resolve_looppart(parts, assign_path, context): + """recursive function to resolve multiple assignments on loops""" + assign_path = assign_path[:] + index = assign_path.pop(0) + for part in parts: + if part is util.Uninferable: + continue + if not hasattr(part, "itered"): + continue + try: + itered = part.itered() + except TypeError: + continue + for stmt in itered: + index_node = nodes.Const(index) + try: + assigned = stmt.getitem(index_node, context) + except ( + AttributeError, + exceptions.AstroidTypeError, + exceptions.AstroidIndexError, + ): + continue + if not assign_path: + # we achieved to resolved the assignment path, + # don't infer the last part + yield assigned + elif assigned is util.Uninferable: + break + else: + # we are not yet on the last part of the path + # search on each possibly inferred value + try: + yield from _resolve_looppart( + assigned.infer(context), assign_path, context + ) + except exceptions.InferenceError: + break + + +@decorators.raise_if_nothing_inferred +def for_assigned_stmts(self, node=None, context=None, assign_path=None): + if isinstance(self, nodes.AsyncFor) or getattr(self, "is_async", False): + # Skip inferring of async code for now + return dict(node=self, unknown=node, assign_path=assign_path, context=context) + if assign_path is None: + for lst in self.iter.infer(context): + if isinstance(lst, (nodes.Tuple, nodes.List)): + yield from lst.elts + else: + yield from _resolve_looppart(self.iter.infer(context), assign_path, context) + return dict(node=self, unknown=node, assign_path=assign_path, context=context) + + +nodes.For.assigned_stmts = for_assigned_stmts +nodes.Comprehension.assigned_stmts = for_assigned_stmts + + +def sequence_assigned_stmts(self, node=None, context=None, assign_path=None): + if assign_path is None: + assign_path = [] + try: + index = self.elts.index(node) + except ValueError as exc: + raise exceptions.InferenceError( + "Tried to retrieve a node {node!r} which does not exist", + node=self, + assign_path=assign_path, + context=context, + ) from exc + + assign_path.insert(0, index) + return self.parent.assigned_stmts( + node=self, context=context, assign_path=assign_path + ) + + +nodes.Tuple.assigned_stmts = sequence_assigned_stmts +nodes.List.assigned_stmts = sequence_assigned_stmts + + +def assend_assigned_stmts(self, node=None, context=None, assign_path=None): + return self.parent.assigned_stmts(node=self, context=context) + + +nodes.AssignName.assigned_stmts = assend_assigned_stmts +nodes.AssignAttr.assigned_stmts = assend_assigned_stmts + + +def _arguments_infer_argname(self, name, context): + # arguments information may be missing, in which case we can't do anything + # more + if not (self.arguments or self.vararg or self.kwarg): + yield util.Uninferable + return + + functype = self.parent.type + # first argument of instance/class method + if ( + self.arguments + and getattr(self.arguments[0], "name", None) == name + and functype != "staticmethod" + ): + cls = self.parent.parent.scope() + is_metaclass = isinstance(cls, nodes.ClassDef) and cls.type == "metaclass" + # If this is a metaclass, then the first argument will always + # be the class, not an instance. + if context.boundnode and isinstance(context.boundnode, bases.Instance): + cls = context.boundnode._proxied + if is_metaclass or functype == "classmethod": + yield cls + return + if functype == "method": + yield cls.instantiate_class() + return + + if context and context.callcontext: + call_site = arguments.CallSite(context.callcontext, context.extra_context) + yield from call_site.infer_argument(self.parent, name, context) + return + + if name == self.vararg: + vararg = nodes.const_factory(()) + vararg.parent = self + if not self.arguments and self.parent.name == "__init__": + cls = self.parent.parent.scope() + vararg.elts = [cls.instantiate_class()] + yield vararg + return + if name == self.kwarg: + kwarg = nodes.const_factory({}) + kwarg.parent = self + yield kwarg + return + # if there is a default value, yield it. And then yield Uninferable to reflect + # we can't guess given argument value + try: + context = contextmod.copy_context(context) + yield from self.default_value(name).infer(context) + yield util.Uninferable + except exceptions.NoDefault: + yield util.Uninferable + + +def arguments_assigned_stmts(self, node=None, context=None, assign_path=None): + if context.callcontext: + # reset call context/name + callcontext = context.callcontext + context = contextmod.copy_context(context) + context.callcontext = None + args = arguments.CallSite(callcontext, context=context) + return args.infer_argument(self.parent, node.name, context) + return _arguments_infer_argname(self, node.name, context) + + +nodes.Arguments.assigned_stmts = arguments_assigned_stmts + + +@decorators.raise_if_nothing_inferred +def assign_assigned_stmts(self, node=None, context=None, assign_path=None): + if not assign_path: + yield self.value + return None + yield from _resolve_assignment_parts( + self.value.infer(context), assign_path, context + ) + + return dict(node=self, unknown=node, assign_path=assign_path, context=context) + + +def assign_annassigned_stmts(self, node=None, context=None, assign_path=None): + for inferred in assign_assigned_stmts(self, node, context, assign_path): + if inferred is None: + yield util.Uninferable + else: + yield inferred + + +nodes.Assign.assigned_stmts = assign_assigned_stmts +nodes.AnnAssign.assigned_stmts = assign_annassigned_stmts +nodes.AugAssign.assigned_stmts = assign_assigned_stmts + + +def _resolve_assignment_parts(parts, assign_path, context): + """recursive function to resolve multiple assignments""" + assign_path = assign_path[:] + index = assign_path.pop(0) + for part in parts: + assigned = None + if isinstance(part, nodes.Dict): + # A dictionary in an iterating context + try: + assigned, _ = part.items[index] + except IndexError: + return + + elif hasattr(part, "getitem"): + index_node = nodes.Const(index) + try: + assigned = part.getitem(index_node, context) + except (exceptions.AstroidTypeError, exceptions.AstroidIndexError): + return + + if not assigned: + return + + if not assign_path: + # we achieved to resolved the assignment path, don't infer the + # last part + yield assigned + elif assigned is util.Uninferable: + return + else: + # we are not yet on the last part of the path search on each + # possibly inferred value + try: + yield from _resolve_assignment_parts( + assigned.infer(context), assign_path, context + ) + except exceptions.InferenceError: + return + + +@decorators.raise_if_nothing_inferred +def excepthandler_assigned_stmts(self, node=None, context=None, assign_path=None): + for assigned in node_classes.unpack_infer(self.type): + if isinstance(assigned, nodes.ClassDef): + assigned = objects.ExceptionInstance(assigned) + + yield assigned + return dict(node=self, unknown=node, assign_path=assign_path, context=context) + + +nodes.ExceptHandler.assigned_stmts = excepthandler_assigned_stmts + + +def _infer_context_manager(self, mgr, context): + inferred = next(mgr.infer(context=context)) + if isinstance(inferred, bases.Generator): + # Check if it is decorated with contextlib.contextmanager. + func = inferred.parent + if not func.decorators: + raise exceptions.InferenceError( + "No decorators found on inferred generator %s", node=func + ) + + for decorator_node in func.decorators.nodes: + decorator = next(decorator_node.infer(context=context)) + if isinstance(decorator, nodes.FunctionDef): + if decorator.qname() == _CONTEXTLIB_MGR: + break + else: + # It doesn't interest us. + raise exceptions.InferenceError(node=func) + + # Get the first yield point. If it has multiple yields, + # then a RuntimeError will be raised. + + possible_yield_points = func.nodes_of_class(nodes.Yield) + # Ignore yields in nested functions + yield_point = next( + (node for node in possible_yield_points if node.scope() == func), None + ) + if yield_point: + if not yield_point.value: + const = nodes.Const(None) + const.parent = yield_point + const.lineno = yield_point.lineno + yield const + else: + yield from yield_point.value.infer(context=context) + elif isinstance(inferred, bases.Instance): + try: + enter = next(inferred.igetattr("__enter__", context=context)) + except (exceptions.InferenceError, exceptions.AttributeInferenceError): + raise exceptions.InferenceError(node=inferred) + if not isinstance(enter, bases.BoundMethod): + raise exceptions.InferenceError(node=enter) + yield from enter.infer_call_result(self, context) + else: + raise exceptions.InferenceError(node=mgr) + + +@decorators.raise_if_nothing_inferred +def with_assigned_stmts(self, node=None, context=None, assign_path=None): + """Infer names and other nodes from a *with* statement. + + This enables only inference for name binding in a *with* statement. + For instance, in the following code, inferring `func` will return + the `ContextManager` class, not whatever ``__enter__`` returns. + We are doing this intentionally, because we consider that the context + manager result is whatever __enter__ returns and what it is binded + using the ``as`` keyword. + + class ContextManager(object): + def __enter__(self): + return 42 + with ContextManager() as f: + pass + + # ContextManager().infer() will return ContextManager + # f.infer() will return 42. + + Arguments: + self: nodes.With + node: The target of the assignment, `as (a, b)` in `with foo as (a, b)`. + context: Inference context used for caching already inferred objects + assign_path: + A list of indices, where each index specifies what item to fetch from + the inference results. + """ + try: + mgr = next(mgr for (mgr, vars) in self.items if vars == node) + except StopIteration: + return None + if assign_path is None: + yield from _infer_context_manager(self, mgr, context) + else: + for result in _infer_context_manager(self, mgr, context): + # Walk the assign_path and get the item at the final index. + obj = result + for index in assign_path: + if not hasattr(obj, "elts"): + raise exceptions.InferenceError( + "Wrong type ({targets!r}) for {node!r} assignment", + node=self, + targets=node, + assign_path=assign_path, + context=context, + ) + try: + obj = obj.elts[index] + except IndexError as exc: + raise exceptions.InferenceError( + "Tried to infer a nonexistent target with index {index} " + "in {node!r}.", + node=self, + targets=node, + assign_path=assign_path, + context=context, + ) from exc + except TypeError as exc: + raise exceptions.InferenceError( + "Tried to unpack a non-iterable value " "in {node!r}.", + node=self, + targets=node, + assign_path=assign_path, + context=context, + ) from exc + yield obj + return dict(node=self, unknown=node, assign_path=assign_path, context=context) + + +nodes.With.assigned_stmts = with_assigned_stmts + + +@decorators.raise_if_nothing_inferred +def named_expr_assigned_stmts(self, node, context=None, assign_path=None): + """Infer names and other nodes from an assignment expression""" + if self.target == node: + yield from self.value.infer(context=context) + else: + raise exceptions.InferenceError( + "Cannot infer NamedExpr node {node!r}", + node=self, + assign_path=assign_path, + context=context, + ) + + +nodes.NamedExpr.assigned_stmts = named_expr_assigned_stmts + + +@decorators.yes_if_nothing_inferred +def starred_assigned_stmts(self, node=None, context=None, assign_path=None): + """ + Arguments: + self: nodes.Starred + node: a node related to the current underlying Node. + context: Inference context used for caching already inferred objects + assign_path: + A list of indices, where each index specifies what item to fetch from + the inference results. + """ + # pylint: disable=too-many-locals,too-many-branches,too-many-statements + def _determine_starred_iteration_lookups(starred, target, lookups): + # Determine the lookups for the rhs of the iteration + itered = target.itered() + for index, element in enumerate(itered): + if ( + isinstance(element, nodes.Starred) + and element.value.name == starred.value.name + ): + lookups.append((index, len(itered))) + break + if isinstance(element, nodes.Tuple): + lookups.append((index, len(element.itered()))) + _determine_starred_iteration_lookups(starred, element, lookups) + + stmt = self.statement() + if not isinstance(stmt, (nodes.Assign, nodes.For)): + raise exceptions.InferenceError( + "Statement {stmt!r} enclosing {node!r} " "must be an Assign or For node.", + node=self, + stmt=stmt, + unknown=node, + context=context, + ) + + if context is None: + context = contextmod.InferenceContext() + + if isinstance(stmt, nodes.Assign): + value = stmt.value + lhs = stmt.targets[0] + + if sum(1 for _ in lhs.nodes_of_class(nodes.Starred)) > 1: + raise exceptions.InferenceError( + "Too many starred arguments in the " " assignment targets {lhs!r}.", + node=self, + targets=lhs, + unknown=node, + context=context, + ) + + try: + rhs = next(value.infer(context)) + except exceptions.InferenceError: + yield util.Uninferable + return + if rhs is util.Uninferable or not hasattr(rhs, "itered"): + yield util.Uninferable + return + + try: + elts = collections.deque(rhs.itered()) + except TypeError: + yield util.Uninferable + return + + # Unpack iteratively the values from the rhs of the assignment, + # until the find the starred node. What will remain will + # be the list of values which the Starred node will represent + # This is done in two steps, from left to right to remove + # anything before the starred node and from right to left + # to remove anything after the starred node. + + for index, left_node in enumerate(lhs.elts): + if not isinstance(left_node, nodes.Starred): + if not elts: + break + elts.popleft() + continue + lhs_elts = collections.deque(reversed(lhs.elts[index:])) + for right_node in lhs_elts: + if not isinstance(right_node, nodes.Starred): + if not elts: + break + elts.pop() + continue + + # We're done unpacking. + elts = list(elts) + packed = nodes.List( + ctx=Store, parent=self, lineno=lhs.lineno, col_offset=lhs.col_offset + ) + packed.postinit(elts=elts) + yield packed + break + + if isinstance(stmt, nodes.For): + try: + inferred_iterable = next(stmt.iter.infer(context=context)) + except exceptions.InferenceError: + yield util.Uninferable + return + if inferred_iterable is util.Uninferable or not hasattr( + inferred_iterable, "itered" + ): + yield util.Uninferable + return + try: + itered = inferred_iterable.itered() + except TypeError: + yield util.Uninferable + return + + target = stmt.target + + if not isinstance(target, nodes.Tuple): + raise exceptions.InferenceError( + "Could not make sense of this, the target must be a tuple", + context=context, + ) + + lookups = [] + _determine_starred_iteration_lookups(self, target, lookups) + if not lookups: + raise exceptions.InferenceError( + "Could not make sense of this, needs at least a lookup", context=context + ) + + # Make the last lookup a slice, since that what we want for a Starred node + last_element_index, last_element_length = lookups[-1] + is_starred_last = last_element_index == (last_element_length - 1) + + lookup_slice = slice( + last_element_index, + None if is_starred_last else (last_element_length - last_element_index), + ) + lookups[-1] = lookup_slice + + for element in itered: + + # We probably want to infer the potential values *for each* element in an + # iterable, but we can't infer a list of all values, when only a list of + # step values are expected: + # + # for a, *b in [...]: + # b + # + # *b* should now point to just the elements at that particular iteration step, + # which astroid can't know about. + + found_element = None + for lookup in lookups: + if not hasattr(element, "itered"): + break + if not isinstance(lookup, slice): + # Grab just the index, not the whole length + lookup = lookup[0] + try: + itered_inner_element = element.itered() + element = itered_inner_element[lookup] + except IndexError: + break + except TypeError: + # Most likely the itered() call failed, cannot make sense of this + yield util.Uninferable + return + else: + found_element = element + + unpacked = nodes.List( + ctx=Store, parent=self, lineno=self.lineno, col_offset=self.col_offset + ) + unpacked.postinit(elts=found_element or []) + yield unpacked + return + + yield util.Uninferable + + +nodes.Starred.assigned_stmts = starred_assigned_stmts diff --git a/env/lib/python3.8/site-packages/astroid/raw_building.py b/env/lib/python3.8/site-packages/astroid/raw_building.py new file mode 100644 index 000000000..b26127787 --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/raw_building.py @@ -0,0 +1,483 @@ +# -*- coding: utf-8 -*- +# Copyright (c) 2006-2014 LOGILAB S.A. (Paris, FRANCE) +# Copyright (c) 2012 FELD Boris +# Copyright (c) 2014-2020 Claudiu Popa +# Copyright (c) 2014 Google, Inc. +# Copyright (c) 2015-2016 Ceridwen +# Copyright (c) 2015 Florian Bruhin +# Copyright (c) 2015 Ovidiu Sabou +# Copyright (c) 2016 Derek Gustafson +# Copyright (c) 2016 Jakub Wilk +# Copyright (c) 2018 Ville Skyttä +# Copyright (c) 2018 Nick Drozd +# Copyright (c) 2018 Bryce Guinta +# Copyright (c) 2020 Robin Jarry + +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER + +"""this module contains a set of functions to create astroid trees from scratch +(build_* functions) or from living object (object_build_* functions) +""" + +import builtins +import inspect +import os +import sys +import types + +from astroid import bases +from astroid import manager +from astroid import node_classes +from astroid import nodes + + +MANAGER = manager.AstroidManager() +# the keys of CONST_CLS eg python builtin types + +_CONSTANTS = tuple(node_classes.CONST_CLS) +_BUILTINS = vars(builtins) +TYPE_NONE = type(None) +TYPE_NOTIMPLEMENTED = type(NotImplemented) +TYPE_ELLIPSIS = type(...) + + +def _io_discrepancy(member): + # _io module names itself `io`: http://bugs.python.org/issue18602 + member_self = getattr(member, "__self__", None) + return ( + member_self + and inspect.ismodule(member_self) + and member_self.__name__ == "_io" + and member.__module__ == "io" + ) + + +def _attach_local_node(parent, node, name): + node.name = name # needed by add_local_node + parent.add_local_node(node) + + +def _add_dunder_class(func, member): + """Add a __class__ member to the given func node, if we can determine it.""" + python_cls = member.__class__ + cls_name = getattr(python_cls, "__name__", None) + if not cls_name: + return + cls_bases = [ancestor.__name__ for ancestor in python_cls.__bases__] + ast_klass = build_class(cls_name, cls_bases, python_cls.__doc__) + func.instance_attrs["__class__"] = [ast_klass] + + +_marker = object() + + +def attach_dummy_node(node, name, runtime_object=_marker): + """create a dummy node and register it in the locals of the given + node with the specified name + """ + enode = nodes.EmptyNode() + enode.object = runtime_object + _attach_local_node(node, enode, name) + + +def _has_underlying_object(self): + return self.object is not None and self.object is not _marker + + +nodes.EmptyNode.has_underlying_object = _has_underlying_object + + +def attach_const_node(node, name, value): + """create a Const node and register it in the locals of the given + node with the specified name + """ + if name not in node.special_attributes: + _attach_local_node(node, nodes.const_factory(value), name) + + +def attach_import_node(node, modname, membername): + """create a ImportFrom node and register it in the locals of the given + node with the specified name + """ + from_node = nodes.ImportFrom(modname, [(membername, None)]) + _attach_local_node(node, from_node, membername) + + +def build_module(name, doc=None): + """create and initialize an astroid Module node""" + node = nodes.Module(name, doc, pure_python=False) + node.package = False + node.parent = None + return node + + +def build_class(name, basenames=(), doc=None): + """create and initialize an astroid ClassDef node""" + node = nodes.ClassDef(name, doc) + for base in basenames: + basenode = nodes.Name() + basenode.name = base + node.bases.append(basenode) + basenode.parent = node + return node + + +def build_function(name, args=None, posonlyargs=None, defaults=None, doc=None): + """create and initialize an astroid FunctionDef node""" + args, defaults, posonlyargs = args or [], defaults or [], posonlyargs or [] + # first argument is now a list of decorators + func = nodes.FunctionDef(name, doc) + func.args = argsnode = nodes.Arguments() + argsnode.args = [] + argsnode.posonlyargs = [] + for arg in args: + argsnode.args.append(nodes.Name()) + argsnode.args[-1].name = arg + argsnode.args[-1].parent = argsnode + for arg in posonlyargs: + argsnode.posonlyargs.append(nodes.Name()) + argsnode.posonlyargs[-1].name = arg + argsnode.posonlyargs[-1].parent = argsnode + argsnode.defaults = [] + for default in defaults: + argsnode.defaults.append(nodes.const_factory(default)) + argsnode.defaults[-1].parent = argsnode + argsnode.kwarg = None + argsnode.vararg = None + argsnode.parent = func + if args: + register_arguments(func) + return func + + +def build_from_import(fromname, names): + """create and initialize an astroid ImportFrom import statement""" + return nodes.ImportFrom(fromname, [(name, None) for name in names]) + + +def register_arguments(func, args=None): + """add given arguments to local + + args is a list that may contains nested lists + (i.e. def func(a, (b, c, d)): ...) + """ + if args is None: + args = func.args.args + if func.args.vararg: + func.set_local(func.args.vararg, func.args) + if func.args.kwarg: + func.set_local(func.args.kwarg, func.args) + for arg in args: + if isinstance(arg, nodes.Name): + func.set_local(arg.name, arg) + else: + register_arguments(func, arg.elts) + + +def object_build_class(node, member, localname): + """create astroid for a living class object""" + basenames = [base.__name__ for base in member.__bases__] + return _base_class_object_build(node, member, basenames, localname=localname) + + +def object_build_function(node, member, localname): + """create astroid for a living function object""" + signature = inspect.signature(member) + args = [] + defaults = [] + posonlyargs = [] + for param_name, param in signature.parameters.items(): + if param.kind == inspect.Parameter.POSITIONAL_ONLY: + posonlyargs.append(param_name) + elif param.kind == inspect.Parameter.POSITIONAL_OR_KEYWORD: + args.append(param_name) + elif param.kind == inspect.Parameter.VAR_POSITIONAL: + args.append(param_name) + elif param.kind == inspect.Parameter.VAR_KEYWORD: + args.append(param_name) + if param.default is not inspect._empty: + defaults.append(param.default) + func = build_function( + getattr(member, "__name__", None) or localname, + args, + posonlyargs, + defaults, + member.__doc__, + ) + node.add_local_node(func, localname) + + +def object_build_datadescriptor(node, member, name): + """create astroid for a living data descriptor object""" + return _base_class_object_build(node, member, [], name) + + +def object_build_methoddescriptor(node, member, localname): + """create astroid for a living method descriptor object""" + # FIXME get arguments ? + func = build_function( + getattr(member, "__name__", None) or localname, doc=member.__doc__ + ) + # set node's arguments to None to notice that we have no information, not + # and empty argument list + func.args.args = None + node.add_local_node(func, localname) + _add_dunder_class(func, member) + + +def _base_class_object_build(node, member, basenames, name=None, localname=None): + """create astroid for a living class object, with a given set of base names + (e.g. ancestors) + """ + klass = build_class( + name or getattr(member, "__name__", None) or localname, + basenames, + member.__doc__, + ) + klass._newstyle = isinstance(member, type) + node.add_local_node(klass, localname) + try: + # limit the instantiation trick since it's too dangerous + # (such as infinite test execution...) + # this at least resolves common case such as Exception.args, + # OSError.errno + if issubclass(member, Exception): + instdict = member().__dict__ + else: + raise TypeError + except TypeError: + pass + else: + for item_name, obj in instdict.items(): + valnode = nodes.EmptyNode() + valnode.object = obj + valnode.parent = klass + valnode.lineno = 1 + klass.instance_attrs[item_name] = [valnode] + return klass + + +def _build_from_function(node, name, member, module): + # verify this is not an imported function + try: + code = member.__code__ + except AttributeError: + # Some implementations don't provide the code object, + # such as Jython. + code = None + filename = getattr(code, "co_filename", None) + if filename is None: + assert isinstance(member, object) + object_build_methoddescriptor(node, member, name) + elif filename != getattr(module, "__file__", None): + attach_dummy_node(node, name, member) + else: + object_build_function(node, member, name) + + +def _safe_has_attribute(obj, member): + try: + return hasattr(obj, member) + except Exception: # pylint: disable=broad-except + return False + + +class InspectBuilder: + """class for building nodes from living object + + this is actually a really minimal representation, including only Module, + FunctionDef and ClassDef nodes and some others as guessed. + """ + + def __init__(self): + self._done = {} + self._module = None + + def inspect_build(self, module, modname=None, path=None): + """build astroid from a living module (i.e. using inspect) + this is used when there is no python source code available (either + because it's a built-in module or because the .py is not available) + """ + self._module = module + if modname is None: + modname = module.__name__ + try: + node = build_module(modname, module.__doc__) + except AttributeError: + # in jython, java modules have no __doc__ (see #109562) + node = build_module(modname) + node.file = node.path = os.path.abspath(path) if path else path + node.name = modname + MANAGER.cache_module(node) + node.package = hasattr(module, "__path__") + self._done = {} + self.object_build(node, module) + return node + + def object_build(self, node, obj): + """recursive method which create a partial ast from real objects + (only function, class, and method are handled) + """ + if obj in self._done: + return self._done[obj] + self._done[obj] = node + for name in dir(obj): + try: + member = getattr(obj, name) + except AttributeError: + # damned ExtensionClass.Base, I know you're there ! + attach_dummy_node(node, name) + continue + if inspect.ismethod(member): + member = member.__func__ + if inspect.isfunction(member): + _build_from_function(node, name, member, self._module) + elif inspect.isbuiltin(member): + if not _io_discrepancy(member) and self.imported_member( + node, member, name + ): + continue + object_build_methoddescriptor(node, member, name) + elif inspect.isclass(member): + if self.imported_member(node, member, name): + continue + if member in self._done: + class_node = self._done[member] + if class_node not in node.locals.get(name, ()): + node.add_local_node(class_node, name) + else: + class_node = object_build_class(node, member, name) + # recursion + self.object_build(class_node, member) + if name == "__class__" and class_node.parent is None: + class_node.parent = self._done[self._module] + elif inspect.ismethoddescriptor(member): + assert isinstance(member, object) + object_build_methoddescriptor(node, member, name) + elif inspect.isdatadescriptor(member): + assert isinstance(member, object) + object_build_datadescriptor(node, member, name) + elif isinstance(member, _CONSTANTS): + attach_const_node(node, name, member) + elif inspect.isroutine(member): + # This should be called for Jython, where some builtin + # methods aren't caught by isbuiltin branch. + _build_from_function(node, name, member, self._module) + elif _safe_has_attribute(member, "__all__"): + module = build_module(name) + _attach_local_node(node, module, name) + # recursion + self.object_build(module, member) + else: + # create an empty node so that the name is actually defined + attach_dummy_node(node, name, member) + return None + + def imported_member(self, node, member, name): + """verify this is not an imported class or handle it""" + # /!\ some classes like ExtensionClass doesn't have a __module__ + # attribute ! Also, this may trigger an exception on badly built module + # (see http://www.logilab.org/ticket/57299 for instance) + try: + modname = getattr(member, "__module__", None) + except TypeError: + modname = None + if modname is None: + if name in ("__new__", "__subclasshook__"): + # Python 2.5.1 (r251:54863, Sep 1 2010, 22:03:14) + # >>> print object.__new__.__module__ + # None + modname = builtins.__name__ + else: + attach_dummy_node(node, name, member) + return True + + real_name = {"gtk": "gtk_gtk", "_io": "io"}.get(modname, modname) + + if real_name != self._module.__name__: + # check if it sounds valid and then add an import node, else use a + # dummy node + try: + getattr(sys.modules[modname], name) + except (KeyError, AttributeError): + attach_dummy_node(node, name, member) + else: + attach_import_node(node, modname, name) + return True + return False + + +### astroid bootstrapping ###################################################### + +_CONST_PROXY = {} + +# TODO : find a nicer way to handle this situation; +def _set_proxied(const): + return _CONST_PROXY[const.value.__class__] + + +def _astroid_bootstrapping(): + """astroid bootstrapping the builtins module""" + # this boot strapping is necessary since we need the Const nodes to + # inspect_build builtins, and then we can proxy Const + builder = InspectBuilder() + astroid_builtin = builder.inspect_build(builtins) + + # pylint: disable=redefined-outer-name + for cls, node_cls in node_classes.CONST_CLS.items(): + if cls is TYPE_NONE: + proxy = build_class("NoneType") + proxy.parent = astroid_builtin + elif cls is TYPE_NOTIMPLEMENTED: + proxy = build_class("NotImplementedType") + proxy.parent = astroid_builtin + elif cls is TYPE_ELLIPSIS: + proxy = build_class("Ellipsis") + proxy.parent = astroid_builtin + else: + proxy = astroid_builtin.getattr(cls.__name__)[0] + if cls in (dict, list, set, tuple): + node_cls._proxied = proxy + else: + _CONST_PROXY[cls] = proxy + + # Set the builtin module as parent for some builtins. + nodes.Const._proxied = property(_set_proxied) + + _GeneratorType = nodes.ClassDef( + types.GeneratorType.__name__, types.GeneratorType.__doc__ + ) + _GeneratorType.parent = astroid_builtin + bases.Generator._proxied = _GeneratorType + builder.object_build(bases.Generator._proxied, types.GeneratorType) + + if hasattr(types, "AsyncGeneratorType"): + # pylint: disable=no-member; AsyncGeneratorType + _AsyncGeneratorType = nodes.ClassDef( + types.AsyncGeneratorType.__name__, types.AsyncGeneratorType.__doc__ + ) + _AsyncGeneratorType.parent = astroid_builtin + bases.AsyncGenerator._proxied = _AsyncGeneratorType + builder.object_build(bases.AsyncGenerator._proxied, types.AsyncGeneratorType) + builtin_types = ( + types.GetSetDescriptorType, + types.GeneratorType, + types.MemberDescriptorType, + TYPE_NONE, + TYPE_NOTIMPLEMENTED, + types.FunctionType, + types.MethodType, + types.BuiltinFunctionType, + types.ModuleType, + types.TracebackType, + ) + for _type in builtin_types: + if _type.__name__ not in astroid_builtin: + cls = nodes.ClassDef(_type.__name__, _type.__doc__) + cls.parent = astroid_builtin + builder.object_build(cls, _type) + astroid_builtin[_type.__name__] = cls + + +_astroid_bootstrapping() diff --git a/env/lib/python3.8/site-packages/astroid/rebuilder.py b/env/lib/python3.8/site-packages/astroid/rebuilder.py new file mode 100644 index 000000000..3fc1a83f2 --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/rebuilder.py @@ -0,0 +1,1010 @@ +# -*- coding: utf-8 -*- +# Copyright (c) 2009-2014 LOGILAB S.A. (Paris, FRANCE) +# Copyright (c) 2013-2020 Claudiu Popa +# Copyright (c) 2013-2014 Google, Inc. +# Copyright (c) 2014 Alexander Presnyakov +# Copyright (c) 2014 Eevee (Alex Munroe) +# Copyright (c) 2015-2016 Ceridwen +# Copyright (c) 2016-2017 Derek Gustafson +# Copyright (c) 2016 Jared Garst +# Copyright (c) 2017 Hugo +# Copyright (c) 2017 Łukasz Rogalski +# Copyright (c) 2017 rr- +# Copyright (c) 2018-2019 Ville Skyttä +# Copyright (c) 2018 Tomas Gavenciak +# Copyright (c) 2018 Serhiy Storchaka +# Copyright (c) 2018 Nick Drozd +# Copyright (c) 2018 Bryce Guinta +# Copyright (c) 2019-2020 Ashley Whetter +# Copyright (c) 2019 Hugo van Kemenade +# Copyright (c) 2019 Zbigniew Jędrzejewski-Szmek + +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER + +"""this module contains utilities for rebuilding an _ast tree in +order to get a single Astroid representation +""" + +import sys +from typing import Optional + +import astroid +from astroid._ast import parse_function_type_comment, get_parser_module, ParserModule +from astroid import nodes + + +CONST_NAME_TRANSFORMS = {"None": None, "True": True, "False": False} + +REDIRECT = { + "arguments": "Arguments", + "comprehension": "Comprehension", + "ListCompFor": "Comprehension", + "GenExprFor": "Comprehension", + "excepthandler": "ExceptHandler", + "keyword": "Keyword", +} +PY37 = sys.version_info >= (3, 7) +PY38 = sys.version_info >= (3, 8) + + +def _visit_or_none(node, attr, visitor, parent, visit="visit", **kws): + """If the given node has an attribute, visits the attribute, and + otherwise returns None. + + """ + value = getattr(node, attr, None) + if value: + return getattr(visitor, visit)(value, parent, **kws) + + return None + + +class TreeRebuilder: + """Rebuilds the _ast tree to become an Astroid tree""" + + def __init__(self, manager, parser_module: Optional[ParserModule] = None): + self._manager = manager + self._global_names = [] + self._import_from_nodes = [] + self._delayed_assattr = [] + self._visit_meths = {} + + if parser_module is None: + self._parser_module = get_parser_module() + else: + self._parser_module = parser_module + self._module = self._parser_module.module + + def _get_doc(self, node): + try: + if PY37 and hasattr(node, "docstring"): + doc = node.docstring + return node, doc + if node.body and isinstance(node.body[0], self._module.Expr): + + first_value = node.body[0].value + if isinstance(first_value, self._module.Str) or ( + PY38 + and isinstance(first_value, self._module.Constant) + and isinstance(first_value.value, str) + ): + doc = first_value.value if PY38 else first_value.s + node.body = node.body[1:] + return node, doc + except IndexError: + pass # ast built from scratch + return node, None + + def _get_context(self, node): + return self._parser_module.context_classes.get(type(node.ctx), astroid.Load) + + def visit_module(self, node, modname, modpath, package): + """visit a Module node by returning a fresh instance of it""" + node, doc = self._get_doc(node) + newnode = nodes.Module( + name=modname, + doc=doc, + file=modpath, + path=[modpath], + package=package, + parent=None, + ) + newnode.postinit([self.visit(child, newnode) for child in node.body]) + return newnode + + def visit(self, node, parent): + cls = node.__class__ + if cls in self._visit_meths: + visit_method = self._visit_meths[cls] + else: + cls_name = cls.__name__ + visit_name = "visit_" + REDIRECT.get(cls_name, cls_name).lower() + visit_method = getattr(self, visit_name) + self._visit_meths[cls] = visit_method + return visit_method(node, parent) + + def _save_assignment(self, node, name=None): + """save assignement situation since node.parent is not available yet""" + if self._global_names and node.name in self._global_names[-1]: + node.root().set_local(node.name, node) + else: + node.parent.set_local(node.name, node) + + def visit_arg(self, node, parent): + """visit an arg node by returning a fresh AssName instance""" + return self.visit_assignname(node, parent, node.arg) + + def visit_arguments(self, node, parent): + """visit an Arguments node by returning a fresh instance of it""" + vararg, kwarg = node.vararg, node.kwarg + newnode = nodes.Arguments( + vararg.arg if vararg else None, kwarg.arg if kwarg else None, parent + ) + args = [self.visit(child, newnode) for child in node.args] + defaults = [self.visit(child, newnode) for child in node.defaults] + varargannotation = None + kwargannotation = None + posonlyargs = [] + # change added in 82732 (7c5c678e4164), vararg and kwarg + # are instances of `_ast.arg`, not strings + if vararg: + if node.vararg.annotation: + varargannotation = self.visit(node.vararg.annotation, newnode) + vararg = vararg.arg + if kwarg: + if node.kwarg.annotation: + kwargannotation = self.visit(node.kwarg.annotation, newnode) + kwarg = kwarg.arg + kwonlyargs = [self.visit(child, newnode) for child in node.kwonlyargs] + kw_defaults = [ + self.visit(child, newnode) if child else None for child in node.kw_defaults + ] + annotations = [ + self.visit(arg.annotation, newnode) if arg.annotation else None + for arg in node.args + ] + kwonlyargs_annotations = [ + self.visit(arg.annotation, newnode) if arg.annotation else None + for arg in node.kwonlyargs + ] + + posonlyargs_annotations = [] + if PY38: + posonlyargs = [self.visit(child, newnode) for child in node.posonlyargs] + posonlyargs_annotations = [ + self.visit(arg.annotation, newnode) if arg.annotation else None + for arg in node.posonlyargs + ] + type_comment_args = [ + self.check_type_comment(child, parent=newnode) for child in node.args + ] + type_comment_kwonlyargs = [ + self.check_type_comment(child, parent=newnode) for child in node.kwonlyargs + ] + type_comment_posonlyargs = [] + if PY38: + type_comment_posonlyargs = [ + self.check_type_comment(child, parent=newnode) + for child in node.posonlyargs + ] + + newnode.postinit( + args=args, + defaults=defaults, + kwonlyargs=kwonlyargs, + posonlyargs=posonlyargs, + kw_defaults=kw_defaults, + annotations=annotations, + kwonlyargs_annotations=kwonlyargs_annotations, + posonlyargs_annotations=posonlyargs_annotations, + varargannotation=varargannotation, + kwargannotation=kwargannotation, + type_comment_args=type_comment_args, + type_comment_kwonlyargs=type_comment_kwonlyargs, + type_comment_posonlyargs=type_comment_posonlyargs, + ) + # save argument names in locals: + if vararg: + newnode.parent.set_local(vararg, newnode) + if kwarg: + newnode.parent.set_local(kwarg, newnode) + return newnode + + def visit_assert(self, node, parent): + """visit a Assert node by returning a fresh instance of it""" + newnode = nodes.Assert(node.lineno, node.col_offset, parent) + if node.msg: + msg = self.visit(node.msg, newnode) + else: + msg = None + newnode.postinit(self.visit(node.test, newnode), msg) + return newnode + + def check_type_comment(self, node, parent): + type_comment = getattr(node, "type_comment", None) + if not type_comment: + return None + + try: + type_comment_ast = self._parser_module.parse(type_comment) + except SyntaxError: + # Invalid type comment, just skip it. + return None + + type_object = self.visit(type_comment_ast.body[0], parent=parent) + if not isinstance(type_object, nodes.Expr): + return None + + return type_object.value + + def check_function_type_comment(self, node): + type_comment = getattr(node, "type_comment", None) + if not type_comment: + return None + + try: + type_comment_ast = parse_function_type_comment(type_comment) + except SyntaxError: + # Invalid type comment, just skip it. + return None + + returns = None + argtypes = [ + self.visit(elem, node) for elem in (type_comment_ast.argtypes or []) + ] + if type_comment_ast.returns: + returns = self.visit(type_comment_ast.returns, node) + + return returns, argtypes + + # Async structs added in Python 3.5 + def visit_asyncfunctiondef(self, node, parent): + return self._visit_functiondef(nodes.AsyncFunctionDef, node, parent) + + def visit_asyncfor(self, node, parent): + return self._visit_for(nodes.AsyncFor, node, parent) + + def visit_await(self, node, parent): + newnode = nodes.Await(node.lineno, node.col_offset, parent) + newnode.postinit(value=self.visit(node.value, newnode)) + return newnode + + def visit_asyncwith(self, node, parent): + return self._visit_with(nodes.AsyncWith, node, parent) + + def visit_assign(self, node, parent): + """visit a Assign node by returning a fresh instance of it""" + newnode = nodes.Assign(node.lineno, node.col_offset, parent) + type_annotation = self.check_type_comment(node, parent=newnode) + newnode.postinit( + targets=[self.visit(child, newnode) for child in node.targets], + value=self.visit(node.value, newnode), + type_annotation=type_annotation, + ) + return newnode + + def visit_annassign(self, node, parent): + """visit an AnnAssign node by returning a fresh instance of it""" + newnode = nodes.AnnAssign(node.lineno, node.col_offset, parent) + annotation = _visit_or_none(node, "annotation", self, newnode) + newnode.postinit( + target=self.visit(node.target, newnode), + annotation=annotation, + simple=node.simple, + value=_visit_or_none(node, "value", self, newnode), + ) + return newnode + + def visit_assignname(self, node, parent, node_name=None): + """visit a node and return a AssignName node""" + newnode = nodes.AssignName( + node_name, + getattr(node, "lineno", None), + getattr(node, "col_offset", None), + parent, + ) + self._save_assignment(newnode) + return newnode + + def visit_augassign(self, node, parent): + """visit a AugAssign node by returning a fresh instance of it""" + newnode = nodes.AugAssign( + self._parser_module.bin_op_classes[type(node.op)] + "=", + node.lineno, + node.col_offset, + parent, + ) + newnode.postinit( + self.visit(node.target, newnode), self.visit(node.value, newnode) + ) + return newnode + + def visit_repr(self, node, parent): + """visit a Backquote node by returning a fresh instance of it""" + newnode = nodes.Repr(node.lineno, node.col_offset, parent) + newnode.postinit(self.visit(node.value, newnode)) + return newnode + + def visit_binop(self, node, parent): + """visit a BinOp node by returning a fresh instance of it""" + newnode = nodes.BinOp( + self._parser_module.bin_op_classes[type(node.op)], + node.lineno, + node.col_offset, + parent, + ) + newnode.postinit( + self.visit(node.left, newnode), self.visit(node.right, newnode) + ) + return newnode + + def visit_boolop(self, node, parent): + """visit a BoolOp node by returning a fresh instance of it""" + newnode = nodes.BoolOp( + self._parser_module.bool_op_classes[type(node.op)], + node.lineno, + node.col_offset, + parent, + ) + newnode.postinit([self.visit(child, newnode) for child in node.values]) + return newnode + + def visit_break(self, node, parent): + """visit a Break node by returning a fresh instance of it""" + return nodes.Break( + getattr(node, "lineno", None), getattr(node, "col_offset", None), parent + ) + + def visit_call(self, node, parent): + """visit a CallFunc node by returning a fresh instance of it""" + newnode = nodes.Call(node.lineno, node.col_offset, parent) + starargs = _visit_or_none(node, "starargs", self, newnode) + kwargs = _visit_or_none(node, "kwargs", self, newnode) + args = [self.visit(child, newnode) for child in node.args] + + if node.keywords: + keywords = [self.visit(child, newnode) for child in node.keywords] + else: + keywords = None + if starargs: + new_starargs = nodes.Starred( + col_offset=starargs.col_offset, + lineno=starargs.lineno, + parent=starargs.parent, + ) + new_starargs.postinit(value=starargs) + args.append(new_starargs) + if kwargs: + new_kwargs = nodes.Keyword( + arg=None, + col_offset=kwargs.col_offset, + lineno=kwargs.lineno, + parent=kwargs.parent, + ) + new_kwargs.postinit(value=kwargs) + if keywords: + keywords.append(new_kwargs) + else: + keywords = [new_kwargs] + + newnode.postinit(self.visit(node.func, newnode), args, keywords) + return newnode + + def visit_classdef(self, node, parent, newstyle=True): + """visit a ClassDef node to become astroid""" + node, doc = self._get_doc(node) + newnode = nodes.ClassDef(node.name, doc, node.lineno, node.col_offset, parent) + metaclass = None + for keyword in node.keywords: + if keyword.arg == "metaclass": + metaclass = self.visit(keyword, newnode).value + break + if node.decorator_list: + decorators = self.visit_decorators(node, newnode) + else: + decorators = None + newnode.postinit( + [self.visit(child, newnode) for child in node.bases], + [self.visit(child, newnode) for child in node.body], + decorators, + newstyle, + metaclass, + [ + self.visit(kwd, newnode) + for kwd in node.keywords + if kwd.arg != "metaclass" + ], + ) + return newnode + + def visit_const(self, node, parent): + """visit a Const node by returning a fresh instance of it""" + return nodes.Const( + node.value, + getattr(node, "lineno", None), + getattr(node, "col_offset", None), + parent, + ) + + def visit_continue(self, node, parent): + """visit a Continue node by returning a fresh instance of it""" + return nodes.Continue( + getattr(node, "lineno", None), getattr(node, "col_offset", None), parent + ) + + def visit_compare(self, node, parent): + """visit a Compare node by returning a fresh instance of it""" + newnode = nodes.Compare(node.lineno, node.col_offset, parent) + newnode.postinit( + self.visit(node.left, newnode), + [ + ( + self._parser_module.cmp_op_classes[op.__class__], + self.visit(expr, newnode), + ) + for (op, expr) in zip(node.ops, node.comparators) + ], + ) + return newnode + + def visit_comprehension(self, node, parent): + """visit a Comprehension node by returning a fresh instance of it""" + newnode = nodes.Comprehension(parent) + newnode.postinit( + self.visit(node.target, newnode), + self.visit(node.iter, newnode), + [self.visit(child, newnode) for child in node.ifs], + getattr(node, "is_async", None), + ) + return newnode + + def visit_decorators(self, node, parent): + """visit a Decorators node by returning a fresh instance of it""" + # /!\ node is actually an _ast.FunctionDef node while + # parent is an astroid.nodes.FunctionDef node + if PY38: + # Set the line number of the first decorator for Python 3.8+. + lineno = node.decorator_list[0].lineno + else: + lineno = node.lineno + newnode = nodes.Decorators(lineno, node.col_offset, parent) + newnode.postinit([self.visit(child, newnode) for child in node.decorator_list]) + return newnode + + def visit_delete(self, node, parent): + """visit a Delete node by returning a fresh instance of it""" + newnode = nodes.Delete(node.lineno, node.col_offset, parent) + newnode.postinit([self.visit(child, newnode) for child in node.targets]) + return newnode + + def _visit_dict_items(self, node, parent, newnode): + for key, value in zip(node.keys, node.values): + rebuilt_value = self.visit(value, newnode) + if not key: + # Python 3.5 and extended unpacking + rebuilt_key = nodes.DictUnpack( + rebuilt_value.lineno, rebuilt_value.col_offset, parent + ) + else: + rebuilt_key = self.visit(key, newnode) + yield rebuilt_key, rebuilt_value + + def visit_dict(self, node, parent): + """visit a Dict node by returning a fresh instance of it""" + newnode = nodes.Dict(node.lineno, node.col_offset, parent) + items = list(self._visit_dict_items(node, parent, newnode)) + newnode.postinit(items) + return newnode + + def visit_dictcomp(self, node, parent): + """visit a DictComp node by returning a fresh instance of it""" + newnode = nodes.DictComp(node.lineno, node.col_offset, parent) + newnode.postinit( + self.visit(node.key, newnode), + self.visit(node.value, newnode), + [self.visit(child, newnode) for child in node.generators], + ) + return newnode + + def visit_expr(self, node, parent): + """visit a Expr node by returning a fresh instance of it""" + newnode = nodes.Expr(node.lineno, node.col_offset, parent) + newnode.postinit(self.visit(node.value, newnode)) + return newnode + + # Not used in Python 3.8+. + def visit_ellipsis(self, node, parent): + """visit an Ellipsis node by returning a fresh instance of it""" + return nodes.Ellipsis( + getattr(node, "lineno", None), getattr(node, "col_offset", None), parent + ) + + def visit_emptynode(self, node, parent): + """visit an EmptyNode node by returning a fresh instance of it""" + return nodes.EmptyNode( + getattr(node, "lineno", None), getattr(node, "col_offset", None), parent + ) + + def visit_excepthandler(self, node, parent): + """visit an ExceptHandler node by returning a fresh instance of it""" + newnode = nodes.ExceptHandler(node.lineno, node.col_offset, parent) + if node.name: + name = self.visit_assignname(node, newnode, node.name) + else: + name = None + newnode.postinit( + _visit_or_none(node, "type", self, newnode), + name, + [self.visit(child, newnode) for child in node.body], + ) + return newnode + + def visit_exec(self, node, parent): + """visit an Exec node by returning a fresh instance of it""" + newnode = nodes.Exec(node.lineno, node.col_offset, parent) + newnode.postinit( + self.visit(node.body, newnode), + _visit_or_none(node, "globals", self, newnode), + _visit_or_none(node, "locals", self, newnode), + ) + return newnode + + # Not used in Python 3.8+. + def visit_extslice(self, node, parent): + """visit an ExtSlice node by returning a fresh instance of it""" + newnode = nodes.ExtSlice(parent=parent) + newnode.postinit([self.visit(dim, newnode) for dim in node.dims]) + return newnode + + def _visit_for(self, cls, node, parent): + """visit a For node by returning a fresh instance of it""" + newnode = cls(node.lineno, node.col_offset, parent) + type_annotation = self.check_type_comment(node, parent=newnode) + newnode.postinit( + target=self.visit(node.target, newnode), + iter=self.visit(node.iter, newnode), + body=[self.visit(child, newnode) for child in node.body], + orelse=[self.visit(child, newnode) for child in node.orelse], + type_annotation=type_annotation, + ) + return newnode + + def visit_for(self, node, parent): + return self._visit_for(nodes.For, node, parent) + + def visit_importfrom(self, node, parent): + """visit an ImportFrom node by returning a fresh instance of it""" + names = [(alias.name, alias.asname) for alias in node.names] + newnode = nodes.ImportFrom( + node.module or "", + names, + node.level or None, + getattr(node, "lineno", None), + getattr(node, "col_offset", None), + parent, + ) + # store From names to add them to locals after building + self._import_from_nodes.append(newnode) + return newnode + + def _visit_functiondef(self, cls, node, parent): + """visit an FunctionDef node to become astroid""" + self._global_names.append({}) + node, doc = self._get_doc(node) + + lineno = node.lineno + if PY38 and node.decorator_list: + # Python 3.8 sets the line number of a decorated function + # to be the actual line number of the function, but the + # previous versions expected the decorator's line number instead. + # We reset the function's line number to that of the + # first decorator to maintain backward compatibility. + # It's not ideal but this discrepancy was baked into + # the framework for *years*. + lineno = node.decorator_list[0].lineno + + newnode = cls(node.name, doc, lineno, node.col_offset, parent) + if node.decorator_list: + decorators = self.visit_decorators(node, newnode) + else: + decorators = None + if node.returns: + returns = self.visit(node.returns, newnode) + else: + returns = None + + type_comment_args = type_comment_returns = None + type_comment_annotation = self.check_function_type_comment(node) + if type_comment_annotation: + type_comment_returns, type_comment_args = type_comment_annotation + newnode.postinit( + args=self.visit(node.args, newnode), + body=[self.visit(child, newnode) for child in node.body], + decorators=decorators, + returns=returns, + type_comment_returns=type_comment_returns, + type_comment_args=type_comment_args, + ) + self._global_names.pop() + return newnode + + def visit_functiondef(self, node, parent): + return self._visit_functiondef(nodes.FunctionDef, node, parent) + + def visit_generatorexp(self, node, parent): + """visit a GeneratorExp node by returning a fresh instance of it""" + newnode = nodes.GeneratorExp(node.lineno, node.col_offset, parent) + newnode.postinit( + self.visit(node.elt, newnode), + [self.visit(child, newnode) for child in node.generators], + ) + return newnode + + def visit_attribute(self, node, parent): + """visit an Attribute node by returning a fresh instance of it""" + context = self._get_context(node) + if context == astroid.Del: + # FIXME : maybe we should reintroduce and visit_delattr ? + # for instance, deactivating assign_ctx + newnode = nodes.DelAttr(node.attr, node.lineno, node.col_offset, parent) + elif context == astroid.Store: + newnode = nodes.AssignAttr(node.attr, node.lineno, node.col_offset, parent) + # Prohibit a local save if we are in an ExceptHandler. + if not isinstance(parent, astroid.ExceptHandler): + self._delayed_assattr.append(newnode) + else: + newnode = nodes.Attribute(node.attr, node.lineno, node.col_offset, parent) + newnode.postinit(self.visit(node.value, newnode)) + return newnode + + def visit_global(self, node, parent): + """visit a Global node to become astroid""" + newnode = nodes.Global( + node.names, + getattr(node, "lineno", None), + getattr(node, "col_offset", None), + parent, + ) + if self._global_names: # global at the module level, no effect + for name in node.names: + self._global_names[-1].setdefault(name, []).append(newnode) + return newnode + + def visit_if(self, node, parent): + """visit an If node by returning a fresh instance of it""" + newnode = nodes.If(node.lineno, node.col_offset, parent) + newnode.postinit( + self.visit(node.test, newnode), + [self.visit(child, newnode) for child in node.body], + [self.visit(child, newnode) for child in node.orelse], + ) + return newnode + + def visit_ifexp(self, node, parent): + """visit a IfExp node by returning a fresh instance of it""" + newnode = nodes.IfExp(node.lineno, node.col_offset, parent) + newnode.postinit( + self.visit(node.test, newnode), + self.visit(node.body, newnode), + self.visit(node.orelse, newnode), + ) + return newnode + + def visit_import(self, node, parent): + """visit a Import node by returning a fresh instance of it""" + names = [(alias.name, alias.asname) for alias in node.names] + newnode = nodes.Import( + names, + getattr(node, "lineno", None), + getattr(node, "col_offset", None), + parent, + ) + # save import names in parent's locals: + for (name, asname) in newnode.names: + name = asname or name + parent.set_local(name.split(".")[0], newnode) + return newnode + + def visit_joinedstr(self, node, parent): + newnode = nodes.JoinedStr(node.lineno, node.col_offset, parent) + newnode.postinit([self.visit(child, newnode) for child in node.values]) + return newnode + + def visit_formattedvalue(self, node, parent): + newnode = nodes.FormattedValue(node.lineno, node.col_offset, parent) + newnode.postinit( + self.visit(node.value, newnode), + node.conversion, + _visit_or_none(node, "format_spec", self, newnode), + ) + return newnode + + def visit_namedexpr(self, node, parent): + newnode = nodes.NamedExpr(node.lineno, node.col_offset, parent) + newnode.postinit( + self.visit(node.target, newnode), self.visit(node.value, newnode) + ) + return newnode + + # Not used in Python 3.8+. + def visit_index(self, node, parent): + """visit a Index node by returning a fresh instance of it""" + newnode = nodes.Index(parent=parent) + newnode.postinit(self.visit(node.value, newnode)) + return newnode + + def visit_keyword(self, node, parent): + """visit a Keyword node by returning a fresh instance of it""" + newnode = nodes.Keyword(node.arg, parent=parent) + newnode.postinit(self.visit(node.value, newnode)) + return newnode + + def visit_lambda(self, node, parent): + """visit a Lambda node by returning a fresh instance of it""" + newnode = nodes.Lambda(node.lineno, node.col_offset, parent) + newnode.postinit(self.visit(node.args, newnode), self.visit(node.body, newnode)) + return newnode + + def visit_list(self, node, parent): + """visit a List node by returning a fresh instance of it""" + context = self._get_context(node) + newnode = nodes.List( + ctx=context, lineno=node.lineno, col_offset=node.col_offset, parent=parent + ) + newnode.postinit([self.visit(child, newnode) for child in node.elts]) + return newnode + + def visit_listcomp(self, node, parent): + """visit a ListComp node by returning a fresh instance of it""" + newnode = nodes.ListComp(node.lineno, node.col_offset, parent) + newnode.postinit( + self.visit(node.elt, newnode), + [self.visit(child, newnode) for child in node.generators], + ) + return newnode + + def visit_name(self, node, parent): + """visit a Name node by returning a fresh instance of it""" + context = self._get_context(node) + # True and False can be assigned to something in py2x, so we have to + # check first the context. + if context == astroid.Del: + newnode = nodes.DelName(node.id, node.lineno, node.col_offset, parent) + elif context == astroid.Store: + newnode = nodes.AssignName(node.id, node.lineno, node.col_offset, parent) + elif node.id in CONST_NAME_TRANSFORMS: + newnode = nodes.Const( + CONST_NAME_TRANSFORMS[node.id], + getattr(node, "lineno", None), + getattr(node, "col_offset", None), + parent, + ) + return newnode + else: + newnode = nodes.Name(node.id, node.lineno, node.col_offset, parent) + # XXX REMOVE me : + if context in (astroid.Del, astroid.Store): # 'Aug' ?? + self._save_assignment(newnode) + return newnode + + # Not used in Python 3.8+. + def visit_nameconstant(self, node, parent): + # in Python 3.4 we have NameConstant for True / False / None + return nodes.Const( + node.value, + getattr(node, "lineno", None), + getattr(node, "col_offset", None), + parent, + ) + + def visit_nonlocal(self, node, parent): + """visit a Nonlocal node and return a new instance of it""" + return nodes.Nonlocal( + node.names, + getattr(node, "lineno", None), + getattr(node, "col_offset", None), + parent, + ) + + def visit_constant(self, node, parent): + """visit a Constant node by returning a fresh instance of Const""" + return nodes.Const( + node.value, + getattr(node, "lineno", None), + getattr(node, "col_offset", None), + parent, + ) + + # Not used in Python 3.8+. + def visit_str(self, node, parent): + """visit a String/Bytes node by returning a fresh instance of Const""" + return nodes.Const( + node.s, + getattr(node, "lineno", None), + getattr(node, "col_offset", None), + parent, + ) + + visit_bytes = visit_str + + # Not used in Python 3.8+. + def visit_num(self, node, parent): + """visit a Num node by returning a fresh instance of Const""" + return nodes.Const( + node.n, + getattr(node, "lineno", None), + getattr(node, "col_offset", None), + parent, + ) + + def visit_pass(self, node, parent): + """visit a Pass node by returning a fresh instance of it""" + return nodes.Pass(node.lineno, node.col_offset, parent) + + def visit_print(self, node, parent): + """visit a Print node by returning a fresh instance of it""" + newnode = nodes.Print(node.nl, node.lineno, node.col_offset, parent) + newnode.postinit( + _visit_or_none(node, "dest", self, newnode), + [self.visit(child, newnode) for child in node.values], + ) + return newnode + + def visit_raise(self, node, parent): + """visit a Raise node by returning a fresh instance of it""" + newnode = nodes.Raise(node.lineno, node.col_offset, parent) + # no traceback; anyway it is not used in Pylint + newnode.postinit( + _visit_or_none(node, "exc", self, newnode), + _visit_or_none(node, "cause", self, newnode), + ) + return newnode + + def visit_return(self, node, parent): + """visit a Return node by returning a fresh instance of it""" + newnode = nodes.Return(node.lineno, node.col_offset, parent) + if node.value is not None: + newnode.postinit(self.visit(node.value, newnode)) + return newnode + + def visit_set(self, node, parent): + """visit a Set node by returning a fresh instance of it""" + newnode = nodes.Set(node.lineno, node.col_offset, parent) + newnode.postinit([self.visit(child, newnode) for child in node.elts]) + return newnode + + def visit_setcomp(self, node, parent): + """visit a SetComp node by returning a fresh instance of it""" + newnode = nodes.SetComp(node.lineno, node.col_offset, parent) + newnode.postinit( + self.visit(node.elt, newnode), + [self.visit(child, newnode) for child in node.generators], + ) + return newnode + + def visit_slice(self, node, parent): + """visit a Slice node by returning a fresh instance of it""" + newnode = nodes.Slice(parent=parent) + newnode.postinit( + _visit_or_none(node, "lower", self, newnode), + _visit_or_none(node, "upper", self, newnode), + _visit_or_none(node, "step", self, newnode), + ) + return newnode + + def visit_subscript(self, node, parent): + """visit a Subscript node by returning a fresh instance of it""" + context = self._get_context(node) + newnode = nodes.Subscript( + ctx=context, lineno=node.lineno, col_offset=node.col_offset, parent=parent + ) + newnode.postinit( + self.visit(node.value, newnode), self.visit(node.slice, newnode) + ) + return newnode + + def visit_starred(self, node, parent): + """visit a Starred node and return a new instance of it""" + context = self._get_context(node) + newnode = nodes.Starred( + ctx=context, lineno=node.lineno, col_offset=node.col_offset, parent=parent + ) + newnode.postinit(self.visit(node.value, newnode)) + return newnode + + def visit_tryexcept(self, node, parent): + """visit a TryExcept node by returning a fresh instance of it""" + newnode = nodes.TryExcept(node.lineno, node.col_offset, parent) + newnode.postinit( + [self.visit(child, newnode) for child in node.body], + [self.visit(child, newnode) for child in node.handlers], + [self.visit(child, newnode) for child in node.orelse], + ) + return newnode + + def visit_try(self, node, parent): + # python 3.3 introduce a new Try node replacing + # TryFinally/TryExcept nodes + if node.finalbody: + newnode = nodes.TryFinally(node.lineno, node.col_offset, parent) + if node.handlers: + body = [self.visit_tryexcept(node, newnode)] + else: + body = [self.visit(child, newnode) for child in node.body] + newnode.postinit(body, [self.visit(n, newnode) for n in node.finalbody]) + return newnode + if node.handlers: + return self.visit_tryexcept(node, parent) + return None + + def visit_tryfinally(self, node, parent): + """visit a TryFinally node by returning a fresh instance of it""" + newnode = nodes.TryFinally(node.lineno, node.col_offset, parent) + newnode.postinit( + [self.visit(child, newnode) for child in node.body], + [self.visit(n, newnode) for n in node.finalbody], + ) + return newnode + + def visit_tuple(self, node, parent): + """visit a Tuple node by returning a fresh instance of it""" + context = self._get_context(node) + newnode = nodes.Tuple( + ctx=context, lineno=node.lineno, col_offset=node.col_offset, parent=parent + ) + newnode.postinit([self.visit(child, newnode) for child in node.elts]) + return newnode + + def visit_unaryop(self, node, parent): + """visit a UnaryOp node by returning a fresh instance of it""" + newnode = nodes.UnaryOp( + self._parser_module.unary_op_classes[node.op.__class__], + node.lineno, + node.col_offset, + parent, + ) + newnode.postinit(self.visit(node.operand, newnode)) + return newnode + + def visit_while(self, node, parent): + """visit a While node by returning a fresh instance of it""" + newnode = nodes.While(node.lineno, node.col_offset, parent) + newnode.postinit( + self.visit(node.test, newnode), + [self.visit(child, newnode) for child in node.body], + [self.visit(child, newnode) for child in node.orelse], + ) + return newnode + + def _visit_with(self, cls, node, parent): + newnode = cls(node.lineno, node.col_offset, parent) + + def visit_child(child): + expr = self.visit(child.context_expr, newnode) + var = _visit_or_none(child, "optional_vars", self, newnode) + return expr, var + + type_annotation = self.check_type_comment(node, parent=newnode) + newnode.postinit( + items=[visit_child(child) for child in node.items], + body=[self.visit(child, newnode) for child in node.body], + type_annotation=type_annotation, + ) + return newnode + + def visit_with(self, node, parent): + return self._visit_with(nodes.With, node, parent) + + def visit_yield(self, node, parent): + """visit a Yield node by returning a fresh instance of it""" + newnode = nodes.Yield(node.lineno, node.col_offset, parent) + if node.value is not None: + newnode.postinit(self.visit(node.value, newnode)) + return newnode + + def visit_yieldfrom(self, node, parent): + newnode = nodes.YieldFrom(node.lineno, node.col_offset, parent) + if node.value is not None: + newnode.postinit(self.visit(node.value, newnode)) + return newnode diff --git a/env/lib/python3.8/site-packages/astroid/scoped_nodes.py b/env/lib/python3.8/site-packages/astroid/scoped_nodes.py new file mode 100644 index 000000000..8561e745a --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/scoped_nodes.py @@ -0,0 +1,2927 @@ +# -*- coding: utf-8 -*- +# Copyright (c) 2006-2014 LOGILAB S.A. (Paris, FRANCE) +# Copyright (c) 2010 Daniel Harding +# Copyright (c) 2011, 2013-2015 Google, Inc. +# Copyright (c) 2013-2020 Claudiu Popa +# Copyright (c) 2013 Phil Schaf +# Copyright (c) 2014 Eevee (Alex Munroe) +# Copyright (c) 2015-2016 Florian Bruhin +# Copyright (c) 2015-2016 Ceridwen +# Copyright (c) 2015 Rene Zhang +# Copyright (c) 2015 Philip Lorenz +# Copyright (c) 2016-2017 Derek Gustafson +# Copyright (c) 2017-2018 Bryce Guinta +# Copyright (c) 2017-2018 Ashley Whetter +# Copyright (c) 2017 Łukasz Rogalski +# Copyright (c) 2017 David Euresti +# Copyright (c) 2018-2019 Nick Drozd +# Copyright (c) 2018 Ville Skyttä +# Copyright (c) 2018 Anthony Sottile +# Copyright (c) 2018 HoverHell +# Copyright (c) 2019 Hugo van Kemenade +# Copyright (c) 2019 Peter de Blanc + +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER + + +""" +This module contains the classes for "scoped" node, i.e. which are opening a +new local scope in the language definition : Module, ClassDef, FunctionDef (and +Lambda, GeneratorExp, DictComp and SetComp to some extent). +""" + +import builtins +import sys +import io +import itertools +from typing import Optional, List + +from astroid import bases +from astroid import context as contextmod +from astroid import exceptions +from astroid import decorators as decorators_mod +from astroid.interpreter import objectmodel +from astroid.interpreter import dunder_lookup +from astroid import manager +from astroid import mixins +from astroid import node_classes +from astroid import util + + +BUILTINS = builtins.__name__ +ITER_METHODS = ("__iter__", "__getitem__") +EXCEPTION_BASE_CLASSES = frozenset({"Exception", "BaseException"}) +objects = util.lazy_import("objects") +BUILTIN_DESCRIPTORS = frozenset( + {"classmethod", "staticmethod", "builtins.classmethod", "builtins.staticmethod"} +) + + +def _c3_merge(sequences, cls, context): + """Merges MROs in *sequences* to a single MRO using the C3 algorithm. + + Adapted from http://www.python.org/download/releases/2.3/mro/. + + """ + result = [] + while True: + sequences = [s for s in sequences if s] # purge empty sequences + if not sequences: + return result + for s1 in sequences: # find merge candidates among seq heads + candidate = s1[0] + for s2 in sequences: + if candidate in s2[1:]: + candidate = None + break # reject the current head, it appears later + else: + break + if not candidate: + # Show all the remaining bases, which were considered as + # candidates for the next mro sequence. + raise exceptions.InconsistentMroError( + message="Cannot create a consistent method resolution order " + "for MROs {mros} of class {cls!r}.", + mros=sequences, + cls=cls, + context=context, + ) + + result.append(candidate) + # remove the chosen candidate + for seq in sequences: + if seq[0] == candidate: + del seq[0] + return None + + +def clean_duplicates_mro(sequences, cls, context): + for sequence in sequences: + names = [ + (node.lineno, node.qname()) if node.name else None for node in sequence + ] + last_index = dict(map(reversed, enumerate(names))) + if names and names[0] is not None and last_index[names[0]] != 0: + raise exceptions.DuplicateBasesError( + message="Duplicates found in MROs {mros} for {cls!r}.", + mros=sequences, + cls=cls, + context=context, + ) + yield [ + node + for i, (node, name) in enumerate(zip(sequence, names)) + if name is None or last_index[name] == i + ] + + +def function_to_method(n, klass): + if isinstance(n, FunctionDef): + if n.type == "classmethod": + return bases.BoundMethod(n, klass) + if n.type == "property": + return n + if n.type != "staticmethod": + return bases.UnboundMethod(n) + return n + + +MANAGER = manager.AstroidManager() + + +def builtin_lookup(name): + """lookup a name into the builtin module + return the list of matching statements and the astroid for the builtin + module + """ + builtin_astroid = MANAGER.ast_from_module(builtins) + if name == "__dict__": + return builtin_astroid, () + try: + stmts = builtin_astroid.locals[name] + except KeyError: + stmts = () + return builtin_astroid, stmts + + +# TODO move this Mixin to mixins.py; problem: 'FunctionDef' in _scope_lookup +class LocalsDictNodeNG(node_classes.LookupMixIn, node_classes.NodeNG): + """ this class provides locals handling common to Module, FunctionDef + and ClassDef nodes, including a dict like interface for direct access + to locals information + """ + + # attributes below are set by the builder module or by raw factories + + locals = {} + """A map of the name of a local variable to the node defining the local. + + :type: dict(str, NodeNG) + """ + + def qname(self): + """Get the 'qualified' name of the node. + + For example: module.name, module.class.name ... + + :returns: The qualified name. + :rtype: str + """ + # pylint: disable=no-member; github.com/pycqa/astroid/issues/278 + if self.parent is None: + return self.name + return "%s.%s" % (self.parent.frame().qname(), self.name) + + def frame(self): + """The first parent frame node. + + A frame node is a :class:`Module`, :class:`FunctionDef`, + or :class:`ClassDef`. + + :returns: The first parent frame node. + :rtype: Module or FunctionDef or ClassDef + """ + return self + + def scope(self): + """The first parent node defining a new scope. + + :returns: The first parent scope node. + :rtype: Module or FunctionDef or ClassDef or Lambda or GenExpr + """ + return self + + def _scope_lookup(self, node, name, offset=0): + """XXX method for interfacing the scope lookup""" + try: + stmts = node._filter_stmts(self.locals[name], self, offset) + except KeyError: + stmts = () + if stmts: + return self, stmts + if self.parent: # i.e. not Module + # nested scope: if parent scope is a function, that's fine + # else jump to the module + pscope = self.parent.scope() + if not pscope.is_function: + pscope = pscope.root() + return pscope.scope_lookup(node, name) + return builtin_lookup(name) # Module + + def set_local(self, name, stmt): + """Define that the given name is declared in the given statement node. + + .. seealso:: :meth:`scope` + + :param name: The name that is being defined. + :type name: str + + :param stmt: The statement that defines the given name. + :type stmt: NodeNG + """ + # assert not stmt in self.locals.get(name, ()), (self, stmt) + self.locals.setdefault(name, []).append(stmt) + + __setitem__ = set_local + + def _append_node(self, child): + """append a child, linking it in the tree""" + # pylint: disable=no-member; depending by the class + # which uses the current class as a mixin or base class. + # It's rewritten in 2.0, so it makes no sense for now + # to spend development time on it. + self.body.append(child) + child.parent = self + + def add_local_node(self, child_node, name=None): + """Append a child that should alter the locals of this scope node. + + :param child_node: The child node that will alter locals. + :type child_node: NodeNG + + :param name: The name of the local that will be altered by + the given child node. + :type name: str or None + """ + if name != "__class__": + # add __class__ node as a child will cause infinite recursion later! + self._append_node(child_node) + self.set_local(name or child_node.name, child_node) + + def __getitem__(self, item): + """The first node the defines the given local. + + :param item: The name of the locally defined object. + :type item: str + + :raises KeyError: If the name is not defined. + """ + return self.locals[item][0] + + def __iter__(self): + """Iterate over the names of locals defined in this scoped node. + + :returns: The names of the defined locals. + :rtype: iterable(str) + """ + return iter(self.keys()) + + def keys(self): + """The names of locals defined in this scoped node. + + :returns: The names of the defined locals. + :rtype: list(str) + """ + return list(self.locals.keys()) + + def values(self): + """The nodes that define the locals in this scoped node. + + :returns: The nodes that define locals. + :rtype: list(NodeNG) + """ + return [self[key] for key in self.keys()] + + def items(self): + """Get the names of the locals and the node that defines the local. + + :returns: The names of locals and their associated node. + :rtype: list(tuple(str, NodeNG)) + """ + return list(zip(self.keys(), self.values())) + + def __contains__(self, name): + """Check if a local is defined in this scope. + + :param name: The name of the local to check for. + :type name: str + + :returns: True if this node has a local of the given name, + False otherwise. + :rtype: bool + """ + return name in self.locals + + +class Module(LocalsDictNodeNG): + """Class representing an :class:`ast.Module` node. + + >>> node = astroid.extract_node('import astroid') + >>> node + + >>> node.parent + + """ + + _astroid_fields = ("body",) + + fromlineno = 0 + """The first line that this node appears on in the source code. + + :type: int or None + """ + lineno = 0 + """The line that this node appears on in the source code. + + :type: int or None + """ + + # attributes below are set by the builder module or by raw factories + + file = None + """The path to the file that this ast has been extracted from. + + This will be ``None`` when the representation has been built from a + built-in module. + + :type: str or None + """ + file_bytes = None + """The string/bytes that this ast was built from. + + :type: str or bytes or None + """ + file_encoding = None + """The encoding of the source file. + + This is used to get unicode out of a source file. + Python 2 only. + + :type: str or None + """ + name = None + """The name of the module. + + :type: str or None + """ + pure_python = None + """Whether the ast was built from source. + + :type: bool or None + """ + package = None + """Whether the node represents a package or a module. + + :type: bool or None + """ + globals = None + """A map of the name of a global variable to the node defining the global. + + :type: dict(str, NodeNG) + """ + + # Future imports + future_imports = None + """The imports from ``__future__``. + + :type: set(str) or None + """ + special_attributes = objectmodel.ModuleModel() + """The names of special attributes that this module has. + + :type: objectmodel.ModuleModel + """ + + # names of module attributes available through the global scope + scope_attrs = {"__name__", "__doc__", "__file__", "__path__", "__package__"} + """The names of module attributes available through the global scope. + + :type: str(str) + """ + + _other_fields = ( + "name", + "doc", + "file", + "path", + "package", + "pure_python", + "future_imports", + ) + _other_other_fields = ("locals", "globals") + + def __init__( + self, + name, + doc, + file=None, + path: Optional[List[str]] = None, + package=None, + parent=None, + pure_python=True, + ): + """ + :param name: The name of the module. + :type name: str + + :param doc: The module docstring. + :type doc: str + + :param file: The path to the file that this ast has been extracted from. + :type file: str or None + + :param path: + :type path: Optional[List[str]] + + :param package: Whether the node represents a package or a module. + :type package: bool or None + + :param parent: The parent node in the syntax tree. + :type parent: NodeNG or None + + :param pure_python: Whether the ast was built from source. + :type pure_python: bool or None + """ + self.name = name + self.doc = doc + self.file = file + self.path = path + self.package = package + self.parent = parent + self.pure_python = pure_python + self.locals = self.globals = {} + """A map of the name of a local variable to the node defining the local. + + :type: dict(str, NodeNG) + """ + self.body = [] + """The contents of the module. + + :type: list(NodeNG) or None + """ + self.future_imports = set() + + # pylint: enable=redefined-builtin + + def postinit(self, body=None): + """Do some setup after initialisation. + + :param body: The contents of the module. + :type body: list(NodeNG) or None + """ + self.body = body + + def _get_stream(self): + if self.file_bytes is not None: + return io.BytesIO(self.file_bytes) + if self.file is not None: + stream = open(self.file, "rb") + return stream + return None + + def stream(self): + """Get a stream to the underlying file or bytes. + + :type: file or io.BytesIO or None + """ + return self._get_stream() + + def block_range(self, lineno): + """Get a range from where this node starts to where this node ends. + + :param lineno: Unused. + :type lineno: int + + :returns: The range of line numbers that this node belongs to. + :rtype: tuple(int, int) + """ + return self.fromlineno, self.tolineno + + def scope_lookup(self, node, name, offset=0): + """Lookup where the given variable is assigned. + + :param node: The node to look for assignments up to. + Any assignments after the given node are ignored. + :type node: NodeNG + + :param name: The name of the variable to find assignments for. + :type name: str + + :param offset: The line offset to filter statements up to. + :type offset: int + + :returns: This scope node and the list of assignments associated to the + given name according to the scope where it has been found (locals, + globals or builtin). + :rtype: tuple(str, list(NodeNG)) + """ + if name in self.scope_attrs and name not in self.locals: + try: + return self, self.getattr(name) + except exceptions.AttributeInferenceError: + return self, () + return self._scope_lookup(node, name, offset) + + def pytype(self): + """Get the name of the type that this node represents. + + :returns: The name of the type. + :rtype: str + """ + return "%s.module" % BUILTINS + + def display_type(self): + """A human readable type of this node. + + :returns: The type of this node. + :rtype: str + """ + return "Module" + + def getattr(self, name, context=None, ignore_locals=False): + if not name: + raise exceptions.AttributeInferenceError( + target=self, attribute=name, context=context + ) + + result = [] + name_in_locals = name in self.locals + + if name in self.special_attributes and not ignore_locals and not name_in_locals: + result = [self.special_attributes.lookup(name)] + elif not ignore_locals and name_in_locals: + result = self.locals[name] + elif self.package: + try: + result = [self.import_module(name, relative_only=True)] + except (exceptions.AstroidBuildingError, SyntaxError) as exc: + raise exceptions.AttributeInferenceError( + target=self, attribute=name, context=context + ) from exc + result = [n for n in result if not isinstance(n, node_classes.DelName)] + if result: + return result + raise exceptions.AttributeInferenceError( + target=self, attribute=name, context=context + ) + + def igetattr(self, name, context=None): + """Infer the possible values of the given variable. + + :param name: The name of the variable to infer. + :type name: str + + :returns: The inferred possible values. + :rtype: iterable(NodeNG) or None + """ + # set lookup name since this is necessary to infer on import nodes for + # instance + context = contextmod.copy_context(context) + context.lookupname = name + try: + return bases._infer_stmts(self.getattr(name, context), context, frame=self) + except exceptions.AttributeInferenceError as error: + raise exceptions.InferenceError( + error.message, target=self, attribute=name, context=context + ) from error + + def fully_defined(self): + """Check if this module has been build from a .py file. + + If so, the module contains a complete representation, + including the code. + + :returns: True if the module has been built from a .py file. + :rtype: bool + """ + return self.file is not None and self.file.endswith(".py") + + def statement(self): + """The first parent node, including self, marked as statement node. + + :returns: The first parent statement. + :rtype: NodeNG + """ + return self + + def previous_sibling(self): + """The previous sibling statement. + + :returns: The previous sibling statement node. + :rtype: NodeNG or None + """ + + def next_sibling(self): + """The next sibling statement node. + + :returns: The next sibling statement node. + :rtype: NodeNG or None + """ + + _absolute_import_activated = True + + def absolute_import_activated(self): + """Whether :pep:`328` absolute import behaviour has been enabled. + + :returns: True if :pep:`328` has been enabled, False otherwise. + :rtype: bool + """ + return self._absolute_import_activated + + def import_module(self, modname, relative_only=False, level=None): + """Get the ast for a given module as if imported from this module. + + :param modname: The name of the module to "import". + :type modname: str + + :param relative_only: Whether to only consider relative imports. + :type relative_only: bool + + :param level: The level of relative import. + :type level: int or None + + :returns: The imported module ast. + :rtype: NodeNG + """ + if relative_only and level is None: + level = 0 + absmodname = self.relative_to_absolute_name(modname, level) + + try: + return MANAGER.ast_from_module_name(absmodname) + except exceptions.AstroidBuildingError: + # we only want to import a sub module or package of this module, + # skip here + if relative_only: + raise + return MANAGER.ast_from_module_name(modname) + + def relative_to_absolute_name(self, modname, level): + """Get the absolute module name for a relative import. + + The relative import can be implicit or explicit. + + :param modname: The module name to convert. + :type modname: str + + :param level: The level of relative import. + :type level: int + + :returns: The absolute module name. + :rtype: str + + :raises TooManyLevelsError: When the relative import refers to a + module too far above this one. + """ + # XXX this returns non sens when called on an absolute import + # like 'pylint.checkers.astroid.utils' + # XXX doesn't return absolute name if self.name isn't absolute name + if self.absolute_import_activated() and level is None: + return modname + if level: + if self.package: + level = level - 1 + if level and self.name.count(".") < level: + raise exceptions.TooManyLevelsError(level=level, name=self.name) + + package_name = self.name.rsplit(".", level)[0] + elif self.package: + package_name = self.name + else: + package_name = self.name.rsplit(".", 1)[0] + + if package_name: + if not modname: + return package_name + return "%s.%s" % (package_name, modname) + return modname + + def wildcard_import_names(self): + """The list of imported names when this module is 'wildcard imported'. + + It doesn't include the '__builtins__' name which is added by the + current CPython implementation of wildcard imports. + + :returns: The list of imported names. + :rtype: list(str) + """ + # We separate the different steps of lookup in try/excepts + # to avoid catching too many Exceptions + default = [name for name in self.keys() if not name.startswith("_")] + try: + all_values = self["__all__"] + except KeyError: + return default + + try: + explicit = next(all_values.assigned_stmts()) + except exceptions.InferenceError: + return default + except AttributeError: + # not an assignment node + # XXX infer? + return default + + # Try our best to detect the exported name. + inferred = [] + try: + explicit = next(explicit.infer()) + except exceptions.InferenceError: + return default + if not isinstance(explicit, (node_classes.Tuple, node_classes.List)): + return default + + str_const = lambda node: ( + isinstance(node, node_classes.Const) and isinstance(node.value, str) + ) + for node in explicit.elts: + if str_const(node): + inferred.append(node.value) + else: + try: + inferred_node = next(node.infer()) + except exceptions.InferenceError: + continue + if str_const(inferred_node): + inferred.append(inferred_node.value) + return inferred + + def public_names(self): + """The list of the names that are publicly available in this module. + + :returns: The list of publc names. + :rtype: list(str) + """ + return [name for name in self.keys() if not name.startswith("_")] + + def bool_value(self, context=None): + """Determine the boolean value of this node. + + :returns: The boolean value of this node. + For a :class:`Module` this is always ``True``. + :rtype: bool + """ + return True + + def get_children(self): + yield from self.body + + +class ComprehensionScope(LocalsDictNodeNG): + """Scoping for different types of comprehensions.""" + + def frame(self): + """The first parent frame node. + + A frame node is a :class:`Module`, :class:`FunctionDef`, + or :class:`ClassDef`. + + :returns: The first parent frame node. + :rtype: Module or FunctionDef or ClassDef + """ + return self.parent.frame() + + scope_lookup = LocalsDictNodeNG._scope_lookup + + +class GeneratorExp(ComprehensionScope): + """Class representing an :class:`ast.GeneratorExp` node. + + >>> node = astroid.extract_node('(thing for thing in things if thing)') + >>> node + + """ + + _astroid_fields = ("elt", "generators") + _other_other_fields = ("locals",) + elt = None + """The element that forms the output of the expression. + + :type: NodeNG or None + """ + generators = None + """The generators that are looped through. + + :type: list(Comprehension) or None + """ + + def __init__(self, lineno=None, col_offset=None, parent=None): + """ + :param lineno: The line that this node appears on in the source code. + :type lineno: int or None + + :param col_offset: The column that this node appears on in the + source code. + :type col_offset: int or None + + :param parent: The parent node in the syntax tree. + :type parent: NodeNG or None + """ + self.locals = {} + """A map of the name of a local variable to the node defining the local. + + :type: dict(str, NodeNG) + """ + + super().__init__(lineno, col_offset, parent) + + def postinit(self, elt=None, generators=None): + """Do some setup after initialisation. + + :param elt: The element that forms the output of the expression. + :type elt: NodeNG or None + + :param generators: The generators that are looped through. + :type generators: list(Comprehension) or None + """ + self.elt = elt + if generators is None: + self.generators = [] + else: + self.generators = generators + + def bool_value(self, context=None): + """Determine the boolean value of this node. + + :returns: The boolean value of this node. + For a :class:`GeneratorExp` this is always ``True``. + :rtype: bool + """ + return True + + def get_children(self): + yield self.elt + + yield from self.generators + + +class DictComp(ComprehensionScope): + """Class representing an :class:`ast.DictComp` node. + + >>> node = astroid.extract_node('{k:v for k, v in things if k > v}') + >>> node + + """ + + _astroid_fields = ("key", "value", "generators") + _other_other_fields = ("locals",) + key = None + """What produces the keys. + + :type: NodeNG or None + """ + value = None + """What produces the values. + + :type: NodeNG or None + """ + generators = None + """The generators that are looped through. + + :type: list(Comprehension) or None + """ + + def __init__(self, lineno=None, col_offset=None, parent=None): + """ + :param lineno: The line that this node appears on in the source code. + :type lineno: int or None + + :param col_offset: The column that this node appears on in the + source code. + :type col_offset: int or None + + :param parent: The parent node in the syntax tree. + :type parent: NodeNG or None + """ + self.locals = {} + """A map of the name of a local variable to the node defining the local. + + :type: dict(str, NodeNG) + """ + + super().__init__(lineno, col_offset, parent) + + def postinit(self, key=None, value=None, generators=None): + """Do some setup after initialisation. + + :param key: What produces the keys. + :type key: NodeNG or None + + :param value: What produces the values. + :type value: NodeNG or None + + :param generators: The generators that are looped through. + :type generators: list(Comprehension) or None + """ + self.key = key + self.value = value + if generators is None: + self.generators = [] + else: + self.generators = generators + + def bool_value(self, context=None): + """Determine the boolean value of this node. + + :returns: The boolean value of this node. + For a :class:`DictComp` this is always :class:`Uninferable`. + :rtype: Uninferable + """ + return util.Uninferable + + def get_children(self): + yield self.key + yield self.value + + yield from self.generators + + +class SetComp(ComprehensionScope): + """Class representing an :class:`ast.SetComp` node. + + >>> node = astroid.extract_node('{thing for thing in things if thing}') + >>> node + + """ + + _astroid_fields = ("elt", "generators") + _other_other_fields = ("locals",) + elt = None + """The element that forms the output of the expression. + + :type: NodeNG or None + """ + generators = None + """The generators that are looped through. + + :type: list(Comprehension) or None + """ + + def __init__(self, lineno=None, col_offset=None, parent=None): + """ + :param lineno: The line that this node appears on in the source code. + :type lineno: int or None + + :param col_offset: The column that this node appears on in the + source code. + :type col_offset: int or None + + :param parent: The parent node in the syntax tree. + :type parent: NodeNG or None + """ + self.locals = {} + """A map of the name of a local variable to the node defining the local. + + :type: dict(str, NodeNG) + """ + + super().__init__(lineno, col_offset, parent) + + def postinit(self, elt=None, generators=None): + """Do some setup after initialisation. + + :param elt: The element that forms the output of the expression. + :type elt: NodeNG or None + + :param generators: The generators that are looped through. + :type generators: list(Comprehension) or None + """ + self.elt = elt + if generators is None: + self.generators = [] + else: + self.generators = generators + + def bool_value(self, context=None): + """Determine the boolean value of this node. + + :returns: The boolean value of this node. + For a :class:`SetComp` this is always :class:`Uninferable`. + :rtype: Uninferable + """ + return util.Uninferable + + def get_children(self): + yield self.elt + + yield from self.generators + + +class _ListComp(node_classes.NodeNG): + """Class representing an :class:`ast.ListComp` node. + + >>> node = astroid.extract_node('[thing for thing in things if thing]') + >>> node + + """ + + _astroid_fields = ("elt", "generators") + elt = None + """The element that forms the output of the expression. + + :type: NodeNG or None + """ + generators = None + """The generators that are looped through. + + :type: list(Comprehension) or None + """ + + def postinit(self, elt=None, generators=None): + """Do some setup after initialisation. + + :param elt: The element that forms the output of the expression. + :type elt: NodeNG or None + + :param generators: The generators that are looped through. + :type generators: list(Comprehension) or None + """ + self.elt = elt + self.generators = generators + + def bool_value(self, context=None): + """Determine the boolean value of this node. + + :returns: The boolean value of this node. + For a :class:`ListComp` this is always :class:`Uninferable`. + :rtype: Uninferable + """ + return util.Uninferable + + def get_children(self): + yield self.elt + + yield from self.generators + + +class ListComp(_ListComp, ComprehensionScope): + """Class representing an :class:`ast.ListComp` node. + + >>> node = astroid.extract_node('[thing for thing in things if thing]') + >>> node + + """ + + _other_other_fields = ("locals",) + + def __init__(self, lineno=None, col_offset=None, parent=None): + self.locals = {} + """A map of the name of a local variable to the node defining it. + + :type: dict(str, NodeNG) + """ + + super().__init__(lineno, col_offset, parent) + + +def _infer_decorator_callchain(node): + """Detect decorator call chaining and see if the end result is a + static or a classmethod. + """ + if not isinstance(node, FunctionDef): + return None + if not node.parent: + return None + try: + result = next(node.infer_call_result(node.parent)) + except exceptions.InferenceError: + return None + if isinstance(result, bases.Instance): + result = result._proxied + if isinstance(result, ClassDef): + if result.is_subtype_of("%s.classmethod" % BUILTINS): + return "classmethod" + if result.is_subtype_of("%s.staticmethod" % BUILTINS): + return "staticmethod" + if isinstance(result, FunctionDef): + if not result.decorators: + return None + # Determine if this function is decorated with one of the builtin descriptors we want. + for decorator in result.decorators.nodes: + if isinstance(decorator, node_classes.Name): + if decorator.name in BUILTIN_DESCRIPTORS: + return decorator.name + if ( + isinstance(decorator, node_classes.Attribute) + and isinstance(decorator.expr, node_classes.Name) + and decorator.expr.name == BUILTINS + and decorator.attrname in BUILTIN_DESCRIPTORS + ): + return decorator.attrname + return None + + +class Lambda(mixins.FilterStmtsMixin, LocalsDictNodeNG): + """Class representing an :class:`ast.Lambda` node. + + >>> node = astroid.extract_node('lambda arg: arg + 1') + >>> node + l.1 at 0x7f23b2e41518> + """ + + _astroid_fields = ("args", "body") + _other_other_fields = ("locals",) + name = "" + is_lambda = True + + def implicit_parameters(self): + return 0 + + # function's type, 'function' | 'method' | 'staticmethod' | 'classmethod' + @property + def type(self): + """Whether this is a method or function. + + :returns: 'method' if this is a method, 'function' otherwise. + :rtype: str + """ + # pylint: disable=no-member + if self.args.arguments and self.args.arguments[0].name == "self": + if isinstance(self.parent.scope(), ClassDef): + return "method" + return "function" + + def __init__(self, lineno=None, col_offset=None, parent=None): + """ + :param lineno: The line that this node appears on in the source code. + :type lineno: int or None + + :param col_offset: The column that this node appears on in the + source code. + :type col_offset: int or None + + :param parent: The parent node in the syntax tree. + :type parent: NodeNG or None + """ + self.locals = {} + """A map of the name of a local variable to the node defining it. + + :type: dict(str, NodeNG) + """ + + self.args = [] + """The arguments that the function takes. + + :type: Arguments or list + """ + + self.body = [] + """The contents of the function body. + + :type: list(NodeNG) + """ + + super().__init__(lineno, col_offset, parent) + + def postinit(self, args, body): + """Do some setup after initialisation. + + :param args: The arguments that the function takes. + :type args: Arguments + + :param body: The contents of the function body. + :type body: list(NodeNG) + """ + self.args = args + self.body = body + + def pytype(self): + """Get the name of the type that this node represents. + + :returns: The name of the type. + :rtype: str + """ + if "method" in self.type: + return "%s.instancemethod" % BUILTINS + return "%s.function" % BUILTINS + + def display_type(self): + """A human readable type of this node. + + :returns: The type of this node. + :rtype: str + """ + if "method" in self.type: + return "Method" + return "Function" + + def callable(self): + """Whether this node defines something that is callable. + + :returns: True if this defines something that is callable, + False otherwise. + For a :class:`Lambda` this is always ``True``. + :rtype: bool + """ + return True + + def argnames(self): + """Get the names of each of the arguments. + + :returns: The names of the arguments. + :rtype: list(str) + """ + # pylint: disable=no-member; github.com/pycqa/astroid/issues/291 + # args is in fact redefined later on by postinit. Can't be changed + # to None due to a strong interaction between Lambda and FunctionDef. + + if self.args.arguments: # maybe None with builtin functions + names = _rec_get_names(self.args.arguments) + else: + names = [] + if self.args.vararg: + names.append(self.args.vararg) + if self.args.kwarg: + names.append(self.args.kwarg) + return names + + def infer_call_result(self, caller, context=None): + """Infer what the function returns when called. + + :param caller: Unused + :type caller: object + """ + # pylint: disable=no-member; github.com/pycqa/astroid/issues/291 + # args is in fact redefined later on by postinit. Can't be changed + # to None due to a strong interaction between Lambda and FunctionDef. + return self.body.infer(context) + + def scope_lookup(self, node, name, offset=0): + """Lookup where the given names is assigned. + + :param node: The node to look for assignments up to. + Any assignments after the given node are ignored. + :type node: NodeNG + + :param name: The name to find assignments for. + :type name: str + + :param offset: The line offset to filter statements up to. + :type offset: int + + :returns: This scope node and the list of assignments associated to the + given name according to the scope where it has been found (locals, + globals or builtin). + :rtype: tuple(str, list(NodeNG)) + """ + # pylint: disable=no-member; github.com/pycqa/astroid/issues/291 + # args is in fact redefined later on by postinit. Can't be changed + # to None due to a strong interaction between Lambda and FunctionDef. + + if node in self.args.defaults or node in self.args.kw_defaults: + frame = self.parent.frame() + # line offset to avoid that def func(f=func) resolve the default + # value to the defined function + offset = -1 + else: + # check this is not used in function decorators + frame = self + return frame._scope_lookup(node, name, offset) + + def bool_value(self, context=None): + """Determine the boolean value of this node. + + :returns: The boolean value of this node. + For a :class:`Lambda` this is always ``True``. + :rtype: bool + """ + return True + + def get_children(self): + yield self.args + yield self.body + + +class FunctionDef(mixins.MultiLineBlockMixin, node_classes.Statement, Lambda): + """Class representing an :class:`ast.FunctionDef`. + + >>> node = astroid.extract_node(''' + ... def my_func(arg): + ... return arg + 1 + ... ''') + >>> node + + """ + + _astroid_fields = ("decorators", "args", "returns", "body") + _multi_line_block_fields = ("body",) + returns = None + decorators = None + """The decorators that are applied to this method or function. + + :type: Decorators or None + """ + special_attributes = objectmodel.FunctionModel() + """The names of special attributes that this function has. + + :type: objectmodel.FunctionModel + """ + is_function = True + """Whether this node indicates a function. + + For a :class:`FunctionDef` this is always ``True``. + + :type: bool + """ + type_annotation = None + """If present, this will contain the type annotation passed by a type comment + + :type: NodeNG or None + """ + type_comment_args = None + """ + If present, this will contain the type annotation for arguments + passed by a type comment + """ + type_comment_returns = None + """If present, this will contain the return type annotation, passed by a type comment""" + # attributes below are set by the builder module or by raw factories + _other_fields = ("name", "doc") + _other_other_fields = ( + "locals", + "_type", + "type_comment_returns", + "type_comment_args", + ) + _type = None + + def __init__(self, name=None, doc=None, lineno=None, col_offset=None, parent=None): + """ + :param name: The name of the function. + :type name: str or None + + :param doc: The function's docstring. + :type doc: str or None + + :param lineno: The line that this node appears on in the source code. + :type lineno: int or None + + :param col_offset: The column that this node appears on in the + source code. + :type col_offset: int or None + + :param parent: The parent node in the syntax tree. + :type parent: NodeNG or None + """ + self.name = name + """The name of the function. + + :type name: str or None + """ + + self.doc = doc + """The function's docstring. + + :type doc: str or None + """ + + self.instance_attrs = {} + super().__init__(lineno, col_offset, parent) + if parent: + frame = parent.frame() + frame.set_local(name, self) + + # pylint: disable=arguments-differ; different than Lambdas + def postinit( + self, + args, + body, + decorators=None, + returns=None, + type_comment_returns=None, + type_comment_args=None, + ): + """Do some setup after initialisation. + + :param args: The arguments that the function takes. + :type args: Arguments or list + + :param body: The contents of the function body. + :type body: list(NodeNG) + + :param decorators: The decorators that are applied to this + method or function. + :type decorators: Decorators or None + :params type_comment_returns: + The return type annotation passed via a type comment. + :params type_comment_args: + The args type annotation passed via a type comment. + """ + self.args = args + self.body = body + self.decorators = decorators + self.returns = returns + self.type_comment_returns = type_comment_returns + self.type_comment_args = type_comment_args + + @decorators_mod.cachedproperty + def extra_decorators(self): + """The extra decorators that this function can have. + + Additional decorators are considered when they are used as + assignments, as in ``method = staticmethod(method)``. + The property will return all the callables that are used for + decoration. + + :type: list(NodeNG) + """ + frame = self.parent.frame() + if not isinstance(frame, ClassDef): + return [] + + decorators = [] + for assign in frame._get_assign_nodes(): + if isinstance(assign.value, node_classes.Call) and isinstance( + assign.value.func, node_classes.Name + ): + for assign_node in assign.targets: + if not isinstance(assign_node, node_classes.AssignName): + # Support only `name = callable(name)` + continue + + if assign_node.name != self.name: + # Interested only in the assignment nodes that + # decorates the current method. + continue + try: + meth = frame[self.name] + except KeyError: + continue + else: + # Must be a function and in the same frame as the + # original method. + if ( + isinstance(meth, FunctionDef) + and assign_node.frame() == frame + ): + decorators.append(assign.value) + return decorators + + @decorators_mod.cachedproperty + def type( + self + ): # pylint: disable=invalid-overridden-method,too-many-return-statements + """The function type for this node. + + Possible values are: method, function, staticmethod, classmethod. + + :type: str + """ + for decorator in self.extra_decorators: + if decorator.func.name in BUILTIN_DESCRIPTORS: + return decorator.func.name + + frame = self.parent.frame() + type_name = "function" + if isinstance(frame, ClassDef): + if self.name == "__new__": + return "classmethod" + if sys.version_info >= (3, 6) and self.name == "__init_subclass__": + return "classmethod" + + type_name = "method" + + if not self.decorators: + return type_name + + for node in self.decorators.nodes: + if isinstance(node, node_classes.Name): + if node.name in BUILTIN_DESCRIPTORS: + return node.name + if ( + isinstance(node, node_classes.Attribute) + and isinstance(node.expr, node_classes.Name) + and node.expr.name == BUILTINS + and node.attrname in BUILTIN_DESCRIPTORS + ): + return node.attrname + + if isinstance(node, node_classes.Call): + # Handle the following case: + # @some_decorator(arg1, arg2) + # def func(...) + # + try: + current = next(node.func.infer()) + except exceptions.InferenceError: + continue + _type = _infer_decorator_callchain(current) + if _type is not None: + return _type + + try: + for inferred in node.infer(): + # Check to see if this returns a static or a class method. + _type = _infer_decorator_callchain(inferred) + if _type is not None: + return _type + + if not isinstance(inferred, ClassDef): + continue + for ancestor in inferred.ancestors(): + if not isinstance(ancestor, ClassDef): + continue + if ancestor.is_subtype_of("%s.classmethod" % BUILTINS): + return "classmethod" + if ancestor.is_subtype_of("%s.staticmethod" % BUILTINS): + return "staticmethod" + except exceptions.InferenceError: + pass + return type_name + + @decorators_mod.cachedproperty + def fromlineno(self): + """The first line that this node appears on in the source code. + + :type: int or None + """ + # lineno is the line number of the first decorator, we want the def + # statement lineno + lineno = self.lineno + if self.decorators is not None: + lineno += sum( + node.tolineno - node.lineno + 1 for node in self.decorators.nodes + ) + + return lineno + + @decorators_mod.cachedproperty + def blockstart_tolineno(self): + """The line on which the beginning of this block ends. + + :type: int + """ + return self.args.tolineno + + def block_range(self, lineno): + """Get a range from the given line number to where this node ends. + + :param lineno: Unused. + :type lineno: int + + :returns: The range of line numbers that this node belongs to, + :rtype: tuple(int, int) + """ + return self.fromlineno, self.tolineno + + def getattr(self, name, context=None): + """this method doesn't look in the instance_attrs dictionary since it's + done by an Instance proxy at inference time. + """ + if not name: + raise exceptions.AttributeInferenceError( + target=self, attribute=name, context=context + ) + + found_attrs = [] + if name in self.instance_attrs: + found_attrs = self.instance_attrs[name] + if name in self.special_attributes: + found_attrs.append(self.special_attributes.lookup(name)) + if found_attrs: + return found_attrs + raise exceptions.AttributeInferenceError(target=self, attribute=name) + + def igetattr(self, name, context=None): + """Inferred getattr, which returns an iterator of inferred statements.""" + try: + return bases._infer_stmts(self.getattr(name, context), context, frame=self) + except exceptions.AttributeInferenceError as error: + raise exceptions.InferenceError( + error.message, target=self, attribute=name, context=context + ) from error + + def is_method(self): + """Check if this function node represents a method. + + :returns: True if this is a method, False otherwise. + :rtype: bool + """ + # check we are defined in a ClassDef, because this is usually expected + # (e.g. pylint...) when is_method() return True + return self.type != "function" and isinstance(self.parent.frame(), ClassDef) + + @decorators_mod.cached + def decoratornames(self, context=None): + """Get the qualified names of each of the decorators on this function. + + :param context: + An inference context that can be passed to inference functions + :returns: The names of the decorators. + :rtype: set(str) + """ + result = set() + decoratornodes = [] + if self.decorators is not None: + decoratornodes += self.decorators.nodes + decoratornodes += self.extra_decorators + for decnode in decoratornodes: + try: + for infnode in decnode.infer(context=context): + result.add(infnode.qname()) + except exceptions.InferenceError: + continue + return result + + def is_bound(self): + """Check if the function is bound to an instance or class. + + :returns: True if the function is bound to an instance or class, + False otherwise. + :rtype: bool + """ + return self.type == "classmethod" + + def is_abstract(self, pass_is_abstract=True): + """Check if the method is abstract. + + A method is considered abstract if any of the following is true: + * The only statement is 'raise NotImplementedError' + * The only statement is 'pass' and pass_is_abstract is True + * The method is annotated with abc.astractproperty/abc.abstractmethod + + :returns: True if the method is abstract, False otherwise. + :rtype: bool + """ + if self.decorators: + for node in self.decorators.nodes: + try: + inferred = next(node.infer()) + except exceptions.InferenceError: + continue + if inferred and inferred.qname() in ( + "abc.abstractproperty", + "abc.abstractmethod", + ): + return True + + for child_node in self.body: + if isinstance(child_node, node_classes.Raise): + if child_node.raises_not_implemented(): + return True + return pass_is_abstract and isinstance(child_node, node_classes.Pass) + # empty function is the same as function with a single "pass" statement + if pass_is_abstract: + return True + + def is_generator(self): + """Check if this is a generator function. + + :returns: True is this is a generator function, False otherwise. + :rtype: bool + """ + return bool(next(self._get_yield_nodes_skip_lambdas(), False)) + + def infer_call_result(self, caller=None, context=None): + """Infer what the function returns when called. + + :returns: What the function returns. + :rtype: iterable(NodeNG or Uninferable) or None + """ + if self.is_generator(): + if isinstance(self, AsyncFunctionDef): + generator_cls = bases.AsyncGenerator + else: + generator_cls = bases.Generator + result = generator_cls(self) + yield result + return + # This is really a gigantic hack to work around metaclass generators + # that return transient class-generating functions. Pylint's AST structure + # cannot handle a base class object that is only used for calling __new__, + # but does not contribute to the inheritance structure itself. We inject + # a fake class into the hierarchy here for several well-known metaclass + # generators, and filter it out later. + if ( + self.name == "with_metaclass" + and len(self.args.args) == 1 + and self.args.vararg is not None + ): + metaclass = next(caller.args[0].infer(context)) + if isinstance(metaclass, ClassDef): + class_bases = [next(arg.infer(context)) for arg in caller.args[1:]] + new_class = ClassDef(name="temporary_class") + new_class.hide = True + new_class.parent = self + new_class.postinit( + bases=[base for base in class_bases if base != util.Uninferable], + body=[], + decorators=[], + metaclass=metaclass, + ) + yield new_class + return + returns = self._get_return_nodes_skip_functions() + + first_return = next(returns, None) + if not first_return: + if self.body and isinstance(self.body[-1], node_classes.Assert): + yield node_classes.Const(None) + return + + raise exceptions.InferenceError( + "The function does not have any return statements" + ) + + for returnnode in itertools.chain((first_return,), returns): + if returnnode.value is None: + yield node_classes.Const(None) + else: + try: + yield from returnnode.value.infer(context) + except exceptions.InferenceError: + yield util.Uninferable + + def bool_value(self, context=None): + """Determine the boolean value of this node. + + :returns: The boolean value of this node. + For a :class:`FunctionDef` this is always ``True``. + :rtype: bool + """ + return True + + def get_children(self): + if self.decorators is not None: + yield self.decorators + + yield self.args + + if self.returns is not None: + yield self.returns + + yield from self.body + + def scope_lookup(self, node, name, offset=0): + """Lookup where the given name is assigned.""" + if name == "__class__": + # __class__ is an implicit closure reference created by the compiler + # if any methods in a class body refer to either __class__ or super. + # In our case, we want to be able to look it up in the current scope + # when `__class__` is being used. + frame = self.parent.frame() + if isinstance(frame, ClassDef): + return self, [frame] + return super().scope_lookup(node, name, offset) + + +class AsyncFunctionDef(FunctionDef): + """Class representing an :class:`ast.FunctionDef` node. + + A :class:`AsyncFunctionDef` is an asynchronous function + created with the `async` keyword. + + >>> node = astroid.extract_node(''' + async def func(things): + async for thing in things: + print(thing) + ''') + >>> node + + >>> node.body[0] + + """ + + +def _rec_get_names(args, names=None): + """return a list of all argument names""" + if names is None: + names = [] + for arg in args: + if isinstance(arg, node_classes.Tuple): + _rec_get_names(arg.elts, names) + else: + names.append(arg.name) + return names + + +def _is_metaclass(klass, seen=None): + """ Return if the given class can be + used as a metaclass. + """ + if klass.name == "type": + return True + if seen is None: + seen = set() + for base in klass.bases: + try: + for baseobj in base.infer(): + baseobj_name = baseobj.qname() + if baseobj_name in seen: + continue + + seen.add(baseobj_name) + if isinstance(baseobj, bases.Instance): + # not abstract + return False + if baseobj is util.Uninferable: + continue + if baseobj is klass: + continue + if not isinstance(baseobj, ClassDef): + continue + if baseobj._type == "metaclass": + return True + if _is_metaclass(baseobj, seen): + return True + except exceptions.InferenceError: + continue + return False + + +def _class_type(klass, ancestors=None): + """return a ClassDef node type to differ metaclass and exception + from 'regular' classes + """ + # XXX we have to store ancestors in case we have an ancestor loop + if klass._type is not None: + return klass._type + if _is_metaclass(klass): + klass._type = "metaclass" + elif klass.name.endswith("Exception"): + klass._type = "exception" + else: + if ancestors is None: + ancestors = set() + klass_name = klass.qname() + if klass_name in ancestors: + # XXX we are in loop ancestors, and have found no type + klass._type = "class" + return "class" + ancestors.add(klass_name) + for base in klass.ancestors(recurs=False): + name = _class_type(base, ancestors) + if name != "class": + if name == "metaclass" and not _is_metaclass(klass): + # don't propagate it if the current class + # can't be a metaclass + continue + klass._type = base.type + break + if klass._type is None: + klass._type = "class" + return klass._type + + +def get_wrapping_class(node): + """Get the class that wraps the given node. + + We consider that a class wraps a node if the class + is a parent for the said node. + + :returns: The class that wraps the given node + :rtype: ClassDef or None + """ + + klass = node.frame() + while klass is not None and not isinstance(klass, ClassDef): + if klass.parent is None: + klass = None + else: + klass = klass.parent.frame() + return klass + + +class ClassDef(mixins.FilterStmtsMixin, LocalsDictNodeNG, node_classes.Statement): + """Class representing an :class:`ast.ClassDef` node. + + >>> node = astroid.extract_node(''' + class Thing: + def my_meth(self, arg): + return arg + self.offset + ''') + >>> node + + """ + + # some of the attributes below are set by the builder module or + # by a raw factories + + # a dictionary of class instances attributes + _astroid_fields = ("decorators", "bases", "body") # name + + decorators = None + """The decorators that are applied to this class. + + :type: Decorators or None + """ + special_attributes = objectmodel.ClassModel() + """The names of special attributes that this class has. + + :type: objectmodel.ClassModel + """ + + _type = None + _metaclass_hack = False + hide = False + type = property( + _class_type, + doc=( + "The class type for this node.\n\n" + "Possible values are: class, metaclass, exception.\n\n" + ":type: str" + ), + ) + _other_fields = ("name", "doc") + _other_other_fields = ("locals", "_newstyle") + _newstyle = None + + def __init__(self, name=None, doc=None, lineno=None, col_offset=None, parent=None): + """ + :param name: The name of the class. + :type name: str or None + + :param doc: The function's docstring. + :type doc: str or None + + :param lineno: The line that this node appears on in the source code. + :type lineno: int or None + + :param col_offset: The column that this node appears on in the + source code. + :type col_offset: int or None + + :param parent: The parent node in the syntax tree. + :type parent: NodeNG or None + """ + self.instance_attrs = {} + self.locals = {} + """A map of the name of a local variable to the node defining it. + + :type: dict(str, NodeNG) + """ + + self.keywords = [] + """The keywords given to the class definition. + + This is usually for :pep:`3115` style metaclass declaration. + + :type: list(Keyword) or None + """ + + self.bases = [] + """What the class inherits from. + + :type: list(NodeNG) + """ + + self.body = [] + """The contents of the class body. + + :type: list(NodeNG) + """ + + self.name = name + """The name of the class. + + :type name: str or None + """ + + self.doc = doc + """The class' docstring. + + :type doc: str or None + """ + + super().__init__(lineno, col_offset, parent) + if parent is not None: + parent.frame().set_local(name, self) + + for local_name, node in self.implicit_locals(): + self.add_local_node(node, local_name) + + def implicit_parameters(self): + return 1 + + def implicit_locals(self): + """Get implicitly defined class definition locals. + + :returns: the the name and Const pair for each local + :rtype: tuple(tuple(str, node_classes.Const), ...) + """ + locals_ = (("__module__", self.special_attributes.attr___module__),) + # __qualname__ is defined in PEP3155 + locals_ += (("__qualname__", self.special_attributes.attr___qualname__),) + return locals_ + + # pylint: disable=redefined-outer-name + def postinit( + self, bases, body, decorators, newstyle=None, metaclass=None, keywords=None + ): + """Do some setup after initialisation. + + :param bases: What the class inherits from. + :type bases: list(NodeNG) + + :param body: The contents of the class body. + :type body: list(NodeNG) + + :param decorators: The decorators that are applied to this class. + :type decorators: Decorators or None + + :param newstyle: Whether this is a new style class or not. + :type newstyle: bool or None + + :param metaclass: The metaclass of this class. + :type metaclass: NodeNG or None + + :param keywords: The keywords given to the class definition. + :type keywords: list(Keyword) or None + """ + self.keywords = keywords + self.bases = bases + self.body = body + self.decorators = decorators + if newstyle is not None: + self._newstyle = newstyle + if metaclass is not None: + self._metaclass = metaclass + + def _newstyle_impl(self, context=None): + if context is None: + context = contextmod.InferenceContext() + if self._newstyle is not None: + return self._newstyle + for base in self.ancestors(recurs=False, context=context): + if base._newstyle_impl(context): + self._newstyle = True + break + klass = self.declared_metaclass() + # could be any callable, we'd need to infer the result of klass(name, + # bases, dict). punt if it's not a class node. + if klass is not None and isinstance(klass, ClassDef): + self._newstyle = klass._newstyle_impl(context) + if self._newstyle is None: + self._newstyle = False + return self._newstyle + + _newstyle = None + newstyle = property( + _newstyle_impl, + doc=("Whether this is a new style class or not\n\n" ":type: bool or None"), + ) + + @decorators_mod.cachedproperty + def blockstart_tolineno(self): + """The line on which the beginning of this block ends. + + :type: int + """ + if self.bases: + return self.bases[-1].tolineno + + return self.fromlineno + + def block_range(self, lineno): + """Get a range from the given line number to where this node ends. + + :param lineno: Unused. + :type lineno: int + + :returns: The range of line numbers that this node belongs to, + :rtype: tuple(int, int) + """ + return self.fromlineno, self.tolineno + + def pytype(self): + """Get the name of the type that this node represents. + + :returns: The name of the type. + :rtype: str + """ + if self.newstyle: + return "%s.type" % BUILTINS + return "%s.classobj" % BUILTINS + + def display_type(self): + """A human readable type of this node. + + :returns: The type of this node. + :rtype: str + """ + return "Class" + + def callable(self): + """Whether this node defines something that is callable. + + :returns: True if this defines something that is callable, + False otherwise. + For a :class:`ClassDef` this is always ``True``. + :rtype: bool + """ + return True + + def is_subtype_of(self, type_name, context=None): + """Whether this class is a subtype of the given type. + + :param type_name: The name of the type of check against. + :type type_name: str + + :returns: True if this class is a subtype of the given type, + False otherwise. + :rtype: bool + """ + if self.qname() == type_name: + return True + for anc in self.ancestors(context=context): + if anc.qname() == type_name: + return True + return False + + def _infer_type_call(self, caller, context): + name_node = next(caller.args[0].infer(context)) + if isinstance(name_node, node_classes.Const) and isinstance( + name_node.value, str + ): + name = name_node.value + else: + return util.Uninferable + + result = ClassDef(name, None) + + # Get the bases of the class. + class_bases = next(caller.args[1].infer(context)) + if isinstance(class_bases, (node_classes.Tuple, node_classes.List)): + bases = [] + for base in class_bases.itered(): + inferred = next(base.infer(context=context)) + if inferred: + bases.append( + node_classes.EvaluatedObject(original=base, value=inferred) + ) + result.bases = bases + else: + # There is currently no AST node that can represent an 'unknown' + # node (Uninferable is not an AST node), therefore we simply return Uninferable here + # although we know at least the name of the class. + return util.Uninferable + + # Get the members of the class + try: + members = next(caller.args[2].infer(context)) + except exceptions.InferenceError: + members = None + + if members and isinstance(members, node_classes.Dict): + for attr, value in members.items: + if isinstance(attr, node_classes.Const) and isinstance(attr.value, str): + result.locals[attr.value] = [value] + + result.parent = caller.parent + return result + + def infer_call_result(self, caller, context=None): + """infer what a class is returning when called""" + if ( + self.is_subtype_of("%s.type" % (BUILTINS,), context) + and len(caller.args) == 3 + ): + result = self._infer_type_call(caller, context) + yield result + return + + dunder_call = None + try: + metaclass = self.metaclass(context=context) + if metaclass is not None: + dunder_call = next(metaclass.igetattr("__call__", context)) + except exceptions.AttributeInferenceError: + pass + + if dunder_call and dunder_call.qname() != "builtins.type.__call__": + # Call type.__call__ if not set metaclass + # (since type is the default metaclass) + context = contextmod.bind_context_to_node(context, self) + yield from dunder_call.infer_call_result(caller, context) + else: + yield self.instantiate_class() + + def scope_lookup(self, node, name, offset=0): + """Lookup where the given name is assigned. + + :param node: The node to look for assignments up to. + Any assignments after the given node are ignored. + :type node: NodeNG + + :param name: The name to find assignments for. + :type name: str + + :param offset: The line offset to filter statements up to. + :type offset: int + + :returns: This scope node and the list of assignments associated to the + given name according to the scope where it has been found (locals, + globals or builtin). + :rtype: tuple(str, list(NodeNG)) + """ + # If the name looks like a builtin name, just try to look + # into the upper scope of this class. We might have a + # decorator that it's poorly named after a builtin object + # inside this class. + lookup_upper_frame = ( + isinstance(node.parent, node_classes.Decorators) + and name in MANAGER.builtins_module + ) + if ( + any(node == base or base.parent_of(node) for base in self.bases) + or lookup_upper_frame + ): + # Handle the case where we have either a name + # in the bases of a class, which exists before + # the actual definition or the case where we have + # a Getattr node, with that name. + # + # name = ... + # class A(name): + # def name(self): ... + # + # import name + # class A(name.Name): + # def name(self): ... + + frame = self.parent.frame() + # line offset to avoid that class A(A) resolve the ancestor to + # the defined class + offset = -1 + else: + frame = self + return frame._scope_lookup(node, name, offset) + + @property + def basenames(self): + """The names of the parent classes + + Names are given in the order they appear in the class definition. + + :type: list(str) + """ + return [bnode.as_string() for bnode in self.bases] + + def ancestors(self, recurs=True, context=None): + """Iterate over the base classes in prefixed depth first order. + + :param recurs: Whether to recurse or return direct ancestors only. + :type recurs: bool + + :returns: The base classes + :rtype: iterable(NodeNG) + """ + # FIXME: should be possible to choose the resolution order + # FIXME: inference make infinite loops possible here + yielded = {self} + if context is None: + context = contextmod.InferenceContext() + if not self.bases and self.qname() != "builtins.object": + yield builtin_lookup("object")[1][0] + return + + for stmt in self.bases: + with context.restore_path(): + try: + for baseobj in stmt.infer(context): + if not isinstance(baseobj, ClassDef): + if isinstance(baseobj, bases.Instance): + baseobj = baseobj._proxied + else: + continue + if not baseobj.hide: + if baseobj in yielded: + continue + yielded.add(baseobj) + yield baseobj + if not recurs: + continue + for grandpa in baseobj.ancestors(recurs=True, context=context): + if grandpa is self: + # This class is the ancestor of itself. + break + if grandpa in yielded: + continue + yielded.add(grandpa) + yield grandpa + except exceptions.InferenceError: + continue + + def local_attr_ancestors(self, name, context=None): + """Iterate over the parents that define the given name. + + :param name: The name to find definitions for. + :type name: str + + :returns: The parents that define the given name. + :rtype: iterable(NodeNG) + """ + # Look up in the mro if we can. This will result in the + # attribute being looked up just as Python does it. + try: + ancestors = self.mro(context)[1:] + except exceptions.MroError: + # Fallback to use ancestors, we can't determine + # a sane MRO. + ancestors = self.ancestors(context=context) + for astroid in ancestors: + if name in astroid: + yield astroid + + def instance_attr_ancestors(self, name, context=None): + """Iterate over the parents that define the given name as an attribute. + + :param name: The name to find definitions for. + :type name: str + + :returns: The parents that define the given name as + an instance attribute. + :rtype: iterable(NodeNG) + """ + for astroid in self.ancestors(context=context): + if name in astroid.instance_attrs: + yield astroid + + def has_base(self, node): + """Whether this class directly inherits from the given node. + + :param node: The node to check for. + :type node: NodeNG + + :returns: True if this class directly inherits from the given node. + :rtype: bool + """ + return node in self.bases + + def local_attr(self, name, context=None): + """Get the list of assign nodes associated to the given name. + + Assignments are looked for in both this class and in parents. + + :returns: The list of assignments to the given name. + :rtype: list(NodeNG) + + :raises AttributeInferenceError: If no attribute with this name + can be found in this class or parent classes. + """ + result = [] + if name in self.locals: + result = self.locals[name] + else: + class_node = next(self.local_attr_ancestors(name, context), None) + if class_node: + result = class_node.locals[name] + result = [n for n in result if not isinstance(n, node_classes.DelAttr)] + if result: + return result + raise exceptions.AttributeInferenceError( + target=self, attribute=name, context=context + ) + + def instance_attr(self, name, context=None): + """Get the list of nodes associated to the given attribute name. + + Assignments are looked for in both this class and in parents. + + :returns: The list of assignments to the given name. + :rtype: list(NodeNG) + + :raises AttributeInferenceError: If no attribute with this name + can be found in this class or parent classes. + """ + # Return a copy, so we don't modify self.instance_attrs, + # which could lead to infinite loop. + values = list(self.instance_attrs.get(name, [])) + # get all values from parents + for class_node in self.instance_attr_ancestors(name, context): + values += class_node.instance_attrs[name] + values = [n for n in values if not isinstance(n, node_classes.DelAttr)] + if values: + return values + raise exceptions.AttributeInferenceError( + target=self, attribute=name, context=context + ) + + def instantiate_class(self): + """Get an :class:`Instance` of the :class:`ClassDef` node. + + :returns: An :class:`Instance` of the :class:`ClassDef` node, + or self if this is not possible. + :rtype: Instance or ClassDef + """ + try: + if any(cls.name in EXCEPTION_BASE_CLASSES for cls in self.mro()): + # Subclasses of exceptions can be exception instances + return objects.ExceptionInstance(self) + except exceptions.MroError: + pass + return bases.Instance(self) + + def getattr(self, name, context=None, class_context=True): + """Get an attribute from this class, using Python's attribute semantic. + + This method doesn't look in the :attr:`instance_attrs` dictionary + since it is done by an :class:`Instance` proxy at inference time. + It may return an :class:`Uninferable` object if + the attribute has not been + found, but a ``__getattr__`` or ``__getattribute__`` method is defined. + If ``class_context`` is given, then it is considered that the + attribute is accessed from a class context, + e.g. ClassDef.attribute, otherwise it might have been accessed + from an instance as well. If ``class_context`` is used in that + case, then a lookup in the implicit metaclass and the explicit + metaclass will be done. + + :param name: The attribute to look for. + :type name: str + + :param class_context: Whether the attribute can be accessed statically. + :type class_context: bool + + :returns: The attribute. + :rtype: list(NodeNG) + + :raises AttributeInferenceError: If the attribute cannot be inferred. + """ + if not name: + raise exceptions.AttributeInferenceError( + target=self, attribute=name, context=context + ) + + values = self.locals.get(name, []) + if name in self.special_attributes and class_context and not values: + result = [self.special_attributes.lookup(name)] + if name == "__bases__": + # Need special treatment, since they are mutable + # and we need to return all the values. + result += values + return result + + # don't modify the list in self.locals! + values = list(values) + for classnode in self.ancestors(recurs=True, context=context): + values += classnode.locals.get(name, []) + + if class_context: + values += self._metaclass_lookup_attribute(name, context) + + if not values: + raise exceptions.AttributeInferenceError( + target=self, attribute=name, context=context + ) + + # Look for AnnAssigns, which are not attributes in the purest sense. + for value in values: + if isinstance(value, node_classes.AssignName): + stmt = value.statement() + if isinstance(stmt, node_classes.AnnAssign) and stmt.value is None: + raise exceptions.AttributeInferenceError( + target=self, attribute=name, context=context + ) + return values + + def _metaclass_lookup_attribute(self, name, context): + """Search the given name in the implicit and the explicit metaclass.""" + attrs = set() + implicit_meta = self.implicit_metaclass() + context = contextmod.copy_context(context) + metaclass = self.metaclass(context=context) + for cls in {implicit_meta, metaclass}: + if cls and cls != self and isinstance(cls, ClassDef): + cls_attributes = self._get_attribute_from_metaclass(cls, name, context) + attrs.update(set(cls_attributes)) + return attrs + + def _get_attribute_from_metaclass(self, cls, name, context): + try: + attrs = cls.getattr(name, context=context, class_context=True) + except exceptions.AttributeInferenceError: + return + + for attr in bases._infer_stmts(attrs, context, frame=cls): + if not isinstance(attr, FunctionDef): + yield attr + continue + + if isinstance(attr, objects.Property): + yield attr + continue + if attr.type == "classmethod": + # If the method is a classmethod, then it will + # be bound to the metaclass, not to the class + # from where the attribute is retrieved. + # get_wrapping_class could return None, so just + # default to the current class. + frame = get_wrapping_class(attr) or self + yield bases.BoundMethod(attr, frame) + elif attr.type == "staticmethod": + yield attr + else: + yield bases.BoundMethod(attr, self) + + def igetattr(self, name, context=None, class_context=True): + """Infer the possible values of the given variable. + + :param name: The name of the variable to infer. + :type name: str + + :returns: The inferred possible values. + :rtype: iterable(NodeNG or Uninferable) + """ + # set lookup name since this is necessary to infer on import nodes for + # instance + context = contextmod.copy_context(context) + context.lookupname = name + + metaclass = self.declared_metaclass(context=context) + try: + attributes = self.getattr(name, context, class_context=class_context) + # If we have more than one attribute, make sure that those starting from + # the second one are from the same scope. This is to account for modifications + # to the attribute happening *after* the attribute's definition (e.g. AugAssigns on lists) + if len(attributes) > 1: + first_attr, attributes = attributes[0], attributes[1:] + first_scope = first_attr.scope() + attributes = [first_attr] + [ + attr + for attr in attributes + if attr.parent and attr.parent.scope() == first_scope + ] + + for inferred in bases._infer_stmts(attributes, context, frame=self): + # yield Uninferable object instead of descriptors when necessary + if not isinstance(inferred, node_classes.Const) and isinstance( + inferred, bases.Instance + ): + try: + inferred._proxied.getattr("__get__", context) + except exceptions.AttributeInferenceError: + yield inferred + else: + yield util.Uninferable + elif isinstance(inferred, objects.Property): + function = inferred.function + if not class_context: + # Through an instance so we can solve the property + yield from function.infer_call_result( + caller=self, context=context + ) + # If we have a metaclass, we're accessing this attribute through + # the class itself, which means we can solve the property + elif metaclass: + # Resolve a property as long as it is not accessed through + # the class itself. + yield from function.infer_call_result( + caller=self, context=context + ) + else: + yield inferred + else: + yield function_to_method(inferred, self) + except exceptions.AttributeInferenceError as error: + if not name.startswith("__") and self.has_dynamic_getattr(context): + # class handle some dynamic attributes, return a Uninferable object + yield util.Uninferable + else: + raise exceptions.InferenceError( + error.message, target=self, attribute=name, context=context + ) + + def has_dynamic_getattr(self, context=None): + """Check if the class has a custom __getattr__ or __getattribute__. + + If any such method is found and it is not from + builtins, nor from an extension module, then the function + will return True. + + :returns: True if the class has a custom + __getattr__ or __getattribute__, False otherwise. + :rtype: bool + """ + + def _valid_getattr(node): + root = node.root() + return root.name != BUILTINS and getattr(root, "pure_python", None) + + try: + return _valid_getattr(self.getattr("__getattr__", context)[0]) + except exceptions.AttributeInferenceError: + # if self.newstyle: XXX cause an infinite recursion error + try: + getattribute = self.getattr("__getattribute__", context)[0] + return _valid_getattr(getattribute) + except exceptions.AttributeInferenceError: + pass + return False + + def getitem(self, index, context=None): + """Return the inference of a subscript. + + This is basically looking up the method in the metaclass and calling it. + + :returns: The inferred value of a subscript to this class. + :rtype: NodeNG + + :raises AstroidTypeError: If this class does not define a + ``__getitem__`` method. + """ + try: + methods = dunder_lookup.lookup(self, "__getitem__") + except exceptions.AttributeInferenceError as exc: + raise exceptions.AstroidTypeError(node=self, context=context) from exc + + method = methods[0] + + # Create a new callcontext for providing index as an argument. + new_context = contextmod.bind_context_to_node(context, self) + new_context.callcontext = contextmod.CallContext(args=[index]) + + try: + return next(method.infer_call_result(self, new_context)) + except exceptions.InferenceError: + return util.Uninferable + + def methods(self): + """Iterate over all of the method defined in this class and its parents. + + :returns: The methods defined on the class. + :rtype: iterable(FunctionDef) + """ + done = {} + for astroid in itertools.chain(iter((self,)), self.ancestors()): + for meth in astroid.mymethods(): + if meth.name in done: + continue + done[meth.name] = None + yield meth + + def mymethods(self): + """Iterate over all of the method defined in this class only. + + :returns: The methods defined on the class. + :rtype: iterable(FunctionDef) + """ + for member in self.values(): + if isinstance(member, FunctionDef): + yield member + + def implicit_metaclass(self): + """Get the implicit metaclass of the current class. + + For newstyle classes, this will return an instance of builtins.type. + For oldstyle classes, it will simply return None, since there's + no implicit metaclass there. + + :returns: The metaclass. + :rtype: builtins.type or None + """ + if self.newstyle: + return builtin_lookup("type")[1][0] + return None + + _metaclass = None + + def declared_metaclass(self, context=None): + """Return the explicit declared metaclass for the current class. + + An explicit declared metaclass is defined + either by passing the ``metaclass`` keyword argument + in the class definition line (Python 3) or (Python 2) by + having a ``__metaclass__`` class attribute, or if there are + no explicit bases but there is a global ``__metaclass__`` variable. + + :returns: The metaclass of this class, + or None if one could not be found. + :rtype: NodeNG or None + """ + for base in self.bases: + try: + for baseobj in base.infer(context=context): + if isinstance(baseobj, ClassDef) and baseobj.hide: + self._metaclass = baseobj._metaclass + self._metaclass_hack = True + break + except exceptions.InferenceError: + pass + + if self._metaclass: + # Expects this from Py3k TreeRebuilder + try: + return next( + node + for node in self._metaclass.infer(context=context) + if node is not util.Uninferable + ) + except (exceptions.InferenceError, StopIteration): + return None + + return None + + def _find_metaclass(self, seen=None, context=None): + if seen is None: + seen = set() + seen.add(self) + + klass = self.declared_metaclass(context=context) + if klass is None: + for parent in self.ancestors(context=context): + if parent not in seen: + klass = parent._find_metaclass(seen) + if klass is not None: + break + return klass + + def metaclass(self, context=None): + """Get the metaclass of this class. + + If this class does not define explicitly a metaclass, + then the first defined metaclass in ancestors will be used + instead. + + :returns: The metaclass of this class. + :rtype: NodeNG or None + """ + return self._find_metaclass(context=context) + + def has_metaclass_hack(self): + return self._metaclass_hack + + def _islots(self): + """ Return an iterator with the inferred slots. """ + if "__slots__" not in self.locals: + return None + for slots in self.igetattr("__slots__"): + # check if __slots__ is a valid type + for meth in ITER_METHODS: + try: + slots.getattr(meth) + break + except exceptions.AttributeInferenceError: + continue + else: + continue + + if isinstance(slots, node_classes.Const): + # a string. Ignore the following checks, + # but yield the node, only if it has a value + if slots.value: + yield slots + continue + if not hasattr(slots, "itered"): + # we can't obtain the values, maybe a .deque? + continue + + if isinstance(slots, node_classes.Dict): + values = [item[0] for item in slots.items] + else: + values = slots.itered() + if values is util.Uninferable: + continue + if not values: + # Stop the iteration, because the class + # has an empty list of slots. + return values + + for elt in values: + try: + for inferred in elt.infer(): + if inferred is util.Uninferable: + continue + if not isinstance( + inferred, node_classes.Const + ) or not isinstance(inferred.value, str): + continue + if not inferred.value: + continue + yield inferred + except exceptions.InferenceError: + continue + + return None + + def _slots(self): + if not self.newstyle: + raise NotImplementedError( + "The concept of slots is undefined for old-style classes." + ) + + slots = self._islots() + try: + first = next(slots) + except StopIteration as exc: + # The class doesn't have a __slots__ definition or empty slots. + if exc.args and exc.args[0] not in ("", None): + return exc.args[0] + return None + return [first] + list(slots) + + # Cached, because inferring them all the time is expensive + @decorators_mod.cached + def slots(self): + """Get all the slots for this node. + + :returns: The names of slots for this class. + If the class doesn't define any slot, through the ``__slots__`` + variable, then this function will return a None. + Also, it will return None in the case the slots were not inferred. + :rtype: list(str) or None + """ + + def grouped_slots(): + # Not interested in object, since it can't have slots. + for cls in self.mro()[:-1]: + try: + cls_slots = cls._slots() + except NotImplementedError: + continue + if cls_slots is not None: + yield from cls_slots + else: + yield None + + if not self.newstyle: + raise NotImplementedError( + "The concept of slots is undefined for old-style classes." + ) + + slots = list(grouped_slots()) + if not all(slot is not None for slot in slots): + return None + + return sorted(set(slots), key=lambda item: item.value) + + def _inferred_bases(self, context=None): + # Similar with .ancestors, but the difference is when one base is inferred, + # only the first object is wanted. That's because + # we aren't interested in superclasses, as in the following + # example: + # + # class SomeSuperClass(object): pass + # class SomeClass(SomeSuperClass): pass + # class Test(SomeClass): pass + # + # Inferring SomeClass from the Test's bases will give + # us both SomeClass and SomeSuperClass, but we are interested + # only in SomeClass. + + if context is None: + context = contextmod.InferenceContext() + if not self.bases and self.qname() != "builtins.object": + yield builtin_lookup("object")[1][0] + return + + for stmt in self.bases: + try: + baseobj = next(stmt.infer(context=context)) + except exceptions.InferenceError: + continue + if isinstance(baseobj, bases.Instance): + baseobj = baseobj._proxied + if not isinstance(baseobj, ClassDef): + continue + if not baseobj.hide: + yield baseobj + else: + yield from baseobj.bases + + def _compute_mro(self, context=None): + inferred_bases = list(self._inferred_bases(context=context)) + bases_mro = [] + for base in inferred_bases: + if base is self: + continue + + try: + mro = base._compute_mro(context=context) + bases_mro.append(mro) + except NotImplementedError: + # Some classes have in their ancestors both newstyle and + # old style classes. For these we can't retrieve the .mro, + # although in Python it's possible, since the class we are + # currently working is in fact new style. + # So, we fallback to ancestors here. + ancestors = list(base.ancestors(context=context)) + bases_mro.append(ancestors) + + unmerged_mro = [[self]] + bases_mro + [inferred_bases] + unmerged_mro = list(clean_duplicates_mro(unmerged_mro, self, context)) + return _c3_merge(unmerged_mro, self, context) + + def mro(self, context=None) -> List["ClassDef"]: + """Get the method resolution order, using C3 linearization. + + :returns: The list of ancestors, sorted by the mro. + :rtype: list(NodeNG) + :raises DuplicateBasesError: Duplicate bases in the same class base + :raises InconsistentMroError: A class' MRO is inconsistent + """ + return self._compute_mro(context=context) + + def bool_value(self, context=None): + """Determine the boolean value of this node. + + :returns: The boolean value of this node. + For a :class:`ClassDef` this is always ``True``. + :rtype: bool + """ + return True + + def get_children(self): + if self.decorators is not None: + yield self.decorators + + yield from self.bases + yield from self.body + + @decorators_mod.cached + def _get_assign_nodes(self): + children_assign_nodes = ( + child_node._get_assign_nodes() for child_node in self.body + ) + return list(itertools.chain.from_iterable(children_assign_nodes)) diff --git a/env/lib/python3.8/site-packages/astroid/test_utils.py b/env/lib/python3.8/site-packages/astroid/test_utils.py new file mode 100644 index 000000000..e22c7a4ff --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/test_utils.py @@ -0,0 +1,73 @@ +# Copyright (c) 2013-2014 Google, Inc. +# Copyright (c) 2014 LOGILAB S.A. (Paris, FRANCE) +# Copyright (c) 2015-2016, 2018-2019 Claudiu Popa +# Copyright (c) 2015-2016 Ceridwen +# Copyright (c) 2016 Jakub Wilk +# Copyright (c) 2018 Anthony Sottile + +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER + +"""Utility functions for test code that uses astroid ASTs as input.""" +import contextlib +import functools +import sys +import warnings + +import pytest + +from astroid import nodes + + +def require_version(minver=None, maxver=None): + """ Compare version of python interpreter to the given one. Skip the test + if older. + """ + + def parse(string, default=None): + string = string or default + try: + return tuple(int(v) for v in string.split(".")) + except ValueError as exc: + raise ValueError( + "{string} is not a correct version : should be X.Y[.Z].".format( + string=string + ) + ) from exc + + def check_require_version(f): + current = sys.version_info[:3] + if parse(minver, "0") < current <= parse(maxver, "4"): + return f + + str_version = ".".join(str(v) for v in sys.version_info) + + @functools.wraps(f) + def new_f(*args, **kwargs): + if minver is not None: + pytest.skip( + "Needs Python > %s. Current version is %s." % (minver, str_version) + ) + elif maxver is not None: + pytest.skip( + "Needs Python <= %s. Current version is %s." % (maxver, str_version) + ) + + return new_f + + return check_require_version + + +def get_name_node(start_from, name, index=0): + return [n for n in start_from.nodes_of_class(nodes.Name) if n.name == name][index] + + +@contextlib.contextmanager +def enable_warning(warning): + warnings.simplefilter("always", warning) + try: + yield + finally: + # Reset it to default value, so it will take + # into account the values from the -W flag. + warnings.simplefilter("default", warning) diff --git a/env/lib/python3.8/site-packages/astroid/transforms.py b/env/lib/python3.8/site-packages/astroid/transforms.py new file mode 100644 index 000000000..e5506cc69 --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/transforms.py @@ -0,0 +1,90 @@ +# Copyright (c) 2015-2016, 2018 Claudiu Popa +# Copyright (c) 2016 Ceridwen +# Copyright (c) 2018 Nick Drozd + +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER + + +import collections +from functools import lru_cache + + +class TransformVisitor: + """A visitor for handling transforms. + + The standard approach of using it is to call + :meth:`~visit` with an *astroid* module and the class + will take care of the rest, walking the tree and running the + transforms for each encountered node. + """ + + TRANSFORM_MAX_CACHE_SIZE = 10000 + + def __init__(self): + self.transforms = collections.defaultdict(list) + + @lru_cache(maxsize=TRANSFORM_MAX_CACHE_SIZE) + def _transform(self, node): + """Call matching transforms for the given node if any and return the + transformed node. + """ + cls = node.__class__ + if cls not in self.transforms: + # no transform registered for this class of node + return node + + transforms = self.transforms[cls] + for transform_func, predicate in transforms: + if predicate is None or predicate(node): + ret = transform_func(node) + # if the transformation function returns something, it's + # expected to be a replacement for the node + if ret is not None: + node = ret + if ret.__class__ != cls: + # Can no longer apply the rest of the transforms. + break + return node + + def _visit(self, node): + if hasattr(node, "_astroid_fields"): + for name in node._astroid_fields: + value = getattr(node, name) + visited = self._visit_generic(value) + if visited != value: + setattr(node, name, visited) + return self._transform(node) + + def _visit_generic(self, node): + if isinstance(node, list): + return [self._visit_generic(child) for child in node] + if isinstance(node, tuple): + return tuple(self._visit_generic(child) for child in node) + if not node or isinstance(node, str): + return node + + return self._visit(node) + + def register_transform(self, node_class, transform, predicate=None): + """Register `transform(node)` function to be applied on the given + astroid's `node_class` if `predicate` is None or returns true + when called with the node as argument. + + The transform function may return a value which is then used to + substitute the original node in the tree. + """ + self.transforms[node_class].append((transform, predicate)) + + def unregister_transform(self, node_class, transform, predicate=None): + """Unregister the given transform.""" + self.transforms[node_class].remove((transform, predicate)) + + def visit(self, module): + """Walk the given astroid *tree* and transform each encountered node + + Only the nodes which have transforms registered will actually + be replaced or changed. + """ + module.body = [self._visit(child) for child in module.body] + return self._transform(module) diff --git a/env/lib/python3.8/site-packages/astroid/util.py b/env/lib/python3.8/site-packages/astroid/util.py new file mode 100644 index 000000000..3ab756155 --- /dev/null +++ b/env/lib/python3.8/site-packages/astroid/util.py @@ -0,0 +1,164 @@ +# Copyright (c) 2015-2018 Claudiu Popa +# Copyright (c) 2015-2016 Ceridwen +# Copyright (c) 2018 Bryce Guinta +# Copyright (c) 2018 Nick Drozd + +# Licensed under the LGPL: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html +# For details: https://github.com/PyCQA/astroid/blob/master/COPYING.LESSER + +import warnings +from itertools import islice + +import importlib +import lazy_object_proxy + + +def lazy_descriptor(obj): + class DescriptorProxy(lazy_object_proxy.Proxy): + def __get__(self, instance, owner=None): + return self.__class__.__get__(self, instance) + + return DescriptorProxy(obj) + + +def lazy_import(module_name): + return lazy_object_proxy.Proxy( + lambda: importlib.import_module("." + module_name, "astroid") + ) + + +@object.__new__ +class Uninferable: + """Special inference object, which is returned when inference fails.""" + + def __repr__(self): + return "Uninferable" + + __str__ = __repr__ + + def __getattribute__(self, name): + if name == "next": + raise AttributeError("next method should not be called") + if name.startswith("__") and name.endswith("__"): + return object.__getattribute__(self, name) + if name == "accept": + return object.__getattribute__(self, name) + return self + + def __call__(self, *args, **kwargs): + return self + + def __bool__(self): + return False + + __nonzero__ = __bool__ + + def accept(self, visitor): + func = getattr(visitor, "visit_uninferable") + return func(self) + + +class BadOperationMessage: + """Object which describes a TypeError occurred somewhere in the inference chain + + This is not an exception, but a container object which holds the types and + the error which occurred. + """ + + +class BadUnaryOperationMessage(BadOperationMessage): + """Object which describes operational failures on UnaryOps.""" + + def __init__(self, operand, op, error): + self.operand = operand + self.op = op + self.error = error + + @property + def _object_type_helper(self): + helpers = lazy_import("helpers") + return helpers.object_type + + def _object_type(self, obj): + # pylint: disable=not-callable; can't infer lazy_import + objtype = self._object_type_helper(obj) + if objtype is Uninferable: + return None + + return objtype + + def __str__(self): + if hasattr(self.operand, "name"): + operand_type = self.operand.name + else: + object_type = self._object_type(self.operand) + if hasattr(object_type, "name"): + operand_type = object_type.name + else: + # Just fallback to as_string + operand_type = object_type.as_string() + + msg = "bad operand type for unary {}: {}" + return msg.format(self.op, operand_type) + + +class BadBinaryOperationMessage(BadOperationMessage): + """Object which describes type errors for BinOps.""" + + def __init__(self, left_type, op, right_type): + self.left_type = left_type + self.right_type = right_type + self.op = op + + def __str__(self): + msg = "unsupported operand type(s) for {}: {!r} and {!r}" + return msg.format(self.op, self.left_type.name, self.right_type.name) + + +def _instancecheck(cls, other): + wrapped = cls.__wrapped__ + other_cls = other.__class__ + is_instance_of = wrapped is other_cls or issubclass(other_cls, wrapped) + warnings.warn( + "%r is deprecated and slated for removal in astroid " + "2.0, use %r instead" % (cls.__class__.__name__, wrapped.__name__), + PendingDeprecationWarning, + stacklevel=2, + ) + return is_instance_of + + +def proxy_alias(alias_name, node_type): + """Get a Proxy from the given name to the given node type.""" + proxy = type( + alias_name, + (lazy_object_proxy.Proxy,), + { + "__class__": object.__dict__["__class__"], + "__instancecheck__": _instancecheck, + }, + ) + return proxy(lambda: node_type) + + +def limit_inference(iterator, size): + """Limit inference amount. + + Limit inference amount to help with performance issues with + exponentially exploding possible results. + + :param iterator: Inference generator to limit + :type iterator: Iterator(NodeNG) + + :param size: Maximum mount of nodes yielded plus an + Uninferable at the end if limit reached + :type size: int + + :yields: A possibly modified generator + :rtype param: Iterable + """ + yield from islice(iterator, size) + has_more = next(iterator, False) + if has_more is not False: + yield Uninferable + return diff --git a/env/lib/python3.8/site-packages/autopep8-1.5.4.dist-info/AUTHORS.rst b/env/lib/python3.8/site-packages/autopep8-1.5.4.dist-info/AUTHORS.rst new file mode 100644 index 000000000..e2781e4c6 --- /dev/null +++ b/env/lib/python3.8/site-packages/autopep8-1.5.4.dist-info/AUTHORS.rst @@ -0,0 +1,48 @@ +Main contributors +----------------- +- Hideo Hattori (https://github.com/hhatto) +- Steven Myint (https://github.com/myint) +- Bill Wendling (https://github.com/gwelymernans) + +Patches +------- +- Fraser Tweedale (https://github.com/frasertweedale) +- clach04 (https://github.com/clach04) +- Marc Abramowitz (https://github.com/msabramo) +- dellis23 (https://github.com/dellis23) +- Sam Vilain (https://github.com/samv) +- Florent Xicluna (https://github.com/florentx) +- Andras Tim (https://github.com/andras-tim) +- tomscytale (https://github.com/tomscytale) +- Filip Noetzel (https://github.com/peritus) +- Erik Bray (https://github.com/iguananaut) +- Christopher Medrela (https://github.com/chrismedrela) +- 小明 (https://github.com/dongweiming) +- Andy Hayden (https://github.com/hayd) +- Fabio Zadrozny (https://github.com/fabioz) +- Alex Chernetz (https://github.com/achernet) +- Marc Schlaich (https://github.com/schlamar) +- E. M. Bray (https://github.com/embray) +- Thomas Hisch (https://github.com/thisch) +- Florian Best (https://github.com/spaceone) +- Ian Clark (https://github.com/evenicoulddoit) +- Khairi Hafsham (https://github.com/khairihafsham) +- Neil Halelamien (https://github.com/neilsh) +- Hashem Nasarat (https://github.com/Hnasar) +- Hugo van Kemenade (https://github.com/hugovk) +- gmbnomis (https://github.com/gmbnomis) +- Samuel Lelièvre (https://github.com/slel) +- bigredengineer (https://github.com/bigredengineer) +- Kai Chen (https://github.com/kx-chen) +- Anthony Sottile (https://github.com/asottile) +- 秋葉 (https://github.com/Hanaasagi) +- Christian Clauss (https://github.com/cclauss) +- tobixx (https://github.com/tobixx) +- bigredengineer (https://github.com/bigredengineer) +- Bastien Gérard (https://github.com/bagerard) +- nicolasbonifas (https://github.com/nicolasbonifas) +- Andrii Yurchuk (https://github.com/Ch00k) +- José M. Guisado (https://github.com/pvxe) +- Dai Truong (https://github.com/NovaDev94) +- jnozsc (https://github.com/jnozsc) +- Edwin Shepherd (https://github.com/shardros) diff --git a/env/lib/python3.8/site-packages/autopep8-1.5.4.dist-info/INSTALLER b/env/lib/python3.8/site-packages/autopep8-1.5.4.dist-info/INSTALLER new file mode 100644 index 000000000..a1b589e38 --- /dev/null +++ b/env/lib/python3.8/site-packages/autopep8-1.5.4.dist-info/INSTALLER @@ -0,0 +1 @@ +pip diff --git a/env/lib/python3.8/site-packages/autopep8-1.5.4.dist-info/LICENSE b/env/lib/python3.8/site-packages/autopep8-1.5.4.dist-info/LICENSE new file mode 100644 index 000000000..df9738f45 --- /dev/null +++ b/env/lib/python3.8/site-packages/autopep8-1.5.4.dist-info/LICENSE @@ -0,0 +1,23 @@ +Copyright (C) 2010-2011 Hideo Hattori +Copyright (C) 2011-2013 Hideo Hattori, Steven Myint +Copyright (C) 2013-2016 Hideo Hattori, Steven Myint, Bill Wendling + +Permission is hereby granted, free of charge, to any person obtaining +a copy of this software and associated documentation files (the +"Software"), to deal in the Software without restriction, including +without limitation the rights to use, copy, modify, merge, publish, +distribute, sublicense, and/or sell copies of the Software, and to +permit persons to whom the Software is furnished to do so, subject to +the following conditions: + +The above copyright notice and this permission notice shall be +included in all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, +EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF +MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS +BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN +ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN +CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/env/lib/python3.8/site-packages/autopep8-1.5.4.dist-info/METADATA b/env/lib/python3.8/site-packages/autopep8-1.5.4.dist-info/METADATA new file mode 100644 index 000000000..9580d55a3 --- /dev/null +++ b/env/lib/python3.8/site-packages/autopep8-1.5.4.dist-info/METADATA @@ -0,0 +1,455 @@ +Metadata-Version: 2.1 +Name: autopep8 +Version: 1.5.4 +Summary: A tool that automatically formats Python code to conform to the PEP 8 style guide +Home-page: https://github.com/hhatto/autopep8 +Author: Hideo Hattori +Author-email: hhatto.jp@gmail.com +License: Expat License +Keywords: automation,pep8,format,pycodestyle +Platform: UNKNOWN +Classifier: Development Status :: 5 - Production/Stable +Classifier: Environment :: Console +Classifier: Intended Audience :: Developers +Classifier: License :: OSI Approved :: MIT License +Classifier: Operating System :: OS Independent +Classifier: Programming Language :: Python +Classifier: Programming Language :: Python :: 2 +Classifier: Programming Language :: Python :: 2.7 +Classifier: Programming Language :: Python :: 3 +Classifier: Programming Language :: Python :: 3.4 +Classifier: Programming Language :: Python :: 3.5 +Classifier: Programming Language :: Python :: 3.6 +Classifier: Programming Language :: Python :: 3.7 +Classifier: Programming Language :: Python :: 3.8 +Classifier: Topic :: Software Development :: Libraries :: Python Modules +Classifier: Topic :: Software Development :: Quality Assurance +Requires-Dist: pycodestyle (>=2.6.0) +Requires-Dist: toml + +======== +autopep8 +======== + +.. image:: https://img.shields.io/pypi/v/autopep8.svg + :target: https://pypi.org/project/autopep8/ + :alt: PyPI Version + +.. image:: https://github.com/hhatto/autopep8/workflows/Python%20package/badge.svg + :target: https://github.com/hhatto/autopep8/actions + :alt: Build status + +.. image:: https://codecov.io/gh/hhatto/autopep8/branch/master/graph/badge.svg + :target: https://codecov.io/gh/hhatto/autopep8 + :alt: Code Coverage + +autopep8 automatically formats Python code to conform to the `PEP 8`_ style +guide. It uses the pycodestyle_ utility to determine what parts of the code +needs to be formatted. autopep8 is capable of fixing most of the formatting +issues_ that can be reported by pycodestyle. + +.. _PEP 8: https://www.python.org/dev/peps/pep-0008/ +.. _issues: https://pycodestyle.readthedocs.org/en/latest/intro.html#error-codes + +.. contents:: + + +Installation +============ + +From pip:: + + $ pip install --upgrade autopep8 + +Consider using the ``--user`` option_. + +.. _option: https://pip.pypa.io/en/latest/user_guide/#user-installs + + +Requirements +============ + +autopep8 requires pycodestyle_. + +.. _pycodestyle: https://github.com/PyCQA/pycodestyle + + +Usage +===== + +To modify a file in place (with aggressive level 2):: + + $ autopep8 --in-place --aggressive --aggressive + +Before running autopep8. + +.. code-block:: python + + import math, sys; + + def example1(): + ####This is a long comment. This should be wrapped to fit within 72 characters. + some_tuple=( 1,2, 3,'a' ); + some_variable={'long':'Long code lines should be wrapped within 79 characters.', + 'other':[math.pi, 100,200,300,9876543210,'This is a long string that goes on'], + 'more':{'inner':'This whole logical line should be wrapped.',some_tuple:[1, + 20,300,40000,500000000,60000000000000000]}} + return (some_tuple, some_variable) + def example2(): return {'has_key() is deprecated':True}.has_key({'f':2}.has_key('')); + class Example3( object ): + def __init__ ( self, bar ): + #Comments should have a space after the hash. + if bar : bar+=1; bar=bar* bar ; return bar + else: + some_string = """ + Indentation in multiline strings should not be touched. + Only actual code should be reindented. + """ + return (sys.path, some_string) + +After running autopep8. + +.. code-block:: python + + import math + import sys + + + def example1(): + # This is a long comment. This should be wrapped to fit within 72 + # characters. + some_tuple = (1, 2, 3, 'a') + some_variable = { + 'long': 'Long code lines should be wrapped within 79 characters.', + 'other': [ + math.pi, + 100, + 200, + 300, + 9876543210, + 'This is a long string that goes on'], + 'more': { + 'inner': 'This whole logical line should be wrapped.', + some_tuple: [ + 1, + 20, + 300, + 40000, + 500000000, + 60000000000000000]}} + return (some_tuple, some_variable) + + + def example2(): return ('' in {'f': 2}) in {'has_key() is deprecated': True} + + + class Example3(object): + def __init__(self, bar): + # Comments should have a space after the hash. + if bar: + bar += 1 + bar = bar * bar + return bar + else: + some_string = """ + Indentation in multiline strings should not be touched. + Only actual code should be reindented. + """ + return (sys.path, some_string) + +Options:: + + usage: autopep8 [-h] [--version] [-v] [-d] [-i] [--global-config filename] + [--ignore-local-config] [-r] [-j n] [-p n] [-a] + [--experimental] [--exclude globs] [--list-fixes] + [--ignore errors] [--select errors] [--max-line-length n] + [--line-range line line] [--hang-closing] [--exit-code] + [files [files ...]] + + Automatically formats Python code to conform to the PEP 8 style guide. + + positional arguments: + files files to format or '-' for standard in + + optional arguments: + -h, --help show this help message and exit + --version show program's version number and exit + -v, --verbose print verbose messages; multiple -v result in more + verbose messages + -d, --diff print the diff for the fixed source + -i, --in-place make changes to files in place + --global-config filename + path to a global pep8 config file; if this file does + not exist then this is ignored (default: + ~/.config/pep8) + --ignore-local-config + don't look for and apply local config files; if not + passed, defaults are updated with any config files in + the project's root directory + -r, --recursive run recursively over directories; must be used with + --in-place or --diff + -j n, --jobs n number of parallel jobs; match CPU count if value is + less than 1 + -p n, --pep8-passes n + maximum number of additional pep8 passes (default: + infinite) + -a, --aggressive enable non-whitespace changes; multiple -a result in + more aggressive changes + --experimental enable experimental fixes + --exclude globs exclude file/directory names that match these comma- + separated globs + --list-fixes list codes for fixes; used by --ignore and --select + --ignore errors do not fix these errors/warnings (default: + E226,E24,W50,W690) + --select errors fix only these errors/warnings (e.g. E4,W) + --max-line-length n set maximum allowed line length (default: 79) + --line-range line line, --range line line + only fix errors found within this inclusive range of + line numbers (e.g. 1 99); line numbers are indexed at + 1 + --hang-closing hang-closing option passed to pycodestyle + --exit-code change to behavior of exit code. default behavior of + return value, 0 is no differences, 1 is error exit. + return 2 when add this option. 2 is exists + differences. + + +Features +======== + +autopep8 fixes the following issues_ reported by pycodestyle_:: + + E101 - Reindent all lines. + E11 - Fix indentation. + E121 - Fix indentation to be a multiple of four. + E122 - Add absent indentation for hanging indentation. + E123 - Align closing bracket to match opening bracket. + E124 - Align closing bracket to match visual indentation. + E125 - Indent to distinguish line from next logical line. + E126 - Fix over-indented hanging indentation. + E127 - Fix visual indentation. + E128 - Fix visual indentation. + E129 - Fix visual indentation. + E131 - Fix hanging indent for unaligned continuation line. + E133 - Fix missing indentation for closing bracket. + E20 - Remove extraneous whitespace. + E211 - Remove extraneous whitespace. + E22 - Fix extraneous whitespace around keywords. + E224 - Remove extraneous whitespace around operator. + E225 - Fix missing whitespace around operator. + E226 - Fix missing whitespace around arithmetic operator. + E227 - Fix missing whitespace around bitwise/shift operator. + E228 - Fix missing whitespace around modulo operator. + E231 - Add missing whitespace. + E241 - Fix extraneous whitespace around keywords. + E242 - Remove extraneous whitespace around operator. + E251 - Remove whitespace around parameter '=' sign. + E252 - Missing whitespace around parameter equals. + E26 - Fix spacing after comment hash for inline comments. + E265 - Fix spacing after comment hash for block comments. + E266 - Fix too many leading '#' for block comments. + E27 - Fix extraneous whitespace around keywords. + E301 - Add missing blank line. + E302 - Add missing 2 blank lines. + E303 - Remove extra blank lines. + E304 - Remove blank line following function decorator. + E305 - Expected 2 blank lines after end of function or class. + E306 - Expected 1 blank line before a nested definition. + E401 - Put imports on separate lines. + E402 - Fix module level import not at top of file + E501 - Try to make lines fit within --max-line-length characters. + E502 - Remove extraneous escape of newline. + E701 - Put colon-separated compound statement on separate lines. + E70 - Put semicolon-separated compound statement on separate lines. + E711 - Fix comparison with None. + E712 - Fix comparison with boolean. + E713 - Use 'not in' for test for membership. + E714 - Use 'is not' test for object identity. + E721 - Use "isinstance()" instead of comparing types directly. + E722 - Fix bare except. + E731 - Use a def when use do not assign a lambda expression. + W291 - Remove trailing whitespace. + W292 - Add a single newline at the end of the file. + W293 - Remove trailing whitespace on blank line. + W391 - Remove trailing blank lines. + W503 - Fix line break before binary operator. + W504 - Fix line break after binary operator. + W601 - Use "in" rather than "has_key()". + W602 - Fix deprecated form of raising exception. + W603 - Use "!=" instead of "<>" + W604 - Use "repr()" instead of backticks. + W605 - Fix invalid escape sequence 'x'. + W690 - Fix various deprecated code (via lib2to3). + +autopep8 also fixes some issues not found by pycodestyle_. + +- Correct deprecated or non-idiomatic Python code (via ``lib2to3``). Use this + for making Python 2.7 code more compatible with Python 3. (This is triggered + if ``W690`` is enabled.) +- Normalize files with mixed line endings. +- Put a blank line between a class docstring and its first method + declaration. (Enabled with ``E301``.) +- Remove blank lines between a function declaration and its docstring. (Enabled + with ``E303``.) + +autopep8 avoids fixing some issues found by pycodestyle_. + +- ``E112``/``E113`` for non comments are reports of bad indentation that break + syntax rules. These should not be modified at all. +- ``E265``, which refers to spacing after comment hash, is ignored if the + comment looks like code. autopep8 avoids modifying these since they are not + real comments. If you really want to get rid of the pycodestyle_ warning, + consider just removing the commented-out code. (This can be automated via + eradicate_.) + +.. _eradicate: https://github.com/myint/eradicate + + +More advanced usage +=================== + +By default autopep8 only makes whitespace changes. Thus, by default, it does +not fix ``E711`` and ``E712``. (Changing ``x == None`` to ``x is None`` may +change the meaning of the program if ``x`` has its ``__eq__`` method +overridden.) Nor does it correct deprecated code ``W6``. To enable these +more aggressive fixes, use the ``--aggressive`` option:: + + $ autopep8 --aggressive + +Use multiple ``--aggressive`` to increase the aggressiveness level. For +example, ``E712`` requires aggressiveness level 2 (since ``x == True`` could be +changed to either ``x`` or ``x is True``, but autopep8 chooses the former). + +``--aggressive`` will also shorten lines more aggressively. It will also remove +trailing whitespace more aggressively. (Usually, we don't touch trailing +whitespace in docstrings and other multiline strings. And to do even more +aggressive changes to docstrings, use docformatter_.) + +.. _docformatter: https://github.com/myint/docformatter + +To enable only a subset of the fixes, use the ``--select`` option. For example, +to fix various types of indentation issues:: + + $ autopep8 --select=E1,W1 + +Similarly, to just fix deprecated code:: + + $ autopep8 --aggressive --select=W6 + +The above is useful when trying to port a single code base to work with both +Python 2 and Python 3 at the same time. + +If the file being fixed is large, you may want to enable verbose progress +messages:: + + $ autopep8 -v + +Passing in ``--experimental`` enables the following functionality: + +- Shortens code lines by taking its length into account + +:: + +$ autopep8 --experimental + +Use as a module +=============== + +The simplest way of using autopep8 as a module is via the ``fix_code()`` +function: + + >>> import autopep8 + >>> autopep8.fix_code('x= 123\n') + 'x = 123\n' + +Or with options: + + >>> import autopep8 + >>> autopep8.fix_code('x.has_key(y)\n', + ... options={'aggressive': 1}) + 'y in x\n' + >>> autopep8.fix_code('print( 123 )\n', + ... options={'ignore': ['E']}) + 'print( 123 )\n' + + +Configuration +============= + +By default, if ``$HOME/.config/pycodestyle`` (``~\.pycodestyle`` in Windows +environment) exists, it will be used as global configuration file. +Alternatively, you can specify the global configuration file with the +``--global-config`` option. + +Also, if ``setup.cfg``, ``tox.ini``, ``.pep8`` and ``.flake8`` files exist +in the directory where the target file exists, it will be used as the +configuration file. + +``pep8``, ``pycodestyle``, and ``flake8`` can be used as a section. + +configuration file example:: + + [pycodestyle] + max_line_length = 120 + ignore = E501 + +pyproject.toml +-------------- + +autopep8 can also use ``pyproject.toml``. +section must use ``[tool.autopep8]``, and ``pyproject.toml`` takes precedence +over any other configuration files. + +configuration file example:: + + [tool.autopep8] + max_line_length = 120 + ignore = "E501,W6" # or ["E501", "W6"] + + +Testing +======= + +Test cases are in ``test/test_autopep8.py``. They can be run directly via +``python test/test_autopep8.py`` or via tox_. The latter is useful for +testing against multiple Python interpreters. (We currently test against +CPython versions 2.7, 3.4, 3.5, 3.6 and 3.7. We also test against PyPy.) + +.. _`tox`: https://pypi.org/project/tox/ + +Broad spectrum testing is available via ``test/acid.py``. This script runs +autopep8 against Python code and checks for correctness and completeness of the +code fixes. It can check that the bytecode remains identical. +``test/acid_pypi.py`` makes use of ``acid.py`` to test against the latest +released packages on PyPI. + + +Troubleshooting +=============== + +``pkg_resources.DistributionNotFound`` +-------------------------------------- + +If you are using an ancient version of ``setuptools``, you might encounter +``pkg_resources.DistributionNotFound`` when trying to run ``autopep8``. Try +upgrading ``setuptools`` to workaround this ``setuptools`` problem:: + + $ pip install --upgrade setuptools + +Use ``sudo`` if you are installing to the system. + + +Links +===== + +* PyPI_ +* GitHub_ +* `Travis CI`_ +* Coveralls_ + +.. _PyPI: https://pypi.org/project/autopep8/ +.. _GitHub: https://github.com/hhatto/autopep8 +.. _`Travis CI`: https://travis-ci.org/hhatto/autopep8 +.. _`Coveralls`: https://coveralls.io/r/hhatto/autopep8 + + diff --git a/env/lib/python3.8/site-packages/autopep8-1.5.4.dist-info/RECORD b/env/lib/python3.8/site-packages/autopep8-1.5.4.dist-info/RECORD new file mode 100644 index 000000000..c9bb6600e --- /dev/null +++ b/env/lib/python3.8/site-packages/autopep8-1.5.4.dist-info/RECORD @@ -0,0 +1,11 @@ +../../../bin/autopep8,sha256=ViKzPDQpNHkwF6IhFw5e8TeR_9Mq68M78ZKyHlBkSDM,287 +__pycache__/autopep8.cpython-38.pyc,, +autopep8-1.5.4.dist-info/AUTHORS.rst,sha256=tiTPsbzGl9dtXCMEWXbWSV1zan1M-BoWtiixs46GIWk,2003 +autopep8-1.5.4.dist-info/INSTALLER,sha256=zuuue4knoyJ-UwPPXg8fezS7VCrXJQrAP7zeNuwvFQg,4 +autopep8-1.5.4.dist-info/LICENSE,sha256=jR0COOSFQ0QZFMqwdB1N4-Bwobg2f3h69fIJr7YLCWo,1181 +autopep8-1.5.4.dist-info/METADATA,sha256=xGWUpO3NYAhFVnkHL7oP9pXmNrTCcl_-fZlCkxJAo48,16767 +autopep8-1.5.4.dist-info/RECORD,, +autopep8-1.5.4.dist-info/WHEEL,sha256=kGT74LWyRUZrL4VgLh6_g12IeVl_9u9ZVhadrgXZUEY,110 +autopep8-1.5.4.dist-info/entry_points.txt,sha256=iHNa5_cSXw2ablVbRmfiFGMG1CNrpEPRCEjn3nspaJ8,44 +autopep8-1.5.4.dist-info/top_level.txt,sha256=s2x-di3QBwGxr7kd5xErt2pom8dsFRdINbmwsOEgLfU,9 +autopep8.py,sha256=95wzvmkg4MH1726gIXpUSFBaE15SdNjTUGPcAp2pLJ8,153775 diff --git a/env/lib/python3.8/site-packages/autopep8-1.5.4.dist-info/WHEEL b/env/lib/python3.8/site-packages/autopep8-1.5.4.dist-info/WHEEL new file mode 100644 index 000000000..ef99c6cf3 --- /dev/null +++ b/env/lib/python3.8/site-packages/autopep8-1.5.4.dist-info/WHEEL @@ -0,0 +1,6 @@ +Wheel-Version: 1.0 +Generator: bdist_wheel (0.34.2) +Root-Is-Purelib: true +Tag: py2-none-any +Tag: py3-none-any + diff --git a/env/lib/python3.8/site-packages/autopep8-1.5.4.dist-info/entry_points.txt b/env/lib/python3.8/site-packages/autopep8-1.5.4.dist-info/entry_points.txt new file mode 100644 index 000000000..e3b2c4f8e --- /dev/null +++ b/env/lib/python3.8/site-packages/autopep8-1.5.4.dist-info/entry_points.txt @@ -0,0 +1,3 @@ +[console_scripts] +autopep8 = autopep8:main + diff --git a/env/lib/python3.8/site-packages/autopep8-1.5.4.dist-info/top_level.txt b/env/lib/python3.8/site-packages/autopep8-1.5.4.dist-info/top_level.txt new file mode 100644 index 000000000..d81c0c257 --- /dev/null +++ b/env/lib/python3.8/site-packages/autopep8-1.5.4.dist-info/top_level.txt @@ -0,0 +1 @@ +autopep8 diff --git a/env/lib/python3.8/site-packages/autopep8.py b/env/lib/python3.8/site-packages/autopep8.py new file mode 100644 index 000000000..47f83bd77 --- /dev/null +++ b/env/lib/python3.8/site-packages/autopep8.py @@ -0,0 +1,4514 @@ +#!/usr/bin/env python + +# Copyright (C) 2010-2011 Hideo Hattori +# Copyright (C) 2011-2013 Hideo Hattori, Steven Myint +# Copyright (C) 2013-2016 Hideo Hattori, Steven Myint, Bill Wendling +# +# Permission is hereby granted, free of charge, to any person obtaining +# a copy of this software and associated documentation files (the +# "Software"), to deal in the Software without restriction, including +# without limitation the rights to use, copy, modify, merge, publish, +# distribute, sublicense, and/or sell copies of the Software, and to +# permit persons to whom the Software is furnished to do so, subject to +# the following conditions: +# +# The above copyright notice and this permission notice shall be +# included in all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, +# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF +# MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS +# BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN +# ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN +# CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. + +"""Automatically formats Python code to conform to the PEP 8 style guide. + +Fixes that only need be done once can be added by adding a function of the form +"fix_(source)" to this module. They should return the fixed source code. +These fixes are picked up by apply_global_fixes(). + +Fixes that depend on pycodestyle should be added as methods to FixPEP8. See the +class documentation for more information. + +""" + +from __future__ import absolute_import +from __future__ import division +from __future__ import print_function +from __future__ import unicode_literals + +import argparse +import codecs +import collections +import copy +import difflib +import fnmatch +import inspect +import io +import itertools +import keyword +import locale +import os +import re +import signal +import sys +import textwrap +import token +import tokenize +import warnings +import ast +try: + from configparser import ConfigParser as SafeConfigParser + from configparser import Error +except ImportError: + from ConfigParser import SafeConfigParser + from ConfigParser import Error + +import toml +import pycodestyle +from pycodestyle import STARTSWITH_INDENT_STATEMENT_REGEX + + +try: + unicode +except NameError: + unicode = str + + +__version__ = '1.5.4' + + +CR = '\r' +LF = '\n' +CRLF = '\r\n' + + +PYTHON_SHEBANG_REGEX = re.compile(r'^#!.*\bpython[23]?\b\s*$') +LAMBDA_REGEX = re.compile(r'([\w.]+)\s=\slambda\s*([\(\)=\w,\s.]*):') +COMPARE_NEGATIVE_REGEX = re.compile(r'\b(not)\s+([^][)(}{]+?)\s+(in|is)\s') +COMPARE_NEGATIVE_REGEX_THROUGH = re.compile(r'\b(not\s+in|is\s+not)\s') +BARE_EXCEPT_REGEX = re.compile(r'except\s*:') +STARTSWITH_DEF_REGEX = re.compile(r'^(async\s+def|def)\s.*\):') +DOCSTRING_START_REGEX = re.compile(r'^u?r?(?P["\']{3})') +ENABLE_REGEX = re.compile(r'# *(fmt|autopep8): *on') +DISABLE_REGEX = re.compile(r'# *(fmt|autopep8): *off') + +EXIT_CODE_OK = 0 +EXIT_CODE_ERROR = 1 +EXIT_CODE_EXISTS_DIFF = 2 + +# For generating line shortening candidates. +SHORTEN_OPERATOR_GROUPS = frozenset([ + frozenset([',']), + frozenset(['%']), + frozenset([',', '(', '[', '{']), + frozenset(['%', '(', '[', '{']), + frozenset([',', '(', '[', '{', '%', '+', '-', '*', '/', '//']), + frozenset(['%', '+', '-', '*', '/', '//']), +]) + + +DEFAULT_IGNORE = 'E226,E24,W50,W690' # TODO: use pycodestyle.DEFAULT_IGNORE +DEFAULT_INDENT_SIZE = 4 +# these fixes conflict with each other, if the `--ignore` setting causes both +# to be enabled, disable both of them +CONFLICTING_CODES = ('W503', 'W504') + +SELECTED_GLOBAL_FIXED_METHOD_CODES = ['W602', ] + +# W602 is handled separately due to the need to avoid "with_traceback". +CODE_TO_2TO3 = { + 'E231': ['ws_comma'], + 'E721': ['idioms'], + 'W601': ['has_key'], + 'W603': ['ne'], + 'W604': ['repr'], + 'W690': ['apply', + 'except', + 'exitfunc', + 'numliterals', + 'operator', + 'paren', + 'reduce', + 'renames', + 'standarderror', + 'sys_exc', + 'throw', + 'tuple_params', + 'xreadlines']} + + +if sys.platform == 'win32': # pragma: no cover + DEFAULT_CONFIG = os.path.expanduser(r'~\.pycodestyle') +else: + DEFAULT_CONFIG = os.path.join(os.getenv('XDG_CONFIG_HOME') or + os.path.expanduser('~/.config'), + 'pycodestyle') +# fallback, use .pep8 +if not os.path.exists(DEFAULT_CONFIG): # pragma: no cover + if sys.platform == 'win32': + DEFAULT_CONFIG = os.path.expanduser(r'~\.pep8') + else: + DEFAULT_CONFIG = os.path.join(os.path.expanduser('~/.config'), 'pep8') +PROJECT_CONFIG = ('setup.cfg', 'tox.ini', '.pep8', '.flake8') + + +MAX_PYTHON_FILE_DETECTION_BYTES = 1024 + + +def open_with_encoding(filename, mode='r', encoding=None, limit_byte_check=-1): + """Return opened file with a specific encoding.""" + if not encoding: + encoding = detect_encoding(filename, limit_byte_check=limit_byte_check) + + return io.open(filename, mode=mode, encoding=encoding, + newline='') # Preserve line endings + + +def detect_encoding(filename, limit_byte_check=-1): + """Return file encoding.""" + try: + with open(filename, 'rb') as input_file: + from lib2to3.pgen2 import tokenize as lib2to3_tokenize + encoding = lib2to3_tokenize.detect_encoding(input_file.readline)[0] + + with open_with_encoding(filename, encoding=encoding) as test_file: + test_file.read(limit_byte_check) + + return encoding + except (LookupError, SyntaxError, UnicodeDecodeError): + return 'latin-1' + + +def readlines_from_file(filename): + """Return contents of file.""" + with open_with_encoding(filename) as input_file: + return input_file.readlines() + + +def extended_blank_lines(logical_line, + blank_lines, + blank_before, + indent_level, + previous_logical): + """Check for missing blank lines after class declaration.""" + if previous_logical.startswith('def '): + if blank_lines and pycodestyle.DOCSTRING_REGEX.match(logical_line): + yield (0, 'E303 too many blank lines ({})'.format(blank_lines)) + elif pycodestyle.DOCSTRING_REGEX.match(previous_logical): + # Missing blank line between class docstring and method declaration. + if ( + indent_level and + not blank_lines and + not blank_before and + logical_line.startswith(('def ')) and + '(self' in logical_line + ): + yield (0, 'E301 expected 1 blank line, found 0') + + +pycodestyle.register_check(extended_blank_lines) + + +def continued_indentation(logical_line, tokens, indent_level, hang_closing, + indent_char, noqa): + """Override pycodestyle's function to provide indentation information.""" + first_row = tokens[0][2][0] + nrows = 1 + tokens[-1][2][0] - first_row + if noqa or nrows == 1: + return + + # indent_next tells us whether the next block is indented. Assuming + # that it is indented by 4 spaces, then we should not allow 4-space + # indents on the final continuation line. In turn, some other + # indents are allowed to have an extra 4 spaces. + indent_next = logical_line.endswith(':') + + row = depth = 0 + valid_hangs = ( + (DEFAULT_INDENT_SIZE,) + if indent_char != '\t' else (DEFAULT_INDENT_SIZE, + 2 * DEFAULT_INDENT_SIZE) + ) + + # Remember how many brackets were opened on each line. + parens = [0] * nrows + + # Relative indents of physical lines. + rel_indent = [0] * nrows + + # For each depth, collect a list of opening rows. + open_rows = [[0]] + # For each depth, memorize the hanging indentation. + hangs = [None] + + # Visual indents. + indent_chances = {} + last_indent = tokens[0][2] + indent = [last_indent[1]] + + last_token_multiline = None + line = None + last_line = '' + last_line_begins_with_multiline = False + for token_type, text, start, end, line in tokens: + + newline = row < start[0] - first_row + if newline: + row = start[0] - first_row + newline = (not last_token_multiline and + token_type not in (tokenize.NL, tokenize.NEWLINE)) + last_line_begins_with_multiline = last_token_multiline + + if newline: + # This is the beginning of a continuation line. + last_indent = start + + # Record the initial indent. + rel_indent[row] = pycodestyle.expand_indent(line) - indent_level + + # Identify closing bracket. + close_bracket = (token_type == tokenize.OP and text in ']})') + + # Is the indent relative to an opening bracket line? + for open_row in reversed(open_rows[depth]): + hang = rel_indent[row] - rel_indent[open_row] + hanging_indent = hang in valid_hangs + if hanging_indent: + break + if hangs[depth]: + hanging_indent = (hang == hangs[depth]) + + visual_indent = (not close_bracket and hang > 0 and + indent_chances.get(start[1])) + + if close_bracket and indent[depth]: + # Closing bracket for visual indent. + if start[1] != indent[depth]: + yield (start, 'E124 {}'.format(indent[depth])) + elif close_bracket and not hang: + # closing bracket matches indentation of opening bracket's line + if hang_closing: + yield (start, 'E133 {}'.format(indent[depth])) + elif indent[depth] and start[1] < indent[depth]: + # Visual indent is broken. + yield (start, 'E128 {}'.format(indent[depth])) + elif (hanging_indent or + (indent_next and + rel_indent[row] == 2 * DEFAULT_INDENT_SIZE)): + # Hanging indent is verified. + if close_bracket and not hang_closing: + yield (start, 'E123 {}'.format(indent_level + + rel_indent[open_row])) + hangs[depth] = hang + elif visual_indent is True: + # Visual indent is verified. + indent[depth] = start[1] + elif visual_indent in (text, unicode): + # Ignore token lined up with matching one from a previous line. + pass + else: + one_indented = (indent_level + rel_indent[open_row] + + DEFAULT_INDENT_SIZE) + # Indent is broken. + if hang <= 0: + error = ('E122', one_indented) + elif indent[depth]: + error = ('E127', indent[depth]) + elif not close_bracket and hangs[depth]: + error = ('E131', one_indented) + elif hang > DEFAULT_INDENT_SIZE: + error = ('E126', one_indented) + else: + hangs[depth] = hang + error = ('E121', one_indented) + + yield (start, '{} {}'.format(*error)) + + # Look for visual indenting. + if ( + parens[row] and + token_type not in (tokenize.NL, tokenize.COMMENT) and + not indent[depth] + ): + indent[depth] = start[1] + indent_chances[start[1]] = True + # Deal with implicit string concatenation. + elif (token_type in (tokenize.STRING, tokenize.COMMENT) or + text in ('u', 'ur', 'b', 'br')): + indent_chances[start[1]] = unicode + # Special case for the "if" statement because len("if (") is equal to + # 4. + elif not indent_chances and not row and not depth and text == 'if': + indent_chances[end[1] + 1] = True + elif text == ':' and line[end[1]:].isspace(): + open_rows[depth].append(row) + + # Keep track of bracket depth. + if token_type == tokenize.OP: + if text in '([{': + depth += 1 + indent.append(0) + hangs.append(None) + if len(open_rows) == depth: + open_rows.append([]) + open_rows[depth].append(row) + parens[row] += 1 + elif text in ')]}' and depth > 0: + # Parent indents should not be more than this one. + prev_indent = indent.pop() or last_indent[1] + hangs.pop() + for d in range(depth): + if indent[d] > prev_indent: + indent[d] = 0 + for ind in list(indent_chances): + if ind >= prev_indent: + del indent_chances[ind] + del open_rows[depth + 1:] + depth -= 1 + if depth: + indent_chances[indent[depth]] = True + for idx in range(row, -1, -1): + if parens[idx]: + parens[idx] -= 1 + break + assert len(indent) == depth + 1 + if ( + start[1] not in indent_chances and + # This is for purposes of speeding up E121 (GitHub #90). + not last_line.rstrip().endswith(',') + ): + # Allow to line up tokens. + indent_chances[start[1]] = text + + last_token_multiline = (start[0] != end[0]) + if last_token_multiline: + rel_indent[end[0] - first_row] = rel_indent[row] + + last_line = line + + if ( + indent_next and + not last_line_begins_with_multiline and + pycodestyle.expand_indent(line) == indent_level + DEFAULT_INDENT_SIZE + ): + pos = (start[0], indent[0] + 4) + desired_indent = indent_level + 2 * DEFAULT_INDENT_SIZE + if visual_indent: + yield (pos, 'E129 {}'.format(desired_indent)) + else: + yield (pos, 'E125 {}'.format(desired_indent)) + + +del pycodestyle._checks['logical_line'][pycodestyle.continued_indentation] +pycodestyle.register_check(continued_indentation) + + +class FixPEP8(object): + + """Fix invalid code. + + Fixer methods are prefixed "fix_". The _fix_source() method looks for these + automatically. + + The fixer method can take either one or two arguments (in addition to + self). The first argument is "result", which is the error information from + pycodestyle. The second argument, "logical", is required only for + logical-line fixes. + + The fixer method can return the list of modified lines or None. An empty + list would mean that no changes were made. None would mean that only the + line reported in the pycodestyle error was modified. Note that the modified + line numbers that are returned are indexed at 1. This typically would + correspond with the line number reported in the pycodestyle error + information. + + [fixed method list] + - e111,e114,e115,e116 + - e121,e122,e123,e124,e125,e126,e127,e128,e129 + - e201,e202,e203 + - e211 + - e221,e222,e223,e224,e225 + - e231 + - e251,e252 + - e261,e262 + - e271,e272,e273,e274 + - e301,e302,e303,e304,e305,e306 + - e401,e402 + - e502 + - e701,e702,e703,e704 + - e711,e712,e713,e714 + - e722 + - e731 + - w291 + - w503,504 + + """ + + def __init__(self, filename, + options, + contents=None, + long_line_ignore_cache=None): + self.filename = filename + if contents is None: + self.source = readlines_from_file(filename) + else: + sio = io.StringIO(contents) + self.source = sio.readlines() + self.options = options + self.indent_word = _get_indentword(''.join(self.source)) + + # collect imports line + self.imports = {} + for i, line in enumerate(self.source): + if (line.find("import ") == 0 or line.find("from ") == 0) and \ + line not in self.imports: + # collect only import statements that first appeared + self.imports[line] = i + + self.long_line_ignore_cache = ( + set() if long_line_ignore_cache is None + else long_line_ignore_cache) + + # Many fixers are the same even though pycodestyle categorizes them + # differently. + self.fix_e115 = self.fix_e112 + self.fix_e121 = self._fix_reindent + self.fix_e122 = self._fix_reindent + self.fix_e123 = self._fix_reindent + self.fix_e124 = self._fix_reindent + self.fix_e126 = self._fix_reindent + self.fix_e127 = self._fix_reindent + self.fix_e128 = self._fix_reindent + self.fix_e129 = self._fix_reindent + self.fix_e133 = self.fix_e131 + self.fix_e202 = self.fix_e201 + self.fix_e203 = self.fix_e201 + self.fix_e211 = self.fix_e201 + self.fix_e221 = self.fix_e271 + self.fix_e222 = self.fix_e271 + self.fix_e223 = self.fix_e271 + self.fix_e226 = self.fix_e225 + self.fix_e227 = self.fix_e225 + self.fix_e228 = self.fix_e225 + self.fix_e241 = self.fix_e271 + self.fix_e242 = self.fix_e224 + self.fix_e252 = self.fix_e225 + self.fix_e261 = self.fix_e262 + self.fix_e272 = self.fix_e271 + self.fix_e273 = self.fix_e271 + self.fix_e274 = self.fix_e271 + self.fix_e306 = self.fix_e301 + self.fix_e501 = ( + self.fix_long_line_logically if + options and (options.aggressive >= 2 or options.experimental) else + self.fix_long_line_physically) + self.fix_e703 = self.fix_e702 + self.fix_w293 = self.fix_w291 + + def _fix_source(self, results): + try: + (logical_start, logical_end) = _find_logical(self.source) + logical_support = True + except (SyntaxError, tokenize.TokenError): # pragma: no cover + logical_support = False + + completed_lines = set() + for result in sorted(results, key=_priority_key): + if result['line'] in completed_lines: + continue + + fixed_methodname = 'fix_' + result['id'].lower() + if hasattr(self, fixed_methodname): + fix = getattr(self, fixed_methodname) + + line_index = result['line'] - 1 + original_line = self.source[line_index] + + is_logical_fix = len(_get_parameters(fix)) > 2 + if is_logical_fix: + logical = None + if logical_support: + logical = _get_logical(self.source, + result, + logical_start, + logical_end) + if logical and set(range( + logical[0][0] + 1, + logical[1][0] + 1)).intersection( + completed_lines): + continue + + modified_lines = fix(result, logical) + else: + modified_lines = fix(result) + + if modified_lines is None: + # Force logical fixes to report what they modified. + assert not is_logical_fix + + if self.source[line_index] == original_line: + modified_lines = [] + + if modified_lines: + completed_lines.update(modified_lines) + elif modified_lines == []: # Empty list means no fix + if self.options.verbose >= 2: + print( + '---> Not fixing {error} on line {line}'.format( + error=result['id'], line=result['line']), + file=sys.stderr) + else: # We assume one-line fix when None. + completed_lines.add(result['line']) + else: + if self.options.verbose >= 3: + print( + "---> '{}' is not defined.".format(fixed_methodname), + file=sys.stderr) + + info = result['info'].strip() + print('---> {}:{}:{}:{}'.format(self.filename, + result['line'], + result['column'], + info), + file=sys.stderr) + + def fix(self): + """Return a version of the source code with PEP 8 violations fixed.""" + pep8_options = { + 'ignore': self.options.ignore, + 'select': self.options.select, + 'max_line_length': self.options.max_line_length, + 'hang_closing': self.options.hang_closing, + } + results = _execute_pep8(pep8_options, self.source) + + if self.options.verbose: + progress = {} + for r in results: + if r['id'] not in progress: + progress[r['id']] = set() + progress[r['id']].add(r['line']) + print('---> {n} issue(s) to fix {progress}'.format( + n=len(results), progress=progress), file=sys.stderr) + + if self.options.line_range: + start, end = self.options.line_range + results = [r for r in results + if start <= r['line'] <= end] + + self._fix_source(filter_results(source=''.join(self.source), + results=results, + aggressive=self.options.aggressive)) + + if self.options.line_range: + # If number of lines has changed then change line_range. + count = sum(sline.count('\n') + for sline in self.source[start - 1:end]) + self.options.line_range[1] = start + count - 1 + + return ''.join(self.source) + + def _fix_reindent(self, result): + """Fix a badly indented line. + + This is done by adding or removing from its initial indent only. + + """ + num_indent_spaces = int(result['info'].split()[1]) + line_index = result['line'] - 1 + target = self.source[line_index] + + self.source[line_index] = ' ' * num_indent_spaces + target.lstrip() + + def fix_e112(self, result): + """Fix under-indented comments.""" + line_index = result['line'] - 1 + target = self.source[line_index] + + if not target.lstrip().startswith('#'): + # Don't screw with invalid syntax. + return [] + + self.source[line_index] = self.indent_word + target + + def fix_e113(self, result): + """Fix unexpected indentation.""" + line_index = result['line'] - 1 + target = self.source[line_index] + indent = _get_indentation(target) + stripped = target.lstrip() + self.source[line_index] = indent[1:] + stripped + + def fix_e116(self, result): + """Fix over-indented comments.""" + line_index = result['line'] - 1 + target = self.source[line_index] + + indent = _get_indentation(target) + stripped = target.lstrip() + + if not stripped.startswith('#'): + # Don't screw with invalid syntax. + return [] + + self.source[line_index] = indent[1:] + stripped + + def fix_e117(self, result): + """Fix over-indented.""" + line_index = result['line'] - 1 + target = self.source[line_index] + + indent = _get_indentation(target) + if indent == '\t': + return [] + + stripped = target.lstrip() + + self.source[line_index] = indent[1:] + stripped + + def fix_e125(self, result): + """Fix indentation undistinguish from the next logical line.""" + num_indent_spaces = int(result['info'].split()[1]) + line_index = result['line'] - 1 + target = self.source[line_index] + + spaces_to_add = num_indent_spaces - len(_get_indentation(target)) + indent = len(_get_indentation(target)) + modified_lines = [] + + while len(_get_indentation(self.source[line_index])) >= indent: + self.source[line_index] = (' ' * spaces_to_add + + self.source[line_index]) + modified_lines.append(1 + line_index) # Line indexed at 1. + line_index -= 1 + + return modified_lines + + def fix_e131(self, result): + """Fix indentation undistinguish from the next logical line.""" + num_indent_spaces = int(result['info'].split()[1]) + line_index = result['line'] - 1 + target = self.source[line_index] + + spaces_to_add = num_indent_spaces - len(_get_indentation(target)) + + if spaces_to_add >= 0: + self.source[line_index] = (' ' * spaces_to_add + + self.source[line_index]) + else: + offset = abs(spaces_to_add) + self.source[line_index] = self.source[line_index][offset:] + + def fix_e201(self, result): + """Remove extraneous whitespace.""" + line_index = result['line'] - 1 + target = self.source[line_index] + offset = result['column'] - 1 + + fixed = fix_whitespace(target, + offset=offset, + replacement='') + + self.source[line_index] = fixed + + def fix_e224(self, result): + """Remove extraneous whitespace around operator.""" + target = self.source[result['line'] - 1] + offset = result['column'] - 1 + fixed = target[:offset] + target[offset:].replace('\t', ' ') + self.source[result['line'] - 1] = fixed + + def fix_e225(self, result): + """Fix missing whitespace around operator.""" + target = self.source[result['line'] - 1] + offset = result['column'] - 1 + fixed = target[:offset] + ' ' + target[offset:] + + # Only proceed if non-whitespace characters match. + # And make sure we don't break the indentation. + if ( + fixed.replace(' ', '') == target.replace(' ', '') and + _get_indentation(fixed) == _get_indentation(target) + ): + self.source[result['line'] - 1] = fixed + error_code = result.get('id', 0) + try: + ts = generate_tokens(fixed) + except (SyntaxError, tokenize.TokenError): + return + if not check_syntax(fixed.lstrip()): + return + errors = list( + pycodestyle.missing_whitespace_around_operator(fixed, ts)) + for e in reversed(errors): + if error_code != e[1].split()[0]: + continue + offset = e[0][1] + fixed = fixed[:offset] + ' ' + fixed[offset:] + self.source[result['line'] - 1] = fixed + else: + return [] + + def fix_e231(self, result): + """Add missing whitespace.""" + line_index = result['line'] - 1 + target = self.source[line_index] + offset = result['column'] + fixed = target[:offset].rstrip() + ' ' + target[offset:].lstrip() + self.source[line_index] = fixed + + def fix_e251(self, result): + """Remove whitespace around parameter '=' sign.""" + line_index = result['line'] - 1 + target = self.source[line_index] + + # This is necessary since pycodestyle sometimes reports columns that + # goes past the end of the physical line. This happens in cases like, + # foo(bar\n=None) + c = min(result['column'] - 1, + len(target) - 1) + + if target[c].strip(): + fixed = target + else: + fixed = target[:c].rstrip() + target[c:].lstrip() + + # There could be an escaped newline + # + # def foo(a=\ + # 1) + if fixed.endswith(('=\\\n', '=\\\r\n', '=\\\r')): + self.source[line_index] = fixed.rstrip('\n\r \t\\') + self.source[line_index + 1] = self.source[line_index + 1].lstrip() + return [line_index + 1, line_index + 2] # Line indexed at 1 + + self.source[result['line'] - 1] = fixed + + def fix_e262(self, result): + """Fix spacing after comment hash.""" + target = self.source[result['line'] - 1] + offset = result['column'] + + code = target[:offset].rstrip(' \t#') + comment = target[offset:].lstrip(' \t#') + + fixed = code + (' # ' + comment if comment.strip() else '\n') + + self.source[result['line'] - 1] = fixed + + def fix_e271(self, result): + """Fix extraneous whitespace around keywords.""" + line_index = result['line'] - 1 + target = self.source[line_index] + offset = result['column'] - 1 + + fixed = fix_whitespace(target, + offset=offset, + replacement=' ') + + if fixed == target: + return [] + else: + self.source[line_index] = fixed + + def fix_e301(self, result): + """Add missing blank line.""" + cr = '\n' + self.source[result['line'] - 1] = cr + self.source[result['line'] - 1] + + def fix_e302(self, result): + """Add missing 2 blank lines.""" + add_linenum = 2 - int(result['info'].split()[-1]) + offset = 1 + if self.source[result['line'] - 2].strip() == "\\": + offset = 2 + cr = '\n' * add_linenum + self.source[result['line'] - offset] = ( + cr + self.source[result['line'] - offset] + ) + + def fix_e303(self, result): + """Remove extra blank lines.""" + delete_linenum = int(result['info'].split('(')[1].split(')')[0]) - 2 + delete_linenum = max(1, delete_linenum) + + # We need to count because pycodestyle reports an offset line number if + # there are comments. + cnt = 0 + line = result['line'] - 2 + modified_lines = [] + while cnt < delete_linenum and line >= 0: + if not self.source[line].strip(): + self.source[line] = '' + modified_lines.append(1 + line) # Line indexed at 1 + cnt += 1 + line -= 1 + + return modified_lines + + def fix_e304(self, result): + """Remove blank line following function decorator.""" + line = result['line'] - 2 + if not self.source[line].strip(): + self.source[line] = '' + + def fix_e305(self, result): + """Add missing 2 blank lines after end of function or class.""" + add_delete_linenum = 2 - int(result['info'].split()[-1]) + cnt = 0 + offset = result['line'] - 2 + modified_lines = [] + if add_delete_linenum < 0: + # delete cr + add_delete_linenum = abs(add_delete_linenum) + while cnt < add_delete_linenum and offset >= 0: + if not self.source[offset].strip(): + self.source[offset] = '' + modified_lines.append(1 + offset) # Line indexed at 1 + cnt += 1 + offset -= 1 + else: + # add cr + cr = '\n' + # check comment line + while True: + if offset < 0: + break + line = self.source[offset].lstrip() + if not line: + break + if line[0] != '#': + break + offset -= 1 + offset += 1 + self.source[offset] = cr + self.source[offset] + modified_lines.append(1 + offset) # Line indexed at 1. + return modified_lines + + def fix_e401(self, result): + """Put imports on separate lines.""" + line_index = result['line'] - 1 + target = self.source[line_index] + offset = result['column'] - 1 + + if not target.lstrip().startswith('import'): + return [] + + indentation = re.split(pattern=r'\bimport\b', + string=target, maxsplit=1)[0] + fixed = (target[:offset].rstrip('\t ,') + '\n' + + indentation + 'import ' + target[offset:].lstrip('\t ,')) + self.source[line_index] = fixed + + def fix_e402(self, result): + (line_index, offset, target) = get_index_offset_contents(result, + self.source) + for i in range(1, 100): + line = "".join(self.source[line_index:line_index+i]) + try: + generate_tokens("".join(line)) + except (SyntaxError, tokenize.TokenError): + continue + break + if not (target in self.imports and self.imports[target] != line_index): + mod_offset = get_module_imports_on_top_of_file(self.source, + line_index) + self.source[mod_offset] = line + self.source[mod_offset] + for offset in range(i): + self.source[line_index+offset] = '' + + def fix_long_line_logically(self, result, logical): + """Try to make lines fit within --max-line-length characters.""" + if ( + not logical or + len(logical[2]) == 1 or + self.source[result['line'] - 1].lstrip().startswith('#') + ): + return self.fix_long_line_physically(result) + + start_line_index = logical[0][0] + end_line_index = logical[1][0] + logical_lines = logical[2] + + previous_line = get_item(self.source, start_line_index - 1, default='') + next_line = get_item(self.source, end_line_index + 1, default='') + + single_line = join_logical_line(''.join(logical_lines)) + + try: + fixed = self.fix_long_line( + target=single_line, + previous_line=previous_line, + next_line=next_line, + original=''.join(logical_lines)) + except (SyntaxError, tokenize.TokenError): + return self.fix_long_line_physically(result) + + if fixed: + for line_index in range(start_line_index, end_line_index + 1): + self.source[line_index] = '' + self.source[start_line_index] = fixed + return range(start_line_index + 1, end_line_index + 1) + + return [] + + def fix_long_line_physically(self, result): + """Try to make lines fit within --max-line-length characters.""" + line_index = result['line'] - 1 + target = self.source[line_index] + + previous_line = get_item(self.source, line_index - 1, default='') + next_line = get_item(self.source, line_index + 1, default='') + + try: + fixed = self.fix_long_line( + target=target, + previous_line=previous_line, + next_line=next_line, + original=target) + except (SyntaxError, tokenize.TokenError): + return [] + + if fixed: + self.source[line_index] = fixed + return [line_index + 1] + + return [] + + def fix_long_line(self, target, previous_line, + next_line, original): + cache_entry = (target, previous_line, next_line) + if cache_entry in self.long_line_ignore_cache: + return [] + + if target.lstrip().startswith('#'): + if self.options.aggressive: + # Wrap commented lines. + return shorten_comment( + line=target, + max_line_length=self.options.max_line_length, + last_comment=not next_line.lstrip().startswith('#')) + return [] + + fixed = get_fixed_long_line( + target=target, + previous_line=previous_line, + original=original, + indent_word=self.indent_word, + max_line_length=self.options.max_line_length, + aggressive=self.options.aggressive, + experimental=self.options.experimental, + verbose=self.options.verbose) + + if fixed and not code_almost_equal(original, fixed): + return fixed + + self.long_line_ignore_cache.add(cache_entry) + return None + + def fix_e502(self, result): + """Remove extraneous escape of newline.""" + (line_index, _, target) = get_index_offset_contents(result, + self.source) + self.source[line_index] = target.rstrip('\n\r \t\\') + '\n' + + def fix_e701(self, result): + """Put colon-separated compound statement on separate lines.""" + line_index = result['line'] - 1 + target = self.source[line_index] + c = result['column'] + + fixed_source = (target[:c] + '\n' + + _get_indentation(target) + self.indent_word + + target[c:].lstrip('\n\r \t\\')) + self.source[result['line'] - 1] = fixed_source + return [result['line'], result['line'] + 1] + + def fix_e702(self, result, logical): + """Put semicolon-separated compound statement on separate lines.""" + if not logical: + return [] # pragma: no cover + logical_lines = logical[2] + + # Avoid applying this when indented. + # https://docs.python.org/reference/compound_stmts.html + for line in logical_lines: + if (result['id'] == 'E702' and ':' in line + and STARTSWITH_INDENT_STATEMENT_REGEX.match(line)): + if self.options.verbose: + print( + '---> avoid fixing {error} with ' + 'other compound statements'.format(error=result['id']), + file=sys.stderr + ) + return [] + + line_index = result['line'] - 1 + target = self.source[line_index] + + if target.rstrip().endswith('\\'): + # Normalize '1; \\\n2' into '1; 2'. + self.source[line_index] = target.rstrip('\n \r\t\\') + self.source[line_index + 1] = self.source[line_index + 1].lstrip() + return [line_index + 1, line_index + 2] + + if target.rstrip().endswith(';'): + self.source[line_index] = target.rstrip('\n \r\t;') + '\n' + return [line_index + 1] + + offset = result['column'] - 1 + first = target[:offset].rstrip(';').rstrip() + second = (_get_indentation(logical_lines[0]) + + target[offset:].lstrip(';').lstrip()) + + # Find inline comment. + inline_comment = None + if target[offset:].lstrip(';').lstrip()[:2] == '# ': + inline_comment = target[offset:].lstrip(';') + + if inline_comment: + self.source[line_index] = first + inline_comment + else: + self.source[line_index] = first + '\n' + second + return [line_index + 1] + + def fix_e704(self, result): + """Fix multiple statements on one line def""" + (line_index, _, target) = get_index_offset_contents(result, + self.source) + match = STARTSWITH_DEF_REGEX.match(target) + if match: + self.source[line_index] = '{}\n{}{}'.format( + match.group(0), + _get_indentation(target) + self.indent_word, + target[match.end(0):].lstrip()) + + def fix_e711(self, result): + """Fix comparison with None.""" + (line_index, offset, target) = get_index_offset_contents(result, + self.source) + + right_offset = offset + 2 + if right_offset >= len(target): + return [] + + left = target[:offset].rstrip() + center = target[offset:right_offset] + right = target[right_offset:].lstrip() + + if center.strip() == '==': + new_center = 'is' + elif center.strip() == '!=': + new_center = 'is not' + else: + return [] + + self.source[line_index] = ' '.join([left, new_center, right]) + + def fix_e712(self, result): + """Fix (trivial case of) comparison with boolean.""" + (line_index, offset, target) = get_index_offset_contents(result, + self.source) + + # Handle very easy "not" special cases. + if re.match(r'^\s*if [\w."\'\[\]]+ == False:$', target): + self.source[line_index] = re.sub(r'if ([\w."\'\[\]]+) == False:', + r'if not \1:', target, count=1) + elif re.match(r'^\s*if [\w."\'\[\]]+ != True:$', target): + self.source[line_index] = re.sub(r'if ([\w."\'\[\]]+) != True:', + r'if not \1:', target, count=1) + else: + right_offset = offset + 2 + if right_offset >= len(target): + return [] + + left = target[:offset].rstrip() + center = target[offset:right_offset] + right = target[right_offset:].lstrip() + + # Handle simple cases only. + new_right = None + if center.strip() == '==': + if re.match(r'\bTrue\b', right): + new_right = re.sub(r'\bTrue\b *', '', right, count=1) + elif center.strip() == '!=': + if re.match(r'\bFalse\b', right): + new_right = re.sub(r'\bFalse\b *', '', right, count=1) + + if new_right is None: + return [] + + if new_right[0].isalnum(): + new_right = ' ' + new_right + + self.source[line_index] = left + new_right + + def fix_e713(self, result): + """Fix (trivial case of) non-membership check.""" + (line_index, offset, target) = get_index_offset_contents(result, + self.source) + + # to convert once 'not in' -> 'in' + before_target = target[:offset] + target = target[offset:] + match_notin = COMPARE_NEGATIVE_REGEX_THROUGH.search(target) + notin_pos_start, notin_pos_end = 0, 0 + if match_notin: + notin_pos_start = match_notin.start(1) + notin_pos_end = match_notin.end() + target = '{}{} {}'.format( + target[:notin_pos_start], 'in', target[notin_pos_end:]) + + # fix 'not in' + match = COMPARE_NEGATIVE_REGEX.search(target) + if match: + if match.group(3) == 'in': + pos_start = match.start(1) + new_target = '{5}{0}{1} {2} {3} {4}'.format( + target[:pos_start], match.group(2), match.group(1), + match.group(3), target[match.end():], before_target) + if match_notin: + # revert 'in' -> 'not in' + pos_start = notin_pos_start + offset + pos_end = notin_pos_end + offset - 4 # len('not ') + new_target = '{}{} {}'.format( + new_target[:pos_start], 'not in', new_target[pos_end:]) + self.source[line_index] = new_target + + def fix_e714(self, result): + """Fix object identity should be 'is not' case.""" + (line_index, offset, target) = get_index_offset_contents(result, + self.source) + + # to convert once 'is not' -> 'is' + before_target = target[:offset] + target = target[offset:] + match_isnot = COMPARE_NEGATIVE_REGEX_THROUGH.search(target) + isnot_pos_start, isnot_pos_end = 0, 0 + if match_isnot: + isnot_pos_start = match_isnot.start(1) + isnot_pos_end = match_isnot.end() + target = '{}{} {}'.format( + target[:isnot_pos_start], 'in', target[isnot_pos_end:]) + + match = COMPARE_NEGATIVE_REGEX.search(target) + if match: + if match.group(3).startswith('is'): + pos_start = match.start(1) + new_target = '{5}{0}{1} {2} {3} {4}'.format( + target[:pos_start], match.group(2), match.group(3), + match.group(1), target[match.end():], before_target) + if match_isnot: + # revert 'is' -> 'is not' + pos_start = isnot_pos_start + offset + pos_end = isnot_pos_end + offset - 4 # len('not ') + new_target = '{}{} {}'.format( + new_target[:pos_start], 'is not', new_target[pos_end:]) + self.source[line_index] = new_target + + def fix_e722(self, result): + """fix bare except""" + (line_index, _, target) = get_index_offset_contents(result, + self.source) + match = BARE_EXCEPT_REGEX.search(target) + if match: + self.source[line_index] = '{}{}{}'.format( + target[:result['column'] - 1], "except BaseException:", + target[match.end():]) + + def fix_e731(self, result): + """Fix do not assign a lambda expression check.""" + (line_index, _, target) = get_index_offset_contents(result, + self.source) + match = LAMBDA_REGEX.search(target) + if match: + end = match.end() + self.source[line_index] = '{}def {}({}): return {}'.format( + target[:match.start(0)], match.group(1), match.group(2), + target[end:].lstrip()) + + def fix_w291(self, result): + """Remove trailing whitespace.""" + fixed_line = self.source[result['line'] - 1].rstrip() + self.source[result['line'] - 1] = fixed_line + '\n' + + def fix_w391(self, _): + """Remove trailing blank lines.""" + blank_count = 0 + for line in reversed(self.source): + line = line.rstrip() + if line: + break + else: + blank_count += 1 + + original_length = len(self.source) + self.source = self.source[:original_length - blank_count] + return range(1, 1 + original_length) + + def fix_w503(self, result): + (line_index, _, target) = get_index_offset_contents(result, + self.source) + one_string_token = target.split()[0] + try: + ts = generate_tokens(one_string_token) + except (SyntaxError, tokenize.TokenError): + return + if not _is_binary_operator(ts[0][0], one_string_token): + return + # find comment + comment_index = 0 + found_not_comment_only_line = False + comment_only_linenum = 0 + for i in range(5): + # NOTE: try to parse code in 5 times + if (line_index - i) < 0: + break + from_index = line_index - i - 1 + if from_index < 0 or len(self.source) <= from_index: + break + to_index = line_index + 1 + strip_line = self.source[from_index].lstrip() + if ( + not found_not_comment_only_line and + strip_line and strip_line[0] == '#' + ): + comment_only_linenum += 1 + continue + found_not_comment_only_line = True + try: + ts = generate_tokens("".join(self.source[from_index:to_index])) + except (SyntaxError, tokenize.TokenError): + continue + newline_count = 0 + newline_index = [] + for index, t in enumerate(ts): + if t[0] in (tokenize.NEWLINE, tokenize.NL): + newline_index.append(index) + newline_count += 1 + if newline_count > 2: + tts = ts[newline_index[-3]:] + else: + tts = ts + old = [] + for t in tts: + if t[0] in (tokenize.NEWLINE, tokenize.NL): + newline_count -= 1 + if newline_count <= 1: + break + if tokenize.COMMENT == t[0] and old and old[0] != tokenize.NL: + comment_index = old[3][1] + break + old = t + break + i = target.index(one_string_token) + fix_target_line = line_index - 1 - comment_only_linenum + self.source[line_index] = '{}{}'.format( + target[:i], target[i + len(one_string_token):].lstrip()) + nl = find_newline(self.source[fix_target_line:line_index]) + before_line = self.source[fix_target_line] + bl = before_line.index(nl) + if comment_index: + self.source[fix_target_line] = '{} {} {}'.format( + before_line[:comment_index], one_string_token, + before_line[comment_index + 1:]) + else: + if before_line[:bl].endswith("#"): + # special case + # see: https://github.com/hhatto/autopep8/issues/503 + self.source[fix_target_line] = '{}{} {}'.format( + before_line[:bl-2], one_string_token, before_line[bl-2:]) + else: + self.source[fix_target_line] = '{} {}{}'.format( + before_line[:bl], one_string_token, before_line[bl:]) + + def fix_w504(self, result): + (line_index, _, target) = get_index_offset_contents(result, + self.source) + # NOTE: is not collect pointed out in pycodestyle==2.4.0 + comment_index = 0 + operator_position = None # (start_position, end_position) + for i in range(1, 6): + to_index = line_index + i + try: + ts = generate_tokens("".join(self.source[line_index:to_index])) + except (SyntaxError, tokenize.TokenError): + continue + newline_count = 0 + newline_index = [] + for index, t in enumerate(ts): + if _is_binary_operator(t[0], t[1]): + if t[2][0] == 1 and t[3][0] == 1: + operator_position = (t[2][1], t[3][1]) + elif t[0] == tokenize.NAME and t[1] in ("and", "or"): + if t[2][0] == 1 and t[3][0] == 1: + operator_position = (t[2][1], t[3][1]) + elif t[0] in (tokenize.NEWLINE, tokenize.NL): + newline_index.append(index) + newline_count += 1 + if newline_count > 2: + tts = ts[:newline_index[-3]] + else: + tts = ts + old = [] + for t in tts: + if tokenize.COMMENT == t[0] and old: + comment_row, comment_index = old[3] + break + old = t + break + if not operator_position: + return + target_operator = target[operator_position[0]:operator_position[1]] + + if comment_index and comment_row == 1: + self.source[line_index] = '{}{}'.format( + target[:operator_position[0]].rstrip(), + target[comment_index:]) + else: + self.source[line_index] = '{}{}{}'.format( + target[:operator_position[0]].rstrip(), + target[operator_position[1]:].lstrip(), + target[operator_position[1]:]) + + next_line = self.source[line_index + 1] + next_line_indent = 0 + m = re.match(r'\s*', next_line) + if m: + next_line_indent = m.span()[1] + self.source[line_index + 1] = '{}{} {}'.format( + next_line[:next_line_indent], target_operator, + next_line[next_line_indent:]) + + def fix_w605(self, result): + (line_index, _, target) = get_index_offset_contents(result, + self.source) + try: + tokens = list(generate_tokens(target)) + except (SyntaxError, tokenize.TokenError): + return + for (pos, _msg) in get_w605_position(tokens): + if target[pos - 1] == "r": + # ignore special case + if self.options.verbose: + print("invalid line: line_number={}, line: {}".format( + line_index + 1, target)) + return + self.source[line_index] = '{}r{}'.format( + target[:pos], target[pos:]) + + +def get_w605_position(tokens): + """workaround get pointing out position by W605.""" + # TODO: When this PR(*) change is released, use pos of pycodestyle + # *: https://github.com/PyCQA/pycodestyle/pull/747 + valid = [ + '\n', '\\', '\'', '"', 'a', 'b', 'f', 'n', 'r', 't', 'v', + '0', '1', '2', '3', '4', '5', '6', '7', 'x', + + # Escape sequences only recognized in string literals + 'N', 'u', 'U', + ] + + for token_type, text, start_pos, _end_pos, _line in tokens: + if token_type == tokenize.STRING: + quote = text[-3:] if text[-3:] in ('"""', "'''") else text[-1] + # Extract string modifiers (e.g. u or r) + quote_pos = text.index(quote) + prefix = text[:quote_pos].lower() + start = quote_pos + len(quote) + string = text[start:-len(quote)] + + if 'r' not in prefix: + pos = string.find('\\') + while pos >= 0: + pos += 1 + if string[pos] not in valid: + yield ( + # No need to search line, token stores position + start_pos[1], + "W605 invalid escape sequence '\\%s'" % + string[pos], + ) + pos = string.find('\\', pos + 1) + + +def get_module_imports_on_top_of_file(source, import_line_index): + """return import or from keyword position + + example: + > 0: import sys + 1: import os + 2: + 3: def function(): + """ + def is_string_literal(line): + if line[0] in 'uUbB': + line = line[1:] + if line and line[0] in 'rR': + line = line[1:] + return line and (line[0] == '"' or line[0] == "'") + + def is_future_import(line): + nodes = ast.parse(line) + for n in nodes.body: + if isinstance(n, ast.ImportFrom) and n.module == '__future__': + return True + return False + + def has_future_import(source): + offset = 0 + line = '' + for _, next_line in source: + for line_part in next_line.strip().splitlines(True): + line = line + line_part + try: + return is_future_import(line), offset + except SyntaxError: + continue + offset += 1 + return False, offset + + allowed_try_keywords = ('try', 'except', 'else', 'finally') + in_docstring = False + docstring_kind = '"""' + source_stream = iter(enumerate(source)) + for cnt, line in source_stream: + if not in_docstring: + m = DOCSTRING_START_REGEX.match(line.lstrip()) + if m is not None: + in_docstring = True + docstring_kind = m.group('kind') + remain = line[m.end(): m.endpos].rstrip() + if remain[-3:] == docstring_kind: # one line doc + in_docstring = False + continue + if in_docstring: + if line.rstrip()[-3:] == docstring_kind: + in_docstring = False + continue + + if not line.rstrip(): + continue + elif line.startswith('#'): + continue + + if line.startswith('import '): + if cnt == import_line_index: + continue + return cnt + elif line.startswith('from '): + if cnt == import_line_index: + continue + hit, offset = has_future_import( + itertools.chain([(cnt, line)], source_stream) + ) + if hit: + # move to the back + return cnt + offset + 1 + return cnt + elif pycodestyle.DUNDER_REGEX.match(line): + return cnt + elif any(line.startswith(kw) for kw in allowed_try_keywords): + continue + elif is_string_literal(line): + return cnt + else: + return cnt + return 0 + + +def get_index_offset_contents(result, source): + """Return (line_index, column_offset, line_contents).""" + line_index = result['line'] - 1 + return (line_index, + result['column'] - 1, + source[line_index]) + + +def get_fixed_long_line(target, previous_line, original, + indent_word=' ', max_line_length=79, + aggressive=False, experimental=False, verbose=False): + """Break up long line and return result. + + Do this by generating multiple reformatted candidates and then + ranking the candidates to heuristically select the best option. + + """ + indent = _get_indentation(target) + source = target[len(indent):] + assert source.lstrip() == source + assert not target.lstrip().startswith('#') + + # Check for partial multiline. + tokens = list(generate_tokens(source)) + + candidates = shorten_line( + tokens, source, indent, + indent_word, + max_line_length, + aggressive=aggressive, + experimental=experimental, + previous_line=previous_line) + + # Also sort alphabetically as a tie breaker (for determinism). + candidates = sorted( + sorted(set(candidates).union([target, original])), + key=lambda x: line_shortening_rank( + x, + indent_word, + max_line_length, + experimental=experimental)) + + if verbose >= 4: + print(('-' * 79 + '\n').join([''] + candidates + ['']), + file=wrap_output(sys.stderr, 'utf-8')) + + if candidates: + best_candidate = candidates[0] + + # Don't allow things to get longer. + if longest_line_length(best_candidate) > longest_line_length(original): + return None + + return best_candidate + + +def longest_line_length(code): + """Return length of longest line.""" + if len(code) == 0: + return 0 + return max(len(line) for line in code.splitlines()) + + +def join_logical_line(logical_line): + """Return single line based on logical line input.""" + indentation = _get_indentation(logical_line) + + return indentation + untokenize_without_newlines( + generate_tokens(logical_line.lstrip())) + '\n' + + +def untokenize_without_newlines(tokens): + """Return source code based on tokens.""" + text = '' + last_row = 0 + last_column = -1 + + for t in tokens: + token_string = t[1] + (start_row, start_column) = t[2] + (end_row, end_column) = t[3] + + if start_row > last_row: + last_column = 0 + if ( + (start_column > last_column or token_string == '\n') and + not text.endswith(' ') + ): + text += ' ' + + if token_string != '\n': + text += token_string + + last_row = end_row + last_column = end_column + + return text.rstrip() + + +def _find_logical(source_lines): + # Make a variable which is the index of all the starts of lines. + logical_start = [] + logical_end = [] + last_newline = True + parens = 0 + for t in generate_tokens(''.join(source_lines)): + if t[0] in [tokenize.COMMENT, tokenize.DEDENT, + tokenize.INDENT, tokenize.NL, + tokenize.ENDMARKER]: + continue + if not parens and t[0] in [tokenize.NEWLINE, tokenize.SEMI]: + last_newline = True + logical_end.append((t[3][0] - 1, t[2][1])) + continue + if last_newline and not parens: + logical_start.append((t[2][0] - 1, t[2][1])) + last_newline = False + if t[0] == tokenize.OP: + if t[1] in '([{': + parens += 1 + elif t[1] in '}])': + parens -= 1 + return (logical_start, logical_end) + + +def _get_logical(source_lines, result, logical_start, logical_end): + """Return the logical line corresponding to the result. + + Assumes input is already E702-clean. + + """ + row = result['line'] - 1 + col = result['column'] - 1 + ls = None + le = None + for i in range(0, len(logical_start), 1): + assert logical_end + x = logical_end[i] + if x[0] > row or (x[0] == row and x[1] > col): + le = x + ls = logical_start[i] + break + if ls is None: + return None + original = source_lines[ls[0]:le[0] + 1] + return ls, le, original + + +def get_item(items, index, default=None): + if 0 <= index < len(items): + return items[index] + + return default + + +def reindent(source, indent_size): + """Reindent all lines.""" + reindenter = Reindenter(source) + return reindenter.run(indent_size) + + +def code_almost_equal(a, b): + """Return True if code is similar. + + Ignore whitespace when comparing specific line. + + """ + split_a = split_and_strip_non_empty_lines(a) + split_b = split_and_strip_non_empty_lines(b) + + if len(split_a) != len(split_b): + return False + + for (index, _) in enumerate(split_a): + if ''.join(split_a[index].split()) != ''.join(split_b[index].split()): + return False + + return True + + +def split_and_strip_non_empty_lines(text): + """Return lines split by newline. + + Ignore empty lines. + + """ + return [line.strip() for line in text.splitlines() if line.strip()] + + +def fix_e265(source, aggressive=False): # pylint: disable=unused-argument + """Format block comments.""" + if '#' not in source: + # Optimization. + return source + + ignored_line_numbers = multiline_string_lines( + source, + include_docstrings=True) | set(commented_out_code_lines(source)) + + fixed_lines = [] + sio = io.StringIO(source) + for (line_number, line) in enumerate(sio.readlines(), start=1): + if ( + line.lstrip().startswith('#') and + line_number not in ignored_line_numbers and + not pycodestyle.noqa(line) + ): + indentation = _get_indentation(line) + line = line.lstrip() + + # Normalize beginning if not a shebang. + if len(line) > 1: + pos = next((index for index, c in enumerate(line) + if c != '#')) + if ( + # Leave multiple spaces like '# ' alone. + (line[:pos].count('#') > 1 or line[1].isalnum() or + not line[1].isspace()) and + line[1] not in ':!' and + # Leave stylistic outlined blocks alone. + not line.rstrip().endswith('#') + ): + line = '# ' + line.lstrip('# \t') + + fixed_lines.append(indentation + line) + else: + fixed_lines.append(line) + + return ''.join(fixed_lines) + + +def refactor(source, fixer_names, ignore=None, filename=''): + """Return refactored code using lib2to3. + + Skip if ignore string is produced in the refactored code. + + """ + from lib2to3 import pgen2 + try: + new_text = refactor_with_2to3(source, + fixer_names=fixer_names, + filename=filename) + except (pgen2.parse.ParseError, + SyntaxError, + UnicodeDecodeError, + UnicodeEncodeError): + return source + + if ignore: + if ignore in new_text and ignore not in source: + return source + + return new_text + + +def code_to_2to3(select, ignore, where='', verbose=False): + fixes = set() + for code, fix in CODE_TO_2TO3.items(): + if code_match(code, select=select, ignore=ignore): + if verbose: + print('---> Applying {} fix for {}'.format(where, + code.upper()), + file=sys.stderr) + fixes |= set(fix) + return fixes + + +def fix_2to3(source, + aggressive=True, select=None, ignore=None, filename='', + where='global', verbose=False): + """Fix various deprecated code (via lib2to3).""" + if not aggressive: + return source + + select = select or [] + ignore = ignore or [] + + return refactor(source, + code_to_2to3(select=select, + ignore=ignore, + where=where, + verbose=verbose), + filename=filename) + + +def fix_w602(source, aggressive=True): + """Fix deprecated form of raising exception.""" + if not aggressive: + return source + + return refactor(source, ['raise'], ignore='with_traceback') + + +def find_newline(source): + """Return type of newline used in source. + + Input is a list of lines. + + """ + assert not isinstance(source, unicode) + + counter = collections.defaultdict(int) + for line in source: + if line.endswith(CRLF): + counter[CRLF] += 1 + elif line.endswith(CR): + counter[CR] += 1 + elif line.endswith(LF): + counter[LF] += 1 + + return (sorted(counter, key=counter.get, reverse=True) or [LF])[0] + + +def _get_indentword(source): + """Return indentation type.""" + indent_word = ' ' # Default in case source has no indentation + try: + for t in generate_tokens(source): + if t[0] == token.INDENT: + indent_word = t[1] + break + except (SyntaxError, tokenize.TokenError): + pass + return indent_word + + +def _get_indentation(line): + """Return leading whitespace.""" + if line.strip(): + non_whitespace_index = len(line) - len(line.lstrip()) + return line[:non_whitespace_index] + + return '' + + +def get_diff_text(old, new, filename): + """Return text of unified diff between old and new.""" + newline = '\n' + diff = difflib.unified_diff( + old, new, + 'original/' + filename, + 'fixed/' + filename, + lineterm=newline) + + text = '' + for line in diff: + text += line + + # Work around missing newline (http://bugs.python.org/issue2142). + if text and not line.endswith(newline): + text += newline + r'\ No newline at end of file' + newline + + return text + + +def _priority_key(pep8_result): + """Key for sorting PEP8 results. + + Global fixes should be done first. This is important for things like + indentation. + + """ + priority = [ + # Fix multiline colon-based before semicolon based. + 'e701', + # Break multiline statements early. + 'e702', + # Things that make lines longer. + 'e225', 'e231', + # Remove extraneous whitespace before breaking lines. + 'e201', + # Shorten whitespace in comment before resorting to wrapping. + 'e262' + ] + middle_index = 10000 + lowest_priority = [ + # We need to shorten lines last since the logical fixer can get in a + # loop, which causes us to exit early. + 'e501', + ] + key = pep8_result['id'].lower() + try: + return priority.index(key) + except ValueError: + try: + return middle_index + lowest_priority.index(key) + 1 + except ValueError: + return middle_index + + +def shorten_line(tokens, source, indentation, indent_word, max_line_length, + aggressive=False, experimental=False, previous_line=''): + """Separate line at OPERATOR. + + Multiple candidates will be yielded. + + """ + for candidate in _shorten_line(tokens=tokens, + source=source, + indentation=indentation, + indent_word=indent_word, + aggressive=aggressive, + previous_line=previous_line): + yield candidate + + if aggressive: + for key_token_strings in SHORTEN_OPERATOR_GROUPS: + shortened = _shorten_line_at_tokens( + tokens=tokens, + source=source, + indentation=indentation, + indent_word=indent_word, + key_token_strings=key_token_strings, + aggressive=aggressive) + + if shortened is not None and shortened != source: + yield shortened + + if experimental: + for shortened in _shorten_line_at_tokens_new( + tokens=tokens, + source=source, + indentation=indentation, + max_line_length=max_line_length): + + yield shortened + + +def _shorten_line(tokens, source, indentation, indent_word, + aggressive=False, previous_line=''): + """Separate line at OPERATOR. + + The input is expected to be free of newlines except for inside multiline + strings and at the end. + + Multiple candidates will be yielded. + + """ + for (token_type, + token_string, + start_offset, + end_offset) in token_offsets(tokens): + + if ( + token_type == tokenize.COMMENT and + not is_probably_part_of_multiline(previous_line) and + not is_probably_part_of_multiline(source) and + not source[start_offset + 1:].strip().lower().startswith( + ('noqa', 'pragma:', 'pylint:')) + ): + # Move inline comments to previous line. + first = source[:start_offset] + second = source[start_offset:] + yield (indentation + second.strip() + '\n' + + indentation + first.strip() + '\n') + elif token_type == token.OP and token_string != '=': + # Don't break on '=' after keyword as this violates PEP 8. + + assert token_type != token.INDENT + + first = source[:end_offset] + + second_indent = indentation + if (first.rstrip().endswith('(') and + source[end_offset:].lstrip().startswith(')')): + pass + elif first.rstrip().endswith('('): + second_indent += indent_word + elif '(' in first: + second_indent += ' ' * (1 + first.find('(')) + else: + second_indent += indent_word + + second = (second_indent + source[end_offset:].lstrip()) + if ( + not second.strip() or + second.lstrip().startswith('#') + ): + continue + + # Do not begin a line with a comma + if second.lstrip().startswith(','): + continue + # Do end a line with a dot + if first.rstrip().endswith('.'): + continue + if token_string in '+-*/': + fixed = first + ' \\' + '\n' + second + else: + fixed = first + '\n' + second + + # Only fix if syntax is okay. + if check_syntax(normalize_multiline(fixed) + if aggressive else fixed): + yield indentation + fixed + + +def _is_binary_operator(token_type, text): + return ((token_type == tokenize.OP or text in ['and', 'or']) and + text not in '()[]{},:.;@=%~') + + +# A convenient way to handle tokens. +Token = collections.namedtuple('Token', ['token_type', 'token_string', + 'spos', 'epos', 'line']) + + +class ReformattedLines(object): + + """The reflowed lines of atoms. + + Each part of the line is represented as an "atom." They can be moved + around when need be to get the optimal formatting. + + """ + + ########################################################################### + # Private Classes + + class _Indent(object): + + """Represent an indentation in the atom stream.""" + + def __init__(self, indent_amt): + self._indent_amt = indent_amt + + def emit(self): + return ' ' * self._indent_amt + + @property + def size(self): + return self._indent_amt + + class _Space(object): + + """Represent a space in the atom stream.""" + + def emit(self): + return ' ' + + @property + def size(self): + return 1 + + class _LineBreak(object): + + """Represent a line break in the atom stream.""" + + def emit(self): + return '\n' + + @property + def size(self): + return 0 + + def __init__(self, max_line_length): + self._max_line_length = max_line_length + self._lines = [] + self._bracket_depth = 0 + self._prev_item = None + self._prev_prev_item = None + + def __repr__(self): + return self.emit() + + ########################################################################### + # Public Methods + + def add(self, obj, indent_amt, break_after_open_bracket): + if isinstance(obj, Atom): + self._add_item(obj, indent_amt) + return + + self._add_container(obj, indent_amt, break_after_open_bracket) + + def add_comment(self, item): + num_spaces = 2 + if len(self._lines) > 1: + if isinstance(self._lines[-1], self._Space): + num_spaces -= 1 + if len(self._lines) > 2: + if isinstance(self._lines[-2], self._Space): + num_spaces -= 1 + + while num_spaces > 0: + self._lines.append(self._Space()) + num_spaces -= 1 + self._lines.append(item) + + def add_indent(self, indent_amt): + self._lines.append(self._Indent(indent_amt)) + + def add_line_break(self, indent): + self._lines.append(self._LineBreak()) + self.add_indent(len(indent)) + + def add_line_break_at(self, index, indent_amt): + self._lines.insert(index, self._LineBreak()) + self._lines.insert(index + 1, self._Indent(indent_amt)) + + def add_space_if_needed(self, curr_text, equal=False): + if ( + not self._lines or isinstance( + self._lines[-1], (self._LineBreak, self._Indent, self._Space)) + ): + return + + prev_text = unicode(self._prev_item) + prev_prev_text = ( + unicode(self._prev_prev_item) if self._prev_prev_item else '') + + if ( + # The previous item was a keyword or identifier and the current + # item isn't an operator that doesn't require a space. + ((self._prev_item.is_keyword or self._prev_item.is_string or + self._prev_item.is_name or self._prev_item.is_number) and + (curr_text[0] not in '([{.,:}])' or + (curr_text[0] == '=' and equal))) or + + # Don't place spaces around a '.', unless it's in an 'import' + # statement. + ((prev_prev_text != 'from' and prev_text[-1] != '.' and + curr_text != 'import') and + + # Don't place a space before a colon. + curr_text[0] != ':' and + + # Don't split up ending brackets by spaces. + ((prev_text[-1] in '}])' and curr_text[0] not in '.,}])') or + + # Put a space after a colon or comma. + prev_text[-1] in ':,' or + + # Put space around '=' if asked to. + (equal and prev_text == '=') or + + # Put spaces around non-unary arithmetic operators. + ((self._prev_prev_item and + (prev_text not in '+-' and + (self._prev_prev_item.is_name or + self._prev_prev_item.is_number or + self._prev_prev_item.is_string)) and + prev_text in ('+', '-', '%', '*', '/', '//', '**', 'in'))))) + ): + self._lines.append(self._Space()) + + def previous_item(self): + """Return the previous non-whitespace item.""" + return self._prev_item + + def fits_on_current_line(self, item_extent): + return self.current_size() + item_extent <= self._max_line_length + + def current_size(self): + """The size of the current line minus the indentation.""" + size = 0 + for item in reversed(self._lines): + size += item.size + if isinstance(item, self._LineBreak): + break + + return size + + def line_empty(self): + return (self._lines and + isinstance(self._lines[-1], + (self._LineBreak, self._Indent))) + + def emit(self): + string = '' + for item in self._lines: + if isinstance(item, self._LineBreak): + string = string.rstrip() + string += item.emit() + + return string.rstrip() + '\n' + + ########################################################################### + # Private Methods + + def _add_item(self, item, indent_amt): + """Add an item to the line. + + Reflow the line to get the best formatting after the item is + inserted. The bracket depth indicates if the item is being + inserted inside of a container or not. + + """ + if self._prev_item and self._prev_item.is_string and item.is_string: + # Place consecutive string literals on separate lines. + self._lines.append(self._LineBreak()) + self._lines.append(self._Indent(indent_amt)) + + item_text = unicode(item) + if self._lines and self._bracket_depth: + # Adding the item into a container. + self._prevent_default_initializer_splitting(item, indent_amt) + + if item_text in '.,)]}': + self._split_after_delimiter(item, indent_amt) + + elif self._lines and not self.line_empty(): + # Adding the item outside of a container. + if self.fits_on_current_line(len(item_text)): + self._enforce_space(item) + + else: + # Line break for the new item. + self._lines.append(self._LineBreak()) + self._lines.append(self._Indent(indent_amt)) + + self._lines.append(item) + self._prev_item, self._prev_prev_item = item, self._prev_item + + if item_text in '([{': + self._bracket_depth += 1 + + elif item_text in '}])': + self._bracket_depth -= 1 + assert self._bracket_depth >= 0 + + def _add_container(self, container, indent_amt, break_after_open_bracket): + actual_indent = indent_amt + 1 + + if ( + unicode(self._prev_item) != '=' and + not self.line_empty() and + not self.fits_on_current_line( + container.size + self._bracket_depth + 2) + ): + + if unicode(container)[0] == '(' and self._prev_item.is_name: + # Don't split before the opening bracket of a call. + break_after_open_bracket = True + actual_indent = indent_amt + 4 + elif ( + break_after_open_bracket or + unicode(self._prev_item) not in '([{' + ): + # If the container doesn't fit on the current line and the + # current line isn't empty, place the container on the next + # line. + self._lines.append(self._LineBreak()) + self._lines.append(self._Indent(indent_amt)) + break_after_open_bracket = False + else: + actual_indent = self.current_size() + 1 + break_after_open_bracket = False + + if isinstance(container, (ListComprehension, IfExpression)): + actual_indent = indent_amt + + # Increase the continued indentation only if recursing on a + # container. + container.reflow(self, ' ' * actual_indent, + break_after_open_bracket=break_after_open_bracket) + + def _prevent_default_initializer_splitting(self, item, indent_amt): + """Prevent splitting between a default initializer. + + When there is a default initializer, it's best to keep it all on + the same line. It's nicer and more readable, even if it goes + over the maximum allowable line length. This goes back along the + current line to determine if we have a default initializer, and, + if so, to remove extraneous whitespaces and add a line + break/indent before it if needed. + + """ + if unicode(item) == '=': + # This is the assignment in the initializer. Just remove spaces for + # now. + self._delete_whitespace() + return + + if (not self._prev_item or not self._prev_prev_item or + unicode(self._prev_item) != '='): + return + + self._delete_whitespace() + prev_prev_index = self._lines.index(self._prev_prev_item) + + if ( + isinstance(self._lines[prev_prev_index - 1], self._Indent) or + self.fits_on_current_line(item.size + 1) + ): + # The default initializer is already the only item on this line. + # Don't insert a newline here. + return + + # Replace the space with a newline/indent combo. + if isinstance(self._lines[prev_prev_index - 1], self._Space): + del self._lines[prev_prev_index - 1] + + self.add_line_break_at(self._lines.index(self._prev_prev_item), + indent_amt) + + def _split_after_delimiter(self, item, indent_amt): + """Split the line only after a delimiter.""" + self._delete_whitespace() + + if self.fits_on_current_line(item.size): + return + + last_space = None + for current_item in reversed(self._lines): + if ( + last_space and + (not isinstance(current_item, Atom) or + not current_item.is_colon) + ): + break + else: + last_space = None + if isinstance(current_item, self._Space): + last_space = current_item + if isinstance(current_item, (self._LineBreak, self._Indent)): + return + + if not last_space: + return + + self.add_line_break_at(self._lines.index(last_space), indent_amt) + + def _enforce_space(self, item): + """Enforce a space in certain situations. + + There are cases where we will want a space where normally we + wouldn't put one. This just enforces the addition of a space. + + """ + if isinstance(self._lines[-1], + (self._Space, self._LineBreak, self._Indent)): + return + + if not self._prev_item: + return + + item_text = unicode(item) + prev_text = unicode(self._prev_item) + + # Prefer a space around a '.' in an import statement, and between the + # 'import' and '('. + if ( + (item_text == '.' and prev_text == 'from') or + (item_text == 'import' and prev_text == '.') or + (item_text == '(' and prev_text == 'import') + ): + self._lines.append(self._Space()) + + def _delete_whitespace(self): + """Delete all whitespace from the end of the line.""" + while isinstance(self._lines[-1], (self._Space, self._LineBreak, + self._Indent)): + del self._lines[-1] + + +class Atom(object): + + """The smallest unbreakable unit that can be reflowed.""" + + def __init__(self, atom): + self._atom = atom + + def __repr__(self): + return self._atom.token_string + + def __len__(self): + return self.size + + def reflow( + self, reflowed_lines, continued_indent, extent, + break_after_open_bracket=False, + is_list_comp_or_if_expr=False, + next_is_dot=False + ): + if self._atom.token_type == tokenize.COMMENT: + reflowed_lines.add_comment(self) + return + + total_size = extent if extent else self.size + + if self._atom.token_string not in ',:([{}])': + # Some atoms will need an extra 1-sized space token after them. + total_size += 1 + + prev_item = reflowed_lines.previous_item() + if ( + not is_list_comp_or_if_expr and + not reflowed_lines.fits_on_current_line(total_size) and + not (next_is_dot and + reflowed_lines.fits_on_current_line(self.size + 1)) and + not reflowed_lines.line_empty() and + not self.is_colon and + not (prev_item and prev_item.is_name and + unicode(self) == '(') + ): + # Start a new line if there is already something on the line and + # adding this atom would make it go over the max line length. + reflowed_lines.add_line_break(continued_indent) + else: + reflowed_lines.add_space_if_needed(unicode(self)) + + reflowed_lines.add(self, len(continued_indent), + break_after_open_bracket) + + def emit(self): + return self.__repr__() + + @property + def is_keyword(self): + return keyword.iskeyword(self._atom.token_string) + + @property + def is_string(self): + return self._atom.token_type == tokenize.STRING + + @property + def is_name(self): + return self._atom.token_type == tokenize.NAME + + @property + def is_number(self): + return self._atom.token_type == tokenize.NUMBER + + @property + def is_comma(self): + return self._atom.token_string == ',' + + @property + def is_colon(self): + return self._atom.token_string == ':' + + @property + def size(self): + return len(self._atom.token_string) + + +class Container(object): + + """Base class for all container types.""" + + def __init__(self, items): + self._items = items + + def __repr__(self): + string = '' + last_was_keyword = False + + for item in self._items: + if item.is_comma: + string += ', ' + elif item.is_colon: + string += ': ' + else: + item_string = unicode(item) + if ( + string and + (last_was_keyword or + (not string.endswith(tuple('([{,.:}]) ')) and + not item_string.startswith(tuple('([{,.:}])')))) + ): + string += ' ' + string += item_string + + last_was_keyword = item.is_keyword + return string + + def __iter__(self): + for element in self._items: + yield element + + def __getitem__(self, idx): + return self._items[idx] + + def reflow(self, reflowed_lines, continued_indent, + break_after_open_bracket=False): + last_was_container = False + for (index, item) in enumerate(self._items): + next_item = get_item(self._items, index + 1) + + if isinstance(item, Atom): + is_list_comp_or_if_expr = ( + isinstance(self, (ListComprehension, IfExpression))) + item.reflow(reflowed_lines, continued_indent, + self._get_extent(index), + is_list_comp_or_if_expr=is_list_comp_or_if_expr, + next_is_dot=(next_item and + unicode(next_item) == '.')) + if last_was_container and item.is_comma: + reflowed_lines.add_line_break(continued_indent) + last_was_container = False + else: # isinstance(item, Container) + reflowed_lines.add(item, len(continued_indent), + break_after_open_bracket) + last_was_container = not isinstance(item, (ListComprehension, + IfExpression)) + + if ( + break_after_open_bracket and index == 0 and + # Prefer to keep empty containers together instead of + # separating them. + unicode(item) == self.open_bracket and + (not next_item or unicode(next_item) != self.close_bracket) and + (len(self._items) != 3 or not isinstance(next_item, Atom)) + ): + reflowed_lines.add_line_break(continued_indent) + break_after_open_bracket = False + else: + next_next_item = get_item(self._items, index + 2) + if ( + unicode(item) not in ['.', '%', 'in'] and + next_item and not isinstance(next_item, Container) and + unicode(next_item) != ':' and + next_next_item and (not isinstance(next_next_item, Atom) or + unicode(next_item) == 'not') and + not reflowed_lines.line_empty() and + not reflowed_lines.fits_on_current_line( + self._get_extent(index + 1) + 2) + ): + reflowed_lines.add_line_break(continued_indent) + + def _get_extent(self, index): + """The extent of the full element. + + E.g., the length of a function call or keyword. + + """ + extent = 0 + prev_item = get_item(self._items, index - 1) + seen_dot = prev_item and unicode(prev_item) == '.' + while index < len(self._items): + item = get_item(self._items, index) + index += 1 + + if isinstance(item, (ListComprehension, IfExpression)): + break + + if isinstance(item, Container): + if prev_item and prev_item.is_name: + if seen_dot: + extent += 1 + else: + extent += item.size + + prev_item = item + continue + elif (unicode(item) not in ['.', '=', ':', 'not'] and + not item.is_name and not item.is_string): + break + + if unicode(item) == '.': + seen_dot = True + + extent += item.size + prev_item = item + + return extent + + @property + def is_string(self): + return False + + @property + def size(self): + return len(self.__repr__()) + + @property + def is_keyword(self): + return False + + @property + def is_name(self): + return False + + @property + def is_comma(self): + return False + + @property + def is_colon(self): + return False + + @property + def open_bracket(self): + return None + + @property + def close_bracket(self): + return None + + +class Tuple(Container): + + """A high-level representation of a tuple.""" + + @property + def open_bracket(self): + return '(' + + @property + def close_bracket(self): + return ')' + + +class List(Container): + + """A high-level representation of a list.""" + + @property + def open_bracket(self): + return '[' + + @property + def close_bracket(self): + return ']' + + +class DictOrSet(Container): + + """A high-level representation of a dictionary or set.""" + + @property + def open_bracket(self): + return '{' + + @property + def close_bracket(self): + return '}' + + +class ListComprehension(Container): + + """A high-level representation of a list comprehension.""" + + @property + def size(self): + length = 0 + for item in self._items: + if isinstance(item, IfExpression): + break + length += item.size + return length + + +class IfExpression(Container): + + """A high-level representation of an if-expression.""" + + +def _parse_container(tokens, index, for_or_if=None): + """Parse a high-level container, such as a list, tuple, etc.""" + + # Store the opening bracket. + items = [Atom(Token(*tokens[index]))] + index += 1 + + num_tokens = len(tokens) + while index < num_tokens: + tok = Token(*tokens[index]) + + if tok.token_string in ',)]}': + # First check if we're at the end of a list comprehension or + # if-expression. Don't add the ending token as part of the list + # comprehension or if-expression, because they aren't part of those + # constructs. + if for_or_if == 'for': + return (ListComprehension(items), index - 1) + + elif for_or_if == 'if': + return (IfExpression(items), index - 1) + + # We've reached the end of a container. + items.append(Atom(tok)) + + # If not, then we are at the end of a container. + if tok.token_string == ')': + # The end of a tuple. + return (Tuple(items), index) + + elif tok.token_string == ']': + # The end of a list. + return (List(items), index) + + elif tok.token_string == '}': + # The end of a dictionary or set. + return (DictOrSet(items), index) + + elif tok.token_string in '([{': + # A sub-container is being defined. + (container, index) = _parse_container(tokens, index) + items.append(container) + + elif tok.token_string == 'for': + (container, index) = _parse_container(tokens, index, 'for') + items.append(container) + + elif tok.token_string == 'if': + (container, index) = _parse_container(tokens, index, 'if') + items.append(container) + + else: + items.append(Atom(tok)) + + index += 1 + + return (None, None) + + +def _parse_tokens(tokens): + """Parse the tokens. + + This converts the tokens into a form where we can manipulate them + more easily. + + """ + + index = 0 + parsed_tokens = [] + + num_tokens = len(tokens) + while index < num_tokens: + tok = Token(*tokens[index]) + + assert tok.token_type != token.INDENT + if tok.token_type == tokenize.NEWLINE: + # There's only one newline and it's at the end. + break + + if tok.token_string in '([{': + (container, index) = _parse_container(tokens, index) + if not container: + return None + parsed_tokens.append(container) + else: + parsed_tokens.append(Atom(tok)) + + index += 1 + + return parsed_tokens + + +def _reflow_lines(parsed_tokens, indentation, max_line_length, + start_on_prefix_line): + """Reflow the lines so that it looks nice.""" + + if unicode(parsed_tokens[0]) == 'def': + # A function definition gets indented a bit more. + continued_indent = indentation + ' ' * 2 * DEFAULT_INDENT_SIZE + else: + continued_indent = indentation + ' ' * DEFAULT_INDENT_SIZE + + break_after_open_bracket = not start_on_prefix_line + + lines = ReformattedLines(max_line_length) + lines.add_indent(len(indentation.lstrip('\r\n'))) + + if not start_on_prefix_line: + # If splitting after the opening bracket will cause the first element + # to be aligned weirdly, don't try it. + first_token = get_item(parsed_tokens, 0) + second_token = get_item(parsed_tokens, 1) + + if ( + first_token and second_token and + unicode(second_token)[0] == '(' and + len(indentation) + len(first_token) + 1 == len(continued_indent) + ): + return None + + for item in parsed_tokens: + lines.add_space_if_needed(unicode(item), equal=True) + + save_continued_indent = continued_indent + if start_on_prefix_line and isinstance(item, Container): + start_on_prefix_line = False + continued_indent = ' ' * (lines.current_size() + 1) + + item.reflow(lines, continued_indent, break_after_open_bracket) + continued_indent = save_continued_indent + + return lines.emit() + + +def _shorten_line_at_tokens_new(tokens, source, indentation, + max_line_length): + """Shorten the line taking its length into account. + + The input is expected to be free of newlines except for inside + multiline strings and at the end. + + """ + # Yield the original source so to see if it's a better choice than the + # shortened candidate lines we generate here. + yield indentation + source + + parsed_tokens = _parse_tokens(tokens) + + if parsed_tokens: + # Perform two reflows. The first one starts on the same line as the + # prefix. The second starts on the line after the prefix. + fixed = _reflow_lines(parsed_tokens, indentation, max_line_length, + start_on_prefix_line=True) + if fixed and check_syntax(normalize_multiline(fixed.lstrip())): + yield fixed + + fixed = _reflow_lines(parsed_tokens, indentation, max_line_length, + start_on_prefix_line=False) + if fixed and check_syntax(normalize_multiline(fixed.lstrip())): + yield fixed + + +def _shorten_line_at_tokens(tokens, source, indentation, indent_word, + key_token_strings, aggressive): + """Separate line by breaking at tokens in key_token_strings. + + The input is expected to be free of newlines except for inside + multiline strings and at the end. + + """ + offsets = [] + for (index, _t) in enumerate(token_offsets(tokens)): + (token_type, + token_string, + start_offset, + end_offset) = _t + + assert token_type != token.INDENT + + if token_string in key_token_strings: + # Do not break in containers with zero or one items. + unwanted_next_token = { + '(': ')', + '[': ']', + '{': '}'}.get(token_string) + if unwanted_next_token: + if ( + get_item(tokens, + index + 1, + default=[None, None])[1] == unwanted_next_token or + get_item(tokens, + index + 2, + default=[None, None])[1] == unwanted_next_token + ): + continue + + if ( + index > 2 and token_string == '(' and + tokens[index - 1][1] in ',(%[' + ): + # Don't split after a tuple start, or before a tuple start if + # the tuple is in a list. + continue + + if end_offset < len(source) - 1: + # Don't split right before newline. + offsets.append(end_offset) + else: + # Break at adjacent strings. These were probably meant to be on + # separate lines in the first place. + previous_token = get_item(tokens, index - 1) + if ( + token_type == tokenize.STRING and + previous_token and previous_token[0] == tokenize.STRING + ): + offsets.append(start_offset) + + current_indent = None + fixed = None + for line in split_at_offsets(source, offsets): + if fixed: + fixed += '\n' + current_indent + line + + for symbol in '([{': + if line.endswith(symbol): + current_indent += indent_word + else: + # First line. + fixed = line + assert not current_indent + current_indent = indent_word + + assert fixed is not None + + if check_syntax(normalize_multiline(fixed) + if aggressive > 1 else fixed): + return indentation + fixed + + return None + + +def token_offsets(tokens): + """Yield tokens and offsets.""" + end_offset = 0 + previous_end_row = 0 + previous_end_column = 0 + for t in tokens: + token_type = t[0] + token_string = t[1] + (start_row, start_column) = t[2] + (end_row, end_column) = t[3] + + # Account for the whitespace between tokens. + end_offset += start_column + if previous_end_row == start_row: + end_offset -= previous_end_column + + # Record the start offset of the token. + start_offset = end_offset + + # Account for the length of the token itself. + end_offset += len(token_string) + + yield (token_type, + token_string, + start_offset, + end_offset) + + previous_end_row = end_row + previous_end_column = end_column + + +def normalize_multiline(line): + """Normalize multiline-related code that will cause syntax error. + + This is for purposes of checking syntax. + + """ + if line.startswith('def ') and line.rstrip().endswith(':'): + return line + ' pass' + elif line.startswith('return '): + return 'def _(): ' + line + elif line.startswith('@'): + return line + 'def _(): pass' + elif line.startswith('class '): + return line + ' pass' + elif line.startswith(('if ', 'elif ', 'for ', 'while ')): + return line + ' pass' + + return line + + +def fix_whitespace(line, offset, replacement): + """Replace whitespace at offset and return fixed line.""" + # Replace escaped newlines too + left = line[:offset].rstrip('\n\r \t\\') + right = line[offset:].lstrip('\n\r \t\\') + if right.startswith('#'): + return line + + return left + replacement + right + + +def _execute_pep8(pep8_options, source): + """Execute pycodestyle via python method calls.""" + class QuietReport(pycodestyle.BaseReport): + + """Version of checker that does not print.""" + + def __init__(self, options): + super(QuietReport, self).__init__(options) + self.__full_error_results = [] + + def error(self, line_number, offset, text, check): + """Collect errors.""" + code = super(QuietReport, self).error(line_number, + offset, + text, + check) + if code: + self.__full_error_results.append( + {'id': code, + 'line': line_number, + 'column': offset + 1, + 'info': text}) + + def full_error_results(self): + """Return error results in detail. + + Results are in the form of a list of dictionaries. Each + dictionary contains 'id', 'line', 'column', and 'info'. + + """ + return self.__full_error_results + + checker = pycodestyle.Checker('', lines=source, reporter=QuietReport, + **pep8_options) + checker.check_all() + return checker.report.full_error_results() + + +def _remove_leading_and_normalize(line): + # ignore FF in first lstrip() + return line.lstrip(' \t\v').rstrip(CR + LF) + '\n' + + +class Reindenter(object): + + """Reindents badly-indented code to uniformly use four-space indentation. + + Released to the public domain, by Tim Peters, 03 October 2000. + + """ + + def __init__(self, input_text): + sio = io.StringIO(input_text) + source_lines = sio.readlines() + + self.string_content_line_numbers = multiline_string_lines(input_text) + + # File lines, rstripped & tab-expanded. Dummy at start is so + # that we can use tokenize's 1-based line numbering easily. + # Note that a line is all-blank iff it is a newline. + self.lines = [] + for line_number, line in enumerate(source_lines, start=1): + # Do not modify if inside a multiline string. + if line_number in self.string_content_line_numbers: + self.lines.append(line) + else: + # Only expand leading tabs. + self.lines.append(_get_indentation(line).expandtabs() + + _remove_leading_and_normalize(line)) + + self.lines.insert(0, None) + self.index = 1 # index into self.lines of next line + self.input_text = input_text + + def run(self, indent_size=DEFAULT_INDENT_SIZE): + """Fix indentation and return modified line numbers. + + Line numbers are indexed at 1. + + """ + if indent_size < 1: + return self.input_text + + try: + stats = _reindent_stats(tokenize.generate_tokens(self.getline)) + except (SyntaxError, tokenize.TokenError): + return self.input_text + # Remove trailing empty lines. + lines = self.lines + # Sentinel. + stats.append((len(lines), 0)) + # Map count of leading spaces to # we want. + have2want = {} + # Program after transformation. + after = [] + # Copy over initial empty lines -- there's nothing to do until + # we see a line with *something* on it. + i = stats[0][0] + after.extend(lines[1:i]) + for i in range(len(stats) - 1): + thisstmt, thislevel = stats[i] + nextstmt = stats[i + 1][0] + have = _leading_space_count(lines[thisstmt]) + want = thislevel * indent_size + if want < 0: + # A comment line. + if have: + # An indented comment line. If we saw the same + # indentation before, reuse what it most recently + # mapped to. + want = have2want.get(have, -1) + if want < 0: + # Then it probably belongs to the next real stmt. + for j in range(i + 1, len(stats) - 1): + jline, jlevel = stats[j] + if jlevel >= 0: + if have == _leading_space_count(lines[jline]): + want = jlevel * indent_size + break + # Maybe it's a hanging comment like this one, + if want < 0: + # in which case we should shift it like its base + # line got shifted. + for j in range(i - 1, -1, -1): + jline, jlevel = stats[j] + if jlevel >= 0: + want = (have + _leading_space_count( + after[jline - 1]) - + _leading_space_count(lines[jline])) + break + if want < 0: + # Still no luck -- leave it alone. + want = have + else: + want = 0 + assert want >= 0 + have2want[have] = want + diff = want - have + if diff == 0 or have == 0: + after.extend(lines[thisstmt:nextstmt]) + else: + for line_number, line in enumerate(lines[thisstmt:nextstmt], + start=thisstmt): + if line_number in self.string_content_line_numbers: + after.append(line) + elif diff > 0: + if line == '\n': + after.append(line) + else: + after.append(' ' * diff + line) + else: + remove = min(_leading_space_count(line), -diff) + after.append(line[remove:]) + + return ''.join(after) + + def getline(self): + """Line-getter for tokenize.""" + if self.index >= len(self.lines): + line = '' + else: + line = self.lines[self.index] + self.index += 1 + return line + + +def _reindent_stats(tokens): + """Return list of (lineno, indentlevel) pairs. + + One for each stmt and comment line. indentlevel is -1 for comment + lines, as a signal that tokenize doesn't know what to do about them; + indeed, they're our headache! + + """ + find_stmt = 1 # Next token begins a fresh stmt? + level = 0 # Current indent level. + stats = [] + + for t in tokens: + token_type = t[0] + sline = t[2][0] + line = t[4] + + if token_type == tokenize.NEWLINE: + # A program statement, or ENDMARKER, will eventually follow, + # after some (possibly empty) run of tokens of the form + # (NL | COMMENT)* (INDENT | DEDENT+)? + find_stmt = 1 + + elif token_type == tokenize.INDENT: + find_stmt = 1 + level += 1 + + elif token_type == tokenize.DEDENT: + find_stmt = 1 + level -= 1 + + elif token_type == tokenize.COMMENT: + if find_stmt: + stats.append((sline, -1)) + # But we're still looking for a new stmt, so leave + # find_stmt alone. + + elif token_type == tokenize.NL: + pass + + elif find_stmt: + # This is the first "real token" following a NEWLINE, so it + # must be the first token of the next program statement, or an + # ENDMARKER. + find_stmt = 0 + if line: # Not endmarker. + stats.append((sline, level)) + + return stats + + +def _leading_space_count(line): + """Return number of leading spaces in line.""" + i = 0 + while i < len(line) and line[i] == ' ': + i += 1 + return i + + +def refactor_with_2to3(source_text, fixer_names, filename=''): + """Use lib2to3 to refactor the source. + + Return the refactored source code. + + """ + from lib2to3.refactor import RefactoringTool + fixers = ['lib2to3.fixes.fix_' + name for name in fixer_names] + tool = RefactoringTool(fixer_names=fixers, explicit=fixers) + + from lib2to3.pgen2 import tokenize as lib2to3_tokenize + try: + # The name parameter is necessary particularly for the "import" fixer. + return unicode(tool.refactor_string(source_text, name=filename)) + except lib2to3_tokenize.TokenError: + return source_text + + +def check_syntax(code): + """Return True if syntax is okay.""" + try: + return compile(code, '', 'exec', dont_inherit=True) + except (SyntaxError, TypeError, ValueError): + return False + + +def find_with_line_numbers(pattern, contents): + """A wrapper around 're.finditer' to find line numbers. + + Returns a list of line numbers where pattern was found in contents. + """ + matches = list(re.finditer(pattern, contents)) + if not matches: + return [] + + end = matches[-1].start() + + # -1 so a failed `rfind` maps to the first line. + newline_offsets = { + -1: 0 + } + for line_num, m in enumerate(re.finditer(r'\n', contents), 1): + offset = m.start() + if offset > end: + break + newline_offsets[offset] = line_num + + def get_line_num(match, contents): + """Get the line number of string in a files contents. + + Failing to find the newline is OK, -1 maps to 0 + + """ + newline_offset = contents.rfind('\n', 0, match.start()) + return newline_offsets[newline_offset] + + return [get_line_num(match, contents) + 1 for match in matches] + + +def get_disabled_ranges(source): + """Returns a list of tuples representing the disabled ranges. + + If disabled and no re-enable will disable for rest of file. + + """ + enable_line_nums = find_with_line_numbers(ENABLE_REGEX, source) + disable_line_nums = find_with_line_numbers(DISABLE_REGEX, source) + total_lines = len(re.findall("\n", source)) + 1 + + enable_commands = {} + for num in enable_line_nums: + enable_commands[num] = True + for num in disable_line_nums: + enable_commands[num] = False + + disabled_ranges = [] + currently_enabled = True + disabled_start = None + + for line, commanded_enabled in sorted(enable_commands.items()): + if currently_enabled is True and commanded_enabled is False: + disabled_start = line + currently_enabled = False + elif currently_enabled is False and commanded_enabled is True: + disabled_ranges.append((disabled_start, line)) + currently_enabled = True + + if currently_enabled is False: + disabled_ranges.append((disabled_start, total_lines)) + + return disabled_ranges + + +def filter_results(source, results, aggressive): + """Filter out spurious reports from pycodestyle. + + If aggressive is True, we allow possibly unsafe fixes (E711, E712). + + """ + non_docstring_string_line_numbers = multiline_string_lines( + source, include_docstrings=False) + all_string_line_numbers = multiline_string_lines( + source, include_docstrings=True) + + commented_out_code_line_numbers = commented_out_code_lines(source) + + # Filter out the disabled ranges + disabled_ranges = get_disabled_ranges(source) + if len(disabled_ranges) > 0: + results = [result for result in results + if any(result['line'] not in range(*disabled_range) + for disabled_range in disabled_ranges) + ] + + has_e901 = any(result['id'].lower() == 'e901' for result in results) + + for r in results: + issue_id = r['id'].lower() + + if r['line'] in non_docstring_string_line_numbers: + if issue_id.startswith(('e1', 'e501', 'w191')): + continue + + if r['line'] in all_string_line_numbers: + if issue_id in ['e501']: + continue + + # We must offset by 1 for lines that contain the trailing contents of + # multiline strings. + if not aggressive and (r['line'] + 1) in all_string_line_numbers: + # Do not modify multiline strings in non-aggressive mode. Remove + # trailing whitespace could break doctests. + if issue_id.startswith(('w29', 'w39')): + continue + + if aggressive <= 0: + if issue_id.startswith(('e711', 'e72', 'w6')): + continue + + if aggressive <= 1: + if issue_id.startswith(('e712', 'e713', 'e714')): + continue + + if aggressive <= 2: + if issue_id.startswith(('e704')): + continue + + if r['line'] in commented_out_code_line_numbers: + if issue_id.startswith(('e26', 'e501')): + continue + + # Do not touch indentation if there is a token error caused by + # incomplete multi-line statement. Otherwise, we risk screwing up the + # indentation. + if has_e901: + if issue_id.startswith(('e1', 'e7')): + continue + + yield r + + +def multiline_string_lines(source, include_docstrings=False): + """Return line numbers that are within multiline strings. + + The line numbers are indexed at 1. + + Docstrings are ignored. + + """ + line_numbers = set() + previous_token_type = '' + try: + for t in generate_tokens(source): + token_type = t[0] + start_row = t[2][0] + end_row = t[3][0] + + if token_type == tokenize.STRING and start_row != end_row: + if ( + include_docstrings or + previous_token_type != tokenize.INDENT + ): + # We increment by one since we want the contents of the + # string. + line_numbers |= set(range(1 + start_row, 1 + end_row)) + + previous_token_type = token_type + except (SyntaxError, tokenize.TokenError): + pass + + return line_numbers + + +def commented_out_code_lines(source): + """Return line numbers of comments that are likely code. + + Commented-out code is bad practice, but modifying it just adds even + more clutter. + + """ + line_numbers = [] + try: + for t in generate_tokens(source): + token_type = t[0] + token_string = t[1] + start_row = t[2][0] + line = t[4] + + # Ignore inline comments. + if not line.lstrip().startswith('#'): + continue + + if token_type == tokenize.COMMENT: + stripped_line = token_string.lstrip('#').strip() + with warnings.catch_warnings(): + # ignore SyntaxWarning in Python3.8+ + # refs: + # https://bugs.python.org/issue15248 + # https://docs.python.org/3.8/whatsnew/3.8.html#other-language-changes + warnings.filterwarnings("ignore", category=SyntaxWarning) + if ( + ' ' in stripped_line and + '#' not in stripped_line and + check_syntax(stripped_line) + ): + line_numbers.append(start_row) + except (SyntaxError, tokenize.TokenError): + pass + + return line_numbers + + +def shorten_comment(line, max_line_length, last_comment=False): + """Return trimmed or split long comment line. + + If there are no comments immediately following it, do a text wrap. + Doing this wrapping on all comments in general would lead to jagged + comment text. + + """ + assert len(line) > max_line_length + line = line.rstrip() + + # PEP 8 recommends 72 characters for comment text. + indentation = _get_indentation(line) + '# ' + max_line_length = min(max_line_length, + len(indentation) + 72) + + MIN_CHARACTER_REPEAT = 5 + if ( + len(line) - len(line.rstrip(line[-1])) >= MIN_CHARACTER_REPEAT and + not line[-1].isalnum() + ): + # Trim comments that end with things like --------- + return line[:max_line_length] + '\n' + elif last_comment and re.match(r'\s*#+\s*\w+', line): + split_lines = textwrap.wrap(line.lstrip(' \t#'), + initial_indent=indentation, + subsequent_indent=indentation, + width=max_line_length, + break_long_words=False, + break_on_hyphens=False) + return '\n'.join(split_lines) + '\n' + + return line + '\n' + + +def normalize_line_endings(lines, newline): + """Return fixed line endings. + + All lines will be modified to use the most common line ending. + + """ + return [line.rstrip('\n\r') + newline for line in lines] + + +def mutual_startswith(a, b): + return b.startswith(a) or a.startswith(b) + + +def code_match(code, select, ignore): + if ignore: + assert not isinstance(ignore, unicode) + for ignored_code in [c.strip() for c in ignore]: + if mutual_startswith(code.lower(), ignored_code.lower()): + return False + + if select: + assert not isinstance(select, unicode) + for selected_code in [c.strip() for c in select]: + if mutual_startswith(code.lower(), selected_code.lower()): + return True + return False + + return True + + +def fix_code(source, options=None, encoding=None, apply_config=False): + """Return fixed source code. + + "encoding" will be used to decode "source" if it is a byte string. + + """ + options = _get_options(options, apply_config) + + if not isinstance(source, unicode): + source = source.decode(encoding or get_encoding()) + + sio = io.StringIO(source) + return fix_lines(sio.readlines(), options=options) + + +def _get_options(raw_options, apply_config): + """Return parsed options.""" + if not raw_options: + return parse_args([''], apply_config=apply_config) + + if isinstance(raw_options, dict): + options = parse_args([''], apply_config=apply_config) + for name, value in raw_options.items(): + if not hasattr(options, name): + raise ValueError("No such option '{}'".format(name)) + + # Check for very basic type errors. + expected_type = type(getattr(options, name)) + if not isinstance(expected_type, (str, unicode)): + if isinstance(value, (str, unicode)): + raise ValueError( + "Option '{}' should not be a string".format(name)) + setattr(options, name, value) + else: + options = raw_options + + return options + + +def fix_lines(source_lines, options, filename=''): + """Return fixed source code.""" + # Transform everything to line feed. Then change them back to original + # before returning fixed source code. + original_newline = find_newline(source_lines) + tmp_source = ''.join(normalize_line_endings(source_lines, '\n')) + + # Keep a history to break out of cycles. + previous_hashes = set() + + if options.line_range: + # Disable "apply_local_fixes()" for now due to issue #175. + fixed_source = tmp_source + else: + pep8_options = { + 'ignore': options.ignore, + 'select': options.select, + 'max_line_length': options.max_line_length, + 'hang_closing': options.hang_closing, + } + sio = io.StringIO(tmp_source) + contents = sio.readlines() + results = _execute_pep8(pep8_options, contents) + codes = {result['id'] for result in results + if result['id'] in SELECTED_GLOBAL_FIXED_METHOD_CODES} + # Apply global fixes only once (for efficiency). + fixed_source = apply_global_fixes(tmp_source, + options, + filename=filename, + codes=codes) + + passes = 0 + long_line_ignore_cache = set() + while hash(fixed_source) not in previous_hashes: + if options.pep8_passes >= 0 and passes > options.pep8_passes: + break + passes += 1 + + previous_hashes.add(hash(fixed_source)) + + tmp_source = copy.copy(fixed_source) + + fix = FixPEP8( + filename, + options, + contents=tmp_source, + long_line_ignore_cache=long_line_ignore_cache) + + fixed_source = fix.fix() + + sio = io.StringIO(fixed_source) + return ''.join(normalize_line_endings(sio.readlines(), original_newline)) + + +def fix_file(filename, options=None, output=None, apply_config=False): + if not options: + options = parse_args([filename], apply_config=apply_config) + + original_source = readlines_from_file(filename) + + fixed_source = original_source + + if options.in_place or options.diff or output: + encoding = detect_encoding(filename) + + if output: + output = LineEndingWrapper(wrap_output(output, encoding=encoding)) + + fixed_source = fix_lines(fixed_source, options, filename=filename) + + if options.diff: + new = io.StringIO(fixed_source) + new = new.readlines() + diff = get_diff_text(original_source, new, filename) + if output: + output.write(diff) + output.flush() + elif options.jobs > 1: + diff = diff.encode(encoding) + return diff + elif options.in_place: + original = "".join(original_source).splitlines() + fixed = fixed_source.splitlines() + original_source_last_line = ( + original_source[-1].split("\n")[-1] if original_source else "" + ) + fixed_source_last_line = fixed_source.split("\n")[-1] + if original != fixed or ( + original_source_last_line != fixed_source_last_line + ): + with open_with_encoding(filename, 'w', encoding=encoding) as fp: + fp.write(fixed_source) + return fixed_source + return None + else: + if output: + output.write(fixed_source) + output.flush() + return fixed_source + + +def global_fixes(): + """Yield multiple (code, function) tuples.""" + for function in list(globals().values()): + if inspect.isfunction(function): + arguments = _get_parameters(function) + if arguments[:1] != ['source']: + continue + + code = extract_code_from_function(function) + if code: + yield (code, function) + + +def _get_parameters(function): + # pylint: disable=deprecated-method + if sys.version_info.major >= 3: + # We need to match "getargspec()", which includes "self" as the first + # value for methods. + # https://bugs.python.org/issue17481#msg209469 + if inspect.ismethod(function): + function = function.__func__ + + return list(inspect.signature(function).parameters) + else: + return inspect.getargspec(function)[0] + + +def apply_global_fixes(source, options, where='global', filename='', + codes=None): + """Run global fixes on source code. + + These are fixes that only need be done once (unlike those in + FixPEP8, which are dependent on pycodestyle). + + """ + if codes is None: + codes = [] + if any(code_match(code, select=options.select, ignore=options.ignore) + for code in ['E101', 'E111']): + source = reindent(source, + indent_size=options.indent_size) + + for (code, function) in global_fixes(): + if code.upper() in SELECTED_GLOBAL_FIXED_METHOD_CODES \ + and code.upper() not in codes: + continue + if code_match(code, select=options.select, ignore=options.ignore): + if options.verbose: + print('---> Applying {} fix for {}'.format(where, + code.upper()), + file=sys.stderr) + source = function(source, + aggressive=options.aggressive) + + source = fix_2to3(source, + aggressive=options.aggressive, + select=options.select, + ignore=options.ignore, + filename=filename, + where=where, + verbose=options.verbose) + + return source + + +def extract_code_from_function(function): + """Return code handled by function.""" + if not function.__name__.startswith('fix_'): + return None + + code = re.sub('^fix_', '', function.__name__) + if not code: + return None + + try: + int(code[1:]) + except ValueError: + return None + + return code + + +def _get_package_version(): + packages = ["pycodestyle: {}".format(pycodestyle.__version__)] + return ", ".join(packages) + + +def create_parser(): + """Return command-line parser.""" + parser = argparse.ArgumentParser(description=docstring_summary(__doc__), + prog='autopep8') + parser.add_argument('--version', action='version', + version='%(prog)s {} ({})'.format( + __version__, _get_package_version())) + parser.add_argument('-v', '--verbose', action='count', + default=0, + help='print verbose messages; ' + 'multiple -v result in more verbose messages') + parser.add_argument('-d', '--diff', action='store_true', + help='print the diff for the fixed source') + parser.add_argument('-i', '--in-place', action='store_true', + help='make changes to files in place') + parser.add_argument('--global-config', metavar='filename', + default=DEFAULT_CONFIG, + help='path to a global pep8 config file; if this file ' + 'does not exist then this is ignored ' + '(default: {})'.format(DEFAULT_CONFIG)) + parser.add_argument('--ignore-local-config', action='store_true', + help="don't look for and apply local config files; " + 'if not passed, defaults are updated with any ' + "config files in the project's root directory") + parser.add_argument('-r', '--recursive', action='store_true', + help='run recursively over directories; ' + 'must be used with --in-place or --diff') + parser.add_argument('-j', '--jobs', type=int, metavar='n', default=1, + help='number of parallel jobs; ' + 'match CPU count if value is less than 1') + parser.add_argument('-p', '--pep8-passes', metavar='n', + default=-1, type=int, + help='maximum number of additional pep8 passes ' + '(default: infinite)') + parser.add_argument('-a', '--aggressive', action='count', default=0, + help='enable non-whitespace changes; ' + 'multiple -a result in more aggressive changes') + parser.add_argument('--experimental', action='store_true', + help='enable experimental fixes') + parser.add_argument('--exclude', metavar='globs', + help='exclude file/directory names that match these ' + 'comma-separated globs') + parser.add_argument('--list-fixes', action='store_true', + help='list codes for fixes; ' + 'used by --ignore and --select') + parser.add_argument('--ignore', metavar='errors', default='', + help='do not fix these errors/warnings ' + '(default: {})'.format(DEFAULT_IGNORE)) + parser.add_argument('--select', metavar='errors', default='', + help='fix only these errors/warnings (e.g. E4,W)') + parser.add_argument('--max-line-length', metavar='n', default=79, type=int, + help='set maximum allowed line length ' + '(default: %(default)s)') + parser.add_argument('--line-range', '--range', metavar='line', + default=None, type=int, nargs=2, + help='only fix errors found within this inclusive ' + 'range of line numbers (e.g. 1 99); ' + 'line numbers are indexed at 1') + parser.add_argument('--indent-size', default=DEFAULT_INDENT_SIZE, + type=int, help=argparse.SUPPRESS) + parser.add_argument('--hang-closing', action='store_true', + help='hang-closing option passed to pycodestyle') + parser.add_argument('--exit-code', action='store_true', + help='change to behavior of exit code.' + ' default behavior of return value, 0 is no ' + 'differences, 1 is error exit. return 2 when' + ' add this option. 2 is exists differences.') + parser.add_argument('files', nargs='*', + help="files to format or '-' for standard in") + + return parser + + +def _expand_codes(codes, ignore_codes): + """expand to individual E/W codes""" + ret = set() + + is_conflict = False + if all( + any( + conflicting_code.startswith(code) + for code in codes + ) + for conflicting_code in CONFLICTING_CODES + ): + is_conflict = True + + is_ignore_w503 = "W503" in ignore_codes + is_ignore_w504 = "W504" in ignore_codes + + for code in codes: + if code == "W": + if is_ignore_w503 and is_ignore_w504: + ret.update({"W1", "W2", "W3", "W505", "W6"}) + elif is_ignore_w503: + ret.update({"W1", "W2", "W3", "W504", "W505", "W6"}) + else: + ret.update({"W1", "W2", "W3", "W503", "W505", "W6"}) + elif code in ("W5", "W50"): + if is_ignore_w503 and is_ignore_w504: + ret.update({"W505"}) + elif is_ignore_w503: + ret.update({"W504", "W505"}) + else: + ret.update({"W503", "W505"}) + elif not (code in ("W503", "W504") and is_conflict): + ret.add(code) + + return ret + + +def parse_args(arguments, apply_config=False): + """Parse command-line options.""" + parser = create_parser() + args = parser.parse_args(arguments) + + if not args.files and not args.list_fixes: + parser.error('incorrect number of arguments') + + args.files = [decode_filename(name) for name in args.files] + + if apply_config: + parser = read_config(args, parser) + # prioritize settings when exist pyproject.toml's tool.autopep8 section + try: + parser_with_pyproject_toml = read_pyproject_toml(args, parser) + except Exception: + parser_with_pyproject_toml = None + if parser_with_pyproject_toml: + parser = parser_with_pyproject_toml + args = parser.parse_args(arguments) + args.files = [decode_filename(name) for name in args.files] + + if '-' in args.files: + if len(args.files) > 1: + parser.error('cannot mix stdin and regular files') + + if args.diff: + parser.error('--diff cannot be used with standard input') + + if args.in_place: + parser.error('--in-place cannot be used with standard input') + + if args.recursive: + parser.error('--recursive cannot be used with standard input') + + if len(args.files) > 1 and not (args.in_place or args.diff): + parser.error('autopep8 only takes one filename as argument ' + 'unless the "--in-place" or "--diff" args are ' + 'used') + + if args.recursive and not (args.in_place or args.diff): + parser.error('--recursive must be used with --in-place or --diff') + + if args.in_place and args.diff: + parser.error('--in-place and --diff are mutually exclusive') + + if args.max_line_length <= 0: + parser.error('--max-line-length must be greater than 0') + + if args.select: + args.select = _expand_codes( + _split_comma_separated(args.select), + (_split_comma_separated(args.ignore) if args.ignore else []) + ) + + if args.ignore: + args.ignore = _split_comma_separated(args.ignore) + if all( + not any( + conflicting_code.startswith(ignore_code) + for ignore_code in args.ignore + ) + for conflicting_code in CONFLICTING_CODES + ): + args.ignore.update(CONFLICTING_CODES) + elif not args.select: + if args.aggressive: + # Enable everything by default if aggressive. + args.select = {'E', 'W1', 'W2', 'W3', 'W6'} + else: + args.ignore = _split_comma_separated(DEFAULT_IGNORE) + + if args.exclude: + args.exclude = _split_comma_separated(args.exclude) + else: + args.exclude = {} + + if args.jobs < 1: + # Do not import multiprocessing globally in case it is not supported + # on the platform. + import multiprocessing + args.jobs = multiprocessing.cpu_count() + + if args.jobs > 1 and not (args.in_place or args.diff): + parser.error('parallel jobs requires --in-place') + + if args.line_range: + if args.line_range[0] <= 0: + parser.error('--range must be positive numbers') + if args.line_range[0] > args.line_range[1]: + parser.error('First value of --range should be less than or equal ' + 'to the second') + + return args + + +def _get_normalize_options(config, section, option_list): + for (k, _) in config.items(section): + norm_opt = k.lstrip('-').replace('-', '_') + if not option_list.get(norm_opt): + continue + opt_type = option_list[norm_opt] + if opt_type is int: + value = config.getint(section, k) + elif opt_type is bool: + value = config.getboolean(section, k) + else: + value = config.get(section, k) + yield norm_opt, k, value + + +def read_config(args, parser): + """Read both user configuration and local configuration.""" + config = SafeConfigParser() + + try: + config.read(args.global_config) + + if not args.ignore_local_config: + parent = tail = args.files and os.path.abspath( + os.path.commonprefix(args.files)) + while tail: + if config.read([os.path.join(parent, fn) + for fn in PROJECT_CONFIG]): + break + (parent, tail) = os.path.split(parent) + + defaults = {} + option_list = {o.dest: o.type or type(o.default) + for o in parser._actions} + + for section in ['pep8', 'pycodestyle', 'flake8']: + if not config.has_section(section): + continue + for norm_opt, k, value in _get_normalize_options(config, section, + option_list): + if args.verbose: + print("enable config: section={}, key={}, value={}".format( + section, k, value)) + defaults[norm_opt] = value + + parser.set_defaults(**defaults) + except Error: + # Ignore for now. + pass + + return parser + + +def read_pyproject_toml(args, parser): + """Read pyproject.toml and load configuration.""" + config = None + + if os.path.exists(args.global_config): + with open(args.global_config) as fp: + config = toml.load(fp) + + if not args.ignore_local_config: + parent = tail = args.files and os.path.abspath( + os.path.commonprefix(args.files)) + while tail: + pyproject_toml = os.path.join(parent, "pyproject.toml") + if os.path.exists(pyproject_toml): + with open(pyproject_toml) as fp: + config = toml.load(fp) + break + (parent, tail) = os.path.split(parent) + + if not config: + return None + + if config.get("tool", {}).get("autopep8") is None: + return None + + config = config.get("tool").get("autopep8") + + defaults = {} + option_list = {o.dest: o.type or type(o.default) + for o in parser._actions} + + TUPLED_OPTIONS = ("ignore", "select") + for (k, v) in config.items(): + norm_opt = k.lstrip('-').replace('-', '_') + if not option_list.get(norm_opt): + continue + if type(v) in (list, tuple) and norm_opt in TUPLED_OPTIONS: + value = ",".join(v) + else: + value = v + if args.verbose: + print("enable pyproject.toml config: " + "key={}, value={}".format(k, value)) + defaults[norm_opt] = value + + if defaults: + # set value when exists key-value in defaults dict + parser.set_defaults(**defaults) + + return parser + + +def _split_comma_separated(string): + """Return a set of strings.""" + return {text.strip() for text in string.split(',') if text.strip()} + + +def decode_filename(filename): + """Return Unicode filename.""" + if isinstance(filename, unicode): + return filename + + return filename.decode(sys.getfilesystemencoding()) + + +def supported_fixes(): + """Yield pep8 error codes that autopep8 fixes. + + Each item we yield is a tuple of the code followed by its + description. + + """ + yield ('E101', docstring_summary(reindent.__doc__)) + + instance = FixPEP8(filename=None, options=None, contents='') + for attribute in dir(instance): + code = re.match('fix_([ew][0-9][0-9][0-9])', attribute) + if code: + yield ( + code.group(1).upper(), + re.sub(r'\s+', ' ', + docstring_summary(getattr(instance, attribute).__doc__)) + ) + + for (code, function) in sorted(global_fixes()): + yield (code.upper() + (4 - len(code)) * ' ', + re.sub(r'\s+', ' ', docstring_summary(function.__doc__))) + + for code in sorted(CODE_TO_2TO3): + yield (code.upper() + (4 - len(code)) * ' ', + re.sub(r'\s+', ' ', docstring_summary(fix_2to3.__doc__))) + + +def docstring_summary(docstring): + """Return summary of docstring.""" + return docstring.split('\n')[0] if docstring else '' + + +def line_shortening_rank(candidate, indent_word, max_line_length, + experimental=False): + """Return rank of candidate. + + This is for sorting candidates. + + """ + if not candidate.strip(): + return 0 + + rank = 0 + lines = candidate.rstrip().split('\n') + + offset = 0 + if ( + not lines[0].lstrip().startswith('#') and + lines[0].rstrip()[-1] not in '([{' + ): + for (opening, closing) in ('()', '[]', '{}'): + # Don't penalize empty containers that aren't split up. Things like + # this "foo(\n )" aren't particularly good. + opening_loc = lines[0].find(opening) + closing_loc = lines[0].find(closing) + if opening_loc >= 0: + if closing_loc < 0 or closing_loc != opening_loc + 1: + offset = max(offset, 1 + opening_loc) + + current_longest = max(offset + len(x.strip()) for x in lines) + + rank += 4 * max(0, current_longest - max_line_length) + + rank += len(lines) + + # Too much variation in line length is ugly. + rank += 2 * standard_deviation(len(line) for line in lines) + + bad_staring_symbol = { + '(': ')', + '[': ']', + '{': '}'}.get(lines[0][-1]) + + if len(lines) > 1: + if ( + bad_staring_symbol and + lines[1].lstrip().startswith(bad_staring_symbol) + ): + rank += 20 + + for lineno, current_line in enumerate(lines): + current_line = current_line.strip() + + if current_line.startswith('#'): + continue + + for bad_start in ['.', '%', '+', '-', '/']: + if current_line.startswith(bad_start): + rank += 100 + + # Do not tolerate operators on their own line. + if current_line == bad_start: + rank += 1000 + + if ( + current_line.endswith(('.', '%', '+', '-', '/')) and + "': " in current_line + ): + rank += 1000 + + if current_line.endswith(('(', '[', '{', '.')): + # Avoid lonely opening. They result in longer lines. + if len(current_line) <= len(indent_word): + rank += 100 + + # Avoid the ugliness of ", (\n". + if ( + current_line.endswith('(') and + current_line[:-1].rstrip().endswith(',') + ): + rank += 100 + + # Avoid the ugliness of "something[\n" and something[index][\n. + if ( + current_line.endswith('[') and + len(current_line) > 1 and + (current_line[-2].isalnum() or current_line[-2] in ']') + ): + rank += 300 + + # Also avoid the ugliness of "foo.\nbar" + if current_line.endswith('.'): + rank += 100 + + if has_arithmetic_operator(current_line): + rank += 100 + + # Avoid breaking at unary operators. + if re.match(r'.*[(\[{]\s*[\-\+~]$', current_line.rstrip('\\ ')): + rank += 1000 + + if re.match(r'.*lambda\s*\*$', current_line.rstrip('\\ ')): + rank += 1000 + + if current_line.endswith(('%', '(', '[', '{')): + rank -= 20 + + # Try to break list comprehensions at the "for". + if current_line.startswith('for '): + rank -= 50 + + if current_line.endswith('\\'): + # If a line ends in \-newline, it may be part of a + # multiline string. In that case, we would like to know + # how long that line is without the \-newline. If it's + # longer than the maximum, or has comments, then we assume + # that the \-newline is an okay candidate and only + # penalize it a bit. + total_len = len(current_line) + lineno += 1 + while lineno < len(lines): + total_len += len(lines[lineno]) + + if lines[lineno].lstrip().startswith('#'): + total_len = max_line_length + break + + if not lines[lineno].endswith('\\'): + break + + lineno += 1 + + if total_len < max_line_length: + rank += 10 + else: + rank += 100 if experimental else 1 + + # Prefer breaking at commas rather than colon. + if ',' in current_line and current_line.endswith(':'): + rank += 10 + + # Avoid splitting dictionaries between key and value. + if current_line.endswith(':'): + rank += 100 + + rank += 10 * count_unbalanced_brackets(current_line) + + return max(0, rank) + + +def standard_deviation(numbers): + """Return standard deviation.""" + numbers = list(numbers) + if not numbers: + return 0 + mean = sum(numbers) / len(numbers) + return (sum((n - mean) ** 2 for n in numbers) / + len(numbers)) ** .5 + + +def has_arithmetic_operator(line): + """Return True if line contains any arithmetic operators.""" + for operator in pycodestyle.ARITHMETIC_OP: + if operator in line: + return True + + return False + + +def count_unbalanced_brackets(line): + """Return number of unmatched open/close brackets.""" + count = 0 + for opening, closing in ['()', '[]', '{}']: + count += abs(line.count(opening) - line.count(closing)) + + return count + + +def split_at_offsets(line, offsets): + """Split line at offsets. + + Return list of strings. + + """ + result = [] + + previous_offset = 0 + current_offset = 0 + for current_offset in sorted(offsets): + if current_offset < len(line) and previous_offset != current_offset: + result.append(line[previous_offset:current_offset].strip()) + previous_offset = current_offset + + result.append(line[current_offset:]) + + return result + + +class LineEndingWrapper(object): + + r"""Replace line endings to work with sys.stdout. + + It seems that sys.stdout expects only '\n' as the line ending, no matter + the platform. Otherwise, we get repeated line endings. + + """ + + def __init__(self, output): + self.__output = output + + def write(self, s): + self.__output.write(s.replace('\r\n', '\n').replace('\r', '\n')) + + def flush(self): + self.__output.flush() + + +def match_file(filename, exclude): + """Return True if file is okay for modifying/recursing.""" + base_name = os.path.basename(filename) + + if base_name.startswith('.'): + return False + + for pattern in exclude: + if fnmatch.fnmatch(base_name, pattern): + return False + if fnmatch.fnmatch(filename, pattern): + return False + + if not os.path.isdir(filename) and not is_python_file(filename): + return False + + return True + + +def find_files(filenames, recursive, exclude): + """Yield filenames.""" + while filenames: + name = filenames.pop(0) + if recursive and os.path.isdir(name): + for root, directories, children in os.walk(name): + filenames += [os.path.join(root, f) for f in children + if match_file(os.path.join(root, f), + exclude)] + directories[:] = [d for d in directories + if match_file(os.path.join(root, d), + exclude)] + else: + is_exclude_match = False + for pattern in exclude: + if fnmatch.fnmatch(name, pattern): + is_exclude_match = True + break + if not is_exclude_match: + yield name + + +def _fix_file(parameters): + """Helper function for optionally running fix_file() in parallel.""" + if parameters[1].verbose: + print('[file:{}]'.format(parameters[0]), file=sys.stderr) + try: + return fix_file(*parameters) + except IOError as error: + print(unicode(error), file=sys.stderr) + + +def fix_multiple_files(filenames, options, output=None): + """Fix list of files. + + Optionally fix files recursively. + + """ + results = [] + filenames = find_files(filenames, options.recursive, options.exclude) + if options.jobs > 1: + import multiprocessing + pool = multiprocessing.Pool(options.jobs) + ret = pool.map(_fix_file, [(name, options) for name in filenames]) + if options.diff: + for r in ret: + sys.stdout.write(r.decode()) + sys.stdout.flush() + results.extend([x for x in ret if x is not None]) + else: + for name in filenames: + ret = _fix_file((name, options, output)) + if ret is None: + continue + if options.diff: + if ret != '': + results.append(ret) + elif options.in_place: + results.append(ret) + else: + original_source = readlines_from_file(name) + if "".join(original_source).splitlines() != ret.splitlines(): + results.append(ret) + return results + + +def is_python_file(filename): + """Return True if filename is Python file.""" + if filename.endswith('.py'): + return True + + try: + with open_with_encoding( + filename, + limit_byte_check=MAX_PYTHON_FILE_DETECTION_BYTES) as f: + text = f.read(MAX_PYTHON_FILE_DETECTION_BYTES) + if not text: + return False + first_line = text.splitlines()[0] + except (IOError, IndexError): + return False + + if not PYTHON_SHEBANG_REGEX.match(first_line): + return False + + return True + + +def is_probably_part_of_multiline(line): + """Return True if line is likely part of a multiline string. + + When multiline strings are involved, pep8 reports the error as being + at the start of the multiline string, which doesn't work for us. + + """ + return ( + '"""' in line or + "'''" in line or + line.rstrip().endswith('\\') + ) + + +def wrap_output(output, encoding): + """Return output with specified encoding.""" + return codecs.getwriter(encoding)(output.buffer + if hasattr(output, 'buffer') + else output) + + +def get_encoding(): + """Return preferred encoding.""" + return locale.getpreferredencoding() or sys.getdefaultencoding() + + +def main(argv=None, apply_config=True): + """Command-line entry.""" + if argv is None: + argv = sys.argv + + try: + # Exit on broken pipe. + signal.signal(signal.SIGPIPE, signal.SIG_DFL) + except AttributeError: # pragma: no cover + # SIGPIPE is not available on Windows. + pass + + try: + args = parse_args(argv[1:], apply_config=apply_config) + + if args.list_fixes: + for code, description in sorted(supported_fixes()): + print('{code} - {description}'.format( + code=code, description=description)) + return EXIT_CODE_OK + + if args.files == ['-']: + assert not args.in_place + + encoding = sys.stdin.encoding or get_encoding() + read_stdin = sys.stdin.read() + fixed_stdin = fix_code(read_stdin, args, encoding=encoding) + + # LineEndingWrapper is unnecessary here due to the symmetry between + # standard in and standard out. + wrap_output(sys.stdout, encoding=encoding).write(fixed_stdin) + + if hash(read_stdin) != hash(fixed_stdin): + if args.exit_code: + return EXIT_CODE_EXISTS_DIFF + else: + if args.in_place or args.diff: + args.files = list(set(args.files)) + else: + assert len(args.files) == 1 + assert not args.recursive + + results = fix_multiple_files(args.files, args, sys.stdout) + if args.diff: + ret = any([len(ret) != 0 for ret in results]) + else: + # with in-place option + ret = any([ret is not None for ret in results]) + if args.exit_code and ret: + return EXIT_CODE_EXISTS_DIFF + except KeyboardInterrupt: + return EXIT_CODE_ERROR # pragma: no cover + + +class CachedTokenizer(object): + + """A one-element cache around tokenize.generate_tokens(). + + Original code written by Ned Batchelder, in coverage.py. + + """ + + def __init__(self): + self.last_text = None + self.last_tokens = None + + def generate_tokens(self, text): + """A stand-in for tokenize.generate_tokens().""" + if text != self.last_text: + string_io = io.StringIO(text) + self.last_tokens = list( + tokenize.generate_tokens(string_io.readline) + ) + self.last_text = text + return self.last_tokens + + +_cached_tokenizer = CachedTokenizer() +generate_tokens = _cached_tokenizer.generate_tokens + + +if __name__ == '__main__': + sys.exit(main()) diff --git a/env/lib/python3.8/site-packages/easy_install.py b/env/lib/python3.8/site-packages/easy_install.py new file mode 100644 index 000000000..d87e98403 --- /dev/null +++ b/env/lib/python3.8/site-packages/easy_install.py @@ -0,0 +1,5 @@ +"""Run the EasyInstall command""" + +if __name__ == '__main__': + from setuptools.command.easy_install import main + main() diff --git a/env/lib/python3.8/site-packages/isort-5.5.3.dist-info/INSTALLER b/env/lib/python3.8/site-packages/isort-5.5.3.dist-info/INSTALLER new file mode 100644 index 000000000..a1b589e38 --- /dev/null +++ b/env/lib/python3.8/site-packages/isort-5.5.3.dist-info/INSTALLER @@ -0,0 +1 @@ +pip diff --git a/env/lib/python3.8/site-packages/isort-5.5.3.dist-info/LICENSE b/env/lib/python3.8/site-packages/isort-5.5.3.dist-info/LICENSE new file mode 100644 index 000000000..b5083a50d --- /dev/null +++ b/env/lib/python3.8/site-packages/isort-5.5.3.dist-info/LICENSE @@ -0,0 +1,21 @@ +The MIT License (MIT) + +Copyright (c) 2013 Timothy Edmund Crosley + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in +all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +THE SOFTWARE. diff --git a/env/lib/python3.8/site-packages/isort-5.5.3.dist-info/METADATA b/env/lib/python3.8/site-packages/isort-5.5.3.dist-info/METADATA new file mode 100644 index 000000000..14d07a5e4 --- /dev/null +++ b/env/lib/python3.8/site-packages/isort-5.5.3.dist-info/METADATA @@ -0,0 +1,713 @@ +Metadata-Version: 2.1 +Name: isort +Version: 5.5.3 +Summary: A Python utility / library to sort Python imports. +Home-page: https://pycqa.github.io/isort/ +License: MIT +Keywords: Refactor,Lint,Imports,Sort,Clean +Author: Timothy Crosley +Author-email: timothy.crosley@gmail.com +Requires-Python: >=3.6,<4.0 +Classifier: Development Status :: 6 - Mature +Classifier: Environment :: Console +Classifier: Intended Audience :: Developers +Classifier: License :: OSI Approved :: MIT License +Classifier: Natural Language :: English +Classifier: Programming Language :: Python +Classifier: Programming Language :: Python :: 3 +Classifier: Programming Language :: Python :: 3 :: Only +Classifier: Programming Language :: Python :: 3.6 +Classifier: Programming Language :: Python :: 3.7 +Classifier: Programming Language :: Python :: 3.8 +Classifier: Programming Language :: Python :: 3.9 +Classifier: Programming Language :: Python :: Implementation :: CPython +Classifier: Programming Language :: Python :: Implementation :: PyPy +Classifier: Topic :: Software Development :: Libraries +Classifier: Topic :: Utilities +Provides-Extra: colors +Provides-Extra: pipfile_deprecated_finder +Provides-Extra: requirements_deprecated_finder +Requires-Dist: colorama (>=0.4.3,<0.5.0); extra == "colors" +Requires-Dist: pip-api; extra == "requirements_deprecated_finder" +Requires-Dist: pipreqs; extra == "pipfile_deprecated_finder" or extra == "requirements_deprecated_finder" +Requires-Dist: requirementslib; extra == "pipfile_deprecated_finder" +Project-URL: Changelog, https://github.com/pycqa/isort/blob/master/CHANGELOG.md +Project-URL: Documentation, https://pycqa.github.io/isort/ +Project-URL: Repository, https://github.com/pycqa/isort +Description-Content-Type: text/markdown + +[![isort - isort your imports, so you don't have to.](https://raw.githubusercontent.com/pycqa/isort/develop/art/logo_large.png)](https://pycqa.github.io/isort/) + +------------------------------------------------------------------------ + +[![PyPI version](https://badge.fury.io/py/isort.svg)](https://badge.fury.io/py/isort) +[![Test Status](https://github.com/pycqa/isort/workflows/Test/badge.svg?branch=develop)](https://github.com/pycqa/isort/actions?query=workflow%3ATest) +[![Lint Status](https://github.com/pycqa/isort/workflows/Lint/badge.svg?branch=develop)](https://github.com/pycqa/isort/actions?query=workflow%3ALint) +[![Code coverage Status](https://codecov.io/gh/pycqa/isort/branch/develop/graph/badge.svg)](https://codecov.io/gh/pycqa/isort) +[![Maintainability](https://api.codeclimate.com/v1/badges/060372d3e77573072609/maintainability)](https://codeclimate.com/github/timothycrosley/isort/maintainability) +[![License](https://img.shields.io/github/license/mashape/apistatus.svg)](https://pypi.org/project/isort/) +[![Join the chat at https://gitter.im/timothycrosley/isort](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/timothycrosley/isort?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) +[![Downloads](https://pepy.tech/badge/isort)](https://pepy.tech/project/isort) +[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black) +[![Imports: isort](https://img.shields.io/badge/%20imports-isort-%231674b1?style=flat&labelColor=ef8336)](https://pycqa.github.io/isort/) +[![DeepSource](https://static.deepsource.io/deepsource-badge-light-mini.svg)](https://deepsource.io/gh/timothycrosley/isort/?ref=repository-badge) +_________________ + +[Read Latest Documentation](https://pycqa.github.io/isort/) - [Browse GitHub Code Repository](https://github.com/pycqa/isort/) +_________________ + +isort your imports, so you don't have to. + +isort is a Python utility / library to sort imports alphabetically, and +automatically separated into sections and by type. It provides a command line +utility, Python library and [plugins for various +editors](https://github.com/pycqa/isort/wiki/isort-Plugins) to +quickly sort all your imports. It requires Python 3.6+ to run but +supports formatting Python 2 code too. + +[Try isort now from your browser!](https://pycqa.github.io/isort/docs/quick_start/0.-try/) + +![Example Usage](https://raw.github.com/pycqa/isort/develop/example.gif) + +Before isort: + +```python +from my_lib import Object + +import os + +from my_lib import Object3 + +from my_lib import Object2 + +import sys + +from third_party import lib15, lib1, lib2, lib3, lib4, lib5, lib6, lib7, lib8, lib9, lib10, lib11, lib12, lib13, lib14 + +import sys + +from __future__ import absolute_import + +from third_party import lib3 + +print("Hey") +print("yo") +``` + +After isort: + +```python +from __future__ import absolute_import + +import os +import sys + +from third_party import (lib1, lib2, lib3, lib4, lib5, lib6, lib7, lib8, + lib9, lib10, lib11, lib12, lib13, lib14, lib15) + +from my_lib import Object, Object2, Object3 + +print("Hey") +print("yo") +``` + +## Installing isort + +Installing isort is as simple as: + +```bash +pip install isort +``` + +Install isort with requirements.txt support: + +```bash +pip install isort[requirements_deprecated_finder] +``` + +Install isort with Pipfile support: + +```bash +pip install isort[pipfile_deprecated_finder] +``` + +Install isort with both formats support: + +```bash +pip install isort[requirements_deprecated_finder,pipfile_deprecated_finder] +``` + +## Using isort + +**From the command line**: + +```bash +isort mypythonfile.py mypythonfile2.py +``` + +or recursively: + +```bash +isort . +``` + +*which is equivalent to:* + +```bash +isort **/*.py +``` + +or to see the proposed changes without applying them: + +```bash +isort mypythonfile.py --diff +``` + +Finally, to atomically run isort against a project, only applying +changes if they don't introduce syntax errors do: + +```bash +isort --atomic . +``` + +(Note: this is disabled by default as it keeps isort from being able to +run against code written using a different version of Python) + +**From within Python**: + +```bash +import isort + +isort.file("pythonfile.py") +``` + +or: + +```bash +import isort + +sorted_code = isort.code("import b\nimport a\n") +``` + +## Installing isort's for your preferred text editor + +Several plugins have been written that enable to use isort from within a +variety of text-editors. You can find a full list of them [on the isort +wiki](https://github.com/pycqa/isort/wiki/isort-Plugins). +Additionally, I will enthusiastically accept pull requests that include +plugins for other text editors and add documentation for them as I am +notified. + +## Multi line output modes + +You will notice above the \"multi\_line\_output\" setting. This setting +defines how from imports wrap when they extend past the line\_length +limit and has 12 possible settings: + +**0 - Grid** + +```python +from third_party import (lib1, lib2, lib3, + lib4, lib5, ...) +``` + +**1 - Vertical** + +```python +from third_party import (lib1, + lib2, + lib3 + lib4, + lib5, + ...) +``` + +**2 - Hanging Indent** + +```python +from third_party import \ + lib1, lib2, lib3, \ + lib4, lib5, lib6 +``` + +**3 - Vertical Hanging Indent** + +```python +from third_party import ( + lib1, + lib2, + lib3, + lib4, +) +``` + +**4 - Hanging Grid** + +```python +from third_party import ( + lib1, lib2, lib3, lib4, + lib5, ...) +``` + +**5 - Hanging Grid Grouped** + +```python +from third_party import ( + lib1, lib2, lib3, lib4, + lib5, ... +) +``` + +**6 - Hanging Grid Grouped, No Trailing Comma** + +In Mode 5 isort leaves a single extra space to maintain consistency of +output when a comma is added at the end. Mode 6 is the same - except +that no extra space is maintained leading to the possibility of lines +one character longer. You can enforce a trailing comma by using this in +conjunction with `-tc` or `include_trailing_comma: True`. + +```python +from third_party import ( + lib1, lib2, lib3, lib4, + lib5 +) +``` + +**7 - NOQA** + +```python +from third_party import lib1, lib2, lib3, ... # NOQA +``` + +Alternatively, you can set `force_single_line` to `True` (`-sl` on the +command line) and every import will appear on its own line: + +```python +from third_party import lib1 +from third_party import lib2 +from third_party import lib3 +... +``` + +**8 - Vertical Hanging Indent Bracket** + +Same as Mode 3 - _Vertical Hanging Indent_ but the closing parentheses +on the last line is indented. + +```python +from third_party import ( + lib1, + lib2, + lib3, + lib4, + ) +``` + +**9 - Vertical Prefix From Module Import** + +Starts a new line with the same `from MODULE import ` prefix when lines are longer than the line length limit. + +```python +from third_party import lib1, lib2, lib3 +from third_party import lib4, lib5, lib6 +``` + +**10 - Hanging Indent With Parentheses** + +Same as Mode 2 - _Hanging Indent_ but uses parentheses instead of backslash +for wrapping long lines. + +```python +from third_party import ( + lib1, lib2, lib3, + lib4, lib5, lib6) +``` + +**11 - Backslash Grid** + +Same as Mode 0 - _Grid_ but uses backslashes instead of parentheses to group imports. + +```python +from third_party import lib1, lib2, lib3, \ + lib4, lib5 +``` + +## Indentation + +To change the how constant indents appear - simply change the +indent property with the following accepted formats: + +- Number of spaces you would like. For example: 4 would cause standard + 4 space indentation. +- Tab +- A verbatim string with quotes around it. + +For example: + +```python +" " +``` + +is equivalent to 4. + +For the import styles that use parentheses, you can control whether or +not to include a trailing comma after the last import with the +`include_trailing_comma` option (defaults to `False`). + +## Intelligently Balanced Multi-line Imports + +As of isort 3.1.0 support for balanced multi-line imports has been +added. With this enabled isort will dynamically change the import length +to the one that produces the most balanced grid, while staying below the +maximum import length defined. + +Example: + +```python +from __future__ import (absolute_import, division, + print_function, unicode_literals) +``` + +Will be produced instead of: + +```python +from __future__ import (absolute_import, division, print_function, + unicode_literals) +``` + +To enable this set `balanced_wrapping` to `True` in your config or pass +the `-e` option into the command line utility. + +## Custom Sections and Ordering + +You can change the section order with `sections` option from the default +of: + +```ini +FUTURE,STDLIB,THIRDPARTY,FIRSTPARTY,LOCALFOLDER +``` + +to your preference: + +```ini +sections=FUTURE,STDLIB,FIRSTPARTY,THIRDPARTY,LOCALFOLDER +``` + +You also can define your own sections and their order. + +Example: + +```ini +known_django=django +known_pandas=pandas,numpy +sections=FUTURE,STDLIB,DJANGO,THIRDPARTY,PANDAS,FIRSTPARTY,LOCALFOLDER +``` + +would create two new sections with the specified known modules. + +The `no_lines_before` option will prevent the listed sections from being +split from the previous section by an empty line. + +Example: + +```ini +sections=FUTURE,STDLIB,THIRDPARTY,FIRSTPARTY,LOCALFOLDER +no_lines_before=LOCALFOLDER +``` + +would produce a section with both FIRSTPARTY and LOCALFOLDER modules +combined. + +**IMPORTANT NOTE**: It is very important to know when setting `known` sections that the naming +does not directly map for historical reasons. For custom settings, the only difference is +capitalization (`known_custom=custom` VS `sections=CUSTOM,...`) for all others reference the +following mapping: + + - `known_standard_library` : `STANDARD_LIBRARY` + - `extra_standard_library` : `STANDARD_LIBRARY` # Like known standard library but appends instead of replacing + - `known_future_library` : `FUTURE` + - `known_first_party`: `FIRSTPARTY` + - `known_third_party`: `THIRDPARTY` + - `known_local_folder`: `LOCALFOLDER` + +This will likely be changed in isort 6.0.0+ in a backwards compatible way. + +## Auto-comment import sections + +Some projects prefer to have import sections uniquely titled to aid in +identifying the sections quickly when visually scanning. isort can +automate this as well. To do this simply set the +`import_heading_{section_name}` setting for each section you wish to +have auto commented - to the desired comment. + +For Example: + +```ini +import_heading_stdlib=Standard Library +import_heading_firstparty=My Stuff +``` + +Would lead to output looking like the following: + +```python +# Standard Library +import os +import sys + +import django.settings + +# My Stuff +import myproject.test +``` + +## Ordering by import length + +isort also makes it easy to sort your imports by length, simply by +setting the `length_sort` option to `True`. This will result in the +following output style: + +```python +from evn.util import ( + Pool, + Dict, + Options, + Constant, + DecayDict, + UnexpectedCodePath, +) +``` + +It is also possible to opt-in to sorting imports by length for only +specific sections by using `length_sort_` followed by the section name +as a configuration item, e.g.: + + length_sort_stdlib=1 + +## Controlling how isort sections `from` imports + +By default isort places straight (`import y`) imports above from imports (`from x import y`): + +```python +import b +from a import a # This will always appear below because it is a from import. +``` + +However, if you prefer to keep strict alphabetical sorting you can set [force sort within sections](https://pycqa.github.io/isort/docs/configuration/options/#force-sort-within-sections) to true. Resulting in: + + +```python +from a import a # This will now appear at top because a appears in the alphabet before b +import b +``` + +You can even tell isort to always place from imports on top, instead of the default of placing them on bottom, using [from first](https://pycqa.github.io/isort/docs/configuration/options/#from-first). + +```python +from b import b # If from first is set to True, all from imports will be placed before non-from imports. +import a +``` + +## Skip processing of imports (outside of configuration) + +To make isort ignore a single import simply add a comment at the end of +the import line containing the text `isort:skip`: + +```python +import module # isort:skip +``` + +or: + +```python +from xyz import (abc, # isort:skip + yo, + hey) +``` + +To make isort skip an entire file simply add `isort:skip_file` to the +module's doc string: + +```python +""" my_module.py + Best module ever + + isort:skip_file +""" + +import b +import a +``` + +## Adding an import to multiple files + +isort makes it easy to add an import statement across multiple files, +while being assured it's correctly placed. + +To add an import to all files: + +```bash +isort -a "from __future__ import print_function" *.py +``` + +To add an import only to files that already have imports: + +```bash +isort -a "from __future__ import print_function" --append-only *.py +``` + + +## Removing an import from multiple files + +isort also makes it easy to remove an import from multiple files, +without having to be concerned with how it was originally formatted. + +From the command line: + +```bash +isort --rm "os.system" *.py +``` + +## Using isort to verify code + +The `--check-only` option +------------------------- + +isort can also be used to used to verify that code is correctly +formatted by running it with `-c`. Any files that contain incorrectly +sorted and/or formatted imports will be outputted to `stderr`. + +```bash +isort **/*.py -c -v + +SUCCESS: /home/timothy/Projects/Open_Source/isort/isort_kate_plugin.py Everything Looks Good! +ERROR: /home/timothy/Projects/Open_Source/isort/isort/isort.py Imports are incorrectly sorted. +``` + +One great place this can be used is with a pre-commit git hook, such as +this one by \@acdha: + + + +This can help to ensure a certain level of code quality throughout a +project. + +Git hook +-------- + +isort provides a hook function that can be integrated into your Git +pre-commit script to check Python code before committing. + +To cause the commit to fail if there are isort errors (strict mode), +include the following in `.git/hooks/pre-commit`: + +```python +#!/usr/bin/env python +import sys +from isort.hooks import git_hook + +sys.exit(git_hook(strict=True, modify=True, lazy=True, settings_file="")) +``` + +If you just want to display warnings, but allow the commit to happen +anyway, call `git_hook` without the strict parameter. If you want to +display warnings, but not also fix the code, call `git_hook` without the +modify parameter. +The `lazy` argument is to support users who are "lazy" to add files +individually to the index and tend to use `git commit -a` instead. +Set it to `True` to ensure all tracked files are properly isorted, +leave it out or set it to `False` to check only files added to your +index. + +If you want to use a specific configuration file for the hook, you can pass its +path to settings_file. If no path is specifically requested, `git_hook` will +search for the configuration file starting at the directory containing the first +staged file, as per `git diff-index` ordering, and going upward in the directory +structure until a valid configuration file is found or +[`MAX_CONFIG_SEARCH_DEPTH`](src/config.py:35) directories are checked. +The settings_file parameter is used to support users who keep their configuration +file in a directory that might not be a parent of all the other files. + +## Setuptools integration + +Upon installation, isort enables a `setuptools` command that checks +Python files declared by your project. + +Running `python setup.py isort` on the command line will check the files +listed in your `py_modules` and `packages`. If any warning is found, the +command will exit with an error code: + +```bash +$ python setup.py isort +``` + +Also, to allow users to be able to use the command without having to +install isort themselves, add isort to the setup\_requires of your +`setup()` like so: + +```python +setup( + name="project", + packages=["project"], + + setup_requires=[ + "isort" + ] +) +``` + +## Spread the word + +[![Imports: isort](https://img.shields.io/badge/%20imports-isort-%231674b1?style=flat&labelColor=ef8336)](https://pycqa.github.io/isort/) + +Place this badge at the top of your repository to let others know your project uses isort. + +For README.md: + +```markdown +[![Imports: isort](https://img.shields.io/badge/%20imports-isort-%231674b1?style=flat&labelColor=ef8336)](https://pycqa.github.io/isort/) +``` + +Or README.rst: + +```rst +.. image:: https://img.shields.io/badge/%20imports-isort-%231674b1?style=flat&labelColor=ef8336 + :target: https://pycqa.github.io/isort/ +``` + +## Security contact information + +To report a security vulnerability, please use the [Tidelift security +contact](https://tidelift.com/security). Tidelift will coordinate the +fix and disclosure. + +## Why isort? + +isort simply stands for import sort. It was originally called +"sortImports" however I got tired of typing the extra characters and +came to the realization camelCase is not pythonic. + +I wrote isort because in an organization I used to work in the manager +came in one day and decided all code must have alphabetically sorted +imports. The code base was huge - and he meant for us to do it by hand. +However, being a programmer - I\'m too lazy to spend 8 hours mindlessly +performing a function, but not too lazy to spend 16 hours automating it. +I was given permission to open source sortImports and here we are :) + +------------------------------------------------------------------------ + +[Get professionally supported isort with the Tidelift +Subscription](https://tidelift.com/subscription/pkg/pypi-isort?utm_source=pypi-isort&utm_medium=referral&utm_campaign=readme) + +Professional support for isort is available as part of the [Tidelift +Subscription](https://tidelift.com/subscription/pkg/pypi-isort?utm_source=pypi-isort&utm_medium=referral&utm_campaign=readme). +Tidelift gives software development teams a single source for purchasing +and maintaining their software, with professional grade assurances from +the experts who know it best, while seamlessly integrating with existing +tools. + +------------------------------------------------------------------------ + +Thanks and I hope you find isort useful! + +~Timothy Crosley + diff --git a/env/lib/python3.8/site-packages/isort-5.5.3.dist-info/RECORD b/env/lib/python3.8/site-packages/isort-5.5.3.dist-info/RECORD new file mode 100644 index 000000000..6fe6665f7 --- /dev/null +++ b/env/lib/python3.8/site-packages/isort-5.5.3.dist-info/RECORD @@ -0,0 +1,96 @@ +../../../bin/isort,sha256=vyuRuMqWGnHobPV0Y1yiR6qAMvD-SLF-wGX-XHLQVz0,289 +isort-5.5.3.dist-info/INSTALLER,sha256=zuuue4knoyJ-UwPPXg8fezS7VCrXJQrAP7zeNuwvFQg,4 +isort-5.5.3.dist-info/LICENSE,sha256=BjKUABw9Uj26y6ud1UrCKZgnVsyvWSylMkCysM3YIGU,1089 +isort-5.5.3.dist-info/METADATA,sha256=hn0IBLgrXcUsrRIWO2qE9dGSkD00UM02PS381O7pf-o,20126 +isort-5.5.3.dist-info/RECORD,, +isort-5.5.3.dist-info/WHEEL,sha256=xSvaL1DM8LOHfdyo0cCcwjZu1tC6CnCsRGWUgazvlbM,83 +isort-5.5.3.dist-info/entry_points.txt,sha256=_Iy7m5GNm89oXcjsXzVEFav4wXWsTqKXiZUARWjFI7M,148 +isort/__init__.py,sha256=u8zdFTPFro_l9J7JzdeNSlu6CU6BboY3fRTuloxbl7c,374 +isort/__main__.py,sha256=iK0trzN9CCXpQX-XPZDZ9JVkm2Lc0q0oiAgsa6FkJb4,36 +isort/__pycache__/__init__.cpython-38.pyc,, +isort/__pycache__/__main__.cpython-38.pyc,, +isort/__pycache__/_version.cpython-38.pyc,, +isort/__pycache__/api.cpython-38.pyc,, +isort/__pycache__/comments.cpython-38.pyc,, +isort/__pycache__/core.cpython-38.pyc,, +isort/__pycache__/exceptions.cpython-38.pyc,, +isort/__pycache__/format.cpython-38.pyc,, +isort/__pycache__/hooks.cpython-38.pyc,, +isort/__pycache__/io.cpython-38.pyc,, +isort/__pycache__/literal.cpython-38.pyc,, +isort/__pycache__/logo.cpython-38.pyc,, +isort/__pycache__/main.cpython-38.pyc,, +isort/__pycache__/output.cpython-38.pyc,, +isort/__pycache__/parse.cpython-38.pyc,, +isort/__pycache__/place.cpython-38.pyc,, +isort/__pycache__/profiles.cpython-38.pyc,, +isort/__pycache__/pylama_isort.cpython-38.pyc,, +isort/__pycache__/sections.cpython-38.pyc,, +isort/__pycache__/settings.cpython-38.pyc,, +isort/__pycache__/setuptools_commands.cpython-38.pyc,, +isort/__pycache__/sorting.cpython-38.pyc,, +isort/__pycache__/utils.cpython-38.pyc,, +isort/__pycache__/wrap.cpython-38.pyc,, +isort/__pycache__/wrap_modes.cpython-38.pyc,, +isort/_future/__init__.py,sha256=wn-Aa4CVe0zZfA_YBTkJqb6LA9HR9NgpAp0uatzNRNs,326 +isort/_future/__pycache__/__init__.cpython-38.pyc,, +isort/_future/__pycache__/_dataclasses.cpython-38.pyc,, +isort/_future/_dataclasses.py,sha256=zrstLAFOKIHRWJjXdZuHJG2EPiyTVFSQuBcsPpXCCnk,44112 +isort/_vendored/toml/LICENSE,sha256=LZKUgj32yJNXyL5JJ_znk2HWVh5e51MtWSbmOTmqpTY,1252 +isort/_vendored/toml/__init__.py,sha256=gKOk-Amczi2juJsOs1D6UEToaPSIIgNh95Yo5N5gneE,703 +isort/_vendored/toml/__pycache__/__init__.cpython-38.pyc,, +isort/_vendored/toml/__pycache__/decoder.cpython-38.pyc,, +isort/_vendored/toml/__pycache__/encoder.cpython-38.pyc,, +isort/_vendored/toml/__pycache__/ordered.cpython-38.pyc,, +isort/_vendored/toml/__pycache__/tz.cpython-38.pyc,, +isort/_vendored/toml/decoder.py,sha256=5etBKNvVLFAR0rhLCJ9fnRTlqkebI4ZQeoJi_myFbd4,37713 +isort/_vendored/toml/encoder.py,sha256=gQOXYnAWo27Jc_przA1FqLX5AgwbdgN-qDHQtKRx300,9668 +isort/_vendored/toml/ordered.py,sha256=aW5woa5xOqR4BjIz9t10_lghxyhF54KQ7FqUNVv7WJ0,334 +isort/_vendored/toml/tz.py,sha256=8TAiXrTqU08sE0ruz2TXH_pFY2rlwNKE47MSE4rDo8Y,618 +isort/_version.py,sha256=2Y257b62i66TmVfznI9gltuV3mV9fDbfFJc2Akce2rI,22 +isort/api.py,sha256=BUoXkRtXsO_VliB2NFyCaf14Fw5TvfwcX4YTo1_eCbk,15903 +isort/comments.py,sha256=23uMZZbUn8y3glMW6_WftnEhECvc-4LW4ysEghpYUUU,962 +isort/core.py,sha256=XOMrW_LCFeAbSFlsQp76sKLWgmYMgCfjRvyrqVLY11c,17583 +isort/deprecated/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0 +isort/deprecated/__pycache__/__init__.cpython-38.pyc,, +isort/deprecated/__pycache__/finders.cpython-38.pyc,, +isort/deprecated/finders.py,sha256=N-ujofD6auS5ZPjtaeIB5S2lZk-7Dmx17w57DeU0Q_U,14488 +isort/exceptions.py,sha256=Jxk4rfvI4TcaBcmVT2VD5LcEzwCbVQh6BXPfjP_Gmvc,4635 +isort/format.py,sha256=z_Mf1T2IXh9TyNYY9dw-nWSAfsqWxLvzLwTov7nojuM,5111 +isort/hooks.py,sha256=VatU5IHqBFfTAtl4gaIqr7WocV7r-_6ndtDcrnwq_Y8,3146 +isort/io.py,sha256=30v6ZH7ntl6hAZGAArB5G1uol1FiQ8qb97s1G71Hwt4,1757 +isort/literal.py,sha256=PQRMWSkbbP3pEhj88pFhSjX6Q3IH-_Pn_XdLf4D7a2M,3548 +isort/logo.py,sha256=cL3al79O7O0G2viqRMRfBPp0qtRZmJw2nHSCZw8XWdQ,388 +isort/main.py,sha256=JenbH7sPC29tSGJ8Yx_npknwrHRSM87Pby4xBsllYB4,32103 +isort/output.py,sha256=H-q0l1MYzwnm3ZMYYSMIImNi5iClQpcsspWn7mczINU,22728 +isort/parse.py,sha256=QxU-j-0sc6qLm5fntHOP2ipAPmmiga8B9Yd8Drzy0F0,20774 +isort/place.py,sha256=S3eRp3EVsIq7LDgb4QN1jb7-dvtfXXr48EqJsMP54-Y,3289 +isort/profiles.py,sha256=CyCEpF1iOgrfxvC2nnRAjuKxxuojVN5NViyE-OlFciU,1502 +isort/pylama_isort.py,sha256=Qk8XqicFOn7EhVVQl-gmlybh4WVWbKaDYM8koDB8Dg8,897 +isort/sections.py,sha256=xG5bwU4tOIKUmeBBhZ45EIfjP8HgDOx796bPvD5zWCw,297 +isort/settings.py,sha256=RPeQ7PyyWTzlOH_GxgbyzrbF-i7eaZAFeWUqvrcQZlE,26707 +isort/setuptools_commands.py,sha256=2EIVYwUYAurcihzYSIDXV6zKHM-DxqxHBW-x7UnI3No,2223 +isort/sorting.py,sha256=YrojJFqcBzoeve3VuYk_cij6fc6VL0VNHIhF7cYM9_0,3021 +isort/stdlibs/__init__.py,sha256=MgiO4yPeJZ6ieWz5qSw2LuY7pVmRjZUaCqyUaLH5qJQ,64 +isort/stdlibs/__pycache__/__init__.cpython-38.pyc,, +isort/stdlibs/__pycache__/all.cpython-38.pyc,, +isort/stdlibs/__pycache__/py2.cpython-38.pyc,, +isort/stdlibs/__pycache__/py27.cpython-38.pyc,, +isort/stdlibs/__pycache__/py3.cpython-38.pyc,, +isort/stdlibs/__pycache__/py35.cpython-38.pyc,, +isort/stdlibs/__pycache__/py36.cpython-38.pyc,, +isort/stdlibs/__pycache__/py37.cpython-38.pyc,, +isort/stdlibs/__pycache__/py38.cpython-38.pyc,, +isort/stdlibs/__pycache__/py39.cpython-38.pyc,, +isort/stdlibs/all.py,sha256=n8Es1WK6UlupYyVvf1PDjGbionqix-afC3LkY8nzTcw,57 +isort/stdlibs/py2.py,sha256=dTgWTa7ggz1cwN8fuI9eIs9-5nTmkRxG_uO61CGwfXI,41 +isort/stdlibs/py27.py,sha256=-Id4l2pjAOMXUfwDNnIBR2o8I_mW_Ghmuek2b82Bczk,4492 +isort/stdlibs/py3.py,sha256=4NpsSHXy9mU4pc3nazM6GTB9RD7iqN2JV9n6SUA672w,101 +isort/stdlibs/py35.py,sha256=SVZp9jaCVq4kSjbKcVgF8dJttyFCqcl20ydodsmHrqE,3283 +isort/stdlibs/py36.py,sha256=tCGWDZXWlJJI4_845yOhTpIvnU0-a3TouD_xsMEIZ3s,3298 +isort/stdlibs/py37.py,sha256=nYZmN-s3qMmAHHddegQv6U0j4cnAH0e5SmqTiG6mmhQ,3322 +isort/stdlibs/py38.py,sha256=KE_65iAHg7icOv2xSGScdJWjwBZGuSQYfYcTSIoo_d8,3307 +isort/stdlibs/py39.py,sha256=gHmC2xbsvrqqxybV9G7vrKRv7UmZpgt9NybAhR1LANk,3295 +isort/utils.py,sha256=D_NmQoPoQSTmLzy5HLcZF1hMK9DIj7vzlGDkMWR0c5E,980 +isort/wrap.py,sha256=GiQNWBSOtDEVO2o5OSUpSvEc_7D_SOVdpdLuBVhJDNc,5828 +isort/wrap_modes.py,sha256=v2frlLau8V-EzylNc8AcM99UIn3j_rSn7zP58QAGgbs,11118 diff --git a/env/lib/python3.8/site-packages/isort-5.5.3.dist-info/WHEEL b/env/lib/python3.8/site-packages/isort-5.5.3.dist-info/WHEEL new file mode 100644 index 000000000..bbb34895a --- /dev/null +++ b/env/lib/python3.8/site-packages/isort-5.5.3.dist-info/WHEEL @@ -0,0 +1,4 @@ +Wheel-Version: 1.0 +Generator: poetry 1.0.5 +Root-Is-Purelib: true +Tag: py3-none-any diff --git a/env/lib/python3.8/site-packages/isort-5.5.3.dist-info/entry_points.txt b/env/lib/python3.8/site-packages/isort-5.5.3.dist-info/entry_points.txt new file mode 100644 index 000000000..ff609bb56 --- /dev/null +++ b/env/lib/python3.8/site-packages/isort-5.5.3.dist-info/entry_points.txt @@ -0,0 +1,9 @@ +[console_scripts] +isort=isort.main:main + +[distutils.commands] +isort=isort.main:ISortCommand + +[pylama.linter] +isort=isort=isort.pylama_isort:Linter + diff --git a/env/lib/python3.8/site-packages/isort/__init__.py b/env/lib/python3.8/site-packages/isort/__init__.py new file mode 100644 index 000000000..236255dd8 --- /dev/null +++ b/env/lib/python3.8/site-packages/isort/__init__.py @@ -0,0 +1,9 @@ +"""Defines the public isort interface""" +from . import settings +from ._version import __version__ +from .api import check_code_string as check_code +from .api import check_file, check_stream, place_module, place_module_with_reason +from .api import sort_code_string as code +from .api import sort_file as file +from .api import sort_stream as stream +from .settings import Config diff --git a/env/lib/python3.8/site-packages/isort/__main__.py b/env/lib/python3.8/site-packages/isort/__main__.py new file mode 100644 index 000000000..94b1d057b --- /dev/null +++ b/env/lib/python3.8/site-packages/isort/__main__.py @@ -0,0 +1,3 @@ +from isort.main import main + +main() diff --git a/env/lib/python3.8/site-packages/isort/_future/__init__.py b/env/lib/python3.8/site-packages/isort/_future/__init__.py new file mode 100644 index 000000000..4d9ef4b76 --- /dev/null +++ b/env/lib/python3.8/site-packages/isort/_future/__init__.py @@ -0,0 +1,12 @@ +import sys + +if sys.version_info.major <= 3 and sys.version_info.minor <= 6: + from . import _dataclasses as dataclasses # type: ignore + +else: + import dataclasses # type: ignore + +dataclass = dataclasses.dataclass # type: ignore +field = dataclasses.field # type: ignore + +__all__ = ["dataclasses", "dataclass", "field"] diff --git a/env/lib/python3.8/site-packages/isort/_future/_dataclasses.py b/env/lib/python3.8/site-packages/isort/_future/_dataclasses.py new file mode 100644 index 000000000..87acb666e --- /dev/null +++ b/env/lib/python3.8/site-packages/isort/_future/_dataclasses.py @@ -0,0 +1,1209 @@ +# type: ignore +# flake8: noqa +# flake8: noqa +"""Backport of Python3.7 dataclasses Library + +Taken directly from here: https://github.com/ericvsmith/dataclasses +Licensed under the Apache License: https://github.com/ericvsmith/dataclasses/blob/master/LICENSE.txt + +Needed due to isorts strict no non-optional requirements stance. + +TODO: Remove once isort only supports 3.7+ +""" +import copy +import inspect +import keyword +import re +import sys +import types + +__all__ = [ + "dataclass", + "field", + "Field", + "FrozenInstanceError", + "InitVar", + "MISSING", + # Helper functions. + "fields", + "asdict", + "astuple", + "make_dataclass", + "replace", + "is_dataclass", +] + +# Conditions for adding methods. The boxes indicate what action the +# dataclass decorator takes. For all of these tables, when I talk +# about init=, repr=, eq=, order=, unsafe_hash=, or frozen=, I'm +# referring to the arguments to the @dataclass decorator. When +# checking if a dunder method already exists, I mean check for an +# entry in the class's __dict__. I never check to see if an attribute +# is defined in a base class. + +# Key: +# +=========+=========================================+ +# + Value | Meaning | +# +=========+=========================================+ +# | | No action: no method is added. | +# +---------+-----------------------------------------+ +# | add | Generated method is added. | +# +---------+-----------------------------------------+ +# | raise | TypeError is raised. | +# +---------+-----------------------------------------+ +# | None | Attribute is set to None. | +# +=========+=========================================+ + +# __init__ +# +# +--- init= parameter +# | +# v | | | +# | no | yes | <--- class has __init__ in __dict__? +# +=======+=======+=======+ +# | False | | | +# +-------+-------+-------+ +# | True | add | | <- the default +# +=======+=======+=======+ + +# __repr__ +# +# +--- repr= parameter +# | +# v | | | +# | no | yes | <--- class has __repr__ in __dict__? +# +=======+=======+=======+ +# | False | | | +# +-------+-------+-------+ +# | True | add | | <- the default +# +=======+=======+=======+ + + +# __setattr__ +# __delattr__ +# +# +--- frozen= parameter +# | +# v | | | +# | no | yes | <--- class has __setattr__ or __delattr__ in __dict__? +# +=======+=======+=======+ +# | False | | | <- the default +# +-------+-------+-------+ +# | True | add | raise | +# +=======+=======+=======+ +# Raise because not adding these methods would break the "frozen-ness" +# of the class. + +# __eq__ +# +# +--- eq= parameter +# | +# v | | | +# | no | yes | <--- class has __eq__ in __dict__? +# +=======+=======+=======+ +# | False | | | +# +-------+-------+-------+ +# | True | add | | <- the default +# +=======+=======+=======+ + +# __lt__ +# __le__ +# __gt__ +# __ge__ +# +# +--- order= parameter +# | +# v | | | +# | no | yes | <--- class has any comparison method in __dict__? +# +=======+=======+=======+ +# | False | | | <- the default +# +-------+-------+-------+ +# | True | add | raise | +# +=======+=======+=======+ +# Raise because to allow this case would interfere with using +# functools.total_ordering. + +# __hash__ + +# +------------------- unsafe_hash= parameter +# | +----------- eq= parameter +# | | +--- frozen= parameter +# | | | +# v v v | | | +# | no | yes | <--- class has explicitly defined __hash__ +# +=======+=======+=======+========+========+ +# | False | False | False | | | No __eq__, use the base class __hash__ +# +-------+-------+-------+--------+--------+ +# | False | False | True | | | No __eq__, use the base class __hash__ +# +-------+-------+-------+--------+--------+ +# | False | True | False | None | | <-- the default, not hashable +# +-------+-------+-------+--------+--------+ +# | False | True | True | add | | Frozen, so hashable, allows override +# +-------+-------+-------+--------+--------+ +# | True | False | False | add | raise | Has no __eq__, but hashable +# +-------+-------+-------+--------+--------+ +# | True | False | True | add | raise | Has no __eq__, but hashable +# +-------+-------+-------+--------+--------+ +# | True | True | False | add | raise | Not frozen, but hashable +# +-------+-------+-------+--------+--------+ +# | True | True | True | add | raise | Frozen, so hashable +# +=======+=======+=======+========+========+ +# For boxes that are blank, __hash__ is untouched and therefore +# inherited from the base class. If the base is object, then +# id-based hashing is used. +# +# Note that a class may already have __hash__=None if it specified an +# __eq__ method in the class body (not one that was created by +# @dataclass). +# +# See _hash_action (below) for a coded version of this table. + + +# Raised when an attempt is made to modify a frozen class. +class FrozenInstanceError(AttributeError): + pass + + +# A sentinel object for default values to signal that a default +# factory will be used. This is given a nice repr() which will appear +# in the function signature of dataclasses' constructors. +class _HAS_DEFAULT_FACTORY_CLASS: + def __repr__(self): + return "" + + +_HAS_DEFAULT_FACTORY = _HAS_DEFAULT_FACTORY_CLASS() + +# A sentinel object to detect if a parameter is supplied or not. Use +# a class to give it a better repr. +class _MISSING_TYPE: + pass + + +MISSING = _MISSING_TYPE() + +# Since most per-field metadata will be unused, create an empty +# read-only proxy that can be shared among all fields. +_EMPTY_METADATA = types.MappingProxyType({}) + +# Markers for the various kinds of fields and pseudo-fields. +class _FIELD_BASE: + def __init__(self, name): + self.name = name + + def __repr__(self): + return self.name + + +_FIELD = _FIELD_BASE("_FIELD") +_FIELD_CLASSVAR = _FIELD_BASE("_FIELD_CLASSVAR") +_FIELD_INITVAR = _FIELD_BASE("_FIELD_INITVAR") + +# The name of an attribute on the class where we store the Field +# objects. Also used to check if a class is a Data Class. +_FIELDS = "__dataclass_fields__" + +# The name of an attribute on the class that stores the parameters to +# @dataclass. +_PARAMS = "__dataclass_params__" + +# The name of the function, that if it exists, is called at the end of +# __init__. +_POST_INIT_NAME = "__post_init__" + +# String regex that string annotations for ClassVar or InitVar must match. +# Allows "identifier.identifier[" or "identifier[". +# https://bugs.python.org/issue33453 for details. +_MODULE_IDENTIFIER_RE = re.compile(r"^(?:\s*(\w+)\s*\.)?\s*(\w+)") + + +class _InitVarMeta(type): + def __getitem__(self, params): + return self + + +class InitVar(metaclass=_InitVarMeta): + pass + + +# Instances of Field are only ever created from within this module, +# and only from the field() function, although Field instances are +# exposed externally as (conceptually) read-only objects. +# +# name and type are filled in after the fact, not in __init__. +# They're not known at the time this class is instantiated, but it's +# convenient if they're available later. +# +# When cls._FIELDS is filled in with a list of Field objects, the name +# and type fields will have been populated. +class Field: + __slots__ = ( + "name", + "type", + "default", + "default_factory", + "repr", + "hash", + "init", + "compare", + "metadata", + "_field_type", # Private: not to be used by user code. + ) + + def __init__(self, default, default_factory, init, repr, hash, compare, metadata): + self.name = None + self.type = None + self.default = default + self.default_factory = default_factory + self.init = init + self.repr = repr + self.hash = hash + self.compare = compare + self.metadata = ( + _EMPTY_METADATA + if metadata is None or len(metadata) == 0 + else types.MappingProxyType(metadata) + ) + self._field_type = None + + def __repr__(self): + return ( + "Field(" + f"name={self.name!r}," + f"type={self.type!r}," + f"default={self.default!r}," + f"default_factory={self.default_factory!r}," + f"init={self.init!r}," + f"repr={self.repr!r}," + f"hash={self.hash!r}," + f"compare={self.compare!r}," + f"metadata={self.metadata!r}," + f"_field_type={self._field_type}" + ")" + ) + + # This is used to support the PEP 487 __set_name__ protocol in the + # case where we're using a field that contains a descriptor as a + # defaul value. For details on __set_name__, see + # https://www.python.org/dev/peps/pep-0487/#implementation-details. + # + # Note that in _process_class, this Field object is overwritten + # with the default value, so the end result is a descriptor that + # had __set_name__ called on it at the right time. + def __set_name__(self, owner, name): + func = getattr(type(self.default), "__set_name__", None) + if func: + # There is a __set_name__ method on the descriptor, call + # it. + func(self.default, owner, name) + + +class _DataclassParams: + __slots__ = ("init", "repr", "eq", "order", "unsafe_hash", "frozen") + + def __init__(self, init, repr, eq, order, unsafe_hash, frozen): + self.init = init + self.repr = repr + self.eq = eq + self.order = order + self.unsafe_hash = unsafe_hash + self.frozen = frozen + + def __repr__(self): + return ( + "_DataclassParams(" + f"init={self.init!r}," + f"repr={self.repr!r}," + f"eq={self.eq!r}," + f"order={self.order!r}," + f"unsafe_hash={self.unsafe_hash!r}," + f"frozen={self.frozen!r}" + ")" + ) + + +# This function is used instead of exposing Field creation directly, +# so that a type checker can be told (via overloads) that this is a +# function whose type depends on its parameters. +def field( + *, + default=MISSING, + default_factory=MISSING, + init=True, + repr=True, + hash=None, + compare=True, + metadata=None, +): + """Return an object to identify dataclass fields. + default is the default value of the field. default_factory is a + 0-argument function called to initialize a field's value. If init + is True, the field will be a parameter to the class's __init__() + function. If repr is True, the field will be included in the + object's repr(). If hash is True, the field will be included in + the object's hash(). If compare is True, the field will be used + in comparison functions. metadata, if specified, must be a + mapping which is stored but not otherwise examined by dataclass. + It is an error to specify both default and default_factory. + """ + + if default is not MISSING and default_factory is not MISSING: + raise ValueError("cannot specify both default and default_factory") + return Field(default, default_factory, init, repr, hash, compare, metadata) + + +def _tuple_str(obj_name, fields): + # Return a string representing each field of obj_name as a tuple + # member. So, if fields is ['x', 'y'] and obj_name is "self", + # return "(self.x,self.y)". + + # Special case for the 0-tuple. + if not fields: + return "()" + # Note the trailing comma, needed if this turns out to be a 1-tuple. + return f'({",".join(f"{obj_name}.{f.name}" for f in fields)},)' + + +def _create_fn(name, args, body, *, globals=None, locals=None, return_type=MISSING): + # Note that we mutate locals when exec() is called. Caller + # beware! The only callers are internal to this module, so no + # worries about external callers. + if locals is None: + locals = {} + return_annotation = "" + if return_type is not MISSING: + locals["_return_type"] = return_type + return_annotation = "->_return_type" + args = ",".join(args) + body = "\n".join(f" {b}" for b in body) + + # Compute the text of the entire function. + txt = f"def {name}({args}){return_annotation}:\n{body}" + + exec(txt, globals, locals) # nosec + return locals[name] + + +def _field_assign(frozen, name, value, self_name): + # If we're a frozen class, then assign to our fields in __init__ + # via object.__setattr__. Otherwise, just use a simple + # assignment. + # + # self_name is what "self" is called in this function: don't + # hard-code "self", since that might be a field name. + if frozen: + return f"object.__setattr__({self_name},{name!r},{value})" + return f"{self_name}.{name}={value}" + + +def _field_init(f, frozen, globals, self_name): + # Return the text of the line in the body of __init__ that will + # initialize this field. + + default_name = f"_dflt_{f.name}" + if f.default_factory is not MISSING: + if f.init: + # This field has a default factory. If a parameter is + # given, use it. If not, call the factory. + globals[default_name] = f.default_factory + value = f"{default_name}() " f"if {f.name} is _HAS_DEFAULT_FACTORY " f"else {f.name}" + else: + # This is a field that's not in the __init__ params, but + # has a default factory function. It needs to be + # initialized here by calling the factory function, + # because there's no other way to initialize it. + + # For a field initialized with a default=defaultvalue, the + # class dict just has the default value + # (cls.fieldname=defaultvalue). But that won't work for a + # default factory, the factory must be called in __init__ + # and we must assign that to self.fieldname. We can't + # fall back to the class dict's value, both because it's + # not set, and because it might be different per-class + # (which, after all, is why we have a factory function!). + + globals[default_name] = f.default_factory + value = f"{default_name}()" + else: + # No default factory. + if f.init: + if f.default is MISSING: + # There's no default, just do an assignment. + value = f.name + elif f.default is not MISSING: + globals[default_name] = f.default + value = f.name + else: + # This field does not need initialization. Signify that + # to the caller by returning None. + return None + + # Only test this now, so that we can create variables for the + # default. However, return None to signify that we're not going + # to actually do the assignment statement for InitVars. + if f._field_type == _FIELD_INITVAR: + return None + + # Now, actually generate the field assignment. + return _field_assign(frozen, f.name, value, self_name) + + +def _init_param(f): + # Return the __init__ parameter string for this field. For + # example, the equivalent of 'x:int=3' (except instead of 'int', + # reference a variable set to int, and instead of '3', reference a + # variable set to 3). + if f.default is MISSING and f.default_factory is MISSING: + # There's no default, and no default_factory, just output the + # variable name and type. + default = "" + elif f.default is not MISSING: + # There's a default, this will be the name that's used to look + # it up. + default = f"=_dflt_{f.name}" + elif f.default_factory is not MISSING: + # There's a factory function. Set a marker. + default = "=_HAS_DEFAULT_FACTORY" + return f"{f.name}:_type_{f.name}{default}" + + +def _init_fn(fields, frozen, has_post_init, self_name): + # fields contains both real fields and InitVar pseudo-fields. + + # Make sure we don't have fields without defaults following fields + # with defaults. This actually would be caught when exec-ing the + # function source code, but catching it here gives a better error + # message, and future-proofs us in case we build up the function + # using ast. + seen_default = False + for f in fields: + # Only consider fields in the __init__ call. + if f.init: + if not (f.default is MISSING and f.default_factory is MISSING): + seen_default = True + elif seen_default: + raise TypeError(f"non-default argument {f.name!r} " "follows default argument") + + globals = {"MISSING": MISSING, "_HAS_DEFAULT_FACTORY": _HAS_DEFAULT_FACTORY} + + body_lines = [] + for f in fields: + line = _field_init(f, frozen, globals, self_name) + # line is None means that this field doesn't require + # initialization (it's a pseudo-field). Just skip it. + if line: + body_lines.append(line) + + # Does this class have a post-init function? + if has_post_init: + params_str = ",".join(f.name for f in fields if f._field_type is _FIELD_INITVAR) + body_lines.append(f"{self_name}.{_POST_INIT_NAME}({params_str})") + + # If no body lines, use 'pass'. + if not body_lines: + body_lines = ["pass"] + + locals = {f"_type_{f.name}": f.type for f in fields} + return _create_fn( + "__init__", + [self_name] + [_init_param(f) for f in fields if f.init], + body_lines, + locals=locals, + globals=globals, + return_type=None, + ) + + +def _repr_fn(fields): + return _create_fn( + "__repr__", + ("self",), + [ + 'return self.__class__.__qualname__ + f"(' + + ", ".join(f"{f.name}={{self.{f.name}!r}}" for f in fields) + + ')"' + ], + ) + + +def _frozen_get_del_attr(cls, fields): + # XXX: globals is modified on the first call to _create_fn, then + # the modified version is used in the second call. Is this okay? + globals = {"cls": cls, "FrozenInstanceError": FrozenInstanceError} + if fields: + fields_str = "(" + ",".join(repr(f.name) for f in fields) + ",)" + else: + # Special case for the zero-length tuple. + fields_str = "()" + return ( + _create_fn( + "__setattr__", + ("self", "name", "value"), + ( + f"if type(self) is cls or name in {fields_str}:", + ' raise FrozenInstanceError(f"cannot assign to field {name!r}")', + f"super(cls, self).__setattr__(name, value)", + ), + globals=globals, + ), + _create_fn( + "__delattr__", + ("self", "name"), + ( + f"if type(self) is cls or name in {fields_str}:", + ' raise FrozenInstanceError(f"cannot delete field {name!r}")', + f"super(cls, self).__delattr__(name)", + ), + globals=globals, + ), + ) + + +def _cmp_fn(name, op, self_tuple, other_tuple): + # Create a comparison function. If the fields in the object are + # named 'x' and 'y', then self_tuple is the string + # '(self.x,self.y)' and other_tuple is the string + # '(other.x,other.y)'. + + return _create_fn( + name, + ("self", "other"), + [ + "if other.__class__ is self.__class__:", + f" return {self_tuple}{op}{other_tuple}", + "return NotImplemented", + ], + ) + + +def _hash_fn(fields): + self_tuple = _tuple_str("self", fields) + return _create_fn("__hash__", ("self",), [f"return hash({self_tuple})"]) + + +def _is_classvar(a_type, typing): + # This test uses a typing internal class, but it's the best way to + # test if this is a ClassVar. + return type(a_type) is typing._ClassVar + + +def _is_initvar(a_type, dataclasses): + # The module we're checking against is the module we're + # currently in (dataclasses.py). + return a_type is dataclasses.InitVar + + +def _is_type(annotation, cls, a_module, a_type, is_type_predicate): + # Given a type annotation string, does it refer to a_type in + # a_module? For example, when checking that annotation denotes a + # ClassVar, then a_module is typing, and a_type is + # typing.ClassVar. + + # It's possible to look up a_module given a_type, but it involves + # looking in sys.modules (again!), and seems like a waste since + # the caller already knows a_module. + + # - annotation is a string type annotation + # - cls is the class that this annotation was found in + # - a_module is the module we want to match + # - a_type is the type in that module we want to match + # - is_type_predicate is a function called with (obj, a_module) + # that determines if obj is of the desired type. + + # Since this test does not do a local namespace lookup (and + # instead only a module (global) lookup), there are some things it + # gets wrong. + + # With string annotations, cv0 will be detected as a ClassVar: + # CV = ClassVar + # @dataclass + # class C0: + # cv0: CV + + # But in this example cv1 will not be detected as a ClassVar: + # @dataclass + # class C1: + # CV = ClassVar + # cv1: CV + + # In C1, the code in this function (_is_type) will look up "CV" in + # the module and not find it, so it will not consider cv1 as a + # ClassVar. This is a fairly obscure corner case, and the best + # way to fix it would be to eval() the string "CV" with the + # correct global and local namespaces. However that would involve + # a eval() penalty for every single field of every dataclass + # that's defined. It was judged not worth it. + + match = _MODULE_IDENTIFIER_RE.match(annotation) + if match: + ns = None + module_name = match.group(1) + if not module_name: + # No module name, assume the class's module did + # "from dataclasses import InitVar". + ns = sys.modules.get(cls.__module__).__dict__ + else: + # Look up module_name in the class's module. + module = sys.modules.get(cls.__module__) + if module and module.__dict__.get(module_name) is a_module: + ns = sys.modules.get(a_type.__module__).__dict__ + if ns and is_type_predicate(ns.get(match.group(2)), a_module): + return True + return False + + +def _get_field(cls, a_name, a_type): + # Return a Field object for this field name and type. ClassVars + # and InitVars are also returned, but marked as such (see + # f._field_type). + + # If the default value isn't derived from Field, then it's only a + # normal default value. Convert it to a Field(). + default = getattr(cls, a_name, MISSING) + if isinstance(default, Field): + f = default + else: + if isinstance(default, types.MemberDescriptorType): + # This is a field in __slots__, so it has no default value. + default = MISSING + f = field(default=default) + + # Only at this point do we know the name and the type. Set them. + f.name = a_name + f.type = a_type + + # Assume it's a normal field until proven otherwise. We're next + # going to decide if it's a ClassVar or InitVar, everything else + # is just a normal field. + f._field_type = _FIELD + + # In addition to checking for actual types here, also check for + # string annotations. get_type_hints() won't always work for us + # (see https://github.com/python/typing/issues/508 for example), + # plus it's expensive and would require an eval for every stirng + # annotation. So, make a best effort to see if this is a ClassVar + # or InitVar using regex's and checking that the thing referenced + # is actually of the correct type. + + # For the complete discussion, see https://bugs.python.org/issue33453 + + # If typing has not been imported, then it's impossible for any + # annotation to be a ClassVar. So, only look for ClassVar if + # typing has been imported by any module (not necessarily cls's + # module). + typing = sys.modules.get("typing") + if typing: + if _is_classvar(a_type, typing) or ( + isinstance(f.type, str) and _is_type(f.type, cls, typing, typing.ClassVar, _is_classvar) + ): + f._field_type = _FIELD_CLASSVAR + + # If the type is InitVar, or if it's a matching string annotation, + # then it's an InitVar. + if f._field_type is _FIELD: + # The module we're checking against is the module we're + # currently in (dataclasses.py). + dataclasses = sys.modules[__name__] + if _is_initvar(a_type, dataclasses) or ( + isinstance(f.type, str) + and _is_type(f.type, cls, dataclasses, dataclasses.InitVar, _is_initvar) + ): + f._field_type = _FIELD_INITVAR + + # Validations for individual fields. This is delayed until now, + # instead of in the Field() constructor, since only here do we + # know the field name, which allows for better error reporting. + + # Special restrictions for ClassVar and InitVar. + if f._field_type in (_FIELD_CLASSVAR, _FIELD_INITVAR): + if f.default_factory is not MISSING: + raise TypeError(f"field {f.name} cannot have a " "default factory") + # Should I check for other field settings? default_factory + # seems the most serious to check for. Maybe add others. For + # example, how about init=False (or really, + # init=)? It makes no sense for + # ClassVar and InitVar to specify init=. + + # For real fields, disallow mutable defaults for known types. + if f._field_type is _FIELD and isinstance(f.default, (list, dict, set)): + raise ValueError( + f"mutable default {type(f.default)} for field " + f"{f.name} is not allowed: use default_factory" + ) + + return f + + +def _set_new_attribute(cls, name, value): + # Never overwrites an existing attribute. Returns True if the + # attribute already exists. + if name in cls.__dict__: + return True + setattr(cls, name, value) + return False + + +# Decide if/how we're going to create a hash function. Key is +# (unsafe_hash, eq, frozen, does-hash-exist). Value is the action to +# take. The common case is to do nothing, so instead of providing a +# function that is a no-op, use None to signify that. + + +def _hash_set_none(cls, fields): + return None + + +def _hash_add(cls, fields): + flds = [f for f in fields if (f.compare if f.hash is None else f.hash)] + return _hash_fn(flds) + + +def _hash_exception(cls, fields): + # Raise an exception. + raise TypeError(f"Cannot overwrite attribute __hash__ " f"in class {cls.__name__}") + + +# +# +-------------------------------------- unsafe_hash? +# | +------------------------------- eq? +# | | +------------------------ frozen? +# | | | +---------------- has-explicit-hash? +# | | | | +# | | | | +------- action +# | | | | | +# v v v v v +_hash_action = { + (False, False, False, False): None, + (False, False, False, True): None, + (False, False, True, False): None, + (False, False, True, True): None, + (False, True, False, False): _hash_set_none, + (False, True, False, True): None, + (False, True, True, False): _hash_add, + (False, True, True, True): None, + (True, False, False, False): _hash_add, + (True, False, False, True): _hash_exception, + (True, False, True, False): _hash_add, + (True, False, True, True): _hash_exception, + (True, True, False, False): _hash_add, + (True, True, False, True): _hash_exception, + (True, True, True, False): _hash_add, + (True, True, True, True): _hash_exception, +} +# See https://bugs.python.org/issue32929#msg312829 for an if-statement +# version of this table. + + +def _process_class(cls, init, repr, eq, order, unsafe_hash, frozen): + # Now that dicts retain insertion order, there's no reason to use + # an ordered dict. I am leveraging that ordering here, because + # derived class fields overwrite base class fields, but the order + # is defined by the base class, which is found first. + fields = {} + + setattr(cls, _PARAMS, _DataclassParams(init, repr, eq, order, unsafe_hash, frozen)) + + # Find our base classes in reverse MRO order, and exclude + # ourselves. In reversed order so that more derived classes + # override earlier field definitions in base classes. As long as + # we're iterating over them, see if any are frozen. + any_frozen_base = False + has_dataclass_bases = False + for b in cls.__mro__[-1:0:-1]: + # Only process classes that have been processed by our + # decorator. That is, they have a _FIELDS attribute. + base_fields = getattr(b, _FIELDS, None) + if base_fields: + has_dataclass_bases = True + for f in base_fields.values(): + fields[f.name] = f + if getattr(b, _PARAMS).frozen: + any_frozen_base = True + + # Annotations that are defined in this class (not in base + # classes). If __annotations__ isn't present, then this class + # adds no new annotations. We use this to compute fields that are + # added by this class. + # + # Fields are found from cls_annotations, which is guaranteed to be + # ordered. Default values are from class attributes, if a field + # has a default. If the default value is a Field(), then it + # contains additional info beyond (and possibly including) the + # actual default value. Pseudo-fields ClassVars and InitVars are + # included, despite the fact that they're not real fields. That's + # dealt with later. + cls_annotations = cls.__dict__.get("__annotations__", {}) + + # Now find fields in our class. While doing so, validate some + # things, and set the default values (as class attributes) where + # we can. + cls_fields = [_get_field(cls, name, type) for name, type in cls_annotations.items()] + for f in cls_fields: + fields[f.name] = f + + # If the class attribute (which is the default value for this + # field) exists and is of type 'Field', replace it with the + # real default. This is so that normal class introspection + # sees a real default value, not a Field. + if isinstance(getattr(cls, f.name, None), Field): + if f.default is MISSING: + # If there's no default, delete the class attribute. + # This happens if we specify field(repr=False), for + # example (that is, we specified a field object, but + # no default value). Also if we're using a default + # factory. The class attribute should not be set at + # all in the post-processed class. + delattr(cls, f.name) + else: + setattr(cls, f.name, f.default) + + # Do we have any Field members that don't also have annotations? + for name, value in cls.__dict__.items(): + if isinstance(value, Field) and not name in cls_annotations: + raise TypeError(f"{name!r} is a field but has no type annotation") + + # Check rules that apply if we are derived from any dataclasses. + if has_dataclass_bases: + # Raise an exception if any of our bases are frozen, but we're not. + if any_frozen_base and not frozen: + raise TypeError("cannot inherit non-frozen dataclass from a " "frozen one") + + # Raise an exception if we're frozen, but none of our bases are. + if not any_frozen_base and frozen: + raise TypeError("cannot inherit frozen dataclass from a " "non-frozen one") + + # Remember all of the fields on our class (including bases). This + # also marks this class as being a dataclass. + setattr(cls, _FIELDS, fields) + + # Was this class defined with an explicit __hash__? Note that if + # __eq__ is defined in this class, then python will automatically + # set __hash__ to None. This is a heuristic, as it's possible + # that such a __hash__ == None was not auto-generated, but it + # close enough. + class_hash = cls.__dict__.get("__hash__", MISSING) + has_explicit_hash = not ( + class_hash is MISSING or (class_hash is None and "__eq__" in cls.__dict__) + ) + + # If we're generating ordering methods, we must be generating the + # eq methods. + if order and not eq: + raise ValueError("eq must be true if order is true") + + if init: + # Does this class have a post-init function? + has_post_init = hasattr(cls, _POST_INIT_NAME) + + # Include InitVars and regular fields (so, not ClassVars). + flds = [f for f in fields.values() if f._field_type in (_FIELD, _FIELD_INITVAR)] + _set_new_attribute( + cls, + "__init__", + _init_fn( + flds, + frozen, + has_post_init, + # The name to use for the "self" + # param in __init__. Use "self" + # if possible. + "__dataclass_self__" if "self" in fields else "self", + ), + ) + + # Get the fields as a list, and include only real fields. This is + # used in all of the following methods. + field_list = [f for f in fields.values() if f._field_type is _FIELD] + + if repr: + flds = [f for f in field_list if f.repr] + _set_new_attribute(cls, "__repr__", _repr_fn(flds)) + + if eq: + # Create _eq__ method. There's no need for a __ne__ method, + # since python will call __eq__ and negate it. + flds = [f for f in field_list if f.compare] + self_tuple = _tuple_str("self", flds) + other_tuple = _tuple_str("other", flds) + _set_new_attribute(cls, "__eq__", _cmp_fn("__eq__", "==", self_tuple, other_tuple)) + + if order: + # Create and set the ordering methods. + flds = [f for f in field_list if f.compare] + self_tuple = _tuple_str("self", flds) + other_tuple = _tuple_str("other", flds) + for name, op in [("__lt__", "<"), ("__le__", "<="), ("__gt__", ">"), ("__ge__", ">=")]: + if _set_new_attribute(cls, name, _cmp_fn(name, op, self_tuple, other_tuple)): + raise TypeError( + f"Cannot overwrite attribute {name} " + f"in class {cls.__name__}. Consider using " + "functools.total_ordering" + ) + + if frozen: + for fn in _frozen_get_del_attr(cls, field_list): + if _set_new_attribute(cls, fn.__name__, fn): + raise TypeError( + f"Cannot overwrite attribute {fn.__name__} " f"in class {cls.__name__}" + ) + + # Decide if/how we're going to create a hash function. + hash_action = _hash_action[bool(unsafe_hash), bool(eq), bool(frozen), has_explicit_hash] + if hash_action: + # No need to call _set_new_attribute here, since by the time + # we're here the overwriting is unconditional. + cls.__hash__ = hash_action(cls, field_list) + + if not getattr(cls, "__doc__"): + # Create a class doc-string. + cls.__doc__ = cls.__name__ + str(inspect.signature(cls)).replace(" -> None", "") + + return cls + + +# _cls should never be specified by keyword, so start it with an +# underscore. The presence of _cls is used to detect if this +# decorator is being called with parameters or not. +def dataclass( + _cls=None, *, init=True, repr=True, eq=True, order=False, unsafe_hash=False, frozen=False +): + """Returns the same class as was passed in, with dunder methods + added based on the fields defined in the class. + Examines PEP 526 __annotations__ to determine fields. + If init is true, an __init__() method is added to the class. If + repr is true, a __repr__() method is added. If order is true, rich + comparison dunder methods are added. If unsafe_hash is true, a + __hash__() method function is added. If frozen is true, fields may + not be assigned to after instance creation. + """ + + def wrap(cls): + return _process_class(cls, init, repr, eq, order, unsafe_hash, frozen) + + # See if we're being called as @dataclass or @dataclass(). + if _cls is None: + # We're called with parens. + return wrap + + # We're called as @dataclass without parens. + return wrap(_cls) + + +def fields(class_or_instance): + """Return a tuple describing the fields of this dataclass. + Accepts a dataclass or an instance of one. Tuple elements are of + type Field. + """ + + # Might it be worth caching this, per class? + try: + fields = getattr(class_or_instance, _FIELDS) + except AttributeError: + raise TypeError("must be called with a dataclass type or instance") + + # Exclude pseudo-fields. Note that fields is sorted by insertion + # order, so the order of the tuple is as the fields were defined. + return tuple(f for f in fields.values() if f._field_type is _FIELD) + + +def _is_dataclass_instance(obj): + """Returns True if obj is an instance of a dataclass.""" + return not isinstance(obj, type) and hasattr(obj, _FIELDS) + + +def is_dataclass(obj): + """Returns True if obj is a dataclass or an instance of a + dataclass.""" + return hasattr(obj, _FIELDS) + + +def asdict(obj, *, dict_factory=dict): + """Return the fields of a dataclass instance as a new dictionary mapping + field names to field values. + Example usage: + @dataclass + class C: + x: int + y: int + c = C(1, 2) + assert asdict(c) == {'x': 1, 'y': 2} + If given, 'dict_factory' will be used instead of built-in dict. + The function applies recursively to field values that are + dataclass instances. This will also look into built-in containers: + tuples, lists, and dicts. + """ + if not _is_dataclass_instance(obj): + raise TypeError("asdict() should be called on dataclass instances") + return _asdict_inner(obj, dict_factory) + + +def _asdict_inner(obj, dict_factory): + if _is_dataclass_instance(obj): + result = [] + for f in fields(obj): + value = _asdict_inner(getattr(obj, f.name), dict_factory) + result.append((f.name, value)) + return dict_factory(result) + elif isinstance(obj, (list, tuple)): + return type(obj)(_asdict_inner(v, dict_factory) for v in obj) + elif isinstance(obj, dict): + return type(obj)( + (_asdict_inner(k, dict_factory), _asdict_inner(v, dict_factory)) for k, v in obj.items() + ) + else: + return copy.deepcopy(obj) + + +def astuple(obj, *, tuple_factory=tuple): + """Return the fields of a dataclass instance as a new tuple of field values. + Example usage:: + @dataclass + class C: + x: int + y: int + c = C(1, 2) + assert astuple(c) == (1, 2) + If given, 'tuple_factory' will be used instead of built-in tuple. + The function applies recursively to field values that are + dataclass instances. This will also look into built-in containers: + tuples, lists, and dicts. + """ + + if not _is_dataclass_instance(obj): + raise TypeError("astuple() should be called on dataclass instances") + return _astuple_inner(obj, tuple_factory) + + +def _astuple_inner(obj, tuple_factory): + if _is_dataclass_instance(obj): + result = [] + for f in fields(obj): + value = _astuple_inner(getattr(obj, f.name), tuple_factory) + result.append(value) + return tuple_factory(result) + elif isinstance(obj, (list, tuple)): + return type(obj)(_astuple_inner(v, tuple_factory) for v in obj) + elif isinstance(obj, dict): + return type(obj)( + (_astuple_inner(k, tuple_factory), _astuple_inner(v, tuple_factory)) + for k, v in obj.items() + ) + else: + return copy.deepcopy(obj) + + +def make_dataclass( + cls_name, + fields, + *, + bases=(), + namespace=None, + init=True, + repr=True, + eq=True, + order=False, + unsafe_hash=False, + frozen=False, +): + """Return a new dynamically created dataclass. + The dataclass name will be 'cls_name'. 'fields' is an iterable + of either (name), (name, type) or (name, type, Field) objects. If type is + omitted, use the string 'typing.Any'. Field objects are created by + the equivalent of calling 'field(name, type [, Field-info])'. + C = make_dataclass('C', ['x', ('y', int), ('z', int, field(init=False))], bases=(Base,)) + is equivalent to: + @dataclass + class C(Base): + x: 'typing.Any' + y: int + z: int = field(init=False) + For the bases and namespace parameters, see the builtin type() function. + The parameters init, repr, eq, order, unsafe_hash, and frozen are passed to + dataclass(). + """ + + if namespace is None: + namespace = {} + else: + # Copy namespace since we're going to mutate it. + namespace = namespace.copy() + + # While we're looking through the field names, validate that they + # are identifiers, are not keywords, and not duplicates. + seen = set() + anns = {} + for item in fields: + if isinstance(item, str): + name = item + tp = "typing.Any" + elif len(item) == 2: + ( + name, + tp, + ) = item + elif len(item) == 3: + name, tp, spec = item + namespace[name] = spec + else: + raise TypeError(f"Invalid field: {item!r}") + + if not isinstance(name, str) or not name.isidentifier(): + raise TypeError(f"Field names must be valid identifers: {name!r}") + if keyword.iskeyword(name): + raise TypeError(f"Field names must not be keywords: {name!r}") + if name in seen: + raise TypeError(f"Field name duplicated: {name!r}") + + seen.add(name) + anns[name] = tp + + namespace["__annotations__"] = anns + # We use `types.new_class()` instead of simply `type()` to allow dynamic creation + # of generic dataclassses. + cls = types.new_class(cls_name, bases, {}, lambda ns: ns.update(namespace)) + return dataclass( + cls, init=init, repr=repr, eq=eq, order=order, unsafe_hash=unsafe_hash, frozen=frozen + ) + + +def replace(obj, **changes): + """Return a new object replacing specified fields with new values. + This is especially useful for frozen classes. Example usage: + @dataclass(frozen=True) + class C: + x: int + y: int + c = C(1, 2) + c1 = replace(c, x=3) + assert c1.x == 3 and c1.y == 2 + """ + + # We're going to mutate 'changes', but that's okay because it's a + # new dict, even if called with 'replace(obj, **my_changes)'. + + if not _is_dataclass_instance(obj): + raise TypeError("replace() should be called on dataclass instances") + + # It's an error to have init=False fields in 'changes'. + # If a field is not in 'changes', read its value from the provided obj. + + for f in getattr(obj, _FIELDS).values(): + if not f.init: + # Error if this field is specified in changes. + if f.name in changes: + raise ValueError( + f"field {f.name} is declared with " + "init=False, it cannot be specified with " + "replace()" + ) + continue + + if f.name not in changes: + changes[f.name] = getattr(obj, f.name) + + # Create the new object, which calls __init__() and + # __post_init__() (if defined), using all of the init fields we've + # added and/or left in 'changes'. If there are values supplied in + # changes that aren't fields, this will correctly raise a + # TypeError. + return obj.__class__(**changes) diff --git a/env/lib/python3.8/site-packages/isort/_vendored/toml/LICENSE b/env/lib/python3.8/site-packages/isort/_vendored/toml/LICENSE new file mode 100644 index 000000000..5010e3075 --- /dev/null +++ b/env/lib/python3.8/site-packages/isort/_vendored/toml/LICENSE @@ -0,0 +1,27 @@ +The MIT License + +Copyright 2013-2019 William Pearson +Copyright 2015-2016 Julien Enselme +Copyright 2016 Google Inc. +Copyright 2017 Samuel Vasko +Copyright 2017 Nate Prewitt +Copyright 2017 Jack Evans +Copyright 2019 Filippo Broggini + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in +all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +THE SOFTWARE. \ No newline at end of file diff --git a/env/lib/python3.8/site-packages/isort/_vendored/toml/__init__.py b/env/lib/python3.8/site-packages/isort/_vendored/toml/__init__.py new file mode 100644 index 000000000..8cefeffad --- /dev/null +++ b/env/lib/python3.8/site-packages/isort/_vendored/toml/__init__.py @@ -0,0 +1,23 @@ +"""Python module which parses and emits TOML. + +Released under the MIT license. +""" +from . import decoder, encoder + +__version__ = "0.10.1" +_spec_ = "0.5.0" + +load = decoder.load +loads = decoder.loads +TomlDecoder = decoder.TomlDecoder +TomlDecodeError = decoder.TomlDecodeError +TomlPreserveCommentDecoder = decoder.TomlPreserveCommentDecoder + +dump = encoder.dump +dumps = encoder.dumps +TomlEncoder = encoder.TomlEncoder +TomlArraySeparatorEncoder = encoder.TomlArraySeparatorEncoder +TomlPreserveInlineDictEncoder = encoder.TomlPreserveInlineDictEncoder +TomlNumpyEncoder = encoder.TomlNumpyEncoder +TomlPreserveCommentEncoder = encoder.TomlPreserveCommentEncoder +TomlPathlibEncoder = encoder.TomlPathlibEncoder diff --git a/env/lib/python3.8/site-packages/isort/_vendored/toml/decoder.py b/env/lib/python3.8/site-packages/isort/_vendored/toml/decoder.py new file mode 100644 index 000000000..b90b6933e --- /dev/null +++ b/env/lib/python3.8/site-packages/isort/_vendored/toml/decoder.py @@ -0,0 +1,1053 @@ +import datetime +import io +import re +import sys +from os import linesep + +from .tz import TomlTz + +if sys.version_info < (3,): + _range = xrange # noqa: F821 +else: + unicode = str + _range = range + basestring = str + unichr = chr + + +def _detect_pathlib_path(p): + if (3, 4) <= sys.version_info: + import pathlib + + if isinstance(p, pathlib.PurePath): + return True + return False + + +def _ispath(p): + if isinstance(p, (bytes, basestring)): + return True + return _detect_pathlib_path(p) + + +def _getpath(p): + if (3, 6) <= sys.version_info: + import os + + return os.fspath(p) + if _detect_pathlib_path(p): + return str(p) + return p + + +try: + FNFError = FileNotFoundError +except NameError: + FNFError = IOError + + +TIME_RE = re.compile(r"([0-9]{2}):([0-9]{2}):([0-9]{2})(\.([0-9]{3,6}))?") + + +class TomlDecodeError(ValueError): + """Base toml Exception / Error.""" + + def __init__(self, msg, doc, pos): + lineno = doc.count("\n", 0, pos) + 1 + colno = pos - doc.rfind("\n", 0, pos) + emsg = "{} (line {} column {} char {})".format(msg, lineno, colno, pos) + ValueError.__init__(self, emsg) + self.msg = msg + self.doc = doc + self.pos = pos + self.lineno = lineno + self.colno = colno + + +# Matches a TOML number, which allows underscores for readability +_number_with_underscores = re.compile("([0-9])(_([0-9]))*") + + +class CommentValue(object): + def __init__(self, val, comment, beginline, _dict): + self.val = val + separator = "\n" if beginline else " " + self.comment = separator + comment + self._dict = _dict + + def __getitem__(self, key): + return self.val[key] + + def __setitem__(self, key, value): + self.val[key] = value + + def dump(self, dump_value_func): + retstr = dump_value_func(self.val) + if isinstance(self.val, self._dict): + return self.comment + "\n" + unicode(retstr) + else: + return unicode(retstr) + self.comment + + +def _strictly_valid_num(n): + n = n.strip() + if not n: + return False + if n[0] == "_": + return False + if n[-1] == "_": + return False + if "_." in n or "._" in n: + return False + if len(n) == 1: + return True + if n[0] == "0" and n[1] not in [".", "o", "b", "x"]: + return False + if n[0] == "+" or n[0] == "-": + n = n[1:] + if len(n) > 1 and n[0] == "0" and n[1] != ".": + return False + if "__" in n: + return False + return True + + +def load(f, _dict=dict, decoder=None): + """Parses named file or files as toml and returns a dictionary + + Args: + f: Path to the file to open, array of files to read into single dict + or a file descriptor + _dict: (optional) Specifies the class of the returned toml dictionary + decoder: The decoder to use + + Returns: + Parsed toml file represented as a dictionary + + Raises: + TypeError -- When f is invalid type + TomlDecodeError: Error while decoding toml + IOError / FileNotFoundError -- When an array with no valid (existing) + (Python 2 / Python 3) file paths is passed + """ + + if _ispath(f): + with io.open(_getpath(f), encoding="utf-8") as ffile: + return loads(ffile.read(), _dict, decoder) + elif isinstance(f, list): + from os import path as op + from warnings import warn + + if not [path for path in f if op.exists(path)]: + error_msg = "Load expects a list to contain filenames only." + error_msg += linesep + error_msg += "The list needs to contain the path of at least one " "existing file." + raise FNFError(error_msg) + if decoder is None: + decoder = TomlDecoder(_dict) + d = decoder.get_empty_table() + for l in f: # noqa: E741 + if op.exists(l): + d.update(load(l, _dict, decoder)) + else: + warn("Non-existent filename in list with at least one valid " "filename") + return d + else: + try: + return loads(f.read(), _dict, decoder) + except AttributeError: + raise TypeError("You can only load a file descriptor, filename or " "list") + + +_groupname_re = re.compile(r"^[A-Za-z0-9_-]+$") + + +def loads(s, _dict=dict, decoder=None): + """Parses string as toml + + Args: + s: String to be parsed + _dict: (optional) Specifies the class of the returned toml dictionary + + Returns: + Parsed toml file represented as a dictionary + + Raises: + TypeError: When a non-string is passed + TomlDecodeError: Error while decoding toml + """ + + implicitgroups = [] + if decoder is None: + decoder = TomlDecoder(_dict) + retval = decoder.get_empty_table() + currentlevel = retval + if not isinstance(s, basestring): + raise TypeError("Expecting something like a string") + + if not isinstance(s, unicode): + s = s.decode("utf8") + + original = s + sl = list(s) + openarr = 0 + openstring = False + openstrchar = "" + multilinestr = False + arrayoftables = False + beginline = True + keygroup = False + dottedkey = False + keyname = 0 + key = "" + prev_key = "" + line_no = 1 + + for i, item in enumerate(sl): + if item == "\r" and sl[i + 1] == "\n": + sl[i] = " " + continue + if keyname: + key += item + if item == "\n": + raise TomlDecodeError( + "Key name found without value." " Reached end of line.", original, i + ) + if openstring: + if item == openstrchar: + oddbackslash = False + k = 1 + while i >= k and sl[i - k] == "\\": + oddbackslash = not oddbackslash + k += 1 + if not oddbackslash: + keyname = 2 + openstring = False + openstrchar = "" + continue + elif keyname == 1: + if item.isspace(): + keyname = 2 + continue + elif item == ".": + dottedkey = True + continue + elif item.isalnum() or item == "_" or item == "-": + continue + elif dottedkey and sl[i - 1] == "." and (item == '"' or item == "'"): + openstring = True + openstrchar = item + continue + elif keyname == 2: + if item.isspace(): + if dottedkey: + nextitem = sl[i + 1] + if not nextitem.isspace() and nextitem != ".": + keyname = 1 + continue + if item == ".": + dottedkey = True + nextitem = sl[i + 1] + if not nextitem.isspace() and nextitem != ".": + keyname = 1 + continue + if item == "=": + keyname = 0 + prev_key = key[:-1].rstrip() + key = "" + dottedkey = False + else: + raise TomlDecodeError( + "Found invalid character in key name: '" + + item + + "'. Try quoting the key name.", + original, + i, + ) + if item == "'" and openstrchar != '"': + k = 1 + try: + while sl[i - k] == "'": + k += 1 + if k == 3: + break + except IndexError: + pass + if k == 3: + multilinestr = not multilinestr + openstring = multilinestr + else: + openstring = not openstring + if openstring: + openstrchar = "'" + else: + openstrchar = "" + if item == '"' and openstrchar != "'": + oddbackslash = False + k = 1 + tripquote = False + try: + while sl[i - k] == '"': + k += 1 + if k == 3: + tripquote = True + break + if k == 1 or (k == 3 and tripquote): + while sl[i - k] == "\\": + oddbackslash = not oddbackslash + k += 1 + except IndexError: + pass + if not oddbackslash: + if tripquote: + multilinestr = not multilinestr + openstring = multilinestr + else: + openstring = not openstring + if openstring: + openstrchar = '"' + else: + openstrchar = "" + if item == "#" and (not openstring and not keygroup and not arrayoftables): + j = i + comment = "" + try: + while sl[j] != "\n": + comment += s[j] + sl[j] = " " + j += 1 + except IndexError: + break + if not openarr: + decoder.preserve_comment(line_no, prev_key, comment, beginline) + if item == "[" and (not openstring and not keygroup and not arrayoftables): + if beginline: + if len(sl) > i + 1 and sl[i + 1] == "[": + arrayoftables = True + else: + keygroup = True + else: + openarr += 1 + if item == "]" and not openstring: + if keygroup: + keygroup = False + elif arrayoftables: + if sl[i - 1] == "]": + arrayoftables = False + else: + openarr -= 1 + if item == "\n": + if openstring or multilinestr: + if not multilinestr: + raise TomlDecodeError("Unbalanced quotes", original, i) + if (sl[i - 1] == "'" or sl[i - 1] == '"') and (sl[i - 2] == sl[i - 1]): + sl[i] = sl[i - 1] + if sl[i - 3] == sl[i - 1]: + sl[i - 3] = " " + elif openarr: + sl[i] = " " + else: + beginline = True + line_no += 1 + elif beginline and sl[i] != " " and sl[i] != "\t": + beginline = False + if not keygroup and not arrayoftables: + if sl[i] == "=": + raise TomlDecodeError("Found empty keyname. ", original, i) + keyname = 1 + key += item + if keyname: + raise TomlDecodeError( + "Key name found without value." " Reached end of file.", original, len(s) + ) + if openstring: # reached EOF and have an unterminated string + raise TomlDecodeError( + "Unterminated string found." " Reached end of file.", original, len(s) + ) + s = "".join(sl) + s = s.split("\n") + multikey = None + multilinestr = "" + multibackslash = False + pos = 0 + for idx, line in enumerate(s): + if idx > 0: + pos += len(s[idx - 1]) + 1 + + decoder.embed_comments(idx, currentlevel) + + if not multilinestr or multibackslash or "\n" not in multilinestr: + line = line.strip() + if line == "" and (not multikey or multibackslash): + continue + if multikey: + if multibackslash: + multilinestr += line + else: + multilinestr += line + multibackslash = False + closed = False + if multilinestr[0] == "[": + closed = line[-1] == "]" + elif len(line) > 2: + closed = ( + line[-1] == multilinestr[0] + and line[-2] == multilinestr[0] + and line[-3] == multilinestr[0] + ) + if closed: + try: + value, vtype = decoder.load_value(multilinestr) + except ValueError as err: + raise TomlDecodeError(str(err), original, pos) + currentlevel[multikey] = value + multikey = None + multilinestr = "" + else: + k = len(multilinestr) - 1 + while k > -1 and multilinestr[k] == "\\": + multibackslash = not multibackslash + k -= 1 + if multibackslash: + multilinestr = multilinestr[:-1] + else: + multilinestr += "\n" + continue + if line[0] == "[": + arrayoftables = False + if len(line) == 1: + raise TomlDecodeError( + "Opening key group bracket on line by " "itself.", original, pos + ) + if line[1] == "[": + arrayoftables = True + line = line[2:] + splitstr = "]]" + else: + line = line[1:] + splitstr = "]" + i = 1 + quotesplits = decoder._get_split_on_quotes(line) + quoted = False + for quotesplit in quotesplits: + if not quoted and splitstr in quotesplit: + break + i += quotesplit.count(splitstr) + quoted = not quoted + line = line.split(splitstr, i) + if len(line) < i + 1 or line[-1].strip() != "": + raise TomlDecodeError("Key group not on a line by itself.", original, pos) + groups = splitstr.join(line[:-1]).split(".") + i = 0 + while i < len(groups): + groups[i] = groups[i].strip() + if len(groups[i]) > 0 and (groups[i][0] == '"' or groups[i][0] == "'"): + groupstr = groups[i] + j = i + 1 + while not groupstr[0] == groupstr[-1]: + j += 1 + if j > len(groups) + 2: + raise TomlDecodeError( + "Invalid group name '" + groupstr + "' Something " + "went wrong.", + original, + pos, + ) + groupstr = ".".join(groups[i:j]).strip() + groups[i] = groupstr[1:-1] + groups[i + 1 : j] = [] + else: + if not _groupname_re.match(groups[i]): + raise TomlDecodeError( + "Invalid group name '" + groups[i] + "'. Try quoting it.", original, pos + ) + i += 1 + currentlevel = retval + for i in _range(len(groups)): + group = groups[i] + if group == "": + raise TomlDecodeError( + "Can't have a keygroup with an empty " "name", original, pos + ) + try: + currentlevel[group] + if i == len(groups) - 1: + if group in implicitgroups: + implicitgroups.remove(group) + if arrayoftables: + raise TomlDecodeError( + "An implicitly defined " "table can't be an array", + original, + pos, + ) + elif arrayoftables: + currentlevel[group].append(decoder.get_empty_table()) + else: + raise TomlDecodeError( + "What? " + group + " already exists?" + str(currentlevel), + original, + pos, + ) + except TypeError: + currentlevel = currentlevel[-1] + if group not in currentlevel: + currentlevel[group] = decoder.get_empty_table() + if i == len(groups) - 1 and arrayoftables: + currentlevel[group] = [decoder.get_empty_table()] + except KeyError: + if i != len(groups) - 1: + implicitgroups.append(group) + currentlevel[group] = decoder.get_empty_table() + if i == len(groups) - 1 and arrayoftables: + currentlevel[group] = [decoder.get_empty_table()] + currentlevel = currentlevel[group] + if arrayoftables: + try: + currentlevel = currentlevel[-1] + except KeyError: + pass + elif line[0] == "{": + if line[-1] != "}": + raise TomlDecodeError( + "Line breaks are not allowed in inline" "objects", original, pos + ) + try: + decoder.load_inline_object(line, currentlevel, multikey, multibackslash) + except ValueError as err: + raise TomlDecodeError(str(err), original, pos) + elif "=" in line: + try: + ret = decoder.load_line(line, currentlevel, multikey, multibackslash) + except ValueError as err: + raise TomlDecodeError(str(err), original, pos) + if ret is not None: + multikey, multilinestr, multibackslash = ret + return retval + + +def _load_date(val): + microsecond = 0 + tz = None + try: + if len(val) > 19: + if val[19] == ".": + if val[-1].upper() == "Z": + subsecondval = val[20:-1] + tzval = "Z" + else: + subsecondvalandtz = val[20:] + if "+" in subsecondvalandtz: + splitpoint = subsecondvalandtz.index("+") + subsecondval = subsecondvalandtz[:splitpoint] + tzval = subsecondvalandtz[splitpoint:] + elif "-" in subsecondvalandtz: + splitpoint = subsecondvalandtz.index("-") + subsecondval = subsecondvalandtz[:splitpoint] + tzval = subsecondvalandtz[splitpoint:] + else: + tzval = None + subsecondval = subsecondvalandtz + if tzval is not None: + tz = TomlTz(tzval) + microsecond = int(int(subsecondval) * (10 ** (6 - len(subsecondval)))) + else: + tz = TomlTz(val[19:]) + except ValueError: + tz = None + if "-" not in val[1:]: + return None + try: + if len(val) == 10: + d = datetime.date(int(val[:4]), int(val[5:7]), int(val[8:10])) + else: + d = datetime.datetime( + int(val[:4]), + int(val[5:7]), + int(val[8:10]), + int(val[11:13]), + int(val[14:16]), + int(val[17:19]), + microsecond, + tz, + ) + except ValueError: + return None + return d + + +def _load_unicode_escapes(v, hexbytes, prefix): + skip = False + i = len(v) - 1 + while i > -1 and v[i] == "\\": + skip = not skip + i -= 1 + for hx in hexbytes: + if skip: + skip = False + i = len(hx) - 1 + while i > -1 and hx[i] == "\\": + skip = not skip + i -= 1 + v += prefix + v += hx + continue + hxb = "" + i = 0 + hxblen = 4 + if prefix == "\\U": + hxblen = 8 + hxb = "".join(hx[i : i + hxblen]).lower() + if hxb.strip("0123456789abcdef"): + raise ValueError("Invalid escape sequence: " + hxb) + if hxb[0] == "d" and hxb[1].strip("01234567"): + raise ValueError( + "Invalid escape sequence: " + hxb + ". Only scalar unicode points are allowed." + ) + v += unichr(int(hxb, 16)) + v += unicode(hx[len(hxb) :]) + return v + + +# Unescape TOML string values. + +# content after the \ +_escapes = ["0", "b", "f", "n", "r", "t", '"'] +# What it should be replaced by +_escapedchars = ["\0", "\b", "\f", "\n", "\r", "\t", '"'] +# Used for substitution +_escape_to_escapedchars = dict(zip(_escapes, _escapedchars)) + + +def _unescape(v): + """Unescape characters in a TOML string.""" + i = 0 + backslash = False + while i < len(v): + if backslash: + backslash = False + if v[i] in _escapes: + v = v[: i - 1] + _escape_to_escapedchars[v[i]] + v[i + 1 :] + elif v[i] == "\\": + v = v[: i - 1] + v[i:] + elif v[i] == "u" or v[i] == "U": + i += 1 + else: + raise ValueError("Reserved escape sequence used") + continue + elif v[i] == "\\": + backslash = True + i += 1 + return v + + +class InlineTableDict(object): + """Sentinel subclass of dict for inline tables.""" + + +class TomlDecoder(object): + def __init__(self, _dict=dict): + self._dict = _dict + + def get_empty_table(self): + return self._dict() + + def get_empty_inline_table(self): + class DynamicInlineTableDict(self._dict, InlineTableDict): + """Concrete sentinel subclass for inline tables. + It is a subclass of _dict which is passed in dynamically at load + time + + It is also a subclass of InlineTableDict + """ + + return DynamicInlineTableDict() + + def load_inline_object(self, line, currentlevel, multikey=False, multibackslash=False): + candidate_groups = line[1:-1].split(",") + groups = [] + if len(candidate_groups) == 1 and not candidate_groups[0].strip(): + candidate_groups.pop() + while len(candidate_groups) > 0: + candidate_group = candidate_groups.pop(0) + try: + _, value = candidate_group.split("=", 1) + except ValueError: + raise ValueError("Invalid inline table encountered") + value = value.strip() + if (value[0] == value[-1] and value[0] in ('"', "'")) or ( + value[0] in "-0123456789" + or value in ("true", "false") + or (value[0] == "[" and value[-1] == "]") + or (value[0] == "{" and value[-1] == "}") + ): + groups.append(candidate_group) + elif len(candidate_groups) > 0: + candidate_groups[0] = candidate_group + "," + candidate_groups[0] + else: + raise ValueError("Invalid inline table value encountered") + for group in groups: + status = self.load_line(group, currentlevel, multikey, multibackslash) + if status is not None: + break + + def _get_split_on_quotes(self, line): + doublequotesplits = line.split('"') + quoted = False + quotesplits = [] + if len(doublequotesplits) > 1 and "'" in doublequotesplits[0]: + singlequotesplits = doublequotesplits[0].split("'") + doublequotesplits = doublequotesplits[1:] + while len(singlequotesplits) % 2 == 0 and len(doublequotesplits): + singlequotesplits[-1] += '"' + doublequotesplits[0] + doublequotesplits = doublequotesplits[1:] + if "'" in singlequotesplits[-1]: + singlequotesplits = singlequotesplits[:-1] + singlequotesplits[-1].split("'") + quotesplits += singlequotesplits + for doublequotesplit in doublequotesplits: + if quoted: + quotesplits.append(doublequotesplit) + else: + quotesplits += doublequotesplit.split("'") + quoted = not quoted + return quotesplits + + def load_line(self, line, currentlevel, multikey, multibackslash): + i = 1 + quotesplits = self._get_split_on_quotes(line) + quoted = False + for quotesplit in quotesplits: + if not quoted and "=" in quotesplit: + break + i += quotesplit.count("=") + quoted = not quoted + pair = line.split("=", i) + strictly_valid = _strictly_valid_num(pair[-1]) + if _number_with_underscores.match(pair[-1]): + pair[-1] = pair[-1].replace("_", "") + while len(pair[-1]) and ( + pair[-1][0] != " " + and pair[-1][0] != "\t" + and pair[-1][0] != "'" + and pair[-1][0] != '"' + and pair[-1][0] != "[" + and pair[-1][0] != "{" + and pair[-1].strip() != "true" + and pair[-1].strip() != "false" + ): + try: + float(pair[-1]) + break + except ValueError: + pass + if _load_date(pair[-1]) is not None: + break + if TIME_RE.match(pair[-1]): + break + i += 1 + prev_val = pair[-1] + pair = line.split("=", i) + if prev_val == pair[-1]: + raise ValueError("Invalid date or number") + if strictly_valid: + strictly_valid = _strictly_valid_num(pair[-1]) + pair = ["=".join(pair[:-1]).strip(), pair[-1].strip()] + if "." in pair[0]: + if '"' in pair[0] or "'" in pair[0]: + quotesplits = self._get_split_on_quotes(pair[0]) + quoted = False + levels = [] + for quotesplit in quotesplits: + if quoted: + levels.append(quotesplit) + else: + levels += [level.strip() for level in quotesplit.split(".")] + quoted = not quoted + else: + levels = pair[0].split(".") + while levels[-1] == "": + levels = levels[:-1] + for level in levels[:-1]: + if level == "": + continue + if level not in currentlevel: + currentlevel[level] = self.get_empty_table() + currentlevel = currentlevel[level] + pair[0] = levels[-1].strip() + elif (pair[0][0] == '"' or pair[0][0] == "'") and (pair[0][-1] == pair[0][0]): + pair[0] = _unescape(pair[0][1:-1]) + k, koffset = self._load_line_multiline_str(pair[1]) + if k > -1: + while k > -1 and pair[1][k + koffset] == "\\": + multibackslash = not multibackslash + k -= 1 + if multibackslash: + multilinestr = pair[1][:-1] + else: + multilinestr = pair[1] + "\n" + multikey = pair[0] + else: + value, vtype = self.load_value(pair[1], strictly_valid) + try: + currentlevel[pair[0]] + raise ValueError("Duplicate keys!") + except TypeError: + raise ValueError("Duplicate keys!") + except KeyError: + if multikey: + return multikey, multilinestr, multibackslash + else: + currentlevel[pair[0]] = value + + def _load_line_multiline_str(self, p): + poffset = 0 + if len(p) < 3: + return -1, poffset + if p[0] == "[" and (p.strip()[-1] != "]" and self._load_array_isstrarray(p)): + newp = p[1:].strip().split(",") + while len(newp) > 1 and newp[-1][0] != '"' and newp[-1][0] != "'": + newp = newp[:-2] + [newp[-2] + "," + newp[-1]] + newp = newp[-1] + poffset = len(p) - len(newp) + p = newp + if p[0] != '"' and p[0] != "'": + return -1, poffset + if p[1] != p[0] or p[2] != p[0]: + return -1, poffset + if len(p) > 5 and p[-1] == p[0] and p[-2] == p[0] and p[-3] == p[0]: + return -1, poffset + return len(p) - 1, poffset + + def load_value(self, v, strictly_valid=True): + if not v: + raise ValueError("Empty value is invalid") + if v == "true": + return (True, "bool") + elif v == "false": + return (False, "bool") + elif v[0] == '"' or v[0] == "'": + quotechar = v[0] + testv = v[1:].split(quotechar) + triplequote = False + triplequotecount = 0 + if len(testv) > 1 and testv[0] == "" and testv[1] == "": + testv = testv[2:] + triplequote = True + closed = False + for tv in testv: + if tv == "": + if triplequote: + triplequotecount += 1 + else: + closed = True + else: + oddbackslash = False + try: + i = -1 + j = tv[i] + while j == "\\": + oddbackslash = not oddbackslash + i -= 1 + j = tv[i] + except IndexError: + pass + if not oddbackslash: + if closed: + raise ValueError( + "Found tokens after a closed " + "string. Invalid TOML." + ) + else: + if not triplequote or triplequotecount > 1: + closed = True + else: + triplequotecount = 0 + if quotechar == '"': + escapeseqs = v.split("\\")[1:] + backslash = False + for i in escapeseqs: + if i == "": + backslash = not backslash + else: + if i[0] not in _escapes and (i[0] != "u" and i[0] != "U" and not backslash): + raise ValueError("Reserved escape sequence used") + if backslash: + backslash = False + for prefix in ["\\u", "\\U"]: + if prefix in v: + hexbytes = v.split(prefix) + v = _load_unicode_escapes(hexbytes[0], hexbytes[1:], prefix) + v = _unescape(v) + if len(v) > 1 and v[1] == quotechar and (len(v) < 3 or v[1] == v[2]): + v = v[2:-2] + return (v[1:-1], "str") + elif v[0] == "[": + return (self.load_array(v), "array") + elif v[0] == "{": + inline_object = self.get_empty_inline_table() + self.load_inline_object(v, inline_object) + return (inline_object, "inline_object") + elif TIME_RE.match(v): + h, m, s, _, ms = TIME_RE.match(v).groups() + time = datetime.time(int(h), int(m), int(s), int(ms) if ms else 0) + return (time, "time") + else: + parsed_date = _load_date(v) + if parsed_date is not None: + return (parsed_date, "date") + if not strictly_valid: + raise ValueError("Weirdness with leading zeroes or " "underscores in your number.") + itype = "int" + neg = False + if v[0] == "-": + neg = True + v = v[1:] + elif v[0] == "+": + v = v[1:] + v = v.replace("_", "") + lowerv = v.lower() + if "." in v or ("x" not in v and ("e" in v or "E" in v)): + if "." in v and v.split(".", 1)[1] == "": + raise ValueError("This float is missing digits after " "the point") + if v[0] not in "0123456789": + raise ValueError("This float doesn't have a leading " "digit") + v = float(v) + itype = "float" + elif len(lowerv) == 3 and (lowerv == "inf" or lowerv == "nan"): + v = float(v) + itype = "float" + if itype == "int": + v = int(v, 0) + if neg: + return (0 - v, itype) + return (v, itype) + + def bounded_string(self, s): + if len(s) == 0: + return True + if s[-1] != s[0]: + return False + i = -2 + backslash = False + while len(s) + i > 0: + if s[i] == "\\": + backslash = not backslash + i -= 1 + else: + break + return not backslash + + def _load_array_isstrarray(self, a): + a = a[1:-1].strip() + if a != "" and (a[0] == '"' or a[0] == "'"): + return True + return False + + def load_array(self, a): + atype = None + retval = [] + a = a.strip() + if "[" not in a[1:-1] or "" != a[1:-1].split("[")[0].strip(): + strarray = self._load_array_isstrarray(a) + if not a[1:-1].strip().startswith("{"): + a = a[1:-1].split(",") + else: + # a is an inline object, we must find the matching parenthesis + # to define groups + new_a = [] + start_group_index = 1 + end_group_index = 2 + open_bracket_count = 1 if a[start_group_index] == "{" else 0 + in_str = False + while end_group_index < len(a[1:]): + if a[end_group_index] == '"' or a[end_group_index] == "'": + if in_str: + backslash_index = end_group_index - 1 + while backslash_index > -1 and a[backslash_index] == "\\": + in_str = not in_str + backslash_index -= 1 + in_str = not in_str + if not in_str and a[end_group_index] == "{": + open_bracket_count += 1 + if in_str or a[end_group_index] != "}": + end_group_index += 1 + continue + elif a[end_group_index] == "}" and open_bracket_count > 1: + open_bracket_count -= 1 + end_group_index += 1 + continue + + # Increase end_group_index by 1 to get the closing bracket + end_group_index += 1 + + new_a.append(a[start_group_index:end_group_index]) + + # The next start index is at least after the closing + # bracket, a closing bracket can be followed by a comma + # since we are in an array. + start_group_index = end_group_index + 1 + while start_group_index < len(a[1:]) and a[start_group_index] != "{": + start_group_index += 1 + end_group_index = start_group_index + 1 + a = new_a + b = 0 + if strarray: + while b < len(a) - 1: + ab = a[b].strip() + while not self.bounded_string(ab) or ( + len(ab) > 2 + and ab[0] == ab[1] == ab[2] + and ab[-2] != ab[0] + and ab[-3] != ab[0] + ): + a[b] = a[b] + "," + a[b + 1] + ab = a[b].strip() + if b < len(a) - 2: + a = a[: b + 1] + a[b + 2 :] + else: + a = a[: b + 1] + b += 1 + else: + al = list(a[1:-1]) + a = [] + openarr = 0 + j = 0 + for i in _range(len(al)): + if al[i] == "[": + openarr += 1 + elif al[i] == "]": + openarr -= 1 + elif al[i] == "," and not openarr: + a.append("".join(al[j:i])) + j = i + 1 + a.append("".join(al[j:])) + for i in _range(len(a)): + a[i] = a[i].strip() + if a[i] != "": + nval, ntype = self.load_value(a[i]) + if atype: + if ntype != atype: + raise ValueError("Not a homogeneous array") + else: + atype = ntype + retval.append(nval) + return retval + + def preserve_comment(self, line_no, key, comment, beginline): + pass + + def embed_comments(self, idx, currentlevel): + pass + + +class TomlPreserveCommentDecoder(TomlDecoder): + def __init__(self, _dict=dict): + self.saved_comments = {} + super(TomlPreserveCommentDecoder, self).__init__(_dict) + + def preserve_comment(self, line_no, key, comment, beginline): + self.saved_comments[line_no] = (key, comment, beginline) + + def embed_comments(self, idx, currentlevel): + if idx not in self.saved_comments: + return + + key, comment, beginline = self.saved_comments[idx] + currentlevel[key] = CommentValue(currentlevel[key], comment, beginline, self._dict) diff --git a/env/lib/python3.8/site-packages/isort/_vendored/toml/encoder.py b/env/lib/python3.8/site-packages/isort/_vendored/toml/encoder.py new file mode 100644 index 000000000..68ec60f9f --- /dev/null +++ b/env/lib/python3.8/site-packages/isort/_vendored/toml/encoder.py @@ -0,0 +1,295 @@ +import datetime +import re +import sys +from decimal import Decimal + +from .decoder import InlineTableDict + +if sys.version_info >= (3,): + unicode = str + + +def dump(o, f, encoder=None): + """Writes out dict as toml to a file + + Args: + o: Object to dump into toml + f: File descriptor where the toml should be stored + encoder: The ``TomlEncoder`` to use for constructing the output string + + Returns: + String containing the toml corresponding to dictionary + + Raises: + TypeError: When anything other than file descriptor is passed + """ + + if not f.write: + raise TypeError("You can only dump an object to a file descriptor") + d = dumps(o, encoder=encoder) + f.write(d) + return d + + +def dumps(o, encoder=None): + """Stringifies input dict as toml + + Args: + o: Object to dump into toml + encoder: The ``TomlEncoder`` to use for constructing the output string + + Returns: + String containing the toml corresponding to dict + + Examples: + ```python + >>> import toml + >>> output = { + ... 'a': "I'm a string", + ... 'b': ["I'm", "a", "list"], + ... 'c': 2400 + ... } + >>> toml.dumps(output) + 'a = "I\'m a string"\nb = [ "I\'m", "a", "list",]\nc = 2400\n' + ``` + """ + + retval = "" + if encoder is None: + encoder = TomlEncoder(o.__class__) + addtoretval, sections = encoder.dump_sections(o, "") + retval += addtoretval + outer_objs = [id(o)] + while sections: + section_ids = [id(section) for section in sections] + for outer_obj in outer_objs: + if outer_obj in section_ids: + raise ValueError("Circular reference detected") + outer_objs += section_ids + newsections = encoder.get_empty_table() + for section in sections: + addtoretval, addtosections = encoder.dump_sections(sections[section], section) + + if addtoretval or (not addtoretval and not addtosections): + if retval and retval[-2:] != "\n\n": + retval += "\n" + retval += "[" + section + "]\n" + if addtoretval: + retval += addtoretval + for s in addtosections: + newsections[section + "." + s] = addtosections[s] + sections = newsections + return retval + + +def _dump_str(v): + if sys.version_info < (3,) and hasattr(v, "decode") and isinstance(v, str): + v = v.decode("utf-8") + v = "%r" % v + if v[0] == "u": + v = v[1:] + singlequote = v.startswith("'") + if singlequote or v.startswith('"'): + v = v[1:-1] + if singlequote: + v = v.replace("\\'", "'") + v = v.replace('"', '\\"') + v = v.split("\\x") + while len(v) > 1: + i = -1 + if not v[0]: + v = v[1:] + v[0] = v[0].replace("\\\\", "\\") + # No, I don't know why != works and == breaks + joinx = v[0][i] != "\\" + while v[0][:i] and v[0][i] == "\\": + joinx = not joinx + i -= 1 + if joinx: + joiner = "x" + else: + joiner = "u00" + v = [v[0] + joiner + v[1]] + v[2:] + return unicode('"' + v[0] + '"') + + +def _dump_float(v): + return "{}".format(v).replace("e+0", "e+").replace("e-0", "e-") + + +def _dump_time(v): + utcoffset = v.utcoffset() + if utcoffset is None: + return v.isoformat() + # The TOML norm specifies that it's local time thus we drop the offset + return v.isoformat()[:-6] + + +class TomlEncoder(object): + def __init__(self, _dict=dict, preserve=False): + self._dict = _dict + self.preserve = preserve + self.dump_funcs = { + str: _dump_str, + unicode: _dump_str, + list: self.dump_list, + bool: lambda v: unicode(v).lower(), + int: lambda v: v, + float: _dump_float, + Decimal: _dump_float, + datetime.datetime: lambda v: v.isoformat().replace("+00:00", "Z"), + datetime.time: _dump_time, + datetime.date: lambda v: v.isoformat(), + } + + def get_empty_table(self): + return self._dict() + + def dump_list(self, v): + retval = "[" + for u in v: + retval += " " + unicode(self.dump_value(u)) + "," + retval += "]" + return retval + + def dump_inline_table(self, section): + """Preserve inline table in its compact syntax instead of expanding + into subsection. + + https://github.com/toml-lang/toml#user-content-inline-table + """ + retval = "" + if isinstance(section, dict): + val_list = [] + for k, v in section.items(): + val = self.dump_inline_table(v) + val_list.append(k + " = " + val) + retval += "{ " + ", ".join(val_list) + " }\n" + return retval + else: + return unicode(self.dump_value(section)) + + def dump_value(self, v): + # Lookup function corresponding to v's type + dump_fn = self.dump_funcs.get(type(v)) + if dump_fn is None and hasattr(v, "__iter__"): + dump_fn = self.dump_funcs[list] + # Evaluate function (if it exists) else return v + return dump_fn(v) if dump_fn is not None else self.dump_funcs[str](v) + + def dump_sections(self, o, sup): + retstr = "" + if sup != "" and sup[-1] != ".": + sup += "." + retdict = self._dict() + arraystr = "" + for section in o: + section = unicode(section) + qsection = section + if not re.match(r"^[A-Za-z0-9_-]+$", section): + qsection = _dump_str(section) + if not isinstance(o[section], dict): + arrayoftables = False + if isinstance(o[section], list): + for a in o[section]: + if isinstance(a, dict): + arrayoftables = True + if arrayoftables: + for a in o[section]: + arraytabstr = "\n" + arraystr += "[[" + sup + qsection + "]]\n" + s, d = self.dump_sections(a, sup + qsection) + if s: + if s[0] == "[": + arraytabstr += s + else: + arraystr += s + while d: + newd = self._dict() + for dsec in d: + s1, d1 = self.dump_sections(d[dsec], sup + qsection + "." + dsec) + if s1: + arraytabstr += "[" + sup + qsection + "." + dsec + "]\n" + arraytabstr += s1 + for s1 in d1: + newd[dsec + "." + s1] = d1[s1] + d = newd + arraystr += arraytabstr + else: + if o[section] is not None: + retstr += qsection + " = " + unicode(self.dump_value(o[section])) + "\n" + elif self.preserve and isinstance(o[section], InlineTableDict): + retstr += qsection + " = " + self.dump_inline_table(o[section]) + else: + retdict[qsection] = o[section] + retstr += arraystr + return (retstr, retdict) + + +class TomlPreserveInlineDictEncoder(TomlEncoder): + def __init__(self, _dict=dict): + super(TomlPreserveInlineDictEncoder, self).__init__(_dict, True) + + +class TomlArraySeparatorEncoder(TomlEncoder): + def __init__(self, _dict=dict, preserve=False, separator=","): + super(TomlArraySeparatorEncoder, self).__init__(_dict, preserve) + if separator.strip() == "": + separator = "," + separator + elif separator.strip(" \t\n\r,"): + raise ValueError("Invalid separator for arrays") + self.separator = separator + + def dump_list(self, v): + t = [] + retval = "[" + for u in v: + t.append(self.dump_value(u)) + while t != []: + s = [] + for u in t: + if isinstance(u, list): + for r in u: + s.append(r) + else: + retval += " " + unicode(u) + self.separator + t = s + retval += "]" + return retval + + +class TomlNumpyEncoder(TomlEncoder): + def __init__(self, _dict=dict, preserve=False): + import numpy as np + + super(TomlNumpyEncoder, self).__init__(_dict, preserve) + self.dump_funcs[np.float16] = _dump_float + self.dump_funcs[np.float32] = _dump_float + self.dump_funcs[np.float64] = _dump_float + self.dump_funcs[np.int16] = self._dump_int + self.dump_funcs[np.int32] = self._dump_int + self.dump_funcs[np.int64] = self._dump_int + + def _dump_int(self, v): + return "{}".format(int(v)) + + +class TomlPreserveCommentEncoder(TomlEncoder): + def __init__(self, _dict=dict, preserve=False): + from toml.decoder import CommentValue + + super(TomlPreserveCommentEncoder, self).__init__(_dict, preserve) + self.dump_funcs[CommentValue] = lambda v: v.dump(self.dump_value) + + +class TomlPathlibEncoder(TomlEncoder): + def _dump_pathlib_path(self, v): + return _dump_str(str(v)) + + def dump_value(self, v): + if (3, 4) <= sys.version_info: + import pathlib + + if isinstance(v, pathlib.PurePath): + v = str(v) + return super(TomlPathlibEncoder, self).dump_value(v) diff --git a/env/lib/python3.8/site-packages/isort/_vendored/toml/ordered.py b/env/lib/python3.8/site-packages/isort/_vendored/toml/ordered.py new file mode 100644 index 000000000..013b31e5c --- /dev/null +++ b/env/lib/python3.8/site-packages/isort/_vendored/toml/ordered.py @@ -0,0 +1,13 @@ +from collections import OrderedDict + +from . import TomlDecoder, TomlEncoder + + +class TomlOrderedDecoder(TomlDecoder): + def __init__(self): + super(self.__class__, self).__init__(_dict=OrderedDict) + + +class TomlOrderedEncoder(TomlEncoder): + def __init__(self): + super(self.__class__, self).__init__(_dict=OrderedDict) diff --git a/env/lib/python3.8/site-packages/isort/_vendored/toml/tz.py b/env/lib/python3.8/site-packages/isort/_vendored/toml/tz.py new file mode 100644 index 000000000..46214bd4d --- /dev/null +++ b/env/lib/python3.8/site-packages/isort/_vendored/toml/tz.py @@ -0,0 +1,21 @@ +from datetime import timedelta, tzinfo + + +class TomlTz(tzinfo): + def __init__(self, toml_offset): + if toml_offset == "Z": + self._raw_offset = "+00:00" + else: + self._raw_offset = toml_offset + self._sign = -1 if self._raw_offset[0] == "-" else 1 + self._hours = int(self._raw_offset[1:3]) + self._minutes = int(self._raw_offset[4:6]) + + def tzname(self, dt): + return "UTC" + self._raw_offset + + def utcoffset(self, dt): + return self._sign * timedelta(hours=self._hours, minutes=self._minutes) + + def dst(self, dt): + return timedelta(0) diff --git a/env/lib/python3.8/site-packages/isort/_version.py b/env/lib/python3.8/site-packages/isort/_version.py new file mode 100644 index 000000000..16b899cb9 --- /dev/null +++ b/env/lib/python3.8/site-packages/isort/_version.py @@ -0,0 +1 @@ +__version__ = "5.5.3" diff --git a/env/lib/python3.8/site-packages/isort/api.py b/env/lib/python3.8/site-packages/isort/api.py new file mode 100644 index 000000000..cbcc3e6ea --- /dev/null +++ b/env/lib/python3.8/site-packages/isort/api.py @@ -0,0 +1,387 @@ +import shutil +import sys +from io import StringIO +from pathlib import Path +from typing import Optional, TextIO, Union, cast +from warnings import warn + +from isort import core + +from . import io +from .exceptions import ( + ExistingSyntaxErrors, + FileSkipComment, + FileSkipSetting, + IntroducedSyntaxErrors, +) +from .format import ask_whether_to_apply_changes_to_file, create_terminal_printer, show_unified_diff +from .io import Empty +from .place import module as place_module # noqa: F401 +from .place import module_with_reason as place_module_with_reason # noqa: F401 +from .settings import DEFAULT_CONFIG, Config + + +def sort_code_string( + code: str, + extension: Optional[str] = None, + config: Config = DEFAULT_CONFIG, + file_path: Optional[Path] = None, + disregard_skip: bool = False, + show_diff: Union[bool, TextIO] = False, + **config_kwargs, +): + """Sorts any imports within the provided code string, returning a new string with them sorted. + + - **code**: The string of code with imports that need to be sorted. + - **extension**: The file extension that contains imports. Defaults to filename extension or py. + - **config**: The config object to use when sorting imports. + - **file_path**: The disk location where the code string was pulled from. + - **disregard_skip**: set to `True` if you want to ignore a skip set in config for this file. + - **show_diff**: If `True` the changes that need to be done will be printed to stdout, if a + TextIO stream is provided results will be written to it, otherwise no diff will be computed. + - ****config_kwargs**: Any config modifications. + """ + input_stream = StringIO(code) + output_stream = StringIO() + config = _config(path=file_path, config=config, **config_kwargs) + sort_stream( + input_stream, + output_stream, + extension=extension, + config=config, + file_path=file_path, + disregard_skip=disregard_skip, + show_diff=show_diff, + ) + output_stream.seek(0) + return output_stream.read() + + +def check_code_string( + code: str, + show_diff: Union[bool, TextIO] = False, + extension: Optional[str] = None, + config: Config = DEFAULT_CONFIG, + file_path: Optional[Path] = None, + disregard_skip: bool = False, + **config_kwargs, +) -> bool: + """Checks the order, format, and categorization of imports within the provided code string. + Returns `True` if everything is correct, otherwise `False`. + + - **code**: The string of code with imports that need to be sorted. + - **show_diff**: If `True` the changes that need to be done will be printed to stdout, if a + TextIO stream is provided results will be written to it, otherwise no diff will be computed. + - **extension**: The file extension that contains imports. Defaults to filename extension or py. + - **config**: The config object to use when sorting imports. + - **file_path**: The disk location where the code string was pulled from. + - **disregard_skip**: set to `True` if you want to ignore a skip set in config for this file. + - ****config_kwargs**: Any config modifications. + """ + config = _config(path=file_path, config=config, **config_kwargs) + return check_stream( + StringIO(code), + show_diff=show_diff, + extension=extension, + config=config, + file_path=file_path, + disregard_skip=disregard_skip, + ) + + +def sort_stream( + input_stream: TextIO, + output_stream: TextIO, + extension: Optional[str] = None, + config: Config = DEFAULT_CONFIG, + file_path: Optional[Path] = None, + disregard_skip: bool = False, + show_diff: Union[bool, TextIO] = False, + **config_kwargs, +) -> bool: + """Sorts any imports within the provided code stream, outputs to the provided output stream. + Returns `True` if anything is modified from the original input stream, otherwise `False`. + + - **input_stream**: The stream of code with imports that need to be sorted. + - **output_stream**: The stream where sorted imports should be written to. + - **extension**: The file extension that contains imports. Defaults to filename extension or py. + - **config**: The config object to use when sorting imports. + - **file_path**: The disk location where the code string was pulled from. + - **disregard_skip**: set to `True` if you want to ignore a skip set in config for this file. + - **show_diff**: If `True` the changes that need to be done will be printed to stdout, if a + TextIO stream is provided results will be written to it, otherwise no diff will be computed. + - ****config_kwargs**: Any config modifications. + """ + if show_diff: + _output_stream = StringIO() + _input_stream = StringIO(input_stream.read()) + changed = sort_stream( + input_stream=_input_stream, + output_stream=_output_stream, + extension=extension, + config=config, + file_path=file_path, + disregard_skip=disregard_skip, + **config_kwargs, + ) + _output_stream.seek(0) + _input_stream.seek(0) + show_unified_diff( + file_input=_input_stream.read(), + file_output=_output_stream.read(), + file_path=file_path, + output=output_stream if show_diff is True else cast(TextIO, show_diff), + color_output=config.color_output, + ) + return changed + + config = _config(path=file_path, config=config, **config_kwargs) + content_source = str(file_path or "Passed in content") + if not disregard_skip: + if file_path and config.is_skipped(file_path): + raise FileSkipSetting(content_source) + + _internal_output = output_stream + + if config.atomic: + try: + file_content = input_stream.read() + compile(file_content, content_source, "exec", 0, 1) + input_stream = StringIO(file_content) + except SyntaxError: + raise ExistingSyntaxErrors(content_source) + + if not output_stream.readable(): + _internal_output = StringIO() + + try: + changed = core.process( + input_stream, + _internal_output, + extension=extension or (file_path and file_path.suffix.lstrip(".")) or "py", + config=config, + ) + except FileSkipComment: + raise FileSkipComment(content_source) + + if config.atomic: + _internal_output.seek(0) + try: + compile(_internal_output.read(), content_source, "exec", 0, 1) + _internal_output.seek(0) + if _internal_output != output_stream: + output_stream.write(_internal_output.read()) + except SyntaxError: # pragma: no cover + raise IntroducedSyntaxErrors(content_source) + + return changed + + +def check_stream( + input_stream: TextIO, + show_diff: Union[bool, TextIO] = False, + extension: Optional[str] = None, + config: Config = DEFAULT_CONFIG, + file_path: Optional[Path] = None, + disregard_skip: bool = False, + **config_kwargs, +) -> bool: + """Checks any imports within the provided code stream, returning `False` if any unsorted or + incorrectly imports are found or `True` if no problems are identified. + + - **input_stream**: The stream of code with imports that need to be sorted. + - **show_diff**: If `True` the changes that need to be done will be printed to stdout, if a + TextIO stream is provided results will be written to it, otherwise no diff will be computed. + - **extension**: The file extension that contains imports. Defaults to filename extension or py. + - **config**: The config object to use when sorting imports. + - **file_path**: The disk location where the code string was pulled from. + - **disregard_skip**: set to `True` if you want to ignore a skip set in config for this file. + - ****config_kwargs**: Any config modifications. + """ + config = _config(path=file_path, config=config, **config_kwargs) + + changed: bool = sort_stream( + input_stream=input_stream, + output_stream=Empty, + extension=extension, + config=config, + file_path=file_path, + disregard_skip=disregard_skip, + ) + printer = create_terminal_printer(color=config.color_output) + if not changed: + if config.verbose: + printer.success(f"{file_path or ''} Everything Looks Good!") + return True + else: + printer.error(f"{file_path or ''} Imports are incorrectly sorted and/or formatted.") + if show_diff: + output_stream = StringIO() + input_stream.seek(0) + file_contents = input_stream.read() + sort_stream( + input_stream=StringIO(file_contents), + output_stream=output_stream, + extension=extension, + config=config, + file_path=file_path, + disregard_skip=disregard_skip, + ) + output_stream.seek(0) + + show_unified_diff( + file_input=file_contents, + file_output=output_stream.read(), + file_path=file_path, + output=None if show_diff is True else cast(TextIO, show_diff), + color_output=config.color_output, + ) + return False + + +def check_file( + filename: Union[str, Path], + show_diff: Union[bool, TextIO] = False, + config: Config = DEFAULT_CONFIG, + file_path: Optional[Path] = None, + disregard_skip: bool = True, + extension: Optional[str] = None, + **config_kwargs, +) -> bool: + """Checks any imports within the provided file, returning `False` if any unsorted or + incorrectly imports are found or `True` if no problems are identified. + + - **filename**: The name or Path of the file to check. + - **show_diff**: If `True` the changes that need to be done will be printed to stdout, if a + TextIO stream is provided results will be written to it, otherwise no diff will be computed. + - **config**: The config object to use when sorting imports. + - **file_path**: The disk location where the code string was pulled from. + - **disregard_skip**: set to `True` if you want to ignore a skip set in config for this file. + - **extension**: The file extension that contains imports. Defaults to filename extension or py. + - ****config_kwargs**: Any config modifications. + """ + with io.File.read(filename) as source_file: + return check_stream( + source_file.stream, + show_diff=show_diff, + extension=extension, + config=config, + file_path=file_path or source_file.path, + disregard_skip=disregard_skip, + **config_kwargs, + ) + + +def sort_file( + filename: Union[str, Path], + extension: Optional[str] = None, + config: Config = DEFAULT_CONFIG, + file_path: Optional[Path] = None, + disregard_skip: bool = True, + ask_to_apply: bool = False, + show_diff: Union[bool, TextIO] = False, + write_to_stdout: bool = False, + **config_kwargs, +) -> bool: + """Sorts and formats any groups of imports imports within the provided file or Path. + Returns `True` if the file has been changed, otherwise `False`. + + - **filename**: The name or Path of the file to format. + - **extension**: The file extension that contains imports. Defaults to filename extension or py. + - **config**: The config object to use when sorting imports. + - **file_path**: The disk location where the code string was pulled from. + - **disregard_skip**: set to `True` if you want to ignore a skip set in config for this file. + - **ask_to_apply**: If `True`, prompt before applying any changes. + - **show_diff**: If `True` the changes that need to be done will be printed to stdout, if a + TextIO stream is provided results will be written to it, otherwise no diff will be computed. + - **write_to_stdout**: If `True`, write to stdout instead of the input file. + - ****config_kwargs**: Any config modifications. + """ + with io.File.read(filename) as source_file: + actual_file_path = file_path or source_file.path + changed: bool = False + try: + if write_to_stdout: + changed = sort_stream( + input_stream=source_file.stream, + output_stream=sys.stdout, + config=config, + file_path=actual_file_path, + disregard_skip=disregard_skip, + extension=extension, + **config_kwargs, + ) + else: + tmp_file = source_file.path.with_suffix(source_file.path.suffix + ".isorted") + try: + with tmp_file.open( + "w", encoding=source_file.encoding, newline="" + ) as output_stream: + shutil.copymode(filename, tmp_file) + changed = sort_stream( + input_stream=source_file.stream, + output_stream=output_stream, + config=config, + file_path=actual_file_path, + disregard_skip=disregard_skip, + extension=extension, + **config_kwargs, + ) + if changed: + if show_diff or ask_to_apply: + source_file.stream.seek(0) + with tmp_file.open( + encoding=source_file.encoding, newline="" + ) as tmp_out: + show_unified_diff( + file_input=source_file.stream.read(), + file_output=tmp_out.read(), + file_path=actual_file_path, + output=None if show_diff is True else cast(TextIO, show_diff), + color_output=config.color_output, + ) + if show_diff or ( + ask_to_apply + and not ask_whether_to_apply_changes_to_file( + str(source_file.path) + ) + ): + return False + source_file.stream.close() + tmp_file.replace(source_file.path) + if not config.quiet: + print(f"Fixing {source_file.path}") + finally: + try: # Python 3.8+: use `missing_ok=True` instead of try except. + tmp_file.unlink() + except FileNotFoundError: + pass + except ExistingSyntaxErrors: + warn(f"{actual_file_path} unable to sort due to existing syntax errors") + except IntroducedSyntaxErrors: # pragma: no cover + warn(f"{actual_file_path} unable to sort as isort introduces new syntax errors") + + return changed + + +def _config( + path: Optional[Path] = None, config: Config = DEFAULT_CONFIG, **config_kwargs +) -> Config: + if path: + if ( + config is DEFAULT_CONFIG + and "settings_path" not in config_kwargs + and "settings_file" not in config_kwargs + ): + config_kwargs["settings_path"] = path + + if config_kwargs: + if config is not DEFAULT_CONFIG: + raise ValueError( + "You can either specify custom configuration options using kwargs or " + "passing in a Config object. Not Both!" + ) + + config = Config(**config_kwargs) + + return config diff --git a/env/lib/python3.8/site-packages/isort/comments.py b/env/lib/python3.8/site-packages/isort/comments.py new file mode 100644 index 000000000..b865b3281 --- /dev/null +++ b/env/lib/python3.8/site-packages/isort/comments.py @@ -0,0 +1,32 @@ +from typing import List, Optional, Tuple + + +def parse(line: str) -> Tuple[str, str]: + """Parses import lines for comments and returns back the + import statement and the associated comment. + """ + comment_start = line.find("#") + if comment_start != -1: + return (line[:comment_start], line[comment_start + 1 :].strip()) + + return (line, "") + + +def add_to_line( + comments: Optional[List[str]], + original_string: str = "", + removed: bool = False, + comment_prefix: str = "", +) -> str: + """Returns a string with comments added if removed is not set.""" + if removed: + return parse(original_string)[0] + + if not comments: + return original_string + else: + unique_comments: List[str] = [] + for comment in comments: + if comment not in unique_comments: + unique_comments.append(comment) + return f"{parse(original_string)[0]}{comment_prefix} {'; '.join(unique_comments)}" diff --git a/env/lib/python3.8/site-packages/isort/core.py b/env/lib/python3.8/site-packages/isort/core.py new file mode 100644 index 000000000..d1e945d48 --- /dev/null +++ b/env/lib/python3.8/site-packages/isort/core.py @@ -0,0 +1,415 @@ +import textwrap +from io import StringIO +from itertools import chain +from typing import List, TextIO, Union + +import isort.literal +from isort.settings import DEFAULT_CONFIG, Config + +from . import output, parse +from .exceptions import FileSkipComment +from .format import format_natural, remove_whitespace +from .settings import FILE_SKIP_COMMENTS + +CIMPORT_IDENTIFIERS = ("cimport ", "cimport*", "from.cimport") +IMPORT_START_IDENTIFIERS = ("from ", "from.import", "import ", "import*") + CIMPORT_IDENTIFIERS +COMMENT_INDICATORS = ('"""', "'''", "'", '"', "#") +CODE_SORT_COMMENTS = ( + "# isort: list", + "# isort: dict", + "# isort: set", + "# isort: unique-list", + "# isort: tuple", + "# isort: unique-tuple", + "# isort: assignments", +) + + +def process( + input_stream: TextIO, + output_stream: TextIO, + extension: str = "py", + config: Config = DEFAULT_CONFIG, +) -> bool: + """Parses stream identifying sections of contiguous imports and sorting them + + Code with unsorted imports is read from the provided `input_stream`, sorted and then + outputted to the specified `output_stream`. + + - `input_stream`: Text stream with unsorted import sections. + - `output_stream`: Text stream to output sorted inputs into. + - `config`: Config settings to use when sorting imports. Defaults settings. + - *Default*: `isort.settings.DEFAULT_CONFIG`. + - `extension`: The file extension or file extension rules that should be used. + - *Default*: `"py"`. + - *Choices*: `["py", "pyi", "pyx"]`. + + Returns `True` if there were changes that needed to be made (errors present) from what + was provided in the input_stream, otherwise `False`. + """ + line_separator: str = config.line_ending + add_imports: List[str] = [format_natural(addition) for addition in config.add_imports] + import_section: str = "" + next_import_section: str = "" + next_cimports: bool = False + in_quote: str = "" + first_comment_index_start: int = -1 + first_comment_index_end: int = -1 + contains_imports: bool = False + in_top_comment: bool = False + first_import_section: bool = True + indent: str = "" + isort_off: bool = False + code_sorting: Union[bool, str] = False + code_sorting_section: str = "" + code_sorting_indent: str = "" + cimports: bool = False + made_changes: bool = False + stripped_line: str = "" + end_of_file: bool = False + + if config.float_to_top: + new_input = "" + current = "" + isort_off = False + for line in chain(input_stream, (None,)): + if isort_off and line is not None: + if line == "# isort: on\n": + isort_off = False + new_input += line + elif line in ("# isort: split\n", "# isort: off\n", None) or str(line).endswith( + "# isort: split\n" + ): + if line == "# isort: off\n": + isort_off = True + if current: + parsed = parse.file_contents(current, config=config) + extra_space = "" + while current and current[-1] == "\n": + extra_space += "\n" + current = current[:-1] + extra_space = extra_space.replace("\n", "", 1) + sorted_output = output.sorted_imports( + parsed, config, extension, import_type="import" + ) + made_changes = made_changes or _has_changed( + before=current, + after=sorted_output, + line_separator=parsed.line_separator, + ignore_whitespace=config.ignore_whitespace, + ) + new_input += sorted_output + new_input += extra_space + current = "" + new_input += line or "" + else: + current += line or "" + + input_stream = StringIO(new_input) + + for index, line in enumerate(chain(input_stream, (None,))): + if line is None: + if index == 0 and not config.force_adds: + return False + + not_imports = True + end_of_file = True + line = "" + if not line_separator: + line_separator = "\n" + + if code_sorting and code_sorting_section: + output_stream.write( + textwrap.indent( + isort.literal.assignment( + code_sorting_section, + str(code_sorting), + extension, + config=_indented_config(config, indent), + ), + code_sorting_indent, + ) + ) + else: + stripped_line = line.strip() + if stripped_line and not line_separator: + line_separator = line[len(line.rstrip()) :].replace(" ", "").replace("\t", "") + + for file_skip_comment in FILE_SKIP_COMMENTS: + if file_skip_comment in line: + raise FileSkipComment("Passed in content") + + if not in_quote and stripped_line == "# isort: off": + isort_off = True + + if ( + (index == 0 or (index in (1, 2) and not contains_imports)) + and stripped_line.startswith("#") + and stripped_line not in config.section_comments + ): + in_top_comment = True + elif in_top_comment: + if not line.startswith("#") or stripped_line in config.section_comments: + in_top_comment = False + first_comment_index_end = index - 1 + + if (not stripped_line.startswith("#") or in_quote) and '"' in line or "'" in line: + char_index = 0 + if first_comment_index_start == -1 and ( + line.startswith('"') or line.startswith("'") + ): + first_comment_index_start = index + while char_index < len(line): + if line[char_index] == "\\": + char_index += 1 + elif in_quote: + if line[char_index : char_index + len(in_quote)] == in_quote: + in_quote = "" + if first_comment_index_end < first_comment_index_start: + first_comment_index_end = index + elif line[char_index] in ("'", '"'): + long_quote = line[char_index : char_index + 3] + if long_quote in ('"""', "'''"): + in_quote = long_quote + char_index += 2 + else: + in_quote = line[char_index] + elif line[char_index] == "#": + break + char_index += 1 + + not_imports = bool(in_quote) or in_top_comment or isort_off + if not (in_quote or in_top_comment): + if isort_off: + if stripped_line == "# isort: on": + isort_off = False + elif stripped_line.endswith("# isort: split"): + not_imports = True + elif stripped_line in CODE_SORT_COMMENTS: + code_sorting = stripped_line.split("isort: ")[1].strip() + code_sorting_indent = line[: -len(line.lstrip())] + not_imports = True + elif code_sorting: + if not stripped_line: + output_stream.write( + textwrap.indent( + isort.literal.assignment( + code_sorting_section, + str(code_sorting), + extension, + config=_indented_config(config, indent), + ), + code_sorting_indent, + ) + ) + not_imports = True + code_sorting = False + code_sorting_section = "" + code_sorting_indent = "" + else: + code_sorting_section += line + line = "" + elif stripped_line in config.section_comments: + if import_section and not contains_imports: + output_stream.write(import_section) + import_section = line + not_imports = False + else: + import_section += line + indent = line[: -len(line.lstrip())] + elif not (stripped_line or contains_imports): + not_imports = True + elif ( + not stripped_line + or stripped_line.startswith("#") + and (not indent or indent + line.lstrip() == line) + and not config.treat_all_comments_as_code + and stripped_line not in config.treat_comments_as_code + ): + import_section += line + elif stripped_line.startswith(IMPORT_START_IDENTIFIERS): + contains_imports = True + + new_indent = line[: -len(line.lstrip())] + import_statement = line + stripped_line = line.strip().split("#")[0] + while stripped_line.endswith("\\") or ( + "(" in stripped_line and ")" not in stripped_line + ): + if stripped_line.endswith("\\"): + while stripped_line and stripped_line.endswith("\\"): + line = input_stream.readline() + stripped_line = line.strip().split("#")[0] + import_statement += line + else: + while ")" not in stripped_line: + line = input_stream.readline() + stripped_line = line.strip().split("#")[0] + import_statement += line + + cimport_statement: bool = False + if ( + import_statement.lstrip().startswith(CIMPORT_IDENTIFIERS) + or " cimport " in import_statement + or " cimport*" in import_statement + or " cimport(" in import_statement + or ".cimport" in import_statement + ): + cimport_statement = True + + if cimport_statement != cimports or (new_indent != indent and import_section): + if import_section: + next_cimports = cimport_statement + next_import_section = import_statement + import_statement = "" + not_imports = True + line = "" + else: + cimports = cimport_statement + + indent = new_indent + import_section += import_statement + else: + not_imports = True + + if not_imports: + raw_import_section: str = import_section + if ( + add_imports + and (stripped_line or end_of_file) + and not config.append_only + and not in_top_comment + and not in_quote + and not import_section + and not line.lstrip().startswith(COMMENT_INDICATORS) + ): + import_section = line_separator.join(add_imports) + line_separator + if end_of_file and index != 0: + output_stream.write(line_separator) + contains_imports = True + add_imports = [] + + if next_import_section and not import_section: # pragma: no cover + raw_import_section = import_section = next_import_section + next_import_section = "" + + if import_section: + if add_imports and not indent: + import_section = ( + line_separator.join(add_imports) + line_separator + import_section + ) + contains_imports = True + add_imports = [] + + if not indent: + import_section += line + raw_import_section += line + if not contains_imports: + output_stream.write(import_section) + + else: + leading_whitespace = import_section[: -len(import_section.lstrip())] + trailing_whitespace = import_section[len(import_section.rstrip()) :] + if first_import_section and not import_section.lstrip( + line_separator + ).startswith(COMMENT_INDICATORS): + import_section = import_section.lstrip(line_separator) + raw_import_section = raw_import_section.lstrip(line_separator) + first_import_section = False + + if indent: + import_section = "".join( + line[len(indent) :] for line in import_section.splitlines(keepends=True) + ) + + sorted_import_section = output.sorted_imports( + parse.file_contents(import_section, config=config), + _indented_config(config, indent), + extension, + import_type="cimport" if cimports else "import", + ) + if not (import_section.strip() and not sorted_import_section): + if indent: + sorted_import_section = ( + leading_whitespace + + textwrap.indent(sorted_import_section, indent).strip() + + trailing_whitespace + ) + + made_changes = made_changes or _has_changed( + before=raw_import_section, + after=sorted_import_section, + line_separator=line_separator, + ignore_whitespace=config.ignore_whitespace, + ) + output_stream.write(sorted_import_section) + if not line and not indent and next_import_section: + output_stream.write(line_separator) + + if indent: + output_stream.write(line) + if not next_import_section: + indent = "" + + if next_import_section: + cimports = next_cimports + contains_imports = True + else: + contains_imports = False + import_section = next_import_section + next_import_section = "" + else: + output_stream.write(line) + not_imports = False + + if stripped_line and not in_quote and not import_section and not next_import_section: + if stripped_line == "yield": + while not stripped_line or stripped_line == "yield": + new_line = input_stream.readline() + if not new_line: + break + + output_stream.write(new_line) + stripped_line = new_line.strip().split("#")[0] + + if stripped_line.startswith("raise") or stripped_line.startswith("yield"): + if "(" in stripped_line: + while ")" not in stripped_line: + new_line = input_stream.readline() + if not new_line: + break + + output_stream.write(new_line) + stripped_line = new_line.strip().split("#")[0] + + while stripped_line.endswith("\\"): + new_line = input_stream.readline() + if not new_line: + break + + output_stream.write(new_line) + stripped_line = new_line.strip().split("#")[0] + + return made_changes + + +def _indented_config(config: Config, indent: str): + if not indent: + return config + + return Config( + config=config, + line_length=max(config.line_length - len(indent), 0), + wrap_length=max(config.wrap_length - len(indent), 0), + lines_after_imports=1, + ) + + +def _has_changed(before: str, after: str, line_separator: str, ignore_whitespace: bool) -> bool: + if ignore_whitespace: + return ( + remove_whitespace(before, line_separator=line_separator).strip() + != remove_whitespace(after, line_separator=line_separator).strip() + ) + else: + return before.strip() != after.strip() diff --git a/env/lib/python3.8/site-packages/isort/deprecated/__init__.py b/env/lib/python3.8/site-packages/isort/deprecated/__init__.py new file mode 100644 index 000000000..e69de29bb diff --git a/env/lib/python3.8/site-packages/isort/deprecated/finders.py b/env/lib/python3.8/site-packages/isort/deprecated/finders.py new file mode 100644 index 000000000..77eb23fa4 --- /dev/null +++ b/env/lib/python3.8/site-packages/isort/deprecated/finders.py @@ -0,0 +1,403 @@ +"""Finders try to find right section for passed module name""" +import importlib.machinery +import inspect +import os +import os.path +import re +import sys +import sysconfig +from abc import ABCMeta, abstractmethod +from fnmatch import fnmatch +from functools import lru_cache +from glob import glob +from pathlib import Path +from typing import Dict, Iterable, Iterator, List, Optional, Pattern, Sequence, Tuple, Type + +from isort import sections +from isort.settings import KNOWN_SECTION_MAPPING, Config +from isort.utils import chdir, exists_case_sensitive + +try: + from pipreqs import pipreqs + +except ImportError: + pipreqs = None + +try: + from pip_api import parse_requirements + +except ImportError: + parse_requirements = None + +try: + from requirementslib import Pipfile + +except ImportError: + Pipfile = None + + +class BaseFinder(metaclass=ABCMeta): + def __init__(self, config: Config) -> None: + self.config = config + + @abstractmethod + def find(self, module_name: str) -> Optional[str]: + raise NotImplementedError + + +class ForcedSeparateFinder(BaseFinder): + def find(self, module_name: str) -> Optional[str]: + for forced_separate in self.config.forced_separate: + # Ensure all forced_separate patterns will match to end of string + path_glob = forced_separate + if not forced_separate.endswith("*"): + path_glob = "%s*" % forced_separate + + if fnmatch(module_name, path_glob) or fnmatch(module_name, "." + path_glob): + return forced_separate + return None + + +class LocalFinder(BaseFinder): + def find(self, module_name: str) -> Optional[str]: + if module_name.startswith("."): + return "LOCALFOLDER" + return None + + +class KnownPatternFinder(BaseFinder): + def __init__(self, config: Config) -> None: + super().__init__(config) + + self.known_patterns: List[Tuple[Pattern[str], str]] = [] + for placement in reversed(config.sections): + known_placement = KNOWN_SECTION_MAPPING.get(placement, placement).lower() + config_key = f"known_{known_placement}" + known_patterns = list( + getattr(self.config, config_key, self.config.known_other.get(known_placement, [])) + ) + known_patterns = [ + pattern + for known_pattern in known_patterns + for pattern in self._parse_known_pattern(known_pattern) + ] + for known_pattern in known_patterns: + regexp = "^" + known_pattern.replace("*", ".*").replace("?", ".?") + "$" + self.known_patterns.append((re.compile(regexp), placement)) + + def _parse_known_pattern(self, pattern: str) -> List[str]: + """Expand pattern if identified as a directory and return found sub packages""" + if pattern.endswith(os.path.sep): + patterns = [ + filename + for filename in os.listdir(os.path.join(self.config.directory, pattern)) + if os.path.isdir(os.path.join(self.config.directory, pattern, filename)) + ] + else: + patterns = [pattern] + + return patterns + + def find(self, module_name: str) -> Optional[str]: + # Try to find most specific placement instruction match (if any) + parts = module_name.split(".") + module_names_to_check = (".".join(parts[:first_k]) for first_k in range(len(parts), 0, -1)) + for module_name_to_check in module_names_to_check: + for pattern, placement in self.known_patterns: + if pattern.match(module_name_to_check): + return placement + return None + + +class PathFinder(BaseFinder): + def __init__(self, config: Config, path: str = ".") -> None: + super().__init__(config) + + # restore the original import path (i.e. not the path to bin/isort) + root_dir = os.path.abspath(path) + src_dir = f"{root_dir}/src" + self.paths = [root_dir, src_dir] + + # virtual env + self.virtual_env = self.config.virtual_env or os.environ.get("VIRTUAL_ENV") + if self.virtual_env: + self.virtual_env = os.path.realpath(self.virtual_env) + self.virtual_env_src = "" + if self.virtual_env: + self.virtual_env_src = f"{self.virtual_env}/src/" + for venv_path in glob(f"{self.virtual_env}/lib/python*/site-packages"): + if venv_path not in self.paths: + self.paths.append(venv_path) + for nested_venv_path in glob(f"{self.virtual_env}/lib/python*/*/site-packages"): + if nested_venv_path not in self.paths: + self.paths.append(nested_venv_path) + for venv_src_path in glob(f"{self.virtual_env}/src/*"): + if os.path.isdir(venv_src_path): + self.paths.append(venv_src_path) + + # conda + self.conda_env = self.config.conda_env or os.environ.get("CONDA_PREFIX") or "" + if self.conda_env: + self.conda_env = os.path.realpath(self.conda_env) + for conda_path in glob(f"{self.conda_env}/lib/python*/site-packages"): + if conda_path not in self.paths: + self.paths.append(conda_path) + for nested_conda_path in glob(f"{self.conda_env}/lib/python*/*/site-packages"): + if nested_conda_path not in self.paths: + self.paths.append(nested_conda_path) + + # handle case-insensitive paths on windows + self.stdlib_lib_prefix = os.path.normcase(sysconfig.get_paths()["stdlib"]) + if self.stdlib_lib_prefix not in self.paths: + self.paths.append(self.stdlib_lib_prefix) + + # add system paths + for system_path in sys.path[1:]: + if system_path not in self.paths: + self.paths.append(system_path) + + def find(self, module_name: str) -> Optional[str]: + for prefix in self.paths: + package_path = "/".join((prefix, module_name.split(".")[0])) + path_obj = Path(package_path).resolve() + is_module = ( + exists_case_sensitive(package_path + ".py") + or any( + exists_case_sensitive(package_path + ext_suffix) + for ext_suffix in importlib.machinery.EXTENSION_SUFFIXES + ) + or exists_case_sensitive(package_path + "/__init__.py") + ) + is_package = exists_case_sensitive(package_path) and os.path.isdir(package_path) + if is_module or is_package: + if ( + "site-packages" in prefix + or "dist-packages" in prefix + or (self.virtual_env and self.virtual_env_src in prefix) + ): + return sections.THIRDPARTY + elif os.path.normcase(prefix) == self.stdlib_lib_prefix: + return sections.STDLIB + elif self.conda_env and self.conda_env in prefix: + return sections.THIRDPARTY + for src_path in self.config.src_paths: + if src_path in path_obj.parents and not self.config.is_skipped(path_obj): + return sections.FIRSTPARTY + + if os.path.normcase(prefix).startswith(self.stdlib_lib_prefix): + return sections.STDLIB # pragma: no cover - edge case for one OS. Hard to test. + + return self.config.default_section + return None + + +class ReqsBaseFinder(BaseFinder): + enabled = False + + def __init__(self, config: Config, path: str = ".") -> None: + super().__init__(config) + self.path = path + if self.enabled: + self.mapping = self._load_mapping() + self.names = self._load_names() + + @abstractmethod + def _get_names(self, path: str) -> Iterator[str]: + raise NotImplementedError + + @abstractmethod + def _get_files_from_dir(self, path: str) -> Iterator[str]: + raise NotImplementedError + + @staticmethod + def _load_mapping() -> Optional[Dict[str, str]]: + """Return list of mappings `package_name -> module_name` + + Example: + django-haystack -> haystack + """ + if not pipreqs: + return None + path = os.path.dirname(inspect.getfile(pipreqs)) + path = os.path.join(path, "mapping") + with open(path) as f: + mappings: Dict[str, str] = {} # pypi_name: import_name + for line in f: + import_name, _, pypi_name = line.strip().partition(":") + mappings[pypi_name] = import_name + return mappings + # return dict(tuple(line.strip().split(":")[::-1]) for line in f) + + def _load_names(self) -> List[str]: + """Return list of thirdparty modules from requirements""" + names = [] + for path in self._get_files(): + for name in self._get_names(path): + names.append(self._normalize_name(name)) + return names + + @staticmethod + def _get_parents(path: str) -> Iterator[str]: + prev = "" + while path != prev: + prev = path + yield path + path = os.path.dirname(path) + + def _get_files(self) -> Iterator[str]: + """Return paths to all requirements files""" + path = os.path.abspath(self.path) + if os.path.isfile(path): + path = os.path.dirname(path) + + for path in self._get_parents(path): + yield from self._get_files_from_dir(path) + + def _normalize_name(self, name: str) -> str: + """Convert package name to module name + + Examples: + Django -> django + django-haystack -> django_haystack + Flask-RESTFul -> flask_restful + """ + if self.mapping: + name = self.mapping.get(name.replace("-", "_"), name) + return name.lower().replace("-", "_") + + def find(self, module_name: str) -> Optional[str]: + # required lib not installed yet + if not self.enabled: + return None + + module_name, _sep, _submodules = module_name.partition(".") + module_name = module_name.lower() + if not module_name: + return None + + for name in self.names: + if module_name == name: + return sections.THIRDPARTY + return None + + +class RequirementsFinder(ReqsBaseFinder): + exts = (".txt", ".in") + enabled = bool(parse_requirements) + + def _get_files_from_dir(self, path: str) -> Iterator[str]: + """Return paths to requirements files from passed dir.""" + yield from self._get_files_from_dir_cached(path) + + @classmethod + @lru_cache(maxsize=16) + def _get_files_from_dir_cached(cls, path: str) -> List[str]: + results = [] + + for fname in os.listdir(path): + if "requirements" not in fname: + continue + full_path = os.path.join(path, fname) + + # *requirements*/*.{txt,in} + if os.path.isdir(full_path): + for subfile_name in os.listdir(full_path): + for ext in cls.exts: + if subfile_name.endswith(ext): + results.append(os.path.join(full_path, subfile_name)) + continue + + # *requirements*.{txt,in} + if os.path.isfile(full_path): + for ext in cls.exts: + if fname.endswith(ext): + results.append(full_path) + break + + return results + + def _get_names(self, path: str) -> Iterator[str]: + """Load required packages from path to requirements file""" + yield from self._get_names_cached(path) + + @classmethod + @lru_cache(maxsize=16) + def _get_names_cached(cls, path: str) -> List[str]: + result = [] + + with chdir(os.path.dirname(path)): + requirements = parse_requirements(path) + for req in requirements.values(): + if req.name: + result.append(req.name) + + return result + + +class PipfileFinder(ReqsBaseFinder): + enabled = bool(Pipfile) + + def _get_names(self, path: str) -> Iterator[str]: + with chdir(path): + project = Pipfile.load(path) + for req in project.packages: + yield req.name + + def _get_files_from_dir(self, path: str) -> Iterator[str]: + if "Pipfile" in os.listdir(path): + yield path + + +class DefaultFinder(BaseFinder): + def find(self, module_name: str) -> Optional[str]: + return self.config.default_section + + +class FindersManager: + _default_finders_classes: Sequence[Type[BaseFinder]] = ( + ForcedSeparateFinder, + LocalFinder, + KnownPatternFinder, + PathFinder, + PipfileFinder, + RequirementsFinder, + DefaultFinder, + ) + + def __init__( + self, config: Config, finder_classes: Optional[Iterable[Type[BaseFinder]]] = None + ) -> None: + self.verbose: bool = config.verbose + + if finder_classes is None: + finder_classes = self._default_finders_classes + finders: List[BaseFinder] = [] + for finder_cls in finder_classes: + try: + finders.append(finder_cls(config)) + except Exception as exception: + # if one finder fails to instantiate isort can continue using the rest + if self.verbose: + print( + ( + f"{finder_cls.__name__} encountered an error ({exception}) during " + "instantiation and cannot be used" + ) + ) + self.finders: Tuple[BaseFinder, ...] = tuple(finders) + + def find(self, module_name: str) -> Optional[str]: + for finder in self.finders: + try: + section = finder.find(module_name) + if section is not None: + return section + except Exception as exception: + # isort has to be able to keep trying to identify the correct + # import section even if one approach fails + if self.verbose: + print( + f"{finder.__class__.__name__} encountered an error ({exception}) while " + f"trying to identify the {module_name} module" + ) + return None diff --git a/env/lib/python3.8/site-packages/isort/exceptions.py b/env/lib/python3.8/site-packages/isort/exceptions.py new file mode 100644 index 000000000..9f45744c7 --- /dev/null +++ b/env/lib/python3.8/site-packages/isort/exceptions.py @@ -0,0 +1,134 @@ +"""All isort specific exception classes should be defined here""" +from .profiles import profiles + + +class ISortError(Exception): + """Base isort exception object from which all isort sourced exceptions should inherit""" + + +class InvalidSettingsPath(ISortError): + """Raised when a settings path is provided that is neither a valid file or directory""" + + def __init__(self, settings_path: str): + super().__init__( + f"isort was told to use the settings_path: {settings_path} as the base directory or " + "file that represents the starting point of config file discovery, but it does not " + "exist." + ) + self.settings_path = settings_path + + +class ExistingSyntaxErrors(ISortError): + """Raised when isort is told to sort imports within code that has existing syntax errors""" + + def __init__(self, file_path: str): + super().__init__( + f"isort was told to sort imports within code that contains syntax errors: " + f"{file_path}." + ) + self.file_path = file_path + + +class IntroducedSyntaxErrors(ISortError): + """Raised when isort has introduced a syntax error in the process of sorting imports""" + + def __init__(self, file_path: str): + super().__init__( + f"isort introduced syntax errors when attempting to sort the imports contained within " + f"{file_path}." + ) + self.file_path = file_path + + +class FileSkipped(ISortError): + """Should be raised when a file is skipped for any reason""" + + def __init__(self, message: str, file_path: str): + super().__init__(message) + self.file_path = file_path + + +class FileSkipComment(FileSkipped): + """Raised when an entire file is skipped due to a isort skip file comment""" + + def __init__(self, file_path: str): + super().__init__( + f"{file_path} contains an file skip comment and was skipped.", file_path=file_path + ) + + +class FileSkipSetting(FileSkipped): + """Raised when an entire file is skipped due to provided isort settings""" + + def __init__(self, file_path: str): + super().__init__( + f"{file_path} was skipped as it's listed in 'skip' setting" + " or matches a glob in 'skip_glob' setting", + file_path=file_path, + ) + + +class ProfileDoesNotExist(ISortError): + """Raised when a profile is set by the user that doesn't exist""" + + def __init__(self, profile: str): + super().__init__( + f"Specified profile of {profile} does not exist. " + f"Available profiles: {','.join(profiles)}." + ) + self.profile = profile + + +class FormattingPluginDoesNotExist(ISortError): + """Raised when a formatting plugin is set by the user that doesn't exist""" + + def __init__(self, formatter: str): + super().__init__(f"Specified formatting plugin of {formatter} does not exist. ") + self.formatter = formatter + + +class LiteralParsingFailure(ISortError): + """Raised when one of isorts literal sorting comments is used but isort can't parse the + the given data structure. + """ + + def __init__(self, code: str, original_error: Exception): + super().__init__( + f"isort failed to parse the given literal {code}. It's important to note " + "that isort literal sorting only supports simple literals parsable by " + f"ast.literal_eval which gave the exception of {original_error}." + ) + self.code = code + self.original_error = original_error + + +class LiteralSortTypeMismatch(ISortError): + """Raised when an isort literal sorting comment is used, with a type that doesn't match the + supplied data structure's type. + """ + + def __init__(self, kind: type, expected_kind: type): + super().__init__( + f"isort was told to sort a literal of type {expected_kind} but was given " + f"a literal of type {kind}." + ) + self.kind = kind + self.expected_kind = expected_kind + + +class AssignmentsFormatMismatch(ISortError): + """Raised when isort is told to sort assignments but the format of the assignment section + doesn't match isort's expectation. + """ + + def __init__(self, code: str): + super().__init__( + "isort was told to sort a section of assignments, however the given code:\n\n" + f"{code}\n\n" + "Does not match isort's strict single line formatting requirement for assignment " + "sorting:\n\n" + "{variable_name} = {value}\n" + "{variable_name2} = {value2}\n" + "...\n\n" + ) + self.code = code diff --git a/env/lib/python3.8/site-packages/isort/format.py b/env/lib/python3.8/site-packages/isort/format.py new file mode 100644 index 000000000..46bb15695 --- /dev/null +++ b/env/lib/python3.8/site-packages/isort/format.py @@ -0,0 +1,149 @@ +import re +import sys +from datetime import datetime +from difflib import unified_diff +from pathlib import Path +from typing import Optional, TextIO + +try: + import colorama +except ImportError: + colorama_unavailable = True +else: + colorama_unavailable = False + colorama.init() + + +ADDED_LINE_PATTERN = re.compile(r"\+[^+]") +REMOVED_LINE_PATTERN = re.compile(r"-[^-]") + + +def format_simplified(import_line: str) -> str: + import_line = import_line.strip() + if import_line.startswith("from "): + import_line = import_line.replace("from ", "") + import_line = import_line.replace(" import ", ".") + elif import_line.startswith("import "): + import_line = import_line.replace("import ", "") + + return import_line + + +def format_natural(import_line: str) -> str: + import_line = import_line.strip() + if not import_line.startswith("from ") and not import_line.startswith("import "): + if "." not in import_line: + return f"import {import_line}" + parts = import_line.split(".") + end = parts.pop(-1) + return f"from {'.'.join(parts)} import {end}" + + return import_line + + +def show_unified_diff( + *, + file_input: str, + file_output: str, + file_path: Optional[Path], + output: Optional[TextIO] = None, + color_output: bool = False, +): + """Shows a unified_diff for the provided input and output against the provided file path. + + - **file_input**: A string that represents the contents of a file before changes. + - **file_output**: A string that represents the contents of a file after changes. + - **file_path**: A Path object that represents the file path of the file being changed. + - **output**: A stream to output the diff to. If non is provided uses sys.stdout. + - **color_output**: Use color in output if True. + """ + printer = create_terminal_printer(color_output, output) + file_name = "" if file_path is None else str(file_path) + file_mtime = str( + datetime.now() if file_path is None else datetime.fromtimestamp(file_path.stat().st_mtime) + ) + unified_diff_lines = unified_diff( + file_input.splitlines(keepends=True), + file_output.splitlines(keepends=True), + fromfile=file_name + ":before", + tofile=file_name + ":after", + fromfiledate=file_mtime, + tofiledate=str(datetime.now()), + ) + for line in unified_diff_lines: + printer.diff_line(line) + + +def ask_whether_to_apply_changes_to_file(file_path: str) -> bool: + answer = None + while answer not in ("yes", "y", "no", "n", "quit", "q"): + answer = input(f"Apply suggested changes to '{file_path}' [y/n/q]? ") # nosec + answer = answer.lower() + if answer in ("no", "n"): + return False + if answer in ("quit", "q"): + sys.exit(1) + return True + + +def remove_whitespace(content: str, line_separator: str = "\n") -> str: + content = content.replace(line_separator, "").replace(" ", "").replace("\x0c", "") + return content + + +class BasicPrinter: + ERROR = "ERROR" + SUCCESS = "SUCCESS" + + def __init__(self, output: Optional[TextIO] = None): + self.output = output or sys.stdout + + def success(self, message: str) -> None: + print(f"{self.SUCCESS}: {message}", file=self.output) + + def error(self, message: str) -> None: + print(f"{self.ERROR}: {message}", file=sys.stderr) + + def diff_line(self, line: str) -> None: + self.output.write(line) + + +class ColoramaPrinter(BasicPrinter): + def __init__(self, output: Optional[TextIO] = None): + self.output = output or sys.stdout + # Note: this constants are instance variables instead ofs class variables + # because they refer to colorama which might not be installed. + self.ERROR = self.style_text("ERROR", colorama.Fore.RED) + self.SUCCESS = self.style_text("SUCCESS", colorama.Fore.GREEN) + self.ADDED_LINE = colorama.Fore.GREEN + self.REMOVED_LINE = colorama.Fore.RED + + @staticmethod + def style_text(text: str, style: Optional[str] = None) -> str: + if style is None: + return text + return style + text + colorama.Style.RESET_ALL + + def diff_line(self, line: str) -> None: + style = None + if re.match(ADDED_LINE_PATTERN, line): + style = self.ADDED_LINE + elif re.match(REMOVED_LINE_PATTERN, line): + style = self.REMOVED_LINE + self.output.write(self.style_text(line, style)) + + +def create_terminal_printer(color: bool, output: Optional[TextIO] = None): + if color and colorama_unavailable: + no_colorama_message = ( + "\n" + "Sorry, but to use --color (color_output) the colorama python package is required.\n\n" + "Reference: https://pypi.org/project/colorama/\n\n" + "You can either install it separately on your system or as the colors extra " + "for isort. Ex: \n\n" + "$ pip install isort[colors]\n" + ) + print(no_colorama_message, file=sys.stderr) + sys.exit(1) + + return ColoramaPrinter(output) if color else BasicPrinter(output) diff --git a/env/lib/python3.8/site-packages/isort/hooks.py b/env/lib/python3.8/site-packages/isort/hooks.py new file mode 100644 index 000000000..acccede59 --- /dev/null +++ b/env/lib/python3.8/site-packages/isort/hooks.py @@ -0,0 +1,90 @@ +"""Defines a git hook to allow pre-commit warnings and errors about import order. + +usage: + exit_code = git_hook(strict=True|False, modify=True|False) +""" +import os +import subprocess # nosec - Needed for hook +from pathlib import Path +from typing import List + +from isort import Config, api, exceptions + + +def get_output(command: List[str]) -> str: + """ + Run a command and return raw output + + :param str command: the command to run + :returns: the stdout output of the command + """ + result = subprocess.run(command, stdout=subprocess.PIPE, check=True) # nosec - trusted input + return result.stdout.decode() + + +def get_lines(command: List[str]) -> List[str]: + """ + Run a command and return lines of output + + :param str command: the command to run + :returns: list of whitespace-stripped lines output by command + """ + stdout = get_output(command) + return [line.strip() for line in stdout.splitlines()] + + +def git_hook( + strict: bool = False, modify: bool = False, lazy: bool = False, settings_file: str = "" +) -> int: + """ + Git pre-commit hook to check staged files for isort errors + + :param bool strict - if True, return number of errors on exit, + causing the hook to fail. If False, return zero so it will + just act as a warning. + :param bool modify - if True, fix the sources if they are not + sorted properly. If False, only report result without + modifying anything. + :param bool lazy - if True, also check/fix unstaged files. + This is useful if you frequently use ``git commit -a`` for example. + If False, ony check/fix the staged files for isort errors. + :param str settings_file - A path to a file to be used as + the configuration file for this run. + When settings_file is the empty string, the configuration file + will be searched starting at the directory containing the first + staged file, if any, and going upward in the directory structure. + + :return number of errors if in strict mode, 0 otherwise. + """ + + # Get list of files modified and staged + diff_cmd = ["git", "diff-index", "--cached", "--name-only", "--diff-filter=ACMRTUXB", "HEAD"] + if lazy: + diff_cmd.remove("--cached") + + files_modified = get_lines(diff_cmd) + if not files_modified: + return 0 + + errors = 0 + config = Config( + settings_file=settings_file, + settings_path=os.path.dirname(os.path.abspath(files_modified[0])), + ) + for filename in files_modified: + if filename.endswith(".py"): + # Get the staged contents of the file + staged_cmd = ["git", "show", f":{filename}"] + staged_contents = get_output(staged_cmd) + + try: + if not api.check_code_string( + staged_contents, file_path=Path(filename), config=config + ): + errors += 1 + if modify: + api.sort_file(filename, config=config) + except exceptions.FileSkipped: # pragma: no cover + pass + + return errors if strict else 0 diff --git a/env/lib/python3.8/site-packages/isort/io.py b/env/lib/python3.8/site-packages/isort/io.py new file mode 100644 index 000000000..a0357347b --- /dev/null +++ b/env/lib/python3.8/site-packages/isort/io.py @@ -0,0 +1,60 @@ +"""Defines any IO utilities used by isort""" +import re +import tokenize +from contextlib import contextmanager +from io import BytesIO, StringIO, TextIOWrapper +from pathlib import Path +from typing import Iterator, NamedTuple, TextIO, Union + +_ENCODING_PATTERN = re.compile(br"^[ \t\f]*#.*?coding[:=][ \t]*([-_.a-zA-Z0-9]+)") + + +class File(NamedTuple): + stream: TextIO + path: Path + encoding: str + + @staticmethod + def from_contents(contents: str, filename: str) -> "File": + encoding, _ = tokenize.detect_encoding(BytesIO(contents.encode("utf-8")).readline) + return File(StringIO(contents), path=Path(filename).resolve(), encoding=encoding) + + @property + def extension(self): + return self.path.suffix.lstrip(".") + + @staticmethod + def _open(filename): + """Open a file in read only mode using the encoding detected by + detect_encoding(). + """ + buffer = open(filename, "rb") + try: + encoding, _ = tokenize.detect_encoding(buffer.readline) + buffer.seek(0) + text = TextIOWrapper(buffer, encoding, line_buffering=True, newline="") + text.mode = "r" # type: ignore + return text + except Exception: + buffer.close() + raise + + @staticmethod + @contextmanager + def read(filename: Union[str, Path]) -> Iterator["File"]: + file_path = Path(filename).resolve() + stream = None + try: + stream = File._open(file_path) + yield File(stream=stream, path=file_path, encoding=stream.encoding) + finally: + if stream is not None: + stream.close() + + +class _EmptyIO(StringIO): + def write(self, *args, **kwargs): + pass + + +Empty = _EmptyIO() diff --git a/env/lib/python3.8/site-packages/isort/literal.py b/env/lib/python3.8/site-packages/isort/literal.py new file mode 100644 index 000000000..28e0855c3 --- /dev/null +++ b/env/lib/python3.8/site-packages/isort/literal.py @@ -0,0 +1,108 @@ +import ast +from pprint import PrettyPrinter +from typing import Any, Callable, Dict, List, Set, Tuple + +from isort.exceptions import ( + AssignmentsFormatMismatch, + LiteralParsingFailure, + LiteralSortTypeMismatch, +) +from isort.settings import DEFAULT_CONFIG, Config + + +class ISortPrettyPrinter(PrettyPrinter): + """an isort customized pretty printer for sorted literals""" + + def __init__(self, config: Config): + super().__init__(width=config.line_length, compact=True) + + +type_mapping: Dict[str, Tuple[type, Callable[[Any, ISortPrettyPrinter], str]]] = {} + + +def assignments(code: str) -> str: + sort_assignments = {} + for line in code.splitlines(keepends=True): + if line: + if " = " not in line: + raise AssignmentsFormatMismatch(code) + else: + variable_name, value = line.split(" = ", 1) + sort_assignments[variable_name] = value + + sorted_assignments = dict(sorted(sort_assignments.items(), key=lambda item: item[1])) + return "".join(f"{key} = {value}" for key, value in sorted_assignments.items()) + + +def assignment(code: str, sort_type: str, extension: str, config: Config = DEFAULT_CONFIG) -> str: + """Sorts the literal present within the provided code against the provided sort type, + returning the sorted representation of the source code. + """ + if sort_type == "assignments": + return assignments(code) + elif sort_type not in type_mapping: + raise ValueError( + "Trying to sort using an undefined sort_type. " + f"Defined sort types are {', '.join(type_mapping.keys())}." + ) + + variable_name, literal = code.split(" = ") + variable_name = variable_name.lstrip() + try: + value = ast.literal_eval(literal) + except Exception as error: + raise LiteralParsingFailure(code, error) + + expected_type, sort_function = type_mapping[sort_type] + if type(value) != expected_type: + raise LiteralSortTypeMismatch(type(value), expected_type) + + printer = ISortPrettyPrinter(config) + sorted_value_code = f"{variable_name} = {sort_function(value, printer)}" + if config.formatting_function: + sorted_value_code = config.formatting_function( + sorted_value_code, extension, config + ).rstrip() + + sorted_value_code += code[len(code.rstrip()) :] + return sorted_value_code + + +def register_type(name: str, kind: type): + """Registers a new literal sort type.""" + + def wrap(function): + type_mapping[name] = (kind, function) + return function + + return wrap + + +@register_type("dict", dict) +def _dict(value: Dict[Any, Any], printer: ISortPrettyPrinter) -> str: + return printer.pformat(dict(sorted(value.items(), key=lambda item: item[1]))) + + +@register_type("list", list) +def _list(value: List[Any], printer: ISortPrettyPrinter) -> str: + return printer.pformat(sorted(value)) + + +@register_type("unique-list", list) +def _unique_list(value: List[Any], printer: ISortPrettyPrinter) -> str: + return printer.pformat(list(sorted(set(value)))) + + +@register_type("set", set) +def _set(value: Set[Any], printer: ISortPrettyPrinter) -> str: + return "{" + printer.pformat(tuple(sorted(value)))[1:-1] + "}" + + +@register_type("tuple", tuple) +def _tuple(value: Tuple[Any, ...], printer: ISortPrettyPrinter) -> str: + return printer.pformat(tuple(sorted(value))) + + +@register_type("unique-tuple", tuple) +def _unique_tuple(value: Tuple[Any, ...], printer: ISortPrettyPrinter) -> str: + return printer.pformat(tuple(sorted(set(value)))) diff --git a/env/lib/python3.8/site-packages/isort/logo.py b/env/lib/python3.8/site-packages/isort/logo.py new file mode 100644 index 000000000..6377d8686 --- /dev/null +++ b/env/lib/python3.8/site-packages/isort/logo.py @@ -0,0 +1,19 @@ +from ._version import __version__ + +ASCII_ART = rf""" + _ _ + (_) ___ ___ _ __| |_ + | |/ _/ / _ \/ '__ _/ + | |\__ \/\_\/| | | |_ + |_|\___/\___/\_/ \_/ + + isort your imports, so you don't have to. + + VERSION {__version__} +""" + +__doc__ = f""" +```python +{ASCII_ART} +``` +""" diff --git a/env/lib/python3.8/site-packages/isort/main.py b/env/lib/python3.8/site-packages/isort/main.py new file mode 100644 index 000000000..5dc2ef188 --- /dev/null +++ b/env/lib/python3.8/site-packages/isort/main.py @@ -0,0 +1,928 @@ +"""Tool for sorting imports alphabetically, and automatically separated into sections.""" +import argparse +import functools +import json +import os +import sys +from io import TextIOWrapper +from pathlib import Path +from typing import Any, Dict, Iterable, Iterator, List, Optional, Sequence, Set +from warnings import warn + +from . import __version__, api, sections +from .exceptions import FileSkipped +from .format import create_terminal_printer +from .logo import ASCII_ART +from .profiles import profiles +from .settings import VALID_PY_TARGETS, Config, WrapModes + +try: + from .setuptools_commands import ISortCommand # noqa: F401 +except ImportError: + pass + +DEPRECATED_SINGLE_DASH_ARGS = { + "-ac", + "-af", + "-ca", + "-cs", + "-df", + "-ds", + "-dt", + "-fas", + "-fass", + "-ff", + "-fgw", + "-fss", + "-lai", + "-lbt", + "-le", + "-ls", + "-nis", + "-nlb", + "-ot", + "-rr", + "-sd", + "-sg", + "-sl", + "-sp", + "-tc", + "-wl", + "-ws", +} +QUICK_GUIDE = f""" +{ASCII_ART} + +Nothing to do: no files or paths have have been passed in! + +Try one of the following: + + `isort .` - sort all Python files, starting from the current directory, recursively. + `isort . --interactive` - Do the same, but ask before making any changes. + `isort . --check --diff` - Check to see if imports are correctly sorted within this project. + `isort --help` - In-depth information about isort's available command-line options. + +Visit https://pycqa.github.io/isort/ for complete information about how to use isort. +""" + + +class SortAttempt: + def __init__(self, incorrectly_sorted: bool, skipped: bool) -> None: + self.incorrectly_sorted = incorrectly_sorted + self.skipped = skipped + + +def sort_imports( + file_name: str, + config: Config, + check: bool = False, + ask_to_apply: bool = False, + write_to_stdout: bool = False, + **kwargs: Any, +) -> Optional[SortAttempt]: + try: + incorrectly_sorted: bool = False + skipped: bool = False + if check: + try: + incorrectly_sorted = not api.check_file(file_name, config=config, **kwargs) + except FileSkipped: + skipped = True + return SortAttempt(incorrectly_sorted, skipped) + else: + try: + incorrectly_sorted = not api.sort_file( + file_name, + config=config, + ask_to_apply=ask_to_apply, + write_to_stdout=write_to_stdout, + **kwargs, + ) + except FileSkipped: + skipped = True + return SortAttempt(incorrectly_sorted, skipped) + except (OSError, ValueError) as error: + warn(f"Unable to parse file {file_name} due to {error}") + return None + except Exception: + printer = create_terminal_printer(color=config.color_output) + printer.error( + f"Unrecoverable exception thrown when parsing {file_name}! " + "This should NEVER happen.\n" + "If encountered, please open an issue: https://github.com/PyCQA/isort/issues/new" + ) + raise + + +def iter_source_code(paths: Iterable[str], config: Config, skipped: List[str]) -> Iterator[str]: + """Iterate over all Python source files defined in paths.""" + visited_dirs: Set[Path] = set() + + for path in paths: + if os.path.isdir(path): + for dirpath, dirnames, filenames in os.walk(path, topdown=True, followlinks=True): + base_path = Path(dirpath) + for dirname in list(dirnames): + full_path = base_path / dirname + resolved_path = full_path.resolve() + if config.is_skipped(full_path): + skipped.append(dirname) + dirnames.remove(dirname) + else: + if resolved_path in visited_dirs: # pragma: no cover + if not config.quiet: + warn(f"Likely recursive symlink detected to {resolved_path}") + dirnames.remove(dirname) + visited_dirs.add(resolved_path) + + for filename in filenames: + filepath = os.path.join(dirpath, filename) + if config.is_supported_filetype(filepath): + if config.is_skipped(Path(filepath)): + skipped.append(filename) + else: + yield filepath + else: + yield path + + +def _build_arg_parser() -> argparse.ArgumentParser: + parser = argparse.ArgumentParser( + description="Sort Python import definitions alphabetically " + "within logical sections. Run with no arguments to see a quick " + "start guide, otherwise, one or more files/directories/stdin must be provided. " + "Use `-` as the first argument to represent stdin. Use --interactive to use the pre 5.0.0 " + "interactive behavior." + " " + "If you've used isort 4 but are new to isort 5, see the upgrading guide:" + "https://pycqa.github.io/isort/docs/upgrade_guides/5.0.0/." + ) + inline_args_group = parser.add_mutually_exclusive_group() + parser.add_argument( + "--src", + "--src-path", + dest="src_paths", + action="append", + help="Add an explicitly defined source path " + "(modules within src paths have their imports automatically categorized as first_party).", + ) + parser.add_argument( + "-a", + "--add-import", + dest="add_imports", + action="append", + help="Adds the specified import line to all files, " + "automatically determining correct placement.", + ) + parser.add_argument( + "--append", + "--append-only", + dest="append_only", + action="store_true", + help="Only adds the imports specified in --add-imports if the file" + " contains existing imports.", + ) + parser.add_argument( + "--ac", + "--atomic", + dest="atomic", + action="store_true", + help="Ensures the output doesn't save if the resulting file contains syntax errors.", + ) + parser.add_argument( + "--af", + "--force-adds", + dest="force_adds", + action="store_true", + help="Forces import adds even if the original file is empty.", + ) + parser.add_argument( + "-b", + "--builtin", + dest="known_standard_library", + action="append", + help="Force isort to recognize a module as part of Python's standard library.", + ) + parser.add_argument( + "--extra-builtin", + dest="extra_standard_library", + action="append", + help="Extra modules to be included in the list of ones in Python's standard library.", + ) + parser.add_argument( + "-c", + "--check-only", + "--check", + action="store_true", + dest="check", + help="Checks the file for unsorted / unformatted imports and prints them to the " + "command line without modifying the file.", + ) + parser.add_argument( + "--ca", + "--combine-as", + dest="combine_as_imports", + action="store_true", + help="Combines as imports on the same line.", + ) + parser.add_argument( + "--cs", + "--combine-star", + dest="combine_star", + action="store_true", + help="Ensures that if a star import is present, " + "nothing else is imported from that namespace.", + ) + parser.add_argument( + "-d", + "--stdout", + help="Force resulting output to stdout, instead of in-place.", + dest="write_to_stdout", + action="store_true", + ) + parser.add_argument( + "--df", + "--diff", + dest="show_diff", + action="store_true", + help="Prints a diff of all the changes isort would make to a file, instead of " + "changing it in place", + ) + parser.add_argument( + "--ds", + "--no-sections", + help="Put all imports into the same section bucket", + dest="no_sections", + action="store_true", + ) + parser.add_argument( + "-e", + "--balanced", + dest="balanced_wrapping", + action="store_true", + help="Balances wrapping to produce the most consistent line length possible", + ) + parser.add_argument( + "-f", + "--future", + dest="known_future_library", + action="append", + help="Force isort to recognize a module as part of the future compatibility libraries.", + ) + parser.add_argument( + "--fas", + "--force-alphabetical-sort", + action="store_true", + dest="force_alphabetical_sort", + help="Force all imports to be sorted as a single section", + ) + parser.add_argument( + "--fass", + "--force-alphabetical-sort-within-sections", + action="store_true", + dest="force_alphabetical_sort_within_sections", + help="Force all imports to be sorted alphabetically within a section", + ) + parser.add_argument( + "--ff", + "--from-first", + dest="from_first", + help="Switches the typical ordering preference, " + "showing from imports first then straight ones.", + ) + parser.add_argument( + "--fgw", + "--force-grid-wrap", + nargs="?", + const=2, + type=int, + dest="force_grid_wrap", + help="Force number of from imports (defaults to 2) to be grid wrapped regardless of line " + "length", + ) + parser.add_argument( + "--fss", + "--force-sort-within-sections", + action="store_true", + dest="force_sort_within_sections", + help="Don't sort straight-style imports (like import sys) before from-style imports " + "(like from itertools import groupby). Instead, sort the imports by module, " + "independent of import style.", + ) + parser.add_argument( + "-i", + "--indent", + help='String to place for indents defaults to " " (4 spaces).', + dest="indent", + type=str, + ) + parser.add_argument( + "-j", "--jobs", help="Number of files to process in parallel.", dest="jobs", type=int + ) + parser.add_argument("--lai", "--lines-after-imports", dest="lines_after_imports", type=int) + parser.add_argument("--lbt", "--lines-between-types", dest="lines_between_types", type=int) + parser.add_argument( + "--le", + "--line-ending", + dest="line_ending", + help="Forces line endings to the specified value. " + "If not set, values will be guessed per-file.", + ) + parser.add_argument( + "--ls", + "--length-sort", + help="Sort imports by their string length.", + dest="length_sort", + action="store_true", + ) + parser.add_argument( + "--lss", + "--length-sort-straight", + help="Sort straight imports by their string length.", + dest="length_sort_straight", + action="store_true", + ) + parser.add_argument( + "-m", + "--multi-line", + dest="multi_line_output", + choices=list(WrapModes.__members__.keys()) + + [str(mode.value) for mode in WrapModes.__members__.values()], + type=str, + help="Multi line output (0-grid, 1-vertical, 2-hanging, 3-vert-hanging, 4-vert-grid, " + "5-vert-grid-grouped, 6-vert-grid-grouped-no-comma, 7-noqa, " + "8-vertical-hanging-indent-bracket, 9-vertical-prefix-from-module-import, " + "10-hanging-indent-with-parentheses).", + ) + parser.add_argument( + "-n", + "--ensure-newline-before-comments", + dest="ensure_newline_before_comments", + action="store_true", + help="Inserts a blank line before a comment following an import.", + ) + inline_args_group.add_argument( + "--nis", + "--no-inline-sort", + dest="no_inline_sort", + action="store_true", + help="Leaves `from` imports with multiple imports 'as-is' " + "(e.g. `from foo import a, c ,b`).", + ) + parser.add_argument( + "--nlb", + "--no-lines-before", + help="Sections which should not be split with previous by empty lines", + dest="no_lines_before", + action="append", + ) + parser.add_argument( + "-o", + "--thirdparty", + dest="known_third_party", + action="append", + help="Force isort to recognize a module as being part of a third party library.", + ) + parser.add_argument( + "--ot", + "--order-by-type", + dest="order_by_type", + action="store_true", + help="Order imports by type, which is determined by case, in addition to alphabetically.\n" + "\n**NOTE**: type here refers to the implied type from the import name capitalization.\n" + ' isort does not do type introspection for the imports. These "types" are simply: ' + "CONSTANT_VARIABLE, CamelCaseClass, variable_or_function. If your project follows PEP8" + " or a related coding standard and has many imports this is a good default, otherwise you " + "likely will want to turn it off. From the CLI the `--dont-order-by-type` option will turn " + "this off.", + ) + parser.add_argument( + "--dt", + "--dont-order-by-type", + dest="dont_order_by_type", + action="store_true", + help="Don't order imports by type, which is determined by case, in addition to " + "alphabetically.\n\n" + "**NOTE**: type here refers to the implied type from the import name capitalization.\n" + ' isort does not do type introspection for the imports. These "types" are simply: ' + "CONSTANT_VARIABLE, CamelCaseClass, variable_or_function. If your project follows PEP8" + " or a related coding standard and has many imports this is a good default. You can turn " + "this on from the CLI using `--order-by-type`.", + ) + parser.add_argument( + "-p", + "--project", + dest="known_first_party", + action="append", + help="Force isort to recognize a module as being part of the current python project.", + ) + parser.add_argument( + "--known-local-folder", + dest="known_local_folder", + action="append", + help="Force isort to recognize a module as being a local folder. " + "Generally, this is reserved for relative imports (from . import module).", + ) + parser.add_argument( + "-q", + "--quiet", + action="store_true", + dest="quiet", + help="Shows extra quiet output, only errors are outputted.", + ) + parser.add_argument( + "--rm", + "--remove-import", + dest="remove_imports", + action="append", + help="Removes the specified import from all files.", + ) + parser.add_argument( + "--rr", + "--reverse-relative", + dest="reverse_relative", + action="store_true", + help="Reverse order of relative imports.", + ) + parser.add_argument( + "-s", + "--skip", + help="Files that sort imports should skip over. If you want to skip multiple " + "files you should specify twice: --skip file1 --skip file2.", + dest="skip", + action="append", + ) + parser.add_argument( + "--sd", + "--section-default", + dest="default_section", + help="Sets the default section for import options: " + str(sections.DEFAULT), + ) + parser.add_argument( + "--sg", + "--skip-glob", + help="Files that sort imports should skip over.", + dest="skip_glob", + action="append", + ) + parser.add_argument( + "--gitignore", + "--skip-gitignore", + action="store_true", + dest="skip_gitignore", + help="Treat project as a git repository and ignore files listed in .gitignore", + ) + inline_args_group.add_argument( + "--sl", + "--force-single-line-imports", + dest="force_single_line", + action="store_true", + help="Forces all from imports to appear on their own line", + ) + parser.add_argument( + "--nsl", + "--single-line-exclusions", + help="One or more modules to exclude from the single line rule.", + dest="single_line_exclusions", + action="append", + ) + parser.add_argument( + "--sp", + "--settings-path", + "--settings-file", + "--settings", + dest="settings_path", + help="Explicitly set the settings path or file instead of auto determining " + "based on file location.", + ) + parser.add_argument( + "-t", + "--top", + help="Force specific imports to the top of their appropriate section.", + dest="force_to_top", + action="append", + ) + parser.add_argument( + "--tc", + "--trailing-comma", + dest="include_trailing_comma", + action="store_true", + help="Includes a trailing comma on multi line imports that include parentheses.", + ) + parser.add_argument( + "--up", + "--use-parentheses", + dest="use_parentheses", + action="store_true", + help="Use parentheses for line continuation on length limit instead of slashes." + " **NOTE**: This is separate from wrap modes, and only affects how individual lines that " + " are too long get continued, not sections of multiple imports.", + ) + parser.add_argument( + "-V", + "--version", + action="store_true", + dest="show_version", + help="Displays the currently installed version of isort.", + ) + parser.add_argument( + "-v", + "--verbose", + action="store_true", + dest="verbose", + help="Shows verbose output, such as when files are skipped or when a check is successful.", + ) + parser.add_argument( + "--virtual-env", + dest="virtual_env", + help="Virtual environment to use for determining whether a package is third-party", + ) + parser.add_argument( + "--conda-env", + dest="conda_env", + help="Conda environment to use for determining whether a package is third-party", + ) + parser.add_argument( + "--vn", + "--version-number", + action="version", + version=__version__, + help="Returns just the current version number without the logo", + ) + parser.add_argument( + "-l", + "-w", + "--line-length", + "--line-width", + help="The max length of an import line (used for wrapping long imports).", + dest="line_length", + type=int, + ) + parser.add_argument( + "--wl", + "--wrap-length", + dest="wrap_length", + type=int, + help="Specifies how long lines that are wrapped should be, if not set line_length is used." + "\nNOTE: wrap_length must be LOWER than or equal to line_length.", + ) + parser.add_argument( + "--ws", + "--ignore-whitespace", + action="store_true", + dest="ignore_whitespace", + help="Tells isort to ignore whitespace differences when --check-only is being used.", + ) + parser.add_argument( + "--case-sensitive", + dest="case_sensitive", + action="store_true", + help="Tells isort to include casing when sorting module names", + ) + parser.add_argument( + "--filter-files", + dest="filter_files", + action="store_true", + help="Tells isort to filter files even when they are explicitly passed in as " + "part of the CLI command.", + ) + parser.add_argument( + "files", nargs="*", help="One or more Python source files that need their imports sorted." + ) + parser.add_argument( + "--py", + "--python-version", + action="store", + dest="py_version", + choices=tuple(VALID_PY_TARGETS) + ("auto",), + help="Tells isort to set the known standard library based on the specified Python " + "version. Default is to assume any Python 3 version could be the target, and use a union " + "of all stdlib modules across versions. If auto is specified, the version of the " + "interpreter used to run isort " + f"(currently: {sys.version_info.major}{sys.version_info.minor}) will be used.", + ) + parser.add_argument( + "--profile", + dest="profile", + type=str, + help="Base profile type to use for configuration. " + f"Profiles include: {', '.join(profiles.keys())}. As well as any shared profiles.", + ) + parser.add_argument( + "--interactive", + dest="ask_to_apply", + action="store_true", + help="Tells isort to apply changes interactively.", + ) + parser.add_argument( + "--old-finders", + "--magic-placement", + dest="old_finders", + action="store_true", + help="Use the old deprecated finder logic that relies on environment introspection magic.", + ) + parser.add_argument( + "--show-config", + dest="show_config", + action="store_true", + help="See isort's determined config, as well as sources of config options.", + ) + parser.add_argument( + "--honor-noqa", + dest="honor_noqa", + action="store_true", + help="Tells isort to honor noqa comments to enforce skipping those comments.", + ) + parser.add_argument( + "--remove-redundant-aliases", + dest="remove_redundant_aliases", + action="store_true", + help=( + "Tells isort to remove redundant aliases from imports, such as `import os as os`." + " This defaults to `False` simply because some projects use these seemingly useless " + " aliases to signify intent and change behaviour." + ), + ) + parser.add_argument( + "--color", + dest="color_output", + action="store_true", + help="Tells isort to use color in terminal output.", + ) + parser.add_argument( + "--float-to-top", + dest="float_to_top", + action="store_true", + help="Causes all non-indented imports to float to the top of the file having its imports " + "sorted. It can be an excellent shortcut for collecting imports every once in a while " + "when you place them in the middle of a file to avoid context switching.\n\n" + "*NOTE*: It currently doesn't work with cimports and introduces some extra over-head " + "and a performance penalty.", + ) + parser.add_argument( + "--treat-comment-as-code", + dest="treat_comments_as_code", + action="append", + help="Tells isort to treat the specified single line comment(s) as if they are code.", + ) + parser.add_argument( + "--treat-all-comment-as-code", + dest="treat_all_comments_as_code", + action="store_true", + help="Tells isort to treat all single line comments as if they are code.", + ) + parser.add_argument( + "--formatter", + dest="formatter", + type=str, + help="Specifies the name of a formatting plugin to use when producing output.", + ) + parser.add_argument( + "--ext", + "--extension", + "--supported-extension", + dest="supported_extensions", + action="append", + help="Specifies what extensions isort can be ran against.", + ) + parser.add_argument( + "--blocked-extension", + dest="blocked_extensions", + action="append", + help="Specifies what extensions isort can never be ran against.", + ) + parser.add_argument( + "--dedup-headings", + dest="dedup_headings", + action="store_true", + help="Tells isort to only show an identical custom import heading comment once, even if" + " there are multiple sections with the comment set.", + ) + + # deprecated options + parser.add_argument( + "--recursive", + dest="deprecated_flags", + action="append_const", + const="--recursive", + help=argparse.SUPPRESS, + ) + parser.add_argument( + "-rc", dest="deprecated_flags", action="append_const", const="-rc", help=argparse.SUPPRESS + ) + parser.add_argument( + "--dont-skip", + dest="deprecated_flags", + action="append_const", + const="--dont-skip", + help=argparse.SUPPRESS, + ) + parser.add_argument( + "-ns", dest="deprecated_flags", action="append_const", const="-ns", help=argparse.SUPPRESS + ) + parser.add_argument( + "--apply", + dest="deprecated_flags", + action="append_const", + const="--apply", + help=argparse.SUPPRESS, + ) + parser.add_argument( + "-k", + "--keep-direct-and-as", + dest="deprecated_flags", + action="append_const", + const="--keep-direct-and-as", + help=argparse.SUPPRESS, + ) + + return parser + + +def parse_args(argv: Optional[Sequence[str]] = None) -> Dict[str, Any]: + argv = sys.argv[1:] if argv is None else list(argv) + remapped_deprecated_args = [] + for index, arg in enumerate(argv): + if arg in DEPRECATED_SINGLE_DASH_ARGS: + remapped_deprecated_args.append(arg) + argv[index] = f"-{arg}" + + parser = _build_arg_parser() + arguments = {key: value for key, value in vars(parser.parse_args(argv)).items() if value} + if remapped_deprecated_args: + arguments["remapped_deprecated_args"] = remapped_deprecated_args + if "dont_order_by_type" in arguments: + arguments["order_by_type"] = False + del arguments["dont_order_by_type"] + multi_line_output = arguments.get("multi_line_output", None) + if multi_line_output: + if multi_line_output.isdigit(): + arguments["multi_line_output"] = WrapModes(int(multi_line_output)) + else: + arguments["multi_line_output"] = WrapModes[multi_line_output] + return arguments + + +def _preconvert(item): + """Preconverts objects from native types into JSONifyiable types""" + if isinstance(item, (set, frozenset)): + return list(item) + elif isinstance(item, WrapModes): + return item.name + elif isinstance(item, Path): + return str(item) + elif callable(item) and hasattr(item, "__name__"): + return item.__name__ + else: + raise TypeError("Unserializable object {} of type {}".format(item, type(item))) + + +def main(argv: Optional[Sequence[str]] = None, stdin: Optional[TextIOWrapper] = None) -> None: + arguments = parse_args(argv) + if arguments.get("show_version"): + print(ASCII_ART) + return + + show_config: bool = arguments.pop("show_config", False) + + if "settings_path" in arguments: + if os.path.isfile(arguments["settings_path"]): + arguments["settings_file"] = os.path.abspath(arguments["settings_path"]) + arguments["settings_path"] = os.path.dirname(arguments["settings_file"]) + else: + arguments["settings_path"] = os.path.abspath(arguments["settings_path"]) + + if "virtual_env" in arguments: + venv = arguments["virtual_env"] + arguments["virtual_env"] = os.path.abspath(venv) + if not os.path.isdir(arguments["virtual_env"]): + warn(f"virtual_env dir does not exist: {arguments['virtual_env']}") + + file_names = arguments.pop("files", []) + if not file_names and not show_config: + print(QUICK_GUIDE) + if arguments: + sys.exit("Error: arguments passed in without any paths or content.") + else: + return + if "settings_path" not in arguments: + arguments["settings_path"] = ( + os.path.abspath(file_names[0] if file_names else ".") or os.getcwd() + ) + if not os.path.isdir(arguments["settings_path"]): + arguments["settings_path"] = os.path.dirname(arguments["settings_path"]) + + config_dict = arguments.copy() + ask_to_apply = config_dict.pop("ask_to_apply", False) + jobs = config_dict.pop("jobs", ()) + check = config_dict.pop("check", False) + show_diff = config_dict.pop("show_diff", False) + write_to_stdout = config_dict.pop("write_to_stdout", False) + deprecated_flags = config_dict.pop("deprecated_flags", False) + remapped_deprecated_args = config_dict.pop("remapped_deprecated_args", False) + wrong_sorted_files = False + + if "src_paths" in config_dict: + config_dict["src_paths"] = { + Path(src_path).resolve() for src_path in config_dict.get("src_paths", ()) + } + + config = Config(**config_dict) + if show_config: + print(json.dumps(config.__dict__, indent=4, separators=(",", ": "), default=_preconvert)) + return + elif file_names == ["-"]: + api.sort_stream( + input_stream=sys.stdin if stdin is None else stdin, + output_stream=sys.stdout, + config=config, + show_diff=show_diff, + ) + else: + skipped: List[str] = [] + + if config.filter_files: + filtered_files = [] + for file_name in file_names: + if config.is_skipped(Path(file_name)): + skipped.append(file_name) + else: + filtered_files.append(file_name) + file_names = filtered_files + + file_names = iter_source_code(file_names, config, skipped) + num_skipped = 0 + if config.verbose: + print(ASCII_ART) + + if jobs: + import multiprocessing + + executor = multiprocessing.Pool(jobs) + attempt_iterator = executor.imap( + functools.partial( + sort_imports, + config=config, + check=check, + ask_to_apply=ask_to_apply, + write_to_stdout=write_to_stdout, + ), + file_names, + ) + else: + # https://github.com/python/typeshed/pull/2814 + attempt_iterator = ( + sort_imports( # type: ignore + file_name, + config=config, + check=check, + ask_to_apply=ask_to_apply, + show_diff=show_diff, + write_to_stdout=write_to_stdout, + ) + for file_name in file_names + ) + + for sort_attempt in attempt_iterator: + if not sort_attempt: + continue # pragma: no cover - shouldn't happen, satisfies type constraint + incorrectly_sorted = sort_attempt.incorrectly_sorted + if arguments.get("check", False) and incorrectly_sorted: + wrong_sorted_files = True + if sort_attempt.skipped: + num_skipped += ( + 1 # pragma: no cover - shouldn't happen, due to skip in iter_source_code + ) + + num_skipped += len(skipped) + if num_skipped and not arguments.get("quiet", False): + if config.verbose: + for was_skipped in skipped: + warn( + f"{was_skipped} was skipped as it's listed in 'skip' setting" + " or matches a glob in 'skip_glob' setting" + ) + print(f"Skipped {num_skipped} files") + + if not config.quiet and (remapped_deprecated_args or deprecated_flags): + if remapped_deprecated_args: + warn( + "W0502: The following deprecated single dash CLI flags were used and translated: " + f"{', '.join(remapped_deprecated_args)}!" + ) + if deprecated_flags: + warn( + "W0501: The following deprecated CLI flags were used and ignored: " + f"{', '.join(deprecated_flags)}!" + ) + warn( + "W0500: Please see the 5.0.0 Upgrade guide: " + "https://pycqa.github.io/isort/docs/upgrade_guides/5.0.0/" + ) + + if wrong_sorted_files: + sys.exit(1) + + +if __name__ == "__main__": + main() diff --git a/env/lib/python3.8/site-packages/isort/output.py b/env/lib/python3.8/site-packages/isort/output.py new file mode 100644 index 000000000..8cf915f95 --- /dev/null +++ b/env/lib/python3.8/site-packages/isort/output.py @@ -0,0 +1,558 @@ +import copy +import itertools +from functools import partial +from typing import Iterable, List, Set, Tuple + +from isort.format import format_simplified + +from . import parse, sorting, wrap +from .comments import add_to_line as with_comments +from .settings import DEFAULT_CONFIG, Config + +STATEMENT_DECLARATIONS: Tuple[str, ...] = ("def ", "cdef ", "cpdef ", "class ", "@", "async def") + + +def sorted_imports( + parsed: parse.ParsedContent, + config: Config = DEFAULT_CONFIG, + extension: str = "py", + import_type: str = "import", +) -> str: + """Adds the imports back to the file. + + (at the index of the first import) sorted alphabetically and split between groups + + """ + if parsed.import_index == -1: + return _output_as_string(parsed.lines_without_imports, parsed.line_separator) + + formatted_output: List[str] = parsed.lines_without_imports.copy() + remove_imports = [format_simplified(removal) for removal in config.remove_imports] + + sections: Iterable[str] = itertools.chain(parsed.sections, config.forced_separate) + + if config.no_sections: + parsed.imports["no_sections"] = {"straight": {}, "from": {}} + base_sections: Tuple[str, ...] = () + for section in sections: + if section == "FUTURE": + base_sections = ("FUTURE",) + continue + parsed.imports["no_sections"]["straight"].update( + parsed.imports[section].get("straight", {}) + ) + parsed.imports["no_sections"]["from"].update(parsed.imports[section].get("from", {})) + sections = base_sections + ("no_sections",) + + output: List[str] = [] + seen_headings: Set[str] = set() + pending_lines_before = False + for section in sections: + straight_modules = parsed.imports[section]["straight"] + straight_modules = sorting.naturally( + straight_modules, + key=lambda key: sorting.module_key( + key, config, section_name=section, straight_import=True + ), + ) + from_modules = parsed.imports[section]["from"] + from_modules = sorting.naturally( + from_modules, key=lambda key: sorting.module_key(key, config, section_name=section) + ) + + straight_imports = _with_straight_imports( + parsed, config, straight_modules, section, remove_imports, import_type + ) + from_imports = _with_from_imports( + parsed, config, from_modules, section, remove_imports, import_type + ) + + lines_between = [""] * ( + config.lines_between_types if from_modules and straight_modules else 0 + ) + if config.from_first: + section_output = from_imports + lines_between + straight_imports + else: + section_output = straight_imports + lines_between + from_imports + + if config.force_sort_within_sections: + # collapse comments + comments_above = [] + new_section_output: List[str] = [] + for line in section_output: + if not line: + continue + if line.startswith("#"): + comments_above.append(line) + elif comments_above: + new_section_output.append(_LineWithComments(line, comments_above)) + comments_above = [] + else: + new_section_output.append(line) + + new_section_output = sorting.naturally( + new_section_output, + key=partial( + sorting.section_key, + order_by_type=config.order_by_type, + force_to_top=config.force_to_top, + lexicographical=config.lexicographical, + length_sort=config.length_sort, + reverse_relative=config.reverse_relative, + ), + ) + + # uncollapse comments + section_output = [] + for line in new_section_output: + comments = getattr(line, "comments", ()) + if comments: + section_output.extend(comments) + section_output.append(str(line)) + + section_name = section + no_lines_before = section_name in config.no_lines_before + + if section_output: + if section_name in parsed.place_imports: + parsed.place_imports[section_name] = section_output + continue + + section_title = config.import_headings.get(section_name.lower(), "") + if section_title and section_title not in seen_headings: + if config.dedup_headings: + seen_headings.add(section_title) + section_comment = f"# {section_title}" + if section_comment not in parsed.lines_without_imports[0:1]: + section_output.insert(0, section_comment) + + if pending_lines_before or not no_lines_before: + output += [""] * config.lines_between_sections + + output += section_output + + pending_lines_before = False + else: + pending_lines_before = pending_lines_before or not no_lines_before + + if config.ensure_newline_before_comments: + output = _ensure_newline_before_comment(output) + + while output and output[-1].strip() == "": + output.pop() # pragma: no cover + while output and output[0].strip() == "": + output.pop(0) + + if config.formatting_function: + output = config.formatting_function( + parsed.line_separator.join(output), extension, config + ).splitlines() + + output_at = 0 + if parsed.import_index < parsed.original_line_count: + output_at = parsed.import_index + formatted_output[output_at:0] = output + + imports_tail = output_at + len(output) + while [ + character.strip() for character in formatted_output[imports_tail : imports_tail + 1] + ] == [""]: + formatted_output.pop(imports_tail) + + if len(formatted_output) > imports_tail: + next_construct = "" + tail = formatted_output[imports_tail:] + + for index, line in enumerate(tail): + should_skip, in_quote, *_ = parse.skip_line( + line, + in_quote="", + index=len(formatted_output), + section_comments=config.section_comments, + needs_import=False, + ) + if not should_skip and line.strip(): + if ( + line.strip().startswith("#") + and len(tail) > (index + 1) + and tail[index + 1].strip() + ): + continue + next_construct = line + break + elif in_quote: + next_construct = line + break + + if config.lines_after_imports != -1: + formatted_output[imports_tail:0] = ["" for line in range(config.lines_after_imports)] + elif extension != "pyi" and next_construct.startswith(STATEMENT_DECLARATIONS): + formatted_output[imports_tail:0] = ["", ""] + else: + formatted_output[imports_tail:0] = [""] + + if parsed.place_imports: + new_out_lines = [] + for index, line in enumerate(formatted_output): + new_out_lines.append(line) + if line in parsed.import_placements: + new_out_lines.extend(parsed.place_imports[parsed.import_placements[line]]) + if ( + len(formatted_output) <= (index + 1) + or formatted_output[index + 1].strip() != "" + ): + new_out_lines.append("") + formatted_output = new_out_lines + + return _output_as_string(formatted_output, parsed.line_separator) + + +def _with_from_imports( + parsed: parse.ParsedContent, + config: Config, + from_modules: Iterable[str], + section: str, + remove_imports: List[str], + import_type: str, +) -> List[str]: + output: List[str] = [] + for module in from_modules: + if module in remove_imports: + continue + + import_start = f"from {module} {import_type} " + from_imports = list(parsed.imports[section]["from"][module]) + if not config.no_inline_sort or ( + config.force_single_line and module not in config.single_line_exclusions + ): + ignore_case = config.force_alphabetical_sort_within_sections + from_imports = sorting.naturally( + from_imports, + key=lambda key: sorting.module_key( + key, + config, + True, + ignore_case, + section_name=section, + ), + ) + if remove_imports: + from_imports = [ + line for line in from_imports if f"{module}.{line}" not in remove_imports + ] + + sub_modules = [f"{module}.{from_import}" for from_import in from_imports] + as_imports = { + from_import: [ + f"{from_import} as {as_module}" for as_module in parsed.as_map["from"][sub_module] + ] + for from_import, sub_module in zip(from_imports, sub_modules) + if sub_module in parsed.as_map["from"] + } + if config.combine_as_imports and not ("*" in from_imports and config.combine_star): + if not config.no_inline_sort: + for as_import in as_imports: + as_imports[as_import] = sorting.naturally(as_imports[as_import]) + for from_import in copy.copy(from_imports): + if from_import in as_imports: + idx = from_imports.index(from_import) + if parsed.imports[section]["from"][module][from_import]: + from_imports[(idx + 1) : (idx + 1)] = as_imports.pop(from_import) + else: + from_imports[idx : (idx + 1)] = as_imports.pop(from_import) + + only_show_as_imports = False + comments = parsed.categorized_comments["from"].pop(module, ()) + above_comments = parsed.categorized_comments["above"]["from"].pop(module, None) + while from_imports: + if above_comments: + output.extend(above_comments) + above_comments = None + + if "*" in from_imports and config.combine_star: + import_statement = wrap.line( + with_comments( + _with_star_comments(parsed, module, list(comments or ())), + f"{import_start}*", + removed=config.ignore_comments, + comment_prefix=config.comment_prefix, + ), + parsed.line_separator, + config, + ) + from_imports = [ + from_import for from_import in from_imports if from_import in as_imports + ] + only_show_as_imports = True + elif config.force_single_line and module not in config.single_line_exclusions: + import_statement = "" + while from_imports: + from_import = from_imports.pop(0) + single_import_line = with_comments( + comments, + import_start + from_import, + removed=config.ignore_comments, + comment_prefix=config.comment_prefix, + ) + comment = ( + parsed.categorized_comments["nested"].get(module, {}).pop(from_import, None) + ) + if comment: + single_import_line += ( + f"{comments and ';' or config.comment_prefix} " f"{comment}" + ) + if from_import in as_imports: + if ( + parsed.imports[section]["from"][module][from_import] + and not only_show_as_imports + ): + output.append( + wrap.line(single_import_line, parsed.line_separator, config) + ) + from_comments = parsed.categorized_comments["straight"].get( + f"{module}.{from_import}" + ) + output.extend( + with_comments( + from_comments, + wrap.line(import_start + as_import, parsed.line_separator, config), + removed=config.ignore_comments, + comment_prefix=config.comment_prefix, + ) + for as_import in sorting.naturally(as_imports[from_import]) + ) + else: + output.append(wrap.line(single_import_line, parsed.line_separator, config)) + comments = None + else: + while from_imports and from_imports[0] in as_imports: + from_import = from_imports.pop(0) + as_imports[from_import] = sorting.naturally(as_imports[from_import]) + from_comments = ( + parsed.categorized_comments["straight"].get(f"{module}.{from_import}") or [] + ) + if ( + parsed.imports[section]["from"][module][from_import] + and not only_show_as_imports + ): + specific_comment = ( + parsed.categorized_comments["nested"] + .get(module, {}) + .pop(from_import, None) + ) + if specific_comment: + from_comments.append(specific_comment) + output.append( + wrap.line( + with_comments( + from_comments, + import_start + from_import, + removed=config.ignore_comments, + comment_prefix=config.comment_prefix, + ), + parsed.line_separator, + config, + ) + ) + from_comments = [] + + for as_import in as_imports[from_import]: + specific_comment = ( + parsed.categorized_comments["nested"] + .get(module, {}) + .pop(as_import, None) + ) + if specific_comment: + from_comments.append(specific_comment) + + output.append( + wrap.line( + with_comments( + from_comments, + import_start + as_import, + removed=config.ignore_comments, + comment_prefix=config.comment_prefix, + ), + parsed.line_separator, + config, + ) + ) + + from_comments = [] + + if "*" in from_imports: + output.append( + with_comments( + _with_star_comments(parsed, module, list(comments or ())), + f"{import_start}*", + removed=config.ignore_comments, + comment_prefix=config.comment_prefix, + ) + ) + from_imports.remove("*") + comments = None + + for from_import in copy.copy(from_imports): + comment = ( + parsed.categorized_comments["nested"].get(module, {}).pop(from_import, None) + ) + if comment: + single_import_line = with_comments( + comments, + import_start + from_import, + removed=config.ignore_comments, + comment_prefix=config.comment_prefix, + ) + single_import_line += ( + f"{comments and ';' or config.comment_prefix} " f"{comment}" + ) + output.append(wrap.line(single_import_line, parsed.line_separator, config)) + from_imports.remove(from_import) + comments = None + + from_import_section = [] + while from_imports and ( + from_imports[0] not in as_imports + or ( + config.combine_as_imports + and parsed.imports[section]["from"][module][from_import] + ) + ): + from_import_section.append(from_imports.pop(0)) + if config.combine_as_imports: + comments = (comments or []) + list( + parsed.categorized_comments["from"].pop(f"{module}.__combined_as__", ()) + ) + import_statement = with_comments( + comments, + import_start + (", ").join(from_import_section), + removed=config.ignore_comments, + comment_prefix=config.comment_prefix, + ) + if not from_import_section: + import_statement = "" + + do_multiline_reformat = False + + force_grid_wrap = config.force_grid_wrap + if force_grid_wrap and len(from_import_section) >= force_grid_wrap: + do_multiline_reformat = True + + if len(import_statement) > config.line_length and len(from_import_section) > 1: + do_multiline_reformat = True + + # If line too long AND have imports AND we are + # NOT using GRID or VERTICAL wrap modes + if ( + len(import_statement) > config.line_length + and len(from_import_section) > 0 + and config.multi_line_output + not in (wrap.Modes.GRID, wrap.Modes.VERTICAL) # type: ignore + ): + do_multiline_reformat = True + + if do_multiline_reformat: + import_statement = wrap.import_statement( + import_start=import_start, + from_imports=from_import_section, + comments=comments, + line_separator=parsed.line_separator, + config=config, + ) + if config.multi_line_output == wrap.Modes.GRID: # type: ignore + other_import_statement = wrap.import_statement( + import_start=import_start, + from_imports=from_import_section, + comments=comments, + line_separator=parsed.line_separator, + config=config, + multi_line_output=wrap.Modes.VERTICAL_GRID, # type: ignore + ) + if max(len(x) for x in import_statement.split("\n")) > config.line_length: + import_statement = other_import_statement + if not do_multiline_reformat and len(import_statement) > config.line_length: + import_statement = wrap.line(import_statement, parsed.line_separator, config) + + if import_statement: + output.append(import_statement) + return output + + +def _with_straight_imports( + parsed: parse.ParsedContent, + config: Config, + straight_modules: Iterable[str], + section: str, + remove_imports: List[str], + import_type: str, +) -> List[str]: + output: List[str] = [] + for module in straight_modules: + if module in remove_imports: + continue + + import_definition = [] + if module in parsed.as_map["straight"]: + if parsed.imports[section]["straight"][module]: + import_definition.append(f"{import_type} {module}") + import_definition.extend( + f"{import_type} {module} as {as_import}" + for as_import in parsed.as_map["straight"][module] + ) + else: + import_definition.append(f"{import_type} {module}") + + comments_above = parsed.categorized_comments["above"]["straight"].pop(module, None) + if comments_above: + output.extend(comments_above) + output.extend( + with_comments( + parsed.categorized_comments["straight"].get(module), + idef, + removed=config.ignore_comments, + comment_prefix=config.comment_prefix, + ) + for idef in import_definition + ) + + return output + + +def _output_as_string(lines: List[str], line_separator: str) -> str: + return line_separator.join(_normalize_empty_lines(lines)) + + +def _normalize_empty_lines(lines: List[str]) -> List[str]: + while lines and lines[-1].strip() == "": + lines.pop(-1) + + lines.append("") + return lines + + +class _LineWithComments(str): + def __new__(cls, value, comments): + instance = super().__new__(cls, value) # type: ignore + instance.comments = comments + return instance + + +def _ensure_newline_before_comment(output): + new_output: List[str] = [] + + def is_comment(line): + return line and line.startswith("#") + + for line, prev_line in zip(output, [None] + output): + if is_comment(line) and prev_line != "" and not is_comment(prev_line): + new_output.append("") + new_output.append(line) + return new_output + + +def _with_star_comments(parsed: parse.ParsedContent, module: str, comments: List[str]) -> List[str]: + star_comment = parsed.categorized_comments["nested"].get(module, {}).pop("*", None) + if star_comment: + return comments + [star_comment] + else: + return comments diff --git a/env/lib/python3.8/site-packages/isort/parse.py b/env/lib/python3.8/site-packages/isort/parse.py new file mode 100644 index 000000000..613f7fa76 --- /dev/null +++ b/env/lib/python3.8/site-packages/isort/parse.py @@ -0,0 +1,494 @@ +"""Defines parsing functions used by isort for parsing import definitions""" +from collections import OrderedDict, defaultdict +from functools import partial +from itertools import chain +from typing import TYPE_CHECKING, Any, Dict, List, NamedTuple, Optional, Tuple +from warnings import warn + +from . import place +from .comments import parse as parse_comments +from .deprecated.finders import FindersManager +from .settings import DEFAULT_CONFIG, Config + +if TYPE_CHECKING: + from mypy_extensions import TypedDict + + CommentsAboveDict = TypedDict( + "CommentsAboveDict", {"straight": Dict[str, Any], "from": Dict[str, Any]} + ) + + CommentsDict = TypedDict( + "CommentsDict", + { + "from": Dict[str, Any], + "straight": Dict[str, Any], + "nested": Dict[str, Any], + "above": CommentsAboveDict, + }, + ) + + +def _infer_line_separator(contents: str) -> str: + if "\r\n" in contents: + return "\r\n" + elif "\r" in contents: + return "\r" + else: + return "\n" + + +def _normalize_line(raw_line: str) -> Tuple[str, str]: + """Normalizes import related statements in the provided line. + + Returns (normalized_line: str, raw_line: str) + """ + line = raw_line.replace("from.import ", "from . import ") + line = line.replace("from.cimport ", "from . cimport ") + line = line.replace("import*", "import *") + line = line.replace(" .import ", " . import ") + line = line.replace(" .cimport ", " . cimport ") + line = line.replace("\t", " ") + return (line, raw_line) + + +def import_type(line: str, config: Config = DEFAULT_CONFIG) -> Optional[str]: + """If the current line is an import line it will return its type (from or straight)""" + if config.honor_noqa and line.lower().rstrip().endswith("noqa"): + return None + elif "isort:skip" in line or "isort: skip" in line or "isort: split" in line: + return None + elif line.startswith(("import ", "cimport ")): + return "straight" + elif line.startswith("from "): + return "from" + return None + + +def _strip_syntax(import_string: str) -> str: + import_string = import_string.replace("_import", "[[i]]") + import_string = import_string.replace("_cimport", "[[ci]]") + for remove_syntax in ["\\", "(", ")", ","]: + import_string = import_string.replace(remove_syntax, " ") + import_list = import_string.split() + for key in ("from", "import", "cimport"): + if key in import_list: + import_list.remove(key) + import_string = " ".join(import_list) + import_string = import_string.replace("[[i]]", "_import") + import_string = import_string.replace("[[ci]]", "_cimport") + return import_string.replace("{ ", "{|").replace(" }", "|}") + + +def skip_line( + line: str, + in_quote: str, + index: int, + section_comments: Tuple[str, ...], + needs_import: bool = True, +) -> Tuple[bool, str]: + """Determine if a given line should be skipped. + + Returns back a tuple containing: + + (skip_line: bool, + in_quote: str,) + """ + should_skip = bool(in_quote) + if '"' in line or "'" in line: + char_index = 0 + while char_index < len(line): + if line[char_index] == "\\": + char_index += 1 + elif in_quote: + if line[char_index : char_index + len(in_quote)] == in_quote: + in_quote = "" + elif line[char_index] in ("'", '"'): + long_quote = line[char_index : char_index + 3] + if long_quote in ('"""', "'''"): + in_quote = long_quote + char_index += 2 + else: + in_quote = line[char_index] + elif line[char_index] == "#": + break + char_index += 1 + + if ";" in line.split("#")[0] and needs_import: + for part in (part.strip() for part in line.split(";")): + if ( + part + and not part.startswith("from ") + and not part.startswith(("import ", "cimport ")) + ): + should_skip = True + + return (bool(should_skip or in_quote), in_quote) + + +class ParsedContent(NamedTuple): + in_lines: List[str] + lines_without_imports: List[str] + import_index: int + place_imports: Dict[str, List[str]] + import_placements: Dict[str, str] + as_map: Dict[str, Dict[str, List[str]]] + imports: Dict[str, Dict[str, Any]] + categorized_comments: "CommentsDict" + change_count: int + original_line_count: int + line_separator: str + sections: Any + + +def file_contents(contents: str, config: Config = DEFAULT_CONFIG) -> ParsedContent: + """Parses a python file taking out and categorizing imports.""" + line_separator: str = config.line_ending or _infer_line_separator(contents) + in_lines = contents.splitlines() + if contents and contents[-1] in ("\n", "\r"): + in_lines.append("") + + out_lines = [] + original_line_count = len(in_lines) + if config.old_finders: + finder = FindersManager(config=config).find + else: + finder = partial(place.module, config=config) + + line_count = len(in_lines) + + place_imports: Dict[str, List[str]] = {} + import_placements: Dict[str, str] = {} + as_map: Dict[str, Dict[str, List[str]]] = { + "straight": defaultdict(list), + "from": defaultdict(list), + } + imports: OrderedDict[str, Dict[str, Any]] = OrderedDict() + for section in chain(config.sections, config.forced_separate): + imports[section] = {"straight": OrderedDict(), "from": OrderedDict()} + categorized_comments: CommentsDict = { + "from": {}, + "straight": {}, + "nested": {}, + "above": {"straight": {}, "from": {}}, + } + + index = 0 + import_index = -1 + in_quote = "" + while index < line_count: + line = in_lines[index] + index += 1 + statement_index = index + (skipping_line, in_quote) = skip_line( + line, in_quote=in_quote, index=index, section_comments=config.section_comments + ) + + if line in config.section_comments and not skipping_line: + if import_index == -1: + import_index = index - 1 + continue + + if "isort:imports-" in line and line.startswith("#"): + section = line.split("isort:imports-")[-1].split()[0].upper() + place_imports[section] = [] + import_placements[line] = section + elif "isort: imports-" in line and line.startswith("#"): + section = line.split("isort: imports-")[-1].split()[0].upper() + place_imports[section] = [] + import_placements[line] = section + + if skipping_line: + out_lines.append(line) + continue + elif ( + config.float_to_top + and import_index == -1 + and line + and not in_quote + and not line.strip().startswith("#") + ): + import_index = index - 1 + while import_index and not in_lines[import_index - 1]: + import_index -= 1 + + line, *end_of_line_comment = line.split("#", 1) + if ";" in line: + statements = [line.strip() for line in line.split(";")] + else: + statements = [line] + if end_of_line_comment: + statements[-1] = f"{statements[-1]}#{end_of_line_comment[0]}" + + for statement in statements: + line, raw_line = _normalize_line(statement) + type_of_import = import_type(line, config) or "" + if not type_of_import: + out_lines.append(raw_line) + continue + + if import_index == -1: + import_index = index - 1 + nested_comments = {} + import_string, comment = parse_comments(line) + comments = [comment] if comment else [] + line_parts = [part for part in _strip_syntax(import_string).strip().split(" ") if part] + if type_of_import == "from" and len(line_parts) == 2 and comments: + nested_comments[line_parts[-1]] = comments[0] + + if "(" in line.split("#", 1)[0] and index < line_count: + while not line.split("#")[0].strip().endswith(")") and index < line_count: + line, new_comment = parse_comments(in_lines[index]) + index += 1 + if new_comment: + comments.append(new_comment) + stripped_line = _strip_syntax(line).strip() + if ( + type_of_import == "from" + and stripped_line + and " " not in stripped_line.replace(" as ", "") + and new_comment + ): + nested_comments[stripped_line] = comments[-1] + import_string += line_separator + line + else: + while line.strip().endswith("\\"): + line, new_comment = parse_comments(in_lines[index]) + index += 1 + if new_comment: + comments.append(new_comment) + + # Still need to check for parentheses after an escaped line + if ( + "(" in line.split("#")[0] + and ")" not in line.split("#")[0] + and index < line_count + ): + stripped_line = _strip_syntax(line).strip() + if ( + type_of_import == "from" + and stripped_line + and " " not in stripped_line.replace(" as ", "") + and new_comment + ): + nested_comments[stripped_line] = comments[-1] + import_string += line_separator + line + + while not line.split("#")[0].strip().endswith(")") and index < line_count: + line, new_comment = parse_comments(in_lines[index]) + index += 1 + if new_comment: + comments.append(new_comment) + stripped_line = _strip_syntax(line).strip() + if ( + type_of_import == "from" + and stripped_line + and " " not in stripped_line.replace(" as ", "") + and new_comment + ): + nested_comments[stripped_line] = comments[-1] + import_string += line_separator + line + + stripped_line = _strip_syntax(line).strip() + if ( + type_of_import == "from" + and stripped_line + and " " not in stripped_line.replace(" as ", "") + and new_comment + ): + nested_comments[stripped_line] = comments[-1] + if import_string.strip().endswith( + (" import", " cimport") + ) or line.strip().startswith(("import ", "cimport ")): + import_string += line_separator + line + else: + import_string = import_string.rstrip().rstrip("\\") + " " + line.lstrip() + + if type_of_import == "from": + cimports: bool + import_string = ( + import_string.replace("import(", "import (") + .replace("\\", " ") + .replace("\n", " ") + ) + if " cimport " in import_string: + parts = import_string.split(" cimport ") + cimports = True + + else: + parts = import_string.split(" import ") + cimports = False + + from_import = parts[0].split(" ") + import_string = (" cimport " if cimports else " import ").join( + [from_import[0] + " " + "".join(from_import[1:])] + parts[1:] + ) + + just_imports = [ + item.replace("{|", "{ ").replace("|}", " }") + for item in _strip_syntax(import_string).split() + ] + straight_import = True + attach_comments_to: Optional[List[Any]] = None + if "as" in just_imports and (just_imports.index("as") + 1) < len(just_imports): + straight_import = False + while "as" in just_imports: + nested_module = None + as_index = just_imports.index("as") + if type_of_import == "from": + nested_module = just_imports[as_index - 1] + top_level_module = just_imports[0] + module = top_level_module + "." + nested_module + as_name = just_imports[as_index + 1] + if nested_module == as_name and config.remove_redundant_aliases: + pass + elif as_name not in as_map["from"][module]: + as_map["from"][module].append(as_name) + + full_name = f"{nested_module} as {as_name}" + associated_comment = nested_comments.get(full_name) + if associated_comment: + categorized_comments["nested"].setdefault(top_level_module, {})[ + full_name + ] = associated_comment + if associated_comment in comments: + comments.pop(comments.index(associated_comment)) + else: + module = just_imports[as_index - 1] + as_name = just_imports[as_index + 1] + if module == as_name and config.remove_redundant_aliases: + pass + elif as_name not in as_map["straight"][module]: + as_map["straight"][module].append(as_name) + + if comments and attach_comments_to is None: + if nested_module and config.combine_as_imports: + attach_comments_to = categorized_comments["from"].setdefault( + f"{top_level_module}.__combined_as__", [] + ) + else: + attach_comments_to = categorized_comments["straight"].setdefault( + module, [] + ) + del just_imports[as_index : as_index + 2] + + if type_of_import == "from": + import_from = just_imports.pop(0) + placed_module = finder(import_from) + if config.verbose: + print(f"from-type place_module for {import_from} returned {placed_module}") + if placed_module == "": + warn( + f"could not place module {import_from} of line {line} --" + " Do you need to define a default section?" + ) + root = imports[placed_module][type_of_import] # type: ignore + for import_name in just_imports: + associated_comment = nested_comments.get(import_name) + if associated_comment: + categorized_comments["nested"].setdefault(import_from, {})[ + import_name + ] = associated_comment + if associated_comment in comments: + comments.pop(comments.index(associated_comment)) + + if comments and attach_comments_to is None: + attach_comments_to = categorized_comments["from"].setdefault(import_from, []) + + if len(out_lines) > max(import_index, 1) - 1: + last = out_lines and out_lines[-1].rstrip() or "" + while ( + last.startswith("#") + and not last.endswith('"""') + and not last.endswith("'''") + and "isort:imports-" not in last + and "isort: imports-" not in last + and not config.treat_all_comments_as_code + and not last.strip() in config.treat_comments_as_code + ): + categorized_comments["above"]["from"].setdefault(import_from, []).insert( + 0, out_lines.pop(-1) + ) + if out_lines: + last = out_lines[-1].rstrip() + else: + last = "" + if statement_index - 1 == import_index: # pragma: no cover + import_index -= len( + categorized_comments["above"]["from"].get(import_from, []) + ) + + if import_from not in root: + root[import_from] = OrderedDict( + (module, straight_import) for module in just_imports + ) + else: + root[import_from].update( + (module, straight_import | root[import_from].get(module, False)) + for module in just_imports + ) + + if comments and attach_comments_to is not None: + attach_comments_to.extend(comments) + else: + if comments and attach_comments_to is not None: + attach_comments_to.extend(comments) + comments = [] + + for module in just_imports: + if comments: + categorized_comments["straight"][module] = comments + comments = [] + + if len(out_lines) > max(import_index, +1, 1) - 1: + + last = out_lines and out_lines[-1].rstrip() or "" + while ( + last.startswith("#") + and not last.endswith('"""') + and not last.endswith("'''") + and "isort:imports-" not in last + and "isort: imports-" not in last + and not config.treat_all_comments_as_code + and not last.strip() in config.treat_comments_as_code + ): + categorized_comments["above"]["straight"].setdefault(module, []).insert( + 0, out_lines.pop(-1) + ) + if out_lines: + last = out_lines[-1].rstrip() + else: + last = "" + if index - 1 == import_index: + import_index -= len( + categorized_comments["above"]["straight"].get(module, []) + ) + placed_module = finder(module) + if config.verbose: + print(f"else-type place_module for {module} returned {placed_module}") + if placed_module == "": + warn( + f"could not place module {module} of line {line} --" + " Do you need to define a default section?" + ) + imports.setdefault("", {"straight": OrderedDict(), "from": OrderedDict()}) + straight_import |= imports[placed_module][type_of_import].get( # type: ignore + module, False + ) + imports[placed_module][type_of_import][module] = straight_import # type: ignore + + change_count = len(out_lines) - original_line_count + + return ParsedContent( + in_lines=in_lines, + lines_without_imports=out_lines, + import_index=import_index, + place_imports=place_imports, + import_placements=import_placements, + as_map=as_map, + imports=imports, + categorized_comments=categorized_comments, + change_count=change_count, + original_line_count=original_line_count, + line_separator=line_separator, + sections=config.sections, + ) diff --git a/env/lib/python3.8/site-packages/isort/place.py b/env/lib/python3.8/site-packages/isort/place.py new file mode 100644 index 000000000..34b2eeb86 --- /dev/null +++ b/env/lib/python3.8/site-packages/isort/place.py @@ -0,0 +1,95 @@ +"""Contains all logic related to placing an import within a certain section.""" +import importlib +from fnmatch import fnmatch +from functools import lru_cache +from pathlib import Path +from typing import Optional, Tuple + +from isort import sections +from isort.settings import DEFAULT_CONFIG, Config +from isort.utils import exists_case_sensitive + +LOCAL = "LOCALFOLDER" + + +def module(name: str, config: Config = DEFAULT_CONFIG) -> str: + """Returns the section placement for the given module name.""" + return module_with_reason(name, config)[0] + + +@lru_cache(maxsize=1000) +def module_with_reason(name: str, config: Config = DEFAULT_CONFIG) -> Tuple[str, str]: + """Returns the section placement for the given module name alongside the reasoning.""" + return ( + _forced_separate(name, config) + or _local(name, config) + or _known_pattern(name, config) + or _src_path(name, config) + or (config.default_section, "Default option in Config or universal default.") + ) + + +def _forced_separate(name: str, config: Config) -> Optional[Tuple[str, str]]: + for forced_separate in config.forced_separate: + # Ensure all forced_separate patterns will match to end of string + path_glob = forced_separate + if not forced_separate.endswith("*"): + path_glob = "%s*" % forced_separate + + if fnmatch(name, path_glob) or fnmatch(name, "." + path_glob): + return (forced_separate, f"Matched forced_separate ({forced_separate}) config value.") + + return None + + +def _local(name: str, config: Config) -> Optional[Tuple[str, str]]: + if name.startswith("."): + return (LOCAL, "Module name started with a dot.") + + return None + + +def _known_pattern(name: str, config: Config) -> Optional[Tuple[str, str]]: + parts = name.split(".") + module_names_to_check = (".".join(parts[:first_k]) for first_k in range(len(parts), 0, -1)) + for module_name_to_check in module_names_to_check: + for pattern, placement in config.known_patterns: + if pattern.match(module_name_to_check): + return (placement, f"Matched configured known pattern {pattern}") + + return None + + +def _src_path(name: str, config: Config) -> Optional[Tuple[str, str]]: + for src_path in config.src_paths: + root_module_name = name.split(".")[0] + module_path = (src_path / root_module_name).resolve() + if ( + _is_module(module_path) + or _is_package(module_path) + or _src_path_is_module(src_path, root_module_name) + ): + return (sections.FIRSTPARTY, f"Found in one of the configured src_paths: {src_path}.") + + return None + + +def _is_module(path: Path) -> bool: + return ( + exists_case_sensitive(str(path.with_suffix(".py"))) + or any( + exists_case_sensitive(str(path.with_suffix(ext_suffix))) + for ext_suffix in importlib.machinery.EXTENSION_SUFFIXES + ) + or exists_case_sensitive(str(path / "__init__.py")) + ) + + +def _is_package(path: Path) -> bool: + return exists_case_sensitive(str(path)) and path.is_dir() + + +def _src_path_is_module(src_path: Path, module_name: str) -> bool: + return ( + module_name == src_path.name and src_path.is_dir() and exists_case_sensitive(str(src_path)) + ) diff --git a/env/lib/python3.8/site-packages/isort/profiles.py b/env/lib/python3.8/site-packages/isort/profiles.py new file mode 100644 index 000000000..cd976cd29 --- /dev/null +++ b/env/lib/python3.8/site-packages/isort/profiles.py @@ -0,0 +1,62 @@ +"""Common profiles are defined here to be easily used within a project using --profile {name}""" +from typing import Any, Dict + +black = { + "multi_line_output": 3, + "include_trailing_comma": True, + "force_grid_wrap": 0, + "use_parentheses": True, + "ensure_newline_before_comments": True, + "line_length": 88, +} +django = { + "combine_as_imports": True, + "include_trailing_comma": True, + "multi_line_output": 5, + "line_length": 79, +} +pycharm = {"multi_line_output": 3, "force_grid_wrap": 2} +google = { + "force_single_line": True, + "force_sort_within_sections": True, + "lexicographical": True, + "single_line_exclusions": ("typing",), +} +open_stack = { + "force_single_line": True, + "force_sort_within_sections": True, + "lexicographical": True, +} +plone = { + "force_alphabetical_sort": True, + "force_single_line": True, + "lines_after_imports": 2, + "line_length": 200, +} +attrs = { + "atomic": True, + "force_grid_wrap": 0, + "include_trailing_comma": True, + "lines_after_imports": 2, + "lines_between_types": 1, + "multi_line_output": 3, + "use_parentheses": True, +} +hug = { + "multi_line_output": 3, + "include_trailing_comma": True, + "force_grid_wrap": 0, + "use_parentheses": True, + "line_length": 100, +} + +profiles: Dict[str, Dict[str, Any]] = { + "black": black, + "django": django, + "pycharm": pycharm, + "google": google, + "open_stack": open_stack, + "plone": plone, + "attrs": attrs, + "hug": hug, +} diff --git a/env/lib/python3.8/site-packages/isort/pylama_isort.py b/env/lib/python3.8/site-packages/isort/pylama_isort.py new file mode 100644 index 000000000..0e14d5696 --- /dev/null +++ b/env/lib/python3.8/site-packages/isort/pylama_isort.py @@ -0,0 +1,33 @@ +import os +import sys +from contextlib import contextmanager +from typing import Any, Dict, List + +from pylama.lint import Linter as BaseLinter + +from . import api + + +@contextmanager +def supress_stdout(): + stdout = sys.stdout + with open(os.devnull, "w") as devnull: + sys.stdout = devnull + yield + sys.stdout = stdout + + +class Linter(BaseLinter): + def allow(self, path: str) -> bool: + """Determine if this path should be linted.""" + return path.endswith(".py") + + def run(self, path: str, **meta: Any) -> List[Dict[str, Any]]: + """Lint the file. Return an array of error dicts if appropriate.""" + with supress_stdout(): + if not api.check_file(path): + return [ + {"lnum": 0, "col": 0, "text": "Incorrectly sorted imports.", "type": "ISORT"} + ] + else: + return [] diff --git a/env/lib/python3.8/site-packages/isort/sections.py b/env/lib/python3.8/site-packages/isort/sections.py new file mode 100644 index 000000000..f59db6926 --- /dev/null +++ b/env/lib/python3.8/site-packages/isort/sections.py @@ -0,0 +1,9 @@ +"""Defines all sections isort uses by default""" +from typing import Tuple + +FUTURE: str = "FUTURE" +STDLIB: str = "STDLIB" +THIRDPARTY: str = "THIRDPARTY" +FIRSTPARTY: str = "FIRSTPARTY" +LOCALFOLDER: str = "LOCALFOLDER" +DEFAULT: Tuple[str, ...] = (FUTURE, STDLIB, THIRDPARTY, FIRSTPARTY, LOCALFOLDER) diff --git a/env/lib/python3.8/site-packages/isort/settings.py b/env/lib/python3.8/site-packages/isort/settings.py new file mode 100644 index 000000000..94a14acf2 --- /dev/null +++ b/env/lib/python3.8/site-packages/isort/settings.py @@ -0,0 +1,702 @@ +"""isort/settings.py. + +Defines how the default settings for isort should be loaded +""" +import configparser +import fnmatch +import os +import posixpath +import re +import stat +import subprocess # nosec: Needed for gitignore support. +import sys +from functools import lru_cache +from pathlib import Path +from typing import Any, Callable, Dict, FrozenSet, Iterable, List, Optional, Pattern, Set, Tuple +from warnings import warn + +from . import stdlibs +from ._future import dataclass, field +from ._vendored import toml +from .exceptions import FormattingPluginDoesNotExist, InvalidSettingsPath, ProfileDoesNotExist +from .profiles import profiles +from .sections import DEFAULT as SECTION_DEFAULTS +from .sections import FIRSTPARTY, FUTURE, LOCALFOLDER, STDLIB, THIRDPARTY +from .wrap_modes import WrapModes +from .wrap_modes import from_string as wrap_mode_from_string + +_SHEBANG_RE = re.compile(br"^#!.*\bpython[23w]?\b") +SUPPORTED_EXTENSIONS = frozenset({"py", "pyi", "pyx"}) +BLOCKED_EXTENSIONS = frozenset({"pex"}) +FILE_SKIP_COMMENTS: Tuple[str, ...] = ( + "isort:" + "skip_file", + "isort: " + "skip_file", +) # Concatenated to avoid this file being skipped +MAX_CONFIG_SEARCH_DEPTH: int = 25 # The number of parent directories to for a config file within +STOP_CONFIG_SEARCH_ON_DIRS: Tuple[str, ...] = (".git", ".hg") +VALID_PY_TARGETS: Tuple[str, ...] = tuple( + target.replace("py", "") for target in dir(stdlibs) if not target.startswith("_") +) +CONFIG_SOURCES: Tuple[str, ...] = ( + ".isort.cfg", + "pyproject.toml", + "setup.cfg", + "tox.ini", + ".editorconfig", +) +DEFAULT_SKIP: FrozenSet[str] = frozenset( + { + ".venv", + "venv", + ".tox", + ".eggs", + ".git", + ".hg", + ".mypy_cache", + ".nox", + ".svn", + ".bzr", + "_build", + "buck-out", + "build", + "dist", + ".pants.d", + ".direnv", + "node_modules", + } +) + +CONFIG_SECTIONS: Dict[str, Tuple[str, ...]] = { + ".isort.cfg": ("settings", "isort"), + "pyproject.toml": ("tool.isort",), + "setup.cfg": ("isort", "tool:isort"), + "tox.ini": ("isort", "tool:isort"), + ".editorconfig": ("*", "*.py", "**.py", "*.{py}"), +} +FALLBACK_CONFIG_SECTIONS: Tuple[str, ...] = ("isort", "tool:isort", "tool.isort") + +IMPORT_HEADING_PREFIX = "import_heading_" +KNOWN_PREFIX = "known_" +KNOWN_SECTION_MAPPING: Dict[str, str] = { + STDLIB: "STANDARD_LIBRARY", + FUTURE: "FUTURE_LIBRARY", + FIRSTPARTY: "FIRST_PARTY", + THIRDPARTY: "THIRD_PARTY", + LOCALFOLDER: "LOCAL_FOLDER", +} + +RUNTIME_SOURCE = "runtime" + +DEPRECATED_SETTINGS = ("not_skip", "keep_direct_and_as_imports") + +_STR_BOOLEAN_MAPPING = { + "y": True, + "yes": True, + "t": True, + "on": True, + "1": True, + "true": True, + "n": False, + "no": False, + "f": False, + "off": False, + "0": False, + "false": False, +} + + +@dataclass(frozen=True) +class _Config: + """Defines the data schema and defaults used for isort configuration. + + NOTE: known lists, such as known_standard_library, are intentionally not complete as they are + dynamically determined later on. + """ + + py_version: str = "3" + force_to_top: FrozenSet[str] = frozenset() + skip: FrozenSet[str] = DEFAULT_SKIP + skip_glob: FrozenSet[str] = frozenset() + skip_gitignore: bool = False + line_length: int = 79 + wrap_length: int = 0 + line_ending: str = "" + sections: Tuple[str, ...] = SECTION_DEFAULTS + no_sections: bool = False + known_future_library: FrozenSet[str] = frozenset(("__future__",)) + known_third_party: FrozenSet[str] = frozenset() + known_first_party: FrozenSet[str] = frozenset() + known_local_folder: FrozenSet[str] = frozenset() + known_standard_library: FrozenSet[str] = frozenset() + extra_standard_library: FrozenSet[str] = frozenset() + known_other: Dict[str, FrozenSet[str]] = field(default_factory=dict) + multi_line_output: WrapModes = WrapModes.GRID # type: ignore + forced_separate: Tuple[str, ...] = () + indent: str = " " * 4 + comment_prefix: str = " #" + length_sort: bool = False + length_sort_straight: bool = False + length_sort_sections: FrozenSet[str] = frozenset() + add_imports: FrozenSet[str] = frozenset() + remove_imports: FrozenSet[str] = frozenset() + append_only: bool = False + reverse_relative: bool = False + force_single_line: bool = False + single_line_exclusions: Tuple[str, ...] = () + default_section: str = THIRDPARTY + import_headings: Dict[str, str] = field(default_factory=dict) + balanced_wrapping: bool = False + use_parentheses: bool = False + order_by_type: bool = True + atomic: bool = False + lines_after_imports: int = -1 + lines_between_sections: int = 1 + lines_between_types: int = 0 + combine_as_imports: bool = False + combine_star: bool = False + include_trailing_comma: bool = False + from_first: bool = False + verbose: bool = False + quiet: bool = False + force_adds: bool = False + force_alphabetical_sort_within_sections: bool = False + force_alphabetical_sort: bool = False + force_grid_wrap: int = 0 + force_sort_within_sections: bool = False + lexicographical: bool = False + ignore_whitespace: bool = False + no_lines_before: FrozenSet[str] = frozenset() + no_inline_sort: bool = False + ignore_comments: bool = False + case_sensitive: bool = False + sources: Tuple[Dict[str, Any], ...] = () + virtual_env: str = "" + conda_env: str = "" + ensure_newline_before_comments: bool = False + directory: str = "" + profile: str = "" + honor_noqa: bool = False + src_paths: FrozenSet[Path] = frozenset() + old_finders: bool = False + remove_redundant_aliases: bool = False + float_to_top: bool = False + filter_files: bool = False + formatter: str = "" + formatting_function: Optional[Callable[[str, str, object], str]] = None + color_output: bool = False + treat_comments_as_code: FrozenSet[str] = frozenset() + treat_all_comments_as_code: bool = False + supported_extensions: FrozenSet[str] = SUPPORTED_EXTENSIONS + blocked_extensions: FrozenSet[str] = BLOCKED_EXTENSIONS + constants: FrozenSet[str] = frozenset() + classes: FrozenSet[str] = frozenset() + variables: FrozenSet[str] = frozenset() + dedup_headings: bool = False + + def __post_init__(self): + py_version = self.py_version + if py_version == "auto": # pragma: no cover + if sys.version_info.major == 2 and sys.version_info.minor <= 6: + py_version = "2" + elif sys.version_info.major == 3 and ( + sys.version_info.minor <= 5 or sys.version_info.minor >= 9 + ): + py_version = "3" + else: + py_version = f"{sys.version_info.major}{sys.version_info.minor}" + + if py_version not in VALID_PY_TARGETS: + raise ValueError( + f"The python version {py_version} is not supported. " + "You can set a python version with the -py or --python-version flag. " + f"The following versions are supported: {VALID_PY_TARGETS}" + ) + + if py_version != "all": + object.__setattr__(self, "py_version", f"py{py_version}") + + if not self.known_standard_library: + object.__setattr__( + self, "known_standard_library", frozenset(getattr(stdlibs, self.py_version).stdlib) + ) + + if self.force_alphabetical_sort: + object.__setattr__(self, "force_alphabetical_sort_within_sections", True) + object.__setattr__(self, "no_sections", True) + object.__setattr__(self, "lines_between_types", 1) + object.__setattr__(self, "from_first", True) + if self.wrap_length > self.line_length: + raise ValueError( + "wrap_length must be set lower than or equal to line_length: " + f"{self.wrap_length} > {self.line_length}." + ) + + def __hash__(self): + return id(self) + + +_DEFAULT_SETTINGS = {**vars(_Config()), "source": "defaults"} + + +class Config(_Config): + def __init__( + self, + settings_file: str = "", + settings_path: str = "", + config: Optional[_Config] = None, + **config_overrides, + ): + self._known_patterns: Optional[List[Tuple[Pattern[str], str]]] = None + self._section_comments: Optional[Tuple[str, ...]] = None + + if config: + config_vars = vars(config).copy() + config_vars.update(config_overrides) + config_vars["py_version"] = config_vars["py_version"].replace("py", "") + config_vars.pop("_known_patterns") + config_vars.pop("_section_comments") + super().__init__(**config_vars) # type: ignore + return + + sources: List[Dict[str, Any]] = [_DEFAULT_SETTINGS] + + config_settings: Dict[str, Any] + project_root: str + if settings_file: + config_settings = _get_config_data( + settings_file, + CONFIG_SECTIONS.get(os.path.basename(settings_file), FALLBACK_CONFIG_SECTIONS), + ) + project_root = os.path.dirname(settings_file) + elif settings_path: + if not os.path.exists(settings_path): + raise InvalidSettingsPath(settings_path) + + settings_path = os.path.abspath(settings_path) + project_root, config_settings = _find_config(settings_path) + else: + config_settings = {} + project_root = os.getcwd() + + profile_name = config_overrides.get("profile", config_settings.get("profile", "")) + profile: Dict[str, Any] = {} + if profile_name: + if profile_name not in profiles: + import pkg_resources + + for plugin in pkg_resources.iter_entry_points("isort.profiles"): + profiles.setdefault(plugin.name, plugin.load()) + + if profile_name not in profiles: + raise ProfileDoesNotExist(profile_name) + + profile = profiles[profile_name].copy() + profile["source"] = f"{profile_name} profile" + sources.append(profile) + + if config_settings: + sources.append(config_settings) + if config_overrides: + config_overrides["source"] = RUNTIME_SOURCE + sources.append(config_overrides) + + combined_config = {**profile, **config_settings, **config_overrides} + if "indent" in combined_config: + indent = str(combined_config["indent"]) + if indent.isdigit(): + indent = " " * int(indent) + else: + indent = indent.strip("'").strip('"') + if indent.lower() == "tab": + indent = "\t" + combined_config["indent"] = indent + + known_other = {} + import_headings = {} + for key, value in tuple(combined_config.items()): + # Collect all known sections beyond those that have direct entries + if key.startswith(KNOWN_PREFIX) and key not in ( + "known_standard_library", + "known_future_library", + "known_third_party", + "known_first_party", + "known_local_folder", + ): + import_heading = key[len(KNOWN_PREFIX) :].lower() + maps_to_section = import_heading.upper() + combined_config.pop(key) + if maps_to_section in KNOWN_SECTION_MAPPING: + section_name = f"known_{KNOWN_SECTION_MAPPING[maps_to_section].lower()}" + if section_name in combined_config and not self.quiet: + warn( + f"Can't set both {key} and {section_name} in the same config file.\n" + f"Default to {section_name} if unsure." + "\n\n" + "See: https://pycqa.github.io/isort/" + "#custom-sections-and-ordering." + ) + else: + combined_config[section_name] = frozenset(value) + else: + known_other[import_heading] = frozenset(value) + if ( + maps_to_section not in combined_config.get("sections", ()) + and not self.quiet + ): + warn( + f"`{key}` setting is defined, but {maps_to_section} is not" + " included in `sections` config option:" + f" {combined_config.get('sections', SECTION_DEFAULTS)}.\n\n" + "See: https://pycqa.github.io/isort/" + "#custom-sections-and-ordering." + ) + if key.startswith(IMPORT_HEADING_PREFIX): + import_headings[key[len(IMPORT_HEADING_PREFIX) :].lower()] = str(value) + + # Coerce all provided config values into their correct type + default_value = _DEFAULT_SETTINGS.get(key, None) + if default_value is None: + continue + + combined_config[key] = type(default_value)(value) + + for section in combined_config.get("sections", ()): + if section in SECTION_DEFAULTS: + continue + elif not section.lower() in known_other: + config_keys = ", ".join(known_other.keys()) + warn( + f"`sections` setting includes {section}, but no known_{section.lower()} " + "is defined. " + f"The following known_SECTION config options are defined: {config_keys}." + ) + + if "directory" not in combined_config: + combined_config["directory"] = ( + os.path.dirname(config_settings["source"]) + if config_settings.get("source", None) + else os.getcwd() + ) + + path_root = Path(combined_config.get("directory", project_root)).resolve() + path_root = path_root if path_root.is_dir() else path_root.parent + if "src_paths" not in combined_config: + combined_config["src_paths"] = frozenset((path_root, path_root / "src")) + else: + combined_config["src_paths"] = frozenset( + path_root / path for path in combined_config.get("src_paths", ()) + ) + + if "formatter" in combined_config: + import pkg_resources + + for plugin in pkg_resources.iter_entry_points("isort.formatters"): + if plugin.name == combined_config["formatter"]: + combined_config["formatting_function"] = plugin.load() + break + else: + raise FormattingPluginDoesNotExist(combined_config["formatter"]) + + # Remove any config values that are used for creating config object but + # aren't defined in dataclass + combined_config.pop("source", None) + combined_config.pop("sources", None) + combined_config.pop("runtime_src_paths", None) + + deprecated_options_used = [ + option for option in combined_config if option in DEPRECATED_SETTINGS + ] + if deprecated_options_used: + for deprecated_option in deprecated_options_used: + combined_config.pop(deprecated_option) + if not self.quiet: + warn( + "W0503: Deprecated config options were used: " + f"{', '.join(deprecated_options_used)}." + "Please see the 5.0.0 upgrade guide: bit.ly/isortv5." + ) + + if known_other: + combined_config["known_other"] = known_other + if import_headings: + for import_heading_key in import_headings: + combined_config.pop(f"{IMPORT_HEADING_PREFIX}{import_heading_key}") + combined_config["import_headings"] = import_headings + + super().__init__(sources=tuple(sources), **combined_config) # type: ignore + + def is_supported_filetype(self, file_name: str): + _root, ext = os.path.splitext(file_name) + ext = ext.lstrip(".") + if ext in self.supported_extensions: + return True + elif ext in self.blocked_extensions: + return False + + # Skip editor backup files. + if file_name.endswith("~"): + return False + + try: + if stat.S_ISFIFO(os.stat(file_name).st_mode): + return False + except OSError: + pass + + try: + with open(file_name, "rb") as fp: + line = fp.readline(100) + except OSError: + return False + else: + return bool(_SHEBANG_RE.match(line)) + + def is_skipped(self, file_path: Path) -> bool: + """Returns True if the file and/or folder should be skipped based on current settings.""" + if self.directory and Path(self.directory) in file_path.resolve().parents: + file_name = os.path.relpath(file_path.resolve(), self.directory) + else: + file_name = str(file_path) + + os_path = str(file_path) + + if self.skip_gitignore: + if file_path.name == ".git": # pragma: no cover + return True + + result = subprocess.run( # nosec + ["git", "-C", str(file_path.parent), "check-ignore", "--quiet", os_path] + ) + if result.returncode == 0: + return True + + normalized_path = os_path.replace("\\", "/") + if normalized_path[1:2] == ":": + normalized_path = normalized_path[2:] + + for skip_path in self.skip: + if posixpath.abspath(normalized_path) == posixpath.abspath( + skip_path.replace("\\", "/") + ): + return True + + position = os.path.split(file_name) + while position[1]: + if position[1] in self.skip: + return True + position = os.path.split(position[0]) + + for glob in self.skip_glob: + if fnmatch.fnmatch(file_name, glob) or fnmatch.fnmatch("/" + file_name, glob): + return True + + if not (os.path.isfile(os_path) or os.path.isdir(os_path) or os.path.islink(os_path)): + return True + + return False + + @property + def known_patterns(self): + if self._known_patterns is not None: + return self._known_patterns + + self._known_patterns = [] + pattern_sections = [STDLIB] + [section for section in self.sections if section != STDLIB] + for placement in reversed(pattern_sections): + known_placement = KNOWN_SECTION_MAPPING.get(placement, placement).lower() + config_key = f"{KNOWN_PREFIX}{known_placement}" + known_modules = getattr(self, config_key, self.known_other.get(known_placement, ())) + extra_modules = getattr(self, f"extra_{known_placement}", ()) + all_modules = set(extra_modules).union(known_modules) + known_patterns = [ + pattern + for known_pattern in all_modules + for pattern in self._parse_known_pattern(known_pattern) + ] + for known_pattern in known_patterns: + regexp = "^" + known_pattern.replace("*", ".*").replace("?", ".?") + "$" + self._known_patterns.append((re.compile(regexp), placement)) + + return self._known_patterns + + @property + def section_comments(self) -> Tuple[str, ...]: + if self._section_comments is not None: + return self._section_comments + + self._section_comments = tuple(f"# {heading}" for heading in self.import_headings.values()) + return self._section_comments + + def _parse_known_pattern(self, pattern: str) -> List[str]: + """Expand pattern if identified as a directory and return found sub packages""" + if pattern.endswith(os.path.sep): + patterns = [ + filename + for filename in os.listdir(os.path.join(self.directory, pattern)) + if os.path.isdir(os.path.join(self.directory, pattern, filename)) + ] + else: + patterns = [pattern] + + return patterns + + +def _get_str_to_type_converter(setting_name: str) -> Callable[[str], Any]: + type_converter: Callable[[str], Any] = type(_DEFAULT_SETTINGS.get(setting_name, "")) + if type_converter == WrapModes: + type_converter = wrap_mode_from_string + return type_converter + + +def _as_list(value: str) -> List[str]: + if isinstance(value, list): + return [item.strip() for item in value] + filtered = [item.strip() for item in value.replace("\n", ",").split(",") if item.strip()] + return filtered + + +def _abspaths(cwd: str, values: Iterable[str]) -> Set[str]: + paths = { + os.path.join(cwd, value) + if not value.startswith(os.path.sep) and value.endswith(os.path.sep) + else value + for value in values + } + return paths + + +@lru_cache() +def _find_config(path: str) -> Tuple[str, Dict[str, Any]]: + current_directory = path + tries = 0 + while current_directory and tries < MAX_CONFIG_SEARCH_DEPTH: + for config_file_name in CONFIG_SOURCES: + potential_config_file = os.path.join(current_directory, config_file_name) + if os.path.isfile(potential_config_file): + config_data: Dict[str, Any] + try: + config_data = _get_config_data( + potential_config_file, CONFIG_SECTIONS[config_file_name] + ) + except Exception: + warn(f"Failed to pull configuration information from {potential_config_file}") + config_data = {} + if config_data: + return (current_directory, config_data) + + for stop_dir in STOP_CONFIG_SEARCH_ON_DIRS: + if os.path.isdir(os.path.join(current_directory, stop_dir)): + return (current_directory, {}) + + new_directory = os.path.split(current_directory)[0] + if new_directory == current_directory: + break + + current_directory = new_directory + tries += 1 + + return (path, {}) + + +@lru_cache() +def _get_config_data(file_path: str, sections: Tuple[str]) -> Dict[str, Any]: + settings: Dict[str, Any] = {} + + with open(file_path) as config_file: + if file_path.endswith(".toml"): + config = toml.load(config_file) + for section in sections: + config_section = config + for key in section.split("."): + config_section = config_section.get(key, {}) + settings.update(config_section) + else: + if file_path.endswith(".editorconfig"): + line = "\n" + last_position = config_file.tell() + while line: + line = config_file.readline() + if "[" in line: + config_file.seek(last_position) + break + last_position = config_file.tell() + + config = configparser.ConfigParser(strict=False) + config.read_file(config_file) + for section in sections: + if section.startswith("*.{") and section.endswith("}"): + extension = section[len("*.{") : -1] + for config_key in config.keys(): + if config_key.startswith("*.{") and config_key.endswith("}"): + if extension in map( + lambda text: text.strip(), config_key[len("*.{") : -1].split(",") + ): + settings.update(config.items(config_key)) + + elif config.has_section(section): + settings.update(config.items(section)) + + if settings: + settings["source"] = file_path + + if file_path.endswith(".editorconfig"): + indent_style = settings.pop("indent_style", "").strip() + indent_size = settings.pop("indent_size", "").strip() + if indent_size == "tab": + indent_size = settings.pop("tab_width", "").strip() + + if indent_style == "space": + settings["indent"] = " " * (indent_size and int(indent_size) or 4) + + elif indent_style == "tab": + settings["indent"] = "\t" * (indent_size and int(indent_size) or 1) + + max_line_length = settings.pop("max_line_length", "").strip() + if max_line_length and (max_line_length == "off" or max_line_length.isdigit()): + settings["line_length"] = ( + float("inf") if max_line_length == "off" else int(max_line_length) + ) + settings = { + key: value + for key, value in settings.items() + if key in _DEFAULT_SETTINGS.keys() or key.startswith(KNOWN_PREFIX) + } + + for key, value in settings.items(): + existing_value_type = _get_str_to_type_converter(key) + if existing_value_type == tuple: + settings[key] = tuple(_as_list(value)) + elif existing_value_type == frozenset: + settings[key] = frozenset(_as_list(settings.get(key))) # type: ignore + elif existing_value_type == bool: + # Only some configuration formats support native boolean values. + if not isinstance(value, bool): + value = _as_bool(value) + settings[key] = value + elif key.startswith(KNOWN_PREFIX): + settings[key] = _abspaths(os.path.dirname(file_path), _as_list(value)) + elif key == "force_grid_wrap": + try: + result = existing_value_type(value) + except ValueError: # backwards compatibility for true / false force grid wrap + result = 0 if value.lower().strip() == "false" else 2 + settings[key] = result + elif key == "comment_prefix": + settings[key] = str(value).strip("'").strip('"') + else: + settings[key] = existing_value_type(value) + + return settings + + +def _as_bool(value: str) -> bool: + """Given a string value that represents True or False, returns the Boolean equivalent. + Heavily inspired from distutils strtobool. + """ + try: + return _STR_BOOLEAN_MAPPING[value.lower()] + except KeyError: + raise ValueError(f"invalid truth value {value}") + + +DEFAULT_CONFIG = Config() diff --git a/env/lib/python3.8/site-packages/isort/setuptools_commands.py b/env/lib/python3.8/site-packages/isort/setuptools_commands.py new file mode 100644 index 000000000..f67008877 --- /dev/null +++ b/env/lib/python3.8/site-packages/isort/setuptools_commands.py @@ -0,0 +1,61 @@ +import glob +import os +import sys +from typing import Any, Dict, Iterator, List +from warnings import warn + +import setuptools + +from . import api +from .settings import DEFAULT_CONFIG + + +class ISortCommand(setuptools.Command): + """The :class:`ISortCommand` class is used by setuptools to perform + imports checks on registered modules. + """ + + description = "Run isort on modules registered in setuptools" + user_options: List[Any] = [] + + def initialize_options(self) -> None: + default_settings = vars(DEFAULT_CONFIG).copy() + for key, value in default_settings.items(): + setattr(self, key, value) + + def finalize_options(self) -> None: + "Get options from config files." + self.arguments: Dict[str, Any] = {} # skipcq: PYL-W0201 + self.arguments["settings_path"] = os.getcwd() + + def distribution_files(self) -> Iterator[str]: + """Find distribution packages.""" + # This is verbatim from flake8 + if self.distribution.packages: + package_dirs = self.distribution.package_dir or {} + for package in self.distribution.packages: + pkg_dir = package + if package in package_dirs: + pkg_dir = package_dirs[package] + elif "" in package_dirs: + pkg_dir = package_dirs[""] + os.path.sep + pkg_dir + yield pkg_dir.replace(".", os.path.sep) + + if self.distribution.py_modules: + for filename in self.distribution.py_modules: + yield "%s.py" % filename + # Don't miss the setup.py file itself + yield "setup.py" + + def run(self) -> None: + arguments = self.arguments + wrong_sorted_files = False + for path in self.distribution_files(): + for python_file in glob.iglob(os.path.join(path, "*.py")): + try: + if not api.check_file(python_file, **arguments): + wrong_sorted_files = True # pragma: no cover + except OSError as error: # pragma: no cover + warn(f"Unable to parse file {python_file} due to {error}") + if wrong_sorted_files: + sys.exit(1) # pragma: no cover diff --git a/env/lib/python3.8/site-packages/isort/sorting.py b/env/lib/python3.8/site-packages/isort/sorting.py new file mode 100644 index 000000000..780747a3d --- /dev/null +++ b/env/lib/python3.8/site-packages/isort/sorting.py @@ -0,0 +1,99 @@ +import re +from typing import Any, Callable, Iterable, List, Optional + +from .settings import Config + +_import_line_intro_re = re.compile("^(?:from|import) ") +_import_line_midline_import_re = re.compile(" import ") + + +def module_key( + module_name: str, + config: Config, + sub_imports: bool = False, + ignore_case: bool = False, + section_name: Optional[Any] = None, + straight_import: Optional[bool] = False, +) -> str: + match = re.match(r"^(\.+)\s*(.*)", module_name) + if match: + sep = " " if config.reverse_relative else "_" + module_name = sep.join(match.groups()) + + prefix = "" + if ignore_case: + module_name = str(module_name).lower() + else: + module_name = str(module_name) + + if sub_imports and config.order_by_type: + if module_name in config.constants: + prefix = "A" + elif module_name in config.classes: + prefix = "B" + elif module_name in config.variables: + prefix = "C" + elif module_name.isupper() and len(module_name) > 1: # see issue #376 + prefix = "A" + elif module_name in config.classes or module_name[0:1].isupper(): + prefix = "B" + else: + prefix = "C" + if not config.case_sensitive: + module_name = module_name.lower() + + length_sort = ( + config.length_sort + or (config.length_sort_straight and straight_import) + or str(section_name).lower() in config.length_sort_sections + ) + _length_sort_maybe = length_sort and (str(len(module_name)) + ":" + module_name) or module_name + return f"{module_name in config.force_to_top and 'A' or 'B'}{prefix}{_length_sort_maybe}" + + +def section_key( + line: str, + order_by_type: bool, + force_to_top: List[str], + lexicographical: bool = False, + length_sort: bool = False, + reverse_relative: bool = False, +) -> str: + section = "B" + + if reverse_relative and line.startswith("from ."): + match = re.match(r"^from (\.+)\s*(.*)", line) + if match: + line = f"from {' '.join(match.groups())}" + + if lexicographical: + line = _import_line_intro_re.sub("", _import_line_midline_import_re.sub(".", line)) + else: + line = re.sub("^from ", "", line) + line = re.sub("^import ", "", line) + if line.split(" ")[0] in force_to_top: + section = "A" + if not order_by_type: + line = line.lower() + + return f"{section}{len(line) if length_sort else ''}{line}" + + +def naturally(to_sort: Iterable[str], key: Optional[Callable[[str], Any]] = None) -> List[str]: + """Returns a naturally sorted list""" + if key is None: + key_callback = _natural_keys + else: + + def key_callback(text: str) -> List[Any]: + return _natural_keys(key(text)) # type: ignore + + return sorted(to_sort, key=key_callback) + + +def _atoi(text: str) -> Any: + return int(text) if text.isdigit() else text + + +def _natural_keys(text: str) -> List[Any]: + return [_atoi(c) for c in re.split(r"(\d+)", text)] diff --git a/env/lib/python3.8/site-packages/isort/stdlibs/__init__.py b/env/lib/python3.8/site-packages/isort/stdlibs/__init__.py new file mode 100644 index 000000000..9021bc455 --- /dev/null +++ b/env/lib/python3.8/site-packages/isort/stdlibs/__init__.py @@ -0,0 +1 @@ +from . import all, py2, py3, py27, py35, py36, py37, py38, py39 diff --git a/env/lib/python3.8/site-packages/isort/stdlibs/all.py b/env/lib/python3.8/site-packages/isort/stdlibs/all.py new file mode 100644 index 000000000..08a365e19 --- /dev/null +++ b/env/lib/python3.8/site-packages/isort/stdlibs/all.py @@ -0,0 +1,3 @@ +from . import py2, py3 + +stdlib = py2.stdlib | py3.stdlib diff --git a/env/lib/python3.8/site-packages/isort/stdlibs/py2.py b/env/lib/python3.8/site-packages/isort/stdlibs/py2.py new file mode 100644 index 000000000..74af019e4 --- /dev/null +++ b/env/lib/python3.8/site-packages/isort/stdlibs/py2.py @@ -0,0 +1,3 @@ +from . import py27 + +stdlib = py27.stdlib diff --git a/env/lib/python3.8/site-packages/isort/stdlibs/py27.py b/env/lib/python3.8/site-packages/isort/stdlibs/py27.py new file mode 100644 index 000000000..87aa67f1f --- /dev/null +++ b/env/lib/python3.8/site-packages/isort/stdlibs/py27.py @@ -0,0 +1,300 @@ +""" +File contains the standard library of Python 2.7. + +DO NOT EDIT. If the standard library changes, a new list should be created +using the mkstdlibs.py script. +""" + +stdlib = { + "AL", + "BaseHTTPServer", + "Bastion", + "CGIHTTPServer", + "Carbon", + "ColorPicker", + "ConfigParser", + "Cookie", + "DEVICE", + "DocXMLRPCServer", + "EasyDialogs", + "FL", + "FrameWork", + "GL", + "HTMLParser", + "MacOS", + "MimeWriter", + "MiniAEFrame", + "Nav", + "PixMapWrapper", + "Queue", + "SUNAUDIODEV", + "ScrolledText", + "SimpleHTTPServer", + "SimpleXMLRPCServer", + "SocketServer", + "StringIO", + "Tix", + "Tkinter", + "UserDict", + "UserList", + "UserString", + "W", + "__builtin__", + "_winreg", + "abc", + "aepack", + "aetools", + "aetypes", + "aifc", + "al", + "anydbm", + "applesingle", + "argparse", + "array", + "ast", + "asynchat", + "asyncore", + "atexit", + "audioop", + "autoGIL", + "base64", + "bdb", + "binascii", + "binhex", + "bisect", + "bsddb", + "buildtools", + "bz2", + "cPickle", + "cProfile", + "cStringIO", + "calendar", + "cd", + "cfmfile", + "cgi", + "cgitb", + "chunk", + "cmath", + "cmd", + "code", + "codecs", + "codeop", + "collections", + "colorsys", + "commands", + "compileall", + "compiler", + "contextlib", + "cookielib", + "copy", + "copy_reg", + "crypt", + "csv", + "ctypes", + "curses", + "datetime", + "dbhash", + "dbm", + "decimal", + "difflib", + "dircache", + "dis", + "distutils", + "dl", + "doctest", + "dumbdbm", + "dummy_thread", + "dummy_threading", + "email", + "encodings", + "ensurepip", + "errno", + "exceptions", + "fcntl", + "filecmp", + "fileinput", + "findertools", + "fl", + "flp", + "fm", + "fnmatch", + "formatter", + "fpectl", + "fpformat", + "fractions", + "ftplib", + "functools", + "future_builtins", + "gc", + "gdbm", + "gensuitemodule", + "getopt", + "getpass", + "gettext", + "gl", + "glob", + "grp", + "gzip", + "hashlib", + "heapq", + "hmac", + "hotshot", + "htmlentitydefs", + "htmllib", + "httplib", + "ic", + "icopen", + "imageop", + "imaplib", + "imgfile", + "imghdr", + "imp", + "importlib", + "imputil", + "inspect", + "io", + "itertools", + "jpeg", + "json", + "keyword", + "lib2to3", + "linecache", + "locale", + "logging", + "macerrors", + "macostools", + "macpath", + "macresource", + "mailbox", + "mailcap", + "marshal", + "math", + "md5", + "mhlib", + "mimetools", + "mimetypes", + "mimify", + "mmap", + "modulefinder", + "msilib", + "msvcrt", + "multifile", + "multiprocessing", + "mutex", + "netrc", + "new", + "nis", + "nntplib", + "ntpath", + "numbers", + "operator", + "optparse", + "os", + "ossaudiodev", + "parser", + "pdb", + "pickle", + "pickletools", + "pipes", + "pkgutil", + "platform", + "plistlib", + "popen2", + "poplib", + "posix", + "posixfile", + "posixpath", + "pprint", + "profile", + "pstats", + "pty", + "pwd", + "py_compile", + "pyclbr", + "pydoc", + "quopri", + "random", + "re", + "readline", + "resource", + "rexec", + "rfc822", + "rlcompleter", + "robotparser", + "runpy", + "sched", + "select", + "sets", + "sgmllib", + "sha", + "shelve", + "shlex", + "shutil", + "signal", + "site", + "smtpd", + "smtplib", + "sndhdr", + "socket", + "spwd", + "sqlite3", + "sre", + "sre_compile", + "sre_constants", + "sre_parse", + "ssl", + "stat", + "statvfs", + "string", + "stringprep", + "struct", + "subprocess", + "sunau", + "sunaudiodev", + "symbol", + "symtable", + "sys", + "sysconfig", + "syslog", + "tabnanny", + "tarfile", + "telnetlib", + "tempfile", + "termios", + "test", + "textwrap", + "thread", + "threading", + "time", + "timeit", + "token", + "tokenize", + "trace", + "traceback", + "ttk", + "tty", + "turtle", + "types", + "unicodedata", + "unittest", + "urllib", + "urllib2", + "urlparse", + "user", + "uu", + "uuid", + "videoreader", + "warnings", + "wave", + "weakref", + "webbrowser", + "whichdb", + "winsound", + "wsgiref", + "xdrlib", + "xml", + "xmlrpclib", + "zipfile", + "zipimport", + "zlib", +} diff --git a/env/lib/python3.8/site-packages/isort/stdlibs/py3.py b/env/lib/python3.8/site-packages/isort/stdlibs/py3.py new file mode 100644 index 000000000..78e0984d5 --- /dev/null +++ b/env/lib/python3.8/site-packages/isort/stdlibs/py3.py @@ -0,0 +1,3 @@ +from . import py35, py36, py37, py38 + +stdlib = py35.stdlib | py36.stdlib | py37.stdlib | py38.stdlib diff --git a/env/lib/python3.8/site-packages/isort/stdlibs/py35.py b/env/lib/python3.8/site-packages/isort/stdlibs/py35.py new file mode 100644 index 000000000..274d8a7d1 --- /dev/null +++ b/env/lib/python3.8/site-packages/isort/stdlibs/py35.py @@ -0,0 +1,222 @@ +""" +File contains the standard library of Python 3.5. + +DO NOT EDIT. If the standard library changes, a new list should be created +using the mkstdlibs.py script. +""" + +stdlib = { + "_dummy_thread", + "_thread", + "abc", + "aifc", + "argparse", + "array", + "ast", + "asynchat", + "asyncio", + "asyncore", + "atexit", + "audioop", + "base64", + "bdb", + "binascii", + "binhex", + "bisect", + "builtins", + "bz2", + "cProfile", + "calendar", + "cgi", + "cgitb", + "chunk", + "cmath", + "cmd", + "code", + "codecs", + "codeop", + "collections", + "colorsys", + "compileall", + "concurrent", + "configparser", + "contextlib", + "copy", + "copyreg", + "crypt", + "csv", + "ctypes", + "curses", + "datetime", + "dbm", + "decimal", + "difflib", + "dis", + "distutils", + "doctest", + "dummy_threading", + "email", + "encodings", + "ensurepip", + "enum", + "errno", + "faulthandler", + "fcntl", + "filecmp", + "fileinput", + "fnmatch", + "formatter", + "fpectl", + "fractions", + "ftplib", + "functools", + "gc", + "getopt", + "getpass", + "gettext", + "glob", + "grp", + "gzip", + "hashlib", + "heapq", + "hmac", + "html", + "http", + "imaplib", + "imghdr", + "imp", + "importlib", + "inspect", + "io", + "ipaddress", + "itertools", + "json", + "keyword", + "lib2to3", + "linecache", + "locale", + "logging", + "lzma", + "macpath", + "mailbox", + "mailcap", + "marshal", + "math", + "mimetypes", + "mmap", + "modulefinder", + "msilib", + "msvcrt", + "multiprocessing", + "netrc", + "nis", + "nntplib", + "ntpath", + "numbers", + "operator", + "optparse", + "os", + "ossaudiodev", + "parser", + "pathlib", + "pdb", + "pickle", + "pickletools", + "pipes", + "pkgutil", + "platform", + "plistlib", + "poplib", + "posix", + "posixpath", + "pprint", + "profile", + "pstats", + "pty", + "pwd", + "py_compile", + "pyclbr", + "pydoc", + "queue", + "quopri", + "random", + "re", + "readline", + "reprlib", + "resource", + "rlcompleter", + "runpy", + "sched", + "select", + "selectors", + "shelve", + "shlex", + "shutil", + "signal", + "site", + "smtpd", + "smtplib", + "sndhdr", + "socket", + "socketserver", + "spwd", + "sqlite3", + "sre", + "sre_compile", + "sre_constants", + "sre_parse", + "ssl", + "stat", + "statistics", + "string", + "stringprep", + "struct", + "subprocess", + "sunau", + "symbol", + "symtable", + "sys", + "sysconfig", + "syslog", + "tabnanny", + "tarfile", + "telnetlib", + "tempfile", + "termios", + "test", + "textwrap", + "threading", + "time", + "timeit", + "tkinter", + "token", + "tokenize", + "trace", + "traceback", + "tracemalloc", + "tty", + "turtle", + "turtledemo", + "types", + "typing", + "unicodedata", + "unittest", + "urllib", + "uu", + "uuid", + "venv", + "warnings", + "wave", + "weakref", + "webbrowser", + "winreg", + "winsound", + "wsgiref", + "xdrlib", + "xml", + "xmlrpc", + "zipapp", + "zipfile", + "zipimport", + "zlib", +} diff --git a/env/lib/python3.8/site-packages/isort/stdlibs/py36.py b/env/lib/python3.8/site-packages/isort/stdlibs/py36.py new file mode 100644 index 000000000..8ae02a150 --- /dev/null +++ b/env/lib/python3.8/site-packages/isort/stdlibs/py36.py @@ -0,0 +1,223 @@ +""" +File contains the standard library of Python 3.6. + +DO NOT EDIT. If the standard library changes, a new list should be created +using the mkstdlibs.py script. +""" + +stdlib = { + "_dummy_thread", + "_thread", + "abc", + "aifc", + "argparse", + "array", + "ast", + "asynchat", + "asyncio", + "asyncore", + "atexit", + "audioop", + "base64", + "bdb", + "binascii", + "binhex", + "bisect", + "builtins", + "bz2", + "cProfile", + "calendar", + "cgi", + "cgitb", + "chunk", + "cmath", + "cmd", + "code", + "codecs", + "codeop", + "collections", + "colorsys", + "compileall", + "concurrent", + "configparser", + "contextlib", + "copy", + "copyreg", + "crypt", + "csv", + "ctypes", + "curses", + "datetime", + "dbm", + "decimal", + "difflib", + "dis", + "distutils", + "doctest", + "dummy_threading", + "email", + "encodings", + "ensurepip", + "enum", + "errno", + "faulthandler", + "fcntl", + "filecmp", + "fileinput", + "fnmatch", + "formatter", + "fpectl", + "fractions", + "ftplib", + "functools", + "gc", + "getopt", + "getpass", + "gettext", + "glob", + "grp", + "gzip", + "hashlib", + "heapq", + "hmac", + "html", + "http", + "imaplib", + "imghdr", + "imp", + "importlib", + "inspect", + "io", + "ipaddress", + "itertools", + "json", + "keyword", + "lib2to3", + "linecache", + "locale", + "logging", + "lzma", + "macpath", + "mailbox", + "mailcap", + "marshal", + "math", + "mimetypes", + "mmap", + "modulefinder", + "msilib", + "msvcrt", + "multiprocessing", + "netrc", + "nis", + "nntplib", + "ntpath", + "numbers", + "operator", + "optparse", + "os", + "ossaudiodev", + "parser", + "pathlib", + "pdb", + "pickle", + "pickletools", + "pipes", + "pkgutil", + "platform", + "plistlib", + "poplib", + "posix", + "posixpath", + "pprint", + "profile", + "pstats", + "pty", + "pwd", + "py_compile", + "pyclbr", + "pydoc", + "queue", + "quopri", + "random", + "re", + "readline", + "reprlib", + "resource", + "rlcompleter", + "runpy", + "sched", + "secrets", + "select", + "selectors", + "shelve", + "shlex", + "shutil", + "signal", + "site", + "smtpd", + "smtplib", + "sndhdr", + "socket", + "socketserver", + "spwd", + "sqlite3", + "sre", + "sre_compile", + "sre_constants", + "sre_parse", + "ssl", + "stat", + "statistics", + "string", + "stringprep", + "struct", + "subprocess", + "sunau", + "symbol", + "symtable", + "sys", + "sysconfig", + "syslog", + "tabnanny", + "tarfile", + "telnetlib", + "tempfile", + "termios", + "test", + "textwrap", + "threading", + "time", + "timeit", + "tkinter", + "token", + "tokenize", + "trace", + "traceback", + "tracemalloc", + "tty", + "turtle", + "turtledemo", + "types", + "typing", + "unicodedata", + "unittest", + "urllib", + "uu", + "uuid", + "venv", + "warnings", + "wave", + "weakref", + "webbrowser", + "winreg", + "winsound", + "wsgiref", + "xdrlib", + "xml", + "xmlrpc", + "zipapp", + "zipfile", + "zipimport", + "zlib", +} diff --git a/env/lib/python3.8/site-packages/isort/stdlibs/py37.py b/env/lib/python3.8/site-packages/isort/stdlibs/py37.py new file mode 100644 index 000000000..0eb1dd6fa --- /dev/null +++ b/env/lib/python3.8/site-packages/isort/stdlibs/py37.py @@ -0,0 +1,224 @@ +""" +File contains the standard library of Python 3.7. + +DO NOT EDIT. If the standard library changes, a new list should be created +using the mkstdlibs.py script. +""" + +stdlib = { + "_dummy_thread", + "_thread", + "abc", + "aifc", + "argparse", + "array", + "ast", + "asynchat", + "asyncio", + "asyncore", + "atexit", + "audioop", + "base64", + "bdb", + "binascii", + "binhex", + "bisect", + "builtins", + "bz2", + "cProfile", + "calendar", + "cgi", + "cgitb", + "chunk", + "cmath", + "cmd", + "code", + "codecs", + "codeop", + "collections", + "colorsys", + "compileall", + "concurrent", + "configparser", + "contextlib", + "contextvars", + "copy", + "copyreg", + "crypt", + "csv", + "ctypes", + "curses", + "dataclasses", + "datetime", + "dbm", + "decimal", + "difflib", + "dis", + "distutils", + "doctest", + "dummy_threading", + "email", + "encodings", + "ensurepip", + "enum", + "errno", + "faulthandler", + "fcntl", + "filecmp", + "fileinput", + "fnmatch", + "formatter", + "fractions", + "ftplib", + "functools", + "gc", + "getopt", + "getpass", + "gettext", + "glob", + "grp", + "gzip", + "hashlib", + "heapq", + "hmac", + "html", + "http", + "imaplib", + "imghdr", + "imp", + "importlib", + "inspect", + "io", + "ipaddress", + "itertools", + "json", + "keyword", + "lib2to3", + "linecache", + "locale", + "logging", + "lzma", + "macpath", + "mailbox", + "mailcap", + "marshal", + "math", + "mimetypes", + "mmap", + "modulefinder", + "msilib", + "msvcrt", + "multiprocessing", + "netrc", + "nis", + "nntplib", + "ntpath", + "numbers", + "operator", + "optparse", + "os", + "ossaudiodev", + "parser", + "pathlib", + "pdb", + "pickle", + "pickletools", + "pipes", + "pkgutil", + "platform", + "plistlib", + "poplib", + "posix", + "posixpath", + "pprint", + "profile", + "pstats", + "pty", + "pwd", + "py_compile", + "pyclbr", + "pydoc", + "queue", + "quopri", + "random", + "re", + "readline", + "reprlib", + "resource", + "rlcompleter", + "runpy", + "sched", + "secrets", + "select", + "selectors", + "shelve", + "shlex", + "shutil", + "signal", + "site", + "smtpd", + "smtplib", + "sndhdr", + "socket", + "socketserver", + "spwd", + "sqlite3", + "sre", + "sre_compile", + "sre_constants", + "sre_parse", + "ssl", + "stat", + "statistics", + "string", + "stringprep", + "struct", + "subprocess", + "sunau", + "symbol", + "symtable", + "sys", + "sysconfig", + "syslog", + "tabnanny", + "tarfile", + "telnetlib", + "tempfile", + "termios", + "test", + "textwrap", + "threading", + "time", + "timeit", + "tkinter", + "token", + "tokenize", + "trace", + "traceback", + "tracemalloc", + "tty", + "turtle", + "turtledemo", + "types", + "typing", + "unicodedata", + "unittest", + "urllib", + "uu", + "uuid", + "venv", + "warnings", + "wave", + "weakref", + "webbrowser", + "winreg", + "winsound", + "wsgiref", + "xdrlib", + "xml", + "xmlrpc", + "zipapp", + "zipfile", + "zipimport", + "zlib", +} diff --git a/env/lib/python3.8/site-packages/isort/stdlibs/py38.py b/env/lib/python3.8/site-packages/isort/stdlibs/py38.py new file mode 100644 index 000000000..9bcea9a16 --- /dev/null +++ b/env/lib/python3.8/site-packages/isort/stdlibs/py38.py @@ -0,0 +1,223 @@ +""" +File contains the standard library of Python 3.8. + +DO NOT EDIT. If the standard library changes, a new list should be created +using the mkstdlibs.py script. +""" + +stdlib = { + "_dummy_thread", + "_thread", + "abc", + "aifc", + "argparse", + "array", + "ast", + "asynchat", + "asyncio", + "asyncore", + "atexit", + "audioop", + "base64", + "bdb", + "binascii", + "binhex", + "bisect", + "builtins", + "bz2", + "cProfile", + "calendar", + "cgi", + "cgitb", + "chunk", + "cmath", + "cmd", + "code", + "codecs", + "codeop", + "collections", + "colorsys", + "compileall", + "concurrent", + "configparser", + "contextlib", + "contextvars", + "copy", + "copyreg", + "crypt", + "csv", + "ctypes", + "curses", + "dataclasses", + "datetime", + "dbm", + "decimal", + "difflib", + "dis", + "distutils", + "doctest", + "dummy_threading", + "email", + "encodings", + "ensurepip", + "enum", + "errno", + "faulthandler", + "fcntl", + "filecmp", + "fileinput", + "fnmatch", + "formatter", + "fractions", + "ftplib", + "functools", + "gc", + "getopt", + "getpass", + "gettext", + "glob", + "grp", + "gzip", + "hashlib", + "heapq", + "hmac", + "html", + "http", + "imaplib", + "imghdr", + "imp", + "importlib", + "inspect", + "io", + "ipaddress", + "itertools", + "json", + "keyword", + "lib2to3", + "linecache", + "locale", + "logging", + "lzma", + "mailbox", + "mailcap", + "marshal", + "math", + "mimetypes", + "mmap", + "modulefinder", + "msilib", + "msvcrt", + "multiprocessing", + "netrc", + "nis", + "nntplib", + "ntpath", + "numbers", + "operator", + "optparse", + "os", + "ossaudiodev", + "parser", + "pathlib", + "pdb", + "pickle", + "pickletools", + "pipes", + "pkgutil", + "platform", + "plistlib", + "poplib", + "posix", + "posixpath", + "pprint", + "profile", + "pstats", + "pty", + "pwd", + "py_compile", + "pyclbr", + "pydoc", + "queue", + "quopri", + "random", + "re", + "readline", + "reprlib", + "resource", + "rlcompleter", + "runpy", + "sched", + "secrets", + "select", + "selectors", + "shelve", + "shlex", + "shutil", + "signal", + "site", + "smtpd", + "smtplib", + "sndhdr", + "socket", + "socketserver", + "spwd", + "sqlite3", + "sre", + "sre_compile", + "sre_constants", + "sre_parse", + "ssl", + "stat", + "statistics", + "string", + "stringprep", + "struct", + "subprocess", + "sunau", + "symbol", + "symtable", + "sys", + "sysconfig", + "syslog", + "tabnanny", + "tarfile", + "telnetlib", + "tempfile", + "termios", + "test", + "textwrap", + "threading", + "time", + "timeit", + "tkinter", + "token", + "tokenize", + "trace", + "traceback", + "tracemalloc", + "tty", + "turtle", + "turtledemo", + "types", + "typing", + "unicodedata", + "unittest", + "urllib", + "uu", + "uuid", + "venv", + "warnings", + "wave", + "weakref", + "webbrowser", + "winreg", + "winsound", + "wsgiref", + "xdrlib", + "xml", + "xmlrpc", + "zipapp", + "zipfile", + "zipimport", + "zlib", +} diff --git a/env/lib/python3.8/site-packages/isort/stdlibs/py39.py b/env/lib/python3.8/site-packages/isort/stdlibs/py39.py new file mode 100644 index 000000000..7bcb8f2b7 --- /dev/null +++ b/env/lib/python3.8/site-packages/isort/stdlibs/py39.py @@ -0,0 +1,223 @@ +""" +File contains the standard library of Python 3.9. + +DO NOT EDIT. If the standard library changes, a new list should be created +using the mkstdlibs.py script. +""" + +stdlib = { + "_thread", + "abc", + "aifc", + "argparse", + "array", + "ast", + "asynchat", + "asyncio", + "asyncore", + "atexit", + "audioop", + "base64", + "bdb", + "binascii", + "binhex", + "bisect", + "builtins", + "bz2", + "cProfile", + "calendar", + "cgi", + "cgitb", + "chunk", + "cmath", + "cmd", + "code", + "codecs", + "codeop", + "collections", + "colorsys", + "compileall", + "concurrent", + "configparser", + "contextlib", + "contextvars", + "copy", + "copyreg", + "crypt", + "csv", + "ctypes", + "curses", + "dataclasses", + "datetime", + "dbm", + "decimal", + "difflib", + "dis", + "distutils", + "doctest", + "email", + "encodings", + "ensurepip", + "enum", + "errno", + "faulthandler", + "fcntl", + "filecmp", + "fileinput", + "fnmatch", + "formatter", + "fractions", + "ftplib", + "functools", + "gc", + "getopt", + "getpass", + "gettext", + "glob", + "graphlib", + "grp", + "gzip", + "hashlib", + "heapq", + "hmac", + "html", + "http", + "imaplib", + "imghdr", + "imp", + "importlib", + "inspect", + "io", + "ipaddress", + "itertools", + "json", + "keyword", + "lib2to3", + "linecache", + "locale", + "logging", + "lzma", + "mailbox", + "mailcap", + "marshal", + "math", + "mimetypes", + "mmap", + "modulefinder", + "msilib", + "msvcrt", + "multiprocessing", + "netrc", + "nis", + "nntplib", + "ntpath", + "numbers", + "operator", + "optparse", + "os", + "ossaudiodev", + "parser", + "pathlib", + "pdb", + "pickle", + "pickletools", + "pipes", + "pkgutil", + "platform", + "plistlib", + "poplib", + "posix", + "posixpath", + "pprint", + "profile", + "pstats", + "pty", + "pwd", + "py_compile", + "pyclbr", + "pydoc", + "queue", + "quopri", + "random", + "re", + "readline", + "reprlib", + "resource", + "rlcompleter", + "runpy", + "sched", + "secrets", + "select", + "selectors", + "shelve", + "shlex", + "shutil", + "signal", + "site", + "smtpd", + "smtplib", + "sndhdr", + "socket", + "socketserver", + "spwd", + "sqlite3", + "sre", + "sre_compile", + "sre_constants", + "sre_parse", + "ssl", + "stat", + "statistics", + "string", + "stringprep", + "struct", + "subprocess", + "sunau", + "symbol", + "symtable", + "sys", + "sysconfig", + "syslog", + "tabnanny", + "tarfile", + "telnetlib", + "tempfile", + "termios", + "test", + "textwrap", + "threading", + "time", + "timeit", + "tkinter", + "token", + "tokenize", + "trace", + "traceback", + "tracemalloc", + "tty", + "turtle", + "turtledemo", + "types", + "typing", + "unicodedata", + "unittest", + "urllib", + "uu", + "uuid", + "venv", + "warnings", + "wave", + "weakref", + "webbrowser", + "winreg", + "winsound", + "wsgiref", + "xdrlib", + "xml", + "xmlrpc", + "zipapp", + "zipfile", + "zipimport", + "zlib", + "zoneinfo", +} diff --git a/env/lib/python3.8/site-packages/isort/utils.py b/env/lib/python3.8/site-packages/isort/utils.py new file mode 100644 index 000000000..27f17b4a5 --- /dev/null +++ b/env/lib/python3.8/site-packages/isort/utils.py @@ -0,0 +1,29 @@ +import os +import sys +from contextlib import contextmanager +from typing import Iterator + + +def exists_case_sensitive(path: str) -> bool: + """Returns if the given path exists and also matches the case on Windows. + + When finding files that can be imported, it is important for the cases to match because while + file os.path.exists("module.py") and os.path.exists("MODULE.py") both return True on Windows, + Python can only import using the case of the real file. + """ + result = os.path.exists(path) + if (sys.platform.startswith("win") or sys.platform == "darwin") and result: # pragma: no cover + directory, basename = os.path.split(path) + result = basename in os.listdir(directory) + return result + + +@contextmanager +def chdir(path: str) -> Iterator[None]: + """Context manager for changing dir and restoring previous workdir after exit.""" + curdir = os.getcwd() + os.chdir(path) + try: + yield + finally: + os.chdir(curdir) diff --git a/env/lib/python3.8/site-packages/isort/wrap.py b/env/lib/python3.8/site-packages/isort/wrap.py new file mode 100644 index 000000000..11542fa07 --- /dev/null +++ b/env/lib/python3.8/site-packages/isort/wrap.py @@ -0,0 +1,131 @@ +import copy +import re +from typing import List, Optional, Sequence + +from .settings import DEFAULT_CONFIG, Config +from .wrap_modes import WrapModes as Modes +from .wrap_modes import formatter_from_string + + +def import_statement( + import_start: str, + from_imports: List[str], + comments: Sequence[str] = (), + line_separator: str = "\n", + config: Config = DEFAULT_CONFIG, + multi_line_output: Optional[Modes] = None, +) -> str: + """Returns a multi-line wrapped form of the provided from import statement.""" + formatter = formatter_from_string((multi_line_output or config.multi_line_output).name) + dynamic_indent = " " * (len(import_start) + 1) + indent = config.indent + line_length = config.wrap_length or config.line_length + statement = formatter( + statement=import_start, + imports=copy.copy(from_imports), + white_space=dynamic_indent, + indent=indent, + line_length=line_length, + comments=comments, + line_separator=line_separator, + comment_prefix=config.comment_prefix, + include_trailing_comma=config.include_trailing_comma, + remove_comments=config.ignore_comments, + ) + if config.balanced_wrapping: + lines = statement.split(line_separator) + line_count = len(lines) + if len(lines) > 1: + minimum_length = min(len(line) for line in lines[:-1]) + else: + minimum_length = 0 + new_import_statement = statement + while len(lines[-1]) < minimum_length and len(lines) == line_count and line_length > 10: + statement = new_import_statement + line_length -= 1 + new_import_statement = formatter( + statement=import_start, + imports=copy.copy(from_imports), + white_space=dynamic_indent, + indent=indent, + line_length=line_length, + comments=comments, + line_separator=line_separator, + comment_prefix=config.comment_prefix, + include_trailing_comma=config.include_trailing_comma, + remove_comments=config.ignore_comments, + ) + lines = new_import_statement.split(line_separator) + if statement.count(line_separator) == 0: + return _wrap_line(statement, line_separator, config) + return statement + + +def line(content: str, line_separator: str, config: Config = DEFAULT_CONFIG) -> str: + """Returns a line wrapped to the specified line-length, if possible.""" + wrap_mode = config.multi_line_output + if len(content) > config.line_length and wrap_mode != Modes.NOQA: # type: ignore + line_without_comment = content + comment = None + if "#" in content: + line_without_comment, comment = content.split("#", 1) + for splitter in ("import ", ".", "as "): + exp = r"\b" + re.escape(splitter) + r"\b" + if re.search(exp, line_without_comment) and not line_without_comment.strip().startswith( + splitter + ): + line_parts = re.split(exp, line_without_comment) + if comment and not (config.use_parentheses and "noqa" in comment): + _comma_maybe = ( + "," if (config.include_trailing_comma and config.use_parentheses) else "" + ) + line_parts[ + -1 + ] = f"{line_parts[-1].strip()}{_comma_maybe}{config.comment_prefix}{comment}" + next_line = [] + while (len(content) + 2) > ( + config.wrap_length or config.line_length + ) and line_parts: + next_line.append(line_parts.pop()) + content = splitter.join(line_parts) + if not content: + content = next_line.pop() + + cont_line = _wrap_line( + config.indent + splitter.join(next_line).lstrip(), line_separator, config + ) + if config.use_parentheses: + if splitter == "as ": + output = f"{content}{splitter}{cont_line.lstrip()}" + else: + _comma = "," if config.include_trailing_comma and not comment else "" + if wrap_mode in ( + Modes.VERTICAL_HANGING_INDENT, # type: ignore + Modes.VERTICAL_GRID_GROUPED, # type: ignore + ): + _separator = line_separator + else: + _separator = "" + _comment = "" + if comment and "noqa" in comment: + _comment = f"{config.comment_prefix}{comment}" + cont_line = cont_line.rstrip() + _comma = "," if config.include_trailing_comma else "" + output = ( + f"{content}{splitter}({_comment}" + f"{line_separator}{cont_line}{_comma}{_separator})" + ) + lines = output.split(line_separator) + if config.comment_prefix in lines[-1] and lines[-1].endswith(")"): + content, comment = lines[-1].split(config.comment_prefix, 1) + lines[-1] = content + ")" + config.comment_prefix + comment[:-1] + return line_separator.join(lines) + return f"{content}{splitter}\\{line_separator}{cont_line}" + elif len(content) > config.line_length and wrap_mode == Modes.NOQA: # type: ignore + if "# NOQA" not in content: + return f"{content}{config.comment_prefix} NOQA" + + return content + + +_wrap_line = line diff --git a/env/lib/python3.8/site-packages/isort/wrap_modes.py b/env/lib/python3.8/site-packages/isort/wrap_modes.py new file mode 100644 index 000000000..ccfc1cd5a --- /dev/null +++ b/env/lib/python3.8/site-packages/isort/wrap_modes.py @@ -0,0 +1,317 @@ +"""Defines all wrap modes that can be used when outputting formatted imports""" +import enum +from inspect import signature +from typing import Any, Callable, Dict, List + +import isort.comments + +_wrap_modes: Dict[str, Callable[[Any], str]] = {} + + +def from_string(value: str) -> "WrapModes": + return getattr(WrapModes, str(value), None) or WrapModes(int(value)) + + +def formatter_from_string(name: str): + return _wrap_modes.get(name.upper(), grid) + + +def _wrap_mode_interface( + statement: str, + imports: List[str], + white_space: str, + indent: str, + line_length: int, + comments: List[str], + line_separator: str, + comment_prefix: str, + include_trailing_comma: bool, + remove_comments: bool, +) -> str: + """Defines the common interface used by all wrap mode functions""" + return "" + + +def _wrap_mode(function): + """Registers an individual wrap mode. Function name and order are significant and used for + creating enum. + """ + _wrap_modes[function.__name__.upper()] = function + function.__signature__ = signature(_wrap_mode_interface) + function.__annotations__ = _wrap_mode_interface.__annotations__ + return function + + +@_wrap_mode +def grid(**interface): + if not interface["imports"]: + return "" + + interface["statement"] += "(" + interface["imports"].pop(0) + while interface["imports"]: + next_import = interface["imports"].pop(0) + next_statement = isort.comments.add_to_line( + interface["comments"], + interface["statement"] + ", " + next_import, + removed=interface["remove_comments"], + comment_prefix=interface["comment_prefix"], + ) + if ( + len(next_statement.split(interface["line_separator"])[-1]) + 1 + > interface["line_length"] + ): + lines = [f"{interface['white_space']}{next_import.split(' ')[0]}"] + for part in next_import.split(" ")[1:]: + new_line = f"{lines[-1]} {part}" + if len(new_line) + 1 > interface["line_length"]: + lines.append(f"{interface['white_space']}{part}") + else: + lines[-1] = new_line + next_import = interface["line_separator"].join(lines) + interface["statement"] = ( + isort.comments.add_to_line( + interface["comments"], + f"{interface['statement']},", + removed=interface["remove_comments"], + comment_prefix=interface["comment_prefix"], + ) + + f"{interface['line_separator']}{next_import}" + ) + interface["comments"] = [] + else: + interface["statement"] += ", " + next_import + return interface["statement"] + ("," if interface["include_trailing_comma"] else "") + ")" + + +@_wrap_mode +def vertical(**interface): + if not interface["imports"]: + return "" + + first_import = ( + isort.comments.add_to_line( + interface["comments"], + interface["imports"].pop(0) + ",", + removed=interface["remove_comments"], + comment_prefix=interface["comment_prefix"], + ) + + interface["line_separator"] + + interface["white_space"] + ) + + _imports = ("," + interface["line_separator"] + interface["white_space"]).join( + interface["imports"] + ) + _comma_maybe = "," if interface["include_trailing_comma"] else "" + return f"{interface['statement']}({first_import}{_imports}{_comma_maybe})" + + +def _hanging_indent_common(use_parentheses=False, **interface): + if not interface["imports"]: + return "" + line_length_limit = interface["line_length"] - (1 if use_parentheses else 3) + + def end_line(line): + if use_parentheses: + return line + if not line.endswith(" "): + line += " " + return line + "\\" + + if use_parentheses: + interface["statement"] += "(" + next_import = interface["imports"].pop(0) + next_statement = interface["statement"] + next_import + # Check for first import + if len(next_statement) > line_length_limit: + next_statement = ( + isort.comments.add_to_line( + interface["comments"], + end_line(interface["statement"]), + removed=interface["remove_comments"], + comment_prefix=interface["comment_prefix"], + ) + + f"{interface['line_separator']}{interface['indent']}{next_import}" + ) + interface["comments"] = [] + interface["statement"] = next_statement + while interface["imports"]: + next_import = interface["imports"].pop(0) + next_statement = isort.comments.add_to_line( + interface["comments"], + interface["statement"] + ", " + next_import, + removed=interface["remove_comments"], + comment_prefix=interface["comment_prefix"], + ) + current_line = next_statement.split(interface["line_separator"])[-1] + if len(current_line) > line_length_limit: + next_statement = ( + isort.comments.add_to_line( + interface["comments"], + end_line(interface["statement"] + ","), + removed=interface["remove_comments"], + comment_prefix=interface["comment_prefix"], + ) + + f"{interface['line_separator']}{interface['indent']}{next_import}" + ) + interface["comments"] = [] + interface["statement"] = next_statement + _comma_maybe = "," if interface["include_trailing_comma"] else "" + _close_parentheses_maybe = ")" if use_parentheses else "" + return interface["statement"] + _comma_maybe + _close_parentheses_maybe + + +@_wrap_mode +def hanging_indent(**interface): + return _hanging_indent_common(use_parentheses=False, **interface) + + +@_wrap_mode +def vertical_hanging_indent(**interface): + _line_with_comments = isort.comments.add_to_line( + interface["comments"], + "", + removed=interface["remove_comments"], + comment_prefix=interface["comment_prefix"], + ) + _imports = ("," + interface["line_separator"] + interface["indent"]).join(interface["imports"]) + _comma_maybe = "," if interface["include_trailing_comma"] else "" + return ( + f"{interface['statement']}({_line_with_comments}{interface['line_separator']}" + f"{interface['indent']}{_imports}{_comma_maybe}{interface['line_separator']})" + ) + + +def _vertical_grid_common(need_trailing_char: bool, **interface): + if not interface["imports"]: + return "" + + interface["statement"] += ( + isort.comments.add_to_line( + interface["comments"], + "(", + removed=interface["remove_comments"], + comment_prefix=interface["comment_prefix"], + ) + + interface["line_separator"] + + interface["indent"] + + interface["imports"].pop(0) + ) + while interface["imports"]: + next_import = interface["imports"].pop(0) + next_statement = f"{interface['statement']}, {next_import}" + current_line_length = len(next_statement.split(interface["line_separator"])[-1]) + if interface["imports"] or need_trailing_char: + # If we have more interface["imports"] we need to account for a comma after this import + # We might also need to account for a closing ) we're going to add. + current_line_length += 1 + if current_line_length > interface["line_length"]: + next_statement = ( + f"{interface['statement']},{interface['line_separator']}" + f"{interface['indent']}{next_import}" + ) + interface["statement"] = next_statement + if interface["include_trailing_comma"]: + interface["statement"] += "," + return interface["statement"] + + +@_wrap_mode +def vertical_grid(**interface) -> str: + return _vertical_grid_common(need_trailing_char=True, **interface) + ")" + + +@_wrap_mode +def vertical_grid_grouped(**interface): + return ( + _vertical_grid_common(need_trailing_char=True, **interface) + + interface["line_separator"] + + ")" + ) + + +@_wrap_mode +def vertical_grid_grouped_no_comma(**interface): + return ( + _vertical_grid_common(need_trailing_char=False, **interface) + + interface["line_separator"] + + ")" + ) + + +@_wrap_mode +def noqa(**interface): + _imports = ", ".join(interface["imports"]) + retval = f"{interface['statement']}{_imports}" + comment_str = " ".join(interface["comments"]) + if interface["comments"]: + if ( + len(retval) + len(interface["comment_prefix"]) + 1 + len(comment_str) + <= interface["line_length"] + ): + return f"{retval}{interface['comment_prefix']} {comment_str}" + elif "NOQA" in interface["comments"]: + return f"{retval}{interface['comment_prefix']} {comment_str}" + else: + return f"{retval}{interface['comment_prefix']} NOQA {comment_str}" + else: + if len(retval) <= interface["line_length"]: + return retval + else: + return f"{retval}{interface['comment_prefix']} NOQA" + + +@_wrap_mode +def vertical_hanging_indent_bracket(**interface): + if not interface["imports"]: + return "" + statement = vertical_hanging_indent(**interface) + return f'{statement[:-1]}{interface["indent"]})' + + +@_wrap_mode +def vertical_prefix_from_module_import(**interface): + if not interface["imports"]: + return "" + prefix_statement = interface["statement"] + interface["statement"] += interface["imports"].pop(0) + while interface["imports"]: + next_import = interface["imports"].pop(0) + next_statement = isort.comments.add_to_line( + interface["comments"], + interface["statement"] + ", " + next_import, + removed=interface["remove_comments"], + comment_prefix=interface["comment_prefix"], + ) + if ( + len(next_statement.split(interface["line_separator"])[-1]) + 1 + > interface["line_length"] + ): + next_statement = ( + isort.comments.add_to_line( + interface["comments"], + f"{interface['statement']}", + removed=interface["remove_comments"], + comment_prefix=interface["comment_prefix"], + ) + + f"{interface['line_separator']}{prefix_statement}{next_import}" + ) + interface["comments"] = [] + interface["statement"] = next_statement + return interface["statement"] + + +@_wrap_mode +def hanging_indent_with_parentheses(**interface): + return _hanging_indent_common(use_parentheses=True, **interface) + + +@_wrap_mode +def backslash_grid(**interface): + interface["indent"] = interface["white_space"][:-1] + return _hanging_indent_common(use_parentheses=False, **interface) + + +WrapModes = enum.Enum( # type: ignore + "WrapModes", {wrap_mode: index for index, wrap_mode in enumerate(_wrap_modes.keys())} +) diff --git a/env/lib/python3.8/site-packages/lazy_object_proxy-1.4.3.dist-info/INSTALLER b/env/lib/python3.8/site-packages/lazy_object_proxy-1.4.3.dist-info/INSTALLER new file mode 100644 index 000000000..a1b589e38 --- /dev/null +++ b/env/lib/python3.8/site-packages/lazy_object_proxy-1.4.3.dist-info/INSTALLER @@ -0,0 +1 @@ +pip diff --git a/env/lib/python3.8/site-packages/lazy_object_proxy-1.4.3.dist-info/METADATA b/env/lib/python3.8/site-packages/lazy_object_proxy-1.4.3.dist-info/METADATA new file mode 100644 index 000000000..6b4b83021 --- /dev/null +++ b/env/lib/python3.8/site-packages/lazy_object_proxy-1.4.3.dist-info/METADATA @@ -0,0 +1,166 @@ +Metadata-Version: 2.1 +Name: lazy-object-proxy +Version: 1.4.3 +Summary: A fast and thorough lazy object proxy. +Home-page: https://github.com/ionelmc/python-lazy-object-proxy +Author: Ionel Cristian Mărieș +Author-email: contact@ionelmc.ro +License: BSD-2-Clause +Project-URL: Documentation, https://python-lazy-object-proxy.readthedocs.io/ +Project-URL: Changelog, https://python-lazy-object-proxy.readthedocs.io/en/latest/changelog.html +Project-URL: Issue Tracker, https://github.com/ionelmc/python-lazy-object-proxy/issues +Platform: UNKNOWN +Classifier: Development Status :: 5 - Production/Stable +Classifier: Intended Audience :: Developers +Classifier: License :: OSI Approved :: BSD License +Classifier: Operating System :: Unix +Classifier: Operating System :: POSIX +Classifier: Operating System :: Microsoft :: Windows +Classifier: Programming Language :: Python +Classifier: Programming Language :: Python :: 2.7 +Classifier: Programming Language :: Python :: 3 +Classifier: Programming Language :: Python :: 3.4 +Classifier: Programming Language :: Python :: 3.5 +Classifier: Programming Language :: Python :: 3.6 +Classifier: Programming Language :: Python :: 3.7 +Classifier: Programming Language :: Python :: 3.8 +Classifier: Programming Language :: Python :: Implementation :: CPython +Classifier: Programming Language :: Python :: Implementation :: PyPy +Classifier: Topic :: Utilities +Requires-Python: >=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.* + +======== +Overview +======== + + + +A fast and thorough lazy object proxy. + +* Free software: BSD 2-Clause License + +Note that this is based on `wrapt`_'s ObjectProxy with one big change: it calls a function the first time the proxy object is +used, while `wrapt.ObjectProxy` just forwards the method calls to the target object. + +In other words, you use `lazy-object-proxy` when you only have the object way later and you use `wrapt.ObjectProxy` when you +want to override few methods (by subclassing) and forward everything else to the target object. + +Example:: + + import lazy_object_proxy + + def expensive_func(): + from time import sleep + print('starting calculation') + # just as example for a very slow computation + sleep(2) + print('finished calculation') + # return the result of the calculation + return 10 + + obj = lazy_object_proxy.Proxy(expensive_func) + # function is called only when object is actually used + print(obj) # now expensive_func is called + + print(obj) # the result without calling the expensive_func + +Installation +============ + +:: + + pip install lazy-object-proxy + +Documentation +============= + +https://python-lazy-object-proxy.readthedocs.io/ + +Development +=========== + +To run the all tests run:: + + tox + +Acknowledgements +================ + +This project is based on some code from `wrapt`_ as you can see in the git history. + +.. _wrapt: https://github.com/GrahamDumpleton/wrapt + + +Changelog +========= + +1.4.3 (2019-10-26) +------------------ + +* Added binary wheels for Python 3.8. +* Fixed license metadata. + +1.4.2 (2019-08-22) +------------------ + +* Included a ``pyproject.toml`` to allow users install the sdist with old python/setuptools, as the + setuptools-scm dep will be fetched by pip instead of setuptools. + Fixes `#30 `_. + +1.4.1 (2019-05-10) +------------------ + +* Fixed wheels being built with ``-coverage`` cflags. No more issues about bogus ``cext.gcda`` files. +* Removed useless C file from wheels. +* Changed ``setup.py`` to use setuptools-scm. + +1.4.0 (2019-05-05) +------------------ + +* Fixed ``__mod__`` for the slots backend. Contributed by Ran Benita in + `#28 `_. +* Dropped support for Python 2.6 and 3.3. Contributed by "hugovk" in + `#24 `_. + +1.3.1 (2017-05-05) +------------------ + +* Fix broken release (``sdist`` had a broken ``MANIFEST.in``). + +1.3.0 (2017-05-02) +------------------ + +* Speed up arithmetic operations involving ``cext.Proxy`` subclasses. + +1.2.2 (2016-04-14) +------------------ + +* Added `manylinux `_ wheels. +* Minor cleanup in readme. + +1.2.1 (2015-08-18) +------------------ + +* Fix a memory leak (the wrapped object would get bogus references). Contributed by Astrum Kuo in + `#10 `_. + +1.2.0 (2015-07-06) +------------------ + +* Don't instantiate the object when __repr__ is called. This aids with debugging (allows one to see exactly in + what state the proxy is). + +1.1.0 (2015-07-05) +------------------ + +* Added support for pickling. The pickled value is going to be the wrapped object *without* any Proxy container. +* Fixed a memory management issue in the C extension (reference cycles weren't garbage collected due to improper + handling in the C extension). Contributed by Alvin Chow in + `#8 `_. + +1.0.2 (2015-04-11) +----------------------------------------- + +* First release on PyPI. + + diff --git a/env/lib/python3.8/site-packages/lazy_object_proxy-1.4.3.dist-info/RECORD b/env/lib/python3.8/site-packages/lazy_object_proxy-1.4.3.dist-info/RECORD new file mode 100644 index 000000000..edc1035ee --- /dev/null +++ b/env/lib/python3.8/site-packages/lazy_object_proxy-1.4.3.dist-info/RECORD @@ -0,0 +1,18 @@ +lazy_object_proxy-1.4.3.dist-info/INSTALLER,sha256=zuuue4knoyJ-UwPPXg8fezS7VCrXJQrAP7zeNuwvFQg,4 +lazy_object_proxy-1.4.3.dist-info/METADATA,sha256=Y2X63wcFbQT4yI3zKpRFwfpA_TCpB6U79MPmRGCMJT0,5088 +lazy_object_proxy-1.4.3.dist-info/RECORD,, +lazy_object_proxy-1.4.3.dist-info/WHEEL,sha256=VEyGcIFAmk_1KbI6gaZGw_mMiT-pdGweASQLX-DzYaY,108 +lazy_object_proxy-1.4.3.dist-info/top_level.txt,sha256=UNH-FQB-j_8bYqPz3gD90kHvaC42TQqY0thHSnbaa0k,18 +lazy_object_proxy/__init__.py,sha256=pMqxzToF24DuzOltm-Q8nZ3jNDXOaSQcJmiNArdUrlU,410 +lazy_object_proxy/__pycache__/__init__.cpython-38.pyc,, +lazy_object_proxy/__pycache__/_version.cpython-38.pyc,, +lazy_object_proxy/__pycache__/compat.cpython-38.pyc,, +lazy_object_proxy/__pycache__/simple.cpython-38.pyc,, +lazy_object_proxy/__pycache__/slots.cpython-38.pyc,, +lazy_object_proxy/__pycache__/utils.cpython-38.pyc,, +lazy_object_proxy/_version.py,sha256=ALahZZsa-Z4ZoXe2iDIs4ajidCOW695W5leOWijWYEc,116 +lazy_object_proxy/cext.cpython-38-x86_64-linux-gnu.so,sha256=-2QmXTob8M7FEpnLb1sI3f0UR1Q_YhV1z1EupaBk3F0,170352 +lazy_object_proxy/compat.py,sha256=DY3HbKwbrbeKY6tkXRNRLJ1go6HJb8HUwWqyw3T5g_g,196 +lazy_object_proxy/simple.py,sha256=guacy8_QbJeBs7vXpPPVoDVkDNXOZ86xyS1dtAeKvOs,8216 +lazy_object_proxy/slots.py,sha256=9DilWUINScpZN26NwmRtscTtaqaEwtCfIoriLq5Nz24,11359 +lazy_object_proxy/utils.py,sha256=x4XTrtlp_mDTWO_EOq_ILIOv2Qol8RLMnRm5M8l3OfU,291 diff --git a/env/lib/python3.8/site-packages/lazy_object_proxy-1.4.3.dist-info/WHEEL b/env/lib/python3.8/site-packages/lazy_object_proxy-1.4.3.dist-info/WHEEL new file mode 100644 index 000000000..ae40efdd2 --- /dev/null +++ b/env/lib/python3.8/site-packages/lazy_object_proxy-1.4.3.dist-info/WHEEL @@ -0,0 +1,5 @@ +Wheel-Version: 1.0 +Generator: bdist_wheel (0.31.1) +Root-Is-Purelib: false +Tag: cp38-cp38-manylinux1_x86_64 + diff --git a/env/lib/python3.8/site-packages/lazy_object_proxy-1.4.3.dist-info/top_level.txt b/env/lib/python3.8/site-packages/lazy_object_proxy-1.4.3.dist-info/top_level.txt new file mode 100644 index 000000000..bdf032e91 --- /dev/null +++ b/env/lib/python3.8/site-packages/lazy_object_proxy-1.4.3.dist-info/top_level.txt @@ -0,0 +1 @@ +lazy_object_proxy diff --git a/env/lib/python3.8/site-packages/lazy_object_proxy/__init__.py b/env/lib/python3.8/site-packages/lazy_object_proxy/__init__.py new file mode 100644 index 000000000..e9a9a763a --- /dev/null +++ b/env/lib/python3.8/site-packages/lazy_object_proxy/__init__.py @@ -0,0 +1,23 @@ +try: + import copy_reg as copyreg +except ImportError: + import copyreg + +from .utils import identity + +copyreg.constructor(identity) + +try: + from .cext import Proxy + from .cext import identity +except ImportError: + from .slots import Proxy +else: + copyreg.constructor(identity) + +try: + from ._version import version as __version__ +except ImportError: + __version__ = '1.4.3' + +__all__ = "Proxy", diff --git a/env/lib/python3.8/site-packages/lazy_object_proxy/_version.py b/env/lib/python3.8/site-packages/lazy_object_proxy/_version.py new file mode 100644 index 000000000..313677135 --- /dev/null +++ b/env/lib/python3.8/site-packages/lazy_object_proxy/_version.py @@ -0,0 +1,4 @@ +# coding: utf-8 +# file generated by setuptools_scm +# don't change, don't track in version control +version = '1.4.3' diff --git a/env/lib/python3.8/site-packages/lazy_object_proxy/cext.cpython-38-x86_64-linux-gnu.so b/env/lib/python3.8/site-packages/lazy_object_proxy/cext.cpython-38-x86_64-linux-gnu.so new file mode 100755 index 000000000..87640cb0d Binary files /dev/null and b/env/lib/python3.8/site-packages/lazy_object_proxy/cext.cpython-38-x86_64-linux-gnu.so differ diff --git a/env/lib/python3.8/site-packages/lazy_object_proxy/compat.py b/env/lib/python3.8/site-packages/lazy_object_proxy/compat.py new file mode 100644 index 000000000..dc6edfa50 --- /dev/null +++ b/env/lib/python3.8/site-packages/lazy_object_proxy/compat.py @@ -0,0 +1,9 @@ +import sys + +PY2 = sys.version_info[0] == 2 +PY3 = sys.version_info[0] == 3 + + +def with_metaclass(meta, *bases): + """Create a base class with a metaclass.""" + return meta("NewBase", bases, {}) diff --git a/env/lib/python3.8/site-packages/lazy_object_proxy/simple.py b/env/lib/python3.8/site-packages/lazy_object_proxy/simple.py new file mode 100644 index 000000000..24b1339af --- /dev/null +++ b/env/lib/python3.8/site-packages/lazy_object_proxy/simple.py @@ -0,0 +1,246 @@ +import operator + +from .compat import PY2 +from .compat import PY3 +from .compat import with_metaclass +from .utils import cached_property +from .utils import identity + + +def make_proxy_method(code): + def proxy_wrapper(self, *args): + return code(self.__wrapped__, *args) + + return proxy_wrapper + + +class _ProxyMethods(object): + # We use properties to override the values of __module__ and + # __doc__. If we add these in ObjectProxy, the derived class + # __dict__ will still be setup to have string variants of these + # attributes and the rules of descriptors means that they appear to + # take precedence over the properties in the base class. To avoid + # that, we copy the properties into the derived class type itself + # via a meta class. In that way the properties will always take + # precedence. + + @property + def __module__(self): + return self.__wrapped__.__module__ + + @__module__.setter + def __module__(self, value): + self.__wrapped__.__module__ = value + + @property + def __doc__(self): + return self.__wrapped__.__doc__ + + @__doc__.setter + def __doc__(self, value): + self.__wrapped__.__doc__ = value + + # Need to also propagate the special __weakref__ attribute for case + # where decorating classes which will define this. If do not define + # it and use a function like inspect.getmembers() on a decorator + # class it will fail. This can't be in the derived classes. + + @property + def __weakref__(self): + return self.__wrapped__.__weakref__ + + +class _ProxyMetaType(type): + def __new__(cls, name, bases, dictionary): + # Copy our special properties into the class so that they + # always take precedence over attributes of the same name added + # during construction of a derived class. This is to save + # duplicating the implementation for them in all derived classes. + + dictionary.update(vars(_ProxyMethods)) + dictionary.pop('__dict__') + + return type.__new__(cls, name, bases, dictionary) + + +class Proxy(with_metaclass(_ProxyMetaType)): + __factory__ = None + + def __init__(self, factory): + self.__dict__['__factory__'] = factory + + @cached_property + def __wrapped__(self): + self = self.__dict__ + if '__factory__' in self: + factory = self['__factory__'] + return factory() + else: + raise ValueError("Proxy hasn't been initiated: __factory__ is missing.") + + __name__ = property(make_proxy_method(operator.attrgetter('__name__'))) + __class__ = property(make_proxy_method(operator.attrgetter('__class__'))) + __annotations__ = property(make_proxy_method(operator.attrgetter('__anotations__'))) + __dir__ = make_proxy_method(dir) + __str__ = make_proxy_method(str) + + if PY3: + __bytes__ = make_proxy_method(bytes) + + def __repr__(self, __getattr__=object.__getattribute__): + if '__wrapped__' in self.__dict__: + return '<{} at 0x{:x} wrapping {!r} at 0x{:x} with factory {!r}>'.format( + type(self).__name__, id(self), + self.__wrapped__, id(self.__wrapped__), + self.__factory__ + ) + else: + return '<{} at 0x{:x} with factory {!r}>'.format( + type(self).__name__, id(self), + self.__factory__ + ) + + __reversed__ = make_proxy_method(reversed) + + if PY3: + __round__ = make_proxy_method(round) + + __lt__ = make_proxy_method(operator.lt) + __le__ = make_proxy_method(operator.le) + __eq__ = make_proxy_method(operator.eq) + __ne__ = make_proxy_method(operator.ne) + __gt__ = make_proxy_method(operator.gt) + __ge__ = make_proxy_method(operator.ge) + __hash__ = make_proxy_method(hash) + __nonzero__ = make_proxy_method(bool) + __bool__ = make_proxy_method(bool) + + def __setattr__(self, name, value): + if hasattr(type(self), name): + self.__dict__[name] = value + else: + setattr(self.__wrapped__, name, value) + + def __getattr__(self, name): + if name in ('__wrapped__', '__factory__'): + raise AttributeError(name) + else: + return getattr(self.__wrapped__, name) + + def __delattr__(self, name): + if hasattr(type(self), name): + del self.__dict__[name] + else: + delattr(self.__wrapped__, name) + + __add__ = make_proxy_method(operator.add) + __sub__ = make_proxy_method(operator.sub) + __mul__ = make_proxy_method(operator.mul) + __div__ = make_proxy_method(operator.div if PY2 else operator.truediv) + __truediv__ = make_proxy_method(operator.truediv) + __floordiv__ = make_proxy_method(operator.floordiv) + __mod__ = make_proxy_method(operator.mod) + __divmod__ = make_proxy_method(divmod) + __pow__ = make_proxy_method(pow) + __lshift__ = make_proxy_method(operator.lshift) + __rshift__ = make_proxy_method(operator.rshift) + __and__ = make_proxy_method(operator.and_) + __xor__ = make_proxy_method(operator.xor) + __or__ = make_proxy_method(operator.or_) + + def __radd__(self, other): + return other + self.__wrapped__ + + def __rsub__(self, other): + return other - self.__wrapped__ + + def __rmul__(self, other): + return other * self.__wrapped__ + + def __rdiv__(self, other): + return operator.div(other, self.__wrapped__) + + def __rtruediv__(self, other): + return operator.truediv(other, self.__wrapped__) + + def __rfloordiv__(self, other): + return other // self.__wrapped__ + + def __rmod__(self, other): + return other % self.__wrapped__ + + def __rdivmod__(self, other): + return divmod(other, self.__wrapped__) + + def __rpow__(self, other, *args): + return pow(other, self.__wrapped__, *args) + + def __rlshift__(self, other): + return other << self.__wrapped__ + + def __rrshift__(self, other): + return other >> self.__wrapped__ + + def __rand__(self, other): + return other & self.__wrapped__ + + def __rxor__(self, other): + return other ^ self.__wrapped__ + + def __ror__(self, other): + return other | self.__wrapped__ + + __iadd__ = make_proxy_method(operator.iadd) + __isub__ = make_proxy_method(operator.isub) + __imul__ = make_proxy_method(operator.imul) + __idiv__ = make_proxy_method(operator.idiv if PY2 else operator.itruediv) + __itruediv__ = make_proxy_method(operator.itruediv) + __ifloordiv__ = make_proxy_method(operator.ifloordiv) + __imod__ = make_proxy_method(operator.imod) + __ipow__ = make_proxy_method(operator.ipow) + __ilshift__ = make_proxy_method(operator.ilshift) + __irshift__ = make_proxy_method(operator.irshift) + __iand__ = make_proxy_method(operator.iand) + __ixor__ = make_proxy_method(operator.ixor) + __ior__ = make_proxy_method(operator.ior) + __neg__ = make_proxy_method(operator.neg) + __pos__ = make_proxy_method(operator.pos) + __abs__ = make_proxy_method(operator.abs) + __invert__ = make_proxy_method(operator.invert) + + __int__ = make_proxy_method(int) + + if PY2: + __long__ = make_proxy_method(long) # noqa + + __float__ = make_proxy_method(float) + __oct__ = make_proxy_method(oct) + __hex__ = make_proxy_method(hex) + __index__ = make_proxy_method(operator.index) + __len__ = make_proxy_method(len) + __contains__ = make_proxy_method(operator.contains) + __getitem__ = make_proxy_method(operator.getitem) + __setitem__ = make_proxy_method(operator.setitem) + __delitem__ = make_proxy_method(operator.delitem) + + if PY2: + __getslice__ = make_proxy_method(operator.getslice) + __setslice__ = make_proxy_method(operator.setslice) + __delslice__ = make_proxy_method(operator.delslice) + + def __enter__(self): + return self.__wrapped__.__enter__() + + def __exit__(self, *args, **kwargs): + return self.__wrapped__.__exit__(*args, **kwargs) + + __iter__ = make_proxy_method(iter) + + def __call__(self, *args, **kwargs): + return self.__wrapped__(*args, **kwargs) + + def __reduce__(self): + return identity, (self.__wrapped__,) + + def __reduce_ex__(self, protocol): + return identity, (self.__wrapped__,) diff --git a/env/lib/python3.8/site-packages/lazy_object_proxy/slots.py b/env/lib/python3.8/site-packages/lazy_object_proxy/slots.py new file mode 100644 index 000000000..efb08db91 --- /dev/null +++ b/env/lib/python3.8/site-packages/lazy_object_proxy/slots.py @@ -0,0 +1,414 @@ +import operator + +from .compat import PY2 +from .compat import PY3 +from .compat import with_metaclass +from .utils import identity + + +class _ProxyMethods(object): + # We use properties to override the values of __module__ and + # __doc__. If we add these in ObjectProxy, the derived class + # __dict__ will still be setup to have string variants of these + # attributes and the rules of descriptors means that they appear to + # take precedence over the properties in the base class. To avoid + # that, we copy the properties into the derived class type itself + # via a meta class. In that way the properties will always take + # precedence. + + @property + def __module__(self): + return self.__wrapped__.__module__ + + @__module__.setter + def __module__(self, value): + self.__wrapped__.__module__ = value + + @property + def __doc__(self): + return self.__wrapped__.__doc__ + + @__doc__.setter + def __doc__(self, value): + self.__wrapped__.__doc__ = value + + # We similar use a property for __dict__. We need __dict__ to be + # explicit to ensure that vars() works as expected. + + @property + def __dict__(self): + return self.__wrapped__.__dict__ + + # Need to also propagate the special __weakref__ attribute for case + # where decorating classes which will define this. If do not define + # it and use a function like inspect.getmembers() on a decorator + # class it will fail. This can't be in the derived classes. + + @property + def __weakref__(self): + return self.__wrapped__.__weakref__ + + +class _ProxyMetaType(type): + def __new__(cls, name, bases, dictionary): + # Copy our special properties into the class so that they + # always take precedence over attributes of the same name added + # during construction of a derived class. This is to save + # duplicating the implementation for them in all derived classes. + + dictionary.update(vars(_ProxyMethods)) + + return type.__new__(cls, name, bases, dictionary) + + +class Proxy(with_metaclass(_ProxyMetaType)): + """ + A proxy implementation in pure Python, using slots. You can subclass this to add + local methods or attributes, or enable __dict__. + + The most important internals: + + * ``__factory__`` is the callback that "materializes" the object we proxy to. + * ``__target__`` will contain the object we proxy to, once it's "materialized". + * ``__wrapped__`` is a property that does either: + + * return ``__target__`` if it's set. + * calls ``__factory__``, saves result to ``__target__`` and returns said result. + """ + + __slots__ = '__target__', '__factory__' + + def __init__(self, factory): + object.__setattr__(self, '__factory__', factory) + + @property + def __wrapped__(self, __getattr__=object.__getattribute__, __setattr__=object.__setattr__, + __delattr__=object.__delattr__): + try: + return __getattr__(self, '__target__') + except AttributeError: + try: + factory = __getattr__(self, '__factory__') + except AttributeError: + raise ValueError("Proxy hasn't been initiated: __factory__ is missing.") + target = factory() + __setattr__(self, '__target__', target) + return target + + @__wrapped__.deleter + def __wrapped__(self, __delattr__=object.__delattr__): + __delattr__(self, '__target__') + + @__wrapped__.setter + def __wrapped__(self, target, __setattr__=object.__setattr__): + __setattr__(self, '__target__', target) + + @property + def __name__(self): + return self.__wrapped__.__name__ + + @__name__.setter + def __name__(self, value): + self.__wrapped__.__name__ = value + + @property + def __class__(self): + return self.__wrapped__.__class__ + + @__class__.setter # noqa + def __class__(self, value): + self.__wrapped__.__class__ = value + + @property + def __annotations__(self): + return self.__wrapped__.__anotations__ + + @__annotations__.setter + def __annotations__(self, value): + self.__wrapped__.__annotations__ = value + + def __dir__(self): + return dir(self.__wrapped__) + + def __str__(self): + return str(self.__wrapped__) + + if PY3: + def __bytes__(self): + return bytes(self.__wrapped__) + + def __repr__(self, __getattr__=object.__getattribute__): + try: + target = __getattr__(self, '__target__') + except AttributeError: + return '<{} at 0x{:x} with factory {!r}>'.format( + type(self).__name__, id(self), + self.__factory__ + ) + else: + return '<{} at 0x{:x} wrapping {!r} at 0x{:x} with factory {!r}>'.format( + type(self).__name__, id(self), + target, id(target), + self.__factory__ + ) + + def __reversed__(self): + return reversed(self.__wrapped__) + + if PY3: + def __round__(self): + return round(self.__wrapped__) + + def __lt__(self, other): + return self.__wrapped__ < other + + def __le__(self, other): + return self.__wrapped__ <= other + + def __eq__(self, other): + return self.__wrapped__ == other + + def __ne__(self, other): + return self.__wrapped__ != other + + def __gt__(self, other): + return self.__wrapped__ > other + + def __ge__(self, other): + return self.__wrapped__ >= other + + def __hash__(self): + return hash(self.__wrapped__) + + def __nonzero__(self): + return bool(self.__wrapped__) + + def __bool__(self): + return bool(self.__wrapped__) + + def __setattr__(self, name, value, __setattr__=object.__setattr__): + if hasattr(type(self), name): + __setattr__(self, name, value) + else: + setattr(self.__wrapped__, name, value) + + def __getattr__(self, name): + if name in ('__wrapped__', '__factory__'): + raise AttributeError(name) + else: + return getattr(self.__wrapped__, name) + + def __delattr__(self, name, __delattr__=object.__delattr__): + if hasattr(type(self), name): + __delattr__(self, name) + else: + delattr(self.__wrapped__, name) + + def __add__(self, other): + return self.__wrapped__ + other + + def __sub__(self, other): + return self.__wrapped__ - other + + def __mul__(self, other): + return self.__wrapped__ * other + + def __div__(self, other): + return operator.div(self.__wrapped__, other) + + def __truediv__(self, other): + return operator.truediv(self.__wrapped__, other) + + def __floordiv__(self, other): + return self.__wrapped__ // other + + def __mod__(self, other): + return self.__wrapped__ % other + + def __divmod__(self, other): + return divmod(self.__wrapped__, other) + + def __pow__(self, other, *args): + return pow(self.__wrapped__, other, *args) + + def __lshift__(self, other): + return self.__wrapped__ << other + + def __rshift__(self, other): + return self.__wrapped__ >> other + + def __and__(self, other): + return self.__wrapped__ & other + + def __xor__(self, other): + return self.__wrapped__ ^ other + + def __or__(self, other): + return self.__wrapped__ | other + + def __radd__(self, other): + return other + self.__wrapped__ + + def __rsub__(self, other): + return other - self.__wrapped__ + + def __rmul__(self, other): + return other * self.__wrapped__ + + def __rdiv__(self, other): + return operator.div(other, self.__wrapped__) + + def __rtruediv__(self, other): + return operator.truediv(other, self.__wrapped__) + + def __rfloordiv__(self, other): + return other // self.__wrapped__ + + def __rmod__(self, other): + return other % self.__wrapped__ + + def __rdivmod__(self, other): + return divmod(other, self.__wrapped__) + + def __rpow__(self, other, *args): + return pow(other, self.__wrapped__, *args) + + def __rlshift__(self, other): + return other << self.__wrapped__ + + def __rrshift__(self, other): + return other >> self.__wrapped__ + + def __rand__(self, other): + return other & self.__wrapped__ + + def __rxor__(self, other): + return other ^ self.__wrapped__ + + def __ror__(self, other): + return other | self.__wrapped__ + + def __iadd__(self, other): + self.__wrapped__ += other + return self + + def __isub__(self, other): + self.__wrapped__ -= other + return self + + def __imul__(self, other): + self.__wrapped__ *= other + return self + + def __idiv__(self, other): + self.__wrapped__ = operator.idiv(self.__wrapped__, other) + return self + + def __itruediv__(self, other): + self.__wrapped__ = operator.itruediv(self.__wrapped__, other) + return self + + def __ifloordiv__(self, other): + self.__wrapped__ //= other + return self + + def __imod__(self, other): + self.__wrapped__ %= other + return self + + def __ipow__(self, other): + self.__wrapped__ **= other + return self + + def __ilshift__(self, other): + self.__wrapped__ <<= other + return self + + def __irshift__(self, other): + self.__wrapped__ >>= other + return self + + def __iand__(self, other): + self.__wrapped__ &= other + return self + + def __ixor__(self, other): + self.__wrapped__ ^= other + return self + + def __ior__(self, other): + self.__wrapped__ |= other + return self + + def __neg__(self): + return -self.__wrapped__ + + def __pos__(self): + return +self.__wrapped__ + + def __abs__(self): + return abs(self.__wrapped__) + + def __invert__(self): + return ~self.__wrapped__ + + def __int__(self): + return int(self.__wrapped__) + + if PY2: + def __long__(self): + return long(self.__wrapped__) # noqa + + def __float__(self): + return float(self.__wrapped__) + + def __oct__(self): + return oct(self.__wrapped__) + + def __hex__(self): + return hex(self.__wrapped__) + + def __index__(self): + return operator.index(self.__wrapped__) + + def __len__(self): + return len(self.__wrapped__) + + def __contains__(self, value): + return value in self.__wrapped__ + + def __getitem__(self, key): + return self.__wrapped__[key] + + def __setitem__(self, key, value): + self.__wrapped__[key] = value + + def __delitem__(self, key): + del self.__wrapped__[key] + + def __getslice__(self, i, j): + return self.__wrapped__[i:j] + + def __setslice__(self, i, j, value): + self.__wrapped__[i:j] = value + + def __delslice__(self, i, j): + del self.__wrapped__[i:j] + + def __enter__(self): + return self.__wrapped__.__enter__() + + def __exit__(self, *args, **kwargs): + return self.__wrapped__.__exit__(*args, **kwargs) + + def __iter__(self): + return iter(self.__wrapped__) + + def __call__(self, *args, **kwargs): + return self.__wrapped__(*args, **kwargs) + + def __reduce__(self): + return identity, (self.__wrapped__,) + + def __reduce_ex__(self, protocol): + return identity, (self.__wrapped__,) diff --git a/env/lib/python3.8/site-packages/lazy_object_proxy/utils.py b/env/lib/python3.8/site-packages/lazy_object_proxy/utils.py new file mode 100644 index 000000000..ceb3050f7 --- /dev/null +++ b/env/lib/python3.8/site-packages/lazy_object_proxy/utils.py @@ -0,0 +1,13 @@ +def identity(obj): + return obj + + +class cached_property(object): + def __init__(self, func): + self.func = func + + def __get__(self, obj, cls): + if obj is None: + return self + value = obj.__dict__[self.func.__name__] = self.func(obj) + return value diff --git a/env/lib/python3.8/site-packages/mccabe-0.6.1.dist-info/DESCRIPTION.rst b/env/lib/python3.8/site-packages/mccabe-0.6.1.dist-info/DESCRIPTION.rst new file mode 100644 index 000000000..de6106859 --- /dev/null +++ b/env/lib/python3.8/site-packages/mccabe-0.6.1.dist-info/DESCRIPTION.rst @@ -0,0 +1,152 @@ +McCabe complexity checker +========================= + +Ned's script to check McCabe complexity. + +This module provides a plugin for ``flake8``, the Python code checker. + + +Installation +------------ + +You can install, upgrade, uninstall ``mccabe`` with these commands:: + + $ pip install mccabe + $ pip install --upgrade mccabe + $ pip uninstall mccabe + + +Standalone script +----------------- + +The complexity checker can be used directly:: + + $ python -m mccabe --min 5 mccabe.py + ("185:1: 'PathGraphingAstVisitor.visitIf'", 5) + ("71:1: 'PathGraph.to_dot'", 5) + ("245:1: 'McCabeChecker.run'", 5) + ("283:1: 'main'", 7) + ("203:1: 'PathGraphingAstVisitor.visitTryExcept'", 5) + ("257:1: 'get_code_complexity'", 5) + + +Plugin for Flake8 +----------------- + +When both ``flake8 2.0`` and ``mccabe`` are installed, the plugin is +available in ``flake8``:: + + $ flake8 --version + 2.0 (pep8: 1.4.2, pyflakes: 0.6.1, mccabe: 0.2) + +By default the plugin is disabled. Use the ``--max-complexity`` switch to +enable it. It will emit a warning if the McCabe complexity of a function is +higher that the value:: + + $ flake8 --max-complexity 10 coolproject + ... + coolproject/mod.py:1204:1: C901 'CoolFactory.prepare' is too complex (14) + +This feature is quite useful to detect over-complex code. According to McCabe, +anything that goes beyond 10 is too complex. + + +Links +----- + +* Feedback and ideas: http://mail.python.org/mailman/listinfo/code-quality + +* Cyclomatic complexity: http://en.wikipedia.org/wiki/Cyclomatic_complexity. + +* Ned Batchelder's script: + http://nedbatchelder.com/blog/200803/python_code_complexity_microtool.html + + +Changes +------- + +0.6.1 - 2017-01-26 +`````````````````` + +* Fix signature for ``PathGraphingAstVisitor.default`` to match the signature + for ``ASTVisitor`` + +0.6.0 - 2017-01-23 +`````````````````` + +* Add support for Python 3.6 + +* Fix handling for missing statement types + +0.5.3 - 2016-12-14 +`````````````````` + +* Report actual column number of violation instead of the start of the line + +0.5.2 - 2016-07-31 +`````````````````` + +* When opening files ourselves, make sure we always name the file variable + +0.5.1 - 2016-07-28 +`````````````````` + +* Set default maximum complexity to -1 on the class itself + +0.5.0 - 2016-05-30 +`````````````````` + +* PyCon 2016 PDX release + +* Add support for Flake8 3.0 + +0.4.0 - 2016-01-27 +`````````````````` + +* Stop testing on Python 3.2 + +* Add support for async/await keywords on Python 3.5 from PEP 0492 + +0.3.1 - 2015-06-14 +`````````````````` + +* Include ``test_mccabe.py`` in releases. + +* Always coerce the ``max_complexity`` value from Flake8's entry-point to an + integer. + +0.3 - 2014-12-17 +```````````````` + +* Computation was wrong: the mccabe complexity starts at 1, not 2. + +* The ``max-complexity`` value is now inclusive. E.g.: if the + value is 10 and the reported complexity is 10, then it passes. + +* Add tests. + + +0.2.1 - 2013-04-03 +`````````````````` + +* Do not require ``setuptools`` in setup.py. It works around an issue + with ``pip`` and Python 3. + + +0.2 - 2013-02-22 +```````````````` + +* Rename project to ``mccabe``. + +* Provide ``flake8.extension`` setuptools entry point. + +* Read ``max-complexity`` from the configuration file. + +* Rename argument ``min_complexity`` to ``threshold``. + + +0.1 - 2013-02-11 +```````````````` +* First release + + diff --git a/env/lib/python3.8/site-packages/mccabe-0.6.1.dist-info/INSTALLER b/env/lib/python3.8/site-packages/mccabe-0.6.1.dist-info/INSTALLER new file mode 100644 index 000000000..a1b589e38 --- /dev/null +++ b/env/lib/python3.8/site-packages/mccabe-0.6.1.dist-info/INSTALLER @@ -0,0 +1 @@ +pip diff --git a/env/lib/python3.8/site-packages/mccabe-0.6.1.dist-info/METADATA b/env/lib/python3.8/site-packages/mccabe-0.6.1.dist-info/METADATA new file mode 100644 index 000000000..f22645f3b --- /dev/null +++ b/env/lib/python3.8/site-packages/mccabe-0.6.1.dist-info/METADATA @@ -0,0 +1,178 @@ +Metadata-Version: 2.0 +Name: mccabe +Version: 0.6.1 +Summary: McCabe checker, plugin for flake8 +Home-page: https://github.com/pycqa/mccabe +Author: Ian Cordasco +Author-email: graffatcolmingov@gmail.com +License: Expat license +Keywords: flake8 mccabe +Platform: UNKNOWN +Classifier: Development Status :: 5 - Production/Stable +Classifier: Environment :: Console +Classifier: Intended Audience :: Developers +Classifier: License :: OSI Approved :: MIT License +Classifier: Operating System :: OS Independent +Classifier: Programming Language :: Python +Classifier: Programming Language :: Python :: 2 +Classifier: Programming Language :: Python :: 2.7 +Classifier: Programming Language :: Python :: 3 +Classifier: Programming Language :: Python :: 3.3 +Classifier: Programming Language :: Python :: 3.4 +Classifier: Programming Language :: Python :: 3.5 +Classifier: Programming Language :: Python :: 3.6 +Classifier: Topic :: Software Development :: Libraries :: Python Modules +Classifier: Topic :: Software Development :: Quality Assurance + +McCabe complexity checker +========================= + +Ned's script to check McCabe complexity. + +This module provides a plugin for ``flake8``, the Python code checker. + + +Installation +------------ + +You can install, upgrade, uninstall ``mccabe`` with these commands:: + + $ pip install mccabe + $ pip install --upgrade mccabe + $ pip uninstall mccabe + + +Standalone script +----------------- + +The complexity checker can be used directly:: + + $ python -m mccabe --min 5 mccabe.py + ("185:1: 'PathGraphingAstVisitor.visitIf'", 5) + ("71:1: 'PathGraph.to_dot'", 5) + ("245:1: 'McCabeChecker.run'", 5) + ("283:1: 'main'", 7) + ("203:1: 'PathGraphingAstVisitor.visitTryExcept'", 5) + ("257:1: 'get_code_complexity'", 5) + + +Plugin for Flake8 +----------------- + +When both ``flake8 2.0`` and ``mccabe`` are installed, the plugin is +available in ``flake8``:: + + $ flake8 --version + 2.0 (pep8: 1.4.2, pyflakes: 0.6.1, mccabe: 0.2) + +By default the plugin is disabled. Use the ``--max-complexity`` switch to +enable it. It will emit a warning if the McCabe complexity of a function is +higher that the value:: + + $ flake8 --max-complexity 10 coolproject + ... + coolproject/mod.py:1204:1: C901 'CoolFactory.prepare' is too complex (14) + +This feature is quite useful to detect over-complex code. According to McCabe, +anything that goes beyond 10 is too complex. + + +Links +----- + +* Feedback and ideas: http://mail.python.org/mailman/listinfo/code-quality + +* Cyclomatic complexity: http://en.wikipedia.org/wiki/Cyclomatic_complexity. + +* Ned Batchelder's script: + http://nedbatchelder.com/blog/200803/python_code_complexity_microtool.html + + +Changes +------- + +0.6.1 - 2017-01-26 +`````````````````` + +* Fix signature for ``PathGraphingAstVisitor.default`` to match the signature + for ``ASTVisitor`` + +0.6.0 - 2017-01-23 +`````````````````` + +* Add support for Python 3.6 + +* Fix handling for missing statement types + +0.5.3 - 2016-12-14 +`````````````````` + +* Report actual column number of violation instead of the start of the line + +0.5.2 - 2016-07-31 +`````````````````` + +* When opening files ourselves, make sure we always name the file variable + +0.5.1 - 2016-07-28 +`````````````````` + +* Set default maximum complexity to -1 on the class itself + +0.5.0 - 2016-05-30 +`````````````````` + +* PyCon 2016 PDX release + +* Add support for Flake8 3.0 + +0.4.0 - 2016-01-27 +`````````````````` + +* Stop testing on Python 3.2 + +* Add support for async/await keywords on Python 3.5 from PEP 0492 + +0.3.1 - 2015-06-14 +`````````````````` + +* Include ``test_mccabe.py`` in releases. + +* Always coerce the ``max_complexity`` value from Flake8's entry-point to an + integer. + +0.3 - 2014-12-17 +```````````````` + +* Computation was wrong: the mccabe complexity starts at 1, not 2. + +* The ``max-complexity`` value is now inclusive. E.g.: if the + value is 10 and the reported complexity is 10, then it passes. + +* Add tests. + + +0.2.1 - 2013-04-03 +`````````````````` + +* Do not require ``setuptools`` in setup.py. It works around an issue + with ``pip`` and Python 3. + + +0.2 - 2013-02-22 +```````````````` + +* Rename project to ``mccabe``. + +* Provide ``flake8.extension`` setuptools entry point. + +* Read ``max-complexity`` from the configuration file. + +* Rename argument ``min_complexity`` to ``threshold``. + + +0.1 - 2013-02-11 +```````````````` +* First release + + diff --git a/env/lib/python3.8/site-packages/mccabe-0.6.1.dist-info/RECORD b/env/lib/python3.8/site-packages/mccabe-0.6.1.dist-info/RECORD new file mode 100644 index 000000000..df6a393d2 --- /dev/null +++ b/env/lib/python3.8/site-packages/mccabe-0.6.1.dist-info/RECORD @@ -0,0 +1,10 @@ +__pycache__/mccabe.cpython-38.pyc,, +mccabe-0.6.1.dist-info/DESCRIPTION.rst,sha256=lGHJ-Y3IviuP3DRqLL_TXPUr3wJ2GZ8XJkAV6ve3O58,3302 +mccabe-0.6.1.dist-info/INSTALLER,sha256=zuuue4knoyJ-UwPPXg8fezS7VCrXJQrAP7zeNuwvFQg,4 +mccabe-0.6.1.dist-info/METADATA,sha256=jawnTfTVrlzBSmeI-cTXSRIwIlijODtZdj-padBqIv0,4324 +mccabe-0.6.1.dist-info/RECORD,, +mccabe-0.6.1.dist-info/WHEEL,sha256=o2k-Qa-RMNIJmUdIc7KU6VWR_ErNRbWNlxDIpl7lm34,110 +mccabe-0.6.1.dist-info/entry_points.txt,sha256=N2NH182GXTUyTm8r8XMgadb9C-CRa5dUr1k8OC91uGE,47 +mccabe-0.6.1.dist-info/metadata.json,sha256=e508OR4t6_G7h7eO2C6NHlHQqVpPZZH1_DlAPrVECYM,1218 +mccabe-0.6.1.dist-info/top_level.txt,sha256=21cXuqZE-lpcfAqqANvX9EjI1ED1p8zcViv064u3RKA,7 +mccabe.py,sha256=XPMywdQshG_5nSjckb-OzNqnCQuXQvy3FTClspKwGQA,10693 diff --git a/env/lib/python3.8/site-packages/mccabe-0.6.1.dist-info/WHEEL b/env/lib/python3.8/site-packages/mccabe-0.6.1.dist-info/WHEEL new file mode 100644 index 000000000..8b6dd1b5a --- /dev/null +++ b/env/lib/python3.8/site-packages/mccabe-0.6.1.dist-info/WHEEL @@ -0,0 +1,6 @@ +Wheel-Version: 1.0 +Generator: bdist_wheel (0.29.0) +Root-Is-Purelib: true +Tag: py2-none-any +Tag: py3-none-any + diff --git a/env/lib/python3.8/site-packages/mccabe-0.6.1.dist-info/entry_points.txt b/env/lib/python3.8/site-packages/mccabe-0.6.1.dist-info/entry_points.txt new file mode 100644 index 000000000..cc6645bdc --- /dev/null +++ b/env/lib/python3.8/site-packages/mccabe-0.6.1.dist-info/entry_points.txt @@ -0,0 +1,3 @@ +[flake8.extension] +C90 = mccabe:McCabeChecker + diff --git a/env/lib/python3.8/site-packages/mccabe-0.6.1.dist-info/metadata.json b/env/lib/python3.8/site-packages/mccabe-0.6.1.dist-info/metadata.json new file mode 100644 index 000000000..ae04d8fac --- /dev/null +++ b/env/lib/python3.8/site-packages/mccabe-0.6.1.dist-info/metadata.json @@ -0,0 +1 @@ +{"classifiers": ["Development Status :: 5 - Production/Stable", "Environment :: Console", "Intended Audience :: Developers", "License :: OSI Approved :: MIT License", "Operating System :: OS Independent", "Programming Language :: Python", "Programming Language :: Python :: 2", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.3", "Programming Language :: Python :: 3.4", "Programming Language :: Python :: 3.5", "Programming Language :: Python :: 3.6", "Topic :: Software Development :: Libraries :: Python Modules", "Topic :: Software Development :: Quality Assurance"], "extensions": {"python.details": {"contacts": [{"email": "graffatcolmingov@gmail.com", "name": "Ian Cordasco", "role": "author"}], "document_names": {"description": "DESCRIPTION.rst"}, "project_urls": {"Home": "https://github.com/pycqa/mccabe"}}, "python.exports": {"flake8.extension": {"C90": "mccabe:McCabeChecker"}}}, "generator": "bdist_wheel (0.29.0)", "keywords": ["flake8", "mccabe"], "license": "Expat license", "metadata_version": "2.0", "name": "mccabe", "summary": "McCabe checker, plugin for flake8", "test_requires": [{"requires": ["pytest"]}], "version": "0.6.1"} \ No newline at end of file diff --git a/env/lib/python3.8/site-packages/mccabe-0.6.1.dist-info/top_level.txt b/env/lib/python3.8/site-packages/mccabe-0.6.1.dist-info/top_level.txt new file mode 100644 index 000000000..8831b36b4 --- /dev/null +++ b/env/lib/python3.8/site-packages/mccabe-0.6.1.dist-info/top_level.txt @@ -0,0 +1 @@ +mccabe diff --git a/env/lib/python3.8/site-packages/mccabe.py b/env/lib/python3.8/site-packages/mccabe.py new file mode 100644 index 000000000..c0cda7568 --- /dev/null +++ b/env/lib/python3.8/site-packages/mccabe.py @@ -0,0 +1,347 @@ +""" Meager code path measurement tool. + Ned Batchelder + http://nedbatchelder.com/blog/200803/python_code_complexity_microtool.html + MIT License. +""" +from __future__ import with_statement + +import optparse +import sys +import tokenize + +from collections import defaultdict +try: + import ast + from ast import iter_child_nodes +except ImportError: # Python 2.5 + from flake8.util import ast, iter_child_nodes + +__version__ = '0.6.1' + + +class ASTVisitor(object): + """Performs a depth-first walk of the AST.""" + + def __init__(self): + self.node = None + self._cache = {} + + def default(self, node, *args): + for child in iter_child_nodes(node): + self.dispatch(child, *args) + + def dispatch(self, node, *args): + self.node = node + klass = node.__class__ + meth = self._cache.get(klass) + if meth is None: + className = klass.__name__ + meth = getattr(self.visitor, 'visit' + className, self.default) + self._cache[klass] = meth + return meth(node, *args) + + def preorder(self, tree, visitor, *args): + """Do preorder walk of tree using visitor""" + self.visitor = visitor + visitor.visit = self.dispatch + self.dispatch(tree, *args) # XXX *args make sense? + + +class PathNode(object): + def __init__(self, name, look="circle"): + self.name = name + self.look = look + + def to_dot(self): + print('node [shape=%s,label="%s"] %d;' % ( + self.look, self.name, self.dot_id())) + + def dot_id(self): + return id(self) + + +class PathGraph(object): + def __init__(self, name, entity, lineno, column=0): + self.name = name + self.entity = entity + self.lineno = lineno + self.column = column + self.nodes = defaultdict(list) + + def connect(self, n1, n2): + self.nodes[n1].append(n2) + # Ensure that the destination node is always counted. + self.nodes[n2] = [] + + def to_dot(self): + print('subgraph {') + for node in self.nodes: + node.to_dot() + for node, nexts in self.nodes.items(): + for next in nexts: + print('%s -- %s;' % (node.dot_id(), next.dot_id())) + print('}') + + def complexity(self): + """ Return the McCabe complexity for the graph. + V-E+2 + """ + num_edges = sum([len(n) for n in self.nodes.values()]) + num_nodes = len(self.nodes) + return num_edges - num_nodes + 2 + + +class PathGraphingAstVisitor(ASTVisitor): + """ A visitor for a parsed Abstract Syntax Tree which finds executable + statements. + """ + + def __init__(self): + super(PathGraphingAstVisitor, self).__init__() + self.classname = "" + self.graphs = {} + self.reset() + + def reset(self): + self.graph = None + self.tail = None + + def dispatch_list(self, node_list): + for node in node_list: + self.dispatch(node) + + def visitFunctionDef(self, node): + + if self.classname: + entity = '%s%s' % (self.classname, node.name) + else: + entity = node.name + + name = '%d:%d: %r' % (node.lineno, node.col_offset, entity) + + if self.graph is not None: + # closure + pathnode = self.appendPathNode(name) + self.tail = pathnode + self.dispatch_list(node.body) + bottom = PathNode("", look='point') + self.graph.connect(self.tail, bottom) + self.graph.connect(pathnode, bottom) + self.tail = bottom + else: + self.graph = PathGraph(name, entity, node.lineno, node.col_offset) + pathnode = PathNode(name) + self.tail = pathnode + self.dispatch_list(node.body) + self.graphs["%s%s" % (self.classname, node.name)] = self.graph + self.reset() + + visitAsyncFunctionDef = visitFunctionDef + + def visitClassDef(self, node): + old_classname = self.classname + self.classname += node.name + "." + self.dispatch_list(node.body) + self.classname = old_classname + + def appendPathNode(self, name): + if not self.tail: + return + pathnode = PathNode(name) + self.graph.connect(self.tail, pathnode) + self.tail = pathnode + return pathnode + + def visitSimpleStatement(self, node): + if node.lineno is None: + lineno = 0 + else: + lineno = node.lineno + name = "Stmt %d" % lineno + self.appendPathNode(name) + + def default(self, node, *args): + if isinstance(node, ast.stmt): + self.visitSimpleStatement(node) + else: + super(PathGraphingAstVisitor, self).default(node, *args) + + def visitLoop(self, node): + name = "Loop %d" % node.lineno + self._subgraph(node, name) + + visitAsyncFor = visitFor = visitWhile = visitLoop + + def visitIf(self, node): + name = "If %d" % node.lineno + self._subgraph(node, name) + + def _subgraph(self, node, name, extra_blocks=()): + """create the subgraphs representing any `if` and `for` statements""" + if self.graph is None: + # global loop + self.graph = PathGraph(name, name, node.lineno, node.col_offset) + pathnode = PathNode(name) + self._subgraph_parse(node, pathnode, extra_blocks) + self.graphs["%s%s" % (self.classname, name)] = self.graph + self.reset() + else: + pathnode = self.appendPathNode(name) + self._subgraph_parse(node, pathnode, extra_blocks) + + def _subgraph_parse(self, node, pathnode, extra_blocks): + """parse the body and any `else` block of `if` and `for` statements""" + loose_ends = [] + self.tail = pathnode + self.dispatch_list(node.body) + loose_ends.append(self.tail) + for extra in extra_blocks: + self.tail = pathnode + self.dispatch_list(extra.body) + loose_ends.append(self.tail) + if node.orelse: + self.tail = pathnode + self.dispatch_list(node.orelse) + loose_ends.append(self.tail) + else: + loose_ends.append(pathnode) + if pathnode: + bottom = PathNode("", look='point') + for le in loose_ends: + self.graph.connect(le, bottom) + self.tail = bottom + + def visitTryExcept(self, node): + name = "TryExcept %d" % node.lineno + self._subgraph(node, name, extra_blocks=node.handlers) + + visitTry = visitTryExcept + + def visitWith(self, node): + name = "With %d" % node.lineno + self.appendPathNode(name) + self.dispatch_list(node.body) + + visitAsyncWith = visitWith + + +class McCabeChecker(object): + """McCabe cyclomatic complexity checker.""" + name = 'mccabe' + version = __version__ + _code = 'C901' + _error_tmpl = "C901 %r is too complex (%d)" + max_complexity = -1 + + def __init__(self, tree, filename): + self.tree = tree + + @classmethod + def add_options(cls, parser): + flag = '--max-complexity' + kwargs = { + 'default': -1, + 'action': 'store', + 'type': 'int', + 'help': 'McCabe complexity threshold', + 'parse_from_config': 'True', + } + config_opts = getattr(parser, 'config_options', None) + if isinstance(config_opts, list): + # Flake8 2.x + kwargs.pop('parse_from_config') + parser.add_option(flag, **kwargs) + parser.config_options.append('max-complexity') + else: + parser.add_option(flag, **kwargs) + + @classmethod + def parse_options(cls, options): + cls.max_complexity = int(options.max_complexity) + + def run(self): + if self.max_complexity < 0: + return + visitor = PathGraphingAstVisitor() + visitor.preorder(self.tree, visitor) + for graph in visitor.graphs.values(): + if graph.complexity() > self.max_complexity: + text = self._error_tmpl % (graph.entity, graph.complexity()) + yield graph.lineno, graph.column, text, type(self) + + +def get_code_complexity(code, threshold=7, filename='stdin'): + try: + tree = compile(code, filename, "exec", ast.PyCF_ONLY_AST) + except SyntaxError: + e = sys.exc_info()[1] + sys.stderr.write("Unable to parse %s: %s\n" % (filename, e)) + return 0 + + complx = [] + McCabeChecker.max_complexity = threshold + for lineno, offset, text, check in McCabeChecker(tree, filename).run(): + complx.append('%s:%d:1: %s' % (filename, lineno, text)) + + if len(complx) == 0: + return 0 + print('\n'.join(complx)) + return len(complx) + + +def get_module_complexity(module_path, threshold=7): + """Returns the complexity of a module""" + with open(module_path, "rU") as mod: + code = mod.read() + return get_code_complexity(code, threshold, filename=module_path) + + +def _read(filename): + if (2, 5) < sys.version_info < (3, 0): + with open(filename, 'rU') as f: + return f.read() + elif (3, 0) <= sys.version_info < (4, 0): + """Read the source code.""" + try: + with open(filename, 'rb') as f: + (encoding, _) = tokenize.detect_encoding(f.readline) + except (LookupError, SyntaxError, UnicodeError): + # Fall back if file encoding is improperly declared + with open(filename, encoding='latin-1') as f: + return f.read() + with open(filename, 'r', encoding=encoding) as f: + return f.read() + + +def main(argv=None): + if argv is None: + argv = sys.argv[1:] + opar = optparse.OptionParser() + opar.add_option("-d", "--dot", dest="dot", + help="output a graphviz dot file", action="store_true") + opar.add_option("-m", "--min", dest="threshold", + help="minimum complexity for output", type="int", + default=1) + + options, args = opar.parse_args(argv) + + code = _read(args[0]) + tree = compile(code, args[0], "exec", ast.PyCF_ONLY_AST) + visitor = PathGraphingAstVisitor() + visitor.preorder(tree, visitor) + + if options.dot: + print('graph {') + for graph in visitor.graphs.values(): + if (not options.threshold or + graph.complexity() >= options.threshold): + graph.to_dot() + print('}') + else: + for graph in visitor.graphs.values(): + if graph.complexity() >= options.threshold: + print(graph.name, graph.complexity()) + + +if __name__ == '__main__': + main(sys.argv[1:]) diff --git a/env/lib/python3.8/site-packages/pip-20.0.2.dist-info/INSTALLER b/env/lib/python3.8/site-packages/pip-20.0.2.dist-info/INSTALLER new file mode 100644 index 000000000..a1b589e38 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip-20.0.2.dist-info/INSTALLER @@ -0,0 +1 @@ +pip diff --git a/env/lib/python3.8/site-packages/pip-20.0.2.dist-info/LICENSE.txt b/env/lib/python3.8/site-packages/pip-20.0.2.dist-info/LICENSE.txt new file mode 100644 index 000000000..737fec5c5 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip-20.0.2.dist-info/LICENSE.txt @@ -0,0 +1,20 @@ +Copyright (c) 2008-2019 The pip developers (see AUTHORS.txt file) + +Permission is hereby granted, free of charge, to any person obtaining +a copy of this software and associated documentation files (the +"Software"), to deal in the Software without restriction, including +without limitation the rights to use, copy, modify, merge, publish, +distribute, sublicense, and/or sell copies of the Software, and to +permit persons to whom the Software is furnished to do so, subject to +the following conditions: + +The above copyright notice and this permission notice shall be +included in all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, +EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF +MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE +LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION +OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION +WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. diff --git a/env/lib/python3.8/site-packages/pip-20.0.2.dist-info/METADATA b/env/lib/python3.8/site-packages/pip-20.0.2.dist-info/METADATA new file mode 100644 index 000000000..5183c4e68 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip-20.0.2.dist-info/METADATA @@ -0,0 +1,84 @@ +Metadata-Version: 2.1 +Name: pip +Version: 20.0.2 +Summary: The PyPA recommended tool for installing Python packages. +Home-page: https://pip.pypa.io/ +Author: The pip developers +Author-email: pypa-dev@groups.google.com +License: MIT +Project-URL: Documentation, https://pip.pypa.io +Project-URL: Source, https://github.com/pypa/pip +Keywords: distutils easy_install egg setuptools wheel virtualenv +Platform: UNKNOWN +Classifier: Development Status :: 5 - Production/Stable +Classifier: Intended Audience :: Developers +Classifier: License :: OSI Approved :: MIT License +Classifier: Topic :: Software Development :: Build Tools +Classifier: Programming Language :: Python +Classifier: Programming Language :: Python :: 2 +Classifier: Programming Language :: Python :: 2.7 +Classifier: Programming Language :: Python :: 3 +Classifier: Programming Language :: Python :: 3.5 +Classifier: Programming Language :: Python :: 3.6 +Classifier: Programming Language :: Python :: 3.7 +Classifier: Programming Language :: Python :: 3.8 +Classifier: Programming Language :: Python :: Implementation :: CPython +Classifier: Programming Language :: Python :: Implementation :: PyPy +Requires-Python: >=2.7,!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.* + +pip - The Python Package Installer +================================== + +.. image:: https://img.shields.io/pypi/v/pip.svg + :target: https://pypi.org/project/pip/ + +.. image:: https://readthedocs.org/projects/pip/badge/?version=latest + :target: https://pip.pypa.io/en/latest + +pip is the `package installer`_ for Python. You can use pip to install packages from the `Python Package Index`_ and other indexes. + +Please take a look at our documentation for how to install and use pip: + +* `Installation`_ +* `Usage`_ + +Updates are released regularly, with a new version every 3 months. More details can be found in our documentation: + +* `Release notes`_ +* `Release process`_ + +If you find bugs, need help, or want to talk to the developers please use our mailing lists or chat rooms: + +* `Issue tracking`_ +* `Discourse channel`_ +* `User IRC`_ + +If you want to get involved head over to GitHub to get the source code, look at our development documentation and feel free to jump on the developer mailing lists and chat rooms: + +* `GitHub page`_ +* `Dev documentation`_ +* `Dev mailing list`_ +* `Dev IRC`_ + +Code of Conduct +--------------- + +Everyone interacting in the pip project's codebases, issue trackers, chat +rooms, and mailing lists is expected to follow the `PyPA Code of Conduct`_. + +.. _package installer: https://packaging.python.org/guides/tool-recommendations/ +.. _Python Package Index: https://pypi.org +.. _Installation: https://pip.pypa.io/en/stable/installing.html +.. _Usage: https://pip.pypa.io/en/stable/ +.. _Release notes: https://pip.pypa.io/en/stable/news.html +.. _Release process: https://pip.pypa.io/en/latest/development/release-process/ +.. _GitHub page: https://github.com/pypa/pip +.. _Dev documentation: https://pip.pypa.io/en/latest/development +.. _Issue tracking: https://github.com/pypa/pip/issues +.. _Discourse channel: https://discuss.python.org/c/packaging +.. _Dev mailing list: https://groups.google.com/forum/#!forum/pypa-dev +.. _User IRC: https://webchat.freenode.net/?channels=%23pypa +.. _Dev IRC: https://webchat.freenode.net/?channels=%23pypa-dev +.. _PyPA Code of Conduct: https://www.pypa.io/en/latest/code-of-conduct/ + + diff --git a/env/lib/python3.8/site-packages/pip-20.0.2.dist-info/RECORD b/env/lib/python3.8/site-packages/pip-20.0.2.dist-info/RECORD new file mode 100644 index 000000000..170f627d2 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip-20.0.2.dist-info/RECORD @@ -0,0 +1,246 @@ +../../../bin/pip,sha256=tbRdDgDeyny0IpgPfD_SFhIQHuGOEuJ1SNN8OTE1Jlc,301 +../../../bin/pip3,sha256=tbRdDgDeyny0IpgPfD_SFhIQHuGOEuJ1SNN8OTE1Jlc,301 +../../../bin/pip3.8,sha256=tbRdDgDeyny0IpgPfD_SFhIQHuGOEuJ1SNN8OTE1Jlc,301 +pip-20.0.2.dist-info/INSTALLER,sha256=zuuue4knoyJ-UwPPXg8fezS7VCrXJQrAP7zeNuwvFQg,4 +pip-20.0.2.dist-info/LICENSE.txt,sha256=W6Ifuwlk-TatfRU2LR7W1JMcyMj5_y1NkRkOEJvnRDE,1090 +pip-20.0.2.dist-info/METADATA,sha256=MSgjT2JTt8usp4Hopp5AGEmc-7sKR2Jd7HTMJqCoRhw,3352 +pip-20.0.2.dist-info/RECORD,, +pip-20.0.2.dist-info/WHEEL,sha256=kGT74LWyRUZrL4VgLh6_g12IeVl_9u9ZVhadrgXZUEY,110 +pip-20.0.2.dist-info/entry_points.txt,sha256=HtfDOwpUlr9s73jqLQ6wF9V0_0qvUXJwCBz7Vwx0Ue0,125 +pip-20.0.2.dist-info/top_level.txt,sha256=zuuue4knoyJ-UwPPXg8fezS7VCrXJQrAP7zeNuwvFQg,4 +pip/__init__.py,sha256=U1AM82iShMaw90K6Yq0Q2-AZ1EsOcqQLQRB-rxwFtII,455 +pip/__main__.py,sha256=NM95x7KuQr-lwPoTjAC0d_QzLJsJjpmAoxZg0mP8s98,632 +pip/__pycache__/__init__.cpython-38.pyc,, +pip/__pycache__/__main__.cpython-38.pyc,, +pip/_internal/__init__.py,sha256=j5fiII6yCeZjpW7_7wAVRMM4DwE-gyARGVU4yAADDeE,517 +pip/_internal/__pycache__/__init__.cpython-38.pyc,, +pip/_internal/__pycache__/build_env.cpython-38.pyc,, +pip/_internal/__pycache__/cache.cpython-38.pyc,, +pip/_internal/__pycache__/configuration.cpython-38.pyc,, +pip/_internal/__pycache__/exceptions.cpython-38.pyc,, +pip/_internal/__pycache__/legacy_resolve.cpython-38.pyc,, +pip/_internal/__pycache__/locations.cpython-38.pyc,, +pip/_internal/__pycache__/main.cpython-38.pyc,, +pip/_internal/__pycache__/pep425tags.cpython-38.pyc,, +pip/_internal/__pycache__/pyproject.cpython-38.pyc,, +pip/_internal/__pycache__/self_outdated_check.cpython-38.pyc,, +pip/_internal/__pycache__/wheel_builder.cpython-38.pyc,, +pip/_internal/build_env.py,sha256=--aNgzIdYrCOclHMwoAdpclCpfdFE_jooRuCy5gczwg,7532 +pip/_internal/cache.py,sha256=16GrnDRLBQNlfKWIuIF6Sa-EFS78kez_w1WEjT3ykTI,11605 +pip/_internal/cli/__init__.py,sha256=FkHBgpxxb-_gd6r1FjnNhfMOzAUYyXoXKJ6abijfcFU,132 +pip/_internal/cli/__pycache__/__init__.cpython-38.pyc,, +pip/_internal/cli/__pycache__/autocompletion.cpython-38.pyc,, +pip/_internal/cli/__pycache__/base_command.cpython-38.pyc,, +pip/_internal/cli/__pycache__/cmdoptions.cpython-38.pyc,, +pip/_internal/cli/__pycache__/command_context.cpython-38.pyc,, +pip/_internal/cli/__pycache__/main.cpython-38.pyc,, +pip/_internal/cli/__pycache__/main_parser.cpython-38.pyc,, +pip/_internal/cli/__pycache__/parser.cpython-38.pyc,, +pip/_internal/cli/__pycache__/req_command.cpython-38.pyc,, +pip/_internal/cli/__pycache__/status_codes.cpython-38.pyc,, +pip/_internal/cli/autocompletion.py,sha256=ekGNtcDI0p7rFVc-7s4T9Tbss4Jgb7vsB649XJIblRg,6547 +pip/_internal/cli/base_command.py,sha256=v6yl5XNRqye8BT9ep8wvpMu6lylP_Hu6D95r_HqbpbQ,7948 +pip/_internal/cli/cmdoptions.py,sha256=f1TVHuu_fR3lLlMo6b367H_GsWFv26tLI9cAS-kZfE0,28114 +pip/_internal/cli/command_context.py,sha256=ygMVoTy2jpNilKT-6416gFSQpaBtrKRBbVbi2fy__EU,975 +pip/_internal/cli/main.py,sha256=8iq3bHe5lxJTB2EvKOqZ38NS0MmoS79_S1kgj4QuH8A,2610 +pip/_internal/cli/main_parser.py,sha256=W9OWeryh7ZkqELohaFh0Ko9sB98ZkSeDmnYbOZ1imBc,2819 +pip/_internal/cli/parser.py,sha256=O9djTuYQuSfObiY-NU6p4MJCfWsRUnDpE2YGA_fwols,9487 +pip/_internal/cli/req_command.py,sha256=pAUAglpTn0mUA6lRs7KN71yOm1KDabD0ySVTQTqWTSA,12463 +pip/_internal/cli/status_codes.py,sha256=F6uDG6Gj7RNKQJUDnd87QKqI16Us-t-B0wPF_4QMpWc,156 +pip/_internal/commands/__init__.py,sha256=uTSj58QlrSKeXqCUSdL-eAf_APzx5BHy1ABxb0j5ZNE,3714 +pip/_internal/commands/__pycache__/__init__.cpython-38.pyc,, +pip/_internal/commands/__pycache__/check.cpython-38.pyc,, +pip/_internal/commands/__pycache__/completion.cpython-38.pyc,, +pip/_internal/commands/__pycache__/configuration.cpython-38.pyc,, +pip/_internal/commands/__pycache__/debug.cpython-38.pyc,, +pip/_internal/commands/__pycache__/download.cpython-38.pyc,, +pip/_internal/commands/__pycache__/freeze.cpython-38.pyc,, +pip/_internal/commands/__pycache__/hash.cpython-38.pyc,, +pip/_internal/commands/__pycache__/help.cpython-38.pyc,, +pip/_internal/commands/__pycache__/install.cpython-38.pyc,, +pip/_internal/commands/__pycache__/list.cpython-38.pyc,, +pip/_internal/commands/__pycache__/search.cpython-38.pyc,, +pip/_internal/commands/__pycache__/show.cpython-38.pyc,, +pip/_internal/commands/__pycache__/uninstall.cpython-38.pyc,, +pip/_internal/commands/__pycache__/wheel.cpython-38.pyc,, +pip/_internal/commands/check.py,sha256=mgLNYT3bd6Kmynwh4zzcBmVlFZ-urMo40jTgk6U405E,1505 +pip/_internal/commands/completion.py,sha256=UFQvq0Q4_B96z1bvnQyMOq82aPSu05RejbLmqeTZjC0,2975 +pip/_internal/commands/configuration.py,sha256=6riioZjMhsNSEct7dE-X8SobGodk3WERKJvuyjBje4Q,7226 +pip/_internal/commands/debug.py,sha256=a8llax2hRkxgK-tvwdJgaCaZCYPIx0fDvrlMDoYr8bQ,4209 +pip/_internal/commands/download.py,sha256=zX_0-IeFb4C8dxSmGHxk-6H5kehtyTSsdWpjNpAhSww,5007 +pip/_internal/commands/freeze.py,sha256=QS-4ib8jbKJ2wrDaDbTuyaB3Y_iJ5CQC2gAVHuAv9QU,3481 +pip/_internal/commands/hash.py,sha256=47teimfAPhpkaVbSDaafck51BT3XXYuL83lAqc5lOcE,1735 +pip/_internal/commands/help.py,sha256=Nhecq--ydFn80Gm1Zvbf9943EcRJfO0TnXUhsF0RO7s,1181 +pip/_internal/commands/install.py,sha256=T4P3J1rw7CQrZX4OUamtcoWMkTrJBfUe6gWpTfZW1bQ,27286 +pip/_internal/commands/list.py,sha256=2l0JiqHxjxDHNTCb2HZOjwwdo4duS1R0MsqZb6HSMKk,10660 +pip/_internal/commands/search.py,sha256=7Il8nKZ9mM7qF5jlnBoPvSIFY9f-0-5IbYoX3miTuZY,5148 +pip/_internal/commands/show.py,sha256=Vzsj2oX0JBl94MPyF3LV8YoMcigl8B2UsMM8zp0pH2s,6792 +pip/_internal/commands/uninstall.py,sha256=8mldFbrQecSoWDZRqxBgJkrlvx6Y9Iy7cs-2BIgtXt4,2983 +pip/_internal/commands/wheel.py,sha256=TMU5ZhjLo7BIZQApGPsYfoCsbGTnvP-N9jkgPJXhj1Y,7170 +pip/_internal/configuration.py,sha256=MgKrLFBJBkF3t2VJM4tvlnEspfSuS4scp_LhHWh53nY,14222 +pip/_internal/distributions/__init__.py,sha256=ECBUW5Gtu9TjJwyFLvim-i6kUMYVuikNh9I5asL6tbA,959 +pip/_internal/distributions/__pycache__/__init__.cpython-38.pyc,, +pip/_internal/distributions/__pycache__/base.cpython-38.pyc,, +pip/_internal/distributions/__pycache__/installed.cpython-38.pyc,, +pip/_internal/distributions/__pycache__/sdist.cpython-38.pyc,, +pip/_internal/distributions/__pycache__/wheel.cpython-38.pyc,, +pip/_internal/distributions/base.py,sha256=ruprpM_L2T2HNi3KLUHlbHimZ1sWVw-3Q0Lb8O7TDAI,1425 +pip/_internal/distributions/installed.py,sha256=YqlkBKr6TVP1MAYS6SG8ojud21wVOYLMZ8jMLJe9MSU,760 +pip/_internal/distributions/sdist.py,sha256=D4XTMlCwgPlK69l62GLYkNSVTVe99fR5iAcVt2EbGok,4086 +pip/_internal/distributions/wheel.py,sha256=95uD-TfaYoq3KiKBdzk9YMN4RRqJ28LNoSTS2K46gek,1294 +pip/_internal/exceptions.py,sha256=6YRuwXAK6F1iyUWKIkCIpWWN2khkAn1sZOgrFA9S8Ro,10247 +pip/_internal/index/__init__.py,sha256=vpt-JeTZefh8a-FC22ZeBSXFVbuBcXSGiILhQZJaNpQ,30 +pip/_internal/index/__pycache__/__init__.cpython-38.pyc,, +pip/_internal/index/__pycache__/collector.cpython-38.pyc,, +pip/_internal/index/__pycache__/package_finder.cpython-38.pyc,, +pip/_internal/index/collector.py,sha256=YS7Ix4oylU7ZbPTPFugh-244GSRqMvdHsGUG6nmz2gE,17892 +pip/_internal/index/package_finder.py,sha256=2Rg75AOpLj8BN1jyL8EI-Iw-Hv6ibJkrYVARCht3bX8,37542 +pip/_internal/legacy_resolve.py,sha256=L7R72I7CjVgJlPTggmA1j4b-H8NmxNu_dKVhrpGXGps,16277 +pip/_internal/locations.py,sha256=VifFEqhc7FWFV8QGoEM3CpECRY8Doq7kTytytxsEgx0,6734 +pip/_internal/main.py,sha256=IVBnUQ-FG7DK6617uEXRB5_QJqspAsBFmTmTesYkbdQ,437 +pip/_internal/models/__init__.py,sha256=3DHUd_qxpPozfzouoqa9g9ts1Czr5qaHfFxbnxriepM,63 +pip/_internal/models/__pycache__/__init__.cpython-38.pyc,, +pip/_internal/models/__pycache__/candidate.cpython-38.pyc,, +pip/_internal/models/__pycache__/format_control.cpython-38.pyc,, +pip/_internal/models/__pycache__/index.cpython-38.pyc,, +pip/_internal/models/__pycache__/link.cpython-38.pyc,, +pip/_internal/models/__pycache__/scheme.cpython-38.pyc,, +pip/_internal/models/__pycache__/search_scope.cpython-38.pyc,, +pip/_internal/models/__pycache__/selection_prefs.cpython-38.pyc,, +pip/_internal/models/__pycache__/target_python.cpython-38.pyc,, +pip/_internal/models/__pycache__/wheel.cpython-38.pyc,, +pip/_internal/models/candidate.py,sha256=Y58Bcm6oXUj0iS-yhmerlGo5CQJI2p0Ww9h6hR9zQDw,1150 +pip/_internal/models/format_control.py,sha256=ICzVjjGwfZYdX-eLLKHjMHLutEJlAGpfj09OG_eMqac,2673 +pip/_internal/models/index.py,sha256=K59A8-hVhBM20Xkahr4dTwP7OjkJyEqXH11UwHFVgqM,1060 +pip/_internal/models/link.py,sha256=y0H2ZOk0P6d1lfGUL2Pl09xFgZcRt5HwN2LElMifOpI,6827 +pip/_internal/models/scheme.py,sha256=vvhBrrno7eVDXcdKHiZWwxhPHf4VG5uSCEkC0QDR2RU,679 +pip/_internal/models/search_scope.py,sha256=2LXbU4wV8LwqdtXQXNXFYKv-IxiDI_QwSz9ZgbwtAfk,3898 +pip/_internal/models/selection_prefs.py,sha256=rPeif2KKjhTPXeMoQYffjqh10oWpXhdkxRDaPT1HO8k,1908 +pip/_internal/models/target_python.py,sha256=c-cFi6zCuo5HYbXNS3rVVpKRaHVh5yQlYEjEW23SidQ,3799 +pip/_internal/models/wheel.py,sha256=6KLuLKH5b0C5goWQXGSISRaq2UZtkHUEAU1y1Zsrwms,2766 +pip/_internal/network/__init__.py,sha256=jf6Tt5nV_7zkARBrKojIXItgejvoegVJVKUbhAa5Ioc,50 +pip/_internal/network/__pycache__/__init__.cpython-38.pyc,, +pip/_internal/network/__pycache__/auth.cpython-38.pyc,, +pip/_internal/network/__pycache__/cache.cpython-38.pyc,, +pip/_internal/network/__pycache__/download.cpython-38.pyc,, +pip/_internal/network/__pycache__/session.cpython-38.pyc,, +pip/_internal/network/__pycache__/utils.cpython-38.pyc,, +pip/_internal/network/__pycache__/xmlrpc.cpython-38.pyc,, +pip/_internal/network/auth.py,sha256=K3G1ukKb3PiH8w_UnpXTz8qQsTULO-qdbfOE9zTo1fE,11119 +pip/_internal/network/cache.py,sha256=51CExcRkXWrgMZ7WsrZ6cmijKfViD5tVgKbBvJHO1IE,2394 +pip/_internal/network/download.py,sha256=3D9vdJmVwmCUMxzC-TaVI_GvVOpQna3BLEYNPCSx3Fc,6260 +pip/_internal/network/session.py,sha256=u1IXQfv21R1xv86ulyiB58-be4sYm90eFB0Wp8fVMYw,14702 +pip/_internal/network/utils.py,sha256=iiixo1OeaQ3niUWiBjg59PN6f1w7vvTww1vFriTD_IU,1959 +pip/_internal/network/xmlrpc.py,sha256=AL115M3vFJ8xiHVJneb8Hi0ZFeRvdPhblC89w25OG5s,1597 +pip/_internal/operations/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0 +pip/_internal/operations/__pycache__/__init__.cpython-38.pyc,, +pip/_internal/operations/__pycache__/check.cpython-38.pyc,, +pip/_internal/operations/__pycache__/freeze.cpython-38.pyc,, +pip/_internal/operations/__pycache__/prepare.cpython-38.pyc,, +pip/_internal/operations/build/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0 +pip/_internal/operations/build/__pycache__/__init__.cpython-38.pyc,, +pip/_internal/operations/build/__pycache__/metadata.cpython-38.pyc,, +pip/_internal/operations/build/__pycache__/metadata_legacy.cpython-38.pyc,, +pip/_internal/operations/build/__pycache__/wheel.cpython-38.pyc,, +pip/_internal/operations/build/__pycache__/wheel_legacy.cpython-38.pyc,, +pip/_internal/operations/build/metadata.py,sha256=yHMi5gHYXcXyHcvUPWHdO-UyOo3McFWljn_nHfM1O9c,1307 +pip/_internal/operations/build/metadata_legacy.py,sha256=4n6N7BTysqVmEpITzT2UVClyt0Peij_Im8Qm965IWB4,3957 +pip/_internal/operations/build/wheel.py,sha256=ntltdNP6D2Tpr4V0agssu6rE0F9LaBpJkYT6zSdhEbw,1469 +pip/_internal/operations/build/wheel_legacy.py,sha256=DYSxQKutwSZnmNvWkwsl2HzE2XQBxV0i0wTphjtUe90,3349 +pip/_internal/operations/check.py,sha256=a6uHG0daoWpmSPCdL7iYJaGQYZ-CRvPvTnCv2PnIIs0,5353 +pip/_internal/operations/freeze.py,sha256=td4BeRnW10EXFTZrx6VgygO3CrjqD5B9f0BGzjQm-Ew,10180 +pip/_internal/operations/install/__init__.py,sha256=mX7hyD2GNBO2mFGokDQ30r_GXv7Y_PLdtxcUv144e-s,51 +pip/_internal/operations/install/__pycache__/__init__.cpython-38.pyc,, +pip/_internal/operations/install/__pycache__/editable_legacy.cpython-38.pyc,, +pip/_internal/operations/install/__pycache__/legacy.cpython-38.pyc,, +pip/_internal/operations/install/__pycache__/wheel.cpython-38.pyc,, +pip/_internal/operations/install/editable_legacy.py,sha256=rJ_xs2qtDUjpY2-n6eYlVyZiNoKbOtZXZrYrcnIELt4,1488 +pip/_internal/operations/install/legacy.py,sha256=eBV8gHbO9sBlBc-4nuR3Sd2nikHgEcnC9khfeLiypio,4566 +pip/_internal/operations/install/wheel.py,sha256=xdCjH6uIUyg39Pf8tUaMFUN4a7eozJAFMb_wKcgQlsY,23012 +pip/_internal/operations/prepare.py,sha256=ro2teBlbBpkRJhBKraP9CoJgVLpueSk62ziWhRToXww,20942 +pip/_internal/pep425tags.py,sha256=SlIQokevkoKnXhoK3PZvXiDoj8hFKoJ7thDifDtga3k,5490 +pip/_internal/pyproject.py,sha256=kB966ZCSxiZQRa3W2RXN9as5pRuKW6Elnb4xdqDxASg,7404 +pip/_internal/req/__init__.py,sha256=UVaYPlHZVGRBQQPjvGC_6jJDQtewXm0ws-8Lxhg_TiY,2671 +pip/_internal/req/__pycache__/__init__.cpython-38.pyc,, +pip/_internal/req/__pycache__/constructors.cpython-38.pyc,, +pip/_internal/req/__pycache__/req_file.cpython-38.pyc,, +pip/_internal/req/__pycache__/req_install.cpython-38.pyc,, +pip/_internal/req/__pycache__/req_set.cpython-38.pyc,, +pip/_internal/req/__pycache__/req_tracker.cpython-38.pyc,, +pip/_internal/req/__pycache__/req_uninstall.cpython-38.pyc,, +pip/_internal/req/constructors.py,sha256=w5-kWWVCqlSqcIBitw86yq7XGMPpKrHDfQZSE2mJ_xc,14388 +pip/_internal/req/req_file.py,sha256=ECqRUicCw5Y08R1YynZAAp8dSKQhDXoc1Q-mY3a9b6I,18485 +pip/_internal/req/req_install.py,sha256=wjsIr4lDpbVSLqANKJI9mXwRVHaRxcnj8q30UiHoLRA,30442 +pip/_internal/req/req_set.py,sha256=GsrKmupRKhNMhjkofVfCEHEHfgEvYBxClaQH5xLBQHg,8066 +pip/_internal/req/req_tracker.py,sha256=27fvVG8Y2MJS1KpU2rBMnQyUEMHG4lkHT_bzbzQK-c0,4723 +pip/_internal/req/req_uninstall.py,sha256=DWnOsuyYGju6-sylyoCm7GtUNevn9qMAVhjAGLcdXUE,23609 +pip/_internal/self_outdated_check.py,sha256=3KO1pTJUuYaiV9X0t87I9PimkGL82HbhLWbocqKZpBU,8009 +pip/_internal/utils/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0 +pip/_internal/utils/__pycache__/__init__.cpython-38.pyc,, +pip/_internal/utils/__pycache__/appdirs.cpython-38.pyc,, +pip/_internal/utils/__pycache__/compat.cpython-38.pyc,, +pip/_internal/utils/__pycache__/deprecation.cpython-38.pyc,, +pip/_internal/utils/__pycache__/distutils_args.cpython-38.pyc,, +pip/_internal/utils/__pycache__/encoding.cpython-38.pyc,, +pip/_internal/utils/__pycache__/entrypoints.cpython-38.pyc,, +pip/_internal/utils/__pycache__/filesystem.cpython-38.pyc,, +pip/_internal/utils/__pycache__/filetypes.cpython-38.pyc,, +pip/_internal/utils/__pycache__/glibc.cpython-38.pyc,, +pip/_internal/utils/__pycache__/hashes.cpython-38.pyc,, +pip/_internal/utils/__pycache__/inject_securetransport.cpython-38.pyc,, +pip/_internal/utils/__pycache__/logging.cpython-38.pyc,, +pip/_internal/utils/__pycache__/marker_files.cpython-38.pyc,, +pip/_internal/utils/__pycache__/misc.cpython-38.pyc,, +pip/_internal/utils/__pycache__/models.cpython-38.pyc,, +pip/_internal/utils/__pycache__/packaging.cpython-38.pyc,, +pip/_internal/utils/__pycache__/pkg_resources.cpython-38.pyc,, +pip/_internal/utils/__pycache__/setuptools_build.cpython-38.pyc,, +pip/_internal/utils/__pycache__/subprocess.cpython-38.pyc,, +pip/_internal/utils/__pycache__/temp_dir.cpython-38.pyc,, +pip/_internal/utils/__pycache__/typing.cpython-38.pyc,, +pip/_internal/utils/__pycache__/ui.cpython-38.pyc,, +pip/_internal/utils/__pycache__/unpacking.cpython-38.pyc,, +pip/_internal/utils/__pycache__/urls.cpython-38.pyc,, +pip/_internal/utils/__pycache__/virtualenv.cpython-38.pyc,, +pip/_internal/utils/__pycache__/wheel.cpython-38.pyc,, +pip/_internal/utils/appdirs.py,sha256=frpKbfJiyKLgpPDYNDrPtkfaZ0akY9SyB7ryPV29sMg,1144 +pip/_internal/utils/compat.py,sha256=D7FKGLBdQwWH-dHIGaoWMawDZWBYApvtJVL1kFPJ930,8869 +pip/_internal/utils/deprecation.py,sha256=pBnNogoA4UGTxa_JDnPXBRRYpKMbExAhXpBwAwklOBs,3318 +pip/_internal/utils/distutils_args.py,sha256=a56mblNxk9BGifbpEETG61mmBrqhjtjRkJ4HYn-oOEE,1350 +pip/_internal/utils/encoding.py,sha256=hxZz0t3Whw3d4MHQEiofxalTlfKwxFdLc8fpeGfhKo8,1320 +pip/_internal/utils/entrypoints.py,sha256=vHcNpnksCv6mllihU6hfifdsKPEjwcaJ1aLIXEaynaU,1152 +pip/_internal/utils/filesystem.py,sha256=PXa3vMcz4mbEKtkD0joFI8pBwddLQxhfPFOkVH5xjfE,5255 +pip/_internal/utils/filetypes.py,sha256=R2FwzoeX7b-rZALOXx5cuO8VPPMhUQ4ne7wm3n3IcWA,571 +pip/_internal/utils/glibc.py,sha256=LOeNGgawCKS-4ke9fii78fwXD73dtNav3uxz1Bf-Ab8,3297 +pip/_internal/utils/hashes.py,sha256=my-wSnAWEDvl_8rQaOQcVIWjwh1-f_QiEvGy9TPf53U,3942 +pip/_internal/utils/inject_securetransport.py,sha256=M17ZlFVY66ApgeASVjKKLKNz0LAfk-SyU0HZ4ZB6MmI,810 +pip/_internal/utils/logging.py,sha256=aJL7NldPhS5KGFof6Qt3o3MG5cjm5TOoo7bGRu9_wsg,13033 +pip/_internal/utils/marker_files.py,sha256=CO5djQlrPIozJpJybViH_insoAaBGY1aqEt6-cC-iW0,741 +pip/_internal/utils/misc.py,sha256=uIb58Hiu_g2HRORo2aMcgnW_7R5d-5wUAuoW0fA2ZME,26085 +pip/_internal/utils/models.py,sha256=IA0hw_T4awQzui0kqfIEASm5yLtgZAB08ag59Nip5G8,1148 +pip/_internal/utils/packaging.py,sha256=VtiwcAAL7LBi7tGL2je7LeW4bE11KMHGCsJ1NZY5XtM,3035 +pip/_internal/utils/pkg_resources.py,sha256=ZX-k7V5q_aNWyDse92nN7orN1aCpRLsaxzpkBZ1XKzU,1254 +pip/_internal/utils/setuptools_build.py,sha256=DouaVolV9olDDFIIN9IszaL-FHdNaZt10ufOZFH9ZAU,5070 +pip/_internal/utils/subprocess.py,sha256=Ph3x5eHQBxFotyGhpZN8asSMBud-BBkmgaNfARG-di8,9922 +pip/_internal/utils/temp_dir.py,sha256=87Ib8aNic_hoSDEmUYJHTQIn5-prL2AYL5u_yZ3s4sI,7768 +pip/_internal/utils/typing.py,sha256=xkYwOeHlf4zsHXBDC4310HtEqwhQcYXFPq2h35Tcrl0,1401 +pip/_internal/utils/ui.py,sha256=0FNxXlGtbpPtTviv2oXS9t8bQG_NBdfUgP4GbubhS9U,13911 +pip/_internal/utils/unpacking.py,sha256=M944JTSiapBOSKLWu7lbawpVHSE7flfzZTEr3TAG7v8,9438 +pip/_internal/utils/urls.py,sha256=aNV9wq5ClUmrz6sG-al7hEWJ4ToitOy7l82CmFGFNW8,1481 +pip/_internal/utils/virtualenv.py,sha256=Q3S1WPlI7JWpGOT2jUVJ8l2chm_k7VPJ9cHA_cUluEU,3396 +pip/_internal/utils/wheel.py,sha256=grTRwZtMQwApwbbSPmRVLtac6FKy6SVKeCXNkWyyePA,7302 +pip/_internal/vcs/__init__.py,sha256=viJxJRqRE_mVScum85bgQIXAd6o0ozFt18VpC-qIJrM,617 +pip/_internal/vcs/__pycache__/__init__.cpython-38.pyc,, +pip/_internal/vcs/__pycache__/bazaar.cpython-38.pyc,, +pip/_internal/vcs/__pycache__/git.cpython-38.pyc,, +pip/_internal/vcs/__pycache__/mercurial.cpython-38.pyc,, +pip/_internal/vcs/__pycache__/subversion.cpython-38.pyc,, +pip/_internal/vcs/__pycache__/versioncontrol.cpython-38.pyc,, +pip/_internal/vcs/bazaar.py,sha256=84q1-kj1_nJ9AMzMu8RmMp-riRZu81M7K9kowcYgi3U,3957 +pip/_internal/vcs/git.py,sha256=X0j5jv_x3ZnM_NP09B1ZDxW-PAmfHzqOqX7Wf5XW--0,14058 +pip/_internal/vcs/mercurial.py,sha256=2mg7BdYI_Fe00fF6omaNccFQLPHBsDBG5CAEzvqn5sA,5110 +pip/_internal/vcs/subversion.py,sha256=Fpwy71AmuqXnoKi6h1SrXRtPjEMn8fieuM1O4j01IBg,12292 +pip/_internal/vcs/versioncontrol.py,sha256=nqoaM1_rzx24WnHtihXA8RcPpnUae0sV2sR_LS_5HFA,22600 +pip/_internal/wheel_builder.py,sha256=gr9jE14W5ZuYblpldo-tpRuyG0e0AVmHLttImuAvXlE,9441 +pip/_vendor/__init__.py,sha256=v3ZUr2bBNfKRfNOPnrjq2aU7ZjgGiamksWvYlJM5nco,4894 +pip/_vendor/__pycache__/__init__.cpython-38.pyc,, diff --git a/env/lib/python3.8/site-packages/pip-20.0.2.dist-info/WHEEL b/env/lib/python3.8/site-packages/pip-20.0.2.dist-info/WHEEL new file mode 100644 index 000000000..ef99c6cf3 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip-20.0.2.dist-info/WHEEL @@ -0,0 +1,6 @@ +Wheel-Version: 1.0 +Generator: bdist_wheel (0.34.2) +Root-Is-Purelib: true +Tag: py2-none-any +Tag: py3-none-any + diff --git a/env/lib/python3.8/site-packages/pip-20.0.2.dist-info/entry_points.txt b/env/lib/python3.8/site-packages/pip-20.0.2.dist-info/entry_points.txt new file mode 100644 index 000000000..d48bd8a85 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip-20.0.2.dist-info/entry_points.txt @@ -0,0 +1,5 @@ +[console_scripts] +pip = pip._internal.cli.main:main +pip3 = pip._internal.cli.main:main +pip3.8 = pip._internal.cli.main:main + diff --git a/env/lib/python3.8/site-packages/pip-20.0.2.dist-info/top_level.txt b/env/lib/python3.8/site-packages/pip-20.0.2.dist-info/top_level.txt new file mode 100644 index 000000000..a1b589e38 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip-20.0.2.dist-info/top_level.txt @@ -0,0 +1 @@ +pip diff --git a/env/lib/python3.8/site-packages/pip/__init__.py b/env/lib/python3.8/site-packages/pip/__init__.py new file mode 100644 index 000000000..827a4e20a --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/__init__.py @@ -0,0 +1,18 @@ +from pip._internal.utils.typing import MYPY_CHECK_RUNNING + +if MYPY_CHECK_RUNNING: + from typing import List, Optional + + +__version__ = "20.0.2" + + +def main(args=None): + # type: (Optional[List[str]]) -> int + """This is an internal API only meant for use by pip's own console scripts. + + For additional details, see https://github.com/pypa/pip/issues/7498. + """ + from pip._internal.utils.entrypoints import _wrapper + + return _wrapper(args) diff --git a/env/lib/python3.8/site-packages/pip/__main__.py b/env/lib/python3.8/site-packages/pip/__main__.py new file mode 100644 index 000000000..e83b9e056 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/__main__.py @@ -0,0 +1,19 @@ +from __future__ import absolute_import + +import os +import sys + +# If we are running from a wheel, add the wheel to sys.path +# This allows the usage python pip-*.whl/pip install pip-*.whl +if __package__ == '': + # __file__ is pip-*.whl/pip/__main__.py + # first dirname call strips of '/__main__.py', second strips off '/pip' + # Resulting path is the name of the wheel itself + # Add that to sys.path so we can import pip + path = os.path.dirname(os.path.dirname(__file__)) + sys.path.insert(0, path) + +from pip._internal.cli.main import main as _main # isort:skip # noqa + +if __name__ == '__main__': + sys.exit(_main()) diff --git a/env/lib/python3.8/site-packages/pip/_internal/__init__.py b/env/lib/python3.8/site-packages/pip/_internal/__init__.py new file mode 100644 index 000000000..3aa8a4693 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/__init__.py @@ -0,0 +1,18 @@ +#!/usr/bin/env python +import pip._internal.utils.inject_securetransport # noqa +from pip._internal.utils.typing import MYPY_CHECK_RUNNING + +if MYPY_CHECK_RUNNING: + from typing import Optional, List + + +def main(args=None): + # type: (Optional[List[str]]) -> int + """This is preserved for old console scripts that may still be referencing + it. + + For additional details, see https://github.com/pypa/pip/issues/7498. + """ + from pip._internal.utils.entrypoints import _wrapper + + return _wrapper(args) diff --git a/env/lib/python3.8/site-packages/pip/_internal/build_env.py b/env/lib/python3.8/site-packages/pip/_internal/build_env.py new file mode 100644 index 000000000..f55f0e6b8 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/build_env.py @@ -0,0 +1,221 @@ +"""Build Environment used for isolation during sdist building +""" + +# The following comment should be removed at some point in the future. +# mypy: strict-optional=False +# mypy: disallow-untyped-defs=False + +import logging +import os +import sys +import textwrap +from collections import OrderedDict +from distutils.sysconfig import get_python_lib +from sysconfig import get_paths + +from pip._vendor.pkg_resources import Requirement, VersionConflict, WorkingSet + +from pip import __file__ as pip_location +from pip._internal.utils.subprocess import call_subprocess +from pip._internal.utils.temp_dir import TempDirectory +from pip._internal.utils.typing import MYPY_CHECK_RUNNING +from pip._internal.utils.ui import open_spinner + +if MYPY_CHECK_RUNNING: + from typing import Tuple, Set, Iterable, Optional, List + from pip._internal.index.package_finder import PackageFinder + +logger = logging.getLogger(__name__) + + +class _Prefix: + + def __init__(self, path): + # type: (str) -> None + self.path = path + self.setup = False + self.bin_dir = get_paths( + 'nt' if os.name == 'nt' else 'posix_prefix', + vars={'base': path, 'platbase': path} + )['scripts'] + # Note: prefer distutils' sysconfig to get the + # library paths so PyPy is correctly supported. + purelib = get_python_lib(plat_specific=False, prefix=path) + platlib = get_python_lib(plat_specific=True, prefix=path) + if purelib == platlib: + self.lib_dirs = [purelib] + else: + self.lib_dirs = [purelib, platlib] + + +class BuildEnvironment(object): + """Creates and manages an isolated environment to install build deps + """ + + def __init__(self): + # type: () -> None + self._temp_dir = TempDirectory(kind="build-env") + + self._prefixes = OrderedDict(( + (name, _Prefix(os.path.join(self._temp_dir.path, name))) + for name in ('normal', 'overlay') + )) + + self._bin_dirs = [] # type: List[str] + self._lib_dirs = [] # type: List[str] + for prefix in reversed(list(self._prefixes.values())): + self._bin_dirs.append(prefix.bin_dir) + self._lib_dirs.extend(prefix.lib_dirs) + + # Customize site to: + # - ensure .pth files are honored + # - prevent access to system site packages + system_sites = { + os.path.normcase(site) for site in ( + get_python_lib(plat_specific=False), + get_python_lib(plat_specific=True), + ) + } + self._site_dir = os.path.join(self._temp_dir.path, 'site') + if not os.path.exists(self._site_dir): + os.mkdir(self._site_dir) + with open(os.path.join(self._site_dir, 'sitecustomize.py'), 'w') as fp: + fp.write(textwrap.dedent( + ''' + import os, site, sys + + # First, drop system-sites related paths. + original_sys_path = sys.path[:] + known_paths = set() + for path in {system_sites!r}: + site.addsitedir(path, known_paths=known_paths) + system_paths = set( + os.path.normcase(path) + for path in sys.path[len(original_sys_path):] + ) + original_sys_path = [ + path for path in original_sys_path + if os.path.normcase(path) not in system_paths + ] + sys.path = original_sys_path + + # Second, add lib directories. + # ensuring .pth file are processed. + for path in {lib_dirs!r}: + assert not path in sys.path + site.addsitedir(path) + ''' + ).format(system_sites=system_sites, lib_dirs=self._lib_dirs)) + + def __enter__(self): + self._save_env = { + name: os.environ.get(name, None) + for name in ('PATH', 'PYTHONNOUSERSITE', 'PYTHONPATH') + } + + path = self._bin_dirs[:] + old_path = self._save_env['PATH'] + if old_path: + path.extend(old_path.split(os.pathsep)) + + pythonpath = [self._site_dir] + + os.environ.update({ + 'PATH': os.pathsep.join(path), + 'PYTHONNOUSERSITE': '1', + 'PYTHONPATH': os.pathsep.join(pythonpath), + }) + + def __exit__(self, exc_type, exc_val, exc_tb): + for varname, old_value in self._save_env.items(): + if old_value is None: + os.environ.pop(varname, None) + else: + os.environ[varname] = old_value + + def cleanup(self): + # type: () -> None + self._temp_dir.cleanup() + + def check_requirements(self, reqs): + # type: (Iterable[str]) -> Tuple[Set[Tuple[str, str]], Set[str]] + """Return 2 sets: + - conflicting requirements: set of (installed, wanted) reqs tuples + - missing requirements: set of reqs + """ + missing = set() + conflicting = set() + if reqs: + ws = WorkingSet(self._lib_dirs) + for req in reqs: + try: + if ws.find(Requirement.parse(req)) is None: + missing.add(req) + except VersionConflict as e: + conflicting.add((str(e.args[0].as_requirement()), + str(e.args[1]))) + return conflicting, missing + + def install_requirements( + self, + finder, # type: PackageFinder + requirements, # type: Iterable[str] + prefix_as_string, # type: str + message # type: Optional[str] + ): + # type: (...) -> None + prefix = self._prefixes[prefix_as_string] + assert not prefix.setup + prefix.setup = True + if not requirements: + return + args = [ + sys.executable, os.path.dirname(pip_location), 'install', + '--ignore-installed', '--no-user', '--prefix', prefix.path, + '--no-warn-script-location', + ] # type: List[str] + if logger.getEffectiveLevel() <= logging.DEBUG: + args.append('-v') + for format_control in ('no_binary', 'only_binary'): + formats = getattr(finder.format_control, format_control) + args.extend(('--' + format_control.replace('_', '-'), + ','.join(sorted(formats or {':none:'})))) + + index_urls = finder.index_urls + if index_urls: + args.extend(['-i', index_urls[0]]) + for extra_index in index_urls[1:]: + args.extend(['--extra-index-url', extra_index]) + else: + args.append('--no-index') + for link in finder.find_links: + args.extend(['--find-links', link]) + + for host in finder.trusted_hosts: + args.extend(['--trusted-host', host]) + if finder.allow_all_prereleases: + args.append('--pre') + args.append('--') + args.extend(requirements) + with open_spinner(message) as spinner: + call_subprocess(args, spinner=spinner) + + +class NoOpBuildEnvironment(BuildEnvironment): + """A no-op drop-in replacement for BuildEnvironment + """ + + def __init__(self): + pass + + def __enter__(self): + pass + + def __exit__(self, exc_type, exc_val, exc_tb): + pass + + def cleanup(self): + pass + + def install_requirements(self, finder, requirements, prefix, message): + raise NotImplementedError() diff --git a/env/lib/python3.8/site-packages/pip/_internal/cache.py b/env/lib/python3.8/site-packages/pip/_internal/cache.py new file mode 100644 index 000000000..abecd78f8 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/cache.py @@ -0,0 +1,329 @@ +"""Cache Management +""" + +# The following comment should be removed at some point in the future. +# mypy: strict-optional=False + +import hashlib +import json +import logging +import os + +from pip._vendor.packaging.tags import interpreter_name, interpreter_version +from pip._vendor.packaging.utils import canonicalize_name + +from pip._internal.exceptions import InvalidWheelFilename +from pip._internal.models.link import Link +from pip._internal.models.wheel import Wheel +from pip._internal.utils.temp_dir import TempDirectory +from pip._internal.utils.typing import MYPY_CHECK_RUNNING +from pip._internal.utils.urls import path_to_url + +if MYPY_CHECK_RUNNING: + from typing import Optional, Set, List, Any, Dict + + from pip._vendor.packaging.tags import Tag + + from pip._internal.models.format_control import FormatControl + +logger = logging.getLogger(__name__) + + +def _hash_dict(d): + # type: (Dict[str, str]) -> str + """Return a stable sha224 of a dictionary.""" + s = json.dumps(d, sort_keys=True, separators=(",", ":"), ensure_ascii=True) + return hashlib.sha224(s.encode("ascii")).hexdigest() + + +class Cache(object): + """An abstract class - provides cache directories for data from links + + + :param cache_dir: The root of the cache. + :param format_control: An object of FormatControl class to limit + binaries being read from the cache. + :param allowed_formats: which formats of files the cache should store. + ('binary' and 'source' are the only allowed values) + """ + + def __init__(self, cache_dir, format_control, allowed_formats): + # type: (str, FormatControl, Set[str]) -> None + super(Cache, self).__init__() + assert not cache_dir or os.path.isabs(cache_dir) + self.cache_dir = cache_dir or None + self.format_control = format_control + self.allowed_formats = allowed_formats + + _valid_formats = {"source", "binary"} + assert self.allowed_formats.union(_valid_formats) == _valid_formats + + def _get_cache_path_parts_legacy(self, link): + # type: (Link) -> List[str] + """Get parts of part that must be os.path.joined with cache_dir + + Legacy cache key (pip < 20) for compatibility with older caches. + """ + + # We want to generate an url to use as our cache key, we don't want to + # just re-use the URL because it might have other items in the fragment + # and we don't care about those. + key_parts = [link.url_without_fragment] + if link.hash_name is not None and link.hash is not None: + key_parts.append("=".join([link.hash_name, link.hash])) + key_url = "#".join(key_parts) + + # Encode our key url with sha224, we'll use this because it has similar + # security properties to sha256, but with a shorter total output (and + # thus less secure). However the differences don't make a lot of + # difference for our use case here. + hashed = hashlib.sha224(key_url.encode()).hexdigest() + + # We want to nest the directories some to prevent having a ton of top + # level directories where we might run out of sub directories on some + # FS. + parts = [hashed[:2], hashed[2:4], hashed[4:6], hashed[6:]] + + return parts + + def _get_cache_path_parts(self, link): + # type: (Link) -> List[str] + """Get parts of part that must be os.path.joined with cache_dir + """ + + # We want to generate an url to use as our cache key, we don't want to + # just re-use the URL because it might have other items in the fragment + # and we don't care about those. + key_parts = {"url": link.url_without_fragment} + if link.hash_name is not None and link.hash is not None: + key_parts[link.hash_name] = link.hash + if link.subdirectory_fragment: + key_parts["subdirectory"] = link.subdirectory_fragment + + # Include interpreter name, major and minor version in cache key + # to cope with ill-behaved sdists that build a different wheel + # depending on the python version their setup.py is being run on, + # and don't encode the difference in compatibility tags. + # https://github.com/pypa/pip/issues/7296 + key_parts["interpreter_name"] = interpreter_name() + key_parts["interpreter_version"] = interpreter_version() + + # Encode our key url with sha224, we'll use this because it has similar + # security properties to sha256, but with a shorter total output (and + # thus less secure). However the differences don't make a lot of + # difference for our use case here. + hashed = _hash_dict(key_parts) + + # We want to nest the directories some to prevent having a ton of top + # level directories where we might run out of sub directories on some + # FS. + parts = [hashed[:2], hashed[2:4], hashed[4:6], hashed[6:]] + + return parts + + def _get_candidates(self, link, canonical_package_name): + # type: (Link, Optional[str]) -> List[Any] + can_not_cache = ( + not self.cache_dir or + not canonical_package_name or + not link + ) + if can_not_cache: + return [] + + formats = self.format_control.get_allowed_formats( + canonical_package_name + ) + if not self.allowed_formats.intersection(formats): + return [] + + candidates = [] + path = self.get_path_for_link(link) + if os.path.isdir(path): + for candidate in os.listdir(path): + candidates.append((candidate, path)) + # TODO remove legacy path lookup in pip>=21 + legacy_path = self.get_path_for_link_legacy(link) + if os.path.isdir(legacy_path): + for candidate in os.listdir(legacy_path): + candidates.append((candidate, legacy_path)) + return candidates + + def get_path_for_link_legacy(self, link): + # type: (Link) -> str + raise NotImplementedError() + + def get_path_for_link(self, link): + # type: (Link) -> str + """Return a directory to store cached items in for link. + """ + raise NotImplementedError() + + def get( + self, + link, # type: Link + package_name, # type: Optional[str] + supported_tags, # type: List[Tag] + ): + # type: (...) -> Link + """Returns a link to a cached item if it exists, otherwise returns the + passed link. + """ + raise NotImplementedError() + + def cleanup(self): + # type: () -> None + pass + + +class SimpleWheelCache(Cache): + """A cache of wheels for future installs. + """ + + def __init__(self, cache_dir, format_control): + # type: (str, FormatControl) -> None + super(SimpleWheelCache, self).__init__( + cache_dir, format_control, {"binary"} + ) + + def get_path_for_link_legacy(self, link): + # type: (Link) -> str + parts = self._get_cache_path_parts_legacy(link) + return os.path.join(self.cache_dir, "wheels", *parts) + + def get_path_for_link(self, link): + # type: (Link) -> str + """Return a directory to store cached wheels for link + + Because there are M wheels for any one sdist, we provide a directory + to cache them in, and then consult that directory when looking up + cache hits. + + We only insert things into the cache if they have plausible version + numbers, so that we don't contaminate the cache with things that were + not unique. E.g. ./package might have dozens of installs done for it + and build a version of 0.0...and if we built and cached a wheel, we'd + end up using the same wheel even if the source has been edited. + + :param link: The link of the sdist for which this will cache wheels. + """ + parts = self._get_cache_path_parts(link) + + # Store wheels within the root cache_dir + return os.path.join(self.cache_dir, "wheels", *parts) + + def get( + self, + link, # type: Link + package_name, # type: Optional[str] + supported_tags, # type: List[Tag] + ): + # type: (...) -> Link + candidates = [] + + if not package_name: + return link + + canonical_package_name = canonicalize_name(package_name) + for wheel_name, wheel_dir in self._get_candidates( + link, canonical_package_name + ): + try: + wheel = Wheel(wheel_name) + except InvalidWheelFilename: + continue + if canonicalize_name(wheel.name) != canonical_package_name: + logger.debug( + "Ignoring cached wheel {} for {} as it " + "does not match the expected distribution name {}.".format( + wheel_name, link, package_name + ) + ) + continue + if not wheel.supported(supported_tags): + # Built for a different python/arch/etc + continue + candidates.append( + ( + wheel.support_index_min(supported_tags), + wheel_name, + wheel_dir, + ) + ) + + if not candidates: + return link + + _, wheel_name, wheel_dir = min(candidates) + return Link(path_to_url(os.path.join(wheel_dir, wheel_name))) + + +class EphemWheelCache(SimpleWheelCache): + """A SimpleWheelCache that creates it's own temporary cache directory + """ + + def __init__(self, format_control): + # type: (FormatControl) -> None + self._temp_dir = TempDirectory(kind="ephem-wheel-cache") + + super(EphemWheelCache, self).__init__( + self._temp_dir.path, format_control + ) + + def cleanup(self): + # type: () -> None + self._temp_dir.cleanup() + + +class WheelCache(Cache): + """Wraps EphemWheelCache and SimpleWheelCache into a single Cache + + This Cache allows for gracefully degradation, using the ephem wheel cache + when a certain link is not found in the simple wheel cache first. + """ + + def __init__(self, cache_dir, format_control): + # type: (str, FormatControl) -> None + super(WheelCache, self).__init__( + cache_dir, format_control, {'binary'} + ) + self._wheel_cache = SimpleWheelCache(cache_dir, format_control) + self._ephem_cache = EphemWheelCache(format_control) + + def get_path_for_link_legacy(self, link): + # type: (Link) -> str + return self._wheel_cache.get_path_for_link_legacy(link) + + def get_path_for_link(self, link): + # type: (Link) -> str + return self._wheel_cache.get_path_for_link(link) + + def get_ephem_path_for_link(self, link): + # type: (Link) -> str + return self._ephem_cache.get_path_for_link(link) + + def get( + self, + link, # type: Link + package_name, # type: Optional[str] + supported_tags, # type: List[Tag] + ): + # type: (...) -> Link + retval = self._wheel_cache.get( + link=link, + package_name=package_name, + supported_tags=supported_tags, + ) + if retval is not link: + return retval + + return self._ephem_cache.get( + link=link, + package_name=package_name, + supported_tags=supported_tags, + ) + + def cleanup(self): + # type: () -> None + self._wheel_cache.cleanup() + self._ephem_cache.cleanup() diff --git a/env/lib/python3.8/site-packages/pip/_internal/cli/__init__.py b/env/lib/python3.8/site-packages/pip/_internal/cli/__init__.py new file mode 100644 index 000000000..e589bb917 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/cli/__init__.py @@ -0,0 +1,4 @@ +"""Subpackage containing all of pip's command line interface related code +""" + +# This file intentionally does not import submodules diff --git a/env/lib/python3.8/site-packages/pip/_internal/cli/autocompletion.py b/env/lib/python3.8/site-packages/pip/_internal/cli/autocompletion.py new file mode 100644 index 000000000..329de6025 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/cli/autocompletion.py @@ -0,0 +1,164 @@ +"""Logic that powers autocompletion installed by ``pip completion``. +""" + +import optparse +import os +import sys +from itertools import chain + +from pip._internal.cli.main_parser import create_main_parser +from pip._internal.commands import commands_dict, create_command +from pip._internal.utils.misc import get_installed_distributions +from pip._internal.utils.typing import MYPY_CHECK_RUNNING + +if MYPY_CHECK_RUNNING: + from typing import Any, Iterable, List, Optional + + +def autocomplete(): + # type: () -> None + """Entry Point for completion of main and subcommand options. + """ + # Don't complete if user hasn't sourced bash_completion file. + if 'PIP_AUTO_COMPLETE' not in os.environ: + return + cwords = os.environ['COMP_WORDS'].split()[1:] + cword = int(os.environ['COMP_CWORD']) + try: + current = cwords[cword - 1] + except IndexError: + current = '' + + parser = create_main_parser() + subcommands = list(commands_dict) + options = [] + + # subcommand + subcommand_name = None # type: Optional[str] + for word in cwords: + if word in subcommands: + subcommand_name = word + break + # subcommand options + if subcommand_name is not None: + # special case: 'help' subcommand has no options + if subcommand_name == 'help': + sys.exit(1) + # special case: list locally installed dists for show and uninstall + should_list_installed = ( + subcommand_name in ['show', 'uninstall'] and + not current.startswith('-') + ) + if should_list_installed: + installed = [] + lc = current.lower() + for dist in get_installed_distributions(local_only=True): + if dist.key.startswith(lc) and dist.key not in cwords[1:]: + installed.append(dist.key) + # if there are no dists installed, fall back to option completion + if installed: + for dist in installed: + print(dist) + sys.exit(1) + + subcommand = create_command(subcommand_name) + + for opt in subcommand.parser.option_list_all: + if opt.help != optparse.SUPPRESS_HELP: + for opt_str in opt._long_opts + opt._short_opts: + options.append((opt_str, opt.nargs)) + + # filter out previously specified options from available options + prev_opts = [x.split('=')[0] for x in cwords[1:cword - 1]] + options = [(x, v) for (x, v) in options if x not in prev_opts] + # filter options by current input + options = [(k, v) for k, v in options if k.startswith(current)] + # get completion type given cwords and available subcommand options + completion_type = get_path_completion_type( + cwords, cword, subcommand.parser.option_list_all, + ) + # get completion files and directories if ``completion_type`` is + # ````, ```` or ```` + if completion_type: + paths = auto_complete_paths(current, completion_type) + options = [(path, 0) for path in paths] + for option in options: + opt_label = option[0] + # append '=' to options which require args + if option[1] and option[0][:2] == "--": + opt_label += '=' + print(opt_label) + else: + # show main parser options only when necessary + + opts = [i.option_list for i in parser.option_groups] + opts.append(parser.option_list) + flattened_opts = chain.from_iterable(opts) + if current.startswith('-'): + for opt in flattened_opts: + if opt.help != optparse.SUPPRESS_HELP: + subcommands += opt._long_opts + opt._short_opts + else: + # get completion type given cwords and all available options + completion_type = get_path_completion_type(cwords, cword, + flattened_opts) + if completion_type: + subcommands = list(auto_complete_paths(current, + completion_type)) + + print(' '.join([x for x in subcommands if x.startswith(current)])) + sys.exit(1) + + +def get_path_completion_type(cwords, cword, opts): + # type: (List[str], int, Iterable[Any]) -> Optional[str] + """Get the type of path completion (``file``, ``dir``, ``path`` or None) + + :param cwords: same as the environmental variable ``COMP_WORDS`` + :param cword: same as the environmental variable ``COMP_CWORD`` + :param opts: The available options to check + :return: path completion type (``file``, ``dir``, ``path`` or None) + """ + if cword < 2 or not cwords[cword - 2].startswith('-'): + return None + for opt in opts: + if opt.help == optparse.SUPPRESS_HELP: + continue + for o in str(opt).split('/'): + if cwords[cword - 2].split('=')[0] == o: + if not opt.metavar or any( + x in ('path', 'file', 'dir') + for x in opt.metavar.split('/')): + return opt.metavar + return None + + +def auto_complete_paths(current, completion_type): + # type: (str, str) -> Iterable[str] + """If ``completion_type`` is ``file`` or ``path``, list all regular files + and directories starting with ``current``; otherwise only list directories + starting with ``current``. + + :param current: The word to be completed + :param completion_type: path completion type(`file`, `path` or `dir`)i + :return: A generator of regular files and/or directories + """ + directory, filename = os.path.split(current) + current_path = os.path.abspath(directory) + # Don't complete paths if they can't be accessed + if not os.access(current_path, os.R_OK): + return + filename = os.path.normcase(filename) + # list all files that start with ``filename`` + file_list = (x for x in os.listdir(current_path) + if os.path.normcase(x).startswith(filename)) + for f in file_list: + opt = os.path.join(current_path, f) + comp_file = os.path.normcase(os.path.join(directory, f)) + # complete regular files when there is not ```` after option + # complete directories when there is ````, ```` or + # ````after option + if completion_type != 'dir' and os.path.isfile(opt): + yield comp_file + elif os.path.isdir(opt): + yield os.path.join(comp_file, '') diff --git a/env/lib/python3.8/site-packages/pip/_internal/cli/base_command.py b/env/lib/python3.8/site-packages/pip/_internal/cli/base_command.py new file mode 100644 index 000000000..628faa3ee --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/cli/base_command.py @@ -0,0 +1,226 @@ +"""Base Command class, and related routines""" + +from __future__ import absolute_import, print_function + +import logging +import logging.config +import optparse +import os +import platform +import sys +import traceback + +from pip._internal.cli import cmdoptions +from pip._internal.cli.command_context import CommandContextMixIn +from pip._internal.cli.parser import ( + ConfigOptionParser, + UpdatingDefaultsHelpFormatter, +) +from pip._internal.cli.status_codes import ( + ERROR, + PREVIOUS_BUILD_DIR_ERROR, + SUCCESS, + UNKNOWN_ERROR, + VIRTUALENV_NOT_FOUND, +) +from pip._internal.exceptions import ( + BadCommand, + CommandError, + InstallationError, + PreviousBuildDirError, + UninstallationError, +) +from pip._internal.utils.deprecation import deprecated +from pip._internal.utils.filesystem import check_path_owner +from pip._internal.utils.logging import BrokenStdoutLoggingError, setup_logging +from pip._internal.utils.misc import get_prog, normalize_path +from pip._internal.utils.temp_dir import global_tempdir_manager +from pip._internal.utils.typing import MYPY_CHECK_RUNNING +from pip._internal.utils.virtualenv import running_under_virtualenv + +if MYPY_CHECK_RUNNING: + from typing import List, Tuple, Any + from optparse import Values + +__all__ = ['Command'] + +logger = logging.getLogger(__name__) + + +class Command(CommandContextMixIn): + usage = None # type: str + ignore_require_venv = False # type: bool + + def __init__(self, name, summary, isolated=False): + # type: (str, str, bool) -> None + super(Command, self).__init__() + parser_kw = { + 'usage': self.usage, + 'prog': '%s %s' % (get_prog(), name), + 'formatter': UpdatingDefaultsHelpFormatter(), + 'add_help_option': False, + 'name': name, + 'description': self.__doc__, + 'isolated': isolated, + } + + self.name = name + self.summary = summary + self.parser = ConfigOptionParser(**parser_kw) + + # Commands should add options to this option group + optgroup_name = '%s Options' % self.name.capitalize() + self.cmd_opts = optparse.OptionGroup(self.parser, optgroup_name) + + # Add the general options + gen_opts = cmdoptions.make_option_group( + cmdoptions.general_group, + self.parser, + ) + self.parser.add_option_group(gen_opts) + + def handle_pip_version_check(self, options): + # type: (Values) -> None + """ + This is a no-op so that commands by default do not do the pip version + check. + """ + # Make sure we do the pip version check if the index_group options + # are present. + assert not hasattr(options, 'no_index') + + def run(self, options, args): + # type: (Values, List[Any]) -> Any + raise NotImplementedError + + def parse_args(self, args): + # type: (List[str]) -> Tuple[Any, Any] + # factored out for testability + return self.parser.parse_args(args) + + def main(self, args): + # type: (List[str]) -> int + try: + with self.main_context(): + return self._main(args) + finally: + logging.shutdown() + + def _main(self, args): + # type: (List[str]) -> int + # Intentionally set as early as possible so globally-managed temporary + # directories are available to the rest of the code. + self.enter_context(global_tempdir_manager()) + + options, args = self.parse_args(args) + + # Set verbosity so that it can be used elsewhere. + self.verbosity = options.verbose - options.quiet + + level_number = setup_logging( + verbosity=self.verbosity, + no_color=options.no_color, + user_log_file=options.log, + ) + + if ( + sys.version_info[:2] == (2, 7) and + not options.no_python_version_warning + ): + message = ( + "A future version of pip will drop support for Python 2.7. " + "More details about Python 2 support in pip, can be found at " + "https://pip.pypa.io/en/latest/development/release-process/#python-2-support" # noqa + ) + if platform.python_implementation() == "CPython": + message = ( + "Python 2.7 reached the end of its life on January " + "1st, 2020. Please upgrade your Python as Python 2.7 " + "is no longer maintained. " + ) + message + deprecated(message, replacement=None, gone_in=None) + + if options.skip_requirements_regex: + deprecated( + "--skip-requirements-regex is unsupported and will be removed", + replacement=( + "manage requirements/constraints files explicitly, " + "possibly generating them from metadata" + ), + gone_in="20.1", + issue=7297, + ) + + # TODO: Try to get these passing down from the command? + # without resorting to os.environ to hold these. + # This also affects isolated builds and it should. + + if options.no_input: + os.environ['PIP_NO_INPUT'] = '1' + + if options.exists_action: + os.environ['PIP_EXISTS_ACTION'] = ' '.join(options.exists_action) + + if options.require_venv and not self.ignore_require_venv: + # If a venv is required check if it can really be found + if not running_under_virtualenv(): + logger.critical( + 'Could not find an activated virtualenv (required).' + ) + sys.exit(VIRTUALENV_NOT_FOUND) + + if options.cache_dir: + options.cache_dir = normalize_path(options.cache_dir) + if not check_path_owner(options.cache_dir): + logger.warning( + "The directory '%s' or its parent directory is not owned " + "or is not writable by the current user. The cache " + "has been disabled. Check the permissions and owner of " + "that directory. If executing pip with sudo, you may want " + "sudo's -H flag.", + options.cache_dir, + ) + options.cache_dir = None + + try: + status = self.run(options, args) + # FIXME: all commands should return an exit status + # and when it is done, isinstance is not needed anymore + if isinstance(status, int): + return status + except PreviousBuildDirError as exc: + logger.critical(str(exc)) + logger.debug('Exception information:', exc_info=True) + + return PREVIOUS_BUILD_DIR_ERROR + except (InstallationError, UninstallationError, BadCommand) as exc: + logger.critical(str(exc)) + logger.debug('Exception information:', exc_info=True) + + return ERROR + except CommandError as exc: + logger.critical('%s', exc) + logger.debug('Exception information:', exc_info=True) + + return ERROR + except BrokenStdoutLoggingError: + # Bypass our logger and write any remaining messages to stderr + # because stdout no longer works. + print('ERROR: Pipe to stdout was broken', file=sys.stderr) + if level_number <= logging.DEBUG: + traceback.print_exc(file=sys.stderr) + + return ERROR + except KeyboardInterrupt: + logger.critical('Operation cancelled by user') + logger.debug('Exception information:', exc_info=True) + + return ERROR + except BaseException: + logger.critical('Exception:', exc_info=True) + + return UNKNOWN_ERROR + finally: + self.handle_pip_version_check(options) + + return SUCCESS diff --git a/env/lib/python3.8/site-packages/pip/_internal/cli/cmdoptions.py b/env/lib/python3.8/site-packages/pip/_internal/cli/cmdoptions.py new file mode 100644 index 000000000..447f31918 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/cli/cmdoptions.py @@ -0,0 +1,957 @@ +""" +shared options and groups + +The principle here is to define options once, but *not* instantiate them +globally. One reason being that options with action='append' can carry state +between parses. pip parses general options twice internally, and shouldn't +pass on state. To be consistent, all options will follow this design. +""" + +# The following comment should be removed at some point in the future. +# mypy: strict-optional=False + +from __future__ import absolute_import + +import logging +import os +import textwrap +import warnings +from distutils.util import strtobool +from functools import partial +from optparse import SUPPRESS_HELP, Option, OptionGroup +from textwrap import dedent + +from pip._internal.exceptions import CommandError +from pip._internal.locations import USER_CACHE_DIR, get_src_prefix +from pip._internal.models.format_control import FormatControl +from pip._internal.models.index import PyPI +from pip._internal.models.target_python import TargetPython +from pip._internal.utils.hashes import STRONG_HASHES +from pip._internal.utils.typing import MYPY_CHECK_RUNNING +from pip._internal.utils.ui import BAR_TYPES + +if MYPY_CHECK_RUNNING: + from typing import Any, Callable, Dict, Optional, Tuple + from optparse import OptionParser, Values + from pip._internal.cli.parser import ConfigOptionParser + +logger = logging.getLogger(__name__) + + +def raise_option_error(parser, option, msg): + # type: (OptionParser, Option, str) -> None + """ + Raise an option parsing error using parser.error(). + + Args: + parser: an OptionParser instance. + option: an Option instance. + msg: the error text. + """ + msg = '{} error: {}'.format(option, msg) + msg = textwrap.fill(' '.join(msg.split())) + parser.error(msg) + + +def make_option_group(group, parser): + # type: (Dict[str, Any], ConfigOptionParser) -> OptionGroup + """ + Return an OptionGroup object + group -- assumed to be dict with 'name' and 'options' keys + parser -- an optparse Parser + """ + option_group = OptionGroup(parser, group['name']) + for option in group['options']: + option_group.add_option(option()) + return option_group + + +def check_install_build_global(options, check_options=None): + # type: (Values, Optional[Values]) -> None + """Disable wheels if per-setup.py call options are set. + + :param options: The OptionParser options to update. + :param check_options: The options to check, if not supplied defaults to + options. + """ + if check_options is None: + check_options = options + + def getname(n): + # type: (str) -> Optional[Any] + return getattr(check_options, n, None) + names = ["build_options", "global_options", "install_options"] + if any(map(getname, names)): + control = options.format_control + control.disallow_binaries() + warnings.warn( + 'Disabling all use of wheels due to the use of --build-option ' + '/ --global-option / --install-option.', stacklevel=2, + ) + + +def check_dist_restriction(options, check_target=False): + # type: (Values, bool) -> None + """Function for determining if custom platform options are allowed. + + :param options: The OptionParser options. + :param check_target: Whether or not to check if --target is being used. + """ + dist_restriction_set = any([ + options.python_version, + options.platform, + options.abi, + options.implementation, + ]) + + binary_only = FormatControl(set(), {':all:'}) + sdist_dependencies_allowed = ( + options.format_control != binary_only and + not options.ignore_dependencies + ) + + # Installations or downloads using dist restrictions must not combine + # source distributions and dist-specific wheels, as they are not + # guaranteed to be locally compatible. + if dist_restriction_set and sdist_dependencies_allowed: + raise CommandError( + "When restricting platform and interpreter constraints using " + "--python-version, --platform, --abi, or --implementation, " + "either --no-deps must be set, or --only-binary=:all: must be " + "set and --no-binary must not be set (or must be set to " + ":none:)." + ) + + if check_target: + if dist_restriction_set and not options.target_dir: + raise CommandError( + "Can not use any platform or abi specific options unless " + "installing via '--target'" + ) + + +def _path_option_check(option, opt, value): + # type: (Option, str, str) -> str + return os.path.expanduser(value) + + +class PipOption(Option): + TYPES = Option.TYPES + ("path",) + TYPE_CHECKER = Option.TYPE_CHECKER.copy() + TYPE_CHECKER["path"] = _path_option_check + + +########### +# options # +########### + +help_ = partial( + Option, + '-h', '--help', + dest='help', + action='help', + help='Show help.', +) # type: Callable[..., Option] + +isolated_mode = partial( + Option, + "--isolated", + dest="isolated_mode", + action="store_true", + default=False, + help=( + "Run pip in an isolated mode, ignoring environment variables and user " + "configuration." + ), +) # type: Callable[..., Option] + +require_virtualenv = partial( + Option, + # Run only if inside a virtualenv, bail if not. + '--require-virtualenv', '--require-venv', + dest='require_venv', + action='store_true', + default=False, + help=SUPPRESS_HELP +) # type: Callable[..., Option] + +verbose = partial( + Option, + '-v', '--verbose', + dest='verbose', + action='count', + default=0, + help='Give more output. Option is additive, and can be used up to 3 times.' +) # type: Callable[..., Option] + +no_color = partial( + Option, + '--no-color', + dest='no_color', + action='store_true', + default=False, + help="Suppress colored output", +) # type: Callable[..., Option] + +version = partial( + Option, + '-V', '--version', + dest='version', + action='store_true', + help='Show version and exit.', +) # type: Callable[..., Option] + +quiet = partial( + Option, + '-q', '--quiet', + dest='quiet', + action='count', + default=0, + help=( + 'Give less output. Option is additive, and can be used up to 3' + ' times (corresponding to WARNING, ERROR, and CRITICAL logging' + ' levels).' + ), +) # type: Callable[..., Option] + +progress_bar = partial( + Option, + '--progress-bar', + dest='progress_bar', + type='choice', + choices=list(BAR_TYPES.keys()), + default='on', + help=( + 'Specify type of progress to be displayed [' + + '|'.join(BAR_TYPES.keys()) + '] (default: %default)' + ), +) # type: Callable[..., Option] + +log = partial( + PipOption, + "--log", "--log-file", "--local-log", + dest="log", + metavar="path", + type="path", + help="Path to a verbose appending log." +) # type: Callable[..., Option] + +no_input = partial( + Option, + # Don't ask for input + '--no-input', + dest='no_input', + action='store_true', + default=False, + help=SUPPRESS_HELP +) # type: Callable[..., Option] + +proxy = partial( + Option, + '--proxy', + dest='proxy', + type='str', + default='', + help="Specify a proxy in the form [user:passwd@]proxy.server:port." +) # type: Callable[..., Option] + +retries = partial( + Option, + '--retries', + dest='retries', + type='int', + default=5, + help="Maximum number of retries each connection should attempt " + "(default %default times).", +) # type: Callable[..., Option] + +timeout = partial( + Option, + '--timeout', '--default-timeout', + metavar='sec', + dest='timeout', + type='float', + default=15, + help='Set the socket timeout (default %default seconds).', +) # type: Callable[..., Option] + +skip_requirements_regex = partial( + Option, + # A regex to be used to skip requirements + '--skip-requirements-regex', + dest='skip_requirements_regex', + type='str', + default='', + help=SUPPRESS_HELP, +) # type: Callable[..., Option] + + +def exists_action(): + # type: () -> Option + return Option( + # Option when path already exist + '--exists-action', + dest='exists_action', + type='choice', + choices=['s', 'i', 'w', 'b', 'a'], + default=[], + action='append', + metavar='action', + help="Default action when a path already exists: " + "(s)witch, (i)gnore, (w)ipe, (b)ackup, (a)bort.", + ) + + +cert = partial( + PipOption, + '--cert', + dest='cert', + type='path', + metavar='path', + help="Path to alternate CA bundle.", +) # type: Callable[..., Option] + +client_cert = partial( + PipOption, + '--client-cert', + dest='client_cert', + type='path', + default=None, + metavar='path', + help="Path to SSL client certificate, a single file containing the " + "private key and the certificate in PEM format.", +) # type: Callable[..., Option] + +index_url = partial( + Option, + '-i', '--index-url', '--pypi-url', + dest='index_url', + metavar='URL', + default=PyPI.simple_url, + help="Base URL of the Python Package Index (default %default). " + "This should point to a repository compliant with PEP 503 " + "(the simple repository API) or a local directory laid out " + "in the same format.", +) # type: Callable[..., Option] + + +def extra_index_url(): + # type: () -> Option + return Option( + '--extra-index-url', + dest='extra_index_urls', + metavar='URL', + action='append', + default=[], + help="Extra URLs of package indexes to use in addition to " + "--index-url. Should follow the same rules as " + "--index-url.", + ) + + +no_index = partial( + Option, + '--no-index', + dest='no_index', + action='store_true', + default=False, + help='Ignore package index (only looking at --find-links URLs instead).', +) # type: Callable[..., Option] + + +def find_links(): + # type: () -> Option + return Option( + '-f', '--find-links', + dest='find_links', + action='append', + default=[], + metavar='url', + help="If a url or path to an html file, then parse for links to " + "archives. If a local path or file:// url that's a directory, " + "then look for archives in the directory listing.", + ) + + +def trusted_host(): + # type: () -> Option + return Option( + "--trusted-host", + dest="trusted_hosts", + action="append", + metavar="HOSTNAME", + default=[], + help="Mark this host or host:port pair as trusted, even though it " + "does not have valid or any HTTPS.", + ) + + +def constraints(): + # type: () -> Option + return Option( + '-c', '--constraint', + dest='constraints', + action='append', + default=[], + metavar='file', + help='Constrain versions using the given constraints file. ' + 'This option can be used multiple times.' + ) + + +def requirements(): + # type: () -> Option + return Option( + '-r', '--requirement', + dest='requirements', + action='append', + default=[], + metavar='file', + help='Install from the given requirements file. ' + 'This option can be used multiple times.' + ) + + +def editable(): + # type: () -> Option + return Option( + '-e', '--editable', + dest='editables', + action='append', + default=[], + metavar='path/url', + help=('Install a project in editable mode (i.e. setuptools ' + '"develop mode") from a local project path or a VCS url.'), + ) + + +def _handle_src(option, opt_str, value, parser): + # type: (Option, str, str, OptionParser) -> None + value = os.path.abspath(value) + setattr(parser.values, option.dest, value) + + +src = partial( + PipOption, + '--src', '--source', '--source-dir', '--source-directory', + dest='src_dir', + type='path', + metavar='dir', + default=get_src_prefix(), + action='callback', + callback=_handle_src, + help='Directory to check out editable projects into. ' + 'The default in a virtualenv is "/src". ' + 'The default for global installs is "/src".' +) # type: Callable[..., Option] + + +def _get_format_control(values, option): + # type: (Values, Option) -> Any + """Get a format_control object.""" + return getattr(values, option.dest) + + +def _handle_no_binary(option, opt_str, value, parser): + # type: (Option, str, str, OptionParser) -> None + existing = _get_format_control(parser.values, option) + FormatControl.handle_mutual_excludes( + value, existing.no_binary, existing.only_binary, + ) + + +def _handle_only_binary(option, opt_str, value, parser): + # type: (Option, str, str, OptionParser) -> None + existing = _get_format_control(parser.values, option) + FormatControl.handle_mutual_excludes( + value, existing.only_binary, existing.no_binary, + ) + + +def no_binary(): + # type: () -> Option + format_control = FormatControl(set(), set()) + return Option( + "--no-binary", dest="format_control", action="callback", + callback=_handle_no_binary, type="str", + default=format_control, + help="Do not use binary packages. Can be supplied multiple times, and " + "each time adds to the existing value. Accepts either :all: to " + "disable all binary packages, :none: to empty the set, or one or " + "more package names with commas between them (no colons). Note " + "that some packages are tricky to compile and may fail to " + "install when this option is used on them.", + ) + + +def only_binary(): + # type: () -> Option + format_control = FormatControl(set(), set()) + return Option( + "--only-binary", dest="format_control", action="callback", + callback=_handle_only_binary, type="str", + default=format_control, + help="Do not use source packages. Can be supplied multiple times, and " + "each time adds to the existing value. Accepts either :all: to " + "disable all source packages, :none: to empty the set, or one or " + "more package names with commas between them. Packages without " + "binary distributions will fail to install when this option is " + "used on them.", + ) + + +platform = partial( + Option, + '--platform', + dest='platform', + metavar='platform', + default=None, + help=("Only use wheels compatible with . " + "Defaults to the platform of the running system."), +) # type: Callable[..., Option] + + +# This was made a separate function for unit-testing purposes. +def _convert_python_version(value): + # type: (str) -> Tuple[Tuple[int, ...], Optional[str]] + """ + Convert a version string like "3", "37", or "3.7.3" into a tuple of ints. + + :return: A 2-tuple (version_info, error_msg), where `error_msg` is + non-None if and only if there was a parsing error. + """ + if not value: + # The empty string is the same as not providing a value. + return (None, None) + + parts = value.split('.') + if len(parts) > 3: + return ((), 'at most three version parts are allowed') + + if len(parts) == 1: + # Then we are in the case of "3" or "37". + value = parts[0] + if len(value) > 1: + parts = [value[0], value[1:]] + + try: + version_info = tuple(int(part) for part in parts) + except ValueError: + return ((), 'each version part must be an integer') + + return (version_info, None) + + +def _handle_python_version(option, opt_str, value, parser): + # type: (Option, str, str, OptionParser) -> None + """ + Handle a provided --python-version value. + """ + version_info, error_msg = _convert_python_version(value) + if error_msg is not None: + msg = ( + 'invalid --python-version value: {!r}: {}'.format( + value, error_msg, + ) + ) + raise_option_error(parser, option=option, msg=msg) + + parser.values.python_version = version_info + + +python_version = partial( + Option, + '--python-version', + dest='python_version', + metavar='python_version', + action='callback', + callback=_handle_python_version, type='str', + default=None, + help=dedent("""\ + The Python interpreter version to use for wheel and "Requires-Python" + compatibility checks. Defaults to a version derived from the running + interpreter. The version can be specified using up to three dot-separated + integers (e.g. "3" for 3.0.0, "3.7" for 3.7.0, or "3.7.3"). A major-minor + version can also be given as a string without dots (e.g. "37" for 3.7.0). + """), +) # type: Callable[..., Option] + + +implementation = partial( + Option, + '--implementation', + dest='implementation', + metavar='implementation', + default=None, + help=("Only use wheels compatible with Python " + "implementation , e.g. 'pp', 'jy', 'cp', " + " or 'ip'. If not specified, then the current " + "interpreter implementation is used. Use 'py' to force " + "implementation-agnostic wheels."), +) # type: Callable[..., Option] + + +abi = partial( + Option, + '--abi', + dest='abi', + metavar='abi', + default=None, + help=("Only use wheels compatible with Python " + "abi , e.g. 'pypy_41'. If not specified, then the " + "current interpreter abi tag is used. Generally " + "you will need to specify --implementation, " + "--platform, and --python-version when using " + "this option."), +) # type: Callable[..., Option] + + +def add_target_python_options(cmd_opts): + # type: (OptionGroup) -> None + cmd_opts.add_option(platform()) + cmd_opts.add_option(python_version()) + cmd_opts.add_option(implementation()) + cmd_opts.add_option(abi()) + + +def make_target_python(options): + # type: (Values) -> TargetPython + target_python = TargetPython( + platform=options.platform, + py_version_info=options.python_version, + abi=options.abi, + implementation=options.implementation, + ) + + return target_python + + +def prefer_binary(): + # type: () -> Option + return Option( + "--prefer-binary", + dest="prefer_binary", + action="store_true", + default=False, + help="Prefer older binary packages over newer source packages." + ) + + +cache_dir = partial( + PipOption, + "--cache-dir", + dest="cache_dir", + default=USER_CACHE_DIR, + metavar="dir", + type='path', + help="Store the cache data in ." +) # type: Callable[..., Option] + + +def _handle_no_cache_dir(option, opt, value, parser): + # type: (Option, str, str, OptionParser) -> None + """ + Process a value provided for the --no-cache-dir option. + + This is an optparse.Option callback for the --no-cache-dir option. + """ + # The value argument will be None if --no-cache-dir is passed via the + # command-line, since the option doesn't accept arguments. However, + # the value can be non-None if the option is triggered e.g. by an + # environment variable, like PIP_NO_CACHE_DIR=true. + if value is not None: + # Then parse the string value to get argument error-checking. + try: + strtobool(value) + except ValueError as exc: + raise_option_error(parser, option=option, msg=str(exc)) + + # Originally, setting PIP_NO_CACHE_DIR to a value that strtobool() + # converted to 0 (like "false" or "no") caused cache_dir to be disabled + # rather than enabled (logic would say the latter). Thus, we disable + # the cache directory not just on values that parse to True, but (for + # backwards compatibility reasons) also on values that parse to False. + # In other words, always set it to False if the option is provided in + # some (valid) form. + parser.values.cache_dir = False + + +no_cache = partial( + Option, + "--no-cache-dir", + dest="cache_dir", + action="callback", + callback=_handle_no_cache_dir, + help="Disable the cache.", +) # type: Callable[..., Option] + +no_deps = partial( + Option, + '--no-deps', '--no-dependencies', + dest='ignore_dependencies', + action='store_true', + default=False, + help="Don't install package dependencies.", +) # type: Callable[..., Option] + + +def _handle_build_dir(option, opt, value, parser): + # type: (Option, str, str, OptionParser) -> None + if value: + value = os.path.abspath(value) + setattr(parser.values, option.dest, value) + + +build_dir = partial( + PipOption, + '-b', '--build', '--build-dir', '--build-directory', + dest='build_dir', + type='path', + metavar='dir', + action='callback', + callback=_handle_build_dir, + help='Directory to unpack packages into and build in. Note that ' + 'an initial build still takes place in a temporary directory. ' + 'The location of temporary directories can be controlled by setting ' + 'the TMPDIR environment variable (TEMP on Windows) appropriately. ' + 'When passed, build directories are not cleaned in case of failures.' +) # type: Callable[..., Option] + +ignore_requires_python = partial( + Option, + '--ignore-requires-python', + dest='ignore_requires_python', + action='store_true', + help='Ignore the Requires-Python information.' +) # type: Callable[..., Option] + +no_build_isolation = partial( + Option, + '--no-build-isolation', + dest='build_isolation', + action='store_false', + default=True, + help='Disable isolation when building a modern source distribution. ' + 'Build dependencies specified by PEP 518 must be already installed ' + 'if this option is used.' +) # type: Callable[..., Option] + + +def _handle_no_use_pep517(option, opt, value, parser): + # type: (Option, str, str, OptionParser) -> None + """ + Process a value provided for the --no-use-pep517 option. + + This is an optparse.Option callback for the no_use_pep517 option. + """ + # Since --no-use-pep517 doesn't accept arguments, the value argument + # will be None if --no-use-pep517 is passed via the command-line. + # However, the value can be non-None if the option is triggered e.g. + # by an environment variable, for example "PIP_NO_USE_PEP517=true". + if value is not None: + msg = """A value was passed for --no-use-pep517, + probably using either the PIP_NO_USE_PEP517 environment variable + or the "no-use-pep517" config file option. Use an appropriate value + of the PIP_USE_PEP517 environment variable or the "use-pep517" + config file option instead. + """ + raise_option_error(parser, option=option, msg=msg) + + # Otherwise, --no-use-pep517 was passed via the command-line. + parser.values.use_pep517 = False + + +use_pep517 = partial( + Option, + '--use-pep517', + dest='use_pep517', + action='store_true', + default=None, + help='Use PEP 517 for building source distributions ' + '(use --no-use-pep517 to force legacy behaviour).' +) # type: Any + +no_use_pep517 = partial( + Option, + '--no-use-pep517', + dest='use_pep517', + action='callback', + callback=_handle_no_use_pep517, + default=None, + help=SUPPRESS_HELP +) # type: Any + +install_options = partial( + Option, + '--install-option', + dest='install_options', + action='append', + metavar='options', + help="Extra arguments to be supplied to the setup.py install " + "command (use like --install-option=\"--install-scripts=/usr/local/" + "bin\"). Use multiple --install-option options to pass multiple " + "options to setup.py install. If you are using an option with a " + "directory path, be sure to use absolute path.", +) # type: Callable[..., Option] + +global_options = partial( + Option, + '--global-option', + dest='global_options', + action='append', + metavar='options', + help="Extra global options to be supplied to the setup.py " + "call before the install command.", +) # type: Callable[..., Option] + +no_clean = partial( + Option, + '--no-clean', + action='store_true', + default=False, + help="Don't clean up build directories." +) # type: Callable[..., Option] + +pre = partial( + Option, + '--pre', + action='store_true', + default=False, + help="Include pre-release and development versions. By default, " + "pip only finds stable versions.", +) # type: Callable[..., Option] + +disable_pip_version_check = partial( + Option, + "--disable-pip-version-check", + dest="disable_pip_version_check", + action="store_true", + default=True, + help="Don't periodically check PyPI to determine whether a new version " + "of pip is available for download. Implied with --no-index.", +) # type: Callable[..., Option] + + +# Deprecated, Remove later +always_unzip = partial( + Option, + '-Z', '--always-unzip', + dest='always_unzip', + action='store_true', + help=SUPPRESS_HELP, +) # type: Callable[..., Option] + + +def _handle_merge_hash(option, opt_str, value, parser): + # type: (Option, str, str, OptionParser) -> None + """Given a value spelled "algo:digest", append the digest to a list + pointed to in a dict by the algo name.""" + if not parser.values.hashes: + parser.values.hashes = {} + try: + algo, digest = value.split(':', 1) + except ValueError: + parser.error('Arguments to %s must be a hash name ' + 'followed by a value, like --hash=sha256:abcde...' % + opt_str) + if algo not in STRONG_HASHES: + parser.error('Allowed hash algorithms for %s are %s.' % + (opt_str, ', '.join(STRONG_HASHES))) + parser.values.hashes.setdefault(algo, []).append(digest) + + +hash = partial( + Option, + '--hash', + # Hash values eventually end up in InstallRequirement.hashes due to + # __dict__ copying in process_line(). + dest='hashes', + action='callback', + callback=_handle_merge_hash, + type='string', + help="Verify that the package's archive matches this " + 'hash before installing. Example: --hash=sha256:abcdef...', +) # type: Callable[..., Option] + + +require_hashes = partial( + Option, + '--require-hashes', + dest='require_hashes', + action='store_true', + default=False, + help='Require a hash to check each requirement against, for ' + 'repeatable installs. This option is implied when any package in a ' + 'requirements file has a --hash option.', +) # type: Callable[..., Option] + + +list_path = partial( + PipOption, + '--path', + dest='path', + type='path', + action='append', + help='Restrict to the specified installation path for listing ' + 'packages (can be used multiple times).' +) # type: Callable[..., Option] + + +def check_list_path_option(options): + # type: (Values) -> None + if options.path and (options.user or options.local): + raise CommandError( + "Cannot combine '--path' with '--user' or '--local'" + ) + + +no_python_version_warning = partial( + Option, + '--no-python-version-warning', + dest='no_python_version_warning', + action='store_true', + default=False, + help='Silence deprecation warnings for upcoming unsupported Pythons.', +) # type: Callable[..., Option] + + +########## +# groups # +########## + +general_group = { + 'name': 'General Options', + 'options': [ + help_, + isolated_mode, + require_virtualenv, + verbose, + version, + quiet, + log, + no_input, + proxy, + retries, + timeout, + skip_requirements_regex, + exists_action, + trusted_host, + cert, + client_cert, + cache_dir, + no_cache, + disable_pip_version_check, + no_color, + no_python_version_warning, + ] +} # type: Dict[str, Any] + +index_group = { + 'name': 'Package Index Options', + 'options': [ + index_url, + extra_index_url, + no_index, + find_links, + ] +} # type: Dict[str, Any] diff --git a/env/lib/python3.8/site-packages/pip/_internal/cli/command_context.py b/env/lib/python3.8/site-packages/pip/_internal/cli/command_context.py new file mode 100644 index 000000000..d1a64a776 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/cli/command_context.py @@ -0,0 +1,36 @@ +from contextlib import contextmanager + +from pip._vendor.contextlib2 import ExitStack + +from pip._internal.utils.typing import MYPY_CHECK_RUNNING + +if MYPY_CHECK_RUNNING: + from typing import Iterator, ContextManager, TypeVar + + _T = TypeVar('_T', covariant=True) + + +class CommandContextMixIn(object): + def __init__(self): + # type: () -> None + super(CommandContextMixIn, self).__init__() + self._in_main_context = False + self._main_context = ExitStack() + + @contextmanager + def main_context(self): + # type: () -> Iterator[None] + assert not self._in_main_context + + self._in_main_context = True + try: + with self._main_context: + yield + finally: + self._in_main_context = False + + def enter_context(self, context_provider): + # type: (ContextManager[_T]) -> _T + assert self._in_main_context + + return self._main_context.enter_context(context_provider) diff --git a/env/lib/python3.8/site-packages/pip/_internal/cli/main.py b/env/lib/python3.8/site-packages/pip/_internal/cli/main.py new file mode 100644 index 000000000..5e97a5103 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/cli/main.py @@ -0,0 +1,75 @@ +"""Primary application entrypoint. +""" +from __future__ import absolute_import + +import locale +import logging +import os +import sys + +from pip._internal.cli.autocompletion import autocomplete +from pip._internal.cli.main_parser import parse_command +from pip._internal.commands import create_command +from pip._internal.exceptions import PipError +from pip._internal.utils import deprecation +from pip._internal.utils.typing import MYPY_CHECK_RUNNING + +if MYPY_CHECK_RUNNING: + from typing import List, Optional + +logger = logging.getLogger(__name__) + + +# Do not import and use main() directly! Using it directly is actively +# discouraged by pip's maintainers. The name, location and behavior of +# this function is subject to change, so calling it directly is not +# portable across different pip versions. + +# In addition, running pip in-process is unsupported and unsafe. This is +# elaborated in detail at +# https://pip.pypa.io/en/stable/user_guide/#using-pip-from-your-program. +# That document also provides suggestions that should work for nearly +# all users that are considering importing and using main() directly. + +# However, we know that certain users will still want to invoke pip +# in-process. If you understand and accept the implications of using pip +# in an unsupported manner, the best approach is to use runpy to avoid +# depending on the exact location of this entry point. + +# The following example shows how to use runpy to invoke pip in that +# case: +# +# sys.argv = ["pip", your, args, here] +# runpy.run_module("pip", run_name="__main__") +# +# Note that this will exit the process after running, unlike a direct +# call to main. As it is not safe to do any processing after calling +# main, this should not be an issue in practice. + +def main(args=None): + # type: (Optional[List[str]]) -> int + if args is None: + args = sys.argv[1:] + + # Configure our deprecation warnings to be sent through loggers + deprecation.install_warning_logger() + + autocomplete() + + try: + cmd_name, cmd_args = parse_command(args) + except PipError as exc: + sys.stderr.write("ERROR: %s" % exc) + sys.stderr.write(os.linesep) + sys.exit(1) + + # Needed for locale.getpreferredencoding(False) to work + # in pip._internal.utils.encoding.auto_decode + try: + locale.setlocale(locale.LC_ALL, '') + except locale.Error as e: + # setlocale can apparently crash if locale are uninitialized + logger.debug("Ignoring error %s when setting locale", e) + command = create_command(cmd_name, isolated=("--isolated" in cmd_args)) + + return command.main(cmd_args) diff --git a/env/lib/python3.8/site-packages/pip/_internal/cli/main_parser.py b/env/lib/python3.8/site-packages/pip/_internal/cli/main_parser.py new file mode 100644 index 000000000..a89821d44 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/cli/main_parser.py @@ -0,0 +1,99 @@ +"""A single place for constructing and exposing the main parser +""" + +import os +import sys + +from pip._internal.cli import cmdoptions +from pip._internal.cli.parser import ( + ConfigOptionParser, + UpdatingDefaultsHelpFormatter, +) +from pip._internal.commands import commands_dict, get_similar_commands +from pip._internal.exceptions import CommandError +from pip._internal.utils.misc import get_pip_version, get_prog +from pip._internal.utils.typing import MYPY_CHECK_RUNNING + +if MYPY_CHECK_RUNNING: + from typing import Tuple, List + + +__all__ = ["create_main_parser", "parse_command"] + + +def create_main_parser(): + # type: () -> ConfigOptionParser + """Creates and returns the main parser for pip's CLI + """ + + parser_kw = { + 'usage': '\n%prog [options]', + 'add_help_option': False, + 'formatter': UpdatingDefaultsHelpFormatter(), + 'name': 'global', + 'prog': get_prog(), + } + + parser = ConfigOptionParser(**parser_kw) + parser.disable_interspersed_args() + + parser.version = get_pip_version() + + # add the general options + gen_opts = cmdoptions.make_option_group(cmdoptions.general_group, parser) + parser.add_option_group(gen_opts) + + # so the help formatter knows + parser.main = True # type: ignore + + # create command listing for description + description = [''] + [ + '%-27s %s' % (name, command_info.summary) + for name, command_info in commands_dict.items() + ] + parser.description = '\n'.join(description) + + return parser + + +def parse_command(args): + # type: (List[str]) -> Tuple[str, List[str]] + parser = create_main_parser() + + # Note: parser calls disable_interspersed_args(), so the result of this + # call is to split the initial args into the general options before the + # subcommand and everything else. + # For example: + # args: ['--timeout=5', 'install', '--user', 'INITools'] + # general_options: ['--timeout==5'] + # args_else: ['install', '--user', 'INITools'] + general_options, args_else = parser.parse_args(args) + + # --version + if general_options.version: + sys.stdout.write(parser.version) # type: ignore + sys.stdout.write(os.linesep) + sys.exit() + + # pip || pip help -> print_help() + if not args_else or (args_else[0] == 'help' and len(args_else) == 1): + parser.print_help() + sys.exit() + + # the subcommand name + cmd_name = args_else[0] + + if cmd_name not in commands_dict: + guess = get_similar_commands(cmd_name) + + msg = ['unknown command "%s"' % cmd_name] + if guess: + msg.append('maybe you meant "%s"' % guess) + + raise CommandError(' - '.join(msg)) + + # all the args without the subcommand + cmd_args = args[:] + cmd_args.remove(cmd_name) + + return cmd_name, cmd_args diff --git a/env/lib/python3.8/site-packages/pip/_internal/cli/parser.py b/env/lib/python3.8/site-packages/pip/_internal/cli/parser.py new file mode 100644 index 000000000..c99456bae --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/cli/parser.py @@ -0,0 +1,265 @@ +"""Base option parser setup""" + +# The following comment should be removed at some point in the future. +# mypy: disallow-untyped-defs=False + +from __future__ import absolute_import + +import logging +import optparse +import sys +import textwrap +from distutils.util import strtobool + +from pip._vendor.six import string_types + +from pip._internal.cli.status_codes import UNKNOWN_ERROR +from pip._internal.configuration import Configuration, ConfigurationError +from pip._internal.utils.compat import get_terminal_size + +logger = logging.getLogger(__name__) + + +class PrettyHelpFormatter(optparse.IndentedHelpFormatter): + """A prettier/less verbose help formatter for optparse.""" + + def __init__(self, *args, **kwargs): + # help position must be aligned with __init__.parseopts.description + kwargs['max_help_position'] = 30 + kwargs['indent_increment'] = 1 + kwargs['width'] = get_terminal_size()[0] - 2 + optparse.IndentedHelpFormatter.__init__(self, *args, **kwargs) + + def format_option_strings(self, option): + return self._format_option_strings(option, ' <%s>', ', ') + + def _format_option_strings(self, option, mvarfmt=' <%s>', optsep=', '): + """ + Return a comma-separated list of option strings and metavars. + + :param option: tuple of (short opt, long opt), e.g: ('-f', '--format') + :param mvarfmt: metavar format string - evaluated as mvarfmt % metavar + :param optsep: separator + """ + opts = [] + + if option._short_opts: + opts.append(option._short_opts[0]) + if option._long_opts: + opts.append(option._long_opts[0]) + if len(opts) > 1: + opts.insert(1, optsep) + + if option.takes_value(): + metavar = option.metavar or option.dest.lower() + opts.append(mvarfmt % metavar.lower()) + + return ''.join(opts) + + def format_heading(self, heading): + if heading == 'Options': + return '' + return heading + ':\n' + + def format_usage(self, usage): + """ + Ensure there is only one newline between usage and the first heading + if there is no description. + """ + msg = '\nUsage: %s\n' % self.indent_lines(textwrap.dedent(usage), " ") + return msg + + def format_description(self, description): + # leave full control over description to us + if description: + if hasattr(self.parser, 'main'): + label = 'Commands' + else: + label = 'Description' + # some doc strings have initial newlines, some don't + description = description.lstrip('\n') + # some doc strings have final newlines and spaces, some don't + description = description.rstrip() + # dedent, then reindent + description = self.indent_lines(textwrap.dedent(description), " ") + description = '%s:\n%s\n' % (label, description) + return description + else: + return '' + + def format_epilog(self, epilog): + # leave full control over epilog to us + if epilog: + return epilog + else: + return '' + + def indent_lines(self, text, indent): + new_lines = [indent + line for line in text.split('\n')] + return "\n".join(new_lines) + + +class UpdatingDefaultsHelpFormatter(PrettyHelpFormatter): + """Custom help formatter for use in ConfigOptionParser. + + This is updates the defaults before expanding them, allowing + them to show up correctly in the help listing. + """ + + def expand_default(self, option): + if self.parser is not None: + self.parser._update_defaults(self.parser.defaults) + return optparse.IndentedHelpFormatter.expand_default(self, option) + + +class CustomOptionParser(optparse.OptionParser): + + def insert_option_group(self, idx, *args, **kwargs): + """Insert an OptionGroup at a given position.""" + group = self.add_option_group(*args, **kwargs) + + self.option_groups.pop() + self.option_groups.insert(idx, group) + + return group + + @property + def option_list_all(self): + """Get a list of all options, including those in option groups.""" + res = self.option_list[:] + for i in self.option_groups: + res.extend(i.option_list) + + return res + + +class ConfigOptionParser(CustomOptionParser): + """Custom option parser which updates its defaults by checking the + configuration files and environmental variables""" + + def __init__(self, *args, **kwargs): + self.name = kwargs.pop('name') + + isolated = kwargs.pop("isolated", False) + self.config = Configuration(isolated) + + assert self.name + optparse.OptionParser.__init__(self, *args, **kwargs) + + def check_default(self, option, key, val): + try: + return option.check_value(key, val) + except optparse.OptionValueError as exc: + print("An error occurred during configuration: %s" % exc) + sys.exit(3) + + def _get_ordered_configuration_items(self): + # Configuration gives keys in an unordered manner. Order them. + override_order = ["global", self.name, ":env:"] + + # Pool the options into different groups + section_items = {name: [] for name in override_order} + for section_key, val in self.config.items(): + # ignore empty values + if not val: + logger.debug( + "Ignoring configuration key '%s' as it's value is empty.", + section_key + ) + continue + + section, key = section_key.split(".", 1) + if section in override_order: + section_items[section].append((key, val)) + + # Yield each group in their override order + for section in override_order: + for key, val in section_items[section]: + yield key, val + + def _update_defaults(self, defaults): + """Updates the given defaults with values from the config files and + the environ. Does a little special handling for certain types of + options (lists).""" + + # Accumulate complex default state. + self.values = optparse.Values(self.defaults) + late_eval = set() + # Then set the options with those values + for key, val in self._get_ordered_configuration_items(): + # '--' because configuration supports only long names + option = self.get_option('--' + key) + + # Ignore options not present in this parser. E.g. non-globals put + # in [global] by users that want them to apply to all applicable + # commands. + if option is None: + continue + + if option.action in ('store_true', 'store_false', 'count'): + try: + val = strtobool(val) + except ValueError: + error_msg = invalid_config_error_message( + option.action, key, val + ) + self.error(error_msg) + + elif option.action == 'append': + val = val.split() + val = [self.check_default(option, key, v) for v in val] + elif option.action == 'callback': + late_eval.add(option.dest) + opt_str = option.get_opt_string() + val = option.convert_value(opt_str, val) + # From take_action + args = option.callback_args or () + kwargs = option.callback_kwargs or {} + option.callback(option, opt_str, val, self, *args, **kwargs) + else: + val = self.check_default(option, key, val) + + defaults[option.dest] = val + + for key in late_eval: + defaults[key] = getattr(self.values, key) + self.values = None + return defaults + + def get_default_values(self): + """Overriding to make updating the defaults after instantiation of + the option parser possible, _update_defaults() does the dirty work.""" + if not self.process_default_values: + # Old, pre-Optik 1.5 behaviour. + return optparse.Values(self.defaults) + + # Load the configuration, or error out in case of an error + try: + self.config.load() + except ConfigurationError as err: + self.exit(UNKNOWN_ERROR, str(err)) + + defaults = self._update_defaults(self.defaults.copy()) # ours + for option in self._get_all_options(): + default = defaults.get(option.dest) + if isinstance(default, string_types): + opt_str = option.get_opt_string() + defaults[option.dest] = option.check_value(opt_str, default) + return optparse.Values(defaults) + + def error(self, msg): + self.print_usage(sys.stderr) + self.exit(UNKNOWN_ERROR, "%s\n" % msg) + + +def invalid_config_error_message(action, key, val): + """Returns a better error message when invalid configuration option + is provided.""" + if action in ('store_true', 'store_false'): + return ("{0} is not a valid value for {1} option, " + "please specify a boolean value like yes/no, " + "true/false or 1/0 instead.").format(val, key) + + return ("{0} is not a valid value for {1} option, " + "please specify a numerical value like 1/0 " + "instead.").format(val, key) diff --git a/env/lib/python3.8/site-packages/pip/_internal/cli/req_command.py b/env/lib/python3.8/site-packages/pip/_internal/cli/req_command.py new file mode 100644 index 000000000..9383b3b8d --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/cli/req_command.py @@ -0,0 +1,333 @@ +"""Contains the Command base classes that depend on PipSession. + +The classes in this module are in a separate module so the commands not +needing download / PackageFinder capability don't unnecessarily import the +PackageFinder machinery and all its vendored dependencies, etc. +""" + +import logging +import os +from functools import partial + +from pip._internal.cli.base_command import Command +from pip._internal.cli.command_context import CommandContextMixIn +from pip._internal.exceptions import CommandError +from pip._internal.index.package_finder import PackageFinder +from pip._internal.legacy_resolve import Resolver +from pip._internal.models.selection_prefs import SelectionPreferences +from pip._internal.network.download import Downloader +from pip._internal.network.session import PipSession +from pip._internal.operations.prepare import RequirementPreparer +from pip._internal.req.constructors import ( + install_req_from_editable, + install_req_from_line, + install_req_from_req_string, +) +from pip._internal.req.req_file import parse_requirements +from pip._internal.self_outdated_check import ( + make_link_collector, + pip_self_version_check, +) +from pip._internal.utils.typing import MYPY_CHECK_RUNNING + +if MYPY_CHECK_RUNNING: + from optparse import Values + from typing import List, Optional, Tuple + from pip._internal.cache import WheelCache + from pip._internal.models.target_python import TargetPython + from pip._internal.req.req_set import RequirementSet + from pip._internal.req.req_tracker import RequirementTracker + from pip._internal.utils.temp_dir import TempDirectory + +logger = logging.getLogger(__name__) + + +class SessionCommandMixin(CommandContextMixIn): + + """ + A class mixin for command classes needing _build_session(). + """ + def __init__(self): + # type: () -> None + super(SessionCommandMixin, self).__init__() + self._session = None # Optional[PipSession] + + @classmethod + def _get_index_urls(cls, options): + # type: (Values) -> Optional[List[str]] + """Return a list of index urls from user-provided options.""" + index_urls = [] + if not getattr(options, "no_index", False): + url = getattr(options, "index_url", None) + if url: + index_urls.append(url) + urls = getattr(options, "extra_index_urls", None) + if urls: + index_urls.extend(urls) + # Return None rather than an empty list + return index_urls or None + + def get_default_session(self, options): + # type: (Values) -> PipSession + """Get a default-managed session.""" + if self._session is None: + self._session = self.enter_context(self._build_session(options)) + # there's no type annotation on requests.Session, so it's + # automatically ContextManager[Any] and self._session becomes Any, + # then https://github.com/python/mypy/issues/7696 kicks in + assert self._session is not None + return self._session + + def _build_session(self, options, retries=None, timeout=None): + # type: (Values, Optional[int], Optional[int]) -> PipSession + assert not options.cache_dir or os.path.isabs(options.cache_dir) + session = PipSession( + cache=( + os.path.join(options.cache_dir, "http") + if options.cache_dir else None + ), + retries=retries if retries is not None else options.retries, + trusted_hosts=options.trusted_hosts, + index_urls=self._get_index_urls(options), + ) + + # Handle custom ca-bundles from the user + if options.cert: + session.verify = options.cert + + # Handle SSL client certificate + if options.client_cert: + session.cert = options.client_cert + + # Handle timeouts + if options.timeout or timeout: + session.timeout = ( + timeout if timeout is not None else options.timeout + ) + + # Handle configured proxies + if options.proxy: + session.proxies = { + "http": options.proxy, + "https": options.proxy, + } + + # Determine if we can prompt the user for authentication or not + session.auth.prompting = not options.no_input + + return session + + +class IndexGroupCommand(Command, SessionCommandMixin): + + """ + Abstract base class for commands with the index_group options. + + This also corresponds to the commands that permit the pip version check. + """ + + def handle_pip_version_check(self, options): + # type: (Values) -> None + """ + Do the pip version check if not disabled. + + This overrides the default behavior of not doing the check. + """ + # Make sure the index_group options are present. + assert hasattr(options, 'no_index') + + if options.disable_pip_version_check or options.no_index: + return + + # Otherwise, check if we're using the latest version of pip available. + session = self._build_session( + options, + retries=0, + timeout=min(5, options.timeout) + ) + with session: + pip_self_version_check(session, options) + + +class RequirementCommand(IndexGroupCommand): + + @staticmethod + def make_requirement_preparer( + temp_build_dir, # type: TempDirectory + options, # type: Values + req_tracker, # type: RequirementTracker + session, # type: PipSession + finder, # type: PackageFinder + use_user_site, # type: bool + download_dir=None, # type: str + wheel_download_dir=None, # type: str + ): + # type: (...) -> RequirementPreparer + """ + Create a RequirementPreparer instance for the given parameters. + """ + downloader = Downloader(session, progress_bar=options.progress_bar) + + temp_build_dir_path = temp_build_dir.path + assert temp_build_dir_path is not None + + return RequirementPreparer( + build_dir=temp_build_dir_path, + src_dir=options.src_dir, + download_dir=download_dir, + wheel_download_dir=wheel_download_dir, + build_isolation=options.build_isolation, + req_tracker=req_tracker, + downloader=downloader, + finder=finder, + require_hashes=options.require_hashes, + use_user_site=use_user_site, + ) + + @staticmethod + def make_resolver( + preparer, # type: RequirementPreparer + finder, # type: PackageFinder + options, # type: Values + wheel_cache=None, # type: Optional[WheelCache] + use_user_site=False, # type: bool + ignore_installed=True, # type: bool + ignore_requires_python=False, # type: bool + force_reinstall=False, # type: bool + upgrade_strategy="to-satisfy-only", # type: str + use_pep517=None, # type: Optional[bool] + py_version_info=None # type: Optional[Tuple[int, ...]] + ): + # type: (...) -> Resolver + """ + Create a Resolver instance for the given parameters. + """ + make_install_req = partial( + install_req_from_req_string, + isolated=options.isolated_mode, + wheel_cache=wheel_cache, + use_pep517=use_pep517, + ) + return Resolver( + preparer=preparer, + finder=finder, + make_install_req=make_install_req, + use_user_site=use_user_site, + ignore_dependencies=options.ignore_dependencies, + ignore_installed=ignore_installed, + ignore_requires_python=ignore_requires_python, + force_reinstall=force_reinstall, + upgrade_strategy=upgrade_strategy, + py_version_info=py_version_info, + ) + + def populate_requirement_set( + self, + requirement_set, # type: RequirementSet + args, # type: List[str] + options, # type: Values + finder, # type: PackageFinder + session, # type: PipSession + wheel_cache, # type: Optional[WheelCache] + ): + # type: (...) -> None + """ + Marshal cmd line args into a requirement set. + """ + for filename in options.constraints: + for req_to_add in parse_requirements( + filename, + constraint=True, finder=finder, options=options, + session=session, wheel_cache=wheel_cache): + req_to_add.is_direct = True + requirement_set.add_requirement(req_to_add) + + for req in args: + req_to_add = install_req_from_line( + req, None, isolated=options.isolated_mode, + use_pep517=options.use_pep517, + wheel_cache=wheel_cache + ) + req_to_add.is_direct = True + requirement_set.add_requirement(req_to_add) + + for req in options.editables: + req_to_add = install_req_from_editable( + req, + isolated=options.isolated_mode, + use_pep517=options.use_pep517, + wheel_cache=wheel_cache + ) + req_to_add.is_direct = True + requirement_set.add_requirement(req_to_add) + + # NOTE: options.require_hashes may be set if --require-hashes is True + for filename in options.requirements: + for req_to_add in parse_requirements( + filename, + finder=finder, options=options, session=session, + wheel_cache=wheel_cache, + use_pep517=options.use_pep517): + req_to_add.is_direct = True + requirement_set.add_requirement(req_to_add) + + # If any requirement has hash options, enable hash checking. + requirements = ( + requirement_set.unnamed_requirements + + list(requirement_set.requirements.values()) + ) + if any(req.has_hash_options for req in requirements): + options.require_hashes = True + + if not (args or options.editables or options.requirements): + opts = {'name': self.name} + if options.find_links: + raise CommandError( + 'You must give at least one requirement to %(name)s ' + '(maybe you meant "pip %(name)s %(links)s"?)' % + dict(opts, links=' '.join(options.find_links))) + else: + raise CommandError( + 'You must give at least one requirement to %(name)s ' + '(see "pip help %(name)s")' % opts) + + @staticmethod + def trace_basic_info(finder): + # type: (PackageFinder) -> None + """ + Trace basic information about the provided objects. + """ + # Display where finder is looking for packages + search_scope = finder.search_scope + locations = search_scope.get_formatted_locations() + if locations: + logger.info(locations) + + def _build_package_finder( + self, + options, # type: Values + session, # type: PipSession + target_python=None, # type: Optional[TargetPython] + ignore_requires_python=None, # type: Optional[bool] + ): + # type: (...) -> PackageFinder + """ + Create a package finder appropriate to this requirement command. + + :param ignore_requires_python: Whether to ignore incompatible + "Requires-Python" values in links. Defaults to False. + """ + link_collector = make_link_collector(session, options=options) + selection_prefs = SelectionPreferences( + allow_yanked=True, + format_control=options.format_control, + allow_all_prereleases=options.pre, + prefer_binary=options.prefer_binary, + ignore_requires_python=ignore_requires_python, + ) + + return PackageFinder.create( + link_collector=link_collector, + selection_prefs=selection_prefs, + target_python=target_python, + ) diff --git a/env/lib/python3.8/site-packages/pip/_internal/cli/status_codes.py b/env/lib/python3.8/site-packages/pip/_internal/cli/status_codes.py new file mode 100644 index 000000000..275360a31 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/cli/status_codes.py @@ -0,0 +1,8 @@ +from __future__ import absolute_import + +SUCCESS = 0 +ERROR = 1 +UNKNOWN_ERROR = 2 +VIRTUALENV_NOT_FOUND = 3 +PREVIOUS_BUILD_DIR_ERROR = 4 +NO_MATCHES_FOUND = 23 diff --git a/env/lib/python3.8/site-packages/pip/_internal/commands/__init__.py b/env/lib/python3.8/site-packages/pip/_internal/commands/__init__.py new file mode 100644 index 000000000..2a311f8fc --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/commands/__init__.py @@ -0,0 +1,114 @@ +""" +Package containing all pip commands +""" + +# The following comment should be removed at some point in the future. +# mypy: disallow-untyped-defs=False + +from __future__ import absolute_import + +import importlib +from collections import OrderedDict, namedtuple + +from pip._internal.utils.typing import MYPY_CHECK_RUNNING + +if MYPY_CHECK_RUNNING: + from typing import Any + from pip._internal.cli.base_command import Command + + +CommandInfo = namedtuple('CommandInfo', 'module_path, class_name, summary') + +# The ordering matters for help display. +# Also, even though the module path starts with the same +# "pip._internal.commands" prefix in each case, we include the full path +# because it makes testing easier (specifically when modifying commands_dict +# in test setup / teardown by adding info for a FakeCommand class defined +# in a test-related module). +# Finally, we need to pass an iterable of pairs here rather than a dict +# so that the ordering won't be lost when using Python 2.7. +commands_dict = OrderedDict([ + ('install', CommandInfo( + 'pip._internal.commands.install', 'InstallCommand', + 'Install packages.', + )), + ('download', CommandInfo( + 'pip._internal.commands.download', 'DownloadCommand', + 'Download packages.', + )), + ('uninstall', CommandInfo( + 'pip._internal.commands.uninstall', 'UninstallCommand', + 'Uninstall packages.', + )), + ('freeze', CommandInfo( + 'pip._internal.commands.freeze', 'FreezeCommand', + 'Output installed packages in requirements format.', + )), + ('list', CommandInfo( + 'pip._internal.commands.list', 'ListCommand', + 'List installed packages.', + )), + ('show', CommandInfo( + 'pip._internal.commands.show', 'ShowCommand', + 'Show information about installed packages.', + )), + ('check', CommandInfo( + 'pip._internal.commands.check', 'CheckCommand', + 'Verify installed packages have compatible dependencies.', + )), + ('config', CommandInfo( + 'pip._internal.commands.configuration', 'ConfigurationCommand', + 'Manage local and global configuration.', + )), + ('search', CommandInfo( + 'pip._internal.commands.search', 'SearchCommand', + 'Search PyPI for packages.', + )), + ('wheel', CommandInfo( + 'pip._internal.commands.wheel', 'WheelCommand', + 'Build wheels from your requirements.', + )), + ('hash', CommandInfo( + 'pip._internal.commands.hash', 'HashCommand', + 'Compute hashes of package archives.', + )), + ('completion', CommandInfo( + 'pip._internal.commands.completion', 'CompletionCommand', + 'A helper command used for command completion.', + )), + ('debug', CommandInfo( + 'pip._internal.commands.debug', 'DebugCommand', + 'Show information useful for debugging.', + )), + ('help', CommandInfo( + 'pip._internal.commands.help', 'HelpCommand', + 'Show help for commands.', + )), +]) # type: OrderedDict[str, CommandInfo] + + +def create_command(name, **kwargs): + # type: (str, **Any) -> Command + """ + Create an instance of the Command class with the given name. + """ + module_path, class_name, summary = commands_dict[name] + module = importlib.import_module(module_path) + command_class = getattr(module, class_name) + command = command_class(name=name, summary=summary, **kwargs) + + return command + + +def get_similar_commands(name): + """Command name auto-correct.""" + from difflib import get_close_matches + + name = name.lower() + + close_commands = get_close_matches(name, commands_dict.keys()) + + if close_commands: + return close_commands[0] + else: + return False diff --git a/env/lib/python3.8/site-packages/pip/_internal/commands/check.py b/env/lib/python3.8/site-packages/pip/_internal/commands/check.py new file mode 100644 index 000000000..968944611 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/commands/check.py @@ -0,0 +1,45 @@ +# The following comment should be removed at some point in the future. +# mypy: disallow-untyped-defs=False + +import logging + +from pip._internal.cli.base_command import Command +from pip._internal.operations.check import ( + check_package_set, + create_package_set_from_installed, +) +from pip._internal.utils.misc import write_output + +logger = logging.getLogger(__name__) + + +class CheckCommand(Command): + """Verify installed packages have compatible dependencies.""" + + usage = """ + %prog [options]""" + + def run(self, options, args): + package_set, parsing_probs = create_package_set_from_installed() + missing, conflicting = check_package_set(package_set) + + for project_name in missing: + version = package_set[project_name].version + for dependency in missing[project_name]: + write_output( + "%s %s requires %s, which is not installed.", + project_name, version, dependency[0], + ) + + for project_name in conflicting: + version = package_set[project_name].version + for dep_name, dep_version, req in conflicting[project_name]: + write_output( + "%s %s has requirement %s, but you have %s %s.", + project_name, version, req, dep_name, dep_version, + ) + + if missing or conflicting or parsing_probs: + return 1 + else: + write_output("No broken requirements found.") diff --git a/env/lib/python3.8/site-packages/pip/_internal/commands/completion.py b/env/lib/python3.8/site-packages/pip/_internal/commands/completion.py new file mode 100644 index 000000000..c532806e3 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/commands/completion.py @@ -0,0 +1,96 @@ +# The following comment should be removed at some point in the future. +# mypy: disallow-untyped-defs=False + +from __future__ import absolute_import + +import sys +import textwrap + +from pip._internal.cli.base_command import Command +from pip._internal.utils.misc import get_prog + +BASE_COMPLETION = """ +# pip %(shell)s completion start%(script)s# pip %(shell)s completion end +""" + +COMPLETION_SCRIPTS = { + 'bash': """ + _pip_completion() + { + COMPREPLY=( $( COMP_WORDS="${COMP_WORDS[*]}" \\ + COMP_CWORD=$COMP_CWORD \\ + PIP_AUTO_COMPLETE=1 $1 2>/dev/null ) ) + } + complete -o default -F _pip_completion %(prog)s + """, + 'zsh': """ + function _pip_completion { + local words cword + read -Ac words + read -cn cword + reply=( $( COMP_WORDS="$words[*]" \\ + COMP_CWORD=$(( cword-1 )) \\ + PIP_AUTO_COMPLETE=1 $words[1] 2>/dev/null )) + } + compctl -K _pip_completion %(prog)s + """, + 'fish': """ + function __fish_complete_pip + set -lx COMP_WORDS (commandline -o) "" + set -lx COMP_CWORD ( \\ + math (contains -i -- (commandline -t) $COMP_WORDS)-1 \\ + ) + set -lx PIP_AUTO_COMPLETE 1 + string split \\ -- (eval $COMP_WORDS[1]) + end + complete -fa "(__fish_complete_pip)" -c %(prog)s + """, +} + + +class CompletionCommand(Command): + """A helper command to be used for command completion.""" + + ignore_require_venv = True + + def __init__(self, *args, **kw): + super(CompletionCommand, self).__init__(*args, **kw) + + cmd_opts = self.cmd_opts + + cmd_opts.add_option( + '--bash', '-b', + action='store_const', + const='bash', + dest='shell', + help='Emit completion code for bash') + cmd_opts.add_option( + '--zsh', '-z', + action='store_const', + const='zsh', + dest='shell', + help='Emit completion code for zsh') + cmd_opts.add_option( + '--fish', '-f', + action='store_const', + const='fish', + dest='shell', + help='Emit completion code for fish') + + self.parser.insert_option_group(0, cmd_opts) + + def run(self, options, args): + """Prints the completion code of the given shell""" + shells = COMPLETION_SCRIPTS.keys() + shell_options = ['--' + shell for shell in sorted(shells)] + if options.shell in shells: + script = textwrap.dedent( + COMPLETION_SCRIPTS.get(options.shell, '') % { + 'prog': get_prog(), + } + ) + print(BASE_COMPLETION % {'script': script, 'shell': options.shell}) + else: + sys.stderr.write( + 'ERROR: You must pass %s\n' % ' or '.join(shell_options) + ) diff --git a/env/lib/python3.8/site-packages/pip/_internal/commands/configuration.py b/env/lib/python3.8/site-packages/pip/_internal/commands/configuration.py new file mode 100644 index 000000000..efcf5bb36 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/commands/configuration.py @@ -0,0 +1,233 @@ +# The following comment should be removed at some point in the future. +# mypy: disallow-untyped-defs=False + +import logging +import os +import subprocess + +from pip._internal.cli.base_command import Command +from pip._internal.cli.status_codes import ERROR, SUCCESS +from pip._internal.configuration import ( + Configuration, + get_configuration_files, + kinds, +) +from pip._internal.exceptions import PipError +from pip._internal.utils.misc import get_prog, write_output + +logger = logging.getLogger(__name__) + + +class ConfigurationCommand(Command): + """Manage local and global configuration. + + Subcommands: + + list: List the active configuration (or from the file specified) + edit: Edit the configuration file in an editor + get: Get the value associated with name + set: Set the name=value + unset: Unset the value associated with name + + If none of --user, --global and --site are passed, a virtual + environment configuration file is used if one is active and the file + exists. Otherwise, all modifications happen on the to the user file by + default. + """ + + ignore_require_venv = True + usage = """ + %prog [] list + %prog [] [--editor ] edit + + %prog [] get name + %prog [] set name value + %prog [] unset name + """ + + def __init__(self, *args, **kwargs): + super(ConfigurationCommand, self).__init__(*args, **kwargs) + + self.configuration = None + + self.cmd_opts.add_option( + '--editor', + dest='editor', + action='store', + default=None, + help=( + 'Editor to use to edit the file. Uses VISUAL or EDITOR ' + 'environment variables if not provided.' + ) + ) + + self.cmd_opts.add_option( + '--global', + dest='global_file', + action='store_true', + default=False, + help='Use the system-wide configuration file only' + ) + + self.cmd_opts.add_option( + '--user', + dest='user_file', + action='store_true', + default=False, + help='Use the user configuration file only' + ) + + self.cmd_opts.add_option( + '--site', + dest='site_file', + action='store_true', + default=False, + help='Use the current environment configuration file only' + ) + + self.parser.insert_option_group(0, self.cmd_opts) + + def run(self, options, args): + handlers = { + "list": self.list_values, + "edit": self.open_in_editor, + "get": self.get_name, + "set": self.set_name_value, + "unset": self.unset_name + } + + # Determine action + if not args or args[0] not in handlers: + logger.error("Need an action ({}) to perform.".format( + ", ".join(sorted(handlers))) + ) + return ERROR + + action = args[0] + + # Determine which configuration files are to be loaded + # Depends on whether the command is modifying. + try: + load_only = self._determine_file( + options, need_value=(action in ["get", "set", "unset", "edit"]) + ) + except PipError as e: + logger.error(e.args[0]) + return ERROR + + # Load a new configuration + self.configuration = Configuration( + isolated=options.isolated_mode, load_only=load_only + ) + self.configuration.load() + + # Error handling happens here, not in the action-handlers. + try: + handlers[action](options, args[1:]) + except PipError as e: + logger.error(e.args[0]) + return ERROR + + return SUCCESS + + def _determine_file(self, options, need_value): + file_options = [key for key, value in ( + (kinds.USER, options.user_file), + (kinds.GLOBAL, options.global_file), + (kinds.SITE, options.site_file), + ) if value] + + if not file_options: + if not need_value: + return None + # Default to user, unless there's a site file. + elif any( + os.path.exists(site_config_file) + for site_config_file in get_configuration_files()[kinds.SITE] + ): + return kinds.SITE + else: + return kinds.USER + elif len(file_options) == 1: + return file_options[0] + + raise PipError( + "Need exactly one file to operate upon " + "(--user, --site, --global) to perform." + ) + + def list_values(self, options, args): + self._get_n_args(args, "list", n=0) + + for key, value in sorted(self.configuration.items()): + write_output("%s=%r", key, value) + + def get_name(self, options, args): + key = self._get_n_args(args, "get [name]", n=1) + value = self.configuration.get_value(key) + + write_output("%s", value) + + def set_name_value(self, options, args): + key, value = self._get_n_args(args, "set [name] [value]", n=2) + self.configuration.set_value(key, value) + + self._save_configuration() + + def unset_name(self, options, args): + key = self._get_n_args(args, "unset [name]", n=1) + self.configuration.unset_value(key) + + self._save_configuration() + + def open_in_editor(self, options, args): + editor = self._determine_editor(options) + + fname = self.configuration.get_file_to_edit() + if fname is None: + raise PipError("Could not determine appropriate file.") + + try: + subprocess.check_call([editor, fname]) + except subprocess.CalledProcessError as e: + raise PipError( + "Editor Subprocess exited with exit code {}" + .format(e.returncode) + ) + + def _get_n_args(self, args, example, n): + """Helper to make sure the command got the right number of arguments + """ + if len(args) != n: + msg = ( + 'Got unexpected number of arguments, expected {}. ' + '(example: "{} config {}")' + ).format(n, get_prog(), example) + raise PipError(msg) + + if n == 1: + return args[0] + else: + return args + + def _save_configuration(self): + # We successfully ran a modifying command. Need to save the + # configuration. + try: + self.configuration.save() + except Exception: + logger.error( + "Unable to save configuration. Please report this as a bug.", + exc_info=1 + ) + raise PipError("Internal Error.") + + def _determine_editor(self, options): + if options.editor is not None: + return options.editor + elif "VISUAL" in os.environ: + return os.environ["VISUAL"] + elif "EDITOR" in os.environ: + return os.environ["EDITOR"] + else: + raise PipError("Could not determine editor to use.") diff --git a/env/lib/python3.8/site-packages/pip/_internal/commands/debug.py b/env/lib/python3.8/site-packages/pip/_internal/commands/debug.py new file mode 100644 index 000000000..fe93b3a39 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/commands/debug.py @@ -0,0 +1,142 @@ +# The following comment should be removed at some point in the future. +# mypy: disallow-untyped-defs=False + +from __future__ import absolute_import + +import locale +import logging +import os +import sys + +from pip._vendor.certifi import where + +from pip._internal.cli import cmdoptions +from pip._internal.cli.base_command import Command +from pip._internal.cli.cmdoptions import make_target_python +from pip._internal.cli.status_codes import SUCCESS +from pip._internal.utils.logging import indent_log +from pip._internal.utils.misc import get_pip_version +from pip._internal.utils.typing import MYPY_CHECK_RUNNING + +if MYPY_CHECK_RUNNING: + from typing import Any, List, Optional + from optparse import Values + +logger = logging.getLogger(__name__) + + +def show_value(name, value): + # type: (str, Optional[str]) -> None + logger.info('{}: {}'.format(name, value)) + + +def show_sys_implementation(): + # type: () -> None + logger.info('sys.implementation:') + if hasattr(sys, 'implementation'): + implementation = sys.implementation # type: ignore + implementation_name = implementation.name + else: + implementation_name = '' + + with indent_log(): + show_value('name', implementation_name) + + +def show_tags(options): + # type: (Values) -> None + tag_limit = 10 + + target_python = make_target_python(options) + tags = target_python.get_tags() + + # Display the target options that were explicitly provided. + formatted_target = target_python.format_given() + suffix = '' + if formatted_target: + suffix = ' (target: {})'.format(formatted_target) + + msg = 'Compatible tags: {}{}'.format(len(tags), suffix) + logger.info(msg) + + if options.verbose < 1 and len(tags) > tag_limit: + tags_limited = True + tags = tags[:tag_limit] + else: + tags_limited = False + + with indent_log(): + for tag in tags: + logger.info(str(tag)) + + if tags_limited: + msg = ( + '...\n' + '[First {tag_limit} tags shown. Pass --verbose to show all.]' + ).format(tag_limit=tag_limit) + logger.info(msg) + + +def ca_bundle_info(config): + levels = set() + for key, value in config.items(): + levels.add(key.split('.')[0]) + + if not levels: + return "Not specified" + + levels_that_override_global = ['install', 'wheel', 'download'] + global_overriding_level = [ + level for level in levels if level in levels_that_override_global + ] + if not global_overriding_level: + return 'global' + + levels.remove('global') + return ", ".join(levels) + + +class DebugCommand(Command): + """ + Display debug information. + """ + + usage = """ + %prog """ + ignore_require_venv = True + + def __init__(self, *args, **kw): + super(DebugCommand, self).__init__(*args, **kw) + + cmd_opts = self.cmd_opts + cmdoptions.add_target_python_options(cmd_opts) + self.parser.insert_option_group(0, cmd_opts) + self.parser.config.load() + + def run(self, options, args): + # type: (Values, List[Any]) -> int + logger.warning( + "This command is only meant for debugging. " + "Do not use this with automation for parsing and getting these " + "details, since the output and options of this command may " + "change without notice." + ) + show_value('pip version', get_pip_version()) + show_value('sys.version', sys.version) + show_value('sys.executable', sys.executable) + show_value('sys.getdefaultencoding', sys.getdefaultencoding()) + show_value('sys.getfilesystemencoding', sys.getfilesystemencoding()) + show_value( + 'locale.getpreferredencoding', locale.getpreferredencoding(), + ) + show_value('sys.platform', sys.platform) + show_sys_implementation() + + show_value("'cert' config value", ca_bundle_info(self.parser.config)) + show_value("REQUESTS_CA_BUNDLE", os.environ.get('REQUESTS_CA_BUNDLE')) + show_value("CURL_CA_BUNDLE", os.environ.get('CURL_CA_BUNDLE')) + show_value("pip._vendor.certifi.where()", where()) + + show_tags(options) + + return SUCCESS diff --git a/env/lib/python3.8/site-packages/pip/_internal/commands/download.py b/env/lib/python3.8/site-packages/pip/_internal/commands/download.py new file mode 100644 index 000000000..24da3eb2a --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/commands/download.py @@ -0,0 +1,147 @@ +# The following comment should be removed at some point in the future. +# mypy: disallow-untyped-defs=False + +from __future__ import absolute_import + +import logging +import os + +from pip._internal.cli import cmdoptions +from pip._internal.cli.cmdoptions import make_target_python +from pip._internal.cli.req_command import RequirementCommand +from pip._internal.req import RequirementSet +from pip._internal.req.req_tracker import get_requirement_tracker +from pip._internal.utils.misc import ensure_dir, normalize_path, write_output +from pip._internal.utils.temp_dir import TempDirectory + +logger = logging.getLogger(__name__) + + +class DownloadCommand(RequirementCommand): + """ + Download packages from: + + - PyPI (and other indexes) using requirement specifiers. + - VCS project urls. + - Local project directories. + - Local or remote source archives. + + pip also supports downloading from "requirements files", which provide + an easy way to specify a whole environment to be downloaded. + """ + + usage = """ + %prog [options] [package-index-options] ... + %prog [options] -r [package-index-options] ... + %prog [options] ... + %prog [options] ... + %prog [options] ...""" + + def __init__(self, *args, **kw): + super(DownloadCommand, self).__init__(*args, **kw) + + cmd_opts = self.cmd_opts + + cmd_opts.add_option(cmdoptions.constraints()) + cmd_opts.add_option(cmdoptions.requirements()) + cmd_opts.add_option(cmdoptions.build_dir()) + cmd_opts.add_option(cmdoptions.no_deps()) + cmd_opts.add_option(cmdoptions.global_options()) + cmd_opts.add_option(cmdoptions.no_binary()) + cmd_opts.add_option(cmdoptions.only_binary()) + cmd_opts.add_option(cmdoptions.prefer_binary()) + cmd_opts.add_option(cmdoptions.src()) + cmd_opts.add_option(cmdoptions.pre()) + cmd_opts.add_option(cmdoptions.no_clean()) + cmd_opts.add_option(cmdoptions.require_hashes()) + cmd_opts.add_option(cmdoptions.progress_bar()) + cmd_opts.add_option(cmdoptions.no_build_isolation()) + cmd_opts.add_option(cmdoptions.use_pep517()) + cmd_opts.add_option(cmdoptions.no_use_pep517()) + + cmd_opts.add_option( + '-d', '--dest', '--destination-dir', '--destination-directory', + dest='download_dir', + metavar='dir', + default=os.curdir, + help=("Download packages into ."), + ) + + cmdoptions.add_target_python_options(cmd_opts) + + index_opts = cmdoptions.make_option_group( + cmdoptions.index_group, + self.parser, + ) + + self.parser.insert_option_group(0, index_opts) + self.parser.insert_option_group(0, cmd_opts) + + def run(self, options, args): + options.ignore_installed = True + # editable doesn't really make sense for `pip download`, but the bowels + # of the RequirementSet code require that property. + options.editables = [] + + cmdoptions.check_dist_restriction(options) + + options.download_dir = normalize_path(options.download_dir) + + ensure_dir(options.download_dir) + + session = self.get_default_session(options) + + target_python = make_target_python(options) + finder = self._build_package_finder( + options=options, + session=session, + target_python=target_python, + ) + build_delete = (not (options.no_clean or options.build_dir)) + + with get_requirement_tracker() as req_tracker, TempDirectory( + options.build_dir, delete=build_delete, kind="download" + ) as directory: + + requirement_set = RequirementSet() + self.populate_requirement_set( + requirement_set, + args, + options, + finder, + session, + None + ) + + preparer = self.make_requirement_preparer( + temp_build_dir=directory, + options=options, + req_tracker=req_tracker, + session=session, + finder=finder, + download_dir=options.download_dir, + use_user_site=False, + ) + + resolver = self.make_resolver( + preparer=preparer, + finder=finder, + options=options, + py_version_info=options.python_version, + ) + + self.trace_basic_info(finder) + + resolver.resolve(requirement_set) + + downloaded = ' '.join([ + req.name for req in requirement_set.successfully_downloaded + ]) + if downloaded: + write_output('Successfully downloaded %s', downloaded) + + # Clean up + if not options.no_clean: + requirement_set.cleanup_files() + + return requirement_set diff --git a/env/lib/python3.8/site-packages/pip/_internal/commands/freeze.py b/env/lib/python3.8/site-packages/pip/_internal/commands/freeze.py new file mode 100644 index 000000000..e96c0833f --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/commands/freeze.py @@ -0,0 +1,103 @@ +# The following comment should be removed at some point in the future. +# mypy: disallow-untyped-defs=False + +from __future__ import absolute_import + +import sys + +from pip._internal.cache import WheelCache +from pip._internal.cli import cmdoptions +from pip._internal.cli.base_command import Command +from pip._internal.models.format_control import FormatControl +from pip._internal.operations.freeze import freeze +from pip._internal.utils.compat import stdlib_pkgs + +DEV_PKGS = {'pip', 'setuptools', 'distribute', 'wheel', 'pkg-resources'} + + +class FreezeCommand(Command): + """ + Output installed packages in requirements format. + + packages are listed in a case-insensitive sorted order. + """ + + usage = """ + %prog [options]""" + log_streams = ("ext://sys.stderr", "ext://sys.stderr") + + def __init__(self, *args, **kw): + super(FreezeCommand, self).__init__(*args, **kw) + + self.cmd_opts.add_option( + '-r', '--requirement', + dest='requirements', + action='append', + default=[], + metavar='file', + help="Use the order in the given requirements file and its " + "comments when generating output. This option can be " + "used multiple times.") + self.cmd_opts.add_option( + '-f', '--find-links', + dest='find_links', + action='append', + default=[], + metavar='URL', + help='URL for finding packages, which will be added to the ' + 'output.') + self.cmd_opts.add_option( + '-l', '--local', + dest='local', + action='store_true', + default=False, + help='If in a virtualenv that has global access, do not output ' + 'globally-installed packages.') + self.cmd_opts.add_option( + '--user', + dest='user', + action='store_true', + default=False, + help='Only output packages installed in user-site.') + self.cmd_opts.add_option(cmdoptions.list_path()) + self.cmd_opts.add_option( + '--all', + dest='freeze_all', + action='store_true', + help='Do not skip these packages in the output:' + ' %s' % ', '.join(DEV_PKGS)) + self.cmd_opts.add_option( + '--exclude-editable', + dest='exclude_editable', + action='store_true', + help='Exclude editable package from output.') + + self.parser.insert_option_group(0, self.cmd_opts) + + def run(self, options, args): + format_control = FormatControl(set(), set()) + wheel_cache = WheelCache(options.cache_dir, format_control) + skip = set(stdlib_pkgs) + if not options.freeze_all: + skip.update(DEV_PKGS) + + cmdoptions.check_list_path_option(options) + + freeze_kwargs = dict( + requirement=options.requirements, + find_links=options.find_links, + local_only=options.local, + user_only=options.user, + paths=options.path, + skip_regex=options.skip_requirements_regex, + isolated=options.isolated_mode, + wheel_cache=wheel_cache, + skip=skip, + exclude_editable=options.exclude_editable, + ) + + try: + for line in freeze(**freeze_kwargs): + sys.stdout.write(line + '\n') + finally: + wheel_cache.cleanup() diff --git a/env/lib/python3.8/site-packages/pip/_internal/commands/hash.py b/env/lib/python3.8/site-packages/pip/_internal/commands/hash.py new file mode 100644 index 000000000..1dc7fb0ea --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/commands/hash.py @@ -0,0 +1,58 @@ +# The following comment should be removed at some point in the future. +# mypy: disallow-untyped-defs=False + +from __future__ import absolute_import + +import hashlib +import logging +import sys + +from pip._internal.cli.base_command import Command +from pip._internal.cli.status_codes import ERROR +from pip._internal.utils.hashes import FAVORITE_HASH, STRONG_HASHES +from pip._internal.utils.misc import read_chunks, write_output + +logger = logging.getLogger(__name__) + + +class HashCommand(Command): + """ + Compute a hash of a local package archive. + + These can be used with --hash in a requirements file to do repeatable + installs. + """ + + usage = '%prog [options] ...' + ignore_require_venv = True + + def __init__(self, *args, **kw): + super(HashCommand, self).__init__(*args, **kw) + self.cmd_opts.add_option( + '-a', '--algorithm', + dest='algorithm', + choices=STRONG_HASHES, + action='store', + default=FAVORITE_HASH, + help='The hash algorithm to use: one of %s' % + ', '.join(STRONG_HASHES)) + self.parser.insert_option_group(0, self.cmd_opts) + + def run(self, options, args): + if not args: + self.parser.print_usage(sys.stderr) + return ERROR + + algorithm = options.algorithm + for path in args: + write_output('%s:\n--hash=%s:%s', + path, algorithm, _hash_of_file(path, algorithm)) + + +def _hash_of_file(path, algorithm): + """Return the hash digest of a file.""" + with open(path, 'rb') as archive: + hash = hashlib.new(algorithm) + for chunk in read_chunks(archive): + hash.update(chunk) + return hash.hexdigest() diff --git a/env/lib/python3.8/site-packages/pip/_internal/commands/help.py b/env/lib/python3.8/site-packages/pip/_internal/commands/help.py new file mode 100644 index 000000000..75af999b4 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/commands/help.py @@ -0,0 +1,41 @@ +# The following comment should be removed at some point in the future. +# mypy: disallow-untyped-defs=False + +from __future__ import absolute_import + +from pip._internal.cli.base_command import Command +from pip._internal.cli.status_codes import SUCCESS +from pip._internal.exceptions import CommandError + + +class HelpCommand(Command): + """Show help for commands""" + + usage = """ + %prog """ + ignore_require_venv = True + + def run(self, options, args): + from pip._internal.commands import ( + commands_dict, create_command, get_similar_commands, + ) + + try: + # 'pip help' with no args is handled by pip.__init__.parseopt() + cmd_name = args[0] # the command we need help for + except IndexError: + return SUCCESS + + if cmd_name not in commands_dict: + guess = get_similar_commands(cmd_name) + + msg = ['unknown command "%s"' % cmd_name] + if guess: + msg.append('maybe you meant "%s"' % guess) + + raise CommandError(' - '.join(msg)) + + command = create_command(cmd_name) + command.parser.print_help() + + return SUCCESS diff --git a/env/lib/python3.8/site-packages/pip/_internal/commands/install.py b/env/lib/python3.8/site-packages/pip/_internal/commands/install.py new file mode 100644 index 000000000..cb2fb280c --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/commands/install.py @@ -0,0 +1,727 @@ +# The following comment should be removed at some point in the future. +# It's included for now because without it InstallCommand.run() has a +# couple errors where we have to know req.name is str rather than +# Optional[str] for the InstallRequirement req. +# mypy: strict-optional=False +# mypy: disallow-untyped-defs=False + +from __future__ import absolute_import + +import errno +import logging +import operator +import os +import shutil +import site +from optparse import SUPPRESS_HELP + +from pip._vendor import pkg_resources +from pip._vendor.packaging.utils import canonicalize_name + +from pip._internal.cache import WheelCache +from pip._internal.cli import cmdoptions +from pip._internal.cli.cmdoptions import make_target_python +from pip._internal.cli.req_command import RequirementCommand +from pip._internal.cli.status_codes import ERROR, SUCCESS +from pip._internal.exceptions import ( + CommandError, + InstallationError, + PreviousBuildDirError, +) +from pip._internal.locations import distutils_scheme +from pip._internal.operations.check import check_install_conflicts +from pip._internal.req import RequirementSet, install_given_reqs +from pip._internal.req.req_tracker import get_requirement_tracker +from pip._internal.utils.deprecation import deprecated +from pip._internal.utils.distutils_args import parse_distutils_args +from pip._internal.utils.filesystem import test_writable_dir +from pip._internal.utils.misc import ( + ensure_dir, + get_installed_version, + protect_pip_from_modification_on_windows, + write_output, +) +from pip._internal.utils.temp_dir import TempDirectory +from pip._internal.utils.typing import MYPY_CHECK_RUNNING +from pip._internal.utils.virtualenv import virtualenv_no_global +from pip._internal.wheel_builder import build, should_build_for_install_command + +if MYPY_CHECK_RUNNING: + from optparse import Values + from typing import Any, Iterable, List, Optional + + from pip._internal.models.format_control import FormatControl + from pip._internal.req.req_install import InstallRequirement + from pip._internal.wheel_builder import BinaryAllowedPredicate + +from pip._internal.locations import running_under_virtualenv + +logger = logging.getLogger(__name__) + + +def get_check_binary_allowed(format_control): + # type: (FormatControl) -> BinaryAllowedPredicate + def check_binary_allowed(req): + # type: (InstallRequirement) -> bool + if req.use_pep517: + return True + canonical_name = canonicalize_name(req.name) + allowed_formats = format_control.get_allowed_formats(canonical_name) + return "binary" in allowed_formats + + return check_binary_allowed + + +class InstallCommand(RequirementCommand): + """ + Install packages from: + + - PyPI (and other indexes) using requirement specifiers. + - VCS project urls. + - Local project directories. + - Local or remote source archives. + + pip also supports installing from "requirements files", which provide + an easy way to specify a whole environment to be installed. + """ + + usage = """ + %prog [options] [package-index-options] ... + %prog [options] -r [package-index-options] ... + %prog [options] [-e] ... + %prog [options] [-e] ... + %prog [options] ...""" + + def __init__(self, *args, **kw): + super(InstallCommand, self).__init__(*args, **kw) + + cmd_opts = self.cmd_opts + + cmd_opts.add_option(cmdoptions.requirements()) + cmd_opts.add_option(cmdoptions.constraints()) + cmd_opts.add_option(cmdoptions.no_deps()) + cmd_opts.add_option(cmdoptions.pre()) + + cmd_opts.add_option(cmdoptions.editable()) + cmd_opts.add_option( + '-t', '--target', + dest='target_dir', + metavar='dir', + default=None, + help='Install packages into . ' + 'By default this will not replace existing files/folders in ' + '. Use --upgrade to replace existing packages in ' + 'with new versions.' + ) + cmdoptions.add_target_python_options(cmd_opts) + + cmd_opts.add_option( + '--user', + dest='use_user_site', + action='store_true', + help="Install to the Python user install directory for your " + "platform. Typically ~/.local/, or %APPDATA%\\Python on " + "Windows. (See the Python documentation for site.USER_BASE " + "for full details.) On Debian systems, this is the " + "default when running outside of a virtual environment " + "and not as root.") + + cmd_opts.add_option( + '--no-user', + dest='use_system_location', + action='store_true', + help=SUPPRESS_HELP) + cmd_opts.add_option( + '--root', + dest='root_path', + metavar='dir', + default=None, + help="Install everything relative to this alternate root " + "directory.") + cmd_opts.add_option( + '--prefix', + dest='prefix_path', + metavar='dir', + default=None, + help="Installation prefix where lib, bin and other top-level " + "folders are placed") + + cmd_opts.add_option( + '--system', + dest='use_system_location', + action='store_true', + help="Install using the system scheme (overrides --user on " + "Debian systems)") + + cmd_opts.add_option(cmdoptions.build_dir()) + + cmd_opts.add_option(cmdoptions.src()) + + cmd_opts.add_option( + '-U', '--upgrade', + dest='upgrade', + action='store_true', + help='Upgrade all specified packages to the newest available ' + 'version. The handling of dependencies depends on the ' + 'upgrade-strategy used.' + ) + + cmd_opts.add_option( + '--upgrade-strategy', + dest='upgrade_strategy', + default='only-if-needed', + choices=['only-if-needed', 'eager'], + help='Determines how dependency upgrading should be handled ' + '[default: %default]. ' + '"eager" - dependencies are upgraded regardless of ' + 'whether the currently installed version satisfies the ' + 'requirements of the upgraded package(s). ' + '"only-if-needed" - are upgraded only when they do not ' + 'satisfy the requirements of the upgraded package(s).' + ) + + cmd_opts.add_option( + '--force-reinstall', + dest='force_reinstall', + action='store_true', + help='Reinstall all packages even if they are already ' + 'up-to-date.') + + cmd_opts.add_option( + '-I', '--ignore-installed', + dest='ignore_installed', + action='store_true', + help='Ignore the installed packages, overwriting them. ' + 'This can break your system if the existing package ' + 'is of a different version or was installed ' + 'with a different package manager!' + ) + + cmd_opts.add_option(cmdoptions.ignore_requires_python()) + cmd_opts.add_option(cmdoptions.no_build_isolation()) + cmd_opts.add_option(cmdoptions.use_pep517()) + cmd_opts.add_option(cmdoptions.no_use_pep517()) + + cmd_opts.add_option(cmdoptions.install_options()) + cmd_opts.add_option(cmdoptions.global_options()) + + cmd_opts.add_option( + "--compile", + action="store_true", + dest="compile", + default=True, + help="Compile Python source files to bytecode", + ) + + cmd_opts.add_option( + "--no-compile", + action="store_false", + dest="compile", + help="Do not compile Python source files to bytecode", + ) + + cmd_opts.add_option( + "--no-warn-script-location", + action="store_false", + dest="warn_script_location", + default=True, + help="Do not warn when installing scripts outside PATH", + ) + cmd_opts.add_option( + "--no-warn-conflicts", + action="store_false", + dest="warn_about_conflicts", + default=True, + help="Do not warn about broken dependencies", + ) + + cmd_opts.add_option(cmdoptions.no_binary()) + cmd_opts.add_option(cmdoptions.only_binary()) + cmd_opts.add_option(cmdoptions.prefer_binary()) + cmd_opts.add_option(cmdoptions.no_clean()) + cmd_opts.add_option(cmdoptions.require_hashes()) + cmd_opts.add_option(cmdoptions.progress_bar()) + + index_opts = cmdoptions.make_option_group( + cmdoptions.index_group, + self.parser, + ) + + self.parser.insert_option_group(0, index_opts) + self.parser.insert_option_group(0, cmd_opts) + + def run(self, options, args): + # type: (Values, List[Any]) -> int + cmdoptions.check_install_build_global(options) + upgrade_strategy = "to-satisfy-only" + if options.upgrade: + upgrade_strategy = options.upgrade_strategy + + cmdoptions.check_dist_restriction(options, check_target=True) + + if options.python_version: + python_versions = [options.python_version] + else: + python_versions = None + + # compute install location defaults + if (not options.use_user_site and not options.prefix_path and not + options.target_dir and not options.use_system_location): + if not running_under_virtualenv() and os.geteuid() != 0: + options.use_user_site = True + + if options.use_system_location: + options.use_user_site = False + + options.src_dir = os.path.abspath(options.src_dir) + install_options = options.install_options or [] + + options.use_user_site = decide_user_install( + options.use_user_site, + prefix_path=options.prefix_path, + target_dir=options.target_dir, + root_path=options.root_path, + isolated_mode=options.isolated_mode, + ) + + target_temp_dir = None # type: Optional[TempDirectory] + target_temp_dir_path = None # type: Optional[str] + if options.target_dir: + options.ignore_installed = True + options.target_dir = os.path.abspath(options.target_dir) + if (os.path.exists(options.target_dir) and not + os.path.isdir(options.target_dir)): + raise CommandError( + "Target path exists but is not a directory, will not " + "continue." + ) + + # Create a target directory for using with the target option + target_temp_dir = TempDirectory(kind="target") + target_temp_dir_path = target_temp_dir.path + + global_options = options.global_options or [] + + session = self.get_default_session(options) + + target_python = make_target_python(options) + finder = self._build_package_finder( + options=options, + session=session, + target_python=target_python, + ignore_requires_python=options.ignore_requires_python, + ) + build_delete = (not (options.no_clean or options.build_dir)) + wheel_cache = WheelCache(options.cache_dir, options.format_control) + + with get_requirement_tracker() as req_tracker, TempDirectory( + options.build_dir, delete=build_delete, kind="install" + ) as directory: + requirement_set = RequirementSet( + check_supported_wheels=not options.target_dir, + ) + + try: + self.populate_requirement_set( + requirement_set, args, options, finder, session, + wheel_cache + ) + + warn_deprecated_install_options( + requirement_set, options.install_options + ) + + preparer = self.make_requirement_preparer( + temp_build_dir=directory, + options=options, + req_tracker=req_tracker, + session=session, + finder=finder, + use_user_site=options.use_user_site, + ) + resolver = self.make_resolver( + preparer=preparer, + finder=finder, + options=options, + wheel_cache=wheel_cache, + use_user_site=options.use_user_site, + ignore_installed=options.ignore_installed, + ignore_requires_python=options.ignore_requires_python, + force_reinstall=options.force_reinstall, + upgrade_strategy=upgrade_strategy, + use_pep517=options.use_pep517, + ) + + self.trace_basic_info(finder) + + resolver.resolve(requirement_set) + + try: + pip_req = requirement_set.get_requirement("pip") + except KeyError: + modifying_pip = None + else: + # If we're not replacing an already installed pip, + # we're not modifying it. + modifying_pip = pip_req.satisfied_by is None + protect_pip_from_modification_on_windows( + modifying_pip=modifying_pip + ) + + check_binary_allowed = get_check_binary_allowed( + finder.format_control + ) + + reqs_to_build = [ + r for r in requirement_set.requirements.values() + if should_build_for_install_command( + r, check_binary_allowed + ) + ] + + _, build_failures = build( + reqs_to_build, + wheel_cache=wheel_cache, + build_options=[], + global_options=[], + ) + + # If we're using PEP 517, we cannot do a direct install + # so we fail here. + # We don't care about failures building legacy + # requirements, as we'll fall through to a direct + # install for those. + pep517_build_failures = [ + r for r in build_failures if r.use_pep517 + ] + if pep517_build_failures: + raise InstallationError( + "Could not build wheels for {} which use" + " PEP 517 and cannot be installed directly".format( + ", ".join(r.name for r in pep517_build_failures))) + + to_install = resolver.get_installation_order( + requirement_set + ) + + # Consistency Checking of the package set we're installing. + should_warn_about_conflicts = ( + not options.ignore_dependencies and + options.warn_about_conflicts + ) + if should_warn_about_conflicts: + self._warn_about_conflicts(to_install) + + # Don't warn about script install locations if + # --target has been specified + warn_script_location = options.warn_script_location + if options.target_dir: + warn_script_location = False + + installed = install_given_reqs( + to_install, + install_options, + global_options, + root=options.root_path, + home=target_temp_dir_path, + prefix=options.prefix_path, + pycompile=options.compile, + warn_script_location=warn_script_location, + use_user_site=options.use_user_site, + ) + + lib_locations = get_lib_location_guesses( + user=options.use_user_site, + home=target_temp_dir_path, + root=options.root_path, + prefix=options.prefix_path, + isolated=options.isolated_mode, + ) + working_set = pkg_resources.WorkingSet(lib_locations) + + installed.sort(key=operator.attrgetter('name')) + items = [] + for result in installed: + item = result.name + try: + installed_version = get_installed_version( + result.name, working_set=working_set + ) + if installed_version: + item += '-' + installed_version + except Exception: + pass + items.append(item) + installed_desc = ' '.join(items) + if installed_desc: + write_output( + 'Successfully installed %s', installed_desc, + ) + except EnvironmentError as error: + show_traceback = (self.verbosity >= 1) + + message = create_env_error_message( + error, show_traceback, options.use_user_site, + ) + logger.error(message, exc_info=show_traceback) + + return ERROR + except PreviousBuildDirError: + options.no_clean = True + raise + finally: + # Clean up + if not options.no_clean: + requirement_set.cleanup_files() + wheel_cache.cleanup() + + if options.target_dir: + self._handle_target_dir( + options.target_dir, target_temp_dir, options.upgrade + ) + + return SUCCESS + + def _handle_target_dir(self, target_dir, target_temp_dir, upgrade): + ensure_dir(target_dir) + + # Checking both purelib and platlib directories for installed + # packages to be moved to target directory + lib_dir_list = [] + + with target_temp_dir: + # Checking both purelib and platlib directories for installed + # packages to be moved to target directory + scheme = distutils_scheme('', home=target_temp_dir.path) + purelib_dir = scheme['purelib'] + platlib_dir = scheme['platlib'] + data_dir = scheme['data'] + + if os.path.exists(purelib_dir): + lib_dir_list.append(purelib_dir) + if os.path.exists(platlib_dir) and platlib_dir != purelib_dir: + lib_dir_list.append(platlib_dir) + if os.path.exists(data_dir): + lib_dir_list.append(data_dir) + + for lib_dir in lib_dir_list: + for item in os.listdir(lib_dir): + if lib_dir == data_dir: + ddir = os.path.join(data_dir, item) + if any(s.startswith(ddir) for s in lib_dir_list[:-1]): + continue + target_item_dir = os.path.join(target_dir, item) + if os.path.exists(target_item_dir): + if not upgrade: + logger.warning( + 'Target directory %s already exists. Specify ' + '--upgrade to force replacement.', + target_item_dir + ) + continue + if os.path.islink(target_item_dir): + logger.warning( + 'Target directory %s already exists and is ' + 'a link. Pip will not automatically replace ' + 'links, please remove if replacement is ' + 'desired.', + target_item_dir + ) + continue + if os.path.isdir(target_item_dir): + shutil.rmtree(target_item_dir) + else: + os.remove(target_item_dir) + + shutil.move( + os.path.join(lib_dir, item), + target_item_dir + ) + + def _warn_about_conflicts(self, to_install): + try: + package_set, _dep_info = check_install_conflicts(to_install) + except Exception: + logger.error("Error checking for conflicts.", exc_info=True) + return + missing, conflicting = _dep_info + + # NOTE: There is some duplication here from pip check + for project_name in missing: + version = package_set[project_name][0] + for dependency in missing[project_name]: + logger.critical( + "%s %s requires %s, which is not installed.", + project_name, version, dependency[1], + ) + + for project_name in conflicting: + version = package_set[project_name][0] + for dep_name, dep_version, req in conflicting[project_name]: + logger.critical( + "%s %s has requirement %s, but you'll have %s %s which is " + "incompatible.", + project_name, version, req, dep_name, dep_version, + ) + + +def get_lib_location_guesses(*args, **kwargs): + scheme = distutils_scheme('', *args, **kwargs) + return [scheme['purelib'], scheme['platlib']] + + +def site_packages_writable(**kwargs): + return all( + test_writable_dir(d) for d in set(get_lib_location_guesses(**kwargs)) + ) + + +def decide_user_install( + use_user_site, # type: Optional[bool] + prefix_path=None, # type: Optional[str] + target_dir=None, # type: Optional[str] + root_path=None, # type: Optional[str] + isolated_mode=False, # type: bool +): + # type: (...) -> bool + """Determine whether to do a user install based on the input options. + + If use_user_site is False, no additional checks are done. + If use_user_site is True, it is checked for compatibility with other + options. + If use_user_site is None, the default behaviour depends on the environment, + which is provided by the other arguments. + """ + # In some cases (config from tox), use_user_site can be set to an integer + # rather than a bool, which 'use_user_site is False' wouldn't catch. + if (use_user_site is not None) and (not use_user_site): + logger.debug("Non-user install by explicit request") + return False + + if use_user_site: + if prefix_path: + raise CommandError( + "Can not combine '--user' and '--prefix' as they imply " + "different installation locations" + ) + if virtualenv_no_global(): + raise InstallationError( + "Can not perform a '--user' install. User site-packages " + "are not visible in this virtualenv." + ) + logger.debug("User install by explicit request") + return True + + # If we are here, user installs have not been explicitly requested/avoided + assert use_user_site is None + + # user install incompatible with --prefix/--target + if prefix_path or target_dir: + logger.debug("Non-user install due to --prefix or --target option") + return False + + # If user installs are not enabled, choose a non-user install + if not site.ENABLE_USER_SITE: + logger.debug("Non-user install because user site-packages disabled") + return False + + # If we have permission for a non-user install, do that, + # otherwise do a user install. + if site_packages_writable(root=root_path, isolated=isolated_mode): + logger.debug("Non-user install because site-packages writeable") + return False + + logger.info("Defaulting to user installation because normal site-packages " + "is not writeable") + return True + + +def warn_deprecated_install_options(requirement_set, options): + # type: (RequirementSet, Optional[List[str]]) -> None + """If any location-changing --install-option arguments were passed for + requirements or on the command-line, then show a deprecation warning. + """ + def format_options(option_names): + # type: (Iterable[str]) -> List[str] + return ["--{}".format(name.replace("_", "-")) for name in option_names] + + requirements = ( + requirement_set.unnamed_requirements + + list(requirement_set.requirements.values()) + ) + + offenders = [] + + for requirement in requirements: + install_options = requirement.options.get("install_options", []) + location_options = parse_distutils_args(install_options) + if location_options: + offenders.append( + "{!r} from {}".format( + format_options(location_options.keys()), requirement + ) + ) + + if options: + location_options = parse_distutils_args(options) + if location_options: + offenders.append( + "{!r} from command line".format( + format_options(location_options.keys()) + ) + ) + + if not offenders: + return + + deprecated( + reason=( + "Location-changing options found in --install-option: {}. " + "This configuration may cause unexpected behavior and is " + "unsupported.".format( + "; ".join(offenders) + ) + ), + replacement=( + "using pip-level options like --user, --prefix, --root, and " + "--target" + ), + gone_in="20.2", + issue=7309, + ) + + +def create_env_error_message(error, show_traceback, using_user_site): + """Format an error message for an EnvironmentError + + It may occur anytime during the execution of the install command. + """ + parts = [] + + # Mention the error if we are not going to show a traceback + parts.append("Could not install packages due to an EnvironmentError") + if not show_traceback: + parts.append(": ") + parts.append(str(error)) + else: + parts.append(".") + + # Spilt the error indication from a helper message (if any) + parts[-1] += "\n" + + # Suggest useful actions to the user: + # (1) using user site-packages or (2) verifying the permissions + if error.errno == errno.EACCES: + user_option_part = "Consider using the `--user` option" + permissions_part = "Check the permissions" + + if not using_user_site: + parts.extend([ + user_option_part, " or ", + permissions_part.lower(), + ]) + else: + parts.append(permissions_part) + parts.append(".\n") + + return "".join(parts).strip() + "\n" diff --git a/env/lib/python3.8/site-packages/pip/_internal/commands/list.py b/env/lib/python3.8/site-packages/pip/_internal/commands/list.py new file mode 100644 index 000000000..d0062063e --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/commands/list.py @@ -0,0 +1,315 @@ +# The following comment should be removed at some point in the future. +# mypy: disallow-untyped-defs=False + +from __future__ import absolute_import + +import json +import logging + +from pip._vendor import six +from pip._vendor.six.moves import zip_longest + +from pip._internal.cli import cmdoptions +from pip._internal.cli.req_command import IndexGroupCommand +from pip._internal.exceptions import CommandError +from pip._internal.index.package_finder import PackageFinder +from pip._internal.models.selection_prefs import SelectionPreferences +from pip._internal.self_outdated_check import make_link_collector +from pip._internal.utils.misc import ( + dist_is_editable, + get_installed_distributions, + write_output, +) +from pip._internal.utils.packaging import get_installer + +from pip._vendor.packaging.version import parse + +logger = logging.getLogger(__name__) + + +class ListCommand(IndexGroupCommand): + """ + List installed packages, including editables. + + Packages are listed in a case-insensitive sorted order. + """ + + usage = """ + %prog [options]""" + + def __init__(self, *args, **kw): + super(ListCommand, self).__init__(*args, **kw) + + cmd_opts = self.cmd_opts + + cmd_opts.add_option( + '-o', '--outdated', + action='store_true', + default=False, + help='List outdated packages') + cmd_opts.add_option( + '-u', '--uptodate', + action='store_true', + default=False, + help='List uptodate packages') + cmd_opts.add_option( + '-e', '--editable', + action='store_true', + default=False, + help='List editable projects.') + cmd_opts.add_option( + '-l', '--local', + action='store_true', + default=False, + help=('If in a virtualenv that has global access, do not list ' + 'globally-installed packages.'), + ) + self.cmd_opts.add_option( + '--user', + dest='user', + action='store_true', + default=False, + help='Only output packages installed in user-site.') + cmd_opts.add_option(cmdoptions.list_path()) + cmd_opts.add_option( + '--pre', + action='store_true', + default=False, + help=("Include pre-release and development versions. By default, " + "pip only finds stable versions."), + ) + + cmd_opts.add_option( + '--format', + action='store', + dest='list_format', + default="columns", + choices=('columns', 'freeze', 'json'), + help="Select the output format among: columns (default), freeze, " + "or json", + ) + + cmd_opts.add_option( + '--not-required', + action='store_true', + dest='not_required', + help="List packages that are not dependencies of " + "installed packages.", + ) + + cmd_opts.add_option( + '--exclude-editable', + action='store_false', + dest='include_editable', + help='Exclude editable package from output.', + ) + cmd_opts.add_option( + '--include-editable', + action='store_true', + dest='include_editable', + help='Include editable package from output.', + default=True, + ) + index_opts = cmdoptions.make_option_group( + cmdoptions.index_group, self.parser + ) + + self.parser.insert_option_group(0, index_opts) + self.parser.insert_option_group(0, cmd_opts) + + def _build_package_finder(self, options, session): + """ + Create a package finder appropriate to this list command. + """ + link_collector = make_link_collector(session, options=options) + + # Pass allow_yanked=False to ignore yanked versions. + selection_prefs = SelectionPreferences( + allow_yanked=False, + allow_all_prereleases=options.pre, + ) + + return PackageFinder.create( + link_collector=link_collector, + selection_prefs=selection_prefs, + ) + + def run(self, options, args): + if options.outdated and options.uptodate: + raise CommandError( + "Options --outdated and --uptodate cannot be combined.") + + cmdoptions.check_list_path_option(options) + + packages = get_installed_distributions( + local_only=options.local, + user_only=options.user, + editables_only=options.editable, + include_editables=options.include_editable, + paths=options.path, + ) + + # get_not_required must be called firstly in order to find and + # filter out all dependencies correctly. Otherwise a package + # can't be identified as requirement because some parent packages + # could be filtered out before. + if options.not_required: + packages = self.get_not_required(packages, options) + + if options.outdated: + packages = self.get_outdated(packages, options) + elif options.uptodate: + packages = self.get_uptodate(packages, options) + + self.output_package_listing(packages, options) + + def get_outdated(self, packages, options): + return [ + dist for dist in self.iter_packages_latest_infos(packages, options) + if parse(str(dist.latest_version)) > parse(str(dist.parsed_version)) + ] + + def get_uptodate(self, packages, options): + return [ + dist for dist in self.iter_packages_latest_infos(packages, options) + if parse(str(dist.latest_version)) == parse(str(dist.parsed_version)) + ] + + def get_not_required(self, packages, options): + dep_keys = set() + for dist in packages: + dep_keys.update(requirement.key for requirement in dist.requires()) + return {pkg for pkg in packages if pkg.key not in dep_keys} + + def iter_packages_latest_infos(self, packages, options): + with self._build_session(options) as session: + finder = self._build_package_finder(options, session) + + for dist in packages: + typ = 'unknown' + all_candidates = finder.find_all_candidates(dist.key) + if not options.pre: + # Remove prereleases + all_candidates = [candidate for candidate in all_candidates + if not candidate.version.is_prerelease] + + evaluator = finder.make_candidate_evaluator( + project_name=dist.project_name, + ) + best_candidate = evaluator.sort_best_candidate(all_candidates) + if best_candidate is None: + continue + + remote_version = best_candidate.version + if best_candidate.link.is_wheel: + typ = 'wheel' + else: + typ = 'sdist' + # This is dirty but makes the rest of the code much cleaner + dist.latest_version = remote_version + dist.latest_filetype = typ + yield dist + + def output_package_listing(self, packages, options): + packages = sorted( + packages, + key=lambda dist: dist.project_name.lower(), + ) + if options.list_format == 'columns' and packages: + data, header = format_for_columns(packages, options) + self.output_package_listing_columns(data, header) + elif options.list_format == 'freeze': + for dist in packages: + if options.verbose >= 1: + write_output("%s==%s (%s)", dist.project_name, + dist.version, dist.location) + else: + write_output("%s==%s", dist.project_name, dist.version) + elif options.list_format == 'json': + write_output(format_for_json(packages, options)) + + def output_package_listing_columns(self, data, header): + # insert the header first: we need to know the size of column names + if len(data) > 0: + data.insert(0, header) + + pkg_strings, sizes = tabulate(data) + + # Create and add a separator. + if len(data) > 0: + pkg_strings.insert(1, " ".join(map(lambda x: '-' * x, sizes))) + + for val in pkg_strings: + write_output(val) + + +def tabulate(vals): + # From pfmoore on GitHub: + # https://github.com/pypa/pip/issues/3651#issuecomment-216932564 + assert len(vals) > 0 + + sizes = [0] * max(len(x) for x in vals) + for row in vals: + sizes = [max(s, len(str(c))) for s, c in zip_longest(sizes, row)] + + result = [] + for row in vals: + display = " ".join([str(c).ljust(s) if c is not None else '' + for s, c in zip_longest(sizes, row)]) + result.append(display) + + return result, sizes + + +def format_for_columns(pkgs, options): + """ + Convert the package data into something usable + by output_package_listing_columns. + """ + running_outdated = options.outdated + # Adjust the header for the `pip list --outdated` case. + if running_outdated: + header = ["Package", "Version", "Latest", "Type"] + else: + header = ["Package", "Version"] + + data = [] + if options.verbose >= 1 or any(dist_is_editable(x) for x in pkgs): + header.append("Location") + if options.verbose >= 1: + header.append("Installer") + + for proj in pkgs: + # if we're working on the 'outdated' list, separate out the + # latest_version and type + row = [proj.project_name, proj.version] + + if running_outdated: + row.append(proj.latest_version) + row.append(proj.latest_filetype) + + if options.verbose >= 1 or dist_is_editable(proj): + row.append(proj.location) + if options.verbose >= 1: + row.append(get_installer(proj)) + + data.append(row) + + return data, header + + +def format_for_json(packages, options): + data = [] + for dist in packages: + info = { + 'name': dist.project_name, + 'version': six.text_type(dist.version), + } + if options.verbose >= 1: + info['location'] = dist.location + info['installer'] = get_installer(dist) + if options.outdated: + info['latest_version'] = six.text_type(dist.latest_version) + info['latest_filetype'] = dist.latest_filetype + data.append(info) + return json.dumps(data) diff --git a/env/lib/python3.8/site-packages/pip/_internal/commands/search.py b/env/lib/python3.8/site-packages/pip/_internal/commands/search.py new file mode 100644 index 000000000..2e880eec2 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/commands/search.py @@ -0,0 +1,145 @@ +# The following comment should be removed at some point in the future. +# mypy: disallow-untyped-defs=False + +from __future__ import absolute_import + +import logging +import sys +import textwrap +from collections import OrderedDict + +from pip._vendor import pkg_resources +from pip._vendor.packaging.version import parse as parse_version +# NOTE: XMLRPC Client is not annotated in typeshed as on 2017-07-17, which is +# why we ignore the type on this import +from pip._vendor.six.moves import xmlrpc_client # type: ignore + +from pip._internal.cli.base_command import Command +from pip._internal.cli.req_command import SessionCommandMixin +from pip._internal.cli.status_codes import NO_MATCHES_FOUND, SUCCESS +from pip._internal.exceptions import CommandError +from pip._internal.models.index import PyPI +from pip._internal.network.xmlrpc import PipXmlrpcTransport +from pip._internal.utils.compat import get_terminal_size +from pip._internal.utils.logging import indent_log +from pip._internal.utils.misc import write_output + +logger = logging.getLogger(__name__) + + +class SearchCommand(Command, SessionCommandMixin): + """Search for PyPI packages whose name or summary contains .""" + + usage = """ + %prog [options] """ + ignore_require_venv = True + + def __init__(self, *args, **kw): + super(SearchCommand, self).__init__(*args, **kw) + self.cmd_opts.add_option( + '-i', '--index', + dest='index', + metavar='URL', + default=PyPI.pypi_url, + help='Base URL of Python Package Index (default %default)') + + self.parser.insert_option_group(0, self.cmd_opts) + + def run(self, options, args): + if not args: + raise CommandError('Missing required argument (search query).') + query = args + pypi_hits = self.search(query, options) + hits = transform_hits(pypi_hits) + + terminal_width = None + if sys.stdout.isatty(): + terminal_width = get_terminal_size()[0] + + print_results(hits, terminal_width=terminal_width) + if pypi_hits: + return SUCCESS + return NO_MATCHES_FOUND + + def search(self, query, options): + index_url = options.index + + session = self.get_default_session(options) + + transport = PipXmlrpcTransport(index_url, session) + pypi = xmlrpc_client.ServerProxy(index_url, transport) + hits = pypi.search({'name': query, 'summary': query}, 'or') + return hits + + +def transform_hits(hits): + """ + The list from pypi is really a list of versions. We want a list of + packages with the list of versions stored inline. This converts the + list from pypi into one we can use. + """ + packages = OrderedDict() + for hit in hits: + name = hit['name'] + summary = hit['summary'] + version = hit['version'] + + if name not in packages.keys(): + packages[name] = { + 'name': name, + 'summary': summary, + 'versions': [version], + } + else: + packages[name]['versions'].append(version) + + # if this is the highest version, replace summary and score + if version == highest_version(packages[name]['versions']): + packages[name]['summary'] = summary + + return list(packages.values()) + + +def print_results(hits, name_column_width=None, terminal_width=None): + if not hits: + return + if name_column_width is None: + name_column_width = max([ + len(hit['name']) + len(highest_version(hit.get('versions', ['-']))) + for hit in hits + ]) + 4 + + installed_packages = [p.project_name for p in pkg_resources.working_set] + for hit in hits: + name = hit['name'] + summary = hit['summary'] or '' + latest = highest_version(hit.get('versions', ['-'])) + if terminal_width is not None: + target_width = terminal_width - name_column_width - 5 + if target_width > 10: + # wrap and indent summary to fit terminal + summary = textwrap.wrap(summary, target_width) + summary = ('\n' + ' ' * (name_column_width + 3)).join(summary) + + line = '%-*s - %s' % (name_column_width, + '%s (%s)' % (name, latest), summary) + try: + write_output(line) + if name in installed_packages: + dist = pkg_resources.get_distribution(name) + with indent_log(): + if dist.version == latest: + write_output('INSTALLED: %s (latest)', dist.version) + else: + write_output('INSTALLED: %s', dist.version) + if parse_version(latest).pre: + write_output('LATEST: %s (pre-release; install' + ' with "pip install --pre")', latest) + else: + write_output('LATEST: %s', latest) + except UnicodeEncodeError: + pass + + +def highest_version(versions): + return max(versions, key=parse_version) diff --git a/env/lib/python3.8/site-packages/pip/_internal/commands/show.py b/env/lib/python3.8/site-packages/pip/_internal/commands/show.py new file mode 100644 index 000000000..a46b08eeb --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/commands/show.py @@ -0,0 +1,180 @@ +# The following comment should be removed at some point in the future. +# mypy: disallow-untyped-defs=False + +from __future__ import absolute_import + +import logging +import os +from email.parser import FeedParser + +from pip._vendor import pkg_resources +from pip._vendor.packaging.utils import canonicalize_name + +from pip._internal.cli.base_command import Command +from pip._internal.cli.status_codes import ERROR, SUCCESS +from pip._internal.utils.misc import write_output + +logger = logging.getLogger(__name__) + + +class ShowCommand(Command): + """ + Show information about one or more installed packages. + + The output is in RFC-compliant mail header format. + """ + + usage = """ + %prog [options] ...""" + ignore_require_venv = True + + def __init__(self, *args, **kw): + super(ShowCommand, self).__init__(*args, **kw) + self.cmd_opts.add_option( + '-f', '--files', + dest='files', + action='store_true', + default=False, + help='Show the full list of installed files for each package.') + + self.parser.insert_option_group(0, self.cmd_opts) + + def run(self, options, args): + if not args: + logger.warning('ERROR: Please provide a package name or names.') + return ERROR + query = args + + results = search_packages_info(query) + if not print_results( + results, list_files=options.files, verbose=options.verbose): + return ERROR + return SUCCESS + + +def search_packages_info(query): + """ + Gather details from installed distributions. Print distribution name, + version, location, and installed files. Installed files requires a + pip generated 'installed-files.txt' in the distributions '.egg-info' + directory. + """ + installed = {} + for p in pkg_resources.working_set: + installed[canonicalize_name(p.project_name)] = p + + query_names = [canonicalize_name(name) for name in query] + missing = sorted( + [name for name, pkg in zip(query, query_names) if pkg not in installed] + ) + if missing: + logger.warning('Package(s) not found: %s', ', '.join(missing)) + + def get_requiring_packages(package_name): + canonical_name = canonicalize_name(package_name) + return [ + pkg.project_name for pkg in pkg_resources.working_set + if canonical_name in + [canonicalize_name(required.name) for required in + pkg.requires()] + ] + + for dist in [installed[pkg] for pkg in query_names if pkg in installed]: + package = { + 'name': dist.project_name, + 'version': dist.version, + 'location': dist.location, + 'requires': [dep.project_name for dep in dist.requires()], + 'required_by': get_requiring_packages(dist.project_name) + } + file_list = None + metadata = None + if isinstance(dist, pkg_resources.DistInfoDistribution): + # RECORDs should be part of .dist-info metadatas + if dist.has_metadata('RECORD'): + lines = dist.get_metadata_lines('RECORD') + paths = [l.split(',')[0] for l in lines] + paths = [os.path.join(dist.location, p) for p in paths] + file_list = [os.path.relpath(p, dist.location) for p in paths] + + if dist.has_metadata('METADATA'): + metadata = dist.get_metadata('METADATA') + else: + # Otherwise use pip's log for .egg-info's + if dist.has_metadata('installed-files.txt'): + paths = dist.get_metadata_lines('installed-files.txt') + paths = [os.path.join(dist.egg_info, p) for p in paths] + file_list = [os.path.relpath(p, dist.location) for p in paths] + + if dist.has_metadata('PKG-INFO'): + metadata = dist.get_metadata('PKG-INFO') + + if dist.has_metadata('entry_points.txt'): + entry_points = dist.get_metadata_lines('entry_points.txt') + package['entry_points'] = entry_points + + if dist.has_metadata('INSTALLER'): + for line in dist.get_metadata_lines('INSTALLER'): + if line.strip(): + package['installer'] = line.strip() + break + + # @todo: Should pkg_resources.Distribution have a + # `get_pkg_info` method? + feed_parser = FeedParser() + feed_parser.feed(metadata) + pkg_info_dict = feed_parser.close() + for key in ('metadata-version', 'summary', + 'home-page', 'author', 'author-email', 'license'): + package[key] = pkg_info_dict.get(key) + + # It looks like FeedParser cannot deal with repeated headers + classifiers = [] + for line in metadata.splitlines(): + if line.startswith('Classifier: '): + classifiers.append(line[len('Classifier: '):]) + package['classifiers'] = classifiers + + if file_list: + package['files'] = sorted(file_list) + yield package + + +def print_results(distributions, list_files=False, verbose=False): + """ + Print the informations from installed distributions found. + """ + results_printed = False + for i, dist in enumerate(distributions): + results_printed = True + if i > 0: + write_output("---") + + write_output("Name: %s", dist.get('name', '')) + write_output("Version: %s", dist.get('version', '')) + write_output("Summary: %s", dist.get('summary', '')) + write_output("Home-page: %s", dist.get('home-page', '')) + write_output("Author: %s", dist.get('author', '')) + write_output("Author-email: %s", dist.get('author-email', '')) + write_output("License: %s", dist.get('license', '')) + write_output("Location: %s", dist.get('location', '')) + write_output("Requires: %s", ', '.join(dist.get('requires', []))) + write_output("Required-by: %s", ', '.join(dist.get('required_by', []))) + + if verbose: + write_output("Metadata-Version: %s", + dist.get('metadata-version', '')) + write_output("Installer: %s", dist.get('installer', '')) + write_output("Classifiers:") + for classifier in dist.get('classifiers', []): + write_output(" %s", classifier) + write_output("Entry-points:") + for entry in dist.get('entry_points', []): + write_output(" %s", entry.strip()) + if list_files: + write_output("Files:") + for line in dist.get('files', []): + write_output(" %s", line.strip()) + if "files" not in dist: + write_output("Cannot locate installed-files.txt") + return results_printed diff --git a/env/lib/python3.8/site-packages/pip/_internal/commands/uninstall.py b/env/lib/python3.8/site-packages/pip/_internal/commands/uninstall.py new file mode 100644 index 000000000..1bde414a6 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/commands/uninstall.py @@ -0,0 +1,82 @@ +# The following comment should be removed at some point in the future. +# mypy: disallow-untyped-defs=False + +from __future__ import absolute_import + +from pip._vendor.packaging.utils import canonicalize_name + +from pip._internal.cli.base_command import Command +from pip._internal.cli.req_command import SessionCommandMixin +from pip._internal.exceptions import InstallationError +from pip._internal.req import parse_requirements +from pip._internal.req.constructors import install_req_from_line +from pip._internal.utils.misc import protect_pip_from_modification_on_windows + + +class UninstallCommand(Command, SessionCommandMixin): + """ + Uninstall packages. + + pip is able to uninstall most installed packages. Known exceptions are: + + - Pure distutils packages installed with ``python setup.py install``, which + leave behind no metadata to determine what files were installed. + - Script wrappers installed by ``python setup.py develop``. + """ + + usage = """ + %prog [options] ... + %prog [options] -r ...""" + + def __init__(self, *args, **kw): + super(UninstallCommand, self).__init__(*args, **kw) + self.cmd_opts.add_option( + '-r', '--requirement', + dest='requirements', + action='append', + default=[], + metavar='file', + help='Uninstall all the packages listed in the given requirements ' + 'file. This option can be used multiple times.', + ) + self.cmd_opts.add_option( + '-y', '--yes', + dest='yes', + action='store_true', + help="Don't ask for confirmation of uninstall deletions.") + + self.parser.insert_option_group(0, self.cmd_opts) + + def run(self, options, args): + session = self.get_default_session(options) + + reqs_to_uninstall = {} + for name in args: + req = install_req_from_line( + name, isolated=options.isolated_mode, + ) + if req.name: + reqs_to_uninstall[canonicalize_name(req.name)] = req + for filename in options.requirements: + for req in parse_requirements( + filename, + options=options, + session=session): + if req.name: + reqs_to_uninstall[canonicalize_name(req.name)] = req + if not reqs_to_uninstall: + raise InstallationError( + 'You must give at least one requirement to %(name)s (see ' + '"pip help %(name)s")' % dict(name=self.name) + ) + + protect_pip_from_modification_on_windows( + modifying_pip="pip" in reqs_to_uninstall + ) + + for req in reqs_to_uninstall.values(): + uninstall_pathset = req.uninstall( + auto_confirm=options.yes, verbose=self.verbosity > 0, + ) + if uninstall_pathset: + uninstall_pathset.commit() diff --git a/env/lib/python3.8/site-packages/pip/_internal/commands/wheel.py b/env/lib/python3.8/site-packages/pip/_internal/commands/wheel.py new file mode 100644 index 000000000..eb44bcee4 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/commands/wheel.py @@ -0,0 +1,197 @@ +# -*- coding: utf-8 -*- + +# The following comment should be removed at some point in the future. +# mypy: disallow-untyped-defs=False + +from __future__ import absolute_import + +import logging +import os +import shutil + +from pip._internal.cache import WheelCache +from pip._internal.cli import cmdoptions +from pip._internal.cli.req_command import RequirementCommand +from pip._internal.exceptions import CommandError, PreviousBuildDirError +from pip._internal.req import RequirementSet +from pip._internal.req.req_tracker import get_requirement_tracker +from pip._internal.utils.misc import ensure_dir, normalize_path +from pip._internal.utils.temp_dir import TempDirectory +from pip._internal.utils.typing import MYPY_CHECK_RUNNING +from pip._internal.wheel_builder import build, should_build_for_wheel_command + +if MYPY_CHECK_RUNNING: + from optparse import Values + from typing import Any, List + + +logger = logging.getLogger(__name__) + + +class WheelCommand(RequirementCommand): + """ + Build Wheel archives for your requirements and dependencies. + + Wheel is a built-package format, and offers the advantage of not + recompiling your software during every install. For more details, see the + wheel docs: https://wheel.readthedocs.io/en/latest/ + + Requirements: setuptools>=0.8, and wheel. + + 'pip wheel' uses the bdist_wheel setuptools extension from the wheel + package to build individual wheels. + + """ + + usage = """ + %prog [options] ... + %prog [options] -r ... + %prog [options] [-e] ... + %prog [options] [-e] ... + %prog [options] ...""" + + def __init__(self, *args, **kw): + super(WheelCommand, self).__init__(*args, **kw) + + cmd_opts = self.cmd_opts + + cmd_opts.add_option( + '-w', '--wheel-dir', + dest='wheel_dir', + metavar='dir', + default=os.curdir, + help=("Build wheels into , where the default is the " + "current working directory."), + ) + cmd_opts.add_option(cmdoptions.no_binary()) + cmd_opts.add_option(cmdoptions.only_binary()) + cmd_opts.add_option(cmdoptions.prefer_binary()) + cmd_opts.add_option( + '--build-option', + dest='build_options', + metavar='options', + action='append', + help="Extra arguments to be supplied to 'setup.py bdist_wheel'.", + ) + cmd_opts.add_option(cmdoptions.no_build_isolation()) + cmd_opts.add_option(cmdoptions.use_pep517()) + cmd_opts.add_option(cmdoptions.no_use_pep517()) + cmd_opts.add_option(cmdoptions.constraints()) + cmd_opts.add_option(cmdoptions.editable()) + cmd_opts.add_option(cmdoptions.requirements()) + cmd_opts.add_option(cmdoptions.src()) + cmd_opts.add_option(cmdoptions.ignore_requires_python()) + cmd_opts.add_option(cmdoptions.no_deps()) + cmd_opts.add_option(cmdoptions.build_dir()) + cmd_opts.add_option(cmdoptions.progress_bar()) + + cmd_opts.add_option( + '--global-option', + dest='global_options', + action='append', + metavar='options', + help="Extra global options to be supplied to the setup.py " + "call before the 'bdist_wheel' command.") + + cmd_opts.add_option( + '--pre', + action='store_true', + default=False, + help=("Include pre-release and development versions. By default, " + "pip only finds stable versions."), + ) + + cmd_opts.add_option(cmdoptions.no_clean()) + cmd_opts.add_option(cmdoptions.require_hashes()) + + index_opts = cmdoptions.make_option_group( + cmdoptions.index_group, + self.parser, + ) + + self.parser.insert_option_group(0, index_opts) + self.parser.insert_option_group(0, cmd_opts) + + def run(self, options, args): + # type: (Values, List[Any]) -> None + cmdoptions.check_install_build_global(options) + + session = self.get_default_session(options) + + finder = self._build_package_finder(options, session) + build_delete = (not (options.no_clean or options.build_dir)) + wheel_cache = WheelCache(options.cache_dir, options.format_control) + + options.wheel_dir = normalize_path(options.wheel_dir) + ensure_dir(options.wheel_dir) + + with get_requirement_tracker() as req_tracker, TempDirectory( + options.build_dir, delete=build_delete, kind="wheel" + ) as directory: + + requirement_set = RequirementSet() + + try: + self.populate_requirement_set( + requirement_set, args, options, finder, session, + wheel_cache + ) + + preparer = self.make_requirement_preparer( + temp_build_dir=directory, + options=options, + req_tracker=req_tracker, + session=session, + finder=finder, + wheel_download_dir=options.wheel_dir, + use_user_site=False, + ) + + resolver = self.make_resolver( + preparer=preparer, + finder=finder, + options=options, + wheel_cache=wheel_cache, + ignore_requires_python=options.ignore_requires_python, + use_pep517=options.use_pep517, + ) + + self.trace_basic_info(finder) + + resolver.resolve(requirement_set) + + reqs_to_build = [ + r for r in requirement_set.requirements.values() + if should_build_for_wheel_command(r) + ] + + # build wheels + build_successes, build_failures = build( + reqs_to_build, + wheel_cache=wheel_cache, + build_options=options.build_options or [], + global_options=options.global_options or [], + ) + for req in build_successes: + assert req.link and req.link.is_wheel + assert req.local_file_path + # copy from cache to target directory + try: + shutil.copy(req.local_file_path, options.wheel_dir) + except OSError as e: + logger.warning( + "Building wheel for %s failed: %s", + req.name, e, + ) + build_failures.append(req) + if len(build_failures) != 0: + raise CommandError( + "Failed to build one or more wheels" + ) + except PreviousBuildDirError: + options.no_clean = True + raise + finally: + if not options.no_clean: + requirement_set.cleanup_files() + wheel_cache.cleanup() diff --git a/env/lib/python3.8/site-packages/pip/_internal/configuration.py b/env/lib/python3.8/site-packages/pip/_internal/configuration.py new file mode 100644 index 000000000..f09a1ae25 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/configuration.py @@ -0,0 +1,422 @@ +"""Configuration management setup + +Some terminology: +- name + As written in config files. +- value + Value associated with a name +- key + Name combined with it's section (section.name) +- variant + A single word describing where the configuration key-value pair came from +""" + +# The following comment should be removed at some point in the future. +# mypy: strict-optional=False + +import locale +import logging +import os +import sys + +from pip._vendor.six.moves import configparser + +from pip._internal.exceptions import ( + ConfigurationError, + ConfigurationFileCouldNotBeLoaded, +) +from pip._internal.utils import appdirs +from pip._internal.utils.compat import WINDOWS, expanduser +from pip._internal.utils.misc import ensure_dir, enum +from pip._internal.utils.typing import MYPY_CHECK_RUNNING + +if MYPY_CHECK_RUNNING: + from typing import ( + Any, Dict, Iterable, List, NewType, Optional, Tuple + ) + + RawConfigParser = configparser.RawConfigParser # Shorthand + Kind = NewType("Kind", str) + +logger = logging.getLogger(__name__) + + +# NOTE: Maybe use the optionx attribute to normalize keynames. +def _normalize_name(name): + # type: (str) -> str + """Make a name consistent regardless of source (environment or file) + """ + name = name.lower().replace('_', '-') + if name.startswith('--'): + name = name[2:] # only prefer long opts + return name + + +def _disassemble_key(name): + # type: (str) -> List[str] + if "." not in name: + error_message = ( + "Key does not contain dot separated section and key. " + "Perhaps you wanted to use 'global.{}' instead?" + ).format(name) + raise ConfigurationError(error_message) + return name.split(".", 1) + + +# The kinds of configurations there are. +kinds = enum( + USER="user", # User Specific + GLOBAL="global", # System Wide + SITE="site", # [Virtual] Environment Specific + ENV="env", # from PIP_CONFIG_FILE + ENV_VAR="env-var", # from Environment Variables +) + + +CONFIG_BASENAME = 'pip.ini' if WINDOWS else 'pip.conf' + + +def get_configuration_files(): + # type: () -> Dict[Kind, List[str]] + global_config_files = [ + os.path.join(path, CONFIG_BASENAME) + for path in appdirs.site_config_dirs('pip') + ] + + site_config_file = os.path.join(sys.prefix, CONFIG_BASENAME) + legacy_config_file = os.path.join( + expanduser('~'), + 'pip' if WINDOWS else '.pip', + CONFIG_BASENAME, + ) + new_config_file = os.path.join( + appdirs.user_config_dir("pip"), CONFIG_BASENAME + ) + return { + kinds.GLOBAL: global_config_files, + kinds.SITE: [site_config_file], + kinds.USER: [legacy_config_file, new_config_file], + } + + +class Configuration(object): + """Handles management of configuration. + + Provides an interface to accessing and managing configuration files. + + This class converts provides an API that takes "section.key-name" style + keys and stores the value associated with it as "key-name" under the + section "section". + + This allows for a clean interface wherein the both the section and the + key-name are preserved in an easy to manage form in the configuration files + and the data stored is also nice. + """ + + def __init__(self, isolated, load_only=None): + # type: (bool, Kind) -> None + super(Configuration, self).__init__() + + _valid_load_only = [kinds.USER, kinds.GLOBAL, kinds.SITE, None] + if load_only not in _valid_load_only: + raise ConfigurationError( + "Got invalid value for load_only - should be one of {}".format( + ", ".join(map(repr, _valid_load_only[:-1])) + ) + ) + self.isolated = isolated # type: bool + self.load_only = load_only # type: Optional[Kind] + + # The order here determines the override order. + self._override_order = [ + kinds.GLOBAL, kinds.USER, kinds.SITE, kinds.ENV, kinds.ENV_VAR + ] + + self._ignore_env_names = ["version", "help"] + + # Because we keep track of where we got the data from + self._parsers = { + variant: [] for variant in self._override_order + } # type: Dict[Kind, List[Tuple[str, RawConfigParser]]] + self._config = { + variant: {} for variant in self._override_order + } # type: Dict[Kind, Dict[str, Any]] + self._modified_parsers = [] # type: List[Tuple[str, RawConfigParser]] + + def load(self): + # type: () -> None + """Loads configuration from configuration files and environment + """ + self._load_config_files() + if not self.isolated: + self._load_environment_vars() + + def get_file_to_edit(self): + # type: () -> Optional[str] + """Returns the file with highest priority in configuration + """ + assert self.load_only is not None, \ + "Need to be specified a file to be editing" + + try: + return self._get_parser_to_modify()[0] + except IndexError: + return None + + def items(self): + # type: () -> Iterable[Tuple[str, Any]] + """Returns key-value pairs like dict.items() representing the loaded + configuration + """ + return self._dictionary.items() + + def get_value(self, key): + # type: (str) -> Any + """Get a value from the configuration. + """ + try: + return self._dictionary[key] + except KeyError: + raise ConfigurationError("No such key - {}".format(key)) + + def set_value(self, key, value): + # type: (str, Any) -> None + """Modify a value in the configuration. + """ + self._ensure_have_load_only() + + fname, parser = self._get_parser_to_modify() + + if parser is not None: + section, name = _disassemble_key(key) + + # Modify the parser and the configuration + if not parser.has_section(section): + parser.add_section(section) + parser.set(section, name, value) + + self._config[self.load_only][key] = value + self._mark_as_modified(fname, parser) + + def unset_value(self, key): + # type: (str) -> None + """Unset a value in the configuration. + """ + self._ensure_have_load_only() + + if key not in self._config[self.load_only]: + raise ConfigurationError("No such key - {}".format(key)) + + fname, parser = self._get_parser_to_modify() + + if parser is not None: + section, name = _disassemble_key(key) + + # Remove the key in the parser + modified_something = False + if parser.has_section(section): + # Returns whether the option was removed or not + modified_something = parser.remove_option(section, name) + + if modified_something: + # name removed from parser, section may now be empty + section_iter = iter(parser.items(section)) + try: + val = next(section_iter) + except StopIteration: + val = None + + if val is None: + parser.remove_section(section) + + self._mark_as_modified(fname, parser) + else: + raise ConfigurationError( + "Fatal Internal error [id=1]. Please report as a bug." + ) + + del self._config[self.load_only][key] + + def save(self): + # type: () -> None + """Save the current in-memory state. + """ + self._ensure_have_load_only() + + for fname, parser in self._modified_parsers: + logger.info("Writing to %s", fname) + + # Ensure directory exists. + ensure_dir(os.path.dirname(fname)) + + with open(fname, "w") as f: + parser.write(f) + + # + # Private routines + # + + def _ensure_have_load_only(self): + # type: () -> None + if self.load_only is None: + raise ConfigurationError("Needed a specific file to be modifying.") + logger.debug("Will be working with %s variant only", self.load_only) + + @property + def _dictionary(self): + # type: () -> Dict[str, Any] + """A dictionary representing the loaded configuration. + """ + # NOTE: Dictionaries are not populated if not loaded. So, conditionals + # are not needed here. + retval = {} + + for variant in self._override_order: + retval.update(self._config[variant]) + + return retval + + def _load_config_files(self): + # type: () -> None + """Loads configuration from configuration files + """ + config_files = dict(self._iter_config_files()) + if config_files[kinds.ENV][0:1] == [os.devnull]: + logger.debug( + "Skipping loading configuration files due to " + "environment's PIP_CONFIG_FILE being os.devnull" + ) + return + + for variant, files in config_files.items(): + for fname in files: + # If there's specific variant set in `load_only`, load only + # that variant, not the others. + if self.load_only is not None and variant != self.load_only: + logger.debug( + "Skipping file '%s' (variant: %s)", fname, variant + ) + continue + + parser = self._load_file(variant, fname) + + # Keeping track of the parsers used + self._parsers[variant].append((fname, parser)) + + def _load_file(self, variant, fname): + # type: (Kind, str) -> RawConfigParser + logger.debug("For variant '%s', will try loading '%s'", variant, fname) + parser = self._construct_parser(fname) + + for section in parser.sections(): + items = parser.items(section) + self._config[variant].update(self._normalized_keys(section, items)) + + return parser + + def _construct_parser(self, fname): + # type: (str) -> RawConfigParser + parser = configparser.RawConfigParser() + # If there is no such file, don't bother reading it but create the + # parser anyway, to hold the data. + # Doing this is useful when modifying and saving files, where we don't + # need to construct a parser. + if os.path.exists(fname): + try: + parser.read(fname) + except UnicodeDecodeError: + # See https://github.com/pypa/pip/issues/4963 + raise ConfigurationFileCouldNotBeLoaded( + reason="contains invalid {} characters".format( + locale.getpreferredencoding(False) + ), + fname=fname, + ) + except configparser.Error as error: + # See https://github.com/pypa/pip/issues/4893 + raise ConfigurationFileCouldNotBeLoaded(error=error) + return parser + + def _load_environment_vars(self): + # type: () -> None + """Loads configuration from environment variables + """ + self._config[kinds.ENV_VAR].update( + self._normalized_keys(":env:", self._get_environ_vars()) + ) + + def _normalized_keys(self, section, items): + # type: (str, Iterable[Tuple[str, Any]]) -> Dict[str, Any] + """Normalizes items to construct a dictionary with normalized keys. + + This routine is where the names become keys and are made the same + regardless of source - configuration files or environment. + """ + normalized = {} + for name, val in items: + key = section + "." + _normalize_name(name) + normalized[key] = val + return normalized + + def _get_environ_vars(self): + # type: () -> Iterable[Tuple[str, str]] + """Returns a generator with all environmental vars with prefix PIP_""" + for key, val in os.environ.items(): + should_be_yielded = ( + key.startswith("PIP_") and + key[4:].lower() not in self._ignore_env_names + ) + if should_be_yielded: + yield key[4:].lower(), val + + # XXX: This is patched in the tests. + def _iter_config_files(self): + # type: () -> Iterable[Tuple[Kind, List[str]]] + """Yields variant and configuration files associated with it. + + This should be treated like items of a dictionary. + """ + # SMELL: Move the conditions out of this function + + # environment variables have the lowest priority + config_file = os.environ.get('PIP_CONFIG_FILE', None) + if config_file is not None: + yield kinds.ENV, [config_file] + else: + yield kinds.ENV, [] + + config_files = get_configuration_files() + + # at the base we have any global configuration + yield kinds.GLOBAL, config_files[kinds.GLOBAL] + + # per-user configuration next + should_load_user_config = not self.isolated and not ( + config_file and os.path.exists(config_file) + ) + if should_load_user_config: + # The legacy config file is overridden by the new config file + yield kinds.USER, config_files[kinds.USER] + + # finally virtualenv configuration first trumping others + yield kinds.SITE, config_files[kinds.SITE] + + def _get_parser_to_modify(self): + # type: () -> Tuple[str, RawConfigParser] + # Determine which parser to modify + parsers = self._parsers[self.load_only] + if not parsers: + # This should not happen if everything works correctly. + raise ConfigurationError( + "Fatal Internal error [id=2]. Please report as a bug." + ) + + # Use the highest priority parser. + return parsers[-1] + + # XXX: This is patched in the tests. + def _mark_as_modified(self, fname, parser): + # type: (str, RawConfigParser) -> None + file_parser_tuple = (fname, parser) + if file_parser_tuple not in self._modified_parsers: + self._modified_parsers.append(file_parser_tuple) diff --git a/env/lib/python3.8/site-packages/pip/_internal/distributions/__init__.py b/env/lib/python3.8/site-packages/pip/_internal/distributions/__init__.py new file mode 100644 index 000000000..d5c1afc5b --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/distributions/__init__.py @@ -0,0 +1,24 @@ +from pip._internal.distributions.sdist import SourceDistribution +from pip._internal.distributions.wheel import WheelDistribution +from pip._internal.utils.typing import MYPY_CHECK_RUNNING + +if MYPY_CHECK_RUNNING: + from pip._internal.distributions.base import AbstractDistribution + from pip._internal.req.req_install import InstallRequirement + + +def make_distribution_for_install_requirement(install_req): + # type: (InstallRequirement) -> AbstractDistribution + """Returns a Distribution for the given InstallRequirement + """ + # Editable requirements will always be source distributions. They use the + # legacy logic until we create a modern standard for them. + if install_req.editable: + return SourceDistribution(install_req) + + # If it's a wheel, it's a WheelDistribution + if install_req.is_wheel: + return WheelDistribution(install_req) + + # Otherwise, a SourceDistribution + return SourceDistribution(install_req) diff --git a/env/lib/python3.8/site-packages/pip/_internal/distributions/base.py b/env/lib/python3.8/site-packages/pip/_internal/distributions/base.py new file mode 100644 index 000000000..b836b98d1 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/distributions/base.py @@ -0,0 +1,45 @@ +import abc + +from pip._vendor.six import add_metaclass + +from pip._internal.utils.typing import MYPY_CHECK_RUNNING + +if MYPY_CHECK_RUNNING: + from typing import Optional + + from pip._vendor.pkg_resources import Distribution + from pip._internal.req import InstallRequirement + from pip._internal.index.package_finder import PackageFinder + + +@add_metaclass(abc.ABCMeta) +class AbstractDistribution(object): + """A base class for handling installable artifacts. + + The requirements for anything installable are as follows: + + - we must be able to determine the requirement name + (or we can't correctly handle the non-upgrade case). + + - for packages with setup requirements, we must also be able + to determine their requirements without installing additional + packages (for the same reason as run-time dependencies) + + - we must be able to create a Distribution object exposing the + above metadata. + """ + + def __init__(self, req): + # type: (InstallRequirement) -> None + super(AbstractDistribution, self).__init__() + self.req = req + + @abc.abstractmethod + def get_pkg_resources_distribution(self): + # type: () -> Optional[Distribution] + raise NotImplementedError() + + @abc.abstractmethod + def prepare_distribution_metadata(self, finder, build_isolation): + # type: (PackageFinder, bool) -> None + raise NotImplementedError() diff --git a/env/lib/python3.8/site-packages/pip/_internal/distributions/installed.py b/env/lib/python3.8/site-packages/pip/_internal/distributions/installed.py new file mode 100644 index 000000000..0d15bf424 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/distributions/installed.py @@ -0,0 +1,24 @@ +from pip._internal.distributions.base import AbstractDistribution +from pip._internal.utils.typing import MYPY_CHECK_RUNNING + +if MYPY_CHECK_RUNNING: + from typing import Optional + + from pip._vendor.pkg_resources import Distribution + from pip._internal.index.package_finder import PackageFinder + + +class InstalledDistribution(AbstractDistribution): + """Represents an installed package. + + This does not need any preparation as the required information has already + been computed. + """ + + def get_pkg_resources_distribution(self): + # type: () -> Optional[Distribution] + return self.req.satisfied_by + + def prepare_distribution_metadata(self, finder, build_isolation): + # type: (PackageFinder, bool) -> None + pass diff --git a/env/lib/python3.8/site-packages/pip/_internal/distributions/sdist.py b/env/lib/python3.8/site-packages/pip/_internal/distributions/sdist.py new file mode 100644 index 000000000..be3d7d97a --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/distributions/sdist.py @@ -0,0 +1,104 @@ +import logging + +from pip._internal.build_env import BuildEnvironment +from pip._internal.distributions.base import AbstractDistribution +from pip._internal.exceptions import InstallationError +from pip._internal.utils.subprocess import runner_with_spinner_message +from pip._internal.utils.typing import MYPY_CHECK_RUNNING + +if MYPY_CHECK_RUNNING: + from typing import Set, Tuple + + from pip._vendor.pkg_resources import Distribution + from pip._internal.index.package_finder import PackageFinder + + +logger = logging.getLogger(__name__) + + +class SourceDistribution(AbstractDistribution): + """Represents a source distribution. + + The preparation step for these needs metadata for the packages to be + generated, either using PEP 517 or using the legacy `setup.py egg_info`. + """ + + def get_pkg_resources_distribution(self): + # type: () -> Distribution + return self.req.get_dist() + + def prepare_distribution_metadata(self, finder, build_isolation): + # type: (PackageFinder, bool) -> None + # Load pyproject.toml, to determine whether PEP 517 is to be used + self.req.load_pyproject_toml() + + # Set up the build isolation, if this requirement should be isolated + should_isolate = self.req.use_pep517 and build_isolation + if should_isolate: + self._setup_isolation(finder) + + self.req.prepare_metadata() + + def _setup_isolation(self, finder): + # type: (PackageFinder) -> None + def _raise_conflicts(conflicting_with, conflicting_reqs): + # type: (str, Set[Tuple[str, str]]) -> None + format_string = ( + "Some build dependencies for {requirement} " + "conflict with {conflicting_with}: {description}." + ) + error_message = format_string.format( + requirement=self.req, + conflicting_with=conflicting_with, + description=', '.join( + '{} is incompatible with {}'.format(installed, wanted) + for installed, wanted in sorted(conflicting) + ) + ) + raise InstallationError(error_message) + + # Isolate in a BuildEnvironment and install the build-time + # requirements. + pyproject_requires = self.req.pyproject_requires + assert pyproject_requires is not None + + self.req.build_env = BuildEnvironment() + self.req.build_env.install_requirements( + finder, pyproject_requires, 'overlay', + "Installing build dependencies" + ) + conflicting, missing = self.req.build_env.check_requirements( + self.req.requirements_to_check + ) + if conflicting: + _raise_conflicts("PEP 517/518 supported requirements", + conflicting) + if missing: + logger.warning( + "Missing build requirements in pyproject.toml for %s.", + self.req, + ) + logger.warning( + "The project does not specify a build backend, and " + "pip cannot fall back to setuptools without %s.", + " and ".join(map(repr, sorted(missing))) + ) + # Install any extra build dependencies that the backend requests. + # This must be done in a second pass, as the pyproject.toml + # dependencies must be installed before we can call the backend. + with self.req.build_env: + runner = runner_with_spinner_message( + "Getting requirements to build wheel" + ) + backend = self.req.pep517_backend + assert backend is not None + with backend.subprocess_runner(runner): + reqs = backend.get_requires_for_build_wheel() + + conflicting, missing = self.req.build_env.check_requirements(reqs) + if conflicting: + _raise_conflicts("the backend dependencies", conflicting) + self.req.build_env.install_requirements( + finder, missing, 'normal', + "Installing backend dependencies" + ) diff --git a/env/lib/python3.8/site-packages/pip/_internal/distributions/wheel.py b/env/lib/python3.8/site-packages/pip/_internal/distributions/wheel.py new file mode 100644 index 000000000..bf3482b15 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/distributions/wheel.py @@ -0,0 +1,36 @@ +from zipfile import ZipFile + +from pip._internal.distributions.base import AbstractDistribution +from pip._internal.utils.typing import MYPY_CHECK_RUNNING +from pip._internal.utils.wheel import pkg_resources_distribution_for_wheel + +if MYPY_CHECK_RUNNING: + from pip._vendor.pkg_resources import Distribution + from pip._internal.index.package_finder import PackageFinder + + +class WheelDistribution(AbstractDistribution): + """Represents a wheel distribution. + + This does not need any preparation as wheels can be directly unpacked. + """ + + def get_pkg_resources_distribution(self): + # type: () -> Distribution + """Loads the metadata from the wheel file into memory and returns a + Distribution that uses it, not relying on the wheel file or + requirement. + """ + # Set as part of preparation during download. + assert self.req.local_file_path + # Wheels are never unnamed. + assert self.req.name + + with ZipFile(self.req.local_file_path, allowZip64=True) as z: + return pkg_resources_distribution_for_wheel( + z, self.req.name, self.req.local_file_path + ) + + def prepare_distribution_metadata(self, finder, build_isolation): + # type: (PackageFinder, bool) -> None + pass diff --git a/env/lib/python3.8/site-packages/pip/_internal/exceptions.py b/env/lib/python3.8/site-packages/pip/_internal/exceptions.py new file mode 100644 index 000000000..dddec789e --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/exceptions.py @@ -0,0 +1,308 @@ +"""Exceptions used throughout package""" + +# The following comment should be removed at some point in the future. +# mypy: disallow-untyped-defs=False + +from __future__ import absolute_import + +from itertools import chain, groupby, repeat + +from pip._vendor.six import iteritems + +from pip._internal.utils.typing import MYPY_CHECK_RUNNING + +if MYPY_CHECK_RUNNING: + from typing import Optional + from pip._vendor.pkg_resources import Distribution + from pip._internal.req.req_install import InstallRequirement + + +class PipError(Exception): + """Base pip exception""" + + +class ConfigurationError(PipError): + """General exception in configuration""" + + +class InstallationError(PipError): + """General exception during installation""" + + +class UninstallationError(PipError): + """General exception during uninstallation""" + + +class NoneMetadataError(PipError): + """ + Raised when accessing "METADATA" or "PKG-INFO" metadata for a + pip._vendor.pkg_resources.Distribution object and + `dist.has_metadata('METADATA')` returns True but + `dist.get_metadata('METADATA')` returns None (and similarly for + "PKG-INFO"). + """ + + def __init__(self, dist, metadata_name): + # type: (Distribution, str) -> None + """ + :param dist: A Distribution object. + :param metadata_name: The name of the metadata being accessed + (can be "METADATA" or "PKG-INFO"). + """ + self.dist = dist + self.metadata_name = metadata_name + + def __str__(self): + # type: () -> str + # Use `dist` in the error message because its stringification + # includes more information, like the version and location. + return ( + 'None {} metadata found for distribution: {}'.format( + self.metadata_name, self.dist, + ) + ) + + +class DistributionNotFound(InstallationError): + """Raised when a distribution cannot be found to satisfy a requirement""" + + +class RequirementsFileParseError(InstallationError): + """Raised when a general error occurs parsing a requirements file line.""" + + +class BestVersionAlreadyInstalled(PipError): + """Raised when the most up-to-date version of a package is already + installed.""" + + +class BadCommand(PipError): + """Raised when virtualenv or a command is not found""" + + +class CommandError(PipError): + """Raised when there is an error in command-line arguments""" + + +class PreviousBuildDirError(PipError): + """Raised when there's a previous conflicting build directory""" + + +class InvalidWheelFilename(InstallationError): + """Invalid wheel filename.""" + + +class UnsupportedWheel(InstallationError): + """Unsupported wheel.""" + + +class HashErrors(InstallationError): + """Multiple HashError instances rolled into one for reporting""" + + def __init__(self): + self.errors = [] + + def append(self, error): + self.errors.append(error) + + def __str__(self): + lines = [] + self.errors.sort(key=lambda e: e.order) + for cls, errors_of_cls in groupby(self.errors, lambda e: e.__class__): + lines.append(cls.head) + lines.extend(e.body() for e in errors_of_cls) + if lines: + return '\n'.join(lines) + + def __nonzero__(self): + return bool(self.errors) + + def __bool__(self): + return self.__nonzero__() + + +class HashError(InstallationError): + """ + A failure to verify a package against known-good hashes + + :cvar order: An int sorting hash exception classes by difficulty of + recovery (lower being harder), so the user doesn't bother fretting + about unpinned packages when he has deeper issues, like VCS + dependencies, to deal with. Also keeps error reports in a + deterministic order. + :cvar head: A section heading for display above potentially many + exceptions of this kind + :ivar req: The InstallRequirement that triggered this error. This is + pasted on after the exception is instantiated, because it's not + typically available earlier. + + """ + req = None # type: Optional[InstallRequirement] + head = '' + + def body(self): + """Return a summary of me for display under the heading. + + This default implementation simply prints a description of the + triggering requirement. + + :param req: The InstallRequirement that provoked this error, with + populate_link() having already been called + + """ + return ' %s' % self._requirement_name() + + def __str__(self): + return '%s\n%s' % (self.head, self.body()) + + def _requirement_name(self): + """Return a description of the requirement that triggered me. + + This default implementation returns long description of the req, with + line numbers + + """ + return str(self.req) if self.req else 'unknown package' + + +class VcsHashUnsupported(HashError): + """A hash was provided for a version-control-system-based requirement, but + we don't have a method for hashing those.""" + + order = 0 + head = ("Can't verify hashes for these requirements because we don't " + "have a way to hash version control repositories:") + + +class DirectoryUrlHashUnsupported(HashError): + """A hash was provided for a version-control-system-based requirement, but + we don't have a method for hashing those.""" + + order = 1 + head = ("Can't verify hashes for these file:// requirements because they " + "point to directories:") + + +class HashMissing(HashError): + """A hash was needed for a requirement but is absent.""" + + order = 2 + head = ('Hashes are required in --require-hashes mode, but they are ' + 'missing from some requirements. Here is a list of those ' + 'requirements along with the hashes their downloaded archives ' + 'actually had. Add lines like these to your requirements files to ' + 'prevent tampering. (If you did not enable --require-hashes ' + 'manually, note that it turns on automatically when any package ' + 'has a hash.)') + + def __init__(self, gotten_hash): + """ + :param gotten_hash: The hash of the (possibly malicious) archive we + just downloaded + """ + self.gotten_hash = gotten_hash + + def body(self): + # Dodge circular import. + from pip._internal.utils.hashes import FAVORITE_HASH + + package = None + if self.req: + # In the case of URL-based requirements, display the original URL + # seen in the requirements file rather than the package name, + # so the output can be directly copied into the requirements file. + package = (self.req.original_link if self.req.original_link + # In case someone feeds something downright stupid + # to InstallRequirement's constructor. + else getattr(self.req, 'req', None)) + return ' %s --hash=%s:%s' % (package or 'unknown package', + FAVORITE_HASH, + self.gotten_hash) + + +class HashUnpinned(HashError): + """A requirement had a hash specified but was not pinned to a specific + version.""" + + order = 3 + head = ('In --require-hashes mode, all requirements must have their ' + 'versions pinned with ==. These do not:') + + +class HashMismatch(HashError): + """ + Distribution file hash values don't match. + + :ivar package_name: The name of the package that triggered the hash + mismatch. Feel free to write to this after the exception is raise to + improve its error message. + + """ + order = 4 + head = ('THESE PACKAGES DO NOT MATCH THE HASHES FROM THE REQUIREMENTS ' + 'FILE. If you have updated the package versions, please update ' + 'the hashes. Otherwise, examine the package contents carefully; ' + 'someone may have tampered with them.') + + def __init__(self, allowed, gots): + """ + :param allowed: A dict of algorithm names pointing to lists of allowed + hex digests + :param gots: A dict of algorithm names pointing to hashes we + actually got from the files under suspicion + """ + self.allowed = allowed + self.gots = gots + + def body(self): + return ' %s:\n%s' % (self._requirement_name(), + self._hash_comparison()) + + def _hash_comparison(self): + """ + Return a comparison of actual and expected hash values. + + Example:: + + Expected sha256 abcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcde + or 123451234512345123451234512345123451234512345 + Got bcdefbcdefbcdefbcdefbcdefbcdefbcdefbcdefbcdef + + """ + def hash_then_or(hash_name): + # For now, all the decent hashes have 6-char names, so we can get + # away with hard-coding space literals. + return chain([hash_name], repeat(' or')) + + lines = [] + for hash_name, expecteds in iteritems(self.allowed): + prefix = hash_then_or(hash_name) + lines.extend((' Expected %s %s' % (next(prefix), e)) + for e in expecteds) + lines.append(' Got %s\n' % + self.gots[hash_name].hexdigest()) + return '\n'.join(lines) + + +class UnsupportedPythonVersion(InstallationError): + """Unsupported python version according to Requires-Python package + metadata.""" + + +class ConfigurationFileCouldNotBeLoaded(ConfigurationError): + """When there are errors while loading a configuration file + """ + + def __init__(self, reason="could not be loaded", fname=None, error=None): + super(ConfigurationFileCouldNotBeLoaded, self).__init__(error) + self.reason = reason + self.fname = fname + self.error = error + + def __str__(self): + if self.fname is not None: + message_part = " in {}.".format(self.fname) + else: + assert self.error is not None + message_part = ".\n{}\n".format(self.error.message) + return "Configuration file {}{}".format(self.reason, message_part) diff --git a/env/lib/python3.8/site-packages/pip/_internal/index/__init__.py b/env/lib/python3.8/site-packages/pip/_internal/index/__init__.py new file mode 100644 index 000000000..7a17b7b3b --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/index/__init__.py @@ -0,0 +1,2 @@ +"""Index interaction code +""" diff --git a/env/lib/python3.8/site-packages/pip/_internal/index/collector.py b/env/lib/python3.8/site-packages/pip/_internal/index/collector.py new file mode 100644 index 000000000..833079317 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/index/collector.py @@ -0,0 +1,544 @@ +""" +The main purpose of this module is to expose LinkCollector.collect_links(). +""" + +import cgi +import itertools +import logging +import mimetypes +import os +from collections import OrderedDict + +from pip._vendor import html5lib, requests +from pip._vendor.distlib.compat import unescape +from pip._vendor.requests.exceptions import HTTPError, RetryError, SSLError +from pip._vendor.six.moves.urllib import parse as urllib_parse +from pip._vendor.six.moves.urllib import request as urllib_request + +from pip._internal.models.link import Link +from pip._internal.utils.filetypes import ARCHIVE_EXTENSIONS +from pip._internal.utils.misc import redact_auth_from_url +from pip._internal.utils.typing import MYPY_CHECK_RUNNING +from pip._internal.utils.urls import path_to_url, url_to_path +from pip._internal.vcs import is_url, vcs + +if MYPY_CHECK_RUNNING: + from typing import ( + Callable, Iterable, List, MutableMapping, Optional, Sequence, Tuple, + Union, + ) + import xml.etree.ElementTree + + from pip._vendor.requests import Response + + from pip._internal.models.search_scope import SearchScope + from pip._internal.network.session import PipSession + + HTMLElement = xml.etree.ElementTree.Element + ResponseHeaders = MutableMapping[str, str] + + +logger = logging.getLogger(__name__) + + +def _match_vcs_scheme(url): + # type: (str) -> Optional[str] + """Look for VCS schemes in the URL. + + Returns the matched VCS scheme, or None if there's no match. + """ + for scheme in vcs.schemes: + if url.lower().startswith(scheme) and url[len(scheme)] in '+:': + return scheme + return None + + +def _is_url_like_archive(url): + # type: (str) -> bool + """Return whether the URL looks like an archive. + """ + filename = Link(url).filename + for bad_ext in ARCHIVE_EXTENSIONS: + if filename.endswith(bad_ext): + return True + return False + + +class _NotHTML(Exception): + def __init__(self, content_type, request_desc): + # type: (str, str) -> None + super(_NotHTML, self).__init__(content_type, request_desc) + self.content_type = content_type + self.request_desc = request_desc + + +def _ensure_html_header(response): + # type: (Response) -> None + """Check the Content-Type header to ensure the response contains HTML. + + Raises `_NotHTML` if the content type is not text/html. + """ + content_type = response.headers.get("Content-Type", "") + if not content_type.lower().startswith("text/html"): + raise _NotHTML(content_type, response.request.method) + + +class _NotHTTP(Exception): + pass + + +def _ensure_html_response(url, session): + # type: (str, PipSession) -> None + """Send a HEAD request to the URL, and ensure the response contains HTML. + + Raises `_NotHTTP` if the URL is not available for a HEAD request, or + `_NotHTML` if the content type is not text/html. + """ + scheme, netloc, path, query, fragment = urllib_parse.urlsplit(url) + if scheme not in {'http', 'https'}: + raise _NotHTTP() + + resp = session.head(url, allow_redirects=True) + resp.raise_for_status() + + _ensure_html_header(resp) + + +def _get_html_response(url, session): + # type: (str, PipSession) -> Response + """Access an HTML page with GET, and return the response. + + This consists of three parts: + + 1. If the URL looks suspiciously like an archive, send a HEAD first to + check the Content-Type is HTML, to avoid downloading a large file. + Raise `_NotHTTP` if the content type cannot be determined, or + `_NotHTML` if it is not HTML. + 2. Actually perform the request. Raise HTTP exceptions on network failures. + 3. Check the Content-Type header to make sure we got HTML, and raise + `_NotHTML` otherwise. + """ + if _is_url_like_archive(url): + _ensure_html_response(url, session=session) + + logger.debug('Getting page %s', redact_auth_from_url(url)) + + resp = session.get( + url, + headers={ + "Accept": "text/html", + # We don't want to blindly returned cached data for + # /simple/, because authors generally expecting that + # twine upload && pip install will function, but if + # they've done a pip install in the last ~10 minutes + # it won't. Thus by setting this to zero we will not + # blindly use any cached data, however the benefit of + # using max-age=0 instead of no-cache, is that we will + # still support conditional requests, so we will still + # minimize traffic sent in cases where the page hasn't + # changed at all, we will just always incur the round + # trip for the conditional GET now instead of only + # once per 10 minutes. + # For more information, please see pypa/pip#5670. + "Cache-Control": "max-age=0", + }, + ) + resp.raise_for_status() + + # The check for archives above only works if the url ends with + # something that looks like an archive. However that is not a + # requirement of an url. Unless we issue a HEAD request on every + # url we cannot know ahead of time for sure if something is HTML + # or not. However we can check after we've downloaded it. + _ensure_html_header(resp) + + return resp + + +def _get_encoding_from_headers(headers): + # type: (ResponseHeaders) -> Optional[str] + """Determine if we have any encoding information in our headers. + """ + if headers and "Content-Type" in headers: + content_type, params = cgi.parse_header(headers["Content-Type"]) + if "charset" in params: + return params['charset'] + return None + + +def _determine_base_url(document, page_url): + # type: (HTMLElement, str) -> str + """Determine the HTML document's base URL. + + This looks for a ```` tag in the HTML document. If present, its href + attribute denotes the base URL of anchor tags in the document. If there is + no such tag (or if it does not have a valid href attribute), the HTML + file's URL is used as the base URL. + + :param document: An HTML document representation. The current + implementation expects the result of ``html5lib.parse()``. + :param page_url: The URL of the HTML document. + """ + for base in document.findall(".//base"): + href = base.get("href") + if href is not None: + return href + return page_url + + +def _clean_link(url): + # type: (str) -> str + """Makes sure a link is fully encoded. That is, if a ' ' shows up in + the link, it will be rewritten to %20 (while not over-quoting + % or other characters).""" + # Split the URL into parts according to the general structure + # `scheme://netloc/path;parameters?query#fragment`. Note that the + # `netloc` can be empty and the URI will then refer to a local + # filesystem path. + result = urllib_parse.urlparse(url) + # In both cases below we unquote prior to quoting to make sure + # nothing is double quoted. + if result.netloc == "": + # On Windows the path part might contain a drive letter which + # should not be quoted. On Linux where drive letters do not + # exist, the colon should be quoted. We rely on urllib.request + # to do the right thing here. + path = urllib_request.pathname2url( + urllib_request.url2pathname(result.path)) + else: + # In addition to the `/` character we protect `@` so that + # revision strings in VCS URLs are properly parsed. + path = urllib_parse.quote(urllib_parse.unquote(result.path), safe="/@") + return urllib_parse.urlunparse(result._replace(path=path)) + + +def _create_link_from_element( + anchor, # type: HTMLElement + page_url, # type: str + base_url, # type: str +): + # type: (...) -> Optional[Link] + """ + Convert an anchor element in a simple repository page to a Link. + """ + href = anchor.get("href") + if not href: + return None + + url = _clean_link(urllib_parse.urljoin(base_url, href)) + pyrequire = anchor.get('data-requires-python') + pyrequire = unescape(pyrequire) if pyrequire else None + + yanked_reason = anchor.get('data-yanked') + if yanked_reason: + # This is a unicode string in Python 2 (and 3). + yanked_reason = unescape(yanked_reason) + + link = Link( + url, + comes_from=page_url, + requires_python=pyrequire, + yanked_reason=yanked_reason, + ) + + return link + + +def parse_links(page): + # type: (HTMLPage) -> Iterable[Link] + """ + Parse an HTML document, and yield its anchor elements as Link objects. + """ + document = html5lib.parse( + page.content, + transport_encoding=page.encoding, + namespaceHTMLElements=False, + ) + + url = page.url + base_url = _determine_base_url(document, url) + for anchor in document.findall(".//a"): + link = _create_link_from_element( + anchor, + page_url=url, + base_url=base_url, + ) + if link is None: + continue + yield link + + +class HTMLPage(object): + """Represents one page, along with its URL""" + + def __init__( + self, + content, # type: bytes + encoding, # type: Optional[str] + url, # type: str + ): + # type: (...) -> None + """ + :param encoding: the encoding to decode the given content. + :param url: the URL from which the HTML was downloaded. + """ + self.content = content + self.encoding = encoding + self.url = url + + def __str__(self): + # type: () -> str + return redact_auth_from_url(self.url) + + +def _handle_get_page_fail( + link, # type: Link + reason, # type: Union[str, Exception] + meth=None # type: Optional[Callable[..., None]] +): + # type: (...) -> None + if meth is None: + meth = logger.debug + meth("Could not fetch URL %s: %s - skipping", link, reason) + + +def _make_html_page(response): + # type: (Response) -> HTMLPage + encoding = _get_encoding_from_headers(response.headers) + return HTMLPage(response.content, encoding=encoding, url=response.url) + + +def _get_html_page(link, session=None): + # type: (Link, Optional[PipSession]) -> Optional[HTMLPage] + if session is None: + raise TypeError( + "_get_html_page() missing 1 required keyword argument: 'session'" + ) + + url = link.url.split('#', 1)[0] + + # Check for VCS schemes that do not support lookup as web pages. + vcs_scheme = _match_vcs_scheme(url) + if vcs_scheme: + logger.debug('Cannot look at %s URL %s', vcs_scheme, link) + return None + + # Tack index.html onto file:// URLs that point to directories + scheme, _, path, _, _, _ = urllib_parse.urlparse(url) + if (scheme == 'file' and os.path.isdir(urllib_request.url2pathname(path))): + # add trailing slash if not present so urljoin doesn't trim + # final segment + if not url.endswith('/'): + url += '/' + url = urllib_parse.urljoin(url, 'index.html') + logger.debug(' file: URL is directory, getting %s', url) + + try: + resp = _get_html_response(url, session=session) + except _NotHTTP: + logger.debug( + 'Skipping page %s because it looks like an archive, and cannot ' + 'be checked by HEAD.', link, + ) + except _NotHTML as exc: + logger.debug( + 'Skipping page %s because the %s request got Content-Type: %s', + link, exc.request_desc, exc.content_type, + ) + except HTTPError as exc: + _handle_get_page_fail(link, exc) + except RetryError as exc: + _handle_get_page_fail(link, exc) + except SSLError as exc: + reason = "There was a problem confirming the ssl certificate: " + reason += str(exc) + _handle_get_page_fail(link, reason, meth=logger.info) + except requests.ConnectionError as exc: + _handle_get_page_fail(link, "connection error: %s" % exc) + except requests.Timeout: + _handle_get_page_fail(link, "timed out") + else: + return _make_html_page(resp) + return None + + +def _remove_duplicate_links(links): + # type: (Iterable[Link]) -> List[Link] + """ + Return a list of links, with duplicates removed and ordering preserved. + """ + # We preserve the ordering when removing duplicates because we can. + return list(OrderedDict.fromkeys(links)) + + +def group_locations(locations, expand_dir=False): + # type: (Sequence[str], bool) -> Tuple[List[str], List[str]] + """ + Divide a list of locations into two groups: "files" (archives) and "urls." + + :return: A pair of lists (files, urls). + """ + files = [] + urls = [] + + # puts the url for the given file path into the appropriate list + def sort_path(path): + # type: (str) -> None + url = path_to_url(path) + if mimetypes.guess_type(url, strict=False)[0] == 'text/html': + urls.append(url) + else: + files.append(url) + + for url in locations: + + is_local_path = os.path.exists(url) + is_file_url = url.startswith('file:') + + if is_local_path or is_file_url: + if is_local_path: + path = url + else: + path = url_to_path(url) + if os.path.isdir(path): + if expand_dir: + path = os.path.realpath(path) + for item in os.listdir(path): + sort_path(os.path.join(path, item)) + elif is_file_url: + urls.append(url) + else: + logger.warning( + "Path '{0}' is ignored: " + "it is a directory.".format(path), + ) + elif os.path.isfile(path): + sort_path(path) + else: + logger.warning( + "Url '%s' is ignored: it is neither a file " + "nor a directory.", url, + ) + elif is_url(url): + # Only add url with clear scheme + urls.append(url) + else: + logger.warning( + "Url '%s' is ignored. It is either a non-existing " + "path or lacks a specific scheme.", url, + ) + + return files, urls + + +class CollectedLinks(object): + + """ + Encapsulates the return value of a call to LinkCollector.collect_links(). + + The return value includes both URLs to project pages containing package + links, as well as individual package Link objects collected from other + sources. + + This info is stored separately as: + + (1) links from the configured file locations, + (2) links from the configured find_links, and + (3) urls to HTML project pages, as described by the PEP 503 simple + repository API. + """ + + def __init__( + self, + files, # type: List[Link] + find_links, # type: List[Link] + project_urls, # type: List[Link] + ): + # type: (...) -> None + """ + :param files: Links from file locations. + :param find_links: Links from find_links. + :param project_urls: URLs to HTML project pages, as described by + the PEP 503 simple repository API. + """ + self.files = files + self.find_links = find_links + self.project_urls = project_urls + + +class LinkCollector(object): + + """ + Responsible for collecting Link objects from all configured locations, + making network requests as needed. + + The class's main method is its collect_links() method. + """ + + def __init__( + self, + session, # type: PipSession + search_scope, # type: SearchScope + ): + # type: (...) -> None + self.search_scope = search_scope + self.session = session + + @property + def find_links(self): + # type: () -> List[str] + return self.search_scope.find_links + + def fetch_page(self, location): + # type: (Link) -> Optional[HTMLPage] + """ + Fetch an HTML page containing package links. + """ + return _get_html_page(location, session=self.session) + + def collect_links(self, project_name): + # type: (str) -> CollectedLinks + """Find all available links for the given project name. + + :return: All the Link objects (unfiltered), as a CollectedLinks object. + """ + search_scope = self.search_scope + index_locations = search_scope.get_index_urls_locations(project_name) + index_file_loc, index_url_loc = group_locations(index_locations) + fl_file_loc, fl_url_loc = group_locations( + self.find_links, expand_dir=True, + ) + + file_links = [ + Link(url) for url in itertools.chain(index_file_loc, fl_file_loc) + ] + + # We trust every directly linked archive in find_links + find_link_links = [Link(url, '-f') for url in self.find_links] + + # We trust every url that the user has given us whether it was given + # via --index-url or --find-links. + # We want to filter out anything that does not have a secure origin. + url_locations = [ + link for link in itertools.chain( + (Link(url) for url in index_url_loc), + (Link(url) for url in fl_url_loc), + ) + if self.session.is_secure_origin(link) + ] + + url_locations = _remove_duplicate_links(url_locations) + lines = [ + '{} location(s) to search for versions of {}:'.format( + len(url_locations), project_name, + ), + ] + for link in url_locations: + lines.append('* {}'.format(link)) + logger.debug('\n'.join(lines)) + + return CollectedLinks( + files=file_links, + find_links=find_link_links, + project_urls=url_locations, + ) diff --git a/env/lib/python3.8/site-packages/pip/_internal/index/package_finder.py b/env/lib/python3.8/site-packages/pip/_internal/index/package_finder.py new file mode 100644 index 000000000..a74d78db5 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/index/package_finder.py @@ -0,0 +1,1013 @@ +"""Routines related to PyPI, indexes""" + +# The following comment should be removed at some point in the future. +# mypy: strict-optional=False + +from __future__ import absolute_import + +import logging +import re + +from pip._vendor.packaging import specifiers +from pip._vendor.packaging.utils import canonicalize_name +from pip._vendor.packaging.version import parse as parse_version + +from pip._internal.exceptions import ( + BestVersionAlreadyInstalled, + DistributionNotFound, + InvalidWheelFilename, + UnsupportedWheel, +) +from pip._internal.index.collector import parse_links +from pip._internal.models.candidate import InstallationCandidate +from pip._internal.models.format_control import FormatControl +from pip._internal.models.link import Link +from pip._internal.models.selection_prefs import SelectionPreferences +from pip._internal.models.target_python import TargetPython +from pip._internal.models.wheel import Wheel +from pip._internal.utils.filetypes import WHEEL_EXTENSION +from pip._internal.utils.logging import indent_log +from pip._internal.utils.misc import build_netloc +from pip._internal.utils.packaging import check_requires_python +from pip._internal.utils.typing import MYPY_CHECK_RUNNING +from pip._internal.utils.unpacking import SUPPORTED_EXTENSIONS +from pip._internal.utils.urls import url_to_path + +if MYPY_CHECK_RUNNING: + from typing import ( + FrozenSet, Iterable, List, Optional, Set, Text, Tuple, Union, + ) + + from pip._vendor.packaging.tags import Tag + from pip._vendor.packaging.version import _BaseVersion + + from pip._internal.index.collector import LinkCollector + from pip._internal.models.search_scope import SearchScope + from pip._internal.req import InstallRequirement + from pip._internal.utils.hashes import Hashes + + BuildTag = Union[Tuple[()], Tuple[int, str]] + CandidateSortingKey = ( + Tuple[int, int, int, _BaseVersion, BuildTag, Optional[int]] + ) + + +__all__ = ['FormatControl', 'BestCandidateResult', 'PackageFinder'] + + +logger = logging.getLogger(__name__) + + +def _check_link_requires_python( + link, # type: Link + version_info, # type: Tuple[int, int, int] + ignore_requires_python=False, # type: bool +): + # type: (...) -> bool + """ + Return whether the given Python version is compatible with a link's + "Requires-Python" value. + + :param version_info: A 3-tuple of ints representing the Python + major-minor-micro version to check. + :param ignore_requires_python: Whether to ignore the "Requires-Python" + value if the given Python version isn't compatible. + """ + try: + is_compatible = check_requires_python( + link.requires_python, version_info=version_info, + ) + except specifiers.InvalidSpecifier: + logger.debug( + "Ignoring invalid Requires-Python (%r) for link: %s", + link.requires_python, link, + ) + else: + if not is_compatible: + version = '.'.join(map(str, version_info)) + if not ignore_requires_python: + logger.debug( + 'Link requires a different Python (%s not in: %r): %s', + version, link.requires_python, link, + ) + return False + + logger.debug( + 'Ignoring failed Requires-Python check (%s not in: %r) ' + 'for link: %s', + version, link.requires_python, link, + ) + + return True + + +class LinkEvaluator(object): + + """ + Responsible for evaluating links for a particular project. + """ + + _py_version_re = re.compile(r'-py([123]\.?[0-9]?)$') + + # Don't include an allow_yanked default value to make sure each call + # site considers whether yanked releases are allowed. This also causes + # that decision to be made explicit in the calling code, which helps + # people when reading the code. + def __init__( + self, + project_name, # type: str + canonical_name, # type: str + formats, # type: FrozenSet[str] + target_python, # type: TargetPython + allow_yanked, # type: bool + ignore_requires_python=None, # type: Optional[bool] + ): + # type: (...) -> None + """ + :param project_name: The user supplied package name. + :param canonical_name: The canonical package name. + :param formats: The formats allowed for this package. Should be a set + with 'binary' or 'source' or both in it. + :param target_python: The target Python interpreter to use when + evaluating link compatibility. This is used, for example, to + check wheel compatibility, as well as when checking the Python + version, e.g. the Python version embedded in a link filename + (or egg fragment) and against an HTML link's optional PEP 503 + "data-requires-python" attribute. + :param allow_yanked: Whether files marked as yanked (in the sense + of PEP 592) are permitted to be candidates for install. + :param ignore_requires_python: Whether to ignore incompatible + PEP 503 "data-requires-python" values in HTML links. Defaults + to False. + """ + if ignore_requires_python is None: + ignore_requires_python = False + + self._allow_yanked = allow_yanked + self._canonical_name = canonical_name + self._ignore_requires_python = ignore_requires_python + self._formats = formats + self._target_python = target_python + + self.project_name = project_name + + def evaluate_link(self, link): + # type: (Link) -> Tuple[bool, Optional[Text]] + """ + Determine whether a link is a candidate for installation. + + :return: A tuple (is_candidate, result), where `result` is (1) a + version string if `is_candidate` is True, and (2) if + `is_candidate` is False, an optional string to log the reason + the link fails to qualify. + """ + version = None + if link.is_yanked and not self._allow_yanked: + reason = link.yanked_reason or '' + # Mark this as a unicode string to prevent "UnicodeEncodeError: + # 'ascii' codec can't encode character" in Python 2 when + # the reason contains non-ascii characters. + return (False, u'yanked for reason: {}'.format(reason)) + + if link.egg_fragment: + egg_info = link.egg_fragment + ext = link.ext + else: + egg_info, ext = link.splitext() + if not ext: + return (False, 'not a file') + if ext not in SUPPORTED_EXTENSIONS: + return (False, 'unsupported archive format: %s' % ext) + if "binary" not in self._formats and ext == WHEEL_EXTENSION: + reason = 'No binaries permitted for %s' % self.project_name + return (False, reason) + if "macosx10" in link.path and ext == '.zip': + return (False, 'macosx10 one') + if ext == WHEEL_EXTENSION: + try: + wheel = Wheel(link.filename) + except InvalidWheelFilename: + return (False, 'invalid wheel filename') + if canonicalize_name(wheel.name) != self._canonical_name: + reason = 'wrong project name (not %s)' % self.project_name + return (False, reason) + + supported_tags = self._target_python.get_tags() + if not wheel.supported(supported_tags): + # Include the wheel's tags in the reason string to + # simplify troubleshooting compatibility issues. + file_tags = wheel.get_formatted_file_tags() + reason = ( + "none of the wheel's tags match: {}".format( + ', '.join(file_tags) + ) + ) + return (False, reason) + + version = wheel.version + + # This should be up by the self.ok_binary check, but see issue 2700. + if "source" not in self._formats and ext != WHEEL_EXTENSION: + return (False, 'No sources permitted for %s' % self.project_name) + + if not version: + version = _extract_version_from_fragment( + egg_info, self._canonical_name, + ) + if not version: + return ( + False, 'Missing project version for %s' % self.project_name, + ) + + match = self._py_version_re.search(version) + if match: + version = version[:match.start()] + py_version = match.group(1) + if py_version != self._target_python.py_version: + return (False, 'Python version is incorrect') + + supports_python = _check_link_requires_python( + link, version_info=self._target_python.py_version_info, + ignore_requires_python=self._ignore_requires_python, + ) + if not supports_python: + # Return None for the reason text to suppress calling + # _log_skipped_link(). + return (False, None) + + logger.debug('Found link %s, version: %s', link, version) + + return (True, version) + + +def filter_unallowed_hashes( + candidates, # type: List[InstallationCandidate] + hashes, # type: Hashes + project_name, # type: str +): + # type: (...) -> List[InstallationCandidate] + """ + Filter out candidates whose hashes aren't allowed, and return a new + list of candidates. + + If at least one candidate has an allowed hash, then all candidates with + either an allowed hash or no hash specified are returned. Otherwise, + the given candidates are returned. + + Including the candidates with no hash specified when there is a match + allows a warning to be logged if there is a more preferred candidate + with no hash specified. Returning all candidates in the case of no + matches lets pip report the hash of the candidate that would otherwise + have been installed (e.g. permitting the user to more easily update + their requirements file with the desired hash). + """ + if not hashes: + logger.debug( + 'Given no hashes to check %s links for project %r: ' + 'discarding no candidates', + len(candidates), + project_name, + ) + # Make sure we're not returning back the given value. + return list(candidates) + + matches_or_no_digest = [] + # Collect the non-matches for logging purposes. + non_matches = [] + match_count = 0 + for candidate in candidates: + link = candidate.link + if not link.has_hash: + pass + elif link.is_hash_allowed(hashes=hashes): + match_count += 1 + else: + non_matches.append(candidate) + continue + + matches_or_no_digest.append(candidate) + + if match_count: + filtered = matches_or_no_digest + else: + # Make sure we're not returning back the given value. + filtered = list(candidates) + + if len(filtered) == len(candidates): + discard_message = 'discarding no candidates' + else: + discard_message = 'discarding {} non-matches:\n {}'.format( + len(non_matches), + '\n '.join(str(candidate.link) for candidate in non_matches) + ) + + logger.debug( + 'Checked %s links for project %r against %s hashes ' + '(%s matches, %s no digest): %s', + len(candidates), + project_name, + hashes.digest_count, + match_count, + len(matches_or_no_digest) - match_count, + discard_message + ) + + return filtered + + +class CandidatePreferences(object): + + """ + Encapsulates some of the preferences for filtering and sorting + InstallationCandidate objects. + """ + + def __init__( + self, + prefer_binary=False, # type: bool + allow_all_prereleases=False, # type: bool + ): + # type: (...) -> None + """ + :param allow_all_prereleases: Whether to allow all pre-releases. + """ + self.allow_all_prereleases = allow_all_prereleases + self.prefer_binary = prefer_binary + + +class BestCandidateResult(object): + """A collection of candidates, returned by `PackageFinder.find_best_candidate`. + + This class is only intended to be instantiated by CandidateEvaluator's + `compute_best_candidate()` method. + """ + + def __init__( + self, + candidates, # type: List[InstallationCandidate] + applicable_candidates, # type: List[InstallationCandidate] + best_candidate, # type: Optional[InstallationCandidate] + ): + # type: (...) -> None + """ + :param candidates: A sequence of all available candidates found. + :param applicable_candidates: The applicable candidates. + :param best_candidate: The most preferred candidate found, or None + if no applicable candidates were found. + """ + assert set(applicable_candidates) <= set(candidates) + + if best_candidate is None: + assert not applicable_candidates + else: + assert best_candidate in applicable_candidates + + self._applicable_candidates = applicable_candidates + self._candidates = candidates + + self.best_candidate = best_candidate + + def iter_all(self): + # type: () -> Iterable[InstallationCandidate] + """Iterate through all candidates. + """ + return iter(self._candidates) + + def iter_applicable(self): + # type: () -> Iterable[InstallationCandidate] + """Iterate through the applicable candidates. + """ + return iter(self._applicable_candidates) + + +class CandidateEvaluator(object): + + """ + Responsible for filtering and sorting candidates for installation based + on what tags are valid. + """ + + @classmethod + def create( + cls, + project_name, # type: str + target_python=None, # type: Optional[TargetPython] + prefer_binary=False, # type: bool + allow_all_prereleases=False, # type: bool + specifier=None, # type: Optional[specifiers.BaseSpecifier] + hashes=None, # type: Optional[Hashes] + ): + # type: (...) -> CandidateEvaluator + """Create a CandidateEvaluator object. + + :param target_python: The target Python interpreter to use when + checking compatibility. If None (the default), a TargetPython + object will be constructed from the running Python. + :param specifier: An optional object implementing `filter` + (e.g. `packaging.specifiers.SpecifierSet`) to filter applicable + versions. + :param hashes: An optional collection of allowed hashes. + """ + if target_python is None: + target_python = TargetPython() + if specifier is None: + specifier = specifiers.SpecifierSet() + + supported_tags = target_python.get_tags() + + return cls( + project_name=project_name, + supported_tags=supported_tags, + specifier=specifier, + prefer_binary=prefer_binary, + allow_all_prereleases=allow_all_prereleases, + hashes=hashes, + ) + + def __init__( + self, + project_name, # type: str + supported_tags, # type: List[Tag] + specifier, # type: specifiers.BaseSpecifier + prefer_binary=False, # type: bool + allow_all_prereleases=False, # type: bool + hashes=None, # type: Optional[Hashes] + ): + # type: (...) -> None + """ + :param supported_tags: The PEP 425 tags supported by the target + Python in order of preference (most preferred first). + """ + self._allow_all_prereleases = allow_all_prereleases + self._hashes = hashes + self._prefer_binary = prefer_binary + self._project_name = project_name + self._specifier = specifier + self._supported_tags = supported_tags + + def get_applicable_candidates( + self, + candidates, # type: List[InstallationCandidate] + ): + # type: (...) -> List[InstallationCandidate] + """ + Return the applicable candidates from a list of candidates. + """ + # Using None infers from the specifier instead. + allow_prereleases = self._allow_all_prereleases or None + specifier = self._specifier + versions = { + str(v) for v in specifier.filter( + # We turn the version object into a str here because otherwise + # when we're debundled but setuptools isn't, Python will see + # packaging.version.Version and + # pkg_resources._vendor.packaging.version.Version as different + # types. This way we'll use a str as a common data interchange + # format. If we stop using the pkg_resources provided specifier + # and start using our own, we can drop the cast to str(). + (str(c.version) for c in candidates), + prereleases=allow_prereleases, + ) + } + + # Again, converting version to str to deal with debundling. + applicable_candidates = [ + c for c in candidates if str(c.version) in versions + ] + + filtered_applicable_candidates = filter_unallowed_hashes( + candidates=applicable_candidates, + hashes=self._hashes, + project_name=self._project_name, + ) + + return sorted(filtered_applicable_candidates, key=self._sort_key) + + def _sort_key(self, candidate): + # type: (InstallationCandidate) -> CandidateSortingKey + """ + Function to pass as the `key` argument to a call to sorted() to sort + InstallationCandidates by preference. + + Returns a tuple such that tuples sorting as greater using Python's + default comparison operator are more preferred. + + The preference is as follows: + + First and foremost, candidates with allowed (matching) hashes are + always preferred over candidates without matching hashes. This is + because e.g. if the only candidate with an allowed hash is yanked, + we still want to use that candidate. + + Second, excepting hash considerations, candidates that have been + yanked (in the sense of PEP 592) are always less preferred than + candidates that haven't been yanked. Then: + + If not finding wheels, they are sorted by version only. + If finding wheels, then the sort order is by version, then: + 1. existing installs + 2. wheels ordered via Wheel.support_index_min(self._supported_tags) + 3. source archives + If prefer_binary was set, then all wheels are sorted above sources. + + Note: it was considered to embed this logic into the Link + comparison operators, but then different sdist links + with the same version, would have to be considered equal + """ + valid_tags = self._supported_tags + support_num = len(valid_tags) + build_tag = () # type: BuildTag + binary_preference = 0 + link = candidate.link + if link.is_wheel: + # can raise InvalidWheelFilename + wheel = Wheel(link.filename) + if not wheel.supported(valid_tags): + raise UnsupportedWheel( + "%s is not a supported wheel for this platform. It " + "can't be sorted." % wheel.filename + ) + if self._prefer_binary: + binary_preference = 1 + pri = -(wheel.support_index_min(valid_tags)) + if wheel.build_tag is not None: + match = re.match(r'^(\d+)(.*)$', wheel.build_tag) + build_tag_groups = match.groups() + build_tag = (int(build_tag_groups[0]), build_tag_groups[1]) + else: # sdist + pri = -(support_num) + has_allowed_hash = int(link.is_hash_allowed(self._hashes)) + yank_value = -1 * int(link.is_yanked) # -1 for yanked. + return ( + has_allowed_hash, yank_value, binary_preference, candidate.version, + build_tag, pri, + ) + + def sort_best_candidate( + self, + candidates, # type: List[InstallationCandidate] + ): + # type: (...) -> Optional[InstallationCandidate] + """ + Return the best candidate per the instance's sort order, or None if + no candidate is acceptable. + """ + if not candidates: + return None + + best_candidate = max(candidates, key=self._sort_key) + + # Log a warning per PEP 592 if necessary before returning. + link = best_candidate.link + if link.is_yanked: + reason = link.yanked_reason or '' + msg = ( + # Mark this as a unicode string to prevent + # "UnicodeEncodeError: 'ascii' codec can't encode character" + # in Python 2 when the reason contains non-ascii characters. + u'The candidate selected for download or install is a ' + 'yanked version: {candidate}\n' + 'Reason for being yanked: {reason}' + ).format(candidate=best_candidate, reason=reason) + logger.warning(msg) + + return best_candidate + + def compute_best_candidate( + self, + candidates, # type: List[InstallationCandidate] + ): + # type: (...) -> BestCandidateResult + """ + Compute and return a `BestCandidateResult` instance. + """ + applicable_candidates = self.get_applicable_candidates(candidates) + + best_candidate = self.sort_best_candidate(applicable_candidates) + + return BestCandidateResult( + candidates, + applicable_candidates=applicable_candidates, + best_candidate=best_candidate, + ) + + +class PackageFinder(object): + """This finds packages. + + This is meant to match easy_install's technique for looking for + packages, by reading pages and looking for appropriate links. + """ + + def __init__( + self, + link_collector, # type: LinkCollector + target_python, # type: TargetPython + allow_yanked, # type: bool + format_control=None, # type: Optional[FormatControl] + candidate_prefs=None, # type: CandidatePreferences + ignore_requires_python=None, # type: Optional[bool] + ): + # type: (...) -> None + """ + This constructor is primarily meant to be used by the create() class + method and from tests. + + :param format_control: A FormatControl object, used to control + the selection of source packages / binary packages when consulting + the index and links. + :param candidate_prefs: Options to use when creating a + CandidateEvaluator object. + """ + if candidate_prefs is None: + candidate_prefs = CandidatePreferences() + + format_control = format_control or FormatControl(set(), set()) + + self._allow_yanked = allow_yanked + self._candidate_prefs = candidate_prefs + self._ignore_requires_python = ignore_requires_python + self._link_collector = link_collector + self._target_python = target_python + + self.format_control = format_control + + # These are boring links that have already been logged somehow. + self._logged_links = set() # type: Set[Link] + + # Don't include an allow_yanked default value to make sure each call + # site considers whether yanked releases are allowed. This also causes + # that decision to be made explicit in the calling code, which helps + # people when reading the code. + @classmethod + def create( + cls, + link_collector, # type: LinkCollector + selection_prefs, # type: SelectionPreferences + target_python=None, # type: Optional[TargetPython] + ): + # type: (...) -> PackageFinder + """Create a PackageFinder. + + :param selection_prefs: The candidate selection preferences, as a + SelectionPreferences object. + :param target_python: The target Python interpreter to use when + checking compatibility. If None (the default), a TargetPython + object will be constructed from the running Python. + """ + if target_python is None: + target_python = TargetPython() + + candidate_prefs = CandidatePreferences( + prefer_binary=selection_prefs.prefer_binary, + allow_all_prereleases=selection_prefs.allow_all_prereleases, + ) + + return cls( + candidate_prefs=candidate_prefs, + link_collector=link_collector, + target_python=target_python, + allow_yanked=selection_prefs.allow_yanked, + format_control=selection_prefs.format_control, + ignore_requires_python=selection_prefs.ignore_requires_python, + ) + + @property + def search_scope(self): + # type: () -> SearchScope + return self._link_collector.search_scope + + @search_scope.setter + def search_scope(self, search_scope): + # type: (SearchScope) -> None + self._link_collector.search_scope = search_scope + + @property + def find_links(self): + # type: () -> List[str] + return self._link_collector.find_links + + @property + def index_urls(self): + # type: () -> List[str] + return self.search_scope.index_urls + + @property + def trusted_hosts(self): + # type: () -> Iterable[str] + for host_port in self._link_collector.session.pip_trusted_origins: + yield build_netloc(*host_port) + + @property + def allow_all_prereleases(self): + # type: () -> bool + return self._candidate_prefs.allow_all_prereleases + + def set_allow_all_prereleases(self): + # type: () -> None + self._candidate_prefs.allow_all_prereleases = True + + def make_link_evaluator(self, project_name): + # type: (str) -> LinkEvaluator + canonical_name = canonicalize_name(project_name) + formats = self.format_control.get_allowed_formats(canonical_name) + + return LinkEvaluator( + project_name=project_name, + canonical_name=canonical_name, + formats=formats, + target_python=self._target_python, + allow_yanked=self._allow_yanked, + ignore_requires_python=self._ignore_requires_python, + ) + + def _sort_links(self, links): + # type: (Iterable[Link]) -> List[Link] + """ + Returns elements of links in order, non-egg links first, egg links + second, while eliminating duplicates + """ + eggs, no_eggs = [], [] + seen = set() # type: Set[Link] + for link in links: + if link not in seen: + seen.add(link) + if link.egg_fragment: + eggs.append(link) + else: + no_eggs.append(link) + return no_eggs + eggs + + def _log_skipped_link(self, link, reason): + # type: (Link, Text) -> None + if link not in self._logged_links: + # Mark this as a unicode string to prevent "UnicodeEncodeError: + # 'ascii' codec can't encode character" in Python 2 when + # the reason contains non-ascii characters. + # Also, put the link at the end so the reason is more visible + # and because the link string is usually very long. + logger.debug(u'Skipping link: %s: %s', reason, link) + self._logged_links.add(link) + + def get_install_candidate(self, link_evaluator, link): + # type: (LinkEvaluator, Link) -> Optional[InstallationCandidate] + """ + If the link is a candidate for install, convert it to an + InstallationCandidate and return it. Otherwise, return None. + """ + is_candidate, result = link_evaluator.evaluate_link(link) + if not is_candidate: + if result: + self._log_skipped_link(link, reason=result) + return None + + return InstallationCandidate( + name=link_evaluator.project_name, + link=link, + # Convert the Text result to str since InstallationCandidate + # accepts str. + version=str(result), + ) + + def evaluate_links(self, link_evaluator, links): + # type: (LinkEvaluator, Iterable[Link]) -> List[InstallationCandidate] + """ + Convert links that are candidates to InstallationCandidate objects. + """ + candidates = [] + for link in self._sort_links(links): + candidate = self.get_install_candidate(link_evaluator, link) + if candidate is not None: + candidates.append(candidate) + + return candidates + + def process_project_url(self, project_url, link_evaluator): + # type: (Link, LinkEvaluator) -> List[InstallationCandidate] + logger.debug( + 'Fetching project page and analyzing links: %s', project_url, + ) + html_page = self._link_collector.fetch_page(project_url) + if html_page is None: + return [] + + page_links = list(parse_links(html_page)) + + with indent_log(): + package_links = self.evaluate_links( + link_evaluator, + links=page_links, + ) + + return package_links + + def find_all_candidates(self, project_name): + # type: (str) -> List[InstallationCandidate] + """Find all available InstallationCandidate for project_name + + This checks index_urls and find_links. + All versions found are returned as an InstallationCandidate list. + + See LinkEvaluator.evaluate_link() for details on which files + are accepted. + """ + collected_links = self._link_collector.collect_links(project_name) + + link_evaluator = self.make_link_evaluator(project_name) + + find_links_versions = self.evaluate_links( + link_evaluator, + links=collected_links.find_links, + ) + + page_versions = [] + for project_url in collected_links.project_urls: + package_links = self.process_project_url( + project_url, link_evaluator=link_evaluator, + ) + page_versions.extend(package_links) + + file_versions = self.evaluate_links( + link_evaluator, + links=collected_links.files, + ) + if file_versions: + file_versions.sort(reverse=True) + logger.debug( + 'Local files found: %s', + ', '.join([ + url_to_path(candidate.link.url) + for candidate in file_versions + ]) + ) + + # This is an intentional priority ordering + return file_versions + find_links_versions + page_versions + + def make_candidate_evaluator( + self, + project_name, # type: str + specifier=None, # type: Optional[specifiers.BaseSpecifier] + hashes=None, # type: Optional[Hashes] + ): + # type: (...) -> CandidateEvaluator + """Create a CandidateEvaluator object to use. + """ + candidate_prefs = self._candidate_prefs + return CandidateEvaluator.create( + project_name=project_name, + target_python=self._target_python, + prefer_binary=candidate_prefs.prefer_binary, + allow_all_prereleases=candidate_prefs.allow_all_prereleases, + specifier=specifier, + hashes=hashes, + ) + + def find_best_candidate( + self, + project_name, # type: str + specifier=None, # type: Optional[specifiers.BaseSpecifier] + hashes=None, # type: Optional[Hashes] + ): + # type: (...) -> BestCandidateResult + """Find matches for the given project and specifier. + + :param specifier: An optional object implementing `filter` + (e.g. `packaging.specifiers.SpecifierSet`) to filter applicable + versions. + + :return: A `BestCandidateResult` instance. + """ + candidates = self.find_all_candidates(project_name) + candidate_evaluator = self.make_candidate_evaluator( + project_name=project_name, + specifier=specifier, + hashes=hashes, + ) + return candidate_evaluator.compute_best_candidate(candidates) + + def find_requirement(self, req, upgrade): + # type: (InstallRequirement, bool) -> Optional[Link] + """Try to find a Link matching req + + Expects req, an InstallRequirement and upgrade, a boolean + Returns a Link if found, + Raises DistributionNotFound or BestVersionAlreadyInstalled otherwise + """ + hashes = req.hashes(trust_internet=False) + best_candidate_result = self.find_best_candidate( + req.name, specifier=req.specifier, hashes=hashes, + ) + best_candidate = best_candidate_result.best_candidate + + installed_version = None # type: Optional[_BaseVersion] + if req.satisfied_by is not None: + installed_version = parse_version(req.satisfied_by.version) + + def _format_versions(cand_iter): + # type: (Iterable[InstallationCandidate]) -> str + # This repeated parse_version and str() conversion is needed to + # handle different vendoring sources from pip and pkg_resources. + # If we stop using the pkg_resources provided specifier and start + # using our own, we can drop the cast to str(). + return ", ".join(sorted( + {str(c.version) for c in cand_iter}, + key=parse_version, + )) or "none" + + if installed_version is None and best_candidate is None: + logger.critical( + 'Could not find a version that satisfies the requirement %s ' + '(from versions: %s)', + req, + _format_versions(best_candidate_result.iter_all()), + ) + + raise DistributionNotFound( + 'No matching distribution found for %s' % req + ) + + best_installed = False + if installed_version and ( + best_candidate is None or + best_candidate.version <= installed_version): + best_installed = True + + if not upgrade and installed_version is not None: + if best_installed: + logger.debug( + 'Existing installed version (%s) is most up-to-date and ' + 'satisfies requirement', + installed_version, + ) + else: + logger.debug( + 'Existing installed version (%s) satisfies requirement ' + '(most up-to-date version is %s)', + installed_version, + best_candidate.version, + ) + return None + + if best_installed: + # We have an existing version, and its the best version + logger.debug( + 'Installed version (%s) is most up-to-date (past versions: ' + '%s)', + installed_version, + _format_versions(best_candidate_result.iter_applicable()), + ) + raise BestVersionAlreadyInstalled + + logger.debug( + 'Using version %s (newest of versions: %s)', + best_candidate.version, + _format_versions(best_candidate_result.iter_applicable()), + ) + return best_candidate.link + + +def _find_name_version_sep(fragment, canonical_name): + # type: (str, str) -> int + """Find the separator's index based on the package's canonical name. + + :param fragment: A + filename "fragment" (stem) or + egg fragment. + :param canonical_name: The package's canonical name. + + This function is needed since the canonicalized name does not necessarily + have the same length as the egg info's name part. An example:: + + >>> fragment = 'foo__bar-1.0' + >>> canonical_name = 'foo-bar' + >>> _find_name_version_sep(fragment, canonical_name) + 8 + """ + # Project name and version must be separated by one single dash. Find all + # occurrences of dashes; if the string in front of it matches the canonical + # name, this is the one separating the name and version parts. + for i, c in enumerate(fragment): + if c != "-": + continue + if canonicalize_name(fragment[:i]) == canonical_name: + return i + raise ValueError("{} does not match {}".format(fragment, canonical_name)) + + +def _extract_version_from_fragment(fragment, canonical_name): + # type: (str, str) -> Optional[str] + """Parse the version string from a + filename + "fragment" (stem) or egg fragment. + + :param fragment: The string to parse. E.g. foo-2.1 + :param canonical_name: The canonicalized name of the package this + belongs to. + """ + try: + version_start = _find_name_version_sep(fragment, canonical_name) + 1 + except ValueError: + return None + version = fragment[version_start:] + if not version: + return None + return version diff --git a/env/lib/python3.8/site-packages/pip/_internal/legacy_resolve.py b/env/lib/python3.8/site-packages/pip/_internal/legacy_resolve.py new file mode 100644 index 000000000..ca269121b --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/legacy_resolve.py @@ -0,0 +1,430 @@ +"""Dependency Resolution + +The dependency resolution in pip is performed as follows: + +for top-level requirements: + a. only one spec allowed per project, regardless of conflicts or not. + otherwise a "double requirement" exception is raised + b. they override sub-dependency requirements. +for sub-dependencies + a. "first found, wins" (where the order is breadth first) +""" + +# The following comment should be removed at some point in the future. +# mypy: strict-optional=False +# mypy: disallow-untyped-defs=False + +import logging +import sys +from collections import defaultdict +from itertools import chain + +from pip._vendor.packaging import specifiers + +from pip._internal.exceptions import ( + BestVersionAlreadyInstalled, + DistributionNotFound, + HashError, + HashErrors, + UnsupportedPythonVersion, +) +from pip._internal.utils.logging import indent_log +from pip._internal.utils.misc import dist_in_usersite, normalize_version_info +from pip._internal.utils.packaging import ( + check_requires_python, + get_requires_python, +) +from pip._internal.utils.typing import MYPY_CHECK_RUNNING + +if MYPY_CHECK_RUNNING: + from typing import Callable, DefaultDict, List, Optional, Set, Tuple + from pip._vendor import pkg_resources + + from pip._internal.distributions import AbstractDistribution + from pip._internal.index.package_finder import PackageFinder + from pip._internal.operations.prepare import RequirementPreparer + from pip._internal.req.req_install import InstallRequirement + from pip._internal.req.req_set import RequirementSet + + InstallRequirementProvider = Callable[ + [str, InstallRequirement], InstallRequirement + ] + DiscoveredDependencies = DefaultDict[str, List[InstallRequirement]] + +logger = logging.getLogger(__name__) + + +def _check_dist_requires_python( + dist, # type: pkg_resources.Distribution + version_info, # type: Tuple[int, int, int] + ignore_requires_python=False, # type: bool +): + # type: (...) -> None + """ + Check whether the given Python version is compatible with a distribution's + "Requires-Python" value. + + :param version_info: A 3-tuple of ints representing the Python + major-minor-micro version to check. + :param ignore_requires_python: Whether to ignore the "Requires-Python" + value if the given Python version isn't compatible. + + :raises UnsupportedPythonVersion: When the given Python version isn't + compatible. + """ + requires_python = get_requires_python(dist) + try: + is_compatible = check_requires_python( + requires_python, version_info=version_info, + ) + except specifiers.InvalidSpecifier as exc: + logger.warning( + "Package %r has an invalid Requires-Python: %s", + dist.project_name, exc, + ) + return + + if is_compatible: + return + + version = '.'.join(map(str, version_info)) + if ignore_requires_python: + logger.debug( + 'Ignoring failed Requires-Python check for package %r: ' + '%s not in %r', + dist.project_name, version, requires_python, + ) + return + + raise UnsupportedPythonVersion( + 'Package {!r} requires a different Python: {} not in {!r}'.format( + dist.project_name, version, requires_python, + )) + + +class Resolver(object): + """Resolves which packages need to be installed/uninstalled to perform \ + the requested operation without breaking the requirements of any package. + """ + + _allowed_strategies = {"eager", "only-if-needed", "to-satisfy-only"} + + def __init__( + self, + preparer, # type: RequirementPreparer + finder, # type: PackageFinder + make_install_req, # type: InstallRequirementProvider + use_user_site, # type: bool + ignore_dependencies, # type: bool + ignore_installed, # type: bool + ignore_requires_python, # type: bool + force_reinstall, # type: bool + upgrade_strategy, # type: str + py_version_info=None, # type: Optional[Tuple[int, ...]] + ): + # type: (...) -> None + super(Resolver, self).__init__() + assert upgrade_strategy in self._allowed_strategies + + if py_version_info is None: + py_version_info = sys.version_info[:3] + else: + py_version_info = normalize_version_info(py_version_info) + + self._py_version_info = py_version_info + + self.preparer = preparer + self.finder = finder + + self.upgrade_strategy = upgrade_strategy + self.force_reinstall = force_reinstall + self.ignore_dependencies = ignore_dependencies + self.ignore_installed = ignore_installed + self.ignore_requires_python = ignore_requires_python + self.use_user_site = use_user_site + self._make_install_req = make_install_req + + self._discovered_dependencies = \ + defaultdict(list) # type: DiscoveredDependencies + + def resolve(self, requirement_set): + # type: (RequirementSet) -> None + """Resolve what operations need to be done + + As a side-effect of this method, the packages (and their dependencies) + are downloaded, unpacked and prepared for installation. This + preparation is done by ``pip.operations.prepare``. + + Once PyPI has static dependency metadata available, it would be + possible to move the preparation to become a step separated from + dependency resolution. + """ + # If any top-level requirement has a hash specified, enter + # hash-checking mode, which requires hashes from all. + root_reqs = ( + requirement_set.unnamed_requirements + + list(requirement_set.requirements.values()) + ) + + # Actually prepare the files, and collect any exceptions. Most hash + # exceptions cannot be checked ahead of time, because + # req.populate_link() needs to be called before we can make decisions + # based on link type. + discovered_reqs = [] # type: List[InstallRequirement] + hash_errors = HashErrors() + for req in chain(root_reqs, discovered_reqs): + try: + discovered_reqs.extend(self._resolve_one(requirement_set, req)) + except HashError as exc: + exc.req = req + hash_errors.append(exc) + + if hash_errors: + raise hash_errors + + def _is_upgrade_allowed(self, req): + # type: (InstallRequirement) -> bool + if self.upgrade_strategy == "to-satisfy-only": + return False + elif self.upgrade_strategy == "eager": + return True + else: + assert self.upgrade_strategy == "only-if-needed" + return req.is_direct + + def _set_req_to_reinstall(self, req): + # type: (InstallRequirement) -> None + """ + Set a requirement to be installed. + """ + # Don't uninstall the conflict if doing a user install and the + # conflict is not a user install. + if not self.use_user_site or dist_in_usersite(req.satisfied_by): + req.should_reinstall = True + req.satisfied_by = None + + def _check_skip_installed(self, req_to_install): + # type: (InstallRequirement) -> Optional[str] + """Check if req_to_install should be skipped. + + This will check if the req is installed, and whether we should upgrade + or reinstall it, taking into account all the relevant user options. + + After calling this req_to_install will only have satisfied_by set to + None if the req_to_install is to be upgraded/reinstalled etc. Any + other value will be a dist recording the current thing installed that + satisfies the requirement. + + Note that for vcs urls and the like we can't assess skipping in this + routine - we simply identify that we need to pull the thing down, + then later on it is pulled down and introspected to assess upgrade/ + reinstalls etc. + + :return: A text reason for why it was skipped, or None. + """ + if self.ignore_installed: + return None + + req_to_install.check_if_exists(self.use_user_site) + if not req_to_install.satisfied_by: + return None + + if self.force_reinstall: + self._set_req_to_reinstall(req_to_install) + return None + + if not self._is_upgrade_allowed(req_to_install): + if self.upgrade_strategy == "only-if-needed": + return 'already satisfied, skipping upgrade' + return 'already satisfied' + + # Check for the possibility of an upgrade. For link-based + # requirements we have to pull the tree down and inspect to assess + # the version #, so it's handled way down. + if not req_to_install.link: + try: + self.finder.find_requirement(req_to_install, upgrade=True) + except BestVersionAlreadyInstalled: + # Then the best version is installed. + return 'already up-to-date' + except DistributionNotFound: + # No distribution found, so we squash the error. It will + # be raised later when we re-try later to do the install. + # Why don't we just raise here? + pass + + self._set_req_to_reinstall(req_to_install) + return None + + def _get_abstract_dist_for(self, req): + # type: (InstallRequirement) -> AbstractDistribution + """Takes a InstallRequirement and returns a single AbstractDist \ + representing a prepared variant of the same. + """ + if req.editable: + return self.preparer.prepare_editable_requirement(req) + + # satisfied_by is only evaluated by calling _check_skip_installed, + # so it must be None here. + assert req.satisfied_by is None + skip_reason = self._check_skip_installed(req) + + if req.satisfied_by: + return self.preparer.prepare_installed_requirement( + req, skip_reason + ) + + upgrade_allowed = self._is_upgrade_allowed(req) + + # We eagerly populate the link, since that's our "legacy" behavior. + require_hashes = self.preparer.require_hashes + req.populate_link(self.finder, upgrade_allowed, require_hashes) + abstract_dist = self.preparer.prepare_linked_requirement(req) + + # NOTE + # The following portion is for determining if a certain package is + # going to be re-installed/upgraded or not and reporting to the user. + # This should probably get cleaned up in a future refactor. + + # req.req is only avail after unpack for URL + # pkgs repeat check_if_exists to uninstall-on-upgrade + # (#14) + if not self.ignore_installed: + req.check_if_exists(self.use_user_site) + + if req.satisfied_by: + should_modify = ( + self.upgrade_strategy != "to-satisfy-only" or + self.force_reinstall or + self.ignore_installed or + req.link.scheme == 'file' + ) + if should_modify: + self._set_req_to_reinstall(req) + else: + logger.info( + 'Requirement already satisfied (use --upgrade to upgrade):' + ' %s', req, + ) + + return abstract_dist + + def _resolve_one( + self, + requirement_set, # type: RequirementSet + req_to_install, # type: InstallRequirement + ): + # type: (...) -> List[InstallRequirement] + """Prepare a single requirements file. + + :return: A list of additional InstallRequirements to also install. + """ + # Tell user what we are doing for this requirement: + # obtain (editable), skipping, processing (local url), collecting + # (remote url or package name) + if req_to_install.constraint or req_to_install.prepared: + return [] + + req_to_install.prepared = True + + # register tmp src for cleanup in case something goes wrong + requirement_set.reqs_to_cleanup.append(req_to_install) + + abstract_dist = self._get_abstract_dist_for(req_to_install) + + # Parse and return dependencies + dist = abstract_dist.get_pkg_resources_distribution() + # This will raise UnsupportedPythonVersion if the given Python + # version isn't compatible with the distribution's Requires-Python. + _check_dist_requires_python( + dist, version_info=self._py_version_info, + ignore_requires_python=self.ignore_requires_python, + ) + + more_reqs = [] # type: List[InstallRequirement] + + def add_req(subreq, extras_requested): + sub_install_req = self._make_install_req( + str(subreq), + req_to_install, + ) + parent_req_name = req_to_install.name + to_scan_again, add_to_parent = requirement_set.add_requirement( + sub_install_req, + parent_req_name=parent_req_name, + extras_requested=extras_requested, + ) + if parent_req_name and add_to_parent: + self._discovered_dependencies[parent_req_name].append( + add_to_parent + ) + more_reqs.extend(to_scan_again) + + with indent_log(): + # We add req_to_install before its dependencies, so that we + # can refer to it when adding dependencies. + if not requirement_set.has_requirement(req_to_install.name): + # 'unnamed' requirements will get added here + # 'unnamed' requirements can only come from being directly + # provided by the user. + assert req_to_install.is_direct + requirement_set.add_requirement( + req_to_install, parent_req_name=None, + ) + + if not self.ignore_dependencies: + if req_to_install.extras: + logger.debug( + "Installing extra requirements: %r", + ','.join(req_to_install.extras), + ) + missing_requested = sorted( + set(req_to_install.extras) - set(dist.extras) + ) + for missing in missing_requested: + logger.warning( + '%s does not provide the extra \'%s\'', + dist, missing + ) + + available_requested = sorted( + set(dist.extras) & set(req_to_install.extras) + ) + for subreq in dist.requires(available_requested): + add_req(subreq, extras_requested=available_requested) + + if not req_to_install.editable and not req_to_install.satisfied_by: + # XXX: --no-install leads this to report 'Successfully + # downloaded' for only non-editable reqs, even though we took + # action on them. + requirement_set.successfully_downloaded.append(req_to_install) + + return more_reqs + + def get_installation_order(self, req_set): + # type: (RequirementSet) -> List[InstallRequirement] + """Create the installation order. + + The installation order is topological - requirements are installed + before the requiring thing. We break cycles at an arbitrary point, + and make no other guarantees. + """ + # The current implementation, which we may change at any point + # installs the user specified things in the order given, except when + # dependencies must come earlier to achieve topological order. + order = [] + ordered_reqs = set() # type: Set[InstallRequirement] + + def schedule(req): + if req.satisfied_by or req in ordered_reqs: + return + if req.constraint: + return + ordered_reqs.add(req) + for dep in self._discovered_dependencies[req.name]: + schedule(dep) + order.append(req) + + for install_req in req_set.requirements.values(): + schedule(install_req) + return order diff --git a/env/lib/python3.8/site-packages/pip/_internal/locations.py b/env/lib/python3.8/site-packages/pip/_internal/locations.py new file mode 100644 index 000000000..0c1155319 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/locations.py @@ -0,0 +1,194 @@ +"""Locations where we look for configs, install stuff, etc""" + +# The following comment should be removed at some point in the future. +# mypy: strict-optional=False + +from __future__ import absolute_import + +import os +import os.path +import platform +import site +import sys +import sysconfig +from distutils import sysconfig as distutils_sysconfig +from distutils.command.install import SCHEME_KEYS # type: ignore +from distutils.command.install import install as distutils_install_command + +from pip._internal.models.scheme import Scheme +from pip._internal.utils import appdirs +from pip._internal.utils.compat import WINDOWS +from pip._internal.utils.typing import MYPY_CHECK_RUNNING, cast +from pip._internal.utils.virtualenv import running_under_virtualenv + +if MYPY_CHECK_RUNNING: + from typing import Dict, List, Optional, Union + + from distutils.cmd import Command as DistutilsCommand + + +# Application Directories +USER_CACHE_DIR = appdirs.user_cache_dir("pip") + + +def get_major_minor_version(): + # type: () -> str + """ + Return the major-minor version of the current Python as a string, e.g. + "3.7" or "3.10". + """ + return '{}.{}'.format(*sys.version_info) + + +def get_src_prefix(): + # type: () -> str + if running_under_virtualenv(): + src_prefix = os.path.join(sys.prefix, 'src') + else: + # FIXME: keep src in cwd for now (it is not a temporary folder) + try: + src_prefix = os.path.join(os.getcwd(), 'src') + except OSError: + # In case the current working directory has been renamed or deleted + sys.exit( + "The folder you are executing pip from can no longer be found." + ) + + # under macOS + virtualenv sys.prefix is not properly resolved + # it is something like /path/to/python/bin/.. + return os.path.abspath(src_prefix) + + +# FIXME doesn't account for venv linked to global site-packages + +site_packages = sysconfig.get_path("purelib") # type: Optional[str] + +# This is because of a bug in PyPy's sysconfig module, see +# https://bitbucket.org/pypy/pypy/issues/2506/sysconfig-returns-incorrect-paths +# for more information. +if platform.python_implementation().lower() == "pypy": + site_packages = distutils_sysconfig.get_python_lib() +try: + # Use getusersitepackages if this is present, as it ensures that the + # value is initialised properly. + user_site = site.getusersitepackages() +except AttributeError: + user_site = site.USER_SITE + +if WINDOWS: + bin_py = os.path.join(sys.prefix, 'Scripts') + bin_user = os.path.join(user_site, 'Scripts') + # buildout uses 'bin' on Windows too? + if not os.path.exists(bin_py): + bin_py = os.path.join(sys.prefix, 'bin') + bin_user = os.path.join(user_site, 'bin') +else: + bin_py = os.path.join(sys.prefix, 'bin') + bin_user = os.path.join(user_site, 'bin') + + # Forcing to use /usr/local/bin for standard macOS framework installs + # Also log to ~/Library/Logs/ for use with the Console.app log viewer + if sys.platform[:6] == 'darwin' and sys.prefix[:16] == '/System/Library/': + bin_py = '/usr/local/bin' + + +def distutils_scheme( + dist_name, user=False, home=None, root=None, isolated=False, prefix=None +): + # type:(str, bool, str, str, bool, str) -> Dict[str, str] + """ + Return a distutils install scheme + """ + from distutils.dist import Distribution + + dist_args = {'name': dist_name} # type: Dict[str, Union[str, List[str]]] + if isolated: + dist_args["script_args"] = ["--no-user-cfg"] + + d = Distribution(dist_args) + d.parse_config_files() + obj = None # type: Optional[DistutilsCommand] + obj = d.get_command_obj('install', create=True) + assert obj is not None + i = cast(distutils_install_command, obj) + # NOTE: setting user or home has the side-effect of creating the home dir + # or user base for installations during finalize_options() + # ideally, we'd prefer a scheme class that has no side-effects. + assert not (user and prefix), "user={} prefix={}".format(user, prefix) + assert not (home and prefix), "home={} prefix={}".format(home, prefix) + i.user = user or i.user + if user or home: + i.prefix = "" + i.prefix = prefix or i.prefix + i.home = home or i.home + i.root = root or i.root + i.finalize_options() + + scheme = {} + for key in SCHEME_KEYS: + scheme[key] = getattr(i, 'install_' + key) + + # install_lib specified in setup.cfg should install *everything* + # into there (i.e. it takes precedence over both purelib and + # platlib). Note, i.install_lib is *always* set after + # finalize_options(); we only want to override here if the user + # has explicitly requested it hence going back to the config + if 'install_lib' in d.get_option_dict('install'): + scheme.update(dict(purelib=i.install_lib, platlib=i.install_lib)) + + if running_under_virtualenv(): + scheme['headers'] = os.path.join( + sys.prefix, + 'include', + 'site', + 'python{}'.format(get_major_minor_version()), + dist_name, + ) + + if root is not None: + path_no_drive = os.path.splitdrive( + os.path.abspath(scheme["headers"]))[1] + scheme["headers"] = os.path.join( + root, + path_no_drive[1:], + ) + + return scheme + + +def get_scheme( + dist_name, # type: str + user=False, # type: bool + home=None, # type: Optional[str] + root=None, # type: Optional[str] + isolated=False, # type: bool + prefix=None, # type: Optional[str] +): + # type: (...) -> Scheme + """ + Get the "scheme" corresponding to the input parameters. The distutils + documentation provides the context for the available schemes: + https://docs.python.org/3/install/index.html#alternate-installation + + :param dist_name: the name of the package to retrieve the scheme for, used + in the headers scheme path + :param user: indicates to use the "user" scheme + :param home: indicates to use the "home" scheme and provides the base + directory for the same + :param root: root under which other directories are re-based + :param isolated: equivalent to --no-user-cfg, i.e. do not consider + ~/.pydistutils.cfg (posix) or ~/pydistutils.cfg (non-posix) for + scheme paths + :param prefix: indicates to use the "prefix" scheme and provides the + base directory for the same + """ + scheme = distutils_scheme( + dist_name, user, home, root, isolated, prefix + ) + return Scheme( + platlib=scheme["platlib"], + purelib=scheme["purelib"], + headers=scheme["headers"], + scripts=scheme["scripts"], + data=scheme["data"], + ) diff --git a/env/lib/python3.8/site-packages/pip/_internal/main.py b/env/lib/python3.8/site-packages/pip/_internal/main.py new file mode 100644 index 000000000..3208d5b88 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/main.py @@ -0,0 +1,16 @@ +from pip._internal.utils.typing import MYPY_CHECK_RUNNING + +if MYPY_CHECK_RUNNING: + from typing import Optional, List + + +def main(args=None): + # type: (Optional[List[str]]) -> int + """This is preserved for old console scripts that may still be referencing + it. + + For additional details, see https://github.com/pypa/pip/issues/7498. + """ + from pip._internal.utils.entrypoints import _wrapper + + return _wrapper(args) diff --git a/env/lib/python3.8/site-packages/pip/_internal/models/__init__.py b/env/lib/python3.8/site-packages/pip/_internal/models/__init__.py new file mode 100644 index 000000000..7855226e4 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/models/__init__.py @@ -0,0 +1,2 @@ +"""A package that contains models that represent entities. +""" diff --git a/env/lib/python3.8/site-packages/pip/_internal/models/candidate.py b/env/lib/python3.8/site-packages/pip/_internal/models/candidate.py new file mode 100644 index 000000000..1dc1a576e --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/models/candidate.py @@ -0,0 +1,36 @@ +from pip._vendor.packaging.version import parse as parse_version + +from pip._internal.utils.models import KeyBasedCompareMixin +from pip._internal.utils.typing import MYPY_CHECK_RUNNING + +if MYPY_CHECK_RUNNING: + from pip._vendor.packaging.version import _BaseVersion + from pip._internal.models.link import Link + + +class InstallationCandidate(KeyBasedCompareMixin): + """Represents a potential "candidate" for installation. + """ + + def __init__(self, name, version, link): + # type: (str, str, Link) -> None + self.name = name + self.version = parse_version(version) # type: _BaseVersion + self.link = link + + super(InstallationCandidate, self).__init__( + key=(self.name, self.version, self.link), + defining_class=InstallationCandidate + ) + + def __repr__(self): + # type: () -> str + return "".format( + self.name, self.version, self.link, + ) + + def __str__(self): + # type: () -> str + return '{!r} candidate (version {} at {})'.format( + self.name, self.version, self.link, + ) diff --git a/env/lib/python3.8/site-packages/pip/_internal/models/format_control.py b/env/lib/python3.8/site-packages/pip/_internal/models/format_control.py new file mode 100644 index 000000000..2e13727ca --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/models/format_control.py @@ -0,0 +1,84 @@ +# The following comment should be removed at some point in the future. +# mypy: strict-optional=False + +from pip._vendor.packaging.utils import canonicalize_name + +from pip._internal.exceptions import CommandError +from pip._internal.utils.typing import MYPY_CHECK_RUNNING + +if MYPY_CHECK_RUNNING: + from typing import Optional, Set, FrozenSet + + +class FormatControl(object): + """Helper for managing formats from which a package can be installed. + """ + + def __init__(self, no_binary=None, only_binary=None): + # type: (Optional[Set[str]], Optional[Set[str]]) -> None + if no_binary is None: + no_binary = set() + if only_binary is None: + only_binary = set() + + self.no_binary = no_binary + self.only_binary = only_binary + + def __eq__(self, other): + # type: (object) -> bool + return self.__dict__ == other.__dict__ + + def __ne__(self, other): + # type: (object) -> bool + return not self.__eq__(other) + + def __repr__(self): + # type: () -> str + return "{}({}, {})".format( + self.__class__.__name__, + self.no_binary, + self.only_binary + ) + + @staticmethod + def handle_mutual_excludes(value, target, other): + # type: (str, Optional[Set[str]], Optional[Set[str]]) -> None + if value.startswith('-'): + raise CommandError( + "--no-binary / --only-binary option requires 1 argument." + ) + new = value.split(',') + while ':all:' in new: + other.clear() + target.clear() + target.add(':all:') + del new[:new.index(':all:') + 1] + # Without a none, we want to discard everything as :all: covers it + if ':none:' not in new: + return + for name in new: + if name == ':none:': + target.clear() + continue + name = canonicalize_name(name) + other.discard(name) + target.add(name) + + def get_allowed_formats(self, canonical_name): + # type: (str) -> FrozenSet[str] + result = {"binary", "source"} + if canonical_name in self.only_binary: + result.discard('source') + elif canonical_name in self.no_binary: + result.discard('binary') + elif ':all:' in self.only_binary: + result.discard('source') + elif ':all:' in self.no_binary: + result.discard('binary') + return frozenset(result) + + def disallow_binaries(self): + # type: () -> None + self.handle_mutual_excludes( + ':all:', self.no_binary, self.only_binary, + ) diff --git a/env/lib/python3.8/site-packages/pip/_internal/models/index.py b/env/lib/python3.8/site-packages/pip/_internal/models/index.py new file mode 100644 index 000000000..ead1efbda --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/models/index.py @@ -0,0 +1,31 @@ +from pip._vendor.six.moves.urllib import parse as urllib_parse + + +class PackageIndex(object): + """Represents a Package Index and provides easier access to endpoints + """ + + def __init__(self, url, file_storage_domain): + # type: (str, str) -> None + super(PackageIndex, self).__init__() + self.url = url + self.netloc = urllib_parse.urlsplit(url).netloc + self.simple_url = self._url_for_path('simple') + self.pypi_url = self._url_for_path('pypi') + + # This is part of a temporary hack used to block installs of PyPI + # packages which depend on external urls only necessary until PyPI can + # block such packages themselves + self.file_storage_domain = file_storage_domain + + def _url_for_path(self, path): + # type: (str) -> str + return urllib_parse.urljoin(self.url, path) + + +PyPI = PackageIndex( + 'https://pypi.org/', file_storage_domain='files.pythonhosted.org' +) +TestPyPI = PackageIndex( + 'https://test.pypi.org/', file_storage_domain='test-files.pythonhosted.org' +) diff --git a/env/lib/python3.8/site-packages/pip/_internal/models/link.py b/env/lib/python3.8/site-packages/pip/_internal/models/link.py new file mode 100644 index 000000000..34fbcbfe7 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/models/link.py @@ -0,0 +1,227 @@ +import os +import posixpath +import re + +from pip._vendor.six.moves.urllib import parse as urllib_parse + +from pip._internal.utils.filetypes import WHEEL_EXTENSION +from pip._internal.utils.misc import ( + redact_auth_from_url, + split_auth_from_netloc, + splitext, +) +from pip._internal.utils.models import KeyBasedCompareMixin +from pip._internal.utils.typing import MYPY_CHECK_RUNNING +from pip._internal.utils.urls import path_to_url, url_to_path + +if MYPY_CHECK_RUNNING: + from typing import Optional, Text, Tuple, Union + from pip._internal.index.collector import HTMLPage + from pip._internal.utils.hashes import Hashes + + +class Link(KeyBasedCompareMixin): + """Represents a parsed link from a Package Index's simple URL + """ + + def __init__( + self, + url, # type: str + comes_from=None, # type: Optional[Union[str, HTMLPage]] + requires_python=None, # type: Optional[str] + yanked_reason=None, # type: Optional[Text] + ): + # type: (...) -> None + """ + :param url: url of the resource pointed to (href of the link) + :param comes_from: instance of HTMLPage where the link was found, + or string. + :param requires_python: String containing the `Requires-Python` + metadata field, specified in PEP 345. This may be specified by + a data-requires-python attribute in the HTML link tag, as + described in PEP 503. + :param yanked_reason: the reason the file has been yanked, if the + file has been yanked, or None if the file hasn't been yanked. + This is the value of the "data-yanked" attribute, if present, in + a simple repository HTML link. If the file has been yanked but + no reason was provided, this should be the empty string. See + PEP 592 for more information and the specification. + """ + + # url can be a UNC windows share + if url.startswith('\\\\'): + url = path_to_url(url) + + self._parsed_url = urllib_parse.urlsplit(url) + # Store the url as a private attribute to prevent accidentally + # trying to set a new value. + self._url = url + + self.comes_from = comes_from + self.requires_python = requires_python if requires_python else None + self.yanked_reason = yanked_reason + + super(Link, self).__init__(key=url, defining_class=Link) + + def __str__(self): + # type: () -> str + if self.requires_python: + rp = ' (requires-python:%s)' % self.requires_python + else: + rp = '' + if self.comes_from: + return '%s (from %s)%s' % (redact_auth_from_url(self._url), + self.comes_from, rp) + else: + return redact_auth_from_url(str(self._url)) + + def __repr__(self): + # type: () -> str + return '' % self + + @property + def url(self): + # type: () -> str + return self._url + + @property + def filename(self): + # type: () -> str + path = self.path.rstrip('/') + name = posixpath.basename(path) + if not name: + # Make sure we don't leak auth information if the netloc + # includes a username and password. + netloc, user_pass = split_auth_from_netloc(self.netloc) + return netloc + + name = urllib_parse.unquote(name) + assert name, ('URL %r produced no filename' % self._url) + return name + + @property + def file_path(self): + # type: () -> str + return url_to_path(self.url) + + @property + def scheme(self): + # type: () -> str + return self._parsed_url.scheme + + @property + def netloc(self): + # type: () -> str + """ + This can contain auth information. + """ + return self._parsed_url.netloc + + @property + def path(self): + # type: () -> str + return urllib_parse.unquote(self._parsed_url.path) + + def splitext(self): + # type: () -> Tuple[str, str] + return splitext(posixpath.basename(self.path.rstrip('/'))) + + @property + def ext(self): + # type: () -> str + return self.splitext()[1] + + @property + def url_without_fragment(self): + # type: () -> str + scheme, netloc, path, query, fragment = self._parsed_url + return urllib_parse.urlunsplit((scheme, netloc, path, query, None)) + + _egg_fragment_re = re.compile(r'[#&]egg=([^&]*)') + + @property + def egg_fragment(self): + # type: () -> Optional[str] + match = self._egg_fragment_re.search(self._url) + if not match: + return None + return match.group(1) + + _subdirectory_fragment_re = re.compile(r'[#&]subdirectory=([^&]*)') + + @property + def subdirectory_fragment(self): + # type: () -> Optional[str] + match = self._subdirectory_fragment_re.search(self._url) + if not match: + return None + return match.group(1) + + _hash_re = re.compile( + r'(sha1|sha224|sha384|sha256|sha512|md5)=([a-f0-9]+)' + ) + + @property + def hash(self): + # type: () -> Optional[str] + match = self._hash_re.search(self._url) + if match: + return match.group(2) + return None + + @property + def hash_name(self): + # type: () -> Optional[str] + match = self._hash_re.search(self._url) + if match: + return match.group(1) + return None + + @property + def show_url(self): + # type: () -> str + return posixpath.basename(self._url.split('#', 1)[0].split('?', 1)[0]) + + @property + def is_file(self): + # type: () -> bool + return self.scheme == 'file' + + def is_existing_dir(self): + # type: () -> bool + return self.is_file and os.path.isdir(self.file_path) + + @property + def is_wheel(self): + # type: () -> bool + return self.ext == WHEEL_EXTENSION + + @property + def is_vcs(self): + # type: () -> bool + from pip._internal.vcs import vcs + + return self.scheme in vcs.all_schemes + + @property + def is_yanked(self): + # type: () -> bool + return self.yanked_reason is not None + + @property + def has_hash(self): + # type: () -> bool + return self.hash_name is not None + + def is_hash_allowed(self, hashes): + # type: (Optional[Hashes]) -> bool + """ + Return True if the link has a hash and it is allowed. + """ + if hashes is None or not self.has_hash: + return False + # Assert non-None so mypy knows self.hash_name and self.hash are str. + assert self.hash_name is not None + assert self.hash is not None + + return hashes.is_hash_allowed(self.hash_name, hex_digest=self.hash) diff --git a/env/lib/python3.8/site-packages/pip/_internal/models/scheme.py b/env/lib/python3.8/site-packages/pip/_internal/models/scheme.py new file mode 100644 index 000000000..af07b4078 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/models/scheme.py @@ -0,0 +1,25 @@ +""" +For types associated with installation schemes. + +For a general overview of available schemes and their context, see +https://docs.python.org/3/install/index.html#alternate-installation. +""" + + +class Scheme(object): + """A Scheme holds paths which are used as the base directories for + artifacts associated with a Python package. + """ + def __init__( + self, + platlib, # type: str + purelib, # type: str + headers, # type: str + scripts, # type: str + data, # type: str + ): + self.platlib = platlib + self.purelib = purelib + self.headers = headers + self.scripts = scripts + self.data = data diff --git a/env/lib/python3.8/site-packages/pip/_internal/models/search_scope.py b/env/lib/python3.8/site-packages/pip/_internal/models/search_scope.py new file mode 100644 index 000000000..138d1b6ee --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/models/search_scope.py @@ -0,0 +1,114 @@ +import itertools +import logging +import os +import posixpath + +from pip._vendor.packaging.utils import canonicalize_name +from pip._vendor.six.moves.urllib import parse as urllib_parse + +from pip._internal.models.index import PyPI +from pip._internal.utils.compat import has_tls +from pip._internal.utils.misc import normalize_path, redact_auth_from_url +from pip._internal.utils.typing import MYPY_CHECK_RUNNING + +if MYPY_CHECK_RUNNING: + from typing import List + + +logger = logging.getLogger(__name__) + + +class SearchScope(object): + + """ + Encapsulates the locations that pip is configured to search. + """ + + @classmethod + def create( + cls, + find_links, # type: List[str] + index_urls, # type: List[str] + ): + # type: (...) -> SearchScope + """ + Create a SearchScope object after normalizing the `find_links`. + """ + # Build find_links. If an argument starts with ~, it may be + # a local file relative to a home directory. So try normalizing + # it and if it exists, use the normalized version. + # This is deliberately conservative - it might be fine just to + # blindly normalize anything starting with a ~... + built_find_links = [] # type: List[str] + for link in find_links: + if link.startswith('~'): + new_link = normalize_path(link) + if os.path.exists(new_link): + link = new_link + built_find_links.append(link) + + # If we don't have TLS enabled, then WARN if anyplace we're looking + # relies on TLS. + if not has_tls(): + for link in itertools.chain(index_urls, built_find_links): + parsed = urllib_parse.urlparse(link) + if parsed.scheme == 'https': + logger.warning( + 'pip is configured with locations that require ' + 'TLS/SSL, however the ssl module in Python is not ' + 'available.' + ) + break + + return cls( + find_links=built_find_links, + index_urls=index_urls, + ) + + def __init__( + self, + find_links, # type: List[str] + index_urls, # type: List[str] + ): + # type: (...) -> None + self.find_links = find_links + self.index_urls = index_urls + + def get_formatted_locations(self): + # type: () -> str + lines = [] + if self.index_urls and self.index_urls != [PyPI.simple_url]: + lines.append( + 'Looking in indexes: {}'.format(', '.join( + redact_auth_from_url(url) for url in self.index_urls)) + ) + if self.find_links: + lines.append( + 'Looking in links: {}'.format(', '.join( + redact_auth_from_url(url) for url in self.find_links)) + ) + return '\n'.join(lines) + + def get_index_urls_locations(self, project_name): + # type: (str) -> List[str] + """Returns the locations found via self.index_urls + + Checks the url_name on the main (first in the list) index and + use this url_name to produce all locations + """ + + def mkurl_pypi_url(url): + # type: (str) -> str + loc = posixpath.join( + url, + urllib_parse.quote(canonicalize_name(project_name))) + # For maximum compatibility with easy_install, ensure the path + # ends in a trailing slash. Although this isn't in the spec + # (and PyPI can handle it without the slash) some other index + # implementations might break if they relied on easy_install's + # behavior. + if not loc.endswith('/'): + loc = loc + '/' + return loc + + return [mkurl_pypi_url(url) for url in self.index_urls] diff --git a/env/lib/python3.8/site-packages/pip/_internal/models/selection_prefs.py b/env/lib/python3.8/site-packages/pip/_internal/models/selection_prefs.py new file mode 100644 index 000000000..f58fdce9c --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/models/selection_prefs.py @@ -0,0 +1,47 @@ +from pip._internal.utils.typing import MYPY_CHECK_RUNNING + +if MYPY_CHECK_RUNNING: + from typing import Optional + from pip._internal.models.format_control import FormatControl + + +class SelectionPreferences(object): + + """ + Encapsulates the candidate selection preferences for downloading + and installing files. + """ + + # Don't include an allow_yanked default value to make sure each call + # site considers whether yanked releases are allowed. This also causes + # that decision to be made explicit in the calling code, which helps + # people when reading the code. + def __init__( + self, + allow_yanked, # type: bool + allow_all_prereleases=False, # type: bool + format_control=None, # type: Optional[FormatControl] + prefer_binary=False, # type: bool + ignore_requires_python=None, # type: Optional[bool] + ): + # type: (...) -> None + """Create a SelectionPreferences object. + + :param allow_yanked: Whether files marked as yanked (in the sense + of PEP 592) are permitted to be candidates for install. + :param format_control: A FormatControl object or None. Used to control + the selection of source packages / binary packages when consulting + the index and links. + :param prefer_binary: Whether to prefer an old, but valid, binary + dist over a new source dist. + :param ignore_requires_python: Whether to ignore incompatible + "Requires-Python" values in links. Defaults to False. + """ + if ignore_requires_python is None: + ignore_requires_python = False + + self.allow_yanked = allow_yanked + self.allow_all_prereleases = allow_all_prereleases + self.format_control = format_control + self.prefer_binary = prefer_binary + self.ignore_requires_python = ignore_requires_python diff --git a/env/lib/python3.8/site-packages/pip/_internal/models/target_python.py b/env/lib/python3.8/site-packages/pip/_internal/models/target_python.py new file mode 100644 index 000000000..97ae85a09 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/models/target_python.py @@ -0,0 +1,107 @@ +import sys + +from pip._internal.pep425tags import get_supported, version_info_to_nodot +from pip._internal.utils.misc import normalize_version_info +from pip._internal.utils.typing import MYPY_CHECK_RUNNING + +if MYPY_CHECK_RUNNING: + from typing import List, Optional, Tuple + + from pip._vendor.packaging.tags import Tag + + +class TargetPython(object): + + """ + Encapsulates the properties of a Python interpreter one is targeting + for a package install, download, etc. + """ + + def __init__( + self, + platform=None, # type: Optional[str] + py_version_info=None, # type: Optional[Tuple[int, ...]] + abi=None, # type: Optional[str] + implementation=None, # type: Optional[str] + ): + # type: (...) -> None + """ + :param platform: A string or None. If None, searches for packages + that are supported by the current system. Otherwise, will find + packages that can be built on the platform passed in. These + packages will only be downloaded for distribution: they will + not be built locally. + :param py_version_info: An optional tuple of ints representing the + Python version information to use (e.g. `sys.version_info[:3]`). + This can have length 1, 2, or 3 when provided. + :param abi: A string or None. This is passed to pep425tags.py's + get_supported() function as is. + :param implementation: A string or None. This is passed to + pep425tags.py's get_supported() function as is. + """ + # Store the given py_version_info for when we call get_supported(). + self._given_py_version_info = py_version_info + + if py_version_info is None: + py_version_info = sys.version_info[:3] + else: + py_version_info = normalize_version_info(py_version_info) + + py_version = '.'.join(map(str, py_version_info[:2])) + + self.abi = abi + self.implementation = implementation + self.platform = platform + self.py_version = py_version + self.py_version_info = py_version_info + + # This is used to cache the return value of get_tags(). + self._valid_tags = None # type: Optional[List[Tag]] + + def format_given(self): + # type: () -> str + """ + Format the given, non-None attributes for display. + """ + display_version = None + if self._given_py_version_info is not None: + display_version = '.'.join( + str(part) for part in self._given_py_version_info + ) + + key_values = [ + ('platform', self.platform), + ('version_info', display_version), + ('abi', self.abi), + ('implementation', self.implementation), + ] + return ' '.join( + '{}={!r}'.format(key, value) for key, value in key_values + if value is not None + ) + + def get_tags(self): + # type: () -> List[Tag] + """ + Return the supported PEP 425 tags to check wheel candidates against. + + The tags are returned in order of preference (most preferred first). + """ + if self._valid_tags is None: + # Pass versions=None if no py_version_info was given since + # versions=None uses special default logic. + py_version_info = self._given_py_version_info + if py_version_info is None: + version = None + else: + version = version_info_to_nodot(py_version_info) + + tags = get_supported( + version=version, + platform=self.platform, + abi=self.abi, + impl=self.implementation, + ) + self._valid_tags = tags + + return self._valid_tags diff --git a/env/lib/python3.8/site-packages/pip/_internal/models/wheel.py b/env/lib/python3.8/site-packages/pip/_internal/models/wheel.py new file mode 100644 index 000000000..f1e3f44c5 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/models/wheel.py @@ -0,0 +1,78 @@ +"""Represents a wheel file and provides access to the various parts of the +name that have meaning. +""" +import re + +from pip._vendor.packaging.tags import Tag + +from pip._internal.exceptions import InvalidWheelFilename +from pip._internal.utils.typing import MYPY_CHECK_RUNNING + +if MYPY_CHECK_RUNNING: + from typing import List + + +class Wheel(object): + """A wheel file""" + + wheel_file_re = re.compile( + r"""^(?P(?P.+?)-(?P.*?)) + ((-(?P\d[^-]*?))?-(?P.+?)-(?P.+?)-(?P.+?) + \.whl|\.dist-info)$""", + re.VERBOSE + ) + + def __init__(self, filename): + # type: (str) -> None + """ + :raises InvalidWheelFilename: when the filename is invalid for a wheel + """ + wheel_info = self.wheel_file_re.match(filename) + if not wheel_info: + raise InvalidWheelFilename( + "%s is not a valid wheel filename." % filename + ) + self.filename = filename + self.name = wheel_info.group('name').replace('_', '-') + # we'll assume "_" means "-" due to wheel naming scheme + # (https://github.com/pypa/pip/issues/1150) + self.version = wheel_info.group('ver').replace('_', '-') + self.build_tag = wheel_info.group('build') + self.pyversions = wheel_info.group('pyver').split('.') + self.abis = wheel_info.group('abi').split('.') + self.plats = wheel_info.group('plat').split('.') + + # All the tag combinations from this file + self.file_tags = { + Tag(x, y, z) for x in self.pyversions + for y in self.abis for z in self.plats + } + + def get_formatted_file_tags(self): + # type: () -> List[str] + """Return the wheel's tags as a sorted list of strings.""" + return sorted(str(tag) for tag in self.file_tags) + + def support_index_min(self, tags): + # type: (List[Tag]) -> int + """Return the lowest index that one of the wheel's file_tag combinations + achieves in the given list of supported tags. + + For example, if there are 8 supported tags and one of the file tags + is first in the list, then return 0. + + :param tags: the PEP 425 tags to check the wheel against, in order + with most preferred first. + + :raises ValueError: If none of the wheel's file tags match one of + the supported tags. + """ + return min(tags.index(tag) for tag in self.file_tags if tag in tags) + + def supported(self, tags): + # type: (List[Tag]) -> bool + """Return whether the wheel is compatible with one of the given tags. + + :param tags: the PEP 425 tags to check the wheel against. + """ + return not self.file_tags.isdisjoint(tags) diff --git a/env/lib/python3.8/site-packages/pip/_internal/network/__init__.py b/env/lib/python3.8/site-packages/pip/_internal/network/__init__.py new file mode 100644 index 000000000..b51bde91b --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/network/__init__.py @@ -0,0 +1,2 @@ +"""Contains purely network-related utilities. +""" diff --git a/env/lib/python3.8/site-packages/pip/_internal/network/auth.py b/env/lib/python3.8/site-packages/pip/_internal/network/auth.py new file mode 100644 index 000000000..1e1da54ca --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/network/auth.py @@ -0,0 +1,298 @@ +"""Network Authentication Helpers + +Contains interface (MultiDomainBasicAuth) and associated glue code for +providing credentials in the context of network requests. +""" + +# The following comment should be removed at some point in the future. +# mypy: disallow-untyped-defs=False + +import logging + +from pip._vendor.requests.auth import AuthBase, HTTPBasicAuth +from pip._vendor.requests.utils import get_netrc_auth +from pip._vendor.six.moves.urllib import parse as urllib_parse + +from pip._internal.utils.misc import ( + ask, + ask_input, + ask_password, + remove_auth_from_url, + split_auth_netloc_from_url, +) +from pip._internal.utils.typing import MYPY_CHECK_RUNNING + +if MYPY_CHECK_RUNNING: + from optparse import Values + from typing import Dict, Optional, Tuple + + from pip._internal.vcs.versioncontrol import AuthInfo + + Credentials = Tuple[str, str, str] + +logger = logging.getLogger(__name__) + +try: + import keyring # noqa +except ImportError: + keyring = None +except Exception as exc: + logger.warning( + "Keyring is skipped due to an exception: %s", str(exc), + ) + keyring = None + + +def get_keyring_auth(url, username): + """Return the tuple auth for a given url from keyring.""" + if not url or not keyring: + return None + + try: + try: + get_credential = keyring.get_credential + except AttributeError: + pass + else: + logger.debug("Getting credentials from keyring for %s", url) + cred = get_credential(url, username) + if cred is not None: + return cred.username, cred.password + return None + + if username: + logger.debug("Getting password from keyring for %s", url) + password = keyring.get_password(url, username) + if password: + return username, password + + except Exception as exc: + logger.warning( + "Keyring is skipped due to an exception: %s", str(exc), + ) + + +class MultiDomainBasicAuth(AuthBase): + + def __init__(self, prompting=True, index_urls=None): + # type: (bool, Optional[Values]) -> None + self.prompting = prompting + self.index_urls = index_urls + self.passwords = {} # type: Dict[str, AuthInfo] + # When the user is prompted to enter credentials and keyring is + # available, we will offer to save them. If the user accepts, + # this value is set to the credentials they entered. After the + # request authenticates, the caller should call + # ``save_credentials`` to save these. + self._credentials_to_save = None # type: Optional[Credentials] + + def _get_index_url(self, url): + """Return the original index URL matching the requested URL. + + Cached or dynamically generated credentials may work against + the original index URL rather than just the netloc. + + The provided url should have had its username and password + removed already. If the original index url had credentials then + they will be included in the return value. + + Returns None if no matching index was found, or if --no-index + was specified by the user. + """ + if not url or not self.index_urls: + return None + + for u in self.index_urls: + prefix = remove_auth_from_url(u).rstrip("/") + "/" + if url.startswith(prefix): + return u + + def _get_new_credentials(self, original_url, allow_netrc=True, + allow_keyring=True): + """Find and return credentials for the specified URL.""" + # Split the credentials and netloc from the url. + url, netloc, url_user_password = split_auth_netloc_from_url( + original_url, + ) + + # Start with the credentials embedded in the url + username, password = url_user_password + if username is not None and password is not None: + logger.debug("Found credentials in url for %s", netloc) + return url_user_password + + # Find a matching index url for this request + index_url = self._get_index_url(url) + if index_url: + # Split the credentials from the url. + index_info = split_auth_netloc_from_url(index_url) + if index_info: + index_url, _, index_url_user_password = index_info + logger.debug("Found index url %s", index_url) + + # If an index URL was found, try its embedded credentials + if index_url and index_url_user_password[0] is not None: + username, password = index_url_user_password + if username is not None and password is not None: + logger.debug("Found credentials in index url for %s", netloc) + return index_url_user_password + + # Get creds from netrc if we still don't have them + if allow_netrc: + netrc_auth = get_netrc_auth(original_url) + if netrc_auth: + logger.debug("Found credentials in netrc for %s", netloc) + return netrc_auth + + # If we don't have a password and keyring is available, use it. + if allow_keyring: + # The index url is more specific than the netloc, so try it first + kr_auth = ( + get_keyring_auth(index_url, username) or + get_keyring_auth(netloc, username) + ) + if kr_auth: + logger.debug("Found credentials in keyring for %s", netloc) + return kr_auth + + return username, password + + def _get_url_and_credentials(self, original_url): + """Return the credentials to use for the provided URL. + + If allowed, netrc and keyring may be used to obtain the + correct credentials. + + Returns (url_without_credentials, username, password). Note + that even if the original URL contains credentials, this + function may return a different username and password. + """ + url, netloc, _ = split_auth_netloc_from_url(original_url) + + # Use any stored credentials that we have for this netloc + username, password = self.passwords.get(netloc, (None, None)) + + if username is None and password is None: + # No stored credentials. Acquire new credentials without prompting + # the user. (e.g. from netrc, keyring, or the URL itself) + username, password = self._get_new_credentials(original_url) + + if username is not None or password is not None: + # Convert the username and password if they're None, so that + # this netloc will show up as "cached" in the conditional above. + # Further, HTTPBasicAuth doesn't accept None, so it makes sense to + # cache the value that is going to be used. + username = username or "" + password = password or "" + + # Store any acquired credentials. + self.passwords[netloc] = (username, password) + + assert ( + # Credentials were found + (username is not None and password is not None) or + # Credentials were not found + (username is None and password is None) + ), "Could not load credentials from url: {}".format(original_url) + + return url, username, password + + def __call__(self, req): + # Get credentials for this request + url, username, password = self._get_url_and_credentials(req.url) + + # Set the url of the request to the url without any credentials + req.url = url + + if username is not None and password is not None: + # Send the basic auth with this request + req = HTTPBasicAuth(username, password)(req) + + # Attach a hook to handle 401 responses + req.register_hook("response", self.handle_401) + + return req + + # Factored out to allow for easy patching in tests + def _prompt_for_password(self, netloc): + username = ask_input("User for %s: " % netloc) + if not username: + return None, None + auth = get_keyring_auth(netloc, username) + if auth: + return auth[0], auth[1], False + password = ask_password("Password: ") + return username, password, True + + # Factored out to allow for easy patching in tests + def _should_save_password_to_keyring(self): + if not keyring: + return False + return ask("Save credentials to keyring [y/N]: ", ["y", "n"]) == "y" + + def handle_401(self, resp, **kwargs): + # We only care about 401 responses, anything else we want to just + # pass through the actual response + if resp.status_code != 401: + return resp + + # We are not able to prompt the user so simply return the response + if not self.prompting: + return resp + + parsed = urllib_parse.urlparse(resp.url) + + # Prompt the user for a new username and password + username, password, save = self._prompt_for_password(parsed.netloc) + + # Store the new username and password to use for future requests + self._credentials_to_save = None + if username is not None and password is not None: + self.passwords[parsed.netloc] = (username, password) + + # Prompt to save the password to keyring + if save and self._should_save_password_to_keyring(): + self._credentials_to_save = (parsed.netloc, username, password) + + # Consume content and release the original connection to allow our new + # request to reuse the same one. + resp.content + resp.raw.release_conn() + + # Add our new username and password to the request + req = HTTPBasicAuth(username or "", password or "")(resp.request) + req.register_hook("response", self.warn_on_401) + + # On successful request, save the credentials that were used to + # keyring. (Note that if the user responded "no" above, this member + # is not set and nothing will be saved.) + if self._credentials_to_save: + req.register_hook("response", self.save_credentials) + + # Send our new request + new_resp = resp.connection.send(req, **kwargs) + new_resp.history.append(resp) + + return new_resp + + def warn_on_401(self, resp, **kwargs): + """Response callback to warn about incorrect credentials.""" + if resp.status_code == 401: + logger.warning( + '401 Error, Credentials not correct for %s', resp.request.url, + ) + + def save_credentials(self, resp, **kwargs): + """Response callback to save credentials on success.""" + assert keyring is not None, "should never reach here without keyring" + if not keyring: + return + + creds = self._credentials_to_save + self._credentials_to_save = None + if creds and resp.status_code < 400: + try: + logger.info('Saving credentials to keyring') + keyring.set_password(*creds) + except Exception: + logger.exception('Failed to save credentials') diff --git a/env/lib/python3.8/site-packages/pip/_internal/network/cache.py b/env/lib/python3.8/site-packages/pip/_internal/network/cache.py new file mode 100644 index 000000000..c9386e173 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/network/cache.py @@ -0,0 +1,81 @@ +"""HTTP cache implementation. +""" + +# The following comment should be removed at some point in the future. +# mypy: disallow-untyped-defs=False + +import os +from contextlib import contextmanager + +from pip._vendor.cachecontrol.cache import BaseCache +from pip._vendor.cachecontrol.caches import FileCache +from pip._vendor.requests.models import Response + +from pip._internal.utils.filesystem import adjacent_tmp_file, replace +from pip._internal.utils.misc import ensure_dir +from pip._internal.utils.typing import MYPY_CHECK_RUNNING + +if MYPY_CHECK_RUNNING: + from typing import Optional + + +def is_from_cache(response): + # type: (Response) -> bool + return getattr(response, "from_cache", False) + + +@contextmanager +def suppressed_cache_errors(): + """If we can't access the cache then we can just skip caching and process + requests as if caching wasn't enabled. + """ + try: + yield + except (OSError, IOError): + pass + + +class SafeFileCache(BaseCache): + """ + A file based cache which is safe to use even when the target directory may + not be accessible or writable. + """ + + def __init__(self, directory): + # type: (str) -> None + assert directory is not None, "Cache directory must not be None." + super(SafeFileCache, self).__init__() + self.directory = directory + + def _get_cache_path(self, name): + # type: (str) -> str + # From cachecontrol.caches.file_cache.FileCache._fn, brought into our + # class for backwards-compatibility and to avoid using a non-public + # method. + hashed = FileCache.encode(name) + parts = list(hashed[:5]) + [hashed] + return os.path.join(self.directory, *parts) + + def get(self, key): + # type: (str) -> Optional[bytes] + path = self._get_cache_path(key) + with suppressed_cache_errors(): + with open(path, 'rb') as f: + return f.read() + + def set(self, key, value): + # type: (str, bytes) -> None + path = self._get_cache_path(key) + with suppressed_cache_errors(): + ensure_dir(os.path.dirname(path)) + + with adjacent_tmp_file(path) as f: + f.write(value) + + replace(f.name, path) + + def delete(self, key): + # type: (str) -> None + path = self._get_cache_path(key) + with suppressed_cache_errors(): + os.remove(path) diff --git a/env/lib/python3.8/site-packages/pip/_internal/network/download.py b/env/lib/python3.8/site-packages/pip/_internal/network/download.py new file mode 100644 index 000000000..c90c4bf42 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/network/download.py @@ -0,0 +1,200 @@ +"""Download files with progress indicators. +""" +import cgi +import logging +import mimetypes +import os + +from pip._vendor import requests +from pip._vendor.requests.models import CONTENT_CHUNK_SIZE + +from pip._internal.models.index import PyPI +from pip._internal.network.cache import is_from_cache +from pip._internal.network.utils import response_chunks +from pip._internal.utils.misc import ( + format_size, + redact_auth_from_url, + splitext, +) +from pip._internal.utils.typing import MYPY_CHECK_RUNNING +from pip._internal.utils.ui import DownloadProgressProvider + +if MYPY_CHECK_RUNNING: + from typing import Iterable, Optional + + from pip._vendor.requests.models import Response + + from pip._internal.models.link import Link + from pip._internal.network.session import PipSession + +logger = logging.getLogger(__name__) + + +def _get_http_response_size(resp): + # type: (Response) -> Optional[int] + try: + return int(resp.headers['content-length']) + except (ValueError, KeyError, TypeError): + return None + + +def _prepare_download( + resp, # type: Response + link, # type: Link + progress_bar # type: str +): + # type: (...) -> Iterable[bytes] + total_length = _get_http_response_size(resp) + + if link.netloc == PyPI.file_storage_domain: + url = link.show_url + else: + url = link.url_without_fragment + + logged_url = redact_auth_from_url(url) + + if total_length: + logged_url = '{} ({})'.format(logged_url, format_size(total_length)) + + if is_from_cache(resp): + logger.info("Using cached %s", logged_url) + else: + logger.info("Downloading %s", logged_url) + + if logger.getEffectiveLevel() > logging.INFO: + show_progress = False + elif is_from_cache(resp): + show_progress = False + elif not total_length: + show_progress = True + elif total_length > (40 * 1000): + show_progress = True + else: + show_progress = False + + chunks = response_chunks(resp, CONTENT_CHUNK_SIZE) + + if not show_progress: + return chunks + + return DownloadProgressProvider( + progress_bar, max=total_length + )(chunks) + + +def sanitize_content_filename(filename): + # type: (str) -> str + """ + Sanitize the "filename" value from a Content-Disposition header. + """ + return os.path.basename(filename) + + +def parse_content_disposition(content_disposition, default_filename): + # type: (str, str) -> str + """ + Parse the "filename" value from a Content-Disposition header, and + return the default filename if the result is empty. + """ + _type, params = cgi.parse_header(content_disposition) + filename = params.get('filename') + if filename: + # We need to sanitize the filename to prevent directory traversal + # in case the filename contains ".." path parts. + filename = sanitize_content_filename(filename) + return filename or default_filename + + +def _get_http_response_filename(resp, link): + # type: (Response, Link) -> str + """Get an ideal filename from the given HTTP response, falling back to + the link filename if not provided. + """ + filename = link.filename # fallback + # Have a look at the Content-Disposition header for a better guess + content_disposition = resp.headers.get('content-disposition') + if content_disposition: + filename = parse_content_disposition(content_disposition, filename) + ext = splitext(filename)[1] # type: Optional[str] + if not ext: + ext = mimetypes.guess_extension( + resp.headers.get('content-type', '') + ) + if ext: + filename += ext + if not ext and link.url != resp.url: + ext = os.path.splitext(resp.url)[1] + if ext: + filename += ext + return filename + + +def _http_get_download(session, link): + # type: (PipSession, Link) -> Response + target_url = link.url.split('#', 1)[0] + resp = session.get( + target_url, + # We use Accept-Encoding: identity here because requests + # defaults to accepting compressed responses. This breaks in + # a variety of ways depending on how the server is configured. + # - Some servers will notice that the file isn't a compressible + # file and will leave the file alone and with an empty + # Content-Encoding + # - Some servers will notice that the file is already + # compressed and will leave the file alone and will add a + # Content-Encoding: gzip header + # - Some servers won't notice anything at all and will take + # a file that's already been compressed and compress it again + # and set the Content-Encoding: gzip header + # By setting this to request only the identity encoding We're + # hoping to eliminate the third case. Hopefully there does not + # exist a server which when given a file will notice it is + # already compressed and that you're not asking for a + # compressed file and will then decompress it before sending + # because if that's the case I don't think it'll ever be + # possible to make this work. + headers={"Accept-Encoding": "identity"}, + stream=True, + ) + resp.raise_for_status() + return resp + + +class Download(object): + def __init__( + self, + response, # type: Response + filename, # type: str + chunks, # type: Iterable[bytes] + ): + # type: (...) -> None + self.response = response + self.filename = filename + self.chunks = chunks + + +class Downloader(object): + def __init__( + self, + session, # type: PipSession + progress_bar, # type: str + ): + # type: (...) -> None + self._session = session + self._progress_bar = progress_bar + + def __call__(self, link): + # type: (Link) -> Download + try: + resp = _http_get_download(self._session, link) + except requests.HTTPError as e: + logger.critical( + "HTTP error %s while getting %s", e.response.status_code, link + ) + raise + + return Download( + resp, + _get_http_response_filename(resp, link), + _prepare_download(resp, link, self._progress_bar), + ) diff --git a/env/lib/python3.8/site-packages/pip/_internal/network/session.py b/env/lib/python3.8/site-packages/pip/_internal/network/session.py new file mode 100644 index 000000000..f5eb15ef2 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/network/session.py @@ -0,0 +1,405 @@ +"""PipSession and supporting code, containing all pip-specific +network request configuration and behavior. +""" + +# The following comment should be removed at some point in the future. +# mypy: disallow-untyped-defs=False + +import email.utils +import json +import logging +import mimetypes +import os +import platform +import sys +import warnings + +from pip._vendor import requests, six, urllib3 +from pip._vendor.cachecontrol import CacheControlAdapter +from pip._vendor.requests.adapters import BaseAdapter, HTTPAdapter +from pip._vendor.requests.models import Response +from pip._vendor.requests.structures import CaseInsensitiveDict +from pip._vendor.six.moves.urllib import parse as urllib_parse +from pip._vendor.urllib3.exceptions import InsecureRequestWarning + +from pip import __version__ +from pip._internal.network.auth import MultiDomainBasicAuth +from pip._internal.network.cache import SafeFileCache +# Import ssl from compat so the initial import occurs in only one place. +from pip._internal.utils.compat import has_tls, ipaddress +from pip._internal.utils.glibc import libc_ver +from pip._internal.utils.misc import ( + build_url_from_netloc, + get_installed_version, + parse_netloc, +) +from pip._internal.utils.typing import MYPY_CHECK_RUNNING +from pip._internal.utils.urls import url_to_path + +if MYPY_CHECK_RUNNING: + from typing import ( + Iterator, List, Optional, Tuple, Union, + ) + + from pip._internal.models.link import Link + + SecureOrigin = Tuple[str, str, Optional[Union[int, str]]] + + +logger = logging.getLogger(__name__) + + +# Ignore warning raised when using --trusted-host. +warnings.filterwarnings("ignore", category=InsecureRequestWarning) + + +SECURE_ORIGINS = [ + # protocol, hostname, port + # Taken from Chrome's list of secure origins (See: http://bit.ly/1qrySKC) + ("https", "*", "*"), + ("*", "localhost", "*"), + ("*", "127.0.0.0/8", "*"), + ("*", "::1/128", "*"), + ("file", "*", None), + # ssh is always secure. + ("ssh", "*", "*"), +] # type: List[SecureOrigin] + + +# These are environment variables present when running under various +# CI systems. For each variable, some CI systems that use the variable +# are indicated. The collection was chosen so that for each of a number +# of popular systems, at least one of the environment variables is used. +# This list is used to provide some indication of and lower bound for +# CI traffic to PyPI. Thus, it is okay if the list is not comprehensive. +# For more background, see: https://github.com/pypa/pip/issues/5499 +CI_ENVIRONMENT_VARIABLES = ( + # Azure Pipelines + 'BUILD_BUILDID', + # Jenkins + 'BUILD_ID', + # AppVeyor, CircleCI, Codeship, Gitlab CI, Shippable, Travis CI + 'CI', + # Explicit environment variable. + 'PIP_IS_CI', +) + + +def looks_like_ci(): + # type: () -> bool + """ + Return whether it looks like pip is running under CI. + """ + # We don't use the method of checking for a tty (e.g. using isatty()) + # because some CI systems mimic a tty (e.g. Travis CI). Thus that + # method doesn't provide definitive information in either direction. + return any(name in os.environ for name in CI_ENVIRONMENT_VARIABLES) + + +def user_agent(): + """ + Return a string representing the user agent. + """ + data = { + "installer": {"name": "pip", "version": __version__}, + "python": platform.python_version(), + "implementation": { + "name": platform.python_implementation(), + }, + } + + if data["implementation"]["name"] == 'CPython': + data["implementation"]["version"] = platform.python_version() + elif data["implementation"]["name"] == 'PyPy': + if sys.pypy_version_info.releaselevel == 'final': + pypy_version_info = sys.pypy_version_info[:3] + else: + pypy_version_info = sys.pypy_version_info + data["implementation"]["version"] = ".".join( + [str(x) for x in pypy_version_info] + ) + elif data["implementation"]["name"] == 'Jython': + # Complete Guess + data["implementation"]["version"] = platform.python_version() + elif data["implementation"]["name"] == 'IronPython': + # Complete Guess + data["implementation"]["version"] = platform.python_version() + + if sys.platform.startswith("linux"): + from pip._vendor import distro + distro_infos = dict(filter( + lambda x: x[1], + zip(["name", "version", "id"], distro.linux_distribution()), + )) + libc = dict(filter( + lambda x: x[1], + zip(["lib", "version"], libc_ver()), + )) + if libc: + distro_infos["libc"] = libc + if distro_infos: + data["distro"] = distro_infos + + if sys.platform.startswith("darwin") and platform.mac_ver()[0]: + data["distro"] = {"name": "macOS", "version": platform.mac_ver()[0]} + + if platform.system(): + data.setdefault("system", {})["name"] = platform.system() + + if platform.release(): + data.setdefault("system", {})["release"] = platform.release() + + if platform.machine(): + data["cpu"] = platform.machine() + + if has_tls(): + import _ssl as ssl + data["openssl_version"] = ssl.OPENSSL_VERSION + + setuptools_version = get_installed_version("setuptools") + if setuptools_version is not None: + data["setuptools_version"] = setuptools_version + + # Use None rather than False so as not to give the impression that + # pip knows it is not being run under CI. Rather, it is a null or + # inconclusive result. Also, we include some value rather than no + # value to make it easier to know that the check has been run. + data["ci"] = True if looks_like_ci() else None + + user_data = os.environ.get("PIP_USER_AGENT_USER_DATA") + if user_data is not None: + data["user_data"] = user_data + + return "{data[installer][name]}/{data[installer][version]} {json}".format( + data=data, + json=json.dumps(data, separators=(",", ":"), sort_keys=True), + ) + + +class LocalFSAdapter(BaseAdapter): + + def send(self, request, stream=None, timeout=None, verify=None, cert=None, + proxies=None): + pathname = url_to_path(request.url) + + resp = Response() + resp.status_code = 200 + resp.url = request.url + + try: + stats = os.stat(pathname) + except OSError as exc: + resp.status_code = 404 + resp.raw = exc + else: + modified = email.utils.formatdate(stats.st_mtime, usegmt=True) + content_type = mimetypes.guess_type(pathname)[0] or "text/plain" + resp.headers = CaseInsensitiveDict({ + "Content-Type": content_type, + "Content-Length": stats.st_size, + "Last-Modified": modified, + }) + + resp.raw = open(pathname, "rb") + resp.close = resp.raw.close + + return resp + + def close(self): + pass + + +class InsecureHTTPAdapter(HTTPAdapter): + + def cert_verify(self, conn, url, verify, cert): + super(InsecureHTTPAdapter, self).cert_verify( + conn=conn, url=url, verify=False, cert=cert + ) + + +class PipSession(requests.Session): + + timeout = None # type: Optional[int] + + def __init__(self, *args, **kwargs): + """ + :param trusted_hosts: Domains not to emit warnings for when not using + HTTPS. + """ + retries = kwargs.pop("retries", 0) + cache = kwargs.pop("cache", None) + trusted_hosts = kwargs.pop("trusted_hosts", []) # type: List[str] + index_urls = kwargs.pop("index_urls", None) + + super(PipSession, self).__init__(*args, **kwargs) + + # Namespace the attribute with "pip_" just in case to prevent + # possible conflicts with the base class. + self.pip_trusted_origins = [] # type: List[Tuple[str, Optional[int]]] + + # Attach our User Agent to the request + self.headers["User-Agent"] = user_agent() + + # Attach our Authentication handler to the session + self.auth = MultiDomainBasicAuth(index_urls=index_urls) + + # Create our urllib3.Retry instance which will allow us to customize + # how we handle retries. + retries = urllib3.Retry( + # Set the total number of retries that a particular request can + # have. + total=retries, + + # A 503 error from PyPI typically means that the Fastly -> Origin + # connection got interrupted in some way. A 503 error in general + # is typically considered a transient error so we'll go ahead and + # retry it. + # A 500 may indicate transient error in Amazon S3 + # A 520 or 527 - may indicate transient error in CloudFlare + status_forcelist=[500, 503, 520, 527], + + # Add a small amount of back off between failed requests in + # order to prevent hammering the service. + backoff_factor=0.25, + ) + + # We want to _only_ cache responses on securely fetched origins. We do + # this because we can't validate the response of an insecurely fetched + # origin, and we don't want someone to be able to poison the cache and + # require manual eviction from the cache to fix it. + if cache: + secure_adapter = CacheControlAdapter( + cache=SafeFileCache(cache), + max_retries=retries, + ) + else: + secure_adapter = HTTPAdapter(max_retries=retries) + + # Our Insecure HTTPAdapter disables HTTPS validation. It does not + # support caching (see above) so we'll use it for all http:// URLs as + # well as any https:// host that we've marked as ignoring TLS errors + # for. + insecure_adapter = InsecureHTTPAdapter(max_retries=retries) + # Save this for later use in add_insecure_host(). + self._insecure_adapter = insecure_adapter + + self.mount("https://", secure_adapter) + self.mount("http://", insecure_adapter) + + # Enable file:// urls + self.mount("file://", LocalFSAdapter()) + + for host in trusted_hosts: + self.add_trusted_host(host, suppress_logging=True) + + def add_trusted_host(self, host, source=None, suppress_logging=False): + # type: (str, Optional[str], bool) -> None + """ + :param host: It is okay to provide a host that has previously been + added. + :param source: An optional source string, for logging where the host + string came from. + """ + if not suppress_logging: + msg = 'adding trusted host: {!r}'.format(host) + if source is not None: + msg += ' (from {})'.format(source) + logger.info(msg) + + host_port = parse_netloc(host) + if host_port not in self.pip_trusted_origins: + self.pip_trusted_origins.append(host_port) + + self.mount(build_url_from_netloc(host) + '/', self._insecure_adapter) + if not host_port[1]: + # Mount wildcard ports for the same host. + self.mount( + build_url_from_netloc(host) + ':', + self._insecure_adapter + ) + + def iter_secure_origins(self): + # type: () -> Iterator[SecureOrigin] + for secure_origin in SECURE_ORIGINS: + yield secure_origin + for host, port in self.pip_trusted_origins: + yield ('*', host, '*' if port is None else port) + + def is_secure_origin(self, location): + # type: (Link) -> bool + # Determine if this url used a secure transport mechanism + parsed = urllib_parse.urlparse(str(location)) + origin_protocol, origin_host, origin_port = ( + parsed.scheme, parsed.hostname, parsed.port, + ) + + # The protocol to use to see if the protocol matches. + # Don't count the repository type as part of the protocol: in + # cases such as "git+ssh", only use "ssh". (I.e., Only verify against + # the last scheme.) + origin_protocol = origin_protocol.rsplit('+', 1)[-1] + + # Determine if our origin is a secure origin by looking through our + # hardcoded list of secure origins, as well as any additional ones + # configured on this PackageFinder instance. + for secure_origin in self.iter_secure_origins(): + secure_protocol, secure_host, secure_port = secure_origin + if origin_protocol != secure_protocol and secure_protocol != "*": + continue + + try: + addr = ipaddress.ip_address( + None + if origin_host is None + else six.ensure_text(origin_host) + ) + network = ipaddress.ip_network( + six.ensure_text(secure_host) + ) + except ValueError: + # We don't have both a valid address or a valid network, so + # we'll check this origin against hostnames. + if ( + origin_host and + origin_host.lower() != secure_host.lower() and + secure_host != "*" + ): + continue + else: + # We have a valid address and network, so see if the address + # is contained within the network. + if addr not in network: + continue + + # Check to see if the port matches. + if ( + origin_port != secure_port and + secure_port != "*" and + secure_port is not None + ): + continue + + # If we've gotten here, then this origin matches the current + # secure origin and we should return True + return True + + # If we've gotten to this point, then the origin isn't secure and we + # will not accept it as a valid location to search. We will however + # log a warning that we are ignoring it. + logger.warning( + "The repository located at %s is not a trusted or secure host and " + "is being ignored. If this repository is available via HTTPS we " + "recommend you use HTTPS instead, otherwise you may silence " + "this warning and allow it anyway with '--trusted-host %s'.", + origin_host, + origin_host, + ) + + return False + + def request(self, method, url, *args, **kwargs): + # Allow setting a default timeout on a session + kwargs.setdefault("timeout", self.timeout) + + # Dispatch the actual request + return super(PipSession, self).request(method, url, *args, **kwargs) diff --git a/env/lib/python3.8/site-packages/pip/_internal/network/utils.py b/env/lib/python3.8/site-packages/pip/_internal/network/utils.py new file mode 100644 index 000000000..a19050b0f --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/network/utils.py @@ -0,0 +1,48 @@ +from pip._vendor.requests.models import CONTENT_CHUNK_SIZE, Response + +from pip._internal.utils.typing import MYPY_CHECK_RUNNING + +if MYPY_CHECK_RUNNING: + from typing import Iterator + + +def response_chunks(response, chunk_size=CONTENT_CHUNK_SIZE): + # type: (Response, int) -> Iterator[bytes] + """Given a requests Response, provide the data chunks. + """ + try: + # Special case for urllib3. + for chunk in response.raw.stream( + chunk_size, + # We use decode_content=False here because we don't + # want urllib3 to mess with the raw bytes we get + # from the server. If we decompress inside of + # urllib3 then we cannot verify the checksum + # because the checksum will be of the compressed + # file. This breakage will only occur if the + # server adds a Content-Encoding header, which + # depends on how the server was configured: + # - Some servers will notice that the file isn't a + # compressible file and will leave the file alone + # and with an empty Content-Encoding + # - Some servers will notice that the file is + # already compressed and will leave the file + # alone and will add a Content-Encoding: gzip + # header + # - Some servers won't notice anything at all and + # will take a file that's already been compressed + # and compress it again and set the + # Content-Encoding: gzip header + # + # By setting this not to decode automatically we + # hope to eliminate problems with the second case. + decode_content=False, + ): + yield chunk + except AttributeError: + # Standard file-like object. + while True: + chunk = response.raw.read(chunk_size) + if not chunk: + break + yield chunk diff --git a/env/lib/python3.8/site-packages/pip/_internal/network/xmlrpc.py b/env/lib/python3.8/site-packages/pip/_internal/network/xmlrpc.py new file mode 100644 index 000000000..121edd930 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/network/xmlrpc.py @@ -0,0 +1,44 @@ +"""xmlrpclib.Transport implementation +""" + +# The following comment should be removed at some point in the future. +# mypy: disallow-untyped-defs=False + +import logging + +from pip._vendor import requests +# NOTE: XMLRPC Client is not annotated in typeshed as on 2017-07-17, which is +# why we ignore the type on this import +from pip._vendor.six.moves import xmlrpc_client # type: ignore +from pip._vendor.six.moves.urllib import parse as urllib_parse + +logger = logging.getLogger(__name__) + + +class PipXmlrpcTransport(xmlrpc_client.Transport): + """Provide a `xmlrpclib.Transport` implementation via a `PipSession` + object. + """ + + def __init__(self, index_url, session, use_datetime=False): + xmlrpc_client.Transport.__init__(self, use_datetime) + index_parts = urllib_parse.urlparse(index_url) + self._scheme = index_parts.scheme + self._session = session + + def request(self, host, handler, request_body, verbose=False): + parts = (self._scheme, host, handler, None, None, None) + url = urllib_parse.urlunparse(parts) + try: + headers = {'Content-Type': 'text/xml'} + response = self._session.post(url, data=request_body, + headers=headers, stream=True) + response.raise_for_status() + self.verbose = verbose + return self.parse_response(response.raw) + except requests.HTTPError as exc: + logger.critical( + "HTTP error %s while getting %s", + exc.response.status_code, url, + ) + raise diff --git a/env/lib/python3.8/site-packages/pip/_internal/operations/__init__.py b/env/lib/python3.8/site-packages/pip/_internal/operations/__init__.py new file mode 100644 index 000000000..e69de29bb diff --git a/env/lib/python3.8/site-packages/pip/_internal/operations/build/__init__.py b/env/lib/python3.8/site-packages/pip/_internal/operations/build/__init__.py new file mode 100644 index 000000000..e69de29bb diff --git a/env/lib/python3.8/site-packages/pip/_internal/operations/build/metadata.py b/env/lib/python3.8/site-packages/pip/_internal/operations/build/metadata.py new file mode 100644 index 000000000..b13fbdef9 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/operations/build/metadata.py @@ -0,0 +1,40 @@ +"""Metadata generation logic for source distributions. +""" + +import logging +import os + +from pip._internal.utils.subprocess import runner_with_spinner_message +from pip._internal.utils.temp_dir import TempDirectory +from pip._internal.utils.typing import MYPY_CHECK_RUNNING + +if MYPY_CHECK_RUNNING: + from pip._internal.build_env import BuildEnvironment + from pip._vendor.pep517.wrappers import Pep517HookCaller + +logger = logging.getLogger(__name__) + + +def generate_metadata(build_env, backend): + # type: (BuildEnvironment, Pep517HookCaller) -> str + """Generate metadata using mechanisms described in PEP 517. + + Returns the generated metadata directory. + """ + metadata_tmpdir = TempDirectory( + kind="modern-metadata", globally_managed=True + ) + + metadata_dir = metadata_tmpdir.path + + with build_env: + # Note that Pep517HookCaller implements a fallback for + # prepare_metadata_for_build_wheel, so we don't have to + # consider the possibility that this hook doesn't exist. + runner = runner_with_spinner_message("Preparing wheel metadata") + with backend.subprocess_runner(runner): + distinfo_dir = backend.prepare_metadata_for_build_wheel( + metadata_dir + ) + + return os.path.join(metadata_dir, distinfo_dir) diff --git a/env/lib/python3.8/site-packages/pip/_internal/operations/build/metadata_legacy.py b/env/lib/python3.8/site-packages/pip/_internal/operations/build/metadata_legacy.py new file mode 100644 index 000000000..b6813f89b --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/operations/build/metadata_legacy.py @@ -0,0 +1,122 @@ +"""Metadata generation logic for legacy source distributions. +""" + +import logging +import os + +from pip._internal.exceptions import InstallationError +from pip._internal.utils.misc import ensure_dir +from pip._internal.utils.setuptools_build import make_setuptools_egg_info_args +from pip._internal.utils.subprocess import call_subprocess +from pip._internal.utils.typing import MYPY_CHECK_RUNNING +from pip._internal.vcs import vcs + +if MYPY_CHECK_RUNNING: + from typing import List, Optional + + from pip._internal.build_env import BuildEnvironment + +logger = logging.getLogger(__name__) + + +def _find_egg_info(source_directory, is_editable): + # type: (str, bool) -> str + """Find an .egg-info in `source_directory`, based on `is_editable`. + """ + + def looks_like_virtual_env(path): + # type: (str) -> bool + return ( + os.path.lexists(os.path.join(path, 'bin', 'python')) or + os.path.exists(os.path.join(path, 'Scripts', 'Python.exe')) + ) + + def locate_editable_egg_info(base): + # type: (str) -> List[str] + candidates = [] # type: List[str] + for root, dirs, files in os.walk(base): + for dir_ in vcs.dirnames: + if dir_ in dirs: + dirs.remove(dir_) + # Iterate over a copy of ``dirs``, since mutating + # a list while iterating over it can cause trouble. + # (See https://github.com/pypa/pip/pull/462.) + for dir_ in list(dirs): + if looks_like_virtual_env(os.path.join(root, dir_)): + dirs.remove(dir_) + # Also don't search through tests + elif dir_ == 'test' or dir_ == 'tests': + dirs.remove(dir_) + candidates.extend(os.path.join(root, dir_) for dir_ in dirs) + return [f for f in candidates if f.endswith('.egg-info')] + + def depth_of_directory(dir_): + # type: (str) -> int + return ( + dir_.count(os.path.sep) + + (os.path.altsep and dir_.count(os.path.altsep) or 0) + ) + + base = source_directory + if is_editable: + filenames = locate_editable_egg_info(base) + else: + base = os.path.join(base, 'pip-egg-info') + filenames = os.listdir(base) + + if not filenames: + raise InstallationError( + "Files/directories not found in {}".format(base) + ) + + # If we have more than one match, we pick the toplevel one. This + # can easily be the case if there is a dist folder which contains + # an extracted tarball for testing purposes. + if len(filenames) > 1: + filenames.sort(key=depth_of_directory) + + return os.path.join(base, filenames[0]) + + +def generate_metadata( + build_env, # type: BuildEnvironment + setup_py_path, # type: str + source_dir, # type: str + editable, # type: bool + isolated, # type: bool + details, # type: str +): + # type: (...) -> str + """Generate metadata using setup.py-based defacto mechanisms. + + Returns the generated metadata directory. + """ + logger.debug( + 'Running setup.py (path:%s) egg_info for package %s', + setup_py_path, details, + ) + + egg_info_dir = None # type: Optional[str] + # For non-editable installs, don't put the .egg-info files at the root, + # to avoid confusion due to the source code being considered an installed + # egg. + if not editable: + egg_info_dir = os.path.join(source_dir, 'pip-egg-info') + # setuptools complains if the target directory does not exist. + ensure_dir(egg_info_dir) + + args = make_setuptools_egg_info_args( + setup_py_path, + egg_info_dir=egg_info_dir, + no_user_config=isolated, + ) + + with build_env: + call_subprocess( + args, + cwd=source_dir, + command_desc='python setup.py egg_info', + ) + + # Return the .egg-info directory. + return _find_egg_info(source_dir, editable) diff --git a/env/lib/python3.8/site-packages/pip/_internal/operations/build/wheel.py b/env/lib/python3.8/site-packages/pip/_internal/operations/build/wheel.py new file mode 100644 index 000000000..1266ce05c --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/operations/build/wheel.py @@ -0,0 +1,46 @@ +import logging +import os + +from pip._internal.utils.subprocess import runner_with_spinner_message +from pip._internal.utils.typing import MYPY_CHECK_RUNNING + +if MYPY_CHECK_RUNNING: + from typing import List, Optional + from pip._vendor.pep517.wrappers import Pep517HookCaller + +logger = logging.getLogger(__name__) + + +def build_wheel_pep517( + name, # type: str + backend, # type: Pep517HookCaller + metadata_directory, # type: str + build_options, # type: List[str] + tempd, # type: str +): + # type: (...) -> Optional[str] + """Build one InstallRequirement using the PEP 517 build process. + + Returns path to wheel if successfully built. Otherwise, returns None. + """ + assert metadata_directory is not None + if build_options: + # PEP 517 does not support --build-options + logger.error('Cannot build wheel for %s using PEP 517 when ' + '--build-option is present' % (name,)) + return None + try: + logger.debug('Destination directory: %s', tempd) + + runner = runner_with_spinner_message( + 'Building wheel for {} (PEP 517)'.format(name) + ) + with backend.subprocess_runner(runner): + wheel_name = backend.build_wheel( + tempd, + metadata_directory=metadata_directory, + ) + except Exception: + logger.error('Failed building wheel for %s', name) + return None + return os.path.join(tempd, wheel_name) diff --git a/env/lib/python3.8/site-packages/pip/_internal/operations/build/wheel_legacy.py b/env/lib/python3.8/site-packages/pip/_internal/operations/build/wheel_legacy.py new file mode 100644 index 000000000..3ebd9fe44 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/operations/build/wheel_legacy.py @@ -0,0 +1,115 @@ +import logging +import os.path + +from pip._internal.utils.setuptools_build import ( + make_setuptools_bdist_wheel_args, +) +from pip._internal.utils.subprocess import ( + LOG_DIVIDER, + call_subprocess, + format_command_args, +) +from pip._internal.utils.typing import MYPY_CHECK_RUNNING +from pip._internal.utils.ui import open_spinner + +if MYPY_CHECK_RUNNING: + from typing import List, Optional, Text + +logger = logging.getLogger(__name__) + + +def format_command_result( + command_args, # type: List[str] + command_output, # type: Text +): + # type: (...) -> str + """Format command information for logging.""" + command_desc = format_command_args(command_args) + text = 'Command arguments: {}\n'.format(command_desc) + + if not command_output: + text += 'Command output: None' + elif logger.getEffectiveLevel() > logging.DEBUG: + text += 'Command output: [use --verbose to show]' + else: + if not command_output.endswith('\n'): + command_output += '\n' + text += 'Command output:\n{}{}'.format(command_output, LOG_DIVIDER) + + return text + + +def get_legacy_build_wheel_path( + names, # type: List[str] + temp_dir, # type: str + name, # type: str + command_args, # type: List[str] + command_output, # type: Text +): + # type: (...) -> Optional[str] + """Return the path to the wheel in the temporary build directory.""" + # Sort for determinism. + names = sorted(names) + if not names: + msg = ( + 'Legacy build of wheel for {!r} created no files.\n' + ).format(name) + msg += format_command_result(command_args, command_output) + logger.warning(msg) + return None + + if len(names) > 1: + msg = ( + 'Legacy build of wheel for {!r} created more than one file.\n' + 'Filenames (choosing first): {}\n' + ).format(name, names) + msg += format_command_result(command_args, command_output) + logger.warning(msg) + + return os.path.join(temp_dir, names[0]) + + +def build_wheel_legacy( + name, # type: str + setup_py_path, # type: str + source_dir, # type: str + global_options, # type: List[str] + build_options, # type: List[str] + tempd, # type: str +): + # type: (...) -> Optional[str] + """Build one unpacked package using the "legacy" build process. + + Returns path to wheel if successfully built. Otherwise, returns None. + """ + wheel_args = make_setuptools_bdist_wheel_args( + setup_py_path, + global_options=global_options, + build_options=build_options, + destination_dir=tempd, + ) + + spin_message = 'Building wheel for %s (setup.py)' % (name,) + with open_spinner(spin_message) as spinner: + logger.debug('Destination directory: %s', tempd) + + try: + output = call_subprocess( + wheel_args, + cwd=source_dir, + spinner=spinner, + ) + except Exception: + spinner.finish("error") + logger.error('Failed building wheel for %s', name) + return None + + names = os.listdir(tempd) + wheel_path = get_legacy_build_wheel_path( + names=names, + temp_dir=tempd, + name=name, + command_args=wheel_args, + command_output=output, + ) + return wheel_path diff --git a/env/lib/python3.8/site-packages/pip/_internal/operations/check.py b/env/lib/python3.8/site-packages/pip/_internal/operations/check.py new file mode 100644 index 000000000..b85a12306 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/operations/check.py @@ -0,0 +1,163 @@ +"""Validation of dependencies of packages +""" + +# The following comment should be removed at some point in the future. +# mypy: strict-optional=False +# mypy: disallow-untyped-defs=False + +import logging +from collections import namedtuple + +from pip._vendor.packaging.utils import canonicalize_name +from pip._vendor.pkg_resources import RequirementParseError + +from pip._internal.distributions import ( + make_distribution_for_install_requirement, +) +from pip._internal.utils.misc import get_installed_distributions +from pip._internal.utils.typing import MYPY_CHECK_RUNNING + +logger = logging.getLogger(__name__) + +if MYPY_CHECK_RUNNING: + from pip._internal.req.req_install import InstallRequirement + from typing import ( + Any, Callable, Dict, Optional, Set, Tuple, List + ) + + # Shorthands + PackageSet = Dict[str, 'PackageDetails'] + Missing = Tuple[str, Any] + Conflicting = Tuple[str, str, Any] + + MissingDict = Dict[str, List[Missing]] + ConflictingDict = Dict[str, List[Conflicting]] + CheckResult = Tuple[MissingDict, ConflictingDict] + +PackageDetails = namedtuple('PackageDetails', ['version', 'requires']) + + +def create_package_set_from_installed(**kwargs): + # type: (**Any) -> Tuple[PackageSet, bool] + """Converts a list of distributions into a PackageSet. + """ + # Default to using all packages installed on the system + if kwargs == {}: + kwargs = {"local_only": False, "skip": ()} + + package_set = {} + problems = False + for dist in get_installed_distributions(**kwargs): + name = canonicalize_name(dist.project_name) + try: + package_set[name] = PackageDetails(dist.version, dist.requires()) + except RequirementParseError as e: + # Don't crash on broken metadata + logger.warning("Error parsing requirements for %s: %s", name, e) + problems = True + return package_set, problems + + +def check_package_set(package_set, should_ignore=None): + # type: (PackageSet, Optional[Callable[[str], bool]]) -> CheckResult + """Check if a package set is consistent + + If should_ignore is passed, it should be a callable that takes a + package name and returns a boolean. + """ + if should_ignore is None: + def should_ignore(name): + return False + + missing = {} + conflicting = {} + + for package_name in package_set: + # Info about dependencies of package_name + missing_deps = set() # type: Set[Missing] + conflicting_deps = set() # type: Set[Conflicting] + + if should_ignore(package_name): + continue + + for req in package_set[package_name].requires: + name = canonicalize_name(req.project_name) # type: str + + # Check if it's missing + if name not in package_set: + missed = True + if req.marker is not None: + missed = req.marker.evaluate() + if missed: + missing_deps.add((name, req)) + continue + + # Check if there's a conflict + version = package_set[name].version # type: str + if not req.specifier.contains(version, prereleases=True): + conflicting_deps.add((name, version, req)) + + if missing_deps: + missing[package_name] = sorted(missing_deps, key=str) + if conflicting_deps: + conflicting[package_name] = sorted(conflicting_deps, key=str) + + return missing, conflicting + + +def check_install_conflicts(to_install): + # type: (List[InstallRequirement]) -> Tuple[PackageSet, CheckResult] + """For checking if the dependency graph would be consistent after \ + installing given requirements + """ + # Start from the current state + package_set, _ = create_package_set_from_installed() + # Install packages + would_be_installed = _simulate_installation_of(to_install, package_set) + + # Only warn about directly-dependent packages; create a whitelist of them + whitelist = _create_whitelist(would_be_installed, package_set) + + return ( + package_set, + check_package_set( + package_set, should_ignore=lambda name: name not in whitelist + ) + ) + + +def _simulate_installation_of(to_install, package_set): + # type: (List[InstallRequirement], PackageSet) -> Set[str] + """Computes the version of packages after installing to_install. + """ + + # Keep track of packages that were installed + installed = set() + + # Modify it as installing requirement_set would (assuming no errors) + for inst_req in to_install: + abstract_dist = make_distribution_for_install_requirement(inst_req) + dist = abstract_dist.get_pkg_resources_distribution() + + name = canonicalize_name(dist.key) + package_set[name] = PackageDetails(dist.version, dist.requires()) + + installed.add(name) + + return installed + + +def _create_whitelist(would_be_installed, package_set): + # type: (Set[str], PackageSet) -> Set[str] + packages_affected = set(would_be_installed) + + for package_name in package_set: + if package_name in packages_affected: + continue + + for req in package_set[package_name].requires: + if canonicalize_name(req.name) in packages_affected: + packages_affected.add(package_name) + break + + return packages_affected diff --git a/env/lib/python3.8/site-packages/pip/_internal/operations/freeze.py b/env/lib/python3.8/site-packages/pip/_internal/operations/freeze.py new file mode 100644 index 000000000..36a5c339a --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/operations/freeze.py @@ -0,0 +1,265 @@ +# The following comment should be removed at some point in the future. +# mypy: strict-optional=False +# mypy: disallow-untyped-defs=False + +from __future__ import absolute_import + +import collections +import logging +import os +import re + +from pip._vendor import six +from pip._vendor.packaging.utils import canonicalize_name +from pip._vendor.pkg_resources import RequirementParseError + +from pip._internal.exceptions import BadCommand, InstallationError +from pip._internal.req.constructors import ( + install_req_from_editable, + install_req_from_line, +) +from pip._internal.req.req_file import COMMENT_RE +from pip._internal.utils.misc import ( + dist_is_editable, + get_installed_distributions, +) +from pip._internal.utils.typing import MYPY_CHECK_RUNNING + +if MYPY_CHECK_RUNNING: + from typing import ( + Iterator, Optional, List, Container, Set, Dict, Tuple, Iterable, Union + ) + from pip._internal.cache import WheelCache + from pip._vendor.pkg_resources import ( + Distribution, Requirement + ) + + RequirementInfo = Tuple[Optional[Union[str, Requirement]], bool, List[str]] + + +logger = logging.getLogger(__name__) + + +def freeze( + requirement=None, # type: Optional[List[str]] + find_links=None, # type: Optional[List[str]] + local_only=None, # type: Optional[bool] + user_only=None, # type: Optional[bool] + paths=None, # type: Optional[List[str]] + skip_regex=None, # type: Optional[str] + isolated=False, # type: bool + wheel_cache=None, # type: Optional[WheelCache] + exclude_editable=False, # type: bool + skip=() # type: Container[str] +): + # type: (...) -> Iterator[str] + find_links = find_links or [] + skip_match = None + + if skip_regex: + skip_match = re.compile(skip_regex).search + + for link in find_links: + yield '-f %s' % link + installations = {} # type: Dict[str, FrozenRequirement] + for dist in get_installed_distributions(local_only=local_only, + skip=(), + user_only=user_only, + paths=paths): + try: + req = FrozenRequirement.from_dist(dist) + except RequirementParseError as exc: + # We include dist rather than dist.project_name because the + # dist string includes more information, like the version and + # location. We also include the exception message to aid + # troubleshooting. + logger.warning( + 'Could not generate requirement for distribution %r: %s', + dist, exc + ) + continue + if exclude_editable and req.editable: + continue + installations[req.canonical_name] = req + + if requirement: + # the options that don't get turned into an InstallRequirement + # should only be emitted once, even if the same option is in multiple + # requirements files, so we need to keep track of what has been emitted + # so that we don't emit it again if it's seen again + emitted_options = set() # type: Set[str] + # keep track of which files a requirement is in so that we can + # give an accurate warning if a requirement appears multiple times. + req_files = collections.defaultdict(list) # type: Dict[str, List[str]] + for req_file_path in requirement: + with open(req_file_path) as req_file: + for line in req_file: + if (not line.strip() or + line.strip().startswith('#') or + (skip_match and skip_match(line)) or + line.startswith(( + '-r', '--requirement', + '-Z', '--always-unzip', + '-f', '--find-links', + '-i', '--index-url', + '--pre', + '--trusted-host', + '--process-dependency-links', + '--extra-index-url'))): + line = line.rstrip() + if line not in emitted_options: + emitted_options.add(line) + yield line + continue + + if line.startswith('-e') or line.startswith('--editable'): + if line.startswith('-e'): + line = line[2:].strip() + else: + line = line[len('--editable'):].strip().lstrip('=') + line_req = install_req_from_editable( + line, + isolated=isolated, + wheel_cache=wheel_cache, + ) + else: + line_req = install_req_from_line( + COMMENT_RE.sub('', line).strip(), + isolated=isolated, + wheel_cache=wheel_cache, + ) + + if not line_req.name: + logger.info( + "Skipping line in requirement file [%s] because " + "it's not clear what it would install: %s", + req_file_path, line.strip(), + ) + logger.info( + " (add #egg=PackageName to the URL to avoid" + " this warning)" + ) + else: + line_req_canonical_name = canonicalize_name( + line_req.name) + if line_req_canonical_name not in installations: + # either it's not installed, or it is installed + # but has been processed already + if not req_files[line_req.name]: + logger.warning( + "Requirement file [%s] contains %s, but " + "package %r is not installed", + req_file_path, + COMMENT_RE.sub('', line).strip(), + line_req.name + ) + else: + req_files[line_req.name].append(req_file_path) + else: + yield str(installations[ + line_req_canonical_name]).rstrip() + del installations[line_req_canonical_name] + req_files[line_req.name].append(req_file_path) + + # Warn about requirements that were included multiple times (in a + # single requirements file or in different requirements files). + for name, files in six.iteritems(req_files): + if len(files) > 1: + logger.warning("Requirement %s included multiple times [%s]", + name, ', '.join(sorted(set(files)))) + + yield( + '## The following requirements were added by ' + 'pip freeze:' + ) + for installation in sorted( + installations.values(), key=lambda x: x.name.lower()): + if installation.canonical_name not in skip: + yield str(installation).rstrip() + + +def get_requirement_info(dist): + # type: (Distribution) -> RequirementInfo + """ + Compute and return values (req, editable, comments) for use in + FrozenRequirement.from_dist(). + """ + if not dist_is_editable(dist): + return (None, False, []) + + location = os.path.normcase(os.path.abspath(dist.location)) + + from pip._internal.vcs import vcs, RemoteNotFoundError + vcs_backend = vcs.get_backend_for_dir(location) + + if vcs_backend is None: + req = dist.as_requirement() + logger.debug( + 'No VCS found for editable requirement "%s" in: %r', req, + location, + ) + comments = [ + '# Editable install with no version control ({})'.format(req) + ] + return (location, True, comments) + + try: + req = vcs_backend.get_src_requirement(location, dist.project_name) + except RemoteNotFoundError: + req = dist.as_requirement() + comments = [ + '# Editable {} install with no remote ({})'.format( + type(vcs_backend).__name__, req, + ) + ] + return (location, True, comments) + + except BadCommand: + logger.warning( + 'cannot determine version of editable source in %s ' + '(%s command not found in path)', + location, + vcs_backend.name, + ) + return (None, True, []) + + except InstallationError as exc: + logger.warning( + "Error when trying to get requirement for VCS system %s, " + "falling back to uneditable format", exc + ) + else: + if req is not None: + return (req, True, []) + + logger.warning( + 'Could not determine repository location of %s', location + ) + comments = ['## !! Could not determine repository location'] + + return (None, False, comments) + + +class FrozenRequirement(object): + def __init__(self, name, req, editable, comments=()): + # type: (str, Union[str, Requirement], bool, Iterable[str]) -> None + self.name = name + self.canonical_name = canonicalize_name(name) + self.req = req + self.editable = editable + self.comments = comments + + @classmethod + def from_dist(cls, dist): + # type: (Distribution) -> FrozenRequirement + req, editable, comments = get_requirement_info(dist) + if req is None: + req = dist.as_requirement() + + return cls(dist.project_name, req, editable, comments=comments) + + def __str__(self): + req = self.req + if self.editable: + req = '-e %s' % req + return '\n'.join(list(self.comments) + [str(req)]) + '\n' diff --git a/env/lib/python3.8/site-packages/pip/_internal/operations/install/__init__.py b/env/lib/python3.8/site-packages/pip/_internal/operations/install/__init__.py new file mode 100644 index 000000000..24d6a5dd3 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/operations/install/__init__.py @@ -0,0 +1,2 @@ +"""For modules related to installing packages. +""" diff --git a/env/lib/python3.8/site-packages/pip/_internal/operations/install/editable_legacy.py b/env/lib/python3.8/site-packages/pip/_internal/operations/install/editable_legacy.py new file mode 100644 index 000000000..a668a61dc --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/operations/install/editable_legacy.py @@ -0,0 +1,52 @@ +"""Legacy editable installation process, i.e. `setup.py develop`. +""" +import logging + +from pip._internal.utils.logging import indent_log +from pip._internal.utils.setuptools_build import make_setuptools_develop_args +from pip._internal.utils.subprocess import call_subprocess +from pip._internal.utils.typing import MYPY_CHECK_RUNNING + +if MYPY_CHECK_RUNNING: + from typing import List, Optional, Sequence + + from pip._internal.build_env import BuildEnvironment + + +logger = logging.getLogger(__name__) + + +def install_editable( + install_options, # type: List[str] + global_options, # type: Sequence[str] + prefix, # type: Optional[str] + home, # type: Optional[str] + use_user_site, # type: bool + name, # type: str + setup_py_path, # type: str + isolated, # type: bool + build_env, # type: BuildEnvironment + unpacked_source_directory, # type: str +): + # type: (...) -> None + """Install a package in editable mode. Most arguments are pass-through + to setuptools. + """ + logger.info('Running setup.py develop for %s', name) + + args = make_setuptools_develop_args( + setup_py_path, + global_options=global_options, + install_options=install_options, + no_user_config=isolated, + prefix=prefix, + home=home, + use_user_site=use_user_site, + ) + + with indent_log(): + with build_env: + call_subprocess( + args, + cwd=unpacked_source_directory, + ) diff --git a/env/lib/python3.8/site-packages/pip/_internal/operations/install/legacy.py b/env/lib/python3.8/site-packages/pip/_internal/operations/install/legacy.py new file mode 100644 index 000000000..2d4adc4f6 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/operations/install/legacy.py @@ -0,0 +1,129 @@ +"""Legacy installation process, i.e. `setup.py install`. +""" + +import logging +import os +from distutils.util import change_root + +from pip._internal.utils.deprecation import deprecated +from pip._internal.utils.logging import indent_log +from pip._internal.utils.misc import ensure_dir +from pip._internal.utils.setuptools_build import make_setuptools_install_args +from pip._internal.utils.subprocess import runner_with_spinner_message +from pip._internal.utils.temp_dir import TempDirectory +from pip._internal.utils.typing import MYPY_CHECK_RUNNING + +if MYPY_CHECK_RUNNING: + from typing import List, Optional, Sequence + + from pip._internal.models.scheme import Scheme + from pip._internal.req.req_install import InstallRequirement + + +logger = logging.getLogger(__name__) + + +def install( + install_req, # type: InstallRequirement + install_options, # type: List[str] + global_options, # type: Sequence[str] + root, # type: Optional[str] + home, # type: Optional[str] + prefix, # type: Optional[str] + use_user_site, # type: bool + pycompile, # type: bool + scheme, # type: Scheme +): + # type: (...) -> None + # Extend the list of global and install options passed on to + # the setup.py call with the ones from the requirements file. + # Options specified in requirements file override those + # specified on the command line, since the last option given + # to setup.py is the one that is used. + global_options = list(global_options) + \ + install_req.options.get('global_options', []) + install_options = list(install_options) + \ + install_req.options.get('install_options', []) + + header_dir = scheme.headers + + with TempDirectory(kind="record") as temp_dir: + record_filename = os.path.join(temp_dir.path, 'install-record.txt') + install_args = make_setuptools_install_args( + install_req.setup_py_path, + global_options=global_options, + install_options=install_options, + record_filename=record_filename, + root=root, + prefix=prefix, + header_dir=header_dir, + home=home, + use_user_site=use_user_site, + no_user_config=install_req.isolated, + pycompile=pycompile, + ) + + runner = runner_with_spinner_message( + "Running setup.py install for {}".format(install_req.name) + ) + with indent_log(), install_req.build_env: + runner( + cmd=install_args, + cwd=install_req.unpacked_source_directory, + ) + + if not os.path.exists(record_filename): + logger.debug('Record file %s not found', record_filename) + return + install_req.install_succeeded = True + + # We intentionally do not use any encoding to read the file because + # setuptools writes the file using distutils.file_util.write_file, + # which does not specify an encoding. + with open(record_filename) as f: + record_lines = f.read().splitlines() + + def prepend_root(path): + # type: (str) -> str + if root is None or not os.path.isabs(path): + return path + else: + return change_root(root, path) + + for line in record_lines: + directory = os.path.dirname(line) + if directory.endswith('.egg-info'): + egg_info_dir = prepend_root(directory) + break + else: + deprecated( + reason=( + "{} did not indicate that it installed an " + ".egg-info directory. Only setup.py projects " + "generating .egg-info directories are supported." + ).format(install_req), + replacement=( + "for maintainers: updating the setup.py of {0}. " + "For users: contact the maintainers of {0} to let " + "them know to update their setup.py.".format( + install_req.name + ) + ), + gone_in="20.2", + issue=6998, + ) + # FIXME: put the record somewhere + return + new_lines = [] + for line in record_lines: + filename = line.strip() + if os.path.isdir(filename): + filename += os.path.sep + new_lines.append( + os.path.relpath(prepend_root(filename), egg_info_dir) + ) + new_lines.sort() + ensure_dir(egg_info_dir) + inst_files_path = os.path.join(egg_info_dir, 'installed-files.txt') + with open(inst_files_path, 'w') as f: + f.write('\n'.join(new_lines) + '\n') diff --git a/env/lib/python3.8/site-packages/pip/_internal/operations/install/wheel.py b/env/lib/python3.8/site-packages/pip/_internal/operations/install/wheel.py new file mode 100644 index 000000000..aac975c3a --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/operations/install/wheel.py @@ -0,0 +1,615 @@ +"""Support for installing and building the "wheel" binary package format. +""" + +# The following comment should be removed at some point in the future. +# mypy: strict-optional=False + +from __future__ import absolute_import + +import collections +import compileall +import csv +import logging +import os.path +import re +import shutil +import stat +import sys +import warnings +from base64 import urlsafe_b64encode +from zipfile import ZipFile + +from pip._vendor import pkg_resources +from pip._vendor.distlib.scripts import ScriptMaker +from pip._vendor.distlib.util import get_export_entry +from pip._vendor.six import StringIO + +from pip._internal.exceptions import InstallationError +from pip._internal.locations import get_major_minor_version +from pip._internal.utils.misc import captured_stdout, ensure_dir, hash_file +from pip._internal.utils.temp_dir import TempDirectory +from pip._internal.utils.typing import MYPY_CHECK_RUNNING +from pip._internal.utils.unpacking import unpack_file +from pip._internal.utils.wheel import parse_wheel + +if MYPY_CHECK_RUNNING: + from email.message import Message + from typing import ( + Dict, List, Optional, Sequence, Tuple, IO, Text, Any, + Iterable, Callable, Set, + ) + + from pip._internal.models.scheme import Scheme + + InstalledCSVRow = Tuple[str, ...] + + +logger = logging.getLogger(__name__) + + +def normpath(src, p): + # type: (str, str) -> str + return os.path.relpath(src, p).replace(os.path.sep, '/') + + +def rehash(path, blocksize=1 << 20): + # type: (str, int) -> Tuple[str, str] + """Return (encoded_digest, length) for path using hashlib.sha256()""" + h, length = hash_file(path, blocksize) + digest = 'sha256=' + urlsafe_b64encode( + h.digest() + ).decode('latin1').rstrip('=') + # unicode/str python2 issues + return (digest, str(length)) # type: ignore + + +def open_for_csv(name, mode): + # type: (str, Text) -> IO[Any] + if sys.version_info[0] < 3: + nl = {} # type: Dict[str, Any] + bin = 'b' + else: + nl = {'newline': ''} # type: Dict[str, Any] + bin = '' + return open(name, mode + bin, **nl) + + +def fix_script(path): + # type: (str) -> Optional[bool] + """Replace #!python with #!/path/to/python + Return True if file was changed. + """ + # XXX RECORD hashes will need to be updated + if os.path.isfile(path): + with open(path, 'rb') as script: + firstline = script.readline() + if not firstline.startswith(b'#!python'): + return False + exename = sys.executable.encode(sys.getfilesystemencoding()) + firstline = b'#!' + exename + os.linesep.encode("ascii") + rest = script.read() + with open(path, 'wb') as script: + script.write(firstline) + script.write(rest) + return True + return None + + +def wheel_root_is_purelib(metadata): + # type: (Message) -> bool + return metadata.get("Root-Is-Purelib", "").lower() == "true" + + +def get_entrypoints(filename): + # type: (str) -> Tuple[Dict[str, str], Dict[str, str]] + if not os.path.exists(filename): + return {}, {} + + # This is done because you can pass a string to entry_points wrappers which + # means that they may or may not be valid INI files. The attempt here is to + # strip leading and trailing whitespace in order to make them valid INI + # files. + with open(filename) as fp: + data = StringIO() + for line in fp: + data.write(line.strip()) + data.write("\n") + data.seek(0) + + # get the entry points and then the script names + entry_points = pkg_resources.EntryPoint.parse_map(data) + console = entry_points.get('console_scripts', {}) + gui = entry_points.get('gui_scripts', {}) + + def _split_ep(s): + # type: (pkg_resources.EntryPoint) -> Tuple[str, str] + """get the string representation of EntryPoint, + remove space and split on '=' + """ + split_parts = str(s).replace(" ", "").split("=") + return split_parts[0], split_parts[1] + + # convert the EntryPoint objects into strings with module:function + console = dict(_split_ep(v) for v in console.values()) + gui = dict(_split_ep(v) for v in gui.values()) + return console, gui + + +def message_about_scripts_not_on_PATH(scripts): + # type: (Sequence[str]) -> Optional[str] + """Determine if any scripts are not on PATH and format a warning. + Returns a warning message if one or more scripts are not on PATH, + otherwise None. + """ + if not scripts: + return None + + # Group scripts by the path they were installed in + grouped_by_dir = collections.defaultdict(set) # type: Dict[str, Set[str]] + for destfile in scripts: + parent_dir = os.path.dirname(destfile) + script_name = os.path.basename(destfile) + grouped_by_dir[parent_dir].add(script_name) + + # We don't want to warn for directories that are on PATH. + not_warn_dirs = [ + os.path.normcase(i).rstrip(os.sep) for i in + os.environ.get("PATH", "").split(os.pathsep) + ] + # If an executable sits with sys.executable, we don't warn for it. + # This covers the case of venv invocations without activating the venv. + not_warn_dirs.append(os.path.normcase(os.path.dirname(sys.executable))) + warn_for = { + parent_dir: scripts for parent_dir, scripts in grouped_by_dir.items() + if os.path.normcase(parent_dir) not in not_warn_dirs + } # type: Dict[str, Set[str]] + if not warn_for: + return None + + # Format a message + msg_lines = [] + for parent_dir, dir_scripts in warn_for.items(): + sorted_scripts = sorted(dir_scripts) # type: List[str] + if len(sorted_scripts) == 1: + start_text = "script {} is".format(sorted_scripts[0]) + else: + start_text = "scripts {} are".format( + ", ".join(sorted_scripts[:-1]) + " and " + sorted_scripts[-1] + ) + + msg_lines.append( + "The {} installed in '{}' which is not on PATH." + .format(start_text, parent_dir) + ) + + last_line_fmt = ( + "Consider adding {} to PATH or, if you prefer " + "to suppress this warning, use --no-warn-script-location." + ) + if len(msg_lines) == 1: + msg_lines.append(last_line_fmt.format("this directory")) + else: + msg_lines.append(last_line_fmt.format("these directories")) + + # Add a note if any directory starts with ~ + warn_for_tilde = any( + i[0] == "~" for i in os.environ.get("PATH", "").split(os.pathsep) if i + ) + if warn_for_tilde: + tilde_warning_msg = ( + "NOTE: The current PATH contains path(s) starting with `~`, " + "which may not be expanded by all applications." + ) + msg_lines.append(tilde_warning_msg) + + # Returns the formatted multiline message + return "\n".join(msg_lines) + + +def sorted_outrows(outrows): + # type: (Iterable[InstalledCSVRow]) -> List[InstalledCSVRow] + """Return the given rows of a RECORD file in sorted order. + + Each row is a 3-tuple (path, hash, size) and corresponds to a record of + a RECORD file (see PEP 376 and PEP 427 for details). For the rows + passed to this function, the size can be an integer as an int or string, + or the empty string. + """ + # Normally, there should only be one row per path, in which case the + # second and third elements don't come into play when sorting. + # However, in cases in the wild where a path might happen to occur twice, + # we don't want the sort operation to trigger an error (but still want + # determinism). Since the third element can be an int or string, we + # coerce each element to a string to avoid a TypeError in this case. + # For additional background, see-- + # https://github.com/pypa/pip/issues/5868 + return sorted(outrows, key=lambda row: tuple(str(x) for x in row)) + + +def get_csv_rows_for_installed( + old_csv_rows, # type: Iterable[List[str]] + installed, # type: Dict[str, str] + changed, # type: Set[str] + generated, # type: List[str] + lib_dir, # type: str +): + # type: (...) -> List[InstalledCSVRow] + """ + :param installed: A map from archive RECORD path to installation RECORD + path. + """ + installed_rows = [] # type: List[InstalledCSVRow] + for row in old_csv_rows: + if len(row) > 3: + logger.warning( + 'RECORD line has more than three elements: {}'.format(row) + ) + # Make a copy because we are mutating the row. + row = list(row) + old_path = row[0] + new_path = installed.pop(old_path, old_path) + row[0] = new_path + if new_path in changed: + digest, length = rehash(new_path) + row[1] = digest + row[2] = length + installed_rows.append(tuple(row)) + for f in generated: + digest, length = rehash(f) + installed_rows.append((normpath(f, lib_dir), digest, str(length))) + for f in installed: + installed_rows.append((installed[f], '', '')) + return installed_rows + + +class MissingCallableSuffix(Exception): + pass + + +def _raise_for_invalid_entrypoint(specification): + # type: (str) -> None + entry = get_export_entry(specification) + if entry is not None and entry.suffix is None: + raise MissingCallableSuffix(str(entry)) + + +class PipScriptMaker(ScriptMaker): + def make(self, specification, options=None): + # type: (str, Dict[str, Any]) -> List[str] + _raise_for_invalid_entrypoint(specification) + return super(PipScriptMaker, self).make(specification, options) + + +def install_unpacked_wheel( + name, # type: str + wheeldir, # type: str + wheel_zip, # type: ZipFile + scheme, # type: Scheme + req_description, # type: str + pycompile=True, # type: bool + warn_script_location=True # type: bool +): + # type: (...) -> None + """Install a wheel. + + :param name: Name of the project to install + :param wheeldir: Base directory of the unpacked wheel + :param wheel_zip: open ZipFile for wheel being installed + :param scheme: Distutils scheme dictating the install directories + :param req_description: String used in place of the requirement, for + logging + :param pycompile: Whether to byte-compile installed Python files + :param warn_script_location: Whether to check that scripts are installed + into a directory on PATH + :raises UnsupportedWheel: + * when the directory holds an unpacked wheel with incompatible + Wheel-Version + * when the .dist-info dir does not match the wheel + """ + # TODO: Investigate and break this up. + # TODO: Look into moving this into a dedicated class for representing an + # installation. + + source = wheeldir.rstrip(os.path.sep) + os.path.sep + + info_dir, metadata = parse_wheel(wheel_zip, name) + + if wheel_root_is_purelib(metadata): + lib_dir = scheme.purelib + else: + lib_dir = scheme.platlib + + subdirs = os.listdir(source) + data_dirs = [s for s in subdirs if s.endswith('.data')] + + # Record details of the files moved + # installed = files copied from the wheel to the destination + # changed = files changed while installing (scripts #! line typically) + # generated = files newly generated during the install (script wrappers) + installed = {} # type: Dict[str, str] + changed = set() + generated = [] # type: List[str] + + # Compile all of the pyc files that we're going to be installing + if pycompile: + with captured_stdout() as stdout: + with warnings.catch_warnings(): + warnings.filterwarnings('ignore') + compileall.compile_dir(source, force=True, quiet=True) + logger.debug(stdout.getvalue()) + + def record_installed(srcfile, destfile, modified=False): + # type: (str, str, bool) -> None + """Map archive RECORD paths to installation RECORD paths.""" + oldpath = normpath(srcfile, wheeldir) + newpath = normpath(destfile, lib_dir) + installed[oldpath] = newpath + if modified: + changed.add(destfile) + + def clobber( + source, # type: str + dest, # type: str + is_base, # type: bool + fixer=None, # type: Optional[Callable[[str], Any]] + filter=None # type: Optional[Callable[[str], bool]] + ): + # type: (...) -> None + ensure_dir(dest) # common for the 'include' path + + for dir, subdirs, files in os.walk(source): + basedir = dir[len(source):].lstrip(os.path.sep) + destdir = os.path.join(dest, basedir) + if is_base and basedir == '': + subdirs[:] = [s for s in subdirs if not s.endswith('.data')] + for f in files: + # Skip unwanted files + if filter and filter(f): + continue + srcfile = os.path.join(dir, f) + destfile = os.path.join(dest, basedir, f) + # directory creation is lazy and after the file filtering above + # to ensure we don't install empty dirs; empty dirs can't be + # uninstalled. + ensure_dir(destdir) + + # copyfile (called below) truncates the destination if it + # exists and then writes the new contents. This is fine in most + # cases, but can cause a segfault if pip has loaded a shared + # object (e.g. from pyopenssl through its vendored urllib3) + # Since the shared object is mmap'd an attempt to call a + # symbol in it will then cause a segfault. Unlinking the file + # allows writing of new contents while allowing the process to + # continue to use the old copy. + if os.path.exists(destfile): + os.unlink(destfile) + + # We use copyfile (not move, copy, or copy2) to be extra sure + # that we are not moving directories over (copyfile fails for + # directories) as well as to ensure that we are not copying + # over any metadata because we want more control over what + # metadata we actually copy over. + shutil.copyfile(srcfile, destfile) + + # Copy over the metadata for the file, currently this only + # includes the atime and mtime. + st = os.stat(srcfile) + if hasattr(os, "utime"): + os.utime(destfile, (st.st_atime, st.st_mtime)) + + # If our file is executable, then make our destination file + # executable. + if os.access(srcfile, os.X_OK): + st = os.stat(srcfile) + permissions = ( + st.st_mode | stat.S_IXUSR | stat.S_IXGRP | stat.S_IXOTH + ) + os.chmod(destfile, permissions) + + changed = False + if fixer: + changed = fixer(destfile) + record_installed(srcfile, destfile, changed) + + clobber(source, lib_dir, True) + + dest_info_dir = os.path.join(lib_dir, info_dir) + + # Get the defined entry points + ep_file = os.path.join(dest_info_dir, 'entry_points.txt') + console, gui = get_entrypoints(ep_file) + + def is_entrypoint_wrapper(name): + # type: (str) -> bool + # EP, EP.exe and EP-script.py are scripts generated for + # entry point EP by setuptools + if name.lower().endswith('.exe'): + matchname = name[:-4] + elif name.lower().endswith('-script.py'): + matchname = name[:-10] + elif name.lower().endswith(".pya"): + matchname = name[:-4] + else: + matchname = name + # Ignore setuptools-generated scripts + return (matchname in console or matchname in gui) + + for datadir in data_dirs: + fixer = None + filter = None + for subdir in os.listdir(os.path.join(wheeldir, datadir)): + fixer = None + if subdir == 'scripts': + fixer = fix_script + filter = is_entrypoint_wrapper + source = os.path.join(wheeldir, datadir, subdir) + dest = getattr(scheme, subdir) + clobber(source, dest, False, fixer=fixer, filter=filter) + + maker = PipScriptMaker(None, scheme.scripts) + + # Ensure old scripts are overwritten. + # See https://github.com/pypa/pip/issues/1800 + maker.clobber = True + + # Ensure we don't generate any variants for scripts because this is almost + # never what somebody wants. + # See https://bitbucket.org/pypa/distlib/issue/35/ + maker.variants = {''} + + # This is required because otherwise distlib creates scripts that are not + # executable. + # See https://bitbucket.org/pypa/distlib/issue/32/ + maker.set_mode = True + + scripts_to_generate = [] + + # Special case pip and setuptools to generate versioned wrappers + # + # The issue is that some projects (specifically, pip and setuptools) use + # code in setup.py to create "versioned" entry points - pip2.7 on Python + # 2.7, pip3.3 on Python 3.3, etc. But these entry points are baked into + # the wheel metadata at build time, and so if the wheel is installed with + # a *different* version of Python the entry points will be wrong. The + # correct fix for this is to enhance the metadata to be able to describe + # such versioned entry points, but that won't happen till Metadata 2.0 is + # available. + # In the meantime, projects using versioned entry points will either have + # incorrect versioned entry points, or they will not be able to distribute + # "universal" wheels (i.e., they will need a wheel per Python version). + # + # Because setuptools and pip are bundled with _ensurepip and virtualenv, + # we need to use universal wheels. So, as a stopgap until Metadata 2.0, we + # override the versioned entry points in the wheel and generate the + # correct ones. This code is purely a short-term measure until Metadata 2.0 + # is available. + # + # To add the level of hack in this section of code, in order to support + # ensurepip this code will look for an ``ENSUREPIP_OPTIONS`` environment + # variable which will control which version scripts get installed. + # + # ENSUREPIP_OPTIONS=altinstall + # - Only pipX.Y and easy_install-X.Y will be generated and installed + # ENSUREPIP_OPTIONS=install + # - pipX.Y, pipX, easy_install-X.Y will be generated and installed. Note + # that this option is technically if ENSUREPIP_OPTIONS is set and is + # not altinstall + # DEFAULT + # - The default behavior is to install pip, pipX, pipX.Y, easy_install + # and easy_install-X.Y. + pip_script = console.pop('pip', None) + if pip_script: + if "ENSUREPIP_OPTIONS" not in os.environ: + scripts_to_generate.append('pip = ' + pip_script) + + if os.environ.get("ENSUREPIP_OPTIONS", "") != "altinstall": + scripts_to_generate.append( + 'pip%s = %s' % (sys.version_info[0], pip_script) + ) + + scripts_to_generate.append( + 'pip%s = %s' % (get_major_minor_version(), pip_script) + ) + # Delete any other versioned pip entry points + pip_ep = [k for k in console if re.match(r'pip(\d(\.\d)?)?$', k)] + for k in pip_ep: + del console[k] + easy_install_script = console.pop('easy_install', None) + if easy_install_script: + if "ENSUREPIP_OPTIONS" not in os.environ: + scripts_to_generate.append( + 'easy_install = ' + easy_install_script + ) + + scripts_to_generate.append( + 'easy_install-%s = %s' % ( + get_major_minor_version(), easy_install_script + ) + ) + # Delete any other versioned easy_install entry points + easy_install_ep = [ + k for k in console if re.match(r'easy_install(-\d\.\d)?$', k) + ] + for k in easy_install_ep: + del console[k] + + # Generate the console and GUI entry points specified in the wheel + scripts_to_generate.extend( + '%s = %s' % kv for kv in console.items() + ) + + gui_scripts_to_generate = [ + '%s = %s' % kv for kv in gui.items() + ] + + generated_console_scripts = [] # type: List[str] + + try: + generated_console_scripts = maker.make_multiple(scripts_to_generate) + generated.extend(generated_console_scripts) + + generated.extend( + maker.make_multiple(gui_scripts_to_generate, {'gui': True}) + ) + except MissingCallableSuffix as e: + entry = e.args[0] + raise InstallationError( + "Invalid script entry point: {} for req: {} - A callable " + "suffix is required. Cf https://packaging.python.org/" + "specifications/entry-points/#use-for-scripts for more " + "information.".format(entry, req_description) + ) + + if warn_script_location: + msg = message_about_scripts_not_on_PATH(generated_console_scripts) + if msg is not None: + logger.warning(msg) + + # Record pip as the installer + installer = os.path.join(dest_info_dir, 'INSTALLER') + temp_installer = os.path.join(dest_info_dir, 'INSTALLER.pip') + with open(temp_installer, 'wb') as installer_file: + installer_file.write(b'pip\n') + shutil.move(temp_installer, installer) + generated.append(installer) + + # Record details of all files installed + record = os.path.join(dest_info_dir, 'RECORD') + temp_record = os.path.join(dest_info_dir, 'RECORD.pip') + with open_for_csv(record, 'r') as record_in: + with open_for_csv(temp_record, 'w+') as record_out: + reader = csv.reader(record_in) + outrows = get_csv_rows_for_installed( + reader, installed=installed, changed=changed, + generated=generated, lib_dir=lib_dir, + ) + writer = csv.writer(record_out) + # Sort to simplify testing. + for row in sorted_outrows(outrows): + writer.writerow(row) + shutil.move(temp_record, record) + + +def install_wheel( + name, # type: str + wheel_path, # type: str + scheme, # type: Scheme + req_description, # type: str + pycompile=True, # type: bool + warn_script_location=True, # type: bool + _temp_dir_for_testing=None, # type: Optional[str] +): + # type: (...) -> None + with TempDirectory( + path=_temp_dir_for_testing, kind="unpacked-wheel" + ) as unpacked_dir, ZipFile(wheel_path, allowZip64=True) as z: + unpack_file(wheel_path, unpacked_dir.path) + install_unpacked_wheel( + name=name, + wheeldir=unpacked_dir.path, + wheel_zip=z, + scheme=scheme, + req_description=req_description, + pycompile=pycompile, + warn_script_location=warn_script_location, + ) diff --git a/env/lib/python3.8/site-packages/pip/_internal/operations/prepare.py b/env/lib/python3.8/site-packages/pip/_internal/operations/prepare.py new file mode 100644 index 000000000..0b61f2052 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/operations/prepare.py @@ -0,0 +1,591 @@ +"""Prepares a distribution for installation +""" + +# The following comment should be removed at some point in the future. +# mypy: strict-optional=False + +import logging +import mimetypes +import os +import shutil +import sys + +from pip._vendor import requests +from pip._vendor.six import PY2 + +from pip._internal.distributions import ( + make_distribution_for_install_requirement, +) +from pip._internal.distributions.installed import InstalledDistribution +from pip._internal.exceptions import ( + DirectoryUrlHashUnsupported, + HashMismatch, + HashUnpinned, + InstallationError, + PreviousBuildDirError, + VcsHashUnsupported, +) +from pip._internal.utils.filesystem import copy2_fixed +from pip._internal.utils.hashes import MissingHashes +from pip._internal.utils.logging import indent_log +from pip._internal.utils.marker_files import write_delete_marker_file +from pip._internal.utils.misc import ( + ask_path_exists, + backup_dir, + display_path, + hide_url, + path_to_display, + rmtree, +) +from pip._internal.utils.temp_dir import TempDirectory +from pip._internal.utils.typing import MYPY_CHECK_RUNNING +from pip._internal.utils.unpacking import unpack_file +from pip._internal.vcs import vcs + +if MYPY_CHECK_RUNNING: + from typing import ( + Callable, List, Optional, Tuple, + ) + + from mypy_extensions import TypedDict + + from pip._internal.distributions import AbstractDistribution + from pip._internal.index.package_finder import PackageFinder + from pip._internal.models.link import Link + from pip._internal.network.download import Downloader + from pip._internal.req.req_install import InstallRequirement + from pip._internal.req.req_tracker import RequirementTracker + from pip._internal.utils.hashes import Hashes + + if PY2: + CopytreeKwargs = TypedDict( + 'CopytreeKwargs', + { + 'ignore': Callable[[str, List[str]], List[str]], + 'symlinks': bool, + }, + total=False, + ) + else: + CopytreeKwargs = TypedDict( + 'CopytreeKwargs', + { + 'copy_function': Callable[[str, str], None], + 'ignore': Callable[[str, List[str]], List[str]], + 'ignore_dangling_symlinks': bool, + 'symlinks': bool, + }, + total=False, + ) + +logger = logging.getLogger(__name__) + + +def _get_prepared_distribution( + req, # type: InstallRequirement + req_tracker, # type: RequirementTracker + finder, # type: PackageFinder + build_isolation # type: bool +): + # type: (...) -> AbstractDistribution + """Prepare a distribution for installation. + """ + abstract_dist = make_distribution_for_install_requirement(req) + with req_tracker.track(req): + abstract_dist.prepare_distribution_metadata(finder, build_isolation) + return abstract_dist + + +def unpack_vcs_link(link, location): + # type: (Link, str) -> None + vcs_backend = vcs.get_backend_for_scheme(link.scheme) + assert vcs_backend is not None + vcs_backend.unpack(location, url=hide_url(link.url)) + + +def _copy_file(filename, location, link): + # type: (str, str, Link) -> None + copy = True + download_location = os.path.join(location, link.filename) + if os.path.exists(download_location): + response = ask_path_exists( + 'The file {} exists. (i)gnore, (w)ipe, (b)ackup, (a)abort'.format( + display_path(download_location) + ), + ('i', 'w', 'b', 'a'), + ) + if response == 'i': + copy = False + elif response == 'w': + logger.warning('Deleting %s', display_path(download_location)) + os.remove(download_location) + elif response == 'b': + dest_file = backup_dir(download_location) + logger.warning( + 'Backing up %s to %s', + display_path(download_location), + display_path(dest_file), + ) + shutil.move(download_location, dest_file) + elif response == 'a': + sys.exit(-1) + if copy: + shutil.copy(filename, download_location) + logger.info('Saved %s', display_path(download_location)) + + +def unpack_http_url( + link, # type: Link + location, # type: str + downloader, # type: Downloader + download_dir=None, # type: Optional[str] + hashes=None, # type: Optional[Hashes] +): + # type: (...) -> str + temp_dir = TempDirectory(kind="unpack", globally_managed=True) + # If a download dir is specified, is the file already downloaded there? + already_downloaded_path = None + if download_dir: + already_downloaded_path = _check_download_dir( + link, download_dir, hashes + ) + + if already_downloaded_path: + from_path = already_downloaded_path + content_type = mimetypes.guess_type(from_path)[0] + else: + # let's download to a tmp dir + from_path, content_type = _download_http_url( + link, downloader, temp_dir.path, hashes + ) + + # unpack the archive to the build dir location. even when only + # downloading archives, they have to be unpacked to parse dependencies + unpack_file(from_path, location, content_type) + + return from_path + + +def _copy2_ignoring_special_files(src, dest): + # type: (str, str) -> None + """Copying special files is not supported, but as a convenience to users + we skip errors copying them. This supports tools that may create e.g. + socket files in the project source directory. + """ + try: + copy2_fixed(src, dest) + except shutil.SpecialFileError as e: + # SpecialFileError may be raised due to either the source or + # destination. If the destination was the cause then we would actually + # care, but since the destination directory is deleted prior to + # copy we ignore all of them assuming it is caused by the source. + logger.warning( + "Ignoring special file error '%s' encountered copying %s to %s.", + str(e), + path_to_display(src), + path_to_display(dest), + ) + + +def _copy_source_tree(source, target): + # type: (str, str) -> None + def ignore(d, names): + # type: (str, List[str]) -> List[str] + # Pulling in those directories can potentially be very slow, + # exclude the following directories if they appear in the top + # level dir (and only it). + # See discussion at https://github.com/pypa/pip/pull/6770 + return ['.tox', '.nox'] if d == source else [] + + kwargs = dict(ignore=ignore, symlinks=True) # type: CopytreeKwargs + + if not PY2: + # Python 2 does not support copy_function, so we only ignore + # errors on special file copy in Python 3. + kwargs['copy_function'] = _copy2_ignoring_special_files + + shutil.copytree(source, target, **kwargs) + + +def unpack_file_url( + link, # type: Link + location, # type: str + download_dir=None, # type: Optional[str] + hashes=None # type: Optional[Hashes] +): + # type: (...) -> Optional[str] + """Unpack link into location. + """ + link_path = link.file_path + # If it's a url to a local directory + if link.is_existing_dir(): + if os.path.isdir(location): + rmtree(location) + _copy_source_tree(link_path, location) + return None + + # If a download dir is specified, is the file already there and valid? + already_downloaded_path = None + if download_dir: + already_downloaded_path = _check_download_dir( + link, download_dir, hashes + ) + + if already_downloaded_path: + from_path = already_downloaded_path + else: + from_path = link_path + + # If --require-hashes is off, `hashes` is either empty, the + # link's embedded hash, or MissingHashes; it is required to + # match. If --require-hashes is on, we are satisfied by any + # hash in `hashes` matching: a URL-based or an option-based + # one; no internet-sourced hash will be in `hashes`. + if hashes: + hashes.check_against_path(from_path) + + content_type = mimetypes.guess_type(from_path)[0] + + # unpack the archive to the build dir location. even when only downloading + # archives, they have to be unpacked to parse dependencies + unpack_file(from_path, location, content_type) + + return from_path + + +def unpack_url( + link, # type: Link + location, # type: str + downloader, # type: Downloader + download_dir=None, # type: Optional[str] + hashes=None, # type: Optional[Hashes] +): + # type: (...) -> Optional[str] + """Unpack link into location, downloading if required. + + :param hashes: A Hashes object, one of whose embedded hashes must match, + or HashMismatch will be raised. If the Hashes is empty, no matches are + required, and unhashable types of requirements (like VCS ones, which + would ordinarily raise HashUnsupported) are allowed. + """ + # non-editable vcs urls + if link.is_vcs: + unpack_vcs_link(link, location) + return None + + # file urls + elif link.is_file: + return unpack_file_url(link, location, download_dir, hashes=hashes) + + # http urls + else: + return unpack_http_url( + link, + location, + downloader, + download_dir, + hashes=hashes, + ) + + +def _download_http_url( + link, # type: Link + downloader, # type: Downloader + temp_dir, # type: str + hashes, # type: Optional[Hashes] +): + # type: (...) -> Tuple[str, str] + """Download link url into temp_dir using provided session""" + download = downloader(link) + + file_path = os.path.join(temp_dir, download.filename) + with open(file_path, 'wb') as content_file: + for chunk in download.chunks: + content_file.write(chunk) + + if hashes: + hashes.check_against_path(file_path) + + return file_path, download.response.headers.get('content-type', '') + + +def _check_download_dir(link, download_dir, hashes): + # type: (Link, str, Optional[Hashes]) -> Optional[str] + """ Check download_dir for previously downloaded file with correct hash + If a correct file is found return its path else None + """ + download_path = os.path.join(download_dir, link.filename) + + if not os.path.exists(download_path): + return None + + # If already downloaded, does its hash match? + logger.info('File was already downloaded %s', download_path) + if hashes: + try: + hashes.check_against_path(download_path) + except HashMismatch: + logger.warning( + 'Previously-downloaded file %s has bad hash. ' + 'Re-downloading.', + download_path + ) + os.unlink(download_path) + return None + return download_path + + +class RequirementPreparer(object): + """Prepares a Requirement + """ + + def __init__( + self, + build_dir, # type: str + download_dir, # type: Optional[str] + src_dir, # type: str + wheel_download_dir, # type: Optional[str] + build_isolation, # type: bool + req_tracker, # type: RequirementTracker + downloader, # type: Downloader + finder, # type: PackageFinder + require_hashes, # type: bool + use_user_site, # type: bool + ): + # type: (...) -> None + super(RequirementPreparer, self).__init__() + + self.src_dir = src_dir + self.build_dir = build_dir + self.req_tracker = req_tracker + self.downloader = downloader + self.finder = finder + + # Where still-packed archives should be written to. If None, they are + # not saved, and are deleted immediately after unpacking. + self.download_dir = download_dir + + # Where still-packed .whl files should be written to. If None, they are + # written to the download_dir parameter. Separate to download_dir to + # permit only keeping wheel archives for pip wheel. + self.wheel_download_dir = wheel_download_dir + + # NOTE + # download_dir and wheel_download_dir overlap semantically and may + # be combined if we're willing to have non-wheel archives present in + # the wheelhouse output by 'pip wheel'. + + # Is build isolation allowed? + self.build_isolation = build_isolation + + # Should hash-checking be required? + self.require_hashes = require_hashes + + # Should install in user site-packages? + self.use_user_site = use_user_site + + @property + def _download_should_save(self): + # type: () -> bool + if not self.download_dir: + return False + + if os.path.exists(self.download_dir): + return True + + logger.critical('Could not find download directory') + raise InstallationError( + "Could not find or access download directory '{}'" + .format(self.download_dir)) + + def prepare_linked_requirement( + self, + req, # type: InstallRequirement + ): + # type: (...) -> AbstractDistribution + """Prepare a requirement that would be obtained from req.link + """ + assert req.link + link = req.link + + # TODO: Breakup into smaller functions + if link.scheme == 'file': + path = link.file_path + logger.info('Processing %s', display_path(path)) + else: + logger.info('Collecting %s', req.req or req) + + with indent_log(): + # @@ if filesystem packages are not marked + # editable in a req, a non deterministic error + # occurs when the script attempts to unpack the + # build directory + # Since source_dir is only set for editable requirements. + assert req.source_dir is None + req.ensure_has_source_dir(self.build_dir) + # If a checkout exists, it's unwise to keep going. version + # inconsistencies are logged later, but do not fail the + # installation. + # FIXME: this won't upgrade when there's an existing + # package unpacked in `req.source_dir` + if os.path.exists(os.path.join(req.source_dir, 'setup.py')): + raise PreviousBuildDirError( + "pip can't proceed with requirements '{}' due to a" + " pre-existing build directory ({}). This is " + "likely due to a previous installation that failed" + ". pip is being responsible and not assuming it " + "can delete this. Please delete it and try again." + .format(req, req.source_dir) + ) + + # Now that we have the real link, we can tell what kind of + # requirements we have and raise some more informative errors + # than otherwise. (For example, we can raise VcsHashUnsupported + # for a VCS URL rather than HashMissing.) + if self.require_hashes: + # We could check these first 2 conditions inside + # unpack_url and save repetition of conditions, but then + # we would report less-useful error messages for + # unhashable requirements, complaining that there's no + # hash provided. + if link.is_vcs: + raise VcsHashUnsupported() + elif link.is_existing_dir(): + raise DirectoryUrlHashUnsupported() + if not req.original_link and not req.is_pinned: + # Unpinned packages are asking for trouble when a new + # version is uploaded. This isn't a security check, but + # it saves users a surprising hash mismatch in the + # future. + # + # file:/// URLs aren't pinnable, so don't complain + # about them not being pinned. + raise HashUnpinned() + + hashes = req.hashes(trust_internet=not self.require_hashes) + if self.require_hashes and not hashes: + # Known-good hashes are missing for this requirement, so + # shim it with a facade object that will provoke hash + # computation and then raise a HashMissing exception + # showing the user what the hash should be. + hashes = MissingHashes() + + download_dir = self.download_dir + if link.is_wheel and self.wheel_download_dir: + # when doing 'pip wheel` we download wheels to a + # dedicated dir. + download_dir = self.wheel_download_dir + + try: + local_path = unpack_url( + link, req.source_dir, self.downloader, download_dir, + hashes=hashes, + ) + except requests.HTTPError as exc: + logger.critical( + 'Could not install requirement %s because of error %s', + req, + exc, + ) + raise InstallationError( + 'Could not install requirement {} because of HTTP ' + 'error {} for URL {}'.format(req, exc, link) + ) + + # For use in later processing, preserve the file path on the + # requirement. + if local_path: + req.local_file_path = local_path + + if link.is_wheel: + if download_dir: + # When downloading, we only unpack wheels to get + # metadata. + autodelete_unpacked = True + else: + # When installing a wheel, we use the unpacked + # wheel. + autodelete_unpacked = False + else: + # We always delete unpacked sdists after pip runs. + autodelete_unpacked = True + if autodelete_unpacked: + write_delete_marker_file(req.source_dir) + + abstract_dist = _get_prepared_distribution( + req, self.req_tracker, self.finder, self.build_isolation, + ) + + if download_dir: + if link.is_existing_dir(): + logger.info('Link is a directory, ignoring download_dir') + elif local_path and not os.path.exists( + os.path.join(download_dir, link.filename) + ): + _copy_file(local_path, download_dir, link) + + if self._download_should_save: + # Make a .zip of the source_dir we already created. + if link.is_vcs: + req.archive(self.download_dir) + return abstract_dist + + def prepare_editable_requirement( + self, + req, # type: InstallRequirement + ): + # type: (...) -> AbstractDistribution + """Prepare an editable requirement + """ + assert req.editable, "cannot prepare a non-editable req as editable" + + logger.info('Obtaining %s', req) + + with indent_log(): + if self.require_hashes: + raise InstallationError( + 'The editable requirement {} cannot be installed when ' + 'requiring hashes, because there is no single file to ' + 'hash.'.format(req) + ) + req.ensure_has_source_dir(self.src_dir) + req.update_editable(not self._download_should_save) + + abstract_dist = _get_prepared_distribution( + req, self.req_tracker, self.finder, self.build_isolation, + ) + + if self._download_should_save: + req.archive(self.download_dir) + req.check_if_exists(self.use_user_site) + + return abstract_dist + + def prepare_installed_requirement( + self, + req, # type: InstallRequirement + skip_reason # type: str + ): + # type: (...) -> AbstractDistribution + """Prepare an already-installed requirement + """ + assert req.satisfied_by, "req should have been satisfied but isn't" + assert skip_reason is not None, ( + "did not get skip reason skipped but req.satisfied_by " + "is set to {}".format(req.satisfied_by) + ) + logger.info( + 'Requirement %s: %s (%s)', + skip_reason, req, req.satisfied_by.version + ) + with indent_log(): + if self.require_hashes: + logger.debug( + 'Since it is already installed, we are trusting this ' + 'package without checking its hash. To ensure a ' + 'completely repeatable environment, install into an ' + 'empty virtualenv.' + ) + abstract_dist = InstalledDistribution(req) + + return abstract_dist diff --git a/env/lib/python3.8/site-packages/pip/_internal/pep425tags.py b/env/lib/python3.8/site-packages/pip/_internal/pep425tags.py new file mode 100644 index 000000000..a2386ee75 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/pep425tags.py @@ -0,0 +1,167 @@ +"""Generate and work with PEP 425 Compatibility Tags.""" +from __future__ import absolute_import + +import logging +import re + +from pip._vendor.packaging.tags import ( + Tag, + compatible_tags, + cpython_tags, + generic_tags, + interpreter_name, + interpreter_version, + mac_platforms, +) + +from pip._internal.utils.typing import MYPY_CHECK_RUNNING + +if MYPY_CHECK_RUNNING: + from typing import List, Optional, Tuple + + from pip._vendor.packaging.tags import PythonVersion + +logger = logging.getLogger(__name__) + +_osx_arch_pat = re.compile(r'(.+)_(\d+)_(\d+)_(.+)') + + +def version_info_to_nodot(version_info): + # type: (Tuple[int, ...]) -> str + # Only use up to the first two numbers. + return ''.join(map(str, version_info[:2])) + + +def _mac_platforms(arch): + # type: (str) -> List[str] + match = _osx_arch_pat.match(arch) + if match: + name, major, minor, actual_arch = match.groups() + mac_version = (int(major), int(minor)) + arches = [ + # Since we have always only checked that the platform starts + # with "macosx", for backwards-compatibility we extract the + # actual prefix provided by the user in case they provided + # something like "macosxcustom_". It may be good to remove + # this as undocumented or deprecate it in the future. + '{}_{}'.format(name, arch[len('macosx_'):]) + for arch in mac_platforms(mac_version, actual_arch) + ] + else: + # arch pattern didn't match (?!) + arches = [arch] + return arches + + +def _custom_manylinux_platforms(arch): + # type: (str) -> List[str] + arches = [arch] + arch_prefix, arch_sep, arch_suffix = arch.partition('_') + if arch_prefix == 'manylinux2014': + # manylinux1/manylinux2010 wheels run on most manylinux2014 systems + # with the exception of wheels depending on ncurses. PEP 599 states + # manylinux1/manylinux2010 wheels should be considered + # manylinux2014 wheels: + # https://www.python.org/dev/peps/pep-0599/#backwards-compatibility-with-manylinux2010-wheels + if arch_suffix in {'i686', 'x86_64'}: + arches.append('manylinux2010' + arch_sep + arch_suffix) + arches.append('manylinux1' + arch_sep + arch_suffix) + elif arch_prefix == 'manylinux2010': + # manylinux1 wheels run on most manylinux2010 systems with the + # exception of wheels depending on ncurses. PEP 571 states + # manylinux1 wheels should be considered manylinux2010 wheels: + # https://www.python.org/dev/peps/pep-0571/#backwards-compatibility-with-manylinux1-wheels + arches.append('manylinux1' + arch_sep + arch_suffix) + return arches + + +def _get_custom_platforms(arch): + # type: (str) -> List[str] + arch_prefix, arch_sep, arch_suffix = arch.partition('_') + if arch.startswith('macosx'): + arches = _mac_platforms(arch) + elif arch_prefix in ['manylinux2014', 'manylinux2010']: + arches = _custom_manylinux_platforms(arch) + else: + arches = [arch] + return arches + + +def _get_python_version(version): + # type: (str) -> PythonVersion + if len(version) > 1: + return int(version[0]), int(version[1:]) + else: + return (int(version[0]),) + + +def _get_custom_interpreter(implementation=None, version=None): + # type: (Optional[str], Optional[str]) -> str + if implementation is None: + implementation = interpreter_name() + if version is None: + version = interpreter_version() + return "{}{}".format(implementation, version) + + +def get_supported( + version=None, # type: Optional[str] + platform=None, # type: Optional[str] + impl=None, # type: Optional[str] + abi=None # type: Optional[str] +): + # type: (...) -> List[Tag] + """Return a list of supported tags for each version specified in + `versions`. + + :param version: a string version, of the form "33" or "32", + or None. The version will be assumed to support our ABI. + :param platform: specify the exact platform you want valid + tags for, or None. If None, use the local system platform. + :param impl: specify the exact implementation you want valid + tags for, or None. If None, use the local interpreter impl. + :param abi: specify the exact abi you want valid + tags for, or None. If None, use the local interpreter abi. + """ + supported = [] # type: List[Tag] + + python_version = None # type: Optional[PythonVersion] + if version is not None: + python_version = _get_python_version(version) + + interpreter = _get_custom_interpreter(impl, version) + + abis = None # type: Optional[List[str]] + if abi is not None: + abis = [abi] + + platforms = None # type: Optional[List[str]] + if platform is not None: + platforms = _get_custom_platforms(platform) + + is_cpython = (impl or interpreter_name()) == "cp" + if is_cpython: + supported.extend( + cpython_tags( + python_version=python_version, + abis=abis, + platforms=platforms, + ) + ) + else: + supported.extend( + generic_tags( + interpreter=interpreter, + abis=abis, + platforms=platforms, + ) + ) + supported.extend( + compatible_tags( + python_version=python_version, + interpreter=interpreter, + platforms=platforms, + ) + ) + + return supported diff --git a/env/lib/python3.8/site-packages/pip/_internal/pyproject.py b/env/lib/python3.8/site-packages/pip/_internal/pyproject.py new file mode 100644 index 000000000..cf614fd6e --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/pyproject.py @@ -0,0 +1,196 @@ +from __future__ import absolute_import + +import io +import os +import sys +from collections import namedtuple + +from pip._vendor import pytoml, six +from pip._vendor.packaging.requirements import InvalidRequirement, Requirement + +from pip._internal.exceptions import InstallationError +from pip._internal.utils.typing import MYPY_CHECK_RUNNING + +if MYPY_CHECK_RUNNING: + from typing import Any, Optional, List + + +def _is_list_of_str(obj): + # type: (Any) -> bool + return ( + isinstance(obj, list) and + all(isinstance(item, six.string_types) for item in obj) + ) + + +def make_pyproject_path(unpacked_source_directory): + # type: (str) -> str + path = os.path.join(unpacked_source_directory, 'pyproject.toml') + + # Python2 __file__ should not be unicode + if six.PY2 and isinstance(path, six.text_type): + path = path.encode(sys.getfilesystemencoding()) + + return path + + +BuildSystemDetails = namedtuple('BuildSystemDetails', [ + 'requires', 'backend', 'check', 'backend_path' +]) + + +def load_pyproject_toml( + use_pep517, # type: Optional[bool] + pyproject_toml, # type: str + setup_py, # type: str + req_name # type: str +): + # type: (...) -> Optional[BuildSystemDetails] + """Load the pyproject.toml file. + + Parameters: + use_pep517 - Has the user requested PEP 517 processing? None + means the user hasn't explicitly specified. + pyproject_toml - Location of the project's pyproject.toml file + setup_py - Location of the project's setup.py file + req_name - The name of the requirement we're processing (for + error reporting) + + Returns: + None if we should use the legacy code path, otherwise a tuple + ( + requirements from pyproject.toml, + name of PEP 517 backend, + requirements we should check are installed after setting + up the build environment + directory paths to import the backend from (backend-path), + relative to the project root. + ) + """ + has_pyproject = os.path.isfile(pyproject_toml) + has_setup = os.path.isfile(setup_py) + + if has_pyproject: + with io.open(pyproject_toml, encoding="utf-8") as f: + pp_toml = pytoml.load(f) + build_system = pp_toml.get("build-system") + else: + build_system = None + + # The following cases must use PEP 517 + # We check for use_pep517 being non-None and falsey because that means + # the user explicitly requested --no-use-pep517. The value 0 as + # opposed to False can occur when the value is provided via an + # environment variable or config file option (due to the quirk of + # strtobool() returning an integer in pip's configuration code). + if has_pyproject and not has_setup: + if use_pep517 is not None and not use_pep517: + raise InstallationError( + "Disabling PEP 517 processing is invalid: " + "project does not have a setup.py" + ) + use_pep517 = True + elif build_system and "build-backend" in build_system: + if use_pep517 is not None and not use_pep517: + raise InstallationError( + "Disabling PEP 517 processing is invalid: " + "project specifies a build backend of {} " + "in pyproject.toml".format( + build_system["build-backend"] + ) + ) + use_pep517 = True + + # If we haven't worked out whether to use PEP 517 yet, + # and the user hasn't explicitly stated a preference, + # we do so if the project has a pyproject.toml file. + elif use_pep517 is None: + use_pep517 = has_pyproject + + # At this point, we know whether we're going to use PEP 517. + assert use_pep517 is not None + + # If we're using the legacy code path, there is nothing further + # for us to do here. + if not use_pep517: + return None + + if build_system is None: + # Either the user has a pyproject.toml with no build-system + # section, or the user has no pyproject.toml, but has opted in + # explicitly via --use-pep517. + # In the absence of any explicit backend specification, we + # assume the setuptools backend that most closely emulates the + # traditional direct setup.py execution, and require wheel and + # a version of setuptools that supports that backend. + + build_system = { + "requires": ["setuptools>=40.8.0", "wheel"], + "build-backend": "setuptools.build_meta:__legacy__", + } + + # If we're using PEP 517, we have build system information (either + # from pyproject.toml, or defaulted by the code above). + # Note that at this point, we do not know if the user has actually + # specified a backend, though. + assert build_system is not None + + # Ensure that the build-system section in pyproject.toml conforms + # to PEP 518. + error_template = ( + "{package} has a pyproject.toml file that does not comply " + "with PEP 518: {reason}" + ) + + # Specifying the build-system table but not the requires key is invalid + if "requires" not in build_system: + raise InstallationError( + error_template.format(package=req_name, reason=( + "it has a 'build-system' table but not " + "'build-system.requires' which is mandatory in the table" + )) + ) + + # Error out if requires is not a list of strings + requires = build_system["requires"] + if not _is_list_of_str(requires): + raise InstallationError(error_template.format( + package=req_name, + reason="'build-system.requires' is not a list of strings.", + )) + + # Each requirement must be valid as per PEP 508 + for requirement in requires: + try: + Requirement(requirement) + except InvalidRequirement: + raise InstallationError( + error_template.format( + package=req_name, + reason=( + "'build-system.requires' contains an invalid " + "requirement: {!r}".format(requirement) + ), + ) + ) + + backend = build_system.get("build-backend") + backend_path = build_system.get("backend-path", []) + check = [] # type: List[str] + if backend is None: + # If the user didn't specify a backend, we assume they want to use + # the setuptools backend. But we can't be sure they have included + # a version of setuptools which supplies the backend, or wheel + # (which is needed by the backend) in their requirements. So we + # make a note to check that those requirements are present once + # we have set up the environment. + # This is quite a lot of work to check for a very specific case. But + # the problem is, that case is potentially quite common - projects that + # adopted PEP 518 early for the ability to specify requirements to + # execute setup.py, but never considered needing to mention the build + # tools themselves. The original PEP 518 code had a similar check (but + # implemented in a different way). + backend = "setuptools.build_meta:__legacy__" + check = ["setuptools>=40.8.0", "wheel"] + + return BuildSystemDetails(requires, backend, check, backend_path) diff --git a/env/lib/python3.8/site-packages/pip/_internal/req/__init__.py b/env/lib/python3.8/site-packages/pip/_internal/req/__init__.py new file mode 100644 index 000000000..d2d027ade --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/req/__init__.py @@ -0,0 +1,92 @@ +# The following comment should be removed at some point in the future. +# mypy: strict-optional=False + +from __future__ import absolute_import + +import logging + +from pip._internal.utils.logging import indent_log +from pip._internal.utils.typing import MYPY_CHECK_RUNNING + +from .req_file import parse_requirements +from .req_install import InstallRequirement +from .req_set import RequirementSet + +if MYPY_CHECK_RUNNING: + from typing import Any, List, Sequence + +__all__ = [ + "RequirementSet", "InstallRequirement", + "parse_requirements", "install_given_reqs", +] + +logger = logging.getLogger(__name__) + + +class InstallationResult(object): + def __init__(self, name): + # type: (str) -> None + self.name = name + + def __repr__(self): + # type: () -> str + return "InstallationResult(name={!r})".format(self.name) + + +def install_given_reqs( + to_install, # type: List[InstallRequirement] + install_options, # type: List[str] + global_options=(), # type: Sequence[str] + *args, # type: Any + **kwargs # type: Any +): + # type: (...) -> List[InstallationResult] + """ + Install everything in the given list. + + (to be called after having downloaded and unpacked the packages) + """ + + if to_install: + logger.info( + 'Installing collected packages: %s', + ', '.join([req.name for req in to_install]), + ) + + installed = [] + + with indent_log(): + for requirement in to_install: + if requirement.should_reinstall: + logger.info('Attempting uninstall: %s', requirement.name) + with indent_log(): + uninstalled_pathset = requirement.uninstall( + auto_confirm=True + ) + try: + requirement.install( + install_options, + global_options, + *args, + **kwargs + ) + except Exception: + should_rollback = ( + requirement.should_reinstall and + not requirement.install_succeeded + ) + # if install did not succeed, rollback previous uninstall + if should_rollback: + uninstalled_pathset.rollback() + raise + else: + should_commit = ( + requirement.should_reinstall and + requirement.install_succeeded + ) + if should_commit: + uninstalled_pathset.commit() + + installed.append(InstallationResult(requirement.name)) + + return installed diff --git a/env/lib/python3.8/site-packages/pip/_internal/req/constructors.py b/env/lib/python3.8/site-packages/pip/_internal/req/constructors.py new file mode 100644 index 000000000..1f3cd8a10 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/req/constructors.py @@ -0,0 +1,436 @@ +"""Backing implementation for InstallRequirement's various constructors + +The idea here is that these formed a major chunk of InstallRequirement's size +so, moving them and support code dedicated to them outside of that class +helps creates for better understandability for the rest of the code. + +These are meant to be used elsewhere within pip to create instances of +InstallRequirement. +""" + +# The following comment should be removed at some point in the future. +# mypy: strict-optional=False + +import logging +import os +import re + +from pip._vendor.packaging.markers import Marker +from pip._vendor.packaging.requirements import InvalidRequirement, Requirement +from pip._vendor.packaging.specifiers import Specifier +from pip._vendor.pkg_resources import RequirementParseError, parse_requirements + +from pip._internal.exceptions import InstallationError +from pip._internal.models.index import PyPI, TestPyPI +from pip._internal.models.link import Link +from pip._internal.models.wheel import Wheel +from pip._internal.pyproject import make_pyproject_path +from pip._internal.req.req_install import InstallRequirement +from pip._internal.utils.filetypes import ARCHIVE_EXTENSIONS +from pip._internal.utils.misc import is_installable_dir, splitext +from pip._internal.utils.typing import MYPY_CHECK_RUNNING +from pip._internal.utils.urls import path_to_url +from pip._internal.vcs import is_url, vcs + +if MYPY_CHECK_RUNNING: + from typing import ( + Any, Dict, Optional, Set, Tuple, Union, + ) + from pip._internal.cache import WheelCache + + +__all__ = [ + "install_req_from_editable", "install_req_from_line", + "parse_editable" +] + +logger = logging.getLogger(__name__) +operators = Specifier._operators.keys() + + +def is_archive_file(name): + # type: (str) -> bool + """Return True if `name` is a considered as an archive file.""" + ext = splitext(name)[1].lower() + if ext in ARCHIVE_EXTENSIONS: + return True + return False + + +def _strip_extras(path): + # type: (str) -> Tuple[str, Optional[str]] + m = re.match(r'^(.+)(\[[^\]]+\])$', path) + extras = None + if m: + path_no_extras = m.group(1) + extras = m.group(2) + else: + path_no_extras = path + + return path_no_extras, extras + + +def convert_extras(extras): + # type: (Optional[str]) -> Set[str] + if not extras: + return set() + return Requirement("placeholder" + extras.lower()).extras + + +def parse_editable(editable_req): + # type: (str) -> Tuple[Optional[str], str, Optional[Set[str]]] + """Parses an editable requirement into: + - a requirement name + - an URL + - extras + - editable options + Accepted requirements: + svn+http://blahblah@rev#egg=Foobar[baz]&subdirectory=version_subdir + .[some_extra] + """ + + url = editable_req + + # If a file path is specified with extras, strip off the extras. + url_no_extras, extras = _strip_extras(url) + + if os.path.isdir(url_no_extras): + if not os.path.exists(os.path.join(url_no_extras, 'setup.py')): + msg = ( + 'File "setup.py" not found. Directory cannot be installed ' + 'in editable mode: {}'.format(os.path.abspath(url_no_extras)) + ) + pyproject_path = make_pyproject_path(url_no_extras) + if os.path.isfile(pyproject_path): + msg += ( + '\n(A "pyproject.toml" file was found, but editable ' + 'mode currently requires a setup.py based build.)' + ) + raise InstallationError(msg) + + # Treating it as code that has already been checked out + url_no_extras = path_to_url(url_no_extras) + + if url_no_extras.lower().startswith('file:'): + package_name = Link(url_no_extras).egg_fragment + if extras: + return ( + package_name, + url_no_extras, + Requirement("placeholder" + extras.lower()).extras, + ) + else: + return package_name, url_no_extras, None + + for version_control in vcs: + if url.lower().startswith('%s:' % version_control): + url = '%s+%s' % (version_control, url) + break + + if '+' not in url: + raise InstallationError( + '{} is not a valid editable requirement. ' + 'It should either be a path to a local project or a VCS URL ' + '(beginning with svn+, git+, hg+, or bzr+).'.format(editable_req) + ) + + vc_type = url.split('+', 1)[0].lower() + + if not vcs.get_backend(vc_type): + error_message = 'For --editable=%s only ' % editable_req + \ + ', '.join([backend.name + '+URL' for backend in vcs.backends]) + \ + ' is currently supported' + raise InstallationError(error_message) + + package_name = Link(url).egg_fragment + if not package_name: + raise InstallationError( + "Could not detect requirement name for '%s', please specify one " + "with #egg=your_package_name" % editable_req + ) + return package_name, url, None + + +def deduce_helpful_msg(req): + # type: (str) -> str + """Returns helpful msg in case requirements file does not exist, + or cannot be parsed. + + :params req: Requirements file path + """ + msg = "" + if os.path.exists(req): + msg = " It does exist." + # Try to parse and check if it is a requirements file. + try: + with open(req, 'r') as fp: + # parse first line only + next(parse_requirements(fp.read())) + msg += " The argument you provided " + \ + "(%s) appears to be a" % (req) + \ + " requirements file. If that is the" + \ + " case, use the '-r' flag to install" + \ + " the packages specified within it." + except RequirementParseError: + logger.debug("Cannot parse '%s' as requirements \ + file" % (req), exc_info=True) + else: + msg += " File '%s' does not exist." % (req) + return msg + + +class RequirementParts(object): + def __init__( + self, + requirement, # type: Optional[Requirement] + link, # type: Optional[Link] + markers, # type: Optional[Marker] + extras, # type: Set[str] + ): + self.requirement = requirement + self.link = link + self.markers = markers + self.extras = extras + + +def parse_req_from_editable(editable_req): + # type: (str) -> RequirementParts + name, url, extras_override = parse_editable(editable_req) + + if name is not None: + try: + req = Requirement(name) + except InvalidRequirement: + raise InstallationError("Invalid requirement: '%s'" % name) + else: + req = None + + link = Link(url) + + return RequirementParts(req, link, None, extras_override) + + +# ---- The actual constructors follow ---- + + +def install_req_from_editable( + editable_req, # type: str + comes_from=None, # type: Optional[str] + use_pep517=None, # type: Optional[bool] + isolated=False, # type: bool + options=None, # type: Optional[Dict[str, Any]] + wheel_cache=None, # type: Optional[WheelCache] + constraint=False # type: bool +): + # type: (...) -> InstallRequirement + + parts = parse_req_from_editable(editable_req) + + source_dir = parts.link.file_path if parts.link.scheme == 'file' else None + + return InstallRequirement( + parts.requirement, comes_from, source_dir=source_dir, + editable=True, + link=parts.link, + constraint=constraint, + use_pep517=use_pep517, + isolated=isolated, + options=options if options else {}, + wheel_cache=wheel_cache, + extras=parts.extras, + ) + + +def _looks_like_path(name): + # type: (str) -> bool + """Checks whether the string "looks like" a path on the filesystem. + + This does not check whether the target actually exists, only judge from the + appearance. + + Returns true if any of the following conditions is true: + * a path separator is found (either os.path.sep or os.path.altsep); + * a dot is found (which represents the current directory). + """ + if os.path.sep in name: + return True + if os.path.altsep is not None and os.path.altsep in name: + return True + if name.startswith("."): + return True + return False + + +def _get_url_from_path(path, name): + # type: (str, str) -> str + """ + First, it checks whether a provided path is an installable directory + (e.g. it has a setup.py). If it is, returns the path. + + If false, check if the path is an archive file (such as a .whl). + The function checks if the path is a file. If false, if the path has + an @, it will treat it as a PEP 440 URL requirement and return the path. + """ + if _looks_like_path(name) and os.path.isdir(path): + if is_installable_dir(path): + return path_to_url(path) + raise InstallationError( + "Directory %r is not installable. Neither 'setup.py' " + "nor 'pyproject.toml' found." % name + ) + if not is_archive_file(path): + return None + if os.path.isfile(path): + return path_to_url(path) + urlreq_parts = name.split('@', 1) + if len(urlreq_parts) >= 2 and not _looks_like_path(urlreq_parts[0]): + # If the path contains '@' and the part before it does not look + # like a path, try to treat it as a PEP 440 URL req instead. + return None + logger.warning( + 'Requirement %r looks like a filename, but the ' + 'file does not exist', + name + ) + return path_to_url(path) + + +def parse_req_from_line(name, line_source): + # type: (str, Optional[str]) -> RequirementParts + if is_url(name): + marker_sep = '; ' + else: + marker_sep = ';' + if marker_sep in name: + name, markers_as_string = name.split(marker_sep, 1) + markers_as_string = markers_as_string.strip() + if not markers_as_string: + markers = None + else: + markers = Marker(markers_as_string) + else: + markers = None + name = name.strip() + req_as_string = None + path = os.path.normpath(os.path.abspath(name)) + link = None + extras_as_string = None + + if is_url(name): + link = Link(name) + else: + p, extras_as_string = _strip_extras(path) + url = _get_url_from_path(p, name) + if url is not None: + link = Link(url) + + # it's a local file, dir, or url + if link: + # Handle relative file URLs + if link.scheme == 'file' and re.search(r'\.\./', link.url): + link = Link( + path_to_url(os.path.normpath(os.path.abspath(link.path)))) + # wheel file + if link.is_wheel: + wheel = Wheel(link.filename) # can raise InvalidWheelFilename + req_as_string = "%s==%s" % (wheel.name, wheel.version) + else: + # set the req to the egg fragment. when it's not there, this + # will become an 'unnamed' requirement + req_as_string = link.egg_fragment + + # a requirement specifier + else: + req_as_string = name + + extras = convert_extras(extras_as_string) + + def with_source(text): + # type: (str) -> str + if not line_source: + return text + return '{} (from {})'.format(text, line_source) + + if req_as_string is not None: + try: + req = Requirement(req_as_string) + except InvalidRequirement: + if os.path.sep in req_as_string: + add_msg = "It looks like a path." + add_msg += deduce_helpful_msg(req_as_string) + elif ('=' in req_as_string and + not any(op in req_as_string for op in operators)): + add_msg = "= is not a valid operator. Did you mean == ?" + else: + add_msg = '' + msg = with_source( + 'Invalid requirement: {!r}'.format(req_as_string) + ) + if add_msg: + msg += '\nHint: {}'.format(add_msg) + raise InstallationError(msg) + else: + req = None + + return RequirementParts(req, link, markers, extras) + + +def install_req_from_line( + name, # type: str + comes_from=None, # type: Optional[Union[str, InstallRequirement]] + use_pep517=None, # type: Optional[bool] + isolated=False, # type: bool + options=None, # type: Optional[Dict[str, Any]] + wheel_cache=None, # type: Optional[WheelCache] + constraint=False, # type: bool + line_source=None, # type: Optional[str] +): + # type: (...) -> InstallRequirement + """Creates an InstallRequirement from a name, which might be a + requirement, directory containing 'setup.py', filename, or URL. + + :param line_source: An optional string describing where the line is from, + for logging purposes in case of an error. + """ + parts = parse_req_from_line(name, line_source) + + return InstallRequirement( + parts.requirement, comes_from, link=parts.link, markers=parts.markers, + use_pep517=use_pep517, isolated=isolated, + options=options if options else {}, + wheel_cache=wheel_cache, + constraint=constraint, + extras=parts.extras, + ) + + +def install_req_from_req_string( + req_string, # type: str + comes_from=None, # type: Optional[InstallRequirement] + isolated=False, # type: bool + wheel_cache=None, # type: Optional[WheelCache] + use_pep517=None # type: Optional[bool] +): + # type: (...) -> InstallRequirement + try: + req = Requirement(req_string) + except InvalidRequirement: + raise InstallationError("Invalid requirement: '%s'" % req_string) + + domains_not_allowed = [ + PyPI.file_storage_domain, + TestPyPI.file_storage_domain, + ] + if (req.url and comes_from and comes_from.link and + comes_from.link.netloc in domains_not_allowed): + # Explicitly disallow pypi packages that depend on external urls + raise InstallationError( + "Packages installed from PyPI cannot depend on packages " + "which are not also hosted on PyPI.\n" + "%s depends on %s " % (comes_from.name, req) + ) + + return InstallRequirement( + req, comes_from, isolated=isolated, wheel_cache=wheel_cache, + use_pep517=use_pep517 + ) diff --git a/env/lib/python3.8/site-packages/pip/_internal/req/req_file.py b/env/lib/python3.8/site-packages/pip/_internal/req/req_file.py new file mode 100644 index 000000000..8c7810481 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/req/req_file.py @@ -0,0 +1,546 @@ +""" +Requirements file parsing +""" + +# The following comment should be removed at some point in the future. +# mypy: strict-optional=False + +from __future__ import absolute_import + +import optparse +import os +import re +import shlex +import sys + +from pip._vendor.six.moves import filterfalse +from pip._vendor.six.moves.urllib import parse as urllib_parse + +from pip._internal.cli import cmdoptions +from pip._internal.exceptions import ( + InstallationError, + RequirementsFileParseError, +) +from pip._internal.models.search_scope import SearchScope +from pip._internal.req.constructors import ( + install_req_from_editable, + install_req_from_line, +) +from pip._internal.utils.encoding import auto_decode +from pip._internal.utils.typing import MYPY_CHECK_RUNNING +from pip._internal.utils.urls import get_url_scheme + +if MYPY_CHECK_RUNNING: + from optparse import Values + from typing import ( + Any, Callable, Iterator, List, NoReturn, Optional, Text, Tuple, + ) + + from pip._internal.req import InstallRequirement + from pip._internal.cache import WheelCache + from pip._internal.index.package_finder import PackageFinder + from pip._internal.network.session import PipSession + + ReqFileLines = Iterator[Tuple[int, Text]] + + LineParser = Callable[[Text], Tuple[str, Values]] + + +__all__ = ['parse_requirements'] + +SCHEME_RE = re.compile(r'^(http|https|file):', re.I) +COMMENT_RE = re.compile(r'(^|\s+)#.*$') + +# Matches environment variable-style values in '${MY_VARIABLE_1}' with the +# variable name consisting of only uppercase letters, digits or the '_' +# (underscore). This follows the POSIX standard defined in IEEE Std 1003.1, +# 2013 Edition. +ENV_VAR_RE = re.compile(r'(?P\$\{(?P[A-Z0-9_]+)\})') + +SUPPORTED_OPTIONS = [ + cmdoptions.index_url, + cmdoptions.extra_index_url, + cmdoptions.no_index, + cmdoptions.constraints, + cmdoptions.requirements, + cmdoptions.editable, + cmdoptions.find_links, + cmdoptions.no_binary, + cmdoptions.only_binary, + cmdoptions.require_hashes, + cmdoptions.pre, + cmdoptions.trusted_host, + cmdoptions.always_unzip, # Deprecated +] # type: List[Callable[..., optparse.Option]] + +# options to be passed to requirements +SUPPORTED_OPTIONS_REQ = [ + cmdoptions.install_options, + cmdoptions.global_options, + cmdoptions.hash, +] # type: List[Callable[..., optparse.Option]] + +# the 'dest' string values +SUPPORTED_OPTIONS_REQ_DEST = [str(o().dest) for o in SUPPORTED_OPTIONS_REQ] + + +class ParsedLine(object): + def __init__( + self, + filename, # type: str + lineno, # type: int + comes_from, # type: str + args, # type: str + opts, # type: Values + constraint, # type: bool + ): + # type: (...) -> None + self.filename = filename + self.lineno = lineno + self.comes_from = comes_from + self.args = args + self.opts = opts + self.constraint = constraint + + +def parse_requirements( + filename, # type: str + session, # type: PipSession + finder=None, # type: Optional[PackageFinder] + comes_from=None, # type: Optional[str] + options=None, # type: Optional[optparse.Values] + constraint=False, # type: bool + wheel_cache=None, # type: Optional[WheelCache] + use_pep517=None # type: Optional[bool] +): + # type: (...) -> Iterator[InstallRequirement] + """Parse a requirements file and yield InstallRequirement instances. + + :param filename: Path or url of requirements file. + :param session: PipSession instance. + :param finder: Instance of pip.index.PackageFinder. + :param comes_from: Origin description of requirements. + :param options: cli options. + :param constraint: If true, parsing a constraint file rather than + requirements file. + :param wheel_cache: Instance of pip.wheel.WheelCache + :param use_pep517: Value of the --use-pep517 option. + """ + skip_requirements_regex = ( + options.skip_requirements_regex if options else None + ) + line_parser = get_line_parser(finder) + parser = RequirementsFileParser( + session, line_parser, comes_from, skip_requirements_regex + ) + + for parsed_line in parser.parse(filename, constraint): + req = handle_line( + parsed_line, finder, options, session, wheel_cache, use_pep517 + ) + if req is not None: + yield req + + +def preprocess(content, skip_requirements_regex): + # type: (Text, Optional[str]) -> ReqFileLines + """Split, filter, and join lines, and return a line iterator + + :param content: the content of the requirements file + :param options: cli options + """ + lines_enum = enumerate(content.splitlines(), start=1) # type: ReqFileLines + lines_enum = join_lines(lines_enum) + lines_enum = ignore_comments(lines_enum) + if skip_requirements_regex: + lines_enum = skip_regex(lines_enum, skip_requirements_regex) + lines_enum = expand_env_variables(lines_enum) + return lines_enum + + +def handle_line( + line, # type: ParsedLine + finder=None, # type: Optional[PackageFinder] + options=None, # type: Optional[optparse.Values] + session=None, # type: Optional[PipSession] + wheel_cache=None, # type: Optional[WheelCache] + use_pep517=None, # type: Optional[bool] +): + # type: (...) -> Optional[InstallRequirement] + """Handle a single parsed requirements line; This can result in + creating/yielding requirements, or updating the finder. + + For lines that contain requirements, the only options that have an effect + are from SUPPORTED_OPTIONS_REQ, and they are scoped to the + requirement. Other options from SUPPORTED_OPTIONS may be present, but are + ignored. + + For lines that do not contain requirements, the only options that have an + effect are from SUPPORTED_OPTIONS. Options from SUPPORTED_OPTIONS_REQ may + be present, but are ignored. These lines may contain multiple options + (although our docs imply only one is supported), and all our parsed and + affect the finder. + """ + + # preserve for the nested code path + line_comes_from = '%s %s (line %s)' % ( + '-c' if line.constraint else '-r', line.filename, line.lineno, + ) + + # return a line requirement + if line.args: + isolated = options.isolated_mode if options else False + if options: + cmdoptions.check_install_build_global(options, line.opts) + # get the options that apply to requirements + req_options = {} + for dest in SUPPORTED_OPTIONS_REQ_DEST: + if dest in line.opts.__dict__ and line.opts.__dict__[dest]: + req_options[dest] = line.opts.__dict__[dest] + line_source = 'line {} of {}'.format(line.lineno, line.filename) + return install_req_from_line( + line.args, + comes_from=line_comes_from, + use_pep517=use_pep517, + isolated=isolated, + options=req_options, + wheel_cache=wheel_cache, + constraint=line.constraint, + line_source=line_source, + ) + + # return an editable requirement + elif line.opts.editables: + isolated = options.isolated_mode if options else False + return install_req_from_editable( + line.opts.editables[0], comes_from=line_comes_from, + use_pep517=use_pep517, + constraint=line.constraint, isolated=isolated, + wheel_cache=wheel_cache + ) + + # percolate hash-checking option upward + elif line.opts.require_hashes: + options.require_hashes = line.opts.require_hashes + + # set finder options + elif finder: + find_links = finder.find_links + index_urls = finder.index_urls + if line.opts.index_url: + index_urls = [line.opts.index_url] + if line.opts.no_index is True: + index_urls = [] + if line.opts.extra_index_urls: + index_urls.extend(line.opts.extra_index_urls) + if line.opts.find_links: + # FIXME: it would be nice to keep track of the source + # of the find_links: support a find-links local path + # relative to a requirements file. + value = line.opts.find_links[0] + req_dir = os.path.dirname(os.path.abspath(line.filename)) + relative_to_reqs_file = os.path.join(req_dir, value) + if os.path.exists(relative_to_reqs_file): + value = relative_to_reqs_file + find_links.append(value) + + search_scope = SearchScope( + find_links=find_links, + index_urls=index_urls, + ) + finder.search_scope = search_scope + + if line.opts.pre: + finder.set_allow_all_prereleases() + + if session: + for host in line.opts.trusted_hosts or []: + source = 'line {} of {}'.format(line.lineno, line.filename) + session.add_trusted_host(host, source=source) + + return None + + +class RequirementsFileParser(object): + def __init__( + self, + session, # type: PipSession + line_parser, # type: LineParser + comes_from, # type: str + skip_requirements_regex, # type: Optional[str] + ): + # type: (...) -> None + self._session = session + self._line_parser = line_parser + self._comes_from = comes_from + self._skip_requirements_regex = skip_requirements_regex + + def parse(self, filename, constraint): + # type: (str, bool) -> Iterator[ParsedLine] + """Parse a given file, yielding parsed lines. + """ + for line in self._parse_and_recurse(filename, constraint): + yield line + + def _parse_and_recurse(self, filename, constraint): + # type: (str, bool) -> Iterator[ParsedLine] + for line in self._parse_file(filename, constraint): + if ( + not line.args and + not line.opts.editables and + (line.opts.requirements or line.opts.constraints) + ): + # parse a nested requirements file + if line.opts.requirements: + req_path = line.opts.requirements[0] + nested_constraint = False + else: + req_path = line.opts.constraints[0] + nested_constraint = True + + # original file is over http + if SCHEME_RE.search(filename): + # do a url join so relative paths work + req_path = urllib_parse.urljoin(filename, req_path) + # original file and nested file are paths + elif not SCHEME_RE.search(req_path): + # do a join so relative paths work + req_path = os.path.join( + os.path.dirname(filename), req_path, + ) + + for inner_line in self._parse_and_recurse( + req_path, nested_constraint, + ): + yield inner_line + else: + yield line + + def _parse_file(self, filename, constraint): + # type: (str, bool) -> Iterator[ParsedLine] + _, content = get_file_content( + filename, self._session, comes_from=self._comes_from + ) + + lines_enum = preprocess(content, self._skip_requirements_regex) + + for line_number, line in lines_enum: + try: + args_str, opts = self._line_parser(line) + except OptionParsingError as e: + # add offending line + msg = 'Invalid requirement: %s\n%s' % (line, e.msg) + raise RequirementsFileParseError(msg) + + yield ParsedLine( + filename, + line_number, + self._comes_from, + args_str, + opts, + constraint, + ) + + +def get_line_parser(finder): + # type: (Optional[PackageFinder]) -> LineParser + def parse_line(line): + # type: (Text) -> Tuple[str, Values] + # Build new parser for each line since it accumulates appendable + # options. + parser = build_parser() + defaults = parser.get_default_values() + defaults.index_url = None + if finder: + defaults.format_control = finder.format_control + + args_str, options_str = break_args_options(line) + # Prior to 2.7.3, shlex cannot deal with unicode entries + if sys.version_info < (2, 7, 3): + # https://github.com/python/mypy/issues/1174 + options_str = options_str.encode('utf8') # type: ignore + + # https://github.com/python/mypy/issues/1174 + opts, _ = parser.parse_args( + shlex.split(options_str), defaults) # type: ignore + + return args_str, opts + + return parse_line + + +def break_args_options(line): + # type: (Text) -> Tuple[str, Text] + """Break up the line into an args and options string. We only want to shlex + (and then optparse) the options, not the args. args can contain markers + which are corrupted by shlex. + """ + tokens = line.split(' ') + args = [] + options = tokens[:] + for token in tokens: + if token.startswith('-') or token.startswith('--'): + break + else: + args.append(token) + options.pop(0) + return ' '.join(args), ' '.join(options) # type: ignore + + +class OptionParsingError(Exception): + def __init__(self, msg): + # type: (str) -> None + self.msg = msg + + +def build_parser(): + # type: () -> optparse.OptionParser + """ + Return a parser for parsing requirement lines + """ + parser = optparse.OptionParser(add_help_option=False) + + option_factories = SUPPORTED_OPTIONS + SUPPORTED_OPTIONS_REQ + for option_factory in option_factories: + option = option_factory() + parser.add_option(option) + + # By default optparse sys.exits on parsing errors. We want to wrap + # that in our own exception. + def parser_exit(self, msg): + # type: (Any, str) -> NoReturn + raise OptionParsingError(msg) + # NOTE: mypy disallows assigning to a method + # https://github.com/python/mypy/issues/2427 + parser.exit = parser_exit # type: ignore + + return parser + + +def join_lines(lines_enum): + # type: (ReqFileLines) -> ReqFileLines + """Joins a line ending in '\' with the previous line (except when following + comments). The joined line takes on the index of the first line. + """ + primary_line_number = None + new_line = [] # type: List[Text] + for line_number, line in lines_enum: + if not line.endswith('\\') or COMMENT_RE.match(line): + if COMMENT_RE.match(line): + # this ensures comments are always matched later + line = ' ' + line + if new_line: + new_line.append(line) + yield primary_line_number, ''.join(new_line) + new_line = [] + else: + yield line_number, line + else: + if not new_line: + primary_line_number = line_number + new_line.append(line.strip('\\')) + + # last line contains \ + if new_line: + yield primary_line_number, ''.join(new_line) + + # TODO: handle space after '\'. + + +def ignore_comments(lines_enum): + # type: (ReqFileLines) -> ReqFileLines + """ + Strips comments and filter empty lines. + """ + for line_number, line in lines_enum: + line = COMMENT_RE.sub('', line) + line = line.strip() + if line: + yield line_number, line + + +def skip_regex(lines_enum, pattern): + # type: (ReqFileLines, str) -> ReqFileLines + """ + Skip lines that match the provided pattern + + Note: the regex pattern is only built once + """ + matcher = re.compile(pattern) + lines_enum = filterfalse(lambda e: matcher.search(e[1]), lines_enum) + return lines_enum + + +def expand_env_variables(lines_enum): + # type: (ReqFileLines) -> ReqFileLines + """Replace all environment variables that can be retrieved via `os.getenv`. + + The only allowed format for environment variables defined in the + requirement file is `${MY_VARIABLE_1}` to ensure two things: + + 1. Strings that contain a `$` aren't accidentally (partially) expanded. + 2. Ensure consistency across platforms for requirement files. + + These points are the result of a discussion on the `github pull + request #3514 `_. + + Valid characters in variable names follow the `POSIX standard + `_ and are limited + to uppercase letter, digits and the `_` (underscore). + """ + for line_number, line in lines_enum: + for env_var, var_name in ENV_VAR_RE.findall(line): + value = os.getenv(var_name) + if not value: + continue + + line = line.replace(env_var, value) + + yield line_number, line + + +def get_file_content(url, session, comes_from=None): + # type: (str, PipSession, Optional[str]) -> Tuple[str, Text] + """Gets the content of a file; it may be a filename, file: URL, or + http: URL. Returns (location, content). Content is unicode. + Respects # -*- coding: declarations on the retrieved files. + + :param url: File path or url. + :param session: PipSession instance. + :param comes_from: Origin description of requirements. + """ + scheme = get_url_scheme(url) + + if scheme in ['http', 'https']: + # FIXME: catch some errors + resp = session.get(url) + resp.raise_for_status() + return resp.url, resp.text + + elif scheme == 'file': + if comes_from and comes_from.startswith('http'): + raise InstallationError( + 'Requirements file %s references URL %s, which is local' + % (comes_from, url)) + + path = url.split(':', 1)[1] + path = path.replace('\\', '/') + match = _url_slash_drive_re.match(path) + if match: + path = match.group(1) + ':' + path.split('|', 1)[1] + path = urllib_parse.unquote(path) + if path.startswith('/'): + path = '/' + path.lstrip('/') + url = path + + try: + with open(url, 'rb') as f: + content = auto_decode(f.read()) + except IOError as exc: + raise InstallationError( + 'Could not open requirements file: %s' % str(exc) + ) + return url, content + + +_url_slash_drive_re = re.compile(r'/*([a-z])\|', re.I) diff --git a/env/lib/python3.8/site-packages/pip/_internal/req/req_install.py b/env/lib/python3.8/site-packages/pip/_internal/req/req_install.py new file mode 100644 index 000000000..22ac24b96 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/req/req_install.py @@ -0,0 +1,830 @@ +# The following comment should be removed at some point in the future. +# mypy: strict-optional=False + +from __future__ import absolute_import + +import logging +import os +import shutil +import sys +import zipfile + +from pip._vendor import pkg_resources, six +from pip._vendor.packaging.requirements import Requirement +from pip._vendor.packaging.utils import canonicalize_name +from pip._vendor.packaging.version import Version +from pip._vendor.packaging.version import parse as parse_version +from pip._vendor.pep517.wrappers import Pep517HookCaller + +from pip._internal import pep425tags +from pip._internal.build_env import NoOpBuildEnvironment +from pip._internal.exceptions import InstallationError +from pip._internal.locations import get_scheme +from pip._internal.models.link import Link +from pip._internal.operations.build.metadata import generate_metadata +from pip._internal.operations.build.metadata_legacy import \ + generate_metadata as generate_metadata_legacy +from pip._internal.operations.install.editable_legacy import \ + install_editable as install_editable_legacy +from pip._internal.operations.install.legacy import install as install_legacy +from pip._internal.operations.install.wheel import install_wheel +from pip._internal.pyproject import load_pyproject_toml, make_pyproject_path +from pip._internal.req.req_uninstall import UninstallPathSet +from pip._internal.utils.deprecation import deprecated +from pip._internal.utils.hashes import Hashes +from pip._internal.utils.logging import indent_log +from pip._internal.utils.marker_files import ( + PIP_DELETE_MARKER_FILENAME, + has_delete_marker_file, + write_delete_marker_file, +) +from pip._internal.utils.misc import ( + ask_path_exists, + backup_dir, + display_path, + dist_in_site_packages, + dist_in_usersite, + get_installed_version, + hide_url, + redact_auth_from_url, + rmtree, +) +from pip._internal.utils.packaging import get_metadata +from pip._internal.utils.temp_dir import TempDirectory +from pip._internal.utils.typing import MYPY_CHECK_RUNNING +from pip._internal.utils.virtualenv import running_under_virtualenv +from pip._internal.vcs import vcs + +if MYPY_CHECK_RUNNING: + from typing import ( + Any, Dict, Iterable, List, Optional, Sequence, Union, + ) + from pip._internal.build_env import BuildEnvironment + from pip._internal.cache import WheelCache + from pip._internal.index.package_finder import PackageFinder + from pip._vendor.pkg_resources import Distribution + from pip._vendor.packaging.specifiers import SpecifierSet + from pip._vendor.packaging.markers import Marker + + +logger = logging.getLogger(__name__) + + +def _get_dist(metadata_directory): + # type: (str) -> Distribution + """Return a pkg_resources.Distribution for the provided + metadata directory. + """ + dist_dir = metadata_directory.rstrip(os.sep) + + # Determine the correct Distribution object type. + if dist_dir.endswith(".egg-info"): + dist_cls = pkg_resources.Distribution + else: + assert dist_dir.endswith(".dist-info") + dist_cls = pkg_resources.DistInfoDistribution + + # Build a PathMetadata object, from path to metadata. :wink: + base_dir, dist_dir_name = os.path.split(dist_dir) + dist_name = os.path.splitext(dist_dir_name)[0] + metadata = pkg_resources.PathMetadata(base_dir, dist_dir) + + return dist_cls( + base_dir, + project_name=dist_name, + metadata=metadata, + ) + + +class InstallRequirement(object): + """ + Represents something that may be installed later on, may have information + about where to fetch the relevant requirement and also contains logic for + installing the said requirement. + """ + + def __init__( + self, + req, # type: Optional[Requirement] + comes_from, # type: Optional[Union[str, InstallRequirement]] + source_dir=None, # type: Optional[str] + editable=False, # type: bool + link=None, # type: Optional[Link] + markers=None, # type: Optional[Marker] + use_pep517=None, # type: Optional[bool] + isolated=False, # type: bool + options=None, # type: Optional[Dict[str, Any]] + wheel_cache=None, # type: Optional[WheelCache] + constraint=False, # type: bool + extras=() # type: Iterable[str] + ): + # type: (...) -> None + assert req is None or isinstance(req, Requirement), req + self.req = req + self.comes_from = comes_from + self.constraint = constraint + if source_dir is None: + self.source_dir = None # type: Optional[str] + else: + self.source_dir = os.path.normpath(os.path.abspath(source_dir)) + self.editable = editable + + self._wheel_cache = wheel_cache + if link is None and req and req.url: + # PEP 508 URL requirement + link = Link(req.url) + self.link = self.original_link = link + # Path to any downloaded or already-existing package. + self.local_file_path = None # type: Optional[str] + if self.link and self.link.is_file: + self.local_file_path = self.link.file_path + + if extras: + self.extras = extras + elif req: + self.extras = { + pkg_resources.safe_extra(extra) for extra in req.extras + } + else: + self.extras = set() + if markers is None and req: + markers = req.marker + self.markers = markers + + # This holds the pkg_resources.Distribution object if this requirement + # is already available: + self.satisfied_by = None # type: Optional[Distribution] + # Whether the installation process should try to uninstall an existing + # distribution before installing this requirement. + self.should_reinstall = False + # Temporary build location + self._temp_build_dir = None # type: Optional[TempDirectory] + # Set to True after successful installation + self.install_succeeded = None # type: Optional[bool] + self.options = options if options else {} + # Set to True after successful preparation of this requirement + self.prepared = False + self.is_direct = False + + self.isolated = isolated + self.build_env = NoOpBuildEnvironment() # type: BuildEnvironment + + # For PEP 517, the directory where we request the project metadata + # gets stored. We need this to pass to build_wheel, so the backend + # can ensure that the wheel matches the metadata (see the PEP for + # details). + self.metadata_directory = None # type: Optional[str] + + # The static build requirements (from pyproject.toml) + self.pyproject_requires = None # type: Optional[List[str]] + + # Build requirements that we will check are available + self.requirements_to_check = [] # type: List[str] + + # The PEP 517 backend we should use to build the project + self.pep517_backend = None # type: Optional[Pep517HookCaller] + + # Are we using PEP 517 for this requirement? + # After pyproject.toml has been loaded, the only valid values are True + # and False. Before loading, None is valid (meaning "use the default"). + # Setting an explicit value before loading pyproject.toml is supported, + # but after loading this flag should be treated as read only. + self.use_pep517 = use_pep517 + + def __str__(self): + # type: () -> str + if self.req: + s = str(self.req) + if self.link: + s += ' from %s' % redact_auth_from_url(self.link.url) + elif self.link: + s = redact_auth_from_url(self.link.url) + else: + s = '' + if self.satisfied_by is not None: + s += ' in %s' % display_path(self.satisfied_by.location) + if self.comes_from: + if isinstance(self.comes_from, six.string_types): + comes_from = self.comes_from # type: Optional[str] + else: + comes_from = self.comes_from.from_path() + if comes_from: + s += ' (from %s)' % comes_from + return s + + def __repr__(self): + # type: () -> str + return '<%s object: %s editable=%r>' % ( + self.__class__.__name__, str(self), self.editable) + + def format_debug(self): + # type: () -> str + """An un-tested helper for getting state, for debugging. + """ + attributes = vars(self) + names = sorted(attributes) + + state = ( + "{}={!r}".format(attr, attributes[attr]) for attr in sorted(names) + ) + return '<{name} object: {{{state}}}>'.format( + name=self.__class__.__name__, + state=", ".join(state), + ) + + def populate_link(self, finder, upgrade, require_hashes): + # type: (PackageFinder, bool, bool) -> None + """Ensure that if a link can be found for this, that it is found. + + Note that self.link may still be None - if Upgrade is False and the + requirement is already installed. + + If require_hashes is True, don't use the wheel cache, because cached + wheels, always built locally, have different hashes than the files + downloaded from the index server and thus throw false hash mismatches. + Furthermore, cached wheels at present have undeterministic contents due + to file modification times. + """ + if self.link is None: + self.link = finder.find_requirement(self, upgrade) + if self._wheel_cache is not None and not require_hashes: + old_link = self.link + supported_tags = pep425tags.get_supported() + self.link = self._wheel_cache.get( + link=self.link, + package_name=self.name, + supported_tags=supported_tags, + ) + if old_link != self.link: + logger.debug('Using cached wheel link: %s', self.link) + + # Things that are valid for all kinds of requirements? + @property + def name(self): + # type: () -> Optional[str] + if self.req is None: + return None + return six.ensure_str(pkg_resources.safe_name(self.req.name)) + + @property + def specifier(self): + # type: () -> SpecifierSet + return self.req.specifier + + @property + def is_pinned(self): + # type: () -> bool + """Return whether I am pinned to an exact version. + + For example, some-package==1.2 is pinned; some-package>1.2 is not. + """ + specifiers = self.specifier + return (len(specifiers) == 1 and + next(iter(specifiers)).operator in {'==', '==='}) + + @property + def installed_version(self): + # type: () -> Optional[str] + return get_installed_version(self.name) + + def match_markers(self, extras_requested=None): + # type: (Optional[Iterable[str]]) -> bool + if not extras_requested: + # Provide an extra to safely evaluate the markers + # without matching any extra + extras_requested = ('',) + if self.markers is not None: + return any( + self.markers.evaluate({'extra': extra}) + for extra in extras_requested) + else: + return True + + @property + def has_hash_options(self): + # type: () -> bool + """Return whether any known-good hashes are specified as options. + + These activate --require-hashes mode; hashes specified as part of a + URL do not. + + """ + return bool(self.options.get('hashes', {})) + + def hashes(self, trust_internet=True): + # type: (bool) -> Hashes + """Return a hash-comparer that considers my option- and URL-based + hashes to be known-good. + + Hashes in URLs--ones embedded in the requirements file, not ones + downloaded from an index server--are almost peers with ones from + flags. They satisfy --require-hashes (whether it was implicitly or + explicitly activated) but do not activate it. md5 and sha224 are not + allowed in flags, which should nudge people toward good algos. We + always OR all hashes together, even ones from URLs. + + :param trust_internet: Whether to trust URL-based (#md5=...) hashes + downloaded from the internet, as by populate_link() + + """ + good_hashes = self.options.get('hashes', {}).copy() + link = self.link if trust_internet else self.original_link + if link and link.hash: + good_hashes.setdefault(link.hash_name, []).append(link.hash) + return Hashes(good_hashes) + + def from_path(self): + # type: () -> Optional[str] + """Format a nice indicator to show where this "comes from" + """ + if self.req is None: + return None + s = str(self.req) + if self.comes_from: + if isinstance(self.comes_from, six.string_types): + comes_from = self.comes_from + else: + comes_from = self.comes_from.from_path() + if comes_from: + s += '->' + comes_from + return s + + def ensure_build_location(self, build_dir): + # type: (str) -> str + assert build_dir is not None + if self._temp_build_dir is not None: + assert self._temp_build_dir.path + return self._temp_build_dir.path + if self.req is None: + # Some systems have /tmp as a symlink which confuses custom + # builds (such as numpy). Thus, we ensure that the real path + # is returned. + self._temp_build_dir = TempDirectory(kind="req-build") + + return self._temp_build_dir.path + if self.editable: + name = self.name.lower() + else: + name = self.name + # FIXME: Is there a better place to create the build_dir? (hg and bzr + # need this) + if not os.path.exists(build_dir): + logger.debug('Creating directory %s', build_dir) + os.makedirs(build_dir) + write_delete_marker_file(build_dir) + return os.path.join(build_dir, name) + + def _set_requirement(self): + # type: () -> None + """Set requirement after generating metadata. + """ + assert self.req is None + assert self.metadata is not None + assert self.source_dir is not None + + # Construct a Requirement object from the generated metadata + if isinstance(parse_version(self.metadata["Version"]), Version): + op = "==" + else: + op = "===" + + self.req = Requirement( + "".join([ + self.metadata["Name"], + op, + self.metadata["Version"], + ]) + ) + + def warn_on_mismatching_name(self): + # type: () -> None + metadata_name = canonicalize_name(self.metadata["Name"]) + if canonicalize_name(self.req.name) == metadata_name: + # Everything is fine. + return + + # If we're here, there's a mismatch. Log a warning about it. + logger.warning( + 'Generating metadata for package %s ' + 'produced metadata for project name %s. Fix your ' + '#egg=%s fragments.', + self.name, metadata_name, self.name + ) + self.req = Requirement(metadata_name) + + def remove_temporary_source(self): + # type: () -> None + """Remove the source files from this requirement, if they are marked + for deletion""" + if self.source_dir and has_delete_marker_file(self.source_dir): + logger.debug('Removing source in %s', self.source_dir) + rmtree(self.source_dir) + self.source_dir = None + if self._temp_build_dir: + self._temp_build_dir.cleanup() + self._temp_build_dir = None + self.build_env.cleanup() + + def check_if_exists(self, use_user_site): + # type: (bool) -> None + """Find an installed distribution that satisfies or conflicts + with this requirement, and set self.satisfied_by or + self.should_reinstall appropriately. + """ + if self.req is None: + return + # get_distribution() will resolve the entire list of requirements + # anyway, and we've already determined that we need the requirement + # in question, so strip the marker so that we don't try to + # evaluate it. + no_marker = Requirement(str(self.req)) + no_marker.marker = None + try: + self.satisfied_by = pkg_resources.get_distribution(str(no_marker)) + except pkg_resources.DistributionNotFound: + return + except pkg_resources.VersionConflict: + existing_dist = pkg_resources.get_distribution( + self.req.name + ) + if use_user_site: + if dist_in_usersite(existing_dist): + self.should_reinstall = True + elif (running_under_virtualenv() and + dist_in_site_packages(existing_dist)): + raise InstallationError( + "Will not install to the user site because it will " + "lack sys.path precedence to %s in %s" % + (existing_dist.project_name, existing_dist.location) + ) + else: + self.should_reinstall = True + else: + if self.editable and self.satisfied_by: + self.should_reinstall = True + # when installing editables, nothing pre-existing should ever + # satisfy + self.satisfied_by = None + + # Things valid for wheels + @property + def is_wheel(self): + # type: () -> bool + if not self.link: + return False + return self.link.is_wheel + + # Things valid for sdists + @property + def unpacked_source_directory(self): + # type: () -> str + return os.path.join( + self.source_dir, + self.link and self.link.subdirectory_fragment or '') + + @property + def setup_py_path(self): + # type: () -> str + assert self.source_dir, "No source dir for %s" % self + setup_py = os.path.join(self.unpacked_source_directory, 'setup.py') + + # Python2 __file__ should not be unicode + if six.PY2 and isinstance(setup_py, six.text_type): + setup_py = setup_py.encode(sys.getfilesystemencoding()) + + return setup_py + + @property + def pyproject_toml_path(self): + # type: () -> str + assert self.source_dir, "No source dir for %s" % self + return make_pyproject_path(self.unpacked_source_directory) + + def load_pyproject_toml(self): + # type: () -> None + """Load the pyproject.toml file. + + After calling this routine, all of the attributes related to PEP 517 + processing for this requirement have been set. In particular, the + use_pep517 attribute can be used to determine whether we should + follow the PEP 517 or legacy (setup.py) code path. + """ + pyproject_toml_data = load_pyproject_toml( + self.use_pep517, + self.pyproject_toml_path, + self.setup_py_path, + str(self) + ) + + if pyproject_toml_data is None: + self.use_pep517 = False + return + + self.use_pep517 = True + requires, backend, check, backend_path = pyproject_toml_data + self.requirements_to_check = check + self.pyproject_requires = requires + self.pep517_backend = Pep517HookCaller( + self.unpacked_source_directory, backend, backend_path=backend_path, + ) + + def _generate_metadata(self): + # type: () -> str + """Invokes metadata generator functions, with the required arguments. + """ + if not self.use_pep517: + assert self.unpacked_source_directory + + return generate_metadata_legacy( + build_env=self.build_env, + setup_py_path=self.setup_py_path, + source_dir=self.unpacked_source_directory, + editable=self.editable, + isolated=self.isolated, + details=self.name or "from {}".format(self.link) + ) + + assert self.pep517_backend is not None + + return generate_metadata( + build_env=self.build_env, + backend=self.pep517_backend, + ) + + def prepare_metadata(self): + # type: () -> None + """Ensure that project metadata is available. + + Under PEP 517, call the backend hook to prepare the metadata. + Under legacy processing, call setup.py egg-info. + """ + assert self.source_dir + + with indent_log(): + self.metadata_directory = self._generate_metadata() + + # Act on the newly generated metadata, based on the name and version. + if not self.name: + self._set_requirement() + else: + self.warn_on_mismatching_name() + + self.assert_source_matches_version() + + @property + def metadata(self): + # type: () -> Any + if not hasattr(self, '_metadata'): + self._metadata = get_metadata(self.get_dist()) + + return self._metadata + + def get_dist(self): + # type: () -> Distribution + return _get_dist(self.metadata_directory) + + def assert_source_matches_version(self): + # type: () -> None + assert self.source_dir + version = self.metadata['version'] + if self.req.specifier and version not in self.req.specifier: + logger.warning( + 'Requested %s, but installing version %s', + self, + version, + ) + else: + logger.debug( + 'Source in %s has version %s, which satisfies requirement %s', + display_path(self.source_dir), + version, + self, + ) + + # For both source distributions and editables + def ensure_has_source_dir(self, parent_dir): + # type: (str) -> None + """Ensure that a source_dir is set. + + This will create a temporary build dir if the name of the requirement + isn't known yet. + + :param parent_dir: The ideal pip parent_dir for the source_dir. + Generally src_dir for editables and build_dir for sdists. + :return: self.source_dir + """ + if self.source_dir is None: + self.source_dir = self.ensure_build_location(parent_dir) + + # For editable installations + def update_editable(self, obtain=True): + # type: (bool) -> None + if not self.link: + logger.debug( + "Cannot update repository at %s; repository location is " + "unknown", + self.source_dir, + ) + return + assert self.editable + assert self.source_dir + if self.link.scheme == 'file': + # Static paths don't get updated + return + assert '+' in self.link.url, "bad url: %r" % self.link.url + vc_type, url = self.link.url.split('+', 1) + vcs_backend = vcs.get_backend(vc_type) + if vcs_backend: + if not self.link.is_vcs: + reason = ( + "This form of VCS requirement is being deprecated: {}." + ).format( + self.link.url + ) + replacement = None + if self.link.url.startswith("git+git@"): + replacement = ( + "git+https://git@example.com/..., " + "git+ssh://git@example.com/..., " + "or the insecure git+git://git@example.com/..." + ) + deprecated(reason, replacement, gone_in="21.0", issue=7554) + hidden_url = hide_url(self.link.url) + if obtain: + vcs_backend.obtain(self.source_dir, url=hidden_url) + else: + vcs_backend.export(self.source_dir, url=hidden_url) + else: + assert 0, ( + 'Unexpected version control type (in %s): %s' + % (self.link, vc_type)) + + # Top-level Actions + def uninstall(self, auto_confirm=False, verbose=False): + # type: (bool, bool) -> Optional[UninstallPathSet] + """ + Uninstall the distribution currently satisfying this requirement. + + Prompts before removing or modifying files unless + ``auto_confirm`` is True. + + Refuses to delete or modify files outside of ``sys.prefix`` - + thus uninstallation within a virtual environment can only + modify that virtual environment, even if the virtualenv is + linked to global site-packages. + + """ + assert self.req + try: + dist = pkg_resources.get_distribution(self.req.name) + except pkg_resources.DistributionNotFound: + logger.warning("Skipping %s as it is not installed.", self.name) + return None + else: + logger.info('Found existing installation: %s', dist) + + uninstalled_pathset = UninstallPathSet.from_dist(dist) + uninstalled_pathset.remove(auto_confirm, verbose) + return uninstalled_pathset + + def _get_archive_name(self, path, parentdir, rootdir): + # type: (str, str, str) -> str + + def _clean_zip_name(name, prefix): + # type: (str, str) -> str + assert name.startswith(prefix + os.path.sep), ( + "name %r doesn't start with prefix %r" % (name, prefix) + ) + name = name[len(prefix) + 1:] + name = name.replace(os.path.sep, '/') + return name + + path = os.path.join(parentdir, path) + name = _clean_zip_name(path, rootdir) + return self.name + '/' + name + + def archive(self, build_dir): + # type: (str) -> None + """Saves archive to provided build_dir. + + Used for saving downloaded VCS requirements as part of `pip download`. + """ + assert self.source_dir + + create_archive = True + archive_name = '%s-%s.zip' % (self.name, self.metadata["version"]) + archive_path = os.path.join(build_dir, archive_name) + + if os.path.exists(archive_path): + response = ask_path_exists( + 'The file %s exists. (i)gnore, (w)ipe, (b)ackup, (a)bort ' % + display_path(archive_path), ('i', 'w', 'b', 'a')) + if response == 'i': + create_archive = False + elif response == 'w': + logger.warning('Deleting %s', display_path(archive_path)) + os.remove(archive_path) + elif response == 'b': + dest_file = backup_dir(archive_path) + logger.warning( + 'Backing up %s to %s', + display_path(archive_path), + display_path(dest_file), + ) + shutil.move(archive_path, dest_file) + elif response == 'a': + sys.exit(-1) + + if not create_archive: + return + + zip_output = zipfile.ZipFile( + archive_path, 'w', zipfile.ZIP_DEFLATED, allowZip64=True, + ) + with zip_output: + dir = os.path.normcase( + os.path.abspath(self.unpacked_source_directory) + ) + for dirpath, dirnames, filenames in os.walk(dir): + if 'pip-egg-info' in dirnames: + dirnames.remove('pip-egg-info') + for dirname in dirnames: + dir_arcname = self._get_archive_name( + dirname, parentdir=dirpath, rootdir=dir, + ) + zipdir = zipfile.ZipInfo(dir_arcname + '/') + zipdir.external_attr = 0x1ED << 16 # 0o755 + zip_output.writestr(zipdir, '') + for filename in filenames: + if filename == PIP_DELETE_MARKER_FILENAME: + continue + file_arcname = self._get_archive_name( + filename, parentdir=dirpath, rootdir=dir, + ) + filename = os.path.join(dirpath, filename) + zip_output.write(filename, file_arcname) + + logger.info('Saved %s', display_path(archive_path)) + + def install( + self, + install_options, # type: List[str] + global_options=None, # type: Optional[Sequence[str]] + root=None, # type: Optional[str] + home=None, # type: Optional[str] + prefix=None, # type: Optional[str] + warn_script_location=True, # type: bool + use_user_site=False, # type: bool + pycompile=True # type: bool + ): + # type: (...) -> None + scheme = get_scheme( + self.name, + user=use_user_site, + home=home, + root=root, + isolated=self.isolated, + prefix=prefix, + ) + + global_options = global_options if global_options is not None else [] + if self.editable: + install_editable_legacy( + install_options, + global_options, + prefix=prefix, + home=home, + use_user_site=use_user_site, + name=self.name, + setup_py_path=self.setup_py_path, + isolated=self.isolated, + build_env=self.build_env, + unpacked_source_directory=self.unpacked_source_directory, + ) + self.install_succeeded = True + return + + if self.is_wheel: + assert self.local_file_path + install_wheel( + self.name, + self.local_file_path, + scheme=scheme, + req_description=str(self.req), + pycompile=pycompile, + warn_script_location=warn_script_location, + ) + self.install_succeeded = True + return + + install_legacy( + self, + install_options=install_options, + global_options=global_options, + root=root, + home=home, + prefix=prefix, + use_user_site=use_user_site, + pycompile=pycompile, + scheme=scheme, + ) diff --git a/env/lib/python3.8/site-packages/pip/_internal/req/req_set.py b/env/lib/python3.8/site-packages/pip/_internal/req/req_set.py new file mode 100644 index 000000000..087ac5925 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/req/req_set.py @@ -0,0 +1,209 @@ +# The following comment should be removed at some point in the future. +# mypy: strict-optional=False + +from __future__ import absolute_import + +import logging +from collections import OrderedDict + +from pip._vendor.packaging.utils import canonicalize_name + +from pip._internal import pep425tags +from pip._internal.exceptions import InstallationError +from pip._internal.models.wheel import Wheel +from pip._internal.utils.logging import indent_log +from pip._internal.utils.typing import MYPY_CHECK_RUNNING + +if MYPY_CHECK_RUNNING: + from typing import Dict, Iterable, List, Optional, Tuple + from pip._internal.req.req_install import InstallRequirement + + +logger = logging.getLogger(__name__) + + +class RequirementSet(object): + + def __init__(self, check_supported_wheels=True): + # type: (bool) -> None + """Create a RequirementSet. + """ + + self.requirements = OrderedDict() # type: Dict[str, InstallRequirement] # noqa: E501 + self.check_supported_wheels = check_supported_wheels + + self.unnamed_requirements = [] # type: List[InstallRequirement] + self.successfully_downloaded = [] # type: List[InstallRequirement] + self.reqs_to_cleanup = [] # type: List[InstallRequirement] + + def __str__(self): + # type: () -> str + requirements = sorted( + (req for req in self.requirements.values() if not req.comes_from), + key=lambda req: canonicalize_name(req.name), + ) + return ' '.join(str(req.req) for req in requirements) + + def __repr__(self): + # type: () -> str + requirements = sorted( + self.requirements.values(), + key=lambda req: canonicalize_name(req.name), + ) + + format_string = '<{classname} object; {count} requirement(s): {reqs}>' + return format_string.format( + classname=self.__class__.__name__, + count=len(requirements), + reqs=', '.join(str(req.req) for req in requirements), + ) + + def add_unnamed_requirement(self, install_req): + # type: (InstallRequirement) -> None + assert not install_req.name + self.unnamed_requirements.append(install_req) + + def add_named_requirement(self, install_req): + # type: (InstallRequirement) -> None + assert install_req.name + + project_name = canonicalize_name(install_req.name) + self.requirements[project_name] = install_req + + def add_requirement( + self, + install_req, # type: InstallRequirement + parent_req_name=None, # type: Optional[str] + extras_requested=None # type: Optional[Iterable[str]] + ): + # type: (...) -> Tuple[List[InstallRequirement], Optional[InstallRequirement]] # noqa: E501 + """Add install_req as a requirement to install. + + :param parent_req_name: The name of the requirement that needed this + added. The name is used because when multiple unnamed requirements + resolve to the same name, we could otherwise end up with dependency + links that point outside the Requirements set. parent_req must + already be added. Note that None implies that this is a user + supplied requirement, vs an inferred one. + :param extras_requested: an iterable of extras used to evaluate the + environment markers. + :return: Additional requirements to scan. That is either [] if + the requirement is not applicable, or [install_req] if the + requirement is applicable and has just been added. + """ + # If the markers do not match, ignore this requirement. + if not install_req.match_markers(extras_requested): + logger.info( + "Ignoring %s: markers '%s' don't match your environment", + install_req.name, install_req.markers, + ) + return [], None + + # If the wheel is not supported, raise an error. + # Should check this after filtering out based on environment markers to + # allow specifying different wheels based on the environment/OS, in a + # single requirements file. + if install_req.link and install_req.link.is_wheel: + wheel = Wheel(install_req.link.filename) + tags = pep425tags.get_supported() + if (self.check_supported_wheels and not wheel.supported(tags)): + raise InstallationError( + "%s is not a supported wheel on this platform." % + wheel.filename + ) + + # This next bit is really a sanity check. + assert install_req.is_direct == (parent_req_name is None), ( + "a direct req shouldn't have a parent and also, " + "a non direct req should have a parent" + ) + + # Unnamed requirements are scanned again and the requirement won't be + # added as a dependency until after scanning. + if not install_req.name: + self.add_unnamed_requirement(install_req) + return [install_req], None + + try: + existing_req = self.get_requirement(install_req.name) + except KeyError: + existing_req = None + + has_conflicting_requirement = ( + parent_req_name is None and + existing_req and + not existing_req.constraint and + existing_req.extras == install_req.extras and + existing_req.req.specifier != install_req.req.specifier + ) + if has_conflicting_requirement: + raise InstallationError( + "Double requirement given: %s (already in %s, name=%r)" + % (install_req, existing_req, install_req.name) + ) + + # When no existing requirement exists, add the requirement as a + # dependency and it will be scanned again after. + if not existing_req: + self.add_named_requirement(install_req) + # We'd want to rescan this requirement later + return [install_req], install_req + + # Assume there's no need to scan, and that we've already + # encountered this for scanning. + if install_req.constraint or not existing_req.constraint: + return [], existing_req + + does_not_satisfy_constraint = ( + install_req.link and + not ( + existing_req.link and + install_req.link.path == existing_req.link.path + ) + ) + if does_not_satisfy_constraint: + self.reqs_to_cleanup.append(install_req) + raise InstallationError( + "Could not satisfy constraints for '%s': " + "installation from path or url cannot be " + "constrained to a version" % install_req.name, + ) + # If we're now installing a constraint, mark the existing + # object for real installation. + existing_req.constraint = False + existing_req.extras = tuple(sorted( + set(existing_req.extras) | set(install_req.extras) + )) + logger.debug( + "Setting %s extras to: %s", + existing_req, existing_req.extras, + ) + # Return the existing requirement for addition to the parent and + # scanning again. + return [existing_req], existing_req + + def has_requirement(self, name): + # type: (str) -> bool + project_name = canonicalize_name(name) + + return ( + project_name in self.requirements and + not self.requirements[project_name].constraint + ) + + def get_requirement(self, name): + # type: (str) -> InstallRequirement + project_name = canonicalize_name(name) + + if project_name in self.requirements: + return self.requirements[project_name] + + raise KeyError("No project with the name %r" % name) + + def cleanup_files(self): + # type: () -> None + """Clean up files, remove builds.""" + logger.debug('Cleaning up...') + with indent_log(): + for req in self.reqs_to_cleanup: + req.remove_temporary_source() diff --git a/env/lib/python3.8/site-packages/pip/_internal/req/req_tracker.py b/env/lib/python3.8/site-packages/pip/_internal/req/req_tracker.py new file mode 100644 index 000000000..84e0c0419 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/req/req_tracker.py @@ -0,0 +1,150 @@ +# The following comment should be removed at some point in the future. +# mypy: strict-optional=False + +from __future__ import absolute_import + +import contextlib +import errno +import hashlib +import logging +import os + +from pip._vendor import contextlib2 + +from pip._internal.utils.temp_dir import TempDirectory +from pip._internal.utils.typing import MYPY_CHECK_RUNNING + +if MYPY_CHECK_RUNNING: + from types import TracebackType + from typing import Dict, Iterator, Optional, Set, Type, Union + from pip._internal.req.req_install import InstallRequirement + from pip._internal.models.link import Link + +logger = logging.getLogger(__name__) + + +@contextlib.contextmanager +def update_env_context_manager(**changes): + # type: (str) -> Iterator[None] + target = os.environ + + # Save values from the target and change them. + non_existent_marker = object() + saved_values = {} # type: Dict[str, Union[object, str]] + for name, new_value in changes.items(): + try: + saved_values[name] = target[name] + except KeyError: + saved_values[name] = non_existent_marker + target[name] = new_value + + try: + yield + finally: + # Restore original values in the target. + for name, original_value in saved_values.items(): + if original_value is non_existent_marker: + del target[name] + else: + assert isinstance(original_value, str) # for mypy + target[name] = original_value + + +@contextlib.contextmanager +def get_requirement_tracker(): + # type: () -> Iterator[RequirementTracker] + root = os.environ.get('PIP_REQ_TRACKER') + with contextlib2.ExitStack() as ctx: + if root is None: + root = ctx.enter_context( + TempDirectory(kind='req-tracker') + ).path + ctx.enter_context(update_env_context_manager(PIP_REQ_TRACKER=root)) + logger.debug("Initialized build tracking at %s", root) + + with RequirementTracker(root) as tracker: + yield tracker + + +class RequirementTracker(object): + + def __init__(self, root): + # type: (str) -> None + self._root = root + self._entries = set() # type: Set[InstallRequirement] + logger.debug("Created build tracker: %s", self._root) + + def __enter__(self): + # type: () -> RequirementTracker + logger.debug("Entered build tracker: %s", self._root) + return self + + def __exit__( + self, + exc_type, # type: Optional[Type[BaseException]] + exc_val, # type: Optional[BaseException] + exc_tb # type: Optional[TracebackType] + ): + # type: (...) -> None + self.cleanup() + + def _entry_path(self, link): + # type: (Link) -> str + hashed = hashlib.sha224(link.url_without_fragment.encode()).hexdigest() + return os.path.join(self._root, hashed) + + def add(self, req): + # type: (InstallRequirement) -> None + """Add an InstallRequirement to build tracking. + """ + + # Get the file to write information about this requirement. + entry_path = self._entry_path(req.link) + + # Try reading from the file. If it exists and can be read from, a build + # is already in progress, so a LookupError is raised. + try: + with open(entry_path) as fp: + contents = fp.read() + except IOError as e: + # if the error is anything other than "file does not exist", raise. + if e.errno != errno.ENOENT: + raise + else: + message = '%s is already being built: %s' % (req.link, contents) + raise LookupError(message) + + # If we're here, req should really not be building already. + assert req not in self._entries + + # Start tracking this requirement. + with open(entry_path, 'w') as fp: + fp.write(str(req)) + self._entries.add(req) + + logger.debug('Added %s to build tracker %r', req, self._root) + + def remove(self, req): + # type: (InstallRequirement) -> None + """Remove an InstallRequirement from build tracking. + """ + + # Delete the created file and the corresponding entries. + os.unlink(self._entry_path(req.link)) + self._entries.remove(req) + + logger.debug('Removed %s from build tracker %r', req, self._root) + + def cleanup(self): + # type: () -> None + for req in set(self._entries): + self.remove(req) + + logger.debug("Removed build tracker: %r", self._root) + + @contextlib.contextmanager + def track(self, req): + # type: (InstallRequirement) -> Iterator[None] + self.add(req) + yield + self.remove(req) diff --git a/env/lib/python3.8/site-packages/pip/_internal/req/req_uninstall.py b/env/lib/python3.8/site-packages/pip/_internal/req/req_uninstall.py new file mode 100644 index 000000000..5971b130e --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/req/req_uninstall.py @@ -0,0 +1,644 @@ +from __future__ import absolute_import + +import csv +import functools +import logging +import os +import sys +import sysconfig + +from pip._vendor import pkg_resources + +from pip._internal.exceptions import UninstallationError +from pip._internal.locations import bin_py, bin_user +from pip._internal.utils.compat import WINDOWS, cache_from_source, uses_pycache +from pip._internal.utils.logging import indent_log +from pip._internal.utils.misc import ( + FakeFile, + ask, + dist_in_usersite, + dist_is_local, + egg_link_path, + is_local, + normalize_path, + renames, + rmtree, +) +from pip._internal.utils.temp_dir import AdjacentTempDirectory, TempDirectory +from pip._internal.utils.typing import MYPY_CHECK_RUNNING + +if MYPY_CHECK_RUNNING: + from typing import ( + Any, Callable, Dict, Iterable, Iterator, List, Optional, Set, Tuple, + ) + from pip._vendor.pkg_resources import Distribution + +logger = logging.getLogger(__name__) + + +def _script_names(dist, script_name, is_gui): + # type: (Distribution, str, bool) -> List[str] + """Create the fully qualified name of the files created by + {console,gui}_scripts for the given ``dist``. + Returns the list of file names + """ + if dist_in_usersite(dist): + bin_dir = bin_user + else: + bin_dir = bin_py + exe_name = os.path.join(bin_dir, script_name) + paths_to_remove = [exe_name] + if WINDOWS: + paths_to_remove.append(exe_name + '.exe') + paths_to_remove.append(exe_name + '.exe.manifest') + if is_gui: + paths_to_remove.append(exe_name + '-script.pyw') + else: + paths_to_remove.append(exe_name + '-script.py') + return paths_to_remove + + +def _unique(fn): + # type: (Callable[..., Iterator[Any]]) -> Callable[..., Iterator[Any]] + @functools.wraps(fn) + def unique(*args, **kw): + # type: (Any, Any) -> Iterator[Any] + seen = set() # type: Set[Any] + for item in fn(*args, **kw): + if item not in seen: + seen.add(item) + yield item + return unique + + +@_unique +def uninstallation_paths(dist): + # type: (Distribution) -> Iterator[str] + """ + Yield all the uninstallation paths for dist based on RECORD-without-.py[co] + + Yield paths to all the files in RECORD. For each .py file in RECORD, add + the .pyc and .pyo in the same directory. + + UninstallPathSet.add() takes care of the __pycache__ .py[co]. + """ + r = csv.reader(FakeFile(dist.get_metadata_lines('RECORD'))) + for row in r: + path = os.path.join(dist.location, row[0]) + yield path + if path.endswith('.py'): + dn, fn = os.path.split(path) + base = fn[:-3] + path = os.path.join(dn, base + '.pyc') + yield path + path = os.path.join(dn, base + '.pyo') + yield path + + +def compact(paths): + # type: (Iterable[str]) -> Set[str] + """Compact a path set to contain the minimal number of paths + necessary to contain all paths in the set. If /a/path/ and + /a/path/to/a/file.txt are both in the set, leave only the + shorter path.""" + + sep = os.path.sep + short_paths = set() # type: Set[str] + for path in sorted(paths, key=len): + should_skip = any( + path.startswith(shortpath.rstrip("*")) and + path[len(shortpath.rstrip("*").rstrip(sep))] == sep + for shortpath in short_paths + ) + if not should_skip: + short_paths.add(path) + return short_paths + + +def compress_for_rename(paths): + # type: (Iterable[str]) -> Set[str] + """Returns a set containing the paths that need to be renamed. + + This set may include directories when the original sequence of paths + included every file on disk. + """ + case_map = dict((os.path.normcase(p), p) for p in paths) + remaining = set(case_map) + unchecked = sorted(set(os.path.split(p)[0] + for p in case_map.values()), key=len) + wildcards = set() # type: Set[str] + + def norm_join(*a): + # type: (str) -> str + return os.path.normcase(os.path.join(*a)) + + for root in unchecked: + if any(os.path.normcase(root).startswith(w) + for w in wildcards): + # This directory has already been handled. + continue + + all_files = set() # type: Set[str] + all_subdirs = set() # type: Set[str] + for dirname, subdirs, files in os.walk(root): + all_subdirs.update(norm_join(root, dirname, d) + for d in subdirs) + all_files.update(norm_join(root, dirname, f) + for f in files) + # If all the files we found are in our remaining set of files to + # remove, then remove them from the latter set and add a wildcard + # for the directory. + if not (all_files - remaining): + remaining.difference_update(all_files) + wildcards.add(root + os.sep) + + return set(map(case_map.__getitem__, remaining)) | wildcards + + +def compress_for_output_listing(paths): + # type: (Iterable[str]) -> Tuple[Set[str], Set[str]] + """Returns a tuple of 2 sets of which paths to display to user + + The first set contains paths that would be deleted. Files of a package + are not added and the top-level directory of the package has a '*' added + at the end - to signify that all it's contents are removed. + + The second set contains files that would have been skipped in the above + folders. + """ + + will_remove = set(paths) + will_skip = set() + + # Determine folders and files + folders = set() + files = set() + for path in will_remove: + if path.endswith(".pyc"): + continue + if path.endswith("__init__.py") or ".dist-info" in path: + folders.add(os.path.dirname(path)) + files.add(path) + + # probably this one https://github.com/python/mypy/issues/390 + _normcased_files = set(map(os.path.normcase, files)) # type: ignore + + folders = compact(folders) + + # This walks the tree using os.walk to not miss extra folders + # that might get added. + for folder in folders: + for dirpath, _, dirfiles in os.walk(folder): + for fname in dirfiles: + if fname.endswith(".pyc"): + continue + + file_ = os.path.join(dirpath, fname) + if (os.path.isfile(file_) and + os.path.normcase(file_) not in _normcased_files): + # We are skipping this file. Add it to the set. + will_skip.add(file_) + + will_remove = files | { + os.path.join(folder, "*") for folder in folders + } + + return will_remove, will_skip + + +class StashedUninstallPathSet(object): + """A set of file rename operations to stash files while + tentatively uninstalling them.""" + def __init__(self): + # type: () -> None + # Mapping from source file root to [Adjacent]TempDirectory + # for files under that directory. + self._save_dirs = {} # type: Dict[str, TempDirectory] + # (old path, new path) tuples for each move that may need + # to be undone. + self._moves = [] # type: List[Tuple[str, str]] + + def _get_directory_stash(self, path): + # type: (str) -> str + """Stashes a directory. + + Directories are stashed adjacent to their original location if + possible, or else moved/copied into the user's temp dir.""" + + try: + save_dir = AdjacentTempDirectory(path) # type: TempDirectory + except OSError: + save_dir = TempDirectory(kind="uninstall") + self._save_dirs[os.path.normcase(path)] = save_dir + + return save_dir.path + + def _get_file_stash(self, path): + # type: (str) -> str + """Stashes a file. + + If no root has been provided, one will be created for the directory + in the user's temp directory.""" + path = os.path.normcase(path) + head, old_head = os.path.dirname(path), None + save_dir = None + + while head != old_head: + try: + save_dir = self._save_dirs[head] + break + except KeyError: + pass + head, old_head = os.path.dirname(head), head + else: + # Did not find any suitable root + head = os.path.dirname(path) + save_dir = TempDirectory(kind='uninstall') + self._save_dirs[head] = save_dir + + relpath = os.path.relpath(path, head) + if relpath and relpath != os.path.curdir: + return os.path.join(save_dir.path, relpath) + return save_dir.path + + def stash(self, path): + # type: (str) -> str + """Stashes the directory or file and returns its new location. + Handle symlinks as files to avoid modifying the symlink targets. + """ + path_is_dir = os.path.isdir(path) and not os.path.islink(path) + if path_is_dir: + new_path = self._get_directory_stash(path) + else: + new_path = self._get_file_stash(path) + + self._moves.append((path, new_path)) + if (path_is_dir and os.path.isdir(new_path)): + # If we're moving a directory, we need to + # remove the destination first or else it will be + # moved to inside the existing directory. + # We just created new_path ourselves, so it will + # be removable. + os.rmdir(new_path) + renames(path, new_path) + return new_path + + def commit(self): + # type: () -> None + """Commits the uninstall by removing stashed files.""" + for _, save_dir in self._save_dirs.items(): + save_dir.cleanup() + self._moves = [] + self._save_dirs = {} + + def rollback(self): + # type: () -> None + """Undoes the uninstall by moving stashed files back.""" + for p in self._moves: + logger.info("Moving to %s\n from %s", *p) + + for new_path, path in self._moves: + try: + logger.debug('Replacing %s from %s', new_path, path) + if os.path.isfile(new_path) or os.path.islink(new_path): + os.unlink(new_path) + elif os.path.isdir(new_path): + rmtree(new_path) + renames(path, new_path) + except OSError as ex: + logger.error("Failed to restore %s", new_path) + logger.debug("Exception: %s", ex) + + self.commit() + + @property + def can_rollback(self): + # type: () -> bool + return bool(self._moves) + + +class UninstallPathSet(object): + """A set of file paths to be removed in the uninstallation of a + requirement.""" + def __init__(self, dist): + # type: (Distribution) -> None + self.paths = set() # type: Set[str] + self._refuse = set() # type: Set[str] + self.pth = {} # type: Dict[str, UninstallPthEntries] + self.dist = dist + self._moved_paths = StashedUninstallPathSet() + + def _permitted(self, path): + # type: (str) -> bool + """ + Return True if the given path is one we are permitted to + remove/modify, False otherwise. + + """ + return is_local(path) + + def add(self, path): + # type: (str) -> None + head, tail = os.path.split(path) + + # we normalize the head to resolve parent directory symlinks, but not + # the tail, since we only want to uninstall symlinks, not their targets + path = os.path.join(normalize_path(head), os.path.normcase(tail)) + + if not os.path.exists(path): + return + if self._permitted(path): + self.paths.add(path) + else: + self._refuse.add(path) + + # __pycache__ files can show up after 'installed-files.txt' is created, + # due to imports + if os.path.splitext(path)[1] == '.py' and uses_pycache: + self.add(cache_from_source(path)) + + def add_pth(self, pth_file, entry): + # type: (str, str) -> None + pth_file = normalize_path(pth_file) + if self._permitted(pth_file): + if pth_file not in self.pth: + self.pth[pth_file] = UninstallPthEntries(pth_file) + self.pth[pth_file].add(entry) + else: + self._refuse.add(pth_file) + + def remove(self, auto_confirm=False, verbose=False): + # type: (bool, bool) -> None + """Remove paths in ``self.paths`` with confirmation (unless + ``auto_confirm`` is True).""" + + if not self.paths: + logger.info( + "Can't uninstall '%s'. No files were found to uninstall.", + self.dist.project_name, + ) + return + + dist_name_version = ( + self.dist.project_name + "-" + self.dist.version + ) + logger.info('Uninstalling %s:', dist_name_version) + + with indent_log(): + if auto_confirm or self._allowed_to_proceed(verbose): + moved = self._moved_paths + + for_rename = compress_for_rename(self.paths) + + for path in sorted(compact(for_rename)): + moved.stash(path) + logger.debug('Removing file or directory %s', path) + + for pth in self.pth.values(): + pth.remove() + + logger.info('Successfully uninstalled %s', dist_name_version) + + def _allowed_to_proceed(self, verbose): + # type: (bool) -> bool + """Display which files would be deleted and prompt for confirmation + """ + + def _display(msg, paths): + # type: (str, Iterable[str]) -> None + if not paths: + return + + logger.info(msg) + with indent_log(): + for path in sorted(compact(paths)): + logger.info(path) + + if not verbose: + will_remove, will_skip = compress_for_output_listing(self.paths) + else: + # In verbose mode, display all the files that are going to be + # deleted. + will_remove = set(self.paths) + will_skip = set() + + _display('Would remove:', will_remove) + _display('Would not remove (might be manually added):', will_skip) + _display('Would not remove (outside of prefix):', self._refuse) + if verbose: + _display('Will actually move:', compress_for_rename(self.paths)) + + return ask('Proceed (y/n)? ', ('y', 'n')) == 'y' + + def rollback(self): + # type: () -> None + """Rollback the changes previously made by remove().""" + if not self._moved_paths.can_rollback: + logger.error( + "Can't roll back %s; was not uninstalled", + self.dist.project_name, + ) + return + logger.info('Rolling back uninstall of %s', self.dist.project_name) + self._moved_paths.rollback() + for pth in self.pth.values(): + pth.rollback() + + def commit(self): + # type: () -> None + """Remove temporary save dir: rollback will no longer be possible.""" + self._moved_paths.commit() + + @classmethod + def from_dist(cls, dist): + # type: (Distribution) -> UninstallPathSet + dist_path = normalize_path(dist.location) + if not dist_is_local(dist): + logger.info( + "Not uninstalling %s at %s, outside environment %s", + dist.key, + dist_path, + sys.prefix, + ) + return cls(dist) + + if dist_path in {p for p in {sysconfig.get_path("stdlib"), + sysconfig.get_path("platstdlib")} + if p}: + logger.info( + "Not uninstalling %s at %s, as it is in the standard library.", + dist.key, + dist_path, + ) + return cls(dist) + + paths_to_remove = cls(dist) + develop_egg_link = egg_link_path(dist) + develop_egg_link_egg_info = '{}.egg-info'.format( + pkg_resources.to_filename(dist.project_name)) + egg_info_exists = dist.egg_info and os.path.exists(dist.egg_info) + # Special case for distutils installed package + distutils_egg_info = getattr(dist._provider, 'path', None) + + # Uninstall cases order do matter as in the case of 2 installs of the + # same package, pip needs to uninstall the currently detected version + if (egg_info_exists and dist.egg_info.endswith('.egg-info') and + not dist.egg_info.endswith(develop_egg_link_egg_info)): + # if dist.egg_info.endswith(develop_egg_link_egg_info), we + # are in fact in the develop_egg_link case + paths_to_remove.add(dist.egg_info) + if dist.has_metadata('installed-files.txt'): + for installed_file in dist.get_metadata( + 'installed-files.txt').splitlines(): + path = os.path.normpath( + os.path.join(dist.egg_info, installed_file) + ) + paths_to_remove.add(path) + # FIXME: need a test for this elif block + # occurs with --single-version-externally-managed/--record outside + # of pip + elif dist.has_metadata('top_level.txt'): + if dist.has_metadata('namespace_packages.txt'): + namespaces = dist.get_metadata('namespace_packages.txt') + else: + namespaces = [] + for top_level_pkg in [ + p for p + in dist.get_metadata('top_level.txt').splitlines() + if p and p not in namespaces]: + path = os.path.join(dist.location, top_level_pkg) + paths_to_remove.add(path) + paths_to_remove.add(path + '.py') + paths_to_remove.add(path + '.pyc') + paths_to_remove.add(path + '.pyo') + + elif distutils_egg_info: + raise UninstallationError( + "Cannot uninstall {!r}. It is a distutils installed project " + "and thus we cannot accurately determine which files belong " + "to it which would lead to only a partial uninstall.".format( + dist.project_name, + ) + ) + + elif dist.location.endswith('.egg'): + # package installed by easy_install + # We cannot match on dist.egg_name because it can slightly vary + # i.e. setuptools-0.6c11-py2.6.egg vs setuptools-0.6rc11-py2.6.egg + paths_to_remove.add(dist.location) + easy_install_egg = os.path.split(dist.location)[1] + easy_install_pth = os.path.join(os.path.dirname(dist.location), + 'easy-install.pth') + paths_to_remove.add_pth(easy_install_pth, './' + easy_install_egg) + + elif egg_info_exists and dist.egg_info.endswith('.dist-info'): + for path in uninstallation_paths(dist): + paths_to_remove.add(path) + + elif develop_egg_link: + # develop egg + with open(develop_egg_link, 'r') as fh: + link_pointer = os.path.normcase(fh.readline().strip()) + assert (link_pointer == dist.location), ( + 'Egg-link %s does not match installed location of %s ' + '(at %s)' % (link_pointer, dist.project_name, dist.location) + ) + paths_to_remove.add(develop_egg_link) + easy_install_pth = os.path.join(os.path.dirname(develop_egg_link), + 'easy-install.pth') + paths_to_remove.add_pth(easy_install_pth, dist.location) + + else: + logger.debug( + 'Not sure how to uninstall: %s - Check: %s', + dist, dist.location, + ) + + # find distutils scripts= scripts + if dist.has_metadata('scripts') and dist.metadata_isdir('scripts'): + for script in dist.metadata_listdir('scripts'): + if dist_in_usersite(dist): + bin_dir = bin_user + else: + bin_dir = bin_py + paths_to_remove.add(os.path.join(bin_dir, script)) + if WINDOWS: + paths_to_remove.add(os.path.join(bin_dir, script) + '.bat') + + # find console_scripts + _scripts_to_remove = [] + console_scripts = dist.get_entry_map(group='console_scripts') + for name in console_scripts.keys(): + _scripts_to_remove.extend(_script_names(dist, name, False)) + # find gui_scripts + gui_scripts = dist.get_entry_map(group='gui_scripts') + for name in gui_scripts.keys(): + _scripts_to_remove.extend(_script_names(dist, name, True)) + + for s in _scripts_to_remove: + paths_to_remove.add(s) + + return paths_to_remove + + +class UninstallPthEntries(object): + def __init__(self, pth_file): + # type: (str) -> None + if not os.path.isfile(pth_file): + raise UninstallationError( + "Cannot remove entries from nonexistent file %s" % pth_file + ) + self.file = pth_file + self.entries = set() # type: Set[str] + self._saved_lines = None # type: Optional[List[bytes]] + + def add(self, entry): + # type: (str) -> None + entry = os.path.normcase(entry) + # On Windows, os.path.normcase converts the entry to use + # backslashes. This is correct for entries that describe absolute + # paths outside of site-packages, but all the others use forward + # slashes. + # os.path.splitdrive is used instead of os.path.isabs because isabs + # treats non-absolute paths with drive letter markings like c:foo\bar + # as absolute paths. It also does not recognize UNC paths if they don't + # have more than "\\sever\share". Valid examples: "\\server\share\" or + # "\\server\share\folder". Python 2.7.8+ support UNC in splitdrive. + if WINDOWS and not os.path.splitdrive(entry)[0]: + entry = entry.replace('\\', '/') + self.entries.add(entry) + + def remove(self): + # type: () -> None + logger.debug('Removing pth entries from %s:', self.file) + with open(self.file, 'rb') as fh: + # windows uses '\r\n' with py3k, but uses '\n' with py2.x + lines = fh.readlines() + self._saved_lines = lines + if any(b'\r\n' in line for line in lines): + endline = '\r\n' + else: + endline = '\n' + # handle missing trailing newline + if lines and not lines[-1].endswith(endline.encode("utf-8")): + lines[-1] = lines[-1] + endline.encode("utf-8") + for entry in self.entries: + try: + logger.debug('Removing entry: %s', entry) + lines.remove((entry + endline).encode("utf-8")) + except ValueError: + pass + with open(self.file, 'wb') as fh: + fh.writelines(lines) + + def rollback(self): + # type: () -> bool + if self._saved_lines is None: + logger.error( + 'Cannot roll back changes to %s, none were made', self.file + ) + return False + logger.debug('Rolling %s back to previous state', self.file) + with open(self.file, 'wb') as fh: + fh.writelines(self._saved_lines) + return True diff --git a/env/lib/python3.8/site-packages/pip/_internal/self_outdated_check.py b/env/lib/python3.8/site-packages/pip/_internal/self_outdated_check.py new file mode 100644 index 000000000..8fc3c594a --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/self_outdated_check.py @@ -0,0 +1,242 @@ +# The following comment should be removed at some point in the future. +# mypy: disallow-untyped-defs=False + +from __future__ import absolute_import + +import datetime +import hashlib +import json +import logging +import os.path +import sys + +from pip._vendor import pkg_resources +from pip._vendor.packaging import version as packaging_version +from pip._vendor.six import ensure_binary + +from pip._internal.index.collector import LinkCollector +from pip._internal.index.package_finder import PackageFinder +from pip._internal.models.search_scope import SearchScope +from pip._internal.models.selection_prefs import SelectionPreferences +from pip._internal.utils.filesystem import ( + adjacent_tmp_file, + check_path_owner, + replace, +) +from pip._internal.utils.misc import ( + ensure_dir, + get_installed_version, + redact_auth_from_url, +) +from pip._internal.utils.packaging import get_installer +from pip._internal.utils.typing import MYPY_CHECK_RUNNING + +if MYPY_CHECK_RUNNING: + import optparse + from optparse import Values + from typing import Any, Dict, Text, Union + + from pip._internal.network.session import PipSession + + +SELFCHECK_DATE_FMT = "%Y-%m-%dT%H:%M:%SZ" + + +logger = logging.getLogger(__name__) + + +def make_link_collector( + session, # type: PipSession + options, # type: Values + suppress_no_index=False, # type: bool +): + # type: (...) -> LinkCollector + """ + :param session: The Session to use to make requests. + :param suppress_no_index: Whether to ignore the --no-index option + when constructing the SearchScope object. + """ + index_urls = [options.index_url] + options.extra_index_urls + if options.no_index and not suppress_no_index: + logger.debug( + 'Ignoring indexes: %s', + ','.join(redact_auth_from_url(url) for url in index_urls), + ) + index_urls = [] + + # Make sure find_links is a list before passing to create(). + find_links = options.find_links or [] + + search_scope = SearchScope.create( + find_links=find_links, index_urls=index_urls, + ) + + link_collector = LinkCollector(session=session, search_scope=search_scope) + + return link_collector + + +def _get_statefile_name(key): + # type: (Union[str, Text]) -> str + key_bytes = ensure_binary(key) + name = hashlib.sha224(key_bytes).hexdigest() + return name + + +class SelfCheckState(object): + def __init__(self, cache_dir): + # type: (str) -> None + self.state = {} # type: Dict[str, Any] + self.statefile_path = None + + # Try to load the existing state + if cache_dir: + self.statefile_path = os.path.join( + cache_dir, "selfcheck", _get_statefile_name(self.key) + ) + try: + with open(self.statefile_path) as statefile: + self.state = json.load(statefile) + except (IOError, ValueError, KeyError): + # Explicitly suppressing exceptions, since we don't want to + # error out if the cache file is invalid. + pass + + @property + def key(self): + return sys.prefix + + def save(self, pypi_version, current_time): + # type: (str, datetime.datetime) -> None + # If we do not have a path to cache in, don't bother saving. + if not self.statefile_path: + return + + # Check to make sure that we own the directory + if not check_path_owner(os.path.dirname(self.statefile_path)): + return + + # Now that we've ensured the directory is owned by this user, we'll go + # ahead and make sure that all our directories are created. + ensure_dir(os.path.dirname(self.statefile_path)) + + state = { + # Include the key so it's easy to tell which pip wrote the + # file. + "key": self.key, + "last_check": current_time.strftime(SELFCHECK_DATE_FMT), + "pypi_version": pypi_version, + } + + text = json.dumps(state, sort_keys=True, separators=(",", ":")) + + with adjacent_tmp_file(self.statefile_path) as f: + f.write(ensure_binary(text)) + + try: + # Since we have a prefix-specific state file, we can just + # overwrite whatever is there, no need to check. + replace(f.name, self.statefile_path) + except OSError: + # Best effort. + pass + + +def was_installed_by_pip(pkg): + # type: (str) -> bool + """Checks whether pkg was installed by pip + + This is used not to display the upgrade message when pip is in fact + installed by system package manager, such as dnf on Fedora. + """ + try: + dist = pkg_resources.get_distribution(pkg) + return "pip" == get_installer(dist) + except pkg_resources.DistributionNotFound: + return False + + +def pip_self_version_check(session, options): + # type: (PipSession, optparse.Values) -> None + """Check for an update for pip. + + Limit the frequency of checks to once per week. State is stored either in + the active virtualenv or in the user's USER_CACHE_DIR keyed off the prefix + of the pip script path. + """ + installed_version = get_installed_version("pip") + if not installed_version: + return + + pip_version = packaging_version.parse(installed_version) + pypi_version = None + + try: + state = SelfCheckState(cache_dir=options.cache_dir) + + current_time = datetime.datetime.utcnow() + # Determine if we need to refresh the state + if "last_check" in state.state and "pypi_version" in state.state: + last_check = datetime.datetime.strptime( + state.state["last_check"], + SELFCHECK_DATE_FMT + ) + if (current_time - last_check).total_seconds() < 7 * 24 * 60 * 60: + pypi_version = state.state["pypi_version"] + + # Refresh the version if we need to or just see if we need to warn + if pypi_version is None: + # Lets use PackageFinder to see what the latest pip version is + link_collector = make_link_collector( + session, + options=options, + suppress_no_index=True, + ) + + # Pass allow_yanked=False so we don't suggest upgrading to a + # yanked version. + selection_prefs = SelectionPreferences( + allow_yanked=False, + allow_all_prereleases=False, # Explicitly set to False + ) + + finder = PackageFinder.create( + link_collector=link_collector, + selection_prefs=selection_prefs, + ) + best_candidate = finder.find_best_candidate("pip").best_candidate + if best_candidate is None: + return + pypi_version = str(best_candidate.version) + + # save that we've performed a check + state.save(pypi_version, current_time) + + remote_version = packaging_version.parse(pypi_version) + + local_version_is_older = ( + pip_version < remote_version and + pip_version.base_version != remote_version.base_version and + was_installed_by_pip('pip') + ) + + # Determine if our pypi_version is older + if not local_version_is_older: + return + + # We cannot tell how the current pip is available in the current + # command context, so be pragmatic here and suggest the command + # that's always available. This does not accommodate spaces in + # `sys.executable`. + pip_cmd = "{} -m pip".format(sys.executable) + logger.warning( + "You are using pip version %s; however, version %s is " + "available.\nYou should consider upgrading via the " + "'%s install --upgrade pip' command.", + pip_version, pypi_version, pip_cmd + ) + except Exception: + logger.debug( + "There was an error checking the latest version of pip", + exc_info=True, + ) diff --git a/env/lib/python3.8/site-packages/pip/_internal/utils/__init__.py b/env/lib/python3.8/site-packages/pip/_internal/utils/__init__.py new file mode 100644 index 000000000..e69de29bb diff --git a/env/lib/python3.8/site-packages/pip/_internal/utils/appdirs.py b/env/lib/python3.8/site-packages/pip/_internal/utils/appdirs.py new file mode 100644 index 000000000..251c5fd59 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/utils/appdirs.py @@ -0,0 +1,41 @@ +""" +This code wraps the vendored appdirs module to so the return values are +compatible for the current pip code base. + +The intention is to rewrite current usages gradually, keeping the tests pass, +and eventually drop this after all usages are changed. +""" + +from __future__ import absolute_import + +import os + +from pip._vendor import appdirs as _appdirs + +from pip._internal.utils.typing import MYPY_CHECK_RUNNING + +if MYPY_CHECK_RUNNING: + from typing import List + + +def user_cache_dir(appname): + # type: (str) -> str + return _appdirs.user_cache_dir(appname, appauthor=False) + + +def user_config_dir(appname, roaming=True): + # type: (str, bool) -> str + return _appdirs.user_config_dir(appname, appauthor=False, roaming=roaming) + + +def user_data_dir(appname, roaming=False): + # type: (str, bool) -> str + return _appdirs.user_data_dir(appname, appauthor=False, roaming=roaming) + + +def site_config_dirs(appname): + # type: (str) -> List[str] + dirval = _appdirs.site_config_dir(appname, appauthor=False, multipath=True) + if _appdirs.system not in ["win32", "darwin"]: + return dirval.split(os.pathsep) + return [dirval] diff --git a/env/lib/python3.8/site-packages/pip/_internal/utils/compat.py b/env/lib/python3.8/site-packages/pip/_internal/utils/compat.py new file mode 100644 index 000000000..6efa52ad2 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/utils/compat.py @@ -0,0 +1,269 @@ +"""Stuff that differs in different Python versions and platform +distributions.""" + +# The following comment should be removed at some point in the future. +# mypy: disallow-untyped-defs=False + +from __future__ import absolute_import, division + +import codecs +import locale +import logging +import os +import shutil +import sys + +from pip._vendor.six import PY2, text_type + +from pip._internal.utils.typing import MYPY_CHECK_RUNNING + +if MYPY_CHECK_RUNNING: + from typing import Optional, Text, Tuple, Union + +try: + import ipaddress +except ImportError: + try: + from pip._vendor import ipaddress # type: ignore + except ImportError: + import ipaddr as ipaddress # type: ignore + ipaddress.ip_address = ipaddress.IPAddress # type: ignore + ipaddress.ip_network = ipaddress.IPNetwork # type: ignore + + +__all__ = [ + "ipaddress", "uses_pycache", "console_to_str", + "get_path_uid", "stdlib_pkgs", "WINDOWS", "samefile", "get_terminal_size", +] + + +logger = logging.getLogger(__name__) + +if PY2: + import imp + + try: + cache_from_source = imp.cache_from_source # type: ignore + except AttributeError: + # does not use __pycache__ + cache_from_source = None + + uses_pycache = cache_from_source is not None +else: + uses_pycache = True + from importlib.util import cache_from_source + + +if PY2: + # In Python 2.7, backslashreplace exists + # but does not support use for decoding. + # We implement our own replace handler for this + # situation, so that we can consistently use + # backslash replacement for all versions. + def backslashreplace_decode_fn(err): + raw_bytes = (err.object[i] for i in range(err.start, err.end)) + # Python 2 gave us characters - convert to numeric bytes + raw_bytes = (ord(b) for b in raw_bytes) + return u"".join(u"\\x%x" % c for c in raw_bytes), err.end + codecs.register_error( + "backslashreplace_decode", + backslashreplace_decode_fn, + ) + backslashreplace_decode = "backslashreplace_decode" +else: + backslashreplace_decode = "backslashreplace" + + +def has_tls(): + # type: () -> bool + try: + import _ssl # noqa: F401 # ignore unused + return True + except ImportError: + pass + + from pip._vendor.urllib3.util import IS_PYOPENSSL + return IS_PYOPENSSL + + +def str_to_display(data, desc=None): + # type: (Union[bytes, Text], Optional[str]) -> Text + """ + For display or logging purposes, convert a bytes object (or text) to + text (e.g. unicode in Python 2) safe for output. + + :param desc: An optional phrase describing the input data, for use in + the log message if a warning is logged. Defaults to "Bytes object". + + This function should never error out and so can take a best effort + approach. It is okay to be lossy if needed since the return value is + just for display. + + We assume the data is in the locale preferred encoding. If it won't + decode properly, we warn the user but decode as best we can. + + We also ensure that the output can be safely written to standard output + without encoding errors. + """ + if isinstance(data, text_type): + return data + + # Otherwise, data is a bytes object (str in Python 2). + # First, get the encoding we assume. This is the preferred + # encoding for the locale, unless that is not found, or + # it is ASCII, in which case assume UTF-8 + encoding = locale.getpreferredencoding() + if (not encoding) or codecs.lookup(encoding).name == "ascii": + encoding = "utf-8" + + # Now try to decode the data - if we fail, warn the user and + # decode with replacement. + try: + decoded_data = data.decode(encoding) + except UnicodeDecodeError: + if desc is None: + desc = 'Bytes object' + msg_format = '{} does not appear to be encoded as %s'.format(desc) + logger.warning(msg_format, encoding) + decoded_data = data.decode(encoding, errors=backslashreplace_decode) + + # Make sure we can print the output, by encoding it to the output + # encoding with replacement of unencodable characters, and then + # decoding again. + # We use stderr's encoding because it's less likely to be + # redirected and if we don't find an encoding we skip this + # step (on the assumption that output is wrapped by something + # that won't fail). + # The double getattr is to deal with the possibility that we're + # being called in a situation where sys.__stderr__ doesn't exist, + # or doesn't have an encoding attribute. Neither of these cases + # should occur in normal pip use, but there's no harm in checking + # in case people use pip in (unsupported) unusual situations. + output_encoding = getattr(getattr(sys, "__stderr__", None), + "encoding", None) + + if output_encoding: + output_encoded = decoded_data.encode( + output_encoding, + errors="backslashreplace" + ) + decoded_data = output_encoded.decode(output_encoding) + + return decoded_data + + +def console_to_str(data): + # type: (bytes) -> Text + """Return a string, safe for output, of subprocess output. + """ + return str_to_display(data, desc='Subprocess output') + + +def get_path_uid(path): + # type: (str) -> int + """ + Return path's uid. + + Does not follow symlinks: + https://github.com/pypa/pip/pull/935#discussion_r5307003 + + Placed this function in compat due to differences on AIX and + Jython, that should eventually go away. + + :raises OSError: When path is a symlink or can't be read. + """ + if hasattr(os, 'O_NOFOLLOW'): + fd = os.open(path, os.O_RDONLY | os.O_NOFOLLOW) + file_uid = os.fstat(fd).st_uid + os.close(fd) + else: # AIX and Jython + # WARNING: time of check vulnerability, but best we can do w/o NOFOLLOW + if not os.path.islink(path): + # older versions of Jython don't have `os.fstat` + file_uid = os.stat(path).st_uid + else: + # raise OSError for parity with os.O_NOFOLLOW above + raise OSError( + "%s is a symlink; Will not return uid for symlinks" % path + ) + return file_uid + + +def expanduser(path): + # type: (str) -> str + """ + Expand ~ and ~user constructions. + + Includes a workaround for https://bugs.python.org/issue14768 + """ + expanded = os.path.expanduser(path) + if path.startswith('~/') and expanded.startswith('//'): + expanded = expanded[1:] + return expanded + + +# packages in the stdlib that may have installation metadata, but should not be +# considered 'installed'. this theoretically could be determined based on +# dist.location (py27:`sysconfig.get_paths()['stdlib']`, +# py26:sysconfig.get_config_vars('LIBDEST')), but fear platform variation may +# make this ineffective, so hard-coding +stdlib_pkgs = {"python", "wsgiref", "argparse"} + + +# windows detection, covers cpython and ironpython +WINDOWS = (sys.platform.startswith("win") or + (sys.platform == 'cli' and os.name == 'nt')) + + +def samefile(file1, file2): + # type: (str, str) -> bool + """Provide an alternative for os.path.samefile on Windows/Python2""" + if hasattr(os.path, 'samefile'): + return os.path.samefile(file1, file2) + else: + path1 = os.path.normcase(os.path.abspath(file1)) + path2 = os.path.normcase(os.path.abspath(file2)) + return path1 == path2 + + +if hasattr(shutil, 'get_terminal_size'): + def get_terminal_size(): + # type: () -> Tuple[int, int] + """ + Returns a tuple (x, y) representing the width(x) and the height(y) + in characters of the terminal window. + """ + return tuple(shutil.get_terminal_size()) # type: ignore +else: + def get_terminal_size(): + # type: () -> Tuple[int, int] + """ + Returns a tuple (x, y) representing the width(x) and the height(y) + in characters of the terminal window. + """ + def ioctl_GWINSZ(fd): + try: + import fcntl + import termios + import struct + cr = struct.unpack_from( + 'hh', + fcntl.ioctl(fd, termios.TIOCGWINSZ, '12345678') + ) + except Exception: + return None + if cr == (0, 0): + return None + return cr + cr = ioctl_GWINSZ(0) or ioctl_GWINSZ(1) or ioctl_GWINSZ(2) + if not cr: + if sys.platform != "win32": + try: + fd = os.open(os.ctermid(), os.O_RDONLY) + cr = ioctl_GWINSZ(fd) + os.close(fd) + except Exception: + pass + if not cr: + cr = (os.environ.get('LINES', 25), os.environ.get('COLUMNS', 80)) + return int(cr[1]), int(cr[0]) diff --git a/env/lib/python3.8/site-packages/pip/_internal/utils/deprecation.py b/env/lib/python3.8/site-packages/pip/_internal/utils/deprecation.py new file mode 100644 index 000000000..2f20cfd49 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/utils/deprecation.py @@ -0,0 +1,104 @@ +""" +A module that implements tooling to enable easy warnings about deprecations. +""" + +# The following comment should be removed at some point in the future. +# mypy: disallow-untyped-defs=False + +from __future__ import absolute_import + +import logging +import warnings + +from pip._vendor.packaging.version import parse + +from pip import __version__ as current_version +from pip._internal.utils.typing import MYPY_CHECK_RUNNING + +if MYPY_CHECK_RUNNING: + from typing import Any, Optional + + +DEPRECATION_MSG_PREFIX = "DEPRECATION: " + + +class PipDeprecationWarning(Warning): + pass + + +_original_showwarning = None # type: Any + + +# Warnings <-> Logging Integration +def _showwarning(message, category, filename, lineno, file=None, line=None): + if file is not None: + if _original_showwarning is not None: + _original_showwarning( + message, category, filename, lineno, file, line, + ) + elif issubclass(category, PipDeprecationWarning): + # We use a specially named logger which will handle all of the + # deprecation messages for pip. + logger = logging.getLogger("pip._internal.deprecations") + logger.warning(message) + else: + _original_showwarning( + message, category, filename, lineno, file, line, + ) + + +def install_warning_logger(): + # type: () -> None + # Enable our Deprecation Warnings + warnings.simplefilter("default", PipDeprecationWarning, append=True) + + global _original_showwarning + + if _original_showwarning is None: + _original_showwarning = warnings.showwarning + warnings.showwarning = _showwarning + + +def deprecated(reason, replacement, gone_in, issue=None): + # type: (str, Optional[str], Optional[str], Optional[int]) -> None + """Helper to deprecate existing functionality. + + reason: + Textual reason shown to the user about why this functionality has + been deprecated. + replacement: + Textual suggestion shown to the user about what alternative + functionality they can use. + gone_in: + The version of pip does this functionality should get removed in. + Raises errors if pip's current version is greater than or equal to + this. + issue: + Issue number on the tracker that would serve as a useful place for + users to find related discussion and provide feedback. + + Always pass replacement, gone_in and issue as keyword arguments for clarity + at the call site. + """ + + # Construct a nice message. + # This is eagerly formatted as we want it to get logged as if someone + # typed this entire message out. + sentences = [ + (reason, DEPRECATION_MSG_PREFIX + "{}"), + (gone_in, "pip {} will remove support for this functionality."), + (replacement, "A possible replacement is {}."), + (issue, ( + "You can find discussion regarding this at " + "https://github.com/pypa/pip/issues/{}." + )), + ] + message = " ".join( + template.format(val) for val, template in sentences if val is not None + ) + + # Raise as an error if it has to be removed. + if gone_in is not None and parse(current_version) >= parse(gone_in): + raise PipDeprecationWarning(message) + + warnings.warn(message, category=PipDeprecationWarning, stacklevel=2) diff --git a/env/lib/python3.8/site-packages/pip/_internal/utils/distutils_args.py b/env/lib/python3.8/site-packages/pip/_internal/utils/distutils_args.py new file mode 100644 index 000000000..e38e402d7 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/utils/distutils_args.py @@ -0,0 +1,48 @@ +from distutils.errors import DistutilsArgError +from distutils.fancy_getopt import FancyGetopt + +from pip._internal.utils.typing import MYPY_CHECK_RUNNING + +if MYPY_CHECK_RUNNING: + from typing import Dict, List + + +_options = [ + ("exec-prefix=", None, ""), + ("home=", None, ""), + ("install-base=", None, ""), + ("install-data=", None, ""), + ("install-headers=", None, ""), + ("install-lib=", None, ""), + ("install-platlib=", None, ""), + ("install-purelib=", None, ""), + ("install-scripts=", None, ""), + ("prefix=", None, ""), + ("root=", None, ""), + ("user", None, ""), +] + + +# typeshed doesn't permit Tuple[str, None, str], see python/typeshed#3469. +_distutils_getopt = FancyGetopt(_options) # type: ignore + + +def parse_distutils_args(args): + # type: (List[str]) -> Dict[str, str] + """Parse provided arguments, returning an object that has the + matched arguments. + + Any unknown arguments are ignored. + """ + result = {} + for arg in args: + try: + _, match = _distutils_getopt.getopt(args=[arg]) + except DistutilsArgError: + # We don't care about any other options, which here may be + # considered unrecognized since our option list is not + # exhaustive. + pass + else: + result.update(match.__dict__) + return result diff --git a/env/lib/python3.8/site-packages/pip/_internal/utils/encoding.py b/env/lib/python3.8/site-packages/pip/_internal/utils/encoding.py new file mode 100644 index 000000000..ab4d4b98e --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/utils/encoding.py @@ -0,0 +1,42 @@ +# The following comment should be removed at some point in the future. +# mypy: strict-optional=False + +import codecs +import locale +import re +import sys + +from pip._internal.utils.typing import MYPY_CHECK_RUNNING + +if MYPY_CHECK_RUNNING: + from typing import List, Tuple, Text + +BOMS = [ + (codecs.BOM_UTF8, 'utf-8'), + (codecs.BOM_UTF16, 'utf-16'), + (codecs.BOM_UTF16_BE, 'utf-16-be'), + (codecs.BOM_UTF16_LE, 'utf-16-le'), + (codecs.BOM_UTF32, 'utf-32'), + (codecs.BOM_UTF32_BE, 'utf-32-be'), + (codecs.BOM_UTF32_LE, 'utf-32-le'), +] # type: List[Tuple[bytes, Text]] + +ENCODING_RE = re.compile(br'coding[:=]\s*([-\w.]+)') + + +def auto_decode(data): + # type: (bytes) -> Text + """Check a bytes string for a BOM to correctly detect the encoding + + Fallback to locale.getpreferredencoding(False) like open() on Python3""" + for bom, encoding in BOMS: + if data.startswith(bom): + return data[len(bom):].decode(encoding) + # Lets check the first two lines as in PEP263 + for line in data.split(b'\n')[:2]: + if line[0:1] == b'#' and ENCODING_RE.search(line): + encoding = ENCODING_RE.search(line).groups()[0].decode('ascii') + return data.decode(encoding) + return data.decode( + locale.getpreferredencoding(False) or sys.getdefaultencoding(), + ) diff --git a/env/lib/python3.8/site-packages/pip/_internal/utils/entrypoints.py b/env/lib/python3.8/site-packages/pip/_internal/utils/entrypoints.py new file mode 100644 index 000000000..befd01c89 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/utils/entrypoints.py @@ -0,0 +1,31 @@ +import sys + +from pip._internal.cli.main import main +from pip._internal.utils.typing import MYPY_CHECK_RUNNING + +if MYPY_CHECK_RUNNING: + from typing import Optional, List + + +def _wrapper(args=None): + # type: (Optional[List[str]]) -> int + """Central wrapper for all old entrypoints. + + Historically pip has had several entrypoints defined. Because of issues + arising from PATH, sys.path, multiple Pythons, their interactions, and most + of them having a pip installed, users suffer every time an entrypoint gets + moved. + + To alleviate this pain, and provide a mechanism for warning users and + directing them to an appropriate place for help, we now define all of + our old entrypoints as wrappers for the current one. + """ + sys.stderr.write( + "WARNING: pip is being invoked by an old script wrapper. This will " + "fail in a future version of pip.\n" + "Please see https://github.com/pypa/pip/issues/5599 for advice on " + "fixing the underlying issue.\n" + "To avoid this problem you can invoke Python with '-m pip' instead of " + "running pip directly.\n" + ) + return main(args) diff --git a/env/lib/python3.8/site-packages/pip/_internal/utils/filesystem.py b/env/lib/python3.8/site-packages/pip/_internal/utils/filesystem.py new file mode 100644 index 000000000..6f1537e40 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/utils/filesystem.py @@ -0,0 +1,171 @@ +import errno +import os +import os.path +import random +import shutil +import stat +import sys +from contextlib import contextmanager +from tempfile import NamedTemporaryFile + +# NOTE: retrying is not annotated in typeshed as on 2017-07-17, which is +# why we ignore the type on this import. +from pip._vendor.retrying import retry # type: ignore +from pip._vendor.six import PY2 + +from pip._internal.utils.compat import get_path_uid +from pip._internal.utils.typing import MYPY_CHECK_RUNNING, cast + +if MYPY_CHECK_RUNNING: + from typing import BinaryIO, Iterator + + class NamedTemporaryFileResult(BinaryIO): + @property + def file(self): + # type: () -> BinaryIO + pass + + +def check_path_owner(path): + # type: (str) -> bool + # If we don't have a way to check the effective uid of this process, then + # we'll just assume that we own the directory. + if sys.platform == "win32" or not hasattr(os, "geteuid"): + return True + + assert os.path.isabs(path) + + previous = None + while path != previous: + if os.path.lexists(path): + # Check if path is writable by current user. + if os.geteuid() == 0: + # Special handling for root user in order to handle properly + # cases where users use sudo without -H flag. + try: + path_uid = get_path_uid(path) + except OSError: + return False + return path_uid == 0 + else: + return os.access(path, os.W_OK) + else: + previous, path = path, os.path.dirname(path) + return False # assume we don't own the path + + +def copy2_fixed(src, dest): + # type: (str, str) -> None + """Wrap shutil.copy2() but map errors copying socket files to + SpecialFileError as expected. + + See also https://bugs.python.org/issue37700. + """ + try: + shutil.copy2(src, dest) + except (OSError, IOError): + for f in [src, dest]: + try: + is_socket_file = is_socket(f) + except OSError: + # An error has already occurred. Another error here is not + # a problem and we can ignore it. + pass + else: + if is_socket_file: + raise shutil.SpecialFileError("`%s` is a socket" % f) + + raise + + +def is_socket(path): + # type: (str) -> bool + return stat.S_ISSOCK(os.lstat(path).st_mode) + + +@contextmanager +def adjacent_tmp_file(path): + # type: (str) -> Iterator[NamedTemporaryFileResult] + """Given a path to a file, open a temp file next to it securely and ensure + it is written to disk after the context reaches its end. + """ + with NamedTemporaryFile( + delete=False, + dir=os.path.dirname(path), + prefix=os.path.basename(path), + suffix='.tmp', + ) as f: + result = cast('NamedTemporaryFileResult', f) + try: + yield result + finally: + result.file.flush() + os.fsync(result.file.fileno()) + + +_replace_retry = retry(stop_max_delay=1000, wait_fixed=250) + +if PY2: + @_replace_retry + def replace(src, dest): + # type: (str, str) -> None + try: + os.rename(src, dest) + except OSError: + os.remove(dest) + os.rename(src, dest) + +else: + replace = _replace_retry(os.replace) + + +# test_writable_dir and _test_writable_dir_win are copied from Flit, +# with the author's agreement to also place them under pip's license. +def test_writable_dir(path): + # type: (str) -> bool + """Check if a directory is writable. + + Uses os.access() on POSIX, tries creating files on Windows. + """ + # If the directory doesn't exist, find the closest parent that does. + while not os.path.isdir(path): + parent = os.path.dirname(path) + if parent == path: + break # Should never get here, but infinite loops are bad + path = parent + + if os.name == 'posix': + return os.access(path, os.W_OK) + + return _test_writable_dir_win(path) + + +def _test_writable_dir_win(path): + # type: (str) -> bool + # os.access doesn't work on Windows: http://bugs.python.org/issue2528 + # and we can't use tempfile: http://bugs.python.org/issue22107 + basename = 'accesstest_deleteme_fishfingers_custard_' + alphabet = 'abcdefghijklmnopqrstuvwxyz0123456789' + for i in range(10): + name = basename + ''.join(random.choice(alphabet) for _ in range(6)) + file = os.path.join(path, name) + try: + fd = os.open(file, os.O_RDWR | os.O_CREAT | os.O_EXCL) + except OSError as e: + if e.errno == errno.EEXIST: + continue + if e.errno == errno.EPERM: + # This could be because there's a directory with the same name. + # But it's highly unlikely there's a directory called that, + # so we'll assume it's because the parent dir is not writable. + return False + raise + else: + os.close(fd) + os.unlink(file) + return True + + # This should never be reached + raise EnvironmentError( + 'Unexpected condition testing for writable directory' + ) diff --git a/env/lib/python3.8/site-packages/pip/_internal/utils/filetypes.py b/env/lib/python3.8/site-packages/pip/_internal/utils/filetypes.py new file mode 100644 index 000000000..daa0ca771 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/utils/filetypes.py @@ -0,0 +1,16 @@ +"""Filetype information. +""" +from pip._internal.utils.typing import MYPY_CHECK_RUNNING + +if MYPY_CHECK_RUNNING: + from typing import Tuple + +WHEEL_EXTENSION = '.whl' +BZ2_EXTENSIONS = ('.tar.bz2', '.tbz') # type: Tuple[str, ...] +XZ_EXTENSIONS = ('.tar.xz', '.txz', '.tlz', + '.tar.lz', '.tar.lzma') # type: Tuple[str, ...] +ZIP_EXTENSIONS = ('.zip', WHEEL_EXTENSION) # type: Tuple[str, ...] +TAR_EXTENSIONS = ('.tar.gz', '.tgz', '.tar') # type: Tuple[str, ...] +ARCHIVE_EXTENSIONS = ( + ZIP_EXTENSIONS + BZ2_EXTENSIONS + TAR_EXTENSIONS + XZ_EXTENSIONS +) diff --git a/env/lib/python3.8/site-packages/pip/_internal/utils/glibc.py b/env/lib/python3.8/site-packages/pip/_internal/utils/glibc.py new file mode 100644 index 000000000..361042441 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/utils/glibc.py @@ -0,0 +1,98 @@ +# The following comment should be removed at some point in the future. +# mypy: strict-optional=False + +from __future__ import absolute_import + +import os +import sys + +from pip._internal.utils.typing import MYPY_CHECK_RUNNING + +if MYPY_CHECK_RUNNING: + from typing import Optional, Tuple + + +def glibc_version_string(): + # type: () -> Optional[str] + "Returns glibc version string, or None if not using glibc." + return glibc_version_string_confstr() or glibc_version_string_ctypes() + + +def glibc_version_string_confstr(): + # type: () -> Optional[str] + "Primary implementation of glibc_version_string using os.confstr." + # os.confstr is quite a bit faster than ctypes.DLL. It's also less likely + # to be broken or missing. This strategy is used in the standard library + # platform module: + # https://github.com/python/cpython/blob/fcf1d003bf4f0100c9d0921ff3d70e1127ca1b71/Lib/platform.py#L175-L183 + if sys.platform == "win32": + return None + try: + # os.confstr("CS_GNU_LIBC_VERSION") returns a string like "glibc 2.17": + _, version = os.confstr("CS_GNU_LIBC_VERSION").split() + except (AttributeError, OSError, ValueError): + # os.confstr() or CS_GNU_LIBC_VERSION not available (or a bad value)... + return None + return version + + +def glibc_version_string_ctypes(): + # type: () -> Optional[str] + "Fallback implementation of glibc_version_string using ctypes." + + try: + import ctypes + except ImportError: + return None + + # ctypes.CDLL(None) internally calls dlopen(NULL), and as the dlopen + # manpage says, "If filename is NULL, then the returned handle is for the + # main program". This way we can let the linker do the work to figure out + # which libc our process is actually using. + process_namespace = ctypes.CDLL(None) + try: + gnu_get_libc_version = process_namespace.gnu_get_libc_version + except AttributeError: + # Symbol doesn't exist -> therefore, we are not linked to + # glibc. + return None + + # Call gnu_get_libc_version, which returns a string like "2.5" + gnu_get_libc_version.restype = ctypes.c_char_p + version_str = gnu_get_libc_version() + # py2 / py3 compatibility: + if not isinstance(version_str, str): + version_str = version_str.decode("ascii") + + return version_str + + +# platform.libc_ver regularly returns completely nonsensical glibc +# versions. E.g. on my computer, platform says: +# +# ~$ python2.7 -c 'import platform; print(platform.libc_ver())' +# ('glibc', '2.7') +# ~$ python3.5 -c 'import platform; print(platform.libc_ver())' +# ('glibc', '2.9') +# +# But the truth is: +# +# ~$ ldd --version +# ldd (Debian GLIBC 2.22-11) 2.22 +# +# This is unfortunate, because it means that the linehaul data on libc +# versions that was generated by pip 8.1.2 and earlier is useless and +# misleading. Solution: instead of using platform, use our code that actually +# works. +def libc_ver(): + # type: () -> Tuple[str, str] + """Try to determine the glibc version + + Returns a tuple of strings (lib, version) which default to empty strings + in case the lookup fails. + """ + glibc_version = glibc_version_string() + if glibc_version is None: + return ("", "") + else: + return ("glibc", glibc_version) diff --git a/env/lib/python3.8/site-packages/pip/_internal/utils/hashes.py b/env/lib/python3.8/site-packages/pip/_internal/utils/hashes.py new file mode 100644 index 000000000..4c41551a2 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/utils/hashes.py @@ -0,0 +1,131 @@ +from __future__ import absolute_import + +import hashlib + +from pip._vendor.six import iteritems, iterkeys, itervalues + +from pip._internal.exceptions import ( + HashMismatch, + HashMissing, + InstallationError, +) +from pip._internal.utils.misc import read_chunks +from pip._internal.utils.typing import MYPY_CHECK_RUNNING + +if MYPY_CHECK_RUNNING: + from typing import ( + Dict, List, BinaryIO, NoReturn, Iterator + ) + from pip._vendor.six import PY3 + if PY3: + from hashlib import _Hash + else: + from hashlib import _hash as _Hash + + +# The recommended hash algo of the moment. Change this whenever the state of +# the art changes; it won't hurt backward compatibility. +FAVORITE_HASH = 'sha256' + + +# Names of hashlib algorithms allowed by the --hash option and ``pip hash`` +# Currently, those are the ones at least as collision-resistant as sha256. +STRONG_HASHES = ['sha256', 'sha384', 'sha512'] + + +class Hashes(object): + """A wrapper that builds multiple hashes at once and checks them against + known-good values + + """ + def __init__(self, hashes=None): + # type: (Dict[str, List[str]]) -> None + """ + :param hashes: A dict of algorithm names pointing to lists of allowed + hex digests + """ + self._allowed = {} if hashes is None else hashes + + @property + def digest_count(self): + # type: () -> int + return sum(len(digests) for digests in self._allowed.values()) + + def is_hash_allowed( + self, + hash_name, # type: str + hex_digest, # type: str + ): + # type: (...) -> bool + """Return whether the given hex digest is allowed.""" + return hex_digest in self._allowed.get(hash_name, []) + + def check_against_chunks(self, chunks): + # type: (Iterator[bytes]) -> None + """Check good hashes against ones built from iterable of chunks of + data. + + Raise HashMismatch if none match. + + """ + gots = {} + for hash_name in iterkeys(self._allowed): + try: + gots[hash_name] = hashlib.new(hash_name) + except (ValueError, TypeError): + raise InstallationError('Unknown hash name: %s' % hash_name) + + for chunk in chunks: + for hash in itervalues(gots): + hash.update(chunk) + + for hash_name, got in iteritems(gots): + if got.hexdigest() in self._allowed[hash_name]: + return + self._raise(gots) + + def _raise(self, gots): + # type: (Dict[str, _Hash]) -> NoReturn + raise HashMismatch(self._allowed, gots) + + def check_against_file(self, file): + # type: (BinaryIO) -> None + """Check good hashes against a file-like object + + Raise HashMismatch if none match. + + """ + return self.check_against_chunks(read_chunks(file)) + + def check_against_path(self, path): + # type: (str) -> None + with open(path, 'rb') as file: + return self.check_against_file(file) + + def __nonzero__(self): + # type: () -> bool + """Return whether I know any known-good hashes.""" + return bool(self._allowed) + + def __bool__(self): + # type: () -> bool + return self.__nonzero__() + + +class MissingHashes(Hashes): + """A workalike for Hashes used when we're missing a hash for a requirement + + It computes the actual hash of the requirement and raises a HashMissing + exception showing it to the user. + + """ + def __init__(self): + # type: () -> None + """Don't offer the ``hashes`` kwarg.""" + # Pass our favorite hash in to generate a "gotten hash". With the + # empty list, it will never match, so an error will always raise. + super(MissingHashes, self).__init__(hashes={FAVORITE_HASH: []}) + + def _raise(self, gots): + # type: (Dict[str, _Hash]) -> NoReturn + raise HashMissing(gots[FAVORITE_HASH].hexdigest()) diff --git a/env/lib/python3.8/site-packages/pip/_internal/utils/inject_securetransport.py b/env/lib/python3.8/site-packages/pip/_internal/utils/inject_securetransport.py new file mode 100644 index 000000000..5b93b1d67 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/utils/inject_securetransport.py @@ -0,0 +1,36 @@ +"""A helper module that injects SecureTransport, on import. + +The import should be done as early as possible, to ensure all requests and +sessions (or whatever) are created after injecting SecureTransport. + +Note that we only do the injection on macOS, when the linked OpenSSL is too +old to handle TLSv1.2. +""" + +import sys + + +def inject_securetransport(): + # type: () -> None + # Only relevant on macOS + if sys.platform != "darwin": + return + + try: + import ssl + except ImportError: + return + + # Checks for OpenSSL 1.0.1 + if ssl.OPENSSL_VERSION_NUMBER >= 0x1000100f: + return + + try: + from pip._vendor.urllib3.contrib import securetransport + except (ImportError, OSError): + return + + securetransport.inject_into_urllib3() + + +inject_securetransport() diff --git a/env/lib/python3.8/site-packages/pip/_internal/utils/logging.py b/env/lib/python3.8/site-packages/pip/_internal/utils/logging.py new file mode 100644 index 000000000..7767111a6 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/utils/logging.py @@ -0,0 +1,398 @@ +# The following comment should be removed at some point in the future. +# mypy: disallow-untyped-defs=False + +from __future__ import absolute_import + +import contextlib +import errno +import logging +import logging.handlers +import os +import sys +from logging import Filter, getLogger + +from pip._vendor.six import PY2 + +from pip._internal.utils.compat import WINDOWS +from pip._internal.utils.deprecation import DEPRECATION_MSG_PREFIX +from pip._internal.utils.misc import ensure_dir + +try: + import threading +except ImportError: + import dummy_threading as threading # type: ignore + + +try: + # Use "import as" and set colorama in the else clause to avoid mypy + # errors and get the following correct revealed type for colorama: + # `Union[_importlib_modulespec.ModuleType, None]` + # Otherwise, we get an error like the following in the except block: + # > Incompatible types in assignment (expression has type "None", + # variable has type Module) + # TODO: eliminate the need to use "import as" once mypy addresses some + # of its issues with conditional imports. Here is an umbrella issue: + # https://github.com/python/mypy/issues/1297 + from pip._vendor import colorama as _colorama +# Lots of different errors can come from this, including SystemError and +# ImportError. +except Exception: + colorama = None +else: + # Import Fore explicitly rather than accessing below as colorama.Fore + # to avoid the following error running mypy: + # > Module has no attribute "Fore" + # TODO: eliminate the need to import Fore once mypy addresses some of its + # issues with conditional imports. This particular case could be an + # instance of the following issue (but also see the umbrella issue above): + # https://github.com/python/mypy/issues/3500 + from pip._vendor.colorama import Fore + + colorama = _colorama + + +_log_state = threading.local() +_log_state.indentation = 0 +subprocess_logger = getLogger('pip.subprocessor') + + +class BrokenStdoutLoggingError(Exception): + """ + Raised if BrokenPipeError occurs for the stdout stream while logging. + """ + pass + + +# BrokenPipeError does not exist in Python 2 and, in addition, manifests +# differently in Windows and non-Windows. +if WINDOWS: + # In Windows, a broken pipe can show up as EINVAL rather than EPIPE: + # https://bugs.python.org/issue19612 + # https://bugs.python.org/issue30418 + if PY2: + def _is_broken_pipe_error(exc_class, exc): + """See the docstring for non-Windows Python 3 below.""" + return (exc_class is IOError and + exc.errno in (errno.EINVAL, errno.EPIPE)) + else: + # In Windows, a broken pipe IOError became OSError in Python 3. + def _is_broken_pipe_error(exc_class, exc): + """See the docstring for non-Windows Python 3 below.""" + return ((exc_class is BrokenPipeError) or # noqa: F821 + (exc_class is OSError and + exc.errno in (errno.EINVAL, errno.EPIPE))) +elif PY2: + def _is_broken_pipe_error(exc_class, exc): + """See the docstring for non-Windows Python 3 below.""" + return (exc_class is IOError and exc.errno == errno.EPIPE) +else: + # Then we are in the non-Windows Python 3 case. + def _is_broken_pipe_error(exc_class, exc): + """ + Return whether an exception is a broken pipe error. + + Args: + exc_class: an exception class. + exc: an exception instance. + """ + return (exc_class is BrokenPipeError) # noqa: F821 + + +@contextlib.contextmanager +def indent_log(num=2): + """ + A context manager which will cause the log output to be indented for any + log messages emitted inside it. + """ + _log_state.indentation += num + try: + yield + finally: + _log_state.indentation -= num + + +def get_indentation(): + return getattr(_log_state, 'indentation', 0) + + +class IndentingFormatter(logging.Formatter): + + def __init__(self, *args, **kwargs): + """ + A logging.Formatter that obeys the indent_log() context manager. + + :param add_timestamp: A bool indicating output lines should be prefixed + with their record's timestamp. + """ + self.add_timestamp = kwargs.pop("add_timestamp", False) + super(IndentingFormatter, self).__init__(*args, **kwargs) + + def get_message_start(self, formatted, levelno): + """ + Return the start of the formatted log message (not counting the + prefix to add to each line). + """ + if levelno < logging.WARNING: + return '' + if formatted.startswith(DEPRECATION_MSG_PREFIX): + # Then the message already has a prefix. We don't want it to + # look like "WARNING: DEPRECATION: ...." + return '' + if levelno < logging.ERROR: + return 'WARNING: ' + + return 'ERROR: ' + + def format(self, record): + """ + Calls the standard formatter, but will indent all of the log message + lines by our current indentation level. + """ + formatted = super(IndentingFormatter, self).format(record) + message_start = self.get_message_start(formatted, record.levelno) + formatted = message_start + formatted + + prefix = '' + if self.add_timestamp: + # TODO: Use Formatter.default_time_format after dropping PY2. + t = self.formatTime(record, "%Y-%m-%dT%H:%M:%S") + prefix = '%s,%03d ' % (t, record.msecs) + prefix += " " * get_indentation() + formatted = "".join([ + prefix + line + for line in formatted.splitlines(True) + ]) + return formatted + + +def _color_wrap(*colors): + def wrapped(inp): + return "".join(list(colors) + [inp, colorama.Style.RESET_ALL]) + return wrapped + + +class ColorizedStreamHandler(logging.StreamHandler): + + # Don't build up a list of colors if we don't have colorama + if colorama: + COLORS = [ + # This needs to be in order from highest logging level to lowest. + (logging.ERROR, _color_wrap(Fore.RED)), + (logging.WARNING, _color_wrap(Fore.YELLOW)), + ] + else: + COLORS = [] + + def __init__(self, stream=None, no_color=None): + logging.StreamHandler.__init__(self, stream) + self._no_color = no_color + + if WINDOWS and colorama: + self.stream = colorama.AnsiToWin32(self.stream) + + def _using_stdout(self): + """ + Return whether the handler is using sys.stdout. + """ + if WINDOWS and colorama: + # Then self.stream is an AnsiToWin32 object. + return self.stream.wrapped is sys.stdout + + return self.stream is sys.stdout + + def should_color(self): + # Don't colorize things if we do not have colorama or if told not to + if not colorama or self._no_color: + return False + + real_stream = ( + self.stream if not isinstance(self.stream, colorama.AnsiToWin32) + else self.stream.wrapped + ) + + # If the stream is a tty we should color it + if hasattr(real_stream, "isatty") and real_stream.isatty(): + return True + + # If we have an ANSI term we should color it + if os.environ.get("TERM") == "ANSI": + return True + + # If anything else we should not color it + return False + + def format(self, record): + msg = logging.StreamHandler.format(self, record) + + if self.should_color(): + for level, color in self.COLORS: + if record.levelno >= level: + msg = color(msg) + break + + return msg + + # The logging module says handleError() can be customized. + def handleError(self, record): + exc_class, exc = sys.exc_info()[:2] + # If a broken pipe occurred while calling write() or flush() on the + # stdout stream in logging's Handler.emit(), then raise our special + # exception so we can handle it in main() instead of logging the + # broken pipe error and continuing. + if (exc_class and self._using_stdout() and + _is_broken_pipe_error(exc_class, exc)): + raise BrokenStdoutLoggingError() + + return super(ColorizedStreamHandler, self).handleError(record) + + +class BetterRotatingFileHandler(logging.handlers.RotatingFileHandler): + + def _open(self): + ensure_dir(os.path.dirname(self.baseFilename)) + return logging.handlers.RotatingFileHandler._open(self) + + +class MaxLevelFilter(Filter): + + def __init__(self, level): + self.level = level + + def filter(self, record): + return record.levelno < self.level + + +class ExcludeLoggerFilter(Filter): + + """ + A logging Filter that excludes records from a logger (or its children). + """ + + def filter(self, record): + # The base Filter class allows only records from a logger (or its + # children). + return not super(ExcludeLoggerFilter, self).filter(record) + + +def setup_logging(verbosity, no_color, user_log_file): + """Configures and sets up all of the logging + + Returns the requested logging level, as its integer value. + """ + + # Determine the level to be logging at. + if verbosity >= 1: + level = "DEBUG" + elif verbosity == -1: + level = "WARNING" + elif verbosity == -2: + level = "ERROR" + elif verbosity <= -3: + level = "CRITICAL" + else: + level = "INFO" + + level_number = getattr(logging, level) + + # The "root" logger should match the "console" level *unless* we also need + # to log to a user log file. + include_user_log = user_log_file is not None + if include_user_log: + additional_log_file = user_log_file + root_level = "DEBUG" + else: + additional_log_file = "/dev/null" + root_level = level + + # Disable any logging besides WARNING unless we have DEBUG level logging + # enabled for vendored libraries. + vendored_log_level = "WARNING" if level in ["INFO", "ERROR"] else "DEBUG" + + # Shorthands for clarity + log_streams = { + "stdout": "ext://sys.stdout", + "stderr": "ext://sys.stderr", + } + handler_classes = { + "stream": "pip._internal.utils.logging.ColorizedStreamHandler", + "file": "pip._internal.utils.logging.BetterRotatingFileHandler", + } + handlers = ["console", "console_errors", "console_subprocess"] + ( + ["user_log"] if include_user_log else [] + ) + + logging.config.dictConfig({ + "version": 1, + "disable_existing_loggers": False, + "filters": { + "exclude_warnings": { + "()": "pip._internal.utils.logging.MaxLevelFilter", + "level": logging.WARNING, + }, + "restrict_to_subprocess": { + "()": "logging.Filter", + "name": subprocess_logger.name, + }, + "exclude_subprocess": { + "()": "pip._internal.utils.logging.ExcludeLoggerFilter", + "name": subprocess_logger.name, + }, + }, + "formatters": { + "indent": { + "()": IndentingFormatter, + "format": "%(message)s", + }, + "indent_with_timestamp": { + "()": IndentingFormatter, + "format": "%(message)s", + "add_timestamp": True, + }, + }, + "handlers": { + "console": { + "level": level, + "class": handler_classes["stream"], + "no_color": no_color, + "stream": log_streams["stdout"], + "filters": ["exclude_subprocess", "exclude_warnings"], + "formatter": "indent", + }, + "console_errors": { + "level": "WARNING", + "class": handler_classes["stream"], + "no_color": no_color, + "stream": log_streams["stderr"], + "filters": ["exclude_subprocess"], + "formatter": "indent", + }, + # A handler responsible for logging to the console messages + # from the "subprocessor" logger. + "console_subprocess": { + "level": level, + "class": handler_classes["stream"], + "no_color": no_color, + "stream": log_streams["stderr"], + "filters": ["restrict_to_subprocess"], + "formatter": "indent", + }, + "user_log": { + "level": "DEBUG", + "class": handler_classes["file"], + "filename": additional_log_file, + "delay": True, + "formatter": "indent_with_timestamp", + }, + }, + "root": { + "level": root_level, + "handlers": handlers, + }, + "loggers": { + "pip._vendor": { + "level": vendored_log_level + } + }, + }) + + return level_number diff --git a/env/lib/python3.8/site-packages/pip/_internal/utils/marker_files.py b/env/lib/python3.8/site-packages/pip/_internal/utils/marker_files.py new file mode 100644 index 000000000..42ea81405 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/utils/marker_files.py @@ -0,0 +1,25 @@ +import os.path + +DELETE_MARKER_MESSAGE = '''\ +This file is placed here by pip to indicate the source was put +here by pip. + +Once this package is successfully installed this source code will be +deleted (unless you remove this file). +''' +PIP_DELETE_MARKER_FILENAME = 'pip-delete-this-directory.txt' + + +def has_delete_marker_file(directory): + # type: (str) -> bool + return os.path.exists(os.path.join(directory, PIP_DELETE_MARKER_FILENAME)) + + +def write_delete_marker_file(directory): + # type: (str) -> None + """ + Write the pip delete marker file into this directory. + """ + filepath = os.path.join(directory, PIP_DELETE_MARKER_FILENAME) + with open(filepath, 'w') as marker_fp: + marker_fp.write(DELETE_MARKER_MESSAGE) diff --git a/env/lib/python3.8/site-packages/pip/_internal/utils/misc.py b/env/lib/python3.8/site-packages/pip/_internal/utils/misc.py new file mode 100644 index 000000000..554af0bf7 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/utils/misc.py @@ -0,0 +1,904 @@ +# The following comment should be removed at some point in the future. +# mypy: strict-optional=False +# mypy: disallow-untyped-defs=False + +from __future__ import absolute_import + +import contextlib +import errno +import getpass +import hashlib +import io +import logging +import os +import posixpath +import shutil +import stat +import sys +from collections import deque + +from pip._vendor import pkg_resources +# NOTE: retrying is not annotated in typeshed as on 2017-07-17, which is +# why we ignore the type on this import. +from pip._vendor.retrying import retry # type: ignore +from pip._vendor.six import PY2, text_type +from pip._vendor.six.moves import input +from pip._vendor.six.moves.urllib import parse as urllib_parse +from pip._vendor.six.moves.urllib.parse import unquote as urllib_unquote + +from pip import __version__ +from pip._internal.exceptions import CommandError +from pip._internal.locations import ( + get_major_minor_version, + site_packages, + user_site, +) +from pip._internal.utils.compat import ( + WINDOWS, + expanduser, + stdlib_pkgs, + str_to_display, +) +from pip._internal.utils.typing import MYPY_CHECK_RUNNING, cast +from pip._internal.utils.virtualenv import ( + running_under_virtualenv, + virtualenv_no_global, +) + +if PY2: + from io import BytesIO as StringIO +else: + from io import StringIO + +if MYPY_CHECK_RUNNING: + from typing import ( + Any, AnyStr, Container, Iterable, List, Optional, Text, + Tuple, Union, + ) + from pip._vendor.pkg_resources import Distribution + + VersionInfo = Tuple[int, int, int] + + +__all__ = ['rmtree', 'display_path', 'backup_dir', + 'ask', 'splitext', + 'format_size', 'is_installable_dir', + 'normalize_path', + 'renames', 'get_prog', + 'captured_stdout', 'ensure_dir', + 'get_installed_version', 'remove_auth_from_url'] + + +logger = logging.getLogger(__name__) + + +def get_pip_version(): + # type: () -> str + pip_pkg_dir = os.path.join(os.path.dirname(__file__), "..", "..") + pip_pkg_dir = os.path.abspath(pip_pkg_dir) + + return ( + 'pip {} from {} (python {})'.format( + __version__, pip_pkg_dir, get_major_minor_version(), + ) + ) + + +def normalize_version_info(py_version_info): + # type: (Tuple[int, ...]) -> Tuple[int, int, int] + """ + Convert a tuple of ints representing a Python version to one of length + three. + + :param py_version_info: a tuple of ints representing a Python version, + or None to specify no version. The tuple can have any length. + + :return: a tuple of length three if `py_version_info` is non-None. + Otherwise, return `py_version_info` unchanged (i.e. None). + """ + if len(py_version_info) < 3: + py_version_info += (3 - len(py_version_info)) * (0,) + elif len(py_version_info) > 3: + py_version_info = py_version_info[:3] + + return cast('VersionInfo', py_version_info) + + +def ensure_dir(path): + # type: (AnyStr) -> None + """os.path.makedirs without EEXIST.""" + try: + os.makedirs(path) + except OSError as e: + # Windows can raise spurious ENOTEMPTY errors. See #6426. + if e.errno != errno.EEXIST and e.errno != errno.ENOTEMPTY: + raise + + +def get_prog(): + # type: () -> str + try: + prog = os.path.basename(sys.argv[0]) + if prog in ('__main__.py', '-c'): + return "%s -m pip" % sys.executable + else: + return prog + except (AttributeError, TypeError, IndexError): + pass + return 'pip' + + +# Retry every half second for up to 3 seconds +@retry(stop_max_delay=3000, wait_fixed=500) +def rmtree(dir, ignore_errors=False): + # type: (str, bool) -> None + shutil.rmtree(dir, ignore_errors=ignore_errors, + onerror=rmtree_errorhandler) + + +def rmtree_errorhandler(func, path, exc_info): + """On Windows, the files in .svn are read-only, so when rmtree() tries to + remove them, an exception is thrown. We catch that here, remove the + read-only attribute, and hopefully continue without problems.""" + try: + has_attr_readonly = not (os.stat(path).st_mode & stat.S_IWRITE) + except (IOError, OSError): + # it's equivalent to os.path.exists + return + + if has_attr_readonly: + # convert to read/write + os.chmod(path, stat.S_IWRITE) + # use the original function to repeat the operation + func(path) + return + else: + raise + + +def path_to_display(path): + # type: (Optional[Union[str, Text]]) -> Optional[Text] + """ + Convert a bytes (or text) path to text (unicode in Python 2) for display + and logging purposes. + + This function should never error out. Also, this function is mainly needed + for Python 2 since in Python 3 str paths are already text. + """ + if path is None: + return None + if isinstance(path, text_type): + return path + # Otherwise, path is a bytes object (str in Python 2). + try: + display_path = path.decode(sys.getfilesystemencoding(), 'strict') + except UnicodeDecodeError: + # Include the full bytes to make troubleshooting easier, even though + # it may not be very human readable. + if PY2: + # Convert the bytes to a readable str representation using + # repr(), and then convert the str to unicode. + # Also, we add the prefix "b" to the repr() return value both + # to make the Python 2 output look like the Python 3 output, and + # to signal to the user that this is a bytes representation. + display_path = str_to_display('b{!r}'.format(path)) + else: + # Silence the "F821 undefined name 'ascii'" flake8 error since + # in Python 3 ascii() is a built-in. + display_path = ascii(path) # noqa: F821 + + return display_path + + +def display_path(path): + # type: (Union[str, Text]) -> str + """Gives the display value for a given path, making it relative to cwd + if possible.""" + path = os.path.normcase(os.path.abspath(path)) + if sys.version_info[0] == 2: + path = path.decode(sys.getfilesystemencoding(), 'replace') + path = path.encode(sys.getdefaultencoding(), 'replace') + if path.startswith(os.getcwd() + os.path.sep): + path = '.' + path[len(os.getcwd()):] + return path + + +def backup_dir(dir, ext='.bak'): + # type: (str, str) -> str + """Figure out the name of a directory to back up the given dir to + (adding .bak, .bak2, etc)""" + n = 1 + extension = ext + while os.path.exists(dir + extension): + n += 1 + extension = ext + str(n) + return dir + extension + + +def ask_path_exists(message, options): + # type: (str, Iterable[str]) -> str + for action in os.environ.get('PIP_EXISTS_ACTION', '').split(): + if action in options: + return action + return ask(message, options) + + +def _check_no_input(message): + # type: (str) -> None + """Raise an error if no input is allowed.""" + if os.environ.get('PIP_NO_INPUT'): + raise Exception( + 'No input was expected ($PIP_NO_INPUT set); question: %s' % + message + ) + + +def ask(message, options): + # type: (str, Iterable[str]) -> str + """Ask the message interactively, with the given possible responses""" + while 1: + _check_no_input(message) + response = input(message) + response = response.strip().lower() + if response not in options: + print( + 'Your response (%r) was not one of the expected responses: ' + '%s' % (response, ', '.join(options)) + ) + else: + return response + + +def ask_input(message): + # type: (str) -> str + """Ask for input interactively.""" + _check_no_input(message) + return input(message) + + +def ask_password(message): + # type: (str) -> str + """Ask for a password interactively.""" + _check_no_input(message) + return getpass.getpass(message) + + +def format_size(bytes): + # type: (float) -> str + if bytes > 1000 * 1000: + return '%.1f MB' % (bytes / 1000.0 / 1000) + elif bytes > 10 * 1000: + return '%i kB' % (bytes / 1000) + elif bytes > 1000: + return '%.1f kB' % (bytes / 1000.0) + else: + return '%i bytes' % bytes + + +def is_installable_dir(path): + # type: (str) -> bool + """Is path is a directory containing setup.py or pyproject.toml? + """ + if not os.path.isdir(path): + return False + setup_py = os.path.join(path, 'setup.py') + if os.path.isfile(setup_py): + return True + pyproject_toml = os.path.join(path, 'pyproject.toml') + if os.path.isfile(pyproject_toml): + return True + return False + + +def read_chunks(file, size=io.DEFAULT_BUFFER_SIZE): + """Yield pieces of data from a file-like object until EOF.""" + while True: + chunk = file.read(size) + if not chunk: + break + yield chunk + + +def normalize_path(path, resolve_symlinks=True): + # type: (str, bool) -> str + """ + Convert a path to its canonical, case-normalized, absolute version. + + """ + path = expanduser(path) + if resolve_symlinks: + path = os.path.realpath(path) + else: + path = os.path.abspath(path) + return os.path.normcase(path) + + +def splitext(path): + # type: (str) -> Tuple[str, str] + """Like os.path.splitext, but take off .tar too""" + base, ext = posixpath.splitext(path) + if base.lower().endswith('.tar'): + ext = base[-4:] + ext + base = base[:-4] + return base, ext + + +def renames(old, new): + # type: (str, str) -> None + """Like os.renames(), but handles renaming across devices.""" + # Implementation borrowed from os.renames(). + head, tail = os.path.split(new) + if head and tail and not os.path.exists(head): + os.makedirs(head) + + shutil.move(old, new) + + head, tail = os.path.split(old) + if head and tail: + try: + os.removedirs(head) + except OSError: + pass + + +def is_local(path): + # type: (str) -> bool + """ + Return True if this is a path pip is allowed to modify. + + If we're in a virtualenv, sys.prefix points to the virtualenv's + prefix; only sys.prefix is considered local. + + If we're not in a virtualenv, in general we can modify anything. + However, if the OS vendor has configured distutils to install + somewhere other than sys.prefix (which could be a subdirectory of + sys.prefix, e.g. /usr/local), we consider sys.prefix itself nonlocal + and the domain of the OS vendor. (In other words, everything _other + than_ sys.prefix is considered local.) + + Caution: this function assumes the head of path has been normalized + with normalize_path. + """ + + path = normalize_path(path) + prefix = normalize_path(sys.prefix) + + if running_under_virtualenv(): + return path.startswith(normalize_path(sys.prefix)) + else: + from pip._internal.locations import distutils_scheme + if path.startswith(prefix): + for local_path in distutils_scheme("").values(): + if path.startswith(normalize_path(local_path)): + return True + return False + else: + return True + + +def dist_is_local(dist): + # type: (Distribution) -> bool + """ + Return True if given Distribution object is installed somewhere pip + is allowed to modify. + + """ + return is_local(dist_location(dist)) + + +def dist_in_usersite(dist): + # type: (Distribution) -> bool + """ + Return True if given Distribution is installed in user site. + """ + return dist_location(dist).startswith(normalize_path(user_site)) + + +def dist_in_site_packages(dist): + # type: (Distribution) -> bool + """ + Return True if given Distribution is installed in + sysconfig.get_python_lib(). + """ + return dist_location(dist).startswith(normalize_path(site_packages)) + + +def dist_is_editable(dist): + # type: (Distribution) -> bool + """ + Return True if given Distribution is an editable install. + """ + for path_item in sys.path: + egg_link = os.path.join(path_item, dist.project_name + '.egg-link') + if os.path.isfile(egg_link): + return True + return False + + +def get_installed_distributions( + local_only=True, # type: bool + skip=stdlib_pkgs, # type: Container[str] + include_editables=True, # type: bool + editables_only=False, # type: bool + user_only=False, # type: bool + paths=None # type: Optional[List[str]] +): + # type: (...) -> List[Distribution] + """ + Return a list of installed Distribution objects. + + If ``local_only`` is True (default), only return installations + local to the current virtualenv, if in a virtualenv. + + ``skip`` argument is an iterable of lower-case project names to + ignore; defaults to stdlib_pkgs + + If ``include_editables`` is False, don't report editables. + + If ``editables_only`` is True , only report editables. + + If ``user_only`` is True , only report installations in the user + site directory. + + If ``paths`` is set, only report the distributions present at the + specified list of locations. + """ + if paths: + working_set = pkg_resources.WorkingSet(paths) + else: + working_set = pkg_resources.working_set + + if local_only: + local_test = dist_is_local + else: + def local_test(d): + return True + + if include_editables: + def editable_test(d): + return True + else: + def editable_test(d): + return not dist_is_editable(d) + + if editables_only: + def editables_only_test(d): + return dist_is_editable(d) + else: + def editables_only_test(d): + return True + + if user_only: + user_test = dist_in_usersite + else: + def user_test(d): + return True + + return [d for d in working_set + if local_test(d) and + d.key not in skip and + editable_test(d) and + editables_only_test(d) and + user_test(d) + ] + + +def egg_link_path(dist): + # type: (Distribution) -> Optional[str] + """ + Return the path for the .egg-link file if it exists, otherwise, None. + + There's 3 scenarios: + 1) not in a virtualenv + try to find in site.USER_SITE, then site_packages + 2) in a no-global virtualenv + try to find in site_packages + 3) in a yes-global virtualenv + try to find in site_packages, then site.USER_SITE + (don't look in global location) + + For #1 and #3, there could be odd cases, where there's an egg-link in 2 + locations. + + This method will just return the first one found. + """ + sites = [] + if running_under_virtualenv(): + sites.append(site_packages) + if not virtualenv_no_global() and user_site: + sites.append(user_site) + else: + if user_site: + sites.append(user_site) + sites.append(site_packages) + + for site in sites: + egglink = os.path.join(site, dist.project_name) + '.egg-link' + if os.path.isfile(egglink): + return egglink + return None + + +def dist_location(dist): + # type: (Distribution) -> str + """ + Get the site-packages location of this distribution. Generally + this is dist.location, except in the case of develop-installed + packages, where dist.location is the source code location, and we + want to know where the egg-link file is. + + The returned location is normalized (in particular, with symlinks removed). + """ + egg_link = egg_link_path(dist) + if egg_link: + return normalize_path(egg_link) + return normalize_path(dist.location) + + +def write_output(msg, *args): + # type: (str, str) -> None + logger.info(msg, *args) + + +class FakeFile(object): + """Wrap a list of lines in an object with readline() to make + ConfigParser happy.""" + def __init__(self, lines): + self._gen = (l for l in lines) + + def readline(self): + try: + try: + return next(self._gen) + except NameError: + return self._gen.next() + except StopIteration: + return '' + + def __iter__(self): + return self._gen + + +class StreamWrapper(StringIO): + + @classmethod + def from_stream(cls, orig_stream): + cls.orig_stream = orig_stream + return cls() + + # compileall.compile_dir() needs stdout.encoding to print to stdout + @property + def encoding(self): + return self.orig_stream.encoding + + +@contextlib.contextmanager +def captured_output(stream_name): + """Return a context manager used by captured_stdout/stdin/stderr + that temporarily replaces the sys stream *stream_name* with a StringIO. + + Taken from Lib/support/__init__.py in the CPython repo. + """ + orig_stdout = getattr(sys, stream_name) + setattr(sys, stream_name, StreamWrapper.from_stream(orig_stdout)) + try: + yield getattr(sys, stream_name) + finally: + setattr(sys, stream_name, orig_stdout) + + +def captured_stdout(): + """Capture the output of sys.stdout: + + with captured_stdout() as stdout: + print('hello') + self.assertEqual(stdout.getvalue(), 'hello\n') + + Taken from Lib/support/__init__.py in the CPython repo. + """ + return captured_output('stdout') + + +def captured_stderr(): + """ + See captured_stdout(). + """ + return captured_output('stderr') + + +class cached_property(object): + """A property that is only computed once per instance and then replaces + itself with an ordinary attribute. Deleting the attribute resets the + property. + + Source: https://github.com/bottlepy/bottle/blob/0.11.5/bottle.py#L175 + """ + + def __init__(self, func): + self.__doc__ = getattr(func, '__doc__') + self.func = func + + def __get__(self, obj, cls): + if obj is None: + # We're being accessed from the class itself, not from an object + return self + value = obj.__dict__[self.func.__name__] = self.func(obj) + return value + + +def get_installed_version(dist_name, working_set=None): + """Get the installed version of dist_name avoiding pkg_resources cache""" + # Create a requirement that we'll look for inside of setuptools. + req = pkg_resources.Requirement.parse(dist_name) + + if working_set is None: + # We want to avoid having this cached, so we need to construct a new + # working set each time. + working_set = pkg_resources.WorkingSet() + + # Get the installed distribution from our working set + dist = working_set.find(req) + + # Check to see if we got an installed distribution or not, if we did + # we want to return it's version. + return dist.version if dist else None + + +def consume(iterator): + """Consume an iterable at C speed.""" + deque(iterator, maxlen=0) + + +# Simulates an enum +def enum(*sequential, **named): + enums = dict(zip(sequential, range(len(sequential))), **named) + reverse = {value: key for key, value in enums.items()} + enums['reverse_mapping'] = reverse + return type('Enum', (), enums) + + +def build_netloc(host, port): + # type: (str, Optional[int]) -> str + """ + Build a netloc from a host-port pair + """ + if port is None: + return host + if ':' in host: + # Only wrap host with square brackets when it is IPv6 + host = '[{}]'.format(host) + return '{}:{}'.format(host, port) + + +def build_url_from_netloc(netloc, scheme='https'): + # type: (str, str) -> str + """ + Build a full URL from a netloc. + """ + if netloc.count(':') >= 2 and '@' not in netloc and '[' not in netloc: + # It must be a bare IPv6 address, so wrap it with brackets. + netloc = '[{}]'.format(netloc) + return '{}://{}'.format(scheme, netloc) + + +def parse_netloc(netloc): + # type: (str) -> Tuple[str, Optional[int]] + """ + Return the host-port pair from a netloc. + """ + url = build_url_from_netloc(netloc) + parsed = urllib_parse.urlparse(url) + return parsed.hostname, parsed.port + + +def split_auth_from_netloc(netloc): + """ + Parse out and remove the auth information from a netloc. + + Returns: (netloc, (username, password)). + """ + if '@' not in netloc: + return netloc, (None, None) + + # Split from the right because that's how urllib.parse.urlsplit() + # behaves if more than one @ is present (which can be checked using + # the password attribute of urlsplit()'s return value). + auth, netloc = netloc.rsplit('@', 1) + if ':' in auth: + # Split from the left because that's how urllib.parse.urlsplit() + # behaves if more than one : is present (which again can be checked + # using the password attribute of the return value) + user_pass = auth.split(':', 1) + else: + user_pass = auth, None + + user_pass = tuple( + None if x is None else urllib_unquote(x) for x in user_pass + ) + + return netloc, user_pass + + +def redact_netloc(netloc): + # type: (str) -> str + """ + Replace the sensitive data in a netloc with "****", if it exists. + + For example: + - "user:pass@example.com" returns "user:****@example.com" + - "accesstoken@example.com" returns "****@example.com" + """ + netloc, (user, password) = split_auth_from_netloc(netloc) + if user is None: + return netloc + if password is None: + user = '****' + password = '' + else: + user = urllib_parse.quote(user) + password = ':****' + return '{user}{password}@{netloc}'.format(user=user, + password=password, + netloc=netloc) + + +def _transform_url(url, transform_netloc): + """Transform and replace netloc in a url. + + transform_netloc is a function taking the netloc and returning a + tuple. The first element of this tuple is the new netloc. The + entire tuple is returned. + + Returns a tuple containing the transformed url as item 0 and the + original tuple returned by transform_netloc as item 1. + """ + purl = urllib_parse.urlsplit(url) + netloc_tuple = transform_netloc(purl.netloc) + # stripped url + url_pieces = ( + purl.scheme, netloc_tuple[0], purl.path, purl.query, purl.fragment + ) + surl = urllib_parse.urlunsplit(url_pieces) + return surl, netloc_tuple + + +def _get_netloc(netloc): + return split_auth_from_netloc(netloc) + + +def _redact_netloc(netloc): + return (redact_netloc(netloc),) + + +def split_auth_netloc_from_url(url): + # type: (str) -> Tuple[str, str, Tuple[str, str]] + """ + Parse a url into separate netloc, auth, and url with no auth. + + Returns: (url_without_auth, netloc, (username, password)) + """ + url_without_auth, (netloc, auth) = _transform_url(url, _get_netloc) + return url_without_auth, netloc, auth + + +def remove_auth_from_url(url): + # type: (str) -> str + """Return a copy of url with 'username:password@' removed.""" + # username/pass params are passed to subversion through flags + # and are not recognized in the url. + return _transform_url(url, _get_netloc)[0] + + +def redact_auth_from_url(url): + # type: (str) -> str + """Replace the password in a given url with ****.""" + return _transform_url(url, _redact_netloc)[0] + + +class HiddenText(object): + def __init__( + self, + secret, # type: str + redacted, # type: str + ): + # type: (...) -> None + self.secret = secret + self.redacted = redacted + + def __repr__(self): + # type: (...) -> str + return ''.format(str(self)) + + def __str__(self): + # type: (...) -> str + return self.redacted + + # This is useful for testing. + def __eq__(self, other): + # type: (Any) -> bool + if type(self) != type(other): + return False + + # The string being used for redaction doesn't also have to match, + # just the raw, original string. + return (self.secret == other.secret) + + # We need to provide an explicit __ne__ implementation for Python 2. + # TODO: remove this when we drop PY2 support. + def __ne__(self, other): + # type: (Any) -> bool + return not self == other + + +def hide_value(value): + # type: (str) -> HiddenText + return HiddenText(value, redacted='****') + + +def hide_url(url): + # type: (str) -> HiddenText + redacted = redact_auth_from_url(url) + return HiddenText(url, redacted=redacted) + + +def protect_pip_from_modification_on_windows(modifying_pip): + # type: (bool) -> None + """Protection of pip.exe from modification on Windows + + On Windows, any operation modifying pip should be run as: + python -m pip ... + """ + pip_names = [ + "pip.exe", + "pip{}.exe".format(sys.version_info[0]), + "pip{}.{}.exe".format(*sys.version_info[:2]) + ] + + # See https://github.com/pypa/pip/issues/1299 for more discussion + should_show_use_python_msg = ( + modifying_pip and + WINDOWS and + os.path.basename(sys.argv[0]) in pip_names + ) + + if should_show_use_python_msg: + new_command = [ + sys.executable, "-m", "pip" + ] + sys.argv[1:] + raise CommandError( + 'To modify pip, please run the following command:\n{}' + .format(" ".join(new_command)) + ) + + +def is_console_interactive(): + # type: () -> bool + """Is this console interactive? + """ + return sys.stdin is not None and sys.stdin.isatty() + + +def hash_file(path, blocksize=1 << 20): + # type: (str, int) -> Tuple[Any, int] + """Return (hash, length) for path using hashlib.sha256() + """ + + h = hashlib.sha256() + length = 0 + with open(path, 'rb') as f: + for block in read_chunks(f, size=blocksize): + length += len(block) + h.update(block) + return h, length + + +def is_wheel_installed(): + """ + Return whether the wheel package is installed. + """ + try: + import wheel # noqa: F401 + except ImportError: + return False + + return True diff --git a/env/lib/python3.8/site-packages/pip/_internal/utils/models.py b/env/lib/python3.8/site-packages/pip/_internal/utils/models.py new file mode 100644 index 000000000..29e144115 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/utils/models.py @@ -0,0 +1,42 @@ +"""Utilities for defining models +""" +# The following comment should be removed at some point in the future. +# mypy: disallow-untyped-defs=False + +import operator + + +class KeyBasedCompareMixin(object): + """Provides comparison capabilities that is based on a key + """ + + def __init__(self, key, defining_class): + self._compare_key = key + self._defining_class = defining_class + + def __hash__(self): + return hash(self._compare_key) + + def __lt__(self, other): + return self._compare(other, operator.__lt__) + + def __le__(self, other): + return self._compare(other, operator.__le__) + + def __gt__(self, other): + return self._compare(other, operator.__gt__) + + def __ge__(self, other): + return self._compare(other, operator.__ge__) + + def __eq__(self, other): + return self._compare(other, operator.__eq__) + + def __ne__(self, other): + return self._compare(other, operator.__ne__) + + def _compare(self, other, method): + if not isinstance(other, self._defining_class): + return NotImplemented + + return method(self._compare_key, other._compare_key) diff --git a/env/lib/python3.8/site-packages/pip/_internal/utils/packaging.py b/env/lib/python3.8/site-packages/pip/_internal/utils/packaging.py new file mode 100644 index 000000000..68aa86edb --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/utils/packaging.py @@ -0,0 +1,94 @@ +from __future__ import absolute_import + +import logging +from email.parser import FeedParser + +from pip._vendor import pkg_resources +from pip._vendor.packaging import specifiers, version + +from pip._internal.exceptions import NoneMetadataError +from pip._internal.utils.misc import display_path +from pip._internal.utils.typing import MYPY_CHECK_RUNNING + +if MYPY_CHECK_RUNNING: + from typing import Optional, Tuple + from email.message import Message + from pip._vendor.pkg_resources import Distribution + + +logger = logging.getLogger(__name__) + + +def check_requires_python(requires_python, version_info): + # type: (Optional[str], Tuple[int, ...]) -> bool + """ + Check if the given Python version matches a "Requires-Python" specifier. + + :param version_info: A 3-tuple of ints representing a Python + major-minor-micro version to check (e.g. `sys.version_info[:3]`). + + :return: `True` if the given Python version satisfies the requirement. + Otherwise, return `False`. + + :raises InvalidSpecifier: If `requires_python` has an invalid format. + """ + if requires_python is None: + # The package provides no information + return True + requires_python_specifier = specifiers.SpecifierSet(requires_python) + + python_version = version.parse('.'.join(map(str, version_info))) + return python_version in requires_python_specifier + + +def get_metadata(dist): + # type: (Distribution) -> Message + """ + :raises NoneMetadataError: if the distribution reports `has_metadata()` + True but `get_metadata()` returns None. + """ + metadata_name = 'METADATA' + if (isinstance(dist, pkg_resources.DistInfoDistribution) and + dist.has_metadata(metadata_name)): + metadata = dist.get_metadata(metadata_name) + elif dist.has_metadata('PKG-INFO'): + metadata_name = 'PKG-INFO' + metadata = dist.get_metadata(metadata_name) + else: + logger.warning("No metadata found in %s", display_path(dist.location)) + metadata = '' + + if metadata is None: + raise NoneMetadataError(dist, metadata_name) + + feed_parser = FeedParser() + # The following line errors out if with a "NoneType" TypeError if + # passed metadata=None. + feed_parser.feed(metadata) + return feed_parser.close() + + +def get_requires_python(dist): + # type: (pkg_resources.Distribution) -> Optional[str] + """ + Return the "Requires-Python" metadata for a distribution, or None + if not present. + """ + pkg_info_dict = get_metadata(dist) + requires_python = pkg_info_dict.get('Requires-Python') + + if requires_python is not None: + # Convert to a str to satisfy the type checker, since requires_python + # can be a Header object. + requires_python = str(requires_python) + + return requires_python + + +def get_installer(dist): + # type: (Distribution) -> str + if dist.has_metadata('INSTALLER'): + for line in dist.get_metadata_lines('INSTALLER'): + if line.strip(): + return line.strip() + return '' diff --git a/env/lib/python3.8/site-packages/pip/_internal/utils/pkg_resources.py b/env/lib/python3.8/site-packages/pip/_internal/utils/pkg_resources.py new file mode 100644 index 000000000..0bc129acc --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/utils/pkg_resources.py @@ -0,0 +1,44 @@ +from pip._vendor.pkg_resources import yield_lines +from pip._vendor.six import ensure_str + +from pip._internal.utils.typing import MYPY_CHECK_RUNNING + +if MYPY_CHECK_RUNNING: + from typing import Dict, Iterable, List + + +class DictMetadata(object): + """IMetadataProvider that reads metadata files from a dictionary. + """ + def __init__(self, metadata): + # type: (Dict[str, bytes]) -> None + self._metadata = metadata + + def has_metadata(self, name): + # type: (str) -> bool + return name in self._metadata + + def get_metadata(self, name): + # type: (str) -> str + try: + return ensure_str(self._metadata[name]) + except UnicodeDecodeError as e: + # Mirrors handling done in pkg_resources.NullProvider. + e.reason += " in {} file".format(name) + raise + + def get_metadata_lines(self, name): + # type: (str) -> Iterable[str] + return yield_lines(self.get_metadata(name)) + + def metadata_isdir(self, name): + # type: (str) -> bool + return False + + def metadata_listdir(self, name): + # type: (str) -> List[str] + return [] + + def run_script(self, script_name, namespace): + # type: (str, str) -> None + pass diff --git a/env/lib/python3.8/site-packages/pip/_internal/utils/setuptools_build.py b/env/lib/python3.8/site-packages/pip/_internal/utils/setuptools_build.py new file mode 100644 index 000000000..4147a650d --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/utils/setuptools_build.py @@ -0,0 +1,181 @@ +import sys + +from pip._internal.utils.typing import MYPY_CHECK_RUNNING + +if MYPY_CHECK_RUNNING: + from typing import List, Optional, Sequence + +# Shim to wrap setup.py invocation with setuptools +# +# We set sys.argv[0] to the path to the underlying setup.py file so +# setuptools / distutils don't take the path to the setup.py to be "-c" when +# invoking via the shim. This avoids e.g. the following manifest_maker +# warning: "warning: manifest_maker: standard file '-c' not found". +_SETUPTOOLS_SHIM = ( + "import sys, setuptools, tokenize; sys.argv[0] = {0!r}; __file__={0!r};" + "f=getattr(tokenize, 'open', open)(__file__);" + "code=f.read().replace('\\r\\n', '\\n');" + "f.close();" + "exec(compile(code, __file__, 'exec'))" +) + + +def make_setuptools_shim_args( + setup_py_path, # type: str + global_options=None, # type: Sequence[str] + no_user_config=False, # type: bool + unbuffered_output=False # type: bool +): + # type: (...) -> List[str] + """ + Get setuptools command arguments with shim wrapped setup file invocation. + + :param setup_py_path: The path to setup.py to be wrapped. + :param global_options: Additional global options. + :param no_user_config: If True, disables personal user configuration. + :param unbuffered_output: If True, adds the unbuffered switch to the + argument list. + """ + args = [sys.executable] + if unbuffered_output: + args += ["-u"] + args += ["-c", _SETUPTOOLS_SHIM.format(setup_py_path)] + if global_options: + args += global_options + if no_user_config: + args += ["--no-user-cfg"] + return args + + +def make_setuptools_bdist_wheel_args( + setup_py_path, # type: str + global_options, # type: Sequence[str] + build_options, # type: Sequence[str] + destination_dir, # type: str +): + # type: (...) -> List[str] + # NOTE: Eventually, we'd want to also -S to the flags here, when we're + # isolating. Currently, it breaks Python in virtualenvs, because it + # relies on site.py to find parts of the standard library outside the + # virtualenv. + args = make_setuptools_shim_args( + setup_py_path, + global_options=global_options, + unbuffered_output=True + ) + args += ["bdist_wheel", "-d", destination_dir] + args += build_options + return args + + +def make_setuptools_clean_args( + setup_py_path, # type: str + global_options, # type: Sequence[str] +): + # type: (...) -> List[str] + args = make_setuptools_shim_args( + setup_py_path, + global_options=global_options, + unbuffered_output=True + ) + args += ["clean", "--all"] + return args + + +def make_setuptools_develop_args( + setup_py_path, # type: str + global_options, # type: Sequence[str] + install_options, # type: Sequence[str] + no_user_config, # type: bool + prefix, # type: Optional[str] + home, # type: Optional[str] + use_user_site, # type: bool +): + # type: (...) -> List[str] + assert not (use_user_site and prefix) + + args = make_setuptools_shim_args( + setup_py_path, + global_options=global_options, + no_user_config=no_user_config, + ) + + args += ["develop", "--no-deps"] + + args += install_options + + if prefix: + args += ["--prefix", prefix] + if home is not None: + args += ["--home", home] + + if use_user_site: + args += ["--user", "--prefix="] + + return args + + +def make_setuptools_egg_info_args( + setup_py_path, # type: str + egg_info_dir, # type: Optional[str] + no_user_config, # type: bool +): + # type: (...) -> List[str] + args = make_setuptools_shim_args(setup_py_path) + if no_user_config: + args += ["--no-user-cfg"] + + args += ["egg_info"] + + if egg_info_dir: + args += ["--egg-base", egg_info_dir] + + return args + + +def make_setuptools_install_args( + setup_py_path, # type: str + global_options, # type: Sequence[str] + install_options, # type: Sequence[str] + record_filename, # type: str + root, # type: Optional[str] + prefix, # type: Optional[str] + header_dir, # type: Optional[str] + home, # type: Optional[str] + use_user_site, # type: bool + no_user_config, # type: bool + pycompile # type: bool +): + # type: (...) -> List[str] + assert not (use_user_site and prefix) + assert not (use_user_site and root) + + args = make_setuptools_shim_args( + setup_py_path, + global_options=global_options, + no_user_config=no_user_config, + unbuffered_output=True + ) + args += ["install", "--record", record_filename] + args += ["--single-version-externally-managed"] + + if root is not None: + args += ["--root", root] + if prefix is not None: + args += ["--prefix", prefix] + if home is not None: + args += ["--home", home] + if use_user_site: + args += ["--user", "--prefix="] + + if pycompile: + args += ["--compile"] + else: + args += ["--no-compile"] + + if header_dir: + args += ["--install-headers", header_dir] + + args += install_options + + return args diff --git a/env/lib/python3.8/site-packages/pip/_internal/utils/subprocess.py b/env/lib/python3.8/site-packages/pip/_internal/utils/subprocess.py new file mode 100644 index 000000000..ea0176d34 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/utils/subprocess.py @@ -0,0 +1,278 @@ +# The following comment should be removed at some point in the future. +# mypy: strict-optional=False + +from __future__ import absolute_import + +import logging +import os +import subprocess + +from pip._vendor.six.moves import shlex_quote + +from pip._internal.exceptions import InstallationError +from pip._internal.utils.compat import console_to_str, str_to_display +from pip._internal.utils.logging import subprocess_logger +from pip._internal.utils.misc import HiddenText, path_to_display +from pip._internal.utils.typing import MYPY_CHECK_RUNNING +from pip._internal.utils.ui import open_spinner + +if MYPY_CHECK_RUNNING: + from typing import ( + Any, Callable, Iterable, List, Mapping, Optional, Text, Union, + ) + from pip._internal.utils.ui import SpinnerInterface + + CommandArgs = List[Union[str, HiddenText]] + + +LOG_DIVIDER = '----------------------------------------' + + +def make_command(*args): + # type: (Union[str, HiddenText, CommandArgs]) -> CommandArgs + """ + Create a CommandArgs object. + """ + command_args = [] # type: CommandArgs + for arg in args: + # Check for list instead of CommandArgs since CommandArgs is + # only known during type-checking. + if isinstance(arg, list): + command_args.extend(arg) + else: + # Otherwise, arg is str or HiddenText. + command_args.append(arg) + + return command_args + + +def format_command_args(args): + # type: (Union[List[str], CommandArgs]) -> str + """ + Format command arguments for display. + """ + # For HiddenText arguments, display the redacted form by calling str(). + # Also, we don't apply str() to arguments that aren't HiddenText since + # this can trigger a UnicodeDecodeError in Python 2 if the argument + # has type unicode and includes a non-ascii character. (The type + # checker doesn't ensure the annotations are correct in all cases.) + return ' '.join( + shlex_quote(str(arg)) if isinstance(arg, HiddenText) + else shlex_quote(arg) for arg in args + ) + + +def reveal_command_args(args): + # type: (Union[List[str], CommandArgs]) -> List[str] + """ + Return the arguments in their raw, unredacted form. + """ + return [ + arg.secret if isinstance(arg, HiddenText) else arg for arg in args + ] + + +def make_subprocess_output_error( + cmd_args, # type: Union[List[str], CommandArgs] + cwd, # type: Optional[str] + lines, # type: List[Text] + exit_status, # type: int +): + # type: (...) -> Text + """ + Create and return the error message to use to log a subprocess error + with command output. + + :param lines: A list of lines, each ending with a newline. + """ + command = format_command_args(cmd_args) + # Convert `command` and `cwd` to text (unicode in Python 2) so we can use + # them as arguments in the unicode format string below. This avoids + # "UnicodeDecodeError: 'ascii' codec can't decode byte ..." in Python 2 + # if either contains a non-ascii character. + command_display = str_to_display(command, desc='command bytes') + cwd_display = path_to_display(cwd) + + # We know the joined output value ends in a newline. + output = ''.join(lines) + msg = ( + # Use a unicode string to avoid "UnicodeEncodeError: 'ascii' + # codec can't encode character ..." in Python 2 when a format + # argument (e.g. `output`) has a non-ascii character. + u'Command errored out with exit status {exit_status}:\n' + ' command: {command_display}\n' + ' cwd: {cwd_display}\n' + 'Complete output ({line_count} lines):\n{output}{divider}' + ).format( + exit_status=exit_status, + command_display=command_display, + cwd_display=cwd_display, + line_count=len(lines), + output=output, + divider=LOG_DIVIDER, + ) + return msg + + +def call_subprocess( + cmd, # type: Union[List[str], CommandArgs] + show_stdout=False, # type: bool + cwd=None, # type: Optional[str] + on_returncode='raise', # type: str + extra_ok_returncodes=None, # type: Optional[Iterable[int]] + command_desc=None, # type: Optional[str] + extra_environ=None, # type: Optional[Mapping[str, Any]] + unset_environ=None, # type: Optional[Iterable[str]] + spinner=None, # type: Optional[SpinnerInterface] + log_failed_cmd=True # type: Optional[bool] +): + # type: (...) -> Text + """ + Args: + show_stdout: if true, use INFO to log the subprocess's stderr and + stdout streams. Otherwise, use DEBUG. Defaults to False. + extra_ok_returncodes: an iterable of integer return codes that are + acceptable, in addition to 0. Defaults to None, which means []. + unset_environ: an iterable of environment variable names to unset + prior to calling subprocess.Popen(). + log_failed_cmd: if false, failed commands are not logged, only raised. + """ + if extra_ok_returncodes is None: + extra_ok_returncodes = [] + if unset_environ is None: + unset_environ = [] + # Most places in pip use show_stdout=False. What this means is-- + # + # - We connect the child's output (combined stderr and stdout) to a + # single pipe, which we read. + # - We log this output to stderr at DEBUG level as it is received. + # - If DEBUG logging isn't enabled (e.g. if --verbose logging wasn't + # requested), then we show a spinner so the user can still see the + # subprocess is in progress. + # - If the subprocess exits with an error, we log the output to stderr + # at ERROR level if it hasn't already been displayed to the console + # (e.g. if --verbose logging wasn't enabled). This way we don't log + # the output to the console twice. + # + # If show_stdout=True, then the above is still done, but with DEBUG + # replaced by INFO. + if show_stdout: + # Then log the subprocess output at INFO level. + log_subprocess = subprocess_logger.info + used_level = logging.INFO + else: + # Then log the subprocess output using DEBUG. This also ensures + # it will be logged to the log file (aka user_log), if enabled. + log_subprocess = subprocess_logger.debug + used_level = logging.DEBUG + + # Whether the subprocess will be visible in the console. + showing_subprocess = subprocess_logger.getEffectiveLevel() <= used_level + + # Only use the spinner if we're not showing the subprocess output + # and we have a spinner. + use_spinner = not showing_subprocess and spinner is not None + + if command_desc is None: + command_desc = format_command_args(cmd) + + log_subprocess("Running command %s", command_desc) + env = os.environ.copy() + if extra_environ: + env.update(extra_environ) + for name in unset_environ: + env.pop(name, None) + try: + proc = subprocess.Popen( + # Convert HiddenText objects to the underlying str. + reveal_command_args(cmd), + stderr=subprocess.STDOUT, stdin=subprocess.PIPE, + stdout=subprocess.PIPE, cwd=cwd, env=env, + ) + proc.stdin.close() + except Exception as exc: + if log_failed_cmd: + subprocess_logger.critical( + "Error %s while executing command %s", exc, command_desc, + ) + raise + all_output = [] + while True: + # The "line" value is a unicode string in Python 2. + line = console_to_str(proc.stdout.readline()) + if not line: + break + line = line.rstrip() + all_output.append(line + '\n') + + # Show the line immediately. + log_subprocess(line) + # Update the spinner. + if use_spinner: + spinner.spin() + try: + proc.wait() + finally: + if proc.stdout: + proc.stdout.close() + proc_had_error = ( + proc.returncode and proc.returncode not in extra_ok_returncodes + ) + if use_spinner: + if proc_had_error: + spinner.finish("error") + else: + spinner.finish("done") + if proc_had_error: + if on_returncode == 'raise': + if not showing_subprocess and log_failed_cmd: + # Then the subprocess streams haven't been logged to the + # console yet. + msg = make_subprocess_output_error( + cmd_args=cmd, + cwd=cwd, + lines=all_output, + exit_status=proc.returncode, + ) + subprocess_logger.error(msg) + exc_msg = ( + 'Command errored out with exit status {}: {} ' + 'Check the logs for full command output.' + ).format(proc.returncode, command_desc) + raise InstallationError(exc_msg) + elif on_returncode == 'warn': + subprocess_logger.warning( + 'Command "%s" had error code %s in %s', + command_desc, proc.returncode, cwd, + ) + elif on_returncode == 'ignore': + pass + else: + raise ValueError('Invalid value: on_returncode=%s' % + repr(on_returncode)) + return ''.join(all_output) + + +def runner_with_spinner_message(message): + # type: (str) -> Callable[..., None] + """Provide a subprocess_runner that shows a spinner message. + + Intended for use with for pep517's Pep517HookCaller. Thus, the runner has + an API that matches what's expected by Pep517HookCaller.subprocess_runner. + """ + + def runner( + cmd, # type: List[str] + cwd=None, # type: Optional[str] + extra_environ=None # type: Optional[Mapping[str, Any]] + ): + # type: (...) -> None + with open_spinner(message) as spinner: + call_subprocess( + cmd, + cwd=cwd, + extra_environ=extra_environ, + spinner=spinner, + ) + + return runner diff --git a/env/lib/python3.8/site-packages/pip/_internal/utils/temp_dir.py b/env/lib/python3.8/site-packages/pip/_internal/utils/temp_dir.py new file mode 100644 index 000000000..65e41bc70 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/utils/temp_dir.py @@ -0,0 +1,250 @@ +from __future__ import absolute_import + +import errno +import itertools +import logging +import os.path +import tempfile +from contextlib import contextmanager + +from pip._vendor.contextlib2 import ExitStack + +from pip._internal.utils.misc import rmtree +from pip._internal.utils.typing import MYPY_CHECK_RUNNING + +if MYPY_CHECK_RUNNING: + from typing import Any, Dict, Iterator, Optional, TypeVar + + _T = TypeVar('_T', bound='TempDirectory') + + +logger = logging.getLogger(__name__) + + +_tempdir_manager = None # type: Optional[ExitStack] + + +@contextmanager +def global_tempdir_manager(): + # type: () -> Iterator[None] + global _tempdir_manager + with ExitStack() as stack: + old_tempdir_manager, _tempdir_manager = _tempdir_manager, stack + try: + yield + finally: + _tempdir_manager = old_tempdir_manager + + +class TempDirectoryTypeRegistry(object): + """Manages temp directory behavior + """ + + def __init__(self): + # type: () -> None + self._should_delete = {} # type: Dict[str, bool] + + def set_delete(self, kind, value): + # type: (str, bool) -> None + """Indicate whether a TempDirectory of the given kind should be + auto-deleted. + """ + self._should_delete[kind] = value + + def get_delete(self, kind): + # type: (str) -> bool + """Get configured auto-delete flag for a given TempDirectory type, + default True. + """ + return self._should_delete.get(kind, True) + + +_tempdir_registry = None # type: Optional[TempDirectoryTypeRegistry] + + +@contextmanager +def tempdir_registry(): + # type: () -> Iterator[TempDirectoryTypeRegistry] + """Provides a scoped global tempdir registry that can be used to dictate + whether directories should be deleted. + """ + global _tempdir_registry + old_tempdir_registry = _tempdir_registry + _tempdir_registry = TempDirectoryTypeRegistry() + try: + yield _tempdir_registry + finally: + _tempdir_registry = old_tempdir_registry + + +class TempDirectory(object): + """Helper class that owns and cleans up a temporary directory. + + This class can be used as a context manager or as an OO representation of a + temporary directory. + + Attributes: + path + Location to the created temporary directory + delete + Whether the directory should be deleted when exiting + (when used as a contextmanager) + + Methods: + cleanup() + Deletes the temporary directory + + When used as a context manager, if the delete attribute is True, on + exiting the context the temporary directory is deleted. + """ + + def __init__( + self, + path=None, # type: Optional[str] + delete=None, # type: Optional[bool] + kind="temp", # type: str + globally_managed=False, # type: bool + ): + super(TempDirectory, self).__init__() + + # If we were given an explicit directory, resolve delete option now. + # Otherwise we wait until cleanup and see what tempdir_registry says. + if path is not None and delete is None: + delete = False + + if path is None: + path = self._create(kind) + + self._path = path + self._deleted = False + self.delete = delete + self.kind = kind + + if globally_managed: + assert _tempdir_manager is not None + _tempdir_manager.enter_context(self) + + @property + def path(self): + # type: () -> str + assert not self._deleted, ( + "Attempted to access deleted path: {}".format(self._path) + ) + return self._path + + def __repr__(self): + # type: () -> str + return "<{} {!r}>".format(self.__class__.__name__, self.path) + + def __enter__(self): + # type: (_T) -> _T + return self + + def __exit__(self, exc, value, tb): + # type: (Any, Any, Any) -> None + if self.delete is not None: + delete = self.delete + elif _tempdir_registry: + delete = _tempdir_registry.get_delete(self.kind) + else: + delete = True + + if delete: + self.cleanup() + + def _create(self, kind): + # type: (str) -> str + """Create a temporary directory and store its path in self.path + """ + # We realpath here because some systems have their default tmpdir + # symlinked to another directory. This tends to confuse build + # scripts, so we canonicalize the path by traversing potential + # symlinks here. + path = os.path.realpath( + tempfile.mkdtemp(prefix="pip-{}-".format(kind)) + ) + logger.debug("Created temporary directory: {}".format(path)) + return path + + def cleanup(self): + # type: () -> None + """Remove the temporary directory created and reset state + """ + self._deleted = True + if os.path.exists(self._path): + rmtree(self._path) + + +class AdjacentTempDirectory(TempDirectory): + """Helper class that creates a temporary directory adjacent to a real one. + + Attributes: + original + The original directory to create a temp directory for. + path + After calling create() or entering, contains the full + path to the temporary directory. + delete + Whether the directory should be deleted when exiting + (when used as a contextmanager) + + """ + # The characters that may be used to name the temp directory + # We always prepend a ~ and then rotate through these until + # a usable name is found. + # pkg_resources raises a different error for .dist-info folder + # with leading '-' and invalid metadata + LEADING_CHARS = "-~.=%0123456789" + + def __init__(self, original, delete=None): + # type: (str, Optional[bool]) -> None + self.original = original.rstrip('/\\') + super(AdjacentTempDirectory, self).__init__(delete=delete) + + @classmethod + def _generate_names(cls, name): + # type: (str) -> Iterator[str] + """Generates a series of temporary names. + + The algorithm replaces the leading characters in the name + with ones that are valid filesystem characters, but are not + valid package names (for both Python and pip definitions of + package). + """ + for i in range(1, len(name)): + for candidate in itertools.combinations_with_replacement( + cls.LEADING_CHARS, i - 1): + new_name = '~' + ''.join(candidate) + name[i:] + if new_name != name: + yield new_name + + # If we make it this far, we will have to make a longer name + for i in range(len(cls.LEADING_CHARS)): + for candidate in itertools.combinations_with_replacement( + cls.LEADING_CHARS, i): + new_name = '~' + ''.join(candidate) + name + if new_name != name: + yield new_name + + def _create(self, kind): + # type: (str) -> str + root, name = os.path.split(self.original) + for candidate in self._generate_names(name): + path = os.path.join(root, candidate) + try: + os.mkdir(path) + except OSError as ex: + # Continue if the name exists already + if ex.errno != errno.EEXIST: + raise + else: + path = os.path.realpath(path) + break + else: + # Final fallback on the default behavior. + path = os.path.realpath( + tempfile.mkdtemp(prefix="pip-{}-".format(kind)) + ) + + logger.debug("Created temporary directory: {}".format(path)) + return path diff --git a/env/lib/python3.8/site-packages/pip/_internal/utils/typing.py b/env/lib/python3.8/site-packages/pip/_internal/utils/typing.py new file mode 100644 index 000000000..8505a29b1 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/utils/typing.py @@ -0,0 +1,38 @@ +"""For neatly implementing static typing in pip. + +`mypy` - the static type analysis tool we use - uses the `typing` module, which +provides core functionality fundamental to mypy's functioning. + +Generally, `typing` would be imported at runtime and used in that fashion - +it acts as a no-op at runtime and does not have any run-time overhead by +design. + +As it turns out, `typing` is not vendorable - it uses separate sources for +Python 2/Python 3. Thus, this codebase can not expect it to be present. +To work around this, mypy allows the typing import to be behind a False-y +optional to prevent it from running at runtime and type-comments can be used +to remove the need for the types to be accessible directly during runtime. + +This module provides the False-y guard in a nicely named fashion so that a +curious maintainer can reach here to read this. + +In pip, all static-typing related imports should be guarded as follows: + + from pip._internal.utils.typing import MYPY_CHECK_RUNNING + + if MYPY_CHECK_RUNNING: + from typing import ... + +Ref: https://github.com/python/mypy/issues/3216 +""" + +MYPY_CHECK_RUNNING = False + + +if MYPY_CHECK_RUNNING: + from typing import cast +else: + # typing's cast() is needed at runtime, but we don't want to import typing. + # Thus, we use a dummy no-op version, which we tell mypy to ignore. + def cast(type_, value): # type: ignore + return value diff --git a/env/lib/python3.8/site-packages/pip/_internal/utils/ui.py b/env/lib/python3.8/site-packages/pip/_internal/utils/ui.py new file mode 100644 index 000000000..87782aa64 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/utils/ui.py @@ -0,0 +1,428 @@ +# The following comment should be removed at some point in the future. +# mypy: strict-optional=False +# mypy: disallow-untyped-defs=False + +from __future__ import absolute_import, division + +import contextlib +import itertools +import logging +import sys +import time +from signal import SIGINT, default_int_handler, signal + +from pip._vendor import six +from pip._vendor.progress import HIDE_CURSOR, SHOW_CURSOR +from pip._vendor.progress.bar import Bar, FillingCirclesBar, IncrementalBar +from pip._vendor.progress.spinner import Spinner + +from pip._internal.utils.compat import WINDOWS +from pip._internal.utils.logging import get_indentation +from pip._internal.utils.misc import format_size +from pip._internal.utils.typing import MYPY_CHECK_RUNNING + +if MYPY_CHECK_RUNNING: + from typing import Any, Iterator, IO + +try: + from pip._vendor import colorama +# Lots of different errors can come from this, including SystemError and +# ImportError. +except Exception: + colorama = None + +logger = logging.getLogger(__name__) + + +def _select_progress_class(preferred, fallback): + encoding = getattr(preferred.file, "encoding", None) + + # If we don't know what encoding this file is in, then we'll just assume + # that it doesn't support unicode and use the ASCII bar. + if not encoding: + return fallback + + # Collect all of the possible characters we want to use with the preferred + # bar. + characters = [ + getattr(preferred, "empty_fill", six.text_type()), + getattr(preferred, "fill", six.text_type()), + ] + characters += list(getattr(preferred, "phases", [])) + + # Try to decode the characters we're using for the bar using the encoding + # of the given file, if this works then we'll assume that we can use the + # fancier bar and if not we'll fall back to the plaintext bar. + try: + six.text_type().join(characters).encode(encoding) + except UnicodeEncodeError: + return fallback + else: + return preferred + + +_BaseBar = _select_progress_class(IncrementalBar, Bar) # type: Any + + +class InterruptibleMixin(object): + """ + Helper to ensure that self.finish() gets called on keyboard interrupt. + + This allows downloads to be interrupted without leaving temporary state + (like hidden cursors) behind. + + This class is similar to the progress library's existing SigIntMixin + helper, but as of version 1.2, that helper has the following problems: + + 1. It calls sys.exit(). + 2. It discards the existing SIGINT handler completely. + 3. It leaves its own handler in place even after an uninterrupted finish, + which will have unexpected delayed effects if the user triggers an + unrelated keyboard interrupt some time after a progress-displaying + download has already completed, for example. + """ + + def __init__(self, *args, **kwargs): + """ + Save the original SIGINT handler for later. + """ + super(InterruptibleMixin, self).__init__(*args, **kwargs) + + self.original_handler = signal(SIGINT, self.handle_sigint) + + # If signal() returns None, the previous handler was not installed from + # Python, and we cannot restore it. This probably should not happen, + # but if it does, we must restore something sensible instead, at least. + # The least bad option should be Python's default SIGINT handler, which + # just raises KeyboardInterrupt. + if self.original_handler is None: + self.original_handler = default_int_handler + + def finish(self): + """ + Restore the original SIGINT handler after finishing. + + This should happen regardless of whether the progress display finishes + normally, or gets interrupted. + """ + super(InterruptibleMixin, self).finish() + signal(SIGINT, self.original_handler) + + def handle_sigint(self, signum, frame): + """ + Call self.finish() before delegating to the original SIGINT handler. + + This handler should only be in place while the progress display is + active. + """ + self.finish() + self.original_handler(signum, frame) + + +class SilentBar(Bar): + + def update(self): + pass + + +class BlueEmojiBar(IncrementalBar): + + suffix = "%(percent)d%%" + bar_prefix = " " + bar_suffix = " " + phases = (u"\U0001F539", u"\U0001F537", u"\U0001F535") # type: Any + + +class DownloadProgressMixin(object): + + def __init__(self, *args, **kwargs): + super(DownloadProgressMixin, self).__init__(*args, **kwargs) + self.message = (" " * (get_indentation() + 2)) + self.message + + @property + def downloaded(self): + return format_size(self.index) + + @property + def download_speed(self): + # Avoid zero division errors... + if self.avg == 0.0: + return "..." + return format_size(1 / self.avg) + "/s" + + @property + def pretty_eta(self): + if self.eta: + return "eta %s" % self.eta_td + return "" + + def iter(self, it): + for x in it: + yield x + self.next(len(x)) + self.finish() + + +class WindowsMixin(object): + + def __init__(self, *args, **kwargs): + # The Windows terminal does not support the hide/show cursor ANSI codes + # even with colorama. So we'll ensure that hide_cursor is False on + # Windows. + # This call needs to go before the super() call, so that hide_cursor + # is set in time. The base progress bar class writes the "hide cursor" + # code to the terminal in its init, so if we don't set this soon + # enough, we get a "hide" with no corresponding "show"... + if WINDOWS and self.hide_cursor: + self.hide_cursor = False + + super(WindowsMixin, self).__init__(*args, **kwargs) + + # Check if we are running on Windows and we have the colorama module, + # if we do then wrap our file with it. + if WINDOWS and colorama: + self.file = colorama.AnsiToWin32(self.file) + # The progress code expects to be able to call self.file.isatty() + # but the colorama.AnsiToWin32() object doesn't have that, so we'll + # add it. + self.file.isatty = lambda: self.file.wrapped.isatty() + # The progress code expects to be able to call self.file.flush() + # but the colorama.AnsiToWin32() object doesn't have that, so we'll + # add it. + self.file.flush = lambda: self.file.wrapped.flush() + + +class BaseDownloadProgressBar(WindowsMixin, InterruptibleMixin, + DownloadProgressMixin): + + file = sys.stdout + message = "%(percent)d%%" + suffix = "%(downloaded)s %(download_speed)s %(pretty_eta)s" + +# NOTE: The "type: ignore" comments on the following classes are there to +# work around https://github.com/python/typing/issues/241 + + +class DefaultDownloadProgressBar(BaseDownloadProgressBar, + _BaseBar): + pass + + +class DownloadSilentBar(BaseDownloadProgressBar, SilentBar): # type: ignore + pass + + +class DownloadBar(BaseDownloadProgressBar, # type: ignore + Bar): + pass + + +class DownloadFillingCirclesBar(BaseDownloadProgressBar, # type: ignore + FillingCirclesBar): + pass + + +class DownloadBlueEmojiProgressBar(BaseDownloadProgressBar, # type: ignore + BlueEmojiBar): + pass + + +class DownloadProgressSpinner(WindowsMixin, InterruptibleMixin, + DownloadProgressMixin, Spinner): + + file = sys.stdout + suffix = "%(downloaded)s %(download_speed)s" + + def next_phase(self): + if not hasattr(self, "_phaser"): + self._phaser = itertools.cycle(self.phases) + return next(self._phaser) + + def update(self): + message = self.message % self + phase = self.next_phase() + suffix = self.suffix % self + line = ''.join([ + message, + " " if message else "", + phase, + " " if suffix else "", + suffix, + ]) + + self.writeln(line) + + +BAR_TYPES = { + "off": (DownloadSilentBar, DownloadSilentBar), + "on": (DefaultDownloadProgressBar, DownloadProgressSpinner), + "ascii": (DownloadBar, DownloadProgressSpinner), + "pretty": (DownloadFillingCirclesBar, DownloadProgressSpinner), + "emoji": (DownloadBlueEmojiProgressBar, DownloadProgressSpinner) +} + + +def DownloadProgressProvider(progress_bar, max=None): + if max is None or max == 0: + return BAR_TYPES[progress_bar][1]().iter + else: + return BAR_TYPES[progress_bar][0](max=max).iter + + +################################################################ +# Generic "something is happening" spinners +# +# We don't even try using progress.spinner.Spinner here because it's actually +# simpler to reimplement from scratch than to coerce their code into doing +# what we need. +################################################################ + +@contextlib.contextmanager +def hidden_cursor(file): + # type: (IO[Any]) -> Iterator[None] + # The Windows terminal does not support the hide/show cursor ANSI codes, + # even via colorama. So don't even try. + if WINDOWS: + yield + # We don't want to clutter the output with control characters if we're + # writing to a file, or if the user is running with --quiet. + # See https://github.com/pypa/pip/issues/3418 + elif not file.isatty() or logger.getEffectiveLevel() > logging.INFO: + yield + else: + file.write(HIDE_CURSOR) + try: + yield + finally: + file.write(SHOW_CURSOR) + + +class RateLimiter(object): + def __init__(self, min_update_interval_seconds): + # type: (float) -> None + self._min_update_interval_seconds = min_update_interval_seconds + self._last_update = 0 # type: float + + def ready(self): + # type: () -> bool + now = time.time() + delta = now - self._last_update + return delta >= self._min_update_interval_seconds + + def reset(self): + # type: () -> None + self._last_update = time.time() + + +class SpinnerInterface(object): + def spin(self): + # type: () -> None + raise NotImplementedError() + + def finish(self, final_status): + # type: (str) -> None + raise NotImplementedError() + + +class InteractiveSpinner(SpinnerInterface): + def __init__(self, message, file=None, spin_chars="-\\|/", + # Empirically, 8 updates/second looks nice + min_update_interval_seconds=0.125): + self._message = message + if file is None: + file = sys.stdout + self._file = file + self._rate_limiter = RateLimiter(min_update_interval_seconds) + self._finished = False + + self._spin_cycle = itertools.cycle(spin_chars) + + self._file.write(" " * get_indentation() + self._message + " ... ") + self._width = 0 + + def _write(self, status): + assert not self._finished + # Erase what we wrote before by backspacing to the beginning, writing + # spaces to overwrite the old text, and then backspacing again + backup = "\b" * self._width + self._file.write(backup + " " * self._width + backup) + # Now we have a blank slate to add our status + self._file.write(status) + self._width = len(status) + self._file.flush() + self._rate_limiter.reset() + + def spin(self): + # type: () -> None + if self._finished: + return + if not self._rate_limiter.ready(): + return + self._write(next(self._spin_cycle)) + + def finish(self, final_status): + # type: (str) -> None + if self._finished: + return + self._write(final_status) + self._file.write("\n") + self._file.flush() + self._finished = True + + +# Used for dumb terminals, non-interactive installs (no tty), etc. +# We still print updates occasionally (once every 60 seconds by default) to +# act as a keep-alive for systems like Travis-CI that take lack-of-output as +# an indication that a task has frozen. +class NonInteractiveSpinner(SpinnerInterface): + def __init__(self, message, min_update_interval_seconds=60): + # type: (str, float) -> None + self._message = message + self._finished = False + self._rate_limiter = RateLimiter(min_update_interval_seconds) + self._update("started") + + def _update(self, status): + assert not self._finished + self._rate_limiter.reset() + logger.info("%s: %s", self._message, status) + + def spin(self): + # type: () -> None + if self._finished: + return + if not self._rate_limiter.ready(): + return + self._update("still running...") + + def finish(self, final_status): + # type: (str) -> None + if self._finished: + return + self._update("finished with status '%s'" % (final_status,)) + self._finished = True + + +@contextlib.contextmanager +def open_spinner(message): + # type: (str) -> Iterator[SpinnerInterface] + # Interactive spinner goes directly to sys.stdout rather than being routed + # through the logging system, but it acts like it has level INFO, + # i.e. it's only displayed if we're at level INFO or better. + # Non-interactive spinner goes through the logging system, so it is always + # in sync with logging configuration. + if sys.stdout.isatty() and logger.getEffectiveLevel() <= logging.INFO: + spinner = InteractiveSpinner(message) # type: SpinnerInterface + else: + spinner = NonInteractiveSpinner(message) + try: + with hidden_cursor(sys.stdout): + yield spinner + except KeyboardInterrupt: + spinner.finish("canceled") + raise + except Exception: + spinner.finish("error") + raise + else: + spinner.finish("done") diff --git a/env/lib/python3.8/site-packages/pip/_internal/utils/unpacking.py b/env/lib/python3.8/site-packages/pip/_internal/utils/unpacking.py new file mode 100644 index 000000000..7252dc217 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/utils/unpacking.py @@ -0,0 +1,272 @@ +"""Utilities related archives. +""" + +# The following comment should be removed at some point in the future. +# mypy: strict-optional=False +# mypy: disallow-untyped-defs=False + +from __future__ import absolute_import + +import logging +import os +import shutil +import stat +import tarfile +import zipfile + +from pip._internal.exceptions import InstallationError +from pip._internal.utils.filetypes import ( + BZ2_EXTENSIONS, + TAR_EXTENSIONS, + XZ_EXTENSIONS, + ZIP_EXTENSIONS, +) +from pip._internal.utils.misc import ensure_dir +from pip._internal.utils.typing import MYPY_CHECK_RUNNING + +if MYPY_CHECK_RUNNING: + from typing import Iterable, List, Optional, Text, Union + + +logger = logging.getLogger(__name__) + + +SUPPORTED_EXTENSIONS = ZIP_EXTENSIONS + TAR_EXTENSIONS + +try: + import bz2 # noqa + SUPPORTED_EXTENSIONS += BZ2_EXTENSIONS +except ImportError: + logger.debug('bz2 module is not available') + +try: + # Only for Python 3.3+ + import lzma # noqa + SUPPORTED_EXTENSIONS += XZ_EXTENSIONS +except ImportError: + logger.debug('lzma module is not available') + + +def current_umask(): + """Get the current umask which involves having to set it temporarily.""" + mask = os.umask(0) + os.umask(mask) + return mask + + +def split_leading_dir(path): + # type: (Union[str, Text]) -> List[Union[str, Text]] + path = path.lstrip('/').lstrip('\\') + if ( + '/' in path and ( + ('\\' in path and path.find('/') < path.find('\\')) or + '\\' not in path + ) + ): + return path.split('/', 1) + elif '\\' in path: + return path.split('\\', 1) + else: + return [path, ''] + + +def has_leading_dir(paths): + # type: (Iterable[Union[str, Text]]) -> bool + """Returns true if all the paths have the same leading path name + (i.e., everything is in one subdirectory in an archive)""" + common_prefix = None + for path in paths: + prefix, rest = split_leading_dir(path) + if not prefix: + return False + elif common_prefix is None: + common_prefix = prefix + elif prefix != common_prefix: + return False + return True + + +def is_within_directory(directory, target): + # type: ((Union[str, Text]), (Union[str, Text])) -> bool + """ + Return true if the absolute path of target is within the directory + """ + abs_directory = os.path.abspath(directory) + abs_target = os.path.abspath(target) + + prefix = os.path.commonprefix([abs_directory, abs_target]) + return prefix == abs_directory + + +def unzip_file(filename, location, flatten=True): + # type: (str, str, bool) -> None + """ + Unzip the file (with path `filename`) to the destination `location`. All + files are written based on system defaults and umask (i.e. permissions are + not preserved), except that regular file members with any execute + permissions (user, group, or world) have "chmod +x" applied after being + written. Note that for windows, any execute changes using os.chmod are + no-ops per the python docs. + """ + ensure_dir(location) + zipfp = open(filename, 'rb') + try: + zip = zipfile.ZipFile(zipfp, allowZip64=True) + leading = has_leading_dir(zip.namelist()) and flatten + for info in zip.infolist(): + name = info.filename + fn = name + if leading: + fn = split_leading_dir(name)[1] + fn = os.path.join(location, fn) + dir = os.path.dirname(fn) + if not is_within_directory(location, fn): + message = ( + 'The zip file ({}) has a file ({}) trying to install ' + 'outside target directory ({})' + ) + raise InstallationError(message.format(filename, fn, location)) + if fn.endswith('/') or fn.endswith('\\'): + # A directory + ensure_dir(fn) + else: + ensure_dir(dir) + # Don't use read() to avoid allocating an arbitrarily large + # chunk of memory for the file's content + fp = zip.open(name) + try: + with open(fn, 'wb') as destfp: + shutil.copyfileobj(fp, destfp) + finally: + fp.close() + mode = info.external_attr >> 16 + # if mode and regular file and any execute permissions for + # user/group/world? + if mode and stat.S_ISREG(mode) and mode & 0o111: + # make dest file have execute for user/group/world + # (chmod +x) no-op on windows per python docs + os.chmod(fn, (0o777 - current_umask() | 0o111)) + finally: + zipfp.close() + + +def untar_file(filename, location): + # type: (str, str) -> None + """ + Untar the file (with path `filename`) to the destination `location`. + All files are written based on system defaults and umask (i.e. permissions + are not preserved), except that regular file members with any execute + permissions (user, group, or world) have "chmod +x" applied after being + written. Note that for windows, any execute changes using os.chmod are + no-ops per the python docs. + """ + ensure_dir(location) + if filename.lower().endswith('.gz') or filename.lower().endswith('.tgz'): + mode = 'r:gz' + elif filename.lower().endswith(BZ2_EXTENSIONS): + mode = 'r:bz2' + elif filename.lower().endswith(XZ_EXTENSIONS): + mode = 'r:xz' + elif filename.lower().endswith('.tar'): + mode = 'r' + else: + logger.warning( + 'Cannot determine compression type for file %s', filename, + ) + mode = 'r:*' + tar = tarfile.open(filename, mode) + try: + leading = has_leading_dir([ + member.name for member in tar.getmembers() + ]) + for member in tar.getmembers(): + fn = member.name + if leading: + # https://github.com/python/mypy/issues/1174 + fn = split_leading_dir(fn)[1] # type: ignore + path = os.path.join(location, fn) + if not is_within_directory(location, path): + message = ( + 'The tar file ({}) has a file ({}) trying to install ' + 'outside target directory ({})' + ) + raise InstallationError( + message.format(filename, path, location) + ) + if member.isdir(): + ensure_dir(path) + elif member.issym(): + try: + # https://github.com/python/typeshed/issues/2673 + tar._extract_member(member, path) # type: ignore + except Exception as exc: + # Some corrupt tar files seem to produce this + # (specifically bad symlinks) + logger.warning( + 'In the tar file %s the member %s is invalid: %s', + filename, member.name, exc, + ) + continue + else: + try: + fp = tar.extractfile(member) + except (KeyError, AttributeError) as exc: + # Some corrupt tar files seem to produce this + # (specifically bad symlinks) + logger.warning( + 'In the tar file %s the member %s is invalid: %s', + filename, member.name, exc, + ) + continue + ensure_dir(os.path.dirname(path)) + with open(path, 'wb') as destfp: + shutil.copyfileobj(fp, destfp) + fp.close() + # Update the timestamp (useful for cython compiled files) + # https://github.com/python/typeshed/issues/2673 + tar.utime(member, path) # type: ignore + # member have any execute permissions for user/group/world? + if member.mode & 0o111: + # make dest file have execute for user/group/world + # no-op on windows per python docs + os.chmod(path, (0o777 - current_umask() | 0o111)) + finally: + tar.close() + + +def unpack_file( + filename, # type: str + location, # type: str + content_type=None, # type: Optional[str] +): + # type: (...) -> None + filename = os.path.realpath(filename) + if ( + content_type == 'application/zip' or + filename.lower().endswith(ZIP_EXTENSIONS) or + zipfile.is_zipfile(filename) + ): + unzip_file( + filename, + location, + flatten=not filename.endswith('.whl') + ) + elif ( + content_type == 'application/x-gzip' or + tarfile.is_tarfile(filename) or + filename.lower().endswith( + TAR_EXTENSIONS + BZ2_EXTENSIONS + XZ_EXTENSIONS + ) + ): + untar_file(filename, location) + else: + # FIXME: handle? + # FIXME: magic signatures? + logger.critical( + 'Cannot unpack file %s (downloaded from %s, content-type: %s); ' + 'cannot detect archive format', + filename, location, content_type, + ) + raise InstallationError( + 'Cannot determine archive format of {}'.format(location) + ) diff --git a/env/lib/python3.8/site-packages/pip/_internal/utils/urls.py b/env/lib/python3.8/site-packages/pip/_internal/utils/urls.py new file mode 100644 index 000000000..9ad40feb3 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/utils/urls.py @@ -0,0 +1,54 @@ +import os +import sys + +from pip._vendor.six.moves.urllib import parse as urllib_parse +from pip._vendor.six.moves.urllib import request as urllib_request + +from pip._internal.utils.typing import MYPY_CHECK_RUNNING + +if MYPY_CHECK_RUNNING: + from typing import Optional, Text, Union + + +def get_url_scheme(url): + # type: (Union[str, Text]) -> Optional[Text] + if ':' not in url: + return None + return url.split(':', 1)[0].lower() + + +def path_to_url(path): + # type: (Union[str, Text]) -> str + """ + Convert a path to a file: URL. The path will be made absolute and have + quoted path parts. + """ + path = os.path.normpath(os.path.abspath(path)) + url = urllib_parse.urljoin('file:', urllib_request.pathname2url(path)) + return url + + +def url_to_path(url): + # type: (str) -> str + """ + Convert a file: URL to a path. + """ + assert url.startswith('file:'), ( + "You can only turn file: urls into filenames (not %r)" % url) + + _, netloc, path, _, _ = urllib_parse.urlsplit(url) + + if not netloc or netloc == 'localhost': + # According to RFC 8089, same as empty authority. + netloc = '' + elif sys.platform == 'win32': + # If we have a UNC path, prepend UNC share notation. + netloc = '\\\\' + netloc + else: + raise ValueError( + 'non-local file URIs are not supported on this platform: %r' + % url + ) + + path = urllib_request.url2pathname(netloc + path) + return path diff --git a/env/lib/python3.8/site-packages/pip/_internal/utils/virtualenv.py b/env/lib/python3.8/site-packages/pip/_internal/utils/virtualenv.py new file mode 100644 index 000000000..d81e6ac54 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/utils/virtualenv.py @@ -0,0 +1,115 @@ +from __future__ import absolute_import + +import logging +import os +import re +import site +import sys + +from pip._internal.utils.typing import MYPY_CHECK_RUNNING + +if MYPY_CHECK_RUNNING: + from typing import List, Optional + +logger = logging.getLogger(__name__) +_INCLUDE_SYSTEM_SITE_PACKAGES_REGEX = re.compile( + r"include-system-site-packages\s*=\s*(?Ptrue|false)" +) + + +def _running_under_venv(): + # type: () -> bool + """Checks if sys.base_prefix and sys.prefix match. + + This handles PEP 405 compliant virtual environments. + """ + return sys.prefix != getattr(sys, "base_prefix", sys.prefix) + + +def _running_under_regular_virtualenv(): + # type: () -> bool + """Checks if sys.real_prefix is set. + + This handles virtual environments created with pypa's virtualenv. + """ + # pypa/virtualenv case + return hasattr(sys, 'real_prefix') + + +def running_under_virtualenv(): + # type: () -> bool + """Return True if we're running inside a virtualenv, False otherwise. + """ + return _running_under_venv() or _running_under_regular_virtualenv() + + +def _get_pyvenv_cfg_lines(): + # type: () -> Optional[List[str]] + """Reads {sys.prefix}/pyvenv.cfg and returns its contents as list of lines + + Returns None, if it could not read/access the file. + """ + pyvenv_cfg_file = os.path.join(sys.prefix, 'pyvenv.cfg') + try: + with open(pyvenv_cfg_file) as f: + return f.read().splitlines() # avoids trailing newlines + except IOError: + return None + + +def _no_global_under_venv(): + # type: () -> bool + """Check `{sys.prefix}/pyvenv.cfg` for system site-packages inclusion + + PEP 405 specifies that when system site-packages are not supposed to be + visible from a virtual environment, `pyvenv.cfg` must contain the following + line: + + include-system-site-packages = false + + Additionally, log a warning if accessing the file fails. + """ + cfg_lines = _get_pyvenv_cfg_lines() + if cfg_lines is None: + # We're not in a "sane" venv, so assume there is no system + # site-packages access (since that's PEP 405's default state). + logger.warning( + "Could not access 'pyvenv.cfg' despite a virtual environment " + "being active. Assuming global site-packages is not accessible " + "in this environment." + ) + return True + + for line in cfg_lines: + match = _INCLUDE_SYSTEM_SITE_PACKAGES_REGEX.match(line) + if match is not None and match.group('value') == 'false': + return True + return False + + +def _no_global_under_regular_virtualenv(): + # type: () -> bool + """Check if "no-global-site-packages.txt" exists beside site.py + + This mirrors logic in pypa/virtualenv for determining whether system + site-packages are visible in the virtual environment. + """ + site_mod_dir = os.path.dirname(os.path.abspath(site.__file__)) + no_global_site_packages_file = os.path.join( + site_mod_dir, 'no-global-site-packages.txt', + ) + return os.path.exists(no_global_site_packages_file) + + +def virtualenv_no_global(): + # type: () -> bool + """Returns a boolean, whether running in venv with no system site-packages. + """ + + if _running_under_regular_virtualenv(): + return _no_global_under_regular_virtualenv() + + if _running_under_venv(): + return _no_global_under_venv() + + return False diff --git a/env/lib/python3.8/site-packages/pip/_internal/utils/wheel.py b/env/lib/python3.8/site-packages/pip/_internal/utils/wheel.py new file mode 100644 index 000000000..837e0afd7 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/utils/wheel.py @@ -0,0 +1,225 @@ +"""Support functions for working with wheel files. +""" + +from __future__ import absolute_import + +import logging +from email.parser import Parser +from zipfile import ZipFile + +from pip._vendor.packaging.utils import canonicalize_name +from pip._vendor.pkg_resources import DistInfoDistribution +from pip._vendor.six import PY2, ensure_str + +from pip._internal.exceptions import UnsupportedWheel +from pip._internal.utils.pkg_resources import DictMetadata +from pip._internal.utils.typing import MYPY_CHECK_RUNNING + +if MYPY_CHECK_RUNNING: + from email.message import Message + from typing import Dict, Tuple + + from pip._vendor.pkg_resources import Distribution + +if PY2: + from zipfile import BadZipfile as BadZipFile +else: + from zipfile import BadZipFile + + +VERSION_COMPATIBLE = (1, 0) + + +logger = logging.getLogger(__name__) + + +class WheelMetadata(DictMetadata): + """Metadata provider that maps metadata decoding exceptions to our + internal exception type. + """ + def __init__(self, metadata, wheel_name): + # type: (Dict[str, bytes], str) -> None + super(WheelMetadata, self).__init__(metadata) + self._wheel_name = wheel_name + + def get_metadata(self, name): + # type: (str) -> str + try: + return super(WheelMetadata, self).get_metadata(name) + except UnicodeDecodeError as e: + # Augment the default error with the origin of the file. + raise UnsupportedWheel( + "Error decoding metadata for {}: {}".format( + self._wheel_name, e + ) + ) + + +def pkg_resources_distribution_for_wheel(wheel_zip, name, location): + # type: (ZipFile, str, str) -> Distribution + """Get a pkg_resources distribution given a wheel. + + :raises UnsupportedWheel: on any errors + """ + info_dir, _ = parse_wheel(wheel_zip, name) + + metadata_files = [ + p for p in wheel_zip.namelist() if p.startswith("{}/".format(info_dir)) + ] + + metadata_text = {} # type: Dict[str, bytes] + for path in metadata_files: + # If a flag is set, namelist entries may be unicode in Python 2. + # We coerce them to native str type to match the types used in the rest + # of the code. This cannot fail because unicode can always be encoded + # with UTF-8. + full_path = ensure_str(path) + _, metadata_name = full_path.split("/", 1) + + try: + metadata_text[metadata_name] = read_wheel_metadata_file( + wheel_zip, full_path + ) + except UnsupportedWheel as e: + raise UnsupportedWheel( + "{} has an invalid wheel, {}".format(name, str(e)) + ) + + metadata = WheelMetadata(metadata_text, location) + + return DistInfoDistribution( + location=location, metadata=metadata, project_name=name + ) + + +def parse_wheel(wheel_zip, name): + # type: (ZipFile, str) -> Tuple[str, Message] + """Extract information from the provided wheel, ensuring it meets basic + standards. + + Returns the name of the .dist-info directory and the parsed WHEEL metadata. + """ + try: + info_dir = wheel_dist_info_dir(wheel_zip, name) + metadata = wheel_metadata(wheel_zip, info_dir) + version = wheel_version(metadata) + except UnsupportedWheel as e: + raise UnsupportedWheel( + "{} has an invalid wheel, {}".format(name, str(e)) + ) + + check_compatibility(version, name) + + return info_dir, metadata + + +def wheel_dist_info_dir(source, name): + # type: (ZipFile, str) -> str + """Returns the name of the contained .dist-info directory. + + Raises AssertionError or UnsupportedWheel if not found, >1 found, or + it doesn't match the provided name. + """ + # Zip file path separators must be / + subdirs = list(set(p.split("/")[0] for p in source.namelist())) + + info_dirs = [s for s in subdirs if s.endswith('.dist-info')] + + if not info_dirs: + raise UnsupportedWheel(".dist-info directory not found") + + if len(info_dirs) > 1: + raise UnsupportedWheel( + "multiple .dist-info directories found: {}".format( + ", ".join(info_dirs) + ) + ) + + info_dir = info_dirs[0] + + info_dir_name = canonicalize_name(info_dir) + canonical_name = canonicalize_name(name) + if not info_dir_name.startswith(canonical_name): + raise UnsupportedWheel( + ".dist-info directory {!r} does not start with {!r}".format( + info_dir, canonical_name + ) + ) + + # Zip file paths can be unicode or str depending on the zip entry flags, + # so normalize it. + return ensure_str(info_dir) + + +def read_wheel_metadata_file(source, path): + # type: (ZipFile, str) -> bytes + try: + return source.read(path) + # BadZipFile for general corruption, KeyError for missing entry, + # and RuntimeError for password-protected files + except (BadZipFile, KeyError, RuntimeError) as e: + raise UnsupportedWheel( + "could not read {!r} file: {!r}".format(path, e) + ) + + +def wheel_metadata(source, dist_info_dir): + # type: (ZipFile, str) -> Message + """Return the WHEEL metadata of an extracted wheel, if possible. + Otherwise, raise UnsupportedWheel. + """ + path = "{}/WHEEL".format(dist_info_dir) + # Zip file path separators must be / + wheel_contents = read_wheel_metadata_file(source, path) + + try: + wheel_text = ensure_str(wheel_contents) + except UnicodeDecodeError as e: + raise UnsupportedWheel("error decoding {!r}: {!r}".format(path, e)) + + # FeedParser (used by Parser) does not raise any exceptions. The returned + # message may have .defects populated, but for backwards-compatibility we + # currently ignore them. + return Parser().parsestr(wheel_text) + + +def wheel_version(wheel_data): + # type: (Message) -> Tuple[int, ...] + """Given WHEEL metadata, return the parsed Wheel-Version. + Otherwise, raise UnsupportedWheel. + """ + version_text = wheel_data["Wheel-Version"] + if version_text is None: + raise UnsupportedWheel("WHEEL is missing Wheel-Version") + + version = version_text.strip() + + try: + return tuple(map(int, version.split('.'))) + except ValueError: + raise UnsupportedWheel("invalid Wheel-Version: {!r}".format(version)) + + +def check_compatibility(version, name): + # type: (Tuple[int, ...], str) -> None + """Raises errors or warns if called with an incompatible Wheel-Version. + + Pip should refuse to install a Wheel-Version that's a major series + ahead of what it's compatible with (e.g 2.0 > 1.1); and warn when + installing a version only minor version ahead (e.g 1.2 > 1.1). + + version: a 2-tuple representing a Wheel-Version (Major, Minor) + name: name of wheel or package to raise exception about + + :raises UnsupportedWheel: when an incompatible Wheel-Version is given + """ + if version[0] > VERSION_COMPATIBLE[0]: + raise UnsupportedWheel( + "%s's Wheel-Version (%s) is not compatible with this version " + "of pip" % (name, '.'.join(map(str, version))) + ) + elif version > VERSION_COMPATIBLE: + logger.warning( + 'Installing from a newer Wheel-Version (%s)', + '.'.join(map(str, version)), + ) diff --git a/env/lib/python3.8/site-packages/pip/_internal/vcs/__init__.py b/env/lib/python3.8/site-packages/pip/_internal/vcs/__init__.py new file mode 100644 index 000000000..2a4eb1375 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/vcs/__init__.py @@ -0,0 +1,15 @@ +# Expose a limited set of classes and functions so callers outside of +# the vcs package don't need to import deeper than `pip._internal.vcs`. +# (The test directory and imports protected by MYPY_CHECK_RUNNING may +# still need to import from a vcs sub-package.) +# Import all vcs modules to register each VCS in the VcsSupport object. +import pip._internal.vcs.bazaar +import pip._internal.vcs.git +import pip._internal.vcs.mercurial +import pip._internal.vcs.subversion # noqa: F401 +from pip._internal.vcs.versioncontrol import ( # noqa: F401 + RemoteNotFoundError, + is_url, + make_vcs_requirement_url, + vcs, +) diff --git a/env/lib/python3.8/site-packages/pip/_internal/vcs/bazaar.py b/env/lib/python3.8/site-packages/pip/_internal/vcs/bazaar.py new file mode 100644 index 000000000..347c06f9d --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/vcs/bazaar.py @@ -0,0 +1,120 @@ +# The following comment should be removed at some point in the future. +# mypy: disallow-untyped-defs=False + +from __future__ import absolute_import + +import logging +import os + +from pip._vendor.six.moves.urllib import parse as urllib_parse + +from pip._internal.utils.misc import display_path, rmtree +from pip._internal.utils.subprocess import make_command +from pip._internal.utils.typing import MYPY_CHECK_RUNNING +from pip._internal.utils.urls import path_to_url +from pip._internal.vcs.versioncontrol import VersionControl, vcs + +if MYPY_CHECK_RUNNING: + from typing import Optional, Tuple + from pip._internal.utils.misc import HiddenText + from pip._internal.vcs.versioncontrol import AuthInfo, RevOptions + + +logger = logging.getLogger(__name__) + + +class Bazaar(VersionControl): + name = 'bzr' + dirname = '.bzr' + repo_name = 'branch' + schemes = ( + 'bzr', 'bzr+http', 'bzr+https', 'bzr+ssh', 'bzr+sftp', 'bzr+ftp', + 'bzr+lp', + ) + + def __init__(self, *args, **kwargs): + super(Bazaar, self).__init__(*args, **kwargs) + # This is only needed for python <2.7.5 + # Register lp but do not expose as a scheme to support bzr+lp. + if getattr(urllib_parse, 'uses_fragment', None): + urllib_parse.uses_fragment.extend(['lp']) + + @staticmethod + def get_base_rev_args(rev): + return ['-r', rev] + + def export(self, location, url): + # type: (str, HiddenText) -> None + """ + Export the Bazaar repository at the url to the destination location + """ + # Remove the location to make sure Bazaar can export it correctly + if os.path.exists(location): + rmtree(location) + + url, rev_options = self.get_url_rev_options(url) + self.run_command( + make_command('export', location, url, rev_options.to_args()), + show_stdout=False, + ) + + def fetch_new(self, dest, url, rev_options): + # type: (str, HiddenText, RevOptions) -> None + rev_display = rev_options.to_display() + logger.info( + 'Checking out %s%s to %s', + url, + rev_display, + display_path(dest), + ) + cmd_args = ( + make_command('branch', '-q', rev_options.to_args(), url, dest) + ) + self.run_command(cmd_args) + + def switch(self, dest, url, rev_options): + # type: (str, HiddenText, RevOptions) -> None + self.run_command(make_command('switch', url), cwd=dest) + + def update(self, dest, url, rev_options): + # type: (str, HiddenText, RevOptions) -> None + cmd_args = make_command('pull', '-q', rev_options.to_args()) + self.run_command(cmd_args, cwd=dest) + + @classmethod + def get_url_rev_and_auth(cls, url): + # type: (str) -> Tuple[str, Optional[str], AuthInfo] + # hotfix the URL scheme after removing bzr+ from bzr+ssh:// readd it + url, rev, user_pass = super(Bazaar, cls).get_url_rev_and_auth(url) + if url.startswith('ssh://'): + url = 'bzr+' + url + return url, rev, user_pass + + @classmethod + def get_remote_url(cls, location): + urls = cls.run_command(['info'], show_stdout=False, cwd=location) + for line in urls.splitlines(): + line = line.strip() + for x in ('checkout of branch: ', + 'parent branch: '): + if line.startswith(x): + repo = line.split(x)[1] + if cls._is_local_repository(repo): + return path_to_url(repo) + return repo + return None + + @classmethod + def get_revision(cls, location): + revision = cls.run_command( + ['revno'], show_stdout=False, cwd=location, + ) + return revision.splitlines()[-1] + + @classmethod + def is_commit_id_equal(cls, dest, name): + """Always assume the versions don't match""" + return False + + +vcs.register(Bazaar) diff --git a/env/lib/python3.8/site-packages/pip/_internal/vcs/git.py b/env/lib/python3.8/site-packages/pip/_internal/vcs/git.py new file mode 100644 index 000000000..7483303a9 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/vcs/git.py @@ -0,0 +1,389 @@ +# The following comment should be removed at some point in the future. +# mypy: disallow-untyped-defs=False + +from __future__ import absolute_import + +import logging +import os.path +import re + +from pip._vendor.packaging.version import parse as parse_version +from pip._vendor.six.moves.urllib import parse as urllib_parse +from pip._vendor.six.moves.urllib import request as urllib_request + +from pip._internal.exceptions import BadCommand +from pip._internal.utils.misc import display_path, hide_url +from pip._internal.utils.subprocess import make_command +from pip._internal.utils.temp_dir import TempDirectory +from pip._internal.utils.typing import MYPY_CHECK_RUNNING +from pip._internal.vcs.versioncontrol import ( + RemoteNotFoundError, + VersionControl, + find_path_to_setup_from_repo_root, + vcs, +) + +if MYPY_CHECK_RUNNING: + from typing import Optional, Tuple + from pip._internal.utils.misc import HiddenText + from pip._internal.vcs.versioncontrol import AuthInfo, RevOptions + + +urlsplit = urllib_parse.urlsplit +urlunsplit = urllib_parse.urlunsplit + + +logger = logging.getLogger(__name__) + + +HASH_REGEX = re.compile('^[a-fA-F0-9]{40}$') + + +def looks_like_hash(sha): + return bool(HASH_REGEX.match(sha)) + + +class Git(VersionControl): + name = 'git' + dirname = '.git' + repo_name = 'clone' + schemes = ( + 'git', 'git+http', 'git+https', 'git+ssh', 'git+git', 'git+file', + ) + # Prevent the user's environment variables from interfering with pip: + # https://github.com/pypa/pip/issues/1130 + unset_environ = ('GIT_DIR', 'GIT_WORK_TREE') + default_arg_rev = 'HEAD' + + @staticmethod + def get_base_rev_args(rev): + return [rev] + + def is_immutable_rev_checkout(self, url, dest): + # type: (str, str) -> bool + _, rev_options = self.get_url_rev_options(hide_url(url)) + if not rev_options.rev: + return False + if not self.is_commit_id_equal(dest, rev_options.rev): + # the current commit is different from rev, + # which means rev was something else than a commit hash + return False + # return False in the rare case rev is both a commit hash + # and a tag or a branch; we don't want to cache in that case + # because that branch/tag could point to something else in the future + is_tag_or_branch = bool( + self.get_revision_sha(dest, rev_options.rev)[0] + ) + return not is_tag_or_branch + + def get_git_version(self): + VERSION_PFX = 'git version ' + version = self.run_command(['version'], show_stdout=False) + if version.startswith(VERSION_PFX): + version = version[len(VERSION_PFX):].split()[0] + else: + version = '' + # get first 3 positions of the git version because + # on windows it is x.y.z.windows.t, and this parses as + # LegacyVersion which always smaller than a Version. + version = '.'.join(version.split('.')[:3]) + return parse_version(version) + + @classmethod + def get_current_branch(cls, location): + """ + Return the current branch, or None if HEAD isn't at a branch + (e.g. detached HEAD). + """ + # git-symbolic-ref exits with empty stdout if "HEAD" is a detached + # HEAD rather than a symbolic ref. In addition, the -q causes the + # command to exit with status code 1 instead of 128 in this case + # and to suppress the message to stderr. + args = ['symbolic-ref', '-q', 'HEAD'] + output = cls.run_command( + args, extra_ok_returncodes=(1, ), show_stdout=False, cwd=location, + ) + ref = output.strip() + + if ref.startswith('refs/heads/'): + return ref[len('refs/heads/'):] + + return None + + def export(self, location, url): + # type: (str, HiddenText) -> None + """Export the Git repository at the url to the destination location""" + if not location.endswith('/'): + location = location + '/' + + with TempDirectory(kind="export") as temp_dir: + self.unpack(temp_dir.path, url=url) + self.run_command( + ['checkout-index', '-a', '-f', '--prefix', location], + show_stdout=False, cwd=temp_dir.path + ) + + @classmethod + def get_revision_sha(cls, dest, rev): + """ + Return (sha_or_none, is_branch), where sha_or_none is a commit hash + if the revision names a remote branch or tag, otherwise None. + + Args: + dest: the repository directory. + rev: the revision name. + """ + # Pass rev to pre-filter the list. + output = cls.run_command(['show-ref', rev], cwd=dest, + show_stdout=False, on_returncode='ignore') + refs = {} + for line in output.strip().splitlines(): + try: + sha, ref = line.split() + except ValueError: + # Include the offending line to simplify troubleshooting if + # this error ever occurs. + raise ValueError('unexpected show-ref line: {!r}'.format(line)) + + refs[ref] = sha + + branch_ref = 'refs/remotes/origin/{}'.format(rev) + tag_ref = 'refs/tags/{}'.format(rev) + + sha = refs.get(branch_ref) + if sha is not None: + return (sha, True) + + sha = refs.get(tag_ref) + + return (sha, False) + + @classmethod + def resolve_revision(cls, dest, url, rev_options): + # type: (str, HiddenText, RevOptions) -> RevOptions + """ + Resolve a revision to a new RevOptions object with the SHA1 of the + branch, tag, or ref if found. + + Args: + rev_options: a RevOptions object. + """ + rev = rev_options.arg_rev + # The arg_rev property's implementation for Git ensures that the + # rev return value is always non-None. + assert rev is not None + + sha, is_branch = cls.get_revision_sha(dest, rev) + + if sha is not None: + rev_options = rev_options.make_new(sha) + rev_options.branch_name = rev if is_branch else None + + return rev_options + + # Do not show a warning for the common case of something that has + # the form of a Git commit hash. + if not looks_like_hash(rev): + logger.warning( + "Did not find branch or tag '%s', assuming revision or ref.", + rev, + ) + + if not rev.startswith('refs/'): + return rev_options + + # If it looks like a ref, we have to fetch it explicitly. + cls.run_command( + make_command('fetch', '-q', url, rev_options.to_args()), + cwd=dest, + ) + # Change the revision to the SHA of the ref we fetched + sha = cls.get_revision(dest, rev='FETCH_HEAD') + rev_options = rev_options.make_new(sha) + + return rev_options + + @classmethod + def is_commit_id_equal(cls, dest, name): + """ + Return whether the current commit hash equals the given name. + + Args: + dest: the repository directory. + name: a string name. + """ + if not name: + # Then avoid an unnecessary subprocess call. + return False + + return cls.get_revision(dest) == name + + def fetch_new(self, dest, url, rev_options): + # type: (str, HiddenText, RevOptions) -> None + rev_display = rev_options.to_display() + logger.info('Cloning %s%s to %s', url, rev_display, display_path(dest)) + self.run_command(make_command('clone', '-q', url, dest)) + + if rev_options.rev: + # Then a specific revision was requested. + rev_options = self.resolve_revision(dest, url, rev_options) + branch_name = getattr(rev_options, 'branch_name', None) + if branch_name is None: + # Only do a checkout if the current commit id doesn't match + # the requested revision. + if not self.is_commit_id_equal(dest, rev_options.rev): + cmd_args = make_command( + 'checkout', '-q', rev_options.to_args(), + ) + self.run_command(cmd_args, cwd=dest) + elif self.get_current_branch(dest) != branch_name: + # Then a specific branch was requested, and that branch + # is not yet checked out. + track_branch = 'origin/{}'.format(branch_name) + cmd_args = [ + 'checkout', '-b', branch_name, '--track', track_branch, + ] + self.run_command(cmd_args, cwd=dest) + + #: repo may contain submodules + self.update_submodules(dest) + + def switch(self, dest, url, rev_options): + # type: (str, HiddenText, RevOptions) -> None + self.run_command( + make_command('config', 'remote.origin.url', url), + cwd=dest, + ) + cmd_args = make_command('checkout', '-q', rev_options.to_args()) + self.run_command(cmd_args, cwd=dest) + + self.update_submodules(dest) + + def update(self, dest, url, rev_options): + # type: (str, HiddenText, RevOptions) -> None + # First fetch changes from the default remote + if self.get_git_version() >= parse_version('1.9.0'): + # fetch tags in addition to everything else + self.run_command(['fetch', '-q', '--tags'], cwd=dest) + else: + self.run_command(['fetch', '-q'], cwd=dest) + # Then reset to wanted revision (maybe even origin/master) + rev_options = self.resolve_revision(dest, url, rev_options) + cmd_args = make_command('reset', '--hard', '-q', rev_options.to_args()) + self.run_command(cmd_args, cwd=dest) + #: update submodules + self.update_submodules(dest) + + @classmethod + def get_remote_url(cls, location): + """ + Return URL of the first remote encountered. + + Raises RemoteNotFoundError if the repository does not have a remote + url configured. + """ + # We need to pass 1 for extra_ok_returncodes since the command + # exits with return code 1 if there are no matching lines. + stdout = cls.run_command( + ['config', '--get-regexp', r'remote\..*\.url'], + extra_ok_returncodes=(1, ), show_stdout=False, cwd=location, + ) + remotes = stdout.splitlines() + try: + found_remote = remotes[0] + except IndexError: + raise RemoteNotFoundError + + for remote in remotes: + if remote.startswith('remote.origin.url '): + found_remote = remote + break + url = found_remote.split(' ')[1] + return url.strip() + + @classmethod + def get_revision(cls, location, rev=None): + if rev is None: + rev = 'HEAD' + current_rev = cls.run_command( + ['rev-parse', rev], show_stdout=False, cwd=location, + ) + return current_rev.strip() + + @classmethod + def get_subdirectory(cls, location): + """ + Return the path to setup.py, relative to the repo root. + Return None if setup.py is in the repo root. + """ + # find the repo root + git_dir = cls.run_command( + ['rev-parse', '--git-dir'], + show_stdout=False, cwd=location).strip() + if not os.path.isabs(git_dir): + git_dir = os.path.join(location, git_dir) + repo_root = os.path.abspath(os.path.join(git_dir, '..')) + return find_path_to_setup_from_repo_root(location, repo_root) + + @classmethod + def get_url_rev_and_auth(cls, url): + # type: (str) -> Tuple[str, Optional[str], AuthInfo] + """ + Prefixes stub URLs like 'user@hostname:user/repo.git' with 'ssh://'. + That's required because although they use SSH they sometimes don't + work with a ssh:// scheme (e.g. GitHub). But we need a scheme for + parsing. Hence we remove it again afterwards and return it as a stub. + """ + # Works around an apparent Git bug + # (see https://article.gmane.org/gmane.comp.version-control.git/146500) + scheme, netloc, path, query, fragment = urlsplit(url) + if scheme.endswith('file'): + initial_slashes = path[:-len(path.lstrip('/'))] + newpath = ( + initial_slashes + + urllib_request.url2pathname(path) + .replace('\\', '/').lstrip('/') + ) + url = urlunsplit((scheme, netloc, newpath, query, fragment)) + after_plus = scheme.find('+') + 1 + url = scheme[:after_plus] + urlunsplit( + (scheme[after_plus:], netloc, newpath, query, fragment), + ) + + if '://' not in url: + assert 'file:' not in url + url = url.replace('git+', 'git+ssh://') + url, rev, user_pass = super(Git, cls).get_url_rev_and_auth(url) + url = url.replace('ssh://', '') + else: + url, rev, user_pass = super(Git, cls).get_url_rev_and_auth(url) + + return url, rev, user_pass + + @classmethod + def update_submodules(cls, location): + if not os.path.exists(os.path.join(location, '.gitmodules')): + return + cls.run_command( + ['submodule', 'update', '--init', '--recursive', '-q'], + cwd=location, + ) + + @classmethod + def controls_location(cls, location): + if super(Git, cls).controls_location(location): + return True + try: + r = cls.run_command(['rev-parse'], + cwd=location, + show_stdout=False, + on_returncode='ignore', + log_failed_cmd=False) + return not r + except BadCommand: + logger.debug("could not determine if %s is under git control " + "because git is not available", location) + return False + + +vcs.register(Git) diff --git a/env/lib/python3.8/site-packages/pip/_internal/vcs/mercurial.py b/env/lib/python3.8/site-packages/pip/_internal/vcs/mercurial.py new file mode 100644 index 000000000..d9b58cfe9 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/vcs/mercurial.py @@ -0,0 +1,155 @@ +# The following comment should be removed at some point in the future. +# mypy: disallow-untyped-defs=False + +from __future__ import absolute_import + +import logging +import os + +from pip._vendor.six.moves import configparser + +from pip._internal.exceptions import BadCommand, InstallationError +from pip._internal.utils.misc import display_path +from pip._internal.utils.subprocess import make_command +from pip._internal.utils.temp_dir import TempDirectory +from pip._internal.utils.typing import MYPY_CHECK_RUNNING +from pip._internal.utils.urls import path_to_url +from pip._internal.vcs.versioncontrol import ( + VersionControl, + find_path_to_setup_from_repo_root, + vcs, +) + +if MYPY_CHECK_RUNNING: + from pip._internal.utils.misc import HiddenText + from pip._internal.vcs.versioncontrol import RevOptions + + +logger = logging.getLogger(__name__) + + +class Mercurial(VersionControl): + name = 'hg' + dirname = '.hg' + repo_name = 'clone' + schemes = ( + 'hg', 'hg+file', 'hg+http', 'hg+https', 'hg+ssh', 'hg+static-http', + ) + + @staticmethod + def get_base_rev_args(rev): + return [rev] + + def export(self, location, url): + # type: (str, HiddenText) -> None + """Export the Hg repository at the url to the destination location""" + with TempDirectory(kind="export") as temp_dir: + self.unpack(temp_dir.path, url=url) + + self.run_command( + ['archive', location], show_stdout=False, cwd=temp_dir.path + ) + + def fetch_new(self, dest, url, rev_options): + # type: (str, HiddenText, RevOptions) -> None + rev_display = rev_options.to_display() + logger.info( + 'Cloning hg %s%s to %s', + url, + rev_display, + display_path(dest), + ) + self.run_command(make_command('clone', '--noupdate', '-q', url, dest)) + self.run_command( + make_command('update', '-q', rev_options.to_args()), + cwd=dest, + ) + + def switch(self, dest, url, rev_options): + # type: (str, HiddenText, RevOptions) -> None + repo_config = os.path.join(dest, self.dirname, 'hgrc') + config = configparser.RawConfigParser() + try: + config.read(repo_config) + config.set('paths', 'default', url.secret) + with open(repo_config, 'w') as config_file: + config.write(config_file) + except (OSError, configparser.NoSectionError) as exc: + logger.warning( + 'Could not switch Mercurial repository to %s: %s', url, exc, + ) + else: + cmd_args = make_command('update', '-q', rev_options.to_args()) + self.run_command(cmd_args, cwd=dest) + + def update(self, dest, url, rev_options): + # type: (str, HiddenText, RevOptions) -> None + self.run_command(['pull', '-q'], cwd=dest) + cmd_args = make_command('update', '-q', rev_options.to_args()) + self.run_command(cmd_args, cwd=dest) + + @classmethod + def get_remote_url(cls, location): + url = cls.run_command( + ['showconfig', 'paths.default'], + show_stdout=False, cwd=location).strip() + if cls._is_local_repository(url): + url = path_to_url(url) + return url.strip() + + @classmethod + def get_revision(cls, location): + """ + Return the repository-local changeset revision number, as an integer. + """ + current_revision = cls.run_command( + ['parents', '--template={rev}'], + show_stdout=False, cwd=location).strip() + return current_revision + + @classmethod + def get_requirement_revision(cls, location): + """ + Return the changeset identification hash, as a 40-character + hexadecimal string + """ + current_rev_hash = cls.run_command( + ['parents', '--template={node}'], + show_stdout=False, cwd=location).strip() + return current_rev_hash + + @classmethod + def is_commit_id_equal(cls, dest, name): + """Always assume the versions don't match""" + return False + + @classmethod + def get_subdirectory(cls, location): + """ + Return the path to setup.py, relative to the repo root. + Return None if setup.py is in the repo root. + """ + # find the repo root + repo_root = cls.run_command( + ['root'], show_stdout=False, cwd=location).strip() + if not os.path.isabs(repo_root): + repo_root = os.path.abspath(os.path.join(location, repo_root)) + return find_path_to_setup_from_repo_root(location, repo_root) + + @classmethod + def controls_location(cls, location): + if super(Mercurial, cls).controls_location(location): + return True + try: + cls.run_command( + ['identify'], + cwd=location, + show_stdout=False, + on_returncode='raise', + log_failed_cmd=False) + return True + except (BadCommand, InstallationError): + return False + + +vcs.register(Mercurial) diff --git a/env/lib/python3.8/site-packages/pip/_internal/vcs/subversion.py b/env/lib/python3.8/site-packages/pip/_internal/vcs/subversion.py new file mode 100644 index 000000000..6c76d1ad4 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/vcs/subversion.py @@ -0,0 +1,333 @@ +# The following comment should be removed at some point in the future. +# mypy: disallow-untyped-defs=False + +from __future__ import absolute_import + +import logging +import os +import re + +from pip._internal.utils.logging import indent_log +from pip._internal.utils.misc import ( + display_path, + is_console_interactive, + rmtree, + split_auth_from_netloc, +) +from pip._internal.utils.subprocess import make_command +from pip._internal.utils.typing import MYPY_CHECK_RUNNING +from pip._internal.vcs.versioncontrol import VersionControl, vcs + +_svn_xml_url_re = re.compile('url="([^"]+)"') +_svn_rev_re = re.compile(r'committed-rev="(\d+)"') +_svn_info_xml_rev_re = re.compile(r'\s*revision="(\d+)"') +_svn_info_xml_url_re = re.compile(r'(.*)') + + +if MYPY_CHECK_RUNNING: + from typing import Optional, Tuple + from pip._internal.utils.subprocess import CommandArgs + from pip._internal.utils.misc import HiddenText + from pip._internal.vcs.versioncontrol import AuthInfo, RevOptions + + +logger = logging.getLogger(__name__) + + +class Subversion(VersionControl): + name = 'svn' + dirname = '.svn' + repo_name = 'checkout' + schemes = ('svn', 'svn+ssh', 'svn+http', 'svn+https', 'svn+svn') + + @classmethod + def should_add_vcs_url_prefix(cls, remote_url): + return True + + @staticmethod + def get_base_rev_args(rev): + return ['-r', rev] + + @classmethod + def get_revision(cls, location): + """ + Return the maximum revision for all files under a given location + """ + # Note: taken from setuptools.command.egg_info + revision = 0 + + for base, dirs, files in os.walk(location): + if cls.dirname not in dirs: + dirs[:] = [] + continue # no sense walking uncontrolled subdirs + dirs.remove(cls.dirname) + entries_fn = os.path.join(base, cls.dirname, 'entries') + if not os.path.exists(entries_fn): + # FIXME: should we warn? + continue + + dirurl, localrev = cls._get_svn_url_rev(base) + + if base == location: + base = dirurl + '/' # save the root url + elif not dirurl or not dirurl.startswith(base): + dirs[:] = [] + continue # not part of the same svn tree, skip it + revision = max(revision, localrev) + return revision + + @classmethod + def get_netloc_and_auth(cls, netloc, scheme): + """ + This override allows the auth information to be passed to svn via the + --username and --password options instead of via the URL. + """ + if scheme == 'ssh': + # The --username and --password options can't be used for + # svn+ssh URLs, so keep the auth information in the URL. + return super(Subversion, cls).get_netloc_and_auth(netloc, scheme) + + return split_auth_from_netloc(netloc) + + @classmethod + def get_url_rev_and_auth(cls, url): + # type: (str) -> Tuple[str, Optional[str], AuthInfo] + # hotfix the URL scheme after removing svn+ from svn+ssh:// readd it + url, rev, user_pass = super(Subversion, cls).get_url_rev_and_auth(url) + if url.startswith('ssh://'): + url = 'svn+' + url + return url, rev, user_pass + + @staticmethod + def make_rev_args(username, password): + # type: (Optional[str], Optional[HiddenText]) -> CommandArgs + extra_args = [] # type: CommandArgs + if username: + extra_args += ['--username', username] + if password: + extra_args += ['--password', password] + + return extra_args + + @classmethod + def get_remote_url(cls, location): + # In cases where the source is in a subdirectory, not alongside + # setup.py we have to look up in the location until we find a real + # setup.py + orig_location = location + while not os.path.exists(os.path.join(location, 'setup.py')): + last_location = location + location = os.path.dirname(location) + if location == last_location: + # We've traversed up to the root of the filesystem without + # finding setup.py + logger.warning( + "Could not find setup.py for directory %s (tried all " + "parent directories)", + orig_location, + ) + return None + + return cls._get_svn_url_rev(location)[0] + + @classmethod + def _get_svn_url_rev(cls, location): + from pip._internal.exceptions import InstallationError + + entries_path = os.path.join(location, cls.dirname, 'entries') + if os.path.exists(entries_path): + with open(entries_path) as f: + data = f.read() + else: # subversion >= 1.7 does not have the 'entries' file + data = '' + + if (data.startswith('8') or + data.startswith('9') or + data.startswith('10')): + data = list(map(str.splitlines, data.split('\n\x0c\n'))) + del data[0][0] # get rid of the '8' + url = data[0][3] + revs = [int(d[9]) for d in data if len(d) > 9 and d[9]] + [0] + elif data.startswith('= 1.7 + # Note that using get_remote_call_options is not necessary here + # because `svn info` is being run against a local directory. + # We don't need to worry about making sure interactive mode + # is being used to prompt for passwords, because passwords + # are only potentially needed for remote server requests. + xml = cls.run_command( + ['info', '--xml', location], + show_stdout=False, + ) + url = _svn_info_xml_url_re.search(xml).group(1) + revs = [ + int(m.group(1)) for m in _svn_info_xml_rev_re.finditer(xml) + ] + except InstallationError: + url, revs = None, [] + + if revs: + rev = max(revs) + else: + rev = 0 + + return url, rev + + @classmethod + def is_commit_id_equal(cls, dest, name): + """Always assume the versions don't match""" + return False + + def __init__(self, use_interactive=None): + # type: (bool) -> None + if use_interactive is None: + use_interactive = is_console_interactive() + self.use_interactive = use_interactive + + # This member is used to cache the fetched version of the current + # ``svn`` client. + # Special value definitions: + # None: Not evaluated yet. + # Empty tuple: Could not parse version. + self._vcs_version = None # type: Optional[Tuple[int, ...]] + + super(Subversion, self).__init__() + + def call_vcs_version(self): + # type: () -> Tuple[int, ...] + """Query the version of the currently installed Subversion client. + + :return: A tuple containing the parts of the version information or + ``()`` if the version returned from ``svn`` could not be parsed. + :raises: BadCommand: If ``svn`` is not installed. + """ + # Example versions: + # svn, version 1.10.3 (r1842928) + # compiled Feb 25 2019, 14:20:39 on x86_64-apple-darwin17.0.0 + # svn, version 1.7.14 (r1542130) + # compiled Mar 28 2018, 08:49:13 on x86_64-pc-linux-gnu + version_prefix = 'svn, version ' + version = self.run_command(['--version'], show_stdout=False) + if not version.startswith(version_prefix): + return () + + version = version[len(version_prefix):].split()[0] + version_list = version.split('.') + try: + parsed_version = tuple(map(int, version_list)) + except ValueError: + return () + + return parsed_version + + def get_vcs_version(self): + # type: () -> Tuple[int, ...] + """Return the version of the currently installed Subversion client. + + If the version of the Subversion client has already been queried, + a cached value will be used. + + :return: A tuple containing the parts of the version information or + ``()`` if the version returned from ``svn`` could not be parsed. + :raises: BadCommand: If ``svn`` is not installed. + """ + if self._vcs_version is not None: + # Use cached version, if available. + # If parsing the version failed previously (empty tuple), + # do not attempt to parse it again. + return self._vcs_version + + vcs_version = self.call_vcs_version() + self._vcs_version = vcs_version + return vcs_version + + def get_remote_call_options(self): + # type: () -> CommandArgs + """Return options to be used on calls to Subversion that contact the server. + + These options are applicable for the following ``svn`` subcommands used + in this class. + + - checkout + - export + - switch + - update + + :return: A list of command line arguments to pass to ``svn``. + """ + if not self.use_interactive: + # --non-interactive switch is available since Subversion 0.14.4. + # Subversion < 1.8 runs in interactive mode by default. + return ['--non-interactive'] + + svn_version = self.get_vcs_version() + # By default, Subversion >= 1.8 runs in non-interactive mode if + # stdin is not a TTY. Since that is how pip invokes SVN, in + # call_subprocess(), pip must pass --force-interactive to ensure + # the user can be prompted for a password, if required. + # SVN added the --force-interactive option in SVN 1.8. Since + # e.g. RHEL/CentOS 7, which is supported until 2024, ships with + # SVN 1.7, pip should continue to support SVN 1.7. Therefore, pip + # can't safely add the option if the SVN version is < 1.8 (or unknown). + if svn_version >= (1, 8): + return ['--force-interactive'] + + return [] + + def export(self, location, url): + # type: (str, HiddenText) -> None + """Export the svn repository at the url to the destination location""" + url, rev_options = self.get_url_rev_options(url) + + logger.info('Exporting svn repository %s to %s', url, location) + with indent_log(): + if os.path.exists(location): + # Subversion doesn't like to check out over an existing + # directory --force fixes this, but was only added in svn 1.5 + rmtree(location) + cmd_args = make_command( + 'export', self.get_remote_call_options(), + rev_options.to_args(), url, location, + ) + self.run_command(cmd_args, show_stdout=False) + + def fetch_new(self, dest, url, rev_options): + # type: (str, HiddenText, RevOptions) -> None + rev_display = rev_options.to_display() + logger.info( + 'Checking out %s%s to %s', + url, + rev_display, + display_path(dest), + ) + cmd_args = make_command( + 'checkout', '-q', self.get_remote_call_options(), + rev_options.to_args(), url, dest, + ) + self.run_command(cmd_args) + + def switch(self, dest, url, rev_options): + # type: (str, HiddenText, RevOptions) -> None + cmd_args = make_command( + 'switch', self.get_remote_call_options(), rev_options.to_args(), + url, dest, + ) + self.run_command(cmd_args) + + def update(self, dest, url, rev_options): + # type: (str, HiddenText, RevOptions) -> None + cmd_args = make_command( + 'update', self.get_remote_call_options(), rev_options.to_args(), + dest, + ) + self.run_command(cmd_args) + + +vcs.register(Subversion) diff --git a/env/lib/python3.8/site-packages/pip/_internal/vcs/versioncontrol.py b/env/lib/python3.8/site-packages/pip/_internal/vcs/versioncontrol.py new file mode 100644 index 000000000..7cfd56882 --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/vcs/versioncontrol.py @@ -0,0 +1,700 @@ +"""Handles all VCS (version control) support""" + +from __future__ import absolute_import + +import errno +import logging +import os +import shutil +import sys + +from pip._vendor import pkg_resources +from pip._vendor.six.moves.urllib import parse as urllib_parse + +from pip._internal.exceptions import BadCommand +from pip._internal.utils.compat import samefile +from pip._internal.utils.misc import ( + ask_path_exists, + backup_dir, + display_path, + hide_url, + hide_value, + rmtree, +) +from pip._internal.utils.subprocess import call_subprocess, make_command +from pip._internal.utils.typing import MYPY_CHECK_RUNNING +from pip._internal.utils.urls import get_url_scheme + +if MYPY_CHECK_RUNNING: + from typing import ( + Any, Dict, Iterable, Iterator, List, Mapping, Optional, Text, Tuple, + Type, Union + ) + from pip._internal.utils.ui import SpinnerInterface + from pip._internal.utils.misc import HiddenText + from pip._internal.utils.subprocess import CommandArgs + + AuthInfo = Tuple[Optional[str], Optional[str]] + + +__all__ = ['vcs'] + + +logger = logging.getLogger(__name__) + + +def is_url(name): + # type: (Union[str, Text]) -> bool + """ + Return true if the name looks like a URL. + """ + scheme = get_url_scheme(name) + if scheme is None: + return False + return scheme in ['http', 'https', 'file', 'ftp'] + vcs.all_schemes + + +def make_vcs_requirement_url(repo_url, rev, project_name, subdir=None): + # type: (str, str, str, Optional[str]) -> str + """ + Return the URL for a VCS requirement. + + Args: + repo_url: the remote VCS url, with any needed VCS prefix (e.g. "git+"). + project_name: the (unescaped) project name. + """ + egg_project_name = pkg_resources.to_filename(project_name) + req = '{}@{}#egg={}'.format(repo_url, rev, egg_project_name) + if subdir: + req += '&subdirectory={}'.format(subdir) + + return req + + +def find_path_to_setup_from_repo_root(location, repo_root): + # type: (str, str) -> Optional[str] + """ + Find the path to `setup.py` by searching up the filesystem from `location`. + Return the path to `setup.py` relative to `repo_root`. + Return None if `setup.py` is in `repo_root` or cannot be found. + """ + # find setup.py + orig_location = location + while not os.path.exists(os.path.join(location, 'setup.py')): + last_location = location + location = os.path.dirname(location) + if location == last_location: + # We've traversed up to the root of the filesystem without + # finding setup.py + logger.warning( + "Could not find setup.py for directory %s (tried all " + "parent directories)", + orig_location, + ) + return None + + if samefile(repo_root, location): + return None + + return os.path.relpath(location, repo_root) + + +class RemoteNotFoundError(Exception): + pass + + +class RevOptions(object): + + """ + Encapsulates a VCS-specific revision to install, along with any VCS + install options. + + Instances of this class should be treated as if immutable. + """ + + def __init__( + self, + vc_class, # type: Type[VersionControl] + rev=None, # type: Optional[str] + extra_args=None, # type: Optional[CommandArgs] + ): + # type: (...) -> None + """ + Args: + vc_class: a VersionControl subclass. + rev: the name of the revision to install. + extra_args: a list of extra options. + """ + if extra_args is None: + extra_args = [] + + self.extra_args = extra_args + self.rev = rev + self.vc_class = vc_class + self.branch_name = None # type: Optional[str] + + def __repr__(self): + # type: () -> str + return ''.format(self.vc_class.name, self.rev) + + @property + def arg_rev(self): + # type: () -> Optional[str] + if self.rev is None: + return self.vc_class.default_arg_rev + + return self.rev + + def to_args(self): + # type: () -> CommandArgs + """ + Return the VCS-specific command arguments. + """ + args = [] # type: CommandArgs + rev = self.arg_rev + if rev is not None: + args += self.vc_class.get_base_rev_args(rev) + args += self.extra_args + + return args + + def to_display(self): + # type: () -> str + if not self.rev: + return '' + + return ' (to revision {})'.format(self.rev) + + def make_new(self, rev): + # type: (str) -> RevOptions + """ + Make a copy of the current instance, but with a new rev. + + Args: + rev: the name of the revision for the new object. + """ + return self.vc_class.make_rev_options(rev, extra_args=self.extra_args) + + +class VcsSupport(object): + _registry = {} # type: Dict[str, VersionControl] + schemes = ['ssh', 'git', 'hg', 'bzr', 'sftp', 'svn'] + + def __init__(self): + # type: () -> None + # Register more schemes with urlparse for various version control + # systems + urllib_parse.uses_netloc.extend(self.schemes) + # Python >= 2.7.4, 3.3 doesn't have uses_fragment + if getattr(urllib_parse, 'uses_fragment', None): + urllib_parse.uses_fragment.extend(self.schemes) + super(VcsSupport, self).__init__() + + def __iter__(self): + # type: () -> Iterator[str] + return self._registry.__iter__() + + @property + def backends(self): + # type: () -> List[VersionControl] + return list(self._registry.values()) + + @property + def dirnames(self): + # type: () -> List[str] + return [backend.dirname for backend in self.backends] + + @property + def all_schemes(self): + # type: () -> List[str] + schemes = [] # type: List[str] + for backend in self.backends: + schemes.extend(backend.schemes) + return schemes + + def register(self, cls): + # type: (Type[VersionControl]) -> None + if not hasattr(cls, 'name'): + logger.warning('Cannot register VCS %s', cls.__name__) + return + if cls.name not in self._registry: + self._registry[cls.name] = cls() + logger.debug('Registered VCS backend: %s', cls.name) + + def unregister(self, name): + # type: (str) -> None + if name in self._registry: + del self._registry[name] + + def get_backend_for_dir(self, location): + # type: (str) -> Optional[VersionControl] + """ + Return a VersionControl object if a repository of that type is found + at the given directory. + """ + for vcs_backend in self._registry.values(): + if vcs_backend.controls_location(location): + logger.debug('Determine that %s uses VCS: %s', + location, vcs_backend.name) + return vcs_backend + return None + + def get_backend_for_scheme(self, scheme): + # type: (str) -> Optional[VersionControl] + """ + Return a VersionControl object or None. + """ + for vcs_backend in self._registry.values(): + if scheme in vcs_backend.schemes: + return vcs_backend + return None + + def get_backend(self, name): + # type: (str) -> Optional[VersionControl] + """ + Return a VersionControl object or None. + """ + name = name.lower() + return self._registry.get(name) + + +vcs = VcsSupport() + + +class VersionControl(object): + name = '' + dirname = '' + repo_name = '' + # List of supported schemes for this Version Control + schemes = () # type: Tuple[str, ...] + # Iterable of environment variable names to pass to call_subprocess(). + unset_environ = () # type: Tuple[str, ...] + default_arg_rev = None # type: Optional[str] + + @classmethod + def should_add_vcs_url_prefix(cls, remote_url): + # type: (str) -> bool + """ + Return whether the vcs prefix (e.g. "git+") should be added to a + repository's remote url when used in a requirement. + """ + return not remote_url.lower().startswith('{}:'.format(cls.name)) + + @classmethod + def get_subdirectory(cls, location): + # type: (str) -> Optional[str] + """ + Return the path to setup.py, relative to the repo root. + Return None if setup.py is in the repo root. + """ + return None + + @classmethod + def get_requirement_revision(cls, repo_dir): + # type: (str) -> str + """ + Return the revision string that should be used in a requirement. + """ + return cls.get_revision(repo_dir) + + @classmethod + def get_src_requirement(cls, repo_dir, project_name): + # type: (str, str) -> Optional[str] + """ + Return the requirement string to use to redownload the files + currently at the given repository directory. + + Args: + project_name: the (unescaped) project name. + + The return value has a form similar to the following: + + {repository_url}@{revision}#egg={project_name} + """ + repo_url = cls.get_remote_url(repo_dir) + if repo_url is None: + return None + + if cls.should_add_vcs_url_prefix(repo_url): + repo_url = '{}+{}'.format(cls.name, repo_url) + + revision = cls.get_requirement_revision(repo_dir) + subdir = cls.get_subdirectory(repo_dir) + req = make_vcs_requirement_url(repo_url, revision, project_name, + subdir=subdir) + + return req + + @staticmethod + def get_base_rev_args(rev): + # type: (str) -> List[str] + """ + Return the base revision arguments for a vcs command. + + Args: + rev: the name of a revision to install. Cannot be None. + """ + raise NotImplementedError + + def is_immutable_rev_checkout(self, url, dest): + # type: (str, str) -> bool + """ + Return true if the commit hash checked out at dest matches + the revision in url. + + Always return False, if the VCS does not support immutable commit + hashes. + + This method does not check if there are local uncommitted changes + in dest after checkout, as pip currently has no use case for that. + """ + return False + + @classmethod + def make_rev_options(cls, rev=None, extra_args=None): + # type: (Optional[str], Optional[CommandArgs]) -> RevOptions + """ + Return a RevOptions object. + + Args: + rev: the name of a revision to install. + extra_args: a list of extra options. + """ + return RevOptions(cls, rev, extra_args=extra_args) + + @classmethod + def _is_local_repository(cls, repo): + # type: (str) -> bool + """ + posix absolute paths start with os.path.sep, + win32 ones start with drive (like c:\\folder) + """ + drive, tail = os.path.splitdrive(repo) + return repo.startswith(os.path.sep) or bool(drive) + + def export(self, location, url): + # type: (str, HiddenText) -> None + """ + Export the repository at the url to the destination location + i.e. only download the files, without vcs informations + + :param url: the repository URL starting with a vcs prefix. + """ + raise NotImplementedError + + @classmethod + def get_netloc_and_auth(cls, netloc, scheme): + # type: (str, str) -> Tuple[str, Tuple[Optional[str], Optional[str]]] + """ + Parse the repository URL's netloc, and return the new netloc to use + along with auth information. + + Args: + netloc: the original repository URL netloc. + scheme: the repository URL's scheme without the vcs prefix. + + This is mainly for the Subversion class to override, so that auth + information can be provided via the --username and --password options + instead of through the URL. For other subclasses like Git without + such an option, auth information must stay in the URL. + + Returns: (netloc, (username, password)). + """ + return netloc, (None, None) + + @classmethod + def get_url_rev_and_auth(cls, url): + # type: (str) -> Tuple[str, Optional[str], AuthInfo] + """ + Parse the repository URL to use, and return the URL, revision, + and auth info to use. + + Returns: (url, rev, (username, password)). + """ + scheme, netloc, path, query, frag = urllib_parse.urlsplit(url) + if '+' not in scheme: + raise ValueError( + "Sorry, {!r} is a malformed VCS url. " + "The format is +://, " + "e.g. svn+http://myrepo/svn/MyApp#egg=MyApp".format(url) + ) + # Remove the vcs prefix. + scheme = scheme.split('+', 1)[1] + netloc, user_pass = cls.get_netloc_and_auth(netloc, scheme) + rev = None + if '@' in path: + path, rev = path.rsplit('@', 1) + url = urllib_parse.urlunsplit((scheme, netloc, path, query, '')) + return url, rev, user_pass + + @staticmethod + def make_rev_args(username, password): + # type: (Optional[str], Optional[HiddenText]) -> CommandArgs + """ + Return the RevOptions "extra arguments" to use in obtain(). + """ + return [] + + def get_url_rev_options(self, url): + # type: (HiddenText) -> Tuple[HiddenText, RevOptions] + """ + Return the URL and RevOptions object to use in obtain() and in + some cases export(), as a tuple (url, rev_options). + """ + secret_url, rev, user_pass = self.get_url_rev_and_auth(url.secret) + username, secret_password = user_pass + password = None # type: Optional[HiddenText] + if secret_password is not None: + password = hide_value(secret_password) + extra_args = self.make_rev_args(username, password) + rev_options = self.make_rev_options(rev, extra_args=extra_args) + + return hide_url(secret_url), rev_options + + @staticmethod + def normalize_url(url): + # type: (str) -> str + """ + Normalize a URL for comparison by unquoting it and removing any + trailing slash. + """ + return urllib_parse.unquote(url).rstrip('/') + + @classmethod + def compare_urls(cls, url1, url2): + # type: (str, str) -> bool + """ + Compare two repo URLs for identity, ignoring incidental differences. + """ + return (cls.normalize_url(url1) == cls.normalize_url(url2)) + + def fetch_new(self, dest, url, rev_options): + # type: (str, HiddenText, RevOptions) -> None + """ + Fetch a revision from a repository, in the case that this is the + first fetch from the repository. + + Args: + dest: the directory to fetch the repository to. + rev_options: a RevOptions object. + """ + raise NotImplementedError + + def switch(self, dest, url, rev_options): + # type: (str, HiddenText, RevOptions) -> None + """ + Switch the repo at ``dest`` to point to ``URL``. + + Args: + rev_options: a RevOptions object. + """ + raise NotImplementedError + + def update(self, dest, url, rev_options): + # type: (str, HiddenText, RevOptions) -> None + """ + Update an already-existing repo to the given ``rev_options``. + + Args: + rev_options: a RevOptions object. + """ + raise NotImplementedError + + @classmethod + def is_commit_id_equal(cls, dest, name): + # type: (str, Optional[str]) -> bool + """ + Return whether the id of the current commit equals the given name. + + Args: + dest: the repository directory. + name: a string name. + """ + raise NotImplementedError + + def obtain(self, dest, url): + # type: (str, HiddenText) -> None + """ + Install or update in editable mode the package represented by this + VersionControl object. + + :param dest: the repository directory in which to install or update. + :param url: the repository URL starting with a vcs prefix. + """ + url, rev_options = self.get_url_rev_options(url) + + if not os.path.exists(dest): + self.fetch_new(dest, url, rev_options) + return + + rev_display = rev_options.to_display() + if self.is_repository_directory(dest): + existing_url = self.get_remote_url(dest) + if self.compare_urls(existing_url, url.secret): + logger.debug( + '%s in %s exists, and has correct URL (%s)', + self.repo_name.title(), + display_path(dest), + url, + ) + if not self.is_commit_id_equal(dest, rev_options.rev): + logger.info( + 'Updating %s %s%s', + display_path(dest), + self.repo_name, + rev_display, + ) + self.update(dest, url, rev_options) + else: + logger.info('Skipping because already up-to-date.') + return + + logger.warning( + '%s %s in %s exists with URL %s', + self.name, + self.repo_name, + display_path(dest), + existing_url, + ) + prompt = ('(s)witch, (i)gnore, (w)ipe, (b)ackup ', + ('s', 'i', 'w', 'b')) + else: + logger.warning( + 'Directory %s already exists, and is not a %s %s.', + dest, + self.name, + self.repo_name, + ) + # https://github.com/python/mypy/issues/1174 + prompt = ('(i)gnore, (w)ipe, (b)ackup ', # type: ignore + ('i', 'w', 'b')) + + logger.warning( + 'The plan is to install the %s repository %s', + self.name, + url, + ) + response = ask_path_exists('What to do? %s' % prompt[0], prompt[1]) + + if response == 'a': + sys.exit(-1) + + if response == 'w': + logger.warning('Deleting %s', display_path(dest)) + rmtree(dest) + self.fetch_new(dest, url, rev_options) + return + + if response == 'b': + dest_dir = backup_dir(dest) + logger.warning( + 'Backing up %s to %s', display_path(dest), dest_dir, + ) + shutil.move(dest, dest_dir) + self.fetch_new(dest, url, rev_options) + return + + # Do nothing if the response is "i". + if response == 's': + logger.info( + 'Switching %s %s to %s%s', + self.repo_name, + display_path(dest), + url, + rev_display, + ) + self.switch(dest, url, rev_options) + + def unpack(self, location, url): + # type: (str, HiddenText) -> None + """ + Clean up current location and download the url repository + (and vcs infos) into location + + :param url: the repository URL starting with a vcs prefix. + """ + if os.path.exists(location): + rmtree(location) + self.obtain(location, url=url) + + @classmethod + def get_remote_url(cls, location): + # type: (str) -> str + """ + Return the url used at location + + Raises RemoteNotFoundError if the repository does not have a remote + url configured. + """ + raise NotImplementedError + + @classmethod + def get_revision(cls, location): + # type: (str) -> str + """ + Return the current commit id of the files at the given location. + """ + raise NotImplementedError + + @classmethod + def run_command( + cls, + cmd, # type: Union[List[str], CommandArgs] + show_stdout=True, # type: bool + cwd=None, # type: Optional[str] + on_returncode='raise', # type: str + extra_ok_returncodes=None, # type: Optional[Iterable[int]] + command_desc=None, # type: Optional[str] + extra_environ=None, # type: Optional[Mapping[str, Any]] + spinner=None, # type: Optional[SpinnerInterface] + log_failed_cmd=True # type: bool + ): + # type: (...) -> Text + """ + Run a VCS subcommand + This is simply a wrapper around call_subprocess that adds the VCS + command name, and checks that the VCS is available + """ + cmd = make_command(cls.name, *cmd) + try: + return call_subprocess(cmd, show_stdout, cwd, + on_returncode=on_returncode, + extra_ok_returncodes=extra_ok_returncodes, + command_desc=command_desc, + extra_environ=extra_environ, + unset_environ=cls.unset_environ, + spinner=spinner, + log_failed_cmd=log_failed_cmd) + except OSError as e: + # errno.ENOENT = no such file or directory + # In other words, the VCS executable isn't available + if e.errno == errno.ENOENT: + raise BadCommand( + 'Cannot find command %r - do you have ' + '%r installed and in your ' + 'PATH?' % (cls.name, cls.name)) + else: + raise # re-raise exception if a different error occurred + + @classmethod + def is_repository_directory(cls, path): + # type: (str) -> bool + """ + Return whether a directory path is a repository directory. + """ + logger.debug('Checking in %s for %s (%s)...', + path, cls.dirname, cls.name) + return os.path.exists(os.path.join(path, cls.dirname)) + + @classmethod + def controls_location(cls, location): + # type: (str) -> bool + """ + Check if a location is controlled by the vcs. + It is meant to be overridden to implement smarter detection + mechanisms for specific vcs. + + This can do more than is_repository_directory() alone. For example, + the Git override checks that Git is actually available. + """ + return cls.is_repository_directory(location) diff --git a/env/lib/python3.8/site-packages/pip/_internal/wheel_builder.py b/env/lib/python3.8/site-packages/pip/_internal/wheel_builder.py new file mode 100644 index 000000000..7c7820d4f --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_internal/wheel_builder.py @@ -0,0 +1,305 @@ +"""Orchestrator for building wheels from InstallRequirements. +""" + +# The following comment should be removed at some point in the future. +# mypy: strict-optional=False + +import logging +import os.path +import re +import shutil + +from pip._internal.models.link import Link +from pip._internal.operations.build.wheel import build_wheel_pep517 +from pip._internal.operations.build.wheel_legacy import build_wheel_legacy +from pip._internal.utils.logging import indent_log +from pip._internal.utils.misc import ensure_dir, hash_file, is_wheel_installed +from pip._internal.utils.setuptools_build import make_setuptools_clean_args +from pip._internal.utils.subprocess import call_subprocess +from pip._internal.utils.temp_dir import TempDirectory +from pip._internal.utils.typing import MYPY_CHECK_RUNNING +from pip._internal.utils.urls import path_to_url +from pip._internal.vcs import vcs + +if MYPY_CHECK_RUNNING: + from typing import ( + Any, Callable, Iterable, List, Optional, Pattern, Tuple, + ) + + from pip._internal.cache import WheelCache + from pip._internal.req.req_install import InstallRequirement + + BinaryAllowedPredicate = Callable[[InstallRequirement], bool] + BuildResult = Tuple[List[InstallRequirement], List[InstallRequirement]] + +logger = logging.getLogger(__name__) + + +def _contains_egg_info( + s, _egg_info_re=re.compile(r'([a-z0-9_.]+)-([a-z0-9_.!+-]+)', re.I)): + # type: (str, Pattern[str]) -> bool + """Determine whether the string looks like an egg_info. + + :param s: The string to parse. E.g. foo-2.1 + """ + return bool(_egg_info_re.search(s)) + + +def _should_build( + req, # type: InstallRequirement + need_wheel, # type: bool + check_binary_allowed, # type: BinaryAllowedPredicate +): + # type: (...) -> bool + """Return whether an InstallRequirement should be built into a wheel.""" + if req.constraint: + # never build requirements that are merely constraints + return False + if req.is_wheel: + if need_wheel: + logger.info( + 'Skipping %s, due to already being wheel.', req.name, + ) + return False + + if need_wheel: + # i.e. pip wheel, not pip install + return True + + # From this point, this concerns the pip install command only + # (need_wheel=False). + + if not req.use_pep517 and not is_wheel_installed(): + # we don't build legacy requirements if wheel is not installed + return False + + if req.editable or not req.source_dir: + return False + + if not check_binary_allowed(req): + logger.info( + "Skipping wheel build for %s, due to binaries " + "being disabled for it.", req.name, + ) + return False + + return True + + +def should_build_for_wheel_command( + req, # type: InstallRequirement +): + # type: (...) -> bool + return _should_build( + req, need_wheel=True, check_binary_allowed=_always_true + ) + + +def should_build_for_install_command( + req, # type: InstallRequirement + check_binary_allowed, # type: BinaryAllowedPredicate +): + # type: (...) -> bool + return _should_build( + req, need_wheel=False, check_binary_allowed=check_binary_allowed + ) + + +def _should_cache( + req, # type: InstallRequirement +): + # type: (...) -> Optional[bool] + """ + Return whether a built InstallRequirement can be stored in the persistent + wheel cache, assuming the wheel cache is available, and _should_build() + has determined a wheel needs to be built. + """ + if not should_build_for_install_command( + req, check_binary_allowed=_always_true + ): + # never cache if pip install would not have built + # (editable mode, etc) + return False + + if req.link and req.link.is_vcs: + # VCS checkout. Do not cache + # unless it points to an immutable commit hash. + assert not req.editable + assert req.source_dir + vcs_backend = vcs.get_backend_for_scheme(req.link.scheme) + assert vcs_backend + if vcs_backend.is_immutable_rev_checkout(req.link.url, req.source_dir): + return True + return False + + base, ext = req.link.splitext() + if _contains_egg_info(base): + return True + + # Otherwise, do not cache. + return False + + +def _get_cache_dir( + req, # type: InstallRequirement + wheel_cache, # type: WheelCache +): + # type: (...) -> str + """Return the persistent or temporary cache directory where the built + wheel need to be stored. + """ + cache_available = bool(wheel_cache.cache_dir) + if cache_available and _should_cache(req): + cache_dir = wheel_cache.get_path_for_link(req.link) + else: + cache_dir = wheel_cache.get_ephem_path_for_link(req.link) + return cache_dir + + +def _always_true(_): + # type: (Any) -> bool + return True + + +def _build_one( + req, # type: InstallRequirement + output_dir, # type: str + build_options, # type: List[str] + global_options, # type: List[str] +): + # type: (...) -> Optional[str] + """Build one wheel. + + :return: The filename of the built wheel, or None if the build failed. + """ + try: + ensure_dir(output_dir) + except OSError as e: + logger.warning( + "Building wheel for %s failed: %s", + req.name, e, + ) + return None + + # Install build deps into temporary directory (PEP 518) + with req.build_env: + return _build_one_inside_env( + req, output_dir, build_options, global_options + ) + + +def _build_one_inside_env( + req, # type: InstallRequirement + output_dir, # type: str + build_options, # type: List[str] + global_options, # type: List[str] +): + # type: (...) -> Optional[str] + with TempDirectory(kind="wheel") as temp_dir: + if req.use_pep517: + wheel_path = build_wheel_pep517( + name=req.name, + backend=req.pep517_backend, + metadata_directory=req.metadata_directory, + build_options=build_options, + tempd=temp_dir.path, + ) + else: + wheel_path = build_wheel_legacy( + name=req.name, + setup_py_path=req.setup_py_path, + source_dir=req.unpacked_source_directory, + global_options=global_options, + build_options=build_options, + tempd=temp_dir.path, + ) + + if wheel_path is not None: + wheel_name = os.path.basename(wheel_path) + dest_path = os.path.join(output_dir, wheel_name) + try: + wheel_hash, length = hash_file(wheel_path) + shutil.move(wheel_path, dest_path) + logger.info('Created wheel for %s: ' + 'filename=%s size=%d sha256=%s', + req.name, wheel_name, length, + wheel_hash.hexdigest()) + logger.info('Stored in directory: %s', output_dir) + return dest_path + except Exception as e: + logger.warning( + "Building wheel for %s failed: %s", + req.name, e, + ) + # Ignore return, we can't do anything else useful. + if not req.use_pep517: + _clean_one_legacy(req, global_options) + return None + + +def _clean_one_legacy(req, global_options): + # type: (InstallRequirement, List[str]) -> bool + clean_args = make_setuptools_clean_args( + req.setup_py_path, + global_options=global_options, + ) + + logger.info('Running setup.py clean for %s', req.name) + try: + call_subprocess(clean_args, cwd=req.source_dir) + return True + except Exception: + logger.error('Failed cleaning build dir for %s', req.name) + return False + + +def build( + requirements, # type: Iterable[InstallRequirement] + wheel_cache, # type: WheelCache + build_options, # type: List[str] + global_options, # type: List[str] +): + # type: (...) -> BuildResult + """Build wheels. + + :return: The list of InstallRequirement that succeeded to build and + the list of InstallRequirement that failed to build. + """ + if not requirements: + return [], [] + + # Build the wheels. + logger.info( + 'Building wheels for collected packages: %s', + ', '.join(req.name for req in requirements), + ) + + with indent_log(): + build_successes, build_failures = [], [] + for req in requirements: + cache_dir = _get_cache_dir(req, wheel_cache) + wheel_file = _build_one( + req, cache_dir, build_options, global_options + ) + if wheel_file: + # Update the link for this. + req.link = Link(path_to_url(wheel_file)) + req.local_file_path = req.link.file_path + assert req.link.is_wheel + build_successes.append(req) + else: + build_failures.append(req) + + # notify success/failure + if build_successes: + logger.info( + 'Successfully built %s', + ' '.join([req.name for req in build_successes]), + ) + if build_failures: + logger.info( + 'Failed to build %s', + ' '.join([req.name for req in build_failures]), + ) + # Return a list of requirements that failed to build + return build_successes, build_failures diff --git a/env/lib/python3.8/site-packages/pip/_vendor/__init__.py b/env/lib/python3.8/site-packages/pip/_vendor/__init__.py new file mode 100644 index 000000000..f8b7ae6fb --- /dev/null +++ b/env/lib/python3.8/site-packages/pip/_vendor/__init__.py @@ -0,0 +1,117 @@ +""" +pip._vendor is for vendoring dependencies of pip to prevent needing pip to +depend on something external. + +Files inside of pip._vendor should be considered immutable and should only be +updated to versions from upstream. +""" +from __future__ import absolute_import + +import glob +import os.path +import sys + +# Downstream redistributors which have debundled our dependencies should also +# patch this value to be true. This will trigger the additional patching +# to cause things like "six" to be available as pip. +DEBUNDLED = True + +# By default, look in this directory for a bunch of .whl files which we will +# add to the beginning of sys.path before attempting to import anything. This +# is done to support downstream re-distributors like Debian and Fedora who +# wish to create their own Wheels for our dependencies to aid in debundling. +prefix = sys.prefix +if sys.prefix.startswith('/usr/lib/pypy'): + prefix = '/usr' +WHEEL_DIR = os.path.abspath(os.path.join(prefix, 'share', 'python-wheels')) + + +# Define a small helper function to alias our vendored modules to the real ones +# if the vendored ones do not exist. This idea of this was taken from +# https://github.com/kennethreitz/requests/pull/2567. +def vendored(modulename): + vendored_name = "{0}.{1}".format(__name__, modulename) + + try: + __import__(modulename, globals(), locals(), level=0) + except ImportError: + # We can just silently allow import failures to pass here. If we + # got to this point it means that ``import pip._vendor.whatever`` + # failed and so did ``import whatever``. Since we're importing this + # upfront in an attempt to alias imports, not erroring here will + # just mean we get a regular import error whenever pip *actually* + # tries to import one of these modules to use it, which actually + # gives us a better error message than we would have otherwise + # gotten. + pass + else: + sys.modules[vendored_name] = sys.modules[modulename] + base, head = vendored_name.rsplit(".", 1) + setattr(sys.modules[base], head, sys.modules[modulename]) + + +# If we're operating in a debundled setup, then we want to go ahead and trigger +# the aliasing of our vendored libraries as well as looking for wheels to add +# to our sys.path. This will cause all of this code to be a no-op typically +# however downstream redistributors can enable it in a consistent way across +# all platforms. +if DEBUNDLED: + # Actually look inside of WHEEL_DIR to find .whl files and add them to the + # front of our sys.path. + sys.path[:] = glob.glob(os.path.join(WHEEL_DIR, "*.whl")) + sys.path + + # Actually alias all of our vendored dependencies. + vendored("appdirs") + vendored("cachecontrol") + vendored("colorama") + vendored("contextlib2") + vendored("distlib") + vendored("distro") + vendored("html5lib") + vendored("six") + vendored("six.moves") + vendored("six.moves.urllib") + vendored("six.moves.urllib.parse") + vendored("packaging") + vendored("packaging.version") + vendored("packaging.specifiers") + vendored("pep517") + vendored("pkg_resources") + vendored("progress") + vendored("pytoml") + vendored("retrying") + vendored("requests") + vendored("requests.exceptions") + vendored("requests.packages") + vendored("requests.packages.urllib3") + vendored("requests.packages.urllib3._collections") + vendored("requests.packages.urllib3.connection") + vendored("requests.packages.urllib3.connectionpool") + vendored("requests.packages.urllib3.contrib") + vendored("requests.packages.urllib3.contrib.ntlmpool") + vendored("requests.packages.urllib3.contrib.pyopenssl") + vendored("requests.packages.urllib3.exceptions") + vendored("requests.packages.urllib3.fields") + vendored("requests.packages.urllib3.filepost") + vendored("requests.packages.urllib3.packages") + try: + vendored("requests.packages.urllib3.packages.ordered_dict") + vendored("requests.packages.urllib3.packages.six") + except ImportError: + # Debian already unbundles these from requests. + pass + vendored("requests.packages.urllib3.packages.ssl_match_hostname") + vendored("requests.packages.urllib3.packages.ssl_match_hostname." + "_implementation") + vendored("requests.packages.urllib3.poolmanager") + vendored("requests.packages.urllib3.request") + vendored("requests.packages.urllib3.response") + vendored("requests.packages.urllib3.util") + vendored("requests.packages.urllib3.util.connection") + vendored("requests.packages.urllib3.util.request") + vendored("requests.packages.urllib3.util.response") + vendored("requests.packages.urllib3.util.retry") + vendored("requests.packages.urllib3.util.ssl_") + vendored("requests.packages.urllib3.util.timeout") + vendored("requests.packages.urllib3.util.url") + vendored("urllib3") diff --git a/env/lib/python3.8/site-packages/pkg_resources-0.0.0.dist-info/AUTHORS.txt b/env/lib/python3.8/site-packages/pkg_resources-0.0.0.dist-info/AUTHORS.txt new file mode 100644 index 000000000..72c87d7d3 --- /dev/null +++ b/env/lib/python3.8/site-packages/pkg_resources-0.0.0.dist-info/AUTHORS.txt @@ -0,0 +1,562 @@ +A_Rog +Aakanksha Agrawal <11389424+rasponic@users.noreply.github.com> +Abhinav Sagar <40603139+abhinavsagar@users.noreply.github.com> +ABHYUDAY PRATAP SINGH +abs51295 +AceGentile +Adam Chainz +Adam Tse +Adam Tse +Adam Wentz +admin +Adrien Morison +ahayrapetyan +Ahilya +AinsworthK +Akash Srivastava +Alan Yee +Albert Tugushev +Albert-Guan +albertg +Aleks Bunin +Alethea Flowers +Alex Gaynor +Alex Grönholm +Alex Loosley +Alex Morega +Alex Stachowiak +Alexander Shtyrov +Alexandre Conrad +Alexey Popravka +Alexey Popravka +Alli +Ami Fischman +Ananya Maiti +Anatoly Techtonik +Anders Kaseorg +Andreas Lutro +Andrei Geacar +Andrew Gaul +Andrey Bulgakov +Andrés Delfino <34587441+andresdelfino@users.noreply.github.com> +Andrés Delfino +Andy Freeland +Andy Freeland +Andy Kluger +Ani Hayrapetyan +Aniruddha Basak +Anish Tambe +Anrs Hu +Anthony Sottile +Antoine Musso +Anton Ovchinnikov +Anton Patrushev +Antonio Alvarado Hernandez +Antony Lee +Antti Kaihola +Anubhav Patel +Anuj Godase +AQNOUCH Mohammed +AraHaan +Arindam Choudhury +Armin Ronacher +Artem +Ashley Manton +Ashwin Ramaswami +atse +Atsushi Odagiri +Avner Cohen +Baptiste Mispelon +Barney Gale +barneygale +Bartek Ogryczak +Bastian Venthur +Ben Darnell +Ben Hoyt +Ben Rosser +Bence Nagy +Benjamin Peterson +Benjamin VanEvery +Benoit Pierre +Berker Peksag +Bernardo B. Marques +Bernhard M. Wiedemann +Bertil Hatt +Bogdan Opanchuk +BorisZZZ +Brad Erickson +Bradley Ayers +Brandon L. Reiss +Brandt Bucher +Brett Randall +Brian Cristante <33549821+brcrista@users.noreply.github.com> +Brian Cristante +Brian Rosner +BrownTruck +Bruno Oliveira +Bruno Renié +Bstrdsmkr +Buck Golemon +burrows +Bussonnier Matthias +c22 +Caleb Martinez +Calvin Smith +Carl Meyer +Carlos Liam +Carol Willing +Carter Thayer +Cass +Chandrasekhar Atina +Chih-Hsuan Yen +Chih-Hsuan Yen +Chris Brinker +Chris Hunt +Chris Jerdonek +Chris McDonough +Chris Wolfe +Christian Heimes +Christian Oudard +Christopher Hunt +Christopher Snyder +Clark Boylan +Clay McClure +Cody +Cody Soyland +Colin Watson +Connor Osborn +Cooper Lees +Cooper Ry Lees +Cory Benfield +Cory Wright +Craig Kerstiens +Cristian Sorinel +Curtis Doty +cytolentino +Damian Quiroga +Dan Black +Dan Savilonis +Dan Sully +daniel +Daniel Collins +Daniel Hahler +Daniel Holth +Daniel Jost +Daniel Shaulov +Daniele Esposti +Daniele Procida +Danny Hermes +Dav Clark +Dave Abrahams +Dave Jones +David Aguilar +David Black +David Bordeynik +David Bordeynik +David Caro +David Evans +David Linke +David Pursehouse +David Tucker +David Wales +Davidovich +derwolfe +Desetude +Diego Caraballo +DiegoCaraballo +Dmitry Gladkov +Domen Kožar +Donald Stufft +Dongweiming +Douglas Thor +DrFeathers +Dustin Ingram +Dwayne Bailey +Ed Morley <501702+edmorley@users.noreply.github.com> +Ed Morley +Eitan Adler +ekristina +elainechan +Eli Schwartz +Eli Schwartz +Emil Burzo +Emil Styrke +Endoh Takanao +enoch +Erdinc Mutlu +Eric Gillingham +Eric Hanchrow +Eric Hopper +Erik M. Bray +Erik Rose +Ernest W Durbin III +Ernest W. Durbin III +Erwin Janssen +Eugene Vereshchagin +everdimension +Felix Yan +fiber-space +Filip Kokosiński +Florian Briand +Florian Rathgeber +Francesco +Francesco Montesano +Frost Ming +Gabriel Curio +Gabriel de Perthuis +Garry Polley +gdanielson +Geoffrey Lehée +Geoffrey Sneddon +George Song +Georgi Valkov +Giftlin Rajaiah +gizmoguy1 +gkdoc <40815324+gkdoc@users.noreply.github.com> +Gopinath M <31352222+mgopi1990@users.noreply.github.com> +GOTO Hayato <3532528+gh640@users.noreply.github.com> +gpiks +Guilherme Espada +Guy Rozendorn +gzpan123 +Hanjun Kim +Hari Charan +Harsh Vardhan +Herbert Pfennig +Hsiaoming Yang +Hugo +Hugo Lopes Tavares +Hugo van Kemenade +hugovk +Hynek Schlawack +Ian Bicking +Ian Cordasco +Ian Lee +Ian Stapleton Cordasco +Ian Wienand +Ian Wienand +Igor Kuzmitshov +Igor Sobreira +Ilya Baryshev +INADA Naoki +Ionel Cristian Mărieș +Ionel Maries Cristian +Ivan Pozdeev +Jacob Kim +jakirkham +Jakub Stasiak +Jakub Vysoky +Jakub Wilk +James Cleveland +James Cleveland +James Firth +James Polley +Jan Pokorný +Jannis Leidel +jarondl +Jason R. Coombs +Jay Graves +Jean-Christophe Fillion-Robin +Jeff Barber +Jeff Dairiki +Jelmer Vernooij +jenix21 +Jeremy Stanley +Jeremy Zafran +Jiashuo Li +Jim Garrison +Jivan Amara +John Paton +John-Scott Atlakson +johnthagen +johnthagen +Jon Banafato +Jon Dufresne +Jon Parise +Jonas Nockert +Jonathan Herbert +Joost Molenaar +Jorge Niedbalski +Joseph Long +Josh Bronson +Josh Hansen +Josh Schneier +Juanjo Bazán +Julian Berman +Julian Gethmann +Julien Demoor +jwg4 +Jyrki Pulliainen +Kai Chen +Kamal Bin Mustafa +kaustav haldar +keanemind +Keith Maxwell +Kelsey Hightower +Kenneth Belitzky +Kenneth Reitz +Kenneth Reitz +Kevin Burke +Kevin Carter +Kevin Frommelt +Kevin R Patterson +Kexuan Sun +Kit Randel +kpinc +Krishna Oza +Kumar McMillan +Kyle Persohn +lakshmanaram +Laszlo Kiss-Kollar +Laurent Bristiel +Laurie Opperman +Leon Sasson +Lev Givon +Lincoln de Sousa +Lipis +Loren Carvalho +Lucas Cimon +Ludovic Gasc +Luke Macken +Luo Jiebin +luojiebin +luz.paz +László Kiss Kollár +László Kiss Kollár +Marc Abramowitz +Marc Tamlyn +Marcus Smith +Mariatta +Mark Kohler +Mark Williams +Mark Williams +Markus Hametner +Masaki +Masklinn +Matej Stuchlik +Mathew Jennings +Mathieu Bridon +Matt Good +Matt Maker +Matt Robenolt +matthew +Matthew Einhorn +Matthew Gilliard +Matthew Iversen +Matthew Trumbell +Matthew Willson +Matthias Bussonnier +mattip +Maxim Kurnikov +Maxime Rouyrre +mayeut +mbaluna <44498973+mbaluna@users.noreply.github.com> +mdebi <17590103+mdebi@users.noreply.github.com> +memoselyk +Michael +Michael Aquilina +Michael E. Karpeles +Michael Klich +Michael Williamson +michaelpacer +Mickaël Schoentgen +Miguel Araujo Perez +Mihir Singh +Mike +Mike Hendricks +Min RK +MinRK +Miro Hrončok +Monica Baluna +montefra +Monty Taylor +Nate Coraor +Nathaniel J. Smith +Nehal J Wani +Neil Botelho +Nick Coghlan +Nick Stenning +Nick Timkovich +Nicolas Bock +Nikhil Benesch +Nitesh Sharma +Nowell Strite +NtaleGrey +nvdv +Ofekmeister +ofrinevo +Oliver Jeeves +Oliver Tonnhofer +Olivier Girardot +Olivier Grisel +Ollie Rutherfurd +OMOTO Kenji +Omry Yadan +Oren Held +Oscar Benjamin +Oz N Tiram +Pachwenko <32424503+Pachwenko@users.noreply.github.com> +Patrick Dubroy +Patrick Jenkins +Patrick Lawson +patricktokeeffe +Patrik Kopkan +Paul Kehrer +Paul Moore +Paul Nasrat +Paul Oswald +Paul van der Linden +Paulus Schoutsen +Pavithra Eswaramoorthy <33131404+QueenCoffee@users.noreply.github.com> +Pawel Jasinski +Pekka Klärck +Peter Lisák +Peter Waller +petr-tik +Phaneendra Chiruvella +Phil Freo +Phil Pennock +Phil Whelan +Philip Jägenstedt +Philip Molloy +Philippe Ombredanne +Pi Delport +Pierre-Yves Rofes +pip +Prabakaran Kumaresshan +Prabhjyotsing Surjit Singh Sodhi +Prabhu Marappan +Pradyun Gedam +Pratik Mallya +Preet Thakkar +Preston Holmes +Przemek Wrzos +Pulkit Goyal <7895pulkit@gmail.com> +Qiangning Hong +Quentin Pradet +R. David Murray +Rafael Caricio +Ralf Schmitt +Razzi Abuissa +rdb +Remi Rampin +Remi Rampin +Rene Dudfield +Riccardo Magliocchetti +Richard Jones +RobberPhex +Robert Collins +Robert McGibbon +Robert T. McGibbon +robin elisha robinson +Roey Berman +Rohan Jain +Rohan Jain +Rohan Jain +Roman Bogorodskiy +Romuald Brunet +Ronny Pfannschmidt +Rory McCann +Ross Brattain +Roy Wellington Ⅳ +Roy Wellington Ⅳ +Ryan Wooden +ryneeverett +Sachi King +Salvatore Rinchiera +Savio Jomton +schlamar +Scott Kitterman +Sean +seanj +Sebastian Jordan +Sebastian Schaetz +Segev Finer +SeongSoo Cho +Sergey Vasilyev +Seth Woodworth +Shlomi Fish +Shovan Maity +Simeon Visser +Simon Cross +Simon Pichugin +sinoroc +Sorin Sbarnea +Stavros Korokithakis +Stefan Scherfke +Stephan Erb +stepshal +Steve (Gadget) Barnes +Steve Barnes +Steve Dower +Steve Kowalik +Steven Myint +stonebig +Stéphane Bidoul (ACSONE) +Stéphane Bidoul +Stéphane Klein +Sumana Harihareswara +Sviatoslav Sydorenko +Sviatoslav Sydorenko +Swat009 +Takayuki SHIMIZUKAWA +tbeswick +Thijs Triemstra +Thomas Fenzl +Thomas Grainger +Thomas Guettler +Thomas Johansson +Thomas Kluyver +Thomas Smith +Tim D. Smith +Tim Gates +Tim Harder +Tim Heap +tim smith +tinruufu +Tom Forbes +Tom Freudenheim +Tom V +Tomas Orsava +Tomer Chachamu +Tony Beswick +Tony Zhaocheng Tan +TonyBeswick +toonarmycaptain +Toshio Kuratomi +Travis Swicegood +Tzu-ping Chung +Valentin Haenel +Victor Stinner +victorvpaulo +Viktor Szépe +Ville Skyttä +Vinay Sajip +Vincent Philippon +Vinicyus Macedo <7549205+vinicyusmacedo@users.noreply.github.com> +Vitaly Babiy +Vladimir Rutsky +W. Trevor King +Wil Tan +Wilfred Hughes +William ML Leslie +William T Olson +Wilson Mo +wim glenn +Wolfgang Maier +Xavier Fernandez +Xavier Fernandez +xoviat +xtreak +YAMAMOTO Takashi +Yen Chi Hsuan +Yeray Diaz Diaz +Yoval P +Yu Jian +Yuan Jing Vincent Yan +Zearin +Zearin +Zhiping Deng +Zvezdan Petkovic +Łukasz Langa +Семён Марьясин diff --git a/env/lib/python3.8/site-packages/pkg_resources-0.0.0.dist-info/INSTALLER b/env/lib/python3.8/site-packages/pkg_resources-0.0.0.dist-info/INSTALLER new file mode 100644 index 000000000..a1b589e38 --- /dev/null +++ b/env/lib/python3.8/site-packages/pkg_resources-0.0.0.dist-info/INSTALLER @@ -0,0 +1 @@ +pip diff --git a/env/lib/python3.8/site-packages/pkg_resources-0.0.0.dist-info/LICENSE.txt b/env/lib/python3.8/site-packages/pkg_resources-0.0.0.dist-info/LICENSE.txt new file mode 100644 index 000000000..737fec5c5 --- /dev/null +++ b/env/lib/python3.8/site-packages/pkg_resources-0.0.0.dist-info/LICENSE.txt @@ -0,0 +1,20 @@ +Copyright (c) 2008-2019 The pip developers (see AUTHORS.txt file) + +Permission is hereby granted, free of charge, to any person obtaining +a copy of this software and associated documentation files (the +"Software"), to deal in the Software without restriction, including +without limitation the rights to use, copy, modify, merge, publish, +distribute, sublicense, and/or sell copies of the Software, and to +permit persons to whom the Software is furnished to do so, subject to +the following conditions: + +The above copyright notice and this permission notice shall be +included in all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, +EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF +MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE +LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION +OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION +WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. diff --git a/env/lib/python3.8/site-packages/pkg_resources-0.0.0.dist-info/METADATA b/env/lib/python3.8/site-packages/pkg_resources-0.0.0.dist-info/METADATA new file mode 100644 index 000000000..cf6c9302c --- /dev/null +++ b/env/lib/python3.8/site-packages/pkg_resources-0.0.0.dist-info/METADATA @@ -0,0 +1,13 @@ +Metadata-Version: 2.1 +Name: pkg_resources +Version: 0.0.0 +Summary: UNKNOWN +Home-page: UNKNOWN +Author: UNKNOWN +Author-email: UNKNOWN +License: UNKNOWN +Platform: UNKNOWN + +UNKNOWN + + diff --git a/env/lib/python3.8/site-packages/pkg_resources-0.0.0.dist-info/RECORD b/env/lib/python3.8/site-packages/pkg_resources-0.0.0.dist-info/RECORD new file mode 100644 index 000000000..2bbae1dbd --- /dev/null +++ b/env/lib/python3.8/site-packages/pkg_resources-0.0.0.dist-info/RECORD @@ -0,0 +1,38 @@ +pkg_resources-0.0.0.dist-info/AUTHORS.txt,sha256=RtqU9KfonVGhI48DAA4-yTOBUhBtQTjFhaDzHoyh7uU,21518 +pkg_resources-0.0.0.dist-info/INSTALLER,sha256=zuuue4knoyJ-UwPPXg8fezS7VCrXJQrAP7zeNuwvFQg,4 +pkg_resources-0.0.0.dist-info/LICENSE.txt,sha256=W6Ifuwlk-TatfRU2LR7W1JMcyMj5_y1NkRkOEJvnRDE,1090 +pkg_resources-0.0.0.dist-info/METADATA,sha256=V9_WPOtD1FnuKrTGv6Ique7kAOn2lasvT8W0_iMCCCk,177 +pkg_resources-0.0.0.dist-info/RECORD,, +pkg_resources-0.0.0.dist-info/WHEEL,sha256=kGT74LWyRUZrL4VgLh6_g12IeVl_9u9ZVhadrgXZUEY,110 +pkg_resources/__init__.py,sha256=0IssxXPnaDKpYZRra8Ime0JG4hwosQljItGD0bnIkGk,108349 +pkg_resources/__pycache__/__init__.cpython-38.pyc,, +pkg_resources/__pycache__/py31compat.cpython-38.pyc,, +pkg_resources/_vendor/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0 +pkg_resources/_vendor/__pycache__/__init__.cpython-38.pyc,, +pkg_resources/_vendor/__pycache__/appdirs.cpython-38.pyc,, +pkg_resources/_vendor/__pycache__/pyparsing.cpython-38.pyc,, +pkg_resources/_vendor/__pycache__/six.cpython-38.pyc,, +pkg_resources/_vendor/appdirs.py,sha256=MievUEuv3l_mQISH5SF0shDk_BNhHHzYiAPrT3ITN4I,24701 +pkg_resources/_vendor/packaging/__about__.py,sha256=zkcCPTN_6TcLW0Nrlg0176-R1QQ_WVPTm8sz1R4-HjM,720 +pkg_resources/_vendor/packaging/__init__.py,sha256=_vNac5TrzwsrzbOFIbF-5cHqc_Y2aPT2D7zrIR06BOo,513 +pkg_resources/_vendor/packaging/__pycache__/__about__.cpython-38.pyc,, +pkg_resources/_vendor/packaging/__pycache__/__init__.cpython-38.pyc,, +pkg_resources/_vendor/packaging/__pycache__/_compat.cpython-38.pyc,, +pkg_resources/_vendor/packaging/__pycache__/_structures.cpython-38.pyc,, +pkg_resources/_vendor/packaging/__pycache__/markers.cpython-38.pyc,, +pkg_resources/_vendor/packaging/__pycache__/requirements.cpython-38.pyc,, +pkg_resources/_vendor/packaging/__pycache__/specifiers.cpython-38.pyc,, +pkg_resources/_vendor/packaging/__pycache__/utils.cpython-38.pyc,, +pkg_resources/_vendor/packaging/__pycache__/version.cpython-38.pyc,, +pkg_resources/_vendor/packaging/_compat.py,sha256=Vi_A0rAQeHbU-a9X0tt1yQm9RqkgQbDSxzRw8WlU9kA,860 +pkg_resources/_vendor/packaging/_structures.py,sha256=RImECJ4c_wTlaTYYwZYLHEiebDMaAJmK1oPARhw1T5o,1416 +pkg_resources/_vendor/packaging/markers.py,sha256=uEcBBtGvzqltgnArqb9c4RrcInXezDLos14zbBHhWJo,8248 +pkg_resources/_vendor/packaging/requirements.py,sha256=SikL2UynbsT0qtY9ltqngndha_sfo0w6XGFhAhoSoaQ,4355 +pkg_resources/_vendor/packaging/specifiers.py,sha256=SAMRerzO3fK2IkFZCaZkuwZaL_EGqHNOz4pni4vhnN0,28025 +pkg_resources/_vendor/packaging/utils.py,sha256=3m6WvPm6NNxE8rkTGmn0r75B_GZSGg7ikafxHsBN1WA,421 +pkg_resources/_vendor/packaging/version.py,sha256=OwGnxYfr2ghNzYx59qWIBkrK3SnB6n-Zfd1XaLpnnM0,11556 +pkg_resources/_vendor/pyparsing.py,sha256=tmrp-lu-qO1i75ZzIN5A12nKRRD1Cm4Vpk-5LR9rims,232055 +pkg_resources/_vendor/six.py,sha256=A6hdJZVjI3t_geebZ9BzUvwRrIXo0lfwzQlM2LcKyas,30098 +pkg_resources/extern/__init__.py,sha256=cHiEfHuLmm6rs5Ve_ztBfMI7Lr31vss-D4wkqF5xzlI,2498 +pkg_resources/extern/__pycache__/__init__.cpython-38.pyc,, +pkg_resources/py31compat.py,sha256=-WQ0e4c3RG_acdhwC3gLiXhP_lg4G5q7XYkZkQg0gxU,558 diff --git a/env/lib/python3.8/site-packages/pkg_resources-0.0.0.dist-info/WHEEL b/env/lib/python3.8/site-packages/pkg_resources-0.0.0.dist-info/WHEEL new file mode 100644 index 000000000..ef99c6cf3 --- /dev/null +++ b/env/lib/python3.8/site-packages/pkg_resources-0.0.0.dist-info/WHEEL @@ -0,0 +1,6 @@ +Wheel-Version: 1.0 +Generator: bdist_wheel (0.34.2) +Root-Is-Purelib: true +Tag: py2-none-any +Tag: py3-none-any + diff --git a/env/lib/python3.8/site-packages/pkg_resources/__init__.py b/env/lib/python3.8/site-packages/pkg_resources/__init__.py new file mode 100644 index 000000000..2f5aa64a6 --- /dev/null +++ b/env/lib/python3.8/site-packages/pkg_resources/__init__.py @@ -0,0 +1,3296 @@ +# coding: utf-8 +""" +Package resource API +-------------------- + +A resource is a logical file contained within a package, or a logical +subdirectory thereof. The package resource API expects resource names +to have their path parts separated with ``/``, *not* whatever the local +path separator is. Do not use os.path operations to manipulate resource +names being passed into the API. + +The package resource API is designed to work with normal filesystem packages, +.egg files, and unpacked .egg files. It can also work in a limited way with +.zip files and with custom PEP 302 loaders that support the ``get_data()`` +method. +""" + +from __future__ import absolute_import + +import sys +import os +import io +import time +import re +import types +import zipfile +import zipimport +import warnings +import stat +import functools +import pkgutil +import operator +import platform +import collections +import plistlib +import email.parser +import errno +import tempfile +import textwrap +import itertools +import inspect +import ntpath +import posixpath +from pkgutil import get_importer + +try: + import _imp +except ImportError: + # Python 3.2 compatibility + import imp as _imp + +try: + FileExistsError +except NameError: + FileExistsError = OSError + +from pkg_resources.extern import six +from pkg_resources.extern.six.moves import urllib, map, filter + +# capture these to bypass sandboxing +from os import utime +try: + from os import mkdir, rename, unlink + WRITE_SUPPORT = True +except ImportError: + # no write support, probably under GAE + WRITE_SUPPORT = False + +from os import open as os_open +from os.path import isdir, split + +try: + import importlib.machinery as importlib_machinery + # access attribute to force import under delayed import mechanisms. + importlib_machinery.__name__ +except ImportError: + importlib_machinery = None + +from . import py31compat +from pkg_resources.extern import appdirs +from pkg_resources.extern import packaging +__import__('pkg_resources.extern.packaging.version') +__import__('pkg_resources.extern.packaging.specifiers') +__import__('pkg_resources.extern.packaging.requirements') +__import__('pkg_resources.extern.packaging.markers') + + +__metaclass__ = type + + +if (3, 0) < sys.version_info < (3, 5): + raise RuntimeError("Python 3.5 or later is required") + +if six.PY2: + # Those builtin exceptions are only defined in Python 3 + PermissionError = None + NotADirectoryError = None + +# declare some globals that will be defined later to +# satisfy the linters. +require = None +working_set = None +add_activation_listener = None +resources_stream = None +cleanup_resources = None +resource_dir = None +resource_stream = None +set_extraction_path = None +resource_isdir = None +resource_string = None +iter_entry_points = None +resource_listdir = None +resource_filename = None +resource_exists = None +_distribution_finders = None +_namespace_handlers = None +_namespace_packages = None + + +class PEP440Warning(RuntimeWarning): + """ + Used when there is an issue with a version or specifier not complying with + PEP 440. + """ + + +def parse_version(v): + try: + return packaging.version.Version(v) + except packaging.version.InvalidVersion: + return packaging.version.LegacyVersion(v) + + +_state_vars = {} + + +def _declare_state(vartype, **kw): + globals().update(kw) + _state_vars.update(dict.fromkeys(kw, vartype)) + + +def __getstate__(): + state = {} + g = globals() + for k, v in _state_vars.items(): + state[k] = g['_sget_' + v](g[k]) + return state + + +def __setstate__(state): + g = globals() + for k, v in state.items(): + g['_sset_' + _state_vars[k]](k, g[k], v) + return state + + +def _sget_dict(val): + return val.copy() + + +def _sset_dict(key, ob, state): + ob.clear() + ob.update(state) + + +def _sget_object(val): + return val.__getstate__() + + +def _sset_object(key, ob, state): + ob.__setstate__(state) + + +_sget_none = _sset_none = lambda *args: None + + +def get_supported_platform(): + """Return this platform's maximum compatible version. + + distutils.util.get_platform() normally reports the minimum version + of Mac OS X that would be required to *use* extensions produced by + distutils. But what we want when checking compatibility is to know the + version of Mac OS X that we are *running*. To allow usage of packages that + explicitly require a newer version of Mac OS X, we must also know the + current version of the OS. + + If this condition occurs for any other platform with a version in its + platform strings, this function should be extended accordingly. + """ + plat = get_build_platform() + m = macosVersionString.match(plat) + if m is not None and sys.platform == "darwin": + try: + plat = 'macosx-%s-%s' % ('.'.join(_macosx_vers()[:2]), m.group(3)) + except ValueError: + # not Mac OS X + pass + return plat + + +__all__ = [ + # Basic resource access and distribution/entry point discovery + 'require', 'run_script', 'get_provider', 'get_distribution', + 'load_entry_point', 'get_entry_map', 'get_entry_info', + 'iter_entry_points', + 'resource_string', 'resource_stream', 'resource_filename', + 'resource_listdir', 'resource_exists', 'resource_isdir', + + # Environmental control + 'declare_namespace', 'working_set', 'add_activation_listener', + 'find_distributions', 'set_extraction_path', 'cleanup_resources', + 'get_default_cache', + + # Primary implementation classes + 'Environment', 'WorkingSet', 'ResourceManager', + 'Distribution', 'Requirement', 'EntryPoint', + + # Exceptions + 'ResolutionError', 'VersionConflict', 'DistributionNotFound', + 'UnknownExtra', 'ExtractionError', + + # Warnings + 'PEP440Warning', + + # Parsing functions and string utilities + 'parse_requirements', 'parse_version', 'safe_name', 'safe_version', + 'get_platform', 'compatible_platforms', 'yield_lines', 'split_sections', + 'safe_extra', 'to_filename', 'invalid_marker', 'evaluate_marker', + + # filesystem utilities + 'ensure_directory', 'normalize_path', + + # Distribution "precedence" constants + 'EGG_DIST', 'BINARY_DIST', 'SOURCE_DIST', 'CHECKOUT_DIST', 'DEVELOP_DIST', + + # "Provider" interfaces, implementations, and registration/lookup APIs + 'IMetadataProvider', 'IResourceProvider', 'FileMetadata', + 'PathMetadata', 'EggMetadata', 'EmptyProvider', 'empty_provider', + 'NullProvider', 'EggProvider', 'DefaultProvider', 'ZipProvider', + 'register_finder', 'register_namespace_handler', 'register_loader_type', + 'fixup_namespace_packages', 'get_importer', + + # Warnings + 'PkgResourcesDeprecationWarning', + + # Deprecated/backward compatibility only + 'run_main', 'AvailableDistributions', +] + + +class ResolutionError(Exception): + """Abstract base for dependency resolution errors""" + + def __repr__(self): + return self.__class__.__name__ + repr(self.args) + + +class VersionConflict(ResolutionError): + """ + An already-installed version conflicts with the requested version. + + Should be initialized with the installed Distribution and the requested + Requirement. + """ + + _template = "{self.dist} is installed but {self.req} is required" + + @property + def dist(self): + return self.args[0] + + @property + def req(self): + return self.args[1] + + def report(self): + return self._template.format(**locals()) + + def with_context(self, required_by): + """ + If required_by is non-empty, return a version of self that is a + ContextualVersionConflict. + """ + if not required_by: + return self + args = self.args + (required_by,) + return ContextualVersionConflict(*args) + + +class ContextualVersionConflict(VersionConflict): + """ + A VersionConflict that accepts a third parameter, the set of the + requirements that required the installed Distribution. + """ + + _template = VersionConflict._template + ' by {self.required_by}' + + @property + def required_by(self): + return self.args[2] + + +class DistributionNotFound(ResolutionError): + """A requested distribution was not found""" + + _template = ("The '{self.req}' distribution was not found " + "and is required by {self.requirers_str}") + + @property + def req(self): + return self.args[0] + + @property + def requirers(self): + return self.args[1] + + @property + def requirers_str(self): + if not self.requirers: + return 'the application' + return ', '.join(self.requirers) + + def report(self): + return self._template.format(**locals()) + + def __str__(self): + return self.report() + + +class UnknownExtra(ResolutionError): + """Distribution doesn't have an "extra feature" of the given name""" + + +_provider_factories = {} + +PY_MAJOR = '{}.{}'.format(*sys.version_info) +EGG_DIST = 3 +BINARY_DIST = 2 +SOURCE_DIST = 1 +CHECKOUT_DIST = 0 +DEVELOP_DIST = -1 + + +def register_loader_type(loader_type, provider_factory): + """Register `provider_factory` to make providers for `loader_type` + + `loader_type` is the type or class of a PEP 302 ``module.__loader__``, + and `provider_factory` is a function that, passed a *module* object, + returns an ``IResourceProvider`` for that module. + """ + _provider_factories[loader_type] = provider_factory + + +def get_provider(moduleOrReq): + """Return an IResourceProvider for the named module or requirement""" + if isinstance(moduleOrReq, Requirement): + return working_set.find(moduleOrReq) or require(str(moduleOrReq))[0] + try: + module = sys.modules[moduleOrReq] + except KeyError: + __import__(moduleOrReq) + module = sys.modules[moduleOrReq] + loader = getattr(module, '__loader__', None) + return _find_adapter(_provider_factories, loader)(module) + + +def _macosx_vers(_cache=[]): + if not _cache: + version = platform.mac_ver()[0] + # fallback for MacPorts + if version == '': + plist = '/System/Library/CoreServices/SystemVersion.plist' + if os.path.exists(plist): + if hasattr(plistlib, 'readPlist'): + plist_content = plistlib.readPlist(plist) + if 'ProductVersion' in plist_content: + version = plist_content['ProductVersion'] + + _cache.append(version.split('.')) + return _cache[0] + + +def _macosx_arch(machine): + return {'PowerPC': 'ppc', 'Power_Macintosh': 'ppc'}.get(machine, machine) + + +def get_build_platform(): + """Return this platform's string for platform-specific distributions + + XXX Currently this is the same as ``distutils.util.get_platform()``, but it + needs some hacks for Linux and Mac OS X. + """ + from sysconfig import get_platform + + plat = get_platform() + if sys.platform == "darwin" and not plat.startswith('macosx-'): + try: + version = _macosx_vers() + machine = os.uname()[4].replace(" ", "_") + return "macosx-%d.%d-%s" % ( + int(version[0]), int(version[1]), + _macosx_arch(machine), + ) + except ValueError: + # if someone is running a non-Mac darwin system, this will fall + # through to the default implementation + pass + return plat + + +macosVersionString = re.compile(r"macosx-(\d+)\.(\d+)-(.*)") +darwinVersionString = re.compile(r"darwin-(\d+)\.(\d+)\.(\d+)-(.*)") +# XXX backward compat +get_platform = get_build_platform + + +def compatible_platforms(provided, required): + """Can code for the `provided` platform run on the `required` platform? + + Returns true if either platform is ``None``, or the platforms are equal. + + XXX Needs compatibility checks for Linux and other unixy OSes. + """ + if provided is None or required is None or provided == required: + # easy case + return True + + # Mac OS X special cases + reqMac = macosVersionString.match(required) + if reqMac: + provMac = macosVersionString.match(provided) + + # is this a Mac package? + if not provMac: + # this is backwards compatibility for packages built before + # setuptools 0.6. All packages built after this point will + # use the new macosx designation. + provDarwin = darwinVersionString.match(provided) + if provDarwin: + dversion = int(provDarwin.group(1)) + macosversion = "%s.%s" % (reqMac.group(1), reqMac.group(2)) + if dversion == 7 and macosversion >= "10.3" or \ + dversion == 8 and macosversion >= "10.4": + return True + # egg isn't macosx or legacy darwin + return False + + # are they the same major version and machine type? + if provMac.group(1) != reqMac.group(1) or \ + provMac.group(3) != reqMac.group(3): + return False + + # is the required OS major update >= the provided one? + if int(provMac.group(2)) > int(reqMac.group(2)): + return False + + return True + + # XXX Linux and other platforms' special cases should go here + return False + + +def run_script(dist_spec, script_name): + """Locate distribution `dist_spec` and run its `script_name` script""" + ns = sys._getframe(1).f_globals + name = ns['__name__'] + ns.clear() + ns['__name__'] = name + require(dist_spec)[0].run_script(script_name, ns) + + +# backward compatibility +run_main = run_script + + +def get_distribution(dist): + """Return a current distribution object for a Requirement or string""" + if isinstance(dist, six.string_types): + dist = Requirement.parse(dist) + if isinstance(dist, Requirement): + dist = get_provider(dist) + if not isinstance(dist, Distribution): + raise TypeError("Expected string, Requirement, or Distribution", dist) + return dist + + +def load_entry_point(dist, group, name): + """Return `name` entry point of `group` for `dist` or raise ImportError""" + return get_distribution(dist).load_entry_point(group, name) + + +def get_entry_map(dist, group=None): + """Return the entry point map for `group`, or the full entry map""" + return get_distribution(dist).get_entry_map(group) + + +def get_entry_info(dist, group, name): + """Return the EntryPoint object for `group`+`name`, or ``None``""" + return get_distribution(dist).get_entry_info(group, name) + + +class IMetadataProvider: + def has_metadata(name): + """Does the package's distribution contain the named metadata?""" + + def get_metadata(name): + """The named metadata resource as a string""" + + def get_metadata_lines(name): + """Yield named metadata resource as list of non-blank non-comment lines + + Leading and trailing whitespace is stripped from each line, and lines + with ``#`` as the first non-blank character are omitted.""" + + def metadata_isdir(name): + """Is the named metadata a directory? (like ``os.path.isdir()``)""" + + def metadata_listdir(name): + """List of metadata names in the directory (like ``os.listdir()``)""" + + def run_script(script_name, namespace): + """Execute the named script in the supplied namespace dictionary""" + + +class IResourceProvider(IMetadataProvider): + """An object that provides access to package resources""" + + def get_resource_filename(manager, resource_name): + """Return a true filesystem path for `resource_name` + + `manager` must be an ``IResourceManager``""" + + def get_resource_stream(manager, resource_name): + """Return a readable file-like object for `resource_name` + + `manager` must be an ``IResourceManager``""" + + def get_resource_string(manager, resource_name): + """Return a string containing the contents of `resource_name` + + `manager` must be an ``IResourceManager``""" + + def has_resource(resource_name): + """Does the package contain the named resource?""" + + def resource_isdir(resource_name): + """Is the named resource a directory? (like ``os.path.isdir()``)""" + + def resource_listdir(resource_name): + """List of resource names in the directory (like ``os.listdir()``)""" + + +class WorkingSet: + """A collection of active distributions on sys.path (or a similar list)""" + + def __init__(self, entries=None): + """Create working set from list of path entries (default=sys.path)""" + self.entries = [] + self.entry_keys = {} + self.by_key = {} + self.callbacks = [] + + if entries is None: + entries = sys.path + + for entry in entries: + self.add_entry(entry) + + @classmethod + def _build_master(cls): + """ + Prepare the master working set. + """ + ws = cls() + try: + from __main__ import __requires__ + except ImportError: + # The main program does not list any requirements + return ws + + # ensure the requirements are met + try: + ws.require(__requires__) + except VersionConflict: + return cls._build_from_requirements(__requires__) + + return ws + + @classmethod + def _build_from_requirements(cls, req_spec): + """ + Build a working set from a requirement spec. Rewrites sys.path. + """ + # try it without defaults already on sys.path + # by starting with an empty path + ws = cls([]) + reqs = parse_requirements(req_spec) + dists = ws.resolve(reqs, Environment()) + for dist in dists: + ws.add(dist) + + # add any missing entries from sys.path + for entry in sys.path: + if entry not in ws.entries: + ws.add_entry(entry) + + # then copy back to sys.path + sys.path[:] = ws.entries + return ws + + def add_entry(self, entry): + """Add a path item to ``.entries``, finding any distributions on it + + ``find_distributions(entry, True)`` is used to find distributions + corresponding to the path entry, and they are added. `entry` is + always appended to ``.entries``, even if it is already present. + (This is because ``sys.path`` can contain the same value more than + once, and the ``.entries`` of the ``sys.path`` WorkingSet should always + equal ``sys.path``.) + """ + self.entry_keys.setdefault(entry, []) + self.entries.append(entry) + for dist in find_distributions(entry, True): + self.add(dist, entry, False) + + def __contains__(self, dist): + """True if `dist` is the active distribution for its project""" + return self.by_key.get(dist.key) == dist + + def find(self, req): + """Find a distribution matching requirement `req` + + If there is an active distribution for the requested project, this + returns it as long as it meets the version requirement specified by + `req`. But, if there is an active distribution for the project and it + does *not* meet the `req` requirement, ``VersionConflict`` is raised. + If there is no active distribution for the requested project, ``None`` + is returned. + """ + dist = self.by_key.get(req.key) + if dist is not None and dist not in req: + # XXX add more info + raise VersionConflict(dist, req) + return dist + + def iter_entry_points(self, group, name=None): + """Yield entry point objects from `group` matching `name` + + If `name` is None, yields all entry points in `group` from all + distributions in the working set, otherwise only ones matching + both `group` and `name` are yielded (in distribution order). + """ + return ( + entry + for dist in self + for entry in dist.get_entry_map(group).values() + if name is None or name == entry.name + ) + + def run_script(self, requires, script_name): + """Locate distribution for `requires` and run `script_name` script""" + ns = sys._getframe(1).f_globals + name = ns['__name__'] + ns.clear() + ns['__name__'] = name + self.require(requires)[0].run_script(script_name, ns) + + def __iter__(self): + """Yield distributions for non-duplicate projects in the working set + + The yield order is the order in which the items' path entries were + added to the working set. + """ + seen = {} + for item in self.entries: + if item not in self.entry_keys: + # workaround a cache issue + continue + + for key in self.entry_keys[item]: + if key not in seen: + seen[key] = 1 + yield self.by_key[key] + + def add(self, dist, entry=None, insert=True, replace=False): + """Add `dist` to working set, associated with `entry` + + If `entry` is unspecified, it defaults to the ``.location`` of `dist`. + On exit from this routine, `entry` is added to the end of the working + set's ``.entries`` (if it wasn't already present). + + `dist` is only added to the working set if it's for a project that + doesn't already have a distribution in the set, unless `replace=True`. + If it's added, any callbacks registered with the ``subscribe()`` method + will be called. + """ + if insert: + dist.insert_on(self.entries, entry, replace=replace) + + if entry is None: + entry = dist.location + keys = self.entry_keys.setdefault(entry, []) + keys2 = self.entry_keys.setdefault(dist.location, []) + if not replace and dist.key in self.by_key: + # ignore hidden distros + return + + self.by_key[dist.key] = dist + if dist.key not in keys: + keys.append(dist.key) + if dist.key not in keys2: + keys2.append(dist.key) + self._added_new(dist) + + def resolve(self, requirements, env=None, installer=None, + replace_conflicting=False, extras=None): + """List all distributions needed to (recursively) meet `requirements` + + `requirements` must be a sequence of ``Requirement`` objects. `env`, + if supplied, should be an ``Environment`` instance. If + not supplied, it defaults to all distributions available within any + entry or distribution in the working set. `installer`, if supplied, + will be invoked with each requirement that cannot be met by an + already-installed distribution; it should return a ``Distribution`` or + ``None``. + + Unless `replace_conflicting=True`, raises a VersionConflict exception + if + any requirements are found on the path that have the correct name but + the wrong version. Otherwise, if an `installer` is supplied it will be + invoked to obtain the correct version of the requirement and activate + it. + + `extras` is a list of the extras to be used with these requirements. + This is important because extra requirements may look like `my_req; + extra = "my_extra"`, which would otherwise be interpreted as a purely + optional requirement. Instead, we want to be able to assert that these + requirements are truly required. + """ + + # set up the stack + requirements = list(requirements)[::-1] + # set of processed requirements + processed = {} + # key -> dist + best = {} + to_activate = [] + + req_extras = _ReqExtras() + + # Mapping of requirement to set of distributions that required it; + # useful for reporting info about conflicts. + required_by = collections.defaultdict(set) + + while requirements: + # process dependencies breadth-first + req = requirements.pop(0) + if req in processed: + # Ignore cyclic or redundant dependencies + continue + + if not req_extras.markers_pass(req, extras): + continue + + dist = best.get(req.key) + if dist is None: + # Find the best distribution and add it to the map + dist = self.by_key.get(req.key) + if dist is None or (dist not in req and replace_conflicting): + ws = self + if env is None: + if dist is None: + env = Environment(self.entries) + else: + # Use an empty environment and workingset to avoid + # any further conflicts with the conflicting + # distribution + env = Environment([]) + ws = WorkingSet([]) + dist = best[req.key] = env.best_match( + req, ws, installer, + replace_conflicting=replace_conflicting + ) + if dist is None: + requirers = required_by.get(req, None) + raise DistributionNotFound(req, requirers) + to_activate.append(dist) + if dist not in req: + # Oops, the "best" so far conflicts with a dependency + dependent_req = required_by[req] + raise VersionConflict(dist, req).with_context(dependent_req) + + # push the new requirements onto the stack + new_requirements = dist.requires(req.extras)[::-1] + requirements.extend(new_requirements) + + # Register the new requirements needed by req + for new_requirement in new_requirements: + required_by[new_requirement].add(req.project_name) + req_extras[new_requirement] = req.extras + + processed[req] = True + + # return list of distros to activate + return to_activate + + def find_plugins( + self, plugin_env, full_env=None, installer=None, fallback=True): + """Find all activatable distributions in `plugin_env` + + Example usage:: + + distributions, errors = working_set.find_plugins( + Environment(plugin_dirlist) + ) + # add plugins+libs to sys.path + map(working_set.add, distributions) + # display errors + print('Could not load', errors) + + The `plugin_env` should be an ``Environment`` instance that contains + only distributions that are in the project's "plugin directory" or + directories. The `full_env`, if supplied, should be an ``Environment`` + contains all currently-available distributions. If `full_env` is not + supplied, one is created automatically from the ``WorkingSet`` this + method is called on, which will typically mean that every directory on + ``sys.path`` will be scanned for distributions. + + `installer` is a standard installer callback as used by the + ``resolve()`` method. The `fallback` flag indicates whether we should + attempt to resolve older versions of a plugin if the newest version + cannot be resolved. + + This method returns a 2-tuple: (`distributions`, `error_info`), where + `distributions` is a list of the distributions found in `plugin_env` + that were loadable, along with any other distributions that are needed + to resolve their dependencies. `error_info` is a dictionary mapping + unloadable plugin distributions to an exception instance describing the + error that occurred. Usually this will be a ``DistributionNotFound`` or + ``VersionConflict`` instance. + """ + + plugin_projects = list(plugin_env) + # scan project names in alphabetic order + plugin_projects.sort() + + error_info = {} + distributions = {} + + if full_env is None: + env = Environment(self.entries) + env += plugin_env + else: + env = full_env + plugin_env + + shadow_set = self.__class__([]) + # put all our entries in shadow_set + list(map(shadow_set.add, self)) + + for project_name in plugin_projects: + + for dist in plugin_env[project_name]: + + req = [dist.as_requirement()] + + try: + resolvees = shadow_set.resolve(req, env, installer) + + except ResolutionError as v: + # save error info + error_info[dist] = v + if fallback: + # try the next older version of project + continue + else: + # give up on this project, keep going + break + + else: + list(map(shadow_set.add, resolvees)) + distributions.update(dict.fromkeys(resolvees)) + + # success, no need to try any more versions of this project + break + + distributions = list(distributions) + distributions.sort() + + return distributions, error_info + + def require(self, *requirements): + """Ensure that distributions matching `requirements` are activated + + `requirements` must be a string or a (possibly-nested) sequence + thereof, specifying the distributions and versions required. The + return value is a sequence of the distributions that needed to be + activated to fulfill the requirements; all relevant distributions are + included, even if they were already activated in this working set. + """ + needed = self.resolve(parse_requirements(requirements)) + + for dist in needed: + self.add(dist) + + return needed + + def subscribe(self, callback, existing=True): + """Invoke `callback` for all distributions + + If `existing=True` (default), + call on all existing ones, as well. + """ + if callback in self.callbacks: + return + self.callbacks.append(callback) + if not existing: + return + for dist in self: + callback(dist) + + def _added_new(self, dist): + for callback in self.callbacks: + callback(dist) + + def __getstate__(self): + return ( + self.entries[:], self.entry_keys.copy(), self.by_key.copy(), + self.callbacks[:] + ) + + def __setstate__(self, e_k_b_c): + entries, keys, by_key, callbacks = e_k_b_c + self.entries = entries[:] + self.entry_keys = keys.copy() + self.by_key = by_key.copy() + self.callbacks = callbacks[:] + + +class _ReqExtras(dict): + """ + Map each requirement to the extras that demanded it. + """ + + def markers_pass(self, req, extras=None): + """ + Evaluate markers for req against each extra that + demanded it. + + Return False if the req has a marker and fails + evaluation. Otherwise, return True. + """ + extra_evals = ( + req.marker.evaluate({'extra': extra}) + for extra in self.get(req, ()) + (extras or (None,)) + ) + return not req.marker or any(extra_evals) + + +class Environment: + """Searchable snapshot of distributions on a search path""" + + def __init__( + self, search_path=None, platform=get_supported_platform(), + python=PY_MAJOR): + """Snapshot distributions available on a search path + + Any distributions found on `search_path` are added to the environment. + `search_path` should be a sequence of ``sys.path`` items. If not + supplied, ``sys.path`` is used. + + `platform` is an optional string specifying the name of the platform + that platform-specific distributions must be compatible with. If + unspecified, it defaults to the current platform. `python` is an + optional string naming the desired version of Python (e.g. ``'3.6'``); + it defaults to the current version. + + You may explicitly set `platform` (and/or `python`) to ``None`` if you + wish to map *all* distributions, not just those compatible with the + running platform or Python version. + """ + self._distmap = {} + self.platform = platform + self.python = python + self.scan(search_path) + + def can_add(self, dist): + """Is distribution `dist` acceptable for this environment? + + The distribution must match the platform and python version + requirements specified when this environment was created, or False + is returned. + """ + py_compat = ( + self.python is None + or dist.py_version is None + or dist.py_version == self.python + ) + return py_compat and compatible_platforms(dist.platform, self.platform) + + def remove(self, dist): + """Remove `dist` from the environment""" + self._distmap[dist.key].remove(dist) + + def scan(self, search_path=None): + """Scan `search_path` for distributions usable in this environment + + Any distributions found are added to the environment. + `search_path` should be a sequence of ``sys.path`` items. If not + supplied, ``sys.path`` is used. Only distributions conforming to + the platform/python version defined at initialization are added. + """ + if search_path is None: + search_path = sys.path + + for item in search_path: + for dist in find_distributions(item): + self.add(dist) + + def __getitem__(self, project_name): + """Return a newest-to-oldest list of distributions for `project_name` + + Uses case-insensitive `project_name` comparison, assuming all the + project's distributions use their project's name converted to all + lowercase as their key. + + """ + distribution_key = project_name.lower() + return self._distmap.get(distribution_key, []) + + def add(self, dist): + """Add `dist` if we ``can_add()`` it and it has not already been added + """ + if self.can_add(dist) and dist.has_version(): + dists = self._distmap.setdefault(dist.key, []) + if dist not in dists: + dists.append(dist) + dists.sort(key=operator.attrgetter('hashcmp'), reverse=True) + + def best_match( + self, req, working_set, installer=None, replace_conflicting=False): + """Find distribution best matching `req` and usable on `working_set` + + This calls the ``find(req)`` method of the `working_set` to see if a + suitable distribution is already active. (This may raise + ``VersionConflict`` if an unsuitable version of the project is already + active in the specified `working_set`.) If a suitable distribution + isn't active, this method returns the newest distribution in the + environment that meets the ``Requirement`` in `req`. If no suitable + distribution is found, and `installer` is supplied, then the result of + calling the environment's ``obtain(req, installer)`` method will be + returned. + """ + try: + dist = working_set.find(req) + except VersionConflict: + if not replace_conflicting: + raise + dist = None + if dist is not None: + return dist + for dist in self[req.key]: + if dist in req: + return dist + # try to download/install + return self.obtain(req, installer) + + def obtain(self, requirement, installer=None): + """Obtain a distribution matching `requirement` (e.g. via download) + + Obtain a distro that matches requirement (e.g. via download). In the + base ``Environment`` class, this routine just returns + ``installer(requirement)``, unless `installer` is None, in which case + None is returned instead. This method is a hook that allows subclasses + to attempt other ways of obtaining a distribution before falling back + to the `installer` argument.""" + if installer is not None: + return installer(requirement) + + def __iter__(self): + """Yield the unique project names of the available distributions""" + for key in self._distmap.keys(): + if self[key]: + yield key + + def __iadd__(self, other): + """In-place addition of a distribution or environment""" + if isinstance(other, Distribution): + self.add(other) + elif isinstance(other, Environment): + for project in other: + for dist in other[project]: + self.add(dist) + else: + raise TypeError("Can't add %r to environment" % (other,)) + return self + + def __add__(self, other): + """Add an environment or distribution to an environment""" + new = self.__class__([], platform=None, python=None) + for env in self, other: + new += env + return new + + +# XXX backward compatibility +AvailableDistributions = Environment + + +class ExtractionError(RuntimeError): + """An error occurred extracting a resource + + The following attributes are available from instances of this exception: + + manager + The resource manager that raised this exception + + cache_path + The base directory for resource extraction + + original_error + The exception instance that caused extraction to fail + """ + + +class ResourceManager: + """Manage resource extraction and packages""" + extraction_path = None + + def __init__(self): + self.cached_files = {} + + def resource_exists(self, package_or_requirement, resource_name): + """Does the named resource exist?""" + return get_provider(package_or_requirement).has_resource(resource_name) + + def resource_isdir(self, package_or_requirement, resource_name): + """Is the named resource an existing directory?""" + return get_provider(package_or_requirement).resource_isdir( + resource_name + ) + + def resource_filename(self, package_or_requirement, resource_name): + """Return a true filesystem path for specified resource""" + return get_provider(package_or_requirement).get_resource_filename( + self, resource_name + ) + + def resource_stream(self, package_or_requirement, resource_name): + """Return a readable file-like object for specified resource""" + return get_provider(package_or_requirement).get_resource_stream( + self, resource_name + ) + + def resource_string(self, package_or_requirement, resource_name): + """Return specified resource as a string""" + return get_provider(package_or_requirement).get_resource_string( + self, resource_name + ) + + def resource_listdir(self, package_or_requirement, resource_name): + """List the contents of the named resource directory""" + return get_provider(package_or_requirement).resource_listdir( + resource_name + ) + + def extraction_error(self): + """Give an error message for problems extracting file(s)""" + + old_exc = sys.exc_info()[1] + cache_path = self.extraction_path or get_default_cache() + + tmpl = textwrap.dedent(""" + Can't extract file(s) to egg cache + + The following error occurred while trying to extract file(s) + to the Python egg cache: + + {old_exc} + + The Python egg cache directory is currently set to: + + {cache_path} + + Perhaps your account does not have write access to this directory? + You can change the cache directory by setting the PYTHON_EGG_CACHE + environment variable to point to an accessible directory. + """).lstrip() + err = ExtractionError(tmpl.format(**locals())) + err.manager = self + err.cache_path = cache_path + err.original_error = old_exc + raise err + + def get_cache_path(self, archive_name, names=()): + """Return absolute location in cache for `archive_name` and `names` + + The parent directory of the resulting path will be created if it does + not already exist. `archive_name` should be the base filename of the + enclosing egg (which may not be the name of the enclosing zipfile!), + including its ".egg" extension. `names`, if provided, should be a + sequence of path name parts "under" the egg's extraction location. + + This method should only be called by resource providers that need to + obtain an extraction location, and only for names they intend to + extract, as it tracks the generated names for possible cleanup later. + """ + extract_path = self.extraction_path or get_default_cache() + target_path = os.path.join(extract_path, archive_name + '-tmp', *names) + try: + _bypass_ensure_directory(target_path) + except Exception: + self.extraction_error() + + self._warn_unsafe_extraction_path(extract_path) + + self.cached_files[target_path] = 1 + return target_path + + @staticmethod + def _warn_unsafe_extraction_path(path): + """ + If the default extraction path is overridden and set to an insecure + location, such as /tmp, it opens up an opportunity for an attacker to + replace an extracted file with an unauthorized payload. Warn the user + if a known insecure location is used. + + See Distribute #375 for more details. + """ + if os.name == 'nt' and not path.startswith(os.environ['windir']): + # On Windows, permissions are generally restrictive by default + # and temp directories are not writable by other users, so + # bypass the warning. + return + mode = os.stat(path).st_mode + if mode & stat.S_IWOTH or mode & stat.S_IWGRP: + msg = ( + "%s is writable by group/others and vulnerable to attack " + "when " + "used with get_resource_filename. Consider a more secure " + "location (set with .set_extraction_path or the " + "PYTHON_EGG_CACHE environment variable)." % path + ) + warnings.warn(msg, UserWarning) + + def postprocess(self, tempname, filename): + """Perform any platform-specific postprocessing of `tempname` + + This is where Mac header rewrites should be done; other platforms don't + have anything special they should do. + + Resource providers should call this method ONLY after successfully + extracting a compressed resource. They must NOT call it on resources + that are already in the filesystem. + + `tempname` is the current (temporary) name of the file, and `filename` + is the name it will be renamed to by the caller after this routine + returns. + """ + + if os.name == 'posix': + # Make the resource executable + mode = ((os.stat(tempname).st_mode) | 0o555) & 0o7777 + os.chmod(tempname, mode) + + def set_extraction_path(self, path): + """Set the base path where resources will be extracted to, if needed. + + If you do not call this routine before any extractions take place, the + path defaults to the return value of ``get_default_cache()``. (Which + is based on the ``PYTHON_EGG_CACHE`` environment variable, with various + platform-specific fallbacks. See that routine's documentation for more + details.) + + Resources are extracted to subdirectories of this path based upon + information given by the ``IResourceProvider``. You may set this to a + temporary directory, but then you must call ``cleanup_resources()`` to + delete the extracted files when done. There is no guarantee that + ``cleanup_resources()`` will be able to remove all extracted files. + + (Note: you may not change the extraction path for a given resource + manager once resources have been extracted, unless you first call + ``cleanup_resources()``.) + """ + if self.cached_files: + raise ValueError( + "Can't change extraction path, files already extracted" + ) + + self.extraction_path = path + + def cleanup_resources(self, force=False): + """ + Delete all extracted resource files and directories, returning a list + of the file and directory names that could not be successfully removed. + This function does not have any concurrency protection, so it should + generally only be called when the extraction path is a temporary + directory exclusive to a single process. This method is not + automatically called; you must call it explicitly or register it as an + ``atexit`` function if you wish to ensure cleanup of a temporary + directory used for extractions. + """ + # XXX + + +def get_default_cache(): + """ + Return the ``PYTHON_EGG_CACHE`` environment variable + or a platform-relevant user cache dir for an app + named "Python-Eggs". + """ + return ( + os.environ.get('PYTHON_EGG_CACHE') + or appdirs.user_cache_dir(appname='Python-Eggs') + ) + + +def safe_name(name): + """Convert an arbitrary string to a standard distribution name + + Any runs of non-alphanumeric/. characters are replaced with a single '-'. + """ + return re.sub('[^A-Za-z0-9.]+', '-', name) + + +def safe_version(version): + """ + Convert an arbitrary string to a standard version string + """ + try: + # normalize the version + return str(packaging.version.Version(version)) + except packaging.version.InvalidVersion: + version = version.replace(' ', '.') + return re.sub('[^A-Za-z0-9.]+', '-', version) + + +def safe_extra(extra): + """Convert an arbitrary string to a standard 'extra' name + + Any runs of non-alphanumeric characters are replaced with a single '_', + and the result is always lowercased. + """ + return re.sub('[^A-Za-z0-9.-]+', '_', extra).lower() + + +def to_filename(name): + """Convert a project or version name to its filename-escaped form + + Any '-' characters are currently replaced with '_'. + """ + return name.replace('-', '_') + + +def invalid_marker(text): + """ + Validate text as a PEP 508 environment marker; return an exception + if invalid or False otherwise. + """ + try: + evaluate_marker(text) + except SyntaxError as e: + e.filename = None + e.lineno = None + return e + return False + + +def evaluate_marker(text, extra=None): + """ + Evaluate a PEP 508 environment marker. + Return a boolean indicating the marker result in this environment. + Raise SyntaxError if marker is invalid. + + This implementation uses the 'pyparsing' module. + """ + try: + marker = packaging.markers.Marker(text) + return marker.evaluate() + except packaging.markers.InvalidMarker as e: + raise SyntaxError(e) + + +class NullProvider: + """Try to implement resources and metadata for arbitrary PEP 302 loaders""" + + egg_name = None + egg_info = None + loader = None + + def __init__(self, module): + self.loader = getattr(module, '__loader__', None) + self.module_path = os.path.dirname(getattr(module, '__file__', '')) + + def get_resource_filename(self, manager, resource_name): + return self._fn(self.module_path, resource_name) + + def get_resource_stream(self, manager, resource_name): + return io.BytesIO(self.get_resource_string(manager, resource_name)) + + def get_resource_string(self, manager, resource_name): + return self._get(self._fn(self.module_path, resource_name)) + + def has_resource(self, resource_name): + return self._has(self._fn(self.module_path, resource_name)) + + def _get_metadata_path(self, name): + return self._fn(self.egg_info, name) + + def has_metadata(self, name): + if not self.egg_info: + return self.egg_info + + path = self._get_metadata_path(name) + return self._has(path) + + def get_metadata(self, name): + if not self.egg_info: + return "" + path = self._get_metadata_path(name) + value = self._get(path) + if six.PY2: + return value + try: + return value.decode('utf-8') + except UnicodeDecodeError as exc: + # Include the path in the error message to simplify + # troubleshooting, and without changing the exception type. + exc.reason += ' in {} file at path: {}'.format(name, path) + raise + + def get_metadata_lines(self, name): + return yield_lines(self.get_metadata(name)) + + def resource_isdir(self, resource_name): + return self._isdir(self._fn(self.module_path, resource_name)) + + def metadata_isdir(self, name): + return self.egg_info and self._isdir(self._fn(self.egg_info, name)) + + def resource_listdir(self, resource_name): + return self._listdir(self._fn(self.module_path, resource_name)) + + def metadata_listdir(self, name): + if self.egg_info: + return self._listdir(self._fn(self.egg_info, name)) + return [] + + def run_script(self, script_name, namespace): + script = 'scripts/' + script_name + if not self.has_metadata(script): + raise ResolutionError( + "Script {script!r} not found in metadata at {self.egg_info!r}" + .format(**locals()), + ) + script_text = self.get_metadata(script).replace('\r\n', '\n') + script_text = script_text.replace('\r', '\n') + script_filename = self._fn(self.egg_info, script) + namespace['__file__'] = script_filename + if os.path.exists(script_filename): + source = open(script_filename).read() + code = compile(source, script_filename, 'exec') + exec(code, namespace, namespace) + else: + from linecache import cache + cache[script_filename] = ( + len(script_text), 0, script_text.split('\n'), script_filename + ) + script_code = compile(script_text, script_filename, 'exec') + exec(script_code, namespace, namespace) + + def _has(self, path): + raise NotImplementedError( + "Can't perform this operation for unregistered loader type" + ) + + def _isdir(self, path): + raise NotImplementedError( + "Can't perform this operation for unregistered loader type" + ) + + def _listdir(self, path): + raise NotImplementedError( + "Can't perform this operation for unregistered loader type" + ) + + def _fn(self, base, resource_name): + self._validate_resource_path(resource_name) + if resource_name: + return os.path.join(base, *resource_name.split('/')) + return base + + @staticmethod + def _validate_resource_path(path): + """ + Validate the resource paths according to the docs. + https://setuptools.readthedocs.io/en/latest/pkg_resources.html#basic-resource-access + + >>> warned = getfixture('recwarn') + >>> warnings.simplefilter('always') + >>> vrp = NullProvider._validate_resource_path + >>> vrp('foo/bar.txt') + >>> bool(warned) + False + >>> vrp('../foo/bar.txt') + >>> bool(warned) + True + >>> warned.clear() + >>> vrp('/foo/bar.txt') + >>> bool(warned) + True + >>> vrp('foo/../../bar.txt') + >>> bool(warned) + True + >>> warned.clear() + >>> vrp('foo/f../bar.txt') + >>> bool(warned) + False + + Windows path separators are straight-up disallowed. + >>> vrp(r'\\foo/bar.txt') + Traceback (most recent call last): + ... + ValueError: Use of .. or absolute path in a resource path \ +is not allowed. + + >>> vrp(r'C:\\foo/bar.txt') + Traceback (most recent call last): + ... + ValueError: Use of .. or absolute path in a resource path \ +is not allowed. + + Blank values are allowed + + >>> vrp('') + >>> bool(warned) + False + + Non-string values are not. + + >>> vrp(None) + Traceback (most recent call last): + ... + AttributeError: ... + """ + invalid = ( + os.path.pardir in path.split(posixpath.sep) or + posixpath.isabs(path) or + ntpath.isabs(path) + ) + if not invalid: + return + + msg = "Use of .. or absolute path in a resource path is not allowed." + + # Aggressively disallow Windows absolute paths + if ntpath.isabs(path) and not posixpath.isabs(path): + raise ValueError(msg) + + # for compatibility, warn; in future + # raise ValueError(msg) + warnings.warn( + msg[:-1] + " and will raise exceptions in a future release.", + DeprecationWarning, + stacklevel=4, + ) + + def _get(self, path): + if hasattr(self.loader, 'get_data'): + return self.loader.get_data(path) + raise NotImplementedError( + "Can't perform this operation for loaders without 'get_data()'" + ) + + +register_loader_type(object, NullProvider) + + +class EggProvider(NullProvider): + """Provider based on a virtual filesystem""" + + def __init__(self, module): + NullProvider.__init__(self, module) + self._setup_prefix() + + def _setup_prefix(self): + # we assume here that our metadata may be nested inside a "basket" + # of multiple eggs; that's why we use module_path instead of .archive + path = self.module_path + old = None + while path != old: + if _is_egg_path(path): + self.egg_name = os.path.basename(path) + self.egg_info = os.path.join(path, 'EGG-INFO') + self.egg_root = path + break + old = path + path, base = os.path.split(path) + + +class DefaultProvider(EggProvider): + """Provides access to package resources in the filesystem""" + + def _has(self, path): + return os.path.exists(path) + + def _isdir(self, path): + return os.path.isdir(path) + + def _listdir(self, path): + return os.listdir(path) + + def get_resource_stream(self, manager, resource_name): + return open(self._fn(self.module_path, resource_name), 'rb') + + def _get(self, path): + with open(path, 'rb') as stream: + return stream.read() + + @classmethod + def _register(cls): + loader_names = 'SourceFileLoader', 'SourcelessFileLoader', + for name in loader_names: + loader_cls = getattr(importlib_machinery, name, type(None)) + register_loader_type(loader_cls, cls) + + +DefaultProvider._register() + + +class EmptyProvider(NullProvider): + """Provider that returns nothing for all requests""" + + module_path = None + + _isdir = _has = lambda self, path: False + + def _get(self, path): + return '' + + def _listdir(self, path): + return [] + + def __init__(self): + pass + + +empty_provider = EmptyProvider() + + +class ZipManifests(dict): + """ + zip manifest builder + """ + + @classmethod + def build(cls, path): + """ + Build a dictionary similar to the zipimport directory + caches, except instead of tuples, store ZipInfo objects. + + Use a platform-specific path separator (os.sep) for the path keys + for compatibility with pypy on Windows. + """ + with zipfile.ZipFile(path) as zfile: + items = ( + ( + name.replace('/', os.sep), + zfile.getinfo(name), + ) + for name in zfile.namelist() + ) + return dict(items) + + load = build + + +class MemoizedZipManifests(ZipManifests): + """ + Memoized zipfile manifests. + """ + manifest_mod = collections.namedtuple('manifest_mod', 'manifest mtime') + + def load(self, path): + """ + Load a manifest at path or return a suitable manifest already loaded. + """ + path = os.path.normpath(path) + mtime = os.stat(path).st_mtime + + if path not in self or self[path].mtime != mtime: + manifest = self.build(path) + self[path] = self.manifest_mod(manifest, mtime) + + return self[path].manifest + + +class ZipProvider(EggProvider): + """Resource support for zips and eggs""" + + eagers = None + _zip_manifests = MemoizedZipManifests() + + def __init__(self, module): + EggProvider.__init__(self, module) + self.zip_pre = self.loader.archive + os.sep + + def _zipinfo_name(self, fspath): + # Convert a virtual filename (full path to file) into a zipfile subpath + # usable with the zipimport directory cache for our target archive + fspath = fspath.rstrip(os.sep) + if fspath == self.loader.archive: + return '' + if fspath.startswith(self.zip_pre): + return fspath[len(self.zip_pre):] + raise AssertionError( + "%s is not a subpath of %s" % (fspath, self.zip_pre) + ) + + def _parts(self, zip_path): + # Convert a zipfile subpath into an egg-relative path part list. + # pseudo-fs path + fspath = self.zip_pre + zip_path + if fspath.startswith(self.egg_root + os.sep): + return fspath[len(self.egg_root) + 1:].split(os.sep) + raise AssertionError( + "%s is not a subpath of %s" % (fspath, self.egg_root) + ) + + @property + def zipinfo(self): + return self._zip_manifests.load(self.loader.archive) + + def get_resource_filename(self, manager, resource_name): + if not self.egg_name: + raise NotImplementedError( + "resource_filename() only supported for .egg, not .zip" + ) + # no need to lock for extraction, since we use temp names + zip_path = self._resource_to_zip(resource_name) + eagers = self._get_eager_resources() + if '/'.join(self._parts(zip_path)) in eagers: + for name in eagers: + self._extract_resource(manager, self._eager_to_zip(name)) + return self._extract_resource(manager, zip_path) + + @staticmethod + def _get_date_and_size(zip_stat): + size = zip_stat.file_size + # ymdhms+wday, yday, dst + date_time = zip_stat.date_time + (0, 0, -1) + # 1980 offset already done + timestamp = time.mktime(date_time) + return timestamp, size + + def _extract_resource(self, manager, zip_path): + + if zip_path in self._index(): + for name in self._index()[zip_path]: + last = self._extract_resource( + manager, os.path.join(zip_path, name) + ) + # return the extracted directory name + return os.path.dirname(last) + + timestamp, size = self._get_date_and_size(self.zipinfo[zip_path]) + + if not WRITE_SUPPORT: + raise IOError('"os.rename" and "os.unlink" are not supported ' + 'on this platform') + try: + + real_path = manager.get_cache_path( + self.egg_name, self._parts(zip_path) + ) + + if self._is_current(real_path, zip_path): + return real_path + + outf, tmpnam = _mkstemp( + ".$extract", + dir=os.path.dirname(real_path), + ) + os.write(outf, self.loader.get_data(zip_path)) + os.close(outf) + utime(tmpnam, (timestamp, timestamp)) + manager.postprocess(tmpnam, real_path) + + try: + rename(tmpnam, real_path) + + except os.error: + if os.path.isfile(real_path): + if self._is_current(real_path, zip_path): + # the file became current since it was checked above, + # so proceed. + return real_path + # Windows, del old file and retry + elif os.name == 'nt': + unlink(real_path) + rename(tmpnam, real_path) + return real_path + raise + + except os.error: + # report a user-friendly error + manager.extraction_error() + + return real_path + + def _is_current(self, file_path, zip_path): + """ + Return True if the file_path is current for this zip_path + """ + timestamp, size = self._get_date_and_size(self.zipinfo[zip_path]) + if not os.path.isfile(file_path): + return False + stat = os.stat(file_path) + if stat.st_size != size or stat.st_mtime != timestamp: + return False + # check that the contents match + zip_contents = self.loader.get_data(zip_path) + with open(file_path, 'rb') as f: + file_contents = f.read() + return zip_contents == file_contents + + def _get_eager_resources(self): + if self.eagers is None: + eagers = [] + for name in ('native_libs.txt', 'eager_resources.txt'): + if self.has_metadata(name): + eagers.extend(self.get_metadata_lines(name)) + self.eagers = eagers + return self.eagers + + def _index(self): + try: + return self._dirindex + except AttributeError: + ind = {} + for path in self.zipinfo: + parts = path.split(os.sep) + while parts: + parent = os.sep.join(parts[:-1]) + if parent in ind: + ind[parent].append(parts[-1]) + break + else: + ind[parent] = [parts.pop()] + self._dirindex = ind + return ind + + def _has(self, fspath): + zip_path = self._zipinfo_name(fspath) + return zip_path in self.zipinfo or zip_path in self._index() + + def _isdir(self, fspath): + return self._zipinfo_name(fspath) in self._index() + + def _listdir(self, fspath): + return list(self._index().get(self._zipinfo_name(fspath), ())) + + def _eager_to_zip(self, resource_name): + return self._zipinfo_name(self._fn(self.egg_root, resource_name)) + + def _resource_to_zip(self, resource_name): + return self._zipinfo_name(self._fn(self.module_path, resource_name)) + + +register_loader_type(zipimport.zipimporter, ZipProvider) + + +class FileMetadata(EmptyProvider): + """Metadata handler for standalone PKG-INFO files + + Usage:: + + metadata = FileMetadata("/path/to/PKG-INFO") + + This provider rejects all data and metadata requests except for PKG-INFO, + which is treated as existing, and will be the contents of the file at + the provided location. + """ + + def __init__(self, path): + self.path = path + + def _get_metadata_path(self, name): + return self.path + + def has_metadata(self, name): + return name == 'PKG-INFO' and os.path.isfile(self.path) + + def get_metadata(self, name): + if name != 'PKG-INFO': + raise KeyError("No metadata except PKG-INFO is available") + + with io.open(self.path, encoding='utf-8', errors="replace") as f: + metadata = f.read() + self._warn_on_replacement(metadata) + return metadata + + def _warn_on_replacement(self, metadata): + # Python 2.7 compat for: replacement_char = '�' + replacement_char = b'\xef\xbf\xbd'.decode('utf-8') + if replacement_char in metadata: + tmpl = "{self.path} could not be properly decoded in UTF-8" + msg = tmpl.format(**locals()) + warnings.warn(msg) + + def get_metadata_lines(self, name): + return yield_lines(self.get_metadata(name)) + + +class PathMetadata(DefaultProvider): + """Metadata provider for egg directories + + Usage:: + + # Development eggs: + + egg_info = "/path/to/PackageName.egg-info" + base_dir = os.path.dirname(egg_info) + metadata = PathMetadata(base_dir, egg_info) + dist_name = os.path.splitext(os.path.basename(egg_info))[0] + dist = Distribution(basedir, project_name=dist_name, metadata=metadata) + + # Unpacked egg directories: + + egg_path = "/path/to/PackageName-ver-pyver-etc.egg" + metadata = PathMetadata(egg_path, os.path.join(egg_path,'EGG-INFO')) + dist = Distribution.from_filename(egg_path, metadata=metadata) + """ + + def __init__(self, path, egg_info): + self.module_path = path + self.egg_info = egg_info + + +class EggMetadata(ZipProvider): + """Metadata provider for .egg files""" + + def __init__(self, importer): + """Create a metadata provider from a zipimporter""" + + self.zip_pre = importer.archive + os.sep + self.loader = importer + if importer.prefix: + self.module_path = os.path.join(importer.archive, importer.prefix) + else: + self.module_path = importer.archive + self._setup_prefix() + + +_declare_state('dict', _distribution_finders={}) + + +def register_finder(importer_type, distribution_finder): + """Register `distribution_finder` to find distributions in sys.path items + + `importer_type` is the type or class of a PEP 302 "Importer" (sys.path item + handler), and `distribution_finder` is a callable that, passed a path + item and the importer instance, yields ``Distribution`` instances found on + that path item. See ``pkg_resources.find_on_path`` for an example.""" + _distribution_finders[importer_type] = distribution_finder + + +def find_distributions(path_item, only=False): + """Yield distributions accessible via `path_item`""" + importer = get_importer(path_item) + finder = _find_adapter(_distribution_finders, importer) + return finder(importer, path_item, only) + + +def find_eggs_in_zip(importer, path_item, only=False): + """ + Find eggs in zip files; possibly multiple nested eggs. + """ + if importer.archive.endswith('.whl'): + # wheels are not supported with this finder + # they don't have PKG-INFO metadata, and won't ever contain eggs + return + metadata = EggMetadata(importer) + if metadata.has_metadata('PKG-INFO'): + yield Distribution.from_filename(path_item, metadata=metadata) + if only: + # don't yield nested distros + return + for subitem in metadata.resource_listdir(''): + if _is_egg_path(subitem): + subpath = os.path.join(path_item, subitem) + dists = find_eggs_in_zip(zipimport.zipimporter(subpath), subpath) + for dist in dists: + yield dist + elif subitem.lower().endswith('.dist-info'): + subpath = os.path.join(path_item, subitem) + submeta = EggMetadata(zipimport.zipimporter(subpath)) + submeta.egg_info = subpath + yield Distribution.from_location(path_item, subitem, submeta) + + +register_finder(zipimport.zipimporter, find_eggs_in_zip) + + +def find_nothing(importer, path_item, only=False): + return () + + +register_finder(object, find_nothing) + + +def _by_version_descending(names): + """ + Given a list of filenames, return them in descending order + by version number. + + >>> names = 'bar', 'foo', 'Python-2.7.10.egg', 'Python-2.7.2.egg' + >>> _by_version_descending(names) + ['Python-2.7.10.egg', 'Python-2.7.2.egg', 'foo', 'bar'] + >>> names = 'Setuptools-1.2.3b1.egg', 'Setuptools-1.2.3.egg' + >>> _by_version_descending(names) + ['Setuptools-1.2.3.egg', 'Setuptools-1.2.3b1.egg'] + >>> names = 'Setuptools-1.2.3b1.egg', 'Setuptools-1.2.3.post1.egg' + >>> _by_version_descending(names) + ['Setuptools-1.2.3.post1.egg', 'Setuptools-1.2.3b1.egg'] + """ + def _by_version(name): + """ + Parse each component of the filename + """ + name, ext = os.path.splitext(name) + parts = itertools.chain(name.split('-'), [ext]) + return [packaging.version.parse(part) for part in parts] + + return sorted(names, key=_by_version, reverse=True) + + +def find_on_path(importer, path_item, only=False): + """Yield distributions accessible on a sys.path directory""" + path_item = _normalize_cached(path_item) + + if _is_unpacked_egg(path_item): + yield Distribution.from_filename( + path_item, metadata=PathMetadata( + path_item, os.path.join(path_item, 'EGG-INFO') + ) + ) + return + + entries = safe_listdir(path_item) + + # for performance, before sorting by version, + # screen entries for only those that will yield + # distributions + filtered = ( + entry + for entry in entries + if dist_factory(path_item, entry, only) + ) + + # scan for .egg and .egg-info in directory + path_item_entries = _by_version_descending(filtered) + for entry in path_item_entries: + fullpath = os.path.join(path_item, entry) + factory = dist_factory(path_item, entry, only) + for dist in factory(fullpath): + yield dist + + +def dist_factory(path_item, entry, only): + """ + Return a dist_factory for a path_item and entry + """ + lower = entry.lower() + is_meta = any(map(lower.endswith, ('.egg-info', '.dist-info'))) + return ( + distributions_from_metadata + if is_meta else + find_distributions + if not only and _is_egg_path(entry) else + resolve_egg_link + if not only and lower.endswith('.egg-link') else + NoDists() + ) + + +class NoDists: + """ + >>> bool(NoDists()) + False + + >>> list(NoDists()('anything')) + [] + """ + def __bool__(self): + return False + if six.PY2: + __nonzero__ = __bool__ + + def __call__(self, fullpath): + return iter(()) + + +def safe_listdir(path): + """ + Attempt to list contents of path, but suppress some exceptions. + """ + try: + return os.listdir(path) + except (PermissionError, NotADirectoryError): + pass + except OSError as e: + # Ignore the directory if does not exist, not a directory or + # permission denied + ignorable = ( + e.errno in (errno.ENOTDIR, errno.EACCES, errno.ENOENT) + # Python 2 on Windows needs to be handled this way :( + or getattr(e, "winerror", None) == 267 + ) + if not ignorable: + raise + return () + + +def distributions_from_metadata(path): + root = os.path.dirname(path) + if os.path.isdir(path): + if len(os.listdir(path)) == 0: + # empty metadata dir; skip + return + metadata = PathMetadata(root, path) + else: + metadata = FileMetadata(path) + entry = os.path.basename(path) + yield Distribution.from_location( + root, entry, metadata, precedence=DEVELOP_DIST, + ) + + +def non_empty_lines(path): + """ + Yield non-empty lines from file at path + """ + with open(path) as f: + for line in f: + line = line.strip() + if line: + yield line + + +def resolve_egg_link(path): + """ + Given a path to an .egg-link, resolve distributions + present in the referenced path. + """ + referenced_paths = non_empty_lines(path) + resolved_paths = ( + os.path.join(os.path.dirname(path), ref) + for ref in referenced_paths + ) + dist_groups = map(find_distributions, resolved_paths) + return next(dist_groups, ()) + + +register_finder(pkgutil.ImpImporter, find_on_path) + +if hasattr(importlib_machinery, 'FileFinder'): + register_finder(importlib_machinery.FileFinder, find_on_path) + +_declare_state('dict', _namespace_handlers={}) +_declare_state('dict', _namespace_packages={}) + + +def register_namespace_handler(importer_type, namespace_handler): + """Register `namespace_handler` to declare namespace packages + + `importer_type` is the type or class of a PEP 302 "Importer" (sys.path item + handler), and `namespace_handler` is a callable like this:: + + def namespace_handler(importer, path_entry, moduleName, module): + # return a path_entry to use for child packages + + Namespace handlers are only called if the importer object has already + agreed that it can handle the relevant path item, and they should only + return a subpath if the module __path__ does not already contain an + equivalent subpath. For an example namespace handler, see + ``pkg_resources.file_ns_handler``. + """ + _namespace_handlers[importer_type] = namespace_handler + + +def _handle_ns(packageName, path_item): + """Ensure that named package includes a subpath of path_item (if needed)""" + + importer = get_importer(path_item) + if importer is None: + return None + + # capture warnings due to #1111 + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + loader = importer.find_module(packageName) + + if loader is None: + return None + module = sys.modules.get(packageName) + if module is None: + module = sys.modules[packageName] = types.ModuleType(packageName) + module.__path__ = [] + _set_parent_ns(packageName) + elif not hasattr(module, '__path__'): + raise TypeError("Not a package:", packageName) + handler = _find_adapter(_namespace_handlers, importer) + subpath = handler(importer, path_item, packageName, module) + if subpath is not None: + path = module.__path__ + path.append(subpath) + loader.load_module(packageName) + _rebuild_mod_path(path, packageName, module) + return subpath + + +def _rebuild_mod_path(orig_path, package_name, module): + """ + Rebuild module.__path__ ensuring that all entries are ordered + corresponding to their sys.path order + """ + sys_path = [_normalize_cached(p) for p in sys.path] + + def safe_sys_path_index(entry): + """ + Workaround for #520 and #513. + """ + try: + return sys_path.index(entry) + except ValueError: + return float('inf') + + def position_in_sys_path(path): + """ + Return the ordinal of the path based on its position in sys.path + """ + path_parts = path.split(os.sep) + module_parts = package_name.count('.') + 1 + parts = path_parts[:-module_parts] + return safe_sys_path_index(_normalize_cached(os.sep.join(parts))) + + new_path = sorted(orig_path, key=position_in_sys_path) + new_path = [_normalize_cached(p) for p in new_path] + + if isinstance(module.__path__, list): + module.__path__[:] = new_path + else: + module.__path__ = new_path + + +def declare_namespace(packageName): + """Declare that package 'packageName' is a namespace package""" + + _imp.acquire_lock() + try: + if packageName in _namespace_packages: + return + + path = sys.path + parent, _, _ = packageName.rpartition('.') + + if parent: + declare_namespace(parent) + if parent not in _namespace_packages: + __import__(parent) + try: + path = sys.modules[parent].__path__ + except AttributeError: + raise TypeError("Not a package:", parent) + + # Track what packages are namespaces, so when new path items are added, + # they can be updated + _namespace_packages.setdefault(parent or None, []).append(packageName) + _namespace_packages.setdefault(packageName, []) + + for path_item in path: + # Ensure all the parent's path items are reflected in the child, + # if they apply + _handle_ns(packageName, path_item) + + finally: + _imp.release_lock() + + +def fixup_namespace_packages(path_item, parent=None): + """Ensure that previously-declared namespace packages include path_item""" + _imp.acquire_lock() + try: + for package in _namespace_packages.get(parent, ()): + subpath = _handle_ns(package, path_item) + if subpath: + fixup_namespace_packages(subpath, package) + finally: + _imp.release_lock() + + +def file_ns_handler(importer, path_item, packageName, module): + """Compute an ns-package subpath for a filesystem or zipfile importer""" + + subpath = os.path.join(path_item, packageName.split('.')[-1]) + normalized = _normalize_cached(subpath) + for item in module.__path__: + if _normalize_cached(item) == normalized: + break + else: + # Only return the path if it's not already there + return subpath + + +register_namespace_handler(pkgutil.ImpImporter, file_ns_handler) +register_namespace_handler(zipimport.zipimporter, file_ns_handler) + +if hasattr(importlib_machinery, 'FileFinder'): + register_namespace_handler(importlib_machinery.FileFinder, file_ns_handler) + + +def null_ns_handler(importer, path_item, packageName, module): + return None + + +register_namespace_handler(object, null_ns_handler) + + +def normalize_path(filename): + """Normalize a file/dir name for comparison purposes""" + return os.path.normcase(os.path.realpath(os.path.normpath(_cygwin_patch(filename)))) + + +def _cygwin_patch(filename): # pragma: nocover + """ + Contrary to POSIX 2008, on Cygwin, getcwd (3) contains + symlink components. Using + os.path.abspath() works around this limitation. A fix in os.getcwd() + would probably better, in Cygwin even more so, except + that this seems to be by design... + """ + return os.path.abspath(filename) if sys.platform == 'cygwin' else filename + + +def _normalize_cached(filename, _cache={}): + try: + return _cache[filename] + except KeyError: + _cache[filename] = result = normalize_path(filename) + return result + + +def _is_egg_path(path): + """ + Determine if given path appears to be an egg. + """ + return path.lower().endswith('.egg') + + +def _is_unpacked_egg(path): + """ + Determine if given path appears to be an unpacked egg. + """ + return ( + _is_egg_path(path) and + os.path.isfile(os.path.join(path, 'EGG-INFO', 'PKG-INFO')) + ) + + +def _set_parent_ns(packageName): + parts = packageName.split('.') + name = parts.pop() + if parts: + parent = '.'.join(parts) + setattr(sys.modules[parent], name, sys.modules[packageName]) + + +def yield_lines(strs): + """Yield non-empty/non-comment lines of a string or sequence""" + if isinstance(strs, six.string_types): + for s in strs.splitlines(): + s = s.strip() + # skip blank lines/comments + if s and not s.startswith('#'): + yield s + else: + for ss in strs: + for s in yield_lines(ss): + yield s + + +MODULE = re.compile(r"\w+(\.\w+)*$").match +EGG_NAME = re.compile( + r""" + (?P[^-]+) ( + -(?P[^-]+) ( + -py(?P[^-]+) ( + -(?P.+) + )? + )? + )? + """, + re.VERBOSE | re.IGNORECASE, +).match + + +class EntryPoint: + """Object representing an advertised importable object""" + + def __init__(self, name, module_name, attrs=(), extras=(), dist=None): + if not MODULE(module_name): + raise ValueError("Invalid module name", module_name) + self.name = name + self.module_name = module_name + self.attrs = tuple(attrs) + self.extras = tuple(extras) + self.dist = dist + + def __str__(self): + s = "%s = %s" % (self.name, self.module_name) + if self.attrs: + s += ':' + '.'.join(self.attrs) + if self.extras: + s += ' [%s]' % ','.join(self.extras) + return s + + def __repr__(self): + return "EntryPoint.parse(%r)" % str(self) + + def load(self, require=True, *args, **kwargs): + """ + Require packages for this EntryPoint, then resolve it. + """ + if not require or args or kwargs: + warnings.warn( + "Parameters to load are deprecated. Call .resolve and " + ".require separately.", + PkgResourcesDeprecationWarning, + stacklevel=2, + ) + if require: + self.require(*args, **kwargs) + return self.resolve() + + def resolve(self): + """ + Resolve the entry point from its module and attrs. + """ + module = __import__(self.module_name, fromlist=['__name__'], level=0) + try: + return functools.reduce(getattr, self.attrs, module) + except AttributeError as exc: + raise ImportError(str(exc)) + + def require(self, env=None, installer=None): + if self.extras and not self.dist: + raise UnknownExtra("Can't require() without a distribution", self) + + # Get the requirements for this entry point with all its extras and + # then resolve them. We have to pass `extras` along when resolving so + # that the working set knows what extras we want. Otherwise, for + # dist-info distributions, the working set will assume that the + # requirements for that extra are purely optional and skip over them. + reqs = self.dist.requires(self.extras) + items = working_set.resolve(reqs, env, installer, extras=self.extras) + list(map(working_set.add, items)) + + pattern = re.compile( + r'\s*' + r'(?P.+?)\s*' + r'=\s*' + r'(?P[\w.]+)\s*' + r'(:\s*(?P[\w.]+))?\s*' + r'(?P\[.*\])?\s*$' + ) + + @classmethod + def parse(cls, src, dist=None): + """Parse a single entry point from string `src` + + Entry point syntax follows the form:: + + name = some.module:some.attr [extra1, extra2] + + The entry name and module name are required, but the ``:attrs`` and + ``[extras]`` parts are optional + """ + m = cls.pattern.match(src) + if not m: + msg = "EntryPoint must be in 'name=module:attrs [extras]' format" + raise ValueError(msg, src) + res = m.groupdict() + extras = cls._parse_extras(res['extras']) + attrs = res['attr'].split('.') if res['attr'] else () + return cls(res['name'], res['module'], attrs, extras, dist) + + @classmethod + def _parse_extras(cls, extras_spec): + if not extras_spec: + return () + req = Requirement.parse('x' + extras_spec) + if req.specs: + raise ValueError() + return req.extras + + @classmethod + def parse_group(cls, group, lines, dist=None): + """Parse an entry point group""" + if not MODULE(group): + raise ValueError("Invalid group name", group) + this = {} + for line in yield_lines(lines): + ep = cls.parse(line, dist) + if ep.name in this: + raise ValueError("Duplicate entry point", group, ep.name) + this[ep.name] = ep + return this + + @classmethod + def parse_map(cls, data, dist=None): + """Parse a map of entry point groups""" + if isinstance(data, dict): + data = data.items() + else: + data = split_sections(data) + maps = {} + for group, lines in data: + if group is None: + if not lines: + continue + raise ValueError("Entry points must be listed in groups") + group = group.strip() + if group in maps: + raise ValueError("Duplicate group name", group) + maps[group] = cls.parse_group(group, lines, dist) + return maps + + +def _remove_md5_fragment(location): + if not location: + return '' + parsed = urllib.parse.urlparse(location) + if parsed[-1].startswith('md5='): + return urllib.parse.urlunparse(parsed[:-1] + ('',)) + return location + + +def _version_from_file(lines): + """ + Given an iterable of lines from a Metadata file, return + the value of the Version field, if present, or None otherwise. + """ + def is_version_line(line): + return line.lower().startswith('version:') + version_lines = filter(is_version_line, lines) + line = next(iter(version_lines), '') + _, _, value = line.partition(':') + return safe_version(value.strip()) or None + + +class Distribution: + """Wrap an actual or potential sys.path entry w/metadata""" + PKG_INFO = 'PKG-INFO' + + def __init__( + self, location=None, metadata=None, project_name=None, + version=None, py_version=PY_MAJOR, platform=None, + precedence=EGG_DIST): + self.project_name = safe_name(project_name or 'Unknown') + if version is not None: + self._version = safe_version(version) + self.py_version = py_version + self.platform = platform + self.location = location + self.precedence = precedence + self._provider = metadata or empty_provider + + @classmethod + def from_location(cls, location, basename, metadata=None, **kw): + project_name, version, py_version, platform = [None] * 4 + basename, ext = os.path.splitext(basename) + if ext.lower() in _distributionImpl: + cls = _distributionImpl[ext.lower()] + + match = EGG_NAME(basename) + if match: + project_name, version, py_version, platform = match.group( + 'name', 'ver', 'pyver', 'plat' + ) + return cls( + location, metadata, project_name=project_name, version=version, + py_version=py_version, platform=platform, **kw + )._reload_version() + + def _reload_version(self): + return self + + @property + def hashcmp(self): + return ( + self.parsed_version, + self.precedence, + self.key, + _remove_md5_fragment(self.location), + self.py_version or '', + self.platform or '', + ) + + def __hash__(self): + return hash(self.hashcmp) + + def __lt__(self, other): + return self.hashcmp < other.hashcmp + + def __le__(self, other): + return self.hashcmp <= other.hashcmp + + def __gt__(self, other): + return self.hashcmp > other.hashcmp + + def __ge__(self, other): + return self.hashcmp >= other.hashcmp + + def __eq__(self, other): + if not isinstance(other, self.__class__): + # It's not a Distribution, so they are not equal + return False + return self.hashcmp == other.hashcmp + + def __ne__(self, other): + return not self == other + + # These properties have to be lazy so that we don't have to load any + # metadata until/unless it's actually needed. (i.e., some distributions + # may not know their name or version without loading PKG-INFO) + + @property + def key(self): + try: + return self._key + except AttributeError: + self._key = key = self.project_name.lower() + return key + + @property + def parsed_version(self): + if not hasattr(self, "_parsed_version"): + self._parsed_version = parse_version(self.version) + + return self._parsed_version + + def _warn_legacy_version(self): + LV = packaging.version.LegacyVersion + is_legacy = isinstance(self._parsed_version, LV) + if not is_legacy: + return + + # While an empty version is technically a legacy version and + # is not a valid PEP 440 version, it's also unlikely to + # actually come from someone and instead it is more likely that + # it comes from setuptools attempting to parse a filename and + # including it in the list. So for that we'll gate this warning + # on if the version is anything at all or not. + if not self.version: + return + + tmpl = textwrap.dedent(""" + '{project_name} ({version})' is being parsed as a legacy, + non PEP 440, + version. You may find odd behavior and sort order. + In particular it will be sorted as less than 0.0. It + is recommended to migrate to PEP 440 compatible + versions. + """).strip().replace('\n', ' ') + + warnings.warn(tmpl.format(**vars(self)), PEP440Warning) + + @property + def version(self): + try: + return self._version + except AttributeError: + version = self._get_version() + if version is None: + path = self._get_metadata_path_for_display(self.PKG_INFO) + msg = ( + "Missing 'Version:' header and/or {} file at path: {}" + ).format(self.PKG_INFO, path) + raise ValueError(msg, self) + + return version + + @property + def _dep_map(self): + """ + A map of extra to its list of (direct) requirements + for this distribution, including the null extra. + """ + try: + return self.__dep_map + except AttributeError: + self.__dep_map = self._filter_extras(self._build_dep_map()) + return self.__dep_map + + @staticmethod + def _filter_extras(dm): + """ + Given a mapping of extras to dependencies, strip off + environment markers and filter out any dependencies + not matching the markers. + """ + for extra in list(filter(None, dm)): + new_extra = extra + reqs = dm.pop(extra) + new_extra, _, marker = extra.partition(':') + fails_marker = marker and ( + invalid_marker(marker) + or not evaluate_marker(marker) + ) + if fails_marker: + reqs = [] + new_extra = safe_extra(new_extra) or None + + dm.setdefault(new_extra, []).extend(reqs) + return dm + + def _build_dep_map(self): + dm = {} + for name in 'requires.txt', 'depends.txt': + for extra, reqs in split_sections(self._get_metadata(name)): + dm.setdefault(extra, []).extend(parse_requirements(reqs)) + return dm + + def requires(self, extras=()): + """List of Requirements needed for this distro if `extras` are used""" + dm = self._dep_map + deps = [] + deps.extend(dm.get(None, ())) + for ext in extras: + try: + deps.extend(dm[safe_extra(ext)]) + except KeyError: + raise UnknownExtra( + "%s has no such extra feature %r" % (self, ext) + ) + return deps + + def _get_metadata_path_for_display(self, name): + """ + Return the path to the given metadata file, if available. + """ + try: + # We need to access _get_metadata_path() on the provider object + # directly rather than through this class's __getattr__() + # since _get_metadata_path() is marked private. + path = self._provider._get_metadata_path(name) + + # Handle exceptions e.g. in case the distribution's metadata + # provider doesn't support _get_metadata_path(). + except Exception: + return '[could not detect]' + + return path + + def _get_metadata(self, name): + if self.has_metadata(name): + for line in self.get_metadata_lines(name): + yield line + + def _get_version(self): + lines = self._get_metadata(self.PKG_INFO) + version = _version_from_file(lines) + + return version + + def activate(self, path=None, replace=False): + """Ensure distribution is importable on `path` (default=sys.path)""" + if path is None: + path = sys.path + self.insert_on(path, replace=replace) + if path is sys.path: + fixup_namespace_packages(self.location) + for pkg in self._get_metadata('namespace_packages.txt'): + if pkg in sys.modules: + declare_namespace(pkg) + + def egg_name(self): + """Return what this distribution's standard .egg filename should be""" + filename = "%s-%s-py%s" % ( + to_filename(self.project_name), to_filename(self.version), + self.py_version or PY_MAJOR + ) + + if self.platform: + filename += '-' + self.platform + return filename + + def __repr__(self): + if self.location: + return "%s (%s)" % (self, self.location) + else: + return str(self) + + def __str__(self): + try: + version = getattr(self, 'version', None) + except ValueError: + version = None + version = version or "[unknown version]" + return "%s %s" % (self.project_name, version) + + def __getattr__(self, attr): + """Delegate all unrecognized public attributes to .metadata provider""" + if attr.startswith('_'): + raise AttributeError(attr) + return getattr(self._provider, attr) + + def __dir__(self): + return list( + set(super(Distribution, self).__dir__()) + | set( + attr for attr in self._provider.__dir__() + if not attr.startswith('_') + ) + ) + + if not hasattr(object, '__dir__'): + # python 2.7 not supported + del __dir__ + + @classmethod + def from_filename(cls, filename, metadata=None, **kw): + return cls.from_location( + _normalize_cached(filename), os.path.basename(filename), metadata, + **kw + ) + + def as_requirement(self): + """Return a ``Requirement`` that matches this distribution exactly""" + if isinstance(self.parsed_version, packaging.version.Version): + spec = "%s==%s" % (self.project_name, self.parsed_version) + else: + spec = "%s===%s" % (self.project_name, self.parsed_version) + + return Requirement.parse(spec) + + def load_entry_point(self, group, name): + """Return the `name` entry point of `group` or raise ImportError""" + ep = self.get_entry_info(group, name) + if ep is None: + raise ImportError("Entry point %r not found" % ((group, name),)) + return ep.load() + + def get_entry_map(self, group=None): + """Return the entry point map for `group`, or the full entry map""" + try: + ep_map = self._ep_map + except AttributeError: + ep_map = self._ep_map = EntryPoint.parse_map( + self._get_metadata('entry_points.txt'), self + ) + if group is not None: + return ep_map.get(group, {}) + return ep_map + + def get_entry_info(self, group, name): + """Return the EntryPoint object for `group`+`name`, or ``None``""" + return self.get_entry_map(group).get(name) + + def insert_on(self, path, loc=None, replace=False): + """Ensure self.location is on path + + If replace=False (default): + - If location is already in path anywhere, do nothing. + - Else: + - If it's an egg and its parent directory is on path, + insert just ahead of the parent. + - Else: add to the end of path. + If replace=True: + - If location is already on path anywhere (not eggs) + or higher priority than its parent (eggs) + do nothing. + - Else: + - If it's an egg and its parent directory is on path, + insert just ahead of the parent, + removing any lower-priority entries. + - Else: add it to the front of path. + """ + + loc = loc or self.location + if not loc: + return + + nloc = _normalize_cached(loc) + bdir = os.path.dirname(nloc) + npath = [(p and _normalize_cached(p) or p) for p in path] + + for p, item in enumerate(npath): + if item == nloc: + if replace: + break + else: + # don't modify path (even removing duplicates) if + # found and not replace + return + elif item == bdir and self.precedence == EGG_DIST: + # if it's an .egg, give it precedence over its directory + # UNLESS it's already been added to sys.path and replace=False + if (not replace) and nloc in npath[p:]: + return + if path is sys.path: + self.check_version_conflict() + path.insert(p, loc) + npath.insert(p, nloc) + break + else: + if path is sys.path: + self.check_version_conflict() + if replace: + path.insert(0, loc) + else: + path.append(loc) + return + + # p is the spot where we found or inserted loc; now remove duplicates + while True: + try: + np = npath.index(nloc, p + 1) + except ValueError: + break + else: + del npath[np], path[np] + # ha! + p = np + + return + + def check_version_conflict(self): + if self.key == 'setuptools': + # ignore the inevitable setuptools self-conflicts :( + return + + nsp = dict.fromkeys(self._get_metadata('namespace_packages.txt')) + loc = normalize_path(self.location) + for modname in self._get_metadata('top_level.txt'): + if (modname not in sys.modules or modname in nsp + or modname in _namespace_packages): + continue + if modname in ('pkg_resources', 'setuptools', 'site'): + continue + fn = getattr(sys.modules[modname], '__file__', None) + if fn and (normalize_path(fn).startswith(loc) or + fn.startswith(self.location)): + continue + issue_warning( + "Module %s was already imported from %s, but %s is being added" + " to sys.path" % (modname, fn, self.location), + ) + + def has_version(self): + try: + self.version + except ValueError: + issue_warning("Unbuilt egg for " + repr(self)) + return False + return True + + def clone(self, **kw): + """Copy this distribution, substituting in any changed keyword args""" + names = 'project_name version py_version platform location precedence' + for attr in names.split(): + kw.setdefault(attr, getattr(self, attr, None)) + kw.setdefault('metadata', self._provider) + return self.__class__(**kw) + + @property + def extras(self): + return [dep for dep in self._dep_map if dep] + + +class EggInfoDistribution(Distribution): + def _reload_version(self): + """ + Packages installed by distutils (e.g. numpy or scipy), + which uses an old safe_version, and so + their version numbers can get mangled when + converted to filenames (e.g., 1.11.0.dev0+2329eae to + 1.11.0.dev0_2329eae). These distributions will not be + parsed properly + downstream by Distribution and safe_version, so + take an extra step and try to get the version number from + the metadata file itself instead of the filename. + """ + md_version = self._get_version() + if md_version: + self._version = md_version + return self + + +class DistInfoDistribution(Distribution): + """ + Wrap an actual or potential sys.path entry + w/metadata, .dist-info style. + """ + PKG_INFO = 'METADATA' + EQEQ = re.compile(r"([\(,])\s*(\d.*?)\s*([,\)])") + + @property + def _parsed_pkg_info(self): + """Parse and cache metadata""" + try: + return self._pkg_info + except AttributeError: + metadata = self.get_metadata(self.PKG_INFO) + self._pkg_info = email.parser.Parser().parsestr(metadata) + return self._pkg_info + + @property + def _dep_map(self): + try: + return self.__dep_map + except AttributeError: + self.__dep_map = self._compute_dependencies() + return self.__dep_map + + def _compute_dependencies(self): + """Recompute this distribution's dependencies.""" + dm = self.__dep_map = {None: []} + + reqs = [] + # Including any condition expressions + for req in self._parsed_pkg_info.get_all('Requires-Dist') or []: + reqs.extend(parse_requirements(req)) + + def reqs_for_extra(extra): + for req in reqs: + if not req.marker or req.marker.evaluate({'extra': extra}): + yield req + + common = frozenset(reqs_for_extra(None)) + dm[None].extend(common) + + for extra in self._parsed_pkg_info.get_all('Provides-Extra') or []: + s_extra = safe_extra(extra.strip()) + dm[s_extra] = list(frozenset(reqs_for_extra(extra)) - common) + + return dm + + +_distributionImpl = { + '.egg': Distribution, + '.egg-info': EggInfoDistribution, + '.dist-info': DistInfoDistribution, +} + + +def issue_warning(*args, **kw): + level = 1 + g = globals() + try: + # find the first stack frame that is *not* code in + # the pkg_resources module, to use for the warning + while sys._getframe(level).f_globals is g: + level += 1 + except ValueError: + pass + warnings.warn(stacklevel=level + 1, *args, **kw) + + +class RequirementParseError(ValueError): + def __str__(self): + return ' '.join(self.args) + + +def parse_requirements(strs): + """Yield ``Requirement`` objects for each specification in `strs` + + `strs` must be a string, or a (possibly-nested) iterable thereof. + """ + # create a steppable iterator, so we can handle \-continuations + lines = iter(yield_lines(strs)) + + for line in lines: + # Drop comments -- a hash without a space may be in a URL. + if ' #' in line: + line = line[:line.find(' #')] + # If there is a line continuation, drop it, and append the next line. + if line.endswith('\\'): + line = line[:-2].strip() + try: + line += next(lines) + except StopIteration: + return + yield Requirement(line) + + +class Requirement(packaging.requirements.Requirement): + def __init__(self, requirement_string): + """DO NOT CALL THIS UNDOCUMENTED METHOD; use Requirement.parse()!""" + try: + super(Requirement, self).__init__(requirement_string) + except packaging.requirements.InvalidRequirement as e: + raise RequirementParseError(str(e)) + self.unsafe_name = self.name + project_name = safe_name(self.name) + self.project_name, self.key = project_name, project_name.lower() + self.specs = [ + (spec.operator, spec.version) for spec in self.specifier] + self.extras = tuple(map(safe_extra, self.extras)) + self.hashCmp = ( + self.key, + self.url, + self.specifier, + frozenset(self.extras), + str(self.marker) if self.marker else None, + ) + self.__hash = hash(self.hashCmp) + + def __eq__(self, other): + return ( + isinstance(other, Requirement) and + self.hashCmp == other.hashCmp + ) + + def __ne__(self, other): + return not self == other + + def __contains__(self, item): + if isinstance(item, Distribution): + if item.key != self.key: + return False + + item = item.version + + # Allow prereleases always in order to match the previous behavior of + # this method. In the future this should be smarter and follow PEP 440 + # more accurately. + return self.specifier.contains(item, prereleases=True) + + def __hash__(self): + return self.__hash + + def __repr__(self): + return "Requirement.parse(%r)" % str(self) + + @staticmethod + def parse(s): + req, = parse_requirements(s) + return req + + +def _always_object(classes): + """ + Ensure object appears in the mro even + for old-style classes. + """ + if object not in classes: + return classes + (object,) + return classes + + +def _find_adapter(registry, ob): + """Return an adapter factory for `ob` from `registry`""" + types = _always_object(inspect.getmro(getattr(ob, '__class__', type(ob)))) + for t in types: + if t in registry: + return registry[t] + + +def ensure_directory(path): + """Ensure that the parent directory of `path` exists""" + dirname = os.path.dirname(path) + py31compat.makedirs(dirname, exist_ok=True) + + +def _bypass_ensure_directory(path): + """Sandbox-bypassing version of ensure_directory()""" + if not WRITE_SUPPORT: + raise IOError('"os.mkdir" not supported on this platform.') + dirname, filename = split(path) + if dirname and filename and not isdir(dirname): + _bypass_ensure_directory(dirname) + try: + mkdir(dirname, 0o755) + except FileExistsError: + pass + + +def split_sections(s): + """Split a string or iterable thereof into (section, content) pairs + + Each ``section`` is a stripped version of the section header ("[section]") + and each ``content`` is a list of stripped lines excluding blank lines and + comment-only lines. If there are any such lines before the first section + header, they're returned in a first ``section`` of ``None``. + """ + section = None + content = [] + for line in yield_lines(s): + if line.startswith("["): + if line.endswith("]"): + if section or content: + yield section, content + section = line[1:-1].strip() + content = [] + else: + raise ValueError("Invalid section heading", line) + else: + content.append(line) + + # wrap up last segment + yield section, content + + +def _mkstemp(*args, **kw): + old_open = os.open + try: + # temporarily bypass sandboxing + os.open = os_open + return tempfile.mkstemp(*args, **kw) + finally: + # and then put it back + os.open = old_open + + +# Silence the PEP440Warning by default, so that end users don't get hit by it +# randomly just because they use pkg_resources. We want to append the rule +# because we want earlier uses of filterwarnings to take precedence over this +# one. +warnings.filterwarnings("ignore", category=PEP440Warning, append=True) + + +# from jaraco.functools 1.3 +def _call_aside(f, *args, **kwargs): + f(*args, **kwargs) + return f + + +@_call_aside +def _initialize(g=globals()): + "Set up global resource manager (deliberately not state-saved)" + manager = ResourceManager() + g['_manager'] = manager + g.update( + (name, getattr(manager, name)) + for name in dir(manager) + if not name.startswith('_') + ) + + +@_call_aside +def _initialize_master_working_set(): + """ + Prepare the master working set and make the ``require()`` + API available. + + This function has explicit effects on the global state + of pkg_resources. It is intended to be invoked once at + the initialization of this module. + + Invocation by other packages is unsupported and done + at their own risk. + """ + working_set = WorkingSet._build_master() + _declare_state('object', working_set=working_set) + + require = working_set.require + iter_entry_points = working_set.iter_entry_points + add_activation_listener = working_set.subscribe + run_script = working_set.run_script + # backward compatibility + run_main = run_script + # Activate all distributions already on sys.path with replace=False and + # ensure that all distributions added to the working set in the future + # (e.g. by calling ``require()``) will get activated as well, + # with higher priority (replace=True). + tuple( + dist.activate(replace=False) + for dist in working_set + ) + add_activation_listener( + lambda dist: dist.activate(replace=True), + existing=False, + ) + working_set.entries = [] + # match order + list(map(working_set.add_entry, sys.path)) + globals().update(locals()) + +class PkgResourcesDeprecationWarning(Warning): + """ + Base class for warning about deprecations in ``pkg_resources`` + + This class is not derived from ``DeprecationWarning``, and as such is + visible by default. + """ diff --git a/env/lib/python3.8/site-packages/pkg_resources/_vendor/__init__.py b/env/lib/python3.8/site-packages/pkg_resources/_vendor/__init__.py new file mode 100644 index 000000000..e69de29bb diff --git a/env/lib/python3.8/site-packages/pkg_resources/_vendor/appdirs.py b/env/lib/python3.8/site-packages/pkg_resources/_vendor/appdirs.py new file mode 100644 index 000000000..ae67001af --- /dev/null +++ b/env/lib/python3.8/site-packages/pkg_resources/_vendor/appdirs.py @@ -0,0 +1,608 @@ +#!/usr/bin/env python +# -*- coding: utf-8 -*- +# Copyright (c) 2005-2010 ActiveState Software Inc. +# Copyright (c) 2013 Eddy Petrișor + +"""Utilities for determining application-specific dirs. + +See for details and usage. +""" +# Dev Notes: +# - MSDN on where to store app data files: +# http://support.microsoft.com/default.aspx?scid=kb;en-us;310294#XSLTH3194121123120121120120 +# - Mac OS X: http://developer.apple.com/documentation/MacOSX/Conceptual/BPFileSystem/index.html +# - XDG spec for Un*x: http://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html + +__version_info__ = (1, 4, 3) +__version__ = '.'.join(map(str, __version_info__)) + + +import sys +import os + +PY3 = sys.version_info[0] == 3 + +if PY3: + unicode = str + +if sys.platform.startswith('java'): + import platform + os_name = platform.java_ver()[3][0] + if os_name.startswith('Windows'): # "Windows XP", "Windows 7", etc. + system = 'win32' + elif os_name.startswith('Mac'): # "Mac OS X", etc. + system = 'darwin' + else: # "Linux", "SunOS", "FreeBSD", etc. + # Setting this to "linux2" is not ideal, but only Windows or Mac + # are actually checked for and the rest of the module expects + # *sys.platform* style strings. + system = 'linux2' +else: + system = sys.platform + + + +def user_data_dir(appname=None, appauthor=None, version=None, roaming=False): + r"""Return full path to the user-specific data dir for this application. + + "appname" is the name of application. + If None, just the system directory is returned. + "appauthor" (only used on Windows) is the name of the + appauthor or distributing body for this application. Typically + it is the owning company name. This falls back to appname. You may + pass False to disable it. + "version" is an optional version path element to append to the + path. You might want to use this if you want multiple versions + of your app to be able to run independently. If used, this + would typically be ".". + Only applied when appname is present. + "roaming" (boolean, default False) can be set True to use the Windows + roaming appdata directory. That means that for users on a Windows + network setup for roaming profiles, this user data will be + sync'd on login. See + + for a discussion of issues. + + Typical user data directories are: + Mac OS X: ~/Library/Application Support/ + Unix: ~/.local/share/ # or in $XDG_DATA_HOME, if defined + Win XP (not roaming): C:\Documents and Settings\\Application Data\\ + Win XP (roaming): C:\Documents and Settings\\Local Settings\Application Data\\ + Win 7 (not roaming): C:\Users\\AppData\Local\\ + Win 7 (roaming): C:\Users\\AppData\Roaming\\ + + For Unix, we follow the XDG spec and support $XDG_DATA_HOME. + That means, by default "~/.local/share/". + """ + if system == "win32": + if appauthor is None: + appauthor = appname + const = roaming and "CSIDL_APPDATA" or "CSIDL_LOCAL_APPDATA" + path = os.path.normpath(_get_win_folder(const)) + if appname: + if appauthor is not False: + path = os.path.join(path, appauthor, appname) + else: + path = os.path.join(path, appname) + elif system == 'darwin': + path = os.path.expanduser('~/Library/Application Support/') + if appname: + path = os.path.join(path, appname) + else: + path = os.getenv('XDG_DATA_HOME', os.path.expanduser("~/.local/share")) + if appname: + path = os.path.join(path, appname) + if appname and version: + path = os.path.join(path, version) + return path + + +def site_data_dir(appname=None, appauthor=None, version=None, multipath=False): + r"""Return full path to the user-shared data dir for this application. + + "appname" is the name of application. + If None, just the system directory is returned. + "appauthor" (only used on Windows) is the name of the + appauthor or distributing body for this application. Typically + it is the owning company name. This falls back to appname. You may + pass False to disable it. + "version" is an optional version path element to append to the + path. You might want to use this if you want multiple versions + of your app to be able to run independently. If used, this + would typically be ".". + Only applied when appname is present. + "multipath" is an optional parameter only applicable to *nix + which indicates that the entire list of data dirs should be + returned. By default, the first item from XDG_DATA_DIRS is + returned, or '/usr/local/share/', + if XDG_DATA_DIRS is not set + + Typical site data directories are: + Mac OS X: /Library/Application Support/ + Unix: /usr/local/share/ or /usr/share/ + Win XP: C:\Documents and Settings\All Users\Application Data\\ + Vista: (Fail! "C:\ProgramData" is a hidden *system* directory on Vista.) + Win 7: C:\ProgramData\\ # Hidden, but writeable on Win 7. + + For Unix, this is using the $XDG_DATA_DIRS[0] default. + + WARNING: Do not use this on Windows. See the Vista-Fail note above for why. + """ + if system == "win32": + if appauthor is None: + appauthor = appname + path = os.path.normpath(_get_win_folder("CSIDL_COMMON_APPDATA")) + if appname: + if appauthor is not False: + path = os.path.join(path, appauthor, appname) + else: + path = os.path.join(path, appname) + elif system == 'darwin': + path = os.path.expanduser('/Library/Application Support') + if appname: + path = os.path.join(path, appname) + else: + # XDG default for $XDG_DATA_DIRS + # only first, if multipath is False + path = os.getenv('XDG_DATA_DIRS', + os.pathsep.join(['/usr/local/share', '/usr/share'])) + pathlist = [os.path.expanduser(x.rstrip(os.sep)) for x in path.split(os.pathsep)] + if appname: + if version: + appname = os.path.join(appname, version) + pathlist = [os.sep.join([x, appname]) for x in pathlist] + + if multipath: + path = os.pathsep.join(pathlist) + else: + path = pathlist[0] + return path + + if appname and version: + path = os.path.join(path, version) + return path + + +def user_config_dir(appname=None, appauthor=None, version=None, roaming=False): + r"""Return full path to the user-specific config dir for this application. + + "appname" is the name of application. + If None, just the system directory is returned. + "appauthor" (only used on Windows) is the name of the + appauthor or distributing body for this application. Typically + it is the owning company name. This falls back to appname. You may + pass False to disable it. + "version" is an optional version path element to append to the + path. You might want to use this if you want multiple versions + of your app to be able to run independently. If used, this + would typically be ".". + Only applied when appname is present. + "roaming" (boolean, default False) can be set True to use the Windows + roaming appdata directory. That means that for users on a Windows + network setup for roaming profiles, this user data will be + sync'd on login. See + + for a discussion of issues. + + Typical user config directories are: + Mac OS X: same as user_data_dir + Unix: ~/.config/ # or in $XDG_CONFIG_HOME, if defined + Win *: same as user_data_dir + + For Unix, we follow the XDG spec and support $XDG_CONFIG_HOME. + That means, by default "~/.config/". + """ + if system in ["win32", "darwin"]: + path = user_data_dir(appname, appauthor, None, roaming) + else: + path = os.getenv('XDG_CONFIG_HOME', os.path.expanduser("~/.config")) + if appname: + path = os.path.join(path, appname) + if appname and version: + path = os.path.join(path, version) + return path + + +def site_config_dir(appname=None, appauthor=None, version=None, multipath=False): + r"""Return full path to the user-shared data dir for this application. + + "appname" is the name of application. + If None, just the system directory is returned. + "appauthor" (only used on Windows) is the name of the + appauthor or distributing body for this application. Typically + it is the owning company name. This falls back to appname. You may + pass False to disable it. + "version" is an optional version path element to append to the + path. You might want to use this if you want multiple versions + of your app to be able to run independently. If used, this + would typically be ".". + Only applied when appname is present. + "multipath" is an optional parameter only applicable to *nix + which indicates that the entire list of config dirs should be + returned. By default, the first item from XDG_CONFIG_DIRS is + returned, or '/etc/xdg/', if XDG_CONFIG_DIRS is not set + + Typical site config directories are: + Mac OS X: same as site_data_dir + Unix: /etc/xdg/ or $XDG_CONFIG_DIRS[i]/ for each value in + $XDG_CONFIG_DIRS + Win *: same as site_data_dir + Vista: (Fail! "C:\ProgramData" is a hidden *system* directory on Vista.) + + For Unix, this is using the $XDG_CONFIG_DIRS[0] default, if multipath=False + + WARNING: Do not use this on Windows. See the Vista-Fail note above for why. + """ + if system in ["win32", "darwin"]: + path = site_data_dir(appname, appauthor) + if appname and version: + path = os.path.join(path, version) + else: + # XDG default for $XDG_CONFIG_DIRS + # only first, if multipath is False + path = os.getenv('XDG_CONFIG_DIRS', '/etc/xdg') + pathlist = [os.path.expanduser(x.rstrip(os.sep)) for x in path.split(os.pathsep)] + if appname: + if version: + appname = os.path.join(appname, version) + pathlist = [os.sep.join([x, appname]) for x in pathlist] + + if multipath: + path = os.pathsep.join(pathlist) + else: + path = pathlist[0] + return path + + +def user_cache_dir(appname=None, appauthor=None, version=None, opinion=True): + r"""Return full path to the user-specific cache dir for this application. + + "appname" is the name of application. + If None, just the system directory is returned. + "appauthor" (only used on Windows) is the name of the + appauthor or distributing body for this application. Typically + it is the owning company name. This falls back to appname. You may + pass False to disable it. + "version" is an optional version path element to append to the + path. You might want to use this if you want multiple versions + of your app to be able to run independently. If used, this + would typically be ".". + Only applied when appname is present. + "opinion" (boolean) can be False to disable the appending of + "Cache" to the base app data dir for Windows. See + discussion below. + + Typical user cache directories are: + Mac OS X: ~/Library/Caches/ + Unix: ~/.cache/ (XDG default) + Win XP: C:\Documents and Settings\\Local Settings\Application Data\\\Cache + Vista: C:\Users\\AppData\Local\\\Cache + + On Windows the only suggestion in the MSDN docs is that local settings go in + the `CSIDL_LOCAL_APPDATA` directory. This is identical to the non-roaming + app data dir (the default returned by `user_data_dir` above). Apps typically + put cache data somewhere *under* the given dir here. Some examples: + ...\Mozilla\Firefox\Profiles\\Cache + ...\Acme\SuperApp\Cache\1.0 + OPINION: This function appends "Cache" to the `CSIDL_LOCAL_APPDATA` value. + This can be disabled with the `opinion=False` option. + """ + if system == "win32": + if appauthor is None: + appauthor = appname + path = os.path.normpath(_get_win_folder("CSIDL_LOCAL_APPDATA")) + if appname: + if appauthor is not False: + path = os.path.join(path, appauthor, appname) + else: + path = os.path.join(path, appname) + if opinion: + path = os.path.join(path, "Cache") + elif system == 'darwin': + path = os.path.expanduser('~/Library/Caches') + if appname: + path = os.path.join(path, appname) + else: + path = os.getenv('XDG_CACHE_HOME', os.path.expanduser('~/.cache')) + if appname: + path = os.path.join(path, appname) + if appname and version: + path = os.path.join(path, version) + return path + + +def user_state_dir(appname=None, appauthor=None, version=None, roaming=False): + r"""Return full path to the user-specific state dir for this application. + + "appname" is the name of application. + If None, just the system directory is returned. + "appauthor" (only used on Windows) is the name of the + appauthor or distributing body for this application. Typically + it is the owning company name. This falls back to appname. You may + pass False to disable it. + "version" is an optional version path element to append to the + path. You might want to use this if you want multiple versions + of your app to be able to run independently. If used, this + would typically be ".". + Only applied when appname is present. + "roaming" (boolean, default False) can be set True to use the Windows + roaming appdata directory. That means that for users on a Windows + network setup for roaming profiles, this user data will be + sync'd on login. See + + for a discussion of issues. + + Typical user state directories are: + Mac OS X: same as user_data_dir + Unix: ~/.local/state/ # or in $XDG_STATE_HOME, if defined + Win *: same as user_data_dir + + For Unix, we follow this Debian proposal + to extend the XDG spec and support $XDG_STATE_HOME. + + That means, by default "~/.local/state/". + """ + if system in ["win32", "darwin"]: + path = user_data_dir(appname, appauthor, None, roaming) + else: + path = os.getenv('XDG_STATE_HOME', os.path.expanduser("~/.local/state")) + if appname: + path = os.path.join(path, appname) + if appname and version: + path = os.path.join(path, version) + return path + + +def user_log_dir(appname=None, appauthor=None, version=None, opinion=True): + r"""Return full path to the user-specific log dir for this application. + + "appname" is the name of application. + If None, just the system directory is returned. + "appauthor" (only used on Windows) is the name of the + appauthor or distributing body for this application. Typically + it is the owning company name. This falls back to appname. You may + pass False to disable it. + "version" is an optional version path element to append to the + path. You might want to use this if you want multiple versions + of your app to be able to run independently. If used, this + would typically be ".". + Only applied when appname is present. + "opinion" (boolean) can be False to disable the appending of + "Logs" to the base app data dir for Windows, and "log" to the + base cache dir for Unix. See discussion below. + + Typical user log directories are: + Mac OS X: ~/Library/Logs/ + Unix: ~/.cache//log # or under $XDG_CACHE_HOME if defined + Win XP: C:\Documents and Settings\\Local Settings\Application Data\\\Logs + Vista: C:\Users\\AppData\Local\\\Logs + + On Windows the only suggestion in the MSDN docs is that local settings + go in the `CSIDL_LOCAL_APPDATA` directory. (Note: I'm interested in + examples of what some windows apps use for a logs dir.) + + OPINION: This function appends "Logs" to the `CSIDL_LOCAL_APPDATA` + value for Windows and appends "log" to the user cache dir for Unix. + This can be disabled with the `opinion=False` option. + """ + if system == "darwin": + path = os.path.join( + os.path.expanduser('~/Library/Logs'), + appname) + elif system == "win32": + path = user_data_dir(appname, appauthor, version) + version = False + if opinion: + path = os.path.join(path, "Logs") + else: + path = user_cache_dir(appname, appauthor, version) + version = False + if opinion: + path = os.path.join(path, "log") + if appname and version: + path = os.path.join(path, version) + return path + + +class AppDirs(object): + """Convenience wrapper for getting application dirs.""" + def __init__(self, appname=None, appauthor=None, version=None, + roaming=False, multipath=False): + self.appname = appname + self.appauthor = appauthor + self.version = version + self.roaming = roaming + self.multipath = multipath + + @property + def user_data_dir(self): + return user_data_dir(self.appname, self.appauthor, + version=self.version, roaming=self.roaming) + + @property + def site_data_dir(self): + return site_data_dir(self.appname, self.appauthor, + version=self.version, multipath=self.multipath) + + @property + def user_config_dir(self): + return user_config_dir(self.appname, self.appauthor, + version=self.version, roaming=self.roaming) + + @property + def site_config_dir(self): + return site_config_dir(self.appname, self.appauthor, + version=self.version, multipath=self.multipath) + + @property + def user_cache_dir(self): + return user_cache_dir(self.appname, self.appauthor, + version=self.version) + + @property + def user_state_dir(self): + return user_state_dir(self.appname, self.appauthor, + version=self.version) + + @property + def user_log_dir(self): + return user_log_dir(self.appname, self.appauthor, + version=self.version) + + +#---- internal support stuff + +def _get_win_folder_from_registry(csidl_name): + """This is a fallback technique at best. I'm not sure if using the + registry for this guarantees us the correct answer for all CSIDL_* + names. + """ + if PY3: + import winreg as _winreg + else: + import _winreg + + shell_folder_name = { + "CSIDL_APPDATA": "AppData", + "CSIDL_COMMON_APPDATA": "Common AppData", + "CSIDL_LOCAL_APPDATA": "Local AppData", + }[csidl_name] + + key = _winreg.OpenKey( + _winreg.HKEY_CURRENT_USER, + r"Software\Microsoft\Windows\CurrentVersion\Explorer\Shell Folders" + ) + dir, type = _winreg.QueryValueEx(key, shell_folder_name) + return dir + + +def _get_win_folder_with_pywin32(csidl_name): + from win32com.shell import shellcon, shell + dir = shell.SHGetFolderPath(0, getattr(shellcon, csidl_name), 0, 0) + # Try to make this a unicode path because SHGetFolderPath does + # not return unicode strings when there is unicode data in the + # path. + try: + dir = unicode(dir) + + # Downgrade to short path name if have highbit chars. See + # . + has_high_char = False + for c in dir: + if ord(c) > 255: + has_high_char = True + break + if has_high_char: + try: + import win32api + dir = win32api.GetShortPathName(dir) + except ImportError: + pass + except UnicodeError: + pass + return dir + + +def _get_win_folder_with_ctypes(csidl_name): + import ctypes + + csidl_const = { + "CSIDL_APPDATA": 26, + "CSIDL_COMMON_APPDATA": 35, + "CSIDL_LOCAL_APPDATA": 28, + }[csidl_name] + + buf = ctypes.create_unicode_buffer(1024) + ctypes.windll.shell32.SHGetFolderPathW(None, csidl_const, None, 0, buf) + + # Downgrade to short path name if have highbit chars. See + # . + has_high_char = False + for c in buf: + if ord(c) > 255: + has_high_char = True + break + if has_high_char: + buf2 = ctypes.create_unicode_buffer(1024) + if ctypes.windll.kernel32.GetShortPathNameW(buf.value, buf2, 1024): + buf = buf2 + + return buf.value + +def _get_win_folder_with_jna(csidl_name): + import array + from com.sun import jna + from com.sun.jna.platform import win32 + + buf_size = win32.WinDef.MAX_PATH * 2 + buf = array.zeros('c', buf_size) + shell = win32.Shell32.INSTANCE + shell.SHGetFolderPath(None, getattr(win32.ShlObj, csidl_name), None, win32.ShlObj.SHGFP_TYPE_CURRENT, buf) + dir = jna.Native.toString(buf.tostring()).rstrip("\0") + + # Downgrade to short path name if have highbit chars. See + # . + has_high_char = False + for c in dir: + if ord(c) > 255: + has_high_char = True + break + if has_high_char: + buf = array.zeros('c', buf_size) + kernel = win32.Kernel32.INSTANCE + if kernel.GetShortPathName(dir, buf, buf_size): + dir = jna.Native.toString(buf.tostring()).rstrip("\0") + + return dir + +if system == "win32": + try: + import win32com.shell + _get_win_folder = _get_win_folder_with_pywin32 + except ImportError: + try: + from ctypes import windll + _get_win_folder = _get_win_folder_with_ctypes + except ImportError: + try: + import com.sun.jna + _get_win_folder = _get_win_folder_with_jna + except ImportError: + _get_win_folder = _get_win_folder_from_registry + + +#---- self test code + +if __name__ == "__main__": + appname = "MyApp" + appauthor = "MyCompany" + + props = ("user_data_dir", + "user_config_dir", + "user_cache_dir", + "user_state_dir", + "user_log_dir", + "site_data_dir", + "site_config_dir") + + print("-- app dirs %s --" % __version__) + + print("-- app dirs (with optional 'version')") + dirs = AppDirs(appname, appauthor, version="1.0") + for prop in props: + print("%s: %s" % (prop, getattr(dirs, prop))) + + print("\n-- app dirs (without optional 'version')") + dirs = AppDirs(appname, appauthor) + for prop in props: + print("%s: %s" % (prop, getattr(dirs, prop))) + + print("\n-- app dirs (without optional 'appauthor')") + dirs = AppDirs(appname) + for prop in props: + print("%s: %s" % (prop, getattr(dirs, prop))) + + print("\n-- app dirs (with disabled 'appauthor')") + dirs = AppDirs(appname, appauthor=False) + for prop in props: + print("%s: %s" % (prop, getattr(dirs, prop))) diff --git a/env/lib/python3.8/site-packages/pkg_resources/_vendor/packaging/__about__.py b/env/lib/python3.8/site-packages/pkg_resources/_vendor/packaging/__about__.py new file mode 100644 index 000000000..95d330ef8 --- /dev/null +++ b/env/lib/python3.8/site-packages/pkg_resources/_vendor/packaging/__about__.py @@ -0,0 +1,21 @@ +# This file is dual licensed under the terms of the Apache License, Version +# 2.0, and the BSD License. See the LICENSE file in the root of this repository +# for complete details. +from __future__ import absolute_import, division, print_function + +__all__ = [ + "__title__", "__summary__", "__uri__", "__version__", "__author__", + "__email__", "__license__", "__copyright__", +] + +__title__ = "packaging" +__summary__ = "Core utilities for Python packages" +__uri__ = "https://github.com/pypa/packaging" + +__version__ = "16.8" + +__author__ = "Donald Stufft and individual contributors" +__email__ = "donald@stufft.io" + +__license__ = "BSD or Apache License, Version 2.0" +__copyright__ = "Copyright 2014-2016 %s" % __author__ diff --git a/env/lib/python3.8/site-packages/pkg_resources/_vendor/packaging/__init__.py b/env/lib/python3.8/site-packages/pkg_resources/_vendor/packaging/__init__.py new file mode 100644 index 000000000..5ee622020 --- /dev/null +++ b/env/lib/python3.8/site-packages/pkg_resources/_vendor/packaging/__init__.py @@ -0,0 +1,14 @@ +# This file is dual licensed under the terms of the Apache License, Version +# 2.0, and the BSD License. See the LICENSE file in the root of this repository +# for complete details. +from __future__ import absolute_import, division, print_function + +from .__about__ import ( + __author__, __copyright__, __email__, __license__, __summary__, __title__, + __uri__, __version__ +) + +__all__ = [ + "__title__", "__summary__", "__uri__", "__version__", "__author__", + "__email__", "__license__", "__copyright__", +] diff --git a/env/lib/python3.8/site-packages/pkg_resources/_vendor/packaging/_compat.py b/env/lib/python3.8/site-packages/pkg_resources/_vendor/packaging/_compat.py new file mode 100644 index 000000000..210bb80b7 --- /dev/null +++ b/env/lib/python3.8/site-packages/pkg_resources/_vendor/packaging/_compat.py @@ -0,0 +1,30 @@ +# This file is dual licensed under the terms of the Apache License, Version +# 2.0, and the BSD License. See the LICENSE file in the root of this repository +# for complete details. +from __future__ import absolute_import, division, print_function + +import sys + + +PY2 = sys.version_info[0] == 2 +PY3 = sys.version_info[0] == 3 + +# flake8: noqa + +if PY3: + string_types = str, +else: + string_types = basestring, + + +def with_metaclass(meta, *bases): + """ + Create a base class with a metaclass. + """ + # This requires a bit of explanation: the basic idea is to make a dummy + # metaclass for one level of class instantiation that replaces itself with + # the actual metaclass. + class metaclass(meta): + def __new__(cls, name, this_bases, d): + return meta(name, bases, d) + return type.__new__(metaclass, 'temporary_class', (), {}) diff --git a/env/lib/python3.8/site-packages/pkg_resources/_vendor/packaging/_structures.py b/env/lib/python3.8/site-packages/pkg_resources/_vendor/packaging/_structures.py new file mode 100644 index 000000000..ccc27861c --- /dev/null +++ b/env/lib/python3.8/site-packages/pkg_resources/_vendor/packaging/_structures.py @@ -0,0 +1,68 @@ +# This file is dual licensed under the terms of the Apache License, Version +# 2.0, and the BSD License. See the LICENSE file in the root of this repository +# for complete details. +from __future__ import absolute_import, division, print_function + + +class Infinity(object): + + def __repr__(self): + return "Infinity" + + def __hash__(self): + return hash(repr(self)) + + def __lt__(self, other): + return False + + def __le__(self, other): + return False + + def __eq__(self, other): + return isinstance(other, self.__class__) + + def __ne__(self, other): + return not isinstance(other, self.__class__) + + def __gt__(self, other): + return True + + def __ge__(self, other): + return True + + def __neg__(self): + return NegativeInfinity + +Infinity = Infinity() + + +class NegativeInfinity(object): + + def __repr__(self): + return "-Infinity" + + def __hash__(self): + return hash(repr(self)) + + def __lt__(self, other): + return True + + def __le__(self, other): + return True + + def __eq__(self, other): + return isinstance(other, self.__class__) + + def __ne__(self, other): + return not isinstance(other, self.__class__) + + def __gt__(self, other): + return False + + def __ge__(self, other): + return False + + def __neg__(self): + return Infinity + +NegativeInfinity = NegativeInfinity() diff --git a/env/lib/python3.8/site-packages/pkg_resources/_vendor/packaging/markers.py b/env/lib/python3.8/site-packages/pkg_resources/_vendor/packaging/markers.py new file mode 100644 index 000000000..892e578ed --- /dev/null +++ b/env/lib/python3.8/site-packages/pkg_resources/_vendor/packaging/markers.py @@ -0,0 +1,301 @@ +# This file is dual licensed under the terms of the Apache License, Version +# 2.0, and the BSD License. See the LICENSE file in the root of this repository +# for complete details. +from __future__ import absolute_import, division, print_function + +import operator +import os +import platform +import sys + +from pkg_resources.extern.pyparsing import ParseException, ParseResults, stringStart, stringEnd +from pkg_resources.extern.pyparsing import ZeroOrMore, Group, Forward, QuotedString +from pkg_resources.extern.pyparsing import Literal as L # noqa + +from ._compat import string_types +from .specifiers import Specifier, InvalidSpecifier + + +__all__ = [ + "InvalidMarker", "UndefinedComparison", "UndefinedEnvironmentName", + "Marker", "default_environment", +] + + +class InvalidMarker(ValueError): + """ + An invalid marker was found, users should refer to PEP 508. + """ + + +class UndefinedComparison(ValueError): + """ + An invalid operation was attempted on a value that doesn't support it. + """ + + +class UndefinedEnvironmentName(ValueError): + """ + A name was attempted to be used that does not exist inside of the + environment. + """ + + +class Node(object): + + def __init__(self, value): + self.value = value + + def __str__(self): + return str(self.value) + + def __repr__(self): + return "<{0}({1!r})>".format(self.__class__.__name__, str(self)) + + def serialize(self): + raise NotImplementedError + + +class Variable(Node): + + def serialize(self): + return str(self) + + +class Value(Node): + + def serialize(self): + return '"{0}"'.format(self) + + +class Op(Node): + + def serialize(self): + return str(self) + + +VARIABLE = ( + L("implementation_version") | + L("platform_python_implementation") | + L("implementation_name") | + L("python_full_version") | + L("platform_release") | + L("platform_version") | + L("platform_machine") | + L("platform_system") | + L("python_version") | + L("sys_platform") | + L("os_name") | + L("os.name") | # PEP-345 + L("sys.platform") | # PEP-345 + L("platform.version") | # PEP-345 + L("platform.machine") | # PEP-345 + L("platform.python_implementation") | # PEP-345 + L("python_implementation") | # undocumented setuptools legacy + L("extra") +) +ALIASES = { + 'os.name': 'os_name', + 'sys.platform': 'sys_platform', + 'platform.version': 'platform_version', + 'platform.machine': 'platform_machine', + 'platform.python_implementation': 'platform_python_implementation', + 'python_implementation': 'platform_python_implementation' +} +VARIABLE.setParseAction(lambda s, l, t: Variable(ALIASES.get(t[0], t[0]))) + +VERSION_CMP = ( + L("===") | + L("==") | + L(">=") | + L("<=") | + L("!=") | + L("~=") | + L(">") | + L("<") +) + +MARKER_OP = VERSION_CMP | L("not in") | L("in") +MARKER_OP.setParseAction(lambda s, l, t: Op(t[0])) + +MARKER_VALUE = QuotedString("'") | QuotedString('"') +MARKER_VALUE.setParseAction(lambda s, l, t: Value(t[0])) + +BOOLOP = L("and") | L("or") + +MARKER_VAR = VARIABLE | MARKER_VALUE + +MARKER_ITEM = Group(MARKER_VAR + MARKER_OP + MARKER_VAR) +MARKER_ITEM.setParseAction(lambda s, l, t: tuple(t[0])) + +LPAREN = L("(").suppress() +RPAREN = L(")").suppress() + +MARKER_EXPR = Forward() +MARKER_ATOM = MARKER_ITEM | Group(LPAREN + MARKER_EXPR + RPAREN) +MARKER_EXPR << MARKER_ATOM + ZeroOrMore(BOOLOP + MARKER_EXPR) + +MARKER = stringStart + MARKER_EXPR + stringEnd + + +def _coerce_parse_result(results): + if isinstance(results, ParseResults): + return [_coerce_parse_result(i) for i in results] + else: + return results + + +def _format_marker(marker, first=True): + assert isinstance(marker, (list, tuple, string_types)) + + # Sometimes we have a structure like [[...]] which is a single item list + # where the single item is itself it's own list. In that case we want skip + # the rest of this function so that we don't get extraneous () on the + # outside. + if (isinstance(marker, list) and len(marker) == 1 and + isinstance(marker[0], (list, tuple))): + return _format_marker(marker[0]) + + if isinstance(marker, list): + inner = (_format_marker(m, first=False) for m in marker) + if first: + return " ".join(inner) + else: + return "(" + " ".join(inner) + ")" + elif isinstance(marker, tuple): + return " ".join([m.serialize() for m in marker]) + else: + return marker + + +_operators = { + "in": lambda lhs, rhs: lhs in rhs, + "not in": lambda lhs, rhs: lhs not in rhs, + "<": operator.lt, + "<=": operator.le, + "==": operator.eq, + "!=": operator.ne, + ">=": operator.ge, + ">": operator.gt, +} + + +def _eval_op(lhs, op, rhs): + try: + spec = Specifier("".join([op.serialize(), rhs])) + except InvalidSpecifier: + pass + else: + return spec.contains(lhs) + + oper = _operators.get(op.serialize()) + if oper is None: + raise UndefinedComparison( + "Undefined {0!r} on {1!r} and {2!r}.".format(op, lhs, rhs) + ) + + return oper(lhs, rhs) + + +_undefined = object() + + +def _get_env(environment, name): + value = environment.get(name, _undefined) + + if value is _undefined: + raise UndefinedEnvironmentName( + "{0!r} does not exist in evaluation environment.".format(name) + ) + + return value + + +def _evaluate_markers(markers, environment): + groups = [[]] + + for marker in markers: + assert isinstance(marker, (list, tuple, string_types)) + + if isinstance(marker, list): + groups[-1].append(_evaluate_markers(marker, environment)) + elif isinstance(marker, tuple): + lhs, op, rhs = marker + + if isinstance(lhs, Variable): + lhs_value = _get_env(environment, lhs.value) + rhs_value = rhs.value + else: + lhs_value = lhs.value + rhs_value = _get_env(environment, rhs.value) + + groups[-1].append(_eval_op(lhs_value, op, rhs_value)) + else: + assert marker in ["and", "or"] + if marker == "or": + groups.append([]) + + return any(all(item) for item in groups) + + +def format_full_version(info): + version = '{0.major}.{0.minor}.{0.micro}'.format(info) + kind = info.releaselevel + if kind != 'final': + version += kind[0] + str(info.serial) + return version + + +def default_environment(): + if hasattr(sys, 'implementation'): + iver = format_full_version(sys.implementation.version) + implementation_name = sys.implementation.name + else: + iver = '0' + implementation_name = '' + + return { + "implementation_name": implementation_name, + "implementation_version": iver, + "os_name": os.name, + "platform_machine": platform.machine(), + "platform_release": platform.release(), + "platform_system": platform.system(), + "platform_version": platform.version(), + "python_full_version": platform.python_version(), + "platform_python_implementation": platform.python_implementation(), + "python_version": platform.python_version()[:3], + "sys_platform": sys.platform, + } + + +class Marker(object): + + def __init__(self, marker): + try: + self._markers = _coerce_parse_result(MARKER.parseString(marker)) + except ParseException as e: + err_str = "Invalid marker: {0!r}, parse error at {1!r}".format( + marker, marker[e.loc:e.loc + 8]) + raise InvalidMarker(err_str) + + def __str__(self): + return _format_marker(self._markers) + + def __repr__(self): + return "".format(str(self)) + + def evaluate(self, environment=None): + """Evaluate a marker. + + Return the boolean from evaluating the given marker against the + environment. environment is an optional argument to override all or + part of the determined environment. + + The environment is determined from the current Python process. + """ + current_environment = default_environment() + if environment is not None: + current_environment.update(environment) + + return _evaluate_markers(self._markers, current_environment) diff --git a/env/lib/python3.8/site-packages/pkg_resources/_vendor/packaging/requirements.py b/env/lib/python3.8/site-packages/pkg_resources/_vendor/packaging/requirements.py new file mode 100644 index 000000000..0c8c4a385 --- /dev/null +++ b/env/lib/python3.8/site-packages/pkg_resources/_vendor/packaging/requirements.py @@ -0,0 +1,127 @@ +# This file is dual licensed under the terms of the Apache License, Version +# 2.0, and the BSD License. See the LICENSE file in the root of this repository +# for complete details. +from __future__ import absolute_import, division, print_function + +import string +import re + +from pkg_resources.extern.pyparsing import stringStart, stringEnd, originalTextFor, ParseException +from pkg_resources.extern.pyparsing import ZeroOrMore, Word, Optional, Regex, Combine +from pkg_resources.extern.pyparsing import Literal as L # noqa +from pkg_resources.extern.six.moves.urllib import parse as urlparse + +from .markers import MARKER_EXPR, Marker +from .specifiers import LegacySpecifier, Specifier, SpecifierSet + + +class InvalidRequirement(ValueError): + """ + An invalid requirement was found, users should refer to PEP 508. + """ + + +ALPHANUM = Word(string.ascii_letters + string.digits) + +LBRACKET = L("[").suppress() +RBRACKET = L("]").suppress() +LPAREN = L("(").suppress() +RPAREN = L(")").suppress() +COMMA = L(",").suppress() +SEMICOLON = L(";").suppress() +AT = L("@").suppress() + +PUNCTUATION = Word("-_.") +IDENTIFIER_END = ALPHANUM | (ZeroOrMore(PUNCTUATION) + ALPHANUM) +IDENTIFIER = Combine(ALPHANUM + ZeroOrMore(IDENTIFIER_END)) + +NAME = IDENTIFIER("name") +EXTRA = IDENTIFIER + +URI = Regex(r'[^ ]+')("url") +URL = (AT + URI) + +EXTRAS_LIST = EXTRA + ZeroOrMore(COMMA + EXTRA) +EXTRAS = (LBRACKET + Optional(EXTRAS_LIST) + RBRACKET)("extras") + +VERSION_PEP440 = Regex(Specifier._regex_str, re.VERBOSE | re.IGNORECASE) +VERSION_LEGACY = Regex(LegacySpecifier._regex_str, re.VERBOSE | re.IGNORECASE) + +VERSION_ONE = VERSION_PEP440 ^ VERSION_LEGACY +VERSION_MANY = Combine(VERSION_ONE + ZeroOrMore(COMMA + VERSION_ONE), + joinString=",", adjacent=False)("_raw_spec") +_VERSION_SPEC = Optional(((LPAREN + VERSION_MANY + RPAREN) | VERSION_MANY)) +_VERSION_SPEC.setParseAction(lambda s, l, t: t._raw_spec or '') + +VERSION_SPEC = originalTextFor(_VERSION_SPEC)("specifier") +VERSION_SPEC.setParseAction(lambda s, l, t: t[1]) + +MARKER_EXPR = originalTextFor(MARKER_EXPR())("marker") +MARKER_EXPR.setParseAction( + lambda s, l, t: Marker(s[t._original_start:t._original_end]) +) +MARKER_SEPERATOR = SEMICOLON +MARKER = MARKER_SEPERATOR + MARKER_EXPR + +VERSION_AND_MARKER = VERSION_SPEC + Optional(MARKER) +URL_AND_MARKER = URL + Optional(MARKER) + +NAMED_REQUIREMENT = \ + NAME + Optional(EXTRAS) + (URL_AND_MARKER | VERSION_AND_MARKER) + +REQUIREMENT = stringStart + NAMED_REQUIREMENT + stringEnd + + +class Requirement(object): + """Parse a requirement. + + Parse a given requirement string into its parts, such as name, specifier, + URL, and extras. Raises InvalidRequirement on a badly-formed requirement + string. + """ + + # TODO: Can we test whether something is contained within a requirement? + # If so how do we do that? Do we need to test against the _name_ of + # the thing as well as the version? What about the markers? + # TODO: Can we normalize the name and extra name? + + def __init__(self, requirement_string): + try: + req = REQUIREMENT.parseString(requirement_string) + except ParseException as e: + raise InvalidRequirement( + "Invalid requirement, parse error at \"{0!r}\"".format( + requirement_string[e.loc:e.loc + 8])) + + self.name = req.name + if req.url: + parsed_url = urlparse.urlparse(req.url) + if not (parsed_url.scheme and parsed_url.netloc) or ( + not parsed_url.scheme and not parsed_url.netloc): + raise InvalidRequirement("Invalid URL given") + self.url = req.url + else: + self.url = None + self.extras = set(req.extras.asList() if req.extras else []) + self.specifier = SpecifierSet(req.specifier) + self.marker = req.marker if req.marker else None + + def __str__(self): + parts = [self.name] + + if self.extras: + parts.append("[{0}]".format(",".join(sorted(self.extras)))) + + if self.specifier: + parts.append(str(self.specifier)) + + if self.url: + parts.append("@ {0}".format(self.url)) + + if self.marker: + parts.append("; {0}".format(self.marker)) + + return "".join(parts) + + def __repr__(self): + return "".format(str(self)) diff --git a/env/lib/python3.8/site-packages/pkg_resources/_vendor/packaging/specifiers.py b/env/lib/python3.8/site-packages/pkg_resources/_vendor/packaging/specifiers.py new file mode 100644 index 000000000..7f5a76cfd --- /dev/null +++ b/env/lib/python3.8/site-packages/pkg_resources/_vendor/packaging/specifiers.py @@ -0,0 +1,774 @@ +# This file is dual licensed under the terms of the Apache License, Version +# 2.0, and the BSD License. See the LICENSE file in the root of this repository +# for complete details. +from __future__ import absolute_import, division, print_function + +import abc +import functools +import itertools +import re + +from ._compat import string_types, with_metaclass +from .version import Version, LegacyVersion, parse + + +class InvalidSpecifier(ValueError): + """ + An invalid specifier was found, users should refer to PEP 440. + """ + + +class BaseSpecifier(with_metaclass(abc.ABCMeta, object)): + + @abc.abstractmethod + def __str__(self): + """ + Returns the str representation of this Specifier like object. This + should be representative of the Specifier itself. + """ + + @abc.abstractmethod + def __hash__(self): + """ + Returns a hash value for this Specifier like object. + """ + + @abc.abstractmethod + def __eq__(self, other): + """ + Returns a boolean representing whether or not the two Specifier like + objects are equal. + """ + + @abc.abstractmethod + def __ne__(self, other): + """ + Returns a boolean representing whether or not the two Specifier like + objects are not equal. + """ + + @abc.abstractproperty + def prereleases(self): + """ + Returns whether or not pre-releases as a whole are allowed by this + specifier. + """ + + @prereleases.setter + def prereleases(self, value): + """ + Sets whether or not pre-releases as a whole are allowed by this + specifier. + """ + + @abc.abstractmethod + def contains(self, item, prereleases=None): + """ + Determines if the given item is contained within this specifier. + """ + + @abc.abstractmethod + def filter(self, iterable, prereleases=None): + """ + Takes an iterable of items and filters them so that only items which + are contained within this specifier are allowed in it. + """ + + +class _IndividualSpecifier(BaseSpecifier): + + _operators = {} + + def __init__(self, spec="", prereleases=None): + match = self._regex.search(spec) + if not match: + raise InvalidSpecifier("Invalid specifier: '{0}'".format(spec)) + + self._spec = ( + match.group("operator").strip(), + match.group("version").strip(), + ) + + # Store whether or not this Specifier should accept prereleases + self._prereleases = prereleases + + def __repr__(self): + pre = ( + ", prereleases={0!r}".format(self.prereleases) + if self._prereleases is not None + else "" + ) + + return "<{0}({1!r}{2})>".format( + self.__class__.__name__, + str(self), + pre, + ) + + def __str__(self): + return "{0}{1}".format(*self._spec) + + def __hash__(self): + return hash(self._spec) + + def __eq__(self, other): + if isinstance(other, string_types): + try: + other = self.__class__(other) + except InvalidSpecifier: + return NotImplemented + elif not isinstance(other, self.__class__): + return NotImplemented + + return self._spec == other._spec + + def __ne__(self, other): + if isinstance(other, string_types): + try: + other = self.__class__(other) + except InvalidSpecifier: + return NotImplemented + elif not isinstance(other, self.__class__): + return NotImplemented + + return self._spec != other._spec + + def _get_operator(self, op): + return getattr(self, "_compare_{0}".format(self._operators[op])) + + def _coerce_version(self, version): + if not isinstance(version, (LegacyVersion, Version)): + version = parse(version) + return version + + @property + def operator(self): + return self._spec[0] + + @property + def version(self): + return self._spec[1] + + @property + def prereleases(self): + return self._prereleases + + @prereleases.setter + def prereleases(self, value): + self._prereleases = value + + def __contains__(self, item): + return self.contains(item) + + def contains(self, item, prereleases=None): + # Determine if prereleases are to be allowed or not. + if prereleases is None: + prereleases = self.prereleases + + # Normalize item to a Version or LegacyVersion, this allows us to have + # a shortcut for ``"2.0" in Specifier(">=2") + item = self._coerce_version(item) + + # Determine if we should be supporting prereleases in this specifier + # or not, if we do not support prereleases than we can short circuit + # logic if this version is a prereleases. + if item.is_prerelease and not prereleases: + return False + + # Actually do the comparison to determine if this item is contained + # within this Specifier or not. + return self._get_operator(self.operator)(item, self.version) + + def filter(self, iterable, prereleases=None): + yielded = False + found_prereleases = [] + + kw = {"prereleases": prereleases if prereleases is not None else True} + + # Attempt to iterate over all the values in the iterable and if any of + # them match, yield them. + for version in iterable: + parsed_version = self._coerce_version(version) + + if self.contains(parsed_version, **kw): + # If our version is a prerelease, and we were not set to allow + # prereleases, then we'll store it for later incase nothing + # else matches this specifier. + if (parsed_version.is_prerelease and not + (prereleases or self.prereleases)): + found_prereleases.append(version) + # Either this is not a prerelease, or we should have been + # accepting prereleases from the begining. + else: + yielded = True + yield version + + # Now that we've iterated over everything, determine if we've yielded + # any values, and if we have not and we have any prereleases stored up + # then we will go ahead and yield the prereleases. + if not yielded and found_prereleases: + for version in found_prereleases: + yield version + + +class LegacySpecifier(_IndividualSpecifier): + + _regex_str = ( + r""" + (?P(==|!=|<=|>=|<|>)) + \s* + (?P + [^,;\s)]* # Since this is a "legacy" specifier, and the version + # string can be just about anything, we match everything + # except for whitespace, a semi-colon for marker support, + # a closing paren since versions can be enclosed in + # them, and a comma since it's a version separator. + ) + """ + ) + + _regex = re.compile( + r"^\s*" + _regex_str + r"\s*$", re.VERBOSE | re.IGNORECASE) + + _operators = { + "==": "equal", + "!=": "not_equal", + "<=": "less_than_equal", + ">=": "greater_than_equal", + "<": "less_than", + ">": "greater_than", + } + + def _coerce_version(self, version): + if not isinstance(version, LegacyVersion): + version = LegacyVersion(str(version)) + return version + + def _compare_equal(self, prospective, spec): + return prospective == self._coerce_version(spec) + + def _compare_not_equal(self, prospective, spec): + return prospective != self._coerce_version(spec) + + def _compare_less_than_equal(self, prospective, spec): + return prospective <= self._coerce_version(spec) + + def _compare_greater_than_equal(self, prospective, spec): + return prospective >= self._coerce_version(spec) + + def _compare_less_than(self, prospective, spec): + return prospective < self._coerce_version(spec) + + def _compare_greater_than(self, prospective, spec): + return prospective > self._coerce_version(spec) + + +def _require_version_compare(fn): + @functools.wraps(fn) + def wrapped(self, prospective, spec): + if not isinstance(prospective, Version): + return False + return fn(self, prospective, spec) + return wrapped + + +class Specifier(_IndividualSpecifier): + + _regex_str = ( + r""" + (?P(~=|==|!=|<=|>=|<|>|===)) + (?P + (?: + # The identity operators allow for an escape hatch that will + # do an exact string match of the version you wish to install. + # This will not be parsed by PEP 440 and we cannot determine + # any semantic meaning from it. This operator is discouraged + # but included entirely as an escape hatch. + (?<====) # Only match for the identity operator + \s* + [^\s]* # We just match everything, except for whitespace + # since we are only testing for strict identity. + ) + | + (?: + # The (non)equality operators allow for wild card and local + # versions to be specified so we have to define these two + # operators separately to enable that. + (?<===|!=) # Only match for equals and not equals + + \s* + v? + (?:[0-9]+!)? # epoch + [0-9]+(?:\.[0-9]+)* # release + (?: # pre release + [-_\.]? + (a|b|c|rc|alpha|beta|pre|preview) + [-_\.]? + [0-9]* + )? + (?: # post release + (?:-[0-9]+)|(?:[-_\.]?(post|rev|r)[-_\.]?[0-9]*) + )? + + # You cannot use a wild card and a dev or local version + # together so group them with a | and make them optional. + (?: + (?:[-_\.]?dev[-_\.]?[0-9]*)? # dev release + (?:\+[a-z0-9]+(?:[-_\.][a-z0-9]+)*)? # local + | + \.\* # Wild card syntax of .* + )? + ) + | + (?: + # The compatible operator requires at least two digits in the + # release segment. + (?<=~=) # Only match for the compatible operator + + \s* + v? + (?:[0-9]+!)? # epoch + [0-9]+(?:\.[0-9]+)+ # release (We have a + instead of a *) + (?: # pre release + [-_\.]? + (a|b|c|rc|alpha|beta|pre|preview) + [-_\.]? + [0-9]* + )? + (?: # post release + (?:-[0-9]+)|(?:[-_\.]?(post|rev|r)[-_\.]?[0-9]*) + )? + (?:[-_\.]?dev[-_\.]?[0-9]*)? # dev release + ) + | + (?: + # All other operators only allow a sub set of what the + # (non)equality operators do. Specifically they do not allow + # local versions to be specified nor do they allow the prefix + # matching wild cards. + (?=": "greater_than_equal", + "<": "less_than", + ">": "greater_than", + "===": "arbitrary", + } + + @_require_version_compare + def _compare_compatible(self, prospective, spec): + # Compatible releases have an equivalent combination of >= and ==. That + # is that ~=2.2 is equivalent to >=2.2,==2.*. This allows us to + # implement this in terms of the other specifiers instead of + # implementing it ourselves. The only thing we need to do is construct + # the other specifiers. + + # We want everything but the last item in the version, but we want to + # ignore post and dev releases and we want to treat the pre-release as + # it's own separate segment. + prefix = ".".join( + list( + itertools.takewhile( + lambda x: (not x.startswith("post") and not + x.startswith("dev")), + _version_split(spec), + ) + )[:-1] + ) + + # Add the prefix notation to the end of our string + prefix += ".*" + + return (self._get_operator(">=")(prospective, spec) and + self._get_operator("==")(prospective, prefix)) + + @_require_version_compare + def _compare_equal(self, prospective, spec): + # We need special logic to handle prefix matching + if spec.endswith(".*"): + # In the case of prefix matching we want to ignore local segment. + prospective = Version(prospective.public) + # Split the spec out by dots, and pretend that there is an implicit + # dot in between a release segment and a pre-release segment. + spec = _version_split(spec[:-2]) # Remove the trailing .* + + # Split the prospective version out by dots, and pretend that there + # is an implicit dot in between a release segment and a pre-release + # segment. + prospective = _version_split(str(prospective)) + + # Shorten the prospective version to be the same length as the spec + # so that we can determine if the specifier is a prefix of the + # prospective version or not. + prospective = prospective[:len(spec)] + + # Pad out our two sides with zeros so that they both equal the same + # length. + spec, prospective = _pad_version(spec, prospective) + else: + # Convert our spec string into a Version + spec = Version(spec) + + # If the specifier does not have a local segment, then we want to + # act as if the prospective version also does not have a local + # segment. + if not spec.local: + prospective = Version(prospective.public) + + return prospective == spec + + @_require_version_compare + def _compare_not_equal(self, prospective, spec): + return not self._compare_equal(prospective, spec) + + @_require_version_compare + def _compare_less_than_equal(self, prospective, spec): + return prospective <= Version(spec) + + @_require_version_compare + def _compare_greater_than_equal(self, prospective, spec): + return prospective >= Version(spec) + + @_require_version_compare + def _compare_less_than(self, prospective, spec): + # Convert our spec to a Version instance, since we'll want to work with + # it as a version. + spec = Version(spec) + + # Check to see if the prospective version is less than the spec + # version. If it's not we can short circuit and just return False now + # instead of doing extra unneeded work. + if not prospective < spec: + return False + + # This special case is here so that, unless the specifier itself + # includes is a pre-release version, that we do not accept pre-release + # versions for the version mentioned in the specifier (e.g. <3.1 should + # not match 3.1.dev0, but should match 3.0.dev0). + if not spec.is_prerelease and prospective.is_prerelease: + if Version(prospective.base_version) == Version(spec.base_version): + return False + + # If we've gotten to here, it means that prospective version is both + # less than the spec version *and* it's not a pre-release of the same + # version in the spec. + return True + + @_require_version_compare + def _compare_greater_than(self, prospective, spec): + # Convert our spec to a Version instance, since we'll want to work with + # it as a version. + spec = Version(spec) + + # Check to see if the prospective version is greater than the spec + # version. If it's not we can short circuit and just return False now + # instead of doing extra unneeded work. + if not prospective > spec: + return False + + # This special case is here so that, unless the specifier itself + # includes is a post-release version, that we do not accept + # post-release versions for the version mentioned in the specifier + # (e.g. >3.1 should not match 3.0.post0, but should match 3.2.post0). + if not spec.is_postrelease and prospective.is_postrelease: + if Version(prospective.base_version) == Version(spec.base_version): + return False + + # Ensure that we do not allow a local version of the version mentioned + # in the specifier, which is techincally greater than, to match. + if prospective.local is not None: + if Version(prospective.base_version) == Version(spec.base_version): + return False + + # If we've gotten to here, it means that prospective version is both + # greater than the spec version *and* it's not a pre-release of the + # same version in the spec. + return True + + def _compare_arbitrary(self, prospective, spec): + return str(prospective).lower() == str(spec).lower() + + @property + def prereleases(self): + # If there is an explicit prereleases set for this, then we'll just + # blindly use that. + if self._prereleases is not None: + return self._prereleases + + # Look at all of our specifiers and determine if they are inclusive + # operators, and if they are if they are including an explicit + # prerelease. + operator, version = self._spec + if operator in ["==", ">=", "<=", "~=", "==="]: + # The == specifier can include a trailing .*, if it does we + # want to remove before parsing. + if operator == "==" and version.endswith(".*"): + version = version[:-2] + + # Parse the version, and if it is a pre-release than this + # specifier allows pre-releases. + if parse(version).is_prerelease: + return True + + return False + + @prereleases.setter + def prereleases(self, value): + self._prereleases = value + + +_prefix_regex = re.compile(r"^([0-9]+)((?:a|b|c|rc)[0-9]+)$") + + +def _version_split(version): + result = [] + for item in version.split("."): + match = _prefix_regex.search(item) + if match: + result.extend(match.groups()) + else: + result.append(item) + return result + + +def _pad_version(left, right): + left_split, right_split = [], [] + + # Get the release segment of our versions + left_split.append(list(itertools.takewhile(lambda x: x.isdigit(), left))) + right_split.append(list(itertools.takewhile(lambda x: x.isdigit(), right))) + + # Get the rest of our versions + left_split.append(left[len(left_split[0]):]) + right_split.append(right[len(right_split[0]):]) + + # Insert our padding + left_split.insert( + 1, + ["0"] * max(0, len(right_split[0]) - len(left_split[0])), + ) + right_split.insert( + 1, + ["0"] * max(0, len(left_split[0]) - len(right_split[0])), + ) + + return ( + list(itertools.chain(*left_split)), + list(itertools.chain(*right_split)), + ) + + +class SpecifierSet(BaseSpecifier): + + def __init__(self, specifiers="", prereleases=None): + # Split on , to break each indidivual specifier into it's own item, and + # strip each item to remove leading/trailing whitespace. + specifiers = [s.strip() for s in specifiers.split(",") if s.strip()] + + # Parsed each individual specifier, attempting first to make it a + # Specifier and falling back to a LegacySpecifier. + parsed = set() + for specifier in specifiers: + try: + parsed.add(Specifier(specifier)) + except InvalidSpecifier: + parsed.add(LegacySpecifier(specifier)) + + # Turn our parsed specifiers into a frozen set and save them for later. + self._specs = frozenset(parsed) + + # Store our prereleases value so we can use it later to determine if + # we accept prereleases or not. + self._prereleases = prereleases + + def __repr__(self): + pre = ( + ", prereleases={0!r}".format(self.prereleases) + if self._prereleases is not None + else "" + ) + + return "".format(str(self), pre) + + def __str__(self): + return ",".join(sorted(str(s) for s in self._specs)) + + def __hash__(self): + return hash(self._specs) + + def __and__(self, other): + if isinstance(other, string_types): + other = SpecifierSet(other) + elif not isinstance(other, SpecifierSet): + return NotImplemented + + specifier = SpecifierSet() + specifier._specs = frozenset(self._specs | other._specs) + + if self._prereleases is None and other._prereleases is not None: + specifier._prereleases = other._prereleases + elif self._prereleases is not None and other._prereleases is None: + specifier._prereleases = self._prereleases + elif self._prereleases == other._prereleases: + specifier._prereleases = self._prereleases + else: + raise ValueError( + "Cannot combine SpecifierSets with True and False prerelease " + "overrides." + ) + + return specifier + + def __eq__(self, other): + if isinstance(other, string_types): + other = SpecifierSet(other) + elif isinstance(other, _IndividualSpecifier): + other = SpecifierSet(str(other)) + elif not isinstance(other, SpecifierSet): + return NotImplemented + + return self._specs == other._specs + + def __ne__(self, other): + if isinstance(other, string_types): + other = SpecifierSet(other) + elif isinstance(other, _IndividualSpecifier): + other = SpecifierSet(str(other)) + elif not isinstance(other, SpecifierSet): + return NotImplemented + + return self._specs != other._specs + + def __len__(self): + return len(self._specs) + + def __iter__(self): + return iter(self._specs) + + @property + def prereleases(self): + # If we have been given an explicit prerelease modifier, then we'll + # pass that through here. + if self._prereleases is not None: + return self._prereleases + + # If we don't have any specifiers, and we don't have a forced value, + # then we'll just return None since we don't know if this should have + # pre-releases or not. + if not self._specs: + return None + + # Otherwise we'll see if any of the given specifiers accept + # prereleases, if any of them do we'll return True, otherwise False. + return any(s.prereleases for s in self._specs) + + @prereleases.setter + def prereleases(self, value): + self._prereleases = value + + def __contains__(self, item): + return self.contains(item) + + def contains(self, item, prereleases=None): + # Ensure that our item is a Version or LegacyVersion instance. + if not isinstance(item, (LegacyVersion, Version)): + item = parse(item) + + # Determine if we're forcing a prerelease or not, if we're not forcing + # one for this particular filter call, then we'll use whatever the + # SpecifierSet thinks for whether or not we should support prereleases. + if prereleases is None: + prereleases = self.prereleases + + # We can determine if we're going to allow pre-releases by looking to + # see if any of the underlying items supports them. If none of them do + # and this item is a pre-release then we do not allow it and we can + # short circuit that here. + # Note: This means that 1.0.dev1 would not be contained in something + # like >=1.0.devabc however it would be in >=1.0.debabc,>0.0.dev0 + if not prereleases and item.is_prerelease: + return False + + # We simply dispatch to the underlying specs here to make sure that the + # given version is contained within all of them. + # Note: This use of all() here means that an empty set of specifiers + # will always return True, this is an explicit design decision. + return all( + s.contains(item, prereleases=prereleases) + for s in self._specs + ) + + def filter(self, iterable, prereleases=None): + # Determine if we're forcing a prerelease or not, if we're not forcing + # one for this particular filter call, then we'll use whatever the + # SpecifierSet thinks for whether or not we should support prereleases. + if prereleases is None: + prereleases = self.prereleases + + # If we have any specifiers, then we want to wrap our iterable in the + # filter method for each one, this will act as a logical AND amongst + # each specifier. + if self._specs: + for spec in self._specs: + iterable = spec.filter(iterable, prereleases=bool(prereleases)) + return iterable + # If we do not have any specifiers, then we need to have a rough filter + # which will filter out any pre-releases, unless there are no final + # releases, and which will filter out LegacyVersion in general. + else: + filtered = [] + found_prereleases = [] + + for item in iterable: + # Ensure that we some kind of Version class for this item. + if not isinstance(item, (LegacyVersion, Version)): + parsed_version = parse(item) + else: + parsed_version = item + + # Filter out any item which is parsed as a LegacyVersion + if isinstance(parsed_version, LegacyVersion): + continue + + # Store any item which is a pre-release for later unless we've + # already found a final version or we are accepting prereleases + if parsed_version.is_prerelease and not prereleases: + if not filtered: + found_prereleases.append(item) + else: + filtered.append(item) + + # If we've found no items except for pre-releases, then we'll go + # ahead and use the pre-releases + if not filtered and found_prereleases and prereleases is None: + return found_prereleases + + return filtered diff --git a/env/lib/python3.8/site-packages/pkg_resources/_vendor/packaging/utils.py b/env/lib/python3.8/site-packages/pkg_resources/_vendor/packaging/utils.py new file mode 100644 index 000000000..942387cef --- /dev/null +++ b/env/lib/python3.8/site-packages/pkg_resources/_vendor/packaging/utils.py @@ -0,0 +1,14 @@ +# This file is dual licensed under the terms of the Apache License, Version +# 2.0, and the BSD License. See the LICENSE file in the root of this repository +# for complete details. +from __future__ import absolute_import, division, print_function + +import re + + +_canonicalize_regex = re.compile(r"[-_.]+") + + +def canonicalize_name(name): + # This is taken from PEP 503. + return _canonicalize_regex.sub("-", name).lower() diff --git a/env/lib/python3.8/site-packages/pkg_resources/_vendor/packaging/version.py b/env/lib/python3.8/site-packages/pkg_resources/_vendor/packaging/version.py new file mode 100644 index 000000000..83b5ee8c5 --- /dev/null +++ b/env/lib/python3.8/site-packages/pkg_resources/_vendor/packaging/version.py @@ -0,0 +1,393 @@ +# This file is dual licensed under the terms of the Apache License, Version +# 2.0, and the BSD License. See the LICENSE file in the root of this repository +# for complete details. +from __future__ import absolute_import, division, print_function + +import collections +import itertools +import re + +from ._structures import Infinity + + +__all__ = [ + "parse", "Version", "LegacyVersion", "InvalidVersion", "VERSION_PATTERN" +] + + +_Version = collections.namedtuple( + "_Version", + ["epoch", "release", "dev", "pre", "post", "local"], +) + + +def parse(version): + """ + Parse the given version string and return either a :class:`Version` object + or a :class:`LegacyVersion` object depending on if the given version is + a valid PEP 440 version or a legacy version. + """ + try: + return Version(version) + except InvalidVersion: + return LegacyVersion(version) + + +class InvalidVersion(ValueError): + """ + An invalid version was found, users should refer to PEP 440. + """ + + +class _BaseVersion(object): + + def __hash__(self): + return hash(self._key) + + def __lt__(self, other): + return self._compare(other, lambda s, o: s < o) + + def __le__(self, other): + return self._compare(other, lambda s, o: s <= o) + + def __eq__(self, other): + return self._compare(other, lambda s, o: s == o) + + def __ge__(self, other): + return self._compare(other, lambda s, o: s >= o) + + def __gt__(self, other): + return self._compare(other, lambda s, o: s > o) + + def __ne__(self, other): + return self._compare(other, lambda s, o: s != o) + + def _compare(self, other, method): + if not isinstance(other, _BaseVersion): + return NotImplemented + + return method(self._key, other._key) + + +class LegacyVersion(_BaseVersion): + + def __init__(self, version): + self._version = str(version) + self._key = _legacy_cmpkey(self._version) + + def __str__(self): + return self._version + + def __repr__(self): + return "".format(repr(str(self))) + + @property + def public(self): + return self._version + + @property + def base_version(self): + return self._version + + @property + def local(self): + return None + + @property + def is_prerelease(self): + return False + + @property + def is_postrelease(self): + return False + + +_legacy_version_component_re = re.compile( + r"(\d+ | [a-z]+ | \.| -)", re.VERBOSE, +) + +_legacy_version_replacement_map = { + "pre": "c", "preview": "c", "-": "final-", "rc": "c", "dev": "@", +} + + +def _parse_version_parts(s): + for part in _legacy_version_component_re.split(s): + part = _legacy_version_replacement_map.get(part, part) + + if not part or part == ".": + continue + + if part[:1] in "0123456789": + # pad for numeric comparison + yield part.zfill(8) + else: + yield "*" + part + + # ensure that alpha/beta/candidate are before final + yield "*final" + + +def _legacy_cmpkey(version): + # We hardcode an epoch of -1 here. A PEP 440 version can only have a epoch + # greater than or equal to 0. This will effectively put the LegacyVersion, + # which uses the defacto standard originally implemented by setuptools, + # as before all PEP 440 versions. + epoch = -1 + + # This scheme is taken from pkg_resources.parse_version setuptools prior to + # it's adoption of the packaging library. + parts = [] + for part in _parse_version_parts(version.lower()): + if part.startswith("*"): + # remove "-" before a prerelease tag + if part < "*final": + while parts and parts[-1] == "*final-": + parts.pop() + + # remove trailing zeros from each series of numeric parts + while parts and parts[-1] == "00000000": + parts.pop() + + parts.append(part) + parts = tuple(parts) + + return epoch, parts + +# Deliberately not anchored to the start and end of the string, to make it +# easier for 3rd party code to reuse +VERSION_PATTERN = r""" + v? + (?: + (?:(?P[0-9]+)!)? # epoch + (?P[0-9]+(?:\.[0-9]+)*) # release segment + (?P
                                          # pre-release
+            [-_\.]?
+            (?P(a|b|c|rc|alpha|beta|pre|preview))
+            [-_\.]?
+            (?P[0-9]+)?
+        )?
+        (?P                                         # post release
+            (?:-(?P[0-9]+))
+            |
+            (?:
+                [-_\.]?
+                (?Ppost|rev|r)
+                [-_\.]?
+                (?P[0-9]+)?
+            )
+        )?
+        (?P                                          # dev release
+            [-_\.]?
+            (?Pdev)
+            [-_\.]?
+            (?P[0-9]+)?
+        )?
+    )
+    (?:\+(?P[a-z0-9]+(?:[-_\.][a-z0-9]+)*))?       # local version
+"""
+
+
+class Version(_BaseVersion):
+
+    _regex = re.compile(
+        r"^\s*" + VERSION_PATTERN + r"\s*$",
+        re.VERBOSE | re.IGNORECASE,
+    )
+
+    def __init__(self, version):
+        # Validate the version and parse it into pieces
+        match = self._regex.search(version)
+        if not match:
+            raise InvalidVersion("Invalid version: '{0}'".format(version))
+
+        # Store the parsed out pieces of the version
+        self._version = _Version(
+            epoch=int(match.group("epoch")) if match.group("epoch") else 0,
+            release=tuple(int(i) for i in match.group("release").split(".")),
+            pre=_parse_letter_version(
+                match.group("pre_l"),
+                match.group("pre_n"),
+            ),
+            post=_parse_letter_version(
+                match.group("post_l"),
+                match.group("post_n1") or match.group("post_n2"),
+            ),
+            dev=_parse_letter_version(
+                match.group("dev_l"),
+                match.group("dev_n"),
+            ),
+            local=_parse_local_version(match.group("local")),
+        )
+
+        # Generate a key which will be used for sorting
+        self._key = _cmpkey(
+            self._version.epoch,
+            self._version.release,
+            self._version.pre,
+            self._version.post,
+            self._version.dev,
+            self._version.local,
+        )
+
+    def __repr__(self):
+        return "".format(repr(str(self)))
+
+    def __str__(self):
+        parts = []
+
+        # Epoch
+        if self._version.epoch != 0:
+            parts.append("{0}!".format(self._version.epoch))
+
+        # Release segment
+        parts.append(".".join(str(x) for x in self._version.release))
+
+        # Pre-release
+        if self._version.pre is not None:
+            parts.append("".join(str(x) for x in self._version.pre))
+
+        # Post-release
+        if self._version.post is not None:
+            parts.append(".post{0}".format(self._version.post[1]))
+
+        # Development release
+        if self._version.dev is not None:
+            parts.append(".dev{0}".format(self._version.dev[1]))
+
+        # Local version segment
+        if self._version.local is not None:
+            parts.append(
+                "+{0}".format(".".join(str(x) for x in self._version.local))
+            )
+
+        return "".join(parts)
+
+    @property
+    def public(self):
+        return str(self).split("+", 1)[0]
+
+    @property
+    def base_version(self):
+        parts = []
+
+        # Epoch
+        if self._version.epoch != 0:
+            parts.append("{0}!".format(self._version.epoch))
+
+        # Release segment
+        parts.append(".".join(str(x) for x in self._version.release))
+
+        return "".join(parts)
+
+    @property
+    def local(self):
+        version_string = str(self)
+        if "+" in version_string:
+            return version_string.split("+", 1)[1]
+
+    @property
+    def is_prerelease(self):
+        return bool(self._version.dev or self._version.pre)
+
+    @property
+    def is_postrelease(self):
+        return bool(self._version.post)
+
+
+def _parse_letter_version(letter, number):
+    if letter:
+        # We consider there to be an implicit 0 in a pre-release if there is
+        # not a numeral associated with it.
+        if number is None:
+            number = 0
+
+        # We normalize any letters to their lower case form
+        letter = letter.lower()
+
+        # We consider some words to be alternate spellings of other words and
+        # in those cases we want to normalize the spellings to our preferred
+        # spelling.
+        if letter == "alpha":
+            letter = "a"
+        elif letter == "beta":
+            letter = "b"
+        elif letter in ["c", "pre", "preview"]:
+            letter = "rc"
+        elif letter in ["rev", "r"]:
+            letter = "post"
+
+        return letter, int(number)
+    if not letter and number:
+        # We assume if we are given a number, but we are not given a letter
+        # then this is using the implicit post release syntax (e.g. 1.0-1)
+        letter = "post"
+
+        return letter, int(number)
+
+
+_local_version_seperators = re.compile(r"[\._-]")
+
+
+def _parse_local_version(local):
+    """
+    Takes a string like abc.1.twelve and turns it into ("abc", 1, "twelve").
+    """
+    if local is not None:
+        return tuple(
+            part.lower() if not part.isdigit() else int(part)
+            for part in _local_version_seperators.split(local)
+        )
+
+
+def _cmpkey(epoch, release, pre, post, dev, local):
+    # When we compare a release version, we want to compare it with all of the
+    # trailing zeros removed. So we'll use a reverse the list, drop all the now
+    # leading zeros until we come to something non zero, then take the rest
+    # re-reverse it back into the correct order and make it a tuple and use
+    # that for our sorting key.
+    release = tuple(
+        reversed(list(
+            itertools.dropwhile(
+                lambda x: x == 0,
+                reversed(release),
+            )
+        ))
+    )
+
+    # We need to "trick" the sorting algorithm to put 1.0.dev0 before 1.0a0.
+    # We'll do this by abusing the pre segment, but we _only_ want to do this
+    # if there is not a pre or a post segment. If we have one of those then
+    # the normal sorting rules will handle this case correctly.
+    if pre is None and post is None and dev is not None:
+        pre = -Infinity
+    # Versions without a pre-release (except as noted above) should sort after
+    # those with one.
+    elif pre is None:
+        pre = Infinity
+
+    # Versions without a post segment should sort before those with one.
+    if post is None:
+        post = -Infinity
+
+    # Versions without a development segment should sort after those with one.
+    if dev is None:
+        dev = Infinity
+
+    if local is None:
+        # Versions without a local segment should sort before those with one.
+        local = -Infinity
+    else:
+        # Versions with a local segment need that segment parsed to implement
+        # the sorting rules in PEP440.
+        # - Alpha numeric segments sort before numeric segments
+        # - Alpha numeric segments sort lexicographically
+        # - Numeric segments sort numerically
+        # - Shorter versions sort before longer versions when the prefixes
+        #   match exactly
+        local = tuple(
+            (i, "") if isinstance(i, int) else (-Infinity, i)
+            for i in local
+        )
+
+    return epoch, release, pre, post, dev, local
diff --git a/env/lib/python3.8/site-packages/pkg_resources/_vendor/pyparsing.py b/env/lib/python3.8/site-packages/pkg_resources/_vendor/pyparsing.py
new file mode 100644
index 000000000..cf75e1e5f
--- /dev/null
+++ b/env/lib/python3.8/site-packages/pkg_resources/_vendor/pyparsing.py
@@ -0,0 +1,5742 @@
+# module pyparsing.py
+#
+# Copyright (c) 2003-2018  Paul T. McGuire
+#
+# Permission is hereby granted, free of charge, to any person obtaining
+# a copy of this software and associated documentation files (the
+# "Software"), to deal in the Software without restriction, including
+# without limitation the rights to use, copy, modify, merge, publish,
+# distribute, sublicense, and/or sell copies of the Software, and to
+# permit persons to whom the Software is furnished to do so, subject to
+# the following conditions:
+#
+# The above copyright notice and this permission notice shall be
+# included in all copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+# MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
+# IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
+# CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
+# TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
+# SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+#
+
+__doc__ = \
+"""
+pyparsing module - Classes and methods to define and execute parsing grammars
+=============================================================================
+
+The pyparsing module is an alternative approach to creating and executing simple grammars,
+vs. the traditional lex/yacc approach, or the use of regular expressions.  With pyparsing, you
+don't need to learn a new syntax for defining grammars or matching expressions - the parsing module
+provides a library of classes that you use to construct the grammar directly in Python.
+
+Here is a program to parse "Hello, World!" (or any greeting of the form 
+C{", !"}), built up using L{Word}, L{Literal}, and L{And} elements 
+(L{'+'} operator gives L{And} expressions, strings are auto-converted to
+L{Literal} expressions)::
+
+    from pyparsing import Word, alphas
+
+    # define grammar of a greeting
+    greet = Word(alphas) + "," + Word(alphas) + "!"
+
+    hello = "Hello, World!"
+    print (hello, "->", greet.parseString(hello))
+
+The program outputs the following::
+
+    Hello, World! -> ['Hello', ',', 'World', '!']
+
+The Python representation of the grammar is quite readable, owing to the self-explanatory
+class names, and the use of '+', '|' and '^' operators.
+
+The L{ParseResults} object returned from L{ParserElement.parseString} can be accessed as a nested list, a dictionary, or an
+object with named attributes.
+
+The pyparsing module handles some of the problems that are typically vexing when writing text parsers:
+ - extra or missing whitespace (the above program will also handle "Hello,World!", "Hello  ,  World  !", etc.)
+ - quoted strings
+ - embedded comments
+
+
+Getting Started -
+-----------------
+Visit the classes L{ParserElement} and L{ParseResults} to see the base classes that most other pyparsing
+classes inherit from. Use the docstrings for examples of how to:
+ - construct literal match expressions from L{Literal} and L{CaselessLiteral} classes
+ - construct character word-group expressions using the L{Word} class
+ - see how to create repetitive expressions using L{ZeroOrMore} and L{OneOrMore} classes
+ - use L{'+'}, L{'|'}, L{'^'}, and L{'&'} operators to combine simple expressions into more complex ones
+ - associate names with your parsed results using L{ParserElement.setResultsName}
+ - find some helpful expression short-cuts like L{delimitedList} and L{oneOf}
+ - find more useful common expressions in the L{pyparsing_common} namespace class
+"""
+
+__version__ = "2.2.1"
+__versionTime__ = "18 Sep 2018 00:49 UTC"
+__author__ = "Paul McGuire "
+
+import string
+from weakref import ref as wkref
+import copy
+import sys
+import warnings
+import re
+import sre_constants
+import collections
+import pprint
+import traceback
+import types
+from datetime import datetime
+
+try:
+    from _thread import RLock
+except ImportError:
+    from threading import RLock
+
+try:
+    # Python 3
+    from collections.abc import Iterable
+    from collections.abc import MutableMapping
+except ImportError:
+    # Python 2.7
+    from collections import Iterable
+    from collections import MutableMapping
+
+try:
+    from collections import OrderedDict as _OrderedDict
+except ImportError:
+    try:
+        from ordereddict import OrderedDict as _OrderedDict
+    except ImportError:
+        _OrderedDict = None
+
+#~ sys.stderr.write( "testing pyparsing module, version %s, %s\n" % (__version__,__versionTime__ ) )
+
+__all__ = [
+'And', 'CaselessKeyword', 'CaselessLiteral', 'CharsNotIn', 'Combine', 'Dict', 'Each', 'Empty',
+'FollowedBy', 'Forward', 'GoToColumn', 'Group', 'Keyword', 'LineEnd', 'LineStart', 'Literal',
+'MatchFirst', 'NoMatch', 'NotAny', 'OneOrMore', 'OnlyOnce', 'Optional', 'Or',
+'ParseBaseException', 'ParseElementEnhance', 'ParseException', 'ParseExpression', 'ParseFatalException',
+'ParseResults', 'ParseSyntaxException', 'ParserElement', 'QuotedString', 'RecursiveGrammarException',
+'Regex', 'SkipTo', 'StringEnd', 'StringStart', 'Suppress', 'Token', 'TokenConverter', 
+'White', 'Word', 'WordEnd', 'WordStart', 'ZeroOrMore',
+'alphanums', 'alphas', 'alphas8bit', 'anyCloseTag', 'anyOpenTag', 'cStyleComment', 'col',
+'commaSeparatedList', 'commonHTMLEntity', 'countedArray', 'cppStyleComment', 'dblQuotedString',
+'dblSlashComment', 'delimitedList', 'dictOf', 'downcaseTokens', 'empty', 'hexnums',
+'htmlComment', 'javaStyleComment', 'line', 'lineEnd', 'lineStart', 'lineno',
+'makeHTMLTags', 'makeXMLTags', 'matchOnlyAtCol', 'matchPreviousExpr', 'matchPreviousLiteral',
+'nestedExpr', 'nullDebugAction', 'nums', 'oneOf', 'opAssoc', 'operatorPrecedence', 'printables',
+'punc8bit', 'pythonStyleComment', 'quotedString', 'removeQuotes', 'replaceHTMLEntity', 
+'replaceWith', 'restOfLine', 'sglQuotedString', 'srange', 'stringEnd',
+'stringStart', 'traceParseAction', 'unicodeString', 'upcaseTokens', 'withAttribute',
+'indentedBlock', 'originalTextFor', 'ungroup', 'infixNotation','locatedExpr', 'withClass',
+'CloseMatch', 'tokenMap', 'pyparsing_common',
+]
+
+system_version = tuple(sys.version_info)[:3]
+PY_3 = system_version[0] == 3
+if PY_3:
+    _MAX_INT = sys.maxsize
+    basestring = str
+    unichr = chr
+    _ustr = str
+
+    # build list of single arg builtins, that can be used as parse actions
+    singleArgBuiltins = [sum, len, sorted, reversed, list, tuple, set, any, all, min, max]
+
+else:
+    _MAX_INT = sys.maxint
+    range = xrange
+
+    def _ustr(obj):
+        """Drop-in replacement for str(obj) that tries to be Unicode friendly. It first tries
+           str(obj). If that fails with a UnicodeEncodeError, then it tries unicode(obj). It
+           then < returns the unicode object | encodes it with the default encoding | ... >.
+        """
+        if isinstance(obj,unicode):
+            return obj
+
+        try:
+            # If this works, then _ustr(obj) has the same behaviour as str(obj), so
+            # it won't break any existing code.
+            return str(obj)
+
+        except UnicodeEncodeError:
+            # Else encode it
+            ret = unicode(obj).encode(sys.getdefaultencoding(), 'xmlcharrefreplace')
+            xmlcharref = Regex(r'&#\d+;')
+            xmlcharref.setParseAction(lambda t: '\\u' + hex(int(t[0][2:-1]))[2:])
+            return xmlcharref.transformString(ret)
+
+    # build list of single arg builtins, tolerant of Python version, that can be used as parse actions
+    singleArgBuiltins = []
+    import __builtin__
+    for fname in "sum len sorted reversed list tuple set any all min max".split():
+        try:
+            singleArgBuiltins.append(getattr(__builtin__,fname))
+        except AttributeError:
+            continue
+            
+_generatorType = type((y for y in range(1)))
+ 
+def _xml_escape(data):
+    """Escape &, <, >, ", ', etc. in a string of data."""
+
+    # ampersand must be replaced first
+    from_symbols = '&><"\''
+    to_symbols = ('&'+s+';' for s in "amp gt lt quot apos".split())
+    for from_,to_ in zip(from_symbols, to_symbols):
+        data = data.replace(from_, to_)
+    return data
+
+class _Constants(object):
+    pass
+
+alphas     = string.ascii_uppercase + string.ascii_lowercase
+nums       = "0123456789"
+hexnums    = nums + "ABCDEFabcdef"
+alphanums  = alphas + nums
+_bslash    = chr(92)
+printables = "".join(c for c in string.printable if c not in string.whitespace)
+
+class ParseBaseException(Exception):
+    """base exception class for all parsing runtime exceptions"""
+    # Performance tuning: we construct a *lot* of these, so keep this
+    # constructor as small and fast as possible
+    def __init__( self, pstr, loc=0, msg=None, elem=None ):
+        self.loc = loc
+        if msg is None:
+            self.msg = pstr
+            self.pstr = ""
+        else:
+            self.msg = msg
+            self.pstr = pstr
+        self.parserElement = elem
+        self.args = (pstr, loc, msg)
+
+    @classmethod
+    def _from_exception(cls, pe):
+        """
+        internal factory method to simplify creating one type of ParseException 
+        from another - avoids having __init__ signature conflicts among subclasses
+        """
+        return cls(pe.pstr, pe.loc, pe.msg, pe.parserElement)
+
+    def __getattr__( self, aname ):
+        """supported attributes by name are:
+            - lineno - returns the line number of the exception text
+            - col - returns the column number of the exception text
+            - line - returns the line containing the exception text
+        """
+        if( aname == "lineno" ):
+            return lineno( self.loc, self.pstr )
+        elif( aname in ("col", "column") ):
+            return col( self.loc, self.pstr )
+        elif( aname == "line" ):
+            return line( self.loc, self.pstr )
+        else:
+            raise AttributeError(aname)
+
+    def __str__( self ):
+        return "%s (at char %d), (line:%d, col:%d)" % \
+                ( self.msg, self.loc, self.lineno, self.column )
+    def __repr__( self ):
+        return _ustr(self)
+    def markInputline( self, markerString = ">!<" ):
+        """Extracts the exception line from the input string, and marks
+           the location of the exception with a special symbol.
+        """
+        line_str = self.line
+        line_column = self.column - 1
+        if markerString:
+            line_str = "".join((line_str[:line_column],
+                                markerString, line_str[line_column:]))
+        return line_str.strip()
+    def __dir__(self):
+        return "lineno col line".split() + dir(type(self))
+
+class ParseException(ParseBaseException):
+    """
+    Exception thrown when parse expressions don't match class;
+    supported attributes by name are:
+     - lineno - returns the line number of the exception text
+     - col - returns the column number of the exception text
+     - line - returns the line containing the exception text
+        
+    Example::
+        try:
+            Word(nums).setName("integer").parseString("ABC")
+        except ParseException as pe:
+            print(pe)
+            print("column: {}".format(pe.col))
+            
+    prints::
+       Expected integer (at char 0), (line:1, col:1)
+        column: 1
+    """
+    pass
+
+class ParseFatalException(ParseBaseException):
+    """user-throwable exception thrown when inconsistent parse content
+       is found; stops all parsing immediately"""
+    pass
+
+class ParseSyntaxException(ParseFatalException):
+    """just like L{ParseFatalException}, but thrown internally when an
+       L{ErrorStop} ('-' operator) indicates that parsing is to stop 
+       immediately because an unbacktrackable syntax error has been found"""
+    pass
+
+#~ class ReparseException(ParseBaseException):
+    #~ """Experimental class - parse actions can raise this exception to cause
+       #~ pyparsing to reparse the input string:
+        #~ - with a modified input string, and/or
+        #~ - with a modified start location
+       #~ Set the values of the ReparseException in the constructor, and raise the
+       #~ exception in a parse action to cause pyparsing to use the new string/location.
+       #~ Setting the values as None causes no change to be made.
+       #~ """
+    #~ def __init_( self, newstring, restartLoc ):
+        #~ self.newParseText = newstring
+        #~ self.reparseLoc = restartLoc
+
+class RecursiveGrammarException(Exception):
+    """exception thrown by L{ParserElement.validate} if the grammar could be improperly recursive"""
+    def __init__( self, parseElementList ):
+        self.parseElementTrace = parseElementList
+
+    def __str__( self ):
+        return "RecursiveGrammarException: %s" % self.parseElementTrace
+
+class _ParseResultsWithOffset(object):
+    def __init__(self,p1,p2):
+        self.tup = (p1,p2)
+    def __getitem__(self,i):
+        return self.tup[i]
+    def __repr__(self):
+        return repr(self.tup[0])
+    def setOffset(self,i):
+        self.tup = (self.tup[0],i)
+
+class ParseResults(object):
+    """
+    Structured parse results, to provide multiple means of access to the parsed data:
+       - as a list (C{len(results)})
+       - by list index (C{results[0], results[1]}, etc.)
+       - by attribute (C{results.} - see L{ParserElement.setResultsName})
+
+    Example::
+        integer = Word(nums)
+        date_str = (integer.setResultsName("year") + '/' 
+                        + integer.setResultsName("month") + '/' 
+                        + integer.setResultsName("day"))
+        # equivalent form:
+        # date_str = integer("year") + '/' + integer("month") + '/' + integer("day")
+
+        # parseString returns a ParseResults object
+        result = date_str.parseString("1999/12/31")
+
+        def test(s, fn=repr):
+            print("%s -> %s" % (s, fn(eval(s))))
+        test("list(result)")
+        test("result[0]")
+        test("result['month']")
+        test("result.day")
+        test("'month' in result")
+        test("'minutes' in result")
+        test("result.dump()", str)
+    prints::
+        list(result) -> ['1999', '/', '12', '/', '31']
+        result[0] -> '1999'
+        result['month'] -> '12'
+        result.day -> '31'
+        'month' in result -> True
+        'minutes' in result -> False
+        result.dump() -> ['1999', '/', '12', '/', '31']
+        - day: 31
+        - month: 12
+        - year: 1999
+    """
+    def __new__(cls, toklist=None, name=None, asList=True, modal=True ):
+        if isinstance(toklist, cls):
+            return toklist
+        retobj = object.__new__(cls)
+        retobj.__doinit = True
+        return retobj
+
+    # Performance tuning: we construct a *lot* of these, so keep this
+    # constructor as small and fast as possible
+    def __init__( self, toklist=None, name=None, asList=True, modal=True, isinstance=isinstance ):
+        if self.__doinit:
+            self.__doinit = False
+            self.__name = None
+            self.__parent = None
+            self.__accumNames = {}
+            self.__asList = asList
+            self.__modal = modal
+            if toklist is None:
+                toklist = []
+            if isinstance(toklist, list):
+                self.__toklist = toklist[:]
+            elif isinstance(toklist, _generatorType):
+                self.__toklist = list(toklist)
+            else:
+                self.__toklist = [toklist]
+            self.__tokdict = dict()
+
+        if name is not None and name:
+            if not modal:
+                self.__accumNames[name] = 0
+            if isinstance(name,int):
+                name = _ustr(name) # will always return a str, but use _ustr for consistency
+            self.__name = name
+            if not (isinstance(toklist, (type(None), basestring, list)) and toklist in (None,'',[])):
+                if isinstance(toklist,basestring):
+                    toklist = [ toklist ]
+                if asList:
+                    if isinstance(toklist,ParseResults):
+                        self[name] = _ParseResultsWithOffset(toklist.copy(),0)
+                    else:
+                        self[name] = _ParseResultsWithOffset(ParseResults(toklist[0]),0)
+                    self[name].__name = name
+                else:
+                    try:
+                        self[name] = toklist[0]
+                    except (KeyError,TypeError,IndexError):
+                        self[name] = toklist
+
+    def __getitem__( self, i ):
+        if isinstance( i, (int,slice) ):
+            return self.__toklist[i]
+        else:
+            if i not in self.__accumNames:
+                return self.__tokdict[i][-1][0]
+            else:
+                return ParseResults([ v[0] for v in self.__tokdict[i] ])
+
+    def __setitem__( self, k, v, isinstance=isinstance ):
+        if isinstance(v,_ParseResultsWithOffset):
+            self.__tokdict[k] = self.__tokdict.get(k,list()) + [v]
+            sub = v[0]
+        elif isinstance(k,(int,slice)):
+            self.__toklist[k] = v
+            sub = v
+        else:
+            self.__tokdict[k] = self.__tokdict.get(k,list()) + [_ParseResultsWithOffset(v,0)]
+            sub = v
+        if isinstance(sub,ParseResults):
+            sub.__parent = wkref(self)
+
+    def __delitem__( self, i ):
+        if isinstance(i,(int,slice)):
+            mylen = len( self.__toklist )
+            del self.__toklist[i]
+
+            # convert int to slice
+            if isinstance(i, int):
+                if i < 0:
+                    i += mylen
+                i = slice(i, i+1)
+            # get removed indices
+            removed = list(range(*i.indices(mylen)))
+            removed.reverse()
+            # fixup indices in token dictionary
+            for name,occurrences in self.__tokdict.items():
+                for j in removed:
+                    for k, (value, position) in enumerate(occurrences):
+                        occurrences[k] = _ParseResultsWithOffset(value, position - (position > j))
+        else:
+            del self.__tokdict[i]
+
+    def __contains__( self, k ):
+        return k in self.__tokdict
+
+    def __len__( self ): return len( self.__toklist )
+    def __bool__(self): return ( not not self.__toklist )
+    __nonzero__ = __bool__
+    def __iter__( self ): return iter( self.__toklist )
+    def __reversed__( self ): return iter( self.__toklist[::-1] )
+    def _iterkeys( self ):
+        if hasattr(self.__tokdict, "iterkeys"):
+            return self.__tokdict.iterkeys()
+        else:
+            return iter(self.__tokdict)
+
+    def _itervalues( self ):
+        return (self[k] for k in self._iterkeys())
+            
+    def _iteritems( self ):
+        return ((k, self[k]) for k in self._iterkeys())
+
+    if PY_3:
+        keys = _iterkeys       
+        """Returns an iterator of all named result keys (Python 3.x only)."""
+
+        values = _itervalues
+        """Returns an iterator of all named result values (Python 3.x only)."""
+
+        items = _iteritems
+        """Returns an iterator of all named result key-value tuples (Python 3.x only)."""
+
+    else:
+        iterkeys = _iterkeys
+        """Returns an iterator of all named result keys (Python 2.x only)."""
+
+        itervalues = _itervalues
+        """Returns an iterator of all named result values (Python 2.x only)."""
+
+        iteritems = _iteritems
+        """Returns an iterator of all named result key-value tuples (Python 2.x only)."""
+
+        def keys( self ):
+            """Returns all named result keys (as a list in Python 2.x, as an iterator in Python 3.x)."""
+            return list(self.iterkeys())
+
+        def values( self ):
+            """Returns all named result values (as a list in Python 2.x, as an iterator in Python 3.x)."""
+            return list(self.itervalues())
+                
+        def items( self ):
+            """Returns all named result key-values (as a list of tuples in Python 2.x, as an iterator in Python 3.x)."""
+            return list(self.iteritems())
+
+    def haskeys( self ):
+        """Since keys() returns an iterator, this method is helpful in bypassing
+           code that looks for the existence of any defined results names."""
+        return bool(self.__tokdict)
+        
+    def pop( self, *args, **kwargs):
+        """
+        Removes and returns item at specified index (default=C{last}).
+        Supports both C{list} and C{dict} semantics for C{pop()}. If passed no
+        argument or an integer argument, it will use C{list} semantics
+        and pop tokens from the list of parsed tokens. If passed a 
+        non-integer argument (most likely a string), it will use C{dict}
+        semantics and pop the corresponding value from any defined 
+        results names. A second default return value argument is 
+        supported, just as in C{dict.pop()}.
+
+        Example::
+            def remove_first(tokens):
+                tokens.pop(0)
+            print(OneOrMore(Word(nums)).parseString("0 123 321")) # -> ['0', '123', '321']
+            print(OneOrMore(Word(nums)).addParseAction(remove_first).parseString("0 123 321")) # -> ['123', '321']
+
+            label = Word(alphas)
+            patt = label("LABEL") + OneOrMore(Word(nums))
+            print(patt.parseString("AAB 123 321").dump())
+
+            # Use pop() in a parse action to remove named result (note that corresponding value is not
+            # removed from list form of results)
+            def remove_LABEL(tokens):
+                tokens.pop("LABEL")
+                return tokens
+            patt.addParseAction(remove_LABEL)
+            print(patt.parseString("AAB 123 321").dump())
+        prints::
+            ['AAB', '123', '321']
+            - LABEL: AAB
+
+            ['AAB', '123', '321']
+        """
+        if not args:
+            args = [-1]
+        for k,v in kwargs.items():
+            if k == 'default':
+                args = (args[0], v)
+            else:
+                raise TypeError("pop() got an unexpected keyword argument '%s'" % k)
+        if (isinstance(args[0], int) or 
+                        len(args) == 1 or 
+                        args[0] in self):
+            index = args[0]
+            ret = self[index]
+            del self[index]
+            return ret
+        else:
+            defaultvalue = args[1]
+            return defaultvalue
+
+    def get(self, key, defaultValue=None):
+        """
+        Returns named result matching the given key, or if there is no
+        such name, then returns the given C{defaultValue} or C{None} if no
+        C{defaultValue} is specified.
+
+        Similar to C{dict.get()}.
+        
+        Example::
+            integer = Word(nums)
+            date_str = integer("year") + '/' + integer("month") + '/' + integer("day")           
+
+            result = date_str.parseString("1999/12/31")
+            print(result.get("year")) # -> '1999'
+            print(result.get("hour", "not specified")) # -> 'not specified'
+            print(result.get("hour")) # -> None
+        """
+        if key in self:
+            return self[key]
+        else:
+            return defaultValue
+
+    def insert( self, index, insStr ):
+        """
+        Inserts new element at location index in the list of parsed tokens.
+        
+        Similar to C{list.insert()}.
+
+        Example::
+            print(OneOrMore(Word(nums)).parseString("0 123 321")) # -> ['0', '123', '321']
+
+            # use a parse action to insert the parse location in the front of the parsed results
+            def insert_locn(locn, tokens):
+                tokens.insert(0, locn)
+            print(OneOrMore(Word(nums)).addParseAction(insert_locn).parseString("0 123 321")) # -> [0, '0', '123', '321']
+        """
+        self.__toklist.insert(index, insStr)
+        # fixup indices in token dictionary
+        for name,occurrences in self.__tokdict.items():
+            for k, (value, position) in enumerate(occurrences):
+                occurrences[k] = _ParseResultsWithOffset(value, position + (position > index))
+
+    def append( self, item ):
+        """
+        Add single element to end of ParseResults list of elements.
+
+        Example::
+            print(OneOrMore(Word(nums)).parseString("0 123 321")) # -> ['0', '123', '321']
+            
+            # use a parse action to compute the sum of the parsed integers, and add it to the end
+            def append_sum(tokens):
+                tokens.append(sum(map(int, tokens)))
+            print(OneOrMore(Word(nums)).addParseAction(append_sum).parseString("0 123 321")) # -> ['0', '123', '321', 444]
+        """
+        self.__toklist.append(item)
+
+    def extend( self, itemseq ):
+        """
+        Add sequence of elements to end of ParseResults list of elements.
+
+        Example::
+            patt = OneOrMore(Word(alphas))
+            
+            # use a parse action to append the reverse of the matched strings, to make a palindrome
+            def make_palindrome(tokens):
+                tokens.extend(reversed([t[::-1] for t in tokens]))
+                return ''.join(tokens)
+            print(patt.addParseAction(make_palindrome).parseString("lskdj sdlkjf lksd")) # -> 'lskdjsdlkjflksddsklfjkldsjdksl'
+        """
+        if isinstance(itemseq, ParseResults):
+            self += itemseq
+        else:
+            self.__toklist.extend(itemseq)
+
+    def clear( self ):
+        """
+        Clear all elements and results names.
+        """
+        del self.__toklist[:]
+        self.__tokdict.clear()
+
+    def __getattr__( self, name ):
+        try:
+            return self[name]
+        except KeyError:
+            return ""
+            
+        if name in self.__tokdict:
+            if name not in self.__accumNames:
+                return self.__tokdict[name][-1][0]
+            else:
+                return ParseResults([ v[0] for v in self.__tokdict[name] ])
+        else:
+            return ""
+
+    def __add__( self, other ):
+        ret = self.copy()
+        ret += other
+        return ret
+
+    def __iadd__( self, other ):
+        if other.__tokdict:
+            offset = len(self.__toklist)
+            addoffset = lambda a: offset if a<0 else a+offset
+            otheritems = other.__tokdict.items()
+            otherdictitems = [(k, _ParseResultsWithOffset(v[0],addoffset(v[1])) )
+                                for (k,vlist) in otheritems for v in vlist]
+            for k,v in otherdictitems:
+                self[k] = v
+                if isinstance(v[0],ParseResults):
+                    v[0].__parent = wkref(self)
+            
+        self.__toklist += other.__toklist
+        self.__accumNames.update( other.__accumNames )
+        return self
+
+    def __radd__(self, other):
+        if isinstance(other,int) and other == 0:
+            # useful for merging many ParseResults using sum() builtin
+            return self.copy()
+        else:
+            # this may raise a TypeError - so be it
+            return other + self
+        
+    def __repr__( self ):
+        return "(%s, %s)" % ( repr( self.__toklist ), repr( self.__tokdict ) )
+
+    def __str__( self ):
+        return '[' + ', '.join(_ustr(i) if isinstance(i, ParseResults) else repr(i) for i in self.__toklist) + ']'
+
+    def _asStringList( self, sep='' ):
+        out = []
+        for item in self.__toklist:
+            if out and sep:
+                out.append(sep)
+            if isinstance( item, ParseResults ):
+                out += item._asStringList()
+            else:
+                out.append( _ustr(item) )
+        return out
+
+    def asList( self ):
+        """
+        Returns the parse results as a nested list of matching tokens, all converted to strings.
+
+        Example::
+            patt = OneOrMore(Word(alphas))
+            result = patt.parseString("sldkj lsdkj sldkj")
+            # even though the result prints in string-like form, it is actually a pyparsing ParseResults
+            print(type(result), result) # ->  ['sldkj', 'lsdkj', 'sldkj']
+            
+            # Use asList() to create an actual list
+            result_list = result.asList()
+            print(type(result_list), result_list) # ->  ['sldkj', 'lsdkj', 'sldkj']
+        """
+        return [res.asList() if isinstance(res,ParseResults) else res for res in self.__toklist]
+
+    def asDict( self ):
+        """
+        Returns the named parse results as a nested dictionary.
+
+        Example::
+            integer = Word(nums)
+            date_str = integer("year") + '/' + integer("month") + '/' + integer("day")
+            
+            result = date_str.parseString('12/31/1999')
+            print(type(result), repr(result)) # ->  (['12', '/', '31', '/', '1999'], {'day': [('1999', 4)], 'year': [('12', 0)], 'month': [('31', 2)]})
+            
+            result_dict = result.asDict()
+            print(type(result_dict), repr(result_dict)) # ->  {'day': '1999', 'year': '12', 'month': '31'}
+
+            # even though a ParseResults supports dict-like access, sometime you just need to have a dict
+            import json
+            print(json.dumps(result)) # -> Exception: TypeError: ... is not JSON serializable
+            print(json.dumps(result.asDict())) # -> {"month": "31", "day": "1999", "year": "12"}
+        """
+        if PY_3:
+            item_fn = self.items
+        else:
+            item_fn = self.iteritems
+            
+        def toItem(obj):
+            if isinstance(obj, ParseResults):
+                if obj.haskeys():
+                    return obj.asDict()
+                else:
+                    return [toItem(v) for v in obj]
+            else:
+                return obj
+                
+        return dict((k,toItem(v)) for k,v in item_fn())
+
+    def copy( self ):
+        """
+        Returns a new copy of a C{ParseResults} object.
+        """
+        ret = ParseResults( self.__toklist )
+        ret.__tokdict = self.__tokdict.copy()
+        ret.__parent = self.__parent
+        ret.__accumNames.update( self.__accumNames )
+        ret.__name = self.__name
+        return ret
+
+    def asXML( self, doctag=None, namedItemsOnly=False, indent="", formatted=True ):
+        """
+        (Deprecated) Returns the parse results as XML. Tags are created for tokens and lists that have defined results names.
+        """
+        nl = "\n"
+        out = []
+        namedItems = dict((v[1],k) for (k,vlist) in self.__tokdict.items()
+                                                            for v in vlist)
+        nextLevelIndent = indent + "  "
+
+        # collapse out indents if formatting is not desired
+        if not formatted:
+            indent = ""
+            nextLevelIndent = ""
+            nl = ""
+
+        selfTag = None
+        if doctag is not None:
+            selfTag = doctag
+        else:
+            if self.__name:
+                selfTag = self.__name
+
+        if not selfTag:
+            if namedItemsOnly:
+                return ""
+            else:
+                selfTag = "ITEM"
+
+        out += [ nl, indent, "<", selfTag, ">" ]
+
+        for i,res in enumerate(self.__toklist):
+            if isinstance(res,ParseResults):
+                if i in namedItems:
+                    out += [ res.asXML(namedItems[i],
+                                        namedItemsOnly and doctag is None,
+                                        nextLevelIndent,
+                                        formatted)]
+                else:
+                    out += [ res.asXML(None,
+                                        namedItemsOnly and doctag is None,
+                                        nextLevelIndent,
+                                        formatted)]
+            else:
+                # individual token, see if there is a name for it
+                resTag = None
+                if i in namedItems:
+                    resTag = namedItems[i]
+                if not resTag:
+                    if namedItemsOnly:
+                        continue
+                    else:
+                        resTag = "ITEM"
+                xmlBodyText = _xml_escape(_ustr(res))
+                out += [ nl, nextLevelIndent, "<", resTag, ">",
+                                                xmlBodyText,
+                                                "" ]
+
+        out += [ nl, indent, "" ]
+        return "".join(out)
+
+    def __lookup(self,sub):
+        for k,vlist in self.__tokdict.items():
+            for v,loc in vlist:
+                if sub is v:
+                    return k
+        return None
+
+    def getName(self):
+        r"""
+        Returns the results name for this token expression. Useful when several 
+        different expressions might match at a particular location.
+
+        Example::
+            integer = Word(nums)
+            ssn_expr = Regex(r"\d\d\d-\d\d-\d\d\d\d")
+            house_number_expr = Suppress('#') + Word(nums, alphanums)
+            user_data = (Group(house_number_expr)("house_number") 
+                        | Group(ssn_expr)("ssn")
+                        | Group(integer)("age"))
+            user_info = OneOrMore(user_data)
+            
+            result = user_info.parseString("22 111-22-3333 #221B")
+            for item in result:
+                print(item.getName(), ':', item[0])
+        prints::
+            age : 22
+            ssn : 111-22-3333
+            house_number : 221B
+        """
+        if self.__name:
+            return self.__name
+        elif self.__parent:
+            par = self.__parent()
+            if par:
+                return par.__lookup(self)
+            else:
+                return None
+        elif (len(self) == 1 and
+               len(self.__tokdict) == 1 and
+               next(iter(self.__tokdict.values()))[0][1] in (0,-1)):
+            return next(iter(self.__tokdict.keys()))
+        else:
+            return None
+
+    def dump(self, indent='', depth=0, full=True):
+        """
+        Diagnostic method for listing out the contents of a C{ParseResults}.
+        Accepts an optional C{indent} argument so that this string can be embedded
+        in a nested display of other data.
+
+        Example::
+            integer = Word(nums)
+            date_str = integer("year") + '/' + integer("month") + '/' + integer("day")
+            
+            result = date_str.parseString('12/31/1999')
+            print(result.dump())
+        prints::
+            ['12', '/', '31', '/', '1999']
+            - day: 1999
+            - month: 31
+            - year: 12
+        """
+        out = []
+        NL = '\n'
+        out.append( indent+_ustr(self.asList()) )
+        if full:
+            if self.haskeys():
+                items = sorted((str(k), v) for k,v in self.items())
+                for k,v in items:
+                    if out:
+                        out.append(NL)
+                    out.append( "%s%s- %s: " % (indent,('  '*depth), k) )
+                    if isinstance(v,ParseResults):
+                        if v:
+                            out.append( v.dump(indent,depth+1) )
+                        else:
+                            out.append(_ustr(v))
+                    else:
+                        out.append(repr(v))
+            elif any(isinstance(vv,ParseResults) for vv in self):
+                v = self
+                for i,vv in enumerate(v):
+                    if isinstance(vv,ParseResults):
+                        out.append("\n%s%s[%d]:\n%s%s%s" % (indent,('  '*(depth)),i,indent,('  '*(depth+1)),vv.dump(indent,depth+1) ))
+                    else:
+                        out.append("\n%s%s[%d]:\n%s%s%s" % (indent,('  '*(depth)),i,indent,('  '*(depth+1)),_ustr(vv)))
+            
+        return "".join(out)
+
+    def pprint(self, *args, **kwargs):
+        """
+        Pretty-printer for parsed results as a list, using the C{pprint} module.
+        Accepts additional positional or keyword args as defined for the 
+        C{pprint.pprint} method. (U{http://docs.python.org/3/library/pprint.html#pprint.pprint})
+
+        Example::
+            ident = Word(alphas, alphanums)
+            num = Word(nums)
+            func = Forward()
+            term = ident | num | Group('(' + func + ')')
+            func <<= ident + Group(Optional(delimitedList(term)))
+            result = func.parseString("fna a,b,(fnb c,d,200),100")
+            result.pprint(width=40)
+        prints::
+            ['fna',
+             ['a',
+              'b',
+              ['(', 'fnb', ['c', 'd', '200'], ')'],
+              '100']]
+        """
+        pprint.pprint(self.asList(), *args, **kwargs)
+
+    # add support for pickle protocol
+    def __getstate__(self):
+        return ( self.__toklist,
+                 ( self.__tokdict.copy(),
+                   self.__parent is not None and self.__parent() or None,
+                   self.__accumNames,
+                   self.__name ) )
+
+    def __setstate__(self,state):
+        self.__toklist = state[0]
+        (self.__tokdict,
+         par,
+         inAccumNames,
+         self.__name) = state[1]
+        self.__accumNames = {}
+        self.__accumNames.update(inAccumNames)
+        if par is not None:
+            self.__parent = wkref(par)
+        else:
+            self.__parent = None
+
+    def __getnewargs__(self):
+        return self.__toklist, self.__name, self.__asList, self.__modal
+
+    def __dir__(self):
+        return (dir(type(self)) + list(self.keys()))
+
+MutableMapping.register(ParseResults)
+
+def col (loc,strg):
+    """Returns current column within a string, counting newlines as line separators.
+   The first column is number 1.
+
+   Note: the default parsing behavior is to expand tabs in the input string
+   before starting the parsing process.  See L{I{ParserElement.parseString}} for more information
+   on parsing strings containing C{}s, and suggested methods to maintain a
+   consistent view of the parsed string, the parse location, and line and column
+   positions within the parsed string.
+   """
+    s = strg
+    return 1 if 0} for more information
+   on parsing strings containing C{}s, and suggested methods to maintain a
+   consistent view of the parsed string, the parse location, and line and column
+   positions within the parsed string.
+   """
+    return strg.count("\n",0,loc) + 1
+
+def line( loc, strg ):
+    """Returns the line of text containing loc within a string, counting newlines as line separators.
+       """
+    lastCR = strg.rfind("\n", 0, loc)
+    nextCR = strg.find("\n", loc)
+    if nextCR >= 0:
+        return strg[lastCR+1:nextCR]
+    else:
+        return strg[lastCR+1:]
+
+def _defaultStartDebugAction( instring, loc, expr ):
+    print (("Match " + _ustr(expr) + " at loc " + _ustr(loc) + "(%d,%d)" % ( lineno(loc,instring), col(loc,instring) )))
+
+def _defaultSuccessDebugAction( instring, startloc, endloc, expr, toks ):
+    print ("Matched " + _ustr(expr) + " -> " + str(toks.asList()))
+
+def _defaultExceptionDebugAction( instring, loc, expr, exc ):
+    print ("Exception raised:" + _ustr(exc))
+
+def nullDebugAction(*args):
+    """'Do-nothing' debug action, to suppress debugging output during parsing."""
+    pass
+
+# Only works on Python 3.x - nonlocal is toxic to Python 2 installs
+#~ 'decorator to trim function calls to match the arity of the target'
+#~ def _trim_arity(func, maxargs=3):
+    #~ if func in singleArgBuiltins:
+        #~ return lambda s,l,t: func(t)
+    #~ limit = 0
+    #~ foundArity = False
+    #~ def wrapper(*args):
+        #~ nonlocal limit,foundArity
+        #~ while 1:
+            #~ try:
+                #~ ret = func(*args[limit:])
+                #~ foundArity = True
+                #~ return ret
+            #~ except TypeError:
+                #~ if limit == maxargs or foundArity:
+                    #~ raise
+                #~ limit += 1
+                #~ continue
+    #~ return wrapper
+
+# this version is Python 2.x-3.x cross-compatible
+'decorator to trim function calls to match the arity of the target'
+def _trim_arity(func, maxargs=2):
+    if func in singleArgBuiltins:
+        return lambda s,l,t: func(t)
+    limit = [0]
+    foundArity = [False]
+    
+    # traceback return data structure changed in Py3.5 - normalize back to plain tuples
+    if system_version[:2] >= (3,5):
+        def extract_stack(limit=0):
+            # special handling for Python 3.5.0 - extra deep call stack by 1
+            offset = -3 if system_version == (3,5,0) else -2
+            frame_summary = traceback.extract_stack(limit=-offset+limit-1)[offset]
+            return [frame_summary[:2]]
+        def extract_tb(tb, limit=0):
+            frames = traceback.extract_tb(tb, limit=limit)
+            frame_summary = frames[-1]
+            return [frame_summary[:2]]
+    else:
+        extract_stack = traceback.extract_stack
+        extract_tb = traceback.extract_tb
+    
+    # synthesize what would be returned by traceback.extract_stack at the call to 
+    # user's parse action 'func', so that we don't incur call penalty at parse time
+    
+    LINE_DIFF = 6
+    # IF ANY CODE CHANGES, EVEN JUST COMMENTS OR BLANK LINES, BETWEEN THE NEXT LINE AND 
+    # THE CALL TO FUNC INSIDE WRAPPER, LINE_DIFF MUST BE MODIFIED!!!!
+    this_line = extract_stack(limit=2)[-1]
+    pa_call_line_synth = (this_line[0], this_line[1]+LINE_DIFF)
+
+    def wrapper(*args):
+        while 1:
+            try:
+                ret = func(*args[limit[0]:])
+                foundArity[0] = True
+                return ret
+            except TypeError:
+                # re-raise TypeErrors if they did not come from our arity testing
+                if foundArity[0]:
+                    raise
+                else:
+                    try:
+                        tb = sys.exc_info()[-1]
+                        if not extract_tb(tb, limit=2)[-1][:2] == pa_call_line_synth:
+                            raise
+                    finally:
+                        del tb
+
+                if limit[0] <= maxargs:
+                    limit[0] += 1
+                    continue
+                raise
+
+    # copy func name to wrapper for sensible debug output
+    func_name = ""
+    try:
+        func_name = getattr(func, '__name__', 
+                            getattr(func, '__class__').__name__)
+    except Exception:
+        func_name = str(func)
+    wrapper.__name__ = func_name
+
+    return wrapper
+
+class ParserElement(object):
+    """Abstract base level parser element class."""
+    DEFAULT_WHITE_CHARS = " \n\t\r"
+    verbose_stacktrace = False
+
+    @staticmethod
+    def setDefaultWhitespaceChars( chars ):
+        r"""
+        Overrides the default whitespace chars
+
+        Example::
+            # default whitespace chars are space,  and newline
+            OneOrMore(Word(alphas)).parseString("abc def\nghi jkl")  # -> ['abc', 'def', 'ghi', 'jkl']
+            
+            # change to just treat newline as significant
+            ParserElement.setDefaultWhitespaceChars(" \t")
+            OneOrMore(Word(alphas)).parseString("abc def\nghi jkl")  # -> ['abc', 'def']
+        """
+        ParserElement.DEFAULT_WHITE_CHARS = chars
+
+    @staticmethod
+    def inlineLiteralsUsing(cls):
+        """
+        Set class to be used for inclusion of string literals into a parser.
+        
+        Example::
+            # default literal class used is Literal
+            integer = Word(nums)
+            date_str = integer("year") + '/' + integer("month") + '/' + integer("day")           
+
+            date_str.parseString("1999/12/31")  # -> ['1999', '/', '12', '/', '31']
+
+
+            # change to Suppress
+            ParserElement.inlineLiteralsUsing(Suppress)
+            date_str = integer("year") + '/' + integer("month") + '/' + integer("day")           
+
+            date_str.parseString("1999/12/31")  # -> ['1999', '12', '31']
+        """
+        ParserElement._literalStringClass = cls
+
+    def __init__( self, savelist=False ):
+        self.parseAction = list()
+        self.failAction = None
+        #~ self.name = ""  # don't define self.name, let subclasses try/except upcall
+        self.strRepr = None
+        self.resultsName = None
+        self.saveAsList = savelist
+        self.skipWhitespace = True
+        self.whiteChars = ParserElement.DEFAULT_WHITE_CHARS
+        self.copyDefaultWhiteChars = True
+        self.mayReturnEmpty = False # used when checking for left-recursion
+        self.keepTabs = False
+        self.ignoreExprs = list()
+        self.debug = False
+        self.streamlined = False
+        self.mayIndexError = True # used to optimize exception handling for subclasses that don't advance parse index
+        self.errmsg = ""
+        self.modalResults = True # used to mark results names as modal (report only last) or cumulative (list all)
+        self.debugActions = ( None, None, None ) #custom debug actions
+        self.re = None
+        self.callPreparse = True # used to avoid redundant calls to preParse
+        self.callDuringTry = False
+
+    def copy( self ):
+        """
+        Make a copy of this C{ParserElement}.  Useful for defining different parse actions
+        for the same parsing pattern, using copies of the original parse element.
+        
+        Example::
+            integer = Word(nums).setParseAction(lambda toks: int(toks[0]))
+            integerK = integer.copy().addParseAction(lambda toks: toks[0]*1024) + Suppress("K")
+            integerM = integer.copy().addParseAction(lambda toks: toks[0]*1024*1024) + Suppress("M")
+            
+            print(OneOrMore(integerK | integerM | integer).parseString("5K 100 640K 256M"))
+        prints::
+            [5120, 100, 655360, 268435456]
+        Equivalent form of C{expr.copy()} is just C{expr()}::
+            integerM = integer().addParseAction(lambda toks: toks[0]*1024*1024) + Suppress("M")
+        """
+        cpy = copy.copy( self )
+        cpy.parseAction = self.parseAction[:]
+        cpy.ignoreExprs = self.ignoreExprs[:]
+        if self.copyDefaultWhiteChars:
+            cpy.whiteChars = ParserElement.DEFAULT_WHITE_CHARS
+        return cpy
+
+    def setName( self, name ):
+        """
+        Define name for this expression, makes debugging and exception messages clearer.
+        
+        Example::
+            Word(nums).parseString("ABC")  # -> Exception: Expected W:(0123...) (at char 0), (line:1, col:1)
+            Word(nums).setName("integer").parseString("ABC")  # -> Exception: Expected integer (at char 0), (line:1, col:1)
+        """
+        self.name = name
+        self.errmsg = "Expected " + self.name
+        if hasattr(self,"exception"):
+            self.exception.msg = self.errmsg
+        return self
+
+    def setResultsName( self, name, listAllMatches=False ):
+        """
+        Define name for referencing matching tokens as a nested attribute
+        of the returned parse results.
+        NOTE: this returns a *copy* of the original C{ParserElement} object;
+        this is so that the client can define a basic element, such as an
+        integer, and reference it in multiple places with different names.
+
+        You can also set results names using the abbreviated syntax,
+        C{expr("name")} in place of C{expr.setResultsName("name")} - 
+        see L{I{__call__}<__call__>}.
+
+        Example::
+            date_str = (integer.setResultsName("year") + '/' 
+                        + integer.setResultsName("month") + '/' 
+                        + integer.setResultsName("day"))
+
+            # equivalent form:
+            date_str = integer("year") + '/' + integer("month") + '/' + integer("day")
+        """
+        newself = self.copy()
+        if name.endswith("*"):
+            name = name[:-1]
+            listAllMatches=True
+        newself.resultsName = name
+        newself.modalResults = not listAllMatches
+        return newself
+
+    def setBreak(self,breakFlag = True):
+        """Method to invoke the Python pdb debugger when this element is
+           about to be parsed. Set C{breakFlag} to True to enable, False to
+           disable.
+        """
+        if breakFlag:
+            _parseMethod = self._parse
+            def breaker(instring, loc, doActions=True, callPreParse=True):
+                import pdb
+                pdb.set_trace()
+                return _parseMethod( instring, loc, doActions, callPreParse )
+            breaker._originalParseMethod = _parseMethod
+            self._parse = breaker
+        else:
+            if hasattr(self._parse,"_originalParseMethod"):
+                self._parse = self._parse._originalParseMethod
+        return self
+
+    def setParseAction( self, *fns, **kwargs ):
+        """
+        Define one or more actions to perform when successfully matching parse element definition.
+        Parse action fn is a callable method with 0-3 arguments, called as C{fn(s,loc,toks)},
+        C{fn(loc,toks)}, C{fn(toks)}, or just C{fn()}, where:
+         - s   = the original string being parsed (see note below)
+         - loc = the location of the matching substring
+         - toks = a list of the matched tokens, packaged as a C{L{ParseResults}} object
+        If the functions in fns modify the tokens, they can return them as the return
+        value from fn, and the modified list of tokens will replace the original.
+        Otherwise, fn does not need to return any value.
+
+        Optional keyword arguments:
+         - callDuringTry = (default=C{False}) indicate if parse action should be run during lookaheads and alternate testing
+
+        Note: the default parsing behavior is to expand tabs in the input string
+        before starting the parsing process.  See L{I{parseString}} for more information
+        on parsing strings containing C{}s, and suggested methods to maintain a
+        consistent view of the parsed string, the parse location, and line and column
+        positions within the parsed string.
+        
+        Example::
+            integer = Word(nums)
+            date_str = integer + '/' + integer + '/' + integer
+
+            date_str.parseString("1999/12/31")  # -> ['1999', '/', '12', '/', '31']
+
+            # use parse action to convert to ints at parse time
+            integer = Word(nums).setParseAction(lambda toks: int(toks[0]))
+            date_str = integer + '/' + integer + '/' + integer
+
+            # note that integer fields are now ints, not strings
+            date_str.parseString("1999/12/31")  # -> [1999, '/', 12, '/', 31]
+        """
+        self.parseAction = list(map(_trim_arity, list(fns)))
+        self.callDuringTry = kwargs.get("callDuringTry", False)
+        return self
+
+    def addParseAction( self, *fns, **kwargs ):
+        """
+        Add one or more parse actions to expression's list of parse actions. See L{I{setParseAction}}.
+        
+        See examples in L{I{copy}}.
+        """
+        self.parseAction += list(map(_trim_arity, list(fns)))
+        self.callDuringTry = self.callDuringTry or kwargs.get("callDuringTry", False)
+        return self
+
+    def addCondition(self, *fns, **kwargs):
+        """Add a boolean predicate function to expression's list of parse actions. See 
+        L{I{setParseAction}} for function call signatures. Unlike C{setParseAction}, 
+        functions passed to C{addCondition} need to return boolean success/fail of the condition.
+
+        Optional keyword arguments:
+         - message = define a custom message to be used in the raised exception
+         - fatal   = if True, will raise ParseFatalException to stop parsing immediately; otherwise will raise ParseException
+         
+        Example::
+            integer = Word(nums).setParseAction(lambda toks: int(toks[0]))
+            year_int = integer.copy()
+            year_int.addCondition(lambda toks: toks[0] >= 2000, message="Only support years 2000 and later")
+            date_str = year_int + '/' + integer + '/' + integer
+
+            result = date_str.parseString("1999/12/31")  # -> Exception: Only support years 2000 and later (at char 0), (line:1, col:1)
+        """
+        msg = kwargs.get("message", "failed user-defined condition")
+        exc_type = ParseFatalException if kwargs.get("fatal", False) else ParseException
+        for fn in fns:
+            def pa(s,l,t):
+                if not bool(_trim_arity(fn)(s,l,t)):
+                    raise exc_type(s,l,msg)
+            self.parseAction.append(pa)
+        self.callDuringTry = self.callDuringTry or kwargs.get("callDuringTry", False)
+        return self
+
+    def setFailAction( self, fn ):
+        """Define action to perform if parsing fails at this expression.
+           Fail acton fn is a callable function that takes the arguments
+           C{fn(s,loc,expr,err)} where:
+            - s = string being parsed
+            - loc = location where expression match was attempted and failed
+            - expr = the parse expression that failed
+            - err = the exception thrown
+           The function returns no value.  It may throw C{L{ParseFatalException}}
+           if it is desired to stop parsing immediately."""
+        self.failAction = fn
+        return self
+
+    def _skipIgnorables( self, instring, loc ):
+        exprsFound = True
+        while exprsFound:
+            exprsFound = False
+            for e in self.ignoreExprs:
+                try:
+                    while 1:
+                        loc,dummy = e._parse( instring, loc )
+                        exprsFound = True
+                except ParseException:
+                    pass
+        return loc
+
+    def preParse( self, instring, loc ):
+        if self.ignoreExprs:
+            loc = self._skipIgnorables( instring, loc )
+
+        if self.skipWhitespace:
+            wt = self.whiteChars
+            instrlen = len(instring)
+            while loc < instrlen and instring[loc] in wt:
+                loc += 1
+
+        return loc
+
+    def parseImpl( self, instring, loc, doActions=True ):
+        return loc, []
+
+    def postParse( self, instring, loc, tokenlist ):
+        return tokenlist
+
+    #~ @profile
+    def _parseNoCache( self, instring, loc, doActions=True, callPreParse=True ):
+        debugging = ( self.debug ) #and doActions )
+
+        if debugging or self.failAction:
+            #~ print ("Match",self,"at loc",loc,"(%d,%d)" % ( lineno(loc,instring), col(loc,instring) ))
+            if (self.debugActions[0] ):
+                self.debugActions[0]( instring, loc, self )
+            if callPreParse and self.callPreparse:
+                preloc = self.preParse( instring, loc )
+            else:
+                preloc = loc
+            tokensStart = preloc
+            try:
+                try:
+                    loc,tokens = self.parseImpl( instring, preloc, doActions )
+                except IndexError:
+                    raise ParseException( instring, len(instring), self.errmsg, self )
+            except ParseBaseException as err:
+                #~ print ("Exception raised:", err)
+                if self.debugActions[2]:
+                    self.debugActions[2]( instring, tokensStart, self, err )
+                if self.failAction:
+                    self.failAction( instring, tokensStart, self, err )
+                raise
+        else:
+            if callPreParse and self.callPreparse:
+                preloc = self.preParse( instring, loc )
+            else:
+                preloc = loc
+            tokensStart = preloc
+            if self.mayIndexError or preloc >= len(instring):
+                try:
+                    loc,tokens = self.parseImpl( instring, preloc, doActions )
+                except IndexError:
+                    raise ParseException( instring, len(instring), self.errmsg, self )
+            else:
+                loc,tokens = self.parseImpl( instring, preloc, doActions )
+
+        tokens = self.postParse( instring, loc, tokens )
+
+        retTokens = ParseResults( tokens, self.resultsName, asList=self.saveAsList, modal=self.modalResults )
+        if self.parseAction and (doActions or self.callDuringTry):
+            if debugging:
+                try:
+                    for fn in self.parseAction:
+                        tokens = fn( instring, tokensStart, retTokens )
+                        if tokens is not None:
+                            retTokens = ParseResults( tokens,
+                                                      self.resultsName,
+                                                      asList=self.saveAsList and isinstance(tokens,(ParseResults,list)),
+                                                      modal=self.modalResults )
+                except ParseBaseException as err:
+                    #~ print "Exception raised in user parse action:", err
+                    if (self.debugActions[2] ):
+                        self.debugActions[2]( instring, tokensStart, self, err )
+                    raise
+            else:
+                for fn in self.parseAction:
+                    tokens = fn( instring, tokensStart, retTokens )
+                    if tokens is not None:
+                        retTokens = ParseResults( tokens,
+                                                  self.resultsName,
+                                                  asList=self.saveAsList and isinstance(tokens,(ParseResults,list)),
+                                                  modal=self.modalResults )
+        if debugging:
+            #~ print ("Matched",self,"->",retTokens.asList())
+            if (self.debugActions[1] ):
+                self.debugActions[1]( instring, tokensStart, loc, self, retTokens )
+
+        return loc, retTokens
+
+    def tryParse( self, instring, loc ):
+        try:
+            return self._parse( instring, loc, doActions=False )[0]
+        except ParseFatalException:
+            raise ParseException( instring, loc, self.errmsg, self)
+    
+    def canParseNext(self, instring, loc):
+        try:
+            self.tryParse(instring, loc)
+        except (ParseException, IndexError):
+            return False
+        else:
+            return True
+
+    class _UnboundedCache(object):
+        def __init__(self):
+            cache = {}
+            self.not_in_cache = not_in_cache = object()
+
+            def get(self, key):
+                return cache.get(key, not_in_cache)
+
+            def set(self, key, value):
+                cache[key] = value
+
+            def clear(self):
+                cache.clear()
+                
+            def cache_len(self):
+                return len(cache)
+
+            self.get = types.MethodType(get, self)
+            self.set = types.MethodType(set, self)
+            self.clear = types.MethodType(clear, self)
+            self.__len__ = types.MethodType(cache_len, self)
+
+    if _OrderedDict is not None:
+        class _FifoCache(object):
+            def __init__(self, size):
+                self.not_in_cache = not_in_cache = object()
+
+                cache = _OrderedDict()
+
+                def get(self, key):
+                    return cache.get(key, not_in_cache)
+
+                def set(self, key, value):
+                    cache[key] = value
+                    while len(cache) > size:
+                        try:
+                            cache.popitem(False)
+                        except KeyError:
+                            pass
+
+                def clear(self):
+                    cache.clear()
+
+                def cache_len(self):
+                    return len(cache)
+
+                self.get = types.MethodType(get, self)
+                self.set = types.MethodType(set, self)
+                self.clear = types.MethodType(clear, self)
+                self.__len__ = types.MethodType(cache_len, self)
+
+    else:
+        class _FifoCache(object):
+            def __init__(self, size):
+                self.not_in_cache = not_in_cache = object()
+
+                cache = {}
+                key_fifo = collections.deque([], size)
+
+                def get(self, key):
+                    return cache.get(key, not_in_cache)
+
+                def set(self, key, value):
+                    cache[key] = value
+                    while len(key_fifo) > size:
+                        cache.pop(key_fifo.popleft(), None)
+                    key_fifo.append(key)
+
+                def clear(self):
+                    cache.clear()
+                    key_fifo.clear()
+
+                def cache_len(self):
+                    return len(cache)
+
+                self.get = types.MethodType(get, self)
+                self.set = types.MethodType(set, self)
+                self.clear = types.MethodType(clear, self)
+                self.__len__ = types.MethodType(cache_len, self)
+
+    # argument cache for optimizing repeated calls when backtracking through recursive expressions
+    packrat_cache = {} # this is set later by enabledPackrat(); this is here so that resetCache() doesn't fail
+    packrat_cache_lock = RLock()
+    packrat_cache_stats = [0, 0]
+
+    # this method gets repeatedly called during backtracking with the same arguments -
+    # we can cache these arguments and save ourselves the trouble of re-parsing the contained expression
+    def _parseCache( self, instring, loc, doActions=True, callPreParse=True ):
+        HIT, MISS = 0, 1
+        lookup = (self, instring, loc, callPreParse, doActions)
+        with ParserElement.packrat_cache_lock:
+            cache = ParserElement.packrat_cache
+            value = cache.get(lookup)
+            if value is cache.not_in_cache:
+                ParserElement.packrat_cache_stats[MISS] += 1
+                try:
+                    value = self._parseNoCache(instring, loc, doActions, callPreParse)
+                except ParseBaseException as pe:
+                    # cache a copy of the exception, without the traceback
+                    cache.set(lookup, pe.__class__(*pe.args))
+                    raise
+                else:
+                    cache.set(lookup, (value[0], value[1].copy()))
+                    return value
+            else:
+                ParserElement.packrat_cache_stats[HIT] += 1
+                if isinstance(value, Exception):
+                    raise value
+                return (value[0], value[1].copy())
+
+    _parse = _parseNoCache
+
+    @staticmethod
+    def resetCache():
+        ParserElement.packrat_cache.clear()
+        ParserElement.packrat_cache_stats[:] = [0] * len(ParserElement.packrat_cache_stats)
+
+    _packratEnabled = False
+    @staticmethod
+    def enablePackrat(cache_size_limit=128):
+        """Enables "packrat" parsing, which adds memoizing to the parsing logic.
+           Repeated parse attempts at the same string location (which happens
+           often in many complex grammars) can immediately return a cached value,
+           instead of re-executing parsing/validating code.  Memoizing is done of
+           both valid results and parsing exceptions.
+           
+           Parameters:
+            - cache_size_limit - (default=C{128}) - if an integer value is provided
+              will limit the size of the packrat cache; if None is passed, then
+              the cache size will be unbounded; if 0 is passed, the cache will
+              be effectively disabled.
+            
+           This speedup may break existing programs that use parse actions that
+           have side-effects.  For this reason, packrat parsing is disabled when
+           you first import pyparsing.  To activate the packrat feature, your
+           program must call the class method C{ParserElement.enablePackrat()}.  If
+           your program uses C{psyco} to "compile as you go", you must call
+           C{enablePackrat} before calling C{psyco.full()}.  If you do not do this,
+           Python will crash.  For best results, call C{enablePackrat()} immediately
+           after importing pyparsing.
+           
+           Example::
+               import pyparsing
+               pyparsing.ParserElement.enablePackrat()
+        """
+        if not ParserElement._packratEnabled:
+            ParserElement._packratEnabled = True
+            if cache_size_limit is None:
+                ParserElement.packrat_cache = ParserElement._UnboundedCache()
+            else:
+                ParserElement.packrat_cache = ParserElement._FifoCache(cache_size_limit)
+            ParserElement._parse = ParserElement._parseCache
+
+    def parseString( self, instring, parseAll=False ):
+        """
+        Execute the parse expression with the given string.
+        This is the main interface to the client code, once the complete
+        expression has been built.
+
+        If you want the grammar to require that the entire input string be
+        successfully parsed, then set C{parseAll} to True (equivalent to ending
+        the grammar with C{L{StringEnd()}}).
+
+        Note: C{parseString} implicitly calls C{expandtabs()} on the input string,
+        in order to report proper column numbers in parse actions.
+        If the input string contains tabs and
+        the grammar uses parse actions that use the C{loc} argument to index into the
+        string being parsed, you can ensure you have a consistent view of the input
+        string by:
+         - calling C{parseWithTabs} on your grammar before calling C{parseString}
+           (see L{I{parseWithTabs}})
+         - define your parse action using the full C{(s,loc,toks)} signature, and
+           reference the input string using the parse action's C{s} argument
+         - explictly expand the tabs in your input string before calling
+           C{parseString}
+        
+        Example::
+            Word('a').parseString('aaaaabaaa')  # -> ['aaaaa']
+            Word('a').parseString('aaaaabaaa', parseAll=True)  # -> Exception: Expected end of text
+        """
+        ParserElement.resetCache()
+        if not self.streamlined:
+            self.streamline()
+            #~ self.saveAsList = True
+        for e in self.ignoreExprs:
+            e.streamline()
+        if not self.keepTabs:
+            instring = instring.expandtabs()
+        try:
+            loc, tokens = self._parse( instring, 0 )
+            if parseAll:
+                loc = self.preParse( instring, loc )
+                se = Empty() + StringEnd()
+                se._parse( instring, loc )
+        except ParseBaseException as exc:
+            if ParserElement.verbose_stacktrace:
+                raise
+            else:
+                # catch and re-raise exception from here, clears out pyparsing internal stack trace
+                raise exc
+        else:
+            return tokens
+
+    def scanString( self, instring, maxMatches=_MAX_INT, overlap=False ):
+        """
+        Scan the input string for expression matches.  Each match will return the
+        matching tokens, start location, and end location.  May be called with optional
+        C{maxMatches} argument, to clip scanning after 'n' matches are found.  If
+        C{overlap} is specified, then overlapping matches will be reported.
+
+        Note that the start and end locations are reported relative to the string
+        being parsed.  See L{I{parseString}} for more information on parsing
+        strings with embedded tabs.
+
+        Example::
+            source = "sldjf123lsdjjkf345sldkjf879lkjsfd987"
+            print(source)
+            for tokens,start,end in Word(alphas).scanString(source):
+                print(' '*start + '^'*(end-start))
+                print(' '*start + tokens[0])
+        
+        prints::
+        
+            sldjf123lsdjjkf345sldkjf879lkjsfd987
+            ^^^^^
+            sldjf
+                    ^^^^^^^
+                    lsdjjkf
+                              ^^^^^^
+                              sldkjf
+                                       ^^^^^^
+                                       lkjsfd
+        """
+        if not self.streamlined:
+            self.streamline()
+        for e in self.ignoreExprs:
+            e.streamline()
+
+        if not self.keepTabs:
+            instring = _ustr(instring).expandtabs()
+        instrlen = len(instring)
+        loc = 0
+        preparseFn = self.preParse
+        parseFn = self._parse
+        ParserElement.resetCache()
+        matches = 0
+        try:
+            while loc <= instrlen and matches < maxMatches:
+                try:
+                    preloc = preparseFn( instring, loc )
+                    nextLoc,tokens = parseFn( instring, preloc, callPreParse=False )
+                except ParseException:
+                    loc = preloc+1
+                else:
+                    if nextLoc > loc:
+                        matches += 1
+                        yield tokens, preloc, nextLoc
+                        if overlap:
+                            nextloc = preparseFn( instring, loc )
+                            if nextloc > loc:
+                                loc = nextLoc
+                            else:
+                                loc += 1
+                        else:
+                            loc = nextLoc
+                    else:
+                        loc = preloc+1
+        except ParseBaseException as exc:
+            if ParserElement.verbose_stacktrace:
+                raise
+            else:
+                # catch and re-raise exception from here, clears out pyparsing internal stack trace
+                raise exc
+
+    def transformString( self, instring ):
+        """
+        Extension to C{L{scanString}}, to modify matching text with modified tokens that may
+        be returned from a parse action.  To use C{transformString}, define a grammar and
+        attach a parse action to it that modifies the returned token list.
+        Invoking C{transformString()} on a target string will then scan for matches,
+        and replace the matched text patterns according to the logic in the parse
+        action.  C{transformString()} returns the resulting transformed string.
+        
+        Example::
+            wd = Word(alphas)
+            wd.setParseAction(lambda toks: toks[0].title())
+            
+            print(wd.transformString("now is the winter of our discontent made glorious summer by this sun of york."))
+        Prints::
+            Now Is The Winter Of Our Discontent Made Glorious Summer By This Sun Of York.
+        """
+        out = []
+        lastE = 0
+        # force preservation of s, to minimize unwanted transformation of string, and to
+        # keep string locs straight between transformString and scanString
+        self.keepTabs = True
+        try:
+            for t,s,e in self.scanString( instring ):
+                out.append( instring[lastE:s] )
+                if t:
+                    if isinstance(t,ParseResults):
+                        out += t.asList()
+                    elif isinstance(t,list):
+                        out += t
+                    else:
+                        out.append(t)
+                lastE = e
+            out.append(instring[lastE:])
+            out = [o for o in out if o]
+            return "".join(map(_ustr,_flatten(out)))
+        except ParseBaseException as exc:
+            if ParserElement.verbose_stacktrace:
+                raise
+            else:
+                # catch and re-raise exception from here, clears out pyparsing internal stack trace
+                raise exc
+
+    def searchString( self, instring, maxMatches=_MAX_INT ):
+        """
+        Another extension to C{L{scanString}}, simplifying the access to the tokens found
+        to match the given parse expression.  May be called with optional
+        C{maxMatches} argument, to clip searching after 'n' matches are found.
+        
+        Example::
+            # a capitalized word starts with an uppercase letter, followed by zero or more lowercase letters
+            cap_word = Word(alphas.upper(), alphas.lower())
+            
+            print(cap_word.searchString("More than Iron, more than Lead, more than Gold I need Electricity"))
+
+            # the sum() builtin can be used to merge results into a single ParseResults object
+            print(sum(cap_word.searchString("More than Iron, more than Lead, more than Gold I need Electricity")))
+        prints::
+            [['More'], ['Iron'], ['Lead'], ['Gold'], ['I'], ['Electricity']]
+            ['More', 'Iron', 'Lead', 'Gold', 'I', 'Electricity']
+        """
+        try:
+            return ParseResults([ t for t,s,e in self.scanString( instring, maxMatches ) ])
+        except ParseBaseException as exc:
+            if ParserElement.verbose_stacktrace:
+                raise
+            else:
+                # catch and re-raise exception from here, clears out pyparsing internal stack trace
+                raise exc
+
+    def split(self, instring, maxsplit=_MAX_INT, includeSeparators=False):
+        """
+        Generator method to split a string using the given expression as a separator.
+        May be called with optional C{maxsplit} argument, to limit the number of splits;
+        and the optional C{includeSeparators} argument (default=C{False}), if the separating
+        matching text should be included in the split results.
+        
+        Example::        
+            punc = oneOf(list(".,;:/-!?"))
+            print(list(punc.split("This, this?, this sentence, is badly punctuated!")))
+        prints::
+            ['This', ' this', '', ' this sentence', ' is badly punctuated', '']
+        """
+        splits = 0
+        last = 0
+        for t,s,e in self.scanString(instring, maxMatches=maxsplit):
+            yield instring[last:s]
+            if includeSeparators:
+                yield t[0]
+            last = e
+        yield instring[last:]
+
+    def __add__(self, other ):
+        """
+        Implementation of + operator - returns C{L{And}}. Adding strings to a ParserElement
+        converts them to L{Literal}s by default.
+        
+        Example::
+            greet = Word(alphas) + "," + Word(alphas) + "!"
+            hello = "Hello, World!"
+            print (hello, "->", greet.parseString(hello))
+        Prints::
+            Hello, World! -> ['Hello', ',', 'World', '!']
+        """
+        if isinstance( other, basestring ):
+            other = ParserElement._literalStringClass( other )
+        if not isinstance( other, ParserElement ):
+            warnings.warn("Cannot combine element of type %s with ParserElement" % type(other),
+                    SyntaxWarning, stacklevel=2)
+            return None
+        return And( [ self, other ] )
+
+    def __radd__(self, other ):
+        """
+        Implementation of + operator when left operand is not a C{L{ParserElement}}
+        """
+        if isinstance( other, basestring ):
+            other = ParserElement._literalStringClass( other )
+        if not isinstance( other, ParserElement ):
+            warnings.warn("Cannot combine element of type %s with ParserElement" % type(other),
+                    SyntaxWarning, stacklevel=2)
+            return None
+        return other + self
+
+    def __sub__(self, other):
+        """
+        Implementation of - operator, returns C{L{And}} with error stop
+        """
+        if isinstance( other, basestring ):
+            other = ParserElement._literalStringClass( other )
+        if not isinstance( other, ParserElement ):
+            warnings.warn("Cannot combine element of type %s with ParserElement" % type(other),
+                    SyntaxWarning, stacklevel=2)
+            return None
+        return self + And._ErrorStop() + other
+
+    def __rsub__(self, other ):
+        """
+        Implementation of - operator when left operand is not a C{L{ParserElement}}
+        """
+        if isinstance( other, basestring ):
+            other = ParserElement._literalStringClass( other )
+        if not isinstance( other, ParserElement ):
+            warnings.warn("Cannot combine element of type %s with ParserElement" % type(other),
+                    SyntaxWarning, stacklevel=2)
+            return None
+        return other - self
+
+    def __mul__(self,other):
+        """
+        Implementation of * operator, allows use of C{expr * 3} in place of
+        C{expr + expr + expr}.  Expressions may also me multiplied by a 2-integer
+        tuple, similar to C{{min,max}} multipliers in regular expressions.  Tuples
+        may also include C{None} as in:
+         - C{expr*(n,None)} or C{expr*(n,)} is equivalent
+              to C{expr*n + L{ZeroOrMore}(expr)}
+              (read as "at least n instances of C{expr}")
+         - C{expr*(None,n)} is equivalent to C{expr*(0,n)}
+              (read as "0 to n instances of C{expr}")
+         - C{expr*(None,None)} is equivalent to C{L{ZeroOrMore}(expr)}
+         - C{expr*(1,None)} is equivalent to C{L{OneOrMore}(expr)}
+
+        Note that C{expr*(None,n)} does not raise an exception if
+        more than n exprs exist in the input stream; that is,
+        C{expr*(None,n)} does not enforce a maximum number of expr
+        occurrences.  If this behavior is desired, then write
+        C{expr*(None,n) + ~expr}
+        """
+        if isinstance(other,int):
+            minElements, optElements = other,0
+        elif isinstance(other,tuple):
+            other = (other + (None, None))[:2]
+            if other[0] is None:
+                other = (0, other[1])
+            if isinstance(other[0],int) and other[1] is None:
+                if other[0] == 0:
+                    return ZeroOrMore(self)
+                if other[0] == 1:
+                    return OneOrMore(self)
+                else:
+                    return self*other[0] + ZeroOrMore(self)
+            elif isinstance(other[0],int) and isinstance(other[1],int):
+                minElements, optElements = other
+                optElements -= minElements
+            else:
+                raise TypeError("cannot multiply 'ParserElement' and ('%s','%s') objects", type(other[0]),type(other[1]))
+        else:
+            raise TypeError("cannot multiply 'ParserElement' and '%s' objects", type(other))
+
+        if minElements < 0:
+            raise ValueError("cannot multiply ParserElement by negative value")
+        if optElements < 0:
+            raise ValueError("second tuple value must be greater or equal to first tuple value")
+        if minElements == optElements == 0:
+            raise ValueError("cannot multiply ParserElement by 0 or (0,0)")
+
+        if (optElements):
+            def makeOptionalList(n):
+                if n>1:
+                    return Optional(self + makeOptionalList(n-1))
+                else:
+                    return Optional(self)
+            if minElements:
+                if minElements == 1:
+                    ret = self + makeOptionalList(optElements)
+                else:
+                    ret = And([self]*minElements) + makeOptionalList(optElements)
+            else:
+                ret = makeOptionalList(optElements)
+        else:
+            if minElements == 1:
+                ret = self
+            else:
+                ret = And([self]*minElements)
+        return ret
+
+    def __rmul__(self, other):
+        return self.__mul__(other)
+
+    def __or__(self, other ):
+        """
+        Implementation of | operator - returns C{L{MatchFirst}}
+        """
+        if isinstance( other, basestring ):
+            other = ParserElement._literalStringClass( other )
+        if not isinstance( other, ParserElement ):
+            warnings.warn("Cannot combine element of type %s with ParserElement" % type(other),
+                    SyntaxWarning, stacklevel=2)
+            return None
+        return MatchFirst( [ self, other ] )
+
+    def __ror__(self, other ):
+        """
+        Implementation of | operator when left operand is not a C{L{ParserElement}}
+        """
+        if isinstance( other, basestring ):
+            other = ParserElement._literalStringClass( other )
+        if not isinstance( other, ParserElement ):
+            warnings.warn("Cannot combine element of type %s with ParserElement" % type(other),
+                    SyntaxWarning, stacklevel=2)
+            return None
+        return other | self
+
+    def __xor__(self, other ):
+        """
+        Implementation of ^ operator - returns C{L{Or}}
+        """
+        if isinstance( other, basestring ):
+            other = ParserElement._literalStringClass( other )
+        if not isinstance( other, ParserElement ):
+            warnings.warn("Cannot combine element of type %s with ParserElement" % type(other),
+                    SyntaxWarning, stacklevel=2)
+            return None
+        return Or( [ self, other ] )
+
+    def __rxor__(self, other ):
+        """
+        Implementation of ^ operator when left operand is not a C{L{ParserElement}}
+        """
+        if isinstance( other, basestring ):
+            other = ParserElement._literalStringClass( other )
+        if not isinstance( other, ParserElement ):
+            warnings.warn("Cannot combine element of type %s with ParserElement" % type(other),
+                    SyntaxWarning, stacklevel=2)
+            return None
+        return other ^ self
+
+    def __and__(self, other ):
+        """
+        Implementation of & operator - returns C{L{Each}}
+        """
+        if isinstance( other, basestring ):
+            other = ParserElement._literalStringClass( other )
+        if not isinstance( other, ParserElement ):
+            warnings.warn("Cannot combine element of type %s with ParserElement" % type(other),
+                    SyntaxWarning, stacklevel=2)
+            return None
+        return Each( [ self, other ] )
+
+    def __rand__(self, other ):
+        """
+        Implementation of & operator when left operand is not a C{L{ParserElement}}
+        """
+        if isinstance( other, basestring ):
+            other = ParserElement._literalStringClass( other )
+        if not isinstance( other, ParserElement ):
+            warnings.warn("Cannot combine element of type %s with ParserElement" % type(other),
+                    SyntaxWarning, stacklevel=2)
+            return None
+        return other & self
+
+    def __invert__( self ):
+        """
+        Implementation of ~ operator - returns C{L{NotAny}}
+        """
+        return NotAny( self )
+
+    def __call__(self, name=None):
+        """
+        Shortcut for C{L{setResultsName}}, with C{listAllMatches=False}.
+        
+        If C{name} is given with a trailing C{'*'} character, then C{listAllMatches} will be
+        passed as C{True}.
+           
+        If C{name} is omitted, same as calling C{L{copy}}.
+
+        Example::
+            # these are equivalent
+            userdata = Word(alphas).setResultsName("name") + Word(nums+"-").setResultsName("socsecno")
+            userdata = Word(alphas)("name") + Word(nums+"-")("socsecno")             
+        """
+        if name is not None:
+            return self.setResultsName(name)
+        else:
+            return self.copy()
+
+    def suppress( self ):
+        """
+        Suppresses the output of this C{ParserElement}; useful to keep punctuation from
+        cluttering up returned output.
+        """
+        return Suppress( self )
+
+    def leaveWhitespace( self ):
+        """
+        Disables the skipping of whitespace before matching the characters in the
+        C{ParserElement}'s defined pattern.  This is normally only used internally by
+        the pyparsing module, but may be needed in some whitespace-sensitive grammars.
+        """
+        self.skipWhitespace = False
+        return self
+
+    def setWhitespaceChars( self, chars ):
+        """
+        Overrides the default whitespace chars
+        """
+        self.skipWhitespace = True
+        self.whiteChars = chars
+        self.copyDefaultWhiteChars = False
+        return self
+
+    def parseWithTabs( self ):
+        """
+        Overrides default behavior to expand C{}s to spaces before parsing the input string.
+        Must be called before C{parseString} when the input grammar contains elements that
+        match C{} characters.
+        """
+        self.keepTabs = True
+        return self
+
+    def ignore( self, other ):
+        """
+        Define expression to be ignored (e.g., comments) while doing pattern
+        matching; may be called repeatedly, to define multiple comment or other
+        ignorable patterns.
+        
+        Example::
+            patt = OneOrMore(Word(alphas))
+            patt.parseString('ablaj /* comment */ lskjd') # -> ['ablaj']
+            
+            patt.ignore(cStyleComment)
+            patt.parseString('ablaj /* comment */ lskjd') # -> ['ablaj', 'lskjd']
+        """
+        if isinstance(other, basestring):
+            other = Suppress(other)
+
+        if isinstance( other, Suppress ):
+            if other not in self.ignoreExprs:
+                self.ignoreExprs.append(other)
+        else:
+            self.ignoreExprs.append( Suppress( other.copy() ) )
+        return self
+
+    def setDebugActions( self, startAction, successAction, exceptionAction ):
+        """
+        Enable display of debugging messages while doing pattern matching.
+        """
+        self.debugActions = (startAction or _defaultStartDebugAction,
+                             successAction or _defaultSuccessDebugAction,
+                             exceptionAction or _defaultExceptionDebugAction)
+        self.debug = True
+        return self
+
+    def setDebug( self, flag=True ):
+        """
+        Enable display of debugging messages while doing pattern matching.
+        Set C{flag} to True to enable, False to disable.
+
+        Example::
+            wd = Word(alphas).setName("alphaword")
+            integer = Word(nums).setName("numword")
+            term = wd | integer
+            
+            # turn on debugging for wd
+            wd.setDebug()
+
+            OneOrMore(term).parseString("abc 123 xyz 890")
+        
+        prints::
+            Match alphaword at loc 0(1,1)
+            Matched alphaword -> ['abc']
+            Match alphaword at loc 3(1,4)
+            Exception raised:Expected alphaword (at char 4), (line:1, col:5)
+            Match alphaword at loc 7(1,8)
+            Matched alphaword -> ['xyz']
+            Match alphaword at loc 11(1,12)
+            Exception raised:Expected alphaword (at char 12), (line:1, col:13)
+            Match alphaword at loc 15(1,16)
+            Exception raised:Expected alphaword (at char 15), (line:1, col:16)
+
+        The output shown is that produced by the default debug actions - custom debug actions can be
+        specified using L{setDebugActions}. Prior to attempting
+        to match the C{wd} expression, the debugging message C{"Match  at loc (,)"}
+        is shown. Then if the parse succeeds, a C{"Matched"} message is shown, or an C{"Exception raised"}
+        message is shown. Also note the use of L{setName} to assign a human-readable name to the expression,
+        which makes debugging and exception messages easier to understand - for instance, the default
+        name created for the C{Word} expression without calling C{setName} is C{"W:(ABCD...)"}.
+        """
+        if flag:
+            self.setDebugActions( _defaultStartDebugAction, _defaultSuccessDebugAction, _defaultExceptionDebugAction )
+        else:
+            self.debug = False
+        return self
+
+    def __str__( self ):
+        return self.name
+
+    def __repr__( self ):
+        return _ustr(self)
+
+    def streamline( self ):
+        self.streamlined = True
+        self.strRepr = None
+        return self
+
+    def checkRecursion( self, parseElementList ):
+        pass
+
+    def validate( self, validateTrace=[] ):
+        """
+        Check defined expressions for valid structure, check for infinite recursive definitions.
+        """
+        self.checkRecursion( [] )
+
+    def parseFile( self, file_or_filename, parseAll=False ):
+        """
+        Execute the parse expression on the given file or filename.
+        If a filename is specified (instead of a file object),
+        the entire file is opened, read, and closed before parsing.
+        """
+        try:
+            file_contents = file_or_filename.read()
+        except AttributeError:
+            with open(file_or_filename, "r") as f:
+                file_contents = f.read()
+        try:
+            return self.parseString(file_contents, parseAll)
+        except ParseBaseException as exc:
+            if ParserElement.verbose_stacktrace:
+                raise
+            else:
+                # catch and re-raise exception from here, clears out pyparsing internal stack trace
+                raise exc
+
+    def __eq__(self,other):
+        if isinstance(other, ParserElement):
+            return self is other or vars(self) == vars(other)
+        elif isinstance(other, basestring):
+            return self.matches(other)
+        else:
+            return super(ParserElement,self)==other
+
+    def __ne__(self,other):
+        return not (self == other)
+
+    def __hash__(self):
+        return hash(id(self))
+
+    def __req__(self,other):
+        return self == other
+
+    def __rne__(self,other):
+        return not (self == other)
+
+    def matches(self, testString, parseAll=True):
+        """
+        Method for quick testing of a parser against a test string. Good for simple 
+        inline microtests of sub expressions while building up larger parser.
+           
+        Parameters:
+         - testString - to test against this expression for a match
+         - parseAll - (default=C{True}) - flag to pass to C{L{parseString}} when running tests
+            
+        Example::
+            expr = Word(nums)
+            assert expr.matches("100")
+        """
+        try:
+            self.parseString(_ustr(testString), parseAll=parseAll)
+            return True
+        except ParseBaseException:
+            return False
+                
+    def runTests(self, tests, parseAll=True, comment='#', fullDump=True, printResults=True, failureTests=False):
+        """
+        Execute the parse expression on a series of test strings, showing each
+        test, the parsed results or where the parse failed. Quick and easy way to
+        run a parse expression against a list of sample strings.
+           
+        Parameters:
+         - tests - a list of separate test strings, or a multiline string of test strings
+         - parseAll - (default=C{True}) - flag to pass to C{L{parseString}} when running tests           
+         - comment - (default=C{'#'}) - expression for indicating embedded comments in the test 
+              string; pass None to disable comment filtering
+         - fullDump - (default=C{True}) - dump results as list followed by results names in nested outline;
+              if False, only dump nested list
+         - printResults - (default=C{True}) prints test output to stdout
+         - failureTests - (default=C{False}) indicates if these tests are expected to fail parsing
+
+        Returns: a (success, results) tuple, where success indicates that all tests succeeded
+        (or failed if C{failureTests} is True), and the results contain a list of lines of each 
+        test's output
+        
+        Example::
+            number_expr = pyparsing_common.number.copy()
+
+            result = number_expr.runTests('''
+                # unsigned integer
+                100
+                # negative integer
+                -100
+                # float with scientific notation
+                6.02e23
+                # integer with scientific notation
+                1e-12
+                ''')
+            print("Success" if result[0] else "Failed!")
+
+            result = number_expr.runTests('''
+                # stray character
+                100Z
+                # missing leading digit before '.'
+                -.100
+                # too many '.'
+                3.14.159
+                ''', failureTests=True)
+            print("Success" if result[0] else "Failed!")
+        prints::
+            # unsigned integer
+            100
+            [100]
+
+            # negative integer
+            -100
+            [-100]
+
+            # float with scientific notation
+            6.02e23
+            [6.02e+23]
+
+            # integer with scientific notation
+            1e-12
+            [1e-12]
+
+            Success
+            
+            # stray character
+            100Z
+               ^
+            FAIL: Expected end of text (at char 3), (line:1, col:4)
+
+            # missing leading digit before '.'
+            -.100
+            ^
+            FAIL: Expected {real number with scientific notation | real number | signed integer} (at char 0), (line:1, col:1)
+
+            # too many '.'
+            3.14.159
+                ^
+            FAIL: Expected end of text (at char 4), (line:1, col:5)
+
+            Success
+
+        Each test string must be on a single line. If you want to test a string that spans multiple
+        lines, create a test like this::
+
+            expr.runTest(r"this is a test\\n of strings that spans \\n 3 lines")
+        
+        (Note that this is a raw string literal, you must include the leading 'r'.)
+        """
+        if isinstance(tests, basestring):
+            tests = list(map(str.strip, tests.rstrip().splitlines()))
+        if isinstance(comment, basestring):
+            comment = Literal(comment)
+        allResults = []
+        comments = []
+        success = True
+        for t in tests:
+            if comment is not None and comment.matches(t, False) or comments and not t:
+                comments.append(t)
+                continue
+            if not t:
+                continue
+            out = ['\n'.join(comments), t]
+            comments = []
+            try:
+                t = t.replace(r'\n','\n')
+                result = self.parseString(t, parseAll=parseAll)
+                out.append(result.dump(full=fullDump))
+                success = success and not failureTests
+            except ParseBaseException as pe:
+                fatal = "(FATAL)" if isinstance(pe, ParseFatalException) else ""
+                if '\n' in t:
+                    out.append(line(pe.loc, t))
+                    out.append(' '*(col(pe.loc,t)-1) + '^' + fatal)
+                else:
+                    out.append(' '*pe.loc + '^' + fatal)
+                out.append("FAIL: " + str(pe))
+                success = success and failureTests
+                result = pe
+            except Exception as exc:
+                out.append("FAIL-EXCEPTION: " + str(exc))
+                success = success and failureTests
+                result = exc
+
+            if printResults:
+                if fullDump:
+                    out.append('')
+                print('\n'.join(out))
+
+            allResults.append((t, result))
+        
+        return success, allResults
+
+        
+class Token(ParserElement):
+    """
+    Abstract C{ParserElement} subclass, for defining atomic matching patterns.
+    """
+    def __init__( self ):
+        super(Token,self).__init__( savelist=False )
+
+
+class Empty(Token):
+    """
+    An empty token, will always match.
+    """
+    def __init__( self ):
+        super(Empty,self).__init__()
+        self.name = "Empty"
+        self.mayReturnEmpty = True
+        self.mayIndexError = False
+
+
+class NoMatch(Token):
+    """
+    A token that will never match.
+    """
+    def __init__( self ):
+        super(NoMatch,self).__init__()
+        self.name = "NoMatch"
+        self.mayReturnEmpty = True
+        self.mayIndexError = False
+        self.errmsg = "Unmatchable token"
+
+    def parseImpl( self, instring, loc, doActions=True ):
+        raise ParseException(instring, loc, self.errmsg, self)
+
+
+class Literal(Token):
+    """
+    Token to exactly match a specified string.
+    
+    Example::
+        Literal('blah').parseString('blah')  # -> ['blah']
+        Literal('blah').parseString('blahfooblah')  # -> ['blah']
+        Literal('blah').parseString('bla')  # -> Exception: Expected "blah"
+    
+    For case-insensitive matching, use L{CaselessLiteral}.
+    
+    For keyword matching (force word break before and after the matched string),
+    use L{Keyword} or L{CaselessKeyword}.
+    """
+    def __init__( self, matchString ):
+        super(Literal,self).__init__()
+        self.match = matchString
+        self.matchLen = len(matchString)
+        try:
+            self.firstMatchChar = matchString[0]
+        except IndexError:
+            warnings.warn("null string passed to Literal; use Empty() instead",
+                            SyntaxWarning, stacklevel=2)
+            self.__class__ = Empty
+        self.name = '"%s"' % _ustr(self.match)
+        self.errmsg = "Expected " + self.name
+        self.mayReturnEmpty = False
+        self.mayIndexError = False
+
+    # Performance tuning: this routine gets called a *lot*
+    # if this is a single character match string  and the first character matches,
+    # short-circuit as quickly as possible, and avoid calling startswith
+    #~ @profile
+    def parseImpl( self, instring, loc, doActions=True ):
+        if (instring[loc] == self.firstMatchChar and
+            (self.matchLen==1 or instring.startswith(self.match,loc)) ):
+            return loc+self.matchLen, self.match
+        raise ParseException(instring, loc, self.errmsg, self)
+_L = Literal
+ParserElement._literalStringClass = Literal
+
+class Keyword(Token):
+    """
+    Token to exactly match a specified string as a keyword, that is, it must be
+    immediately followed by a non-keyword character.  Compare with C{L{Literal}}:
+     - C{Literal("if")} will match the leading C{'if'} in C{'ifAndOnlyIf'}.
+     - C{Keyword("if")} will not; it will only match the leading C{'if'} in C{'if x=1'}, or C{'if(y==2)'}
+    Accepts two optional constructor arguments in addition to the keyword string:
+     - C{identChars} is a string of characters that would be valid identifier characters,
+          defaulting to all alphanumerics + "_" and "$"
+     - C{caseless} allows case-insensitive matching, default is C{False}.
+       
+    Example::
+        Keyword("start").parseString("start")  # -> ['start']
+        Keyword("start").parseString("starting")  # -> Exception
+
+    For case-insensitive matching, use L{CaselessKeyword}.
+    """
+    DEFAULT_KEYWORD_CHARS = alphanums+"_$"
+
+    def __init__( self, matchString, identChars=None, caseless=False ):
+        super(Keyword,self).__init__()
+        if identChars is None:
+            identChars = Keyword.DEFAULT_KEYWORD_CHARS
+        self.match = matchString
+        self.matchLen = len(matchString)
+        try:
+            self.firstMatchChar = matchString[0]
+        except IndexError:
+            warnings.warn("null string passed to Keyword; use Empty() instead",
+                            SyntaxWarning, stacklevel=2)
+        self.name = '"%s"' % self.match
+        self.errmsg = "Expected " + self.name
+        self.mayReturnEmpty = False
+        self.mayIndexError = False
+        self.caseless = caseless
+        if caseless:
+            self.caselessmatch = matchString.upper()
+            identChars = identChars.upper()
+        self.identChars = set(identChars)
+
+    def parseImpl( self, instring, loc, doActions=True ):
+        if self.caseless:
+            if ( (instring[ loc:loc+self.matchLen ].upper() == self.caselessmatch) and
+                 (loc >= len(instring)-self.matchLen or instring[loc+self.matchLen].upper() not in self.identChars) and
+                 (loc == 0 or instring[loc-1].upper() not in self.identChars) ):
+                return loc+self.matchLen, self.match
+        else:
+            if (instring[loc] == self.firstMatchChar and
+                (self.matchLen==1 or instring.startswith(self.match,loc)) and
+                (loc >= len(instring)-self.matchLen or instring[loc+self.matchLen] not in self.identChars) and
+                (loc == 0 or instring[loc-1] not in self.identChars) ):
+                return loc+self.matchLen, self.match
+        raise ParseException(instring, loc, self.errmsg, self)
+
+    def copy(self):
+        c = super(Keyword,self).copy()
+        c.identChars = Keyword.DEFAULT_KEYWORD_CHARS
+        return c
+
+    @staticmethod
+    def setDefaultKeywordChars( chars ):
+        """Overrides the default Keyword chars
+        """
+        Keyword.DEFAULT_KEYWORD_CHARS = chars
+
+class CaselessLiteral(Literal):
+    """
+    Token to match a specified string, ignoring case of letters.
+    Note: the matched results will always be in the case of the given
+    match string, NOT the case of the input text.
+
+    Example::
+        OneOrMore(CaselessLiteral("CMD")).parseString("cmd CMD Cmd10") # -> ['CMD', 'CMD', 'CMD']
+        
+    (Contrast with example for L{CaselessKeyword}.)
+    """
+    def __init__( self, matchString ):
+        super(CaselessLiteral,self).__init__( matchString.upper() )
+        # Preserve the defining literal.
+        self.returnString = matchString
+        self.name = "'%s'" % self.returnString
+        self.errmsg = "Expected " + self.name
+
+    def parseImpl( self, instring, loc, doActions=True ):
+        if instring[ loc:loc+self.matchLen ].upper() == self.match:
+            return loc+self.matchLen, self.returnString
+        raise ParseException(instring, loc, self.errmsg, self)
+
+class CaselessKeyword(Keyword):
+    """
+    Caseless version of L{Keyword}.
+
+    Example::
+        OneOrMore(CaselessKeyword("CMD")).parseString("cmd CMD Cmd10") # -> ['CMD', 'CMD']
+        
+    (Contrast with example for L{CaselessLiteral}.)
+    """
+    def __init__( self, matchString, identChars=None ):
+        super(CaselessKeyword,self).__init__( matchString, identChars, caseless=True )
+
+    def parseImpl( self, instring, loc, doActions=True ):
+        if ( (instring[ loc:loc+self.matchLen ].upper() == self.caselessmatch) and
+             (loc >= len(instring)-self.matchLen or instring[loc+self.matchLen].upper() not in self.identChars) ):
+            return loc+self.matchLen, self.match
+        raise ParseException(instring, loc, self.errmsg, self)
+
+class CloseMatch(Token):
+    """
+    A variation on L{Literal} which matches "close" matches, that is, 
+    strings with at most 'n' mismatching characters. C{CloseMatch} takes parameters:
+     - C{match_string} - string to be matched
+     - C{maxMismatches} - (C{default=1}) maximum number of mismatches allowed to count as a match
+    
+    The results from a successful parse will contain the matched text from the input string and the following named results:
+     - C{mismatches} - a list of the positions within the match_string where mismatches were found
+     - C{original} - the original match_string used to compare against the input string
+    
+    If C{mismatches} is an empty list, then the match was an exact match.
+    
+    Example::
+        patt = CloseMatch("ATCATCGAATGGA")
+        patt.parseString("ATCATCGAAXGGA") # -> (['ATCATCGAAXGGA'], {'mismatches': [[9]], 'original': ['ATCATCGAATGGA']})
+        patt.parseString("ATCAXCGAAXGGA") # -> Exception: Expected 'ATCATCGAATGGA' (with up to 1 mismatches) (at char 0), (line:1, col:1)
+
+        # exact match
+        patt.parseString("ATCATCGAATGGA") # -> (['ATCATCGAATGGA'], {'mismatches': [[]], 'original': ['ATCATCGAATGGA']})
+
+        # close match allowing up to 2 mismatches
+        patt = CloseMatch("ATCATCGAATGGA", maxMismatches=2)
+        patt.parseString("ATCAXCGAAXGGA") # -> (['ATCAXCGAAXGGA'], {'mismatches': [[4, 9]], 'original': ['ATCATCGAATGGA']})
+    """
+    def __init__(self, match_string, maxMismatches=1):
+        super(CloseMatch,self).__init__()
+        self.name = match_string
+        self.match_string = match_string
+        self.maxMismatches = maxMismatches
+        self.errmsg = "Expected %r (with up to %d mismatches)" % (self.match_string, self.maxMismatches)
+        self.mayIndexError = False
+        self.mayReturnEmpty = False
+
+    def parseImpl( self, instring, loc, doActions=True ):
+        start = loc
+        instrlen = len(instring)
+        maxloc = start + len(self.match_string)
+
+        if maxloc <= instrlen:
+            match_string = self.match_string
+            match_stringloc = 0
+            mismatches = []
+            maxMismatches = self.maxMismatches
+
+            for match_stringloc,s_m in enumerate(zip(instring[loc:maxloc], self.match_string)):
+                src,mat = s_m
+                if src != mat:
+                    mismatches.append(match_stringloc)
+                    if len(mismatches) > maxMismatches:
+                        break
+            else:
+                loc = match_stringloc + 1
+                results = ParseResults([instring[start:loc]])
+                results['original'] = self.match_string
+                results['mismatches'] = mismatches
+                return loc, results
+
+        raise ParseException(instring, loc, self.errmsg, self)
+
+
+class Word(Token):
+    """
+    Token for matching words composed of allowed character sets.
+    Defined with string containing all allowed initial characters,
+    an optional string containing allowed body characters (if omitted,
+    defaults to the initial character set), and an optional minimum,
+    maximum, and/or exact length.  The default value for C{min} is 1 (a
+    minimum value < 1 is not valid); the default values for C{max} and C{exact}
+    are 0, meaning no maximum or exact length restriction. An optional
+    C{excludeChars} parameter can list characters that might be found in 
+    the input C{bodyChars} string; useful to define a word of all printables
+    except for one or two characters, for instance.
+    
+    L{srange} is useful for defining custom character set strings for defining 
+    C{Word} expressions, using range notation from regular expression character sets.
+    
+    A common mistake is to use C{Word} to match a specific literal string, as in 
+    C{Word("Address")}. Remember that C{Word} uses the string argument to define
+    I{sets} of matchable characters. This expression would match "Add", "AAA",
+    "dAred", or any other word made up of the characters 'A', 'd', 'r', 'e', and 's'.
+    To match an exact literal string, use L{Literal} or L{Keyword}.
+
+    pyparsing includes helper strings for building Words:
+     - L{alphas}
+     - L{nums}
+     - L{alphanums}
+     - L{hexnums}
+     - L{alphas8bit} (alphabetic characters in ASCII range 128-255 - accented, tilded, umlauted, etc.)
+     - L{punc8bit} (non-alphabetic characters in ASCII range 128-255 - currency, symbols, superscripts, diacriticals, etc.)
+     - L{printables} (any non-whitespace character)
+
+    Example::
+        # a word composed of digits
+        integer = Word(nums) # equivalent to Word("0123456789") or Word(srange("0-9"))
+        
+        # a word with a leading capital, and zero or more lowercase
+        capital_word = Word(alphas.upper(), alphas.lower())
+
+        # hostnames are alphanumeric, with leading alpha, and '-'
+        hostname = Word(alphas, alphanums+'-')
+        
+        # roman numeral (not a strict parser, accepts invalid mix of characters)
+        roman = Word("IVXLCDM")
+        
+        # any string of non-whitespace characters, except for ','
+        csv_value = Word(printables, excludeChars=",")
+    """
+    def __init__( self, initChars, bodyChars=None, min=1, max=0, exact=0, asKeyword=False, excludeChars=None ):
+        super(Word,self).__init__()
+        if excludeChars:
+            initChars = ''.join(c for c in initChars if c not in excludeChars)
+            if bodyChars:
+                bodyChars = ''.join(c for c in bodyChars if c not in excludeChars)
+        self.initCharsOrig = initChars
+        self.initChars = set(initChars)
+        if bodyChars :
+            self.bodyCharsOrig = bodyChars
+            self.bodyChars = set(bodyChars)
+        else:
+            self.bodyCharsOrig = initChars
+            self.bodyChars = set(initChars)
+
+        self.maxSpecified = max > 0
+
+        if min < 1:
+            raise ValueError("cannot specify a minimum length < 1; use Optional(Word()) if zero-length word is permitted")
+
+        self.minLen = min
+
+        if max > 0:
+            self.maxLen = max
+        else:
+            self.maxLen = _MAX_INT
+
+        if exact > 0:
+            self.maxLen = exact
+            self.minLen = exact
+
+        self.name = _ustr(self)
+        self.errmsg = "Expected " + self.name
+        self.mayIndexError = False
+        self.asKeyword = asKeyword
+
+        if ' ' not in self.initCharsOrig+self.bodyCharsOrig and (min==1 and max==0 and exact==0):
+            if self.bodyCharsOrig == self.initCharsOrig:
+                self.reString = "[%s]+" % _escapeRegexRangeChars(self.initCharsOrig)
+            elif len(self.initCharsOrig) == 1:
+                self.reString = "%s[%s]*" % \
+                                      (re.escape(self.initCharsOrig),
+                                      _escapeRegexRangeChars(self.bodyCharsOrig),)
+            else:
+                self.reString = "[%s][%s]*" % \
+                                      (_escapeRegexRangeChars(self.initCharsOrig),
+                                      _escapeRegexRangeChars(self.bodyCharsOrig),)
+            if self.asKeyword:
+                self.reString = r"\b"+self.reString+r"\b"
+            try:
+                self.re = re.compile( self.reString )
+            except Exception:
+                self.re = None
+
+    def parseImpl( self, instring, loc, doActions=True ):
+        if self.re:
+            result = self.re.match(instring,loc)
+            if not result:
+                raise ParseException(instring, loc, self.errmsg, self)
+
+            loc = result.end()
+            return loc, result.group()
+
+        if not(instring[ loc ] in self.initChars):
+            raise ParseException(instring, loc, self.errmsg, self)
+
+        start = loc
+        loc += 1
+        instrlen = len(instring)
+        bodychars = self.bodyChars
+        maxloc = start + self.maxLen
+        maxloc = min( maxloc, instrlen )
+        while loc < maxloc and instring[loc] in bodychars:
+            loc += 1
+
+        throwException = False
+        if loc - start < self.minLen:
+            throwException = True
+        if self.maxSpecified and loc < instrlen and instring[loc] in bodychars:
+            throwException = True
+        if self.asKeyword:
+            if (start>0 and instring[start-1] in bodychars) or (loc4:
+                    return s[:4]+"..."
+                else:
+                    return s
+
+            if ( self.initCharsOrig != self.bodyCharsOrig ):
+                self.strRepr = "W:(%s,%s)" % ( charsAsStr(self.initCharsOrig), charsAsStr(self.bodyCharsOrig) )
+            else:
+                self.strRepr = "W:(%s)" % charsAsStr(self.initCharsOrig)
+
+        return self.strRepr
+
+
+class Regex(Token):
+    r"""
+    Token for matching strings that match a given regular expression.
+    Defined with string specifying the regular expression in a form recognized by the inbuilt Python re module.
+    If the given regex contains named groups (defined using C{(?P...)}), these will be preserved as 
+    named parse results.
+
+    Example::
+        realnum = Regex(r"[+-]?\d+\.\d*")
+        date = Regex(r'(?P\d{4})-(?P\d\d?)-(?P\d\d?)')
+        # ref: http://stackoverflow.com/questions/267399/how-do-you-match-only-valid-roman-numerals-with-a-regular-expression
+        roman = Regex(r"M{0,4}(CM|CD|D?C{0,3})(XC|XL|L?X{0,3})(IX|IV|V?I{0,3})")
+    """
+    compiledREtype = type(re.compile("[A-Z]"))
+    def __init__( self, pattern, flags=0):
+        """The parameters C{pattern} and C{flags} are passed to the C{re.compile()} function as-is. See the Python C{re} module for an explanation of the acceptable patterns and flags."""
+        super(Regex,self).__init__()
+
+        if isinstance(pattern, basestring):
+            if not pattern:
+                warnings.warn("null string passed to Regex; use Empty() instead",
+                        SyntaxWarning, stacklevel=2)
+
+            self.pattern = pattern
+            self.flags = flags
+
+            try:
+                self.re = re.compile(self.pattern, self.flags)
+                self.reString = self.pattern
+            except sre_constants.error:
+                warnings.warn("invalid pattern (%s) passed to Regex" % pattern,
+                    SyntaxWarning, stacklevel=2)
+                raise
+
+        elif isinstance(pattern, Regex.compiledREtype):
+            self.re = pattern
+            self.pattern = \
+            self.reString = str(pattern)
+            self.flags = flags
+            
+        else:
+            raise ValueError("Regex may only be constructed with a string or a compiled RE object")
+
+        self.name = _ustr(self)
+        self.errmsg = "Expected " + self.name
+        self.mayIndexError = False
+        self.mayReturnEmpty = True
+
+    def parseImpl( self, instring, loc, doActions=True ):
+        result = self.re.match(instring,loc)
+        if not result:
+            raise ParseException(instring, loc, self.errmsg, self)
+
+        loc = result.end()
+        d = result.groupdict()
+        ret = ParseResults(result.group())
+        if d:
+            for k in d:
+                ret[k] = d[k]
+        return loc,ret
+
+    def __str__( self ):
+        try:
+            return super(Regex,self).__str__()
+        except Exception:
+            pass
+
+        if self.strRepr is None:
+            self.strRepr = "Re:(%s)" % repr(self.pattern)
+
+        return self.strRepr
+
+
+class QuotedString(Token):
+    r"""
+    Token for matching strings that are delimited by quoting characters.
+    
+    Defined with the following parameters:
+        - quoteChar - string of one or more characters defining the quote delimiting string
+        - escChar - character to escape quotes, typically backslash (default=C{None})
+        - escQuote - special quote sequence to escape an embedded quote string (such as SQL's "" to escape an embedded ") (default=C{None})
+        - multiline - boolean indicating whether quotes can span multiple lines (default=C{False})
+        - unquoteResults - boolean indicating whether the matched text should be unquoted (default=C{True})
+        - endQuoteChar - string of one or more characters defining the end of the quote delimited string (default=C{None} => same as quoteChar)
+        - convertWhitespaceEscapes - convert escaped whitespace (C{'\t'}, C{'\n'}, etc.) to actual whitespace (default=C{True})
+
+    Example::
+        qs = QuotedString('"')
+        print(qs.searchString('lsjdf "This is the quote" sldjf'))
+        complex_qs = QuotedString('{{', endQuoteChar='}}')
+        print(complex_qs.searchString('lsjdf {{This is the "quote"}} sldjf'))
+        sql_qs = QuotedString('"', escQuote='""')
+        print(sql_qs.searchString('lsjdf "This is the quote with ""embedded"" quotes" sldjf'))
+    prints::
+        [['This is the quote']]
+        [['This is the "quote"']]
+        [['This is the quote with "embedded" quotes']]
+    """
+    def __init__( self, quoteChar, escChar=None, escQuote=None, multiline=False, unquoteResults=True, endQuoteChar=None, convertWhitespaceEscapes=True):
+        super(QuotedString,self).__init__()
+
+        # remove white space from quote chars - wont work anyway
+        quoteChar = quoteChar.strip()
+        if not quoteChar:
+            warnings.warn("quoteChar cannot be the empty string",SyntaxWarning,stacklevel=2)
+            raise SyntaxError()
+
+        if endQuoteChar is None:
+            endQuoteChar = quoteChar
+        else:
+            endQuoteChar = endQuoteChar.strip()
+            if not endQuoteChar:
+                warnings.warn("endQuoteChar cannot be the empty string",SyntaxWarning,stacklevel=2)
+                raise SyntaxError()
+
+        self.quoteChar = quoteChar
+        self.quoteCharLen = len(quoteChar)
+        self.firstQuoteChar = quoteChar[0]
+        self.endQuoteChar = endQuoteChar
+        self.endQuoteCharLen = len(endQuoteChar)
+        self.escChar = escChar
+        self.escQuote = escQuote
+        self.unquoteResults = unquoteResults
+        self.convertWhitespaceEscapes = convertWhitespaceEscapes
+
+        if multiline:
+            self.flags = re.MULTILINE | re.DOTALL
+            self.pattern = r'%s(?:[^%s%s]' % \
+                ( re.escape(self.quoteChar),
+                  _escapeRegexRangeChars(self.endQuoteChar[0]),
+                  (escChar is not None and _escapeRegexRangeChars(escChar) or '') )
+        else:
+            self.flags = 0
+            self.pattern = r'%s(?:[^%s\n\r%s]' % \
+                ( re.escape(self.quoteChar),
+                  _escapeRegexRangeChars(self.endQuoteChar[0]),
+                  (escChar is not None and _escapeRegexRangeChars(escChar) or '') )
+        if len(self.endQuoteChar) > 1:
+            self.pattern += (
+                '|(?:' + ')|(?:'.join("%s[^%s]" % (re.escape(self.endQuoteChar[:i]),
+                                               _escapeRegexRangeChars(self.endQuoteChar[i]))
+                                    for i in range(len(self.endQuoteChar)-1,0,-1)) + ')'
+                )
+        if escQuote:
+            self.pattern += (r'|(?:%s)' % re.escape(escQuote))
+        if escChar:
+            self.pattern += (r'|(?:%s.)' % re.escape(escChar))
+            self.escCharReplacePattern = re.escape(self.escChar)+"(.)"
+        self.pattern += (r')*%s' % re.escape(self.endQuoteChar))
+
+        try:
+            self.re = re.compile(self.pattern, self.flags)
+            self.reString = self.pattern
+        except sre_constants.error:
+            warnings.warn("invalid pattern (%s) passed to Regex" % self.pattern,
+                SyntaxWarning, stacklevel=2)
+            raise
+
+        self.name = _ustr(self)
+        self.errmsg = "Expected " + self.name
+        self.mayIndexError = False
+        self.mayReturnEmpty = True
+
+    def parseImpl( self, instring, loc, doActions=True ):
+        result = instring[loc] == self.firstQuoteChar and self.re.match(instring,loc) or None
+        if not result:
+            raise ParseException(instring, loc, self.errmsg, self)
+
+        loc = result.end()
+        ret = result.group()
+
+        if self.unquoteResults:
+
+            # strip off quotes
+            ret = ret[self.quoteCharLen:-self.endQuoteCharLen]
+
+            if isinstance(ret,basestring):
+                # replace escaped whitespace
+                if '\\' in ret and self.convertWhitespaceEscapes:
+                    ws_map = {
+                        r'\t' : '\t',
+                        r'\n' : '\n',
+                        r'\f' : '\f',
+                        r'\r' : '\r',
+                    }
+                    for wslit,wschar in ws_map.items():
+                        ret = ret.replace(wslit, wschar)
+
+                # replace escaped characters
+                if self.escChar:
+                    ret = re.sub(self.escCharReplacePattern, r"\g<1>", ret)
+
+                # replace escaped quotes
+                if self.escQuote:
+                    ret = ret.replace(self.escQuote, self.endQuoteChar)
+
+        return loc, ret
+
+    def __str__( self ):
+        try:
+            return super(QuotedString,self).__str__()
+        except Exception:
+            pass
+
+        if self.strRepr is None:
+            self.strRepr = "quoted string, starting with %s ending with %s" % (self.quoteChar, self.endQuoteChar)
+
+        return self.strRepr
+
+
+class CharsNotIn(Token):
+    """
+    Token for matching words composed of characters I{not} in a given set (will
+    include whitespace in matched characters if not listed in the provided exclusion set - see example).
+    Defined with string containing all disallowed characters, and an optional
+    minimum, maximum, and/or exact length.  The default value for C{min} is 1 (a
+    minimum value < 1 is not valid); the default values for C{max} and C{exact}
+    are 0, meaning no maximum or exact length restriction.
+
+    Example::
+        # define a comma-separated-value as anything that is not a ','
+        csv_value = CharsNotIn(',')
+        print(delimitedList(csv_value).parseString("dkls,lsdkjf,s12 34,@!#,213"))
+    prints::
+        ['dkls', 'lsdkjf', 's12 34', '@!#', '213']
+    """
+    def __init__( self, notChars, min=1, max=0, exact=0 ):
+        super(CharsNotIn,self).__init__()
+        self.skipWhitespace = False
+        self.notChars = notChars
+
+        if min < 1:
+            raise ValueError("cannot specify a minimum length < 1; use Optional(CharsNotIn()) if zero-length char group is permitted")
+
+        self.minLen = min
+
+        if max > 0:
+            self.maxLen = max
+        else:
+            self.maxLen = _MAX_INT
+
+        if exact > 0:
+            self.maxLen = exact
+            self.minLen = exact
+
+        self.name = _ustr(self)
+        self.errmsg = "Expected " + self.name
+        self.mayReturnEmpty = ( self.minLen == 0 )
+        self.mayIndexError = False
+
+    def parseImpl( self, instring, loc, doActions=True ):
+        if instring[loc] in self.notChars:
+            raise ParseException(instring, loc, self.errmsg, self)
+
+        start = loc
+        loc += 1
+        notchars = self.notChars
+        maxlen = min( start+self.maxLen, len(instring) )
+        while loc < maxlen and \
+              (instring[loc] not in notchars):
+            loc += 1
+
+        if loc - start < self.minLen:
+            raise ParseException(instring, loc, self.errmsg, self)
+
+        return loc, instring[start:loc]
+
+    def __str__( self ):
+        try:
+            return super(CharsNotIn, self).__str__()
+        except Exception:
+            pass
+
+        if self.strRepr is None:
+            if len(self.notChars) > 4:
+                self.strRepr = "!W:(%s...)" % self.notChars[:4]
+            else:
+                self.strRepr = "!W:(%s)" % self.notChars
+
+        return self.strRepr
+
+class White(Token):
+    """
+    Special matching class for matching whitespace.  Normally, whitespace is ignored
+    by pyparsing grammars.  This class is included when some whitespace structures
+    are significant.  Define with a string containing the whitespace characters to be
+    matched; default is C{" \\t\\r\\n"}.  Also takes optional C{min}, C{max}, and C{exact} arguments,
+    as defined for the C{L{Word}} class.
+    """
+    whiteStrs = {
+        " " : "",
+        "\t": "",
+        "\n": "",
+        "\r": "",
+        "\f": "",
+        }
+    def __init__(self, ws=" \t\r\n", min=1, max=0, exact=0):
+        super(White,self).__init__()
+        self.matchWhite = ws
+        self.setWhitespaceChars( "".join(c for c in self.whiteChars if c not in self.matchWhite) )
+        #~ self.leaveWhitespace()
+        self.name = ("".join(White.whiteStrs[c] for c in self.matchWhite))
+        self.mayReturnEmpty = True
+        self.errmsg = "Expected " + self.name
+
+        self.minLen = min
+
+        if max > 0:
+            self.maxLen = max
+        else:
+            self.maxLen = _MAX_INT
+
+        if exact > 0:
+            self.maxLen = exact
+            self.minLen = exact
+
+    def parseImpl( self, instring, loc, doActions=True ):
+        if not(instring[ loc ] in self.matchWhite):
+            raise ParseException(instring, loc, self.errmsg, self)
+        start = loc
+        loc += 1
+        maxloc = start + self.maxLen
+        maxloc = min( maxloc, len(instring) )
+        while loc < maxloc and instring[loc] in self.matchWhite:
+            loc += 1
+
+        if loc - start < self.minLen:
+            raise ParseException(instring, loc, self.errmsg, self)
+
+        return loc, instring[start:loc]
+
+
+class _PositionToken(Token):
+    def __init__( self ):
+        super(_PositionToken,self).__init__()
+        self.name=self.__class__.__name__
+        self.mayReturnEmpty = True
+        self.mayIndexError = False
+
+class GoToColumn(_PositionToken):
+    """
+    Token to advance to a specific column of input text; useful for tabular report scraping.
+    """
+    def __init__( self, colno ):
+        super(GoToColumn,self).__init__()
+        self.col = colno
+
+    def preParse( self, instring, loc ):
+        if col(loc,instring) != self.col:
+            instrlen = len(instring)
+            if self.ignoreExprs:
+                loc = self._skipIgnorables( instring, loc )
+            while loc < instrlen and instring[loc].isspace() and col( loc, instring ) != self.col :
+                loc += 1
+        return loc
+
+    def parseImpl( self, instring, loc, doActions=True ):
+        thiscol = col( loc, instring )
+        if thiscol > self.col:
+            raise ParseException( instring, loc, "Text not in expected column", self )
+        newloc = loc + self.col - thiscol
+        ret = instring[ loc: newloc ]
+        return newloc, ret
+
+
+class LineStart(_PositionToken):
+    """
+    Matches if current position is at the beginning of a line within the parse string
+    
+    Example::
+    
+        test = '''\
+        AAA this line
+        AAA and this line
+          AAA but not this one
+        B AAA and definitely not this one
+        '''
+
+        for t in (LineStart() + 'AAA' + restOfLine).searchString(test):
+            print(t)
+    
+    Prints::
+        ['AAA', ' this line']
+        ['AAA', ' and this line']    
+
+    """
+    def __init__( self ):
+        super(LineStart,self).__init__()
+        self.errmsg = "Expected start of line"
+
+    def parseImpl( self, instring, loc, doActions=True ):
+        if col(loc, instring) == 1:
+            return loc, []
+        raise ParseException(instring, loc, self.errmsg, self)
+
+class LineEnd(_PositionToken):
+    """
+    Matches if current position is at the end of a line within the parse string
+    """
+    def __init__( self ):
+        super(LineEnd,self).__init__()
+        self.setWhitespaceChars( ParserElement.DEFAULT_WHITE_CHARS.replace("\n","") )
+        self.errmsg = "Expected end of line"
+
+    def parseImpl( self, instring, loc, doActions=True ):
+        if loc len(instring):
+            return loc, []
+        else:
+            raise ParseException(instring, loc, self.errmsg, self)
+
+class WordStart(_PositionToken):
+    """
+    Matches if the current position is at the beginning of a Word, and
+    is not preceded by any character in a given set of C{wordChars}
+    (default=C{printables}). To emulate the C{\b} behavior of regular expressions,
+    use C{WordStart(alphanums)}. C{WordStart} will also match at the beginning of
+    the string being parsed, or at the beginning of a line.
+    """
+    def __init__(self, wordChars = printables):
+        super(WordStart,self).__init__()
+        self.wordChars = set(wordChars)
+        self.errmsg = "Not at the start of a word"
+
+    def parseImpl(self, instring, loc, doActions=True ):
+        if loc != 0:
+            if (instring[loc-1] in self.wordChars or
+                instring[loc] not in self.wordChars):
+                raise ParseException(instring, loc, self.errmsg, self)
+        return loc, []
+
+class WordEnd(_PositionToken):
+    """
+    Matches if the current position is at the end of a Word, and
+    is not followed by any character in a given set of C{wordChars}
+    (default=C{printables}). To emulate the C{\b} behavior of regular expressions,
+    use C{WordEnd(alphanums)}. C{WordEnd} will also match at the end of
+    the string being parsed, or at the end of a line.
+    """
+    def __init__(self, wordChars = printables):
+        super(WordEnd,self).__init__()
+        self.wordChars = set(wordChars)
+        self.skipWhitespace = False
+        self.errmsg = "Not at the end of a word"
+
+    def parseImpl(self, instring, loc, doActions=True ):
+        instrlen = len(instring)
+        if instrlen>0 and loc maxExcLoc:
+                    maxException = err
+                    maxExcLoc = err.loc
+            except IndexError:
+                if len(instring) > maxExcLoc:
+                    maxException = ParseException(instring,len(instring),e.errmsg,self)
+                    maxExcLoc = len(instring)
+            else:
+                # save match among all matches, to retry longest to shortest
+                matches.append((loc2, e))
+
+        if matches:
+            matches.sort(key=lambda x: -x[0])
+            for _,e in matches:
+                try:
+                    return e._parse( instring, loc, doActions )
+                except ParseException as err:
+                    err.__traceback__ = None
+                    if err.loc > maxExcLoc:
+                        maxException = err
+                        maxExcLoc = err.loc
+
+        if maxException is not None:
+            maxException.msg = self.errmsg
+            raise maxException
+        else:
+            raise ParseException(instring, loc, "no defined alternatives to match", self)
+
+
+    def __ixor__(self, other ):
+        if isinstance( other, basestring ):
+            other = ParserElement._literalStringClass( other )
+        return self.append( other ) #Or( [ self, other ] )
+
+    def __str__( self ):
+        if hasattr(self,"name"):
+            return self.name
+
+        if self.strRepr is None:
+            self.strRepr = "{" + " ^ ".join(_ustr(e) for e in self.exprs) + "}"
+
+        return self.strRepr
+
+    def checkRecursion( self, parseElementList ):
+        subRecCheckList = parseElementList[:] + [ self ]
+        for e in self.exprs:
+            e.checkRecursion( subRecCheckList )
+
+
+class MatchFirst(ParseExpression):
+    """
+    Requires that at least one C{ParseExpression} is found.
+    If two expressions match, the first one listed is the one that will match.
+    May be constructed using the C{'|'} operator.
+
+    Example::
+        # construct MatchFirst using '|' operator
+        
+        # watch the order of expressions to match
+        number = Word(nums) | Combine(Word(nums) + '.' + Word(nums))
+        print(number.searchString("123 3.1416 789")) #  Fail! -> [['123'], ['3'], ['1416'], ['789']]
+
+        # put more selective expression first
+        number = Combine(Word(nums) + '.' + Word(nums)) | Word(nums)
+        print(number.searchString("123 3.1416 789")) #  Better -> [['123'], ['3.1416'], ['789']]
+    """
+    def __init__( self, exprs, savelist = False ):
+        super(MatchFirst,self).__init__(exprs, savelist)
+        if self.exprs:
+            self.mayReturnEmpty = any(e.mayReturnEmpty for e in self.exprs)
+        else:
+            self.mayReturnEmpty = True
+
+    def parseImpl( self, instring, loc, doActions=True ):
+        maxExcLoc = -1
+        maxException = None
+        for e in self.exprs:
+            try:
+                ret = e._parse( instring, loc, doActions )
+                return ret
+            except ParseException as err:
+                if err.loc > maxExcLoc:
+                    maxException = err
+                    maxExcLoc = err.loc
+            except IndexError:
+                if len(instring) > maxExcLoc:
+                    maxException = ParseException(instring,len(instring),e.errmsg,self)
+                    maxExcLoc = len(instring)
+
+        # only got here if no expression matched, raise exception for match that made it the furthest
+        else:
+            if maxException is not None:
+                maxException.msg = self.errmsg
+                raise maxException
+            else:
+                raise ParseException(instring, loc, "no defined alternatives to match", self)
+
+    def __ior__(self, other ):
+        if isinstance( other, basestring ):
+            other = ParserElement._literalStringClass( other )
+        return self.append( other ) #MatchFirst( [ self, other ] )
+
+    def __str__( self ):
+        if hasattr(self,"name"):
+            return self.name
+
+        if self.strRepr is None:
+            self.strRepr = "{" + " | ".join(_ustr(e) for e in self.exprs) + "}"
+
+        return self.strRepr
+
+    def checkRecursion( self, parseElementList ):
+        subRecCheckList = parseElementList[:] + [ self ]
+        for e in self.exprs:
+            e.checkRecursion( subRecCheckList )
+
+
+class Each(ParseExpression):
+    """
+    Requires all given C{ParseExpression}s to be found, but in any order.
+    Expressions may be separated by whitespace.
+    May be constructed using the C{'&'} operator.
+
+    Example::
+        color = oneOf("RED ORANGE YELLOW GREEN BLUE PURPLE BLACK WHITE BROWN")
+        shape_type = oneOf("SQUARE CIRCLE TRIANGLE STAR HEXAGON OCTAGON")
+        integer = Word(nums)
+        shape_attr = "shape:" + shape_type("shape")
+        posn_attr = "posn:" + Group(integer("x") + ',' + integer("y"))("posn")
+        color_attr = "color:" + color("color")
+        size_attr = "size:" + integer("size")
+
+        # use Each (using operator '&') to accept attributes in any order 
+        # (shape and posn are required, color and size are optional)
+        shape_spec = shape_attr & posn_attr & Optional(color_attr) & Optional(size_attr)
+
+        shape_spec.runTests('''
+            shape: SQUARE color: BLACK posn: 100, 120
+            shape: CIRCLE size: 50 color: BLUE posn: 50,80
+            color:GREEN size:20 shape:TRIANGLE posn:20,40
+            '''
+            )
+    prints::
+        shape: SQUARE color: BLACK posn: 100, 120
+        ['shape:', 'SQUARE', 'color:', 'BLACK', 'posn:', ['100', ',', '120']]
+        - color: BLACK
+        - posn: ['100', ',', '120']
+          - x: 100
+          - y: 120
+        - shape: SQUARE
+
+
+        shape: CIRCLE size: 50 color: BLUE posn: 50,80
+        ['shape:', 'CIRCLE', 'size:', '50', 'color:', 'BLUE', 'posn:', ['50', ',', '80']]
+        - color: BLUE
+        - posn: ['50', ',', '80']
+          - x: 50
+          - y: 80
+        - shape: CIRCLE
+        - size: 50
+
+
+        color: GREEN size: 20 shape: TRIANGLE posn: 20,40
+        ['color:', 'GREEN', 'size:', '20', 'shape:', 'TRIANGLE', 'posn:', ['20', ',', '40']]
+        - color: GREEN
+        - posn: ['20', ',', '40']
+          - x: 20
+          - y: 40
+        - shape: TRIANGLE
+        - size: 20
+    """
+    def __init__( self, exprs, savelist = True ):
+        super(Each,self).__init__(exprs, savelist)
+        self.mayReturnEmpty = all(e.mayReturnEmpty for e in self.exprs)
+        self.skipWhitespace = True
+        self.initExprGroups = True
+
+    def parseImpl( self, instring, loc, doActions=True ):
+        if self.initExprGroups:
+            self.opt1map = dict((id(e.expr),e) for e in self.exprs if isinstance(e,Optional))
+            opt1 = [ e.expr for e in self.exprs if isinstance(e,Optional) ]
+            opt2 = [ e for e in self.exprs if e.mayReturnEmpty and not isinstance(e,Optional)]
+            self.optionals = opt1 + opt2
+            self.multioptionals = [ e.expr for e in self.exprs if isinstance(e,ZeroOrMore) ]
+            self.multirequired = [ e.expr for e in self.exprs if isinstance(e,OneOrMore) ]
+            self.required = [ e for e in self.exprs if not isinstance(e,(Optional,ZeroOrMore,OneOrMore)) ]
+            self.required += self.multirequired
+            self.initExprGroups = False
+        tmpLoc = loc
+        tmpReqd = self.required[:]
+        tmpOpt  = self.optionals[:]
+        matchOrder = []
+
+        keepMatching = True
+        while keepMatching:
+            tmpExprs = tmpReqd + tmpOpt + self.multioptionals + self.multirequired
+            failed = []
+            for e in tmpExprs:
+                try:
+                    tmpLoc = e.tryParse( instring, tmpLoc )
+                except ParseException:
+                    failed.append(e)
+                else:
+                    matchOrder.append(self.opt1map.get(id(e),e))
+                    if e in tmpReqd:
+                        tmpReqd.remove(e)
+                    elif e in tmpOpt:
+                        tmpOpt.remove(e)
+            if len(failed) == len(tmpExprs):
+                keepMatching = False
+
+        if tmpReqd:
+            missing = ", ".join(_ustr(e) for e in tmpReqd)
+            raise ParseException(instring,loc,"Missing one or more required elements (%s)" % missing )
+
+        # add any unmatched Optionals, in case they have default values defined
+        matchOrder += [e for e in self.exprs if isinstance(e,Optional) and e.expr in tmpOpt]
+
+        resultlist = []
+        for e in matchOrder:
+            loc,results = e._parse(instring,loc,doActions)
+            resultlist.append(results)
+
+        finalResults = sum(resultlist, ParseResults([]))
+        return loc, finalResults
+
+    def __str__( self ):
+        if hasattr(self,"name"):
+            return self.name
+
+        if self.strRepr is None:
+            self.strRepr = "{" + " & ".join(_ustr(e) for e in self.exprs) + "}"
+
+        return self.strRepr
+
+    def checkRecursion( self, parseElementList ):
+        subRecCheckList = parseElementList[:] + [ self ]
+        for e in self.exprs:
+            e.checkRecursion( subRecCheckList )
+
+
+class ParseElementEnhance(ParserElement):
+    """
+    Abstract subclass of C{ParserElement}, for combining and post-processing parsed tokens.
+    """
+    def __init__( self, expr, savelist=False ):
+        super(ParseElementEnhance,self).__init__(savelist)
+        if isinstance( expr, basestring ):
+            if issubclass(ParserElement._literalStringClass, Token):
+                expr = ParserElement._literalStringClass(expr)
+            else:
+                expr = ParserElement._literalStringClass(Literal(expr))
+        self.expr = expr
+        self.strRepr = None
+        if expr is not None:
+            self.mayIndexError = expr.mayIndexError
+            self.mayReturnEmpty = expr.mayReturnEmpty
+            self.setWhitespaceChars( expr.whiteChars )
+            self.skipWhitespace = expr.skipWhitespace
+            self.saveAsList = expr.saveAsList
+            self.callPreparse = expr.callPreparse
+            self.ignoreExprs.extend(expr.ignoreExprs)
+
+    def parseImpl( self, instring, loc, doActions=True ):
+        if self.expr is not None:
+            return self.expr._parse( instring, loc, doActions, callPreParse=False )
+        else:
+            raise ParseException("",loc,self.errmsg,self)
+
+    def leaveWhitespace( self ):
+        self.skipWhitespace = False
+        self.expr = self.expr.copy()
+        if self.expr is not None:
+            self.expr.leaveWhitespace()
+        return self
+
+    def ignore( self, other ):
+        if isinstance( other, Suppress ):
+            if other not in self.ignoreExprs:
+                super( ParseElementEnhance, self).ignore( other )
+                if self.expr is not None:
+                    self.expr.ignore( self.ignoreExprs[-1] )
+        else:
+            super( ParseElementEnhance, self).ignore( other )
+            if self.expr is not None:
+                self.expr.ignore( self.ignoreExprs[-1] )
+        return self
+
+    def streamline( self ):
+        super(ParseElementEnhance,self).streamline()
+        if self.expr is not None:
+            self.expr.streamline()
+        return self
+
+    def checkRecursion( self, parseElementList ):
+        if self in parseElementList:
+            raise RecursiveGrammarException( parseElementList+[self] )
+        subRecCheckList = parseElementList[:] + [ self ]
+        if self.expr is not None:
+            self.expr.checkRecursion( subRecCheckList )
+
+    def validate( self, validateTrace=[] ):
+        tmp = validateTrace[:]+[self]
+        if self.expr is not None:
+            self.expr.validate(tmp)
+        self.checkRecursion( [] )
+
+    def __str__( self ):
+        try:
+            return super(ParseElementEnhance,self).__str__()
+        except Exception:
+            pass
+
+        if self.strRepr is None and self.expr is not None:
+            self.strRepr = "%s:(%s)" % ( self.__class__.__name__, _ustr(self.expr) )
+        return self.strRepr
+
+
+class FollowedBy(ParseElementEnhance):
+    """
+    Lookahead matching of the given parse expression.  C{FollowedBy}
+    does I{not} advance the parsing position within the input string, it only
+    verifies that the specified parse expression matches at the current
+    position.  C{FollowedBy} always returns a null token list.
+
+    Example::
+        # use FollowedBy to match a label only if it is followed by a ':'
+        data_word = Word(alphas)
+        label = data_word + FollowedBy(':')
+        attr_expr = Group(label + Suppress(':') + OneOrMore(data_word, stopOn=label).setParseAction(' '.join))
+        
+        OneOrMore(attr_expr).parseString("shape: SQUARE color: BLACK posn: upper left").pprint()
+    prints::
+        [['shape', 'SQUARE'], ['color', 'BLACK'], ['posn', 'upper left']]
+    """
+    def __init__( self, expr ):
+        super(FollowedBy,self).__init__(expr)
+        self.mayReturnEmpty = True
+
+    def parseImpl( self, instring, loc, doActions=True ):
+        self.expr.tryParse( instring, loc )
+        return loc, []
+
+
+class NotAny(ParseElementEnhance):
+    """
+    Lookahead to disallow matching with the given parse expression.  C{NotAny}
+    does I{not} advance the parsing position within the input string, it only
+    verifies that the specified parse expression does I{not} match at the current
+    position.  Also, C{NotAny} does I{not} skip over leading whitespace. C{NotAny}
+    always returns a null token list.  May be constructed using the '~' operator.
+
+    Example::
+        
+    """
+    def __init__( self, expr ):
+        super(NotAny,self).__init__(expr)
+        #~ self.leaveWhitespace()
+        self.skipWhitespace = False  # do NOT use self.leaveWhitespace(), don't want to propagate to exprs
+        self.mayReturnEmpty = True
+        self.errmsg = "Found unwanted token, "+_ustr(self.expr)
+
+    def parseImpl( self, instring, loc, doActions=True ):
+        if self.expr.canParseNext(instring, loc):
+            raise ParseException(instring, loc, self.errmsg, self)
+        return loc, []
+
+    def __str__( self ):
+        if hasattr(self,"name"):
+            return self.name
+
+        if self.strRepr is None:
+            self.strRepr = "~{" + _ustr(self.expr) + "}"
+
+        return self.strRepr
+
+class _MultipleMatch(ParseElementEnhance):
+    def __init__( self, expr, stopOn=None):
+        super(_MultipleMatch, self).__init__(expr)
+        self.saveAsList = True
+        ender = stopOn
+        if isinstance(ender, basestring):
+            ender = ParserElement._literalStringClass(ender)
+        self.not_ender = ~ender if ender is not None else None
+
+    def parseImpl( self, instring, loc, doActions=True ):
+        self_expr_parse = self.expr._parse
+        self_skip_ignorables = self._skipIgnorables
+        check_ender = self.not_ender is not None
+        if check_ender:
+            try_not_ender = self.not_ender.tryParse
+        
+        # must be at least one (but first see if we are the stopOn sentinel;
+        # if so, fail)
+        if check_ender:
+            try_not_ender(instring, loc)
+        loc, tokens = self_expr_parse( instring, loc, doActions, callPreParse=False )
+        try:
+            hasIgnoreExprs = (not not self.ignoreExprs)
+            while 1:
+                if check_ender:
+                    try_not_ender(instring, loc)
+                if hasIgnoreExprs:
+                    preloc = self_skip_ignorables( instring, loc )
+                else:
+                    preloc = loc
+                loc, tmptokens = self_expr_parse( instring, preloc, doActions )
+                if tmptokens or tmptokens.haskeys():
+                    tokens += tmptokens
+        except (ParseException,IndexError):
+            pass
+
+        return loc, tokens
+        
+class OneOrMore(_MultipleMatch):
+    """
+    Repetition of one or more of the given expression.
+    
+    Parameters:
+     - expr - expression that must match one or more times
+     - stopOn - (default=C{None}) - expression for a terminating sentinel
+          (only required if the sentinel would ordinarily match the repetition 
+          expression)          
+
+    Example::
+        data_word = Word(alphas)
+        label = data_word + FollowedBy(':')
+        attr_expr = Group(label + Suppress(':') + OneOrMore(data_word).setParseAction(' '.join))
+
+        text = "shape: SQUARE posn: upper left color: BLACK"
+        OneOrMore(attr_expr).parseString(text).pprint()  # Fail! read 'color' as data instead of next label -> [['shape', 'SQUARE color']]
+
+        # use stopOn attribute for OneOrMore to avoid reading label string as part of the data
+        attr_expr = Group(label + Suppress(':') + OneOrMore(data_word, stopOn=label).setParseAction(' '.join))
+        OneOrMore(attr_expr).parseString(text).pprint() # Better -> [['shape', 'SQUARE'], ['posn', 'upper left'], ['color', 'BLACK']]
+        
+        # could also be written as
+        (attr_expr * (1,)).parseString(text).pprint()
+    """
+
+    def __str__( self ):
+        if hasattr(self,"name"):
+            return self.name
+
+        if self.strRepr is None:
+            self.strRepr = "{" + _ustr(self.expr) + "}..."
+
+        return self.strRepr
+
+class ZeroOrMore(_MultipleMatch):
+    """
+    Optional repetition of zero or more of the given expression.
+    
+    Parameters:
+     - expr - expression that must match zero or more times
+     - stopOn - (default=C{None}) - expression for a terminating sentinel
+          (only required if the sentinel would ordinarily match the repetition 
+          expression)          
+
+    Example: similar to L{OneOrMore}
+    """
+    def __init__( self, expr, stopOn=None):
+        super(ZeroOrMore,self).__init__(expr, stopOn=stopOn)
+        self.mayReturnEmpty = True
+        
+    def parseImpl( self, instring, loc, doActions=True ):
+        try:
+            return super(ZeroOrMore, self).parseImpl(instring, loc, doActions)
+        except (ParseException,IndexError):
+            return loc, []
+
+    def __str__( self ):
+        if hasattr(self,"name"):
+            return self.name
+
+        if self.strRepr is None:
+            self.strRepr = "[" + _ustr(self.expr) + "]..."
+
+        return self.strRepr
+
+class _NullToken(object):
+    def __bool__(self):
+        return False
+    __nonzero__ = __bool__
+    def __str__(self):
+        return ""
+
+_optionalNotMatched = _NullToken()
+class Optional(ParseElementEnhance):
+    """
+    Optional matching of the given expression.
+
+    Parameters:
+     - expr - expression that must match zero or more times
+     - default (optional) - value to be returned if the optional expression is not found.
+
+    Example::
+        # US postal code can be a 5-digit zip, plus optional 4-digit qualifier
+        zip = Combine(Word(nums, exact=5) + Optional('-' + Word(nums, exact=4)))
+        zip.runTests('''
+            # traditional ZIP code
+            12345
+            
+            # ZIP+4 form
+            12101-0001
+            
+            # invalid ZIP
+            98765-
+            ''')
+    prints::
+        # traditional ZIP code
+        12345
+        ['12345']
+
+        # ZIP+4 form
+        12101-0001
+        ['12101-0001']
+
+        # invalid ZIP
+        98765-
+             ^
+        FAIL: Expected end of text (at char 5), (line:1, col:6)
+    """
+    def __init__( self, expr, default=_optionalNotMatched ):
+        super(Optional,self).__init__( expr, savelist=False )
+        self.saveAsList = self.expr.saveAsList
+        self.defaultValue = default
+        self.mayReturnEmpty = True
+
+    def parseImpl( self, instring, loc, doActions=True ):
+        try:
+            loc, tokens = self.expr._parse( instring, loc, doActions, callPreParse=False )
+        except (ParseException,IndexError):
+            if self.defaultValue is not _optionalNotMatched:
+                if self.expr.resultsName:
+                    tokens = ParseResults([ self.defaultValue ])
+                    tokens[self.expr.resultsName] = self.defaultValue
+                else:
+                    tokens = [ self.defaultValue ]
+            else:
+                tokens = []
+        return loc, tokens
+
+    def __str__( self ):
+        if hasattr(self,"name"):
+            return self.name
+
+        if self.strRepr is None:
+            self.strRepr = "[" + _ustr(self.expr) + "]"
+
+        return self.strRepr
+
+class SkipTo(ParseElementEnhance):
+    """
+    Token for skipping over all undefined text until the matched expression is found.
+
+    Parameters:
+     - expr - target expression marking the end of the data to be skipped
+     - include - (default=C{False}) if True, the target expression is also parsed 
+          (the skipped text and target expression are returned as a 2-element list).
+     - ignore - (default=C{None}) used to define grammars (typically quoted strings and 
+          comments) that might contain false matches to the target expression
+     - failOn - (default=C{None}) define expressions that are not allowed to be 
+          included in the skipped test; if found before the target expression is found, 
+          the SkipTo is not a match
+
+    Example::
+        report = '''
+            Outstanding Issues Report - 1 Jan 2000
+
+               # | Severity | Description                               |  Days Open
+            -----+----------+-------------------------------------------+-----------
+             101 | Critical | Intermittent system crash                 |          6
+              94 | Cosmetic | Spelling error on Login ('log|n')         |         14
+              79 | Minor    | System slow when running too many reports |         47
+            '''
+        integer = Word(nums)
+        SEP = Suppress('|')
+        # use SkipTo to simply match everything up until the next SEP
+        # - ignore quoted strings, so that a '|' character inside a quoted string does not match
+        # - parse action will call token.strip() for each matched token, i.e., the description body
+        string_data = SkipTo(SEP, ignore=quotedString)
+        string_data.setParseAction(tokenMap(str.strip))
+        ticket_expr = (integer("issue_num") + SEP 
+                      + string_data("sev") + SEP 
+                      + string_data("desc") + SEP 
+                      + integer("days_open"))
+        
+        for tkt in ticket_expr.searchString(report):
+            print tkt.dump()
+    prints::
+        ['101', 'Critical', 'Intermittent system crash', '6']
+        - days_open: 6
+        - desc: Intermittent system crash
+        - issue_num: 101
+        - sev: Critical
+        ['94', 'Cosmetic', "Spelling error on Login ('log|n')", '14']
+        - days_open: 14
+        - desc: Spelling error on Login ('log|n')
+        - issue_num: 94
+        - sev: Cosmetic
+        ['79', 'Minor', 'System slow when running too many reports', '47']
+        - days_open: 47
+        - desc: System slow when running too many reports
+        - issue_num: 79
+        - sev: Minor
+    """
+    def __init__( self, other, include=False, ignore=None, failOn=None ):
+        super( SkipTo, self ).__init__( other )
+        self.ignoreExpr = ignore
+        self.mayReturnEmpty = True
+        self.mayIndexError = False
+        self.includeMatch = include
+        self.asList = False
+        if isinstance(failOn, basestring):
+            self.failOn = ParserElement._literalStringClass(failOn)
+        else:
+            self.failOn = failOn
+        self.errmsg = "No match found for "+_ustr(self.expr)
+
+    def parseImpl( self, instring, loc, doActions=True ):
+        startloc = loc
+        instrlen = len(instring)
+        expr = self.expr
+        expr_parse = self.expr._parse
+        self_failOn_canParseNext = self.failOn.canParseNext if self.failOn is not None else None
+        self_ignoreExpr_tryParse = self.ignoreExpr.tryParse if self.ignoreExpr is not None else None
+        
+        tmploc = loc
+        while tmploc <= instrlen:
+            if self_failOn_canParseNext is not None:
+                # break if failOn expression matches
+                if self_failOn_canParseNext(instring, tmploc):
+                    break
+                    
+            if self_ignoreExpr_tryParse is not None:
+                # advance past ignore expressions
+                while 1:
+                    try:
+                        tmploc = self_ignoreExpr_tryParse(instring, tmploc)
+                    except ParseBaseException:
+                        break
+            
+            try:
+                expr_parse(instring, tmploc, doActions=False, callPreParse=False)
+            except (ParseException, IndexError):
+                # no match, advance loc in string
+                tmploc += 1
+            else:
+                # matched skipto expr, done
+                break
+
+        else:
+            # ran off the end of the input string without matching skipto expr, fail
+            raise ParseException(instring, loc, self.errmsg, self)
+
+        # build up return values
+        loc = tmploc
+        skiptext = instring[startloc:loc]
+        skipresult = ParseResults(skiptext)
+        
+        if self.includeMatch:
+            loc, mat = expr_parse(instring,loc,doActions,callPreParse=False)
+            skipresult += mat
+
+        return loc, skipresult
+
+class Forward(ParseElementEnhance):
+    """
+    Forward declaration of an expression to be defined later -
+    used for recursive grammars, such as algebraic infix notation.
+    When the expression is known, it is assigned to the C{Forward} variable using the '<<' operator.
+
+    Note: take care when assigning to C{Forward} not to overlook precedence of operators.
+    Specifically, '|' has a lower precedence than '<<', so that::
+        fwdExpr << a | b | c
+    will actually be evaluated as::
+        (fwdExpr << a) | b | c
+    thereby leaving b and c out as parseable alternatives.  It is recommended that you
+    explicitly group the values inserted into the C{Forward}::
+        fwdExpr << (a | b | c)
+    Converting to use the '<<=' operator instead will avoid this problem.
+
+    See L{ParseResults.pprint} for an example of a recursive parser created using
+    C{Forward}.
+    """
+    def __init__( self, other=None ):
+        super(Forward,self).__init__( other, savelist=False )
+
+    def __lshift__( self, other ):
+        if isinstance( other, basestring ):
+            other = ParserElement._literalStringClass(other)
+        self.expr = other
+        self.strRepr = None
+        self.mayIndexError = self.expr.mayIndexError
+        self.mayReturnEmpty = self.expr.mayReturnEmpty
+        self.setWhitespaceChars( self.expr.whiteChars )
+        self.skipWhitespace = self.expr.skipWhitespace
+        self.saveAsList = self.expr.saveAsList
+        self.ignoreExprs.extend(self.expr.ignoreExprs)
+        return self
+        
+    def __ilshift__(self, other):
+        return self << other
+    
+    def leaveWhitespace( self ):
+        self.skipWhitespace = False
+        return self
+
+    def streamline( self ):
+        if not self.streamlined:
+            self.streamlined = True
+            if self.expr is not None:
+                self.expr.streamline()
+        return self
+
+    def validate( self, validateTrace=[] ):
+        if self not in validateTrace:
+            tmp = validateTrace[:]+[self]
+            if self.expr is not None:
+                self.expr.validate(tmp)
+        self.checkRecursion([])
+
+    def __str__( self ):
+        if hasattr(self,"name"):
+            return self.name
+        return self.__class__.__name__ + ": ..."
+
+        # stubbed out for now - creates awful memory and perf issues
+        self._revertClass = self.__class__
+        self.__class__ = _ForwardNoRecurse
+        try:
+            if self.expr is not None:
+                retString = _ustr(self.expr)
+            else:
+                retString = "None"
+        finally:
+            self.__class__ = self._revertClass
+        return self.__class__.__name__ + ": " + retString
+
+    def copy(self):
+        if self.expr is not None:
+            return super(Forward,self).copy()
+        else:
+            ret = Forward()
+            ret <<= self
+            return ret
+
+class _ForwardNoRecurse(Forward):
+    def __str__( self ):
+        return "..."
+
+class TokenConverter(ParseElementEnhance):
+    """
+    Abstract subclass of C{ParseExpression}, for converting parsed results.
+    """
+    def __init__( self, expr, savelist=False ):
+        super(TokenConverter,self).__init__( expr )#, savelist )
+        self.saveAsList = False
+
+class Combine(TokenConverter):
+    """
+    Converter to concatenate all matching tokens to a single string.
+    By default, the matching patterns must also be contiguous in the input string;
+    this can be disabled by specifying C{'adjacent=False'} in the constructor.
+
+    Example::
+        real = Word(nums) + '.' + Word(nums)
+        print(real.parseString('3.1416')) # -> ['3', '.', '1416']
+        # will also erroneously match the following
+        print(real.parseString('3. 1416')) # -> ['3', '.', '1416']
+
+        real = Combine(Word(nums) + '.' + Word(nums))
+        print(real.parseString('3.1416')) # -> ['3.1416']
+        # no match when there are internal spaces
+        print(real.parseString('3. 1416')) # -> Exception: Expected W:(0123...)
+    """
+    def __init__( self, expr, joinString="", adjacent=True ):
+        super(Combine,self).__init__( expr )
+        # suppress whitespace-stripping in contained parse expressions, but re-enable it on the Combine itself
+        if adjacent:
+            self.leaveWhitespace()
+        self.adjacent = adjacent
+        self.skipWhitespace = True
+        self.joinString = joinString
+        self.callPreparse = True
+
+    def ignore( self, other ):
+        if self.adjacent:
+            ParserElement.ignore(self, other)
+        else:
+            super( Combine, self).ignore( other )
+        return self
+
+    def postParse( self, instring, loc, tokenlist ):
+        retToks = tokenlist.copy()
+        del retToks[:]
+        retToks += ParseResults([ "".join(tokenlist._asStringList(self.joinString)) ], modal=self.modalResults)
+
+        if self.resultsName and retToks.haskeys():
+            return [ retToks ]
+        else:
+            return retToks
+
+class Group(TokenConverter):
+    """
+    Converter to return the matched tokens as a list - useful for returning tokens of C{L{ZeroOrMore}} and C{L{OneOrMore}} expressions.
+
+    Example::
+        ident = Word(alphas)
+        num = Word(nums)
+        term = ident | num
+        func = ident + Optional(delimitedList(term))
+        print(func.parseString("fn a,b,100"))  # -> ['fn', 'a', 'b', '100']
+
+        func = ident + Group(Optional(delimitedList(term)))
+        print(func.parseString("fn a,b,100"))  # -> ['fn', ['a', 'b', '100']]
+    """
+    def __init__( self, expr ):
+        super(Group,self).__init__( expr )
+        self.saveAsList = True
+
+    def postParse( self, instring, loc, tokenlist ):
+        return [ tokenlist ]
+
+class Dict(TokenConverter):
+    """
+    Converter to return a repetitive expression as a list, but also as a dictionary.
+    Each element can also be referenced using the first token in the expression as its key.
+    Useful for tabular report scraping when the first column can be used as a item key.
+
+    Example::
+        data_word = Word(alphas)
+        label = data_word + FollowedBy(':')
+        attr_expr = Group(label + Suppress(':') + OneOrMore(data_word).setParseAction(' '.join))
+
+        text = "shape: SQUARE posn: upper left color: light blue texture: burlap"
+        attr_expr = (label + Suppress(':') + OneOrMore(data_word, stopOn=label).setParseAction(' '.join))
+        
+        # print attributes as plain groups
+        print(OneOrMore(attr_expr).parseString(text).dump())
+        
+        # instead of OneOrMore(expr), parse using Dict(OneOrMore(Group(expr))) - Dict will auto-assign names
+        result = Dict(OneOrMore(Group(attr_expr))).parseString(text)
+        print(result.dump())
+        
+        # access named fields as dict entries, or output as dict
+        print(result['shape'])        
+        print(result.asDict())
+    prints::
+        ['shape', 'SQUARE', 'posn', 'upper left', 'color', 'light blue', 'texture', 'burlap']
+
+        [['shape', 'SQUARE'], ['posn', 'upper left'], ['color', 'light blue'], ['texture', 'burlap']]
+        - color: light blue
+        - posn: upper left
+        - shape: SQUARE
+        - texture: burlap
+        SQUARE
+        {'color': 'light blue', 'posn': 'upper left', 'texture': 'burlap', 'shape': 'SQUARE'}
+    See more examples at L{ParseResults} of accessing fields by results name.
+    """
+    def __init__( self, expr ):
+        super(Dict,self).__init__( expr )
+        self.saveAsList = True
+
+    def postParse( self, instring, loc, tokenlist ):
+        for i,tok in enumerate(tokenlist):
+            if len(tok) == 0:
+                continue
+            ikey = tok[0]
+            if isinstance(ikey,int):
+                ikey = _ustr(tok[0]).strip()
+            if len(tok)==1:
+                tokenlist[ikey] = _ParseResultsWithOffset("",i)
+            elif len(tok)==2 and not isinstance(tok[1],ParseResults):
+                tokenlist[ikey] = _ParseResultsWithOffset(tok[1],i)
+            else:
+                dictvalue = tok.copy() #ParseResults(i)
+                del dictvalue[0]
+                if len(dictvalue)!= 1 or (isinstance(dictvalue,ParseResults) and dictvalue.haskeys()):
+                    tokenlist[ikey] = _ParseResultsWithOffset(dictvalue,i)
+                else:
+                    tokenlist[ikey] = _ParseResultsWithOffset(dictvalue[0],i)
+
+        if self.resultsName:
+            return [ tokenlist ]
+        else:
+            return tokenlist
+
+
+class Suppress(TokenConverter):
+    """
+    Converter for ignoring the results of a parsed expression.
+
+    Example::
+        source = "a, b, c,d"
+        wd = Word(alphas)
+        wd_list1 = wd + ZeroOrMore(',' + wd)
+        print(wd_list1.parseString(source))
+
+        # often, delimiters that are useful during parsing are just in the
+        # way afterward - use Suppress to keep them out of the parsed output
+        wd_list2 = wd + ZeroOrMore(Suppress(',') + wd)
+        print(wd_list2.parseString(source))
+    prints::
+        ['a', ',', 'b', ',', 'c', ',', 'd']
+        ['a', 'b', 'c', 'd']
+    (See also L{delimitedList}.)
+    """
+    def postParse( self, instring, loc, tokenlist ):
+        return []
+
+    def suppress( self ):
+        return self
+
+
+class OnlyOnce(object):
+    """
+    Wrapper for parse actions, to ensure they are only called once.
+    """
+    def __init__(self, methodCall):
+        self.callable = _trim_arity(methodCall)
+        self.called = False
+    def __call__(self,s,l,t):
+        if not self.called:
+            results = self.callable(s,l,t)
+            self.called = True
+            return results
+        raise ParseException(s,l,"")
+    def reset(self):
+        self.called = False
+
+def traceParseAction(f):
+    """
+    Decorator for debugging parse actions. 
+    
+    When the parse action is called, this decorator will print C{">> entering I{method-name}(line:I{current_source_line}, I{parse_location}, I{matched_tokens})".}
+    When the parse action completes, the decorator will print C{"<<"} followed by the returned value, or any exception that the parse action raised.
+
+    Example::
+        wd = Word(alphas)
+
+        @traceParseAction
+        def remove_duplicate_chars(tokens):
+            return ''.join(sorted(set(''.join(tokens))))
+
+        wds = OneOrMore(wd).setParseAction(remove_duplicate_chars)
+        print(wds.parseString("slkdjs sld sldd sdlf sdljf"))
+    prints::
+        >>entering remove_duplicate_chars(line: 'slkdjs sld sldd sdlf sdljf', 0, (['slkdjs', 'sld', 'sldd', 'sdlf', 'sdljf'], {}))
+        <3:
+            thisFunc = paArgs[0].__class__.__name__ + '.' + thisFunc
+        sys.stderr.write( ">>entering %s(line: '%s', %d, %r)\n" % (thisFunc,line(l,s),l,t) )
+        try:
+            ret = f(*paArgs)
+        except Exception as exc:
+            sys.stderr.write( "< ['aa', 'bb', 'cc']
+        delimitedList(Word(hexnums), delim=':', combine=True).parseString("AA:BB:CC:DD:EE") # -> ['AA:BB:CC:DD:EE']
+    """
+    dlName = _ustr(expr)+" ["+_ustr(delim)+" "+_ustr(expr)+"]..."
+    if combine:
+        return Combine( expr + ZeroOrMore( delim + expr ) ).setName(dlName)
+    else:
+        return ( expr + ZeroOrMore( Suppress( delim ) + expr ) ).setName(dlName)
+
+def countedArray( expr, intExpr=None ):
+    """
+    Helper to define a counted list of expressions.
+    This helper defines a pattern of the form::
+        integer expr expr expr...
+    where the leading integer tells how many expr expressions follow.
+    The matched tokens returns the array of expr tokens as a list - the leading count token is suppressed.
+    
+    If C{intExpr} is specified, it should be a pyparsing expression that produces an integer value.
+
+    Example::
+        countedArray(Word(alphas)).parseString('2 ab cd ef')  # -> ['ab', 'cd']
+
+        # in this parser, the leading integer value is given in binary,
+        # '10' indicating that 2 values are in the array
+        binaryConstant = Word('01').setParseAction(lambda t: int(t[0], 2))
+        countedArray(Word(alphas), intExpr=binaryConstant).parseString('10 ab cd ef')  # -> ['ab', 'cd']
+    """
+    arrayExpr = Forward()
+    def countFieldParseAction(s,l,t):
+        n = t[0]
+        arrayExpr << (n and Group(And([expr]*n)) or Group(empty))
+        return []
+    if intExpr is None:
+        intExpr = Word(nums).setParseAction(lambda t:int(t[0]))
+    else:
+        intExpr = intExpr.copy()
+    intExpr.setName("arrayLen")
+    intExpr.addParseAction(countFieldParseAction, callDuringTry=True)
+    return ( intExpr + arrayExpr ).setName('(len) ' + _ustr(expr) + '...')
+
+def _flatten(L):
+    ret = []
+    for i in L:
+        if isinstance(i,list):
+            ret.extend(_flatten(i))
+        else:
+            ret.append(i)
+    return ret
+
+def matchPreviousLiteral(expr):
+    """
+    Helper to define an expression that is indirectly defined from
+    the tokens matched in a previous expression, that is, it looks
+    for a 'repeat' of a previous expression.  For example::
+        first = Word(nums)
+        second = matchPreviousLiteral(first)
+        matchExpr = first + ":" + second
+    will match C{"1:1"}, but not C{"1:2"}.  Because this matches a
+    previous literal, will also match the leading C{"1:1"} in C{"1:10"}.
+    If this is not desired, use C{matchPreviousExpr}.
+    Do I{not} use with packrat parsing enabled.
+    """
+    rep = Forward()
+    def copyTokenToRepeater(s,l,t):
+        if t:
+            if len(t) == 1:
+                rep << t[0]
+            else:
+                # flatten t tokens
+                tflat = _flatten(t.asList())
+                rep << And(Literal(tt) for tt in tflat)
+        else:
+            rep << Empty()
+    expr.addParseAction(copyTokenToRepeater, callDuringTry=True)
+    rep.setName('(prev) ' + _ustr(expr))
+    return rep
+
+def matchPreviousExpr(expr):
+    """
+    Helper to define an expression that is indirectly defined from
+    the tokens matched in a previous expression, that is, it looks
+    for a 'repeat' of a previous expression.  For example::
+        first = Word(nums)
+        second = matchPreviousExpr(first)
+        matchExpr = first + ":" + second
+    will match C{"1:1"}, but not C{"1:2"}.  Because this matches by
+    expressions, will I{not} match the leading C{"1:1"} in C{"1:10"};
+    the expressions are evaluated first, and then compared, so
+    C{"1"} is compared with C{"10"}.
+    Do I{not} use with packrat parsing enabled.
+    """
+    rep = Forward()
+    e2 = expr.copy()
+    rep <<= e2
+    def copyTokenToRepeater(s,l,t):
+        matchTokens = _flatten(t.asList())
+        def mustMatchTheseTokens(s,l,t):
+            theseTokens = _flatten(t.asList())
+            if  theseTokens != matchTokens:
+                raise ParseException("",0,"")
+        rep.setParseAction( mustMatchTheseTokens, callDuringTry=True )
+    expr.addParseAction(copyTokenToRepeater, callDuringTry=True)
+    rep.setName('(prev) ' + _ustr(expr))
+    return rep
+
+def _escapeRegexRangeChars(s):
+    #~  escape these chars: ^-]
+    for c in r"\^-]":
+        s = s.replace(c,_bslash+c)
+    s = s.replace("\n",r"\n")
+    s = s.replace("\t",r"\t")
+    return _ustr(s)
+
+def oneOf( strs, caseless=False, useRegex=True ):
+    """
+    Helper to quickly define a set of alternative Literals, and makes sure to do
+    longest-first testing when there is a conflict, regardless of the input order,
+    but returns a C{L{MatchFirst}} for best performance.
+
+    Parameters:
+     - strs - a string of space-delimited literals, or a collection of string literals
+     - caseless - (default=C{False}) - treat all literals as caseless
+     - useRegex - (default=C{True}) - as an optimization, will generate a Regex
+          object; otherwise, will generate a C{MatchFirst} object (if C{caseless=True}, or
+          if creating a C{Regex} raises an exception)
+
+    Example::
+        comp_oper = oneOf("< = > <= >= !=")
+        var = Word(alphas)
+        number = Word(nums)
+        term = var | number
+        comparison_expr = term + comp_oper + term
+        print(comparison_expr.searchString("B = 12  AA=23 B<=AA AA>12"))
+    prints::
+        [['B', '=', '12'], ['AA', '=', '23'], ['B', '<=', 'AA'], ['AA', '>', '12']]
+    """
+    if caseless:
+        isequal = ( lambda a,b: a.upper() == b.upper() )
+        masks = ( lambda a,b: b.upper().startswith(a.upper()) )
+        parseElementClass = CaselessLiteral
+    else:
+        isequal = ( lambda a,b: a == b )
+        masks = ( lambda a,b: b.startswith(a) )
+        parseElementClass = Literal
+
+    symbols = []
+    if isinstance(strs,basestring):
+        symbols = strs.split()
+    elif isinstance(strs, Iterable):
+        symbols = list(strs)
+    else:
+        warnings.warn("Invalid argument to oneOf, expected string or iterable",
+                SyntaxWarning, stacklevel=2)
+    if not symbols:
+        return NoMatch()
+
+    i = 0
+    while i < len(symbols)-1:
+        cur = symbols[i]
+        for j,other in enumerate(symbols[i+1:]):
+            if ( isequal(other, cur) ):
+                del symbols[i+j+1]
+                break
+            elif ( masks(cur, other) ):
+                del symbols[i+j+1]
+                symbols.insert(i,other)
+                cur = other
+                break
+        else:
+            i += 1
+
+    if not caseless and useRegex:
+        #~ print (strs,"->", "|".join( [ _escapeRegexChars(sym) for sym in symbols] ))
+        try:
+            if len(symbols)==len("".join(symbols)):
+                return Regex( "[%s]" % "".join(_escapeRegexRangeChars(sym) for sym in symbols) ).setName(' | '.join(symbols))
+            else:
+                return Regex( "|".join(re.escape(sym) for sym in symbols) ).setName(' | '.join(symbols))
+        except Exception:
+            warnings.warn("Exception creating Regex for oneOf, building MatchFirst",
+                    SyntaxWarning, stacklevel=2)
+
+
+    # last resort, just use MatchFirst
+    return MatchFirst(parseElementClass(sym) for sym in symbols).setName(' | '.join(symbols))
+
+def dictOf( key, value ):
+    """
+    Helper to easily and clearly define a dictionary by specifying the respective patterns
+    for the key and value.  Takes care of defining the C{L{Dict}}, C{L{ZeroOrMore}}, and C{L{Group}} tokens
+    in the proper order.  The key pattern can include delimiting markers or punctuation,
+    as long as they are suppressed, thereby leaving the significant key text.  The value
+    pattern can include named results, so that the C{Dict} results can include named token
+    fields.
+
+    Example::
+        text = "shape: SQUARE posn: upper left color: light blue texture: burlap"
+        attr_expr = (label + Suppress(':') + OneOrMore(data_word, stopOn=label).setParseAction(' '.join))
+        print(OneOrMore(attr_expr).parseString(text).dump())
+        
+        attr_label = label
+        attr_value = Suppress(':') + OneOrMore(data_word, stopOn=label).setParseAction(' '.join)
+
+        # similar to Dict, but simpler call format
+        result = dictOf(attr_label, attr_value).parseString(text)
+        print(result.dump())
+        print(result['shape'])
+        print(result.shape)  # object attribute access works too
+        print(result.asDict())
+    prints::
+        [['shape', 'SQUARE'], ['posn', 'upper left'], ['color', 'light blue'], ['texture', 'burlap']]
+        - color: light blue
+        - posn: upper left
+        - shape: SQUARE
+        - texture: burlap
+        SQUARE
+        SQUARE
+        {'color': 'light blue', 'shape': 'SQUARE', 'posn': 'upper left', 'texture': 'burlap'}
+    """
+    return Dict( ZeroOrMore( Group ( key + value ) ) )
+
+def originalTextFor(expr, asString=True):
+    """
+    Helper to return the original, untokenized text for a given expression.  Useful to
+    restore the parsed fields of an HTML start tag into the raw tag text itself, or to
+    revert separate tokens with intervening whitespace back to the original matching
+    input text. By default, returns astring containing the original parsed text.  
+       
+    If the optional C{asString} argument is passed as C{False}, then the return value is a 
+    C{L{ParseResults}} containing any results names that were originally matched, and a 
+    single token containing the original matched text from the input string.  So if 
+    the expression passed to C{L{originalTextFor}} contains expressions with defined
+    results names, you must set C{asString} to C{False} if you want to preserve those
+    results name values.
+
+    Example::
+        src = "this is test  bold text  normal text "
+        for tag in ("b","i"):
+            opener,closer = makeHTMLTags(tag)
+            patt = originalTextFor(opener + SkipTo(closer) + closer)
+            print(patt.searchString(src)[0])
+    prints::
+        [' bold text ']
+        ['text']
+    """
+    locMarker = Empty().setParseAction(lambda s,loc,t: loc)
+    endlocMarker = locMarker.copy()
+    endlocMarker.callPreparse = False
+    matchExpr = locMarker("_original_start") + expr + endlocMarker("_original_end")
+    if asString:
+        extractText = lambda s,l,t: s[t._original_start:t._original_end]
+    else:
+        def extractText(s,l,t):
+            t[:] = [s[t.pop('_original_start'):t.pop('_original_end')]]
+    matchExpr.setParseAction(extractText)
+    matchExpr.ignoreExprs = expr.ignoreExprs
+    return matchExpr
+
+def ungroup(expr): 
+    """
+    Helper to undo pyparsing's default grouping of And expressions, even
+    if all but one are non-empty.
+    """
+    return TokenConverter(expr).setParseAction(lambda t:t[0])
+
+def locatedExpr(expr):
+    """
+    Helper to decorate a returned token with its starting and ending locations in the input string.
+    This helper adds the following results names:
+     - locn_start = location where matched expression begins
+     - locn_end = location where matched expression ends
+     - value = the actual parsed results
+
+    Be careful if the input text contains C{} characters, you may want to call
+    C{L{ParserElement.parseWithTabs}}
+
+    Example::
+        wd = Word(alphas)
+        for match in locatedExpr(wd).searchString("ljsdf123lksdjjf123lkkjj1222"):
+            print(match)
+    prints::
+        [[0, 'ljsdf', 5]]
+        [[8, 'lksdjjf', 15]]
+        [[18, 'lkkjj', 23]]
+    """
+    locator = Empty().setParseAction(lambda s,l,t: l)
+    return Group(locator("locn_start") + expr("value") + locator.copy().leaveWhitespace()("locn_end"))
+
+
+# convenience constants for positional expressions
+empty       = Empty().setName("empty")
+lineStart   = LineStart().setName("lineStart")
+lineEnd     = LineEnd().setName("lineEnd")
+stringStart = StringStart().setName("stringStart")
+stringEnd   = StringEnd().setName("stringEnd")
+
+_escapedPunc = Word( _bslash, r"\[]-*.$+^?()~ ", exact=2 ).setParseAction(lambda s,l,t:t[0][1])
+_escapedHexChar = Regex(r"\\0?[xX][0-9a-fA-F]+").setParseAction(lambda s,l,t:unichr(int(t[0].lstrip(r'\0x'),16)))
+_escapedOctChar = Regex(r"\\0[0-7]+").setParseAction(lambda s,l,t:unichr(int(t[0][1:],8)))
+_singleChar = _escapedPunc | _escapedHexChar | _escapedOctChar | CharsNotIn(r'\]', exact=1)
+_charRange = Group(_singleChar + Suppress("-") + _singleChar)
+_reBracketExpr = Literal("[") + Optional("^").setResultsName("negate") + Group( OneOrMore( _charRange | _singleChar ) ).setResultsName("body") + "]"
+
+def srange(s):
+    r"""
+    Helper to easily define string ranges for use in Word construction.  Borrows
+    syntax from regexp '[]' string range definitions::
+        srange("[0-9]")   -> "0123456789"
+        srange("[a-z]")   -> "abcdefghijklmnopqrstuvwxyz"
+        srange("[a-z$_]") -> "abcdefghijklmnopqrstuvwxyz$_"
+    The input string must be enclosed in []'s, and the returned string is the expanded
+    character set joined into a single string.
+    The values enclosed in the []'s may be:
+     - a single character
+     - an escaped character with a leading backslash (such as C{\-} or C{\]})
+     - an escaped hex character with a leading C{'\x'} (C{\x21}, which is a C{'!'} character) 
+         (C{\0x##} is also supported for backwards compatibility) 
+     - an escaped octal character with a leading C{'\0'} (C{\041}, which is a C{'!'} character)
+     - a range of any of the above, separated by a dash (C{'a-z'}, etc.)
+     - any combination of the above (C{'aeiouy'}, C{'a-zA-Z0-9_$'}, etc.)
+    """
+    _expanded = lambda p: p if not isinstance(p,ParseResults) else ''.join(unichr(c) for c in range(ord(p[0]),ord(p[1])+1))
+    try:
+        return "".join(_expanded(part) for part in _reBracketExpr.parseString(s).body)
+    except Exception:
+        return ""
+
+def matchOnlyAtCol(n):
+    """
+    Helper method for defining parse actions that require matching at a specific
+    column in the input text.
+    """
+    def verifyCol(strg,locn,toks):
+        if col(locn,strg) != n:
+            raise ParseException(strg,locn,"matched token not at column %d" % n)
+    return verifyCol
+
+def replaceWith(replStr):
+    """
+    Helper method for common parse actions that simply return a literal value.  Especially
+    useful when used with C{L{transformString}()}.
+
+    Example::
+        num = Word(nums).setParseAction(lambda toks: int(toks[0]))
+        na = oneOf("N/A NA").setParseAction(replaceWith(math.nan))
+        term = na | num
+        
+        OneOrMore(term).parseString("324 234 N/A 234") # -> [324, 234, nan, 234]
+    """
+    return lambda s,l,t: [replStr]
+
+def removeQuotes(s,l,t):
+    """
+    Helper parse action for removing quotation marks from parsed quoted strings.
+
+    Example::
+        # by default, quotation marks are included in parsed results
+        quotedString.parseString("'Now is the Winter of our Discontent'") # -> ["'Now is the Winter of our Discontent'"]
+
+        # use removeQuotes to strip quotation marks from parsed results
+        quotedString.setParseAction(removeQuotes)
+        quotedString.parseString("'Now is the Winter of our Discontent'") # -> ["Now is the Winter of our Discontent"]
+    """
+    return t[0][1:-1]
+
+def tokenMap(func, *args):
+    """
+    Helper to define a parse action by mapping a function to all elements of a ParseResults list.If any additional 
+    args are passed, they are forwarded to the given function as additional arguments after
+    the token, as in C{hex_integer = Word(hexnums).setParseAction(tokenMap(int, 16))}, which will convert the
+    parsed data to an integer using base 16.
+
+    Example (compare the last to example in L{ParserElement.transformString}::
+        hex_ints = OneOrMore(Word(hexnums)).setParseAction(tokenMap(int, 16))
+        hex_ints.runTests('''
+            00 11 22 aa FF 0a 0d 1a
+            ''')
+        
+        upperword = Word(alphas).setParseAction(tokenMap(str.upper))
+        OneOrMore(upperword).runTests('''
+            my kingdom for a horse
+            ''')
+
+        wd = Word(alphas).setParseAction(tokenMap(str.title))
+        OneOrMore(wd).setParseAction(' '.join).runTests('''
+            now is the winter of our discontent made glorious summer by this sun of york
+            ''')
+    prints::
+        00 11 22 aa FF 0a 0d 1a
+        [0, 17, 34, 170, 255, 10, 13, 26]
+
+        my kingdom for a horse
+        ['MY', 'KINGDOM', 'FOR', 'A', 'HORSE']
+
+        now is the winter of our discontent made glorious summer by this sun of york
+        ['Now Is The Winter Of Our Discontent Made Glorious Summer By This Sun Of York']
+    """
+    def pa(s,l,t):
+        return [func(tokn, *args) for tokn in t]
+
+    try:
+        func_name = getattr(func, '__name__', 
+                            getattr(func, '__class__').__name__)
+    except Exception:
+        func_name = str(func)
+    pa.__name__ = func_name
+
+    return pa
+
+upcaseTokens = tokenMap(lambda t: _ustr(t).upper())
+"""(Deprecated) Helper parse action to convert tokens to upper case. Deprecated in favor of L{pyparsing_common.upcaseTokens}"""
+
+downcaseTokens = tokenMap(lambda t: _ustr(t).lower())
+"""(Deprecated) Helper parse action to convert tokens to lower case. Deprecated in favor of L{pyparsing_common.downcaseTokens}"""
+    
+def _makeTags(tagStr, xml):
+    """Internal helper to construct opening and closing tag expressions, given a tag name"""
+    if isinstance(tagStr,basestring):
+        resname = tagStr
+        tagStr = Keyword(tagStr, caseless=not xml)
+    else:
+        resname = tagStr.name
+
+    tagAttrName = Word(alphas,alphanums+"_-:")
+    if (xml):
+        tagAttrValue = dblQuotedString.copy().setParseAction( removeQuotes )
+        openTag = Suppress("<") + tagStr("tag") + \
+                Dict(ZeroOrMore(Group( tagAttrName + Suppress("=") + tagAttrValue ))) + \
+                Optional("/",default=[False]).setResultsName("empty").setParseAction(lambda s,l,t:t[0]=='/') + Suppress(">")
+    else:
+        printablesLessRAbrack = "".join(c for c in printables if c not in ">")
+        tagAttrValue = quotedString.copy().setParseAction( removeQuotes ) | Word(printablesLessRAbrack)
+        openTag = Suppress("<") + tagStr("tag") + \
+                Dict(ZeroOrMore(Group( tagAttrName.setParseAction(downcaseTokens) + \
+                Optional( Suppress("=") + tagAttrValue ) ))) + \
+                Optional("/",default=[False]).setResultsName("empty").setParseAction(lambda s,l,t:t[0]=='/') + Suppress(">")
+    closeTag = Combine(_L("")
+
+    openTag = openTag.setResultsName("start"+"".join(resname.replace(":"," ").title().split())).setName("<%s>" % resname)
+    closeTag = closeTag.setResultsName("end"+"".join(resname.replace(":"," ").title().split())).setName("" % resname)
+    openTag.tag = resname
+    closeTag.tag = resname
+    return openTag, closeTag
+
+def makeHTMLTags(tagStr):
+    """
+    Helper to construct opening and closing tag expressions for HTML, given a tag name. Matches
+    tags in either upper or lower case, attributes with namespaces and with quoted or unquoted values.
+
+    Example::
+        text = 'More info at the pyparsing wiki page'
+        # makeHTMLTags returns pyparsing expressions for the opening and closing tags as a 2-tuple
+        a,a_end = makeHTMLTags("A")
+        link_expr = a + SkipTo(a_end)("link_text") + a_end
+        
+        for link in link_expr.searchString(text):
+            # attributes in the  tag (like "href" shown here) are also accessible as named results
+            print(link.link_text, '->', link.href)
+    prints::
+        pyparsing -> http://pyparsing.wikispaces.com
+    """
+    return _makeTags( tagStr, False )
+
+def makeXMLTags(tagStr):
+    """
+    Helper to construct opening and closing tag expressions for XML, given a tag name. Matches
+    tags only in the given upper/lower case.
+
+    Example: similar to L{makeHTMLTags}
+    """
+    return _makeTags( tagStr, True )
+
+def withAttribute(*args,**attrDict):
+    """
+    Helper to create a validating parse action to be used with start tags created
+    with C{L{makeXMLTags}} or C{L{makeHTMLTags}}. Use C{withAttribute} to qualify a starting tag
+    with a required attribute value, to avoid false matches on common tags such as
+    C{} or C{
}.
+
+    Call C{withAttribute} with a series of attribute names and values. Specify the list
+    of filter attributes names and values as:
+     - keyword arguments, as in C{(align="right")}, or
+     - as an explicit dict with C{**} operator, when an attribute name is also a Python
+          reserved word, as in C{**{"class":"Customer", "align":"right"}}
+     - a list of name-value tuples, as in ( ("ns1:class", "Customer"), ("ns2:align","right") )
+    For attribute names with a namespace prefix, you must use the second form.  Attribute
+    names are matched insensitive to upper/lower case.
+       
+    If just testing for C{class} (with or without a namespace), use C{L{withClass}}.
+
+    To verify that the attribute exists, but without specifying a value, pass
+    C{withAttribute.ANY_VALUE} as the value.
+
+    Example::
+        html = '''
+            

+            Some text
+            
1 4 0 1 0

+            
1,3 2,3 1,1

+            
this has no type

+            

+                
+        '''
+        div,div_end = makeHTMLTags("div")
+
+        # only match div tag having a type attribute with value "grid"
+        div_grid = div().setParseAction(withAttribute(type="grid"))
+        grid_expr = div_grid + SkipTo(div | div_end)("body")
+        for grid_header in grid_expr.searchString(html):
+            print(grid_header.body)
+        
+        # construct a match with any div tag having a type attribute, regardless of the value
+        div_any_type = div().setParseAction(withAttribute(type=withAttribute.ANY_VALUE))
+        div_expr = div_any_type + SkipTo(div | div_end)("body")
+        for div_header in div_expr.searchString(html):
+            print(div_header.body)
+    prints::
+        1 4 0 1 0
+
+        1 4 0 1 0
+        1,3 2,3 1,1
+    """
+    if args:
+        attrs = args[:]
+    else:
+        attrs = attrDict.items()
+    attrs = [(k,v) for k,v in attrs]
+    def pa(s,l,tokens):
+        for attrName,attrValue in attrs:
+            if attrName not in tokens:
+                raise ParseException(s,l,"no matching attribute " + attrName)
+            if attrValue != withAttribute.ANY_VALUE and tokens[attrName] != attrValue:
+                raise ParseException(s,l,"attribute '%s' has value '%s', must be '%s'" %
+                                            (attrName, tokens[attrName], attrValue))
+    return pa
+withAttribute.ANY_VALUE = object()
+
+def withClass(classname, namespace=''):
+    """
+    Simplified version of C{L{withAttribute}} when matching on a div class - made
+    difficult because C{class} is a reserved word in Python.
+
+    Example::
+        html = '''
+            

+            Some text
+            
1 4 0 1 0

+            
1,3 2,3 1,1

+            
this <div> has no class

+            

+                
+        '''
+        div,div_end = makeHTMLTags("div")
+        div_grid = div().setParseAction(withClass("grid"))
+        
+        grid_expr = div_grid + SkipTo(div | div_end)("body")
+        for grid_header in grid_expr.searchString(html):
+            print(grid_header.body)
+        
+        div_any_type = div().setParseAction(withClass(withAttribute.ANY_VALUE))
+        div_expr = div_any_type + SkipTo(div | div_end)("body")
+        for div_header in div_expr.searchString(html):
+            print(div_header.body)
+    prints::
+        1 4 0 1 0
+
+        1 4 0 1 0
+        1,3 2,3 1,1
+    """
+    classattr = "%s:class" % namespace if namespace else "class"
+    return withAttribute(**{classattr : classname})        
+
+opAssoc = _Constants()
+opAssoc.LEFT = object()
+opAssoc.RIGHT = object()
+
+def infixNotation( baseExpr, opList, lpar=Suppress('('), rpar=Suppress(')') ):
+    """
+    Helper method for constructing grammars of expressions made up of
+    operators working in a precedence hierarchy.  Operators may be unary or
+    binary, left- or right-associative.  Parse actions can also be attached
+    to operator expressions. The generated parser will also recognize the use 
+    of parentheses to override operator precedences (see example below).
+    
+    Note: if you define a deep operator list, you may see performance issues
+    when using infixNotation. See L{ParserElement.enablePackrat} for a
+    mechanism to potentially improve your parser performance.
+
+    Parameters:
+     - baseExpr - expression representing the most basic element for the nested
+     - opList - list of tuples, one for each operator precedence level in the
+      expression grammar; each tuple is of the form
+      (opExpr, numTerms, rightLeftAssoc, parseAction), where:
+       - opExpr is the pyparsing expression for the operator;
+          may also be a string, which will be converted to a Literal;
+          if numTerms is 3, opExpr is a tuple of two expressions, for the
+          two operators separating the 3 terms
+       - numTerms is the number of terms for this operator (must
+          be 1, 2, or 3)
+       - rightLeftAssoc is the indicator whether the operator is
+          right or left associative, using the pyparsing-defined
+          constants C{opAssoc.RIGHT} and C{opAssoc.LEFT}.
+       - parseAction is the parse action to be associated with
+          expressions matching this operator expression (the
+          parse action tuple member may be omitted); if the parse action
+          is passed a tuple or list of functions, this is equivalent to
+          calling C{setParseAction(*fn)} (L{ParserElement.setParseAction})
+     - lpar - expression for matching left-parentheses (default=C{Suppress('(')})
+     - rpar - expression for matching right-parentheses (default=C{Suppress(')')})
+
+    Example::
+        # simple example of four-function arithmetic with ints and variable names
+        integer = pyparsing_common.signed_integer
+        varname = pyparsing_common.identifier 
+        
+        arith_expr = infixNotation(integer | varname,
+            [
+            ('-', 1, opAssoc.RIGHT),
+            (oneOf('* /'), 2, opAssoc.LEFT),
+            (oneOf('+ -'), 2, opAssoc.LEFT),
+            ])
+        
+        arith_expr.runTests('''
+            5+3*6
+            (5+3)*6
+            -2--11
+            ''', fullDump=False)
+    prints::
+        5+3*6
+        [[5, '+', [3, '*', 6]]]
+
+        (5+3)*6
+        [[[5, '+', 3], '*', 6]]
+
+        -2--11
+        [[['-', 2], '-', ['-', 11]]]
+    """
+    ret = Forward()
+    lastExpr = baseExpr | ( lpar + ret + rpar )
+    for i,operDef in enumerate(opList):
+        opExpr,arity,rightLeftAssoc,pa = (operDef + (None,))[:4]
+        termName = "%s term" % opExpr if arity < 3 else "%s%s term" % opExpr
+        if arity == 3:
+            if opExpr is None or len(opExpr) != 2:
+                raise ValueError("if numterms=3, opExpr must be a tuple or list of two expressions")
+            opExpr1, opExpr2 = opExpr
+        thisExpr = Forward().setName(termName)
+        if rightLeftAssoc == opAssoc.LEFT:
+            if arity == 1:
+                matchExpr = FollowedBy(lastExpr + opExpr) + Group( lastExpr + OneOrMore( opExpr ) )
+            elif arity == 2:
+                if opExpr is not None:
+                    matchExpr = FollowedBy(lastExpr + opExpr + lastExpr) + Group( lastExpr + OneOrMore( opExpr + lastExpr ) )
+                else:
+                    matchExpr = FollowedBy(lastExpr+lastExpr) + Group( lastExpr + OneOrMore(lastExpr) )
+            elif arity == 3:
+                matchExpr = FollowedBy(lastExpr + opExpr1 + lastExpr + opExpr2 + lastExpr) + \
+                            Group( lastExpr + opExpr1 + lastExpr + opExpr2 + lastExpr )
+            else:
+                raise ValueError("operator must be unary (1), binary (2), or ternary (3)")
+        elif rightLeftAssoc == opAssoc.RIGHT:
+            if arity == 1:
+                # try to avoid LR with this extra test
+                if not isinstance(opExpr, Optional):
+                    opExpr = Optional(opExpr)
+                matchExpr = FollowedBy(opExpr.expr + thisExpr) + Group( opExpr + thisExpr )
+            elif arity == 2:
+                if opExpr is not None:
+                    matchExpr = FollowedBy(lastExpr + opExpr + thisExpr) + Group( lastExpr + OneOrMore( opExpr + thisExpr ) )
+                else:
+                    matchExpr = FollowedBy(lastExpr + thisExpr) + Group( lastExpr + OneOrMore( thisExpr ) )
+            elif arity == 3:
+                matchExpr = FollowedBy(lastExpr + opExpr1 + thisExpr + opExpr2 + thisExpr) + \
+                            Group( lastExpr + opExpr1 + thisExpr + opExpr2 + thisExpr )
+            else:
+                raise ValueError("operator must be unary (1), binary (2), or ternary (3)")
+        else:
+            raise ValueError("operator must indicate right or left associativity")
+        if pa:
+            if isinstance(pa, (tuple, list)):
+                matchExpr.setParseAction(*pa)
+            else:
+                matchExpr.setParseAction(pa)
+        thisExpr <<= ( matchExpr.setName(termName) | lastExpr )
+        lastExpr = thisExpr
+    ret <<= lastExpr
+    return ret
+
+operatorPrecedence = infixNotation
+"""(Deprecated) Former name of C{L{infixNotation}}, will be dropped in a future release."""
+
+dblQuotedString = Combine(Regex(r'"(?:[^"\n\r\\]|(?:"")|(?:\\(?:[^x]|x[0-9a-fA-F]+)))*')+'"').setName("string enclosed in double quotes")
+sglQuotedString = Combine(Regex(r"'(?:[^'\n\r\\]|(?:'')|(?:\\(?:[^x]|x[0-9a-fA-F]+)))*")+"'").setName("string enclosed in single quotes")
+quotedString = Combine(Regex(r'"(?:[^"\n\r\\]|(?:"")|(?:\\(?:[^x]|x[0-9a-fA-F]+)))*')+'"'|
+                       Regex(r"'(?:[^'\n\r\\]|(?:'')|(?:\\(?:[^x]|x[0-9a-fA-F]+)))*")+"'").setName("quotedString using single or double quotes")
+unicodeString = Combine(_L('u') + quotedString.copy()).setName("unicode string literal")
+
+def nestedExpr(opener="(", closer=")", content=None, ignoreExpr=quotedString.copy()):
+    """
+    Helper method for defining nested lists enclosed in opening and closing
+    delimiters ("(" and ")" are the default).
+
+    Parameters:
+     - opener - opening character for a nested list (default=C{"("}); can also be a pyparsing expression
+     - closer - closing character for a nested list (default=C{")"}); can also be a pyparsing expression
+     - content - expression for items within the nested lists (default=C{None})
+     - ignoreExpr - expression for ignoring opening and closing delimiters (default=C{quotedString})
+
+    If an expression is not provided for the content argument, the nested
+    expression will capture all whitespace-delimited content between delimiters
+    as a list of separate values.
+
+    Use the C{ignoreExpr} argument to define expressions that may contain
+    opening or closing characters that should not be treated as opening
+    or closing characters for nesting, such as quotedString or a comment
+    expression.  Specify multiple expressions using an C{L{Or}} or C{L{MatchFirst}}.
+    The default is L{quotedString}, but if no expressions are to be ignored,
+    then pass C{None} for this argument.
+
+    Example::
+        data_type = oneOf("void int short long char float double")
+        decl_data_type = Combine(data_type + Optional(Word('*')))
+        ident = Word(alphas+'_', alphanums+'_')
+        number = pyparsing_common.number
+        arg = Group(decl_data_type + ident)
+        LPAR,RPAR = map(Suppress, "()")
+
+        code_body = nestedExpr('{', '}', ignoreExpr=(quotedString | cStyleComment))
+
+        c_function = (decl_data_type("type") 
+                      + ident("name")
+                      + LPAR + Optional(delimitedList(arg), [])("args") + RPAR 
+                      + code_body("body"))
+        c_function.ignore(cStyleComment)
+        
+        source_code = '''
+            int is_odd(int x) { 
+                return (x%2); 
+            }
+                
+            int dec_to_hex(char hchar) { 
+                if (hchar >= '0' && hchar <= '9') { 
+                    return (ord(hchar)-ord('0')); 
+                } else { 
+                    return (10+ord(hchar)-ord('A'));
+                } 
+            }
+        '''
+        for func in c_function.searchString(source_code):
+            print("%(name)s (%(type)s) args: %(args)s" % func)
+
+    prints::
+        is_odd (int) args: [['int', 'x']]
+        dec_to_hex (int) args: [['char', 'hchar']]
+    """
+    if opener == closer:
+        raise ValueError("opening and closing strings cannot be the same")
+    if content is None:
+        if isinstance(opener,basestring) and isinstance(closer,basestring):
+            if len(opener) == 1 and len(closer)==1:
+                if ignoreExpr is not None:
+                    content = (Combine(OneOrMore(~ignoreExpr +
+                                    CharsNotIn(opener+closer+ParserElement.DEFAULT_WHITE_CHARS,exact=1))
+                                ).setParseAction(lambda t:t[0].strip()))
+                else:
+                    content = (empty.copy()+CharsNotIn(opener+closer+ParserElement.DEFAULT_WHITE_CHARS
+                                ).setParseAction(lambda t:t[0].strip()))
+            else:
+                if ignoreExpr is not None:
+                    content = (Combine(OneOrMore(~ignoreExpr + 
+                                    ~Literal(opener) + ~Literal(closer) +
+                                    CharsNotIn(ParserElement.DEFAULT_WHITE_CHARS,exact=1))
+                                ).setParseAction(lambda t:t[0].strip()))
+                else:
+                    content = (Combine(OneOrMore(~Literal(opener) + ~Literal(closer) +
+                                    CharsNotIn(ParserElement.DEFAULT_WHITE_CHARS,exact=1))
+                                ).setParseAction(lambda t:t[0].strip()))
+        else:
+            raise ValueError("opening and closing arguments must be strings if no content expression is given")
+    ret = Forward()
+    if ignoreExpr is not None:
+        ret <<= Group( Suppress(opener) + ZeroOrMore( ignoreExpr | ret | content ) + Suppress(closer) )
+    else:
+        ret <<= Group( Suppress(opener) + ZeroOrMore( ret | content )  + Suppress(closer) )
+    ret.setName('nested %s%s expression' % (opener,closer))
+    return ret
+
+def indentedBlock(blockStatementExpr, indentStack, indent=True):
+    """
+    Helper method for defining space-delimited indentation blocks, such as
+    those used to define block statements in Python source code.
+
+    Parameters:
+     - blockStatementExpr - expression defining syntax of statement that
+            is repeated within the indented block
+     - indentStack - list created by caller to manage indentation stack
+            (multiple statementWithIndentedBlock expressions within a single grammar
+            should share a common indentStack)
+     - indent - boolean indicating whether block must be indented beyond the
+            the current level; set to False for block of left-most statements
+            (default=C{True})
+
+    A valid block must contain at least one C{blockStatement}.
+
+    Example::
+        data = '''
+        def A(z):
+          A1
+          B = 100
+          G = A2
+          A2
+          A3
+        B
+        def BB(a,b,c):
+          BB1
+          def BBA():
+            bba1
+            bba2
+            bba3
+        C
+        D
+        def spam(x,y):
+             def eggs(z):
+                 pass
+        '''
+
+
+        indentStack = [1]
+        stmt = Forward()
+
+        identifier = Word(alphas, alphanums)
+        funcDecl = ("def" + identifier + Group( "(" + Optional( delimitedList(identifier) ) + ")" ) + ":")
+        func_body = indentedBlock(stmt, indentStack)
+        funcDef = Group( funcDecl + func_body )
+
+        rvalue = Forward()
+        funcCall = Group(identifier + "(" + Optional(delimitedList(rvalue)) + ")")
+        rvalue << (funcCall | identifier | Word(nums))
+        assignment = Group(identifier + "=" + rvalue)
+        stmt << ( funcDef | assignment | identifier )
+
+        module_body = OneOrMore(stmt)
+
+        parseTree = module_body.parseString(data)
+        parseTree.pprint()
+    prints::
+        [['def',
+          'A',
+          ['(', 'z', ')'],
+          ':',
+          [['A1'], [['B', '=', '100']], [['G', '=', 'A2']], ['A2'], ['A3']]],
+         'B',
+         ['def',
+          'BB',
+          ['(', 'a', 'b', 'c', ')'],
+          ':',
+          [['BB1'], [['def', 'BBA', ['(', ')'], ':', [['bba1'], ['bba2'], ['bba3']]]]]],
+         'C',
+         'D',
+         ['def',
+          'spam',
+          ['(', 'x', 'y', ')'],
+          ':',
+          [[['def', 'eggs', ['(', 'z', ')'], ':', [['pass']]]]]]] 
+    """
+    def checkPeerIndent(s,l,t):
+        if l >= len(s): return
+        curCol = col(l,s)
+        if curCol != indentStack[-1]:
+            if curCol > indentStack[-1]:
+                raise ParseFatalException(s,l,"illegal nesting")
+            raise ParseException(s,l,"not a peer entry")
+
+    def checkSubIndent(s,l,t):
+        curCol = col(l,s)
+        if curCol > indentStack[-1]:
+            indentStack.append( curCol )
+        else:
+            raise ParseException(s,l,"not a subentry")
+
+    def checkUnindent(s,l,t):
+        if l >= len(s): return
+        curCol = col(l,s)
+        if not(indentStack and curCol < indentStack[-1] and curCol <= indentStack[-2]):
+            raise ParseException(s,l,"not an unindent")
+        indentStack.pop()
+
+    NL = OneOrMore(LineEnd().setWhitespaceChars("\t ").suppress())
+    INDENT = (Empty() + Empty().setParseAction(checkSubIndent)).setName('INDENT')
+    PEER   = Empty().setParseAction(checkPeerIndent).setName('')
+    UNDENT = Empty().setParseAction(checkUnindent).setName('UNINDENT')
+    if indent:
+        smExpr = Group( Optional(NL) +
+            #~ FollowedBy(blockStatementExpr) +
+            INDENT + (OneOrMore( PEER + Group(blockStatementExpr) + Optional(NL) )) + UNDENT)
+    else:
+        smExpr = Group( Optional(NL) +
+            (OneOrMore( PEER + Group(blockStatementExpr) + Optional(NL) )) )
+    blockStatementExpr.ignore(_bslash + LineEnd())
+    return smExpr.setName('indented block')
+
+alphas8bit = srange(r"[\0xc0-\0xd6\0xd8-\0xf6\0xf8-\0xff]")
+punc8bit = srange(r"[\0xa1-\0xbf\0xd7\0xf7]")
+
+anyOpenTag,anyCloseTag = makeHTMLTags(Word(alphas,alphanums+"_:").setName('any tag'))
+_htmlEntityMap = dict(zip("gt lt amp nbsp quot apos".split(),'><& "\''))
+commonHTMLEntity = Regex('&(?P' + '|'.join(_htmlEntityMap.keys()) +");").setName("common HTML entity")
+def replaceHTMLEntity(t):
+    """Helper parser action to replace common HTML entities with their special characters"""
+    return _htmlEntityMap.get(t.entity)
+
+# it's easy to get these comment structures wrong - they're very common, so may as well make them available
+cStyleComment = Combine(Regex(r"/\*(?:[^*]|\*(?!/))*") + '*/').setName("C style comment")
+"Comment of the form C{/* ... */}"
+
+htmlComment = Regex(r"").setName("HTML comment")
+"Comment of the form C{}"
+
+restOfLine = Regex(r".*").leaveWhitespace().setName("rest of line")
+dblSlashComment = Regex(r"//(?:\\\n|[^\n])*").setName("// comment")
+"Comment of the form C{// ... (to end of line)}"
+
+cppStyleComment = Combine(Regex(r"/\*(?:[^*]|\*(?!/))*") + '*/'| dblSlashComment).setName("C++ style comment")
+"Comment of either form C{L{cStyleComment}} or C{L{dblSlashComment}}"
+
+javaStyleComment = cppStyleComment
+"Same as C{L{cppStyleComment}}"
+
+pythonStyleComment = Regex(r"#.*").setName("Python style comment")
+"Comment of the form C{# ... (to end of line)}"
+
+_commasepitem = Combine(OneOrMore(Word(printables, excludeChars=',') +
+                                  Optional( Word(" \t") +
+                                            ~Literal(",") + ~LineEnd() ) ) ).streamline().setName("commaItem")
+commaSeparatedList = delimitedList( Optional( quotedString.copy() | _commasepitem, default="") ).setName("commaSeparatedList")
+"""(Deprecated) Predefined expression of 1 or more printable words or quoted strings, separated by commas.
+   This expression is deprecated in favor of L{pyparsing_common.comma_separated_list}."""
+
+# some other useful expressions - using lower-case class name since we are really using this as a namespace
+class pyparsing_common:
+    """
+    Here are some common low-level expressions that may be useful in jump-starting parser development:
+     - numeric forms (L{integers}, L{reals}, L{scientific notation})
+     - common L{programming identifiers}
+     - network addresses (L{MAC}, L{IPv4}, L{IPv6})
+     - ISO8601 L{dates} and L{datetime}
+     - L{UUID}
+     - L{comma-separated list}
+    Parse actions:
+     - C{L{convertToInteger}}
+     - C{L{convertToFloat}}
+     - C{L{convertToDate}}
+     - C{L{convertToDatetime}}
+     - C{L{stripHTMLTags}}
+     - C{L{upcaseTokens}}
+     - C{L{downcaseTokens}}
+
+    Example::
+        pyparsing_common.number.runTests('''
+            # any int or real number, returned as the appropriate type
+            100
+            -100
+            +100
+            3.14159
+            6.02e23
+            1e-12
+            ''')
+
+        pyparsing_common.fnumber.runTests('''
+            # any int or real number, returned as float
+            100
+            -100
+            +100
+            3.14159
+            6.02e23
+            1e-12
+            ''')
+
+        pyparsing_common.hex_integer.runTests('''
+            # hex numbers
+            100
+            FF
+            ''')
+
+        pyparsing_common.fraction.runTests('''
+            # fractions
+            1/2
+            -3/4
+            ''')
+
+        pyparsing_common.mixed_integer.runTests('''
+            # mixed fractions
+            1
+            1/2
+            -3/4
+            1-3/4
+            ''')
+
+        import uuid
+        pyparsing_common.uuid.setParseAction(tokenMap(uuid.UUID))
+        pyparsing_common.uuid.runTests('''
+            # uuid
+            12345678-1234-5678-1234-567812345678
+            ''')
+    prints::
+        # any int or real number, returned as the appropriate type
+        100
+        [100]
+
+        -100
+        [-100]
+
+        +100
+        [100]
+
+        3.14159
+        [3.14159]
+
+        6.02e23
+        [6.02e+23]
+
+        1e-12
+        [1e-12]
+
+        # any int or real number, returned as float
+        100
+        [100.0]
+
+        -100
+        [-100.0]
+
+        +100
+        [100.0]
+
+        3.14159
+        [3.14159]
+
+        6.02e23
+        [6.02e+23]
+
+        1e-12
+        [1e-12]
+
+        # hex numbers
+        100
+        [256]
+
+        FF
+        [255]
+
+        # fractions
+        1/2
+        [0.5]
+
+        -3/4
+        [-0.75]
+
+        # mixed fractions
+        1
+        [1]
+
+        1/2
+        [0.5]
+
+        -3/4
+        [-0.75]
+
+        1-3/4
+        [1.75]
+
+        # uuid
+        12345678-1234-5678-1234-567812345678
+        [UUID('12345678-1234-5678-1234-567812345678')]
+    """
+
+    convertToInteger = tokenMap(int)
+    """
+    Parse action for converting parsed integers to Python int
+    """
+
+    convertToFloat = tokenMap(float)
+    """
+    Parse action for converting parsed numbers to Python float
+    """
+
+    integer = Word(nums).setName("integer").setParseAction(convertToInteger)
+    """expression that parses an unsigned integer, returns an int"""
+
+    hex_integer = Word(hexnums).setName("hex integer").setParseAction(tokenMap(int,16))
+    """expression that parses a hexadecimal integer, returns an int"""
+
+    signed_integer = Regex(r'[+-]?\d+').setName("signed integer").setParseAction(convertToInteger)
+    """expression that parses an integer with optional leading sign, returns an int"""
+
+    fraction = (signed_integer().setParseAction(convertToFloat) + '/' + signed_integer().setParseAction(convertToFloat)).setName("fraction")
+    """fractional expression of an integer divided by an integer, returns a float"""
+    fraction.addParseAction(lambda t: t[0]/t[-1])
+
+    mixed_integer = (fraction | signed_integer + Optional(Optional('-').suppress() + fraction)).setName("fraction or mixed integer-fraction")
+    """mixed integer of the form 'integer - fraction', with optional leading integer, returns float"""
+    mixed_integer.addParseAction(sum)
+
+    real = Regex(r'[+-]?\d+\.\d*').setName("real number").setParseAction(convertToFloat)
+    """expression that parses a floating point number and returns a float"""
+
+    sci_real = Regex(r'[+-]?\d+([eE][+-]?\d+|\.\d*([eE][+-]?\d+)?)').setName("real number with scientific notation").setParseAction(convertToFloat)
+    """expression that parses a floating point number with optional scientific notation and returns a float"""
+
+    # streamlining this expression makes the docs nicer-looking
+    number = (sci_real | real | signed_integer).streamline()
+    """any numeric expression, returns the corresponding Python type"""
+
+    fnumber = Regex(r'[+-]?\d+\.?\d*([eE][+-]?\d+)?').setName("fnumber").setParseAction(convertToFloat)
+    """any int or real number, returned as float"""
+    
+    identifier = Word(alphas+'_', alphanums+'_').setName("identifier")
+    """typical code identifier (leading alpha or '_', followed by 0 or more alphas, nums, or '_')"""
+    
+    ipv4_address = Regex(r'(25[0-5]|2[0-4][0-9]|1?[0-9]{1,2})(\.(25[0-5]|2[0-4][0-9]|1?[0-9]{1,2})){3}').setName("IPv4 address")
+    "IPv4 address (C{0.0.0.0 - 255.255.255.255})"
+
+    _ipv6_part = Regex(r'[0-9a-fA-F]{1,4}').setName("hex_integer")
+    _full_ipv6_address = (_ipv6_part + (':' + _ipv6_part)*7).setName("full IPv6 address")
+    _short_ipv6_address = (Optional(_ipv6_part + (':' + _ipv6_part)*(0,6)) + "::" + Optional(_ipv6_part + (':' + _ipv6_part)*(0,6))).setName("short IPv6 address")
+    _short_ipv6_address.addCondition(lambda t: sum(1 for tt in t if pyparsing_common._ipv6_part.matches(tt)) < 8)
+    _mixed_ipv6_address = ("::ffff:" + ipv4_address).setName("mixed IPv6 address")
+    ipv6_address = Combine((_full_ipv6_address | _mixed_ipv6_address | _short_ipv6_address).setName("IPv6 address")).setName("IPv6 address")
+    "IPv6 address (long, short, or mixed form)"
+    
+    mac_address = Regex(r'[0-9a-fA-F]{2}([:.-])[0-9a-fA-F]{2}(?:\1[0-9a-fA-F]{2}){4}').setName("MAC address")
+    "MAC address xx:xx:xx:xx:xx (may also have '-' or '.' delimiters)"
+
+    @staticmethod
+    def convertToDate(fmt="%Y-%m-%d"):
+        """
+        Helper to create a parse action for converting parsed date string to Python datetime.date
+
+        Params -
+         - fmt - format to be passed to datetime.strptime (default=C{"%Y-%m-%d"})
+
+        Example::
+            date_expr = pyparsing_common.iso8601_date.copy()
+            date_expr.setParseAction(pyparsing_common.convertToDate())
+            print(date_expr.parseString("1999-12-31"))
+        prints::
+            [datetime.date(1999, 12, 31)]
+        """
+        def cvt_fn(s,l,t):
+            try:
+                return datetime.strptime(t[0], fmt).date()
+            except ValueError as ve:
+                raise ParseException(s, l, str(ve))
+        return cvt_fn
+
+    @staticmethod
+    def convertToDatetime(fmt="%Y-%m-%dT%H:%M:%S.%f"):
+        """
+        Helper to create a parse action for converting parsed datetime string to Python datetime.datetime
+
+        Params -
+         - fmt - format to be passed to datetime.strptime (default=C{"%Y-%m-%dT%H:%M:%S.%f"})
+
+        Example::
+            dt_expr = pyparsing_common.iso8601_datetime.copy()
+            dt_expr.setParseAction(pyparsing_common.convertToDatetime())
+            print(dt_expr.parseString("1999-12-31T23:59:59.999"))
+        prints::
+            [datetime.datetime(1999, 12, 31, 23, 59, 59, 999000)]
+        """
+        def cvt_fn(s,l,t):
+            try:
+                return datetime.strptime(t[0], fmt)
+            except ValueError as ve:
+                raise ParseException(s, l, str(ve))
+        return cvt_fn
+
+    iso8601_date = Regex(r'(?P\d{4})(?:-(?P\d\d)(?:-(?P\d\d))?)?').setName("ISO8601 date")
+    "ISO8601 date (C{yyyy-mm-dd})"
+
+    iso8601_datetime = Regex(r'(?P\d{4})-(?P\d\d)-(?P\d\d)[T ](?P\d\d):(?P\d\d)(:(?P\d\d(\.\d*)?)?)?(?PZ|[+-]\d\d:?\d\d)?').setName("ISO8601 datetime")
+    "ISO8601 datetime (C{yyyy-mm-ddThh:mm:ss.s(Z|+-00:00)}) - trailing seconds, milliseconds, and timezone optional; accepts separating C{'T'} or C{' '}"
+
+    uuid = Regex(r'[0-9a-fA-F]{8}(-[0-9a-fA-F]{4}){3}-[0-9a-fA-F]{12}').setName("UUID")
+    "UUID (C{xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx})"
+
+    _html_stripper = anyOpenTag.suppress() | anyCloseTag.suppress()
+    @staticmethod
+    def stripHTMLTags(s, l, tokens):
+        """
+        Parse action to remove HTML tags from web page HTML source
+
+        Example::
+            # strip HTML links from normal text 
+            text = 'More info at the pyparsing wiki page'
+            td,td_end = makeHTMLTags("TD")
+            table_text = td + SkipTo(td_end).setParseAction(pyparsing_common.stripHTMLTags)("body") + td_end
+            
+            print(table_text.parseString(text).body) # -> 'More info at the pyparsing wiki page'
+        """
+        return pyparsing_common._html_stripper.transformString(tokens[0])
+
+    _commasepitem = Combine(OneOrMore(~Literal(",") + ~LineEnd() + Word(printables, excludeChars=',') 
+                                        + Optional( White(" \t") ) ) ).streamline().setName("commaItem")
+    comma_separated_list = delimitedList( Optional( quotedString.copy() | _commasepitem, default="") ).setName("comma separated list")
+    """Predefined expression of 1 or more printable words or quoted strings, separated by commas."""
+
+    upcaseTokens = staticmethod(tokenMap(lambda t: _ustr(t).upper()))
+    """Parse action to convert tokens to upper case."""
+
+    downcaseTokens = staticmethod(tokenMap(lambda t: _ustr(t).lower()))
+    """Parse action to convert tokens to lower case."""
+
+
+if __name__ == "__main__":
+
+    selectToken    = CaselessLiteral("select")
+    fromToken      = CaselessLiteral("from")
+
+    ident          = Word(alphas, alphanums + "_$")
+
+    columnName     = delimitedList(ident, ".", combine=True).setParseAction(upcaseTokens)
+    columnNameList = Group(delimitedList(columnName)).setName("columns")
+    columnSpec     = ('*' | columnNameList)
+
+    tableName      = delimitedList(ident, ".", combine=True).setParseAction(upcaseTokens)
+    tableNameList  = Group(delimitedList(tableName)).setName("tables")
+    
+    simpleSQL      = selectToken("command") + columnSpec("columns") + fromToken + tableNameList("tables")
+
+    # demo runTests method, including embedded comments in test string
+    simpleSQL.runTests("""
+        # '*' as column list and dotted table name
+        select * from SYS.XYZZY
+
+        # caseless match on "SELECT", and casts back to "select"
+        SELECT * from XYZZY, ABC
+
+        # list of column names, and mixed case SELECT keyword
+        Select AA,BB,CC from Sys.dual
+
+        # multiple tables
+        Select A, B, C from Sys.dual, Table2
+
+        # invalid SELECT keyword - should fail
+        Xelect A, B, C from Sys.dual
+
+        # incomplete command - should fail
+        Select
+
+        # invalid column name - should fail
+        Select ^^^ frox Sys.dual
+
+        """)
+
+    pyparsing_common.number.runTests("""
+        100
+        -100
+        +100
+        3.14159
+        6.02e23
+        1e-12
+        """)
+
+    # any int or real number, returned as float
+    pyparsing_common.fnumber.runTests("""
+        100
+        -100
+        +100
+        3.14159
+        6.02e23
+        1e-12
+        """)
+
+    pyparsing_common.hex_integer.runTests("""
+        100
+        FF
+        """)
+
+    import uuid
+    pyparsing_common.uuid.setParseAction(tokenMap(uuid.UUID))
+    pyparsing_common.uuid.runTests("""
+        12345678-1234-5678-1234-567812345678
+        """)
diff --git a/env/lib/python3.8/site-packages/pkg_resources/_vendor/six.py b/env/lib/python3.8/site-packages/pkg_resources/_vendor/six.py
new file mode 100644
index 000000000..190c0239c
--- /dev/null
+++ b/env/lib/python3.8/site-packages/pkg_resources/_vendor/six.py
@@ -0,0 +1,868 @@
+"""Utilities for writing code that runs on Python 2 and 3"""
+
+# Copyright (c) 2010-2015 Benjamin Peterson
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+
+from __future__ import absolute_import
+
+import functools
+import itertools
+import operator
+import sys
+import types
+
+__author__ = "Benjamin Peterson "
+__version__ = "1.10.0"
+
+
+# Useful for very coarse version differentiation.
+PY2 = sys.version_info[0] == 2
+PY3 = sys.version_info[0] == 3
+PY34 = sys.version_info[0:2] >= (3, 4)
+
+if PY3:
+    string_types = str,
+    integer_types = int,
+    class_types = type,
+    text_type = str
+    binary_type = bytes
+
+    MAXSIZE = sys.maxsize
+else:
+    string_types = basestring,
+    integer_types = (int, long)
+    class_types = (type, types.ClassType)
+    text_type = unicode
+    binary_type = str
+
+    if sys.platform.startswith("java"):
+        # Jython always uses 32 bits.
+        MAXSIZE = int((1 << 31) - 1)
+    else:
+        # It's possible to have sizeof(long) != sizeof(Py_ssize_t).
+        class X(object):
+
+            def __len__(self):
+                return 1 << 31
+        try:
+            len(X())
+        except OverflowError:
+            # 32-bit
+            MAXSIZE = int((1 << 31) - 1)
+        else:
+            # 64-bit
+            MAXSIZE = int((1 << 63) - 1)
+        del X
+
+
+def _add_doc(func, doc):
+    """Add documentation to a function."""
+    func.__doc__ = doc
+
+
+def _import_module(name):
+    """Import module, returning the module after the last dot."""
+    __import__(name)
+    return sys.modules[name]
+
+
+class _LazyDescr(object):
+
+    def __init__(self, name):
+        self.name = name
+
+    def __get__(self, obj, tp):
+        result = self._resolve()
+        setattr(obj, self.name, result)  # Invokes __set__.
+        try:
+            # This is a bit ugly, but it avoids running this again by
+            # removing this descriptor.
+            delattr(obj.__class__, self.name)
+        except AttributeError:
+            pass
+        return result
+
+
+class MovedModule(_LazyDescr):
+
+    def __init__(self, name, old, new=None):
+        super(MovedModule, self).__init__(name)
+        if PY3:
+            if new is None:
+                new = name
+            self.mod = new
+        else:
+            self.mod = old
+
+    def _resolve(self):
+        return _import_module(self.mod)
+
+    def __getattr__(self, attr):
+        _module = self._resolve()
+        value = getattr(_module, attr)
+        setattr(self, attr, value)
+        return value
+
+
+class _LazyModule(types.ModuleType):
+
+    def __init__(self, name):
+        super(_LazyModule, self).__init__(name)
+        self.__doc__ = self.__class__.__doc__
+
+    def __dir__(self):
+        attrs = ["__doc__", "__name__"]
+        attrs += [attr.name for attr in self._moved_attributes]
+        return attrs
+
+    # Subclasses should override this
+    _moved_attributes = []
+
+
+class MovedAttribute(_LazyDescr):
+
+    def __init__(self, name, old_mod, new_mod, old_attr=None, new_attr=None):
+        super(MovedAttribute, self).__init__(name)
+        if PY3:
+            if new_mod is None:
+                new_mod = name
+            self.mod = new_mod
+            if new_attr is None:
+                if old_attr is None:
+                    new_attr = name
+                else:
+                    new_attr = old_attr
+            self.attr = new_attr
+        else:
+            self.mod = old_mod
+            if old_attr is None:
+                old_attr = name
+            self.attr = old_attr
+
+    def _resolve(self):
+        module = _import_module(self.mod)
+        return getattr(module, self.attr)
+
+
+class _SixMetaPathImporter(object):
+
+    """
+    A meta path importer to import six.moves and its submodules.
+
+    This class implements a PEP302 finder and loader. It should be compatible
+    with Python 2.5 and all existing versions of Python3
+    """
+
+    def __init__(self, six_module_name):
+        self.name = six_module_name
+        self.known_modules = {}
+
+    def _add_module(self, mod, *fullnames):
+        for fullname in fullnames:
+            self.known_modules[self.name + "." + fullname] = mod
+
+    def _get_module(self, fullname):
+        return self.known_modules[self.name + "." + fullname]
+
+    def find_module(self, fullname, path=None):
+        if fullname in self.known_modules:
+            return self
+        return None
+
+    def __get_module(self, fullname):
+        try:
+            return self.known_modules[fullname]
+        except KeyError:
+            raise ImportError("This loader does not know module " + fullname)
+
+    def load_module(self, fullname):
+        try:
+            # in case of a reload
+            return sys.modules[fullname]
+        except KeyError:
+            pass
+        mod = self.__get_module(fullname)
+        if isinstance(mod, MovedModule):
+            mod = mod._resolve()
+        else:
+            mod.__loader__ = self
+        sys.modules[fullname] = mod
+        return mod
+
+    def is_package(self, fullname):
+        """
+        Return true, if the named module is a package.
+
+        We need this method to get correct spec objects with
+        Python 3.4 (see PEP451)
+        """
+        return hasattr(self.__get_module(fullname), "__path__")
+
+    def get_code(self, fullname):
+        """Return None
+
+        Required, if is_package is implemented"""
+        self.__get_module(fullname)  # eventually raises ImportError
+        return None
+    get_source = get_code  # same as get_code
+
+_importer = _SixMetaPathImporter(__name__)
+
+
+class _MovedItems(_LazyModule):
+
+    """Lazy loading of moved objects"""
+    __path__ = []  # mark as package
+
+
+_moved_attributes = [
+    MovedAttribute("cStringIO", "cStringIO", "io", "StringIO"),
+    MovedAttribute("filter", "itertools", "builtins", "ifilter", "filter"),
+    MovedAttribute("filterfalse", "itertools", "itertools", "ifilterfalse", "filterfalse"),
+    MovedAttribute("input", "__builtin__", "builtins", "raw_input", "input"),
+    MovedAttribute("intern", "__builtin__", "sys"),
+    MovedAttribute("map", "itertools", "builtins", "imap", "map"),
+    MovedAttribute("getcwd", "os", "os", "getcwdu", "getcwd"),
+    MovedAttribute("getcwdb", "os", "os", "getcwd", "getcwdb"),
+    MovedAttribute("range", "__builtin__", "builtins", "xrange", "range"),
+    MovedAttribute("reload_module", "__builtin__", "importlib" if PY34 else "imp", "reload"),
+    MovedAttribute("reduce", "__builtin__", "functools"),
+    MovedAttribute("shlex_quote", "pipes", "shlex", "quote"),
+    MovedAttribute("StringIO", "StringIO", "io"),
+    MovedAttribute("UserDict", "UserDict", "collections"),
+    MovedAttribute("UserList", "UserList", "collections"),
+    MovedAttribute("UserString", "UserString", "collections"),
+    MovedAttribute("xrange", "__builtin__", "builtins", "xrange", "range"),
+    MovedAttribute("zip", "itertools", "builtins", "izip", "zip"),
+    MovedAttribute("zip_longest", "itertools", "itertools", "izip_longest", "zip_longest"),
+    MovedModule("builtins", "__builtin__"),
+    MovedModule("configparser", "ConfigParser"),
+    MovedModule("copyreg", "copy_reg"),
+    MovedModule("dbm_gnu", "gdbm", "dbm.gnu"),
+    MovedModule("_dummy_thread", "dummy_thread", "_dummy_thread"),
+    MovedModule("http_cookiejar", "cookielib", "http.cookiejar"),
+    MovedModule("http_cookies", "Cookie", "http.cookies"),
+    MovedModule("html_entities", "htmlentitydefs", "html.entities"),
+    MovedModule("html_parser", "HTMLParser", "html.parser"),
+    MovedModule("http_client", "httplib", "http.client"),
+    MovedModule("email_mime_multipart", "email.MIMEMultipart", "email.mime.multipart"),
+    MovedModule("email_mime_nonmultipart", "email.MIMENonMultipart", "email.mime.nonmultipart"),
+    MovedModule("email_mime_text", "email.MIMEText", "email.mime.text"),
+    MovedModule("email_mime_base", "email.MIMEBase", "email.mime.base"),
+    MovedModule("BaseHTTPServer", "BaseHTTPServer", "http.server"),
+    MovedModule("CGIHTTPServer", "CGIHTTPServer", "http.server"),
+    MovedModule("SimpleHTTPServer", "SimpleHTTPServer", "http.server"),
+    MovedModule("cPickle", "cPickle", "pickle"),
+    MovedModule("queue", "Queue"),
+    MovedModule("reprlib", "repr"),
+    MovedModule("socketserver", "SocketServer"),
+    MovedModule("_thread", "thread", "_thread"),
+    MovedModule("tkinter", "Tkinter"),
+    MovedModule("tkinter_dialog", "Dialog", "tkinter.dialog"),
+    MovedModule("tkinter_filedialog", "FileDialog", "tkinter.filedialog"),
+    MovedModule("tkinter_scrolledtext", "ScrolledText", "tkinter.scrolledtext"),
+    MovedModule("tkinter_simpledialog", "SimpleDialog", "tkinter.simpledialog"),
+    MovedModule("tkinter_tix", "Tix", "tkinter.tix"),
+    MovedModule("tkinter_ttk", "ttk", "tkinter.ttk"),
+    MovedModule("tkinter_constants", "Tkconstants", "tkinter.constants"),
+    MovedModule("tkinter_dnd", "Tkdnd", "tkinter.dnd"),
+    MovedModule("tkinter_colorchooser", "tkColorChooser",
+                "tkinter.colorchooser"),
+    MovedModule("tkinter_commondialog", "tkCommonDialog",
+                "tkinter.commondialog"),
+    MovedModule("tkinter_tkfiledialog", "tkFileDialog", "tkinter.filedialog"),
+    MovedModule("tkinter_font", "tkFont", "tkinter.font"),
+    MovedModule("tkinter_messagebox", "tkMessageBox", "tkinter.messagebox"),
+    MovedModule("tkinter_tksimpledialog", "tkSimpleDialog",
+                "tkinter.simpledialog"),
+    MovedModule("urllib_parse", __name__ + ".moves.urllib_parse", "urllib.parse"),
+    MovedModule("urllib_error", __name__ + ".moves.urllib_error", "urllib.error"),
+    MovedModule("urllib", __name__ + ".moves.urllib", __name__ + ".moves.urllib"),
+    MovedModule("urllib_robotparser", "robotparser", "urllib.robotparser"),
+    MovedModule("xmlrpc_client", "xmlrpclib", "xmlrpc.client"),
+    MovedModule("xmlrpc_server", "SimpleXMLRPCServer", "xmlrpc.server"),
+]
+# Add windows specific modules.
+if sys.platform == "win32":
+    _moved_attributes += [
+        MovedModule("winreg", "_winreg"),
+    ]
+
+for attr in _moved_attributes:
+    setattr(_MovedItems, attr.name, attr)
+    if isinstance(attr, MovedModule):
+        _importer._add_module(attr, "moves." + attr.name)
+del attr
+
+_MovedItems._moved_attributes = _moved_attributes
+
+moves = _MovedItems(__name__ + ".moves")
+_importer._add_module(moves, "moves")
+
+
+class Module_six_moves_urllib_parse(_LazyModule):
+
+    """Lazy loading of moved objects in six.moves.urllib_parse"""
+
+
+_urllib_parse_moved_attributes = [
+    MovedAttribute("ParseResult", "urlparse", "urllib.parse"),
+    MovedAttribute("SplitResult", "urlparse", "urllib.parse"),
+    MovedAttribute("parse_qs", "urlparse", "urllib.parse"),
+    MovedAttribute("parse_qsl", "urlparse", "urllib.parse"),
+    MovedAttribute("urldefrag", "urlparse", "urllib.parse"),
+    MovedAttribute("urljoin", "urlparse", "urllib.parse"),
+    MovedAttribute("urlparse", "urlparse", "urllib.parse"),
+    MovedAttribute("urlsplit", "urlparse", "urllib.parse"),
+    MovedAttribute("urlunparse", "urlparse", "urllib.parse"),
+    MovedAttribute("urlunsplit", "urlparse", "urllib.parse"),
+    MovedAttribute("quote", "urllib", "urllib.parse"),
+    MovedAttribute("quote_plus", "urllib", "urllib.parse"),
+    MovedAttribute("unquote", "urllib", "urllib.parse"),
+    MovedAttribute("unquote_plus", "urllib", "urllib.parse"),
+    MovedAttribute("urlencode", "urllib", "urllib.parse"),
+    MovedAttribute("splitquery", "urllib", "urllib.parse"),
+    MovedAttribute("splittag", "urllib", "urllib.parse"),
+    MovedAttribute("splituser", "urllib", "urllib.parse"),
+    MovedAttribute("uses_fragment", "urlparse", "urllib.parse"),
+    MovedAttribute("uses_netloc", "urlparse", "urllib.parse"),
+    MovedAttribute("uses_params", "urlparse", "urllib.parse"),
+    MovedAttribute("uses_query", "urlparse", "urllib.parse"),
+    MovedAttribute("uses_relative", "urlparse", "urllib.parse"),
+]
+for attr in _urllib_parse_moved_attributes:
+    setattr(Module_six_moves_urllib_parse, attr.name, attr)
+del attr
+
+Module_six_moves_urllib_parse._moved_attributes = _urllib_parse_moved_attributes
+
+_importer._add_module(Module_six_moves_urllib_parse(__name__ + ".moves.urllib_parse"),
+                      "moves.urllib_parse", "moves.urllib.parse")
+
+
+class Module_six_moves_urllib_error(_LazyModule):
+
+    """Lazy loading of moved objects in six.moves.urllib_error"""
+
+
+_urllib_error_moved_attributes = [
+    MovedAttribute("URLError", "urllib2", "urllib.error"),
+    MovedAttribute("HTTPError", "urllib2", "urllib.error"),
+    MovedAttribute("ContentTooShortError", "urllib", "urllib.error"),
+]
+for attr in _urllib_error_moved_attributes:
+    setattr(Module_six_moves_urllib_error, attr.name, attr)
+del attr
+
+Module_six_moves_urllib_error._moved_attributes = _urllib_error_moved_attributes
+
+_importer._add_module(Module_six_moves_urllib_error(__name__ + ".moves.urllib.error"),
+                      "moves.urllib_error", "moves.urllib.error")
+
+
+class Module_six_moves_urllib_request(_LazyModule):
+
+    """Lazy loading of moved objects in six.moves.urllib_request"""
+
+
+_urllib_request_moved_attributes = [
+    MovedAttribute("urlopen", "urllib2", "urllib.request"),
+    MovedAttribute("install_opener", "urllib2", "urllib.request"),
+    MovedAttribute("build_opener", "urllib2", "urllib.request"),
+    MovedAttribute("pathname2url", "urllib", "urllib.request"),
+    MovedAttribute("url2pathname", "urllib", "urllib.request"),
+    MovedAttribute("getproxies", "urllib", "urllib.request"),
+    MovedAttribute("Request", "urllib2", "urllib.request"),
+    MovedAttribute("OpenerDirector", "urllib2", "urllib.request"),
+    MovedAttribute("HTTPDefaultErrorHandler", "urllib2", "urllib.request"),
+    MovedAttribute("HTTPRedirectHandler", "urllib2", "urllib.request"),
+    MovedAttribute("HTTPCookieProcessor", "urllib2", "urllib.request"),
+    MovedAttribute("ProxyHandler", "urllib2", "urllib.request"),
+    MovedAttribute("BaseHandler", "urllib2", "urllib.request"),
+    MovedAttribute("HTTPPasswordMgr", "urllib2", "urllib.request"),
+    MovedAttribute("HTTPPasswordMgrWithDefaultRealm", "urllib2", "urllib.request"),
+    MovedAttribute("AbstractBasicAuthHandler", "urllib2", "urllib.request"),
+    MovedAttribute("HTTPBasicAuthHandler", "urllib2", "urllib.request"),
+    MovedAttribute("ProxyBasicAuthHandler", "urllib2", "urllib.request"),
+    MovedAttribute("AbstractDigestAuthHandler", "urllib2", "urllib.request"),
+    MovedAttribute("HTTPDigestAuthHandler", "urllib2", "urllib.request"),
+    MovedAttribute("ProxyDigestAuthHandler", "urllib2", "urllib.request"),
+    MovedAttribute("HTTPHandler", "urllib2", "urllib.request"),
+    MovedAttribute("HTTPSHandler", "urllib2", "urllib.request"),
+    MovedAttribute("FileHandler", "urllib2", "urllib.request"),
+    MovedAttribute("FTPHandler", "urllib2", "urllib.request"),
+    MovedAttribute("CacheFTPHandler", "urllib2", "urllib.request"),
+    MovedAttribute("UnknownHandler", "urllib2", "urllib.request"),
+    MovedAttribute("HTTPErrorProcessor", "urllib2", "urllib.request"),
+    MovedAttribute("urlretrieve", "urllib", "urllib.request"),
+    MovedAttribute("urlcleanup", "urllib", "urllib.request"),
+    MovedAttribute("URLopener", "urllib", "urllib.request"),
+    MovedAttribute("FancyURLopener", "urllib", "urllib.request"),
+    MovedAttribute("proxy_bypass", "urllib", "urllib.request"),
+]
+for attr in _urllib_request_moved_attributes:
+    setattr(Module_six_moves_urllib_request, attr.name, attr)
+del attr
+
+Module_six_moves_urllib_request._moved_attributes = _urllib_request_moved_attributes
+
+_importer._add_module(Module_six_moves_urllib_request(__name__ + ".moves.urllib.request"),
+                      "moves.urllib_request", "moves.urllib.request")
+
+
+class Module_six_moves_urllib_response(_LazyModule):
+
+    """Lazy loading of moved objects in six.moves.urllib_response"""
+
+
+_urllib_response_moved_attributes = [
+    MovedAttribute("addbase", "urllib", "urllib.response"),
+    MovedAttribute("addclosehook", "urllib", "urllib.response"),
+    MovedAttribute("addinfo", "urllib", "urllib.response"),
+    MovedAttribute("addinfourl", "urllib", "urllib.response"),
+]
+for attr in _urllib_response_moved_attributes:
+    setattr(Module_six_moves_urllib_response, attr.name, attr)
+del attr
+
+Module_six_moves_urllib_response._moved_attributes = _urllib_response_moved_attributes
+
+_importer._add_module(Module_six_moves_urllib_response(__name__ + ".moves.urllib.response"),
+                      "moves.urllib_response", "moves.urllib.response")
+
+
+class Module_six_moves_urllib_robotparser(_LazyModule):
+
+    """Lazy loading of moved objects in six.moves.urllib_robotparser"""
+
+
+_urllib_robotparser_moved_attributes = [
+    MovedAttribute("RobotFileParser", "robotparser", "urllib.robotparser"),
+]
+for attr in _urllib_robotparser_moved_attributes:
+    setattr(Module_six_moves_urllib_robotparser, attr.name, attr)
+del attr
+
+Module_six_moves_urllib_robotparser._moved_attributes = _urllib_robotparser_moved_attributes
+
+_importer._add_module(Module_six_moves_urllib_robotparser(__name__ + ".moves.urllib.robotparser"),
+                      "moves.urllib_robotparser", "moves.urllib.robotparser")
+
+
+class Module_six_moves_urllib(types.ModuleType):
+
+    """Create a six.moves.urllib namespace that resembles the Python 3 namespace"""
+    __path__ = []  # mark as package
+    parse = _importer._get_module("moves.urllib_parse")
+    error = _importer._get_module("moves.urllib_error")
+    request = _importer._get_module("moves.urllib_request")
+    response = _importer._get_module("moves.urllib_response")
+    robotparser = _importer._get_module("moves.urllib_robotparser")
+
+    def __dir__(self):
+        return ['parse', 'error', 'request', 'response', 'robotparser']
+
+_importer._add_module(Module_six_moves_urllib(__name__ + ".moves.urllib"),
+                      "moves.urllib")
+
+
+def add_move(move):
+    """Add an item to six.moves."""
+    setattr(_MovedItems, move.name, move)
+
+
+def remove_move(name):
+    """Remove item from six.moves."""
+    try:
+        delattr(_MovedItems, name)
+    except AttributeError:
+        try:
+            del moves.__dict__[name]
+        except KeyError:
+            raise AttributeError("no such move, %r" % (name,))
+
+
+if PY3:
+    _meth_func = "__func__"
+    _meth_self = "__self__"
+
+    _func_closure = "__closure__"
+    _func_code = "__code__"
+    _func_defaults = "__defaults__"
+    _func_globals = "__globals__"
+else:
+    _meth_func = "im_func"
+    _meth_self = "im_self"
+
+    _func_closure = "func_closure"
+    _func_code = "func_code"
+    _func_defaults = "func_defaults"
+    _func_globals = "func_globals"
+
+
+try:
+    advance_iterator = next
+except NameError:
+    def advance_iterator(it):
+        return it.next()
+next = advance_iterator
+
+
+try:
+    callable = callable
+except NameError:
+    def callable(obj):
+        return any("__call__" in klass.__dict__ for klass in type(obj).__mro__)
+
+
+if PY3:
+    def get_unbound_function(unbound):
+        return unbound
+
+    create_bound_method = types.MethodType
+
+    def create_unbound_method(func, cls):
+        return func
+
+    Iterator = object
+else:
+    def get_unbound_function(unbound):
+        return unbound.im_func
+
+    def create_bound_method(func, obj):
+        return types.MethodType(func, obj, obj.__class__)
+
+    def create_unbound_method(func, cls):
+        return types.MethodType(func, None, cls)
+
+    class Iterator(object):
+
+        def next(self):
+            return type(self).__next__(self)
+
+    callable = callable
+_add_doc(get_unbound_function,
+         """Get the function out of a possibly unbound function""")
+
+
+get_method_function = operator.attrgetter(_meth_func)
+get_method_self = operator.attrgetter(_meth_self)
+get_function_closure = operator.attrgetter(_func_closure)
+get_function_code = operator.attrgetter(_func_code)
+get_function_defaults = operator.attrgetter(_func_defaults)
+get_function_globals = operator.attrgetter(_func_globals)
+
+
+if PY3:
+    def iterkeys(d, **kw):
+        return iter(d.keys(**kw))
+
+    def itervalues(d, **kw):
+        return iter(d.values(**kw))
+
+    def iteritems(d, **kw):
+        return iter(d.items(**kw))
+
+    def iterlists(d, **kw):
+        return iter(d.lists(**kw))
+
+    viewkeys = operator.methodcaller("keys")
+
+    viewvalues = operator.methodcaller("values")
+
+    viewitems = operator.methodcaller("items")
+else:
+    def iterkeys(d, **kw):
+        return d.iterkeys(**kw)
+
+    def itervalues(d, **kw):
+        return d.itervalues(**kw)
+
+    def iteritems(d, **kw):
+        return d.iteritems(**kw)
+
+    def iterlists(d, **kw):
+        return d.iterlists(**kw)
+
+    viewkeys = operator.methodcaller("viewkeys")
+
+    viewvalues = operator.methodcaller("viewvalues")
+
+    viewitems = operator.methodcaller("viewitems")
+
+_add_doc(iterkeys, "Return an iterator over the keys of a dictionary.")
+_add_doc(itervalues, "Return an iterator over the values of a dictionary.")
+_add_doc(iteritems,
+         "Return an iterator over the (key, value) pairs of a dictionary.")
+_add_doc(iterlists,
+         "Return an iterator over the (key, [values]) pairs of a dictionary.")
+
+
+if PY3:
+    def b(s):
+        return s.encode("latin-1")
+
+    def u(s):
+        return s
+    unichr = chr
+    import struct
+    int2byte = struct.Struct(">B").pack
+    del struct
+    byte2int = operator.itemgetter(0)
+    indexbytes = operator.getitem
+    iterbytes = iter
+    import io
+    StringIO = io.StringIO
+    BytesIO = io.BytesIO
+    _assertCountEqual = "assertCountEqual"
+    if sys.version_info[1] <= 1:
+        _assertRaisesRegex = "assertRaisesRegexp"
+        _assertRegex = "assertRegexpMatches"
+    else:
+        _assertRaisesRegex = "assertRaisesRegex"
+        _assertRegex = "assertRegex"
+else:
+    def b(s):
+        return s
+    # Workaround for standalone backslash
+
+    def u(s):
+        return unicode(s.replace(r'\\', r'\\\\'), "unicode_escape")
+    unichr = unichr
+    int2byte = chr
+
+    def byte2int(bs):
+        return ord(bs[0])
+
+    def indexbytes(buf, i):
+        return ord(buf[i])
+    iterbytes = functools.partial(itertools.imap, ord)
+    import StringIO
+    StringIO = BytesIO = StringIO.StringIO
+    _assertCountEqual = "assertItemsEqual"
+    _assertRaisesRegex = "assertRaisesRegexp"
+    _assertRegex = "assertRegexpMatches"
+_add_doc(b, """Byte literal""")
+_add_doc(u, """Text literal""")
+
+
+def assertCountEqual(self, *args, **kwargs):
+    return getattr(self, _assertCountEqual)(*args, **kwargs)
+
+
+def assertRaisesRegex(self, *args, **kwargs):
+    return getattr(self, _assertRaisesRegex)(*args, **kwargs)
+
+
+def assertRegex(self, *args, **kwargs):
+    return getattr(self, _assertRegex)(*args, **kwargs)
+
+
+if PY3:
+    exec_ = getattr(moves.builtins, "exec")
+
+    def reraise(tp, value, tb=None):
+        if value is None:
+            value = tp()
+        if value.__traceback__ is not tb:
+            raise value.with_traceback(tb)
+        raise value
+
+else:
+    def exec_(_code_, _globs_=None, _locs_=None):
+        """Execute code in a namespace."""
+        if _globs_ is None:
+            frame = sys._getframe(1)
+            _globs_ = frame.f_globals
+            if _locs_ is None:
+                _locs_ = frame.f_locals
+            del frame
+        elif _locs_ is None:
+            _locs_ = _globs_
+        exec("""exec _code_ in _globs_, _locs_""")
+
+    exec_("""def reraise(tp, value, tb=None):
+    raise tp, value, tb
+""")
+
+
+if sys.version_info[:2] == (3, 2):
+    exec_("""def raise_from(value, from_value):
+    if from_value is None:
+        raise value
+    raise value from from_value
+""")
+elif sys.version_info[:2] > (3, 2):
+    exec_("""def raise_from(value, from_value):
+    raise value from from_value
+""")
+else:
+    def raise_from(value, from_value):
+        raise value
+
+
+print_ = getattr(moves.builtins, "print", None)
+if print_ is None:
+    def print_(*args, **kwargs):
+        """The new-style print function for Python 2.4 and 2.5."""
+        fp = kwargs.pop("file", sys.stdout)
+        if fp is None:
+            return
+
+        def write(data):
+            if not isinstance(data, basestring):
+                data = str(data)
+            # If the file has an encoding, encode unicode with it.
+            if (isinstance(fp, file) and
+                    isinstance(data, unicode) and
+                    fp.encoding is not None):
+                errors = getattr(fp, "errors", None)
+                if errors is None:
+                    errors = "strict"
+                data = data.encode(fp.encoding, errors)
+            fp.write(data)
+        want_unicode = False
+        sep = kwargs.pop("sep", None)
+        if sep is not None:
+            if isinstance(sep, unicode):
+                want_unicode = True
+            elif not isinstance(sep, str):
+                raise TypeError("sep must be None or a string")
+        end = kwargs.pop("end", None)
+        if end is not None:
+            if isinstance(end, unicode):
+                want_unicode = True
+            elif not isinstance(end, str):
+                raise TypeError("end must be None or a string")
+        if kwargs:
+            raise TypeError("invalid keyword arguments to print()")
+        if not want_unicode:
+            for arg in args:
+                if isinstance(arg, unicode):
+                    want_unicode = True
+                    break
+        if want_unicode:
+            newline = unicode("\n")
+            space = unicode(" ")
+        else:
+            newline = "\n"
+            space = " "
+        if sep is None:
+            sep = space
+        if end is None:
+            end = newline
+        for i, arg in enumerate(args):
+            if i:
+                write(sep)
+            write(arg)
+        write(end)
+if sys.version_info[:2] < (3, 3):
+    _print = print_
+
+    def print_(*args, **kwargs):
+        fp = kwargs.get("file", sys.stdout)
+        flush = kwargs.pop("flush", False)
+        _print(*args, **kwargs)
+        if flush and fp is not None:
+            fp.flush()
+
+_add_doc(reraise, """Reraise an exception.""")
+
+if sys.version_info[0:2] < (3, 4):
+    def wraps(wrapped, assigned=functools.WRAPPER_ASSIGNMENTS,
+              updated=functools.WRAPPER_UPDATES):
+        def wrapper(f):
+            f = functools.wraps(wrapped, assigned, updated)(f)
+            f.__wrapped__ = wrapped
+            return f
+        return wrapper
+else:
+    wraps = functools.wraps
+
+
+def with_metaclass(meta, *bases):
+    """Create a base class with a metaclass."""
+    # This requires a bit of explanation: the basic idea is to make a dummy
+    # metaclass for one level of class instantiation that replaces itself with
+    # the actual metaclass.
+    class metaclass(meta):
+
+        def __new__(cls, name, this_bases, d):
+            return meta(name, bases, d)
+    return type.__new__(metaclass, 'temporary_class', (), {})
+
+
+def add_metaclass(metaclass):
+    """Class decorator for creating a class with a metaclass."""
+    def wrapper(cls):
+        orig_vars = cls.__dict__.copy()
+        slots = orig_vars.get('__slots__')
+        if slots is not None:
+            if isinstance(slots, str):
+                slots = [slots]
+            for slots_var in slots:
+                orig_vars.pop(slots_var)
+        orig_vars.pop('__dict__', None)
+        orig_vars.pop('__weakref__', None)
+        return metaclass(cls.__name__, cls.__bases__, orig_vars)
+    return wrapper
+
+
+def python_2_unicode_compatible(klass):
+    """
+    A decorator that defines __unicode__ and __str__ methods under Python 2.
+    Under Python 3 it does nothing.
+
+    To support Python 2 and 3 with a single code base, define a __str__ method
+    returning text and apply this decorator to the class.
+    """
+    if PY2:
+        if '__str__' not in klass.__dict__:
+            raise ValueError("@python_2_unicode_compatible cannot be applied "
+                             "to %s because it doesn't define __str__()." %
+                             klass.__name__)
+        klass.__unicode__ = klass.__str__
+        klass.__str__ = lambda self: self.__unicode__().encode('utf-8')
+    return klass
+
+
+# Complete the moves implementation.
+# This code is at the end of this module to speed up module loading.
+# Turn this module into a package.
+__path__ = []  # required for PEP 302 and PEP 451
+__package__ = __name__  # see PEP 366 @ReservedAssignment
+if globals().get("__spec__") is not None:
+    __spec__.submodule_search_locations = []  # PEP 451 @UndefinedVariable
+# Remove other six meta path importers, since they cause problems. This can
+# happen if six is removed from sys.modules and then reloaded. (Setuptools does
+# this for some reason.)
+if sys.meta_path:
+    for i, importer in enumerate(sys.meta_path):
+        # Here's some real nastiness: Another "instance" of the six module might
+        # be floating around. Therefore, we can't use isinstance() to check for
+        # the six meta path importer, since the other six instance will have
+        # inserted an importer with different class.
+        if (type(importer).__name__ == "_SixMetaPathImporter" and
+                importer.name == __name__):
+            del sys.meta_path[i]
+            break
+    del i, importer
+# Finally, add the importer to the meta path import hook.
+sys.meta_path.append(_importer)
diff --git a/env/lib/python3.8/site-packages/pkg_resources/extern/__init__.py b/env/lib/python3.8/site-packages/pkg_resources/extern/__init__.py
new file mode 100644
index 000000000..c1eb9e998
--- /dev/null
+++ b/env/lib/python3.8/site-packages/pkg_resources/extern/__init__.py
@@ -0,0 +1,73 @@
+import sys
+
+
+class VendorImporter:
+    """
+    A PEP 302 meta path importer for finding optionally-vendored
+    or otherwise naturally-installed packages from root_name.
+    """
+
+    def __init__(self, root_name, vendored_names=(), vendor_pkg=None):
+        self.root_name = root_name
+        self.vendored_names = set(vendored_names)
+        self.vendor_pkg = vendor_pkg or root_name.replace('extern', '_vendor')
+
+    @property
+    def search_path(self):
+        """
+        Search first the vendor package then as a natural package.
+        """
+        yield self.vendor_pkg + '.'
+        yield ''
+
+    def find_module(self, fullname, path=None):
+        """
+        Return self when fullname starts with root_name and the
+        target module is one vendored through this importer.
+        """
+        root, base, target = fullname.partition(self.root_name + '.')
+        if root:
+            return
+        if not any(map(target.startswith, self.vendored_names)):
+            return
+        return self
+
+    def load_module(self, fullname):
+        """
+        Iterate over the search path to locate and load fullname.
+        """
+        root, base, target = fullname.partition(self.root_name + '.')
+        for prefix in self.search_path:
+            try:
+                extant = prefix + target
+                __import__(extant)
+                mod = sys.modules[extant]
+                sys.modules[fullname] = mod
+                # mysterious hack:
+                # Remove the reference to the extant package/module
+                # on later Python versions to cause relative imports
+                # in the vendor package to resolve the same modules
+                # as those going through this importer.
+                if prefix and sys.version_info > (3, 3):
+                    del sys.modules[extant]
+                return mod
+            except ImportError:
+                pass
+        else:
+            raise ImportError(
+                "The '{target}' package is required; "
+                "normally this is bundled with this package so if you get "
+                "this warning, consult the packager of your "
+                "distribution.".format(**locals())
+            )
+
+    def install(self):
+        """
+        Install this importer into sys.meta_path if not already present.
+        """
+        if self not in sys.meta_path:
+            sys.meta_path.append(self)
+
+
+names = 'packaging', 'pyparsing', 'six', 'appdirs'
+VendorImporter(__name__, names).install()
diff --git a/env/lib/python3.8/site-packages/pkg_resources/py31compat.py b/env/lib/python3.8/site-packages/pkg_resources/py31compat.py
new file mode 100644
index 000000000..a381c424f
--- /dev/null
+++ b/env/lib/python3.8/site-packages/pkg_resources/py31compat.py
@@ -0,0 +1,23 @@
+import os
+import errno
+import sys
+
+from .extern import six
+
+
+def _makedirs_31(path, exist_ok=False):
+    try:
+        os.makedirs(path)
+    except OSError as exc:
+        if not exist_ok or exc.errno != errno.EEXIST:
+            raise
+
+
+# rely on compatibility behavior until mode considerations
+#  and exists_ok considerations are disentangled.
+# See https://github.com/pypa/setuptools/pull/1083#issuecomment-315168663
+needs_makedirs = (
+    six.PY2 or
+    (3, 4) <= sys.version_info < (3, 4, 1)
+)
+makedirs = _makedirs_31 if needs_makedirs else os.makedirs
diff --git a/env/lib/python3.8/site-packages/pycodestyle-2.6.0.dist-info/INSTALLER b/env/lib/python3.8/site-packages/pycodestyle-2.6.0.dist-info/INSTALLER
new file mode 100644
index 000000000..a1b589e38
--- /dev/null
+++ b/env/lib/python3.8/site-packages/pycodestyle-2.6.0.dist-info/INSTALLER
@@ -0,0 +1 @@
+pip
diff --git a/env/lib/python3.8/site-packages/pycodestyle-2.6.0.dist-info/LICENSE b/env/lib/python3.8/site-packages/pycodestyle-2.6.0.dist-info/LICENSE
new file mode 100644
index 000000000..72d9921cc
--- /dev/null
+++ b/env/lib/python3.8/site-packages/pycodestyle-2.6.0.dist-info/LICENSE
@@ -0,0 +1,25 @@
+Copyright © 2006-2009 Johann C. Rocholl 
+Copyright © 2009-2014 Florent Xicluna 
+Copyright © 2014-2020 Ian Lee 
+
+Licensed under the terms of the Expat License
+
+Permission is hereby granted, free of charge, to any person
+obtaining a copy of this software and associated documentation files
+(the "Software"), to deal in the Software without restriction,
+including without limitation the rights to use, copy, modify, merge,
+publish, distribute, sublicense, and/or sell copies of the Software,
+and to permit persons to whom the Software is furnished to do so,
+subject to the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
diff --git a/env/lib/python3.8/site-packages/pycodestyle-2.6.0.dist-info/METADATA b/env/lib/python3.8/site-packages/pycodestyle-2.6.0.dist-info/METADATA
new file mode 100644
index 000000000..7287b4e5b
--- /dev/null
+++ b/env/lib/python3.8/site-packages/pycodestyle-2.6.0.dist-info/METADATA
@@ -0,0 +1,1035 @@
+Metadata-Version: 2.1
+Name: pycodestyle
+Version: 2.6.0
+Summary: Python style guide checker
+Home-page: https://pycodestyle.readthedocs.io/
+Author: Johann C. Rocholl
+Author-email: johann@rocholl.net
+Maintainer: Ian Lee
+Maintainer-email: IanLee1521@gmail.com
+License: Expat license
+Keywords: pycodestyle,pep8,PEP 8,PEP-8,PEP8
+Platform: UNKNOWN
+Classifier: Development Status :: 5 - Production/Stable
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python
+Classifier: Programming Language :: Python :: 2
+Classifier: Programming Language :: Python :: 2.7
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.4
+Classifier: Programming Language :: Python :: 3.5
+Classifier: Programming Language :: Python :: 3.6
+Classifier: Programming Language :: Python :: 3.7
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: Implementation :: CPython
+Classifier: Programming Language :: Python :: Implementation :: PyPy
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Requires-Python: >=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*
+
+pycodestyle (formerly called pep8) - Python style guide checker
+===============================================================
+
+.. image:: https://img.shields.io/travis/PyCQA/pycodestyle.svg
+   :target: https://travis-ci.org/PyCQA/pycodestyle
+   :alt: Build status
+
+.. image:: https://readthedocs.org/projects/pycodestyle/badge/?version=latest
+    :target: https://pycodestyle.readthedocs.io
+    :alt: Documentation Status
+
+.. image:: https://img.shields.io/pypi/wheel/pycodestyle.svg
+   :target: https://pypi.org/project/pycodestyle/
+   :alt: Wheel Status
+
+.. image:: https://badges.gitter.im/PyCQA/pycodestyle.svg
+   :alt: Join the chat at https://gitter.im/PyCQA/pycodestyle
+   :target: https://gitter.im/PyCQA/pycodestyle?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge
+
+pycodestyle is a tool to check your Python code against some of the style
+conventions in `PEP 8`_.
+
+.. _PEP 8: http://www.python.org/dev/peps/pep-0008/
+
+.. note::
+
+    This package used to be called ``pep8`` but was renamed to ``pycodestyle``
+    to reduce confusion. Further discussion can be found `in the issue where
+    Guido requested this
+    change `_, or in the
+    lightning talk at PyCon 2016 by @IanLee1521:
+    `slides `_
+    `video `_.
+
+Features
+--------
+
+* Plugin architecture: Adding new checks is easy.
+
+* Parseable output: Jump to error location in your editor.
+
+* Small: Just one Python file, requires only stdlib. You can use just
+  the ``pycodestyle.py`` file for this purpose.
+
+* Comes with a comprehensive test suite.
+
+Installation
+------------
+
+You can install, upgrade, and uninstall ``pycodestyle.py`` with these commands::
+
+  $ pip install pycodestyle
+  $ pip install --upgrade pycodestyle
+  $ pip uninstall pycodestyle
+
+There's also a package for Debian/Ubuntu, but it's not always the
+latest version.
+
+Example usage and output
+------------------------
+
+::
+
+  $ pycodestyle --first optparse.py
+  optparse.py:69:11: E401 multiple imports on one line
+  optparse.py:77:1: E302 expected 2 blank lines, found 1
+  optparse.py:88:5: E301 expected 1 blank line, found 0
+  optparse.py:222:34: W602 deprecated form of raising exception
+  optparse.py:347:31: E211 whitespace before '('
+  optparse.py:357:17: E201 whitespace after '{'
+  optparse.py:472:29: E221 multiple spaces before operator
+  optparse.py:544:21: W601 .has_key() is deprecated, use 'in'
+
+You can also make ``pycodestyle.py`` show the source code for each error, and
+even the relevant text from PEP 8::
+
+  $ pycodestyle --show-source --show-pep8 testsuite/E40.py
+  testsuite/E40.py:2:10: E401 multiple imports on one line
+  import os, sys
+           ^
+      Imports should usually be on separate lines.
+
+      Okay: import os\nimport sys
+      E401: import sys, os
+
+
+Or you can display how often each error was found::
+
+  $ pycodestyle --statistics -qq Python-2.5/Lib
+  232     E201 whitespace after '['
+  599     E202 whitespace before ')'
+  631     E203 whitespace before ','
+  842     E211 whitespace before '('
+  2531    E221 multiple spaces before operator
+  4473    E301 expected 1 blank line, found 0
+  4006    E302 expected 2 blank lines, found 1
+  165     E303 too many blank lines (4)
+  325     E401 multiple imports on one line
+  3615    E501 line too long (82 characters)
+  612     W601 .has_key() is deprecated, use 'in'
+  1188    W602 deprecated form of raising exception
+
+Links
+-----
+
+* `Read the documentation `_
+
+* `Fork me on GitHub `_
+
+
+Changelog
+=========
+
+2.6.0 (2020-05-11)
+------------------
+
+Announcements:
+
+* Anthony Sottile (@asottile) joined the team as a core developer. :tada:
+
+Changes:
+
+* E306: fix detection inside ``async def``.  PR #929.
+* E301: fix regression disallowing decorated one-liners.  PR #927.
+* E714: fix false positive with chained ``is not``.  PR #931.
+
+
+2.6.0a1 (2020-04-23)
+--------------------
+
+New checks:
+
+* E225: require whitespace around ``and`` ``in`` ``is`` and ``or``.  PR #847.
+
+Changes:
+
+* E117: fix indentation using tabs by treating as 8-space indents.  PR #837.
+* E721: fix false positive with names containg ``istype``.  PR #850.
+* E741: allow ``l`` as a named argument in a function call.  PR #853.
+* E302: fix false-negative with decorated functions.  PR #859.
+* W504: ellipsis (``...``) is no longer treated as a binary operator.  PR #875.
+* E402: allow ``with``, ``if``, ``elif``, ``else`` to guard imports.  PR #834.
+* Add support for assignment expressions ``:=`` (PEP 572).  PR #879.
+* Add support for positional-only arguments ``/`` (PEP 570).  PR #872, #918.
+* Add support for python 3.8.
+* Add support for matrix multiplication operator ``@`` (PEP 465).  PR #897.
+* Support visual indent for continuation lines for ``with`` / ``assert`` /
+  ``raise``.  PR #912.
+* E302: allow two blank lines after a block of one-liners.  PR #913.
+* E302: allow two-and-fewer newlines at the top of the file.  PR #919.
+
+
+2.5.0 (2019-01-29)
+------------------
+
+New checks:
+
+* E117: Over-indented code blocks
+* W505: Maximum doc-string length only when configured with --max-doc-length
+
+Changes:
+
+* Remove support for EOL Python 2.6 and 3.3. PR #720.
+* Add E117 error for over-indented code blocks.
+* Allow W605 to be silenced by `# noqa` and fix the position reported by W605
+* Allow users to omit blank lines around one-liner definitions of classes and
+  functions
+* Include the function return annotation (``->``) as requiring surrounding
+  whitespace only on Python 3
+* Verify that only names can follow ``await``. Previously we allowed numbers
+  and strings.
+* Add support for Python 3.7
+* Fix detection of annotated argument defaults for E252
+* Correct the position reported by W504
+
+
+2.4.0 (2018-04-10)
+------------------
+
+New checks:
+
+* Add W504 warning for checking that a break doesn't happen after a binary
+  operator. This check is ignored by default. PR #502.
+* Add W605 warning for invalid escape sequences in string literals. PR #676.
+* Add W606 warning for 'async' and 'await' reserved keywords being introduced
+  in Python 3.7. PR #684.
+* Add E252 error for missing whitespace around equal sign in type annotated
+  function arguments with defaults values. PR #717.
+
+Changes:
+
+* An internal bisect search has replaced a linear search in order to improve
+  efficiency. PR #648.
+* pycodestyle now uses PyPI trove classifiers in order to document supported
+  python versions on PyPI. PR #654.
+* 'setup.cfg' '[wheel]' section has been renamed to '[bdist_wheel]', as
+  the former is legacy. PR #653.
+* pycodestyle now handles very long lines much more efficiently for python
+  3.2+. Fixes #643. PR #644.
+* You can now write 'pycodestyle.StyleGuide(verbose=True)' instead of
+  'pycodestyle.StyleGuide(verbose=True, paths=['-v'])' in order to achieve
+  verbosity. PR #663.
+* The distribution of pycodestyle now includes the license text in order to
+  comply with open source licenses which require this. PR #694.
+* 'maximum_line_length' now ignores shebang ('#!') lines. PR #736.
+* Add configuration option for the allowed number of blank lines. It is
+  implemented as a top level dictionary which can be easily overwritten. Fixes
+  #732. PR #733.
+
+Bugs:
+
+* Prevent a 'DeprecationWarning', and a 'SyntaxError' in future python, caused
+  by an invalid escape sequence. PR #625.
+* Correctly report E501 when the first line of a docstring is too long.
+  Resolves #622. PR #630.
+* Support variable annotation when variable start by a keyword, such as class
+  variable type annotations in python 3.6. PR #640.
+* pycodestyle internals have been changed in order to allow 'python3 -m
+  cProfile' to report correct metrics. PR #647.
+* Fix a spelling mistake in the description of E722. PR #697.
+* 'pycodestyle --diff' now does not break if your 'gitconfig' enables
+  'mnemonicprefix'. PR #706.
+
+2.3.1 (2017-01-31)
+------------------
+
+Bugs:
+
+* Fix regression in detection of E302 and E306; #618, #620
+
+2.3.0 (2017-01-30)
+------------------
+
+New Checks:
+
+* Add E722 warning for bare ``except`` clauses
+* Report E704 for async function definitions (``async def``)
+
+Bugs:
+
+* Fix another E305 false positive for variables beginning with "class" or
+  "def"
+* Fix detection of multiple spaces between ``async`` and ``def``
+* Fix handling of variable annotations. Stop reporting E701 on Python 3.6 for
+  variable annotations.
+
+2.2.0 (2016-11-14)
+------------------
+
+Announcements:
+
+* Added Make target to obtain proper tarball file permissions; #599
+
+Bugs:
+
+* Fixed E305 regression caused by #400; #593
+
+2.1.0 (2016-11-04)
+------------------
+
+Announcements:
+
+* Change all references to the pep8 project to say pycodestyle; #530
+
+Changes:
+
+* Report E302 for blank lines before an "async def"; #556
+* Update our list of tested and supported Python versions which are 2.6, 2.7,
+  3.2, 3.3, 3.4 and 3.5 as well as the nightly Python build and PyPy.
+* Report E742 and E743 for functions and classes badly named 'l', 'O', or 'I'.
+* Report E741 on 'global' and 'nonlocal' statements, as well as prohibited
+  single-letter variables.
+* Deprecated use of `[pep8]` section name in favor of `[pycodestyle]`; #591
+* Report E722 when bare except clause is used; #579
+
+Bugs:
+
+* Fix opt_type AssertionError when using Flake8 2.6.2 and pycodestyle; #561
+* Require two blank lines after toplevel def, class; #536
+* Remove accidentally quadratic computation based on the number of colons. This
+  will make pycodestyle faster in some cases; #314
+
+2.0.0 (2016-05-31)
+------------------
+
+Announcements:
+
+* Repository renamed to `pycodestyle`; Issue #466 / #481.
+* Added joint Code of Conduct as member of PyCQA; #483
+
+Changes:
+
+* Added tox test support for Python 3.5 and pypy3
+* Added check E275 for whitespace on `from ... import ...` lines; #489 / #491
+* Added W503 to the list of codes ignored by default ignore list; #498
+* Removed use of project level `.pep8` configuration file; #364
+
+Bugs:
+
+* Fixed bug with treating `~` operator as binary; #383 / #384
+* Identify binary operators as unary; #484 / #485
+
+1.7.0 (2016-01-12)
+------------------
+
+Announcements:
+
+* Repository moved to PyCQA Organization on GitHub:
+  https://github.com/pycqa/pep8
+
+Changes:
+
+* Reverted the fix in #368, "options passed on command line are only ones
+  accepted" feature. This has many unintended consequences in pep8 and flake8
+  and needs to be reworked when I have more time.
+* Added support for Python 3.5. (Issue #420 & #459)
+* Added support for multi-line config_file option parsing. (Issue #429)
+* Improved parameter parsing. (Issues #420 & #456)
+
+Bugs:
+
+* Fixed BytesWarning on Python 3. (Issue #459)
+
+1.6.2 (2015-02-15)
+------------------
+
+Changes:
+
+* Added check for breaking around a binary operator. (Issue #197, Pull #305)
+
+Bugs:
+
+* Restored config_file parameter in process_options(). (Issue #380)
+
+
+1.6.1 (2015-02-08)
+------------------
+
+Changes:
+
+* Assign variables before referenced. (Issue #287)
+
+Bugs:
+
+* Exception thrown due to unassigned ``local_dir`` variable. (Issue #377)
+
+
+1.6.0 (2015-02-06)
+------------------
+
+News:
+
+* Ian Lee  joined the project as a maintainer.
+
+Changes:
+
+* Report E731 for lambda assignment. (Issue #277)
+
+* Report E704 for one-liner def instead of E701.
+  Do not report this error in the default configuration. (Issue #277)
+
+* Replace codes E111, E112 and E113 with codes E114, E115 and E116
+  for bad indentation of comments. (Issue #274)
+
+* Report E266 instead of E265 when the block comment starts with
+  multiple ``#``. (Issue #270)
+
+* Report E402 for import statements not at the top of the file. (Issue #264)
+
+* Do not enforce whitespaces around ``**`` operator. (Issue #292)
+
+* Strip whitespace from around paths during normalization. (Issue #339 / #343)
+
+* Update ``--format`` documentation. (Issue #198 / Pull Request #310)
+
+* Add ``.tox/`` to default excludes. (Issue #335)
+
+* Do not report E121 or E126 in the default configuration. (Issues #256 / #316)
+
+* Allow spaces around the equals sign in an annotated function. (Issue #357)
+
+* Allow trailing backslash if in an inline comment. (Issue #374)
+
+* If ``--config`` is used, only that configuration is processed. Otherwise,
+  merge the user and local configurations are merged. (Issue #368 / #369)
+
+Bug fixes:
+
+* Don't crash if Checker.build_tokens_line() returns None. (Issue #306)
+
+* Don't crash if os.path.expanduser() throws an ImportError. (Issue #297)
+
+* Missing space around keyword parameter equal not always reported, E251.
+  (Issue #323)
+
+* Fix false positive E711/E712/E713. (Issues #330 and #336)
+
+* Do not skip physical checks if the newline is escaped. (Issue #319)
+
+* Flush sys.stdout to avoid race conditions with printing. See flake8 bug:
+  https://gitlab.com/pycqa/flake8/issues/17 for more details. (Issue #363)
+
+
+1.5.7 (2014-05-29)
+------------------
+
+Bug fixes:
+
+* Skip the traceback on "Broken pipe" signal. (Issue #275)
+
+* Do not exit when an option in ``setup.cfg`` or ``tox.ini``
+  is not recognized.
+
+* Check the last line even if it does not end with a newline. (Issue #286)
+
+* Always open files in universal newlines mode in Python 2. (Issue #288)
+
+
+1.5.6 (2014-04-14)
+------------------
+
+Bug fixes:
+
+* Check the last line even if it has no end-of-line. (Issue #273)
+
+
+1.5.5 (2014-04-10)
+------------------
+
+Bug fixes:
+
+* Fix regression with E22 checks and inline comments. (Issue #271)
+
+
+1.5.4 (2014-04-07)
+------------------
+
+Bug fixes:
+
+* Fix negative offset with E303 before a multi-line docstring.
+  (Issue #269)
+
+
+1.5.3 (2014-04-04)
+------------------
+
+Bug fixes:
+
+* Fix wrong offset computation when error is on the last char
+  of a physical line. (Issue #268)
+
+
+1.5.2 (2014-04-04)
+------------------
+
+Changes:
+
+* Distribute a universal wheel file.
+
+Bug fixes:
+
+* Report correct line number for E303 with comments. (Issue #60)
+
+* Do not allow newline after parameter equal. (Issue #252)
+
+* Fix line number reported for multi-line strings. (Issue #220)
+
+* Fix false positive E121/E126 with multi-line strings. (Issue #265)
+
+* Fix E501 not detected in comments with Python 2.5.
+
+* Fix caret position with ``--show-source`` when line contains tabs.
+
+
+1.5.1 (2014-03-27)
+------------------
+
+Bug fixes:
+
+* Fix a crash with E125 on multi-line strings. (Issue #263)
+
+
+1.5 (2014-03-26)
+----------------
+
+Changes:
+
+* Report E129 instead of E125 for visually indented line with same
+  indent as next logical line.  (Issue #126)
+
+* Report E265 for space before block comment. (Issue #190)
+
+* Report E713 and E714 when operators ``not in`` and ``is not`` are
+  recommended. (Issue #236)
+
+* Allow long lines in multiline strings and comments if they cannot
+  be wrapped. (Issue #224).
+
+* Optionally disable physical line checks inside multiline strings,
+  using ``# noqa``. (Issue #242)
+
+* Change text for E121 to report "continuation line under-indented
+  for hanging indent" instead of indentation not being a
+  multiple of 4.
+
+* Report E131 instead of E121 / E126 if the hanging indent is not
+  consistent within the same continuation block.  It helps when
+  error E121 or E126 is in the ``ignore`` list.
+
+* Report E126 instead of E121 when the continuation line is hanging
+  with extra indentation, even if indentation is not a multiple of 4.
+
+Bug fixes:
+
+* Allow the checkers to report errors on empty files. (Issue #240)
+
+* Fix ignoring too many checks when ``--select`` is used with codes
+  declared in a flake8 extension. (Issue #216)
+
+* Fix regression with multiple brackets. (Issue #214)
+
+* Fix ``StyleGuide`` to parse the local configuration if the
+  keyword argument ``paths`` is specified. (Issue #246)
+
+* Fix a false positive E124 for hanging indent. (Issue #254)
+
+* Fix a false positive E126 with embedded colon. (Issue #144)
+
+* Fix a false positive E126 when indenting with tabs. (Issue #204)
+
+* Fix behaviour when ``exclude`` is in the configuration file and
+  the current directory is not the project directory. (Issue #247)
+
+* The logical checks can return ``None`` instead of an empty iterator.
+  (Issue #250)
+
+* Do not report multiple E101 if only the first indentation starts
+  with a tab. (Issue #237)
+
+* Fix a rare false positive W602. (Issue #34)
+
+
+1.4.6 (2013-07-02)
+------------------
+
+Changes:
+
+* Honor ``# noqa`` for errors E711 and E712. (Issue #180)
+
+* When both a ``tox.ini`` and a ``setup.cfg`` are present in the project
+  directory, merge their contents.  The ``tox.ini`` file takes
+  precedence (same as before). (Issue #182)
+
+* Give priority to ``--select`` over ``--ignore``. (Issue #188)
+
+* Compare full path when excluding a file. (Issue #186)
+
+* New option ``--hang-closing`` to switch to the alternative style of
+  closing bracket indentation for hanging indent.  Add error E133 for
+  closing bracket which is missing indentation. (Issue #103)
+
+* Accept both styles of closing bracket indentation for hanging indent.
+  Do not report error E123 in the default configuration. (Issue #103)
+
+Bug fixes:
+
+* Do not crash when running AST checks and the document contains null bytes.
+  (Issue #184)
+
+* Correctly report other E12 errors when E123 is ignored. (Issue #103)
+
+* Fix false positive E261/E262 when the file contains a BOM. (Issue #193)
+
+* Fix E701, E702 and E703 not detected sometimes. (Issue #196)
+
+* Fix E122 not detected in some cases. (Issue #201 and #208)
+
+* Fix false positive E121 with multiple brackets. (Issue #203)
+
+
+1.4.5 (2013-03-06)
+------------------
+
+* When no path is specified, do not try to read from stdin.  The feature
+  was added in 1.4.3, but it is not supported on Windows.  Use ``-``
+  filename argument to read from stdin.  This usage is supported
+  since 1.3.4. (Issue #170)
+
+* Do not require ``setuptools`` in setup.py.  It works around an issue
+  with ``pip`` and Python 3. (Issue #172)
+
+* Add ``__pycache__`` to the ignore list.
+
+* Change misleading message for E251. (Issue #171)
+
+* Do not report false E302 when the source file has a coding cookie or a
+  comment on the first line. (Issue #174)
+
+* Reorganize the tests and add tests for the API and for the command line
+  usage and options. (Issues #161 and #162)
+
+* Ignore all checks which are not explicitly selected when ``select`` is
+  passed to the ``StyleGuide`` constructor.
+
+
+1.4.4 (2013-02-24)
+------------------
+
+* Report E227 or E228 instead of E225 for whitespace around bitwise, shift
+  or modulo operators. (Issue #166)
+
+* Change the message for E226 to make clear that it is about arithmetic
+  operators.
+
+* Fix a false positive E128 for continuation line indentation with tabs.
+
+* Fix regression with the ``--diff`` option. (Issue #169)
+
+* Fix the ``TestReport`` class to print the unexpected warnings and
+  errors.
+
+
+1.4.3 (2013-02-22)
+------------------
+
+* Hide the ``--doctest`` and ``--testsuite`` options when installed.
+
+* Fix crash with AST checkers when the syntax is invalid. (Issue #160)
+
+* Read from standard input if no path is specified.
+
+* Initiate a graceful shutdown on ``Control+C``.
+
+* Allow changing the ``checker_class`` for the ``StyleGuide``.
+
+
+1.4.2 (2013-02-10)
+------------------
+
+* Support AST checkers provided by third-party applications.
+
+* Register new checkers with ``register_check(func_or_cls, codes)``.
+
+* Allow constructing a ``StyleGuide`` with a custom parser.
+
+* Accept visual indentation without parenthesis after the ``if``
+  statement. (Issue #151)
+
+* Fix UnboundLocalError when using ``# noqa`` with continued lines.
+  (Issue #158)
+
+* Re-order the lines for the ``StandardReport``.
+
+* Expand tabs when checking E12 continuation lines. (Issue #155)
+
+* Refactor the testing class ``TestReport`` and the specific test
+  functions into a separate test module.
+
+
+1.4.1 (2013-01-18)
+------------------
+
+* Allow sphinx.ext.autodoc syntax for comments. (Issue #110)
+
+* Report E703 instead of E702 for the trailing semicolon. (Issue #117)
+
+* Honor ``# noqa`` in addition to ``# nopep8``. (Issue #149)
+
+* Expose the ``OptionParser`` factory for better extensibility.
+
+
+1.4 (2012-12-22)
+----------------
+
+* Report E226 instead of E225 for optional whitespace around common
+  operators (``*``, ``**``, ``/``, ``+`` and ``-``).  This new error
+  code is ignored in the default configuration because PEP 8 recommends
+  to "use your own judgement". (Issue #96)
+
+* Lines with a ``# nopep8`` at the end will not issue errors on line
+  length E501 or continuation line indentation E12*. (Issue #27)
+
+* Fix AssertionError when the source file contains an invalid line
+  ending ``"\r\r\n"``. (Issue #119)
+
+* Read the ``[pep8]`` section of ``tox.ini`` or ``setup.cfg`` if present.
+  (Issue #93 and #141)
+
+* Add the Sphinx-based documentation, and publish it
+  on https://pycodestyle.readthedocs.io/. (Issue #105)
+
+
+1.3.4 (2012-12-18)
+------------------
+
+* Fix false positive E124 and E128 with comments. (Issue #100)
+
+* Fix error on stdin when running with bpython. (Issue #101)
+
+* Fix false positive E401. (Issue #104)
+
+* Report E231 for nested dictionary in list. (Issue #142)
+
+* Catch E271 at the beginning of the line. (Issue #133)
+
+* Fix false positive E126 for multi-line comments. (Issue #138)
+
+* Fix false positive E221 when operator is preceded by a comma. (Issue #135)
+
+* Fix ``--diff`` failing on one-line hunk. (Issue #137)
+
+* Fix the ``--exclude`` switch for directory paths. (Issue #111)
+
+* Use ``-`` filename to read from standard input. (Issue #128)
+
+
+1.3.3 (2012-06-27)
+------------------
+
+* Fix regression with continuation line checker. (Issue #98)
+
+
+1.3.2 (2012-06-26)
+------------------
+
+* Revert to the previous behaviour for ``--show-pep8``:
+  do not imply ``--first``. (Issue #89)
+
+* Add E902 for IO errors. (Issue #87)
+
+* Fix false positive for E121, and missed E124. (Issue #92)
+
+* Set a sensible default path for config file on Windows. (Issue #95)
+
+* Allow ``verbose`` in the configuration file. (Issue #91)
+
+* Show the enforced ``max-line-length`` in the error message. (Issue #86)
+
+
+1.3.1 (2012-06-18)
+------------------
+
+* Explain which configuration options are expected.  Accept and recommend
+  the options names with hyphen instead of underscore. (Issue #82)
+
+* Do not read the user configuration when used as a module
+  (except if ``config_file=True`` is passed to the ``StyleGuide`` constructor).
+
+* Fix wrong or missing cases for the E12 series.
+
+* Fix cases where E122 was missed. (Issue #81)
+
+
+1.3 (2012-06-15)
+----------------
+
+.. warning::
+   The internal API is backwards incompatible.
+
+* Remove global configuration and refactor the library around
+  a ``StyleGuide`` class; add the ability to configure various
+  reporters. (Issue #35 and #66)
+
+* Read user configuration from ``~/.config/pep8``
+  and local configuration from ``./.pep8``. (Issue #22)
+
+* Fix E502 for backslash embedded in multi-line string. (Issue #68)
+
+* Fix E225 for Python 3 iterable unpacking (PEP 3132). (Issue #72)
+
+* Enable the new checkers from the E12 series in the default
+  configuration.
+
+* Suggest less error-prone alternatives for E712 errors.
+
+* Rewrite checkers to run faster (E22, E251, E27).
+
+* Fixed a crash when parsed code is invalid (too many
+  closing brackets).
+
+* Fix E127 and E128 for continuation line indentation. (Issue #74)
+
+* New option ``--format`` to customize the error format. (Issue #23)
+
+* New option ``--diff`` to check only modified code.  The unified
+  diff is read from STDIN.  Example: ``hg diff | pep8 --diff``
+  (Issue #39)
+
+* Correctly report the count of failures and set the exit code to 1
+  when the ``--doctest`` or the ``--testsuite`` fails.
+
+* Correctly detect the encoding in Python 3. (Issue #69)
+
+* Drop support for Python 2.3, 2.4 and 3.0. (Issue #78)
+
+
+1.2 (2012-06-01)
+----------------
+
+* Add E121 through E128 for continuation line indentation.  These
+  checks are disabled by default.  If you want to force all checks,
+  use switch ``--select=E,W``.  Patch by Sam Vilain. (Issue #64)
+
+* Add E721 for direct type comparisons. (Issue #47)
+
+* Add E711 and E712 for comparisons to singletons. (Issue #46)
+
+* Fix spurious E225 and E701 for function annotations. (Issue #29)
+
+* Add E502 for explicit line join between brackets.
+
+* Fix E901 when printing source with ``--show-source``.
+
+* Report all errors for each checker, instead of reporting only the
+  first occurrence for each line.
+
+* Option ``--show-pep8`` implies ``--first``.
+
+
+1.1 (2012-05-24)
+----------------
+
+* Add E901 for syntax errors. (Issues #63 and #30)
+
+* Add E271, E272, E273 and E274 for extraneous whitespace around
+  keywords. (Issue #57)
+
+* Add ``tox.ini`` configuration file for tests. (Issue #61)
+
+* Add ``.travis.yml`` configuration file for continuous integration.
+  (Issue #62)
+
+
+1.0.1 (2012-04-06)
+------------------
+
+* Fix inconsistent version numbers.
+
+
+1.0 (2012-04-04)
+----------------
+
+* Fix W602 ``raise`` to handle multi-char names. (Issue #53)
+
+
+0.7.0 (2012-03-26)
+------------------
+
+* Now ``--first`` prints only the first occurrence of each error.
+  The ``--repeat`` flag becomes obsolete because it is the default
+  behaviour. (Issue #6)
+
+* Allow specifying ``--max-line-length``. (Issue #36)
+
+* Make the shebang more flexible. (Issue #26)
+
+* Add testsuite to the bundle. (Issue #25)
+
+* Fixes for Jython. (Issue #49)
+
+* Add PyPI classifiers. (Issue #43)
+
+* Fix the ``--exclude`` option. (Issue #48)
+
+* Fix W602, accept ``raise`` with 3 arguments. (Issue #34)
+
+* Correctly select all tests if ``DEFAULT_IGNORE == ''``.
+
+
+0.6.1 (2010-10-03)
+------------------
+
+* Fix inconsistent version numbers. (Issue #21)
+
+
+0.6.0 (2010-09-19)
+------------------
+
+* Test suite reorganized and enhanced in order to check more failures
+  with fewer test files.  Read the ``run_tests`` docstring for details
+  about the syntax.
+
+* Fix E225: accept ``print >>sys.stderr, "..."`` syntax.
+
+* Fix E501 for lines containing multibyte encoded characters. (Issue #7)
+
+* Fix E221, E222, E223, E224 not detected in some cases. (Issue #16)
+
+* Fix E211 to reject ``v = dic['a'] ['b']``. (Issue #17)
+
+* Exit code is always 1 if any error or warning is found. (Issue #10)
+
+* ``--ignore`` checks are now really ignored, especially in
+  conjunction with ``--count``. (Issue #8)
+
+* Blank lines with spaces yield W293 instead of W291: some developers
+  want to ignore this warning and indent the blank lines to paste their
+  code easily in the Python interpreter.
+
+* Fix E301: do not require a blank line before an indented block. (Issue #14)
+
+* Fix E203 to accept NumPy slice notation ``a[0, :]``. (Issue #13)
+
+* Performance improvements.
+
+* Fix decoding and checking non-UTF8 files in Python 3.
+
+* Fix E225: reject ``True+False`` when running on Python 3.
+
+* Fix an exception when the line starts with an operator.
+
+* Allow a new line before closing ``)``, ``}`` or ``]``. (Issue #5)
+
+
+0.5.0 (2010-02-17)
+------------------
+
+* Changed the ``--count`` switch to print to sys.stderr and set
+  exit code to 1 if any error or warning is found.
+
+* E241 and E242 are removed from the standard checks. If you want to
+  include these checks, use switch ``--select=E,W``. (Issue #4)
+
+* Blank line is not mandatory before the first class method or nested
+  function definition, even if there's a docstring. (Issue #1)
+
+* Add the switch ``--version``.
+
+* Fix decoding errors with Python 3. (Issue #13 [1]_)
+
+* Add ``--select`` option which is mirror of ``--ignore``.
+
+* Add checks E261 and E262 for spaces before inline comments.
+
+* New check W604 warns about deprecated usage of backticks.
+
+* New check W603 warns about the deprecated operator ``<>``.
+
+* Performance improvement, due to rewriting of E225.
+
+* E225 now accepts:
+
+  - no whitespace after unary operator or similar. (Issue #9 [1]_)
+
+  - lambda function with argument unpacking or keyword defaults.
+
+* Reserve "2 blank lines" for module-level logical blocks. (E303)
+
+* Allow multi-line comments. (E302, issue #10 [1]_)
+
+
+0.4.2 (2009-10-22)
+------------------
+
+* Decorators on classes and class methods are OK now.
+
+
+0.4 (2009-10-20)
+----------------
+
+* Support for all versions of Python from 2.3 to 3.1.
+
+* New and greatly expanded self tests.
+
+* Added ``--count`` option to print the total number of errors and warnings.
+
+* Further improvements to the handling of comments and blank lines.
+  (Issue #1 [1]_ and others changes.)
+
+* Check all py files in directory when passed a directory (Issue
+  #2 [1]_). This also prevents an exception when traversing directories
+  with non ``*.py`` files.
+
+* E231 should allow commas to be followed by ``)``. (Issue #3 [1]_)
+
+* Spaces are no longer required around the equals sign for keyword
+  arguments or default parameter values.
+
+
+.. [1] These issues refer to the `previous issue tracker`__.
+.. __:  http://github.com/cburroughs/pep8.py/issues
+
+
+0.3.1 (2009-09-14)
+------------------
+
+* Fixes for comments: do not count them when checking for blank lines between
+  items.
+
+* Added setup.py for pypi upload and easy_installability.
+
+
+0.2 (2007-10-16)
+----------------
+
+* Loads of fixes and improvements.
+
+
+0.1 (2006-10-01)
+----------------
+
+* First release.
+
+
diff --git a/env/lib/python3.8/site-packages/pycodestyle-2.6.0.dist-info/RECORD b/env/lib/python3.8/site-packages/pycodestyle-2.6.0.dist-info/RECORD
new file mode 100644
index 000000000..72a2c6228
--- /dev/null
+++ b/env/lib/python3.8/site-packages/pycodestyle-2.6.0.dist-info/RECORD
@@ -0,0 +1,11 @@
+../../../bin/pycodestyle,sha256=af5O5dhqEQfCCIeuMvnaAPx-deoh2V9H4cjibHhhJxM,292
+__pycache__/pycodestyle.cpython-38.pyc,,
+pycodestyle-2.6.0.dist-info/INSTALLER,sha256=zuuue4knoyJ-UwPPXg8fezS7VCrXJQrAP7zeNuwvFQg,4
+pycodestyle-2.6.0.dist-info/LICENSE,sha256=93IpXoGvNHjTTojlLQdiACMOx91qOeEjvFyzWqZqva4,1254
+pycodestyle-2.6.0.dist-info/METADATA,sha256=WI4-bMnR66kT7MKGLVFW7xqmuotPGP0uLJProv7nhD4,30287
+pycodestyle-2.6.0.dist-info/RECORD,,
+pycodestyle-2.6.0.dist-info/WHEEL,sha256=kGT74LWyRUZrL4VgLh6_g12IeVl_9u9ZVhadrgXZUEY,110
+pycodestyle-2.6.0.dist-info/entry_points.txt,sha256=6JU_7SAppC93MBSQi1_QxDwEQUyg6cgK71ab9q_Hxco,51
+pycodestyle-2.6.0.dist-info/namespace_packages.txt,sha256=AbpHGcgLb-kRsJGnwFEktk7uzpZOCcBY74-YBdrKVGs,1
+pycodestyle-2.6.0.dist-info/top_level.txt,sha256=rHbIEiXmvsJ016mFcLVcF_d-dKgP3VdfOB6CWbivZug,12
+pycodestyle.py,sha256=xYcAkNSMHMAlz6cmdkfhwK7QC5RsRwK2pfG2_uPI2xM,103376
diff --git a/env/lib/python3.8/site-packages/pycodestyle-2.6.0.dist-info/WHEEL b/env/lib/python3.8/site-packages/pycodestyle-2.6.0.dist-info/WHEEL
new file mode 100644
index 000000000..ef99c6cf3
--- /dev/null
+++ b/env/lib/python3.8/site-packages/pycodestyle-2.6.0.dist-info/WHEEL
@@ -0,0 +1,6 @@
+Wheel-Version: 1.0
+Generator: bdist_wheel (0.34.2)
+Root-Is-Purelib: true
+Tag: py2-none-any
+Tag: py3-none-any
+
diff --git a/env/lib/python3.8/site-packages/pycodestyle-2.6.0.dist-info/entry_points.txt b/env/lib/python3.8/site-packages/pycodestyle-2.6.0.dist-info/entry_points.txt
new file mode 100644
index 000000000..71bd88504
--- /dev/null
+++ b/env/lib/python3.8/site-packages/pycodestyle-2.6.0.dist-info/entry_points.txt
@@ -0,0 +1,3 @@
+[console_scripts]
+pycodestyle = pycodestyle:_main
+
diff --git a/env/lib/python3.8/site-packages/pycodestyle-2.6.0.dist-info/namespace_packages.txt b/env/lib/python3.8/site-packages/pycodestyle-2.6.0.dist-info/namespace_packages.txt
new file mode 100644
index 000000000..8b1378917
--- /dev/null
+++ b/env/lib/python3.8/site-packages/pycodestyle-2.6.0.dist-info/namespace_packages.txt
@@ -0,0 +1 @@
+
diff --git a/env/lib/python3.8/site-packages/pycodestyle-2.6.0.dist-info/top_level.txt b/env/lib/python3.8/site-packages/pycodestyle-2.6.0.dist-info/top_level.txt
new file mode 100644
index 000000000..282a93fbd
--- /dev/null
+++ b/env/lib/python3.8/site-packages/pycodestyle-2.6.0.dist-info/top_level.txt
@@ -0,0 +1 @@
+pycodestyle
diff --git a/env/lib/python3.8/site-packages/pycodestyle.py b/env/lib/python3.8/site-packages/pycodestyle.py
new file mode 100644
index 000000000..deb45395d
--- /dev/null
+++ b/env/lib/python3.8/site-packages/pycodestyle.py
@@ -0,0 +1,2763 @@
+#!/usr/bin/env python
+# pycodestyle.py - Check Python source code formatting, according to
+# PEP 8
+#
+# Copyright (C) 2006-2009 Johann C. Rocholl 
+# Copyright (C) 2009-2014 Florent Xicluna 
+# Copyright (C) 2014-2016 Ian Lee 
+#
+# Permission is hereby granted, free of charge, to any person
+# obtaining a copy of this software and associated documentation files
+# (the "Software"), to deal in the Software without restriction,
+# including without limitation the rights to use, copy, modify, merge,
+# publish, distribute, sublicense, and/or sell copies of the Software,
+# and to permit persons to whom the Software is furnished to do so,
+# subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be
+# included in all copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+# MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+# BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+# ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+# CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+
+r"""
+Check Python source code formatting, according to PEP 8.
+
+For usage and a list of options, try this:
+$ python pycodestyle.py -h
+
+This program and its regression test suite live here:
+https://github.com/pycqa/pycodestyle
+
+Groups of errors and warnings:
+E errors
+W warnings
+100 indentation
+200 whitespace
+300 blank lines
+400 imports
+500 line length
+600 deprecation
+700 statements
+900 syntax error
+"""
+from __future__ import with_statement
+
+import inspect
+import keyword
+import os
+import re
+import sys
+import time
+import tokenize
+import warnings
+import bisect
+
+try:
+    from functools import lru_cache
+except ImportError:
+    def lru_cache(maxsize=128):  # noqa as it's a fake implementation.
+        """Does not really need a real a lru_cache, it's just
+        optimization, so let's just do nothing here. Python 3.2+ will
+        just get better performances, time to upgrade?
+        """
+        return lambda function: function
+
+from fnmatch import fnmatch
+from optparse import OptionParser
+
+try:
+    from configparser import RawConfigParser
+    from io import TextIOWrapper
+except ImportError:
+    from ConfigParser import RawConfigParser
+
+__version__ = '2.6.0'
+
+DEFAULT_EXCLUDE = '.svn,CVS,.bzr,.hg,.git,__pycache__,.tox'
+DEFAULT_IGNORE = 'E121,E123,E126,E226,E24,E704,W503,W504'
+try:
+    if sys.platform == 'win32':
+        USER_CONFIG = os.path.expanduser(r'~\.pycodestyle')
+    else:
+        USER_CONFIG = os.path.join(
+            os.getenv('XDG_CONFIG_HOME') or os.path.expanduser('~/.config'),
+            'pycodestyle'
+        )
+except ImportError:
+    USER_CONFIG = None
+
+PROJECT_CONFIG = ('setup.cfg', 'tox.ini')
+TESTSUITE_PATH = os.path.join(os.path.dirname(__file__), 'testsuite')
+MAX_LINE_LENGTH = 79
+# Number of blank lines between various code parts.
+BLANK_LINES_CONFIG = {
+    # Top level class and function.
+    'top_level': 2,
+    # Methods and nested class and function.
+    'method': 1,
+}
+MAX_DOC_LENGTH = 72
+REPORT_FORMAT = {
+    'default': '%(path)s:%(row)d:%(col)d: %(code)s %(text)s',
+    'pylint': '%(path)s:%(row)d: [%(code)s] %(text)s',
+}
+
+PyCF_ONLY_AST = 1024
+SINGLETONS = frozenset(['False', 'None', 'True'])
+KEYWORDS = frozenset(keyword.kwlist + ['print', 'async']) - SINGLETONS
+UNARY_OPERATORS = frozenset(['>>', '**', '*', '+', '-'])
+ARITHMETIC_OP = frozenset(['**', '*', '/', '//', '+', '-', '@'])
+WS_OPTIONAL_OPERATORS = ARITHMETIC_OP.union(['^', '&', '|', '<<', '>>', '%'])
+# Warn for -> function annotation operator in py3.5+ (issue 803)
+FUNCTION_RETURN_ANNOTATION_OP = ['->'] if sys.version_info >= (3, 5) else []
+ASSIGNMENT_EXPRESSION_OP = [':='] if sys.version_info >= (3, 8) else []
+WS_NEEDED_OPERATORS = frozenset([
+    '**=', '*=', '/=', '//=', '+=', '-=', '!=', '<>', '<', '>',
+    '%=', '^=', '&=', '|=', '==', '<=', '>=', '<<=', '>>=', '=',
+    'and', 'in', 'is', 'or'] +
+    FUNCTION_RETURN_ANNOTATION_OP +
+    ASSIGNMENT_EXPRESSION_OP)
+WHITESPACE = frozenset(' \t')
+NEWLINE = frozenset([tokenize.NL, tokenize.NEWLINE])
+SKIP_TOKENS = NEWLINE.union([tokenize.INDENT, tokenize.DEDENT])
+# ERRORTOKEN is triggered by backticks in Python 3
+SKIP_COMMENTS = SKIP_TOKENS.union([tokenize.COMMENT, tokenize.ERRORTOKEN])
+BENCHMARK_KEYS = ['directories', 'files', 'logical lines', 'physical lines']
+
+INDENT_REGEX = re.compile(r'([ \t]*)')
+RAISE_COMMA_REGEX = re.compile(r'raise\s+\w+\s*,')
+RERAISE_COMMA_REGEX = re.compile(r'raise\s+\w+\s*,.*,\s*\w+\s*$')
+ERRORCODE_REGEX = re.compile(r'\b[A-Z]\d{3}\b')
+DOCSTRING_REGEX = re.compile(r'u?r?["\']')
+EXTRANEOUS_WHITESPACE_REGEX = re.compile(r'[\[({] | [\]}),;]| :(?!=)')
+WHITESPACE_AFTER_COMMA_REGEX = re.compile(r'[,;:]\s*(?:  |\t)')
+COMPARE_SINGLETON_REGEX = re.compile(r'(\bNone|\bFalse|\bTrue)?\s*([=!]=)'
+                                     r'\s*(?(1)|(None|False|True))\b')
+COMPARE_NEGATIVE_REGEX = re.compile(r'\b(?%&^]+)(\s*)')
+LAMBDA_REGEX = re.compile(r'\blambda\b')
+HUNK_REGEX = re.compile(r'^@@ -\d+(?:,\d+)? \+(\d+)(?:,(\d+))? @@.*$')
+STARTSWITH_DEF_REGEX = re.compile(r'^(async\s+def|def)\b')
+STARTSWITH_TOP_LEVEL_REGEX = re.compile(r'^(async\s+def\s+|def\s+|class\s+|@)')
+STARTSWITH_INDENT_STATEMENT_REGEX = re.compile(
+    r'^\s*({0})\b'.format('|'.join(s.replace(' ', r'\s+') for s in (
+        'def', 'async def',
+        'for', 'async for',
+        'if', 'elif', 'else',
+        'try', 'except', 'finally',
+        'with', 'async with',
+        'class',
+        'while',
+    )))
+)
+DUNDER_REGEX = re.compile(r'^__([^\s]+)__ = ')
+
+_checks = {'physical_line': {}, 'logical_line': {}, 'tree': {}}
+
+
+def _get_parameters(function):
+    if sys.version_info >= (3, 3):
+        return [parameter.name
+                for parameter
+                in inspect.signature(function).parameters.values()
+                if parameter.kind == parameter.POSITIONAL_OR_KEYWORD]
+    else:
+        return inspect.getargspec(function)[0]
+
+
+def register_check(check, codes=None):
+    """Register a new check object."""
+    def _add_check(check, kind, codes, args):
+        if check in _checks[kind]:
+            _checks[kind][check][0].extend(codes or [])
+        else:
+            _checks[kind][check] = (codes or [''], args)
+    if inspect.isfunction(check):
+        args = _get_parameters(check)
+        if args and args[0] in ('physical_line', 'logical_line'):
+            if codes is None:
+                codes = ERRORCODE_REGEX.findall(check.__doc__ or '')
+            _add_check(check, args[0], codes, args)
+    elif inspect.isclass(check):
+        if _get_parameters(check.__init__)[:2] == ['self', 'tree']:
+            _add_check(check, 'tree', codes, None)
+    return check
+
+
+########################################################################
+# Plugins (check functions) for physical lines
+########################################################################
+
+@register_check
+def tabs_or_spaces(physical_line, indent_char):
+    r"""Never mix tabs and spaces.
+
+    The most popular way of indenting Python is with spaces only.  The
+    second-most popular way is with tabs only.  Code indented with a
+    mixture of tabs and spaces should be converted to using spaces
+    exclusively.  When invoking the Python command line interpreter with
+    the -t option, it issues warnings about code that illegally mixes
+    tabs and spaces.  When using -tt these warnings become errors.
+    These options are highly recommended!
+
+    Okay: if a == 0:\n    a = 1\n    b = 1
+    E101: if a == 0:\n        a = 1\n\tb = 1
+    """
+    indent = INDENT_REGEX.match(physical_line).group(1)
+    for offset, char in enumerate(indent):
+        if char != indent_char:
+            return offset, "E101 indentation contains mixed spaces and tabs"
+
+
+@register_check
+def tabs_obsolete(physical_line):
+    r"""On new projects, spaces-only are strongly recommended over tabs.
+
+    Okay: if True:\n    return
+    W191: if True:\n\treturn
+    """
+    indent = INDENT_REGEX.match(physical_line).group(1)
+    if '\t' in indent:
+        return indent.index('\t'), "W191 indentation contains tabs"
+
+
+@register_check
+def trailing_whitespace(physical_line):
+    r"""Trailing whitespace is superfluous.
+
+    The warning returned varies on whether the line itself is blank,
+    for easier filtering for those who want to indent their blank lines.
+
+    Okay: spam(1)\n#
+    W291: spam(1) \n#
+    W293: class Foo(object):\n    \n    bang = 12
+    """
+    physical_line = physical_line.rstrip('\n')    # chr(10), newline
+    physical_line = physical_line.rstrip('\r')    # chr(13), carriage return
+    physical_line = physical_line.rstrip('\x0c')  # chr(12), form feed, ^L
+    stripped = physical_line.rstrip(' \t\v')
+    if physical_line != stripped:
+        if stripped:
+            return len(stripped), "W291 trailing whitespace"
+        else:
+            return 0, "W293 blank line contains whitespace"
+
+
+@register_check
+def trailing_blank_lines(physical_line, lines, line_number, total_lines):
+    r"""Trailing blank lines are superfluous.
+
+    Okay: spam(1)
+    W391: spam(1)\n
+
+    However the last line should end with a new line (warning W292).
+    """
+    if line_number == total_lines:
+        stripped_last_line = physical_line.rstrip()
+        if physical_line and not stripped_last_line:
+            return 0, "W391 blank line at end of file"
+        if stripped_last_line == physical_line:
+            return len(lines[-1]), "W292 no newline at end of file"
+
+
+@register_check
+def maximum_line_length(physical_line, max_line_length, multiline,
+                        line_number, noqa):
+    r"""Limit all lines to a maximum of 79 characters.
+
+    There are still many devices around that are limited to 80 character
+    lines; plus, limiting windows to 80 characters makes it possible to
+    have several windows side-by-side.  The default wrapping on such
+    devices looks ugly.  Therefore, please limit all lines to a maximum
+    of 79 characters. For flowing long blocks of text (docstrings or
+    comments), limiting the length to 72 characters is recommended.
+
+    Reports error E501.
+    """
+    line = physical_line.rstrip()
+    length = len(line)
+    if length > max_line_length and not noqa:
+        # Special case: ignore long shebang lines.
+        if line_number == 1 and line.startswith('#!'):
+            return
+        # Special case for long URLs in multi-line docstrings or
+        # comments, but still report the error when the 72 first chars
+        # are whitespaces.
+        chunks = line.split()
+        if ((len(chunks) == 1 and multiline) or
+            (len(chunks) == 2 and chunks[0] == '#')) and \
+                len(line) - len(chunks[-1]) < max_line_length - 7:
+            return
+        if hasattr(line, 'decode'):   # Python 2
+            # The line could contain multi-byte characters
+            try:
+                length = len(line.decode('utf-8'))
+            except UnicodeError:
+                pass
+        if length > max_line_length:
+            return (max_line_length, "E501 line too long "
+                    "(%d > %d characters)" % (length, max_line_length))
+
+
+########################################################################
+# Plugins (check functions) for logical lines
+########################################################################
+
+
+def _is_one_liner(logical_line, indent_level, lines, line_number):
+    if not STARTSWITH_TOP_LEVEL_REGEX.match(logical_line):
+        return False
+
+    line_idx = line_number - 1
+
+    if line_idx < 1:
+        prev_indent = 0
+    else:
+        prev_indent = expand_indent(lines[line_idx - 1])
+
+    if prev_indent > indent_level:
+        return False
+
+    while line_idx < len(lines):
+        line = lines[line_idx].strip()
+        if not line.startswith('@') and STARTSWITH_TOP_LEVEL_REGEX.match(line):
+            break
+        else:
+            line_idx += 1
+    else:
+        return False  # invalid syntax: EOF while searching for def/class
+
+    next_idx = line_idx + 1
+    while next_idx < len(lines):
+        if lines[next_idx].strip():
+            break
+        else:
+            next_idx += 1
+    else:
+        return True  # line is last in the file
+
+    return expand_indent(lines[next_idx]) <= indent_level
+
+
+@register_check
+def blank_lines(logical_line, blank_lines, indent_level, line_number,
+                blank_before, previous_logical,
+                previous_unindented_logical_line, previous_indent_level,
+                lines):
+    r"""Separate top-level function and class definitions with two blank
+    lines.
+
+    Method definitions inside a class are separated by a single blank
+    line.
+
+    Extra blank lines may be used (sparingly) to separate groups of
+    related functions.  Blank lines may be omitted between a bunch of
+    related one-liners (e.g. a set of dummy implementations).
+
+    Use blank lines in functions, sparingly, to indicate logical
+    sections.
+
+    Okay: def a():\n    pass\n\n\ndef b():\n    pass
+    Okay: def a():\n    pass\n\n\nasync def b():\n    pass
+    Okay: def a():\n    pass\n\n\n# Foo\n# Bar\n\ndef b():\n    pass
+    Okay: default = 1\nfoo = 1
+    Okay: classify = 1\nfoo = 1
+
+    E301: class Foo:\n    b = 0\n    def bar():\n        pass
+    E302: def a():\n    pass\n\ndef b(n):\n    pass
+    E302: def a():\n    pass\n\nasync def b(n):\n    pass
+    E303: def a():\n    pass\n\n\n\ndef b(n):\n    pass
+    E303: def a():\n\n\n\n    pass
+    E304: @decorator\n\ndef a():\n    pass
+    E305: def a():\n    pass\na()
+    E306: def a():\n    def b():\n        pass\n    def c():\n        pass
+    """  # noqa
+    top_level_lines = BLANK_LINES_CONFIG['top_level']
+    method_lines = BLANK_LINES_CONFIG['method']
+
+    if not previous_logical and blank_before < top_level_lines:
+        return  # Don't expect blank lines before the first line
+    if previous_logical.startswith('@'):
+        if blank_lines:
+            yield 0, "E304 blank lines found after function decorator"
+    elif (blank_lines > top_level_lines or
+            (indent_level and blank_lines == method_lines + 1)
+          ):
+        yield 0, "E303 too many blank lines (%d)" % blank_lines
+    elif STARTSWITH_TOP_LEVEL_REGEX.match(logical_line):
+        # allow a group of one-liners
+        if (
+            _is_one_liner(logical_line, indent_level, lines, line_number) and
+            blank_before == 0
+        ):
+            return
+        if indent_level:
+            if not (blank_before == method_lines or
+                    previous_indent_level < indent_level or
+                    DOCSTRING_REGEX.match(previous_logical)
+                    ):
+                ancestor_level = indent_level
+                nested = False
+                # Search backwards for a def ancestor or tree root
+                # (top level).
+                for line in lines[line_number - top_level_lines::-1]:
+                    if line.strip() and expand_indent(line) < ancestor_level:
+                        ancestor_level = expand_indent(line)
+                        nested = STARTSWITH_DEF_REGEX.match(line.lstrip())
+                        if nested or ancestor_level == 0:
+                            break
+                if nested:
+                    yield 0, "E306 expected %s blank line before a " \
+                        "nested definition, found 0" % (method_lines,)
+                else:
+                    yield 0, "E301 expected %s blank line, found 0" % (
+                        method_lines,)
+        elif blank_before != top_level_lines:
+            yield 0, "E302 expected %s blank lines, found %d" % (
+                top_level_lines, blank_before)
+    elif (logical_line and
+            not indent_level and
+            blank_before != top_level_lines and
+            previous_unindented_logical_line.startswith(('def ', 'class '))
+          ):
+        yield 0, "E305 expected %s blank lines after " \
+            "class or function definition, found %d" % (
+                top_level_lines, blank_before)
+
+
+@register_check
+def extraneous_whitespace(logical_line):
+    r"""Avoid extraneous whitespace.
+
+    Avoid extraneous whitespace in these situations:
+    - Immediately inside parentheses, brackets or braces.
+    - Immediately before a comma, semicolon, or colon.
+
+    Okay: spam(ham[1], {eggs: 2})
+    E201: spam( ham[1], {eggs: 2})
+    E201: spam(ham[ 1], {eggs: 2})
+    E201: spam(ham[1], { eggs: 2})
+    E202: spam(ham[1], {eggs: 2} )
+    E202: spam(ham[1 ], {eggs: 2})
+    E202: spam(ham[1], {eggs: 2 })
+
+    E203: if x == 4: print x, y; x, y = y , x
+    E203: if x == 4: print x, y ; x, y = y, x
+    E203: if x == 4 : print x, y; x, y = y, x
+    """
+    line = logical_line
+    for match in EXTRANEOUS_WHITESPACE_REGEX.finditer(line):
+        text = match.group()
+        char = text.strip()
+        found = match.start()
+        if text == char + ' ':
+            # assert char in '([{'
+            yield found + 1, "E201 whitespace after '%s'" % char
+        elif line[found - 1] != ',':
+            code = ('E202' if char in '}])' else 'E203')  # if char in ',;:'
+            yield found, "%s whitespace before '%s'" % (code, char)
+
+
+@register_check
+def whitespace_around_keywords(logical_line):
+    r"""Avoid extraneous whitespace around keywords.
+
+    Okay: True and False
+    E271: True and  False
+    E272: True  and False
+    E273: True and\tFalse
+    E274: True\tand False
+    """
+    for match in KEYWORD_REGEX.finditer(logical_line):
+        before, after = match.groups()
+
+        if '\t' in before:
+            yield match.start(1), "E274 tab before keyword"
+        elif len(before) > 1:
+            yield match.start(1), "E272 multiple spaces before keyword"
+
+        if '\t' in after:
+            yield match.start(2), "E273 tab after keyword"
+        elif len(after) > 1:
+            yield match.start(2), "E271 multiple spaces after keyword"
+
+
+@register_check
+def missing_whitespace_after_import_keyword(logical_line):
+    r"""Multiple imports in form from x import (a, b, c) should have
+    space between import statement and parenthesised name list.
+
+    Okay: from foo import (bar, baz)
+    E275: from foo import(bar, baz)
+    E275: from importable.module import(bar, baz)
+    """
+    line = logical_line
+    indicator = ' import('
+    if line.startswith('from '):
+        found = line.find(indicator)
+        if -1 < found:
+            pos = found + len(indicator) - 1
+            yield pos, "E275 missing whitespace after keyword"
+
+
+@register_check
+def missing_whitespace(logical_line):
+    r"""Each comma, semicolon or colon should be followed by whitespace.
+
+    Okay: [a, b]
+    Okay: (3,)
+    Okay: a[1:4]
+    Okay: a[:4]
+    Okay: a[1:]
+    Okay: a[1:4:2]
+    E231: ['a','b']
+    E231: foo(bar,baz)
+    E231: [{'a':'b'}]
+    """
+    line = logical_line
+    for index in range(len(line) - 1):
+        char = line[index]
+        next_char = line[index + 1]
+        if char in ',;:' and next_char not in WHITESPACE:
+            before = line[:index]
+            if char == ':' and before.count('[') > before.count(']') and \
+                    before.rfind('{') < before.rfind('['):
+                continue  # Slice syntax, no space required
+            if char == ',' and next_char == ')':
+                continue  # Allow tuple with only one element: (3,)
+            if char == ':' and next_char == '=' and sys.version_info >= (3, 8):
+                continue  # Allow assignment expression
+            yield index, "E231 missing whitespace after '%s'" % char
+
+
+@register_check
+def indentation(logical_line, previous_logical, indent_char,
+                indent_level, previous_indent_level):
+    r"""Use 4 spaces per indentation level.
+
+    For really old code that you don't want to mess up, you can continue
+    to use 8-space tabs.
+
+    Okay: a = 1
+    Okay: if a == 0:\n    a = 1
+    E111:   a = 1
+    E114:   # a = 1
+
+    Okay: for item in items:\n    pass
+    E112: for item in items:\npass
+    E115: for item in items:\n# Hi\n    pass
+
+    Okay: a = 1\nb = 2
+    E113: a = 1\n    b = 2
+    E116: a = 1\n    # b = 2
+    """
+    c = 0 if logical_line else 3
+    tmpl = "E11%d %s" if logical_line else "E11%d %s (comment)"
+    if indent_level % 4:
+        yield 0, tmpl % (1 + c, "indentation is not a multiple of four")
+    indent_expect = previous_logical.endswith(':')
+    if indent_expect and indent_level <= previous_indent_level:
+        yield 0, tmpl % (2 + c, "expected an indented block")
+    elif not indent_expect and indent_level > previous_indent_level:
+        yield 0, tmpl % (3 + c, "unexpected indentation")
+
+    if indent_expect:
+        expected_indent_amount = 8 if indent_char == '\t' else 4
+        expected_indent_level = previous_indent_level + expected_indent_amount
+        if indent_level > expected_indent_level:
+            yield 0, tmpl % (7, 'over-indented')
+
+
+@register_check
+def continued_indentation(logical_line, tokens, indent_level, hang_closing,
+                          indent_char, noqa, verbose):
+    r"""Continuation lines indentation.
+
+    Continuation lines should align wrapped elements either vertically
+    using Python's implicit line joining inside parentheses, brackets
+    and braces, or using a hanging indent.
+
+    When using a hanging indent these considerations should be applied:
+    - there should be no arguments on the first line, and
+    - further indentation should be used to clearly distinguish itself
+      as a continuation line.
+
+    Okay: a = (\n)
+    E123: a = (\n    )
+
+    Okay: a = (\n    42)
+    E121: a = (\n   42)
+    E122: a = (\n42)
+    E123: a = (\n    42\n    )
+    E124: a = (24,\n     42\n)
+    E125: if (\n    b):\n    pass
+    E126: a = (\n        42)
+    E127: a = (24,\n      42)
+    E128: a = (24,\n    42)
+    E129: if (a or\n    b):\n    pass
+    E131: a = (\n    42\n 24)
+    """
+    first_row = tokens[0][2][0]
+    nrows = 1 + tokens[-1][2][0] - first_row
+    if noqa or nrows == 1:
+        return
+
+    # indent_next tells us whether the next block is indented; assuming
+    # that it is indented by 4 spaces, then we should not allow 4-space
+    # indents on the final continuation line; in turn, some other
+    # indents are allowed to have an extra 4 spaces.
+    indent_next = logical_line.endswith(':')
+
+    row = depth = 0
+    valid_hangs = (4,) if indent_char != '\t' else (4, 8)
+    # remember how many brackets were opened on each line
+    parens = [0] * nrows
+    # relative indents of physical lines
+    rel_indent = [0] * nrows
+    # for each depth, collect a list of opening rows
+    open_rows = [[0]]
+    # for each depth, memorize the hanging indentation
+    hangs = [None]
+    # visual indents
+    indent_chances = {}
+    last_indent = tokens[0][2]
+    visual_indent = None
+    last_token_multiline = False
+    # for each depth, memorize the visual indent column
+    indent = [last_indent[1]]
+    if verbose >= 3:
+        print(">>> " + tokens[0][4].rstrip())
+
+    for token_type, text, start, end, line in tokens:
+
+        newline = row < start[0] - first_row
+        if newline:
+            row = start[0] - first_row
+            newline = not last_token_multiline and token_type not in NEWLINE
+
+        if newline:
+            # this is the beginning of a continuation line.
+            last_indent = start
+            if verbose >= 3:
+                print("... " + line.rstrip())
+
+            # record the initial indent.
+            rel_indent[row] = expand_indent(line) - indent_level
+
+            # identify closing bracket
+            close_bracket = (token_type == tokenize.OP and text in ']})')
+
+            # is the indent relative to an opening bracket line?
+            for open_row in reversed(open_rows[depth]):
+                hang = rel_indent[row] - rel_indent[open_row]
+                hanging_indent = hang in valid_hangs
+                if hanging_indent:
+                    break
+            if hangs[depth]:
+                hanging_indent = (hang == hangs[depth])
+            # is there any chance of visual indent?
+            visual_indent = (not close_bracket and hang > 0 and
+                             indent_chances.get(start[1]))
+
+            if close_bracket and indent[depth]:
+                # closing bracket for visual indent
+                if start[1] != indent[depth]:
+                    yield (start, "E124 closing bracket does not match "
+                           "visual indentation")
+            elif close_bracket and not hang:
+                # closing bracket matches indentation of opening
+                # bracket's line
+                if hang_closing:
+                    yield start, "E133 closing bracket is missing indentation"
+            elif indent[depth] and start[1] < indent[depth]:
+                if visual_indent is not True:
+                    # visual indent is broken
+                    yield (start, "E128 continuation line "
+                           "under-indented for visual indent")
+            elif hanging_indent or (indent_next and rel_indent[row] == 8):
+                # hanging indent is verified
+                if close_bracket and not hang_closing:
+                    yield (start, "E123 closing bracket does not match "
+                           "indentation of opening bracket's line")
+                hangs[depth] = hang
+            elif visual_indent is True:
+                # visual indent is verified
+                indent[depth] = start[1]
+            elif visual_indent in (text, str):
+                # ignore token lined up with matching one from a
+                # previous line
+                pass
+            else:
+                # indent is broken
+                if hang <= 0:
+                    error = "E122", "missing indentation or outdented"
+                elif indent[depth]:
+                    error = "E127", "over-indented for visual indent"
+                elif not close_bracket and hangs[depth]:
+                    error = "E131", "unaligned for hanging indent"
+                else:
+                    hangs[depth] = hang
+                    if hang > 4:
+                        error = "E126", "over-indented for hanging indent"
+                    else:
+                        error = "E121", "under-indented for hanging indent"
+                yield start, "%s continuation line %s" % error
+
+        # look for visual indenting
+        if (parens[row] and
+                token_type not in (tokenize.NL, tokenize.COMMENT) and
+                not indent[depth]):
+            indent[depth] = start[1]
+            indent_chances[start[1]] = True
+            if verbose >= 4:
+                print("bracket depth %s indent to %s" % (depth, start[1]))
+        # deal with implicit string concatenation
+        elif (token_type in (tokenize.STRING, tokenize.COMMENT) or
+              text in ('u', 'ur', 'b', 'br')):
+            indent_chances[start[1]] = str
+        # visual indent after assert/raise/with
+        elif not row and not depth and text in ["assert", "raise", "with"]:
+            indent_chances[end[1] + 1] = True
+        # special case for the "if" statement because len("if (") == 4
+        elif not indent_chances and not row and not depth and text == 'if':
+            indent_chances[end[1] + 1] = True
+        elif text == ':' and line[end[1]:].isspace():
+            open_rows[depth].append(row)
+
+        # keep track of bracket depth
+        if token_type == tokenize.OP:
+            if text in '([{':
+                depth += 1
+                indent.append(0)
+                hangs.append(None)
+                if len(open_rows) == depth:
+                    open_rows.append([])
+                open_rows[depth].append(row)
+                parens[row] += 1
+                if verbose >= 4:
+                    print("bracket depth %s seen, col %s, visual min = %s" %
+                          (depth, start[1], indent[depth]))
+            elif text in ')]}' and depth > 0:
+                # parent indents should not be more than this one
+                prev_indent = indent.pop() or last_indent[1]
+                hangs.pop()
+                for d in range(depth):
+                    if indent[d] > prev_indent:
+                        indent[d] = 0
+                for ind in list(indent_chances):
+                    if ind >= prev_indent:
+                        del indent_chances[ind]
+                del open_rows[depth + 1:]
+                depth -= 1
+                if depth:
+                    indent_chances[indent[depth]] = True
+                for idx in range(row, -1, -1):
+                    if parens[idx]:
+                        parens[idx] -= 1
+                        break
+            assert len(indent) == depth + 1
+            if start[1] not in indent_chances:
+                # allow lining up tokens
+                indent_chances[start[1]] = text
+
+        last_token_multiline = (start[0] != end[0])
+        if last_token_multiline:
+            rel_indent[end[0] - first_row] = rel_indent[row]
+
+    if indent_next and expand_indent(line) == indent_level + 4:
+        pos = (start[0], indent[0] + 4)
+        if visual_indent:
+            code = "E129 visually indented line"
+        else:
+            code = "E125 continuation line"
+        yield pos, "%s with same indent as next logical line" % code
+
+
+@register_check
+def whitespace_before_parameters(logical_line, tokens):
+    r"""Avoid extraneous whitespace.
+
+    Avoid extraneous whitespace in the following situations:
+    - before the open parenthesis that starts the argument list of a
+      function call.
+    - before the open parenthesis that starts an indexing or slicing.
+
+    Okay: spam(1)
+    E211: spam (1)
+
+    Okay: dict['key'] = list[index]
+    E211: dict ['key'] = list[index]
+    E211: dict['key'] = list [index]
+    """
+    prev_type, prev_text, __, prev_end, __ = tokens[0]
+    for index in range(1, len(tokens)):
+        token_type, text, start, end, __ = tokens[index]
+        if (token_type == tokenize.OP and
+            text in '([' and
+            start != prev_end and
+            (prev_type == tokenize.NAME or prev_text in '}])') and
+            # Syntax "class A (B):" is allowed, but avoid it
+            (index < 2 or tokens[index - 2][1] != 'class') and
+                # Allow "return (a.foo for a in range(5))"
+                not keyword.iskeyword(prev_text)):
+            yield prev_end, "E211 whitespace before '%s'" % text
+        prev_type = token_type
+        prev_text = text
+        prev_end = end
+
+
+@register_check
+def whitespace_around_operator(logical_line):
+    r"""Avoid extraneous whitespace around an operator.
+
+    Okay: a = 12 + 3
+    E221: a = 4  + 5
+    E222: a = 4 +  5
+    E223: a = 4\t+ 5
+    E224: a = 4 +\t5
+    """
+    for match in OPERATOR_REGEX.finditer(logical_line):
+        before, after = match.groups()
+
+        if '\t' in before:
+            yield match.start(1), "E223 tab before operator"
+        elif len(before) > 1:
+            yield match.start(1), "E221 multiple spaces before operator"
+
+        if '\t' in after:
+            yield match.start(2), "E224 tab after operator"
+        elif len(after) > 1:
+            yield match.start(2), "E222 multiple spaces after operator"
+
+
+@register_check
+def missing_whitespace_around_operator(logical_line, tokens):
+    r"""Surround operators with a single space on either side.
+
+    - Always surround these binary operators with a single space on
+      either side: assignment (=), augmented assignment (+=, -= etc.),
+      comparisons (==, <, >, !=, <=, >=, in, not in, is, is not),
+      Booleans (and, or, not).
+
+    - If operators with different priorities are used, consider adding
+      whitespace around the operators with the lowest priorities.
+
+    Okay: i = i + 1
+    Okay: submitted += 1
+    Okay: x = x * 2 - 1
+    Okay: hypot2 = x * x + y * y
+    Okay: c = (a + b) * (a - b)
+    Okay: foo(bar, key='word', *args, **kwargs)
+    Okay: alpha[:-i]
+
+    E225: i=i+1
+    E225: submitted +=1
+    E225: x = x /2 - 1
+    E225: z = x **y
+    E225: z = 1and 1
+    E226: c = (a+b) * (a-b)
+    E226: hypot2 = x*x + y*y
+    E227: c = a|b
+    E228: msg = fmt%(errno, errmsg)
+    """
+    parens = 0
+    need_space = False
+    prev_type = tokenize.OP
+    prev_text = prev_end = None
+    operator_types = (tokenize.OP, tokenize.NAME)
+    for token_type, text, start, end, line in tokens:
+        if token_type in SKIP_COMMENTS:
+            continue
+        if text in ('(', 'lambda'):
+            parens += 1
+        elif text == ')':
+            parens -= 1
+        if need_space:
+            if start != prev_end:
+                # Found a (probably) needed space
+                if need_space is not True and not need_space[1]:
+                    yield (need_space[0],
+                           "E225 missing whitespace around operator")
+                need_space = False
+            elif text == '>' and prev_text in ('<', '-'):
+                # Tolerate the "<>" operator, even if running Python 3
+                # Deal with Python 3's annotated return value "->"
+                pass
+            elif (
+                    # def f(a, /, b):
+                    #           ^
+                    # def f(a, b, /):
+                    #              ^
+                    prev_text == '/' and text in {',', ')'} or
+                    # def f(a, b, /):
+                    #               ^
+                    prev_text == ')' and text == ':'
+            ):
+                # Tolerate the "/" operator in function definition
+                # For more info see PEP570
+                pass
+            else:
+                if need_space is True or need_space[1]:
+                    # A needed trailing space was not found
+                    yield prev_end, "E225 missing whitespace around operator"
+                elif prev_text != '**':
+                    code, optype = 'E226', 'arithmetic'
+                    if prev_text == '%':
+                        code, optype = 'E228', 'modulo'
+                    elif prev_text not in ARITHMETIC_OP:
+                        code, optype = 'E227', 'bitwise or shift'
+                    yield (need_space[0], "%s missing whitespace "
+                           "around %s operator" % (code, optype))
+                need_space = False
+        elif token_type in operator_types and prev_end is not None:
+            if text == '=' and parens:
+                # Allow keyword args or defaults: foo(bar=None).
+                pass
+            elif text in WS_NEEDED_OPERATORS:
+                need_space = True
+            elif text in UNARY_OPERATORS:
+                # Check if the operator is used as a binary operator
+                # Allow unary operators: -123, -x, +1.
+                # Allow argument unpacking: foo(*args, **kwargs).
+                if (prev_text in '}])' if prev_type == tokenize.OP
+                        else prev_text not in KEYWORDS):
+                    need_space = None
+            elif text in WS_OPTIONAL_OPERATORS:
+                need_space = None
+
+            if need_space is None:
+                # Surrounding space is optional, but ensure that
+                # trailing space matches opening space
+                need_space = (prev_end, start != prev_end)
+            elif need_space and start == prev_end:
+                # A needed opening space was not found
+                yield prev_end, "E225 missing whitespace around operator"
+                need_space = False
+        prev_type = token_type
+        prev_text = text
+        prev_end = end
+
+
+@register_check
+def whitespace_around_comma(logical_line):
+    r"""Avoid extraneous whitespace after a comma or a colon.
+
+    Note: these checks are disabled by default
+
+    Okay: a = (1, 2)
+    E241: a = (1,  2)
+    E242: a = (1,\t2)
+    """
+    line = logical_line
+    for m in WHITESPACE_AFTER_COMMA_REGEX.finditer(line):
+        found = m.start() + 1
+        if '\t' in m.group():
+            yield found, "E242 tab after '%s'" % m.group()[0]
+        else:
+            yield found, "E241 multiple spaces after '%s'" % m.group()[0]
+
+
+@register_check
+def whitespace_around_named_parameter_equals(logical_line, tokens):
+    r"""Don't use spaces around the '=' sign in function arguments.
+
+    Don't use spaces around the '=' sign when used to indicate a
+    keyword argument or a default parameter value, except when
+    using a type annotation.
+
+    Okay: def complex(real, imag=0.0):
+    Okay: return magic(r=real, i=imag)
+    Okay: boolean(a == b)
+    Okay: boolean(a != b)
+    Okay: boolean(a <= b)
+    Okay: boolean(a >= b)
+    Okay: def foo(arg: int = 42):
+    Okay: async def foo(arg: int = 42):
+
+    E251: def complex(real, imag = 0.0):
+    E251: return magic(r = real, i = imag)
+    E252: def complex(real, image: float=0.0):
+    """
+    parens = 0
+    no_space = False
+    require_space = False
+    prev_end = None
+    annotated_func_arg = False
+    in_def = bool(STARTSWITH_DEF_REGEX.match(logical_line))
+
+    message = "E251 unexpected spaces around keyword / parameter equals"
+    missing_message = "E252 missing whitespace around parameter equals"
+
+    for token_type, text, start, end, line in tokens:
+        if token_type == tokenize.NL:
+            continue
+        if no_space:
+            no_space = False
+            if start != prev_end:
+                yield (prev_end, message)
+        if require_space:
+            require_space = False
+            if start == prev_end:
+                yield (prev_end, missing_message)
+        if token_type == tokenize.OP:
+            if text in '([':
+                parens += 1
+            elif text in ')]':
+                parens -= 1
+            elif in_def and text == ':' and parens == 1:
+                annotated_func_arg = True
+            elif parens == 1 and text == ',':
+                annotated_func_arg = False
+            elif parens and text == '=':
+                if annotated_func_arg and parens == 1:
+                    require_space = True
+                    if start == prev_end:
+                        yield (prev_end, missing_message)
+                else:
+                    no_space = True
+                    if start != prev_end:
+                        yield (prev_end, message)
+            if not parens:
+                annotated_func_arg = False
+
+        prev_end = end
+
+
+@register_check
+def whitespace_before_comment(logical_line, tokens):
+    r"""Separate inline comments by at least two spaces.
+
+    An inline comment is a comment on the same line as a statement.
+    Inline comments should be separated by at least two spaces from the
+    statement. They should start with a # and a single space.
+
+    Each line of a block comment starts with a # and a single space
+    (unless it is indented text inside the comment).
+
+    Okay: x = x + 1  # Increment x
+    Okay: x = x + 1    # Increment x
+    Okay: # Block comment
+    E261: x = x + 1 # Increment x
+    E262: x = x + 1  #Increment x
+    E262: x = x + 1  #  Increment x
+    E265: #Block comment
+    E266: ### Block comment
+    """
+    prev_end = (0, 0)
+    for token_type, text, start, end, line in tokens:
+        if token_type == tokenize.COMMENT:
+            inline_comment = line[:start[1]].strip()
+            if inline_comment:
+                if prev_end[0] == start[0] and start[1] < prev_end[1] + 2:
+                    yield (prev_end,
+                           "E261 at least two spaces before inline comment")
+            symbol, sp, comment = text.partition(' ')
+            bad_prefix = symbol not in '#:' and (symbol.lstrip('#')[:1] or '#')
+            if inline_comment:
+                if bad_prefix or comment[:1] in WHITESPACE:
+                    yield start, "E262 inline comment should start with '# '"
+            elif bad_prefix and (bad_prefix != '!' or start[0] > 1):
+                if bad_prefix != '#':
+                    yield start, "E265 block comment should start with '# '"
+                elif comment:
+                    yield start, "E266 too many leading '#' for block comment"
+        elif token_type != tokenize.NL:
+            prev_end = end
+
+
+@register_check
+def imports_on_separate_lines(logical_line):
+    r"""Place imports on separate lines.
+
+    Okay: import os\nimport sys
+    E401: import sys, os
+
+    Okay: from subprocess import Popen, PIPE
+    Okay: from myclas import MyClass
+    Okay: from foo.bar.yourclass import YourClass
+    Okay: import myclass
+    Okay: import foo.bar.yourclass
+    """
+    line = logical_line
+    if line.startswith('import '):
+        found = line.find(',')
+        if -1 < found and ';' not in line[:found]:
+            yield found, "E401 multiple imports on one line"
+
+
+@register_check
+def module_imports_on_top_of_file(
+        logical_line, indent_level, checker_state, noqa):
+    r"""Place imports at the top of the file.
+
+    Always put imports at the top of the file, just after any module
+    comments and docstrings, and before module globals and constants.
+
+    Okay: import os
+    Okay: # this is a comment\nimport os
+    Okay: '''this is a module docstring'''\nimport os
+    Okay: r'''this is a module docstring'''\nimport os
+    Okay:
+    try:\n\timport x\nexcept ImportError:\n\tpass\nelse:\n\tpass\nimport y
+    Okay:
+    try:\n\timport x\nexcept ImportError:\n\tpass\nfinally:\n\tpass\nimport y
+    E402: a=1\nimport os
+    E402: 'One string'\n"Two string"\nimport os
+    E402: a=1\nfrom sys import x
+
+    Okay: if x:\n    import os
+    """  # noqa
+    def is_string_literal(line):
+        if line[0] in 'uUbB':
+            line = line[1:]
+        if line and line[0] in 'rR':
+            line = line[1:]
+        return line and (line[0] == '"' or line[0] == "'")
+
+    allowed_keywords = (
+        'try', 'except', 'else', 'finally', 'with', 'if', 'elif')
+
+    if indent_level:  # Allow imports in conditional statement/function
+        return
+    if not logical_line:  # Allow empty lines or comments
+        return
+    if noqa:
+        return
+    line = logical_line
+    if line.startswith('import ') or line.startswith('from '):
+        if checker_state.get('seen_non_imports', False):
+            yield 0, "E402 module level import not at top of file"
+    elif re.match(DUNDER_REGEX, line):
+        return
+    elif any(line.startswith(kw) for kw in allowed_keywords):
+        # Allow certain keywords intermixed with imports in order to
+        # support conditional or filtered importing
+        return
+    elif is_string_literal(line):
+        # The first literal is a docstring, allow it. Otherwise, report
+        # error.
+        if checker_state.get('seen_docstring', False):
+            checker_state['seen_non_imports'] = True
+        else:
+            checker_state['seen_docstring'] = True
+    else:
+        checker_state['seen_non_imports'] = True
+
+
+@register_check
+def compound_statements(logical_line):
+    r"""Compound statements (on the same line) are generally
+    discouraged.
+
+    While sometimes it's okay to put an if/for/while with a small body
+    on the same line, never do this for multi-clause statements.
+    Also avoid folding such long lines!
+
+    Always use a def statement instead of an assignment statement that
+    binds a lambda expression directly to a name.
+
+    Okay: if foo == 'blah':\n    do_blah_thing()
+    Okay: do_one()
+    Okay: do_two()
+    Okay: do_three()
+
+    E701: if foo == 'blah': do_blah_thing()
+    E701: for x in lst: total += x
+    E701: while t < 10: t = delay()
+    E701: if foo == 'blah': do_blah_thing()
+    E701: else: do_non_blah_thing()
+    E701: try: something()
+    E701: finally: cleanup()
+    E701: if foo == 'blah': one(); two(); three()
+    E702: do_one(); do_two(); do_three()
+    E703: do_four();  # useless semicolon
+    E704: def f(x): return 2*x
+    E731: f = lambda x: 2*x
+    """
+    line = logical_line
+    last_char = len(line) - 1
+    found = line.find(':')
+    prev_found = 0
+    counts = {char: 0 for char in '{}[]()'}
+    while -1 < found < last_char:
+        update_counts(line[prev_found:found], counts)
+        if ((counts['{'] <= counts['}'] and   # {'a': 1} (dict)
+             counts['['] <= counts[']'] and   # [1:2] (slice)
+             counts['('] <= counts[')']) and  # (annotation)
+            not (sys.version_info >= (3, 8) and
+                 line[found + 1] == '=')):  # assignment expression
+            lambda_kw = LAMBDA_REGEX.search(line, 0, found)
+            if lambda_kw:
+                before = line[:lambda_kw.start()].rstrip()
+                if before[-1:] == '=' and isidentifier(before[:-1].strip()):
+                    yield 0, ("E731 do not assign a lambda expression, use a "
+                              "def")
+                break
+            if STARTSWITH_DEF_REGEX.match(line):
+                yield 0, "E704 multiple statements on one line (def)"
+            elif STARTSWITH_INDENT_STATEMENT_REGEX.match(line):
+                yield found, "E701 multiple statements on one line (colon)"
+        prev_found = found
+        found = line.find(':', found + 1)
+    found = line.find(';')
+    while -1 < found:
+        if found < last_char:
+            yield found, "E702 multiple statements on one line (semicolon)"
+        else:
+            yield found, "E703 statement ends with a semicolon"
+        found = line.find(';', found + 1)
+
+
+@register_check
+def explicit_line_join(logical_line, tokens):
+    r"""Avoid explicit line join between brackets.
+
+    The preferred way of wrapping long lines is by using Python's
+    implied line continuation inside parentheses, brackets and braces.
+    Long lines can be broken over multiple lines by wrapping expressions
+    in parentheses.  These should be used in preference to using a
+    backslash for line continuation.
+
+    E502: aaa = [123, \\n       123]
+    E502: aaa = ("bbb " \\n       "ccc")
+
+    Okay: aaa = [123,\n       123]
+    Okay: aaa = ("bbb "\n       "ccc")
+    Okay: aaa = "bbb " \\n    "ccc"
+    Okay: aaa = 123  # \\
+    """
+    prev_start = prev_end = parens = 0
+    comment = False
+    backslash = None
+    for token_type, text, start, end, line in tokens:
+        if token_type == tokenize.COMMENT:
+            comment = True
+        if start[0] != prev_start and parens and backslash and not comment:
+            yield backslash, "E502 the backslash is redundant between brackets"
+        if end[0] != prev_end:
+            if line.rstrip('\r\n').endswith('\\'):
+                backslash = (end[0], len(line.splitlines()[-1]) - 1)
+            else:
+                backslash = None
+            prev_start = prev_end = end[0]
+        else:
+            prev_start = start[0]
+        if token_type == tokenize.OP:
+            if text in '([{':
+                parens += 1
+            elif text in ')]}':
+                parens -= 1
+
+
+_SYMBOLIC_OPS = frozenset("()[]{},:.;@=%~") | frozenset(("...",))
+
+
+def _is_binary_operator(token_type, text):
+    is_op_token = token_type == tokenize.OP
+    is_conjunction = text in ['and', 'or']
+    # NOTE(sigmavirus24): Previously the not_a_symbol check was executed
+    # conditionally. Since it is now *always* executed, text may be
+    # None. In that case we get a TypeError for `text not in str`.
+    not_a_symbol = text and text not in _SYMBOLIC_OPS
+    # The % character is strictly speaking a binary operator, but the
+    # common usage seems to be to put it next to the format parameters,
+    # after a line break.
+    return ((is_op_token or is_conjunction) and not_a_symbol)
+
+
+def _break_around_binary_operators(tokens):
+    """Private function to reduce duplication.
+
+    This factors out the shared details between
+    :func:`break_before_binary_operator` and
+    :func:`break_after_binary_operator`.
+    """
+    line_break = False
+    unary_context = True
+    # Previous non-newline token types and text
+    previous_token_type = None
+    previous_text = None
+    for token_type, text, start, end, line in tokens:
+        if token_type == tokenize.COMMENT:
+            continue
+        if ('\n' in text or '\r' in text) and token_type != tokenize.STRING:
+            line_break = True
+        else:
+            yield (token_type, text, previous_token_type, previous_text,
+                   line_break, unary_context, start)
+            unary_context = text in '([{,;'
+            line_break = False
+            previous_token_type = token_type
+            previous_text = text
+
+
+@register_check
+def break_before_binary_operator(logical_line, tokens):
+    r"""
+    Avoid breaks before binary operators.
+
+    The preferred place to break around a binary operator is after the
+    operator, not before it.
+
+    W503: (width == 0\n + height == 0)
+    W503: (width == 0\n and height == 0)
+    W503: var = (1\n       & ~2)
+    W503: var = (1\n       / -2)
+    W503: var = (1\n       + -1\n       + -2)
+
+    Okay: foo(\n    -x)
+    Okay: foo(x\n    [])
+    Okay: x = '''\n''' + ''
+    Okay: foo(x,\n    -y)
+    Okay: foo(x,  # comment\n    -y)
+    """
+    for context in _break_around_binary_operators(tokens):
+        (token_type, text, previous_token_type, previous_text,
+         line_break, unary_context, start) = context
+        if (_is_binary_operator(token_type, text) and line_break and
+                not unary_context and
+                not _is_binary_operator(previous_token_type,
+                                        previous_text)):
+            yield start, "W503 line break before binary operator"
+
+
+@register_check
+def break_after_binary_operator(logical_line, tokens):
+    r"""
+    Avoid breaks after binary operators.
+
+    The preferred place to break around a binary operator is before the
+    operator, not after it.
+
+    W504: (width == 0 +\n height == 0)
+    W504: (width == 0 and\n height == 0)
+    W504: var = (1 &\n       ~2)
+
+    Okay: foo(\n    -x)
+    Okay: foo(x\n    [])
+    Okay: x = '''\n''' + ''
+    Okay: x = '' + '''\n'''
+    Okay: foo(x,\n    -y)
+    Okay: foo(x,  # comment\n    -y)
+
+    The following should be W504 but unary_context is tricky with these
+    Okay: var = (1 /\n       -2)
+    Okay: var = (1 +\n       -1 +\n       -2)
+    """
+    prev_start = None
+    for context in _break_around_binary_operators(tokens):
+        (token_type, text, previous_token_type, previous_text,
+         line_break, unary_context, start) = context
+        if (_is_binary_operator(previous_token_type, previous_text) and
+                line_break and
+                not unary_context and
+                not _is_binary_operator(token_type, text)):
+            yield prev_start, "W504 line break after binary operator"
+        prev_start = start
+
+
+@register_check
+def comparison_to_singleton(logical_line, noqa):
+    r"""Comparison to singletons should use "is" or "is not".
+
+    Comparisons to singletons like None should always be done
+    with "is" or "is not", never the equality operators.
+
+    Okay: if arg is not None:
+    E711: if arg != None:
+    E711: if None == arg:
+    E712: if arg == True:
+    E712: if False == arg:
+
+    Also, beware of writing if x when you really mean if x is not None
+    -- e.g. when testing whether a variable or argument that defaults to
+    None was set to some other value.  The other value might have a type
+    (such as a container) that could be false in a boolean context!
+    """
+    match = not noqa and COMPARE_SINGLETON_REGEX.search(logical_line)
+    if match:
+        singleton = match.group(1) or match.group(3)
+        same = (match.group(2) == '==')
+
+        msg = "'if cond is %s:'" % (('' if same else 'not ') + singleton)
+        if singleton in ('None',):
+            code = 'E711'
+        else:
+            code = 'E712'
+            nonzero = ((singleton == 'True' and same) or
+                       (singleton == 'False' and not same))
+            msg += " or 'if %scond:'" % ('' if nonzero else 'not ')
+        yield match.start(2), ("%s comparison to %s should be %s" %
+                               (code, singleton, msg))
+
+
+@register_check
+def comparison_negative(logical_line):
+    r"""Negative comparison should be done using "not in" and "is not".
+
+    Okay: if x not in y:\n    pass
+    Okay: assert (X in Y or X is Z)
+    Okay: if not (X in Y):\n    pass
+    Okay: zz = x is not y
+    E713: Z = not X in Y
+    E713: if not X.B in Y:\n    pass
+    E714: if not X is Y:\n    pass
+    E714: Z = not X.B is Y
+    """
+    match = COMPARE_NEGATIVE_REGEX.search(logical_line)
+    if match:
+        pos = match.start(1)
+        if match.group(2) == 'in':
+            yield pos, "E713 test for membership should be 'not in'"
+        else:
+            yield pos, "E714 test for object identity should be 'is not'"
+
+
+@register_check
+def comparison_type(logical_line, noqa):
+    r"""Object type comparisons should always use isinstance().
+
+    Do not compare types directly.
+
+    Okay: if isinstance(obj, int):
+    E721: if type(obj) is type(1):
+
+    When checking if an object is a string, keep in mind that it might
+    be a unicode string too! In Python 2.3, str and unicode have a
+    common base class, basestring, so you can do:
+
+    Okay: if isinstance(obj, basestring):
+    Okay: if type(a1) is type(b1):
+    """
+    match = COMPARE_TYPE_REGEX.search(logical_line)
+    if match and not noqa:
+        inst = match.group(1)
+        if inst and isidentifier(inst) and inst not in SINGLETONS:
+            return  # Allow comparison for types which are not obvious
+        yield match.start(), "E721 do not compare types, use 'isinstance()'"
+
+
+@register_check
+def bare_except(logical_line, noqa):
+    r"""When catching exceptions, mention specific exceptions when
+    possible.
+
+    Okay: except Exception:
+    Okay: except BaseException:
+    E722: except:
+    """
+    if noqa:
+        return
+
+    regex = re.compile(r"except\s*:")
+    match = regex.match(logical_line)
+    if match:
+        yield match.start(), "E722 do not use bare 'except'"
+
+
+@register_check
+def ambiguous_identifier(logical_line, tokens):
+    r"""Never use the characters 'l', 'O', or 'I' as variable names.
+
+    In some fonts, these characters are indistinguishable from the
+    numerals one and zero. When tempted to use 'l', use 'L' instead.
+
+    Okay: L = 0
+    Okay: o = 123
+    Okay: i = 42
+    E741: l = 0
+    E741: O = 123
+    E741: I = 42
+
+    Variables can be bound in several other contexts, including class
+    and function definitions, 'global' and 'nonlocal' statements,
+    exception handlers, and 'with' and 'for' statements.
+    In addition, we have a special handling for function parameters.
+
+    Okay: except AttributeError as o:
+    Okay: with lock as L:
+    Okay: foo(l=12)
+    Okay: for a in foo(l=12):
+    E741: except AttributeError as O:
+    E741: with lock as l:
+    E741: global I
+    E741: nonlocal l
+    E741: def foo(l):
+    E741: def foo(l=12):
+    E741: l = foo(l=12)
+    E741: for l in range(10):
+    E742: class I(object):
+    E743: def l(x):
+    """
+    is_func_def = False  # Set to true if 'def' is found
+    parameter_parentheses_level = 0
+    idents_to_avoid = ('l', 'O', 'I')
+    prev_type, prev_text, prev_start, prev_end, __ = tokens[0]
+    for token_type, text, start, end, line in tokens[1:]:
+        ident = pos = None
+        # find function definitions
+        if prev_text == 'def':
+            is_func_def = True
+        # update parameter parentheses level
+        if parameter_parentheses_level == 0 and \
+                prev_type == tokenize.NAME and \
+                token_type == tokenize.OP and text == '(':
+            parameter_parentheses_level = 1
+        elif parameter_parentheses_level > 0 and \
+                token_type == tokenize.OP:
+            if text == '(':
+                parameter_parentheses_level += 1
+            elif text == ')':
+                parameter_parentheses_level -= 1
+        # identifiers on the lhs of an assignment operator
+        if token_type == tokenize.OP and '=' in text and \
+                parameter_parentheses_level == 0:
+            if prev_text in idents_to_avoid:
+                ident = prev_text
+                pos = prev_start
+        # identifiers bound to values with 'as', 'for',
+        # 'global', or 'nonlocal'
+        if prev_text in ('as', 'for', 'global', 'nonlocal'):
+            if text in idents_to_avoid:
+                ident = text
+                pos = start
+        # function parameter definitions
+        if is_func_def:
+            if text in idents_to_avoid:
+                ident = text
+                pos = start
+        if prev_text == 'class':
+            if text in idents_to_avoid:
+                yield start, "E742 ambiguous class definition '%s'" % text
+        if prev_text == 'def':
+            if text in idents_to_avoid:
+                yield start, "E743 ambiguous function definition '%s'" % text
+        if ident:
+            yield pos, "E741 ambiguous variable name '%s'" % ident
+        prev_type = token_type
+        prev_text = text
+        prev_start = start
+
+
+@register_check
+def python_3000_has_key(logical_line, noqa):
+    r"""The {}.has_key() method is removed in Python 3: use the 'in'
+    operator.
+
+    Okay: if "alph" in d:\n    print d["alph"]
+    W601: assert d.has_key('alph')
+    """
+    pos = logical_line.find('.has_key(')
+    if pos > -1 and not noqa:
+        yield pos, "W601 .has_key() is deprecated, use 'in'"
+
+
+@register_check
+def python_3000_raise_comma(logical_line):
+    r"""When raising an exception, use "raise ValueError('message')".
+
+    The older form is removed in Python 3.
+
+    Okay: raise DummyError("Message")
+    W602: raise DummyError, "Message"
+    """
+    match = RAISE_COMMA_REGEX.match(logical_line)
+    if match and not RERAISE_COMMA_REGEX.match(logical_line):
+        yield match.end() - 1, "W602 deprecated form of raising exception"
+
+
+@register_check
+def python_3000_not_equal(logical_line):
+    r"""New code should always use != instead of <>.
+
+    The older syntax is removed in Python 3.
+
+    Okay: if a != 'no':
+    W603: if a <> 'no':
+    """
+    pos = logical_line.find('<>')
+    if pos > -1:
+        yield pos, "W603 '<>' is deprecated, use '!='"
+
+
+@register_check
+def python_3000_backticks(logical_line):
+    r"""Use repr() instead of backticks in Python 3.
+
+    Okay: val = repr(1 + 2)
+    W604: val = `1 + 2`
+    """
+    pos = logical_line.find('`')
+    if pos > -1:
+        yield pos, "W604 backticks are deprecated, use 'repr()'"
+
+
+@register_check
+def python_3000_invalid_escape_sequence(logical_line, tokens, noqa):
+    r"""Invalid escape sequences are deprecated in Python 3.6.
+
+    Okay: regex = r'\.png$'
+    W605: regex = '\.png$'
+    """
+    if noqa:
+        return
+
+    # https://docs.python.org/3/reference/lexical_analysis.html#string-and-bytes-literals
+    valid = [
+        '\n',
+        '\\',
+        '\'',
+        '"',
+        'a',
+        'b',
+        'f',
+        'n',
+        'r',
+        't',
+        'v',
+        '0', '1', '2', '3', '4', '5', '6', '7',
+        'x',
+
+        # Escape sequences only recognized in string literals
+        'N',
+        'u',
+        'U',
+    ]
+
+    for token_type, text, start, end, line in tokens:
+        if token_type == tokenize.STRING:
+            start_line, start_col = start
+            quote = text[-3:] if text[-3:] in ('"""', "'''") else text[-1]
+            # Extract string modifiers (e.g. u or r)
+            quote_pos = text.index(quote)
+            prefix = text[:quote_pos].lower()
+            start = quote_pos + len(quote)
+            string = text[start:-len(quote)]
+
+            if 'r' not in prefix:
+                pos = string.find('\\')
+                while pos >= 0:
+                    pos += 1
+                    if string[pos] not in valid:
+                        line = start_line + string.count('\n', 0, pos)
+                        if line == start_line:
+                            col = start_col + len(prefix) + len(quote) + pos
+                        else:
+                            col = pos - string.rfind('\n', 0, pos) - 1
+                        yield (
+                            (line, col - 1),
+                            "W605 invalid escape sequence '\\%s'" %
+                            string[pos],
+                        )
+                    pos = string.find('\\', pos + 1)
+
+
+@register_check
+def python_3000_async_await_keywords(logical_line, tokens):
+    """'async' and 'await' are reserved keywords starting at Python 3.7.
+
+    W606: async = 42
+    W606: await = 42
+    Okay: async def read(db):\n    data = await db.fetch('SELECT ...')
+    """
+    # The Python tokenize library before Python 3.5 recognizes
+    # async/await as a NAME token. Therefore, use a state machine to
+    # look for the possible async/await constructs as defined by the
+    # Python grammar:
+    # https://docs.python.org/3/reference/grammar.html
+
+    state = None
+    for token_type, text, start, end, line in tokens:
+        error = False
+
+        if token_type == tokenize.NL:
+            continue
+
+        if state is None:
+            if token_type == tokenize.NAME:
+                if text == 'async':
+                    state = ('async_stmt', start)
+                elif text == 'await':
+                    state = ('await', start)
+                elif (token_type == tokenize.NAME and
+                      text in ('def', 'for')):
+                    state = ('define', start)
+
+        elif state[0] == 'async_stmt':
+            if token_type == tokenize.NAME and text in ('def', 'with', 'for'):
+                # One of funcdef, with_stmt, or for_stmt. Return to
+                # looking for async/await names.
+                state = None
+            else:
+                error = True
+        elif state[0] == 'await':
+            if token_type == tokenize.NAME:
+                # An await expression. Return to looking for async/await
+                # names.
+                state = None
+            elif token_type == tokenize.OP and text == '(':
+                state = None
+            else:
+                error = True
+        elif state[0] == 'define':
+            if token_type == tokenize.NAME and text in ('async', 'await'):
+                error = True
+            else:
+                state = None
+
+        if error:
+            yield (
+                state[1],
+                "W606 'async' and 'await' are reserved keywords starting with "
+                "Python 3.7",
+            )
+            state = None
+
+    # Last token
+    if state is not None:
+        yield (
+            state[1],
+            "W606 'async' and 'await' are reserved keywords starting with "
+            "Python 3.7",
+        )
+
+
+########################################################################
+@register_check
+def maximum_doc_length(logical_line, max_doc_length, noqa, tokens):
+    r"""Limit all doc lines to a maximum of 72 characters.
+
+    For flowing long blocks of text (docstrings or comments), limiting
+    the length to 72 characters is recommended.
+
+    Reports warning W505
+    """
+    if max_doc_length is None or noqa:
+        return
+
+    prev_token = None
+    skip_lines = set()
+    # Skip lines that
+    for token_type, text, start, end, line in tokens:
+        if token_type not in SKIP_COMMENTS.union([tokenize.STRING]):
+            skip_lines.add(line)
+
+    for token_type, text, start, end, line in tokens:
+        # Skip lines that aren't pure strings
+        if token_type == tokenize.STRING and skip_lines:
+            continue
+        if token_type in (tokenize.STRING, tokenize.COMMENT):
+            # Only check comment-only lines
+            if prev_token is None or prev_token in SKIP_TOKENS:
+                lines = line.splitlines()
+                for line_num, physical_line in enumerate(lines):
+                    if hasattr(physical_line, 'decode'):  # Python 2
+                        # The line could contain multi-byte characters
+                        try:
+                            physical_line = physical_line.decode('utf-8')
+                        except UnicodeError:
+                            pass
+                    if start[0] + line_num == 1 and line.startswith('#!'):
+                        return
+                    length = len(physical_line)
+                    chunks = physical_line.split()
+                    if token_type == tokenize.COMMENT:
+                        if (len(chunks) == 2 and
+                                length - len(chunks[-1]) < MAX_DOC_LENGTH):
+                            continue
+                    if len(chunks) == 1 and line_num + 1 < len(lines):
+                        if (len(chunks) == 1 and
+                                length - len(chunks[-1]) < MAX_DOC_LENGTH):
+                            continue
+                    if length > max_doc_length:
+                        doc_error = (start[0] + line_num, max_doc_length)
+                        yield (doc_error, "W505 doc line too long "
+                                          "(%d > %d characters)"
+                               % (length, max_doc_length))
+        prev_token = token_type
+
+
+########################################################################
+# Helper functions
+########################################################################
+
+
+if sys.version_info < (3,):
+    # Python 2: implicit encoding.
+    def readlines(filename):
+        """Read the source code."""
+        with open(filename, 'rU') as f:
+            return f.readlines()
+    isidentifier = re.compile(r'[a-zA-Z_]\w*$').match
+    stdin_get_value = sys.stdin.read
+else:
+    # Python 3
+    def readlines(filename):
+        """Read the source code."""
+        try:
+            with open(filename, 'rb') as f:
+                (coding, lines) = tokenize.detect_encoding(f.readline)
+                f = TextIOWrapper(f, coding, line_buffering=True)
+                return [line.decode(coding) for line in lines] + f.readlines()
+        except (LookupError, SyntaxError, UnicodeError):
+            # Fall back if file encoding is improperly declared
+            with open(filename, encoding='latin-1') as f:
+                return f.readlines()
+    isidentifier = str.isidentifier
+
+    def stdin_get_value():
+        """Read the value from stdin."""
+        return TextIOWrapper(sys.stdin.buffer, errors='ignore').read()
+
+noqa = lru_cache(512)(re.compile(r'# no(?:qa|pep8)\b', re.I).search)
+
+
+def expand_indent(line):
+    r"""Return the amount of indentation.
+
+    Tabs are expanded to the next multiple of 8.
+
+    >>> expand_indent('    ')
+    4
+    >>> expand_indent('\t')
+    8
+    >>> expand_indent('       \t')
+    8
+    >>> expand_indent('        \t')
+    16
+    """
+    line = line.rstrip('\n\r')
+    if '\t' not in line:
+        return len(line) - len(line.lstrip())
+    result = 0
+    for char in line:
+        if char == '\t':
+            result = result // 8 * 8 + 8
+        elif char == ' ':
+            result += 1
+        else:
+            break
+    return result
+
+
+def mute_string(text):
+    """Replace contents with 'xxx' to prevent syntax matching.
+
+    >>> mute_string('"abc"')
+    '"xxx"'
+    >>> mute_string("'''abc'''")
+    "'''xxx'''"
+    >>> mute_string("r'abc'")
+    "r'xxx'"
+    """
+    # String modifiers (e.g. u or r)
+    start = text.index(text[-1]) + 1
+    end = len(text) - 1
+    # Triple quotes
+    if text[-3:] in ('"""', "'''"):
+        start += 2
+        end -= 2
+    return text[:start] + 'x' * (end - start) + text[end:]
+
+
+def parse_udiff(diff, patterns=None, parent='.'):
+    """Return a dictionary of matching lines."""
+    # For each file of the diff, the entry key is the filename,
+    # and the value is a set of row numbers to consider.
+    rv = {}
+    path = nrows = None
+    for line in diff.splitlines():
+        if nrows:
+            if line[:1] != '-':
+                nrows -= 1
+            continue
+        if line[:3] == '@@ ':
+            hunk_match = HUNK_REGEX.match(line)
+            (row, nrows) = [int(g or '1') for g in hunk_match.groups()]
+            rv[path].update(range(row, row + nrows))
+        elif line[:3] == '+++':
+            path = line[4:].split('\t', 1)[0]
+            # Git diff will use (i)ndex, (w)ork tree, (c)ommit and
+            # (o)bject instead of a/b/c/d as prefixes for patches
+            if path[:2] in ('b/', 'w/', 'i/'):
+                path = path[2:]
+            rv[path] = set()
+    return {
+        os.path.join(parent, filepath): rows
+        for (filepath, rows) in rv.items()
+        if rows and filename_match(filepath, patterns)
+    }
+
+
+def normalize_paths(value, parent=os.curdir):
+    """Parse a comma-separated list of paths.
+
+    Return a list of absolute paths.
+    """
+    if not value:
+        return []
+    if isinstance(value, list):
+        return value
+    paths = []
+    for path in value.split(','):
+        path = path.strip()
+        if '/' in path:
+            path = os.path.abspath(os.path.join(parent, path))
+        paths.append(path.rstrip('/'))
+    return paths
+
+
+def filename_match(filename, patterns, default=True):
+    """Check if patterns contains a pattern that matches filename.
+
+    If patterns is unspecified, this always returns True.
+    """
+    if not patterns:
+        return default
+    return any(fnmatch(filename, pattern) for pattern in patterns)
+
+
+def update_counts(s, counts):
+    r"""Adds one to the counts of each appearance of characters in s,
+        for characters in counts"""
+    for char in s:
+        if char in counts:
+            counts[char] += 1
+
+
+def _is_eol_token(token):
+    return token[0] in NEWLINE or token[4][token[3][1]:].lstrip() == '\\\n'
+
+
+########################################################################
+# Framework to run all checks
+########################################################################
+
+
+class Checker(object):
+    """Load a Python source file, tokenize it, check coding style."""
+
+    def __init__(self, filename=None, lines=None,
+                 options=None, report=None, **kwargs):
+        if options is None:
+            options = StyleGuide(kwargs).options
+        else:
+            assert not kwargs
+        self._io_error = None
+        self._physical_checks = options.physical_checks
+        self._logical_checks = options.logical_checks
+        self._ast_checks = options.ast_checks
+        self.max_line_length = options.max_line_length
+        self.max_doc_length = options.max_doc_length
+        self.multiline = False  # in a multiline string?
+        self.hang_closing = options.hang_closing
+        self.verbose = options.verbose
+        self.filename = filename
+        # Dictionary where a checker can store its custom state.
+        self._checker_states = {}
+        if filename is None:
+            self.filename = 'stdin'
+            self.lines = lines or []
+        elif filename == '-':
+            self.filename = 'stdin'
+            self.lines = stdin_get_value().splitlines(True)
+        elif lines is None:
+            try:
+                self.lines = readlines(filename)
+            except IOError:
+                (exc_type, exc) = sys.exc_info()[:2]
+                self._io_error = '%s: %s' % (exc_type.__name__, exc)
+                self.lines = []
+        else:
+            self.lines = lines
+        if self.lines:
+            ord0 = ord(self.lines[0][0])
+            if ord0 in (0xef, 0xfeff):  # Strip the UTF-8 BOM
+                if ord0 == 0xfeff:
+                    self.lines[0] = self.lines[0][1:]
+                elif self.lines[0][:3] == '\xef\xbb\xbf':
+                    self.lines[0] = self.lines[0][3:]
+        self.report = report or options.report
+        self.report_error = self.report.error
+        self.noqa = False
+
+    def report_invalid_syntax(self):
+        """Check if the syntax is valid."""
+        (exc_type, exc) = sys.exc_info()[:2]
+        if len(exc.args) > 1:
+            offset = exc.args[1]
+            if len(offset) > 2:
+                offset = offset[1:3]
+        else:
+            offset = (1, 0)
+        self.report_error(offset[0], offset[1] or 0,
+                          'E901 %s: %s' % (exc_type.__name__, exc.args[0]),
+                          self.report_invalid_syntax)
+
+    def readline(self):
+        """Get the next line from the input buffer."""
+        if self.line_number >= self.total_lines:
+            return ''
+        line = self.lines[self.line_number]
+        self.line_number += 1
+        if self.indent_char is None and line[:1] in WHITESPACE:
+            self.indent_char = line[0]
+        return line
+
+    def run_check(self, check, argument_names):
+        """Run a check plugin."""
+        arguments = []
+        for name in argument_names:
+            arguments.append(getattr(self, name))
+        return check(*arguments)
+
+    def init_checker_state(self, name, argument_names):
+        """Prepare custom state for the specific checker plugin."""
+        if 'checker_state' in argument_names:
+            self.checker_state = self._checker_states.setdefault(name, {})
+
+    def check_physical(self, line):
+        """Run all physical checks on a raw input line."""
+        self.physical_line = line
+        for name, check, argument_names in self._physical_checks:
+            self.init_checker_state(name, argument_names)
+            result = self.run_check(check, argument_names)
+            if result is not None:
+                (offset, text) = result
+                self.report_error(self.line_number, offset, text, check)
+                if text[:4] == 'E101':
+                    self.indent_char = line[0]
+
+    def build_tokens_line(self):
+        """Build a logical line from tokens."""
+        logical = []
+        comments = []
+        length = 0
+        prev_row = prev_col = mapping = None
+        for token_type, text, start, end, line in self.tokens:
+            if token_type in SKIP_TOKENS:
+                continue
+            if not mapping:
+                mapping = [(0, start)]
+            if token_type == tokenize.COMMENT:
+                comments.append(text)
+                continue
+            if token_type == tokenize.STRING:
+                text = mute_string(text)
+            if prev_row:
+                (start_row, start_col) = start
+                if prev_row != start_row:    # different row
+                    prev_text = self.lines[prev_row - 1][prev_col - 1]
+                    if prev_text == ',' or (prev_text not in '{[(' and
+                                            text not in '}])'):
+                        text = ' ' + text
+                elif prev_col != start_col:  # different column
+                    text = line[prev_col:start_col] + text
+            logical.append(text)
+            length += len(text)
+            mapping.append((length, end))
+            (prev_row, prev_col) = end
+        self.logical_line = ''.join(logical)
+        self.noqa = comments and noqa(''.join(comments))
+        return mapping
+
+    def check_logical(self):
+        """Build a line from tokens and run all logical checks on it."""
+        self.report.increment_logical_line()
+        mapping = self.build_tokens_line()
+        if not mapping:
+            return
+
+        mapping_offsets = [offset for offset, _ in mapping]
+        (start_row, start_col) = mapping[0][1]
+        start_line = self.lines[start_row - 1]
+        self.indent_level = expand_indent(start_line[:start_col])
+        if self.blank_before < self.blank_lines:
+            self.blank_before = self.blank_lines
+        if self.verbose >= 2:
+            print(self.logical_line[:80].rstrip())
+        for name, check, argument_names in self._logical_checks:
+            if self.verbose >= 4:
+                print('   ' + name)
+            self.init_checker_state(name, argument_names)
+            for offset, text in self.run_check(check, argument_names) or ():
+                if not isinstance(offset, tuple):
+                    # As mappings are ordered, bisecting is a fast way
+                    # to find a given offset in them.
+                    token_offset, pos = mapping[bisect.bisect_left(
+                        mapping_offsets, offset)]
+                    offset = (pos[0], pos[1] + offset - token_offset)
+                self.report_error(offset[0], offset[1], text, check)
+        if self.logical_line:
+            self.previous_indent_level = self.indent_level
+            self.previous_logical = self.logical_line
+            if not self.indent_level:
+                self.previous_unindented_logical_line = self.logical_line
+        self.blank_lines = 0
+        self.tokens = []
+
+    def check_ast(self):
+        """Build the file's AST and run all AST checks."""
+        try:
+            tree = compile(''.join(self.lines), '', 'exec', PyCF_ONLY_AST)
+        except (ValueError, SyntaxError, TypeError):
+            return self.report_invalid_syntax()
+        for name, cls, __ in self._ast_checks:
+            checker = cls(tree, self.filename)
+            for lineno, offset, text, check in checker.run():
+                if not self.lines or not noqa(self.lines[lineno - 1]):
+                    self.report_error(lineno, offset, text, check)
+
+    def generate_tokens(self):
+        """Tokenize file, run physical line checks and yield tokens."""
+        if self._io_error:
+            self.report_error(1, 0, 'E902 %s' % self._io_error, readlines)
+        tokengen = tokenize.generate_tokens(self.readline)
+        try:
+            for token in tokengen:
+                if token[2][0] > self.total_lines:
+                    return
+                self.noqa = token[4] and noqa(token[4])
+                self.maybe_check_physical(token)
+                yield token
+        except (SyntaxError, tokenize.TokenError):
+            self.report_invalid_syntax()
+
+    def maybe_check_physical(self, token):
+        """If appropriate for token, check current physical line(s)."""
+        # Called after every token, but act only on end of line.
+        if _is_eol_token(token):
+            # Obviously, a newline token ends a single physical line.
+            self.check_physical(token[4])
+        elif token[0] == tokenize.STRING and '\n' in token[1]:
+            # Less obviously, a string that contains newlines is a
+            # multiline string, either triple-quoted or with internal
+            # newlines backslash-escaped. Check every physical line in
+            # the string *except* for the last one: its newline is
+            # outside of the multiline string, so we consider it a
+            # regular physical line, and will check it like any other
+            # physical line.
+            #
+            # Subtleties:
+            # - we don't *completely* ignore the last line; if it
+            #   contains the magical "# noqa" comment, we disable all
+            #   physical checks for the entire multiline string
+            # - have to wind self.line_number back because initially it
+            #   points to the last line of the string, and we want
+            #   check_physical() to give accurate feedback
+            if noqa(token[4]):
+                return
+            self.multiline = True
+            self.line_number = token[2][0]
+            _, src, (_, offset), _, _ = token
+            src = self.lines[self.line_number - 1][:offset] + src
+            for line in src.split('\n')[:-1]:
+                self.check_physical(line + '\n')
+                self.line_number += 1
+            self.multiline = False
+
+    def check_all(self, expected=None, line_offset=0):
+        """Run all checks on the input file."""
+        self.report.init_file(self.filename, self.lines, expected, line_offset)
+        self.total_lines = len(self.lines)
+        if self._ast_checks:
+            self.check_ast()
+        self.line_number = 0
+        self.indent_char = None
+        self.indent_level = self.previous_indent_level = 0
+        self.previous_logical = ''
+        self.previous_unindented_logical_line = ''
+        self.tokens = []
+        self.blank_lines = self.blank_before = 0
+        parens = 0
+        for token in self.generate_tokens():
+            self.tokens.append(token)
+            token_type, text = token[0:2]
+            if self.verbose >= 3:
+                if token[2][0] == token[3][0]:
+                    pos = '[%s:%s]' % (token[2][1] or '', token[3][1])
+                else:
+                    pos = 'l.%s' % token[3][0]
+                print('l.%s\t%s\t%s\t%r' %
+                      (token[2][0], pos, tokenize.tok_name[token[0]], text))
+            if token_type == tokenize.OP:
+                if text in '([{':
+                    parens += 1
+                elif text in '}])':
+                    parens -= 1
+            elif not parens:
+                if token_type in NEWLINE:
+                    if token_type == tokenize.NEWLINE:
+                        self.check_logical()
+                        self.blank_before = 0
+                    elif len(self.tokens) == 1:
+                        # The physical line contains only this token.
+                        self.blank_lines += 1
+                        del self.tokens[0]
+                    else:
+                        self.check_logical()
+        if self.tokens:
+            self.check_physical(self.lines[-1])
+            self.check_logical()
+        return self.report.get_file_results()
+
+
+class BaseReport(object):
+    """Collect the results of the checks."""
+
+    print_filename = False
+
+    def __init__(self, options):
+        self._benchmark_keys = options.benchmark_keys
+        self._ignore_code = options.ignore_code
+        # Results
+        self.elapsed = 0
+        self.total_errors = 0
+        self.counters = dict.fromkeys(self._benchmark_keys, 0)
+        self.messages = {}
+
+    def start(self):
+        """Start the timer."""
+        self._start_time = time.time()
+
+    def stop(self):
+        """Stop the timer."""
+        self.elapsed = time.time() - self._start_time
+
+    def init_file(self, filename, lines, expected, line_offset):
+        """Signal a new file."""
+        self.filename = filename
+        self.lines = lines
+        self.expected = expected or ()
+        self.line_offset = line_offset
+        self.file_errors = 0
+        self.counters['files'] += 1
+        self.counters['physical lines'] += len(lines)
+
+    def increment_logical_line(self):
+        """Signal a new logical line."""
+        self.counters['logical lines'] += 1
+
+    def error(self, line_number, offset, text, check):
+        """Report an error, according to options."""
+        code = text[:4]
+        if self._ignore_code(code):
+            return
+        if code in self.counters:
+            self.counters[code] += 1
+        else:
+            self.counters[code] = 1
+            self.messages[code] = text[5:]
+        # Don't care about expected errors or warnings
+        if code in self.expected:
+            return
+        if self.print_filename and not self.file_errors:
+            print(self.filename)
+        self.file_errors += 1
+        self.total_errors += 1
+        return code
+
+    def get_file_results(self):
+        """Return the count of errors and warnings for this file."""
+        return self.file_errors
+
+    def get_count(self, prefix=''):
+        """Return the total count of errors and warnings."""
+        return sum(self.counters[key]
+                   for key in self.messages if key.startswith(prefix))
+
+    def get_statistics(self, prefix=''):
+        """Get statistics for message codes that start with the prefix.
+
+        prefix='' matches all errors and warnings
+        prefix='E' matches all errors
+        prefix='W' matches all warnings
+        prefix='E4' matches all errors that have to do with imports
+        """
+        return ['%-7s %s %s' % (self.counters[key], key, self.messages[key])
+                for key in sorted(self.messages) if key.startswith(prefix)]
+
+    def print_statistics(self, prefix=''):
+        """Print overall statistics (number of errors and warnings)."""
+        for line in self.get_statistics(prefix):
+            print(line)
+
+    def print_benchmark(self):
+        """Print benchmark numbers."""
+        print('%-7.2f %s' % (self.elapsed, 'seconds elapsed'))
+        if self.elapsed:
+            for key in self._benchmark_keys:
+                print('%-7d %s per second (%d total)' %
+                      (self.counters[key] / self.elapsed, key,
+                       self.counters[key]))
+
+
+class FileReport(BaseReport):
+    """Collect the results of the checks and print the filenames."""
+
+    print_filename = True
+
+
+class StandardReport(BaseReport):
+    """Collect and print the results of the checks."""
+
+    def __init__(self, options):
+        super(StandardReport, self).__init__(options)
+        self._fmt = REPORT_FORMAT.get(options.format.lower(),
+                                      options.format)
+        self._repeat = options.repeat
+        self._show_source = options.show_source
+        self._show_pep8 = options.show_pep8
+
+    def init_file(self, filename, lines, expected, line_offset):
+        """Signal a new file."""
+        self._deferred_print = []
+        return super(StandardReport, self).init_file(
+            filename, lines, expected, line_offset)
+
+    def error(self, line_number, offset, text, check):
+        """Report an error, according to options."""
+        code = super(StandardReport, self).error(line_number, offset,
+                                                 text, check)
+        if code and (self.counters[code] == 1 or self._repeat):
+            self._deferred_print.append(
+                (line_number, offset, code, text[5:], check.__doc__))
+        return code
+
+    def get_file_results(self):
+        """Print results and return the overall count for this file."""
+        self._deferred_print.sort()
+        for line_number, offset, code, text, doc in self._deferred_print:
+            print(self._fmt % {
+                'path': self.filename,
+                'row': self.line_offset + line_number, 'col': offset + 1,
+                'code': code, 'text': text,
+            })
+            if self._show_source:
+                if line_number > len(self.lines):
+                    line = ''
+                else:
+                    line = self.lines[line_number - 1]
+                print(line.rstrip())
+                print(re.sub(r'\S', ' ', line[:offset]) + '^')
+            if self._show_pep8 and doc:
+                print('    ' + doc.strip())
+
+            # stdout is block buffered when not stdout.isatty().
+            # line can be broken where buffer boundary since other
+            # processes write to same file.
+            # flush() after print() to avoid buffer boundary.
+            # Typical buffer size is 8192. line written safely when
+            # len(line) < 8192.
+            sys.stdout.flush()
+        return self.file_errors
+
+
+class DiffReport(StandardReport):
+    """Collect and print the results for the changed lines only."""
+
+    def __init__(self, options):
+        super(DiffReport, self).__init__(options)
+        self._selected = options.selected_lines
+
+    def error(self, line_number, offset, text, check):
+        if line_number not in self._selected[self.filename]:
+            return
+        return super(DiffReport, self).error(line_number, offset, text, check)
+
+
+class StyleGuide(object):
+    """Initialize a PEP-8 instance with few options."""
+
+    def __init__(self, *args, **kwargs):
+        # build options from the command line
+        self.checker_class = kwargs.pop('checker_class', Checker)
+        parse_argv = kwargs.pop('parse_argv', False)
+        config_file = kwargs.pop('config_file', False)
+        parser = kwargs.pop('parser', None)
+        # build options from dict
+        options_dict = dict(*args, **kwargs)
+        arglist = None if parse_argv else options_dict.get('paths', None)
+        verbose = options_dict.get('verbose', None)
+        options, self.paths = process_options(
+            arglist, parse_argv, config_file, parser, verbose)
+        if options_dict:
+            options.__dict__.update(options_dict)
+            if 'paths' in options_dict:
+                self.paths = options_dict['paths']
+
+        self.runner = self.input_file
+        self.options = options
+
+        if not options.reporter:
+            options.reporter = BaseReport if options.quiet else StandardReport
+
+        options.select = tuple(options.select or ())
+        if not (options.select or options.ignore or
+                options.testsuite or options.doctest) and DEFAULT_IGNORE:
+            # The default choice: ignore controversial checks
+            options.ignore = tuple(DEFAULT_IGNORE.split(','))
+        else:
+            # Ignore all checks which are not explicitly selected
+            options.ignore = ('',) if options.select else tuple(options.ignore)
+        options.benchmark_keys = BENCHMARK_KEYS[:]
+        options.ignore_code = self.ignore_code
+        options.physical_checks = self.get_checks('physical_line')
+        options.logical_checks = self.get_checks('logical_line')
+        options.ast_checks = self.get_checks('tree')
+        self.init_report()
+
+    def init_report(self, reporter=None):
+        """Initialize the report instance."""
+        self.options.report = (reporter or self.options.reporter)(self.options)
+        return self.options.report
+
+    def check_files(self, paths=None):
+        """Run all checks on the paths."""
+        if paths is None:
+            paths = self.paths
+        report = self.options.report
+        runner = self.runner
+        report.start()
+        try:
+            for path in paths:
+                if os.path.isdir(path):
+                    self.input_dir(path)
+                elif not self.excluded(path):
+                    runner(path)
+        except KeyboardInterrupt:
+            print('... stopped')
+        report.stop()
+        return report
+
+    def input_file(self, filename, lines=None, expected=None, line_offset=0):
+        """Run all checks on a Python source file."""
+        if self.options.verbose:
+            print('checking %s' % filename)
+        fchecker = self.checker_class(
+            filename, lines=lines, options=self.options)
+        return fchecker.check_all(expected=expected, line_offset=line_offset)
+
+    def input_dir(self, dirname):
+        """Check all files in this directory and all subdirectories."""
+        dirname = dirname.rstrip('/')
+        if self.excluded(dirname):
+            return 0
+        counters = self.options.report.counters
+        verbose = self.options.verbose
+        filepatterns = self.options.filename
+        runner = self.runner
+        for root, dirs, files in os.walk(dirname):
+            if verbose:
+                print('directory ' + root)
+            counters['directories'] += 1
+            for subdir in sorted(dirs):
+                if self.excluded(subdir, root):
+                    dirs.remove(subdir)
+            for filename in sorted(files):
+                # contain a pattern that matches?
+                if ((filename_match(filename, filepatterns) and
+                     not self.excluded(filename, root))):
+                    runner(os.path.join(root, filename))
+
+    def excluded(self, filename, parent=None):
+        """Check if the file should be excluded.
+
+        Check if 'options.exclude' contains a pattern matching filename.
+        """
+        if not self.options.exclude:
+            return False
+        basename = os.path.basename(filename)
+        if filename_match(basename, self.options.exclude):
+            return True
+        if parent:
+            filename = os.path.join(parent, filename)
+        filename = os.path.abspath(filename)
+        return filename_match(filename, self.options.exclude)
+
+    def ignore_code(self, code):
+        """Check if the error code should be ignored.
+
+        If 'options.select' contains a prefix of the error code,
+        return False.  Else, if 'options.ignore' contains a prefix of
+        the error code, return True.
+        """
+        if len(code) < 4 and any(s.startswith(code)
+                                 for s in self.options.select):
+            return False
+        return (code.startswith(self.options.ignore) and
+                not code.startswith(self.options.select))
+
+    def get_checks(self, argument_name):
+        """Get all the checks for this category.
+
+        Find all globally visible functions where the first argument
+        name starts with argument_name and which contain selected tests.
+        """
+        checks = []
+        for check, attrs in _checks[argument_name].items():
+            (codes, args) = attrs
+            if any(not (code and self.ignore_code(code)) for code in codes):
+                checks.append((check.__name__, check, args))
+        return sorted(checks)
+
+
+def get_parser(prog='pycodestyle', version=__version__):
+    """Create the parser for the program."""
+    parser = OptionParser(prog=prog, version=version,
+                          usage="%prog [options] input ...")
+    parser.config_options = [
+        'exclude', 'filename', 'select', 'ignore', 'max-line-length',
+        'max-doc-length', 'hang-closing', 'count', 'format', 'quiet',
+        'show-pep8', 'show-source', 'statistics', 'verbose']
+    parser.add_option('-v', '--verbose', default=0, action='count',
+                      help="print status messages, or debug with -vv")
+    parser.add_option('-q', '--quiet', default=0, action='count',
+                      help="report only file names, or nothing with -qq")
+    parser.add_option('-r', '--repeat', default=True, action='store_true',
+                      help="(obsolete) show all occurrences of the same error")
+    parser.add_option('--first', action='store_false', dest='repeat',
+                      help="show first occurrence of each error")
+    parser.add_option('--exclude', metavar='patterns', default=DEFAULT_EXCLUDE,
+                      help="exclude files or directories which match these "
+                           "comma separated patterns (default: %default)")
+    parser.add_option('--filename', metavar='patterns', default='*.py',
+                      help="when parsing directories, only check filenames "
+                           "matching these comma separated patterns "
+                           "(default: %default)")
+    parser.add_option('--select', metavar='errors', default='',
+                      help="select errors and warnings (e.g. E,W6)")
+    parser.add_option('--ignore', metavar='errors', default='',
+                      help="skip errors and warnings (e.g. E4,W) "
+                           "(default: %s)" % DEFAULT_IGNORE)
+    parser.add_option('--show-source', action='store_true',
+                      help="show source code for each error")
+    parser.add_option('--show-pep8', action='store_true',
+                      help="show text of PEP 8 for each error "
+                           "(implies --first)")
+    parser.add_option('--statistics', action='store_true',
+                      help="count errors and warnings")
+    parser.add_option('--count', action='store_true',
+                      help="print total number of errors and warnings "
+                           "to standard error and set exit code to 1 if "
+                           "total is not null")
+    parser.add_option('--max-line-length', type='int', metavar='n',
+                      default=MAX_LINE_LENGTH,
+                      help="set maximum allowed line length "
+                           "(default: %default)")
+    parser.add_option('--max-doc-length', type='int', metavar='n',
+                      default=None,
+                      help="set maximum allowed doc line length and perform "
+                           "these checks (unchecked if not set)")
+    parser.add_option('--hang-closing', action='store_true',
+                      help="hang closing bracket instead of matching "
+                           "indentation of opening bracket's line")
+    parser.add_option('--format', metavar='format', default='default',
+                      help="set the error format [default|pylint|]")
+    parser.add_option('--diff', action='store_true',
+                      help="report changes only within line number ranges in "
+                           "the unified diff received on STDIN")
+    group = parser.add_option_group("Testing Options")
+    if os.path.exists(TESTSUITE_PATH):
+        group.add_option('--testsuite', metavar='dir',
+                         help="run regression tests from dir")
+        group.add_option('--doctest', action='store_true',
+                         help="run doctest on myself")
+    group.add_option('--benchmark', action='store_true',
+                     help="measure processing speed")
+    return parser
+
+
+def read_config(options, args, arglist, parser):
+    """Read and parse configurations.
+
+    If a config file is specified on the command line with the
+    "--config" option, then only it is used for configuration.
+
+    Otherwise, the user configuration (~/.config/pycodestyle) and any
+    local configurations in the current directory or above will be
+    merged together (in that order) using the read method of
+    ConfigParser.
+    """
+    config = RawConfigParser()
+
+    cli_conf = options.config
+
+    local_dir = os.curdir
+
+    if USER_CONFIG and os.path.isfile(USER_CONFIG):
+        if options.verbose:
+            print('user configuration: %s' % USER_CONFIG)
+        config.read(USER_CONFIG)
+
+    parent = tail = args and os.path.abspath(os.path.commonprefix(args))
+    while tail:
+        if config.read(os.path.join(parent, fn) for fn in PROJECT_CONFIG):
+            local_dir = parent
+            if options.verbose:
+                print('local configuration: in %s' % parent)
+            break
+        (parent, tail) = os.path.split(parent)
+
+    if cli_conf and os.path.isfile(cli_conf):
+        if options.verbose:
+            print('cli configuration: %s' % cli_conf)
+        config.read(cli_conf)
+
+    pycodestyle_section = None
+    if config.has_section(parser.prog):
+        pycodestyle_section = parser.prog
+    elif config.has_section('pep8'):
+        pycodestyle_section = 'pep8'  # Deprecated
+        warnings.warn('[pep8] section is deprecated. Use [pycodestyle].')
+
+    if pycodestyle_section:
+        option_list = {o.dest: o.type or o.action for o in parser.option_list}
+
+        # First, read the default values
+        (new_options, __) = parser.parse_args([])
+
+        # Second, parse the configuration
+        for opt in config.options(pycodestyle_section):
+            if opt.replace('_', '-') not in parser.config_options:
+                print("  unknown option '%s' ignored" % opt)
+                continue
+            if options.verbose > 1:
+                print("  %s = %s" % (opt,
+                                     config.get(pycodestyle_section, opt)))
+            normalized_opt = opt.replace('-', '_')
+            opt_type = option_list[normalized_opt]
+            if opt_type in ('int', 'count'):
+                value = config.getint(pycodestyle_section, opt)
+            elif opt_type in ('store_true', 'store_false'):
+                value = config.getboolean(pycodestyle_section, opt)
+            else:
+                value = config.get(pycodestyle_section, opt)
+                if normalized_opt == 'exclude':
+                    value = normalize_paths(value, local_dir)
+            setattr(new_options, normalized_opt, value)
+
+        # Third, overwrite with the command-line options
+        (options, __) = parser.parse_args(arglist, values=new_options)
+    options.doctest = options.testsuite = False
+    return options
+
+
+def process_options(arglist=None, parse_argv=False, config_file=None,
+                    parser=None, verbose=None):
+    """Process options passed either via arglist or command line args.
+
+    Passing in the ``config_file`` parameter allows other tools, such as
+    flake8 to specify their own options to be processed in pycodestyle.
+    """
+    if not parser:
+        parser = get_parser()
+    if not parser.has_option('--config'):
+        group = parser.add_option_group("Configuration", description=(
+            "The project options are read from the [%s] section of the "
+            "tox.ini file or the setup.cfg file located in any parent folder "
+            "of the path(s) being processed.  Allowed options are: %s." %
+            (parser.prog, ', '.join(parser.config_options))))
+        group.add_option('--config', metavar='path', default=config_file,
+                         help="user config file location")
+    # Don't read the command line if the module is used as a library.
+    if not arglist and not parse_argv:
+        arglist = []
+    # If parse_argv is True and arglist is None, arguments are
+    # parsed from the command line (sys.argv)
+    (options, args) = parser.parse_args(arglist)
+    options.reporter = None
+
+    # If explicitly specified verbosity, override any `-v` CLI flag
+    if verbose is not None:
+        options.verbose = verbose
+
+    if options.ensure_value('testsuite', False):
+        args.append(options.testsuite)
+    elif not options.ensure_value('doctest', False):
+        if parse_argv and not args:
+            if options.diff or any(os.path.exists(name)
+                                   for name in PROJECT_CONFIG):
+                args = ['.']
+            else:
+                parser.error('input not specified')
+        options = read_config(options, args, arglist, parser)
+        options.reporter = parse_argv and options.quiet == 1 and FileReport
+
+    options.filename = _parse_multi_options(options.filename)
+    options.exclude = normalize_paths(options.exclude)
+    options.select = _parse_multi_options(options.select)
+    options.ignore = _parse_multi_options(options.ignore)
+
+    if options.diff:
+        options.reporter = DiffReport
+        stdin = stdin_get_value()
+        options.selected_lines = parse_udiff(stdin, options.filename, args[0])
+        args = sorted(options.selected_lines)
+
+    return options, args
+
+
+def _parse_multi_options(options, split_token=','):
+    r"""Split and strip and discard empties.
+
+    Turns the following:
+
+    A,
+    B,
+
+    into ["A", "B"]
+    """
+    if options:
+        return [o.strip() for o in options.split(split_token) if o.strip()]
+    else:
+        return options
+
+
+def _main():
+    """Parse options and run checks on Python source."""
+    import signal
+
+    # Handle "Broken pipe" gracefully
+    try:
+        signal.signal(signal.SIGPIPE, lambda signum, frame: sys.exit(1))
+    except AttributeError:
+        pass    # not supported on Windows
+
+    style_guide = StyleGuide(parse_argv=True)
+    options = style_guide.options
+
+    if options.doctest or options.testsuite:
+        from testsuite.support import run_tests
+        report = run_tests(style_guide)
+    else:
+        report = style_guide.check_files()
+
+    if options.statistics:
+        report.print_statistics()
+
+    if options.benchmark:
+        report.print_benchmark()
+
+    if options.testsuite and not options.quiet:
+        report.print_results()
+
+    if report.total_errors:
+        if options.count:
+            sys.stderr.write(str(report.total_errors) + '\n')
+        sys.exit(1)
+
+
+if __name__ == '__main__':
+    _main()
diff --git a/env/lib/python3.8/site-packages/pylint-2.6.0.dist-info/COPYING b/env/lib/python3.8/site-packages/pylint-2.6.0.dist-info/COPYING
new file mode 100644
index 000000000..8c4c849e2
--- /dev/null
+++ b/env/lib/python3.8/site-packages/pylint-2.6.0.dist-info/COPYING
@@ -0,0 +1,340 @@
+		    GNU GENERAL PUBLIC LICENSE
+		       Version 2, June 1991
+
+ Copyright (C) 1989, 1991 Free Software Foundation, Inc.
+	51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+ Everyone is permitted to copy and distribute verbatim copies
+ of this license document, but changing it is not allowed.
+
+			    Preamble
+
+  The licenses for most software are designed to take away your
+freedom to share and change it.  By contrast, the GNU General Public
+License is intended to guarantee your freedom to share and change free
+software--to make sure the software is free for all its users.  This
+General Public License applies to most of the Free Software
+Foundation's software and to any other program whose authors commit to
+using it.  (Some other Free Software Foundation software is covered by
+the GNU Library General Public License instead.)  You can apply it to
+your programs, too.
+
+  When we speak of free software, we are referring to freedom, not
+price.  Our General Public Licenses are designed to make sure that you
+have the freedom to distribute copies of free software (and charge for
+this service if you wish), that you receive source code or can get it
+if you want it, that you can change the software or use pieces of it
+in new free programs; and that you know you can do these things.
+
+  To protect your rights, we need to make restrictions that forbid
+anyone to deny you these rights or to ask you to surrender the rights.
+These restrictions translate to certain responsibilities for you if you
+distribute copies of the software, or if you modify it.
+
+  For example, if you distribute copies of such a program, whether
+gratis or for a fee, you must give the recipients all the rights that
+you have.  You must make sure that they, too, receive or can get the
+source code.  And you must show them these terms so they know their
+rights.
+
+  We protect your rights with two steps: (1) copyright the software, and
+(2) offer you this license which gives you legal permission to copy,
+distribute and/or modify the software.
+
+  Also, for each author's protection and ours, we want to make certain
+that everyone understands that there is no warranty for this free
+software.  If the software is modified by someone else and passed on, we
+want its recipients to know that what they have is not the original, so
+that any problems introduced by others will not reflect on the original
+authors' reputations.
+
+  Finally, any free program is threatened constantly by software
+patents.  We wish to avoid the danger that redistributors of a free
+program will individually obtain patent licenses, in effect making the
+program proprietary.  To prevent this, we have made it clear that any
+patent must be licensed for everyone's free use or not licensed at all.
+
+  The precise terms and conditions for copying, distribution and
+modification follow.
+
+		    GNU GENERAL PUBLIC LICENSE
+   TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
+
+  0. This License applies to any program or other work which contains
+a notice placed by the copyright holder saying it may be distributed
+under the terms of this General Public License.  The "Program", below,
+refers to any such program or work, and a "work based on the Program"
+means either the Program or any derivative work under copyright law:
+that is to say, a work containing the Program or a portion of it,
+either verbatim or with modifications and/or translated into another
+language.  (Hereinafter, translation is included without limitation in
+the term "modification".)  Each licensee is addressed as "you".
+
+Activities other than copying, distribution and modification are not
+covered by this License; they are outside its scope.  The act of
+running the Program is not restricted, and the output from the Program
+is covered only if its contents constitute a work based on the
+Program (independent of having been made by running the Program).
+Whether that is true depends on what the Program does.
+
+  1. You may copy and distribute verbatim copies of the Program's
+source code as you receive it, in any medium, provided that you
+conspicuously and appropriately publish on each copy an appropriate
+copyright notice and disclaimer of warranty; keep intact all the
+notices that refer to this License and to the absence of any warranty;
+and give any other recipients of the Program a copy of this License
+along with the Program.
+
+You may charge a fee for the physical act of transferring a copy, and
+you may at your option offer warranty protection in exchange for a fee.
+
+  2. You may modify your copy or copies of the Program or any portion
+of it, thus forming a work based on the Program, and copy and
+distribute such modifications or work under the terms of Section 1
+above, provided that you also meet all of these conditions:
+
+    a) You must cause the modified files to carry prominent notices
+    stating that you changed the files and the date of any change.
+
+    b) You must cause any work that you distribute or publish, that in
+    whole or in part contains or is derived from the Program or any
+    part thereof, to be licensed as a whole at no charge to all third
+    parties under the terms of this License.
+
+    c) If the modified program normally reads commands interactively
+    when run, you must cause it, when started running for such
+    interactive use in the most ordinary way, to print or display an
+    announcement including an appropriate copyright notice and a
+    notice that there is no warranty (or else, saying that you provide
+    a warranty) and that users may redistribute the program under
+    these conditions, and telling the user how to view a copy of this
+    License.  (Exception: if the Program itself is interactive but
+    does not normally print such an announcement, your work based on
+    the Program is not required to print an announcement.)
+
+These requirements apply to the modified work as a whole.  If
+identifiable sections of that work are not derived from the Program,
+and can be reasonably considered independent and separate works in
+themselves, then this License, and its terms, do not apply to those
+sections when you distribute them as separate works.  But when you
+distribute the same sections as part of a whole which is a work based
+on the Program, the distribution of the whole must be on the terms of
+this License, whose permissions for other licensees extend to the
+entire whole, and thus to each and every part regardless of who wrote it.
+
+Thus, it is not the intent of this section to claim rights or contest
+your rights to work written entirely by you; rather, the intent is to
+exercise the right to control the distribution of derivative or
+collective works based on the Program.
+
+In addition, mere aggregation of another work not based on the Program
+with the Program (or with a work based on the Program) on a volume of
+a storage or distribution medium does not bring the other work under
+the scope of this License.
+
+  3. You may copy and distribute the Program (or a work based on it,
+under Section 2) in object code or executable form under the terms of
+Sections 1 and 2 above provided that you also do one of the following:
+
+    a) Accompany it with the complete corresponding machine-readable
+    source code, which must be distributed under the terms of Sections
+    1 and 2 above on a medium customarily used for software interchange; or,
+
+    b) Accompany it with a written offer, valid for at least three
+    years, to give any third party, for a charge no more than your
+    cost of physically performing source distribution, a complete
+    machine-readable copy of the corresponding source code, to be
+    distributed under the terms of Sections 1 and 2 above on a medium
+    customarily used for software interchange; or,
+
+    c) Accompany it with the information you received as to the offer
+    to distribute corresponding source code.  (This alternative is
+    allowed only for noncommercial distribution and only if you
+    received the program in object code or executable form with such
+    an offer, in accord with Subsection b above.)
+
+The source code for a work means the preferred form of the work for
+making modifications to it.  For an executable work, complete source
+code means all the source code for all modules it contains, plus any
+associated interface definition files, plus the scripts used to
+control compilation and installation of the executable.  However, as a
+special exception, the source code distributed need not include
+anything that is normally distributed (in either source or binary
+form) with the major components (compiler, kernel, and so on) of the
+operating system on which the executable runs, unless that component
+itself accompanies the executable.
+
+If distribution of executable or object code is made by offering
+access to copy from a designated place, then offering equivalent
+access to copy the source code from the same place counts as
+distribution of the source code, even though third parties are not
+compelled to copy the source along with the object code.
+
+  4. You may not copy, modify, sublicense, or distribute the Program
+except as expressly provided under this License.  Any attempt
+otherwise to copy, modify, sublicense or distribute the Program is
+void, and will automatically terminate your rights under this License.
+However, parties who have received copies, or rights, from you under
+this License will not have their licenses terminated so long as such
+parties remain in full compliance.
+
+  5. You are not required to accept this License, since you have not
+signed it.  However, nothing else grants you permission to modify or
+distribute the Program or its derivative works.  These actions are
+prohibited by law if you do not accept this License.  Therefore, by
+modifying or distributing the Program (or any work based on the
+Program), you indicate your acceptance of this License to do so, and
+all its terms and conditions for copying, distributing or modifying
+the Program or works based on it.
+
+  6. Each time you redistribute the Program (or any work based on the
+Program), the recipient automatically receives a license from the
+original licensor to copy, distribute or modify the Program subject to
+these terms and conditions.  You may not impose any further
+restrictions on the recipients' exercise of the rights granted herein.
+You are not responsible for enforcing compliance by third parties to
+this License.
+
+  7. If, as a consequence of a court judgment or allegation of patent
+infringement or for any other reason (not limited to patent issues),
+conditions are imposed on you (whether by court order, agreement or
+otherwise) that contradict the conditions of this License, they do not
+excuse you from the conditions of this License.  If you cannot
+distribute so as to satisfy simultaneously your obligations under this
+License and any other pertinent obligations, then as a consequence you
+may not distribute the Program at all.  For example, if a patent
+license would not permit royalty-free redistribution of the Program by
+all those who receive copies directly or indirectly through you, then
+the only way you could satisfy both it and this License would be to
+refrain entirely from distribution of the Program.
+
+If any portion of this section is held invalid or unenforceable under
+any particular circumstance, the balance of the section is intended to
+apply and the section as a whole is intended to apply in other
+circumstances.
+
+It is not the purpose of this section to induce you to infringe any
+patents or other property right claims or to contest validity of any
+such claims; this section has the sole purpose of protecting the
+integrity of the free software distribution system, which is
+implemented by public license practices.  Many people have made
+generous contributions to the wide range of software distributed
+through that system in reliance on consistent application of that
+system; it is up to the author/donor to decide if he or she is willing
+to distribute software through any other system and a licensee cannot
+impose that choice.
+
+This section is intended to make thoroughly clear what is believed to
+be a consequence of the rest of this License.
+
+  8. If the distribution and/or use of the Program is restricted in
+certain countries either by patents or by copyrighted interfaces, the
+original copyright holder who places the Program under this License
+may add an explicit geographical distribution limitation excluding
+those countries, so that distribution is permitted only in or among
+countries not thus excluded.  In such case, this License incorporates
+the limitation as if written in the body of this License.
+
+  9. The Free Software Foundation may publish revised and/or new versions
+of the General Public License from time to time.  Such new versions will
+be similar in spirit to the present version, but may differ in detail to
+address new problems or concerns.
+
+Each version is given a distinguishing version number.  If the Program
+specifies a version number of this License which applies to it and "any
+later version", you have the option of following the terms and conditions
+either of that version or of any later version published by the Free
+Software Foundation.  If the Program does not specify a version number of
+this License, you may choose any version ever published by the Free Software
+Foundation.
+
+  10. If you wish to incorporate parts of the Program into other free
+programs whose distribution conditions are different, write to the author
+to ask for permission.  For software which is copyrighted by the Free
+Software Foundation, write to the Free Software Foundation; we sometimes
+make exceptions for this.  Our decision will be guided by the two goals
+of preserving the free status of all derivatives of our free software and
+of promoting the sharing and reuse of software generally.
+
+			    NO WARRANTY
+
+  11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY
+FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW.  EXCEPT WHEN
+OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES
+PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED
+OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
+MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE.  THE ENTIRE RISK AS
+TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU.  SHOULD THE
+PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING,
+REPAIR OR CORRECTION.
+
+  12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
+WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR
+REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES,
+INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING
+OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED
+TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY
+YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER
+PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE
+POSSIBILITY OF SUCH DAMAGES.
+
+		     END OF TERMS AND CONDITIONS
+
+	    How to Apply These Terms to Your New Programs
+
+  If you develop a new program, and you want it to be of the greatest
+possible use to the public, the best way to achieve this is to make it
+free software which everyone can redistribute and change under these terms.
+
+  To do so, attach the following notices to the program.  It is safest
+to attach them to the start of each source file to most effectively
+convey the exclusion of warranty; and each file should have at least
+the "copyright" line and a pointer to where the full notice is found.
+
+    
+    Copyright (C)   
+
+    This program is free software; you can redistribute it and/or modify
+    it under the terms of the GNU General Public License as published by
+    the Free Software Foundation; either version 2 of the License, or
+    (at your option) any later version.
+
+    This program is distributed in the hope that it will be useful,
+    but WITHOUT ANY WARRANTY; without even the implied warranty of
+    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+    GNU General Public License for more details.
+
+    You should have received a copy of the GNU General Public License
+    along with this program; if not, write to the Free Software
+    Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301 USA
+
+
+Also add information on how to contact you by electronic and paper mail.
+
+If the program is interactive, make it output a short notice like this
+when it starts in an interactive mode:
+
+    Gnomovision version 69, Copyright (C) year  name of author
+    Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
+    This is free software, and you are welcome to redistribute it
+    under certain conditions; type `show c' for details.
+
+The hypothetical commands `show w' and `show c' should show the appropriate
+parts of the General Public License.  Of course, the commands you use may
+be called something other than `show w' and `show c'; they could even be
+mouse-clicks or menu items--whatever suits your program.
+
+You should also get your employer (if you work as a programmer) or your
+school, if any, to sign a "copyright disclaimer" for the program, if
+necessary.  Here is a sample; alter the names:
+
+  Yoyodyne, Inc., hereby disclaims all copyright interest in the program
+  `Gnomovision' (which makes passes at compilers) written by James Hacker.
+
+  , 1 April 1989
+  Ty Coon, President of Vice
+
+This General Public License does not permit incorporating your program into
+proprietary programs.  If your program is a subroutine library, you may
+consider it more useful to permit linking proprietary applications with the
+library.  If this is what you want to do, use the GNU Library General
+Public License instead of this License.
diff --git a/env/lib/python3.8/site-packages/pylint-2.6.0.dist-info/INSTALLER b/env/lib/python3.8/site-packages/pylint-2.6.0.dist-info/INSTALLER
new file mode 100644
index 000000000..a1b589e38
--- /dev/null
+++ b/env/lib/python3.8/site-packages/pylint-2.6.0.dist-info/INSTALLER
@@ -0,0 +1 @@
+pip
diff --git a/env/lib/python3.8/site-packages/pylint-2.6.0.dist-info/METADATA b/env/lib/python3.8/site-packages/pylint-2.6.0.dist-info/METADATA
new file mode 100644
index 000000000..f2a378517
--- /dev/null
+++ b/env/lib/python3.8/site-packages/pylint-2.6.0.dist-info/METADATA
@@ -0,0 +1,205 @@
+Metadata-Version: 2.1
+Name: pylint
+Version: 2.6.0
+Summary: python code static checker
+Home-page: https://github.com/PyCQA/pylint
+Author: Python Code Quality Authority
+Author-email: code-quality@python.org
+License: GPL
+Project-URL: What's New, https://pylint.pycqa.org/en/latest/whatsnew/
+Platform: UNKNOWN
+Classifier: Development Status :: 6 - Mature
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: GNU General Public License (GPL)
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.5
+Classifier: Programming Language :: Python :: 3.6
+Classifier: Programming Language :: Python :: 3.7
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Programming Language :: Python :: Implementation :: CPython
+Classifier: Programming Language :: Python :: Implementation :: PyPy
+Classifier: Topic :: Software Development :: Debuggers
+Classifier: Topic :: Software Development :: Quality Assurance
+Classifier: Topic :: Software Development :: Testing
+Requires-Python: >=3.5.*
+Requires-Dist: astroid (<=2.5,>=2.4.0)
+Requires-Dist: isort (<6,>=4.2.5)
+Requires-Dist: mccabe (<0.7,>=0.6)
+Requires-Dist: toml (>=0.7.1)
+Requires-Dist: colorama ; sys_platform=="win32"
+
+
+README for Pylint - https://pylint.pycqa.org/
+=============================================
+
+.. image:: https://travis-ci.org/PyCQA/pylint.svg?branch=master
+    :target: https://travis-ci.org/PyCQA/pylint
+
+.. image:: https://ci.appveyor.com/api/projects/status/rbvwhakyj1y09atb/branch/master?svg=true
+    :alt: AppVeyor Build Status
+    :target: https://ci.appveyor.com/project/PCManticore/pylint
+
+.. image:: https://coveralls.io/repos/github/PyCQA/pylint/badge.svg?branch=master
+    :target: https://coveralls.io/github/PyCQA/pylint?branch=master
+
+
+.. image:: https://img.shields.io/pypi/v/pylint.svg
+    :alt: Pypi Package version
+    :target: https://pypi.python.org/pypi/pylint
+
+.. image:: https://readthedocs.org/projects/pylint/badge/?version=latest
+    :target: https://pylint.readthedocs.io/en/latest/?badge=latest
+    :alt: Documentation Status
+
+.. image:: https://img.shields.io/badge/code%20style-black-000000.svg
+    :target: https://github.com/ambv/black
+
+.. |tideliftlogo| image:: doc/media/Tidelift_Logos_RGB_Tidelift_Shorthand_On-White_small.png
+   :width: 75
+   :height: 60
+   :alt: Tidelift
+
+.. list-table::
+   :widths: 10 100
+
+   * - |tideliftlogo|
+     - Professional support for pylint is available as part of the `Tidelift
+       Subscription`_.  Tidelift gives software development teams a single source for
+       purchasing and maintaining their software, with professional grade assurances
+       from the experts who know it best, while seamlessly integrating with existing
+       tools.
+
+.. _Tidelift Subscription: https://tidelift.com/subscription/pkg/pypi-pylint?utm_source=pypi-pylint&utm_medium=referral&utm_campaign=readme
+
+
+======
+Pylint
+======
+
+**It's not just a linter that annoys you!**
+
+Pylint is a Python static code analysis tool which looks for programming errors,
+helps enforcing a coding standard, sniffs for code smells and offers simple refactoring
+suggestions.
+
+It's highly configurable, having special pragmas to control its errors and warnings
+from within your code, as well as from an extensive configuration file.
+It is also possible to write your own plugins for adding your own checks or for
+extending pylint in one way or another.
+
+It's a free software distributed under the GNU General Public Licence unless
+otherwise specified.
+
+Development is hosted on GitHub: https://github.com/PyCQA/pylint/
+
+You can use the code-quality@python.org mailing list to discuss about
+Pylint. Subscribe at https://mail.python.org/mailman/listinfo/code-quality/
+or read the archives at https://mail.python.org/pipermail/code-quality/
+
+Pull requests are amazing and most welcome.
+
+Install
+-------
+
+Pylint can be simply installed by running::
+
+    pip install pylint
+
+If you are using Python 3.6+, upgrade to get full support for your version::
+
+    pip install pylint --upgrade
+
+If you want to install from a source distribution, extract the tarball and run
+the following command ::
+
+    python setup.py install
+
+
+Do make sure to do the same for astroid, which is used internally by pylint.
+
+For debian and rpm packages, use your usual tools according to your Linux distribution.
+
+More information about installation and available distribution format
+can be found here_.
+
+Documentation
+-------------
+
+The documentation lives at https://pylint.pycqa.org/.
+
+Pylint is shipped with following additional commands:
+
+* pyreverse: an UML diagram generator
+* symilar: an independent similarities checker
+* epylint: Emacs and Flymake compatible Pylint
+
+
+Testing
+-------
+
+We use tox_ for running the test suite. You should be able to install it with::
+
+    pip install tox pytest
+
+
+To run the test suite for a particular Python version, you can do::
+
+    tox -e py37
+
+
+To run individual tests with ``tox``, you can do::
+
+    tox -e py37 -- -k name_of_the_test
+
+
+We use pytest_ for testing ``pylint``, which you can use without using ``tox`` for a faster development cycle.
+
+If you want to run tests on a specific portion of the code with pytest_, (pytest-cov_) and your local python version::
+
+    # ( pip install pytest-cov )
+    # Everything:
+    python3 -m pytest tests/
+    # Everything in tests/message with coverage for the relevant code:
+    python3 -m pytest tests/message/ --cov=pylint.message
+    coverage html
+    # Only the functional test "missing_kwoa_py3":
+    python3 -m pytest "tests/test_functional.py::test_functional[missing_kwoa_py3]"
+
+
+Do not forget to clone astroid_ and install the last version::
+
+
+    git clone https://github.com/PyCQA/astroid.git
+
+    # From source
+    python3 astroid/setup.py build sdist
+    pip3 install astroid/dist/astroid*.tar.gz
+
+    # Using an editable installation
+    cd astroid
+    python3 -m pip install -e .
+
+
+For more detailed information, check the documentation.
+
+.. _here: https://pylint.pycqa.org/en/latest/user_guide/installation.html
+.. _tox: https://tox.readthedocs.io/en/latest/
+.. _pytest: https://docs.pytest.org/en/latest/
+.. _pytest-cov: https://pypi.org/project/pytest-cov/
+.. _astroid: https://github.com/PyCQA/astroid
+
+License
+-------
+
+pylint is, with a few exceptions listed below, `GPLv2 `_.
+
+The icon files are licensed under the `CC BY-SA 4.0 `_ license:
+
+- `doc/logo.png `_
+- `doc/logo.svg `_
+
+
diff --git a/env/lib/python3.8/site-packages/pylint-2.6.0.dist-info/RECORD b/env/lib/python3.8/site-packages/pylint-2.6.0.dist-info/RECORD
new file mode 100644
index 000000000..c4ae768d8
--- /dev/null
+++ b/env/lib/python3.8/site-packages/pylint-2.6.0.dist-info/RECORD
@@ -0,0 +1,187 @@
+../../../bin/epylint,sha256=5xw_mxm_Q6O8y5ez6Ga2z8UgOxqGqLfrYSKVNuinkPY,299
+../../../bin/pylint,sha256=jXu977mqJKFTgb7g0nm8TgA4dbgW6pOLfdQ4QYtu6kQ,297
+../../../bin/pyreverse,sha256=guFB-e-p0gs5Ixrw2rJTMs6p5NQm__k3Wyp_XnBV7P8,303
+../../../bin/symilar,sha256=ZUTz9wljHIrVHC12-fuH1XH-u5z5v2sptjvw2HyW6ak,299
+pylint-2.6.0.data/scripts/epylint,sha256=ebDphNeMoKus049k5MQbxN1JYsHUsOXZxws0Do6gCG0,51
+pylint-2.6.0.data/scripts/pylint,sha256=wXf1V2_-AB_S1uuYztSS90GiTeCkJ4eBOGEQ7CO2Nmc,53
+pylint-2.6.0.data/scripts/pyreverse,sha256=4UQf7-hfOAx6Ux8d5g0d2KIjpUPRMwFhBdsKsu0gWg0,59
+pylint-2.6.0.data/scripts/symilar,sha256=iz6DGtePyfs0haoFobDfsRsMjaFOizh7E3vsevB2Ipw,55
+pylint-2.6.0.dist-info/COPYING,sha256=-XsUCA3ouEkNYOs9Yg68QZlD4HeUZtHdDV1vaP4ZXc0,17984
+pylint-2.6.0.dist-info/INSTALLER,sha256=zuuue4knoyJ-UwPPXg8fezS7VCrXJQrAP7zeNuwvFQg,4
+pylint-2.6.0.dist-info/METADATA,sha256=tyNZytHzEEwcJ9VzfKzbveXToNRzjOdN0XDpdJfsk14,6705
+pylint-2.6.0.dist-info/RECORD,,
+pylint-2.6.0.dist-info/WHEEL,sha256=EVRjI69F5qVjm_YgqcTXPnTAv3BfSUr0WVAHuSP3Xoo,92
+pylint-2.6.0.dist-info/entry_points.txt,sha256=WUTwHM2ZcExO-VSvss18AMFsQL-XcWg6O3_MwobWfmw,137
+pylint-2.6.0.dist-info/top_level.txt,sha256=j6Z9i__pIuaiCka6Ul9YIy6yI5aw5QbCtldLvZlMksE,7
+pylint/__init__.py,sha256=v4P6u-dsxp2oq7QRVH29e7BT4M5bdn-8YZbwjebaWe0,1159
+pylint/__main__.py,sha256=HRja-2If7jX6FMF5dY7uMVUO2kJ3I0Y0uflK2ckDA5s,585
+pylint/__pkginfo__.py,sha256=3JpKkNXffks0hqvIVBZDhr5bPzHAOPBi-VWcCq7GtU4,4121
+pylint/__pycache__/__init__.cpython-38.pyc,,
+pylint/__pycache__/__main__.cpython-38.pyc,,
+pylint/__pycache__/__pkginfo__.cpython-38.pyc,,
+pylint/__pycache__/constants.cpython-38.pyc,,
+pylint/__pycache__/epylint.cpython-38.pyc,,
+pylint/__pycache__/exceptions.cpython-38.pyc,,
+pylint/__pycache__/graph.cpython-38.pyc,,
+pylint/__pycache__/interfaces.cpython-38.pyc,,
+pylint/__pycache__/testutils.cpython-38.pyc,,
+pylint/checkers/__init__.py,sha256=Xs6vVoNtulCm3eLzOKZXzLydO3CkZoo-LCUi02V9DSw,2203
+pylint/checkers/__pycache__/__init__.cpython-38.pyc,,
+pylint/checkers/__pycache__/async.cpython-38.pyc,,
+pylint/checkers/__pycache__/base.cpython-38.pyc,,
+pylint/checkers/__pycache__/base_checker.cpython-38.pyc,,
+pylint/checkers/__pycache__/classes.cpython-38.pyc,,
+pylint/checkers/__pycache__/design_analysis.cpython-38.pyc,,
+pylint/checkers/__pycache__/exceptions.cpython-38.pyc,,
+pylint/checkers/__pycache__/format.cpython-38.pyc,,
+pylint/checkers/__pycache__/imports.cpython-38.pyc,,
+pylint/checkers/__pycache__/logging.cpython-38.pyc,,
+pylint/checkers/__pycache__/misc.cpython-38.pyc,,
+pylint/checkers/__pycache__/newstyle.cpython-38.pyc,,
+pylint/checkers/__pycache__/python3.cpython-38.pyc,,
+pylint/checkers/__pycache__/raw_metrics.cpython-38.pyc,,
+pylint/checkers/__pycache__/refactoring.cpython-38.pyc,,
+pylint/checkers/__pycache__/similar.cpython-38.pyc,,
+pylint/checkers/__pycache__/spelling.cpython-38.pyc,,
+pylint/checkers/__pycache__/stdlib.cpython-38.pyc,,
+pylint/checkers/__pycache__/strings.cpython-38.pyc,,
+pylint/checkers/__pycache__/typecheck.cpython-38.pyc,,
+pylint/checkers/__pycache__/utils.cpython-38.pyc,,
+pylint/checkers/__pycache__/variables.cpython-38.pyc,,
+pylint/checkers/async.py,sha256=JsT7Tz40BzExCxOM1lxOazq8a5U5Qt30NVH7skaVVeg,3524
+pylint/checkers/base.py,sha256=k6S26UCUdr2KhKibI4Py_DXZsVP72oEgJVQSbdu8m5c,96488
+pylint/checkers/base_checker.py,sha256=6EiWxqNjJ69sQYbLNobRY--GDByqH-HUfivDc2EUC6Y,7664
+pylint/checkers/classes.py,sha256=RAiGs_3bN7fLpSq9DE8jwVTfFpQzIEQ7sv5NrzUoTsw,79023
+pylint/checkers/design_analysis.py,sha256=CHrofelHlAVPMx_NzDlLAU4A7lh_F5tOTa47BSEiCBE,16861
+pylint/checkers/exceptions.py,sha256=28OsFP9Qka-WIJEBwSj0058REHMkCt7jTM7sLPVO4Q4,23729
+pylint/checkers/format.py,sha256=HMAD_0DRrLOLHV2vl6dzEpF96FocYlIxbc0EDxyvXGo,28135
+pylint/checkers/imports.py,sha256=XC9a9KhVg3wUT2bytUbDeoQPZX3DUX3Wn_ebD-21nvc,38230
+pylint/checkers/logging.py,sha256=t_EM5pASCXB4ZLpgVm1O4WEXCP08K8ZzyFCqSTfz33M,15919
+pylint/checkers/misc.py,sha256=gVe__qHnXK1RDHXXrByndQd6_VSnWnZcObWKNgViiqE,7241
+pylint/checkers/newstyle.py,sha256=BRTLrHos8vvfB5vnmK2q_XHrDHPlImM6e1ykybq_5_o,5067
+pylint/checkers/python3.py,sha256=qSnqN2HBpELbFDBJ6VOShoUoMyvWhvbHu7ln2vg9rAM,53095
+pylint/checkers/raw_metrics.py,sha256=bi52QkvDD8rBH1tM5wh15-T9KdAXeM_rrSzwdDFJ73s,4040
+pylint/checkers/refactoring.py,sha256=ToWYmRszGgmNfln_eqeqCh_KKjxCPpvhxLvtMtn75tQ,63239
+pylint/checkers/similar.py,sha256=xbPADFB8VPBu4XrVj1LBB1FOMQpg0jQyGJWCw16TXpc,15458
+pylint/checkers/spelling.py,sha256=Dx1ruls8wi0yIqMDb2kJ2kh2tO3_cAGieGDn8Y-dx8k,13867
+pylint/checkers/stdlib.py,sha256=-5laS4-VByRWyD5uJsP7ijHUELdLWT8o4v5FB_ptK44,17690
+pylint/checkers/strings.py,sha256=CIjSzGeyGbNbxsJ86UMKLHZKBtGIodOUMQsPyYV9_MU,38113
+pylint/checkers/typecheck.py,sha256=FgD4diwuE6OXtQBpyPV-8Xl4AFtsX5a5gpWzeMt4LQQ,71732
+pylint/checkers/utils.py,sha256=5j4dc1NEpZywEG1loQ2pe5qlFdJH2I9P1-mhjTEKrn8,45213
+pylint/checkers/variables.py,sha256=cDH63Hwj2l7_nejxVBsn7rtv5S7Bgm23vNWITVJBa48,80398
+pylint/config/__init__.py,sha256=3USGhpgN7m5H7VqsLPjOJsRyjIQswaUv5z1rY1dyBiM,4459
+pylint/config/__pycache__/__init__.cpython-38.pyc,,
+pylint/config/__pycache__/configuration_mixin.cpython-38.pyc,,
+pylint/config/__pycache__/find_default_config_files.cpython-38.pyc,,
+pylint/config/__pycache__/man_help_formatter.cpython-38.pyc,,
+pylint/config/__pycache__/option.cpython-38.pyc,,
+pylint/config/__pycache__/option_manager_mixin.cpython-38.pyc,,
+pylint/config/__pycache__/option_parser.cpython-38.pyc,,
+pylint/config/__pycache__/options_provider_mixin.cpython-38.pyc,,
+pylint/config/configuration_mixin.py,sha256=2gvNKg1LZBtlnAUqO2OOHFZvgdigfZLswAFO58CzO1Q,1105
+pylint/config/find_default_config_files.py,sha256=sPIxomIQUXvvh-11Ygc-NiBYIrK5wQO_7LMkzqxfo4Y,2103
+pylint/config/man_help_formatter.py,sha256=JZEeNKvjSpdYTY0jV5_PahKqSP8Ei489JH7Uk4oPkkI,3777
+pylint/config/option.py,sha256=jWXPFt3YCALhgmuo5T87TZQ5EbochLAGAj7nArllxrc,5406
+pylint/config/option_manager_mixin.py,sha256=Z2yMMhjb4oUOSuzs7xYYxN__F0Sa9hCha80DeqF2UD8,14470
+pylint/config/option_parser.py,sha256=7-PmG8r5i5N4r5tgRUq1A45sq_pRz8lFMtw33nRST4Y,1689
+pylint/config/options_provider_mixin.py,sha256=vXWVrsnz-zDLW6YlHflJTtvI_QqvqusajJy_YxK8iZg,4176
+pylint/constants.py,sha256=vnKjbOfqog3sjuSN48ZXO6VeQrb0sV43D028rVeZDss,1405
+pylint/epylint.py,sha256=kKt3on5YJw5vXLI9AeO9gIB3u9lH8fkE1ReGlyzPNJw,7240
+pylint/exceptions.py,sha256=WuVt9BlJW92g8fggqy7aCoC9GgSIy9AEpHsiCKo8psU,1153
+pylint/extensions/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+pylint/extensions/__pycache__/__init__.cpython-38.pyc,,
+pylint/extensions/__pycache__/_check_docs_utils.cpython-38.pyc,,
+pylint/extensions/__pycache__/bad_builtin.cpython-38.pyc,,
+pylint/extensions/__pycache__/broad_try_clause.cpython-38.pyc,,
+pylint/extensions/__pycache__/check_docs.cpython-38.pyc,,
+pylint/extensions/__pycache__/check_elif.cpython-38.pyc,,
+pylint/extensions/__pycache__/comparetozero.cpython-38.pyc,,
+pylint/extensions/__pycache__/docparams.cpython-38.pyc,,
+pylint/extensions/__pycache__/docstyle.cpython-38.pyc,,
+pylint/extensions/__pycache__/emptystring.cpython-38.pyc,,
+pylint/extensions/__pycache__/mccabe.cpython-38.pyc,,
+pylint/extensions/__pycache__/overlapping_exceptions.cpython-38.pyc,,
+pylint/extensions/__pycache__/redefined_variable_type.cpython-38.pyc,,
+pylint/extensions/_check_docs_utils.py,sha256=X-phX6z3Znthcuq4BpgMZR-3Z-cS3iRgIMHlwq4uIXM,23564
+pylint/extensions/bad_builtin.py,sha256=nCYJ4nGEvbTnB6AlPMx_IBjb2bWFu3rkz6ThW_pQ6Ac,2510
+pylint/extensions/broad_try_clause.py,sha256=nIOTZfOVGsM4fp50yyaes-z--iAlYvgQcbvlA_W5i3Q,2331
+pylint/extensions/check_docs.py,sha256=SJvB9OaaPeA5GsWZRofYuUlfP6u_Wny4Ugog0OqtzGs,796
+pylint/extensions/check_elif.py,sha256=rj3FgaXcCylZTb_PWraCCOnmzkPuA0hLc3CZo3lg8hA,2614
+pylint/extensions/comparetozero.py,sha256=MeVUIYTv4kFEHbBZHAjQXbIa1LbnHK9VncEoZKxHe5w,2469
+pylint/extensions/docparams.py,sha256=fCQC3z2gX6CbUCv6cGaIeF1ps_icrvIYI8Eu_ul32MY,20437
+pylint/extensions/docstyle.py,sha256=bG8V4n9fXjWrSyXl3ZKi5gNnVogEgcroJo9wxykKGEM,3035
+pylint/extensions/emptystring.py,sha256=kkk35STu_Bsm9Boi1sjVTCa7ZeB5duo3I1fbjWKjUfI,2568
+pylint/extensions/mccabe.py,sha256=7jupZPjE0a5SsfxhSj9O3al0j6vjR_J18q7cCSTzGLY,6278
+pylint/extensions/overlapping_exceptions.py,sha256=Qsi7qIF6sx36HYV8cjKjPcgYRZNPiFIg4NEcefjGFyg,3266
+pylint/extensions/redefined_variable_type.py,sha256=qopv8bgRcPOiQoa8c6btH0trTQEt2tCRm4sug6qOwjM,4333
+pylint/graph.py,sha256=jj1v2fqQrGAhjdPOLVAzOsqLFR9ce7LTVyyFKjaAhTQ,6528
+pylint/interfaces.py,sha256=jSpQ-c3PGqLZF7LsJue5gHz6IdYire9ki7cieG6xdfE,3235
+pylint/lint/__init__.py,sha256=FkrvfVY1gTb6syeGkvNDM37-0pXXOmR8qu0ig95CQ9M,4188
+pylint/lint/__pycache__/__init__.cpython-38.pyc,,
+pylint/lint/__pycache__/check_parallel.cpython-38.pyc,,
+pylint/lint/__pycache__/pylinter.cpython-38.pyc,,
+pylint/lint/__pycache__/report_functions.cpython-38.pyc,,
+pylint/lint/__pycache__/run.cpython-38.pyc,,
+pylint/lint/__pycache__/utils.cpython-38.pyc,,
+pylint/lint/check_parallel.py,sha256=Jov2r9_0_hjRBfOC0ZYSyU67dY99pv-idEIE5fBnN3I,3973
+pylint/lint/pylinter.py,sha256=ZRjQ7iSTbI6lC0HroM7urKdooqcfCpVN1ss05mjOzRo,45795
+pylint/lint/report_functions.py,sha256=4jB7bRHENFdvY4xXMamIC1VaQdrYdyp-629KujV-pI8,2750
+pylint/lint/run.py,sha256=lq3iu62kTYIIXXsHbDFnf8rIx7MxEMpKJ30fHNmYZA8,17223
+pylint/lint/utils.py,sha256=glQ5R9CNfpDL6Kj9lysnlUOOpOGuMP_sLA2gUpA1m2w,2308
+pylint/message/__init__.py,sha256=sgVx9YmL9b84Y6TgsfOKHJchXnxOd-EGV52wEKWLLe8,2807
+pylint/message/__pycache__/__init__.cpython-38.pyc,,
+pylint/message/__pycache__/message.cpython-38.pyc,,
+pylint/message/__pycache__/message_definition.cpython-38.pyc,,
+pylint/message/__pycache__/message_definition_store.cpython-38.pyc,,
+pylint/message/__pycache__/message_handler_mix_in.cpython-38.pyc,,
+pylint/message/__pycache__/message_id_store.cpython-38.pyc,,
+pylint/message/message.py,sha256=eJQ6x3SKMQpD6aLn4x2tYT4NSUy4qWzVhl_0Am5R5WI,1300
+pylint/message/message_definition.py,sha256=o7zckciMEunKoi3wz7U2m507HsXD4IeHuPJ-anURyi8,2993
+pylint/message/message_definition_store.py,sha256=aQi_4Z_tEw9lxQyn07-w_BapJLd04OcZIJ5A4tLgxho,3535
+pylint/message/message_handler_mix_in.py,sha256=BcHlarVGm-ySZOmMOKSmJ-jQlQhVXA6HYpFw4p1_ZYc,15208
+pylint/message/message_id_store.py,sha256=Nri4iRo9t5QefzJP9MU0phz66dBZ3ienVNRqQNQoDac,5291
+pylint/pyreverse/__init__.py,sha256=runafCn0veg0di-i8TztMGlKEJO3Qg01MICGqDgZ0c0,202
+pylint/pyreverse/__pycache__/__init__.cpython-38.pyc,,
+pylint/pyreverse/__pycache__/diadefslib.cpython-38.pyc,,
+pylint/pyreverse/__pycache__/diagrams.cpython-38.pyc,,
+pylint/pyreverse/__pycache__/inspector.cpython-38.pyc,,
+pylint/pyreverse/__pycache__/main.cpython-38.pyc,,
+pylint/pyreverse/__pycache__/utils.cpython-38.pyc,,
+pylint/pyreverse/__pycache__/vcgutils.cpython-38.pyc,,
+pylint/pyreverse/__pycache__/writer.cpython-38.pyc,,
+pylint/pyreverse/diadefslib.py,sha256=9y9COGVkaeWlMbuMh0PH0ka5FXuN5jvOjEZAPbtZAdI,8754
+pylint/pyreverse/diagrams.py,sha256=PAuxP25QXqEmQgFpJDHxCEKHo9EIpB7o8cYFITe3FGc,9035
+pylint/pyreverse/inspector.py,sha256=RyFmZ_8IEH59EpXOqkS1d3KsqTdn_LWu-_a3fNu08QU,12257
+pylint/pyreverse/main.py,sha256=FIrqeVfxM8fje0QprImOFLeiz4s7oawOMXoa5yJEPqA,6528
+pylint/pyreverse/utils.py,sha256=-pmBeypy9i6GRD4oBdaudBYCZfMoXXQUENWZBS9geU0,6351
+pylint/pyreverse/vcgutils.py,sha256=5IsM9m21JZ5n-1edmqi1EHFK2ZYy2IX1WzfI8Uoe4Ts,6584
+pylint/pyreverse/writer.py,sha256=10wgUZ-8ANHgCvI2c1ZkukcAeZ-dfbJ_MWL75MoRCuI,8003
+pylint/reporters/__init__.py,sha256=dVj9-p_ireGR0jF2qxsu54a-0H_jxDafNiMUUVuHWH4,1650
+pylint/reporters/__pycache__/__init__.cpython-38.pyc,,
+pylint/reporters/__pycache__/base_reporter.cpython-38.pyc,,
+pylint/reporters/__pycache__/collecting_reporter.cpython-38.pyc,,
+pylint/reporters/__pycache__/json_reporter.cpython-38.pyc,,
+pylint/reporters/__pycache__/reports_handler_mix_in.cpython-38.pyc,,
+pylint/reporters/__pycache__/text.cpython-38.pyc,,
+pylint/reporters/base_reporter.py,sha256=HilNSVW_QAG34uY73b01eLlK-RPMgA62O4gk19rVIm8,2031
+pylint/reporters/collecting_reporter.py,sha256=tFJ0rsTNz9h4C0U5lgbe1jlQAOy0Of0WLGrq2QO7Sug,478
+pylint/reporters/json_reporter.py,sha256=JzakpeEjYZVFjtkpjmiU_die-zSugOONpSb6UHW91Ls,1996
+pylint/reporters/reports_handler_mix_in.py,sha256=t4Ucmk5Wa2CagNbEbehlr1n6H-4zmRnrCEvI57qVuig,2704
+pylint/reporters/text.py,sha256=tAMiL676mppZonTqGGUK9EAXv9q7rT_Np8GE0L_phj4,8087
+pylint/reporters/ureports/__init__.py,sha256=UteHDRjY2qMHwLUpA3McgaBMTdQnMTKrVRZ5A_WZybA,3235
+pylint/reporters/ureports/__pycache__/__init__.cpython-38.pyc,,
+pylint/reporters/ureports/__pycache__/nodes.cpython-38.pyc,,
+pylint/reporters/ureports/__pycache__/text_writer.cpython-38.pyc,,
+pylint/reporters/ureports/nodes.py,sha256=7ze_4YQHtNpx8wdsQHtU0sAMOSWVcIvaoKe4eVWnYJM,5193
+pylint/reporters/ureports/text_writer.py,sha256=SuKF0eKZcLvF457lXJyxFVk-M3epvAXCesKo6T2aBlU,3373
+pylint/testutils.py,sha256=bWQ0rsxptVTQRIEavbDahP_pzgy1Nbh4hYe5PicAu8w,20452
+pylint/utils/__init__.py,sha256=LR5PAyLJhLu7dDWCBkeCGN00MqPYz6pgkr_wFSrfqCA,2984
+pylint/utils/__pycache__/__init__.cpython-38.pyc,,
+pylint/utils/__pycache__/ast_walker.cpython-38.pyc,,
+pylint/utils/__pycache__/file_state.cpython-38.pyc,,
+pylint/utils/__pycache__/pragma_parser.cpython-38.pyc,,
+pylint/utils/__pycache__/utils.cpython-38.pyc,,
+pylint/utils/ast_walker.py,sha256=6ewsiRTz1W4v9UFm-lXMZoBWEe8b89kZRrSN7bW3aiM,2907
+pylint/utils/file_state.py,sha256=zw-4J5JasNZPhmcTbDUwiqaXpPWCIpPdYizvyGlnvCo,5962
+pylint/utils/pragma_parser.py,sha256=FuAydaJb_O6RMbVVyZg83eACKVuWxurtc5n1Gxz6E3Q,4799
+pylint/utils/utils.py,sha256=ahZtvd-OXZfJA_4l9KWoBb5TIfx_6zHdKvdSqZeNtns,14497
diff --git a/env/lib/python3.8/site-packages/pylint-2.6.0.dist-info/WHEEL b/env/lib/python3.8/site-packages/pylint-2.6.0.dist-info/WHEEL
new file mode 100644
index 000000000..83ff02e96
--- /dev/null
+++ b/env/lib/python3.8/site-packages/pylint-2.6.0.dist-info/WHEEL
@@ -0,0 +1,5 @@
+Wheel-Version: 1.0
+Generator: bdist_wheel (0.35.1)
+Root-Is-Purelib: true
+Tag: py3-none-any
+
diff --git a/env/lib/python3.8/site-packages/pylint-2.6.0.dist-info/entry_points.txt b/env/lib/python3.8/site-packages/pylint-2.6.0.dist-info/entry_points.txt
new file mode 100644
index 000000000..063b5e456
--- /dev/null
+++ b/env/lib/python3.8/site-packages/pylint-2.6.0.dist-info/entry_points.txt
@@ -0,0 +1,6 @@
+[console_scripts]
+epylint = pylint:run_epylint
+pylint = pylint:run_pylint
+pyreverse = pylint:run_pyreverse
+symilar = pylint:run_symilar
+
diff --git a/env/lib/python3.8/site-packages/pylint-2.6.0.dist-info/top_level.txt b/env/lib/python3.8/site-packages/pylint-2.6.0.dist-info/top_level.txt
new file mode 100644
index 000000000..7fb0ea150
--- /dev/null
+++ b/env/lib/python3.8/site-packages/pylint-2.6.0.dist-info/top_level.txt
@@ -0,0 +1 @@
+pylint
diff --git a/env/lib/python3.8/site-packages/pylint/__init__.py b/env/lib/python3.8/site-packages/pylint/__init__.py
new file mode 100644
index 000000000..8c4755818
--- /dev/null
+++ b/env/lib/python3.8/site-packages/pylint/__init__.py
@@ -0,0 +1,44 @@
+# Copyright (c) 2008, 2012 LOGILAB S.A. (Paris, FRANCE) 
+# Copyright (c) 2014, 2016-2020 Claudiu Popa 
+# Copyright (c) 2014 Arun Persaud 
+# Copyright (c) 2015 Ionel Cristian Maries 
+# Copyright (c) 2018 Nick Drozd 
+# Copyright (c) 2020 Pierre Sassoulas 
+
+# Licensed under the GPL: https://www.gnu.org/licenses/old-licenses/gpl-2.0.html
+# For details: https://github.com/PyCQA/pylint/blob/master/COPYING
+
+import sys
+
+from pylint.__pkginfo__ import version as __version__
+
+# pylint: disable=import-outside-toplevel
+
+
+def run_pylint():
+    from pylint.lint import Run as PylintRun
+
+    try:
+        PylintRun(sys.argv[1:])
+    except KeyboardInterrupt:
+        sys.exit(1)
+
+
+def run_epylint():
+    from pylint.epylint import Run as EpylintRun
+
+    EpylintRun()
+
+
+def run_pyreverse():
+    """run pyreverse"""
+    from pylint.pyreverse.main import Run as PyreverseRun
+
+    PyreverseRun(sys.argv[1:])
+
+
+def run_symilar():
+    """run symilar"""
+    from pylint.checkers.similar import Run as SimilarRun
+
+    SimilarRun(sys.argv[1:])
diff --git a/env/lib/python3.8/site-packages/pylint/__main__.py b/env/lib/python3.8/site-packages/pylint/__main__.py
new file mode 100644
index 000000000..badb5002a
--- /dev/null
+++ b/env/lib/python3.8/site-packages/pylint/__main__.py
@@ -0,0 +1,18 @@
+# Licensed under the GPL: https://www.gnu.org/licenses/old-licenses/gpl-2.0.html
+# For details: https://github.com/PyCQA/pylint/blob/master/COPYING
+
+#!/usr/bin/env python
+import os
+import sys
+
+import pylint
+
+# Strip out the current working directory from sys.path.
+# Having the working directory in `sys.path` means that `pylint` might
+# inadvertently import user code from modules having the same name as
+# stdlib or pylint's own modules.
+# CPython issue: https://bugs.python.org/issue33053
+if sys.path[0] == "" or sys.path[0] == os.getcwd():
+    sys.path.pop(0)
+
+pylint.run_pylint()
diff --git a/env/lib/python3.8/site-packages/pylint/__pkginfo__.py b/env/lib/python3.8/site-packages/pylint/__pkginfo__.py
new file mode 100644
index 000000000..11e46003d
--- /dev/null
+++ b/env/lib/python3.8/site-packages/pylint/__pkginfo__.py
@@ -0,0 +1,102 @@
+# -*- coding: utf-8 -*-
+# Copyright (c) 2006-2015 LOGILAB S.A. (Paris, FRANCE) 
+# Copyright (c) 2010 Julien Jehannet 
+# Copyright (c) 2013-2014 Google, Inc.
+# Copyright (c) 2014-2020 Claudiu Popa 
+# Copyright (c) 2014 Brett Cannon 
+# Copyright (c) 2014 Ricardo Gemignani 
+# Copyright (c) 2014 Arun Persaud 
+# Copyright (c) 2015 Ionel Cristian Maries 
+# Copyright (c) 2016 Moises Lopez 
+# Copyright (c) 2016 Florian Bruhin 
+# Copyright (c) 2016 Jakub Wilk 
+# Copyright (c) 2017-2018 Hugo 
+# Copyright (c) 2018-2019 Ashley Whetter 
+# Copyright (c) 2018 ssolanki 
+# Copyright (c) 2018 Sushobhit <31987769+sushobhit27@users.noreply.github.com>
+# Copyright (c) 2019-2020 Ville Skyttä 
+# Copyright (c) 2019 Hugo van Kemenade 
+# Copyright (c) 2019 Dan Hemberger <846186+hemberger@users.noreply.github.com>
+# Copyright (c) 2019 jab 
+# Copyright (c) 2020 hippo91 
+# Copyright (c) 2020 Damien Baty 
+# Copyright (c) 2020 Pierre Sassoulas 
+
+# Licensed under the GPL: https://www.gnu.org/licenses/old-licenses/gpl-2.0.html
+# For details: https://github.com/PyCQA/pylint/blob/master/COPYING
+
+# pylint: disable=redefined-builtin,invalid-name
+"""pylint packaging information"""
+
+
+from os.path import join
+
+# For an official release, use dev_version = None
+numversion = (2, 6, 0)
+dev_version = None
+
+version = ".".join(str(num) for num in numversion)
+if dev_version is not None:
+    version += "-dev" + str(dev_version)
+
+install_requires = [
+    "astroid>=2.4.0,<=2.5",
+    "isort>=4.2.5,<6",
+    "mccabe>=0.6,<0.7",
+    "toml>=0.7.1",
+]
+
+dependency_links = []  # type: ignore
+
+extras_require = {}
+extras_require[':sys_platform=="win32"'] = ["colorama"]
+
+license = "GPL"
+description = "python code static checker"
+web = "https://github.com/PyCQA/pylint"
+mailinglist = "mailto:code-quality@python.org"
+project_urls = {"What's New": "https://pylint.pycqa.org/en/latest/whatsnew/"}
+author = "Python Code Quality Authority"
+author_email = "code-quality@python.org"
+
+classifiers = [
+    "Development Status :: 6 - Mature",
+    "Environment :: Console",
+    "Intended Audience :: Developers",
+    "License :: OSI Approved :: GNU General Public License (GPL)",
+    "Operating System :: OS Independent",
+    "Programming Language :: Python",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.5",
+    "Programming Language :: Python :: 3.6",
+    "Programming Language :: Python :: 3.7",
+    "Programming Language :: Python :: 3.8",
+    "Programming Language :: Python :: 3 :: Only",
+    "Programming Language :: Python :: Implementation :: CPython",
+    "Programming Language :: Python :: Implementation :: PyPy",
+    "Topic :: Software Development :: Debuggers",
+    "Topic :: Software Development :: Quality Assurance",
+    "Topic :: Software Development :: Testing",
+]
+
+
+long_desc = """\
+ Pylint is a Python source code analyzer which looks for programming
+ errors, helps enforcing a coding standard and sniffs for some code
+ smells (as defined in Martin Fowler's Refactoring book)
+ .
+ Pylint can be seen as another PyChecker since nearly all tests you
+ can do with PyChecker can also be done with Pylint. However, Pylint
+ offers some more features, like checking length of lines of code,
+ checking if variable names are well-formed according to your coding
+ standard, or checking if declared interfaces are truly implemented,
+ and much more.
+ .
+ Additionally, it is possible to write plugins to add your own checks.
+ .
+ Pylint is shipped with "pyreverse" (UML diagram generator)
+ and "symilar" (an independent similarities checker)."""
+
+scripts = [
+    join("bin", filename) for filename in ("pylint", "symilar", "epylint", "pyreverse")
+]
diff --git a/env/lib/python3.8/site-packages/pylint/checkers/__init__.py b/env/lib/python3.8/site-packages/pylint/checkers/__init__.py
new file mode 100644
index 000000000..921a54a2f
--- /dev/null
+++ b/env/lib/python3.8/site-packages/pylint/checkers/__init__.py
@@ -0,0 +1,67 @@
+# Copyright (c) 2006-2014 LOGILAB S.A. (Paris, FRANCE) 
+# Copyright (c) 2013-2014 Google, Inc.
+# Copyright (c) 2013 buck@yelp.com 
+# Copyright (c) 2014-2018, 2020 Claudiu Popa 
+# Copyright (c) 2014 Brett Cannon 
+# Copyright (c) 2014 Arun Persaud 
+# Copyright (c) 2015 Ionel Cristian Maries 
+# Copyright (c) 2016 Moises Lopez 
+# Copyright (c) 2017-2018 Bryce Guinta 
+# Copyright (c) 2018-2019 Pierre Sassoulas 
+# Copyright (c) 2018 ssolanki 
+# Copyright (c) 2019 Bruno P. Kinoshita 
+
+# Licensed under the GPL: https://www.gnu.org/licenses/old-licenses/gpl-2.0.html
+# For details: https://github.com/PyCQA/pylint/blob/master/COPYING
+
+"""utilities methods and classes for checkers
+
+Base id of standard checkers (used in msg and report ids):
+01: base
+02: classes
+03: format
+04: import
+05: misc
+06: variables
+07: exceptions
+08: similar
+09: design_analysis
+10: newstyle
+11: typecheck
+12: logging
+13: string_format
+14: string_constant
+15: stdlib
+16: python3
+17: refactoring
+18-50: not yet used: reserved for future internal checkers.
+51-99: perhaps used: reserved for external checkers
+
+The raw_metrics checker has no number associated since it doesn't emit any
+messages nor reports. XXX not true, emit a 07 report !
+
+"""
+
+from pylint.checkers.base_checker import BaseChecker, BaseTokenChecker
+from pylint.utils import register_plugins
+
+
+def table_lines_from_stats(stats, _, columns):
+    """get values listed in  from  and ,
+    and return a formated list of values, designed to be given to a
+    ureport.Table object
+    """
+    lines = []
+    for m_type in columns:
+        new = stats[m_type]
+        new = "%.3f" % new if isinstance(new, float) else str(new)
+        lines += (m_type.replace("_", " "), new, "NC", "NC")
+    return lines
+
+
+def initialize(linter):
+    """initialize linter with checkers in this package """
+    register_plugins(linter, __path__[0])
+
+
+__all__ = ("BaseChecker", "BaseTokenChecker", "initialize")
diff --git a/env/lib/python3.8/site-packages/pylint/checkers/async.py b/env/lib/python3.8/site-packages/pylint/checkers/async.py
new file mode 100644
index 000000000..1b581c0f1
--- /dev/null
+++ b/env/lib/python3.8/site-packages/pylint/checkers/async.py
@@ -0,0 +1,90 @@
+# Copyright (c) 2015-2018, 2020 Claudiu Popa 
+# Copyright (c) 2017 Derek Gustafson 
+# Copyright (c) 2019 Pierre Sassoulas 
+
+# Licensed under the GPL: https://www.gnu.org/licenses/old-licenses/gpl-2.0.html
+# For details: https://github.com/PyCQA/pylint/blob/master/COPYING
+
+"""Checker for anything related to the async protocol (PEP 492)."""
+
+import sys
+
+import astroid
+from astroid import bases, exceptions
+
+from pylint import checkers, interfaces, utils
+from pylint.checkers import utils as checker_utils
+from pylint.checkers.utils import decorated_with
+
+
+class AsyncChecker(checkers.BaseChecker):
+    __implements__ = interfaces.IAstroidChecker
+    name = "async"
+    msgs = {
+        "E1700": (
+            "Yield inside async function",
+            "yield-inside-async-function",
+            "Used when an `yield` or `yield from` statement is "
+            "found inside an async function.",
+            {"minversion": (3, 5)},
+        ),
+        "E1701": (
+            "Async context manager '%s' doesn't implement __aenter__ and __aexit__.",
+            "not-async-context-manager",
+            "Used when an async context manager is used with an object "
+            "that does not implement the async context management protocol.",
+            {"minversion": (3, 5)},
+        ),
+    }
+
+    def open(self):
+        self._ignore_mixin_members = utils.get_global_option(
+            self, "ignore-mixin-members"
+        )
+        self._async_generators = ["contextlib.asynccontextmanager"]
+
+    @checker_utils.check_messages("yield-inside-async-function")
+    def visit_asyncfunctiondef(self, node):
+        for child in node.nodes_of_class(astroid.Yield):
+            if child.scope() is node and (
+                sys.version_info[:2] == (3, 5) or isinstance(child, astroid.YieldFrom)
+            ):
+                self.add_message("yield-inside-async-function", node=child)
+
+    @checker_utils.check_messages("not-async-context-manager")
+    def visit_asyncwith(self, node):
+        for ctx_mgr, _ in node.items:
+            inferred = checker_utils.safe_infer(ctx_mgr)
+            if inferred is None or inferred is astroid.Uninferable:
+                continue
+
+            if isinstance(inferred, bases.AsyncGenerator):
+                # Check if we are dealing with a function decorated
+                # with contextlib.asynccontextmanager.
+                if decorated_with(inferred.parent, self._async_generators):
+                    continue
+            else:
+                try:
+                    inferred.getattr("__aenter__")
+                    inferred.getattr("__aexit__")
+                except exceptions.NotFoundError:
+                    if isinstance(inferred, astroid.Instance):
+                        # If we do not know the bases of this class,
+                        # just skip it.
+                        if not checker_utils.has_known_bases(inferred):
+                            continue
+                        # Just ignore mixin classes.
+                        if self._ignore_mixin_members:
+                            if inferred.name[-5:].lower() == "mixin":
+                                continue
+                else:
+                    continue
+
+            self.add_message(
+                "not-async-context-manager", node=node, args=(inferred.name,)
+            )
+
+
+def register(linter):
+    """required method to auto register this checker"""
+    linter.register_checker(AsyncChecker(linter))
diff --git a/env/lib/python3.8/site-packages/pylint/checkers/base.py b/env/lib/python3.8/site-packages/pylint/checkers/base.py
new file mode 100644
index 000000000..d830a68ba
--- /dev/null
+++ b/env/lib/python3.8/site-packages/pylint/checkers/base.py
@@ -0,0 +1,2507 @@
+# -*- coding: utf-8 -*-
+# Copyright (c) 2006-2016 LOGILAB S.A. (Paris, FRANCE) 
+# Copyright (c) 2010 Daniel Harding 
+# Copyright (c) 2012-2014 Google, Inc.
+# Copyright (c) 2013-2020 Claudiu Popa 
+# Copyright (c) 2014 Brett Cannon 
+# Copyright (c) 2014 Arun Persaud 
+# Copyright (c) 2015 Nick Bastin 
+# Copyright (c) 2015 Michael Kefeder 
+# Copyright (c) 2015 Dmitry Pribysh 
+# Copyright (c) 2015 Stephane Wirtel 
+# Copyright (c) 2015 Cosmin Poieana 
+# Copyright (c) 2015 Florian Bruhin 
+# Copyright (c) 2015 Radu Ciorba 
+# Copyright (c) 2015 Ionel Cristian Maries 
+# Copyright (c) 2016, 2019 Ashley Whetter 
+# Copyright (c) 2016, 2018 Jakub Wilk 
+# Copyright (c) 2016-2017 Łukasz Rogalski 
+# Copyright (c) 2016 Glenn Matthews 
+# Copyright (c) 2016 Elias Dorneles 
+# Copyright (c) 2016 Yannack 
+# Copyright (c) 2016 Alex Jurkiewicz 
+# Copyright (c) 2017 Pierre Sassoulas 
+# Copyright (c) 2017, 2019 hippo91 
+# Copyright (c) 2017 danields 
+# Copyright (c) 2017 Jacques Kvam 
+# Copyright (c) 2017 ttenhoeve-aa 
+# Copyright (c) 2018-2019 Nick Drozd 
+# Copyright (c) 2018-2019 Ville Skyttä 
+# Copyright (c) 2018 Sergei Lebedev <185856+superbobry@users.noreply.github.com>
+# Copyright (c) 2018 Lucas Cimon 
+# Copyright (c) 2018 ssolanki 
+# Copyright (c) 2018 Natalie Serebryakova 
+# Copyright (c) 2018 Sushobhit <31987769+sushobhit27@users.noreply.github.com>
+# Copyright (c) 2018 SergeyKosarchuk 
+# Copyright (c) 2018 Steven M. Vascellaro 
+# Copyright (c) 2018 Mike Frysinger 
+# Copyright (c) 2018 Chris Lamb 
+# Copyright (c) 2018 glmdgrielson <32415403+glmdgrielson@users.noreply.github.com>
+# Copyright (c) 2019 Daniel Draper 
+# Copyright (c) 2019 Hugo van Kemenade 
+# Copyright (c) 2019 Pierre Sassoulas 
+# Copyright (c) 2019 Niko Wenselowski 
+# Copyright (c) 2019 Nikita Sobolev 
+# Copyright (c) 2019 Oisín Moran 
+# Copyright (c) 2019 Fantix King 
+# Copyright (c) 2020 Damien Baty 
+# Copyright (c) 2020 Ram Rachum 
+# Copyright (c) 2020 Anthony Sottile 
+# Copyright (c) 2020 bernie gray 
+# Copyright (c) 2020 Gabriel R Sezefredo 
+# Copyright (c) 2020 Benny 
+# Copyright (c) 2020 Anubhav <35621759+anubh-v@users.noreply.github.com>
+
+# Licensed under the GPL: https://www.gnu.org/licenses/old-licenses/gpl-2.0.html
+# For details: https://github.com/PyCQA/pylint/blob/master/COPYING
+
+"""basic checker for Python code"""
+import builtins
+import collections
+import itertools
+import re
+import sys
+from typing import Pattern
+
+import astroid
+import astroid.bases
+import astroid.scoped_nodes
+from astroid.arguments import CallSite
+
+from pylint import checkers, exceptions, interfaces
+from pylint import utils as lint_utils
+from pylint.checkers import utils
+from pylint.checkers.utils import (
+    is_overload_stub,
+    is_property_deleter,
+    is_property_setter,
+)
+from pylint.reporters.ureports import nodes as reporter_nodes
+
+
+class NamingStyle:
+    # It may seem counterintuitive that single naming style
+    # has multiple "accepted" forms of regular expressions,
+    # but we need to special-case stuff like dunder names
+    # in method names.
+    CLASS_NAME_RGX = None  # type: Pattern[str]
+    MOD_NAME_RGX = None  # type: Pattern[str]
+    CONST_NAME_RGX = None  # type: Pattern[str]
+    COMP_VAR_RGX = None  # type: Pattern[str]
+    DEFAULT_NAME_RGX = None  # type: Pattern[str]
+    CLASS_ATTRIBUTE_RGX = None  # type: Pattern[str]
+
+    @classmethod
+    def get_regex(cls, name_type):
+        return {
+            "module": cls.MOD_NAME_RGX,
+            "const": cls.CONST_NAME_RGX,
+            "class": cls.CLASS_NAME_RGX,
+            "function": cls.DEFAULT_NAME_RGX,
+            "method": cls.DEFAULT_NAME_RGX,
+            "attr": cls.DEFAULT_NAME_RGX,
+            "argument": cls.DEFAULT_NAME_RGX,
+            "variable": cls.DEFAULT_NAME_RGX,
+            "class_attribute": cls.CLASS_ATTRIBUTE_RGX,
+            "inlinevar": cls.COMP_VAR_RGX,
+        }[name_type]
+
+
+class SnakeCaseStyle(NamingStyle):
+    """Regex rules for snake_case naming style."""
+
+    CLASS_NAME_RGX = re.compile(r"[^\W\dA-Z][^\WA-Z]+$")
+    MOD_NAME_RGX = re.compile(r"[^\W\dA-Z][^\WA-Z]*$")
+    CONST_NAME_RGX = re.compile(r"([^\W\dA-Z][^\WA-Z]*|__.*__)$")
+    COMP_VAR_RGX = re.compile(r"[^\W\dA-Z][^\WA-Z]*$")
+    DEFAULT_NAME_RGX = re.compile(
+        r"([^\W\dA-Z][^\WA-Z]{2,}|_[^\WA-Z]*|__[^\WA-Z\d_][^\WA-Z]+__)$"
+    )
+    CLASS_ATTRIBUTE_RGX = re.compile(r"([^\W\dA-Z][^\WA-Z]{2,}|__.*__)$")
+
+
+class CamelCaseStyle(NamingStyle):
+    """Regex rules for camelCase naming style."""
+
+    CLASS_NAME_RGX = re.compile(r"[^\W\dA-Z][^\W_]+$")
+    MOD_NAME_RGX = re.compile(r"[^\W\dA-Z][^\W_]*$")
+    CONST_NAME_RGX = re.compile(r"([^\W\dA-Z][^\W_]*|__.*__)$")
+    COMP_VAR_RGX = re.compile(r"[^\W\dA-Z][^\W_]*$")
+    DEFAULT_NAME_RGX = re.compile(r"([^\W\dA-Z][^\W_]{2,}|__[^\W\dA-Z_]\w+__)$")
+    CLASS_ATTRIBUTE_RGX = re.compile(r"([^\W\dA-Z][^\W_]{2,}|__.*__)$")
+
+
+class PascalCaseStyle(NamingStyle):
+    """Regex rules for PascalCase naming style."""
+
+    CLASS_NAME_RGX = re.compile(r"[^\W\da-z][^\W_]+$")
+    MOD_NAME_RGX = re.compile(r"[^\W\da-z][^\W_]+$")
+    CONST_NAME_RGX = re.compile(r"([^\W\da-z][^\W_]*|__.*__)$")
+    COMP_VAR_RGX = re.compile(r"[^\W\da-z][^\W_]+$")
+    DEFAULT_NAME_RGX = re.compile(r"([^\W\da-z][^\W_]{2,}|__[^\W\dA-Z_]\w+__)$")
+    CLASS_ATTRIBUTE_RGX = re.compile(r"[^\W\da-z][^\W_]{2,}$")
+
+
+class UpperCaseStyle(NamingStyle):
+    """Regex rules for UPPER_CASE naming style."""
+
+    CLASS_NAME_RGX = re.compile(r"[^\W\da-z][^\Wa-z]+$")
+    MOD_NAME_RGX = re.compile(r"[^\W\da-z][^\Wa-z]+$")
+    CONST_NAME_RGX = re.compile(r"([^\W\da-z][^\Wa-z]*|__.*__)$")
+    COMP_VAR_RGX = re.compile(r"[^\W\da-z][^\Wa-z]+$")
+    DEFAULT_NAME_RGX = re.compile(r"([^\W\da-z][^\Wa-z]{2,}|__[^\W\dA-Z_]\w+__)$")
+    CLASS_ATTRIBUTE_RGX = re.compile(r"[^\W\da-z][^\Wa-z]{2,}$")
+
+
+class AnyStyle(NamingStyle):
+    @classmethod
+    def get_regex(cls, name_type):
+        return re.compile(".*")
+
+
+NAMING_STYLES = {
+    "snake_case": SnakeCaseStyle,
+    "camelCase": CamelCaseStyle,
+    "PascalCase": PascalCaseStyle,
+    "UPPER_CASE": UpperCaseStyle,
+    "any": AnyStyle,
+}
+
+# do not require a doc string on private/system methods
+NO_REQUIRED_DOC_RGX = re.compile("^_")
+REVERSED_PROTOCOL_METHOD = "__reversed__"
+SEQUENCE_PROTOCOL_METHODS = ("__getitem__", "__len__")
+REVERSED_METHODS = (SEQUENCE_PROTOCOL_METHODS, (REVERSED_PROTOCOL_METHOD,))
+TYPECHECK_COMPARISON_OPERATORS = frozenset(("is", "is not", "==", "!="))
+LITERAL_NODE_TYPES = (astroid.Const, astroid.Dict, astroid.List, astroid.Set)
+UNITTEST_CASE = "unittest.case"
+BUILTINS = builtins.__name__
+TYPE_QNAME = "%s.type" % BUILTINS
+ABC_METACLASSES = {"_py_abc.ABCMeta", "abc.ABCMeta"}  # Python 3.7+,
+
+# Name categories that are always consistent with all naming conventions.
+EXEMPT_NAME_CATEGORIES = {"exempt", "ignore"}
+
+# A mapping from qname -> symbol, to be used when generating messages
+# about dangerous default values as arguments
+DEFAULT_ARGUMENT_SYMBOLS = dict(
+    zip(
+        [".".join([BUILTINS, x]) for x in ("set", "dict", "list")],
+        ["set()", "{}", "[]"],
+    ),
+    **{
+        x: "%s()" % x
+        for x in (
+            "collections.deque",
+            "collections.ChainMap",
+            "collections.Counter",
+            "collections.OrderedDict",
+            "collections.defaultdict",
+            "collections.UserDict",
+            "collections.UserList",
+        )
+    },
+)
+REVERSED_COMPS = {"<": ">", "<=": ">=", ">": "<", ">=": "<="}
+COMPARISON_OPERATORS = frozenset(("==", "!=", "<", ">", "<=", ">="))
+# List of methods which can be redefined
+REDEFINABLE_METHODS = frozenset(("__module__",))
+TYPING_FORWARD_REF_QNAME = "typing.ForwardRef"
+
+
+def _redefines_import(node):
+    """ Detect that the given node (AssignName) is inside an
+    exception handler and redefines an import from the tryexcept body.
+    Returns True if the node redefines an import, False otherwise.
+    """
+    current = node
+    while current and not isinstance(current.parent, astroid.ExceptHandler):
+        current = current.parent
+    if not current or not utils.error_of_type(current.parent, ImportError):
+        return False
+    try_block = current.parent.parent
+    for import_node in try_block.nodes_of_class((astroid.ImportFrom, astroid.Import)):
+        for name, alias in import_node.names:
+            if alias:
+                if alias == node.name:
+                    return True
+            elif name == node.name:
+                return True
+    return False
+
+
+def in_loop(node):
+    """return True if the node is inside a kind of for loop"""
+    parent = node.parent
+    while parent is not None:
+        if isinstance(
+            parent,
+            (
+                astroid.For,
+                astroid.ListComp,
+                astroid.SetComp,
+                astroid.DictComp,
+                astroid.GeneratorExp,
+            ),
+        ):
+            return True
+        parent = parent.parent
+    return False
+
+
+def in_nested_list(nested_list, obj):
+    """return true if the object is an element of  or of a nested
+    list
+    """
+    for elmt in nested_list:
+        if isinstance(elmt, (list, tuple)):
+            if in_nested_list(elmt, obj):
+                return True
+        elif elmt == obj:
+            return True
+    return False
+
+
+def _get_break_loop_node(break_node):
+    """
+    Returns the loop node that holds the break node in arguments.
+
+    Args:
+        break_node (astroid.Break): the break node of interest.
+
+    Returns:
+        astroid.For or astroid.While: the loop node holding the break node.
+    """
+    loop_nodes = (astroid.For, astroid.While)
+    parent = break_node.parent
+    while not isinstance(parent, loop_nodes) or break_node in getattr(
+        parent, "orelse", []
+    ):
+        break_node = parent
+        parent = parent.parent
+        if parent is None:
+            break
+    return parent
+
+
+def _loop_exits_early(loop):
+    """
+    Returns true if a loop may ends up in a break statement.
+
+    Args:
+        loop (astroid.For, astroid.While): the loop node inspected.
+
+    Returns:
+        bool: True if the loop may ends up in a break statement, False otherwise.
+    """
+    loop_nodes = (astroid.For, astroid.While)
+    definition_nodes = (astroid.FunctionDef, astroid.ClassDef)
+    inner_loop_nodes = [
+        _node
+        for _node in loop.nodes_of_class(loop_nodes, skip_klass=definition_nodes)
+        if _node != loop
+    ]
+    return any(
+        _node
+        for _node in loop.nodes_of_class(astroid.Break, skip_klass=definition_nodes)
+        if _get_break_loop_node(_node) not in inner_loop_nodes
+    )
+
+
+def _is_multi_naming_match(match, node_type, confidence):
+    return (
+        match is not None
+        and match.lastgroup is not None
+        and match.lastgroup not in EXEMPT_NAME_CATEGORIES
+        and (node_type != "method" or confidence != interfaces.INFERENCE_FAILURE)
+    )
+
+
+BUILTIN_PROPERTY = "builtins.property"
+
+
+def _get_properties(config):
+    """Returns a tuple of property classes and names.
+
+    Property classes are fully qualified, such as 'abc.abstractproperty' and
+    property names are the actual names, such as 'abstract_property'.
+    """
+    property_classes = {BUILTIN_PROPERTY}
+    property_names = set()  # Not returning 'property', it has its own check.
+    if config is not None:
+        property_classes.update(config.property_classes)
+        property_names.update(
+            prop.rsplit(".", 1)[-1] for prop in config.property_classes
+        )
+    return property_classes, property_names
+
+
+def _determine_function_name_type(node: astroid.FunctionDef, config=None):
+    """Determine the name type whose regex the a function's name should match.
+
+    :param node: A function node.
+    :param config: Configuration from which to pull additional property classes.
+    :type config: :class:`optparse.Values`
+
+    :returns: One of ('function', 'method', 'attr')
+    :rtype: str
+    """
+    property_classes, property_names = _get_properties(config)
+    if not node.is_method():
+        return "function"
+
+    if is_property_setter(node) or is_property_deleter(node):
+        # If the function is decorated using the prop_method.{setter,getter}
+        # form, treat it like an attribute as well.
+        return "attr"
+
+    if node.decorators:
+        decorators = node.decorators.nodes
+    else:
+        decorators = []
+    for decorator in decorators:
+        # If the function is a property (decorated with @property
+        # or @abc.abstractproperty), the name type is 'attr'.
+        if isinstance(decorator, astroid.Name) or (
+            isinstance(decorator, astroid.Attribute)
+            and decorator.attrname in property_names
+        ):
+            inferred = utils.safe_infer(decorator)
+            if (
+                inferred
+                and hasattr(inferred, "qname")
+                and inferred.qname() in property_classes
+            ):
+                return "attr"
+    return "method"
+
+
+def _has_abstract_methods(node):
+    """
+    Determine if the given `node` has abstract methods.
+
+    The methods should be made abstract by decorating them
+    with `abc` decorators.
+    """
+    return len(utils.unimplemented_abstract_methods(node)) > 0
+
+
+def report_by_type_stats(sect, stats, _):
+    """make a report of
+
+    * percentage of different types documented
+    * percentage of different types with a bad name
+    """
+    # percentage of different types documented and/or with a bad name
+    nice_stats = {}
+    for node_type in ("module", "class", "method", "function"):
+        try:
+            total = stats[node_type]
+        except KeyError as e:
+            raise exceptions.EmptyReportError() from e
+        nice_stats[node_type] = {}
+        if total != 0:
+            try:
+                documented = total - stats["undocumented_" + node_type]
+                percent = (documented * 100.0) / total
+                nice_stats[node_type]["percent_documented"] = "%.2f" % percent
+            except KeyError:
+                nice_stats[node_type]["percent_documented"] = "NC"
+            try:
+                percent = (stats["badname_" + node_type] * 100.0) / total
+                nice_stats[node_type]["percent_badname"] = "%.2f" % percent
+            except KeyError:
+                nice_stats[node_type]["percent_badname"] = "NC"
+    lines = ("type", "number", "old number", "difference", "%documented", "%badname")
+    for node_type in ("module", "class", "method", "function"):
+        new = stats[node_type]
+        lines += (
+            node_type,
+            str(new),
+            "NC",
+            "NC",
+            nice_stats[node_type].get("percent_documented", "0"),
+            nice_stats[node_type].get("percent_badname", "0"),
+        )
+    sect.append(reporter_nodes.Table(children=lines, cols=6, rheaders=1))
+
+
+def redefined_by_decorator(node):
+    """return True if the object is a method redefined via decorator.
+
+    For example:
+        @property
+        def x(self): return self._x
+        @x.setter
+        def x(self, value): self._x = value
+    """
+    if node.decorators:
+        for decorator in node.decorators.nodes:
+            if (
+                isinstance(decorator, astroid.Attribute)
+                and getattr(decorator.expr, "name", None) == node.name
+            ):
+                return True
+    return False
+
+
+class _BasicChecker(checkers.BaseChecker):
+    __implements__ = interfaces.IAstroidChecker
+    name = "basic"
+
+
+class BasicErrorChecker(_BasicChecker):
+    msgs = {
+        "E0100": (
+            "__init__ method is a generator",
+            "init-is-generator",
+            "Used when the special class method __init__ is turned into a "
+            "generator by a yield in its body.",
+        ),
+        "E0101": (
+            "Explicit return in __init__",
+            "return-in-init",
+            "Used when the special class method __init__ has an explicit "
+            "return value.",
+        ),
+        "E0102": (
+            "%s already defined line %s",
+            "function-redefined",
+            "Used when a function / class / method is redefined.",
+        ),
+        "E0103": (
+            "%r not properly in loop",
+            "not-in-loop",
+            "Used when break or continue keywords are used outside a loop.",
+        ),
+        "E0104": (
+            "Return outside function",
+            "return-outside-function",
+            'Used when a "return" statement is found outside a function or method.',
+        ),
+        "E0105": (
+            "Yield outside function",
+            "yield-outside-function",
+            'Used when a "yield" statement is found outside a function or method.',
+        ),
+        "E0106": (
+            "Return with argument inside generator",
+            "return-arg-in-generator",
+            'Used when a "return" statement with an argument is found '
+            "outside in a generator function or method (e.g. with some "
+            '"yield" statements).',
+            {"maxversion": (3, 3)},
+        ),
+        "E0107": (
+            "Use of the non-existent %s operator",
+            "nonexistent-operator",
+            "Used when you attempt to use the C-style pre-increment or "
+            "pre-decrement operator -- and ++, which doesn't exist in Python.",
+        ),
+        "E0108": (
+            "Duplicate argument name %s in function definition",
+            "duplicate-argument-name",
+            "Duplicate argument names in function definitions are syntax errors.",
+        ),
+        "E0110": (
+            "Abstract class %r with abstract methods instantiated",
+            "abstract-class-instantiated",
+            "Used when an abstract class with `abc.ABCMeta` as metaclass "
+            "has abstract methods and is instantiated.",
+        ),
+        "W0120": (
+            "Else clause on loop without a break statement",
+            "useless-else-on-loop",
+            "Loops should only have an else clause if they can exit early "
+            "with a break statement, otherwise the statements under else "
+            "should be on the same scope as the loop itself.",
+        ),
+        "E0112": (
+            "More than one starred expression in assignment",
+            "too-many-star-expressions",
+            "Emitted when there are more than one starred "
+            "expressions (`*x`) in an assignment. This is a SyntaxError.",
+        ),
+        "E0113": (
+            "Starred assignment target must be in a list or tuple",
+            "invalid-star-assignment-target",
+            "Emitted when a star expression is used as a starred assignment target.",
+        ),
+        "E0114": (
+            "Can use starred expression only in assignment target",
+            "star-needs-assignment-target",
+            "Emitted when a star expression is not used in an assignment target.",
+        ),
+        "E0115": (
+            "Name %r is nonlocal and global",
+            "nonlocal-and-global",
+            "Emitted when a name is both nonlocal and global.",
+        ),
+        "E0116": (
+            "'continue' not supported inside 'finally' clause",
+            "continue-in-finally",
+            "Emitted when the `continue` keyword is found "
+            "inside a finally clause, which is a SyntaxError.",
+            {"maxversion": (3, 8)},
+        ),
+        "E0117": (
+            "nonlocal name %s found without binding",
+            "nonlocal-without-binding",
+            "Emitted when a nonlocal variable does not have an attached "
+            "name somewhere in the parent scopes",
+        ),
+        "E0118": (
+            "Name %r is used prior to global declaration",
+            "used-prior-global-declaration",
+            "Emitted when a name is used prior a global declaration, "
+            "which results in an error since Python 3.6.",
+            {"minversion": (3, 6)},
+        ),
+    }
+
+    @utils.check_messages("function-redefined")
+    def visit_classdef(self, node):
+        self._check_redefinition("class", node)
+
+    def _too_many_starred_for_tuple(self, assign_tuple):
+        starred_count = 0
+        for elem in assign_tuple.itered():
+            if isinstance(elem, astroid.Tuple):
+                return self._too_many_starred_for_tuple(elem)
+            if isinstance(elem, astroid.Starred):
+                starred_count += 1
+        return starred_count > 1
+
+    @utils.check_messages("too-many-star-expressions", "invalid-star-assignment-target")
+    def visit_assign(self, node):
+        # Check *a, *b = ...
+        assign_target = node.targets[0]
+        # Check *a = b
+        if isinstance(node.targets[0], astroid.Starred):
+            self.add_message("invalid-star-assignment-target", node=node)
+
+        if not isinstance(assign_target, astroid.Tuple):
+            return
+        if self._too_many_starred_for_tuple(assign_target):
+            self.add_message("too-many-star-expressions", node=node)
+
+    @utils.check_messages("star-needs-assignment-target")
+    def visit_starred(self, node):
+        """Check that a Starred expression is used in an assignment target."""
+        if isinstance(node.parent, astroid.Call):
+            # f(*args) is converted to Call(args=[Starred]), so ignore
+            # them for this check.
+            return
+        if isinstance(
+            node.parent, (astroid.List, astroid.Tuple, astroid.Set, astroid.Dict)
+        ):
+            # PEP 448 unpacking.
+            return
+
+        stmt = node.statement()
+        if not isinstance(stmt, astroid.Assign):
+            return
+
+        if stmt.value is node or stmt.value.parent_of(node):
+            self.add_message("star-needs-assignment-target", node=node)
+
+    @utils.check_messages(
+        "init-is-generator",
+        "return-in-init",
+        "function-redefined",
+        "return-arg-in-generator",
+        "duplicate-argument-name",
+        "nonlocal-and-global",
+        "used-prior-global-declaration",
+    )
+    def visit_functiondef(self, node):
+        self._check_nonlocal_and_global(node)
+        self._check_name_used_prior_global(node)
+        if not redefined_by_decorator(
+            node
+        ) and not utils.is_registered_in_singledispatch_function(node):
+            self._check_redefinition(node.is_method() and "method" or "function", node)
+        # checks for max returns, branch, return in __init__
+        returns = node.nodes_of_class(
+            astroid.Return, skip_klass=(astroid.FunctionDef, astroid.ClassDef)
+        )
+        if node.is_method() and node.name == "__init__":
+            if node.is_generator():
+                self.add_message("init-is-generator", node=node)
+            else:
+                values = [r.value for r in returns]
+                # Are we returning anything but None from constructors
+                if any(v for v in values if not utils.is_none(v)):
+                    self.add_message("return-in-init", node=node)
+        # Check for duplicate names by clustering args with same name for detailed report
+        arg_clusters = collections.defaultdict(list)
+        arguments = filter(None, [node.args.args, node.args.kwonlyargs])
+
+        for arg in itertools.chain.from_iterable(arguments):
+            arg_clusters[arg.name].append(arg)
+
+        # provide detailed report about each repeated argument
+        for argument_duplicates in arg_clusters.values():
+            if len(argument_duplicates) != 1:
+                for argument in argument_duplicates:
+                    self.add_message(
+                        "duplicate-argument-name",
+                        line=argument.lineno,
+                        node=argument,
+                        args=(argument.name,),
+                    )
+
+    visit_asyncfunctiondef = visit_functiondef
+
+    def _check_name_used_prior_global(self, node):
+
+        scope_globals = {
+            name: child
+            for child in node.nodes_of_class(astroid.Global)
+            for name in child.names
+            if child.scope() is node
+        }
+
+        if not scope_globals:
+            return
+
+        for node_name in node.nodes_of_class(astroid.Name):
+            if node_name.scope() is not node:
+                continue
+
+            name = node_name.name
+            corresponding_global = scope_globals.get(name)
+            if not corresponding_global:
+                continue
+
+            global_lineno = corresponding_global.fromlineno
+            if global_lineno and global_lineno > node_name.fromlineno:
+                self.add_message(
+                    "used-prior-global-declaration", node=node_name, args=(name,)
+                )
+
+    def _check_nonlocal_and_global(self, node):
+        """Check that a name is both nonlocal and global."""
+
+        def same_scope(current):
+            return current.scope() is node
+
+        from_iter = itertools.chain.from_iterable
+        nonlocals = set(
+            from_iter(
+                child.names
+                for child in node.nodes_of_class(astroid.Nonlocal)
+                if same_scope(child)
+            )
+        )
+
+        if not nonlocals:
+            return
+
+        global_vars = set(
+            from_iter(
+                child.names
+                for child in node.nodes_of_class(astroid.Global)
+                if same_scope(child)
+            )
+        )
+        for name in nonlocals.intersection(global_vars):
+            self.add_message("nonlocal-and-global", args=(name,), node=node)
+
+    @utils.check_messages("return-outside-function")
+    def visit_return(self, node):
+        if not isinstance(node.frame(), astroid.FunctionDef):
+            self.add_message("return-outside-function", node=node)
+
+    @utils.check_messages("yield-outside-function")
+    def visit_yield(self, node):
+        self._check_yield_outside_func(node)
+
+    @utils.check_messages("yield-outside-function")
+    def visit_yieldfrom(self, node):
+        self._check_yield_outside_func(node)
+
+    @utils.check_messages("not-in-loop", "continue-in-finally")
+    def visit_continue(self, node):
+        self._check_in_loop(node, "continue")
+
+    @utils.check_messages("not-in-loop")
+    def visit_break(self, node):
+        self._check_in_loop(node, "break")
+
+    @utils.check_messages("useless-else-on-loop")
+    def visit_for(self, node):
+        self._check_else_on_loop(node)
+
+    @utils.check_messages("useless-else-on-loop")
+    def visit_while(self, node):
+        self._check_else_on_loop(node)
+
+    @utils.check_messages("nonexistent-operator")
+    def visit_unaryop(self, node):
+        """check use of the non-existent ++ and -- operator operator"""
+        if (
+            (node.op in "+-")
+            and isinstance(node.operand, astroid.UnaryOp)
+            and (node.operand.op == node.op)
+        ):
+            self.add_message("nonexistent-operator", node=node, args=node.op * 2)
+
+    def _check_nonlocal_without_binding(self, node, name):
+        current_scope = node.scope()
+        while True:
+            if current_scope.parent is None:
+                break
+
+            if not isinstance(current_scope, (astroid.ClassDef, astroid.FunctionDef)):
+                self.add_message("nonlocal-without-binding", args=(name,), node=node)
+                return
+
+            if name not in current_scope.locals:
+                current_scope = current_scope.parent.scope()
+                continue
+
+            # Okay, found it.
+            return
+
+        if not isinstance(current_scope, astroid.FunctionDef):
+            self.add_message("nonlocal-without-binding", args=(name,), node=node)
+
+    @utils.check_messages("nonlocal-without-binding")
+    def visit_nonlocal(self, node):
+        for name in node.names:
+            self._check_nonlocal_without_binding(node, name)
+
+    @utils.check_messages("abstract-class-instantiated")
+    def visit_call(self, node):
+        """ Check instantiating abstract class with
+        abc.ABCMeta as metaclass.
+        """
+        try:
+            for inferred in node.func.infer():
+                self._check_inferred_class_is_abstract(inferred, node)
+        except astroid.InferenceError:
+            return
+
+    def _check_inferred_class_is_abstract(self, inferred, node):
+        if not isinstance(inferred, astroid.ClassDef):
+            return
+
+        klass = utils.node_frame_class(node)
+        if klass is inferred:
+            # Don't emit the warning if the class is instantiated
+            # in its own body or if the call is not an instance
+            # creation. If the class is instantiated into its own
+            # body, we're expecting that it knows what it is doing.
+            return
+
+        # __init__ was called
+        abstract_methods = _has_abstract_methods(inferred)
+
+        if not abstract_methods:
+            return
+
+        metaclass = inferred.metaclass()
+
+        if metaclass is None:
+            # Python 3.4 has `abc.ABC`, which won't be detected
+            # by ClassNode.metaclass()
+            for ancestor in inferred.ancestors():
+                if ancestor.qname() == "abc.ABC":
+                    self.add_message(
+                        "abstract-class-instantiated", args=(inferred.name,), node=node
+                    )
+                    break
+
+            return
+
+        if metaclass.qname() in ABC_METACLASSES:
+            self.add_message(
+                "abstract-class-instantiated", args=(inferred.name,), node=node
+            )
+
+    def _check_yield_outside_func(self, node):
+        if not isinstance(node.frame(), (astroid.FunctionDef, astroid.Lambda)):
+            self.add_message("yield-outside-function", node=node)
+
+    def _check_else_on_loop(self, node):
+        """Check that any loop with an else clause has a break statement."""
+        if node.orelse and not _loop_exits_early(node):
+            self.add_message(
+                "useless-else-on-loop",
+                node=node,
+                # This is not optimal, but the line previous
+                # to the first statement in the else clause
+                # will usually be the one that contains the else:.
+                line=node.orelse[0].lineno - 1,
+            )
+
+    def _check_in_loop(self, node, node_name):
+        """check that a node is inside a for or while loop"""
+        _node = node.parent
+        while _node:
+            if isinstance(_node, (astroid.For, astroid.While)):
+                if node not in _node.orelse:
+                    return
+
+            if isinstance(_node, (astroid.ClassDef, astroid.FunctionDef)):
+                break
+            if (
+                isinstance(_node, astroid.TryFinally)
+                and node in _node.finalbody
+                and isinstance(node, astroid.Continue)
+            ):
+                self.add_message("continue-in-finally", node=node)
+
+            _node = _node.parent
+
+        self.add_message("not-in-loop", node=node, args=node_name)
+
+    def _check_redefinition(self, redeftype, node):
+        """check for redefinition of a function / method / class name"""
+        parent_frame = node.parent.frame()
+
+        # Ignore function stubs created for type information
+        redefinitions = parent_frame.locals[node.name]
+        defined_self = next(
+            (local for local in redefinitions if not utils.is_overload_stub(local)),
+            node,
+        )
+        if defined_self is not node and not astroid.are_exclusive(node, defined_self):
+            # Additional checks for methods which are not considered
+            # redefined, since they are already part of the base API.
+            if (
+                isinstance(parent_frame, astroid.ClassDef)
+                and node.name in REDEFINABLE_METHODS
+            ):
+                return
+
+            # Skip typing.overload() functions.
+            if utils.is_overload_stub(node):
+                return
+
+            # Exempt functions redefined on a condition.
+            if isinstance(node.parent, astroid.If):
+                # Exempt "if not " cases
+                if (
+                    isinstance(node.parent.test, astroid.UnaryOp)
+                    and node.parent.test.op == "not"
+                    and isinstance(node.parent.test.operand, astroid.Name)
+                    and node.parent.test.operand.name == node.name
+                ):
+                    return
+
+                # Exempt "if  is not None" cases
+                # pylint: disable=too-many-boolean-expressions
+                if (
+                    isinstance(node.parent.test, astroid.Compare)
+                    and isinstance(node.parent.test.left, astroid.Name)
+                    and node.parent.test.left.name == node.name
+                    and node.parent.test.ops[0][0] == "is"
+                    and isinstance(node.parent.test.ops[0][1], astroid.Const)
+                    and node.parent.test.ops[0][1].value is None
+                ):
+                    return
+
+            # Check if we have forward references for this node.
+            try:
+                redefinition_index = redefinitions.index(node)
+            except ValueError:
+                pass
+            else:
+                for redefinition in redefinitions[:redefinition_index]:
+                    inferred = utils.safe_infer(redefinition)
+                    if (
+                        inferred
+                        and isinstance(inferred, astroid.Instance)
+                        and inferred.qname() == TYPING_FORWARD_REF_QNAME
+                    ):
+                        return
+
+            dummy_variables_rgx = lint_utils.get_global_option(
+                self, "dummy-variables-rgx", default=None
+            )
+            if dummy_variables_rgx and dummy_variables_rgx.match(node.name):
+                return
+            self.add_message(
+                "function-redefined",
+                node=node,
+                args=(redeftype, defined_self.fromlineno),
+            )
+
+
+class BasicChecker(_BasicChecker):
+    """checks for :
+    * doc strings
+    * number of arguments, local variables, branches, returns and statements in
+    functions, methods
+    * required module attributes
+    * dangerous default values as arguments
+    * redefinition of function / method / class
+    * uses of the global statement
+    """
+
+    __implements__ = interfaces.IAstroidChecker
+
+    name = "basic"
+    msgs = {
+        "W0101": (
+            "Unreachable code",
+            "unreachable",
+            'Used when there is some code behind a "return" or "raise" '
+            "statement, which will never be accessed.",
+        ),
+        "W0102": (
+            "Dangerous default value %s as argument",
+            "dangerous-default-value",
+            "Used when a mutable value as list or dictionary is detected in "
+            "a default value for an argument.",
+        ),
+        "W0104": (
+            "Statement seems to have no effect",
+            "pointless-statement",
+            "Used when a statement doesn't have (or at least seems to) any effect.",
+        ),
+        "W0105": (
+            "String statement has no effect",
+            "pointless-string-statement",
+            "Used when a string is used as a statement (which of course "
+            "has no effect). This is a particular case of W0104 with its "
+            "own message so you can easily disable it if you're using "
+            "those strings as documentation, instead of comments.",
+        ),
+        "W0106": (
+            'Expression "%s" is assigned to nothing',
+            "expression-not-assigned",
+            "Used when an expression that is not a function call is assigned "
+            "to nothing. Probably something else was intended.",
+        ),
+        "W0108": (
+            "Lambda may not be necessary",
+            "unnecessary-lambda",
+            "Used when the body of a lambda expression is a function call "
+            "on the same argument list as the lambda itself; such lambda "
+            "expressions are in all but a few cases replaceable with the "
+            "function being called in the body of the lambda.",
+        ),
+        "W0109": (
+            "Duplicate key %r in dictionary",
+            "duplicate-key",
+            "Used when a dictionary expression binds the same key multiple times.",
+        ),
+        "W0122": (
+            "Use of exec",
+            "exec-used",
+            'Used when you use the "exec" statement (function for Python '
+            "3), to discourage its usage. That doesn't "
+            "mean you cannot use it !",
+        ),
+        "W0123": (
+            "Use of eval",
+            "eval-used",
+            'Used when you use the "eval" function, to discourage its '
+            "usage. Consider using `ast.literal_eval` for safely evaluating "
+            "strings containing Python expressions "
+            "from untrusted sources. ",
+        ),
+        "W0150": (
+            "%s statement in finally block may swallow exception",
+            "lost-exception",
+            "Used when a break or a return statement is found inside the "
+            "finally clause of a try...finally block: the exceptions raised "
+            "in the try clause will be silently swallowed instead of being "
+            "re-raised.",
+        ),
+        "W0199": (
+            "Assert called on a 2-item-tuple. Did you mean 'assert x,y'?",
+            "assert-on-tuple",
+            "A call of assert on a tuple will always evaluate to true if "
+            "the tuple is not empty, and will always evaluate to false if "
+            "it is.",
+        ),
+        "W0124": (
+            'Following "as" with another context manager looks like a tuple.',
+            "confusing-with-statement",
+            "Emitted when a `with` statement component returns multiple values "
+            "and uses name binding with `as` only for a part of those values, "
+            "as in with ctx() as a, b. This can be misleading, since it's not "
+            "clear if the context manager returns a tuple or if the node without "
+            "a name binding is another context manager.",
+        ),
+        "W0125": (
+            "Using a conditional statement with a constant value",
+            "using-constant-test",
+            "Emitted when a conditional statement (If or ternary if) "
+            "uses a constant value for its test. This might not be what "
+            "the user intended to do.",
+        ),
+        "W0126": (
+            "Using a conditional statement with potentially wrong function or method call due to missing parentheses",
+            "missing-parentheses-for-call-in-test",
+            "Emitted when a conditional statement (If or ternary if) "
+            "seems to wrongly call a function due to missing parentheses",
+        ),
+        "W0127": (
+            "Assigning the same variable %r to itself",
+            "self-assigning-variable",
+            "Emitted when we detect that a variable is assigned to itself",
+        ),
+        "W0128": (
+            "Redeclared variable %r in assignment",
+            "redeclared-assigned-name",
+            "Emitted when we detect that a variable was redeclared in the same assignment.",
+        ),
+        "E0111": (
+            "The first reversed() argument is not a sequence",
+            "bad-reversed-sequence",
+            "Used when the first argument to reversed() builtin "
+            "isn't a sequence (does not implement __reversed__, "
+            "nor __getitem__ and __len__",
+        ),
+        "E0119": (
+            "format function is not called on str",
+            "misplaced-format-function",
+            "Emitted when format function is not called on str object. "
+            'e.g doing print("value: {}").format(123) instead of '
+            'print("value: {}".format(123)). This might not be what the user '
+            "intended to do.",
+        ),
+        "W0129": (
+            "Assert statement has a string literal as its first argument. The assert will %s fail.",
+            "assert-on-string-literal",
+            "Used when an assert statement has a string literal as its first argument, which will "
+            "cause the assert to always pass.",
+        ),
+    }
+
+    reports = (("RP0101", "Statistics by type", report_by_type_stats),)
+
+    def __init__(self, linter):
+        _BasicChecker.__init__(self, linter)
+        self.stats = None
+        self._tryfinallys = None
+
+    def open(self):
+        """initialize visit variables and statistics
+        """
+        self._tryfinallys = []
+        self.stats = self.linter.add_stats(module=0, function=0, method=0, class_=0)
+
+    @utils.check_messages("using-constant-test", "missing-parentheses-for-call-in-test")
+    def visit_if(self, node):
+        self._check_using_constant_test(node, node.test)
+
+    @utils.check_messages("using-constant-test", "missing-parentheses-for-call-in-test")
+    def visit_ifexp(self, node):
+        self._check_using_constant_test(node, node.test)
+
+    @utils.check_messages("using-constant-test", "missing-parentheses-for-call-in-test")
+    def visit_comprehension(self, node):
+        if node.ifs:
+            for if_test in node.ifs:
+                self._check_using_constant_test(node, if_test)
+
+    def _check_using_constant_test(self, node, test):
+        const_nodes = (
+            astroid.Module,
+            astroid.scoped_nodes.GeneratorExp,
+            astroid.Lambda,
+            astroid.FunctionDef,
+            astroid.ClassDef,
+            astroid.bases.Generator,
+            astroid.UnboundMethod,
+            astroid.BoundMethod,
+            astroid.Module,
+        )
+        structs = (astroid.Dict, astroid.Tuple, astroid.Set)
+
+        # These nodes are excepted, since they are not constant
+        # values, requiring a computation to happen.
+        except_nodes = (
+            astroid.Call,
+            astroid.BinOp,
+            astroid.BoolOp,
+            astroid.UnaryOp,
+            astroid.Subscript,
+        )
+        inferred = None
+        emit = isinstance(test, (astroid.Const,) + structs + const_nodes)
+        if not isinstance(test, except_nodes):
+            inferred = utils.safe_infer(test)
+
+        if emit:
+            self.add_message("using-constant-test", node=node)
+        elif isinstance(inferred, const_nodes):
+            # If the constant node is a FunctionDef or Lambda then
+            #  it may be a illicit function call due to missing parentheses
+            call_inferred = None
+            try:
+                if isinstance(inferred, astroid.FunctionDef):
+                    call_inferred = inferred.infer_call_result()
+                elif isinstance(inferred, astroid.Lambda):
+                    call_inferred = inferred.infer_call_result(node)
+            except astroid.InferenceError:
+                call_inferred = None
+            if call_inferred:
+                try:
+                    for inf_call in call_inferred:
+                        if inf_call != astroid.Uninferable:
+                            self.add_message(
+                                "missing-parentheses-for-call-in-test", node=node
+                            )
+                            break
+                except astroid.InferenceError:
+                    pass
+            self.add_message("using-constant-test", node=node)
+
+    def visit_module(self, _):
+        """check module name, docstring and required arguments
+        """
+        self.stats["module"] += 1
+
+    def visit_classdef(self, node):  # pylint: disable=unused-argument
+        """check module name, docstring and redefinition
+        increment branch counter
+        """
+        self.stats["class"] += 1
+
+    @utils.check_messages(
+        "pointless-statement", "pointless-string-statement", "expression-not-assigned"
+    )
+    def visit_expr(self, node):
+        """Check for various kind of statements without effect"""
+        expr = node.value
+        if isinstance(expr, astroid.Const) and isinstance(expr.value, str):
+            # treat string statement in a separated message
+            # Handle PEP-257 attribute docstrings.
+            # An attribute docstring is defined as being a string right after
+            # an assignment at the module level, class level or __init__ level.
+            scope = expr.scope()
+            if isinstance(
+                scope, (astroid.ClassDef, astroid.Module, astroid.FunctionDef)
+            ):
+                if isinstance(scope, astroid.FunctionDef) and scope.name != "__init__":
+                    pass
+                else:
+                    sibling = expr.previous_sibling()
+                    if (
+                        sibling is not None
+                        and sibling.scope() is scope
+                        and isinstance(sibling, (astroid.Assign, astroid.AnnAssign))
+                    ):
+                        return
+            self.add_message("pointless-string-statement", node=node)
+            return
+
+        # Ignore if this is :
+        # * a direct function call
+        # * the unique child of a try/except body
+        # * a yield statement
+        # * an ellipsis (which can be used on Python 3 instead of pass)
+        # warn W0106 if we have any underlying function call (we can't predict
+        # side effects), else pointless-statement
+        if (
+            isinstance(
+                expr, (astroid.Yield, astroid.Await, astroid.Ellipsis, astroid.Call)
+            )
+            or (
+                isinstance(node.parent, astroid.TryExcept)
+                and node.parent.body == [node]
+            )
+            or (isinstance(expr, astroid.Const) and expr.value is Ellipsis)
+        ):
+            return
+        if any(expr.nodes_of_class(astroid.Call)):
+            self.add_message(
+                "expression-not-assigned", node=node, args=expr.as_string()
+            )
+        else:
+            self.add_message("pointless-statement", node=node)
+
+    @staticmethod
+    def _filter_vararg(node, call_args):
+        # Return the arguments for the given call which are
+        # not passed as vararg.
+        for arg in call_args:
+            if isinstance(arg, astroid.Starred):
+                if (
+                    isinstance(arg.value, astroid.Name)
+                    and arg.value.name != node.args.vararg
+                ):
+                    yield arg
+            else:
+                yield arg
+
+    @staticmethod
+    def _has_variadic_argument(args, variadic_name):
+        if not args:
+            return True
+        for arg in args:
+            if isinstance(arg.value, astroid.Name):
+                if arg.value.name != variadic_name:
+                    return True
+            else:
+                return True
+        return False
+
+    @utils.check_messages("unnecessary-lambda")
+    def visit_lambda(self, node):
+        """check whether or not the lambda is suspicious
+        """
+        # if the body of the lambda is a call expression with the same
+        # argument list as the lambda itself, then the lambda is
+        # possibly unnecessary and at least suspicious.
+        if node.args.defaults:
+            # If the arguments of the lambda include defaults, then a
+            # judgment cannot be made because there is no way to check
+            # that the defaults defined by the lambda are the same as
+            # the defaults defined by the function called in the body
+            # of the lambda.
+            return
+        call = node.body
+        if not isinstance(call, astroid.Call):
+            # The body of the lambda must be a function call expression
+            # for the lambda to be unnecessary.
+            return
+        if isinstance(node.body.func, astroid.Attribute) and isinstance(
+            node.body.func.expr, astroid.Call
+        ):
+            # Chained call, the intermediate call might
+            # return something else (but we don't check that, yet).
+            return
+
+        call_site = CallSite.from_call(call)
+        ordinary_args = list(node.args.args)
+        new_call_args = list(self._filter_vararg(node, call.args))
+        if node.args.kwarg:
+            if self._has_variadic_argument(call.kwargs, node.args.kwarg):
+                return
+
+        if node.args.vararg:
+            if self._has_variadic_argument(call.starargs, node.args.vararg):
+                return
+        elif call.starargs:
+            return
+
+        if call.keywords:
+            # Look for additional keyword arguments that are not part
+            # of the lambda's signature
+            lambda_kwargs = {keyword.name for keyword in node.args.defaults}
+            if len(lambda_kwargs) != len(call_site.keyword_arguments):
+                # Different lengths, so probably not identical
+                return
+            if set(call_site.keyword_arguments).difference(lambda_kwargs):
+                return
+
+        # The "ordinary" arguments must be in a correspondence such that:
+        # ordinary_args[i].name == call.args[i].name.
+        if len(ordinary_args) != len(new_call_args):
+            return
+        for arg, passed_arg in zip(ordinary_args, new_call_args):
+            if not isinstance(passed_arg, astroid.Name):
+                return
+            if arg.name != passed_arg.name:
+                return
+
+        self.add_message("unnecessary-lambda", line=node.fromlineno, node=node)
+
+    @utils.check_messages("dangerous-default-value")
+    def visit_functiondef(self, node):
+        """check function name, docstring, arguments, redefinition,
+        variable names, max locals
+        """
+        self.stats["method" if node.is_method() else "function"] += 1
+        self._check_dangerous_default(node)
+
+    visit_asyncfunctiondef = visit_functiondef
+
+    def _check_dangerous_default(self, node):
+        # check for dangerous default values as arguments
+        is_iterable = lambda n: isinstance(n, (astroid.List, astroid.Set, astroid.Dict))
+        defaults = node.args.defaults or [] + node.args.kw_defaults or []
+        for default in defaults:
+            if not default:
+                continue
+            try:
+                value = next(default.infer())
+            except astroid.InferenceError:
+                continue
+
+            if (
+                isinstance(value, astroid.Instance)
+                and value.qname() in DEFAULT_ARGUMENT_SYMBOLS
+            ):
+
+                if value is default:
+                    msg = DEFAULT_ARGUMENT_SYMBOLS[value.qname()]
+                elif isinstance(value, astroid.Instance) or is_iterable(value):
+                    # We are here in the following situation(s):
+                    #   * a dict/set/list/tuple call which wasn't inferred
+                    #     to a syntax node ({}, () etc.). This can happen
+                    #     when the arguments are invalid or unknown to
+                    #     the inference.
+                    #   * a variable from somewhere else, which turns out to be a list
+                    #     or a dict.
+                    if is_iterable(default):
+                        msg = value.pytype()
+                    elif isinstance(default, astroid.Call):
+                        msg = "%s() (%s)" % (value.name, value.qname())
+                    else:
+                        msg = "%s (%s)" % (default.as_string(), value.qname())
+                else:
+                    # this argument is a name
+                    msg = "%s (%s)" % (
+                        default.as_string(),
+                        DEFAULT_ARGUMENT_SYMBOLS[value.qname()],
+                    )
+                self.add_message("dangerous-default-value", node=node, args=(msg,))
+
+    @utils.check_messages("unreachable", "lost-exception")
+    def visit_return(self, node):
+        """1 - check is the node has a right sibling (if so, that's some
+        unreachable code)
+        2 - check is the node is inside the finally clause of a try...finally
+        block
+        """
+        self._check_unreachable(node)
+        # Is it inside final body of a try...finally bloc ?
+        self._check_not_in_finally(node, "return", (astroid.FunctionDef,))
+
+    @utils.check_messages("unreachable")
+    def visit_continue(self, node):
+        """check is the node has a right sibling (if so, that's some unreachable
+        code)
+        """
+        self._check_unreachable(node)
+
+    @utils.check_messages("unreachable", "lost-exception")
+    def visit_break(self, node):
+        """1 - check is the node has a right sibling (if so, that's some
+        unreachable code)
+        2 - check is the node is inside the finally clause of a try...finally
+        block
+        """
+        # 1 - Is it right sibling ?
+        self._check_unreachable(node)
+        # 2 - Is it inside final body of a try...finally bloc ?
+        self._check_not_in_finally(node, "break", (astroid.For, astroid.While))
+
+    @utils.check_messages("unreachable")
+    def visit_raise(self, node):
+        """check if the node has a right sibling (if so, that's some unreachable
+        code)
+        """
+        self._check_unreachable(node)
+
+    @utils.check_messages("exec-used")
+    def visit_exec(self, node):
+        """just print a warning on exec statements"""
+        self.add_message("exec-used", node=node)
+
+    def _check_misplaced_format_function(self, call_node):
+        if not isinstance(call_node.func, astroid.Attribute):
+            return
+        if call_node.func.attrname != "format":
+            return
+
+        expr = utils.safe_infer(call_node.func.expr)
+        if expr is astroid.Uninferable:
+            return
+        if not expr:
+            # we are doubtful on inferred type of node, so here just check if format
+            # was called on print()
+            call_expr = call_node.func.expr
+            if not isinstance(call_expr, astroid.Call):
+                return
+            if (
+                isinstance(call_expr.func, astroid.Name)
+                and call_expr.func.name == "print"
+            ):
+                self.add_message("misplaced-format-function", node=call_node)
+
+    @utils.check_messages(
+        "eval-used", "exec-used", "bad-reversed-sequence", "misplaced-format-function"
+    )
+    def visit_call(self, node):
+        """visit a Call node -> check if this is not a blacklisted builtin
+        call and check for * or ** use
+        """
+        self._check_misplaced_format_function(node)
+        if isinstance(node.func, astroid.Name):
+            name = node.func.name
+            # ignore the name if it's not a builtin (i.e. not defined in the
+            # locals nor globals scope)
+            if not (name in node.frame() or name in node.root()):
+                if name == "exec":
+                    self.add_message("exec-used", node=node)
+                elif name == "reversed":
+                    self._check_reversed(node)
+                elif name == "eval":
+                    self.add_message("eval-used", node=node)
+
+    @utils.check_messages("assert-on-tuple", "assert-on-string-literal")
+    def visit_assert(self, node):
+        """check whether assert is used on a tuple or string literal."""
+        if (
+            node.fail is None
+            and isinstance(node.test, astroid.Tuple)
+            and len(node.test.elts) == 2
+        ):
+            self.add_message("assert-on-tuple", node=node)
+
+        if isinstance(node.test, astroid.Const) and isinstance(node.test.value, str):
+            if node.test.value:
+                when = "never"
+            else:
+                when = "always"
+            self.add_message("assert-on-string-literal", node=node, args=(when,))
+
+    @utils.check_messages("duplicate-key")
+    def visit_dict(self, node):
+        """check duplicate key in dictionary"""
+        keys = set()
+        for k, _ in node.items:
+            if isinstance(k, astroid.Const):
+                key = k.value
+                if key in keys:
+                    self.add_message("duplicate-key", node=node, args=key)
+                keys.add(key)
+
+    def visit_tryfinally(self, node):
+        """update try...finally flag"""
+        self._tryfinallys.append(node)
+
+    def leave_tryfinally(self, node):  # pylint: disable=unused-argument
+        """update try...finally flag"""
+        self._tryfinallys.pop()
+
+    def _check_unreachable(self, node):
+        """check unreachable code"""
+        unreach_stmt = node.next_sibling()
+        if unreach_stmt is not None:
+            self.add_message("unreachable", node=unreach_stmt)
+
+    def _check_not_in_finally(self, node, node_name, breaker_classes=()):
+        """check that a node is not inside a finally clause of a
+        try...finally statement.
+        If we found before a try...finally bloc a parent which its type is
+        in breaker_classes, we skip the whole check."""
+        # if self._tryfinallys is empty, we're not an in try...finally block
+        if not self._tryfinallys:
+            return
+        # the node could be a grand-grand...-children of the try...finally
+        _parent = node.parent
+        _node = node
+        while _parent and not isinstance(_parent, breaker_classes):
+            if hasattr(_parent, "finalbody") and _node in _parent.finalbody:
+                self.add_message("lost-exception", node=node, args=node_name)
+                return
+            _node = _parent
+            _parent = _node.parent
+
+    def _check_reversed(self, node):
+        """ check that the argument to `reversed` is a sequence """
+        try:
+            argument = utils.safe_infer(utils.get_argument_from_call(node, position=0))
+        except utils.NoSuchArgumentError:
+            pass
+        else:
+            if argument is astroid.Uninferable:
+                return
+            if argument is None:
+                # Nothing was inferred.
+                # Try to see if we have iter().
+                if isinstance(node.args[0], astroid.Call):
+                    try:
+                        func = next(node.args[0].func.infer())
+                    except astroid.InferenceError:
+                        return
+                    if getattr(
+                        func, "name", None
+                    ) == "iter" and utils.is_builtin_object(func):
+                        self.add_message("bad-reversed-sequence", node=node)
+                return
+
+            if isinstance(argument, (astroid.List, astroid.Tuple)):
+                return
+
+            if isinstance(argument, astroid.Instance):
+                if argument._proxied.name == "dict" and utils.is_builtin_object(
+                    argument._proxied
+                ):
+                    self.add_message("bad-reversed-sequence", node=node)
+                    return
+                if any(
+                    ancestor.name == "dict" and utils.is_builtin_object(ancestor)
+                    for ancestor in argument._proxied.ancestors()
+                ):
+                    # Mappings aren't accepted by reversed(), unless
+                    # they provide explicitly a __reversed__ method.
+                    try:
+                        argument.locals[REVERSED_PROTOCOL_METHOD]
+                    except KeyError:
+                        self.add_message("bad-reversed-sequence", node=node)
+                    return
+
+            if hasattr(argument, "getattr"):
+                # everything else is not a proper sequence for reversed()
+                for methods in REVERSED_METHODS:
+                    for meth in methods:
+                        try:
+                            argument.getattr(meth)
+                        except astroid.NotFoundError:
+                            break
+                    else:
+                        break
+                else:
+                    self.add_message("bad-reversed-sequence", node=node)
+            else:
+                self.add_message("bad-reversed-sequence", node=node)
+
+    @utils.check_messages("confusing-with-statement")
+    def visit_with(self, node):
+        # a "with" statement with multiple managers corresponds
+        # to one AST "With" node with multiple items
+        pairs = node.items
+        if pairs:
+            for prev_pair, pair in zip(pairs, pairs[1:]):
+                if isinstance(prev_pair[1], astroid.AssignName) and (
+                    pair[1] is None and not isinstance(pair[0], astroid.Call)
+                ):
+                    # Don't emit a message if the second is a function call
+                    # there's no way that can be mistaken for a name assignment.
+                    # If the line number doesn't match
+                    # we assume it's a nested "with".
+                    self.add_message("confusing-with-statement", node=node)
+
+    def _check_self_assigning_variable(self, node):
+        # Detect assigning to the same variable.
+
+        scope = node.scope()
+        scope_locals = scope.locals
+
+        rhs_names = []
+        targets = node.targets
+        if isinstance(targets[0], astroid.Tuple):
+            if len(targets) != 1:
+                # A complex assignment, so bail out early.
+                return
+            targets = targets[0].elts
+            if len(targets) == 1:
+                # Unpacking a variable into the same name.
+                return
+
+        if isinstance(node.value, astroid.Name):
+            if len(targets) != 1:
+                return
+            rhs_names = [node.value]
+        elif isinstance(node.value, astroid.Tuple):
+            rhs_count = len(node.value.elts)
+            if len(targets) != rhs_count or rhs_count == 1:
+                return
+            rhs_names = node.value.elts
+
+        for target, lhs_name in zip(targets, rhs_names):
+            if not isinstance(lhs_name, astroid.Name):
+                continue
+            if not isinstance(target, astroid.AssignName):
+                continue
+            if isinstance(scope, astroid.ClassDef) and target.name in scope_locals:
+                # Check that the scope is different than a class level, which is usually
+                # a pattern to expose module level attributes as class level ones.
+                continue
+            if target.name == lhs_name.name:
+                self.add_message(
+                    "self-assigning-variable", args=(target.name,), node=target
+                )
+
+    def _check_redeclared_assign_name(self, targets):
+        dummy_variables_rgx = lint_utils.get_global_option(
+            self, "dummy-variables-rgx", default=None
+        )
+
+        for target in targets:
+            if not isinstance(target, astroid.Tuple):
+                continue
+
+            found_names = []
+            for element in target.elts:
+                if isinstance(element, astroid.Tuple):
+                    self._check_redeclared_assign_name([element])
+                elif isinstance(element, astroid.AssignName) and element.name != "_":
+                    if dummy_variables_rgx and dummy_variables_rgx.match(element.name):
+                        return
+                    found_names.append(element.name)
+
+            names = collections.Counter(found_names)
+            for name, count in names.most_common():
+                if count > 1:
+                    self.add_message(
+                        "redeclared-assigned-name", args=(name,), node=target
+                    )
+
+    @utils.check_messages("self-assigning-variable", "redeclared-assigned-name")
+    def visit_assign(self, node):
+        self._check_self_assigning_variable(node)
+        self._check_redeclared_assign_name(node.targets)
+
+    @utils.check_messages("redeclared-assigned-name")
+    def visit_for(self, node):
+        self._check_redeclared_assign_name([node.target])
+
+
+KNOWN_NAME_TYPES = {
+    "module",
+    "const",
+    "class",
+    "function",
+    "method",
+    "attr",
+    "argument",
+    "variable",
+    "class_attribute",
+    "inlinevar",
+}
+
+
+HUMAN_READABLE_TYPES = {
+    "module": "module",
+    "const": "constant",
+    "class": "class",
+    "function": "function",
+    "method": "method",
+    "attr": "attribute",
+    "argument": "argument",
+    "variable": "variable",
+    "class_attribute": "class attribute",
+    "inlinevar": "inline iteration",
+}
+
+DEFAULT_NAMING_STYLES = {
+    "module": "snake_case",
+    "const": "UPPER_CASE",
+    "class": "PascalCase",
+    "function": "snake_case",
+    "method": "snake_case",
+    "attr": "snake_case",
+    "argument": "snake_case",
+    "variable": "snake_case",
+    "class_attribute": "any",
+    "inlinevar": "any",
+}
+
+
+def _create_naming_options():
+    name_options = []
+    for name_type in sorted(KNOWN_NAME_TYPES):
+        human_readable_name = HUMAN_READABLE_TYPES[name_type]
+        default_style = DEFAULT_NAMING_STYLES[name_type]
+        name_type = name_type.replace("_", "-")
+        name_options.append(
+            (
+                "%s-naming-style" % (name_type,),
+                {
+                    "default": default_style,
+                    "type": "choice",
+                    "choices": list(NAMING_STYLES.keys()),
+                    "metavar": "