Upgrade pantsbuild from 2.18.3 to 2.20.4 (+regen lockfiles/st2.lock)

[cognifloyd]

The commits step through updating from 2.18 to 2.19 to 2.20.
This PR also regenerates lockfiles/st2.lock (and related files) which, among other updates, pulls in the st2-auth-ldap repo update so that CI stops complaining about that.

Interesting new features in 2.19

• https://www.pantsbuild.org/blog/2024/01/24/pants-2-19
• https://github.com/pantsbuild/pants/blob/2.19.x/src/python/pants/notes/2.19.x.md

Pants can auto retry tests to deal with flaky tests in CI. I enabled that in pants.ci.toml so it retries a test up to 3x.

Interesting new features in 2.20

• https://www.pantsbuild.org/blog/2024/03/27/pants-2-20
• https://github.com/pantsbuild/pants/blob/2.20.x/src/python/pants/notes/2.20.x.md

Pants can now experimentally use GHA as a remote cache.

Pants has a pex_binary target for generating targets that run a particular entry point or script. Pants 2.20 gained a pex_binary(executable=...) field which would allow us to wrap our st2-* python scripts in a pex at some point. This feature means that pex will embed the script contents instead of attempting to import from a file that is not a valid package name (The "-" character is not valid). For now, I just left a comment in a BUILD file about this.
Note: I asked for this feature some time ago. It was first added to pex 2.1.93, and then to pants. Now its available.

pants-plugins lockfile diffs

From 2.18 to 2.19

Lockfile diff: lockfiles/pants-plugins.lock [pants-plugins]

==                    Upgraded dependencies                     ==

  pantsbuild-pants               2.18.3       -->   2.19.3
  pantsbuild-pants-testutil      2.18.3       -->   2.19.3
  pex                            2.1.137      -->   2.2.1
  pluggy                         1.4.0        -->   1.5.0
  psutil                         5.9.0        -->   5.9.8
  ujson                          5.9.0        -->   5.10.0

From 2.19 to 2.20

Lockfile diff: lockfiles/pants-plugins.lock [pants-plugins]

==                    Upgraded dependencies                     ==

  attrs                          23.2.0       -->   24.2.0
  pantsbuild-pants               2.19.3       -->   2.20.4
  pantsbuild-pants-testutil      2.19.3       -->   2.20.4
  pyparsing                      3.1.2        -->   3.1.4
  pyyaml                         6.0.1        -->   6.0.2

st2 lockfile diff

Lockfile diff: lockfiles/st2.lock [st2]
                                                                  
==                    Upgraded dependencies                     ==

  argcomplete                    3.4.0        -->   3.5.0
  attrs                          23.2.0       -->   24.2.0
  certifi                        2024.7.4     -->   2024.8.30
  cffi                           1.16.0       -->   1.17.1
  cryptography                   43.0.0       -->   43.0.1
  filelock                       3.15.4       -->   3.16.0
  gunicorn                       22.0.0       -->   23.0.0
  idna                           3.7          -->   3.8
  importlib-metadata             8.2.0        -->   8.4.0
  kombu                          5.3.7        -->   5.4.0
  orjson                         3.10.6       -->   3.10.7
  oslo-config                    9.5.0        -->   9.6.0
  oslo-i18n                      6.3.0        -->   6.4.0
  oslo-serialization             5.4.0        -->   5.5.0
  oslo-utils                     7.2.0        -->   7.3.0
  paramiko                       3.4.0        -->   3.4.1
  pbr                            6.0.0        -->   6.1.0
  platformdirs                   4.2.2        -->   4.3.2
  prettytable                    3.10.2       -->   3.11.0
  pyparsing                      3.1.2        -->   3.1.4
  pyyaml                         6.0.1        -->   6.0.2
  redis                          5.0.7        -->   5.0.8
  setuptools                     72.1.0       -->   74.1.2
  simplejson                     3.19.2       -->   3.19.3
  soupsieve                      2.5          -->   2.6
  stevedore                      5.2.0        -->   5.3.0
  tooz                           6.2.0        -->   6.3.0
  virtualenv                     20.26.3      -->   20.26.4
  webob                          1.8.7        -->   1.8.8
  webtest                        3.0.0        -->   3.0.1
  wheel                          0.43.0       -->   0.44.0
  zipp                           3.19.2       -->   3.20.1

[nzlosh]

LGTM

[mamercad]

🚢