WIP: release v4.6.0

[jameslamb]

Release checklist:

Copied from #6538 with a few changes.

deadline

February 14, 2024 (ref: #6791)

before merge

• Update VERSION.txt number.
• Update version in Appveyor config file.
• Update version in configure file of R-package: /gha run r-configure.
• Change development.mode from unreleased to release in pkgdown config file.
• Update version in python-package/pyproject.toml
• Add release branch to RTD versions, trigger a new build, check docs
• All docs for new behavior have Sphinx versionadded:: annotations (docs on those)
• All new parameters in config.h have *New in version {version}* comments added
• resolve all issues and merge all PRs we want to get into the release:
  • [ci] conda-based CI jobs broken: "conda/activate.d/libarrow_activate.sh: line 25: CONDA_BUILD: unbound variable" #6799
  • [python-package] Separately check whether pyarrow and cffi are installed #6785
  • [python-package] support sub-classing scikit-learn estimators #6783
  • [python-package] do not copy column-major numpy arrays when creating Dataset from list of arrays #6773
  • WIP: [R-package] require lgb.Dataset, remove support for passing 'colnames' and 'categorical_feature' for lgb.train() and lgb.cv() #6714
  • [python-package] support Python 3.13 #6668
• Run the valgrind checks with /gha run r-valgrind (docs link)

after merge

• Push v* tag to trigger GitHubRelease action at Azure Pipelines.
• Update stable tag at GitHub.

code to update tags (click me)

NEW_VERSION="4.6.0"
git checkout master
git pull upstream master
git fetch upstream --tags
git tag -d stable
git push upstream :refs/tags/stable
git tag stable
git tag "v${NEW_VERSION}"
git push upstream stable "v${NEW_VERSION}"

• Add new tag to RTD versions and trigger a new build.
• Trigger a new build of the stable version docs on readthedocs (see [ci] Fix version matching between RTD pages and R-package pages #6673 (comment))
• Remove the release branch release/v{n}.{n}.{n} from RTD versions

after the release is created by Azure DevOps job

• Add release notes from release-drafter to the automatically-created GitHub release, convert it from Draft to published.
• Manually attach arm64 macOS wheel to the release
• Upload release to CRAN
• Upload release to test PyPI
• Upload release to PyPI.

code to publish to PyPI (summary)

mkdir /tmp/lgb-release
cd /tmp/lgb-release

NEW_VERSION="4.6.0"

# NOTE: requires a GitHub personal access token with "repo" scope
gh release download \
    --repo microsoft/LightGBM \
    --dir ./artifacts \
    --pattern 'lightgbm*-py3-*.whl' \
    --pattern "lightgbm-${NEW_VERSION}.tar.gz" \
    "v${NEW_VERSION}"

# NOTE: manually downloaded arm64 wheels from CI

# config docs: https://packaging.python.org/en/latest/specifications/pypirc/
twine upload \
    -r testpypi \
    ./artifacts/*

(gh is the GitHub CLI, see https://cli.github.com/manual/gh_release_download)

Then confirmed that installing the latest wheel works.

pip install -i https://test.pypi.org/simple/ "lightgbm==${NEW_VERSION}"
python ./examples/python-guide/logistic_regression.py

Then pushed them to real PyPI.

twine upload \
    ./artifacts/*

• update conda-forge recipe
  • usually a PR will automatically be opened a day or 2 after publishing an sdist to PyPI, e.g. lightgbm v4.5.0 conda-forge/lightgbm-feedstock#60
• Update cran-comments.md when new release is accepted on CRAN.
• Upload release to NuGet.
• Update version and commit hash in Homebrew formula (docs)
• put up another PR incrementing development version here in GitHub (example: bump development version to 4.4.0.99 #6485)

code to open that dev version PR (click me)

RELEASE_PR_NUMBER="6439"
RELEASE_VERSION=$(cat ./VERSION.txt)
DEV_VERSION="${RELEASE_VERSION}.99"

git checkout -b ci/dev-version

echo "${DEV_VERSION}" > ./VERSION.txt
sed \
    -i .bak \
    "s|${RELEASE_VERSION}|${DEV_VERSION}|g" \
    .appveyor.yml

sed \
    -i .bak \
    "s|version = \"${RELEASE_VERSION}\"|version = \"${DEV_VERSION}\"|g" \
    ./python-package/pyproject.toml

sed \
    -i .bak \
    's|mode\: release|mode\: unreleased|g' \
    ./R-package/pkgdown/_pkgdown.yml

docker run \
    --rm \
    -v $(pwd):/opt/LightGBM \
    -w /opt/LightGBM \
    ubuntu:22.04 \
    ./R-package/recreate-configure.sh

git add \
    ./.appveyor.yml \
    ./R-package/configure \
    ./R-package/pkgdown/_pkgdown.yml \
    ./VERSION.txt \
    ./python-package/pyproject.toml

commit_msg="bump development version to ${DEV_VERSION}"
git commit -m "${commit_msg}"
git push upstream ci/dev-version

gh pr create \
    --repo microsoft/LightGBM \
    --base 'master' \
    --label 'maintenance' \
    --title "${commit_msg}" \
    --body "now that v${RELEASE_VERSION} has been released: #${RELEASE_PR_NUMBER}"

Notes for Reviewers

I believe this should be v4.6.0 instead of v4.5.1 because of some small breaking changes:

• [python-package] remove support for passing 'feature_name' and 'categorical_feature' through train() and cv() #6706
• [python-package] require scikit-learn>=0.24.2, make scikit-learn estimators compatible with scikit-learn>=1.6.0dev #6651
• [ci] Require CMake 3.28 and replace FetchContent_Populate with FetchContent_MakeAvailable #6550

See https://github.com/microsoft/LightGBM/releases.

[jameslamb]

/gha run r-configure

[jameslamb]

I think it's time to start another release!

We need to at least release the R package by February 14th, to stay on CRAN (#6791). But even without that, it's been about 6 months since v4.5.0 and a lot of good changes have piled up.

I've listed the PRs / issues that I think we should try to get into the release. Please comment if there are others you want to try to get in.

@guolinke @shiyu1994 @StrikerRUS @jmoralez @borchero @btrotta

[jameslamb]

/gha run r-configure

This failed with the following:

{
  "message": "The 'Microsoft Open Source' enterprise forbids access via a personal access tokens (classic) if the token's lifetime is greater than 365 days. Please adjust your token's lifetime at the following URL: https://github.com/settings/tokens/503169623",
  "documentation_url": "https://docs.github.com/rest/repos/repos#create-a-repository-dispatch-event",
  "status": "403"
}

(build link)

@shiyu1994 can you please fix that in the repo settings? I can update R-package/configure manually and push it, but we'll need to get comment-triggered workflows working again to run the valgrind checks.

[jameslamb]

Add release branch to RTD versions,

Did this. Here's the first build: https://readthedocs.org/projects/lightgbm/builds/26974345/

[StrikerRUS]

I guess we need to replace old WORKFLOW token with a new one that has 365 days lifetime.
Refer to #3439 (comment) for required token permissions.