Misc\NEWS in the Windows installer

[AA-Turner]

As discovered in python/blurb#44 (python/blurb#44 (comment)), the Windows installer ships the merged NEWS file in Doc\html\, but has a 'broken' Doc\html\whatsnew\changelog.html file, reading "The NEWS file is not available".

1. Can we fix the changelog.html creation? My first guess would be that the NEWS file has not been created or moved to Doc\build\NEWS at the time that Sphinx is invoked.

2. Do we need to provide the NEWS file to users? It is a ~2MB ~50,000 line text file with fairly poor usability and little discoverability (it is in the html directory but not linked from anywhere).

A

[hugovk]

My first guess would be that the NEWS file has not been created or moved to Doc\build\NEWS at the time that Sphinx is invoked.

Seems that way (but I'm not too familiar with the Windows build process!). Looking at a recent-ish build: https://dev.azure.com/Python/cpython/_build/results?buildId=157746&view=results

Docs are built and uploaded as an artifact in the first "Build bianries" stage:

https://dev.azure.com/Python/cpython/_build/results?buildId=157746&view=logs&s=6884a131-87da-5381-61f3-d7acc3b91d76

"Merge NEWS file" happens later, in "Build MSI installer", after the readymade docs artifact has been downloaded and moved:

https://dev.azure.com/Python/cpython/_build/results?buildId=157746&view=logs&s=ee5e2201-3607-5d60-495a-8553549f51c0

[AA-Turner]

My first guess would be that the NEWS file has not been created or moved to Doc\build\NEWS at the time that Sphinx is invoked.

Seems that way (but I'm not too familiar with the Windows build process!).

...

Docs are built and uploaded as an artifact in the first "Build bianries" stage:

dev.azure.com/Python/cpython/_build/results?buildId=157746&view=logs&s=6884a131-87da-5381-61f3-d7acc3b91d76

Aha, Line 127 of "Build HTML docs" has "Merging Misc/NEWS with py -3.12 -m blurb", which is echoed by make.bat.

This copies the file to "%PY_MISC_NEWS_DIR%\NEWS". The build process (as far as I can tell) does not define PY_MISC_NEWS_DIR, so this line is executed:

if not defined PY_MISC_NEWS_DIR set PY_MISC_NEWS_DIR=%BUILDDIR%\%1

%1 is the output format, html, and builddir is presumably build, where the cwd is Doc\. This means that PY_MISC_NEWS_DIR expands to Doc\build\html, and the NEWS file is copied here.

I think therefore that this first point was accidentally solved in python/cpython#129577, as make.bat now copies to build\NEWS explicitly. I'm not sure how we'd test this though, is there a way to do dry-runs of the Windows release steps?

A

[AA-Turner]

To second point, the released tarballs include the merged NEWS file as Misc\NEWS, but I can't find such a file in my local Python installations, and it is unclear if/where in msi-steps.yml that Misc\News gets used. Steve is probably best placed to comment here.

[hugovk]

I'm not sure how we'd test this though, is there a way to do dry-runs of the Windows release steps?

A

Yes, @zooba and @ambv can trigger dry runs.

Although we'll get wet runs with the 3.12 and 3.13 releases tomorrow...

[zooba]

I think therefore that this first point was accidentally solved in python/cpython#129577,

Does this change need to be backported then?

I'll get a run going so we can see the result.

[AA-Turner]

Does this change need to be backported then?

It's been backported to 3.12 & 3.13 but not security branches.

[zooba]

The changelog.html from the build is about 6MB - does that seem right? You should be able to download the doc artifact from https://dev.azure.com/Python/cpython/_build/results?buildId=158873&view=artifacts&pathAsName=false&type=publishedArtifacts to check it yourself.

[hugovk]

Yep, that's about right -- it's a decade of blurbs since 3.5! (see python/docs-community#98)

I downloaded the artifact and changelog.html matches my local.

[zooba]

Guess there's nothing more to fix then!

[AA-Turner]

Do we need to keep blurb merge in msi-steps.yml?

release-tools/windows-release/msi-steps.yml

Lines 41 to 45 in 6922a3c

	- script: |
	%PYTHON% -m pip install blurb
	%PYTHON% -m blurb merge -f Misc\NEWS
	displayName: 'Merge NEWS file'

[zooba]

May as well. No change is easier than making a change, and I'm hoping to deprecate these steps soon (see PEP 773).

[AA-Turner]

Ok, it seems indeed there is nothing to do!