Allow to exclude tracked files from the package release archive

[olexandr-konovalov]

The TransGrp package (https://github.com/hulpke/transgrp) has data for groups of degree 32 that need to be downloaded separately (~300MB from http://www.math.colostate.edu/~hulpke/transgrp/trans32.tgz). This archive is not updated for every TransGrp release. If TransGrp would be using ReleaseTools, it would be useful to have some kind of an .ignore file to be able to specify which files should not be included into the package distribution.

See also #48 which suggests functionality for adding non-tracked files to the package archive.

[hulpke]

For what its worth, transgrp has a shell scrip for packing that works perfectly fine and at first glance I fear that release tools would make the packing process longer and introduce other break points. Thus please don't take transgrp as an indicator of urgency.

Otherwise, of course this might make perfect sense for private package addition.

[fingolfin]

So, I am really completely neutral as to whether people use this tool or not -- if it is helpful to somebody, great; if you have your own working tool, also fine by me!

My experience simply was that everybody has their own cobbled together script to do this job, and most (well, all) I have seen so far are very limited, with little to no error checking, no validation, etc. Which makes sense, as they are written as throw-away tools, to be used by a single person, so it's not necessary to have them clean and nice, because their author knows what they are doing.

... except that when you don't make a release for half a year, that "you" from back then is a stranger, and you run into problems with the release, and it gets painful, and you have to try again, and then it works, but a day later you are told your package does not validate, and you thought you had it behind, but now you need to look at it again, and BOY, making releases sucks!. My experience based on discussion with maybe half a dozen package authors is that this then leads to them delaying releases, because doing them is so frustrating (which of course makes the problem worse, because the longer you put it off, the more details about it you forget).

Well, at least that was my personal situation. So I sat down to write a tool with documentation, and proper error handling, and I looked at all the steps I have to take during a release, and I made notes about the steps I tended to forget (and also asked other people for what happened for them), and then I tried to automate it as much as possible. And at least for my purposes, I succeeded. But of course there is a limit of time I can spend on this, and so there are certainly use cases which ReleaseTools does not (yet?) supported adequately, such as the one described by @alex-konovalov above.

So, @hulpke, if your scripts work fine for you: great! Although they clearly don't run ValidatePackageInfo ... :-).

Still, I'd love to here what makes you say that ReleaseTools "would make the packing process longer and introduce other break points" -- other than the fact that based on what I read in this issue,, ReleaseTools is missing at last on feature you'd need (though I think it should be pretty simple to add).

Perhaps you could briefly elaborate?

I am wondering if you were perhaps mislead by the fact that in the README, I took pains to point out things people should not forget to do when they make a release, which makes the list indeed rather long? Well, I made that list based on experience that people do forget them all the time.

Yet in practice, when I release a package, the steps I take are these:

1. Update PackageInfo.g and whatever else is needed for a release and commit that.
2. Run the release script (without any options)

That's it. Of course there are often "break points", like the script pointing out to me that I did something wrong, e.g. forgot to adjust the version, put in a malformed date string into my PackageInfo.g, that I forgot to commit some files, forgot to update the VERSION file in my package (if any etc.) -- which saves me a lot of time, because I can then quickly fix those and run release again, as opposed to getting an email some time later from @alex-konovalov that I need to make another release to fix the problems.

Writing and polishing this script took quite some effort, but it paid of for myself already, and I know several other people use it quite effectively, and that makes me happy :-). That said, there are certainly still tons of things it could do better, and I am very interested in any suggestions for that (and of course also in pull requests).

[olexandr-konovalov]

I had own release scripts in the past, and I ditched all of them for the purpose of using ReleaseTools (see e.g. gap-packages/scscp@689089f). That saved me a lot of time, and I agree with every @fingolfin's word above.

Also, while not necessary to use both, it works well with GitHubPagesForGAP. Since TransGrp is a so important package for GAP, it should have a proper webpage...

[fingolfin]

This is getting more important now, since lots of packages now have a scripts dir and .travis.yml, .codecov.yml files which shouldn't be in the tarball. I'll push this up on my agenda (likely by addressing it as discussed on PR #48)