Debloated distribution: base-modules-doc

[bbkr]

Hi

In company I'm working for we use Docker extensively.
One of the issues we're struggling with is the fact that Perl package became huge bloatware - full installation size exceeds 50MB and consists of:

• 2MB Perl interpreter itself
• 5MB core stuff (unicode definitions, common pragmas)
• 0.5MB of weird tools (zipdetails, perlivp, ...)
• 0.5MB of diagnostic tools (perlbug, podchecker, ...)
• 45MB of 350(!) CPAN modules (some of them like Memoize::NDBM_File are included without evaluation of nowadays usage stats, some made obsolete by better alternatives like Time::Piece vs DateTime, some like Tap::* or Pod::* namespaces are used only during development and make no sense on production machines).
• 7 MBs of PODs and manpages

This "distribution" approach made sense in 90's, when internet connection was not common / slow and it was reasonable to provide user as feature-rich ecosystem as possible for local installation. However in Docker service-oriented environments this bloats FS layer in every container that wants to use cperl by 50 MB, which causes slow production deploys and docker-compose pulls for developers to get updated environment. Not to mention it looks bizarre if 50MB Perl package is applied to 5MB Alpine container for example :)

My humble request is to add bare core installation option to Makefile: interpreter + unicode defs + pragmas + minimal set of modules required to get cpanm rolling. Something like Debian did with splitting Perl distribution into perl-base, perl-modules and perl-doc: https://packages.debian.org/search?keywords=perl-base . It's much more easier nowadays to compose slim and well-tuned environment directly from CPAN than to clog every service with hundreds of modules and scripts bundled with Perl that will never be used.

Thanks!

[rurban]

I appreciate your concerns. I packaged perl also by myself for cygwin for a decade, and had to download updates by myself. But it will not happen.

Why so much bloat?

• The biggest problem is that packagers often do not include cpan essential packages, so the core package had to include the cpan client to install the missing packages by himself. Only fedora and debian can afford a split. And they do already. But even they don't include the compilers. Which is a major problem for users. Nobody is using it.
• 90% of the bloat are cpan client dependencies.
• 2nd: some packages are essential, like serializers (JSON, YAML, Storable), compiler (B::C), debugger, profiler (Devel::NYTProf), and cannot be maintained nor packaged by others. In fact about half of the essential packages are not maintained at all, neither by cpan nor by p5p.
  Relying on external packages turned out to be fatal for frameworks and infrastructure. JSON::PP and friends are used over Cpanel::JSON::XS which is 10x faster and slimmer. YAML is used over YAML::Syck or YAML::XS, even if they are completely broken or over architectured. E.g. cpan defaults to YAML on distroprefs (~20s boot delay). cperl fixed and switched to YAML::XS to avoid this delay. YAML::Syck would also be fine but uses just 1.1, and this should not be in core.
  Cpan::Meta default to it's own YAML::Tiny, because they need more security, but cperl got rid of that and switched to the fixed YAML::XS also. But the whole toolchain and testing infra got over architectured lately.
  Then there are few packages included which are needed to catch core errors, which would otherwise not been found. Such as Internals::DumpArenas.
  Technically Math::BigInt::GMP needs to be in core also, because the pure perl variants are a joke, but I didn't get there yet.
  Everything which has a proper XS variant should be in core instead of the slow pure perl variant. Unicode should be compiled shared XS modules, not the slow and huge script nonsense. That's 2MB run-time bloatware, unshared amongst forks.

But in the long term who won?
Strawberry by providing everything, even the toolchain. redhat and debian got beaten for providing base. Many issues arise from that, people missing core modules.

But your request of course does make sense, to provide targets and build rules/scripts for splitted packages. Pull requests for a base/modules/doc split will be accepted. I"ll eventually provide them myself also.
It's a trivial packaging splitup, p5p doesn't provide anything there, only cperl in Porting.
My packages are currently 10 MB, without docs much smaller.

[bbkr]

Thank you for such fast and detailed response.

Yes, my point was to provide slim base installation as an option, not to dump whole bundled ecosystem and maintain interpreter alone.

I've tried to tinker with Makefile and installperl scripts but it's too tangled for my poor knowledge about Perl toolchain to do it right. However I'll be happy to help testing base installation option when it's added in cperl.

[rurban]

Technically it might be done via installperl --base/--no-base (docs already done seperately).
And a Makefile target for install-base, install-nobase.
But usually you would just strip them apart as every other packager does it: In the seperate packaging script, after make install DESTDIR=.
E.g. fedora/debian/cygwin split debug symbols into its own packages, and create seperate tars from the one destdir. In cperl such a script is in Porting/do-make-cperl-release, which also takes care of reproducible builds.
There can be added: Porting/do-make-cperl-base, Porting/do-make-cperl-modules, Porting/do-make-cperl-docs. Simple bash scripts