Allow build-type Configure to work with Components

[angerman]

When components were introduced, cabal had support for build-type: Configure, which allows to run a configure script prior to building the package. Upon the introduction of components, it became obvious that we can't just configure each and every component, as this easily runs into the situation where the configure script is run potentially concurrently for all components.

However, build-type is a global option for cabal files, and is usually used to produce a .buildinfo file. This file is then used to augment the Main Library and Executable components only.

This change now tracks whether or not we are building a component to the setup invocation, and if we do, ignores a build-type: Configure built-type from the package description. For end users who want to depend on
package configured values, they will need to set their sub libraries, ... and other components to depend on the main library, and import any configured values from there.

For lib:Cabal, which doesn't know about components when configure is invoked, we will continue to execute configure, even though cabal will never use that info.

Fixes #4548

---

• Patches conform to the coding conventions.
• Any changes that could be relevant to users have been recorded in the changelog.
  • Is the change significant? If so, remember to add significance: significant in the changelog file.
• The documentation has been updated, if necessary.
• Manual QA notes have been included.
• Tests have been added. (Ask for help if you don’t know how to write them! Ask for an exemption if tests are too complex for too little coverage!)

[angerman]

Upon closer reflection, maybe we can use use buildinfo for other items, I'll need to try

library: foo
<buildinfo data>

Still we'd want to run configure only one at most, as again all invocations should be identical.

EIDT: Update, this does not work.

[angerman]

I'll try to use a slightly different approach, by ensuring we run configure at least once. I'm sill unsure if this is sound as outlined in #10965; however it would be a strict improvement over the status quo nonetheless.

[Copilot]

Pull Request Overview

A PR to update cabal's handling of build-type: Configure to work with components by distinguishing main library and executable components from other components.

• Update changelog entries to reflect significant changes.
• Adjust test output for expected configuration messages.
• Modify setup and project planning logic to bypass running configure for non-main-library/executable components.

Reviewed Changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
changelog.d/pr-10966.md	Updated changelog to describe the new behavior for build-type: Configure.
cabal-testsuite/PackageTests/Configure/cabal.out	Revised expected output to reflect updated configure messages.
cabal-install/src/Distribution/Client/SetupWrapper.hs	Added isMainLibOrExeComponent flag and modified build-type override logic.
cabal-install/src/Distribution/Client/ProjectPlanning.hs	Revised configuration planning to conditionally apply build-type.
cabal-install/src/Distribution/Client/Configure.hs	Propagated default flag value for isMainLibOrExeComponent in configuration.

Comments suppressed due to low confidence (1)

cabal-testsuite/PackageTests/Configure/cabal.out:7

• The expected output strings have changed (e.g., updating from '(lib:zlib)' to '(lib)' and modifying the configuration message). Please confirm that all tests have been updated accordingly to reflect these new messages.

- zlib-1.1 (lib:zlib) (first run)

[mpickering]

It's a good thing to work on to remove these reasons for whole package builds but I'm not sure this is the right approach.

Can you explain how this approach avoids the issue which is described in #4548 ?

It seems that at a glance, you will still get two concurrent runs of configure (for the library and executable components)

It would be good to also add a test.

My instinct is that you should instead modify the Configure build-type to use autoconfSetupHooks rather than autoconfUserHooks. Then when cabal-install supports per-component builds for Hooks build type, then ./configure will automatically work. SetupHooks are designed to allow you to specify that some parts run once per package, and other parts once per component.

[sheaf]

Currently, build-type: Configure uses autoconfSetupHooks, which means it is using build-type: Hooks in practice. Hence, it seems to me the correct fix would be to ensure packages with build-type: Hooks use per-component builds. I don't remember the precise details of what is left to do; but #9986 has some information.

I don't think we can simply remove the cuz_buildtype check and call it a day. The check is there because the implementation can't deal with e.g. building a package with multiple sublibraries. If you remove the check, then you expose users to internal errors.

[angerman]

It's a good thing to work on to remove these reasons for whole package builds but I'm not sure this is the right approach.

Can you explain how this approach avoids the issue which is described in #4548 ?

I first opened #10965, because I though the naive thing would be to just ensure configure is run in separate folders. Turns out, it already is. So there is no more running concurrently in the same folder, sure it still runs n times (once per component) and this is wasteful. Even worse while you can do codegen (e.g. creation of header files in configure script, well you can do all kinds of stuff after all it's just a shell script), you can't influence anything but the main library and the executable components with the .buildinfo file. The other is that the directory layout right now would even mean you aren't really in what cabal-install considers the final build directory for components.

It seems that at a glance, you will still get two concurrent runs of configure (for the library and executable components)

Yes you will. But they will be in separate directories. Hooray for cabal-install being sensbile about no global CWD state anymore.

It would be good to also add a test.

That can be added.

My instinct is that you should instead modify the Configure build-type to use autoconfSetupHooks rather than autoconfUserHooks. Then when cabal-install supports per-component builds for Hooks build type, then ./configure will automatically work. SetupHooks are designed to allow you to specify that some parts run once per package, and other parts once per component.

While making the Custom and Hooks setups separate components is indeed a worthwhile goal, I'll leave this to @andreabedini who has already been working on that for Setup/SetupHooks based build-types.

@sheaf

Hence, it seems to me the correct fix would be to ensure packages with build-type: Hooks use per-component builds. I don't remember the precise details of what is left to do; but #9986 has some information.

As outlined above SetupHooks.hs (and Setup.hs) would basically require to have a proper Setup component inserted on which other components depend. Whether or not build-type: Configure could be improved once a Setup component exists, would need to be seen. Right now without the existence of Setup components, making build-type: Configure work, is a step in the right direction, I think.

---

I thought for a while if I were to call this out or not. I've got fairly thick skin and am not much bothered by this, but:

I don't think we can simply remove the cuz_buildtype check and call it a day.

comes across extremely hostile. This change did a lot more than simply remove the cuz_buildtype check; it also passed the complete cabal test-suite, across multiple compilers and operating systems, without disabling any existing tests. The only change to the test-suite is a pretty-print change that stems from the change of Package to Component building which do not have identical textual output.

[mpickering]

While making the Custom and Hooks setups separate components is indeed a worthwhile goal, I'll leave this to @andreabedini who has already been working on that for Setup/SetupHooks based build-types.

I think there is a misunderstanding here. I am suggesting that extending the Hooks build-type work so that using Hooks build type does not force you to do per-package builds would also fix this issue.

Making Setup into a proper component does also sound good though!

To be clear as well, what is the motivation for this change? Is it self-motivated or is it because you want to support Configure type with multiple public sublibraries?

Yes you will. But they will be in separate directories. Hooray for cabal-install being sensbile about no global CWD state anymore.

I see, that's an important difference, I tried building network (after modify cabal-install to remove the Cuz_Configure clause) and that does work running ./configure concurrently.

---

To summarise my thoughts:

• ./configure should only be run once per package, so a solution using Hooks (which allows you express precisely this) would be better than the proposed solution, but much more work.
• As an interim solution, it doesn't seem too bad to run ./configure once per component. Since the original check was put in place, it seems we now run ./configure in a separate directory.
• If the motivation is to do with public sublibraries not working with 'Configure' build-type, then that is a separate issue to be fixed in cabal-install (and it doesn't look insurmountable, a medium amount of work).

[sheaf]

I thought for a while if I were to call this out or not. I've got fairly thick skin and am not much bothered by this, but [...] comes across extremely hostile.

I apologise, it wasn't at all my intention to dismiss the work you put into finding, implementing and testing a surgical change that improves the situation for build-type: Configure.

[angerman]

@mpickering

I think there is a misunderstanding here. I am suggesting that extending the Hooks build-type work so that using Hooks build type does not force you to do per-package builds would also fix this issue.

I'm not sure there is. Hooks and Custom both require a proper phase distinction, they both come with haskell scripts. The hooks documentation mentions creating a SetupHooks.hs, and my experience with Hooks is that cabal will ask for that file. In my case I am specifically looking for an option that doesn't need Setup.hs or SetupHooks.hs. Thus build-type: Hooks does not seem like an option, build-type: Configure (and build-type: Make, although that's a much bigger hammer), do work.

Making Setup into a proper component does also sound good though!

Yes, I'm glad we can all agree on this!

To be clear as well, what is the motivation for this change? Is it self-motivated or is it because you want to support Configure type with multiple public sublibraries?

This is to turn a build-type: Configure package into one with public sub libraries. build-type: Hooks (or build-type: Custom) are not viable options in this case.

In general I see build-type: Configure and distinct from build-type: Hooks and build-type: Custom specifically because the latter two come with a dependency on a native haskell compiler, while the former does not.

@mpickering

If the motivation is to do with public sublibraries not working with 'Configure' build-type, then that is a separate issue to be fixed in cabal-install (and it doesn't look insurmountable, a medium amount of work).

I'm not sure I understand that line. Yes this is motivated by a public sublibraries package with build-type: Configure, and this PR touches cabal-install only. So we are in agreement here with this PR? Good!

@sheaf

I apologise, it wasn't at all my intention to dismiss the work you put into finding, implementing and testing a surgical change that improves the situation for build-type: Configure.

Much appreciated! Let's make lib:Cabal and cabal-install better, one step at a time!

[andreabedini]

Allow me to add one data point. I have only recently realised that Haskell.nix build cabal packages by component, always. For any build type, haskell.nix will compile Setup.hs (or use the default one if missing) and then call ./Setup configure <component> and ./Setup build <component>. Regardless whether this is "proper" or not; Haskell.nix is extensively used in the Cardano ecosystem and very well battle tested. The simple fact that I didn't even notice seems to suggest that in practice this works. I haven't looked back at the survey that @mpickering, @sheaf and others had done for the SetupHooks proposal but my gut feeling is that the documented use of custom build-types would still work per-component.

I only skimmed the discussion but I agree that moving configure build-type to autoconfSetupHooks and making setup a real component¹ would be both great improvements. That said I think allowing build-type configure to work by component is something we can safely do right away because 1) it goes in the same direction of the other two changes above 2) I believe any unintended breakage can be dealt with by fixing the package (or moving it to setup-hooks?).

Footnotes

1. I might have a go at this very soon. ↩

[mpickering]

@angerman,

I mean that cabal-install should support doing whole package builds of packages which have multiple sublibraries. At the moment, this patch doesn't fix that since you can't use Hooks or Custom build-type.

If you are using cabal-install then currently using build-type: Configure depends on having a native Haskell compiler available, since it is implemented by creating a Setup.hs script and compiling it. If you are using Cabal directly via Setup.hs then you must also compile that with a native compiler. It does seem possible in the future to run the configure step directly from cabal-install and @sheaf is working on some patches towards that.

@andreabedini That is interesting to know, I didn't know haskell.nix was implemented like that. It is an interesting idea to ignore the package/component difference. However, I do think it is important to maintain the difference, this is codified in the design of 'Hooks' and also means you can avoid repeating expensive work (such as running ./configure scripts once per component).

As explained in the previous summary, I think it is fine to merge this patch but I think there are some trade-offs.

[mpickering]

I will approve once the commit message/comments are updated to reflect the discussion and there is a testcase.

Thanks @angerman :)

[andreabedini]

If you are using cabal-install then currently using build-type: Configure depends on having a native Haskell compiler available, since it is implemented by creating a Setup.hs script and compiling it.

This is not correct. Only Custom and Hooks need an external setup. Simple, Configure and Make are allowed to use internal setup or self-exec.

https://github.com/haskell/cabal/blob/96ce5a8dd46978425d0f4217ab5c5620991b2757/cabal-install/src/Distribution/Client/SetupWrapper.hs#L405C1-L416

https://github.com/haskell/cabal/blob/96ce5a8dd46978425d0f4217ab5c5620991b2757/cabal-install/src/Distribution/Client/SetupWrapper.hs#L553C1-L560C56

[andreabedini]

@angerman You write:

  -- ^ Let the setup script logic know if it is being run to build a main
  -- library or executable component.  This is used to determine if we should
  -- use the configure command, if the build-type is 'Configure'.  For
  -- configure, only the main library and execomponents have 'configure'
  -- support, and thus we can skip running configure for other components.

I don't understand this comment. The configure command for Build-Type: Configure can work per component.

Whole package:

λ cabal act-as-setup --build-type=Configure -- build
Preprocessing library 'an-internal-library' for my-pkg-0.1.0.0...
Building library 'an-internal-library' for my-pkg-0.1.0.0...
[1 of 1] Compiling Lib              ( src/Lib.hs, dist/build/an-internal-library/Lib.o, dist/build/an-internal-library/Lib.dyn_o )
Preprocessing executable 'the-executable' for my-pkg-0.1.0.0...
Building executable 'the-executable' for my-pkg-0.1.0.0...
[1 of 1] Compiling Main             ( app/Main.hs, dist/build/the-executable/the-executable-tmp/Main.o )
[2 of 2] Linking dist/build/the-executable/the-executable

A single component:

λ cabal act-as-setup --build-type=Configure -- clean
cleaning...

λ cabal act-as-setup --build-type=Configure -- configure an-internal-library
Configuring library 'an-internal-library' for my-pkg-0.1.0.0...

λ cabal act-as-setup --build-type=Configure -- build an-internal-library
Preprocessing library 'an-internal-library' for my-pkg-0.1.0.0...
Building library 'an-internal-library' for my-pkg-0.1.0.0...
[1 of 1] Compiling Lib              ( src/Lib.hs, dist/build/an-internal-library/Lib.o, dist/build/an-internal-library/Lib.dyn_o )

(Note that after configure an-internal-library, a simple build would have built only the configured component).

[mpickering]

If you are using cabal-install then currently using build-type: Configure depends on having a native Haskell compiler available, since it is implemented by creating a Setup.hs script and compiling it.

This is not correct. Only Custom and Hooks need an external setup. Simple, Configure and Make are allowed to use internal setup or self-exec.

https://github.com/haskell/cabal/blob/96ce5a8dd46978425d0f4217ab5c5620991b2757/cabal-install/src/Distribution/Client/SetupWrapper.hs#L405C1-L416

https://github.com/haskell/cabal/blob/96ce5a8dd46978425d0f4217ab5c5620991b2757/cabal-install/src/Distribution/Client/SetupWrapper.hs#L553C1-L560C56

Thanks @andreabedini, you are right. I was confused by the code in getExternalSetupMethod. Sorry for the confusion and confidently asserting something I hadn't checked myself recently.

[angerman]

@andreabedini

I don't understand this comment. The configure command for Build-Type: Configure can work per component.

Sure you can run it per for components other than the main library or executables (also components), but there is no way that the configure run can influence any such components. The .buildinfo support only applies to the main library and executables, there is no way to produce a .buildinfo file that impacts other components than the main library or executables. As outlined in #10965, any generated files are inaccessible anyway. The configure script is also not informed about the component it's ran against. Thus any configure run is identical for all components, and can produce .buildinfo files only for the main library and executable components. Thus there seems no use of running configure for any other components than the main library and executable components.

[angerman]

@mpickering amended the commit message, I hope it includes all the relevant information you want, if not, please let me know which ones specifically you'd like me to add.

Also: test-case added.

[Mikolaj]

@mpickering amended the commit message, I hope it includes all the relevant information you want, if not, please let me know which ones specifically you'd like me to add.

Also: test-case added.

After @mpickering approves, we still need one more review. @philderbeast: would you be inclined to provide it?