Add Fastjet #21052

lgray · 2022-11-07T16:14:44Z

Checklist

conda-forge-linter · 2022-11-07T16:15:06Z

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipes/fastjet) and found it was in an excellent condition.

lgray · 2022-11-07T16:17:06Z

@jpivarski @chrispap95 @jmduarte could you please give a thumbs-up here to indicate your willingness to be a maintainer?

jpivarski · 2022-11-07T16:23:48Z

I can be listed as a maintainer for the conda-forge recipe, just as I am for uproot and awkward, but I'm not actually familiar with how conda-forge recipes work. I can't be the only maintainer because if something goes wrong, I'll need to reach out for help.

But I'm happy to be in the loop about whether or not the conda-forge fastjet packages are being deployed correctly.

lgray · 2022-11-07T17:21:58Z

OK - so the problem appears to be that boost isn't being put into the include path in a reasonable way. It's being a bit stubborn. Perhaps @chrisburr has some idea how to patch it in.

lgray · 2022-11-07T21:49:02Z

  /home/conda/staged-recipes/build_artifacts/fastjet_1667857508475/work/CGAL-5.5.1/include/CGAL/config.h:127:10: fatal error: boost/config.hpp: No such file or directory
    127 | #include <boost/config.hpp>
        |          ^~~~~~~~~~~~~~~~~~

Yep - quite unsure what else to try to fiddle with. Boost doesn't seem to install correctly for the compiler to pick it up via conda (despite mpfr and gmp working fine).

chrisburr · 2022-11-08T09:34:05Z

Why are you builing CGAL?

chrispap95 · 2022-11-08T09:37:59Z

CGAL headers are needed to provide NlnN methods at very high multiplicities. We can build without CGAL but that would strip some functionality off fastjet.

chrisburr · 2022-11-08T09:48:56Z

You should be able to get CGAL as a dependency rather than building it: https://github.com/conda-forge/cgal-cpp-feedstock

chrispap95 · 2022-11-08T09:55:22Z

We are not building CGAL. We are only using the headers. Is this going to make the headers available?

chrisburr · 2022-11-08T10:08:40Z

We are not building CGAL. We are only using the headers. Is this going to make the headers available?

Yes.

chrispap95 · 2022-11-08T10:14:39Z

Okay, I see. Unfortunately, I don't have write access to the PR. I guess @lgray will try this later.

chrisburr · 2022-11-08T15:17:24Z

recipes/fastjet/meta.yaml

+  missing_dso_whitelist:
+    - $RPATH/libfastjet.so.0
+    - $RPATH/libfastjettools.so.0


You probably shouldn't be using this, if the CI fails because of it there is probably a bug.

That was your addition to the meta.yaml on the previous iteration of this pr. a9b9037 . There is very likely some bug or weird workaround going on (looks like you were debugging it), judging from that.

How do I kick the CI to get it to start again? It seems to have stopped at linting this time.

That was me? 😕 I have no recollection so I say remove it and maybe I'll get to the same conclusion again 😆

How do I kick the CI to get it to start again? It seems to have stopped at linting this time.

Either force push an amended commit, push an empty commit or close and reopen. I think GitHub is having issues as I'm seeing lost web hooks in other organisations as well.

@chrisburr can you give some references on how to debug overlinking errors that are releated to things similar to this? I'm not experienced here, but that seems to be the only thing that is fialing in my local tests.

lgray · 2022-11-08T16:06:44Z

indeed --with-cgaldir was causing problems and now it seems to build fine, but it fails when messing about with the libraries.

chrisburr · 2022-11-08T16:23:23Z

recipes/fastjet/meta.yaml

@@ -26,11 +26,13 @@ requirements:
    - automake
    - autoconf
    - swig
+    - cgal-cpp


This belongs in host.

yes just noticed that!

chrisburr · 2022-11-08T16:59:12Z

You also need gmp in host:

  ERROR (fastjet,lib/python3.10/site-packages/fastjet/_fastjet_core/lib/libfastjet.so.0.0.0): Needed DSO lib/libgmp.so.10 found in ['gmp']

lgray · 2022-11-08T17:00:30Z

@chrisburr so it's getting all the way to the point where it's built the libraries, etc. However, there are massive complaints of overlinking in the final products. I've put the meta.yaml back into a state without the dso_whitelist and that's about the limit of what I know how to do effectively as far as debugging goes.

lgray · 2022-11-08T17:01:41Z

OK - trying that... Let's see :D

chrisburr · 2022-11-08T17:05:33Z

You'll probably also need to add the dso whitelist stuff to at least see it pass (or see it fail with a missing shared library when testing).

Tomorrow I can run a build locally and figure out why the fastjet libs are badly linked.

chrisburr · 2022-11-08T17:39:32Z

I suspect you're now seeing a swig 4.1+ thing, basically ClusterSequence_fastjet_banner_stream -> ClusterSequence.fastjet.banner.stream. (This also works with older swig versions.)

conda-forge-webservices · 2023-03-07T15:17:00Z

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipes/fastjet) and found it was in an excellent condition.

lgray · 2023-03-07T16:06:29Z

Something weird is happening with fastjet contrib. @chrisburr help appreciated. The rest seems to be building happily now, though, so we're perhaps closer than before.

lgray · 2023-03-07T16:17:34Z

~~Oh, I think I see...~~ No, no idea why it's suddenly failing.

chrisburr · 2023-03-07T16:28:52Z

It's trying to use g++ and not respecting $CXX when building fastjet-contrib.

lgray · 2023-03-07T16:40:09Z

Glorious.

lgray · 2023-03-28T17:48:44Z

@chrisburr so what's going on is that I need to supply CXX to the ./configure of fastjet-contrib but it's parsing the rest of the arguments in a completely obtuse way that I don't understand so if I specify more than CXXFLAGS or CXX individually one argument eats the other.

Help appreciated. :-)

chrisburr · 2023-04-20T05:43:42Z

From a glance at the commits you've tried I've noticed something odd:

                    f'CXX={env.get("CXX", "g++")} CXXFLAGS=-O3 -Bstatic -Bdynamic -std=c++17',

These aren't split correctly, you're setting CXX to {env["CXX"} CXXFLAGS=-O3 -Bstatic -Bdynamic -std=c++17.

You probably need something like:

                    f'CXX={env.get("CXX", "g++")}',
                    "CXXFLAGS=-O3 -Bstatic -Bdynamic -std=c++17",

lgray · 2023-04-20T13:46:36Z

The current issue with not completing the build seems unrelated or am I reading that wrong? It looks like somehow the fastjet contrib package is missing files or they are mislocated???

Co-authored-by: Matthew Feickert <[email protected]>

* fastjet-contrib's .Makefile.inc is created at ./configure time, which means that is the environment's CXX is accessible inside of setup.py it can be passed in as an argument to configure. Use this to set CXX for fastjet-contrib/.Makefile.inc and use a default value of g++ if no environment value is set. * Remove g++ from build requirements.

* Add the libsiscone.so.0, libsiscone_spherical.so.0, and libfastjetplugins.so.0 shared libraries to the missing_dso_whilelist. All exist under lib/python3.*/site-packages/fastjet/_fastjet_core/lib/. * Add gmp to the 'run' requirements as fastjet does not currently package this shared library but it is required at runtime. * Amend the 0001-Don-t-overwrite-external-CXXFLAGS-and-LDFLAGS.patch. - Fully remove the logic for donwloading CGAL as this is covered by the 'host' requirements. - Move the update of the CXX environment variable till after the os.environ.copy() called just before the fastjet-contrib ./configure to ensure that a default exists. - Remove the '$$' from '$ORIGIN', keeping ony one '$'. This is important, else the LDFLAGS will not resolve properly.

Co-authored-by: Matthew Feickert <[email protected]>

* Patch fastjet-contrib/Makefile.in to use the environment variable $LDFLAGS during the build of the libfastjetcontribfragile shared library. * Patch setup.py to properly set LDFLAGS and provide the environment variables at ./autogen.sh time and ./configure time when the fastjet-core and fastjet-contrib Makefiles are being generated. * Move gmp to the 'host' requirements. * Sort the 'build' and 'host' requirements.

* Use single quotes and '$$' to properly guard ORIGIN when LDFLAGS are injected into Makefiles. e.g. '$$ORIGIN/_fastjet_core/lib' Without this guard, when Make evaluates the variable $ORIGIN will get evaluated as RIGIN.

* As conda-forge's build procedure was trying to search through other CPython versions than the target's CPython in the build_artifacts it was then attempting to link incorreclty and causing overlinking errors. If a wheel is built first and then fastjet is installed from that wheel, then the resulting shared libraries will be linked correctly and the build can pass without errors.

Co-authored-by: Matthew Feickert <[email protected]>

xhochy

Overall looking good. We can merge on gree CI

matthewfeickert · 2024-02-22T20:45:34Z

Overall looking good. We can merge on green CI

Fantastic! Thank you for your reviews @xhochy!

matthewfeickert · 2024-02-22T21:06:49Z

(While this is obvious to people who know, for those who might get linked here by me) The feedstock generated from this PR can be found over on https://github.com/conda-forge/fastjet-feedstock 👍

lgray changed the title ~~Fastjet~~ Add Fastjet Nov 7, 2022

chrisburr reviewed Nov 8, 2022

View reviewed changes

jmduarte mentioned this pull request Nov 10, 2022

Add fastjet to conda-forge scikit-hep/fastjet#133

Open

lgray force-pushed the fastjet branch from b0a783a to 3510304 Compare April 20, 2023 13:46

lgray and others added 20 commits February 22, 2024 13:45

also patch CXX in fjcontrib

d153b21

brackets

af98cd9

extract CXX from python env

398ca78

CXX was gobbling up the CXXFLAGS def?

78333c5

try CXX at cli, config parsing seems weird

ed73d88

will fix with patch in fastjet pip; wait for next release

2d3346d

see if 3.4.0.6 gets us anywhere

315ea59

magical build recipe

d3be59b

Co-authored-by: Matthew Feickert <[email protected]>

appease linty boye

d6ffdca

ok...

d51d5c1

even more ok...

2677e5b

just to try...

4c34ec1

add Matthew

efe9be1

Co-authored-by: Matthew Feickert <[email protected]>

if you build a runway the planes will come

2110ef3

fix: Use ORIGIN in RPATH inside of Makefiles

5daf4fd

* Use single quotes and '$$' to properly guard ORIGIN when LDFLAGS are injected into Makefiles. e.g. '$$ORIGIN/_fastjet_core/lib' Without this guard, when Make evaluates the variable $ORIGIN will get evaluated as RIGIN.

build -> python-build

22e74fe

Co-authored-by: Matthew Feickert <[email protected]>

lgray force-pushed the fastjet branch from 8f632cc to 22e74fe Compare February 22, 2024 19:45

xhochy approved these changes Feb 22, 2024

View reviewed changes

ocefpaf merged commit 4c9c6dc into conda-forge:main Feb 22, 2024
5 checks passed

lgray deleted the fastjet branch February 22, 2024 21:05

matthewfeickert mentioned this pull request Feb 22, 2024

Fix linking of shared libraries for local pip install . scikit-hep/fastjet#280

Open

matthewfeickert mentioned this pull request Mar 20, 2024

feat: Add hepconvert #25789

Merged

9 tasks

matthewfeickert mentioned this pull request Apr 29, 2024

Add to conda-forge ssl-hep/ServiceX_frontend#352

Closed

matthewfeickert mentioned this pull request May 13, 2024

feat: Add macos to build conda-forge/fastjet-feedstock#9

Draft

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Fastjet #21052

Add Fastjet #21052

lgray commented Nov 7, 2022 •

edited

Loading

conda-forge-linter commented Nov 7, 2022

lgray commented Nov 7, 2022

jpivarski commented Nov 7, 2022

lgray commented Nov 7, 2022

lgray commented Nov 7, 2022

chrisburr commented Nov 8, 2022

chrispap95 commented Nov 8, 2022

chrisburr commented Nov 8, 2022

chrispap95 commented Nov 8, 2022

chrisburr commented Nov 8, 2022

chrispap95 commented Nov 8, 2022

chrisburr Nov 8, 2022

lgray Nov 8, 2022 •

edited

Loading

chrisburr Nov 8, 2022

matthewfeickert Feb 16, 2024

lgray commented Nov 8, 2022

chrisburr Nov 8, 2022

lgray Nov 8, 2022

chrisburr commented Nov 8, 2022

lgray commented Nov 8, 2022

lgray commented Nov 8, 2022

chrisburr commented Nov 8, 2022

chrisburr commented Nov 8, 2022

conda-forge-webservices bot commented Mar 7, 2023

lgray commented Mar 7, 2023

lgray commented Mar 7, 2023 •

edited

Loading

chrisburr commented Mar 7, 2023

lgray commented Mar 7, 2023

lgray commented Mar 28, 2023

chrisburr commented Apr 20, 2023

lgray commented Apr 20, 2023

xhochy left a comment

matthewfeickert commented Feb 22, 2024 •

edited

Loading

matthewfeickert commented Feb 22, 2024

Add Fastjet #21052

Add Fastjet #21052

Conversation

lgray commented Nov 7, 2022 • edited Loading

conda-forge-linter commented Nov 7, 2022

lgray commented Nov 7, 2022

jpivarski commented Nov 7, 2022

lgray commented Nov 7, 2022

lgray commented Nov 7, 2022

chrisburr commented Nov 8, 2022

chrispap95 commented Nov 8, 2022

chrisburr commented Nov 8, 2022

chrispap95 commented Nov 8, 2022

chrisburr commented Nov 8, 2022

chrispap95 commented Nov 8, 2022

chrisburr Nov 8, 2022

Choose a reason for hiding this comment

lgray Nov 8, 2022 • edited Loading

Choose a reason for hiding this comment

chrisburr Nov 8, 2022

Choose a reason for hiding this comment

matthewfeickert Feb 16, 2024

Choose a reason for hiding this comment

lgray commented Nov 8, 2022

chrisburr Nov 8, 2022

Choose a reason for hiding this comment

lgray Nov 8, 2022

Choose a reason for hiding this comment

chrisburr commented Nov 8, 2022

lgray commented Nov 8, 2022

lgray commented Nov 8, 2022

chrisburr commented Nov 8, 2022

chrisburr commented Nov 8, 2022

conda-forge-webservices bot commented Mar 7, 2023

lgray commented Mar 7, 2023

lgray commented Mar 7, 2023 • edited Loading

chrisburr commented Mar 7, 2023

lgray commented Mar 7, 2023

lgray commented Mar 28, 2023

chrisburr commented Apr 20, 2023

lgray commented Apr 20, 2023

xhochy left a comment

Choose a reason for hiding this comment

matthewfeickert commented Feb 22, 2024 • edited Loading

matthewfeickert commented Feb 22, 2024

lgray commented Nov 7, 2022 •

edited

Loading

lgray Nov 8, 2022 •

edited

Loading

lgray commented Mar 7, 2023 •

edited

Loading

matthewfeickert commented Feb 22, 2024 •

edited

Loading