Repository navigation
Speed Up Content Resolver #95
Description
Content Resolver takes a long time to run. I'm creating this as an "Epic" to discuss, and spin off other issues/feature requests.
My initial thoughts on what we can do to speed up Content Resolver:
- Parallelize analyze_srpm_buildroots
- Parallelize analyze_workloads
- Split ELN Extras to a separate builder.
- Split each "View" to it's own separate builder.
Activity
tdawson commentedon Sep 12, 2025
Parallelize analyze_workloads
Although it is running "async" the reality is that it's happening one at a time. Let's see if we can truly parallelize it.
tdawson commentedon Sep 12, 2025
Split each "View" and / or "ELN Extras" to it's own builder
I currently don't know how to do this and keep them all in sync. But it would be good to discuss and possibly implement.
Possibly it wouldn't matter if they keep in sync, possibly it does.
tdawson commentedon Oct 3, 2025
Splitting out the views.
Currently we have ELN, ELN-Extras and C10S.
ELN and ELN-Extras are tied together, because Extras depends on ELN.
But C10S could easily be split apart.
I have a Content Resolver branch that has a command line option to do different views.
https://github.com/tdawson/content-resolver/tree/feature/filter-views-at-config-load
I've tested it with c10s successfully. I'm currently testing it with eln-extras, which should automatically pull in eln.
It needs a few tweaks, because it's currently requiring the view_id (view-eln) instead of the label (eln), but that shouldn't be hard to change. But other than an awkward command line option, it seems to be working very well.
But that brings up the question. If we split them apart, and put them both on separate machines, what would we do with the website.
Would we create a static front page that points to the various views?
Would we try to combine them into one?
Would we get a seperate URL for C10S and just tell people to go there? ( https://tiny.distro.builders/c10s/ )
tdawson commentedon Oct 22, 2025
I have updated the original code to use labels instead of views, plus a few other tweaks to show when it's being used and what has changed.
It works pretty good, and I have a merge request here:
#98