Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[question] #3849

Open
1 task done
maitrey opened this issue Sep 24, 2024 · 6 comments · May be fixed by #3799
Open
1 task done

[question] #3849

maitrey opened this issue Sep 24, 2024 · 6 comments · May be fixed by #3799
Assignees

Comments

@maitrey
Copy link

maitrey commented Sep 24, 2024

What is your question?

Dear Conan Folks,

Conan does not support reverse dependencies.
In that case, how does a package know its consumer?
Why is this needed as we would like to have fast feedback on already available packages from feature branches and /or pull requests via CI/CD.

Is not this an actual use-case?
Thanks!
Br,
Maitrey

Have you read the CONTRIBUTING guide?

  • I've read the CONTRIBUTING guide
@memsharded memsharded self-assigned this Sep 24, 2024
@memsharded
Copy link
Member

Hi @maitrey

Thanks for your question.

Conan does not support reverse dependencies.
In that case, how does a package know its consumer?

This is not a Conan specific problem, this is intrinsic to any package manager with a decentralized approach, you cannot ask many package managers what packages depend on a given one in the different servers you might have packages for that package manager.

The solution to this is that it is not necessary to know the consumers of a package for doing CI, in fact, even if it was possible it is still not a good approach, as it doesn't really scale.

We are working right not in a new CI tutorial, and you will see there the recommended approach, like the "products pipeline". You can have a look in #3799, it is still missing some parts, but it is already very advanced, so you can read most of it.

Thanks for your feedback!

@memsharded memsharded linked a pull request Sep 24, 2024 that will close this issue
@memsharded memsharded transferred this issue from conan-io/conan Sep 25, 2024
@memsharded
Copy link
Member

Moved this ticket to docs repo

@maitrey
Copy link
Author

maitrey commented Sep 25, 2024

Thank you , I was wondering to have the lockfiles and introduce atleast 3 artifactory repositories(devel, staging and release). For branching strategy there shall be a develop branch and main. Feature branches shall be created from develop branch.At the first feature branch, stage only trigger a build do not upload artifacts to artifactory. When a PR is raised then also build and test and upload to staging repository along with lock files. Use this and pass the recipe revision to next stage where using conan list it gets all the packages from devel and finds out recursively if the rrev is matching in the lockfiles. If not match query for the graph build order and get the nodes with the help of "binary=missing". Collect and merge all the graph build orders. Since we have package names, repository and pipeline names as same, we need to trigger the correct Pipeline and only when they are successful the creator package is allowed to merge to develop. It might also be necessary to develop a queue so that if there are multiple branches in the creator package than the one that was pushed first is considered or all Pull requests are clubbed together so that the costs for the builds are controlled. After merging to develop the packages are published in the devel repo. On a regular basis the devel repo is merged to main along with same steps as for develop branch meaning that build and test , identify consumers of the package trigger their pipelines and once succesful push the artifacts to devel repo. Finally when a tag is created it shall also follow the same procedure as for main along wish publishing artifacts to release repo.

@memsharded
Copy link
Member

Use this and pass the recipe revision to next stage where using conan list it gets all the packages from devel and finds out recursively if the rrev is matching in the lockfiles.

Most likely not necessary. The conan graph build-order will already check for you, and it will able to return an "empty order" or an order without packages to build, because everything is good with the existing revisions.

If not match query for the graph build order and get the nodes with the help of "binary=missing"

No, a binary: Missing in the build-order means that build-order is broken and can't be used. Conan will launch an exception for it. The build order must be calculated with conan graph build-order --build=missing and then the packages with binary: Build are the ones that needs to be built.

@maitrey
Copy link
Author

maitrey commented Sep 25, 2024

Ok, I just upgraded to 2.7.0 and if a lockfile is present, then it says that the current package is not in the requires.If I delete the lock file locally, then its possible to get the build-order and yes you are right about binary:Build. Command that I used is:

conan graph build-order --build=missing -r autosar-releases-conan-local --requires=errmngr/0.0.0-dev.0@auotosar/testfeature --format=json > build-order.json

@memsharded
Copy link
Member

Ok, I just upgraded to 2.7.0 and if a lockfile is present, then it says that the current package is not in the requires.

What do you mean by the "current package"? Why it is not in the lockfile, maybe it should be in the lockfile?
For cases that we want to apply partial locking, that is lock whatever is locked in the conan.lock file, but let Conan resolve the dependencies versions that are not locked in the lockfile, the --lockfile-partial argument can be used. But still it sounds that there is something different in the process that might need to be changed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants