Lightweight project planning process

[raghotham]

As discussed in the community office hours on 11/20, we would like to add a lightweight project planning process for llama stack so that we can provide some level of predictability to the community on what to expect by when.

As a first iteration, here's what we could have:

Schedule for releases

• Patch (Z stream) releases - roughly every week
• Minor version (Y stream) releases - roughly every 2 months
• Major version (X stream) releases - roughly every 6-8 months

Milestones

• Create for each minor version and major version releases
• Add only issues to milestones, not PRs. Small PRs that come and landed quickly without issues dont need to be tracked in a milestone. Large PRs that remain open for longer should have a corresponding issue which would be in the milestone.
• A version is not released until all issues for the corresponding milestone are completed OR remaining issues are moved to a future milestone with a rationale.
• Milestones are triaged by TRIAGERS, discussed in the #triage discord channel, and on the weekly community calls as time permits

Release process

• Each release will have a release owner - who would be one of the CODEOWNERS
• Release owner would create a thread in the #release discord channel corresponding to the release
• Community members are welcome to help test release candidates
  • Z stream releases would mostly rely on automated tests in the CI workflows
  • Y stream releases would include manual testing of new features on the day of the release and documentation checks in addition to CI
  • X stream releases would include a scheduling testing period with test plans prior to the release date
• Once all tests pass on the release candidates, release owner would trigger the publish flows to publish artifacts and update documentation
• Once the release is done, the release owner will announce the release and release notes in #announcements channel in discord

[cdoern]

I think this is a great idea, a few ideas for additional criteria for our Milestones:

• A version is not released until all issues and PRs for the corresponding milestone are completed OR remaining issues/PRs are moved to a future milestone with a rationale.

We should probably add one of these:

• Only issues are allowed in Milestones, otherwise we end up having duplicate work tracked in a single milestone, causing them to be hard to refine.

Or:

• Issues and PRs are allowed in milestones. If a large singleton PR is important for a release but not associated with an issue, it can be added to a milestone.

I have seen both of these work. Having both PRs and issues is more natural and people tend to gravitate towards that, but it can be hard to refine a milestone with PRs + issues. I do think that if PRs and issues are both allowed, then we should only allow "singleton" PRs. This means that if someone opens an important PR without an issue, it can be in the milestone. However, an issue and its associated PRs should not both be in the milestone.

[ashwinb]

I think it would be simpler to do the first of these -- just use Issues. Any PR which needs to be in a Release officially should have an Issue attached. Of course I am assuming it is not a requirement for all PRs but only for PRs that tend to be open for longer: small PRs which come and land relatively quickly automatically do get included in Releases.

[raghotham]

Updated the description based on the comments. Please check

[cdoern]

looks good! thanks @raghotham

[ashwinb]

Should we mention something about how discussion about a release will be managed -- right now we do this ad-hoc in the Releases channel.

[raghotham]

Something like this?

Release process

• Each release will have a release owner - who would be one of the CODEOWNERS
• Release owner would create a thread in the #release discord channel corresponding to the release
• Community members are welcome to help test release candidates
  • Z stream releases would mostly rely on automated tests in the CI workflows
  • Y stream releases would include manual testing of new features on the day of the release and documentation checks in addition to CI
  • X stream releases would include a scheduling testing period with test plans prior to the release date
• Once all tests pass on the release candidates, release owner would trigger the publish flows to publish artifacts and update documentation
• Once the release is done, the release owner will announce the release and release notes in #announcements channel in discord

[ashwinb]

sure, sounds good

[raghotham]

updated

[nathan-weinberg]

This looks very good @raghotham - I would maybe make one addition stating something like "this process will be displayed in docs/RELEASE.md file in the llama-stack repo. the codeowners/maintainers reserve the right to update this process via PRs updating the document as they see fit"

[leseb]

Is this a good opportunity to clarify a few more things like:

• what's the content of each release (feature only? bug fixes?)
• the support life cycle of a given release