New RFC: Switching from `publishDir` to workflow output

[maxulysse]

Have you read the RFC docs?

• Yes, I have read and understood the RFC docs

Summary

Workflow output are still in preview mode, but the current third syntax is stable enough and I think vastly superior to what we do with publishDir.
Because instead of publishing files from the process, we publish them from the channels.

Champion

@maxulysse

Background & Motivation

For me the new workflow output is one of the step to help go done the way to pipeline chaining.
And I think publishing files from the channels makes more sense in a Nextflow kind of way than doing it at the process level.

Goals

• Migrate selected pipelines
• Write blogpost to help migrating
• update TEMPLATE

Non-Goals

• Migrate all pipelines

References

• Ben's PR prototype: Update to Nextflow 25.04 fetchngs#347
• Nextflow docs: https://nextflow.io/docs/latest/workflow.html#publishing-outputs
• Podcast: https://www.youtube.com/watch?v=aRpsuub5BsA
• Pipelines already using this: https://github.com/nf-core/references/

[nf-core-bot]

RFC approval status: 🕐 Pending

RFC has approvals from 0/7 required @core-team quorum.

Review Status	Core Team members
❌ Rejected	@mashehu, @ewels, @FriederikeHanssen, @jfy133, @nvnieuwk, @mribeirodantas
🕐 Pending	@maxulysse, @JoseEspinosa, @mirpedrol, @kenibrewer, @edmundmiller, @sateeshperi, @FranBonath, @christopher-hakkaart

[maxulysse]

I nominate @nvnieuwk @edmundmiller and @FriederikeHanssen as co-champion

[mashehu]

/reject too soon. needs to be out of preview mode first imo

[ewels]

/reject - agree with @mashehu.

I'd be happy to see work towards POC branches for specific pipelines, early development work etc. But I do not think that we should be releasing production pipelines or touching the template with this yet.

[maxulysse]

The way I see it, this can still be planned and not put into production until it's phased outside of preview.
We used DSL2 in preview mode for some time, no-one seemed to see any issue there.

[FriederikeHanssen]

Agree with @ewels . Sorry :(

I would vote for some of the bigger pipeline with elaborate publishing logic making a serious effort to implement this though so we can catch any remaining edge cases that are clunky or unsupported (although time will be the enemy here I think). We started some of this but abandoned it quickly with v1 or v2 because of limitations.

/reject