Denoise one or more samples independently — the equivalent of R DADA2's
dada(..., pool = FALSE). For pooled and pseudo-pooled inference see
dada-pooled and dada-pseudo.
# single sample -> stdout or -o
dada2-rs dada sample.derep.json.gz --error-model err.json -o sample.dada.json
# many samples, processed independently and serially
dada2-rs dada derep/*.json.gz --error-model err.json --output-dir dada/ --threads 24Each input may be a FASTQ file (dereplicated in memory) or a JSON file produced
by derep or sample (.json / .json.gz). Pre-dereplicated input avoids
re-reading the FASTQ when iterating on parameters. Output is a JSON object
describing the inferred ASVs.
With a single input the result goes to --output / -o (or stdout). With more
than one input the samples are processed independently and serially — not
pooled — and one {sample}.json per sample is written to --output-dir, which
is then required.
--sample-name — sample identifier written to the output JSON. Defaults to
the filename stem of the input.
--phred-offset — 33 for Sanger / Illumina 1.8+, 64 for Illumina 1.3–1.7.
--prior — FASTA file of prior sequences (uncompressed or gzipped). Each
sequence in the file that matches a dereplicated unique exactly is flagged as a
prior, making it immune to the abundance p-value filter. Prior-based splitting
uses --omega-p instead of --omega-a.
--error-model (required) — JSON error model produced by learn-errors, or converted from an R DADA2 learnErrors() .rds with learnerrors_to_dada2rs.R.
--use-err-in — use the model's err_in matrix instead of err_out.
err_out (the rates estimated from the final transition counts) is the default
and is what R DADA2 uses downstream.
--inherit-err-params — inherit any unspecified algorithm parameters
(omega_*, min_*, detect_singletons, band, homo_gap_p, kdist_cutoff,
kmer_size, no_kmer_screen) from the error model JSON's params block. Any
flag passed explicitly on the CLI still wins.
Without this flag the built-in CLI defaults apply, and a warning is emitted for each CLI value that disagrees with the error model's value. That warning is the point: denoising with parameters that differ from the ones the model was fit under is usually a mistake, and this makes it visible rather than silent.
Note that OMEGA_C is not inherited — see the
parameters page.
These are R's setDadaOpt() parameters and all default to the R values; the
parameters page has the full equivalency table. Omitting a
flag means "inherit from the error model if --inherit-err-params is set,
otherwise use the R default".
| Flag | R option | Default |
|---|---|---|
--omega-a |
OMEGA_A |
1e-40 |
--omega-c |
OMEGA_C |
1e-40 |
--omega-p |
OMEGA_P |
1e-4 |
--min-fold |
MIN_FOLD |
1 |
--min-hamming |
MIN_HAMMING |
1 |
--min-abund |
MIN_ABUNDANCE |
1 |
--detect-singletons |
DETECT_SINGLETONS |
false |
--max-clust |
MAX_CLUST |
0 (unlimited) |
--greedy |
GREEDY |
true |
--use-quals |
USE_QUALS |
true |
--detect-singletons, --greedy, --use-quals and --no-kmer-screen are
tri-state: omit to inherit or use the default, or pass true / false to set
them explicitly.
--band — alignment band radius, R's BAND_SIZE. 16 is the Illumina
default; 32 is recommended for PacBio HiFi 16S amplicons (per the DADA2 LRAS
manuscript); -1 is unbanded (O(n²), rarely needed).
The platform split is not cosmetic. On MiSeq data the band can drop to 8 safely, but 4 breaks; on PacBio HiFi it cannot drop from 32 at all — 32 → 16 already changes the ASV set. See Band size & platform defaults.
--gap-p — gap penalty, R's GAP_PENALTY (default −8).
--homo-gap-p — homopolymer-run gap penalty, R's
HOMOPOLYMER_GAP_PENALTY. When unset it falls back to whatever --gap-p
resolves to, reproducing R's NULL default. PacBio pipelines sometimes set this
closer to 0 (e.g. -1) on the theory that homopolymer indels dominate — but
note that the -1 recommendation originates with 454, not PacBio, and HiFi
data does not have that error mode.
--match / --mismatch — R's MATCH (+5) and MISMATCH (−4). R's
full 4×4 score matrix is commented out upstream; the active path is these
scalars, and dada2-rs matches that path.
--align-backend — nw (default) is Needleman-Wunsch. wfa2 is the
experimental WFA backend (wfa2lib-rs): ASV-equivalent on the Illumina and PacBio
HiFi data tested, but alignments are not byte-identical. wfa2 requires a build
with --features wfa; a default build — and the published crate — errors if it
is selected.
The screen decides which pairs are worth aligning. It does not define the clusters. Set it too aggressively and you lose real comparisons; set it loosely and you pay in alignment time.
--kdist-cutoff — k-mer distance cutoff, R's KDIST_CUTOFF (default 0.42).
Pairs above this distance are not aligned. Lower screens more aggressively
(faster, more false negatives); raise it for divergent sequences.
Tightening this is not free: it acts through partition and error-model bias, not
just by screening out error copies, and n_asv is a misleading metric for
judging it — use set identity. The clean lever is to decouple this cutoff from
the one learn-errors runs under, leaving error learning at 0.42 while
denoising tightens to 0.30; see
KDIST cutoff decoupling.
--kmer-size — k-mer size for the screen and the Raw k-mer vectors, R's
KMER_SIZE (default 5, range 3–8; 8 is a hard ceiling because k-mer indices
must fit in a u16).
For PacBio HiFi, do not leave this at 5: on ~1.4 kb reads the screen becomes
a no-op — nearly every pair is aligned, roughly 4–5× slower at scale. Use k=7
for speed or k=6 to cap memory; both give effectively identical ASVs. Memory
scales as 4^k per Raw (k=5 → 1 KB, k=6 → 4 KB, k=7 → 16 KB, k=8 → 64 KB).
See K-mer screen size.
--no-kmer-screen — disable the screen and align every pair. Much slower;
use only when you suspect the screen is filtering valid comparisons.
Neither flag affects results.
--threads (default 1) — threads for both dereplication and the DADA2
comparison map.
--sample-jobs — multi-input only: how many samples to denoise
concurrently, each on its own threads / sample-jobs sub-pool. A single
sample's comparison map is often too small to feed many threads, so fanning
samples across smaller sub-pools keeps every core fed — roughly 4 threads per
sample is the sweet spot — and bounds memory to this many samples in flight.
Defaults to round(threads / 4), i.e. serial at ≤4 threads. Dial it down for
very large or complex samples if memory is tight.
--output / -o — single input only; write JSON here instead of stdout.
--output-dir — required for more than one input; one {sample}.json per
sample, the same convention as dada-pooled. Created if absent.
--compact — minified JSON instead of pretty-printed.
--gzip — gzip the per-sample files as {sample}.json.gz.
ASV lineage (#306). Each
record in asvs carries, beside its sequence, abundance and birth statistics:
| field | meaning |
|---|---|
cluster |
the ASV's cluster id in this denoising result |
birth_from |
the cluster it split from; null for the initial cluster |
nearest |
the closest other ASV, by substitutions plus indel events |
nearest_hamming, nearest_indels |
substitutions and internal indel events to it; end overhangs are not counted |
birth_from is the lineage at the time of the split; centres move afterwards,
so it need not be the final nearest ASV. Ids are per sample under dada and
dada-pseudo, and pool-wide under dada-pooled, where a referenced ASV may be
absent from a given sample's file. nearest is found after denoising (a k-mer
screen, then one alignment per candidate) and is omitted when there is a single
ASV.
None of these change the ASVs.
--verbose — the run's shape and results to stderr: CPU allocation,
alignment backend, active tuning gates, resident footprint, the index decision,
a phase-times one-liner, and per-cluster progress. The phase attribution and
optimisation counters are not printed; pass
--metrics-json for those.
Under --sample-jobs > 1 the per-round progress records are prefixed with the
sample that produced them, since concurrent output is otherwise
unattributable. With a single sample in flight — or under dada-pooled — the
text is left exactly as DADA2's C layer writes it.
--aux-outputs — emit R-DADA2-parity per-cluster diagnostics in the output
JSON: cluster_stats (n0/n1/nunq/birth_qave/post-hoc p-value),
cluster_quality (mean quality at each reference position), birth_subs (the
substitutions that drove each cluster split), and transitions (a 16 × nq
transition-by-quality matrix). This costs one extra alignment per Raw against
its cluster center, so enable it only when you need the diagnostics.
--cluster-trace — write a single full cluster trace (clusters.json)
here: cluster centers, members with hamming distance / λ / p-value, and birth
metadata. Useful for ASV-calling QC and one-off plots; see
examples/cluster_trace/.
--trace-no-members — omit the per-cluster members array, emitting only
centers and birth metadata.
--trace-min-abund (default 1) — only include trace members at or above
this abundance.
--metrics-json — write a structured record of the run's phase times,
compare attribution, shuffle split, footprint and index decision to this file.
Free to leave on: it collects only what the run already pays for.
--metrics-attribution — add the per-comparison timings (busy, map
parallel efficiency, the screen / DP / al2subs split). This slows the run,
so do not combine it with a timing measurement. Requires --metrics-json.
See Run metrics for the schema and the two levels.
--failed-uniques — write a TSV of uniques that failed to denoise (final
p-value < omega_c, map == null). Tidy long format with a header —
sequence<TAB>sample<TAB>reads — one row per failed unique per sample it
appears in. This answers "what failed to denoise, and how many reads did it
cost?" without hand-joining map back to the derep uniques.
--indel-events, --indel-opportunities (experimental) — tabulate
the insertions and deletions DADA2's error model ignores, from the same final
unique-to-centre alignments that give trans
(#291). Neither changes the
denoising output.
--indel-events: one row per indel event per unique, for every cluster member (correct1 or 0). Columns:sample,cluster,center_len,unique(input index),reads,correct,p,lambdaandcluster_reads(the inputs ofp),nsubs,n_internal(non-terminal events in this unique),type(I/D),center_pos,len,bases,hp_len,terminal,qual_min,qual_mean(inserted bases),qual_left,qual_right(flanking unique bases).hp_lenis the centre length of the homopolymer run the event lengthens or shortens; 0 when the bases are mixed or match neither flanking base.terminalgaps lie outside the first and last aligned centre positions.--indel-opportunities: read-weighted rate tables overcorrectmembers and internal positions, assample,table,key,opportunities,events.sub_qandins_qare keyed by quality (aligned unique bases and substitutions; all unique bases and inserted bases);del_hpby centre run length (bases and deleted bases);ins_hpby centre run length (runs and insertions lengthening them);del_fqby the lower quality of the aligned unique bases either side (centre bases and deleted bases);ins_lenanddel_lenby event length (events only).sub_qsums totrans. Further tables split these:ins_q_novelandins_q_run(insertions matching neither neighbour, or lengthening a run),del_fq_L1…del_fq_L5(del_fqper centre run length, 5 = 5 or more),del_hp_A…ins_hp_T(per run base),del_cqanddel_cq_L<n>(asdel_fq, keyed by the centre's own mean quality at the base instead), anddel_fqcq_L<n>(both, keyedflanking × 128 + centre).
Qualities are the unique's per-position mean, as in trans. With several
inputs, each file holds every sample's rows, sorted by sample.
Both files open with # key: value lines naming the run: dada2_rs_command,
dada2_rs_version, error_model (the path given), error_matrix (err_out,
or err_in under --use-err-in), and for dada-pseudo also round and
reestimated_err_between_rounds. Read them with comment = "#" (pandas) or
comment.char = "#" (R). Fuller provenance is tracked in
#297.
These flags are unstable and off by default. They are not drop-in replacements for the defaults.
--screen-backend — an alternative to the k-mer screen. kmer (default) is
the ESPRIT-style 4^k frequency vector that R/C++ DADA2 uses; minimizer is a
winnowed-minimizer sketch. Neither defines the clusters.
Expect mostly-concordant but not identical results. On every dataset tested the ASV sets and counts agree closely, but there are real differences: typically a fraction of a percent of reads and a small number of low-abundance ASVs (all under ~15 reads in the datasets measured).
It pays off where a lot of screening happens — diverse pools, where most pairs are dissimilar, so the screen runs on everything and the aligner on almost nothing. There the k-mer screen can reach 44–77% of denoising time and this replaces it with under 10%. On low-diversity data the screen is ~1% of runtime and there is nothing to win.
It requires tuning. The default --kdist-cutoff 0.42 is wrong for this
backend — it over-screens by roughly 3×. Use --minimizer-k 8 with a cutoff
derived for your data (~0.64 on diverse Illumina pools, ~0.50 on PacBio HiFi)
via kdist-calibrate --screen-backend minimizer. See
Minimizers as the screen.
--minimizer-k (default 8, range 5–31) — k-mer size for the minimizer
sketch. Independent of --kmer-size, which still governs the frequency screen.
k=8 was chosen on Illumina via a pair audit (k=11 misses real neighbours) and
has not been re-derived per platform the way --kmer-size had to be.
--minimizer-w (default 5, range 1–64) — winnowing window in k-mers. The
sketch retains ~2/(w+1) of positions, so larger w is smaller and faster but
less sensitive. This was never varied in any published measurement — every
result on the findings page is a single w=5 point.
--screen-audit — evaluate both screens on every comparison and align the
union, reporting how often they disagree and, for each disagreement, how many
substitutions the alignment actually found. Answers whether the minimizer screen
passes a superset, a subset, or a different set of the pairs --kdist-cutoff
passes. ASVs are unaffected (audit-only alignments are discarded), but the run
is substantially slower and its timings are not comparable to a normal run.
--wfa-max-edits (default 50, 0 = unbounded) — WFA edit-budget cap, in
edit operations, used with --align-backend wfa2. WFA aborts a pair once it
needs more than this many edits and falls back to the NW path for that pair
(NW-identical there). The budget is an absolute edit count, not a fraction
of read length: denoising only aligns near-identical reads (~99.9% identity), so
real error copies stay a few edits apart regardless of read length, while
divergent non-error-copy pairs that slip past the screen stay bounded. Ignored
by the nw backend.
Internally an edit budget E maps to a WFA cost of E·|gap_p| — e.g. 50·8 =
400 with default scoring. The DADA2RS_WFA_MAX_STEPS environment override is
specified in those raw cost units, not edits. DADA2RS_WFA_LINEAR=1 runs WFA
with gap-linear penalties instead of affine with gap opening 0: the same scores
with one wavefront instead of three, for benchmarking (#51).
--indel-rates, --hp-ceiling, --indel-pair-window,
--indel-end-zone — an indel-aware λ
(#291). R DADA2's λ scores
substitutions only, so a variant that differs from its cluster centre only by
indels is absorbed however abundant it is. Given --indel-rates, each indel
event between a unique and its centre multiplies λ by its probability and
counts toward hamming. The rates are either fitted rates from
learn-errors --indel-rates-out, used as given,
or the --indel-opportunities table of a prior run on the same kind of data,
fitted on load by pooling each quality with its neighbours until it holds 20
events.
| event | probability |
|---|---|
| insertion of k bases | rate per base at the inserted bases' lowest quality (novel and run-lengthening bases fitted apart) × r^(k−1) |
| deletion of k bases | rate per base at the centre run length (1–4, 5+) and the lower flanking quality × r^(k−1) |
| homopolymer-length change (largest run involved ≥ 2) | --hp-ceiling (default 0.05) |
insertion and deletion of equal length within --indel-pair-window (default 5) |
the substitutions they stand for, from the error model |
terminal gaps; indels within --indel-end-zone (default 10) of the 3′ end |
ignored |
The homopolymer ceiling sits above the rate of recurring slippage hotspots
(at most ~8% of the parent on PacBio HiFi, ~0.3% from PCR on Illumina), so those
stay absorbed while a homopolymer-length variant far above it can split. Lower
it for high-fidelity, low-cycle protocols; 1 never splits on homopolymer
changes and off scores them at the measured rates. The end zone suits reads
with a low-quality 3′ tail; set it to 0 where read ends are reliable, such as
HiFi or recent Illumina chemistries. The settings are recorded under
params.indel_lambda. The error model is unchanged: learn-errors still
learns from R's partition. Evidence for each choice is on #291.
For PacBio HiFi reads trimmed of both primers, use the indel-aware λ with
--end-model global --gap-open-p -1 --indel-end-zone 0
(#309):
- Near-end indels. On spike-ins placed 3–25 bases from either end it
recovered 176 of 192 on Zymo and 128 on BEI, against 167 and 119 for
ends-free, with no false splits.ends-freemisses indels in the first few 5′ bases, which a free leading gap absorbs; the default 3′ end zone misses most indels within 8 bases of the 3′ end. - Truncated reads. With linear gaps,
globalsplits reads truncated at one end: a long end gap fragments into interior gaps that λ then scores. The opening cost keeps it one gap, and the read is absorbed. - Full run. On 95 PacBio samples (
dada-pooled), truncated reads are absorbed, insertion variants such as 16S intervening sequences are kept, and the ASVs matchends-freeexcept for a few low-abundance ones. Pooled denoising takes about 1.5× as long asends-free.
--end-model ends-free --indel-end-zone 0 remains the conservative choice. Both
are experimental.
--end-model (default ends-free) — which alignment end gaps are free
(#289). An end gap is a run of
gaps at the start or end of an alignment, where one sequence overhangs the other.
A free end gap costs nothing; a penalised one costs --gap-p per column, like
an interior gap.
| value | leading (5') gaps | trailing (3') gaps | use for |
|---|---|---|---|
ends-free |
free | free | anything; R DADA2's model |
free-3p |
penalised | free | reads that start right after a trimmed primer and end at an arbitrary point |
global |
penalised | penalised | reads trimmed of both primers, so both ends sit at fixed biological positions |
ends-free and free-3p differ only at the 5' end. ends-free lets one
read start later than the other at no cost. free-3p assumes primer
removal has put every read's first base at the same position, so a leading gap
must be a real indel and is charged as one.
Why not global for Illumina or other paired-end data. Each read is
anchored only at its own primer. Its 3' end is set by the number of sequencing
cycles or by --trunc-len, not by the amplicon, so the last base of two reads
falls at different biological positions whenever an indel sits between them. A
trailing overhang there is expected, and global would charge it as gaps or
force mismatches near the 3' end, which then count as errors in the error model.
Forward and reverse reads are denoised separately, before merging, so the same
applies to both. Use free-3p for these reads. global fits only when
every read spans the whole amplicon and both primers are removed: full-length
PacBio 16S, or short amplicons whose reads all run through to the reverse
primer. Amplicons of variable length, such as ITS2, mix reads that run through
with reads that do not, so their 3' ends are not consistently anchored; use
free-3p there.
Ragged 5' ends. A read that keeps part of a primer, or loses a few bases
past it, is charged a leading gap under free-3p and global that ends-free
leaves free. This does not change the ASVs: a read's error probability (λ) is
computed from substitutions only, as in R, so a read that differs from its
centre only by a leading gap is absorbed the same way under every model. On the
MSA-1003 mock, giving 1–20% of reads a ragged start (±1–3 bases), or leaving
the full forward primer on 1–5% of them, gave the same ASVs under all three
models (#289).
Both backends honour the setting; chimera detection keeps R's model. The value
is recorded in the error model, and dada warns when its own differs. Against
truth (MiSeq SOP Mock, PacBio MSA-1003) neither opt-in model changes ASV calls;
free-3p gives a byte-identical table on the MiSeq SOP. global on truncated
Illumina reads raises reverse-read error rates by up to 24%, which is the 3'
mismatch effect described above (#289).
--gap-open-p (default 0) — affine gaps: a gap of length L costs
--gap-open-p + L × --gap-p. 0 is R's linear model and leaves every
output unchanged; a positive value means the same penalty, as for --gap-p.
With linear gaps, one long gap and the same number of scattered gaps cost the same, so a read truncated at one end, or an inserted block such as a 16S intervening sequence, can align as dozens of scattered gaps that hide its substitutions or, with the indel-aware λ, count as dozens of indels (#309). An opening cost makes the single block the best alignment. Keep it small:
--end-model global |
learned substitutions vs ends-free: Zymo / BEI / PacBio 95 |
reads truncated at one end, with --indel-rates |
|---|---|---|
--gap-open-p 0 |
+0.1% / +0.05% / −0.15% | split off as ASVs |
--gap-open-p -1 |
+0.5% / +0.2% / +0.9% | absorbed |
--gap-open-p -8 |
+11.5% / +4.7% / +13% | absorbed |
A large opening cost re-describes nearby insertion–deletion pairs as
substitutions, which the error model then learns. On the near-end spike-ins
above, global with --gap-open-p -1 and --indel-end-zone 0 recovered 176
of 192 on Zymo and 128 on BEI, against 167 and 119 for ends-free, with no
false splits.
It works with every end model and both backends, and cannot be combined with a
--homo-gap-p different from --gap-p. The value is recorded in the error
model and the output params, and dada warns when its own differs. Each
affine alignment costs about 1.3× a linear one (learn-errors on the PacBio mocks
took 1.1–1.2× as long).