Skip to content

Latest commit

 

History

History
136 lines (99 loc) · 5.93 KB

dada-pooled.md

File metadata and controls

136 lines (99 loc) · 5.93 KB

dada-pooled

Denoise multiple samples with full pooling — R DADA2's dada(..., pool = TRUE).

dada2-rs dada-pooled derep/*.json.gz \
  --error-model err.json -o dada/ --threads 24 --gzip

Per-sample uniques are merged into one combined table (abundances summed, qualities abundance-weighted-averaged), DADA2 runs once on the merged table, and one JSON file per sample is written to the output directory containing only the ASVs present in that sample.

Pooling is the most sensitive mode for rare variants, and the most expensive. Peak memory is driven by the merged unique table rather than any single sample.

Input

<INPUT>... — FASTQ or derep/sample JSON files, one per sample. Their order matters, as it does in R, under the default --pool-tiebreak first-seen (see below). Pass an explicit, byte-sorted list (e.g. LC_ALL=C ls ...) rather than a shell glob, whose order follows the locale, and keep the list with the results; each output records its sample's pool_input_index.

--sample-names — comma-separated, one per input. Defaults to filename stems.

--phred-offset — 33 for Sanger / Illumina 1.8+, 64 for Illumina 1.3–1.7.

--prior — FASTA of prior sequences; exact matches against the merged table are exempt from the abundance p-value filter and split via --omega-p.

Error model

--error-model (required), --use-err-in, --inherit-err-params — identical in meaning to dada's error-model flags.

Denoising, Alignment, Screening

Identical in meaning to dada: see Denoising, Alignment and Screening. One flag is specific to pooling:

--pool-tiebreak (default first-seen) — how equal-abundance uniques are ordered in the merged table. Order matters where a birth saturates at pA = 0 and position decides which of two tied variants names an ASV.

value ties ordered by
first-seen first appearance across the inputs, in the order given, as R's combineDereps2
lexical sequence, so results do not depend on input order; the default before #260

first-seen reproduces R: on the #260 comparison it gave R's pooled ASV set exactly on MiSeq and NovaSeq ITS2 (both reads), where lexical differed by up to 12 ASVs, and narrowed PacBio HiFi from 17 to 10. Choose lexical when independence from input order matters more than R parity. The differences are renames within equal-abundance pairs. Each output records the value in params.pool_tiebreak.

--kmer-size deserves particular attention here. Pooling produces a large, diverse unique table, which is exactly the regime where the k-mer screen runs on everything — so on PacBio HiFi, leaving it at 5 is costly. See K-mer screen size.

Performance

--threads (default 1) — threads for dereplication and the DADA2 comparison map. Pooled runs scale to high thread counts, but the shuffle phase has a serial floor that grows as a share of runtime as threads rise; see Thread scaling & memory placement.

Output

--output-dir / -o (required) — directory for per-sample {sample}.json, created if absent.

--compact — minified JSON.

--gzip — write {sample}.json.gz.

ASV records carry lineage fields; here cluster ids are pool-wide, the same in every sample's file.

Diagnostics

--verbose — the run's shape and results to stderr: CPU allocation, alignment backend, active tuning gates, resident footprint, the index decision, a phase-times one-liner, and per-cluster progress. The phase attribution and optimisation counters are not printed; pass --metrics-json for those.

--failed-uniques — TSV of uniques that failed to denoise. Because pooled denoising runs once on the merged unique table, "failed" is a global property (map == null on the merged index); for each failed merged unique a row is emitted per sample it appears in, carrying that sample's read count.

--indel-events, --indel-opportunities (experimental) — the indel tables described under dada, from the pooled partition. cluster, unique (merged index), correct, p, lambda and cluster_reads are global; each event is emitted per sample its merged unique appears in, with that sample's reads. The rate tables are pool-level (sample is pooled), weighted by pooled reads as dada weighs them.

--pooled-record — write a self-contained pooled record (merged uniques with pooled abundance, the global map, global ASVs, and the inputs in pooled order) for kdist-calibrate --from-dada-pooled. Off by default. Give a path outside --output-dir so it does not join the per-sample *.json.gz glob; gzip follows the path's .gz extension.

--metrics-json — write a structured record of the run's phase times, compare attribution, shuffle split, footprint and index decision to this file. Free to leave on: it collects only what the run already pays for.

--metrics-attribution — add the per-comparison timings (busy, map parallel efficiency, the screen / DP / al2subs split). This slows the run, so do not combine it with a timing measurement. Requires --metrics-json.

See Run metrics for the schema and the two levels.

--cluster-trace, --trace-no-members, --trace-min-abund — as in dada.

Experimental

Identical in meaning to dada's experimental flags. Note that on pooled PacBio the minimizer index choice reversed relative to Illumina, which is why the backend now probes rather than predicts; see Minimizers as the screen.

See also