garbage collect zero-value UTXOs #1832

darioAnongba · 2025-10-01T15:04:36Z

Garbage collect the residue zero-value UTXOs when creating transactions. Zero-value UTXOs occur when creating tombstones or full burns.
Currently, these UTXOs accumulate in the DB and are never cleaned. This PR introduces a garbage collection mechanism to collect these UTXOs and use them as inputs of transactions initiated by tapd:

Transfers
Burns

The PR adds a new swept flag to the managed_utxo table because UTXOs are not removed from the table when spent. This flag is also returned by the ListUtxos RPC endpoint.

The mechanism preserves the liveness and safety properties, ensuring that zero-value UTXOs can never accumulate in the DB. Adding garbage collection to Mint transactions is not necessary to ensure these properties.

Fixes #514

Note to reviewers

UTXO meaning unspent, keeping the spent outputs in a table called manage_utxos seems like a contradiction, same logic applies for the need of a swept flag in that table. We should either rename the table or store spent utxos somewhere else?
The pkScript is retrieved by querying the chain_txs table. Is it possible/more efficient to avoid this and somehow recompute it from the data we have?

coveralls · 2025-10-01T15:30:55Z

Pull Request Test Coverage Report for Build 18382547235

Details

21 of 354 (5.93%) changed or added relevant lines in 10 files are covered.
28020 unchanged lines in 195 files lost coverage.
Overall coverage decreased (-15.2%) to 34.469%

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
tapfreighter/parcel.go	0	4	0.0%
tapdb/sqlc/assets.sql.go	0	6	0.0%
rpcserver.go	0	11	0.0%
tapfreighter/coin_select.go	0	29	0.0%
tapfreighter/chain_porter.go	0	40	0.0%
tapfreighter/wallet.go	0	70	0.0%
tapdb/assets_store.go	3	176	1.7%

Files with Coverage Reduction	New Missed Lines	%
universe/supplyverifier/util.go	1	98.75%
proof/util.go	2	81.63%
tapdb/migrations.go	2	76.19%
address/log.go	3	0.0%
commitment/log.go	3	0.0%
internal/pedersen/commitment.go	3	95.31%
lndservices/log.go	3	0.0%
rfq/log.go	3	0.0%
tapchannel/log.go	3	0.0%
tapdb/supply_syncer.go	3	73.85%

Totals
Change from base Build 18359865357:	-15.2%
Covered Lines:	30524
Relevant Lines:	88554

💛 - Coveralls

ffranr

Commit message is incomplete for the commit with subject tapdb: add support for zero-value UTXO sweeping.

tapdb/assets_store.go

tapfreighter/wallet.go

ffranr · 2025-10-14T17:49:56Z

rpcserver.go

+	// Fetch zero-value UTXOs that should be swept as additional inputs.
+	zeroValueInputs, err := r.cfg.AssetStore.FetchZeroValueAnchorUTXOs(ctx)
+	if err != nil {
+		return nil, fmt.Errorf("unable to fetch zero-value "+
+			"UTXOs: %w", err)
+	}
+


this is probably fine as the default behaviour for now, even though it does have privacy implications. Maybe we should add a flag to AnchorVirtualPsbtsRequest to skip spending zero value inputs? default to: spend zero value

I wasn't sure what to do with this endpoint either. Adding a flag bothers me because zero value utxos are an internal side effect of the protocol and the notion should never be exposed externally so I added the same behavior here as we would in burning or transfers.
Exposing the zero value utxos notion externally implies adding descriptions to the API, adding to the CLI and asking users to be aware of what they are.
Let's see what @GeorgeTsagk says.

zero value utxos are an internal side effect of the protocol

The task of cleaning up UTXOs, regardless of why they exist (whether as an internal side effect of the protocol or not), is separate from anchoring a virtual PSBT. It’s distinct from the primary function of the endpoint. The current change opportunistically extends the endpoint’s core behavior by adding cleanup logic, which introduces privacy implications.

I think we should let the user choose whether to at least skip that cleanup so that the expected (and previous) behaviour is at least still available.

Thinking about this now. I'm not even sure if we should sweep zero value UTXOs in AnchorVirtualPsbts at all. Maybe it should be a completely distinct cleanup focused endpoint. Maybe for now we can just make this opt-in.

The current change opportunistically extends the endpoint’s core behavior by adding cleanup logic

I do agree with this and personally hate unintended side effects. I agree we should remove this. Only issue is if some user is always using low-level endpoints to create transactions, then he will never be able to sweep I think. A dedicated endpoint might make sense or a way to get the UTXOs when querying ListUtxos.

Keeping the cleanup integrated with other wallet actions feels more natural. It is also more cost-efficient as we naturally batch on-the-go.

I think we should definitely wrap all these behind a config flag. Whether it should be on/off by default I'm not sure about, we can start with off-by-default and maybe enable in a future release.

wrap all these behind a config flag.

This I believe should be a daemon-level flag and should extend all the way to tapfreighter/tapsend

The issue I see here is we're starting to diverge from the basic need that is avoiding "garbage" to grow indefinitely.

Zero-value UTXOs garbage collection is part of the realm of coin selection. Most Bitcoin wallets perform coin selection on behalf of the user and only few advanced wallets allow selecting and freezing specific UTXOs (like Sparrow). This is even more important for tapd where the coin selection is tied to the assets.

We initially started with a feature that naturally preserved liveness and safety properties and abstracted from the user the entire notion of zero-value UTXOs, which is a notion internal to the protocol, but now we are:

Worrying about privacy when the user calls AnchorVirtualPsbts if we select zero value UTXOs the same way we do for burn or send.

Adding a global config that defines whether or not zero-value UTXOs are sweeped, which implies that the set of zero-value UTXOs can now grow outside the limits of a transaction, breaking liveness and safety so we need to verify the size of the tx not to exceed the weight limit and maybe other side effects.

Taproot assets inherits from the same coin selection behavior as Bitcoin. If you have a small change UTXO lying around, a simple thing to do is consolidate it when sending a transaction. but the most efficient thing to do can be quite complex. Unless we want to implement advanced coin selection, I say we:

do not add a global flag

do not add a flag to the requests

always sweep in send and burn

never expose the notion of zero-value UTXOs in the API.

think about coin selection strategies (optimize fees, minimize utxos, consolidate, etc).

docs/release-notes/release-notes-0.7.0.md

ffranr · 2025-10-14T17:54:02Z

itest/burn_test.go

-	AssertBalances(
-		t.t, t.tapd, burnAmt+simpleCollectible.Amount+multiBurnAmt,
-		WithNumUtxos(3), WithNumAnchorUtxos(3),
-		WithScriptKeyType(asset.ScriptKeyBurn),
-	)
-


why remove this assertion? (and below)

This assertion is not relevant anymore for burns because it sums the amounts of the assets in the UTXOs it finds. Here it's used to verify the amounts burnt but you can't do that anymore unless you keep track of the swept UTXOs which is not the scope of this test.
As an e2e test, you only care that after you burn, the balance of the asset is correctly decreased.

another reason to wrap the sweeping behind a flag is to make sure we preserve old behavior and not alter test coverage

this way we'd disable sweeping for tests like this one and not have to delete the assertion

itest/zero_value_anchor_test.go

ffranr · 2025-10-15T13:33:57Z

Please rebase on to the latest in [0-8-0-staging](https://github.com/lightninglabs/taproot-assets/tree/0-8-0-staging). Might get you more PSBT funding test coverage now that #1418 is merged.

GeorgeTsagk

awesome work!

GeorgeTsagk · 2025-10-21T16:34:19Z

tapdb/sqlc/migrations/000048_add_swept_flag.up.sql

@@ -0,0 +1,2 @@
+-- Add swept flag to managed_utxos table to track when UTXOs have been swept
+ALTER TABLE managed_utxos ADD COLUMN swept BOOLEAN NOT NULL DEFAULT FALSE;


shouldn't we also add a golang-level migration step to automatically detect which of our managed utxos are already spent?

Normally not because zero-value anchor UTXOs cannot have been swept by normal usage of tapd. We're not responsible if a user happened to sweep these UTXOs somehow manually and corrupted the state of his DB.

GeorgeTsagk · 2025-10-21T16:59:51Z

tapsend/allocation.go

+	// Bip32Derivation is the BIP32 derivation info for the internal key.
+	// This is used to preserve the key derivation through the allocation
+	// flow so that PSBTs can be properly signed.
+	Bip32Derivation []*psbt.Bip32Derivation


why is this needed now?

Before, we were not persisting the necessary information in the DB to be able to sign the input for zero value UTXOs. Laolu described the bug here: https://github.com/lightninglabs/taproot-assets/blob/main/docs/debug-handbook/psbt-finalization-failures.md#a-bip32-key-derivation-mismatch.

Specifically the KeyFamily or KeyIndex not preserved when storing keys.

GeorgeTsagk · 2025-10-21T17:03:22Z

rpcserver.go

+	// Fetch zero-value UTXOs that should be swept as additional inputs.
+	zeroValueInputs, err := r.cfg.AssetStore.FetchZeroValueAnchorUTXOs(ctx)
+	if err != nil {
+		return nil, fmt.Errorf("unable to fetch zero-value "+
+			"UTXOs: %w", err)
+	}
+


Keeping the cleanup integrated with other wallet actions feels more natural. It is also more cost-efficient as we naturally batch on-the-go.

I think we should definitely wrap all these behind a config flag. Whether it should be on/off by default I'm not sure about, we can start with off-by-default and maybe enable in a future release.

GeorgeTsagk · 2025-10-21T17:05:55Z

rpcserver.go

+	// Fetch zero-value UTXOs that should be swept as additional inputs.
+	zeroValueInputs, err := r.cfg.AssetStore.FetchZeroValueAnchorUTXOs(ctx)
+	if err != nil {
+		return nil, fmt.Errorf("unable to fetch zero-value "+
+			"UTXOs: %w", err)
+	}
+


wrap all these behind a config flag.

This I believe should be a daemon-level flag and should extend all the way to tapfreighter/tapsend

GeorgeTsagk · 2025-10-21T17:10:40Z

itest/burn_test.go

-	AssertBalances(
-		t.t, t.tapd, burnAmt+simpleCollectible.Amount+multiBurnAmt,
-		WithNumUtxos(3), WithNumAnchorUtxos(3),
-		WithScriptKeyType(asset.ScriptKeyBurn),
-	)
-


another reason to wrap the sweeping behind a flag is to make sure we preserve old behavior and not alter test coverage

this way we'd disable sweeping for tests like this one and not have to delete the assertion

itest/zero_value_anchor_test.go

GeorgeTsagk · 2025-10-21T17:17:42Z

itest/zero_value_anchor_test.go

+		t.t, t.tapd, 0, WithScriptKeyType(asset.ScriptKeyBurn),
+		WithNumUtxos(0), WithNumAnchorUtxos(0),
+	)
+}


I think the config flag that controls whether sweeping is activated will also help us add slightly better coverage here too.

We could accumulate multiple burn UTXOs / tombstones and then turn the flag on and see them all being swept.

agree but to test this we would need to start tapd with a given config, stop it in the middle of the test and restart it with a different config. Is this what you mean?

GeorgeTsagk · 2025-10-21T17:18:12Z

also needs a rebase

This includes the addition of a "swept" field in the "managed_utxos" table with the corresponding migration and the "MarkManagedUTXOAsSwept" function.

…value inputs

…ng family and index

darioAnongba self-assigned this Oct 1, 2025

levmi added this to Taproot-Assets Project Board Oct 2, 2025

github-project-automation bot moved this to 🆕 New in Taproot-Assets Project Board Oct 2, 2025

levmi moved this from 🆕 New to 🏗 In progress in Taproot-Assets Project Board Oct 2, 2025

darioAnongba force-pushed the feat/zero-value-utxo-selection branch from f3a3bff to f55b6c5 Compare October 3, 2025 16:21

darioAnongba changed the base branch from main to 0-8-0-staging October 6, 2025 09:43

darioAnongba force-pushed the feat/zero-value-utxo-selection branch 3 times, most recently from 3bd769d to 6fe11a5 Compare October 6, 2025 15:03

darioAnongba changed the base branch from 0-8-0-staging to main October 6, 2025 15:03

darioAnongba force-pushed the feat/zero-value-utxo-selection branch 10 times, most recently from a8f3ce4 to afbfebf Compare October 9, 2025 13:04

darioAnongba marked this pull request as ready for review October 9, 2025 13:09

darioAnongba force-pushed the feat/zero-value-utxo-selection branch 4 times, most recently from bf3e3ee to d812a8e Compare October 9, 2025 16:04

darioAnongba requested review from Roasbeef and ffranr October 9, 2025 16:25

darioAnongba force-pushed the feat/zero-value-utxo-selection branch from d812a8e to 0ceaf76 Compare October 9, 2025 16:32

darioAnongba moved this from 🏗 In progress to 👀 In review in Taproot-Assets Project Board Oct 9, 2025

darioAnongba changed the base branch from main to 0-8-0-staging October 13, 2025 10:47

darioAnongba force-pushed the feat/zero-value-utxo-selection branch 2 times, most recently from a0a71be to 1a7100d Compare October 13, 2025 11:04

darioAnongba requested a review from GeorgeTsagk October 13, 2025 11:42

ffranr reviewed Oct 14, 2025

View reviewed changes

darioAnongba force-pushed the feat/zero-value-utxo-selection branch 2 times, most recently from f4fb800 to 127946d Compare October 16, 2025 14:17

darioAnongba requested review from ffranr and removed request for Roasbeef October 16, 2025 14:18

GeorgeTsagk force-pushed the 0-8-0-staging branch from 965998d to 83a4116 Compare October 20, 2025 16:16

GeorgeTsagk reviewed Oct 21, 2025

View reviewed changes

darioAnongba added 6 commits October 21, 2025 21:58

tapdb: add support for zero-value utxo sweeping

1b52974

This includes the addition of a "swept" field in the "managed_utxos" table with the corresponding migration and the "MarkManagedUTXOAsSwept" function.

tapfreighter: add call to FetchZeroValueAnchorUTXOs to retrieve zero-…

bb78e82

…value inputs

tapsend: include BIP32 key derivation to allocation to preserve signi…

2ef2cbe

…ng family and index

taprpc+rpc: Apply garbage collection in AnchorVirtualPsbts

2964429

itest: add test for zero-value utxo garbage collection

f74b150

docs: add release notes

8120455

darioAnongba force-pushed the feat/zero-value-utxo-selection branch from 127946d to 8120455 Compare October 21, 2025 20:27

		@@ -0,0 +1,2 @@
		-- Add swept flag to managed_utxos table to track when UTXOs have been swept
		ALTER TABLE managed_utxos ADD COLUMN swept BOOLEAN NOT NULL DEFAULT FALSE;

garbage collect zero-value UTXOs #1832

Are you sure you want to change the base?

garbage collect zero-value UTXOs #1832

Conversation

darioAnongba commented Oct 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coveralls commented Oct 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull Request Test Coverage Report for Build 18382547235

Details

💛 - Coveralls

Uh oh!

ffranr left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

darioAnongba Oct 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ffranr commented Oct 15, 2025

Uh oh!

GeorgeTsagk left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

GeorgeTsagk commented Oct 21, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

darioAnongba commented Oct 1, 2025 •

edited

Loading

coveralls commented Oct 1, 2025 •

edited

Loading

darioAnongba Oct 21, 2025 •

edited

Loading