Expose transaction validation cost in mempool transaction JSON

[K-Singh]

Problem

Block assembly is bounded by two limits, maxBlockSize and maxBlockCost. Both are enforced by CandidateGenerator.correctLimits:

blockTxs.map(_._2).sum < maxBlockCost && blockTxs.map(_._1.size).sum < maxBlockSize

The mempool API returns size, so external tooling can enforce the size limit. It does not return cost, so the cost limit cannot be enforced outside the node.

Anyone assembling blocks externally (block template builders, mempool simulators, throughput analysis) has to re-run script validation against a UTXO set, or guess from size and input/output counts. The node already computed the exact value during mempool admission and discarded it at the serialization boundary.

Cost is also already load-bearing inside the pool: SortingOption.FeePerCycle orders by fee-per-cost via lastCost in ErgoMemPool.feeFactor. A node in that mode exposes an ordering clients cannot reproduce.

Change

Add a cost field to the mempool transaction JSON, sourced from UnconfirmedTransaction.lastCost.

{
  "id": "...",
  "inputs": [ ... ],
  "dataInputs": [],
  "outputs": [ ... ],
  "size": 344,
  "cost": 21456
}

Affected endpoints:

• GET /transactions/unconfirmed
• GET /transactions/unconfirmed/byTransactionId/{txId}
• POST /transactions/unconfirmed/byErgoTree

Null semantics

cost is null when the node never measured it:

1. Digest state. No script validation happens, so lastCost is never set.
2. Rollback. ErgoNodeViewHolder.updateMemPool returns transactions from disconnected blocks with UnconfirmedTransaction(tx, None) and puts them via put, which skips validateWithCost. They stay null until CleanupWorker re-validates them.

The key is always present on a node carrying this change, so clients can distinguish "node too old" (key absent) from "cost unknown" (key null).

Implementation notes

unconfirmedById added to ErgoMemPoolReader. The by-id endpoint used modifierById, which returns Option[ErgoTransaction] and drops the wrapper before the route sees it. The new method returns the UnconfirmedTransaction; modifierById is now defined in terms of it, so there is one lookup path. ErgoMemPool implements it as pool.get(id), identical to what modifierById did.

byErgoTree keeps the wrapper. Its two collect blocks mapped to tx.transaction, discarding the cost. They become filters. Deduplication across the output-match and input-match sets is unchanged, since UnconfirmedTransaction.equals and hashCode are defined on the transaction id.

Digest-state branch of byErgoTree now uses the shared JSON builder. It previously serialized a bare ErgoTransaction, which did not match the ErgoTransactionWithInputBoxes schema the spec declares for that response. Routing it through the same builder fixes that and means cost is never silently absent from a response that otherwise carries it.

Compatibility

Additive and optional. Not in the schema's required list. Clients that ignore unknown keys are unaffected. Confirmed-transaction schemas (ErgoTransaction, IndexedErgoTransaction, WalletTransaction) are untouched, since lastCost only exists for unconfirmed transactions.

Tests

Six cases added to TransactionApiRouteSpec, covering cost reported by id, in the list, via byErgoTree, agreement between the list and by-id endpoints, the null case, and that size still reports correctly alongside cost.

Run locally on v6.0.6:

• TransactionApiRouteSpec: 29 passed, 0 failed (23 pre-existing, 6 new)
• Mempool, mempool auditor, wallet service and all HTTP route suites: 191 passed across 19 suites, 0 failed

Known follow-up, not in this PR

Rollback-restored transactions report null until the next cleanup pass. Closing that would mean carrying per-transaction cost from block validation through updateMemPool, which is a larger change than exposing a value the pool already holds.

🤖 Generated with Claude Code

[jozanek]

Thanks for the description — the implementation notes made this quick to follow, and the equals/hashCode reasoning behind the collect → filter change holds up.

I read this as: publish a number the pool already holds, so tooling can reason about the mempool without re-running validation. On that reading the field belongs, the nullable-field shape looks right to me, and the blast radius is small — explorer-backend (circe) and ergo-appkit (Gson) both ignore unknown keys.

Where I got stuck is the stated why. Both justifications came apart against the code they cite:

• Block-budget enforcement. collectTxs never reads lastCost — it re-runs validateWithCost against its own upcomingContext (CandidateGenerator.scala:979-987), and budgets maxBlockCost - safeGap, with safeGap up to 500,000 (:680-696). correctLimits also counts the fee transaction the node synthesizes itself. A builder summing these against maxBlockCost overshoots, and collectTxs truncates silently rather than rejecting (:1000-1003).
• Reproducing FeePerCycle ordering. mempoolSorting = "random" is the shipped default (application.conf:88) — a per-process coin flip that no endpoint exposes. And OrderedTxPool.put discards the new feeFactor when the id is already present (:101-110), so a cleanup pass moves the reported cost but not the weight.

Neither sinks the field. They point at a different framing — mempool observability: fee-market analysis, simulation, "why isn't my transaction being mined" — which it serves well. Would you be open to re-justifying it that way?

Questions:

1. Is there a consumer that would sum these against a budget? If yes, I think the shape needs to change and /mining/candidateWithTxs is the better vehicle. If every consumer is descriptive, the field is right and the rest is wording.
2. #2534 describes this change but isn't linked — was that deliberate? It also records getAll(Seq(id)) as an alternative to touching the trait. I lean toward unconfirmedById being the better call, so I'd mainly like the reasoning captured where the next reader finds it.
3. POST /transactions/check already computes this exact number and discards it (ErgoBaseApiRoute.scala:136-141). Worth a line on why the pool-side view was the one you needed?
4. Base v6.0.6 matches convention, but the carry-forward PR #2473 is currently CONFLICTING — is there a plan for how this reaches 6.0.7?

I couldn't find any third-party request for this: #1854, the wishlist that produced these endpoints, doesn't mention cost. Not evidence against the change, but the constituency isn't visible to a reviewer.

What would move me to approve: corrected wording at the two description sites, a line preserving the required-list tri-state, and the null-cost test pointed at digestReadersRef.

[jozanek]

This says cost is measured when the transaction entered the mempool, but CleanupWorker.scala:90-92 overwrites it via head.withCost(txCost) on every re-validation pass for transactions older than mempoolCleanupDuration (30s default) — which is what UnconfirmedTransaction.scala:12 means by "during last check".

An integrator reading this will treat the number as pinned to admission and may cache it. Would you be open to something like "as of this node's most recent validation of this transaction, against the state at that time"? Since the re-check pass is itself budget-limited (CostLimit at CleanupWorker.scala:27), two honest nodes can also report different costs for the same transaction, which may be worth a clause.

[K-Singh]

Yea that is fair to change

[jozanek]

The tri-state this sets up — key absent means an older node, null means unmeasured, a number means measured — works because cost stays out of the required list above. That reads deliberate to me and I'd keep it exactly as is.

What I think is missing is anything in the file saying so. A later tidy-up that "completes" the required list would quietly remove the version detection the description relies on, and no test would catch it. Would you be open to a sentence in the description: always present on nodes carrying this field, absence indicates an older node?

[K-Singh]

That seems like a reasonable way to handle it.

[jozanek]

Same wording as the spec description — "measured when the transaction entered the pool". Given CleanupWorker refreshes it, the next reader of this method may take the value as fixed for the lifetime of the pool entry.

Would you mind matching whatever wording you settle on in openapi.yaml, so the two don't drift apart later?

[jozanek]

The description presents this as bringing the digest branch into line with the ErgoTransactionWithInputBoxes schema. Reading the schema, I don't think it arrives there: ErgoBoxWithSpendingProof is allOf[ErgoTransactionOutput, ...], and ErgoTransactionOutput requires value, ergoTree, additionalRegisters and creationHeight (openapi.yaml:110-116). With Map.empty the inputs are still {boxId, spendingProof}.

The change itself still looks right to me — it makes this branch consistent with the two sibling endpoints and keeps cost from going missing on one of them. Would you be open to describing it as a consistency fix rather than a conformance fix, so a later spec audit doesn't treat this as closed?

[K-Singh]

Sure that makes sense

[jozanek]

Both requests here read lastCost from the same in-process pool object, so I think this can only fail if one of the two tests above it has already failed.

No action needed if it's here to document the invariant — I mostly want to check I'm not missing a path where the list and by-id routes could diverge.

[K-Singh]

This was mostly to document the change

[jozanek]

The comment describes digest state and rollback, but this request goes to route, which line 42 wires to utxoReadersRef — and txs.head comes from the Stubs pool, built as UnconfirmedTransaction(tx, None) and never validated. So it asserts null on a fixture that was constructed null, rather than on a node that couldn't measure.

digestReadersRef already exists at Stubs.scala:362. Would you be open to pointing this one at it? The digest branch of byErgoTree — the behavior change the implementation notes single out — currently has no coverage at all, and that's the case most likely to regress.

[K-Singh]

Yes this makes sense, can fix it

[K-Singh]

Where I got stuck is the stated why. Both justifications came apart against the code they cite:

• Block-budget enforcement. collectTxs never reads lastCost — it re-runs validateWithCost against its own upcomingContext (CandidateGenerator.scala:979-987), and budgets maxBlockCost - safeGap, with safeGap up to 500,000 (:680-696). correctLimits also counts the fee transaction the node synthesizes itself. A builder summing these against maxBlockCost overshoots, and collectTxs truncates silently rather than rejecting (:1000-1003).
• Reproducing FeePerCycle ordering. mempoolSorting = "random" is the shipped default (application.conf:88) — a per-process coin flip that no endpoint exposes. And OrderedTxPool.put discards the new feeFactor when the id is already present (:101-110), so a cleanup pass moves the reported cost but not the weight.

Neither sinks the field. They point at a different framing — mempool observability: fee-market analysis, simulation, "why isn't my transaction being mined" — which it serves well. Would you be open to re-justifying it that way?

Questions:

1. Is there a consumer that would sum these against a budget? If yes, I think the shape needs to change and /mining/candidateWithTxs is the better vehicle. If every consumer is descriptive, the field is right and the rest is wording.

Yes, this will be used to build blocks according to cost budgets, but I am still quite confident that this is the best way to do it. A few reasons:

• We shouldn't add unnecessary info to candidateWithTxs, miners are in a race to mine the block as fast as possible, so we should deliver the required info alone in that call.
• You are correct, the cost given is the mempool cost, and the transactions actual cost is determined in the upcoming context. However in most cases this is acceptable. Miners will optimize MEV by looking at specific mempool transactions they want, and including them or superseding them where appropriate. In the large majority of cases (though you can correct me if I'm wrong) validation cost in the mempool should not significantly differ from costing in the upcomingContext. This does not hold for transactions whose cost is significantly effected by changes in the pre-header, but most DeFi protocols on Ergo do not fall into this category.
• The goal is not necessarily to build up to the max cost budget ourselves, in Lithos we are setting a limit (default 50% of the cost limit). So we fill up to our own budget, then let the node fill the rest of the block with whichever mempool transactions that were not explicitly included. However the safeGap clarification is useful to know, so thanks!

2. Expose transaction validation cost in mempool transaction JSON #2534 describes this change but isn't linked — was that deliberate? It also records getAll(Seq(id)) as an alternative to touching the trait. I lean toward unconfirmedById being the better call, so I'd mainly like the reasoning captured where the next reader finds it.

• Not deliberate, I forgot! But yes I think unconfirmedById is better than what was proposed there.

3. POST /transactions/check already computes this exact number and discards it (ErgoBaseApiRoute.scala:136-141). Worth a line on why the pool-side view was the one you needed?

This is true, but as miners are in a race to build the block, it is best for them to avoid making additional API calls which would require the transaction to be revalidated. As the cost is already computed while the transaction is in the mempool, it is most efficient to just return it as part of the mempool call.

4. Base v6.0.6 matches convention, but the carry-forward PR Candidate for 6.0.6 release #2473 is currently CONFLICTING — is there a plan for how this reaches 6.0.7?

I am mostly focused on getting this ready for 6.0.6 due to the upcoming Lithos launch, as we need both the merkle proof fix in #2463 for proper Lithos functionality and this fix for #2547 to allow proper MEV. But I can look into what is needed for 6.0.7