PoC: Add metrics to TXM by GabrielMartinezRodriguez · Pull Request #503 · HappyChainDevs/happychain

GabrielMartinezRodriguez · 2025-03-07T14:21:41Z

Linked Issues

Added OpenTelemetry instrumentation to monitor txm

Implemented block monitoring metrics for current block number and block delay
Added nonce management metrics for tracking nonce allocation and queue status
Created transaction status change monitoring
Added transaction collector metrics
Implemented repository metrics for non-finalized transactions
Set up Prometheus exporter on port 9090 for metrics collection

Toggle Checklist

Checklist

Basics

B1. I have applied the proper label & proper branch name (e.g. norswap/build-system-caching).
B2. This PR is not so big that it should be split & addresses only one concern.
B3. The PR targets the lowest branch it can (ideally master).

Correctness

C1. Builds and passes tests.
C2. The code is properly parameterized & compatible with different environments.
C3. I have manually tested my changes & connected features.
C4. I have performed a thorough self-review of my code.

Architecture & Documentation

D1. I made it easy to reason locally about the code.
D2. All public-facing APIs & meaningful internal APIs are properly documented.
D3. Architecture is documented appropriately.
D4. Appropriate Changeset has been generated for npm published packages.

cloudflare-workers-and-pages · 2025-03-07T14:21:43Z

Deploying happychain with Cloudflare Pages

Latest commit:	`1610388`
Status:	⚡️ Build in progress...

View logs

GabrielMartinezRodriguez · 2025-03-07T14:22:03Z

Randomness monitor service #584
Txm traces #521
fix(txm): returned nonce queue order #578
fix(txm): not process the same block multiple times #577
Fix: Viem sends random undefined blocks #575
fix(txm): heap out of memory #573
Avoid initiating new attempts if the gas conditions persist unchanged #571
Dynamic priority fee #570
TXM: Rpc liveness #562
fix(txm): nonce gap #561
PoC: Add metrics to TXM #503 👈 (View in Graphite)
master

This stack of pull requests is managed by Graphite. Learn more about stacking.

packages/txm/lib/NonceManager.ts

norswap · 2025-03-14T02:07:12Z

packages/txm/lib/telemetry/metrics.ts

+    description: "Count of transactions transitioning to a different status",
+    unit: "count",
+    valueType: ValueType.INT,
+})


Is this one useful?

Yes, because then we can track how many transactions we have in any state, as you can filter by the specific status. You can filter by status because when I add the counter, I specify the status:

transactionStatusChangeCounter.add(1, { status: this.status, });

This actually generates a global counter for all statuses and one for each specific status

norswap · 2025-03-14T02:25:17Z

packages/txm/lib/telemetry/metrics.ts

+    unit: "count",
+    valueType: ValueType.INT,
+})
+


Seems to be missing more transaction metrics (processing, reverted, cancelled, successfull, and I would add failed in multiple buckets (RPC, timeout, simulation failure, ...), cf. notion doc)

Yes, I agree with you. This was just to show what I think is a good way to implement metrics, but yes, we need to add more. I'm going to do it

norswap · 2025-03-14T02:26:54Z

packages/txm/lib/telemetry/metrics.ts

+    description: "Number of transactions collected",
+    unit: "count",
+    valueType: ValueType.INT,
+})


Numba go up type of stuff. We can keep it, it's fun.

We might want a metric for the rate of growth over time? Like bucket into a time period (1 min? 10 min?) and see how many new txs arrive in that time and chart that.

That was my intention, but to achieve that kind of metric, the right way to do it is to have a counter and then visualize the rate of increase of this counter over a period of time in the Grafana panel. This is the correct approach because it is more versatile and easier to implement

norswap · 2025-03-14T02:29:30Z

This is good, probably add a few more metrics (esp. transaction breakdown) and then we can merge this initial batch.

not-reed · 2025-03-31T14:16:21Z

support/common/lib/utils/safeViemClients.ts

+            // biome-ignore format: tidy
+            console.warn(
+                "No signTransaction method found on the account, using signMessage instead. " +
+                "A viem update probably change the internal signing API.");


interesting 🧐

not-reed · 2025-03-31T14:20:14Z

support/common/lib/utils/safeViemClients.ts

+
+        return ResultAsync.fromThrowable(() => {
+            if (client.account.signTransaction) {
+                return client.account.signTransaction({


I think client.signTransaction should always be the correct approach. Some account types like JSON-RPC are essentially just a string as they defer signing elsewhere. client.signTransaction takes into account what type of account is configured and signs accordingly

(i think)

That was made a couple of months ago. But I remember that we tried to use client.account.signTransaction because using that method allowed us to avoid calling getChainId. I’m not exactly sure why, but when using client.signTransaction directly—even if you specify the chainId—it still makes a call to the blockchain

That's what I remember too. We should a comment there to explain why we're doing this though!

norswap · 2025-04-04T12:44:08Z

Gave this a cursory look, good to merge from my side imho.
Just left a little comment to document our use of the internal signing function.

GabrielMartinezRodriguez changed the title ~~Add OpenTelemetry metrics to TXM components~~ PoC: Add metrics to TXM Mar 7, 2025

GabrielMartinezRodriguez self-assigned this Mar 7, 2025

GabrielMartinezRodriguez added the no-merge For showcase, not to be merged label Mar 7, 2025

aodhgan reviewed Mar 11, 2025

View reviewed changes

packages/txm/lib/NonceManager.ts Outdated Show resolved Hide resolved

GabrielMartinezRodriguez force-pushed the gabriel/txm-telemetry branch from f6a00d3 to e7703ac Compare March 13, 2025 13:53

GabrielMartinezRodriguez mentioned this pull request Mar 13, 2025

Txm traces #521

Merged

11 tasks

norswap reviewed Mar 14, 2025

View reviewed changes

GabrielMartinezRodriguez force-pushed the gabriel/txm-telemetry branch from 7b586dc to ee3e75d Compare March 17, 2025 14:32

GabrielMartinezRodriguez added reviewing-2 Ready for, or undergoing final review and removed no-merge For showcase, not to be merged labels Mar 17, 2025

GabrielMartinezRodriguez requested review from aodhgan and norswap March 17, 2025 15:25

GabrielMartinezRodriguez marked this pull request as ready for review March 17, 2025 15:25

GabrielMartinezRodriguez force-pushed the gabriel/txm-telemetry branch 2 times, most recently from cf5bf43 to 96a7504 Compare March 18, 2025 14:17

This was referenced Mar 26, 2025

fix(txm): nonce gap #561

Merged

TXM: Rpc liveness #562

Merged

GabrielMartinezRodriguez force-pushed the gabriel/txm-telemetry branch from 1727040 to 4303d66 Compare March 27, 2025 10:24

This was referenced Mar 27, 2025

Dynamic priority fee #570

Merged

Avoid initiating new attempts if the gas conditions persist unchanged #571

Merged

fix(txm): heap out of memory #573

Merged

GabrielMartinezRodriguez force-pushed the gabriel/txm-telemetry branch from 4365362 to e783a93 Compare March 31, 2025 09:54

GabrielMartinezRodriguez mentioned this pull request Mar 31, 2025

Fix: Viem sends random undefined blocks #575

Merged

11 tasks

not-reed approved these changes Mar 31, 2025

View reviewed changes

GabrielMartinezRodriguez mentioned this pull request Apr 1, 2025

fix(txm): not process the same block multiple times #577

Merged

11 tasks

GabrielMartinezRodriguez mentioned this pull request Apr 1, 2025

fix(txm): returned nonce queue order #578

Merged

11 tasks

aodhgan approved these changes Apr 2, 2025

View reviewed changes

GabrielMartinezRodriguez mentioned this pull request Apr 3, 2025

Randomness monitor service #584

Merged

11 tasks

norswap added merge-after-changes Can be merged after requested changes are made and removed reviewing-2 Ready for, or undergoing final review labels Apr 4, 2025

GabrielMartinezRodriguez added 12 commits April 7, 2025 12:42

feat(txm): implemented metrics

346531c

chore: format

b2c57ab

chore(txm): move metrics to specific file

acfe772

chore(txm): include instrumentation in tests

7fcfcca

feat(txm): added metrics

5ee1aad

feat(txm): instrumentation is configurable

eed296e

fix(txm): test

2af7bf7

fix: bun.lock

d7f2e29

feat(txm): os metrics

6f9a21c

chore(txm): remove metrics on tests

b86fe5e

fix(txm): fixed attemptsUntilFinalization metric

daad565

chore(txm): use safeGetTransactionCount

16ff508

GabrielMartinezRodriguez force-pushed the gabriel/txm-telemetry branch from f9da5c8 to 16ff508 Compare April 7, 2025 11:42

chore: format

1610388

GabrielMartinezRodriguez merged commit b08d67b into master Apr 7, 2025
3 of 4 checks passed

GabrielMartinezRodriguez deleted the gabriel/txm-telemetry branch April 7, 2025 11:58

This was referenced Apr 8, 2025

feat(randomness): submit drands asap #590

Merged

Txm transactions support arbitrary calldata #596

Merged

This was referenced Apr 21, 2025

feat(txm): transactions with value #647

Merged

Faucet service & Iframe integration #661

Merged

feat: deploy faucet #668

Merged

feat(randomness): drand prune #693

Merged

Conversation

GabrielMartinezRodriguez commented Mar 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Linked Issues

Checklist

Basics

Correctness

Architecture & Documentation

Uh oh!

cloudflare-workers-and-pages bot commented Mar 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Deploying happychain with Cloudflare Pages

Uh oh!

GabrielMartinezRodriguez commented Mar 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

norswap Mar 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

norswap commented Mar 14, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

norswap commented Apr 4, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

GabrielMartinezRodriguez commented Mar 7, 2025 •

edited

Loading

cloudflare-workers-and-pages bot commented Mar 7, 2025 •

edited

Loading

GabrielMartinezRodriguez commented Mar 7, 2025 •

edited

Loading

norswap Mar 14, 2025 •

edited

Loading