LSPS5 implementation #3662

martinsaposnic · 2025-03-11T20:12:16Z

A complete implementation for LSPS5 (spec defined here lightning/blips#55)

Reviewing commit by commit is recommended (~40% of the added lines are tests)

Notes:

Persistence of the state will come on a future PR
Will rebase from lightning-liquidity: Introduce EventQueue notifier and wake BP for message processing #3509 once that PR is merged

ldk-reviews-bot · 2025-03-11T20:12:19Z

👋 Thanks for assigning @tnull as a reviewer!
I'll wait for their review and will help manage the review process.
Once they submit their review, I'll check if a second reviewer would be helpful.

martinsaposnic · 2025-03-11T20:16:28Z

This is a huge PR, but it wasn’t obvious to me how to split it in a way that would still make sense (I did split it into small commits to make it easier to review.). I’m open to suggestions if you have ideas on how this could be structured differently.

codecov · 2025-03-11T20:21:35Z

Codecov Report

Attention: Patch coverage is 86.55804% with 264 lines in your changes missing coverage. Please review.

Project coverage is 89.16%. Comparing base (830ffa0) to head (b70e9fb).
Report is 4 commits behind head on main.

Files with missing lines	Patch %	Lines
lightning-liquidity/src/lsps5/client.rs	81.89%	86 Missing and 17 partials ⚠️
lightning-liquidity/src/lsps5/service.rs	85.33%	52 Missing and 19 partials ⚠️
lightning-liquidity/src/lsps5/msgs.rs	88.52%	35 Missing and 10 partials ⚠️
lightning-liquidity/src/lsps0/ser.rs	73.78%	9 Missing and 18 partials ⚠️
lightning-liquidity/src/manager.rs	87.05%	6 Missing and 5 partials ⚠️
lightning-liquidity/src/lsps5/url_utils.rs	98.14%	5 Missing and 1 partial ⚠️
lightning-liquidity/src/lsps0/msgs.rs	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3662      +/-   ##
==========================================
- Coverage   89.26%   89.16%   -0.11%     
==========================================
  Files         155      159       +4     
  Lines      119968   121932    +1964     
  Branches   119968   121932    +1964     
==========================================
+ Hits       107092   108719    +1627     
- Misses      10265    10515     +250     
- Partials     2611     2698      +87

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

tnull

Wow, thank you for looking into this! I did a first pass, and it looks pretty amazing already!

Before going too much into further details, here are a few general comments upfront:

I'm generally no fan of introducing additional dependencies here, an in particular not reqwest and tokio. I think following the pattern so far BroadcastNotifications could be a request that the user handles with any HTTP client they want and then could call back into LSPS5ServiceHandler. Alternatively, we could also use a trait similar to the current HTTPClient, but I don't think we want to keep the default implementation. Note that the blocking reqwest variant wraps a tokio runtime internally, and therefore should never (1, 2, ...) be used together. I guess technically we could consider a default async version of the trait that uses async reqwest, but I would prefer to simply have well-documented trait on our end that the user can implement however they choose to. Also note that stacking tokio runtimes is heavily discouraged in general, so assuming our users would themselves use a tokio runtime, we shouldn't wrap one in LSPS5ServiceHandler.
Note that lightning-liquidity is optionally no-std compliant, so please don't rely on std wherever possible, often it's just a matter of using core instead and importing the respective types from crate::prelude. If you really find yourselves needing to use std, make sure it's feature gated behind feature = "std" and we provide an alternative for users that don't support it.
Minor: Regarding formatting we're using tabs, not spaces. Feel free to run ./contrib/run-rustfmt.sh after each commit to run our formatting scripts.
This PR in its current scope is great, just want to note that eventually we need to add persistence for the state. As we haven't fully fleshed out the persistence strategy for lightning-liquidity in general yet, it's actually preferred to defer this to a follow-up, but just wanted to mention it. Also note that some changes to MessageQueue/EventQueue will happen in lightning-liquidity: Introduce EventQueue notifier and wake BP for message processing #3509, but will ping you to rebase once that has been merged. (just sidenotes here).

I hope these initial points make sense, let me know if you have any questions, or once you made corresponding changes and think this is ready for the next round of review!

ldk-reviews-bot · 2025-03-13T12:45:04Z

👋 The first review has been submitted!

Do you think this PR is ready for a second reviewer? If so, click here to assign a second reviewer.

tnull · 2025-03-13T12:48:30Z

This is a huge PR, but it wasn’t obvious to me how to split it in a way that would still make sense (I did split it into small commits to make it easier to review.). I’m open to suggestions if you have ideas on how this could be structured differently.

Thanks for asking! I'm totally fine to keep this (with its current scope) in a single PR, as long as we keep the commit history pretty clean to allow continuing review to happen commit-by-commit. To this end, please make sure add any fixup commits clearly marked (e.g. via a f or fixup prefix in the commit message header) directly under the commit they belong to, so they can cleanly be squashed into the respective feature commits in-between review rounds.

tnull · 2025-03-13T12:54:53Z

Btw, I'm not sure if you're familiar with the previous attempt of implementing LSPS5: #3499

Given this is a clean slate, not sure how much there is to learn, but still might be worth a look. Also not sure if @johncantrell97 would be interested in reviewing this PR, too, as he's familiar with the codebase and LSPS5.

martinsaposnic · 2025-03-14T17:11:07Z

@tnull, ready for the next review round!

Removed reqwest and tokio.
HttpClient is now a generic trait, allowing users to pass any implementation.
Removed std.
Small refactor on client and service to be able to delete some silly / unnecessary code.
Ran formatting on every commit.
All new changes are in commits prefixed with fixup:.

CI is failing because of the usage of the url crate (which I guess does not support rust 1.63?), which I need for validating the webhook URL. A new commit will come addressing that shortly

ldk-reviews-bot · 2025-03-17T00:00:52Z

🔔 1st Reminder

Hey @tnull @valentinewallace! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2025-03-17T00:00:54Z

🔔 1st Reminder

Hey @tnull @valentinewallace! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

tnull · 2025-03-18T12:41:03Z

FYI I fixed the Rust 1.63.0 build failures by downgrading the url crate to 1.7.2 in this commit.

I'm not thrilled about using such an old version (5+ years). It works, but maybe not ideal.

A few other options:
* I could write a small URL validator that just handles what we need for the webhook validation. It can bring other bugs / issues but eliminates the dependency headache.

* Use feature flags in Cargo.toml

* Bump the MSRV for lightning-liquidity
I can implement any of these alternatives if you think it's worth it. Let me know what you think!

Yes, let's please also drop the url dependency and just write the few checks we need to run on the URLs ourselves. I'd rather avoid introducing feature flags and have LSPS5 available to all users. Bumping MSRV is really no option currently, as we're looking to expose lightning-liquidity in our language bindings, which currently relies on said 1.63 MSRV.

martinsaposnic · 2025-03-24T20:30:30Z

Hey @tnull, thank you so much for reviewing and answering my questions!

I just pushed some updates, addressing all your comments, including:

All new crates added before are now deleted
Correct notification signature
URL validation is now handled by us (not an external crate)
Generic time trait for user-provided implementations (with a default std implementation)
Added tests with spec examples and test vectors (more tests are coming, so we are sure the PR is following the spec byte-for-byte)

This should be ready for the next round of review.

Thanks again!

tnull

Thanks for the changes so far, this is def. quickly moving in the right direction!

Added a bunch of more high-level comments, will see to get more into the details in the next round. Feel free to squash current fixups!

lightning-liquidity/src/lsps5/msgs.rs

lightning-liquidity/src/lsps5/url_utils.rs

lightning-liquidity/src/lsps5/msgs.rs

lightning-liquidity/src/lsps5/client.rs

lightning-liquidity/src/lsps5/mod.rs

- Add 'time' feature flag to allow disabling time-dependent functionality - Include 'time' in default features - Allow users to disable SystemTime::now without disabling all std features - Follows pattern established in other crates (e.g., lightning-transaction-sync) - Improves compatibility with WASM environments

Adds a new url_utils.rs module that provides: - A lightweight URL parser specialized for LSPS5 webhook validation - A URL trait and Url implementation focusing on scheme and host extraction - RFC-compliant scheme validation - Comprehensive test coverage for various URL scenarios This implementation allows validating webhook URLs without depending on the external url crate

- Remove unnecessary URL trait (no other implementations expected) - Rename Url to LSPSUrl to avoid namespace conflicts - Clean up redundant comments

- Define LSPS5Request and LSPS5Response enums for webhook registration, listing, and removal. - Implement WebhookNotification and associated helper constructors for different notification types. - Implement serialization/deserialization support with comprehensive tests. - Improve LSPS5 message types, validation, and testing - Replace generic String types with strongly-typed Lsps5AppName and Lsps5WebhookUrl with built-in length and format validation - Restructure imports to follow one-per-line convention - Add constants for notification method strings - Make WebhookNotificationMethod enum more consistent with LSPS5 prefix - Use explicit serde_json::json and serde_json::Value instead of imports - Improve code documentation with proper ticks and references - Add comprehensive test vectors from the BLIP-0055 specification

- Rename types to use consistent UPPERCASE prefix (LSPS5AppName, LSPS5WebhookUrl) - Use LSPSUrl type for webhook URLs instead of raw strings - Replace LightningError with LSPSResponseError for better separation of concerns - Add appName and webhookUrl validation on msgs so they can be reused both in service and client - Embed notification parameters directly in WebhookNotificationMethod - Improve serialization/deserialization for webhook notifications

- Introduce LSPS5ServiceEvent for LSPS-side webhook events including registration, listing, removal, and notification. - Define LSPS5ClientEvent for handling webhook outcomes on the client (Lightning node) side. - Outline WebhookNotificationParams enum to support notification-specific parameters. - Improve LSPS5 event documentation and field naming - Rename client/lsp fields to counterparty_node_id for consistent terminology - Replace generic String types with more specific Lsps5AppName and Lsps5WebhookUrl - Add comprehensive documentation for all events and fields - Include format specifications (UTF-8, ISO8601) and size constraints - Add request_id field to all relevant events for consistent request tracking - Provide detailed descriptions of error codes and their meanings - Use complete sentences in documentation comments

- Rename types to LSPS5AppName and LSPS5WebhookUrl for naming consistency - Change WebhookNotificationSent to SendWebhookNotifications for better model - Update event to indicate a notification needs to be sent (not was sent) - Add headers field to support HTTP POST requests with headers - Remove redundant WebhookNotificationParams enum (was moved to msgs) - Fix bLIP-50 references to correctly reference bLIP-55

Implements the LSPS5 webhook registration service that allows LSPs to notify clients of important events via webhooks. This service handles webhook registration, listing, removal, and notification delivery according to the LSPS5 specification. Some details: - A generic HttpClient trait is defined so users can provide their own HTTP implementation - A generic TimeProvider trait is defined with a DefaultTimeProvider that uses std functionality - Uses URL utils to validate webhook URLs according to LSPS5 requirements - Uses secure message signing logic from the lightning::util::message_signing module - Works with the events and messages defined in earlier commits - Tests will be provided in a future commit

martinsaposnic · 2025-03-27T21:46:08Z

Thanks for the changes so far! This is definitely moving in the right direction quickly.

I’ve added more high-level comments and will get into the details in the next round. Feel free to squash the fixups!

Thanks for the review @tnull!

I've addressed all new comments and pushed them as fixup: commits.

Key changes:

http_client has been removed. Instead, there's now a SendWebhookNotifications event that informs the user which URLs to call, leaving the actual POST requests to them.
No more panics or unsafe unwraps.
TimeProvider can now be injected via an alternative constructor.

CI is taking forever, so I may need to push a few fixes if anything breaks, but this should be ready for review again!

A couple of things I know will get comments:

I'm using fromRFC3339 and toRFC3339 from LSPSDateTime, but since timestamps are handled as Duration, I had to introduce a function to convert Duration into LSPSDateTime:

/// Returns the number of seconds since the Unix epoch.
pub fn new_from_duration(duration: Duration) -> Self {
    Self(DateTime::from_timestamp(duration.as_secs() as i64, duration.subsec_nanos()).unwrap())
}

It works, but it’s not the cleanest approach—open to suggestions.

The client receives app_name and webhook_url as plain strings, which are then wrapped in LSPS5AppName and LSPS5WebhookUrl via a new method that validates them.

When the event is triggered, the service picks it up and receives already-validated LSPS5AppName and LSPS5WebhookUrl, but it calls validate again to ensure nothing changed.

This pattern—new calls validate, and then validate runs again later—works, but feels a bit redundant. Curious to hear thoughts on whether this is a good approach.

Looking forward to your feedback!

…client logic - Reorder and group imports (e.g. LSPS5 msgs) for clarity - Rename time provider method to duration_since_epoch and - - use LSPSDateTime serialization instead of custom functions - Remove dead code and configuration related to HTTP client - Replace magic numbers with well-named constants and document their purpose - Update tests to use the new time provider method consistently

Implements the client-side functionality for LSPS5 webhook registration, allowing Lightning clients to register, list, and remove webhooks with LSPs. This client handler processes responses and verifies webhook notification signatures. Key features: - Full client API for webhook registration operations - Per-peer state tracking for pending requests - Automatic request timeout and cleanup - Security validation for webhook URLs - Notification signature verification - Some tests are provided but more will come in a future commit This implementation pairs with the service-side LSPS5 webhook handler to complete the webhook registration protocol according to the LSPS5 specification.

- Replace unwrapping with safe Option handling - Remove redundant URL security validation code from client methods, they are now in msgs - Adjust timestamp subtraction to properly handle potential underflow - Group and reorganize imports for clarity and consistency - Add store_signature and check_signature to prevent replay attacks

Fully integrates the LSPS5 webhook components into the lightning-liquidity framework, enabling usage through the LiquidityManager. It includes - Registering LSPS5 events in the event system - Adding LSPS5 module to the main library exports - Updating LSPS0 serialization to handle LSPS5 messages - Adding LSPS5 configuration options to client and service config structures - Implementing message handling for LSPS5 requests and responses - Adding accessor methods for LSPS5 client and service handlers With this change, LSPS5 webhook functionality can now be accessed through the standard LiquidityManager interface, following the same pattern as other LSPS protocols.

…orrectness of the signing logic

tnull

Cool, thank you for the quick turnaround! I did a rather quick round on the fixups, they look good so far, feel free to squash them!

I think CI is currently failing due to some use of moved value: _time_provider`` errors. Will see to do a more thorough round on the entire PR once it passes again.

tnull · 2025-03-28T13:35:54Z

lightning-liquidity/Cargo.toml

@@ -46,4 +47,5 @@ check-cfg = [
 	"cfg(c_bindings)",
 	"cfg(backtrace)",
 	"cfg(ldk_bench)",
+	"cfg(time)",


This is likely not needed, as time is a feature, not a cfg flag, no?

tnull · 2025-03-28T13:38:54Z

lightning-liquidity/src/lsps0/ser.rs

@@ -27,6 +28,7 @@ use bitcoin::secp256k1::PublicKey;
 use core::fmt::{self, Display};
 use core::str::FromStr;

+use core::time::Duration;
 #[cfg(feature = "std")]


I think this now needs to be feature-gated on time, same below, no? Or are we doing this in the follow-up also addressing LSPS2?

tnull · 2025-03-28T13:43:48Z

lightning-liquidity/src/lsps0/ser.rs

+
+	/// Returns the number of seconds since the Unix epoch.
+	pub fn new_from_duration(duration: Duration) -> Self {
+		Self(DateTime::from_timestamp(duration.as_secs() as i64, duration.subsec_nanos()).unwrap())


As Add<Duration> is implemented for chrono::DateTime (see https://docs.rs/chrono/latest/chrono/struct.DateTime.html#impl-Add%3CDuration%3E-for-DateTime%3CTz%3E), we should be able to make this infallible by simply doing

DateTime::UNIX_EPOCH + duration

tnull · 2025-03-28T13:44:13Z

lightning-liquidity/src/lsps0/ser.rs

+	}
+
+	/// Returns the number of seconds since the Unix epoch.
+	pub fn new_from_duration(duration: Duration) -> Self {


Can we make this an impl From<Duration> for LSPSDateTime?

tnull · 2025-03-28T13:45:35Z

lightning-liquidity/src/lsps0/ser.rs

@@ -212,6 +214,16 @@ impl LSPSDateTime {
 			self.0.timestamp().try_into().expect("expiration to be ahead of unix epoch");
 		now_seconds_since_epoch > datetime_seconds_since_epoch
 	}
+
+	/// Returns the number of seconds since the Unix epoch.
+	pub fn timestamp(&self) -> u64 {


Part of introducing LSPSDateTime was exactly to avoid such ambiguous timestamp types. Are we sure we need this, and does it need to be part of the public API?

tnull · 2025-03-28T15:49:21Z

lightning-liquidity/src/lsps5/event.rs

-	/// This event occurs after the LSP successfully contacts a client's webhook.
-	WebhookNotificationSent {
-		/// Client node ID that was notified
+	/// This event occurs when the LSP needs to send a notification to a client's webhook.


Please provide more concrete guidance how users need to handle this event.

martinsaposnic mentioned this pull request Mar 11, 2025

Add bLIP 55: Webhook Registration (LSPS5) lightning/blips#55

Open

ldk-reviews-bot requested a review from arik-so March 11, 2025 20:22

tnull self-requested a review March 11, 2025 20:37

wpaulino removed the request for review from arik-so March 11, 2025 20:39

tnull reviewed Mar 13, 2025

View reviewed changes

martinsaposnic force-pushed the lsps5 branch from 3aca4fc to 4dbe74d Compare March 14, 2025 14:48

martinsaposnic marked this pull request as draft March 14, 2025 14:49

martinsaposnic force-pushed the lsps5 branch 2 times, most recently from 1d4b47c to edf5346 Compare March 14, 2025 16:31

martinsaposnic marked this pull request as ready for review March 14, 2025 16:31

ldk-reviews-bot requested a review from valentinewallace March 14, 2025 16:32

martinsaposnic requested a review from tnull March 14, 2025 17:12

martinsaposnic force-pushed the lsps5 branch 7 times, most recently from 9809682 to af5929e Compare March 16, 2025 14:24

tnull removed the request for review from valentinewallace March 17, 2025 08:33

martinsaposnic force-pushed the lsps5 branch from af5929e to e018300 Compare March 17, 2025 16:10

martinsaposnic force-pushed the lsps5 branch 3 times, most recently from 2c51275 to b0b7bad Compare March 24, 2025 20:06

martinsaposnic requested a review from tnull March 25, 2025 13:08

tnull reviewed Mar 25, 2025

View reviewed changes

martinsaposnic force-pushed the lsps5 branch 2 times, most recently from cc8a9ea to f057bda Compare March 27, 2025 21:16

martinsaposnic added 9 commits March 27, 2025 18:18

fixup: Add timestamp and new_from_duration functions to LSPSDateTime

b9f3569

fixup: Simplify URL utilities by removing trait, renaming Url to LSPSUrl

84d14c7

- Remove unnecessary URL trait (no other implementations expected) - Rename Url to LSPSUrl to avoid namespace conflicts - Clean up redundant comments

martinsaposnic force-pushed the lsps5 branch from f057bda to 979abf1 Compare March 27, 2025 21:27

martinsaposnic requested a review from tnull March 27, 2025 21:46

martinsaposnic added 7 commits March 28, 2025 12:01

Add LSPS5 module structure

da24aaf

fixup: Change BLIP to bLIP

123d929

Add tests for LSPS5 client and service. Also tests for checking the c…

b70e9fb

…orrectness of the signing logic

martinsaposnic force-pushed the lsps5 branch from 979abf1 to b70e9fb Compare March 28, 2025 15:05

tnull reviewed Mar 28, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

LSPS5 implementation #3662

LSPS5 implementation #3662

martinsaposnic commented Mar 11, 2025 •

edited

Loading

ldk-reviews-bot commented Mar 11, 2025 •

edited

Loading

martinsaposnic commented Mar 11, 2025 •

edited

Loading

codecov bot commented Mar 11, 2025 •

edited

Loading

tnull left a comment •

edited

Loading

ldk-reviews-bot commented Mar 13, 2025

tnull commented Mar 13, 2025 •

edited

Loading

tnull commented Mar 13, 2025

martinsaposnic commented Mar 14, 2025 •

edited

Loading

ldk-reviews-bot commented Mar 17, 2025

ldk-reviews-bot commented Mar 17, 2025

tnull commented Mar 18, 2025

martinsaposnic commented Mar 24, 2025

tnull left a comment

martinsaposnic commented Mar 27, 2025

tnull left a comment

tnull Mar 28, 2025

tnull Mar 28, 2025

tnull Mar 28, 2025

tnull Mar 28, 2025

tnull Mar 28, 2025

tnull Mar 28, 2025

LSPS5 implementation #3662

Are you sure you want to change the base?

LSPS5 implementation #3662

Conversation

martinsaposnic commented Mar 11, 2025 • edited Loading

ldk-reviews-bot commented Mar 11, 2025 • edited Loading

martinsaposnic commented Mar 11, 2025 • edited Loading

codecov bot commented Mar 11, 2025 • edited Loading

Codecov Report

tnull left a comment • edited Loading

Choose a reason for hiding this comment

ldk-reviews-bot commented Mar 13, 2025

tnull commented Mar 13, 2025 • edited Loading

tnull commented Mar 13, 2025

martinsaposnic commented Mar 14, 2025 • edited Loading

ldk-reviews-bot commented Mar 17, 2025

ldk-reviews-bot commented Mar 17, 2025

tnull commented Mar 18, 2025

martinsaposnic commented Mar 24, 2025

tnull left a comment

Choose a reason for hiding this comment

martinsaposnic commented Mar 27, 2025

tnull left a comment

Choose a reason for hiding this comment

tnull Mar 28, 2025

Choose a reason for hiding this comment

tnull Mar 28, 2025

Choose a reason for hiding this comment

tnull Mar 28, 2025

Choose a reason for hiding this comment

tnull Mar 28, 2025

Choose a reason for hiding this comment

tnull Mar 28, 2025

Choose a reason for hiding this comment

tnull Mar 28, 2025

Choose a reason for hiding this comment

martinsaposnic commented Mar 11, 2025 •

edited

Loading

ldk-reviews-bot commented Mar 11, 2025 •

edited

Loading

martinsaposnic commented Mar 11, 2025 •

edited

Loading

codecov bot commented Mar 11, 2025 •

edited

Loading

tnull left a comment •

edited

Loading

tnull commented Mar 13, 2025 •

edited

Loading

martinsaposnic commented Mar 14, 2025 •

edited

Loading