Peer Storage Feature – Part 2 #3623

adi2011 · 2025-02-26T15:00:46Z

This is the second PR in the peer storage feature series.

Key Changes

Enable ChainMonitor to send peer storage to all channel partners whenever a new block is mined.
Add get_peer_storage_key() to NodeSigner.
Implement decryption logic for peer storage in handle_peer_storage_retrieval().

In the next one, I will add serialisation logic for ChannelMonitors inside peer storage.

tnull · 2025-02-27T08:52:04Z

Seems this isn't properly rebased, hence CI is failing?

adi2011 · 2025-02-27T11:45:11Z

Seems this isn't properly rebased, hence CI is failing?

I think #3626 will fix this.

tnull · 2025-02-27T11:58:21Z

Seems this isn't properly rebased, hence CI is failing?

I think #3626 will fix this.

No, it fails with:

src/ln/our_peer_storage.rs - ln::our_peer_storage::OurPeerStorage (line 48) stdout ----
error[E0433]: failed to resolve: use of undeclared type `OurPeerStorage`
 --> src/ln/our_peer_storage.rs:49:28
  |
3 | let mut our_peer_storage = OurPeerStorage::new();
  |                            ^^^^^^^^^^^^^^ not found in this scope
  |
help: consider importing this struct
  |
2 | use lightning::ln::our_peer_storage::OurPeerStorage;
  |

error[E0433]: failed to resolve: use of undeclared type `OurPeerStorage`
 --> src/ln/our_peer_storage.rs:54:1
  |
8 | OurPeerStorage::decrypt_our_peer_storage(&mut decrypted, &encrypted).unwrap();
  | ^^^^^^^^^^^^^^ not found in this scope
  |
help: consider importing this struct
  |
2 | use lightning::ln::our_peer_storage::OurPeerStorage;
  |

error: aborting due to 2 previous errors

For more information about this error, try `rustc --explain E0433`.
Couldn't compile the test.

failures:
    src/ln/our_peer_storage.rs - ln::our_peer_storage::OurPeerStorage (line 48)

test result: FAILED. 28 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 3.00s

error: test failed, to rerun pass '--doc'

adi2011 · 2025-02-27T12:06:38Z

That is because it’s trying to compile the commented code in the documentation, I will fix it in a bit.

Add get_peer_storage_key method to derive a 32-byte encryption key for securing Peer Storage. This method utilizes HKDF with the node's secret key as input and a fixed info string to generate the encryption key. - Add 'get_peer_storage_key' to NodeSigner. - Implement 'get_peer_storage_key' for KeysManager & PhantomKeysManager.

Introduce the OurPeerStorage struct to manage serialized channel data for peer storage backups. This struct facilitates the distribution of peer storage to channel partners and includes versioning and timestamping for comparison between retrieved peer storage instances. - Add the OurPeerStorage struct with fields for version, timestamp, and serialized channel data (ser_channels). - Implement methods to encrypt and decrypt peer storage securely. - Add functionality to update channel data within OurPeerStorage.

To enable ChainMonitor sending peer storage to channel partners whenever a new block is added, a new message handler, `SendingOnlyMessageHandler`, has been introduced. This allows the `ChainMonitor` to handle the peer storage distribution. Key changes: - Introduce the SendingOnlyMessageHandler trait to facilitate peer storage distribution. - Add SendingOnlyMessageHandler into the MessageHandler. - Implement SendingOnlyMessageHandler for ChainMonitor and IgnoringMessageHandler. - Process SendingOnlyMessageHandler events inside process_events().

Everytime a new block is added we send PeerStorage to all of our channel partner. - Add our_peer_storage and our_peerstorage_encryption_key to ChainMonitor - Write send_peer_storage() and send it to all channel partners whenever a new block is added

Ensure ChannelManager properly handles peer_storage_retrieval. - Write internal_peer_storage_retreival to verify if we recv correct peer storage. - Send error if we get invalid peer_storage data.

Ensure that we correctly handle the sendpeerstorage message event from chainmonitor and process it through channelmonitor. Key Changes: - Retrieve sendpeerstorage message event from chainmonitor for both nodes. - Handle peer storage messages exchanged between nodes and verify correct decryption.

tnull

Unfortunately CI is still failing as the code doesn't work in no_std.

tnull · 2025-02-28T08:38:21Z

lightning/src/ln/our_peer_storage.rs

+use crate::ln::msgs::DecodeError;
+
+/// [`OurPeerStorage`] is used to store channel information that allows for the creation of a
+/// PeerStorage backup. It includes versioning and timestamping for comparison between


nit: IIRC, we settled on snake_case here?

Suggested change

/// PeerStorage backup. It includes versioning and timestamping for comparison between

/// `peer_storage` backup. It includes versioning and timestamping for comparison between

Sorry, my bad. Fixed!

tnull · 2025-02-28T08:43:27Z

lightning/src/ln/our_peer_storage.rs

+impl OurPeerStorage {
+	/// Returns a [`OurPeerStorage`] with version 1 and current timestamp.
+	pub fn new() -> Self {
+		let duration_since_epoch = std::time::SystemTime::now()


As LDK also supports no_std, we need to make this code work in no_std environments. As accessing time is only available in std, it means we either need to find a way to have the user provide the timestamp themselves, or feature-gate the peer storage to only work on std.

That said, are we positive that we need a timestamp in the first place?

No, I have added a timestamp so that if we receive multiple peer storage instances, we can select the latest one. I was thinking of replacing the timestamp with block height...

lightning/src/ln/our_peer_storage.rs

adi2011 · 2025-03-01T07:09:52Z

Thanks for the review @tnull.

Fixed the CI :)

codecov · 2025-03-01T07:54:17Z

Codecov Report

Attention: Patch coverage is 79.91071% with 45 lines in your changes missing coverage. Please review.

Project coverage is 89.98%. Comparing base (eaeed77) to head (4759f21).
Report is 23 commits behind head on main.

Files with missing lines	Patch %	Lines
lightning/src/ln/channelmanager.rs	75.38%	15 Missing and 1 partial ⚠️
lightning/src/ln/our_peer_storage.rs	80.95%	8 Missing and 4 partials ⚠️
lightning/src/util/test_utils.rs	70.00%	9 Missing ⚠️
lightning/src/ln/peer_handler.rs	63.15%	6 Missing and 1 partial ⚠️
lightning/src/ln/blinded_payment_tests.rs	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3623      +/-   ##
==========================================
+ Coverage   89.16%   89.98%   +0.82%     
==========================================
  Files         152      153       +1     
  Lines      118791   124698    +5907     
  Branches   118791   124698    +5907     
==========================================
+ Hits       105921   112213    +6292     
+ Misses      10312     9992     -320     
+ Partials     2558     2493      -65

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

…aListen

adi2011 force-pushed the peer-storage/encrypt-decrypt branch 3 times, most recently from d8df7cf to 361738d Compare February 27, 2025 03:31

adi2011 marked this pull request as ready for review February 27, 2025 07:46

adi2011 force-pushed the peer-storage/encrypt-decrypt branch from 361738d to a1006e0 Compare February 27, 2025 11:25

adi2011 force-pushed the peer-storage/encrypt-decrypt branch 2 times, most recently from ff7fec7 to 465da4a Compare February 27, 2025 12:09

jkczyz added the weekly goal Someone wants to land this this week label Feb 27, 2025

adi2011 force-pushed the peer-storage/encrypt-decrypt branch from 465da4a to 4f29813 Compare February 28, 2025 04:49

Aditya Sharma and others added 6 commits February 28, 2025 10:20

Handle PeerStorageRetrieval in ChannelManager

ae9f30b

Ensure ChannelManager properly handles peer_storage_retrieval. - Write internal_peer_storage_retreival to verify if we recv correct peer storage. - Send error if we get invalid peer_storage data.

adi2011 force-pushed the peer-storage/encrypt-decrypt branch from 4f29813 to 2113034 Compare February 28, 2025 04:51

tnull reviewed Feb 28, 2025

View reviewed changes

adi2011 added 3 commits March 1, 2025 12:17

fixup: nit: Fix comment formatting in our_peer_storage.rs

9ab8833

fixup: our_peer_storage: Replace timestamp with block height

b2d5a8e

fixup: add missing doc for get_block_height

d509fa2

adi2011 requested a review from tnull March 1, 2025 07:08

fixup: fix doctest lightning-liquidity

4759f21

adi2011 added 2 commits March 2, 2025 08:23

fixup: Send peer storage even when using ConnectionStyle::FullBlockVi…

d99b069

…aListen

fixup: Place chainmonitor import at appropriate place.

437513d

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Peer Storage Feature – Part 2 #3623

Peer Storage Feature – Part 2 #3623

adi2011 commented Feb 26, 2025

tnull commented Feb 27, 2025

adi2011 commented Feb 27, 2025

tnull commented Feb 27, 2025

adi2011 commented Feb 27, 2025

tnull left a comment

tnull Feb 28, 2025

adi2011 Mar 1, 2025

tnull Feb 28, 2025

adi2011 Mar 1, 2025

adi2011 commented Mar 1, 2025

codecov bot commented Mar 1, 2025

	/// PeerStorage backup. It includes versioning and timestamping for comparison between
	/// `peer_storage` backup. It includes versioning and timestamping for comparison between

Peer Storage Feature – Part 2 #3623

Are you sure you want to change the base?

Peer Storage Feature – Part 2 #3623

Conversation

adi2011 commented Feb 26, 2025

tnull commented Feb 27, 2025

adi2011 commented Feb 27, 2025

tnull commented Feb 27, 2025

adi2011 commented Feb 27, 2025

tnull left a comment

Choose a reason for hiding this comment

tnull Feb 28, 2025

Choose a reason for hiding this comment

adi2011 Mar 1, 2025

Choose a reason for hiding this comment

tnull Feb 28, 2025

Choose a reason for hiding this comment

adi2011 Mar 1, 2025

Choose a reason for hiding this comment

adi2011 commented Mar 1, 2025

codecov bot commented Mar 1, 2025

Codecov Report