Skip to content

Commit

Permalink
Properly pre-allocate NetworkGraph channel/node maps
Browse files Browse the repository at this point in the history
When we build a new `NetworkGraph` from empty, we're generally
doing an initial startup and will be syncing the graph very soon.
Using an initially-empty `IndexedMap` for the `channels` and
`nodes` results in quite some memory churn, with the initial RGS
application benchmark showing 15% of its time in pagefault handling
alone (i.e. allocating new memory from the OS, let alone the 23%
of time in `memmove`).

Further, when deserializing a `NetworkGraph`, we'd swapped the
expected node and channel count constants, leaving the node map
too small and causing map doubling as we read entries from disk.

Finally, when deserializing, allocating only exactly the amount of
map entries we need is likely to lead to at least one doubling, so
we're better off just over-estimating the number of nodes and
channels and allocating what we want.

Here we just always allocate `channels` and `nodes` based on
constants, leading to a 20%-ish speedup in the initial RGS
application benchmark.
  • Loading branch information
TheBlueMatt committed Jan 31, 2025
1 parent c06b284 commit a5810bf
Showing 1 changed file with 9 additions and 6 deletions.
15 changes: 9 additions & 6 deletions lightning/src/routing/gossip.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1660,8 +1660,7 @@ where

let chain_hash: ChainHash = Readable::read(reader)?;
let channels_count: u64 = Readable::read(reader)?;
// In Nov, 2023 there were about 15,000 nodes; we cap allocations to 1.5x that.
let mut channels = IndexedMap::with_capacity(cmp::min(channels_count as usize, 22500));
let mut channels = IndexedMap::with_capacity(CHAN_COUNT_ESTIMATE);
for _ in 0..channels_count {
let chan_id: u64 = Readable::read(reader)?;
let chan_info: ChannelInfo = Readable::read(reader)?;
Expand All @@ -1673,8 +1672,7 @@ where
if nodes_count > u32::max_value() as u64 / 2 {
return Err(DecodeError::InvalidValue);
}
// In Nov, 2023 there were about 69K channels; we cap allocations to 1.5x that.
let mut nodes = IndexedMap::with_capacity(cmp::min(nodes_count as usize, 103500));
let mut nodes = IndexedMap::with_capacity(NODE_COUNT_ESTIMATE);
for i in 0..nodes_count {
let node_id = Readable::read(reader)?;
let mut node_info: NodeInfo = Readable::read(reader)?;
Expand Down Expand Up @@ -1750,6 +1748,11 @@ where
}
}

// In Jan, 2025 there were about 45K channels
const CHAN_COUNT_ESTIMATE: usize = 50_000;
// In Jan, 2025 there were about 11K nodes
const NODE_COUNT_ESTIMATE: usize = 15_000;

impl<L: Deref> NetworkGraph<L>
where
L::Target: Logger,
Expand All @@ -1760,8 +1763,8 @@ where
secp_ctx: Secp256k1::verification_only(),
chain_hash: ChainHash::using_genesis_block(network),
logger,
channels: RwLock::new(IndexedMap::new()),
nodes: RwLock::new(IndexedMap::new()),
channels: RwLock::new(IndexedMap::with_capacity(CHAN_COUNT_ESTIMATE)),
nodes: RwLock::new(IndexedMap::with_capacity(NODE_COUNT_ESTIMATE)),
next_node_counter: AtomicUsize::new(0),
removed_node_counters: Mutex::new(Vec::new()),
last_rapid_gossip_sync_timestamp: Mutex::new(None),
Expand Down

0 comments on commit a5810bf

Please sign in to comment.