Syncing the Contact Buffer in Small Chunks for Delta Updates

[AndreaCatania]

Hello @jrouwe,

I'm integrating a delta update feature to sync the contact buffer across multiple frames. This is necessary because the Contact array is large and creates a bottleneck in the number of objects we can simulate.

Due to the current SaveState implementation, which creates a black-box type buffer, merging two buffers on the client for a single RestoreState call is impossible. The solution I’ve found involves generating partial buffers on the server and sending them to the client:

Buffer partial_state;
Jolt->SaveState(partial_state):
peer->SendToClient(partial_state):

Once the client receives all partial buffers, instead of merging, it calls RestoreState multiple times:

for(partial_state : states){
    Jolt->RestoreState(partial_state);
}

The problem is that RestoreState clears the previous contacts with this function:

bool ContactConstraintManager::RestoreState(StateRecorder &inStream)
{
	bool success = mCache[mCacheWriteIdx].RestoreState(mCache[mCacheWriteIdx ^ 1], inStream);
	mCacheWriteIdx ^= 1;
	mCache[mCacheWriteIdx].Clear();
	return success;
}

I have some ideas on fixing this, but I’d appreciate your thoughts before proceeding. Do you have any suggestions on how we could address this?

[jrouwe]

Hello,

I'm super busy this week, so I haven't had much time to think about this. I should be able to take a look at this next week.

In the mean time: The contact cache only contains contacts for active objects, so I'm not so sure you can get away with partially synching it as I expect a large part of it to change from frame to frame.

Can you describe exactly how the partial sync would work? Is the bottleneck that the buffer needs to be synched across the network? Or is it a CPU issue where it takes too much time to save the entire buffer?

[AndreaCatania]

The server periodically generates a snapshot and sends it to the client, which uses it to validate its predicted simulation. The snapshot includes a list of active objects, their contacts, and the body information only for the synchronized objects.

However, due to the snapshot size, syncing the snapshot at 30Hz (ideally 60Hz or more) is challenging, as it saturates the bandwidth. Currently, I'm limited to syncing it at a max of 4Hz or less.

To address this, I'm synchronizing only one or two bodies at a higher frequency. I generate a snapshot containing the information for those specific objects. When the client receives this, it merges the state with its locally generated snapshot. This allows the client to reset the state for all the synchronized object, where only few have their actual state coming from the server.

We can currently restore the state for all active bodies, not selectively for certain ones, and this creates another problem. In games like Overwatch, destructible objects and physical particles are part of the simulation but aren't synchronized. For example, characters and synchronized bodies interact with these unsynchronized objects but aren't affected by their contacts. Be able to exclude those contacts from the snapshot, would be a great way shrink the snapshot size.

In conclusion, we need a solution that allows restoring partial contact buffers for selected objects, similarly we do with bodies.

[jrouwe]

Hello,

I'm still having a hard time visualizing how you would be able to send partial snapshots.

You're probably filtering out the 'destructibles' through the various functions in StateRecorderFilter resulting in a partial snapshot that has only the 'synchronized' bodies and contacts that involve those bodies. The 'destructibles' must be configured to see the 'synchronized' objects as kinematic or else they will affect the simulation and break determinism. Now you send that partial snapshot to the client and restore it. This would clear the contact cache which would enforce a deterministic simulation for all 'synchronized' objects. Downside is that it also resets the contact cache for the 'destructibles', which means that they will have more trouble coming to rest.

For this particular problem, we could perhaps have special functionality that allows you to selectively copy entries from the previous contact cache (i.e. everything that involves a 'destructible'). We don't care about determinism for these objects, so that should be fine.

If I understand you correctly, you also want to send a subset of the 'synchronized' objects. And this is where I don't really see how that could work. Determinism is only achieved when the contact cache is 100% the same at the beginning of the time step. If you were to copy the contact cache from the previous time step and then apply a partial delta on top of that, you run the risk of retaining a contact in the cache that had been removed on the server, which would break determinism. If anything changed in objects that you did not send, you would also break determinism.

To me it sounds you should not attempt to send a partial snapshot, but try to detect parts of the shapshot that didn't change. This could be done by 'diffing' the previous frame's snapshot with the current and sending that 'diff' over the wire. To help the diffing algorithm, Jolt could tell the StateRecorder when a new object is written (e.g. through a StateRecorder::OnBodyBegin/End/OnConstraintBegin/End/OnContactBegin/End callback). This would allow you to more easily find the corresponding binary blob from the previous frame. On the client, you would convert the 'diff' back into a full stream and present that to Jolt.

[AndreaCatania]

Hello,

I'm still having a hard time visualizing how you would be able to send partial snapshots.

To explain the synchronization mechanism, I need to start by clarifying that each generated snapshot is stored with an index that allows it to be recalled, and also informs us about the execution time.

Applying the snapshot is the first step in rewinding and re-synchronizing the client. This process only occurs when the state of an object differs between the server and the client. The rewinding process consists of two phases:

1. Snapshot Application: This allows us to go back in time to apply a correction to the state of one or more objects.
2. Execution of all inputs made by the player after the applied snapshot.

Since some objects are more important for synchronization—think of the ball in Rocket League compared to the rest—we need a method to sync data for the ball without generating a snapshot that includes all objects' information. This way, we can synchronize the ball at a much higher framerate than other objects.

⭐ When the client receives the partial snapshot from the server, as you mentioned, before applying it, we need to merge it with the snapshot generated by the client so that all client-simulated objects can be brought back in time.

The client then selects the locally generated snapshot using the index from the partial snapshot just received, and the two are merged to form a single snapshot, correctly restoring the state of ALL objects at that specific moment in time.

After applying the snapshot, the inputs are re-executed one by one, and the re-synchronization (rewinding) operation is completed.

Determinism

What you mentioned is correct—determinism is not guaranteed unless the entire snapshot is applied. However, syncing a snapshot only once per second often results in very noticeable corrections, which can cause issues.

The partial snapshot system is designed to reduce the severity of these corrections and simplify the error masking process. This system doesn't eliminate full synchronization, which would still occur.

To recap, we’ll have a partial snapshot running at 30Hz and a full snapshot running at 1Hz or slightly higher. This will minimize the error (the difference between the objects' current positions and those on the server) as much as possible.

To me it sounds you should .... to detect parts of the shapshot that didn't change.

This is something that we will need to address at later stage to optimize sent buffer.

The problem

The issue is that Jolt doesn't allow merging two contact buffers, which is necessary to combine the server's partial snapshot with the local one, as explained here: ⭐

The Solution

I believe the best solution is the one you suggested—enabling the ability to copy specific parts of the contact buffer. This would allow me to apply the server’s partial snapshot after the client’s, while keeping the contact buffer intact for destructible objects that aren’t part of the synchronized simulation.

We just need to ensure this doesn’t require additional metadata or redundant information, which would increase the snapshot size.

[jrouwe]

Do you have any suggestions for what the API should look like (besides the one I posted)?

[AndreaCatania]

I think an API like this would be more than sufficient:

• vector<BodyPairContacts> contacts= PhysicsSystem->GetBodyPairsInContact()
• PhysicsSystem->SetBodyPairContacts(body_pair);

Where BodyPairContacts is a struct similar to:

struct BodyPairContacts 
{
    BodyPair BodyID_A;
    BodyPair BodyID_B;
    vector<Contacts> contacts;
}

This could be the usage:

// Buffer creation
MyBuffer my_buffer
for(auto& body_pair : PhysicsSystem->GetBodyPairsInContact())
{
    if( is_body_sync(body_pair.GetBodyID_A()) || is_body_sync(body_pair.GetBodyID_B()) )
    {
        body_pair.StoreState(my_buffer);
    }
}

// Buffer(s) restore
MyBuffer my_buffer_1
while( !my_buffer_1.is_empty() )
{

    BodyPairContacts body_pair_contacts;
    body_pair.RestoreState(my_buffer);
    PhysicsSystem->SetBodyPairContacts(body_pair);

}

MyBuffer my_buffer_2
while( !my_buffer_2.is_empty() )
{

    BodyPairContacts body_pair_contacts;
    body_pair.RestoreState(my_buffer);
    PhysicsSystem->SetBodyPairContacts(body_pair);

}

[jrouwe]

I've been thinking about this for a while. I'd prefer not to directly expose all the body pair contacts as the lock free hash set that is being used is quite restrictive.

Instead I've created a very simple change that I hope will solve your issue:

#1282

As long as you have disjoint sets of bodies, constraints and contacts, you can use the StateRecorderFilter to create multiple snapshots.
You can then call PhysicsSystem::RestoreState multiple times, as long as you call StateRecorder::SetLastPart(false) on all but the last call to PhysicsSystem::RestoreState. Note that an assert will trigger if you try to restore the same contact from multiple streams.

The unit tests contain an example.

Can you let me know if this indeed works for you?

[AndreaCatania]

Thanks a lot. I'll try this and let you know!

[AndreaCatania]

Hello Jorrit,

Apologies for the long delay. I've been sidetracked with other tasks, but I’m fully focused on this now and can respond quickly. I’ve reviewed the code you proposed, and I really like it.

The only issue is that I can’t restore the same contacts twice without causing undefined behavior. This complicates things since the contact buffer is a black box for my application, and I can't filter out contacts that were already applied when restoring the two buffers.

This is problematic because the client snapshots all synchronizing bodies and doesn’t know which bodies the server will include in the contact buffer.

A potential solution could be to add two functions to the StateRecorderFilter class to allow the RestoreState function to determine whether contacts for specific body pairs should be restored. Something like:

• bool ShouldRestoreContacts(const BodyID &inBody1, const BodyID &inBody2) const
  Used by StateRecorderFilter before restoring a contact.

• void NotifyContactsRestored(const BodyID &inBody1, const BodyID &inBody2).
  Used by StateRecorderFilter after restoring the contacts to notify the filter that will mark them as restored and will skip restoring them twice.

With these functions, I’d be able to skip restoring contacts that were already applied, avoiding undefined behavior or asserts.

Does it sounds ok to you? If so, I can PR your branch with such change.
Thanks for the support.

[jrouwe]

I think NotifyContactsRestored is redundant. If ShouldRestoreContacts is called and returns true, you know that NotifyContactsRestored will be called. If it returns false, you know that NotifyContactsRestored won't be called. So you may as well do the logic in NotifyContactsRestored inside the ShouldRestoreContacts function.

[jrouwe]

I've updated #1282. Ideally the interface should now also have a ShouldRestoreBody and ShouldRestoreConstraint call, but those are a bit more difficult to implement. Also it doesn't matter if you restore the same constraint/body multiple times, the last one will win.

[AndreaCatania]

Thanks a lot, I'm going to try this in a bit and let you know!

Ideally the interface should now also have a ShouldRestoreBody and ShouldRestoreConstraint call, but those are a bit more difficult to implement. Also it doesn't matter if you restore the same constraint/body multiple times, the last one will win.

For me, this is totally fine, but it’s worth documenting: I’m not using this feature because I’m networking the body info separately.

However, it’s important to mention that my approach is to restore contacts from the server buffer first, then the client buffer, excluding those already applied (giving precedence to the server buffer).

The body restoration works in the opposite way, ensuring that the client buffer's body is applied (the last wins).

This functionality doesn’t affect me and likely won’t affect others (either you fully restore the world state or you partition it as much as you can), but it should be documented in the filter class header for clarity.

[jrouwe]

The body restoration works in the opposite way, ensuring that the client buffer's body is applied (the last wins).

Doesn't this get you into trouble? If you use the contact cache of the server and the body configuration of the client, you're not going to be applying the correct contact impulses. This will at least lead to a non-deterministic simulation and it is probably also not physically correct.

[AndreaCatania]

Oh yes, I meant that I restore the body with a separate function call during the body state restoration.

I have an algorithm that performs incremental updates, so I’m trying to break up the buffer as much as possible to take advantage of it.

This algorithm is already capable of discarding the client buffer, so I will not use this new feature when restoring the body. Instead, I’ll use the standard approach.