Questions on MoveRobot scheduler

[cwrx777]

release: main/iron 20231229

I notice that the robot fleet adapter follow_new_path method is invoked from here.
1 What calls the operator() method?
2. There may be a delay between from the time it is scheduled until the time it is executed, see below snippet. and in the delay maybe very high, in the order of tens of seconds, eg during busy traffic, multiple robots etc. how can the delay be minimized?

auto schedule_time = std::chrono::high_resolution_clock::now();
_context->worker().schedule([schedule_time, w = weak_from_this(), s](const auto&)
{
      auto execute_time = std::chrono::high_resolution_clock::now();
      std::chrono::duration<double, std::milli> scheduling_delay = execute_time - schedule_time;
      std::cout <<  "Scheduling  delay: " << scheduling_delay.count() << " ms" << std::endl;

      const auto self = w.lock();
      if (!self)
        return;
    ...
}

3. what is the impact if the whole code block is run outside of the worker/scheduler?

[mxgrey]

What calls the operator() method?

It gets called via the rxcpp job pipeline, with the entry point being here.

how can the delay be minimized?

The best way to prevent any delay would be if all long-running jobs, such as planning and negotiation, would always take place on threads / workers that are not the main event loop.

There is a mechanism in rxcpp to ensure that a certain job will always be on a specific worker, but I haven't found a clean way to prevent a job from being done on a certain worker. Without that ability I don't see a way to prevent long running jobs from sometimes blocking the main event loop. This limitation was one of the motivating factors for migrating to Rust+Bevy.

what is the impact if the whole code block is run outside of the worker/scheduler?

Data races, which means corrupted memory, undefined behavior, and segmentation faults. We could try to prevent data races using mutexes, but those are very error prone and easily end up causing deadlocks when small subtle mistakes are made.

[mxgrey]

If you're interested in this problem, one idea I've had in the past is to have a pool of workers specifically dedicated to planning and negotiation. Whenever a new planning or negotiation job is started, we would assign it to one of the workers in the pool in a round-robin pattern.

The worker for the main event loop would be kept out of this pool. That way we guarantee that planning and negotiation never gets assigned to the main event loop worker.

I haven't done this because it will be a sensitive system to implement and then it will take extensive testing to make sure that it has been done correctly and there aren't any loose ends that could cause memory corruption problems. So instead I want to focus my time on the long term solution that will eliminate these (and many other) problems completely with guaranteed correctness.