Why are some statges of the sequential selector process repeated?

[arilwan]

I am using sequential selector with floating option (SFFS) and I noticed in the history log, certain stages of the process are repeated, some even more than twice.

[2023-06-22 02:38:04] Features: 42/80 -- score: 0.9336672488489401[Parallel(n_jobs=-1)]: Using backend LokyBackend with 4 concurrent workers.
[Parallel(n_jobs=-1)]: Done  38 out of  38 | elapsed: 23.4min finished
[Parallel(n_jobs=-1)]: Using backend LokyBackend with 4 concurrent workers.
[Parallel(n_jobs=-1)]: Done  33 tasks      | elapsed: 22.6min
[Parallel(n_jobs=-1)]: Done  42 out of  42 | elapsed: 26.4min finished

[2023-06-22 03:27:52] Features: 43/80 -- score: 0.933633990782854[Parallel(n_jobs=-1)]: Using backend LokyBackend with 4 concurrent workers.
[Parallel(n_jobs=-1)]: Done  37 out of  37 | elapsed: 23.2min finished
[Parallel(n_jobs=-1)]: Using backend LokyBackend with 4 concurrent workers.
[Parallel(n_jobs=-1)]: Done  33 tasks      | elapsed: 22.4min
[Parallel(n_jobs=-1)]: Done  43 out of  43 | elapsed: 26.8min finished
[Parallel(n_jobs=-1)]: Using backend LokyBackend with 4 concurrent workers.
[Parallel(n_jobs=-1)]: Done  33 tasks      | elapsed: 22.9min
[Parallel(n_jobs=-1)]: Done  42 out of  42 | elapsed: 26.7min finished

[2023-06-22 04:44:31] Features: 43/80 -- score: 0.9338236416950221[Parallel(n_jobs=-1)]: Using backend LokyBackend with 4 concurrent workers.
[Parallel(n_jobs=-1)]: Done  37 out of  37 | elapsed: 23.3min finished
[Parallel(n_jobs=-1)]: Using backend LokyBackend with 4 concurrent workers.
[Parallel(n_jobs=-1)]: Done  33 tasks      | elapsed: 22.5min
[Parallel(n_jobs=-1)]: Done  43 out of  43 | elapsed: 26.9min finished
[Parallel(n_jobs=-1)]: Using backend LokyBackend with 4 concurrent workers.
[Parallel(n_jobs=-1)]: Done  33 tasks      | elapsed: 22.3min
[Parallel(n_jobs=-1)]: Done  42 out of  42 | elapsed: 26.0min finished

[2023-06-22 06:00:39] Features: 43/80 -- score: 0.9339489338990343[Parallel(n_jobs=-1)]: Using backend LokyBackend with 4 concurrent workers.
[Parallel(n_jobs=-1)]: Done  37 out of  37 | elapsed: 22.8min finished
[Parallel(n_jobs=-1)]: Using backend LokyBackend with 4 concurrent workers.
[Parallel(n_jobs=-1)]: Done  33 tasks      | elapsed: 22.0min
[Parallel(n_jobs=-1)]: Done  43 out of  43 | elapsed: 26.2min finished

[2023-06-22 06:49:42] Features: 44/80 -- score: 0.9334836704105907[Parallel(n_jobs=-1)]:

In the above log example, Features: 43/80 is repeated 3x.

In the second example below, the process jumped back to stage 21:

[2023-06-21 04:31:23] Features: 24/80 -- score: 0.9333800957337954[Parallel(n_jobs=-1)]: Using backend LokyBackend with 4 concurrent workers.
[Parallel(n_jobs=-1)]: Done  33 tasks      | elapsed: 21.0min
[Parallel(n_jobs=-1)]: Done  56 out of  56 | elapsed: 32.6min finished
[Parallel(n_jobs=-1)]: Using backend LokyBackend with 4 concurrent workers.
[Parallel(n_jobs=-1)]: Done  24 out of  24 | elapsed: 11.4min finished
[Parallel(n_jobs=-1)]: Using backend LokyBackend with 4 concurrent workers.
[Parallel(n_jobs=-1)]: Done  23 out of  23 | elapsed: 11.2min finished
[Parallel(n_jobs=-1)]: Using backend LokyBackend with 4 concurrent workers.
[Parallel(n_jobs=-1)]: Done  22 out of  22 | elapsed: 11.0min finished
[Parallel(n_jobs=-1)]: Using backend LokyBackend with 4 concurrent workers.
[Parallel(n_jobs=-1)]: Done  21 out of  21 | elapsed: 10.4min finished
[Parallel(n_jobs=-1)]: Using backend LokyBackend with 4 concurrent workers.
[Parallel(n_jobs=-1)]: Done  20 out of  20 | elapsed:  9.9min finished

[2023-06-21 05:57:56] Features: 21/80 -- score: 0.9340906959399483[Parallel(n_jobs=-1)]: Using backend LokyBackend with 4 concurrent workers.
[Parallel(n_jobs=-1)]: Done  33 tasks      | elapsed: 17.7min
[Parallel(n_jobs=-1)]: Done  59 out of  59 | elapsed: 28.8min finished
[Parallel(n_jobs=-1)]: Using backend LokyBackend with 4 concurrent workers.
[Parallel(n_jobs=-1)]: Done  21 out of  21 | elapsed: 10.4min finished

[2023-06-21 06:37:11] Features: 22/80 -- score: 0.9342523854195989
...

Any idea why this happens?

[rasbt]

Hm, that's interesting! It could be due to the floating option. Say you have features [1, 2, 3, 4, 5], and we are currently at stage [1, 4, 6, 9]. Then maybe removing 4 improves the performance: [1, 6, 9]. Now adding 5 improves the performance [1, 6, 9, 5] and we are back at a 4-feature set.

My guess is this duplication probably wouldn't happen without the floating=True option?