Change names of knn modes

[jmazanec15]

Description

Currently, the 2 "modes" we support are on_disk and in_memory. on_disk mode will configure defaults to use less memory at the expense of potential search latency. in_memory mode configures the defaults to optimized for fast search latency.

So, a better, more general name for these 2 parameters would be:

1. on_disk -> memory_optimized
2. in_memory -> latency_optimized

That being said, without breaking backwards compatibility, Im wondering if we should change the names of these 2 parameters to better describe their intent. This would generalize the mode parameters to focus more on user workload intent as opposed to implementation.

[navneet1v]

@jmazanec15 is there a reason we should change this mode? latency_optimized doesn't really mean that search will be latency optimized.

[jmazanec15]

The main reason is to focus the parameters more on the behavior they will strive for as opposed to how it will be achieved. That way, in the future, we can add different optimizations based on the behavior. For instance, for "memory_optimized", we should enable #2401 by default. For "latency_optimized", we can exercise tradeoffs around more aggressive caching in memory, etc.

latency_optimized doesn't really mean that search will be latency optimized.

I think it does - the defaults are configured to optimize for search latency trading off memory. Its very similar to the index.codec BEST_SPEED vs. BEST_COMPRESSION.

[navneet1v]

For issue #2401, my thought on that was always this, we should have a option to run our native engine in memory constraint environments. Now if the native engine is building a quantized index it doesn't mean that index is memory_optimized.

on_disk -> memory_optimized

On disk, I feel is always a separate feature/capability on how to build the index. #2401 is more about how to load the data in memory. We should not jumble both the things. A binary quantized index may or may not always lives in memory.

[jmazanec15]

mode is supposed to represent a hint for the user configured workload, and then, based on the other information provided, we select defaults to the best of our ability that will achieve that hint.

Now if the native engine is building a quantized index it doesn't mean that index is memory_optimized.

Not sure I understand, this point.

On disk, I feel is always a separate feature/capability on how to build the index. #2401 is more about how to load the data in memory. We should not jumble both the things. A binary quantized index may or may not always lives in memory.

But it also controls default search behavior, via the re-score parameter.