Fixed concurrent problems for document indexing and user query

[danielaskdd]

Problems Addressed

1. When enable_llm_cache_for_entity_extract is True and enable_llm_cache is False, there is race condition when doing document indexing and user query at the same time. Func extract_entities temporarily modifies global config enable_llm_cache to True, causing user queries to also temporarily use the cache.
2. User query is almost blocked by document indexing. The limit_async_func_call decorator implementation using sleep to wait for another thread to release resource is inefficient, that user queries will not have a chance to run until document indexing is complete.
3. Thread safety issues in disk persistence operations of NanoVectorDB.

Solutions

1. Entity Extraction Improvements:

• A force_llm_cache param is introduced to cache handle func, forcing handle_cache to check the cache no matter what global config settings.
• Modified extract_entities func to use this new param to control the behavior of handle_cache.

2. Concurrent Operation Optimization:

• Replaced max concurrent instances control method of limit_async_func_call with asyncio.Semaphore, ensuring FIFO order to prevent user query wait too long after document indexing start.
• Removed redundant semaphore logic from EmbeddingFunc.

3. Introduced asyncio.Lock to protect file save operations in NanoVectorDBStorage.

Bug fixes

1. Fix quantize_embedding func for not support list type embedding param.
2. Fix prompt response cache fail when is_embedding_cache_enabled is true, by adding cache_type to llm_response_cache, and introduced a new cache_type call extract for entities extraction phase .
3. Fix LLM similarity verification fail by correcting the llm_model_func retrieval method.
4. Update similarity_check prompt to avoid LLM generating two scores at times.

Change API Server Config on rag initialization

1. Disable llm cache for entity extraction
2. Enable embedding cache