You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
[ ] I have checked the documentation and related resources and couldn't resolve my bug.
Describe the bug
When I use the evaluation function, the timeoutError is frequently displayed, and the final evaluation metrics all show NAN, I refer to the documentation to set up various runconfigs, but it has no effect
evalsets = load_test_data(uploaded_file)
if evalsets is not None:
batch_size = 4
final_df = pd.DataFrame()
try:
result = evaluate(
evalsets,
metrics=metrics,
batch_size=batch_size,
raise_exceptions=True,
callbacks=[TestCallback()],
run_config=RunConfig(timeout=60,max_retries=3, max_wait=60, max_workers=1)
)
final_df = result.to_pandas()
print("final_df:", final_df.head())
except Exception as e:
print(f"评估时出错: {e}")
Error trace
Evaluating: 0%| | 0/8 [00:00<?, ?it/s]Exception raised in Job[0]: TimeoutError()
Evaluating: 25%|████████████████████▊ | 2/8 [02:03<06:10, 61.83s/it]Exception raised in Job[2]: TimeoutError()
Evaluating: 38%|███████████████████████████████▏ | 3/8 [03:03<05:04, 61.00s/it]Exception raised in Job[3]: TimeoutError()
Evaluating: 50%|█████████████████████████████████████████▌ | 4/8 [04:03<04:02, 60.61s/it]Exception raised in Job[4]: TimeoutError()
Evaluating: 62%|███████████████████████████████████████████████████▉ | 5/8 [05:03<03:01, 60.39s/it]Exception raised in Job[5]: TimeoutError()
Evaluating: 75%|██████████████████████████████████████████████████████████████▎ | 6/8 [06:03<02:00, 60.26s/it]Exception raised in Job[6]: TimeoutError()
Evaluating: 88%|████████████████████████████████████████████████████████████████████████▋ | 7/8 [07:03<01:00, 60.18s/it]Exception raised in Job[7]: TimeoutError()
Evaluating: 100%|███████████████████████████████████████████████████████████████████████████████████| 8/8 [08:03<00:00, 60.39s/it]
final_df: user_input retrieved_contexts ... context_precision context_recall
0 请详细介绍奥运会的历史与发展。 [奥运会,全称为奥林匹克运动会(Olympic Games),是世界上规模最大、影响最广的综... ...
NaN NaN
1 请详细介绍奥运会的历史与发展。 [奥运会,全称为奥林匹克运动会(Olympic Games),是世界上规模最大、影响最广的综... ...
NaN NaN
Expected behavior
This issue didn't come up at first, and I wondered if it was due to server load or other factors such as the network, or if there was a problem with the evaluation parameters I had configured. Thanks.
Additional context
Add any other context about the problem here.
The text was updated successfully, but these errors were encountered:
[ ] I have checked the documentation and related resources and couldn't resolve my bug.
Describe the bug
When I use the evaluation function, the timeoutError is frequently displayed, and the final evaluation metrics all show NAN, I refer to the documentation to set up various runconfigs, but it has no effect
Ragas version:0.2.9
Python version:3.12
Code to Reproduce
faithfulness.llm = qwen_llm
answer_relevancy.llm = qwen_llm
answer_relevancy.embeddings = bge_m3
context_recall.llm = qwen_llm
context_precision.llm = qwen_llm
Error trace
Evaluating: 0%| | 0/8 [00:00<?, ?it/s]Exception raised in Job[0]: TimeoutError()
Evaluating: 25%|████████████████████▊ | 2/8 [02:03<06:10, 61.83s/it]Exception raised in Job[2]: TimeoutError()
Evaluating: 38%|███████████████████████████████▏ | 3/8 [03:03<05:04, 61.00s/it]Exception raised in Job[3]: TimeoutError()
Evaluating: 50%|█████████████████████████████████████████▌ | 4/8 [04:03<04:02, 60.61s/it]Exception raised in Job[4]: TimeoutError()
Evaluating: 62%|███████████████████████████████████████████████████▉ | 5/8 [05:03<03:01, 60.39s/it]Exception raised in Job[5]: TimeoutError()
Evaluating: 75%|██████████████████████████████████████████████████████████████▎ | 6/8 [06:03<02:00, 60.26s/it]Exception raised in Job[6]: TimeoutError()
Evaluating: 88%|████████████████████████████████████████████████████████████████████████▋ | 7/8 [07:03<01:00, 60.18s/it]Exception raised in Job[7]: TimeoutError()
Evaluating: 100%|███████████████████████████████████████████████████████████████████████████████████| 8/8 [08:03<00:00, 60.39s/it]
final_df: user_input retrieved_contexts ... context_precision context_recall
0 请详细介绍奥运会的历史与发展。 [奥运会,全称为奥林匹克运动会(Olympic Games),是世界上规模最大、影响最广的综... ...
NaN NaN
1 请详细介绍奥运会的历史与发展。 [奥运会,全称为奥林匹克运动会(Olympic Games),是世界上规模最大、影响最广的综... ...
NaN NaN
Expected behavior
This issue didn't come up at first, and I wondered if it was due to server load or other factors such as the network, or if there was a problem with the evaluation parameters I had configured. Thanks.
Additional context
Add any other context about the problem here.
The text was updated successfully, but these errors were encountered: