feat: completing text /chat-completion and /completion tests

[LESSuseLESS]

What does this PR do?

The goal is to have a fairly complete set of provider and e2e tests for /chat-completion and /completion. This is the current list,

grep -oE "def test_[a-zA-Z_+]*" llama_stack/providers/tests/inference/test_text_inference.py | cut -d' ' -f2

• test_model_list
• test_text_completion_non_streaming
• test_text_completion_streaming
• test_text_completion_logprobs_non_streaming
• test_text_completion_logprobs_streaming
• test_text_completion_structured_output
• test_text_chat_completion_non_streaming
• test_text_chat_completion_structured_output
• test_text_chat_completion_streaming
• test_text_chat_completion_with_tool_calling
• test_text_chat_completion_with_tool_calling_streaming

grep -oE "def test_[a-zA-Z_+]*" tests/client-sdk/inference/test_text_inference.py | cut -d' ' -f2

• test_text_completion_non_streaming
• test_text_completion_streaming
• test_text_completion_log_probs_non_streaming
• test_text_completion_log_probs_streaming
• test_text_completion_structured_output
• test_text_chat_completion_non_streaming
• test_text_chat_completion_streaming
• test_text_chat_completion_with_tool_calling_and_non_streaming
• test_text_chat_completion_with_tool_calling_and_streaming
• test_text_chat_completion_with_tool_choice_required
• test_text_chat_completion_with_tool_choice_none
• test_text_chat_completion_structured_output
• test_text_chat_completion_tool_calling_tools_not_in_request

Test plan

== Set up Ollama local server

OLLAMA_HOST=127.0.0.1:8321 with-proxy ollama serve
OLLAMA_HOST=127.0.0.1:8321 ollama run llama3.2:3b-instruct-fp16 --keepalive 60m

== Run a provider test

conda activate stack
OLLAMA_URL="http://localhost:8321" \
pytest -v -s -k "ollama" --inference-model="llama3.2:3b-instruct-fp16" \
llama_stack/providers/tests/inference/test_text_inference.py::TestInference

== Run an e2e test

conda activate sherpa
with-proxy pip install llama-stack
export INFERENCE_MODEL=llama3.2:3b-instruct-fp16
export LLAMA_STACK_PORT=8322
with-proxy llama stack build --template ollama
with-proxy llama stack run --env OLLAMA_URL=http://localhost:8321 ollama

conda activate stack
LLAMA_STACK_PORT=8322 LLAMA_STACK_BASE_URL="http://localhost:8322" \
pytest -v -s --inference-model="llama3.2:3b-instruct-fp16" \
tests/client-sdk/inference/test_text_inference.py

[ashwinb]

lgtm. cc @hardikjshah @ehhuang to take one look given you have been in the test world lately.

[hardikjshah]

left a couple of small comments

[hardikjshah]

Can you please update the MANIFEST.in to include these files and remove the deleted ones ? https://github.com/meta-llama/llama-stack/blob/main/MANIFEST.in

[LESSuseLESS]

ah, didn't know this before. fixed.

[hardikjshah]

Lets just drop this as we recently added (https://github.com/meta-llama/llama-stack/pull/1214/files) defaults to be inferred based on model-id at the api level when not provided.

[LESSuseLESS]

let me fix this with a new diff, so that i can keep this one iso-transformation without any coding logic change

[hardikjshah]

provider_tool_format does not seem to be used

[LESSuseLESS]

same here, there are several places that need some cleanup, and I'll fix them with one more diff, instead of this one