server defaults to port 8321 ignoring configured server port from yaml

[thoraxe]

System Info

N/A

Information

• The official example scripts
• My own modified scripts

🐛 Describe the bug

I did a build with venv and modified config.yaml to specify the server port:

...
server:
  port: 5001

When running, if a port argument isn't specified, the port is automatically forcibly set by default, an the configured port is ignored:

INFERENCE_MODEL=meta-llama/Llama-3.2-1B-Instruct VLLM_URL=http://192.168.1.252:4000/v1 VLLM_API_TOKEN=sk-1234 llama stack run --image-type venv ~/.llama/distributions/remote-vllm/remote-vllm-run.yaml 
Using run configuration: /home/thoraxe/.llama/distributions/remote-vllm/remote-vllm-run.yaml
+ python -m llama_stack.distribution.server.server --yaml-config /home/thoraxe/.llama/distributions/remote-vllm/remote-vllm-run.yaml --port 8321
Using config file: /home/thoraxe/.llama/distributions/remote-vllm/remote-vllm-run.yaml
...
...
INFO:     Uvicorn running on http://['::', '0.0.0.0']:8321 (Press CTRL+C to quit)

Error logs

n/a

Expected behavior

server should use the configured port from the yaml and not forcibly set one ignoring the config.

[leseb]

At first glance, it seems to me that the CLI flag takes precedence over the config file, this is a common behavior. At least that's what you example shows.

Did I miss something?

[thoraxe]

@leseb in the absence of specifying a CLI flag, llama stack run sets its own CLI flag, so the config yaml will ALWAYS be ignored.

As you can see from my copy/paste, I did not set a CLI flag, but run set one for me, even though I didn't set one.

So perhaps the subject of the issue is not quite accurate -- perhaps it should be:

llama stack run sets a default CLI flag for port which will always override the config

[leseb]

@leseb in the absence of specifying a CLI flag, llama stack run sets its own CLI flag, so the config yaml will ALWAYS be ignored.

As you can see from my copy/paste, I did not set a CLI flag, but run set one for me, even though I didn't set one.

So perhaps the subject of the issue is not quite accurate -- perhaps it should be:

llama stack run sets a default CLI flag for port which will always override the config

Oh right thanks for clarifying, I could repro and sent a patch!

[thoraxe]

@leseb nice! glad I was able to clear it up.