api_url fallback

[poetinger]

I took a look around the issues and haven't seen this request yet, nor do I know if anybody has a use case similar to mine. I have just forked the project to see if I can add it myself whenever I get some spare time to do so, but in the mean time I will open this and hope somebody has some recommendation or pointers before I get started.

The (dreaded) user stories:

• As a developer who works in multiple locations, I find it tedious to update the ollama api_url for each location that I work in. I want to have the ability to have more than one url to attempt to connect to.
• As a developer who has a small local LLM for quick queries and an expensive separate machine for large and complex queries, I would like to be able to have both available without requiring configuration tinkering between AI prompts (potentially losing context).

A super fast peek and ripgrep through the codebase, I can see that api_url is currently handled as just a &str, and the connection attempts are bubbled immediately on HTML error responses. I can envision two different solutions to my problems:

1. Make the api_url setting a Vec and treat all listed apis as one glob. This would run into the issue of which model belongs to which api though, requiring extra work to ensure requests are sent to the correct server.
2. Only introduce the concept of a "fallback". When the primary api_url is unreachable, use the fallback_url and bubble its errors instead. This does not solve the first user story, but the second is much more important for my daily usage as it also handles the case where there are multiple servers in one LAN, and I can shutdown the primary as needed.

I'm leaning toward the second solution, but I am curious if this has been considered by anybody else? Hoping this becomes my daily driver!