[Bug]: How to configure Litellm to use direct Google API instead of Vertex AI for Gemini API calls?

[Arijit-jktech]

What happened?

I am using litellm for both completion and embedding methods with Gemini models. However, when making API calls with the gemini model name, I am encountering an error indicating that the request is being routed through Vertex AI instead of directly calling the Google Gemini API.

Code Snippet
Here is the code I am using to invoke the gemini model:

from litellm import completion


response_object = completion(
                model="gemini/gemini-1.5-flash",
                prompt = "Hi, how are you?"
)
print(response_object)

Issue/Clarification Needed

1. Is Litellm designed to use Vertex AI by default when calling Gemini models?
2. Is there a way to configure Litellm to directly call the Google Gemini API instead of Vertex AI?
3. If routing through Vertex AI is the intended behavior, are there any settings or configurations to bypass this?

Becuse vertex ai has significantly less rate limit than direct gemini api call, I want to use direct gemini call, because the same code snippet when i am running using the below code snippet it is working as expected.

import google.generativeai as genai
model = genai.GenerativeModel(model)
response = model.generate_content(prompt)
print(response)

For reference: https://www.reddit.com/r/googlecloud/comments/1dr7at1/vertex_ai_api_vs_gemini_api/

Any insights on whether this is expected behaviour or a configuration issue would be appreciated. Thanks!

Relevant log output

litellm.exceptions.InternalServerError: litellm.InternalServerError: litellm.InternalServerError: VertexAIException - {
  "error": {
    "code": 503,
    "message": "The model is overloaded. Please try again later.",
    "status": "UNAVAILABLE"
  }
}

Are you a ML Ops Team?

Yes

What LiteLLM version are you on ?

1.58.2

Twitter / LinkedIn details

No response