[Feat]: tokenized prompt

[0wwafa]

Is your feature request related to a problem? Please describe.

When using the REST api, the json object has "text": "this is a prompt" fields.
Instead of text what other types are possible?
it would be great to be able to send already tokenized data (it would speed up the inference and save network data too).

Describe the solution you'd like

a more efficient way to send prompts and history or chubnked prompts.

Describe alternatives you've considered

No response

Additional context

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[holtskinner]

You can pass in multiple types of data, such as Images/Video/Audio.
Example: https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/image-understanding

And you can send a tokenized prompt as a list of strings like this:

import vertexai
from vertexai.generative_models import GenerativeModel

# TODO(developer): Update project_id
# PROJECT_ID = "your-project-id"
vertexai.init(project=PROJECT_ID, location="us-central1")

model = GenerativeModel("gemini-1.5-flash-001")

response = model.generate_content(
    ["What", "'s", " a", " good", " name", " for", " a", " flower", " shop", " that", " specializes", " in", " selling", " bou", "quets", " of", " dried", " flowers", "?"]
)

print(response.text)