Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

No thinking tokens/step in response from flash-thinking model #4924

Open
amrit073 opened this issue Jan 24, 2025 · 0 comments
Open

No thinking tokens/step in response from flash-thinking model #4924

amrit073 opened this issue Jan 24, 2025 · 0 comments
Labels
api: vertex-ai Issues related to the googleapis/python-aiplatform API.

Comments

@amrit073
Copy link

Environment details

  • OS type and version: Arch linux
  • Python version: 3.10.12
  • pip version: 23.0.1
  • google-cloud-aiplatform version: 1.78.0

Code example

    async def simple_llm_call():
        client = GenerativeModel(model_name="gemini-2.0-flash-thinking-exp-1219")
        res = await client.generate_content_async(
            stream=True,
            generation_config={"max_output_tokens": 10000},
            contents=Part.from_text("Solve 3*x^3-5*x=1"),
        )
        async for chunk in res:
            print(chunk.candidates[0].content)

    asyncio.run(simple_llm_call())

Output

role: "model"
parts {
  text: "We want to solve the equation $3x^3 - 5x = 1$,"
}

role: "model"
parts {
  text: " which can be rewritten as $3x^3 - 5x - 1 = 0$.\nThis is a cubic equation of the form $ax^3 + bx + c = 0$, where $a=3$, $b=-5$, and $c=-1$.\nWe use the trigonometric"
}

role: "model"
parts {
  text: " method for solving cubic equations of this form.\nLet $x = k \\cos \\theta$. Substituting this into the equation, we get:\n$3(k \\cos \\theta)^3 - 5(k \\cos \\theta) - 1 = 0$\n$3k^3 
\\cos"
}

Acc to docs and linked notebook ,
The first part should include the model's thoughts instead of an answer. And it would also have part[0].thought True but it is not showing up in our case.

@product-auto-label product-auto-label bot added the api: vertex-ai Issues related to the googleapis/python-aiplatform API. label Jan 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: vertex-ai Issues related to the googleapis/python-aiplatform API.
Projects
None yet
Development

No branches or pull requests

1 participant