[Feature Suggest] Output streaming and more complete output logging/callback support

[MRziyi]

Issue Description:

I am currently working on implementing a web frontend for CrewAI. While I am able to use callback functions to update the output from agents, I have noticed that not all content displayed in the terminal is accessible through these callback functions. Specifically, certain outputs such as the "Thoughts" from the agent are not captured.

Problem:
The current implementation of callback functions appears to be incomplete, as it does not capture all the information outputted by the agents. This limitation makes it challenging to fully utilize the agent's output within a web frontend.

Suggestion:
I suggest implementing stream output and more comprehensive logging/callback support. This enhancement would allow users to capture all outputs generated by the agents, thus providing a more complete and useful interface for web frontend implementations.

Current Limitation:
With the current version, only a very limited amount of content outputted to the terminal can be captured by the user through callback functions. This significantly restricts the usability and effectiveness of the callback mechanism.

Request:
Could you please provide guidance on how to capture the complete output from the agents, including the "Thoughts"? Additionally, I would appreciate it if you could consider enhancing the callback functions to support more comprehensive logging and streaming of agent outputs.

Thank you for your attention to this matter. I look forward to any suggestions or improvements you can provide.

[pazevedo-hyland]

Any plans on this @joaomdmoura ?