Provide Peak Usage Insights to Improve User Experience and Load Distribution

[kslamph]

Idea

I’ve observed that many issues arise due to slow response times from the model, often without users being fully aware of the cause. Impatient actions, such as repeatedly canceling and retrying requests, can exacerbate the problem, leading to more severe server strain. To address this, it would be beneficial to provide users with clear insights into peak usage times for the cloud models. By making this information readily available, users can make informed decisions about when to engage with the service, opting for non-peak hours when the system is under less pressure. This approach would not only help distribute the load more evenly but also improve the overall user experience by setting realistic expectations about potential delays during peak times. Additionally, transparency about the service’s health and performance would foster better user understanding and trust.

[theskcd]

this is super important for us, so thank you for highlighting it. We are on top of this