Best way to structure real time data dashboard with updates from external source

[fuithecat]

I want to structure the site to have multiple buckets so users can access a dashboard url like this: /livefeed/bucket1 /livefeed/bucket2...etc. The bucketID is a uuid and system generated. Data is changed by an external app (I can have the external app to call a url on Wave to indicate data has been updated). There will be many different browser sessions connected to the bucket dashboards any any one time. I'd like to have the external api call to Wave with the changed data, and have the data broadcast to the change to all browser sessions connected to /livefeed/bucket1.

You examples mention about having a While loop on the server to poll the data and update the dashboard but it doesn't seem very efficient and won't be able to support many users sessions.

thanks

[mturoci]

I assume you are using Wave scripts (please correct me if that's not true).

You examples mention about having a While loop on the server to poll the data and update the dashboard but it doesn't seem very efficient and won't be able to support many users sessions.

Looking at the architecture diagram, you may notice that browsers connect to Wave server, not python Wave script which means the python process can use while True without any effects on the actual dashboard. Thus you can achieve multiple browser sessions without issues.

Another alternative may be a regular cron job that periodically polls your API endpoint for the new data and updates the dashboard as needed.

[fuithecat]

Thanks for the clarifications. I am using the app instead of scripts, I supposed that wouldn't make a difference?

Let's say I want to have a job or api to update the data and write to q.app.myvar, I can set the app mode to 'broadcast' and all browser sessions would see the updated widget content based on q.app.myvar without requiring a loop, correct?

[mturoci]

I supposed that wouldn't make a difference?

Scripts are generally simpler and more suitable for read-only dashboards. Hence while True would be fine. The read-only means you will not be able to handle button clicks or any other interactions. Users will be only able to watch the updating dashboard.

Let's say I want to have a job or api to update the data and write to q.app.myvar, I can set the app mode to 'broadcast' and all browser sessions would see the updated widget content based on q.app.myvar without requiring a loop, correct?

Correct, that's one way to do it, but requires you to jump through more hoops compared to a simple python (Wave) script. However, for apps, you are correct, while True is not an option. You need to poll your API in a non-blocking (async) way. I heavily recommend reading this blog post to get a full grasp of how background jobs work in Python in general.

[fuithecat]

Thanks for sharing the blog post, I will read it thoroughly.

[mturoci]

Cool, let me know in case you have other questions. Data polling section is the most relevant for your usecase.