2019.04.10 Prod Play back end changes and user workflow

Attendees

@billyc @janekdererste

---

Summary

Back-end servers. Based on Kai's wishes, we will switch to a setup with a "play" backend and a "production" back-end.

• The Play back-end will link just to a play front-end; Janek and Billy can break this and test things out however they wish.
• The Production back-end will be connected to two front-ends: dev and prod. This way, Billy and Kai can iterate on front-end visualization changes that use the production back-end database. The prod/prod must always be as stable as possible.
• The production back-end database should always, always have datasets uploaded and prepared visualizations available in the "viz" user login. That way, Kai will always have something to demo.

Discussion re: other teams building their own viz instance. It's not exactly clear how to get people outside of VSP access to the viz platform.

• Currently, we would need to create user accounts and host all the data for any external users. Obviously this doesn't scale.
• The back-end is already dockerized. How much work would it be to make the system transferable; i.e. so that ETH or some other team could spin up their own back end and point a front-end to that?
• Jakub should give us a seminar on how the docker system is currently configured: how are files stored outside dockerized containers, where are the database files, etc.

Tags. Some updates to how we'll be using tags:

• Visualizations need to be associated with model runs, among other things. Billy did some preliminary work in March to attach tags to visualizations. Now that Janek is back, he can make the necessary changes to handle visualization tags properly - and Billy will refactor the front-end to use that.
• The front-end can also tag files with file types, initially based on file name but possibly later through sniffing.

Visualization database fields. In addition to the tag functionality, the visualization database needs a few more searchable fields. We will add title, thumbnail, created-on and modified-on timestamps, and owner/creator fields. This will help with front page layout and eventual search/sort capabilities.

• Thumbnail support will be a good impetus for improving the project page

Public projects and visualizations. We still need the API changes for publicly viewable projects and visualizations. My project list will remain separate from publicly shared projects.

Auto-generated visualizations. When a user uploads a pile of files, the UI should be able to sense what visualizations are possible, based on the types of files that have been uploaded. For example, the dashboard already shows zoomable image file thumbnails if they are tagged with a model run.

• After upload, or possibly as part of the upload dialog, the user should be able to "just create everything for me". This will likely require some configuration.

Performance and GPU usage. Mapbox does not scale to large networks. The Switzerland network crashes Firefox completely. So, what to do?

• Janek's work with 3js has shown that it can handle large networks such as the NEMO Project without trouble.
• A background layer (Mapbox or even Leaflet) could be synchronized as a separate canvas layer, similar to how [Transit-Thingy] does it. (link?)
• Mapbox could still be used for some types of visualizations if they focus on aggregate data, for example.
• Billy needs to learn 3js

---

Actions

@janekderste

• play/dev/prod back-end server changes
• allow visualizations to have tags
• add visualization database fields for title, tags, owner, created/modified timestamps, thumbnail
• create public-project API

@billyc

• implement viz tags properly
• build nicer cards (or other UI) to handle thumbnails & reveal viz details
• On import, auto-sense file types and add type-tags to files
• Think about workflow for auto-creating visualizations based on uploaded files
• Reintroduce public projects after the API changes are made

@jakub

• Give us seminar on back-end docker architecture