-
Notifications
You must be signed in to change notification settings - Fork 35
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Reduce complexity of data model to only expose City to external users for shipping purposes #55
Comments
One idea could be to use 'seaport codes', which uniquely identify a port (similar to airport codes) rather than cities - eg. Oakland is It would be nice to provide autocomplete on this field, so that as a user types "Oak", the suggestion appears as |
I like the idea of Seaport Codes, as that is pretty static and allows us to manage a smaller data model and list of input (which also allows for dropdown selects instead of freeform text in the UI). We can pull that information into a basic superset from online resources (reference 1 or reference 2). Would you imagine exposing the available set of Seaport Codes based on existing Routes via an API call to the potentially new Routes functionality? |
Yes I think it makes sense for the MS that manages Routes to also manage the Ports. There are several places where this information is required, for example the container microservice for associating containers with ports (information about Cities is currently hard-coded into it - but I'd imagine we would replace Cities with Ports). How would you envisage the Routes microservice exposing this information? It could be:
|
I propose to use another topic to put this reference data of seaport code. This is one problem of data pipeline I'm addressing with a customer. ETL jobs are doing this reference data table and other transactional data loading in batch every day to put downstream... but when using kafka we can have done this one time and use new event only when code change. So once those reference data are in the topic. any MS that needs those reference data (meaning they are not millions of codes, but thousand) then they can load them in memory reading from offset 0. |
As long as that makes sense to @djones6 and team, that's fine with me. There's some interesting tradeoffs that we get with that pattern and reduce the need for explicit long-term storage of data that does not change often. I still think we'd want the option to have a microservice to query a subset of this data (since we would not want to require a read from offset 0 all the time)... so that would either require an existing microservice to contain some of this lookup API or a new microservice to provide a KTable-like facade & API. @jbcodeforce, would you expect this implementation to cover just the reference data of seaport codes or the routes journey-data as well? I imagine the information comprising a route would be more dynamic in a real-world implementation, so this may not be as appropriate of an implementation for that type of data. |
As City is the only relevant piece of information that we currently use for shipping, we should reduce the overall complexity of the exposed interfaces that require street address, state, country, zip code, etc.
Ideally, we could reduce this to city & country? (I know this would not give us absolute specificity for every real-world example, but it would be fine for demo purposes... The rest of the address elements would be a programming exercise.)
Depending on how far this change proliferates through the system - whether or not it's just the UI and we default the other fields to acceptable values OR we remove the deprecated elements from the data model altogether, this should proliferate from the beginning to the end of the data flow through the application.
The text was updated successfully, but these errors were encountered: