Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce complexity of data model to only expose City to external users for shipping purposes #55

Open
osowski opened this issue Mar 2, 2020 · 5 comments
Labels
enhancement New feature or request help wanted Extra attention is needed

Comments

@osowski
Copy link
Contributor

osowski commented Mar 2, 2020

As City is the only relevant piece of information that we currently use for shipping, we should reduce the overall complexity of the exposed interfaces that require street address, state, country, zip code, etc.

Ideally, we could reduce this to city & country? (I know this would not give us absolute specificity for every real-world example, but it would be fine for demo purposes... The rest of the address elements would be a programming exercise.)

Depending on how far this change proliferates through the system - whether or not it's just the UI and we default the other fields to acceptable values OR we remove the deprecated elements from the data model altogether, this should proliferate from the beginning to the end of the data flow through the application.

@osowski osowski added enhancement New feature or request help wanted Extra attention is needed labels Mar 2, 2020
@djones6
Copy link
Collaborator

djones6 commented Mar 3, 2020

One idea could be to use 'seaport codes', which uniquely identify a port (similar to airport codes) rather than cities - eg. Oakland is US OAK. This might be more familiar to someone in the industry?

It would be nice to provide autocomplete on this field, so that as a user types "Oak", the suggestion appears as [US OAK] Oakland, CA, USA or similar. But not essential as long as there's some guidance as to what locations the system will recognize.

@osowski
Copy link
Contributor Author

osowski commented Mar 3, 2020

I like the idea of Seaport Codes, as that is pretty static and allows us to manage a smaller data model and list of input (which also allows for dropdown selects instead of freeform text in the UI).

We can pull that information into a basic superset from online resources (reference 1 or reference 2).

Would you imagine exposing the available set of Seaport Codes based on existing Routes via an API call to the potentially new Routes functionality?

@djones6
Copy link
Collaborator

djones6 commented Mar 5, 2020

Yes I think it makes sense for the MS that manages Routes to also manage the Ports. There are several places where this information is required, for example the container microservice for associating containers with ports (information about Cities is currently hard-coded into it - but I'd imagine we would replace Cities with Ports).

How would you envisage the Routes microservice exposing this information? It could be:

  • a REST call to get a list of ports (not very event driven)
  • an entity event that the responsible MS publishes to each time a Port is added or changed
  • an interested party sends a 'query' event, the responsible MS publishes a response event with a list

@jbcodeforce
Copy link
Contributor

I propose to use another topic to put this reference data of seaport code. This is one problem of data pipeline I'm addressing with a customer. ETL jobs are doing this reference data table and other transactional data loading in batch every day to put downstream... but when using kafka we can have done this one time and use new event only when code change. So once those reference data are in the topic. any MS that needs those reference data (meaning they are not millions of codes, but thousand) then they can load them in memory reading from offset 0.

@osowski
Copy link
Contributor Author

osowski commented Mar 9, 2020

As long as that makes sense to @djones6 and team, that's fine with me. There's some interesting tradeoffs that we get with that pattern and reduce the need for explicit long-term storage of data that does not change often.

I still think we'd want the option to have a microservice to query a subset of this data (since we would not want to require a read from offset 0 all the time)... so that would either require an existing microservice to contain some of this lookup API or a new microservice to provide a KTable-like facade & API.

@jbcodeforce, would you expect this implementation to cover just the reference data of seaport codes or the routes journey-data as well? I imagine the information comprising a route would be more dynamic in a real-world implementation, so this may not be as appropriate of an implementation for that type of data.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

3 participants