[FEATURE REQUEST] Sample to define AsyncAPI Schema with references to protobuf schema definitions

[racheliurui]

I have tried to modify the streetlights.yml sample to achieve below goals,

1. change all the message payload definition to proto3
2. group the proto3 definition file in the relative path along with the interface yaml

The sample api is hotsted at below location,
https://github.com/etpartners-com-au/asyncapi-schema-sample/blob/master/SampleAPI.yaml

The issue I currently found is,
I haven't found a validator to help me validate the modified yml, so currently it's hard to me to understand how to draft the api schema to meet my 2 goals

Hope some feedback regards to this. As we are thinking of changing an existing XSD schema lib into proto3 and using AsyncAPI to define the event based interfaces.

Thanks very much !

[fmvilas]

Hey @racheliurui. I'm afraid we'll not support protobuf in version 2 for now. We found some challenges and looks like it will need more thinking. One of them being #216.

In the meantime, you may want to convert protobuf to jsonschema so you can use it on AsyncAPI. You can automate this process using https://github.com/devongovett/protobuf-jsonschema.

Hope it helps!

[racheliurui]

@fmvilas thanks for the explanation.
The AsyncAPI official website mentioned the support for protobuf in rc1
https://www.asyncapi.com/docs/specifications/2.0.0-rc1/

So is it confirmed that version 2 official version won't support protobuf?

BTW, We will put into consideration of the tools to do transformation between jsonschema and protobuf, but eventually, the payload we want to use protobuf (as it's quite mature).

[fmvilas]

Yes, it is confirmed. During RC1 we found some questions that weren't clear so we're being conservative here. Injecting non-JSON schemas into a JSON object is possible but raises a lot of issues because some of the artifacts of the spec (like $ref and JSON Pointers) are not meant for non-JSON structures.

It doesn't mean we give up. We're just being cautious and I hope we can make it for 2.1.0. My suggestion, for now, is that you automate the conversion from protobuf to jsonschema in your CI system. You work on the protobuf version and reference the jsonschema one from your AsyncAPI documents. I know it's not the best UX but should be enough for you to continue until we give first-class support for protobuf. Sorry for this.

[racheliurui]

Thanks, @fmvilas
Hoping to hear the good news very soon as we see AsyncAPI is the correct way to define the service/interface as regards to ESB and streaming platforms.
We will try to keep the message schema in both formats, and at least the JSON Schema can be transformed into Protobuff and vise Versa.

[fmvilas]

Thank you, @racheliurui. We'll do all that's in our hands to make it happen :)

[dskvr]

It's been almost 2 years, are protobufs out of the picture for good?

[derberg]

@dskvr we don't know how to overcome #216 and also not many people ask for it, so noone I know is actively researching for a solution. If you have ideas and can help, that would be great

[derberg]

@dskvr @saryeHaddadi, @juanitocalero, @simonbrownsw, @eliasrdrgz, @conbon

fyi we had a chat about it on our spec 3.0 meeting: https://youtu.be/6zImMBBxUwI?t=2173
Please have a look if you are interested in this topic and join the conversation

[simonbrownsw]

thanks — i worked around it as suggested (by auto-generating jsonschema from the protobufs as part of ci build pipeline, with protoc --jsonschema_out), and actually that works just fine