Compatibility with TensorFlow XLA/NVIDIA TensorRT?

[c00lrain]

Hi

This is general question about deep learning inference acceleration with coriander. TF XLA good idea for inference optimization but limited available CUDA. And NVIDIA also release TensorRT as inference optimizer.
Thus anyone try using coriander for TF-XLA and TensorRT?

[Titaniumtown]

that would be cool.

[hughperkins]

I imagine that from a theoretical point of view, there's no reason why the Coriander concept couldn't work for TF-XLA and TensorRT. There would be some amount of leg-work for someone to take Coriander, and make it work for TF-XLA or TensorRT.

Probably no point in mixing the opencl version of coriander code with TF-XLA version of Coriander. Whether it's better to start from afresh, and copy start across as-and-when, or start from this repo, fork it, comment out most stuff, then progressively uncomment it, is unclear to me.

If it was me, I might try both ways: get a small prototype of running stuff in TF-XLA at all, and then take some really simple CUDA example, and gradually create code to convert that into TF-XLA.

Note that I personally have no plans on working on this in the near future. I'm focused on NLP research currently, and there's only so many hours in the day. I think that getting Coriander working on TF-XLA and TensorRT does sound potentially a great opportunity for someone who wants to make a name for themselves.

At the time that DeepCL, opencl Torch, and Coriander were big, I got tons of attention from different companies, it was awesome. Definitely worth it. It's not the only approach; eg you could do a PhD, or do some other project, but I managed to write Coriander in a few months, whilst holding down a full-time job, which is a fairly rapid pay-off I think. (nuance: I was spending 40-50 hours a week on this :P )

[hughperkins]

for the labels:

• 'wontfix': I'm not going to work on this myself
• 'helpwanted': there's definitely an opportunity here for someone else to work on this (if it was me, I'd probably fork it, but whatever works well for you)
• 'enhancement': since, it is