Implementing interpolation methods beyond bi-linear in tfa.image.resample

[EiffL]

Here is a written description of this issue, where we want to add support for other interpolation methods in this function tfa.image.resampler

So, essentially it's a matter of modifying the following files, that I've already started to mess with:

• tensorflow_addons/custom_ops/image/cc/kernels/resampler_ops.cc : This is where the CPU kernel and its gradients are defined
• tensorflow_addons/custom_ops/image/cc/kernels/resampler_ops.h : That's the corresponding header
• tensorflow_addons/custom_ops/image/cc/ops/resampler_ops.cc : That's where the C++ TF ops are defined

Another important file will be:

• tensorflow_addons/custom_ops/image/cc/kernels/resampler_ops_gpu.cu.cc : This is where the GPU kernel and its gradients are defined

All of this should compile easily following the instructions in this part of the README

So essentially, what I've tried to start doing for CPU is removing the default bilinear interpolation scheme and instead replace it by something generic using the resampling kernels already defined in TensorFlow here: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/core/kernels/image/sampling_kernels.h

If we could get Cubic interpolation to work we would already be pretty happy :-)

So concretely, the steps would be:

• : Implement for GPU the equivalent of

  addons/tensorflow_addons/custom_ops/image/cc/kernels/resampler_ops.cc

  Line 62 in 849ceab

  struct Resampler2DFunctor<CPUDevice, T> {

  i.e. replace the logic for bilinear interpolation by something that uses these kernels (if possible)
• : Implement the gradients of the CPU ops here

  addons/tensorflow_addons/custom_ops/image/cc/kernels/resampler_ops.cc

  Line 245 in 849ceab

  struct ResamplerGrad2DFunctor<CPUDevice, T> {

• : Implement the gradients of the GPU ops here

  addons/tensorflow_addons/custom_ops/image/cc/kernels/resampler_ops_gpu.cu.cc

  Line 145 in 849ceab

  __global__ void ResamplerGrad2DKernel(

Alright, this description is not super heavy in explanations so don't hesitate if you have questions @Dr-Zero!

[EiffL]

@Dr-Zero @andrevitorelli how are we doing on this :-) any cool results? or any questions? On implementation itself or on test cases?

[EiffL]

So how are we doing guys :-) ?

[andrevitorelli]

@EiffL, I have written the issue to the tfa repo (you're linked there), they seem to understand it as a bug, also. @Dr-Zero told me he could have it by next Tue.
The code provided there is not as small as 6-7 lines, and is essentially the same of your gist notebook, but that will do, and serves as a unit test.