Migrate AutoAugment and RandAugment to TensorFlow Addons.

[dynamicwebpaige]

Describe the feature and the current behavior/state.
RandAugment and AutoAugment are both policies for enhanced image preprocessing that are included in EfficientNet, but are still using tf.contrib.

https://github.com/tensorflow/tpu/blob/master/models/official/efficientnet/autoaugment.py

The only tf.contrib image operations that they use, however, are rotate, translate and transform - all of which have been included in TensorFlow Addons.

Relevant information

• Are you willing to contribute it (yes/no):
  No, but am hoping that someone from the community will pick it up (potentially a Google Summer of Code student)?

• Are you willing to maintain it going forward? (yes/no):
  Yes

• Is there a relevant academic paper? (if so, where):
  AutoAugment Reference: https://arxiv.org/abs/1805.09501
  RandAugment Reference: https://arxiv.org/abs/1909.13719

• Is there already an implementation in another framework? (if so, where):
  See link above; this would be a standard migration from tf.contrib.

• Was it part of tf.contrib? (if so, where):
  Yes

Which API type would this fall under (layer, metric, optimizer, etc.)
Image

Who will benefit with this feature?
Anyone doing image preprocessing, especially for EfficientNet.

[bhack]

For the Gsoc probably we need to think more generally about preprocessing.
We have still numpy based operations in https://keras.io/preprocessing/image/.
Also It needs to be investigated if there will be any integration between AutoAugment/Autokeras.

[stmugisha]

For the Gsoc probably we need to think more generally about preprocessing.
We have still numpy based operations in https://keras.io/preprocessing/image/.

@bhack can you please elaborate more on what you mean by this. Thanks

[bhack]

Many images operations in Keras are still not in addons or in tf.image but numpy implemented:
https://github.com/keras-team/keras-preprocessing/tree/master/keras_preprocessing/image
Other image operations are going in TF.IO or TF.graphics?
I think that we need to unify image processing so that eventually ops could benefit from the compiler stack soon or later (MLIR & friends).

[bhack]

Also about AutoKeras can the policy be handled by Autokeras so that autoaugment could be used more generally in other projects/experiments with Autokeras+keras_preprocessing instead of being embedded in Efficientnet?

[abhichou4]

Also about AutoKeras can the policy be handled by Autokeras so that autoaugment could be used more generally in other projects/experiments with Autokeras+keras_preprocessing instead of being embedded in Efficientnet?

I don't think AutoKeras handles policies as such. And if policies are just series of operations we could store them maybe as named tuples. A parser (for the lack of a better word) could then return a ImageAugmentation object using a policy. The methods of this class could use Keras preprocessing, tf.image etc. Seems messy. Is this better than porting already implemented image ops in tfa.image?

We could train models with various policies using this as proof of concept.

[bhack]

AutoKeras was "recently" already integrated with keras preprocessing https://github.com/keras-team/autokeras/pull/922/files.
I agree that probably we can investigate what is needed to expand over Autoaugment (or other kind of) policies.
But I think that in AutoAugmentation is also important to have ops that could benefit maximum performance (also in scheduling) cause you will not do these operations offline.

[fsx950223]

I'm interested in it and I have benefited a lot from AutoAugment.

[seanpmorgan]

So while looking at how AutoKeras is handling hyperparameter containers, at first glance it seems like a suitable replacement for Hparams:
https://github.com/keras-team/keras-tuner/blob/master/kerastuner/engine/hyperparameters.py#L464

We initially decided not to move HParams when we left tf.contrib (even though its convient and works very well) since we didn't want to diverge from officially support APIs.

Adding KerasTuner as a dependency has its own challenges, but I'm wondering if there is a better way to align with the eco-system though using it.

@omalleyt12 @gabrieldemarmiesse Do you have any thoughts about use re-using KerasTuner's HyperParameter object (likely overkill for whats needed in AutoAugment). Or just general thoughts on how AutoAugment in addons fits with the KerasTuner and Keras Preprocessing advances?

[bhack]

We still have also AutoAugment variant policy in https://github.com/google-research/remixmatch

[abhichou4]

Should these augmentations be added as part of a submodule in addons?

[bhack]

I don't know. Probably.. It had its own sub "library" for augmentation 😄 https://github.com/google-research/remixmatch/tree/master/libml
It is quite standard to find fragmentation under google-research repos.