The filter transforms provides SQL-based expressions for filtering rows and optionally column removal from parquet files, per the set of transform project conventions the following runtimes are available:
- python - provides the base python-based transformation implementation.
- ray - enables the running of the python-based transformation in a Ray runtime
- spark - enables the running of a spark-based transformation in a Spark runtime.
- kfp - enables running the ray docker image
in a kubernetes cluster using a generated
yamlfile.