Save DF with specific Avro schema

[sixers]

Currently spark-avro constructs avro schema from Spark schema on save. It's limited, e.g. we can't set namespace and name of nested records.

One scenario is reading Avro DF, and then saving it again: original schema, and new schema are different.

Having an option to pass Avro schema explictly should solve this.

[rxin]

I'm not super familiar with avro - can avro schema be defined as a string? If yes, maybe we can just take a string option.

@sixers would you like to give this a try?

[sixers]

@rxin Avro schema is just a json.

I'm not super familiar with Scala, but I can give it a try.

[lukdumi]

Voting and waiting for this 👍

[mnarrell]

Any progress on this issue?

[cbyn]

Would also love to see this

[JoshRosen]

This turns out to be pretty easy to support now that we we honor user-provided Avro schemas during writes. I've opened #197 to fix this for 3.2.0.