-
Notifications
You must be signed in to change notification settings - Fork 2
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #2031 from aml-org/publish-5.5.4
W-16224755: Publish 5.5.4
- Loading branch information
Showing
123 changed files
with
14,239 additions
and
389 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,59 @@ | ||
| # 13. AWS OAS parsing | ||
|
|
||
| Date: 2024-07-02 | ||
|
|
||
|
|
||
| ## Status | ||
|
|
||
| Accepted | ||
|
|
||
|
|
||
| ## Context | ||
|
|
||
| Async 2.x supports AVRO Schemas and we currently don't. | ||
| We want to add support AVRO Schemas inside Async APIs and as a standalone documents. | ||
|
|
||
| We need to decide: | ||
| - how are we going to map AVRO Schemas to the AMF Model | ||
| - how are we going to validate AVRO Schemas | ||
|
|
||
| an AVRO Schema has the following properties: | ||
| - It's defined in plain JSON, they MAY be also be defined as a `.avsc` file | ||
| - It doesn't have a special key that indicates it's an AVRO Schema, nor it's version (like JSON Schema does with it's `$schema`) | ||
|
|
||
|
|
||
| ## Decision | ||
|
|
||
| Implement AVRO Schema parsing as a new specification, following the [AVRO Schema 1.9.0 specification](https://avro.apache.org/docs/1.9.0/spec.html#schemas). | ||
|
|
||
| An AVRO Schema may be a: | ||
| - Map | ||
| - Array | ||
| - Record (with fields, each one being any of the possible types) | ||
| - Enum | ||
| - Fixed Type | ||
| - Primitive Type ("null", "boolean", "int", "long", "float", "double", "bytes", "string") | ||
|
|
||
| We've parsed each AVRO Type to the following AMF Shape: | ||
| - Map --> NodeShape with `AdditionalProperties` field for the values shape | ||
| - Array --> ArrayShape with `Items` field for the items shape | ||
| - Record --> NodeShape with `Properties` with a PropertyShape that contains each field shape | ||
| - Enum --> ScalarShape with `Values` field for it's symbols | ||
| - Fixed Type --> ScalarShape with `Datatype` field for its type and `Size` for its size | ||
| - Primitive Type --> ScalarShape with `Datatype` field, or NilShape if its type 'null' | ||
|
|
||
| Given that in this mapping, several AVRO Types correspond to a ScalarShape or a NodeShape, **we've added the `avro-schema` annotation** with an `avroType` that contains the avro type declared before parsing. | ||
| This way, we can know the exact type for rendering or other purposes, for example having a NodeShape and knowing if it's an avro record or a map (both are parsed as NodeShapes). | ||
|
|
||
| We've also added 3 AVRO-specific fields to the `AnyShape` Model via the `AvroFields` trait, adding the following fields: | ||
| - AvroNamespace | ||
| - Aliases | ||
| - Size | ||
|
|
||
|
|
||
| For now only parsing is done | ||
|
|
||
|
|
||
| ## Consequences | ||
|
|
||
| None so far. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
56 changes: 56 additions & 0 deletions
56
...-contract/shared/src/main/scala/amf/apicontract/internal/spec/avro/AvroRenderPlugin.scala
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,56 @@ | ||
| package amf.apicontract.internal.spec.avro | ||
|
|
||
| import amf.core.client.common.{NormalPriority, PluginPriority} | ||
| import amf.core.client.scala.errorhandling.AMFErrorHandler | ||
| import amf.core.client.scala.model.document.BaseUnit | ||
| import amf.core.internal.plugins.render.{RenderConfiguration, RenderInfo, SYAMLBasedRenderPlugin} | ||
| import amf.core.internal.remote.Mimes._ | ||
| import amf.core.internal.remote.Spec | ||
| import amf.apicontract.internal.spec.avro.emitters.context.AvroShapeEmitterContext | ||
| import amf.apicontract.internal.spec.avro.emitters.document.AvroSchemaDocumentEmitter | ||
| import amf.shapes.client.scala.model.document.AvroSchemaDocument | ||
| import org.yaml.model.YDocument | ||
|
|
||
| object AvroRenderPlugin extends SYAMLBasedRenderPlugin { | ||
|
|
||
| override val id: String = Spec.AVRO_SCHEMA.id | ||
|
|
||
| override def priority: PluginPriority = NormalPriority | ||
|
|
||
| override def defaultSyntax(): String = `application/json` | ||
|
|
||
| override def mediaTypes: Seq[String] = Seq(`application/json`) | ||
|
|
||
| override def applies(element: RenderInfo): Boolean = element.unit match { | ||
| case _: AvroSchemaDocument => true | ||
| case _ => false | ||
| } | ||
|
|
||
| override protected def unparseAsYDocument( | ||
| unit: BaseUnit, | ||
| renderConfig: RenderConfiguration, | ||
| errorHandler: AMFErrorHandler | ||
| ): Option[YDocument] = { | ||
| unit match { | ||
| case document: AvroSchemaDocument => Some(emitDocument(renderConfig, errorHandler, document)) | ||
| case _ => None | ||
| } | ||
| } | ||
|
|
||
| private def emitDocument( | ||
| renderConfig: RenderConfiguration, | ||
| errorHandler: AMFErrorHandler, | ||
| document: AvroSchemaDocument | ||
| ): YDocument = { | ||
| new AvroSchemaDocumentEmitter(document)( | ||
| specContext(renderConfig, errorHandler) | ||
| ).emit() | ||
| } | ||
|
|
||
| private def specContext( | ||
| renderConfig: RenderConfiguration, | ||
| errorHandler: AMFErrorHandler | ||
| ): AvroShapeEmitterContext = { | ||
| new AvroShapeEmitterContext(errorHandler, renderConfig) | ||
| } | ||
| } |
37 changes: 37 additions & 0 deletions
37
...n/scala/amf/apicontract/internal/spec/avro/emitters/context/AvroShapeEmitterContext.scala
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,37 @@ | ||
| package amf.apicontract.internal.spec.avro.emitters.context | ||
|
|
||
| import amf.core.client.scala.errorhandling.AMFErrorHandler | ||
| import amf.core.client.scala.model.domain.Shape | ||
| import amf.core.internal.plugins.render.RenderConfiguration | ||
| import amf.shapes.internal.annotations.AVROSchemaType | ||
|
|
||
| class AvroShapeEmitterContext( | ||
| eh: AMFErrorHandler, | ||
| config: RenderConfiguration | ||
| ) extends AvroSpecEmitterContext(eh, AvroRefEmitter, config) { | ||
|
|
||
| override val factory: AvroSpecEmitterFactory = new AvroSpecEmitterFactory()(this) | ||
|
|
||
| def getAvroType(shape: Shape): Option[String] = { | ||
| shape.annotations.find(classOf[AVROSchemaType]).map(_.avroType) | ||
| } | ||
|
|
||
| def isPrimitive(avroType: String): Boolean = | ||
| Seq("null", "boolean", "int", "long", "float", "double", "bytes", "string").contains(avroType) | ||
|
|
||
| def isComplex(avroType: String): Boolean = | ||
| Seq("record", "enum", "array", "map", "union", "fixed").contains(avroType) | ||
|
|
||
| def isComplex(shape: Shape): Boolean = getAvroType(shape).exists(isComplex) | ||
|
|
||
| def isPrimitive(shape: Shape): Boolean = getAvroType(shape).exists(isPrimitive) | ||
| } | ||
|
|
||
| object AvroShapeEmitterContext { | ||
| implicit def fromSpecEmitterContext(implicit spec: AvroSpecEmitterContext): AvroShapeEmitterContext = | ||
| new AvroShapeEmitterContext(spec.eh, spec.config) | ||
|
|
||
| def apply(eh: AMFErrorHandler, config: RenderConfiguration): AvroShapeEmitterContext = { | ||
| new AvroShapeEmitterContext(eh, config) | ||
| } | ||
| } |
87 changes: 87 additions & 0 deletions
87
...in/scala/amf/apicontract/internal/spec/avro/emitters/context/AvroSpecEmitterContext.scala
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,87 @@ | ||
| package amf.apicontract.internal.spec.avro.emitters.context | ||
|
|
||
| import amf.apicontract.client.scala.model.domain.Parameter | ||
| import amf.apicontract.client.scala.model.domain.security.{ParametrizedSecurityScheme, SecurityRequirement} | ||
| import amf.apicontract.internal.spec.avro.emitters.document.AvroDeclaredTypesEmitters | ||
| import amf.apicontract.internal.spec.avro.emitters.domain.AvroShapeEmitter | ||
| import amf.apicontract.internal.spec.common.emitter.{ | ||
| AbstractSecurityRequirementEmitter, | ||
| AnnotationTypeEmitter, | ||
| ParametrizedSecuritySchemeEmitter | ||
| } | ||
| import amf.apicontract.internal.spec.oas.emitter.context.{OasLikeSpecEmitterContext, OasLikeSpecEmitterFactory} | ||
| import amf.core.client.scala.errorhandling.AMFErrorHandler | ||
| import amf.core.client.scala.model.document.BaseUnit | ||
| import amf.core.client.scala.model.domain.extensions.{CustomDomainProperty, ShapeExtension} | ||
| import amf.core.client.scala.model.domain.{DomainElement, RecursiveShape, Shape} | ||
| import amf.core.internal.metamodel.Field | ||
| import amf.core.internal.parser.domain.FieldEntry | ||
| import amf.core.internal.plugins.render.RenderConfiguration | ||
| import amf.core.internal.remote.{AvroSchema, Spec} | ||
| import amf.core.internal.render.BaseEmitters.MapEntryEmitter | ||
| import amf.core.internal.render.SpecOrdering | ||
| import amf.core.internal.render.emitters.{Emitter, EntryEmitter} | ||
| import amf.shapes.internal.spec.common.SchemaVersion | ||
| import amf.shapes.internal.spec.common.emitter.annotations.FacetsInstanceEmitter | ||
| import amf.shapes.internal.spec.common.emitter.{CustomFacetsEmitter, RefEmitter, TagToReferenceEmitter} | ||
| import org.yaml.model.YDocument.PartBuilder | ||
|
|
||
| import scala.util.matching.Regex | ||
|
|
||
| class AvroSpecEmitterContext( | ||
| eh: AMFErrorHandler, | ||
| refEmitter: RefEmitter = AvroRefEmitter, | ||
| val config: RenderConfiguration | ||
| ) extends OasLikeSpecEmitterContext(eh, refEmitter, config) { | ||
|
|
||
| override val factory: AvroSpecEmitterFactory = new AvroSpecEmitterFactory()(this) | ||
| val spec: Spec = AvroSchema | ||
| def schemasDeclarationsPath: String = "/definitions/" | ||
| override def schemaVersion: SchemaVersion = ??? | ||
| override def nameRegex: Regex = ??? | ||
| } | ||
|
|
||
| class AvroSpecEmitterFactory(implicit override val spec: AvroSpecEmitterContext) extends OasLikeSpecEmitterFactory { | ||
|
|
||
| override def declaredTypesEmitter: (Seq[Shape], Seq[BaseUnit], SpecOrdering) => EntryEmitter = | ||
| (types, references, ordering) => | ||
| AvroDeclaredTypesEmitters.obtainEmitter(types, references, ordering, spec.renderConfig) | ||
|
|
||
| def typeEmitters( | ||
| shape: Shape, | ||
| ordering: SpecOrdering, | ||
| ignored: Seq[Field] = Nil, | ||
| references: Seq[BaseUnit], | ||
| pointer: Seq[String] = Nil, | ||
| schemaPath: Seq[(String, String)] = Nil | ||
| ): Seq[Emitter] = { | ||
| implicit val shapeContext: AvroShapeEmitterContext = AvroShapeEmitterContext.fromSpecEmitterContext(spec) | ||
| AvroShapeEmitter(shape, ordering)(shapeContext).entries() | ||
| } | ||
|
|
||
| override def recursiveShapeEmitter( | ||
| shape: RecursiveShape, | ||
| ordering: SpecOrdering, | ||
| schemaPath: Seq[(String, String)] | ||
| ): EntryEmitter = ??? | ||
|
|
||
| override def tagToReferenceEmitter: (DomainElement, Seq[BaseUnit]) => TagToReferenceEmitter = ??? | ||
|
|
||
| override def customFacetsEmitter: (FieldEntry, SpecOrdering, Seq[BaseUnit]) => CustomFacetsEmitter = ??? | ||
|
|
||
| override def facetsInstanceEmitter: (ShapeExtension, SpecOrdering) => FacetsInstanceEmitter = ??? | ||
|
|
||
| override def parametrizedSecurityEmitter | ||
| : (ParametrizedSecurityScheme, SpecOrdering) => ParametrizedSecuritySchemeEmitter = ??? | ||
|
|
||
| override def securityRequirementEmitter: (SecurityRequirement, SpecOrdering) => AbstractSecurityRequirementEmitter = | ||
| ??? | ||
|
|
||
| override def annotationTypeEmitter: (CustomDomainProperty, SpecOrdering) => AnnotationTypeEmitter = ??? | ||
|
|
||
| override def headerEmitter: (Parameter, SpecOrdering, Seq[BaseUnit]) => EntryEmitter = ??? | ||
| } | ||
|
|
||
| object AvroRefEmitter extends RefEmitter { | ||
| override def ref(url: String, b: PartBuilder): Unit = b.obj(MapEntryEmitter("$ref", url).emit(_)) | ||
| } |
Oops, something went wrong.