AppendData Logical Command and INSERT INTOs

Author: jaceklaskowski

docs/logical-analysis-rules/ResolveInsertInto.md

@@ -8,7 +8,7 @@
 
 `ResolveInsertInto` takes no parameters to be created.
 
-`ResolveInsertInto` is created when `Analyzer` is requested for [batches](../Analyzer.md#batches).
+`ResolveInsertInto` is created when `Analyzer` is requested for the [batches](../Analyzer.md#batches).
 
 ## <span id="apply"> Executing Rule
 

docs/logical-operators/AppendData.md

@@ -17,9 +17,12 @@
 
 `AppendData` is created using [byName](#byName) and [byPosition](#byPosition) operators.
 
-## <span id="isByName"> isByName flag
+### <span id="isByName"> isByName flag
 
-`AppendData` is given `isByName` flag when [created](#creating-instance).
+`AppendData` is given `isByName` flag when [created](#creating-instance):
+
+* [byName](#byName) with the flag enabled (`true`)
+* [byPosition](#byPosition) with the flag disabled (`false`)
 
 `isByName` is part of the [V2WriteCommand](V2WriteCommand.md#isByName) abstraction.
 
@@ -55,9 +58,9 @@ byPosition(
 * [ResolveInsertInto](../logical-analysis-rules/ResolveInsertInto.md) logical resolution rule is executed
 * `DataFrameWriter` is requested to [insertInto](../DataFrameWriter.md#insertInto)
 
-## Query Planning
+## Execution Planning
 
-`AppendData` is planned to one of the physical operators in [DataSourceV2Strategy](../execution-planning-strategies/DataSourceV2Strategy.md) execution planning strategy:
+`AppendData` is planned as one of the physical operators by [DataSourceV2Strategy](../execution-planning-strategies/DataSourceV2Strategy.md) execution planning strategy:
 
 * `AppendDataExecV1`
 * `AppendDataExec`

docs/logical-operators/InsertIntoStatement.md

@@ -1,19 +1,36 @@
 # InsertIntoStatement
 
-`InsertIntoStatement` is a [ParsedStatement](ParsedStatement.md).
+`InsertIntoStatement` is a [UnaryParsedStatement](ParsedStatement.md).
 
 ## Creating Instance
 
 `InsertIntoStatement` takes the following to be created:
 
-* <span id="table"> [Logical Query Plan](LogicalPlan.md)
-* <span id="partitionSpec"> Partition Specification
+* <span id="table"> Table ([LogicalPlan](LogicalPlan.md))
+* <span id="partitionSpec"> Partition Specification (`Map[String, Option[String]]`)
+* <span id="userSpecifiedCols"> User-specified column names
 * <span id="query"> [Query](LogicalPlan.md)
 * <span id="overwrite"> `overwrite` flag
 * <span id="ifPartitionNotExists"> `ifPartitionNotExists` flag
 
 `InsertIntoStatement` is created when:
 
-* Catalyst DSL (`DslLogicalPlan`) is used to [insertInto](../catalyst-dsl/index.md#insertInto)
+* Catalyst DSL is used to [insertInto](../catalyst-dsl/index.md#insertInto)
 * `AstBuilder` is requested to [withInsertInto](../sql/AstBuilder.md#withInsertInto)
 * `DataFrameWriter` is requested to [insertInto](../DataFrameWriter.md#insertInto)
+
+## Logical Resolution
+
+`InsertIntoStatement` is resolved to the following logical operators:
+
+* [InsertIntoDataSourceCommand](InsertIntoDataSourceCommand.md) (for `InsertIntoStatement`s over [LogicalRelation](LogicalRelation.md) over [InsertableRelation](../InsertableRelation.md)) by [DataSourceAnalysis](../logical-analysis-rules/DataSourceAnalysis.md)
+* [InsertIntoHadoopFsRelationCommand](InsertIntoHadoopFsRelationCommand.md) (for `InsertIntoStatement`s over [LogicalRelation](LogicalRelation.md) over [HadoopFsRelation](../InsertableRelation.md)) by [DataSourceAnalysis](../logical-analysis-rules/DataSourceAnalysis.md)
+
+## Logical Analysis
+
+`InsertIntoStatement`s with `UnresolvedCatalogRelation`s are resolved by the following logical analysis rules:
+
+* [FindDataSourceTable](../logical-analysis-rules/FindDataSourceTable.md)
+* `FallBackFileSourceV2`
+* `PreprocessTableInsertion`
+* [PreWriteCheck](../logical-analysis-rules/PreWriteCheck.md)

docs/sql/AstBuilder.md

@@ -416,12 +416,29 @@ ANTLR rule: `singleExpression`
 
 ### <span id="visitSingleInsertQuery"> visitSingleInsertQuery
 
-Creates a [LogicalPlan](../logical-operators/LogicalPlan.md) with a [InsertIntoTable](../logical-operators/InsertIntoTable.md)
+Calls [withInsertInto](#withInsertInto) (with an `InsertIntoContext`) for the following SQLs:
 
 ```sql
-INSERT INTO TABLE? tableIdentifier partitionSpec? #insertIntoTable
+INSERT OVERWRITE TABLE? multipartIdentifier
+(partitionSpec (IF NOT EXISTS)?)?
+identifierList?
+```
+
+```sql
+INSERT INTO TABLE? multipartIdentifier
+partitionSpec?
+(IF NOT EXISTS)?
+identifierList?
+```
 
-INSERT OVERWRITE TABLE tableIdentifier (partitionSpec (IF NOT EXISTS)?)? #insertOverwriteTable
+```sql
+INSERT OVERWRITE LOCAL? DIRECTORY path=STRING rowFormat? createFileFormat?
+```
+
+```sql
+INSERT OVERWRITE LOCAL? DIRECTORY (path=STRING)?
+tableProvider
+(OPTIONS options=tablePropertyList)?
 ```
 
 ANTLR labeled alternative: `#singleInsertQuery`