Skip to content

Commit

Permalink
@pgsql/parser for PG16 — no deparse yet supported
Browse files Browse the repository at this point in the history
  • Loading branch information
pyramation committed Apr 4, 2024
1 parent a4751a2 commit 001ad1f
Show file tree
Hide file tree
Showing 59 changed files with 1,178 additions and 92 deletions.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -150,7 +150,7 @@ Our latest is built with `13-latest` branch from libpg_query

* [pgsql-parser](https://github.com/launchql/pgsql-parser): The real PostgreSQL parser for Node.js, providing symmetric parsing and deparsing of SQL statements with actual PostgreSQL parser integration.
* [pgsql-deparser](https://github.com/launchql/pgsql-parser/tree/main/packages/deparser): A streamlined tool designed for converting PostgreSQL ASTs back into SQL queries, focusing solely on deparser functionality to complement `pgsql-parser`.
* [pgsql-enums](https://github.com/launchql/pgsql-parser/tree/main/packages/enums-json): A utility package offering easy access to PostgreSQL enumeration types in JSON format, aiding in string and integer conversions of enums used within ASTs to compliment `pgsql-parser`
* [pgsql-enums](https://github.com/launchql/pgsql-parser/tree/main/packages/pgsql-enums): A utility package offering easy access to PostgreSQL enumeration types in JSON format, aiding in string and integer conversions of enums used within ASTs to compliment `pgsql-parser`
* [@pgsql/enums](https://github.com/launchql/pgsql-parser/tree/main/packages/enums): Provides PostgreSQL AST enums in TypeScript, enhancing type safety and usability in projects interacting with PostgreSQL AST nodes.
* [@pgsql/types](https://github.com/launchql/pgsql-parser/tree/main/packages/types): Offers TypeScript type definitions for PostgreSQL AST nodes, facilitating type-safe construction, analysis, and manipulation of ASTs.
* [@pgsql/utils](https://github.com/launchql/pgsql-parser/tree/main/packages/utils): A comprehensive utility library for PostgreSQL, offering type-safe AST node creation and enum value conversions, simplifying the construction and manipulation of PostgreSQL ASTs.
Expand Down
2 changes: 1 addition & 1 deletion packages/deparser/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@ Our latest is built with `13-latest` branch from libpg_query

* [pgsql-parser](https://github.com/launchql/pgsql-parser): The real PostgreSQL parser for Node.js, providing symmetric parsing and deparsing of SQL statements with actual PostgreSQL parser integration.
* [pgsql-deparser](https://github.com/launchql/pgsql-parser/tree/main/packages/deparser): A streamlined tool designed for converting PostgreSQL ASTs back into SQL queries, focusing solely on deparser functionality to complement `pgsql-parser`.
* [pgsql-enums](https://github.com/launchql/pgsql-parser/tree/main/packages/enums-json): A utility package offering easy access to PostgreSQL enumeration types in JSON format, aiding in string and integer conversions of enums used within ASTs to compliment `pgsql-parser`
* [pgsql-enums](https://github.com/launchql/pgsql-parser/tree/main/packages/pgsql-enums): A utility package offering easy access to PostgreSQL enumeration types in JSON format, aiding in string and integer conversions of enums used within ASTs to compliment `pgsql-parser`
* [@pgsql/enums](https://github.com/launchql/pgsql-parser/tree/main/packages/enums): Provides PostgreSQL AST enums in TypeScript, enhancing type safety and usability in projects interacting with PostgreSQL AST nodes.
* [@pgsql/types](https://github.com/launchql/pgsql-parser/tree/main/packages/types): Offers TypeScript type definitions for PostgreSQL AST nodes, facilitating type-safe construction, analysis, and manipulation of ASTs.
* [@pgsql/utils](https://github.com/launchql/pgsql-parser/tree/main/packages/utils): A comprehensive utility library for PostgreSQL, offering type-safe AST node creation and enum value conversions, simplifying the construction and manipulation of PostgreSQL ASTs.
Expand Down
2 changes: 1 addition & 1 deletion packages/enums/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ In this example, `A_Expr_Kind` is an enum imported from `@pgsql/enums`, and expr

* [pgsql-parser](https://github.com/launchql/pgsql-parser): The real PostgreSQL parser for Node.js, providing symmetric parsing and deparsing of SQL statements with actual PostgreSQL parser integration.
* [pgsql-deparser](https://github.com/launchql/pgsql-parser/tree/main/packages/deparser): A streamlined tool designed for converting PostgreSQL ASTs back into SQL queries, focusing solely on deparser functionality to complement `pgsql-parser`.
* [pgsql-enums](https://github.com/launchql/pgsql-parser/tree/main/packages/enums-json): A utility package offering easy access to PostgreSQL enumeration types in JSON format, aiding in string and integer conversions of enums used within ASTs to compliment `pgsql-parser`
* [pgsql-enums](https://github.com/launchql/pgsql-parser/tree/main/packages/pgsql-enums): A utility package offering easy access to PostgreSQL enumeration types in JSON format, aiding in string and integer conversions of enums used within ASTs to compliment `pgsql-parser`
* [@pgsql/enums](https://github.com/launchql/pgsql-parser/tree/main/packages/enums): Provides PostgreSQL AST enums in TypeScript, enhancing type safety and usability in projects interacting with PostgreSQL AST nodes.
* [@pgsql/types](https://github.com/launchql/pgsql-parser/tree/main/packages/types): Offers TypeScript type definitions for PostgreSQL AST nodes, facilitating type-safe construction, analysis, and manipulation of ASTs.
* [@pgsql/utils](https://github.com/launchql/pgsql-parser/tree/main/packages/utils): A comprehensive utility library for PostgreSQL, offering type-safe AST node creation and enum value conversions, simplifying the construction and manipulation of PostgreSQL ASTs.
Expand Down
65 changes: 12 additions & 53 deletions packages/parser/README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# pgsql-parser
# @pgsql/parser

<p align="center" width="100%">
<img height="120" src="https://github.com/launchql/pgsql-parser/assets/545047/6440fa7d-918b-4a3b-8d1b-755d85de8bea" />
Expand All @@ -14,12 +14,14 @@
<a href="https://www.npmjs.com/package/pgsql-parser"><img height="20" src="https://img.shields.io/github/package-json/v/launchql/pgsql-parser?filename=packages%2Fparser%2Fpackage.json"/></a>
</p>

The real PostgreSQL parser for Node.js, `pgsql-parser` provides symmetric parsing and deparsing of SQL statements using the actual [PostgreSQL parser](https://github.com/pganalyze/libpg_query). It allows you to parse SQL queries into AST and modify or reconstruct SQL queries from the AST.
**NOTE** This is a version of `pgsql-parser` that only contains the parser, for those who don't need deparsing. This is for PG16 only.

The real PostgreSQL parser for Node.js, `@pgsql/parser` provides symmetric parsing and deparsing of SQL statements using the actual [PostgreSQL parser](https://github.com/pganalyze/libpg_query). It allows you to parse SQL queries into AST and modify or reconstruct SQL queries from the AST.

## Installation

```sh
npm install pgsql-parser
npm install @pgsql/parser
```

## Key Features
Expand All @@ -33,61 +35,15 @@ npm install pgsql-parser
Rewrite part of a SQL query:

```js
import { parse, deparse } from 'pgsql-parser';
import { parse } from '@pgsql/parser';

const stmts = parse('SELECT * FROM test_table');

// Assuming the structure of stmts is known and matches the expected type
stmts[0].RawStmt.stmt.SelectStmt.fromClause[0].RangeVar.relname = 'another_table';

console.log(deparse(stmts));

// SELECT * FROM "another_table"
```

## Deparser Example

The `pgsql-deparser` module serializes ASTs to SQL in pure TypeScript, avoiding the full parser's native dependencies. It's useful when only SQL string conversion from ASTs is needed, and is written in pure TypeScript for easy cross-environment deployment.

Here's how you can use the deparser in your TypeScript code, using [`@pgsql/utils`](https://github.com/launchql/pgsql-parser/tree/main/packages/utils) to create an AST for `deparse`:

```ts
import ast, { SelectStmt } from '@pgsql/utils';
import { deparse } from 'pgsql-deparser';

// This could have been obtained from any JSON or AST, not necessarily @pgsql/utils
const stmt: SelectStmt = ast.selectStmt({
targetList: [
ast.resTarget({
val: ast.columnRef({
fields: [ast.aStar()]
})
})
],
fromClause: [
ast.rangeVar({
relname: 'some_table',
inh: true,
relpersistence: 'p'
})
],
limitOption: 'LIMIT_OPTION_DEFAULT',
op: 'SETOP_NONE'
});

// Modify the AST if needed
stmt.SelectStmt.fromClause[0].RangeVar.relname = 'another_table';

// Deparse the modified AST back to a SQL string
console.log(deparse(stmts));

// Output: SELECT * FROM another_table
```

## CLI

```
npm install -g pgsql-parser
npm install -g @pgsql/parser
```

### usage
Expand Down Expand Up @@ -140,7 +96,10 @@ Our latest is built with `13-latest` branch from libpg_query

| PostgreSQL Major Version | libpg_query | Status | npm
|--------------------------|-------------|---------------------|---------|
| 13 | 13-latest | Active development | `latest`
| 16 | 16-latest | Active development | `latest`
| 15 | (n/a) | Not supported |
| 14 | (n/a) | Not supported |
| 13 | 13-latest | use `pgsql-parser` |
| 12 | (n/a) | Not supported |
| 11 | (n/a) | Not supported |
| 10 | 10-latest | Not supported | `@1.3.1` ([tree](https://github.com/launchql/pgsql-parser/tree/39b7b1adc8914253226e286a48105785219a81ca)) |
Expand All @@ -150,7 +109,7 @@ Our latest is built with `13-latest` branch from libpg_query

* [pgsql-parser](https://github.com/launchql/pgsql-parser): The real PostgreSQL parser for Node.js, providing symmetric parsing and deparsing of SQL statements with actual PostgreSQL parser integration.
* [pgsql-deparser](https://github.com/launchql/pgsql-parser/tree/main/packages/deparser): A streamlined tool designed for converting PostgreSQL ASTs back into SQL queries, focusing solely on deparser functionality to complement `pgsql-parser`.
* [pgsql-enums](https://github.com/launchql/pgsql-parser/tree/main/packages/enums-json): A utility package offering easy access to PostgreSQL enumeration types in JSON format, aiding in string and integer conversions of enums used within ASTs to compliment `pgsql-parser`
* [pgsql-enums](https://github.com/launchql/pgsql-parser/tree/main/packages/pgsql-enums): A utility package offering easy access to PostgreSQL enumeration types in JSON format, aiding in string and integer conversions of enums used within ASTs to compliment `pgsql-parser`
* [@pgsql/enums](https://github.com/launchql/pgsql-parser/tree/main/packages/enums): Provides PostgreSQL AST enums in TypeScript, enhancing type safety and usability in projects interacting with PostgreSQL AST nodes.
* [@pgsql/types](https://github.com/launchql/pgsql-parser/tree/main/packages/types): Offers TypeScript type definitions for PostgreSQL AST nodes, facilitating type-safe construction, analysis, and manipulation of ASTs.
* [@pgsql/utils](https://github.com/launchql/pgsql-parser/tree/main/packages/utils): A comprehensive utility library for PostgreSQL, offering type-safe AST node creation and enum value conversions, simplifying the construction and manipulation of PostgreSQL ASTs.
Expand Down
22 changes: 9 additions & 13 deletions packages/parser/__tests__/__snapshots__/async.test.ts.snap
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ exports[`works 1`] = `
"fields": [
{
"String": {
"str": "health_grade",
"sval": "health_grade",
},
},
],
Expand All @@ -57,17 +57,15 @@ exports[`works 1`] = `
"name": [
{
"String": {
"str": "=",
"sval": "=",
},
},
],
"rexpr": {
"A_Const": {
"location": undefined,
"val": {
"String": {
"str": "A",
},
"sval": {
"sval": "A",
},
},
},
Expand All @@ -81,7 +79,7 @@ exports[`works 1`] = `
"fields": [
{
"String": {
"str": "average_rating",
"sval": "average_rating",
},
},
],
Expand All @@ -92,18 +90,16 @@ exports[`works 1`] = `
"name": [
{
"String": {
"str": ">=",
"sval": ">=",
},
},
],
"rexpr": {
"A_Const": {
"location": undefined,
"val": {
"Float": {
"str": "4.5",
},
"fval": {
"fval": "4.5",
},
"location": undefined,
},
},
},
Expand Down
24 changes: 16 additions & 8 deletions packages/parser/__tests__/__snapshots__/funcs.test.ts.snap
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,8 @@ exports[`works 1`] = `
"PLpgSQL_stmt_assign": {
"expr": {
"PLpgSQL_expr": {
"query": "SELECT ast.jsonb_set(result, '{IndexElem, name}', to_jsonb(v_name))",
"parseMode": 3,
"query": "result = ast.jsonb_set(result, '{IndexElem, name}', to_jsonb(v_name))",
},
},
"lineno": 5,
Expand All @@ -22,7 +23,8 @@ exports[`works 1`] = `
"PLpgSQL_stmt_assign": {
"expr": {
"PLpgSQL_expr": {
"query": "SELECT ast.jsonb_set(result, '{IndexElem, ordering}', to_jsonb(v_ordering))",
"parseMode": 3,
"query": "result = ast.jsonb_set(result, '{IndexElem, ordering}', to_jsonb(v_ordering))",
},
},
"lineno": 6,
Expand All @@ -33,7 +35,8 @@ exports[`works 1`] = `
"PLpgSQL_stmt_assign": {
"expr": {
"PLpgSQL_expr": {
"query": "SELECT ast.jsonb_set(result, '{IndexElem, nulls_ordering}', to_jsonb(v_nulls_ordering))",
"parseMode": 3,
"query": "result = ast.jsonb_set(result, '{IndexElem, nulls_ordering}', to_jsonb(v_nulls_ordering))",
},
},
"lineno": 7,
Expand All @@ -44,7 +47,8 @@ exports[`works 1`] = `
"PLpgSQL_stmt_assign": {
"expr": {
"PLpgSQL_expr": {
"query": "SELECT ast.jsonb_set(result, '{IndexElem, expr}', v_expr)",
"parseMode": 3,
"query": "result = ast.jsonb_set(result, '{IndexElem, expr}', v_expr)",
},
},
"lineno": 8,
Expand All @@ -55,7 +59,8 @@ exports[`works 1`] = `
"PLpgSQL_stmt_assign": {
"expr": {
"PLpgSQL_expr": {
"query": "SELECT ast.jsonb_set(result, '{IndexElem, opclass}', v_opclass)",
"parseMode": 3,
"query": "result = ast.jsonb_set(result, '{IndexElem, opclass}', v_opclass)",
},
},
"lineno": 9,
Expand All @@ -66,7 +71,8 @@ exports[`works 1`] = `
"PLpgSQL_stmt_assign": {
"expr": {
"PLpgSQL_expr": {
"query": "SELECT ast.jsonb_set(result, '{IndexElem, collation}', v_collation)",
"parseMode": 3,
"query": "result = ast.jsonb_set(result, '{IndexElem, collation}', v_collation)",
},
},
"lineno": 10,
Expand All @@ -77,7 +83,8 @@ exports[`works 1`] = `
"PLpgSQL_stmt_return": {
"expr": {
"PLpgSQL_expr": {
"query": "SELECT result",
"parseMode": 2,
"query": "result",
},
},
"lineno": 11,
Expand Down Expand Up @@ -167,7 +174,8 @@ exports[`works 1`] = `
},
"default_val": {
"PLpgSQL_expr": {
"query": "SELECT '{"IndexElem":{}}'::jsonb",
"parseMode": 2,
"query": "'{"IndexElem":{}}'::jsonb",
},
},
"lineno": 3,
Expand Down
9 changes: 4 additions & 5 deletions packages/parser/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "pgsql-parser",
"version": "13.15.0",
"name": "@pgsql/parser",
"version": "16.0.0-testing",
"description": "The real PostgreSQL query parser",
"author": "Dan Lynch <[email protected]>",
"homepage": "https://github.com/launchql/pgsql-parser",
Expand Down Expand Up @@ -84,9 +84,8 @@
"typescript": "^5.0.4"
},
"dependencies": {
"libpg-query": "13.3.2",
"libpg-query": "16.1.0",
"minimist": "^1.2.6",
"pgsql-deparser": "^13.14.0",
"pgsql-enums": "^13.9.0"
"@pgsql/types": "13.8.0"
}
}
7 changes: 4 additions & 3 deletions packages/parser/src/index.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import { Deparser, deparse } from 'pgsql-deparser';
import { Node } from '@pgsql/types';

import {
parseQuery,
parseQuerySync,
Expand All @@ -15,7 +16,7 @@ function mapStmt({ stmt, stmt_len, stmt_location }) {
};
}

export const parse = (sql) => {
export const parse = (sql): Node => {
if (!sql) throw new Error('no SQL provided to parser');
const result = parseQuerySync(sql);
return result.stmts.map(mapStmt);
Expand All @@ -27,4 +28,4 @@ export const parseAsync = async (sql) => {
return result.stmts.map(mapStmt);
};

export { deparse, Deparser, parseFunction };
export { parseFunction };
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,7 @@ console.log(getConstraintFromConstrType('CONSTR_CHECK')); // Outputs 'CHECK'

* [pgsql-parser](https://github.com/launchql/pgsql-parser): The real PostgreSQL parser for Node.js, providing symmetric parsing and deparsing of SQL statements with actual PostgreSQL parser integration.
* [pgsql-deparser](https://github.com/launchql/pgsql-parser/tree/main/packages/deparser): A streamlined tool designed for converting PostgreSQL ASTs back into SQL queries, focusing solely on deparser functionality to complement `pgsql-parser`.
* [pgsql-enums](https://github.com/launchql/pgsql-parser/tree/main/packages/enums-json): A utility package offering easy access to PostgreSQL enumeration types in JSON format, aiding in string and integer conversions of enums used within ASTs to compliment `pgsql-parser`
* [pgsql-enums](https://github.com/launchql/pgsql-parser/tree/main/packages/pgsql-enums): A utility package offering easy access to PostgreSQL enumeration types in JSON format, aiding in string and integer conversions of enums used within ASTs to compliment `pgsql-parser`
* [@pgsql/enums](https://github.com/launchql/pgsql-parser/tree/main/packages/enums): Provides PostgreSQL AST enums in TypeScript, enhancing type safety and usability in projects interacting with PostgreSQL AST nodes.
* [@pgsql/types](https://github.com/launchql/pgsql-parser/tree/main/packages/types): Offers TypeScript type definitions for PostgreSQL AST nodes, facilitating type-safe construction, analysis, and manipulation of ASTs.
* [@pgsql/utils](https://github.com/launchql/pgsql-parser/tree/main/packages/utils): A comprehensive utility library for PostgreSQL, offering type-safe AST node creation and enum value conversions, simplifying the construction and manipulation of PostgreSQL ASTs.
Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
12 changes: 12 additions & 0 deletions packages/pgsql-parser/.editorconfig
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
root = true

[*]
indent_style = space
indent_size = 2
end_of_line = lf
charset = utf-8
trim_trailing_whitespace = true
insert_final_newline = true

[*.md]
trim_trailing_whitespace = false
5 changes: 5 additions & 0 deletions packages/pgsql-parser/.eslintignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
node_modules/
dist/
main/
module/
coverage/
Loading

0 comments on commit 001ad1f

Please sign in to comment.