Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Query service implementations for database/sql interfaces #1552

Merged
merged 54 commits into from
Dec 10, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
54 commits
Select commit Hold shift + click to select a range
d242af3
fix(sql): tmp queryservice default option
vladDotH Nov 10, 2024
ef54737
feat(internal/query/xsql): conn boilerplate
vladDotH Oct 30, 2024
152fd71
feat(internal/query/conn): driver initial implemetations
vladDotH Nov 10, 2024
6ee564b
feat(internal/query/conn): normalize & execContext fns
vladDotH Nov 10, 2024
76921eb
Merge remote-tracking branch 'ydb/master'
vladDotH Nov 10, 2024
0732759
feat(internal/query/xsql): tx options transform
vladDotH Nov 10, 2024
4ca5144
feat(internal/query/xsql): tx initials
vladDotH Nov 10, 2024
2678662
feat(intenral/query/xsql): rows impls
vladDotH Nov 13, 2024
e69f310
Merge branch 'ydb-platform:master' into master
vladDotH Nov 13, 2024
daa6a68
Merge branch 'ydb-platform:master' into master
vladDotH Nov 15, 2024
ff26c0b
feat(internal/query/conn): stmts
vladDotH Nov 15, 2024
29d44eb
feat(internal/query/conn/tx): ExecContext, PrepareContext impls
vladDotH Nov 15, 2024
2d20df5
feat(internal/query/conn): PrepareContext impl
vladDotH Nov 15, 2024
e750be4
fix(sql): xsql renamed
vladDotH Nov 17, 2024
f756a76
fix(internal/query/conn): currentTx interface
vladDotH Nov 17, 2024
c1d27ac
fix(internal/query): xsql rm
vladDotH Nov 17, 2024
ed723c4
feat(internal/query/conn): QueryContext impl
vladDotH Nov 17, 2024
10c3bd4
chore(internal/query/conn): linted
vladDotH Nov 17, 2024
28cd198
feat(internal/query/conn): remained impls
vladDotH Nov 17, 2024
fc89935
chore(internal/query/conn): linted
vladDotH Nov 17, 2024
691008c
Merge remote-tracking branch 'origin' into fork/vladDotH/master
asmyasnikov Nov 19, 2024
f479b3e
added YDB_DATABASE_SQL_OVER_QUERY_SERVICE env for run database/sql ov…
asmyasnikov Nov 19, 2024
1370a72
chore(internal/query/conn/rows): linted
vladDotH Nov 19, 2024
f976dd6
fix(internal/query/conn/tx): nil txc rm
vladDotH Nov 19, 2024
6ad28a9
fix(internal/query/conn/rows/Next): correct dst pointers array
vladDotH Nov 19, 2024
380baac
fix(internal/query/conn/rows): nextResultSets check & columns cache
vladDotH Nov 19, 2024
a0cc5a0
refactor(internal/table/context): public QueryModeFromContext
vladDotH Nov 19, 2024
b292289
feat(internal/query/conn): explain mode support
vladDotH Nov 19, 2024
8e5eaab
fix(internal/query/conn/rows): pure eof returning & NextResultSet fix
vladDotH Nov 19, 2024
9f78824
chore(internal/query/conn): linted
vladDotH Nov 19, 2024
a9f83c1
added YDB_DATABASE_SQL_OVER_QUERY_SERVICE env to experimental tests
asmyasnikov Nov 19, 2024
cc37fad
feat(internal/value): driver.Value force cast
vladDotH Nov 20, 2024
c7e7a04
fix(internal/query): resultSetIndex overflow check
vladDotH Nov 20, 2024
f4a2325
Apply suggestions from code review
asmyasnikov Nov 20, 2024
dbac1d7
fix(internal/value): cast error joining
vladDotH Nov 20, 2024
2f1071a
fix(internal/value/castTo): inner castTo calling revert
vladDotH Nov 20, 2024
410e804
fix(internal/value/castTo): Value.CastTo error wrapping
vladDotH Nov 20, 2024
dc27c84
refactor: QueryModes & querymodes funtions transfered to xcontext
vladDotH Nov 23, 2024
2ffc490
chore(internal/table/conn): linted
vladDotH Nov 23, 2024
b7e83f1
fix(internal/query/conn): badconn rm
vladDotH Nov 23, 2024
a948510
refactor(internal/value): force ptr value set in missing type casts
vladDotH Nov 24, 2024
60f2b54
fix(internal/query/conn): yql type & castResult
vladDotH Nov 24, 2024
75c7061
feat(interna/value/tupleValue.castTo): driver.Value switch
vladDotH Nov 24, 2024
5c47229
feat(internal/query/conn/rows): discarded columns support
vladDotH Nov 24, 2024
756153b
ci(database_sql_with_tx_control_test): temporary query service suppor…
vladDotH Nov 24, 2024
d974d4c
Merge branch 'ydb-platform:master' into master
vladDotH Nov 24, 2024
9f8f706
fix(internal/table/conn): query modes definitions
vladDotH Nov 24, 2024
85da81d
chore(internal/query/conn): linted
vladDotH Nov 24, 2024
21b6d88
Merge branch 'ydb-platform:master' into master
vladDotH Dec 9, 2024
3d61b65
fixed parameters
asmyasnikov Dec 9, 2024
9f7962d
extracted ExplainQueryMode to internal/connector, other query modes m…
asmyasnikov Dec 9, 2024
1ebcc3c
extraced internal/{table,query}/conn/stmt.go into internal/connector …
asmyasnikov Dec 10, 2024
e52398c
moved internal/{table,query}/conn to internal/connector/{table,query}
asmyasnikov Dec 10, 2024
46b1007
rename internal/connector to internal/xsql
asmyasnikov Dec 10, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,7 @@ jobs:
YDB_CONNECTION_STRING_SECURE: grpcs://localhost:2135/local
YDB_SSL_ROOT_CERTIFICATES_FILE: /tmp/ydb_certs/ca.pem
YDB_SESSIONS_SHUTDOWN_URLS: http://localhost:8765/actors/kqp_proxy?force_shutdown=all
YDB_DATABASE_SQL_OVER_QUERY_SERVICE: 1
HIDE_APPLICATION_OUTPUT: 1
steps:
- name: Checkout code
Expand Down
4 changes: 2 additions & 2 deletions driver.go
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,6 @@ import (
"github.com/ydb-platform/ydb-go-sdk/v3/discovery"
"github.com/ydb-platform/ydb-go-sdk/v3/internal/balancer"
"github.com/ydb-platform/ydb-go-sdk/v3/internal/conn"
"github.com/ydb-platform/ydb-go-sdk/v3/internal/connector"
internalCoordination "github.com/ydb-platform/ydb-go-sdk/v3/internal/coordination"
coordinationConfig "github.com/ydb-platform/ydb-go-sdk/v3/internal/coordination/config"
"github.com/ydb-platform/ydb-go-sdk/v3/internal/credentials"
Expand All @@ -37,6 +36,7 @@ import (
"github.com/ydb-platform/ydb-go-sdk/v3/internal/topic/topicclientinternal"
"github.com/ydb-platform/ydb-go-sdk/v3/internal/xcontext"
"github.com/ydb-platform/ydb-go-sdk/v3/internal/xerrors"
"github.com/ydb-platform/ydb-go-sdk/v3/internal/xsql"
"github.com/ydb-platform/ydb-go-sdk/v3/internal/xsync"
"github.com/ydb-platform/ydb-go-sdk/v3/log"
"github.com/ydb-platform/ydb-go-sdk/v3/operation"
Expand Down Expand Up @@ -93,7 +93,7 @@ type (
topic *xsync.Once[*topicclientinternal.Client]
topicOptions []topicoptions.TopicOption

databaseSQLOptions []connector.Option
databaseSQLOptions []xsql.Option

pool *conn.Pool

Expand Down
20 changes: 10 additions & 10 deletions dsn.go
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,10 @@ import (
"github.com/ydb-platform/ydb-go-sdk/v3/balancers"
"github.com/ydb-platform/ydb-go-sdk/v3/credentials"
"github.com/ydb-platform/ydb-go-sdk/v3/internal/bind"
"github.com/ydb-platform/ydb-go-sdk/v3/internal/connector"
"github.com/ydb-platform/ydb-go-sdk/v3/internal/dsn"
tableSql "github.com/ydb-platform/ydb-go-sdk/v3/internal/table/conn"
"github.com/ydb-platform/ydb-go-sdk/v3/internal/xerrors"
"github.com/ydb-platform/ydb-go-sdk/v3/internal/xsql"
tableSql "github.com/ydb-platform/ydb-go-sdk/v3/internal/xsql/table/conn"
)

const tablePathPrefixTransformer = "table_path_prefix"
Expand Down Expand Up @@ -64,41 +64,41 @@ func parseConnectionString(dataSourceName string) (opts []Option, _ error) {
if mode == tableSql.UnknownQueryMode {
return nil, xerrors.WithStackTrace(fmt.Errorf("unknown query mode: %s", queryMode))
}
opts = append(opts, withConnectorOptions(connector.WithDefaultQueryMode(mode)))
opts = append(opts, withConnectorOptions(xsql.WithDefaultQueryMode(mode)))
} else if queryMode := info.Params.Get("query_mode"); queryMode != "" {
mode := tableSql.QueryModeFromString(queryMode)
if mode == tableSql.UnknownQueryMode {
return nil, xerrors.WithStackTrace(fmt.Errorf("unknown query mode: %s", queryMode))
}
opts = append(opts, withConnectorOptions(connector.WithDefaultQueryMode(mode)))
opts = append(opts, withConnectorOptions(xsql.WithDefaultQueryMode(mode)))
}
if fakeTx := info.Params.Get("go_fake_tx"); fakeTx != "" {
for _, queryMode := range strings.Split(fakeTx, ",") {
mode := tableSql.QueryModeFromString(queryMode)
if mode == tableSql.UnknownQueryMode {
return nil, xerrors.WithStackTrace(fmt.Errorf("unknown query mode: %s", queryMode))
}
opts = append(opts, withConnectorOptions(connector.WithFakeTx(mode)))
opts = append(opts, withConnectorOptions(xsql.WithFakeTx(mode)))
}
}
if info.Params.Has("go_query_bind") {
var binders []connector.Option
var binders []xsql.Option
queryTransformers := strings.Split(info.Params.Get("go_query_bind"), ",")
for _, transformer := range queryTransformers {
switch transformer {
case "declare":
binders = append(binders, connector.WithQueryBind(bind.AutoDeclare{}))
binders = append(binders, xsql.WithQueryBind(bind.AutoDeclare{}))
case "positional":
binders = append(binders, connector.WithQueryBind(bind.PositionalArgs{}))
binders = append(binders, xsql.WithQueryBind(bind.PositionalArgs{}))
case "numeric":
binders = append(binders, connector.WithQueryBind(bind.NumericArgs{}))
binders = append(binders, xsql.WithQueryBind(bind.NumericArgs{}))
default:
if strings.HasPrefix(transformer, tablePathPrefixTransformer) {
prefix, err := extractTablePathPrefixFromBinderName(transformer)
if err != nil {
return nil, xerrors.WithStackTrace(err)
}
binders = append(binders, connector.WithQueryBind(bind.TablePathPrefix(prefix)))
binders = append(binders, xsql.WithQueryBind(bind.TablePathPrefix(prefix)))
} else {
return nil, xerrors.WithStackTrace(
fmt.Errorf("unknown query rewriter: %s", transformer),
Expand Down
76 changes: 38 additions & 38 deletions dsn_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,14 +8,14 @@ import (

"github.com/ydb-platform/ydb-go-sdk/v3/config"
"github.com/ydb-platform/ydb-go-sdk/v3/internal/bind"
"github.com/ydb-platform/ydb-go-sdk/v3/internal/connector"
querySql "github.com/ydb-platform/ydb-go-sdk/v3/internal/query/conn"
tableSql "github.com/ydb-platform/ydb-go-sdk/v3/internal/table/conn"
"github.com/ydb-platform/ydb-go-sdk/v3/internal/xsql"
conn2 "github.com/ydb-platform/ydb-go-sdk/v3/internal/xsql/query/conn"
"github.com/ydb-platform/ydb-go-sdk/v3/internal/xsql/table/conn"
)

func TestParse(t *testing.T) {
newConnector := func(opts ...connector.Option) *connector.Connector {
c := &connector.Connector{}
newConnector := func(opts ...xsql.Option) *xsql.Connector {
c := &xsql.Connector{}
for _, opt := range opts {
if opt != nil {
if err := opt.Apply(c); err != nil {
Expand All @@ -26,11 +26,11 @@ func TestParse(t *testing.T) {

return c
}
newTableConn := func(opts ...tableSql.Option) *tableSql.Conn {
return tableSql.New(context.Background(), nil, nil, opts...)
newTableConn := func(opts ...conn.Option) *conn.Conn {
return conn.New(context.Background(), nil, nil, opts...)
}
newQueryConn := func(opts ...querySql.Option) *querySql.Conn {
return querySql.New(context.Background(), nil, nil, opts...)
newQueryConn := func(opts ...conn2.Option) *conn2.Conn {
return conn2.New(context.Background(), nil, nil, opts...)
}
compareConfigs := func(t *testing.T, lhs, rhs *config.Config) {
require.Equal(t, lhs.Secure(), rhs.Secure())
Expand All @@ -40,7 +40,7 @@ func TestParse(t *testing.T) {
for _, tt := range []struct {
dsn string
opts []config.Option
connectorOpts []connector.Option
connectorOpts []xsql.Option
err error
}{
{
Expand Down Expand Up @@ -70,8 +70,8 @@ func TestParse(t *testing.T) {
config.WithEndpoint("localhost:2135"),
config.WithDatabase("/local"),
},
connectorOpts: []connector.Option{
connector.WithDefaultQueryMode(tableSql.ScriptingQueryMode),
connectorOpts: []xsql.Option{
xsql.WithDefaultQueryMode(conn.ScriptingQueryMode),
},
err: nil,
},
Expand All @@ -82,9 +82,9 @@ func TestParse(t *testing.T) {
config.WithEndpoint("localhost:2135"),
config.WithDatabase("/local"),
},
connectorOpts: []connector.Option{
connector.WithDefaultQueryMode(tableSql.ScriptingQueryMode),
connector.WithQueryBind(bind.TablePathPrefix("path/to/tables")),
connectorOpts: []xsql.Option{
xsql.WithDefaultQueryMode(conn.ScriptingQueryMode),
xsql.WithQueryBind(bind.TablePathPrefix("path/to/tables")),
},
err: nil,
},
Expand All @@ -95,10 +95,10 @@ func TestParse(t *testing.T) {
config.WithEndpoint("localhost:2135"),
config.WithDatabase("/local"),
},
connectorOpts: []connector.Option{
connector.WithDefaultQueryMode(tableSql.ScriptingQueryMode),
connector.WithQueryBind(bind.TablePathPrefix("path/to/tables")),
connector.WithQueryBind(bind.NumericArgs{}),
connectorOpts: []xsql.Option{
xsql.WithDefaultQueryMode(conn.ScriptingQueryMode),
xsql.WithQueryBind(bind.TablePathPrefix("path/to/tables")),
xsql.WithQueryBind(bind.NumericArgs{}),
},
err: nil,
},
Expand All @@ -109,10 +109,10 @@ func TestParse(t *testing.T) {
config.WithEndpoint("localhost:2135"),
config.WithDatabase("/local"),
},
connectorOpts: []connector.Option{
connector.WithDefaultQueryMode(tableSql.ScriptingQueryMode),
connector.WithQueryBind(bind.TablePathPrefix("path/to/tables")),
connector.WithQueryBind(bind.PositionalArgs{}),
connectorOpts: []xsql.Option{
xsql.WithDefaultQueryMode(conn.ScriptingQueryMode),
xsql.WithQueryBind(bind.TablePathPrefix("path/to/tables")),
xsql.WithQueryBind(bind.PositionalArgs{}),
},
err: nil,
},
Expand All @@ -123,10 +123,10 @@ func TestParse(t *testing.T) {
config.WithEndpoint("localhost:2135"),
config.WithDatabase("/local"),
},
connectorOpts: []connector.Option{
connector.WithDefaultQueryMode(tableSql.ScriptingQueryMode),
connector.WithQueryBind(bind.TablePathPrefix("path/to/tables")),
connector.WithQueryBind(bind.AutoDeclare{}),
connectorOpts: []xsql.Option{
xsql.WithDefaultQueryMode(conn.ScriptingQueryMode),
xsql.WithQueryBind(bind.TablePathPrefix("path/to/tables")),
xsql.WithQueryBind(bind.AutoDeclare{}),
},
err: nil,
},
Expand All @@ -137,9 +137,9 @@ func TestParse(t *testing.T) {
config.WithEndpoint("localhost:2135"),
config.WithDatabase("/local"),
},
connectorOpts: []connector.Option{
connector.WithDefaultQueryMode(tableSql.ScriptingQueryMode),
connector.WithQueryBind(bind.TablePathPrefix("path/to/tables")),
connectorOpts: []xsql.Option{
xsql.WithDefaultQueryMode(conn.ScriptingQueryMode),
xsql.WithQueryBind(bind.TablePathPrefix("path/to/tables")),
},
err: nil,
},
Expand All @@ -150,11 +150,11 @@ func TestParse(t *testing.T) {
config.WithEndpoint("localhost:2135"),
config.WithDatabase("/local"),
},
connectorOpts: []connector.Option{
connector.WithDefaultQueryMode(tableSql.ScriptingQueryMode),
connector.WithQueryBind(bind.TablePathPrefix("path/to/tables")),
connector.WithQueryBind(bind.PositionalArgs{}),
connector.WithQueryBind(bind.AutoDeclare{}),
connectorOpts: []xsql.Option{
xsql.WithDefaultQueryMode(conn.ScriptingQueryMode),
xsql.WithQueryBind(bind.TablePathPrefix("path/to/tables")),
xsql.WithQueryBind(bind.PositionalArgs{}),
xsql.WithQueryBind(bind.AutoDeclare{}),
},
err: nil,
},
Expand All @@ -165,9 +165,9 @@ func TestParse(t *testing.T) {
config.WithEndpoint("localhost:2135"),
config.WithDatabase("/local"),
},
connectorOpts: []connector.Option{
connector.WithFakeTx(tableSql.ScriptingQueryMode),
connector.WithFakeTx(tableSql.SchemeQueryMode),
connectorOpts: []xsql.Option{
xsql.WithFakeTx(conn.ScriptingQueryMode),
xsql.WithFakeTx(conn.SchemeQueryMode),
},
err: nil,
},
Expand Down
133 changes: 0 additions & 133 deletions internal/query/conn/conn.go

This file was deleted.

5 changes: 0 additions & 5 deletions internal/query/conn/errors.go

This file was deleted.

Loading
Loading