Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Moving experimental to testkit #876

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .codegen.json
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,8 @@
".codegen/service-pkg.go.tmpl": "service/pkg.go",
".codegen/error_mapping.go.tmpl": "apierr/error_mapping.go",
".codegen/error_alias.go.tmpl": "error_alias.go",
".codegen/mock_workspace_client.go.tmpl": "experimental/mocks/mock_workspace_client.go",
".codegen/mock_account_client.go.tmpl": "experimental/mocks/mock_account_client.go"
".codegen/mock_workspace_client.go.tmpl": "testkit/mocks/mock_workspace_client.go",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you should not rename this to anything else just yet.

".codegen/mock_account_client.go.tmpl": "testkit/mocks/mock_account_client.go"
},
"examples": {
".codegen/examples_test.go.tmpl": "service/{{.Package}}/{{.SnakeName}}_usage_test.go"
Expand Down
4 changes: 2 additions & 2 deletions .codegen/mock_account_client.go.tmpl
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
// Code generated from OpenAPI specs by Databricks SDK Generator. DO NOT EDIT.

package mocks
package testkit

import (
"fmt"
"github.com/databricks/databricks-sdk-go"
{{range .Packages}}
"github.com/databricks/databricks-sdk-go/experimental/mocks/service/{{.Name}}"{{end}}
"github.com/databricks/databricks-sdk-go/testkit/mocks/service/{{.Name}}"{{end}}
)

type MockAccountClient struct {
Expand Down
2 changes: 1 addition & 1 deletion .codegen/mock_workspace_client.go.tmpl
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
// Code generated from OpenAPI specs by Databricks SDK Generator. DO NOT EDIT.

package mocks
package testkit

import (
"fmt"
Expand Down
208 changes: 104 additions & 104 deletions .gitattributes

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion .mockery.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
with-expecter: true
filename: "mock_{{.InterfaceName | snakecase}}.go"
dir: "experimental/mocks/service/{{.PackageName | snakecase}}"
dir: "testkit/mocks/service/{{.PackageName | snakecase}}"
mockname: "Mock{{.InterfaceName}}"
outpkg: "{{.PackageName}}"
packages:
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -559,7 +559,7 @@ import (
"context"
"testing"

"github.com/databricks/databricks-sdk-go/experimental/mocks"
"github.com/databricks/databricks-sdk-go/testkit/mocks"
"github.com/databricks/databricks-sdk-go/listing"
"github.com/databricks/databricks-sdk-go/qa/poll"
"github.com/databricks/databricks-sdk-go/service/compute"
Expand Down
75 changes: 19 additions & 56 deletions examples/mocking/README.md
Original file line number Diff line number Diff line change
@@ -1,62 +1,25 @@
# Interoperability with `gomock`
# Using the testkit

When developing large applications, you find yourself in need of mocking APIs. For Go, there's [`gomock`](https://github.com/golang/mock) framework for code generating testing mocks. In this small example, we'll show how to use `gomock` with Databricks SDK for Go.
To test your application it is often necessary to mock external dependency.
The SDK contains a testkit you can use for such a purpose, so you don't have to
use the [`gomock`](https://github.com/golang/mock) framework for code generating
testing mocks.

Please read through [`dbfs_test.go`](dbfs_test.go) test example.

## Declaring which mocks to generate

```go
//go:generate go run github.com/golang/mock/mockgen@latest -package=mocks -destination=mocks/dbfs.go github.com/databricks/databricks-sdk-go/service/dbfs DbfsService
```

* `go run github.com/golang/mock/mockgen@latest` downloads and executes the latest version of `mockgen` command
* `-package=mocks` instructs to generate mocks in the `mocks` package
* `-destination=mocks/dbfs.go` instructs to create `dbfs.go` file with mock stubs.
* `github.com/databricks/databricks-sdk-go/service/dbfs` tells which Databricks package to look services in.
* `DbfsService` tells which services to generate mocks for.

## Initializing `gomock`

Every test needs the following preamble:
The entrypoint `mocks.NewMockWorkspaceClient` exposes an API that allows you
to retrieve a mock object, on which you can set expectations like so:

```go
ctrl := gomock.NewController(t)
defer ctrl.Finish()
client := mocks.NewMockWorkspaceClient(t)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please limit the changes only to readme.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What should we do with the example? I think it refers to a service that has been renamed from dbfs to files maybe?

client.GetMockJobsAPI().On("ListAll", mock.Anything, jobs.ListJobsRequest{Limit: 10}).Return(
[]jobs.BaseJob{
{
JobId: 64,
},
{
JobId: 65,
},
}, nil,
)
```

## Mocking individual methods with `gomock`

Every actual method call must be mocked for the test to pass:

```go
mockDbfs := mocks.NewMockDbfsService(ctrl)
mockDbfs.EXPECT().Create(gomock.Any(), gomock.Eq(dbfs.Create{
Path: "/a/b/c",
Overwrite: true,
})).Return(&dbfs.CreateResponse{
Handle: 123,
}, nil)
```

## Testing idioms with Databricks SDK for Go

You can stub out the HTTP request flow with `httpclient/fixtures.MappingTransport` and `httpclient/fixtures.SliceTransport`.

Every service has a public `WithImpl()` method, that you can use to set the stubs for every service that is called in the unit tests.

```go
w := workspaces.New(&databricks.Config{
HTTPTransport: fixtures.MappingTransport{
//...
}
})
w.Dbfs.WithImpl(mockDbfs)
```

## Running this example

1. Run `go mod tidy` in this folder to create `go.sum` file to pick dependency versions.
2. Run `go mod vendor` to download dependencies into `vendor/` directory.
3. Run `go generate ./...` to create `mocks/` directory.
4. Run `go test ./...` to invoke tests with mocks.
Please read through [`listjobs_test.go`](listjobs_test.go) test example.
48 changes: 0 additions & 48 deletions examples/mocking/dbfs_test.go

This file was deleted.

38 changes: 37 additions & 1 deletion examples/mocking/go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,43 @@ go 1.18
require (
github.com/databricks/databricks-sdk-go v0.0.0
github.com/golang/mock v1.6.0
github.com/stretchr/testify v1.8.1
)

require (
cloud.google.com/go/compute v1.23.4 // indirect
cloud.google.com/go/compute/metadata v0.2.3 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/felixge/httpsnoop v1.0.4 // indirect
github.com/go-logr/logr v1.4.1 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
github.com/golang/protobuf v1.5.3 // indirect
github.com/google/go-querystring v1.1.0 // indirect
github.com/google/s2a-go v0.1.7 // indirect
github.com/googleapis/enterprise-certificate-proxy v0.3.2 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/stretchr/objx v0.5.2 // indirect
github.com/stretchr/testify v1.9.0 // indirect
go.opencensus.io v0.24.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.49.0 // indirect
go.opentelemetry.io/otel v1.24.0 // indirect
go.opentelemetry.io/otel/metric v1.24.0 // indirect
go.opentelemetry.io/otel/trace v1.24.0 // indirect
golang.org/x/crypto v0.21.0 // indirect
golang.org/x/exp v0.0.0-20240222234643-814bf88cf225 // indirect
golang.org/x/mod v0.16.0 // indirect
golang.org/x/net v0.22.0 // indirect
golang.org/x/oauth2 v0.18.0 // indirect
golang.org/x/sys v0.18.0 // indirect
golang.org/x/text v0.14.0 // indirect
golang.org/x/time v0.5.0 // indirect
google.golang.org/api v0.169.0 // indirect
google.golang.org/appengine v1.6.8 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20240304161311-37d4d3c04a78 // indirect
google.golang.org/grpc v1.62.0 // indirect
google.golang.org/protobuf v1.33.0 // indirect
gopkg.in/ini.v1 v1.67.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
)

replace github.com/databricks/databricks-sdk-go v0.0.0 => ../..
32 changes: 32 additions & 0 deletions examples/mocking/listjobs_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
package mocking

import (
"context"
"testing"

_ "github.com/golang/mock/mockgen/model"
"github.com/stretchr/testify/mock"
"github.com/stretchr/testify/require"

// "github.com/databricks/databricks-sdk-go/service/dbfs"
"github.com/databricks/databricks-sdk-go/service/jobs"
"github.com/databricks/databricks-sdk-go/testkit/mocks"
)

func TestListJobs(t *testing.T) {
client := mocks.NewMockWorkspaceClient(t)
client.GetMockJobsAPI().On("ListAll", mock.Anything, jobs.ListJobsRequest{Limit: 10}).Return(
[]jobs.BaseJob{
{
JobId: 64,
},
{
JobId: 65,
},
}, nil,
)
jobs, err := client.WorkspaceClient.Jobs.ListAll(context.Background(), jobs.ListJobsRequest{Limit: 10})
require.NoError(t, err)
require.Equal(t, jobs[0].JobId, int64(64))
require.Equal(t, jobs[1].JobId, int64(65))
}
2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
module github.com/databricks/databricks-sdk-go

go 1.18
go 1.19

require (
github.com/google/go-querystring v1.1.0
Expand Down
2 changes: 1 addition & 1 deletion openapi/generator/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,7 @@ func (c *Generator) Apply(ctx context.Context, batch *code.Batch, suite *roll.Su
filenames = append(filenames, pass.Filenames...)
}

mockDir := filepath.Join(c.dir, "experimental", "mocks", "service")
mockDir := filepath.Join(c.dir, "testkit", "mocks", "service")
mockFilenames := []string{}
info, err := os.Stat(mockDir)
if err == nil && info.IsDir() {
Expand Down
2 changes: 1 addition & 1 deletion qa/lock/databricks/databricks_backend_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,9 @@ import (
"testing"

"github.com/databricks/databricks-sdk-go/apierr"
"github.com/databricks/databricks-sdk-go/experimental/mocks"
"github.com/databricks/databricks-sdk-go/qa/lock/core"
"github.com/databricks/databricks-sdk-go/service/workspace"
"github.com/databricks/databricks-sdk-go/testkit/mocks"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/mock"
)
Expand Down

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

File renamed without changes.