Cloud Code develop Cloud Run locally

[Kevintjeb]

Expected Behavior

Running locally starts the Cloud Run container

Actual Behavior

Failure when preparing Google Cloud SDK

Additional Information

• IDE type : IntelliJ 2020.2.1
• Cloud Code version: 20.8.2-202
• Cloud SDK (Settings > Cloud Code > Cloud SDK)
  • Are you allowing the plugin to manage the Cloud SDK: Yes
  • Version of the Cloud SDK: 307.0.0
• Operating System: MacOs Cataline 10.15.6 (19G2021)

I've found that when using my local gcloud sdk, it doens't have the 'setting up gcloud sdk', but it doesn't do anything either. It hangs forever..

[fsladkey]

Hey there @Kevintjeb, thanks for the report!

Sounds like a bug to me. I'm going to do some preliminary investigation and get back to you shortly with some troubleshooting steps.

If by any chance you have a minimal version of the project to reproduce the problem that you're comfortable sharing with us we'd be happy to troubleshoot with you that way. If not, no problem I understand.

Hope to have this resolved for you soon!

[fsladkey]

To investigate the issue with the managed SDK, would you mind running the following command in your project directory and seeing if it succeeds, and if not how it fails? This is the command which appears to be failing in your original issue (and will use the Cloud Code managed SDK).

"/Users/<YOUR_USER_HERE>/Library/Application Support/google-cloud-tools-java/managed-cloud-sdk/LATEST/google-cloud-sdk/bin/gcloud" alpha code export \
    --service-name dispatcher \
    --image dispatcher \
    --no-skaffold-file \
    --kubernetes-file ./export-test.yaml \
    --env-vars K_SERVICE=dispatcher,K_REVISION=local,K_CONFIGURATION=dispatcher \
     --memory 256MiB

If the command succeeds, it will create a yaml file in the location specified by --kubernetes-file, feel free to change the location.

As for the custom SDK hanging, I'm able to reproduce that, and I believe it is likely because the alpha components have not been installed. So another thing to try is to run gcloud components install alpha, then try again with your custom SDK. I'll file a bug to improve the error handling here regardless of whether that's the issue you're running into.

[Kevintjeb]

Thanks for the quick, extended response wow!

The supplied command works and generates :

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    service: dispatcher
  name: dispatcher
spec:
  replicas: 1
  selector:
    matchLabels:
      app: dispatcher
  template:
    metadata:
      labels:
        app: dispatcher
    spec:
      containers:
      - env:
        - name: PORT
          value: '8080'
        - name: K_CONFIGURATION
          value: dev
        - name: K_REVISION
          value: dev-0001
        - name: K_SERVICE
          value: dispatcher
        - name: K_CONFIGURATION
          value: dispatcher
        - name: K_REVISION
          value: local
        - name: K_SERVICE
          value: dispatcher
        image: dispatcher
        name: dispatcher-container
        ports:
        - containerPort: 8080
        resources:
          limits:
            memory: 268435456
      terminationGracePeriodSeconds: 0
---
apiVersion: v1
kind: Service
metadata:
  name: dispatcher
spec:
  ports:
  - port: 8080
    protocol: TCP
    targetPort: 8080
  selector:
    app: dispatcher
  type: LoadBalancer

However, the cloud code still fails with the same error. After I installed the alpha components I now also have the same error with the custom SDK.

Using Cloud Code in VsCode I received the following error:

,* Failed to start hyperkit VM. "minikube start -p cloud-run-dev-internal" may fix it: creating host: create: Error with pre-create check: "docker-machine-driver-hyperkit needs to run with elevated permissions. Please run the following command, then try again: sudo chown root:wheel /Users/kvandenbroek/.minikube/bin/docker-machine-driver-hyperkit && sudo chmod u+s /Users/kvandenbroek/.minikube/bin/docker-machine-driver-hyperkit"
,* 
,X error provisioning host: Failed to start host: creating host: create: Error with pre-create check: "docker-machine-driver-hyperkit needs to run with elevated permissions. Please run the following command, then try again: sudo chown root:wheel /Users/kvandenbroek/.minikube/bin/docker-machine-driver-hyperkit && sudo chmod u+s /Users/kvandenbroek/.minikube/bin/docker-machine-driver-hyperkit"
* 
* minikube is exiting due to an error. If the above message is not useful, open an issue:
  - https://github.com/kubernetes/minikube/issues/new/choose

Running the command in the error solves that, however there is still some troubles with starting minikube. So that might be related, but not because of cloud code?

it created the following launch.json

{
    "configurations": [
        {
            "name": "Cloud Run: Run/Debug Locally",
            "type": "cloudcode.cloudrun",
            "request": "launch",
            "build": {
                "docker": {
                    "path": "cloud-run/dispatcher/Dockerfile"
                }
            },
            "image": "dispatcher",
            "service": {
                "name": "dispatcher",
                "containerPort": 8080,
                "resources": {
                    "limits": {
                        "memory": "256Mi"
                    }
                }
            },
            "target": {
                "minikube": {}
            },
            "watch": true
        }
    ]
}

EDIT:

So I think I tracked it down to Minikube (IntelliJ still has the error with spec generation though, but VsCode goes to start minikube etc). I'm on Cisco Anyconnect VPN and that messes with Minikube. I can't find any documentation on how to setup all dependencies for the Cloud Code Cloud Run locally feature, is there any?

[fsladkey]

Thanks for the quick, extended response wow!

No problem, happy to help, and appreciate you reporting the issue so we can improve the feature. :)

So I think I tracked it down to Minikube (IntelliJ still has the error with spec generation though, but VsCode goes to start minikube etc). I'm on Cisco Anyconnect VPN and that messes with Minikube.

Hmm interesting. Do you have any proxy settings specified for IntelliJ? They're in Preferences -> System Settings -> HTTP Proxy.

I can't find any documentation on how to setup all dependencies for the Cloud Code Cloud Run locally feature, is there any?

Ideally, no additional set up is required if you're using the managed dependencies, which is why it isn't documented. Unfortunately they're not behaving as expected. I found this minikube issue which seems to be the same as some of the output you're seeing, so it may be worth trying some of the suggestions from that thread.

Because the command to generate the deployment/service spec succeeded when run in isolation it's not clear right now why it's failing, but I'll take another look and see if I can find another way to troubleshoot it.

[Kevintjeb]

I don't have proxy settings enabled for intelliJ, so that shouldn't interfere.

The main problem seems to be minikube with the vpn, but I simply can't disable the VPN and I've tried all kinds of different solutions for the VPN problem. None of them worked..

It does work like a charm when using VSCode and no VPN! I'd prefer to use IntelliJ, and VPN though :)

So for this project/issue it might be worthwhile to try and work together on getting the IntelliJ part working, up until it deploys to minikube/vpn?

[fsladkey]

Hey @Kevintjeb thanks for the info!

To summarize where we're at, sounds like there's still two problems.

1): Minikube access with the VPN turned on - You can try to see if any of these solutions work for you, in which case you should be able to proceed in VSCode. Example 2 shows specifics for Cisco VPN, so hopefully that will be helpful.

2): Cloud Code fails to create pods and services - we're a little bit stuck on this one since running the command manually succeeded. We're going to improve the error reporting there, and if you're up for it once it's part of our nightly build (probably tomorrow) you can download the nightly build of the plugin, and while it probably won't fix your issue, it should give us a clearer error to continue troubleshooting.

Thanks for your detailed responses, really appreciate it and hope we can get this all working for you. 👍

[fsladkey]

Just following up, the improved error reporting is available on the nightly channel now. If you'd like to try it, the installation instructions are available here.

[Kevintjeb]

Hi,

I've looked at the nightly build, and the error is pretty clear. It seems like the cloud code plugins doesn't honor Dockerfile location configuration:

ERROR: (gcloud.alpha.code.export) path/go/src/gitlab.bol.io/mirage/fox-public/Dockerfile does not exist.

While in the cloud code config I've got the following:

Where /cloud-function is a dir in /fox-public/

Tree is:

fox-public
    cloud-function
       dispatcher
          code files
          Dockerfile

[fsladkey]

Hey there, @Kevintjeb Great, that's certainly a much clearer error message. I just tried it out myself and I'm seeing the same behavior. I'll file a bug for that and update this thread when it's resolved.

Thanks for your patience in helping find these issues, really appreciated!

[j-frost]

I had this same issue but also found it worked better once I turned the Jetbrains Kubernetes plugin off.

However now I'm getting <projectPath>/Dockerfile does not exist. - which is true, because my Dockerfile is somewhere further down in the file hierarchy. I tried changing it in the UI but it looks correct there - I guess it's auto detected?

[SUSTAPLE117]

I get the same issue when Dockerfiles are in subfolders. I guess the build context stays at the root folder even when pointing to a Dockerfile in a subdirectory.

[etanshaul]

thanks for the extra details @j-frost, @SUSTAPLE117 . Looks like we need to revisit this.

@fsladkey did your initial investigation lead to any workarounds that can be used in the meantime? Do we know if this is a Skaffold limitation or a Cloud Code limitation?

[fsladkey]

Hey folks! Thanks again for additional details!

When I was looking into this a few months ago and was still running into issues, however I just tried again this today and I think I've found a solution. Looks like we need to specify the Dockerfile location when calling gcloud alpha code export and I believe that should resolve the problem.

Unfortunately, in the meantime I can't think of a good workaround other than opening the directory containing the Dockerfile and running from there. However, I'm pretty hopeful we can get this fix in for the next release.

[etanshaul]

@fsladkey nice find! that makes sense. We can let this group know once the fix is merged in and in the nightly release for those eager to try it before the next release (early December).

[fsladkey]

Closing as this issue is fixed and released.