Cannot connect to Azure DevOps git repo

[mikemowgli]

my git repo is on Azure DevOps. I tried both ssh and http, and both cli config and terraform backend config, always with the same result:

[terraform-backend-git]: empty git-upload-pack given
[terraform-backend-git]: empty git-upload-pack given
[terraform-backend-git]: empty git-upload-pack given

Error: Error loading state:
    Failed to get state: GET http://localhost:6061/?type=git&repository=git@ssh.dev.azure.com:v3/REDACTED/infra-analytics/tooling-tf&ref=state&state=tfstate.json giving up after 3 attempts

Just to make sure, I already created the branch, and the tfstate.json file, but got the same result.

Here's my config:

terraform {
  backend "http" {
    address        = "http://localhost:6061/?type=git&repository=git@ssh.dev.azure.com:v3/REDACTED/infra-analytics/tooling-tf&ref=state&state=tfstate.json"
    lock_address   = "http://localhost:6061/?type=git&repository=git@ssh.dev.azure.com:v3/REDACTED/infra-analytics/tooling-tf&ref=state&state=tfstate.json"
    unlock_address = "http://localhost:6061/?type=git&repository=git@ssh.dev.azure.com:v3/REDACTED/infra-analytics/tooling-tf&ref=state&state=tfstate.json"
  }
}

I also have the TF_BACKEND_HTTP_ENCRYPTION_PASSPHRASE env var set.

When enabling the access logs, I get

[terraform-backend-git]: ssh: handshake failed: EOF
127.0.0.1 - - [19/Oct/2020:11:24:48 +0200] "GET /?type=git&repository=git@ssh.dev.azure.com:v3/REDACTED/infra-analytics/tooling-tf&ref=state&state=tfstate.json HTTP/1.1" 500 27
[terraform-backend-git]: empty git-upload-pack given
127.0.0.1 - - [19/Oct/2020:11:25:10 +0200] "GET /?type=git&repository=git@ssh.dev.azure.com:v3/REDACTED/infra-analytics/tooling-tf&ref=state&state=tfstate.json HTTP/1.1" 500 27
[terraform-backend-git]: empty git-upload-pack given
127.0.0.1 - - [19/Oct/2020:11:25:13 +0200] "GET /?type=git&repository=git@ssh.dev.azure.com:v3/REDACTED/infra-analytics/tooling-tf&ref=state&state=tfstate.json HTTP/1.1" 500 27

[dee-kryvenko]

Thanks for your interest in this project @mikemowgli.

Briefly looking into this issue, it looks like the error comes from go-git. After a little bit of research it looks like this is due to the fact go-git does not support multi-ack (so as many other git client implementations). Azure DevOps (as well as VSTS) in turn does not support older non-multi-ack protocols. All the major competitors like GitHub/GitLab/Bitbucket does support it.

There's a number of issues being reported both in go-git as well as it's consumers, just to list the few:

• add multi-ack client support (clone broken on VisualStudio Team Services, repo returns 500 error) src-d/go-git#335
• Git Fetch does not work with Azure DevOps repos src-d/go-git#1058
• Git Clone does not work with Azure DevOps repos go-git/go-git#64
• Azure DevOps not able fetch code from Git repo buildpacks-community/kpack#213
• Azure DevOps compatibility issues fluxcd/source-controller#104
• helm-op: VSTS git repositories are not cloning fluxcd/flux#1224

It goes on and on to as early as 2017. Together with the fact that Azure DevOps/VSTS seems to be the only one who requires multi-ack, I wouldn't hope for a prompt resolution in go-git. It seems like even Go itself struggle with that: golang/go#28236

That said, Microsoft seems to be firmly refusing to relax protocol requirements: fluxcd/source-controller#104 (comment)

Theoretically it should be possible to swap out go-git by delegating to git command, but that would significantly increase complexity, introduce external dependencies as well as cause potential portability issues - so I am not sure I want to go down that route. This backend design allows for external storage implementations though, so maybe someone else will want to go for it. I'm gonna leave this issue open to not to forget to update go-git once they solve this issue. Other than that there's not much I can help with at this time. Sorry.

[dee-kryvenko]

Fun fact: looking into the GitHub repo stats you seems to be the first one who's trying to use this backend. And you happen to use Azure DevOps which this backend turns out to have compatibility issues with. What a shame.