Sourcing module with git by cloning in shallow mode

[samber]

Hi there,

We have a heavy git repository with a lot of Terraform modules. When we execute a terraform get it take ~20min to download dependencies (a git clone for each module call !).

We removed from the repository some binary files on the master branch. But unfortunately, nothing changed because Terraform do a git clone instead of git clone --depth 1.

[in4mer]

Related #14036

[clumsysnake]

also having this issue.

[scottconnerly]

Related #19277

[czerasz]

Currently using depth=1 together ref parameter will not always work.

Example main.tf:

module "test" {
  source="git::git@gitlab.com:czerasz/terraform-aws-container-override.git?depth=1&ref=tags/v0.1.0"
}

The terraform init for this configuration returns:

...
Error: Failed to download module

Could not download module "test" (main.tf:1) source code from
"git::git@gitlab.com:czerasz/terraform-aws-container-override.git?depth=1&ref=tags/v0.1.0":
error downloading
'ssh://git@gitlab.com/czerasz/terraform-aws-container-override.git?depth=1&ref=tags%2Fv0.1.0':
/usr/bin/git exited with 1: error: pathspec 'tags/v0.1.0' did not match any
file(s) known to git.

This is because Terraform (through the hashicorp/go-getter library) does the following:

• clone the repo with the --depth 1 flag:

  /usr/bin/git clone --depth 1 ssh://git@gitlab.com/czerasz/terraform-aws-container-override.git .terraform/modules/test

• then it tries to checkout the 0.1.0 tag:

  /usr/bin/git checkout tags/v0.1.0

The problem is that by using --depth 1 git does NOT "see" all available tags/branches:

$ cd .terraform/modules/test 
$ git tag | cat 
v0.1.1

[in4mer]

Please correct me if I'm wrong, but your question doesn't make sense.. tags are treeish objects. You've checked out a shallow copy of the repo, meaning you have no chunks besides those used to comprise the currently specified commit you've checked out. So in essence, that means you have no history besides "now."

Ah, yeah, I see. You've asked for tag=123 with a depth=1, and gotten tag=125, and of course no other tags. Have you tried reversing the URL query fields, just as a test?

[czerasz]

@in4mer I tried reversing the fields, but it's not relevant.

The issue is, as You mentioned, there is no history besides "now."