use_coverage causes "Can only merge Directories with no duplicates" error for Go packages with _test packaged files

[nickbruun]

Describe the bug
When use_coverage is enabled, Go packages that have <package>_test packaged test files (ie. for the package foo, the tests are in the same folder but are in the foo_test package and import foo, see reproduction repository for an example) causes an IntrinsicError of the following shape when the test goal is run:

IntrinsicError: Can only merge Directories with no duplicates, but found 2 duplicate entries in __pkgs__/foo_pkg_repro:

`__pkg__.a`: 1.) file digest=58ba11fe1afd8ddcccdd2a69c32f391ab4314b10430c13dfc8e00501f2368b5f size=1960:

<snip>

`__pkg__.a`: 2.) file digest=085a4521ca0c1cca011a41fc59dd11487677270df92568eead5cd16548eeb5da size=8798:

<snip>

A simple reproduction case can be found at nickbruun/pants-use-coverage-repro. The repository has two commits:

1. nickbruun/pants-use-coverage-repro@c3ca0a0: With use_coverage enabled. Failure can be seen in GitHub Actions log
2. nickbruun/pants-use-coverage-repro@0b7a49d: With use_coverage disabled. The successful test run can be seen in GitHub Actions log.

Pants version
2.21.1, but confirmed at least as far back as 2.17 (I just hadn't gotten around to reporting it until now -- sorry!)

OS
Reproduced on Linux and MacOS.

Additional info
See nickbruun/pants-use-coverage-repro#1 for evidence that if the test has the non-_test package, the test goal succeeds.

The following addition to src/python/pants/backend/go/goals/test_test.py will reproduce the issue in tests:

def test_external_test_with_use_coverage(rule_runner: RuleRunner) -> None:
    rule_runner.write_files(
        {
            "foo/BUILD": "go_mod(name='mod')\ngo_package()",
            "foo/go.mod": "module foo",
            "foo/add.go": textwrap.dedent(
                """
                package foo
                func Add(x, y int) int {
                  return x + y
                }
                """
            ),
            "foo/add_test.go": textwrap.dedent(
                """
                package foo_test
                import (
                  "foo"
                  "testing"
                )
                func TestAdd(t *testing.T) {
                  if foo.Add(2, 3) != 5 {
                    t.Fail()
                  }
                }
                """
            ),
        }
    )
    tgt = rule_runner.get_target(Address("foo", generated_name="./"))
    rule_runner.set_options(
        [
            "--test-use-coverage",
        ],
        env_inherit={"PATH"},
    )
    result = rule_runner.request(
        TestResult, [GoTestRequest.Batch("", (GoTestFieldSet.create(tgt),), None)]
    )
    assert result.exit_code == 0

[huonw]

Sorry for the trouble and thanks for the reproducer and test case!

Is this something you're interested in trying to fix?

[nickbruun]

I'll likely give it a swing some evening when I have a minute, but if someone has any quick pointers as to why duplicate libraries are being built, I'm all ears!

[huonw]

Great. Maybe @tdyas or @tgolsson might have a hint or two? (I'm not personally familiar with the Go backend or Go in general.)

[tdyas]

What version of Go are you using?

[nickbruun]

@tdyas I reproduced this in 1.21, 1.22 and now 1.23.