Skip to content

Creating test or performance reproducers

Jump to bottom
Paul Jolly edited this page Jan 4, 2021 · 7 revisions

When you run into an issue with CUE, where possible reducing this down to a simple example helps others who have no context quickly understand where the issue might be.

But sometimes that's simply not possible: it's too complex to reduce, spread across multiple files, etc. In such cases, providing a means by which someone can easily and precisely reproduce your setup locally on their machine is critical. These details should be included in a GitHub bug report.

Here are two ways in which you can provide such steps:

Sequences of shell commands

One way is to provide a sequence of shell commands that can be copy-pasted locally. For example:

cd $(mktemp -d)
(
set -e
git clone https://github.com/play-with-go/preguide
cd preguide
git checkout 9dcd6cce2ddbe9e8d44649815bb6916830da8cb7
cue def
)

Points to note from this approach:

  • the entire block above can be copy-pasted an run locally - no need to remove any shell prompt text like $
  • the reproduction is run within a new temporary directory, cd $(mktemp -d)
  • the main command block is run in a subshell, denoted by the outer parentheses. set -e causes the subshell to exit on any subsequent command errors
  • we git checkout to a specific commit so anyone investigating can be sure of running exactly the same code (otherwise we can't be sure we're not running another commit, because branch references like main/master etc move)
  • the version of cue is given by the GitHub bug report issue template (which should also be completed)

Best practice is to include the output from copy-pasting this same block locally on your machine. That way you can be sure the reproduction is a genuine reproduction.

Creating a txtar archive

Another way is to provide an entirely hermetic reproduction file in the form of a txtar archive. txtar is a trivial text-based file archive format. You can use txtar-c to create a txtar file from an existing directory structure of files.

To quickly demonstrate the use of txtar-c, let's create a setup locally which contains a problem we want to report:

cd $(mktemp -d)
(
set -e
cue mod init mod.com
cat <<EOD > x.cue
package x

a: 41
a: 42
EOD
)

If we now run txtar-c we get the following output:

-- cue.mod/module.cue --
module: "mod.com"
-- x.cue --
package x

a: 41
a: 42

Each file in the archive is delineated by -- filename -- blocks. You can optionally put the commands you ran above the first file in the archive:

cue def

-- cue.mod/module.cue --
module: "mod.com"
-- x.cue --
package x

a: 41
a: 42

Alternatively, simply include the command(s) run in a separate part of the bug report.

Given the above report (assuming the bug report told use we should test against cue v0.2.2) we should see the output:

a: conflicting values 41 and 42:
    ./x.cue:3:4
    ./x.cue:4:4

Someone investigating the problem can then trivially run txtar-x to do the reverse, expanding this archive, at which point they will have an identical local setup and should be able to reproduce the problem.

Clone this wiki locally