tidy: check for invalid file names

[samueltardieu]

Check for file names added to git with:

• non-UTF8 filenames (this would fail "fmt check" with a decoding error for the moment, but maybe we should not count on it as it is an accidental failure)
• control characters (such as "\n" or "\r" in file names)
• ":" (which is a special character on Windows, made rustdoc-json: Structured attributes #142936 fail in bors while it could have be caught earlier)

It only checks files known by git as a developer might want to have "strange" file names alongside their local repository as long as they don't check them in.

r? jieyouxu
as he stumbled upon such a file in #142936

[rustbot]

jieyouxu is currently at their maximum review capacity.
They may take a while to respond.

[rustbot]

There are changes to the tidy tool.

cc @jieyouxu

[Kobzol]

Just out of curiosity, how long does it take to run this on your machine?

[samueltardieu]

To run git ls-files in my working directory, around 16ms wall time. Timing x test tidy gives no perceivable difference in walltime (4.22s vs. 4.26s, but it can vary in both directions – 24 virtual cores laptop):

with this PR:

Executed in    4.26 secs    fish           external
   usr time   14.15 secs    3.70 millis   14.15 secs
   sys time    5.47 secs    1.12 millis    5.47 secs

without this PR:

Executed in    4.22 secs    fish           external
   usr time   14.35 secs    0.79 millis   14.35 secs
   sys time    5.40 secs    1.13 millis    5.40 secs

[Kobzol]

Okay, that should be fine. Hopefully it won't be sluggish e.g. on Windows, but I think that tidy does so much I/O anyway that this won't be bad.

[samueltardieu]

Yeah, since it uses only file names and not even file contents, there is a good chance that the cache is primed already (or that this primes it for other parts of tidy). If it gets r+, maybe rollup=never would make it easier to roll it back in case it is heavier than expected on some platforms.

[Kobzol]

I think it will be fine, I realized that it won't even read all the filenames from disk, which I first wrongly assumed, just return the filenames from the internal git DB, that should be fine.

The git -C argument wouldn't work on some of our CI runners with old OS, but I think these should be just the dist builders that don't run tests anyway. Let's see.

@bors r+

[bors]

📌 Commit cbaaf15 has been approved by Kobzol

It is now in the queue for this repository.