Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

posix filesystem call test suite based on pjdfstest #104

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

sequix
Copy link

@sequix sequix commented May 28, 2020

#101 covers performance test, this PR covers posix calls with (pjd/pjdfstest)[https://github.com/pjd/pjdfstest]. It uses perl TAP-Harness to mock posix calls and its test case is written in script like.

desc="rename returns ENAMETOOLONG if a component of either pathname exceeded {NAME_MAX} characters"

dir=`dirname $0`
. ${dir}/../misc.sh

echo "1..8"

n0=`namegen`
nx=`namegen_max`
nxx="${nx}x"

expect 0 create ${nx} 0644
expect 0 rename ${nx} ${n0}
expect 0 rename ${n0} ${nx}
expect 0 unlink ${nx}

expect 0 create ${n0} 0644
expect ENAMETOOLONG rename ${n0} ${nxx}
expect 0 unlink ${n0}
expect ENAMETOOLONG rename ${nxx} ${n0}

You can see this will create files on the rw layer, but I still got some errors, At first I thought this is about the container environment, but the errors disappear when I switched to oci image.

Test Summary Report
-------------------
/work/pjdfstest/tests/chmod/02.t          (Wstat: 0 Tests: 5 Failed: 1)
  Failed test:  5
/work/pjdfstest/tests/chown/02.t          (Wstat: 0 Tests: 10 Failed: 2)
  Failed tests:  5, 10
/work/pjdfstest/tests/ftruncate/02.t      (Wstat: 0 Tests: 5 Failed: 1)
  Failed test:  5
/work/pjdfstest/tests/link/02.t           (Wstat: 0 Tests: 10 Failed: 1)
  Failed test:  10
/work/pjdfstest/tests/mknod/02.t          (Wstat: 0 Tests: 12 Failed: 1)
  Failed test:  12
/work/pjdfstest/tests/rename/01.t         (Wstat: 0 Tests: 8 Failed: 1)
  Failed test:  8
/work/pjdfstest/tests/rmdir/02.t          (Wstat: 0 Tests: 4 Failed: 1)
  Failed test:  4
/work/pjdfstest/tests/truncate/02.t       (Wstat: 0 Tests: 5 Failed: 1)
  Failed test:  5
/work/pjdfstest/tests/unlink/02.t         (Wstat: 0 Tests: 4 Failed: 1)
  Failed test:  4
Files=235, Tests=8813, 220 wallclock secs ( 2.12 usr  0.60 sys + 25.95 cusr 36.24 csys = 64.91 CPU)
Result: FAIL

Interpretation of above log:

/work/pjdfstest/tests/chown/02.t          (Wstat: 0 Tests: 10 Failed: 2)
  Failed tests:  5, 10

Correspond to the 5th and 10th expect statement of file https://github.com/pjd/pjdfstest/blob/master/tests/chown/02.t. After exmaine all failed tests, they all expect ENAMETOOLONG.

Maybe later we can build tests in ro layer based on this.

@fuweid
Copy link
Member

fuweid commented May 28, 2020

@sequix please sign off thanks

@sequix
Copy link
Author

sequix commented May 28, 2020

@sequix please sign off thanks

signed

Copy link
Member

@ktock ktock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this! How do you think about ltp(Linux Test Project)?

# basic packages
RUN apt-get update -y && \
apt-get --no-install-recommends install -y libbtrfs-dev libseccomp-dev fuse \
apt-transport-https gnupg2 software-properties-common
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need them?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean apt-transport-https gnupg2 software-properties-common.

set -euo pipefail

# Check Dockerfile at script/posix/Dockerfile
IMAGE_PJDFSTEST="${IMAGE_PJDFSTEST:-docker.io/sequix/pjdfstest:v1-stargz}"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll also upload it when we merge it.

Comment on lines +86 to +87
ctr-remote run --rm --snapshotter=stargz "$IMAGE_PJDFSTEST" "$containerID" >/output || \
echo -e "\e[91mPosix test failed!\e[0m"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should propagate the exit code to the caller scripts (test.sh) so that Github Actions can know the success/failure of this test.

@fuweid
Copy link
Member

fuweid commented May 28, 2020

@sequix thanks! but we need your real name :P

@sequix
Copy link
Author

sequix commented Jun 1, 2020

Thanks for this! How do you think about ltp(Linux Test Project)?

Looks like the ltp community is pretty active, not like pjdfstest. I will git ltp a try.

@AkihiroSuda
Copy link
Member

Is this still on plan?

@AkihiroSuda AkihiroSuda marked this pull request as draft August 26, 2021 01:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants