Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chaos experiment container-kill hepler pod throws error unknown service runtime.v1alpha2.RuntimeService by helper pod. #4064

Closed
bmanikantareddy1991 opened this issue Jul 5, 2023 · 6 comments
Labels

Comments

@bmanikantareddy1991
Copy link

What happened:
while running the contianer kill experiment , the hepler pod failed to kill the container. Review the below logs.
k logs container-kill-helper-zzvsvi -n litmus time="2023-07-05T13:36:45Z" level=info msg="Helper Name: container-kill" time="2023-07-05T13:36:45Z" level=info msg="[PreReq]: Getting the ENV variables" time="2023-07-05T13:36:46Z" level=info msg="container ID of carts container, containerID: ce18a09b8742f5202f1325602f3d6a9b61d647b47c3b47e920a595b2c6d7d97f" time="2023-07-05T13:36:46Z" level=info msg="[Info]: Details of application under chaos injection" RestartCountBefore=0 PodName=carts-588d45d467-cxhfz ContainerName=carts time="2023-07-05T13:36:46Z" level=fatal msg="helper pod failed, err: Unable to run command, err: exit status 1; error output: time=\"2023-07-05T13:36:46Z\" level=fatal msg=\"Stopping the container \\\"ce18a09b8742f5202f1325602f3d6a9b61d647b47c3b47e920a595b2c6d7d97f\\\" failed: rpc error: code = Unimplemented desc = unknown service runtime.v1alpha2.RuntimeService\"\n"

What you expected to happen:
the helper should able to kill the container and pod should recover to its original state

Where can this issue be corrected? (optional)

How to reproduce it (as minimally and precisely as possible):
Runing choas experiment on OCI OKE Server Version: v1.26.2. with CRIO as runtime.
the same issue seen in multiple versions.

Anything else we need to know?:

@bmanikantareddy1991
Copy link
Author

@bmanikantareddy1991
Copy link
Author

Can you please help me on this @hrishavjha

@bmanikantareddy1991
Copy link
Author

bmanikantareddy1991 commented Jul 13, 2023

@hrishavjha for the above case when I take the latest image (go-runner) ,
later I was facing some other issue with litmuschaos/litmus-checker image

when I pull the litmuschaos/litmus-checker image and pushed to my private repository and utilising it from the internal repo. I am getting the below error.

"test.com/litmuschaos/litmus-checker:3.0.0-beta8", you must either explicitly specify the command, or list the image's command in the index: https://argoproj.github.io/argo-workflows/workflow-executors/#emissary-emissary: GET https://test.com/v2/oalprod/litmuschaos/litmus-checker/manifests/3.0.0-beta8: DENIED: Anonymous users are only allowed read access on public repos; map[reqId:30e13a018fe865580781b3fe]

When I try to use the same image from docker hub its working. but we are not allowed to use the images from docker hub due to rate limit.

I am using argoworkflow to submit the chaos experiment to kubernetes. In this case why we are not able to see them in litmus UI.

@Calvinaud
Copy link
Contributor

Hello,

I aslo have this problem related to the runtime.
It's the 1.26 of CRI-O, the support for v1alpha2 of the runtime was removed: cri-o/cri-o#6347
It may be the same problem you can have with some version of crictl: kubernetes-sigs/cri-tools#1130

@Calvinaud
Copy link
Contributor

In the custom hardened alpine use as base image for the go-runner it's using the version 1.16: https://github.com/litmuschaos/test-tools/blob/master/custom/hardened-alpine/experiment/Dockerfile#L54C14-L54C14

Which as explain is related to this issue: kubernetes-sigs/cri-tools#1130

@ksatchit
Copy link
Member

ksatchit commented Jan 2, 2024

Fixed with litmuschaos/test-tools#419. Please re-open if you are still facing this issue in the latest version of litmus.

@ksatchit ksatchit closed this as completed Jan 2, 2024
@ksatchit ksatchit added kind/bug and removed bug labels Jan 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants