Skip to content

Commit

Permalink
(chore): update the readme, rbac and enforcement policy (#29)
Browse files Browse the repository at this point in the history
Signed-off-by: Shubham Chaudhary <[email protected]>
  • Loading branch information
ispeakc0de authored Aug 13, 2024
1 parent 003fa7a commit 8daf11f
Show file tree
Hide file tree
Showing 8 changed files with 150 additions and 84 deletions.
102 changes: 75 additions & 27 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ A list of service accounts that are allowed to launch pods with the target servi

```yaml
- name: ALLOWED_ORIGIN_SERVICE_ACCOUNTS
value: '["litmus-admin","replicaset-controller"]'
value: '["litmus-admin"]'
```
### 2. AllowedOriginImages
Expand Down Expand Up @@ -58,16 +58,56 @@ The `SELF_MANAGED_DEPENDENCIES` environment variable controls how the TLS certif

If `SELF_MANAGED_DEPENDENCIES` is set to `false`, you must manually provide the TLS certificates:

1. **Create a Kubernetes Secret**: Create a secret containing the TLS certificate and key.
1. **Generate TLS Certificate**: Generate a TLS certificate and key for the admission controller.

a. Create the CA Certificate:
```bash
openssl req -nodes -new -x509 -keyout certs/ca.key -out certs/ca.crt -subj "/CN=admission-server-ca"
```

b. Generate the Server Key:
```bash
openssl genrsa -out certs/tls.key 2048
```

c. Generate the Certificate Signing Request (CSR) with SANs:
First, create a configuration file for the SANs:
san.cnf
```bash
[req]
distinguished_name = req_distinguished_name
req_extensions = req_ext
prompt = no
[req_distinguished_name]
CN = litmus-admission-server-service.litmus.svc
[req_ext]
subjectAltName = @alt_names
[alt_names]
DNS.1 = litmus-admission-server-service.litmus.svc
DNS.2 = litmus-admission-server-service.litmus.svc.cluster.local
```
Then, generate the CSR using this configuration:
````bash
openssl req -new -key certs/tls.key -out certs/tls.csr -config san.cnf
````
d. Sign the CSR with the CA:
```bash
openssl x509 -req -in certs/tls.csr -CA certs/ca.crt -CAkey certs/ca.key -CAcreateserial -out certs/tls.crt -days 365 -extensions req_ext -extfile san.cnf
```

2. **Create a Kubernetes Secret**: Create a secret containing the TLS certificate and key.

```bash
kubectl create secret tls admission-server-tls \
--cert=/path/to/tls.crt \
--key=/path/to/tls.key \
--cert=certs/tls.crt \
--key=certs/tls.key \
-n <namespace>
```

2. **Mount the Secret**: Mount the secret into the admission controller at the path `/etc/certs`.
3. **Mount the Secret**: Mount the secret into the admission controller at the path `/etc/certs`.

## ValidatingWebhookConfiguration

Expand All @@ -79,35 +119,43 @@ To enforce these restrictions, a `ValidatingWebhookConfiguration` is created. Th
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:
name: pod-validation
name: pod-validation
webhooks:
- name: pod-validation.default.svc
namespaceSelector:
matchExpressions:
- key: kubernetes.io/metadata.name
operator: In
values: [ "controller" ]
clientConfig:
service:
name: admission-server
namespace: default
path: "/validate/pods"
caBundle: "${CA_BUNDLE}"
rules:
- operations: ["CREATE"]
apiGroups: [""]
apiVersions: ["v1"]
resources: ["pods"]
scope: "Namespaced"
admissionReviewVersions: ["v1"]
sideEffects: None
- name: pod-validation.litmus.svc
namespaceSelector:
matchExpressions:
- key: kubernetes.io/metadata.name
operator: In
values: ["litmus"]
clientConfig:
service:
name: litmus-admission-server
namespace: litmus
path: "/validate/pods"
caBundle: "${CA_BUNDLE}"
rules:
- operations: ["CREATE", "UPDATE"]
apiGroups: [""]
apiVersions: ["v1"]
resources: ["pods"]
scope: "Namespaced"
admissionReviewVersions: ["v1"]
sideEffects: None
failurePolicy: Ignore
timeoutSeconds: 5
```

NOTE: Replace the `${CA_BUNDLE}` placeholder with the base64-encoded CA certificate.

````bash
CA_BUNDLE=$(cat certs/ca.crt | base64 | tr -d '\n')
````

### Explanation:

- **Namespace Selector**: The webhook is configured to operate only within specific namespaces, as defined by the `matchExpressions`.
- **Client Config**: Specifies the admission server service and the path to be used for validation.
- **Rules**: The webhook only triggers on `CREATE` operations for pod resources within a namespace.
- **Rules**: The webhook only triggers on `CREATE` and `UPDATE` operations for pod resources within a namespace.
- **Admission Review Versions**: Specifies the admission review versions supported by the webhook.
- **Side Effects**: Indicates that the webhook does not have side effects.

Expand Down
20 changes: 11 additions & 9 deletions deploy/deployment.yaml
Original file line number Diff line number Diff line change
@@ -1,21 +1,22 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: admission-server
name: litmus-admission-server
namespace: litmus
labels:
app: admission-server
litmuschaos.io/component-name: admission-controller
app: litmus-admission-server
litmuschaos.io/component-name: litmus-admission-server
spec:
replicas: 1
selector:
matchLabels:
app: admission-server
app: litmus-admission-server
template:
metadata:
labels:
app: admission-server
app: litmus-admission-server
spec:
serviceAccountName: admission-server
serviceAccountName: litmus-admission-server
containers:
- name: server
image: litmuschaos/admission-controller:ci
Expand All @@ -24,7 +25,7 @@ spec:
- containerPort: 8443
env:
- name: ALLOWED_ORIGIN_SERVICE_ACCOUNTS
value: '["litmus-admin","replicaset-controller"]'
value: '["litmus-admin","litmus"]'
- name: ALLOWED_ORIGIN_IMAGES
value: '["^.*/litmuschaos/go-runner:.*$","^.*/litmuschaos/chaos-operator:.*$"]'
- name: ALLOWED_TARGET_IMAGES
Expand All @@ -41,10 +42,11 @@ spec:
apiVersion: v1
kind: Service
metadata:
name: admission-server-service
name: litmus-admission-server-service
namespace: litmus
spec:
selector:
app: admission-server
app: litmus-admission-server
ports:
- port: 443
targetPort: 8443
13 changes: 7 additions & 6 deletions deploy/rbac.yaml → deploy/rbac-with-managed-dependencies.yaml
Original file line number Diff line number Diff line change
@@ -1,14 +1,15 @@
apiVersion: v1
kind: ServiceAccount
metadata:
name: admission-server
name: litmus-admission-server
namespace: litmus
labels:
app.kubernetes.io/name: litmus
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: admission-server-cluster-role
name: litmus-admission-server-cluster-role
labels:
app.kubernetes.io/name: litmus
rules:
Expand All @@ -25,14 +26,14 @@ rules:
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: admission-server-cluster-role-binding
name: litmus-admission-server-cluster-role-binding
labels:
app.kubernetes.io/name: litmus
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: admission-server-cluster-role
name: litmus-admission-server-cluster-role
subjects:
- kind: ServiceAccount
name: admission-server
namespace: controller
name: litmus-admission-server
namespace: litmus
35 changes: 35 additions & 0 deletions deploy/rbac-without-managed-dependencies.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
apiVersion: v1
kind: ServiceAccount
metadata:
name: litmus-admission-server
namespace: litmus
labels:
app.kubernetes.io/name: litmus
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: litmus-admission-server-role
namespace: litmus
labels:
app.kubernetes.io/name: litmus
rules:
- apiGroups: [ "" ]
resources: [ "pods" ]
verbs: [ "get","list" ]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: litmus-admission-server-role-binding
namespace: litmus
labels:
app.kubernetes.io/name: litmus
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: litmus-admission-server-role
subjects:
- kind: ServiceAccount
name: litmus-admission-server
namespace: litmus
8 changes: 4 additions & 4 deletions deploy/webhook.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,20 +3,20 @@ kind: ValidatingWebhookConfiguration
metadata:
name: pod-validation
webhooks:
- name: pod-validation.default.svc
- name: pod-validation.litmus.svc
namespaceSelector:
matchExpressions:
- key: kubernetes.io/metadata.name
operator: In
values: ["litmus"]
clientConfig:
service:
name: admission-server
namespace: default
name: litmus-admission-server-service
namespace: litmus
path: "/validate/pods"
caBundle: "${CA_BUNDLE}"
rules:
- operations: ["CREATE"]
- operations: ["CREATE", "UPDATE"]
apiGroups: [""]
apiVersions: ["v1"]
resources: ["pods"]
Expand Down
40 changes: 5 additions & 35 deletions internal/pods/restrict.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,9 @@ import (
)

const (
podNameKey = "authentication.kubernetes.io/pod-name"
jobControllerServiceAccount = "system:serviceaccount:kube-system:job-controller"
podNameKey = "authentication.kubernetes.io/pod-name"
jobControllerServiceAccount = "system:serviceaccount:kube-system:job-controller"
controllerServiceAccountPrefix = "system:serviceaccount:kube-system"
)

func validateOriginServiceAccount(serviceAccount string) (bool, string) {
Expand Down Expand Up @@ -44,8 +45,8 @@ func validateOriginImage(serviceAccount string, pod *corev1.Pod, extras map[stri
return true, ""
}

if serviceAccount == jobControllerServiceAccount {
return validateOriginJobImage(pod, clients)
if strings.Contains(serviceAccount, controllerServiceAccountPrefix) {
return true, ""
}

return validateOriginPodImage(pod.Namespace, extras, clients)
Expand Down Expand Up @@ -77,37 +78,6 @@ func validateOriginPodImage(namespace string, extras map[string]v1.ExtraValue, c
return false, ""
}

func validateOriginJobImage(pod *corev1.Pod, clients clients.ClientSets) (bool, string) {
jobName := getJobName(pod)
if jobName == "" {
return false, fmt.Sprintf("could not get origin job name")
}

job, err := clients.KubeClient.BatchV1().Jobs(pod.Namespace).Get(context.Background(), jobName, metav1.GetOptions{})
if err != nil {
return false, fmt.Sprintf("could not get origin job: %v", err)
}

for _, c := range job.Spec.Template.Spec.Containers {
for _, v := range utils.WebHookFilters.AllowedOriginImages.AllowedList {
if utils.MatchRegex(v, c.Image) {
return true, ""
}
}
}

return false, ""
}

func getJobName(pod *corev1.Pod) string {
for _, owner := range pod.OwnerReferences {
if owner.Kind == "Job" {
return owner.Name
}
}
return ""
}

func originFromTerminal(serviceAccount string) bool {
if strings.Contains(serviceAccount, "system:serviceaccount") {
return false
Expand Down
5 changes: 2 additions & 3 deletions internal/webhook/wehook.go
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ import (
)

const (
controllerLabel = "litmuschaos.io/component-name=admission-controller"
controllerLabel = "litmuschaos.io/component-name=litmus-admission-server"
)

func ManageDependencies(clients clients.ClientSets) (*tls.Certificate, error) {
Expand Down Expand Up @@ -61,7 +61,6 @@ func createValidator(serviceName string, signingCert []byte, clients clients.Cli
sideEffect = v1.SideEffectClassNone
validatePodsPath = "/validate/pods"
timeout = int32(5)
failPolicy = v1.Ignore
)

webhookHandler := v1.ValidatingWebhook{
Expand All @@ -73,7 +72,7 @@ func createValidator(serviceName string, signingCert []byte, clients clients.Cli
},
AdmissionReviewVersions: []string{"v1"},
SideEffects: &sideEffect,
FailurePolicy: &failPolicy,
FailurePolicy: &utils.WebHookFilters.EnforcementPolicy,
Rules: []v1.RuleWithOperations{{
Operations: []v1.OperationType{
v1.Create,
Expand Down
Loading

0 comments on commit 8daf11f

Please sign in to comment.