Fix all of the following violations that were found against the API server:-

 a. Ensure that the RotateKubeletServerCertificate argument is set to true.

apiVersion: v1

kind: Pod

metadata:

creationTimestamp: null

labels:

component: kubelet

tier: control-plane

name: kubelet

namespace: kube-system

spec:

containers:

- command:

- kube-controller-manager

+ - --feature-gates=RotateKubeletServerCertificate=true

image: gcr.io/google_containers/kubelet-amd64:v1.6.0

livenessProbe:

failureThreshold: 8

httpGet:

host: 127.0.0.1

path: /healthz

port: 6443

scheme: HTTPS

initialDelaySeconds: 15

timeoutSeconds: 15

name: kubelet

resources:

requests:

cpu: 250m

volumeMounts:

- mountPath: /etc/kubernetes/

name: k8s

readOnly: true

- mountPath: /etc/ssl/certs

name: certs

- mountPath: /etc/pki

name: pki

hostNetwork: true

volumes:

- hostPath:

path: /etc/kubernetes

name: k8s

- hostPath:

path: /etc/ssl/certs

name: certs

- hostPath:

path: /etc/pki

name: pki

  b. Ensure that the admission control plugin PodSecurityPolicy is set.

audit: "/bin/ps -ef | grep $apiserverbin | grep -v grep"

tests:

test_items:

- flag: "--enable-admission-plugins"

compare:

op: has

value: "PodSecurityPolicy"

set: true

remediation: |

Follow the documentation and create Pod Security Policy objects as per your environment.

Then, edit the API server pod specification file $apiserverconf

on the master node and set the --enable-admission-plugins parameter to a

value that includes PodSecurityPolicy :

--enable-admission-plugins=...,PodSecurityPolicy,...

Then restart the API Server.

scored: true

c. Ensure that the --kubelet-certificate-authority argument is set as appropriate.

audit: "/bin/ps -ef | grep $apiserverbin | grep -v grep"

tests:

test_items:

- flag: "--kubelet-certificate-authority"

set: true

remediation: |

Follow the Kubernetes documentation and setup the TLS connection between the

apiserver and kubelets. Then, edit the API server pod specification file

$apiserverconf on the master node and set the --kubelet-certificate-authority

parameter to the path to the cert file for the certificate authority.

--kubelet-certificate-authority=

scored: true

Fix all of the following violations that were found against the ETCD:-

a. Ensure that the --auto-tls argument is not set to true

Edit the etcd pod specification file $etcdconf on the master

node and either remove the --auto-tls parameter or set it to false.

--auto-tls=false

b. Ensure that the --peer-auto-tls argument is not set to true

Edit the etcd pod specification file $etcdconf on the master

node and either remove the --peer-auto-tls parameter or set it to false.

--peer-auto-tls=false

Below is the CKS / CKA exam-style, exact step-by-step solution for Upgrading a kubeadm worker node.

Follow in order, type exact commands, no extra actions.

QUESTION — Upgrade node compute-0 (EXAM MODE)

1) Connect to the correct host (control plane)

ssh cks000034

sudo -i

export KUBECONFIG=/etc/kubernetes/admin.conf

2) Identify the control plane Kubernetes version

This is the target version for compute-0.

kubectl get nodes

Example output:

NAME STATUS ROLES VERSION

control-plane Ready control-plane v1.27.4

compute-0 Ready v1.26.6

???? Note the control-plane version

Example: v1.27.4

3) Drain the compute node (do NOT modify workloads manually)

kubectl drain compute-0 --ignore-daemonsets --delete-emptydir-data

Wait until drain completes successfully.

4) SSH into the compute node

ssh compute-0

sudo -i

5) Check current kubeadm version on compute node

kubeadm version

6) Upgrade kubeadm to match control plane version

Replace 1.27.4 with the exact control-plane version you observed.

apt-get update

apt-get install -y kubeadm=1.27.4-00

Verify:

kubeadm version

7) Run kubeadm upgrade for the node

kubeadm upgrade node

✅ This updates node-specific configs (NO workloads touched).

8) Upgrade kubelet and kubectl to the same version

apt-get install -y kubelet=1.27.4-00 kubectl=1.27.4-00

9) Restart kubelet

systemctl daemon-reload

systemctl restart kubelet

systemctl status kubelet --no-pager

10) Exit the compute node (IMPORTANT)

exit

11) Uncordon the compute node (back on control plane)

kubectl uncordon compute-0

12) Final verification

kubectl get nodes

Expected:

NAME STATUS VERSION

compute-0 Ready v1.27.4

ssh-add ~/.ssh/tempprivate

eval "$(ssh-agent -s)"

cd contrib/terraform/aws

vi terraform.tfvars

terraform init

terraform apply -var-file=credentials.tfvars

ansible-playbook -i ./inventory/hosts ./cluster.yml -e ansible_ssh_user=core -e bootstrap_os=coreos -b --become-user=root --flush-cache -e ansible_user=core

apiVersion: networking.k8s.io/v1

kind: NetworkPolicy

metadata:

name: network-policy

spec:

podSelector: {} #selects all the pods in the namespace deployed

policyTypes:

- Ingress

ingress:

- ports: #in input traffic allowed only through 80 port only

- protocol: TCP

port: 80

API server:

Ensure the --authorization-mode argument includes RBAC

Turn on Role Based Access Control.

Role Based Access Control (RBAC) allows fine-grained control over the operations that different entities can perform on different objects in the cluster. It is recommended to use the RBAC authorization mode.

Fix - BuildtimeKubernetesapiVersion: v1

kind: Pod

metadata:

creationTimestamp: null

labels:

component: kube-apiserver

tier: control-plane

name: kube-apiserver

namespace: kube-system

spec:

containers:

- command:

+ - kube-apiserver

+ - --authorization-mode=RBAC,Node

image: gcr.io/google_containers/kube-apiserver-amd64:v1.6.0

livenessProbe:

failureThreshold: 8

httpGet:

host: 127.0.0.1

path: /healthz

port: 6443

scheme: HTTPS

initialDelaySeconds: 15

timeoutSeconds: 15

name: kube-apiserver-should-pass

resources:

requests:

cpu: 250m

volumeMounts:

- mountPath: /etc/kubernetes/

name: k8s

readOnly: true

- mountPath: /etc/ssl/certs

name: certs

- mountPath: /etc/pki

name: pki

hostNetwork: true

volumes:

- hostPath:

path: /etc/kubernetes

name: k8s

- hostPath:

path: /etc/ssl/certs

name: certs

- hostPath:

path: /etc/pki

name: pki

Ensure the --authorization-mode argument includes Node

Remediation: Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml on the master node and set the --authorization-mode parameter to a value that includes Node.

--authorization-mode=Node,RBAC

Audit:

/bin/ps -ef | grep kube-apiserver | grep -v grep

Expected result:

'Node,RBAC' has 'Node'

Ensure that the --profiling argument is set to false

Remediation: Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml on the master node and set the below parameter.

--profiling=false

Audit:

/bin/ps -ef | grep kube-apiserver | grep -v grep

Expected result:

'false' is equal to 'false'

Fix all of the following violations that were found against the Kubelet:-

Ensure the --anonymous-auth argument is set to false.

Remediation: If using a Kubelet config file, edit the file to set authentication: anonymous: enabled to false. If using executable arguments, edit the kubelet service file /etc/systemd/system/kubelet.service.d/10-kubeadm.conf on each worker node and set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.

--anonymous-auth=false

Based on your system, restart the kubelet service. For example:

systemctl daemon-reload

systemctl restart kubelet.service

Audit:

/bin/ps -fC kubelet

Audit Config:

/bin/cat /var/lib/kubelet/config.yaml

Expected result:

'false' is equal to 'false'

2) Ensure that the --authorization-mode argument is set to Webhook.

Audit

docker inspect kubelet | jq -e '.[0].Args[] | match("--authorization-mode=Webhook").string'

Returned Value: --authorization-mode=Webhook

Fix all of the following violations that were found against the ETCD:-

      a. Ensure that the --auto-tls argument is not set to true

Do not use self-signed certificates for TLS. etcd is a highly-available key value store used by Kubernetes deployments for persistent storage of all of its REST API objects. These objects are sensitive in nature and should not be available to unauthenticated clients. You should enable the client authentication via valid certificates to secure the access to the etcd service.

apiVersion: v1

kind: Pod

metadata:

annotations:

scheduler.alpha.kubernetes.io/critical-pod: ""

creationTimestamp: null

labels:

component: etcd

tier: control-plane

name: etcd

namespace: kube-system

spec:

containers:

- command:

+ - etcd

+ - --auto-tls=true

image: k8s.gcr.io/etcd-amd64:3.2.18

imagePullPolicy: IfNotPresent

livenessProbe:

exec:

command:

- /bin/sh

- -ec

- ETCDCTL_API=3 etcdctl --endpoints=https://[192.168.22.9]:2379 --cacert=/etc/kubernetes/pki/etcd/ca.crt

--cert=/etc/kubernetes/pki/etcd/healthcheck-client.crt --key=/etc/kubernetes/pki/etcd/healthcheck-client.key

get foo

failureThreshold: 8

initialDelaySeconds: 15

timeoutSeconds: 15

name: etcd-should-fail

resources: {}

volumeMounts:

- mountPath: /var/lib/etcd

name: etcd-data

- mountPath: /etc/kubernetes/pki/etcd

name: etcd-certs

hostNetwork: true

priorityClassName: system-cluster-critical

volumes:

- hostPath:

path: /var/lib/etcd

type: DirectoryOrCreate

name: etcd-data

- hostPath:

path: /etc/kubernetes/pki/etcd

type: DirectoryOrCreate

name: etcd-certs

status: {}

1) Connect to the correct host

ssh cks000028

sudo -i

(If hostname differs in your exam, use the one shown in the question banner.)

2) Edit the API server static pod manifest

API server is a static pod in kubeadm.

vi /etc/kubernetes/manifests/kube-apiserver.yaml

3) Configure API server to enable auditing

Inside the command: section, ensure ALL of the following flags exist

(add them if missing, modify if present).

3.1 Use the given audit policy file

- --audit-policy-file=/etc/kubernetes/logpolicy/audit-policy.yaml

3.2 Store audit logs at the required location

- --audit-log-path=/var/log/kubernetes/audit-logs.txt

3.3 Retain a maximum of 2 log files

- --audit-log-maxbackup=2

3.4 Retain logs for 10 days

- --audit-log-maxage=10

✅ Example (your file may have more flags — that’s fine):

- command:

- kube-apiserver

- --audit-policy-file=/etc/kubernetes/logpolicy/audit-policy.yaml

- --audit-log-path=/var/log/kubernetes/audit-logs.txt

- --audit-log-maxbackup=2

- --audit-log-maxage=10

Save and exit:

wq

???? The API server will auto-restart (static pod).

Optional quick check:

docker ps | grep kube-apiserver

4) Edit and EXTEND the audit policy

Open the given basic policy:

vi /etc/kubernetes/logpolicy/audit-policy.yaml

The file already contains rules for what NOT to log.

You must ADD rules BELOW them (do not delete existing ones).

5) Add the required audit rules (EXACT ORDER)

Append the following rules in this order ⬇️

(order matters in audit policies).

5.1 Log namespaces interactions at RequestResponse

- level: RequestResponse

resources:

- group: ""

resources: ["namespaces"]

5.2 Log deployment request bodies in namespace webapps

- level: RequestResponse

namespaces: ["webapps"]

resources:

- group: "apps"

resources: ["deployments"]

5.3 Log ConfigMap and Secret interactions (all namespaces) at Metadata

- level: Metadata

resources:

- group: ""

resources: ["configmaps", "secrets"]

5.4 Log all other requests at Metadata

⚠️ This must be LAST

- level: Metadata

5.5 Final audit-policy.yaml should END like this

# (existing "do not log" rules above)

- level: RequestResponse

resources:

- group: ""

resources: ["namespaces"]

- level: RequestResponse

namespaces: ["webapps"]

resources:

- group: "apps"

resources: ["deployments"]

- level: Metadata

resources:

- group: ""

resources: ["configmaps", "secrets"]

- level: Metadata

Save and exit:

wq

6) Make sure API server uses the EXTENDED policy

Touch the manifest to guarantee reload:

touch /etc/kubernetes/manifests/kube-apiserver.yaml

Wait a few seconds.

7) Verify auditing is working

7.1 Check audit log file exists

ls -l /var/log/kubernetes/audit-logs.txt

7.2 Generate test activity

kubectl get namespaces

kubectl get configmaps -A

7.3 Confirm logs are written

tail -n 20 /var/log/kubernetes/audit-logs.txt

You should see audit entries.

1) Connect to the correct host

ssh cks000032

sudo -i

2) Use admin kubeconfig

export KUBECONFIG=/etc/kubernetes/admin.conf

3) Verify prerequisites (quick check)

These should already exist per task.

kubectl -n prod get svc web

kubectl -n prod get secret web-cert

kubectl get pods -n ingress-nginx

(If the ingress controller pods exist, you’re good.)

4) Create the Ingress resource

Create Ingress named web in namespace prod with:

host: web.k8s.local

all paths → Service web

TLS using Secret web-cert

HTTP → HTTPS redirect (NGINX)

cat <

apiVersion: networking.k8s.io/v1

kind: Ingress

metadata:

name: web

namespace: prod

annotations:

nginx.ingress.kubernetes.io/ssl-redirect: "true"

nginx.ingress.kubernetes.io/force-ssl-redirect: "true"

spec:

ingressClassName: nginx

tls:

- hosts:

- web.k8s.local

secretName: web-cert

rules:

- host: web.k8s.local

http:

paths:

- path: /

pathType: Prefix

backend:

service:

name: web

port:

number: 80

EOF

5) Verify Ingress creation

kubectl -n prod get ingress web

kubectl -n prod describe ingress web

Confirm:

Host = web.k8s.local

TLS Secret = web-cert

Backend Service = web

6) Test HTTP → HTTPS redirect

curl -L http://web.k8s.local

Expected:

Redirects to https://web.k8s.local

Returns application response over HTTPS

To add a Kubernetes cluster to your project, group, or instance:

Navigate to your:

Project’s Operations > Kubernetes page, for a project-level cluster.

Group’s Kubernetes page, for a group-level cluster.

Admin Area > Kubernetes page, for an instance-level cluster.

Click Add Kubernetes cluster.

Click the Add existing cluster tab and fill in the details:

Kubernetes cluster name (required) - The name you wish to give the cluster.

Environment scope (required) - The associated environment to this cluster.

API URL (required) - It’s the URL that GitLab uses to access the Kubernetes API. Kubernetes exposes several APIs, we want the “base” URL that is common to all of them. For example, https://kubernetes.example.com  rather than https://kubernetes.example.com/api/v1 .

Get the API URL by running this command:

kubectl cluster-info | grep -E 'Kubernetes master|Kubernetes control plane' | awk '/http/ {print $NF}'

CA certificate (required) - A valid Kubernetes certificate is needed to authenticate to the cluster. We use the certificate created by default.

List the secrets with kubectl get secrets, and one should be named similar to default-token-xxxxx. Copy that token name for use below.

Get the certificate by running this command:

kubectl get secret -o jsonpath="{['data']['ca\.crt']}"

se kubectl to create a CSR and approve it.

Get the list of CSRs:

kubectl get csr

Approve the CSR:

kubectl certificate approve myuser

Retrieve the certificate from the CSR:

kubectl get csr/myuser -o yaml

here are the role and role-binding to give john permission to create NEW_CRD resource:

kubectl apply -f roleBindingJohn.yaml --as=john

rolebinding.rbac.authorization.k8s.io/john_external-rosource-rb created

kind: RoleBinding

apiVersion: rbac.authorization.k8s.io/v1

metadata:

name: john_crd

namespace: development-john

subjects:

- kind: User

name: john

apiGroup: rbac.authorization.k8s.io

roleRef:

kind: ClusterRole

name: crd-creation

kind: ClusterRole

apiVersion: rbac.authorization.k8s.io/v1

metadata:

name: crd-creation

rules:

- apiGroups: ["kubernetes-client.io/v1"]

resources: ["NEW_CRD"]

verbs: ["create, list, get"]

worker1 $ vim /var/lib/kubelet/config.yaml

anonymous:

enabled: true #Delete this

enabled: false #Replace by this

authorization:

mode: AlwaysAllow #Delete this

mode: Webhook #Replace by this

worker1 $ systemctl restart kubelet.    # To reload kubelet config

ssh to master1

master1 $ vim /etc/kubernetes/manifests/kube-apiserver.yaml

- -- authorization-mode=Node,RBAC

master1 $ vim /etc/kubernetes/manifests/etcd.yaml

- --client-cert-auth=true

Explanationssh to worker1

worker1 $ vim /var/lib/kubelet/config.yaml

apiVersion: kubelet.config.k8s.io/v1beta1

authentication:

anonymous:

enabled: true #Delete this

enabled: false #Replace by this

webhook:

cacheTTL: 0s

enabled: true

x509:

clientCAFile: /etc/kubernetes/pki/ca.crt

authorization:

mode: AlwaysAllow #Delete this

mode: Webhook #Replace by this

webhook:

cacheAuthorizedTTL: 0s

cacheUnauthorizedTTL: 0s

cgroupDriver: systemd

clusterDNS:

- 10.96.0.10

clusterDomain: cluster.local

cpuManagerReconcilePeriod: 0s

evictionPressureTransitionPeriod: 0s

fileCheckFrequency: 0s

healthzBindAddress: 127.0.0.1

healthzPort: 10248

httpCheckFrequency: 0s

imageMinimumGCAge: 0s

kind: KubeletConfiguration

logging: {}

nodeStatusReportFrequency: 0s

nodeStatusUpdateFrequency: 0s

resolvConf: /run/systemd/resolve/resolv.conf

rotateCertificates: true

runtimeRequestTimeout: 0s

staticPodPath: /etc/kubernetes/manifests

streamingConnectionIdleTimeout: 0s

syncFrequency: 0s

volumeStatsAggPeriod: 0s

worker1 $ systemctl restart kubelet.    # To reload kubelet config

ssh to master1

master1 $ vim /etc/kubernetes/manifests/kube-apiserver.yaml

master1 $ vim /etc/kubernetes/manifests/etcd.yaml

kubesec scan k8s-deployment.yaml

cat < kubesec-test.yaml

apiVersion: v1

kind: Pod

metadata:

name: kubesec-demo

spec:

containers:

- name: kubesec-demo

image: gcr.io/google-samples/node-hello:1.0

securityContext:

readOnlyRootFilesystem: true

EOF

kubesec scan kubesec-test.yaml

docker run -i kubesec/kubesec:512c5e0 scan /dev/stdin < kubesec-test.yaml

kubesec http 8080 &

[1] 12345

{"severity":"info","timestamp":"2019-05-12T11:58:34.662+0100","caller":"server/server.go:69","message":"Starting HTTP server on port 8080"}

curl -sSX POST --data-binary @test/asset/score-0-cap-sys-admin.yml http://localhost:8080/scan

[

{

"object": "Pod/security-context-demo.default",

"valid": true,

"message": "Failed with a score of -30 points",

"score": -30,

"scoring": {

"critical": [

{

"selector": "containers[] .securityContext .capabilities .add == SYS_ADMIN",

"reason": "CAP_SYS_ADMIN is the most privileged capability and should always be avoided"

},

{

"selector": "containers[] .securityContext .runAsNonRoot == true",

"reason": "Force the running image to run as a non-root user to ensure least privilege"

},

// ...

1. For Dockerfile: Fix the image version & user name in Dockerfile

2. For mydeployment.yaml : Fix security contexts

Explanation[desk@cli] $ vim /home/cert_masters/Dockerfile

FROM ubuntu:latest # Remove this

FROM ubuntu:18.04 # Add this

USER root                                                                            # Remove this

USER nobody                                                                          # Add this

RUN apt get install -y lsof=4.72 wget=1.17.1 nginx=4.2

ENV ENVIRONMENT=testing

USER root                                                                            # Remove this

USER nobody                                                                          # Add this

CMD ["nginx -d"]

[desk@cli] $ vim /home/cert_masters/mydeployment.yaml

apiVersion: apps/v1

kind: Deployment

metadata:

creationTimestamp: null

labels:

app: kafka

name: kafka

spec:

replicas: 1

selector:

matchLabels:

app: kafka

strategy: {}

template:

metadata:

creationTimestamp: null

labels:

app: kafka

spec:

containers:

- image: bitnami/kafka

name: kafka

volumeMounts:

- name: kafka-vol

mountPath: /var/lib/kafka

securityContext:

{"capabilities":{"add":["NET_ADMIN"],"drop":["all"]},"privileged": True,"readOnlyRootFilesystem": False, "runAsUser": 65535} # Delete This

{"capabilities":{"add":["NET_ADMIN"],"drop":["all"]},"privileged": False,"readOnlyRootFilesystem": True, "runAsUser": 65535} # Add This

resources: {}

volumes:

- name: kafka-vol

emptyDir: {}

status: {}

Pictorial View:

[desk@cli] $ vim /home/cert_masters/mydeployment.yaml

1) Connect to the correct host

ssh cks000035

sudo -i

export KUBECONFIG=/etc/kubernetes/admin.conf

2) List the 3 container names + images in the Deployment

kubectl -n alpine get deploy alpine -o jsonpath='{range .spec.template.spec.containers[*]}{.name}{"\t"}{.image}{"\n"}{end}'

You’ll get 3 lines like:

c1 alpine:3.xx

c2 alpine:3.yy

c3 alpine:3.zz

3) Identify which alpine image has libcrypto3 at 3.1.4-r5

Fastest reliable method (since it’s Alpine, just query apk inside each image):

Run these one-by-one for each image you saw in step 2:

docker run --rm sh -c 'apk info -v libcrypto3 2>/dev/null | head -n1'

docker run --rm sh -c 'apk info -v libcrypto3 2>/dev/null | head -n1'

docker run --rm sh -c 'apk info -v libcrypto3 2>/dev/null | head -n1'

✅ The correct image is the one that prints exactly:

libcrypto3-3.1.4-r5

Note that full image tag, e.g.:

IMG=alpine:3.xx

4) Create SPDX document with bom for that identified image

(Use the identified image from step 3.)

bom generate --image $IMG --format spdx --output /home/candidate/alpine.spdx

Verify file exists:

ls -l /home/candidate/alpine.spdx

5) Remove ONLY the container that uses that image version

The manifest to edit is:

vi /home/candidate/alpine-deployment.yaml

In the spec.template.spec.containers: list, find the container entry whose image: equals the identified $IMG, and delete that one container block only (name/image/ports/etc for that container).

Save:

wq

6) Apply the updated Deployment (do not change other containers)

kubectl apply -f /home/candidate/alpine-deployment.yaml

Wait rollout:

kubectl -n alpine rollout status deployment/alpine

7) Verify only 2 containers remain

kubectl -n alpine get deploy alpine -o jsonpath='{range .spec.template.spec.containers[*]}{.name}{"\t"}{.image}{"\n"}{end}'

You should now see 2 lines, and the $IMG line should be gone.

If bom generate ... errors (quick fix)

Check exact syntax on that system:

bom --help

bom generate --help

Then rerun with the flags it expects, keeping:

image = $IMG

output = /home/candidate/alpine.spdx

format = spdx