Anhang A: Spickzettel
Die wichtigsten Befehle und Schnipsel aus allen Tagen, nach Lehrplanbereich geordnet. Die Tage stehen in Klammern. Platzhalter in spitzen Klammern ersetzt du; IPs sind die des Labors.
Prüfungshandwerk (Tag 1)
source <(kubectl completion bash)
alias k=kubectl
complete -o default -F __start_kubectl k
export do="--dry-run=client -o yaml" # YAML erzeugen statt anlegen
export now="--force --grace-period=0" # sofort löschen
k config use-context <kontext> # erste Aktion jeder Aufgabe
k config set-context --current --namespace=<ns>
ssh [email protected] # Node-Arbeit, am Ende exitStatic-Pod-Manifest sichern, ändern, prüfen, zurückrollen:
M=/etc/kubernetes/manifests
B=~/cks/<tagNN>/backup && mkdir -p $B
sudo cp $M/kube-apiserver.yaml $B/ # NIE ins Manifest-Verzeichnis
sudo vim $M/kube-apiserver.yaml
sudo diff $B/kube-apiserver.yaml $M/kube-apiserver.yaml
sudo crictl ps -a --name kube-apiserver # bis zu 1 min warten
sudo crictl logs <container-id>
sudo sh -c 'tail -n 2 \
/var/log/pods/kube-system_kube-apiserver-cp1_*/kube-apiserver/*.log'
sudo journalctl -u kubelet --since -2min | grep -i apiserver | tail -2
k get --raw /readyz; echo
sudo cp $B/kube-apiserver.yaml $M/ # Rückweg
sudo crictl stop $(sudo crictl ps --name kube-apiserver -q) # Neustart erzwingen
ps -o args= -C kubelet | tr ' ' '\n' | grep -- --config
sudo grep staticPodPath /var/lib/kubelet/config.yaml- Das kubelet liest jede Datei im
Manifest-Verzeichnis als Pod, egal welche Endung (nur versteckte nicht).
Ein
.bakdaneben ist ein zweiter API-Server-Pod. - Nach jeder Änderung
1/1 Readyabwarten, nicht nurRunning.
Cluster Setup (Tag 2–4)
Default Deny, DNS erlauben, Metadaten sperren (Tag 2):
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: default-deny
spec:
podSelector: {}
policyTypes:
- Ingress
- Egress
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: allow-dns
spec:
podSelector: {}
policyTypes:
- Egress
egress:
- to:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: kube-system
podSelector:
matchLabels:
k8s-app: kube-dns
ports:
- port: 53
protocol: UDP
- port: 53
protocol: TCP
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: client-world
spec:
podSelector:
matchLabels:
run: client
policyTypes:
- Egress
egress:
- to:
- ipBlock:
cidr: 0.0.0.0/0
except:
- 169.254.169.254/32from: from:
- namespaceSelector: {...} - namespaceSelector: {...}
- podSelector: {...} podSelector: {...}
-> ODER (zwei Einträge) -> UND (ein Eintrag)
CiliumNetworkPolicy: Entities, L7, FQDN, clusterweites Deny (Tag 2):
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
name: client-egress
spec:
endpointSelector:
matchLabels:
run: client
egress:
- toEndpoints:
- matchLabels:
k8s:io.kubernetes.pod.namespace: kube-system
k8s-app: kube-dns
toPorts:
- ports:
- port: "53"
protocol: ANY
rules:
dns:
- matchPattern: "*"
- toEntities:
- kube-apiserver
toPorts:
- ports:
- port: "6443"
protocol: TCP
- toFQDNs:
- matchName: kubernetes.io
toPorts:
- ports:
- port: "443"
protocol: TCP
---
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
name: web-l7
spec:
endpointSelector:
matchLabels:
app: web
ingress:
- fromEndpoints:
- matchLabels:
run: client
toPorts:
- ports:
- port: "80"
protocol: TCP
rules:
http:
- method: GET
path: "/"
---
apiVersion: cilium.io/v2
kind: CiliumClusterwideNetworkPolicy
metadata:
name: deny-metadata
spec:
endpointSelector:
matchLabels:
k8s:io.kubernetes.pod.namespace: <ns>
enableDefaultDeny:
egress: false
ingress: false
egressDeny:
- toCIDR:
- 169.254.169.254/32- Unter Cilium erfasst
ipBlockkeine Node-IPs (Identitätenhost,remote-node). Für API-Server und NodestoEntitiesnutzen. Deny schlägt Allow. k get cnp,ccnp -Azeigt alle Cilium-Policies.
kube-bench (Tag 3):
sudo kube-bench run --targets=master --benchmark=cis-1.12 > master.txt 2>&1
grep "^\[FAIL\]" master.txt | cut -c1-62
sudo kube-bench run --targets=master --benchmark=cis-1.12 --check=1.2.15,1.4.1
sudo kube-bench run --targets=master --benchmark=cis-1.12 --check=1.4.1 \
2>/dev/null | sed -n '/== Remediations/,/^$/p'
sudo kube-bench run --targets=master --benchmark=cis-1.12 --group=1.4 \
--noremediations
sudo kube-bench run --targets=node --benchmark=cis-1.12 --json 2>/dev/null \
| jq .TotalsTypische Remediations aus dem Buch: --profiling=false an
API-Server, Controller-Manager und Scheduler;
chown -R etcd:etcd /var/lib/etcd; kubelet-Dateien
chmod 600 (kubelet.service,
10-kubeadm.conf,
/var/lib/kubelet/config.yaml), danach
systemctl daemon-reload und
restart kubelet.
Ingress mit TLS (Tag 4):
openssl req -x509 -newkey rsa:2048 -nodes -days 365 \
-keyout web.key -out web.crt -subj "/CN=web.lab.local" \
-addext "subjectAltName=DNS:web.lab.local"
k create secret tls web-tls --cert=web.crt --key=web.key
k create ingress web --class=traefik --rule="web.lab.local/*=web:80,tls=web-tls"
curl -s -o /dev/null -w "%{http_code}\n" --cacert web.crt \
--resolve web.lab.local:30443:172.16.0.126 https://web.lab.local:30443/
k get secret web-tls -o jsonpath='{.data.tls\.crt}' | base64 -d \
| openssl x509 -noout -subject -enddateBinaries prüfen (Tag 4):
kubelet --version # Version exakt ablesen
V=$(kubelet --version | awk '{print $2}')
ARCH=$(dpkg --print-architecture)
U=https://dl.k8s.io/release/$V/bin/linux/${ARCH}
curl -sSLO $U/kubelet.sha512
echo "$(cat kubelet.sha512) /usr/bin/kubelet" | sha512sum -c
echo "$(curl -sSL $U/kubeadm.sha512) /usr/bin/kubeadm" | sha512sum -c
P=$(pidof kube-apiserver) # laufendes Binary prüfen
echo "$(curl -sSL $U/kube-apiserver.sha512) /proc/$P/exe" | sudo sha512sum -cCluster Hardening (Tag 5, 6)
RBAC prüfen und minimieren (Tag 5):
SA=system:serviceaccount:<ns>:<sa>
k auth can-i --list --as=$SA
k auth can-i get secrets -A --as=$SA
k auth can-i get secret/<name> --as=$SA
k get clusterrolebindings \
-o jsonpath='{range .items[*]}{.metadata.name} {.subjects[*].name}{"\n"}{end}'
k get rolebindings -o wide
k get clusterrolebinding <crb> -o jsonpath='{.roleRef.kind}/{.roleRef.name}{"\n"}'
k create role <r> --verb=get,list,watch,patch,update --resource=deployments
k create role <r> --verb=get --resource=secrets --resource-name=<secret>
k create rolebinding <rb> --role=<r> --serviceaccount=<ns>:<sa>
k create rolebinding <rb> --clusterrole=view --serviceaccount=<ns>:<sa>
k delete clusterrolebinding <crb>ServiceAccounts und Tokens (Tag 5):
k patch sa default -p '{"automountServiceAccountToken":false}'
k create token <sa> --duration=10m
k get secrets -A --field-selector type=kubernetes.io/service-account-token
k exec <pod> -- ls /var/run/secrets/kubernetes.io/serviceaccountautomountServiceAccountToken: falseam ServiceAccount oder inpod.spec; der Wert am Pod gewinnt.kubectl --token=…mit Admin-kubeconfig bleibt Admin (Zertifikat gewinnt). Token testen mit--kubeconfig=/dev/null --server=https://172.16.0.126:6443 --certificate-authority=ca.crt --token=$TOKEN.
Anonymen Zugriff nur für Health-Checks (Tag 6), Datei
/etc/kubernetes/authn/authn-config.yaml:
apiVersion: apiserver.config.k8s.io/v1
kind: AuthenticationConfiguration
anonymous:
enabled: true
conditions:
- path: /livez
- path: /readyz
- path: /healthzIm API-Server-Manifest Flag, Mount und Volume (dasselbe Muster gilt für Encryption, Audit und Admission-Konfiguration):
- --authentication-config=/etc/kubernetes/authn/authn-config.yaml
# unter volumeMounts:
- mountPath: /etc/kubernetes/authn
name: authn
readOnly: true
# unter volumes:
- hostPath:
path: /etc/kubernetes/authn
type: Directory
name: authncurl -sk https://172.16.0.126:6443/readyz; echo # ok
curl -sk https://172.16.0.126:6443/version -o /dev/null -w "%{http_code}\n" # 401
sudo grep -- "--enable-admission-plugins" $M/kube-apiserver.yaml # NodeRestriction
k get svc kubernetes -o jsonpath='{.spec.type}{"\n"}' # ClusterIP
curl -sk https://172.16.0.228:10250/pods; echo # kubelet: Unauthorized
ssh [email protected] \
'sudo grep -A1 -E "^authorization:| anonymous:" /var/lib/kubelet/config.yaml'--anonymous-auth=falselegt den API-Server in eine Neustart-Schleife (Probes sind anonym).--anonymous-authundanonymous:in der AuthenticationConfiguration zusammen sind verboten.- kubelet:
authentication.anonymous.enabled: false,authorization.mode: Webhook.
Härtungen upgradefest: dieselben Flags und Volumes in die ConfigMap
kubeadm-config (Tag 6, 10, 15):
k -n kube-system edit cm kubeadm-config
sudo kubeadm upgrade diff v1.37.2 | grep -E "^[-+] |^---" apiServer:
extraArgs:
- name: profiling
value: "false"
- name: authentication-config
value: /etc/kubernetes/authn/authn-config.yaml
extraVolumes:
- name: authn
hostPath: /etc/kubernetes/authn
mountPath: /etc/kubernetes/authn
readOnly: true
pathType: DirectoryUpgrade (Tag 6), Control Plane:
sudo apt-mark unhold kubeadm
sudo apt-get update
sudo apt-get install -y kubeadm=1.37.2-1.1
sudo apt-mark hold kubeadm
sudo kubeadm upgrade plan
sudo kubeadm upgrade apply v1.37.2 # nur erster Control-Plane-Node
k drain cp1 --ignore-daemonsets --delete-emptydir-data
sudo apt-mark unhold kubelet kubectl
sudo apt-get install -y kubelet=1.37.2-1.1 kubectl=1.37.2-1.1
sudo apt-mark hold kubelet kubectl
sudo systemctl daemon-reload
sudo systemctl restart kubelet
k uncordon cp1Worker: drain (von cp1), auf dem Worker kubeadm-Paket,
sudo kubeadm upgrade node, kubelet/kubectl,
daemon-reload, restart, dann uncordon. Paket-Updates setzen
Dateirechte von kubelet.service und
10-kubeadm.conf auf 644 zurück.
System Hardening (Tag 7, 8)
Host minimieren (Tag 7):
sudo ss -tulpn # ohne sudo keine Prozesse
sudo ss -tlnp 'sport = :111'
P=$(pidof <prozess>)
sudo readlink /proc/$P/exe
ps -o unit= -p $P # systemd-Unit
dpkg -S /usr/sbin/<binary> # Paket
sudo systemctl disable --now <dienst>
sudo systemctl disable --now rpcbind.socket rpcbind.service
sudo systemctl mask rpcbind.socket rpcbind.service
sudo apt-get purge -y <paket>
sudo apt-get autoremove --purge -y
awk -F: '$3==0' /etc/passwd
getent passwd | awk -F: '$7 !~ /(nologin|false)$/ {print $1, $3, $7}'
sudo grep -rn NOPASSWD /etc/sudoers /etc/sudoers.d
sudo usermod -L -s /usr/sbin/nologin <user>
sudo userdel -r <user>
sudo find / -xdev -type f -perm -4000 -not -path '/var/lib/containerd/*' 2>/dev/null
sudo chmod u-s <datei>
echo 'install dccp /bin/false' | sudo tee -a /etc/modprobe.d/cks-blacklist.conf
sudo modprobe -n -v dccpAppArmor (Tag 8), Profil auf jedem möglichen Node laden:
sudo apparmor_parser -r /etc/apparmor.d/cks-deny-write
sudo aa-status | grep cks
sudo journalctl -k --since -5min | grep 'apparmor="DENIED"'
k exec <pod> -- cat /proc/1/attr/current; echo securityContext:
appArmorProfile:
type: Localhost
localhostProfile: cks-deny-write # Profilname, nicht Dateipfadseccomp (Tag 8), Profil unter
/var/lib/kubelet/seccomp/profiles/no-mkdir.json:
{
"defaultAction": "SCMP_ACT_ALLOW",
"syscalls": [
{
"names": ["mkdir", "mkdirat"],
"action": "SCMP_ACT_ERRNO"
}
]
} securityContext:
seccompProfile:
type: Localhost
localhostProfile: profiles/no-mkdir.json # relativ zu /var/lib/kubelet/seccompk exec <pod> -- grep Seccomp: /proc/1/status # 2 = Filter aktiv, 0 = keinertype bei beiden: RuntimeDefault,
Localhost oder Unconfined. Fehlt das Profil
auf dem Node: CreateContainerError, Name steht in
k describe pod.
Minimize Microservice Vulnerabilities (Tag 9–12)
Pod Security Admission (Tag 9):
k label --dry-run=server --overwrite ns <ns> \
pod-security.kubernetes.io/enforce=restricted # zeigt verletzende Pods
k label --overwrite ns <ns> \
pod-security.kubernetes.io/enforce=restricted \
pod-security.kubernetes.io/enforce-version=v1.37 \
pod-security.kubernetes.io/warn=restricted \
pod-security.kubernetes.io/audit=restricted
k get ns -l pod-security.kubernetes.io/enforce=restricted
k get events --field-selector reason=FailedCreate -o jsonpath='{.items[0].message}'SecurityContext für restricted:
spec:
securityContext:
runAsNonRoot: true
runAsUser: 1000 # nötig, wenn das Image als root startet
seccompProfile:
type: RuntimeDefault
containers:
- name: app
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop: ["ALL"]Gefährliche Pods finden (Tag 9):
k get pods -A -o json | jq -r '.items[]
| select(.spec.hostNetwork or .spec.hostPID or .spec.hostIPC)
| .metadata.namespace + "/" + .metadata.name'
k get pods -A -o json | jq -r '.items[]
| select(any(.spec.containers[], .spec.initContainers[]?;
.securityContext.privileged))
| .metadata.namespace + "/" + .metadata.name'Secrets verschlüsseln (Tag 10), Datei
/etc/kubernetes/enc/enc.yaml (chmod 600), Flag
--encryption-provider-config=/etc/kubernetes/enc/enc.yaml
plus Mount und Volume:
apiVersion: apiserver.config.k8s.io/v1
kind: EncryptionConfiguration
resources:
- resources:
- secrets
providers:
- aescbc: # erster Provider schreibt
keys:
- name: key1
secret: <head -c 32 /dev/urandom | base64>
- identity: {} # liest AltbestandP=/etc/kubernetes/pki
E="--cacert=$P/etcd/ca.crt --cert=$P/apiserver-etcd-client.crt"
E="$E --key=$P/apiserver-etcd-client.key"
sudo etcdctl $E get /registry/secrets/<ns>/<name> --print-value-only | head -c 23
k get secrets -A -o json | k replace -f - # Bestand neu verschlüsseln
sudo etcdctl $E get /registry/secrets --prefix --print-value-only \
| grep -ac "^k8s:enc:aescbc"Rotation: neuen Schlüssel oben einfügen, API-Server neu starten, alle Secrets ersetzen, alten Schlüssel entfernen, erneut neu starten.
Sandbox mit RuntimeClass (Tag 11):
sudo crictl info | jq -r '.config.containerd.runtimes | keys[]' # Handler-NameapiVersion: node.k8s.io/v1
kind: RuntimeClass
metadata:
name: gvisor
handler: runsc
scheduling:
nodeSelector:
sandbox: gvisor
---
# im Pod bzw. Pod-Template:
spec:
runtimeClassName: gvisork exec <pod> -- dmesg | head -1 # gVisor meldet sich
k exec <pod> -- uname -r # anderer Kernel als der NodePod-zu-Pod-Verschlüsselung mit Cilium-WireGuard (Tag 12):
helm get values cilium -n kube-system -o yaml > cilium-values-backup.yaml
cilium upgrade --version 1.20.5 --set encryption.enabled=true \
--set encryption.type=wireguard
k -n kube-system rollout restart ds/cilium
k -n kube-system rollout status ds/cilium --timeout=300s
k -n kube-system exec ds/cilium -c cilium-agent -- cilium-dbg encrypt status
k -n kube-system exec ds/cilium -c cilium-agent -- cilium-dbg status | grep Encryption
ssh [email protected] 'sudo timeout 10 tcpdump -lni eth0 -A udp port 8472'Ohne Neustart der Agents bleibt encrypt status auf
Disabled. Mit WireGuard läuft Pod-Verkehr über UDP 51871
statt im Klartext über VXLAN (UDP 8472).
Supply Chain Security (Tag 13, 14)
Trivy und SBOM (Tag 13):
trivy image -q --scanners vuln <image> | grep ^Total
trivy image -q --scanners vuln --severity HIGH,CRITICAL --ignore-unfixed <image>
trivy image -q --scanners vuln --severity CRITICAL --exit-code 1 <image>
trivy image -q --scanners vuln --insecure <registry-ohne-tls>/<image>
trivy image -q --input <archiv>.tar
trivy config -q --severity HIGH,CRITICAL --exit-code 1 <manifest-oder-ordner>
trivy image -q -f cyclonedx -o app.cdx.json <image>
trivy image -q -f spdx-json -o app.spdx.json <image>
trivy sbom -q app.cdx.json | grep ^Total
bom generate --image <image> --format json --output app.bom.spdx.json
for i in $(k get pods -o jsonpath='{.items[*].spec.containers[*].image}' \
| tr ' ' '\n' | sort -u); do
echo "== $i"; trivy image -q --scanners vuln --severity CRITICAL $i | grep ^Total
doneMinimales Image:
FROM gcr.io/distroless/static-debian13:nonroot (oder
scratch), USER 65532:65532, keine Shell, kein
Paketmanager. Debuggen per
printf 'securityContext:\n runAsUser: 65532\n' > dbg.yaml,
dann
k debug <pod> --image=busybox:1.37 --target=<container> --profile=restricted --custom=dbg.yaml.
Erlaubte Registry per ValidatingAdmissionPolicy (Tag 14), Policy und Binding:
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingAdmissionPolicy
metadata:
name: allowed-registries
spec:
failurePolicy: Fail
matchConstraints:
resourceRules:
- apiGroups: [""]
apiVersions: ["v1"]
operations: ["CREATE", "UPDATE"]
resources: ["pods"]
validations:
- expression: >-
object.spec.containers.all(c, c.image.startsWith('172.16.0.126:30500/'))
&& (!has(object.spec.initContainers) ||
object.spec.initContainers.all(c, c.image.startsWith('172.16.0.126:30500/')))
message: "Nur Images aus 172.16.0.126:30500/ erlaubt"
- expression: "object.spec.containers.all(c, c.image.contains('@sha256:'))"
messageExpression: >-
"Image ohne Digest: " + object.spec.containers.map(c, c.image).join(", ")
---
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingAdmissionPolicyBinding
metadata:
name: allowed-registries
spec:
policyName: allowed-registries
validationActions: ["Deny"]
matchResources:
namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: <ns>k get pod <p> -o jsonpath='{.status.containerStatuses[0].imageID}{"\n"}' # Digest
k get validatingadmissionpolicy <name> -o yaml # CEL-Fehler: status.typeCheckingcosign (Tag 14; HTTP-Registry nur im Labor):
export COSIGN_PASSWORD=lab-cosign-pw
cosign generate-key-pair
cosign signing-config create --out signing-config.json
cosign sign --key cosign.key --signing-config signing-config.json \
--allow-http-registry --yes <image>@sha256:<digest>
cosign verify --key cosign.pub --allow-http-registry --insecure-ignore-tlog <image>ImagePolicyWebhook (Tag 14): ImagePolicyWebhook in
--enable-admission-plugins,
--admission-control-config-file=/etc/kubernetes/admission/admission-config.yaml,
Verzeichnis mounten.
apiVersion: apiserver.config.k8s.io/v1
kind: AdmissionConfiguration
plugins:
- name: ImagePolicyWebhook
configuration:
imagePolicy:
kubeConfigFile: /etc/kubernetes/admission/imagepolicy.kubeconfig
allowTTL: 50
denyTTL: 50
retryBackoff: 500
defaultAllow: false # "deny if backend unreachable"Statische Analyse (Tag 14):
kubesec scan notes.yaml | jq -r '.[0] | .score, .message'
kubesec scan notes.yaml | jq -r '.[0].scoring.critical[] | .id + ": " + .selector'
kube-linter lint notes.yamlMonitoring, Logging and Runtime Security (Tag 15–18)
Audit-Policy (Tag 15), Datei
/etc/kubernetes/audit/policy.yaml – erste passende Regel
gilt:
apiVersion: audit.k8s.io/v1
kind: Policy
omitStages:
- RequestReceived
rules:
- level: None
nonResourceURLs: ["/healthz*", "/livez*", "/readyz*", "/version"]
- level: None
resources:
- group: coordination.k8s.io
resources: ["leases"]
- level: None
userGroups: ["system:nodes"]
verbs: ["get", "list", "watch"]
- level: None
verbs: ["watch"]
- level: Metadata
resources:
- group: ""
resources: ["secrets", "configmaps", "serviceaccounts/token"]
- level: RequestResponse
resources:
- group: ""
resources: ["pods", "pods/exec"]
verbs: ["create", "update", "patch", "delete"]
- level: MetadataFünf Flags, zwei Volumes (Policy readOnly,
Log-Verzeichnis DirectoryOrCreate):
- --audit-policy-file=/etc/kubernetes/audit/policy.yaml
- --audit-log-path=/var/log/kubernetes/audit/audit.log
- --audit-log-maxage=7
- --audit-log-maxbackup=2
- --audit-log-maxsize=20L=/var/log/kubernetes/audit/audit.log
sudo jq -c 'select(.objectRef.resource=="secrets" and .objectRef.name=="<s>")
| [.user.username, .verb, .responseStatus.code, .sourceIPs[0]]' $L
sudo jq -c 'select(.verb=="delete" and .objectRef.resource=="pods")
| [.user.username, .objectRef.name, .stageTimestamp]' $L
sudo jq -r 'select(.responseStatus.code==403) | .user.username' $L | sort | uniq -cFalco (Tag 16), eigene Regel und Override (Helm
customRules oder auf dem Node
/etc/falco/falco_rules.local.yaml, nie
falco_rules.yaml):
- list: etc_writer
items: [touch, tee, cp, mv, sed, vi]
- rule: Write below etc in container
desc: Ein Prozess im Container schreibt unter /etc
condition: >
open_write and fd.name startswith /etc/ and container
and proc.name in (etc_writer)
output: >
Schreibzugriff unter /etc (file=%fd.name process=%proc.name
user=%user.name container=%container.name)
priority: ERROR
- rule: Terminal shell in container
output: "%evt.time,%container.id,%container.name,%user.name"
override:
output: replace
- rule: Contact K8S API Server From Container
enabled: false
override:
enabled: replacek get pod <p> -o jsonpath='{.spec.nodeName}'; echo
F=$(k -n falco get pod -l app.kubernetes.io/name=falco \
--field-selector spec.nodeName=<node> -o name)
k -n falco logs $F -c falco --since=5m | grep <stichwort>
k -n falco exec $F -c falco -- falco --list | grep '^container\.'
k -n falco exec $F -c falco -- falco -l 'Terminal shell in container'
k -n falco exec $F -c falco -- falco --dry-run # Syntaxtest der Regeln
k -n falco exec $F -c falco -- grep -A3 '^rules_files' /etc/falco/falco.yamlAls Systemdienst (nicht live geprüft):
journalctl -u falco-modern-bpf, nach Änderung
sudo systemctl restart falco-modern-bpf.
Tetragon (Tag 17), TracingPolicyNamespaced mit
Durchsetzung:
apiVersion: cilium.io/v1alpha1
kind: TracingPolicyNamespaced
metadata:
name: shadow-kill
namespace: <ns>
spec:
kprobes:
- call: security_file_permission
syscall: false
args:
- index: 0
type: file
- index: 1
type: int
selectors:
- matchArgs:
- index: 0
operator: Equal
values:
- /etc/shadow
matchActions:
- action: SigkillT=$(k -n kube-system get pod -l app.kubernetes.io/name=tetragon \
--field-selector spec.nodeName=<node> -o name)
k -n kube-system exec $T -c tetragon -- tetra tracingpolicy list
k -n kube-system exec $T -c tetragon -- \
tetra getevents -o compact --namespace <ns> --process curlOverride mit argError: -1 lädt im Labor nur
an Syscalls (call: sys_openat, syscall: true);
an security_file_permission endet es in
load_error, ohne dass k apply meckert.
podSelector grenzt die Policy auf Pods ein.
Unveränderliche Container (Tag 18):
securityContext:
readOnlyRootFilesystem: true
volumeMounts:
- name: tmp
mountPath: /tmp
volumes:
- name: tmp
emptyDir: {}k get pods -o json | jq -r '.items[]
| select(any(.spec.containers[]; .securityContext.readOnlyRootFilesystem != true))
| .metadata.name'Spurensuche auf dem Node mit crictl und
/proc (Tag 18):
ID=$(sudo crictl ps --namespace <ns> --name <container> -q)
PID=$(sudo crictl inspect -o go-template --template '{{.info.pid}}' $ID)
U=$(sudo grep -o -m1 'upperdir=[^,]*' /proc/$PID/mounts | cut -d= -f2)
sudo find $U -type f -printf '%P\n' # was im Container geschrieben wurde
ps -o pid,ppid,user,cmd --ppid $PID
sudo ls -l /proc/<pid>/exe
sudo cat /proc/<pid>/cmdline | tr '\0' ' '; echo
sudo nsenter -t <pid> -n ss -tlnp # Ports im Netz-Namespace des Pods
CID=$(grep -o 'cri-containerd-[0-9a-f]*' /proc/<pid>/cgroup | cut -d- -f3)
sudo crictl ps --id $CID -o json | jq -r '.containers[0] | .metadata.name + " "
+ .labels["io.kubernetes.pod.name"] + " " + .labels["io.kubernetes.pod.namespace"]'Isolieren, sichern, entfernen:
k label pod <p> app- quarantine=true, Deny-all-Policy
auf quarantine: "true",
k get pod <p> -o yaml > evidence/pod.yaml,
k logs <p> > evidence/pod.log, dann den Besitzer
(Deployment) skalieren oder reparieren, nicht nur den Pod löschen.
Symptom und erster Befehl
| Symptom | Erster Befehl |
|---|---|
kubectl hängt nach Manifest-Änderung |
sudo crictl ps -a --name kube-apiserver |
API-Server-Container Exited |
sudo crictl logs <id>, Backup zurück |
API-Server startet, Pod 0/1 |
k -n kube-system get events, Probes |
RuntimeClass "…" not found |
k get runtimeclass |
CreateContainerError |
k describe pod: AppArmor/seccomp-Profil |
ContainerCreating mit Sandbox-Fehler |
Events FailedCreatePodSandBox: Handler |
Deployment 0/1, keine Pods |
Events FailedCreate: PodSecurity, VAP |
CreateContainerConfigError |
runAsNonRoot und Image-User |
| Pod startet nicht, read-only FS | k logs: Schreibpfad, emptyDir mounten |
| Policy ohne Wirkung | k get netpol,cnp,ccnp -A, Labels, Namespace |
encrypt status: Disabled |
k -n kube-system rollout restart ds/cilium |
| Falco startet nicht | k -n falco logs <pod> -c falco --previous |
| TracingPolicy ohne Ereignisse | tetra tracingpolicy list: Zustand |
| Secret in etcd lesbar | Provider-Reihenfolge, k replace vergessen |
Doku-Pfade
Alle kubernetes.io-Pfade beginnen mit
https://kubernetes.io/docs/.
| Thema | Pfad |
|---|---|
| Static Pods | tasks/configure-pod-container/static-pod/ |
| crictl | tasks/debug/debug-cluster/crictl/ |
| NetworkPolicy | concepts/services-networking/network-policies/ |
| Ingress, TLS | concepts/services-networking/ingress/ |
| Cluster absichern | tasks/administer-cluster/securing-a-cluster/ |
| RBAC | reference/access-authn-authz/rbac/ |
| ServiceAccounts | tasks/configure-pod-container/configure-service-account/ |
| Authentifizierung, anonym | reference/access-authn-authz/authentication/ |
| kubelet authn/authz | reference/access-authn-authz/kubelet-authn-authz/ |
| Admission Controllers | reference/access-authn-authz/admission-controllers/ |
| VAP | reference/access-authn-authz/validating-admission-policy/ |
| kubeadm-Upgrade | tasks/administer-cluster/kubeadm/kubeadm-upgrade/ |
| kube-apiserver-Flags | reference/command-line-tools-reference/kube-apiserver/ |
| AppArmor | tutorials/security/apparmor/ |
| seccomp | tutorials/security/seccomp/ |
| SecurityContext | tasks/configure-pod-container/security-context/ |
| Pod Security Standards | concepts/security/pod-security-standards/ |
| Pod Security Admission | concepts/security/pod-security-admission/ |
| PSA-Labels | tasks/configure-pod-container/enforce-standards-namespace-labels/ |
| Secrets | concepts/configuration/secret/ |
| Encryption at Rest | tasks/administer-cluster/encrypt-data/ |
| RuntimeClass | concepts/containers/runtime-class/ |
| Auditing | tasks/debug/debug-cluster/audit/ |
| Ephemere Container | tasks/debug/debug-application/debug-running-pod/ |
Weitere Projekte (jeweils mit https:// davor):
| Thema | URL |
|---|---|
| Binaries, Prüfsummen | kubernetes.io/releases/download/ |
| Cilium Policy-Sprache | docs.cilium.io/en/stable/security/policy/language/ |
| Cilium WireGuard | docs.cilium.io/en/stable/security/network/encryption-wireguard/ |
| kube-bench | github.com/aquasecurity/kube-bench/blob/main/docs/running.md |
| gVisor mit containerd | gvisor.dev/docs/user_guide/containerd/quick_start/ |
| Trivy | trivy.dev/docs/latest/ |
| bom | github.com/kubernetes-sigs/bom |
| cosign | docs.sigstore.dev/cosign/signing/signing_with_containers/ |
| kubesec, KubeLinter | kubesec.io/, docs.kubelinter.io/ |
| Falco Regeln | falco.org/docs/concepts/rules/basic-elements/ |
| Falco Overrides | falco.org/docs/concepts/rules/overriding/ |
| Falco Felder | falco.org/docs/reference/rules/supported-fields/ |
| Tetragon TracingPolicy | tetragon.io/docs/concepts/tracing-policy/ |
| Tetragon Selectors | tetragon.io/docs/concepts/tracing-policy/selectors/ |
Welche dieser Seiten in der Prüfung geöffnet werden dürfen, steht in den aktuellen Prüfungshinweisen der Linux Foundation.