Zum Inhalt springen
Trainingslager IT-Trainingslager

Anhang A: Spickzettel

Die wichtigsten Befehle und Schnipsel aus allen Tagen, nach Lehrplanbereich geordnet. Die Tage stehen in Klammern. Platzhalter in spitzen Klammern ersetzt du; IPs sind die des Labors.

Prüfungshandwerk (Tag 1)

source <(kubectl completion bash)
alias k=kubectl
complete -o default -F __start_kubectl k
export do="--dry-run=client -o yaml"         # YAML erzeugen statt anlegen
export now="--force --grace-period=0"        # sofort löschen
k config use-context <kontext>                # erste Aktion jeder Aufgabe
k config set-context --current --namespace=<ns>
ssh [email protected]                       # Node-Arbeit, am Ende exit

Static-Pod-Manifest sichern, ändern, prüfen, zurückrollen:

M=/etc/kubernetes/manifests
B=~/cks/<tagNN>/backup && mkdir -p $B
sudo cp $M/kube-apiserver.yaml $B/            # NIE ins Manifest-Verzeichnis
sudo vim $M/kube-apiserver.yaml
sudo diff $B/kube-apiserver.yaml $M/kube-apiserver.yaml
sudo crictl ps -a --name kube-apiserver       # bis zu 1 min warten
sudo crictl logs <container-id>
sudo sh -c 'tail -n 2 \
  /var/log/pods/kube-system_kube-apiserver-cp1_*/kube-apiserver/*.log'
sudo journalctl -u kubelet --since -2min | grep -i apiserver | tail -2
k get --raw /readyz; echo
sudo cp $B/kube-apiserver.yaml $M/            # Rückweg
sudo crictl stop $(sudo crictl ps --name kube-apiserver -q)  # Neustart erzwingen
ps -o args= -C kubelet | tr ' ' '\n' | grep -- --config
sudo grep staticPodPath /var/lib/kubelet/config.yaml
  • Das kubelet liest jede Datei im Manifest-Verzeichnis als Pod, egal welche Endung (nur versteckte nicht). Ein .bak daneben ist ein zweiter API-Server-Pod.
  • Nach jeder Änderung 1/1 Ready abwarten, nicht nur Running.

Cluster Setup (Tag 2–4)

Default Deny, DNS erlauben, Metadaten sperren (Tag 2):

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny
spec:
  podSelector: {}
  policyTypes:
  - Ingress
  - Egress
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-dns
spec:
  podSelector: {}
  policyTypes:
  - Egress
  egress:
  - to:
    - namespaceSelector:
        matchLabels:
          kubernetes.io/metadata.name: kube-system
      podSelector:
        matchLabels:
          k8s-app: kube-dns
    ports:
    - port: 53
      protocol: UDP
    - port: 53
      protocol: TCP
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: client-world
spec:
  podSelector:
    matchLabels:
      run: client
  policyTypes:
  - Egress
  egress:
  - to:
    - ipBlock:
        cidr: 0.0.0.0/0
        except:
        - 169.254.169.254/32
from:                           from:
- namespaceSelector: {...}      - namespaceSelector: {...}
- podSelector: {...}              podSelector: {...}
  -> ODER (zwei Einträge)         -> UND (ein Eintrag)

CiliumNetworkPolicy: Entities, L7, FQDN, clusterweites Deny (Tag 2):

apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
  name: client-egress
spec:
  endpointSelector:
    matchLabels:
      run: client
  egress:
  - toEndpoints:
    - matchLabels:
        k8s:io.kubernetes.pod.namespace: kube-system
        k8s-app: kube-dns
    toPorts:
    - ports:
      - port: "53"
        protocol: ANY
      rules:
        dns:
        - matchPattern: "*"
  - toEntities:
    - kube-apiserver
    toPorts:
    - ports:
      - port: "6443"
        protocol: TCP
  - toFQDNs:
    - matchName: kubernetes.io
    toPorts:
    - ports:
      - port: "443"
        protocol: TCP
---
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
  name: web-l7
spec:
  endpointSelector:
    matchLabels:
      app: web
  ingress:
  - fromEndpoints:
    - matchLabels:
        run: client
    toPorts:
    - ports:
      - port: "80"
        protocol: TCP
      rules:
        http:
        - method: GET
          path: "/"
---
apiVersion: cilium.io/v2
kind: CiliumClusterwideNetworkPolicy
metadata:
  name: deny-metadata
spec:
  endpointSelector:
    matchLabels:
      k8s:io.kubernetes.pod.namespace: <ns>
  enableDefaultDeny:
    egress: false
    ingress: false
  egressDeny:
  - toCIDR:
    - 169.254.169.254/32
  • Unter Cilium erfasst ipBlock keine Node-IPs (Identitäten host, remote-node). Für API-Server und Nodes toEntities nutzen. Deny schlägt Allow.
  • k get cnp,ccnp -A zeigt alle Cilium-Policies.

kube-bench (Tag 3):

sudo kube-bench run --targets=master --benchmark=cis-1.12 > master.txt 2>&1
grep "^\[FAIL\]" master.txt | cut -c1-62
sudo kube-bench run --targets=master --benchmark=cis-1.12 --check=1.2.15,1.4.1
sudo kube-bench run --targets=master --benchmark=cis-1.12 --check=1.4.1 \
  2>/dev/null | sed -n '/== Remediations/,/^$/p'
sudo kube-bench run --targets=master --benchmark=cis-1.12 --group=1.4 \
  --noremediations
sudo kube-bench run --targets=node --benchmark=cis-1.12 --json 2>/dev/null \
  | jq .Totals

Typische Remediations aus dem Buch: --profiling=false an API-Server, Controller-Manager und Scheduler; chown -R etcd:etcd /var/lib/etcd; kubelet-Dateien chmod 600 (kubelet.service, 10-kubeadm.conf, /var/lib/kubelet/config.yaml), danach systemctl daemon-reload und restart kubelet.

Ingress mit TLS (Tag 4):

openssl req -x509 -newkey rsa:2048 -nodes -days 365 \
  -keyout web.key -out web.crt -subj "/CN=web.lab.local" \
  -addext "subjectAltName=DNS:web.lab.local"
k create secret tls web-tls --cert=web.crt --key=web.key
k create ingress web --class=traefik --rule="web.lab.local/*=web:80,tls=web-tls"
curl -s -o /dev/null -w "%{http_code}\n" --cacert web.crt \
  --resolve web.lab.local:30443:172.16.0.126 https://web.lab.local:30443/
k get secret web-tls -o jsonpath='{.data.tls\.crt}' | base64 -d \
  | openssl x509 -noout -subject -enddate

Binaries prüfen (Tag 4):

kubelet --version                             # Version exakt ablesen
V=$(kubelet --version | awk '{print $2}')
ARCH=$(dpkg --print-architecture)
U=https://dl.k8s.io/release/$V/bin/linux/${ARCH}
curl -sSLO $U/kubelet.sha512
echo "$(cat kubelet.sha512)  /usr/bin/kubelet" | sha512sum -c
echo "$(curl -sSL $U/kubeadm.sha512)  /usr/bin/kubeadm" | sha512sum -c
P=$(pidof kube-apiserver)                     # laufendes Binary prüfen
echo "$(curl -sSL $U/kube-apiserver.sha512)  /proc/$P/exe" | sudo sha512sum -c

Cluster Hardening (Tag 5, 6)

RBAC prüfen und minimieren (Tag 5):

SA=system:serviceaccount:<ns>:<sa>
k auth can-i --list --as=$SA
k auth can-i get secrets -A --as=$SA
k auth can-i get secret/<name> --as=$SA
k get clusterrolebindings \
  -o jsonpath='{range .items[*]}{.metadata.name} {.subjects[*].name}{"\n"}{end}'
k get rolebindings -o wide
k get clusterrolebinding <crb> -o jsonpath='{.roleRef.kind}/{.roleRef.name}{"\n"}'
k create role <r> --verb=get,list,watch,patch,update --resource=deployments
k create role <r> --verb=get --resource=secrets --resource-name=<secret>
k create rolebinding <rb> --role=<r> --serviceaccount=<ns>:<sa>
k create rolebinding <rb> --clusterrole=view --serviceaccount=<ns>:<sa>
k delete clusterrolebinding <crb>

ServiceAccounts und Tokens (Tag 5):

k patch sa default -p '{"automountServiceAccountToken":false}'
k create token <sa> --duration=10m
k get secrets -A --field-selector type=kubernetes.io/service-account-token
k exec <pod> -- ls /var/run/secrets/kubernetes.io/serviceaccount
  • automountServiceAccountToken: false am ServiceAccount oder in pod.spec; der Wert am Pod gewinnt.
  • kubectl --token=… mit Admin-kubeconfig bleibt Admin (Zertifikat gewinnt). Token testen mit --kubeconfig=/dev/null --server=https://172.16.0.126:6443 --certificate-authority=ca.crt --token=$TOKEN.

Anonymen Zugriff nur für Health-Checks (Tag 6), Datei /etc/kubernetes/authn/authn-config.yaml:

apiVersion: apiserver.config.k8s.io/v1
kind: AuthenticationConfiguration
anonymous:
  enabled: true
  conditions:
  - path: /livez
  - path: /readyz
  - path: /healthz

Im API-Server-Manifest Flag, Mount und Volume (dasselbe Muster gilt für Encryption, Audit und Admission-Konfiguration):

    - --authentication-config=/etc/kubernetes/authn/authn-config.yaml
    # unter volumeMounts:
    - mountPath: /etc/kubernetes/authn
      name: authn
      readOnly: true
  # unter volumes:
  - hostPath:
      path: /etc/kubernetes/authn
      type: Directory
    name: authn
curl -sk https://172.16.0.126:6443/readyz; echo          # ok
curl -sk https://172.16.0.126:6443/version -o /dev/null -w "%{http_code}\n"  # 401
sudo grep -- "--enable-admission-plugins" $M/kube-apiserver.yaml  # NodeRestriction
k get svc kubernetes -o jsonpath='{.spec.type}{"\n"}'    # ClusterIP
curl -sk https://172.16.0.228:10250/pods; echo           # kubelet: Unauthorized
ssh [email protected] \
  'sudo grep -A1 -E "^authorization:|  anonymous:" /var/lib/kubelet/config.yaml'
  • --anonymous-auth=false legt den API-Server in eine Neustart-Schleife (Probes sind anonym). --anonymous-auth und anonymous: in der AuthenticationConfiguration zusammen sind verboten.
  • kubelet: authentication.anonymous.enabled: false, authorization.mode: Webhook.

Härtungen upgradefest: dieselben Flags und Volumes in die ConfigMap kubeadm-config (Tag 6, 10, 15):

k -n kube-system edit cm kubeadm-config
sudo kubeadm upgrade diff v1.37.2 | grep -E "^[-+] |^---"
    apiServer:
      extraArgs:
      - name: profiling
        value: "false"
      - name: authentication-config
        value: /etc/kubernetes/authn/authn-config.yaml
      extraVolumes:
      - name: authn
        hostPath: /etc/kubernetes/authn
        mountPath: /etc/kubernetes/authn
        readOnly: true
        pathType: Directory

Upgrade (Tag 6), Control Plane:

sudo apt-mark unhold kubeadm
sudo apt-get update
sudo apt-get install -y kubeadm=1.37.2-1.1
sudo apt-mark hold kubeadm
sudo kubeadm upgrade plan
sudo kubeadm upgrade apply v1.37.2            # nur erster Control-Plane-Node
k drain cp1 --ignore-daemonsets --delete-emptydir-data
sudo apt-mark unhold kubelet kubectl
sudo apt-get install -y kubelet=1.37.2-1.1 kubectl=1.37.2-1.1
sudo apt-mark hold kubelet kubectl
sudo systemctl daemon-reload
sudo systemctl restart kubelet
k uncordon cp1

Worker: drain (von cp1), auf dem Worker kubeadm-Paket, sudo kubeadm upgrade node, kubelet/kubectl, daemon-reload, restart, dann uncordon. Paket-Updates setzen Dateirechte von kubelet.service und 10-kubeadm.conf auf 644 zurück.

System Hardening (Tag 7, 8)

Host minimieren (Tag 7):

sudo ss -tulpn                                # ohne sudo keine Prozesse
sudo ss -tlnp 'sport = :111'
P=$(pidof <prozess>)
sudo readlink /proc/$P/exe
ps -o unit= -p $P                             # systemd-Unit
dpkg -S /usr/sbin/<binary>                    # Paket
sudo systemctl disable --now <dienst>
sudo systemctl disable --now rpcbind.socket rpcbind.service
sudo systemctl mask rpcbind.socket rpcbind.service
sudo apt-get purge -y <paket>
sudo apt-get autoremove --purge -y
awk -F: '$3==0' /etc/passwd
getent passwd | awk -F: '$7 !~ /(nologin|false)$/ {print $1, $3, $7}'
sudo grep -rn NOPASSWD /etc/sudoers /etc/sudoers.d
sudo usermod -L -s /usr/sbin/nologin <user>
sudo userdel -r <user>
sudo find / -xdev -type f -perm -4000 -not -path '/var/lib/containerd/*' 2>/dev/null
sudo chmod u-s <datei>
echo 'install dccp /bin/false' | sudo tee -a /etc/modprobe.d/cks-blacklist.conf
sudo modprobe -n -v dccp

AppArmor (Tag 8), Profil auf jedem möglichen Node laden:

sudo apparmor_parser -r /etc/apparmor.d/cks-deny-write
sudo aa-status | grep cks
sudo journalctl -k --since -5min | grep 'apparmor="DENIED"'
k exec <pod> -- cat /proc/1/attr/current; echo
    securityContext:
      appArmorProfile:
        type: Localhost
        localhostProfile: cks-deny-write    # Profilname, nicht Dateipfad

seccomp (Tag 8), Profil unter /var/lib/kubelet/seccomp/profiles/no-mkdir.json:

{
  "defaultAction": "SCMP_ACT_ALLOW",
  "syscalls": [
    {
      "names": ["mkdir", "mkdirat"],
      "action": "SCMP_ACT_ERRNO"
    }
  ]
}
  securityContext:
    seccompProfile:
      type: Localhost
      localhostProfile: profiles/no-mkdir.json  # relativ zu /var/lib/kubelet/seccomp
k exec <pod> -- grep Seccomp: /proc/1/status  # 2 = Filter aktiv, 0 = keiner

type bei beiden: RuntimeDefault, Localhost oder Unconfined. Fehlt das Profil auf dem Node: CreateContainerError, Name steht in k describe pod.

Minimize Microservice Vulnerabilities (Tag 9–12)

Pod Security Admission (Tag 9):

k label --dry-run=server --overwrite ns <ns> \
  pod-security.kubernetes.io/enforce=restricted     # zeigt verletzende Pods
k label --overwrite ns <ns> \
  pod-security.kubernetes.io/enforce=restricted \
  pod-security.kubernetes.io/enforce-version=v1.37 \
  pod-security.kubernetes.io/warn=restricted \
  pod-security.kubernetes.io/audit=restricted
k get ns -l pod-security.kubernetes.io/enforce=restricted
k get events --field-selector reason=FailedCreate -o jsonpath='{.items[0].message}'

SecurityContext für restricted:

spec:
  securityContext:
    runAsNonRoot: true
    runAsUser: 1000          # nötig, wenn das Image als root startet
    seccompProfile:
      type: RuntimeDefault
  containers:
  - name: app
    securityContext:
      allowPrivilegeEscalation: false
      capabilities:
        drop: ["ALL"]

Gefährliche Pods finden (Tag 9):

k get pods -A -o json | jq -r '.items[]
  | select(.spec.hostNetwork or .spec.hostPID or .spec.hostIPC)
  | .metadata.namespace + "/" + .metadata.name'
k get pods -A -o json | jq -r '.items[]
  | select(any(.spec.containers[], .spec.initContainers[]?;
      .securityContext.privileged))
  | .metadata.namespace + "/" + .metadata.name'

Secrets verschlüsseln (Tag 10), Datei /etc/kubernetes/enc/enc.yaml (chmod 600), Flag --encryption-provider-config=/etc/kubernetes/enc/enc.yaml plus Mount und Volume:

apiVersion: apiserver.config.k8s.io/v1
kind: EncryptionConfiguration
resources:
- resources:
  - secrets
  providers:
  - aescbc:                  # erster Provider schreibt
      keys:
      - name: key1
        secret: <head -c 32 /dev/urandom | base64>
  - identity: {}             # liest Altbestand
P=/etc/kubernetes/pki
E="--cacert=$P/etcd/ca.crt --cert=$P/apiserver-etcd-client.crt"
E="$E --key=$P/apiserver-etcd-client.key"
sudo etcdctl $E get /registry/secrets/<ns>/<name> --print-value-only | head -c 23
k get secrets -A -o json | k replace -f -     # Bestand neu verschlüsseln
sudo etcdctl $E get /registry/secrets --prefix --print-value-only \
  | grep -ac "^k8s:enc:aescbc"

Rotation: neuen Schlüssel oben einfügen, API-Server neu starten, alle Secrets ersetzen, alten Schlüssel entfernen, erneut neu starten.

Sandbox mit RuntimeClass (Tag 11):

sudo crictl info | jq -r '.config.containerd.runtimes | keys[]'   # Handler-Name
apiVersion: node.k8s.io/v1
kind: RuntimeClass
metadata:
  name: gvisor
handler: runsc
scheduling:
  nodeSelector:
    sandbox: gvisor
---
# im Pod bzw. Pod-Template:
spec:
  runtimeClassName: gvisor
k exec <pod> -- dmesg | head -1               # gVisor meldet sich
k exec <pod> -- uname -r                      # anderer Kernel als der Node

Pod-zu-Pod-Verschlüsselung mit Cilium-WireGuard (Tag 12):

helm get values cilium -n kube-system -o yaml > cilium-values-backup.yaml
cilium upgrade --version 1.20.5 --set encryption.enabled=true \
  --set encryption.type=wireguard
k -n kube-system rollout restart ds/cilium
k -n kube-system rollout status ds/cilium --timeout=300s
k -n kube-system exec ds/cilium -c cilium-agent -- cilium-dbg encrypt status
k -n kube-system exec ds/cilium -c cilium-agent -- cilium-dbg status | grep Encryption
ssh [email protected] 'sudo timeout 10 tcpdump -lni eth0 -A udp port 8472'

Ohne Neustart der Agents bleibt encrypt status auf Disabled. Mit WireGuard läuft Pod-Verkehr über UDP 51871 statt im Klartext über VXLAN (UDP 8472).

Supply Chain Security (Tag 13, 14)

Trivy und SBOM (Tag 13):

trivy image -q --scanners vuln <image> | grep ^Total
trivy image -q --scanners vuln --severity HIGH,CRITICAL --ignore-unfixed <image>
trivy image -q --scanners vuln --severity CRITICAL --exit-code 1 <image>
trivy image -q --scanners vuln --insecure <registry-ohne-tls>/<image>
trivy image -q --input <archiv>.tar
trivy config -q --severity HIGH,CRITICAL --exit-code 1 <manifest-oder-ordner>
trivy image -q -f cyclonedx -o app.cdx.json <image>
trivy image -q -f spdx-json -o app.spdx.json <image>
trivy sbom -q app.cdx.json | grep ^Total
bom generate --image <image> --format json --output app.bom.spdx.json
for i in $(k get pods -o jsonpath='{.items[*].spec.containers[*].image}' \
  | tr ' ' '\n' | sort -u); do
  echo "== $i"; trivy image -q --scanners vuln --severity CRITICAL $i | grep ^Total
done

Minimales Image: FROM gcr.io/distroless/static-debian13:nonroot (oder scratch), USER 65532:65532, keine Shell, kein Paketmanager. Debuggen per printf 'securityContext:\n runAsUser: 65532\n' > dbg.yaml, dann k debug <pod> --image=busybox:1.37 --target=<container> --profile=restricted --custom=dbg.yaml.

Erlaubte Registry per ValidatingAdmissionPolicy (Tag 14), Policy und Binding:

apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingAdmissionPolicy
metadata:
  name: allowed-registries
spec:
  failurePolicy: Fail
  matchConstraints:
    resourceRules:
    - apiGroups: [""]
      apiVersions: ["v1"]
      operations: ["CREATE", "UPDATE"]
      resources: ["pods"]
  validations:
  - expression: >-
      object.spec.containers.all(c, c.image.startsWith('172.16.0.126:30500/'))
      && (!has(object.spec.initContainers) ||
      object.spec.initContainers.all(c, c.image.startsWith('172.16.0.126:30500/')))
    message: "Nur Images aus 172.16.0.126:30500/ erlaubt"
  - expression: "object.spec.containers.all(c, c.image.contains('@sha256:'))"
    messageExpression: >-
      "Image ohne Digest: " + object.spec.containers.map(c, c.image).join(", ")
---
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingAdmissionPolicyBinding
metadata:
  name: allowed-registries
spec:
  policyName: allowed-registries
  validationActions: ["Deny"]
  matchResources:
    namespaceSelector:
      matchLabels:
        kubernetes.io/metadata.name: <ns>
k get pod <p> -o jsonpath='{.status.containerStatuses[0].imageID}{"\n"}'  # Digest
k get validatingadmissionpolicy <name> -o yaml   # CEL-Fehler: status.typeChecking

cosign (Tag 14; HTTP-Registry nur im Labor):

export COSIGN_PASSWORD=lab-cosign-pw
cosign generate-key-pair
cosign signing-config create --out signing-config.json
cosign sign --key cosign.key --signing-config signing-config.json \
  --allow-http-registry --yes <image>@sha256:<digest>
cosign verify --key cosign.pub --allow-http-registry --insecure-ignore-tlog <image>

ImagePolicyWebhook (Tag 14): ImagePolicyWebhook in --enable-admission-plugins, --admission-control-config-file=/etc/kubernetes/admission/admission-config.yaml, Verzeichnis mounten.

apiVersion: apiserver.config.k8s.io/v1
kind: AdmissionConfiguration
plugins:
- name: ImagePolicyWebhook
  configuration:
    imagePolicy:
      kubeConfigFile: /etc/kubernetes/admission/imagepolicy.kubeconfig
      allowTTL: 50
      denyTTL: 50
      retryBackoff: 500
      defaultAllow: false    # "deny if backend unreachable"

Statische Analyse (Tag 14):

kubesec scan notes.yaml | jq -r '.[0] | .score, .message'
kubesec scan notes.yaml | jq -r '.[0].scoring.critical[] | .id + ": " + .selector'
kube-linter lint notes.yaml

Monitoring, Logging and Runtime Security (Tag 15–18)

Audit-Policy (Tag 15), Datei /etc/kubernetes/audit/policy.yaml – erste passende Regel gilt:

apiVersion: audit.k8s.io/v1
kind: Policy
omitStages:
- RequestReceived
rules:
- level: None
  nonResourceURLs: ["/healthz*", "/livez*", "/readyz*", "/version"]
- level: None
  resources:
  - group: coordination.k8s.io
    resources: ["leases"]
- level: None
  userGroups: ["system:nodes"]
  verbs: ["get", "list", "watch"]
- level: None
  verbs: ["watch"]
- level: Metadata
  resources:
  - group: ""
    resources: ["secrets", "configmaps", "serviceaccounts/token"]
- level: RequestResponse
  resources:
  - group: ""
    resources: ["pods", "pods/exec"]
  verbs: ["create", "update", "patch", "delete"]
- level: Metadata

Fünf Flags, zwei Volumes (Policy readOnly, Log-Verzeichnis DirectoryOrCreate):

    - --audit-policy-file=/etc/kubernetes/audit/policy.yaml
    - --audit-log-path=/var/log/kubernetes/audit/audit.log
    - --audit-log-maxage=7
    - --audit-log-maxbackup=2
    - --audit-log-maxsize=20
L=/var/log/kubernetes/audit/audit.log
sudo jq -c 'select(.objectRef.resource=="secrets" and .objectRef.name=="<s>")
  | [.user.username, .verb, .responseStatus.code, .sourceIPs[0]]' $L
sudo jq -c 'select(.verb=="delete" and .objectRef.resource=="pods")
  | [.user.username, .objectRef.name, .stageTimestamp]' $L
sudo jq -r 'select(.responseStatus.code==403) | .user.username' $L | sort | uniq -c

Falco (Tag 16), eigene Regel und Override (Helm customRules oder auf dem Node /etc/falco/falco_rules.local.yaml, nie falco_rules.yaml):

- list: etc_writer
  items: [touch, tee, cp, mv, sed, vi]
- rule: Write below etc in container
  desc: Ein Prozess im Container schreibt unter /etc
  condition: >
    open_write and fd.name startswith /etc/ and container
    and proc.name in (etc_writer)
  output: >
    Schreibzugriff unter /etc (file=%fd.name process=%proc.name
    user=%user.name container=%container.name)
  priority: ERROR
- rule: Terminal shell in container
  output: "%evt.time,%container.id,%container.name,%user.name"
  override:
    output: replace
- rule: Contact K8S API Server From Container
  enabled: false
  override:
    enabled: replace
k get pod <p> -o jsonpath='{.spec.nodeName}'; echo
F=$(k -n falco get pod -l app.kubernetes.io/name=falco \
  --field-selector spec.nodeName=<node> -o name)
k -n falco logs $F -c falco --since=5m | grep <stichwort>
k -n falco exec $F -c falco -- falco --list | grep '^container\.'
k -n falco exec $F -c falco -- falco -l 'Terminal shell in container'
k -n falco exec $F -c falco -- falco --dry-run      # Syntaxtest der Regeln
k -n falco exec $F -c falco -- grep -A3 '^rules_files' /etc/falco/falco.yaml

Als Systemdienst (nicht live geprüft): journalctl -u falco-modern-bpf, nach Änderung sudo systemctl restart falco-modern-bpf.

Tetragon (Tag 17), TracingPolicyNamespaced mit Durchsetzung:

apiVersion: cilium.io/v1alpha1
kind: TracingPolicyNamespaced
metadata:
  name: shadow-kill
  namespace: <ns>
spec:
  kprobes:
  - call: security_file_permission
    syscall: false
    args:
    - index: 0
      type: file
    - index: 1
      type: int
    selectors:
    - matchArgs:
      - index: 0
        operator: Equal
        values:
        - /etc/shadow
      matchActions:
      - action: Sigkill
T=$(k -n kube-system get pod -l app.kubernetes.io/name=tetragon \
  --field-selector spec.nodeName=<node> -o name)
k -n kube-system exec $T -c tetragon -- tetra tracingpolicy list
k -n kube-system exec $T -c tetragon -- \
  tetra getevents -o compact --namespace <ns> --process curl

Override mit argError: -1 lädt im Labor nur an Syscalls (call: sys_openat, syscall: true); an security_file_permission endet es in load_error, ohne dass k apply meckert. podSelector grenzt die Policy auf Pods ein.

Unveränderliche Container (Tag 18):

    securityContext:
      readOnlyRootFilesystem: true
    volumeMounts:
    - name: tmp
      mountPath: /tmp
  volumes:
  - name: tmp
    emptyDir: {}
k get pods -o json | jq -r '.items[]
  | select(any(.spec.containers[]; .securityContext.readOnlyRootFilesystem != true))
  | .metadata.name'

Spurensuche auf dem Node mit crictl und /proc (Tag 18):

ID=$(sudo crictl ps --namespace <ns> --name <container> -q)
PID=$(sudo crictl inspect -o go-template --template '{{.info.pid}}' $ID)
U=$(sudo grep -o -m1 'upperdir=[^,]*' /proc/$PID/mounts | cut -d= -f2)
sudo find $U -type f -printf '%P\n'           # was im Container geschrieben wurde
ps -o pid,ppid,user,cmd --ppid $PID
sudo ls -l /proc/<pid>/exe
sudo cat /proc/<pid>/cmdline | tr '\0' ' '; echo
sudo nsenter -t <pid> -n ss -tlnp             # Ports im Netz-Namespace des Pods
CID=$(grep -o 'cri-containerd-[0-9a-f]*' /proc/<pid>/cgroup | cut -d- -f3)
sudo crictl ps --id $CID -o json | jq -r '.containers[0] | .metadata.name + " "
  + .labels["io.kubernetes.pod.name"] + " " + .labels["io.kubernetes.pod.namespace"]'

Isolieren, sichern, entfernen: k label pod <p> app- quarantine=true, Deny-all-Policy auf quarantine: "true", k get pod <p> -o yaml > evidence/pod.yaml, k logs <p> > evidence/pod.log, dann den Besitzer (Deployment) skalieren oder reparieren, nicht nur den Pod löschen.

Symptom und erster Befehl

Symptom Erster Befehl
kubectl hängt nach Manifest-Änderung sudo crictl ps -a --name kube-apiserver
API-Server-Container Exited sudo crictl logs <id>, Backup zurück
API-Server startet, Pod 0/1 k -n kube-system get events, Probes
RuntimeClass "…" not found k get runtimeclass
CreateContainerError k describe pod: AppArmor/seccomp-Profil
ContainerCreating mit Sandbox-Fehler Events FailedCreatePodSandBox: Handler
Deployment 0/1, keine Pods Events FailedCreate: PodSecurity, VAP
CreateContainerConfigError runAsNonRoot und Image-User
Pod startet nicht, read-only FS k logs: Schreibpfad, emptyDir mounten
Policy ohne Wirkung k get netpol,cnp,ccnp -A, Labels, Namespace
encrypt status: Disabled k -n kube-system rollout restart ds/cilium
Falco startet nicht k -n falco logs <pod> -c falco --previous
TracingPolicy ohne Ereignisse tetra tracingpolicy list: Zustand
Secret in etcd lesbar Provider-Reihenfolge, k replace vergessen

Doku-Pfade

Alle kubernetes.io-Pfade beginnen mit https://kubernetes.io/docs/.

Thema Pfad
Static Pods tasks/configure-pod-container/static-pod/
crictl tasks/debug/debug-cluster/crictl/
NetworkPolicy concepts/services-networking/network-policies/
Ingress, TLS concepts/services-networking/ingress/
Cluster absichern tasks/administer-cluster/securing-a-cluster/
RBAC reference/access-authn-authz/rbac/
ServiceAccounts tasks/configure-pod-container/configure-service-account/
Authentifizierung, anonym reference/access-authn-authz/authentication/
kubelet authn/authz reference/access-authn-authz/kubelet-authn-authz/
Admission Controllers reference/access-authn-authz/admission-controllers/
VAP reference/access-authn-authz/validating-admission-policy/
kubeadm-Upgrade tasks/administer-cluster/kubeadm/kubeadm-upgrade/
kube-apiserver-Flags reference/command-line-tools-reference/kube-apiserver/
AppArmor tutorials/security/apparmor/
seccomp tutorials/security/seccomp/
SecurityContext tasks/configure-pod-container/security-context/
Pod Security Standards concepts/security/pod-security-standards/
Pod Security Admission concepts/security/pod-security-admission/
PSA-Labels tasks/configure-pod-container/enforce-standards-namespace-labels/
Secrets concepts/configuration/secret/
Encryption at Rest tasks/administer-cluster/encrypt-data/
RuntimeClass concepts/containers/runtime-class/
Auditing tasks/debug/debug-cluster/audit/
Ephemere Container tasks/debug/debug-application/debug-running-pod/

Weitere Projekte (jeweils mit https:// davor):

Thema URL
Binaries, Prüfsummen kubernetes.io/releases/download/
Cilium Policy-Sprache docs.cilium.io/en/stable/security/policy/language/
Cilium WireGuard docs.cilium.io/en/stable/security/network/encryption-wireguard/
kube-bench github.com/aquasecurity/kube-bench/blob/main/docs/running.md
gVisor mit containerd gvisor.dev/docs/user_guide/containerd/quick_start/
Trivy trivy.dev/docs/latest/
bom github.com/kubernetes-sigs/bom
cosign docs.sigstore.dev/cosign/signing/signing_with_containers/
kubesec, KubeLinter kubesec.io/, docs.kubelinter.io/
Falco Regeln falco.org/docs/concepts/rules/basic-elements/
Falco Overrides falco.org/docs/concepts/rules/overriding/
Falco Felder falco.org/docs/reference/rules/supported-fields/
Tetragon TracingPolicy tetragon.io/docs/concepts/tracing-policy/
Tetragon Selectors tetragon.io/docs/concepts/tracing-policy/selectors/

Welche dieser Seiten in der Prüfung geöffnet werden dürfen, steht in den aktuellen Prüfungshinweisen der Linux Foundation.