Current State: 2019 Third Party Security Audit of Kubernetes

Authors (in alphabetical order): Cailyn Edwards (Shopify), Pushkar Joglekar (VMware), Rey Lejano (SUSE) and Rory McCune (DataDog)

We expect the brand new Third Party Security Audit of Kubernetes will be published later this month (Oct 2022).

In preparation for that, let's look at the state of findings that were made public as part of the last third party security audit of 2019 that was based on Kubernetes v1.13.4.

Motivation

Craig Ingram has graciously attempted over the years to keep track of the status of the findings reported in the last audit in this issue: kubernetes/kubernetes#81146. This blog post will attempt to dive deeper into this, address any gaps in tracking and become a point in time summary of the state of the findings reported from 2019.

This article should also help readers gain confidence through transparent communication, of work done by the community to address these findings and bubble up any findings that need help from community contributors.

Current State

The status of each issue / finding here is represented in a best effort manner. Authors do not claim to be 100% accurate on the status and welcome any corrections or feedback if the current state is not reflected accurately by commenting directly on the relevant issue.

# Title Issue Status
1 hostPath PersistentVolumes enable PodSecurityPolicy bypass #81110 closed, addressed by kubernetes/website#15756 and kubernetes/kubernetes#109798
2 Kubernetes does not facilitate certificate revocation #81111 duplicate of #18982 and needs a KEP
3 HTTPS connections are not authenticated #81112 Largely left as an end user exercise in setting up the right configuration
4 TOCTOU when moving PID to manager's cgroup via kubelet #81113 Requires Node access for successful exploitation. Fix needed
5 Improperly patched directory traversal in kubectl cp #76788 closed, assigned CVE-2019-11249, fixed in #80436
6 Bearer tokens are revealed in logs #81114 closed, assigned CVE-2019-11250, fixed in #81330
7 Seccomp is disabled by default #81115 closed, addressed by #101943
8 Pervasive world-accessible file permissions #81116 #112384 ( in progress)
9 Environment variables expose sensitive data #81117 closed, addressed by #84992 and #84677
10 Use of InsecureIgnoreHostKey in SSH connections #81118 This feature was removed in v1.22: #102297
11 Use of InsecureSkipVerify and other TLS weaknesses #81119 Needs a KEP
12 kubeadm performs potentially-dangerous reset operations #81120 closed, fixed by #81495, #81494, and kubernetes/website#15881
13 Overflows when using strconv.Atoi and downcasting the result #81121 closed, fixed by #89120
14 kubelet can cause an Out of Memory error with a malicious manifest #81122 closed, fixed by #76518
15 kubectl can cause an Out Of Memory error with a malicious Pod specification #81123 Fix needed
16 Improper fetching of PIDs allows incorrect cgroup movement #81124 Fix needed
17 Directory traversal of host logs running kube-apiserver and kubelet #81125 closed, fixed by #87273
18 Non-constant time password comparison #81126 closed, fixed by #81152
19 Encryption recommendations not in accordance with best practices #81127 Work in Progress
20 Adding credentials to containers by default is unsafe #81128 Closed, fixed by #89193
21 kubelet liveness probes can be used to enumerate host network #81129 Needs a KEP
22 iSCSI volume storage cleartext secrets in logs #81130 closed, fixed by #81215
23 Hard coded credential paths #81131 closed, awaiting more evidence
24 Log rotation is not atomic #81132 Fix needed
25 Arbitrary file paths without bounding #81133 Fix needed.
26 Unsafe JSON construction #81134 Partially fixed
27 kubelet crash due to improperly handled errors #81135 Closed. Fixed by #81135
28 Legacy tokens do not expire #81136 closed, fixed as part of #70679
29 CoreDNS leaks internal cluster information across namespaces #81137 Closed, resolved with CoreDNS v1.6.2. #81137 (comment)
30 Services use questionable default functions #81138 Fix needed
31 Incorrect docker daemon process name in container manager #81139 closed, fixed by #81083
32 Use standard formats everywhere #81140 Needs a KEP
33 Superficial health check provides false sense of safety #81141 closed, fixed by #81319
34 Hardcoded use of insecure gRPC transport #81142 Needs a KEP
35 Incorrect handling of Retry-After #81143 closed, fixed by #91048
36 Incorrect isKernelPid check #81144 closed, fixed by #81086
37 Kubelet supports insecure TLS ciphersuites #81145 closed but fix needed for #91444 (see this comment)

Inspired outcomes

Apart from fixes to the specific issues, the 2019 third party security audit also motivated security focussed enhancements in the next few releases of Kubernetes. One such example is Kubernetes Enhancement Proposal (KEP) 1933 Defend Against Logging Secrets via Static Analysis to prevent exposing secrets to logs with Patrick Rhomberg driving the implementation. As a result of this KEP, go-flow-levee, a taint propagation analysis tool configured to detect logging of secrets, is executed in a script as a Prow presubmit job. This KEP was introduced in v1.20.0 as an alpha feature, then graduated to beta in v1.21.0, and graduated to stable in v1.23.0. As stable, the analysis runs as a blocking presubmit test. This KEP also helped resolve the following issues from the 2019 third party security audit:

Remaining Work

Many of the 37 findings identified were fixed by work from our community members over the last 3 years. However, we still have some work left to do. Here's a breakdown of remaining work with rough estimates on time commitment, complexity and benefits to the ecosystem on fixing these pending issues.

Title Issue Time Commitment Complexity Benefit to Ecosystem
Kubernetes does not facilitate certificate revocation #81111 High High Medium
Use of InsecureSkipVerify and other TLS weaknesses #81119 High High Medium
kubectl can cause a local Out Of Memory error with a malicious Pod specification #81123 Medium Medium Medium
Improper fetching of PIDs allows incorrect cgroup movement #81124 Medium Medium Medium
kubelet liveness probes can be used to enumerate host network #81129 High High Medium
API Server supports insecure TLS ciphersuites #81145 Medium Medium Low
TOCTOU when moving PID to manager's cgroup via kubelet #81113 Medium Medium Low
Log rotation is not atomic #81132 Medium Medium Low
Arbitrary file paths without bounding #81133 Medium Medium Low
Services use questionable default functions #81138 Medium Medium Low
Use standard formats everywhere #81140 High High Very Low
Hardcoded use of insecure gRPC transport #81142 High High Very Low

To get started on fixing any of these findings that need help, please consider getting involved in Kubernetes SIG Security by joining our bi-weekly meetings or hanging out with us on our Slack Channel.