Kubernetes Security Best Practices. Enhancing Runtime Security and Policy Enforcement (3 of 3)

Introduction

Welcome to the third part of our Kubernetes Security Best Practices series!

After laying a solid foundation by hardening our Kubernetes cluster and application environment in the previous parts, we will now focus on securing the runtime environment and implementing robust policy enforcement within our cluster.

In this part, we will take a deep dive into Seccomp profiles and AppArmor, two essential Linux kernel security features used in containerized environments. Seccomp (Secure Computing Mode) restricts the system calls a process can make, limiting its access to the kernel, while AppArmor (Application Armor) provides mandatory access controls to confine programs to a limited set of resources.

Next, we will explore Falco, a cloud-native runtime security project from the Cloud Native Computing Foundation (CNCF) which continuously monitors container, application, host, and network activity, alerting on behaviors that violate predefined rules.

Lastly, we will delve into Open Policy Agent (OPA), an open-source, general-purpose policy engine that enables unified, context-aware policy enforcement across your entire stack.

By combining these approaches, we can greatly enhance the security posture of our Kubernetes clusters. Our goal is to provide a robust framework that allows us to protect, monitor, and respond to any potential threats in our runtime environment, while ensuring compliance with our defined policies.

Join us as we navigate these important aspects of Kubernetes security, further strengthening our clusters against potential threats.

• Hardening Kubernetes Cluster

• Hardening Application Environment

• Enhancing Runtime Security and Policy Enforcement

Seccomp in Kubernetes

Seccomp, or Secure Computing Mode, is a Linux kernel feature that allows a process to specify a filter for incoming system calls. It's a sandboxing tool that provides a means of limiting the system calls a process can make, reducing the kernel's attack surface and adding an extra layer of security.

Containers share the host system's kernel with other containers and the host itself. If a container's process could make any system call, it could potentially exploit a vulnerability in the kernel, leading to a container breakout and compromising the host system.

Seccomp adds an extra layer of security by limiting the system calls a container can make. It enables processes to either define a whitelist, specifying only the system calls necessary for proper execution, or a blacklist, naming the system calls that are forbidden. In the whitelist approach, any system calls made that aren't listed will result in the kernel terminating the process. Alternatively, in the blacklist approach, only the specified system calls are prohibited, while all others are permitted, which can be useful in scenarios where blocking specific known risky system calls is necessary.

Seccomp rules are defined in profiles written in JSON format and specify an array of sys calls names or numbers that the process is allowed (in whitelist format) or denied (in a blacklist format) to make. Here is a very simple example of a seccomp profile:

{
  "defaultAction": "SCMP_ACT_ERRNO",
  "architectures": [
    "SCMP_ARCH_X86_64",
    "SCMP_ARCH_X86",
    "SCMP_ARCH_X32"
  ],
  "syscalls": [
    {
      "names": [
        "write",
        "exit",
        "exit_group"
      ],
      "action": "SCMP_ACT_ALLOW"
    }
  ]
}

In this example defaultAction is SCMP_ACT_ERRNO, which means that any sys call not explicitly allowed will return an error. It is a white list model. If we set value to SCMP_ACT_ALLOW, only listed syscalls will be denied. It is a black list model. architectures specifies the architectures that the profile applies to. syscalls is an array of syscalls objects, and each syscall object has a name property and action property. In this seccomp profile only "write", "exit", and "exit_group" syscalls are allowed.

In Kubernetes, you can specify the seccomp profile for a Pod or specific container. Here is how you can specify a seccomp profile:

apiVersion: v1
kind: Pod
metadata:
  name: seccomp-demo
spec:
  securityContext:
    seccompProfile:
      type: RuntimeDefault
  containers:
  - name: redis
    image: redis:6.2.5
    securityContext:
      seccompProfile:
        type: Unconfined

This feature provides you with more control over your containers' security, allowing you to restrict the system calls a container can make, enhancing the overall security of your Kubernetes cluster. Remember, it's always a good practice to define the minimal set of system calls required for your application to work, thereby following the principle of least privilege.

AppArmor in Kubernetes

AppArmor (Application Armor) is a Linux kernel security module that supplements the standard Linux user and group-based permissions model by restricting programs to certain resources. It is a Mandatory Access Control (MAC) system, which allows administrators to specify, with a high level of granularity, the files, capabilities, network access, and more that are available to applications.

AppArmor profiles are written in a simple text format and specify the actions that are allowed for files, networking, and more. Here is a very simple example:

#include <tunables/global>
profile example-profile flags=(attach_disconnected,mediate_deleted) {
  #include <abstractions/base>
  
  file,
  network,
  capability,
  umount,
  
  /path/to/file r,
  /path/to/other/file w,
  
  deny /path/to/file/to/deny rw,
}

In this example, the profile is named example-profile and allows for reading (r) of /path/to/file, writing (w) to /path/to/other/file, and denies read/write (rw) access to /path/to/file/to/deny. The #include <abstractions/base> line includes a base set of permissions that are typically needed for most applications to function properly.

In Kubernetes, you can specify the AppArmor profile for a Pod or a specific container in the Pod's security context. Here's how you can specify an AppArmor profile:

apiVersion: v1
kind: Pod
metadata:
  name: apparmor-demo
  annotations:
    container.apparmor.security.beta.kubernetes.io/nginx: localhost/example-profile
spec:
  containers:
  - name: nginx
    image: nginx:1.19

In this example, the container.apparmor.security.beta.kubernetes.io/nginx annotation is set to localhost/example-profile, which means that the nginx container will use the example-profile AppArmor profile that was loaded on the host.

Remember, as with any security measures, to achieve the best results, you should tailor your AppArmor profiles according to the specific needs and behaviors of your applications, thereby following the principle of least privilege.

Falco in Kubernetes

Falco is an open-source project originally created by Sysdig and now hosted by the Cloud Native Computing Foundation (CNCF). It's a behavioral activity monitoring tool designed to detect anomalous activity in your applications. Powered by Sysdig’s system call capture infrastructure, Falco lets administrators define highly granular rules to check for activities involving file and network activity, process execution, IPC, and more.

In the dynamic world of containers and microservices, detecting and responding to threats at runtime is a crucial aspect of security. Traditional methods of threat detection are often ill-suited for the ephemeral nature of containers. This is where Falco steps in.

Falco taps into the Linux kernel and keeps a watchful eye on all system calls, the low-level invocations used by applications to request action from the kernel. By monitoring these system calls, Falco can provide deep visibility into system and application behavior and detect anomalous activity indicative of a breach or attack.

Falco can be deployed as a DaemonSet in your Kubernetes cluster, ensuring a running instance on each node for cluster-wide coverage. Here's an example of how to install Falco using Helm:

helm repo add falcosecurity https://falcosecurity.github.io/charts
helm install falco falcosecurity/falco

Once installed, Falco begins monitoring system calls, applying a set of default rules that look for a range of suspicious behavior, including:

• Unexpected spawn of a binary or shell by a container

• Attempts to change namespaces, a potential sign of container escape attempts

• Write to a non-standard directory

• Use of unexpected outbound network connections

These rules can be customized based on your specific needs, and you can create new ones based on the application and environment behaviors.

Alerts can be output to standard error, a file, or Syslog. But more commonly in a cloud-native environment, alerts might be routed to other systems like Prometheus, Grafana, or a SIEM system for further analysis.

Falco serves as a powerful tool to enhance the security in your Kubernetes environment, providing runtime security and intrusion detection capabilities. With its flexible rule-set, you can define the behaviors and activities to keep an eye on, thereby ensuring that any suspicious activity in your environment doesn't go unnoticed.

Open Policy Agent in Kubernetes

The Open Policy Agent (OPA), a project hosted by the Cloud Native Computing Foundation (CNCF), is a general-purpose policy engine with uses ranging from authorization and admission control to data filtering. OPA provides a high-level declarative language (Rego) for policy decision-making, allowing you to offload policy decisions from your software.

Managing policies and access control in a complex, microservices-based environment like Kubernetes can be challenging. OPA offers a unified, context-aware policy enforcement across the stack. It allows policy decisions to be made in a consistent way, regardless of the technology or components involved, which greatly simplifies policy management and ensures more reliable enforcement.

OPA can be used as an Admission Controller in a Kubernetes environment. In this context, OPA makes admission decisions, determining whether certain API requests should be allowed. OPA's policies dictate what is and isn't permitted, such as which users can access certain resources, how resources can be changed, and so forth.

Here's a simple example of how you can use OPA to enforce a policy that ensures all namespaces have a "label" set:

package kubernetes.admission
deny[msg] {
  input.request.kind.kind = "Namespace"
  not input.request.object.metadata.labels
  msg := "all namespaces must have a label"
}

In this example, if a request to create a Namespace does not include a label, the deny rule triggers, and the message "all namespaces must have a label" is returned, causing Kubernetes to deny the request.

To further simplify the process of managing and enforcing policies in Kubernetes, you can use OPA in conjunction with Gatekeeper. Gatekeeper is an open-source project that provides a more Kubernetes-native OPA integration, adding a set of CRDs that you can use to define and enforce policies.

With OPA and Gatekeeper, you can effectively decouple policy decision-making from policy enforcement. This allows you to centralize policy management and simplify enforcement across all your Kubernetes clusters.

In conclusion, OPA offers a powerful, flexible, and unified approach to policy enforcement in a Kubernetes environment. It allows for fine-grained, context-aware policy decisions, which is crucial in the complex and dynamic world of containerized microservices.

Conclusion

As we conclude our series on Kubernetes Security Best Practices, it's worth recapping the broad landscape we've covered, emphasizing the fact that Kubernetes security is multi-faceted and requires a thorough, layered approach.

In the first part of the series, "Hardening the Kubernetes Cluster," we explored vital aspects such as securing the Kubernetes API server, kubelet, and etcd encryption. We focused on the foundational aspects of securing your cluster at the infrastructure level.

In the second installment, "Hardening the Application Environment," we delved into application-level security practices including network policies, RBAC, Security Contexts, Runtime Classes, and Admission Controllers. These practices help to ensure a secure environment for your applications to run.

Finally, in this last part, "Enhancing Security with Advanced Tools and Practices," we examined advanced topics such as Seccomp, AppArmor, Falco, and Open Policy Agent. These tools and practices offer additional layers of protection, improving the overall security posture of your Kubernetes environments.

But our journey does not end here. Security is a constant process of learning and adaptation. Tools such as kube-bench, which can perform a CIS Kubernetes Benchmark test, are invaluable for regularly auditing your cluster and providing best-practice recommendations for configuration and setup.

Looking ahead, we'll continue to explore more topics related to Kubernetes security. Remember, the best security strategy is a proactive one. Continuously monitor and assess your environments, stay informed about the latest threats and mitigation techniques, and adapt your practices accordingly.

Stay tuned for more deep dives and discussions on how to secure your Kubernetes deployments effectively and efficiently. Thank you for following along this series on Kubernetes Security Best Practices. We hope it has been instructive and beneficial for your Kubernetes journey.