Secure exposing Kubernetes resources to the Internet

As organizations increasingly adopt cloud-based technologies and distributed development workflows, exposing internal resources such as Kubernetes clusters has become a necessity.

In this article, we will explore the challenges associated with exposing internal resources and discuss how we can make it more secure.

Also, check examples of code to provide secure Kubernetes configuration.

What are the challenges?

Authentication and Authorization — we should implement strong authentication mechanisms, such as multi-factor authentication (MFA) and role-based access control (RBAC), to ensure that only authorized personnel can access our sensitive resources.

Network Security — secure network is also a crucial thing to take care of. We can implement virtual private networks, use secure communication protocols, can help protect data in transit.

Vulnerability Management — we should regularly patch, and update software and applications running on exposed resources. In order to keep track of any security issues in our infrastructure and code, we should take a closer look at vulnerability management processes. A good example of these kinds of tools is security scanners like Snyk, Sonarqube, Kics or scanners dedicated to container security like eg. Trivy or Clair.

Monitoring and Logging — monitoring and logging systems a key to detecting any issues within our infrastructure and generating alerts for suspicious, malicious behavior.

Encryption — encrypting data at rest and in transit using strong encryption algorithms is an additional layer of security to protect sensitive information.

Example of Kubernetes configuration

Exposing Kubernetes containers externally requires careful consideration of security best practices to ensure a secure deployment. Below are examples of code that can be implemented to achieve this:

Kubernetes Ingress Controller

By leveraging an Ingress controller, you can expose your Kubernetes services securely. Here’s an example using the Nginx Ingress controller:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: my-app-ingress
  annotations:
    kubernetes.io/ingress.class: "nginx"
spec:
  rules:
    - host: example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: my-app-service
                port:
                  number: 80

Ensure to set up proper authentication, authorization, and TLS termination within the Ingress controller to enhance security. Let’s see how to extend our code in order to achieve this:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: my-app-ingress
  annotations:
    kubernetes.io/ingress.class: "nginx"
    nginx.ingress.kubernetes.io/rewrite-target: /
    nginx.ingress.kubernetes.io/ssl-redirect: "true" 
    nginx.ingress.kubernetes.io/secure-backends: "true" 
    nginx.ingress.kubernetes.io/auth-url: "https://auth.example.com/auth" 
spec:
  tls:
    - hosts:
        - example.com
      secretName: tls-secret
  rules:
    - host: example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: my-app-service
                port:
                  number: 80

tls-secret — it’s the name of a Kubernetes Secret that contains SSL/TLS certificate and private key. This ensures that HTTPS is used for secure communication.

nginx.ingress.kubernetes.io/ssl-redirect: "true" — this annotation will enforce an HTTP to HTTPS redirect, ensuring that all traffic is encrypted.

nginx.ingress.kubernetes.io/secure-backends: "true" — this annotation ensures that the Ingress controller establishes secure connections to the backend service.

nginx.ingress.kubernetes.io/auth-url — this annotation is used to specify an external authentication service URL. Requests to your application can be authenticated by this external service, which can be configured to perform user authentication and authorization checks. This is a place to define the URL of our authentication service like eg. Azure Active Directory.

Network Policies

Kubernetes Network Policies allow you to define rules for network traffic within your cluster. By specifying which pods can communicate with each other, you can reduce the attack surface and limit traffic. Here’s an example of a Network Policy:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: my-app-network-policy
spec:
  podSelector:
    matchLabels:
      app: my-app
  policyTypes:
    - Ingress
  ingress:
    - from:
        - podSelector:
            matchLabels:
              app: trusted-app

app: my-app— Pods with this label are the pods that you want this network policy to apply to. These are the pods you are protecting or restricting access to.

app: trusted-app— this label is specifying the source of traffic that is allowed to access pods labeled with app: my-app. In this case, it's allowing pods with the label app: trusted-app to communicate with pods labeled app: my-app.

What it basically mean is that the pods labeled app: trusted-app are considered trusted sources and are allowed to send traffic to the pods labeled app: my-app.

Pod Security Policies

Update: With the release of Kubernetes v1.25, PodSecurityPolicy has been removed. You can read more information about the removal of PodSecurityPolicy in the Kubernetes 1.25 release notes.

Pod Security Policies (PSPs) provide fine-grained control over the security configurations of pods. You can define policies such as restricting privileged containers, host namespace access, and host networking. Here’s an example of a Pod Security Policy:

apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  name: my-app-pod-security-policy
spec:
  privileged: false
  seLinux:
    rule: RunAsAny
  runAsUser:
    rule: MustRunAsNonRoot
  fsGroup:
    rule: RunAsAny
  volumes:
    - '*'

This Pod Security Policy disallows privileged containers, enforces running as a non-root user, and allows any filesystem group. I explain this policy below in detail below.

privileged: false- specifies if pods are allowed to run in privileged mode. Setting it to false means that pods are not allowed to run with privileged access, which can execute processes with full system privileges.

seLinux: rule: RunAsAny- this section allows pods to run with any SELinux context. It doesn’t enforce any specific SELinux context for the pods.

runAsUser: rule: MustRunAsNonRoot- this section specifies the rules for the user under which the pods are allowed to run. Rule enforces that pods must run as a non-root user. This can reduce the potential impact of security vulnerabilities by running processes with limited privileges.

fsGroup: rule: RunAsAny- defines the rules for the pod’s file system group. Rule allows pods to use any file system group. Similar to SELinux, it provides flexibility in terms of file system group usage.

volumes:- '*'- lists the volume types that pods are allowed to use. The asterisk * allows pods to use any type of volume.

Container Image Scanning

Regularly scanning container images for vulnerabilities is essential. Integrate container image scanning tools dedicated to containers like Trivy, or Clair into your CI/CD pipelines to detect any known vulnerabilities before deploying containers to the cluster. You can also use other scanners like eg. Kics which is dedicated to IaC security scans, but it also provides container scanning as a one of feature.

Check my article on how to scan containers to find security issues using Trivy -> https://medium.com/@kbrzozova/devsecops-how-to-secure-containers-using-trivy-22205d9825f6

RBAC and Least Privilege Principles

Implement Role-Based Access Control (RBAC) to enforce granular permissions for users and services within the cluster. As always we should stick to the principle of least privilege, granting only the necessary permissions to each entity.

Secure Secrets Management

Ensure that sensitive information, such as API keys, database credentials, or certificates, is securely managed using Kubernetes Secrets. We should avoid hardcoding secrets in code and configuration files, and instead, inject them into containers through Secrets, Key Management Systems (KMS), or secure environment variables.

As an example, you can create a Secret like this:

apiVersion: v1
kind: Secret
metadata:
  name: app-secret
type: Opaque
data:
  password: bXlzZWNyZXRwYXNzd29yZA==

The string bXlzZWNyZXRwYXNzd29yZA== is a Base64 encoded representation of the actual secret value. In Kubernetes Secrets, sensitive data such as passwords or API keys is stored in a Base64 encoded format to provide a level of obfuscation.

Worth mentioning that Base64 is not a type of encryption, but encoding. The encoded data can be easily decoded to retrieve the original binary data. Encryption is a process that transforms data into a format that is not easily readable without a decryption key or algorithm.

Apply the secret:

kubectl apply -f app-secret.yaml

Inject Secret in your pod configuration. You basically reference the Secret and inject it as an environment variable into your container.

apiVersion: v1
kind: Pod
metadata:
  name: app-pod
spec:
  containers:
    - name: app-container
      image: app-image
      env:
        - name: APP_PASSWORD
          valueFrom:
            secretKeyRef:
              name: app-secret
              key: password

The password from the app-secret Secret as an environment variable named APP_PASSWORD into the app-container. The secretKeyRef includes the Secret and the specific key (password) within it.

SUMMARY

As you can see exposing internal resources like Kubernetes clusters can be a complex process and we should implement a multi-layered security approach including authentication, secure network connections, regular vulnerability management, monitoring, and encryption.