Securing Your Helm Charts: A Complete Guide

Oleksandr Erm • 2/28/2024 • 8 min read

Securing Your Helm Charts: A Comprehensive Guide

Picture this: You’ve just deployed a critical application to your Kubernetes cluster using Helm. Everything seems perfect until you discover a security breach that could have been prevented with proper chart security. This scenario is all too common in the world of cloud-native applications, where the convenience of Helm charts sometimes overshadows the critical need for security.

In this guide, we’ll transform your Helm charts from potential security liabilities into robust, fortress-like deployments. Whether you’re a seasoned Kubernetes architect or just starting with Helm, you’ll learn practical, battle-tested approaches to securing your charts at every level.

Helm Chart Security Fundamentals

Think of your Helm chart as a blueprint for your application’s infrastructure. Just as an architect ensures the structural integrity of a building, we need to verify and protect the integrity of our Helm charts. Let’s explore how.

Chart Verification and Integrity

In the world of container orchestration, trust is everything. Would you deploy code from an unknown source to your production environment? Probably not. This is where chart verification comes in - it’s your first line of defense against compromised or tampered charts.

Here’s how to implement a robust verification system:

# Generate a signing key
gpg --full-generate-key

# Sign a helm chart
helm package --sign mychart/ --key 'John Smith' --keyring ~/.gnupg/secring.gpg

# Verify a signed chart
helm verify mychart-1.0.0.tgz --keyring ~/.gnupg/pubring.gpg

Think of these signatures as your chart’s DNA - they uniquely identify trusted sources and ensure that what you’re deploying is exactly what was intended.

Secure Values Management

One of the most common security pitfalls in Helm charts is the mishandling of sensitive data. It’s like leaving your house keys under the doormat - sure, it’s convenient, but it’s also the first place an attacker will look.

Instead of hardcoding sensitive values, let’s implement a more secure approach:

# values.yaml with sensitive data references
config:
  dbPassword: "${SECRET_DB_PASSWORD}"
  apiKey: "${SECRET_API_KEY}"
  
  # Use external secrets management
  secretRef:
    name: app-secrets
    key: credentials

By leveraging environment variables and external secrets management tools, we create a separation between our chart configuration and sensitive data. This approach not only enhances security but also makes our charts more portable and maintainable.

Implementing Security Best Practices

Security in Kubernetes isn’t just about keeping the bad actors out - it’s about implementing a comprehensive defense strategy that protects your applications at every level. Let’s dive into two critical components of this strategy: RBAC and Pod Security Policies.

RBAC Configuration

Remember the principle of least privilege? It’s like giving each employee in a company exactly the access they need to do their job - no more, no less. Role-Based Access Control (RBAC) in Kubernetes follows this same principle, allowing you to define precise permissions for each component of your application.

Let’s look at how to implement granular access control:

# templates/rbac.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: {{ .Release.Name }}-minimal-access
rules:
- apiGroups: [""]
  resources: ["pods", "services"]
  verbs: ["get", "list"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: {{ .Release.Name }}-binding
subjects:
- kind: ServiceAccount
  name: {{ .Release.Name }}-sa
roleRef:
  kind: Role
  name: {{ .Release.Name }}-minimal-access
  apiGroup: rbac.authorization.k8s.io

This RBAC configuration creates a role that can only view pods and services - perfect for monitoring tools or readonly access. Notice how we’re not granting any write permissions (create, update, delete) unless explicitly needed. This minimalistic approach significantly reduces your attack surface.

Pod Security Policies

If RBAC is about who can access what, Pod Security Policies (PSPs) are about what those pods can do once they’re running. Think of PSPs as your application’s behavioral boundaries - they ensure your containers can’t do anything that might compromise your cluster’s security.

Here’s how to implement a security-focused PSP:

# templates/pod-security-policy.yaml
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  name: {{ .Release.Name }}-psp
spec:
  privileged: false
  seLinux:
    rule: RunAsAny
  runAsUser:
    rule: MustRunAsNonRoot
  fsGroup:
    rule: RunAsAny
  volumes:
  - 'configMap'
  - 'secret'
  - 'emptyDir'

This policy enforces several crucial security practices:

Prevents privileged containers (equivalent to root access on the host)
Requires containers to run as non-root users
Restricts the types of volumes that can be mounted

By implementing these policies, you create a secure foundation that prevents common container escape vectors and privilege escalation attempts. It’s like setting up guardrails - your applications can still do their job, but they can’t accidentally (or intentionally) stray into dangerous territory.

Secure Chart Development

When developing Helm charts, security isn’t an afterthought - it’s a fundamental design principle that should be woven into every aspect of your chart. Let’s explore two critical areas where security begins at the container level and extends to your network architecture.

Container Security

The security of your Kubernetes deployments starts with your containers. Just like you wouldn’t run a production service as root on a traditional server, you shouldn’t do it in a container either. Modern container security requires a defense-in-depth approach, implementing multiple layers of protection.

Here’s how to build security into your container configurations:

# templates/deployment.yaml
spec:
  template:
    spec:
      securityContext:
        runAsNonRoot: true
        runAsUser: 1000
        fsGroup: 2000
      containers:
      - name: {{ .Chart.Name }}
        securityContext:
          allowPrivilegeEscalation: false
          capabilities:
            drop:
            - ALL
          readOnlyRootFilesystem: true

Let’s break down these security measures:

runAsNonRoot ensures containers can’t run as root - a fundamental security best practice
allowPrivilegeEscalation: false prevents processes from gaining more privileges than their parent
Dropping ALL capabilities and using a read-only filesystem creates a minimal attack surface

These configurations create a hardened container environment that follows the principle of least privilege while still maintaining full functionality for your applications.

Network Policies

In today’s microservices architecture, network security is as crucial as container security. Network Policies in Kubernetes act like a sophisticated firewall, controlling the flow of traffic between your pods with precision and purpose.

Here’s how to implement fine-grained network controls:

# templates/network-policy.yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: {{ .Release.Name }}-network-policy
spec:
  podSelector:
    matchLabels:
      app: {{ .Release.Name }}
  policyTypes:
  - Ingress
  - Egress
  ingress:
  - from:
    - podSelector:
        matchLabels:
          role: frontend
    ports:
    - protocol: TCP
      port: 80

This policy implements the zero-trust networking principle: all network traffic is denied by default, and only explicitly allowed connections are permitted. In this example, we’re only allowing incoming traffic from pods labeled as ‘frontend’ on port 80.

Think of it as building secure corridors between your services - each service can only communicate with the ones it absolutely needs to, reducing the blast radius of potential security breaches.

Secure Chart Distribution

Distribution security is often overlooked, but it’s a critical link in your security chain. After all, what good is a secure chart if it can be tampered with during distribution? Let’s explore how to ensure your charts remain secure from development to deployment.

Private Repository Configuration

Public chart repositories are convenient, but when it comes to proprietary or security-sensitive applications, private repositories are essential. They provide controlled access to your charts and ensure that only authorized users and systems can pull them.

Here’s how to set up secure repository access:

# Example of secure repository configuration
apiVersion: v1
kind: Secret
metadata:
  name: helm-repo-creds
type: kubernetes.io/dockerconfigjson
data:
  .dockerconfigjson: {{ .Values.helmRepoCredentials | b64enc }}

This configuration creates a Kubernetes secret containing your repository credentials. By integrating it with your CI/CD pipeline, you can automate secure chart pulls while keeping credentials safely encrypted.

A few best practices to consider:

Rotate repository credentials regularly
Use different credentials for development and production environments
Implement IP-based access restrictions where possible
Enable audit logging for all repository access

Chart Provenance

In the software supply chain, trust is earned through verification. Chart provenance provides cryptographic verification that a chart hasn’t been modified since it was packaged by the original author. It’s like a digital seal of authenticity for your Helm charts.

Here’s how to implement chart signing and verification:

# Generate provenance files
helm package --sign --key 'maintainer@example.com' ./mychart

# Verify chart with provenance
helm install --verify myrelease ./mychart-1.0.0.tgz

When you sign a chart, you create a provenance file (.prov) containing:

A cryptographic hash of the chart archive
The signature of that hash
Metadata about when and who signed it

By verifying provenance before installation, you ensure that:

The chart hasn’t been tampered with during distribution
It comes from a trusted source
The contents match exactly what the maintainer signed

This creates an unbroken chain of trust from the chart maintainer to your cluster. Always require provenance verification in production environments - it’s a small step that provides significant security benefits.

Security Scanning and Validation

Prevention is better than cure - especially in Kubernetes security. Automated scanning and validation help catch security issues before they reach production. Let’s explore how to implement comprehensive security checks in your deployment pipeline.

Chart Testing

Quality assurance isn’t just about functionality - it’s about security too. Automated chart testing helps ensure that your security configurations are consistent and correctly implemented across all your charts.

Here’s a robust testing configuration:

# chart-testing.yaml
remote: origin
target-branch: main
chart-dirs:
  - charts
chart-repos:
  - bitnami=https://charts.bitnami.com/bitnami
validate-maintainers: true
validate-chart-schema: true
validate-yaml: true
check-version-increment: true

This configuration validates several critical aspects:

Chart schema compliance ensures consistent security configurations
Version increment checks prevent accidental deployments of untested versions
YAML validation catches misconfigurations that could lead to security vulnerabilities
Maintainer validation ensures accountability for chart changes

Think of it as your pre-flight security checklist - each validation step confirms that your chart meets security standards before takeoff.

Automated Security Checks

Static analysis is your first line of defense against security vulnerabilities. By integrating security scanning into your CI/CD pipeline, you create a consistent security baseline for all your charts.

Here’s how to implement automated security scanning:

# .github/workflows/security-scan.yml
name: Security Scan
on:
  pull_request:
    paths:
      - 'charts/**'
jobs:
  security-scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Run Kubesec
        uses: controlplaneio/kubesec-action@master
        with:
          input: charts/*/templates/*.yaml

This GitHub Action automatically scans your Helm charts for:

Insecure container configurations
Missing security contexts
Dangerous capabilities or privilege settings
Network policy gaps
Resource limit omissions

Pro Tips for Effective Security Scanning:

Set up blocking checks for critical security issues
Configure different severity levels for different types of findings
Maintain a whitelist for necessary exceptions
Archive scan results for security audit trails

Remember: Security scanning isn’t about achieving a perfect score - it’s about understanding and managing your security posture. Use scan results to make informed decisions about security trade-offs and improvements.

Runtime Security

Security doesn’t end with deployment - it’s an ongoing process that requires careful attention to runtime configurations and monitoring. Let’s explore two critical aspects of runtime security that your Helm charts should address.

Secret Management

Secrets are the crown jewels of your application - database credentials, API keys, encryption keys, and other sensitive data that need special protection. Modern applications require a sophisticated approach to secret management that goes beyond basic Kubernetes secrets.

Here’s how to implement secure secret handling using HashiCorp Vault:

# templates/secrets.yaml
apiVersion: v1
kind: Secret
metadata:
  name: {{ .Release.Name }}-secrets
  annotations:
    vault.hashicorp.com/agent-inject: "true"
    vault.hashicorp.com/role: "{{ .Values.vaultRole }}"
type: Opaque
data:
  {{- range $key, $value := .Values.secrets }}
  {{ $key }}: {{ $value | b64enc }}
  {{- end }}

This configuration provides several security advantages:

Dynamic secret injection at runtime
Automatic secret rotation
Fine-grained access control
Audit logging of all secret access

Best Practices for Secret Management:

Never store secrets in version control
Implement secret rotation policies
Use separate secret stores for different environments
Monitor and alert on unusual secret access patterns

Resource Limits

Resource management isn’t just about efficiency - it’s a critical security measure. Without proper limits, a compromised container could consume excessive resources, potentially affecting other applications or even the entire cluster.

Here’s how to implement comprehensive resource controls:

# templates/deployment.yaml
spec:
  template:
    spec:
      containers:
      - name: {{ .Chart.Name }}
        resources:
          limits:
            cpu: {{ .Values.resources.limits.cpu }}
            memory: {{ .Values.resources.limits.memory }}
          requests:
            cpu: {{ .Values.resources.requests.cpu }}
            memory: {{ .Values.resources.requests.memory }}

Key considerations for resource limits:

Set both requests and limits to prevent resource hogging
Consider the application’s normal operating requirements
Include headroom for traffic spikes
Monitor resource usage patterns to refine limits

Think of resource limits as a safety valve - they ensure that even if something goes wrong, the impact is contained. This is especially important in multi-tenant clusters where resource isolation is crucial for security.

Pro Tips for Resource Management:

Use horizontal pod autoscaling instead of oversized limits
Set up monitoring alerts for consistent high resource usage
Document the reasoning behind your resource allocations
Regularly review and adjust limits based on actual usage patterns

Security Monitoring and Auditing

Even with the best security controls in place, visibility into your cluster’s security posture is crucial. Effective monitoring and auditing help you detect, investigate, and respond to security events before they become incidents.

Audit Logging

Audit logs are your security camera footage - they provide a detailed record of who did what and when in your cluster. Proper audit logging is essential for security investigations, compliance requirements, and understanding your security posture.

Here’s how to configure comprehensive audit logging:

# templates/audit-policy.yaml
apiVersion: audit.k8s.io/v1
kind: Policy
rules:
- level: Metadata
  resources:
  - group: ""
    resources: ["secrets", "configmaps"]

This policy captures important security events:

Access to sensitive resources like secrets and configmaps
Authentication and authorization decisions
Changes to security-relevant configurations
Resource creation, modification, and deletion

Best Practices for Audit Logging:

Define retention periods based on compliance requirements
Implement log forwarding to a secure storage location
Set up alerts for suspicious patterns
Ensure logs are tamper-proof and encrypted

Security Metrics

Metrics provide real-time insights into your cluster’s security health. By monitoring key security indicators, you can spot trends and address issues before they escalate into security incidents.

Here’s how to set up security-focused monitoring:

# templates/servicemonitor.yaml
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: {{ .Release.Name }}-security-metrics
spec:
  endpoints:
  - port: metrics
    interval: 15s
  selector:
    matchLabels:
      app: {{ .Release.Name }}

Key Security Metrics to Monitor:

Failed authentication attempts
Resource access patterns
Pod security policy violations
Network policy denials
Resource utilization anomalies

Pro Tips for Security Monitoring:

Set up baseline metrics for normal behavior
Configure graduated alerting thresholds
Correlate security events across different components
Maintain an incident response playbook

Remember: Good security monitoring isn’t about collecting everything - it’s about collecting the right data and knowing how to use it effectively.

Best Practices Checklist

Let’s wrap up with a comprehensive checklist to ensure you’ve covered all the security bases in your Helm charts:

Chart Development
- ✅ Use version pinning for all dependencies
- ✅ Implement RBAC with least privilege
- ✅ Configure security contexts
- ✅ Define network policies
Distribution Security
- ✅ Sign all charts
- ✅ Use private repositories
- ✅ Implement access controls
- ✅ Verify chart integrity
Runtime Security
- ✅ Use secure secret management
- ✅ Configure resource limits
- ✅ Implement pod security policies
- ✅ Enable audit logging

Conclusion

Securing Helm charts is a multi-faceted challenge that requires attention to detail at every stage - from development through distribution to runtime. By following these security best practices, you create a robust security posture that protects your Kubernetes deployments from common threats while maintaining operational efficiency.

Remember that security is not a destination but a journey. Regularly review and update your security measures, stay informed about new security features and threats, and always prioritize security in your Helm chart development process.