Redpanda in Kubernetes

Kubernetes is a container orchestration tool that helps you manage Redpanda deployments using declarative configuration files called manifests. Kubernetes provides a standardized way of achieving the following:

  • High availability

  • Disaster recovery

  • Scalability

Deployment tools

To deploy Redpanda in Kubernetes, you can choose between Helm for its simplicity or the Redpanda Operator for advanced lifecycle management. While Helm provides an easy way to install and upgrade Redpanda, it has limitations in managing complex, stateful workloads at scale. The Redpanda Operator is the recommended option for production deployments as it enables better upgrade management, dynamic configuration, and improved lifecycle automation.

Feature Redpanda Operator Helm Description

Managed upgrade and rollback

✅

⚠️

Helm provides basic upgrades with rollback capabilities but requires manual intervention and increases operational risk. The Redpanda Operator automates safe, rolling upgrades with reconciliation, significantly reducing risk.

Dynamic configuration

✅

❌

Helm configurations must be updated manually through Helm values and redeployment. The Redpanda Operator dynamically applies real-time configuration changes through custom resources (CRDs).

Advanced health checks and metrics

✅

⚠️

Helm relies on standard Kubernetes-level health metrics, while the Redpanda Operator includes advanced, application-specific metrics and health checks.

Lifecycle automation

✅

⚠️

Helm has limited automation and depends on manual management for scaling, failover, and cleanup. The Redpanda Operator automates scaling, failover, resource reconciliation, and cleanup tasks.

Multi-tenancy management

✅

⚠️

Helm requires separate releases to manage multiple clusters, making management more complex. The Redpanda Operator simplifies this through declarative custom resources.
If you are already using the Redpanda Helm chart and want to migrate to the latest Redpanda Operator for better lifecycle management, see Migrate from the Redpanda Helm chart.

Redpanda Operator

The Redpanda Operator is designed for production-grade Redpanda deployments, offering enhanced lifecycle management, automation, and GitOps compatibility. It supports two deployment modes controlled by the chartRef.useFlux flag.

Redpanda-managed mode

(Default: chartRef.useFlux: false)

The Redpanda Operator manages and reconciles Redpanda resources directly, without involving Flux. This mode works as follows:

  1. The Redpanda Operator creates both HelmRelease and HelmRepository resources but suspends them, keeping Flux inactive.

    Although these resources are created in a suspended state, the Redpanda Operator still requires the Flux CRDs to be installed.

  2. The Redpanda Operator directly reconciles Redpanda resources, performing tasks such as installations, updates, and cleanup, normally handled by Flux.

  3. If useFlux is later re-enabled, the Redpanda Operator unsuspends the Flux resources, and Flux resumes lifecycle management.

Flux-managed mode

(Enabled with chartRef.useFlux: true)

The Redpanda Operator delegates reconciliation to Flux controllers. The workflow is as follows:

  1. The Redpanda Operator defines the necessary CRDs for Redpanda. A Redpanda resource represents a Redpanda cluster, configured through the Redpanda Helm chart values.

  2. When the Redpanda Operator detects changes to Redpanda resources, it triggers Flux to reconcile the corresponding HelmRelease resource.

  3. Flux uses the HelmRelease resource to manage the lifecycle of the Redpanda Helm chart, applying updates as needed.

Example configuration to specify the deployment mode in Redpanda’s custom resource:

apiVersion: cluster.redpanda.com/v1alpha2
kind: Redpanda
metadata:
  name: redpanda
spec:
  chartRef:
    useFlux: false
CRDs for the Redpanda Operator include Flux CRDs. If you are running in Flux-managed mode

Helm

Helm is a package manager for Kubernetes that simplifies defining, installing, and upgrading Kubernetes applications. Helm uses charts, a collection of files describing Kubernetes resources, to deploy applications in a Kubernetes cluster.

The Redpanda Helm chart provides all the manifest files required to deploy Redpanda in Kubernetes, including:

  • A StatefulSet to manage Redpanda brokers

  • A Headless ClusterIP Service for internal communication with the Redpanda cluster

  • A NodePort Service for external communication with the Redpanda cluster

Kubernetes deployment environments

You can run Redpanda on managed Kubernetes services as well as in bare-metal environments. Managed Kubernetes services offer simpler deployment and maintenance, while bare-metal environments provide complete control and cost efficiencies.

Managed Kubernetes

Managed Kubernetes services, such as Google Kubernetes Engine (GKE) and Amazon Elastic Kubernetes Service (EKS), handle core components of a Kubernetes cluster, offering benefits such as:

  • Ease of deployment: Pre-configured instances join your Kubernetes cluster automatically.

  • Control plane maintenance: The provider maintains the control plane, ensuring security and reliability.

  • Health monitoring and repairs: The provider monitors master nodes and repairs them as needed.

You remain responsible for deploying and maintaining Redpanda instances on worker nodes.

Deploy Kubernetes clusters with unmanaged (manual) node updates. Managed (automatic) updates during cluster deployment can lead to service downtime, data loss, or quorum instability. Transitioning from managed updates to unmanaged updates after deployment may require downtime. To avoid these disruptions, plan for unmanaged node updates from the start. See Kubernetes Cluster Requirements and Recommendations.

Bare-metal Kubernetes environments

Bare-metal Kubernetes environments give you complete control over both the control plane and the worker nodes, which can be advantageous when you want the following:

  • Complete control: Bare-metal Kubernetes offers control over every aspect of deployment, suited for highly customized environments.

  • Custom configurations: You have granular control to fine-tune the Kubernetes setup.

  • Cost efficiency: Owning and operating your hardware may be more economical over time.

Documentation conventions

This documentation follows conventions to help users easily identify Kubernetes resource types and Helm values:

  • Resource names: Kubernetes resource names, such as Service or PersistentVolume, are capitalized and in Pascal case to match the manifest files.

  • Helm values: Helm values, like storage.persistentVolume.enabled, are displayed in monospace font.

Next steps

Suggested reading

Suggested labs

Search all labs