Skip to main content

External etcd

Supported Configurations
Running the control plane as a container with:

When using this backing store option, etcd is deployed on the control plane cluster in the same namespace as the vClustervClusterAn open-source software product that creates and manages tenant clusters within Kubernetes infrastructure. vCluster provides tenant isolation capabilities while reducing infrastructure costs.Related: Tenant cluster, Control plane cluster control plane pod. vCluster deploys etcd with a StatefulSet, Service, and headless Service.

controlPlane:
  backingStore:
    etcd:
      deploy:
        enabled: true

Customize the resources​

You can customize the resources that is deployed for etcd. Here are some basic examples, but more options exist in the configuration.

Set resource requests for the StatefulSet​

controlPlane:
  backingStore:
    etcd:
      deploy:
        enabled: true
        statefulSet:
          resources:
            requests:
              cpu: 20m
              memory: 150Mi

Add annotations to each etcd component​

controlPlane:
  backingStore:
    etcd:
      deploy:
        enabled: true
        statefulSet:
          annotations:
            app.kubernetes.io/part-of: "etcd"
            cluster-autoscaler.kubernetes.io/safe-to-evict: "false"
        service:
          annotations:
            app.kubernetes.io/part-of: "etcd"
        headlessService:
          annotations:
            app.kubernetes.io/part-of: "etcd"

Set the security context of the StatefulSet​

controlPlane:
  backingStore:
    etcd:
      deploy:
        enabled: true
        statefulSet:
          security:
            podSecurityContext:
              allowPrivilegeEscalation: false

Migration options​

Migrate to embedded etcd​

You can migrate from deployed etcdetcdA distributed key-value store that provides reliable storage for Kubernetes cluster data. In vCluster, etcd can be deployed externally or embedded within the vCluster pod.Related: Control Plane to embedded etcd without data loss. This migration simplifies your deployment and reduces resource consumption.

Migration guide available

For step-by-step instructions on migrating from deployed to embedded etcd while preserving all your data, see the complete migration guide.

vcluster.yaml
controlPlane:
  backingStore:
    etcd:
      embedded:
        enabled: true
        migrateFromDeployedEtcd: true

Deploying with migrateFromDeployedEtcd: true retains the external etcd StatefulSet to perform the migration. After a successful migration, you should see the Successfully migrated etcd database to embedded etcd log message in vCluster. To remove the external etcd after migration, delete migrateFromDeployedEtcd: true from the vcluster.yaml:

vcluster.yaml after etcd migration
controlPlane:
  backingStore:
    etcd:
      embedded:
        enabled: true

Config reference​

deploy object ​

Deploy defines to use an external etcd that is deployed by the helm chart

enabled boolean false ​

Enabled defines that an external etcd should be deployed.

statefulSet object ​

StatefulSet holds options for the external etcd statefulSet.

enabled boolean true ​

Enabled defines if the statefulSet should be deployed

EnableServiceLinks for the StatefulSet pod

image object ​

Image is the image to use for the external etcd statefulSet

registry string registry.k8s.io ​

Registry is the registry of the container image, e.g. my-registry.com or ghcr.io. This setting can be globally overridden via the controlPlane.advanced.defaultImageRegistry option. Empty means docker hub.

repository string etcd ​

Repository is the repository of the container image, e.g. my-repo/my-image

tag string 3.6.8-0 ​

Tag is the tag of the container image, and is the default version.

imagePullPolicy string ​

ImagePullPolicy is the pull policy for the external etcd image

env object[] [] ​

Env are extra environment variables

extraArgs string[] [] ​

ExtraArgs are appended to the etcd command.

resources object ​

Resources the etcd can consume

limits object ​

Limits are resource limits for the container

requests object map[cpu:20m memory:150Mi] ​

Requests are minimal resources that will be consumed by the container

pods object ​

Pods defines extra metadata for the etcd pods.

annotations object {} ​

Annotations are extra annotations for this resource.

labels object {} ​

Labels are extra labels for this resource.

highAvailability object ​

HighAvailability are high availability options

replicas integer 1 ​

Replicas are the amount of pods to use.

scheduling object ​

Scheduling options for the etcd pods.

nodeSelector object {} ​

NodeSelector is the node selector to apply to the pod.

affinity object {} ​

Affinity is the affinity to apply to the pod.

tolerations object[] [] ​

Tolerations are the tolerations to apply to the pod.

priorityClassName string ​

PriorityClassName is the priority class name for the the pod.

podManagementPolicy string Parallel ​

PodManagementPolicy is the statefulSet pod management policy.

topologySpreadConstraints object[] [] ​

TopologySpreadConstraints are the topology spread constraints for the pod.

security object ​

Security options for the etcd pods.

podSecurityContext object {} ​

PodSecurityContext specifies security context options on the pod level.

containerSecurityContext object {} ​

ContainerSecurityContext specifies security context options on the container level.

persistence object ​

Persistence options for the etcd pods.

volumeClaim object ​

VolumeClaim can be used to configure the persistent volume claim.

enabled boolean true ​

Enabled enables deploying a persistent volume claim.

accessModes string[] [ReadWriteOnce] ​

AccessModes are the persistent volume claim access modes.

retentionPolicy string Retain ​

RetentionPolicy is the persistent volume claim retention policy.

size string 5Gi ​

Size is the persistent volume claim storage size.

storageClass string ​

StorageClass is the persistent volume claim storage class.

volumeClaimTemplates object[] [] ​

VolumeClaimTemplates defines the volumeClaimTemplates for the statefulSet

addVolumes object[] [] ​

AddVolumes defines extra volumes for the pod

addVolumeMounts object[] ​

AddVolumeMounts defines extra volume mounts for the container

name string ​

This must match the Name of a Volume.

readOnly boolean ​

Mounted read-only if true, read-write otherwise (false or unspecified). Defaults to false.

mountPath string ​

Path within the container at which the volume should be mounted. Must not contain ':'.

subPath string ​

Path within the volume from which the container's volume should be mounted. Defaults to "" (volume's root).

mountPropagation string ​

mountPropagation determines how mounts are propagated from the host to container and the other way around. When not set, MountPropagationNone is used. This field is beta in 1.10.

subPathExpr string ​

Expanded path within the volume from which the container's volume should be mounted. Behaves similarly to SubPath but environment variable references $(VAR_NAME) are expanded using the container's environment. Defaults to "" (volume's root). SubPathExpr and SubPath are mutually exclusive.

annotations object {} ​

Annotations are extra annotations for this resource.

labels object {} ​

Labels are extra labels for this resource.

service object ​

Service holds options for the external etcd service.

enabled boolean true ​

Enabled defines if the etcd service should be deployed

annotations object {} ​

Annotations are extra annotations for the external etcd service

headlessService object ​

HeadlessService holds options for the external etcd headless service.

annotations object {} ​

Annotations are extra annotations for the external etcd headless service