Elastic Cloud for Kubernetes (ECK) 1.3 is here , oh yeah ready for Cloud Native

Introduction

yes ECK 1.3 is there. Finally we have a promising update on Elastic Cloud for Kubernetes, better known as the ECK Operator. If the operator wasn’t cool enough to help you safely orchestrate and deploy Elasticsearch (and more) on Kubernetes this version has some great improvements.

Please note that by default now elasticsearch version 7.10 is deployed. For an upgrade, always consult the breaking changes at breaking-changes-7.10

Elasticsearch 7.10 improvements

With elasticsearch 7.10 what can you can expect in a nutshell?

• Better search and usability experience
• Improved performance due memory optimization
• Lower storage consumption, due new stored field compression
• Searchable snapshots, which make Cold tier (topology) data searchable on your favorite Object storage like AWS S3, Azure or GCP. More info see here.
• Event Query Language (EQL), one of the unique capabilities from Endgame

Small syntax snippet below.

GET /virus-index-000001/_eql/search
{
  "query": """
    process where (process.name == "corona.exe" and process.pid != 2020)
  """
}

For more information read the blog here.

ECK 1.3 Improvements

Below my Top 4 of improvements you need to know

1. Volume expansion support
2. Configurable timeouts for ECK Operator
3. Official Helm chart available (beta)
4. IPv6 is supported and auto-detected.

ECK Must have (future) features

Below are some features I’m still waiting for. Please read the issue

• Horizontal Pod scaling, Zone awareness so Elasticsearch can handle vertical Node (cluster) scaling like , see Issue 3933 (Github)
• ECK has support for ILM configuration through manifests, see Issue 2157 (Github)
• Still missing Logstash support, see Issue 1453 (Github)

My first impression.

Volume expansion support

Definitely my number one for this release. ECK now supports online volume expansion when allowed by the underlying storage class. You just have to update the storage request size value of your Elasticsearch manifest (volumeClaimTemplates section). After applying the Statefulset (Pod) and PVC are recreated, but when the volume driver supports ExpandInUsePersistentVolumes, you don’t even have to recycle your elasticsearch processes or Pod recreation is needed. Nice !

Deploy a demo example

cat <<EOF | kubectl apply -f -
apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
  name: volume-demo
spec:
  version: 7.10.0
  nodeSets:
  - name: default
    count: 1
    volumeClaimTemplates:
    - metadata:
        name: elasticsearch-data
      spec:
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: 1Gi
        storageClassName: standard
    config:
      node.store.allow_mmap: false
EOF

Now edit the elasticsearch volume-demo and update the 1Gi to 2Gi. Now watch the elasticsearch instance being updated.

$ kubectl get elasticsearch -o wide -w
NAME          HEALTH    NODES   VERSION   PHASE             AGE

volume-demo   green           7.10.0    ApplyingChanges   16m25s

$ kubectl get pvc
NAME                                          STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
elasticsearch-data-volume-demo-es-default-0   Bound    pvc-fc422743-93a3-4825-9e01-b1dfdd6b36b5   2Gi        RWO            standard       22m

Please take attention this will not work on minikube or as said with unsupported storage drivers. Check the kubernetes documentation if your storage class is supported, see here. . Otherwise you can expect the error below from the admission controller.

error: elasticsearches.elasticsearch.k8s.elastic.co "volume-demo" could not be patched: admission webhook "elastic-es-validation-v1.k8s.elastic.co" denied the request: Elasticsearch.elasticsearch.k8s.elastic.co "volume-demo" is invalid: spec.nodeSet[0].volumeClaimTemplates: Invalid value: []v1.PersistentVolumeClaim{v1.PersistentVolumeClaim{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"elasticsearch-data", GenerateName:"", Namespace:"", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v1.PersistentVolumeClaimSpec{AccessModes:[]v1.PersistentVolumeAccessMode{"ReadWriteOnce"}, Selector:(*v1.LabelSelector)(nil), Resources:v1.ResourceRequirements{Limits:v1.ResourceList(nil), Requests:v1.ResourceList{"storage":resource.Quantity{i:resource.int64Amount{value:2147483648, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"2Gi", Format:"BinarySI"}}}, VolumeName:"", StorageClassName:(*string)(0xc00039f520), VolumeMode:(*v1.PersistentVolumeMode)(nil), DataSource:(*v1.TypedLocalObjectReference)(nil)}, Status:v1.PersistentVolumeClaimStatus{Phase:"", AccessModes:[]v1.PersistentVolumeAccessMode(nil), Capacity:v1.ResourceList(nil), Conditions:[]v1.PersistentVolumeClaimCondition(nil)}}}: claim elasticsearch-data (storage class standard) does not support volume expansion
You can run `kubectl replace -f /var/folders/17/6mxl5kws1zl5szg69jzlh9y40000gp/T/kubectl-edit-ozffb.yaml` to try this update again.

This option is available next to extending through adding an additional NodeSet.

Configurable timeouts for ECK Operator

In production heavy loaded environments there could be moments when the Kubernetes API or the Elasticsearch API isn’t responding fast enough. In these cases it could be helpful to configure a timeout, which is now possible. Preferred way is now by adding this to your Helm values.yaml. Below the default settings, which in most cases should be fine.

# kubeClientTimeout sets the request timeout for Kubernetes API calls made by the operator.
kubeClientTimeout: 60s
  # elasticsearchClientTimeout sets the request timeout for Elasticsearch API calls made by the operator.
elasticsearchClientTimeout: 180s

Official Helm Chart

Before you start, ensure that you are using Helm 3X instead of Helm 2. You got a message about the — create-namespace parameter if Helm 2X is used.

All-in-one installation goes flawless, but you have enough possibilities to parameterize within a Values.yaml. One downside you still have to create Helm charts for the product deployments.

$ helm repo add elastic https://helm.elastic.co
"elastic" has been added to your repositories
$ helm repo update
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "elastic" chart repository
Update Complete. ⎈Happy Helming!⎈
$ helm install elastic-operator elastic/eck-operator -n elastic-system --create-namespace
NAME: elastic-operator
LAST DEPLOYED: Wed Nov 18 21:06:21 2020
NAMESPACE: elastic-system
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
1. Inspect the operator logs by running the following command:
   kubectl logs -n elastic-system sts/elastic-operator

What is available to customize further in the values.yaml

$ helm show values elastic/eck-operator
# nameOverride is the short name for the deployment. Leave empty to let Helm generate a name using chart values.
nameOverride: "elastic-operator"
# fullnameOverride is the full name for the deployment. Leave empty to let Helm generate a name using chart values.
fullnameOverride: "elastic-operator"
# managedNamespaces is the set of namespaces that the operator manages. Leave empty to manage all namespaces.
managedNamespaces: []
# installCRDs determines whether Custom Resource Definitions (CRD) are installed by the chart.
# Note that CRDs are global resources and require cluster admin privileges to install. 
# If you are sharing a cluster with other users who may want to install ECK on their own namespaces, setting this to true can have unintended consequences.
# 1. Upgrades will overwrite the global CRDs and could disrupt the other users of ECK who may be running a different version.
# 2. Uninstalling the chart will delete the CRDs and potentially cause Elastic resources deployed by other users to be removed as well.
installCRDs: true
# replicaCount is the number of operator pods to run.
replicaCount: 1

image:
  # repository is the container image prefixed by the registry name.
  repository: docker.elastic.co/eck/eck-operator
  # pullPolicy is the container image pull policy.
  pullPolicy: IfNotPresent
  # tag is the container image tag. If not defined, defaults to chart appVersion.
  tag: null

# imagePullSecrets defines the secrets to use when pulling the operator container image.
imagePullSecrets: []

# resources define the container resource limits for the operator.
resources:
  limits:
    cpu: 1
    memory: 512Mi
  requests:
    cpu: 100m
    memory: 150Mi

# podAnnotations define the annotations that should be added to the operator pod. 
podAnnotations: {}

# podSecurityContext defines the pod security context for the operator pod.
podSecurityContext: 
  runAsNonRoot: true 

# securityContext defines the security context of the operator container.
securityContext: {} 

# nodeSelector defines the node selector for the operator pod.
nodeSelector: {}

# tolerations defines the node tolerations for the operator pod.
tolerations: []

# affinity defines the node affinity rules for the operator pod.
affinity: {}

# createClusterScopedResources determines whether cluster-scoped resources (ClusterRoles, ClusterRoleBindings) should be created.
createClusterScopedResources: true
serviceAccount:
  # create specifies whether a service account should be created for the operator.
  create: true
  # annotations to add to the service account
  annotations: {}
  # name of the service account to use. If not set and create is true, a name is generated using the fullname template.
  name: ""
tracing:
  # enabled specifies whether APM tracing is enabled for the operator.
  enabled: false
  # config is a map of APM Server configuration variables that should be set in the environment.
  config:
    ELASTIC_APM_SERVER_URL: http://localhost:8200
    ELASTIC_APM_SERVER_TIMEOUT: 30s
refs:
  # enforceRBAC specifies whether RBAC should be enforced for cross-namespace associations between resources.
  enforceRBAC: false
webhook:
  # enabled determines whether the webhook is installed.
  enabled: true
  # caBundle is the PEM-encoded CA trust bundle for the webhook certificate. Only required if manageCerts is false and certManagerCert is null.
  caBundle: Cg==
  # certManagerCert is the name of the cert-manager certificate to use with the webhook. 
  certManagerCert: null
  # certsDir is the directory to mount the certificates.
  certsDir: "/tmp/k8s-webhook-server/serving-certs"
  # certsSecret is the name of the secret containing the webhook certificates.
  certsSecret: elastic-webhook-server-cert
  # failurePolicy of the webhook.
  failurePolicy: Ignore
  # manageCerts determines whether the operator manages the webhook certificates automatically.
  manageCerts: true
  # name of the webhook
  name: elastic-webhook.k8s.elastic.co
  # name of the service used to expose the webhook.
  serviceName: elastic-webhook-server

softMultiTenancy:
  # enabled determines whether the operator is installed with soft multi-tenancy extensions. 
  # This requires network policies to be enabled on the Kubernetes cluster.
  enabled: false
# kubeAPIServerIP is required when softMultiTenancy is enabled.
kubeAPIServerIP: null
telemetry:
  # disabled determines whether the operator periodically updates ECK telemetry data for Kibana to consume.
  disabled: false
  # distibutionChannel denotes which distribution channel was used to install the operator.
  distributionChannel: "helm"
# config values for the operator.
config:
  # logVerbosity defines the logging level. Valid values are as follows:
  # -2: Errors only
  # -1: Errors and warnings
  #  0: Errors, warnings, and information
  #  number greater than 0: Errors, warnings, information, and debug details. 
  logVerbosity: "0"
  # metricsPort defines the port to expose operator metrics. Set to 0 to disable metrics reporting.
  metricsPort: "0"
  # containerRegistry to use for pulling Elasticsearch and other application container images.
  containerRegistry: docker.elastic.co
  # maxConcurrentReconciles is the number of concurrent reconciliation operations to perform per controller.
  maxConcurrentReconciles: "3"
  # caValidity defines the validity period of the CA certificates generated by the operator.
  caValidity: 8760h

  # caRotateBefore defines when to rotate a CA certificate that is due to expire.
  caRotateBefore: 24h
  # certificatesValidity defines the validity period of certificates generated by the operator.
  certificatesValidity: 8760h
  # certificatesRotateBefore defines when to rotate a certificate that is due to expire.
  certificatesRotateBefore: 24h
  # setDefaultSecurityContext determines whether a default security context is set on application containers created by the operator.
  setDefaultSecurityContext: true
  # kubeClientTimeout sets the request timeout for Kubernetes API calls made by the operator.
  kubeClientTimeout: 60s
  # elasticsearchClientTimeout sets the request timeout for Elasticsearch API calls made by the operator.
  elasticsearchClientTimeout: 180s
  # validateStorageClass specifies whether storage classes volume expansion support should be verified.
  # Can be disabled if cluster-wide storage class RBAC access is not available.
  validateStorageClass: true
# Internal use only
internal:
  manifestGen: false

More information needed? You can always read the release notes, documentation or contact me!

https://fullstaq.com

https://www.elastic.co/guide/en/cloud-on-k8s/master/release-highlights-1.3.0.html

https://www.elastic.co/guide/en/cloud-on-k8s/current/release-notes-1.3.0.html

https://www.elastic.co/guide/en/cloud-on-k8s/current/index.html