Equinix SmartKey Secures Kubernetes Secret Stores

Anand Ozarkar

Kubernetes has become an inherent part of efficiently developing, managing and deploying modern cloud applications. It offers capabilities to facilitate the smooth execution of containers that reduce overall compute costs. The Secrets Store for passwords, OAuth tokens, and SSH keys is a critical component of Kubernetes. By design, the Kubernetes Secret Store provides a secure means of storing confidential information external to container images or Pod specifications. Developers determine which information is kept in the Secrets Store. The system can also create and maintain secrets there.

The sensitive nature of the information maintained in the Kubernetes Secrets Store requires tight security to prevent attacks that can result in damage to corporate reputation and revenue. Security threats aren’t just external. Knowledgeable, disgruntled employees with systems admin privileges and access to an unencrypted Kubernetes Secrets Store (or a copy) can be equally destructive.

Get security control for all clouds

See how Equinix SmartKey, offered on cloud-neutral Platform Equinix™, simplifies data protection across any cloud architecture.

Start Your Free Trial
smartkey-key-lock-dark-1-300x200

Best Practices for Kubernetes Secret Store Security

Although Kubernetes maintain secrets separately from container images, a few additional best practices enhance the security level of your Kubernetes infrastructure—separation of duties,  auditing, rotating secrets and encryption.

  • Separation of duties – Developers, systems administrators and users should not have overlapping access to secrets. Often times, there is a very thin line between who can manage the secret and who can consume it. Corporate security strategy should clearly designate separated responsibilities and enforce access rights based on well-defined roles and access needs.
  • Auditable security – Well-implemented security strategies ensure the assignment and changes to access rights are auditable, including the creation of secrets. Every access to a secret should also be recorded and auditable in the event there’s any question regarding access
  • Regular rotation – Rotate secrets on a regular basis or at the minimum, anytime you suspect they may have been compromised.
  • Encryption – Encryption of secrets is the most fundamental means of preventing access by unauthorized parties. Kubernetes enables enterprises to meet encryption requirements with a variety of key management service plug-ins, and enterprises are urged to take advantage of this option.

SmartKey Kubernetes KMS Plugin

Kubernetes uses the etcd key-value store to maintain secrets. Kubernetes v1.13 introduced support for key management service (KMS) plugins to protect secrets maintained in etcd. Encrypting secrets prevents any hacker who hijacks etcd from actually accessing those secrets.

Equinix, a global colocation data center and interconnection provider that hosts 1,800+ network and 2,900+ cloud and IT service providers recently launched an open-source initiative that enables Kubernetes infrastructure security through the SmartKey Kubernetes KMS Plugin. SmartKey is built on Intel’s SGX technology that ensures encryption keys are only available inside a secure enclave. Key material is never available in plaintext to any software component. SmartKey brings the advantages of  Intel’s SGX technology to the Kubernetes container environment.

The SmartKey Kubernetes KMS Plugin project and a detailed user guide are available at https://github.com/equinix/smartkey-kubernetes-kms. Implementing the SmartKey Kubernetes KMS Plugin is straightforward. The following example shows the clear advantages of using the SmartKey plugin to protect the Kubernetes Secret Store.

Unencrypted Secrets Are Security Risks

Without encrypting the Kubernetes Secret Store, keys are available and readable to anyone who has access, as the example below shows. If a systems (IT) administrator decides to go rogue then secrets can easily be stolen. An administrator can take a snapshot of the secrets store or steal the disk containing secrets. If secrets are encrypted before they are stored then risks are mitigated and damage is minimized. A rogue administrator can only destroy them but not use them for any other purpose.

Create a secret using the kubectl command.

Without encryption, the secret is available in plaintext when read from etcd using etcdctl utility.

Using kubectl, the etcdctl utility can reveal the secret to anyone—authorized users or a hacker. In the event the etcdctl is backed up to a remote location and the backup is stolen then the secrets stored in etcd are readable.

Kubernetes Secrets Encrypted with SmartKey

When the SmartKey Kubernetes KMS Plugin is installed and configured, Kubernetes secrets attain a significantly higher level of security, as the following examples demonstrate.

Install the SmartKey Kubernetes KMS Plugin.

Two small configuration steps are required prior to # running. First, configure smartkey.yaml.

Then configure smartkey-grpc.yaml with your appropriate SmartKey account:

Start the SmartKey GRPC service by using the command “sudo service smartkey-grpc start”.

Then configure api-server.yaml to include the special flag “—encryption-provider”. This flag tells Kubernetes to contact the SmartKey KMS while interacting with etcd.

At this point, SmartKey Kubernetes KMS Plugin has been installed, configured and is running.

To confirm the effect of SmartKey, we create a new secret using kubectl.

The result? The secret is encrypted and unintelligible to anyone—authorized employees or hackers.

Even if someone tries to bypass Kubernetes, the secrets remain unintelligible, as shown when trying to read the secret using kubectl just like an application would.

SmartKey Advantages in Hybrid and Multicloud Environments

There are other providers of Kubernetes key management service (KMS). However, the SmartKey Kubernetes KMS Plugin offers more to enterprises that have made investments in cloud environments:

  • Cloud-neutral, and accessible from Kubernetes containers running on leading cloud providers like Amazon, Azure, Google, Oracle and Salesforce.
  • Multi-site and hybrid-cloud support, including support for single enterprise-wide key across cloud and IT data centers.
  • Single, centrally-managed encryption key management service, regardless of where encrypted data and secrets may actually reside,
  • Enterprise-level access controls and audit logging.
  • Maintain encryption keys proximate, but physically and logically separate from encrypted data to provide an added level of data security as well as minimize latency.

For enterprises confronted with the challenges of efficiently managing encryption across multiple cloud environments, the SmartKey Kubernetes KMS Plugin provides an ideal solution.

Developers, network security professionals or cloud architects interested in learning first-hand about the advantages of SmartKey are invited to register for the free trial.

Anand Ozarkar
Anand Ozarkar Senior Manager, Product Software Architecture and Engineering