Configure a Vault server to hold your Code42 archive keystore
Overview
External keystores mean that even in the Code42 cloud environment, you can maintain copies of your Code42 encryption keys that secure your backed-up data.
While the keys reside securely on your users' devices for use in encrypting and decrypting files, you can store a copy of the encryption keys separately from the Code42 cloud, in a Vault keystore system. Vault is a third-party application specifically built to secure secrets. This article describes how to configure a private, self-administered Vault server to store copies of your Code42 encryption keys.
Once you have your Vault server running and tested as described in this article, move copies of your Code42 keys into Vault by following the instructions for migrating keys to a new keystore.
Considerations
External keystores, like a Vault keystore system, are not available for use in the Code42 federal environment.
Vault versions
To ensure secure operation, install the latest Vault version from the Vault downloads page. Previous versions are available from the Vault releases page.
Vault uses two SSL certificates
Before you begin
Install a Vault server
Install a Vault server at a location available to the Code42 cloud. For system requirements, see the Vault Reference Architecture guide.
- Set up a machine or virtual machine with network access that allows TLS/HTTPS communication with the Code42 cloud.
- Plan storage capacity of roughly 1 KB for each of your Code42 users.
- Download a Vault package for that machine's operating system.
- Install and configure a Vault server as described by the Vault documentation.
- Connect your Vault server to whatever storage backend suits your purposes.
Back up Vault
To guard against loss of your encryption keys, make sure that you back up your Vault. For Vault's backup instructions, see the Vault Disaster Recovery Replication guide.
Configure the Vault server
The following steps describe configuring a Vault server on a Linux operating system.
The commands below are samples only. Code42 does not guarantee that these commands are suitable for your environment.
Step 1: Configure your Vault to work with Code42
To allow the Code42 cloud access to your encryption keys, configure your Vault server. For more information about configuring Vault, see Vault's documentation.
Change the values in the examples below as needed for your environment.
- Set environment variables:
- Set Vault_ADDR to the Vault external DNS, including the protocol (https) and port mapping (8200):
export VAULT_ADDR="https://vault.example.com:8200"
- Set Vault_DNS to the Vault external DNS name:
export VAULT_DNS="vault.example.com"
- Set Vault_ADDR to the Vault external DNS, including the protocol (https) and port mapping (8200):
- Enable the secrets engine.
- Enable the secrets engine with the
kv
option run in KV Version 1 mode:vault secrets enable -path=secret kv
- Enable certificate authentication:
vault auth enable cert
- Tune Vault to issue short-lived authentication tokens for TLS connections:
vault secrets tune -default-lease-ttl=60s auth/cert
- Mount the Vault public key infrastructure (PKI):
vault secrets enable pki
- Tune Vault to set the life span of the SSL certificates it generates;
vault secrets tune -max-lease-ttl=87600h pki
- Enable the secrets engine with the
- Generate certificates.
- Generate and configure CA_cert.crt:
vault write -field=certificate pki/root/generate/internal common_name="code42.com" ttl=87600h > CA_cert.crt vault write pki/config/urls issuing_certificates="http://${VAULT_DNS}/v1/pki/ca" crl_distribution_points="http://${VAULT_DNS}:8200/v1/pki/crl"
- Generate intermediate certificates:
vault secrets enable -path=pki_int pki vault secrets tune -max-lease-ttl=43800h pki_int vault write -format=json pki_int/intermediate/generate/internal common_name="code42.com Intermediate Authority" ttl="43800h" | jq -r '.data.csr' > pki_intermediate.csr vault write -format=json pki/root/sign-intermediate csr=@pki_intermediate.csr format=pem_bundle ttl="43800h" | jq -r '.data.certificate' > intermediate.cert.pem vault write pki_int/intermediate/set-signed certificate=@intermediate.cert.pem
- Create a role to generate credentials:
vault write pki_int/roles/cpRole allow_any_name="true" allow_subdomains=true client_flag=true
- Issue the code42.com administration certificate and parse it:
vault write pki_int/issue/cpRole common_name="code42.com" ttl="8760h" -format=json > certs.json cat certs.json | jq -r '.data.ca_chain[0]' > ca_chain.pem cat certs.json | jq -r '.data.certificate' > certificate.pem cat certs.json | jq -r '.data.private_key' > private_key.pem
Set TTL as needed
A settingttl="8760h"
sets the "time to live" for the code42.com administration certificate to one year. Update this setting according to your company's best practices for certificate management.
Before this certificate reaches its expiration date as set by the TTL setting, you must make a new certificate to maintain uninterrupted connection to Code42. See Create a new administration certificate below.
- Generate and configure CA_cert.crt:
- Create policies.
- Create code42PolicyFile.hcl with the following policy entries:
path "pki/issue/*" {capabilities = ["create","read", "update", "delete", "list"]} path "sys/policy/*" {capabilities = ["create", "read", "update", "delete", "list"]} path "auth/cert/certs/*" {capabilities = ["create", "read", "update", "delete", "list"]}
- Write the policy and authorize the certificate:
vault policy write code42Policy code42PolicyFile.hcl vault write auth/cert/certs/code42User display_name="code42User" policies=code42Policy certificate=@certificate.pem ttl=60
- Create code42PolicyFile.hcl with the following policy entries:
- Create a PKCS12 key and certificate file for import to the Code42 cloud:
export LC_CTYPE=C openssl_pw=$(head /dev/urandom | tr -dc A-Za-z0-9 | head -c 16 ; echo '') echo "openssl_pw: ${openssl_pw}" >> secrets.env openssl pkcs12 -export -out code42.p12 -inkey private_key.pem -in certificate.pem -password pass:${openssl_pw}
- Clean up files:
rm ./CA_cert.crt rm ./pki_intermediate.csr rm ./intermediate.cert.pem rm ./certs.json rm ./certificate.pem rm ./private_key.pem rm ./ca_chain.pem rm ./code42PolicyFile unset openssl_pw
Step 2: Open your firewall to the Code42 cloud
Open your firewall to our Code42 IP addresses.
Implement firewall rules to open an inbound port at your Vault server that allows requests from the Code42 cloud. This allows the Code42 cloud to access keys in your Vault to encrypt and decrypt user backup data.
Step 3: Configure the Code42 cloud to work with your Vault
The code42.p12 file created at the end of Step 1 is a PKCS12 certificate (also called a PFX or P12 file) that identifies your Code42 cloud organization to your Vault server.
Provide the file and its password to the Code42 cloud by following the instructions for migrating keys to a new keystore.
The maximum file size is 5 MB.
Create a new administration certificate
The preceding steps describe how to configure your Vault installation to work with Code42 for the first time, and demonstrate how to set the Time To Live (TTL) for the original administration certificate (code42.com certificate). Before the original certificate expires per the TTL setting, you must create a new certificate to replace the original certificate to maintain an uninterrupted connection to Code42.
Following are sample steps to create the new administration certificate. Adapt the steps as needed to fit your environment.
-
Issue a new code42.com administration certificate and split the result into separate files:
vault write pki_int/issue/cpRole common_name="code42.com" ttl="8760h" -format=json > certs.json cat certs.json | jq -r '.data.ca_chain[0]' > ca_chain.pem cat certs.json | jq -r '.data.certificate' > certificate.pem cat certs.json | jq -r '.data.private_key' > private_key.pem
- Update the system certificate in the Vault certificate store:
vault write auth/cert/certs/code42User display_name="code42User" policies=code42Policy certificate=@certificate.pem ttl=60
- Generate a PFX/P12 file from the issued certificate:
export LC_CTYPE=C openssl_pw=$(head /dev/urandom | tr -dc A-Za-z0-9 | head -c 16 ; echo '') echo "openssl_pw: ${openssl_pw}" >> secrets.env openssl pkcs12 -export -out code42.p12 -inkey private_key.pem -in certificate.pem -password pass:${openssl_pw}
- Clean up stray files and the environment variable:
rm ./certs.json rm ./certificate.pem rm ./private_key.pem rm ./ca_chain.pem unset openssl_pw
- Update the certificate in the Code42 cloud.
Vault best practices
Following are best practices for configuring Vault. For more information, see Vault's Production Hardening guide.
Use a tested and trusted backend
Vault is compatible with different kinds of storage backends. Choose a storage backend that fits your production needs. Consul is a backend developed by HashiCorp that can function as both a backend for storing data and also service discovery in a multi-cluster setup of Vault. DynamoDB is also widely-used because it supports high availability mode while providing persistent data for Vault.
Implement a disaster recovery strategy
In a Vault disaster recovery (DR) strategy, you configure your deployment to replicate data across datacenters. This setup provides performance improvements in addition to disaster recovery. Enable both DR primary replication and secondary replication. Disaster recovery is only available for Vault Enterprise and Vault Enterprise Modules but not Vault Open Source (compare Vault packages).
Run in high availability mode
High availability mode (HA) requires multiple Vault nodes deployed as members of the same cluster. This ensures that when the active node goes offline, the secondary node can continue to respond to API calls. Certain storage backends do not support an HA deployment, so it is important to ensure this is considered when choosing a backend. Some storage backends that support HA include Consul, DynamoDB, Etcd, FoundationDB, and Zookeeper.
Do not run in development mode
You can run Vault in development mode with the following command: vault server -dev
Never use this command to launch Vault in production environments. Launch Vault from a script like init.d or systemd as found in a Linux distribution (distro). You should also create a Vault user and group, and preferably a volume mount that only Vault can read and write to.
Use a network load balancer
Use a network load balancer compatible with TLS to distribute traffic across a cluster of Vault servers. Load balancing will improve system responsiveness and increase availability of Vault data. You can perform network load balancing with Consul using a number of different tools.
Always use TLS
Vault should always be used with Transport Layer Security (TLS) in production. Most high availability production environments would normally have a load balancer or reverse proxy route traffic to the Vault nodes. It is important to ensure that this traffic is not terminated on these devices. There should end-to-end TLS communication from Code42 authorities to the load balancer to Vault nodes.
Run as a single tenant
Run Vault as a single tenant on a machine. This reduces the risk of compromising the Vault service and exposing the keys because there is no other process on the machine that can be compromised.
Configure firewall rules
Vault listens on ports 8200 and 8201. In a production deployment you should use a local firewall to control all traffic to Vault. In cloud provider environments, use a security group or firewall rule to achieve the same objective.
Enable auditing
Enable auditing to provide a record of Vault operations. You can send audit logs to a different destination or other system processes like syslog.
Consider your upgrade procedure
If you use automated deployments for upgrading applications, it is important to consider how you upgrade Vault and your storage backends. Choose a backend that allows for persistent data during automated deployment upgrades. DynamoDB and Google Cloud Spanner both ensure persistency during a rolling upgrade.