This page will walk you through the main steps required to manually back up and restore the state of a Pachyderm cluster in production. Details on how to perform those steps might vary depending on your infrastructure and cloud provider / on-premises setup. Refer to your provider’s documentation.
Overview #
Pachyderm state is stored in two main places:
- An object-store holding Pachyderm’s data.
- A PostgreSQL instance made up of one or two databases:
pachyderm
holding Pachyderm’s metadata anddex
holding authentication data.
Backing up a Pachyderm cluster involves snapshotting both the object store and the PostgreSQL database(s), in a consistent state, at a given point in time.
Restoring it involves re-populating the database(s) and the object store using those backups, then recreating a Pachyderm cluster.
- Make sure that you have a bucket for backup use, separate from the object store used by your cluster.
- Depending on the reasons behind your cluster recovery, you might choose to use an existing vs. a new instance of PostgreSQL and/or the object store.
Backup A Standalone Enterprise Server #
Backing up and restoring an Enterprise Server is similar to the backing up and restoring of a regular cluster, with three slight variations:
- The name of its Kubernetes deployment is
pach-enterprise
versuspachd
in the case of a regular cluster. - The Enterprise Server does not use an Object Store.
- An Enterprise server only requires a
dex
database.
Make sure that pachctl
and kubectl
are pointing to the right cluster. Check your Enterprise Server context: pachctl config get active-enterprise-context
, or pachctl config set active-enterprise-context <my-enterprise-context-name> --overwrite
to set it.
- Pause the Enterprise Server like you would pause a regular cluster by running
pachctl enterprise pause
(Enterprise users), or usingkubectl
.
kubectl users:
There is a difference with the pause of a regular cluster. The deployment of the enterprise server is named pach-enterprise
; therefore, the first command should be:
kubectl scale deployment pach-enterprise --replicas 0
There is no need to pause all the Pachyderm clusters registered to the Enterprise Server to backup the enterprise server; however, pausing the Enterprise server will result in your clusters becoming unavailable.
As a reminder, the Enterprise Server does not use any object-store. Therefore, the backup of the Enterprise Server only consists in backing up the database
dex
.Resume the operations on your Enterprise Server by running
pachctl enterprise unpause
(Enterprise users) to scale thepach-enterprise
deployment back up. Alternatively, if you usedkubectl
, run:kubectl scale deployment pach-enterprise --replicas 1
Restore An Enterprise Server #
Follow the cluster restoration steps while skipping all tasks related to creating and populating a new object-store.
Once your cluster is up and running, check that all your clusters are automatically registered with your new Enterprise Server.