Backups

Creating a backup

The backup CR represent a backup of the cluster. Backups are created automatically by the CronJob generated using the settings in backup configuration or manually by creating a backup CR.


Kind: SGBackup

listKind: SGBackupList

plural: sgbackups

singular: sgbackup


Spec

Property Required Updatable Type Default Description
sgCluster string
The name of the SGCluster from which this backup is/will be taken.
subjectToRetentionPolicy booolean false
Indicate if this backup is permanent and should not be removed by the automated retention policy.

Status

Property Type Description
internalName string
The name of the backup.
process object
backupInformation object
sgBackupConfig object
The name of the backup configuration used to perform this backup.

Backup Process

Property Type Description
status string
The status of the backup.
jobPod string
The name of pod assigned to this backup. StackGres utilizes internally a locking mechanism based on the pod name of the job that creates the backup.
failure string
If the status is failed this field will contain a message indicating the failure reason.
subjectToRetentionPolicy boolean
Status (may be transient) until converging to spec.subjectToRetentionPolicy.
timing object

Backup Timing

Property Type Description
start string
The start time of backup.
end string
The end time of backup.
stored string
The time at which the backup is safely stored in the object storage.

Backup Intormation

Property Type Description
hostname string
The hostname of instance where the backup is taken.
systemIdentifier string
The Postgres system identifier of the cluster this backup is taken from.
postgresVersion string
The Postgres version of the server where backup is taken.
pgData string
The data directory where the backup is taken.
size object
lsn object
startWalFile string
The WAL segment file name when backup was started.
controlData object
An object containing data from the output of pg_controldata on the backup.

Backup Size

Property Type Description
compressed integer
Size (in bytes) of the compressed backup.
uncompressed integer
Size (in bytes) of the uncompressed backup.

Backup LSN

Property Type Description
start string
The LSN of when backup started.
end string
The LSN of when backup finished.

Example:

apiVersion: stackgres.io/v1beta1
kind: SGBackup
metadata:
  name: backup
spec:
  sgCluster: stackgres
  subjectToRetentionPolicy: true
status:
  internalName: base_00000002000000000000000E 
  sgBackupConfig:
    compression: lz4
    storage:
      s3Compatible:
        awsCredentials:
          secretKeySelectors:
            accessKeyId:
              key: accesskey
              name: minio
            secretAccessKey:
              key: secretkey
              name: minio
        endpoint: http://minio:9000
        enablePathStyleAddressing: true
        bucket: stackgres
        region: k8s
      type: s3Compatible
  process:
    status: Completed
    jobPod: backup-backup-q79zq
    subjectToRetentionPolicy: true
    timing:
      start: "2020-01-22T10:17:24.983902Z"
      stored: "2020-01-22T10:17:27.183Z"
      end: "2020-01-22T10:17:27.165204Z"
  backupInformation:
    hostname: stackgres-1
    systemIdentifier: "6784708504968245298"
    postgresVersion: "110006"
    pgData: /var/lib/postgresql/data
    size:
      compressed: 6691164     
      uncompressed: 24037844
    lsn:
      start: "234881064"
      end: "234881272"
    startWalFile: 00000002000000000000000E

Configuration

Backup configuration allow to specify when and how backups are performed. By default this is done at 5am UTC in a window of 1 hour, you may change this value in order to perform backups for another time zone and period of time. The backup configuration CR represent the backups configuration of the cluster.


Kind: SGBackupConfig

listKind: SGBackupConfigList

plural: sgbackupconfigs

singular: sgbackupconfig


Spec

Property Required Updatable Type Default Description
baseBackups object
Back backups configuration.
storage object
Backup storage configuration.

Example:

apiVersion: stackgres.io/v1beta1
kind: SGBackupConfig
metadata:
  name: backupconf
spec:
  baseBackups:
    retention: 5
    cronSchedule: 0 5 * * *
    compression: lz4
    performance:
      maxDiskBandwitdh: 26214400 #25 MB per seceod
      maxNetworkBandwitdh: 52428800 #50 MB per second
      uploadDiskConcurrency: 2
  storage:
    type: s3Compatible
    s3Compatible:
      bucket: stackgres
      region: k8s
      enablePathStyleAddressing: true
      endpoint: http://my-cluster-minio:9000
      awsCredentials:
        secretKeySelectors:
          accessKeyId:
            key: accesskey
            name: my-cluster-minio
          secretAccessKey:
            key: secretkey
            name: my-cluster-minio

Default settings are stored in the same namespaces of the stackgres operator, with the name defaultbackupconfig

Given a stackgres operator installed in the stackgres namespace we can see the backup default values with de command:

kubectl get sgbackupconfig -n stackgres defaultbackupconfig -o yaml

If a backup configuration is not specified in the cluster settings, a new one will be created with the default values.

The default name of backup configuration CR is defaultbackupconfig

Base Backups

Property Required Updatable Type Default Description
retention integer 5

Based on this parameter, an automatic retention policy is defined to delete old base backups. This parameter specifies the number of base backups to keep, in a sliding window. Consequently, the time range covered by backups is periodicity*retention, where periodicity is the separation between backups as specified by the cronSchedule property.

Default is 5.

cronSchedule string 05:00 UTC

Continuous Archiving backups are composed of periodic base backups and all the WAL segments produced in between those base backups. This parameter specifies at what time and with what frequency to start performing a new base backup.

Use cron syntax (m h dom mon dow) for this parameter, i.e., 5 values separated by spaces:

  • m: minute, 0 to 59
  • h: hour, 0 to 23
  • dom: day of month, 1 to 31 (recommended not to set it higher than 28)
  • mon: month, 1 to 12
  • dow: day of week, 0 to 7 (0 and 7 both represent Sunday)

Also ranges of values (start-end), the symbol * (meaning first-last) or even */N, where N is a number, meaning “every N, may be used. All times are UTC. It is recommended to avoid 00:00 as base backup time, to avoid overlapping with any other external operations happening at this time. If not provided, full backups will be performed each day at 05:00 UTC

compression string lz4
Select the backup compression algorithm. Possible options are: lz4, lzma, brotli. The default method is lz4. LZ4 is the fastest method, but compression ratio is the worst. LZMA is way slower, but it compresses backups about 6 times better than LZ4. Brotli is a good trade-off between speed and compression ratio, being about 3 times better than LZ4.
performance object

Base Backup Performance

Property Required Updatable Type Default Description
maxDiskBandwitdh integer unlimited
Maximum disk read I/O when performing a backup. In bytes (per second).
maxNetworkBandwitdh integer unlimited
Maximum storage upload bandwidth to be used when storing the backup. In bytes (per second).
uploadDiskConcurrency integer 1
Backup storage may use several concurrent streams to store the data. This parameter configures the number of parallel streams to use. By default, it will use 1 (one stream).

Storage Configuration

Property Required Updatable Type Default Description
type string
Determine the type of object storage used for storing the base backups and WAL segments. Possible values:

  • s3: Amazon Web Services S3 (Simple Storage Service).
  • s3Compatible: non-AWS services that implement a compatibility API with AWS S3.
  • gcs: Google Cloud Storage.
  • azureBlob: Microsoft Azure Blob Storage.
s3 if type = s3 object
Amazon Web Services S3 configuration.
s3Compatible if type = s3Compatible object
AWS S3-Compatible API configuration
gcs if type = gcs object
Google Cloud Storage configuration.
azureBlob if type = azureblob object
Azure Blob Storage configuration.

S3

S3 - Amazon Web Services S3 configuration

Property Required Updatable Type Default Description
bucket string
AWS S3 bucket name.
path string
Optional path within the S3 bucket. Note that StackGres generates in any case a folder per StackGres cluster, using the SGCluster.metadata.name.
awsCredentials object
The credentials to access AWS S3 for writing and reading.
region string
The AWS S3 region. The Region may be detected using s3:GetBucketLocation, but if you wish to avoid giving permissions to this API call or forbid it from the applicable IAM policy, you must then specify this property.
storageClass string
The Amazon S3 Storage Class to use for the backup object storage. By default, the STANDARD storage class is used. Other supported values include STANDARD_IA for Infrequent Access and REDUCED_REDUNDANCY.

S3 - Amazon Web Services S3 Compatible configuration

Property Required Updatable Type Default Description
bucket string
Bucket name.
path string
Optional path within the S3 bucket. Note that StackGres generates in any case a folder per StackGres cluster, using the SGCluster.metadata.name.
awsCredentials object
The credentials to access AWS S3 for writing and reading.
region string
The AWS S3 region. The Region may be detected using s3:GetBucketLocation, but if you wish to avoid giving permissions to this API call or forbid it from the applicable IAM policy, you must then specify this property.
storageClass string
The Amazon S3 Storage Class to use for the backup object storage. By default, the STANDARD storage class is used. Other supported values include STANDARD_IA for Infrequent Access and REDUCED_REDUNDANCY.
endpoint string
Overrides the default url to connect to an S3-compatible service. For example: http://s3-like-service:9000.
enablePathStyleAddressing boolean
Enable path-style addressing (i.e. http://s3.amazonaws.com/BUCKET/KEY) when connecting to an S3-compatible service that lacks support for sub-domain style bucket URLs (i.e. http://BUCKET.s3.amazonaws.com/KEY). Defaults to false.

Amazon Web Services Credentials

Property Required Updatable Type Default Description
secretKeySelectors object
A Kubernetes SecretKeySelector to reference the Secrets that contain the information about the awsCredentials.

Amazon Web Services Secret Key Selector

Property Required Updatable Type Default Description
accessKeyId object
SecretKeySelector containing the AWS Access Key ID secret.
secretAccessKey object
SecretKeySelector containing the AWS Secret Access Key secret.

GSC - Google Cloud Storage configuration

Property Required Updatable Type Default Description
bucket string
GCS bucket name.
path string
Optional path within the GCS bucket. Note that StackGres generates in any case a folder per StackGres cluster, using the SGCluster.metadata.name.
gcpCredentials object
The credentials to access GCS for writing and reading.

GCP Credentials

Property Required Updatable Type Default Description
secretKeySelectors object
A Kubernetes SecretKeySelector to reference the Secrets that contain the information about the Service Account to access GCS.

GCP Secret Key Selector

Property Required Updatable Type Default Description
serviceAccountJSON object
A service account key from GCP. In JSON format, as downloaded from the GCP Console.

AZURE - Azure Blob Storage configuration

Property Required Updatable Type Default Description
bucket string
Azure Blob Storage bucket name.
path string
Optional path within the Azure Blobk bucket. Note that StackGres generates in any case a folder per StackGres cluster, using the SGCluster.metadata.name.
azureCredentials object
The credentials to access Azure Blob Storage for writing and reading.

Azure Credentials

Property Required Updatable Type Default Description
secretKeySelectors object
Kubernetes SecretKeySelectors to reference the Secrets that contain the information about the azureCredentials.

Azure Secret Key Selector

Property Required Updatable Type Default Description
storageAccount object
SecretKeySelector containing the name of the storage account.
accessKey object
SecretKeySelector containing the primary or secondary access key for the storage account.