SGBackup

Kind: SGBackup

listKind: SGBackupList

plural: sgbackups

singular: sgbackup

shortNames sgbkp

The SGBackup custom resource represents a backup of the Postgres cluster. Backups are created automatically by a cron job configured using the settings in the backup configuration or manually by creating a SGBackup.

Example:

apiVersion: stackgres.io/v1
kind: SGBackup
metadata:
  name: backup
spec:
  sgCluster: stackgres
  managedLifecycle: true
status:
  internalName: base_00000002000000000000000E
  sgBackupConfig:
    compression: lz4
    storage:
      s3Compatible:
        awsCredentials:
          secretKeySelectors:
            accessKeyId:
              key: accesskey
              name: minio
            secretAccessKey:
              key: secretkey
              name: minio
        endpoint: http://minio:9000
        enablePathStyleAddressing: true
        bucket: stackgres
        region: k8s
      type: s3Compatible
  process:
    status: Completed
    jobPod: backup-backup-q79zq
    managedLifecycle: true
    timing:
      start: "2020-01-22T10:17:24.983902Z"
      stored: "2020-01-22T10:17:27.183Z"
      end: "2020-01-22T10:17:27.165204Z"
  backupInformation:
    hostname: stackgres-1
    systemIdentifier: "6784708504968245298"
    postgresVersion: "110006"
    pgData: /var/lib/postgresql/data
    size:
      compressed: 6691164
      uncompressed: 24037844
    lsn:
      start: "234881064"
      end: "234881272"
    startWalFile: 00000002000000000000000E

See also Backups section.

The SGBackup represents a manual or automatically generated backup of an SGCluster configured with an SGObjectStorage.

When a SGBackup is created a Job will perform a full backup of the database and update the status of the SGBackup with the all the information required to restore it and some stats (or a failure message in case something unexpected happened). After an SGBackup is created the same Job performs a reconciliation of the backups by applying the retention window that has been configured in the SGObjectStorage and removing the backups with managed lifecycle and the WAL files older than the ones that fit in the retention window. The reconciliation also removes backups (excluding WAL files) that do not belongs to any SGBackup. If the target storage of the SGObjectStorage is changed deletion of an SGBackup backups with managed lifecycle and the WAL files older than the ones that fit in the retention window and of backups that do not belongs to any SGBackup will not be performed anymore on the previous storage, only on the new target storage.

A manual or automatically generated backup of an SGCluster configured with backups.

When a SGBackup is created a Job will perform a full backup of the database and update the status of the SGBackup with the all the information required to restore it and some stats (or a failure message in case something unexpected happened). Backup generated by SGBackup are stored in the object storage configured with an SGObjectStorage together with the WAL files or in a VolumeSnapshot (separated from the WAL files that will be still stored in an object storage) depending on the backup configuration of the targeted SGCluster. After an SGBackup is created the same Job performs a reconciliation of the backups by applying the retention window that has been configured in the SGCluster and removing the backups with managed lifecycle and the WAL files older than the ones that fit in the retention window. The reconciliation also removes backups (excluding WAL files) that do not belongs to any SGBackup (including copies). If the target storage is changed deletion of an SGBackup backups with managed lifecycle and the WAL files older than the ones that fit in the retention window and of backups that do not belongs to any SGBackup will not be performed anymore on the previous storage, only on the new target storage. If the reconciliation of backups fails the backup itself do not fail and will be re-tried the next time a SGBackup or shecduled backup Job take place.

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

apiVersion ✓ string stackgres.io/v1
kind ✓ string SGBackup
metadata ✓ ✓ object Refer to the Kubernetes API documentation for the fields of the metadata field.
spec ✓ ✓ object
status ✓ object

SGBackup.spec

↩ Parent

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

managedLifecycle ✓ boolean Indicate if this backup is not permanent and should be removed by the automated retention policy. Default is false.
maxRetries ✓ integer The maximum number of retries the backup operation is allowed to do after a failure.

A value of 0 (zero) means no retries are made. Defaults to: 3.

reconciliationTimeout ✓ integer Allow to set a timeout for the reconciliation process that take place after the backup.

If not set defaults to 300 (5 minutes). If set to 0 it will disable timeout.

Failure of reconciliation will not make the backup fail and will be re-tried the next time a SGBackup or shecduled backup Job take place.

Default: 300
sgCluster ✓ string The name of the SGCluster from which this backup is/will be taken.

If this is a copy of an existing completed backup in a different namespace the value must be prefixed with the namespace of the source backup and a dot . (e.g. <cluster namespace>.<cluster name>) or have the same value if the source backup is also a copy.

timeout ✓ integer Allow to set a timeout for the backup creation.

If not set it will be disabled and the backup operation will continue until the backup completes or fail. If set to 0 is the same as not being set.

Make sure to set a reasonable high value in order to allow for any unexpected delays during backup creation (network low bandwidth, disk low throughput and so forth).

SGBackup.status

↩ Parent

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

backupInformation ✓ object
backupPath ✓ string The path were the backup is stored.
internalName ✓ string The name of the backup.
process ✓ object
sgBackupConfig ✓ object The backup configuration used to perform this backup.
volumeSnapshot ✓ object The volume snapshot configuration used to restore this backup.

SGBackup.status.backupInformation

↩ Parent

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

controlData ✓ object An object containing data from the output of pg_controldata on the backup.
hostname ✓ string Hostname of the instance where the backup is taken from.
lsn ✓ object
pgData ✓ string Data directory where the backup is taken from.
postgresVersion ✓ string Postgres version of the server where the backup is taken from.
size ✓ object
sourcePod ✓ string Pod where the backup is taken from.
startWalFile ✓ string WAL segment file name when the backup was started.
systemIdentifier ✓ string Postgres system identifier of the cluster this backup is taken from.
timeline ✓ string Backup timeline.

SGBackup.status.backupInformation.controlData

↩ Parent

An object containing data from the output of pg_controldata on the backup.

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

Backup end location ✓ string
Backup start location ✓ string
Blocks per segment of large relation ✓ string
Bytes per WAL segment ✓ string
Catalog version number ✓ string
Data page checksum version ✓ string
Database block size ✓ string
Database cluster state ✓ string
Database system identifier ✓ string
Date/time type storage ✓ string
End-of-backup record required ✓ string
Fake LSN counter for unlogged rels ✓ string
Float4 argument passing ✓ string
Float8 argument passing ✓ string
Latest checkpoint location ✓ string
Latest checkpoint’s NextMultiOffset ✓ string
Latest checkpoint’s NextMultiXactId ✓ string
Latest checkpoint’s NextOID ✓ string
Latest checkpoint’s NextXID ✓ string
Latest checkpoint’s PrevTimeLineID ✓ string
Latest checkpoint’s REDO WAL file ✓ string
Latest checkpoint’s REDO location ✓ string
Latest checkpoint’s TimeLineID ✓ string
Latest checkpoint’s full_page_writes ✓ string
Latest checkpoint’s newestCommitTsXid ✓ string
Latest checkpoint’s oldestActiveXID ✓ string
Latest checkpoint’s oldestCommitTsXid ✓ string
Latest checkpoint’s oldestMulti’s DB ✓ string
Latest checkpoint’s oldestMultiXid ✓ string
Latest checkpoint’s oldestXID ✓ string
Latest checkpoint’s oldestXID’s DB ✓ string
Maximum columns in an index ✓ string
Maximum data alignment ✓ string
Maximum length of identifiers ✓ string
Maximum size of a TOAST chunk ✓ string
Min recovery ending loc’s timeline ✓ string
Minimum recovery ending location ✓ string
Mock authentication nonce ✓ string
Size of a large-object chunk ✓ string
Time of latest checkpoint ✓ string
WAL block size ✓ string
max_connections setting ✓ string
max_locks_per_xact setting ✓ string
max_prepared_xacts setting ✓ string
max_wal_senders setting ✓ string
max_worker_processes setting ✓ string
pg_control last modified ✓ string
pg_control version number ✓ string
track_commit_timestamp setting ✓ string
wal_level setting ✓ string
wal_log_hints setting ✓ string

SGBackup.status.backupInformation.lsn

↩ Parent

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

end ✓ string LSN of when the backup finished.
start ✓ string LSN of when the backup started.

SGBackup.status.backupInformation.size

↩ Parent

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

compressed ✓ integer Size (in bytes) of the compressed backup.

Format: int64
uncompressed ✓ integer Size (in bytes) of the uncompressed backup.

Format: int64

SGBackup.status.process

↩ Parent

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

failure ✓ string If the status is failed this field will contain a message indicating the failure reason.
jobPod ✓ string Name of the pod assigned to the backup. StackGres utilizes internally a locking mechanism based on the pod name of the job that creates the backup.
managedLifecycle ✓ boolean Status (may be transient) until converging to spec.managedLifecycle.
status ✓ string Status of the backup.
timing ✓ object

SGBackup.status.process.timing

↩ Parent

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

end ✓ string End time of backup.
start ✓ string Start time of backup.
stored ✓ string Time at which the backup is safely stored in the object storage.

SGBackup.status.sgBackupConfig

↩ Parent

The backup configuration used to perform this backup.

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

storage ✓ ✓ object Object Storage configuration
baseBackups ✓ object Back backups configuration.

SGBackup.status.sgBackupConfig.storage

↩ Parent

Object Storage configuration

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

type ✓ ✓ enum Determine the type of object storage used for storing the base backups and WAL segments. Possible values: * s3: Amazon Web Services S3 (Simple Storage Service). * s3Compatible: non-AWS services that implement a compatibility API with AWS S3. * gcs: Google Cloud Storage. * azureBlob: Microsoft Azure Blob Storage.

Enum: s3, s3Compatible, gcs, azureBlob
azureBlob ✓ object Azure Blob Storage configuration.
gcs ✓ object Google Cloud Storage configuration.
s3 ✓ object Amazon Web Services S3 configuration.
s3Compatible ✓ object AWS S3-Compatible API configuration
SGBackup.status.sgBackupConfig.storage.azureBlob

↩ Parent

Azure Blob Storage configuration.

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

azureCredentials ✓ ✓ object The credentials to access Azure Blob Storage for writing and reading.
bucket ✓ ✓ string Azure Blob Storage bucket name.
SGBackup.status.sgBackupConfig.storage.azureBlob.azureCredentials

↩ Parent

The credentials to access Azure Blob Storage for writing and reading.

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

secretKeySelectors ✓ object Kubernetes SecretKeySelector(s) to reference the Secret(s) that contain the information about the azureCredentials. . Note that you may use the same or different Secrets for the storageAccount and the accessKey. In the former case, the keys that identify each must be, obviously, different.
SGBackup.status.sgBackupConfig.storage.azureBlob.azureCredentials.secretKeySelectors

↩ Parent

Kubernetes SecretKeySelector(s) to reference the Secret(s) that contain the information about the azureCredentials. . Note that you may use the same or different Secrets for the storageAccount and the accessKey. In the former case, the keys that identify each must be, obviously, different.

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

accessKey ✓ ✓ object The storage account access key.
storageAccount ✓ ✓ object The Storage Account that contains the Blob bucket to be used.
SGBackup.status.sgBackupConfig.storage.azureBlob.azureCredentials.secretKeySelectors.accessKey

↩ Parent

The storage account access key.

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

key ✓ ✓ string The key of the secret to select from. Must be a valid secret key.
name ✓ ✓ string Name of the referent. More information.
SGBackup.status.sgBackupConfig.storage.azureBlob.azureCredentials.secretKeySelectors.storageAccount

↩ Parent

The Storage Account that contains the Blob bucket to be used.

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

key ✓ ✓ string The key of the secret to select from. Must be a valid secret key.
name ✓ ✓ string Name of the referent. More information.
SGBackup.status.sgBackupConfig.storage.gcs

↩ Parent

Google Cloud Storage configuration.

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

bucket ✓ ✓ string GCS bucket name.
gcpCredentials ✓ ✓ object The credentials to access GCS for writing and reading.
SGBackup.status.sgBackupConfig.storage.gcs.gcpCredentials

↩ Parent

The credentials to access GCS for writing and reading.

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

fetchCredentialsFromMetadataService ✓ boolean If true, the credentials will be fetched from the GCE/GKE metadata service and the field secretKeySelectors have to be set to null or omitted.

This is useful when running StackGres inside a GKE cluster using Workload Identity.

secretKeySelectors ✓ object A Kubernetes SecretKeySelector to reference the Secrets that contain the information about the Service Account to access GCS.
SGBackup.status.sgBackupConfig.storage.gcs.gcpCredentials.secretKeySelectors

↩ Parent

A Kubernetes SecretKeySelector to reference the Secrets that contain the information about the Service Account to access GCS.

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

serviceAccountJSON ✓ ✓ object A service account key from GCP. In JSON format, as downloaded from the GCP Console.
SGBackup.status.sgBackupConfig.storage.gcs.gcpCredentials.secretKeySelectors.serviceAccountJSON

↩ Parent

A service account key from GCP. In JSON format, as downloaded from the GCP Console.

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

key ✓ ✓ string The key of the secret to select from. Must be a valid secret key.
name ✓ ✓ string Name of the referent. More information.
SGBackup.status.sgBackupConfig.storage.s3

↩ Parent

Amazon Web Services S3 configuration.

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

awsCredentials ✓ ✓ object The credentials to access AWS S3 for writing and reading.
bucket ✓ ✓ string AWS S3 bucket name.
region ✓ string The AWS S3 region. The Region may be detected using s3:GetBucketLocation, but if you wish to avoid giving permissions to this API call or forbid it from the applicable IAM policy, you must then specify this property.
storageClass ✓ string The Amazon S3 Storage Class to use for the backup object storage. By default, the STANDARD storage class is used. Other supported values include STANDARD_IA for Infrequent Access and REDUCED_REDUNDANCY.
SGBackup.status.sgBackupConfig.storage.s3.awsCredentials

↩ Parent

The credentials to access AWS S3 for writing and reading.

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

secretKeySelectors ✓ ✓ object Kubernetes SecretKeySelector(s) to reference the Secrets that contain the information about the awsCredentials. Note that you may use the same or different Secrets for the accessKeyId and the secretAccessKey. In the former case, the keys that identify each must be, obviously, different.
SGBackup.status.sgBackupConfig.storage.s3.awsCredentials.secretKeySelectors

↩ Parent

Kubernetes SecretKeySelector(s) to reference the Secrets that contain the information about the awsCredentials. Note that you may use the same or different Secrets for the accessKeyId and the secretAccessKey. In the former case, the keys that identify each must be, obviously, different.

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

accessKeyId ✓ ✓ object AWS access key ID. For example, AKIAIOSFODNN7EXAMPLE.
secretAccessKey ✓ ✓ object AWS secret access key. For example, wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY.
SGBackup.status.sgBackupConfig.storage.s3.awsCredentials.secretKeySelectors.accessKeyId

↩ Parent

AWS access key ID. For example, AKIAIOSFODNN7EXAMPLE.

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

key ✓ ✓ string The key of the secret to select from. Must be a valid secret key.
name ✓ ✓ string Name of the referent. More information.
SGBackup.status.sgBackupConfig.storage.s3.awsCredentials.secretKeySelectors.secretAccessKey

↩ Parent

AWS secret access key. For example, wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY.

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

key ✓ ✓ string The key of the secret to select from. Must be a valid secret key.
name ✓ ✓ string Name of the referent. More information.
SGBackup.status.sgBackupConfig.storage.s3Compatible

↩ Parent

AWS S3-Compatible API configuration

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

awsCredentials ✓ ✓ object The credentials to access AWS S3 for writing and reading.
bucket ✓ ✓ string Bucket name.
enablePathStyleAddressing ✓ boolean Enable path-style addressing (i.e. http://s3.amazonaws.com/BUCKET/KEY) when connecting to an S3-compatible service that lacks support for sub-domain style bucket URLs (i.e. http://BUCKET.s3.amazonaws.com/KEY).

Defaults to false.

endpoint ✓ string Overrides the default url to connect to an S3-compatible service. For example: http://s3-like-service:9000.
region ✓ string The AWS S3 region. The Region may be detected using s3:GetBucketLocation, but if you wish to avoid giving permissions to this API call or forbid it from the applicable IAM policy, you must then specify this property.
storageClass ✓ string The Amazon S3 Storage Class to use for the backup object storage. By default, the STANDARD storage class is used. Other supported values include STANDARD_IA for Infrequent Access and REDUCED_REDUNDANCY.
SGBackup.status.sgBackupConfig.storage.s3Compatible.awsCredentials

↩ Parent

The credentials to access AWS S3 for writing and reading.

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

secretKeySelectors ✓ ✓ object Kubernetes SecretKeySelector(s) to reference the Secret(s) that contain the information about the awsCredentials. Note that you may use the same or different Secrets for the accessKeyId and the secretAccessKey. In the former case, the keys that identify each must be, obviously, different.
SGBackup.status.sgBackupConfig.storage.s3Compatible.awsCredentials.secretKeySelectors

↩ Parent

Kubernetes SecretKeySelector(s) to reference the Secret(s) that contain the information about the awsCredentials. Note that you may use the same or different Secrets for the accessKeyId and the secretAccessKey. In the former case, the keys that identify each must be, obviously, different.

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

accessKeyId ✓ ✓ object AWS access key ID. For example, AKIAIOSFODNN7EXAMPLE.
secretAccessKey ✓ ✓ object AWS secret access key. For example, wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY.
caCertificate ✓ object CA Certificate file to be used when connecting to the S3 Compatible Service.
SGBackup.status.sgBackupConfig.storage.s3Compatible.awsCredentials.secretKeySelectors.accessKeyId

↩ Parent

AWS access key ID. For example, AKIAIOSFODNN7EXAMPLE.

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

key ✓ ✓ string The key of the secret to select from. Must be a valid secret key.
name ✓ ✓ string Name of the referent. More information.
SGBackup.status.sgBackupConfig.storage.s3Compatible.awsCredentials.secretKeySelectors.secretAccessKey

↩ Parent

AWS secret access key. For example, wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY.

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

key ✓ ✓ string The key of the secret to select from. Must be a valid secret key.
name ✓ ✓ string Name of the referent. More information.
SGBackup.status.sgBackupConfig.storage.s3Compatible.awsCredentials.secretKeySelectors.caCertificate

↩ Parent

CA Certificate file to be used when connecting to the S3 Compatible Service.

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

key ✓ ✓ string The key of the secret to select from. Must be a valid secret key.
name ✓ ✓ string Name of the referent. More information.

SGBackup.status.sgBackupConfig.baseBackups

↩ Parent

Back backups configuration.

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

compression ✓ enum Select the backup compression algorithm. Possible options are: lz4, lzma, brotli. The default method is lz4. LZ4 is the fastest method, but compression ratio is the worst. LZMA is way slower, but it compresses backups about 6 times better than LZ4. Brotli is a good trade-off between speed and compression ratio, being about 3 times better than LZ4.

Enum: lz4, lzma, brotli

SGBackup.status.volumeSnapshot

↩ Parent

The volume snapshot configuration used to restore this backup.

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

backupLabel ✓ string The content of backup_label column returned by pg_backup_stop encoded in Base64
name ✓ string The volume snapshot used to store this backup.
tablespaceMap ✓ string The content of tablespace_map column returned by pg_backup_stop encoded in Base64