SGBackup

Kind: SGBackup

listKind: SGBackupList

plural: sgbackups

singular: sgbackup

shortNames sgbkp

The SGBackup custom resource represents a backup of the Postgres cluster. Backups are created automatically by a cron job configured using the settings in the backup configuration or manually by creating a SGBackup.

Example:

apiVersion: stackgres.io/v1
kind: SGBackup
metadata:
  name: backup
spec:
  sgCluster: stackgres
  managedLifecycle: true
status:
  internalName: base_00000002000000000000000E
  sgBackupConfig:
    compression: lz4
    storage:
      s3Compatible:
        awsCredentials:
          secretKeySelectors:
            accessKeyId:
              key: accesskey
              name: minio
            secretAccessKey:
              key: secretkey
              name: minio
        endpoint: http://minio:9000
        enablePathStyleAddressing: true
        bucket: stackgres
        region: k8s
      type: s3Compatible
  process:
    status: Completed
    jobPod: backup-backup-q79zq
    managedLifecycle: true
    timing:
      start: "2020-01-22T10:17:24.983902Z"
      stored: "2020-01-22T10:17:27.183Z"
      end: "2020-01-22T10:17:27.165204Z"
  backupInformation:
    hostname: stackgres-1
    systemIdentifier: "6784708504968245298"
    postgresVersion: "110006"
    pgData: /var/lib/postgresql/data
    size:
      compressed: 6691164
      uncompressed: 24037844
    lsn:
      start: "234881064"
      end: "234881272"
    startWalFile: 00000002000000000000000E

See also Backups section.

The SGBackup represents a manual or automatically generated backup of an SGCluster configured with an SGObjectStorage.

When a SGBackup is created a Job will perform a full backup of the database and update the status of the SGBackup with the all the information required to restore it and some stats (or a failure message in case something unexpected happened). After an SGBackup is created the same Job performs a reconciliation of the backups by applying the retention window that has been configured in the SGObjectStorage and removing the backups with managed lifecycle and the WAL files older than the ones that fit in the retention window. The reconciliation also removes backups (excluding WAL files) that do not belongs to any SGBackup. If the target storage of the SGObjectStorage is changed deletion of an SGBackup backups with managed lifecycle and the WAL files older than the ones that fit in the retention window and of backups that do not belongs to any SGBackup will not be performed anymore on the previous storage, only on the new target storage.

A manual or automatically generated backup of an SGCluster configured with backups.

When a SGBackup is created a Job will perform a full backup of the database and update the status of the SGBackup with the all the information required to restore it and some stats (or a failure message in case something unexpected happened). Backup generated by SGBackup are stored in the object storage configured with an SGObjectStorage together with the WAL files or in a VolumeSnapshot (separated from the WAL files that will be still stored in an object storage) depending on the backup configuration of the targeted SGCluster. After an SGBackup is created the same Job performs a reconciliation of the backups by applying the retention window that has been configured in the SGCluster and removing the backups with managed lifecycle and the WAL files older than the ones that fit in the retention window. The reconciliation also removes backups (excluding WAL files) that do not belongs to any SGBackup (including copies). If the target storage is changed deletion of an SGBackup backups with managed lifecycle and the WAL files older than the ones that fit in the retention window and of backups that do not belongs to any SGBackup will not be performed anymore on the previous storage, only on the new target storage. If the reconciliation of backups fails the backup itself do not fail and will be re-tried the next time a SGBackup or shecduled backup Job take place.

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

apiVersion ✓ string stackgres.io/v1
kind ✓ string SGBackup
metadata ✓ ✓ object Refer to the Kubernetes API documentation for the fields of the metadata field.
spec ✓ ✓ object
status ✓ object

SGBackup.spec

↩ Parent

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

managedLifecycle ✓ boolean Indicate if this backup is not permanent and should be removed by the automated retention policy. Default is false.
reconciliationTimeout ✓ integer Allow to set a timeout for the reconciliation process that take place after the backup.

If not set defaults to 300 (5 minutes). If set to 0 it will disable timeout.

Failure of reconciliation will not make the backup fail and will be re-tried the next time a SGBackup or shecduled backup Job take place.

Default: 300
sgCluster ✓ string The name of the SGCluster from which this backup is/will be taken.

If this is a copy of an existing completed backup in a different namespace the value must be prefixed with the namespace of the source backup and a dot . (e.g. <cluster namespace>.<cluster name>) or have the same value if the source backup is also a copy.

timeout ✓ integer Allow to set a timeout for the backup creation.

If not set it will be disabled and the backup operation will continue until the backup completes or fail. If set to 0 is the same as not being set.

Make sure to set a reasonable high value in order to allow for any unexpected delays during backup creation (network low bandwidth, disk low throughput and so forth).

SGBackup.status

↩ Parent

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

backupInformation ✓ object
backupPath ✓ string The path were the backup is stored.
internalName ✓ string The name of the backup.
process ✓ object
sgBackupConfig ✓ object The backup configuration used to perform this backup.
volumeSnapshot ✓ object The volume snapshot configuration used to restore this backup.

SGBackup.status.backupInformation

↩ Parent

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

controlData ✓ object An object containing data from the output of pg_controldata on the backup.
hostname ✓ string Hostname of the instance where the backup is taken from.
lsn ✓ object
pgData ✓ string Data directory where the backup is taken from.
postgresVersion ✓ string Postgres version of the server where the backup is taken from.
size ✓ object
sourcePod ✓ string Pod where the backup is taken from.
startWalFile ✓ string WAL segment file name when the backup was started.
systemIdentifier ✓ string Postgres system identifier of the cluster this backup is taken from.
timeline ✓ string Backup timeline.

SGBackup.status.backupInformation.controlData

↩ Parent

An object containing data from the output of pg_controldata on the backup.

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

Backup end location ✓ string
Backup start location ✓ string
Blocks per segment of large relation ✓ string
Bytes per WAL segment ✓ string
Catalog version number ✓ string
Data page checksum version ✓ string
Database block size ✓ string
Database cluster state ✓ string
Database system identifier ✓ string
Date/time type storage ✓ string
End-of-backup record required ✓ string
Fake LSN counter for unlogged rels ✓ string
Float4 argument passing ✓ string
Float8 argument passing ✓ string
Latest checkpoint location ✓ string
Latest checkpoint’s NextMultiOffset ✓ string
Latest checkpoint’s NextMultiXactId ✓ string
Latest checkpoint’s NextOID ✓ string
Latest checkpoint’s NextXID ✓ string
Latest checkpoint’s PrevTimeLineID ✓ string
Latest checkpoint’s REDO WAL file ✓ string
Latest checkpoint’s REDO location ✓ string
Latest checkpoint’s TimeLineID ✓ string
Latest checkpoint’s full_page_writes ✓ string
Latest checkpoint’s newestCommitTsXid ✓ string
Latest checkpoint’s oldestActiveXID ✓ string
Latest checkpoint’s oldestCommitTsXid ✓ string
Latest checkpoint’s oldestMulti’s DB ✓ string
Latest checkpoint’s oldestMultiXid ✓ string
Latest checkpoint’s oldestXID ✓ string
Latest checkpoint’s oldestXID’s DB ✓ string
Maximum columns in an index ✓ string
Maximum data alignment ✓ string
Maximum length of identifiers ✓ string
Maximum size of a TOAST chunk ✓ string
Min recovery ending loc’s timeline ✓ string
Minimum recovery ending location ✓ string
Mock authentication nonce ✓ string
Size of a large-object chunk ✓ string
Time of latest checkpoint ✓ string
WAL block size ✓ string
max_connections setting ✓ string
max_locks_per_xact setting ✓ string
max_prepared_xacts setting ✓ string
max_wal_senders setting ✓ string
max_worker_processes setting ✓ string
pg_control last modified ✓ string
pg_control version number ✓ string
track_commit_timestamp setting ✓ string
wal_level setting ✓ string
wal_log_hints setting ✓ string

SGBackup.status.backupInformation.lsn

↩ Parent

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

end ✓ string LSN of when the backup finished.
start ✓ string LSN of when the backup started.

SGBackup.status.backupInformation.size

↩ Parent

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

compressed ✓ integer Size (in bytes) of the compressed backup.

Format: int64
uncompressed ✓ integer Size (in bytes) of the uncompressed backup.

Format: int64

SGBackup.status.process

↩ Parent

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

failure ✓ string If the status is failed this field will contain a message indicating the failure reason.
jobPod ✓ string Name of the pod assigned to the backup. StackGres utilizes internally a locking mechanism based on the pod name of the job that creates the backup.
managedLifecycle ✓ boolean Status (may be transient) until converging to spec.managedLifecycle.
status ✓ string Status of the backup.
timing ✓ object

SGBackup.status.process.timing

↩ Parent

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

end ✓ string End time of backup.
start ✓ string Start time of backup.
stored ✓ string Time at which the backup is safely stored in the object storage.

SGBackup.status.sgBackupConfig

↩ Parent

The backup configuration used to perform this backup.

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

storage ✓ ✓ object Backup storage configuration.
baseBackups ✓ object Back backups configuration.
compression ✓ enum Select the backup compression algorithm. Possible options are: lz4, lzma, brotli. The default method is lz4. LZ4 is the fastest method, but compression ratio is the worst. LZMA is way slower, but it compresses backups about 6 times better than LZ4. Brotli is a good trade-off between speed and compression ratio, being about 3 times better than LZ4.

Enum: lz4, lzma, brotli

SGBackup.status.sgBackupConfig.storage

↩ Parent

Backup storage configuration.

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

type ✓ ✓ enum Specifies the type of object storage used for storing the base backups and WAL segments. Possible values:

  • s3: Amazon Web Services S3 (Simple Storage Service).
  • s3Compatible: non-AWS services that implement a compatibility API with AWS S3.
  • gcs: Google Cloud Storage.
  • azureBlob: Microsoft Azure Blob Storage.

    Enum: s3, s3Compatible, gcs, azureBlob
azureBlob ✓ object Azure Blob Storage configuration.
gcs ✓ object Google Cloud Storage configuration.
s3 ✓ object Amazon Web Services S3 configuration.
s3Compatible ✓ object AWS S3-Compatible API configuration
SGBackup.status.sgBackupConfig.storage.azureBlob

↩ Parent

Azure Blob Storage configuration.

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

azureCredentials ✓ ✓ object Credentials to access Azure Blob Storage for writing and reading.
bucket ✓ ✓ string Azure Blob Storage bucket name.
path ✓ string Optional path within the Azure Blobk bucket. Note that StackGres generates in any case a folder per StackGres cluster, using the SGCluster.metadata.name.
SGBackup.status.sgBackupConfig.storage.azureBlob.azureCredentials

↩ Parent

Credentials to access Azure Blob Storage for writing and reading.

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

secretKeySelectors ✓ object Kubernetes SecretKeySelectors to reference the Secrets that contain the information about the azureCredentials.
SGBackup.status.sgBackupConfig.storage.azureBlob.azureCredentials.secretKeySelectors

↩ Parent

Kubernetes SecretKeySelectors to reference the Secrets that contain the information about the azureCredentials.

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

accessKey ✓ ✓ object SecretKeySelector containing the primary or secondary access key for the storage account.
storageAccount ✓ ✓ object SecretKeySelector containing the name of the storage account.
SGBackup.status.sgBackupConfig.storage.azureBlob.azureCredentials.secretKeySelectors.accessKey

↩ Parent

SecretKeySelector containing the primary or secondary access key for the storage account.

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

key ✓ ✓ string The key of the secret to select from. Must be a valid secret key.
name ✓ ✓ string Name of the referent. More information.
SGBackup.status.sgBackupConfig.storage.azureBlob.azureCredentials.secretKeySelectors.storageAccount

↩ Parent

SecretKeySelector containing the name of the storage account.

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

key ✓ ✓ string The key of the secret to select from. Must be a valid secret key.
name ✓ ✓ string Name of the referent. More information.
SGBackup.status.sgBackupConfig.storage.gcs

↩ Parent

Google Cloud Storage configuration.

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

bucket ✓ ✓ string GCS bucket name.
gcpCredentials ✓ ✓ object Credentials to access GCS for writing and reading.
path ✓ string Optional path within the GCS bucket. Note that StackGres generates in any case a folder per StackGres cluster, using the SGCluster.metadata.name.
SGBackup.status.sgBackupConfig.storage.gcs.gcpCredentials

↩ Parent

Credentials to access GCS for writing and reading.

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

fetchCredentialsFromMetadataService ✓ boolean If true, the credentials will be fetched from the GCE/GKE metadata service and the credentials from secretKeySelectors field will not be used.

This is useful when running StackGres inside a GKE cluster using Workload Identity.

secretKeySelectors ✓ object A Kubernetes SecretKeySelector to reference the Secrets that contain the information about the Service Account to access GCS.
SGBackup.status.sgBackupConfig.storage.gcs.gcpCredentials.secretKeySelectors

↩ Parent

A Kubernetes SecretKeySelector to reference the Secrets that contain the information about the Service Account to access GCS.

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

serviceAccountJSON ✓ ✓ object A service account key from GCP. In JSON format, as downloaded from the GCP Console.
SGBackup.status.sgBackupConfig.storage.gcs.gcpCredentials.secretKeySelectors.serviceAccountJSON

↩ Parent

A service account key from GCP. In JSON format, as downloaded from the GCP Console.

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

key ✓ ✓ string The key of the secret to select from. Must be a valid secret key.
name ✓ ✓ string Name of the referent. More information.
SGBackup.status.sgBackupConfig.storage.s3

↩ Parent

Amazon Web Services S3 configuration.

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

awsCredentials ✓ ✓ object Credentials to access AWS S3 for writing and reading.
bucket ✓ ✓ string AWS S3 bucket name.
path ✓ string Optional path within the S3 bucket. Note that StackGres generates in any case a folder per StackGres cluster, using the SGCluster.metadata.name.
region ✓ string AWS S3 region. The Region may be detected using s3:GetBucketLocation, but to avoid giving permissions to this API call or forbid it from the applicable IAM policy, this property must be explicitely specified.
storageClass ✓ string Amazon S3 Storage Class used for the backup object storage. By default, the STANDARD storage class is used. Other supported values include STANDARD_IA for Infrequent Access and REDUCED_REDUNDANCY.
SGBackup.status.sgBackupConfig.storage.s3.awsCredentials

↩ Parent

Credentials to access AWS S3 for writing and reading.

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

secretKeySelectors ✓ ✓ object Kubernetes SecretKeySelectors to reference the Secrets that contain the information about the awsCredentials.
SGBackup.status.sgBackupConfig.storage.s3.awsCredentials.secretKeySelectors

↩ Parent

Kubernetes SecretKeySelectors to reference the Secrets that contain the information about the awsCredentials.

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

accessKeyId ✓ ✓ object SecretKeySelector containing the AWS Access Key ID secret.
secretAccessKey ✓ ✓ object SecretKeySelector containing the AWS Secret Access Key secret.
SGBackup.status.sgBackupConfig.storage.s3.awsCredentials.secretKeySelectors.accessKeyId

↩ Parent

SecretKeySelector containing the AWS Access Key ID secret.

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

key ✓ ✓ string The key of the secret to select from. Must be a valid secret key.
name ✓ ✓ string Name of the referent. More information.
SGBackup.status.sgBackupConfig.storage.s3.awsCredentials.secretKeySelectors.secretAccessKey

↩ Parent

SecretKeySelector containing the AWS Secret Access Key secret.

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

key ✓ ✓ string The key of the secret to select from. Must be a valid secret key.
name ✓ ✓ string Name of the referent. More information.
SGBackup.status.sgBackupConfig.storage.s3Compatible

↩ Parent

AWS S3-Compatible API configuration

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

awsCredentials ✓ ✓ object Credentials to access AWS S3 for writing and reading.
bucket ✓ ✓ string Bucket name.
enablePathStyleAddressing ✓ boolean Enable path-style addressing (i.e. http://s3.amazonaws.com/BUCKET/KEY) when connecting to an S3-compatible service that lacks support for sub-domain style bucket URLs (i.e. http://BUCKET.s3.amazonaws.com/KEY). Defaults to false.
endpoint ✓ string Overrides the default url to connect to an S3-compatible service. For example: http://s3-like-service:9000.
path ✓ string Optional path within the S3 bucket. Note that StackGres generates in any case a folder per StackGres cluster, using the SGCluster.metadata.name.
region ✓ string AWS S3 region. The Region may be detected using s3:GetBucketLocation, but to avoid giving permissions to this API call or forbid it from the applicable IAM policy, this property must be explicitely specified.
storageClass ✓ string Amazon S3 Storage Class used for the backup object storage. By default, the STANDARD storage class is used. Other supported values include STANDARD_IA for Infrequent Access and REDUCED_REDUNDANCY.
SGBackup.status.sgBackupConfig.storage.s3Compatible.awsCredentials

↩ Parent

Credentials to access AWS S3 for writing and reading.

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

secretKeySelectors ✓ ✓ object A Kubernetes SecretKeySelector to reference the Secrets that contain the information about the awsCredentials.
SGBackup.status.sgBackupConfig.storage.s3Compatible.awsCredentials.secretKeySelectors

↩ Parent

A Kubernetes SecretKeySelector to reference the Secrets that contain the information about the awsCredentials.

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

accessKeyId ✓ ✓ object SecretKeySelector containing the AWS Access Key ID secret.
secretAccessKey ✓ ✓ object SecretKeySelector containing the AWS Secret Access Key secret.
SGBackup.status.sgBackupConfig.storage.s3Compatible.awsCredentials.secretKeySelectors.accessKeyId

↩ Parent

SecretKeySelector containing the AWS Access Key ID secret.

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

key ✓ ✓ string The key of the secret to select from. Must be a valid secret key.
name ✓ ✓ string Name of the referent. More information.
SGBackup.status.sgBackupConfig.storage.s3Compatible.awsCredentials.secretKeySelectors.secretAccessKey

↩ Parent

SecretKeySelector containing the AWS Secret Access Key secret.

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

key ✓ ✓ string The key of the secret to select from. Must be a valid secret key.
name ✓ ✓ string Name of the referent. More information.

SGBackup.status.sgBackupConfig.baseBackups

↩ Parent

Back backups configuration.

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

compression ✓ enum Select the backup compression algorithm. Possible options are: lz4, lzma, brotli. The default method is lz4. LZ4 is the fastest method, but compression ratio is the worst. LZMA is way slower, but it compresses backups about 6 times better than LZ4. Brotli is a good trade-off between speed and compression ratio, being about 3 times better than LZ4.

Enum: lz4, lzma, brotli
cronSchedule ✓ string Continuous Archiving backups are composed of periodic base backups and all the WAL segments produced in between those base backups. This parameter specifies at what time and with what frequency to start performing a new base backup.

Use cron syntax (m h dom mon dow) for this parameter, i.e., 5 values separated by spaces:

  • m: minute, 0 to 59
  • h: hour, 0 to 23
  • dom: day of month, 1 to 31 (recommended not to set it higher than 28)
  • mon: month, 1 to 12
  • dow: day of week, 0 to 7 (0 and 7 both represent Sunday)

Also ranges of values (start-end), the symbol * (meaning first-last) or even */N, where N is a number, meaning every N, may be used. All times are UTC. It is recommended to avoid 00:00 as base backup time, to avoid overlapping with any other external operations happening at this time.

performance ✓ object
retention ✓ integer Based on this parameter, an automatic retention policy is defined to delete old base backups. This parameter specifies the number of base backups to keep, in a sliding window. Consequently, the time range covered by backups is periodicity*retention, where periodicity is the separation between backups as specified by the cronSchedule property.

Default is 5.

Minimum: 1
SGBackup.status.sgBackupConfig.baseBackups.performance

↩ Parent

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

maxDiskBandwidth ✓ integer Maximum disk read I/O when performing a backup. In bytes (per second).
maxDiskBandwitdh ✓ integer Deprecated: use instead maxDiskBandwidth.

Maximum disk read I/O when performing a backup. In bytes (per second).

maxNetworkBandwidth ✓ integer Maximum storage upload bandwidth to be used when storing the backup. In bytes (per second).
maxNetworkBandwitdh ✓ integer Deprecated: use instead maxNetworkBandwidth.

Maximum storage upload bandwidth to be used when storing the backup. In bytes (per second).

uploadConcurrency ✓ integer Backup storage may use several concurrent streams to store the data. This parameter configures the number of parallel streams to use. By default, it’s set to 1 (use one stream).

Minimum: 1
uploadDiskConcurrency ✓ integer Backup storage may use several concurrent streams to store the data. This parameter configures the number of parallel streams to use to reading from disk. By default, it’s set to 1 (use one stream).

Minimum: 1

SGBackup.status.volumeSnapshot

↩ Parent

The volume snapshot configuration used to restore this backup.

Property
Required
Updatable
May Require Restart
Type
 Description

Workaround for hugo bug not rendering first table row

backupLabel ✓ string The content of backup_label column returned by pg_backup_stop encoded in Base64
name ✓ string The volume snapshot used to store this backup.
tablespaceMap ✓ string The content of tablespace_map column returned by pg_backup_stop encoded in Base64