Kind: SGBackup
listKind: SGBackupList
plural: sgbackups
singular: sgbackup
shortNames sgbkp
The SGBackup
custom resource represents a backup of the Postgres cluster.
Backups are created automatically by a cron job configured using the settings in the backup configuration or manually by creating a SGBackup
.
Example:
apiVersion: stackgres.io/v1
kind: SGBackup
metadata:
name: backup
spec:
sgCluster: stackgres
managedLifecycle: true
status:
internalName: base_00000002000000000000000E
sgBackupConfig:
compression: lz4
storage:
s3Compatible:
awsCredentials:
secretKeySelectors:
accessKeyId:
key: accesskey
name: minio
secretAccessKey:
key: secretkey
name: minio
endpoint: http://minio:9000
enablePathStyleAddressing: true
bucket: stackgres
region: k8s
type: s3Compatible
process:
status: Completed
jobPod: backup-backup-q79zq
managedLifecycle: true
timing:
start: "2020-01-22T10:17:24.983902Z"
stored: "2020-01-22T10:17:27.183Z"
end: "2020-01-22T10:17:27.165204Z"
backupInformation:
hostname: stackgres-1
systemIdentifier: "6784708504968245298"
postgresVersion: "110006"
pgData: /var/lib/postgresql/data
size:
compressed: 6691164
uncompressed: 24037844
lsn:
start: "234881064"
end: "234881272"
startWalFile: 00000002000000000000000E
See also Backups section.
The SGBackup represents a manual or automatically generated backup of an SGCluster configured with an SGObjectStorage.
When a SGBackup is created a Job will perform a full backup of the database and update the status of the SGBackup with the all the information required to restore it and some stats (or a failure message in case something unexpected happened). After an SGBackup is created the same Job performs a reconciliation of the backups by applying the retention window that has been configured in the SGObjectStorage and removing the backups with managed lifecycle and the WAL files older than the ones that fit in the retention window. The reconciliation also removes backups (excluding WAL files) that do not belongs to any SGBackup. If the target storage of the SGObjectStorage is changed deletion of an SGBackup backups with managed lifecycle and the WAL files older than the ones that fit in the retention window and of backups that do not belongs to any SGBackup will not be performed anymore on the previous storage, only on the new target storage.
A manual or automatically generated backup of an SGCluster configured with backups.
When a SGBackup is created a Job will perform a full backup of the database and update the status of the SGBackup with the all the information required to restore it and some stats (or a failure message in case something unexpected happened). Backup generated by SGBackup are stored in the object storage configured with an SGObjectStorage together with the WAL files or in a VolumeSnapshot (separated from the WAL files that will be still stored in an object storage) depending on the backup configuration of the targeted SGCluster. After an SGBackup is created the same Job performs a reconciliation of the backups by applying the retention window that has been configured in the SGCluster and removing the backups with managed lifecycle and the WAL files older than the ones that fit in the retention window. The reconciliation also removes backups (excluding WAL files) that do not belongs to any SGBackup (including copies). If the target storage is changed deletion of an SGBackup backups with managed lifecycle and the WAL files older than the ones that fit in the retention window and of backups that do not belongs to any SGBackup will not be performed anymore on the previous storage, only on the new target storage. If the reconciliation of backups fails the backup itself do not fail and will be re-tried the next time a SGBackup or shecduled backup Job take place.
Property |
Required |
Updatable |
May Require Restart |
Type |
Description |
---|---|---|---|---|---|
apiVersion | ✓ | string | stackgres.io/v1 | ||
kind | ✓ | string | SGBackup | ||
metadata | ✓ | ✓ | object | Refer to the Kubernetes API documentation for the fields of the metadata field. |
|
spec | ✓ | ✓ | object |
|
|
status | ✓ | object |
|
Property |
Required |
Updatable |
May Require Restart |
Type |
Description |
---|---|---|---|---|---|
managedLifecycle | ✓ | boolean |
Indicate if this backup is not permanent and should be removed by the automated
retention policy. Default is false .
|
||
maxRetries | ✓ | integer |
The maximum number of retries the backup operation is allowed to do after a failure.
A value of |
||
reconciliationTimeout | ✓ | integer |
Allow to set a timeout for the reconciliation process that take place after the backup.
If not set defaults to 300 (5 minutes). If set to 0 it will disable timeout. Failure of reconciliation will not make the backup fail and will be re-tried the next time a SGBackup
or shecduled backup Job take place.
|
||
sgCluster | ✓ | string |
The name of the SGCluster from which this backup is/will be taken.
If this is a copy of an existing completed backup in a different namespace
the value must be prefixed with the namespace of the source backup and a
dot |
||
timeout | ✓ | integer |
Allow to set a timeout for the backup creation.
If not set it will be disabled and the backup operation will continue until the backup completes or fail. If set to 0 is the same as not being set. Make sure to set a reasonable high value in order to allow for any unexpected delays during backup creation (network low bandwidth, disk low throughput and so forth).
|
Property |
Required |
Updatable |
May Require Restart |
Type |
Description |
---|---|---|---|---|---|
backupInformation | ✓ | object |
|
||
backupPath | ✓ | string |
The path were the backup is stored.
|
||
internalName | ✓ | string |
The name of the backup.
|
||
process | ✓ | object |
|
||
sgBackupConfig | ✓ | object |
The backup configuration used to perform this backup. |
||
volumeSnapshot | ✓ | object |
The volume snapshot configuration used to restore this backup. |
Property |
Required |
Updatable |
May Require Restart |
Type |
Description |
---|---|---|---|---|---|
controlData | ✓ | object |
An object containing data from the output of pg_controldata on the backup.
|
||
hostname | ✓ | string |
Hostname of the instance where the backup is taken from.
|
||
lsn | ✓ | object |
|
||
pgData | ✓ | string |
Data directory where the backup is taken from.
|
||
postgresVersion | ✓ | string |
Postgres version of the server where the backup is taken from.
|
||
size | ✓ | object |
|
||
sourcePod | ✓ | string |
Pod where the backup is taken from.
|
||
startWalFile | ✓ | string |
WAL segment file name when the backup was started.
|
||
systemIdentifier | ✓ | string |
Postgres system identifier of the cluster this backup is taken from.
|
||
timeline | ✓ | string |
Backup timeline.
|
An object containing data from the output of pg_controldata on the backup.
Property |
Required |
Updatable |
May Require Restart |
Type |
Description |
---|---|---|---|---|---|
Backup end location | ✓ | string |
|
||
Backup start location | ✓ | string |
|
||
Blocks per segment of large relation | ✓ | string |
|
||
Bytes per WAL segment | ✓ | string |
|
||
Catalog version number | ✓ | string |
|
||
Data page checksum version | ✓ | string |
|
||
Database block size | ✓ | string |
|
||
Database cluster state | ✓ | string |
|
||
Database system identifier | ✓ | string |
|
||
Date/time type storage | ✓ | string |
|
||
End-of-backup record required | ✓ | string |
|
||
Fake LSN counter for unlogged rels | ✓ | string |
|
||
Float4 argument passing | ✓ | string |
|
||
Float8 argument passing | ✓ | string |
|
||
Latest checkpoint location | ✓ | string |
|
||
Latest checkpoint’s NextMultiOffset | ✓ | string |
|
||
Latest checkpoint’s NextMultiXactId | ✓ | string |
|
||
Latest checkpoint’s NextOID | ✓ | string |
|
||
Latest checkpoint’s NextXID | ✓ | string |
|
||
Latest checkpoint’s PrevTimeLineID | ✓ | string |
|
||
Latest checkpoint’s REDO WAL file | ✓ | string |
|
||
Latest checkpoint’s REDO location | ✓ | string |
|
||
Latest checkpoint’s TimeLineID | ✓ | string |
|
||
Latest checkpoint’s full_page_writes | ✓ | string |
|
||
Latest checkpoint’s newestCommitTsXid | ✓ | string |
|
||
Latest checkpoint’s oldestActiveXID | ✓ | string |
|
||
Latest checkpoint’s oldestCommitTsXid | ✓ | string |
|
||
Latest checkpoint’s oldestMulti’s DB | ✓ | string |
|
||
Latest checkpoint’s oldestMultiXid | ✓ | string |
|
||
Latest checkpoint’s oldestXID | ✓ | string |
|
||
Latest checkpoint’s oldestXID’s DB | ✓ | string |
|
||
Maximum columns in an index | ✓ | string |
|
||
Maximum data alignment | ✓ | string |
|
||
Maximum length of identifiers | ✓ | string |
|
||
Maximum size of a TOAST chunk | ✓ | string |
|
||
Min recovery ending loc’s timeline | ✓ | string |
|
||
Minimum recovery ending location | ✓ | string |
|
||
Mock authentication nonce | ✓ | string |
|
||
Size of a large-object chunk | ✓ | string |
|
||
Time of latest checkpoint | ✓ | string |
|
||
WAL block size | ✓ | string |
|
||
max_connections setting | ✓ | string |
|
||
max_locks_per_xact setting | ✓ | string |
|
||
max_prepared_xacts setting | ✓ | string |
|
||
max_wal_senders setting | ✓ | string |
|
||
max_worker_processes setting | ✓ | string |
|
||
pg_control last modified | ✓ | string |
|
||
pg_control version number | ✓ | string |
|
||
track_commit_timestamp setting | ✓ | string |
|
||
wal_level setting | ✓ | string |
|
||
wal_log_hints setting | ✓ | string |
|
Property |
Required |
Updatable |
May Require Restart |
Type |
Description |
---|---|---|---|---|---|
end | ✓ | string |
LSN of when the backup finished.
|
||
start | ✓ | string |
LSN of when the backup started.
|
Property |
Required |
Updatable |
May Require Restart |
Type |
Description |
---|---|---|---|---|---|
compressed | ✓ | integer |
Size (in bytes) of the compressed backup.
Format: int64 |
||
uncompressed | ✓ | integer |
Size (in bytes) of the uncompressed backup.
Format: int64 |
Property |
Required |
Updatable |
May Require Restart |
Type |
Description |
---|---|---|---|---|---|
failure | ✓ | string |
If the status is failed this field will contain a message indicating the failure reason.
|
||
jobPod | ✓ | string |
Name of the pod assigned to the backup. StackGres utilizes internally a locking mechanism based on the pod name of the job that creates the backup.
|
||
managedLifecycle | ✓ | boolean |
Status (may be transient) until converging to spec.managedLifecycle .
|
||
status | ✓ | string |
Status of the backup.
|
||
timing | ✓ | object |
|
Property |
Required |
Updatable |
May Require Restart |
Type |
Description |
---|---|---|---|---|---|
end | ✓ | string |
End time of backup.
|
||
start | ✓ | string |
Start time of backup.
|
||
stored | ✓ | string |
Time at which the backup is safely stored in the object storage.
|
The backup configuration used to perform this backup.
Property |
Required |
Updatable |
May Require Restart |
Type |
Description |
---|---|---|---|---|---|
storage | ✓ | ✓ | object |
Backup storage configuration.
|
|
baseBackups | ✓ | object |
Back backups configuration.
|
||
compression | ✓ | enum |
Select the backup compression algorithm. Possible options are: lz4, lzma, brotli. The default method is lz4 . LZ4 is the fastest method, but compression ratio is the worst. LZMA is way slower, but it compresses backups about 6 times better than LZ4. Brotli is a good trade-off between speed and compression ratio, being about 3 times better than LZ4.
Enum: lz4, lzma, brotli |
Backup storage configuration.
Property |
Required |
Updatable |
May Require Restart |
Type |
Description |
---|---|---|---|---|---|
type | ✓ | ✓ | enum |
Specifies the type of object storage used for storing the base backups and WAL segments.
Possible values:
|
|
azureBlob | ✓ | object |
Azure Blob Storage configuration.
|
||
gcs | ✓ | object |
Google Cloud Storage configuration.
|
||
s3 | ✓ | object |
Amazon Web Services S3 configuration.
|
||
s3Compatible | ✓ | object |
AWS S3-Compatible API configuration |
Azure Blob Storage configuration.
Property |
Required |
Updatable |
May Require Restart |
Type |
Description |
---|---|---|---|---|---|
azureCredentials | ✓ | ✓ | object |
Credentials to access Azure Blob Storage for writing and reading.
|
|
bucket | ✓ | ✓ | string |
Azure Blob Storage bucket name.
|
|
path | ✓ | string |
Optional path within the Azure Blobk bucket. Note that StackGres generates in any case a folder per StackGres cluster, using the SGCluster.metadata.name .
|
Credentials to access Azure Blob Storage for writing and reading.
Property |
Required |
Updatable |
May Require Restart |
Type |
Description |
---|---|---|---|---|---|
secretKeySelectors | ✓ | object |
Kubernetes SecretKeySelectors to reference the Secrets that contain the information about the azureCredentials .
|
Kubernetes SecretKeySelectors to reference the Secrets that contain the information about the azureCredentials
.
Property |
Required |
Updatable |
May Require Restart |
Type |
Description |
---|---|---|---|---|---|
accessKey | ✓ | ✓ | object |
SecretKeySelector containing the primary or secondary access key for the storage account.
|
|
storageAccount | ✓ | ✓ | object |
SecretKeySelector containing the name of the storage account.
|
SecretKeySelector containing the primary or secondary access key for the storage account.
Property |
Required |
Updatable |
May Require Restart |
Type |
Description |
---|---|---|---|---|---|
key | ✓ | ✓ | string |
The key of the secret to select from. Must be a valid secret key.
|
|
name | ✓ | ✓ | string |
Name of the referent. More information.
|
SecretKeySelector containing the name of the storage account.
Property |
Required |
Updatable |
May Require Restart |
Type |
Description |
---|---|---|---|---|---|
key | ✓ | ✓ | string |
The key of the secret to select from. Must be a valid secret key.
|
|
name | ✓ | ✓ | string |
Name of the referent. More information.
|
Google Cloud Storage configuration.
Property |
Required |
Updatable |
May Require Restart |
Type |
Description |
---|---|---|---|---|---|
bucket | ✓ | ✓ | string |
GCS bucket name.
|
|
gcpCredentials | ✓ | ✓ | object |
Credentials to access GCS for writing and reading.
|
|
path | ✓ | string |
Optional path within the GCS bucket. Note that StackGres generates in any case a folder per StackGres cluster, using the SGCluster.metadata.name .
|
Credentials to access GCS for writing and reading.
Property |
Required |
Updatable |
May Require Restart |
Type |
Description |
---|---|---|---|---|---|
fetchCredentialsFromMetadataService | ✓ | boolean |
If true, the credentials will be fetched from the GCE/GKE metadata service and the credentials from secretKeySelectors field will not be used.
This is useful when running StackGres inside a GKE cluster using Workload Identity.
|
||
secretKeySelectors | ✓ | object |
A Kubernetes SecretKeySelector to reference the Secrets that contain the information about the Service Account to access GCS.
|
A Kubernetes SecretKeySelector to reference the Secrets that contain the information about the Service Account to access GCS.
Property |
Required |
Updatable |
May Require Restart |
Type |
Description |
---|---|---|---|---|---|
serviceAccountJSON | ✓ | ✓ | object |
A service account key from GCP. In JSON format, as downloaded from the GCP Console.
|
A service account key from GCP. In JSON format, as downloaded from the GCP Console.
Property |
Required |
Updatable |
May Require Restart |
Type |
Description |
---|---|---|---|---|---|
key | ✓ | ✓ | string |
The key of the secret to select from. Must be a valid secret key.
|
|
name | ✓ | ✓ | string |
Name of the referent. More information.
|
Amazon Web Services S3 configuration.
Property |
Required |
Updatable |
May Require Restart |
Type |
Description |
---|---|---|---|---|---|
awsCredentials | ✓ | ✓ | object |
Credentials to access AWS S3 for writing and reading.
|
|
bucket | ✓ | ✓ | string |
AWS S3 bucket name.
|
|
path | ✓ | string |
Optional path within the S3 bucket. Note that StackGres generates in any case a folder per
StackGres cluster, using the SGCluster.metadata.name .
|
||
region | ✓ | string |
AWS S3 region. The Region may be detected using s3:GetBucketLocation, but to avoid giving permissions to this API call or forbid it from the applicable IAM policy, this property must be explicitely specified.
|
||
storageClass | ✓ | string |
Amazon S3 Storage Class used for the backup object storage. By default, the STANDARD storage class is used. Other supported values include STANDARD_IA for Infrequent Access and REDUCED_REDUNDANCY .
|
Credentials to access AWS S3 for writing and reading.
Property |
Required |
Updatable |
May Require Restart |
Type |
Description |
---|---|---|---|---|---|
secretKeySelectors | ✓ | ✓ | object |
Kubernetes SecretKeySelectors to reference the Secrets that contain the information about the awsCredentials .
|
Kubernetes SecretKeySelectors to reference the Secrets that contain the information about the awsCredentials
.
Property |
Required |
Updatable |
May Require Restart |
Type |
Description |
---|---|---|---|---|---|
accessKeyId | ✓ | ✓ | object |
SecretKeySelector containing the AWS Access Key ID secret.
|
|
secretAccessKey | ✓ | ✓ | object |
SecretKeySelector containing the AWS Secret Access Key secret.
|
SecretKeySelector containing the AWS Access Key ID secret.
Property |
Required |
Updatable |
May Require Restart |
Type |
Description |
---|---|---|---|---|---|
key | ✓ | ✓ | string |
The key of the secret to select from. Must be a valid secret key.
|
|
name | ✓ | ✓ | string |
Name of the referent. More information.
|
SecretKeySelector containing the AWS Secret Access Key secret.
Property |
Required |
Updatable |
May Require Restart |
Type |
Description |
---|---|---|---|---|---|
key | ✓ | ✓ | string |
The key of the secret to select from. Must be a valid secret key.
|
|
name | ✓ | ✓ | string |
Name of the referent. More information.
|
AWS S3-Compatible API configuration
Property |
Required |
Updatable |
May Require Restart |
Type |
Description |
---|---|---|---|---|---|
awsCredentials | ✓ | ✓ | object |
Credentials to access AWS S3 for writing and reading.
|
|
bucket | ✓ | ✓ | string |
Bucket name.
|
|
enablePathStyleAddressing | ✓ | boolean |
Enable path-style addressing (i.e. http://s3.amazonaws.com/BUCKET/KEY ) when connecting to an S3-compatible service that lacks support for sub-domain style bucket URLs (i.e. http://BUCKET.s3.amazonaws.com/KEY ). Defaults to false.
|
||
endpoint | ✓ | string |
Overrides the default url to connect to an S3-compatible service.
For example: http://s3-like-service:9000 .
|
||
path | ✓ | string |
Optional path within the S3 bucket. Note that StackGres generates in any case a folder per StackGres cluster, using the SGCluster.metadata.name .
|
||
region | ✓ | string |
AWS S3 region. The Region may be detected using s3:GetBucketLocation, but to avoid giving permissions to this API call or forbid it from the applicable IAM policy, this property must be explicitely specified.
|
||
storageClass | ✓ | string |
Amazon S3 Storage Class used for the backup object storage. By default, the STANDARD storage class is used. Other supported values include STANDARD_IA for Infrequent Access and REDUCED_REDUNDANCY .
|
Credentials to access AWS S3 for writing and reading.
Property |
Required |
Updatable |
May Require Restart |
Type |
Description |
---|---|---|---|---|---|
secretKeySelectors | ✓ | ✓ | object |
A Kubernetes SecretKeySelector to reference the Secrets that contain the information about the awsCredentials .
|
A Kubernetes SecretKeySelector to reference the Secrets that contain the information about the awsCredentials
.
Property |
Required |
Updatable |
May Require Restart |
Type |
Description |
---|---|---|---|---|---|
accessKeyId | ✓ | ✓ | object |
SecretKeySelector containing the AWS Access Key ID secret.
|
|
secretAccessKey | ✓ | ✓ | object |
SecretKeySelector containing the AWS Secret Access Key secret.
|
SecretKeySelector containing the AWS Access Key ID secret.
Property |
Required |
Updatable |
May Require Restart |
Type |
Description |
---|---|---|---|---|---|
key | ✓ | ✓ | string |
The key of the secret to select from. Must be a valid secret key.
|
|
name | ✓ | ✓ | string |
Name of the referent. More information.
|
SecretKeySelector containing the AWS Secret Access Key secret.
Property |
Required |
Updatable |
May Require Restart |
Type |
Description |
---|---|---|---|---|---|
key | ✓ | ✓ | string |
The key of the secret to select from. Must be a valid secret key.
|
|
name | ✓ | ✓ | string |
Name of the referent. More information.
|
Back backups configuration.
Property |
Required |
Updatable |
May Require Restart |
Type |
Description |
---|---|---|---|---|---|
compression | ✓ | enum |
Select the backup compression algorithm. Possible options are: lz4, lzma, brotli. The default method is lz4 . LZ4 is the fastest method, but compression ratio is the worst. LZMA is way slower, but it compresses backups about 6 times better than LZ4. Brotli is a good trade-off between speed and compression ratio, being about 3 times better than LZ4.
Enum: lz4, lzma, brotli |
||
cronSchedule | ✓ | string |
Continuous Archiving backups are composed of periodic base backups and all the WAL segments produced in between those base backups. This parameter specifies at what time and with what frequency to start performing a new base backup.
Use cron syntax (
Also ranges of values ( |
||
performance | ✓ | object |
|
||
retention | ✓ | integer |
Based on this parameter, an automatic retention policy is defined to delete old base backups.
This parameter specifies the number of base backups to keep, in a sliding window.
Consequently, the time range covered by backups is periodicity*retention , where periodicity is the separation between backups as specified by the cronSchedule property.
Default is 5.
|
Property |
Required |
Updatable |
May Require Restart |
Type |
Description |
---|---|---|---|---|---|
maxDiskBandwidth | ✓ | integer |
Maximum disk read I/O when performing a backup. In bytes (per second).
|
||
maxDiskBandwitdh | ✓ | integer |
Deprecated: use instead maxDiskBandwidth.
Maximum disk read I/O when performing a backup. In bytes (per second).
|
||
maxNetworkBandwidth | ✓ | integer |
Maximum storage upload bandwidth to be used when storing the backup. In bytes (per second).
|
||
maxNetworkBandwitdh | ✓ | integer |
Deprecated: use instead maxNetworkBandwidth.
Maximum storage upload bandwidth to be used when storing the backup. In bytes (per second).
|
||
uploadConcurrency | ✓ | integer |
Backup storage may use several concurrent streams to store the data. This parameter configures the number of parallel streams to use. By default, it’s set to 1 (use one stream).
Minimum: 1 |
||
uploadDiskConcurrency | ✓ | integer |
Backup storage may use several concurrent streams to store the data. This parameter configures the number of parallel streams to use to reading from disk. By default, it’s set to 1 (use one stream).
Minimum: 1 |
The volume snapshot configuration used to restore this backup.
Property |
Required |
Updatable |
May Require Restart |
Type |
Description |
---|---|---|---|---|---|
backupLabel | ✓ | string |
The content of backup_label column returned by pg_backup_stop encoded in Base64
|
||
name | ✓ | string |
The volume snapshot used to store this backup.
|
||
tablespaceMap | ✓ | string |
The content of tablespace_map column returned by pg_backup_stop encoded in Base64
|