S3

AWS S3 Setup

This section will illustrate how to configure backups on StackGres using AWS S3. To do so, you will need to have the AWS CLI installed to create the right permissions and the bucket on AWS S3.

Create the right permissions and the user with following characteristics (that you may change):

  • Zone: us-west-2
  • Bucket name: backup-demo-of-stackgres-io
  • IAM username: stackgres-demo-k8s-sa-user
  • Secret Credentials: eks-backup-bucket-secret
aws iam create-user --region us-west-2 --user-name stackgres-demo-k8s-sa-user
aws iam put-user-policy --region us-west-2 --user-name stackgres-demo-k8s-sa-user --policy-name stackgres-demo-k8s-user-policy --policy-document '{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Action":["s3:ListBucket","s3:GetBucketLocation"],"Resource":["arn:aws:s3:::backup-demo-of-stackgres-io"]},{"Effect":"Allow","Action":["s3:PutObject","s3:GetObject","s3:DeleteObject"],"Resource":["arn:aws:s3:::backup-demo-of-stackgres-io/*"]}]}'

Now we need to create the access key to be used on backup creation. As output a file access_key.json will be generated:

aws --output json iam create-access-key --region us-west-2 --user-name stackgres-demo-k8s-sa-user | tee access_keys.json

Finally to create the bucket:

aws s3 mb s3://backup-demo-of-stackgres-io --region us-west-2

Kubernetes Setup

To proceed, a Kubernetes Secret with the folling shape needs to be created:

kubectl create secret generic eks-backup-bucket-secret --from-literal="accessKeyId=<YOUR_ACCESS_KEY_HERE>"   --from-literal="secretAccessKey=<YOUR_SECRET_KEY_HERE>"

secret/sg-demo-jira-arm-secret created

Having the credentials secret created, we just need to create the object storage configuration and set the backup configuration. The object storage configuration it is governed by the CRD SGObjectStorage. This CRD allows to specify the object storage technology and parameters required and a reference to the above secret.

Create the file sgobjectstorage-backupconfig1.yaml:

apiVersion: stackgres.io/v1beta1
kind: SGObjectStorage
metadata:
  name: backup-config-stackgres-demo
spec:
  type: s3
  s3:
    bucket: backup-demo-of-stackgres-io
    awsCredentials:
      secretKeySelectors:
        accessKeyId:
          name: eks-backup-bucket-secret
          key: accessKeyId
        secretAccessKey:
          name: eks-backup-bucket-secret
          key: secretAccessKey

and deploy to Kubernetes:

kubectl apply -f sgobjectstorage-backupconfig1.yaml

The backup configuration can be set unser the section .spec.configurations.backups of the CRD SGCluster, among others, the retention window for the automated backups, when base backups are performed and performance parameters of the backup process.

apiVersion: stackgres.io/v1
kind: SGCluster
spec:
  configurations:
    backups:
    - sgObjectStorage: backupconfig1
      cronSchedule: '*/5 * * * *'
      retention: 6

Note that for this tutorial and demo purposes, backups are created every 5 minutes. Modify the .spec.backups[0].cronSchedule parameter above to adjust to your own needs.

The above configuration will be applied when creating the SGCluster resource.