SGDistributedLogs

Creating a distributed logs cluster

The distributed logs CR represent a distributed logs cluster. When a cluster is configured to use a distributed logs cluster it will forward all logs from different sources to the distributed logs cluster. Under the hood, distributed log cluster use a SGCluster, therefore the distributed log cluster could be receive SQL queries in a postgres-util fashion but not with it.

For more information about distributed log usage please review the Distributed Log Cluster Administration Guide


Kind: SGDistributedLogs

listKind: SGDistributedLogsList

plural: sgdistributedlogs

singular: sgdistributedlogs


Spec

Property Required Updatable Type Default Description
persistentVolume string
Pod’s persistent volume configuration
postgresServices object
Kubernetes services created or managed by StackGres.
scheduling object
Pod custom scheduling configuration.
sgInstanceProfile string generated
Name of the SGInstanceProfile. A SGInstanceProfile defines CPU and memory limits. Must exist before creating a distributed logs. When no profile is set, a default (currently: 1 core, 2 GiB RAM) one is used.
configurations object generated
Cluster custom configurations.
metadata object
Metadata information from cluster created resources.
nonProductionOptions array

Example:

apiVersion: stackgres.io/v1
kind: SGDistributedLogs
metadata:
  name: distributedlogs
spec:
  persistentVolume:
    size: 10Gi

Persistent volume

Property Required Updatable Type Default Description
size string
Size of the PersistentVolume set for the pod of the cluster for distributed logs. This size is specified either in Mebibytes, Gibibytes or Tebibytes (multiples of 2^20, 2^30 or 2^40, respectively).
storageClass string default storage class
Name of an existing StorageClass in the Kubernetes cluster, used to create the PersistentVolumes for the instances of the cluster.

Postgres Services

Property Required Updatable Type Default Description
primary object primary
Configuration for the -primary service. It provides a stable connection (regardless of primary failures or switchovers) to the read-write Postgres server of the cluster.
replicas object replicas
Configuration for the -replicas service. It provides a stable connection (regardless of replica node failures) to any read-only Postgres server of the cluster. Read-only servers are load-balanced via this service.

Primary service type

Property Required Updatable Type Default Description
type string ClusterIP
Specifies the type of Kubernetes service(ClusterIP, LoadBalancer, NodePort)
loadBalancerIP string
Specify loadBalancer IP of Postgres primary service for Distributed Log
annotations object
Custom Kubernetes annotations passed to the -primary service.

Replicas service type

Property Required Updatable Type Default Description
enabled boolean true
Specify if the -replicas service should be created or not.
type string ClusterIP
Specifies the type of Kubernetes service(ClusterIP, LoadBalancer, NodePort).
loadBalancerIP string
Specify loadBalancer IP of Postgres replica service for Distributed Log
annotations object
Custom Kubernetes annotations passed to the -replicas service.

Example:

apiVersion: stackgres.io/v1
kind: SGDistributedLogs
metadata:
  name: stackgres
spec:
  postgresServices:
    primary:
      type: ClusterIP
    replicas:
      enabled: true
      type: ClusterIP

Scheduling

Holds scheduling configuration for StackGres pods to have.

Property Required Updatable Type Default Description
nodeSelector object
Pod custom node selector.
nodeAffinity object
Node affinity is a group of node affinity scheduling rules.
tolerations array
Pod custom node tolerations

Node Affinity

Sets the pod’s affinity to restrict it to run only on a certain set of node(s)

Property Required Updatable Type Default Description
requiredDuringSchedulingIgnoredDuringExecution object
If the affinity requirements specified by this field are not met at scheduling time, the pod will not be scheduled onto the node. This type of affinity can restrict to only run the pod on specifically nodes like “Intel CPUs”.
preferredDuringSchedulingIgnoredDuringExecution array
The scheduler will prefer to schedule pods to nodes that satisfy the affinity expressions specified by this field, but it may choose a node that violates one or more of the expressions. The node that is most preferred is the one with the greatest sum of weights, i.e. for each node that meets all of the scheduling requirements (resource request, requiredDuringScheduling affinity expressions, etc.), compute a sum by iterating through the elements of this field and adding “weight” to the sum if the node matches the corresponding matchExpressions; the node(s) with the highest sum are the most preferred.

See Kubernetes pod node affinity definition for more details.

Tolerations

Holds scheduling configuration for StackGres pods to have.

Property Required Updatable Type Default Description
key string
Pod custom node selector.
operator string Equal
Pod custom node tolerations
value string
Pod custom node tolerations
effect string match all taint effects
Pod custom node tolerations

Configurations

Property Required Updatable Type Default Description
sgPostgresConfig string
Name of the SGPostgresConfig used for the distributed logs. It must exist. When not set, a default Postgres config, for the major version selected, is used.

Metadata

Holds custom metadata information for StackGres generated resources to have.

Property Required Updatable Type Default Description
annotations object
Custom Kubernetes annotations to be passed to resources created and managed by StackGres.

Annotations

Holds custom annotations for StackGres generated resources to have.

Property Required Updatable Type Default Description
allResources object
Annotations to attach to any resource created or managed by StackGres.
pods object
Annotations to attach to pods created or managed by StackGres.
services object
Annotations to attach to services created or managed by StackGres.
apiVersion: stackgres.io/v1
kind: SGDistributedLogs
metadata:
  name: stackgres
spec:
  pods:
    metadata:
      annotations:
        allResources:
          customAnnotations: customAnnotationValue

Non Production options

The following options should NOT be enabled in a production environment.

Property Required Updatable Type Default Description
disableClusterPodAntiAffinity boolean false

It is a best practice, on non-containerized environments, when running production workloads, to run each database server on a different server (virtual or physical), i.e., not to co-locate more than one database server per host.

The same best practice applies to databases on containers. By default, StackGres will not allow to run more than one StackGres or Distributed Logs pod on a given Kubernetes node. If set to true it will allow more than one StackGres pod per node.

disablePatroniResourceRequirements boolean false

It is a best practice, on containerized environments, when running production workloads, to enforce container’s resources requirements.

The same best practice applies to databases on containers. By default, StackGres will configure resource requirements for patroni container. Set this property to true to prevent StackGres from setting patroni container’s resources requirement.

disableClusterResourceRequirements boolean false

It is a best practice, on containerized environments, when running production workloads, to enforce container’s resources requirements.

By default, StackGres will configure resource requirements for all the containers. Set this property to true to prevent StackGres from setting container’s resources requirements (except for patroni container, see disablePatroniResourceRequirements).

enableSetPatroniCpuRequests boolean false

On containerized environments, when running production workloads, enforcing container’s cpu requirements request to be equals to the limit allow to achieve the highest level of performance. Doing so, reduces the chances of leaving the workload with less cpu than it requires. It also allow to set static CPU management policy that allows to guarantee a pod the usage exclusive CPUs on the node.

By default, StackGres will configure cpu requirements to have the same limit and request for the patroni container. Set this property to true to prevent StackGres from setting patroni container’s cpu requirements request equals to the limit when .spec.requests.cpu is configured in the referenced SGInstanceProfile.

enableSetClusterCpuRequests boolean false

On containerized environments, when running production workloads, enforcing container’s cpu requirements request to be equals to the limit allow to achieve the highest level of performance. Doing so, reduces the chances of leaving the workload with less cpu than it requires. It also allow to set static CPU management policy that allows to guarantee a pod the usage exclusive CPUs on the node.

By default, StackGres will configure cpu requirements to have the same limit and request for all the containers. Set this property to true to prevent StackGres from setting container’s cpu requirements request equals to the limit (except for patroni container, see enablePatroniCpuRequests) when .spec.requests.containers.<container name>.cpu .spec.requests.initContainers.<container name>.cpu is configured in the referenced SGInstanceProfile.

enableSetPatroniMemoryRequests boolean false

On containerized environments, when running production workloads, enforcing container’s memory requirements request to be equals to the limit allow to achieve the highest level of performance. Doing so, reduces the chances of leaving the workload with less memory than it requires.

By default, StackGres will configure memory requirements to have the same limit and request for the patroni container. Set this property to true to prevent StackGres from setting patroni container’s memory requirements request equals to the limit when .spec.requests.memory is configured in the referenced SGInstanceProfile.

enableSetClusterMemoryRequests boolean false

On containerized environments, when running production workloads, enforcing container’s memory requirements request to be equals to the limit allow to achieve the highest level of performance. Doing so, reduces the chances of leaving the workload with less memory than it requires.

By default, StackGres will configure memory requirements to have the same limit and request for all the containers. Set this property to true to prevent StackGres from setting container’s memory requirements request equals to the limit (except for patroni container, see enablePatroniCpuRequests) when .spec.requests.containers.<container name>.memory .spec.requests.initContainers.<container name>.memory is configured in the referenced SGInstanceProfile.