Custom resource

SparkConnectServer

An Apache Spark Connect server component. This resource is managed by the Stackable operator for Apache Spark.

View SDP dev release Download CRD (YAML)

API group: spark.stackable.tech
API versions in dev: v1alpha1
Scope: Namespaced
Operator: Apache Spark
Roles: none

Schema

SparkConnectServer.spark.stackable.tech .spec fields for SDP dev.

Schema

image # object required *
Specify which image to use, the easiest way is to only configure the productVersion. You can also configure a custom image registry to pull from, as well as completely custom images. Consult the Product image selection documentation for details.

Specify which image to use, the easiest way is to only configure the productVersion. You can also configure a custom image registry to pull from, as well as completely custom images. Consult the Product image selection documentation for details.
args # array = []
User provided command line arguments appended to the server entry point.

User provided command line arguments appended to the server entry point.
clusterOperation # object = {"stopped":false,"reconciliationPaused":false}
Cluster operations properties, allow stopping the product instance as well as pausing reconciliation.

Cluster operations properties, allow stopping the product instance as well as pausing reconciliation.
connectors # object = {"s3buckets":[]} One or more S3 connections to be used by the Spark Connect server.

One or more S3 connections to be used by the Spark Connect server.
executor # object Spark Connect executor properties.

Spark Connect executor properties.
objectOverrides # array = []
A list of generic Kubernetes objects, which are merged into the objects that the operator creates. List entries are arbitrary YAML objects, which need to be valid Kubernetes objects. Read the Object overrides documentation for more information.

A list of generic Kubernetes objects, which are merged into the objects that the operator creates. List entries are arbitrary YAML objects, which need to be valid Kubernetes objects. Read the Object overrides documentation for more information.
server # object = {"roleConfig":{"listenerClass":"cluster-internal"}} A Spark Connect server definition.

A Spark Connect server definition.
vectorAggregatorConfigMapName # string
Name of the Vector aggregator discovery ConfigMap. It must contain the key ADDRESS with the address of the Vector aggregator.

Name of the Vector aggregator discovery ConfigMap. It must contain the key ADDRESS with the address of the Vector aggregator.