SDP 26.7 is planned as the next major release. Its contents and planned date may change before publication. Planned release date: July 2026.

Custom resource

SparkHistoryServer

A Spark history server component. This resource is managed by the Stackable operator for Apache Spark.

View SDP 26.7 release Download CRD (YAML)

API group: spark.stackable.tech
API versions in 26.7: v1alpha1
Scope: Namespaced
Operator: Apache Spark
Roles: 1 (nodes)

Schema

SparkHistoryServer.spark.stackable.tech .spec fields for SDP 26.7.

Schema

nodes # role object required * A history server node role definition.

A history server node role definition.
image # object required *
Specify which image to use, the easiest way is to only configure the productVersion. You can also configure a custom image registry to pull from, as well as completely custom images. Consult the Product image selection documentation for details.

Specify which image to use, the easiest way is to only configure the productVersion. You can also configure a custom image registry to pull from, as well as completely custom images. Consult the Product image selection documentation for details.
logFileDirectory # object required * The log file directory definition used by the Spark history server.

The log file directory definition used by the Spark history server.
objectOverrides # array = []
A list of generic Kubernetes objects, which are merged into the objects that the operator creates. List entries are arbitrary YAML objects, which need to be valid Kubernetes objects. Read the Object overrides documentation for more information.

A list of generic Kubernetes objects, which are merged into the objects that the operator creates. List entries are arbitrary YAML objects, which need to be valid Kubernetes objects. Read the Object overrides documentation for more information.
sparkConf # object = {}
A map of key/value strings that will be passed directly to Spark when deploying the history server.

A map of key/value strings that will be passed directly to Spark when deploying the history server.
vectorAggregatorConfigMapName # string
Name of the Vector aggregator discovery ConfigMap. It must contain the key ADDRESS with the address of the Vector aggregator.

Name of the Vector aggregator discovery ConfigMap. It must contain the key ADDRESS with the address of the Vector aggregator.