DataStax Cassandra Operator readiness for multi-Operator environments
DevOps teams are building their Kubernetes native stacks assembling and developing required Kubernetes Operators for their workload. But what does it take to develop an Operator that is a good citizen of such a multi-Operator environment? To help Operator developers systematically approach this problem we have developed the Operator Maturity Model. Various community Operators have benefited from this model. This model also has been used by the Operator development team at DataStax in developing their Cassandra Operator.
The Operator Maturity Model is divided into six categories (Consumability, Configurability, Security, Robustness, Debuggability, and Portability) with a set of guidelines in each category. It collectively offers 26 guidelines that help Operator developers ensure that their Custom Resources are ready to be part of application workflows that may use Custom Resources coming from a variety of Operators.
The DataStax team was referring to these open-source guidelines from the start. So when we started engaging with them officially to certify their Operator for multi-Operator environment readiness, they already were satisfying 50–60% of the guidelines. We then worked together to bring in additional improvements to satisfy 100% of the Operator Maturity Model guidelines.
DataStax Cassandra open-source Operator is enterprise-ready now and can be used on any of the supported Kubernetes setups along with other required Operators. Its central Custom Resource “CassandraDatacenter” is easy to consume and configure while being designed with adequate security boundaries and can be used to set up multi-tenant workflows.
In order to demonstrate use of DataStax Cassandra Operator with another Operator, we considered the problem of adding SSL certificate to an instance of CassandraDatacenter. DataStax Cassandra Operator docs have outlined the steps towards this here. We automate these steps using the CertManager Operator.
Refer to the sample workflow in the following diagram. We use CertManager to enable creating SSL certificates for an instance of CassandraDatacenter. We added an Ingress object to point to the Service object that Cassandra Operator creates when instantiating CassandraDatacenter Custom Resource. The Ingress object was annotated with CertManager annotation to link it with ClusterIssuer Custom Resource and enable creating SSL cert for that Ingress. Cassandra Operator’s Github documentation includes information about how it creates Service objects for each CassandraDatacenter instance and which of these Service objects should be used for external access.
When performing multi-Operator evaluation two key aspects that we consider are:
- How easy it is for the end-users of an Operator to use the Custom Resources in creating their Kubernetes platform workflows.
- How easy it is to visualize such platform workflows for debugging/troubleshooting purposes.
To simplify this for end users, we have developed Platform-as-Code resource annotations and KubePlus kubectl plugins. The resource annotations are added on Operator CRDs to generically capture assumptions made by the Operator developers. From here KubePlus starts working on the Custom Resources of that Operator. For DataStax Cassandra Operator we have added the following annotations on the CassandraDatacenter CRD.
resource/man: cassandradatacenter.readmeresource/composition: StatefulSet, Service, PodDisruptionBudgetresource/annotation-relationship: “on:Secret, key:cassandra.datastax.com/watched-by, value:INSTANCE.metadata.name”
The ‘resource/man’ annotation provides a way to include ‘man page’ like information about a Custom Resource. The value of this annotation is the name of the ConfigMap that contains this information. ‘resource/composition’ annotation lists the sub-resources that are created by an Operator when instantiating a Custom Resource. Finally, the ‘resource/annotation-relationship’ annotation defines any annotation-based relationships that an Operator needs for its operation.
Once these resource annotations were added on the CassandraDatacenter CRD KubePlus started working on the platform workflows involving CassandraDatacenter Custom Resource instance:
- kubectl man CassandraDatacenter
- kubectl composition CassandraDatacenter
- kubectl connections CassandraDatacenter
Here is the output of the ‘kubectl connections’ KubePlus kubectl plugin that shows the platform workflow between ClusterIssuer and CassandraDatacenter Custom Resource presented in the above figure.
You can try the above example by following these steps.
Once you have KubePlus kubectl plugins enabled for all the CRDs on your clusters, it significantly simplifies building and troubleshooting your platform workflows.
Conclusion:
- Go ahead and try DataStax Cassandra Operator to run your Cassandra workload on Kubernetes: https://github.com/datastax/cass-operator
- Check out KubePlus to build, visualize and troubleshoot your Kubernetes automation involving multiple Operators: https://github.com/cloud-ark/kubeplus
- If you are developing a new Operator, don’t miss on checking out the Operator Maturity Model: https://github.com/cloud-ark/kubeplus/blob/master/Guidelines.md
