curl -sfL https://get.k3s.io | K3S_KUBECONFIG_MODE="644" sh -
curl -sfL https://get.k3s.io | K3S_KUBECONFIG_MODE="644" sh -
Let’s continue in a new directory and also create a separate namespace for the Strimzi operator:
mkdir ~/strimzi
cd ~/strimzi
kubectl create namespace strimzi-operator
kubectl create namespace kafka
Now we download the Strimzi CRDs and the operator and adjust the namespace configuration so that the operator runs in the namespace strimzi-operator
:
wget https://github.com/strimzi/strimzi-kafka-operator/releases/download/0.40.0/strimzi-crds-0.40.0.yaml -O strimzi-crds.yaml
wget https://github.com/strimzi/strimzi-kafka-operator/releases/download/0.40.0/strimzi-cluster-operator-0.40.0.yaml -O strimzi-cluster-operator.yaml
sed -i 's/namespace: .*/namespace: strimzi-operator/' strimzi-cluster-operator.yaml
We want Strimzi Operator to monitor all namespaces so that we can also start several Kafka clusters per Kubernetes.
However, we need to customise the Strimzi deployment for this. To do this, replace the following lines in the strimzi-cluster-operator.yaml
file:
strimzi-cluster-operator.yaml
# Old (for me line 7452
env:
- name: STRIMZI_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
# New
env:
- name: STRIMZI_NAMESPACE
value: "*"
The operator requires some additional rights that are not available in the deployment:
kubectl create clusterrolebinding strimzi-cluster-operator-namespaced --clusterrole=strimzi-cluster-operator-namespaced --serviceaccount strimzi-operator:strimzi-cluster-operator
kubectl create clusterrolebinding strimzi-cluster-operator-watched --clusterrole=strimzi-cluster-operator-watched --serviceaccount strimzi-operator:strimzi-cluster-operator
kubectl create clusterrolebinding strimzi-cluster-operator-entity-operator-delegation --clusterrole=strimzi-entity-operator --serviceaccount strimzi-operator:strimzi-cluster-operator
Now we can install the CRDs and the operator:
kubectl apply -f strimzi-crds.yaml
kubectl apply -f strimzi-cluster-operator.yaml -n strimzi-operator
Wait a moment until the operator is ready:
kubectl get pods -n strimzi-operator -w
We would now like to set up the Kafka cluster "correctly" with KRaft:
3 KRaft Controllers with 5GB memory each
3 Kafka Brokers with 50GB memory each
To do this, we create a node pool for both the controllers and the brokers:
kafka-controller-pool.yaml
apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaNodePool
metadata:
name: controller
namespace: kafka
labels:
strimzi.io/cluster: kafka
spec:
replicas: 3
roles:
- controller
storage:
type: jbod
volumes:
- id: 0
type: persistent-claim
size: 5Gi
deleteClaim: false
kafka-broker-pool.yaml
apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaNodePool
metadata:
name: broker
namespace: kafka
labels:
strimzi.io/cluster: kafka
spec:
replicas: 3
roles:
- broker
storage:
type: jbod
volumes:
- id: 0
type: persistent-claim
size: 50Gi
deleteClaim: false
Finally, the resources for the Kafka Cluster itself:
kafka-cluster.yaml
apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
name: kafka
namespace: kafka
annotations:
strimzi.io/node-pools: enabled
strimzi.io/kraft: enabled
spec:
kafka:
version: 3.7.0
listeners:
- name: plain
port: 9092
type: internal
tls: false
config:
offsets.topic.replication.factor: 3
transaction.state.log.replication.factor: 3
transaction.state.log.min.isr: 2
default.replication.factor: 3
min.insync.replicas: 2
entityOperator:
topicOperator: {}
userOperator: {}
Now we can create the resources:
# We are still in the namespace kafka
kubectl apply -f kafka-controller-pool.yaml
kubectl apply -f kafka-broker-pool.yaml
kubectl apply -f kafka-cluster.yaml
Take a closer look at the Kafka resource and then wait until the pods have started:
kubectl config set-context --current --namespace=kafka
kubectl describe kafka kafka
kubectl get pods -w