Creating Kubernetes Controllers

Kubernetes is widely used to build cloud native apps and work in cloud environments. Since kubernetes is open source, the architecture used behind the robust tool to orchestrate your running micro services containers is can be extended in your own project.

Industries using cloud or building their own cloud, use kubernetes. We can understand the importance of kubernetes by looking at all the Cloud providers services which are providing kubernetes to create multi cluster environment to be used by their clients. I will not go into what is kubernetes and how does it work, please read kubernetes documentation on kubernetes.io to understand more about the use of it.

Kubernetes Documentation

Kubernetes provide a way to build your own custom kubernetes resources called as Kubernetes Custom Resource in order to extend its functionality according to your requirements. To give an example you can create your own custom object using kubernetes schema.

Kubernetes is been part of cloud development for a long time and there are open source libraries build to integrate it with our project. Kubernetes operators provides the best way to manage your kubernetes resources and it is easy to implement. Quoted in kubernetes.io blog:-

Operators are proving to be an excellent solution to running stateful distributed applications in Kubernetes. Open source tools like the Operator SDK provide ways to build reliable and maintainable operators, making it easier to extend Kubernetes and implement custom scheduling

Controllers are control loops used to manage your kubernetes resources.

In Kubernetes, controllers are control loops that watch the state of your cluster, then make or request changes where needed. Each controller tries to move the current cluster state closer to the desired state.

Let’s start with creating a controller to watch for an update in kubernetes pod and then add a label to it. To write the controller we are going to use operator sdk which will create a bootstrap code. If you have minikube installed in your local environment or any other local tool (like k3d), we can start with creating a pod using:-

$ kubectl run --image=nginx my-nginx

Install operator-sdk to create a bootstrap controller:-

Installation

Once the installation is done, we can verify the version of operator-sdk to make sure that it is matching with updated version of kubernetes.

$ operator-sdk version

Next step will be to create a directory, initialise operator which will handle the controller for pod resource and finally we create a controller with a kind set to pod.

$ mkdir custom-controller && cd custom-controller
$ operator-sdk init --domain=domain.com --repo=github.com/<user>/custom-controller
$ operator-sdk create api --group=core --version=v1 --kind=Pod --controller=true --resource=false

If you face an error in go version when trying to initialise an operator skip the go version check in init operator command but still it is suited to work with updated versions of go and kubernetes.

$ operator-sdk init --domain=domain.com --repo=github.com/<user>/custom-controller --skip-go-version-check

This will generate a controller/pod_controller.go inside custom-controller repo. Inside that file we need to update our logic to watch for annotation and add a label to the pod using controller.

func (r *PodReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
    _ = r.Log.WithValues("pod", req.NamespacedName)

    // your logic here

    return ctrl.Result{}, nil
}

Reconcile method will be called for any operation on pod, the name and namespace of the pod can be extracted from ctrl.Request struct.

Complete code to watch for annotation inside the pod and adding the label to it:-

Now run the local kubernetes cluster setup and use make run command to run the code from a terminal, make sure you have minikube installed to run cluster in your local environment.

2021-10-31T13:30:04.771+0530 INFO controller-runtime.manager.controller.pod Starting EventSource {"reconciler group": "", "reconciler kind": "Pod", "source": "kind source: /, Kind="}
2021-10-31T13:30:04.771+0530 INFO controller-runtime.manager.controller.pod Starting Controller {"reconciler group": "", "reconciler kind": "Pod"}
2021-10-31T13:30:04.872+0530 INFO controller-runtime.manager.controller.pod Starting workers {"reconciler group": "", "reconciler kind": "Pod", "worker count": 1}

Create a pod with image nginx in another terminal and add annotation to view the events triggered on another terminal where go server is running using make run:-

$ kubectl run --image=nginx my-nginx
NAME       READY   STATUS    RESTARTS   AGE   LABELS
my-nginx   1/1     Running   0          18s   run=my-nginx

Once the pod is running, add the annotation to the running pod which will trigger an event leading to adding the name of pod as label as defined in the pod_controller.go code above.

$ kubectl annotate pod my-nginx custom/add-pod-name-label=true
pod/my-nginx annotated

The go controller running will show the event triggered adding label and we can see the label added to the pod using kubectl show labels command:-

$ kubectl get pods my-nginx --show-labels
NAME       READY   STATUS    RESTARTS   AGE   LABELS
my-nginx   1/1     Running   1          7d    custom/pod-name=my-nginx,run=my-nginx

Concluding the article, operator-sdk is really convenient sdk to create a kubernetes controller. And we just need to worry about updating the logic inside Reconcile method of controller. Full code available at https://github.com/Himanshuxone/label-operator.

Follow https://golang-geek.medium.com/ for more articles on cloud and other technologies. Mention in comments for any question or write an email to himanshusdec8@gmail.com for information on Cloud.

Contact: himanshusdec8@gmail.com