In Defense Of Kubernetes

Vinter, Fri Jan 28 2022 • cloud kubernetes programming

[DISCLAIMER: I don't consider myself an expert Kubernetes developer. This is just a personal opinion on the matter]

I've recently started working on a project that involved Kubernetes. I was the first to propose the technology to my company, and I've never used it before. To be fully honest, this was my first time building a production ready application and my first introduction to webservers / cloud architecture / CDNs. We had one requisite: build a fast website that will be able to manage an unknown number of concurrent users. We knew we had to host it on Azure, and we knew that the business requirements would change in a matter of days. This is why I went with K8S even though no one in our company used it before.

The first impact was rough, I barely used Docker before and I had no idea where to start. All of the tutorials online featured giant diagrams, pipelines and arrows.

This is undoubtedly hostile to a new user that wants to start using the technology. Let's not talk about everything that surrounds it. Minikube? k3s? AWS? Azure? Everyone is doing the same thing in a slightly different manner and you need to find the right tutorial and documentation for your set up.

With this said? I spent a month reading the documentation and now I love it. I understand it. It doesn't seem that complicated.

And I want to argue that it's because I started with it

Artificial Complexity?

One of the main points I always see online against kubernetes goes something like this.

"Why would you need Kubernetes? You can just build your own service mesh with on premise Apache+SQLServer and a couple of NGINX instances, you just need to configure iptables, routings, build a couple of make scripts or maybe link it to something like Nomad or Consul"

To be honest? I almost have no clue on what any of that means. I don't have a 10-year experience, I never had to set up a production server, my NGINX and Apache experiences are limited and I find iptables, host configs and routing in general very complex.

And I personally think that demanding knowing something like that from a developer isn't fair. And I also think that people find these technologies easier because they grew up with them, they're not new and radical.

It's under my assumption that most experienced programmers have layers of knowledge that help them understand way better the "on premise" infrastructure, but what if you don't have those layers?

Setting Up A Production Infrastructure in a Week*

Common Glossary

Before continuing I would like to do a quick rundown of the terms for anyone that is unfamiliar with kubernetes:

Container: A containerized application with Docker
Pod: The smallest kubernetes unit: it can run multiple containers inside of it
Node: A physical machine that can host multiple Pods
ReplicaSet: A list of rules describing the desired state of our pods (mostly the number of replicas we want)
Deployment: A yaml file containing the deployment rules for any ReplicaSet

(please note that I'm not saying that just because it was easy for me it will be for you)

So, how did my learning process go? Wonderfully. After the initial hurdles everything made sense. I'm simplifying here a bit but, a Container is almost a Pod. A Pod is scheduled and maintained by a Replica Set that tells how to replicate it, a Replica Set is managed by a Deployment that tells how to update and rollout Replica Sets. You have Jobs and Services but that's pretty much anything that someone should know to start.

After starting a Kubernetes Cluster (more on this later) you can have copies of Nginx running in seconds (and this works for most of your apps)

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.14.2
        ports:
        - containerPort: 80

Then I dig into pipelines. In a few hours I had a bare bones Argo + Tekton CI/CD pipeline integrated with Git and Azure with multiple namespaces. I understand that people don't like magic, and for me at the time it was. But sometimes you want something working before you fully understand what it does under the hood. (Since then I had the time to research and understand almost everything, and I still don't find it complicated)

To add ArgoCD to your cluster you just simply type

kubectl create namespace argocd
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

And expose it either locally or globally

kubectl patch svc argocd-server -n argocd -p '{"spec": {"type": "LoadBalancer"}}' #globally
kubectl port-forward svc/argocd-server -n argocd 8080:443 #locally

Our devs had the ability to write code, push it, see the tests and the previews and automatically have it pushed to staging after a week of research.

Internal Comunication

Comunication between Services is a vast topic and significantly different from the standard deployment tecnologies I thought I'd briefly touch it here

Since we have multiple replicated instances running together sending messages between them might seem daunting at first. But I really think it's not:

Containers in the same Pod can talk via localhost since they run in the same network namespace
Each Pod has its own IP address and Kubernetes fakes an ethernet connection for them, then they talk by using a Network Bridge
Each Pod also has an internal proxy that allows us to map a single IP to a set of pods
By creating a Service (using a .yaml) we can simply tell Kubernetes to route all connections for that IP to all pods that match a selector specified in the yaml file itself

That's it! Kubernetes abstracts most of the complexity and allows us to think about Pods like single machines running on a LAN network!

Cloud Native Landscape

You're probably familiar with the following image that represents the technologies surrounding Kubernetes:

Good to see that Kubernetes has simplified software development.

I might be missing something, but to me it seems so unfair to use something like this to criticize a product and its complexity? You don't need any of those tools to work on the cloud and/or with k8s. Did anyone ever do a similar image for node modules? Python packages?

I'm honestly grateful for many of these products. Observability? It was as simple as applying a .yaml to my cluster. Same with logging and testing. All of the developers at my company are thankful for Kubernetes because they don't have to think about trivial things anymore. And I know that some might say that it's not because of k8s and I might've done all of this in a different way. But the point is that with kubernetes it was incredibly easy, I did it just with a couple of .yaml files and in a matter of minutes.

Just like we did with ArgoCD we can deploy a Grafana instance just by copying the yaml and applying it

kubectl apply -f grafana.yaml

Intellectual Honesty

I know, I'm avoiding some things. Cloud Providers manage most of the complexity for me, it's more expensive, I don't actually understand the fine details of the system yet etc. Also I don't want to argue about Cloud Provider lock in, that's a topic for another time. But the point I'm trying to make is that if you have to start from scratch I don't think that it kubernetes is inherently harder to learn (unless you need to set it up on premise). Sure, if you need a simple server then a NGINX instance on a server somewhere might suffice, but if you need to do something more than that? I would definitely choose Kubernetes again.

And on a side note, once I set up the whole infrastructure I almost never had to change it. We're currently working with ~30 services, but the last time I had to kubectl something was probably 3 months ago.

I will update the post once the product goes live, and I don't want to sound like a source of objective truth. Kubernetes worked for me, it might not work for you. But the phrase "you shouldn't be using it unless you're Google" will keep sounding unfair to me.