Kubernetes Deployments
In this chapter, you’ll see how Deployments bring self-healing, scalability, rolling updates, and versioned rollbacks to Kubernetes.
We’ll divide the chapter as follows:
• Deployment theory
• How to create a Deployment
• How to perform a rolling update
• How to perform a rollback
Deployment theory
At a high level, you start with application code. That gets packaged as a container and wrapped in a Pod so it can run on Kubernetes. However, Pods don’t self-heal, they don’t scale, and they don’t allow for easy updates or rollbacks. Deployments do all of these. As a result, you’ll almost always deploy Pods via a Deployment controller.
Figure 5.1 shows some Pods being managed by a Deployment controller.
Figure 5.1
It’s important to know that a single Deployment object can only manage a single Pod template. For example, if you have an application with a Pod template for the web front-end and another Pod template for the catalog service, you’ll need two Deployments. However, as you saw in Figure 5.1, a Deployment can manage multiple replicas of the same Pod. For example, Figure 5.1 could be a Deployment that currently manages two replicated web server Pods.
The next thing to know is that Deployments are fully-fledged objects in the Kubernetes API. This means you define them in manifest files that you POST to the API Server.
The last thing to note, is that behind-the-scenes, Deployments leverage another object called a ReplicaSet. While it’s best practice that you don’t interact directly with ReplicaSets, it’s important to understand the role they play.
Keeping it high-level, Deployments use ReplicaSets to provide self-healing and scaling.
Figure 5.2. shows the same Pods managed by the same Deployment. However, this time we’ve added a ReplicaSet object into the relationship and shown which object is responsible for which feature.
Figure 5.2
In summary, think of Deployments as managing ReplicaSets, and ReplicaSets as managing Pods. Put them all together, and you’ve got a great way to deploy and manage applications on Kubernetes.
Self-healing and scalability
Pods are great. They augment containers by allowing co-location of containers, sharing of volumes, sharing of memory, simplified networking, and a lot more. But they offer nothing in the way of self-healing and scalability
– if the node a Pod is running on fails, the Pod will not be restarted.
Enter Deployments…
Deployments augment Pods by adding things like self-healing and scalability. This means:
• If a Pod managed by a Deployment fails, it will be replaced – self-healing.
• If a Pod managed by a Deployment sees increased load, you can easily add more of the same Pod to deal with the load – scaling.
Remember though, behind-the-scenes, Deployments use an object called a ReplicaSet to accomplish self-healing and scalability. However, ReplicaSets operate in the background and you should always carry out operations against the Deployment. For this reason, we’ll focus on Deployments.
It’s all about the state
Before going any further, it’s critical to understand three concepts that are fundamental to everything about Kubernetes:
• Desired state
• Current state (sometimes called actual state or observed state)
• Declarative model
Desired state is what you want. Current state is what you have. If the two match, everybody’s happy.
The declarative model is a way of telling Kubernetes what your desired state is, without having to get into the detail of how to implement it. You leave the how up to Kubernetes.
The declarative model
There are two competing models. The declarative model and the imperative model.
The declarative model is all about describing the end-goal – telling Kubernetes what you want. The imperative model is all about long lists of commands to reach the end-goal – telling Kubernetes how to do something.
The following is an extremely simple analogy that might help:
• Declarative: I need a chocolate cake that will feed 10 people.
• Imperative: Drive to the store. Buy; eggs, milk, flour, cocoa powder… Drive home. Turn on oven. Mix ingredients. Place in baking tray. Place tray in oven for 30 minutes. Remove from oven and turn oven off.
Add icing. Leave to stand.
The declarative model is stating what you want (chocolate cake for 10). The imperative model is a long list of steps required to make a chocolate cake for 10.
Let’s look at a more concrete example.
Assume you’ve got an application with two services – front-end and back-end. You’ve built container images so that you can have a Pod for the front-end service, and a separate Pod for the back-end service. To meet expected demand, you always need 5 instances of the front-end Pod, and 2 instances of the back-end Pod.
Taking the declarative approach, you write a configuration file that tells Kubernetes what you want your application to look like. For example, I want 5 replicas of the front-end Pod all listening externally on port 80
please. And I also want 2 back-end Pods listening internally on port 27017. That’s the desired state. Obviously, the YAML format of the config file will be different, but you get the picture.
Once you’ve described the desired state, you give the config file to Kubernetes and sit back while Kubernetes does the hard work of implementing it.
But things don’t stop there… Kubernetes implements watch loops that are constantly checking that you’ve got what you asked for – does current state match desired state.
Believe me when I tell you, it’s a beautiful thing.
The opposite of the declarative model is the imperative model. In the imperative model, there’s no concept of what you actually want. At least there’s no record of what you want, all you get is a list of instructions.
To make things worse, imperative instructions might have multiple variations. For example, the commands to start containerd containers are different from the commands to start gVisor containers. This ends up being more work, prone to more errors, and because it’s not declaring a desired state, there’s no self-healing.
Believe me when I tell you, this isn’t so beautiful.
Kubernetes supports both models, but strongly prefers the declarative model.
Reconciliation loops
Fundamental to desired state is the concept of background reconciliation loops (a.k.a. control loops).
For example, ReplicaSets implement a background reconciliation loop that is constantly checking whether the right number of Pod replicas are present on the cluster. If there aren’t enough, it adds more. If there are too many, it terminates some.
To be crystal clear, Kubernetes is constantly making sure that current state matches desired state.
If they don’t match – maybe desired state is 10 replicas, but only 8 are running – Kubernetes declares a red-alert condition, orders the control plane to battle-stations and brings up two more replicas. And the best part… it does all of this without calling you at 04:20 am!
But it’s not just failure scenarios. These very-same reconciliation loops enable scaling.
For example, if you POST an updated config that changes replica count from 3 to 5, the new value of 5 will be registered as the application’s new desired state. The next time the ReplicaSet reconciliation loop runs, it will notice the discrepancy and follow the same process – sounding the claxon horn for red alert and spinning up two more replicas.
It really is a beautiful thing.
Rolling updates with Deployments
As well as self-healing and scaling, Deployments give us zero-downtime rolling-updates.
As previously mentioned, Deployments use ReplicaSets for some of the background legwork. In fact, every time you create a Deployment, you automatically get a ReplicaSet that manages the Deployment’s Pods.
Note: Best practice states that you should not manage ReplicaSets directly. You should perform all actions against the Deployment object and leave the Deployment to manage ReplicaSets.
It works like this. You design applications with each discrete service as a Pod. For convenience – self-healing, scaling, rolling updates and more – you wrap Pods in Deployments. This means creating a YAML configuration file describing all of the following:
• How many Pod replicas
• What image to use for the Pod’s container(s)
• What network ports to use
• Details about how to perform rolling updates
You POST the YAML file to the API server and Kubernetes does the rest.
Once everything is up and running, Kubernetes sets up watch loops to make sure observed state matches desired state.
All good so far.
Now, assume you’ve experienced a bug, and you need to deploy an updated image that implements a fix. To do this, you update the same Deployment YAML file with the new image version and re-POST it to the API server.
This registers a new desired state on the cluster, requesting the same number of Pods, but all running the new version of the image. To make this happen, Kubernetes creates a new ReplicaSet for the Pods with the new image.
You now have two ReplicaSets – the original one for the Pods with the old version of the image, and a new one for the Pods with the updated version. Each time Kubernetes increases the number of Pods in the new ReplicaSet (with the new version of the image) it decreases the number of Pods in the old ReplicaSet (with the old version of the image). Net result, you get a smooth rolling update with zero downtime. And you can rinse and repeat the process for future updates – just keep updating that manifest file (which should be stored in a version control system).
Brilliant.
Figure 5.3 shows a Deployment that has been updated once. The initial deployment created the ReplicaSet on the left, and the update created the ReplicaSet on the right. You can see that the ReplicaSet for the initial deployment has been wound down and no longer has any Pod replicas. The ReplicaSet associated with the update is active and owns all of the Pods.
Figure 5.3
It’s important to understand that the old ReplicaSet still has its entire configuration, including the older version of the image it used. This will be important in the next section.
Rollbacks
As we’ve seen in Figure 5.3, older ReplicaSets are wound down and no longer manage any Pods. However, they still exist with their full configuration. This makes them a great option for reverting to previous versions.
The process of rolling back is essentially the opposite of a rolling update – wind one of the old ReplicaSets up, and wind the current one down. Simple.
Figure 5.4 shows the same app rolled back to the initial revision.
Figure 5.4
That’s not the end though. There’s built-in intelligence that lets us say things like “wait X number of seconds after each Pod comes up before proceeding to the next Pod”. There’s also startup probes, readiness probes, and liveness probes that can check the health and status of Pods. All-in-all, Deployments are excellent for performing rolling updates and versioned rollbacks.
With all of that in mind, let’s get your hands dirty and create a Deployment.
How to create a Deployment
In this section, you’ll create a brand-new Kubernetes Deployment from a YAML file. You can do the same thing imperatively using the kubectl run command, but you shouldn’t. The right way is the declarative way.
The following YAML snippet is the Deployment manifest file that you’ll use. It’s available in the book’s GitHub repo in the “deployments” folder and is called deploy.yml.
The examples assume you’ve got a copy in your system’s PATH, and is called deploy.yml.
apiVersion: apps/v1
#Older versions of k8s use apps/v1beta1
kind: Deployment
metadata:
name: hello-deploy
spec:
replicas: 10
selector:
matchLabels:
app: hello-world
minReadySeconds: 10
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1
maxSurge: 1
template:
metadata:
labels:
app: hello-world
spec:
containers:
- name: hello-pod
image: nigelpoulton/k8sbook:latest
ports:
- containerPort: 8080
Warning: The images used in this book are not maintained and may contain vulnerabilities and other security issues. Use with caution.
Let’s step through the config and explain some of the important parts.
Right at the very top you specify the API version to use. Assuming that you’re using an up-to-date version of Kubernetes, Deployment objects are in the apps/v1 API group.
Next, the .kind field tells Kubernetes you’re defining a Deployment object.
The .metadata section is where we give the Deployment a name and labels.
The .spec section is where most of the action happens. Anything directly below .spec relates to the Pod. Anything nested below .spec.template relates to the Pod template that the Deployment will manage. In this example, the Pod template defines a single container.
.spec.replicas tells Kubernetes how may Pod replicas to deploy. spec.selector is a list of labels that Pods must have in order for the Deployment to manage them. And .spec.strategy tells Kubernetes how to perform updates to the Pods managed by the Deployment.
Use kubectl apply to implement it on the cluster.
Note: kubectl apply POSTs the YAML file to the Kubernetes API server.
$ kubectl apply -f deploy.yml
deployment.apps/hello-deploy created
The Deployment is now instantiated on the cluster.
Inspecting Deployments
You can use the normal kubectl get and kubectl describe commands to see details of the Deployment.
$ kubectl get deploy hello-deploy
NAME
DESIRED
CURRENT
UP-TO-DATE
AVAILABLE
AGE
hello-deploy
10
10
10
10
24s
$ kubectl describe deploy hello-deploy
Name:
hello-deploy
Namespace:
default
Selector:
app=hello-world
Replicas:
10 desired | 10 updated | 10 total ...
StrategyType:
RollingUpdate
MinReadySeconds:
10
RollingUpdateStrategy:
1 max unavailable, 1 max surge
Pod Template:
Labels:
app=hello-world
Containers:
hello-pod:
Image:
nigelpoulton/k8sbook:latest
Port:
8080/TCP
<SNIP>
The command outputs have been trimmed for readability. Yours will show more information.
As we mentioned earlier, Deployments automatically create associated ReplicaSets. Use the following kubectl command to confirm this.
$ kubectl get rs
NAME
DESIRED
CURRENT
READY
AGE
hello-deploy-7bbd...
10
10
10
1m
Right now you only have one ReplicaSet. This is because you’ve only performed the initial rollout of the Deployment. You can also see that the name of the ReplicaSet matches the name of the Deployment with a hash on the end. The hash is a hash of the Pod template section (anything below .spec.template) of the YAML
manifest file.
You can get more detailed information about the ReplicaSet with the usual kubectl describe command.
Accessing the app
In order to access the application from a stable name or IP address, or even from outside the cluster, you need a Kubernetes Service object. We’ll discuss Service objects in detail in the next chapter, but for now it’s enough to know they provide a stable DNS name and IP address for a set of Pods.
The following YAML defines a Service that will work with the Pod replicas previously deployed. The YAML is included in the “deployments” folder of the book’s GitHub repo called svc.yml.
apiVersion: v1
kind: Service
metadata:
name: hello-svc
labels:
app: hello-world
spec:
type: NodePort
ports:
- port: 8080
nodePort: 30001
protocol: TCP
selector:
app: hello-world
Deploy it with the following command (the command assumes the manifest file is called svc.yml and is in your system’s PATH).
$ kubectl apply -f svc.yml
service/hello-svc created
Now that the Service is deployed, you can access the app from either of the following: 1. From inside the cluster using the DNS name hello-svc on port 8080
2. From outside the cluster by hitting any of the cluster nodes on port 30001
Figure 5.5 shows the Service being accessed from outside of the cluster via a node called node1 on port 30001. It assumes that node1 is resolvable, and that port 30001 is allowed by any intervening firewalls.
If you are using Minikube, you should append port 30001 to the end of the Minikube IP address. Use the minikube ip command to get the IP address of your Minikube.
Figure 5.5
Performing a rolling update
In this section, you’ll see how to perform a rolling update on the app you’ve just deployed. We’ll assume the new version of the app has already been created and containerized as a Docker image with the edge tag. All that is left to do is use Kubernetes to push the update to production. For this example, we’re ignoring real-world CI/CD
workflows and version control tools.
The first thing you need to do is update the image tag used in the Deployment’s manifest file. The initial version of the app used an image tagged as nigelpoulton/k8sbook:latest. You’ll update the .spec.template.spec.containers section of the Deployment manifest to reference the new nigelpoulton/k8sbook:edge image. This will ensure that next time the manifest is POSTed to the API server, all Pods in the Deployment will be replaced with new ones running the new edge image.
The following is the updated deploy.yml manifest file – the only change is to .spec.template.spec.containers.image indicated by the commented line.
apiVersion: apps/v1
kind: Deployment
metadata:
name: hello-deploy
spec:
replicas: 10
selector:
matchLabels:
app: hello-world
minReadySeconds: 10
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1
maxSurge: 1
template:
metadata:
labels:
app: hello-world
spec:
containers:
- name: hello-pod
image: nigelpoulton/k8sbook:edge
# This line changed
ports:
- containerPort: 8080
Before POSTing the updated configuration to Kubernetes, let’s look at the settings that govern how the update will proceed.
The .spec section of the manifest contains all of the settings relating to how updates will be performed. The first value of interest is .spec.minReadySeconds. This is set to 10, telling Kubernetes to wait for 10 seconds between each Pod being updated. This is useful for throttling the rate at which updates occur – longer waits give you a chance to spot problems and avoid situations where you update all Pods to a faulty configuration.
There is also a nested .spec.strategy map that tells Kubernetes you want this Deployment to:
• Update using the RollingUpdate strategy
• Never have more than one Pod below desired state (maxUnavailable: 1)
• Never have more than one Pod above desired state (maxSurge: 1) As the desired state of the app demands 10 replicas, maxSurge: 1 means you will never have more than 11 Pods during the update process, and maxUnavailable: 1 means you’ll never have less than 9. The net result will be a rolling update that updates two Pods at a time (the delta between 9 and 11 is 2).
With the updated manifest ready, you can initiate the update by re-POSTing the updated YAML file to the API server.
$ kubectl apply -f deploy.yml --record
deployment.apps/hello-deploy configured
The update may take some time to complete. This is because it will iterate two Pods at a time, pulling down the new image on each node, starting the new Pods, and then waiting 10 seconds before moving on to the next two.
You can monitor the progress of the update with kubectl rollout status.
$ kubectl rollout status deployment hello-deploy
Waiting for rollout to finish: 4 out of 10 new replicas...
Waiting for rollout to finish: 4 out of 10 new replicas...
Waiting for rollout to finish: 5 out of 10 new replicas...
^C
If you press Ctrl+C to stop watching the progress of the update, you can run kubectl get deploy commands while the update is in process. This lets you see the effect of some of the update-related settings in the manifest. For example, the following command shows that 5 of the replicas have been updated and you currently have 11. 11 is 1 more than the desired state of 10. This is a result of the maxSurge=1 value in the manifest.
$ kubectl get deploy
NAME
DESIRED
CURRENT
UP-TO-DATE
AVAILABLE
AGE
hello-deploy
10
11
5
9
28m
Once the update is complete, we can verify with kubectl get deploy.
$ kubectl get deploy hello-deploy
NAME
DESIRED
CURRENT
UP-TO-DATE
AVAILABLE
AGE
hello-deploy
10
10
10
10
39m
The output shows the update as complete – 10 Pods are up to date.
You can get more detailed information about the state of the Deployment with the kubectl describe deploy command. This will include the new version of the image in the Pod Template section of the output.
If you’ve been following along with the examples, you’ll be able to hit refresh in your browser and see the updated app (Figure 5.6).The old version of the app displayed “Kubernetes Rocks!”, the new version displays “The Kubernetes Book!!!”.
Figure 5.6
How to perform a rollback
A moment ago, you used kubectl apply to perform a rolling update on a Deployment. You used the --record flag so that Kubernetes would maintain a documented revision history of the Deployment. The following kubectl rollout history command shows the Deployment with two revisions.
$ kubectl rollout history deployment hello-deploy
deployment.apps/hello-deploy
REVISION
CHANGE-CAUSE
1
<none>
2
kubectl apply --filename-deploy.yml --record=true
Revision 1 was the initial deployment that used the latest image tag. Revision 2 is the rolling update you just performed. You can see that the command used to invoke the update has been recorded in the object’s history.
This is only there because you used the --record flag as part of the command to invoke the update. This might be a good reason for you to use the --record flag.
Earlier in the chapter we said that updating a Deployment creates a new ReplicaSet, and that any previous ReplicaSets are not deleted. You can verify this with a kubectl get rs.
$ kubectl get rs
NAME
DESIRED
CURRENT
READY
AGE
hello-deploy-6bc8...
10
10
10
10m
hello-deploy-7bbd...
0
0
0
52m
The output shows that the ReplicaSet for the initial revision still exists (hello-deploy-7bbd...) but that it has been wound down and is not managing any replicas. The hello-deploy-6bc8... ReplicaSet is the one from the latest revision and is active with 10 replicas under management. However, the fact that the previous version still exists makes rollbacks extremely simple.
If you’re following along, it’s worth running a kubectl describe rs against the old ReplicaSet to prove that its configuration still exists.
The following example uses the kubectl rollout command to roll the application back to revision 1. This is an imperative operation and not recommended. However, it can be convenient for quick rollbacks, just remember to update your source YAML files to reflect the imperative changes you make to the cluster.
$ kubectl rollout undo deployment hello-deploy --to-revision=1
deployment.apps "hello-deploy" rolled back
Although it might look like the rollback operation is instantaneous, it’s not. Rollbacks follow the same rules set out in the rolling update sections of the Deployment manifest – minReadySeconds: 10, maxUnavailable: 1, and maxSurge: 1. You can verify this and track the progress with the following kubectl get deploy and kubectl rollout commands.
$ kubectl get deploy hello-deploy
NAME
DESIRED
CURRNET
UP-TO-DATE
AVAILABE
AGE
hello-deploy
10
11
4
9
45m
$ kubectl rollout status deployment hello-deploy
Waiting for rollout to finish: 6 out of 10 new replicas have been updated...
Waiting for rollout to finish: 7 out of 10 new replicas have been updated...
Waiting for rollout to finish: 8 out of 10 new replicas have been updated...
Waiting for rollout to finish: 1 old replicas are pending termination...
Waiting for rollout to finish: 9 of 10 updated replicas are available...
^C
Congratulations. You’ve performed a rolling update and a successful rollback.
Use kubectl delete -f deploy.yml and kubectl delete -f svc.yml to delete the Deployment and Service used in the examples.
Just a quick reminder. The rollback operation you just initiated was an imperative operation. This means that the current state of the cluster will not match your source YAML files – the latest version of the YAML file lists the edge image, but you’ve rolled the cluster back to the latest image. This is a problem with the imperative approach. In the real world, following a rollback operation like this, you should manually update your source YAML files to reflect the changes incurred by the rollback.
Chapter summary
In this chapter, you learned that Deployments are a great way to manage Kubernetes apps. They build on top of Pods by adding self-healing, scalability, rolling updates, and rollbacks. Behind-the-scenes, they leverage ReplicaSets for the self-healing and scalability parts.
Like Pods, Deployments are objects in the Kubernetes API, and you should work with them declaratively.
When you perform updates with the kubectl apply command, older versions of ReplicaSets get wound down, but they stick around making it easy to perform rollbacks.