Reducing infrastructure costs is quite simple: just make sure your application environments are not running, when you don't need them. Shutting down a single VM has an immediate effect on the operation costs. Undeploying a Kubernetes environment allows the IT team to implement and benefit from a cluster auto-downscaling policy. Even shutting down an environment at night and weekends can save the organization a lot of money.
Scaling resources on Kubernetes can be quite tricky. The "Horizontal Pod Autoscaler" is a good solution to scale the number of pods based on observed CPU utilization in Production. Normally ony 1 instance of each pod is required in a development environment. The HPA can down scale your deployment, but you can't set the minimum pods to 0.
So, how can Pods be scaled based on time? What if you want to scale down the pods at night or to scale up a deployment for the same peak hours every day?
How to solve it?
We can use Kubernetes CronJobs to scale deployments up or down.
bitnami/kubectl:latest provides us a pod with the
kubectl tool, which allows us to run commands from a cron job directly on the Kubernetes platform.
To scale the deployment we use the following command
kubectl scale deployment <deployment-name> --replicas=<number-of-pods>.
If you need to scale a group of deployment, you can use the selector argument.
For example the
-l environment=dev will select all deployments with the dev environment label.
The Cron expression is a bit tricky again.
The times are based on the timezone of the kube-controller-manager.
If not customized the default
UTC timezone is used.
Of course the job requires a service account with the permission to manipulate the deployment specification.
Stop all pods at night
Scale down all deployments in the dev environment at 7pm UTC
apiVersion: batch/v1beta1 kind: CronJob metadata: name: scale-down-job spec: schedule: "0 19 * * 1-5" # <1> successfulJobsHistoryLimit: 1 failedJobsHistoryLimit: 1 jobTemplate: spec: template: spec: restartPolicy: OnFailure # <2> containers: - name: kubectl image: bitnami/kubectl:latest # <3> imagePullPolicy: IfNotPresent args: - scale - deployment - -l - environment=dev # <4> - --replicas=0 # <5> serviceAccountName: cicd # <6>
<1> Cron expression to run every working day at 7pm UTC
<2> The default restartPolicy is not allowed and must be set to
<3> Docker image with the kubectl tool
<4> Selector to match all deployments with label environment=dev
<5> Set the number of pods. Use 0 to remove all pods.
<6> Service account to run the job. Requires permission to manipulate the deployment specs.
Scale up all deployments in the dev environment at 7am UTC
apiVersion: batch/v1beta1 kind: CronJob metadata: name: scale-up-job spec: schedule: "0 7 * * 1-5" # <7> successfulJobsHistoryLimit: 1 failedJobsHistoryLimit: 1 jobTemplate: spec: template: spec: restartPolicy: OnFailure containers: - name: kubectl image: bitnami/kubectl:latest imagePullPolicy: IfNotPresent args: - scale - deployment - -l - environment=dev - --replicas=1 serviceAccountName: cicd
<7> Cron expression to run every working day at 7am UTC