Auto-magic Pre-emptible Starter for GCP

Google Cloud Platform offers a “pre-emptible” option for it’s virtual machines. This means that google will shut your instance(s) down within a 24-hour period. This gets you an 80% discount (see: https://cloud.google.com/preemptible-vms).

What if we want to auto-magically start our instances back up? I have the solution. We will create a CronJob on Kubernetes which will run every 15-minutes and will automatically start our stopped instances.

The ConfigMap

First we need to create a ConfigMap to hold our values:

CONFIGMAP.YAML

apiVersion: v1
kind: ConfigMap
metadata:
  name: gcloud-config
data:
  PROJECT: <your google cloud project name>
  ZONE: <the zone of the instance>
  INSTANCE_NAME: <name of the instance>

Now we need to install it with:

$ kubectl apply -f configmap.yaml

The Secret

We need to store our Google Cloud Platform Service Account in a Kubernetes Secret. This will be mounted by our Pod and will be available on the file system:

$ kubectl create secret generic gcloud-service-account \
                    --from-file=<path to your .json key>

The CronJob

The meat of the puzzle is the CronJob. This will run every minute (or you can change it to your liking) and will start our instance:

CRONJOB.YAML

apiVersion: batch/v1beta1
kind: CronJob

metadata:
  name: gcloud-preemptible-starter
spec:
  schedule: "* * * * *"
  concurrencyPolicy: Forbid
  successfulJobsHistoryLimit: 1
  failedJobsHistoryLimit: 1
  jobTemplate:
    spec:
      template:
        spec:
          containers:
            - name: gcloud-preemptible-starter
              image: google/cloud-sdk:183.0.0-alpine
              resources:
                requests:
                  cpu: 0.1
                  memory: 128M
                limits:
                  memory: 512M
              args:
                - /bin/sh
                - -c
                - gcloud auth activate-service-account --key-file /config/*.json && gcloud compute instances start $(INSTANCE_NAME) --project $(PROJECT) --zone $(ZONE)
              env:
                - name: PROJECT
                  valueFrom:
                    configMapKeyRef:
                      name: gcloud-config
                      key: PROJECT
                - name: ZONE
                  valueFrom:
                    configMapKeyRef:
                      name: gcloud-config
                      key: ZONE
                - name: INSTANCE_NAME
                  valueFrom:
                    configMapKeyRef:
                      name: gcloud-config
                      key: INSTANCE_NAME
              volumeMounts:
                - name: gcloud-service-account
                  mountPath: /config
                  readOnly: true
          restartPolicy: Never
          volumes:
            - name: gcloud-config
              configMap:
                name: gcloud-config
            - name: gcloud-service-account
              secret:
                secretName: gcloud-service-account

Now we can install it with:

$ kubectl apply -f cronjob.yaml

You should now be able to see your cronjob:

$ kubectl get cronjob

NAME                                  SCHEDULE      SUSPEND   ACTIVE   LAST SCHEDULE   AGE
gcloud-preemptible-starter            * * * * *     False     0        20s             7m33s

To get the logs out of our cronjob run:

$ kubectl logs pod/gcloud-preemptible-starter-1560486060-hcnmh

Activated service account credentials for: [preemptible-restarter@streaming-platform-production.iam.gserviceaccount.com]
Starting instance(s) centos-1...
.done.
Updated [https://www.googleapis.com/compute/v1/projects/streaming-platform-production/zones/us-east1-b/instances/centos-1].

See Also

Have questions, comments or concerns? Contact Me!

This tutorial also has a complimentary repository where you can simply make install,
See https://github.com/mateothegreat/k8-byexamples-gcloud-preemptible-starter.