Deploy example to GKE cluster

Once the application is built you need to publish the docker image to some registry that stores, manages and secures your application image. In this example we will use the GCloud registry eu.gcr.io.

Follow these steps to complete the process of publishing:

  • Set up the target environment of publishing in your local build process. Create a file named target-env.sbt in the root of your project folder (the same level where you have build.sbt). The file needs to have the following 2 lines:

ThisBuild / cloudflowDockerRegistry := Some("eu.gcr.io")
ThisBuild / cloudflowDockerRepository := Some("<gcloud project id>")

Here project id refers to the ID of your Google Cloud Platform project. This can be found on the project page of your Google Cloud Platform.

  • Publish the docker image of your project to the above registry. This has 3 steps:

  • First make sure that you have access to the cluster gcloud container clusters get-credentials <cluster-name>

  • And that you have access to the Google docker registry gcloud auth configure-docker

  • Then publish sbt buildAndPublish

Once the image is successfully published, you will see something like the following as output of buildAndPublish in your console:

[info] 2-89ce8a7: digest: sha256:ee496e8cf3a3d9ab71c3ef4a4929ed8eeb6129845f981c33005942314ad30f18 size: 6804
[info]
[info] Successfully built and published the following Cloudflow application image:
[info]
[info]   eu.gcr.io/<gcloud project id>/sensor-data:2-89ce8a7
[info]
[info] You can deploy the application to a Kubernetes cluster using any of the the following commands:
[info]
[info]   kubectl-cloudflow deploy eu.gcr.io/<gcloud project id>/sensor-data:2-89ce8a7

So now the image is available in the registry for deployment. We will next use Cloudflow CLI to deploy the application to the cluster.

Deploy Application to the Cluster

This is quite straightforward and the output of buildAndPublish actually tells you what command to run for deployment. In addition you need to supply the credentials along with the CLI.

$ kubectl-cloudflow deploy -u oauth2accesstoken eu.gcr.io/<gcloud project id>/sensor-data:2-89ce8a7 -p "$(gcloud auth print-access-token)"

If the command goes through you will see the following output:

[Done] Deployment of application `sensor-data` has started.

Once all streamlets are up in their pods, you can see them running using the following command:

$ kubectl get pods -n sensor-data
NAME                                          READY   STATUS    RESTARTS   AGE
sensor-data-http-ingress-fd9cdb66f-jbsrm      1/1     Running   0          3m1s
sensor-data-invalid-logger-549d687d89-m64l7   1/1     Running   0          3m1s
sensor-data-metrics-6c47bfb489-xd644          1/1     Running   0          3m1s
sensor-data-valid-logger-76884bb775-86pwh     1/1     Running   0          3m1s
sensor-data-validation-7dd858b6c5-lcp2n       1/1     Running   0          3m1s

Note that all streamlets run in a namespace that matches the name of the application.

Congratulations! You have deployed and started your first Cloudflow application.

What’s next

Now, we are ready to Exercise the example.