Deploy the Example to a Kubernetes Cluster

If you have not already installed Cloudflow on a Kubernetes Cluster, please install Cloudflow first.

If you want to deploy the application to minikube, please follow this guide.

Once the application is built you need to publish the docker images to a container registry that stores, manages and secures your application images. In this example we will use a Docker Hub registry.

Follow these steps to complete the process of publishing:

Set up the target environment of publishing in your local build process. Create a file named target-env.sbt in the root of your project folder (the same level where you have build.sbt). The file needs to have the following 2 lines:

Configure your Docker repository

ThisBuild / cloudflowDockerRegistry := Some("docker.io")
ThisBuild / cloudflowDockerRepository := Some("<your docker hub username>")

Replace <docker hub username> with your Docker Hub username. Publish the docker image of your project to the above registry. Make sure you are logged in with `

docker login --username <your docker hub username>

and then build and publish the application with

sbt buildApp

Once the images are successfully published, you will see something like the following as output of buildApp in your console:

[info] Successfully built and published the following image:
[info]   docker.io/your-user-name/sensor-data-scala:471-89ca008
[success] Cloudflow application CR generated in /.../sensor-data-scala/target/sensor-data-scala.json
[success] Use the following command to deploy the Cloudflow application:
[success] kubectl cloudflow deploy /path/to/sensor-data-scala/target/sensor-data-scala.json
[success] Total time: 41 s, completed Jun 10, 2020 10:10:10 PM

At this point, the Docker image containing the streamlets has been published to the Docker repository. The sensor-data-scala.json file that the build produces, contains the application descriptor. We use the Cloudflow CLI to deploy that application description to a cluster.

Deploy Application to the Cluster

This is quite straightforward and the output of buildApp actually tells you what command to run for deployment. If your Docker Hub repository is private, you need to supply the credentials along with the CLI deploy command, so that Kubernetes can pull the images when the containers are created. The following example reads a password from a file, and passes it to the kubectl cloudflow deploy command using STDIN:

$ cat my-dockerhub-password.txt | kubectl cloudflow deploy /path/to/sensor-data-scala/target/sensor-data-scala.json -u <your docker hub username> --password-stdin

If your docker repository is public, or if image pull secrets are already setup in your Kubernetes cluster to pull from your docker repository, you can omit credentials with the --no-registry-credentials flag, which is shown in the following example:

$ kubectl cloudflow deploy /path/to/sensor-data-scala/target/sensor-data-scala.json --no-registry-credentials

If the command goes through you will see the following output:

[Done] Deployment of application `sensor-data-scala` has started.

Once all streamlets are up in their pods, you can see them running using the following command:

$ kubectl cloudflow status sensor-data-scala
Name:             sensor-data-scala
Namespace:        sensor-data-scala
Version:          484-199a3c1-dirty
Created:          2020-06-10 22:28:32 +0200 CEST
Status:           Running

STREAMLET         POD                                              READY             STATUS            RESTARTS
http-ingress      sensor-data-scala-http-ingress-6f4b677c84-8gtk7  1/1               Running           0
invalid-logger    sensor-data-scala-invalid-logger-f76667866-lffmk 1/1               Running           0
metrics           sensor-data-scala-metrics-859d9cf974-vrpzs       1/1               Running           0
valid-logger      sensor-data-scala-valid-logger-6946746b4b-bpqdc  1/1               Running           0
validation        sensor-data-scala-validation-b7f85669d-kkpx9     1/1               Running           0

Note that all streamlets run in a namespace that matches the name of the application.

Congratulations! You have deployed and started your first Cloudflow application.

What’s next

Now, we are ready to Exercise the example.