Exercising the Deployed Example Application

Once you have the sensor-data Cloudflow application deployed, you can use some CLI helpers to monitor the status of your application.

--help to see all options available

$ kubectl cloudflow --help

This command line tool can be used to deploy and operate Cloudflow applications.

Usage:
  cloudflow [command]

Available Commands:
  configure                 Configures a deployed Cloudflow application.
  deploy                    Deploys a Cloudflow application to the cluster.
  help                      Help about any command
  list                      Lists deployed Cloudflow application in the current cluster.
  scale                     Scales a streamlet of a deployed Cloudflow application to the specified number of replicas.
  status                    Gets the status of a Cloudflow application.
  undeploy                  Undeploys a Cloudflow application.
  update-docker-credentials Updates docker registry credentials that are used to pull Cloudflow application images.
  version                   Prints the plugin version.

Flags:
  -h, --help   help for cloudflow

Use "cloudflow [command] --help" for more information about a command.

list shows all applications deployed in the cluster

$ kubectl cloudflow list
NAME              NAMESPACE         VERSION           CREATION-TIME
sensor-data-scala sensor-data-scala 484-199a3c1       2020-06-10 22:28:32 +0200 CEST

status shows details of a running application

$ kubectl cloudflow status sensor-data-scala
Name:             sensor-data-scala
Namespace:        sensor-data-scala
Version:          484-199a3c1
Created:          2020-06-10 22:28:32 +0200 CEST
Status:           Running

STREAMLET         POD                                                 READY             STATUS            RESTARTS
http-ingress      sensor-data-scala-http-ingress-86dd5b8747-vnmxq     1/1               Running           0
invalid-logger    sensor-data-scala-invalid-logger-789d679855-9l7qz   1/1               Running           0
metrics           sensor-data-scala-metrics-6b7475c4f9-sxxw6          1/1               Running           0
valid-logger      sensor-data-scala-valid-logger-78594f47ff-v82zt     1/1               Running           0
validation        sensor-data-scala-validation-d8858dff-6xswj         1/1               Running           0

Push data to the Application

Our application uses an http based ingress to ingest data. Follow the following steps to push JSON data through the ingress into the application.

Get the port details of our ingress streamlet

$ kubectl describe pod -n sensor-data-scala sensor-data-http-ingress-fd9cdb66f-jbsrm
Name:               sensor-data-http-ingress-fd9cdb66f-jbsrm
Namespace:          sensor-data-scala
Priority:           0
PriorityClassName:  <none>
Node:               gke-dg-gke-1-default-pool-162a09d5-ddnq/10.132.0.21
Start Time:         Tue, 12 Nov 2019 12:47:20 +0530
Labels:             app.kubernetes.io/component=streamlet
                    app.kubernetes.io/managed-by=cloudflow
                    app.kubernetes.io/name=sensor-data-http-ingress
                    app.kubernetes.io/part-of=sensor-data
                    app.kubernetes.io/version=2-89ce8a7
                    com.lightbend.cloudflow/app-id=sensor-data
                    com.lightbend.cloudflow/streamlet-name=http-ingress
                    pod-template-hash=fd9cdb66f
Annotations:        prometheus.io/scrape: true
Status:             Running
IP:                 10.44.1.6
Controlled By:      ReplicaSet/sensor-data-http-ingress-fd9cdb66f
Containers:
  sensor-data-http-ingress:
    Container ID:  docker://9149cd757094e7ea1b943076048b7efc7aa343da8c2d598bba31295ef3cbfd6b
    Image:         eu.gcr.io/bubbly-observer-178213/sensor-data@sha256:ee496e8cf3a3d9ab71c3ef4a4929ed8eeb6129845f981c33005942314ad30f18
    Image ID:      docker-pullable://eu.gcr.io/bubbly-observer-178213/sensor-data@sha256:ee496e8cf3a3d9ab71c3ef4a4929ed8eeb6129845f981c33005942314ad30f18
    Ports:         3000/TCP, 2048/TCP, 2049/TCP, 2050/TCP
...

The streamlet exposes its HTTP endpoint for uploading data on port 3000 - let’s set up a port forwarding on it.

Check the created svc

$ kubectl get svc -n sensor-data-scala
NAME                                     TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)    AGE
sensor-data-scala-http-ingress-service   ClusterIP   10.0.22.206   <none>        3000/TCP   1m

Setup local port forwarding for the Pod port

$ kubectl port-forward svc/sensor-data-scala-http-ingress-service -n sensor-data-scala 3000:3000
Forwarding from 127.0.0.1:3000 -> 3000
Forwarding from [::1]:3000 -> 3000
Handling connection for 3000

The port-forward solution used here is a temporary route to the application from localhost. If you want to create a permanent route that can be accessed by anyone, please see the following chapter on Kubernetes ingresses: Providing External Access to Cloudflow Services

Push data to the application. In order to do that follow the steps:

create a json file named data.json with the following content:

data.json

{
  "deviceId": "c75cb448-df0e-4692-8e06-0321b7703992",
  "timestamp": 1495545346279,
  "measurements": {
    "power": 1.7,
    "rotorSpeed": 23.4,
    "windSpeed": 100.1
  }
}

run the following command:

curl -i -X POST http://localhost:3000 -H "Content-Type: application/json" --data @data.json

Note that we are using the port number 3000 of localhost to which we forward the pod port. This JSON record will pass through the stages of transformation within the pipeline that we defined in the blueprint.

When using this port-forward method, it’s also possible to use the same send-local-data.sh script that we used to send data to the local sandbox. For that, run the following command:

./send-local-data.sh

And you should see output similar to the following:

Sending {"deviceId":"c75cb448-df0e-4692-8e06-0321b7703992","timestamp":1495545646279,"measurements":{"power":1.7,"rotorSpeed":3.9,"windSpeed":105.9}}
HTTP/1.1 202 Accepted
Server: akka-http/10.1.11
Date: Thu, 11 Jun 2020 10:20:48 GMT
Content-Type: text/plain; charset=UTF-8
Content-Length: 88

The request has been accepted for processing, but the processing has not been completed.Sending {"deviceId":"c75cb448-df0e-4692-8e06-0321b7703992","timestamp":1495545646279,"measurements":{"power":1.7,"rotorSpeed":3.9,"windSpeed":105.9}}
HTTP/1.1 202 Accepted
Server: akka-http/10.1.11
Date: Thu, 11 Jun 2020 10:20:48 GMT
Content-Type: text/plain; charset=UTF-8
Content-Length: 88

Verify the Application works

Check the log of the streamlet valid-logger to verify that you get the proper transformed metric.

kubectl logs -n sensor-data-scala sensor-data-valid-logger-76884bb775-86pwh

Towards the end of the log you will see something like the following getting printed out:

[INFO] [06/11/2020 12:20:49.288] [akka_streamlet-akka.actor.default-dispatcher-5] [akka.actor.ActorSystemImpl(akka_streamlet)] valid-logger {"deviceId": "c75cb448-df0e-4692-8e06-0321b7703992", "timestamp": 1495545646279, "name": "rotorSpeed", "value": 3.9}
[INFO] [06/11/2020 12:20:49.293] [akka_streamlet-akka.actor.default-dispatcher-6] [akka.actor.ActorSystemImpl(akka_streamlet)] valid-logger {"deviceId": "c75cb448-df0e-4692-8e06-0321b7703992", "timestamp": 1495545646279, "name": "windSpeed", "value": 105.9}

Undeploy the Application

When we are done experimenting with the application, we can undeploy it to terminate its operations.

kubectl cloudflow undeploy sensor-data-scala