Set Up Example Project and Configure Build
Full sources for all Cloudflow example applications can be found in the
examples folder of the
cloudflow project on Github.
The sources for the example described below can be found in the application called
sensor-data-scala.
|
A typical Cloudflow application uses the organization shown below. We will implement the example in Scala.
-
In a convenient location, such as
my-cloudflow-example
create the following directory structure:|-project |--cloudflow-plugins.sbt |-src |---main |-----avro |-----blueprint |-----resources |-----scala |-------sensordata |-build.sbt
As we move through the process, the leaf level directories of the above tree will contain the following:
-
project/cloudflow-plugins.sbt : contains the Cloudflow
sbt
plugin name and version. -
avro : the avro schema of the domain objects
-
blueprint : the blueprint of the application in a file named
blueprint.conf
-
scala : the source code of the application under the package name
sensordata
-
build.sbt : the sbt build script
-
-
From the top-level directory, initialize
git
(after adding files you will commit them):git init
The Cloudflow sbt
plugin assumes that your project is version-managed usinggit
. It will usegit commit
information to generate dynamic versioning for the Docker images it produces. If the project doesn’t have an initialized git index with at least one commit, some commands will fail with anot a git repository
error.
The sbt build script
Cloudflow provides sbt plugins for the supported runtimes, Akka, Spark and Flink. The plugins speed up development by adding the necessary dependencies and abstracting much of the boilerplate necessary to build a complete application. You can use multiple runtimes in the same application, provided that each runtime is defined in its own sub-project.
In this example, we use the CloudflowAkkaPlugin
that provides the building blocks for developing a Cloudflow application with Akka Streams.
In addition to the backend-specific plugin, we need to add the CloudflowApplicationPlugin
that provides the image-building and local-running capabilities to the project.
-
Create a
build.sbt
file with the following contents and save it in at the same level as yoursrc
directory:import sbt._ import sbt.Keys._ lazy val sensorData = (project in file(".")) .enablePlugins(CloudflowApplicationPlugin, CloudflowAkkaPlugin, ScalafmtPlugin) .settings( scalaVersion := "2.12.10", scalafmtOnCompile := true, libraryDependencies ++= Seq( "com.lightbend.akka" %% "akka-stream-alpakka-file" % "1.1.2", "com.typesafe.akka" %% "akka-http-spray-json" % "10.1.11", "ch.qos.logback" % "logback-classic" % "1.2.3", "com.typesafe.akka" %% "akka-http-testkit" % "10.1.11" % "test", "org.scalatest" %% "scalatest" % "3.0.8" % "test" ), name := "sensor-data-scala", organization := "com.lightbend.cloudflow", headerLicense := Some(HeaderLicense.ALv2("(C) 2016-2020", "Lightbend Inc. <https://www.lightbend.com>")), crossScalaVersions := Vector(scalaVersion.value), scalacOptions ++= Seq( "-encoding", "UTF-8", "-target:jvm-1.8", "-Xlog-reflective-calls", "-Xlint", "-Ywarn-unused", "-Ywarn-unused-import", "-deprecation", "-feature", "-language:_", "-unchecked" ), runLocalConfigFile := Some("src/main/resources/local.conf"), scalacOptions in (Compile, console) --= Seq("-Ywarn-unused", "-Ywarn-unused-import"), scalacOptions in (Test, console) := (scalacOptions in (Compile, console)).value )
The script is a standard Scala sbt build file—with the addition of the Cloudflow plugin for Akka Streams, the Cloudflow application pluging, and also the Scalafmt plugin that we suggest to keep the style consistent in the application (optional).
-
Create the file
project/cloudflow-plugins.sbt
and add the Cloudflow plugin dependency in it:// Resolver for the cloudflow-sbt plugin resolvers += Resolver.url("cloudflow", url("https://lightbend.bintray.com/cloudflow"))(Resolver.ivyStylePatterns) addSbtPlugin("com.lightbend.cloudflow" % "sbt-cloudflow" % "2.0.0")
What’s next
Now, let’s define the Avro schema.