Saturday, January 28, 2017

Spring Data support for Cassandra 3

One of the items that caught my eye from the announcement of the new Spring Data release train named Ingalls was that the Spring Data Cassandra finally supports Cassandra 3+. So I revisited one of my old samples and tried it with a newer version of Cassandra.


Installing Cassandra


The first step is to install a local version of Cassandra and I continue to find the ccm tool to be outstanding in being able to bring up and tear down a small cluster. Here is the command that I am running to bring up a 3 node Apache Cassandra 3.9 based cluster.

ccm create test -v 3.9 -n 3 -s --vnodes


Create Schemas



Connect to a node in the cluster:
ccm node1 cqlsh

CREATE KEYSPACE IF NOT EXISTS sample WITH replication = {'class':'SimpleStrategy', 'replication_factor':1};

Next, I need to create the tables to hold the data. A general Cassandra recommendation is to model the tables based on query patterns - given this let me first define a table to hold the basic "hotel" information:

CREATE TABLE IF NOT EXISTS  sample.hotels (
    id UUID,
    name varchar,
    address varchar,
    state varchar,
    zip varchar,
    primary key((id), name)
);


Assuming I have to support two query patterns - a retrieval of hotels based on say the first letter, and a retrieval of hotels by state, I have a "hotels_by_letter" denormalized table to support retrieval by "first letter":

CREATE TABLE IF NOT EXISTS  sample.hotels_by_letter (
    first_letter varchar,
    hotel_name varchar,
    hotel_id UUID,
    address varchar,
    state varchar,
    zip varchar,
    primary key((first_letter), hotel_name, hotel_id)
);


And just for variety a "hotels_by_state" materialized view to support retrieval by state that the hotels are in:

CREATE MATERIALIZED VIEW sample.hotels_by_state AS
    SELECT id, name, address, state, zip FROM hotels
        WHERE state IS NOT NULL AND id IS NOT NULL AND name IS NOT NULL
    PRIMARY KEY ((state), name, id)
    WITH CLUSTERING ORDER BY (name DESC)


Coding Repositories


On the Java side, since I am persisting and querying a simple domain type called "Hotel", it looks like this:

@Table("hotels")
public class Hotel implements Serializable {
    @PrimaryKey
    private UUID id;
    private String name;
    private String address;
    private String state;
    private String zip;
    ...
}

Now, to be able to perform a basic CRUD operation on this entity all that is required is a repository interface as shown in the following code:
import cass.domain.Hotel;
import org.springframework.data.repository.CrudRepository;

import java.util.UUID;

public interface HotelRepository extends CrudRepository<Hotel, UUID>, HotelRepositoryCustom {}

This repository is additionally inheriting from a HotelRepositoryCustom interface which is to provide the custom finders to support retrieval by first name and state.

Now to persist a Hotel entity all I have to do is to call the repository method:

hotelRepository.save(hotel);

The data in the materialized view is automatically synchronized and maintained by Cassandra, however the data in the "hotels_by_letter" table has to be managed through code, so I have another repository defined to maintain data in this table:

public interface HotelByLetterRepository 
        extends CrudRepository<HotelByLetter, HotelByLetterKey>, HotelByLetterRepositoryCustom {}


The custom interface and its implementation is to facilitate searching this table on queries based on first letter of the hotel name and is implemented this way through the a custom repository implementation feature of Spring data Cassandra.

import com.datastax.driver.core.querybuilder.QueryBuilder;
import com.datastax.driver.core.querybuilder.Select;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.data.cassandra.core.CassandraTemplate;
import org.springframework.stereotype.Repository;

import java.util.List;

@Repository
public class HotelRepositoryImpl implements HotelRepositoryCustom {

    private final CassandraTemplate cassandraTemplate;

    @Autowired
    public HotelRepositoryImpl(CassandraTemplate cassandraTemplate) {
        this.cassandraTemplate = cassandraTemplate;
    }

    @Override
    public List<Hotel> findByState(String state) {
        Select select = QueryBuilder.select().from("hotels_by_state");
        select.where(QueryBuilder.eq("state", state));
        return this.cassandraTemplate.select(select, Hotel.class);
    }
}

@Repository
public class HotelByLetterRepositoryImpl implements HotelByLetterRepositoryCustom {
    private final CassandraTemplate cassandraTemplate;

    public HotelByLetterRepositoryImpl(CassandraTemplate cassandraTemplate) {
        this.cassandraTemplate = cassandraTemplate;
    }

    @Override
    public List<HotelByLetter> findByFirstLetter(String letter) {
        Select select = QueryBuilder.select().from("hotels_by_letter");
        select.where(QueryBuilder.eq("first_letter", letter));
        return this.cassandraTemplate.select(select, HotelByLetter.class);
    }

}


Given these repository classes, custom repositories that provide query support, the rest of the code is to wire everything together which Spring Boot's Cassandra Auto Configuration facilitates.

That is essentially all there is to it, the Spring Data Cassandra makes it ridiculously simple to interact with Cassandra 3+.

A complete working project is I believe a far better way to get familiar with this excellent library and I have such a sample available in my github repo here - https://github.com/bijukunjummen/sample-boot-with-cassandra





Sunday, January 15, 2017

Gradle Plugins DSL and Spring-Boot Plugin

Gradle Plugins DSL is a new gradle feature which provides a very succinct way of adding a plugin to a Gradle based project. A good way to show the utility of this new mechanism is in how it simplifies a sample Spring Boot based gradle build file.

If I were to generate a sample gradle based Spring boot project from the excellent http://start.spring.io site, a snippet of the gradle file which adds in the Spring Boot gradle plugin looks like this:

buildscript {
 ext {
  springBootVersion = '1.4.3.RELEASE'
 }
 repositories {
  mavenCentral()
 }
 dependencies {
  classpath("org.springframework.boot:spring-boot-gradle-plugin:${springBootVersion}")
 }
}

apply plugin: 'org.springframework.boot'

The new Gradle Plugins DSL simplifies this boilerplate drastically. An equivalent declaration using the new Plugins DSL is the following:

plugins {
  id "org.springframework.boot" version "1.4.3.RELEASE"
}

This IMHO reads far better, though it does require some level of mental parsing. The best way to understand this new syntax though may to know that this works in concert with the Gradle plugins portal, a centralized repository of plugins, to resolve the plugin related dependencies. The page for the Spring Boot plugin is here - https://plugins.gradle.org/plugin/org.springframework.boot.

Wednesday, January 11, 2017

Deploying akka-http app to Cloud Foundry - Part 2

In a preceding post I had gone over the steps to deploy a simple akka-http app to Cloud Foundry. The gist of it was that as long there is a way to create a runnable fat(uber) jar, the deployment is very straightforward - Cloud Foundry's Java buildpack can take the bits and wire up everything needed to get it up an running in the Cloud Foundry environment.

Here I wanted to go over a slightly more involved scenario - this is where the app has an external database dependency say to a MySQL database.

In a local environment the details of the database would have been resolved using a configuration typically specified like this:

sampledb = {
  url = "jdbc:mysql://localhost:3306/mydb?useSSL=false"
  user = "myuser"
  password = "mypass"
}

If the Mysql database were to be outside of Cloud Foundry environment this approach of specifying the database configuration will continue to work nicely. However if the service resides in a Cloud Foundry market place , then the details of the service is created dynamically at bind time with the Application.

Just to make this a little more concrete, in my local PCF Dev, I have a marketplace with "p-mysql" service available.



And if I were to create a "service instance" out of this:


and bind this instance to an app:


essentially what happens at this point is that the application has an environment variable called VCAP_SERVICES available to it and this has to be parsed to get the db creds. VCAP_SERVICES in the current scenario looks something like this:

{
  "p-mysql": [
   {
    "credentials": {
     "hostname": "mysql-broker.local.pcfdev.io",
     "jdbcUrl": "jdbc:mysql://mysql-broker.local.pcfdev.io:3306/myinstance?user=user\u0026password=pwd",
     "name": "myinstance",
     "password": "pwd",
     "port": 3306,
     "uri": "mysql://user:pwd@mysql-broker.local.pcfdev.io:3306/myinstance?reconnect=true",
     "username": "user"
    },
    "label": "p-mysql",
    "name": "mydb",
    "plan": "512mb",
    "provider": null,
    "syslog_drain_url": null,
    "tags": [
     "mysql"
    ]
   }
  ]
 }

This can be parsed very easily using Typesafe config, a sample (admittedly hacky) code looks like this:

  def getConfigFor(serviceType: String, name: String): Config = {
    val vcapServices = env("VCAP_SERVICES")
    val rootConfig = ConfigFactory.parseString(vcapServices)
    val configs = rootConfig.getConfigList(serviceType).asScala
      .filter(_.getString("name") == name)
      .map(instance => instance.getConfig("credentials"))

    if (configs.length > 0) configs.head
    else ConfigFactory.empty()
  }

and called the following way:
val dbConfig = cfServicesHelper.getConfigFor("p-mysql", "mydb")

This would dynamically resolve the credentials for mysql and would allow the application to connect to the database.

An easier way to follow all this may be to look at a sample code available in my github repo here - https://github.com/bijukunjummen/sample-akka-http-rest.

Tuesday, January 3, 2017

Deploying akka-http app to Cloud Foundry - Part 1

It is easy to deploy an akka-http application to Cloud Foundry. I experimented with a few variations recently and will cover ways to deploy an Akka-http based REST app in two parts - first a simple app with no external resource dependencies, the second a little more complex CRUD app that maintains state in a MySQL database.


Pre Requisites


A quick way to get a running Cloud Foundry instance is using PCF Dev, a small footprint distribution of Cloud Foundry that can be started up on a developer laptop.

The sample app that I am using is a stock demo app available via the Lightbend Activator, if you have activator binaries available locally, you can create a quick project using the following command:


Generating the sample App and running it locally

activator new sample-akka-http akka-http-microservice


The application can be brought up by running sbt and using the "re-start" task

$ sbt
> re-start

By default the app comes up on port 9000 and can be tested with a sample CURL call - more here:

$ curl http://localhost:9000/ip/8.8.8.8
{
  "city": "Mountain View",
  "query": "8.8.8.8",
  "country": "United States",
  "lon": -122.0881,
  "lat": 37.3845
}


Deploying to Cloud Foundry

There is one change that needs to be made to the application to get it to work in Cloud Foundry - adjusting the port where the application listens on. When the app is deployed to Cloud Foundry, an environment variable called "PORT" is the port that the application is expected to listen on. This change is the following for the sample app:

val port = if (sys.env.contains("PORT")) sys.env("PORT").toInt else config.getInt("http.port")
Http().bindAndHandle(routes, config.getString("http.interface"), port)

Here I look for the PORT environment variable and use that port if available.

There is already the "assembly" sbt plugin available which creates a fat jar with the appropriate entries, to be able to start up the main class of the application

> assembly

Go to "target/scala-2.11" folder and run the fat jar:
$ java -jar sample-akka-http-assembly-1.0.jar

And the application should come up cleanly.

Having a fat jar greatly simplifies the deployment to Cloud Foundry - In the Cloud Foundry world, a buildpack takes the application binaries and layers in the runtime(jvm, a container like tomcat, application certs, monitoring agents etc). Given this fat jar all that is needed to deploy to Cloud Foundry is a command which looks like this:

$ cf push -p sample-akka-http-assembly-1.0.jar sample-akka-http  

Assuming that you are targeting the local PCF Dev environment the application should get cleanly deployed using the appropriate buildpack(java buildpack in this instance) and be available to handle requests in a few minutes:


Which I can test using a curl command similar to what I had before:
$ curl http:// sample-akka-http.local.pcfdev.io/ip/8.8.8.8
{
  "city": "Mountain View",
  "query": "8.8.8.8",
  "country": "United States",
  "lon": -122.0881,
  "lat": 37.3845
}

That is all there is to it - if say some customizations need to be made to the application, say more jvm heap size, this can be easily done via other command line flags or using an application manifest. The process to deploy with external resource dependencies is a little more complex and I will cover this in a follow up post.