Spring Cloud Sidecar - Initialization of Nodes

In the last blog post I had described how the Sidecar application can be used for registering the Cassandra nodes with Eureka and more generally can be used for registering any non-JVM application with Eureka.

In this post I will cover how an application can go about querying the Sidecar registered nodes.

Discovering Registered Nodes - Post Initialization

If the registered nodes are not required during the bean initialization phase then discovering the nodes is fairly straightforward along these lines:

@Component
public class SampleCommandLineRunner implements CommandLineRunner {

    @Autowired
    private DiscoveryClient discoveryClient;

    @PostConstruct
    public void postConstruct() {
//        System.out.println("Printing from postConstruct");
//        printDiscoveredNodes();
    }

    @Override
    public void run(String... strings) throws Exception {
        System.out.println("Printing from run method");
        printDiscoveredNodes();
    }

    public void printDiscoveredNodes() {
        System.out.println(" Printing Discovered Nodes ");

        for (ServiceInstance instance: discoveryClient.getInstances("samplecassandra.vip")) {
            System.out.println("Host: Port = " + instance.getHost() + ":" + instance.getPort());
        }
    }
}

These would print the nodes registered with a name of "samplecasssandra.vip" VIP.

Note that the nodes are being printed from the run method which gets called past the initialization of Spring container. If however the nodes were attempted to be listed from one of the lifecycle stages say the postConstruct method then very likely an exception will be thrown (this behavior is seen with "Angel.SR3" release of Spring Cloud, but appears to work cleanly with "Brixton.*" versions)

Discovering Registered Nodes - During Initialization

Now if an application needs to discover the nodes during initialization the flow is a little more complicated, for a potential issue look at this ticket.

The DiscoveryClient is initialized very late in the Spring Lifecycle and if DiscoveryClient is used in any post-processing activity of a bean it is likely to give an exception.

As an example, say the Cassandra nodes registered using Sidecar is now used by an application to initialize Cassandra connectivity, a way to do it would be to create a wrapper around Cassandra connectivity this way:

import org.springframework.data.cassandra.core.CassandraTemplate;


public class CassandraTemplateWrapper extends CassandraTemplate {

    @Override
    public void afterPropertiesSet() {
        
    }
}

Here CassandraTemplate is being overridden to prevent the check in afterPropertiesSet method that a Cassandra session exists, as a session will be established much later in the start-up cycle.

A Cassandra session can be injected into this custom CassandraTemplate lazily in a bean that implements SmartLifecyle along these lines:

package mvctest.cassandra;

import com.datastax.driver.core.Cluster;
import com.datastax.driver.core.Session;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.FactoryBean;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.SmartLifecycle;
import org.springframework.core.Ordered;
import org.springframework.data.cassandra.core.CassandraTemplate;
import org.springframework.stereotype.Component;

@Component("cassandraTemplate")
public class EurekaCassandraTemplateFactoryBean implements SmartLifecycle, FactoryBean<CassandraTemplate>, Ordered {

    ....


    @Override
    public boolean isAutoStartup() {
        return true;
    }

    @Override
    public void start() {
        LOGGER.info("About to start Discovery client lookup of Cassandra Cluster!!!");
        final Cluster cluster = this.eurekaClusterBuilder.build();
        Session session = cluster.connect(this.cassandraProperties.getKeyspace());
        this.cassandraTemplateWrapper.setSession(session);
        LOGGER.info("Completed Discovery client lookup of Cassandra Cluster!!!");
        running = true;
    }

    @Override
    public boolean isRunning() {
        return this.running;
    }

    @Override
    public int getPhase() {
        return Integer.MAX_VALUE;
    }

    @Override
    public int getOrder() {
        return 1;
    }
}

This way the Cassandra session can be created very late in the cycle. Somewhat rough, but the approach works.

If you are interested in exploring this sample further I have this code available in my github repo here.

Spring Cloud Sidecar

I have an application deployed to a NetflixOSS based cloud which has a structure along these lines:

Essentially a service which persists information to a Cassandra cluster. All the applications are registered to Eureka - so in this instance the service as well as the Cassandra nodes are registered with Eureka, further the service connects to the Cassandra cluster by looking up the nodes via Eureka. 

I will deal with this in two parts - 

1. Registering Cassandra nodes with Eureka
2. Service using Eureka to connect to the Cassandra Cluster

Registering Cassandra Nodes with Eureka

This is where a Sidecar application fits in - the purpose of  Sidecar is to facilitate some of things that make an application a good citizen in a Cloud environment, in this specific instance it enables Cassandra to register with Eureka, respond to health checks. Spring Cloud Netflix Sidecar project provides the necessary support to create a Sidecar application.

The amount of coding required to get a Sidecar application up and running is very minimal, Sidecar behaves like a typically Spring Cloud application except that instead of registering itself to Eureka it has to register another application, so the configuration is mostly the same. 

This is my entire code for the Sidecar application!:

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cloud.netflix.sidecar.EnableSidecar;

@SpringBootApplication
@EnableSidecar
public class SampleSidecarApplication {

    public static void main(String[] args) {
        SpringApplication.run(SampleSidecarApplication.class, args);
    }
}

and the properties to go with this:

application.yml

eureka:
  instance:
    virtual-host-name: samplecassandra.vip

spring:
  application:
    name: samplecassandra

sidecar:
  port: 9042

Here the port is declared to be the port that is relevant to Cassandra.

There is one more aspect to handle, healthcheck, the Sidecar exposes an endpoint which can test the health of the supported application in whatever makes sense for the application. For Cassandra it could be connecting to the local node and firing a small CQL query.

Conclusion

Assuming that the Cassandra nodes are now registered with Eureka, there is a good level of complication in trying to create a Cassandra Session on the consuming service side, this is mainly because of the timing involved in instantiating the Eureka client and the point at which the code tries to find the list of nodes. I will cover this in a subsequent post. If you would like to explore this sample further, here is the github repo.

Rest client calls with Spring Cloud

There are a few interesting ways to make REST client calls with the Spring-Cloud project.
Spring-Cloud rest support builds on top of the core Netflix OSS libraries, but abstracts them and in the process simplifies using the libraries.

RestTemplate

As a first step let us consider the traditional way to make Rest calls through Spring based applications, using RestTemplate:

public class RestTemplateIntegrationTest {

    @Autowired
    private RestTemplate restTemplate;

    @Test
    public void testCallPongService() {
        ResponseEntity<MessageAcknowledgement> ack =
                restTemplate.exchange("http://servicehost/message",
                        HttpMethod.POST,
                        new HttpEntity<>(new Message("test", "hello")),
                        MessageAcknowledgement.class,
                        Collections.emptyMap());
        assertThat(ack.getBody().getPayload(), equalTo("Pong From Configuration Server"));
    }
}

In this specific instance, the host part of the url is expected to be completely known to the client, RestTemplate will take care of marshalling the Java object to the appropriate media type, making the REST call, and unmarshalling the response back to a Java Object.

RestTemplate with Ribbon and Eureka

Netflix Ribbon provides a library for making REST based calls, whereas with RestTemplate the host is expected to be completely known to the client, with Ribbon the host is typically resolved through the Centralized Netflix Eureka server and Ribbon takes care of load-balancing the calls if multiple hosts are found for a service. If Spring-cloud libraries and Ribbon related libraries are present in the classpath, then Spring-Cloud enhances RestTemplate to be based on Ribbon instead with no additional configuration required, with Spring-Cloud in place the call would exactly like before, with a few twists.

ResponseEntity<MessageAcknowledgement> ack =
        restTemplate.exchange("http://sample-pong/message",
                HttpMethod.POST,
                new HttpEntity<>(new Message("test", "hello")),
                MessageAcknowledgement.class,
                Collections.emptyMap());

The twist is that the hostname which in this instance is "sample-pong" is significant, it is not the real host name - instead, an attempt is made to find the list of servers with this name as the registration name in Eureka and the resulting host/port is used for making the request.


If customizations are required a named client can be provided with Ribbon specific properties specified for the named client, along these lines:

ResponseEntity<MessageAcknowledgement> ack =
        restTemplate.exchange("http://samplepong/message",
                HttpMethod.POST,
                new HttpEntity<>(new Message("test", "hello")),
                MessageAcknowledgement.class,
                Collections.emptyMap());

The named client above is "samplepong" and the ribbon specific properties for this client is along these lines:

samplepong:
  ribbon:
    DeploymentContextBasedVipAddresses: sample-pong
    NIWSServerListClassName: com.netflix.niws.loadbalancer.DiscoveryEnabledNIWSServerList
    ReadTimeout: 5000
    MaxAutoRetries: 2

If you are interested in more low level configurations for Ribbon, refer here

Ribbon is a fairly complicated low-level way of making a REST call, RestTemplate abstracts the Ribbon implementation and makes it look easy from a clients perspective.

Netflix Feign

Netflix Feign is another simplified way to make calls to REST based services, all it requires is an interface with relevant annotations which is best demonstrated with an example:

import org.bk.consumer.domain.Message;
import org.bk.consumer.domain.MessageAcknowledgement;
import org.springframework.cloud.netflix.feign.FeignClient;
import org.springframework.http.MediaType;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;

@FeignClient("samplepong")
public interface PongClient {

    @RequestMapping(method = RequestMethod.POST, value = "/message",
            produces = MediaType.APPLICATION_JSON_VALUE, consumes = MediaType.APPLICATION_JSON_VALUE)
    MessageAcknowledgement sendMessage(@RequestBody Message message);
}

The annotations are Spring specific though, Spring-Cloud facilitates this by adding in encoders and decoders which support Spring MVC annotations.

@FeignClient annotation on the interface identifies this a FeignClient code. @EnableFeignClients is required in a Spring Configuration to load up all such FeignClient's.

@SpringBootApplication
@EnableEurekaClient
@EnableFeignClients
public class PingApplication {

    public static void main(String[] args) {
        SpringApplication.run(PingApplication.class, args);
    }
}

References

• Spring Cloud Documentation
• Blog by Josh Long on Microservice Registration and Discovery with Spring Cloud and Netflix's Eureka