Latency and Fault Tolerance by:
Stop cascading failures. Fallbacks and graceful degradation. Fail fast and rapid recovery.
Thread and semaphore isolation with circuit breakers.
I saw a sample demonstrated by Josh Long(@starbuxman) which makes use of Hystrix integrated with Spring - the specific code is here. The sample makes use of annotations to hystrix enable a service class.
My objective here is to recreate a similar set-up in a smaller unit test mode. With that in mind, consider the following interface which is going to be made fault tolerant using Hystrix library:
package hystrixtest; public interface RemoteCallService { String call(String request) throws Exception; }
And a dummy implementation for it. The dummy implementation delegates to a mock implementation which in-turn fails the first two times it is called and succeeds with the third call:
package hystrixtest; import com.netflix.hystrix.contrib.javanica.annotation.HystrixCommand; import org.mockito.invocation.InvocationOnMock; import org.mockito.stubbing.Answer; import static org.mockito.Mockito.*; public class DummyRemoteCallService implements RemoteCallService { private RemoteCallService mockedDelegate; public DummyRemoteCallService() { try { mockedDelegate = mock(RemoteCallService.class); when(mockedDelegate.call(anyString())) .thenThrow(new RuntimeException("Deliberately throwing an exception 1")) .thenThrow(new RuntimeException("Deliberately throwing an exception 2")) .thenAnswer(new Answer<String>() { @Override public String answer(InvocationOnMock invocationOnMock) throws Throwable { return (String) invocationOnMock.getArguments()[0]; } }); }catch(Exception e) { throw new IllegalStateException(e); } } @Override @HystrixCommand(fallbackMethod = "fallBackCall") public String call(String request) throws Exception { return this.mockedDelegate.call(request); } public String fallBackCall(String request) { return "FALLBACK: " + request; } }
The remote call has been annotated with the @Hystrixcommand annotation with a basic configuration to fall back to a "fallBackCall" method in case of a failed remote call.
Now, as you can imagine, there has to be something in the Hystrix library which should intercept calls annotated with @HystrixCommand annotation and makes it fault tolerant. This is a working test which wraps the necessary infrastructure together - in essence, Hystrix library provides a companion AOP based library that intercepts the calls. I have used Spring testing support here to bootstrap the AOP infrastructure, to create the HystrixCommandAspect as a bean, the call goes to the "fallBackCall" for the first two failed calls and succeeds the third time around:
package hystrixtest; import com.netflix.hystrix.contrib.javanica.aop.aspectj.HystrixCommandAspect; import org.junit.Test; import org.junit.runner.RunWith; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; import org.springframework.context.annotation.EnableAspectJAutoProxy; import org.springframework.test.context.ContextConfiguration; import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; import static org.hamcrest.MatcherAssert.assertThat; import static org.hamcrest.Matchers.is; @RunWith(SpringJUnit4ClassRunner.class) @ContextConfiguration public class TestRemoteCallServiceHystrix { @Autowired private RemoteCallService remoteCallService ; @Test public void testRemoteCall() throws Exception{ assertThat(this.remoteCallService.call("test"), is("FALLBACK: test")); assertThat(this.remoteCallService.call("test"), is("FALLBACK: test")); assertThat(this.remoteCallService.call("test"), is("test")); } @Configuration @EnableAspectJAutoProxy public static class SpringConfig { @Bean public HystrixCommandAspect hystrixCommandAspect() { return new HystrixCommandAspect(); } @Bean public RemoteCallService remoteCallService() { return new DummyRemoteCallService(); } } }
Spring-Cloud provides an easier way to configure the Netflix libraries for Spring-Boot based projects and if I were to use this library the test would transform to this, a bunch of configuration is now commented out with the help of Spring-Boot:
package hystrixtest; import org.junit.Test; import org.junit.runner.RunWith; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.boot.autoconfigure.EnableAutoConfiguration; import org.springframework.boot.test.SpringApplicationConfiguration; import org.springframework.cloud.netflix.hystrix.EnableHystrix; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; import static org.hamcrest.MatcherAssert.assertThat; import static org.hamcrest.Matchers.is; @RunWith(SpringJUnit4ClassRunner.class) @SpringApplicationConfiguration public class TestRemoteCallServiceHystrix { @Autowired private RemoteCallService remoteCallService; @Test public void testRemoteCall() throws Exception { assertThat(this.remoteCallService.call("test"), is("FALLBACK: test")); assertThat(this.remoteCallService.call("test"), is("FALLBACK: test")); assertThat(this.remoteCallService.call("test"), is("test")); } @Configuration @EnableAutoConfiguration // @EnableAspectJAutoProxy @EnableHystrix public static class SpringConfig { // @Bean // public HystrixCommandAspect hystrixCommandAspect() { // return new HystrixCommandAspect(); // } @Bean public RemoteCallService remoteCallService() { return new DummyRemoteCallService(); } } }
If you are interested in exploring this sample further, here is the github repo with the working tests.
Hey Biju, what's the difference between the Hystrix and the spring-retry project? are they somewhat overlapping?
ReplyDelete