Sunday, September 23, 2012

Spring Testing Support and Context caching

Spring provides a comprehensive support for unit and integration testing - through annotations to load up a Spring application context, integrate with unit testing frameworks like JUnit and TestNG. Since loading up a large application context for every test takes time, Spring intelligently caches the application context for a test suite - typically when we execute tests for a project, say through ant or maven, a suite is created encompassing all the tests in the project.

There are a few points to note with caching which is what I intend to cover here, this is not likely to be comprehensive but is based on some situations which I have encountered:

1. Caching is based on the locations of Spring application context files

Consider a sample Spring configuration file:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<beans xmlns="http://www.springframework.org/schema/beans"
 xmlns:context="http://www.springframework.org/schema/context"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xmlns:p="http://www.springframework.org/schema/p"
 xsi:schemaLocation="
  http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
  http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd">
 
 <bean id="user1"  class="org.bk.lmt.domain.TaskUser" p:username="user1" p:fullname="testUser1" />
 <bean name="user2" class="org.bk.lmt.domain.TaskUser" p:username="user2" p:fullname="testUser" />
 
 <bean class="org.bk.contextcaching.DelayBean"/>
 
</beans>


And a sample test to load up this context file and verify something.:

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(locations = { "contexttest.xml" })
public class Test1 {
 @Autowired Map<String, TaskUser> usersMap;

 @Test
 public void testGetAUser() {
  TaskUser user = usersMap.get("user1");
  assertThat(user.getFullname(), is("testUser1"));
 }
}

I have deliberately added in an additional bean(DelayBean) which takes about 2 seconds to instantiate, to simulate Spring Application Contexts which are slow to load up.

If I now run a small test suite with two tests, both using the same application context, the behavior is that the first test takes about 2 seconds to run through, but the second test runs through quickly because of context caching.

If there were a third test using a different application context, this test would again take time to run through as the new application context has to be loaded up:

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(locations = { "contexttest2.xml" })
public class Test3 {
...
}



2. Caching of application contexts respects the active profile under which the test is run - essentially the profile is also part of the internal key that Spring uses to cache the context, so if two tests use the exact same application context, but different profiles are active for each of the tests, then the cached application context will not be used for the second test:
@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(locations = { "contexttest.xml" })
@ActiveProfiles("dev1")
public class Test1 {
....


@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(locations = { "contexttest.xml" })
@ActiveProfiles("dev2")
public class Test2 {
....




3. Caching of application context applies even with the new @Configuration style of defining a application context and using it in tests:

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(classes={TestConfiguration.class})
public class Test1 {
...


@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(classes={TestConfiguration.class})
public class Test2 {
....



One implication of caching is that if a test class modifies the state of a bean, then another class in the test suite which uses the cached application context will end up seeing the modified bean instead of the bean the way it was defined in the application context:

For eg. consider two tests, both of which modify a bean in the context, but are asserting on a state the way it is defined in the application context - Here one of the tests would end up failing(based on the order in which Junit executes the tests):

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(classes={TestConfiguration.class})
public class Test1 {
 @Autowired Map<String, TaskUser> usersMap;


 @Test
 public void testGetAUser1() {
  TaskUser user = usersMap.get("user1");
  assertThat(user.getFullname(), is("testUser1"));
  user.setFullname("New Name");
 }
 
 @Test
 public void testGetAUser2() {
  TaskUser user = usersMap.get("user1");
  assertThat(user.getFullname(), is("testUser1"));
  user.setFullname("New Name");
 }
}


The fix is to instruct Spring test support that the application context is now dirty and needs to be reloaded for other tests, and this is done with @DirtiesContext annotation which can specified at the test class level or test method level.

@Test
@DirtiesContext
public void testGetAUser2() {
...



Tuesday, September 11, 2012

Java Annotations - Retention

Consider a Java annotation:


public @interface AnAnnotaton {

}

A class with this annotation applied on it:

@AnAnnotaton
class AnAnnotatedClass{
 
}

And a test which checks if this annotation is present on a class:

import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.Matchers.is;

import java.lang.annotation.Annotation;
import org.junit.Test;

public class AnAnnotationTest {
 
 
 @Test
 public void testAnAnnotation() throws Exception {
  AnAnnotatedClass anAnnotatedClass = new AnAnnotatedClass();
  Annotation[] annotationsOnClass = anAnnotatedClass.getClass().getAnnotations();
  assertThat(annotationsOnClass.length, is(1));
 }

}

Sounds reasonable right, one would expect the above test to pass since the class does have an annotation of AnAnnotation on it.

But, this test fails, and the reason is...


a missing meta annotation(@Retention) on the annotation which indicates how long the annotation is to be retained, if the annotation above is changed as follows, the test would work as expected.

@Retention(RetentionPolicy.RUNTIME)
public @interface AnAnnotaton {

}


So what does @Retention do - to quote from the Javadoc:


Indicates how long annotations with the annotated type are to be retained. If no Retention annotation is present on an annotation type declaration, the retention policy defaults to RetentionPolicy.CLASS

there are three different retention policies:
1. SOURCE - where the annotations are removed by the compiler
2. CLASS - Annotations are present in the bytecode, but are not present at runtime and hence not available when trying to reflectively determine if a class has the annotation.
3. RUNTIME - Annotations are retained in the byte code and is available at runtime and hence can be reflectively found on a class.

This is the reason why when the annotation definition was changed to include @Retention(RetentionPolicy.RUNTIME), the test now runs through.

Something basic, but easy to miss out.

Monday, September 3, 2012

Ways to wire dependencies for an object outside of a Spring Container

There are a few interesting ways of setting the properties and dependencies of an object instantiated outside of a Spring container.

Use Cases

To start with, why would we need to do inject in dependencies outside of a Spring container - I am aware of three use cases where I have instantiated objects outside of the Spring container and needed to inject in dependencies.

Consider first the case of a series of tasks executed using a Spring TaskExecutor, the tasks highlighted below are instantiated outside of a Spring container:

List<Callable<ReportPart>> tasks = new ArrayList<Callable<ReportPart>>();
        List<ReportRequestPart> reportRequestParts = reportRequest.getRequestParts();
        for (ReportRequestPart reportRequestPart : reportRequestParts) {
            tasks.add(new ReportPartRequestCallable(reportRequestPart, reportPartGenerator));
        }

        List<Future<ReportPart>> responseForReportPartList;
        List<ReportPart> reportParts = new ArrayList<ReportPart>();
        try {
            responseForReportPartList = executors.invokeAll(tasks);
            for (Future<ReportPart> reportPartFuture : responseForReportPartList) {
                reportParts.add(reportPartFuture.get());
            }

        } catch (Exception e) {
            logger.error(e.getMessage(), e);
            throw new RuntimeException(e);
        }

public class ReportPartRequestCallable implements Callable<ReportPart> {
 private final ReportRequestPart reportRequestPart;
 private final ReportPartGenerator reportPartGenerator;

 public ReportPartRequestCallable(ReportRequestPart reportRequestPart, ReportPartGenerator reportPartGenerator) {
     this.reportRequestPart = reportRequestPart;
     this.reportPartGenerator = reportPartGenerator;
    }

 @Override
    public ReportPart call() {
    return this.reportPartGenerator.generateReportPart(reportRequestPart);
    } 
}

The second use case is with ActiveRecord pattern say with the samples that come with Spring Roo, consider the following method where a Pet class needs to persist itself and needs an entity manager to do this:
@Transactional
    public void Pet.persist() {
        if (this.entityManager == null) this.entityManager = entityManager();
        this.entityManager.persist(this);
    }

The third use case is for a tag library which is instantiated by a web container, but needs some dependencies from Spring.


Solutions

1. The first approach is actually simple, to provide the dependencies at the point of object instantiation, through constructors or setters. This is what I have used with the first use case where the task has two dependencies which are being provided by the service instantiating the task:

tasks.add(new ReportPartRequestCallable(reportRequestPart, reportPartGenerator));


2. The second approach is a to create a factory that is aware of the Spring container, declaring the beans that are required with a prototype scope within the container and getting the beans by a getBeans method of the application context,

Declaring the bean as a prototype scoped bean:
    <bean name="reportPartRequestCallable" class="org.bk.sisample.taskexecutor.ReportPartRequestCallable" scope="prototype">
     <property name="reportPartGenerator" ref="reportPartGenerator"></property>
    </bean>
    
    <bean name="reportPartRequestCallableFactory" class="org.bk.sisample.taskexecutor.ReportPartRequestCallableFactory"/>

and the factory serving out the bean:
public class ReportPartRequestCallableFactory implements ApplicationContextAware{
 private ApplicationContext applicationContext;

 @Override
 public void setApplicationContext(ApplicationContext applicationContext) throws BeansException {
  this.applicationContext = applicationContext;
 }
 
 public ReportPartRequestCallable getReportPartRequestCallable(){
  return this.applicationContext.getBean("reportPartRequestCallable", ReportPartRequestCallable.class);
 }
}


3.  The third approach is a variation of the above approach is to instantiate the bean and then inject dependencies using AutoWireCapableBeanFactory.autowireBean(instance), this way:
public class ReportPartRequestCallableFactory implements ApplicationContextAware{
 private GenericApplicationContext applicationContext;

 @Override
 public void setApplicationContext(ApplicationContext applicationContext) throws BeansException {
  this.applicationContext = (GenericApplicationContext)applicationContext;
 }
 
 public ReportPartRequestCallable getReportPartRequestCallable(){
  ReportPartRequestCallable reportPartRequestCallable = new ReportPartRequestCallable();
  applicationContext.getBeanFactory().autowireBean(reportPartRequestCallable);
  return reportPartRequestCallable;
 }
}



4.  The fourth approach is using @Configurable, the catch though is that it requires AspectJ to work. Spring essentially enhances the constructor of the class to inject in the dependencies along the lines of what is being explicitly done in the third approach above:

import org.springframework.beans.factory.annotation.Configurable;

@Configurable("reportPartRequestCallable")
public class ReportPartRequestCallable implements Callable<ReportPart> {
    private ReportRequestPart reportRequestPart;
    @Autowired private ReportPartGenerator reportPartGenerator;

    public ReportPartRequestCallable() {
    }

    @Override
    public ReportPart call() {
       return this.reportPartGenerator.generateReportPart(reportRequestPart);
    }

    public void setReportRequestPart(ReportRequestPart reportRequestPart) {
        this.reportRequestPart = reportRequestPart;
    }

    public void setReportPartGenerator(ReportPartGenerator reportPartGenerator) {
        this.reportPartGenerator = reportPartGenerator;
    }
}

The following is also required to configure the Aspect responsible for @Configurable weaving:
<context:spring-configured/>

With these changes in place, any dependency for a class annotated with @Configurable is handled by Spring even if the construction is done completely outside of the container:

    @Override
    public Report generateReport(ReportRequest reportRequest) {
        List<Callable<ReportPart>> tasks = new ArrayList<Callable<ReportPart>>();
        List<ReportRequestPart> reportRequestParts = reportRequest.getRequestParts();
        for (ReportRequestPart reportRequestPart : reportRequestParts) {
            ReportPartRequestCallable reportPartRequestCallable = new ReportPartRequestCallable(); 
            reportPartRequestCallable.setReportRequestPart(reportRequestPart);
            tasks.add(reportPartRequestCallable);
        }
    .......


Conclusion
All of the above approaches are effective with injecting in dependencies in objects which are instantiated outside of a container. I personally prefer to use Approach 4 (using @Configurable) in cases where AspectJ support is available, else I would go with Approach 2(hiding behind a factory and using a prototype bean).

Saturday, August 25, 2012

Spring Scoped Proxy

Consider two Spring beans defined this way:

@Component
class SingletonScopedBean{
 @Autowired private PrototypeScopedBean prototypeScopedBean;
 
 public String getState(){
  return this.prototypeScopedBean.getState();
 }
}

@Component
@Scope(value="prototype")
class PrototypeScopedBean{
 private final String state;
 
 public PrototypeScopedBean(){
  this.state = UUID.randomUUID().toString();
 }

 public String getState() {
  return state;
 }
}

Here a prototype scoped bean is injected into a Singleton scoped bean.

Now, consider this test using these beans:

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration
public class ScopedProxyTest {
 
 @Autowired private SingletonScopedBean singletonScopedBean;
 
 @Test
 public void testScopedProxy() {
  assertThat(singletonScopedBean.getState(), not(equalTo(singletonScopedBean.getState())));
 }
 
 @Configuration
 @ComponentScan("org.bk.samples.scopedproxy")
 public static class SpringContext{}

}

The point to note is that there is only 1 instance of PrototypeScopedBean that is created here - and that 1 instance is injected into the SingletonScopedBean, so the above test which actually expects a new instance of PrototypeScopedBean with each invocation of getState() method will fail.

If a new instance is desired with every request to PrototypeScopedBean (and in general if a bean with longer scope has a bean with shorter scope as a dependency, and the shorter scope needs to be respected), then there are a few solutions:

1. Lookup method injection - which can be read about here
2. A better solution is using Scoped proxies -

A scoped proxy can be specified this way using @Configuration:
@Component
@Scope(value="prototype", proxyMode=ScopedProxyMode.TARGET_CLASS)
class PrototypeScopedBean{
 private final String state;
 
 public PrototypeScopedBean(){
  this.state = UUID.randomUUID().toString();
 }

 public String getState() {
  return state;
 }

}

With this change, the bean injected into the SingletonScopedBean is not the PrototypeScopedBean itself, but a proxy to the bean (created using CGLIB or Dynamic proxies) and this proxy understands the scope and returns instances based on the requirements of the scope, the test should now work as expected.

Saturday, August 18, 2012

@ContextConfiguration defaults

Spring @ContextConfiguration is a way to specify the Application Context for a test.

The location of a xml based test application context can be specified using the locations attribute:
@ContextConfiguration(locations={"test-context.xml"})

and if @Configuration is used as the context, then a classes attibute can be specified:

@ContextConfiguration(classes={TestConfiguration.class})

There are intelligent defaults for these attributes though and that is what I wanted to highlight in this post.

If the locations or the classes attribute is not specified, the default behavior is to first look for a xml configuration with a name as the test class name - "context.xml" file

For eg. if I have a Test class this way:
@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration
public class TestSpringCache {


The location of the configuration that will be tried first is "TestSpringCache-context.xml"

If a context is not found at this location, then a @Configuration default is looked for by scanning all static inner classes annotated with @Configuration of the test class. So if I had the following:


@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration
public class TestSpringCache {
   ...
 
 @Configuration
 @EnableCaching
 @ComponentScan("org.bk.samples.cache")
 public static class TestConfiguration{
 .. 


the inner class TestConfiguration would be used as the source of the Application context.

The default behaviour can be changed by supplying a loader attribute to the @ContextConfiguration, say for eg, if I want to default to @Configuration with its defaults, it can done this way:

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(loader=AnnotationConfigContextLoader.class)
public class TestSpringCache {
..

So to conclude, the defaults provided by @Configuration are a great way to make the tests a little more concise!

Friday, August 10, 2012

Spring @Configuration and FactoryBean

Consider a FactoryBean  for defining a cache using a Spring configuration file:

 <cache:annotation-driven />
 <context:component-scan base-package="org.bk.samples.cachexml"></context:component-scan>
 
 <bean id="cacheManager" class="org.springframework.cache.support.SimpleCacheManager">
  <property name="caches">
   <set>
    <ref bean="defaultCache"/>
   </set>
  </property>
 </bean>
 
 <bean name="defaultCache" class="org.springframework.cache.concurrent.ConcurrentMapCacheFactoryBean">
  <property name="name" value="default"/>
 </bean>

The factory bean ConcurrentMapCacheFactoryBean is a bean which is in turn responsible for creating a Cache bean.

My first attempt at translating this setup to a @Configuration style was the following:


@Bean
public SimpleCacheManager cacheManager(){
 SimpleCacheManager cacheManager = new SimpleCacheManager();
 List<Cache> caches = new ArrayList<Cache>();
 ConcurrentMapCacheFactoryBean cacheFactoryBean = new ConcurrentMapCacheFactoryBean();
 cacheFactoryBean.setName("default");
 caches.add(cacheFactoryBean.getObject());
 cacheManager.setCaches(caches );
 return cacheManager;
}

This did not work however, the reason is that here I have bypassed some Spring bean lifecycle mechanisms altogether. It turns out that ConcurrentMapCacheFactoryBean also implements the InitializingBean interface and does a eager initialization of the cache in the "afterPropertiesSet" method of InitializingBean. Now by directly calling factoryBean.getObject() , I was completely bypassing the afterPropertiesSet method.

There are two possible solutions:
1. Define the FactoryBean the same way it is defined in the XML:
@Bean
public SimpleCacheManager cacheManager(){
 SimpleCacheManager cacheManager = new SimpleCacheManager();
 List<Cache> caches = new ArrayList<Cache>();
 caches.add(cacheBean().getObject());
 cacheManager.setCaches(caches );
 return cacheManager;
}

@Bean
public ConcurrentMapCacheFactoryBean cacheBean(){
 ConcurrentMapCacheFactoryBean cacheFactoryBean = new ConcurrentMapCacheFactoryBean();
 cacheFactoryBean.setName("default");
 return cacheFactoryBean;
}
In this case, there is an explicit FactoryBean being returned from a @Bean method, and Spring will take care of calling the lifecycle methods on this bean.

2. Replicate the behavior in the relevant lifecycle methods, in this specific instance I know that the FactoryBean instantiates the ConcurrentMapCache in the afterPropertiesSet method, I can replicate this behavior directly this way:

@Bean
public SimpleCacheManager cacheManager(){
 SimpleCacheManager cacheManager = new SimpleCacheManager();
 List<Cache> caches = new ArrayList<Cache>();
 caches.add(cacheBean());
 cacheManager.setCaches(caches );
 return cacheManager;
}

@Bean
public Cache  cacheBean(){
 Cache  cache = new ConcurrentMapCache("default");
 return cache;
}

Something to keep in mind when translating a FactoryBean from xml to @Configuration.

Note:
A working one page test as a gist is available here:

Tuesday, August 7, 2012

Mergesort using Fork/Join Framework

The objective of this entry is to show a simple example of a Fork/Join RecursiveAction, not to delve too much into the possible optimizations to merge sort or the relative advantages of using Fork/Join Pool over the existing Java 6 based implementations like ExecutorService.

The following is a typical implementation of a Top Down Merge sort algorithm using Java:

import java.lang.reflect.Array;

public class MergeSort {
 public static <T extends Comparable<? super T>> void sort(T[] a) {
  @SuppressWarnings("unchecked")
  T[] helper = (T[])Array.newInstance(a[0].getClass() , a.length);
  mergesort(a, helper, 0, a.length-1);
 }
 
 private static <T extends Comparable<? super T>> void mergesort(T[] a, T[] helper, int lo, int hi){
  if (lo>=hi) return;
  int mid = lo + (hi-lo)/2;
  mergesort(a, helper, lo, mid);
  mergesort(a, helper, mid+1, hi);
  merge(a, helper, lo, mid, hi);  
 }

 private static <T extends Comparable<? super T>> void merge(T[] a, T[] helper, int lo, int mid, int hi){
  for (int i=lo;i<=hi;i++){
   helper[i]=a[i];
  }
  int i=lo,j=mid+1;
  for(int k=lo;k<=hi;k++){
   if (i>mid){
    a[k]=helper[j++];
   }else if (j>hi){
    a[k]=helper[i++];
   }else if(isLess(helper[i], helper[j])){
    a[k]=helper[i++];
   }else{
    a[k]=helper[j++];
   }
  }
 }

 private static <T extends Comparable<? super T>> boolean isLess(T a, T b) {
  return a.compareTo(b) < 0;
 }
}

To quickly describe the algorithm -
The following steps are performed recursively:

  1.  The input data is divided into 2 halves
  2.  Each half is sorted
  3. The sorted data is then merged

Merge sort is a canonical example for an implementation using Java Fork/Join pool, and the following is a blind implementation of Merge sort using the Fork/Join framework:

The  recursive task in Merge sort can be succinctly expressed as an implementation of RecursiveAction -


 private static class MergeSortTask<T extends Comparable<? super T>> extends RecursiveAction{
  private static final long serialVersionUID = -749935388568367268L;
  private final T[] a;
  private final T[] helper;
  private final int lo;
  private final int hi;
  
  public MergeSortTask(T[] a, T[] helper, int lo, int hi){
   this.a = a;
   this.helper = helper;
   this.lo = lo;
   this.hi = hi;
  }
  @Override
  protected void compute() {
   if (lo>=hi) return;
   int mid = lo + (hi-lo)/2;
   MergeSortTask<T> left = new MergeSortTask<>(a, helper, lo, mid);
   MergeSortTask<T> right = new MergeSortTask<>(a, helper, mid+1, hi);
   invokeAll(left, right);
   merge(this.a, this.helper, this.lo, mid, this.hi);
   
   
  }
  private void merge(T[] a, T[] helper, int lo, int mid, int hi){
   for (int i=lo;i<=hi;i++){
    helper[i]=a[i];
   }
   int i=lo,j=mid+1;
   for(int k=lo;k<=hi;k++){
    if (i>mid){
     a[k]=helper[j++];
    }else if (j>hi){
     a[k]=helper[i++];
    }else if(isLess(helper[i], helper[j])){
     a[k]=helper[i++];
    }else{
     a[k]=helper[j++];
    }
   }
  }
  private boolean isLess(T a, T b) {
   return a.compareTo(b) < 0;
  }
 }

MergeSortTask above implements a compute method, which takes in a array of values, split it up into two parts, creates a MergeSortTask out of each of the parts and forks off two more tasks(hence it is called RecursiveAction!). The specific API used here to spawn of the task is invokeAll which returns only when the submitted subtasks are marked as completed. So once the left and right subtasks return the result is merged in a merge routine.

Given this the only work left is to use a ForkJoinPool to submit this task. ForkJoinPool is analogous to the ExecutorService used for distributing tasks in a threadpool, the difference to quote the ForkJoinPool's API docs:

A ForkJoinPool differs from other kinds of ExecutorService mainly by virtue of employing work-stealing: all threads in the pool attempt to find and execute subtasks created by other active tasks (eventually blocking waiting for work if none exist)

This is how the task of submitting the task to the Fork/Join Pool looks like:

 public static <T extends Comparable<? super T>> void sort(T[] a) {
  @SuppressWarnings("unchecked")
  T[] helper = (T[])Array.newInstance(a[0].getClass() , a.length);
  ForkJoinPool forkJoinPool = new ForkJoinPool(10);
  forkJoinPool.invoke(new MergeSortTask<T>(a, helper, 0, a.length-1));
 }

A complete sample is also available here: https://github.com/bijukunjummen/algos/blob/master/src/main/java/org/bk/algo/sort/algo04/merge/MergeSortForkJoin.java