Thursday, December 27, 2012

Spring Data JPA and pagination

Let us start with the classic JPA way to support pagination.

Consider a simple domain class - A "Member" with attributes first name, last name. To support pagination on a list of members, the JPA way is to support a finder which takes in the offset of the first result(firstResult) and the size of the result(maxResults) to retrieve, this way:

import java.util.List;

import javax.persistence.TypedQuery;

import org.springframework.stereotype.Repository;

import mvcsample.domain.Member;

@Repository
public class JpaMemberDao extends JpaDao<Long, Member> implements MemberDao{

 public JpaMemberDao(){
  super(Member.class);
 }
 @Override
 public List<Member> findAll(int firstResult, int maxResults) {
  TypedQuery<Member> query = this.entityManager.createQuery("select m from Member m", Member.class);
  return query.setFirstResult(firstResult).setMaxResults(maxResults).getResultList();
 }

 @Override
 public Long countMembers() {
  TypedQuery<Long> query = this.entityManager.createQuery("select count(m) from Member m", Long.class);
  return query.getSingleResult();
 }
}

An additional API which returns the count of the records is needed to determine the number of pages for the list of entity, as shown above.

Given this API, two parameters are typically required from the UI:


  • the current page being displayed (say "page.page")
  • the size of list per page (say "page.size")
The controller will be responsible for transforming these inputs to the one required by the JPA - firstResult and maxResults this way:

@RequestMapping(produces="text/html")
public String list(@RequestParam(defaultValue="1", value="page.page", required=false) Integer page, 
   @RequestParam(defaultValue="10", value="page.size", required=false) Integer size, Model model){
 int firstResult = (page==null)?0:(page-1) * size;
 model.addAttribute("members",this.memberDao.findAll(firstResult, size));
 float nrOfPages = (float)this.memberDao.countMembers()/size;
 int maxPages = (int)( ((nrOfPages>(int)nrOfPages) || nrOfPages==0.0)?nrOfPages+1:nrOfPages);
 model.addAttribute("maxPages", maxPages);
 return "members/list";
}

Given a list as a model attribute and the count of all pages(maxPages above), the list can be transformed to a simple table in a jsp, there is a nice tag library that is packaged with Spring Roo which can be used to present the pagination element in a jsp page, I have included it with the reference.



So this is the approach to pagination using JPA and Spring MVC.

Spring-Data-JPA makes this even simpler, first is the repository interface to support retrieving a paginated list - in its simplest form the repository simply requires extending Spring-Data-JPA interfaces and at runtime generates the proxies which implements the real JPA calls:

import mvcsample.domain.Member;

import org.springframework.data.jpa.repository.JpaRepository;
import org.springframework.stereotype.Repository;

public interface MemberRepository extends JpaRepository<Member, Long>{
 //
}

Given this, the controller method which accesses the repository interface is also very simple:

@RequestMapping(produces="text/html")
public String list(Pageable pageable, Model model){
 Page<Member> members = this.memberRepository.findAll(pageable);
    model.addAttribute("members", members.getContent());
    float nrOfPages = members.getTotalPages();
    model.addAttribute("maxPages", nrOfPages);
 return "members/list";
}

The controller method accepts a parameter called Pageable, this parameter is populated using a Spring MVC HandlerMethodArgumentResolver that looks for request parameters by name "page.page" and "page.size" and converts them into the Pageable argument. This custom HandlerMethodArgumentResolver is registered with Spring MVC this way:

<mvc:annotation-driven>
 <mvc:argument-resolvers>
  <bean class="org.springframework.data.web.PageableArgumentResolver"></bean>
 </mvc:argument-resolvers>
</mvc:annotation-driven> 

the JpaRepository API takes in the pageable argument and returns a page, internally automatically populating the count of pages also which can retrieved from the Page methods.

If the queries need to be explicitly specified then this can be done in a number of ways, one of which is the following:

@Query(value="select m from Member m", countQuery="select count(m) from Member m")
Page<Member> findMembers(Pageable pageable);


One catch which I could see is that that pageable's page number is 0 indexed, whereas the one passed from the UI is 1 indexed, however the PageableArgumentResolver internally handles and converts the 1 indexed UI page parameter to the required 0 indexed value.

Spring Data JPA thus makes it really simple to implement a paginated list page.

I am including a sample project which ties all this together, along with the pagination tag library which makes it simple to show the paginated list.

Reference:
A sample projects which implements a paginated list is available here : https://github.com/bijukunjummen/spring-mvc-test-sample.git

Spring-Data-JPA reference: http://static.springsource.org/spring-data/data-jpa/docs/current/reference/html/




Sunday, December 23, 2012

context:component-scan, context:annotation-config and list of beans registered

A <context:component-scan/> custom tag registers the same set of bean definitions as is done by <context:annotation-config/>, apart from its primary responsibility of scanning the java packages and registering bean definitions from the classpath.

The following bean types are registered by either of these custom tags:


Bean Purpose
ConfigurationClassPostProcessor
A bean factory post processor used for bootstrapping classes annotated with @Configuration
AutowiredAnnotationBeanPostProcessor
A bean post processor which processes fields, setter methods, arbitrary methods annotated with @AutoWired, @Value and supports JSR-330 @Inject annotation
RequiredAnnotationBeanPostProcessor
A bean post processor which enforces that the required bean properties have been set - by looking for fields annotated with @Required annotation
CommonAnnotationBeanPostProcessor
A bean post processor which supports JSR-250 annotations - eg. @Resource
PersistenceAnnotationBeanPostProcessor
A bean post processor that processes @PersistenceUnit and @PersistenceContext annotations for injecting in a EntityManagerFactory and EntityManager respectively.

This is the reason why if component-scan is present, annotation-config can be kept out.

If for some reason this registration of default bean definitions are to be avoided, the way to do that is to specify an additional "annotation-config" attribute in component-scan, this way:



<context:component-scan basePackages=".." annotation-config="false"/>

Reference:
Spring Reference site

Sunday, December 9, 2012

Composing Java annotations

The allowed attribute types of a Java annotations are deliberately very restrictive, however some clean composite annotation types are possible with the allowed types.

Consider a sample annotation from the tutorial site:

package annotation;
@interface ClassPreamble {
   String author();
   String[] reviewers();
}

Here the author and reviewers are of String and array types which is in keeping with the allowed types of annotation attributes. The following is a comprehensive list of allowed types(as of Java 7):


  • String
  • Class
  • any parameterized invocation of Class
  • an enum type
  • an annotation type, do note that cycles are not allowed, the annotated type cannot refer to itself
  • an array type whose element type is one of the preceding types.

Now, to make a richer ClassPreable consider two more annotation types defined this way:

package annotation;

public @interface Author {
 String first() default "";
 String last() default "";
}


package annotation;

public @interface Reviewer {
 String first() default "";
 String last() default "";
}


With these, the ClassPreamble can be composed from the richer Author and Reviewer annotation types, this way:

package annotation;
@interface ClassPreamble {
   Author author();
   Reviewer[] reviewers();
}

Now an annotation applied on a class looks like this:

package annotation;

@ClassPreamble(author = @Author(first = "John", last = "Doe")
    , reviewers = {@Reviewer(first = "first1", last = "last1"), @Reviewer(last = "last2") }
)
public class MyClass {
....
}

This is a contrived example just to demonstrate composition of annotations, however this approach is used extensively for real world annotations, for eg, to define a many to many relationship between two JPA entities:

    @ManyToMany
    @JoinTable(name="Employee_Project",
          joinColumns=@JoinColumn(name="Employee_ID"),
          inverseJoinColumns=@JoinColumn(name="Project_ID"))
    private Collection<Project> projects;

Tuesday, November 27, 2012

Spring test mvc with Spring 3.2RC1

I had an opportunity earlier to try out Spring-test-mvc and was very impressed with how easy it now makes integration testing of the Spring MVC controllers.

Spring-test-mvc is now packaged with Spring-test module as of Spring 3.2RC1 and has made a few changes which is what I wanted to record for myself here:

First is to include the Spring-test-mvc module, the following is the dependency now that it is packaged with spring-test module:

<dependency>
 <groupId>org.springframework</groupId>
 <artifactId>spring-test</artifactId>
 <version>3.2.0.RC1</version>
 <scope>test</scope>
</dependency>

Now on the test side the first change is to include a @WebAppConfiguration annotation:

@WebAppConfiguration
@ContextConfiguration(locations={"file:src/main/webapp/WEB-INF/spring/mvc-config.xml", "classpath:/META-INF/spring/applicationContext.xml", "classpath:/META-INF/spring/applicationContext-jpa.xml"})
@RunWith(SpringJUnit4ClassRunner.class)
public class MemberJsonControllerTest {
 
 @Autowired
 private WebApplicationContext wac;




This new annotation instructs Spring test support to load up a WebApplicationContext - a mock ServletContext is automatically assigned to this WebApplicationContext.

Another change from previous version of Spring-test-mvc is the updated package names, the following are the new static imports:

import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.*;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.*;
import static org.springframework.test.web.servlet.setup.MockMvcBuilders.*;

Now that a WebApplicationContext is available in the scope of the test, the next step is to get hold of MockMvc which will drive all the controller related tests:

private MockMvc mockMvc;

@Before
public void setup() {
 this.mockMvc = webAppContextSetup(this.wac).build();
}

Once a MockMvc instance is available all the controller tests can be driven using this, consider the following test where I have a series of CRUD actions and tests on the results being performed on a controller:

@Test
public void testAControllerFlow() throws Exception {
 this.mockMvc.perform(post("/membersjson").contentType(MediaType.APPLICATION_JSON).content(String.format(createJson, 1,"One","One", 0).getBytes()))
   .andExpect(status().isOk());
 this.mockMvc.perform(post("/membersjson").contentType(MediaType.APPLICATION_JSON).content(String.format(createJson, 2,"Two","Two", 0).getBytes()))
  .andExpect(status().isOk());
 this.mockMvc.perform(post("/membersjson").contentType(MediaType.APPLICATION_JSON).content(String.format(createJson, 3,"Three","Three", 0).getBytes()))
  .andExpect(status().isOk());  
 
 this.mockMvc.perform(get("/membersjson").contentType(MediaType.APPLICATION_JSON))
   .andExpect(status().isOk())
   .andExpect(content().string(containsString(membersJson)));
 
 this.mockMvc.perform(put("/membersjson").contentType(MediaType.APPLICATION_JSON).content(String.format(createJson, 1,"One","OneUpdated", 0).getBytes()))
  .andExpect(status().isOk());
 
 mockMvc.perform(get("/membersjson").contentType(MediaType.APPLICATION_JSON))
  .andExpect(status().isOk())
  .andExpect(jsonPath("$[0].first").value("One"));
 
 mockMvc.perform(get("/membersjson/1").contentType(MediaType.APPLICATION_JSON))
  .andExpect(status().isOk())
  .andExpect(content().string(containsString("OneUpdated")));

}


The tests are very readable thanks to the fluent interfaces and supports matchers on response data based on Hamcrest which I have used above(containsString) and also supports jsonpath for validating json responses.


I have a github project which provides the complete working sample: https://github.com/bijukunjummen/spring-mvc-test-sample.git



Friday, November 16, 2012

Spring Integration Standalone application

Creating a standalone Spring application and by extension a Spring Integration application is very easy. As long as a non-daemon thread is active, the main thread does not terminate, so if the Spring Integration application has some task related or polling related threads active the main task will be alive.

Consider the Spring Integration application that I had outlined in my previous Blog entry:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xmlns:int="http://www.springframework.org/schema/integration"
 xmlns:int-http="http://www.springframework.org/schema/integration/http"
 xsi:schemaLocation="http://www.springframework.org/schema/integration http://www.springframework.org/schema/integration/spring-integration.xsd
  http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
  http://www.springframework.org/schema/integration/http http://www.springframework.org/schema/integration/http/spring-integration-http.xsd">

 <int:inbound-channel-adapter channel="quakeinfotrigger.channel" expression="''">
  <int:poller fixed-delay="60000"></int:poller>
 </int:inbound-channel-adapter>
 
 <int:channel id="quakeinfo.channel">
  <int:queue capacity="10"/>
 </int:channel>

 <int:channel id="quakeinfotrigger.channel"></int:channel> 

 <int-http:outbound-gateway id="quakerHttpGateway"
     request-channel="quakeinfotrigger.channel"
     url="http://earthquake.usgs.gov/earthquakes/feed/geojson/all/hour"
     http-method="GET"
     expected-response-type="java.lang.String"
     charset="UTF-8"
     reply-timeout="5000"
     reply-channel="quakeinfo.channel">     
 </int-http:outbound-gateway>
 
 <int:logging-channel-adapter id="messageLogger" log-full-message="true" channel="quakeinfo.channel" level="ERROR">
  <int:poller fixed-delay="5000" ></int:poller>
 </int:logging-channel-adapter>

</beans>

The above Spring Integration flow polls a http endpoint every minute and prints the contents to System out. Now to write a standalone application out of this, is as simple as the following:
package standalone;

import org.springframework.context.support.ClassPathXmlApplicationContext;

public class Main {

 public static void main(String[] args) {
  ClassPathXmlApplicationContext applicationContext = new ClassPathXmlApplicationContext("classpath:/httpgateway.xml");
  applicationContext.registerShutdownHook();
 }
}

Initialize the ApplicationContext, and register a shutdown hook for orderly shutdown of the context, that's it! Since the sample flow above spawns threads to poll an http endpoint the application stays active with just the above code.

This sample is available at the following github repo:
https://github.com/bijukunjummen/si-standalone-sample

Thursday, November 15, 2012

Polling an http end point using Spring Integration

It is a little non-intuitive if you want to write a flow with Spring Integration which polls an http end point and gathers some content from the http end point for further processing.

Spring Integration provides a couple of ways for integrating with a HTTP endpoint -

1. Http Outbound adapter - to send the messages to an http endpoint
2. Http Outbound gateway - to send messages to an http endpoint and to collect the response as a message

My first instinct to poll the http endpoint was to use a Http Inbound channel adapter, the wrong assumption that I made was that the adapter will be responsible for getting the information from an endpoint - what Http Inbound Gateway actually does is to expose an Http endpoint and wait for requests to come in! , this is why I started by saying that it was a little non-intuitive to me that to poll a URL and collect content from it I will actually have to use a Http Outbound gateway

With this clarified, consider an example where I want to poll the USGS Earth Quake information feed available at this url - http://earthquake.usgs.gov/earthquakes/feed/geojson/all/hour

This is how my sample http Outbound component looks like:

<int:channel id="quakeinfo.channel">
  <int:queue capacity="10"/>
 </int:channel>

 <int:channel id="quakeinfotrigger.channel"></int:channel> 

 <int-http:outbound-gateway id="quakerHttpGateway"
     request-channel="quakeinfotrigger.channel"
     url="http://earthquake.usgs.gov/earthquakes/feed/geojson/all/hour"
     http-method="GET"
     expected-response-type="java.lang.String"
     charset="UTF-8"
     reply-timeout="5000"
     reply-channel="quakeinfo.channel">     
 </int-http:outbound-gateway>

Here the http outbound gateway waits for messages to come into the quakeinfotrigger channel, sends out a GET request to the "http://earthquake.usgs.gov/earthquakes/feed/geojson/all/hour" url, and places the response json string into the "quakeinfo.channel" channel

Testing this is easy:

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration("httpgateway.xml")
public class TestHttpOutboundGateway {
 
 @Autowired @Qualifier("quakeinfo.channel") PollableChannel quakeinfoChannel;
 @Autowired @Qualifier("quakeinfotrigger.channel") MessageChannel quakeinfoTriggerChannel;

 @Test
 public void testHttpOutbound() {
  quakeinfoTriggerChannel.send(MessageBuilder.withPayload("").build());
  Message<?> message = quakeinfoChannel.receive();
  assertThat(message.getPayload(), is(notNullValue()));
 }

}

What I am doing here is getting a reference to the channel which triggers the outbound gateway to send a message to the http endpoint and reference to another channel where the response from the http endpoint is placed. I am triggering the test flow by placing a dummy empty message in the trigger channel and then waiting on message to be available on the response channel and asserting on the contents.

This works cleanly, however my original intent was to write a poller which would trigger polling of this endpoint once every minute or so, to do this what I have to do is essentially place a dummy message into the "quakeinfotrigger.channel" channel every minute and this is easily accomplished using a Spring Integration "poller" and a bit of Spring Expression language:

<int:inbound-channel-adapter channel="quakeinfotrigger.channel" expression="''">
 <int:poller fixed-delay="60000"></int:poller>
</int:inbound-channel-adapter>

Here I have a Spring inbound-channel-adapter triggered attached to a poller , with the poller triggering an empty message every minute.

All this looks a little convoluted but works nicely - here is a gist with a working code

References:

1. Based on a question I had posed at the Spring forum http://forum.springsource.org/showthread.php?130711-Need-help-with-polling-to-a-json-based-HTTP-service


Friday, November 2, 2012

JSP Expression Language and .class

This is something which I faced when using JSP-EL this week, consider the following JSP-EL statement which displays content based on the type of an object:

<c:choose>
 <c:when test="${container.class.simpleName=='TableContainer'}">
  <table>...</table>
 </c:when>
 <c:when test="${container.class.simpleName=='ListContainer'}">
  <ul>...</ul>
 </c:when>
 <c:otherwise>
  <c:out value="Wrong container Type"/>
 </c:otherwise>
</c:choose>



However if this is run on Tomcat 7.0+, you are likely to see a very unhelpful exception, along these lines:

org.apache.jasper.JasperException: /WEB-INF/tags/formlayout/display/displaycontainer.tag (line: 11, column: 1) "${container.class.simpleName=='TableContainer'}" contains invalid expression(s): javax.el.ELException: Failed to parse the expression [${container.class.simpleName=='TableContainer'}]


The actual cause of the issue is the use of the .class above - The expression language parser in Tomcat 7+ by default performs a check to ensure that all identifiers conform to Java Language specifications, since class(which is actually a shortcut for getClass()) is a java Keyword it fails the check for the name of an identifier and hence the error -

There are multiple fixes possible for this issue, the first fix is simply to use .getClass() instead of just .class -


....
 <c:when test="${container.getClass().simpleName=='TableContainer'}">
...
</c:choose>


The second fix is a more sensible one, don't depend on the type of the class, instead introduce a method in the class to describe its type and avoiding Java identifier issue altogether:


....
<c:when test="${container.type=='table'}">
...


Another fix is to soften the identifier check in Tomcat 7+, this can be done using a system property set at Tomcat startup -

org.apache.el.parser. SKIP_IDENTIFIER_CHECK

and is described in more detail here - http://tomcat.apache.org/tomcat-7.0-doc/config/systemprops.html