Polling an http end point using Spring Integration

It is a little non-intuitive if you want to write a flow with Spring Integration which polls an http end point and gathers some content from the http end point for further processing.

Spring Integration provides a couple of ways for integrating with a HTTP endpoint -

1. Http Outbound adapter - to send the messages to an http endpoint
2. Http Outbound gateway - to send messages to an http endpoint and to collect the response as a message

My first instinct to poll the http endpoint was to use a Http Inbound channel adapter, the wrong assumption that I made was that the adapter will be responsible for getting the information from an endpoint - what Http Inbound Gateway actually does is to expose an Http endpoint and wait for requests to come in! , this is why I started by saying that it was a little non-intuitive to me that to poll a URL and collect content from it I will actually have to use a Http Outbound gateway

With this clarified, consider an example where I want to poll the USGS Earth Quake information feed available at this url - http://earthquake.usgs.gov/earthquakes/feed/geojson/all/hour

This is how my sample http Outbound component looks like:

<int:channel id="quakeinfo.channel">
  <int:queue capacity="10"/>
 </int:channel>

 <int:channel id="quakeinfotrigger.channel"></int:channel> 

 <int-http:outbound-gateway id="quakerHttpGateway"
     request-channel="quakeinfotrigger.channel"
     url="http://earthquake.usgs.gov/earthquakes/feed/geojson/all/hour"
     http-method="GET"
     expected-response-type="java.lang.String"
     charset="UTF-8"
     reply-timeout="5000"
     reply-channel="quakeinfo.channel">     
 </int-http:outbound-gateway>

Here the http outbound gateway waits for messages to come into the quakeinfotrigger channel, sends out a GET request to the "http://earthquake.usgs.gov/earthquakes/feed/geojson/all/hour" url, and places the response json string into the "quakeinfo.channel" channel

Testing this is easy:

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration("httpgateway.xml")
public class TestHttpOutboundGateway {
 
 @Autowired @Qualifier("quakeinfo.channel") PollableChannel quakeinfoChannel;
 @Autowired @Qualifier("quakeinfotrigger.channel") MessageChannel quakeinfoTriggerChannel;

 @Test
 public void testHttpOutbound() {
  quakeinfoTriggerChannel.send(MessageBuilder.withPayload("").build());
  Message<?> message = quakeinfoChannel.receive();
  assertThat(message.getPayload(), is(notNullValue()));
 }

}

What I am doing here is getting a reference to the channel which triggers the outbound gateway to send a message to the http endpoint and reference to another channel where the response from the http endpoint is placed. I am triggering the test flow by placing a dummy empty message in the trigger channel and then waiting on message to be available on the response channel and asserting on the contents.

This works cleanly, however my original intent was to write a poller which would trigger polling of this endpoint once every minute or so, to do this what I have to do is essentially place a dummy message into the "quakeinfotrigger.channel" channel every minute and this is easily accomplished using a Spring Integration "poller" and a bit of Spring Expression language:

<int:inbound-channel-adapter channel="quakeinfotrigger.channel" expression="''">
 <int:poller fixed-delay="60000"></int:poller>
</int:inbound-channel-adapter>

Here I have a Spring inbound-channel-adapter triggered attached to a poller , with the poller triggering an empty message every minute.

All this looks a little convoluted but works nicely - here is a gist with a working code

References:

1. Based on a question I had posed at the Spring forum http://forum.springsource.org/showthread.php?130711-Need-help-with-polling-to-a-json-based-HTTP-service