Json Patch and Json Merge Patch in Java

 Json Patch and Json Merge Patch both do one job well - a way to represent a change to a source json structure.  

Json Patch does it as a series of operations which transforms a source document and Json Merge Patch represents the change as a lite version of the source document.

It is easier to show these as an example, and this is straight from the Json Merge Patch's RFC.

Let's start with a source document:

{
  "title": "Goodbye!",
  "author": {
    "givenName": "John",
    "familyName": "Doe"
  },
  "tags": [
    "example",
    "sample"
  ],
  "content": "This will be unchanged"
}

and the objective is to transform it to this document:

{
  "title": "Hello!",
  "author": {
    "givenName": "John"
  },
  "tags": [
    "example"
  ],
  "content": "This will be unchanged",
  "phoneNumber": "+01-123-456-7890"
}

Which may be easier to visualize in a diff view:

The consolidated set of changes are:

1. The title is being changed
2. Author/familyName is removed
3. One of the tags is removed
4. A phone number is added

Json Patch

This change can be represented the following way using Json Patch document:

[
  { "op": "replace", "path": "/title", "value": "Hello!"},
  { "op": "remove", "path": "/author/familyName"},
  { "op": "add", "path": "/phoneNumber", "value": "+01-123-456-7890"},
  { "op": "replace", "path": "/tags", "value": ["example"]}
]

A series of operations transforms the source document into the target document. An operation can be one of "add", "remove", "replace", "move", "copy" or "test" and in the example exactly matches the diff.

 

Json Merge Patch

A Json merge patch for the change looks like this:

{
  "title": "Hello!",
  "author": {
    "familyName": null
  },
  "phoneNumber": "+01-123-456-7890",
  "tags": [
    "example"
  ]
}   

There is a little bit of interpretation required on how the change gets applied, it is very intuitive though: 1. The presence of "title" with a new value indicates that the title needs to be changed. 2. An explicit "null" for the family name indicates that the field should be removed 3. A phoneNumber field indicates that a new field needs to be added 4. Updated tags indicates that the tags need to be modified.

 

Using Json Patch with Java

json-patch is an awesome java library that provides support for both Json Patch and Json Merge Patch. It integrates with the excellent Jackson library and provides patch tooling on top of the the library. The sample is in kotlin:

val s = """
{
    "title": "Goodbye!",
    "author": {
      "givenName": "John",
      "familyName": "Doe"
    },
    "tags": [
      "example",
      "sample"
    ],
    "content": "This will be unchanged"
}        
""".trimIndent()


val patch = """
    [
        { "op": "replace", "path": "/title", "value": "Hello!"},
        { "op": "remove", "path": "/author/familyName"},
        { "op": "add", "path": "/phoneNumber", "value": "+01-123-456-7890"},
        { "op": "replace", "path": "/tags", "value": ["example"]}
    ]
""".trimIndent()
val jsonPatch: JsonPatch = JsonPatch.fromJson(objectMapper.readTree(patch))
val target = jsonPatch.apply(objectMapper.readTree(s))

Using Json Merge Patch with Java

The library makes using Json Merge patch equally easy:

val s = """
{
    "title": "Goodbye!",
    "author": {
      "givenName": "John",
      "familyName": "Doe"
    },
    "tags": [
      "example",
      "sample"
    ],
    "content": "This will be unchanged"
}        
""".trimIndent()


val patch = """
{
    "title": "Hello!",
    "author": {
      "familyName": null
    },
    "phoneNumber": "+01-123-456-7890",
    "tags": ["example"]
}   
""".trimIndent()

val jsonMergePatch: JsonMergePatch = JsonMergePatch.fromJson(objectMapper.readTree(patch))
val target = jsonMergePatch.apply(objectMapper.readTree(s))

Conclusion

Json Patch and Json Merge Patch are ways to represent a change to a json document. Both approaches do it a little differently but both are equally intuitive.

Jackson Kotlin extension and reified types

Jackson Kotlin module library is a pleasure to use. It greatly simplifies gnarly code, specifically one's involving TypeReference. 

Consider a sample json which looks like this:

{
    "a" : ["b", "c"],
    "b" : ["a", "c"],
    "c" : ["a", "b"]
}

This content can be represented as a "Map<List<String>>" type in Java. 

So now, if I were to use straight Java to convert the string to the appropriate type, the code would look like this:

Map<String, List<String>> result = objectMapper.readValue(json, new TypeReference<>() {
});

What exactly is that "TypeReference" doing there... think of it as a way of making the type parameters of the generic type "Map" which are "String" and "List" available at runtime, without that the types are erased at runtime and Java would not know that it has to create a "Map<String, List<String>>".

Kotlin can hide this detail behind an extension function which if defined from scratch would look like this:

inline fun <reified T> ObjectMapper.readValue(src: String): T = readValue(src, object : TypeReference<T>() {})

and a code using such an extension function:

val result: Map<String, List<String>> = objectMapper.readValue(json)

See how all the TypeReference related code is well hidden in the extension function. 

This is the kind of capability that is provided by the Jackson Kotlin Module With the right packages imported a sample code with this module looks like this:

import com.fasterxml.jackson.module.kotlin.jacksonObjectMapper
import com.fasterxml.jackson.module.kotlin.readValue
val objectMapper = jacksonObjectMapper()
val result: Map<String, List<String>> = objectMapper.readValue(json)

This is just one of the simplifications offered by the use of the module. It also supports features like Kotlin Data classes, Kotlin built-in types like Pair, Triple etc out of the box.

Highly recommended if you are using Jackson with Kotlin.

Expressing a conditional expression using Json - Java Implementation

I had a need recently to express a conditional expression in a form that a front end Javascript application and a backend Java application could both create and read. Expressing the conditional expression as a Json felt logical and after a quick search, JsonLogic library appeared to fit exactly what I was looking for.

JsonLogic follows a prefix notation for its expressions, along these lines:

{"operator" : ["values" ... ]}

So for eg, given a JSON input data that looks like this:

{
  "a": "hello",
  "b": 1,
  "c": [
    "elem1",
    "elem2"
  ]
}

For equality, an expression using JsonLogic is the following:

{"==" : [ { "var" : "a" }, "hello" ] }

Here the data is being looked up using "var" expression and the equality is checked using the "==" operator. 

Though it is a good fit, I decided to go with an alternate way of expressing the conditional expression but heavily inspired by JsonLogic. So, in my implementation, equality with the sample JSON looks like this:

{
    "equals": [
        "/a", "hello"
    ]
}

Fairly similar, the location to the data is expressed as a Json Pointer and the operators are textual ("equals" vs "==") The full set of supported features are also much smaller than JsonLogic as that sufficed my needs for the project. So now I have a small Java-based library that supports these simplified conditional expressions and this post will go into the details of the operators and the usage of the library.

Sample Expressions

Once more to touch on the sample conditional expressions, everything takes the form of:

{"operator" : ["values" ... ]}

A check for equality looks like this:

{
    "equals": [
        "/a", "hello"
    ]
}

Not operator:

{
    "not": [
        {
            "equals": [
                "/a",
                "hello"
            ]
        }
    ]
}

And/Or operator:

{
    "and": [
        {
            "equal": [
                "/a", "hello"
            ]
        },
        {
            "equal": [
                "/b", 1
            ]
        }
    ]
}

There are a few operators that work on collections, for eg, to check if "c" in the sample JSON has elements "elem1", "elem2":

{
    "contains": [
        "/c", ["elem1", "elem2"]
    ]
}

or to check if the collection has any of the elements "elem1", "elem2":

{
    "containsAnyOf": [
        "/c", ["elem1", "elem2"]
    ]
}

Details of the Library

The Java-based library is built on top of the excellent Jackson JSON parser library and uses it to parse the expression which once parsed is interpreted by the library. A Gradle based project can pull in the dependency the following way(published currently to JCenter):

implementation 'com.github.bijukunjummen:json-conditional-expression:0.4.0'

and use the library along these lines, using a sample Kotlin code:

val jsonExpressionEvaluator: JsonExpressionEvaluator = JsonExpressionEvaluator(ObjectMapper())
jsonExpressionEvaluator.matches(expression, json) //returns true

Conclusion

The conditional expression and the corresponding Java-based interpreter are fairly simple and have sufficed my needs with the kind of operators that I needed for my project, however, I will be more than happy to extend and support more extensive operators if there is enough interest in using the library.

Reference

1. JsonLogic which provided the inspiration for using a prefix notation to represent a conditional expression

2. The project resides in GitHub here - https://github.com/bijukunjummen/json-conditional-expression

Json deserialization with Jackson and Super type tokens

Datatables is a jquery plugin to present tabular information - it can enhance a simple table or can use a AJAX based data and present the information in a tabular form.

Datatables requires the data from the server to follow a specific JSON format for it to be displayed on screen. Consider the case where a list of Member entities is to be displayed, the expected json structure for Members then has to be along these lines:

{
   "aaData":[
      {
         "id":1,
         "first":"one",
         "last":"one",
         "addresses":[

         ],
         "version":0
      },
      {
         "id":2,
         "first":"two",
         "last":"two",
         "addresses":[

         ],
         "version":0
      }
   ],
   "iTotalRecords":100,
   "iTotalDisplayRecords":10,
   "success":true
}

A generic java type can be defined which Jackson can use to generate json of the type shown above, consider the following Java generic type:

package mvcsample.types;
import java.util.List;

public class ListWrapper<T> {
    private List<T> aaData;
    private int iTotalRecords;
    private int iTotalDisplayRecords;
    private  Boolean success;

    public List<T> getAaData() {
  return aaData;
 }
 public void setAaData(List<T> aaData) {
  this.aaData = aaData;
 }
 public int getiTotalRecords() {
  return iTotalRecords;
 }
 public void setiTotalRecords(int iTotalRecords) {
  this.iTotalRecords = iTotalRecords;
 }
 public int getiTotalDisplayRecords() {
  return iTotalDisplayRecords;
 }
 public void setiTotalDisplayRecords(int iTotalDisplayRecords) {
  this.iTotalDisplayRecords = iTotalDisplayRecords;
 }
 public Boolean getSuccess() {
  return success;
 }
 public void setSuccess(Boolean success) {
  this.success = success;
 }   
}

So, with this generic type, to generate a list of Members I would have a parameterized type defined as in this test:

List<Member> members = new ArrayList<>();
members.add(new Member("one", "one"));
members.add(new Member("two", "two"));
ListWrapper<Member> membersWrapper = new ListWrapper<>();
membersWrapper.setAaData(members);
membersWrapper.setiTotalDisplayRecords(10);
membersWrapper.setiTotalRecords(100);
ObjectMapper objectMapper = new ObjectMapper();

StringWriter w = new StringWriter();
objectMapper.writeValue(w, membersWrapper);
String json = w.toString();
System.out.println(json);

And similarly a json for any other type can be generated.

However, what about the other way around, generating the Java type given the json.

Again, consider a case where the json given in the beginning is to be converted to ListWrapper<Member> , I can try a deserialization this way:

ObjectMapper objectMapper = new ObjectMapper();  
ListWrapper<Member> membersUpdated = objectMapper.readValue(json, ListWrapper.class);

Note that above I cannot refer to the class type as ListWrapper<Member>.class, I can only refer to it as ListWrapper.class.

This however will not work and the resulting type will not be a wrapper around Member class, as at runtime Jackson has no idea that it has to generate a ListWrapper<Member>.

The fix is to somehow pass the information about the ListWrapper's type to Jackson and this is where Super type tokens fits in. The article explains how this works in great detail, the essence is that
while type erasure does remove the type information from parameterized instances of generic type, however the type is retained in subclasses of generic classes.

For eg. Consider the following StringList class which derives from ArrayList<String> , it is possible to find that the type parameter of the base class is a String as shown in the test below:

import java.lang.reflect.ParameterizedType;
import java.lang.reflect.Type;
import java.util.ArrayList;

public class StringList extends ArrayList<String>{

 public static void main(String[] args) {
  StringList list = new StringList();
  Type superClassType = list.getClass().getGenericSuperclass();
  ParameterizedType parameterizedType = (ParameterizedType)superClassType;
  System.out.println(parameterizedType.getActualTypeArguments()[0]);
 }
}

This is applicable for a case where a subclass is defined as an anonymous class also this way:

ArrayList<String> list = new ArrayList<String>(){};
Type superClassType = list.getClass().getGenericSuperclass();
ParameterizedType parameterizedType = (ParameterizedType)superClassType;
System.out.println(parameterizedType.getActualTypeArguments()[0]);

This is what is used internally with the Super Type tokens pattern to find the type of the parameterized type. Jackson's com.fasterxml.jackson.core.type.TypeReference abstract class implements this and using this the Jackson deserialization would work this way:

import com.fasterxml.jackson.core.type.TypeReference;

....
ListWrapper<Member> membersWrapper = objectMapper.readValue(json, new TypeReference<ListWrapper<Member>>() {});

ListWrapper<Address> addressWrapper = objectMapper.readValue(json, new TypeReference<ListWrapper<Address>>() {});

This way two different parameterized types can be deserialized given a generic type and a json representation.


References:

Reflecting Generics: http://www.artima.com/weblogs/viewpost.jsp?thread=208860

Neal Gafter's Super type tokens: http://gafter.blogspot.com/2006/12/super-type-tokens.html

Ext-JS, Json - Date/time mapping in Spring MVC with Jackson

The default date/time handling with Jackson and Spring is:

• To serialize dates as a Unix epoch time - number of millseconds since Jan 1, 1970 (UTC). 
• To always use GMT as the timezone, irrespective of the default timezone on the server. 

I prefer to see the dates in a string format with ISO-8601 standard. The way to handle this in Spring MVC with version 3.1 of the Spring framework is to provide a custom Jackson ObjectMapper, to handle the date serialization differently. A custom Object Mapper would be along these lines:

import org.codehaus.jackson.map.ObjectMapper;
import org.codehaus.jackson.map.SerializationConfig.Feature;

public class CustomObjectMapper extends ObjectMapper {
    public CustomObjectMapper(){
        super.configure(Feature.WRITE_DATES_AS_TIMESTAMPS, false);
    }
}

Now Spring 3.1 has made it really simple to register this Custom Object Mapper with the Handler Adapters:

<mvc:annotation-driven > 
    <mvc:message-converters register-defaults="false">
        <bean class="org.springframework.http.converter.json.MappingJacksonHttpMessageConverter" >
            <property name="objectMapper">
                <bean class="org.bk.simplygtd.web.spring.CustomObjectMapper"/>
            </property>
        </bean>
    </mvc:message-converters>
 </mvc:annotation-driven>

Prior to Spring 3.1, a way to register this Custom Object Mapper would have been to write a custom BeanPostProcessor, to iterate through the handler adapters, find the relevant message converter(MappingJacksonHttpMessageConverter) and register the custom object mapper, something along these lines:

import org.springframework.beans.BeansException;
import org.springframework.beans.factory.config.BeanPostProcessor;
import org.springframework.http.converter.HttpMessageConverter;
import org.springframework.http.converter.json.MappingJacksonHttpMessageConverter;
import org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter;

public class CustomObjectMapperBeanPostProcessor implements BeanPostProcessor{

    @Override
    public Object postProcessBeforeInitialization(Object bean, String beanName) throws BeansException {
        return bean;
    }

    @Override
    public Object postProcessAfterInitialization(Object bean, String beanName) throws BeansException {
        if (bean instanceof AnnotationMethodHandlerAdapter){
            AnnotationMethodHandlerAdapter methodHandlerAdapter = (AnnotationMethodHandlerAdapter)bean;
            for (HttpMessageConverter<?> messageConverter: methodHandlerAdapter.getMessageConverters()){
                if (messageConverter instanceof MappingJacksonHttpMessageConverter){
                    ((MappingJacksonHttpMessageConverter)messageConverter).setObjectMapper(new CustomObjectMapper());
                }
            }
        }
        return bean;
    }
}

Now the json responses from Spring MVC should be formatted correctly in this format: 2011-12-06T00:00:00.000+0000


If the user interface is using created using Ext-JS, the model for Ext-JS can be customized to use this date format along these lines - date format of 'c' indicates that the date string should be interpreted using ISO-8601 standard:

Ext.define('CUSTOM.model.Proj',{
    extend: 'Ext.data.Model',
    fields:[{name:'id', type:'string'},{name:'name', type:'string'}, 
            {name:'startDate', type:'date', dateFormat:'c'}, {name:'completedDate', type:'date', dateFormat:'c'},
            {name:'isDone', type:'boolean'}, {name:'version', type:'string'}]
});

There is still one problem, when submitting this date Ext-JS still does not format the date string correctly - it would, by default, send the date without the timezone offset(eg. 2011-12-25T00:15:00), which would be interpreted by the JSON deserializer as  a GMT date/time. To fix this just have this global function:

Ext.JSON.encodeDate = function(o)
{
   return '"' + Ext.Date.format(o, 'c') + '"';
};

Now the dates should be correctly posted to the server with timezone offsets also eg. 2011-12-25T01:15:00-05:00



References:
Jackson JSON Processing: http://jackson.codehaus.org/
Jackson Date Handling FAQ: http://wiki.fasterxml.com/JacksonFAQDateHandling
Some ideas from this blog entry: http://magicmonster.com/kb/prg/java/spring/webmvc/jackson_custom.html
Date type in Ext-JS: http://docs.sencha.com/ext-js/4-0/#!/api/Ext.Date