No, I am better than that!

I’m attempting to establish a new development pattern at work around how services transfer objects across transports that require serialization and deserialization.

That is, if you have two services which communicate across REST or a message bus or carrier pigeon, how do you make that bridge transparent?

+---------+             +---------+
|   Red   |  Transport  |  Green  |
| Service |  <------->  | Service |
+---------+    Medium   +---------+

At the same time, I’m pushing a strong division of client libraries: the developers of each service publish not just a service package but also a client package. In the above scenario, let’s pretend that the Green service needs data from the Red service, so Red is the Producer and Green is the Consumer. In a pseudocode dependency tree:

Package	Dependencies
red-client	(none)
red-service	red-client
green-service	red-client

Why, you may ask, does the Red service have a dependency on its own client? A client package contains all of the interface definitions and data structures needed to communicate with a service across the transport. These interfaces and structures will need to be deserialized by the service, and you don’t want to duplicate them in the service package.

Example

As a concrete example, let’s say that the data structure being passed around is a person. In Java-land, we could make a class Person. I’m going to skip over the getters and setters here for brevity, using Lombok‘s @Data instead:

@Data
public class Person {
    protected String fullName;
}

Using a concrete class with fields is the simplest way of defining this data structure. But it suffers from the same problems that inheritance always has: Java’s lack of support for multiple inheritance forces you to either extend it or do some really goofy gymnastics to copy to/from it. To see this, imagine that a service stores Person instances with JPA. This requires a few annotations:

@Data
@Entity
public class Person {
    @Column(length = 255)
    protected String fullName;
}

But that’s a problem, right? You need the JPA annotations for storage, but only one service should be storing instances—clients of that service are only using transient instances. So you don’t want to introduce a JPA dependency in the client package, only in the service package.

You might try:

@Entity
public class PersonEntity extends Person {
...

But then you start to run into weirdness with JPA: if you need to add that @Column annotation on the fullName field, where do you do it? If you create another fullName field in PersonEntity to mask the one in Person, you’ll find that it doesn’t work quite right. Similarly, if you put the annotation on the getFullName() accessor method, you’re locked into putting all JPA annotations on methods, as mixing them with field-level annotations causes even more weird behavior.

So let’s back up.

What if our client package instead defines a Person with an interface instead of a concrete class?

public interface Person {
    String getFullName();
}

This works out well for our service because our entity annotations do the right thing, and there’s only the one concrete class:

@Data
@Entity
public class PersonEntity implements Person {
    @Column(length = 255)
    protected String fullName;
}

But what about consumers of our service—other services on the other side of the client? When our client library deserializes an instance that has been sent across our transport, what does it get?

For that, your client could define a value object concrete implementation of the interface:

@Data
public class PersonImpl implements Person {
    protected String fullName;
}

If you define it within the Person interface, you’re sending a clear message that consumers of the client don’t need to know the details:

public interface Person {
    String getFullName();
    @Data
    class PersonImpl implements Person {
        protected String fullName;
    }
}

You might even have a builder:

public interface Person {
    ...
    static Person build (final String name) {
        return new PersonImpl(name);
    }
}

This seems nice and clean:

• Your -client package defines data structures as interfaces, with value object default implementations. It has minimal dependencies.
• Your -service producer package has concrete implementations of those interfaces which may include additional functionality, annotations, etc. It depends on the -client and anything else it needs.
• Your consumer packages don’t need concrete implementations of the data structures, as they get the value object defaults from the client package.

It gets a little hairy when you want to serialize and deserialize. Let’s talk serialization first.

Serialization

Let’s presume we’re using Jackson for JSON serialization, which means we just add jackson-databind as a dependency of our client package.

One scenario covers the “save a new Person” story: the consumer builds a new Person and sends it to the service. In a Spring world, it might look something like:

@RestController
public class PersonResource {
    @RequestMapping(method = POST, value = "/person")
    public Person newPerson(@Param("fullName") final String name) {
        final Person unsaved = Person.build(name);
        return personClient.save(unsaved);
    }
}

Behind the scenes, that save method needs to serialize the Person to send it across the transport. Because the instance is a simple value object, this works without much fuss:

public Person save(final Person unsaved) {
    try {
        final String json = new ObjectMapper().writeValueAsString(unsaved);
    } catch (final JsonProcessingException e) {
        e.printStackTrace();
        return null;
    }
    ...

The JSON representation would be exactly what you’d expect:

{"fullName":"Rick Osborne"}

So far so good.

Before we look at serialization from the producer service side, let’s make our JPA entity class a little more realistic and add a primary key and a super-secret hashed password:

@Data
@Entity
public class PersonEntity implements Person {
    @Id protected UUID id;
    @Column(length = 255)
    protected String fullName;
    @Column(length = 64)
    protected String hashedPassword;
}

If we serialized a PersonEntity naively as we did above, we’d have those fields in our JSON:

{"id":"ABC...","fullName":"Rick...","hashedPassword":"..."}

We’ve got a conundrum: we could add Jackson annotations such as @JsonIgnore to our PersonEntity so that we didn’t accidentally send out passwords (even hashed). But that would mean that we could never include such fields, even in cases where we knew the JSON would stay local. Alternatively, we could write a custom serializer for transport … but yuck.

A simpler option might be use our existing value object: give the Person interface a method that reduces an instance to only what is needed for transport:

public interface Person {
    ...
    default Person forTransport() {
        if (this instanceof PersonImpl) return this;
        return build(getName());
    }
    ...

Presuming we have a PersonEntity#from(Person) method to upgrade a Person to a PersonEntity, the client on the producer service can then convert just before transport:

public Person onSaveEvent(final Person person) {
    final PersonEntity entity = PersonEntity.from(unsaved);
    final PersonEntity saved = saveHandler.save(entity);
    return saved.forTransport();
}

That from method might try to load an existing entity, or just naively use Jackson’s readValue(String, Class) or readTree(String) or similar. We’ve got serialization worked out for both sides, so let’s dig into that deserialization.

Deserialization

This is where things start to get really nasty. Using Jackson, how does the client package deserialize a Person? Remember that it doesn’t know anything about PersonEntity and that it really shouldn’t know about PersonImpl, even if it can see that it’s there. Instead, it should stick to the Person interface.

But doing this naively causes an error:

new ObjectMapper().readValue(json, Person.class)

JsonMappingException: Can not construct instance of Person: abstract types either need to be mapped to concrete types, have custom deserializer, or contain additional type information

Presuming we really don’t want to go down the custom deserializer route, that leaves us with two main options.

@JsonDeserialize

We could tell Jackson to always deserialize a Person using its PersonImpl class:

@JsonDeserialize(as = PersonImpl.class)
public interface Person {
    ...

This allows the readValue(json, Person.class) call to work as expected.

If you want to handle JSON with extra fields silently drops the extras without causing errors, you need to do that on PersonImpl:

@JsonIgnoreProperties(ignoreUnknown = true)
class PersonImpl implements Person {
    ...

All of this works great … until you try something like this in your producer service:

new ObjectMapper().readValue(json, PersonEntity.class)

You’ll find that the the @JsonDeserialize on the Person means you end up with a PersonImpl instead of a PersonEntity! You can override this on PersonEntity:

@Entity
@JsonDeserialize(as = PersonEntity.class)
public class PersonEntity implements Person {
    ...

But yuck. Maybe @JsonDeserialize isn’t the best route. It works, but it’s janky.

Let’s try something else.

@JsonCreator

We already have our Person#build(String) method, to which we could just add @JsonCreator and @JsonProperty:

public interface Person {
    ...
    @JsonCreator
    static Person build (@JsonProperty("fullName") final String name) {
        return new PersonImpl(name);
    }
}

This has the same effect when calling readValue(json, Person.class): you get back a PersonImpl. But when doing the same with PersonEntity.class you get back a PersonEntity. You still may want to add the @JsonIgnoreProperties to PersonImpl, of course.

Summary

We’ve managed to work out a clean separation between local (storage) representations and wire (transport) representations of objects. We’ve also made it so that you can serialize as much or as little as you want, without leaking implementation details (such as local class names) across the wire. And most importantly we’ve done it all declaratively, without writing any custom serializers or deserializers. Packages can include client libraries with an absolute minimum of dependencies, while services can hide all of their implementation details from the packages which talk to them.