Data Classes and parsing JSON — A Story about converting Models to Kotlin

Published in

Product Engineering @ Kitchen Stories

8 min readAug 23, 2017

With Google’s announcement of first class support for Kotlin, the time had come to finally integrate it into our app. The first big application — after converting some tests and small helper classes — was to convert all models reassembling the JSON-Schema of our API.

This article summarizes the journey along the way and has a few tips on how to enforce Kotlins Null Safety.

A little Introduction

Think about an API which exposes editorial content in a semi structured manner. As an example for this article we examine an actual, abbreviated Article model from our API.

A few weeks ago in Java world, a simplified version of Article looked like this. It was able to parse an example JSON from the API.

{
  "id": "article_123",
  "title": "Title of the Article",
  "subtitle": "and a subtitle",
  "cell_image": {
    "url": "https://images.kitchenstories.de/wagtailOriginalImages/A453-photo-content-18.jpg"
  },
  "author": {
    "username": "Stefan"
  },
  "comments": [
      ...
  ],
  "content": [
      ...
  ]
}

With the right JSON-Parser (more on that later), there was no problem in transforming this JSON into an Java instance of Article. However, when using Java models to describe JSON, omitted fields in the JSON become null in the Java model. For example if the JSON does not contain author, that field would be initialized with null in Java.

Views and presenters accessing nested fields of this model need quite a few nested null checks. Again, if the field author or username are missing, they become null in the parsed Java model. So if we want to access the authors name to set it on a TextView, we need to check if the whole author instance is not null and if the username is not null to cover every possible edge case. It does not look like a big problem in this example, but in real world Apps the data structure is likely more sophisticated and more nested, leading to quite some tedious chains of null checks.

The last problem we would like to emphasize on is inherent to APIs themselves. As mentioned before, our API delivers editorial content to all clients. In rare occasions, this content does not look as expected. For example an Article JSON could have a cell_image, but url is missing from cell_image. This is an example for edge cases that can lead to NullPointerExceptions when they have not been accounted for via extensive null checking. But also, a cell_image without an url can not be displayed in the App, because it makes actually no sense. We identified all crashes based on API inconsistencies comparable to this, are usually linked to content which was not yet ready for publishing, but unintentionally got published. Therefore we decided we want to prevent that kind of App crashes by filtering broken content.

To summarize, our goal with reworking our models was to get rid of tedious null checks when accessing fields in presenters and views and also filter content from the parsed models, that can not be properly displayed. Lets switch over to Kotlin and have a look at how this new language can help us solve the problems above with ease.

The Kotlin way

After converting the model using ⇧ ctrl ⌥ + K and a few changes, we have a new model written in Kotlin (yeah!).

The differences to our Java model are small but important. First of all no field in our model can be null anymore. The only exception is video, because we support articles without videos.

The second improvement are default values. If our article does not have any comments, the comments field will not (and can not) be initialized with null. Instead it would consist of an empty list. In any presenter accessing comments, we can spare the null check of the field and iterate the list of comments directly. Be aware that you still have to define a field as nullable, if the JSON could potentially define this field explicitly as null.

There is an even bigger side effect on this new model, which is not coincidental but crucial to what we wanted to achieve by this conversion. All fields which can not be null and do not have a default value defined in the constructor are now mandatory fields: id, title, image and author. It is impossible to create an instance of Article without providing these fields, therefore any presenter or view accessing an Article can be sure that none of the mandatory fields are missing. So in the calling code we never have to check again whether an Article has an image or an author (or - what is part of Image.kt - if the Image has a url).

Now we deferred all the concerns on nullable fields and mandatory fields into the Article model itself, but can we still parse this?

Let’s talk about JSON-Parser

The important part we have so carefully avoided talking about until now is the JSON-Parser. When we first introduced Retrofit to our Android App, we decided to use Moshi to parse JSON. The rest of this article will describe a Moshi setup, but the concepts can most likely be adapted to any other JSON parser.

Moshi can already parse from JSON into Kotlin (data) classes and vice versa out of the box. For Article.kthowever, we also need to add the KotlinJsonAdapterFactory to the initialization of Moshi. Without the KotlinJsonAdapter, @JSON annotations and default values in constructors do not work. A minimal example initialization of Moshi with KotlinJsonAdapter looks like this:

val moshi = Moshi.Builder()
    .add(KotlinJsonAdapterFactory())
    .build()

The KotlinJsonAdapterFactory is part of the moshi-kotlin plugin, so your build.gradle should import this plugin. Be aware that moshi-kotlin uses kotlin.reflect library at the moment. This can lead to bigger APKs and surpass the dex method count limit of your App. You should probably configure Proguard when using this solution. There is already a discussion on possible improvements by the team, so improvements on this drawback are likely.

implementation "com.squareup.moshi:moshi-adapters:1.5.0"
implementation "com.squareup.moshi:moshi-kotlin:1.5.0"

Problem solved?

We have come a long way. The Article model now enforces which fields are mandatory and sets defaults on the fields which are not mandatory by itself. That is a big step forward, but to be honest, we have left the most important part for the grand finale.

As briefly mentioned before, in some unfortunate situations, the API can send back broken JSON. The Article model, which we have so carefully crafted and designed to match all our expectations on the calling side, could be contradicted by the JSON sent from the API. For example, consider a case where the Article JSON does not contain a cell_image. The App would crash with a JsonDataException because we declared it as a mandatory field. Do we have a way to avoid this crash and tell Moshi what we'd rather want?

DefaultOnDataMismatchAdapter.kt is an adapter for Moshi, which maps a class type to a default return value in case of parsing errors. On the one hand it could be used to return null as default to recover from the parsing errors, so that content which could not be parsed is factually filtered from the parsed result. On the other hand we can use DefaultOnDataMismatchAdapter.kt to return meaningful values replacing the wrong content. If the JSON describing our Article does not contain a cell_image, this Article is most likely not supposed to be published and can not be correctly displayed in the App. Therefore we filter it by returning null as the default value.

Before we wrap up our example, we may also need a second adapter for some cases.

FilterNullValuesFromListAdapter.kt fixes an issue in the latest Moshi version. A null-safe list defined in Kotlin (e.g. List<Article> instead of List<Article?> ) can still contain nulls. This means that the previously mentioned case with a broken Article where our adapter implementation returns null might lead to a list of non-nullable Articles that includes null values. The FilterNullValuesFromListAdapter hooks into the parsing process, parses a list which might contain nulls and filters all nulls afterwards, returning a list without any nulls.

The easiest way to understand both adapters is to apply them on our Article example. Lets add the two adapters to the Moshi.Builder instantiation and have a look.

And now we are really done. Everything we need to ensure the consistency of all classes used in Article.kt is added to our JSON parser.

The first add statement is an example of a meaningful return value. The model Video.kt (gist) has a field type, which can be unknown, recipe or article. If the JSON does not provide a video type, the default constructor of Video already has unknown defined as the default value. But if the JSON does provide a video type we do not know in our App yet, the App would crash again without this line. For example, if the VideoType would be howto, the first add statement would prevent the App from crashing, initializing an instance of Video with VideoType.unknown instead, because we have not implemented this VideoType yet.

Since there is still the possibility of Video being error prone (e.g. video_url is missing), the second add statement filters error prone Video instances. In this example, the Article can be displayed in the App as an Article without a Video.

The third and fourth line ensure that all content in the article meets the requirements defined in Content.kt. How Content.kt is defined in detail is not important in order to understand these two lines. In case of a parsing error of Content, the third line enables Moshi to return null instead of throwing an Exception. The fourth line filters all nulls from List<Content> defined in Article.kt.

In the last line we ensure that error prone Articles do not lead to crashes in the App, but are filtered from all instances defining Article as a field.

And we are good to go. With this example, you should now be able to convert all your models to Kotlin and also benefit from non-nullable fields and default parameters.

TL;DR

With Kotlin we can declare models with non-nullable fields and provide default values for missing fields
With little effort, it is possible to enforce non-null fields in Kotlin classes when parsing JSON
On the one hand Moshi 1.5.0 + kotlin-adapter enable @Json annotation and constructor default values in Kotlin when parsing JSON
On the other hand you have to consider increased APK size by using both
Be aware, that fields defined in the constructor have to be defined as nullable when the API could return a null for that field

Addition #1:

One discussion on reddit made us think, that it could be confusing to the reader how the different ways of defining fields in JSON are handled:

There are actually three states a field can have in JSON:

{
    "case_1" : "This field has a value"
    // case_2 (omitted) 
    "case_3" : null
}

Only the first and the second case can benefit from Kotlins non-nullable fields. But be aware, that case_2 needs a default value defined in the constructor to be non-nullable. If your API can send an explicit null like case_3, and you define this field as non-nullable, the parser crashes!

Addition #2

For a presentation at our local Kotlin Meetup and Android Meetup I prepared a Live-Coding-Example which is now available via Github:

https://github.com/stefanmedack/MoshiKotlinExample

If you have any questions or feedback, you can contact the author on Twitter or you can also see our Kotlin code in action in our Android App

Stefan Medack