Kotlin JSON Benchmark on Android (2022): Moshi vs Kotlin Serialization

When it comes to automatic serialization and deserialization of Kotlin classes using the JSON format, the two main libraries compatible with Kotlin metadata are currently Moshi and Kotlin Serialization. This compatibility is especially important for non-null types and default values during deserialization, where lack of proper Kotlin support could result in unexpected values occurring at runtime, such as null values in non-null fields. If you’re still using legacy libraries like Gson to parse Kotlin classes, it’s time to reconsider.

Moshi has been supporting Kotlin classes since version 1.5.0, released in 2017. One year later, the next major release 1.6.0 added an annotation processor to generate adapters for Kotlin classes at compile time. I was quite interested in the performance gains allowed by this solution and wrote an article detailing what the generated code does.

JetBrains released version 1.0.0 of Kotlin Serialization in 2020 with built-in support for JSON. This library generates adapters at compile time similarly to Moshi’s annotation processor while being compatible with more platforms and formats. The lack of initial support for streaming was disappointing, so I didn’t even consider using it in production until streaming support was eventually added in version 1.3.0 in 2021. The JSON engine had also been rewritten in the meantime in order to improve performance. Recently, version 1.4.0 added integration with the Okio library, the same library used by Moshi under the hood.

Now that Kotlin Serialization looks full-featured and well-optimized, I thought it would be a good time to compare its performance against Moshi on Android devices with some benchmarks. Which one is the fastest? Take your bets.

The contenders

Moshi-Kotlin Reflection

The runtime Kotlin plugin of the Moshi library. The JSON adapters are generated at runtime using reflection. The main downside of this library is that it adds a runtime dependency to the big kotlin-reflect library (currently a 3 MiB jar file).

Moshi-Kotlin Codegen

The annotation processor Kotlin plugin of the Moshi library. The JSON adapters are generated at compile time by an annotation processor (compatible with both kapt and KSP). It increases build times, but adds no extra runtime dependency (besides the generated adapters).

Moshi-IR

An alternative Kotlin IR implementation of Moshi Codegen, created by Zac Sweers (the current main maintainer of Moshi). It does the exact same thing as Moshi-Kotlin Codegen, but is implemented as a Kotlin compiler plugin instead of an annotation processor for faster build times.

Note: Unlike Moshi-Kotlin Codegen, Moshi-IR doesn’t use very limited reflection at runtime to read default parameter values.
But since the Kotlin model classes used in the following benchmarks don’t have any default parameter values, this optimization will not be put to the test.

Kotlin Serialization

The official Kotlin multiplatform / multi-format serialization library created and maintained by JetBrains. The JSON adapters are generated at compile time by a compiler plugin. Contrary to Moshi which is only compatible with the JSON format, the adapters generated by Kotlin Serialization support other formats: Protocol Buffers, CBOR, HOCON and Properties. Support for more formats is also provided by third-party plugins. The benchmarks in this article will only test the JSON format.

Benchmark setup

This benchmark is based on Zac Sweers’ JSON Serialization Benchmarking github project, updated to add new tests and support the most recent versions of the libraries. You can also find it on Github.

Two different tests are run for each library, in both reading an writing modes:

• Encoding and decoding Kotlin data classes to and from a JSON String;
• Encoding and decoding Kotlin data classes to and from an in-memory byte buffer containing JSON data encoded in UTF-8. This recreates the conditions of reading or writing JSON files or JSON network responses, without the cost of I/O.
  Kotlin Serialization supports 2 different APIs to do this: java.io and okio. Both will be tested.

The benchmarks are run using the AndroidX Benchmark (Microbenchmark) library for more accuracy.

Test data

The test data consists of a 230 KiB JSON file encoded in UTF-8 containing a list of 60 users. Each user contains 23 fields of various types, including nested objects and nested lists. For reference, here is the Kotlin data class definition:

data class User(
    val id: String,
    val index: Int,
    val guid: String,
    val isActive: Boolean,
    val balance: String,
    val pictureUrl: String,
    val age: Int,
    val name: Name,
    val company: String,
    val email: String,
    val address: String,
    val about: String,
    val registered: String,
    val latitude: Double,
    val longitude: Double,
    val tags: List<String>,
    val range: List<Int>,
    val friends: List<Friend>,
    val images: List<Image>,
    val greeting: String,
    val favoriteFruit: String,
    val eyeColor: String,
    val phone: String
)

The same data is used for read and write tests.

Libraries versions

• AndroidX Benchmark 1.1.0
• Kotlin 1.7.10
• Okio 3.2.0
• Moshi 1.14.0
• MoshiX (Moshi-IR) 0.18.3
• Kotlin Serialization (KSerizalizer) 1.4.0

Test hardware and operating system

• Google Pixel 4a (2020) running Android 13
• Google Nexus 5 (2013) running Android 6

Write performance

Note: the scale on the Nexus 5 graph is about 10x the scale of the Pixel 4a graph, meaning the tests ran about 10 times slower on this older device.

The first thing we can notice is that the reflection-based adapters are much slower than the compile-time generated adapters, especially on the older Android version.

We can verify that performance of Moshi-Kotlin Codegen and Moshi-IR is virtually identical, since the generated code is supposed to be the same.

The Moshi tests producing a String run a bit slower than the tests writing to a byte buffer. This can be explained by the way this library works:

Moshi is built on top of the Okio library and works primarily with byte buffers, not characters or Strings. Obtaining a String requires an extra step: reading back and decoding the entire byte buffer into a String. That extra step is the overhead that can be seen in the Moshi String benchmarks. It’s less visible on the slower Nexus 5 because producing the bytes takes much more time.

In comparison, the Kotlin Serialization tests producing a String run faster than the tests writing to a byte buffer. This can again be explained by the way this library works:

The JSON module of Kotlin Serialization works primarily with sequences of characters and Strings. Writing to a byte buffer requires an extra step: converting the characters to bytes. This conversion is performed either by the java.io.Writer class (used by Json.encodeToStream() in the buffer test) or the okio.BufferedSink class (used by Json.encodeToBufferedSink() in the okiobuffer test). We can see that the performance of these two methods is very close so which one to use doesn’t really matter.

In both cases, writing to byte buffers on the lower-end Nexus 5 is more than two times slower using Kotlin Serialization JSON compared to using Moshi. On the newer Pixel 4a, the performance of the two libraries is very close. This could be partially explained by the fact that Kotlin Serialization JSON allocated two times more objects than Moshi during the test (1373 allocations vs 769 for Moshi), and the memory allocation and garbage collector performance improved significantly in newer Android versions.

Winner: Moshi-IR / Moshi-Kotlin Codegen, even if Kotlin Serialization is close on newer devices.

Loser: Moshi-Kotlin Reflection.

Read performance

Now let’s test the more common operation of converting JSON data to Kotlin objects.

On average, the reading tests are almost two times slower than the writing tests.

The reflection-based Moshi adapters are still slower than the compile-time generated adapters, but this time the overhead is lower than for writing tests.

The performance of Moshi-Kotlin Codegen and Moshi-IR is still virtually identical as expected.

Again, the Moshi tests taking a String as input run a bit slower than the tests reading directly from a byte buffer for a similar reason: Moshi needs to encode the String to a byte buffer first.

The most important outtake from this chart is that the buffering performance of Moshi-Kotlin Codegen/Moshi-IR is on par with Kotlin Serialization Json. The teams at JetBrains did an impressive job at improving performance since version 1.0.0 of the library, while also adding buffering support. It’s still a bit slower on the Nexus 5, probably because of the lower byte-to-char decoding performance of java.io.Reader compared to Okio on older Android versions.

Last but not least, we need to talk about the elephant in the room (the last bar on the right): there is something really wrong with the reading performance of the new Okio module of Kotlin Serialization JSON. While it’s supposed to reduce overhead, it’s actually two times slower than the “regular” streaming API based on java.io.InputStream and java.io.Reader.

The reason for this huge performance gap becomes obvious when looking at the source code of the current implementation:

internal class OkioSerialReader(private val source: BufferedSource): SerialReader {
    override fun read(buffer: CharArray, bufferOffset: Int, count: Int): Int {
        var i = 0
        while (i < count && !source.exhausted()) {
            buffer[bufferOffset + i] = source.readUtf8CodePoint().toChar()
            i++
        }
        return if (i > 0) i else -1
    }
}

Sure enough, the character array buffer is filled by decoding one character at a time, and checking for buffer exhaustion after each character. This is definitely a performance bottleneck, but an understandable one since the Okio API doesn’t provide a way to decode UTF-8 characters in batch to a CharArray yet. The best it can do is return a new String to copy to the array, which would allocate memory but probably still be faster.

In its current state, I see no reason to use the new Okio APIs of Kotlin Serialization JSON. Just create a regular java.io.InputStream from your okio.BufferedSource instead:

Json.decodeFromStream(serializer, bufferedSource.inputStream())

Winner: Moshi-IR / Moshi-Kotlin Codegen and Kotlin Serialization using InputStream (tie).

Loser: Kotlin Serialization using BufferedSource.

Read performance — Selective

In this final test, we’ll measure the performance of partial decoding: sometimes we’re not interested in parsing the entire JSON file and fields can be skipped. To test this, we modify the User data class to keep only 5 fields on 23:

data class User(
    val id: String,
    val index: Int,
    val guid: String,
    val age: Int,
    val name: Name
)

The test JSON file remains unchanged. All fields not present in the User class will be skipped by the JSON adapters.

In the previous test, the buffered reading performance of Moshi and Kotlin Serialization was identical or very close. With partial decoding, we can see that the gap widens and Moshi is ahead of Kotlin Serialization JSON, especially on older devices where it’s almost twice as fast.

This is probably due to the fact that Moshi is able to directly skip unwanted bytes in the stream without having to decode them to characters first or allocate any memory, thanks to the powerful API of the Okio library. Kotlin Serialization JSON, on the other hand, has to decode all bytes to characters and also performs memory allocations while skipping the unwanted fields.

Conclusions

As with any microbenchmark, one needs to be careful when it comes to drawing conclusions. That being said, it’s safe to claim that:

• The performance of Moshi 1.14.0 and Kotlin Serialization JSON 1.4.0 is very close and they both perfom very well;
• Moshi-Kotlin Reflection should be avoided because compile-time generated adapters are always faster and don’t require extra runtime dependencies;
• The current Okio integration in Kotlin Serialization JSON is mediocre and I would advise not to use it. It provides no substantial performance gains for encoding and hurts performance for decoding. I suspect that either this integration will be phased out in a future release, or the core JSON engine will be rewritten to take real advantage of Okio (like Moshi) instead of the limited high-level integration we have now.

And the most important conclusion of all:

The time required to encode or decode the JSON data is so small compared to the latency of the network or storage access that it doesn’t really matter and won’t be noticed, unless it’s so bad it goes off the charts. This is not the case for these two libraries.

Which one should you pick? The one that fits your needs best.
Both have their qualities:

• Moshi has more advanced features and provides a lower-level JSON API for maximum control;
• Kotlin Serialization is multiplatform (Javascript, Kotlin native) and its adapters work with other formats than JSON without extra configuration in the model classes. It’s also compatible with unsigned integer types, value classes and sealed classes out-of-the-box which is something to consider.

Thank you for reading and please share your own benchmark results if you have some.