Scala Circe, in depth: the Json model, cursors, derivation in Scala 2 and 3, configured codecs, error accumulation and production pitfalls

The three layers: parser, AST and type classes

Everything in circe goes through one immutable data type, io.circe.Json. A value is a null, a boolean, a number, a string, an array or an object. Objects keep field order and are represented by JsonObject. Numbers are held as JsonNumber, which keeps the original textual form until you ask for a specific numeric type. That matters: a 20-digit ID is not silently squashed into a Double, and you choose whether it becomes a Long, a BigInt or a BigDecimal.

The parser sits in a separate module, circe-parser. On the JVM it is backed by the jawn parser. io.circe.parser.parse returns Either[ParsingFailure, Json] and io.circe.parser.decode[A] parses and decodes in one step, returning Either[Error, A], where Error is either a parsing or a decoding failure. Nothing throws on bad input. Parsing does not need your types; it only checks that the text is JSON.

The type classes carry all type-specific logic. Encoder[A] is essentially A => Json. Decoder[A] is essentially HCursor => Decoder.Result[A], where Decoder.Result[A] is Either[DecodingFailure, A]. Codec[A] bundles both. Encoder.AsObject and Codec.AsObject are refinements that guarantee the output is a JSON object, which is what you need when you later merge extra fields into it.

Cursors: how a Decoder walks the tree

A decoder does not receive a Json value directly. It receives an HCursor: a position in the tree plus the history of moves that led there. Moves such as downField("user"), downArray or downN(2) return an ACursor, which may have failed. Failure does not throw; it is carried to the point where you call .as[A] or .get[A]("name"), and the resulting DecodingFailure includes the history. That history, available as failure.history, is why circe errors can say where the problem is: in the example below, the failure records the moves into order, items, element 1 and qty.

import io.circe._, io.circe.parser._

val doc: Json = parse("""{"order":{"items":[{"sku":"A1","qty":2},{"sku":"B7","qty":"x"}]}}""")
  .getOrElse(Json.Null)

val c: HCursor = doc.hcursor
val firstSku: Decoder.Result[String] =
  c.downField("order").downField("items").downN(0).get[String]("sku")      // Right("A1")

val secondQty: Decoder.Result[Int] =
  c.downField("order").downField("items").downN(1).get[Int]("qty")         // Left(DecodingFailure ...)

You will use cursor code for the awkward cases: envelopes, values that can be a string or a number, nested fields. Custom decoders should use c.downField(...).as[A] rather than extracting a raw Json and decoding it separately, because the second form loses the path and your errors lose their location.

Getting instances: by hand, forProductN or derivation

There are four ways to obtain an Encoder or a Decoder. They trade effort against control, and a mature codebase usually uses all four.

circe also offers fully automatic derivation through import io.circe.generic.auto._, which materialises instances at every use site. It is convenient in a script and a poor default in a service. Each use site re-derives, which costs compile time, and the wire format of a type changes silently whenever someone edits the case class, because no instance declaration exists to review. Prefer semiauto: one explicit instance in the companion object, derived once, found by implicit scope everywhere else.

// Scala 2.13 or Scala 3, circe-generic
import io.circe._, io.circe.generic.semiauto._

final case class Money(amount: BigDecimal, currency: String)
object Money {
  implicit val codec: Codec.AsObject[Money] = deriveCodec[Money]
}

// Scala 3, circe-core only: the derives clause calls Codec.AsObject.derived
final case class Sku(code: String, qty: Int) derives Codec.AsObject

Derived instances need instances for every field type. When compilation fails with a missing implicit for Decoder[Money], the fix is almost always to give Money its own instance rather than reaching for auto derivation. On Scala 2, the @JsonCodec annotation from circe-generic generates the companion instances for you but needs the -Ymacro-annotations compiler flag.

Controlling the wire format with Configuration

Default derivation maps Scala field names verbatim, ignores case class default values when decoding, tolerates unknown fields, and encodes a sealed trait as a single-key wrapper object such as {"Card": {...}}. Real APIs usually want something else. In Scala 3, circe-core's io.circe.derivation.Configuration controls these choices, and ConfiguredCodec derives an instance that honours the given configuration in scope. On Scala 2 the equivalent lived in the separate circe-generic-extras module; check which your build uses before copying examples between versions.

import io.circe.derivation.{Configuration, ConfiguredCodec}

given Configuration =
  Configuration.default
    .withSnakeCaseMemberNames
    .withDefaults
    .withDiscriminator("type")

enum Payment derives ConfiguredCodec:
  case Card(last4: String, network: String)
  case BankTransfer(iban: String, reference: Option[String] = None)

Treat a Configuration as part of your public contract. Define it once per API in an object with a descriptive name, import it explicitly, and never let two different configurations be in implicit scope for the same module. A subtle bug class comes from a type derived in one package with snake_case and reused in another package where the default applies. For enums whose cases have no fields, ConfiguredEnumCodec encodes each case as a plain string rather than an object.

Writing codecs by hand, and combinators that save you from it

Hand-written instances are short when you lean on combinators. Decoder#map and Encoder#contramap adapt an existing instance to a wrapper type. Decoder#emap adds validation that returns Either[String, B] and turns a Left into a DecodingFailure that keeps the cursor path. Decoder#or tries an alternative. With these, value classes, refined strings and tolerant legacy fields take a few lines each.

final case class OrderId(value: String) extends AnyVal
object OrderId {
  private val Pattern = "ord_[a-z0-9]{12}".r
  implicit val decoder: Decoder[OrderId] =
    Decoder[String].emap {
      case s @ Pattern() => Right(OrderId(s))
      case other         => Left(s"not an order id: $other")
    }
  implicit val encoder: Encoder[OrderId] = Encoder[String].contramap(_.value)
}

// An upstream that sometimes sends qty as a string
val lenientInt: Decoder[Int] =
  Decoder[Int].or(Decoder[String].emap(s => s.toIntOption.toRight(s"bad int: $s")))

// An envelope: {"data": {...}, "meta": {...}}
def enveloped[A: Decoder]: Decoder[A] = Decoder.instance(_.downField("data").as[A])

On the encoding side, Encoder.AsObject#mapJsonObject adds or removes keys after derivation, so you can attach a computed field without abandoning the derived instance.

Worked example: decoding a partner order webhook

A partner posts order events. Fields are snake_case, discount may be absent, amounts arrive as JSON numbers with two decimal places, and payment is a tagged union. We want one decoder, clear errors and the same format on the way out.

import io.circe._, io.circe.parser.decode
import io.circe.derivation.{Configuration, ConfiguredCodec}

object PartnerWire:
  given Configuration =
    Configuration.default.withSnakeCaseMemberNames.withDefaults.withDiscriminator("type")

import PartnerWire.given

// Payment (previous section) must be derived under this same given, not a second one
final case class Line(sku: String, unitPrice: BigDecimal, qty: Int) derives ConfiguredCodec
final case class OrderEvent(
    orderId: String,
    placedAt: java.time.Instant,
    lines: List[Line],
    discount: BigDecimal = BigDecimal(0),
    payment: Payment
) derives ConfiguredCodec

val body = """{"order_id":"ord_7g2k","placed_at":"2026-09-30T10:15:00Z",
  "lines":[{"sku":"A1","unit_price":12.50,"qty":2}],
  "payment":{"type":"Card","last4":"4242","network":"visa"}}"""

decode[OrderEvent](body)
// Right(OrderEvent(ord_7g2k, 2026-09-30T10:15:00Z, List(Line(A1,12.50,2)), 0, Card(4242,visa)))

Three details deserve attention. First, java.time.Instant decodes from an ISO-8601 string because circe ships instances for the java.time types. Second, discount is absent in the payload and gets the default only because the configuration says withDefaults; without it the decode fails with a missing-field error. Third, the amounts are BigDecimal, so 12.50 is kept exactly. Decoding money into Double is a classic bug that passes tests with round numbers.

If the partner sends "qty":"two", the error reads as a failure to decode an Int with a history pointing at qty inside the first element of lines. Log that history with every rejected payload.

Reporting every error, not just the first

Decoders are fail-fast by default: the first bad field stops decoding. For a user-facing form or a bulk import, you usually want every problem at once. circe supports that through Decoder#decodeAccumulating and io.circe.parser.decodeAccumulating, which return a ValidatedNel of failures. Derived instances accumulate across fields; a hand-written decoder built with a for-comprehension over Either still stops at the first error unless you also give it an accumulating implementation.

import io.circe.parser.decodeAccumulating
import cats.data.Validated

decodeAccumulating[OrderEvent](badBody) match
  case Validated.Valid(evt)     => handle(evt)
  case Validated.Invalid(errs)  =>
    errs.toList.foreach(e => log.warn(s"${e.getMessage}"))

Printing, nulls and numbers

Output goes through Printer. json.noSpaces and json.spaces2 are shortcuts, and Printer.noSpaces.copy(dropNullValues = true) omits keys whose value is null. By default a derived encoder writes None as null, which some consumers treat differently from an absent key. Decide per API, then use one shared printer so every endpoint behaves the same. Json#deepDropNullValues does the same cleanup on a tree before printing.

Numbers deserve their own rule. circe preserves large integers, but a JavaScript consumer parses numbers as IEEE doubles and loses precision above 2^53. If an ID can exceed that, encode it as a string on the wire. Be deliberate with Double: NaN and infinity are not valid JSON numbers, and circe encodes them as null.

Every document is fully materialised as a Json tree before decoding. For multi-megabyte arrays, decode records incrementally with a streaming integration such as fs2's circe support.

Integrating with HTTP and messaging

HTTP libraries delegate to circe rather than reinventing codecs. In http4s, the http4s-circe module provides entity decoders and encoders from your circe instances, so a route can decode a request body into OrderEvent and failures become a 4xx response. Akka HTTP and Pekko HTTP have community marshallers that do the same. For Kafka or SQS, keep decoding in one small function that turns bytes into Either[Error, A] and route failures to a dead-letter destination with the original payload and the failure history attached. When the external contract is not yours, decode into a wire type that mirrors it and convert to your domain type in a pure function, so a partner's rename stops at the boundary.

Failure modes seen in production

• Silent contract drift. A field renamed in a case class changes the JSON. Pin the format with round-trip tests against checked-in sample payloads, not only decode(encode(x)) == x.
• Two configurations in scope. The same type encodes with snake_case in one module and camelCase in another. Keep one named configuration per API and import it explicitly.
• Defaults not applied. A new optional field breaks old clients because withDefaults was not enabled. Use Option or enable defaults, and test decoding of the previous payload version.
• Money as Double. Rounding errors appear in reconciliation. Use BigDecimal for amounts.
• Auto derivation everywhere. Compile times climb and wire formats change without review. Move to semiauto instances in companions.
• Lost error paths. Custom decoders that extract raw Json and decode it separately return errors with no location. Stay on the cursor.
• Unknown discriminator values. A partner adds a new payment type and every event fails. Decide whether to fail, route to a dead-letter queue, or decode into an explicit Unknown case with a hand-written fallback.

Related reading

Continue with http4s for serving and consuming these codecs over HTTP, Cats Effect for the effect runtime around them, implicits and givens for how instance resolution finds your codecs, Scala 3 macros for how derivation works under the hood, and effect systems compared when choosing a stack.

What to do next

1. Find every import io.circe.generic.auto._ in your services and replace it with semiauto instances or derives clauses in companion objects.
2. Define one named Configuration per external API and import it explicitly; document it next to the API spec.
3. Check in real sample payloads for each external contract and add tests that decode them and compare encoded output with the sample.
4. Audit money, IDs and timestamps: BigDecimal for amounts, strings for IDs above 2^53, Instant for timestamps.
5. Use decodeAccumulating at user-facing boundaries and log the cursor history of every failure.
6. Pick one shared Printer and decide once whether absent values are null or omitted.
7. Decide the policy for unknown enum or subtype values before a partner forces it on you.