GenshIv/silentjson

Contribute to GenshIv/silentjson development by creating an account on GitHub.

Summary

silentjson is a Go JSON library designed for decoding large arrays of objects and streaming JSON with low allocation and no code-generation step. It builds a type registry once, then provides strict and tolerant parsing, parallel batch decoding, and serialization; the README’s performance figures are specific to its documented benchmark environment.

What it is useful for

The main use case is processing large JSON arrays where throughput and memory pressure matter. For example, UnmarshalArrayParallel distributes array records across CPU cores for batch jobs, while UnmarshalSlice is intended for serial parsing or individual requests. The README reports roughly 3.0 GB/s for parallel struct binding on a 16 MB payload in its Ryzen 9 7950X3D benchmark, but also recommends running the included benchmarks on your own hardware.

It is also useful for JSON arriving through an io.Reader. StreamDecoder can extract raw array blocks with NextRawBlock, or bind streamed objects to Go values with APIs such as NextChanParallel. This avoids loading an entire large array into memory at once.

For feeds containing damaged records, tolerant mode is notable: UnmarshalSliceTolerant returns decoded values plus a slice of Problem values, while NewStreamDecoder(r, reg).Tolerant(sink) reports a bad record and continues the stream. Engineers can therefore log, count, or reject malformed input instead of losing an entire batch.

There are important constraints. The library relies heavily on unsafe; ARM64 support is described as experimental; and structs containing interface{}, maps, uint or uint64, channels, or functions fail registry construction. It also accepts raw control characters in strings and preserves invalid UTF-8 differently from encoding/json. For strict specification compliance, the README recommends the standard library.

How engineers can use it

Install the package with the documented command:

go get github.com/GenshIv/silentjson

Define the destination struct, then build its registry once during startup:

var empRegistry = silentjson.BuildRegistry(reflect.TypeOf(Employee{}))

Use UnmarshalArrayParallel for large batch arrays, UnmarshalSlice for serial or request-level work, and StreamDecoder for io.Reader input. Check Registry.Err() at startup so unsupported field kinds fail during initialization rather than later.

The source buffer passed to parsing must not be modified while results are in use. Because unescaped strings can reference that buffer directly, retaining one parsed string may retain the entire input array; copy long-lived strings when that is undesirable.

The repository includes a basic example that can be run with:

go run example/main.go

It also documents go test -bench . ./... for measuring the project on your own hardware.