This tests various parsers agains the toml-test suite.

"Decoder tests" check if valid TOML documents are processed correctly, "encoder tests" checks writing TOML files, and "invalid input" checks that the parser rejects invalid TOML documents. Implementations are marked as "compliant" (✅) if both the "valid" and "encoder" tests pass, otherwise they're marked as non-compliant (❌). Usually it's not a big deal to accept some technically invalid TOML; almost all failing "invalid" tests are uncommon edge cases.

The benchmarks are a simple decode of a 15k and 5M file. Results are displayed as "good", "slow", and "very slow" only. Doing a fully fair comparison is not straight-forward as some implementations will parse numbers, dates, etc. when reading the file, and others delay this parsing until the data is used. It's also not a contest on who can write the fastest parser – it's just to establish that performance is within reasonable bounds. Some parsers seem to be O(n²) or thereabouts and scale very poorly to large files.

Show only:
Name Decoder tests Encoder tests Invalid input 15k file 5M file
C++ toml++
pass: 182 fail: 0
pass: 182 fail: 0
pass: 371 fail: 0
good good output
C++ toml11
pass: 182 fail: 0
(TODO)
pass: 371 fail: 0
good slow output
C toml-c
pass: 182 fail: 0
(doesn't include encoder)
pass: 350 fail: 21
good slow output
C tomlc99
pass: 175 fail: 7
(doesn't include encoder)
pass: 293 fail: 78
good slow output
C# tomlyn
pass: 179 fail: 3
(TODO)
pass: 366 fail: 5
slow good output
Dart toml.dart
pass: 182 fail: 0
pass: 182 fail: 0
pass: 332 fail: 39
good good output
Fortran toml-f
pass: 171 fail: 11
pass: 170 fail: 12
pass: 339 fail: 32
good very slow output
Go go-toml
pass: 182 fail: 0
pass: 182 fail: 0
pass: 371 fail: 0
good slow output
Go toml
pass: 182 fail: 0
pass: 182 fail: 0
pass: 356 fail: 15
good good output
Guile toml
pass: 177 fail: 5
pass: 178 fail: 4
pass: 356 fail: 15
(TODO) (TODO) output
Haskell toml-parser
pass: 181 fail: 1
pass: 182 fail: 0
pass: 369 fail: 2
good good output
Haskell toml-reader
pass: 177 fail: 2
(doesn't include encoder)
pass: 366 fail: 2
good good output
JS j-toml
pass: 182 fail: 0
pass: 182 fail: 0
pass: 357 fail: 14
good good output
JS smol-toml
pass: 181 fail: 0
pass: 174 fail: 7
pass: 357 fail: 14
(error) (error) output
JS toml-eslint-parser
pass: 182 fail: 0
(doesn't include encoder)
pass: 362 fail: 9
good good output
Lisp clop
pass: 179 fail: 3
(doesn't include encoder)
pass: 371 fail: 0
(TODO) (TODO) output
OCAML otoml
pass: 181 fail: 1
pass: 174 fail: 8
pass: 319 fail: 52
(TODO) (TODO) output
Python toml
pass: 164 fail: 18
pass: 168 fail: 14
pass: 298 fail: 73
good good output
Python tomli
pass: 182 fail: 0
pass: 182 fail: 0
pass: 371 fail: 0
good good output
Python tomlkit
pass: 182 fail: 0
pass: 182 fail: 0
pass: 365 fail: 6
slow very slow output
Python tomllib
pass: 182 fail: 0
(doesn't include encoder)
pass: 371 fail: 0
good good output
Racket toml-racket
pass: 0 fail: 182
pass: 175 fail: 7
pass: 0 fail: 371
(TODO) (TODO) output
Ruby perfect_toml
pass: 182 fail: 0
pass: 181 fail: 1
pass: 362 fail: 9
good good output
Ruby toml-rb
pass: 155 fail: 27
pass: 154 fail: 28
pass: 281 fail: 90
(TODO) (TODO) output
Ruby tomlrb
pass: 182 fail: 0
(doesn't include encoder)
pass: 351 fail: 20
good slow output
Rust basic-toml
pass: 161 fail: 21
(TODO)
pass: 367 fail: 4
good good output
Rust taplo
pass: 178 fail: 4
(TODO)
pass: 368 fail: 3
good very slow output
Rust toml
pass: 182 fail: 0
pass: 182 fail: 0
pass: 371 fail: 0
good good output
Rust toml_edit
pass: 182 fail: 0
pass: 182 fail: 0
pass: 371 fail: 0
good good output