Benchmarks
Clause compared with Erlang/OTP on one machine: the same Erlang programs, compiled by erlc and by clau -O2, time their kernels after a warm-up call. These numbers change with every optimization.
- Geometric mean of kernel times of the compiled programs, library tests left out, Clause -O2 against the BEAM JIT (OTP)
- 1.60× faster on average
- Geometric mean of library test times, OTP library modules compiled by Clause -O2 against the BEAM JIT (OTP)
- 1.39× slower on average
- Kernels where Clause is faster
- 38 / 44
Kernel times
Median of 3 runs in milliseconds. The bar shows Clause's time divided by OTP's on a logarithmic scale.
| Kernel | OTP JIT | Clause -O2 | Clause ÷ OTP | |
|---|---|---|---|---|
| Data generation | ||||
| bigint_factorialComputes the factorial of 3,000 with big integers, 330 times | 1,110.12 | 207.56 | 0.19× | |
| binary_appendAppends 10,000 32-bit integers to a growing binary, 23,000 times | 1,076.22 | 803.84 | 0.75× | |
| list_of_tuplesBuilds a list of 200,000 small tuples, 260 times | 1,473.95 | 1,121.96 | 0.76× | |
| list_to_tupleConverts 1,000-element lists to tuples, 170,000 times | 1,500.47 | 1,191.70 | 0.79× | |
| map_putInserts 2,000 integer keys into a map one by one, 4,900 times | 1,576.24 | 1,478.73 | 0.94× | |
| string_buildBuilds 20,000 strings from integers and atoms, 1,000 times | 1,031.48 | 772.52 | 0.75× | |
| Data processing | ||||
| binary_matchSums 20,000 32-bit integers matched out of a binary, 49,000 times | 1,274.78 | 1,076.12 | 0.84× | |
| comprehensionList comprehension filtering and doubling 20,000 integers, 15,000 times | 1,418.04 | 664.73 | 0.47× | |
| int_to_listConverts 20,000 integers to strings and back, 2,000 times | 899.69 | 410.86 | 0.46× | |
| map_foldlMaps, then folds a list of 20,000 integers, 4,500 times | 1,459.92 | 616.89 | 0.42× | |
| map_updateIncrements every value of a 500-key map, 24,000 times | 1,188.35 | 648.98 | 0.55× | |
| small_mapUpdates a small state map 80 million times | 1,484.29 | 1,027.86 | 0.69× | |
| sortSorts a list of 20,000 random integers, 600 times | 1,038.95 | 454.89 | 0.44× | |
| term_to_binaryEncodes and decodes a large nested term, 4,300 times | 1,427.56 | 1,060.32 | 0.74× | |
| Distribution | ||||
| data_exchange200 round trips of a 64 KB binary and 1,000 tuples with a peer node, 3 times | 148.07 | 426.42 | 2.88× | |
| ping_pong5,000 message round trips with a peer node over TCP, 3 times | 831.18 | 1,239.78 | 1.49× | |
| reconnect20 disconnects and reconnects to a peer node, each with a round trip, 3 times | 137.11 | 66.42 | 0.48× | |
| spawnSpawns 500 processes on a peer node one after another, 3 times | 91.03 | 166.03 | 1.82× | |
| File I/O | ||||
| large_readReads a 16 MB file in 64 KB chunks, 370 times | 1,416.09 | 1,029.62 | 0.73× | |
| large_writeWrites a 16 MB file in 64 KB chunks, 200 times | 975.05 | 922.76 | 0.95× | |
| read_fileReads a whole 16 MB file in one call, 330 times | 1,488.38 | 1,418.65 | 0.95× | |
| read_lineReads a 20,000-line text file line by line, 52 times | 1,507.64 | 1,385.17 | 0.92× | |
| small_filesWrites, reads back and deletes 500 small files, 11 times | 1,256.45 | 1,143.67 | 0.91× | |
| Floating point | ||||
| callsTrapezoid sum of a polynomial, one call per step, 160 times | 1,227.88 | 487.21 | 0.40× | |
| guardsAn is_float/1 guard in a 1-million-step float loop, 120 times | 1,173.50 | 433.07 | 0.37× | |
| limitGrows a float step by step past 1e100, 80 times | 1,167.16 | 257.38 | 0.22× | |
| mandelbrotMandelbrot escape counts over a 120×120 grid, 120 times | 1,251.12 | 249.26 | 0.20× | |
| negateUnary minus and float/1 in a 1-million-step float loop, 70 times | 1,222.35 | 633.83 | 0.52× | |
| sumSums a float over a 1-million-step loop, 200 times | 1,115.14 | 472.37 | 0.42× | |
| Garbage collection | ||||
| explicit_collectForces garbage collection with 10,000 live tuples, 5,100 times | 1,524.12 | 580.09 | 0.38× | |
| grow_shrinkGrows the heap with a 200,000-element list, then drops it, 300 times | 1,052.06 | 849.05 | 0.81× | |
| long_lived_churnKeeps a sliding window of 10,000 live elements among garbage, 370 times | 1,308.67 | 1,088.48 | 0.83× | |
| short_processesSpawns 5,000 short-lived processes that each make garbage, 69 times | 1,652.53 | 1,026.00 | 0.62× | |
| Libraries | ||||
| qlcRuns the qlc transform over 27 queries and 3 modules, 11 times | 1,858.46 | 1,685.38 | 0.91× | |
| Message passing | ||||
| fan_in100 processes each send 1,000 messages to one, 25 times | 1,374.31 | 277.80 | 0.20× | |
| large_messageSends a 100,000-element list to another process and back, 560 times | 1,075.30 | 780.68 | 0.73× | |
| ping_pongBounces one message between two processes 2 million times | 1,494.43 | 810.77 | 0.54× | |
| ringPasses a token around a 1,000-process ring 3,050 times | 1,435.96 | 1,028.15 | 0.72× | |
| Loopback sockets | ||||
| tcp_connectOpens 900 TCP connections over loopback, echoing 25 messages of 1 KB on each | 1,487.16 | 1,152.30 | 0.77× | |
| tcp_echoEchoes 30,000 messages of 1 KB over loopback TCP | 1,574.40 | 1,274.58 | 0.81× | |
| udp_round_tripBounces a 1 KB datagram 27,900 times over loopback UDP | 1,504.67 | 1,234.78 | 0.82× | |
| TLS | ||||
| tls_bulkSends 1,024 chunks of 16 KB over loopback TLS 1.3, 3 times | 139.37 | 145.92 | 1.05× | |
| tls_echo5,000 TLS 1.3 echo round trips of 1 KB over loopback, 3 times | 1,018.57 | 2,052.42 | 2.01× | |
| tls_handshake50 TLS 1.3 connections, each a full handshake and one echo, 3 times | 324.40 | 636.09 | 1.96× | |
Whole programs
Wall time in seconds, startup included, median of the runs.
| Program | OTP JIT | Clause -O2 | Clause ÷ OTP | |
|---|---|---|---|---|
| bench_datagenBuilds lists, tuples, maps, binaries and big integers in bulk | 7.91 | 5.61 | 0.71× | |
| bench_dataprocSorting, folds, map updates, binary matching and conversions | 10.29 | 5.97 | 0.58× | |
| bench_distRound trips, data, remote spawns and reconnects between nodes; OTP also runs them over TLS | 5.67 | 4.78 | 0.84× | |
| bench_fileSmall and large file writes and reads, line reads | 6.96 | 6.08 | 0.87× | |
| bench_floatFloat loops: sums, Mandelbrot, limits, negation, guards and calls | 7.36 | 2.57 | 0.35× | |
| bench_gcShort-lived processes, heap growth and shrinking, live data churn | 5.79 | 3.60 | 0.62× | |
| bench_libraryThe qlc parse transform at run time, as qlc:q/1 uses it | 2.23 | 1.86 | 0.83× | |
| bench_msgPing-pong, fan-in, a process ring and large message copies | 5.63 | 3.00 | 0.53× | |
| bench_socketTCP echo, TCP connects and UDP round trips over loopback | 6.26 | 4.92 | 0.79× | |
| bench_sslTLS 1.3 echo, handshakes and bulk transfer over loopback | 2.21 | 3.87 | 1.75× |
Progress
Geometric mean of Clause's time divided by OTP's on each recorded run day. Runs before 2026-10-21 were not pinned to a CPU set, so part of the change on that day comes from the pinning.
| Date | Commit | Clause ÷ OTP |
|---|---|---|
| 2026-10-12 | a7bc0c9b | 20.2× |
| 2026-10-13 | 8a244d9c | 16.9× |
| 2026-10-14 | 55f42a9d | 16.9× |
| 2026-10-15 | 6ed3851c | 2.73× |
| 2026-10-16 | 97295c84 | 1.95× |
| 2026-10-17 | a8c3f8f6 | 1.38× |
| 2026-10-18 | 336caea7 | 1.41× |
| 2026-10-20 | 7d7a2f2e | 1.00× |
| 2026-10-21 | 71ec52f8 | 0.78× |
| 2026-10-23 | dea68508 | 0.69× |
| 2026-10-24 | af105043 | 0.65× |
| 2026-10-26 | d6eafa72 | 0.71× |
| 2026-10-30 | f4e5f3e0 | 0.67× |
| 2026-11-01 | 3265232a | 0.67× |
Executable sizes
Size of the benchmark programs as Clause builds them at each optimization level, in megabytes.
| Program | -O0 | -O2 | -O2 --lto |
|---|---|---|---|
| bench_datagen | 10.22 MB | 10.14 MB | 10.14 MB |
| bench_dataproc | 10.74 MB | 10.33 MB | 10.33 MB |
| bench_dist | 39.41 MB | 24.51 MB | 24.52 MB |
| bench_file | 30.25 MB | 18.09 MB | 18.07 MB |
| bench_float | 10.30 MB | 10.17 MB | 10.17 MB |
| bench_gc | 10.21 MB | 10.13 MB | 10.14 MB |
| bench_library | 10.56 MB | 10.26 MB | 10.27 MB |
| bench_msg | 10.23 MB | 10.14 MB | 10.14 MB |
| bench_socket | 30.33 MB | 18.13 MB | 18.10 MB |
| bench_ssl | 43.62 MB | 26.76 MB | 26.76 MB |
-O2 sizes of the last 6 benchmark runs, oldest first, one Clause commit each. The table below adds the total of all bench_* programs and the size of the clau compiler.
- 2026-10-2171ec52f8
- 2026-10-23dea68508
- 2026-10-24af105043
- 2026-10-26d6eafa72
- 2026-10-30f4e5f3e0
- 2026-11-013265232a
| Date | Commit | All bench_* programs | clau |
|---|---|---|---|
| 2026-10-21 | 71ec52f8 | 38.78 MB | 59.54 MB |
| 2026-10-23 | dea68508 | 38.79 MB | 59.54 MB |
| 2026-10-24 | af105043 | 38.81 MB | 59.65 MB |
| 2026-10-26 | d6eafa72 | 62.21 MB | 59.98 MB |
| 2026-10-30 | f4e5f3e0 | 70.63 MB | 60.75 MB |
| 2026-11-01 | 3265232a | 148.67 MB | 60.82 MB |
Setup
- Machine
- Intel64 Family 6 Model 183 Stepping 1, GenuineIntel, 32 logical CPUs, Windows 11 (10.0.26200)
- CPU set
- Performance cores 0-15, 16 of 32 logical CPUs
- Erlang/OTP
- 29, ERTS 17.1
- Clause commit
- 3265232a
- Run
- 2026-11-01
OTP and Clause run on the same CPUs: every benchmark program is pinned to the performance cores of the processor, so on a hybrid CPU no kernel lands on a slower efficiency core. Both keep one scheduler online per CPU of that set, unbound within it.
To run the benchmarks yourself, use make benchmark or make-benchmark.bat in the Clause repository; the benchmark notes describe the programs and options. Benchmark notes
Clause