Clause
← All documentation

Process heap contract

Process heaps follow the classic ERTS design: one contiguous heap block per process plus overflow fragments, self-describing cells, large binaries in shared buffers off the heap, and a copying collector that runs only at safe points. This note is the contract; runtime.md summarizes the host API.

Word layout

A term is one target word (32 or 64 bits); encodings are in abi.md. Every heap area is a sequence of objects that a walker parses from its first word:

No cell needs alignment stronger than a word. memory/heap_walk parses an area cell by cell and ProcessHeap::verify checks a whole heap. Cells hold only words and bytes, except the off-heap binary's std::shared_ptr (below), so a cell moves by copying its words.

KindWords after the headerTraced words
cons (no header)2 words in totalhead, tail
tuplen element slotsall
map2n slots: keys in exact term order, each followed by its value; at most 32 entriesall
tree_mapentry count, edit word, root map_node: 3 words (map trees)the root
map_nodeentries below, edit word, then n keys and n values (a leaf) or n children (a branch); never a termkeys and values or children
native_recordaddress of the runtime's RecordDefinition, then n field values in definition ordervalues
fun_closureaddress of the runtime's FunDefinition, then n captured values (funs)values
bignumsign word, then magnitude limbs, least significant firstnone
floating8 bytes: 1 word (64-bit) or 2 words (32-bit)none
reference8 bytes: the reference number (pids and references)none
heap_binarybit length, then data bytes rounded up to words (at most 64 bytes)none
refc_binarybit offset, bit length, the buffer's first byte (read by generated code), append end and limit, std::shared_ptr (2 words), off-heap link: 8 wordsnone
sub_binarybit offset into the original, bit length, the original heap_binary or refc_binary (never a sub-binary): 3 wordsthe original
fillern unused wordsnone

The map count is in words (entries = count / 2). Pids and ports are immediates, admitted against the runtime's issued numbers.

Off-heap binaries

A binary larger than 64 bytes is an immutable buffer that floats outside every process heap, shared by reference count (BEAM ProcBin and Binary).

Buffer reuse

A freed buffer of a power-of-two size of at least 256 bytes (the capacities of appendable binaries) stays with the thread that dropped its last reference, for that thread's next buffer of exactly that size (like heap blocks). Each thread keeps at most 32 buffers and 8 MB; the cache is freed when the thread ends. A reused buffer's bytes after the new binary's own are left as they were: no view covers them. Without this, binary append loops took fresh pages for every grown buffer (the C++ heap returns the pages of large freed blocks). New buffers are ByteBuffers (runtime/include/byte_buffer.hpp), which never zero their room either, so a port driver's reply (a whole file read) is written once and taken over as it is.

Handle references

A handle reference is a reference that also holds one off-heap buffer, as an OTP magic reference holds a resource; atomics arrays live in them (atomics and counters).

Map trees

A map with more than 32 entries (OTP's MAP_SMALL_MAP_LIMIT) is a 4-word tree_map stub naming the root of a persistent B+ tree (terms/map_tree) whose nodes are map_node cells of the same heap, as OTP keeps its HAMT nodes on the heap. Smaller maps stay map cells.

Areas

Sizing and budget

Heap blocks

Heap blocks, fragments and old areas come from memory/heap_blocks. A block a collection releases stays with its thread for that thread's next collections (as ERTS's carriers keep memory): a new block takes the smallest cached block of at least its size and at most twice it, else fresh memory. Each thread keeps at most 8 blocks and 16M words (128 MB on 64-bit targets); blocks below 4,096 words are not kept, since the C++ heap's free lists serve them cheaply. A released block that does not fit frees the oldest kept blocks first: a growing heap releases ever larger blocks, and without eviction the blocks of its early, small collections filled the cache, so a live set of millions of words took fresh pages for every young block, old area and full sweep. Without this, every collection's to-space took fresh pages from the operating system and paid a page fault per page it touched, and freeing a large block returned its pages. Cached blocks are not charged to any process or to the runtime-wide account; a thread frees its cache when it ends.

Runtime memory limit

Admission

Pointers into a process heap are created only by the compiler and the runtime inside that process, and always name an object start; there are no interior pointers to detect. Admission is an ownership check for words handed back to a process. Term::from_word(word, context) / TermAccess::admit first check storage liveness (expired_context); atoms and pids/ports must belong to that runtime (fixed atoms are shared). For compound words:

  1. The address is word-aligned inside one of the process's areas, below its top: the heap block is checked first, then fragments sorted by address, then the literal areas (areas). Addresses outside these ranges fail without a load; this does not detect old words at reused addresses.
  2. A boxed word names a header of an admitted kind (not filler); a list word names a cons cell (a word that is not a header).

After admission, TermAccess::object decodes kind, count and payload through trusted_cell, without a heap lookup; child wraps the stored word and validate checks representation only. Factories still re-admit inputs through ProcessHeap::retain, rejecting foreign compound inputs with wrong_owner.

These checks run in Debug builds only (CHECKED_ADMISSION). A Release build trusts every compound, atom, pid and port word handed to a builtin, service or factory, as ERTS trusts its Eterms: generated code only produces words of its own process, literal areas and the runtime's atoms, so the range and owner lookups bought nothing but cost on every argument. Other immediates are still classified. Tests that feed forged or foreign words expect rejection only when CHECKED_ADMISSION holds. map_find may compare another process's key read-only without that rejection. verify() remains the full check that every slot names an object start, for tests.

Roots and safe points

ProcessContext::visit_roots enumerates every root word for the collector. At a safe point nothing else holds heap words of the process:

OwnerRoot wordsNotes
Frame term slotsThe first roots slots of every frame on the stackBottom frames have none; resume and handler indices are integers
Raw frame slotsNoneSpilled native values; generated code keeps no heap word there at a safe point (reload rule)
Registersx[0..live) (ProcessStack::keep_registers)A suspended entry's arguments; every push and pop clears live
Failure channelError payload (BEAM fvalue), erlang:error/2,3 argument list, stack trace termGeneratedCallState Terms rewritten by Term::rebind, which stores only the new word; captured trace frames are descriptor pointers into code
Trap stateThe term words of a trapping builtin's TrapState (portions)Released when the builtin finishes or fails
Process dictionaryEvery key and valueUntil erased or the process ends
MailboxEvery message in the signal inbox and the message queue, including 'EXIT' and 'DOWN' messagesUntil a receive takes it; rewritten in place, so the receive cursor (a list position) and timeout deadline stay valid
Explicit rootsThe span a host passes to collect(roots)Read back after the call
Off-heap listNoneLinks are swept and relinked, not traced

No heap cell holds a pin. Atoms are immediates and the atom table is never collected. Fun cells name code through their untraced FunDefinition, which lives as long as the runtime; loaded modules are never unloaded, so neither funs nor trace descriptors need a pin. Small immediates are not roots.

As in ERTS C code, a host Term is exactly one trivially copyable word, valid until its heap's next collection or process exit; it has no pin, heap pointer, lifetime token or collection epoch (ownership). Access after either boundary reads moved or freed memory and is undefined, with no stale_term check. Registered roots are rewritten; an unregistered host Term must be rebuilt from its rewritten root word. A Term belongs to its process; other processes may only read it while the source stays stable.

The heap moves only at a safe point, and never while a reservation is open:

Any other request returns unsafe_point and changes nothing, not even the failure channel of a running generated call. Allocation never moves the heap: a request that does not fit creates a fragment.

Message buffers

A MessageBuffer (message_buffer.hpp) keeps a term outside every process: copy_of(value, sender) copies the graph from the sender's process heap into a HeapStorage of its own. The sender's ProcessContext supplies heap options, literal areas, atoms, identities and the runtime memory account; the buffer keeps its own lifetime token and shares off-heap binaries. It uses the same GraphCopy as a send; copy_to copies it into a receiver's heap. Immediates and literal roots need no storage. Factories, reservations and message buffers keep their lifetime checks (expired_context); a Term read from a buffer pins nothing and must not outlive its backing. The buffer is never collected and is freed with its last owner; its block is exactly the term's words, whatever the sender's minimum heap. Sends to other processes (arrival buffers), timer messages and monitor tags use it, so they are never re-encoded; persistent terms live in their own address range and are copied, not shared, into buffers.

Single message walk. A send to another process walks its message's graph once, before any lock (OutgoingMessage, with the GraphCopy of graph_copy.hpp); that walk sizes every later copy, and like any copy outside a process it copies persistent terms.

Message hand-off. A send whose message takes at least MessageBuffer::ADOPTED_WORDS (512) words always goes through a buffer, copied from that walk before any lock, so the sender holds no lock while copying. The receiver then takes the buffer over instead of copying it (MessageBuffer::take, HeapStorage::adopt), when the buffer has no other owner and the receiver has a heap block and no budget: the block joins its fragments at the same addresses, the buffer's off-heap binary and map lists join its own with their charges, and the buffer is left empty. Nobody else can reach the block before or after, so no word is copied twice. Adopted fragments ask for a collection only once a receive took their message and their words reach the young block's, as ERTS counts a message fragment only once received (HeapStorage::received); the next collection treats them as young data, and copies a message still queued like any root. Smaller messages are copied once, cheaper than a fragment each: straight into the heap of a receiver that does not run, under the executor lock, or right behind an arrival node for a running one. The receiver takes such an arrival by copying its words as they are and moving the pointers between them by the distance of the copy, reading the objects in address order as the collector scans its copies (relocated arrivals); the graph is not walked again.

Collection in generated code

Generated code collects only at a few safepoints where every live term already sits in a root. Everything else, including every allocating service, is a critical section that never moves the heap.

Triggers

A safepoint collects when the heap asks for it (wants_collection()); otherwise it costs one check.

TriggerCondition at the safepointERTS counterpart
Heap fullAny fragment exists: an allocation did not fit the heap block since the last collectionHeap top reaches the heap end
Off-heap binary pressure (a minor collection unless the last one freed less than half, binary pressure)Off-heap words reach the virtual binary heap limit: 46,422 words at first, after each collection twice the surviving off-heap words or the survivors plus the live heap words, whichever is more (never less than 46,422), but at most the survivors plus half of the budget left free after the heap blockbin_vheap_sz / binary virtual heap
erlang:garbage_collect/0,1,2A request of the builtin (introspection), taken at the target's next safepointExplicit full sweep

A safepoint runs a minor collection unless a full sweep is due: a garbage_collect request, off-heap binary pressure, a heap budget, or an old area without room for the mature words (generations). The public ProcessHeap::collect is always a full sweep.

The new block is sized for the live words plus the stack words in use (ERTS keeps the stack inside the heap block): a deep stack gets a larger heap, so a long recursion collects in proportion to its allocation rather than rescanning the whole stack every few hundred words.

Safepoints

PointWhereLive outside frame term slots
Function entryIn CLAUSE_enter_v1 / CLAUSE_tail_v1 (and so host invocation), before the callee frame is pushed; an inline entry takes this path whenever the heap wants a collectionThe callee's arguments x[0..arity), kept as roots (keep_registers)
Loop headCLAUSE_safepoint_v1(context) at the head of every comprehension generator loop, called only while the heap wants a collectionNothing

Every Erlang loop is either recursion, which passes a function entry per step, or a comprehension, which passes its loop head, so garbage between two safepoints is bounded by straight-line code and single service results.

Not safepoints (critical sections, which keep allocating into fragments): every other runtime service, including allocation, construction and matching services; CLAUSE_return_v1; exception propagation; and message delivery. Services may therefore hold raw heap words in C++ for their whole run, and their input arrays and outputs need no reload.

Waiting and suspended processes

A process that is not running is never collected: it is waiting in a receive, queued after a yield or trap, or not yet started, and everything it holds is already a root (its frames, the registers of the entry or continuation it will resume at, trap state and its messages). Messages sent to it are copied into fragments of its heap. Every delivery wakes a waiting process, and resuming it repeats the entry of its continuation (the wait builtin, a trap continuation or the function it yielded at), which is a function-entry safepoint: the first thing a resumed process does is collect when its heap asks for it. A process waiting in a selective receive that skips many messages therefore collects as they arrive, as ERTS collects a process when it is next scheduled. The golden executables_mailbox_collection covers hoarding, waiting with a timeout, deep recursion under message load, and a consumer acknowledging 3,000 messages within --max-heap 65536.

Allocation is not a safepoint (unlike BEAM's test_heap): that would need every service input and every SSA term live across any allocation in a root, a reload after each allocating service and a retry protocol in every service, while the two safepoints above already bound the garbage.

Reload rule

No SSA value (a value in a native register) holds a heap word across a safepoint, and no native pointer crosses one at all (already an error in lower_frames).

Failure behavior

Implementation

Prototype

tests/prototypes/safepoint holds one comprehension-style loop in post-lower_frames form (loop.ll): a term Y computed before the loop is stored to a term slot and reloaded after the loop-head safepoint. python tests/prototypes/safepoint/run.py compiles it for x86_64-pc-windows-msvc, aarch64-unknown-linux-gnu, i686-pc-windows-msvc and armv7-unknown-linux-gnueabihf at O0 and O2 and checks that a load of Y's frame word follows the safepoint call, so the optimizer never reuses the register that held Y.

Collection

A Cheney copy (memory/heap_collect), a full sweep or a minor collection (generations): allocate the new block first (failure is out_of_memory and leaves the heap untouched), then copy the object behind every root word and scan the new block left to right, copying the children of each copy. A slot is first checked against the from-space area of the previous slot (one range test), and immediates are told apart inline. Cons cells are scanned without a parse, and a boxed copy's slots are found from its header's kind alone (a per-kind count of untraced words, no size checks: the copies are of verified objects); objects of up to four words are copied word by word. A moved boxed object's header is replaced by a boxed pointer to its copy; a moved cons cell gets a zero head and a tail pointing to its copy. Forwarding preserves sharing. Roots are rewritten in place, the off-heap list is swept (copies relinked in list order, dead cells destroyed), and the old block and fragments are freed. A heap that was never allocated is not collected. CollectionStats reports words before, live words, the new heap block, the merged fragments, stack slot capacity and off-heap words.

Generations

Generational collection, as ERTS does it, without its write barrier:

Copying between heaps

ProcessHeap::add(value), equivalently value.copy_to(heap), returns a term of the destination heap (BEAM size_object and copy_struct). Source cells are read through trusted pointers, and a cons cell needs no parse:

Measurements

runtime_heap_measurements (full-mode CTest; numbers printed, not gated) builds a 100,000-element list of {Index, Float} tuples through TermFactory, walks it back through checked accessors, collects with the list as the only root and walks the copy, and creates 1,000 contexts that each hold one small tuple. It reports build and walk times, heap used and capacity words, side bytes (host allocations beyond heap backing) and bytes and heap words per context.