Process heap contract
Process heaps follow the classic ERTS design: one contiguous heap block per process plus overflow fragments, self-describing cells, large binaries in shared buffers off the heap, and a copying collector that runs only at safe points. This note is the contract; runtime.md summarizes the host API.
Word layout
A term is one target word (32 or 64 bits); encodings are in abi.md. Every heap area is a sequence of objects that a walker parses from its first word:
- Header word (primary tag
00): bits 2–6 hold theBoxedKind, bits 7 and up the number of words that follow the header. A boxed term points at its header. The count covers every prefix, payload and padding word, so the walker skips untraced payload without interpreting it. It has 25 bits on 32-bit targets (at most 33,554,431 words) and 57 on 64-bit ones (BoxHeader::MAX_COUNT): the tuple arity limit (2^24 - 1) and the largest bignum (131,072 words on 32-bit) fit by static assertion, maps check it when built, and cons cells and fixed-size cells have no large count. - Cons cell: two term words (head, tail) with no header. A list term points
at the head. A head is never a header because no term has tag
00. - Filler: the all-zero word (kind
tuple, count 0) is a one-word filler; kindfillerwith count n covers n further words. Reservations start zeroed, so reserved but unused words parse as filler. Nonempty tuples always have a nonzero count and{}is an immediate.
No cell needs alignment stronger than a word. memory/heap_walk parses an area
cell by cell and ProcessHeap::verify checks a whole heap. Cells hold only
words and bytes, except the off-heap binary's std::shared_ptr (below), so a
cell moves by copying its words.
| Kind | Words after the header | Traced words |
|---|---|---|
| cons (no header) | 2 words in total | head, tail |
tuple | n element slots | all |
map | 2n slots: keys in exact term order, each followed by its value; at most 32 entries | all |
tree_map | entry count, edit word, root map_node: 3 words (map trees) | the root |
map_node | entries below, edit word, then n keys and n values (a leaf) or n children (a branch); never a term | keys and values or children |
native_record | address of the runtime's RecordDefinition, then n field values in definition order | values |
fun_closure | address of the runtime's FunDefinition, then n captured values (funs) | values |
bignum | sign word, then magnitude limbs, least significant first | none |
floating | 8 bytes: 1 word (64-bit) or 2 words (32-bit) | none |
reference | 8 bytes: the reference number (pids and references) | none |
heap_binary | bit length, then data bytes rounded up to words (at most 64 bytes) | none |
refc_binary | bit offset, bit length, the buffer's first byte (read by generated code), append end and limit, std::shared_ptr (2 words), off-heap link: 8 words | none |
sub_binary | bit offset into the original, bit length, the original heap_binary or refc_binary (never a sub-binary): 3 words | the original |
filler | n unused words | none |
The map count is in words (entries = count / 2). Pids and ports are
immediates, admitted against the runtime's issued numbers.
Off-heap binaries
A binary larger than 64 bytes is an immutable buffer that floats outside every
process heap, shared by reference count (BEAM ProcBin and Binary).
- Its boxed
refc_binarycell holds astd::shared_ptrto the buffer, a bit offset and a bit length. Slices of a large binary (and slices of those) aresub_binarycells naming the original cell, as ERTS sub-binaries do: a slice takes no buffer reference and keeps the original alive. Copying a cell to another process copies theshared_ptr(copying between heaps), never the bytes. - Each cell is linked into its process's off-heap list through its link word. The list is the only way to find these cells' C++ state.
- Moving a cell copies its other words and move-constructs the
shared_ptrinto the new cell, so the old copy owns nothing. After a collection the list sweep relinks moved cells and destroys theshared_ptrof dead ones; teardown destroys all of them. A buffer is freed when its last cell, in any process, dies. - Each process counts its cells per buffer (
HeapStorage::buffers_); a buffer is charged once to every process that references it (its off-heap words, the virtual binary heap of ERTS) until that process's last cell for it dies, and once to the runtime-wide account from creation until the buffer is freed. std::shared_ptris two pointers on every supported STL; astatic_assertkeeps the cell size fixed at eight words after the header on both widths.
Buffer reuse
A freed buffer of a power-of-two size of at least 256 bytes (the capacities of
appendable binaries) stays with the thread that dropped its last reference, for
that thread's next buffer of exactly that size (like
heap blocks). Each thread keeps at most 32 buffers and 8 MB;
the cache is freed when the thread ends. A reused buffer's bytes after the new
binary's own are left as they were: no view covers them. Without this, binary
append loops took fresh pages for every grown buffer (the C++ heap returns the
pages of large freed blocks). New buffers are ByteBuffers
(runtime/include/byte_buffer.hpp), which never zero their room either, so a
port driver's reply (a whole file read) is written once and taken over as it is.
Handle references
A handle reference is a reference that also holds one off-heap buffer, as an OTP
magic reference holds a resource; atomics arrays live in them
(atomics and counters).
- Its cell has kind
referenceand therefc_binarylayout: eight words after the header, the 64-bit reference number in the offset and bit-length words (where a plain reference cell keeps it), then theshared_ptrand the link word. Every reference service reads only the number, so it compares, hashes, prints and encodes like a plain reference. - It is listed, moved, swept, charged and copied exactly like an off-heap binary
cell (
layout::off_heap_cell); its buffer is freed when the last copy of the reference in any process dies. - A process-wide registry maps the numbers of live handles to their buffers
(weak references), so
binary_to_term/1of an encoded handle rebuilds a handle sharing the buffer while it lives, and a plain reference after. Expired entries are dropped as the registry grows.
Map trees
A map with more than 32 entries (OTP's MAP_SMALL_MAP_LIMIT) is a 4-word
tree_map stub naming the root of a persistent B+ tree (terms/map_tree) whose
nodes are map_node cells of the same heap, as OTP keeps its HAMT nodes on the
heap. Smaller maps stay map cells.
- Leaves hold up to 16 key and value words in exact key order; branches hold up to 16 children with their first keys, and every node counts the entries below it, so lookup and access by position are O(log n). Every node but the root is at least half full.
- An update copies the path to the changed leaf (about 35 words per level) and shares every other node with the source map, which stays unchanged, so building a map key by key costs O(n log n) instead of O(n^2). It counts the words of the path first and allocates them once; a failed update rolls them back.
- Nodes are ordinary cells: collections move, promote and free them, dead
versions cost nothing, and they count in
heap_size, not as off-heap words. Copying a map to another heap (messages) copies its nodes like any cells. - Each stub and node carries an edit word: the token of the update that created it above a leaf bit. Nodes of the update's own token change in place (a multi-key update writes each node once). Tokens issued before the latest collection allow no change in place, since the collection may have promoted the nodes, and a copy to another heap clears the token.
- An owned update (
MapOperation::update_owned) whose source the compiler proved unread afterwards consumes the source stub (root[], size 0) and reuses its token while it is current, so it changes the nodes the source's own update created in place and copies the rest of the path. A failed owned update gives the tree back to the stub. The compiler marks an update owned only when the function built the source map itself (M = <map expression or comprehension>in the same clause body), readsMnowhere else and never captures it in a fun (semantic/map_ownership), and in fold loops from the second step on (owned accumulators), where the operation is chosen at run time. - One update at a small-integer key is decided by words alone
(
MapAccess::update_small_key, noTermper argument, the key search calls no virtual probe). Owned, it reuses the source stub too while its token is current: the stub is then younger than the latest collection, so writing the new root into it makes no old-to-young pointer. A fold that updates every entry copies each node at most once per collection and otherwise writes one value word per step.
Areas
- Heap. One block
[start, top, end)per process with bump allocation. It is created by the process's first allocation, sizedmax(min_heap_words, request)so that request always fits (fragments start at the minimum, at most 233 words, then grow, below), and owned by that process alone. Generated code bump-allocates small tuples, lists, floats and closures in it directly throughFrameState::heap_top/heap_limit:HeapStorage::absorbtakes those words into the top andused_words_before any heap operation (reservation, admission, collection, verification, usage queries) andexposeshows the free words again after one. Inline allocation never makes a fragment and stays off under a per-process budget. Once the block has fewer than 64 free words,exposeshows the process's newest own fragment instead (never an adopted message block), so allocation between two safepoints, such as the returns of a body recursion, stays inline after the first fragment opens (fragment windows). Without a budget each new fragment is at least as large as the fragments before it. - Literal areas (literal heap). Each module's compound literals —
strings, constant lists, tuples and maps, floats, bignum literals and
constant binaries of any size — are encoded by the compiler into one
constant byte string (
ModuleDescriptor::literals,abi/literals.hpp), which stays in the executable's read-only data. Registration decodes them once per runtime on a scratch context (pid number 0, never issued) with the runtime's atoms and term factories, then copies them into one area of the persistent term range that never retires;detail::LiteralAreasowns the area for the runtime's life and registers it as a module literal area (a lock-free sorted snapshot), so even copies that copy live persistent terms share it. Binaries over 64 bytes keep a buffer of their own. Terms of every process may point into them: admission checksHeapStorage::readable(heap, fragments, then literals), whileTermAccess::objectreads trusted pointers. The collector leaves literal words as they are (ownedstays heap-only); copies between heaps and into message buffers keep pointing at them (a literal root needs no copy); the verifier accepts them. Literal words are the runtime's own (atoms included), so each runtime has its own copy; generated code reads the claiming runtime's words fromliteral_cacheand other runtimes' throughCLAUSE_literal_v1. An owned map update never changes a literal in place: only a stub in the process's own heap qualifies. - Fragments. When a request does not fit and the heap may not move, it goes into the newest fragment if it fits there, else into a new fragment sized to fit (at least the minimum heap size) and chained to the process. The next collection merges fragments into the new heap block. A reservation lives in one area; rollback resets that area's top and drops a fragment (or the heap block) the reservation created. Allocation never moves the heap: overflow stays in fragments until the next safepoint or host collection (collection in generated code).
- Stack. Generated frames live on one flat stack per process, apart from the
heap (
ProcessStack, execution model). It has no cap by default; an optional per-processStackOptions::limit_wordsbounds it separately from the heap. - Off-heap list. As above.
- Old area. Objects that survived two collections (generations); minor collections leave it in place, a full sweep copies it into the new block and frees it.
Sizing and budget
min_heap_wordsis the process'smin_heap_size: 233 words by default (as ERTS),--min-heap-sizefor every process (ERTS+hms),{min_heap_size, N}inspawn_optorprocess_flag(min_heap_size, N)for one; a setting is rounded up to a heap size and never below the runtime default, andprocess_flagtakes effect at the next collection. Since no block is smaller, a process does not collect before it has allocated that many words.- The heap starts at
min_heap_wordsand grows along the ERTS size sequence: 12, 38, then each size is the sum of the previous two plus one up to 833,026 words, then 20% steps (heap_size_at_least). - A collection's new block is the smallest such size that keeps the words it may
receive below 75% of it: first all used words, since live data is not known
before copying. A result less than 25% live is copied once more into the size
its live data needs; failing to allocate that block keeps the larger one.
Neither is smaller than
min_heap_words. Both sizes count the process stack's words as live, as ERTS keeps the stack inside the heap block. - With a budget set (below), a new block holds at most its live words plus half
of the budget left after them and off-heap buffers (
block_limit), never less thanmin_heap_words. The other half stays free for fragments and new off-heap buffers, so garbage allocated after a collection reaches the next safepoint as a trigger instead of exhausting the budget. The first copy is sized from all used words, so a block above the limit for the words that survived is copied once more into its policy size. - There is no memory cap by default, per process or for the runtime: the heap
grows until the host refuses memory (
out_of_memory), while the sizing above keeps it near its live size. An optional per-process budget,HeapOptions::limit_bytes(defaultUNLIMITED_HEAP_BYTES), covers the heap block, fragments and the bytes of off-heap buffers this process references; exceeding it islimit_exceeded. During a collection the old and new blocks coexist; only the new block is checked against the budget, capped at the budget left after off-heap buffers. A buffer's charge returns when the process drops its last cell for it. The stack keeps its own optional cap,StackOptions::limit_words. Programs set both caps with--max-heapand--max-stack(runtime options).
Heap blocks
Heap blocks, fragments and old areas come from memory/heap_blocks. A block a
collection releases stays with its thread for that thread's next collections
(as ERTS's carriers keep memory): a new block takes the smallest cached
block of at least its size and at most twice it, else fresh memory. Each thread
keeps at most 8 blocks and 16M words (128 MB on 64-bit targets); blocks below
4,096 words are not kept, since the C++ heap's free lists serve them cheaply.
A released block that does not fit frees the oldest kept blocks first:
a growing heap releases ever larger blocks, and without eviction the blocks of
its early, small collections filled the cache, so a live set of millions of
words took fresh pages for every young block, old area and full sweep.
Without this, every collection's to-space took fresh pages from the operating
system and paid a page fault per page it touched, and freeing a large block
returned its pages. Cached blocks are not charged to any process or to the
runtime-wide account; a thread frees its cache when it ends.
Runtime memory limit
- An optional runtime-wide limit,
RuntimeOptions::memory_limit_bytes(defaultUNLIMITED_HEAP_BYTES; programs set it with--max-memory), bounds the memory of all processes together: heap blocks, fragments, off-heap buffers and stack capacity. OTP has no such limit; it is closest to running the VM under an OS memory limit, but fails one process instead of the node. - One account per runtime (
detail::RuntimeMemory, shared by every heap storage and stack) is charged when a block, fragment, off-heap buffer or stack capacity is created and released when it is dropped; a buffer shared by several processes is charged once and released when its last reference dies. Teardown returns every charge of the process.Runtime::memory_bytes()reports the total. - To each process the limit acts as a budget of the storage it owns plus what
the limit leaves (
HeapStorage::budget,room), so the sizing above keeps half of the free memory free after each collection, and a request beyond it islimit_exceeded(resource_limit) for the requesting process only; other processes keep running. Another process's garbage counts until that process collects. - A collection's to-space is charged even past the limit, because it replaces the blocks it releases at the end of the same collection.
- The stack's first 64 words live inside the process context and are charged by the first frame pushed. The stack then doubles while the limit allows it, then grows only by the frame being pushed.
Admission
Pointers into a process heap are created only by the compiler and the runtime
inside that process, and always name an object start; there are no interior
pointers to detect. Admission is an ownership check for words handed back to a
process. Term::from_word(word, context) / TermAccess::admit first check
storage liveness (expired_context); atoms and pids/ports must belong to that
runtime (fixed atoms are shared). For compound words:
- The address is word-aligned inside one of the process's areas, below its
top: the heap block is checked first, then fragments sorted by address, then the literal areas (areas). Addresses outside these ranges fail without a load; this does not detect old words at reused addresses. - A boxed word names a header of an admitted kind (not filler); a list word names a cons cell (a word that is not a header).
After admission, TermAccess::object decodes kind, count and payload through
trusted_cell, without a heap lookup; child wraps the stored word and
validate checks representation only. Factories still re-admit inputs through
ProcessHeap::retain, rejecting foreign compound inputs with wrong_owner.
These checks run in Debug builds only (CHECKED_ADMISSION). A
Release build trusts every compound, atom, pid and port word handed to a
builtin, service or factory, as ERTS trusts its Eterms: generated code only
produces words of its own process, literal areas and the runtime's atoms, so
the range and owner lookups bought nothing but cost on every argument. Other
immediates are still classified. Tests that feed forged or foreign words
expect rejection only when CHECKED_ADMISSION holds.
map_find may compare another process's key read-only without that rejection.
verify() remains the full check that every slot names an object start, for tests.
Roots and safe points
ProcessContext::visit_roots enumerates every root word for the collector. At
a safe point nothing else holds heap words of the process:
| Owner | Root words | Notes |
|---|---|---|
| Frame term slots | The first roots slots of every frame on the stack | Bottom frames have none; resume and handler indices are integers |
| Raw frame slots | None | Spilled native values; generated code keeps no heap word there at a safe point (reload rule) |
| Registers | x[0..live) (ProcessStack::keep_registers) | A suspended entry's arguments; every push and pop clears live |
| Failure channel | Error payload (BEAM fvalue), erlang:error/2,3 argument list, stack trace term | GeneratedCallState Terms rewritten by Term::rebind, which stores only the new word; captured trace frames are descriptor pointers into code |
| Trap state | The term words of a trapping builtin's TrapState (portions) | Released when the builtin finishes or fails |
| Process dictionary | Every key and value | Until erased or the process ends |
| Mailbox | Every message in the signal inbox and the message queue, including 'EXIT' and 'DOWN' messages | Until a receive takes it; rewritten in place, so the receive cursor (a list position) and timeout deadline stay valid |
| Explicit roots | The span a host passes to collect(roots) | Read back after the call |
| Off-heap list | None | Links are swept and relinked, not traced |
No heap cell holds a pin. Atoms are immediates and the atom table is never
collected. Fun cells name code through their untraced FunDefinition, which
lives as long as the runtime; loaded modules are never unloaded, so neither funs
nor trace descriptors need a pin. Small immediates are not roots.
As in ERTS C code, a host Term is exactly one trivially copyable word, valid
until its heap's next collection or process exit; it has no pin, heap pointer,
lifetime token or collection epoch (ownership). Access
after either boundary reads moved or freed memory and is undefined, with no
stale_term check. Registered roots are rewritten; an unregistered host Term
must be rebuilt from its rewritten root word. A Term belongs to its process;
other processes may only read it while the source stays stable.
The heap moves only at a safe point, and never while a reservation is open:
- an explicit host
collect()while the context runs no generated code; - a
collect()while running generated code has declared aSafePointscope, promising that it holds heap words only in the roots above. The runtime opens one only at the generated-code safepoints of the next section.
Any other request returns unsafe_point and changes nothing, not even the
failure channel of a running generated call. Allocation never moves the heap: a
request that does not fit creates a fragment.
Message buffers
A MessageBuffer
(message_buffer.hpp) keeps a term
outside every process: copy_of(value, sender) copies the graph from the
sender's process heap into a HeapStorage of its own. The sender's
ProcessContext supplies heap options, literal areas, atoms, identities and the
runtime memory account; the buffer keeps its own lifetime token and shares
off-heap binaries. It uses the same GraphCopy as a send; copy_to copies it
into a receiver's heap. Immediates and literal roots need no storage. Factories,
reservations and message buffers keep their lifetime checks (expired_context);
a Term read from a buffer pins nothing and must not outlive its backing.
The buffer is never collected and is freed with its last owner; its block is
exactly the term's words, whatever the sender's minimum heap. Sends to other
processes (arrival buffers), timer messages
and monitor tags use it, so they are never re-encoded;
persistent terms live in their own address range
and are copied, not shared, into buffers.
Single message walk. A send to another process walks its message's graph
once, before any lock (OutgoingMessage, with the GraphCopy of
graph_copy.hpp); that walk sizes every
later copy, and like any copy outside a process it copies persistent terms.
Message hand-off. A send whose message takes at least
MessageBuffer::ADOPTED_WORDS (512) words always goes through a buffer, copied
from that walk before any lock, so the sender holds no lock while copying. The
receiver then takes the buffer over instead of
copying it (MessageBuffer::take, HeapStorage::adopt), when the buffer has
no other owner and the receiver has a heap block and no budget: the block joins
its fragments at the same addresses, the buffer's off-heap binary and map lists
join its own with their charges, and the buffer is left empty. Nobody else can
reach the block before or after, so no word is copied twice. Adopted fragments
ask for a collection only once a receive took their message and their words
reach the young block's, as ERTS counts a message fragment only once received
(HeapStorage::received); the next collection treats them as young data, and
copies a message still queued like any root.
Smaller messages are copied once, cheaper than a fragment each: straight into
the heap of a receiver that does not run, under the executor lock, or right
behind an arrival node for a running one. The receiver takes such an arrival
by copying its words as they are and moving the pointers between them by the
distance of the copy, reading the objects in address order as the collector
scans its copies (relocated arrivals); the graph is not walked again.
Collection in generated code
Generated code collects only at a few safepoints where every live term already sits in a root. Everything else, including every allocating service, is a critical section that never moves the heap.
Triggers
A safepoint collects when the heap asks for it (wants_collection());
otherwise it costs one check.
| Trigger | Condition at the safepoint | ERTS counterpart |
|---|---|---|
| Heap full | Any fragment exists: an allocation did not fit the heap block since the last collection | Heap top reaches the heap end |
| Off-heap binary pressure (a minor collection unless the last one freed less than half, binary pressure) | Off-heap words reach the virtual binary heap limit: 46,422 words at first, after each collection twice the surviving off-heap words or the survivors plus the live heap words, whichever is more (never less than 46,422), but at most the survivors plus half of the budget left free after the heap block | bin_vheap_sz / binary virtual heap |
erlang:garbage_collect/0,1,2 | A request of the builtin (introspection), taken at the target's next safepoint | Explicit full sweep |
A safepoint runs a minor collection unless a full sweep is due: a
garbage_collect request, off-heap binary pressure, a heap budget, or an old
area without room for the mature words (generations). The
public ProcessHeap::collect is always a full sweep.
The new block is sized for the live words plus the stack words in use (ERTS keeps the stack inside the heap block): a deep stack gets a larger heap, so a long recursion collects in proportion to its allocation rather than rescanning the whole stack every few hundred words.
Safepoints
| Point | Where | Live outside frame term slots |
|---|---|---|
| Function entry | In CLAUSE_enter_v1 / CLAUSE_tail_v1 (and so host invocation), before the callee frame is pushed; an inline entry takes this path whenever the heap wants a collection | The callee's arguments x[0..arity), kept as roots (keep_registers) |
| Loop head | CLAUSE_safepoint_v1(context) at the head of every comprehension generator loop, called only while the heap wants a collection | Nothing |
Every Erlang loop is either recursion, which passes a function entry per step, or a comprehension, which passes its loop head, so garbage between two safepoints is bounded by straight-line code and single service results.
Not safepoints (critical sections, which keep allocating into fragments): every
other runtime service, including allocation, construction and matching services;
CLAUSE_return_v1; exception propagation; and message delivery. Services may
therefore hold raw heap words in C++ for their whole run, and their input arrays
and outputs need no reload.
Waiting and suspended processes
A process that is not running is never collected: it is waiting in a receive,
queued after a yield or trap, or not yet started, and everything it holds is
already a root (its frames, the registers of the entry or continuation it will
resume at, trap state and its messages). Messages sent to it are copied into
fragments of its heap. Every delivery wakes a waiting process, and resuming it
repeats the entry of its continuation (the wait builtin, a trap continuation or
the function it yielded at), which is a function-entry safepoint: the first
thing a resumed process does is collect when its heap asks for it. A process
waiting in a selective receive that skips many messages therefore collects as
they arrive, as ERTS collects a process when it is next scheduled. The golden
executables_mailbox_collection covers hoarding, waiting with a timeout, deep
recursion under message load, and a consumer acknowledging 3,000 messages
within --max-heap 65536.
Allocation is not a safepoint (unlike BEAM's test_heap): that would need every
service input and every SSA term live across any allocation in a root, a reload
after each allocating service and a retry protocol in every service, while the
two safepoints above already bound the garbage.
Reload rule
No SSA value (a value in a native register) holds a heap word across a
safepoint, and no native pointer crosses one at all (already an error in
lower_frames).
lower_framestreats a loop-head safepoint call like the resume point of a call: it splits the block after the call and spills every value read after it that was computed before it. A term value (a load from a term slot or a register, a value stored into a term slot, or a PHI of such values) is stored after its definition into its existing term slot or a new term slot counted in the descriptor'sroots, and reloaded before each use. Other words (small immediates, atoms, raw integers, flags) keep raw slots: a collection never changes them.- Calls spill and reload the same way, terms into term slots, so the entry safepoint sees every live term of every caller.
- The frame base stays valid across a loop-head safepoint, because a collection rewrites stack words in place and never moves the stack; every transfer re-reads it in the body prologue as before.
- Optimization runs after
lower_framesand cannot replace a reload by the older SSA value: the frame address is loaded from the context'sFrameState, and the opaque safepoint call may write every slot through it (prototype).
Failure behavior
- A collection at a safepoint never records a failure. When its new block cannot
be allocated (
out_of_memory) the heap stays as it is and execution continues with fragments. - Memory exhaustion. With no cap, memory runs out only when the host refuses
a heap block, fragment, off-heap buffer or stack growth:
out_of_memory. With an optional budget or runtime-wide limit set, a request beyond it islimit_exceeded, reported asresource_limit. Both are infrastructure failures: no handler runs, the frames unwind to the bottom frame and a program printsclau: runtime failure: entry call failed: <status>and exits with status 70 after flushing stdout and destroying the process (executables, differences). Because every collection keeps half of the budget left after its survivors free (sizing, triggers), a budget fails only when the live set no longer fits or when straight-line code between two safepoints allocates more than that half; garbage is collected first.
Implementation
ProcessStack::safepoint(live)asksProcessHeap::wants_collection()(a fragment exists, off-heap words reachedbinary_limit_words_, or a collection was requested), keepsx[0..live)as roots, opens aSafePointand collects; a failed collection is ignored.enter(and sotailandinvoke) calls it with the callee's arity before pushing;CLAUSE_safepoint_v1calls it with 0.- Comprehension lowering emits the loop-head check at the head of every
generator loop (
lowering_comprehensions). lower_framessplits each body after a safepoint call and spills crossing values as after a call. Arguments arrive in registers and are not stored on entry: an argument that crosses a call or safepoint is stored into its own argument slot after it arrives.home()keeps the argument slot or a same-block term slot when one holds the value; otherwise a term value (term_value: loaded from a term slot or register, stored into a term slot, or a PHI of those) gets a new term slot, whichplace_slotsappends to the leading term slots before the raw slots, and any other value a raw slot.collected_sizecaps the block atblock_limit(live);shrinkalso runs when the block exceeds the limit of the surviving words;collectcapsbinary_limit_words_at the survivors plus half of the budget left free after the block.- Coverage: the OTP goldens
executables_garbage_collection(more than 64 MiB allocated with a small live set: tail loops, off-heap binaries, allocating comprehension filters, deep body recursion, a caught error payload; with capped runs under--max-heap,--max-stackand--max-memory) andexecutables_heap_growth(72 MB of retained binaries);runtime_collectionnear_budgetandshared_limitcheck the budget and the runtime-wide limit.
Prototype
tests/prototypes/safepoint holds one
comprehension-style loop in post-lower_frames form (loop.ll): a term Y
computed before the loop is stored to a term slot and reloaded after the
loop-head safepoint. python tests/prototypes/safepoint/run.py compiles it for
x86_64-pc-windows-msvc, aarch64-unknown-linux-gnu, i686-pc-windows-msvc
and armv7-unknown-linux-gnueabihf at O0 and O2 and checks that a load of Y's
frame word follows the safepoint call, so the optimizer never reuses the
register that held Y.
Collection
A Cheney copy (memory/heap_collect), a full sweep or a minor collection
(generations): allocate the new block first
(failure is out_of_memory and leaves the heap untouched), then copy the
object behind every root word and scan the new block left to right, copying
the children of each copy. A slot is first checked against the from-space area
of the previous slot (one range test), and immediates are told apart inline.
Cons cells are scanned without a parse, and a boxed copy's slots are found from
its header's kind alone (a per-kind count of untraced words, no size checks:
the copies are of verified objects); objects of up to four words are copied
word by word. A moved boxed object's header
is replaced by a boxed pointer to its copy; a moved cons cell gets a zero head
and a tail pointing to its copy. Forwarding preserves sharing. Roots are
rewritten in place, the off-heap list is swept (copies relinked in list order,
dead cells destroyed), and the old block and fragments are freed. A heap that
was never allocated is not collected. CollectionStats reports words before,
live words, the new heap block, the merged fragments, stack slot capacity and
off-heap words.
Generations
Generational collection, as ERTS does it, without its write barrier:
- High-water mark. After a collection every object in the new block has survived one; the block's top becomes the high-water mark. The next minor collection copies the young block and fragments, promoting objects below the mark into the old area and the rest into a new young block.
- Old area. One block that never moves between full sweeps; a minor collection neither copies nor scans it. Created by the first minor collection that promotes, sized twice the mature words. When it cannot take the mature words, the safepoint runs a full sweep instead, which leaves everything in the new block below its high-water mark and frees the old area.
- Invariant. An immutable object only points at objects older than itself, so promoting by address never leaves an old object pointing at a young one. The runtime's in-place writes keep it: binary loops and appendable binaries write integers, and a map update changes in place only nodes created since the latest collection. Debug builds check after every minor collection that no old object points into the young block or a fragment.
- Sweeps. The off-heap binary list keeps cells of the old area, relinks forwarded ones and releases dead young ones; dead old cells wait for the next full sweep.
- Young sizing. A minor collection keeps the young block's size, as ERTS
does, and at least an eighth of the old area (ERTS). ERTS's minor block is
next_heap_size(young + stack), but its stack fills the rest of a full block, so that never grows it; Clause's stack is separate, and counting it grew the block every minor collection. Clause also grows the block so the young words above the high-water mark (all that can stay young) and the stack fill at most 75% of it: measured, fewer collections pay off more than ERTS's tighter blocks. Generated code collects only at safepoints, so allocation between two of them (the returns of a body recursion) may overflow far into fragments; when they hold a quarter of the block or more, the next block grows by them and is not shrunk. Otherwise a young block over 8,000 words less than 25% live shrinks to three times its live words (ERTS's minor rule). - Shrinking in place. Both shrinks (this one and a full sweep's) lower the block's capacity in place, as ERTS reallocs it: nothing is copied again, and the block's unused tail stays allocated, uncharged, until the next collection releases the block.
- Off-heap limit. After a minor collection the virtual binary heap limit counts the young live words only: the old area may hold dead words, which must not postpone the full sweep that frees dead old binaries.
- Binary pressure. Reaching the limit collects the young generation, as
ERTS's binary virtual heap does, which frees the binaries of dead young
cells. When such a minor collection frees less than half of the off-heap
words, the next one for binary pressure is a full sweep, which also frees the
binaries of dead old cells (
binary_sweep_due_).
Copying between heaps
ProcessHeap::add(value), equivalently value.copy_to(heap), returns a term of
the destination heap (BEAM size_object and copy_struct). Source cells are
read through trusted pointers, and a cons cell needs no parse:
- A term the destination can read (its own heap or a literal area, checked by
storage_->readable(address)) keeps its identity; other graphs are copied. Literal areas recognized by the destination runtime's registry stay shared. Both heaps must be live and belong to one runtime: a caller contract, with no cross-runtime check on compound copies or per-handle source lifetime check. Reading a source after collection or exit is undefined. Root atoms, pids and ports still go through destination admission; an expired destination reportsexpired_context. Factories still refuse foreign compound inputs (ProcessHeap::retain), so only an explicit copy moves a graph. - Copies are flat, as ERTS's default
copy_struct(without its optional sharing-preserving variant): a subterm is copied once per reach, so{T, T}copiesTtwice, and a term whose sharing doubles at every level grows exponentially, as on OTP. One walk with an explicit stack (no recursion) sums the words of every reached object; the copy is one reservation of that sum, in the heap block or a fragment, filled by a second walk in the same order that reads each cell again and points each slot at its child's copy. Both walks follow a list's tails in place, so only heads wait on the stack, and tell shared literals from heap cells with one range test against the span of all module literal areas. The first walk sizes the second's stack, so the second allocates nothing and a message buffer's words need no zeroing. Both stacks and the binary list hold their first entries inline, so a small message's copy makes no host allocation. No table maps source objects to copies, and only binary cells and map stubs keep their offsets. - An off-heap binary's copy is a new cell sharing the buffer; the destination holds the buffer (charging its own off-heap words if it held none of it) before reserving, and lists the cell only after the reservation commits.
- The source is only read. A failure (
resource_limitfor the destination budget or runtime limit,out_of_memoryfor the host) drops the buffer holds and rolls the reservation back, so both heaps and every charge are as before. A copy owns no source storage: it survives the source's collection and teardown.
Measurements
runtime_heap_measurements (full-mode CTest; numbers printed, not gated) builds
a 100,000-element list of {Index, Float} tuples through TermFactory, walks
it back through checked accessors, collects with the list as the only root and
walks the copy, and creates 1,000 contexts that each hold one small tuple. It
reports build and walk times, heap used and capacity words, side bytes (host
allocations beyond heap backing) and bytes and heap words per context.
Clause