D4 Programming Language
Official Foundation Specification — D4/1
Language: D4 Extension: ".dem4" Primary paradigm: Mapped native systems programming Compilation: Ahead-of-time Backend: LLVM exclusively Primary executable format: PE ".exe" Primary object format: PE/COFF ".obj" Link stage: automatic "ld" invocation Machine mapping model: MASM-oriented instant lookup + reference lowering
«Describe high. Resolve immediately. Execute low.»
- What D4 Is
D4 is an ahead-of-time compiled mapped systems programming language.
Its defining idea is simple:
«Most compilation should not require repeatedly rediscovering what a construct means.»
D4 therefore organizes the language around:
- standard definitions
- mapping tables
- reference tables
- declarative labels
- operand descriptions
- typed semantic signatures
- ladder structures
- definitive lowering rules
A D4 construct does not travel through a long sequence of increasingly vague intermediate interpretations.
Instead, the compiler attempts to answer:
What is this? What does it reference? Which semantic rule owns it? Which primitive does that rule resolve to? Which machine operation family implements that primitive?
The answers are largely obtained through indexed lookup.
Conceptually:
SOURCE ↓ REFERENCE ↓ DEFINITION ↓ MAP ↓ PRIMITIVE ↓ OPCODE FAMILY
This makes D4 extremely high-level to write while deliberately making its compiler grammar small and mechanical.
- Why the Name D4 Exists
The 4 represents the four foundational programming dimensions of D4.
Dimension I — Reference-Oriented Programming
D4 programs are networks of references.
Names do not merely identify storage.
They can identify:
- values
- definitions
- layouts
- labels
- operations
- cases
- memory compartments
- boxes
- tracks
- courses
- ladders
- mappings
- error continuations
- reusable ledger entries
Reference resolution is therefore one of the central compiler operations.
Dimension II — Case-Oriented Programming
A concrete realization of a description is a:
case
A case may represent:
- an instance
- a specialization
- a scenario
- a concrete configuration
- an executable entry case
- a contextual implementation
Example:
case player of Player: name := "Lena" health := 100
D4 deliberately uses case rather than forcing every concrete entity into an object-oriented class model.
Dimension III — Description-Oriented Programming
D4 heavily embraces operand-description-oriented programming.
The programmer describes what operands are.
Operations are resolved using those descriptions.
For example:
a: i64 b: i64
already tells D4 an enormous amount about:
a + b
The compiler does not need to guess whether "+" represents:
- string concatenation
- arbitrary object dispatch
- decimal arithmetic
- vector arithmetic
- matrix composition
- pointer arithmetic
The operand descriptions constrain the available semantic mapping immediately.
Dimension IV — Aspect-Oriented Programming
Orthogonal behavior can be attached through:
directives layers labels categories policies
without rebuilding the underlying definition.
Examples:
buffer: ptr @raw @nonnull @aligned(64)
numbers: list @contiguous @bounds
box Frame @arena @stacking
The principal description stays clean.
Its operational aspects are layered onto it.
- The D4 Compilation Architecture
The canonical pipeline is:
.dem4 ↓ D4 Page Scanner ↓ Structural Token Matrix ↓ Standard Definition Resolution ↓ Reference Graph ↓ Ruleset Resolution ↓ Ladder Resolution ↓ Reference Lowering ↓ MASM Mapping Layer ↓ D4IR ↓ LLVM IR ↓ LLVM Optimization ↓ LLVM Target Backend ↓ PE/COFF .obj ↓ ld ↓ PE .exe
There is:
- no required VM
- no required bytecode
- no JIT
- no mandatory garbage collector
- no mandatory object runtime
- no requirement for the Microsoft MASM assembler
- no secondary native code generator
LLVM is the sole production machine-code backend.
- MASM Mapping Without Replacing LLVM
D4's relationship with MASM is intentionally unusual.
D4 uses a MASM Mapping Layer, or MML, as its canonical machine-facing description system.
For example, a resolved operation may map conceptually as:
i64.add ↓ MML:add.r64.r64 ↓ LLVM:add i64 ↓ target-selected ADD sequence
A memory copy could resolve:
copy ↓ MML:mov/movdqu/rep.movsb family ↓ LLVM memcpy/intrinsic/vector operations ↓ target machine code
D4 therefore has an almost direct vocabulary relationship with conventional MASM operations while allowing LLVM to decide:
- register allocation
- scheduling
- instruction selection
- SIMD selection
- instruction fusion
- vectorization
- target features
- peephole optimization
D4 may optionally emit a MASM-style assembly mirror:
d4 build app.dem4 --emit=masm
but that file is diagnostic output.
It is not required to produce the executable.
- Automatic Linking
A normal build:
d4 build app.dem4
performs:
app.dem4 ↓ LLVM ↓ app.obj ↓ ld ↓ app.exe
The D4 compiler driver resolves the appropriate PE-capable "ld".
The reference implementation permits a compatible LLVM "lld" implementation behind the "ld" contract.
The user does not ordinarily perform the linking manually.
Example:
d4 build game.dem4
produces:
game.exe
unless another output format is explicitly requested.
- The Frontend Is Natively Portable
D4's frontend contains no inherent dependency on:
- Windows APIs
- PE structures
- x86
- MASM binaries
- host calling conventions
The frontend operates upon:
source pages definitions references labels types rules maps ladders
Target knowledge enters through target descriptions after semantic resolution.
Therefore a D4 compiler running on:
Windows Linux macOS BSD
can process the same frontend language.
A Linux-hosted compiler may, for example, target:
x86-64 Windows PE
provided the corresponding LLVM target and PE-capable linker are installed.
- D4 Is a Mapped Language
A conventional compiler often asks:
What could this construct mean?
D4 prefers:
What definition does this reference?
The standard compiler contains indexed maps for:
SYMBOL MAP TYPE MAP OPERAND MAP OPERATION MAP REFERENCE MAP DIRECTIVE MAP LAYOUT MAP FAULT MAP ABI MAP CONTAINER MAP LADDER MAP TARGET MAP MACHINE MAP
Suppose the compiler encounters:
total := a + b
with:
a: i64 b: i64
The resolution is approximately:
operator + + left operand i64 + right operand i64 ↓ lookup ↓ core.add<i64,i64> ↓ primitive integer.add.64 ↓ D4IR add.i64
No generalized runtime dispatch is implied.
- Ambiguity Is Structurally Removed
D4 does not treat ambiguity as something for the compiler to creatively interpret.
Resolution follows a strict order.
Standard resolution order
- Explicit label
- Exact local definition
- Exact case definition
- Imported ledger definition
- Standard definition
- Exact typed operand mapping
- Directive-constrained mapping
- Compilation failure
There is no arbitrary "best match."
There is no hidden reinterpretation.
If two possibilities remain semantically valid, D4 requires the programmer or a standard rule to identify the intended one.
For example:
value := use convert(input) @as(i64)
or:
value := use convert(input) @map(integer)
The label or directive collapses the ambiguity.
- Declarative Labeling
Labels are one of D4's primary methods of completing definitions.
label binary_mode label hot_path label network_order
Labels can participate in resolution:
use encode(packet) @network_order
A definition may provide label-specific maps:
ledger Encode: packet @network_order -> Network.Encode
packet @host_order
-> Host.Encode
Anything that would otherwise remain undefined must eventually become resolved through:
- a standard definition
- a ruleset
- an explicit type
- a reference
- a label
- a directive
By the time D4IR exists, semantic uncertainty is forbidden.
- The Definitive D4IR
D4 uses a definitive intermediate representation called:
D4IR
The word definitive is literal.
D4IR is created only after the frontend has frozen:
types references operation identities container representations memory regions error behavior branch destinations calling rules concurrency relationships parallel relationships layout requirements pointer policies range behavior labels ABI requirements
D4IR contains no unresolved overloads.
No abstract operation reaches LLVM.
No "figure this out later" node reaches LLVM.
For example, source:
x := a + b
might become:
%r8 = add.i64 %r3, %r6
not:
%r8 = mysterious_add(%r3,%r6)
LLVM receives concrete operations.
LLVM is free to optimize their realization.
It is not free to reinterpret their meaning.
- High-Level Syntax, Low-Level Grammar
D4 deliberately separates:
surface expressiveness
from:
grammar complexity
The surface language is highly expressive.
The underlying grammar understands only a small collection of fundamental forms:
description reference label case directive connector ladder block call literal map ledger
Large language features are normally defined through combinations of those primitives.
Conceptually:
HIGH-LEVEL FEATURE
↓ definition lookup
SMALL GRAMMAR FORM
↓ map
SEMANTIC PRIMITIVES
This is one reason D4 can grow without continuously inflating its parser.
- Whitespace Is Active
D4 is indentation-sensitive by default.
case player of Player: health := 100
branch health:
0:
use die(player)
1..25:
use warn(player)
Indentation defines containment.
The compiler does not require:
{} begin end endif
for ordinary blocks.
- Horizontal Spacing
Horizontal spacing is syntactically meaningful where it separates grammatical components.
For example:
a+b
may be tokenized differently from:
a + b
Standard D4 formatting therefore strongly prefers explicit spacing around operators.
Canonical:
total := left + right
rather than:
total:=left+right
Spacing is part of the language's structural readability contract.
- Indentation
The standard indentation width is:
4 spaces
Tabs are normalized by the frontend only when the source page declares a tab policy.
Otherwise mixed indentation is rejected.
Example:
@indent(4)
Indentation creates executable structure rather than merely presentation.
- Comments
Single-line comments use:
velocity := distance / time
Block comments use:
This operation performs the complete frame-state reconstruction. *
Inline block form is also legal where unambiguous:
value := 10 * temporary diagnostic note *
Multiplication remains an infix operator:
area := width * height
The scanner distinguishes the forms using structural position and operand context.
- Pages, Not Lines
D4's compiler is fundamentally page-scanned.
It does not semantically compile one line at a time.
A source file is divided internally into logical source pages.
Each page receives its own:
token matrix label index reference index definition index map delta ladder graph dependency summary content fingerprint
The scanner first understands the page.
It then resolves relationships within that page.
Cross-page references are connected through the ledger/reference system.
Conceptually:
PAGE 1 scan → index → resolve
PAGE 2 scan → index → resolve
PAGE 3 scan → index → resolve
rather than:
line line line line line
A physical line is merely one formatting component of a larger structural page.
- Why Page Scanning Matters
Page scanning makes several operations natural:
incremental compilation parallel frontend work cached semantic pages instant reference indexing definition reuse editor integration localized recompilation dependency invalidation large-codebase navigation
If page 47 changes, the compiler can compare its new reference fingerprint against the stored ledger.
Unrelated pages do not automatically require complete semantic reconstruction.
- Primitive Types
D4's standard primitive foundation includes:
bit bool
i8 i16 i32 i64 i128
u8 u16 u32 u64 u128
f16 f32 f64 f128
byte char rune
usize isize addr
unit never
These are non-negotiable semantic primitives.
Higher-level forms ultimately resolve into combinations of these primitives plus references and machine operations.
- Explicit-Type-Driven Inference
D4 inference begins with semantic type information.
Example:
left: u64 right: u64
total := left + right
"total" requires no explicit annotation.
The mapping already determines:
u64 + u64 -> u64
Therefore:
total
becomes "u64".
Inference is not speculative.
It follows semantic signatures.
- Literals
Context determines literals whenever possible.
count: u64 := 10
Here:
10
is immediately resolved to "u64".
Without contextual information:
count := 10
the standard integer default applies.
Default:
i32
Typed literals may force another interpretation:
10u64 10i128 4.0f32
- Descriptions
A description declares semantic properties.
Player: id: u64 health: f32 name: text
A description does not inherently require one particular object model.
Its eventual representation is selected through mappings, usage, and directives.
- Cases
An instantiated or specialized description is a:
case
Example:
case hero of Player: id := 7 health := 100.0 name := "Nia"
A case may also specialize behavior:
case FastEncoder of Encoder: @vectorize @inline
Or establish an executable root:
case main of Program: use run()
- Calls Use "use"
All ordinary invocation is visibly represented by:
use
Example:
use render(scene)
With a return value:
frame := use render(scene)
Nested:
checksum := use hash(use encode(packet))
The word "use" makes invocation semantically explicit.
Merely naming something does not secretly execute it.
- Reusables Use Ledger Logic
Reusable definitions belong to D4's ledger system.
Example:
ledger Math: add(a: i64, b: i64) -> i64: a + b
square(x: i64) -> i64:
use add(x * x, 0)
Usage:
value := use Math.square(12)
The ledger records:
definition identity semantic fingerprint type signature references dependencies lowering map compiled specialization target variant
Once a compatible reusable has been definitively resolved, subsequent use becomes principally a ledger lookup rather than semantic rediscovery.
- Write Once, Resolve Once, Reuse Repeatedly
Ledger logic allows D4 to reason approximately:
Have I already solved this exact semantic problem?
If yes:
reuse definitive entry
If no:
resolve lower record reuse thereafter
Different concrete specializations receive distinct ledger identities.
For example:
sort sort sort
may share the same high-level definition while owning separate resolved entries.
- References
References are foundational.
Standard reference forms include:
ref ptr addr
"ref" is semantic and managed according to its enclosing policies.
"ptr" exposes machine-oriented addressing.
Example:
player: ref memory: ptr
- Smart References
A semantic reference may carry guarantees:
player: ref @nonnull @stable
Possible directives include:
@nonnull @stable @readonly @unique @shared @atomic @pinned @bounds @noalias
The compiler incorporates these properties into reference lowering.
- Raw Pointers
Raw memory remains available.
data: ptr @raw
Raw pointers may still receive useful directives:
data: ptr @raw @nonnull @aligned(64) @noalias
D4 therefore does not force the programmer to choose between:
complete abstraction
and:
complete compiler ignorance
A raw pointer can still communicate known facts.
- Smart Containers
Containers are semantic structures whose concrete representation is resolved before D4IR.
Standard containers include:
list array<T, N> map<K, V> set stack queue ring span slice grid text bytes
Example:
numbers: list
This does not force D4 to retain a giant generic runtime representation forever.
The container map considers:
mutability maximum size growth lifetime access patterns directives ownership alignment target
and chooses a concrete implementation.
D4IR records that choice.
- Container Directives
The programmer can constrain container lowering.
vertices: list @contiguous @aligned(64) @reserve(4096)
Or:
indices: array<u32, 8192> @stack
Or:
lookup: map<Key, Value> @fixed @readonly
Smart containers are not magical runtime objects.
They are compiler-resolved descriptions.
- Memory Uses Boxes
The highest ordinary D4 memory ownership region is a:
box
Example:
box World: ...
A box defines broad properties such as:
ownership lifetime allocation strategy visibility reclamation strategy thread accessibility
- Boxes Contain Compartments
Memory inside a box is subdivided into:
compartment
Example:
box World @heap: compartment Entities: players: list enemies: list
compartment Geometry @aligned(64):
vertices: list<Vertex>
indices: list<u32>
The model is therefore:
BOX ├── COMPARTMENT ├── COMPARTMENT └── COMPARTMENT
A box owns memory responsibility.
A compartment organizes concrete memory behavior within that responsibility.
- Memory Directives
Example:
box Frame @arena @stacking: compartment Scratch @aligned(64): temporary: bytes
Possible policies include:
@stack @heap @arena @static @thread @shared @local @aligned(N) @fixed @pinned @readonly
- Garbage and Reclamation
D4 does not collapse all reclamation into one garbage-collection mechanism.
It provides four principal reclamation concepts:
shedding trimming defer stacking
- Shedding
Shedding releases resources that are no longer needed according to their box/compartment policy.
shed Scratch
Automatic:
box Requests @shedding:
Shedding can represent:
- object destruction
- arena reclamation
- page release
- reference cleanup
- container backing-store release
depending upon the definitive representation.
- Trimming
Trimming removes excess capacity while retaining the active structure.
trim cache to 64mb
Or:
trim results
A list may remain alive while surrendering unused backing memory.
- Defer
Defer postpones reclamation or another action until an explicitly declared point.
defer shed packet until track Network done
Or:
defer use close(file) until case done
Defer is explicit continuation scheduling.
- Stacking
Stacking makes a compartment or allocation sequence reclaim in reverse acquisition order.
box Frame @stacking:
Conceptually:
allocate A allocate B allocate C
release C release B release A
The compiler may lower this into a literal stack pointer discipline, arena marker, destruction ladder, or another equivalent primitive.
- Errors
D4's fundamental error actions are:
skip rewrite reorg quit
There is no requirement for exception-style stack unwinding.
- Skip
"skip" removes a failed contribution and continues where continuation remains valid.
item := use read_item(source) ? Missing -> skip
In iteration:
for file in files: content := use read(file) ? Missing -> skip
use process(content)
A missing file simply does not contribute to that iteration.
- Rewrite
"rewrite" replaces the failed result or operation with another valid one.
config := use load_config(path) ? Missing -> rewrite DefaultConfig
Or:
result := use GPU.render(scene) ? DeviceLost -> rewrite use CPU.render(scene)
Execution continues from the original continuation point with the replacement.
- Reorg
"reorg" reorganizes control around a declared recovery structure.
state := use load_state() ? Corrupt -> reorg StateRecovery
Example recovery ladder:
ladder StateRecovery: use locate_backup() |> use validate_backup() |> use restore_backup() |> resume
Unlike "rewrite", reorganization can replace an entire execution path.
- Quit
"quit" exits an explicitly identified execution scope.
use connect(server) ? PermissionDenied -> quit case
Other forms:
quit track quit course quit box quit program
D4 does not require an uncontrolled process abort when only a smaller semantic region needs termination.
- Error Mapping Is Definitive
By D4IR, every potentially propagating operation has a definitive fault action.
The compiler cannot simply leave:
maybe this fails somehow
in the IR.
The result must be:
skip rewrite reorg quit
or a statically proven impossibility.
- Branching
Standard branching is expressive:
branch status: Ready: use begin()
Waiting:
use hold()
Failed:
use recover()
Ranges may participate:
branch temperature: ..0: use freeze_warning()
1..80:
use normal()
81..:
use heat_warning()
- Fibonacci Branch Sequencing
D4 branch lowering uses Fibonacci sequencing as one of its standard decision-layout strategies.
The sequence:
1 1 2 3 5 8 13 21 34 ...
is used to partition increasingly large branch spaces into lookup and comparison tiers.
This is not permitted to change observable program meaning.
It is a lowering strategy.
For a large decision set, D4 may organize candidate regions into Fibonacci-sized groups rather than constructing a naive linear test chain.
This helps the compiler generate:
- balanced comparison trees
- compact decision tables
- branch clusters
- hot/cold layouts
- range partitions
- speculative lookup groups
- Exponential Expansion
A complex branch description may initially imply many possible paths.
D4 is allowed to expand these relationships during resolution.
Conceptually:
A ├─ B │ ├─ D │ └─ E └─ C ├─ D └─ E
The semantic model may temporarily grow exponentially when necessary to prove exact outcomes.
- Folding and Collapsing
Equivalent paths are then folded.
Previous example:
A ├─ B ─┐ └─ C ─┴→ shared D/E continuation
Common:
- predicates
- results
- continuations
- tails
- state transitions
- memory actions
can collapse into shared definitive paths.
Thus D4 willingly permits:
EXPAND ↓ PROVE ↓ FOLD ↓ COLLAPSE
before machine lowering.
- Dynamic Ranges
Ranges are first-class descriptors.
1..10
Inclusive.
1..<10
Upper-exclusive.
Open ranges:
1.. ..100
Dynamic:
0..items.count
The endpoint is an executable description.
By default, dynamic range components are evaluated according to the enclosing iteration policy.
They can be frozen:
for i in 0..items.count @snapshot:
or remain live:
for i in 0..items.count @live:
- Dynamic Range Steps
0..100 by 4
The step itself may be dynamic:
start..finish by stride
Or produced by a call:
start..finish by use stride_for(mode)
A range therefore describes traversal rather than merely storing two integers.
- Ladders
D4 uses ladders to represent ordered semantic progression.
Many higher structures share the same underlying mechanism.
Example:
ladder Build: source |> scan |> resolve |> lower |> optimize |> emit
Each:
|>
represents a rung transition.
- Derivatives Use Ladders
derivative FullName: first_name |> separator |> last_name |> blend text
The derivative is calculated from upstream references.
- Deductions Use Ladders
deduction SafeCopy: source @nonnull |> destination @nonnull |> destination.size >= source.size |> copy_allowed
A deduction describes facts established rung by rung.
- Chains Use Ladders
chain PrepareFrame: use acquire() |> use update() |> use render() |> use present()
- Categories Use Ladders
category Integer: i8 |> i16 |> i32 |> i64 |> i128
The ladder may represent widening, capability progression, resolution priority, or another relation explicitly defined by the category.
- Families Use Ladders
family Numeric: Integer |> Floating |> VectorNumeric
Families organize related semantic domains without forcing inheritance.
- Layers Use Ladders
layer Network: RawSocket |> Transport |> Session |> Protocol |> Application
Each layer can introduce rules and references while preserving the underlying system.
- Grouping Uses Ladders
group RenderWork: Geometry |> Materials |> Lighting |> Post
Groups are semantic organization, not necessarily physical containers.
- Linear Connectors
D4 uses horizontal connectors when structure naturally reads left-to-right.
Principal examples:
-> mapping/result/direction => definitive resolution ~> deferred transition <-> bidirectional relationship
Example:
Source -> Decoder -> Frame
Or:
i32 + i32 => i32
- Vertical Connectors
Vertical structures use ladder notation.
Input |> Parse |> Resolve |> Lower |> Emit
This is semantically equivalent to an explicitly declared ladder.
Formatting therefore becomes part of D4's expressive shorthand.
- Parallelism Uses Courses
A course represents work that may execute in parallel with sibling courses.
course Geometry: mesh := use build_geometry(scene)
course Lighting: lights := use build_lighting(scene)
course Audio: sound := use prepare_audio(scene)
Courses express parallel opportunity.
They do not inherently imply three operating-system threads.
The compiler/runtime policy may realize them using:
threads work stealing thread pools SIMD GPU dispatch task graphs inline sequential execution
provided observable semantics remain unchanged.
- Concurrency Uses Tracks
A track represents independently progressing execution.
track Network: while active: packet := use receive(socket) use dispatch(packet)
track Simulation: while active: use update_world()
Tracks model concurrency.
Courses model parallel decomposition.
That distinction is fundamental.
- Courses vs Tracks
COURSE "These pieces of work may proceed together."
TRACK "These execution histories independently continue."
A course generally seeks eventual completion and combination.
A track may remain active for an extended or indefinite lifetime.
- Merging Uses Blends
Parallel or independent results are combined through:
blend
Example:
frame := blend Geometry.mesh + Lighting.lights
With a declared combination rule:
final := blend left + right by Sum
Multiple inputs:
scene := blend geometry + lighting + effects + ui by RenderScene
A blend must have deterministic merge semantics by the time D4IR is formed.
- Track Blending
Concurrent results may also blend:
state := blend track Input + track Network by EventOrder
The blending policy resolves ordering.
D4 therefore avoids leaving concurrency conflicts semantically unspecified.
- Brute Force Uses "push"
D4 makes exhaustive computation explicit with:
push
Example:
push candidate in 0..max_key: branch use test(candidate): true: quit push with candidate
"push" means:
«Intentionally exhaust this search domain until its declared success or completion condition is reached.»
The compiler may transform a push into:
- parallel courses
- SIMD batches
- GPU work
- vector searches
- partitioned ranges
- specialized machine loops
where valid.
- "push" Is Semantically Different From a Loop
A normal loop says:
perform this iteration structure
A push says:
exhaustively attack this search space
That additional semantic meaning gives the compiler much more optimization freedom.
- Rollback Uses "recall"
State checkpoints are identified through labels.
label StableState @recall
Later:
recall StableState
"recall" restores the state governed by that checkpoint.
The exact mechanism may resolve into:
snapshot restore transaction rollback journal rollback copy restore version-pointer reset inverse ledger
depending upon the affected state.
- Undo Uses "retreat"
"retreat" is not identical to rollback.
"recall" returns to a known state checkpoint.
"retreat" reverses one or more reversible semantic operations.
Example:
retreat last
Or:
retreat 3
Or:
retreat RenameUser
A ledger entry may declare its inverse mapping.
That mapping makes retreat cheap and deterministic.
- Recall vs Retreat
RECALL restore a declared previous state
RETREAT reverse declared operations
Example:
state A operation B operation C state D
recall A
restores A directly.
retreat 2
reverses C and B.
- Formatting as Shorthand
D4 allows structure itself to communicate relationships.
Verbose:
chain Pipeline: use scan(source) |> use parse(source) |> use resolve(source)
Compact:
Pipeline: scan -> parse -> resolve
If the standard definitions prove these equivalent, both resolve to the same ladder.
The compiler normalizes expressive shorthand before D4IR.
- A Complete Small Program
d4 1
ledger Math: sum(values: span) -> i64: total: i64 := 0
for value in values:
total <- total + value
total
box ProgramMemory @heap @shedding: compartment Numbers @aligned(64): values: list @contiguous
case main of Program: ProgramMemory.Numbers.values := [ 1, 1, 2, 3, 5, 8, 13, 21 ]
midpoint := ProgramMemory.Numbers.values.count / 2
course Left:
value := use Math.sum(
ProgramMemory.Numbers.values[0..<midpoint]
)
course Right:
value := use Math.sum(
ProgramMemory.Numbers.values[midpoint..]
)
total := blend Left.value + Right.value by add
use print(total)
? Busy ->
rewrite use print("output busy")
? BrokenPipe ->
quit case
- Creation and Replacement
D4 standardizes two fundamental binding actions.
Creation:
count := 10
Replacement:
count <- 20
Equality remains:
count = 20
Therefore:
:= establish <- replace = compare equality
The meanings never overlap.
- Control Flow
Conditional:
if health <= 0: use destroy(entity)
Alternative:
if ready: use begin() else: use wait()
Iteration:
for item in items: use process(item)
Conditional iteration:
while running: use update()
Pattern-style branching:
branch result: Success(value): use accept(value)
Missing:
skip
Invalid:
use reject()
- Inline MASM
Machine-specific operations remain available.
asm @masm: mov rax, rcx add rax, rdx
Inline machine assembly is treated as an explicitly constrained lowering region.
LLVM owns final integration into the surrounding function.
Register constraints, clobbers, inputs, and outputs must be declared when they cannot be derived.
Example:
asm @masm input a -> rcx input b -> rdx output rax -> result: mov rax, rcx add rax, rdx
- The Primitive Boundary
Every high-level D4 abstraction must eventually dissolve.
For example:
smart list ↓ layout ↓ pointer size capacity ↓ load/store branch arithmetic ↓ LLVM operations ↓ machine instructions
There is no permanent abstraction tax simply because the source was expressive.
The compiler is responsible for establishing a primitive realization.
- Non-Negotiable Primitive Resolution
Before definitive lowering, constructs must reduce to combinations of:
primitive values primitive arithmetic primitive comparison primitive loads/stores primitive references primitive branches primitive calls primitive atomics primitive vector operations primitive target operations
Anything incapable of reaching this boundary is not a complete D4 program.
- Standard Definitions Are Part of the Language
A large portion of D4's apparent intelligence comes from its standard definition database.
For example, the compiler may know:
list + @contiguous + fixed maximum => fixed contiguous allocation
list + unknown growth => dynamic contiguous allocation
ref + @unique => non-shared semantic reference
ptr + @raw => native machine address semantics
These are defined rules.
They are not arbitrary compiler guesses.
- Definitions Are Extensible
Libraries can contribute mappings.
Example:
ledger VectorMath: map vec4 + vec4 => simd.add.f32x4
The compiler records the extension in the ledger.
A new abstraction can therefore become almost as cheap to resolve as a built-in construct after it has been definitively mapped.
- Rulesets
Rulesets establish constraints shared by definitions.
ruleset SafeBuffer: pointer must @nonnull length must >= 0 capacity must >= length
Applied:
Buffer @rules(SafeBuffer): pointer: ptr length: usize capacity: usize
A ruleset participates directly in ambiguity elimination and lowering.
- Rules Before Optimization
D4 separates:
WHAT IS LEGAL?
from:
WHAT IS FASTEST?
First:
definitions types rules references labels
establish legal meaning.
Then optimization chooses the implementation.
This prevents optimization from becoming a semantic guessing mechanism.
- Optimization Freedom
Once D4IR exists, LLVM may aggressively perform:
constant folding dead-code elimination inlining loop unrolling vectorization SLP GVN LICM tail merging branch folding instruction combining register promotion interprocedural optimization LTO PGO machine scheduling
because D4 has already provided a definitive operation graph.
- Native Runtime Philosophy
D4 does not require a heavyweight runtime.
A program that does not request runtime services can approach:
D4 executable + minimal startup + native operating-system calls
Higher facilities can be linked only when used.
Examples:
allocator scheduler filesystem network stack wrappers thread pool Unicode support reflection metadata debug services
are modular rather than universally mandatory.
- Standard Library Shape
The foundational library is organized approximately as:
core core.math core.mem core.ref core.container core.range core.ledger
sys sys.process sys.thread sys.sync sys.file sys.net
simd atomic crypto text
platform.win platform.posix
Platform-specific facilities remain layered over portable frontend semantics.
- D4's Fundamental Compilation Principle
The language should transform:
complex source structure
into:
simple definitive execution
rather than transforming:
simple source
into:
a giant runtime responsible for discovering meaning later.
That is central to D4.
- The D4 Identity
D4 is simultaneously:
High-level
because programmers work with:
cases descriptions references families categories ladders boxes compartments tracks courses blends dynamic ranges rules ledgers
Low-level
because all of them must definitively reduce into primitive executable operations.
Fast to resolve
because definitions and mappings are aggressively indexed.
Compiler-friendly
because operand types, labels, references, and rules eliminate semantic uncertainty early.
Systems-oriented
because memory, pointers, layout, concurrency, native execution, ABI behavior, and assembly remain first-class concepts.
-
D4 in One Diagram
D4 │┌──────────────┼──────────────┐ │ │ │ DESCRIPTION REFERENCE CASE │ │ │ └──────────────┼──────────────┘ │ STANDARD MAP │ RULESET │ LABEL │ LADDER │ REFERENCE LOWERING │ DEFINITIVE D4IR │ LLVM IR │ LLVM BACKEND │ PE/COFF OBJ │ ld │ .exe
- The Four Dimensions
The conceptual architecture finally reduces to the language's namesake:
D1 — REFERENCE What does this point to?
D2 — CASE What concrete situation exists?
D3 — DESCRIPTION What does this operand/entity mean?
D4 — ASPECT What additional operational constraints apply?
Together:
REFERENCE + CASE + DESCRIPTION + ASPECT ↓ D4
The compiler uses those four dimensions to move from expressive human-facing descriptions toward definitive machine-facing execution.
- Official Language Motto
«Map the meaning. Resolve the reference. Collapse the abstraction. Execute the primitive.»
And the shorter engineering motto:
«High descriptions. Hard mappings. Definitive execution.»
- Core D4 Vocabulary
case concrete instance/specialization use invocation ledger reusable definition system label explicit semantic identity ruleset declarative constraints
box memory responsibility region compartment memory subdivision
ref semantic reference ptr machine pointer
ladder ordered semantic progression derivative value/dependency derived through a ladder deduction proven progression chain ordered execution progression category capability/type grouping family related semantic grouping layer staged semantic organization group contextual association
course parallel work track concurrent execution blend deterministic merge
push exhaustive/brute-force search
skip omit failed contribution rewrite substitute failed result/path reorg reorganize around recovery path quit terminate declared execution scope
shed release obsolete resources trim reduce excess capacity defer postpone action/reclamation stack impose LIFO resource discipline
recall rollback to checkpoint retreat semantically undo reversible operations
branch decision structure range dynamic traversal descriptor
- D4's Foundational Character
D4 does not ask the programmer to live at machine level.
It asks the compiler to be extremely good at getting there.
The source may say:
course Left: value := use solve(left)
course Right: value := use solve(right)
answer := blend Left.value + Right.value
while the final executable may contain little more than:
loads integer/vector operations calls branches register transfers stores
The distance between those two representations is bridged through mappings rather than semantic improvisation.
That is D4's defining strength.
The language is not attempting to hide the machine.
It is attempting to make the route to the machine predefined, indexed, deterministic, reusable, and brutally direct.
D4 Programming Language
Official Foundation Specification — D4/1
Language: D4 Extension: ".dem4" Primary paradigm: Mapped native systems programming Compilation: Ahead-of-time Backend: LLVM exclusively Primary executable format: PE ".exe" Primary object format: PE/COFF ".obj" Link stage: automatic "ld" invocation Machine mapping model: MASM-oriented instant lookup + reference lowering
«Describe high. Resolve immediately. Execute low.»
- What D4 Is
D4 is an ahead-of-time compiled mapped systems programming language.
Its defining idea is simple:
«Most compilation should not require repeatedly rediscovering what a construct means.»
D4 therefore organizes the language around:
- standard definitions
- mapping tables
- reference tables
- declarative labels
- operand descriptions
- typed semantic signatures
- ladder structures
- definitive lowering rules
A D4 construct does not travel through a long sequence of increasingly vague intermediate interpretations.
Instead, the compiler attempts to answer:
What is this? What does it reference? Which semantic rule owns it? Which primitive does that rule resolve to? Which machine operation family implements that primitive?
The answers are largely obtained through indexed lookup.
Conceptually:
SOURCE ↓ REFERENCE ↓ DEFINITION ↓ MAP ↓ PRIMITIVE ↓ OPCODE FAMILY
This makes D4 extremely high-level to write while deliberately making its compiler grammar small and mechanical.
- Why the Name D4 Exists
The 4 represents the four foundational programming dimensions of D4.
Dimension I — Reference-Oriented Programming
D4 programs are networks of references.
Names do not merely identify storage.
They can identify:
- values
- definitions
- layouts
- labels
- operations
- cases
- memory compartments
- boxes
- tracks
- courses
- ladders
- mappings
- error continuations
- reusable ledger entries
Reference resolution is therefore one of the central compiler operations.
Dimension II — Case-Oriented Programming
A concrete realization of a description is a:
case
A case may represent:
- an instance
- a specialization
- a scenario
- a concrete configuration
- an executable entry case
- a contextual implementation
Example:
case player of Player: name := "Lena" health := 100
D4 deliberately uses case rather than forcing every concrete entity into an object-oriented class model.
Dimension III — Description-Oriented Programming
D4 heavily embraces operand-description-oriented programming.
The programmer describes what operands are.
Operations are resolved using those descriptions.
For example:
a: i64 b: i64
already tells D4 an enormous amount about:
a + b
The compiler does not need to guess whether "+" represents:
- string concatenation
- arbitrary object dispatch
- decimal arithmetic
- vector arithmetic
- matrix composition
- pointer arithmetic
The operand descriptions constrain the available semantic mapping immediately.
Dimension IV — Aspect-Oriented Programming
Orthogonal behavior can be attached through:
directives layers labels categories policies
without rebuilding the underlying definition.
Examples:
buffer: ptr @raw @nonnull @aligned(64)
numbers: list @contiguous @bounds
box Frame @arena @stacking
The principal description stays clean.
Its operational aspects are layered onto it.
- The D4 Compilation Architecture
The canonical pipeline is:
.dem4 ↓ D4 Page Scanner ↓ Structural Token Matrix ↓ Standard Definition Resolution ↓ Reference Graph ↓ Ruleset Resolution ↓ Ladder Resolution ↓ Reference Lowering ↓ MASM Mapping Layer ↓ D4IR ↓ LLVM IR ↓ LLVM Optimization ↓ LLVM Target Backend ↓ PE/COFF .obj ↓ ld ↓ PE .exe
There is:
- no required VM
- no required bytecode
- no JIT
- no mandatory garbage collector
- no mandatory object runtime
- no requirement for the Microsoft MASM assembler
- no secondary native code generator
LLVM is the sole production machine-code backend.
- MASM Mapping Without Replacing LLVM
D4's relationship with MASM is intentionally unusual.
D4 uses a MASM Mapping Layer, or MML, as its canonical machine-facing description system.
For example, a resolved operation may map conceptually as:
i64.add ↓ MML:add.r64.r64 ↓ LLVM:add i64 ↓ target-selected ADD sequence
A memory copy could resolve:
copy ↓ MML:mov/movdqu/rep.movsb family ↓ LLVM memcpy/intrinsic/vector operations ↓ target machine code
D4 therefore has an almost direct vocabulary relationship with conventional MASM operations while allowing LLVM to decide:
- register allocation
- scheduling
- instruction selection
- SIMD selection
- instruction fusion
- vectorization
- target features
- peephole optimization
D4 may optionally emit a MASM-style assembly mirror:
d4 build app.dem4 --emit=masm
but that file is diagnostic output.
It is not required to produce the executable.
- Automatic Linking
A normal build:
d4 build app.dem4
performs:
app.dem4 ↓ LLVM ↓ app.obj ↓ ld ↓ app.exe
The D4 compiler driver resolves the appropriate PE-capable "ld".
The reference implementation permits a compatible LLVM "lld" implementation behind the "ld" contract.
The user does not ordinarily perform the linking manually.
Example:
d4 build game.dem4
produces:
game.exe
unless another output format is explicitly requested.
- The Frontend Is Natively Portable
D4's frontend contains no inherent dependency on:
- Windows APIs
- PE structures
- x86
- MASM binaries
- host calling conventions
The frontend operates upon:
source pages definitions references labels types rules maps ladders
Target knowledge enters through target descriptions after semantic resolution.
Therefore a D4 compiler running on:
Windows Linux macOS BSD
can process the same frontend language.
A Linux-hosted compiler may, for example, target:
x86-64 Windows PE
provided the corresponding LLVM target and PE-capable linker are installed.
- D4 Is a Mapped Language
A conventional compiler often asks:
What could this construct mean?
D4 prefers:
What definition does this reference?
The standard compiler contains indexed maps for:
SYMBOL MAP TYPE MAP OPERAND MAP OPERATION MAP REFERENCE MAP DIRECTIVE MAP LAYOUT MAP FAULT MAP ABI MAP CONTAINER MAP LADDER MAP TARGET MAP MACHINE MAP
Suppose the compiler encounters:
total := a + b
with:
a: i64 b: i64
The resolution is approximately:
operator + + left operand i64 + right operand i64 ↓ lookup ↓ core.add<i64,i64> ↓ primitive integer.add.64 ↓ D4IR add.i64
No generalized runtime dispatch is implied.
- Ambiguity Is Structurally Removed
D4 does not treat ambiguity as something for the compiler to creatively interpret.
Resolution follows a strict order.
Standard resolution order
- Explicit label
- Exact local definition
- Exact case definition
- Imported ledger definition
- Standard definition
- Exact typed operand mapping
- Directive-constrained mapping
- Compilation failure
There is no arbitrary "best match."
There is no hidden reinterpretation.
If two possibilities remain semantically valid, D4 requires the programmer or a standard rule to identify the intended one.
For example:
value := use convert(input) @as(i64)
or:
value := use convert(input) @map(integer)
The label or directive collapses the ambiguity.
- Declarative Labeling
Labels are one of D4's primary methods of completing definitions.
label binary_mode label hot_path label network_order
Labels can participate in resolution:
use encode(packet) @network_order
A definition may provide label-specific maps:
ledger Encode: packet @network_order -> Network.Encode
packet @host_order
-> Host.Encode
Anything that would otherwise remain undefined must eventually become resolved through:
- a standard definition
- a ruleset
- an explicit type
- a reference
- a label
- a directive
By the time D4IR exists, semantic uncertainty is forbidden.
- The Definitive D4IR
D4 uses a definitive intermediate representation called:
D4IR
The word definitive is literal.
D4IR is created only after the frontend has frozen:
types references operation identities container representations memory regions error behavior branch destinations calling rules concurrency relationships parallel relationships layout requirements pointer policies range behavior labels ABI requirements
D4IR contains no unresolved overloads.
No abstract operation reaches LLVM.
No "figure this out later" node reaches LLVM.
For example, source:
x := a + b
might become:
%r8 = add.i64 %r3, %r6
not:
%r8 = mysterious_add(%r3,%r6)
LLVM receives concrete operations.
LLVM is free to optimize their realization.
It is not free to reinterpret their meaning.
- High-Level Syntax, Low-Level Grammar
D4 deliberately separates:
surface expressiveness
from:
grammar complexity
The surface language is highly expressive.
The underlying grammar understands only a small collection of fundamental forms:
description reference label case directive connector ladder block call literal map ledger
Large language features are normally defined through combinations of those primitives.
Conceptually:
HIGH-LEVEL FEATURE
↓ definition lookup
SMALL GRAMMAR FORM
↓ map
SEMANTIC PRIMITIVES
This is one reason D4 can grow without continuously inflating its parser.
- Whitespace Is Active
D4 is indentation-sensitive by default.
case player of Player: health := 100
branch health:
0:
use die(player)
1..25:
use warn(player)
Indentation defines containment.
The compiler does not require:
{} begin end endif
for ordinary blocks.
- Horizontal Spacing
Horizontal spacing is syntactically meaningful where it separates grammatical components.
For example:
a+b
may be tokenized differently from:
a + b
Standard D4 formatting therefore strongly prefers explicit spacing around operators.
Canonical:
total := left + right
rather than:
total:=left+right
Spacing is part of the language's structural readability contract.
- Indentation
The standard indentation width is:
4 spaces
Tabs are normalized by the frontend only when the source page declares a tab policy.
Otherwise mixed indentation is rejected.
Example:
@indent(4)
Indentation creates executable structure rather than merely presentation.
- Comments
Single-line comments use:
velocity := distance / time
Block comments use:
This operation performs the complete frame-state reconstruction. *
Inline block form is also legal where unambiguous:
value := 10 * temporary diagnostic note *
Multiplication remains an infix operator:
area := width * height
The scanner distinguishes the forms using structural position and operand context.
- Pages, Not Lines
D4's compiler is fundamentally page-scanned.
It does not semantically compile one line at a time.
A source file is divided internally into logical source pages.
Each page receives its own:
token matrix label index reference index definition index map delta ladder graph dependency summary content fingerprint
The scanner first understands the page.
It then resolves relationships within that page.
Cross-page references are connected through the ledger/reference system.
Conceptually:
PAGE 1 scan → index → resolve
PAGE 2 scan → index → resolve
PAGE 3 scan → index → resolve
rather than:
line line line line line
A physical line is merely one formatting component of a larger structural page.
- Why Page Scanning Matters
Page scanning makes several operations natural:
incremental compilation parallel frontend work cached semantic pages instant reference indexing definition reuse editor integration localized recompilation dependency invalidation large-codebase navigation
If page 47 changes, the compiler can compare its new reference fingerprint against the stored ledger.
Unrelated pages do not automatically require complete semantic reconstruction.
- Primitive Types
D4's standard primitive foundation includes:
bit bool
i8 i16 i32 i64 i128
u8 u16 u32 u64 u128
f16 f32 f64 f128
byte char rune
usize isize addr
unit never
These are non-negotiable semantic primitives.
Higher-level forms ultimately resolve into combinations of these primitives plus references and machine operations.
- Explicit-Type-Driven Inference
D4 inference begins with semantic type information.
Example:
left: u64 right: u64
total := left + right
"total" requires no explicit annotation.
The mapping already determines:
u64 + u64 -> u64
Therefore:
total
becomes "u64".
Inference is not speculative.
It follows semantic signatures.
- Literals
Context determines literals whenever possible.
count: u64 := 10
Here:
10
is immediately resolved to "u64".
Without contextual information:
count := 10
the standard integer default applies.
Default:
i32
Typed literals may force another interpretation:
10u64 10i128 4.0f32
- Descriptions
A description declares semantic properties.
Player: id: u64 health: f32 name: text
A description does not inherently require one particular object model.
Its eventual representation is selected through mappings, usage, and directives.
- Cases
An instantiated or specialized description is a:
case
Example:
case hero of Player: id := 7 health := 100.0 name := "Nia"
A case may also specialize behavior:
case FastEncoder of Encoder: @vectorize @inline
Or establish an executable root:
case main of Program: use run()
- Calls Use "use"
All ordinary invocation is visibly represented by:
use
Example:
use render(scene)
With a return value:
frame := use render(scene)
Nested:
checksum := use hash(use encode(packet))
The word "use" makes invocation semantically explicit.
Merely naming something does not secretly execute it.
- Reusables Use Ledger Logic
Reusable definitions belong to D4's ledger system.
Example:
ledger Math: add(a: i64, b: i64) -> i64: a + b
square(x: i64) -> i64:
use add(x * x, 0)
Usage:
value := use Math.square(12)
The ledger records:
definition identity semantic fingerprint type signature references dependencies lowering map compiled specialization target variant
Once a compatible reusable has been definitively resolved, subsequent use becomes principally a ledger lookup rather than semantic rediscovery.
- Write Once, Resolve Once, Reuse Repeatedly
Ledger logic allows D4 to reason approximately:
Have I already solved this exact semantic problem?
If yes:
reuse definitive entry
If no:
resolve lower record reuse thereafter
Different concrete specializations receive distinct ledger identities.
For example:
sort sort sort
may share the same high-level definition while owning separate resolved entries.
- References
References are foundational.
Standard reference forms include:
ref ptr addr
"ref" is semantic and managed according to its enclosing policies.
"ptr" exposes machine-oriented addressing.
Example:
player: ref memory: ptr
- Smart References
A semantic reference may carry guarantees:
player: ref @nonnull @stable
Possible directives include:
@nonnull @stable @readonly @unique @shared @atomic @pinned @bounds @noalias
The compiler incorporates these properties into reference lowering.
- Raw Pointers
Raw memory remains available.
data: ptr @raw
Raw pointers may still receive useful directives:
data: ptr @raw @nonnull @aligned(64) @noalias
D4 therefore does not force the programmer to choose between:
complete abstraction
and:
complete compiler ignorance
A raw pointer can still communicate known facts.
- Smart Containers
Containers are semantic structures whose concrete representation is resolved before D4IR.
Standard containers include:
list array<T, N> map<K, V> set stack queue ring span slice grid text bytes
Example:
numbers: list
This does not force D4 to retain a giant generic runtime representation forever.
The container map considers:
mutability maximum size growth lifetime access patterns directives ownership alignment target
and chooses a concrete implementation.
D4IR records that choice.
- Container Directives
The programmer can constrain container lowering.
vertices: list @contiguous @aligned(64) @reserve(4096)
Or:
indices: array<u32, 8192> @stack
Or:
lookup: map<Key, Value> @fixed @readonly
Smart containers are not magical runtime objects.
They are compiler-resolved descriptions.
- Memory Uses Boxes
The highest ordinary D4 memory ownership region is a:
box
Example:
box World: ...
A box defines broad properties such as:
ownership lifetime allocation strategy visibility reclamation strategy thread accessibility
- Boxes Contain Compartments
Memory inside a box is subdivided into:
compartment
Example:
box World @heap: compartment Entities: players: list enemies: list
compartment Geometry @aligned(64):
vertices: list<Vertex>
indices: list<u32>
The model is therefore:
BOX ├── COMPARTMENT ├── COMPARTMENT └── COMPARTMENT
A box owns memory responsibility.
A compartment organizes concrete memory behavior within that responsibility.
- Memory Directives
Example:
box Frame @arena @stacking: compartment Scratch @aligned(64): temporary: bytes
Possible policies include:
@stack @heap @arena @static @thread @shared @local @aligned(N) @fixed @pinned @readonly
- Garbage and Reclamation
D4 does not collapse all reclamation into one garbage-collection mechanism.
It provides four principal reclamation concepts:
shedding trimming defer stacking
- Shedding
Shedding releases resources that are no longer needed according to their box/compartment policy.
shed Scratch
Automatic:
box Requests @shedding:
Shedding can represent:
- object destruction
- arena reclamation
- page release
- reference cleanup
- container backing-store release
depending upon the definitive representation.
- Trimming
Trimming removes excess capacity while retaining the active structure.
trim cache to 64mb
Or:
trim results
A list may remain alive while surrendering unused backing memory.
- Defer
Defer postpones reclamation or another action until an explicitly declared point.
defer shed packet until track Network done
Or:
defer use close(file) until case done
Defer is explicit continuation scheduling.
- Stacking
Stacking makes a compartment or allocation sequence reclaim in reverse acquisition order.
box Frame @stacking:
Conceptually:
allocate A allocate B allocate C
release C release B release A
The compiler may lower this into a literal stack pointer discipline, arena marker, destruction ladder, or another equivalent primitive.
- Errors
D4's fundamental error actions are:
skip rewrite reorg quit
There is no requirement for exception-style stack unwinding.
- Skip
"skip" removes a failed contribution and continues where continuation remains valid.
item := use read_item(source) ? Missing -> skip
In iteration:
for file in files: content := use read(file) ? Missing -> skip
use process(content)
A missing file simply does not contribute to that iteration.
- Rewrite
"rewrite" replaces the failed result or operation with another valid one.
config := use load_config(path) ? Missing -> rewrite DefaultConfig
Or:
result := use GPU.render(scene) ? DeviceLost -> rewrite use CPU.render(scene)
Execution continues from the original continuation point with the replacement.
- Reorg
"reorg" reorganizes control around a declared recovery structure.
state := use load_state() ? Corrupt -> reorg StateRecovery
Example recovery ladder:
ladder StateRecovery: use locate_backup() |> use validate_backup() |> use restore_backup() |> resume
Unlike "rewrite", reorganization can replace an entire execution path.
- Quit
"quit" exits an explicitly identified execution scope.
use connect(server) ? PermissionDenied -> quit case
Other forms:
quit track quit course quit box quit program
D4 does not require an uncontrolled process abort when only a smaller semantic region needs termination.
- Error Mapping Is Definitive
By D4IR, every potentially propagating operation has a definitive fault action.
The compiler cannot simply leave:
maybe this fails somehow
in the IR.
The result must be:
skip rewrite reorg quit
or a statically proven impossibility.
- Branching
Standard branching is expressive:
branch status: Ready: use begin()
Waiting:
use hold()
Failed:
use recover()
Ranges may participate:
branch temperature: ..0: use freeze_warning()
1..80:
use normal()
81..:
use heat_warning()
- Fibonacci Branch Sequencing
D4 branch lowering uses Fibonacci sequencing as one of its standard decision-layout strategies.
The sequence:
1 1 2 3 5 8 13 21 34 ...
is used to partition increasingly large branch spaces into lookup and comparison tiers.
This is not permitted to change observable program meaning.
It is a lowering strategy.
For a large decision set, D4 may organize candidate regions into Fibonacci-sized groups rather than constructing a naive linear test chain.
This helps the compiler generate:
- balanced comparison trees
- compact decision tables
- branch clusters
- hot/cold layouts
- range partitions
- speculative lookup groups
- Exponential Expansion
A complex branch description may initially imply many possible paths.
D4 is allowed to expand these relationships during resolution.
Conceptually:
A ├─ B │ ├─ D │ └─ E └─ C ├─ D └─ E
The semantic model may temporarily grow exponentially when necessary to prove exact outcomes.
- Folding and Collapsing
Equivalent paths are then folded.
Previous example:
A ├─ B ─┐ └─ C ─┴→ shared D/E continuation
Common:
- predicates
- results
- continuations
- tails
- state transitions
- memory actions
can collapse into shared definitive paths.
Thus D4 willingly permits:
EXPAND ↓ PROVE ↓ FOLD ↓ COLLAPSE
before machine lowering.
- Dynamic Ranges
Ranges are first-class descriptors.
1..10
Inclusive.
1..<10
Upper-exclusive.
Open ranges:
1.. ..100
Dynamic:
0..items.count
The endpoint is an executable description.
By default, dynamic range components are evaluated according to the enclosing iteration policy.
They can be frozen:
for i in 0..items.count @snapshot:
or remain live:
for i in 0..items.count @live:
- Dynamic Range Steps
0..100 by 4
The step itself may be dynamic:
start..finish by stride
Or produced by a call:
start..finish by use stride_for(mode)
A range therefore describes traversal rather than merely storing two integers.
- Ladders
D4 uses ladders to represent ordered semantic progression.
Many higher structures share the same underlying mechanism.
Example:
ladder Build: source |> scan |> resolve |> lower |> optimize |> emit
Each:
|>
represents a rung transition.
- Derivatives Use Ladders
derivative FullName: first_name |> separator |> last_name |> blend text
The derivative is calculated from upstream references.
- Deductions Use Ladders
deduction SafeCopy: source @nonnull |> destination @nonnull |> destination.size >= source.size |> copy_allowed
A deduction describes facts established rung by rung.
- Chains Use Ladders
chain PrepareFrame: use acquire() |> use update() |> use render() |> use present()
- Categories Use Ladders
category Integer: i8 |> i16 |> i32 |> i64 |> i128
The ladder may represent widening, capability progression, resolution priority, or another relation explicitly defined by the category.
- Families Use Ladders
family Numeric: Integer |> Floating |> VectorNumeric
Families organize related semantic domains without forcing inheritance.
- Layers Use Ladders
layer Network: RawSocket |> Transport |> Session |> Protocol |> Application
Each layer can introduce rules and references while preserving the underlying system.
- Grouping Uses Ladders
group RenderWork: Geometry |> Materials |> Lighting |> Post
Groups are semantic organization, not necessarily physical containers.
- Linear Connectors
D4 uses horizontal connectors when structure naturally reads left-to-right.
Principal examples:
-> mapping/result/direction => definitive resolution ~> deferred transition <-> bidirectional relationship
Example:
Source -> Decoder -> Frame
Or:
i32 + i32 => i32
- Vertical Connectors
Vertical structures use ladder notation.
Input |> Parse |> Resolve |> Lower |> Emit
This is semantically equivalent to an explicitly declared ladder.
Formatting therefore becomes part of D4's expressive shorthand.
- Parallelism Uses Courses
A course represents work that may execute in parallel with sibling courses.
course Geometry: mesh := use build_geometry(scene)
course Lighting: lights := use build_lighting(scene)
course Audio: sound := use prepare_audio(scene)
Courses express parallel opportunity.
They do not inherently imply three operating-system threads.
The compiler/runtime policy may realize them using:
threads work stealing thread pools SIMD GPU dispatch task graphs inline sequential execution
provided observable semantics remain unchanged.
- Concurrency Uses Tracks
A track represents independently progressing execution.
track Network: while active: packet := use receive(socket) use dispatch(packet)
track Simulation: while active: use update_world()
Tracks model concurrency.
Courses model parallel decomposition.
That distinction is fundamental.
- Courses vs Tracks
COURSE "These pieces of work may proceed together."
TRACK "These execution histories independently continue."
A course generally seeks eventual completion and combination.
A track may remain active for an extended or indefinite lifetime.
- Merging Uses Blends
Parallel or independent results are combined through:
blend
Example:
frame := blend Geometry.mesh + Lighting.lights
With a declared combination rule:
final := blend left + right by Sum
Multiple inputs:
scene := blend geometry + lighting + effects + ui by RenderScene
A blend must have deterministic merge semantics by the time D4IR is formed.
- Track Blending
Concurrent results may also blend:
state := blend track Input + track Network by EventOrder
The blending policy resolves ordering.
D4 therefore avoids leaving concurrency conflicts semantically unspecified.
- Brute Force Uses "push"
D4 makes exhaustive computation explicit with:
push
Example:
push candidate in 0..max_key: branch use test(candidate): true: quit push with candidate
"push" means:
«Intentionally exhaust this search domain until its declared success or completion condition is reached.»
The compiler may transform a push into:
- parallel courses
- SIMD batches
- GPU work
- vector searches
- partitioned ranges
- specialized machine loops
where valid.
- "push" Is Semantically Different From a Loop
A normal loop says:
perform this iteration structure
A push says:
exhaustively attack this search space
That additional semantic meaning gives the compiler much more optimization freedom.
- Rollback Uses "recall"
State checkpoints are identified through labels.
label StableState @recall
Later:
recall StableState
"recall" restores the state governed by that checkpoint.
The exact mechanism may resolve into:
snapshot restore transaction rollback journal rollback copy restore version-pointer reset inverse ledger
depending upon the affected state.
- Undo Uses "retreat"
"retreat" is not identical to rollback.
"recall" returns to a known state checkpoint.
"retreat" reverses one or more reversible semantic operations.
Example:
retreat last
Or:
retreat 3
Or:
retreat RenameUser
A ledger entry may declare its inverse mapping.
That mapping makes retreat cheap and deterministic.
- Recall vs Retreat
RECALL restore a declared previous state
RETREAT reverse declared operations
Example:
state A operation B operation C state D
recall A
restores A directly.
retreat 2
reverses C and B.
- Formatting as Shorthand
D4 allows structure itself to communicate relationships.
Verbose:
chain Pipeline: use scan(source) |> use parse(source) |> use resolve(source)
Compact:
Pipeline: scan -> parse -> resolve
If the standard definitions prove these equivalent, both resolve to the same ladder.
The compiler normalizes expressive shorthand before D4IR.
- A Complete Small Program
d4 1
ledger Math: sum(values: span) -> i64: total: i64 := 0
for value in values:
total <- total + value
total
box ProgramMemory @heap @shedding: compartment Numbers @aligned(64): values: list @contiguous
case main of Program: ProgramMemory.Numbers.values := [ 1, 1, 2, 3, 5, 8, 13, 21 ]
midpoint := ProgramMemory.Numbers.values.count / 2
course Left:
value := use Math.sum(
ProgramMemory.Numbers.values[0..<midpoint]
)
course Right:
value := use Math.sum(
ProgramMemory.Numbers.values[midpoint..]
)
total := blend Left.value + Right.value by add
use print(total)
? Busy ->
rewrite use print("output busy")
? BrokenPipe ->
quit case
- Creation and Replacement
D4 standardizes two fundamental binding actions.
Creation:
count := 10
Replacement:
count <- 20
Equality remains:
count = 20
Therefore:
:= establish <- replace = compare equality
The meanings never overlap.
- Control Flow
Conditional:
if health <= 0: use destroy(entity)
Alternative:
if ready: use begin() else: use wait()
Iteration:
for item in items: use process(item)
Conditional iteration:
while running: use update()
Pattern-style branching:
branch result: Success(value): use accept(value)
Missing:
skip
Invalid:
use reject()
- Inline MASM
Machine-specific operations remain available.
asm @masm: mov rax, rcx add rax, rdx
Inline machine assembly is treated as an explicitly constrained lowering region.
LLVM owns final integration into the surrounding function.
Register constraints, clobbers, inputs, and outputs must be declared when they cannot be derived.
Example:
asm @masm input a -> rcx input b -> rdx output rax -> result: mov rax, rcx add rax, rdx
- The Primitive Boundary
Every high-level D4 abstraction must eventually dissolve.
For example:
smart list ↓ layout ↓ pointer size capacity ↓ load/store branch arithmetic ↓ LLVM operations ↓ machine instructions
There is no permanent abstraction tax simply because the source was expressive.
The compiler is responsible for establishing a primitive realization.
- Non-Negotiable Primitive Resolution
Before definitive lowering, constructs must reduce to combinations of:
primitive values primitive arithmetic primitive comparison primitive loads/stores primitive references primitive branches primitive calls primitive atomics primitive vector operations primitive target operations
Anything incapable of reaching this boundary is not a complete D4 program.
- Standard Definitions Are Part of the Language
A large portion of D4's apparent intelligence comes from its standard definition database.
For example, the compiler may know:
list + @contiguous + fixed maximum => fixed contiguous allocation
list + unknown growth => dynamic contiguous allocation
ref + @unique => non-shared semantic reference
ptr + @raw => native machine address semantics
These are defined rules.
They are not arbitrary compiler guesses.
- Definitions Are Extensible
Libraries can contribute mappings.
Example:
ledger VectorMath: map vec4 + vec4 => simd.add.f32x4
The compiler records the extension in the ledger.
A new abstraction can therefore become almost as cheap to resolve as a built-in construct after it has been definitively mapped.
- Rulesets
Rulesets establish constraints shared by definitions.
ruleset SafeBuffer: pointer must @nonnull length must >= 0 capacity must >= length
Applied:
Buffer @rules(SafeBuffer): pointer: ptr length: usize capacity: usize
A ruleset participates directly in ambiguity elimination and lowering.
- Rules Before Optimization
D4 separates:
WHAT IS LEGAL?
from:
WHAT IS FASTEST?
First:
definitions types rules references labels
establish legal meaning.
Then optimization chooses the implementation.
This prevents optimization from becoming a semantic guessing mechanism.
- Optimization Freedom
Once D4IR exists, LLVM may aggressively perform:
constant folding dead-code elimination inlining loop unrolling vectorization SLP GVN LICM tail merging branch folding instruction combining register promotion interprocedural optimization LTO PGO machine scheduling
because D4 has already provided a definitive operation graph.
- Native Runtime Philosophy
D4 does not require a heavyweight runtime.
A program that does not request runtime services can approach:
D4 executable + minimal startup + native operating-system calls
Higher facilities can be linked only when used.
Examples:
allocator scheduler filesystem network stack wrappers thread pool Unicode support reflection metadata debug services
are modular rather than universally mandatory.
- Standard Library Shape
The foundational library is organized approximately as:
core core.math core.mem core.ref core.container core.range core.ledger
sys sys.process sys.thread sys.sync sys.file sys.net
simd atomic crypto text
platform.win platform.posix
Platform-specific facilities remain layered over portable frontend semantics.
- D4's Fundamental Compilation Principle
The language should transform:
complex source structure
into:
simple definitive execution
rather than transforming:
simple source
into:
a giant runtime responsible for discovering meaning later.
That is central to D4.
- The D4 Identity
D4 is simultaneously:
High-level
because programmers work with:
cases descriptions references families categories ladders boxes compartments tracks courses blends dynamic ranges rules ledgers
Low-level
because all of them must definitively reduce into primitive executable operations.
Fast to resolve
because definitions and mappings are aggressively indexed.
Compiler-friendly
because operand types, labels, references, and rules eliminate semantic uncertainty early.
Systems-oriented
because memory, pointers, layout, concurrency, native execution, ABI behavior, and assembly remain first-class concepts.
-
D4 in One Diagram
D4 │┌──────────────┼──────────────┐ │ │ │ DESCRIPTION REFERENCE CASE │ │ │ └──────────────┼──────────────┘ │ STANDARD MAP │ RULESET │ LABEL │ LADDER │ REFERENCE LOWERING │ DEFINITIVE D4IR │ LLVM IR │ LLVM BACKEND │ PE/COFF OBJ │ ld │ .exe
- The Four Dimensions
The conceptual architecture finally reduces to the language's namesake:
D1 — REFERENCE What does this point to?
D2 — CASE What concrete situation exists?
D3 — DESCRIPTION What does this operand/entity mean?
D4 — ASPECT What additional operational constraints apply?
Together:
REFERENCE + CASE + DESCRIPTION + ASPECT ↓ D4
The compiler uses those four dimensions to move from expressive human-facing descriptions toward definitive machine-facing execution.
- Official Language Motto
«Map the meaning. Resolve the reference. Collapse the abstraction. Execute the primitive.»
And the shorter engineering motto:
«High descriptions. Hard mappings. Definitive execution.»
- Core D4 Vocabulary
case concrete instance/specialization use invocation ledger reusable definition system label explicit semantic identity ruleset declarative constraints
box memory responsibility region compartment memory subdivision
ref semantic reference ptr machine pointer
ladder ordered semantic progression derivative value/dependency derived through a ladder deduction proven progression chain ordered execution progression category capability/type grouping family related semantic grouping layer staged semantic organization group contextual association
course parallel work track concurrent execution blend deterministic merge
push exhaustive/brute-force search
skip omit failed contribution rewrite substitute failed result/path reorg reorganize around recovery path quit terminate declared execution scope
shed release obsolete resources trim reduce excess capacity defer postpone action/reclamation stack impose LIFO resource discipline
recall rollback to checkpoint retreat semantically undo reversible operations
branch decision structure range dynamic traversal descriptor
- D4's Foundational Character
D4 does not ask the programmer to live at machine level.
It asks the compiler to be extremely good at getting there.
The source may say:
course Left: value := use solve(left)
course Right: value := use solve(right)
answer := blend Left.value + Right.value
while the final executable may contain little more than:
loads integer/vector operations calls branches register transfers stores
The distance between those two representations is bridged through mappings rather than semantic improvisation.
That is D4's defining strength.
The language is not attempting to hide the machine.
It is attempting to make the route to the machine predefined, indexed, deterministic, reusable, and brutally direct.
- D4 Core Grammar Contract
D4 deliberately keeps its grammatical foundation much smaller than its surface vocabulary.
The canonical grammar recognizes only a limited set of structural forms:
page declaration description case binding reference call directive label ledger ruleset ladder connector branch range block expression literal
Everything else is defined in terms of these structures plus standard maps.
The practical consequence is important:
new high-level construct ≠ necessarily new parser machinery
Instead:
new construct ↓ existing grammar form ↓ new definition or map
This is a foundational D4 design rule.
- Canonical Source Form
A D4 source page follows this conceptual structure:
PAGE HEADER* IMPORT* LABEL* RULESET* DESCRIPTION* LEDGER* BOX* CASE*
The order is not universally mandatory because page scanning builds a complete page index before semantic resolution.
This means the following is legal:
case main of Program: use Boot.start()
ledger Boot: start() -> unit: use print("D4")
The page scanner discovers both definitions before reference lowering begins.
- Source Header
The canonical source-version declaration is:
d4 1
Optional page directives may follow:
d4 1 @indent(4) @target(x86_64_windows) @overflow(checked)
The version declaration governs:
grammar version standard definitions primitive set operator tables mapping rules D4IR compatibility ABI defaults
Source semantics are therefore versioned independently from compiler implementation version.
- Lexical Classes
D4 recognizes these principal lexical classes:
identifier keyword directive label-reference integer-literal floating-literal text-literal character-literal operator connector delimiter indent dedent newline page-boundary
Identifiers use Unicode source text but normalize to a stable canonical identifier form before symbol indexing.
The default recommended identifier style is:
player_count FrameState Network.Reader
Case sensitivity is preserved.
Therefore:
player Player PLAYER
are three distinct identifiers.
- Reserved Core Keywords
The D4/1 core reserves:
d4 case of use ledger label ruleset box compartment ladder derivative deduction chain category family layer group course track blend push skip rewrite reorg quit shed trim defer recall retreat branch if else for while in by from until map must with resume true false unit never
Standard-library names are not automatically reserved unless explicitly promoted into the language version.
- Delimiters
D4 uses:
( ) grouping, argument lists [ ] indexing, literal containers, parameterized declarations < > type parameters and comparisons where structurally valid : description/block introduction , sequence separation . qualified reference
Curly braces are intentionally absent from canonical D4 block syntax.
Indentation owns ordinary block structure.
- Fundamental Operator Table
D4/1 freezes these core operators:
:= establish binding <- replace existing binding = equality != inequality < less than <= less than or equal
greater than= greater than or equal
-
addition / mapped combine
-
subtraction / unary negation
-
multiplication
/ division % remainder
& bitwise AND | bitwise OR ^ bitwise XOR ~ bitwise NOT << logical/arithmetic shift by typed mapping
logical/arithmetic shift by typed mapping
and logical conjunction or logical disjunction not logical negation
.. inclusive range ..< upper-exclusive range
-> relation / result / directional map => definitive resolution ~> deferred transition <-> bidirectional relation |> ladder rung
? fault-match introducer @ directive introducer
Operators do not acquire arbitrary user-defined meaning.
Their extension must map into a declared semantic family with an exact typed signature.
- Operator Precedence
D4/1 canonical precedence, highest to lowest:
-
qualification/index/call attachment . [] use-target
-
unary not ~ unary - unary +
-
multiplicative * / %
-
additive + -
-
shifts << >>
-
comparison < <= > >=
-
equality = !=
-
bitwise AND &
-
bitwise XOR ^
-
bitwise OR |
-
logical AND and
-
logical OR or
-
ranges .. ..<
-
connectors -> => ~> <-> |>
-
binding/replacement := <-
Grouping with parentheses always wins.
- Binding Semantics
Creation is single-purpose:
x := value
This creates a binding in the current semantic scope.
Replacement is separate:
x <- value
Replacement requires:
existing target replacement authority type compatibility lifetime compatibility ruleset validity
There is no assignment/equality ambiguity.
- Binding Stability
A newly established binding is stable unless its description or enclosing authority permits replacement.
Example:
count: i32 := 10
If the binding is immutable by default under the active ruleset, then:
count <- 20
is rejected unless replacement authority exists.
Mutable declaration shorthand:
count: i32 @replace := 10
The important rule is:
mutability is a semantic permission not merely an assignment operator side effect
- Description Grammar
Canonical description form:
Name: field: Type field: Type := default field: Type @directive
Example:
Packet: id: u64 size: usize payload: span @readonly
A description is initially representation-neutral unless a rule or directive makes representation observable.
- Description Composition
Descriptions compose through explicit references rather than implicit inheritance.
Transform: position: Vec3 rotation: Quat
Entity: id: u64 transform: Transform
Shared capability is expressed through categories, families, rulesets, or references.
D4 avoids making inheritance a mandatory structural primitive.
- Case Grammar
Canonical case form:
case Name of Description: body
Anonymous local cases are permitted:
enemy := case of Enemy: health := 100 level := 3
Parameterized cases may be defined through ledgers:
ledger EnemyCases: make(level: u32) -> Enemy: case of Enemy: level := level health := level * 25
- Call Grammar
Calls always use "use".
Canonical forms:
use function() use function(a, b) result := use function(a, b) use Module.function(value)
The grammar treats:
function(a)
without "use" as a non-call form and therefore invalid unless another standard definition explicitly owns that syntax.
This guarantees that invocation remains visible in source.
- Call Resolution Key
A callable is resolved by a canonical key:
qualified-name
- argument-count
- argument-types
- labels
- directives
- active ruleset
Example:
value := use convert(source) @as(u64)
may generate a lookup key conceptually equivalent to:
convert arity=1 arg0=SourceType label/as=u64 rules=current
The first exact legal map wins according to the fixed resolution hierarchy.
There is no fuzzy overload scoring.
- Ledger Grammar
A ledger is D4's canonical reusable-definition table.
ledger Name: definition definition map inverse
Callable entry:
ledger Math: add(a: i64, b: i64) -> i64: a + b
Mapped entry:
ledger NumericMaps: map i64 + i64 => core.add.i64
Inverse entry:
ledger Editing: Rename(old: text, new: text) -> unit: ...
inverse Rename => RenameInverse
The inverse can be used by "retreat".
- Ledger Identity
Every resolved ledger entry receives a canonical semantic identity.
Conceptually:
module ledger entry type-signature directive-set ruleset-hash target-contract
The compiler hashes this identity into a stable lookup key.
This powers:
incremental compilation specialization reuse cross-page reuse cross-module reuse cached lowering cached object emission
- Ruleset Grammar
Canonical form:
ruleset Name: predicate predicate mapping-rule
Example:
ruleset PositiveIndex: value must >= 0 value must < length
Rules may be applied to descriptions:
Index @rules(PositiveIndex): value: isize length: usize
or operations:
use access(buffer, index) @rules(BoundsSafe)
- Rule Classification
D4 rules belong to four semantic classes:
definition rule validity rule resolution rule lowering rule
Definition rule
Establishes what something is.
Validity rule
Establishes whether a state or operation is legal.
Resolution rule
Determines which exact semantic mapping applies.
Lowering rule
Constrains which primitive implementation may represent the resolved meaning.
Optimization is not itself a fifth rule class.
Optimization happens only after these four have produced legal definitive behavior.
- Label Grammar
Labels may be declared:
label fast label network_order label StableState @recall
or attached inline:
use encode(packet) @network_order
The "@" form identifies directive-like semantic qualifiers.
Named labels can also be referenced explicitly:
recall StableState
Labels must be resolvable within the current page, imported ledger space, or module namespace.
- Directive Grammar
Directive forms:
@name @name(value) @name(a, b)
Examples:
@inline @aligned(64) @target(x86_64_windows) @vector(width=8)
Directives cannot silently change source-level observable meaning unless the language definition explicitly declares that directive semantic.
Most directives constrain:
representation lowering optimization ABI memory scheduling diagnostics
- Directive Collision Resolution
Contradictory directives are compile-time errors unless a standard precedence rule exists.
For example:
buffer: ptr @nonnull @nullable
is invalid.
Likewise:
storage: array<byte, 64> @stack @heap
is invalid unless a surrounding map explicitly defines a split representation.
D4 does not silently discard one directive.
- Page Scanner Architecture
The D4 Page Scanner processes source in six fixed passes.
P0 — page boundary recognition P1 — lexical matrix construction P2 — indentation topology P3 — declaration indexing P4 — reference indexing P5 — structural validation
Semantic lowering does not begin during these passes.
The scanner's job is to establish a complete page model first.
- Page Boundaries
A physical ".dem4" file may contain one or more logical pages.
The default implementation may create pages using compiler-managed structural boundaries.
Explicit page boundaries are available:
--- page CoreMath ---
or:
@page(CoreMath)
The exact serialized marker is normalized before structural scanning.
Page names are optional but recommended for large modules.
- Structural Token Matrix
Unlike a flat token stream, D4 stores page tokens in a structural matrix.
Conceptually:
row column indent-depth token-class scope-candidate connector-role reference-role
This allows the compiler to recognize structures such as:
Pipeline: Read |> Decode |> Validate |> Store
without pretending that each line is an isolated statement.
- Page-First Semantic Resolution
After scanning, the semantic frontend resolves each page approximately as:
definitions ↓ labels ↓ local references ↓ external references ↓ rulesets ↓ typed maps ↓ ladders ↓ execution graph
The resulting page artifact is a Resolved Page Record, or RPR.
An RPR is cacheable.
- Resolved Page Record
Each RPR contains:
page identity source fingerprint definition table reference table type table label table ruleset table ladder graph fault table memory declarations dependency edges exported ledger entries target-independent execution graph
Only the portions invalidated by dependency changes need re-resolution.
- Cross-Page References
Cross-page references use indexed identities rather than repeated textual search.
Conceptually:
Page A: references Math.sum
Page B: exports Math.sum
The global reference index stores:
Math.sum -> semantic-id 0x...
Page A then stores the semantic identity directly in its resolved record.
Text lookup is no longer required during subsequent lowering passes.
- Import Semantics
Canonical import syntax is:
use ledger Math use ledger Network.Socket
Alias form:
use ledger Network.Socket as Socket
This use of "use" is consistent with D4's philosophy:
use means activate/reference an executable or reusable semantic resource
Import resolution remains compile-time only.
- Type System Overview
D4 is:
statically typed strongly typed nominal where identity matters structural where descriptions explicitly permit it reference-aware representation-aware only after resolution
Its central type principle is:
«Type information should eliminate choices, not create more guesses.»
- Primitive Integer Semantics
Integer widths are exact.
i8 i16 i32 i64 i128 u8 u16 u32 u64 u128
Signedness is part of the type.
No implicit signed/unsigned conversion occurs when the conversion could change represented meaning.
Example:
a: i32 b: u32 c := a + b
is rejected unless a standard exact map exists under the active ruleset.
The normal solution is explicit:
c := use cast(a) + use cast(b)
- Integer Overflow Policy
Overflow behavior is never undefined.
Standard policies are:
@overflow(checked) @overflow(wrap) @overflow(saturate) @overflow(trap)
The module default is "checked" unless overridden by the target profile or explicit source policy.
Example:
count: u32 @overflow(wrap)
LLVM receives the correct concrete operation rather than being allowed to invent source semantics from undefined behavior.
- Floating-Point Semantics
Floating types are:
f16 f32 f64 f128
Default semantics preserve IEEE-compatible behavior where the target supports it.
Optimization-affecting behavior requires directives:
@fp(strict) @fp(contract) @fp(fast)
"@fp(fast)" is an explicit semantic relaxation.
It is never silently enabled merely because optimization level is high.
- Boolean Semantics
"bool" represents semantic truth.
"bit" represents a one-bit numeric/storage value.
They are distinct.
flag: bool mask: bit
Conversions require exact maps.
This avoids conflating control truth with raw packed state.
- Pointer and Reference Type Semantics
ref ptr addr
mean different things.
"ref":
semantic reference compiler-known target type policy-rich may carry lifetime/authority facts
"ptr":
machine-addressable pointer explicit memory semantics may be raw
"addr":
opaque target-sized address value no implied pointed-to type
- Nullability
References are non-null by default unless declared otherwise.
owner: ref
cannot contain null.
Nullable form:
owner: ref @nullable
Raw pointers may be nullable unless constrained:
buffer: ptr @nonnull
This distinction becomes concrete in D4IR reference metadata.
- Optional Values
D4 provides an explicit optional semantic type:
optional
Example:
result: optional
An optional is not semantically equivalent to a null pointer.
Its representation may nevertheless collapse to a niche value when the compiler proves that representation legal.
- Tuples
Tuple syntax:
(i32, text, bool)
Value:
entry := (7, "seven", true)
Named decomposition:
(id, name, active) := entry
Tuple layout remains unobservable unless constrained by ABI or layout directives.
- Arrays and Spans
Fixed arrays:
array<T, N>
Borrowed contiguous views:
span
Example:
values: array<i32, 128> view: span
"span" semantically contains a bounded contiguous reference, not ownership.
The common primitive lowering is:
pointer + length
but this representation is frozen only at D4IR generation.
- Type Compatibility
D4 defines four compatibility relations:
exact lossless explicit forbidden
Exact
No conversion required.
Lossless
The standard map proves all source values are representable.
Explicit
A programmer-visible conversion is required.
Forbidden
No standard or declared semantic map permits the conversion.
There is no hidden implementation-defined conversion class.
- Generic Descriptions
Parameterized descriptions use angle syntax:
Pair<A, B>: first: A second: B
Instantiation:
coords: Pair<i32, i32>
D4 generics resolve through ledger specialization.
They do not require a runtime generic object model.
- Generic Specialization
For each concretely used generic combination, D4 produces or reuses a semantic specialization entry.
Pair<i32,i32> ↓ ledger lookup ↓ existing specialization?
If yes:
reuse
If no:
resolve once freeze mapping store semantic fingerprint
This avoids repeated template-style rediscovery.
- Compile-Time Constants
Values proven at compile time may be labeled:
PageSize: usize @const := 4096
The compiler may also infer compile-time constancy when a value has no runtime dependency.
Such values are eligible for immediate map folding.
- Runtime Values
A value is runtime if any definitive dependency requires runtime state.
The compiler records the boundary explicitly.
Conceptually:
constant ladder ↓ runtime input rung ↓ runtime remainder
This allows D4 to partially execute ladders during compilation.
- Ladder Grammar
Canonical ladder:
ladder Name: rung |> rung |> rung
Compact form:
Name: A |> B |> C
Each rung has:
input reference set operation or relation output reference set rule context fault behavior
- Ladder Resolution
Ladders resolve top-to-bottom.
Each rung must produce all requirements needed by the next rung.
Example:
ladder ParsePacket: bytes |> header |> payload |> checksum |> Packet
The compiler validates each adjacent transition:
bytes -> header header -> payload payload -> checksum checksum -> Packet
An invalid transition stops compilation at the exact rung.
- Ladder Folding
If multiple consecutive rungs resolve to compile-time identities or equivalent primitives, D4 folds them.
Example:
A |> Identity |> CastExact |> B
may collapse to:
A |> B
provided all observable semantics are preserved.
- Ladder Branches
Ladders may branch:
ladder Decode: input |> branch format: PNG -> use decode_png(input) JPG -> use decode_jpg(input) RAW -> use decode_raw(input) |> image
Branch outputs must converge on a compatible next-rung description.
This makes convergence explicit and verifiable.
- Fibonacci Branch Planner
The Fibonacci Branch Planner is defined as a pre-LLVM structural optimization.
Its job is not to blindly emit Fibonacci comparisons.
Its job is to use Fibonacci-sized partitions as a stable heuristic for exploring branch-layout candidates.
For "N" branch alternatives, the planner builds candidate groups using the greatest Fibonacci numbers not exceeding the remaining branch population.
Example for 20 alternatives:
13 + 5 + 2
These clusters are then scored using known information such as:
case density range continuity hotness shared tails comparison cost jump-table suitability
The result may become:
jump table binary decision tree range tree direct compare chain hybrid dispatch
LLVM still performs final target-specific branch optimization.
- Exponential Expansion Limit
D4 permits exponential semantic expansion only as a bounded proof technique.
The compiler must never require unbounded combinatorial materialization.
Every implementation provides an expansion budget.
When the budget would be exceeded, the resolver switches to symbolic equivalence classes instead of enumerating every path.
Thus:
exponential reasoning is permitted exponential memory explosion is not required
- Branch Collapse Rules
Two branch nodes may collapse when all of the following match:
result type observable side effects memory authority fault behavior continuation destination ordering requirements
Different source paths can therefore share one definitive D4IR block.
- Range Semantic Model
A range is represented semantically as:
start end inclusivity step direction evaluation-policy
These are descriptions, not necessarily stored fields.
Example:
0..count by 2
may lower directly into loop induction primitives without allocating any range object.
- Range Direction
Direction may be inferred when exact:
10..0 by -1
Dynamic direction requires a rule:
start..finish by stride
If "stride" may be zero, compilation requires one of:
proof stride != 0 fault mapping explicit checked policy
Infinite accidental ranges are not silently accepted.
- Containers as Resolved Protocols
Every smart container implements a semantic protocol rather than one universal representation.
The core container protocol includes operations such as:
count capacity access insert remove iterate reserve trim shed
The container map decides which of these exist and how they lower.
For a fixed array:
capacity == count == N reserve = invalid trim = no-op
For a dynamic list:
count <= capacity reserve = valid trim = backing-store adjustment
- Container Representation Selection
Representation selection uses a deterministic key:
container-kind element-type known-size growth-policy ownership lifetime alignment directives target
The same key must produce the same representation under the same D4 language version and target profile.
That makes build behavior reproducible.
- Box Grammar
Canonical box:
box Name @policy: compartment Name: bindings
Example:
box Engine @heap @shedding: compartment Permanent: world: World
compartment Frame @arena:
scratch: bytes
Boxes establish memory authority boundaries.
- Compartment Semantics
A compartment defines:
allocation domain reclamation domain reference visibility alignment policy sharing policy lifetime relation to containing box
References crossing compartments are checked against these properties.
- Cross-Compartment References
Example:
box World: compartment Permanent: player: Player
compartment Frame @arena:
current: ref<Player>
This is legal if:
Permanent outlives Frame
The reverse reference may be rejected:
Permanent -> Frame
if it would outlive the target compartment.
D4 therefore performs lifetime reasoning through box/compartment topology without requiring lifetime syntax on every source reference.
- Memory Authority
Every memory operation resolves to an authority class:
read replace move share atomic raw
These classes feed reference lowering.
For example:
player: ref @readonly
cannot satisfy a map requiring replace authority.
- Shedding Semantics
"shed" terminates ownership responsibility for eligible resources.
It must not leave live references to destroyed storage.
The compiler therefore verifies:
outstanding references deferred uses track access course access retreat history recall checkpoints
before lowering a shed operation.
- Trimming Semantics
"trim" may change storage capacity but not logical content.
Thus:
trim values
must preserve:
values.count element order element identity where promised semantic references where promised
If the representation cannot preserve required reference stability, trimming is rejected or requires an explicit relocation policy.
- Defer Stack
Deferred operations are recorded in a deterministic defer stack attached to the declared scope.
Example:
defer use close(file) until case done
The compiler lowers this into the minimum machinery required by the scope's possible exits.
If the scope has one exit, the deferred operation may become a direct tail action.
If it has many exits, D4 may collapse them into a shared cleanup block.
- Stacking Policy
"@stacking" guarantees reverse-order reclamation for resources acquired in the governed region.
It does not necessarily mean use of the processor stack.
Possible implementations include:
native stack allocation arena marker stack destructor stack index rollback stack region cursor rollback
The representation is selected by the memory map.
- Error Value Model
An operation may declare a typed fault set.
Example:
ledger File: read(path: text) -> bytes faults Missing | Permission | IO
At each use site:
data := use File.read(path) ? Missing -> skip ? Permission -> quit case ? IO -> reorg RetryIO
Unhandled faults must either:
propagate through an explicitly compatible fault signature or cause compilation failure
- "skip" Formal Semantics
"skip" is valid only when the enclosing construct has a legal continuation without the failed contribution.
Valid examples include:
iteration element optional contribution blend input marked omittable collection transform element
Invalid example:
x: i64 := use required_value() ? Missing -> skip
if "x" must exist afterward.
The compiler diagnoses the missing continuation value.
- "rewrite" Formal Semantics
"rewrite" must produce a value or operation result compatible with the original continuation.
Example:
count: u64 := use read_count() ? Missing -> rewrite 0u64
The replacement type is checked exactly like an ordinary result.
- "reorg" Formal Semantics
"reorg" transfers control to a declared recovery ladder.
That ladder must explicitly declare whether it:
resumes original continuation returns replacement result quits scope recalls checkpoint
No implicit recovery destination exists.
- "quit" Formal Semantics
"quit" targets a named or structural execution region.
Valid targets include:
push course track case box program named label
Value-returning quit:
quit push with candidate
is legal when the target construct defines a result type.
- Courses: Parallel Semantics
Courses form a parallel work set.
Example:
course A: left := use solve_left()
course B: right := use solve_right()
result := blend A.left + B.right by Sum
The compiler builds a dependency graph.
Only courses with no ordering dependency may execute simultaneously.
Thus parallelism is proven rather than assumed.
- Course Scheduling Freedom
Because courses describe parallel opportunity, the compiler may realize them as:
sequential execution OS threads task-pool jobs SIMD lanes GPU workgroups distributed jobs through an explicit runtime profile
provided:
observable ordering memory effects fault behavior blend semantics
remain identical.
- Tracks: Concurrent Semantics
Tracks create independent execution histories.
Canonical form:
track Name: body
Track references can be shared only through explicitly compatible memory or synchronization policies.
Example:
box State @shared: compartment Counters: packets: atomic
Tracks may safely access "packets" through atomic maps.
- Track Lifetime
A track must belong to a lifetime owner.
Possible owners:
case box program explicit track group
Detached tracks require:
track Logger @detached:
and must prove that all referenced resources outlive the detached track or are independently owned.
- Data-Race Rule
D4 rejects unsynchronized conflicting access when it can prove that two tracks or courses may overlap.
Conflicting means:
write/write read/write
to the same aliased storage without a valid synchronization map.
Raw pointers can bypass parts of this protection only under explicit raw authority.
That bypass is visible in the semantic record.
- Blend Semantics
Every blend has three pieces:
inputs combiner ordering policy
Example:
total := blend A.value + B.value by Sum
For associative and commutative combiners, the ruleset may declare:
Sum @associative @commutative
which gives the compiler maximum parallel reduction freedom.
- Deterministic Blends
If a blend is order-sensitive, its order must be declared or inferable.
Example:
events := blend track Input + track Network by TimestampThenSource
D4 never defines nondeterministic merge order merely because execution was concurrent.
- Push Grammar
Canonical brute-force search:
push item in range: body
Optional policy:
push candidate in candidates @parallel:
Optional result:
answer := push candidate in candidates: if use valid(candidate): quit push with candidate
The result type is inferred from all value-returning push exits.
- Push Search Contract
"push" guarantees exhaustive coverage only under the declared search policy.
The compiler may reorder candidates unless:
@ordered
is present.
This distinction is crucial.
push candidate in values @ordered:
preserves source iteration order.
Without "@ordered", the compiler may partition or vectorize the domain.
- Recall Checkpoints
A recall checkpoint records a semantic state boundary.
label BeforeUpdate @recall
The compiler computes the smallest state set that must be recoverable.
It need not snapshot an entire box if only three bindings changed after the checkpoint.
Possible optimization:
checkpoint ↓ change-set analysis ↓ store only reversible delta
- Retreat History
Retreat uses ledger-defined inverse operations.
An operation is retreatable only if an inverse exists or the compiler can prove a reversible primitive transformation.
Example:
x <- x + 5
may be reversible only under an overflow policy that preserves invertibility.
Therefore "retreat" is semantic, not merely textual undo.
- Recall and Retreat Interaction
Recall checkpoints establish hard state anchors.
Retreat histories may exist between anchors.
Conceptually:
[Recall A] Op1 Op2 Op3 [Recall B] Op4
"retreat 2" after "Op4" reverses "Op4" and "Op3" if both are reversible.
"recall A" restores A according to its checkpoint strategy regardless of individual retreatability.
- Definitive Reference Lowering
Reference lowering is the stage where semantic references become concrete address/value relationships.
Example:
player: ref use damage(player, 10)
may lower conceptually into:
%player.addr = boxaddr World.Entities.player %health.addr = fieldaddr %player.addr, Player.health %health = load.f32 %health.addr %next = sub.f32 %health, 10.0 store.f32 %next, %health.addr
The high-level "ref" no longer needs interpretation afterward.
- Reference Lowering Inputs
The reference lowerer receives:
resolved type box compartment ownership lifetime alignment alias policy nullability authority target data layout ABI exposure
It produces one of the definitive reference classes:
value address bounded-address fat-reference handle constant register-candidate
- MASM Mapping Layer Structure
MML entries have canonical identities such as:
mov.r64.r64 mov.r64.mem64 add.r64.r64 sub.r32.imm32 cmp.r64.r64 jcc.eq call.rel ret lea.r64.mem
These names describe machine-operation families.
They do not force a literal instruction before LLVM instruction selection.
Example:
D4IR add.i64 ↓ MML candidate add.r64.* ↓ LLVM add i64
LLVM may later fold the operation into an addressing mode or another equivalent instruction sequence.
- Instant Lookup Tables
The standard compiler precomputes immutable lookup tables for primitive resolution.
Principal tables:
type-pair -> arithmetic primitive type-pair -> comparison primitive reference-class -> address primitive container-key -> layout recipe fault-kind -> continuation primitive ladder-kind -> graph recipe directive-set -> lowering constraints D4IR opcode -> LLVM builder action D4IR opcode -> MML operation family
The intent is approximately constant-time resolution for the common path.
- Lookup Table Key Stability
Lookup keys use interned semantic identities rather than source strings wherever possible.
Instead of repeatedly comparing:
"unsigned 64-bit integer"
the resolver operates upon a canonical type ID such as:
T_U64
Likewise:
OP_ADD REF_UNIQUE FAULT_MISSING BOX_ARENA
This is one reason D4 is designed for fast semantic resolution.
- D4IR Design Goals
D4IR is:
typed SSA-friendly control-flow explicit memory explicit fault explicit target-neutral where possible target-constrained where necessary free of unresolved language abstractions
D4IR sits between language semantics and LLVM semantics.
Its purpose is not to compete with LLVM IR.
Its purpose is to guarantee that LLVM receives a fully resolved program.
- D4IR Module Structure
A D4IR module contains:
module header type table constant table global table box layouts function table fault signatures track/course metadata debug map target contract
Functions contain:
parameters basic blocks SSA values memory operations branches calls fault edges returns
- D4IR Primitive Opcode Families
D4IR/1 defines these major opcode families:
const copy
add sub mul div rem neg
and or xor not shl shr
cmp select
load store addr fieldaddr indexaddr
cast bitcast extend truncate
call tailcall return
branch jump switch
phi
atomic.load atomic.store atomic.rmw atomic.cmpxchg fence
vector.make vector.extract vector.insert vector.shuffle
course.spawn course.join track.spawn track.join
fault.raise fault.route
checkpoint recall retreat
mem.copy mem.move mem.fill mem.shed
intrinsic target
Higher source constructs must reduce into these or a later D4IR extension standardized by the language version.
- D4IR Typed Opcodes
Every arithmetic opcode is typed.
Examples:
add.i32 add.i64 add.u64 add.f32 mul.i128 cmp.eq.i64 cmp.lt.f64
The textual suffix is diagnostic notation.
Internally the opcode and type IDs may be stored separately.
- D4IR Example
Source:
ledger Math: double(x: i64) -> i64: x + x
D4IR:
func @Math.double(%x: i64) -> i64 entry: %r0 = add.i64 %x, %x return %r0
LLVM IR may then become conceptually:
define i64 @Math.double(i64 %x) { entry: %r0 = add i64 %x, %x ret i64 %r0 }
The semantic distance between D4IR and LLVM is intentionally small.
- D4IR Memory Example
Source:
player.health <- player.health - damage
D4IR:
%p0 = fieldaddr %player, Player.health %v0 = load.f32 %p0 %v1 = sub.f32 %v0, %damage store.f32 %v1, %p0
No property abstraction survives into LLVM lowering.
- D4IR Fault Example
Source:
data := use File.read(path) ? Missing -> rewrite EmptyBytes ? Permission -> quit case
D4IR may become:
%result, %fault = call.checked @File.read(%path) branch.fault %fault: Missing -> fault.rewrite @EmptyBytes Permission -> fault.quit case none -> continue
During LLVM lowering, this may reduce to ordinary branches, tagged results, platform error codes, or another proven representation.
- D4IR Course Example
Source:
course A: x := use left()
course B: y := use right()
z := blend A.x + B.y by Sum
D4IR conceptually:
%ca = course.spawn @left %cb = course.spawn @right %x = course.join %ca %y = course.join %cb %z = add.i64 %x, %y
If parallel execution is not profitable, the course planner may legally reduce this before LLVM to:
%x = call @left %y = call @right %z = add.i64 %x, %y
because course semantics describe parallel opportunity, not mandatory threading.
- D4IR Track Example
Tracks are different.
track Worker: use process_queue()
must retain independent execution semantics unless the compiler proves complete equivalence under the track contract.
D4IR:
%track = track.spawn @process_queue
The runtime or platform lowering determines the concrete thread/task primitive.
- D4IR Verification
Every D4IR module is verified before LLVM translation.
The verifier checks:
type correctness SSA consistency dominance valid branches valid fault routes memory authority box/compartment lifetime reference legality track/course dependency legality blend determinism recall validity retreat validity target constraints ABI completeness
LLVM must never be used as the primary detector for an unresolved D4 semantic problem.
- LLVM Lowering Contract
The D4IR-to-LLVM layer is intentionally mechanical.
Examples:
D4IR add.i64 -> LLVM add i64
D4IR load.i32 -> LLVM load i32
D4IR store.f64 -> LLVM store double
D4IR cmp.eq.i32 -> LLVM icmp eq i32
D4IR cmp.lt.f32 -> LLVM fcmp ordered/unordered form fixed by D4 fp policy
D4IR phi -> LLVM phi
Complex D4IR operations lower to small deterministic LLVM patterns.
- LLVM Is Not the Semantic Authority
LLVM is never asked to decide:
what a case means what a reference refers to what a ladder means which error behavior applies whether two descriptions are semantically compatible which box owns a resource what blend ordering means
Those questions are already answered.
LLVM receives only executable facts.
- Target Profiles
D4 target profiles define machine-facing defaults.
Example:
@target(x86_64_windows)
A profile contains:
LLVM triple data layout pointer width endianness ABI calling convention defaults object format linker mode CPU baseline feature baseline MML table
- x86-64 Windows Profile
The canonical initial D4 target is:
x86_64-pc-windows-msvc compatible object semantics PE/COFF 64-bit pointers Windows x64 calling convention by default
The compiler emits ".obj" through LLVM.
The driver invokes a PE-capable "ld" contract to produce the executable.
- Link Contract
The canonical command:
d4 build app.dem4
performs conceptually:
d4 frontend ↓ D4IR ↓ LLVM ↓ app.obj ↓ ld-compatible linker ↓ app.exe
The driver controls:
entry point subsystem libraries runtime selection debug data LTO mode output path
- Runtime Profiles
D4 defines runtime profiles rather than one mandatory runtime.
Standard profiles:
@runtime(none) @runtime(minimal) @runtime(system) @runtime(full)
none
No D4 runtime support is linked.
minimal
Startup, panic termination, basic allocation hooks where required.
system
Adds standardized OS abstractions, tracks, synchronization, file and networking helpers.
full
Adds optional high-level library services.
Unused facilities remain link-elidable.
- ABI Declarations
Foreign calls must declare an ABI when it cannot be inferred.
Example:
ledger Win32 @abi(win64): external MessageBoxW( hwnd: addr, text: ptr, caption: ptr, flags: u32 ) -> i32
D4 lowers the signature into LLVM calling convention and attribute metadata.
- External Symbols
External symbol declaration:
external malloc(size: usize) -> ptr @abi(c)
Import-library binding may be attached:
external CreateFileW(...) -> addr @abi(win64) @library("Kernel32")
The linker driver converts library references into target-appropriate link arguments.
- Inline MASM Contract
Because LLVM remains the sole backend, inline MASM is represented as an LLVM inline-assembly region.
D4 therefore requires enough information to construct a safe inline-assembly contract.
Canonical form:
asm @masm input left -> rcx input right -> rdx output rax -> result clobber flags: mov rax, rcx add rax, rdx
The frontend verifies operand widths and constraints before producing D4IR "target.asm".
- Raw Opcode Escape
D4 permits an even lower-level target escape:
opcode @x86_64 "pause"
or an intrinsic map:
use cpu.pause()
The intrinsic form is preferred because it remains portable across compatible backends and architectures.
Raw opcode forms mark the containing definition target-specific.
- Formatting Rules
D4's formatter is semantic rather than cosmetic.
Canonical formatter rules include:
4-space indentation one space around binary operators one space after commas no spaces inside ordinary parentheses vertical ladders aligned by rung directives remain attached to governed construct fault handlers indent beneath producing operation
Example:
result := use load(path) ? Missing -> rewrite Empty ? Corrupt -> reorg Recovery
The formatter never changes semantic indentation.
- Expressive Shorthand Normalization
The formatter/parser may accept:
Build: Scan -> Resolve -> Lower
and normalize it internally to:
ladder Build: Scan |> Resolve |> Lower
Likewise compact descriptions may normalize into canonical multiline forms.
The canonical normalized tree is what receives the semantic fingerprint.
- Standard Compilation Phases
The official frontend pipeline is:
- LOAD
- PAGE
- SCAN
- INDEX
- RESOLVE
- PROVE
- MAP
- LOWER REFERENCES
- BUILD D4IR
- VERIFY D4IR
- LOWER LLVM
- OPTIMIZE LLVM
- EMIT OBJ
- LINK
Each phase has a narrow responsibility.
- LOAD
The loader resolves:
source encoding module identity language version imports target profile
No semantic operation lookup occurs yet.
- PAGE
The page phase partitions source into logical compilation units.
It computes initial fingerprints used for incremental compilation.
- SCAN
The scanner creates the structural token matrix.
It resolves:
lexical class indentation comments delimiters connectors literal boundaries
It does not determine high-level semantics.
- INDEX
The indexer records:
declarations labels ledger entries rulesets case names description names box names compartment names
All definitions become addressable before resolution begins.
- RESOLVE
Resolution connects:
identifiers -> definitions calls -> ledger entries operators -> typed maps containers -> protocols directives -> semantic constraints labels -> declared meanings
Any unresolved ambiguity is a compiler error.
- PROVE
The proof phase validates:
type compatibility rulesets memory lifetimes authority fault coverage range legality track/course safety blend determinism ladder transitions
Failed proof means the program does not reach lowering.
- MAP
Mapping converts resolved semantic operations into exact primitive operation identities.
Example:
u64 + u64 ↓ core.numeric.add.u64 ↓ D4 primitive ADD_U64
The MASM mapping table also records the corresponding machine-operation family.
- LOWER REFERENCES
All abstract references are converted into concrete D4IR-ready reference classes.
After this phase, the compiler knows whether every access is:
SSA value constant address bounded address handle global box-relative location
- BUILD D4IR
The execution graph becomes typed D4IR basic blocks.
All control flow is explicit.
All error continuations are explicit.
All memory operations are explicit.
All unresolved language syntax is gone.
- VERIFY D4IR
No invalid D4IR is permitted to reach LLVM.
Verification failure is a D4 compiler implementation error if the source had already passed semantic proof.
This gives the compiler a hard internal correctness boundary.
- LOWER LLVM
The LLVM translator is intentionally thin.
Its purpose is predominantly table-driven opcode conversion plus metadata construction.
This is exactly where D4 benefits from its definitive IR architecture.
- OPTIMIZE LLVM
Optimization levels:
-O0 -O1 -O2 -O3 -Os -Oz
do not change D4 semantics.
They influence only legal implementation choices.
Profile-guided optimization and LTO may be enabled independently.
- EMIT OBJ
LLVM emits the target object file directly.
For the primary Windows target:
PE/COFF .obj
No MASM assembly step is required.
Optional emitted assembly remains diagnostic or interoperability output.
- LINK
The driver invokes the configured "ld" contract.
The reference Windows toolchain may resolve that contract to LLVM "lld-link" or an equivalent PE-capable linker adapter.
The user-facing model remains simply:
.obj -> ld -> .exe
- Incremental Compilation
D4 incremental compilation operates primarily at page and ledger-entry granularity.
Each page stores:
source hash export hash reference hash ruleset hash target-independent semantic hash
If source changes but exported semantics do not, dependent pages may remain valid.
- Semantic Fingerprints
A semantic fingerprint intentionally ignores irrelevant formatting differences.
For example:
x := a + b
and a formatter-normalized equivalent generate the same semantic fingerprint.
Changes to:
types rules labels maps fault behavior memory authority
do alter the fingerprint.
- Rebuild Propagation
Dependency invalidation follows semantic edges rather than entire-module timestamps.
Conceptually:
changed entry ↓ changed fingerprint? ├─ no -> stop └─ yes -> invalidate direct dependents ↓ repeat
This is essential to D4's intended compile-time performance.
- Diagnostics
D4 diagnostics are required to report the semantic failure, not merely parser position.
Instead of:
error near token 57
D4 prefers:
D4E2214: replacement authority missing
health <- value
^^^^^^
health is readonly through reference player.
Required authority: replace
Available authority: read
Diagnostics are part of the language implementation quality contract.
- Ambiguity Diagnostics
When ambiguity remains after all standard resolution stages, D4 reports the exact competing maps.
Example:
D4E1402: unresolved mapping
use convert(input)
Candidates: convert(Source) -> UTF8Text convert(Source) -> BinaryPacket
Resolve with an explicit type, label, or directive.
The compiler never silently chooses based on declaration order.
- Page Scanner Diagnostics
Because spacing and indentation are semantic, the scanner gives structural diagnostics.
Example:
D4E0108: inconsistent indentation
expected depth: 2 observed depth: 3 page: RenderPipeline
Mixed tabs/spaces without an explicit policy are rejected before semantic analysis.
- Zero-Undefined-Behavior Goal
D4/1 defines source-level behavior for operations that commonly become undefined in low-level languages wherever practical.
Examples include:
integer overflow policy nullability out-of-range indexing invalid shifts fault propagation reference lifetime concurrent conflicting access
Raw target escapes may opt outside these guarantees, but the opt-out is explicit.
- Raw Mode
Raw mode is attached narrowly.
buffer: ptr @raw
or:
case DeviceDriver @raw:
Raw does not mean "turn off the compiler."
It means:
permit operations whose safety cannot be fully proven while retaining all other available type and mapping information
This keeps unsafe regions inspectable.
- Directives Enhance Raw Code
Even raw pointers can retain compiler knowledge.
pixels: ptr @raw @nonnull @aligned(64) @noalias @count(width * height * 4)
This information can flow into LLVM attributes and optimization metadata.
Raw code therefore does not automatically become opaque code.
- Portable Frontend Boundary
The portable frontend ends at verified target-independent D4IR plus target constraints.
Host-specific code begins only in:
target mapping ABI lowering LLVM target emission linking platform runtime modules
This keeps the scanner, parser, resolver, proof engine, ledger engine, and most of D4IR host-independent.
- Compiler Implementation Layers
The reference compiler is organized as:
d4scan page scanner d4index page/global indexer d4resolve reference/type resolver d4proof rules and safety verifier d4map mapping engine d4lower reference + primitive lowerer d4ir IR builder/verifier d4llvm LLVM translator d4link linker driver d4ledger reusable semantic cache d4fmt formatter d4diag diagnostics engine
The components may be libraries rather than separate executables.
The names describe architectural responsibilities.
- Compiler Fast Path
The optimized common compilation path is:
scan page ↓ fingerprint hit? ├─ yes -> reuse RPR └─ no -> resolve changed page
for each operation: typed lookup key ↓ primitive map hit ↓ emit D4IR
D4IR verified ↓ LLVM translation
This is the practical realization of the phrase mapped language.
- Compiler Slow Path
The slow path is used only when:
new generic specialization is required new ledger map must be resolved cross-page semantic dependencies changed symbolic ladder proof is needed branch expansion is needed target constraints force remapping
Once resolved, results are ledgered for reuse.
- D4 Build Modes
Canonical commands:
d4 check app.dem4 d4 build app.dem4 d4 run app.dem4 d4 emit-ir app.dem4 d4 emit-llvm app.dem4 d4 emit-asm app.dem4 d4 emit-obj app.dem4 d4 fmt app.dem4
Meaning:
check scan through D4IR verification build produce final executable run build then execute emit-ir output D4IR emit-llvm output LLVM IR emit-asm output target assembly mirror emit-obj stop after object emission fmt canonical source formatting
- D4 Example: Native Hello World
d4 1
case main of Program: use print("Hello from D4.")
Conceptual lowering:
text literal ↓ standard print ledger lookup ↓ resolved platform output function ↓ D4IR call ↓ LLVM call ↓ native executable
- D4 Example: Memory-Oriented Systems Code
d4 1
box Frame @arena @stacking: compartment Geometry @aligned(64): vertices: array<f32, 4096>
ledger GeometryOps: clear(data: span) -> unit: for i in 0..<data.count: data[i] <- 0.0f32
case main of Program: use GeometryOps.clear(Frame.Geometry.vertices) shed Frame
No tracing garbage collector is required.
The array may lower to one contiguous fixed region and the clear loop may become LLVM memset/vector stores.
- D4 Example: Parallel Search
d4 1
ledger Search: find(values: span, wanted: i64) -> optional: result := push i in 0..<values.count @parallel: if values[i] = wanted: quit push with i
result
The compiler may lower the push into:
vector compare partitioned courses thread-pool search ordinary scalar loop
according to target and cost policy.
The semantic result remains the same under the declared ordering contract.
- D4 Example: Concurrent Server Skeleton
d4 1
box ServerState @shared: compartment Stats: accepted: atomic
track Listener: while true: socket := use accept(listener) ? Temporary -> skip ? Fatal -> quit track
use ServerState.Stats.accepted.increment()
use dispatch(socket)
case main of Program: use Listener.start() use Listener.join()
The track model exposes independent execution while memory rules make sharing explicit.
- D4 Example: Recovery
d4 1
ladder RecoverConfig: use find_backup() |> use validate() |> use load() |> resume
case main of Program: config := use read_config("app.cfg") ? Missing -> rewrite DefaultConfig ? Corrupt -> reorg RecoverConfig ? Permission -> quit case
use start(config)
Every failure outcome is explicit and locally readable.
- D4 Example: Recall and Retreat
d4 1
case Editor of Program: document := use open("story.txt")
label Loaded @recall
use insert(document, 10, "Hello")
use replace(document, 20, 5, "world")
retreat last
if use validate(document) = false:
recall Loaded
"retreat" reverses the most recent reversible edit.
"recall Loaded" restores the checkpoint if validation fails.
- D4 Example: Explicit Low-Level Mapping
d4 1
ledger FastMath: map i64 + i64 => core.add.i64
add(a: i64, b: i64) -> i64 @inline:
a + b
case main of Program: x: i64 := 20 y: i64 := 22 z := use FastMath.add(x, y) use print(z)
The semantic path is direct:
FastMath.add ↓ ledger identity ↓ i64 addition map ↓ D4IR add.i64 ↓ LLVM add i64 ↓ native ADD-family realization or equivalent optimization
- Standard Mapping Principle
Every standard operation should have the shortest stable semantic path possible.
Ideal:
source operation ↓ one lookup primitive identity ↓ direct builder action D4IR
Additional resolution layers exist only when the source actually contains additional semantic information.
D4 does not celebrate compiler complexity.
It tries to eliminate it.
- D4's Compiler-Friendliness Contract
Language features are considered compiler-friendly when they satisfy these rules:
- Syntax has one structural interpretation.
- Types narrow operation meaning.
- Definitions are indexable.
- Reusables have stable ledger identities.
- Errors have explicit continuations.
- Memory has explicit ownership regions.
- Parallelism and concurrency are distinct.
- Branches have definitive destinations.
- High-level abstractions must reach primitive boundaries.
- D4IR contains no unresolved semantics.
These rules are not recommendations.
They are the foundation of the language architecture.
- D4's Performance Philosophy
D4 does not promise speed merely because it is native.
Its performance strategy is architectural:
fast source scanning indexed reference resolution constant-time common maps ledger specialization reuse page-granular incremental compilation early abstraction collapse explicit memory knowledge explicit parallel intent definitive D4IR LLVM machine optimization
Compile-time and runtime performance are treated as two different engineering problems.
The language is designed to attack both.
- D4's Syntax Philosophy
D4 syntax follows four rules:
read high parse small resolve exact lower hard
The source should look like the programmer's intended operation.
The grammar should look like a compiler engineer's carefully constrained machine.
Those are not opposing goals in D4.
They are deliberately the same design.
- D4/1 Frozen Architectural Invariants
The following are frozen for D4/1:
.dem4 source extension
AOT compilation
LLVM-only production backend
PE/COFF primary object target
automatic ld-compatible linking
MASM-oriented machine mapping layer
page-first scanning
active indentation
semantic spacing
use for calls
case for instances/specializations
ledger reuse model
box/compartment memory model
skip/rewrite/reorg/quit faults
course parallelism
track concurrency
blend merging
push exhaustive search
recall rollback
retreat undo
ladder structural model
definitive D4IR
typed lookup-driven resolution
non-negotiable primitive boundary
Future D4 versions may add facilities but cannot reinterpret these meanings within D4/1 source.
- The D4 Execution Doctrine
D4 can now be summarized as an execution doctrine:
DESCRIBE what exists
REFERENCE exactly what is meant
LABEL whatever remains context-dependent
RULE what is legal
MAP each legal construct to one primitive meaning
LADDER ordered semantic relationships
LOWER references into concrete values and addresses
FREEZE the result into definitive D4IR
TRANSLATE D4IR mechanically into LLVM
OPTIMIZE without changing meaning
EMIT native object code
LINK automatically into the final executable
This is the core promise of D4:
«The programmer is allowed to think at a high level because the language refuses to leave the machine-level answer unresolved.»
And its compiler architecture can be reduced to one final chain:
.dem4 ↓ PAGE ↓ REFERENCE ↓ RULE ↓ MAP ↓ LADDER ↓ PRIMITIVE ↓ D4IR ↓ LLVM ↓ OBJ ↓ LD ↓ PE .EXE
D4 is high-level systems expression with low-level semantic finality.