Chapter 16: The embedding boundary

This chapter:

Build vm_register_native, the whole host-registration mechanism a real embedding needs: one call binds a builtin into both the VM's globals and the resolver's known-builtin set.
Build the value-marshalling layer: how Odin data crosses into a Value a script can hold, and back, and the rule that only handles, never raw pointers, ever cross that boundary.
Make a boundary mistake (wrong argument count, wrong argument type) a graceful runtime_error, never an Odin-level panic.
Register Muspel's first real, non-language builtin set, log, spawn, move_to, against a small stub host built entirely inside musc, proving the whole mechanism works from outside muspel/ before Embers exists to need it for real.

Where we left off: Chapter 15 made the VM resumable and gave it host_seconds, one native builtin the language itself always provides. Every other builtin a host wants to expose has, until now, meant hand-writing two lines in two different files, exactly what vm_init and resolver_init already do for print and host_seconds. Nothing yet says how a host outside muspel/ does the same thing for its own functions.

16.1 What "embedding" means concretely

An embedded scripting language is only useful once something outside the language can hand it work to do, and get called back into when the script wants to do something to the world. That's two directions across one boundary: Odin calls into Muspel (this book has had this since Chapter 13: compile a chunk, run it), and Muspel calls back into Odin (this book has had a sliver of this since print, but only ever one, fixed, language-provided function).

The host, whatever program embeds Muspel, owns every VM instance. It decides when to create one, when to resume it, and when to throw it away (Chapter 15's own answer for a suspended script that needs cancelling). It exposes a curated set of builtins, the entire vocabulary a script is allowed to affect the world through, and nothing more: a script never gets raw access to host memory, never holds an Odin pointer it could dereference, never sees anything except the seven Value cases Chapter 13 already defined (f64, string, bool, ^Func_Proto, Builtin, ^Struct_Instance, ^Array_Instance). Section 16.3 is what makes that last sentence true in practice, not just in principle.

16.2 Registering a builtin

Builtin (value.odin, since Chapter 8/11) is already the type every native function has had since print:

Builtin :: proc(vm: ^VM, args: []Value) -> Value

What's been missing is a single, host-facing way to install one. print and host_seconds are registered by hand, in two places each, vm.globals[name] = ... in vm_init and r.builtins[name] = true in resolver_init, because they're the language's own always-present builtins: every script gets them, regardless of host, so they belong to vm_init/resolver_init directly. A host's own builtins are different: nothing about spawn or move_to belongs inside muspel/ itself (the package never imports embers/, and never will), so a host needs its own way to reach both tables from outside. That's the whole of host.odin:

// muspel/host.odin
vm_register_native :: proc(vm: ^VM, r: ^Resolver, name: string, fn: Builtin) {
    vm.globals[name] = fn
    r.builtins[name] = true
}

One call, both tables updated together, so a host can never register a builtin the VM knows about but the resolver doesn't (or the reverse), the exact bug hand-writing both lines separately would eventually invite. Section 16.7's stub host calls this three times, once per builtin, before resolve_program ever runs, so log, spawn, and move_to are already known globals by the time the resolver looks for them, exactly like print always has been.

16.3 Marshalling: what crosses the boundary, and what doesn't

Every argument a script passes to a builtin arrives as a Value; every result a builtin returns becomes one. Numbers and strings need no translation at all, they already are Muspel values on the Odin side; a builtin just receives f64/string and hands one back the same way. The interesting case is anything that isn't naturally one of those five cases: an entity, a texture, a file handle, anything the host owns that a script needs to be able to refer to without holding it directly.

Design decision is too strong a name for this one, since there's really only one reasonable answer once Chapter 13's Value union is fixed: represent a host object as a handle, a plain number the host can use to look the real thing back up in its own table, never a pointer the script could hold and dereference. Section 16.7's spawn returns exactly this: an entity's id, an ordinary f64 as far as Muspel is concerned, meaningless to the script beyond "the thing to pass back into move_to later." This mirrors the dinn monorepo's own JS-handle-table convention (odindom's u32 keys into a JS-side object table, credited as the direct model): the same idea, a different boundary, solving the same underlying problem, that Odin/Muspel and JS/WASM alike have no safe way to hand a live reference across a language boundary, so neither tries to.

The host reaches its own state from inside a native builtin through VM.user_data, an opaque rawptr field added this chapter:

// vm.odin
VM :: struct {
    // ...
    user_data: rawptr,
}

A host sets it once, after vm_init, and every builtin it registers casts it back:

// tools/musc/host_cmd.odin
native_spawn :: proc(vm: ^muspel.VM, args: []muspel.Value) -> muspel.Value {
    host := (^Stub_Host)(vm.user_data)
    // ...
}

This is the standard shape a C-style embedding API gives native code a way back to host state without baking that state into the VM itself (Lua's own lua_State carries an equivalent void * extra-data slot for exactly this reason). It costs nothing when a host doesn't need it (user_data just stays nil), and it's what lets muspel/ stay entirely ignorant of what any particular host's own data even looks like.

16.4 Marshalling structs: reading a `Vec2` apart, and handing one back

Numbers, strings, and handles cover most of a host API, but not all of it: spawn/move_to naturally want a position, and Chapter 5 already gave Muspel scripts the right shape for one, Vec2{ x: .., y: .. }, rather than a builtin taking two loose numbers. arg_struct_field is arg_number/arg_string's own sibling for this case, the same by-name scan Get_Field (Chapter 13) does at the bytecode level, so a host never needs its own copy of a struct's field layout to unpack an argument:

// muspel/host.odin
arg_struct_field :: proc(args: []Value, i: int, field: string) -> (Value, bool) {
	inst, ok := arg_struct(args, i)
	if !ok do return nil, false
	for name, fi in inst.field_names {
		if name == field do return inst.values[fi], true
	}
	return nil, false
}

Handing a struct value back to a script (a builtin's return value, or an argument to a fired callback, Chapter 17) is marshalling's other direction, and it has one rule Chapter 10's arena design makes non-negotiable: the new instance has to be allocated from the calling VM instance's own arena, never the host's general heap and never another instance's arena, the same handles-not-pointers boundary rule extended from section 16.3 to values as well as entities.

// muspel/host.odin
make_struct_value :: proc(vm: ^VM, type_name: string, field_names: []string, values: []Value) -> Value {
	arena_values := make([]Value, len(values), value_arena_allocator(&vm.value_arena))
	inst := new(Struct_Instance, value_arena_allocator(&vm.value_arena))
	if len(arena_values) != len(values) || inst == nil {
		runtime_error(vm, "arena exhausted: cannot allocate struct '%s'", type_name)
		return nil
	}
	copy(arena_values, values)
	inst^ = Struct_Instance{type_name = type_name, field_names = field_names, values = arena_values}
	return inst
}

field_names is expected to be the same backing slice every instance of a given type shares, the same convention Struct_Layout already established (Chapter 10): a host that hands back the same struct shape repeatedly (every call to entity_pos, section 16.7) keeps one small, fixed table for it rather than building a fresh one per call.

16.5 Error handling at the boundary

A script calling a builtin with too few arguments, or an argument of the wrong type, is a mistake a script author will make, not a hypothetical. move_to(entity), missing its Vec2, or spawn(Vec2{x:"not a number", y:4}), a string where a number belongs inside an otherwise-correct struct, both compile and resolve cleanly (arity checking, section 7.4, only ever covers calls to known Muspel functions, never native ones, and Chapter 7's field checking validates a struct literal's keys, never the runtime type of the values behind them), so both would reach a builtin's own Odin code at run time. An unchecked args[0].(f64) there is exactly Chapter 14 exercise 3's 1 + true all over again: a bare Odin type-assertion panic, is not callable's ungraceful cousin.

arg_number and arg_string (host.odin) are the fix, and they're deliberately small:

// muspel/host.odin
arg_number :: proc(args: []Value, i: int) -> (f64, bool) {
    if i >= len(args) do return 0, false
    v, ok := args[i].(f64)
    return v, ok
}

arg_string :: proc(args: []Value, i: int) -> (string, bool) {
    if i >= len(args) do return "", false
    v, ok := args[i].(string)
    return v, ok
}

Both return ok = false rather than panicking, whether the argument is missing entirely or present with the wrong type, the same two-value comma-ok idiom every other type assertion in this book already uses. A builtin checks ok and, on failure, reaches for runtime_error (vm.odin), Chapter 14's own graceful-failure mechanism, no longer file-private from this chapter onward specifically so code outside vm.odin (any host's own builtins) can call it too:

// tools/musc/host_cmd.odin
x, x_ok := muspel.arg_struct_field(args, 0, "x")
y, y_ok := muspel.arg_struct_field(args, 0, "y")
xn, xn_ok := x.(f64)
yn, yn_ok := y.(f64)
if !x_ok || !y_ok || !xn_ok || !yn_ok {
    muspel.runtime_error(vm, "spawn expects (Vec2)")
}

./run.sh host examples/host_error.mus

runtime error: spawn expects (Vec2)
  at <script> (line 3)

A script gets one consistent kind of error, complete with the same call-stack trace machinery regardless of whether an opcode (Div, Call) or a native builtin is what actually detected the problem. This is still Chapter 14's own honest scope, not Chapter 18's: a boundary error here still ends the whole process, exactly as 1 + 1/0 already did. What's new is only that it is a runtime_error at all, not an unchecked panic.

16.6 `host_seconds`, revisited

Chapter 15's host_seconds was this book's first native builtin registered by a host-shaped need (a clock wait/wait_until could read), even though it was written before this chapter existed to name the pattern. It's worth looking back at it now that the pattern has a name: vm_init registers it directly (vm.globals["host_seconds"] = Builtin(native_host_seconds)) rather than through vm_register_native, because it's a language-level builtin, not a host one, every script gets it regardless of what embeds the VM. wait/wait_until themselves, built entirely in Muspel on top of it (section 15.7), needed no registration at all: they're ordinary functions a script's own source defines, the same way any func declaration is. Nothing about either changes this chapter; they're named here only to draw the line this chapter's own builtins sit on the other side of.

16.7 A first builtin set for Embers, stubbed against a fake host

tools/musc/host_cmd.odin builds the smallest possible thing worth calling a host: an id and a position, nothing else.

// tools/musc/host_cmd.odin
Stub_Entity :: struct {
    id:   int,
    x, y: f64,
}

Stub_Host :: struct {
    entities: [dynamic]Stub_Entity,
    next_id:  int,
}

Three builtins, registered onto a fresh VM/Resolver pair before anything is resolved:

r := muspel.resolver_init()
vm := muspel.vm_init()
host := Stub_Host{}
muspel.vm_register_native(&vm, &r, "log", native_log)
muspel.vm_register_native(&vm, &r, "spawn", native_spawn)
muspel.vm_register_native(&vm, &r, "move_to", native_move_to)
vm.user_data = &host

native_log is print in every way but name, proving the registration mechanism costs nothing extra for the simplest possible builtin. native_spawn and native_move_to are where section 16.3's handle convention and section 16.4's struct marshalling earn their keep together: spawn/move_to thread a Vec2 struct value, not two loose numbers, reading it apart with arg_struct_field; entity_pos is the return-direction proof, handing a freshly built Vec2 back into the calling VM instance's own arena:

// tools/musc/host_cmd.odin
Vec2_Fields := []string{"x", "y"}

native_spawn :: proc(vm: ^muspel.VM, args: []muspel.Value) -> muspel.Value {
	host := (^Stub_Host)(vm.user_data)
	x, x_ok := muspel.arg_struct_field(args, 0, "x")
	y, y_ok := muspel.arg_struct_field(args, 0, "y")
	xn, xn_ok := x.(f64)
	yn, yn_ok := y.(f64)
	if !x_ok || !y_ok || !xn_ok || !yn_ok {
		muspel.runtime_error(vm, "spawn expects (Vec2)")
		return nil
	}
	id := host.next_id
	host.next_id += 1
	append(&host.entities, Stub_Entity{id = id, x = xn, y = yn})
	return f64(id)
}

native_entity_pos :: proc(vm: ^muspel.VM, args: []muspel.Value) -> muspel.Value {
	host := (^Stub_Host)(vm.user_data)
	handle, ok := muspel.arg_number(args, 0)
	if !ok do return nil
	for e in host.entities {
		if e.id == int(handle) {
			return muspel.make_struct_value(vm, "Vec2", Vec2_Fields, {e.x, e.y})
		}
	}
	return nil
}

struct Vec2 { x, y }

log("hello from a script")
var e = spawn(Vec2{x:3, y:4})
move_to(e, Vec2{x:10, y:0})
print(entity_pos(e))

./run.sh host examples/host_demo.mus

hello from a script
spawn -> entity 0 at (3, 4)
move_to -> entity 0 now at (10, 0)
Vec2{x: 10, y: 0}

e is a plain number as far as the script is concerned, spawn's own Value result, and the only thing move_to ever does with it is look it back up in host.entities by id. The Vec2{x:.., y:..} values themselves cross the boundary in both directions, in and via arg_struct_field, out via make_struct_value, and the printed entity_pos(e) result is a struct instance built fresh from the calling script's own arena, indistinguishable from one the script had constructed itself. Nothing about this proves out only in principle: musc host runs it, end to end, with no embers/ package anywhere in this book's code/ yet.

The example script now needs its own struct Vec2 { x, y } declaration before it can build one to pass in: unlike a loose x, y number pair, a struct literal's type has to be a real, resolved struct (Chapter 7), so a host's own API surface and a script calling into it agree, by name, on what shape the data takes, the same discipline any two ends of a typed boundary need, enforced here entirely by Chapter 7's ordinary resolver, no separate host-side schema required.

Note the isolation this buys, worth confirming directly: the same script run through plain musc run (not musc host) fails to resolve at all, 'log' is undefined, because log/spawn/ move_to were never registered onto that command's own Resolver/VM pair. Nothing about Chapter 16 leaks into commands that didn't ask for it; musc run's own checkpoint from every earlier chapter is exactly as it was.

Checkpoint

musc host examples/host_demo.mus calls a script-defined log, spawn/move_to (both threading a Vec2 struct value), and entity_pos (handing one back) against a stub host built entirely inside musc, end to end, proving the registration and marshalling mechanism, structs included, before Embers exists to need it for real. A boundary mistake (spawn with a string where a number belongs inside its Vec2, move_to against an entity id that was never spawned) reports a graceful runtime_error with a full trace, never an Odin panic. Every earlier chapter's musc run/compile/debug/parse checkpoint is unaffected: log/spawn/move_to/entity_pos exist only inside musc host's own command.

Exercises

Add a fourth stub-host builtin, despawn(entity), that removes an entity from host.entities by id (or reports a graceful runtime_error if the id isn't found, matching move_to's own pattern). Register it alongside the other three in cmd_host.
native_spawn returns a plain f64 handle. Write a Muspel script that calls spawn twice and stores both results, then calls move_to on the first one after the second has been created, and confirm (by reading host.entities's own state, or by adding a temporary log-based dump) that the handle still refers to the right entity.
Section 16.3 calls a raw pointer crossing the boundary "the thing neither Odin/Muspel nor JS/WASM tries." Suppose a future builtin needed to hand a script read-only access to a host struct's several fields (an entity's x, y, and health, say) rather than one opaque handle. Sketch, in prose, how you'd extend this chapter's handle convention to cover that without ever giving the script a pointer, no code required.
Trigger vm_register_native being called with a name that collides with an existing Muspel function the script itself declares (func log(x) { ... } in a script also run through musc host). Read what actually happens (which one wins, and why) by tracing through compile_program's own function-hoisting pass (Chapter 12) and resolver_init's builtin table together; this isn't a bug this chapter fixes, just a real interaction worth understanding before Chapter 21 registers a much larger builtin set.
Add a builtin that takes an array argument instead of a struct one (sum(list), adding every element, following arg_array's shape alongside arg_struct), and confirm a script calling it with a non-array argument gets a graceful runtime_error, not a panic.