Chapter 20: Modules: scripts that import scripts

This chapter:

Add import "path" as alias: one .mus file pulling named declarations, structs, enums, functions, from another, always by a qualified name, never a bare one.
Resolve a whole graph of files, dependencies before dependents, detecting an import cycle as a clean error instead of unbounded recursion.
Compile each module exactly once and share it, the same "compiled once, shared across every user" split Chapter 17 already used for many VM instances of one script.
Turn common.mus, one shared struct, enum, and function set, into the thing every wave and boss script in Chapter 21 imports, instead of every file copy-pasting its own Vec2.

Where we left off: Every script this book has run has been one file. common.mus doesn't exist yet, and there's no way for one .mus file to reuse another's declarations at all.

20.1 The problem, made concrete

By the time a script codebase has more than one or two wave files, every one of them redefines the same Vec2 struct and the same lerp/distance helpers. Copy-paste drift, fix a bug in one copy, forget the other three, is a real, recurring bug source in any project shaped like this, not a hypothetical one. The requirement is precise: one script needs to say "give me the declarations from that other file," without the compiler re-typechecking or re-emitting them from scratch every time it's asked.

20.2 Grammar

importDecl ::= "import" STRING "as" IDENT ";" ;

// Stmt_Import is import "path" as alias: a path, relative to the importing file's own directory,
// and a mandatory local alias.
Stmt_Import :: struct {
	line:  int,
	path:  string,
	alias: string,
}

Like struct/enum/func, an import is top-level only, rejected with a clear Parse_Error (not silently accepted) if the parser finds one nested inside a block, the same shape Chapter 4 and Chapter 5 already established for every other top-level-only declaration.

20.3 Qualified imports only

Design decision: every reference to an imported declaration is qualified by its import alias. No bare, unqualified names ever enter an importing file's own namespace. Alternatives considered: import "common.mus" pulling every declaration directly into the importing file's own namespace, the way a wildcard or using import works in some languages. Trade-off: unqualified imports read shorter at the call site (Vec2 instead of common.Vec2), at the cost of silent name collisions as a script codebase grows: two modules each declaring a distance function, or a local Vec2 accidentally shadowing an imported one, are exactly the kind of confusing failure mode a modestly-experienced script author (a designer, not necessarily a programmer, Chapter 1's own audience) is least equipped to debug. Choice: qualified only. common.Vec2, common.Phase.Attacking, common.lerp(...), every single time, full stop. The extra typing is small and constant; the failure mode it closes off (a growing script codebase where it's no longer obvious which file a bare name came from) is exactly the kind that gets worse, not better, as a project grows, which is precisely when a script author has the least attention to spare for tracking it down.

20.4 Resolving the import graph

Gathering source text needs core:os, which doesn't exist on -target:js_wasm32 (the same constraint that already kept the step debugger's own I/O out of muspel/, Chapter 14). The split this chapter draws is the same one: muspel/modules.odin does the host-agnostic graph work (scanning, ordering, resolving, compiling), and tools/musc/module_loader.odin does the actual file reads, resolving each import's path relative to the importing file's own directory.

// muspel/modules.odin
Loaded_Module :: struct {
	path:    string,
	stmts:   []Stmt,
	imports: []Import_Ref,
}

scan_imports :: proc(stmts: []Stmt, allocator := context.allocator) -> []Import_Ref {
	refs := make([dynamic]Import_Ref, allocator)
	for s in stmts {
		if imp, ok := s.(^Stmt_Import); ok {
			append(&refs, Import_Ref{path = imp.path, alias = imp.alias, line = imp.line})
		}
	}
	return refs[:]
}

build_module_order walks that graph and returns every module in dependency order, an import always appearing before the file that imports it, using exactly the visited/visiting two-set pattern a topological sort needs:

@(private = "file")
visit_module :: proc(
	modules: ^map[string]Loaded_Module, path: string,
	visited, visiting: ^map[string]bool, result: ^[dynamic]string,
) -> (cycle_path: string, has_cycle: bool) {
	if visited[path] do return "", false
	if visiting[path] do return path, true
	visiting[path] = true
	if mod, ok := modules[path]; ok {
		for imp in mod.imports {
			cp, cyc := visit_module(modules, imp.path, visited, visiting, result)
			if cyc do return cp, true
		}
	}
	visiting[path] = false
	visited[path] = true
	append(result, path)
	return "", false
}

Gotcha: a naive walk with only a "already done" set, no "currently in progress" one, doesn't just miss a cycle, it stack-overflows trying to find it. The first version of this chapter's loader tracked only modules, the completed-module map, and checked if _, exists := modules[path]; exists do return true before recursing. That looks like it should catch a cycle: surely re-visiting an already-loaded module is a no-op. It isn't, because modules[path] is only written after a file's own imports have all been loaded, at the very end of the function that's still in the middle of loading a.mus when b.mus's own import of a.mus fires. a.mus importing b.mus importing a.mus genuinely crashed with a real stack overflow (exit code 139) before this was caught, because neither file was in modules yet when the cycle actually needed detecting. visiting, a second set, marked before recursing into a module's own imports and cleared only after they've all been attempted, is what actually closes the gap: it answers "is this file still being loaded right now, somewhere up the call stack," a question modules alone structurally cannot answer. Test the actual cycle, not just the happy path, is the lesson: a two-file mutual import is the smallest case that exercises it, and it's worth writing by hand rather than trusting the shape of the code to be obviously correct.

resolve_module_graph then resolves every module, dependencies first, copying each dependency's own struct/enum/function tables into every module that imports it, namespaced under that importer's own chosen alias:

resolve_module_graph :: proc(modules: map[string]Loaded_Module, order: []string, allocator := context.allocator) -> map[string]Resolver {
	resolvers := make(map[string]Resolver, allocator)
	for path in order {
		mod := modules[path]
		r := resolver_init(allocator)
		for imp in mod.imports {
			imp_r, ok := resolvers[imp.path]
			if !ok do continue
			for name, info in imp_r.structs {
				r.structs[strings.concatenate({imp.alias, ".", name}, allocator)] = info
			}
			for name, info in imp_r.enums {
				r.enums[strings.concatenate({imp.alias, ".", name}, allocator)] = info
			}
			for name, info in imp_r.functions {
				r.functions[strings.concatenate({imp.alias, ".", name}, allocator)] = info
			}
		}
		resolve_program(&r, mod.stmts, allocator)
		resolvers[path] = r
	}
	return resolvers
}

This is the entire mechanism behind common.Vec2 resolving correctly inside an importing script: by the time resolve_program runs on the entry file's own statements, r.structs["common.Vec2"] already exists, filled in from common.mus's own, already-resolved Struct_Info, keyed exactly the way section 7.5's Expr_Struct_Literal resolution already expects, r.structs[node.type_name] with no special-casing for where node.type_name came from. common.Phase.Attacking and common.lerp(...) resolve the identical way, through the qualified_prefix walk section 7.6 already built for a single level of Enum.Member, extended naturally to a dotted chain of any length by the same recursive structure.

20.5 Compiling a module

A module's own top-level declarations compile exactly as they would standing alone, compile_stmt and compile_expr (Chapters 11 and 12) unchanged, but an imported module's functions need one more step: re-exporting them under their qualified name, so an importing script's Get_Global "common.lerp" finds something.

// muspel/modules.odin
compile_module :: proc(chunk: ^Chunk, stmts: []Stmt, alias: string, allocator := context.allocator) {
	c := compiler_init(chunk, allocator)
	for s in stmts {
		if _, ok := s.(^Stmt_Func); ok do compile_stmt(&c, s)
	}
	for s in stmts {
		if _, ok := s.(^Stmt_Func); ok do continue
		compile_stmt(&c, s)
	}
	if alias != "" {
		for s in stmts {
			if fn, ok := s.(^Stmt_Func); ok {
				name_idx := chunk_add_const(c.chunk, Const_Value(fn.name))
				chunk_write_op(c.chunk, .Get_Global, fn.line)
				chunk_write(c.chunk, name_idx, fn.line)

				qualified := strings.concatenate({alias, ".", fn.name}, allocator)
				qual_idx := chunk_add_const(c.chunk, Const_Value(qualified))
				chunk_write_op(c.chunk, .Define_Global, fn.line)
				chunk_write(c.chunk, qual_idx, fn.line)
			}
		}
	}
	chunk_write_op(c.chunk, .Nil, 0)
	chunk_write_op(c.chunk, .Return, 0)
}

After each function's own ordinary Get_Global "lerp"; ... Define_Global "lerp" (Chapter 12), one extra pair, Get_Global "lerp"; Define_Global "common.lerp", re-binds the exact same ^Func_Proto under its qualified name too. The function is compiled exactly once; only a second, cheap global binding is added, not a second copy of its bytecode. compile_expr's Expr_Field_Access case reads this back with nothing new at all: a reference the resolver marked is_qualified_ref compiles straight to Get_Global on node.qualified_name, indistinguishable from any other global lookup.

Structs and enums need no such re-export: they're purely compile-time (a struct is a Struct_Layout constant baked into whoever constructs one, an enum member a plain integer), so there is no "the struct itself" runtime value that would need a qualified binding the way a function does.

20.6 Running the graph: shared bytecode, sequenced correctly

Sharing one module's compiled code across everything that imports it is the identical split Chapter 17 already made for many VM instances of one script: compile once, run its own top-level code once (defining its globals, plain and qualified), and every importer just calls into the functions that are now sitting in vm.globals, no recompilation, no re-running.

// muspel/modules.odin
run_module_graph :: proc(vm: ^VM, modules: map[string]Loaded_Module, order: []string, allocator := context.allocator) {
	aliases := canonical_aliases(modules, order, allocator)
	for path, i in order {
		mod := modules[path]
		is_entry := i == len(order) - 1
		chunk := new(Chunk, allocator)
		chunk^ = chunk_init(allocator)
		if is_entry {
			compile_program(chunk, mod.stmts, allocator)
		} else {
			compile_module(chunk, mod.stmts, aliases[path], allocator)
		}
		vm.status = .Running
		vm_push_script(vm, chunk)
		vm_run(vm)
		if vm.status == .Error do return
	}
}

Order matters here in exactly one way: every dependency's own chunk has to finish running, populating vm.globals with its bare and qualified bindings, before the entry script's own chunk starts, which is precisely what build_module_order's dependency-first ordering guarantees. vm.status = .Running before each vm_push_script/vm_run pair resets the previous module's own .Done status; skipping this would make vm_resume (Chapter 15) treat the next module as already finished before it ever runs a single instruction, a real bug caught by testing a genuine two-file import, not by reasoning about the code in isolation.

A program with no imports at all is the order == {entry_path} case: one module, itself, compiled with compile_program exactly as every earlier chapter's checkpoint already did. This is a strict superset of the pre-Chapter-20 single-file path, not a parallel one; every existing example in this book runs through this exact same function, unaffected, because it always has exactly one module in its own order.

20.7 `musc run` against a real multi-file example

examples/modules/common.mus
examples/modules/entry.mus

// common.mus
struct Vec2 { x, y }
enum Phase { Idle, Attacking, Fleeing }

func lerp(a, b, t) {
    return a + (b - a) * t
}
func distance(a, b) {
    var dx = a.x - b.x
    var dy = a.y - b.y
    return dx * dx + dy * dy
}

// entry.mus
import "common.mus" as common

var a = common.Vec2{x: 0, y: 0}
var b = common.Vec2{x: 3, y: 4}
print(common.distance(a, b))
print(common.lerp(0, 10, 0.25))

var phase = common.Phase.Idle
print(phase == common.Phase.Idle)
phase = common.Phase.Attacking
print(phase == common.Phase.Attacking)

./run.sh run examples/modules/entry.mus

25
2.5
true
true

Struct, enum, and function, all imported, all resolved and compiled through the graph machinery sections 20.4-20.6 built, no special-casing anywhere in entry.mus's own resolved AST to mark which declarations came from somewhere else, only the qualified names themselves.

An import that fails, a missing file, or an actual cycle, reports a clean error instead of a crash or a hang:

./run.sh run examples/modules/missing_import_entry.mus

examples/modules/missing_import_entry.mus:1: error: could not read imported module "does_not_exist.mus": Not_Exist

Checkpoint

musc run against examples/modules/entry.mus compiles and runs a multi-file program correctly: a shared module's struct, enum, and functions all resolve and execute through their qualified names. A missing import is reported cleanly. Two files importing each other is reported as an import cycle, not a stack overflow, caught during loading itself, not only during the later graph walk.

Exercises

Write a three-file chain, a.mus imports b.mus imports c.mus, each declaring one function the next one's own function calls by qualified name, and confirm build_module_order returns [c, b, a], dependencies before dependents, by tracing visit_module's own recursion on paper before checking it against what actually runs.
Reproduce this chapter's own Gotcha directly: write a.mus importing b.mus and b.mus importing a.mus, and confirm musc run reports a clean cycle error rather than crashing. Then, in a scratch copy of tools/musc/module_loader.odin, remove the loading set entirely (check only modules) and confirm the crash the Gotcha describes actually reproduces.
Have two different files import the same third file under two different aliases (import "common.mus" as c in one, import "common.mus" as shared in the other). Read canonical_aliases's own doc comment and predict, before running anything, which alias the compiled qualified functions end up bound under, then confirm your prediction against musc compile's own disassembly of the shared module's chunk.
common.mus declares Phase with three members. Add a fourth, and confirm every importing script's own common.Phase.<new member> reference resolves correctly with no changes needed anywhere in entry.mus itself, just recompiling.