Foreword

"Why am I making a programming language?"

This question keeps rolling around in my head as I'm typing out the code that is slowly shaping into Rune. Programming is like magic. You imagine it in your mind, write it out, and there it is. Doing stuff which wasn't being done before.

Truth be told, I'm scared that people will tell me that I'm wasting my time. This has already been done, or "Why not just use X?". A thing so glaringly obvious that all of my efforts are wasted.

But you actually don't need a reason. It can simply be for The Joy of Creating, and then it's just you. Spending your own time. No harm done.

But I want to talk about why I'm making Rune beyond just for fun. So I'm dedicating this foreword to it. I feel obligated to describe why this might matter to others.

So here's why I'm making a new programming language.

I've spent a lot of effort working on OxidizeBot, a Twitch bot that streamers can use to add commands and other interactive things in their chat. I built it for myself while streaming. When adding features I always spend way too much time tinkering with it. Making it as generic as possible so it can solve more than just one problem. When it's a personal project, I don't care about being efficient. I care much more about doing things the right way.

...

Ok, I sometimes do that professionally as well. But a working environment is much more constrained. Personal projects should be fun!

Anyway, that means the bot isn't overly specialized to only suit my needs and can be used by others. It's starting to see a little bit of that use now which is a lot of fun. I made something which helps people do something cool.

All the commands in the bot are written in Rust, and compiled straight into the bot. This is nice because Rust is an incredible language. But Rust is also complex. Not needlessly mind you. I believe it's complex because it tackles really hard problems. And that usually comes with a base level of complexity it's very hard to get rid of.

But it's still tricky enough that streamers who have limited programming experience struggle getting up and running. I wanted them to be able to write their own commands. Ones they could just drop into a folder and presto - you're up and running.

To this day I've tutored two of these streamers who were interested in learning Rust to write their own commands.

Embedding a Rust compiler isn't feasible. So I started looking into dynamic programming languages. Ones that could be embedded into an existing application with little to no effort. That seamlessly integrates with its environment. A number of candidates came up, but the one that stood out the most to me was Rhai.

So why is Rhai awesome? It has Rust-like syntax. The runtime is fully written in mostly safe Rust, and can be easily embedded. Hooking up Rust functions is a piece of cake.

But Rhai has a set of design decisions which didn't exactly scratch my itch. The more I used it, the more I got inspired and started thinking about things that could be changed or added. I contributed a bit to the project. And it started to dawn on me that Rhai's approach wasn't exactly what I wanted. There's nothing wrong with this. The authors of Rhai have specific goals and ideas of what they want to accomplish. While it would be feasible to push Rhai in a different direction, the project would emerge looking much different on the other side. Which wouldn't be fair towards the people leveraging Rhai's strengths today. So I wanted a clean slate to find my own compromises. To discover freely what works and doesn't work well.

When I started working on Rune I had the following rough goals in mind:

• Performance should be comparable to Lua and Python (And eventually LuaJIT when we have cranelift).
• Scripts should compile quickly.
• Rune should feel like "Rust without types".
• Excellent support for asynchronous programming (i.e. native select statements).
• Be as good as Rhai when it comes to integrating with native Rust.
• Work well through C bindings.
• A minimalistic stack-based runtime that is strictly single threaded*.

*: If this feels like a step backwards to you, don't worry too much. We can still have concurrency and threading using async code as you'll see later in this book.

Rune is now in a state where I want people to poke at it. Not too hard mind you. It's still early days. The compiler is very much in flux and a miscompilation will definitely cause the wrong closure to be called. You know, the one that doesn't perform your security checks 😅.

But the more poking and prodding people do, the more issues will be found. Every solved issue brings Rune one step close to being production ready. Every set of eyeballs that takes a look can bring fresh perspective and ideas, making the project better for me and everyone else.

I really want to thank Jonathan Turner and all the contributors to the Rhai project. They have been an an immense inspiration to me.

You can find the project on its GitHub page. I hope you'll enjoy using it as much as I've enjoyed making it!

— John-John Tedro

Introduction

Welcome the The Rune Programming Language, a reference guide that will familiarize yourself with Rune.

Rune is an open source embeddable dynamic programming language that compiles and runs on a virtual machine called Runestick (thanks Brendan).

The goal of Rune is to reimagine Rust as a dynamic programming language. Trying to mimic as many concepts as possible, and remixing the ones which do not translate directly. We do this by using the same syntax as Rust. But a few additions are inevitable because certain things are just done differently when you have a dynamic environment.

I also concede that a number of program correctness features you get through static typing will be sorely lacking. The tradeoff you get for this are fast compilation times and duck typing, sometimes leading to more concise and compact code. Python is a great example of this, and is along with Rhai and Lua biggest inspirations for this project.

To read this book, you will definitely want to go to the GitHub project and grab yourself a clone of it. All the examples are in there, and it's highly recommended that you run and tinker them yourself as you encounter them.

With that out of the way, let's get started. We have a bit to go through.

Getting Started

The first thing you need to learn about in Rune is the dbg function. This is used to "debug" values provided to it in order to understand them. Anything can be provided to it, and it will do its best to describe it.

fn function() {
    42
}

let a = [1, 2, 3];
let b = '今';
let closure = || println!("Hello");

dbg!(a);
dbg!(b);
dbg!(function);
dbg!(drop);
dbg!(closure);

Note: by convention Rune uses files ending in .rn.

$> cargo run -- run scripts/book/getting_started/dbg.rn
[1, 2, 3]
'今'
dynamic function (at: 0x1a)
native function (0x1bd03b8ee40)
dynamic function (at: 0x17)

The default dbg implementation outputs information on its arguments to stdout. But its exact behavior can differ depending on how the environment is configured. When Rune is embedded into a larger application it might for example be more suitable to output to a log file.

Rune also provides print! and println! macros which can be used to format directly to stdout, but these cannot be relied on to be present to the same degree as dbg. However for our purposes we will be using rune-cli, which has all of these modules installed. This is also what was used to run the above code.

So for a more formal introduction, here is the official Rune "Hello World":

println!("Hello World");

$> cargo run -- run scripts/book/getting_started/hello_world.rn
Hello World

So now you know how to run Rune scripts. Well done! Let's move on to the next chapter.

Concepts

This chapter covers common concepts that appear in almost all programming languages, and how they work in Rune.

Hopefully these should be familiar to anyone who's used imperative programming languages before. We'll try to take each concept and describe how they work with examples, one at a time.

Items and imports

Everything in Rune has a unique name. Every function, type, and import. This name is what identifies that thing, and is called its item. Rune performs compile time checks to make sure that every item we try to use actually exists.

The following are examples of items in Rune:

• std::result::Result (a type)
• std::iter::range (a function)

The first refers to the Result enum, and the second is the range function. They both live within their corresponding std module. Result is a bit special even, since it's part of the prelude, allowing us to use it without importing it. But what about range?

If we wanted to use range we would have to import it first with a use statement:

use std::iter::once;

let it = once(0);

dbg!(it.next());
dbg!(it.next());

$> cargo run -- run scripts/book/items_imports/example_import.rn
std::iter::Range

Trying to use an item which doesn't exist results in a compile error:

pub fn main() {
    let foo = Foo::new();
}

$> cargo run -- run scripts/book/items_imports/missing_item.rn.fail
error: compile error
  ┌─ scripts/book/items_imports/missing_item.rn.fail:2:15
  │
2 │     let foo = Foo::new();
  │               ^^^^^^^^ missing item `Foo::new`

Every item used in a Rune program must be known at compile time. This is one of the static guarantees every Rune script are has to fulfill. And is one important point where it differs from Lua or Python.

Modules

Rune has support for modules purely defined in Rune itself. This is done using the mod keyword. And the module can either be loaded from a different file matching the name of the module or defined directly inside of the source file.

The following is an example of an inline module:

mod foo {
    pub fn number() {
        1
    }
}

mod bar {
    pub fn number() {
        2
    }
}

pub fn main() {
    dbg!(foo::number() + bar::number());
}

$> cargo run -- run scripts/book/items_imports/inline_modules.rn
3

And this is the equivalent modules loaded from the filesystem. These are three separate files:

mod foo;
mod bar;

dbg!(foo::number() + bar::number());

// file: ./foo/mod.rn
pub fn number() {
    2
}

// file: ./bar.rn
pub fn number() {
    1
}

$> cargo run -- run scripts/book/items_imports/modules.rn
3

Disambiguating imports

Normally an item would simply be used through its local name, such as foo::number above. But what happens if we need to reference a module which is not a direct descendent of the current one or there is some ambiguation?

To this end Rune supports the following Rust keywords:

• self - which will resolve items from the root of the current module.
• crate - which will look up items from the entrypoint of the current project.
• super - which will look up items from the parent of the current module.

mod first {
    pub fn number() {
        crate::number() + 2
    }
}

mod second {
    pub fn number() {
        super::first::number() + 4
    }
}

pub fn number() {
    1
}

dbg!(self::second::number());

$> cargo run -- run scripts/book/items_imports/item_keywords.rn
7

Visibility

Every item used has to be visible to that item. This is governed by Runes visibility rules, which are the following:

• An item can have inherited (empty) or a specified visibility like pub or pub(crate).
• For an item to be visible, all of its parent items have to be visible.
• Items with inherited visibility are equivalent to pub(self), making the item only visible in the module in which they are defined.

The available visibility modifiers are:

• pub - the item is visible from anywhere.
• pub(crate) - the item is visible in the same crate.
• pub(super) - the item is visible in the parent item only.
• pub(self) - the item is only visible to other items in the same module.
• pub(in path) - the item is only visible in the specified path. This is not supported yet.

Note that Rune doesn't have support for crates yet, meaning pub(crate) and pub are currently effectively equivalent.

Functions

One of the most common things in all of programming are functions. These are stored procedures which take a arguments, do some work, and then return. Functions are used because they encapsulate what they do so that the programmer only needs to concern itself with the protocol of the function.

What does it do? What kind of arguments does it take? The alternative would be to copy the code around and that wouldn't be very modular. Functions instead provide a modular piece of code that can be called and re-used. Over and over again.

fn keyword

In Rune, functions are declared with the fn keyword. You've already seen one which is used in every example, main. This is not a special function, but is simply what the Rune cli looks for when deciding what to execute.

pub fn main() {
    println!("Hello World");
}

$> cargo run -- run scripts/book/functions/main_function.rn
Hello World

In Rune, you don't have to specify the return type of a function. Given that Rune is a dynamic programming language, this allows a function to return anything, even completely distinct types.

fn foo(condition) {
    if condition {
        "Hello"
    } else {
        1
    }
}

pub fn main() {
    println!("{}", foo(true));
    println!("{}", foo(false));
}

$> cargo run -- run scripts/book/functions/return_value.rn
Hello
1

Depending on who you talk to, this is either the best thing since sliced bread or quite scary. It allows for a larger ability to express a program, but at the same time it can be harder to reason on what your program will do.

Calling functions in Rust

Rune functions can be easily set up and called from Rust.

use rune::termcolor::{ColorChoice, StandardStream};
use rune::{Diagnostics, Vm};

use std::sync::Arc;

fn main() -> rune::support::Result<()> {
    let context = rune_modules::default_context()?;

    let mut sources = rune::sources!(
        entry => {
            pub fn main(number) {
                number + 10
            }
        }
    );

    let mut diagnostics = Diagnostics::new();

    let result = rune::prepare(&mut sources)
        .with_context(&context)
        .with_diagnostics(&mut diagnostics)
        .build();

    if !diagnostics.is_empty() {
        let mut writer = StandardStream::stderr(ColorChoice::Always);
        diagnostics.emit(&mut writer, &sources)?;
    }

    let unit = result?;

    let mut vm = Vm::new(Arc::new(context.runtime()?), Arc::new(unit));
    let output = vm.execute(["main"], (33i64,))?.complete().into_result()?;
    let output: i64 = rune::from_value(output)?;

    println!("output: {}", output);
    Ok(())
}

$> cargo run --example minimal
output: 43

Control Flow

Rune supports a number of control flow expressions. We will be dedicating this section to describe the most common ones.

return expression

In the previous section we talked about functions. And one of the primary things a function does is return things. The return expression allows for returning from the current function. If used without an argument, the function will return a unit ().

The last statement in a function is known as an implicit return, and will be what the function returns by default unless a return is specified.

fn foo(n) {
    if n < 1 {
        return "less than one";
    }

    "something else"
}

println!("{}", foo(0)); // => outputs: "less than one"
println!("{}", foo(10)); // => outputs: "something else"

$> cargo run -- run scripts/book/control_flow/numbers_game.rn
less than one
something else

if expressions

If expressions allow you to provide a condition with one or more code branches. If the condition is true, the provided block of code will run.

let number = 3;

if number < 5 {
    println!("The number is smaller than 5");
}

$> cargo run -- run scripts/book/control_flow/conditional.rn
The number *is* smaller than 5

Optionally, we can add another branch under else, which will execute in case the condition is false.

let number = 3;

if number < 5 {
    println!("the number is smaller than 5");
} else {
    println!("the number is 5 or bigger");
}

$> cargo run -- run scripts/book/control_flow/conditional_else.rn
the number is smaller than 5

We can also add an arbitrary number of else if branches, which allow us to specify many different conditions.

let number = 3;

if number < 5 {
    println!("the number is smaller than 5");
} else if number == 5 {
    println!("the number is exactly 5");
} else {
    println!("the number is bigger than 5");
}

$> cargo run -- run scripts/book/control_flow/conditional_else_ifs.rn
the number is smaller than 5

Do note however that if you have many conditions, it might be cleaner to use a match.

This will be covered in a later section, but here is a sneak peek:

let number = 3;

match number {
    n if n < 5 => {
        println!("the number is smaller than 5");
    }
    5 => {
        println!("the number is exactly 5");
    }
    n => {
        println!("the number is bigger than 5");
    }
}

$> cargo run -- run scripts/book/control_flow/first_match.rn
the number is smaller than 5

Variables and memory

Variables in Rune are defined using the let keyword. In contrast to Rust, all variables in Rune are mutable and can be changed at any time.

pub fn main() {
    let x = 5;
    println!("The value of x is: {}", x);
    x = 6;
    println!("The value of x is: {}", x);
}

$> cargo run -- run scripts/book/variables/variables.rn
The value of x is: 5
The value of x is: 6

Rune is a memory safe language. Regardless of what you write in a Rune script, we maintain the same memory safety guarantees as safe Rust. This is accomplished through reference counting.

Unless a value is Copy, they are reference counted and can be used at multiple locations. This means that they have shared ownership. Every variable that points to that value therefore points to the same instance of that value. You can think of every nontrivial value being automatically wrapped in an Rc<RefCell<T>> if that helps you out.

This is not exactly what's going on. If you're interested to learn more, Rune uses a container called Shared<T> which is like an Rc<RefCell<T>>, but has a few more tricks.

We can see how this works by sharing and mutating one object across two variables:

pub fn main() {
    let object = #{ field: 1 };
    let object2 = object;
    println!("{}", object.field);
    object2.field = 2;

    // Note: we changed `object2`, but read out `object`
    println!("{}", object.field);
}

$> cargo run -- run scripts/book/variables/shared_ownership.rn
1
2

This can cause issues if we call an external function which expects to take ownership of its arguments. We say that functions like these move their argument, and if we try to use a variable which has been moved an error will be raised in the virtual machine.

Note: Below we use the drop function, which is a built-in function that will take its argument and free it.

pub fn main() {
    let object = #{ field: 1 };
    let object2 = object;
    println!("field: {}", object.field);
    drop(object2);
    println!("field: {}", object.field);
}

$> cargo run -- run scripts/book/variables/take_argument.rn
field: 1
== ! (cannot read, value is moved (at 14)) (469µs)
error: virtual machine error
  ┌─ scripts/book/variables/take_argument.rn:6:27
  │
6 │     println!("field: {}", object.field);
  │                           ^^^^^^^^^^^^ cannot read, value is moved

If you need to, you can test if a variable is still accessible for reading with is_readable, and for writing with is_writable. These are both imported in the prelude. An object which is writable is also movable, and can be provided to functions which need to move the value, like drop.

pub fn main() {
    let object = #{ field: 1 };
    let object2 = object;
    println!("field: {}", object.field);
    drop(object2);

    if is_readable(object) {
        println!("field: {}", object.field);
    } else {
        println!("object is no longer readable 😢");
    }
}

$> cargo run -- run scripts/book/variables/is_readable.rn
field: 1
object is no longer readable 😢

Loops

Loops are a fundamental building block common to many programming languages. This is no exception in Rune. Loops allow you to execute a block of code until a specific condition is reached, which can be a powerful tool for accomplishing programming tasks.

break Keyword

Every loop documented in this section can be terminated early using the break keyword.

When Rune encounters a break, it will immediately jump out of the loop it is currently in and continue running right after it.

let value = 0;

while value < 100 {
    if value >= 50 {
        break;
    }

    value = value + 1;
}

println!("The value is {}", value); // => The value is 50

$> cargo run -- run scripts/book/loops/while_loop.rn
The value is 50

loop Expressions

The loop keyword builds the most fundamental form of loop in Rune. One that repeats unconditionally forever, until it is exited using another control flow operator like a break or a return.

use time::Duration;

loop {
    println!("Hello forever!");
    time::sleep(Duration::from_secs(1)).await;
}

$> cargo run -- run scripts/book/loops/loop_forever.rn
Hello forever!
Hello forever!
Hello forever!
...

Hint: If you want this one to end, you're gonna have to kill it with CTRL+C.

We're also using an asynchronous function called sleep above to avoid spamming our terminals too much. Well talk more about these in a later section.

When broken out of, loops produce the value provided as an argument to the break keyword. By default, this is simply a unit ().

let counter = 0;

let total = loop {
    counter = counter + 1;

    if counter > 10 {
        break counter;
    }
};

println!("The final count is: {}", total);

$> cargo run -- run scripts/book/loops/loop_break.rn
The final count is: 11

Pattern matching

In this section we will be discussing Pattern Matching.

Pattern matching is a flexible mechanism that allows for validating the structure and type of the argument, while also destructuring it to give easy access to what you need.

Below are some examples of its common uses to match on branch conditions:

fn match_input(n) {
    match n {
        1 => println!("The number one."),
        n if n is i64 => println!("Another number: {}.", n),
        [1, 2, n, ..] => println!("A vector starting with one and two, followed by {}.", n),
        "one" => println!("One, but this time as a string."),
        _ => println!("Something else. Can I go eat now?"),
    }
}

match_input(1);
match_input(2);
match_input([1, 2, 42, 84]);
match_input("one");
match_input(#{ field: 42 });

$> cargo run -- run scripts/book/pattern_matching/big_match.rn
The number one.
Another number: 2.
A vector starting with one and two, followed by 42.
One, but this time as a string.
Something else. Can I go eat now?

We will be covering each of these variants in detail in the coming sections.

Patterns

Things that can be matched over are called patterns, and there's a fairly large number of them. In this section we'll try to document the most common ones.

Patterns that can be matched over are the following:

• A unit, simply ().
• A boolean value, like true or false.
• A byte, like b'a' or b'\x10'.
• A character, like 'a' or 'あ'.
• An integer, like 42.
• A string, like "Steven Universe".
• A vector, like the numbers [1, _, ..], or simply the empty vector []. The values in the vectors are patterns themselves.
• A tuple, like ("Steven Universe", _, 42). The values in the tuple are patterns themselves.
• An object, like the numbers {"name": "Steven Universe", "age": _}, or the empty {}. The values in the object are patterns themselves.

Structs can be matched over by prefixing the match with their name:

• A unit struct: Foo.
• A tuple struct: Foo(1, _).
• An object struct: Foo { bar: 1, .. }.

Similarly, variants in an enum can be matched over as well in the same way:

• A unit variant: Foo::Variant.
• A tuple variant: Foo::Variant(1, _).
• An object variant: Foo::Variant { bar: 1, .. }.

Patterns can be almost any combination of the above. Even {"items": ["Sword", "Bow", "Axe"]} is a pattern that can be matched over.

Anything that qualifies as a collection can have .. as a suffix to match the case that there are extra fields or values which are not covered in the pattern. This is called a rest pattern.

let value = #{ a: 0, b: 1 };

let matched = match value {
    // this doesn't match, because a pattern without a rest pattern in it
    // must match exactly.
    #{ a } => false,
    // this matches, because it only requires `a` to be present.
    #{ a, .. } => true,
};

assert!(matched, "rest pattern matched");

$> cargo run -- run scripts/book/pattern_matching/rest_pattern.rn

Binding and ignoring

In a pattern, every value can be replaced with a binding or an ignore pattern. The ignore pattern is a single underscore _, which informs Rune that it should ignore that value, causing it to match unconditionally regardless of what it is.

fn test_ignore(vector) {
    match vector {
        [_, 2] => println!("Second item in vector is 2."),
    }
}

test_ignore([1, 2]);

$> cargo run -- run scripts/book/pattern_matching/ignore.rn
Second item in vector is 2.

In contrast to ignoring, we can also bind the value to a variable that is then in scope of the match arm. This will also match the value unconditionally, but give us access to it in the match arm.

fn test_ignore(vector) {
    match vector {
        [_, b] => println!("Second item in vector is {}.", b),
    }
}

test_ignore([1, 2]);

$> cargo run -- run scripts/book/pattern_matching/bind.rn
Second item in vector is 2.

Here are some more examples:

• [_, a, b] which will ignore the first value in the vector, but then bind the second and third as a and b.
• {"name": name} will bind the value name out of the specified object.

fn describe_car(car) {
    match car {
        #{ "make": year, .. } if year < 1950 => "What, where did you get that?",
        #{ "model": "Ford", "make": year, .. } if year >= 2000 => "Pretty fast!",
        _ => "Can't tell 😞",
    }
}

println!("{}", describe_car(#{"model": "Ford", "make": 2000}));
println!("{}", describe_car(#{"model": "Honda", "make": 1980}));
println!("{}", describe_car(#{"model": "Volvo", "make": 1910}));

$> cargo run -- run scripts/book/pattern_matching/fast_cars.rn
Pretty fast!
Can't tell 😞
What, where did you get that?

Template literals

If you've been paying attention on previous sections you might have seen odd looking strings like `Hello ${name}`. These are called template literals, and allow you to conveniently build strings using variables from the environment.

Template literals are a concept borrowed from EcmaScript.

let age = 30;
dbg!(`I am ${age} years old!`);

$> cargo run -- run scripts/book/template_literals/basic_template.rn
"I am 30 years old!"

Template strings are accelerated by the Vm, each argument uses a display protocol and it can be very efficient to build complex strings out of it.

The DISPLAY_FMT protocol

The DISPLAY_FMT protocol is a function that can be implemented by any external type which allows it to be used in a template string.

It expects a function with the signature fn(&self, buf: &mut String) -> fmt::Result.

use rune::{ContextError, Module};
use rune::runtime::{Protocol, Formatter};
use std::fmt::Write as _;
use std::fmt;

#[derive(Debug)]
pub struct StatusCode {
    inner: u32,
}

impl StatusCode {
    #[rune::function(protocol = DISPLAY_FMT)]
    fn display_fmt(&self, f: &mut Formatter) -> fmt::Result {
        write!(f, "{}", self.inner)
    }
}

pub fn module() -> Result<Module, ContextError> {
    let mut module = Module::new(["http"]);
    module.function_meta(StatusCode::display_fmt)?;
    Ok(module)
}

This is what allows status codes to be formatted into template strings, any types which do not implement this protocol will fail to run.

let vec = [1, 2, 3];
dbg!(`${vec}`);

$> cargo run -- run scripts/book/template_literals/not_a_template.rn
== ! (`Vec` does not implement the `display_fmt` protocol (at 5)) (77.7µs)
error: virtual machine error
  ┌─ scripts/book/template_literals/not_a_template.rn:3:9
  │
3 │     dbg!(`${vec}`);
  │          ^^^^^^^^ `Vec` does not implement the `display_fmt` protocol

Instance functions

Instance functions are functions that are associated to a specific type of variable. When called they take the form value.foo(), where the instance is the first part value. And the instance function is foo().

These are a bit special in Rune. Since Rune is a dynamic programming language we can't tell at compile time which instance any specific value can be. So instance functions must be looked up at runtime.

struct Foo;

impl Foo {
    fn new() {
        Foo
    }
}

let foo = Foo::new();
foo.bar();

$> cargo run -- run scripts/book/instance_functions/missing_instance_fn.rn
error: virtual machine error
   ┌─ scripts/book/instance_functions/missing_instance_fn.rn:11:5
   │
11 │     foo.bar();
   │     ^^^^^^^^^ missing instance function `0xfb67fa086988a22d` for `type(0xc153807c3ddc98d7)``

Note: The error is currently a bit nondescript. But in the future we will be able to provide better diagnostics by adding debug information.

What you're seeing above are type and function hashes. These uniquely identify the item in the virtual machine and is the result of a deterministic computation based on its item. So the hash for the item Foo::new will always be the same.

In Rust, we can calculate this hash using the Hash::type_hash method:

use rune::{Hash, ItemBuf};

fn main() -> rune::support::Result<()> {
    println!("{}", Hash::type_hash(&ItemBuf::with_item(["Foo", "new"])?));
    println!("{}", Hash::type_hash(["Foo", "new"]));
    Ok(())
}

$> cargo run --example function_hash
0xb5dc92ab43cb37d9
0xb5dc92ab43cb37d9

The exact implementation of the hash function is currently not defined, but will be stabilized and documented in a future release.

Defining instance functions in Rust

Native instance functions are added to a runtime environment using the Module::associated_function function. The type is identified as the first argument of the instance function, and must be a type registered in the module using Module::ty.

use rune::termcolor::{ColorChoice, StandardStream};
use rune::{ContextError, Diagnostics, Module, Vm};

use std::sync::Arc;

#[rune::function(instance)]
fn divide_by_three(value: i64) -> i64 {
    value / 3
}

#[tokio::main]
async fn main() -> rune::support::Result<()> {
    let m = module()?;

    let mut context = rune_modules::default_context()?;
    context.install(m)?;
    let runtime = Arc::new(context.runtime()?);

    let mut sources = rune::sources!(entry => {
        pub fn main(number) {
            number.divide_by_three()
        }
    });

    let mut diagnostics = Diagnostics::new();

    let result = rune::prepare(&mut sources)
        .with_context(&context)
        .with_diagnostics(&mut diagnostics)
        .build();

    if !diagnostics.is_empty() {
        let mut writer = StandardStream::stderr(ColorChoice::Always);
        diagnostics.emit(&mut writer, &sources)?;
    }

    let unit = result?;

    let mut vm = Vm::new(runtime, Arc::new(unit));
    let output = vm.execute(["main"], (33i64,))?.complete().into_result()?;
    let output: i64 = rune::from_value(output)?;

    println!("output: {}", output);
    Ok(())
}

fn module() -> Result<Module, ContextError> {
    let mut m = Module::with_item(["mymodule"])?;
    m.function_meta(divide_by_three)?;
    Ok(m)
}

$> cargo run --example custom_instance_fn
output: 11

For more examples on how modules can be used you can have a look at the source for the rune-modules crate.

Field functions

Field functions are special operations which operate on fields. These are distinct from associated functions, because they are invoked by using the operation associated with the kind of the field function.

The most common forms of fields functions are getters and setters, which are defined through the Protocol::GET and Protocol::SET protocols.

The Any derive can also generate default implementations of these through various #[rune(...)] attributes:

#[derive(Any)]
struct External {
    #[rune(get, set, add_assign, copy)]
    number: i64,
    #[rune(get, set)]
    string: String,
}

Once registered, this allows External to be used like this in Rune:

pub fn main(external) {
    external.number = external.number + 1;
    external.number += 1;
    external.string = `${external.string} World`;
}

The full list of available field functions and their corresponding attributes are:

The manual way to register these functions is to use the new Module::field_function function. This clearly showcases that there's no relationship between the field used and the function registered:

use rune::{Any, Module};
use rune::runtime::Protocol;

#[derive(Any)]
struct External {
}

impl External {
    fn field_get(&self) -> String {
        String::from("Hello World")
    }
}

let mut module = Module::new();
module.field_function(&Protocol::GET, "field", External::field_get)?;

Would allow for this in Rune:

pub fn main(external) {
    println!("{}", external.field);
}

Custom field function

Using the Any derive, you can specify a custom field function by using an argument to the corresponding attribute pointing to the function to use instead.

The following uses an implementation of add_assign which performs checked addition:

use rune::runtime::{VmError, VmResult};
use rune::termcolor::{ColorChoice, StandardStream};
use rune::{Any, ContextError, Diagnostics, Module, Vm};

use std::sync::Arc;

#[derive(Any)]
struct External {
    #[rune(add_assign = External::value_add_assign)]
    value: i64,
}

#[allow(clippy::unnecessary_lazy_evaluations)]
impl External {
    fn value_add_assign(&mut self, other: i64) -> VmResult<()> {
        self.value = rune::vm_try!(self.value.checked_add(other).ok_or_else(VmError::overflow));
        VmResult::Ok(())
    }
}

fn main() -> rune::support::Result<()> {
    let m = module()?;

    let mut context = rune_modules::default_context()?;
    context.install(m)?;
    let runtime = Arc::new(context.runtime()?);

    let mut sources = rune::sources! {
        entry => {
            pub fn main(e) {
                e.value += 1;
            }
        }
    };

    let mut diagnostics = Diagnostics::new();

    let result = rune::prepare(&mut sources)
        .with_context(&context)
        .with_diagnostics(&mut diagnostics)
        .build();

    if !diagnostics.is_empty() {
        let mut writer = StandardStream::stderr(ColorChoice::Always);
        diagnostics.emit(&mut writer, &sources)?;
    }

    let unit = result?;

    let mut vm = Vm::new(runtime, Arc::new(unit));

    let input = External { value: i64::MAX };
    let err = vm.call(["main"], (input,)).unwrap_err();
    println!("{:?}", err);
    Ok(())
}

fn module() -> Result<Module, ContextError> {
    let mut m = Module::new();
    m.ty::<External>()?;
    Ok(m)
}

$> cargo run --example checked_add_assign
Error: numerical overflow (at inst 2)

Traits

Traits in rune defines a collection associated items. Once a trait is implemented by a type we can be sure that all the associated names it defines are present on the type.

Traits allow us to reason about types more abstractly, such as this is an iterator.

Limits

As usual, Rune doesn't permit more than one definition of an associated name. Attempting to define more than one with the same name results in a build-time error. This is in contrast to Rust which allows multiple traits with overlapping methods to be defined. So why doesn't Rune allow for this?

Since Rune is a dynamic language, consider what would happen in a situation like this:

#![allow(unused)]
fn main() {
struct Foo {
    /* .. */
}

impl Iterator for Foo {
    fn next(self) {
        /* .. */
    }
}

impl OtherIterator for Foo {
    fn next(self) {
        /* .. */
    }
}

let foo = Foo {
    /* .. */
};

// Which implementation of `next` should we call?
while let Some(value) = foo.next() {

}
}

Since there are no type parameters we can't solve the ambiguity by either only having one trait defining the method in scope or by using an unambigious function qualified function call.

Implementation

In the background the user-facing implementation of traits is done by implementing protocols just before. Protocols are still used by the virtual machine to call functions.

The separation that protocols provide is important because we don't want a user to accidentally implement an associated method which would then be picked up by a trait. Protocols are uniquely defined in their own namespace and cannot be invoked in user code.

As an example, to implement the Iterator trait you have to implement the NEXT protocol. So if the NEXT protocol is present and we request that the ::std::iter::Iterator trait should be implemented, the NEXT protocol implementation is used to construct all the relevant associated methods. This is done by calling Module::implement_trait.

#![allow(unused)]
fn main() {
let mut m = Module::with_item(["module"]);
m.ty::<Iter>()?;
m.function_meta(Iter::next__meta)?;
m.function_meta(Iter::size_hint__meta)?;
m.implement_trait::<Iter>(rune::item!(::std::iter::Iterator))?;

#[derive(Any)]
#[rune(item = "module")]
struct Iter {
    /* .. */
}

impl Iter {
    #[rune::function(keep, protocol = NEXT)]
    fn size_hint(&self) -> Option<bool> {
        Some(true)
    }

    #[rune::function(keep, protocol = SIZE_HINT)]
    fn size_hint(&self) -> (usize, Option<usize>) {
        (1, None)
    }
}
}

Note that this allows the Iter type above to specialize its SIZE_HINT implementation. If the SIZE_HINT protocol was not defined, a default implementation would be provided by the trait.

As a result of implementing the ::std::iter::Iterator trait, the Iter type now automatically gets all the iterator-associated function added to it. So not only can you call Iter::next to advance the iterator, but also make use of combinators such as filter:

#![allow(unused)]
fn main() {
let it = /* construct Iter */;

for value in it.filter(|v| v != true) {
    dbg!(value);
}
}

Defining a trait

Defining a trait is currently a low-level module operation. It's done by implementing a handler which will be called to populate the relevant methods when the trait is implement. Such as this snippet for the Iterator trait:

#![allow(unused)]
fn main() {
let mut m = Module::with_crate("std", ["iter"]);

let mut t = m.define_trait(["Iterator"])?;

t.handler(|cx| {
    let next = cx.find(&Protocol::NEXT)?;

    let size_hint = cx.find_or_define(&Protocol::SIZE_HINT, |_: Value| (0usize, None::<usize>))?;

    /* more methods */
    Ok(())
})?;
}

Calling find requires that NEXT is implemented. We can also see that the implementation for SIZE_HINT will fall back to a default implementation if it's not implemented. The appropriate protocol is also populated if it's missing. All the relevant associated functions are also provided, such as value.next() and value.size_hint().

Rune types

Types in Rune are identified uniquely by their item. An item path is a scope-separated identifier, like std::f64. This particular item identifies a type.

These items can be used to perform basic type checking using the is and is not operations, like this:

assert!(() is Tuple, "tuples should be tuples");
assert!((1, 2) is Tuple, "tuples should be tuples");
assert!(true is bool, "bools should be bools");
assert!('a' is char, "chars should be chars");
assert!(b'a' is u64, "bytes should be unsigned integers");
assert!(42 is i64, "integers should be integers");
assert!(42.1 is f64, "floats should be floats");
assert!("hello" is String, "strings should be strings");
assert!("x" is not char, "strings are not chars");
assert!(#{"hello": "world"} is Object, "objects should be objects");
assert!(["hello", "world"] is Vec, "vectors should be vectors");

$> cargo run -- run scripts/book/types/types.rn

Conversely, the type check would fail if you're providing a value which is not of that type.

assert!(["hello", "world"] is String, "vectors should be strings");

$> cargo run -- run scripts/book/types/bad_type_check.rn
== ! (panicked `assertion failed: vectors should be strings` (at 12)) (133.3µs)
error: virtual machine error
  ┌─ scripts/book/types/bad_type_check.rn:2:5
  │
2 │     assert!(["hello", "world"] is String, "vectors should be strings");
  │     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ panicked `assertion failed: vectors should be strings`

This gives us insight at runtime which type is which, and allows Rune scripts to make decisions depending on what type a value has.

fn dynamic_type(n) {
    if n is String {
        "n is a String"
    } else if n is Vec {
        "n is a vector"
    } else {
        "n is unknown"
    }
}

println!("{}", dynamic_type("Hello"));
println!("{}", dynamic_type([1, 2, 3, 4]));
println!("{}", dynamic_type(42));

$> cargo run -- run scripts/book/types/type_check.rn
n is a String
n is a vector
n is unknown

A tighter way to accomplish this would be by using pattern matching, a mechanism especially suited for many conditional branches. Especially when the branches are different types or variants in an enum.

fn dynamic_type(n) {
    match n {
        n if n is String => "n is a String",
        n if n is Vec => "n is a vector",
        _ => "n is unknown",
    }
}

println!("{}", dynamic_type("Hello"));
println!("{}", dynamic_type([1, 2, 3, 4]));
println!("{}", dynamic_type(42));

$> cargo run -- run scripts/book/types/type_check_patterns.rn
n is a String
n is a vector
n is unknown

Primitive and reference types

Primitives are values stored immediately on the stack. In Rust terminology, these types are Copy, so reassigning them to different values will create distinct copies of the underlying value.

The primitives available in Rune are:

• The unit ().
• Booleans, true and false.
• Bytes, like b'\xff'.
• Characters, like '今'. Which are 4 byte wide characters.
• Integers, like 42. Which are 64-bit signed integers.
• Floats, like 3.1418. Which are 64-bit floating point numbers.
• Static strings, like "Hello World".
• Type hashes.

You can see that these bytes are Copy when assigning them to a different variable, because a separate copy of the value will be used automatically.

let a = 1;
let b = a;
a = 2;
println!("{a}");
println!("{b}");

$> cargo run -- run scripts/book/primitives/copy.rn
2
1

Other types like strings are stored by reference. Assigning them to a different variable will only copy their reference, but they still point to the same underlying data.

let a = String::from("Hello");
let b = a;
a.push_str(" World");
println!("{a}");
println!("{b}");

$> cargo run -- run scripts/book/primitives/primitives.rn
Hello World
Hello World

Vectors

A vector is a native data structure of Rune which is a dynamic list of values. A vector isn't typed, and can store any Rune values.

pub fn main() {
    let values = ["Hello", 42];

    println!("{}", values[0]);
    println!("{}", values.1); // items in vectors can be accessed like tuple fields.

    for v in values {
        println!("{}", v);
    }
}

$> cargo run -- run scripts/book/vectors/vectors.rn
Hello
42
Hello
42

As you can see, you can iterate over a vector because it implements the iterator protocol. It is also possible to create and use an iterator manually using Vec::iter, giving you more control over it.

pub fn main() {
    let values = ["Hello", 42];

    for v in values.iter().rev() {
        println!("{}", v);
    }
}

$> cargo run -- run scripts/book/vectors/vectors_rev.rn
42
Hello

Using vectors from Rust

Vectors are represented externally as the standard Vec.

use rune::termcolor::{ColorChoice, StandardStream};
use rune::{Diagnostics, Vm};

use std::sync::Arc;

fn main() -> rune::support::Result<()> {
    let context = rune_modules::default_context()?;
    let runtime = Arc::new(context.runtime()?);

    let mut sources = rune::sources! {
        entry => {
            pub fn calc(input) {
                let output = 0;

                for value in input {
                    output += value;
                }

                [output]
            }
        }
    };

    let mut diagnostics = Diagnostics::new();

    let result = rune::prepare(&mut sources)
        .with_context(&context)
        .with_diagnostics(&mut diagnostics)
        .build();

    if !diagnostics.is_empty() {
        let mut writer = StandardStream::stderr(ColorChoice::Always);
        diagnostics.emit(&mut writer, &sources)?;
    }

    let unit = result?;
    let mut vm = Vm::new(runtime, Arc::new(unit));

    let input = vec![1, 2, 3, 4];
    let output = vm.call(["calc"], (input,))?;
    let output: Vec<i64> = rune::from_value(output)?;

    println!("{:?}", output);
    Ok(())
}

$> cargo run --example vector
[10]

If you have a vector which has values of non-uniform types, you can use VecTuple to deal with them.

use rune::runtime::VecTuple;
use rune::termcolor::{ColorChoice, StandardStream};
use rune::{Diagnostics, Vm};

use std::sync::Arc;

fn main() -> rune::support::Result<()> {
    let context = rune_modules::default_context()?;
    let runtime = Arc::new(context.runtime()?);

    let mut sources = rune::sources! {
        entry => {
            pub fn calc(input) {
                let a = input[0] + 1;
                let b = format!("{} World", input[1]);
                [a, b]
            }
        }
    };

    let mut diagnostics = Diagnostics::new();

    let result = rune::prepare(&mut sources)
        .with_context(&context)
        .with_diagnostics(&mut diagnostics)
        .build();

    if !diagnostics.is_empty() {
        let mut writer = StandardStream::stderr(ColorChoice::Always);
        diagnostics.emit(&mut writer, &sources)?;
    }

    let unit = result?;
    let mut vm = Vm::new(runtime, Arc::new(unit));

    let input: VecTuple<(i64, String)> = VecTuple::new((1, String::from("Hello")));
    let output = vm.call(["calc"], (input,))?;
    let VecTuple((a, b)): VecTuple<(u32, String)> = rune::from_value(output)?;

    println!("({:?}, {:?})", a, b);
    Ok(())
}

$> cargo run --example vec_tuple
(2, "Hello World")

Objects

Objects are anonymous maps, which support defining and using arbitrary string keys.

let values = #{};
values["first"] = "bar";
values["second"] = 42;

dbg!(values["first"]);
dbg!(values.second); // items be accessed like struct fields.

if let Some(key) = values.get("not a key") {
    dbg!(key);
} else {
    println!("key did not exist");
}

for entry in values {
    dbg!(entry);
}

$> cargo run -- run scripts/book/objects/objects.rn
"bar"
42
key did not exist
("second", 42)
("first", "bar")

These are useful because they allow their data to be specified dynamically, which is exactly the same use case as storing unknown JSON.

One of the biggest motivations for Rune to have anonymous objects is so that we can natively handle data with unknown structure.

async fn get_commits(repo, limit) {
    let limit = limit.unwrap_or(10);

    let client = http::Client::new();
    let request = client.get(`https://api.github.com/repos/${repo}/commits`);
    let response = request.header("User-Agent", "Rune").send().await?;
    let text = response.text().await?;
    let json = json::from_string(text)?;

    let commits = json.iter().take(limit).map(|e| e.sha).collect::<Vec>();
    Ok(commits)
}

for commit in get_commits("rune-rs/rune", Some(5)).await? {
    println!("{commit}");
}

$> cargo run -- run scripts/book/objects/json.rn
9c4bdaf194410d8b2f5d7f9f52eb3e64709d3414
06419f2580e7a18838f483321055fc06c0d75c4c
cba225dad143779a0a9543cfb05cde9710083af5
15133745237c014ff8bae53d8ff8f3c137c732c7
39ac97ab4ebe26118e807eb91c7656ab95b1fcac
3f6310eeeaca22d0373cc11d8b34d346bd12a364

Using objects from Rust

Objects are represented externally as the Object type alias. The keys are always strings, but its value must be specified as the sole type parameter. Note that the dynamic Value can be used if the type is unknown.

use rune::alloc;
use rune::runtime::Object;
use rune::termcolor::{ColorChoice, StandardStream};
use rune::{Diagnostics, Vm};

use std::sync::Arc;

fn main() -> rune::support::Result<()> {
    let context = rune_modules::default_context()?;
    let runtime = Arc::new(context.runtime()?);

    let mut sources = rune::sources! {
        entry => {
            pub fn calc(input) {
                dbg(input["key"]);
                input["key"] = "World";
                input
            }
        }
    };

    let mut diagnostics = Diagnostics::new();

    let result = rune::prepare(&mut sources)
        .with_context(&context)
        .with_diagnostics(&mut diagnostics)
        .build();

    if !diagnostics.is_empty() {
        let mut writer = StandardStream::stderr(ColorChoice::Always);
        diagnostics.emit(&mut writer, &sources)?;
    }

    let unit = result?;

    let mut vm = Vm::new(runtime, Arc::new(unit));

    let mut object = Object::new();
    object.insert(alloc::String::try_from("key")?, rune::to_value(42i64)?)?;

    let output = vm.call(["calc"], (object,))?;
    let output: Object = rune::from_value(output)?;

    println!("{:?}", output.get("key"));
    Ok(())
}

$> cargo run --example object
42
Some("World")

Tuples

Tuples in Rune are fixed-size sequences of values. Similarly to a vector, tuples can contain any sequence of values. But there's no way to change the size of a tuple.

let values = ("Now", "You", "See", "Me");
dbg!(values);

values.2 = "Don't";
values.3 = "!";
dbg!(values);

$> cargo run -- run scripts/book/tuples/tuple_masquerade.rn
("Now", "You", "See", "Me")
("Now", "You", "Don\'t", "!")

The following is a simple example of a function returning a tuple:

fn foo() {
    (1, "test")
}

dbg!(foo());

$> cargo run -- run scripts/book/tuples/basic_tuples.rn
(1, "test")

Tuples can also be pattern matched:

match ("test", 1) {
    ("test", n) => {
        dbg!("the first part was a number:", n);
    }
    _ => {
        dbg!("matched something we did not understand");
    }
}

$> cargo run -- run scripts/book/tuples/tuple_patterns.rn
"the first part was a number:"
1

Using tuples from Rust

Tuples are represented externally as primitive tuple types.

use rune::termcolor::{ColorChoice, StandardStream};
use rune::{Diagnostics, Vm};

use std::sync::Arc;

fn main() -> rune::support::Result<()> {
    let context = rune_modules::default_context()?;
    let runtime = Arc::new(context.runtime()?);

    let mut sources = rune::sources! {
        entry => {
            pub fn calc(input) {
                (input.0 + 1, input.1 + 2)
            }
        }
    };

    let mut diagnostics = Diagnostics::new();

    let result = rune::prepare(&mut sources)
        .with_context(&context)
        .with_diagnostics(&mut diagnostics)
        .build();

    if !diagnostics.is_empty() {
        let mut writer = StandardStream::stderr(ColorChoice::Always);
        diagnostics.emit(&mut writer, &sources)?;
    }

    let unit = result?;

    let mut vm = Vm::new(runtime, Arc::new(unit));
    let output = vm.call(["calc"], ((1u32, 2u32),))?;
    let output: (i32, i32) = rune::from_value(output)?;

    println!("{:?}", output);
    Ok(())
}

$> cargo run --example tuple
(2, 4)

Dynamic types

Dynamic types are types which can be defined and used solely within a Rune script. They provide the ability to structure data and associate functions with it.

The following is a quick example of a struct:

struct Person {
    name,
}

impl Person {
    fn greet(self) {
        println!("Greetings from {}, and good luck with this section!", self.name);
    }
}

let person = Person { name: "John-John Tedro" };

person.greet();

$> cargo run -- run scripts/book/dynamic_types/greeting.rn
Greetings from John-John Tedro, and good luck with this section!

Structs

Structs are like objects, except that they have a predefined structure with a set of keys that are known at compile time and guaranteed to be defined.

Structs can also, like most types, have an impl block associated with them which creates instance functions that you can call on an instance of that struct.

struct User {
    username,
    active,
}

impl User {
    fn set_active(self, active) {
        self.active = active;
    }

    fn describe(self) {
        if self.active {
            println!("{} is active.", self.username);
        } else {
            println!("{} is inactive.", self.username);
        }
    }
}

let user = User { username: "setbac", active: false };

user.describe();
user.set_active(true);
user.describe();

$> cargo run -- run scripts/book/structs/user_database.rn
setbac is inactive
setbac is active

Structs can also be pattern matched, like most types.

But since the fields of a struct are known at compile time, the compiler can ensure that you're only using fields which are defined.

struct User {
    username,
    active,
}

impl User {
    fn describe(self) {
        match self {
            User { username: "setbac", .. } => {
                println!("Yep, it's setbac.");
            }
            User { username, .. } => {
                println!("Other user: {username}.");
            }
        }
    }
}

let user = User { username: "setbac", active: false };

user.describe();
user.username = "newt";
user.describe();

$> cargo run -- run scripts/book/structs/struct_matching.rn
Yep, it's setbac.
Other user: newt.

Enums

Rune has support for enumerations. These allow you to define a type with zero or more variants, where each variant can hold a distinct set of data.

In a dynamic programming language enums might not seem quite as useful, but it's important for Rune to support them to have a level of feature parity with Rust.

Even so, in this section we'll explore some cases where enums are useful.

The Option enum

Rune has native support for Option, the same enum available in Rust that allows you to represent data that can either be present with Option::Some, or absent with Option::None.

fn count_numbers(limit) {
    let limit = limit.unwrap_or(10);

    for n in 0..limit {
        println!("Count: {}", n);
    }
}

println!("First count!");
count_numbers(None);

println!("Second count!");
count_numbers(Some(2));

$> cargo run -- run scripts/book/enums/count_numbers.rn
First count!
Count: 0
Count: 1
Count: 2
Count: 3
Count: 4
Count: 5
Count: 6
Count: 7
Count: 8
Count: 9
Second count!
Count: 0
Count: 1

Using an Option allows us to easily model the scenario where we have an optional function parameter, with a default fallback value.

In the next section we'll be looking into a control flow construct which gives Option superpowers.

The try operator.

External types

When a type is declared outside of Rune it is said to be external. External types are declared when setting up native modules. And Rune allows various levels of integration with the language.

On the simplest level an external type is entirely opaque. Rune knows nothing about it except that it is a value bound to a variable.

Below is the most simple example of an external type. It's implemented by deriving Any which can do a lot of heavy lifting for us.

use rune::runtime::Vm;
use rune::termcolor::{ColorChoice, StandardStream};
use rune::{Any, ContextError, Diagnostics, Module};

use std::sync::Arc;

#[derive(Debug, Any)]
struct External {
    value: u32,
}

pub fn module() -> Result<Module, ContextError> {
    let mut module = Module::new();
    module.ty::<External>()?;
    Ok(module)
}

fn main() -> rune::support::Result<()> {
    let m = module()?;

    let mut context = rune_modules::default_context()?;
    context.install(m)?;
    let runtime = Arc::new(context.runtime()?);

    let mut sources = rune::sources! {
        entry => {
            pub fn main(external) {
                external
            }
        }
    };

    let mut diagnostics = Diagnostics::new();

    let result = rune::prepare(&mut sources)
        .with_context(&context)
        .with_diagnostics(&mut diagnostics)
        .build();

    if !diagnostics.is_empty() {
        let mut writer = StandardStream::stderr(ColorChoice::Always);
        diagnostics.emit(&mut writer, &sources)?;
    }

    let unit = result?;

    let mut vm = Vm::new(runtime, Arc::new(unit));

    let output = vm.call(["main"], (External { value: 42 },))?;
    let output: External = rune::from_value(output)?;
    println!("{:?}", output);
    assert_eq!(output.value, 42);
    Ok(())
}

This type isn't particularly useful. Attempting to access a field on external would simply error. We have to instruct Rune how the field is accessed.

Luckily Any allows us to easily do that by marking the fields we want to make accessible to Rune with #[rune(get)].

#[derive(Debug, Any)]
struct External {
    #[rune(get)]
    value: u32,
}

With our newfound power we can now read external.value.

pub fn main(external) {
    println!("{}", external.value);
}

Setting the value is similarly simple. We simply mark the field with #[rune(set)].

#[derive(Debug, Any)]
struct External {
    #[rune(get, set)]
    value: u32,
}

And now we can both read and write to external.value.

pub fn main(external) {
    external.value = external.value + 1;
}

Note: See the section about Field Functions for a complete reference of the available attributes.

External enums

Enums have a few more tricks that we need to cover. We want to be able to pattern match and construct external enums.

There are three kinds of variants in an enum:

• Unit variants which have no fields. E.g. External::Unit.
• Tuple variants which have numerical fields. E.g. External::Tuple(1, 2, 3).
• Struct variants which have named fields. E.g. External::Struct { a: 1, b: 2, c: 3 }.

Pattern matching is supported out of the box. The only thing to take note of is that pattern matching will only see fields that are annotated with #[rune(get)].

So the following type:

enum External {
    First(#[rune(get)] u32, u32),
    Second(#[rune(get)] u32),
}

Could be pattern matched like this in Rune:

pub fn main(external) {
    match external {
        External::First(a) => a,
        External::Second(b) => b,
    }
}

Take note on how External::First only "sees" the field marked with #[rune(get)].

Let's add a struct variant and see what we can do then:

enum External {
    First(#[rune(get)] u32, u32),
    Second(#[rune(get)] u32),
    Third {
        a: u32,
        b: u32,
        #[rune(get)]
        c: u32,
    },
}

And let's add Third to our example:

pub fn main(external) {
    match external {
        External::First(a) => a,
        External::Second(b) => b,
        External::Third { c } => b,
    }
}

Constructing enum variants

Unit and tuple variants can be annotated with #[rune(constructor)] which is necessary to allow for building enums in Rune. But in order for the constructor to work, all fields must be annotated with #[rune(get)].

enum External {
    #[rune(constructor)]
    First(#[rune(get)] u32, #[rune(get)] u32),
    #[rune(constructor)]
    Second(#[rune(get)] u32),
    Third {
        a: u32,
        b: u32,
        #[rune(get)]
        c: u32,
    },
}

pub fn main() {
    External::First(1, 2)
}

But why do we have the #[rune(get)] requirement? Consider what would happen otherwise. How would we construct an instance of External::First without being able to specify what the values of all fields are? The answer is that all fields must be visible. Alternatively we can declare another constructor as an associated function. The same way we'd do it in Rust.

Try operator

The try operator (?) is a control flow operator which causes a function to return early in case the value being tried over has a certain value.

For Option, this causes the function to return if it has the Option::None variant.

fn checked_div_mod(a, b) {
    let div = a.checked_div(b)?;
    Some((div, a % b))
}

if let Some((div, m)) = checked_div_mod(5, 2) {
    println!("Result: {}, {}", div, m);
}

if let Some((div, m)) = checked_div_mod(5, 0) {
    println!("Result: {}, {}", div, m);
}

$> cargo run -- run scripts/book/try_operator/basic_try.rn
Result: 2, 1

Generators

Generators are a convenient method for constructing functions which are capable of suspending themselves and their state.

The simplest use case for generators is to create a kind of iterator, whose state is stored in the generator function.

With this, we can create a fairly efficient generator to build fibonacci numbers.

fn fib() {
    let a = 0;
    let b = 1;

    loop {
        yield a;
        let c = a + b;
        a = b;
        b = c;
    }
}

let g = fib();

while let Some(n) = g.next() {
    println!("{n}");

    if n > 100 {
        break;
    }
}

$> cargo run -- run scripts/book/generators/fib_generator.rn
0
1
1
2
3
5
8
13
21
34
55
89
144

Advanced generators with GeneratorState

Generators internally are a bit more complex than that. The next function simply slates over some of that complexity to make simple things easier to do.

The first thing to know is that yield itself can actually produce a value, allowing the calling procedure to send values to the generator.

fn printer() {
    let collected = [];

    for _ in 0..2 {
        let out = yield;
        println!("{:?}", out);
        collected.push(out);
    }

    assert_eq!(collected, ["John", (1, 2, 3)]);
}

let printer = printer();
printer.resume(());
printer.resume("John");
printer.resume((1, 2, 3));

$> cargo run -- run scripts/book/generators/send_values.rn
"John"
(1, 2, 3)

But wait, what happened to the first value we sent, 1?

Well, generators don't run immediately, they need to be "warmed up" by calling resume once. At that point it runs the block prior to the first yield, we can see this by instrumenting our code a little.

fn printer() {
    loop {
        println!("waiting for value...");
        let out = yield;
        println!("{out:?}");
    }
}

let printer = printer();

println!("firing off the printer...");
printer.resume(());
println!("ready to go!");

printer.resume("John");
printer.resume((1, 2, 3));

$> cargo run -- run scripts/book/generators/bootup.rn
firing off the printer...
waiting for value...
ready to go!
"John"
waiting for value...
(1, 2, 3)
waiting for value...

Ok, so we understand how to send values into a generator. But how do we receive them?

This adds a bit of complexity, since we need to pull out GeneratorState. This enum has two variants: Yielded and Complete, and represents all the possible states a generator can suspend itself into.

fn print_once() {
    let out = yield 1;
    println!("{:?}", out);
    2
}

let printer = print_once();
dbg!(printer.resume(()));
dbg!(printer.resume("John"));

$> cargo run -- run scripts/book/generators/states.rn
Yielded(1)
"John"
Complete(2)

After the first call to resume, we see that the generator produced Yielded(1). This corresponds to the yield 1 statement in the generator.

The second value we get is Complete(2). This corresponds to the return value of the generator.

Trying to resume the generator after this will cause the virtual machine to error.

fn print_once() {
    yield 1
}

let printer = print_once();
dbg!(printer);
dbg!(printer.resume(()));
dbg!(printer.resume("John"));
dbg!(printer);
dbg!(printer.resume(()));

$> cargo run -- run scripts/book/generators/error.rn
Generator { completed: false }
Yielded(1)
Complete("John")
Generator { completed: true }
error: virtual machine error
   ┌─ scripts/book/generators/error.rn:11:9
   │
11 │     dbg!(printer.resume(()));
   │          ^^^^^^^^^^^^^^^^^^ cannot resume a generator that has completed

Closures

We've gone over functions before, and while incredibly useful there's a few more tricks worth mentioning.

We'll also be talking about closures, an anonymous function with the ability to close over its environment, allowing the function to use and manipulate things from its environment.

Function pointers

Every function can be converted into a function pointer simply by referencing its name without calling it.

This allows for some really neat tricks, like passing in a function which represents the operation you want another function to use.

fn do_thing(op) {
    op(1, 2)
}

fn add(a, b) {
    a + b
}

fn sub(a, b) {
    a - b
}

println!("Result: {}", do_thing(add));
println!("Result: {}", do_thing(sub));

$> cargo run -- run scripts/book/closures/function_pointers.rn
Result: 3
Result: -1

Closures

Closures are anonymous functions which closes over their environment. This means that they capture any variables used inside of the closure, allowing them to be used when the function is being called.

fn work(op) {
    op(1, 2)
}

let n = 1;
println!("Result: {}", work(|a, b| n + a + b));
println!("Result: {}", work(|a, b| n + a * b));

$> cargo run -- run scripts/book/closures/basic_closure.rn
Result: 4
Result: 3

Hint: Closures which do not capture their environment are identical in representation to a function.

Functions outside of the Vm

Now things get really interesting. Runestick, the virtual machine driving Rune, has support for passing function pointers out of the virtual machine using the Function type.

This allows you to write code that takes a function constructed in Rune, and use it for something else.

use rune::runtime::Function;
use rune::termcolor::{ColorChoice, StandardStream};
use rune::{Diagnostics, Vm};

use std::sync::Arc;

fn main() -> rune::support::Result<()> {
    let context = rune_modules::default_context()?;
    let runtime = Arc::new(context.runtime()?);

    let mut sources = rune::sources! {
        entry => {
            fn foo(a, b) {
                a + b
            }

            pub fn main() {
                foo
            }
        }
    };

    let mut diagnostics = Diagnostics::new();

    let result = rune::prepare(&mut sources)
        .with_context(&context)
        .with_diagnostics(&mut diagnostics)
        .build();

    if !diagnostics.is_empty() {
        let mut writer = StandardStream::stderr(ColorChoice::Always);
        diagnostics.emit(&mut writer, &sources)?;
    }

    let unit = result?;

    let mut vm = Vm::new(runtime, Arc::new(unit));
    let output = vm.call(["main"], ())?;
    let output: Function = rune::from_value(output)?;

    println!("{}", output.call::<i64>((1, 3)).into_result()?);
    println!("{}", output.call::<i64>((2, 6)).into_result()?);
    Ok(())
}

$> cargo run --example rune_function
4
8

Note that these functions by necessity have to capture their entire context and can take up quite a bit of space if you keep them around while cycling many contexts or units.

Values used in a closure can also be moved into it using the move keyword, guaranteeing that no one else can use it afterwards. An attempt to do so will cause a compile error.

fn work(op) {
    op(1, 2)
}

pub fn main() {
    let n = 1;
    println!("Result: {}", work(move |a, b| n + a + b));
    assert!(!is_readable(n));
}

$> cargo run -- run scripts/book/closures/closure_move.rn.fail
error: compile error
  ┌─ scripts/book/closures/closure_move.rn.fail:7:33
  │
7 │     println!("Result: {}", work(move |a, b| n + a + b));
  │                                 --------------------- moved here
8 │     assert!(!is_readable(n));
  │                          ^ variable moved

Moving indiscriminately applies to types which in principle could be copied (like integers). We simply don't have the necessary type information available right now to make that decision. If you know that the value can be copied and you want to do so: assign it to a separate variable.

Asynchronous programming

Rune has first class support for Rust-like asynchronous programming. In this section we'll be briefly covering what asynchronous programming is, and how it applies to Rune as a dynamic programming language.

What is it?

Asynchronous code allows us to run multiple tasks concurrently, and work with the result of those tasks.

A typical example would be if we want to perform multiple HTTP requests at once:

let a = http::get("https://google.com");
let b = http::get("https://amazon.com");

loop {
    let res = select {
        res = a => res?,
        res = b => res?,
    };

    match res {
        () => break,
        result => {
            println!("{}", result.status());
        }
    }
}

$> cargo run -- run scripts/book/async/async_http.rn
200 OK
200 OK

In the above code we send two requests concurrently. They are both processed at the same time and we collect the result.

select blocks

A fundamental construct of async programming in Rune is the select block. It enables us to wait on a set of futures at the same time.

A simple example of this is if we were to implement a simple request with a timeout:

struct Timeout;

async fn request(timeout) {
    let request = http::get(`http://httpstat.us/200?sleep=${timeout}`);
    let timeout = time::sleep(time::Duration::from_secs(2));

    let result = select {
        _ = timeout => Err(Timeout),
        res = request => res,
    }?;

    println!("{}", result.status());
    Ok(())
}

if let Err(Timeout) = request(1000).await {
    println!("Request timed out!");
}

if let Err(Timeout) = request(4000).await {
    println!("Request timed out!");
}

$> cargo run -- run scripts/book/async/async_http_timeout.rn
200 OK
Request timed out!

But wait, this is taking three seconds. We're not running the requests concurrently any longer!

Well, while the request and the timeout is run concurrently, the request function is run one at-a-time.

To fix this we need two new things: async functions and .await.

async functions

async functions are just like regular functions, except that when called they produce a Future.

In order to get the result of this Future it must be .awaited. And .await is only permitted inside of async functions and closures.

use std::future;

struct Timeout;

async fn request(timeout) {
    let request = http::get(`http://httpstat.us/200?sleep=${timeout}`);
    let timeout = time::sleep(time::Duration::from_secs(2));

    let result = select {
        _ = timeout => Err(Timeout),
        res = request => res,
    }?;

    Ok(result)
}

for result in future::join([request(1000), request(4000)]).await {
    match result {
        Ok(result) => println!("Result: {}", result.status()),
        Err(Timeout) => println!("Request timed out!"),
    }
}

$> cargo run -- run scripts/book/async/async_http_concurrent.rn
Result: 200 OK
Request timed out!

async closures

Closures can be prefixed with the async keyword, meaning calling them will produce a future.

fn do_request(url) {
    async || {
        Ok(http::get(url).await?.status())
    }
}

let future = do_request("https://google.com");
let status = future().await?;
println!("Status: {status}");

$> cargo run -- run scripts/book/async/async_closure.rn
Status: 200 OK

async blocks

Blocks can be marked with async to produce on-the-fly futures. These blocks can capture variables the same way as closures do, but take no arguments.

fn do_request(url) {
    async {
        Ok(http::get(url).await?.status())
    }
}

let future = do_request("https://google.com");
let status = future.await?;
println!("Status: {status}");

$> cargo run -- run scripts/book/async/async_blocks.rn
Status: 200 OK

Streams

Streams are the asynchronous version of Generators.

They have almost identical next and resume functions, but each must be used with .await, and we are now allowed to use asynchronous functions inside of the generator.

async fn send_requests(list) {
    let client = http::Client::new();

    let do_request = async |url| {
        Ok(client.get(url).send().await?.status())
    };

    for url in list {
        yield do_request(url).await;
    }
}

let requests = send_requests(["https://google.com", "https://amazon.com"]);

while let Some(status) = requests.next().await.transpose()? {
    println!("{}", status);
}

$> cargo run -- run scripts/book/streams/basic_stream.rn
200 OK
200 OK

Multithreading

Rune is thread safe, but the Vm does not implement Sync so cannot directly be shared across threads. This section details instead how you are intended to use Rune in a multithreaded environment.

Compiling a Unit and a RuntimeContext are expensive operations compared to the cost of calling a function. So you should try to do this as little as possible. It is appropriate to recompile a script when the source of the script changes. See the Hot reloading section for more information on this.

Once you have a Unit and a RuntimeContext they are thread safe and can be used by multiple threads simultaneously through Arc<Unit> and Arc<RuntimeContext>. Constructing a Vm with these through Vm::new is a very cheap operation.

#![allow(unused)]
fn main() {
let unit: Arc<Unit> = /* todo */;
let context: Arc<RuntimeContext> = /* todo */;

std::thread::spawn(move || {
    let mut vm = Vm::new(unit, context);
    let value = vm.call(["function"], (42,))?;
    Ok(())
});
}

Virtual machines do allocate memory. To avoide this overhead you'd have to employ more advanced techniques, such as storing virtual machines in a pool or thread locals. Once a machine has been acquired the Unit and RuntimeContext associated with it can be swapped out to the ones you need using Vm::unit_mut and Vm::context_mut respectively.

Using Vm::send_execute is a way to assert that a given execution is thread safe. And allows you to use Rune in asynchronous multithreaded environments, such as Tokio. This is achieved by ensuring that all captured arguments are ConstValue's, which in contrast to Value's are guaranteed to be thread-safe:

use rune::alloc::prelude::*;
use rune::termcolor::{ColorChoice, StandardStream};
use rune::{Diagnostics, Vm};

use std::sync::Arc;

#[tokio::main]
async fn main() -> rune::support::Result<()> {
    let context = rune_modules::default_context()?;
    let runtime = Arc::new(context.runtime()?);

    let mut sources = rune::sources! {
        entry => {
            async fn main(timeout) {
                time::sleep(time::Duration::from_secs(timeout)).await
            }
        }
    };

    let mut diagnostics = Diagnostics::new();

    let result = rune::prepare(&mut sources)
        .with_context(&context)
        .with_diagnostics(&mut diagnostics)
        .build();

    if !diagnostics.is_empty() {
        let mut writer = StandardStream::stderr(ColorChoice::Always);
        diagnostics.emit(&mut writer, &sources)?;
    }

    let unit = result?;

    let vm = Vm::new(runtime, Arc::new(unit));

    let execution = vm.try_clone()?.send_execute(["main"], (5u32,))?;
    let t1 = tokio::spawn(async move {
        execution.async_complete().await.unwrap();
        println!("timer ticked");
    });

    let execution = vm.try_clone()?.send_execute(["main"], (2u32,))?;
    let t2 = tokio::spawn(async move {
        execution.async_complete().await.unwrap();
        println!("timer ticked");
    });

    tokio::try_join!(t1, t2).unwrap();
    Ok(())
}

Finally Function::into_sync exists to coerce a function into a SyncFunction, which is a thread-safe variant of a regular Function. This is a fallible operation since all values which are captured in the function-type in case its a closure has to be coerced to ConstValue. If this is not the case, the conversion will fail.

Hot reloading

Compiling a Unit and a [RuntimeContext] are expensive operations compared to the cost of calling a function. So you should try to do this as little as possible. It is appropriate to recompile a script when the source of the script changes. This section provides you with details for how this can be done when loading scripts from the filesystem.

A typical way to accomplish this is to watch a scripts directory using the notify crate. This allow the application to generate events whenever changes to the directory are detected. See the hot_reloading example and in particular the PathReloader type.

#[path = "hot_reloading/path_reloader.rs"]
mod path_reloader;

use std::path::PathBuf;
use std::pin::pin;
use std::sync::Arc;

use anyhow::{Context as _, Result};
use rune::{Context, Vm};

#[tokio::main(flavor = "current_thread")]
async fn main() -> Result<()> {
    let root =
        PathBuf::from(std::env::var("CARGO_MANIFEST_DIR").context("missing CARGO_MANIFEST_DIR")?);

    let context = Context::with_default_modules()?;

    let mut exit = pin!(tokio::signal::ctrl_c());
    let mut reloader = pin!(path_reloader::PathReloader::new(
        root.join("scripts"),
        &context
    )?);

    let context = Arc::new(context.runtime()?);

    let mut events = Vec::new();

    loop {
        tokio::select! {
            _ = exit.as_mut() => {
                break;
            }
            result = reloader.as_mut().watch(&mut events) => {
                result?;
            }
        }

        for event in events.drain(..) {
            let mut vm = Vm::new(context.clone(), event.unit);

            match event.kind {
                path_reloader::EventKind::Added => {
                    if let Err(error) = vm.call(["hello"], ()) {
                        println!("Error: {}", error);
                    }
                }
                path_reloader::EventKind::Removed => {
                    if let Err(error) = vm.call(["goodbye"], ()) {
                        println!("Error: {}", error);
                    }
                }
            }
        }
    }

    Ok(())
}

Macros

Rune has support for macros. These are functions which expand into code, and can be used by library writers to "extend the compiler".

For now, the following type of macros are support:

• Function-like macros expanding to items (functions, type declarations, ..).
• Function-like macros expanding to expression (statements, blocks, async blocks, ..).
• Attribute macros expanding around a function.

Macros can currently only be defined natively. This is to get around the rather tricky issue that the code of a macro has to be runnable during compilation. Native modules have an edge here, because they have to be defined at a time when they are definitely available to the compiler.

Don't worry though, we will be playing around with macro fn as well, but at a later stage 😉 (See issue #27).

Native modules also means we can re-use all the existing compiler infrastructure for Rune as a library for macro authors. Which is really nice!

Writing a native macro

The following is the definition of the stringy_math! macro. Which is a macro that can be invoked on expressions.

This relies heavily on a Rune-specific quote! macro. Which is inspired by its famed counterpart in the Rust world. A major difference with Rune quote! is that we need to pass in the MacroContext when invoking it. This is a detail which will be covered in one of the advanced sections.

use crate as rune;
use crate::ast;
use crate::compile;
use crate::macros::{quote, MacroContext, TokenStream};
use crate::parse::Parser;

/// Implementation of the `stringy_math!` macro.
#[rune::macro_]
pub fn stringy_math(
    cx: &mut MacroContext<'_, '_, '_>,
    stream: &TokenStream,
) -> compile::Result<TokenStream> {
    let mut parser = Parser::from_token_stream(stream, cx.input_span());

    let mut output = quote!(0);

    while !parser.is_eof()? {
        let op = parser.parse::<ast::Ident>()?;
        let arg = parser.parse::<ast::Expr>()?;

        output = match cx.resolve(op)? {
            "add" => quote!((#output) + #arg),
            "sub" => quote!((#output) - #arg),
            "div" => quote!((#output) / #arg),
            "mul" => quote!((#output) * #arg),
            _ => return Err(compile::Error::msg(op, "unsupported operation")),
        }
    }

    parser.eof()?;
    Ok(output.into_token_stream(cx)?)
}

A macro is added to a Module using the Module::macro_ function.

pub fn module() -> Result<rune::Module, rune::ContextError> {
    let mut module = rune::Module::new(["test", "macros"]);
    module.macro_meta(stringy_math)?;
    Ok(module)
}

With this module installed, we can now take stringy_math! for a spin.

use ::test::macros::stringy_math;

pub fn main() {
    stringy_math!(add 10 sub 2 div 3 mul 100)
}

Running this would return 200.

Advanced

This chapter is dedicated to the advanced topics of Rune. Here we will discuss the advanced and internal details of the language. This chapter is primarily targeted at people who want to understand Rune and Runestick in detail.

Drop order

Rune implements the following rules when determining when a place should be dropped.

Places are distinct from values, in that they refer to a place where a value is stored, not the value itself.

The distinction becomes apparent when we note that the same value can be referenced by multiple places:

let a = 42;
let b = a;

Above, b is a distinct place that refers to the same value as a.

There are two ways for a value to be dropped:

• All the places referencing go out of scope.
• The value is explicitly dropped with drop(value).

The second variant causes the value to be dropped. Using any of the places referencing that value after it has been dropped will cause an error.

Variables

A variable declaration like this:

let var = 42;

Defines a place called var.

Once variables like these go out of scope, their place is dropped. However, dropping a place doesn't necessarily mean the value is dropped. This only happens when that is the last place referencing that variable.

let object = {
    let var = 42;
    let object = #{ var };
};

// object can be used here and `var` is still live.

Temporaries

Temporaries are constructed when evaluating any non-trivial expression, such as this:

let var = [42, (), "hello"];

The drop order for temporaries is not strictly defined and can be extended. But never beyond the block in which they are defined.

Safety

Rune is implemented in Rust, but that doesn't automatically make the language safe (as Rust defines safety) since there are some uses of unsafe. In this section we'll be documenting the pieces of the implementation which are currently unsafe, rationalize, and document potential soundness holes.

Internal Any type

Rune uses an internal Any type.

Apart from the hash conflict documented above, the implementation should be sound. We have an internal Any type instead of relying on Box<dyn Any> to allow AnyObjVtable to be implementable by external types to support external types through a C ffi.

Shared<T> and UnsafeToRef / UnsafeToMut

A large chunk of the Shared<T> container is unsafe. This is a container which is behaviorally equivalent to Rc<RefCell<T>>.

We have this because it merges Rc and RefCell and provides the ability to have "owned borrows" and the ability to unsafely decompose these into a raw pointer and a raw guard, which is used in many implementations of UnsafeToRef or UnsafeToMut.

UnsafeToRef and UnsafeToMut are conversion traits which are strictly used internally to convert values into references. Its safety is documented in the trait.

Sandboxing

Rune is capable of enforcing the following types of limitations:

• Memory limiting, where you specify the maxium amount of memory that Rune may use either during compilation or execution.
• Instruction budgeting, where you can specify how many instructions the virtual machine is permitted to execute.

Instruction budgeting

Instruction budgeting is performed using the with function in the rune::budget module.

The with function is capable of wrapping functions and futures. When wrapping a future it ensures that the budget is suspended appropriately with the execution of the future.

Budgeting is only performed on a per-instruction basis in the virtual machine. What exactly constitutes an instruction might be a bit vague. But important to note is that without explicit co-operation from native functions the budget cannot be enforced. So care must be taken with the native functions that you provide to Rune to ensure that the limits you impose cannot be circumvented.

Memory limiting

Memory limiting is performed using the with function in the rune::alloc::limit module.

#![allow(unused)]
fn main() {
use rune::alloc::prelude::*;
use rune::alloc::limit;

let f = limit::with(1024, || {
    let mut vec = Vec::<u32>::try_with_capacity(256)?;

    for n in 0..256u32 {
        vec.try_push(n)?;
    }

    Ok::<_, rune_alloc::Error>(vec.into_iter().sum::<u32>())
});

let sum = f.call()?;
assert_eq!(sum, 32640);
}

In order for memory limiting to work as intended, you're may only use the collections provided in the rune::alloc module. These contain forks of popular collections such as std::collections and hashbrown.

The with function is capable of wrapping functions and [futures]. When wrapping a future it ensures that the limit is suspended appropriately with the execution of the future.

The stack

Runestick is a stack-based virtual machine. It has two primary places where things are stored. The stack and the heap. It has no registers.

Instructions in the virtual machine operate off the stack. Let's take a look at the add operation with --trace and --dump-stack.

1 + 3

$> cargo run -- run scripts/book/the_stack/add.rn --trace --dump-stack
fn main() (0xe7fc1d6083100dcd):
  0000 = integer 1
    0+0 = 1
  0001 = integer 3
    0+0 = 1
    0+1 = 3
  0002 = add
    0+0 = 4
  0003 = return
    *empty*
== 4 (7.7691ms)
# stack dump after halting
frame #0 (+0)
    *empty*

Let's examine the stack after each instruction.

  0000 = integer 1
    0+0 = 1

We evaluate the integer 1 instruction, which pushes an integer with the value 1 onto the stack.

  0001 = integer 3
    0+0 = 1
    0+1 = 3

We evaluate the integer 3 instruction, which pushes an integer with the value 3 onto the stack.

  0002 = add
    0+0 = 4

We evaluate the add instruction which pops two values from the stack and adds them together. Two integers in this instance would use built-in accelerated implementations which performs addition.

  0003 = return
== 4 (7.7691ms)

We return from the virtual machine. The last value of the stack will be popped as the return value.

# stack dump
frame #0 (+0)

This is the stack dump we see after the virtual machine has exited. It tells us that at call frame #0 (+0), the last and empty call frame at stack position +0 there is nothing on the stack.

Call frames

Call frames are a cheap isolation mechanism available in the virtual machine. They define a subslice in the stack, preventing the vm from accessing values that are outside of the slice.

They have the following rules:

• Instructions cannot access values outside of their current call frame.
• When we return from the call frame the subslice must be empty.

If any these two conditions aren't maintained, the virtual machine will error.

Call frames fill two purposes. The subslice provides a well-defined variable region. Stack-relative operations like copy 0 are always defined relative to the top of their call frame. Where copy 0 would mean "copy from offset 0 of the current stack frame".

They also provide a cheap security mechanism against miscompilations. This might be made optional in the future once Rune is more stable, but for now it's helpful to detect errors early and protect the user against bad instructions. But don't mistake it for perfect security. Like stack protection which is common in modern operating systems, the mechanism can be circumvented by malicious code.

To look close at the mechanism, let's trace the following program:

fn foo(a, b) {
    a + b
}

let a = 3;
foo(1, 2) + a

$> cargo run -- run scripts/book/the_stack/call_and_add.rn --trace --dump-stack
fn main() (0xe7fc1d6083100dcd):
  0005 = integer 3
    0+0 = 3
  0006 = integer 1
    0+0 = 3
    0+1 = 1
  0007 = integer 2
    0+0 = 3
    0+1 = 1
    0+2 = 2
  0008 = call 0xbfd58656ec9a8ebe, 2 // fn `foo`
=> frame 1 (1):
    1+0 = 1
    1+1 = 2
fn foo(arg, arg) (0xbfd58656ec9a8ebe):
  0000 = copy 0 // var `a`
    1+0 = 1
    1+1 = 2
    1+2 = 1
  0001 = copy 1 // var `b`
    1+0 = 1
    1+1 = 2
    1+2 = 1
    1+3 = 2
  0002 = add
    1+0 = 1
    1+1 = 2
    1+2 = 3
  0003 = clean 2
    1+0 = 3
  0004 = return
<= frame 0 (0):
    0+0 = 3
    0+1 = 3
  0009 = copy 0 // var `a`
    0+0 = 3
    0+1 = 3
    0+2 = 3
  0010 = add
    0+0 = 3
    0+1 = 6
  0011 = clean 1
    0+0 = 6
  0012 = return
    *empty*
== 6 (45.8613ms)
# full stack dump after halting
  frame #0 (+0)
    *empty*

We're not going to go through each instruction step-by-step like in the last section. Instead we will only examine the parts related to call frames.

We have a call 0xbfd58656ec9a8ebe, 2 instruction, which tells the virtual machine to jump to the function corresponding to the type hash 0xbfd58656ec9a8ebe, and isolate the top two values on the stack in the next call frame.

We can see that the first argument a is in the lowest position, and the second argument b is on the highest position. Let's examine the effects this function call has on the stack.

    0+0 = 3
    0+1 = 1
    0+2 = 2
  0008 = call 0xbfd58656ec9a8ebe, 2 // fn `foo`
=> frame 1 (1):
    1+0 = 1
    1+1 = 2

Here we can see a new call frame frame 1 being allocated, and that it contains two items: 1 and 2.

We can also see that the items are offset from position 1, which is the base of the current call frame. This is shown as the addresses 1+0 and 1+1. The value 3 at 0+0 is no longer visible, because it is outside of the current call frame.

Let's have a look at what happens when we return:

    1+0 = 1
    1+1 = 2
    1+2 = 3
  0003 = clean 2
    1+0 = 3
  0004 = return
<= frame 0 (0):
    0+0 = 3
    0+1 = 3

We call the clean 2 instruction, which tells the vm to preserve the top of the stack (1+2), and clean two items below it, leaving us with 3. We then return, which jumps us back to frame 0, which now has 0+0 visible and our return value at 0+1.

Compiler guide

This is intended to be a guide into the compiler architecture for Rune for people who want to hack on it.

Rune is in heavy development and this section is likely to change a lot.

Compiling a rune program involves the following stages:

• Queue the initial source files specified by Source::insert.
• Indexing and macro expansion, which processes tasks in the Worker queue until it is empty. These are:
  • Task::LoadFile - Loads a single source into AST file and indexes it.
  • Task::ExpandUnitWildcard - A deferred expansion of a wildcard import. This must happen after indexing because macros might expand into imports.
• Compilation which processes a queue of items to be compiled and assembled.

Indexing

Indexing is primarily handled through the Index trait, which are implemented for the type being indexed with the helper of the Indexer.

This walks through the AST to be indexed and construct components into an item path for every:

• Functions, which adds components named after the function. fn foo would add foo.
• Closures, blocks, and nested functions, which adds an id component, like $10 where the number depends on how many sibling components there are. These are effectively anonymous, and can't be referenced through the language directly.

Compilation

The compilation stage processed the entire AST of every function that is queued to be compiled and generates a sequence of instructions for them through implementations of the Assemble trait.

This stage uses the Query system to look up metadata about external items, and any external item queried for is subsequently queued up to be built.

Consider the following unit:

return foo();

fn foo() {
    2
}

fn bar() {
    3
}

Let's dump all dynamic functions in it:

$> cargo run -- run scripts/book/compiler_guide/dead_code.rn --dump-functions --warnings
# dynamic functions
0x0 = {root}()
0x481411c4bd0a5f6 = foo()
---
== 2 (59.8µs)

As you can see, the code for main::$0::bar was never generated. This is because it's a local function that is never called. And therefore never queried for. So it's never queued to be built in the compilation stage.

State during compilation

Each item in the AST is relatively isolated while they are being compiled. This is one of the benefits of compiling for a stack-based virtual machine - the compilation stage is relatively simple and most reasoning about what instructions to emit can be made locally.

Note that this quickly changes if you want to perform most forms of optimizations. But it's definitely true for naive (and therefore fast!) code generation.

While compiling we keep track of the following state in the Compiler

The source file and id that we are compiling for and global storage used for macro-generated identifiers and literals. This is used to resolve values from the AST through the corresponding Resolve implementation. An example of this is the Resolve implementation of LitStr.

We keep track of local variables using Scopes. Each block creates a new scope of local variables, and this is simply a number that is incremented each time a variable is allocated. These can either be named or anonymous. Each named variable is associated with an offset relative to the current call frame that can be looked up when a variable needs to be used.

We maintain information on loops we're through Loops. This is a stack that contains every loop we are nested in, information on the label in which the loop terminates, and locals that would have to be cleaned up in case we encounter a break expression.

There are a couple more traits which are interesting during compilation:

• AssembleConst - used for assembling constants.
• AssembleFn - used for assembling the content of functions.
• AssembleClosure - used for assembling closures.

Let's look closer at how closures are assembled through AssembleClosure. Once a closure is queried for, it is queued up to be built by the query system. The closure procedure would be compiled and inserted into the unit separately at a given item (like main::$0::$0). And when we invoke the closure, we assemble a call to this procedure.

We can see this call by dumping all the dynamic functions in the following script:

let callable = || 42;
dbg!(callable());

$> cargo run -- run scripts/book/compiler_guide/closures.rn --emit-instructions --dump-functions
# instructions
fn main() (0x1c69d5964e831fc1):
  0000 = load-fn hash=0xbef6d5f6276cd45e // closure `3`
  0001 = copy offset=0 // var `callable`
  0002 = call-fn args=0
  0003 = pop
  0004 = pop
  0005 = return-unit

fn main::$0::$0() (0xbef6d5f6276cd45e):
  0006 = push value=42
  0007 = return address=top, clean=0
# dynamic functions
0xbef6d5f6276cd45e = main::$0::$0()
0x1c69d5964e831fc1 = main()

A function pointer is pushed on the stack load-fn 0xca35663d3c51a903, then copied and called with zero arguments.

Deprecated

These are section of the book which have been deprecated for one reason or another, but we still want to provide links to.

You're probably looking for the section about Template literals.