Releases · mech-lang/mech

x := 1 + 2

Compiles into...

00000000: 4d 45 43 48 01 3a 02 00 00 04 00 00 00 05 00 00  MECH.:..........
00000010: 00 02 00 00 00 81 00 00 00 00 00 00 00 01 00 00  ................
00000020: 00 95 00 00 00 00 00 00 00 04 00 00 00 a5 00 00  ................
00000030: 00 00 00 00 00 60 00 00 00 00 00 00 00 05 01 00  .....`..........
00000040: 00 00 00 00 00 20 00 00 00 00 00 00 00 0c 00 00  ..... ..........
00000050: 00 00 00 00 00 25 01 00 00 00 00 00 00 31 01 00  .....%.......1..
00000060: 00 00 00 00 00 39 00 00 00 00 00 00 00 6a 01 00  .....9.......j..
00000070: 00 00 00 00 00 0d 00 00 00 00 00 00 00 00 00 00  ................
00000080: 00 02 00 00 00 32 00 00 00 00 00 00 00 0c 00 00  .....2..........
00000090: 00 00 00 00 00 01 00 00 00 0c 00 00 00 01 00 00  ................
000000a0: 00 00 00 00 00 00 00 00 00 01 08 00 00 00 00 00  ................
000000b0: 00 00 00 00 00 08 00 00 00 00 00 00 00 00 00 00  ................
000000c0: 00 01 08 00 00 08 00 00 00 00 00 00 00 08 00 00  ................
000000d0: 00 00 00 00 00 00 00 00 00 01 08 00 00 10 00 00  ................
000000e0: 00 00 00 00 00 08 00 00 00 00 00 00 00 00 00 00  ................
000000f0: 00 01 08 00 00 18 00 00 00 00 00 00 00 08 00 00  ................
00000100: 00 00 00 00 00 00 00 00 00 00 00 08 40 00 00 00  ............@...
00000110: 00 00 00 f0 3f 00 00 00 00 00 00 00 40 00 00 00  ....?.......@...
00000120: 00 00 00 08 40 a3 de 9c 71 ad 11 03 00 03 00 00  [email protected].......
00000130: 00 01 00 00 00 00 00 00 00 00 01 01 00 00 00 01  ................
00000140: 00 00 00 01 02 00 00 00 02 00 00 00 10 e8 ff dd  ................
00000150: bc 70 41 b5 00 00 00 00 00 01 00 00 00 02 00 00  .pA.............
00000160: 00 01 03 00 00 00 03 00 00 00 a3 de 9c 71 ad 11  .............q..
00000170: 03 00 01 00 00 00 78 d9 25 66 5c                 ......x.%f\

Sections include:

1. Header
  1. Magic number at the beginning `0x4d 45 43 48` spells "MECH".
  2. Bytecode version
  3. Mech Version
  4. Flags
  5. A whole bunch of offsets to read the various sections
2. Feature flag array - indicates to the compiler what features to include
3. Type array - interns types
4. Constants
  1. First a directory of constant entries 
  2. Then a blob of constants
5. Symbol table - mapping from symbol IDs to registers
6. Instructions - the actual program to be executed. Each instruction is:
  - Op code
  - Array of input registers
  - Output register
7. Dictionary - A mapping from symbol IDs to strings. This section is optional and is only needed if human-readable IDs are required.
8. Checksum

Here's a parsed representation of this particular program, which is a little more readable:

ParsedProgram {
    header: ByteCodeHeader {
        magic: [77,69,67,72],
        version: 1,
        mech_ver: 570,
        flags: 0,
        reg_count: 4,
        instr_count: 5,
        feature_count: 2,
        feature_off: 129,
        types_count: 1,
        types_off: 149,
        const_count: 4,
        const_tbl_off: 165,
        const_tbl_len: 96,
        const_blob_off: 261,
        const_blob_len: 32,
        symbols_len: 12,
        symbols_off: 293,
        instr_off: 305,
        instr_len: 57,
        dict_off: 362,
        dict_len: 13,
        reserved: 0,
    },
    features: [
        50,
        12,
    ],
    types: TypeSection {
        interner: {},
        entries: [
            TypeEntry {
                tag: F64,
                bytes: [],
            },
        ],
    },
    const_entries: [
        ParsedConstEntry {
            type_id: 0,
            enc: 1,
            align: 8,
            flags: 0,
            reserved: 0,
            offset: 0,
            length: 8,
        },
        ParsedConstEntry {
            type_id: 0,
            enc: 1,
            align: 8,
            flags: 0,
            reserved: 0,
            offset: 8,
            length: 8,
        },
        ParsedConstEntry {
            type_id: 0,
            enc: 1,
            align: 8,
            flags: 0,
            reserved: 0,
            offset: 16,
            length: 8,
        },
        ParsedConstEntry {
            type_id: 0,
            enc: 1,
            align: 8,
            flags: 0,
            reserved: 0,
            offset: 24,
            length: 8,
        },
    ],
    const_blob: [0,0,0,0,0,0,8,64,0,0,0,0,0,0,240,63,0,0,0,0,0,0,0,64,0,0,0,0,0,0,8,64],
    instr_bytes: [1,0,0,0,0,0,0,0,0,1,1,0,0,0,1,0,0,0,1,2,0,0,0,2,0,0,0,16,232,255,221,188,112,65,181,0,0,0,0,0,1,0,0,0,2,0,0,0,1,3,0,0,0,3,0,0,0,],
    symbols: {
        863861563252387: 3,
    },
    instrs: [
        ConstLoad {
            dst: 0,
            const_id: 0,
        },
        ConstLoad {
            dst: 1,
            const_id: 1,
        },
        ConstLoad {
            dst: 2,
            const_id: 2,
        },
        BinOp {
            fxn_id: 51018923245436904,
            dst: 0,
            lhs: 1,
            rhs: 2,
        },
        ConstLoad {
            dst: 3,
            const_id: 3,
        },
    ],
    dictionary: {
        863861563252387: "x",
    },
}

Byte Compiler / Interpreter

Generating Bytecode

The bytecode compiler has been integrated into the Mech runtime via the new "build command"

mech build file.mec -o .\output.mecb
[Load] Loaded: test.mec
[Watch] Watching: test.mec
[Output] Mech bytecode written to: .\output.mecb

.mecb is the extension for Mech bytecode.

Running Bytecode

You can run these generated bytecode files by feeding them back into the Mech tool:

mech output.mecb
[Load] Loaded: output.mecb
[Watch] Watching: output.mecb

  3

Supplying the --debug flag will print the above parsed bytecode program for inspection. This will be augmented in the future with more helpful debug output, but for now it will suffice.

This only works for a few functions so far.

Tiny Binaries

Now that we have bytecode files, we have unlocked the ability to compile Mech code to machinecode binaries. Sort of. Let me explain...

Mech right now is an interpreted language; the source code is parsed, and then the AST is run on the interpreter. Mech manages to be faster than other interpreted languages because it is statically typed, so we can unbox many values at compile time so they don't have to be at runtime.

Going from an AST to bytecode means that we can strip out the very hefty parser, and use a lightweight bytecode parser instead. Of course we can only do this because the proper parser and compiler did all the heavy lifting already, so the bytecode is very straightforward to parse, and that makes it fast as well.

So now we can take Mech beyond an AST interpreter which walks the AST, to a bytecode interpreter which will run a very concise program.

This brings me to the topic of this section: tiny binares.

If you look at the size of the Mech tool over time, you will see it has gone from 1MB to almost 100MB over time. This is because we have added more functions to the standard library over time, and the Mech tool bundles them all for convenience.

To reduce binary size, we can limit the contents of the executable to only those functions needed to carry out the program's instructions, excluding the rest. This can cut the resulting binary size from MB to KB.

To accomplish this, we leverage Rust's conditional compilation flags: [#cfg feature="xyz" ]

We've augmented the entire compiler with these flags, which isn't very ergonomic, but now that the work is done I'm very pleased with the results:

Build	Size	Dependencies	Build Time
Full	40 MB	55	6m 13s
Matrix	538KB	26	18.78s
Minimal	226KB	3	4.34s
Minimal Rust	133KB	0	0.30s

The minimal version of Mech takes up only 93KB of space on top of a Rust runtime, making it suitable for embedding in other applications. This is a truly minimal version, without any standard library or even any types aside from atoms, so it can't do much. It can't even store variables. Numbers like u32 and f64, strings, floats, data structures, functions, etc. can all be enabled "a la carte" as features.

The minimal install requires just these three dependencies:

paste - helps with writing macros, I don't think it should have any code in the executable.
byteoder - required to decode Mech bytecode, so it's necessary.
seahash - for hashing IDs

The "matrix" row was compiled with the features f64 matrix_matmul, and matrix, making it suitable for doing matrix multiplication operations, and only clocks in at 538KB. Most of that comes from nalgebra, so if we had a lighter linear algebra backend that could be used.

This also opens the door for embedded use cases, where we can use this feature to restrict use of dynamic memory allocation, and only allow stack allocation.

Code Reorganization

Some files had gotten too long so they've been broken into modules. For example, types.rs has become a module with complex_numbers, rational_numers, floats as members.

Full Changelog: v0.2.56-beta...v0.2.58-beta

Assets 5

05 Aug 21:01

cmontella

v0.2.56-beta

8365d0a

v0.2.56-beta

Added Complex and Rational Numbers

Added two new kinds, complex numbers c64 and rational numbers r64.

Complex Numbers

Complex numbers have a real and imaginary part. The imaginary part can be indicated with i or j notation.

Examples:

a := 3+4i
b := 1-2j
c := a + b -- c will be 4+2i

We've added support for matrix operations as well:

A := [1+2i, 3-4i; 5+6i, 7-8i]
B := [2+3i, 4-5i; 6+7i, 8-9i]
C := A * B -- multiplies the two matrices element-wise

The implementation uses the type frome nalgebra crate, so in the future we will be able to support all of the linear algebra functions that incorporate complex numbers.

These values are represented as two f64 values, one for the real part and one for the imaginary part.

Rational Numbers

Rational numbers r64 are represented as a pair of 64-bit integers, the numerator and denominator.

a := 3/4   -- represents the rational number 3/4
b := 5/6   -- represents the rational number 5/6
c := a + b -- c will be 19/24

Matrix operations are also supported for rational numbers:

A := [1/2 3/4] + 1/2

Rational numbers are useful for representing fractions and performing exact arithmetic without floating-point errors.

a := 1/10 + 2/10 + 3/10 -- a will be exactly 3/5
b := 0.1 + 0.2 + 0.3    -- b will be approximately 3/5
a != b<r64>             -- true, because a is exact while b is approximate

Note, b represented as a rational is 1688849860263938/2814749767106563.

And Operator

Changed the and operator from & to && to align with or operator change from last version.

Table and Set Operators

Added parsers for the table and set operators such as join, union, intersection, and difference. The functions aren't added yet but they are coming.

Upgraded to Rust v0.1.88.

Upgraded to nalgebra v0.34.0.

Various bug fixes as well.

Full Changelog: v0.2.52-beta...v0.2.56-beta

Assets 6

29 Jul 03:07

cmontella

v0.2.52-beta

720cb28

v0.2.52-beta

Table Append

Appending a Row to a Table

This example starts with a table and appends a new row using a record with matching column types.

~x := |a<f64> b<f64>| 1 2 | 3 4 |;
x += {a<f64>: 5, b<f64>: 6};

Appending a Row with Columns in Different Order

You can append a row even if the record fields are in a different order.

~x := |a<f64> b<f64>| 1 2 | 3 4 |;
x += {b<f64>: 6, a<f64>: 5};

Appending a Row from Variables

Rows can also be constructed dynamically using variables.

~x := |a<u64> b<u8>| 1 2 | 3 4 |;
a := 13;
b := 14;
y := {c<bool>: false, a<u64>: a, b<u8>: b};
x += y;

Appending Another Table

You can append one table to another when they share the same schema.

~x := |a<u64> b<u8>| 1 2 | 3 4 |;
y := |a<u64> b<u8>| 5 6 | 7 8 |;
x += y;

Table Select

Selecting Specific Rows by Index

Selecting specific rows can be done by providing a list of row indices.

x := |a<u64> b<u8>| 1 2 | 3 4 | 5 6 |;
x{[1,3]}

Selecting Rows Using Logical Conditions

Rows can also be filtered using a logical column.

a := |x<u64> y<bool>| 2 true | 3 false | 4 false | 5 true |;
a{a.y}

Selecting Rows Using Expressions

You can filter rows using an expression over the table's columns.

a := |x<u64> y<bool>| 2 true | 3 false | 4 false | 5 true |;
a{a.x > 3<u64>}

Table Create

Creating a Table from a Numeric Matrix

Tables can be initialized directly from a matrix with numeric values.

x := [1 2; 3 4];
a<|foo<f64>, bar<f64>|> := x

Creating a Table from a String Matrix

You can also create a table from a matrix of strings.

x := ["true" "false"; "true" "false"];
a<|x<string> y<string>|> := x

Creating a Table from a Boolean Matrix

Boolean matrices can be converted into tables as well.

x := [true false; true false];
a<|x<bool> y<bool>|> := x;

Creating a Table from an Integer Matrix

Here's an example of converting an integer matrix into a table with specific numeric types.

x := [1 2; 3 4];
a<|x<u8> y<i8>|> := x;

Matrix Reshape

Reshaping a Matrix

You can reshape a matrix into a different shape by specifying the new dimensions.

x := [1 3; 2 4];
y<[u64]:4,1> := x

Reshaping a Matrix of Strings

Similarly, matrices of strings can be reshaped.

x := [1 2 3 4];
y<[string]:2,2> := x

Converting a Range to a String Matrix

A range can be converted into a matrix of strings.

x := 1..=4;
out<[string]> := x

Assets 3

23 Jul 18:53

cmontella

v0.2.51-beta

c452b25

v0.2.51-beta

Fixed Table Kind

This is now a valid table kind:

<|x<u64> y<bool>|:3>

Table Row Select

You can now select a table row by index, which returns a record:

x := |a<f32> b<u8>|
     | 1.2    3   |
     | 1.3    4   |
x{2} == {a<f32>: 1.3, b<u8>: 4}     -- true

Mechdown Improvements.

Multi Paragraph Blocks

You can now insert a abstract or quote block with multiple paragraphs.

%% This is an abstract
This is the second paragraph of the abstract

> This is a quote block
This is the second paragraph of the quote block

This is a paragraph outside of the quote block.

Question and Info Blocks

Added syntax for a Question Block and an Info Block, which are like a Quote Block but intended to contain FAQs and notes for the user.

!!> This is an info block, for information the reader needs highlighted

??> This is a question block, to answer frequently asked questions

Fixed a problem with markdown tables where blank cells were not being parsed, leaving the table malformed. Wasn't causing a parse error.

Cool 3D mech logo

Added a cool 3D logo. I don't know where I'll use it yet but it took a while to draw so it's going in the source.

https://github.com/mech-lang/mech/blob/fd4c57a524bf40412e296c38aa8fa60b3fbc8dee/src/bin/mech.rs#L50

Full Changelog: v0.2.50-beta...v0.2.51-beta

Assets 8

15 Jul 22:26

cmontella

v0.2.50-beta

dd0af21

v0.2.50-beta

Revamped table parsing. This is how it works now:

Regular:

     |x<f64> y<u8>|
     |  1      2  |
     |  3      4  |
     |  5      6  |

Fancy:

   ╭────────┬────────╮
   │ x<u64> │ y<f32> │
   ├────────┼────────┤
   │   1    │   2    │
   │   3    │   4    │
   ╰────────┴────────╯

Max Fancy:

   ╭────────┬────────╮
   │ x<u64> │ y<f32> │
   ├────────┼────────┤
   │   1    │   2    │
   ├────────┼────────┤
   │   3    │   4    │
   ╰────────┴────────╯

Min Fancy:

   ╭────────┬────────╮
   │ x<u64> │ y<f32> │
   │   1    │   2    │
   │   3    │   4    │
   ╰────────┴────────╯

To support this, we changed the logic/or operator from | to ||, which is standard so I don't think it's a big deal. logic/and is still &,but that will probably change to && in the next version.

Full Changelog: v0.2.49-beta...v0.2.50-beta

Assets 7

09 Jul 16:24

cmontella

v0.2.49-beta

6f98bc4

v0.2.49-beta

Tuple Destructure

This now works:

x := (1, true, "hello")
(a,b,c) := x
a == 1          -- true
b == true       -- true
c == "hello"    -- true

Git History

Added legacy Mech code from 2014 to Git repo history as best I could. This code wasn't version controlled, but there are some release notes in the files and creation dates that indicate when they were worked on.

Documentation

Added value docs
Added matrix docs

Various bug fixes as well

Full Changelog: v0.2.48-beta...v0.2.49-beta

Assets 9

01 Jul 15:20

cmontella

v0.2.48-beta

e09904a

v0.2.48-beta

Tauri App

I'm experimenting with a Tauri app, which is kind of like Electron but for Rust. It allows one to use a webview as a user interface, which is a little nicer than egui, which is canvas based. Right now the wasm build is powering it, but I should be able to bundle a native runtime in the tauri app. This is important because the wasm build is very slow, which is fine on the web, but this app is supposed to be native. So I'll have to work on it some more before it's really useful. For now what it does is it exposes the wasm REPL in the webview.

Kind Values

Kinds are now proper values, so you can write this:

x := <u32>

There's not much you can do with this yet, but it allows for writing out types in a document and formatting them properly. In the future this will be more useful with state machines.

Tuple Destructuring

You can now destructure tuples with a new tuple destructure statement.

q := (10, "b", true); 

r := (q.3, q.2, q.1)   -- access elements with index

s := (q.0, q.4)        -- compile time errors

(x,y,z) := q           -- destructure elements of q into x, y and z

t := (z,y,x)

t == r   -- true

Implicit conversion

Turns out this is strange to implement widely, so this is no longer supported... for now:

x := [8<u8> 9 10]   -- type error

Full Changelog: v0.2.47-beta...v0.2.48-beta

Assets 9

24 Jun 14:10

cmontella

v0.2.47-beta

c02a898

v0.2.47-beta

Wasm REPL

Added a REPL environment to docs. Supports evaluating code in the browser. You can try it here: https://try.mech-lang.org

Some features:

Click on variables in the document, their values are displayed in the repl
evaluate expressions in the repl
:whos command will print all variables or indicated variables
:docs command will print out the indicated document in the repl. Try it with :docs math/sin

Implicit type conversions

x := [8<u8> 9 10]    -- evaluates to [u8]

Full Changelog: v0.2.46-beta...v0.2.47-beta

Assets 9

Releases: mech-lang/mech

v0.2.66-beta - Mechdown Beta Preview

Uh oh!

v0.2.58-beta 🎂

Mech Bytecode

Byte Compiler / Interpreter

Generating Bytecode

Running Bytecode

Tiny Binaries

Code Reorganization

Uh oh!

v0.2.56-beta

Added Complex and Rational Numbers

Complex Numbers

Rational Numbers

And Operator

Table and Set Operators

Uh oh!

v0.2.52-beta

Table Append

Appending a Row to a Table

Appending a Row with Columns in Different Order

Appending a Row from Variables

Appending Another Table

Table Select

Selecting Specific Rows by Index

Selecting Rows Using Logical Conditions

Selecting Rows Using Expressions

Table Create

Creating a Table from a Numeric Matrix

Creating a Table from a String Matrix

Creating a Table from a Boolean Matrix

Creating a Table from an Integer Matrix

Matrix Reshape

Reshaping a Matrix

Reshaping a Matrix of Strings

Converting a Range to a String Matrix

Uh oh!

v0.2.51-beta

Fixed Table Kind

Table Row Select

Mechdown Improvements.

Multi Paragraph Blocks

Question and Info Blocks

Cool 3D mech logo

Uh oh!

v0.2.50-beta

Uh oh!

v0.2.49-beta

Tuple Destructure

Git History

Documentation

Uh oh!

v0.2.48-beta

Tauri App

Kind Values

Tuple Destructuring

Implicit conversion

Uh oh!

v0.2.47-beta

Wasm REPL

Implicit type conversions

Uh oh!