assemblyscript documentation

README.md

@@ -3,21 +3,19 @@
 
 [![PyPI](https://img.shields.io/pypi/v/beschi)](https://pypi.org/project/beschi/) [![GitHub Workflow Status](https://img.shields.io/github/actions/workflow/status/sjml/beschi/ci.yml)](https://github.com/sjml/beschi/actions/workflows/ci.yml)
 
-This is a custom bit-packing and unpacking code generator for C, C#, Go, Rust, Swift, TypeScript, and Zig.[^1] You feed it a data description and it generates source files for writing/reading buffers of that data, along the lines of [FlatBuffers](https://google.github.io/flatbuffers/) or [Cap'n Proto](https://capnproto.org), but with much less functionality for much simpler use cases. It was initially written for a larger project that was passing data back and forth between a Unity game, a Go server, and a web client, but I extracted it into its own thing. If all you need is a simple way to pack a data structure into a compact, portable binary form, this might be useful for you.
+This is a custom bit-packing and unpacking code generator for C, C#, Go, Rust, Swift, TypeScript, AssemblyScript, and Zig. You feed it a data description and it generates source files for writing/reading buffers of that data, along the lines of [FlatBuffers](https://google.github.io/flatbuffers/) or [Cap'n Proto](https://capnproto.org), but with much less functionality for much simpler use cases. It was initially written for a larger project that was passing data back and forth between a Unity game, a Go server, and a web client, but I extracted it into its own thing. If all you need is a simple way to pack a data structure into a compact, portable binary form, this might be useful for you.
 
 I go into more explanation for why this exists [in the documentation](https://github.com/sjml/beschi/tree/main/docs/), but I'll be honest, too: it **was** kind of fun to write a code generator. 😝 
 
-[^1]: There is also experimental support for AssemblyScript, but it's not run through the test suite so THERE BE DRAGONS. 🐉
-
 ## Documentation
 
 * [Introduction](https://github.com/sjml/beschi/tree/main/docs/introduction.md)
 * [Protocols](https://github.com/sjml/beschi/tree/main/docs/protocols.md)
 
 Language-Specific Documentation: 
 
-| [C](https://github.com/sjml/beschi/tree/main/docs/languages/c.md) | [C#](https://github.com/sjml/beschi/tree/main/docs/languages/csharp.md) | [Go](https://github.com/sjml/beschi/tree/main/docs/languages/go.md) | [Rust](https://github.com/sjml/beschi/tree/main/docs/languages/rust.md) | [Swift](https://github.com/sjml/beschi/tree/main/docs/languages/swift.md) | [TypeScript](https://github.com/sjml/beschi/tree/main/docs/languages/typescript.md) | [Zig](https://github.com/sjml/beschi/tree/main/docs/languages/zig.md)
-|-|-|-|-|-|-|-|
+| [C](https://github.com/sjml/beschi/tree/main/docs/languages/c.md) | [C#](https://github.com/sjml/beschi/tree/main/docs/languages/csharp.md) | [Go](https://github.com/sjml/beschi/tree/main/docs/languages/go.md) | [Rust](https://github.com/sjml/beschi/tree/main/docs/languages/rust.md) | [Swift](https://github.com/sjml/beschi/tree/main/docs/languages/swift.md) | [TypeScript](https://github.com/sjml/beschi/tree/main/docs/languages/typescript.md) | [AssemblyScript](https://github.com/sjml/beschi/tree/main/docs/languages/assemblyscript.md) | [Zig](https://github.com/sjml/beschi/tree/main/docs/languages/zig.md)
+|-|-|-|-|-|-|-|-|
 
 * [Dev Notes](https://github.com/sjml/beschi/tree/main/docs/dev)
 
@@ -150,19 +148,19 @@ Note that while [the TOML spec](https://toml.io/en/v1.0.0) does not guarantee th
 
 These are the base types from which you can build up whatever structures and messages you need to, along with what they correspond to in the various languages. 
 
-| Protocol Type | C          | C#       | Go        | Rust     | Swift     | TypeScript | Zig           |
-|---------------|------------|----------|-----------|----------|-----------|------------|---------------|
-| **`byte`**    | `uint8_t`  | `byte`   | `byte`    | `u8`     | `UInt8`   | `number`   | `u8`          |
-| **`bool`**    | `bool`     | `bool`   | `bool`    | `bool`   | `Bool`    | `boolean`  | `bool`        |
-| **`int16`**   | `uint16_t` | `short`  | `int16`   | `i16`    | `Int16`   | `number`   | `i16`         |
-| **`uint16`**  | `int16_t`  | `ushort` | `uint16`  | `u16`    | `UInt16`  | `number`   | `u16`         |
-| **`int32`**   | `uint32_t` | `int`    | `int32`   | `i32`    | `Int32`   | `number`   | `i32`         |
-| **`uint32`**  | `int32_t`  | `uint`   | `uint32`  | `u32`    | `UInt32`  | `number`   | `u32`         |
-| **`int64`**   | `uint64_t` | `long`   | `int64`   | `i64`    | `Int64`   | `bigint`   | `i64`         |
-| **`uint64`**  | `int64_t`  | `ulong`  | `uint64`  | `u64`    | `UInt64`  | `bigint`   | `u64`         |
-| **`float`**   | `float`    | `float`  | `float32` | `f32`    | `Float32` | `number`   | `f32`         |
-| **`double`**  | `double`   | `double` | `float64` | `f64`    | `Float64` | `number`   | `f64`         |
-| **`string`**  | `char*`    | `string` | `string`  | `String` | `String`  | `string`   | `[]const u8`  |
+| Protocol Type | C          | C#       | Go        | Rust     | Swift     | TypeScript | AssemblyScript | Zig           |
+|---------------|------------|----------|-----------|----------|-----------|------------|----------------|---------------|
+| **`byte`**    | `uint8_t`  | `byte`   | `byte`    | `u8`     | `UInt8`   | `number`   | `u8`           | `u8`         |
+| **`bool`**    | `bool`     | `bool`   | `bool`    | `bool`   | `Bool`    | `boolean`  | `bool`         | `bool`       |
+| **`int16`**   | `uint16_t` | `short`  | `int16`   | `i16`    | `Int16`   | `number`   | `i16`          | `i16`        |
+| **`uint16`**  | `int16_t`  | `ushort` | `uint16`  | `u16`    | `UInt16`  | `number`   | `u16`          | `u16`        |
+| **`int32`**   | `uint32_t` | `int`    | `int32`   | `i32`    | `Int32`   | `number`   | `i32`          | `i32`        |
+| **`uint32`**  | `int32_t`  | `uint`   | `uint32`  | `u32`    | `UInt32`  | `number`   | `u32`          | `u32`        |
+| **`int64`**   | `uint64_t` | `long`   | `int64`   | `i64`    | `Int64`   | `bigint`   | `i64`          | `i64`        |
+| **`uint64`**  | `int64_t`  | `ulong`  | `uint64`  | `u64`    | `UInt64`  | `bigint`   | `u64`          | `u64`        |
+| **`float`**   | `float`    | `float`  | `float32` | `f32`    | `Float32` | `number`   | `f32`          | `f32`        |
+| **`double`**  | `double`   | `double` | `float64` | `f64`    | `Float64` | `number`   | `f64`          | `f64`        |
+| **`string`**  | `char*`    | `string` | `string`  | `String` | `String`  | `string`   | `string`       | `[]const u8` |
 
 All the numbers are stored as little-endian in the buffer, if that matters for you. 
 

beschi/writers/boilerplate/AssemblyScript.ts

@@ -273,7 +273,6 @@ export function UnpackMessages(data: DataView): Message[] {
   const da = new DataAccess(data);
   if (da.data.byteLength < 12) {
     throw new Error("Packed message buffer is too short.");
-    return [];
   }
   const headerBuffer = new Uint8Array(4);
   for (let i = 0; i < 4; i++) {
@@ -282,7 +281,6 @@ export function UnpackMessages(data: DataView): Message[] {
   const headerLabel = String.UTF8.decode(headerBuffer.buffer, false);
   if (headerLabel !== "BSCI") {
     throw new Error("Packed message buffer has invalid header: " + headerLabel);
-    return [];
   }
   const msgCount = da.getUint32();
   if (msgCount == 0) {
@@ -292,7 +290,6 @@ export function UnpackMessages(data: DataView): Message[] {
   const msgList = ProcessRawBytes(dv, msgCount);
   if (msgList.length == 0) {
     throw new Error("No messages in buffer.");
-    return [];
   }
   if (msgList.length != msgCount) {
     throw new Error("Unexpected number of messages in buffer.");

docs/dev/todo.md

@@ -5,8 +5,12 @@ This file is a rough todo list for the tool itself.
 - update documentation before publishing 0.3.*
     - if you call from_bytes on a generic message, it has to be tagged
     - Swift: the PackMessages and stuff hangs off the array because of Reasons™
+- AssemblyScript documentation
+    - fix to use generics 
+    - make the base class fromBytes a generated delegator thing like Rust's
 - UnpackMessages should check for EOF on reads and throw/return errors as appropriate
     - make sure we're consuming the terminator, too
+    - update test suite with packed_broken (sigh)
 
 ## possible future protocol features:
 - ?? inline string and array length types so they don't have to be protocol-wide like they are now

docs/introduction.md

@@ -2,7 +2,7 @@
 
 This is a general introduction to Beschi as a whole. Once you're done reading this, you'll probably want to check out the [more specific documentation](./languages/) for your desired language(s). 
 
-| [C](./languages/c.md) | [C#](./languages/csharp.md) | [Go](./languages/go.md) | [Rust](./languages/rust.md) | [Swift](./languages/swift.md) | [TypeScript](./languages/typescript.md) | [Zig](./languages/zig.md)
+| [C](./languages/c.md) | [C#](./languages/csharp.md) | [Go](./languages/go.md) | [Rust](./languages/rust.md) | [Swift](./languages/swift.md) | [TypeScript](./languages/typescript.md) | [AssemblyScript](./languages/assemblyscript.md) | [Zig](./languages/zig.md)
 |-|-|-|-|-|-|-|
 
 

docs/languages/assemblyscript.md

@@ -1,2 +1,32 @@
-enums not in range will become _Unknown
-might return null from message stuff
+# Using Beschi with AssemblyScript
+
+(You can see [an example AssemblyScript file](../generated_examples/assemblyscript_example.ts) generated from [this annotated protocol](../../test/_protocols/annotated.toml). Example code can be found in [the test harnesses directory](../../test/_harnesses/assemblyscript/).) 
+
+
+## Data Types
+
+The base data types map to AssemblyScript accordingly: 
+
+| Protocol Type | AssemblyScript type |
+|---------------|---------------------|
+| `byte`        | `u8`                |
+| `bool`        | `bool`              |
+| `int16`       | `i16`               |
+| `uint16`      | `u16`               |
+| `int32`       | `i32`               |
+| `uint32`      | `u32`               |
+| `int64`       | `i64`               |
+| `uint64`      | `u64`               |
+| `float`       | `f32`               |
+| `double`      | `f64`               |
+| `string`      | `string`            |
+
+
+## Caveats
+
+* [AssemblyScript](https://www.assemblyscript.org/) is a bit of a strange beast. I had expected it to be "a strongly-typed subset of TypeScript that compiles to WebAssembly" but it turns out it's closer to "C, but with TypeScript syntax and a bunch of convenience functions." That's not a bad thing to be, at all, but it just took me a little by surprise. 
+    * The AssemblyScript generator is implemented as a bunch of modifications to the TypeScript generator. Knowing the above now, I probably would have written it from scratch. But it works now, so it stays until I get frustrated with it. 
+    * In fairness to AssemblyScript, its self-description is "A TypeScript-like language for WebAssembly" so it's being pretty honest about the value proposition. 
+* In particular, AssemblyScript lacks exceptions in any kind of recoverable way; so Beschi-generated code does not throw when it encounters a problem. If possible (like in the `writeBytes` functions), it returns a bool indicating success or failure. Reading messages might return `null`. An out-of-range enum, instead of throwing an exception, will be set to a value of `_Unknown`. 
+    * The only exceptions (hehe) to this are anything dealing with PackedMessage -- if you're trying to unpack a buffer that doesn't have the right header, or miscounts the number of messages or something, there's no real good way to recover from that anyway. 
+* While it passes the test suite, this is probably the least mature and least tested-in-a-real-setting of the supported languages. 

docs/languages/typescript.md

@@ -29,4 +29,3 @@ The base data types map to TypeScript accordingly:
 * 64-bit integers (both signed and unsigned) are implemented with [BigInt](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/BigInt), which has [pretty broad support at this point](https://caniuse.com/?search=bigint). Your client code may need to handle them differently though -- you can't seamlessly do math with a regular `number` and a `bigint`. Users of other languages are used to these kinds of folds, but JavaScript/TypeScript users may find them new and annoying. :) 
 * Reading and writing messages is wrapped in a custom `DataAccess` class that tracks position in a `DataView`. You can either construct it yourself if you want to, for instance, do multiple passes of writing to the same buffer. If you pass a `DataView` to a function that expects a `DataAccess`, the latter will be used internally but not returned to you. 
     * Be careful of long-lasting `DataView` (and thus `DataAccess`) objects, though, since they become invalid if their underlying `ArrayBuffer` gets detached. Usually you're aware of that happening, but if you're working with a WebAssembly module's memory, it happens whenever the memory grows (which you might **not** be aware of). You can always just make a new `DataView` object from the memory when that happens, though. (Documenting here because I was bitten by it myself!)
-* There is an experimental generator for AssemblyScript which leverages the TypeScript support. It's not run through the test suite (because testing WebAssembly is a pain) but seems to work provisionally. 

test/_harnesses/assemblyscript/builder.py

@@ -25,7 +25,7 @@ def build(self):
                 "npm", "install"
             ])
 
-        if builder_util.needs_build(self.intermediate_path, ["harness.mjs", "assembly/_harness.ts", f"assembly/{self.srcfile}"]):
+        if builder_util.needs_build(self.intermediate_path, ["harness.mjs", "assembly/_harness.ts", f"assembly/{self.srcfile}", *self.gen_files]):
             subprocess.check_call([
                 "npx", "asc", "--outFile", self.intermediate_path, f"assembly/{self.srcfile}"
             ])

test/_harnesses/assemblyscript/harness.mjs

@@ -31,13 +31,16 @@ function liftString(pointer, mem) {
     return string + String.fromCharCode(...memoryU16.subarray(start, end));
 }
 
+let abortMessageExpectation = null;
+let receivedAbortMessage = null;
 try {
     const instance = await WebAssembly.instantiate(wasmModule, {
         env: {
             abort: (msg, file, line, col) => {
                 msg = liftString(msg, instance.exports.memory);
                 file = liftString(file, instance.exports.memory);
                 console.error(`ASSEMBLYSCRIPT FATAL ERROR: ${file}:${line}:${col}\n${msg}`);
+                receivedAbortMessage = msg;
             },
             log: (msg) => {
                 console.log(liftString(msg, instance.exports.memory));
@@ -50,7 +53,10 @@ try {
                     console.error(`FAILED! AssemblyScript: ${label}`);
                     ok = false;
                 }
-            }
+            },
+            expectAbort: (message) => {
+                abortMessageExpectation = message;
+            },
         }
     });
 
@@ -70,9 +76,15 @@ try {
         instance.exports.read();
     }
 
-} catch(e) {
-    console.error(`WebAssembly Error: ${programName}`, e);
-    process.exit(2);
+}
+catch(e) {
+    if (abortMessageExpectation !== null && abortMessageExpectation === receivedAbortMessage) {
+        // it failed like we expected
+    }
+    else {
+        console.error(`WebAssembly Error: ${programName}`, e);
+        process.exit(2);
+    }
 }
 
 if (!ok) {