Release 1.4.0.

changelog.md

@@ -6,7 +6,7 @@ Note that since 0.3.0, no deprecations have actually resulted in
 removals. They are advisory only and we have no plans to break the
 deprecated forms.
 
-## 1.4.0 / ???
+## 1.4.0 / 2023-12-01
 
 ### Bug Fixes
 * Prevent metadata from being accessible in compiler sandbox.

man/man3/fennel-api.3

@@ -1,4 +1,4 @@
-.\" Automatically generated by Pandoc 3.1.2
+.\" Automatically generated by Pandoc 2.17.1.1
 .\"
 .\" Define V font for inline verbatim, using C font in formats
 .\" that render this, and otherwise B font.
@@ -14,7 +14,7 @@
 . ftr VB CB
 . ftr VBI CBI
 .\}
-.TH "fennel-api" "3" "2023-09-25" "fennel 1.3.2-dev" "Fennel\[aq]s Lua API"
+.TH "fennel-api" "3" "2023-12-01" "fennel 1.4.0" "Fennel\[aq]s Lua API"
 .hy
 .SH NAME
 .PP
@@ -365,8 +365,9 @@ The list also contains these keys indicating where it was defined:
 This data is used for stack traces and for pinpointing compiler error
 messages.
 Note that column numbers are based on character count, which does not
-always correspond to visual columns; for instance \[dq]วัด\[dq] is three
-characters but only two visual columns.
+always correspond to visual columns; for instance
+\[dq]\[u0E27]\[u0E31]\[u0E14]\[dq] is three characters but only two
+visual columns.
 .SS sequence/key-value table
 .PP
 These are table literals in Fennel code produced by square brackets
@@ -644,8 +645,8 @@ this with Fennel later would result in an incorrect escape code in Lua
 .SH WORK WITH DOCSTRINGS AND METADATA
 .PP
 When running a REPL or using compile/eval with metadata enabled, each
-function declared with \f[V]fn\f[R] or \f[V]λ/lambda\f[R] will use the
-created function as a key on \f[V]fennel.metadata\f[R] to store the
+function declared with \f[V]fn\f[R] or \f[V]\[*l]/lambda\f[R] will use
+the created function as a key on \f[V]fennel.metadata\f[R] to store the
 function\[aq]s arglist and (if provided) docstring.
 The metadata table is weakly-referenced by key, so each function\[aq]s
 metadata will be garbage collected along with the function itself.
@@ -671,7 +672,7 @@ operate on that\[aq]s bound to the function.
 .IP
 .nf
 \f[C]
-local greet = fennel.eval(\[aq](λ greet [name] \[dq]Say hello\[dq] (print \[dq]Hello,\[dq] name))\[aq],
+local greet = fennel.eval(\[aq](\[*l] greet [name] \[dq]Say hello\[dq] (print \[dq]Hello,\[dq] name))\[aq],
                           {useMetadata = true})
 
 fennel.metadata[greet]

man/man5/fennel-reference.5

@@ -1,4 +1,4 @@
-.\" Automatically generated by Pandoc 3.1.2
+.\" Automatically generated by Pandoc 2.17.1.1
 .\"
 .\" Define V font for inline verbatim, using C font in formats
 .\" that render this, and otherwise B font.
@@ -14,7 +14,7 @@
 . ftr VB CB
 . ftr VBI CBI
 .\}
-.TH "fennel-reference" "5" "2023-09-25" "fennel 1.3.2-dev" "Fennel Reference"
+.TH "fennel-reference" "5" "2023-12-01" "fennel 1.4.0" "Fennel Reference"
 .hy
 .SH NAME
 .PP
@@ -61,7 +61,16 @@ For example: \f[V](print \[dq]hello world\[dq])\f[R]
 .PP
 \f[V]{curly brackets}\f[R]: used to denote key/value table literals,
 also known as dictionaries.
-For example: \f[V]{:a 1 :b 2}\f[R]
+For example: \f[V]{:a 1 :b 2}\f[R] In a table if you have a string key
+followed by a symbol of the same name as the string, you can use
+\f[V]:\f[R] as the key and it will be expanded to a string containing
+the name of the following symbol.
+.IP
+.nf
+\f[C]
+{: this} ; is shorthand for {:this this}
+\f[R]
+.fi
 .PP
 \f[V][square brackets]\f[R]: used to denote sequential tables, which can
 be used for literal data structures and also in specials and macros to
@@ -194,7 +203,7 @@ overflowing the stack, provided the call is in a tail position.
 .PP
 The final form in this and all other function forms is used as the
 return value.
-.SS \f[V]lambda\f[R]/\f[V]λ\f[R] nil-checked function
+.SS \f[V]lambda\f[R]/\f[V]\[*l]\f[R] nil-checked function
 .PP
 Creates a function like \f[V]fn\f[R] does, but throws an error at
 runtime if any of the listed arguments are nil, unless its identifier
@@ -224,11 +233,11 @@ the \f[V]?foo\f[R] notation is used by the compiler to indicate that a
 nil value is allowed, but it is a useful notation to communicate intent
 anywhere a new local is introduced.
 .PP
-The \f[V]λ\f[R] form is an alias for \f[V]lambda\f[R] and behaves
+The \f[V]\[*l]\f[R] form is an alias for \f[V]lambda\f[R] and behaves
 identically.
 .SS Docstrings and metadata
 .PP
-The \f[V]fn\f[R], \f[V]lambda\f[R], \f[V]λ\f[R] and \f[V]macro\f[R]
+The \f[V]fn\f[R], \f[V]lambda\f[R], \f[V]\[*l]\f[R] and \f[V]macro\f[R]
 forms accept an optional docstring.
 .IP
 .nf
@@ -237,7 +246,7 @@ forms accept an optional docstring.
   \[dq]Print the sum of x and y\[dq]
   (print (+ x y)))
 
-(λ pxyz [x ?y z]
+(\[*l] pxyz [x ?y z]
   \[dq]Print the sum of x, y, and z. If y is not provided, defaults to 0.\[dq]
   (print (+ x (or ?y 0) z)))
 \f[R]
@@ -497,6 +506,7 @@ Example:
 If a table key is a string with the same name as the local you want to
 bind to, you can use shorthand of just \f[V]:\f[R] for the key name
 followed by the local name.
+This works for both creating tables and destructuring them.
 .PP
 Example:
 .IP
@@ -1578,9 +1588,10 @@ Example:
 \f[R]
 .fi
 .PP
-If the name of the method isn\[aq]t known at compile time, you can use
-\f[V]:\f[R] followed by the table and then the method\[aq]s name as a
-string.
+In the example above, \f[V]f:write\f[R] is a single multisym.
+If the name of the method or the table containing it isn\[aq]t fixed,
+you can use \f[V]:\f[R] followed by the table and then the method\[aq]s
+name to allow it to be a dynamic string instead:
 .PP
 Example:
 .IP
@@ -1697,6 +1708,26 @@ in rather than using the value from the previous form for the next form.
 .PP
 The first form becomes the return value for the whole expression, and
 subsequent forms are evaluated solely for side-effects.
+.SS \f[V]tail!\f[R]
+.PP
+The \f[V]tail!\f[R] form asserts that its argument is called in a tail
+position.
+You can use this when using tail calls to recurse over large data sets
+in a way that might not be obvious; that way if the code is changed so
+that the recursive call is no longer a tail call, it will cause a
+compile error instead of overflowing the stack on large data sets.
+.IP
+.nf
+\f[C]
+(fn process-all [data i]
+  (case (process (. data i))
+    :done (print \[dq]Process completed.\[dq])
+    :next (process-all data (+ i 1))
+    :skip (do (process-all data (+ i 2))
+;;             \[ha]\[ha]\[ha]\[ha]\[ha]\[ha]\[ha]\[ha]\[ha]\[ha]\[ha] Compile error: Must be in tail position
+              (print \[dq]Skipped\[dq] (+ i 1)))))
+\f[R]
+.fi
 .SS \f[V]include\f[R]
 .IP
 .nf
@@ -1728,46 +1759,39 @@ Starting from version 0.10.0 \f[V]include\f[R] and hence
 \f[V]--require-as-include\f[R] support semi-dynamic compile-time
 resolution of module paths similarly to \f[V]import-macros\f[R].
 See the relative require section in the tutorial for more information.
-.SS \f[V]debug-repl\f[R]
+.SS \f[V]assert-repl\f[R]
 .PP
 \f[I](Since 1.4.0)\f[R]
 .PP
 Sometimes it\[aq]s helpful for debugging purposes to drop a repl right
 into the middle of your code to see what\[aq]s really going on.
-You can use the \f[V]debug-repl\f[R] macro to do this:
+You can use the \f[V]assert-repl\f[R] macro to do this:
 .IP
 .nf
 \f[C]
 (let [input (get-input)
       value []]
   (fn helper [x]
     (table.insert value (calculate x)))
-  (debug-repl)
-  value)
+  (assert-repl (transform helper value) \[dq]could not transform\[dq]))
 \f[R]
 .fi
 .PP
-This will drop you into a repl when you hit that point in the code.
-The repl will have access to all the locals that are in scope.
+This works like the built-in \f[V]assert\f[R] function, but when the
+condition is false or nil, instead of an error, it drops into a repl
+which has access to all the locals that are in scope.
 (This would be \f[V]input\f[R], \f[V]value\f[R], and \f[V]helper\f[R] in
 the example above.)
-It takes an optional options table which accepts all the same values as
-the \f[V]fennel.repl\f[R] function in the API.
+It takes an optional options table as its third argument which accepts
+all the same values as the \f[V]fennel.repl\f[R] function in the API.
+.PP
+You can \f[V],return EXPRESSION\f[R] from the repl to replace the
+original failing condition with a different arbitrary value.
+Returning false or nil will trigger a regular \f[V]assert\f[R] failure.
 .PP
 Note that this is meant for use in development and will not work with
 ahead-of-time compilation unless your build also includes Fennel as a
 library.
-.SH \f[V]assert-repl\f[R]
-.PP
-\f[I](Since 1.4.0)\f[R]
-.PP
-Sometimes you want to get a debug repl but only under certain
-conditions.
-The \f[V]assert-repl\f[R] macro is more or less a drop-in replacement
-for \f[V]assert\f[R], except instead of raising an error, it allows you
-to debug with a repl.
-It takes the same arguments as assert plus an optional opts table which
-is the same as \f[V]fennel.repl\f[R].
 .PP
 If you use the \f[V]--assert-as-repl\f[R] flag when running Fennel,
 calls to \f[V]assert\f[R] will be replaced with \f[V]assert-repl\f[R]

man/man7/fennel-tutorial.7

@@ -1,4 +1,4 @@
-.\" Automatically generated by Pandoc 3.1.2
+.\" Automatically generated by Pandoc 2.17.1.1
 .\"
 .\" Define V font for inline verbatim, using C font in formats
 .\" that render this, and otherwise B font.
@@ -14,7 +14,7 @@
 . ftr VB CB
 . ftr VBI CBI
 .\}
-.TH "fennel-tutorial" "7" "2023-09-25" "fennel 1.3.2-dev" "Getting Started with Fennel"
+.TH "fennel-tutorial" "7" "2023-12-01" "fennel 1.4.0" "Getting Started with Fennel"
 .hy
 .SH NAME
 .PP

release-checklist.md

@@ -7,10 +7,11 @@ This document is intended for Fennel maintainers.
 1. Make sure tests pass for all versions of Lua using `make ci`.
 2. Update and date the changelog.
 3. Update version number in `src/fennel/utils.fnl`.
-4. Check for changes which need to be mentioned in help text or man page.
-5. Update the download links in `setup.md`.
-6. Run `make rockspec VERSION=$VERSION`
-7. Run `git add . && git commit -m "Release $VERSION" && git tag -s $VERSION -m $VERSION`
+4. Run `make man`
+5. Check for changes which need to be mentioned in help text or man page.
+6. Update the download links in `setup.md`.
+7. Run `make rockspec VERSION=$VERSION`
+8. Run `git add . && git commit -m "Release $VERSION" && git tag -s $VERSION -m $VERSION`
 
 ## Uploading
 

rockspecs/fennel-1.4.0-1.rockspec

@@ -0,0 +1,8 @@
+package = "fennel"
+local fennel_version = "1.4.0"
+version = (fennel_version .. "-1")
+source = {url = ("https://fennel-lang.org/downloads/fennel-" .. fennel_version .. ".tar.gz")}
+description = {summary = "A lisp that compiles to Lua", detailed = ("Get your parens on--write macros and " .. "homoiconic code on the Lua runtime!"), license = "MIT", homepage = "https://fennel-lang.org/"}
+dependencies = {"lua >= 5.1"}
+build = {type = "builtin", install = {bin = {fennel = "fennel"}}, modules = {fennel = "fennel.lua"}}
+return nil

security.md

@@ -22,6 +22,22 @@ From 1.0 onwards, releases are also signed with `.sig` files using SSH keys:
 You can compare the key in the [allowed][5] file with the keys
 published at [technomancy.us][6], [SourceHut][7], or [GitHub][8].
 
+## Historical Issues
+
+In versions from 1.0.0 to 1.3.1, it was possible for code running in
+the compiler sandbox to call un-sandboxed functions from applications
+or Fennel libraries when running with metadata enabled. This could
+result in RCE when evaluating untrusted code in a way that relied on
+the sandbox for services running with metadata enabled.
+
+In addition, even when metadata was disabled, it was still possible
+for sandboxed code to trigger loading of a module already on the load
+path. In most cases if an attacker can get a file on the load-path
+then they've already won, but in the context of tools that run static
+analysis on untrusted code, this could result in a vulnerability.
+
+Versions prior to 1.0.0 did not sandbox macros.
+
 [1]: mailto:[email protected]
 [2]: mailto:[email protected]
 [3]: https://technomancy.us/8F2C85FFC1EBC016A3B683DE8BD38C28CCFD2DA6.txt

setup.md

@@ -37,12 +37,12 @@ installed on your system.
 This method requires you to manually update the `fennel` script when
 you want to use a newer version that has come out.
 
- 1. Download [the fennel script](https://fennel-lang.org/downloads/fennel-1.3.1)
- 2. Run `chmod +x fennel-1.3.1` to make it executable
- 3. Download [the signature](https://fennel-lang.org/downloads/fennel-1.3.1.asc)
- 4. Run `gpg --verify fennel-1.3.1.asc` to verify that the fennel
+ 1. Download [the fennel script](https://fennel-lang.org/downloads/fennel-1.4.0)
+ 2. Run `chmod +x fennel-1.4.0` to make it executable
+ 3. Download [the signature](https://fennel-lang.org/downloads/fennel-1.4.0.asc)
+ 4. Run `gpg --verify fennel-1.4.0.asc` to verify that the fennel
     script is from the Fennel creators (optional but recommended)
- 5. Move `fennel-1.3.1` to a directory on your `$PATH`, such as `/usr/local/bin`
+ 5. Move `fennel-1.4.0` to a directory on your `$PATH`, such as `/usr/local/bin`
 
 **Note**: You can rename the script to `fennel` for convenience. Or
 you can leave the version in the name, which makes it easy to keep
@@ -58,14 +58,14 @@ This method requires you to manually update the `fennel` binary when
 you want to use a newer version that has come out.
 
  1. Choose one the options below, depending on your system:
-      - [GNU/Linux x86_64](https://fennel-lang.org/downloads/fennel-1.3.1-x86_64)
-        ([signature](https://fennel-lang.org/downloads/fennel-1.3.1-x86_64.asc))
-      - [GNU/Linux arm32](https://fennel-lang.org/downloads/fennel-1.3.1-arm32)
-        ([signature](https://fennel-lang.org/downloads/fennel-1.3.1-arm32.asc))
-      - [Windows x86 32-bit](https://fennel-lang.org/downloads/fennel-1.3.1-windows32.exe)
-        ([signature](https://fennel-lang.org/downloads/fennel-1.3.1-windows32.exe.asc))
- 2. Run `chmod +x fennel-1.3.1*` to make it executable (not needed on Windows).
- 3. Download the signature and confirm it matches using `gpg --verify fennel-1.3.1*.asc`
+      - [GNU/Linux x86_64](https://fennel-lang.org/downloads/fennel-1.4.0-x86_64)
+        ([signature](https://fennel-lang.org/downloads/fennel-1.4.0-x86_64.asc))
+      - [GNU/Linux arm32](https://fennel-lang.org/downloads/fennel-1.4.0-arm32)
+        ([signature](https://fennel-lang.org/downloads/fennel-1.4.0-arm32.asc))
+      - [Windows x86 32-bit](https://fennel-lang.org/downloads/fennel-1.4.0-windows32.exe)
+        ([signature](https://fennel-lang.org/downloads/fennel-1.4.0-windows32.exe.asc))
+ 2. Run `chmod +x fennel-1.4.0*` to make it executable (not needed on Windows).
+ 3. Download the signature and confirm it matches using `gpg --verify fennel-1.4.0*.asc`
     (optional but recommended).
  4. Move the downloaded binary to a directory on your `$PATH`, such as `/usr/local/bin`
 
@@ -98,7 +98,7 @@ The Fennel compiler can be added to your code repository, and then
 loaded from Lua.
 
  1. Get the `fennel.lua` library. You can get this from a
-    [release tarball](https://fennel-lang.org/downloads/fennel-1.3.1.tar.gz)
+    [release tarball](https://fennel-lang.org/downloads/fennel-1.4.0.tar.gz)
     or by running `make` in a source checkout.
  2. Add `fennel.lua` to your code repository.
  3. Add the following lines to your Lua code:

src/fennel/utils.fnl

@@ -5,7 +5,7 @@
 
 (local view (require :fennel.view))
 
-(local version :1.4.0-dev)
+(local version :1.4.0)
 
 ;;; Lua VM detection helper functions