Irken vs Clojure

One way to understand a language is to compare it to its contemporaries. Since Clojure and Irken are both s-expression languages with compile time macros, it's a good comparison to make to learn something about Irken.

Before we get into the details, two of the biggest differences between the languages, and which we'd like to call out first, are..

• Clojure is mature, while Irken still in it's infancy with few programs written in it
• Clojure has extensive libraries and documentation, while Irken is sparsely documented with few libraries

In other words, Irken is very much still a research project, while Clojure has lots of 'real things' written in it. With that disclaimer aside...

Similarities

Both languages:

• use an s-expression syntax
• use Tracing Garbage Collection (though Irken's is much simpler and less performant)
• keep functions and data in the same namespace (like Scheme and Lisp-1)
• have compile time macros
• have Exceptions
• may be written with or without type specifications
• support functional and imperative programming (though Clojure prefers functional)
• support lightweight threading (though Clojure uses explicit decorators from core.async)

Differences

• Irken is statically type-checked at compile time, while Clojure is dynamically typed
• Irken is compiled to native code or run on it's own VM, while Clojure runs on the JVM
• Irken supports first-class continuations, while Clojure and the JVM do not
• Irken supports general and mutual tail-recursion optimization, while Clojure does not
• Irken supports exhaustive pattern match checking, while Clojure does not
• Irken does not allow null pointers, while Clojure has runtime null pointer exceptions
• Clojure supports runtime type casts, while Irken does not
• Clojure procedures directly support variable length argument lists and named arguments, while Irken can only express this in macros
• Clojure has a module and namespacing system, while Irken does not
• Clojure supports program transformation at runtime, while Irken does not
• Clojure has a REPL, while Irken (currently) does not
• Clojure has nice syntax for creating vectors, maps, and sets, while Irken does not
• Clojure has thread-local vars and STM datatypes for concurrency, Irken doesn't yet have a concurrency story

Lots of surface similarity

Because Irken and Clojure are both s-expression languages based on Lisp and Scheme, lots of code looks incredibly similar. As you consider the examples below, pay attention to how much similarity between the languages there is, given that Irken can type-infer and statically type-check programs at compile time.

;; Clojure
(defn add_one [x] (+ x 1))
(map add_one '(1 2 3))       ;; -> (2 3 4)
(sort < '(3 45 2 88))        ;; -> (2 3 45 88)

;; Irken
(define (add_one x) (+ x 1))
(map add_one '(1 2 3))       ;; -> (2 3 4)      : (list int)
(sort < '(3 45 2 88))        ;; -> (2 3 45 88)  : (list int)

Tuples

In Clojure, like Lisp, tuples are often represented as lists (or vectors). However, the semantics of tuples and lists are very different. Tuples are of fixed size where position has meaning, while lists are of variable size where position has no special meaning. Here are some tuples and lists in Clojure

;; Clojure
[x y z]                ;; a 3d coordinate tuple, as a vector
[username password]    ;; a username and password tuple, as a vector
(list 1 2 3 4 5 2 3 4)      ;; a list of numbers

In Irken, tuples and lists are represented by different data-structures. Lists are represented by list and are homomorphic, meaning they always contain elements of the same type. Obviously this makes them ill-suited to represent tuples, as tuple elements often vary in type. Instead tuples are represented by a data-structure called a variant, which may be either static or polymorphic. Static variants require type-declarations, while polymorphic variants may be used ad-hoc without declarations. Here we will consider only polymorphic variants, since they are closer to programming in Clojure. For more information, see Datatypes.

;; Irken
(:vec3d  x y z)              ;; a polymorphic variant tagged :vec3d
(:user   username password)  ;; a polymorphic variant tagged :user
(list 1 2 3 4 5 2 3 4)       ;; a list of numbers

Irken variants get their safety from type-inference and exhaustive pattern matching, which assures the type of a polymorphic variant is always the same along the same code path.

Pattern Matching

In Clojure we might destructure and print a 3d coordinate using pattern matching. We can call this function with any arguments, and only at runtime will we be given an error if it receives an argument which isn't a three tuple of floats. Clojure has no idea if that argument was intended to be a 3d coordinate or not.

;; Clojure
(defn print-vec [vec3d]
   (match [vec3d]
      [x y z] (print "(%f,%f,%f)\n" x y z)))
(print-vec [3 2 4])
(print-vec [3 2])   ;; runtime error

In Irken, pattern matching is type-inferred and statically checked at compile time, and tuples are represented using variants. If the shape of our pattern match isn't the same as the shape of our data, it's a compile time error. If we pass a variant :tag to this function that it's pattern matching doesn't handle, it's a compile time error. If the number and type of the arguments in our pattern match are not the same as when we packed our variant, it's a compile time error.

;; Irken
(define (print-vec vec3d)
  (match vec3d with
    (:vec3d x y z) -> (printf "(" (float x) "," (float y) "," (float z) ")\n")))
(print-vec (:vec3d 3 2 4))
(print-vec (:vec3d 2 1))    ;; compile time error
(print-vec (:other 1 2 3))  ;; compile time error

We might intend to use 2d vectors as well, in which case we can name them with their own variant type, and extend our pattern match. Irken also supports a compact syntax for omitting match and redundant listing of arguments in a function that starts with pattern matching:

;; Irken
(define print-vec
  (:vec3d x y z) -> (printf "vec3d (" (float x) "," (float y) "," (float z) ")\n")
  (:vec2d x y)   -> (printf "vec2d (" (float x) "," (float y) ")\n"))
(print-vec (:vec3d 3 2 4))
(print-vec (:vec2d 2 1))
(map print-vec (list (:vec3d 3 2 4) (:vec2d 2 1))) 

Clojure Maps vs Irken Records

Another way to represent aggregate data in Clojure is using Maps. Fields of a map are accessed by name, offering readability benefits for code. However, because Clojure is dynamic typed, attempts to access missing fields are not caught until runtime. This can make larger codebases fragile, and increase the number of tests you need to write and run to verify the code is working correctly after a change.

;; Clojure
(def my-data {:firstName "John" :lastName "Smith"} )
(defn my-function [person]
   (do
       (:firstName person)    ;; -> "John"
       (:last person)         ;; runtime error, missing :last in map
   ))
(my-function my-data)

Irken records are a static typed data structure akin to a type-inferred C-struct. While they don't need to be explicitly declared, their structure is known at compile time, allowing code to be typechecked. Also their in-memory representation is efficiently packed, similar to a C-struct.

;; Irken
(define my-data { firstName="John" lastName="Smith" } )
(define (my-function person)
   (begin 
       person.firstName ;; -> "John"
       person.last      ;; compile time error, "last" is not a field in record
   ))
(my-function my-data)

Macros

Both Clojure and Irken support creating macros with a pattern matching style and variable hygiene. Clojure macros are based on Lisp defmacro and are similar to Scheme syntax-case, which runs arbitrary code at compile time. Irken macros are based are based on Scheme syntax-rules which doesn't run arbitrary code, but instead performs pattern based substitutions at compile time. This makes Irken macros easier to write, but also less powerful. A future version of Irken could support syntax-case style macros.

Here is a simple Clojure macro which both prints the value of an expression and returns it. We place the result of the expression into result so it isn't evaluated twice. Because of the way Clojure macros are evaluated, we need to explicitly make lists for the output forms, and quote symbols we don't want defmacro to resolve.

;; clojure
(defmacro my-print
  [expression]
  (list 'let ['result expression]
        (list 'println 'result)
        'result))

Irken's macros are based on scheme syntax-rules, modified to fit Irken's pattern matching syntax. This style does not require the forms to be quoted in the result expression, making writing macros a bit simpler. You write the shape of the input form on the left, and the output form on the right.

;; Irken
(defmacro my-print
  (my-print expr) ->  (let ((result expr))
                        (printn result)
                         result))

Let's look at an example with multiple pattern cases. Here is an example of Clojure's or macro:

;; Clojure
(defmacro or
  ([] nil)
  ([x] x)
  ([x & next]
      `(let [or# ~x]
         (if or# or# (or ~@next)))))

Below we see the same macro in Irken. It's easy to see the similarities, but it's also easy to see some differences. Clojure uses or# for alpha renamed variables, while Irken uses $or. Clojure uses & next to match against the rest of a list in the source, and @next to place that pattern into the output, while Irken uses rest ... in both cases. Because the Clojure macro body is running code at compile time, returning code requires syntax quoting the output expression with back-tick, and then selectively un-quoting variables with tilde ~. Irken's result form is a pattern substitution which does not run at compile time.

;; Irken
(defmacro or
   (or ()) -> #f                        ;; false 
   (or x)  -> x 
   (or x rest ...) -> (let (($or x))
                        (if $or $or (or rest ...))))

For more detail, see Irken's macro documentation.

Variable Length Argument Lists, Named Arguments

Clojure procedures directly support variable length argument lists and optional named arguments. Of course with no compile time checking.

(defn my-fn-varargs [& rest] (str rest))
(defn my-fn-named [a & {:keys [optional] :or {optional "Missing"} }] (str a optional))

(my-fn-varargs "1" "2" "3" "4")             ; -> "1234"
(my-fn-named "1" )                      ; -> "1Missing"
(my-fn-named "1" :optional "Present")   ; -> "1Present"
(my-fn-named "1" :foobar "Blah")        ; runtime error

In Irken, variable length argument lists and optional named arguments are decoded with macros into statically typed function calls with fixed length argument lists. For example, a string-concatenation macro application can take any number of strings by composing occurrences of a two argument string-concat function.

(define (concat-2 a b) (format a b))
(defmacro concat
  (concat a) -> a
  (concat a b) -> (concat-2 a b)
  (concat a b c ...) -> (concat c ... (concat-2 a b)))
(concat "1" "2" "3" "4")
; (concat "1" "2" 3 4)      ; compile time error

Optional named arguments can also be decoded using macros. In the future, define could be extended to generate these definitions out of a simpler syntax.

The __rec__ parameter below is not hygienic. fixing this requires either this patch or this issue

(define (my-fn-inner a rest) (format a rest.optional))
(defmacro my-fn-args                                
  (my-fn-args <optional=> opt)     ->  (set! __rec__.optional opt)
  (my-fn-args name value rest ...) -> 
      (begin (my-fn-args name value) (my-fn-args rest ...)))
(defmacro my-fn
  (my-fn a rest ...) ->
     (let ((__rec__ {optional="Missing"}))
        (my-fn-args rest ...)
        (my-fn-inner a __rec__)))

(my-fn "1")                       ; "1Missing"
(my-fn "1" optional= "Present")   ; "1Present"
;(my-fn "1" foobar= "Blah")       ; compile time error

Tail Recursion

Irken supports automatic tail-recursion optimization. Any call in the tail position of a function is tail recursive. This allows mutual tail recursion.

;; automatic tail recursion
(define (some-join coll result)
  (if (= 1 (length coll)) (format result (car coll))
    (some-join (cdr coll) (format result (car coll) ", "))
  ))
(some-join (LIST "hello" "world" "love" "coding") "Words: ")
;; -> "Words: hello, world, love, coding"


;; automatic tail recursion and pattern matching
(define sum-up
    () result             -> result
    (first . rest) result -> (sum-up rest (+ result first)))
(sum-up (range 100000) 0)   
;; --> 4999950000

Clojure is limited by running on the JVM, which does not support tail recursion. As a result, clojure has an explicit tail recursion construct called recur. This allows Clojure programs to achieve optimized tail-recursion for single function recursion only. It can not support TCO on mutual tail recursion.

;; explicit tail recursion
(defn some-join [coll result]
        (if (= 1 (count coll)) (str result (first coll))
          (recur (rest coll) (str result (first coll) ", "))))
(some-join ["hello" "world" "love" "coding"] "Words: ")
;; -> "Words: hello, world, love, coding"

;; explicit tail recursion
(defn sum-up-with-recur [coll result]
        (if (empty? coll) result
          (recur (rest coll) (+ result (first coll)))))
(sum-up-with-recur (range 100000) 0)

Clojure Metadata

Clojure has a really great facility for attaching metadata to symbols and collections. One way this is used is docstrings. For example:

(def ^{:doc "Number of executions required"
       :dynamic true} *sample-count* 60)

Irken currently has no such facility.

TODO:

• add examples similar to Clojure JSON