Macros and syntax extensions in MetaLua

When it comes to macros there are at least two ways they can be implemented. The C/C++ way, which is a text processing approach or Lisp/Nemerle/MetaLua and others way in which macros are a natural property or an extension of the core parser itself. The former is significantly simpler to implement, the latter gives macros access to language-specific information like types, AST, etc.

I’d like to write a few more words about MetaLua. It extends the regular Lua syntax with -{ … } and +{ … } blocks. -{ … } on the topmost level means a block that will be executed at compile time. +{ … } can be used inside of -{ … } and is supposed to contain quoted code, that is the code for which an AST will be generated for further use during compile time. -{ … } can be used again inside of +{ … } to put in some expressions from the outer -{ … } scope which will be evaluated at compile time. This interleaving of blocks is called splicing. The nesting of these two types of blocks isn’t limited by any explicit rule, except perhaps some compiler constraints. Additionally MetaLua makes mlp ‘class’ available at compile time, mlp standing for MetaLua Parser. mlp allows for syntax extensions, e.g. mlp.lexer:add(“let”, “in”) registers two new keywords in the metalua lexer, while mlp.expr:add(“let”, mlp.id, “=”, mlp.expr, “in”, mlp.expr, builder=let_in_builder) adds a new syntax for MetaLua expression, consisting of ‘let’ keyword, followed by an identifier (mlp.id), assignment operator (=), an expression (mlp.expr, this is a recursive relationship), ‘in’ keyword and another expression (mlp.expr) with builder (i.e. AST generator for such syntax) specified as let_in_builder. Now let’s take another look at let_in_builder:

local function let_in_builder (x)
  local variable, value, expr = unpack (x)
  return +{
    function (-{variable})
      return -{expr}
    end (-{value})
  }
end

It’s pretty much a regular LUA function with some MetaLua syntax. It returns a +{ … } block, meaning an AST for the code inside the block. Now look at the code inside, it is a function definition and call. In pure Lua, you can do something like this: (function(x) return x*x; end) (2). The above is really similar, except that instead of x there is whatever the identifier after ‘let’ keyword is. Instead of 2 there is the the first expression of the ‘let’ syntax and instead of x*x there is the second expression from the ‘let’ syntax. In other words, the ‘let’ syntax transforms this:

let identifier = expr1 in expr2

into this:

function (identifier) return expr2; end (expr1)

In the quadratic equation example it transforms this:

let sqrt_delta = sqrt(b*b - 4*a*c) in [(-b - sqrt_delta) / (2*a), (-b + sqrt_delta) / (2*a)]

into this:

function (sqrt_delta)
  return [(-b - sqrt_delta) / (2*a), (-b + sqrt_delta) / (2*a)]
end (sqrt(b*b - 4*a*c))

The concept of quoting and splicing is fundamental to a flexible macro system. MetaLua can be used to create domain-specific scripting languages. It allows Lua to be extended with paradigms like object-oriented programming, effectively transforming it into quite a different language 🙂

I hope the entries in this cycle aren’t too chaotic. I’m writing them in what you might call ad hoc fashion at the moment but as soon as I put all the knowledge on the table I will write a summary article, probably a compilation of all the previous ones 😉