--- title: "Interface with R from Lua" output: rmarkdown::html_vignette vignette: > %\VignetteIndexEntry{Interface with R from Lua} %\VignetteEngine{knitr::rmarkdown} %\VignetteEncoding{UTF-8} --- ```{r, include = FALSE} knitr::opts_chunk$set( collapse = TRUE, comment = "#>" ) ``` Whenever the `luajr` R package opens a new Lua state, it opens all the [standard Lua libraries](https://www.lua.org/manual/5.1/manual.html#5) in that state, as well as the `luajr` module, which is designed to provide some key capabilities for your Lua code to interface well with R code. This vignette documents the `luajr` module. # Vector types The vector types are modelled after C++'s `std::vector` class. A vector type is what you end up with if you pass an argument into a Lua function using `lua_func()` with arg code `"v"`, but you can also create new vectors within Lua. Like their C++ counterparts, vector types all maintain an internal "capacity" which is equal to or greater than their "length", or actual number of elements. The exception is the character vector, which is implemented internally as a Lua table but has the same interface as the other vector types. Note that for vector types, indexes start at 1, not at 0. You must be very careful not to access or write out of these bounds, as the `luajr` module does **not** do bounds checking. Going out of bounds will cause crashes or other undefined behaviour. Unlike Lua tables, vector types can only be indexed with integers from 1 to the vector length, not with strings or any other types. ## Creating and testing vector types {#vcreate} **`luajr.logical(a, b)`, `luajr.integer(a, b)`, `luajr.numeric(a, b)`, `luajr.character(a, b)`** These functions can be used to create "vector types" in Lua code. The meaning of `a` and `b` depends on their type. For example: ```lua luajr.numeric() -- Empty numeric vector luajr.numeric(n, x) -- Size-n numeric vector, all entries equal to x luajr.numeric(v) -- Copied from vector v luajr.numeric(z) -- Copied from "vector-ish" object z ``` Above, `n` is a nonnegative number, `x` is a boolean, number, string (as appropriate for the vector type) or `nil`, `v` is another vector object of the same type, and `z` is a table, vector type, or reference type. If `x == nil` in the `(n, x)` form of the above functions, then the values of the vector are left uninitialized (except for `luajr.character`, where the values are set to the empty string). **`luajr.is_logical(obj)`, `luajr.is_integer(obj)`, `luajr.is_numeric(obj)`, `luajr.is_character(obj)`** Check whether a value `obj` is one of the corresponding vector types. These return `true` if `obj` is of the corresponding type, and `false` otherwise. ## Vector type methods All the vector types have the following methods: **`#v`** Returns the length of the vector. **`x = v[i]`, `v[i] = x`** Get or set the `i`th element of the vector. Again: Note that for vector types, indexes start at 1, not at 0. You must be very careful not to access or write out of these bounds, as the `luajr` module does **not** do bounds checking. Going out of bounds will cause crashes or other undefined behaviour. Unlike Lua tables, vector types can only be indexed with integers from 1 to the vector length, not with strings or any other types. **`pairs(v)`, `ipairs(v)`** Use with a Lua `for` loop to iterate over the values of the vector. For example: ```lua for i,x in pairs(v) do print(i, x) end ``` Note that for vector types, `pairs` and `ipairs` are interchangeable and the same loop above can be written as ```lua for i = 1, #v do print(i, v[i]) end ``` **`v:assign(a, b)`** Assign a new value to the vector; `a` and `b` have the same meaning as in the [vector type constructors](#vcreate). **`v:print()`** Print each value of the vector next to its index, each value on a new line. **`v:concat(sep)`** Returns a string comprised of the elements of the vector converted into strings and concatenated together with `sep` as a separator. If `sep` is missing, the default is `","`. **`v:debug_str()`** Returns a compact string representation of the vector, useful mainly for debugging. This contains the length of the vector, then the capacity of the vector, then each of the vector elements separated by commas. **`v:reserve(n)`** If `n` is larger than the vector's current capacity, suggests that the vector be enlarged to a capacity of `n`. Otherwise, does nothing. **`v:capacity()`** Returns the capacity of the vector. **`v:shrink_to_fit()`** If the vector's capacity is larger than its length, reallocates the vector so that its capacity is equal to its length. **`v:clear()`** Sets the size of the vector to 0. **`v:resize(n, val)`** Sets the size of the vector to `n`. If `n` is smaller than the vector's current length, removes elements at the end of the vector. If `n` is larger than the vector's current length, adds new elements at the end equal to `val`. `val` can be `nil` or missing. **`v:push_back(val)`** Adds `val` to the end of the vector. **`v:pop_back()`** Removes one element from the end of the vector. **`v:insert(i, a, b)`** Inserts new elements before position `i`, which must be between 1 and `#v + 1`. `a` and `b` have the same meaning as in the [vector type constructors](#vcreate). **`v:erase(first, last)`** Removes elements from position `first` to position `last`, inclusive (e.g. so that `v:erase(1, #v)` erases the whole vector). If `last` is `nil` or missing, just erases the single element at position `first`. # Reference types The reference types are similar to the vector types, but they are more "low-level" and hence are missing much of the functionality of the vector types. They map directly onto memory managed by R, so judicious use of reference types when passing to or returning from Lua can avoid unnecessary memory allocations and copies. Although reference types are missing lots of methods that the vector types have, they do have one feature that the vector types don't have, namely that you can set and get the attributes associated with the underlying R value. A reference type is what you end up with if you pass an argument into a Lua function using `lua_func()` with arg code `"r"`, but you can also create new reference types within Lua. As with the vector types, indexes start at 1, not at 0, and you must be very careful not to access or write out of these bounds. Finally, because reference types depend upon R, they are not completely thread safe and therefore they cannot safely be used with `lua_parallel()`. ## Creating and testing reference types **`luajr.logical_r(a, b)`, `luajr.integer_r(a, b)`, `luajr.numeric_r(a, b)`, `luajr.character_r(a, b)`** These functions can be used to create "vector types" in Lua code. The meaning of `a` and `b` depends on their type. Namely: ```lua luajr.numeric_r(n, x) -- Size n, all entries equal to x if x ~= nil luajr.numeric_r(z) -- Copied from "vector-ish" object z ``` Above, `n` is a nonnegative number, `x` is a boolean, number, or string (as appropriate for the vector type) or `nil`, and `z` is a table, vector type, or reference type. **`luajr.is_logical_r(obj)`, `luajr.is_integer_r(obj)`, `luajr.is_numeric_r(obj)`, `luajr.is_character_r(obj)`** Check whether a value `obj` is one of the corresponding reference types. These return `true` if `obj` is of the corresponding type, and `false` otherwise. ## Reference type methods All the reference types have the following methods: **`#v`** Returns the length of the referenced vector. **`x = v[i]`, `v[i] = x`** Get or set the `i`th element of the referenced vector. **`pairs(v)`, `ipairs(v)`** Use with a Lua `for` loop to iterate over the values of the vector. **`x = v(attr)`, `v(attr, x)`** Get or set the attribute named `attr`. The second form, with two arguments, sets the attribute to `x`. Note that even though you can get and manipulate the `names` attribute, you cannot access a reference vector's elements by their names. # List type A list is a special kind of Lua table that can be indexed either with positive integers or with strings. Unlike a Lua table, a list remembers the order in which you have added elements to it, like an R list does. ## Creating and testing lists **`luajr.list()`** Creates a new, empty list. **`luajr.is_list(obj)`** Check whether a value `obj` is a list. Returns `true` if `obj` is a list, and `false` otherwise. ## List methods Lists have the following methods: **`#v`** Returns the length of the list, including both integer- and string-keyed entries. This differs from the behaviour of the `#` operator on normal Lua tables, which only report the number of integer-keyed entries. **`x = v[i]`, `v[i] = x`** Get or set element `i` of the list. `i` can be either a positive integer or a string. Note that you cannot write to any integer keys greater than `#v + 1`. **`pairs(v)`, `ipairs(v)`** Use with a Lua `for` loop to iterate over the values of the vector. Unlike with vector and reference types, `pairs` and `ipairs` are not exactly the same when applied to lists; the former will provide either string or number keys as you iterate through the list (strings if they are present, numbers otherwise) and the latter will give number keys only. Either one iterates through every element of the list and both iterate through the list in the same order. **`x = v(attr)`, `v(attr, x)`** Get or set the attribute named `attr`. The second form, with two arguments, sets the attribute to `x`. Note that, for a list, the `"names"` attribute is not a simple vector of names, like in R, but is an associative array linking keys to their indices. For example, for a list with elements `a = 1`, `2`, and `c = 3`, `v("names")` is equal to `{ "a" = 1, "c" = 3 }`. However, when a list is returned to R, its `"names"` attribute has the normal R format. Note that lists have this interface for setting and getting attributes, but unlike the reference types (which also have this capability), they are not internally managed by R. This means that they are safe to use with `lua_parallel()`, as long as they don't contain any reference types. # Data frame and matrix types There are a handful of additional types based on the above types, but which have some special behaviour when returned to R. ## Data frame A data frame can be created with ```lua df = luajr.dataframe() ``` This is just a `luajr.list` with the `"class"` attribute set to `"data.frame"`. However, when a list with this class gets returned to R, it gets turned into a data frame. ## Matrix A matrix can be created with ```lua df = luajr.matrix_r(nrow, ncol) ``` This is a special kind of `luajr.numeric_r` with the `"dim"` attribute set to `luajr.integer_r({ nrow, ncol })`. It gets recognized as a matrix when returned to R. However, you can only access elements with a single index that starts at 1 and goes in column-major order. So, for example, for a 2x2 matrix, the top-left element has index 1, bottom-left has index 2, top-right has index 3 and bottom-right has index 4. Note that it is a reference type. ## Data matrix A data matrix can be created with ```lua dm = luajr.datamatrix_r(nrow, ncol, names) ``` This is similar to `luajr.matrix_r`, but it has `names` as column names. `names` can be passed in as a Lua table (e.g. `{ "foo", "bar" }`) or as a `luajr.character` or `luajr.character_r` object. Using a data matrix is very slightly faster than using a data frame because only one memory allocation needs to be made, but this difference is on the order of microseconds for normal sized data. Also, if you are just going to convert the returned matrix into a data frame anyway, you lose the speed advantage. So don't worry about it too much. The column names can be get and set within Lua using the special attribute name `"/matrix/colnames"`. # Constants The following constants are defined in the `luajr` table: ```lua luajr.TRUE luajr.FALSE luajr.NA_logical_ luajr.NA_integer_ luajr.NA_real_ luajr.NA_character_ luajr.NULL ``` Note that Lua's semantics in dealing with logical types and NA values are very different from R's semantics. Lua does not "understand" NA values in the same way that R does, and it sees the R logical type as fundamentally an integer, not a boolean. Specifically, whereas in R you could do the following: ```r x = c(TRUE, FALSE, NA) if (x[1]) print("First element of x is TRUE!") ``` in Lua you have to do this: ```lua x = luajr.logical({ luajr.TRUE, luajr.FALSE, luajr.NA_logical_ }) if x[1] == luajr.TRUE then print("First element of x is TRUE!") end ``` This is because under the hood, R's TRUE is defined as the integer 1 (though any nonzero integer besides NA will also test as TRUE), FALSE is defined as 0, and NA (when logical or integer) is defined as a special 'flag' value of -2147483648 (i.e. -2^31). However, in Lua, anything other than `nil` or `false` evaluates as `true`, meaning that the following Lua code ```lua x = luajr.logical({ luajr.TRUE, luajr.FALSE, luajr.NA_logical_ }) for i = 1, #x do if x[i] then print("Element", i, "of x is TRUE!") end end ``` will incorrectly claim that `luajr.TRUE`, `luajr.FALSE`, and `luajr.NA_logical_` are all "`TRUE`". So, instead, explicitly set logical values to either `luajr.TRUE`, or `luajr.FALSE`, or `luajr.NA_logical_`, and explicitly test them against the same values. Note that the different NA constants are not interchangeable. So, when testing for NA in Lua, you have to check if a value is equal to (`==`) or not equal to (`~=`) the corresponding NA constant above, depending on the type of the variable in question. All of this only applies to the vector and reference types described above. Lua also has its own boolean type (with possible values `true` and `false`) which will never compare equal with either `luajr.TRUE`, `luajr.FALSE` or any of the NA values. A Lua string can never compare equal with `luajr.NA_character_`, but conversely a Lua number may sometimes compare equal with `luajr.NA_logical_`, `luajr.NA_integer_`, or `luajr.NA_real_`, so do be careful not to mix Lua types and these R constants. Finally, `luajr.NULL` can be used to represent a `NULL` value, either on its own or as part of a table or `luajr.list` that gets returned to R. If you pass NULL in to Lua through arg code `"s"`, it will come out as `nil` in Lua; but if you use arg code `"r"` or `"v"`, it will come out as `luajr.NULL`.