Introduction

After writing, a time ago, my article about Embedding Lua in C, I decided to follow up with a more real-world example than exposing simple functions and variables.

This is the first part of a series of articles where I write an entire application that is closest to real-world development. The application is not explained entirely here, but it is used as a reference. You may look at the complete source code in the repository to have the full view.

I started writing for myself during my journey of learning how Lua works and using it for some of my projects. Any doubts or comments are welcome.

The Lua role in the application

Today we have an HTTP server in C that supports Lua for a web application. The server will handle simple HTTP requests and return responses from the web application to the client.

The server uses socket, signalfd, and epoll. The latter helps to handle connections and system signals asynchronously. I will not cover those topics in this article, I may do them in another one in the future. How I create the HTTP requests and responses is not explained either. If you are interested, take a look at the project. It is really straightforward.

The idea is to use a Lua script to handle our requests and produce for us the response content to send back to the client connection. It will only support simple HTTP requests.

We are going to send all requests, independent of path or method, to a single function called index. It will receive a table that contains the path, method, and headers that were requested.

Some conventions

I do not comment on using the lua_State as an argument in the functions. The functions that interact with the stack, registry, and state require it, usually we named just L.

Handling the client connection

We are going to jump straight to a single client connection, all the socket handling will not be discussed here.

After accepting the connection from the socket, we start the client_worker function that will handle a single connection for us. It starts reading from the socket the necessary data to create the Request structure. Next, lua_State is initialized with the custom function lua_init:

1
2
3
4
5

void client_worker(Client *client) {
    Request *r = http_read_request(client);
    LOG_INFO("Request: %s %s", r->method, r->path);

    lua_State *L = lua_init();

Initializing Lua State

The function lua_init is just a shortcut to call luaL_newstate and luaL_openlibs from Lua API:

1
2
3
4
5

lua_State *lua_init() {
   lua_State *L = luaL_newstate();
   luaL_openlibs(L);
   return L;
}

With the new state, we are ready to use Lua.

The goal is to invoke the function called index inside the default script(app.lua). It will be the entry point of our application. It will receive a single argument, the Request, and it will return two values: the status code, and a string with the content to be written into the socket as a response to the client.

Loading Lua script

Before calling the index function, we need to load its definition into the Registry Table. Evaluating the script will register the functions definitions and global variables. It can be done with the macro luaL_dofile:

• luaL_dofile is a macro that does the same as luaL_loadfile and lua_pcall.
• luaL_loadfile loads Lua code chunks from a file and lua_pcall execute those chunks in protected mode.

1
2

lua_State *L = lua_init();
if (luaL_dofile(L, SCRIPT_FILENAME) == LUA_OK) {

luaL_dofile requires the path to the Lua script and returns the result of its execution. We check if the result of the operation is LUA_OK. Otherwise, we have to treat the error.

Getting index function reference

1
2
3

if (luaL_dofile(L, SCRIPT_FILENAME) == LUA_OK) {
        lua_getglobal(L, SCRIPT_ENTRYPOINT);
        request_to_lua_arg(L, r);

After loading the Lua function reference into the Registry Table, we push its reference to the Virtual Stack using lua_getglobal. lua_pcall requires the function reference and all the arguments to be in the stack before invoking the function.

Preparing the Request argument

We want to pass the Request to the Lua function, to do this, we have to push the structure into the Virtual Stack to be used as the argument to the function index.

This structure will be converted to a table:

1
2
3
4
5
6
7

typedef struct Request {
    char method[8];                   // GET, POST, ...
    char path[HTTP_PATH_SIZE];        // Eg: /, /about
    uint32_t header_count;            // How many headers in the request
    KeyValue *headers[HEADER_COUNT];  // Headers
    size_t size;                      // Request size
} Request;

We have a custom function request_to_lua_arg which does that:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21

void request_to_lua_arg(lua_State *L, Request *r) {
   lua_newtable(L);                  // creates a new table
   lua_pushstring(L, r->method);     // pushes new value to field
   lua_setfield(L, 2, "method");     // creates a new field
   lua_pushstring(L, r->path);       // another value
   lua_setfield(L, 2, "path");       // new field


   lua_newtable(L); // new nested table

   // fill the nested table with one field per header   
   for (size_t h = 0; h < r->header_count; h++) {
       KeyValue *kv = r->headers[h];
       lua_pushstring(L, kv->value);
       lua_setfield(L, 3, kv->key);
   }


   // set the header table as a new field to request table
   lua_setfield(L, 2, "headers");
}

The function request_to_lua_arg does an important step, it copies the data from the Request structure that contains the fields method, path, and headers to a new Lua table. Let’s see how the routine works.

Creating a table

To create a new table and fill it with fields, we have to pay attention to the API call order, because the operations depend on the Virtual Stack and the order in which values are pushed on it before setting the fields.

We start calling lua_newtable to create an empty table and put it on top of the stack:

1

lua_newtable(L);

With the new table on top of the stack, we can call a sequence of steps to create new fields.

Creating new fields

To create new fields we use lua_setfield, which has to know which table to create a new field, which name identifies the field, and which value will be bound to such field.

The name is one of the arguments of the function. The value has to be on top of the stack, and then the function will pop it. And after popping out the value, the table must take its place on top of the stack, so the function will have access to it and know that it is the table to set the new field.

Another example

Let's leave the HTTP server implementation on the side for a moment and check another example below.

With a table on the stack, a new field can be created by pushing a value into the stack, it can be of any type, including another table:

1

lua_pushstring(L,"Luke");

And calling lua_setfield that will use the value on the stack to create a new field in the table:

1

lua_setfield(L, 1, "name");

If the type that you want to add as a field is another table(nested table), you need to set its fields before setting it as a field in the parent table.

Calling the index function

Let’s go back to the server implementation.

Now that we have the request argument on top of the stack and the function reference, we call it with lua_pcall. It will execute the function in protected mode.

1

int lua_pcall (lua_State *L, int nargs, int nresults, int msgh);

lua_pcall has the following arguments:

• nargs inform the number of values to be popped from the stack and used as arguments for the function call
• nresults tells how many values the Lua function will return. It can be set to LUA_MULTRET to return an arbitrary number of results.
• msgh is the stack index for a custom error handler(not covered here)

1
2
3

request_to_lua_arg(L, r);
if (lua_pcall(L, 1, LUA_MULTRET, 0) == LUA_OK) {
   Response *resp = response_from_lua(L);

We pass the request argument as a table, but It is possible to create a more complex structure as we are going to see in the following articles.

We also check if the function lua_pcall returns LUA_OK to continue for a call that worked.

After calling the index function, its returned values are put on the stack.

Getting the returned values

The values returned will be retrieved from the stack. Checking their types are also important to make sure that they are the expected ones.

• Status code, an integer
• Response Body, a string

Preparing the Response

The custom function response_from_lua is in charge of retrieving the data from the stack and creating a new structure called Response:

1

Response *resp = response_from_lua(L);

Inside the function, we just use the macros luaL_checkinteger and luaL_checklstring. It lookups at the values from the stack and checks their respective types.

Note about luaL_checklstring, it returns the size of the string if the size_t *l argument is not NULL. There are also some caveats about the string in memory, check the manual.

1
2
3
4
5
6
7
8
9

Response *response_from_lua(lua_State *L) {
    // first returned value
    lua_Integer status = luaL_checkinteger(L, 1);
    size_t body_sz;

    // second returned value
    const char *body = luaL_checklstring(L, 2, &body_sz);
    return response_html_new(status, body, body_sz);
}

The luaL_check* macros require the int arg argument which is the index of the arguments in the stack.

This is important because the values are lookup without being popped. The order is the same as the return order in the Lua function.

With the response, we just need to write to the socket with an HTTP Response containing the status code and the body.

And that is it, the main use of Lua. But before we finish, let's take a look at error handling and some notes.

Error Handling

When a lua_pcall-like function does not return LUA_OK, and there is no custom error handling configured, the error will be put on the top of the stack. The common approach is to pop the error from the stack and take an action for it. Our lua_log_error function does this:

1
2
3
4

void lua_log_error(lua_State *L) {
   LOG_ERR("lua: %s", lua_tostring(L, lua_gettop(L)));
   lua_pop(L, lua_gettop(L));
}

It gets the error from the top of the stack with lua_gettop and converts the error to a string with lua_tostring. After using the error, we pop it from the stack.

Notes

• When using the macro luaL_checklstring:
  • it uses internally lua_tolstring, which converter strings and numbers to strings and any other type returns a NULL.
  • the strings are a reference to the string in the stack, make sure to copy it before it gets garbage collected by Lua.
  • numbers are converted in place to string
• When calling a function, make sure to pass the correct number of arguments. And if you aren't sure about the number of results, use LUA_MULTRET

Conclusion

This concludes how I use Lua inside this simple HTTP Server. I hope it was clear and well explained enough to be useful for you in your journey to use Lua. As you can see, using Lua is a good starting point for learning about embedding script languages into an application.

There is not much more to be explained about Lua for this application. It is the core use of Lua inside the HTTP server. The code has more routines to handle sockets, signals, epoll events, and memory allocation.

Check the repository for the complete project and be ready for the following articles I will explain other topics.

Thank you!

Resources

• HTTP Server - Repository Reference for this article
• Lua 5.3 Manual
• Linux: socket
• Linux: signalfd
• Linux: epoll
• Embedding Lua in C