Safe and Efficient Lua Integration

Lua is a popular scripting language in the games industry.  It has an easy to use C API, a decently fast virtual machine, and as fully featured language environment.

Unfortunately, as with all sufficiently advanced technologies, there are some things to watch out for when integrating Lua into your own game engine.  The trickiest to deal with properly is handling safe access to your game objects from Lua.  Lua implements its own garbage collector which doesn't know anything about the life cycle management of your game objects, and its not hard at all to end up in a situation where Lua tries to access an object you've already deallocated in your game engine proper.

A related but quite different issue with Lua integration is the exact reverse of the previous problem: how to ensure your game engine isn't accessing Lua objects that have already been collected by the Lua engine.

Let's start out with an example of basic (but unsafe) Lua integration.

Example 1 - Basic Lua Integration

[cc lang="C++"] // global Lua state lua_State* L;

// methods for the GameObject class static int gameobjectgetid(luaState* s) {
GameObject* self = (GameObject*)luaLcheckudata(s, 1, "GameObject"); luapushinteger(s, self->getID()); return 1; };

static luaLReg gameobjectmethods[] = {
{ "getID", gameobject_getid }, { NULL, NULL } };

// initialize Lua for the engine void initlua() {
L = lua
newstate(); luaL_openlibs(L);

// create metatable luaLnewmetatable(L, "GameObject"); luapushstring(L, "_index"); luapushvalue(L, -2); lua_settable(L, -3);

// register mtehods luaLregister(L, NULL, gameobjectmethods); }

// push a GameObject onto the Lua stack void pushgameobject(GameObject* go) {
GameObject** udata = (GameObject**)lua
newuserdata(L, sizeof(GameObject*)); *udata = go; luaL_getmetatable(L, "GameObject");

lua_setmetatable(L, -2); } [/cc]

Pretty straightforward. We define a single method for our GameObject class to get the object's unique identifier. We initialize Lua by defining a unique meta table for our GameObject class, and we have a method to push GameObject instances into the Lua stack as a userdata with the GameObject metatable.

By far the most serious problem with this setup is that the gameobject_getid() method wrapper is entirely unsafe. Let's say a developer writes a script that receives a GameObject that represents an enemy unit and stores it in a table somewhere. In a few frames, the enemy is killed by the player, and the game engine deallocates the GameObject representing the enemy. At this point, there is now a Lua userdata that has a dangling pointer to the old GameObject. If any script tries to call the getID method on the userdata, it's going to dereference that pointer and crash the game.

This is by no means a new problem for game developers. The issues surrounding storing references to game objects are pretty numerous and generally well understood. One popular solution -- the use of unique game object handlers -- can be easily applied to Lua userdata.

Example 2 - Using Entity Handles

[cc lang="C++"] // getID method using entity handle static int gameobjectgetid(luaState* s) {
GameObjectID id = (GameObjectID)luaLcheckudata(s, 1, "GameObject"); GameObject* self = LookupGameObjectByID(id); if (self != 0) { luapushinteger(s, self->getID()); return 1; } else { return 0; } }

// push a GameObject onto the stack void pushgameobject(GameObject* go) {
GameObjectID* udata = (GameObjectID*)lua
newuserdata(L, sizeof(GameObjectID)); *udata = go->setID(); luaLgetmetatable(L, "GameObject"); luasetmetatable(L, -2); } [/cc]

Our new version does not store a direct pointer in the Lua userdata. If the GameObject is deallocated, the getID method will fail to look up the object and return nil. (You could argue that its more efficient to just return the GameObjectID directly instead of looking up the object for this particular methods; it's just an example.)

There is still a second issue, however, which may frustrate some developers until it is solved. Our current code creates a new Lua userdata every time a GameObject is pushed onto the stack. This means that a developer cannot attach Lua data to the GameObject in any useful way, as there is no way to attach a table to a GameObject. It can also feel a little wasteful to generate a new userdata every time a GameObject is to be pushed onto the stack, although in practice the overhead is pretty minimal.

We would like to be able to attach arbitrary Lua data to our GameObject userdatas so that we can extend the behavior of GameObjects purely using Lua code. This means that we are going to need an efficient way to ensure that there is one and only one userdata associated with each GameObject. However, that alone isnt going to solve our problems! Look at the case where a script receives a GameObject userdata, attaches some additional data to it, and then finishes. Besides the Lua userdata itself is not stored anywhere by the script, the userdata will be garbage collected, and any data associated with it is also lost. We're going to need an additional mechanism to ensure that our GameObject userdata is not garbage collected until the GameObject is destroyed in the engine.

Fortunately, there's a pretty simple way to accomplish both goals. Lua offers a special table called the registry. The registry allows C/C++ to store objects that will not be garbage collected. All we need to do is store our GameObject userdata into the registry using a key that uniquely identifies our object and we're set. Best of all, Lua includes a function for doing this very thing.

The function we're interested in is luaLref(). This function will create a unique integer key for the object at the top of the Lua stack and store it in a specified table. In our case, we're going to use the registry table. So long as we don't create our own integer keys in the registry, the luaLref() function will guarantee that unique keys are generated.

Now that we have the means store a unique userdata for every game object, we actually need a means to associate the userdata with a GameObject instance. The easiest approach here is to simply add an int member variable to the GameObject class in which to store the result of luaLref(). The member should be initialized to the value LUANOREF when the GameObject is initialized.

The first time the GameObject is pushed, we can create our userdata, add a reference to it in the registry, and store the reference identifier in our GameObject. The next time the GameObject is pushed and we see that the registry identifier is not equal to LUANOREF, we can simple get the userdata back from the registry using luagettable(). The only thing left to do is to ensure that the userdata is removed from the registry when the game object is destroyed so that Lua can garbage collect the userdata.

Example 3 - Unique Userdata in Registry
[cc lang="C++"] // push a GameObject onto the stack void pushgameobject(GameObject* go) {
int ref = go->GetLuaRef(); if (ref != LUA
NOREF) { luapushinteger(L, ref); luagettable(L, LUAREGISTRYINDEX); } else { GameObjectID* udata = (GameObjectID*)luanewuserdata(L, sizeof(GameObjectID)); *udata = go->setID(); luaLgetmetatable(L, "GameObject"); luasetmetatable(L, -2);

// generate the registry reference
lua_push(L, -1);
ref = luaL_ref(L, LUA_REGISTRYINDEX);
go->SetLuaRef(ref);

}

// call this when the GameObject is destroyed by the engine void freegameobject(GameObject* go) {
if (go->GetLuaRef() != LUA
NOREF) { luapushinteger(L, go->GetLuaRef()); luapushnil(L); luasettable(L, LUAREGISTRYINDEX); go->SetLuaRef(LUA_NOREF); } } [/cc]

There we go. We've now got a safe userdata wrapper for our engine's game objects, each game object has only a single userdata wrapper, and the userdata will not be collected by Lua's garbage collector until the game object is destroyed by the engine.

I will leave as an exercise to the reader how one can obtain similar behavior for objects that cannot be modified to store an integer for the reference identifier. (Hint: remember that you can create a lightuserdata to wrap simple pointers, and that lightuserdata can be used as a key into a table.)