Rendering images with Emscripten, WASM and OpenGL
March 7, 2018
TL;DR - The source for the demo application documented in this post is available on GitHub.
After joining Madefire in September of last year, one of my first goals was to get our Motion Book rendering engine – a C++ and OpenGL application – running on the web using WebAssembly. While I knew this could theoretically be accomplished by using Emscripten – particularly after hearing about how the team at Figma used it to compile their renderer – I'd never actually tried Emscripten myself.
The OpenGL Image Viewer
I find that reading through examples, then building my own practical mini-applications to be one of the best ways to get up to speed with a new technology. Early in my learning process I stumbled upon this excellent example on how to build a simple OpenGL-based web application using Emscripten, C++ and TypeScript. It demonstrates how one might use OpenGL and C++ to render a simple triangle in a browser canvas using a procedural texture. But it also goes beyond this, illustrating some of the compiler options needed to make C++ functions visible in the module interface, as well as some of the tools available for debugging (yes, debugging!) C++ code in the browser.
Passing Large Data Buffers into an Emscripten-compiled Module
The "proper" way to do things replicates how you would accomplish the same task in C or C++. Namely, allocate memory on the heap, fill the allocated space with the image data, and pass a pointer to that data to your function. If you have a typed array (say a
Uint8Array) with your image data (called
imageData), the boilerplate for such an operation might look as follows:
You can see this at work as part of my example application in
In Emscripten, the heap is just a giant
ArrayBuffer with various typed array "views" onto that buffer made available through the constants:
HEAPU16 and so forth and so on, up to 64 bits.
Here, we use
Module._malloc – similar to C's
malloc – to allocate space for 8-bit unsigned data on the array buffer. The "pointer" we get back from
Module._malloc is just an integer offset into the buffer. With it we can create a typed array backed by the heap at the offset, and fill it with the image data. From there we simply pass the "pointer" – aka the buffer offset – to our C function and do with it what we will. Finally, we clean up after ourselves using
Module._free (similar to C
If you don't need a new typed array view onto the heap data (and in this example we don't), the code above can be simplified to:
A Note About Performance
The use of
Unfortunately, at least for my example application, the
FileReader API does not allow you to read a file directly into a pre-allocated buffer. So, I have no choice, and must do the copy. But, if you have the opportunity to do so, it would be far better to write your data directly into the allocated space on the heap.
Alternatively, for the adventurous, Emscripten offers a split memory compilation mode that could theoretically allow you to use an existing buffer (via
allocateSplitChunk) as part of Emscripten's heap. Given the warnings and performance implications of this mode, I did not explore it here and, as they say, leave it up as an exercise to the reader.
Making Things Nicer with
free is less than ideal. Fortunately, you can hide these pesky implementation details.
One option is to use the
ccall to call the compiled C code. The
This code will get appended to your module during compilation and exposed as
someImageFunction directly on the module.
Given the code above, you might be wondering if the C function,
makefile in my example, you will see that the native
EXPORTED_FUNCTIONS all begin with an underscore. Note that setting the list of
EXPORTED_FUNCTIONS – which are specified as C
extern – is required, else the compiler will optimize them away if they aren't used by any C code.
Getting Tricky with Embind
If you're already using Embind to expose a richer C++ API, you can also use the
emscripten::val class. For instance:
The code above accomplishes the same task as before, namely fetching the length of the typed array, allocating space on the heap, creating a new typed array that is a "view" onto the heap memory, then copying the typed array onto the heap using
If this isn't enough, the folks at Figma have graciously provided a small library, IndirectBuffer, which allows for access of out-of-heap memory in Emscripten. I haven't yet tried this myself, but plan to do so very soon. It looks promising, particularly if you're trying to target a wide range of browsers on a wide range of platforms.
The API surface for the Emscripten run-time is rather large – covering everything everything from networking to filesystems to I/O to runloop considerations. This small example only scratches the surface of what's possible, and for a very limited use case, but hopefully some of you will have found it valuable as a means for getting started in porting – or using – C code on the web using WebAssembly.