Loading an external image into Emscripten in ARGB

I have been working on an archery game and was investigating the ability to dynamically load images into the app. Most Emscripten apps are designed where the files are prepackaged into the game - a loading bar shows when starting the app and this loads all of the assets. I could have done it this way as well, but I wanted to explore if I could load an image externally and then bring it into emscripten.

The advantages are at least a couple:

• I don't need any image loading code - jpeg decoder libraries etc and the image will be loaded in ARGB32 format (Note, I am not using opengl either)
• I can have dynamic content loaded from another website or service.

So let's begin, if you haven't followed my other posts at this point I have a few handy things setup -

• I am working on an amazon ami, which builds and publishes to a web folder
• I wrote a program to combine html, javascript, into emscripten output so that I can run a build script and have all of these pieces combined into the output

The first thing to do is add the javascript and html to your page which will load the image:

<img id="funnyturtle" style="display:none" src="turtle.jpg"/>
<canvas id="imageloader" style="display:none;width:250px;height:200px"></canvas>

<script type='text/javascript'>
setTimeout(loadImage,2000);

var context;
function loadImage()
{
    var elem = document.getElementById('imageloader');
    if (elem && elem.getContext)
    {
        elem.setAttribute('width', '250px'); //###IMPORTANT
        elem.setAttribute('height', '200px');

        context = elem.getContext('2d');
        if (context)
        {
            var img=document.getElementById('funnyturtle');

            var width = 250;
            var height = 200;
           
            context.drawImage(img,0,0, img.width, img.height, 0, 0, width, height);

            var imgd = context.getImageData(0, 0, width, height);

            var numBytes = width * height * 4;
            var ptr= Module._malloc(numBytes);
            var heapBytes= new Uint8Array(Module.HEAPU8.buffer, ptr, numBytes);

            // copy data into heapBytes
            heapBytes.set(new Uint8Array(imgd.data));

            var imageloaded = Module.cwrap('imageloaded', 'number', ['number','number','number'])
            var n = imageloaded(heapBytes.byteOffset, width, height );

            Module._free(heapBytes.byteOffset);
        }
    }
}
</script>

What goes on here is 2 seconds after the page is loaded it will call loadImage(). 2 seconds is because for some reason the javascript for the Emscripten engine is not ready in the normal ready() from jquery. Once loadimage() gets called, we assume that the image is loaded, and basically draw the image to a hidden canvas element. From the hidden canvas we can get the ARGB32 values for the image and call imageLoaded() passing the image rgb values into our c++ code.

Now about the C++ code - one big gotcha is that when you are exporting functions via build options, remember that everything is lopped off unless you add it to your export list. I spent some time figuring out why Module._malloc did not exist and hopefully I can save you some time. So in your build script make sure you have something similar to this:

~/emscripten/emscripten/emcc main.cpp image.cpp game.cpp ShootingScreen.cpp drawing.cpp mainmenu.cpp -o /var/www/html/archery.html -s EXPORTED_FUNCTIONS="['_imageloaded','_main','_malloc','_free']"

From the above, you can see I exported _imageloaded and the other more standard functions. Here is the code in c++ that I have:

extern "C" int imageloaded( char * buffer, int width, int height )
{
    g_game.SetImageLoaded( buffer, width, height );
    return 0;
}

void CGame::SetImageLoaded( char * buffer, int width, int height )
{
    for ( int n = 0; n < IMAGE_CACHE_COUNT; n++ )
    {
        if ( m_images[n] == NULL )
        {
            m_images[n] = new CImage("mike.jpg", buffer, width, height, 200 );
        }
    }
}

CImage::CImage( const char * name, char * imageData, int width, int height, int alpha /*255*/ )
{
    m_imageData = (unsigned int*)malloc( width*height*4 );
    memcpy( m_imageData, imageData, width * height * 4 );

    m_name = strdup( name );
    m_width = width;
    m_height = height;
    m_alpha = alpha;
}

You can see that imageloaded gets called with my rgb32 buffer. I then call into my global game object, to save off the image. To be stupid simple I have an array of CImage* (I haven't gotten std::vector to work yet!). Each image object allocates the size of the image and then makes a copy.

Just for added kicks, here is my routine to draw the image.
void CDrawing::DrawImage( SURFACE & surface, CImage * image, int x, int y )
{
    if ( image->m_alpha == 255 )
    {
        int bufferOffset = x + y * surface.screenWidth;
       
        for ( int n = 0; n < image->m_height; n++ )
        {
            memcpy( surface.buffer + bufferOffset, image->m_imageData + n * image->m_width, image->m_width * 4 );
            bufferOffset += surface.screenWidth;
        }
    }
    else
    {
        unsigned int * src = image->m_imageData;
        unsigned int * dest = surface.buffer + x + y * surface.screenWidth;
       
        double alphaPercent = image->m_alpha / 255.0;
        double destAlpha = 1.0 - alphaPercent;
       
        for ( int n = 0; n < image->m_height; n++ )
        {
            for ( int a = 0; a < image->m_width; a++ )
            {
                *dest = ARGB(   0,
                                (int)(GET_RED(*src) * alphaPercent + GET_RED(*dest) * destAlpha),
                                (int)(GET_GREEN(*src) * alphaPercent + GET_GREEN(*dest) * destAlpha),
                                (int)(GET_BLUE(*src) * alphaPercent + GET_BLUE(*dest) * destAlpha) );
           
                src++;
                dest++;
            }
           
            dest += surface.screenWidth - image->m_width;
        }
    }
}

This just draws the image without stretching, nothing fancy. The alpha blending is split out as it should be slower! That's pretty much all there is too it.

How to modify the output html page to interact with Javascript - Emscripten candy!

I described in the previous post how to call your compiled code from Javascript. Well, that's cool and all, but... every time I build emscripten generates a new html file and wipes any editing I may wish to put inside the html file! To solve this I created a little utility which will combine javascript with your output html so that you can test the interaction between both in the html page. I just added the utility to the end of my build script and am set after that!

This is my build script at this point:

~/emscripten/emscripten/emcc main.cpp game.cpp ShootingScreen.cpp drawing.cpp mainmenu.cpp -o /var/www/html/archery.html -s EXPORTED_FUNCTIONS="['_imageloaded','_main']"

~/utils/htmljoin /var/www/html/archery.html javascript /var/www/html/archery2.html

The text in red, takes my emscripten output, archery.html and javascript and combines it into one file archery2.html. The contents in javascript file are inserted before the <body> tag.

My javascript file

This script code gets injected into the html output. Using this code I can call "imageloaded" which exists in my c++/c code. I found that you need to delay until you can call a function for things to load. My guess is that you could use ready() from jquery to know when you can call a c function. Also, adding jquery library links to this file might be a great idea!

Code to htmljoin utility

Save what is below to htmljoin.cpp and run "gcc  htmljoin.cpp -o htmljoin" to compile this code. It's that simple!

#include <cstdio>
#include <cstdlib>
#include <cstring>

int getFileSize( FILE * file )
{
    fseek (file , 0 , SEEK_END);
    int size = ftell (file);
    rewind (file);
    return size;
}

char * readContentsFromFile( char * filePath )
{
    FILE * file = fopen( filePath, "r" );
   
    if ( file == NULL )
    {
        printf( "ERROR: could not load file \"%s\"\n", filePath );
        return NULL;
    }

    int size = getFileSize( file );
   
    char * buffer = (char*) malloc (sizeof(char)*size + 1);   
   
    if (buffer == NULL)
    {
        printf("ERROR: error allocating memory for reading the file\n");
        fclose( file );
        return NULL;
    }

    printf( "reading %s file, %d bytes\n", filePath, size );
   
    // copy the file into the buffer:
    int result = fread( buffer, 1, size, file );
   
    buffer[size] = '\0';

    fclose( file );
   
    if (result != size)
    {
        printf("ERROR: error reading the contents of the file\n");
        free( buffer );
        return NULL;
    }

    return buffer;
}

void CreateJoinedFile( char * emscriptenHtmlFileData, char * joinHtmlFileData, FILE * outputHtmlFilePath )
{
    //find where body is in the string
    char * bodyLocation = strstr( emscriptenHtmlFileData, "<body>" );
   
    if ( bodyLocation == NULL )
    {
        printf( "ERROR: could not find the <body> tag\n" );
        return;
    }
   
    printf( "writing first piece of emscripten file...\n" );
   
    size_t toWrite = bodyLocation - emscriptenHtmlFileData;
    fwrite( emscriptenHtmlFileData, 1, toWrite, outputHtmlFilePath );

    printf( "joining files...\n" );
   
    //insert the code
    int lengthJoin = strlen( joinHtmlFileData );
    fwrite( joinHtmlFileData, 1, lengthJoin, outputHtmlFilePath );
   
    printf( "writing last piece of emscripten file...\n" );
   
    int bodyLength = strlen( bodyLocation );
    fwrite ( bodyLocation, 1, bodyLength, outputHtmlFilePath );   
}

int main( int argc, char *argv[] )
{
    char emscriptenHtmlFilePath[500], joinHtmlFilePath[500], outputHtmlFilePath[500];

    if ( argc < 4 )
    {
        printf( "you can run this program via command line with three arguments:\n" );
        printf( "arg 1 - emscripten html file\n");
        printf( "arg 2 - html file to join\n");
        printf( "arg 3 - output html file\n");
        printf( "please enter these arguments manually\n" );
        printf( "emscripten html file: " );
        gets( emscriptenHtmlFilePath );
        printf( "html file to join: " );
        gets( joinHtmlFilePath );
        printf( "output html file: " );
        gets( outputHtmlFilePath );
    }
    else
    {
        strcpy( emscriptenHtmlFilePath, argv[1] );
        strcpy( joinHtmlFilePath, argv[2] );
        strcpy( outputHtmlFilePath, argv[3] );
    }

    ///////////////////////////////////////
    // merge the files here
   
    char * emscriptenHtmlFileData = NULL, * joinHtmlFileData = NULL, * outputHtmlFileData = NULL;
    FILE * outputHtmlFile = NULL;
   
    emscriptenHtmlFileData = readContentsFromFile( emscriptenHtmlFilePath );
   
    if ( emscriptenHtmlFileData == NULL )
        goto cleanup;
       
    joinHtmlFileData = readContentsFromFile( joinHtmlFilePath );
   
    if ( joinHtmlFileData == NULL )
        goto cleanup;
       
    printf( "opening the file to write to...\n" );
   
    //open the file for writing
    outputHtmlFile = fopen( outputHtmlFilePath, "w+" );
   
    if ( outputHtmlFile == NULL )
    {
        printf( "ERROR: could not load file \"%s\"\n", outputHtmlFilePath );
        goto cleanup;
    }
   
    printf( "Creating joined file...\n" );
   
    CreateJoinedFile( emscriptenHtmlFileData, joinHtmlFileData, outputHtmlFile );
   
cleanup:

    if ( outputHtmlFile != NULL )
        fclose( outputHtmlFile );
       
    free( emscriptenHtmlFileData );
    free( joinHtmlFileData );
    free( outputHtmlFileData );
   
    return 0;
}

Calling a C function from Javascript in your Emscripten program

One thing you may want to do is pass dynamic content into your Emscripten program. This is possible! I'll show in a little bit.

A little background

I am in the process of writing a program in emscripten - this is a little different then what most people do, which is port an existing project into emscripten. When you port into emscripten, you can package all of your content (images, soundes, etc) into the build. This works out pretty well for most C programs as they tend not to interact with external pieces. Say you write a notepad app - it is just interacting with the file system or if you write an asteroid game, all of the content is static and doesn't rely on external dynamic content. For my program, I want to access an image that can change and isn't bundled with the app.

When emscripten was first released I could lazy load it. Lazy loading loads it byte per byte on demand from a location on the web. This is no longer supported though. For me lazy loading isn't really ideal either for a couple of reasons. First it would be syncronous which would stop other parts of my program from loading while it fetched a chunk of the image. Secondly I don't want to deal with writing code to decode an image! It's not 1995 all over again! I'd prefer the browser to decode the image and send me the data in rgb32. So my solution is to have javascript load the image, map it to a canvas object, then use the canvas to pass into my program the rgb32 array. Simple enough right!

Let's dig in and figure out how to call a C function from js, pass it some paramaters and have the program react to those arguements!

Calling a C function from Javascript

1) First, create a function in a .c file or tag a function with extern "C". This is what you will be allowing to be called from javascript. Extern "C" will prevent name mangling c++ does. All exported functions need to retain their original names to be called from javascript.

Here is a function I exported:

2) Second, you'll need to create some variable to interact that is global. For an archery game, I created a CGame object that is accessible everywhere. You may have a CApplication object. Whatever makes sense for you, but somehow you need to interact with the state of the app before you leave your C style function.

3) The build command must export the function. If you don't do this, the compiler will see it is not used and remove the function. When you export this function, the hidden gotcha is you need to export your main() function as well to continue to allow your program to work as it was before - that is assuming you have an application. Here is what I added to my build command to export main and imageloaded:


~/emscripten/emscripten/emcc main.cpp javascriptmapper.c game.cpp ShootingScreen.cpp drawing.cpp mainmenu.cpp -o /var/www/html/archery.html -s EXPORTED_FUNCTIONS="['_imageloaded','_main']"

Note! The magic emscripten is doing in the background is prefixing functions with an _ so they don't collide with other function names. That is why when you export them you add an _.

4) Call the function in javascript. The easy way to test this is load up your html program, then run the debugger in your browser using the f12 key. Go to the console, and run this javascript:

That's how you can call your function in javascript code. I changed imageloaded to return the integer 5 and you can see that result.

There is another way to call functions that is more performent when calling a function lots of times. I would check out the emscripten documentation for how to do this and also check out  the documentations for  how to send in parameters.

Here is a link:
https://github.com/kripken/emscripten/wiki/Interacting-with-code

Dynamic loading of content in Emscripten (Packaging files and lazy loading)

It appears Emscripten allows two types of loading or at least did. The first type is where you package files with your build command. The compiled code then shows a nice download progress bar. Once it's loaded then your program runs. This is the majority of how I have seen the apps run. The other method lazy loading no longer works. Lazy loading was intended to work by synchronously requesting chunks of bytes. Say you need X bytes of a file, lazy loading would request these bytes via http and then wait on the call until those bytes were returned. Just like reading from a hard drive so everything seems to work the same right? Firefox and Chrome disabled this synchronous call or supporting technology used behind the hood of the engine. So lazy loading is unlikely to work for you. My thoughts to get around this are to load the file via javascript and then use a hook from the C code to get at that data. I will report back if I am able to do this.

Getting started guide with Emscripten on an Amazon EC2 instance

You may find yourself in my same circumstance - wanting to try out Emscripten but not having an available linux box to install and run everything you need. So the shortcut to setting up a linux box is to use a cloud linux box. It's also a great opportunity to get more familar with amazon services. So to do this there are a lot of steps, that I've boiled down into different blog posts. This is the high level blog post which if you follow all of these steps you'll be up and running emscripten in short order.

Getting started using Emscripten

1) Launch a new Amazon Linux ami instance. 

http://faantasticcoder.blogspot.com/2013/05/creating-amazon-linux-ami-for-emscripten.html

2) Setup an Amazon EC2 machine to compile Emscripten

http://faantasticcoder.blogspot.com/2013/05/setting-up-amazon-ec2-machine-to.html

3) Get setup to write code for Emscripten on an Amazon EC2 machine

http://faantasticcoder.blogspot.com/2013/05/setting-up-to-write-code-for-emscripten.html

Setting up an Amazon EC2 machine to compile Emscripten

To use this tutorial, you should go to the amazon ec2 management console and create an instance of Amazon Linux ami. Make sure for the group policy of your ami, you allow ports 21, 22, 80, 3389, 8080. This will make your life easier as it allows you to setup a web service and view the emscripten output live. Also, you will most likely like to sftp into the site along with ssh.

Once your Amazon Linux ami is up and running, connect via ssh and let's begin setting up your machine to build with emscripten!

Get emscripten code

Install git with this command:
$ sudo yum install git

Get the source code for emscripten:
$ git clone git://github.com/kripken/emscripten.git

Get clang/llvm

Go to the webpage http://llvm.org/releases/download.html
You're linux box should be 64 bit so you will want the download for "Clang Binaries for Ubuntu-12.04/x86_64"
Download this file, then rename it to something simpler like llvm.tar.gz
Copy this to your amazon box user folder - I used winscp to copy it to where I wanted.

Now run the command to unzip the file:
$tar -xzvf llvm.tar.gz

Rename the folder it unzipped to "llvm" to make life easier. Again I used winscp to do the rename, use whatever is comfortable for you.

Create the config file for emscripten

Go to your emscripten folder
$cd emscripten

Run the command to build the config files
$./emcc

Fix the path to the llvm code you downloaded
$nano ~/.emscripten
In the file change the path to be something like:
LLVM_ROOT='home/ec2-user/llvm/bin'

Upgrade Glib
The image comes with 2.12. Emscripten needs 2.15!

Change directory so you are in your user folder.
To get the installer run:
$wget http://ftp.gnu.org/gnu/glibc/glibc-2.15.tar.gz

Unzip the installer and navigate to the folder:
$tar -xvzf glibc-2.15.tar.gz
$cd glibc-2.15

Make and navigate to a build folder
$mkdir glibc-build
$cd glibc-build

Create the config file, build and install
$../configure --prefix='/usr'
$make
$sudo make install

This process takes a while so get coffee after you run $make!

Install JS Node

Follow these instructions:
http://faantasticcoder.blogspot.com/2013/05/installing-nodejs-on-amazon-ec2-linux.html

Run emscripten

At this point emscripten is installed and you should be able to run emscripten.

Navigate to the emscripten folder under your user folder (where you ran git).

Run:
$cd emscripten
$./emcc




You should see the following output:

Setting up to write code for Emscripten on an Amazon EC2 machine

This is my recommendation on how to set up your environment to edit. Maybe not the best as I'm not a linux guru, but this gets the job done. You should have apache installed on the ec2 machine to get this to work.

Here is the process we are going to create:

• Edit the files in notepad++ and save.
• Alt-tab to putty ssh session. Run ./build.
• Go to a web browser, hit refresh on your output page and view the results.

Simple enough right?

Making a Build Script

Connect to your amazon EC2 machine with putty

Create a folder for your projects:
$ mkdir projects

Create a folder for your new project
$ mkdir archery

Create a build script
$ nano build

Enter something similar to:
~/emscripten/emscripten/emcc main.cpp -o /var/www/test.html

The first is a path to where your emscripten is installed respective to the root.
The last path is a public web folder so you can see the output.

Use Crtl-X to exit out nano and save the file.

Now you can run:
$./build
and your program will build.

Editing your files

Nano is not really my tool of choice. I'm familiar with notepad++ and that's the tool I use. So here goes:

Install notepad++ on your development machine (don't be a goof and try on the ec2 machine!)

Install the notepad++ plugin for NppFtp. Check out the notepad++ site for tips on doing this.

Once you have NppFtp installed it's time to connect!

Open notepad++ go to the menu plugins > nppftp > show nppftp window.

After you do this, you will now see the nppftp window.

Choose the option show

Setup a profile to connect - so you can connect easily each time!

1)

Click the add new button.
Name it something good, emscripten maybe..
Enter your ami url in the hostname box and select sftp.
NOTE!! you must have enabled port 22 on your group policy when you setup your ami machine
Fill in ec2-user
Wait you're not done yet, go to step 2 below.

2)

Click the authentication tab, select the location to your pem file for the ami. At some point hopefully you've created the pem file and kept it in a safe place for things like this!
Click the options shown in the picture below.
After you do this, hit the close button, it'll be saved don't worry!

3)

You're ready to connect!
Hit that connect button, select your profile.

After connecting in notepad++ you will be able to browse your folders to get to projects/archery/main.cpp. Double click to open and edit that file. When you hit save it will go right to your ec2 machine.

Viewing results in a web browser

Get your AMI url from EC2 management console. If you are not there google it, and then login!
Click on instances, to see the machine you are working on. The url is right there.

The combined url will be something like:
ec2-172-123-50-66.compute-1.amazonaws.com/test.html

Test it in your browser after you compile your program.

You must have apache installed on the ec2 machine for this to work!!

You're ready to write code!

Creating an Amazon Linux ami for Emscripten

Go to the amazon web console and navigate to EC2.

Click next 3 times on "Request instances wizard", until you get to a page with key/values.

 

The next screen will have a Launch button. Launch the machine!

After launching it may ask you to setup status alarms. Just hit close.

 

Next you'll want to connect to your AMI with ssh and with a file explorer. ->

http://faantasticcoder.blogspot.com/2013/05/connecting-to-amazon-linux-ami-with-ssh.html

 

 

 

Installing Node.JS on an Amazon EC2 linux box

Open putty and connect to your EC2 machine.
Pro tip: keep putty in a dropbox folder so you can easily get to your ec2 machine where ever you may choose to work.

Get the source code from GitHub:
$ git clone git://github.com/joyent/node.git

Move to the node folder after it downloads:
$ cd node
$ ./configure
$ ./make

Go get coffee....

$./sudo make install

It's installed at this point.

Connecting to Amazon Linux AMI with ssh and winscp

Connecting via SSH

Download putty and puttygen. Highly recommended - put these files in a dropbox folder.

http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html

Create a PPK file that putty will use to connect

Run puttygen to create the ppk file.

Click okay to the informational box.
Next click "save private key" and click yes to saving without a private key.
Save the ppk file to a dropbox folder so it is handy for you from all locations!

Login to ssh using putty

Installing apache on your EC2 instance

This is very simple!

Installing apache

Open up putty and ssh into your ec2 machine.

Run this:
$ sudo yum -y install httpd

Start the service -
$ sudo service httpd start

You should see a status message that says OK.

Next, navigate in the browser to your EC2 machine.

To get that url go to your ec2 management console, and copy the url shown below.

You should see a generic page shown. If you tried to use this url before you'd be staring at a 404 page!

Uploading web pages

The http web folder has a path of /var/www/html
By default you cannot write to this location!

To take ownership of this folder, connect with putty ssh. And run this command:

$sudo chown -R ec2-user /var/www/html 

After you take ownership of this folder, then you can copy files to this location. I recommend using winSCP to copy files.

Files you copy to this location, will show up using the ami url and the filename. Slick eh?