13 Jan 2013

Multithreading in emscripten with HTML5 WebWorkers

Multithreading in emscripten is different from what us C/C++ coders are used to. There is no concept of threads with shared memory state in Javascript, so emscripten can't simply offer a pthreads wrapper like NaCl does. Instead it uses HTML5 WebWorkers and a highlevel message-passing API to spread work across several CPU cores.

You basically pass a memory buffer over to the worker thread as input data, the worker thread does its processing and passes a memory buffer with the result data back to the main thread.

The downsides are (1) you can't simply port your existing multi-threaded code over to emscripten, (2) it is (currently) somewhat expensive to pass data around since it involves copying, and (3) you cannot express all multithreading patterns in emscripten. The upside is though, that it's really hard to shoot yourself in the foot, since there's no shared state, and all the multithreading primitives you love to hate (like mutexes, semaphores, cond-vars, atomic-ops) simply don't exist.

Let's have a quick look at emscripten's worker API, only 4 API-functions and 2 user-provided functions are necessary:

worker_handle emscripten_create_worker(const char* url);

This create a new worker object, it takes the URL of a separate emscripten-generated Javascript file.

The worker file must export at least one C-function (name doesn't matter, but the function name must be explicitely exported using emscripten's new "-s EXPORTED_FUNCTIONS" switch so that it isn't removed by dead-code elimination. The worker function prototype looks like this:

void dowork(char* data, int size);

The arguments define the location and size of the input data.

The function to invoke the worker is:

void emscripten_call_worker(worker_handle worker, const char *funcname, char *data, int size, void (*callback)(char *, int, void*), void *arg);

This takes the worker handle returned by emscripten_create_worker(), the name of the worker function (in our case "dowork"), a pointer to and size of the input data, a completion callback function pointer, and finally a custom argument which is passed through to the completion callback to associate the completion call with the invocation call.

At some point after emscripten_call_worker() is called, the dowork-function will be called in the worker thread with a data pointer and size. Since the worker has its own address space, the actual pointer value will be different from the pointer value in the emscripten_call_worker call of course.

The worker function now uses this input data to compute a result, and (optionally) hands this result back to the main thread using this function:

void emscripten_worker_respond(char* data, int size);

The return-data will be copied inside the function, so if the worker function had allocated a result buffer it remains the owner of that buffer and is responsible to release it.

Finally, once the worker has finished, the completion callback will be called on the main thread with the result data, and the custom arg given in the emscripten_call_worker() call:

void completion_callback(char* data, int size, void* arg);

The callee does not gain ownership of the data buffer, thus it must read / copy the received data but not write to, or free the buffer.

Finally there's a function to destroy a worker:

void emscripten_destroy_worker(worker_handle worker);

As with threads, creating and destroying workers is not cheap, so you should create a couple of workers at the start of the application and keep them around, instead of creating and destroying workers repeatedly. It's also wise to batch as much work as possible per worker invocation to offset the call-overhead as much as possible (don't call a worker many times per frames, ideally only once), but this is all pretty much common sense.

The worker Javascript file must be created as a separate compilation unit, it's a bit like on the PS3 where the SPU code also must be compiled into small, complete "SPU executables". To keep the code size small I decided to keep the runtime environment in the worker scripts as slim as possible, there's no complete Nebula3 environment, only a minimal C runtime environment. But this is not a limitation of emscripten, only a decision on my part. Most of the time the workers will contain simple math code which loops over arrays of data instead of high-level object-oriented code. To avoid downloading redundant code it might also make sense to put several worker functions into a single JS file.

The updated Nebula3/emscripten demos at http://n3emscripten.appspot.com now decompress the downloaded asset files in up to 4 WebWorker threads in parallel to the main thread, this speeds up asset loading tremendously and avoids the excessive frame hickups which happened before. This is important, since real-world Nebula3 apps stream asset data on demand while the render loop is running.  The whole stuff took me about half a day, but unfortunately I stumbled across a Chrome bug which required a small workaround (see here: http://code.google.com/p/chromium/issues/detail?id=169705).

It's not completely perfect yet. There's data copying happening on the main thread, and there's also some expensive stuff going on when creating the WebGL resources (for instance vertex and index data is unrolled for the instanced rendering hack). The ultimate goal is to move as much resource creation work off the main thread in order to guarantee smooth rendering while resources are created.

There are also browser improvements in sight which will make WebWorkers more efficient in the future, mainly to avoid extra data copies by transferring ownership of the passed data over to the web worker, basically a move instead of a copy.

And that's it for today :)



26 comments:

Unknown said...

Happy New Year Floh! Exiting stuff! :] I'm following very curious!

Unknown said...

The incoming branches (hopefully merged master soon) provide an additional worker response method with the same signature as `emscripten_worker_respond`: `emscripten_worker_respond_provisionally`. This new method allows the worker to continue taking messages until a finalizing call to `emscripten_worker_respond` gets made.

Z said...

Transferrable ArrayBuffers are now implemented (at least in Chrome), but they're not yet in emscripten (see https://github.com/kripken/emscripten/blob/06961a0ef6e3d8d92d5e36ff904262fefec62bec/src/library_browser.js for the current implementation).

emscripten is still passing the TypedArray, but only the ArrayBuffer (that the TypedArray wraps) is actually Transferrable.

Great post - thanks for the insights!

IT said...

Wow. This really made my day. Thanks a lot!

HTML5 Training in ChennaiHTML5 Training in Chennai JavaScript Training in Chennai JavaScript Training in Chennai

JavaScript Training Courses JavaScript Training Courses | Javascript Online Training Angular 2 Training in Chennai Angular 2 Training in Chennai

Alex said...

Excellent read, I just passed this onto a colleague who was doing a little research on that.
PDO Threads
thread lift
thread face lift
face thread lift
thread lift face
thread lift treatment
thread lift for face
thread lift procedure
non surgical thread face lift
skin threading face lift
neck thread lift
facelift with thread
thread neck lift
threading for face lift
fine thread lifting
thread lift neck
thread nose lift
non surgical facelift threading
thread lift surgery
polydioxanone thread lift
face lifting with threads
facelift by threading
non surgical thread lift
golden thread face lift
face lift thread facelift
nova lift face lift

Unknown said...

Well Said, you have furnished the right information that will be useful to anyone at all time. Thanks for sharing your Ideas.
Java training in Chennai

Java training in Bangalore

sathish said...

I appreciate that you produced this wonderful article to help us get more knowledge about this topic.
I know, it is not an easy task to write such a big article in one day, I've tried that and I've failed. But, here you are, trying the big task and finishing it off and getting good comments and ratings. That is one hell of a job done!


Selenium training in bangalore
Selenium training in Chennai
Selenium training in Bangalore
Selenium training in Pune
Selenium Online training

Jagadeesh said...

Thanks, for the blog.
Learn Digital Academy  offers, Digital Marketing Course in Bangalore.
intense In-class training program.

Customer relationship management said...

Your articles really impressed for me,because of all information so nice.mulesoft training in bangalore

Customer relationship management said...

Linking is very useful thing.you have really helped lots of people who visit blog and provide them use full information.salesforce developer training in bangalore

Customer relationship management said...

Being new to the blogging world I feel like there is still so much to learn. Your tips helped to clarify a few things for me as well as giving.salesforce admin training in bangalore

Customer relationship management said...

Really it was an awesome article,very interesting to read.You have provided an nice article,Thanks for sharing.servicenow training in bangalore

Customer relationship management said...

This is really an awesome post, thanks for it. Keep adding more information to this.cloud computing training in bangalore

watson said...

Thanks for sharing this blog. This very important and informative blog.hadoop training institutes in bangalore

Bangalore Training Academy said...

Your topic is very nice and helpful to us … Thank you for the information you wrote.

Bangalore Training Academy located in Bangalore, is one of the best Workday Training institute with 100% Placement support. Workday Training in Bangalore provided by Workday Certified Experts and real-time Working Professionals with handful years of experience in real time Workday Projects.

Softgen Infotech said...

Such a great information for blogger i am a professional blogger thanks…

Became An Expert In Selenium ! Learn from experienced Trainers and get the knowledge to crack a coding interview, @Softgen Infotech Located in BTM Layout.

Softgen Infotech said...

Post is very useful. Thank you this useful information.

Softgen Infotech have the best Python Training in Bangalore . Any professional who is looking out to switch their career can enroll with us.

eTechno Soft Solutions said...

wonderful thanks for sharing an amazing idea. keep it...

Looking for SAP FICO Training in Bangalore , learn from eTechno Soft Solutions SAP HR HCM Training on online training and classroom training. Join today!

watson said...

Great post!I am actually getting ready to across this information,i am very happy to this commands.Also great blog here with all of the valuable information you have.Well done,its a great knowledge.

sap abap training in bangalore

sap abap courses in bangalore

sap abap classes in bangalore

sap abap course syllabus

best sap abap training

sap abap training center

sap abap training institute in bangalore

watson said...

It is amazing and wonderful to visit your site.Thanks for sharing this information,this is useful

sap hana courses in bangalore

sap hana classes in bangalore

sap hana training institute in bangalore

sap hana course syllabus

best sap hana training

sap hana training centers

best sap hana training

watson said...

Great post!I am actually getting ready to across this information,i am very happy to this commands.Also great blog here with all of the valuable information you have.Well done,its a great knowledge.

sap hr courses in bangalore

sap hr classes in bangalore

sap hr training institute in bangalore

sap hr course syllabus

best sap hr training

sap hr training centers

sap hr training in bangalore

watson said...

Thank you for your post. This is excellent information. It is amazing and wonderful to visit your site.

sap mm training in bangalore

sap mm courses in bangalore

sap mm classes in bangalore

sap mm training institute in bangalore

sap mm course syllabus

best sap mm training

sap mm training centers

watson said...

Learned a lot of new things from your post! Good creation and HATS OFF to the creativity of your mind

sap fico training in bangalore

sap fico courses in bangalore

sap fico classes in bangalore

sap fico training institute in bangalore

sap fico course syllabus

best sap fico training

sap fico training centers

Cars2u said...

Hire the best self-drive cars in Coimbatore we are here our name Cars2u all brand cars and pick your favorite color cars2u is the best choice, No1. service providers, cheapest self-driving cars in Coimbatore visit our web page. choose your favorite cars all times available free doorstep delivery

Salman said...

Effective blog with a lot of information. I just Shared you the link below for Courses .They really provide good level of training and Placement,I just Had HTML Classes in this institute , Just Check This Link You can get it more information about the HTML course.


Java training in chennai | Java training in annanagar | Java training in omr | Java training in porur | Java training in tambaram | Java training in velachery

datasciencecourse said...

This is a wonderful article, Given so much info in it, These type of articles keeps the users interest in the website, and keep on sharing more ... good luck.

Digital marketing courses in Bangalore