Create 3D animated emoticons with Jeeliz Weboji
With Jeeliz Weboji library, you can build and embed 3D animated emoticons in your web app. It is fully based on JavaScript and WebGL. With this library, Anyone can create Animoji-like experiences working into their web browser.
Under the hood, the library uses our custom Javascript deep learning framework. It has been trained to detect facial expressions. Then they are reproduced on an animated 3D model. Any head 3D model can be used with our library with a few adjustments and the creation of a bunch of morphs. A morph is the original mesh which has been slightly deformed in order to match a particular expression, for example mouth opening.
Screenshot of the Fox Weboji demonstration, included in the library Github repository. You can test it here: jeeliz.com/demos/weboji/demos/threejs/raccoon. In this tutorial we explain how to replace the Fox mesh by your own 3D model.
When it is done, the Jeeliz Weboji meshConverter puts the base model and the morphs together to build a usable Weboji usable by the API. Let’s see together how to get this done.
Preparing the 3D model
The 3D model should ideally be a head. Although it is possible to add a neck, the more your model is high poly, the more costly it will be to be rendered and animated.
The Weboji API uses a collection of 10 morphs to reproduce the face expressions. To create those 10 morphs, we create ten additional versions of your base 3D model. Each version is modified to follow the instructions below, regarding the expressions bound to each morph index:
Morph 0: smileRight → closed mouth smile rightMorph 1: smileLeft → closed mouth smile leftMorph 2: eyeBrowLeftDown → eyebrow left frownedMorph 3: eyeBrowRightDown → eyebrow right frownedMorph 4: eyeBrowLeftUp → eyebrow left up (surprised)Morph 5: eyeBrowRightUp → eyebrow right up (surprised)Morph 6: mouthOpen → mouth openMorph 7: mouthRound → mouth roundMorph 8: eyeRightClose → close right eyeMorph 9: eyeLeftClose → close left eyeMorph 10: mouthNasty → mouth nasty (upper lip raised)
As we can see below, we have readied the morphs and the base model for my fox weboji:
<
p style=”text-align: center”>
On the Github repository, we can find the base morphs of the fox here: meshConverter/meshes/Fox_V2. We can now create our weboji by using the meshConverter.
Using the meshConverter
The meshConverter is included in the Github repository for the Jeeliz Weboji. In the created folder we’ll find a subfolder “meshConverter”. We can open it and copy/paste the folder where you stored you morphs and base 3D model in this meshConverter folder.
As we can see in the documentation for the meshConverter, the next step is to open our terminal, change directories to yourJeelizWebojiFolder/meshConverter and type in:
python buildJson.py <name_of_your_3dmodel_folder>
The meshConverter should bundle our morphs and our base 3D model into a JSON file readable by the Jeeliz Weboji API. Our newly created JSON file should be in the ../assets/3D/meshes/ folder, named name_of_your_3dmodel_folder.json.
Wrap up
Congrats! You have created your first Weboji! You can now use it in your web app with the Jeeliz Weboji API. You will find the documentation on how to use the API on the Github repo. Stay tuned for an upcoming tutorial on this topic. And give us the link to your creations, we love to communicate and to share applications and demos using our libraries!
The source code of the meshConverter is very readable, with some 3D programming skills you can adapt it to your own 3D format or tweak it to your own workflow. The THREE.JS middleware importing the generated JSON file is also included in the Weboji repository, so you can change it easily.