Menu

Creating a Snapchat-like face filter 2/2: User interactions and particles

, , , , , , , , ,

Recap for the previous part: In the first part, we saw how to export a 3D model to JSON with Blender. We then added our hat to our Three.js scene, and thus became a true pirate of the Grand Line! Pretty neat, huh? In this tutorial, we’ll use a user interaction to trigger the addition of a magnificent bowtie to our filter. We’ll simultaneously add animated particles to create a cool apparition effect. Shazaa! Prerequisites: as for the previous tutorial, you will need to have good knowledge of Javascript and a good grasp of Three.js basics. Ready to build some kick-ass filter?! !

Creating our project folder

We start from the same point as with the first tutorial of this series. Let’s duplicate the cube folder, and rename it bowtie.

N.B: Running the FaceFilter API requires our assets to be served through a local HTTP server. In the command line from the root of our project folder, launch the httpServer.py script and visit localhost:8455 to check out the demos.

Creating our bowtie object

Okay, time to create our bowtie. First we search for a 3D model on the web, then we export it to JSON using Blender. The whole process is detailed in the first part of the tutorial.

Alright, now that we have our model in the right format. Let’s bring it to the code. In demo.js, we create a function named initListeners() to process user interactions. It will handle click events on desktop and touch events on mobile.

// here we initialize all the listeners for user interactions
function initListeners() {
  // your code goes here!
}

Then we create our bowtie mesh by loading our model, creating a material then mixing both into a mesh:

// here we initialize all the listeners for user interactions
function initListeners() {
  // we create our loader
  const loader = new THREE.BufferGeometryLoader()
  // we then load our model
  loader.load(
    'models/bowtie.json',
    function (geometry) {
      // then in the callback of our loader, we firstly create a material
      // we choose a MeshPhongMaterial, which will allow us to get a more realistic
      // result (see: https://threejs.org/docs/#api/materials/MeshPhongMaterial)
      const material = new THREE.MeshPhongMaterial({
      // here we load a texture for our model
        map: new THREE.TextureLoader().load('models/texture.jpg'),
        shininess: 3,
        specular: 0xffffff
      });
      // here we create our mesh with the geometry newly loaded and the material we just created
     const bowtieMesh = new THREE.Mesh(geometry, mat);
   )
}

Creating our particles

The next step consists in creating our particles object. The particles will spread in random directions to produce a cool magic-like apparition effect if the user clicked on the <canvas>. We got inspired by this THREE.js example. Overall, those examples or my go-to when I’m stuck somewhere in a three.js project as most of the basics, and even more complex features, are covered. We’ll create that kind of particles: Particles Beautiful, isn’t it? Let’s dive in!

// we create an empty array where we'll keep a reverence to our particles
// in order to animate them later
const PARTICLES = [];
// we instantiate our particle object 3D and set its “visible” property to false
// we'll be using this property to hide this object went necessary through our animation
const PARTICLESOBJ3D = new THREE.Object3D();
PARTICLESOBJ3D.visible = false;
// add the object to our face object 3D
THREEFACEOBJ3D.add(PARTICLESOBJ3D);
// create a material
// we'll be using the helper method generateSprite taken from the threejs example
const particleMaterial = new THREE.SpriteMaterial({
  map: new THREE.CanvasTexture(generateSprite()),
  blending: THREE.AdditiveBlending
});
// we then generate 200 particles that we'll show on click
for ( let i = 0; i < = 200; i++ ) {
  particle = new THREE.Sprite(particleMaterial);
  PARTICLES.push(particle);
  PARTICLESOBJ3D.add(particle);
}

I adapted a little the generateSprite() method, taken from the three.js example:

function generateSprite() {
  const canvas = document.createElement('canvas');
  canvas.width = 8;
  canvas.height = 8;
  const context = canvas.getContext('2d');
  const gradient = context.createRadialGradient(canvas.width / 2, canvas.height / 2, 0, canvas.width / 2, canvas.height / 2, canvas.width / 2);
  gradient.addColorStop(0, 'rgba(255,255,255,0.5)');
  gradient.addColorStop(0.2, 'rgba(255,255,255,0.5)');
  gradient.addColorStop(1, 'rgba(0,0,0,0.5)');
  context.fillStyle = gradient;
  context.fillRect(0, 0, canvas.width, canvas.height);
  return canvas;
}

Creating our event listener

We want to detect when a user clicks on the <canvas>. It will trigger the appearance of the bowtie at the position of the mouse pointer. It will also launch the particles animation. Let’s begin by creating an event listener. In our loader callback:

// We create a reference to our canvas and add our event listener
const canvas = document.getElementById('jeeFaceFilterCanvas');
canvas.addEventListener('click', (event) => {
  // we position our mesh by using our setPosition helper function
  bowtieMesh = setPosition(bowtieMesh, event);
  // we set .frustumCulled to false to render elements which have part of their body
  // out of the frustum
  bowtieMesh.frustumCulled = false;
  // set the renderOrder high to layer our bowtie on top of our video texture
  bowtieMesh.renderOrder = 100000;
  // we create a bowtie object 3D and add our mesh to it
  const BOWTIE = new THREE.Object3D();
  BOWTIE.add(bowtieMesh);
  // we then set our particle object3D visible property to “true” so that it gets rendered and use our setPosition helper to position the object
  PARTICLESOBJ3D.visible = true;
  setPosition(PARTICLESOBJ3D, event);
  // we then call the animateParticle method, that we'll create right after, on each element from our PARTICLES array
  // this method will handle the animation of our particles
  PARTICLES.forEach((particle, index) => {
    animateParticle(particle, 50*index);
  });
  // we add a setTimeout to set our particle object's visible property
  // to false after the animation is over
  setTimeout(function() {
     PARTICLESOBJ3D.visible = false;
  }, 50*PARTICLES.length);
  // we finally add our bowtie object
  THREEFACEOBJ3D.add(BOWTIE);
});

Phew! That was a big one but we are almost done here! We still have to add our helper methods setPosition() and animateParticles().

Creating our setPosition() helper

We paste setPosition() because the underlying concepts are out of the scope of this tutorial. It you want to learn more about it, you can browse this tutorial on WebGLAcademy.com. Here’s our function:

let MOUSEVECTOR = new THREE.Vector3()
let DIRECTIONVECTOR = new THREE.Vector3()
let VIEWPORTVECTOR = new THREE.Vector3()
// this function will position the bowtie mesh on our THREEFACEOBJ3D
function setBowtiePosition (mesh, event) {
  // we calculate the position of the mouse inside the canvas
  // you might have to fiddle with these depending on if your canvas is absolutely/normally positioned 
  const x = -(2 * (event.clientX — canvas.offsetLeft + canvas.offsetWidth / 2) / canvas.offsetWidth — 1);
  const y = -(2 * (event.clientY — canvas.offsetTop + canvas.offsetHeight / 2) / canvas.offsetHeight — 1);
  // this is used to find the point where to add the mesh on click
  MOUSEVECTOR.set(x, y);
  VIEWPORTVECTOR.set(x, y, 1);
  DIRECTIONVECTOR.copy(VIEWPORTVECTOR);
  DIRECTIONVECTOR.unproject(THREECAMERA);
  DIRECTIONVECTOR.sub(THREECAMERA.position);
  DIRECTIONVECTOR.normalize();
  // we calculate the coefficient that will allow us to find our mesh's position
  const k = _headCenterZ / DIRECTIONVECTOR.z;
  mesh.position.copy(DIRECTIONVECTOR).multiplyScalar(k);
  mesh.position.sub(THREEFACEOBJ3D.position);
}

Animating our particles with animateParticle()

We can position any element we add under the mouse pointer when the user clicks. Let’s create our animateParticle() function. It takes two arguments:

  • The particle mesh we want to animate,
  • The delay value is the time until the animation is completed. We use Tween.js to animate our particles. The mathematical logic involved in this animation is out of the scope of this tutorial. The best way to understand better what’s going on is playing with the parameters and testing again.
function animateParticle( particle, delay ) {
  //tween position :
  const radiusEnd = 100;
  particle.position.set(0,0,0)
  const theta = Math.random()*2*Math.PI; //angle in the plane XY
  const phi = (Math.random()*2–1)*Math.PI/4 //angle between plane XY and the particle. 0-> in the plane XY
  new TWEEN.Tween( particle.position )
    .to( {
      x: 0.008 * radiusEnd * Math.cos(theta) * Math.cos(phi),
      y: 0.008 * radiusEnd * Math.sin(theta) * Math.cos(phi),
    }, delay).start();
  //tween scale :
  particle.scale.x = particle.scale.y = Math.random() * 0.15
  new TWEEN.Tween( particle.scale )
    .to( {x: 0.0001, y: 0.0001}, delay)
    .start();
}

The last step is to add a line of code in our render loop to update our Tweens over time. In the main() function, in the callbackTrack():

if (ISDETECTED) {
  …
  // we add our line of code in the ISDETECTED if block
  TWEEN.update()
}

And here we are! A fabulous bowtie is added each time the user clicks on the <canvas> under the mouse pointer. A funny apparition effect produced by the animation of our particles is simultaneously triggered. How great is that?!

Exercises:

  1. Play a bit more with the particles: try making them fall from the sky like rain or make their colours change over time, change their movement
  2. Select and add to your project other 3D models that could be a good fit for virtual try-on: earrings, tee-shirts…

In conclusion

The second part of this series on creating Snapchat-like filters is completed! Yeaay!

Throughout this tutorial, we saw how to add an 3D object upon user interaction. We also saw how to create and animate particles to produce beautiful effect!

Related Posts