In this article, we provide some context related to our project of virtual mirror for eyewear, highlight our main goals, share a demo, and finally mention some of the results we got…  

Why did Jeeliz try to fix the problem of glasses virtual try-on? Most of the current solutions of virtual try-on in augmented reality and real-time (virtual mirrors) were designed in the 2010’s. At that time, there was a lot of hype around Adobe Flash and consumers were not mobile-first…

Few people cared about interoperability and a trivial combination of desktop web app + native mobile app (for the most motivated users) was widely accepted. Deep-learning algorithms got improved since 2012 and now undoubtedly outperform Stasm-like algorithms for facial detection in real time in the browser. Deep-learning models are 100x faster than the previous the state-of-the-art and allow a real face detection on each frame of the webcam video feed. With these algorithms, no key-point tracking is required anymore (in most of the solutions, key-point tracking was involved to take over the real face detection on the video feed, to compensate a weakness, a lack of efficiency) … Users habits, when visiting a website with a huge catalog, changed drastically. Their level of expectation in term of UI/UX is much higher. And since Tinder released its innovative and famous swipe left/right, in a lot of cases swiping is the new browsing… GPU is improving like crazy! Using hardware accelerated workflows for deep-learning & real-time 3D renders of glasses in the browser is definitely a good bet.  

What did Jeeliz want to achieve in the optical e-commerce space? With this project, we wanted to:

  • Build a virtual mirror from a new technology stack, which is faster than the state-of-the-art and most of all a working solution supported by mobile browsers, obviously in real time.
  • Enhance the 3D renders of items to be tried on (here sunglasses) to improve the realism of the virtual trial.
  • Create a UX that matches the users’ new habits, and helps finding the right pair of glasses quickly.
Jeeliz virtual mirror as a web app powered by webGL and deep-learning technologies. Progressive web apps are the future; our demos were progressive from the beginning. Note: in 2018, major actors/tech companies confirmed it might be the good place to be in the future. See these recent and interesting articles: 

Apple goes back to the future with web apps, Progressive Web Apps are here. What’s the big deal?. See also which great companies  have already moved towards progressive web apps on the website PWA rocks. Augmented reality experiences are going to move from native apps to web apps (not convinced? See webAR on ARCore , webXR by Mozilla and WebXR: Reflecting on the Needs of Real AR on the Web). Thus, we found we had to control the whole workflow to perform great AR experiences in web environment. We’ve created for instance our own physically-based rendering engine to reconstruct the real lighting over the virtual items in a web context. As a side note, available libraries (like “three.js”) won’t do the job expected especially because rendering AR experiences for mobile on the web is very challenging (real-time processing with a very limited computational power). We have created a framework to build neural networks in javascript/webGL to perfectly fit mobile GPU capabilities (instead of trying to use GPU to compute oversized algorithms). And build a 100% GPU workflow taking advantage of 100% of the computational power available on a given device. We have created a new way to browse a catalog of eyeglasses without having to load more than one page (sure, it widely inspired by Tinder). We have also added a layer of price comparison and a doodle-like poll systems to get instant feedbacks from friends and family.

 

Some results of the commercial test of a sunglasses version of our webapp Jeeliz sunglasses is currently linked to Amazon, Flipkart and AliExpress through their APIs (a good way to have a huge offer for free…). It can be connected to any existing platform.  Over few weeks, with sponsored traffic mainly from Adwords it recorded over 3000+ unique visitors (at less than 20 cents per acquired visitor). Here is what we recorded in terms of:
  • engagement: 15 items tried out per session.
  • conversion: 15% of visitors redirected towards Amazon, 30% conversion on Amazon.  

Feel free to contact us for more information! The demo version of the virtual mirror for sunglasses is available on https://jeeliz.com/sunglasses/ We have also made public a virtual mirror widget easy to implement on a existing website. On GitHub: https://github.com/jeeliz/jeelizGlassesVTOWidget :

  • featuring: face detection, lighting reconstruction over virtual objects
  • suitable for any head accesories: glasses, helmet, scarfs…
  • tested on desktop and mobile (iPhoneX, S8, Sony Xperia, Huawei P9)