I spent couple days digging into web AR research due to some growing requests about making AR experience more accessible and sharable. The result isn't beyond my expectation, with the benefit of no-need-to-install separate AR app, it comes with the scarification of weaker power, instability and attachments with experimental browsers which allows to use the system level ARKit/Core functionality.
Maker based 3D model displaying using AR.js and THREE.js. I designed a marker using the tool with 3 different sizes, to see which one works better. I simply load and place a free car model using THREE.ColladaLoader and tested on 3 phones (Iphone 6S, iphone x, samsung galaxy s7 ). I updated the simple demo on github.
The recognization works the best with smaller size markers, even tracks the rotations of paper while the bigger marker is harder to contain because the whole marker has to be in the camera view to be recognized otherwise it will lose the model, different from Vuforia / image recognization when it would still work even if it detects only parts of the image. The model is laggy and pixelated in the mobile view with iphoneX. In 6s, it refuses to open due to "denied permission".
Couple great things:
tons of libraries made it easy to create 3D scenes without a deep knowledge of 3D development or native programming experiences with ARkit/ARCore/Game engines
most libraries are compatible with each other, so can easily integrated with other modern web components, frameworks and server APIs
easy to make sharable gaming / AR experience without app installation as long as it doesn't require heavy modeling and processing
requires low poly assets (wouldn't work well for displaying or interacting products with high details and textures )
user Interface design is very different from the standard web
most browsers and APIs are marked as experimental, work in progress and unstable
no native AR support unless installing external browser or use printed Marker
Libraries / Tools:
- JS Library: Marker-based web AR technology
- by detecting its position in the camera's video stream and laying 3D-models on top of it.
- extension for A-Frame and Three.js for marker-based AR scenes
- An emerging technology from Mozilla that allows to build web 3D/VR/AR experience with simply HTML and DOMs.
- an Entity-Component Framework that built on top of WebGL, Three.js, which provides a declarative, extensible, and composable structure to three.js
- it follows the Composition over inheritance principle that allows greater flexibility in defining entities where every object in a scene is an entity (e.g. enemies, bullets, vehicles, etc.).
- a growing ecosystem of plug ins
- use three.ar.js and aframe-ar as helper libraries for 3D contents, animations and effects
- browser support: WebARonARCore, WebARonARKit
- based on three.ar.js
- Basic A-Frame support for browser-based augmented reality (AR), supporting the new three.ar.js library
- use the power of ARCore (for Android) or ARKit (for IOS) both are native SDK’s
- work on WebARonARKit/Core browsers, as well as WebXR Viewer
- based on three.xr.js
- System & components to build WebXR experiences with A-frame
- complies with the proposed WebXR Device API using the webxr-polyfill
- allows you to move between AR and VR seamlessly
- X stands for: anything
- gateway which the devices use to communicate with the browser and give it information about their capabilities, so that users/devs are able to use this information and adjust the website to any device
- XR Session: a communication channel between the browser and the device
- Enable XR applications on the web by allowing pages to do the following:
- Detect available VR/AR devices.
- Query the devices capabilities.
- Poll the device’s position and orientation.
- Display imagery on the device at the appropriate frame rate.
- This API will take over the current WebVR API and expand it to even more devices
- an open-source library that we use to localize it through the phone camera.
Browsers that supports native AR powers and features from ARkit/Core but requires to install separately as a application to view AR contents
WebARonARCore: Experimental web browser by Google with the goal to bringing ARCore to Web
WebARonARKit: Similar as above, for iOS version
WebXR Viewer: Similar as above for Mozilla
An important feature of AR.js is the possibility to use custom markers, the default type is ‘pattern’. I did some research on pattern markers’ limitations, on their size and shape. I learned that:
The maximum resolution of a marker is 16x16 pixels
They must be square in shape
They cannot have white/transparent areas, only light grey (e.g. #F0F0F0)
They cannot contain colors, only black and light grey
They have to contain simple text, like one letter, a number, or a symbol.
You may use this online tool
cant do very complex sides and edges.
check Patt file, if too complex, the marker trainer just generated a bunch of 0's
have an high contrast between the ‘background’ of the marker and the surrounding environment
Low Poly Asset Libraries:
Google Poly: thousands of models for VR/AR use
Web AR/VR Resources