15703314754_07fb760bd4_smartphone-lens

Going Deep – the race for depth imaging smartphones is on

Our smartphones will soon be depth imaging devices, featuring cameras that measure depth information for each part of the image.  Lenovo already came out with the Google Tango-based Phab 2 phone last year and with the upcoming release of the iPhone 8 we’ll finally find out why Apple paid $350M to acquire the Israeli company PrimeSense back in 2013 (PrimeSense originally provided the technology behind the Microsoft Kinect game accessory).

The market for depth imaging and sensing devices is booming, according to the recently released Smartphone Depth Sensing report from Woodside Capital Partners and Yole Développement. With a CAGR of 37.7% over the next five years, the depth imaging market is expected to reach $9B by 2022.

Last week I spoke with the primary author of this report Rudy Burger, Managing Partner & Managing Director of Woodside and alumnus speaker at Mobile Photo Connect, to hear his perspectives on the depth imaging market.

Here is a synopsis of my takeaways from this interview, the report, and other sources:

What’s driving this market?  It’s primarily AR, the technology that gives you the illusion that virtual objects are placed in the physical world. The big players are all scrambling to deliver the default development environment for the next generation of AR apps and services. We’ll cover these in an upcoming publication, but for now it suffices to say that Google is driving this hard (Project Tango), and so are Snap (for internal use, that is), Facebook (AR Studio), Microsoft (Mixed Reality platform), Amazon (Echo Look) and, last but not least, Apple (ARKit).

Why do we need depth imaging when we can already enjoy AR on our phones with apps like Pokémon Go and Snapchat? Depth imaging phones will take us beyond today’s AR implementations, which basically detect flat planes to which virtual content is “anchored” (e.g. by showing the user a virtual framed photograph hanging on their living room wall when they view their wall through their smartphone camera). AR will become a lot more real when these smartphones can measure depth so that, for instance, a virtual assistant can walk you through the department store aisles towards the product that you’d like to buy.

Which types of depth imaging cameras are available today?

First, let’s start with a quick primer on the main depth image measuring technologies available today:

 [Image from Project Tango video]

  • Structured light: Structured light cameras send out infrared dot patterns to illuminate the contours of an object. It then measures the size of the dots: the larger the dots, the further away (that part of) the object.
  • Time-of-flight: Time-of flight cameras measure the time it takes for an infra-red beam to reach an object and reflect back to the source. The longer this takes, the farther away from the object.
  • Stereovision: Stereovision cameras use two sensors, placed about 2.5″ apart, to infer depth from triangulation, similar to how human do this with a set of eyes (Edwin Azzam, CTO of Stereolabs, one of the innovators in this space, discussed stereovision at Mobile Photo Connect in 2015). Stereovision does not use infrared and is well suited to outdoor environments. The iPhone 7 Plus offers some form of stereovision with its dual cameras (a telephoto and a standard camera positioned close together) that are capable of creating bokeh-like effects (“Portrait mode”).

Which depth imaging camera type is best suited for AR consumption on smartphones? While the dust hasn’t settled yet, Burger believes it is structured light cameras, which are uniquely suitable for 3D mapping. These cameras produce higher resolution imagery than time-of-flight cameras (although not as high as stereovision cameras), require less computational power than stereovision cameras, and perform better in low light settings than stereovision cameras. Infrared-based time-of-flight and structured light cameras fall short in sunny outdoor environments when measuring objects more than 50 meters away, but this is not a typical AR use case.

What will happen with the iPhone’s (stereovision-based) Portrait mode if Apple were to replace the current depth imaging approach with PrimeSense’s structured light solution? No worries – structured light cameras apparently are even better suited to produce high-quality depth of field effects.

So, stay tuned:

  • In the next few months leading up to the holiday season, we’ll hear quite a few depth imaging announcements from major smartphone vendors;
  • At Mobile Photo Connect we’ll hear from high-profile brands how they leverage AR to promote their brands;
  • We’re working on a publication that dives into how AR will let us augment and enrich visual memories.  Please contact us with any questions or suggestions.

A few more things…

Snap. When only two years ago it was Facebook that was declared “over and out” with reports of youngsters abandoning their service in droves, now it is Snap’s turn to feel the heat. After pricing its IPO at $17 and then reaching highs of $28, the company’s stock has fallen to less than half that. Snap closed Monday at $11. User growth is slower than projected, revenue targets are missed, but still, the average user creates 20 Snaps per day.  8 Ways to fix Snapchat include launching an augmented reality developer program rather than keeping their AR development platform proprietary and embracing brands as content creators.

PhotolemurPhotolemur 2.0: one-click photo enhancements without the one-size-fits all effects. Photolemur 2.0’s AI recognizes photo content, such as objects, faces, trees, and sky, and applies custom enhancements that are unique to a specific photo. Alas, no app yet – available only on Windows and Mac.

Framafoto. The new Framafoto app lets you interactively design your framed photos. Simply upload your photo(s), make a grid-collage if so desired, select your frame, preview your creation, and they then print, frame, and deliver the result to you.

Google. Anna Dickson, Photo Lead, Content and Community at Google, will join us in a special Fireside Chat session at Mobile Photo Connect.

Kwilt. Kwilt is joining Mobile Photo Connect as Gold sponsor. CEO Marc-Antoine Benglia will discuss his company’s next-generation photo/video storage solution in our “Photo and video storage solutions” session.

Author: Hans Hartman

Hans Hartman is president of Suite 48 Analytics, the leading research and analysis firm for the mobile photography market and organizer of Mobile Photo Connect, a yearly industry conference about mobile photography.

Share This!Share on FacebookTweet about this on TwitterShare on LinkedInShare on Google+Share on RedditBuffer this pageEmail this to someone

Leave a Reply