New AI imaging products, research papers and scandals keep popping up – so, I thought it was a good idea to present a roundup of the AI imaging announcements of the last two weeks. Needless to say, there was a lot of commotion around FaceApp but, before we get into that, let’s start with the other AI imaging news.
UC Berkeley, the Univ. of Washington, and the Univ. of Chicago: Natural adversarial images. Pictures that are routinely misidentified by AI are known as “adversarial images.” These pictures can be artificially created to trick (and teach) image recognition systems, but they also exist in the wild. Researchers from three universities created a dataset of some 7,500 “natural adversarial examples.” They tested a number of machine vision systems on this data and found that their accuracy dropped by as much as 90 percent, with the software only able to identify just two or three percent of images in some cases. More actionably, they also identified a range of possible causes for the mistakes. Things ain’t exactly perfect yet!
Megvii: Facial recognition for pets. China-based Megvii Technology, a company that raised $750M back in May while planning for an IPO and widely known for its Face++ face recognition software also used by Chinese law enforcement agencies, now offers face recognition for pets. Megvii’s app can register your dog simply by scanning the snout through your phone’s camera. Just like how a phone registers your fingerprint for biometric unlocks, the app asks you to take photos of your dog’s nose from multiple angles. Megvii says it has an accuracy rate of 95% and has reunited 15,000 pets with their owners through the app. But here comes the creepy part: using Megvii’s close relationships with the Chinese government, the Megvii app can also be used to monitor “uncivilized dog keeping” to fine civilians who don’t pick up after their dogs or allow them to walk without leases in public spaces.
Runway ML: AI Imaging development for non-developers. A new program called Runway ML aims to make AI development easier by providing artists, designers, filmmakers, and others with an “app store” of machine learning applications that can be activated with a few clicks, while providing an easy interface for creating AI applications. Not a new idea, but Runway stands out by targeting the creative market. Want to create a motion capture, masking or style transfer module? Do it all without coding, according to the company.
Wyze: Face recognition built into super cheap security cameras. Seattle-based Wyze is the larger of the smaller smart security camera vendors. Wyze is known for selling its security cams ridiculously cheap, starting at $19.99. What was missing? People detection – and that’s what Wyze just added through an over-the-air firmware update with technology provided by Xnor.ai. When the camera detects a person, Wyze Cam can push an optional notification to the Wyze app, delivering real-time actionable alerts to users.
Apple FaceTime: Faking eye contact. A new update included in Apple’s iOS 13 gives iPhone X users’ gaze a “nudge” to achieve eye contact. The adjustment will give people the illusion of being directly looked at by callers. Fun fact: it appears the effect is being achieved using ARKit, which is used to map a user’s face and adjust the positioning of their eyes accordingly.
Google SMILY: Reverse image search for medical diagnoses. AI-based medical visual tech is booming, as was apparent at the LDVision conference I attended a few months ago. Solutions are progressing rapidly beyond simply recognizing patterns in X-rays, MRIs, or pictures of skin tissues. A just-announced case in point: SMILY (Similar Medical Images Like Yours), a solution described in a research paper to which Google researchers contributed. SMILY is a sort of heavily augmented reverse image search built specifically for tissue inspection and cancer diagnosis.
UC San Diego & Google: Relighting images. A group of researchers and engineers from UC San Diego and Google have submitted a paper for Siggraph describing that they have trained a neural network to “relight” portraits after the fact, such that their system can take any photo and adjust the lighting at will—including the direction, temperature, and quality of the light.
[Much more about computational photography in our Fireside Chat session at Visual 1st with Alexander Schiffhauer, Product Lead, Computational Photography at Google. Much more about the evolution of lighting solutions in our “The camera is dead – Long live the camera” session at Visual 1st, featuring Erik Bjernulf, VP Product Management at Profoto.]
City of Oakland: Banning the police from using facial recognition. Oakland becomes the third U.S. city to ban police use of facial recognition, after San Francisco and Somerville, Mass. banned the technology in May and June respectively. Why? According to the city ordinance, facial recognition systems “rely on biased datasets with high levels of inaccuracy,” there is a “lack of standards around the use and sharing of this technology” as well as its “invasive nature.” The ordinance also raises the issue of “potential abuses of data by our government that could lead to persecution of minority groups.”
GitHub: Banning DeepNude code. Code hosting site GitHub is banning code on its site from DeepNude, the self-proclaimed “AI X-Ray App,” an app that used AI to create fake nude pictures of women. After much protest, the app was shut down, but apparently, copies of the app were still accessible online, including on GitHub.
FaceApp: Fun & Panic galore. Since the days of Prisma, the app that introduced the benefits of transfer styles to a larger audience by enabling users to “make your photo look as if Picasso, Munch, or even Salvador Dali himself painted it for you,” the media haven’t paid so much attention to AI imaging as they have in recent days when covering Russia-based FaceApp. Even politicians got into the fray with the Democratic National Committee warning its presidential candidates not to use the app and Senate Minority Leader Chuck Schumer calling on the FBI and FTC to investigate the app developer.
For those of you who were happily off the grid last week while enjoying the summer, here is a summary, as short as I can make it:
FaceApp is a vastly popular app (as of this writing the most downloaded iOS and Android app in the US; average user ratings of 4.8 out of 5 on iOS and 4.5 on Android) that lets users have fun by uploading their photos and then easily adding a smile, making the person look younger or older, or swapping gender. Like many AI-based apps, it does the image processing in the cloud, i.e. the user needs to upload their photos.
Photo credit. Fun fact: not only the elderly versions of the same person in this photo are fake, but the younger originals are also fake! (They were created by thispersondoesnotexist.com, using NVIDIA’s StyleGAN)
What’s so bad about FaceApp?
So, it all comes down to trust or weighing the advantages of a fun app to possible misuse of your photo. Given the app’s download successes, consumers have made their choice – at least, for now.
And a few more things…
Google. Google introduces Gallery Go, a photo gallery designed to work offline, using machine learning to automatically organize and enhances one’s photos. It appears to target smartphone users in low-bandwidth, i.e. developing, countries.
Skylum. The sky is the limit: Skylum announces Luminar 4, the latest version of its desktop photo editing program, which includes AI Sky Replacement, a feature that allows users to automatically replace skies in a photo-realistic fashion. [Skylum CEO Alex Tsepko will speak in our Visual 1st “AI and Imaging” panel].
Square. Payment processing provider Square is getting into photography – by eliminating the photographer. Square launches a robotic Photo Studio service for online sellers. For $9.95, any online merchant – not just those that use Square’s suite of products – can send in a product and have it robotically photographed, and Square will send back a pack of three high-res, multi-angle digital images. Sellers can also purchase a spinning, interactive animation for $29.95. Why do we care? Well, a vendor with, give or take, 2 Million business customers is entering the imaging fray.
photobook.ai & Sarah.ai. photobook.ai, has released its first white-label app for the Japanese pocket photobook consumer market. Sarah.ai, from Cheetah, automatically analyzes photos taken on mobile phones to create stories. It does this by curating the photoset using machine learning computer vision algorithms to remove duplicates, to find the best photos, and to discard the bad ones. The app then creates a 24-page 14x14cm square photobook. [photobook.ai is a Silver sponsor of our Visual 1st conference]
DJI. DJI announces the Ronin-S Compact (SC), a single-handed 3-axis gimbal designed for popular mirrorless cameras. Using over a decade of experience creating aerial and handheld gimbals, the Ronin-SC offers a highly compact design, high-grade materials, and advanced technology for users capturing videos and photographs.
SIGMA. Going small: SIGMA announces the SIGMA fp, the world’s smallest and lightest mirrorless digital camera with a full-frame image sensor. The SIGMA fp incorporates a 35mm full-frame Bayer sensor with 24.6 effective megapixels in a compact 112.6×69.9×45.3mm (4.4”x2.7”x1.8”) body. No release date or price announced as of yet.
Author: Hans Hartman
Hans Hartman is president of Suite 48 Analytics, the leading research and analysis firm for the mobile photography market and organizer of Mobile Visual 1st, a yearly industry conference about mobile photography.