There is more to the world around us than we can see. Much more. While we compensate for that lack of vision with our other senses, like touch or smell, machines do have this luxury. Thermal sensors, who can capture and render heat signatures, fill in this void and combined with advance imaging, deliver far deeper insight on our environment, even to humans. We caught up with Travis Merill, CMO of FLIR, a global leader in thermal imaging, who reveals how critical this technology has become and how it will very soon end up in our cell phones. Travis will be speaking at the fast approaching LDV Vision Summit.
– A little about you, what is your background?
Originally hailing from the great state of Indiana, I spent the early part of my career in the broadband and mobile communications industries in both large company and start-up environments in operations, marketing, and strategy functions. After attending grad school, I joined Samsung for an eight-year stint in a variety of strategy and marketing roles both in Korea and the US. My final role there was launching and running the GALAXY Tab business in the US market for three years before I moved to FLIR Systems as its Chief Marketing Officer in 2014. I lead our global marketing organization to drive awareness for not only the FLIR brand but also for thermal imaging technology in general. I now live in Portland, Oregon, where FLIR is based. Portland is my sixth home city in the past 15 years, so I’ve been a bit of a nomad.
– Tell us more about FLIR. What does the company do?
FLIR is a global leader in thermal imaging and other sensing technologies. The company has been around since the late 1970s, but there are parts of the company that dates back considerably earlier than that. We are NASDAQ-traded with around $1.6B in annual revenue. Originally focused on airborne thermal sensor systems for military and law enforcement aircraft, FLIR has subsequently expanded to leverage thermal imaging technology into a wide variety of commercial and industrial markets like building inspection, oil and gas inspection, factory automation, firefighting, home and professional security, outdoor recreation, recreational boating, among numerous others.
FLIR is vertically integrated so we are not only thermal camera leaders in many of these markets, but we also lead in the development, manufacturing, and selling of the main component of a thermal imager – the camera core module. Most recently, with the launch of our Lepton thermal micro-camera core – approximately ten times smaller and less costly than anything before it – we’ve taken steps into mass markets to deliver the wonder of this powerful technology to more users, including consumers. We position ourselves as “The World’s Sixth Sense” because we bring a new way for people to see and experience the world around them and we are passionate about our mission of leveraging our technologies to save lives, enhance productivity, and protect the environment.
– How important is thermal imaging for managing our world?
It’s critically important, as the ability to see and manage thermal energy is becoming more important every day. Every object in the universe emits or reflects thermal radiation. There is an entire world of infrared temperature signatures that we cannot see with the naked eye that can tell us a lot about the environment around us. Thermal imaging allows us to visualize that world and offers a number of powerful differentiators as compared to conventional visible imaging. These include: the ability to see in total darkness, see through obscurants like smoke or haze, measure temperature differences that could alert us to an unseen safety or equipment failure issue, discern humans or animals from inanimate objects, and visualize gasses that are invisible to the naked eye. Having those capabilities is like having a superpower.
It’s used by firefighters to see through smoke, militaries and coast guards for search & rescue in total darkness, electricians to pinpoint hidden problems in homes or buildings, engineers to evaluate product performance, airports and critical infrastructure for perimeter security, energy companies to identify gas leaks, cities for smarter traffic management, and everyday people for outdoor recreation and DIY home projects. The list goes on and on, though it has only been in the last couple of years that the cost of the technology has reached a point to make it much more widely accessible.
– What are the challenges in recognizing content with thermal cameras?
Thermal imaging allows you to visualize and measure the surface temperate of most objects. As a result, things look very different with a thermal camera. Hot objects appear lighter and cool objects appear darker. Shadows and colors of an object typically don’t appear in a thermal image the way they do in the visible spectrum. A red object and a blue object would look the same if they are the same temperature. Colorization is typically added to help distinguish hotter areas from cooler ones.
Glass is transparent to visible light but is opaque to thermal wavelengths, so you can’t see through it with a thermal camera. Our world is tailored to visible light and we are used to it; some objects like a newspaper look very different through a thermal camera because the print will not show up, as the text and the paper are the same temperature. One way that FLIR solves this issue for users is through something we call MSX (Multi-Spectral Dynamic Imaging), which we’ve patented. On many of our devices, we pair a higher resolution visible camera alongside the thermal camera. Two separate images – one thermal and one visible — are simultaneously captured and the high fidelity details from the visible image (edges, contours, text, etc.) are embossed onto the thermal image. That much more detailed imagery provides the viewer a much better sense of what they are looking at and the relative orientation of various objects in a scene.
Another potential issue with recognizing content thermally is that the resolution of thermal cameras is typically lower than that of visible cameras. In many of our cameras we use an image processing algorithm called FLIR UltraMax that upsamples the native thermal image to a substantially higher resolution, providing a crisper and clearer final image for the user. FLIR UltraMax is only one of numerous image processing algorithms that enable us to provide enhanced imagery and better interpretation for users.
– Does thermal imaging benefit from the advances in deep learning?
Historically, cameras only captured and stored images for the user. Any post-analysis was done on a separate computer. With the advancements of mobile computing power, an increasing level of analysis and processing is done within the camera. Thermal cameras are being increasingly favored for their video analytics capabilities. Leveraging the ability to image in all light levels, measuring temperatures remotely and having a great ability to detect people and animals in a scene has been leveraged by modern video analytic algorithms. These algorithms can be extended and improved by deep learning algorithms and there are indeed a lot of advancements in this field recently, allowing users to transform a camera into an automated detection and/or control device.
FLIR actually just released a new thermal camera core last month, Boson, that incorporates the capability to embed deep learning powered algorithms on the camera itself. FLIR has already provided system-level functionality like pedestrian detection for the automobile market, intrusion alerts for security, intelligent traffic light control functionality, and we are planning many more of these solutions for our customers using the Boson core and deep-learning-based algorithms.
– What are the most unexpected/creative usage of FLIR thermal imaging?
We’ve seen many new and creative uses of thermal imaging since launching the FLIR One in 2014, the world’s first thermal imaging accessory for a smartphone. There is obviously substantial innovation that takes place among the user community and we’ve tried to ensure we have a great feedback loop to capture many of these new use cases. With the FLIR One, we’ve seen new everyday uses of thermal imaging such as finding pets in the dark, checking a baby’s bathwater, finding insects or pests in walls, scanning the bread rack at the bakery to quickly identify which bread is the freshest, avoiding the warm sushi at the grocery store, and checking to see if the campfire has truly been extinguished. Through our outreach efforts with makers and developers, we’ve seen our Lepton core used conceptually for more accurate people counting, gesture control in console gaming, and a wearable alert system for sight-impaired individuals. The FLIR One has also had nearly 30 third-party apps released in the iOS App Store and in Google Play – everything from practical DIY home project-related apps to fun photography apps and games.
– Can we expect thermal imaging in our camera phones soon?
Probably sooner than you realize. Just a couple of months ago at Mobile World Congress, we collaborated with CAT Phones to announce the first smartphone with integrated thermal imaging. This phone, the CAT S60, is a ruggedized phone targeted primarily at commercial and industrial users. Given FLIR’s brand strength and the strong proven value proposition for thermal imaging in those same trade spaces, there has been a tremendous amount of enthusiasm since the announcement. The phone should be available for purchase starting this summer. The smartphone is a natural delivery platform for the integration of various sensors with software to interpret and apply those sensing capabilities. Thermal is just one more very powerful and unique sensor that can provide imagery or more data about the environment around you and I believe we’re still in the early days of its broader evolution.
– What do you expect from LDV Vision Summit and why?
I’m looking forward to hearing from thought leaders in the imaging space about Virtual Reality, machine learning, and various image processing technologies that could be applied to thermal imaging. I also look forward to being able to share with the community more about the power of thermal imaging and what makes it a true “superpower.”
– What would you like to see FLIR create that technology cannot yet deliver?
Sensors of different wavelengths provide various advantages in their own spectral areas. The combination of several of these wavelengths into a compact, affordable multi-sensor system could provide an all-seeing instrument. FLIR will continue to drive down the cost and size of thermal imagers and all other FLIR sensor technology to enable this. Ultimately, I would like to see the benefits of the various wavelength sensors combined into a merged output. FLIR is already combining thermal and visible sensor outputs for enhanced imagery but there are many more wavelengths available to combine and enhance the benefits for society.
As in previous years, Kaptur is a proud media sponsor of the LDV Vision Summit. On top of advance previews of speakers and panels, we offer our readers 25% discount pricing. Act fast, there are only 10 available.
Go to: www.ldv.co/visionsummit/2016/tickets , enter KAPTUR25
Photo by scott1346
Author: Paul Melcher
Paul Melcher is the founder of Kaptur and Managing Director of Melcher System, a consultancy for visual technology firms. He is an entrepreneur, advisor, and consultant with a rich background in visual tech, content licensing, business strategy, and technology with more than 20 years experience in developing world-renowned photo-based companies with already two successful exits.