In the title of their celebrated 1956 documentary film, ocean explorer Jacques Cousteau and cinematographer Louis Malle famously dubbed the realm beneath the seas The Silent World. Using phosphorous torches for lighting and an ultra-wide-angle lens, they created an international cinema sensation—one of the first underwater films ever to be shown in living color.
Cousteau, who became a tireless advocate for marine animals and ecosystems, understood that most of what lies beneath the seas—some 80 percent of the world's oceans, according to the US National Oceanic and Atmospheric Administration (NOAA)—remains unseen by human eyes. His 1960s television series, The Undersea World of Jacques Cousteau, shot from aboard the Calypso, pushed the bounds of new optics and lighting technology to bring a window on Earth's oceans into the homes of millions of people around the globe.
Today's filmmakers are no less inspired by the sometimes strange, often captivating beauty of the underwater world. They have an array of imaging technologies to choose from, but like Cousteau, passion and a knack for making things happen also drives their work.
NOAA National Marine Sanctuaries filmmaker Nick Zachar started diving when he was 11 years old and says, "I have been diving ever since." He studied biology as an undergraduate, but then, after seeing documentaries like BBC's Blue Planet series, he enrolled in film school.
Cinematographer Nick Zachar uses a Boxfish 360-degree camera to film a turtle cleaning station in the Hawaiian Islands' Humpback Whale National Marine Sanctuary. Credit: NOAA National Marine Sanctuaries
For a thesis project, Zachar decided to make a film about the underwater world surrounding the tiny island of Saba in the Caribbean. But with little money and a diver's do-it-yourself sense of adventure, "I made it all on a GoPro-fake it till you make it."
Now, Zachar works with 360-degree virtual reality (VR) cameras to produce NOAA's 360 degree: Explore the Blue immersive virtual reality films showcasing the US National Marine Sanctuaries. They can be viewed on a PC or a VR headset.
For Cousteau, "making it" came from his close association with another pioneer, Harold "Doc" Edgerton, a professor at Massachusetts Institute of Technology (MIT) whose mid-20th century optics inventions, like the use of powerful strobe lighting, resulted in such iconic photos as a bullet at the instant it tears through a piece of fruit. According to MIT's online Edgerton memorial and archive, the underwater world, for Doc, held special interest. Indeed, his deep-sea strobe light enabled the first images of the deep ocean and seafloor.
Both artistic creativity and scientific curiosity continue to drive optics and photonics innovations that enlarge our window on the ocean world. Torches have been replaced by LED lighting—if lighting is even needed by some of today's professional video cameras from Sony, RED, and other manufacturers, that record with high fidelity in even very low-light conditions.
And from TV and movie screens, directors and cinematographers now hope to bring viewers new immersive and VR experiences via technology like multicamera arrays for 360-degree filming underwater.
Scientists have graduated from simple still and video cameras locked in underwater housings, to optics innovations like laser line scan underwater imaging, robots that can track and film elusive creatures in the mid-ocean twilight zone, and handheld or fixed underwater microscopes to image plankton and other animals in the water column where they live, to name just a few.
"Science probably leads the technology," says Evan Kovacs, an underwater cinematographer and CEO of Marine Imaging Technologies, who began his professional life as a lighting designer for "Jazz at Lincoln Center" in New York City. Today, his long list of daring underwater film projects includes both 2D and 3D imaging of iconic deep-water shipwrecks, the Titanic and her sister ship, the Britannic.
His work has shown him that "Usually science has a question," Kovacs says, "and so okay, how can we solve it via imaging?" Once the scientific community comes up with a way to capture the type and quality of images it needs from underwater, he says, "the entertainment industry realizes that something can be done, and they want it. They often come up with funding to push it to the next level."
Cinematographer and 360-degree camera inventor Casey Sapp says his company, VRTUL (Virtual Reality Technology Underwater Limited), "specializes in prototypes that are purpose built for VR headsets and large-format screens." When asked to create images of hundreds of millions of pixels at 60 frames-per-second, he says, for example, "there's no [off-the-shelf] camera that shoots at that. We buy cameras and then build software and pipelines to reach what's available and make it work."
A VRTUL 360-degree camera system by Casey Sapp. Credit: Casey Sapp
Working at the cross section of cinema and science, Sapp's inventions include the first 360-degree 3D underwater camera system; the first remotely operated vehicle/VR piloting system in partnership with the Monterey Bay Aquarium Research Institute; the first 360-degree cinematic camera on a submarine; and, he says, in 2019, the highest resolution underwater cinematic camera system in the world.
It's a high-stakes game to shoot immersive film productions that are unique in every way, Sapp says. Diving has inherent dangers, while expensive cameras, lights, and other equipment add to the risk. He says the nature of his projects—which require specialized software to stitch together images from multiple cameras and make it all look seamless—rest on big budgets and narrow windows of opportunity to get the shot.
But he also describes the reward: images he will share with the world of baby humpback whales as they are birthed in the waters off Tonga in the South Pacific, for example, or great white sharks on patrol in the ocean near Guadalupe Island in Mexico.
However sublime the experience, shooting still or motion pictures in the world's oceans—whether for Hollywood hits or scientific expeditions to hydrothermal vents—requires an understanding of key differences with the surface world, says Stan Logan of Deepsea Power & Light. He notes a substantial divide between shallow- and deep-water photography.
"The dive market is the largest user of underwater lighting and filming, and that goes back to the 1960s and 70s, in particular with James Bond," Logan says. "The technical challenges of course are much simpler at diver depths of 100 feet where lighting is also less of a problem."
However, in deep-ocean imaging, every piece of equipment must be able to withstand high pressure and account for the way light behaves underwater.
Water has a higher refractive index than air—it bends the light more. As such, filmmakers must make careful equipment choices, such as camera-housing dome ports that allow rays of light to enter camera lenses unrefracted, eliminating upper limits on the field of view, versus flat ports, which narrow the field of view and make the image appear magnified.
Water absorbs wavelengths of light, with long wavelengths the first to go—red, orange, yellow. In fact, colors disappear with depth in the same order as the spectrum. Lighting a scene underwater can bring the color back, though marine biologists now recognize that artificial light can drive some animals away while attracting others by providing a spotlight on prey.
In the 20th century, two major technical advances propelled underwater imaging, says oceanographer Jules S. Jaffe of the Scripps Institution of Oceanography and University of California, San Diego. First was the development of the film camera, and then the Aqua-Lung, Cousteau's invention with Émile Gagnon, that gave rise to scuba diving as a popular sport.
A new revolution in underwater optical imaging began in the mid-1980s. Jaffe cites, for example, the towed sled, ARGO, developed by Robert Ballard at Woods Hole Oceanographic Institution (WHOI). Ballard and his crew watched in real time as video cameras sent live images via cable of the discovery of the Titanic. Today, interested people can join Ballard virtually on a deep-sea marine science expedition which includes live video feeds from the seafloor.
Jaffe's own research has been part of the revolution, attesting to his interest in using new technology to observe ocean phenomena. Or, as he puts it, "optics in service to global ecology."
He is inventor or coinventor of instruments such as the benthic underwater microscope (BUM), a diver-deployed imaging system capable of near-micrometer resolution. Jaffe describes the device's three principle optical components: a long working distance microscope objective lens, a shape-changing electrically tunable lens, and focused LEDs to provide reflectance illumination.
Many innovative underwater imaging devices, like the benthic underwater microscope, shown here, have been developed in Jules Jaffe's lab at the Scripps Institution of Oceanography. Credit: Jules Jaffe
Recording images with up to 2.2 µm resolution, the BUM can capture images of fine anatomical details, like dinoflagellates or zooxanthellae living symbiotically inside coral. Jaffe and his team used it to examine various levels of coral bleaching—which is really expulsion of the zooxanthellae. The instrument, he says, is handheld, or it can be left in place for autonomous time-sequence imaging, the only constraints being battery life and biofouling.
"They get all schmutzed up," Jaffe says of the gunk that builds up on the instrument.
The importance of imaging these fine details is "an intrinsic need to more fully characterize small marine organisms with small microscopes," Jaffe continues. For example, his lab has supported or codeveloped instruments like the in-situ plankton assemblage explorer, an inexpensive underwater imaging system to study zooplankton, as well as a video velocimeter that uses a particle image correlation technique to reconstruct underwater bulk flow velocity-an important measurement, since ocean currents determine the fate of many marine organisms.
Visual information from such instruments, Jaffe says, can aid understanding of larger scale ecological processes and events like coral bleaching and algal blooms. He sees a role for AI in helping to process the rich visual data stream captured by today's optics technologies.
Alan Adams, a former MIT physics professor and string theorist, now ocean-optics designer, says oceanography as a discipline needs to step back from its historic reliance on highly specialized, one-of-a-kind imaging and exploration devices and perhaps embrace mass production.
Adams says researchers see a growing need to have many cameras—mass-produced cameras—that would provide, for example, very wide-area monitoring of changing coastlines, ecological restoration projects, or to keep a close eye on aquaculture operations.
The needs of science and commerce, Adams says, should be strong incentives to build mass-produced camera systems. "There's so many reasons that we need good cameras underwater and the problem is not that we're too stuck to do it properly," he says. "You must start with the assumption that you're going to build hundreds of thousands of cameras. And then it's possible to build a device that really ought to be built."
For other ocean researchers, there is a need for instruments that fall somewhere between the extremes of microscopes and minisubs. That's especially true of the twilight zone of the world's oceans-anywhere from 200- to 1,000-meters deep. Marine scientists say it's an area teeming with life—a vast biomass of giant larvaceans, dinner-plate size jellyfish, krill, zooplankton, and more. These animals, probably some of the most surreal lifeforms on Earth, are thought to move up and down in the water column, rising to feed at night near surface waters, then submerging with daylight to evade predation.
"By scuba diving, we could make the kind of behavior observations and collections that were hard to do other ways," says marine biologist and twilight zone expert, Laurence Madin. "That was all great, but eventually your air ran out and you had to go back up and you lost track of whatever you were looking at," the former WHOI deputy director and vice president for research recalls telling his friend, WHOI robotics expert Dana Yoerger.
"I said, it'd be great if there was some kind of machine that would either carry on past the time when the diver's air ran out, or better yet, go ahead and go deeper," Madin recalls.
Yoerger continues the story, "And so Larry asked me, he said, Dana, can you make me a robot that can show me what I missed when I ran out of air?" Yoerger laughs. "And that started the whole project. I said, oh yeah, we can make a robot, it can even follow the animals around."
Enter Mesobot, a new-this-year autonomous underwater vehicle developed at WHOI with primary funding from the US National Science Foundation.
Yoerger and colleagues seemingly thought of everything. Lights used on Mesobot are in red or infrared wavelengths to which deep sea animals have minimal sensitivity. Its thrusters are designed to minimize noise and water disturbances, and the vehicle has an overall unobtrusive profile that maybe doesn't quite fit in with the locals, but on test runs they don't seem to mind, either.
And, as Yoerger promised, Mesobot can, indeed, find and track target animals with low-res stereoscopic cameras and, for scientific imaging, a high-res cinema-quality video camera.
"Now, you could go for something really seriously sensitive," says Adams, who consulted with Yoerger on Mesobot, "but that fails the beautiful image problem, because you're not just interested in science, you're also interested in making beautiful, compelling, powerful images that communicate the science."
William G. Schulz is managing editor of Photonics Focus and a PADI Open Water Scuba Instructor.
Related SPIE content:
Read a transcript of Jacques Cousteau's 1968 plenary talk at the SPIE conference on Underwater Photo-Optical Instrumentation Applications.
