Researchers hacked a weird Franken-lens to a Canon EOS R10 to take photos with everything in focus – without computational photography

Computational photography and in-camera focus stacking can help photographers keep more of the image in focus, but what if a camera could focus on everything all at once using only optics? Researchers at Carnegie Mellon University used four different lenses and 3D display tech to create a Franken-lens that can keep every pixel in a photo in focus.

Traditional optics can only focus on one distance range at a time, adjusting a single focus plane to bring objects in focus. Techniques like focus stacking can help, but are based on editing and computational photography – not optics.

But the research group reverse-engineered 3D display technology to essentially enable each pixel to have a different focal plane, creating a lens that can keep everything in focus without computational photography. The result is that the objects and background are all in focus, a method that the researchers dubbed Spatially-Varying Autofocus.

The key to the unusual lens is Split-Lohmann, an optical arrangement made for 3D displays for interactive gaming and movies, also developed by Carnegie Mellon researchers in 2023. But instead of using the Split-Lohmann for a display, the team used the idea to add a Spatial Light Modulator to the lens.

See how the prototype works in the video below:

ABOVE: Watch the prototype camera lens in action

The Spatial Light Modulator tilts inside the lens to redirect the light, which changes the focal plane. But this modulator can also be controlled locally.

The researchers used a depth map created from the camera’s autofocus to adjust the modulator to match the objects in the frame. This enables the focal point to be controlled for each pixel, rather than using one focal plane for the entire image.

Light comes through the lens, bounces off the custom Spatial Light Modulator and heads for the camera’s sensor. The result is an image where every object in the photo is in focus, without any editing tricks.

The researchers’ prototype lens is T-shaped, with the light entering from the left (so the photographer is actually taking pictures of something to their left, not right in front of them).

Light comes in from the top-left of the T through the variable lens, through one relay lens, through a beam splitter and cubic phase plate, through another relay lens, then to the Spatial Light Modulator, bouncing back where that beam splitter then redirects the light into another relay lens and, finally, to the camera, which is a Canon EOS R10.

The researchers used a variety of lenses on the first entry point (the video below seems to show a Nikon, which feels a bit blasphemous on a Canon, but I digress) and three Samyang 85mm f/1.4 AS as “relay lenses.”

While the technique is all optics and no computational photography, the process still requires at least two images. The first one is to create the depth map to adjust the Spatial Light Modulator to the objects in the scene, and the second one is to actually take the photo of everything in focus.

The researchers were able to use both contrast detection and phase detection autofocus to create those depth maps.

The team’s prototype with the R10 was able to take photos with everything all in focus at a speed of about 3fps. But the group also created another prototype using a machine vision sensor to improve the speed to 21fps.

While genres like portraits seldom want everything in focus, the unusual lens could potentially be used in genres like macro photography, where keeping enough of the scene in focus is a challenge.

But perhaps even more interesting is that the researchers' prototype was also able to take those photos using a wide open aperture. Narrower apertures can help keep more (though not all) of the image in focus, but cut back the amount of light entering the lens. The unusual T-shaped lens could keep that wide aperture while still getting everything in focus.

Another potential use of the technology is to customize the Spatial Light Modulator, which would give the photographer the ability to pick and choose which objects to leave in focus.

As an example, the researchers demonstrated this concept by eliminating a thin wire fence from the focal map. This enables the background to be sharp along with the subject, but blurs the thin wire enough that it nearly disappears in the photo.

This disappearing trick only works with very thin objects like wire fences, but it’s an interesting concept that feels a bit like editing a photo before it is taken.

The team – which included Yingsi Qin, Aswin C Sankaranarayanan and Matthew O’Toole – presented their work at the 2025 International Conference on Computer Vision.

You may also like…

Browse the best Canon lenses or the best macro lenses.