DxO, the company behind the highly-regarded DxOMark benchmarks that rank camera sensor and camera phone image quality, has announced that its expanding into testing the image quality of security cameras.
This may seem like an odd decision, given few people demand Hollywood-grade video quality from their doorbell cam, but as DxO points out, security camera image quality is worth scrutinizing for other reasons:
Why benchmark security cameras?
Home security cameras are usually small so that they can be discreet and unobtrusive. The sensors are about the same size as those found in smartphones, meaning they are much smaller than what you find in DSLR cameras. This presents some major challenges and limitations when it comes to image quality. Small apertures limit the flow of light to the sensor, which affects exposure and dynamic range. Small sensors mean less light is captured, which could affect the final image.
Another challenge, especially for outdoor cameras, is capturing a wide field of view. Moreover, they use fisheyes lenses, which have smaller apertures, to cover a wider field of view. Fisheye lenses can sometimes create artefacts like distortion and color fringing.
The third challenge is detection, and image quality plays a role in this. The most important and basic function of home security cameras is that they must be able to quickly and accurately detect relevant motion in a use case such as a home invasion. The keyword here is relevant, and image quality from the camera is crucial in this case. If the image quality is poor, the security system could easily send too many false alerts, or worse, it might not send any at all because it did not detect anything.(opens in new tab)
The way a home security camera is installed also presents a challenge. Is it wired or battery-powered? A system that is battery-powered could be placed anywhere, but it could potentially limit the camera’s performance in terms of image processing so as to not drain the battery too quickly. A system that requires to be plugged in, on the other hand, might have to be placed in a less than ideal location and might struggle to capture good images. Other installation considerations include the ease of setup, the network, and power, application control as well as cloud usage and storage.(opens in new tab)
Another constraint on image quality is the network the security system runs on. The uploading of recorded images is subject to compression, which affects the final image, especially on battery-powered systems. The quality of the network will also affect the final image. If two bars of Wi-Fi are lost during an upload, compression artefacts will most likely show up in the final image.
Home surveillance cameras also face audio quality challenges and limitations. What’s important is whether voices are intelligible in these kinds of products—in playback and recording. Outdoor cameras are prone to artefacts such as wind noise, and as a result, speech becomes hard to understand. In the case of a doorbell camera, the audio quality is affected by the distance of the speaker to the device. The loudness of the captured of voices is important, especially in comparison with the volume of the background (Signal to Noise Ratio) which may be affected either by the distance from the user to the device or by the environmental conditions (wind, rain, road noises, …).(opens in new tab)
DxO doorbell camera group test
With all these potential image quality issues in mind, DxO conducted a group test of four popular doorbell cameras to assess their daylight and infra-red night time image quality, focussing on attributes such as exposure, detail preservation, artefacts, and color.(opens in new tab)
To ensure good testing realism while also maintaining consistent repeatability, DxO has used realistic mannequins rather than human testers, as well as studio laboratory lighting for daylight and night-time illumination.(opens in new tab)
In this four-camera round-up, Google's Nest Hello doorbell cam produced the best image quality benchmark results. DxO says that it's planning to evaluate other types of security camera - outdoor and indoor - in the near future.(opens in new tab)