Google, MIT create algorithms for retouching photos before taking them

PanARMENIAN.Net - It’s getting harder and harder to squeeze more performance out of your phone’s camera hardware. That’s why companies like Google are turning to computational photography: using algorithms and machine learning to improve your snaps. The latest research from the search giant, conducted with scientists from MIT, takes this work to a new level, producing algorithms that are capable of retouching your photos like a professional photographer in real time, before you take them, The verge says.

The researchers used machine learning to create their software, training neural networks on a dataset of 5,000 images created by Adobe and MIT. Each image in this collection has been retouched by five different photographers, and Google and MIT’s algorithms used this data to learn what sort of improvements to make to different photos. This might mean increasing the brightness here, reducing the saturation there, and so on.

Using machine learning to improve photos has been done before, but the real advance with this research is slimming down the algorithms so that they are small and efficient enough to run on a user’s device without any lag. The software itself is no bigger than a single digital image, and, according to a blog post from MIT, could be equipped “to process images in a range of styles.”

This means the neural networks could be trained on new sets of images, and could even learn to reproduce an individual photographer’s particular look, in the same way companies like Facebook and Prisma have created artistic filters that mimic famous painters. Of course, it’s worth pointing out that smartphones and cameras already process imaging data in real time, but these new techniques are more subtle and reactive, responding to the needs of individual images, rather than applying general rules.

Google has previously used its HDR+ algorithms to bring out more detail in light and shadow on mobile devices since the Nexus 6. And last year, Google’s computational photography lead, Marc Levoy, said that we’re “only beginning to scratch the surface” with this work.