mobile photography technology, culture and community

MIT's new chip promises 'professional-looking' photos on your smartphone

This is a die-colored image of MIT's chip that it claims will revolutionize mobile photography.

Mobile photographers may see better images from their devices with a new processor developed at the Massachusetts Institute of Technology (MIT). MIT claims the new technology will convert "smartphone snapshots" into "professional-looking photographs" with just the touch of a button. The technology aims to instantly create more realistic or enhanced lighting in a photo without affecting the image’s ambience.

According to MIT, the chip can perform super fast HDR processing for both still images and video, but will be especially helpful with low-light photography. By embedding the technology into the chip, rather than running it as software, the processes can be more energy efficient — ideal for mobile devices.

MIT explained the low-light imaging in a statement from its news service:

So in this instance the processor takes two images, one with a flash and one without. It then splits both into a base layer, containing just the large-scale features within the shot, and a detailed layer. Finally, it merges the two images, preserving the natural ambience from the base layer of the nonflash shot, while extracting the details from the picture taken with the flash.

To remove unwanted features from the image, such as noise — the unexpected variations in color or brightness created by digital cameras — the system blurs any undesired pixel with its surrounding neighbors, so that it matches those around it. In conventional filtering, however, this means even those pixels at the edges of objects are also blurred, which results in a less detailed image. 

But by using what is called a bilateral filter, the researchers are able to preserve these outlines, Rithe says. That is because bilateral filters will only blur pixels with their neighbors if they have been assigned a similar brightness value. Since any objects within the image are likely to have a very different level of brightness than that of their background, this prevents the system from blurring across any edges, he says. 

To perform each of these tasks, the chip’s processing unit uses a method of organizing and storing data called a bilateral grid. The image is first divided into smaller blocks. For each block, a histogram is then created. This results in a 3-D representation of the image, with the x and y axes representing the position of the block, and the brightness histogram representing the third dimension.

This makes it easy for the filter to avoid blurring across edges, since pixels with different brightness levels are separated in this third axis in the grid structure, no matter how close together they are in the image itself.

The research, funded by Taiwanese manufacturer Foxconn, has been turned into a prototype by Taiwan Semiconductor Manufacturing Company. This devicem built using 40-nanometer CMOS technology,  has been integrated with a camera and display for testing. No plans for commercialization have been revealed.


Total comments: 16

Foxconn is more like the iPhone assembler. It's not like Foxconn makes the CPU or the Wifi chip, or the screen, or likely the touch interface, or of course the part Apple makes: The OS.

Does Foxconn even make the battery? Foxconn very likely does make the printed circuit board on to which parts from many other manufactures are soldered.


LOL I think the MIT guys need to redefine what they mean by "professional" looking images. To be more specific they should explain they are trying to improve the image quality from mobile sensors, that alone does not constitute what the general description of a "professional" image really.
I'm disappointed guys that are scientifically inclined and used to being precise and accurate, are so blatantly misusing descriptions to get notice for this development.
It's like watching Superman pick a half eaten burger out of a trash "dude" c'mon have a little self respect.


Yes, and I also have a pan that promises to turn my meals into "professional tasting" food. Life isn't all mathematically calculated science...


Onboard HDR, Hardware noise reduction, Battery saving.

NOTHING SPECIAL !!!!!!!!!!!!!!!!!!!!

Sony has done it decades ago.

1 upvote

Sony made a sensor that does all that? Which camera is it in?

Also "decades ago"?

Edited 11 minutes after posting

One advantage of smartphones over standard cameras is that they use much more sophisticated processors inside, which allows them to use better image processing algorithms. The new ARM15 is equivalent to a desktop PC from not too long ago.

1 upvote

And one big disadvantage is these camera's phones don't record raw data, so like tiny P+S cams they toss out the capacity to do much better images. This includes the Nokia 808.


Hopefully this useful tech will appear in the upcoming iPhone 5s/6. Foxconn (and apparently in the near future TSMC also) are major Apple manufacturing partners.

Edited 1 minute after posting
Aleo Veuliah

Good news.

1 upvote

sounds like hdr and photostacking, but on the chip


Frankly speaking, this job shall be done with a common smartphone app.

Take 2 photos, flash+non flash, then use a joint filtering technique.

Plus, this bilateral grid method is kind of old fashioned --- neither the most efficient, nor produces the best result.

Edited 1 minute after posting

Yeah, but like the article said...
"By embedding the technology into the chip, rather than running it as software, the processes can be more energy efficient — ideal for mobile devices."

Like networking, multimedia, and graphics, we all benefit when common operations are moved from software to hardware on a mobile device like a laptop or phone. The integration has helped reduce space and power requirements, while also expanding the baseline for the features you get in an affordable device.

1 upvote

Links to both the source and to a page with more detailed info would have been useful.

Scott Everett

the link is after the second paragraph:


"the processes can be more energy efficient"

Wording noted. Not "is", or even "will be", but "can be".

That said, at least they've managed to put together a prototype device; that's further along the development track than many get!

1 upvote

This is probably a very scientific way of expressing their expectations.

Keep in mind: These guys are engineers, not marketing people! :-)

Total comments: 16
About us