mobile photography technology, culture and community
www.dpreview.com

MIT's new chip promises 'professional-looking' photos on your smartphone

16
This is a die-colored image of MIT's chip that it claims will revolutionize mobile photography.

Mobile photographers may see better images from their devices with a new processor developed at the Massachusetts Institute of Technology (MIT). MIT claims the new technology will convert "smartphone snapshots" into "professional-looking photographs" with just the touch of a button. The technology aims to instantly create more realistic or enhanced lighting in a photo without affecting the image’s ambience.

According to MIT, the chip can perform super fast HDR processing for both still images and video, but will be especially helpful with low-light photography. By embedding the technology into the chip, rather than running it as software, the processes can be more energy efficient — ideal for mobile devices.

MIT explained the low-light imaging in a statement from its news service:

So in this instance the processor takes two images, one with a flash and one without. It then splits both into a base layer, containing just the large-scale features within the shot, and a detailed layer. Finally, it merges the two images, preserving the natural ambience from the base layer of the nonflash shot, while extracting the details from the picture taken with the flash.

To remove unwanted features from the image, such as noise — the unexpected variations in color or brightness created by digital cameras — the system blurs any undesired pixel with its surrounding neighbors, so that it matches those around it. In conventional filtering, however, this means even those pixels at the edges of objects are also blurred, which results in a less detailed image. 

But by using what is called a bilateral filter, the researchers are able to preserve these outlines, Rithe says. That is because bilateral filters will only blur pixels with their neighbors if they have been assigned a similar brightness value. Since any objects within the image are likely to have a very different level of brightness than that of their background, this prevents the system from blurring across any edges, he says. 

To perform each of these tasks, the chip’s processing unit uses a method of organizing and storing data called a bilateral grid. The image is first divided into smaller blocks. For each block, a histogram is then created. This results in a 3-D representation of the image, with the x and y axes representing the position of the block, and the brightness histogram representing the third dimension.

This makes it easy for the filter to avoid blurring across edges, since pixels with different brightness levels are separated in this third axis in the grid structure, no matter how close together they are in the image itself.

The research, funded by Taiwanese manufacturer Foxconn, has been turned into a prototype by Taiwan Semiconductor Manufacturing Company. This devicem built using 40-nanometer CMOS technology,  has been integrated with a camera and display for testing. No plans for commercialization have been revealed.

Comments

Total comments: 16
HowaboutRAW
By HowaboutRAW (Feb 26, 2013)

Foxconn is more like the iPhone assembler. It's not like Foxconn makes the CPU or the Wifi chip, or the screen, or likely the touch interface, or of course the part Apple makes: The OS.

Does Foxconn even make the battery? Foxconn very likely does make the printed circuit board on to which parts from many other manufactures are soldered.

0 upvotes
remylebeau
By remylebeau (Feb 24, 2013)

LOL I think the MIT guys need to redefine what they mean by "professional" looking images. To be more specific they should explain they are trying to improve the image quality from mobile sensors, that alone does not constitute what the general description of a "professional" image really.
I'm disappointed guys that are scientifically inclined and used to being precise and accurate, are so blatantly misusing descriptions to get notice for this development.
It's like watching Superman pick a half eaten burger out of a trash can...like "dude" c'mon have a little self respect.

0 upvotes
happypoppeye
By happypoppeye (Feb 24, 2013)

Yes, and I also have a pan that promises to turn my meals into "professional tasting" food. Life isn't all mathematically calculated science...

0 upvotes
rondhamalam
By rondhamalam (Feb 24, 2013)

Onboard HDR, Hardware noise reduction, Battery saving.

NOTHING SPECIAL !!!!!!!!!!!!!!!!!!!!

Sony has done it decades ago.

1 upvote
HowaboutRAW
By HowaboutRAW (Feb 26, 2013)

Sony made a sensor that does all that? Which camera is it in?

Also "decades ago"?

Comment edited 11 minutes after posting
0 upvotes
Benarm
By Benarm (Feb 23, 2013)

One advantage of smartphones over standard cameras is that they use much more sophisticated processors inside, which allows them to use better image processing algorithms. The new ARM15 is equivalent to a desktop PC from not too long ago.

1 upvote
HowaboutRAW
By HowaboutRAW (Feb 26, 2013)

And one big disadvantage is these camera's phones don't record raw data, so like tiny P+S cams they toss out the capacity to do much better images. This includes the Nokia 808.

0 upvotes
GabrielZ
By GabrielZ (Feb 23, 2013)

Hopefully this useful tech will appear in the upcoming iPhone 5s/6. Foxconn (and apparently in the near future TSMC also) are major Apple manufacturing partners.

Comment edited 1 minute after posting
0 upvotes
Aleo Veuliah
By Aleo Veuliah (Feb 23, 2013)

Good news.

1 upvote
Spectro
By Spectro (Feb 23, 2013)

sounds like hdr and photostacking, but on the chip

0 upvotes
Macro Lover
By Macro Lover (Feb 23, 2013)

Frankly speaking, this job shall be done with a common smartphone app.

Take 2 photos, flash+non flash, then use a joint filtering technique.

Plus, this bilateral grid method is kind of old fashioned --- neither the most efficient, nor produces the best result.

Comment edited 1 minute after posting
0 upvotes
graybalanced
By graybalanced (Feb 23, 2013)

Yeah, but like the article said...
"By embedding the technology into the chip, rather than running it as software, the processes can be more energy efficient — ideal for mobile devices."

Like networking, multimedia, and graphics, we all benefit when common operations are moved from software to hardware on a mobile device like a laptop or phone. The integration has helped reduce space and power requirements, while also expanding the baseline for the features you get in an affordable device.

1 upvote
zaurus
By zaurus (Feb 22, 2013)

Links to both the source and to a page with more detailed info would have been useful.

0 upvotes
Scott Everett
By Scott Everett (Feb 22, 2013)

the link is after the second paragraph:

http://web.mit.edu/newsoffice/2013/computational-photography-chip-0219.html

2 upvotes
Lan
By Lan (Feb 22, 2013)

"the processes can be more energy efficient"

Wording noted. Not "is", or even "will be", but "can be".

That said, at least they've managed to put together a prototype device; that's further along the development track than many get!

1 upvote
Michael_13
By Michael_13 (Feb 24, 2013)

This is probably a very scientific way of expressing their expectations.

Keep in mind: These guys are engineers, not marketing people! :-)

0 upvotes
Total comments: 16
About us
Sitemap
Connect