mobile photography technology, culture and community
www.dpreview.com

Apple applies for dual-sensor camera patent

71
In this illustration in the patent application, the concept is shown on a laptop but the dual-sensor camera design could help build thinner smartphones, too.

Having two distinct sensors in one camera appears to be the flavor of the week. HTC just launched the One M8, the first smartphone to feature a dual-sensor camera. At the same time Apple Insider spotted an Apple patent application that was published on Thursday by the U.S. Patent and Trademark Office. The application is titled: "Electronic device with two image sensors."

It describes a camera design that uses two sensors, one for capturing color and the other for capturing luminance, or brightness information. The final image is then created by combining the captured values of both sensors. The idea is that each sensor module on its own can be thinner than a combination of the two, allowing for the design of slimmer device bodies.

According to Apple Insider, Apple also says splitting luminance and color capture between two sensors improves the signal-to-noise ratio and eliminates color reproduction issues caused by optical filters. There might be cost advantages in production, too. 

Via: Apple Insider


Comments

Total comments: 71
hazydave

They can play a nice trick here, maybe one not done before. Camera devices are getting larger as resolution increases. The sensor must increase to avoid diffraction limitations, so the same basic lens gets a longer focal length to cover that larger sensor, mandating more glass and a longer lens to sensor difference.

And then they de-Bayer the sensor's color matrix, to run a 4:2:0 or 4:1:1 color decimation anyway prior to JPEG or MPEG encoding.

So, how about two cameras. One is monochrome-only. Leaving off the Bayer filter gets you 3 the light capture per pixel. Diffraction is still an issue, but this can allow smaller pixels without losing sensitivity. The second camera only needs capture at 1/4 the resolution to get you 4:2:0 color, so that sensor can also be smaller. No good for close up shots, but camera phones are the lowest form of camera, and the only place where "thin" is more of a concern than image quality. Nice hack, this one.

1 upvote
janvanbogaert

This patent application was already published on June 28, 2012
US2012162465 (A1) - ELECTRONIC DEVICE WITH TWO IMAGE SENSORS The granted patent was published 2 days ago.

Another related one was published on August 20,2009:
US2009207272 (A1) - ELECTRONIC DEVICE WITH TWO IMAGE SENSORS

You can them when you search with these numers on http://worldwide.espacenet.com

All this excitement comes a bit late ...

Comment edited 2 minutes after posting
1 upvote
graybalanced

Those patents could be completely irrelevant to this one unless you show they are for the same purpose. Some dual-sensor implementations are for better dynamic range, others for depth of field, others for noise reduction...you can't just point at any old two-sensor patent and cry "Prior art!!"

0 upvotes
janvanbogaert

@graybalanced
I know. But the first I cited is the published application of the patent that was published a few days ago, and contains the same technical information.
The first I cited is a "Continuation" of the earlier, thus also related, but probably contains also new information.

0 upvotes
InTheMist

Copied by Samsung in 3..2..1...

1 upvote
chadley_chad

You're comment is quite ironic (and stupid!) seeing as its Apple looking to copy others on this.

Now go back under your bridge itroll .... You're embarrassing!

2 upvotes
joyclick

Apple likes to put its name on everything under the sun and make money from it.Android is the answer.

2 upvotes
aaonz

this has nothing to do with android of apple. It's a patent on all electronic to have dual sensor. Samsung will be sued if it copies.

2 upvotes
Entropius

Samsung will be sued no matter what. If Apple can sue over rounded rectangles, it can sue over anything.

4 upvotes
graybalanced

You're kidding...right? Android is hardly the answer, and ironically, the reasons are exactly the ones you listed.

Apple files patents the same way Microsoft, Google, etc. do. Samsung probably does as well.

But if you are trying to argue that it is companies who put their name on everything under the sun who are evil, Samsung, not Apple, is your target.

Apple associates its name with specific product lines and areas. Apple isn't going to build refrigerators, baby monitors, or televisions.

But Samsung really does put its name on everything under the sun. I searched on Amazon and Samsung was under 35 categories. Because Samsung will put their name on a washing machine. Apple will not.

Finally, Android is Samsung. If you take Samsung out of Android, Android has practically no market share.

0 upvotes
Mike Walters

Its a good job Apple didn't patent the single sensor camera, then they could sue everyone who produces a digital camera.

2 upvotes
jmajors

Apple's patent was to address the problem with thick protruding lens that you'd find on very thin phones or iTouch. When you have dual lens, you can reduce the size of lens thus reducing the thickness.

0 upvotes
vratnik

it's said in the article.

4 upvotes
JF69

Yes, but the apple haters don't seem to get it. They are shocked that Apple had the cheek to file a patent.

2 upvotes
Ademeion

I don't think that you can patent a reason why you do something. Perhaps you'd like to rephrase.

0 upvotes
inspiredan

Kodak used a dual sensor/lens design back in 2006 in the Kodak EasyShare V570. But the intent was to extend the zoom range while keeping the size of the camera compact. Apple's novelty relative to Kodak is capturing of luminance information.

1 upvote
Sergey Borachev

Did someone file a patent for the steering wheel+brake+petrol pedal interface in a motor car? If not, maybe I should quickly do that and get very rich. One of the football teams filing a patent for the slam dunk in basketball to prevent other teams doing it? A patent for a recipe? Where will this end and how much is this costing consumers?

Comment edited 1 minute after posting
7 upvotes
nawknai

This is nothing like what is being done these days.

It's costing consumers nothing. If anything, they're benefitting because with patents, there's a reason companies are spending money to create something totally new. Sure, patents sometimes such the life out of an industry, but the majority of the time, we've all benefitted.

0 upvotes
QuarterToDoom

>Electronic device with two image sensors

>Unlike many other digital SLRs, the E-330 used a second sensor in the viewfinder chamber which was fed by splitting 20% of the light from the viewfinder. The advantage of this implementation is that the camera's autofocus and exposure systems are fully functional and there is no shutter lag

0 upvotes
FRANCISCO ARAGAO

Now apple wil sue HTC for having the same idea first.

19 upvotes
Mike Griffin

HTC's idea is not the same. HTC captures depth information with the second sensor.

2 upvotes
Peiasdf

No FRANCISCO ARAGAO. HTC have licencse agreement with Apple. For every HTC phone sold, Apple gets $7.

0 upvotes
h2k

Seeing all the critical and dismissive comments here, i understand Apple is doomed and not a respectable tech company.

5 upvotes
ProfHankD

Apple is, and always has been, primarily driven by marketing, not technology. The best Apple thing to buy is their stock. ;-)

Comment edited 2 minutes after posting
10 upvotes
Tom Zimmer

I'm glad we are all on the same page here, that the best thing to buy from Apple is their stock, and of course the second best thing to buy from Apple is any of their products. ;-)

5 upvotes
LamarLamb

Sounds like a rethink of old beam splitter tech.....

3 upvotes
Scottelly

Before long they'll have multi-sensor cameras with one color captured from each. STOOOOOOOPIIIIIIID! WTF do they think a regular Bayer pattern sensor is?!? A 20 megapixel camera with a Bayer pattern sensor is basically a 3 color module with 5 million red sensors, 5 million blue sensors, and 10 million green sensors!

This dual sensor stuff would make more sense if they were planning to include 3D for better face recognition or something.

0 upvotes
steveh0607

But the colored filters are placed on top of the sensor. Having two sensors, one for color, one for luminance should produce a cleaner image and so is a good idea for phones which generally look crummy at any reasonable size.

4 upvotes
88SAL

Dude... Not so much different from foveon.

0 upvotes
hazydave

Keep in mind a few things. First, the Bayer filtering cuts out 2/3 of the light to each sensor. So, ignoring diffraction issues, the monochrome sensor will work just as well at 1/3 the pixel area.

Next, we're not really talkin' raw here. These cameras capture JPEG or MPEG. So after an image is de-Bayered, it's then 4:2:0 decimated... 3/4 of the color information is just tossed out. Then lossy compressed. So they could include a color sensor at 1/4 the monochrome resolution to deliver that that same color.

Comment edited 1 minute after posting
0 upvotes
solarider

One sensor for the NSA, the other for you ;-)

38 upvotes
D-Man67

Sounds almost like they took from the Sigma Quattro!

Comment edited 38 seconds after posting
2 upvotes
zodiacfml
4 upvotes
frenchdream

dual sensor and dual lens? side by side? there will be parallax issues, no?

4 upvotes
Mario G

Exactly. They say that it "eliminates color reproduction issues caused by optical filters", well, that might avoid them some mighty 1px purple fringing, but that's gonna introduce a whole new type of chromatic aberration, with luminance and colour slightly shifted from each other for close subjects...

2 upvotes
Mike Griffin

It will be interesting to see just how much color resolution can be reduced before it becomes obvious, given that one option is to have a higher pixel count for the luminance sensor.

0 upvotes
instamatic

Solutions like these are the future, but it seems that Apple is a bit late to the game with this one.

0 upvotes
rfsIII

What they need to do is combine this technology with the Lytro approach and then you would really have something—awesome color and adjustable focus....next step: the ability to take photos in any point along the temporal stream.

1 upvote
TheProv

Lytro or refocus is useless if you have a depth of field equivalent to a 28mm f16 FF lens.....
What do you want to refocus? Everything is in focus!

5 upvotes
sportyaccordy

Why not just do 4 sensors... R, G, B, luminance

Less filters -> more sensitivity; more resolution too.

3 upvotes
zodiacfml

Damn, you beat me to that.

1 upvote
Tom Zimmer

Wait, why not a lens for each pixel? Oh, that's right, we already have that in every sensor being made for the past 10 years :-\

0 upvotes
88SAL

*cough* foveon.

0 upvotes
peterwr

*cough* Every colour video camera since the days of tubes...

0 upvotes
ProfHankD

The patent application does not even require that there be two sensors, but just color-sensitive and luma-sensitive "sensor areas," which I think would overlap the Kodak patent for RGB+clear (white) CFAs. In any case, I'm not impressed.

4 upvotes
mma173

I'm not impressed too; I had the same idea in my mind as you, but this is a typical Apple patent. It seems they have connections in the patents office :D

Comment edited 45 seconds after posting
8 upvotes
AlexRuiz

Mma173, no. The issue is there are lots of copy cats chasing after Apple. So they are just being careful.

3 upvotes
hazydave

Sony's been using the RGBW sensor in camcorders for years. In fact, JVC did too, at least in the very first HD camcorder, the GR-HD1. Samsung uses the inverse, RGBW pixels on some of their phone/tablet screens, to increase brightness relative to power consumption.

That part is a very old idea.

Comment edited 32 seconds after posting
0 upvotes
SergioMO

Who will sue who ???

4 upvotes
Revenant

The HTC doesn't have one sensor for colour and one for luminance, so it's a different idea.

6 upvotes
ricklacey

Neither will sue.
Apple and HTC have a 10 year (as of 2012) agreement.

2 upvotes
DenisBBergeron

DS3d have this feature.

0 upvotes
MikeFairbanks

I'm looking forward to smartphones without a dead back.

In other words, put a screen on the back of the camera for more mundane things (time, temperature, better selfies, etc.).

It seems that both faces should be used.

1 upvote
88SAL

I agree, didnt Samsung have compacts that did that. Would it be cheaper to just put identical front and back cameras though

1 upvote
rbebbin2

There are already devices with screens front and back. The back face using ebook technology. Do a search and you will find them.

0 upvotes
hazydave

They make these things in the tens of millions, pennies count. So you get a $3 camera in the front and a $10 camera in the back. Some of these companies, like Samsung and Apple, may strain the capacity of any chip supplier... There's no way doubling volume on the back camera gets you the front camera price. It may not even be an option to buy that volume, even at the higher price.

0 upvotes
Jogger

Now the NSA will have access to even clearer photos from your iPhone.

8 upvotes
Wally626

Apple also introduced the two color flash, which I hope could be translated to larger camera flashes at some point. Being able to match the room color temperature does make for nice images. It can be done today using the correct gels over the flash but is a much slower process then an automatic flash system that can change color by using a ratio of the power from different color lamps.

So will this be followed by a four camera system with one camera for each color plus B&W?

Comment edited 42 seconds after posting
3 upvotes
mma173

A single coloring flash is possible. No need for multiple ones.

1 upvote
photo_rb

I understand how a two color flash works...what's the color-shifting technology behind a single color flash?

Comment edited 10 seconds after posting
0 upvotes
mma173

Get ideas from coloring bulbs that are available in the market or take a look at U.S. Patent 6,755,555.

0 upvotes
agentul

Panasonic has been making video cameras with three sensors for years.

2 upvotes
hjulenissen

But they have shared a common lens, thus are quite different from the illustration

4 upvotes
Wally626

Three sensor camera are quite common in the high end of video, but it does the opposite of the Apple invention. Three sensors requires a splitting prism system and a much deeper camera.

3 upvotes
mma173

They were R, G, and B sensors. I wish some used the same arrangement with different sensors e.g.
Multi-exposure for HDR.
IR and UV.
..

0 upvotes
Mario G

Also, the Panasonic 3D lens for Micro Four Thirds is a dual lens and effectively turns the one physical sensor into two virtual sensors, and it actually makes a good use of the separate optics by adding some 3D to the image.

1 upvote
rfsIII

This sounds like the breakthrough we've been waiting for from the big camera companies.

1 upvote
jkoch2

Some employee at Canon or Nikon probably proposed multi-sensor cameras, instead of single large ones, years ago. The likely reply: "Can't do. Would challenge our DLSR and big lens business. Be careful or soon you'll get a pink slip."

3 upvotes
ThePhilips

It is, unfortunately, not a breakthrough. By far.

Closest example of two lenses. Recall why "(D)SLR" has "Single" in the name: because previously there were "TLR" - twin-lens reflex - cameras. (Though second lens was used exclusively for composition.)

The general problem with the dual-lens solution is the alignment of the two sensor/lens units. For good IQ, required precision of alignment is simply doesn't hold long in real life due to stress and wear. Plus you can't just dumbly combine the two images: the combining should account for slightly different perspectives and as such requires also depth information.

But for the video chat, with the sub 1MP resolutions, it might probably work.

1 upvote
Jonas Palm

This is a sensible concern. But I suspect that modern assembly tech, encased in a mobile device, would be a lot more resistant to alignment issues than my TLRs of old.

I'd suspect that the benefits do not justify the cost of dual sensors, dual lenses, extra sturdy precision assembly and so on, and that we won't see the patent actually being used.

0 upvotes
hazydave

DSLR sensors are 50x-100x larger than camera phone sensors. And there's no win for making the sensor smaller and thinner. This really doesn't make any sense for traditional cameras... the lens costs more than the sensor, and the lens is also most of the size factor. Minolta made a 3-chip DSLRs (RD-175, RD-3000), split via dichroic prism like a camcorder. But just the one lens. These did not catch on.

0 upvotes
Total comments: 71
About us
Sitemap
Connect