mobile photography technology, culture and community
www.dpreview.com

Apple applies for dual-sensor camera patent

71
In this illustration in the patent application, the concept is shown on a laptop but the dual-sensor camera design could help build thinner smartphones, too.

Having two distinct sensors in one camera appears to be the flavor of the week. HTC just launched the One M8, the first smartphone to feature a dual-sensor camera. At the same time Apple Insider spotted an Apple patent application that was published on Thursday by the U.S. Patent and Trademark Office. The application is titled: "Electronic device with two image sensors."

It describes a camera design that uses two sensors, one for capturing color and the other for capturing luminance, or brightness information. The final image is then created by combining the captured values of both sensors. The idea is that each sensor module on its own can be thinner than a combination of the two, allowing for the design of slimmer device bodies.

According to Apple Insider, Apple also says splitting luminance and color capture between two sensors improves the signal-to-noise ratio and eliminates color reproduction issues caused by optical filters. There might be cost advantages in production, too. 

Via: Apple Insider


Comments

Total comments: 71
hazydave
By hazydave (4 months ago)

They can play a nice trick here, maybe one not done before. Camera devices are getting larger as resolution increases. The sensor must increase to avoid diffraction limitations, so the same basic lens gets a longer focal length to cover that larger sensor, mandating more glass and a longer lens to sensor difference.

And then they de-Bayer the sensor's color matrix, to run a 4:2:0 or 4:1:1 color decimation anyway prior to JPEG or MPEG encoding.

So, how about two cameras. One is monochrome-only. Leaving off the Bayer filter gets you 3 the light capture per pixel. Diffraction is still an issue, but this can allow smaller pixels without losing sensitivity. The second camera only needs capture at 1/4 the resolution to get you 4:2:0 color, so that sensor can also be smaller. No good for close up shots, but camera phones are the lowest form of camera, and the only place where "thin" is more of a concern than image quality. Nice hack, this one.

1 upvote
janvanbogaert
By janvanbogaert (4 months ago)

This patent application was already published on June 28, 2012
US2012162465 (A1) - ELECTRONIC DEVICE WITH TWO IMAGE SENSORS The granted patent was published 2 days ago.

Another related one was published on August 20,2009:
US2009207272 (A1) - ELECTRONIC DEVICE WITH TWO IMAGE SENSORS

You can them when you search with these numers on http://worldwide.espacenet.com

All this excitement comes a bit late ...

Comment edited 2 minutes after posting
1 upvote
graybalanced
By graybalanced (4 months ago)

Those patents could be completely irrelevant to this one unless you show they are for the same purpose. Some dual-sensor implementations are for better dynamic range, others for depth of field, others for noise reduction...you can't just point at any old two-sensor patent and cry "Prior art!!"

0 upvotes
janvanbogaert
By janvanbogaert (4 months ago)

@graybalanced
I know. But the first I cited is the published application of the patent that was published a few days ago, and contains the same technical information.
The first I cited is a "Continuation" of the earlier, thus also related, but probably contains also new information.

0 upvotes
InTheMist
By InTheMist (4 months ago)

Copied by Samsung in 3..2..1...

1 upvote
chadley_chad
By chadley_chad (4 months ago)

You're comment is quite ironic (and stupid!) seeing as its Apple looking to copy others on this.

Now go back under your bridge itroll .... You're embarrassing!

2 upvotes
joyclick
By joyclick (4 months ago)

Apple likes to put its name on everything under the sun and make money from it.Android is the answer.

2 upvotes
aaonz
By aaonz (4 months ago)

this has nothing to do with android of apple. It's a patent on all electronic to have dual sensor. Samsung will be sued if it copies.

2 upvotes
Entropius
By Entropius (4 months ago)

Samsung will be sued no matter what. If Apple can sue over rounded rectangles, it can sue over anything.

4 upvotes
graybalanced
By graybalanced (4 months ago)

You're kidding...right? Android is hardly the answer, and ironically, the reasons are exactly the ones you listed.

Apple files patents the same way Microsoft, Google, etc. do. Samsung probably does as well.

But if you are trying to argue that it is companies who put their name on everything under the sun who are evil, Samsung, not Apple, is your target.

Apple associates its name with specific product lines and areas. Apple isn't going to build refrigerators, baby monitors, or televisions.

But Samsung really does put its name on everything under the sun. I searched on Amazon and Samsung was under 35 categories. Because Samsung will put their name on a washing machine. Apple will not.

Finally, Android is Samsung. If you take Samsung out of Android, Android has practically no market share.

0 upvotes
Mike Walters
By Mike Walters (4 months ago)

Its a good job Apple didn't patent the single sensor camera, then they could sue everyone who produces a digital camera.

2 upvotes
jmajors
By jmajors (4 months ago)

Apple's patent was to address the problem with thick protruding lens that you'd find on very thin phones or iTouch. When you have dual lens, you can reduce the size of lens thus reducing the thickness.

0 upvotes
vratnik
By vratnik (4 months ago)

it's said in the article.

4 upvotes
JF69
By JF69 (4 months ago)

Yes, but the apple haters don't seem to get it. They are shocked that Apple had the cheek to file a patent.

2 upvotes
Ademeion
By Ademeion (4 months ago)

I don't think that you can patent a reason why you do something. Perhaps you'd like to rephrase.

0 upvotes
inspiredan
By inspiredan (4 months ago)

Kodak used a dual sensor/lens design back in 2006 in the Kodak EasyShare V570. But the intent was to extend the zoom range while keeping the size of the camera compact. Apple's novelty relative to Kodak is capturing of luminance information.

1 upvote
Sergey Borachev
By Sergey Borachev (4 months ago)

Did someone file a patent for the steering wheel+brake+petrol pedal interface in a motor car? If not, maybe I should quickly do that and get very rich. One of the football teams filing a patent for the slam dunk in basketball to prevent other teams doing it? A patent for a recipe? Where will this end and how much is this costing consumers?

Comment edited 1 minute after posting
7 upvotes
nawknai
By nawknai (4 months ago)

This is nothing like what is being done these days.

It's costing consumers nothing. If anything, they're benefitting because with patents, there's a reason companies are spending money to create something totally new. Sure, patents sometimes such the life out of an industry, but the majority of the time, we've all benefitted.

0 upvotes
QuarterToDoom
By QuarterToDoom (4 months ago)

>Electronic device with two image sensors

>Unlike many other digital SLRs, the E-330 used a second sensor in the viewfinder chamber which was fed by splitting 20% of the light from the viewfinder. The advantage of this implementation is that the camera's autofocus and exposure systems are fully functional and there is no shutter lag

0 upvotes
FRANCISCO ARAGAO
By FRANCISCO ARAGAO (4 months ago)

Now apple wil sue HTC for having the same idea first.

19 upvotes
Mike Griffin
By Mike Griffin (4 months ago)

HTC's idea is not the same. HTC captures depth information with the second sensor.

2 upvotes
Peiasdf
By Peiasdf (4 months ago)

No FRANCISCO ARAGAO. HTC have licencse agreement with Apple. For every HTC phone sold, Apple gets $7.

0 upvotes
h2k
By h2k (4 months ago)

Seeing all the critical and dismissive comments here, i understand Apple is doomed and not a respectable tech company.

5 upvotes
ProfHankD
By ProfHankD (4 months ago)

Apple is, and always has been, primarily driven by marketing, not technology. The best Apple thing to buy is their stock. ;-)

Comment edited 2 minutes after posting
10 upvotes
Tom Zimmer
By Tom Zimmer (4 months ago)

I'm glad we are all on the same page here, that the best thing to buy from Apple is their stock, and of course the second best thing to buy from Apple is any of their products. ;-)

5 upvotes
LamarLamb
By LamarLamb (4 months ago)

Sounds like a rethink of old beam splitter tech.....

3 upvotes
Scottelly
By Scottelly (4 months ago)

Before long they'll have multi-sensor cameras with one color captured from each. STOOOOOOOPIIIIIIID! WTF do they think a regular Bayer pattern sensor is?!? A 20 megapixel camera with a Bayer pattern sensor is basically a 3 color module with 5 million red sensors, 5 million blue sensors, and 10 million green sensors!

This dual sensor stuff would make more sense if they were planning to include 3D for better face recognition or something.

0 upvotes
steveh0607
By steveh0607 (4 months ago)

But the colored filters are placed on top of the sensor. Having two sensors, one for color, one for luminance should produce a cleaner image and so is a good idea for phones which generally look crummy at any reasonable size.

4 upvotes
88SAL
By 88SAL (4 months ago)

Dude... Not so much different from foveon.

0 upvotes
hazydave
By hazydave (4 months ago)

Keep in mind a few things. First, the Bayer filtering cuts out 2/3 of the light to each sensor. So, ignoring diffraction issues, the monochrome sensor will work just as well at 1/3 the pixel area.

Next, we're not really talkin' raw here. These cameras capture JPEG or MPEG. So after an image is de-Bayered, it's then 4:2:0 decimated... 3/4 of the color information is just tossed out. Then lossy compressed. So they could include a color sensor at 1/4 the monochrome resolution to deliver that that same color.

Comment edited 1 minute after posting
0 upvotes
solarider
By solarider (4 months ago)

One sensor for the NSA, the other for you ;-)

38 upvotes
D-Man67
By D-Man67 (4 months ago)

Sounds almost like they took from the Sigma Quattro!

Comment edited 38 seconds after posting
2 upvotes
zodiacfml
By zodiacfml (4 months ago)

nope.

4 upvotes
frenchdream
By frenchdream (4 months ago)

dual sensor and dual lens? side by side? there will be parallax issues, no?

4 upvotes
Mario G
By Mario G (4 months ago)

Exactly. They say that it "eliminates color reproduction issues caused by optical filters", well, that might avoid them some mighty 1px purple fringing, but that's gonna introduce a whole new type of chromatic aberration, with luminance and colour slightly shifted from each other for close subjects...

2 upvotes
Mike Griffin
By Mike Griffin (4 months ago)

It will be interesting to see just how much color resolution can be reduced before it becomes obvious, given that one option is to have a higher pixel count for the luminance sensor.

0 upvotes
instamatic
By instamatic (4 months ago)

Solutions like these are the future, but it seems that Apple is a bit late to the game with this one.

0 upvotes
rfsIII
By rfsIII (4 months ago)

What they need to do is combine this technology with the Lytro approach and then you would really have something—awesome color and adjustable focus....next step: the ability to take photos in any point along the temporal stream.

1 upvote
TheProv
By TheProv (4 months ago)

Lytro or refocus is useless if you have a depth of field equivalent to a 28mm f16 FF lens.....
What do you want to refocus? Everything is in focus!

5 upvotes
sportyaccordy
By sportyaccordy (4 months ago)

Why not just do 4 sensors... R, G, B, luminance

Less filters -> more sensitivity; more resolution too.

3 upvotes
zodiacfml
By zodiacfml (4 months ago)

Damn, you beat me to that.

1 upvote
Tom Zimmer
By Tom Zimmer (4 months ago)

Wait, why not a lens for each pixel? Oh, that's right, we already have that in every sensor being made for the past 10 years :-\

0 upvotes
88SAL
By 88SAL (4 months ago)

*cough* foveon.

0 upvotes
peterwr
By peterwr (4 months ago)

*cough* Every colour video camera since the days of tubes...

0 upvotes
ProfHankD
By ProfHankD (4 months ago)

The patent application does not even require that there be two sensors, but just color-sensitive and luma-sensitive "sensor areas," which I think would overlap the Kodak patent for RGB+clear (white) CFAs. In any case, I'm not impressed.

4 upvotes
mma173
By mma173 (4 months ago)

I'm not impressed too; I had the same idea in my mind as you, but this is a typical Apple patent. It seems they have connections in the patents office :D

Comment edited 45 seconds after posting
8 upvotes
AlexRuiz
By AlexRuiz (4 months ago)

Mma173, no. The issue is there are lots of copy cats chasing after Apple. So they are just being careful.

3 upvotes
hazydave
By hazydave (4 months ago)

Sony's been using the RGBW sensor in camcorders for years. In fact, JVC did too, at least in the very first HD camcorder, the GR-HD1. Samsung uses the inverse, RGBW pixels on some of their phone/tablet screens, to increase brightness relative to power consumption.

That part is a very old idea.

Comment edited 32 seconds after posting
0 upvotes
SergioMO
By SergioMO (4 months ago)

Who will sue who ???

4 upvotes
Revenant
By Revenant (4 months ago)

The HTC doesn't have one sensor for colour and one for luminance, so it's a different idea.

6 upvotes
ricklacey
By ricklacey (4 months ago)

Neither will sue.
Apple and HTC have a 10 year (as of 2012) agreement.

2 upvotes
DenisBBergeron
By DenisBBergeron (4 months ago)

DS3d have this feature.

0 upvotes
MikeFairbanks
By MikeFairbanks (4 months ago)

I'm looking forward to smartphones without a dead back.

In other words, put a screen on the back of the camera for more mundane things (time, temperature, better selfies, etc.).

It seems that both faces should be used.

1 upvote
88SAL
By 88SAL (4 months ago)

I agree, didnt Samsung have compacts that did that. Would it be cheaper to just put identical front and back cameras though

1 upvote
rbebbin2
By rbebbin2 (4 months ago)

There are already devices with screens front and back. The back face using ebook technology. Do a search and you will find them.

0 upvotes
hazydave
By hazydave (4 months ago)

They make these things in the tens of millions, pennies count. So you get a $3 camera in the front and a $10 camera in the back. Some of these companies, like Samsung and Apple, may strain the capacity of any chip supplier... There's no way doubling volume on the back camera gets you the front camera price. It may not even be an option to buy that volume, even at the higher price.

0 upvotes
Jogger
By Jogger (4 months ago)

Now the NSA will have access to even clearer photos from your iPhone.

8 upvotes
Wally626
By Wally626 (4 months ago)

Apple also introduced the two color flash, which I hope could be translated to larger camera flashes at some point. Being able to match the room color temperature does make for nice images. It can be done today using the correct gels over the flash but is a much slower process then an automatic flash system that can change color by using a ratio of the power from different color lamps.

So will this be followed by a four camera system with one camera for each color plus B&W?

Comment edited 42 seconds after posting
3 upvotes
mma173
By mma173 (4 months ago)

A single coloring flash is possible. No need for multiple ones.

1 upvote
photo_rb
By photo_rb (4 months ago)

I understand how a two color flash works...what's the color-shifting technology behind a single color flash?

Comment edited 10 seconds after posting
0 upvotes
mma173
By mma173 (4 months ago)

Get ideas from coloring bulbs that are available in the market or take a look at U.S. Patent 6,755,555.

0 upvotes
agentul
By agentul (4 months ago)

Panasonic has been making video cameras with three sensors for years.

2 upvotes
hjulenissen
By hjulenissen (4 months ago)

But they have shared a common lens, thus are quite different from the illustration

4 upvotes
Wally626
By Wally626 (4 months ago)

Three sensor camera are quite common in the high end of video, but it does the opposite of the Apple invention. Three sensors requires a splitting prism system and a much deeper camera.

3 upvotes
mma173
By mma173 (4 months ago)

They were R, G, and B sensors. I wish some used the same arrangement with different sensors e.g.
Multi-exposure for HDR.
IR and UV.
..

0 upvotes
Mario G
By Mario G (4 months ago)

Also, the Panasonic 3D lens for Micro Four Thirds is a dual lens and effectively turns the one physical sensor into two virtual sensors, and it actually makes a good use of the separate optics by adding some 3D to the image.

1 upvote
rfsIII
By rfsIII (4 months ago)

This sounds like the breakthrough we've been waiting for from the big camera companies.

1 upvote
jkoch2
By jkoch2 (4 months ago)

Some employee at Canon or Nikon probably proposed multi-sensor cameras, instead of single large ones, years ago. The likely reply: "Can't do. Would challenge our DLSR and big lens business. Be careful or soon you'll get a pink slip."

3 upvotes
ThePhilips
By ThePhilips (4 months ago)

It is, unfortunately, not a breakthrough. By far.

Closest example of two lenses. Recall why "(D)SLR" has "Single" in the name: because previously there were "TLR" - twin-lens reflex - cameras. (Though second lens was used exclusively for composition.)

The general problem with the dual-lens solution is the alignment of the two sensor/lens units. For good IQ, required precision of alignment is simply doesn't hold long in real life due to stress and wear. Plus you can't just dumbly combine the two images: the combining should account for slightly different perspectives and as such requires also depth information.

But for the video chat, with the sub 1MP resolutions, it might probably work.

1 upvote
Jonas Palm
By Jonas Palm (4 months ago)

This is a sensible concern. But I suspect that modern assembly tech, encased in a mobile device, would be a lot more resistant to alignment issues than my TLRs of old.

I'd suspect that the benefits do not justify the cost of dual sensors, dual lenses, extra sturdy precision assembly and so on, and that we won't see the patent actually being used.

0 upvotes
hazydave
By hazydave (4 months ago)

DSLR sensors are 50x-100x larger than camera phone sensors. And there's no win for making the sensor smaller and thinner. This really doesn't make any sense for traditional cameras... the lens costs more than the sensor, and the lens is also most of the size factor. Minolta made a 3-chip DSLRs (RD-175, RD-3000), split via dichroic prism like a camcorder. But just the one lens. These did not catch on.

0 upvotes
Total comments: 71
About us
Sitemap
Connect