mobile photography technology, culture and community
www.dpreview.com

Apple aiming to use image stabilization to create high-resolution images

45

Apple Insider has discovered another interesting camera-related Apple patent application. The document titled "Super-resolution based on optical image stabilization" describes a way to use a camera's optical image stabilization system to capture a series of images at slightly offset angles and then stitch them together to create one large high-resolution file. 

Typically an OIS shifts either the imaging sensor or a lens element to counteract the movements of a photographer's unsteady hand. According to the patent application, Apple is looking at alternative ways of using this technology, which is interesting because none of Apple's current smartphones even use an OIS in the traditional way.

It's not clear at this point how high the resulting pixel-count and the image quality from such a system could be, nor what it would be used for. Image resolution is hardly the most pressing problem of smartphone cameras and most mobile images get downsized for sharing and editing anyway. Even Apple's current 8MP sensors offer more than enough pixels for most purposes.

However, more pixels can also be used in creative ways as Nokia has shown with the Lumia 1020 that uses the 41MP from its 1/1.5-inch sensor to enable a lossless digital zoom and create high quality 5MP files through clever downscaling algorithms.  If done right, Apple's patent could allow for a similar system without the use of a large and expensive sensor. We might find out with one of the next generation iPhones.  

Source: USPTO | Via: Apple Insider


  

Comments

Total comments: 45
cmorse
By cmorse (4 months ago)

My camera already has super high resolution based on offsetting the sensor though hand shake. Apple's method is pretty meaningless unless there is a tripod involve .

0 upvotes
bill nu
By bill nu (4 months ago)

Camera makers are asking for poor sales by selling cameras with less functionality than an Iphone. All cameras should have wifi and easy photo transfer to the web, etc.

1 upvote
Mike Griffin
By Mike Griffin (4 months ago)

Tilting the sensor as Apple proposes results in a non-linear solution. The pixels in the centre of the sensor shift not at all. the pixels in the corners shift greatly. So most of the resolution enhancement they propose occurs away from the centre. This is not the same as Hasselblad's solution in the H4D-200MS or Canon patent 2013-74337 that shifts the pixels in a linear fashion to achieve the same resolution enhancement.

0 upvotes
G Davidson
By G Davidson (4 months ago)

I've been thinking of this, too, combined with HDR bracketing for dynamic range and (why not) focus bracketing, too- all in a split second. I expect faster processing is a bottleneck for now. We've barely scratched the surface of what electronics can bring to cameras.

1 upvote
Menneisyys
By Menneisyys (4 months ago)

"HDR bracketing for dynamic range ... all in a split second "

Impossible as of now. The splitsecond HDR currently available in the iPhones isn't true, proper HDR as it uses the electronic shutter and not the mechanical one and otherwise also restricted in what it can do: generally, it can only increase the DR of the resulting image by about 1.5 stops, not with even 10 stops a proper mechanical shutter-shot, multiframe, bracketed shot would be able to achieve.

Currently, there aren't real HDR split-second, electronic shutter-only implementations in any mobile phone. All they can deliver is 2-3 stops enhanced DR at most, not 10+ ones.

I've published some iPhone shots clearly showing this in my Exposure Compensation and Bracketing Bible at http://www.iphonelife.com/blog/87/exposure-compensation-and-bracketing-bible

Comment edited 2 times, last edit 2 minutes after posting
1 upvote
G Davidson
By G Davidson (3 months ago)

Thank you for getting back with that information. Currently the HDR in the iPhone is pretty poor for sure, I didn't realise it was quite that limited, though. Some apps, like TrueHDR (my favourite of the various ones I've tried), can produce excellent results, even if taking their time in doing so. Much better than my Nikon camera can do in it's HDR mode.

It would be nice if some app could output a 16-bit TIFF, or even native HDR file for use later.

0 upvotes
Bruce McL
By Bruce McL (4 months ago)

Apple does not use everything that they patent. This is common knowledge. It is also common practice in the tech industry. Every company takes out more patents than they use.

Yet you wrote “Apple aiming to use...” as the title of the article. There is nothing in the article that supports this statement. The closest you come to that is using “could” and “might” in the last paragraph.

I understand you get more page views if you put made up things about Apple in headlines, but the cost is that it makes you look stupid.

0 upvotes
ZorSy
By ZorSy (4 months ago)

True, they 'aim' to use it. But this latest Apple 'patent frenzy' has an aim to sue, sue and sue more. This chapter is Apple's history is reminiscence of what already happened. This time, though, there is no SJ to pull out and salvage the ship. Bureaucrats have taken over and, sadly, they might share the same 'vision' Kodak had at time.

0 upvotes
Mike Griffin
By Mike Griffin (4 months ago)

Apple have sued the life out of mobile phone makers, are they about to start on the camera industry?

0 upvotes
Bruce McL
By Bruce McL (4 months ago)

@ZorSy What does your reply have to do with my comment? Apple has thousands of patents that they don’t use to sue people, this one is routine. By the way Steve Jobs started this patent war, it was his idea to “go nuclear” on Samsung.

@Mike Griffin Samsung is not dying, they are doing quite well. What company in the mobile phone business did Apple sue that is dying?

1 upvote
Mike Griffin
By Mike Griffin (4 months ago)

"Sued the life out of" is a figure of speech commenting more on Apple's obsessive activity in the courts rather than causing the collapse of any particular company.

0 upvotes
Peiasdf
By Peiasdf (4 months ago)

Samsung should just pay Apple a license fee for using Apple patents like HTC. Samsung won't because if it did, it would need to fork over $1 billion a year to Apple with the volume Samsung is pushing. Just to note, HTC is paying Apple $6 for ever phone sold because Android's core system is copied from Apple.

0 upvotes
Daniel Bliss
By Daniel Bliss (4 months ago)

Wow, it's getting hard to remember the last time I saw a patent diagram of something that ought to be patentable! Bravo Apple!

Now, about that ridiculous patent on round corners . . .

0 upvotes
technic
By technic (4 months ago)

as explained below, just another rip-off patent application from Apple. Probably they include free sleeping pills for the clerks at the patent office with every application.

1 upvote
vFunct
By vFunct (4 months ago)

Not a rip-off at all.

The below mentioned systems, like in Hasselbad, are based on sensor shift, not sensor tilt. That's a completely different mechanism that needs extra equipment, while this just takes advantage of existing OIS equipment.

This is an original idea, deserving of a patent.

Good job Apple!

1 upvote
robmanueb
By robmanueb (4 months ago)

If it is 'derserving' of a patent they should just make it and not try to stop someone else using it. That goes for most companies and most patents.

1 upvote
Peiasdf
By Peiasdf (4 months ago)

@robmanueb
What are you talking about? Of course company should patent first before making the product.

0 upvotes
robmanueb
By robmanueb (4 months ago)

Why?

0 upvotes
robmanueb
By robmanueb (4 months ago)

I tell you why you patent, it is greed. Companies that are manufacturing giants don't need a patent, patent are for two purposes to get a design made if you are a small time inventor and to collect royalties. Apple doesn't need a patent to use this technology, once they start making it they have the rights to make it through prior art. Patents are just there to stop anyone else making it. Car companies are good example of an industry that doens't care about patents, they care about producing cars not lawsuits. The Apple and Samsung war has highlited the corrupt way that lawyers make money from copyright trolling, this has nothing to do with making money from building products which they could easily do.

0 upvotes
Michael Piziak
By Michael Piziak (4 months ago)

I agree with the person that says how about Apple concentrate on a battery that will let the iPhone work all day.

8 upvotes
Peiasdf
By Peiasdf (4 months ago)

Agree but current tech aren't up to bar yet. Put in a large battery is easy but charging will take forever. Using rapid-charging technique available on OPPO Find 7 is possible but it requires a much bigger charger-brick and the battery will die earlier, a lot earlier.

Still, longer charging time is an acceptable trade off for me. I don't run around that often.

0 upvotes
Dennis
By Dennis (4 months ago)

That's interesting ... for years, there was speculation that Konica Minolta would do something similar with its sensor shift stabilization (which sounds easier to control via computer). Yet that hasn't been done yet.
(Whoops : I just read the other comments that point to Hasselblad's implementation).
Doing the same via shifting lens elements is clearly a different technology, though.

Comment edited 3 minutes after posting
1 upvote
MikeFairbanks
By MikeFairbanks (4 months ago)

How about a better battery for the phone? Come on, Apple. Oh, and more freedom for the user (in terms of syncing, cloud storage, etc.)?

And I just bought a new Dell with specs equal to a Mac but at half the price.

Obviously Apple makes outstanding products, and I have a few (Ipod, Iphone, Ipad), but they aren't as in the lead as they used to be in these areas, and are actually falling behind in some others.

What Apple needs to do is meet the needs of users more efficiently or invent something that everyone will want (or need) that others haven't thought up yet.

And for goodness sake, use the back of the phone already (for low-voltage display of time, weather, email feed, news, etc. At-a-glance stuff). The back of the phone is valuable real estate. Use it.

1 upvote
olakiril2
By olakiril2 (4 months ago)

How can be Apple patent something that Hasselblad has been offering with H4D-200MS for 3 years now?

4 upvotes
Peiasdf
By Peiasdf (4 months ago)

Hasselblad use sensor-shift, not OIS which is lens-shift. Apple's is also the first application of OIS + frame stitching which no other company think of before.

Think of a large-photodiode sensor coupled with this Apple tech. High resolution, low noise image with no blur.

0 upvotes
l_d_allan
By l_d_allan (4 months ago)

IIRC, Hasselblad does something similar to this to get the equivalent of 200 mpx from a 50 mpx sensor.

Interesting concept.

3 upvotes
Peiasdf
By Peiasdf (4 months ago)

Hasselblad use sensor-shift, not OIS which is lens-shift. Apple's is also the first application of OIS + frame stitching which no other company think of before.

0 upvotes
Gonard
By Gonard (4 months ago)

I wonder, somewhere down the line, if Apple will again start offering digital cameras. I think I saw the first (?) and perhaps only (?) Apple digital camera ~ 1995.

0 upvotes
AV Janus
By AV Janus (4 months ago)

Wouldn't this be possible with say Olympus 5-axis IBIS?
Maybe a ultra quick global shutter would come in handy to?

If yes, then I dont see sensor resolution, diffraction etc. as being the obstacle in the future for higher res photos.

2 upvotes
LukeDuciel
By LukeDuciel (4 months ago)

It appears Apple is only using shifting. If Olympus can control the 5-Axis IBIS actuators, it will be possible to do tilting with normal lens.

In body tilting and shifting with normal lens would be super news.

1 upvote
AV Janus
By AV Janus (4 months ago)

Front to back movements where talked about regarding in body AF function, I guess tilting would also be cool.

Lot of inbody progress to go IMO. Cool features to make!

Fine up-down, side to side movements paired with global shutter with a 5-6MP supersensitive sensor. WOW, superresolution! :-)

0 upvotes
barry reid
By barry reid (4 months ago)

Contax did in-body AF many years ago with the AX film camera. It made for a very fat bodied camera and slow AF.

0 upvotes
Meuh
By Meuh (4 months ago)

@Jokica people have been wanting pentax to do this for years now ;)

Hassled do something similar with there MF camera's,

http://www.dpreview.com/news/2011/5/26/hasselbladMS

4 upvotes
ZorSy
By ZorSy (4 months ago)

agree, that just shows true face of Apple Co. 'H4D-200MS utilizes Hasselblad's patented piezo frame module, which now captures 6 shots and combines them into an astounding 200 megapixel file.'.... apple's rip-off is called 'iOS'. Oh, and despite the name, it does not stabilise the image...

0 upvotes
Jokica
By Jokica (4 months ago)

Oh, I see. Thank you for your clarification.

0 upvotes
Mike Griffin
By Mike Griffin (4 months ago)

Canon was awarded patent 2013-74337 on 2013.4.22 for technology similar to Hasselblad's
http://www.canonwatch.com/canon-files-patent-for-high-res-images-obtained-by-pixel-shift/

0 upvotes
Mike Griffin
By Mike Griffin (4 months ago)

Can't find any indication that Hasselblad patented their idea of applying sensor shift to increase resolution. Doesn't the Patent Office consider prior art any more?

0 upvotes
Peiasdf
By Peiasdf (4 months ago)

@ZorSy
Hasselblad use sensor-shift, not OIS which is lens-shift. Apple's is also the first application of OIS + frame stitching which no other company think of before.

Read and think before mouthing off.

1 upvote
Menneisyys
By Menneisyys (4 months ago)

Too bad Apple is so much obsessed with producing as thin phones as possible, ruling entirely out large sensors. (FYI: both the Nokia 808 and the 1020 are comparatively thick because of their large sensors.)

Large sensors would make it possible to use considerably more pixels, making such sensor shifting stuff pretty much unneeded.

Nevertheless, I don't think this tech would ever make a debut in iPhones. After all, OIS also takes up some physical space and, assuming it does increase the thickness of the phone, it'll be a no-go for Apple. "Form over functionality", after all...

0 upvotes
Doc Pixel
By Doc Pixel (4 months ago)

You might be interested in the fact that Apple has hired away the lead manager from Nokia's camera division. See here: http://www.theverge.com/2014/5/9/5698550/apple-hires-ari-partinen-from-nokia

Will be interesting to see what goodies Ari and this patent can put together... wouldn't ya say?

1 upvote
Menneisyys
By Menneisyys (4 months ago)

Doc, still I don't think we'd ever see any large sensor iPhones. Not even top Nokia engineers can beat the laws of physics. There was a reason for both the 1020 and particularly the 808 being so thick...

That is, don't expect 2/3" (or larger) sensor iPhones. There won't be one. For Apple, slim profile is far-far more important than image quality.

0 upvotes
Franz Kerschbaum
By Franz Kerschbaum (4 months ago)

And we in Astronomy use it for decades...

3 upvotes
new boyz
By new boyz (4 months ago)

Well, you may not use it anymore without getting sued by Apple in the future, hahaha.

1 upvote
Mike Griffin
By Mike Griffin (4 months ago)

Funny that. You'd think those astronomy guys with their huge telescopes would have seen Apple coming up the rear.

1 upvote
Jokica
By Jokica (4 months ago)

Funny, I have contacted Pentax / Ricoh with this idea two weeks ago.

1 upvote
Total comments: 45
About us
Sitemap
Connect