mobile photography technology, culture and community
www.dpreview.com

DxOMark Mobile Report: Samsung Galaxy S4 lab tests look outstanding

35

DxOMark Mobile Report: Samsung Galaxy S4

We covered Samsung's new superphone, the Galaxy S4, at its launch, tried some of its new camera features and also had a good look at its image capturing capabilities in our recent Smartphone Super Shootout. Thanks to our partners at DxOMark we can now also get a better idea what the Samsung's image output means in numbers and graphs. DxO's imaging experts have analyzed 14 aspects of mobile imaging including detailed image quality assessment, flash performance, autofocus reliability and more to calculate a final score. This report will be integrated into our full review once it is finished but for now read on to find out how the Samsung Galaxy S4's camera performed in the DxO lab tests. For DxO's own report, head to the Galaxy S4 page on DxOMark.com.

Samsung Galaxy S4 Headline features:

  • Android 4.2.2
  • 5-inch 1080p (441ppi) Super-Amoled screen
  • 7.9mm thin
  • 13MP rear and 2MP front cameras
  • 2GB RAM
  • 16/32/64 GB storage versions, Micro-SD card support up to 64 GB
  • 130g
  • 4G LTE 100Mbs down, 50Mps up
  • Barometer, temperature and humidity sensors
  • 2600MAh battery, removable

Summary

With a DxOMark Mobile score of 75 the Samsung Galaxy S4 is ranked number two in the DxO smartphone rankings, between Nokia's 808 and the Apple iPhone 5. The DxOMark team report that the Samsung Galaxy S4 captures images with "impressive detail preservation in bright light conditions," "good auto-exposure even with difficult outdoor situations" and "good image quality with the flash." Images also showed "pleasant and rich colors in all lighting conditions" and the "autofocus is fast and accurate in both auto and trigger mode."

On the downside: DxOMark observed "important loss of detail in low light due to noise reduction," "strong ringing" and "slight color shading under low tungsten light."

In video mode, Galaxy S4 cannot quite repeat the impressive stills image performance. DxOMark's engineers reported that the Samsung produced video with "good colors and good texture reproduction" but the report also found the S4 video showed "visible aliasing and staircase effect" and "strong noise in low-light conditions." Testers also found the video stabilization disappointing and that "in low light conditions the AF can trigger without any change in focus distance."

Still Photography

The DxOMark team found that the Samsung Galaxy S4 images show "good auto-exposure, even with difficult outdoor situations," "pleasant and rich colors in all lighting situations" and "good color shading correction in most situations although a slight error was noticed in low tungsten light."

The only downside in terms of color, exposure and contrast found by DxO testers is a tendency to clip highlights in contrasty scenes. DxOMark scored the Samsung Galaxy S4 at 83/100 for color in bright light and 77/100 in low light.   

Overall DxOMark awarded the Samsung Galaxy S4 scores of:

  • 4.4 out of 5 for Exposure
  • 4.5 out of 5 for White Balance accuracy
  • 4.0 out of 5 for Color shading in low light*
  • 4.0 out of 5 for Color shading in bright light*
  • 3.0 out of 5 for Color Rendering in low light
  • 4.5 out of 5 for Color Rendering in bright light

*Color Shading is the nasty habit cellphone cameras have of rendering different areas of the frame with different color shifts, resulting in pictures with, for example, pinkish centers and greenish corners.

Noise and Details

DxOMark's engineers reported that the Samsung Galaxy S4 image output shows "impressive detail preservation in bright light conditions," with the 13MP sensor showing significant improvement over the best 8MP devices. The DxO testers also found that "noise levels remain acceptable even in low light conditions without chromatic components in flat areas."

However, on the downside "noise reduction is too strong, causing significant loss of detail in low light" and "in low light edges may either be sharp with chroma noise or slightly blurry depending on background." As the image stablization is of the digital variant, it "raises ISO values which significantly increases noise levels."

Texture Acutance

Texture Acutance is a way of measuring the ability of a camera to capture images that preserve fine details, particularly the kind of low contrast detail (textures such as fine foliage, hair, fur) that can be blurred away by noise reduction or obliterated by excessive sharpening.

Sharpness is an important part of the quality of an image, but while it is easy to look at an image and decide visually whether it's sharp or not, the objective measurement of sharpness is less straightforward.

An image can be defined as "sharp" if its edges are sharp and if fine details are visible, but in-camera processing means it's possible to have one of these (sharp edges) but not the other (fine details). Conventional MTF measurements tell us how sharp an edge is, but have drawbacks when it comes to measuring fine detail preservation. Image processing algorithms can detect edges and enhance their sharpness, but they can also find homogeneous areas and smooth them out to reduce noise.

Texture Acutance, on the other hand, can qualify sharpness in terms of preservation of fine details, without being fooled by edge enhancement algorithms.

A detail of a target made of a dead leaves pattern, designed to measure Texture Acutance. It is obtained by drawing random shapes that occlude each other in the plane, like dead leaves falling from a tree. The statistics of this model follow the distribution of the same statistics in natural images.

In this example from a DSLR without edge enhancement, sharpness seems equal on edge and on texture. Many details are visible in the texture.

In this second example edges have been digitally enhanced, and the edge looks over-sharp, with visible processing halos ("ringing"). On the texture part, many details have disappeared.

At first sight, the images from these two cameras may appear equally sharp. A sharpness measurement on edges will indeed confirm this impression, and will even show that the second camera is sharper. But a closer examination of low contrasted textures shows that the first camera has a better preservation of fine details than the second. The purpose of the Texture Acutance measurement is to qualify this difference.

Note: Acutance is a single value metric calculated from a MTF result. Acutance is used to assess the sharpness of an image as viewed by the human visual system, and is dependent on the viewing conditions (size of image, size of screen or print, viewing distance). Only the values of texture acutance are given here. The measurements are expressed as a percentage of the theoretical maximum for the chosen viewing condition. The higher the score, the more details can be seen in an image. 

For all DxOMark Mobile data presented on connect.dpreview.com we're showing only the 8MP equivalent values, which gives us a level playing field for comparison between phone cameras with different megapixel values by normalizing all to 8MP, suitable for fairly large prints. DxOMark also offer this data for lower resolution use-cases (web and on-screen). For more information on DxOMark's testing methodology and Acutance measurements please visit the website at www.dxomark.com.
Texture acutance is better in bright than in lower light but there is not much difference between tungsten and daylight.
In terms of texture acutance the Galaxy S4 is at the lower end of the scale among its peers in low light but takes the lead at medium and bright light levels.

Edge Acutance

Edge acutance is a measure of the sharpness of the edges in images captured by the phone's camera, and again we're only looking at the most demanding of the three viewing conditions that DxOMark reports on, "8MP equivalent."
The Samsung Galaxy S4's increases from 20 to 100 Lux and then stays on a high level.
In terms of edge acutance in good light the Samsung Galaxy S4 shows the best performance of all tested smartphones so far, including the Nokia 808.

Visual Noise

Visual Noise is a value designed to assess the noise in an image as perceived by the human visual system, depending on the viewing condition (size of image, size of screen or print, viewing distance). The measurements have no units and can be simply viewed as a weighted average of noise standard deviation for each channel in the CIE L*a*b* color space. The lower the measurement, the less noise in the image.

 Noise levels in daylight conditions are low across all light levels.
 Noise levels are slightly higher under tungsten lighting.
The Samsung Galaxy S4's noise levels are lower than most of its competitors at lower light levels, but images exhibit slightly higher levels of noise in brighter conditions.

Noise and Detail Perceptual scoring

DxOMark engineers don't just point camera phones at charts, they also take and analyze scores of real-world shots and score them accordingly. Their findings for the Samsung Galaxy S4 were:

Natural scene

  • Texture (bright light): 5.0 out of 5
  • Texture (low light): 3.0 out of 5
  • Noise (bright light): 4.0 out of 5
  • Noise (low light) 3.0 out of 5
Bright light sample shot
100% crop shows dood detail in bright light.
Low light (20 Lux) studio shot
100% crop shows poor texture preservation in low light.
In low tungsten light crops show average noise levels but loss of detail through noise reduction.

Artifacts

Phone cameras, like entry-level compact cameras, tend to suffer from artifacts such as sharpening halos, color fringing, vignetting (shading) and distortion, which can impact on the visual appeal of the end result. DxOMark engineers measure and analyze a range of artifacts. Their findings after testing the Samsung Galaxy S4 are shown below:

Pros:

  • Uniform sharpness across the image field
  • Almost no color fringing

Cons 

  • Strong ringing noticeable
  • Moiré sometimes noticeable

Perceptual scores

  • Sharpness 4.0 out of 5
  • Color fringing 4.5 out of 5

Measured findings

  • Ringing center 30.3%
  • Ringing corner 21.6%
  • Max geometric distortion -0.48%
  • Luminance shading 11%

Distortion and Chromatic Aberrations

The graph shows the magnification from center to edge (with the center normalized to 1). The Samsung Galaxy S4 shows a slight pincushion distortion, which you are not going to notice in normal photography.
 The Samsung's lateral chromatic aberrations are well within acceptable limits.

Autofocus

DxOMark also tests autofocus accuracy and reliability by measuring how much the acutance — sharpness — varies with each shot over a series of 30 exposures (defocusing then using the autofocus for each one). As with other tests these results are dependent on the viewing conditions (a little bit out of focus matters a lot less with a small web image than a full 8MP shot viewed at 100%). Using the 8MP equivalent condition the Samsung Galaxy S4 results are excellent in good light and still very good in lower light. The overall score is 87/100 in bright light and 74/100 in low light.

Pros: 

  • Autofocus is fast and accurate both in auto and trigger mode
  • Very little overshooting
  • Good scene change detection
  • Automatic face detection

Cons: 

  • Autofocus is slightly less repeatable in low light.
 Autofocus repeatability - average acutance difference with best focus: low light 3.3%, bright light 4.25%

Flash

DxOMark scored the Samsung Galaxy S4 82/100 overall for its flash performance.

Pros: 

  • Good overall image quality with or without additional light source: nice colors, good exposure and sharp details

Cons: 

  • Some white balance errors when flash is mixed with a tungsten illuminant
  • No red eye correction

Overall DxOMark Mobile score for Photo: 79 / 100


Video Capture

DxOMark engineers put phone cameras through a similarly grueling set of video tests, and you can read their full findings on the DxOMark website here. We'll simply summarize for you. DxOMark found the Samsung Galaxy S4's video to have good texture reproduction. However, there is visible aliasing and the digital video stabilization is not very efficient.

Pros: 

  • Good colors
  • Good texture reproduction

Cons: 

  • Visible aliasing and staircase effect
  • Strong noise in low-light conditions
  • Disappointing video stabilization
  • In low light conditions, AF can trigger without any change in focus distance

Overall DxOMark Mobile score for Video: 68 / 100

DXOMark Mobile Score
75

DXOMark Image Quality Assessment

The Samsung Galaxy S4 achieves a DxO Mark score of 75, which means it ranks second on the DxO list of smartphone cameras, between Nokia's 808 PureView and the iPhone 5.

The Samsung Galaxy S4 offers "impressive detail preservation in bright light" and the "13MP sensor shows a great improvement over the best 8MP cameras." DxO testers also found the auto exposure works well, even in difficult outdoor situations and the flash image quality to be good. On the downside: DxOMark reports "heavy loss of detail in low light due to excessive noise reduction" and "strong ringing."

In video mode, the Galaxy S4 captured footage with "good texture reproduction" and "good colors" but also showed "visible aliasing and staircase effects" and "strong noise in low-light conditions." Image stabilization in video mode was also found to be inefficient. For DxO's report, visit the Samsung Galaxy S4 page on www.dxomark.com.

Photo Mobile Score 79   Video Mobile Score 68
Exposure and Contrast 85   Exposure and Contrast 88
Color 80   Color 91
Autofocus 80   Autofocus 52
Texture 76   Texture 78
Noise 74   Noise 66
Photo Artifacts 71   Video Artifacts 71
Flash 82   Stabilization 35

Comments

Total comments: 35
TheCameraEngineer

One key takeaway from this is that no "magic" processing could compensate for a good sensor. It's inevitable that texture/detail will be lost in low light because of the aggressive denoise that has to be applied.

0 upvotes
peevee1

"DxOMark's engineers reported "

You mean testers, not engineers. Engineers are people who engineer something.

0 upvotes
TheCameraEngineer

What do you know - maybe the same guys that developed the test methods also do the actual testing.

0 upvotes
Jostian

Give me the Nokia 808 sensor at 5X larger than the S4's anyday... pity about the Symbian though.

2 upvotes
Uzinuzin

I am really impressed by the scientific process being followed here to test the camera in still & video mode. Its clearly seen Still mode scores outstandingly well however, Video mode has got slight stablization issues. Anyway, with camera modes like - Shot and sound, Dual camera etc. Galaxy S4 will definitely impress you. It has also left behind Apple iPhone 5 in a popularity poll I came across at http://www.brandcollage.com/faceoff/apple-iphone5-vs-samsung-galaxy-s4/

1 upvote
Michael Pardee

I would like to see the S4 scores for the metrics dxo normally uses for camera sensors: Color Depth, Dynamic Range, Low-Light ISO. I know they might be pitiful, maybe they don't want to make the camera phones look bad since they get their money from advertising.

Alternatively I would like to see normal dpreview camera reviews list those 7 metrics : exposure/contrast, color, autofocus, texture, noise, photo artifacts, flash. (at least the ones that are applicable) Some numbers to back up dpreview's pulled-out-of-the-air final percentage scores wouldn't hurt.

Why are there two separate sets of criteria? I know camera phones, compacts, and interchangable lens camera could be considered completely different categories and might use different scales for rating, but I would think there could at least be a common set of metrics.

0 upvotes
TrojMacReady

They normally measure RAW data which isn't possible here. Curves, NR, colour profiles and sharpening.... a whole different beast to deal with.

1 upvote
Durandalfr

So it's the occasion to criticize the over-postreatment on some phone camera :o

1 upvote
ARTASHES

How Samsung achived to obtain such detailed jpeg at 100% from 13mp 1/3 sensor when their 1/2.3 camreas are so bad ?

1 upvote
Decoboy

Why is there a DX-Mark-Mobile score? Why not the same test as per a P&S. What is different & how can one compare results?
If phone cameras are as good as the hype, why the need to create a differant score - if they are supposed to be useful as a camera then just test them using the same scoring as other (compact) fixed lens cameras.

2 upvotes
Lars Rehm

how many compact fixed lens point and shoot cameras can you name?

3 upvotes
Durandalfr

what's the point lars? do you test dslr without lens when they come naked or use only the combo lens? You don't adapt the camera position to match the same composition in the studio scene? There are a few fixed lens compact cameras. DPR reviewed some of them? how do you think they did?

If the cameraphones should replace compact camera one day, better test now in what they are equal to them by giving compact cameras a fair comparison, by including, for exemple, the phone cameras results in the DPR studio scene comparator

Comment edited 5 minutes after posting
0 upvotes
fastlass

I appreciate the work that went into this review, but it's too analytical. Could you please distill these charts and graphs into something more digestible?

Perhaps have side by side output from the cameras that zero-in on how the metrics you're presenting translate into real world aspects of an image.

You have 13 graphs and 5 OOC pictures. It should really be reversed to craft a review that's useful to a wide audience.

2 upvotes
spidermoon

I agree, and a comparaison between iphone and sIII or note II
You dont even have picture with HDR mode or Panorama. It's not a true DPR review, but a copy/paste DXO review.

0 upvotes
Lars Rehm

You get side-by-side studio and real-life shots in our smartphone shotout: http://connect.dpreview.com/post/9219904986/shootout-samsunggalaxys4-vs-htcone-vs-iphone5-vs-lumia920

And there will of course be more analysis and sample images in our full review which is still to come.

0 upvotes
itsastickup

For photographers, the crucial Dxo report will be in to the HTC One and its APS-C photosites and dynamic range. Until that comes no decision on a smartphone is really possible.

Comment edited 22 seconds after posting
0 upvotes
TrojMacReady

What are "APS-C" photosites? That's a measure of sensor size and has zero to do with the size of photosites. Photosites come in all kinds of sizes, rather irrespective of format. And to underline that point, the photosites from the HTC One are no larger than those found in my 12 year old P&S.

Noise and DR for a given output size are in practise mostly related to sensor size, metering and processing, not the size of photosites.

2 upvotes
itsastickup

It was handy shorthand; pedanticness should be done in private with other consenting adults.

0 upvotes
d3xmeister

That's, right, you can put full frame pixels in a 2MP camera phone, the image will be camera phone quality, maybe even worse.
Sensor size is what matter, to put it simply to you: What amount of light is used to produce the photo ? If you take a half sensor for example, it will capture about half of the amount of light. You can make it 1MP, it will actually make the photo worse, since resolution does matter when it comes to image quality, and the amount of light captured will still be half.

Comment edited 2 times, last edit 58 seconds after posting
1 upvote
MistyFog

@itsastickup

And the Galaxy S4, with its Full-Frame DSLR-like photosites trumps the HTC One. Oh wait, it's not Full-Frame DSLR-like... but let's not be pedantic.

1 upvote
Lars Rehm

The HTC One's photosite size is comparable to that of a 'enthusiast compact camera' such as the Panasonic LX 7. However, sensor size itself is still the same as on any other smartphone, i.e. the total light gathering capability is no different.

1 upvote
Hobbit13

Very impressed by the results of the S4 as a camera. The image quality is definitely worse than the Nokia 808, but the S4 easily compensates this difference by running Android.

It's amazing how much detail they got from such a small sensor! (the Nokia has a larger sensor than many high end compact camera's, so no "magic" that it performs well)

2 upvotes
Jostian

yip, the S4 performs well indeed, pity about the loss of detail in low light but for the sensor size and MP's it does a great job.

1 upvote
Jostian

rating the S4's LED flash better than a Xenon (Nokia 808)... mmm, what a joke, DxO ???

2 upvotes
MistyFog

@Jostian

I suppose as a commercial enterprise it would be difficult for DxO to inform the millions of people who invested $700 on a spanking brand new S4 that their camera is worse than a phone from the digital Jurassic era.

I suppose if Symbian were the more popular platform today all the sudden Xenon will look brighter than LED flash.

0 upvotes
Sirandar

Brightness does not necessarily a good flash make ...

3 upvotes
MikeNeufeld30

Fantastic results IMO. I have yet to see a "relevant" flagship device outperform the gs4.

2 upvotes
Peiasdf

Good result but of the current generation, DXO only tested BlackBerry Z10 and Sony Xperia Z. Apple and Nokia haven't show their hands yet and they are the one to watch.

0 upvotes
TrojMacReady

The iPhone 5 is just over half a year old and is current generation. Maybe you're referring to future generation with regards to Apple.

Comment edited 1 minute after posting
1 upvote
Durandalfr

lol 82% for the tiny led "flash" while they give 81% to the 808 xenon flash. are they out of their mind?

10 upvotes
Jostian

+1, since when is an LED flash anywhere near a Xenon?? another DxO farse...

3 upvotes
thx1138

What's the colour gamut of this screen.

I read Qualcomm's new Mirasol screen ahs 94% AdobeRGB gamut. I'd guess these screens are maybe 40% at best.

1 upvote
Just another Canon shooter

I hope the gamut is not too high.

0 upvotes
Doug Pardee

Don't hold your breath on Mirasol. Qualcomm canceled production of Mirasol last July. Right now there's a lot of news flap over Qualcomm having unexpectedly shown a mock-up of a Mirasol-based smartphone, but Qualcomm has had a many-years-long history of showing Mirasol mock-ups and demo units but never delivering. Even Qualcomm has reportedly said that it's "a few years away from mass production."

Surprisingly (to me, anyway), even the mock-up didn't look all that impressive.

Comment edited 3 minutes after posting
1 upvote
mcantu

DisplayMate goes into the gamut details here... http://www.displaymate.com/Galaxy_S4_ShootOut_1.htm

Comment edited 2 minutes after posting
1 upvote
Total comments: 35
About us
Sitemap
Connect