Histogram Myth explicated:
By “Mike” Michael L. Baird mike {at] mikebaird d o t com
19 Feb 2012: Join the related discussion at naturescapes.net (ref. this https://photomorrobay.wordpress.com/ )
Many photography instructors and experts in the field are innocently and incorrectly making inaccurate claims, such as those three below, about how to interpret a histogram as seen on your camera’s LCD. The representation that the “tonal x-axis value” is somehow logarithmic in scale, as suggested by bringing “stops” or “amount of light collected” into the discussion, is the cause of confusion. By the very definition of what the histogram is, we should know that it is just a linear scale. We can clearly see that this is the case just by watching the histogram shift on the camera LCD while stopping down. I’ve known otherwise good photographers to give up trying to understand things in this field because they can’t comprehend what’s being thrown at them when histograms are being discussed by the experts… all the experts couldn’t be wrong could they? Well, in this case, yes. Those of you previously driven to insanity can now come back to school.
Claim: A 12 bit image is capable of recording 4,096 (2^12) discrete tonal values. One would think that therefore each F/Stop of the 5 stop range would be able to record some 850 (4096 / 5) of these steps. But, alas, this is not the case. The way that it really works is that the first (brightest) stop’s worth of data contains 2048 of these steps — fully half of those available… This realization carries with it a number of important lessons, the most important of them being that if you do not use the right-hand fifth of the histogram for recording some of your image you are in fact wasting fully half of the available encoding levels of your camera. [Michael Reichmann] [repeated by Ron Day]
Wrong: losing half of your light by stopping down one stop, doesn’t mean you will lose half your tonal values.
Claim: “You’ll lose half your tonal values for each one full stop of under-exposure.” Wrong, losing half of your light by stopping down one stop, doesn’t mean you will lose half your tonal values.
Claim: “Fully half of the tonal values are in the brightest fifth of a histogram. So, if you don’t have at least some pixels heading out into that rightmost fifth of the histogram, you’re wasting half of the potential tonal information that your camera can capture!”
Claim: “If we array the stops of dynamic range along our monochromatic or luminance histogram, we’ll notice that each stop (from left to right) contains two times more information than the previous stop. And notice also that most of the possible color values are in the brightest areas. That is, our camera can capture only a relatively few dark tonal values and lots of bright tonal values.” [Greg Basco]
Illustration: An example of the myth being propagated.
Histogram Stop Myth Illustrated - this is not a correct interpretation
Illustration is courtesy of, and many thanks to, Costa Rica photographer Greg Basco who was great in helping me to finally understand this issue as he tolerated my barrage of challenges with grace and patience.
In a sense this whole argument is silly, because if you define a histogram to be a plot of brightness or “tonal” values ranging from 1 to N, and the horizontal axis is treated as and shown as linear, N/2 being in the center, etc., then saying that “there are more tonal values in one part of the histogram than another” is like saying there are more years in the last 1/5th of your life than the first 1/5th.
I’ve seen this myth repeated over and over for years throughout the literature. This stop metaphor applied to the interpretation of histograms is misleading and inaccurate, and may teach photographers poor technique. The resultant implied prescription to not unnecessarily under-expose, because you’ll lose up to half your tonal values in just a one stop change, is quantitatively false, and may encourage photographers to overexpose their images in avoiding loss of tonal information that is in fact not as at-risk as implied.
By stopping down one stop, you are indeed capturing only one-half the number of photons, but you are not losing one-half of your measured and perceptible tonal values. There is no argument that we should almost never intentionally over-underexpose an image if we can keep all pixel values within the histogram and not blown out to the right (at least if the contrast range of the scene you are shooting is less than the dynamic range of the camera’s sensor, and you are sure you are not blowing out any of the brightest pixels… not so easily done or verifiable in practice) . Collecting more light will increase the signal and information in the scene, and minimize noise (although at the trade-off of tolerating a longer shutter speed or wider aperture – but that’s getting us off the point at hand).
<<< below paragraph added for clarity 14 Feb 2012>>
Understanding the definition of Lightness is essential to understanding the histogram on your camera and as seen in Photoshop, etc., and thus accepting the fact that lightness is not doubled or halved when one aperture stop adjustment is made, doubling or halving the amount of light impinging on the sensor. Wiki says “… scientists… converged on a roughly cube-root curve, consistent with the Stevens power law for brightness perception, reflecting the fact that lightness is proportional to the number of nerve impulses per nerve fiber per unit time… at first glance, you might approximate the lightness function by a cube root, an approximation that is found in much of the technical literature. However, the linear segment near black is significant… an 18% grey card, having a reflectance of 0.18, has lightness very close to 50. It is called ‘mid grey’ because its lightness is midway between black and white.”
To learn more about the 18% grey card measure see http://en.wikipedia.org/wiki/Middle_gray “… middle gray or middle grey is a tone that is perceptually about half way between black and white on a lightness scale;[1] in photography, it is typically defined as 18% reflectance in visible light.[2]”
The following chart can be thought of, in the first order of interpretation, as the related plot on the horizontal axis of the amount of light impinging on the sensor (reflectance), and the vertical axis representing the perceived lightness or brightness in the human visual perception system. “Yn is the Y tristimulus value of a ‘specified white object.'”
You can pretty much see this curve in action if you just open a Canon CR2 RAW file in Canon’s DPP Digital Photo Professional – clicking the “linear” check box under the “RAW” tab to see what the sensor has recorded, and then un-checking “linear” and see how the transformation/relationship curve below changes the RAW reflectance values into perceived brightness or tonal values.
http://en.wikipedia.org/wiki/Lightness_%28color%29
Observe that the lightness is 50% for a luminance of around 18% relative to the reference white.
A histogram is, by definition, a plot of frequency-of-occurrence of perceived brightness [~Lightness] or tonal values of pixels, and you can see for yourself, say using live view (in a camera that has say just 5-stops of dynamic range) that as you stop down your camera, for each stop, the histogram shifts smoothly and linearly to the left by one-fifth of the horizontal-axis tonal value range.
That’s by definition of what the histogram does – it plots the frequency of occurrence of tonal values or perceived brightness of the pixels in an image.
<<< below paragraph added for clarity 12 Jan 2012>> The key to understanding is to first define what the histogram is supposed to represent, including the terms used in the definition. So.. if the definition of the histogram in this context is “the frequency of occurrence of tonal values plotted linearly” then any claim that the horizontal-axis is not linear would immediately seem silly on the surface, as it indeed is. And you can really clearly see in Liveview that the histogram as implemented by camera manufacturers, just scoots along in a nice linear fashion, as the lens is stopped down. The histogram isn’t an abstract concept or metaphor, it’s a simple data visualization tool. Each pixel in an image has a tonal value as perceived by a human in say a grey scale chart test pattern with 256 gradations from left-to-right. The values range from say 1 or black at the left, to say 256 or white at the right. If the brightest pixels are 256, and the darkest are 1, then 128 represents the tonal value of the center pixels of the test pattern. There are by definition 256 tonal values. To state that fully half of these values are found in the upper 1/5th or 1/8th portion of the plot (depending on your dynamic range model) is silly. So once again, “losing half of your light by stopping down one stop, doesn’t mean you will lose half your tonal values.” <<< this paragraph added for clarity 12 Jan 2012>>
The dynamic range of the camera is totally accommodated and compensated for by this point in time in the production of a linear brightness scale (~luminosity) seen in the histogram. It is unimportant, for purposes of this argument, where and how this conversion takes place (the more usual place for this to occur is in RAW conversion).
It is easy to see how the “stops” myth started, because people did recognize that a lot more photons were being measured and recorded in the brighter regions of an image because of the non-linear response of the human visual perception system to reflected light.
The myth arises primarily because (1) people try to make photography more complicated than it is, and (2) it isn’t appreciated that human perceived brightness is a “logarithmic” function of the number of photons detected and recorded by a sensor. Doubling the number of photons hitting pixels (by changing aperture by one stop) does not double the perceived brightness. Doubling the intensity (photon density) of a point of light does not double our perception of it’s brightness (this is a biological phenomenon). And similarly, by design, the conversion of voltages read out from a sensor chip eventually takes this relationship between reflectance in visible light and the perception of brightness or lightness, into consideration. If shooting JPGs in-camera, or in the rendering of JPG preview files for RAW images, for LCD viewing on the camera, then RAW conversion is done for you, in-camera, using the reflectance/brightness relationship being discussed. When shooting RAW files for your workflow, this conversion is usually done during RAW conversion in software on a computer, as part of your workflow. (But, (and we are getting off track her) in theory, it would certainly be possible to do some of the transformation and processing in-camera (this would depend on the camera manufacturer’s implementation, especially if not using the only real RAW standard, DNG. Most RAW and all DNG images would be expected to record more reflectance values in the brighter regions. But remember, reflectance isn’t brightness!).
In the end, the histograms we see on the camera LCD and in Lightroom or Photoshop are just plots of the perceived brightness vs. frequency-of-occurrence of pixels.
Because photon-density vs. brightness is logarithmic and not linear in human perception, doubling the amount of light collected (or doubling the number of photons counted or equivalently the signal voltage created) does not double the perceived brightness (tonal level) quantized by the camera’s electronics and written into RAW or jpg values. By definition, our camera and Photoshop histograms display the distribution of pixels by perceived brightness.
Laboratory assignment. Prove this yourself.
To show this empirically, I downloaded a 10-tonal-level grey scale chart from the internet, which has ten vertical intensity bands, then opened it in Photoshop, viewed Histogram, and selected regions over each pattern.
The histogram looks like a flat or evenly pulsed line, since the grey-levels are distributed evenly and the frequency of occurrence of pixels in each of these tonal value zones is equal in number to any other because of the design of the test pattern.
Photoshop quantizes the histogram view into 256 columns/containers for display purposes.
The RBG tonal values will be linearly different from sample pattern column to sample pattern column by about 28.4 on this projection to 256 plotting points (256/9), or 11% (100/9)
I measured (in 8-bits/channel RGB mode, these mean values)
0, 28, 56, 85, 113, 141, 170, 198, 226, 255
and converted to Grey-levels and measured in these percentages:
0, 15, 28, 41, 53, 64, 75, 84, 92, 100
Another example using a 21-tonal level test pattern
yields the same, as similarly expected, these linear values…
0, 13, 26, 38, 51, 64, 77, 89, 102, 115, 128, 140, 153, 166, 178, 191, 204, 217, 229, 242, 255 (~13 difference per step or 256/20=12.8)
So again, the top 1/5th values, in these cases,
[226, 255] and
[204, 217, 229, 242, 255]
range across 20% of the tonal range, not the 50% as claimed.
QED
15 Feb 2012 here are some notes I’ve included in private responses that may be useful to preserve.
I basically agree with Jason Row on exposing to the right here, Click Here: Why Shooting to the Right Gives You Better Final Images “
and he does not make the myth mistake https://photomorrobay.wordpress.com/ of stating that losing one stop loses you half your tonal values.
f 11
Focal length 50 mm (which would be 80 mm full frame)
Distance from wall 1.1 m (3’10”)
TripodI varied the exposure in 1 stop increments for a total of 8 exposures. Here are the shutter speeds and the sizes of the files for each exposure. These exposures were chosen to to produce data on my histogram from the far left to the far right.1/500 sec 18.55 MB
1/250 sec 19.18 MB
1/125 sec 20.18 MB
1/60 sec 21.85 MB
1/30 sec 24.58 MB
1/15 sec 27.17 MB
1/8 sec 30.51 MB
1/4 sec 33.72 MBSo the size of the files I’m assuming is more or less a linear progression (I have not plotted it). If the myth were true, based on a far right value of 33.72 MB, the file size on the far left at 1/500 sec should be 2.11 MB, not 18.55 MB. This is probably easier to understand if you can see the histograms. I hope you are able to see the two PowerPoint slides that I have attached. I’ll send it as an online Office document, which Hotmail says everyone is supposed to be able to view. Let me know if you can’t see it. Pat
Histogram Experiment.ppt
1/250 sec 19.18 MB
1/125 sec 20.18 MB
1/60 sec 21.85 MB
1/30 sec 24.58 MB
1/15 sec 27.17 MB
1/8 sec 30.51 MB
1/4 sec 33.72 MB
Resume Bio
“Mike” Michael L. Baird is an enthusiast photographer doing nature, wildlife, people, and surf photography along the California Central Coast. He is a CA State Park volunteer docent, and retired Silicon Valley Internet entrepreneur, now living in Morro Bay, CA, where he encourages collaboration via http://photomorrobay.com http://bairdphotos.com http://flickr.bairdphotos.com and http:// morro-bay.com
Mike has an MBA, and a PhD in Computer Science, and was the first VP of Engineering of the $7 billion ask.com search engine company.
He is author of the 20-year best-selling entrepreneurship book entitled “Engineering Your Start-up: A Guide for the High-Tech Entrepreneur” http:// eysu.org.
Contact Mike at mike [at} mikebaird d o t com. (805) 704-2064.
http://bit.ly/MyGoogleScholarCitations lists some of my image-processing related patents and referred papers.
285×285 heqdshot at http://www.flickr.com/photos/mikebaird/6556151331/
Photomorrobay 