How We Test Printers
Like all of the sites in the reviewed.com family, our reviews here at Printerinfo.com are designed to be the most scientific, accurate, objective and in-depth review you’ll find on the Internet. We use the latest scientific methods to analyze every aspect of the performance of a printer, testing it in a number of ways to ascertain how well it does the job of producing attractive prints. Our tests are objective; we rely on scienitific measurements rather than people's perceptions of prints. We feel that this is prefereable to a set of so-called "golden eyes", where a tester looks at a print to judge color, etc. People have preconceptions, and the human eye changes over time, but out tests don't, meaning that we can test two printers months apart and be able to still compare them. We also test all printer the same way: every printer gets put through the same set of tests, irrispective of it is a cheap printer or a large, expensive professional model. Many of our tests have been developed in co-operation with Norman Koren of Iamtest, and we gratefully acknowledge his assistance. NOTE: this description of our testing process reflects the testing procedure as of April 2009; reviews produced prior to this time were produced using a different process, and the results are not directly comparable.
Equipment
X-rite i1 Xtreme - To accurately measure colors, we use an X-Rite i1 XTreme spectrophotometer. This professional color measurement device allows us to measure the colors that printers produce very accurately without relying on outside light sources that change: the 1IXtreme includes its own tightly calibrated internal light source.
 |
| GamutVision in action |
Imatest & Gamutvision - We use two programs to help us analyze the quality of prints: Imatest and Gamutvision, both produced by Norman Koren. These two programs allow us to do in-depth analysis of the color accuracy, depth of black and color gamut of the printers that we test.
Tests
Print Speed
We do two print speed tests: a photo print and a document print speed test.
In the photo print speed test, we measure how long the printer takes to output a reference photo at a number of different print sizes. To test the print time of the device itself, we do five prints, and measure the time between the second print beginning and the final print being ejected from the printer, and divide the time by 4. We measure the time at the highest quality settings that the printer offers, and our score is based on the time to produce a standard 4 by 6 print: the shorter this time, the higher the score.
For the document print speed test, we use the ISO 24712 document, which is designed to mimic a typical business document. We print this PDF document five times, measuring the time between the second print run beginning and the final print run ending, and divide that time by 4 to see how long each print run of the document took. We then divide this by 5 (the document is 5 pages) and calculate the number of pages per minute that the printer can produce. Our score is based on this number; the larger the number of pages per minute, the
Color Accuracy
 |
| Our virtual color chart |
To test the color accuracy of printers, we use a special image that contains 24 colors that are commonly found in photos. These represent things such as skin tones, flowers, blue skies, etc. We then print this image three ways: using the printer’s default color profile onto the manufacturer’s recommended paper, using a custom profile created with the X-rite i1 onto the same recommended paper, and onto Ilford Galerie Smooth Glossy paper, with another custom profile. We then measure the colors on all of these prints, and find the difference between the original colors in the image and the printed result. We then use the distance between the two to calculate the color difference, using the CIDE 2000 color difference formula to work out the ΔE 00 value. We use this approach as the CIEDE 2000 formula more accurately models how the human eye perceives color than the various other color error formulaes. Wikipedia has a good explanation of the various color formulaes here, and Norman Koren has a more in-depth explanation of the CIEDE 2000 formulae here. We Then produce a mean of the 24 color patch errors, and our scoring is based on this; the larger the mean, the lower the score.
Color Gamut
 |
| The Gamut and DMax test image |
The color gamut of a printer indicates the range of colors that it can reproduce; the wider this gamut is, the better job the printer will do of reproducing the wide range of colors that there are in this big, beautiful world of ours. Our test of the color gamut on printers uses another special image that contains the entire range of the Adobe RGB color gamut. We print this out using the same three combinations of paper and profile that we mention above. We then scan the resulting print (using a high-end Epson Perfection V700 scanner) and run it through GamutVision, which produces the graph that you see in our reviews. This graph shows the Adobe RGB color gamut (the grid) and the color gamut that the printer can accurately reproduce (the solid block in the middle). We then determine the percentage of the Adobe RGB gamut that the printer can manage; we typically see figures in the range of 30 to 60%. Our score is based on this percentage; the bigger the percentage, the higher the score.
Depth of Black (DMax)
We also test the depth of the blacks using the same gamut testing image, which contains several areas of black. Gamutvision measures the density of these blacks, and produces a figure for the mean dMax. The larger this number, the deeper and denser the blacks in the print are. Again, we test this using all three test combinations, and our score is based on the combination of profile and paper that produces the highest dMax. The higher the dMax, the higher the score.
Detail
All of the tests above ascertain the ability of the printer to reproduce color, but printers also need to be able to accrately reproduce the fine details in photos and documents. To test this, we print out a number of test files, scan the results and closely examine and compare them. These images were specially chosen to contain small, fine details that are difficult to reproduce, such as small lines (in the Alice woodcut), slanted edges and small details in photos. We then score the printer based on how good of a job it did of reproducing these images; printers that do a job of accurately representing them get a high score, while those that loose detail or produce an inaccurate representation of it get low points.