Bench Testing StereoCore™ PhotoLog 2.0 (Part 2)

This post is a continuation from Bench Testing StereoCore™ PhotoLog Part 1.

The results from the first StereoCore™ PhotoLog 2.0 bench test are now available. In summary, for a sample of 300 length measurements between 45 cm and 55 cm the maximum absolute error encountered was 9 millimetres. The average was 3 mm with a standard deviation of 2 mm. In relative error terms, for length measurements the maximum relative error was 1.6% and the mean relative error was 0.6% with a standard deviation of 0.3%. This means that length measurements made with the program can be expected to be within 1%.

A small systematic error was detected in length measurements, which I am inclined to attribute to the human element - myself.

For a sample of 300 angle measurements, 3 were disqualified as the generated photograph was not sufficiently clear to take a reading. A further 31 were excluded after analysis showed that StereoCore™ PhotoLog 2.0 is not so hot on measuring structures which have StereoCore α angles of less than 5 degrees. Thus for a sample of 266 structures which had StereoCore α angles greater than or equal to 5 degrees, the results (see Figure 1) are as follows: the mean angular error in measuring the plane pole direction was 2.5 degrees, with a standard deviation of 1.7 degrees. The maximum angular error encountered in measuring the direction of a plane was 9.4 degrees. This means that plane pole measurements made with the program can be expected to be within 4 degrees, provided that the StereoCore α angle is greater than 5 degrees. For an explanation of the StereoCore™ PhotoLog angle convention see this blog post.

Figure 1: Angular error in 266 structures (with α angle > 5 degrees) measured using StereoCore™ PhotoLog 2.0.

It should be noted that the angular error is calculated by calculating the angle between plane poles, not by directly comparing α or β measurements. However, looking at the α and β angles separately, the mean error in measuring the α angle was 1.0 degrees with a standard deviation of 1.3 degrees, and for the β angles the mean and standard deviation were 3.5 and 4.8 degrees respectively.

For the 31 excluded measurements with α less than 5 degrees, the mean α angle error was 0.4 degrees with a standard deviation of 0.3 degrees, so StereoCore™ PhotoLog 2.0 is still reliable for α measurements even though for β measurements the results are not as reliable for α angles less than 5 degrees (specifically, for structures with α angles < 5 degrees the angular errors - calculated using the plane poles - had the following statistics: Mean: 19.6 degrees. Standard deviation 29.7 degrees. Minimum 0.3 degrees. Maximum 91.5 degrees.)

Discussion

Obviously we're pretty chuffed with these results. To be getting length measurements mostly within millimetres, and angle measurements within a couple of degrees for the plane poles is pretty good, and to be able to prove it is also great.

There are a couple of things that need mentioning. For a start, there is a human element involved in these measurements as measurements are taken by a process of a user marking features on a photograph. The best we can hope for is to make it easy for the user to make measurements. Secondly, for the reasons discussed in Bench testing StereoCore™ PhotoLog 2.0 Part 1, artificially generated photographs were used to obtain these bench test results. In the field structures may not be so well behaved as to have beautiful, perfect ellipses. Lastly, the bench test does not cover all test cases - in particular the one missing test case is for what happens when you can't see the whole structure. In this bench test all structure ellipses were fully visible.

During the bench test I did notice that the photographs not taken from close to vertically above the core tray had the largest errors in measurements, so one message to take away from the bench test results is that being careful to take photos from vertically above the core tray will improve the accuracy of your data.

Bench testing is an ongoing procedure and there is still scope for a lot more study in this area for StereoCore™ PhotoLog 2.0. The good news is that StereoCore™ PhotoLog is extremely accurate for measuring both lengths and angles.

Cheers for now :)

Dave (member of the StereoCore™ PhotoLog Team).