|
Abstract: The accuracy of image based length and angle measurements made using StereoCore™ PhotoLog 3 was tested by processing a set of artificially generated photographs. The simulated core segments in the images were of known length, and the simulated structures had known (α, β) angles. The measured values were compared with the known values and the errors were calculated. The error data was used to calculate summary statistics showing that in this bench test the length of core segments was measured with a mean error of 3mm and plane poles were measured with a mean error of 4.0°, or 3.3° if α angles less than 5° are excluded. Introduction As part of our internal quality assurance process for StereoCore™ PhotoLog, we bench test the software to make sure that it is returning accurate measurements. This article presents the method and results of the second bench test for StereoCore™ PhotoLog 3. Bench testing of any measurement tool is done to evaluate its accuracy and precision. Accuracy is a measure of how close the measurement is to the actual real-world value, and precision is a measure of the measurement's repeatability. See this Wikipedia article for more details. In order to test this, we take a number of measurements of known quantities with the tool, and compare the known quantities with the measurements. In the case of StereoCore™ PhotoLog we are interested specifically in core segment length measurements and structure angle measurements. Once we have the actual and measured quantities we can calculate a number of descriptive statistics which give us some idea of how well StereoCore™ PhotoLog can be expected to measure lengths and angles. Bench Testing Method We are attempting to test how well StereoCore™ PhotoLog measures lengths or core segments and angles of core structures. The most straightforward way of testing this would be to take photographs of core segments and structures with known lengths and angles, and compare them to the values measured by StereoCore™ PhotoLog. Unfortunately measuring actual core segment lengths and structure angles with sufficient accuracy for a meaningful bench test is difficult, and especially in the quantities needed. To circumvent this problem, we decided to artificially generate the photographs, and simulate structures and core segments within the artificial core trays (see Figure 1). To remove the element of human bias the core segment lengths and the structure plane pole directions are randomized during the generation of the images, and an Excel file is generated containing the precise lengths and angles measured.  Figure 1: Artificial photograph showing simulated core segments and structures. The artificial photographs have simulated lens distortion and were taken from a randomized camera position as well, to more accurately simulate the kind of conditions that StereoCore™ PhotoLog is intended to be used in. The generated photographs were undistorted and calibrated as per StereoCore™ PhotoLog standard operating procedure, and then segment lines and structures were marked on each structure. Once "logging" was complete, the data was exported to Excel. The measured data was then compared to the original data and errors were calculated. Randomizing the data A concern that was raised when we last published bench test results was whether or not the input data was properly generated in such a way that the instrument would be fully tested. Specifically, the question of the distribution of α angles in the test sample was raised. In the original bench test we were concerned to test that the program would return reasonable results for any combination of (α,β) measurements. Accordingly, the structure data was generated so that the plane poles would fall evenly on a hemisphere as illustrated in Figure 2. However, this distribution does not have an even distribution of α angles, as shown in Figure 3, so for the second bench test we generated the data so that there would be an even distribution of α angles, as shown in Figure 4 and Figure 5.  Figure 2: Pole distribution for the first bench test.  Figure 3: α distribution for the first bench test.  Figure 4: Pole distribution for the second bench test.  Figure 5: α distribution for the second bench test. Results Length Measurements Mean absolute error was 3.1mm with a standard deviation of 2.0mm. Mean relative error was 0.6% with a standard deviation of 0.4%. 92.7% of the data was within 6mm and 100.0% of the data was within 10mm. A chart of the measured absolute error is shown in Figure 6.  Figure 6: Length measurement error. Angle Measurements In the table below we present two sets of statistics for angle measurements here, the one set includes α angles of less than 5°, and the other excludes them.  Table 1: Summary statistics for angle measurements. The really important columns of Table 1 are the ones labelled "Angle Diff". They show the summary statistics for the angular error between the measured and actual plane poles. Again we can look at cumulative relative frequencies for the data as well. For α angle measurements, 73.3% of measurements are within 2° and 91.3% are within 5°. For all data, the pole angle difference is less than 5° for 74.0% of the data and less than 10° for 98.7% of the data. After removing the structures with α < 5°, the pole angle difference is less than 5° for 74.7% of the data and less than 10° for 100.0% of the data.  Figure 7: α angle errors.  Figure 8: β angle errors. Something to take note of in Figure 8 is the strange shape of the histogram. Why are there so many β measurements with a high error? What's actually going on is that the β angle becomes harder to measure as α tends towards 90°. This is because the elliptical trace of the structure where it intersects the core barrel becomes closer to a circle. Since the β angle is defined relative to the semi-major axis of the elliptical trace, when there is no clear semi-major axis, measuring the β angle becomes impossible. Since the data for this particular bench test included lots of plane poles with a high α, naturally the β measurements were more varied. However, all is not lost, because it is the (α,β) pair that is important after all - the two measurements together define the plane pole. When we calculate the angle differences between the actual plane poles and measured plane poles, we find a much more satisfactory situation, as illustrated in Figure 9.  Figure 9: Pole angle error. As you can see, most plane poles are measured to within 10°, and excluding α angles of less than 5° removes the outliers, although there are only 4 measurements in 300 that were out by more than 10°, so it may be that there's a case for not even bothering with excluding those measurements with α < 5°. Caveats As always there are a few things to bear in mind with this data. This is artificial data and the structures that one sees in the core shed rarely look as pleasantly elliptical as the ones in the generated photographs. Conclusion Lengths were on average measured to within 3mm and 93% of length measurements were within 6mm. For angles the situation is slightly more complicated because it is the plane pole measurement that we're interested in, i.e. the (α,β) pair rather than the individual α and β values. Looking at the angle between the measured and actual plane poles, the mean measurement error was 4.0°, and 98.7% of measured plane poles were within 10° of the actual poles, in fact only 4 measurements out of 300, all with very low α angles - less than 2° (as measured by StereoCore™ PhotoLog) had angle errors greater than 10°. Looking at α and β separately, the mean measurement error for α angles was 1.5°, with a standard deviation of 1.8°. Compare this with β, where the mean measurement error was 10.8° and the standard deviation was 21°. Despite the large errors in β, the plane poles were still measured accurately to within 10°. This is explained by the fact that as α gets close to 90°, β becomes much more difficult to measure but also has less of an effect on the direction of the plane pole. It also serves to drive home the point that (α, β) measurements must be considered as a pair. The result of this bench test shows that StereoCore™ PhotoLog measures both lengths and angles with a reasonable degree of accuracy and precision. David Orpen StereoCore™ PhotoLog Lead Programmer You can download a pdf copy of this article here: 
0 Comments
Leave a Reply. |
Categories
Archives
August 2018
|
||
RSS Feed