Bench testing StereoCore™ PhotoLog (Part one)

When you use a tool for measurements, you typically want to know how much you can trust the measurements to be a true reflection of the real world. When using StereoCore™ PhotoLog for example, if it reports an alpha angle of 56 degrees for a structure, is that in fact true?

When one wants to check the precision and accuracy of a measurement tool, one way of doing this is by doing a bench test. What you do is take a set of measurements using your tool, and you compare them to their actual real world values, or at least, to the closest approximation to their real world values that you possess. The more measurements the better, obviously, as that lets you have confidence that your test results are not a fluke. Also you've got to be careful to test the tool thoroughly - to put it through it's paces. It's all very well if your ruler measures lengths of 1 m very very well, but fails on measurements less than 20 cm.

If your tool's measurements compare well to the known real-world measurements then you're home and dry. If not, you have to find the reason why not - this may lead to improvements in your tool or better guidelines for its use.

How we bench test StereoCore™ PhotoLog

When we first decided to bench test StereoCore™ PhotoLog we had a bit of a problem in that we'd need lots of measurements of lengths and angles in drill core, and we'd need to know that those measurements were accurate, and then we'd need to take photographs of the same core and measure the lengths and angles of the same structures and core segments using StereoCore™ PhotoLog. My major problem is with the goniometers and tape measures that people use to make these measurements - I think they're not accurate enough for testing purposes. 

If I saw a discrepancy between StereoCore™ PhotoLog results and the manually measured results there'd be a temptation (as the guy who wrote the software) to dismiss the manually measured results. It's bad because I can't be sure if the error is due to 
StereoCore™ PhotoLog.

We have actually done this kind of bench test for the first version of StereoCore™ PhotoLog using data we collected from one of the mine projects, and it is important to check that one's measurements do correspond to actual measurements in the real world, but in order to get reference measurements of sufficient accuracy and in order to get enough of them, we resorted to the following sneaky trick: we make photographs of core with known lengths and angles. We do this by generating 3D models of core and structures on a computer. I'm not the most hectic graphics programmer around, otherwise the synthetic photos would have really come out looking awesome, but I just wrote what's called a wireframe engine - and even that was basic for a wireframe engine - which could take pictures of my 3D models which I created.

An example of the kind of photograph my wireframe engine generates.

I've been doing this kind of bench testing for a while, since about midway through development of StereoCore™ PhotoLog. So I've discovered a few more sneaky tricks. Firstly, you as the bench tester mustn't know the lengths and angles you're trying to measure, otherwise you subconsciously fiddle with the results - you can't help wanting them to be more accurate, so instead of for example placing the structure as best you can and matching the curve on the photograph, you place it and fiddle with it until the reported alpha and beta angles match closely to the ones which you know should be there. Secondly randomizing the photographs is the name of the game. The idea is that you want data that matches closely to the kind of data that one would encounter in a real world drill core. So in the real world people take photos from odd angles, and they don't always align the reference line to the top of the core. The structure angles must also be random - this falls under the heading "testing the tool thoroughly" - if I only test a small range of structure angles and the tool works well to measure them, I haven't really checked that it works well in all situations.

I'm wrapping up the first bench test right now, the program has been performing beyond what I expected actually - it's pretty darned accurate. I will post the results shortly :)

Have yourselves a very merry festive season :) 

Regards - Dave (Member of the StereoCore PhotoLog Team)