Friday, May 23, 2008
The Curse of Proof by Example
What is a proof that a method in image analysis (machine vision) does what it claims to do? For example, how do we know that a given method of image retrieval will retrieve images similar to a given query? Unfortunately, the publications standards are such that most papers present only proofs by example.
This a big topic but I cannot help add one more "joke" to the list "How X proves that all odd numbers are prime?" (see http://www.gdargaud.net/Humor/OddPrime.html) A machine vision person's proof: "3 is prime, 5 is prime, 7 is prime, 9 hmm, 11 is prime, 13 is prime, oh well, we have an 83% success rate, let's publish."
The problem with machine vision publications is not only that the reported success rate may be too low for practical applications. It is also that the choice of examples is fairly sloppy with little thought given on how representative are of the of the population of interest.
This a big topic but I cannot help add one more "joke" to the list "How X proves that all odd numbers are prime?" (see http://www.gdargaud.net/Humor/OddPrime.html) A machine vision person's proof: "3 is prime, 5 is prime, 7 is prime, 9 hmm, 11 is prime, 13 is prime, oh well, we have an 83% success rate, let's publish."
The problem with machine vision publications is not only that the reported success rate may be too low for practical applications. It is also that the choice of examples is fairly sloppy with little thought given on how representative are of the of the population of interest.
