Issue
In JUnit you can use @Ignore before methods to tell the test runner to automatically skip those tests. From what I can gather, this is really only a convenient way to document/mark off incomplete/no longer functional tests that you want to get back to later.
Am I correct in saying then, that at runtime there is no difference between an @Ignore test, a method with no annotation, and a commented out method? (Assuming these tests are all self contained.) Is there some way to obtain a list of the ignored test cases in JUnit on Netbeans? If not, how much usefulness does the @Ignore tag really have, since it might be more useful to fail the test so it's not overlooked?
Solution
Am I correct in saying then, that at runtime there is no difference between an @Ignore test, a method with no annotation, and a commented out method?
An @Ignored method can be found via reflection. A method with no annotation can't (or, to be precise, it can't be identified with certainty as an ignored test method), and a commented out method does not even get into the bytecode.
Albeit I don't think there would be much practical value in finding @Ignored methods runtime, it may be useful to generate statistics / reports.
how much usefulness does the @Ignore tag really have
One thing I can think of is searchability. You can easily identify all @Ignore annotations in the source code, while unannotated or commented out tests are not so simple to find.
it might be more useful to fail the test so it's not overlooked?
If you want (and can) fix it right away, it is fine to have it fail. There are cases when you can't, but you still want to method to be around, precisely so that it does not get forgotten. Then @Ignore makes sense.
Answered By - Péter Török
