Transmutator4j Overview

Transmutator4j is a Java Mutation Testing library that dynamically modifies byte code during class loading when running your tests. If the tests still all pass, then you know the portion of the code that was modified is not properly tested. (Or in rare cases, not needed).

What is Mutation Testing?

Mutation Testing makes small changes to the software under test's code and then runs the software under test's test suite. Since the software has been modified, there should be at least one test that now fails. If all of the tests still pass, then the software is not well tested.

Transmutator4j Features

• Uses a custom classloader that will modify byte code for the system under test during class load time - no recompilation required.
• Exercises the entire given JUnit test suite, not just a single class, to see if anything breaks.
• Stops running the test suite as soon as a failing test is found. This greatly speeds up the runtime of Transmutator4j.
• Will detect if a test is taking too long to run and kill the process. This usually happens if the mutation causes an infinite loop. Infinite loops are considered "test fails" and therefore a well covered test.
• Able to run on already compiled .class or .jar files assuming you also have access to the JUnit tests (which also can be pre-compiled)

Mutations Transmutation4j Makes to your code

Arithmetic mutations

• Java opcodes for int, long, float and double addition, subtraction, multiplication and division are mutated into their opposite opcode. For example, the "int add" opcode is turned into an "int subtract" opcode.
• Java opcodes for the modulus operation are turned into multiplication opcodes.

Boolean mutations

• The constants for boolean true and false are mutated to their inverse.
• Since the JVM uses the constants 0 for false and 1 for true, this also means any hardcoded values of 0 or 1 will also be mutated. So int myCounter =1; will mutate into int myCounter =0; and vice versa.

Increment Mutations

• ++ will mutate into --
• -- will mutate into ++

##Conditional Jump Mutations Since all conditional statements are compiled into conditional jump opcodes, Transmutation4j can mutate conditions inside if/else blocks, and while and for loops by only mutating the jump condition. The end result is the same as mutating if(condition) into if(!condition).

• == becomes !=
• != becomes ==
• == null becomes != null
• != null becomes == null
• >= becomes <
• < becomes >=
• <= becomes >
• > becomes <=

#How to run Transmutator4j

How to run Transmutator4j from the Commandline

Transmutator4j's main jar is called transmutator4j-core.jar (or the full jar containing all dependencies: transmutator4j-core-jar-with-dependencies.jar) which contains an main method in the class net.transmutator4j.RunTransmutator4j

RunTransmutator4j takes several parameters:

 -include <classes>   regular expression of classes to transmutate
 -exclude <classes>   regular expression of classes to NOT transmutate but would otherwise be included
 -out <out>           xml output file to write results
 -src <src>           source folder(s) to transmutated classes
 -test <test>         run given unit test or test suite
 -timeout <timeout>   number of milliseconds to wait for before
                  considering tests have mutated into infinite loop.  By default timeout is 2* the 
                  runtime of the given un-transmutated test suite 

so to run Transmutator4j from command line invoke this command

 % java -classpath <classpath> -jar transmutator4j-core.jar -include <regex> -exclude <regex> -out <path/to/out.file> -src <path/to/src> -test <qualified class name of test suite to run>

How to run Transmutator4j as an Ant Task

Transmutator4j has an optional jar transmutator4j-ant.jar that contains code to run Transmutator4j as an ant task. By including both the transmutator4j-core.jar and transmutator4j-ant.jar in your classpath, you can can run Transmutator as an ant target by adding this to your build.xml file:

<taskdef name="transmutator4j" classname="net.transmutator4j.ant.Transmutator4jTask" 
                                classpathref="test.classpath"/>
	
 <target name = "mutation-tests">
	<transmutator4j
		classpathRef="test.classpath"
		include="my\.project\.Foo\..+"
                    exclude = ".*Test.*"
		outputfile="transmutator4j.foo.xml"
		testsuite="my.project.TestFoo"
	/>
</target>

The transmutator4j target will transmutate all classes in the package my.project.Foo whose class names do not have the word "Test" in them.

How to run Transmutator4j as a Maven plugin

Coming Soon

How is Transmutator4j Different than other Mutation Testers?

When Transmutator4j was written, there were few non-commercial mutation test libraries available. The few programs that were available had problems:

• Jester makes mutations on the actual source code. This is very slow since after each mutation, the source has to be recompiled. In addition, if the program crashes or the user kills the mutation program before it completes, the sourcecode is left in a broken state. Jester recommends that users mutation test a copy of their source for that reason. Also since the source is required, Jester is unable to run on java code that the source is not available or not compilable. Finally, Jester executable displays its status and mutation results in a Swing dialog which makes it difficult to incorporate in a Continuous Integration system.

• Jumble is able to mutate byte code like Transmutator4j does. However Jumble uses the BCEL library to read, mutate and write java byte code. Transmutator4j uses the ASM library to modify byte code which is faster than BCEL. Jumble will only find and run test classes that follow specific naming conventions (usually only the tests for the class that was mutated). Transmutator4j runs the entire test suite every time in case only a dependency breaks by the modification.