Authors: Ulrich Krömer and Stefan Steiner
This article explains how Silk4J tests can be executed within Team Foundation Server (TFS) 2012. First it describes which tools and programs have to be installed to build Java applications within TFS and how to access them from within Eclipse. Following that is a description of how to set up a Silk4J project inside TFS. Finally, the article explains how to execute those tests and store the results in TFS and how to enable TrueLog results.
Prerequisites
Install Microsoft’s Build Extensions on the TFS Build Controller (2010 or 2012) in order to enable building Java projects with Ant or Maven (http://visualstudiogallery.msdn.microsoft.com/45b4e378-5b81-4186-9b7f-21cc5c2f743d for TFS2012 or http://visualstudiogallery.msdn.microsoft.com/2d7c8577-54b8-47ce-82a5-8649f579dcb6 for TFS2010).
Install JDK (not JRE) and set JAVA_HOME on the same machine.
Install Ant (or Maven) and set ANT_HOME on the same machine.
Install Silk Test on the build controller.
Install Team Explorer Everywhere on the client machine, which is the machine where the tester works, and integrate it into your Eclipse environment (http://www.microsoft.com/en-us/download/details.aspx?id=30661).
Note: When Eclipse is restarted, you will not see many changes. The TFS-related windows are more or less hidden by default. You can show them by selecting Window > Show View > Other > Team Foundation Server. Connect to TFS from within this window.
Creating a Silk4J project inside TFS
Setting the project up
Create a new Silk4J project inside Eclipse.
Connect to TFS by right-clicking your project and selecting Team > Share Project. A dialog asks you for the TFS settings (URL & credentials). Supply them correctly and click Next.
You get a list with your team projects. Select the team project you need and click Next. In the list you see now, select the Team Foundation Server workspace to which your project will be connected and click Next (of course you can also change the default values of your workspace). The last step is to select the folder for your shared projects (the project folder path). Click Next. The last dialog displays an overview of your selected settings which you can confirm by clicking Finish.
Check the newly created project in.
Setting the build up
Create an Ant file which contains the definitions for building and for executing the Silk4J tests. See Appendix A for an example. Note: TFS sets some special properties for the Ant file which you can utilize with the help of condition properties. In our example file we make use of the BinariesRoot folder.
Test your Ant file locally (build and run tests) and check it in when it works. Note: The default target should build, deploy, and test as TFS will only call this target.
Create a build definition from the created Ant file. The following webpage describes which steps you have to take: http://www.alexandervanwynsberghe.be/tfs-build-automation-for-a-java-project-using-ant-2/. Follow the instructions in the Create a build definition section to create the build definition.
Make sure that the JUnit JAR and the JTF JAR are also deployed to the build controller machine (possibly by adding them as libraries to the project under source control) so that they are found during test execution.
Now you are ready to schedule builds which already execute Silk Test tests. The test results will automatically be attached to the build. Note: The JUnit result collector will search for the test results in the BinariesRoot folder, so you have to make sure the test creates the JUnit result there. Check the Ant file in Appendix A for an example.
For a more detailed result analysis, you can view the result from within Visual Studio.
Enabling TrueLog for your Silk4J tests
Create a TestSuite with the SilkTestSuite annotation to make use of the Silk Test test runner that can write TrueLog files. See Appendix B for an example.
Update the build definition so that the TrueLog file can be stored along with the JUnit results and thus be archived together with the build. In order to do this you have to first check out the build definition which is located in TFS under Team Build Files/$Your Build Definition name$.
Open the TFSBuild.proj file and overwrite AfterPublishJUnit as shown in the example in Appendix C. Note: By default, the .xlg file will be created in the directory where the BuildType is defined and it has the name of the TestSuite that is executed.
Conclusions
By using this approach it is possible to execute Silk4J tests within a TFS environment and store the test results in TFS.
Limitations
The tests are executed on the build controller machine and are not distributed to the test agents. This would be the case if Silk4NET tests were executed through MSTest. You can best compare this scenario to a Jenkins/Hudson environment where some slave build controllers are responsible for building, while others are responsible for executing functional tests. Instead of having just one build definition for the build and for the tests you will therefore need two: one for executing the build and one for executing the tests.
The TrueLog results do not get attached to the test results within the test results view, as would be the case with Silk4NET. This is also due to the fact that the tests are run on the build machine and not as tests on a test agent where DataCollectors would be executed.
Appendix A
Appendix B
import org.junit.runner.RunWith;
import org.junit.runners.Suite.SuiteClasses;
import com.borland.silktest.jtf.SilkTestSuite;
@RunWith(SilkTestSuite.class)
@SuiteClasses(TestImdb.class)
public class TestSuite {
}
Appendix C