Last modified 7 weeks ago Last modified on 2017-02-06 22:04:51

ComTest » ComTest for C#


From this page you'll find step-by-step guidance for installing the ComTest tool to use with Visual Studio.

Please note: Comtest is in early alpha stage so bugs may occur. Keep calm and report your sorrow to (at)

Tutorial video in Finnish: Vaikka videossa sanotaan ettei toimi ilmaisversioiden kanssa, niin Visual Studio Community Edition 2015 kanssa toimii: Katso ComTestin asennus videona (07:07 min).

System requirements

  • ComTest uses Visual Studio's built-in test system and requires Community, Professional, Premium, Enterprise or Ultimate version of Visual Studio.
  • Java


Following list is only Quick Reference. Do not start with them, go further and follow more detailed (Step 1 to Step 4) instructions carefully!

  1. Download and install Java
  2. Download ComTest.jar (hakee automaattisesti kun käytetään 1. kerran)
  3. Download plugin for Visual Studio / Xamarin
    • (Note: Make sure the file extension is .vsix (not .zip).
  4. Download ComTest.snippet file and import it into Visual Studio snippets (Tools -> Snippet manager -> Import). (VS 2010 users only: use installer.)

Save these e.g. to C:\Mytemp\ComTest.

Avoid path names with spaces. They will cause problems with installation.

Without any installation you can try and use ComTest for C# just by Copy/Pasting your code to test to TIM's ComTest-template

Step 1: Java

ComTest is writen by Java and that for Java runtime is need to run it.

  • Java must be installed before using ComTest. Before installing ComTest please check that you have Java.
    • On Windows: Go to "Control Panel" >> "Programs and Features" (Ohjelmat ja toiminnot suomeksi) , there should be Java(TM) 6 (or some never version of Java)
    • You can start the old Control panel from command line by writing Control.

Step 2: ComTest.jar

You can jump over this step and return back here if this is not loaded on first ComTest run.

  1. Save ComTest.jar somewhere on your machine, e.g. to C:\Mytemp\Comtest
  1. Note down the file's path (e.g. C:\MyTemp\ComTest\ComTest.jar), for you will need it when you are configuring the Visual Studio plugin in Step 2.3
  • Avoid path names with spaces. They will cause problems with installation.

Step 3: ComTest plugin for Visual Studio 2010-2015

Requirements: Visual Studio 2010-2015 Community, Professional, Premium or Ultimate version installed with C# language support. This plugin does not work with previous versions of Visual Studio or with other languages such as Visual Basic or Visual C++.

  1. Download and run the plugin installer.
    • Follow the installer's instructions and relaunch Visual Studio.
    • If the installation was successful, you'll see a new ComTest item in the Visual Studio's Tools menu when a C# project is open.


  • If the ComTest does not appear under Tools-menu, check if it is under Tools/Extension Manager.... If it is, enable it and restart VS.
  • If you have older version installed, uninstall it from Tools/Extension Manager/All.
  1. Open the ComTest configure panel from the VS menu: Tools >> ComTest >> Options

  • A pop-up opens:

  1. Under Path to ComTest.jar executable, write the full path of the file (this is the path you made a note of during Step 1.2). Make sure to include the .jar. For example:

In Finnish: Agoran mikroluokissa voit käyttää jar-tiedostoa varten polkua N:\bin\comtest.jar, joka on aina uusin versio ComTestista. Troubleshooting: If the path has spaces in the folder names, you may experience problems. You can try to avoid the problem by renaming the directory or by surround the problematic name by quotation marks (""). For example:

"C:\Documents\Visual Studio 2013\ComTest.jar".

Sometimes you may need to give full path for java.exe like:

"C:\Program Files (x86)\Java\jre7\bin\java.exe"

Remember to use double quotes around the path if there is spaces.

Additional notes:

You can run the tests normally or using the debugger. If you use the debugger, you can set breakpoints in the code. However, if a test fails, it causes Visual Studio to throw an exception and not run any subsequent tests.

Optional: You can also use a different Java interpreter if the one in your path is not working for you.

Step 4: Install the Visual Studio 2010-2015 code snippet

This is an optional component that makes creating new tests in Visual Studio easier.

There are two ways to install:

Method 1: (VS2010, this MAY not work for VS2015)

  1. Download the ComTest snippet installer
  2. Run the file and follow the installer's instructions.
    • If you get a warning about no signature found, you can skip it safely.
  3. Restart Visual Studio after installation.
    • If the installer does not work(Windows Phone SDK seems to cause this), try the second method...

Method 2: (VS2010-VS2015)

  1. Save the ComTest.snippet file to the Visual Studio's Snippets folder in your own documents folder:
    • Press mouse right button over the previous link and choose "save to disk" (or whatever it is in different browers)
    • Save the file to Visual Studio's snippet folder within your own Documents folder. The directory place depends Windows version and might be something like some of the following (if you use VS2012/VS2013, change the directory, and of course use your own name instead of vesal):
...\My Documents\Visual Studio 2015\Code Snippets\Visual C#\My Code Snippets 
C:\Users\vesal\Documents\Visual Studio 2015\Code Snippets\Visual C#\My Code Snippets
C:\Käyttäjät\vesal\tiedostot\Visual Studio 2015\Code Snippets\Visual C#\My Code Snippets
Libraries: Documents\Visual Studio 2015\Code Snippets\Visual C#\My Code Snippets
  • If there is no such directory, you must create it by yourself.
  • Attention! If you use IE (please don't :)), then be careful that the snippet file will not get an extension .xml. But if it does, use the command line to rename it to ComTest.snippet.
  1. Restart Visual Studio after installation.

Test the snippet:

  1. Write "comt" after a XML comment section in your program...
/// <summary>
/// Multiplies to numbers
/// </summary>
/// <param name="a">first number</param>
/// <param name="b">second number</param>
/// <returns>a*b</returns>
public static double Multiply(double a, double b)
  1. ...and press twice the Tab key and the code should look like this:
/// <summary>
/// Multiplies to numbers
/// </summary>
/// <param name="a">first number</param>
/// <param name="b">second number</param>
/// <returns>a*b</returns>
/// <example>
/// <pre name="test">
/// |
/// </pre>
/// </example>
public static double Multiply(double a, double b)
 // ...


  • Make sure you hit the tab key when the cursor is placed right after the comt text, no space in between.
  • Make sure there is no /// before comt text.

To learn more, read on :)


Usage with the Visual Studio 2010-2013 plugin

This is the preferred way of usage.

  1. Run the ComTest:
  • with mouse: select from the VS menu Tools >> !ComTest Project
  • keyboard shortcut: press Ctrl+Shift+Q
  1. After a while, a test project should appear in Project Explorer window. The tests are also run automatically, and the results should appear in a window at the bottom part of the Visual Studio IDE.

  • if this does not apper, check that on ToolBox? line the target is x86, not Any CPU. Same for bot Release and Debug.
  1. Remember to click on the "Error List" tab on the bottom of the window if you want to go back to editing and view the errors.

Usage with the Visual Studio 2012/2013 plugin

Mainly the usage is same with VS2012/13, but for first test run you must choose the project from Solution Explorer and then right click it and choose Comtest Project.

You may get error that RunAllTests is not available. Press OK and go to Test Explorer and press Run All. Next time just Crtl-Q (if assigned for ComTest Project) should be enough.

Usage from command line

ComTest-C# accepts C# (.cs) files and Visual Studio projects (.csproj) and solutions (.sln), so that

  • test code files are generated from individual source files (ProgramTest.cs from Program.cs)
  • test project files are generated from project files (Project1Test.csproj from Project1.csproj) and all containing code files are processed
  • test solution files are MODIFIED so that all generated test project files are included. All containing non-test C# projects are processed. This is what the Visual Studio plugin does. The original solution file is stored in {SolutionName.sln.ctbackup}.

If you have ComTest.jar stored in your CLASSPATH environment variable, you can run it with

java comtest.ComTest file1.cs file2.cs file3.cs

If not, run

java -jar path\to\ComTest.jar file1.cs file2.cs file3.cs

The content of a file is determined by it's extension. For instructions how to override this and other advanced options, run ComTest with no parameters.

java comtest.ComTest


java -jar path\to\ComTest.jar

Writing tests

The ComTest tests should be written in between the comment block and the method that is going to be tested. Different commenting styles can be used.

/// This is a XML comment block.
/// Use XML comments if you want to document something.
/// Doxygen can read XML comments.
/// Generally, these are the ones you want to use with tests.
 * This is a Javadoc comment block.
 * These are used to comment Java code, but not so much in C#.
 * Doxygen can read these, too.
// This is a C++ comment block.
// If you want to create tests and not include them in documentation,
// use these comments.
 * This is a C comment block.
 * Notice the single star on the first line.
 * These comments are not used for documentation, either.

The start of the test block is signified by

<pre name="test">

After that comes the test code. The test block is terminated by


You can find more information about the syntax of ComTest macro language here.

Example test with ComTest

public class Matrix
    public static void Main(string[] args)

    /// <summary>
    /// Find what line of matrix contains the search value
    /// </summary>
    /// <param name="mat">Matrix where we search</param>
    /// <param name="value">value we search</param>
    /// <returns>row index where we found the value.
    /// If many lines contains the value, we return the first index.
    /// If the matrix does not contain value, return -1</returns>
    /// <example>
    /// <pre name="test">
    /// int[,] m = {{2,4,1}, {9,2,0}, {5,6,1}, {0,12,3}};
    /// Matrix.WhatRowContains(m, 1) === 0;
    /// Matrix.WhatRowContains(m, 0) === 1;
    /// Matrix.WhatRowContains(m, 12) === 3;
    /// Matrix.WhatRowContains(m, 11) === -1;
    /// </pre>
    /// </example>
    public static int WhatRowContains(int[,] mat, int value)
        for (int iy = 0; iy < mat.GetLength(0); iy++)
            for (int ix = 0; ix < mat.GetLength(1); ix++)
                int luku = mat[iy, ix];
                if (luku == value) return iy;
        return -1;

Testing 1-dim array

/// <summary>
/// Returns an arrays with numbers [0 - n-1]
/// </summary>
/// <param name="n">How many numbers needed</param>
/// <returns>Arrys with numbers 0,1,...,n-1</returns>
/// <example>
/// <pre name="test">
///  String.Join(",",Numbers.GiveNumbers(0)) === "";
///  String.Join(",",Numbers.GiveNumbers(1)) === "0";
///  String.Join(",",Numbers.GiveNumbers(3)) === "0,1,2";
/// </pre>
/// </example>
public static int[] GiveNumbers(int n)
    int[] result = new int[n];
    for (int i = 0; i < n; i++) result[i] = i;
    return result;

Testing 2-dim array

In Finnish only for now, sorry.

Mikäli tuloksena on matriisi, on helpointa tehdä oma Jonoksi funktio, joka muuttaa matriisin jonoksi (ja ehkä jopa aina desimaalit pisteeksi), jolloin testin voi kirjoittaa:

/// <summary>
/// Luodaan taulukko aloittaen x1:stä dx:n välein nx x ny kpl lukuja
/// </summary>
/// <param name="ny">montaka riviä tulee</param>
/// <param name="nx">montaka saraketta tulee</param>
/// <param name="x1">alkuarvo</param>
/// <param name="dx">kasvatusaskel</param>
/// <returns>taulukko jossa n reaalilukua</returns>
/// <example>
/// <pre name="test">
///    Matriisit.Jonoksi(Matriisit.Taulukko(0,0,0.1,0.1)) === "";
///    Matriisit.Jonoksi(Matriisit.Taulukko(1,1,0.2,0.1)) === "0.2";
///    Matriisit.Jonoksi(Matriisit.Taulukko(2,2,0.1,0.2)) === "0.1 0.3\n0.5 0.7";
/// </pre>
/// </example>
public static double[,] Taulukko(int ny, int nx, double x1, double dx)
    double[,] luvut = new double[ny,nx];
    double x = x1;
    for (int iy = 0; iy < ny; iy++)
        for (int ix = 0; ix < nx; ix++, x += dx)
            luvut[iy,ix] = x;
    return luvut;

Edellä Jonoksi voisi olla esimerkiksi seuraavaa.

/// <summary>
/// Palauttaa taulukon merkkijonona niin, että alkioiden välissä on erotin.
/// Alkioden formaatin voi valita itse, samoin mitä tulostetaan rivien väliin.
/// Desimaalierotin on aina .
/// </summary>
/// <param name="luvut">Taulukko josta tehdään merkkijono</param>
/// <param name="sarakeErotin">Jono, jolla rivin alkiot erotetaan toisistaan</param>
/// <param name="sarakeformaatti"></param>
/// <param name="rivierotin">Jono, jolla rivit erotetaan toisistaan</param>
/// <param name="riviformaatti">C# tyylinen formaattijono, jolla yksi rivi käsitellään.
/// Jos formaattijonossa ei ole lainkaan {-merkkiä, käytetään jonoa sellaisenaan rivien
/// erottomina
/// </param>
/// <example>
/// <pre name="test">
///  double[,] luvut = {{1,2.1,3},{4,5,6},{7,8,9}};
///  Matriisit.Jonoksi(luvut) === "1 2.1 3\n4 5 6\n7 8 9";
///  Matriisit.Jonoksi(luvut," ","{0}",",") === "1 2.1 3,4 5 6,7 8 9";
///  Matriisit.Jonoksi(luvut,":","{0:0.0}","|","[ {0} ]") === "[ 1.0:2.1:3.0 ]|[ 4.0:5.0:6.0 ]|[ 7.0:8.0:9.0 ]";
///  Matriisit.Jonoksi((double[,])null) === "null";
///  Matriisit.Jonoksi(new double[0,0]) === "";
///  Matriisit.Jonoksi(new double[1,0]) === "";
///  Matriisit.Jonoksi(new double[2,0]) === "\n";
/// </pre>
/// </example>
public static string Jonoksi<T>(T[,] luvut, string sarakeErotin = " ",
                             string sarakeformaatti = "{0}",
                             string riviErotin = "\n",
                             string riviformaatti = "{0}")
    if (luvut == null) return "null";
    System.Globalization.CultureInfo ci = System.Globalization.CultureInfo.CreateSpecificCulture("en-US");
    StringBuilder tulos = new StringBuilder();
    string formaatti = riviformaatti;
    if (formaatti.IndexOf("{") < 0) formaatti = "{0}" + formaatti;
    string rivivali = "";

    for (int iy = 0; iy < luvut.GetLength(0); iy++)
        string vali = "";
        StringBuilder rivi = new StringBuilder();
        for (int ix = 0; ix < luvut.GetLength(1); ix++)
            rivi.Append(String.Format(ci, sarakeformaatti, luvut[iy, ix]));
            vali = sarakeErotin;
        tulos.Append(rivivali + String.Format(formaatti, rivi));
        rivivali = riviErotin;
    return tulos.ToString();

Testing floating point numbers

Because of the internal representation of the floating point numbers, they can not be compared with an "equal" sign, but compare whether they fall within a certain "tolerance", a margin.

/// <example>
/// <pre name="test">
/// double d = 0.1; 
/// [...]
/// d ~~~ 0.1; 
/// </pre>
/// </example>

The default tolerance is 6 digits and it can be changed by #TOLERANCE:

/// <example>
/// <pre name="test">
/// #TOLERANCE=0.01
/// double d = 0.101; 
/// d ~~~ 0.1; 
/// </pre>
/// </example>

Expecting an exception

If some test SHOULD throw an exception, it can be written like:

    ///    "cat"[7] === 'a'  #THROWS IndexOutOfRangeException

and then if it DOES NOT throw an excption, it is test error.

VS solution verkkolevyllä

Eli jos solution sijaitsee verkkolevyllä, ei Comtest toimi, koska VS ei luota verkkolevyihin eikä siksi suostu lataamaan Comtestin solutioniin tekemiä tiedostoja.

Tilanteen voi korjata avaamalla C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE -kansiossa (tai minne ikinä asensitkaan VS:n) olevan devenv.exe.config -tiedoston ja lisäämällä siihen yhden rivin. Mutta jotta kaikki olisi mahdollisimman vaikeaa, on ko. tiedostoon kirjoittaminen estetty (tai ainakin minulla oli), joten joudut avaamaan suosikki tekstieditorisi administratorina (hiiren kakkosnappi -> Run as Administrator) että voit muokata tuota tiedostoa. Tuohon tiedostoon pitää lisätä rivi <loadFromRemoteSources enabled="true" /> heti rivin <runtime> perään. Tallenna tiedosto ja käynnistä VS varmuuden vuoksi uudelleen, ja ComTestin? pitäisi nyt toimia.

Known problems

  • does not work with Visual Studio Express (contains no test framework)
  • may not work on mounted or substed drives (sometimes works???). The project must be in real local physical disk drive.
  • if you get an error saying something like No output from ComTest, there is two options:
    • your path to ComTest.jar is not correct in Tools/ComTest/Options. Please check it and correct.
    • you have a virus scanner that prevenst the run, see later below.
  • if you get an error that says something like: ComTest: File not found, then your Java is not found. Either you have not installed the Java or it is not in the path. In first case install Java, in second case add the full path for Java:
    • go to command line and write:
      dir c:\java.exe /s
      you should get something like:
       Directory of c:\Program Files (x86)\Java\jre7\bin
      12.06.2013  21:43           175 016 java.exe
                     1 File(s)        175 016 bytes
    • in this example case (the path can be something else also), go to your Visual Studion and open Tools/ComTest/Options and write to Java-path:(note also the "):
      "c:\Program Files (x86)\Java\jre7\bin\java.exe"
  • if you use namespace and your namespace have same name than your class, then you have problems. Like:
    public namespace Matrix
      public class Matrix
    Solution: either change the name of namespace or the class. Or you can write the the test line like:
        /// Matrix.Matrix.WhatRowContains(m, 12) === 3;
  • classes, methods and possible namespaces must be public.
  • if you load new Plugin, load the new ComTest.jar at the same time.
  • sometimes when run tests in Solution that Contains Jypeli (XNA), says that "Specified cast is not valid" or "No such interface supported...". Then it helps to delete all generated Test-projects that include Jypeli (XNA).
  • some virus scanners may prevent the run for Java or ComTest. The result is a screen where its says: "No output from ComTest".
  • the snippet installer does not work if you have the Windows Phone SDK installed. This is not something we can fix, unfortunately. See: MSDN forum post
  • if you get error like:
    C:\Users\documents\visual (Unknown) || skipped: file not found
    studion (Unknown) || skipped: file not found
    then your project (or solution) is under path where is spaces in pathname. Please move to directory where is no spaces in the name.
  • if you get error like:
    Picked up _JAVA_OPTIONS: -Djava.vendor="Sun Microsystems Inc."
    Error occurred during initialization of VM
    java/lang/NoClassDefFoundError: java/lang/Object
    then at least one case the java was somehow damaged. Changed to use other java.exe in other directory helped. You may try to go to command line and look what its says ig you run exactly same Java like:
    "C:\Program Files (x86)\Java\jre7\bin\java.exe" -version
    if you still get the error, the Java is damaged.