Formatting PHPUnit Test Results As HTML Table

Blog
About

Formatting PHPUnit Test Results As HTML Table

Posted by Parth - 14/05/08 at 12:05:55 am

PHPUnit is an excellent Unit Test framework for PHP based on XUnit. If you haven’t used PHPUnit or unit testing you should definitely read about it at http://phpunit.de. In this tutorial I explain how you can custom format the test results from PHPUnit. The tutorial assumes familiarity with PHPUnit.
I am a big fan of PHPUnit and use it very often during PHP development. I wanted to view the results of tests run on remote servers in my web browser. The default test result output for phpunit is in the TAP format and its not convenient to parse this output. XML is more suitable for parsing and we will see how to get our test results in xml format.
We will consider the following example Unit Test for this tutorial

<?php
    require_once 'PHPUnit/Framework.php';
    class ArrayTest extends PHPUnit_Framework_TestCase
    {
        public function testNewArrayIsEmpty()
        {
            // Create the Array fixture.    
            $my_array = array();
 
            // Assert that the size of the Array fixture is 0.
            $this->assertEquals(0, sizeof($my_array));
        }
 
        public function testArrayContainsAnElement()
        {
            // Create the Array fixture. 
            $my_array = array();
 
            // Add an element to the Array fixture.
            $my_array[] = 'Element';
 
            // Make this test fail on purpose 
            $this->assertEquals(2, sizeof($my_array));
        }
    }
?>

The test is run on the command line as follows

phpunit ArrayTest

The output from running the above test on command line looks as follows

As I mentioned before, for formating the results of the tests into HTML table its convenient to have the results in the form of xml. There are two ways to get the test results in xml

1. Specifying an option to PHPUnit on the command line.

phpunit --log-xml report.xml ArrayTest

2. By doing what the PHPUnit does internally to get the output in xml format.

For the first option you will have to execute the phpunit command from inside the PHP script, the xml output is written to a file which has to be again read by your PHP script. The second option does not require you to run the phpunit command from inside PHP script and gives you lot more flexibility than the first one. It will also give you an idea of how to use custom formatters for PHPUnit results.
I couldn’t find much documentation on how to use/write custom formatters for PHPUnit test results. After tracing PHPUnit executable(the “phpunit” command itself) using xdebug I found out how PHPUnit outputs test in xml format when you specify “–log-xml” option on the command line. PHPUnit uses something called TestListener for custom formatting of test results. The TestListener follows the observer pattern. The TestListener subscribes to “test failure” and “test success” events of the PHPUnit_Framework_TestResult class.

I have created a Test Runner that uses the xml TestListener. The test runner looks as follows.

<?php
require_once('PHPUnit/Framework/TestSuite.php');
require_once('PHPUnit/Framework/TestResult.php');
require_once('PHPUnit/Util/Log/XML.php');
require_once('ArrayTest.php');
 
class MyTestRunner
{
    public static function run()
    {
        // Create the test suite instance
        $suite = new PHPUnit_Framework_TestSuite();
        $suite->setName('MyTestRunner');
 
        // Add files to the TestSuite
        $suite->addTestSuite('ArrayTest');
 
        // Create a xml listener object 
        $listener = new PHPUnit_Util_Log_XML;
 
        // Create TestResult object and pass the xml listener to it
        $testResult = new PHPUnit_Framework_TestResult();
        $testResult->addListener($listener);
 
        // Run the TestSuite
        $result = $suite->run($testResult);
 
        // Get the results from the listener
        $xml_result = $listener->getXML();
        return $xml_result;
    }
}

I have simplified the code for the The Test Runner for purpose of this tutorial. An actual Test Runner would have some mechanism to get the list of tests to run and a loop to add all those tests to the TestSuite.

The Test runner code is pretty straight forward. I first create a TestSuite . Then I add the ArrayTest to the TestSuite. After that I instantiate an xml TestListener from the PHPUnit_Util_Log_XML class that comes bundled with PHPUnit. Then I instantiate a PHPUnit_Framework_TestResult object and add the TestListener object to the TestResult object. Then I run the TestSuite and get the test result in xml format from the listener object.
This is how the xml looks like

<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
  <testsuite name="MyTestRunner" file="/home/parth/test/MyTestRunner.php" tests="2" failures="1" errors="0" time="0.002406">
    <testsuite name="ArrayTest" file="/home/parth/test/ArrayTest.php" tests="2" failures="1" errors="0" time="0.002406">
      <testcase name="testNewArrayIsEmpty" class="ArrayTest" file="/home/parth/test/ArrayTest.php" line="5" time="0.000514"/>
      <testcase name="testArrayContainsAnElement" class="ArrayTest" file="/home/parth/test/ArrayTest.php" line="14" time="0.001892">
        <failure type="PHPUnit_Framework_ExpectationFailedException"><![CDATA[testArrayContainsAnElement(ArrayTest)
Failed asserting that <integer:1> matches expected value <integer:2>.
 
/home/parth/test/ArrayTest.php:23
/home/parth/test/MyTestRunner.php:26
/home/parth/test/MyTestRunner.php:34
]]></failure>
      </testcase>
    </testsuite>
  </testsuite>
</testsuites>

As expected the xml shows the same results as before(when run on the command line). The xml is self descriptive and has lot of metadata for the test results. Next step is to parse the above xml into an associative array. Following is how I parsed the above result xml using SimpleXML.

<?php
   $test_xml = MyTestRunner::run();
   $simple = new SimpleXMLElement($test_xml);
 
   $test_results = array();
 
   foreach($simple->{'testsuite'}->{'testsuite'}->testcase as $testcase)
   {
       $result = array();
       // Don't froget to cast SimpleXMLElement to string!
       $result['name'] = (string)$testcase['name'];
       if(isset($testcase->{'failure'}))
       {
           $result['result'] = 'Fail';
           $result['message'] = (string)$testcase->{'failure'};
       }
       else
       {
            $result['result'] = 'Pass';
            $result['message'] = '';
       }
       $test_results[] = $result;     
   }
   $test_results;
?>

And finally I rendered the result in the form of a html table

<table cellspacing="0" class="test_results">
  <thead>
    <tr><th>Test Name</th><th>Result</th><th>Message</th></tr>
  </thead>
  <tbody>
  <?php foreach($test_results as $test_result): ?>
  <tr>
      <td><?php echo $test_result['name'] ?></td>
      <?php if($test_result['result'] == 'Fail') : ?>
      <td class="test_fail"/>
      <?php else: ?> 
      <td class="test_pass"/>
      <?php endif; ?>
      <td><?php echo $test_result['message'] ?></td>
  </tr>
  <?php endforeach; ?>
  </tbody>
</table>

Here is how the results look in html table. I borrowed the css from Symfony’s admin generator module.

I hope this tutorial has been helpful. Thanks for reading!

May 14, 2008 | In PHP, Programming, Testing |

9 Comments »

RSS feed for comments on this post. TrackBack URI

Thanks. It’s very helpful

Comment by Bảo — July 7, 2008 #
This is exactly what I was looking for - it just pains me to see all of the trivial examples being documented, while the advanced features need to be discovered by either tracing the source or trying to decipher the API documentation (personally, I’m not very familiar with their style of documenting the API, so I’m a bit lost).

Comment by Klemen Slavič — July 8, 2008 #
Great stuff! Good think I’ve found your blog, it helped me a lot!

Comment by Bruno B. — July 22, 2008 #
This script seems to be acting up sometimes, especially when php errors occur.

I noticed this while running the command-line phpunit and getting errors which were not getting picked up by the web script - it was showing all tests as “PASSED”.

Comment by Bruno B. — July 23, 2008 #
The problem above can be sorted by checking the error tag, in the same way it’s been done for “failure”

The scripts works fine, it’s only not checking for the error tag in the xml - which *does* contain eventual failures.

Comment by Bruno B. — July 23, 2008 #
Hi Parth,

I’m having an issue, maybe you can help me out.
I’ve created my test runner object to work with Zend Framework, but I’m facing a problem.
Everything is going fine, but when I run the test, I’m getting only a blank screen.
Any hint?

Thanks and Regards

Comment by Ramses Paiva — November 7, 2008 #
Hi,

I have tried you code and it seems to fail for me. I don’t know why but, an error like “Couldn’t fetch DomDocument” occurs. While searching, I have found out, that beside XML, you could also use JSON too.

All you have to to instead of creating an instance of a PHPUnit_Util_Log_XML class, create an instance of a PHPUnit_Util_Log_JSON class. And it will give a json output.

Also you could try to pass a file name to the class constructor like:

$boo = new PHPUnit_Util_Log_JSON(”logfile.txt”);

Instead of giving a browser output, it will write it on the file.

Comment by roy simkes — November 13, 2008 #
[...] forget to check out PHPUnit’s Manual and here for how to use [...]

Pingback by PHPUnit Custom Listener | Kingdom of Roi — November 13, 2008 #
Could you publish the CSS for the table?

Comment by Queso — November 30, 2008 #

Parth Patil’s Blog