Parth Patil’s Blog

Formatting PHPUnit Test Results As HTML Table

PHPUnit is an excellent Unit Test framework for PHP based on XUnit. If you haven’t used PHPUnit or unit testing you should definitely read about it at http://phpunit.de. In this tutorial I explain how you can custom format the test results from PHPUnit. The tutorial assumes familiarity with PHPUnit.
I am a big fan of PHPUnit and use it very often during PHP development. I wanted to view the results of tests run on remote servers in my web browser. The default test result output for phpunit is in the TAP format and its not convenient to parse this output. XML is more suitable for parsing and we will see how to get our test results in xml format.
We will consider the following example Unit Test for this tutorial

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

<?php
    require_once 'PHPUnit/Framework.php';
    class ArrayTest extends PHPUnit_Framework_TestCase
    {
        public function testNewArrayIsEmpty()
        {
            // Create the Array fixture.    
            $my_array = array();
 
            // Assert that the size of the Array fixture is 0.
            $this->assertEquals(0, sizeof($my_array));
        }
 
        public function testArrayContainsAnElement()
        {
            // Create the Array fixture. 
            $my_array = array();
 
            // Add an element to the Array fixture.
            $my_array[] = 'Element';
 
            // Make this test fail on purpose 
            $this->assertEquals(2, sizeof($my_array));
        }
    }
?>

The test is run on the command line as follows

phpunit ArrayTest

The output from running the above test on command line looks as follows

As I mentioned before, for formating the results of the tests into HTML table its convenient to have the results in the form of xml. There are two ways to get the test results in xml

1. Specifying an option to PHPUnit on the command line.

phpunit --log-xml report.xml ArrayTest

2. By doing what the PHPUnit does internally to get the output in xml format.

For the first option you will have to execute the phpunit command from inside the PHP script, the xml output is written to a file which has to be again read by your PHP script. The second option does not require you to run the phpunit command from inside PHP script and gives you lot more flexibility than the first one. It will also give you an idea of how to use custom formatters for PHPUnit results.
I couldn’t find much documentation on how to use/write custom formatters for PHPUnit test results. After tracing PHPUnit executable(the “phpunit” command itself) using xdebug I found out how PHPUnit outputs test in xml format when you specify “–log-xml” option on the command line. PHPUnit uses something called TestListener for custom formatting of test results. The TestListener follows the observer pattern. The TestListener subscribes to “test failure” and “test success” events of the PHPUnit_Framework_TestResult class.

I have created a Test Runner that uses the xml TestListener. The test runner looks as follows.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

<?php
require_once('PHPUnit/Framework/TestSuite.php');
require_once('PHPUnit/Framework/TestResult.php');
require_once('PHPUnit/Util/Log/XML.php');
require_once('ArrayTest.php');
 
class MyTestRunner
{
    public static function run()
    {
        // Create the test suite instance
        $suite = new PHPUnit_Framework_TestSuite();
        $suite->setName('MyTestRunner');
 
        // Add files to the TestSuite
        $suite->addTestSuite('ArrayTest');
 
        // Create a xml listener object 
        $listener = new PHPUnit_Util_Log_XML;
 
        // Create TestResult object and pass the xml listener to it
        $testResult = new PHPUnit_Framework_TestResult();
        $testResult->addListener($listener);
 
        // Run the TestSuite
        $result = $suite->run($testResult);
 
        // Get the results from the listener
        $xml_result = $listener->getXML();
        return $xml_result;
    }
}

I have simplified the code for the The Test Runner for purpose of this tutorial. An actual Test Runner would have some mechanism to get the list of tests to run and a loop to add all those tests to the TestSuite.

The Test runner code is pretty straight forward. I first create a TestSuite . Then I add the ArrayTest to the TestSuite. After that I instantiate an xml TestListener from the PHPUnit_Util_Log_XML class that comes bundled with PHPUnit. Then I instantiate a PHPUnit_Framework_TestResult object and add the TestListener object to the TestResult object. Then I run the TestSuite and get the test result in xml format from the listener object.
This is how the xml looks like

<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
  <testsuite name="MyTestRunner" file="/home/parth/test/MyTestRunner.php" tests="2" failures="1" errors="0" time="0.002406">
    <testsuite name="ArrayTest" file="/home/parth/test/ArrayTest.php" tests="2" failures="1" errors="0" time="0.002406">
      <testcase name="testNewArrayIsEmpty" class="ArrayTest" file="/home/parth/test/ArrayTest.php" line="5" time="0.000514"/>
      <testcase name="testArrayContainsAnElement" class="ArrayTest" file="/home/parth/test/ArrayTest.php" line="14" time="0.001892">
        <failure type="PHPUnit_Framework_ExpectationFailedException"><![CDATA[testArrayContainsAnElement(ArrayTest)
Failed asserting that <integer:1> matches expected value <integer:2>.
 
/home/parth/test/ArrayTest.php:23
/home/parth/test/MyTestRunner.php:26
/home/parth/test/MyTestRunner.php:34
]]></failure>
      </testcase>
    </testsuite>
  </testsuite>
</testsuites>

As expected the xml shows the same results as before(when run on the command line). The xml is self descriptive and has lot of metadata for the test results. Next step is to parse the above xml into an associative array. Following is how I parsed the above result xml using SimpleXML.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

<?php
   $test_xml = MyTestRunner::run();
   $simple = new SimpleXMLElement($test_xml);
 
   $test_results = array();
 
   foreach($simple->{'testsuite'}->{'testsuite'}->testcase as $testcase)
   {
       $result = array();
       // Don't froget to cast SimpleXMLElement to string!
       $result['name'] = (string)$testcase['name'];
       if(isset($testcase->{'failure'}))
       {
           $result['result'] = 'Fail';
           $result['message'] = (string)$testcase->{'failure'};
       }
       else
       {
            $result['result'] = 'Pass';
            $result['message'] = '';
       }
       $test_results[] = $result;     
   }
   $test_results;
?>

And finally I rendered the result in the form of a html table

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

<table cellspacing="0" class="test_results">
  <thead>
    <tr><th>Test Name</th><th>Result</th><th>Message</th></tr>
  </thead>
  <tbody>
  <?php foreach($test_results as $test_result): ?>
  <tr>
      <td><?php echo $test_result['name'] ?></td>
      <?php if($test_result['result'] == 'Fail') : ?>
      <td class="test_fail"/>
      <?php else: ?> 
      <td class="test_pass"/>
      <?php endif; ?>
      <td><?php echo $test_result['message'] ?></td>
  </tr>
  <?php endforeach; ?>
  </tbody>
</table>

Here is how the results look in html table. I borrowed the css from Symfony’s admin generator module.

I hope this tutorial has been helpful. Thanks for reading!

Using Perl Expect To Automate SFTP Access

Recently I had to do a task where I had to automate the downloading of files from a sftp server. Automating plain ftp download is much easier than automating sftp because there are prompts from the remote server that have to be handled. For automating such interactive applications there is a wonderful tool called Expect. The automation scripts for Expect are written in the Tcl language. There was an existing Tcl Expect script to automate downloading of files from sftp. I found that I did not have Tcl Expect installed and I had heard that Perl Expect is very similar to TCL Expect. So I decided to give it a shot. I ported the original Expect script to Perl version and added few features.
Following is the script and I have provided some explanation below the code.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51

use strict;
use Expect;
 
# Uncomment the following line if you want to see what expect is doing 
#$Expect::Exp_Internal = 1;
 
# Uncomment the following line if you don't want to see any output from the script
#$Expect::Log_Stdout = 0;
 
# set your username/password here
my $sftpUsername = "username" ;
my $sftpPassword = "password"; 
my $sftpServer = "yourserver";
my $fileToFetch = "myfile";
my $timeout = 10; 
 
# If sftp is not in your path replace with absolute path of sftp program
my $command = 'sftp';
my $params = ("$sftpUsername\@$sftpServer");
 
# Create the Expect object
my $exp = Expect->spawn($command, $params) or die "Cannot spawn sftp command \n";
 
# If this is the first time you are running this , expect will send "yes" to add the key
# for the sftp server to the ~/.ssh/known_hosts file else
# wait for "Password Authentication" string to show up
$exp->expect($timeout,
	["Password Authentication"],
	["Are you sure you want to continue connecting", sub {my $self = shift; $self->send("yes\n");}]
	);
 
# Wait for Password prompt to show up 
$exp->expect($timeout, ["Password:"]);
 
# Sent the sftp password 
$exp->send("$sftpPassword\n");
 
# Wait for sftp prompt
$exp->expect($timeout, ["sftp>"]);
 
# Get yesterday's report file/s and put it in reportsPath directory on the local machine
$exp->send("get $fileToFetch\n");
 
# Wait for sftp prompt
$exp->expect($timeout, ["sftp>"]);
 
# Close ftp session
$exp->send("bye\n");
 
# Destroy the expect object
$exp->soft_close();

You need to install Expect from cpan if you don’t have it installed. To check if the Expect is installed you could check it in your terminal by executing this “perl -MExpect -e1″.
On line 22 I am creating an Expect object using using the spawn function. spawn takes the command to execute as the first argument and the parameters to the command as an array of arguments, hence $params is an array containing single element which is “username@sftpserver”.
Now when the script is run for the first time, there is a prompt to save the key of the sftp server in the ~/.ssh/know_hosts file and if the entry for the remote sftp server is already present in the known_hosts file a “Password Authentication” prompt will be sent by the sftp server. Both these situations have to be handled in the Expect script. Fortunately its very easy to to write handlers for these two possibilities in a single expect function call. The expect function can take any number of possible strings to expect from the remote server at a given point. You can see this on line 28 & 29, each possible string is provided as an array reference. And if there is an action that you want to perform corresponding to a particular string you can do this by providing a subroutine to handle this. You could either pass a reference to a subroutine or provide an anonymous subroutine. On line 29 I have provided an anonymous subroutine to send “yes” when prompted for accepting the key from the sftp server. Its also possible to provide the regular expression instead of literal string matches to the expect function.
The rest of the script is self explanatory.

Read the Expect module documentation on cpan for more information :
http://search.cpan.org/~rgiersig/Expect-1.21/Expect.pod

Search