1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
|
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<Title>Phoronix Test Suite - A Quick Guide To Writing A Test Profile</Title>
<link href="includes/pts-documentation.css" rel="stylesheet" type="text/css" />
</head>
<body>
<div id="pts_doc_header"><div class="pts_doc_fixed"><a href="http://www.phoronix-test-suite.com/"><div id="pts_doc_logo"></div></a></div></div>
<div class="pts_doc_fixed">
<div class="pts_doc_notice"><div style="float: left"><a href="index.html"><< Documentation Home</a></div><div style="float: right;">Writing A Test Profile</div></div>
<div class="pts_doc_main">
<!-- PTS AREA -->
<h1>Writing A Test Profile</h1>
<p>Writing a test profile for the Phoronix Test Suite is a relatively quick and easy
process for anyone familiar with common Linux commands and the basics of XML.
To help you understand the design of the Phoronix Test Suite, this guide covers
the steps needed to write a testing profile for a very simple application.</p>
<p>The first step in the profile writing process is to, well, have a piece of
software you'd like to use with the Phoronix Test Suite. This software can be
closed-source or open-source and be virtually anything as long as it is compatible
with the Linux operating system.</p>
<p>For this guide, the piece of software being used for demonstration is just
a simple C++ program that calculates Pi to 8,765,4321 digits using the Leibniz
formula. Below is this sample piece of software intended just for demonstration
purposes.</p>
<blockquote>#include <iostream><br>
#include <math.h></p>
<p>int main()<br>
{<br>
double pi = 0;</p>
<p> for(long int i = 1; i <= 87654321; i++)<br>
pi += (double) pow(-1, i + 1) / (2 * i - 1);</p>
<p> pi *= 4;<br>
std::cout << "Done Calculating Pi..." << endl;<br>
return 0;<br>
}</blockquote>
<p>The first step in the actual profile writing process is to name it. If you're
looking to ultimately push this profile to be included in the Phoronix Test Suite,
its name must be all lower case and consist of just alpha-numeric characters,
but can contain dashes (-). A more advanced test profile capability is operating system prefixes, and if using those there is an
underscore separating the prefix from the normal profile name. For this sample profile, we're calling it <em>sample-program</em>
and the file-name would be <em>sample-program.xml</em>. Our (very basic) profile is showcased
below.</p>
<blockquote><PhoronixTestSuite><br>
<TestProfile><br>
<Version>1.1.0</Version><br>
<TestType>Processor</TestType><br>
<SoftwareType>Utility</SoftwareType><br>
<License>FREE</License><br>
<Status>PRIVATE</Status><br>
<Maintainer>Phoronix Media</Maintainer><br>
</TestProfile><br>
<TestInformation><br>
<Title>Sample Pi Program</Title><br>
<TimesToRun>3</TimesToRun><br>
<ResultScale>Seconds</ResultScale><br>
<Proportion>LIB</Proportion><br>
<Description>A simple C++ program that calculates Pi to 8,765,4321 digits using the Leibniz formula. This test can be used for showcasing how to write a basic test profile.</Description><br>
<ExternalDependencies>build-utilities</ExternalDependencies><br>
</TestInformation><br>
</PhoronixTestSuite></blockquote>
<p>This XML profile is what interfaces with the Phoronix Test Suite and provides
all the needed information about the test as well as other attributes. For a complete
listing of all the supported profile options, look at the specification files
in the documentation folder. In the case of <em>sample-program</em>, it lets the Phoronix
Test Suite know that it's composed of free software, is designed to test the processor,
is intended for private use only, and this profile is maintained by Phoronix Media.
In addition, it tells the Phoronix Test Suite to execute this program three times
and as no result quantifier is set, the average of the three runs will be taken.
This profile also tells the Phoronix Test Suite that the generic <em>build-utilities</em>
package is needed, which will attempt to ensure that GCC and the standard development
utilities/libraries are installed on your Linux distribution. This is needed as
the C++ source-code will need to be built from source.</p>
<p>The next step is to write the <em>install.sh</em> file, which once called by the Phoronix
Test Suite is intended to install the test locally for benchmarking purposes.
The <em>install.sh</em> file is technically optional, but is generally used by all tests.
Alternatively, the file may be named <em>install.php</em> if you wish to write the install
script in PHP instead of a shell script. Note: The first argument supplied to
the install script is the directory that the test needs to be installed to. The
<em>install.sh</em> file (in our instance) is to be placed inside <em>pts/test-resources/sample-program</em>.
Below is the <em>install.sh</em> for the <em>sample-program</em>.</p>
<blockquote>#!/bin/sh<br /><br />
tar -xjf sample-pi-program-1.tar.bz2<br />
g++ sample-pi-program.cpp -o sample-pi-program<br />
echo "#!/bin/sh<br />
\$TIMER_START<br />
./sample-pi-program 2>&1<br />
\$TIMER_STOP<br />
" > sample-program<br>
chmod +x sample-program</blockquote>
<p>Phoronix Test Suite 1.4 and later provides an integrated multi-platform micro-timer framework that provides the <em>$TIMER_START</em> and <em>$TIMER_STOP</em> functionality.</p>
<p>This install file builds the code with GCC, and then creates a small script
that is run by the Phoronix Test Suite, which times how long it takes to run the
software. Where does the source-code come into play? Well, it needs to be downloaded
now from a web server. The Phoronix Test Suite has built-in support for managing
downloads from multiple servers in a random over, fall-back support if one mirror
is done, and verification of MD5 check-sums. Below is the <em>downloads.xml</em> file for
<em>sample-program</em> that covers all of this.</p>
<blockquote><PhoronixTestSuite><br>
<Downloads><br>
<Package><br>
<URL>http://www.phoronix-test-suite.com/benchmark-files/sample-pi-program.cpp</URL><br>
<MD5>e90fb790df8d1544696a1439c9b5bd8d</MD5><br>
</Package><br>
</Downloads><br>
</PhoronixTestSuite></blockquote>
<p>The final step in the profile writing process is to write a parser to strip
all information but the reported result from the standard output. The standard
output is submitted to <em>parse-results.sh</em> or <em>parse-results.php</em> as the first argument
in quotes or using <em>$LOG_FILE</em> if the test profile writes to that variable's location.</p>
<p>If the test profile uses the integrated micro-timer framework with <em>$TIMER_START</em> and <em>$TIMER_STOP</em> but no parse-results file is provided, the Phoronix Test Suite will automatically use the difference between <em>$TIMER_START</em> and <em>$TIMER_STOP</em> and use that as the test result. This is the time (in seconds) that elapsed between starting and stopping the timer.</p>
<p>After that, with all the files in their correct locations, just run: <em>phoronix-test-suite
benchmark sample-program</em>. The Phoronix Test Suite should now handle the rest by
installing the test, running the test, and recording the results (if you so choose).
There is no additional work that needs to be done for the results to be recorded
in the PTS Results Viewer or even reporting the results to Phoronix Global.</p>
<p><strong>For more information, visit <a href="http://www.phoronix-test-suite.com/">Phoronix-Test-Suite.com</a> and read the <a href="index.html">included documentation</a>. It should also be of great help to just look at the existing test profiles, which can be found inside the <em>phoronix-test-suite/pts/</em> folder with the <em>test-resources</em> and <em>test-profiles</em> sub-directories.</strong></p>
<!-- END OF PTS AREA -->
</div></div><div class="pts_doc_fixed"><div class="pts_doc_bottom"><div style="float: left;"><a href="http://www.phoronix-test-suite.com/">Phoronix-Test-Suite.com</a></div><div style="float: right;">Copyright © 2008 by <a href="http://www.phoronixmedia.com/">Phoronix Media</a>.</div></div></div>
</body>
</html>
|
