1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
|
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<Title>Phoronix Test Suite - XML Test Syntax</Title>
<link href="includes/pts-documentation.css" rel="stylesheet" type="text/css" />
</head>
<body>
<div id="pts_doc_header"><div class="pts_doc_fixed"><a href="http://www.phoronix-test-suite.com/"><div id="pts_doc_logo"></div></a></div></div>
<div class="pts_doc_fixed">
<div class="pts_doc_notice"><div style="float: left"><a href="index.html"><< Documentation Home</a></div><div style="float: right;">PTS Test Specification</div></div>
<div class="pts_doc_main">
<!-- PTS AREA -->
<h1>Phoronix Test Suite 1.0 Test Specification</h1>
<p>This document describes the XML tags for writing a test profile.</p>
<h2>PhoronixTestSuite/TestProfile/</h2>
<p><strong>Tag:</strong> Version<br>
<strong>Location:</strong> PhoronixTestSuite/TestProfile/<br>
<strong>Internal Reference:</strong> P_TEST_PTSVERSION<br>
<strong>Required:</strong> Yes<br>
<strong>Description:</strong> The version of the test profile itself, not dependent upon the main
Phoronix Test Suite version or the version of the software package itself.<br>
<strong>Supported Values:</strong> ( Formatted: x.x.x )</p>
<p><strong>Tag:</strong> TestType<br>
<strong>Location:</strong> PhoronixTestSuite/TestProfile/<br>
<strong>Internal Reference:</strong> P_TEST_HARDWARE_TYPE<br>
<strong>Required:</strong> Recommended<br>
<strong>Description:</strong> The hardware area of the system that this test is designed to stress.<br>
<strong>Supported Values:</strong> System, Processor, Graphics, Disk, Memory, Motherboard</p>
<p><strong>Tag:</strong> SoftwareType<br>
<strong>Location:</strong> PhoronixTestSuite/TestProfile/<br>
<strong>Internal Reference:</strong> P_TEST_SOFTWARE_TYPE<br>
<strong>Required:</strong> Recommended<br>
<strong>Description:</strong> The type of software that this test consists of.<br>
<strong>Supported Values:</strong> Game, Utility, Simulator, Other</p>
<p><strong>Tag:</strong> Maintainer<br>
<strong>Location:</strong> PhoronixTestSuite/TestProfile/<br>
<strong>Internal Reference:</strong> P_TEST_MAINTAINER<br>
<strong>Required:</strong> Yes<br>
<strong>Description:</strong> The name of this profile's maintainer.</p>
<p><strong>Tag:</strong> License<br>
<strong>Location:</strong> PhoronixTestSuite/TestProfile/<br>
<strong>Internal Reference:</strong> P_TEST_LICENSE<br>
<strong>Required:</strong> Yes<br>
<strong>Description:</strong> The license of the software itself.<br>
<strong>Supported Values:</strong> FREE, NON-FREE, RESTRICTED (Will only work with certain hardware
configurations), UNKNOWN</p>
<p><strong>Tag:</strong> Status<br>
<strong>Location:</strong> PhoronixTestSuite/TestProfile/<br>
<strong>Internal Reference:</strong> P_TEST_STATUS<br>
<strong>Required:</strong> Yes<br>
<strong>Description:</strong> The status of the test profile itself.<br>
<strong>Supported Values:</strong> VERIFIED, UNVERIFIED, BROKEN</p>
<p><strong>Tag:</strong> SupportedArchitectures<br>
<strong>Location:</strong> PhoronixTestSuite/TestProfile/<br>
<strong>Internal Reference:</strong> P_TEST_SUPPORTEDARCHS<br>
<strong>Required:</strong> No - Defaults to assume all architectures are supported<br>
<strong>Description:</strong> If the test software is limited to running on select system architectures,
list them in this tag with each architecture delimited by a comma.<br>
<strong>Supported Values:</strong> ( Examples: x86, x86_64 )</p>
<p><strong>Tag:</strong> SupportedPlatforms<br>
<strong>Location:</strong> PhoronixTestSuite/TestProfile/<br>
<strong>Internal Reference:</strong> P_TEST_SUPPORTEDPLATFORMS<br>
<strong>Required:</strong> No - Defaults to assume all operating systems are supported<br>
<strong>Description:</strong> If the test software is limited to running on select operating systems,
list them in this tag with each OS being delimited by a comma.<br>
<strong>Supported Values:</strong> Linux, Solaris, FreeBSD, BSD</p>
<p><strong>Tag:</strong> UnsupportedPlatforms<br>
<strong>Location:</strong> PhoronixTestSuite/TestProfile/<br>
<strong>Internal Reference:</strong> P_TEST_UNSUPPORTEDPLATFORMS<br>
<strong>Required:</strong> No - Defaults to assume all operating systems are supported<br>
<strong>Description:</strong> If the test software is limited to running on select operating systems,
list the ones NOT supported by this test. This tag is the opposite of SupportedPlatforms. Operating systems
not listed here are assumed to be supported.<br>
<strong>Supported Values:</strong> Linux, Solaris, FreeBSD, BSD</p>
<p><strong>Tag:</strong> ExternalDependencies<br>
<strong>Location:</strong> PhoronixTestSuite/TestProfile/<br>
<strong>Internal Reference:</strong> P_TEST_EXDEP<br>
<strong>Required:</strong> No<br>
<strong>Description:</strong> The external dependencies needed for the test. Multiple dependencies
need to be delimited by a comma.<br>
<strong>Supported Values:</strong> ( See pts/distro-xml/generic-packages.xml for a list of currently
supported external dependencies. )</p>
<p><strong>Tag:</strong> Extends<br>
<strong>Location:</strong> PhoronixTestSuite/TestProfile/<br>
<strong>Internal Reference:</strong> P_TEST_CTPEXTENDS<br>
<strong>Required:</strong> No - Defaults to assume there is no extension<br>
<strong>Description:</strong> If the test relies upon another test within the Phoronix Test Suite, list its
name in this tag. This is similar to extends in programming languages in the capabilities that it provides.
When running a test that extends another one, it will require all test extensions be built as well as the
PATH environment variable will contain each of the test directories below this test. This is part of the
Cascading Test Profiles functionality.<br>
<strong>Supported Values:</strong> ( Any name of another test identifer shipping with the Phoronix Test Suite. )</p>
<p><strong>Tag:</strong> RequiresRoot<br>
<strong>Location:</strong> PhoronixTestSuite/TestProfile/<br>
<strong>Internal Reference:</strong> P_TEST_ROOTNEEDED<br>
<strong>Required:</strong> No - Defaults to assume false<br>
<strong>Description:</strong> If root access is needed by the software encapsulated by this profile,
the value of this tag should be set to <em>TRUE</em>.<br>
<strong>Supported Values:</strong> FALSE, TRUE</p>
<p><strong>Tag:</strong> DownloadSize<br>
<strong>Location:</strong> PhoronixTestSuite/TestProfile/<br>
<strong>Internal Reference:</strong> P_TEST_DOWNLOADSIZE<br>
<strong>Required:</strong> No<br>
<strong>Description:</strong> The estimated size (in megabytes) that this test takes up in files
downloaded. <strong>Note:</strong> As of Phoronix Test Suite 1.4, the DownloadSize tag has been deprecated and test profile
writers should instead use the <em>FileSize</em> within the <em>downloads.xml</em> specification for each file to be downloaded.<br>
<strong>Supported Values:</strong> ( Values must be in MB )</p>
<p><strong>Tag:</strong> EnvironmentSize<br>
<strong>Location:</strong> PhoronixTestSuite/TestProfile/<br>
<strong>Internal Reference:</strong> P_TEST_ENVIRONMENTSIZE<br>
<strong>Required:</strong> No<br>
<strong>Description:</strong> The estimated size (in megabytes) that this test takes up in its
environment directory when downloaded files have been extracted, etc.<br>
<strong>Supported Values:</strong> ( Values must be in MB )</p>
<p><strong>Tag:</strong> EstimatedLength<br>
<strong>Location:</strong> PhoronixTestSuite/TestProfile/<br>
<strong>Internal Reference:</strong> P_TEST_ESTIMATEDTIME<br>
<strong>Required:</strong> No<br>
<strong>Description:</strong> The estimated length (in minutes) that it takes
this benchmark to run in total.<br>
<strong>Supported Values:</strong> ( Values must be in minutes )</p>
<p><strong>Tag:</strong> ProjectURL<br>
<strong>Location:</strong> PhoronixTestSuite/TestProfile/<br>
<strong>Internal Reference:</strong> P_TEST_PROJECTURL<br>
<strong>Required:</strong> No<br>
<strong>Description:</strong> The web-site for the software vendor or project
of the software being used for testing.<br>
<strong>Supported Values:</strong> ( The URL must start with http:// and be a
valid address )</p>
<p><strong>Tag:</strong> RequiresCoreVersion<br>
<strong>Location:</strong> PhoronixTestSuite/TestProfile/<br>
<strong>Internal Reference:</strong> P_TEST_SUPPORTS_COREVERSION<br>
<strong>Required:</strong> No<br>
<strong>Description:</strong> If this test profile will only work with specific versions
of the Phoronix Test Suite, they should be specified in this tag.<br>
<strong>Supported Values:</strong> ( Values must be in the PTS version format of X.X.X. If this test
profile requires version 1.4.0 of the Phoronix Test Suite or later, the value would simply be <em>1.4.0</em>. If
it requires a version between 1.4.0 and 1.8.0, it must be written as <em>1.4.0 - 1.8.0</em>. )</p>
<h2>PhoronixTestSuite/TestInformation/</h2>
<p><strong>Tag:</strong> Title<br>
<strong>Location:</strong> PhoronixTestSuite/TestInformation/<br>
<strong>Internal Reference:</strong> P_TEST_TITLE<br>
<strong>Required:</strong> Yes<br>
<strong>Description:</strong> The string title of the test, used for display purposes.</p>
<p><strong>Tag:</strong> SubTitle<br>
<strong>Location:</strong> PhoronixTestSuite/TestInformation/<br>
<strong>Internal Reference:</strong> P_TEST_SUBTITLE<br>
<strong>Required:</strong> No<br>
<strong>Description:</strong> If a value is supplied, this is a static sub-title when rendering
the graph results. Otherwise the test options (if applicable) are displayed.</p>
<p><strong>Tag:</strong> Version<br>
<strong>Location:</strong> PhoronixTestSuite/TestInformation/<br>
<strong>Internal Reference:</strong> P_TEST_VERSION<br>
<strong>Required:</strong> Recommended<br>
<strong>Description:</strong> The version of the software game/application/program itself</p>
<p><strong>Tag:</strong> Description<br>
<strong>Location:</strong> PhoronixTestSuite/TestInformation/<br>
<strong>Internal Reference:</strong> P_TEST_DESCRIPTION<br>
<strong>Required:</strong> Recommended<br>
<strong>Description:</strong> A few sentence description of the test.</p>
<p><strong>Tag:</strong> ResultScale<br>
<strong>Location:</strong> PhoronixTestSuite/TestInformation/<br>
<strong>Internal Reference:</strong> P_TEST_SCALE<br>
<strong>Required:</strong> Yes<br>
<strong>Description:</strong> The scale the results are reported in.<br>
<strong>Supported Values:</strong> ( Examples: Seconds, Average FPS, Mflops )</p>
<p><strong>Tag:</strong> ResultQuantifier<br>
<strong>Location:</strong> PhoronixTestSuite/TestInformation/<br>
<strong>Internal Reference:</strong> P_TEST_QUANTIFIER<br>
<strong>Required:</strong> No - Defaults to AVG<br>
<strong>Description:</strong> When running a test multiple times, how should the final result be
calculated.<br>
<strong>Supported Values:</strong> AVG (Average), MAX (Maximum), MIN (Minimum)</p>
<p><strong>Tag:</strong> ResultFormat<br>
<strong>Location:</strong> PhoronixTestSuite/TestInformation/<br>
<strong>Internal Reference:</strong> P_TEST_RESULTFORMAT<br>
<strong>Required:</strong> No - Defaults to BAR_GRAPH<br>
<strong>Description:</strong> The type of graph/visual representation to be used when viewing the
test results.<br>
<strong>Supported Values:</strong> BAR_GRAPH (Bar Graph), LINE_GRAPH (Line Graph), PASS_FAIL (Pass-Fail
Graph)</p>
<p><strong>Tag:</strong> Proportion<br>
<strong>Location:</strong> PhoronixTestSuite/TestInformation/<br>
<strong>Internal Reference:</strong> P_TEST_PROPORTION<br>
<strong>Required:</strong> No - Defaults to HIB<br>
<strong>Description:</strong> The scale for the test, whether lower or higher results yield better
performance.<br>
<strong>Supported Values:</strong> LIB (Lower Is Better), HIB (Higher Is Better)</p>
<p><strong>Tag:</strong> Executable<br>
<strong>Location:</strong> PhoronixTestSuite/TestInformation/<br>
<strong>Internal Reference:</strong> P_TEST_EXECUTABLE<br>
<strong>Required:</strong> No - Defaults to name (identifier) of test<br>
<strong>Description:</strong> When looking for the test to execute in the path directory or the
benchmarking environment directory, this is the string of the file-name.</p>
<p><strong>Tag:</strong> TimesToRun<br>
<strong>Location:</strong> PhoronixTestSuite/TestInformation/<br>
<strong>Internal Reference:</strong> P_TEST_RUNCOUNT<br>
<strong>Required:</strong> No - Defaults to 1<br>
<strong>Description:</strong> The number of times to run the test.<br>
<strong>Supported Values:</strong> ( Integer Values )</p>
<p><strong>Tag:</strong> IgnoreFirstRun<br>
<strong>Location:</strong> PhoronixTestSuite/TestInformation/<br>
<strong>Internal Reference:</strong> P_TEST_IGNOREFIRSTRUN<br>
<strong>Required:</strong> No - Defaults to 0<br>
<strong>Description:</strong> If the test is being run multiple times, should the first value be
ignored. If the value is set to 1, the first result will be void.<br>
<strong>Supported Values:</strong> 0, 1</p>
<p><strong>Tag:</strong> PreInstallMessage<br>
<strong>Location:</strong> PhoronixTestSuite/TestInformation/<br>
<strong>Internal Reference:</strong> P_TEST_PREINSTALLMSG<br>
<strong>Required:</strong> No<br>
<strong>Description:</strong> If the user needs to be alerted about something dealing with this
test, this tag can contain a string that will be shown to the user before the
test is installed but after the files are downloaded.</p>
<p><strong>Tag:</strong> PostInstallMessage<br>
<strong>Location:</strong> PhoronixTestSuite/TestInformation/<br>
<strong>Internal Reference:</strong> P_TEST_POSTINSTALLMSG<br>
<strong>Required:</strong> No<br>
<strong>Description:</strong> If the user needs to be alerted about something dealing with this
test, this tag can contain a string that will be shown to the user after the test installation script has been executed.</p>
<p><strong>Tag:</strong> PreRunMessage<br>
<strong>Location:</strong> PhoronixTestSuite/TestInformation/<br>
<strong>Internal Reference:</strong> P_TEST_PRERUNMSG<br>
<strong>Required:</strong> No<br>
<strong>Description:</strong> If the user needs to be alerted about something dealing with this
test, this tag can contain a string that will be shown to the user before the
test begins.</p>
<p><strong>Tag:</strong> PostRunMessage<br>
<strong>Location:</strong> PhoronixTestSuite/TestInformation/<br>
<strong>Internal Reference:</strong> P_TEST_POSTRUNMSG<br>
<strong>Required:</strong> No<br>
<strong>Description:</strong> If the user needs to be alerted about something dealing with this
test, this tag can contain a string that will be shown to the user after the
test ends.</p>
<h2>PhoronixTestSuite/TestSettings/Default/</h2>
<p><strong>Tag:</strong> Arguments<br>
<strong>Location:</strong> PhoronixTestSuite/TestSettings/Default/<br>
<strong>Internal Reference:</strong> P_TEST_DEFAULTARGUMENTS<br>
<strong>Required:</strong> No<br>
<strong>Description:</strong> Any arguments to supply to the test executable by default each and
every time.</p>
<p><strong>Tag:</strong> PossiblePaths<br>
<strong>Location:</strong> PhoronixTestSuite/TestSettings/Default/<br>
<strong>Internal Reference:</strong> P_TEST_POSSIBLEPATHS<br>
<strong>Required:</strong> No<br>
<strong>Description:</strong> As an alternative to using the testing environment directory, this
tag can containe different paths to check for the test executable. Multiple paths
need to be separated by a comma.</p>
<h2>PhoronixTestSuite/TestSettings/Option/<br>
NOTE: Multiple instances of these tags are allowed for multiple options in the
test</h2>
<p><strong>Tag:</strong> DisplayName<br>
<strong>Location:</strong> PhoronixTestSuite/TestSettings/Option/<br>
<strong>Internal Reference:</strong> P_TEST_OPTIONS_DISPLAYNAME<br>
<strong>Required:</strong> No<br>
<strong>Description:</strong> If the end-user is presented with options for this test, this tag
is the full string shown to the user to describe this option.</p>
<p><strong>Tag:</strong> ArgumentName<br>
<strong>Location:</strong> PhoronixTestSuite/TestSettings/Option/<br>
<strong>Internal Reference:</strong> P_TEST_OPTIONS_ARGUMENTNAME<br>
<strong>Required:</strong> No<br>
<strong>Description:</strong> The argument that will be appended to the Arguments tag and is pre-fixed
to the Identifier tag selected by the user. <strong>Note:</strong> The use of this tag has been deprecated in
Phoronix Test Suite 1.4.0 and all test profile maintainers are encouraged to use ArgumentPrefix instead.</p>
<p><strong>Tag:</strong> ArgumentPrefix<br>
<strong>Location:</strong> PhoronixTestSuite/TestSettings/Option/<br>
<strong>Internal Reference:</strong> P_TEST_OPTIONS_ARGPREFIX<br>
<strong>Required:</strong> No<br>
<strong>Description:</strong> The argument that will be appended to the Arguments tag and is pre-fixed
to the Identifier tag selected by the user.</p>
<p><strong>Tag:</strong> ArgumentPostfix<br>
<strong>Location:</strong> PhoronixTestSuite/TestSettings/Option/<br>
<strong>Internal Reference:</strong> P_TEST_OPTIONS_ARGPOSTFIX<br>
<strong>Required:</strong> No<br>
<strong>Description:</strong> The argument that will be appended to the Arguments tag and is post-fixed
to the Identifier tag selected by the user.</p>
<p><strong>Tag:</strong> Identifier<br>
<strong>Location:</strong> PhoronixTestSuite/TestSettings/Option/<br>
<strong>Internal Reference:</strong> P_TEST_OPTIONS_IDENTIFIER<br>
<strong>Required:</strong> No<br>
<strong>Description:</strong> The internal ID to describe this option.</p>
<p><strong>Tag:</strong> DefaultEntry<br>
<strong>Location:</strong> PhoronixTestSuite/TestSettings/Option/<br>
<strong>Internal Reference:</strong> P_TEST_OPTIONS_DEFAULTENTRY<br>
<strong>Required:</strong> No<br>
<strong>Description:</strong> The integer option (starting from 1) of the menu entry to select for that option when running in defaults mode. For tests that have no
menu options or just one, this tag isn't needed. For menus with two entries, both of them are used in the defaults mode. When dealing with more than two entries, use this
tag to select the option to use otherwise the last entry will automatically be used.</p>
<p><strong>Tag:</strong> Name<br>
<strong>Location:</strong> PhoronixTestSuite/TestSettings/Option/Menu/<br>
<strong>Internal Reference:</strong> P_TEST_OPTIONS_MENU_GROUP_NAME<br>
<strong>Required:</strong> No<br>
<strong>Description:</strong> The string shown to the user to describe one of the available options
within the option.</p>
<p><strong>Tag:</strong> Value<br>
<strong>Location:</strong> PhoronixTestSuite/TestSettings/Option/Menu/<br>
<strong>Internal Reference:</strong> P_TEST_OPTIONS_MENU_GROUP_VALUE<br>
<strong>Required:</strong> No<br>
<strong>Description:</strong> The string post-fixed to the ArgumentName tag and appended to the
end of the Arguments tag, which is then supplied to the executable binary as arguments.</p>
<h2>PhoronixTestSuite/SelfContained/<br>
NOTE: These options are only available for Self-Contained Test Profiles (SCTP).</h2>
<p><strong>Tag:</strong> Installation<br>
<strong>Location:</strong> PhoronixTestSuite/SelfContained/<br>
<strong>Internal Reference:</strong> P_TEST_SCTP_INSTALLSCRIPT<br>
<strong>Required:</strong> No<br>
<strong>Description:</strong> This contains the PHP or SH script used to install the SCTP test.</p>
<p><strong>Tag:</strong> Downloads<br>
<strong>Location:</strong> PhoronixTestSuite/SelfContained/<br>
<strong>Internal Reference:</strong> P_TEST_SCTP_DOWNLOADS<br>
<strong>Required:</strong> No<br>
<strong>Description:</strong> This contains URL(s) of the file(s) that need to be downloaded for the SCTP test. Multiple URLs should be put between commas.</p>
<p><strong>Tag:</strong> ResultsParser<br>
<strong>Location:</strong> PhoronixTestSuite/SelfContained/<br>
<strong>Internal Reference:</strong> P_TEST_SCTP_RESULTSPARSER<br>
<strong>Required:</strong> No<br>
<strong>Description:</strong> This contains the PHP or SH script used for results parsing with the SCTP test.</p>
<p><strong>Tag:</strong> PreRun<br>
<strong>Location:</strong> PhoronixTestSuite/SelfContained/<br>
<strong>Internal Reference:</strong> P_TEST_SCTP_PRERUN<br>
<strong>Required:</strong> No<br>
<strong>Description:</strong> This contains the PHP or SH script that is executed prior to running a test.</p>
<p><strong>Tag:</strong> PostRun<br>
<strong>Location:</strong> PhoronixTestSuite/SelfContained/<br>
<strong>Internal Reference:</strong> P_TEST_SCTP_POSTRUN<br>
<strong>Required:</strong> No<br>
<strong>Description:</strong> This contains the PHP or SH script that is executed after running a test.</p>
<!-- END OF PTS AREA -->
</div></div><div class="pts_doc_fixed"><div class="pts_doc_bottom"><div style="float: left;"><a href="http://www.phoronix-test-suite.com/">Phoronix-Test-Suite.com</a></div><div style="float: right;">Copyright © 2008 by <a href="http://www.phoronixmedia.com/">Phoronix Media</a>.</div></div></div>
</body>
</html>
|
