Previous: Hypothesis Testing, Up: Getstats

6.4 Evaluating Predicates for TEST Directives

Auto-pilot can run an arbitrary program to determine if a given benchmark requires more iterations. If the program returns true (zero), then the benchmark is complete, otherwise more iterations are required.

Getstats supports a predicate mode specifically designed to evaluate these conditions. For each quantity (e.g., Elapsed time) that is measured the predicate is evaluated independently. If the predicate is false for any of the quantities, then Getstats returns false (non-zero).

To evaluate a predicate Getstats performs replacement, and then executes Perl's eval function on the string. The primary replacements that we use are $name, $mean, and $delta (i.e., the half-width of a 95% confidence interval). For example, the following Getstats command ensures that the User, System, and Elapsed have half-widths that are less than 5% of the mean, and no test may run more than 30 times: getstats --predicate '("$name" ne "User" && "$name" ne "System" && "$name" ne "Elapsed") || ("$delta" < 0.05 * $mean) || ($count > 30)". See Replacements, for detailed information about available replacements.