Previous: New Workloads, Up: Custom Scripts



5.3 New Benchmark Domains

Writing benchmark scripts for new domains requires forethought, and the same principles that apply to writing scripts for new workloads applies to writing scripts for new benchmark domains, See New Workloads. There are not necessarily any hard and fast rules, since everyone's benchmarks are different, but we have found that it helps to keep the following things in mind when developing file-system benchmarks.

The setup, benchmark, and cleanup should be clearly separated. If you are benchmarking multiple systems of the same type, then you should have common setup and cleanup scripts, with small blocks of if statements that change the behavior for the different systems (e.g., we have one fs-setup.sh script that handles all file systems). Instead of simply failing if your script does not recognize the system you are benchmarking, use ap_requirehook to search for a plugin.

Use variables for all of your options, and make sure you support host-specific options. If you include commonsettings, the host-specific options are already taken care of.

Finally, your setup script should actually perform a bit of cleanup as well. For example, our fs-setup.sh script unmounts any file system that is already mounted. This is important, because you don't want a previous bad run of your benchmarks to cause the next iteration to fail.