Re: [vsipl++] patch: Set up parallel service(s) during testing.

[Jules Bergmann <jules@xxxxxxxxxxxxxxxx>]

Stefan Seefeld wrote:
> The attached patch adds a new 'ParallelService' resource to the
> test database which, if lam is used, runs 'lamboot' in its SetUp
> method and 'wipe' during CleanUp. (We can complete this whenever
> other services are needed for other MPI implementations.)
>
> This new resource is associated with all tests that live under
> the 'parallel/' subdirectory. (The next logical step is thus to
> identify all tests requiring this service and move them.)
>
> Ok to commit ?
Stefan, looks good, please check it in.

Here are some thoughts on how we might want to do parallel testing.

All of our parallel tests work (in theory) with any number of procesors 
from 1 to infinity.  We should continue to run all tests with a single 
processor by default.  I.e. 'qmtest run' would run every test either 
directly, or via 'mpirun -np 1 test".
There should be some way to run the parallel tests with a specifed 
number of processors.  Since they're off in a separate directory, maybe 
it looks something like:
	qmtest run -mpiopt "-np 2" parallel

If qmtest just does an 'lamboot' with no other options, all processors 
are virtual that run on the same host.  I.e. even though cugel only has 
2 CPUs, you could run parallel tests with 8 processors.  This won't run 
8 times faster, but it will stress communications, etc. and is a 
valuable form of testing.
However, it is possible to pass a configuration file (called the "LAM 
boot schema") to lamboot to tell it what set of machines to run 
processes on, other than the local host.  We could in theory pass a 
configuration file with sethra, cugel, and sparrowhack and then run 
tests on 6 phyiscal processors.  We shouldn't hardcode this config file 
into either qmtest or configure, it is enough to add an option that 
passes an optional bhost file to lamboot, i.e.
	qmtest run -mpiopt "-np 2" -bhost BHOSTFILE parallel

With this in place, when building a release on cugel, we might do:

	# test everything with 1 processor:
	qmtest run

	# test parallel tests with 2 and 4 processors:
	qmtest run -mpiopt "-np 2" parallel
	qmtest run -mpiopt "-np 4" parallel

When running the GTRI cluster, we could be bold and try larger 
configurations:
	qmtest run -bhost cluster -mpiopt "-mp 64" parallel


If you want to avoid cluttering up the qmtest option space with MPI 
specific options like -mpiopt and -bhost, you might have a single option 
that allows parameters to be passed to the chosen par-services:
	qmtest run -paropt "np=2" -paropt "bhost=host.file" parallel

				-- Jules