HDF5 interface printer class member function definitions.
Gambit operates in an iterative fashion, collecting likelihood samples in a multidimensional space. Along with the likelihoods, we also want to store other information collected at each parameter space point. Naively the results of this search could be stored in a record array, however Gambit operates at too high a level to determine the type information necessary to define the type of the record we want to store in any given scan.
To solve this problem, while maintaining the ability to store a wide variety of types, the Gambit HDF5 interface is implemented with the following strategy:
- Every vertex ID becomes associated with its own, independent, HDF5 dataset, whose type is suited to that vertex (i.e. Gambit functor). This also means that vertices are permitted to have variable-length arrays as their record type (useful for perhaps storing event data of some kind). It is better for later analysis to store the data this way anyway, since one almost always wants one or two or several "columns" of data at a time, rather than every column (complete record) for several parameter points. We effectively store each "column" separately, which makes retrieval of just one "column" very fast.
- Each dataset contains a boolean field which indicates whether a record has been filled or not (some points fail to produce results in all functors).
- When the scanner moves to a new point, all the datasets have an entry added (via their individual buffers). The boolean flag is set to zero (indicating no result) if no result was provided before this. This keeps all the datasets synchronised, so that e.g. record 5 in all datasets refers to the same parameter space point.
- The rank 0 process handles all dataset access, which allows it to keep the datasets synchronised even when point information is coming in from several points at once.
– To elaborate on the above two points further:
- the MPI handling is a little tricky here. Every type needs to know how to serialise itself for transmission to the rank 0 process. Each process can keep a series of buffers (instead of the actual datasets) synchronised by the above method, and when they are full this can trigger the MPI send.
- The MPI receive must happen when the rank 0 process knows that it has completed one of its own points, so that a clean write to all the datasets can occur.
In order to handle the auxilliary printers, i.e. the case where extra information about points is added after the scan has moved on, the following strategy is used:
- The auxilliary printers communicate with the main printer, to let it know what information will appear in the future. The main printer then creates datasets for this future information, and keeps them (and their boolean flags) in lockstep with all the other datasets.
- When the auxilliary printer decides that it wants to print its information, a lookup is done in the datasets containing the pointID and rank information, to figure out which absolute index the auxilliary data belongs to. The pre-existing dataset created by the main printer is then updated accordingly. It might not be possible to allow variable length arrays for this auxilliary data.
- It may occur that the above lookup fails, because the auxilliary data has been computed for a point which hasn't yet been delivered to the rank 0 process (i.e. the main printer data is still in the buffer of another process). In this case the auxilliary data will remain in a buffer (with its associated pointID and rank) and the lookup will be attempted again at a later stage (unless the auxilliary printer signals that the auxilliary data is going to be completely overwritten, in which case the buffer is simply erased).
Authors (add name and date if you modify):
- Ben Farmer (email@example.com..firstname.lastname@example.org@email@example.com@firstname.lastname@example.org)
- 2015 May
Definition in file hdf5printer.cpp.