Detailed DescriptionCombine data from several hdf5 files created by HDF5Printer into a single file Function Documentation◆ sigint_handler()
Definition at line 5 of file combine_hdf5.py. ◆ usage()
Definition at line 21 of file combine_hdf5.py. 22 print " Usage: python combine_hdf5.py <path-to-target-hdf5-file> <root group in hdf5 files> <tmp file 1> <tmp file 2> ..." 24 print " Attempts to combine the data in a group of hdf5 files produced by HDF5Printer in separate processes during a GAMBIT run."
Variable Documentation◆ all_RA_dsets
Definition at line 161 of file combine_hdf5.py. ◆ all_sync_dsets
Definition at line 160 of file combine_hdf5.py. ◆ alltargetindices
Definition at line 289 of file combine_hdf5.py. ◆ bufferlength
Definition at line 18 of file combine_hdf5.py. Referenced by Gambit::Printers::VertexBufferNumeric1D< T, CHUNKLENGTH >.append(), Gambit::Printers::VertexBufferNumeric1D< T, CHUNKLENGTH >.clear(), Gambit::Printers::asciiPrinter.common_constructor(), and Gambit::Printers::VertexBufferNumeric1D< T, CHUNKLENGTH >.skip_append(). ◆ chunks
Definition at line 165 of file combine_hdf5.py. ◆ chunksize
Definition at line 16 of file combine_hdf5.py. Referenced by Gambit::PostProcessor.get_done_points(). ◆ datasets
Definition at line 78 of file combine_hdf5.py. Referenced by Gambit::Printers::HDF5::hdf5_stuff.Enter_Aux_Parameters(). ◆ delete_tmp
Definition at line 44 of file combine_hdf5.py. ◆ dset_length
Definition at line 190 of file combine_hdf5.py. Referenced by Gambit::Printers::HDF5._checkDatasetReadable_helper(), Gambit::Printers::HDF5Printer2.check_consistency(), Gambit::Printers::HDF5Printer.finalise(), Gambit::Printers::DataSetInterfaceScalar< double, CHUNKLENGTH >.get_chunk(), Gambit::Printers::VertexBufferNumeric1D_HDF5< T, CHUNKLENGTH >.get_dataset_length(), Gambit::Printers::HDF5Printer.get_highest_PPID_from_HDF5(), Gambit::Printers::HDF5MasterBuffer.get_highest_PPIDs(), Gambit::PostProcessor.get_my_chunk(), Gambit::Printers::HDF5MasterBuffer.get_position_map(), Gambit::Printers::HDF5.get_RA_write_hash(), Gambit::Printers::HDF5Reader.HDF5Reader(), Gambit::Printers::DataSetInterfaceScalar< double, CHUNKLENGTH >.select_chunk(), Gambit::Printers::HDF5.selectChunk(), and Gambit::Printers::VertexBufferNumeric1D_HDF5< T, CHUNKLENGTH >.write_to_disk(). ◆ dsetlengths
Definition at line 138 of file combine_hdf5.py. ◆ dt
Definition at line 165 of file combine_hdf5.py. ◆ dtype
Definition at line 165 of file combine_hdf5.py. Referenced by Gambit::Printers::HDF5.Enter_HDF5(), Gambit::Printers::HDF5.getType(), Gambit::Printers::HDF5DataSet< uint >.write_buffer(), and Gambit::Printers::HDF5DataSet< uint >.write_RA_buffer(). ◆ existing_dsets
Definition at line 137 of file combine_hdf5.py. ◆ f
Definition at line 72 of file combine_hdf5.py. ◆ fancyindicesDefinition at line 370 of file combine_hdf5.py. ◆ files
Definition at line 61 of file combine_hdf5.py. ◆ fin
Definition at line 183 of file combine_hdf5.py. Referenced by Gambit::PostProcessor.get_done_points(), Gambit::FlavBit::Flav_reader.read_yaml(), and Gambit::FlavBit::Flav_reader.read_yaml_measurement(). ◆ fnames
Definition at line 68 of file combine_hdf5.py. ◆ fout
Definition at line 126 of file combine_hdf5.py. Referenced by Gambit::Printers::asciiPrinter.common_constructor(), and Gambit::PostProcessor.record_done_points(). ◆ gout
Definition at line 129 of file combine_hdf5.py. ◆ group
Definition at line 48 of file combine_hdf5.py. Referenced by Gambit::module_functor_common.backendreqs(), colouring.combine(), Gambit::module_functor_common.getChosenReqFromGroup(), Gambit::DRes::DependencyResolver.getObsLike(), Gambit::Printers::HDF5Printer2.HDF5Printer2(), Gambit::functor.in_allowed_combo(), main(), Gambit::functor.modelComboAllowed(), Gambit::functor.modelComboExplicitlyAllowed(), Gambit::module_functor_common.resolveBackendReq(), Gambit::module_functor_common.setBackendReq(), Gambit::functor.setModelGroup(), Gambit::functor.setSubCap(), and Gambit::DRes::DependencyResolver.solveRequirement(). ◆ ids
Definition at line 272 of file combine_hdf5.py. Referenced by Gambit::Scanner.TWalk(). ◆ IDs_in
Definition at line 234 of file combine_hdf5.py. ◆ IDs_out
Initial value:
Definition at line 262 of file combine_hdf5.py. ◆ imax
Definition at line 223 of file combine_hdf5.py. Referenced by Gambit::DarkBit::SimpleHist.addBox(). ◆ imin
Definition at line 222 of file combine_hdf5.py. Referenced by Gambit::DarkBit::SimpleHist.addBox(). ◆ index
Initial value:
Definition at line 301 of file combine_hdf5.py. Referenced by Gambit::Printers._common_print(), arg_parser(), Gambit::DarkBit.calc_lnL_Haloscope_ADMX1(), Gambit::DarkBit.calc_lnL_Haloscope_RBF(), Gambit::Priors::CompositePrior.CompositePrior(), Gambit::HiggsCouplingsTable.compute_effective_coupling(), Gambit::PolyChord::LogLikeWrapper.dumper(), Gambit::HiggsCouplingsTable.get_charged_decays(), Gambit::HiggsCouplingsTable.get_neutral_decays(), Gambit::HiggsCouplingsTable.get_neutral_decays_SM(), Gambit::DarkBit::SimYieldTable.getChannel(), Gambit::ColliderBit::BaseLimitContainer.limitAverage(), main(), Gambit::DarkBit::SimYieldTable.operator()(), Gambit::dep_bucket< TYPE >.operator*(), Gambit::dep_bucket< TYPE >.operator->(), Gambit::safe_ptr< const double >.operator[](), Gambit::safe_variable_ptr< TYPE >.operator[](), Gambit::gambit_core.process_primary_options(), daFunk::FunkIntegrate_gsl1d.resolve(), Gambit::Spectrum.safeget(), Gambit::HiggsCouplingsTable.set_charged_decays(), Gambit::HiggsCouplingsTable.set_neutral_decays(), Gambit::HiggsCouplingsTable.set_neutral_decays_SM(), Gambit.SLHAea_check_block(), Gambit.SLHAea_overwrite_block(), Gambit::Utils::specialised_threadsafe_rng< Engine >.specialised_threadsafe_rng(), Gambit::DarkBit.TH_ProcessCatalog_DS5_MSSM(), and daFunk::FunkIntegrate_gsl1d.value(). ◆ indexid
Definition at line 300 of file combine_hdf5.py. ◆ indset
Definition at line 391 of file combine_hdf5.py. ◆ init_output_length
Definition at line 147 of file combine_hdf5.py. ◆ itemDefinition at line 192 of file combine_hdf5.py. Referenced by Gambit::Printers::HDF5Reader._retrieve_template(), and Gambit::Printers::asciiPrinter.dump_buffer(). ◆ mask_in
Definition at line 230 of file combine_hdf5.py. ◆ mask_out
Definition at line 258 of file combine_hdf5.py. ◆ maskindices
Definition at line 290 of file combine_hdf5.py. ◆ MatchDefinition at line 279 of file combine_hdf5.py. ◆ max_ppidpairs
Definition at line 19 of file combine_hdf5.py. ◆ maxshape
Definition at line 165 of file combine_hdf5.py. ◆ mpiranks_inDefinition at line 226 of file combine_hdf5.py. ◆ mpiranks_isvalid_in
Definition at line 228 of file combine_hdf5.py. ◆ mpiranks_isvalid_outDefinition at line 256 of file combine_hdf5.py. ◆ mpiranks_outDefinition at line 254 of file combine_hdf5.py. ◆ N
Definition at line 50 of file combine_hdf5.py. ◆ nchunks
Definition at line 220 of file combine_hdf5.py. ◆ nextempty
Definition at line 177 of file combine_hdf5.py. ◆ Nmatches
Definition at line 276 of file combine_hdf5.py. ◆ nsources
Definition at line 317 of file combine_hdf5.py. ◆ ntargets
Definition at line 316 of file combine_hdf5.py. ◆ outdsetDefinition at line 393 of file combine_hdf5.py. ◆ outfname
Definition at line 47 of file combine_hdf5.py. ◆ pid
Definition at line 273 of file combine_hdf5.py. Referenced by Gambit::ColliderBit.convertParticleEvent(), Gambit::Printers::HDF5Reader.get_current_point(), get_HEPUtils_event(), Gambit::ColliderBit.getOSpairs(), Gambit::ColliderBit.getSSpairs(), Gambit::ColliderBit.sortByPT_l(), and Gambit::ColliderBit.SortJets(). ◆ pointIDs_inDefinition at line 225 of file combine_hdf5.py. ◆ pointIDs_isvalid_in
Definition at line 227 of file combine_hdf5.py. ◆ pointIDs_isvalid_outDefinition at line 255 of file combine_hdf5.py. ◆ pointIDs_outDefinition at line 253 of file combine_hdf5.py. ◆ RA_dsets
Definition at line 64 of file combine_hdf5.py. ◆ RA_dsets_exclude
Definition at line 65 of file combine_hdf5.py. ◆ RA_group
Definition at line 51 of file combine_hdf5.py. ◆ RA_lengths
Definition at line 67 of file combine_hdf5.py. ◆ rank
Definition at line 274 of file combine_hdf5.py. Referenced by Gambit::Printers::HDF5Reader._retrieve_template(), Gambit::Printers::HDF5Printer.common_constructor(), Gambit::Printers::HDF5::hdf5_stuff.Enter_Aux_Parameters(), Gambit::Printers::HDF5.find_temporary_files(), Gambit::PostProcessor.get_done_points(), Gambit::PostProcessor.get_effective_chunk(), Gambit::Printers::HDF5Printer2.HDF5Printer2(), Gambit::IniParser.importRound(), main(), Gambit::Likelihood_Container.main(), Gambit::SpecBit.make_vpp_inputs(), Gambit::Printers::SQLiteReader.move_to_next_point(), Gambit::Printers::HDF5DataSetBase.open_dataset(), Gambit::Scanner::like_ptr.operator()(), Gambit::SpecBit.prepare_pass_MSSM_spectrum_to_vevacious(), Gambit::module_functor< ModelParameters >.print(), Gambit::IniParser.recursiveImport(), Gambit::Printers::HDF5Printer2.resume_reader_options(), Gambit::Printers::BaseReader.retrieve(), Gambit::Printers::HDF5Reader.retrieve_and_add_to_SLHAea(), Gambit::Scanner::Scan_Manager.Run(), Gambit::Printers::HDF5::ra_copy_hdf5.run(), scanner_plugin(), Gambit::Printers::HDF5MasterBuffer.schedule_print(), Gambit::Scanner.TWalk(), and Gambit::SpecBit.vevacious_file_location_MSSM(). ◆ runchecks
Definition at line 40 of file combine_hdf5.py. ◆ sync_dsets
Definition at line 63 of file combine_hdf5.py. ◆ sync_lengths
Definition at line 66 of file combine_hdf5.py. ◆ target_dsets
Definition at line 159 of file combine_hdf5.py. ◆ target_lengthDefinition at line 288 of file combine_hdf5.py. ◆ target_mask
Definition at line 292 of file combine_hdf5.py. ◆ target_mask_smallDefinition at line 284 of file combine_hdf5.py. ◆ tmp_dset_metadata
Definition at line 79 of file combine_hdf5.py. ◆ tmp_files
Definition at line 49 of file combine_hdf5.py. Referenced by Gambit::Printers::HDF5Printer.clear_previous_points(), Gambit::Printers::HDF5Printer.common_constructor(), main(), and Gambit::Printers::HDF5Printer.prepare_and_combine_tmp_files(). ◆ tmp_RA_dset_metadata
Definition at line 80 of file combine_hdf5.py. ◆ total_sync_length
Definition at line 98 of file combine_hdf5.py. ◆ xsort
Just some test code which I decided to keep around since it is helpful for understand how the rearrangment of the input data to match the output selection works. # Compute sorting index array for rearranging the source entries to match the target locations # The way this works is a bit trippy, but it is fast. #y = target (IDs_out) #x = sources (IDs_in) #result = array of length(y), containing positions of #e.g. #x = np.array([3,5,7,1 ,9 ,8,6,6]) #y = np.array([2,1,5,10,100,6]) # out = [ - 3 1 - - 6 ] # i.e. "1" in y, is in position index 3 of x. x1 = np.array([0,1,2,3,4,5,6,7,8]) y1 = np.array([4,3,5,0,1,2,6,8,7]) # where in x is each element of y? # should give back y, since e.g. 4 is at index 4 in x # then using ypos as indices on x[sort1], should again return y. xsort1 = np.argsort(x1) ypos1 = np.searchsorted(x1[xsort1], y1) print "verifying..." print y1 print ypos1 print xsort1[ypos1] # less trivial test: x2 = np.array([0,1,7,2,7,8,6,4]) y2 = np.array([4,0,1,2,6,8,7]) # indices array should be: # [8,1,2,3,7,5,2] # (but in our case we don't want duplicates in either the target or input arrays!) xsort2 = np.argsort(x2) ypos2 = np.searchsorted(x2[xsort2], y2) indices = xsort2[ypos2] print "verifying..." print x2 print xsort2 print x2[xsort2] print "y2" print y2 print ypos2 print indices print x2[indices] Definition at line 368 of file combine_hdf5.py. ◆ yindex
Definition at line 369 of file combine_hdf5.py. |