edm::StoredMergeableRunProductMetadata Class Reference

#include <StoredMergeableRunProductMetadata.h>

Classes
class	SingleRunEntry
class	SingleRunEntryAndProcess

Public Member Functions
bool &	allValidAndUseIndexIntoFile ()
bool	getLumiContent (unsigned long long runEntry, std::string const &process, bool &valid, std::vector< LuminosityBlockNumber_t >::const_iterator &lumisBegin, std::vector< LuminosityBlockNumber_t >::const_iterator &lumisEnd) const
std::vector< LuminosityBlockNumber_t > &	lumis ()
void	optimizeBeforeWrite ()
std::vector< std::string > const &	processesWithMergeableRunProducts () const
std::vector< SingleRunEntry > &	singleRunEntries ()
std::vector< SingleRunEntryAndProcess > &	singleRunEntryAndProcesses ()
	StoredMergeableRunProductMetadata ()
	StoredMergeableRunProductMetadata (std::vector< std::string > const &processesWithMergeableRunProducts)

Private Attributes
bool	allValidAndUseIndexIntoFile_
std::vector< LuminosityBlockNumber_t >	lumis_
std::vector< std::string >	processesWithMergeableRunProducts_
std::vector< SingleRunEntry >	singleRunEntries_
std::vector< SingleRunEntryAndProcess >	singleRunEntryAndProcesses_

Detailed Description

This class holds information used to decide how to merge together run products when multiple run entries with the same run number and ProcessHistoryID are read from input files contiguously. This class is persistent and stores the information that needs to be remembered from one process to the next. Most of the work related to this decision is performed by the class MergeableRunProductMetadata. The main purpose of this class is to hold the information that needs to be persistently stored. PoolSource and PoolOutputModule interface with this class to read and write it.

Note that the information is not stored for each product. The information is stored for each run entry in Run TTree in the input file and also for each process in which at least one mergeable run product was selected to be written to the output file. It is not necessary to save information for each product individually, it will be the same for every product created in the same process and in the same run entry.

The main piece of information stored is the list of luminosity block numbers processed when the product was created. Often, this list can be obtained from the IndexIntoFile and we do not need to duplicate this information here and so as an optimization we don't. There are also cases where we can detect that the merging has created invalid run products where part of the content has probably been double counted. We save a value to record this problem.

To improve performance, the data structure has been flattened into 4 vectors instead of containing a vector containing vectors containing vectors.

When the user of this class fails to find a run entry with a particular process, the assumption should be made that the lumi numbers are in IndexIntoFile and valid.

Another optimization is that if in all cases the lumi numbers can be obtained from IndexIntoFile and are valid, then all the vectors are cleared and a boolean value is set to indicate this.

Author

W. David Dagenhart, created 23 May, 2018

Definition at line 57 of file StoredMergeableRunProductMetadata.h.

Constructor & Destructor Documentation

edm::StoredMergeableRunProductMetadata::StoredMergeableRunProductMetadata

(

)

Definition at line 5 of file StoredMergeableRunProductMetadata.cc.

                                                                       :
    allValidAndUseIndexIntoFile_(true) { }

edm::StoredMergeableRunProductMetadata::StoredMergeableRunProductMetadata

(

std::vector< std::string > const &

processesWithMergeableRunProducts

)

Definition at line 9 of file StoredMergeableRunProductMetadata.cc.

                                                                                                  :
    processesWithMergeableRunProducts_(processesWithMergeableRunProducts),
    allValidAndUseIndexIntoFile_(true) { }

Member Function Documentation

bool& edm::StoredMergeableRunProductMetadata::allValidAndUseIndexIntoFile ( )

inline

Definition at line 131 of file StoredMergeableRunProductMetadata.h.

References allValidAndUseIndexIntoFile_, getLumiContent(), optimizeBeforeWrite(), LaserDQM_cfg::process, and AlCaHLTBitMon_QueryRunRegistry::string.

Referenced by edm::MergeableRunProductMetadata::addProcess().

{ return allValidAndUseIndexIntoFile_; }

bool edm::StoredMergeableRunProductMetadata::getLumiContent	(	unsigned long long	runEntry,
		std::string const &	process,
		bool &	valid,
		std::vector< LuminosityBlockNumber_t >::const_iterator &	lumisBegin,
		std::vector< LuminosityBlockNumber_t >::const_iterator &	lumisEnd
	)		const

Definition at line 51 of file StoredMergeableRunProductMetadata.cc.

References allValidAndUseIndexIntoFile_, edm::StoredMergeableRunProductMetadata::SingleRunEntryAndProcess::beginLumi(), edm::StoredMergeableRunProductMetadata::SingleRunEntry::beginProcess(), edm::StoredMergeableRunProductMetadata::SingleRunEntryAndProcess::endLumi(), edm::StoredMergeableRunProductMetadata::SingleRunEntry::endProcess(), lumis_, edm::StoredMergeableRunProductMetadata::SingleRunEntryAndProcess::process(), processesWithMergeableRunProducts_, singleRunEntries_, singleRunEntryAndProcesses_, edm::StoredMergeableRunProductMetadata::SingleRunEntryAndProcess::useIndexIntoFile(), and edm::StoredMergeableRunProductMetadata::SingleRunEntryAndProcess::valid().

Referenced by allValidAndUseIndexIntoFile(), optimizeBeforeWrite(), and edm::MergeableRunProductMetadata::readRun().

                                                                                    {

    valid = true;
    if (allValidAndUseIndexIntoFile_) {
      return false;
    }

    SingleRunEntry const& singleRunEntry = singleRunEntries_.at(runEntry);
    for (unsigned long long j = singleRunEntry.beginProcess(); j < singleRunEntry.endProcess(); ++j) {
      SingleRunEntryAndProcess const& singleRunEntryAndProcess = singleRunEntryAndProcesses_.at(j);
      // This string comparison could be optimized away by storing an index mapping in
      // MergeableRunProductMetadata that gets recalculated each time a new input
      // file is opened
      if (processesWithMergeableRunProducts_.at(singleRunEntryAndProcess.process()) == process) {
        valid = singleRunEntryAndProcess.valid();
        if (singleRunEntryAndProcess.useIndexIntoFile()) {
          return false;
        } else {
          lumisBegin = lumis_.begin() + singleRunEntryAndProcess.beginLumi();
          lumisEnd = lumis_.begin() + singleRunEntryAndProcess.endLumi();
          return true;
        }
      }
    }
    return false;
  }

std::vector<LuminosityBlockNumber_t>& edm::StoredMergeableRunProductMetadata::lumis ( )

inline

Definition at line 130 of file StoredMergeableRunProductMetadata.h.

References lumis_.

Referenced by edm::MergeableRunProductMetadata::addProcess().

{ return lumis_; }

void edm::StoredMergeableRunProductMetadata::optimizeBeforeWrite

(

)

Definition at line 41 of file StoredMergeableRunProductMetadata.cc.

References allValidAndUseIndexIntoFile_, getLumiContent(), lumis_, processesWithMergeableRunProducts_, singleRunEntries_, and singleRunEntryAndProcesses_.

Referenced by allValidAndUseIndexIntoFile(), and edm::RootOutputFile::writeStoredMergeableRunProductMetadata().

                                                              {
    if (allValidAndUseIndexIntoFile_) {
      processesWithMergeableRunProducts_.clear();
      singleRunEntries_.clear();
      singleRunEntryAndProcesses_.clear();
      lumis_.clear();
    }
  }

std::vector<std::string> const& edm::StoredMergeableRunProductMetadata::processesWithMergeableRunProducts ( ) const

inline

Definition at line 67 of file StoredMergeableRunProductMetadata.h.

References processesWithMergeableRunProducts_.

Referenced by edm::MergeableRunProductMetadata::addEntryToStoredMetadata().

                                                                          {
      return processesWithMergeableRunProducts_;
    }

std::vector<SingleRunEntry>& edm::StoredMergeableRunProductMetadata::singleRunEntries ( )

inline

Definition at line 128 of file StoredMergeableRunProductMetadata.h.

References singleRunEntries_.

Referenced by edm::MergeableRunProductMetadata::addEntryToStoredMetadata().

{ return singleRunEntries_; }

std::vector<SingleRunEntryAndProcess>& edm::StoredMergeableRunProductMetadata::singleRunEntryAndProcesses ( )

inline

Definition at line 129 of file StoredMergeableRunProductMetadata.h.

References singleRunEntryAndProcesses_.

Referenced by edm::MergeableRunProductMetadata::addEntryToStoredMetadata(), and edm::MergeableRunProductMetadata::addProcess().

{ return singleRunEntryAndProcesses_; }

Member Data Documentation

bool edm::StoredMergeableRunProductMetadata::allValidAndUseIndexIntoFile_

private

Definition at line 149 of file StoredMergeableRunProductMetadata.h.

Referenced by allValidAndUseIndexIntoFile(), getLumiContent(), and optimizeBeforeWrite().

std::vector<LuminosityBlockNumber_t> edm::StoredMergeableRunProductMetadata::lumis_

private

Definition at line 148 of file StoredMergeableRunProductMetadata.h.

Referenced by getLumiContent(), lumis(), and optimizeBeforeWrite().

std::vector<std::string> edm::StoredMergeableRunProductMetadata::processesWithMergeableRunProducts_

private

Definition at line 145 of file StoredMergeableRunProductMetadata.h.

Referenced by getLumiContent(), optimizeBeforeWrite(), and processesWithMergeableRunProducts().

std::vector<SingleRunEntry> edm::StoredMergeableRunProductMetadata::singleRunEntries_

private

Definition at line 146 of file StoredMergeableRunProductMetadata.h.

Referenced by getLumiContent(), optimizeBeforeWrite(), and singleRunEntries().

std::vector<SingleRunEntryAndProcess> edm::StoredMergeableRunProductMetadata::singleRunEntryAndProcesses_

private

Definition at line 147 of file StoredMergeableRunProductMetadata.h.

Referenced by getLumiContent(), optimizeBeforeWrite(), and singleRunEntryAndProcesses().