|
|
|
#include <IndexIntoFile.h>
Used to quickly find the Events, Lumis, and Runs in a single ROOT format data file and step through them in the desired order.
A list of the most important functions that a client would use directly follows. There are detailed comments below with the declaration of each function.
The begin and end functions are used to start and stop an iteration loop. An argument to the iterator constructor determines the order of iteration.
The functions findPosition, findEventPosition, findRunPosition, and findLumiPosition are used to navigate directly to specific runs, lumis, and events.
The functions mentioned above return an object of type IndexIntoFileItr. The IndexIntoFileItr class has member functions which allow one to navigate forward and backward through the runs, lumis, and events in alternative ways. See more comments with the declaration of each public member function in IndexIntoFileItr.
The iterator will know what the current item is (as one would expect). This could be a run, lumi, or event. It knows more than that though, it knows all three as is explained below.
In the run state, IndexIntoFileItr knows which lumi will be processed next after the run and also which event will be processed after the lumi. These may not be the first ones in the run if the skip functions were used.
In the lumi state, the IndexIntoFileItr will always point at the last associated run and the next event to be processed after the lumi. This may not be the first event if the skip function was used.
In the event state, the IndexIntoFileItr will always point at the last corresponding run and also the last corresponding lumi.
There can be multiple run entries in a TTree associated with the same run number and ProcessHistoryID in a file. There can also be multiple lumi entries associated with the same lumi number, run number, and ProcessHistoryID. Both sorting orders will make these subgroups contiguous, but beyond that is up to the client (normally PoolSource, which passes them up to the EventProcessor and EPStates) to deal with merging the multiple run (or lumi) entries together.
One final comment with regards to IndexIntoFileItr. This is not an STL iterator and it cannot be used with std:: algorithms. The data structures are complex and designed to optimize memory usage. It would be difficult or impossible implement an iterator that is STL compliant.
Here is a summary of the data structures in IndexIntoFile. The persistent data consists of two vectors.
processHistoryIDs_ is a std::vector<ProcessHistoryID> that contains the ProcessHistoryIDs with one element in the vector for each unique ProcessHistoryID. On output they are ordered as they first written out for each output file. On input they are ordered as they are first seen in each process. Note that each ProcessHistoryID is stored once in this separate vector. Everywhere else it is needed it stored as an index into this vector because the ProcessHistoryID itself is large and it would take more memory to store them repeatedly in the other vectors.
runOrLumiEntries_ is a std::vector<RunOrLumiEntry>. This vector holds one element per entry in the run TTree and one element per entry in the lumi TTree. When sorted, everything associated with a given run and ProcessHistoryID will be contiguous in the vector. These groups of entries will be put in the order they first appear in the input file. Within each of these groups the run entries come first in entry order, followed by the entries associated with the lumis. The lumis are also contiguous and sorted by first appearance in the input file. Within a lumi they are sorted by entry order.
There are a number of transient data members also. The 3 most important of these are vectors. To save memory, these are only filled when needed.
runOrLumiIndexes_ is a std::vector<RunOrLumiIndexes>. There is a one to one correspondence between the elements of this vector and the elements of runOrLumiEntries_. The elements of this vector are sorted in numerical order using the ProcessHistoryID index, the run number, and the lumi number. This ordering allows iteration in numerical order and also fast lookup based on run number and lumi number. Each element also has indexes into the eventNumbers_ and eventEntries_ vectors which hold the information giving the event numbers and event entry numbers.
eventNumbers_ is a std::vector containing EventNumber_t's. Each element is a 4 byte int. eventEntries_ is a std::vector containing EventEntry's. Each EventEntry contains a 4 byte event number and an 8 byte entry number. If filled, both vectors contain the same number of entries with identical event numbers sorted in the same order. The only difference is that one includes the entry numbers and thus takes more memory. Each element of runOrLumiIndexes_ has the indexes necessary to find the range inside eventNumbers_ or eventEntries_ corresponding to its lumi. Within that range the elements are sorted by event number, which is used for the numerical order iteration and to find an event by the event number.
The details of the data structure are a little different when reading files written before release 3_8_0 (backward compatibility, see RootFile::fillIndexIntoFile for the details).
This data structure is optimized for low memory usage when there are large numbers of events. The optimal case would occur when there was was one run in a file, one luminosity block in that run and everything had the same ProcessHistoryID. If duplicate checking were off and the process simply iterated through the file in the default order, then only the persistent vectors would be filled. One vector would contain 2 elements, one for the run and the other for the lumi. The other vector would contain one element, the ProcessHistoryID. Even if there were a billion events, that would be all that would exist and take up memory. The data structure is not the optimal structure for a very sparse skim, but the overheads should be tolerable given the number of runs and luminosity blocks that should occur in CMS data.
Normally the only the persistent part of the data structure is filled in the output module using two functions designed specifically for that purpose. The functions are addEntry and sortVector_Run_Or_Lumi_Entries.
There are some complexities associated with filling the data structure, mostly revolving around optimizations to minimize the per event memory usage. The client needs to know which parts of the data structure to fill. See the functions below named fixIndexes, setNumberOfEvents, setEventFinder, fillEventNumbers, fillEventEntries, and inputFileClosed.
Note that this class is not intended to be used directly by the average CMS user. PoolSource and PoolOutputModule are the main clients. Other executables that read ROOT format data files, but do not use PoolSource may also need to use it directly (FWLite, Fireworks, edmFileUtil ...). The interface is too complex for general use.
Definition at line 180 of file IndexIntoFile.h.
| typedef long long edm::IndexIntoFile::EntryNumber_t |
Definition at line 184 of file IndexIntoFile.h.
This enum is used to specify the order of iteration. In firstAppearanceOrder there are 3 sort criteria, in order of precedence these are:
1. firstAppearance of the ProcessHistoryID and run number in the file
2. firstAppearance of the ProcessHistoryID, run number and lumi number in the file
3. entry number
In numerical order the criteria are in order of precedence are:
1. processHistoryID index (which are normally in order of appearance in the process)
2. run number
3. lumi number
4. event number
5. entry number
Definition at line 221 of file IndexIntoFile.h.
| edm::IndexIntoFile::IndexIntoFile | ( | ) |
Definition at line 17 of file IndexIntoFile.cc.
: transients_(), processHistoryIDs_(), runOrLumiEntries_() { }
| edm::IndexIntoFile::~IndexIntoFile | ( | ) |
Definition at line 22 of file IndexIntoFile.cc.
{
}
| void edm::IndexIntoFile::addEntry | ( | ProcessHistoryID const & | processHistoryID, |
| RunNumber_t | run, | ||
| LuminosityBlockNumber_t | lumi, | ||
| EventNumber_t | event, | ||
| EntryNumber_t | entry | ||
| ) |
Used by RootOutputModule to fill the persistent data. This will not work properly if entries are not added in the same order as in RootOutputModule
Definition at line 34 of file IndexIntoFile.cc.
References beginEvents(), currentIndex(), currentLumi(), currentRun(), endEvents(), Exception, getHLTprescales::index, invalidEntry, invalidEvent, invalidIndex, invalidLumi, invalidRun, edm::errors::LogicError, fjr2json::lumi, lumiToFirstEntry(), numberOfEvents(), previousAddedIndex(), processHistoryIDs_, DTTTrigCorrFirst::run, runOrLumiEntries_, runToFirstEntry(), and setNumberOfEvents().
Referenced by edm::RootOutputFile::writeLuminosityBlock(), edm::RootOutputFile::writeOne(), and edm::RootOutputFile::writeRun().
{
int index = 0;
// First see if the ProcessHistoryID is the same as the previous one.
// This is just a performance optimization. We expect to usually get
// many in a row that are the same.
if (previousAddedIndex() != invalidIndex &&
processHistoryID == processHistoryIDs_[previousAddedIndex()]) {
index = previousAddedIndex();
}
// If it was not the same as the previous one then search through the
// entire vector. If it is not there, it needs to be added at the
// end.
else {
index = 0;
while (index < static_cast<int>(processHistoryIDs_.size()) &&
processHistoryIDs_[index] != processHistoryID) {
++index;
}
if (index == static_cast<int>(processHistoryIDs_.size())) {
processHistoryIDs_.push_back(processHistoryID);
}
}
previousAddedIndex() = index;
assert((currentRun() == run && currentIndex() == index) || currentRun() == invalidRun);
if (lumi == invalidLumi) {
if (currentLumi() != invalidLumi) {
throw Exception(errors::LogicError)
<< "In IndexIntoFile::addEntry. Entries were added in illegal order.\n"
<< "This means the IndexIntoFile product in the output file will be corrupted.\n"
<< "The output file will be unusable for most purposes.\n"
<< "If this occurs after an unrelated exception was thrown in\n"
<< "endLuminosityBlock or endRun then ignore this exception and fix\n"
<< "the primary exception. This is an expected side effect.\n"
<< "Otherwise please report this to the core framework developers\n";
}
currentIndex() = invalidIndex;
currentRun() = invalidRun;
currentLumi() = invalidLumi;
std::pair<IndexRunKey, EntryNumber_t> firstRunEntry(IndexRunKey(index, run), entry);
runToFirstEntry().insert(firstRunEntry);
RunOrLumiEntry runEntry(runToFirstEntry()[IndexRunKey(index, run)], invalidEntry, entry, index, run, lumi, invalidEntry, invalidEntry);
runOrLumiEntries_.push_back(runEntry);
}
else {
assert(currentLumi() == lumi || currentLumi() == invalidLumi);
if (currentRun() == invalidRun) {
currentRun() = run;
currentIndex() = index;
}
if (event == invalidEvent) {
currentLumi() = invalidLumi;
std::pair<IndexRunLumiKey, EntryNumber_t> firstLumiEntry(IndexRunLumiKey(index, run, lumi), entry);
lumiToFirstEntry().insert(firstLumiEntry);
RunOrLumiEntry lumiEntry(invalidEntry, lumiToFirstEntry()[IndexRunLumiKey(index, run, lumi)],
entry, index, run, lumi, beginEvents(), endEvents());
runOrLumiEntries_.push_back(lumiEntry);
beginEvents() = invalidEntry;
endEvents() = invalidEntry;
}
else {
setNumberOfEvents(numberOfEvents() + 1);
if (beginEvents() == invalidEntry) {
currentLumi() = lumi;
beginEvents() = entry;
endEvents() = beginEvents() + 1;
}
else {
assert(currentLumi() == lumi);
assert(entry == endEvents());
++endEvents();
}
}
}
}
| IndexIntoFile::IndexIntoFileItr edm::IndexIntoFile::begin | ( | SortOrder | sortOrder | ) | const |
Used to start an iteration over the Runs, Lumis, and Events in a file. Note the argument specifies the order
Definition at line 394 of file IndexIntoFile.cc.
References empty(), end(), edm::IndexIntoFile::IndexIntoFileItr::initializeRun(), invalidIndex, and kRun.
Referenced by containsDuplicateEvents(), findPosition(), iterationWillBeInEntryOrder(), edm::postIndexIntoFilePrintEventLists(), edm::postIndexIntoFilePrintEventsInLumis(), edm::RootFile::RootFile(), set_intersection(), sortEventEntries(), and sortEvents().
{
if (empty()) {
return end(sortOrder);
}
IndexIntoFileItr iter(this,
sortOrder,
kRun,
0,
invalidIndex,
invalidIndex,
0,
0);
iter.initializeRun();
return iter;
}
| EntryNumber_t& edm::IndexIntoFile::beginEvents | ( | ) | const [inline, private] |
Definition at line 1009 of file IndexIntoFile.h.
References edm::Transient< T >::get(), and transients_.
Referenced by addEntry(), edm::IndexIntoFile::IndexIntoFileItrNoSort::initializeLumi_(), edm::IndexIntoFile::IndexIntoFileItrNoSort::nextEventRange(), edm::IndexIntoFile::IndexIntoFileItrNoSort::previousEventRange(), and edm::IndexIntoFile::IndexIntoFileItrNoSort::setToLastEventInRange().
{return transients_.get().beginEvents_;}
| IndexIntoFile::SortedRunOrLumiItr edm::IndexIntoFile::beginRunOrLumi | ( | ) | const |
Definition at line 670 of file IndexIntoFile.cc.
Referenced by containsDuplicateEvents(), fillEventNumbersOrEntries(), and set_intersection().
{
return SortedRunOrLumiItr(this, 0);
}
| bool edm::IndexIntoFile::containsDuplicateEvents | ( | ) | const |
Returns true if the IndexIntoFile contains 2 events with the same ProcessHistoryID index, run number, lumi number and event number.
Definition at line 786 of file IndexIntoFile.cc.
References begin(), edm::IndexIntoFile::RunOrLumiIndexes::beginEventNumbers(), beginRunOrLumi(), empty(), edm::IndexIntoFile::RunOrLumiIndexes::endEventNumbers(), endRunOrLumi(), eventEntries(), eventNumbers(), fillEventNumbers(), edm::IndexIntoFile::RunOrLumiIndexes::isRun(), and prof2calltree::last.
Referenced by edm::DuplicateChecker::inputFileOpened().
{
RunOrLumiIndexes const* previousIndexes = 0;
for (SortedRunOrLumiItr iter = beginRunOrLumi(),
iEnd = endRunOrLumi();
iter != iEnd; ++iter) {
RunOrLumiIndexes const& indexes = iter.runOrLumiIndexes();
// Skip it if it is a run or the same lumi
if (indexes.isRun() ||
(previousIndexes && !(*previousIndexes < indexes))) {
continue;
}
previousIndexes = &indexes;
long long beginEventNumbers = indexes.beginEventNumbers();
long long endEventNumbers = indexes.endEventNumbers();
// there must be more than 1 event in the lumi for there to be any duplicates
if (beginEventNumbers + 1 >= endEventNumbers) continue;
if (!eventEntries().empty()) {
std::vector<EventEntry>::iterator last = eventEntries().begin() + endEventNumbers;
if (std::adjacent_find(eventEntries().begin() + beginEventNumbers, last) != last) {
return true;
}
} else {
fillEventNumbers();
std::vector<EventNumber_t>::iterator last = eventNumbers().begin() + endEventNumbers;
if (std::adjacent_find(eventNumbers().begin() + beginEventNumbers, last) != last) {
return true;
}
}
}
return false;
}
| bool edm::IndexIntoFile::containsEvent | ( | RunNumber_t | run, |
| LuminosityBlockNumber_t | lumi, | ||
| EventNumber_t | event | ||
| ) | const |
Definition at line 656 of file IndexIntoFile.cc.
References findEventPosition(), edm::IndexIntoFile::IndexIntoFileItr::getEntryType(), and kEnd.
Referenced by containsItem().
{
return findEventPosition(run, lumi, event).getEntryType() != kEnd;
}
| bool edm::IndexIntoFile::containsItem | ( | RunNumber_t | run, |
| LuminosityBlockNumber_t | lumi, | ||
| EventNumber_t | event | ||
| ) | const |
Definition at line 651 of file IndexIntoFile.cc.
References containsEvent(), containsLumi(), and containsRun().
{
return event ? containsEvent(run, lumi, event) : (lumi ? containsLumi(run, lumi) : containsRun(run));
}
| bool edm::IndexIntoFile::containsLumi | ( | RunNumber_t | run, |
| LuminosityBlockNumber_t | lumi | ||
| ) | const |
Definition at line 661 of file IndexIntoFile.cc.
References findLumiPosition(), edm::IndexIntoFile::IndexIntoFileItr::getEntryType(), and kEnd.
Referenced by containsItem().
{
return findLumiPosition(run, lumi).getEntryType() != kEnd;
}
| bool edm::IndexIntoFile::containsRun | ( | RunNumber_t | run | ) | const |
Definition at line 666 of file IndexIntoFile.cc.
References findRunPosition(), edm::IndexIntoFile::IndexIntoFileItr::getEntryType(), and kEnd.
Referenced by containsItem().
{
return findRunPosition(run).getEntryType() != kEnd;
}
| int& edm::IndexIntoFile::currentIndex | ( | ) | const [inline, private] |
Definition at line 1011 of file IndexIntoFile.h.
References edm::Transient< T >::get(), and transients_.
Referenced by addEntry().
{return transients_.get().currentIndex_;}
| LuminosityBlockNumber_t& edm::IndexIntoFile::currentLumi | ( | ) | const [inline, private] |
Definition at line 1013 of file IndexIntoFile.h.
References edm::Transient< T >::get(), and transients_.
Referenced by addEntry().
{return transients_.get().currentLumi_;}
| RunNumber_t& edm::IndexIntoFile::currentRun | ( | ) | const [inline, private] |
Definition at line 1012 of file IndexIntoFile.h.
References edm::Transient< T >::get(), and transients_.
Referenced by addEntry().
{return transients_.get().currentRun_;}
| void edm::IndexIntoFile::doneFileInitialization | ( | ) | const |
Clears the temporary vector of event numbers to reduce memory usage.
Definition at line 288 of file IndexIntoFile.cc.
References edm::swap(), and unsortedEventNumbers().
Referenced by edm::RootFile::RootFile().
{
std::vector<EventNumber_t>().swap(unsortedEventNumbers());
}
| bool edm::IndexIntoFile::empty | ( | ) | const |
True if no runs, lumis, or events are in the file.
Definition at line 434 of file IndexIntoFile.cc.
References runOrLumiEntries().
Referenced by begin(), containsDuplicateEvents(), fwlite::EntryFinder::empty(), fillEventNumbersOrEntries(), fillUnsortedEventNumbers(), fwlite::EntryFinder::findEvent(), fwlite::EntryFinder::findLumi(), findPosition(), fwlite::EntryFinder::findRun(), and set_intersection().
{
return runOrLumiEntries().empty();
}
| IndexIntoFile::IndexIntoFileItr edm::IndexIntoFile::end | ( | SortOrder | sortOrder | ) | const |
Used to end an iteration over the Runs, Lumis, and Events in a file.
Definition at line 410 of file IndexIntoFile.cc.
References invalidIndex, and kEnd.
Referenced by begin(), fwlite::EntryFinder::findEvent(), fwlite::EntryFinder::findLumi(), findPosition(), fwlite::EntryFinder::findRun(), iterationWillBeInEntryOrder(), edm::postIndexIntoFilePrintEventLists(), edm::postIndexIntoFilePrintEventsInLumis(), and edm::RootFile::RootFile().
{
return IndexIntoFileItr(this,
sortOrder,
kEnd,
invalidIndex,
invalidIndex,
invalidIndex,
0,
0);
}
| EntryNumber_t& edm::IndexIntoFile::endEvents | ( | ) | const [inline, private] |
Definition at line 1010 of file IndexIntoFile.h.
References edm::Transient< T >::get(), and transients_.
Referenced by addEntry(), edm::IndexIntoFile::IndexIntoFileItrNoSort::initializeLumi_(), edm::IndexIntoFile::IndexIntoFileItrNoSort::nextEventRange(), edm::IndexIntoFile::IndexIntoFileItrNoSort::previousEventRange(), and edm::IndexIntoFile::IndexIntoFileItrNoSort::setToLastEventInRange().
{return transients_.get().endEvents_;}
| IndexIntoFile::SortedRunOrLumiItr edm::IndexIntoFile::endRunOrLumi | ( | ) | const |
Definition at line 674 of file IndexIntoFile.cc.
References runOrLumiEntries(), and findQualityFiles::size.
Referenced by containsDuplicateEvents(), fillEventNumbersOrEntries(), and set_intersection().
{
return SortedRunOrLumiItr(this, runOrLumiEntries().size());
}
| std::vector<EventEntry>& edm::IndexIntoFile::eventEntries | ( | ) | const [inline, private] |
Definition at line 1002 of file IndexIntoFile.h.
References edm::Transient< T >::get(), and transients_.
Referenced by containsDuplicateEvents(), fillEventNumbersOrEntries(), findPosition(), inputFileClosed(), set_intersection(), and sortEventEntries().
{return transients_.get().eventEntries_;}
| std::vector<EventNumber_t>& edm::IndexIntoFile::eventNumbers | ( | ) | const [inline, private] |
Definition at line 1003 of file IndexIntoFile.h.
References edm::Transient< T >::get(), and transients_.
Referenced by containsDuplicateEvents(), fillEventNumbersOrEntries(), findPosition(), set_intersection(), and sortEvents().
{return transients_.get().eventNumbers_;}
| void edm::IndexIntoFile::fillEventEntries | ( | ) | const |
Fills a vector of objects that contain a 4 byte event number and the corresponding TTree entry number (8 bytes) for the event. Not filling it reduces the memory used by IndexIntoFile. As long as the event finder is still pointing at an open file this will automatically be called on demand (when the event numbers and entries are are needed). It makes sense for the client to fill this explicitly in advance if it is known that it will be needed, because in some cases this will prevent the event numbers vector from being unnecessarily filled (wasting memory). This vector will be needed when iterating over events in numerical order or looking up specific events. The entry numbers are needed if the events are actually read from the input file.
Definition at line 177 of file IndexIntoFile.cc.
References fillEventNumbersOrEntries().
{
fillEventNumbersOrEntries(false, true);
}
| void edm::IndexIntoFile::fillEventNumbers | ( | ) | const |
Fills a vector of 4 byte event numbers. Not filling it reduces the memory used by IndexIntoFile. As long as the event finder is still pointing at an open file this will automatically be called on demand (when the event numbers are are needed). In cases, where the input file may be closed when the need arises, the client code must call this explicitly and fill the vector before the file is closed. In PoolSource, this is necessary when duplicate checking across all files and when doing lookups to see if an event is in a previously opened file. Either this vector or the one that also contains event entry numbers can be used when looking for duplicate events within the same file or looking up events in in the current file without reading them.
Definition at line 172 of file IndexIntoFile.cc.
References fillEventNumbersOrEntries().
Referenced by containsDuplicateEvents(), findPosition(), and set_intersection().
{
fillEventNumbersOrEntries(true, false);
}
| void edm::IndexIntoFile::fillEventNumbersOrEntries | ( | bool | needEventNumbers, |
| bool | needEventEntries | ||
| ) | const |
If needEventNumbers is true then this function does the same thing as fillEventNumbers. If NeedEventEntries is true, then this function does the same thing as fillEventEntries. If both are true, it fills both within the same loop and it uses less CPU than calling those two functions separately.
Definition at line 182 of file IndexIntoFile.cc.
References asciidump::at, beginRunOrLumi(), empty(), endRunOrLumi(), event(), eventEntries(), eventNumbers(), fillUnsortedEventNumbers(), invalidEvent, numberOfEvents(), evf::evtn::offset(), findQualityFiles::size, sortEventEntries(), sortEvents(), and unsortedEventNumbers().
Referenced by fillEventEntries(), and fillEventNumbers().
{
if (numberOfEvents() == 0) {
return;
}
if (needEventNumbers && !eventNumbers().empty()) {
needEventNumbers = false;
}
if (needEventEntries && !eventEntries().empty()) {
needEventEntries = false;
}
if (needEventNumbers && !eventEntries().empty()) {
assert(numberOfEvents() == eventEntries().size());
eventNumbers().reserve(eventEntries().size());
for (std::vector<EventNumber_t>::size_type entry = 0U; entry < numberOfEvents(); ++entry) {
eventNumbers().push_back(eventEntries()[entry].event());
}
return;
}
if (!needEventNumbers && !needEventEntries) {
return;
}
fillUnsortedEventNumbers();
if (needEventNumbers) {
eventNumbers().resize(numberOfEvents(), IndexIntoFile::invalidEvent);
}
if (needEventEntries) {
eventEntries().resize(numberOfEvents());
}
long long offset = 0;
long long previousBeginEventNumbers = -1LL;
for (SortedRunOrLumiItr runOrLumi = beginRunOrLumi(), runOrLumiEnd = endRunOrLumi();
runOrLumi != runOrLumiEnd; ++runOrLumi) {
if (runOrLumi.isRun()) continue;
long long beginEventNumbers = 0;
long long endEventNumbers = 0;
EntryNumber_t beginEventEntry = -1LL;
EntryNumber_t endEventEntry = -1LL;
runOrLumi.getRange(beginEventNumbers, endEventNumbers, beginEventEntry, endEventEntry);
// This is true each time one hits a new lumi section (except if the previous lumi had
// no events, in which case the offset is still 0 anyway)
if (beginEventNumbers != previousBeginEventNumbers) offset = 0;
for (EntryNumber_t entry = beginEventEntry; entry != endEventEntry; ++entry) {
if (needEventNumbers) {
eventNumbers().at((entry - beginEventEntry) + offset + beginEventNumbers) = unsortedEventNumbers().at(entry);
}
if (needEventEntries) {
eventEntries().at((entry - beginEventEntry) + offset + beginEventNumbers) =
EventEntry(unsortedEventNumbers().at(entry), entry);
}
}
previousBeginEventNumbers = beginEventNumbers;
offset += endEventEntry - beginEventEntry;
}
if (needEventNumbers) {
sortEvents();
assert(numberOfEvents() == eventNumbers().size());
}
if (needEventEntries) {
sortEventEntries();
assert(numberOfEvents() == eventEntries().size());
}
}
| void edm::IndexIntoFile::fillRunOrLumiIndexes | ( | ) | const [private] |
This function will automatically get called when needed. It depends only on the fact that the persistent data has been filled already.
Definition at line 114 of file IndexIntoFile.cc.
References getHLTprescales::index, invalidEntry, nEvents, runOrLumiEntries_, runOrLumiIndexes(), findQualityFiles::size, and edm::stable_sort_all().
Referenced by findPosition(), edm::IndexIntoFile::IndexIntoFileItrSorted::IndexIntoFileItrSorted(), set_intersection(), edm::IndexIntoFile::SortedRunOrLumiItr::SortedRunOrLumiItr(), sortEventEntries(), and sortEvents().
{
if (runOrLumiEntries_.empty() || !runOrLumiIndexes().empty()) {
return;
}
runOrLumiIndexes().reserve(runOrLumiEntries_.size());
int index = 0;
for (std::vector<RunOrLumiEntry>::const_iterator iter = runOrLumiEntries_.begin(),
iEnd = runOrLumiEntries_.end();
iter != iEnd;
++iter, ++index) {
runOrLumiIndexes().push_back(RunOrLumiIndexes(iter->processHistoryIDIndex(),
iter->run(),
iter->lumi(),
index));
}
stable_sort_all(runOrLumiIndexes());
long long beginEventNumbers = 0;
std::vector<RunOrLumiIndexes>::iterator beginOfLumi = runOrLumiIndexes().begin();
std::vector<RunOrLumiIndexes>::iterator endOfLumi = beginOfLumi;
std::vector<RunOrLumiIndexes>::iterator iEnd = runOrLumiIndexes().end();
while (true) {
while (beginOfLumi != iEnd && beginOfLumi->isRun()) {
++beginOfLumi;
}
if (beginOfLumi == iEnd) break;
endOfLumi = beginOfLumi + 1;
while (endOfLumi != iEnd &&
beginOfLumi->processHistoryIDIndex() == endOfLumi->processHistoryIDIndex() &&
beginOfLumi->run() == endOfLumi->run() &&
beginOfLumi->lumi() == endOfLumi->lumi()) {
++endOfLumi;
}
int nEvents = 0;
for (std::vector<RunOrLumiIndexes>::iterator iter = beginOfLumi;
iter != endOfLumi;
++iter) {
if (runOrLumiEntries_[iter->indexToGetEntry()].beginEvents() != invalidEntry) {
nEvents += runOrLumiEntries_[iter->indexToGetEntry()].endEvents() -
runOrLumiEntries_[iter->indexToGetEntry()].beginEvents();
}
}
for (std::vector<RunOrLumiIndexes>::iterator iter = beginOfLumi;
iter != endOfLumi;
++iter) {
iter->setBeginEventNumbers(beginEventNumbers);
iter->setEndEventNumbers(beginEventNumbers + nEvents);
}
beginEventNumbers += nEvents;
beginOfLumi = endOfLumi;
}
assert(runOrLumiIndexes().size() == runOrLumiEntries_.size());
}
| void edm::IndexIntoFile::fillUnsortedEventNumbers | ( | ) | const [private] |
Definition at line 259 of file IndexIntoFile.cc.
References empty(), getEventNumberOfEntry(), numberOfEvents(), and unsortedEventNumbers().
Referenced by fillEventNumbersOrEntries().
{
if (numberOfEvents() == 0 || !unsortedEventNumbers().empty()) {
return;
}
unsortedEventNumbers().reserve(numberOfEvents());
// The main purpose for the existence of the unsortedEventNumbers
// vector is that it can easily be filled by reading through
// the EventAuxiliary branch in the same order as the TTree
// entries. fillEventNumbersOrEntries can then use this information
// instead of using getEventNumberOfEntry directly and reading
// the branch in a different order.
for (std::vector<EventNumber_t>::size_type entry = 0U; entry < numberOfEvents(); ++entry) {
unsortedEventNumbers().push_back(getEventNumberOfEntry(entry));
}
}
| IndexIntoFile::IndexIntoFileItr edm::IndexIntoFile::findEventPosition | ( | RunNumber_t | run, |
| LuminosityBlockNumber_t | lumi, | ||
| EventNumber_t | event | ||
| ) | const |
Same as findPosition,except the entry type of the returned iterator will be kEvent or kEnd and the event argument must be nonzero. This means the next thing to be processed will be the event if it is found.
Definition at line 630 of file IndexIntoFile.cc.
References edm::IndexIntoFile::IndexIntoFileItr::advanceToEvent(), findPosition(), and invalidEvent.
Referenced by containsEvent(), and fwlite::EntryFinder::findEvent().
{
assert(event != invalidEvent);
IndexIntoFileItr iter = findPosition(run, lumi, event);
iter.advanceToEvent();
return iter;
}
| IndexIntoFile::IndexIntoFileItr edm::IndexIntoFile::findLumiPosition | ( | RunNumber_t | run, |
| LuminosityBlockNumber_t | lumi | ||
| ) | const |
Same as findPosition,except the entry type of the returned iterator will be kLumi or kEnd and the lumi argument must be nonzero. This means the next thing to be processed will be the lumi if it is found.
Definition at line 638 of file IndexIntoFile.cc.
References edm::IndexIntoFile::IndexIntoFileItr::advanceToLumi(), findPosition(), and invalidLumi.
Referenced by containsLumi(), and fwlite::EntryFinder::findLumi().
{
assert(lumi != invalidLumi);
IndexIntoFileItr iter = findPosition(run, lumi, 0U);
iter.advanceToLumi();
return iter;
}
| IndexIntoFile::IndexIntoFileItr edm::IndexIntoFile::findPosition | ( | SortOrder | sortOrder, |
| RunNumber_t | run, | ||
| LuminosityBlockNumber_t | lumi = 0U, |
||
| EventNumber_t | event = 0U |
||
| ) | const |
Definition at line 586 of file IndexIntoFile.cc.
References edm::IndexIntoFile::IndexIntoFileItr::advanceToNextRun(), begin(), end(), findPosition(), getEventNumberOfEntry(), invalidEvent, invalidLumi, fjr2json::lumi, numericalOrder, edm::IndexIntoFile::IndexIntoFileItr::peekAheadAtEventEntry(), edm::IndexIntoFile::IndexIntoFileItr::peekAheadAtLumi(), edm::IndexIntoFile::IndexIntoFileItr::run(), DTTTrigCorrFirst::run, edm::IndexIntoFile::IndexIntoFileItr::skipLumiInRun(), and edm::IndexIntoFile::IndexIntoFileItr::skipToNextEventInLumi().
{
if (sortOrder == IndexIntoFile::numericalOrder) {
return findPosition(run, lumi, event); // a faster algorithm
}
IndexIntoFileItr itr = begin(sortOrder);
IndexIntoFileItr itrEnd = end(sortOrder);
while (itr != itrEnd) {
if (itr.run() != run) {
itr.advanceToNextRun();
}
else {
if (lumi == invalidLumi && event == invalidEvent) {
return itr;
}
else if (lumi != invalidLumi && itr.peekAheadAtLumi() != lumi) {
if (!itr.skipLumiInRun()) {
itr.advanceToNextRun();
}
}
else {
if (event == invalidEvent) {
return itr;
}
else {
EventNumber_t eventNumber = getEventNumberOfEntry(itr.peekAheadAtEventEntry());
if (eventNumber == event) {
return itr;
}
else {
if (!itr.skipToNextEventInLumi()) {
if (!itr.skipLumiInRun()) {
itr.advanceToNextRun();
}
}
}
}
}
}
}
return itrEnd;
}
| IndexIntoFile::IndexIntoFileItr edm::IndexIntoFile::findPosition | ( | RunNumber_t | run, |
| LuminosityBlockNumber_t | lumi = 0U, |
||
| EventNumber_t | event = 0U |
||
| ) | const |
Find a run, lumi, or event. Returns an iterator pointing at it. The iterator will always be in numericalOrder mode. If it is not found the entry type of the iterator will be kEnd. If it is found the entry type of the iterator will always be kRun so the next thing to be processed is the run containing the desired lumi or event or if looking for a run, the run itself. If the lumi and event arguments are 0 (invalid), then it will find a run. If only the event argument is 0 (invalid), then it will find a lumi. If will look for an event if all three arguments are nonzero or if only the lumi argument is 0 (invalid). Note that it will find the first match only so if there is more than one match then the others cannot be found with this method. The order of the search is by processHistoryID index, then run number, then lumi number, then event entry. If searching for a lumi the iterator will advance directly to the desired lumi after the run even if it is not the first lumi in the run. If searching for an event, the iterator will advance to the lumi containing the run and then the requested event after run even if there are other lumis earlier in that run and other events earlier in that lumi.
Definition at line 439 of file IndexIntoFile.cc.
References begin(), empty(), event(), eventEntries(), eventNumbers(), fillEventNumbers(), fillRunOrLumiIndexes(), edm::IndexIntoFile::IndexIntoFileItr::initializeLumi(), edm::IndexIntoFile::IndexIntoFileItr::initializeRun(), invalidEntry, invalidEvent, invalidIndex, invalidLumi, kEnd, kRun, fjr2json::lumi, numericalOrder, DTTTrigCorrFirst::run, and runOrLumiIndexes().
Referenced by findEventPosition(), findLumiPosition(), findPosition(), and findRunPosition().
{
fillRunOrLumiIndexes();
bool lumiMissing = (lumi == 0 && event != 0);
std::vector<RunOrLumiIndexes>::const_iterator it;
std::vector<RunOrLumiIndexes>::const_iterator iEnd = runOrLumiIndexes().end();
std::vector<RunOrLumiIndexes>::const_iterator phEnd;
// Loop over ranges of entries with the same ProcessHistoryID
for (std::vector<RunOrLumiIndexes>::const_iterator phBegin = runOrLumiIndexes().begin();
phBegin != iEnd;
phBegin = phEnd) {
RunOrLumiIndexes el(phBegin->processHistoryIDIndex(), run, lumi, 0);
phEnd = std::upper_bound(phBegin, iEnd, el, Compare_Index());
std::vector<RunOrLumiIndexes>::const_iterator iRun = std::lower_bound(phBegin, phEnd, el, Compare_Index_Run());
if (iRun == phEnd || iRun->run() != run) continue;
if (lumi == invalidLumi && event == invalidEvent) {
IndexIntoFileItr indexItr(this,
numericalOrder,
kRun,
iRun - runOrLumiIndexes().begin(),
invalidIndex,
invalidIndex,
0,
0);
indexItr.initializeRun();
return indexItr;
}
std::vector<RunOrLumiIndexes>::const_iterator iRunEnd = std::upper_bound(iRun, phEnd, el, Compare_Index_Run());
if (!lumiMissing) {
std::vector<RunOrLumiIndexes>::const_iterator iLumi = std::lower_bound(iRun, iRunEnd, el);
if (iLumi == iRunEnd || iLumi->lumi() != lumi) continue;
if (event == invalidEvent) {
IndexIntoFileItr indexItr(this,
numericalOrder,
kRun,
iRun - runOrLumiIndexes().begin(),
iLumi - runOrLumiIndexes().begin(),
invalidIndex,
0,
0);
indexItr.initializeLumi();
return indexItr;
}
long long beginEventNumbers = iLumi->beginEventNumbers();
long long endEventNumbers = iLumi->endEventNumbers();
if (beginEventNumbers >= endEventNumbers) continue;
long long indexToEvent = 0;
if (!eventEntries().empty()) {
std::vector<EventEntry>::const_iterator eventIter = std::lower_bound(eventEntries().begin() + beginEventNumbers,
eventEntries().begin() + endEventNumbers,
EventEntry(event, invalidEntry));
if (eventIter == (eventEntries().begin() + endEventNumbers) ||
eventIter->event() != event) continue;
indexToEvent = eventIter - eventEntries().begin() - beginEventNumbers;
} else {
fillEventNumbers();
std::vector<EventNumber_t>::const_iterator eventIter = std::lower_bound(eventNumbers().begin() + beginEventNumbers,
eventNumbers().begin() + endEventNumbers,
event);
if (eventIter == (eventNumbers().begin() + endEventNumbers) ||
*eventIter != event) continue;
indexToEvent = eventIter - eventNumbers().begin() - beginEventNumbers;
}
return IndexIntoFileItr(this,
numericalOrder,
kRun,
iRun - runOrLumiIndexes().begin(),
iLumi - runOrLumiIndexes().begin(),
iLumi - runOrLumiIndexes().begin(),
indexToEvent,
endEventNumbers - beginEventNumbers);
}
if (lumiMissing) {
std::vector<RunOrLumiIndexes>::const_iterator iLumi = iRun;
while (iLumi != iRunEnd && iLumi->lumi() == invalidLumi) {
++iLumi;
}
if (iLumi == iRunEnd) continue;
std::vector<RunOrLumiIndexes>::const_iterator lumiEnd;
for ( ;
iLumi != iRunEnd;
iLumi = lumiEnd) {
RunOrLumiIndexes elWithLumi(phBegin->processHistoryIDIndex(), run, iLumi->lumi(), 0);
lumiEnd = std::upper_bound(iLumi, iRunEnd, elWithLumi);
long long beginEventNumbers = iLumi->beginEventNumbers();
long long endEventNumbers = iLumi->endEventNumbers();
if (beginEventNumbers >= endEventNumbers) continue;
long long indexToEvent = 0;
if (!eventEntries().empty()) {
std::vector<EventEntry>::const_iterator eventIter = std::lower_bound(eventEntries().begin() + beginEventNumbers,
eventEntries().begin() + endEventNumbers,
EventEntry(event, invalidEntry));
if (eventIter == (eventEntries().begin() + endEventNumbers) ||
eventIter->event() != event) continue;
indexToEvent = eventIter - eventEntries().begin() - beginEventNumbers;
} else {
fillEventNumbers();
std::vector<EventNumber_t>::const_iterator eventIter = std::lower_bound(eventNumbers().begin() + beginEventNumbers,
eventNumbers().begin() + endEventNumbers,
event);
if (eventIter == (eventNumbers().begin() + endEventNumbers) ||
*eventIter != event) continue;
indexToEvent = eventIter - eventNumbers().begin() - beginEventNumbers;
}
return IndexIntoFileItr(this,
numericalOrder,
kRun,
iRun - runOrLumiIndexes().begin(),
iLumi - runOrLumiIndexes().begin(),
iLumi - runOrLumiIndexes().begin(),
indexToEvent,
endEventNumbers - beginEventNumbers);
}
}
} // Loop over ProcessHistoryIDs
return IndexIntoFileItr(this,
numericalOrder,
kEnd,
invalidIndex,
invalidIndex,
invalidIndex,
0,
0);
}
| IndexIntoFile::IndexIntoFileItr edm::IndexIntoFile::findRunPosition | ( | RunNumber_t | run | ) | const |
Same as findPosition.
Definition at line 646 of file IndexIntoFile.cc.
References findPosition().
Referenced by containsRun(), and fwlite::EntryFinder::findRun().
{
return findPosition(run, 0U, 0U);
}
| void edm::IndexIntoFile::fixIndexes | ( | std::vector< ProcessHistoryID > & | processHistoryIDs | ) |
Used by PoolSource to force the ProcessHistoryID indexes to be consistent across all input files. Currently this consistency is important when duplicate checking across all input files. It may be important for other reasons in the future. It is important this be called immediately after reading in the object from the input file, before filling the transient data members or using the indexes in any way.
Definition at line 293 of file IndexIntoFile.cc.
References spr::find(), processHistoryIDs(), processHistoryIDs_, and runOrLumiEntries_.
{
std::map<int, int> oldToNewIndex;
for (std::vector<ProcessHistoryID>::const_iterator iter = processHistoryIDs_.begin(),
iEnd = processHistoryIDs_.end();
iter != iEnd;
++iter) {
std::vector<ProcessHistoryID>::const_iterator iterExisting =
std::find(processHistoryIDs.begin(), processHistoryIDs.end(), *iter);
if (iterExisting == processHistoryIDs.end()) {
oldToNewIndex[iter - processHistoryIDs_.begin()] = processHistoryIDs.size();
processHistoryIDs.push_back(*iter);
}
else {
oldToNewIndex[iter - processHistoryIDs_.begin()] = iterExisting - processHistoryIDs.begin();
}
}
processHistoryIDs_ = processHistoryIDs;
for (std::vector<RunOrLumiEntry>::iterator iter = runOrLumiEntries_.begin(),
iEnd = runOrLumiEntries_.end();
iter != iEnd;
++iter) {
iter->setProcessHistoryIDIndex(oldToNewIndex[iter->processHistoryIDIndex()]);
}
}
| EventNumber_t edm::IndexIntoFile::getEventNumberOfEntry | ( | EntryNumber_t | entry | ) | const [inline, private] |
Definition at line 1016 of file IndexIntoFile.h.
References edm::Transient< T >::get(), and transients_.
Referenced by fillUnsortedEventNumbers(), and findPosition().
{
return transients_.get().eventFinder_->getEventNumberOfEntry(entry);
}
| void edm::IndexIntoFile::inputFileClosed | ( | ) | const |
Clear some vectors and eventFinder when an input file is closed. This reduces the memory used by IndexIntoFile
Definition at line 280 of file IndexIntoFile.cc.
References eventEntries(), resetEventFinder(), runOrLumiIndexes(), edm::swap(), and unsortedEventNumbers().
{
std::vector<EventEntry>().swap(eventEntries());
std::vector<RunOrLumiIndexes>().swap(runOrLumiIndexes());
std::vector<EventNumber_t>().swap(unsortedEventNumbers());
resetEventFinder();
}
| bool edm::IndexIntoFile::iterationWillBeInEntryOrder | ( | SortOrder | sortOrder | ) | const |
Used to determine whether or not to disable fast cloning.
Definition at line 421 of file IndexIntoFile.cc.
References begin(), end(), invalidEntry, and kEvent.
Referenced by edm::postIndexIntoFilePrintEventLists().
{
EntryNumber_t maxEntry = invalidEntry;
for(IndexIntoFileItr it = begin(sortOrder), itEnd = end(sortOrder); it != itEnd; ++it) {
if(it.getEntryType() == kEvent) {
if(it.entry() < maxEntry) {
return false;
}
maxEntry = it.entry();
}
}
return true;
}
| std::map<IndexRunLumiKey, EntryNumber_t>& edm::IndexIntoFile::lumiToFirstEntry | ( | ) | const [inline, private] |
Definition at line 1008 of file IndexIntoFile.h.
References edm::Transient< T >::get(), and transients_.
Referenced by addEntry().
{return transients_.get().lumiToFirstEntry_;}
| size_t edm::IndexIntoFile::numberOfEvents | ( | ) | const [inline, private] |
Definition at line 1015 of file IndexIntoFile.h.
References edm::Transient< T >::get(), and transients_.
Referenced by addEntry(), fillEventNumbersOrEntries(), and fillUnsortedEventNumbers().
{return transients_.get().numberOfEvents_;}
| int& edm::IndexIntoFile::previousAddedIndex | ( | ) | const [inline, private] |
Definition at line 1006 of file IndexIntoFile.h.
References edm::Transient< T >::get(), and transients_.
Referenced by addEntry().
{return transients_.get().previousAddedIndex_;}
| ProcessHistoryID const & edm::IndexIntoFile::processHistoryID | ( | int | i | ) | const |
Definition at line 25 of file IndexIntoFile.cc.
References processHistoryIDs_.
{
return processHistoryIDs_.at(i);
}
| std::vector< ProcessHistoryID > const & edm::IndexIntoFile::processHistoryIDs | ( | ) | const |
Definition at line 29 of file IndexIntoFile.cc.
References processHistoryIDs_.
Referenced by fixIndexes().
{
return processHistoryIDs_;
}
| void edm::IndexIntoFile::resetEventFinder | ( | ) | const [inline, private] |
Definition at line 1001 of file IndexIntoFile.h.
References edm::Transient< T >::get(), and transients_.
Referenced by inputFileClosed().
{transients_.get().eventFinder_.reset();}
| std::vector<RunOrLumiEntry> const& edm::IndexIntoFile::runOrLumiEntries | ( | ) | const [inline] |
Used internally and for test purposes.
Definition at line 992 of file IndexIntoFile.h.
References runOrLumiEntries_.
Referenced by empty(), endRunOrLumi(), edm::IndexIntoFile::IndexIntoFileItrNoSort::getRunOrLumiEntryType(), edm::IndexIntoFile::IndexIntoFileItrNoSort::initializeLumi_(), edm::IndexIntoFile::IndexIntoFileItrNoSort::nextEventRange(), edm::IndexIntoFile::IndexIntoFileItrNoSort::previousEventRange(), edm::IndexIntoFile::IndexIntoFileItrNoSort::setToLastEventInRange(), edm::IndexIntoFile::IndexIntoFileItrNoSort::skipLumiInRun(), and edm::IndexIntoFile::SortedRunOrLumiItr::SortedRunOrLumiItr().
{return runOrLumiEntries_;}
| std::vector<RunOrLumiIndexes>& edm::IndexIntoFile::runOrLumiIndexes | ( | ) | const [inline, private] |
Definition at line 1014 of file IndexIntoFile.h.
References edm::Transient< T >::get(), and transients_.
Referenced by fillRunOrLumiIndexes(), findPosition(), edm::IndexIntoFile::IndexIntoFileItrSorted::getRunOrLumiEntryType(), edm::IndexIntoFile::IndexIntoFileItrSorted::initializeLumi_(), inputFileClosed(), set_intersection(), edm::IndexIntoFile::IndexIntoFileItrSorted::setToLastEventInRange(), edm::IndexIntoFile::IndexIntoFileItrSorted::skipLumiInRun(), sortEventEntries(), and sortEvents().
{return transients_.get().runOrLumiIndexes_;}
| std::map<IndexRunKey, EntryNumber_t>& edm::IndexIntoFile::runToFirstEntry | ( | ) | const [inline, private] |
Definition at line 1007 of file IndexIntoFile.h.
References edm::Transient< T >::get(), and transients_.
Referenced by addEntry(), and sortVector_Run_Or_Lumi_Entries().
{return transients_.get().runToFirstEntry_;}
| void edm::IndexIntoFile::set_intersection | ( | IndexIntoFile const & | indexIntoFile, |
| std::set< IndexRunLumiEventKey > & | intersection | ||
| ) | const |
The intersection argument will be filled with an entry for each event in both IndexIntoFile objects. To be added the event must have the same ProcessHistoryID index, run number, lumi number and event number.
Definition at line 678 of file IndexIntoFile.cc.
References begin(), edm::IndexIntoFile::RunOrLumiIndexes::beginEventNumbers(), beginRunOrLumi(), empty(), edm::IndexIntoFile::RunOrLumiIndexes::endEventNumbers(), endRunOrLumi(), eventEntries(), eventNumbers(), fillEventNumbers(), fillRunOrLumiIndexes(), prof2calltree::front, iEvent, edm::IndexIntoFile::SortedRunOrLumiItr::isRun(), edm::IndexIntoFile::RunOrLumiIndexes::isRun(), edm::IndexIntoFile::RunOrLumiIndexes::lumi(), edm::IndexIntoFile::RunOrLumiIndexes::processHistoryIDIndex(), edm::IndexIntoFile::RunOrLumiIndexes::run(), edm::IndexIntoFile::SortedRunOrLumiItr::runOrLumiIndexes(), and runOrLumiIndexes().
Referenced by edm::DuplicateChecker::inputFileOpened().
{
if (empty() || indexIntoFile.empty()) return;
fillRunOrLumiIndexes();
indexIntoFile.fillRunOrLumiIndexes();
RunOrLumiIndexes const& back1 = runOrLumiIndexes().back();
RunOrLumiIndexes const& back2 = indexIntoFile.runOrLumiIndexes().back();
// Very quick decision if the run ranges in the two files do not overlap
if (back2 < runOrLumiIndexes().front()) return;
if (back1 < indexIntoFile.runOrLumiIndexes().front()) return;
SortedRunOrLumiItr iter1 = beginRunOrLumi();
SortedRunOrLumiItr iEnd1 = endRunOrLumi();
SortedRunOrLumiItr iter2 = indexIntoFile.beginRunOrLumi();
SortedRunOrLumiItr iEnd2 = indexIntoFile.endRunOrLumi();
// Quick decision if the lumi ranges in the two files do not overlap
while (iter1 != iEnd1 && iter1.isRun()) ++iter1;
if (iter1 == iEnd1) return;
if (back2 < iter1.runOrLumiIndexes()) return;
while (iter2 != iEnd2 && iter2.isRun()) ++iter2;
if (iter2 == iEnd2) return;
if (back1 < iter2.runOrLumiIndexes()) return;
RunOrLumiIndexes const* previousIndexes = 0;
// Loop through the both IndexIntoFile objects and look for matching lumis
while (iter1 != iEnd1 && iter2 != iEnd2) {
RunOrLumiIndexes const& indexes1 = iter1.runOrLumiIndexes();
RunOrLumiIndexes const& indexes2 = iter2.runOrLumiIndexes();
if (indexes1 < indexes2) {
++iter1;
}
else if (indexes2 < indexes1) {
++iter2;
}
else { // they are equal
// Skip them if it is a run or the same lumi
if (indexes1.isRun() ||
(previousIndexes && !(*previousIndexes < indexes1))) {
++iter1;
++iter2;
}
else {
previousIndexes = &indexes1;
// Found a matching lumi, now look for matching events
long long beginEventNumbers1 = indexes1.beginEventNumbers();
long long endEventNumbers1 = indexes1.endEventNumbers();
long long beginEventNumbers2 = indexes2.beginEventNumbers();
long long endEventNumbers2 = indexes2.endEventNumbers();
// there must be at least 1 event in each lumi for there to be any matches
if ((beginEventNumbers1 >= endEventNumbers1) ||
(beginEventNumbers2 >= endEventNumbers2)) {
++iter1;
++iter2;
continue;
}
if (!eventEntries().empty() && !indexIntoFile.eventEntries().empty()) {
std::vector<EventEntry> matchingEvents;
std::insert_iterator<std::vector<EventEntry> > insertIter(matchingEvents, matchingEvents.begin());
std::set_intersection(eventEntries().begin() + beginEventNumbers1,
eventEntries().begin() + endEventNumbers1,
indexIntoFile.eventEntries().begin() + beginEventNumbers2,
indexIntoFile.eventEntries().begin() + endEventNumbers2,
insertIter);
for (std::vector<EventEntry>::const_iterator iEvent = matchingEvents.begin(),
iEnd = matchingEvents.end();
iEvent != iEnd; ++iEvent) {
intersection.insert(IndexRunLumiEventKey(indexes1.processHistoryIDIndex(),
indexes1.run(),
indexes1.lumi(),
iEvent->event()));
}
} else {
fillEventNumbers();
indexIntoFile.fillEventNumbers();
std::vector<EventNumber_t> matchingEvents;
std::insert_iterator<std::vector<EventNumber_t> > insertIter(matchingEvents, matchingEvents.begin());
std::set_intersection(eventNumbers().begin() + beginEventNumbers1,
eventNumbers().begin() + endEventNumbers1,
indexIntoFile.eventNumbers().begin() + beginEventNumbers2,
indexIntoFile.eventNumbers().begin() + endEventNumbers2,
insertIter);
for (std::vector<EventNumber_t>::const_iterator iEvent = matchingEvents.begin(),
iEnd = matchingEvents.end();
iEvent != iEnd; ++iEvent) {
intersection.insert(IndexRunLumiEventKey(indexes1.processHistoryIDIndex(),
indexes1.run(),
indexes1.lumi(),
*iEvent));
}
}
}
}
}
}
| void edm::IndexIntoFile::setEventFinder | ( | boost::shared_ptr< EventFinder > | ptr | ) | const [inline] |
Calling this enables the functions that fill the event vectors to get the event numbers. It needs to be called before filling the events vectors This implies the client needs to define a class that inherits from EventFinder and then create one. This function is used to pass in a pointer to its base class.
Definition at line 927 of file IndexIntoFile.h.
References edm::Transient< T >::get(), and transients_.
Referenced by fwlite::EntryFinder::fillIndex().
{transients_.get().eventFinder_ = ptr;}
| void edm::IndexIntoFile::setNumberOfEvents | ( | EntryNumber_t | nevents | ) | const [inline] |
The number of events needs to be set before filling the transient event vectors. It is used to resize them.
Definition at line 918 of file IndexIntoFile.h.
References edm::Transient< T >::get(), nevents, and transients_.
Referenced by addEntry(), and fwlite::EntryFinder::fillIndex().
{
transients_.get().numberOfEvents_ = nevents;
}
| std::vector<ProcessHistoryID>& edm::IndexIntoFile::setProcessHistoryIDs | ( | ) | [inline] |
Used for backward compatibility and tests. RootFile::fillIndexIntoFile uses this to deal with input files created with releases before 3_8_0 which do not contain an IndexIntoFile.
Definition at line 986 of file IndexIntoFile.h.
References processHistoryIDs_.
{return processHistoryIDs_;}
| std::vector<RunOrLumiEntry>& edm::IndexIntoFile::setRunOrLumiEntries | ( | ) | [inline] |
Used for backward compatibility and tests. RootFile::fillIndexIntoFile uses this to deal with input files created with releases before 3_8_0 which do not contain an IndexIntoFile.
Definition at line 981 of file IndexIntoFile.h.
References runOrLumiEntries_.
{return runOrLumiEntries_;}
| void edm::IndexIntoFile::sortEventEntries | ( | ) | const [private] |
Definition at line 368 of file IndexIntoFile.cc.
References begin(), eventEntries(), fillRunOrLumiIndexes(), runOrLumiIndexes(), and python::multivaluedict::sort().
Referenced by fillEventNumbersOrEntries().
{
fillRunOrLumiIndexes();
std::vector<RunOrLumiIndexes>::iterator beginOfLumi = runOrLumiIndexes().begin();
std::vector<RunOrLumiIndexes>::iterator endOfLumi = beginOfLumi;
std::vector<RunOrLumiIndexes>::iterator iEnd = runOrLumiIndexes().end();
while (true) {
while (beginOfLumi != iEnd && beginOfLumi->isRun()) {
++beginOfLumi;
}
if (beginOfLumi == iEnd) break;
endOfLumi = beginOfLumi + 1;
while (endOfLumi != iEnd &&
beginOfLumi->processHistoryIDIndex() == endOfLumi->processHistoryIDIndex() &&
beginOfLumi->run() == endOfLumi->run() &&
beginOfLumi->lumi() == endOfLumi->lumi()) {
++endOfLumi;
}
assert(beginOfLumi->endEventNumbers() >= 0);
assert(beginOfLumi->endEventNumbers() <= static_cast<long long>(eventEntries().size()));
std::sort(eventEntries().begin() + beginOfLumi->beginEventNumbers(),
eventEntries().begin() + beginOfLumi->endEventNumbers());
beginOfLumi = endOfLumi;
}
}
| void edm::IndexIntoFile::sortEvents | ( | ) | const [private] |
Definition at line 342 of file IndexIntoFile.cc.
References begin(), eventNumbers(), fillRunOrLumiIndexes(), runOrLumiIndexes(), and python::multivaluedict::sort().
Referenced by fillEventNumbersOrEntries().
{
fillRunOrLumiIndexes();
std::vector<RunOrLumiIndexes>::iterator beginOfLumi = runOrLumiIndexes().begin();
std::vector<RunOrLumiIndexes>::iterator endOfLumi = beginOfLumi;
std::vector<RunOrLumiIndexes>::iterator iEnd = runOrLumiIndexes().end();
while (true) {
while (beginOfLumi != iEnd && beginOfLumi->isRun()) {
++beginOfLumi;
}
if (beginOfLumi == iEnd) break;
endOfLumi = beginOfLumi + 1;
while (endOfLumi != iEnd &&
beginOfLumi->processHistoryIDIndex() == endOfLumi->processHistoryIDIndex() &&
beginOfLumi->run() == endOfLumi->run() &&
beginOfLumi->lumi() == endOfLumi->lumi()) {
++endOfLumi;
}
assert(beginOfLumi->endEventNumbers() >= 0);
assert(beginOfLumi->endEventNumbers() <= static_cast<long long>(eventNumbers().size()));
std::sort(eventNumbers().begin() + beginOfLumi->beginEventNumbers(),
eventNumbers().begin() + beginOfLumi->endEventNumbers());
beginOfLumi = endOfLumi;
}
}
| void edm::IndexIntoFile::sortVector_Run_Or_Lumi_Entries | ( | ) |
Used by RootOutputModule after all entries have been added. This only works after the correct sequence of addEntry calls, because it makes some corrections before sorting. A std::stable_sort works in cases where those corrections are not needed.
Definition at line 320 of file IndexIntoFile.cc.
References Exception, edm::errors::LogicError, runOrLumiEntries_, runToFirstEntry(), and edm::stable_sort_all().
Referenced by edm::RootOutputFile::writeIndexIntoFile().
{
for (std::vector<RunOrLumiEntry>::iterator iter = runOrLumiEntries_.begin(),
iEnd = runOrLumiEntries_.end();
iter != iEnd;
++iter) {
std::map<IndexRunKey, EntryNumber_t>::const_iterator firstRunEntry =
runToFirstEntry().find(IndexRunKey(iter->processHistoryIDIndex(), iter->run()));
if (firstRunEntry == runToFirstEntry().end()) {
throw Exception(errors::LogicError)
<< "In IndexIntoFile::sortVector_Run_Or_Lumi_Entries. A run entry is missing.\n"
<< "This means the IndexIntoFile product in the output file will be corrupted.\n"
<< "The output file will be unusable for most purposes.\n"
<< "If this occurs after an unrelated exception was thrown in\n"
<< "endLuminosityBlock or endRun then ignore this exception and fix\n"
<< "the primary exception. This is an expected side effect.\n"
<< "Otherwise please report this to the core framework developers\n";
}
iter->setOrderPHIDRun(firstRunEntry->second);
}
stable_sort_all(runOrLumiEntries_);
}
| std::vector<EventNumber_t>& edm::IndexIntoFile::unsortedEventNumbers | ( | ) | const [inline] |
If something external to IndexIntoFile is reading through the EventAuxiliary then it could use this to fill in the event numbers so that IndexIntoFile will not read through it again.
Definition at line 969 of file IndexIntoFile.h.
References edm::Transient< T >::get(), and transients_.
Referenced by doneFileInitialization(), fillEventNumbersOrEntries(), fillUnsortedEventNumbers(), and inputFileClosed().
{return transients_.get().unsortedEventNumbers_;}
IndexIntoFile::EntryNumber_t const edm::IndexIntoFile::invalidEntry = -1LL [static] |
Definition at line 191 of file IndexIntoFile.h.
Referenced by addEntry(), edm::IndexIntoFile::IndexIntoFileItrNoSort::entry(), edm::IndexIntoFile::IndexIntoFileItrSorted::entry(), fillRunOrLumiIndexes(), findPosition(), edm::IndexIntoFile::IndexIntoFileItrNoSort::initializeLumi_(), iterationWillBeInEntryOrder(), edm::IndexIntoFile::IndexIntoFileItrNoSort::nextEventRange(), edm::IndexIntoFile::IndexIntoFileItrSorted::peekAheadAtEventEntry(), edm::IndexIntoFile::IndexIntoFileItrNoSort::peekAheadAtEventEntry(), edm::IndexIntoFile::IndexIntoFileItrNoSort::previousEventRange(), edm::IndexIntoFile::IndexIntoFileItrNoSort::setToLastEventInRange(), edm::IndexIntoFile::IndexIntoFileItrImpl::skipEventBackward(), and edm::IndexIntoFile::IndexIntoFileItrImpl::skipEventForward().
EventNumber_t const edm::IndexIntoFile::invalidEvent = 0U [static] |
Definition at line 190 of file IndexIntoFile.h.
Referenced by addEntry(), fillEventNumbersOrEntries(), findEventPosition(), and findPosition().
int const edm::IndexIntoFile::invalidIndex = -1 [static] |
Definition at line 187 of file IndexIntoFile.h.
Referenced by addEntry(), edm::IndexIntoFile::IndexIntoFileItrImpl::advanceToNextLumiOrRun(), begin(), end(), findPosition(), edm::IndexIntoFile::IndexIntoFileItrNoSort::initializeLumi_(), edm::IndexIntoFile::IndexIntoFileItrSorted::initializeLumi_(), edm::IndexIntoFile::IndexIntoFileItrImpl::initializeRun(), edm::IndexIntoFile::IndexIntoFileItrNoSort::nextEventRange(), edm::IndexIntoFile::IndexIntoFileItrSorted::peekAheadAtEventEntry(), edm::IndexIntoFile::IndexIntoFileItrNoSort::peekAheadAtEventEntry(), edm::IndexIntoFile::IndexIntoFileItrNoSort::peekAheadAtLumi(), edm::IndexIntoFile::IndexIntoFileItrSorted::peekAheadAtLumi(), edm::IndexIntoFile::IndexIntoFileItrNoSort::previousEventRange(), edm::IndexIntoFile::IndexIntoFileItrImpl::previousLumiWithEvents(), edm::IndexIntoFile::IndexIntoFileItrNoSort::processHistoryIDIndex(), edm::IndexIntoFile::IndexIntoFileItrSorted::processHistoryIDIndex(), edm::IndexIntoFile::IndexIntoFileItrImpl::setInvalid(), edm::IndexIntoFile::IndexIntoFileItrImpl::skipEventBackward(), edm::IndexIntoFile::IndexIntoFileItrImpl::skipEventForward(), edm::IndexIntoFile::IndexIntoFileItrNoSort::skipLumiInRun(), and edm::IndexIntoFile::IndexIntoFileItrSorted::skipLumiInRun().
LuminosityBlockNumber_t const edm::IndexIntoFile::invalidLumi = 0U [static] |
Definition at line 189 of file IndexIntoFile.h.
Referenced by addEntry(), findLumiPosition(), findPosition(), edm::IndexIntoFile::SortedRunOrLumiItr::isRun(), edm::IndexIntoFile::RunOrLumiIndexes::isRun(), edm::IndexIntoFile::RunOrLumiEntry::isRun(), edm::IndexIntoFile::IndexIntoFileItrSorted::lumi(), edm::IndexIntoFile::IndexIntoFileItrNoSort::lumi(), edm::IndexIntoFile::IndexIntoFileItrNoSort::peekAheadAtLumi(), edm::IndexIntoFile::IndexIntoFileItrSorted::peekAheadAtLumi(), edm::IndexIntoFile::IndexIntoFileItrImpl::skipEventBackward(), and edm::IndexIntoFile::IndexIntoFileItrImpl::skipEventForward().
RunNumber_t const edm::IndexIntoFile::invalidRun = 0U [static] |
Definition at line 188 of file IndexIntoFile.h.
Referenced by addEntry(), edm::IndexIntoFile::IndexIntoFileItrSorted::run(), edm::IndexIntoFile::IndexIntoFileItrNoSort::run(), edm::IndexIntoFile::IndexIntoFileItrImpl::skipEventBackward(), and edm::IndexIntoFile::IndexIntoFileItrImpl::skipEventForward().
std::vector<ProcessHistoryID> edm::IndexIntoFile::processHistoryIDs_ [private] |
Definition at line 1022 of file IndexIntoFile.h.
Referenced by addEntry(), fixIndexes(), processHistoryID(), processHistoryIDs(), and setProcessHistoryIDs().
std::vector<RunOrLumiEntry> edm::IndexIntoFile::runOrLumiEntries_ [private] |
Definition at line 1023 of file IndexIntoFile.h.
Referenced by addEntry(), fillRunOrLumiIndexes(), fixIndexes(), runOrLumiEntries(), setRunOrLumiEntries(), and sortVector_Run_Or_Lumi_Entries().
Transient<Transients> edm::IndexIntoFile::transients_ [mutable, private] |
Definition at line 1020 of file IndexIntoFile.h.
Referenced by beginEvents(), currentIndex(), currentLumi(), currentRun(), endEvents(), eventEntries(), eventNumbers(), getEventNumberOfEntry(), lumiToFirstEntry(), numberOfEvents(), previousAddedIndex(), resetEventFinder(), runOrLumiIndexes(), runToFirstEntry(), setEventFinder(), setNumberOfEvents(), and unsortedEventNumbers().
1.7.3