CMS 3D CMS Logo

 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Properties Friends Macros Pages
List of all members | Public Member Functions | Static Public Member Functions | Private Types | Private Attributes
edm::DuplicateChecker Class Reference

#include <DuplicateChecker.h>

Public Member Functions

bool checkDisabled () const
 
bool checkingAllFiles () const
 
void disable ()
 
 DuplicateChecker (ParameterSet const &pset)
 
void inputFileClosed ()
 
void inputFileOpened (bool realData, IndexIntoFile const &indexIntoFile, std::vector< boost::shared_ptr< IndexIntoFile > > const &indexesIntoFiles, std::vector< boost::shared_ptr< IndexIntoFile > >::size_type currentIndexIntoFile)
 
bool isDuplicateAndCheckActive (int index, RunNumber_t run, LuminosityBlockNumber_t lumi, EventNumber_t event, std::string const &fileName)
 
bool noDuplicatesInFile () const
 

Static Public Member Functions

static void fillDescription (ParameterSetDescription &desc)
 

Private Types

enum  DataType { isRealData, isSimulation, unknown }
 
enum  DuplicateCheckMode { noDuplicateCheck, checkEachFile, checkEachRealDataFile, checkAllFilesOpened }
 

Private Attributes

DataType dataType_
 
bool disabled_
 
DuplicateCheckMode duplicateCheckMode_
 
bool itIsKnownTheFileHasNoDuplicates_
 
std::set
< IndexIntoFile::IndexRunLumiEventKey
relevantPreviousEvents_
 

Detailed Description

Definition at line 32 of file DuplicateChecker.h.

Member Enumeration Documentation

Constructor & Destructor Documentation

edm::DuplicateChecker::DuplicateChecker ( ParameterSet const &  pset)

Definition at line 13 of file DuplicateChecker.cc.

References checkAllFilesOpened, checkEachFile, checkEachRealDataFile, CastorDigiReco::duplicateCheckMode, duplicateCheckMode_, edm::hlt::Exception, edm::ParameterSet::getUntrackedParameter(), and noDuplicateCheck.

13  :
16  disabled_(false)
17  {
18  // The default value provided as the second argument to the getUntrackedParameter function call
19  // is not used when the ParameterSet has been validated and the parameters are not optional
20  // in the description. This is currently true when PoolSource is the primary input source.
21  // The modules that use PoolSource as a SecSource have not defined their fillDescriptions function
22  // yet, so the ParameterSet does not get validated yet. As soon as all the modules with a SecSource
23  // have defined descriptions, the defaults in the getUntrackedParameterSet function calls can
24  // and should be deleted from the code.
25  std::string duplicateCheckMode =
26  pset.getUntrackedParameter<std::string>("duplicateCheckMode", std::string("checkAllFilesOpened"));
27 
28  if (duplicateCheckMode == std::string("noDuplicateCheck")) duplicateCheckMode_ = noDuplicateCheck;
29  else if (duplicateCheckMode == std::string("checkEachFile")) duplicateCheckMode_ = checkEachFile;
30  else if (duplicateCheckMode == std::string("checkEachRealDataFile")) duplicateCheckMode_ = checkEachRealDataFile;
31  else if (duplicateCheckMode == std::string("checkAllFilesOpened")) duplicateCheckMode_ = checkAllFilesOpened;
32  else {
33  throw cms::Exception("Configuration")
34  << "Illegal configuration parameter value passed to PoolSource for\n"
35  << "the \"duplicateCheckMode\" parameter, legal values are:\n"
36  << "\"noDuplicateCheck\", \"checkEachFile\", \"checkEachRealDataFile\", \"checkAllFilesOpened\"\n";
37  }
38  }
tuple duplicateCheckMode
DuplicateCheckMode duplicateCheckMode_

Member Function Documentation

bool edm::DuplicateChecker::checkDisabled ( ) const
inline
bool edm::DuplicateChecker::checkingAllFiles ( ) const
inline
void edm::DuplicateChecker::disable ( )

Definition at line 40 of file DuplicateChecker.cc.

References dataType_, disabled_, itIsKnownTheFileHasNoDuplicates_, relevantPreviousEvents_, and unknown.

40  {
41  disabled_ = true;
45  }
std::set< IndexIntoFile::IndexRunLumiEventKey > relevantPreviousEvents_
void edm::DuplicateChecker::fillDescription ( ParameterSetDescription desc)
static

Definition at line 119 of file DuplicateChecker.cc.

References edm::ParameterSetDescription::addUntracked().

Referenced by edm::RootInputFileSequence::fillDescription().

119  {
120  std::string defaultString("checkAllFilesOpened");
121  desc.addUntracked<std::string>("duplicateCheckMode", defaultString)->setComment(
122  "'checkAllFilesOpened': check across all input files\n"
123  "'checkEachFile': check each input file independently\n"
124  "'checkEachRealDataFile': check each real data input file independently\n"
125  "'noDuplicateCheck': no duplicate checking\n"
126  );
127  }
void edm::DuplicateChecker::inputFileClosed ( )

Definition at line 77 of file DuplicateChecker.cc.

References dataType_, itIsKnownTheFileHasNoDuplicates_, relevantPreviousEvents_, and unknown.

78  {
82  }
std::set< IndexIntoFile::IndexRunLumiEventKey > relevantPreviousEvents_
void edm::DuplicateChecker::inputFileOpened ( bool  realData,
IndexIntoFile const &  indexIntoFile,
std::vector< boost::shared_ptr< IndexIntoFile > > const &  indexesIntoFiles,
std::vector< boost::shared_ptr< IndexIntoFile > >::size_type  currentIndexIntoFile 
)

Definition at line 47 of file DuplicateChecker.cc.

References checkAllFilesOpened, checkDisabled(), edm::IndexIntoFile::containsDuplicateEvents(), dataType_, duplicateCheckMode_, i, isRealData, isSimulation, itIsKnownTheFileHasNoDuplicates_, relevantPreviousEvents_, and edm::IndexIntoFile::set_intersection().

51  {
52 
53  dataType_ = realData ? isRealData : isSimulation;
54  if (checkDisabled()) return;
55 
58 
60 
61  // Compares the current IndexIntoFile to all the previous ones and saves any duplicates.
62  // One unintended thing, it also saves the duplicate runs and lumis.
63  for(std::vector<boost::shared_ptr<IndexIntoFile> >::size_type i = 0; i < currentIndexIntoFile; ++i) {
64  if (indexesIntoFiles[i].get() != 0) {
65 
66  indexIntoFile.set_intersection(*indexesIntoFiles[i], relevantPreviousEvents_);
67  }
68  }
69  }
70  if (relevantPreviousEvents_.empty()) {
71  if(!indexIntoFile.containsDuplicateEvents()) {
73  }
74  }
75  }
int i
Definition: DBlmapReader.cc:9
bool checkDisabled() const
uint16_t size_type
std::set< IndexIntoFile::IndexRunLumiEventKey > relevantPreviousEvents_
DuplicateCheckMode duplicateCheckMode_
bool edm::DuplicateChecker::isDuplicateAndCheckActive ( int  index,
RunNumber_t  run,
LuminosityBlockNumber_t  lumi,
EventNumber_t  event,
std::string const &  fileName 
)

Definition at line 84 of file DuplicateChecker.cc.

References checkAllFilesOpened, checkDisabled(), duplicateCheckMode_, itIsKnownTheFileHasNoDuplicates_, and relevantPreviousEvents_.

88  {
89  if (itIsKnownTheFileHasNoDuplicates_) return false;
90  if (checkDisabled()) return false;
91 
92  IndexIntoFile::IndexRunLumiEventKey newEvent(index, run, lumi, event);
93  bool duplicate = !relevantPreviousEvents_.insert(newEvent).second;
94 
95  if (duplicate) {
97  LogWarning("DuplicateEvent")
98  << "Duplicate Events found in entire set of input files.\n"
99  << "Both events were from run " << run
100  << " and luminosity block " << lumi
101  << " with event number " << event << ".\n"
102  << "The duplicate was from file " << fileName << ".\n"
103  << "The duplicate will be skipped.\n";
104  }
105  else {
106  LogWarning("DuplicateEvent")
107  << "Duplicate Events found in file " << fileName << ".\n"
108  << "Both events were from run " << run
109  << " and luminosity block " << lumi
110  << " with event number " << event << ".\n"
111  << "The duplicate will be skipped.\n";
112  }
113  return true;
114  }
115  return false;
116  }
bool checkDisabled() const
tuple lumi
Definition: fjr2json.py:35
How EventSelector::AcceptEvent() decides whether to accept an event for output otherwise it is excluding the probing of A single or multiple positive and the trigger will pass if any such matching triggers are PASS or EXCEPTION[A criterion thatmatches no triggers at all is detected and causes a throw.] A single negative with an expectation of appropriate bit checking in the decision and the trigger will pass if any such matching triggers are FAIL or EXCEPTION A wildcarded negative criterion that matches more than one trigger in the trigger but the state exists so we define the behavior If all triggers are the negative crieriion will lead to accepting the event(this again matches the behavior of"!*"before the partial wildcard feature was incorporated).The per-event"cost"of each negative criterion with multiple relevant triggers is about the same as!*was in the past
std::set< IndexIntoFile::IndexRunLumiEventKey > relevantPreviousEvents_
DuplicateCheckMode duplicateCheckMode_
bool edm::DuplicateChecker::noDuplicatesInFile ( ) const
inline

Definition at line 47 of file DuplicateChecker.h.

References itIsKnownTheFileHasNoDuplicates_.

Member Data Documentation

DataType edm::DuplicateChecker::dataType_
private

Definition at line 75 of file DuplicateChecker.h.

Referenced by checkDisabled(), disable(), inputFileClosed(), and inputFileOpened().

bool edm::DuplicateChecker::disabled_
private

Definition at line 86 of file DuplicateChecker.h.

Referenced by checkDisabled(), and disable().

DuplicateCheckMode edm::DuplicateChecker::duplicateCheckMode_
private
bool edm::DuplicateChecker::itIsKnownTheFileHasNoDuplicates_
private
std::set<IndexIntoFile::IndexRunLumiEventKey> edm::DuplicateChecker::relevantPreviousEvents_
private