CMS 3D CMS Logo

List of all members | Public Member Functions | Static Public Member Functions | Public Attributes | Private Member Functions
confdb.HLTProcess Class Reference
Inheritance diagram for confdb.HLTProcess:

Public Member Functions

def __init__ (self, configuration)
 
def addEras (self)
 
def addGlobalOptions (self)
 
def append_filenames (self, name, filenames)
 
def build_source (self)
 
def buildOptions (self)
 
def buildPathList (self)
 
def customize (self)
 
def dump (self)
 
def expand_filenames (self, input)
 
def fixPrescales (self)
 
def getPathList (self)
 
def getRawConfigurationFromDB (self)
 
def getSetupConfigurationFromDB (self)
 
def instrumentDQM (self)
 
def instrumentErrorEventType (self)
 
def instrumentOpenMode (self)
 
def instrumentTiming (self)
 
def loadAdditionalConditions (self, comment, conditions)
 
def loadCff (self, module)
 
def loadCffCommand (self, module)
 
def loadSetupCff (self)
 
def overrideGlobalTag (self)
 
def overrideL1MenuXml (self)
 
def overrideOutput (self)
 
def overrideParameters (self, module, parameters)
 
def overrideProcessName (self)
 
def removeElementFromSequencesTasksAndPaths (self, label)
 
def runL1Emulator (self)
 
def specificCustomize (self)
 
def updateMessageLogger (self)
 

Static Public Member Functions

def consolidateNegativeList (elements)
 
def consolidatePositiveList (elements)
 
def dumppaths (paths)
 
def expandWildcards (globs, collection)
 

Public Attributes

 all_paths
 
 config
 
 converter
 
 data
 
 labels
 
 options
 
 parent
 
 source
 

Private Member Functions

def _fix_parameter (self, args)
 

Detailed Description

Definition at line 21 of file confdb.py.

Constructor & Destructor Documentation

◆ __init__()

def confdb.HLTProcess.__init__ (   self,
  configuration 
)

Definition at line 23 of file confdb.py.

23  def __init__(self, configuration):
24  self.config = configuration
25  self.data = None
26  self.source = []
27  self.parent = []
28 
29  self.options = {
30  'essources' : [],
31  'esmodules' : [],
32  'modules' : [],
33  'sequences' : [],
34  'services' : [],
35  'paths' : [],
36  'psets' : [],
37  'blocks' : [],
38  }
39 
40  self.labels = {}
41  if self.config.fragment:
42  self.labels['process'] = 'fragment'
43  self.labels['dict'] = 'fragment.__dict__'
44  else:
45  self.labels['process'] = 'process'
46  self.labels['dict'] = 'process.__dict__'
47 
48  if self.config.prescale and (self.config.prescale.lower() != 'none'):
49  self.labels['prescale'] = self.config.prescale
50 
51  # get the configuration from ConfdB
52  from .confdbOfflineConverter import OfflineConverter
53  self.converter = OfflineConverter(version = self.config.menu.version, database = self.config.menu.database, proxy = self.config.proxy, proxyHost = self.config.proxy_host, proxyPort = self.config.proxy_port)
54  self.buildPathList()
55  self.buildOptions()
56  self.getSetupConfigurationFromDB()
57  self.getRawConfigurationFromDB()
58  self.customize()
59 
def __init__(self, dataset, job_number, job_id, job_name, isDA, isMC, applyBOWS, applyEXTRACOND, extraconditions, runboundary, lumilist, intlumi, maxevents, gt, allFromGT, alignmentDB, alignmentTAG, apeDB, apeTAG, bowDB, bowTAG, vertextype, tracktype, refittertype, ttrhtype, applyruncontrol, ptcut, CMSSW_dir, the_dir)

Member Function Documentation

◆ _fix_parameter()

def confdb.HLTProcess._fix_parameter (   self,
  args 
)
private
arguments:
name:     parameter name (optional)
type:     parameter type (look for tracked and untracked variants)
value:    original value
replace:  replacement value

Definition at line 341 of file confdb.py.

References mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ecal::raw::InputDataCPU.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, KDTreeNodeInfo< DATA, DIM >.data, calo::multifit::MapSymM< T, Stride, Order >.data, hcal::raw::InputDataCPU.data, LASModuleProfile.data, dqmTnP::AbstractFitter.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, calo::multifit::MapMForPM< T >.data, GeometryComparisonPlotter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, edm::MapOfVectors< K, T >::Iter.data, hcal::raw::InputDataGPU.data, data, EcalShowerContainmentCorrections::Coefficients.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, and edmStreamStallGrapher.Stack.data.

Referenced by confdb.HLTProcess.customize(), and confdb.HLTProcess.instrumentErrorEventType().

341  def _fix_parameter(self, **args):
342  """arguments:
343  name: parameter name (optional)
344  type: parameter type (look for tracked and untracked variants)
345  value: original value
346  replace: replacement value
347  """
348  if 'name' in args:
349  self.data = re.sub(
350  r'%(name)s = cms(?P<tracked>(?:\.untracked)?)\.%(type)s\( (?P<quote>["\']?)%(value)s(?P=quote)' % args,
351  r'%(name)s = cms\g<tracked>.%(type)s( \g<quote>%(replace)s\g<quote>' % args,
352  self.data)
353  else:
354  self.data = re.sub(
355  r'cms(?P<tracked>(?:\.untracked)?)\.%(type)s\( (?P<quote>["\']?)%(value)s(?P=quote)' % args,
356  r'cms\g<tracked>.%(type)s( \g<quote>%(replace)s\g<quote>' % args,
357  self.data)
358 
359 

◆ addEras()

def confdb.HLTProcess.addEras (   self)

Definition at line 534 of file confdb.py.

References geometry.Alignables.config, confdb.HLTProcess.config, preexistingValidation.PreexistingValidation.config, plottingOptions.BasePlottingOptions.config, pkg.AbstractPkg.config, cscdqm::EventProcessorMutex.config, genericValidation.GenericValidation.config, CSCOfflineClient.config, validateAlignments.ValidationBase.config, CSCMonitorModule.config, cscdqm::Collection.config, cscdqm::Dispatcher.config, cscdqm::EventProcessor.config, validateAlignments.ValidationJobMultiIOV.config, mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, ecal::raw::InputDataCPU.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, calo::multifit::MapSymM< T, Stride, Order >.data, KDTreeNodeInfo< DATA, DIM >.data, hcal::raw::InputDataCPU.data, dqmTnP::AbstractFitter.data, LASModuleProfile.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, GeometryComparisonPlotter.data, calo::multifit::MapMForPM< T >.data, edm::MapOfVectors< K, T >::Iter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, hcal::raw::InputDataGPU.data, data, EcalShowerContainmentCorrections::Coefficients.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, edmStreamStallGrapher.Stack.data, and join().

Referenced by confdb.HLTProcess.customize().

534  def addEras(self):
535  if self.config.eras is None:
536  return
537  from Configuration.StandardSequences.Eras import eras
538  erasSplit = self.config.eras.split(',')
539  self.data = re.sub(r'process = cms.Process\( *"\w+"', '\n'.join(eras.pythonCfgLines[era] for era in erasSplit)+'\n\g<0>, '+', '.join(era for era in erasSplit), self.data)
540 
static std::string join(char **cmd)
Definition: RemoteFile.cc:19

◆ addGlobalOptions()

def confdb.HLTProcess.addGlobalOptions (   self)

Definition at line 325 of file confdb.py.

References geometry.Alignables.config, confdb.HLTProcess.config, preexistingValidation.PreexistingValidation.config, plottingOptions.BasePlottingOptions.config, pkg.AbstractPkg.config, cscdqm::EventProcessorMutex.config, genericValidation.GenericValidation.config, CSCOfflineClient.config, validateAlignments.ValidationBase.config, CSCMonitorModule.config, cscdqm::Collection.config, cscdqm::Dispatcher.config, cscdqm::EventProcessor.config, validateAlignments.ValidationJobMultiIOV.config, mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ecal::raw::InputDataCPU.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, KDTreeNodeInfo< DATA, DIM >.data, calo::multifit::MapSymM< T, Stride, Order >.data, hcal::raw::InputDataCPU.data, LASModuleProfile.data, dqmTnP::AbstractFitter.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, calo::multifit::MapMForPM< T >.data, GeometryComparisonPlotter.data, edm::MapOfVectors< K, T >::Iter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, hcal::raw::InputDataGPU.data, data, EcalShowerContainmentCorrections::Coefficients.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, and edmStreamStallGrapher.Stack.data.

Referenced by confdb.HLTProcess.customize().

325  def addGlobalOptions(self):
326  # add global options
327  self.data += """
328 # limit the number of events to be processed
329 %%(process)s.maxEvents = cms.untracked.PSet(
330  input = cms.untracked.int32( %d )
331 )
332 """ % self.config.events
333 
334  self.data += """
335 # enable TrigReport, TimeReport and MultiThreading
336 %(process)s.options.wantSummary = True
337 %(process)s.options.numberOfThreads = 4
338 %(process)s.options.numberOfStreams = 0
339 """
340 

◆ append_filenames()

def confdb.HLTProcess.append_filenames (   self,
  name,
  filenames 
)

Definition at line 892 of file confdb.py.

References mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ecal::raw::InputDataCPU.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, KDTreeNodeInfo< DATA, DIM >.data, calo::multifit::MapSymM< T, Stride, Order >.data, hcal::raw::InputDataCPU.data, LASModuleProfile.data, dqmTnP::AbstractFitter.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, calo::multifit::MapMForPM< T >.data, GeometryComparisonPlotter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, edm::MapOfVectors< K, T >::Iter.data, hcal::raw::InputDataGPU.data, data, EcalShowerContainmentCorrections::Coefficients.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, and edmStreamStallGrapher.Stack.data.

Referenced by confdb.HLTProcess.build_source().

892  def append_filenames(self, name, filenames):
893  if len(filenames) > 255:
894  token_open = "( *("
895  token_close = ") )"
896  else:
897  token_open = "("
898  token_close = ")"
899 
900  self.data += " %s = cms.untracked.vstring%s\n" % (name, token_open)
901  for line in filenames:
902  self.data += " '%s',\n" % line
903  self.data += " %s,\n" % (token_close)
904 
905 

◆ build_source()

def confdb.HLTProcess.build_source (   self)

Definition at line 918 of file confdb.py.

References confdb.HLTProcess.append_filenames(), geometry.Alignables.config, confdb.HLTProcess.config, preexistingValidation.PreexistingValidation.config, plottingOptions.BasePlottingOptions.config, pkg.AbstractPkg.config, cscdqm::EventProcessorMutex.config, genericValidation.GenericValidation.config, CSCOfflineClient.config, validateAlignments.ValidationBase.config, CSCMonitorModule.config, cscdqm::Collection.config, cscdqm::Dispatcher.config, cscdqm::EventProcessor.config, validateAlignments.ValidationJobMultiIOV.config, mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ecal::raw::InputDataCPU.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, calo::multifit::MapSymM< T, Stride, Order >.data, KDTreeNodeInfo< DATA, DIM >.data, hcal::raw::InputDataCPU.data, dqmTnP::AbstractFitter.data, LASModuleProfile.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, GeometryComparisonPlotter.data, calo::multifit::MapMForPM< T >.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, edm::MapOfVectors< K, T >::Iter.data, hcal::raw::InputDataGPU.data, data, EcalShowerContainmentCorrections::Coefficients.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, edmStreamStallGrapher.Stack.data, confdb.HLTProcess.expand_filenames(), confdb.HLTProcess.parent, FWPSetTableManager::PSetData.parent, SlimmedCount.parent, BeautifulSoup.PageElement.parent, dqm::implementation::IBooker::UseScope< SCOPE >.parent, confdb.HLTProcess.source, lhef::XMLDocument.source, DQMNet::Peer.source, Config.Process.source, and BeautifulSoup.ResultSet.source.

Referenced by confdb.HLTProcess.customize().

918  def build_source(self):
919  if self.config.hilton:
920  # use the DAQ source
921  return
922 
923  if self.config.input:
924  # if a dataset or a list of input files was given, use it
925  self.source = self.expand_filenames(self.config.input)
926  elif self.config.data:
927  # offline we can run on data...
928  self.source = [ "file:RelVal_Raw_%s_DATA.root" % self.config.type ]
929  else:
930  # ...or on mc
931  self.source = [ "file:RelVal_Raw_%s_MC.root" % self.config.type ]
932 
933  if self.config.parent:
934  # if a dataset or a list of input files was given for the parent data, use it
935  self.parent = self.expand_filenames(self.config.parent)
936 
937  self.data += """
938 # source module (EDM inputs)
939 %(process)s.source = cms.Source( "PoolSource",
940 """
941  self.append_filenames("fileNames", self.source)
942  if (self.parent):
943  self.append_filenames("secondaryFileNames", self.parent)
944  self.data += """\
945  inputCommands = cms.untracked.vstring(
946  'keep *'
947  )
948 )
949 """
950 

◆ buildOptions()

def confdb.HLTProcess.buildOptions (   self)

Definition at line 793 of file confdb.py.

References mps_setup.append, geometry.Alignables.config, confdb.HLTProcess.config, preexistingValidation.PreexistingValidation.config, plottingOptions.BasePlottingOptions.config, pkg.AbstractPkg.config, cscdqm::EventProcessorMutex.config, genericValidation.GenericValidation.config, CSCOfflineClient.config, validateAlignments.ValidationBase.config, CSCMonitorModule.config, cscdqm::Collection.config, cscdqm::Dispatcher.config, cscdqm::EventProcessor.config, validateAlignments.ValidationJobMultiIOV.config, DTCalibrationWorker.DTCalibrationWorker.options, DTWorkflow.DTWorkflow.options, TestProcess.TestProcess.options, confdb.HLTProcess.options, Config.Process.options, and validateAlignments.ValidationJobMultiIOV.options.

793  def buildOptions(self):
794  # common configuration for all scenarios
795  self.options['services'].append( "-DQM" )
796  self.options['services'].append( "-FUShmDQMOutputService" )
797  self.options['services'].append( "-MicroStateService" )
798  self.options['services'].append( "-ModuleWebRegistry" )
799  self.options['services'].append( "-TimeProfilerService" )
800 
801  # remove the DAQ modules and the online definition of the DQMStore and DQMFileSaver
802  # unless a hilton-like configuration has been requested
803  if not self.config.hilton:
804  self.options['services'].append( "-EvFDaqDirector" )
805  self.options['services'].append( "-FastMonitoringService" )
806  self.options['services'].append( "-DQMStore" )
807  self.options['modules'].append( "-hltDQMFileSaver" )
808  self.options['modules'].append( "-hltDQMFileSaverPB" )
809 
810  if self.config.fragment:
811  # extract a configuration file fragment
812  self.options['essources'].append( "-GlobalTag" )
813  self.options['essources'].append( "-HepPDTESSource" )
814  self.options['essources'].append( "-XMLIdealGeometryESSource" )
815  self.options['essources'].append( "-eegeom" )
816  self.options['essources'].append( "-es_hardcode" )
817  self.options['essources'].append( "-magfield" )
818 
819  self.options['esmodules'].append( "-SlaveField0" )
820  self.options['esmodules'].append( "-SlaveField20" )
821  self.options['esmodules'].append( "-SlaveField30" )
822  self.options['esmodules'].append( "-SlaveField35" )
823  self.options['esmodules'].append( "-SlaveField38" )
824  self.options['esmodules'].append( "-SlaveField40" )
825  self.options['esmodules'].append( "-VBF0" )
826  self.options['esmodules'].append( "-VBF20" )
827  self.options['esmodules'].append( "-VBF30" )
828  self.options['esmodules'].append( "-VBF35" )
829  self.options['esmodules'].append( "-VBF38" )
830  self.options['esmodules'].append( "-VBF40" )
831  self.options['esmodules'].append( "-CSCGeometryESModule" )
832  self.options['esmodules'].append( "-CaloGeometryBuilder" )
833  self.options['esmodules'].append( "-CaloTowerHardcodeGeometryEP" )
834  self.options['esmodules'].append( "-CastorHardcodeGeometryEP" )
835  self.options['esmodules'].append( "-DTGeometryESModule" )
836  self.options['esmodules'].append( "-EcalBarrelGeometryEP" )
837  self.options['esmodules'].append( "-EcalElectronicsMappingBuilder" )
838  self.options['esmodules'].append( "-EcalEndcapGeometryEP" )
839  self.options['esmodules'].append( "-EcalLaserCorrectionService" )
840  self.options['esmodules'].append( "-EcalPreshowerGeometryEP" )
841  self.options['esmodules'].append( "-GEMGeometryESModule" )
842  self.options['esmodules'].append( "-HcalHardcodeGeometryEP" )
843  self.options['esmodules'].append( "-HcalTopologyIdealEP" )
844  self.options['esmodules'].append( "-MuonNumberingInitialization" )
845  self.options['esmodules'].append( "-ParametrizedMagneticFieldProducer" )
846  self.options['esmodules'].append( "-RPCGeometryESModule" )
847  self.options['esmodules'].append( "-SiStripGainESProducer" )
848  self.options['esmodules'].append( "-SiStripRecHitMatcherESProducer" )
849  self.options['esmodules'].append( "-SiStripQualityESProducer" )
850  self.options['esmodules'].append( "-StripCPEfromTrackAngleESProducer" )
851  self.options['esmodules'].append( "-TrackerAdditionalParametersPerDetESModule" )
852  self.options['esmodules'].append( "-TrackerDigiGeometryESModule" )
853  self.options['esmodules'].append( "-TrackerGeometricDetESModule" )
854  self.options['esmodules'].append( "-VolumeBasedMagneticFieldESProducer" )
855  self.options['esmodules'].append( "-ZdcHardcodeGeometryEP" )
856  self.options['esmodules'].append( "-hcal_db_producer" )
857  self.options['esmodules'].append( "-L1GtTriggerMaskAlgoTrigTrivialProducer" )
858  self.options['esmodules'].append( "-L1GtTriggerMaskTechTrigTrivialProducer" )
859  self.options['esmodules'].append( "-hltESPEcalTrigTowerConstituentsMapBuilder" )
860  self.options['esmodules'].append( "-hltESPGlobalTrackingGeometryESProducer" )
861  self.options['esmodules'].append( "-hltESPMuonDetLayerGeometryESProducer" )
862  self.options['esmodules'].append( "-hltESPTrackerRecoGeometryESProducer" )
863  self.options['esmodules'].append( "-trackerTopology" )
864 
865  self.options['esmodules'].append( "-CaloTowerGeometryFromDBEP" )
866  self.options['esmodules'].append( "-CastorGeometryFromDBEP" )
867  self.options['esmodules'].append( "-EcalBarrelGeometryFromDBEP" )
868  self.options['esmodules'].append( "-EcalEndcapGeometryFromDBEP" )
869  self.options['esmodules'].append( "-EcalPreshowerGeometryFromDBEP" )
870  self.options['esmodules'].append( "-HcalGeometryFromDBEP" )
871  self.options['esmodules'].append( "-ZdcGeometryFromDBEP" )
872  self.options['esmodules'].append( "-XMLFromDBSource" )
873  self.options['esmodules'].append( "-sistripconn" )
874 
875  self.options['services'].append( "-MessageLogger" )
876 
877  self.options['psets'].append( "-maxEvents" )
878  self.options['psets'].append( "-options" )
879 
880  # remove Scouting OutputModules even though the EndPaths are kept
881  self.options['modules'].append( "-hltOutputScoutingCaloMuon" )
882  self.options['modules'].append( "-hltOutputScoutingPF" )
883 
884  if self.config.fragment or (self.config.prescale and (self.config.prescale.lower() == 'none')):
885  self.options['services'].append( "-PrescaleService" )
886 
887  if self.config.fragment or self.config.timing:
888  self.options['services'].append( "-FastTimerService" )
889  self.options['services'].append( "-ThroughputService" )
890 
891 

◆ buildPathList()

def confdb.HLTProcess.buildPathList (   self)

Definition at line 728 of file confdb.py.

728  def buildPathList(self):
729  self.all_paths = self.getPathList()
730 
731  if self.config.paths:
732  # no path list was requested, dump the full table, minus unsupported / unwanted paths
733  paths = self.config.paths.split(',')
734  else:
735  # dump only the requested paths, plus the eventual output endpaths
736  paths = []
737 
738  # 'none' should remove all outputs
739  # 'dqm' should remove all outputs but DQMHistograms
740  # 'minimal' should remove all outputs but DQMHistograms, and add a single output module to keep the TriggerResults and TriggerEvent
741  # 'full' should remove all outputs but DQMHistograms, and add a single output module to "keep *"
742  # See also the `overrideOutput` method
743  if self.config.fragment or self.config.output in ('none', ):
744  if self.config.paths:
745  # keep only the Paths and EndPaths requested explicitly
746  pass
747  else:
748  # drop all output EndPaths but the Scouting ones, and drop the RatesMonitoring and DQMHistograms
749  paths.append( "-*Output" )
750  paths.append( "-RatesMonitoring")
751  paths.append( "-DQMHistograms")
752  if self.config.fragment: paths.append( "Scouting*Output" )
753 
754  elif self.config.output in ('dqm', 'minimal', 'full'):
755  if self.config.paths:
756  # keep only the Paths and EndPaths requested explicitly, and the DQMHistograms
757  paths.append( "DQMHistograms" )
758  else:
759  # drop all output EndPaths but the Scouting ones, and drop the RatesMonitoring
760  paths.append( "-*Output" )
761  paths.append( "-RatesMonitoring")
762  if self.config.fragment: paths.append( "Scouting*Output" )
763 
764  else:
765  if self.config.paths:
766  # keep all output EndPaths, including the DQMHistograms
767  paths.append( "*Output" )
768  paths.append( "DQMHistograms" )
769  else:
770  # keep all Paths and EndPaths
771  pass
772 
773  # drop unwanted paths for profiling (and timing studies)
774  if self.config.profiling:
775  paths.append( "-HLTAnalyzerEndpath" )
776 
777  # this should never be in any dump (nor online menu)
778  paths.append( "-OfflineOutput" )
779 
780  # expand all wildcards
781  paths = self.expandWildcards(paths, self.all_paths)
782 
783  if self.config.paths:
784  # do an "additive" consolidation
785  paths = self.consolidatePositiveList(paths)
786  if not paths:
787  raise RuntimeError('Error: option "--paths %s" does not select any valid paths' % self.config.paths)
788  else:
789  # do a "subtractive" consolidation
790  paths = self.consolidateNegativeList(paths)
791  self.options['paths'] = paths
792 

◆ consolidateNegativeList()

def confdb.HLTProcess.consolidateNegativeList (   elements)
static

Definition at line 142 of file confdb.py.

142  def consolidateNegativeList(elements):
143  # consolidate a list of path exclusions and re-inclusions
144  # the result is the list of paths to be removed from the dump
145  result = set()
146  for element in elements:
147  if element[0] == '-':
148  result.add( element )
149  else:
150  result.discard( '-' + element )
151  return sorted( element for element in result )
152 

◆ consolidatePositiveList()

def confdb.HLTProcess.consolidatePositiveList (   elements)
static

Definition at line 154 of file confdb.py.

154  def consolidatePositiveList(elements):
155  # consolidate a list of path selection and re-exclusions
156  # the result is the list of paths to be included in the dump
157  result = set()
158  for element in elements:
159  if element[0] == '-':
160  result.discard( element[1:] )
161  else:
162  result.add( element )
163  return sorted( element for element in result )
164 
165 

◆ customize()

def confdb.HLTProcess.customize (   self)

Definition at line 241 of file confdb.py.

References confdb.HLTProcess._fix_parameter(), confdb.HLTProcess.addEras(), confdb.HLTProcess.addGlobalOptions(), confdb.HLTProcess.all_paths, confdb.HLTProcess.build_source(), geometry.Alignables.config, confdb.HLTProcess.config, preexistingValidation.PreexistingValidation.config, plottingOptions.BasePlottingOptions.config, pkg.AbstractPkg.config, cscdqm::EventProcessorMutex.config, genericValidation.GenericValidation.config, CSCOfflineClient.config, validateAlignments.ValidationBase.config, CSCMonitorModule.config, cscdqm::Collection.config, cscdqm::Dispatcher.config, cscdqm::EventProcessor.config, validateAlignments.ValidationJobMultiIOV.config, mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ecal::raw::InputDataCPU.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, calo::multifit::MapSymM< T, Stride, Order >.data, KDTreeNodeInfo< DATA, DIM >.data, hcal::raw::InputDataCPU.data, dqmTnP::AbstractFitter.data, LASModuleProfile.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, calo::multifit::MapMForPM< T >.data, GeometryComparisonPlotter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, edm::MapOfVectors< K, T >::Iter.data, hcal::raw::InputDataGPU.data, EcalShowerContainmentCorrections::Coefficients.data, data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, edmStreamStallGrapher.Stack.data, confdb.HLTProcess.fixPrescales(), confdb.HLTProcess.instrumentDQM(), confdb.HLTProcess.instrumentErrorEventType(), confdb.HLTProcess.instrumentOpenMode(), confdb.HLTProcess.instrumentTiming(), confdb.HLTProcess.loadSetupCff(), confdb.HLTProcess.overrideGlobalTag(), confdb.HLTProcess.overrideL1MenuXml(), confdb.HLTProcess.overrideOutput(), confdb.HLTProcess.overrideProcessName(), python.rootplot.root2matplotlib.replace(), confdb.HLTProcess.runL1Emulator(), confdb.HLTProcess.specificCustomize(), and confdb.HLTProcess.updateMessageLogger().

241  def customize(self):
242 
243  # adapt the source to the current scenario
244  if not self.config.fragment:
245  self.build_source()
246 
247  # manual override some parameters
248  if self.config.type in ('HIon', ):
249  if self.config.data:
250  if not self.config.fragment:
251  self._fix_parameter( type = 'InputTag', value = 'rawDataCollector', replace = 'rawDataRepacker')
252 
253  # if requested, remove the HLT prescales
254  self.fixPrescales()
255 
256  # if requested, override all ED/HLTfilters to always pass ("open" mode)
257  self.instrumentOpenMode()
258 
259  # if requested, change all HLTTriggerTypeFilter EDFilters to accept only error events (SelectedTriggerType = 0)
260  self.instrumentErrorEventType()
261 
262  # if requested, instrument the self with the modules and EndPath needed for timing studies
263  self.instrumentTiming()
264 
265  # if requested, override the L1 self from the GlobalTag (Xml)
266  self.overrideL1MenuXml()
267 
268  # if requested, run the L1 emulator
269  self.runL1Emulator()
270 
271  # add process.load("setup_cff")
272  self.loadSetupCff()
273 
274  if self.config.fragment:
275  self.data += """
276 # dummify hltGetConditions in cff's
277 if 'hltGetConditions' in %(dict)s and 'HLTriggerFirstPath' in %(dict)s :
278  %(process)s.hltDummyConditions = cms.EDFilter( "HLTBool",
279  result = cms.bool( True )
280  )
281  %(process)s.HLTriggerFirstPath.replace(%(process)s.hltGetConditions,%(process)s.hltDummyConditions)
282 """
283 
284  # the scouting path issue:
285  # 1) for config fragments, we remove all output modules
286  # 2) however in old style datasets, the scouting output paths also run the unpackers which are needed
287  # 3) therefore they have to keep the scouting path but remove the scouting output module
288  # 4) in new style datasets, aka datasetpaths & finalpaths, the scouting unpackers are on another path and all of this is unnecessary
289  # 5) however its hard to detect whether we have new style or old style so we run this for both
290  # 6) therefore we end up with a superfluous Scouting*OutputPaths which are empty
291  for path in self.all_paths:
292  match = re.match(r'(Scouting\w+)Output$', path)
293  if match:
294  module = 'hltOutput' + match.group(1)
295  self.data = self.data.replace(path+' = cms.EndPath', path+' = cms.Path')
296  self.data = self.data.replace(' + process.'+module, '')
297  self.data = self.data.replace(' process.'+module, '')
298  else:
299 
300  # override the process name and adapt the relevant filters
301  self.overrideProcessName()
302 
303  # select specific Eras
304  self.addEras()
305 
306  # override the output modules to output root files
307  self.overrideOutput()
308 
309  # add global options
310  self.addGlobalOptions()
311 
312  # if requested or necessary, override the GlobalTag and connection strings (incl. L1!)
313  self.overrideGlobalTag()
314 
315  # request summary informations from the MessageLogger
316  self.updateMessageLogger()
317 
318  # replace DQMStore and DQMRootOutputModule with a configuration suitable for running offline
319  self.instrumentDQM()
320 
321  # add specific customisations
322  self.specificCustomize()
323 
324 
def replace(string, replacements)

◆ dump()

def confdb.HLTProcess.dump (   self)

Definition at line 167 of file confdb.py.

References geometry.Alignables.config, confdb.HLTProcess.config, preexistingValidation.PreexistingValidation.config, plottingOptions.BasePlottingOptions.config, pkg.AbstractPkg.config, cscdqm::EventProcessorMutex.config, genericValidation.GenericValidation.config, CSCOfflineClient.config, validateAlignments.ValidationBase.config, CSCMonitorModule.config, cscdqm::Collection.config, cscdqm::Dispatcher.config, cscdqm::EventProcessor.config, validateAlignments.ValidationJobMultiIOV.config, mat4.data, CombinedTauTagCalibration.data, CombinedSVCalibration.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ecal::raw::InputDataCPU.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, KDTreeNodeInfo< DATA, DIM >.data, calo::multifit::MapSymM< T, Stride, Order >.data, hcal::raw::InputDataCPU.data, LASModuleProfile.data, dqmTnP::AbstractFitter.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, calo::multifit::MapMForPM< T >.data, GeometryComparisonPlotter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, edm::MapOfVectors< K, T >::Iter.data, hcal::raw::InputDataGPU.data, data, EcalShowerContainmentCorrections::Coefficients.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, edmStreamStallGrapher.Stack.data, confdb.HLTProcess.labels, ecaldqm::binning::AxisSpecs.labels, and svgfig.Ticks.labels.

167  def dump(self):
168  self.data = self.data % self.labels
169  if self.config.fragment:
170  self.data = re.sub( r'\bprocess\b', 'fragment', self.data )
171  self.data = re.sub( r'\bProcess\b', 'ProcessFragment', self.data )
172  return self.data
173 
174 

◆ dumppaths()

def confdb.HLTProcess.dumppaths (   paths)
static

Definition at line 722 of file confdb.py.

722  def dumppaths(paths):
723  sys.stderr.write('Path selection:\n')
724  for path in paths:
725  sys.stderr.write('\t%s\n' % path)
726  sys.stderr.write('\n\n')
727 

◆ expand_filenames()

def confdb.HLTProcess.expand_filenames (   self,
  input 
)

Definition at line 906 of file confdb.py.

Referenced by confdb.HLTProcess.build_source().

906  def expand_filenames(self, input):
907  # check if the input is a dataset or a list of files
908  if input[0:8] == 'dataset:':
909  from .dasFileQuery import dasFileQuery
910  # extract the dataset name, and use DAS to fine the list of LFNs
911  dataset = input[8:]
912  files = dasFileQuery(dataset)
913  else:
914  # assume a comma-separated list of input files
915  files = input.split(',')
916  return files
917 

◆ expandWildcards()

def confdb.HLTProcess.expandWildcards (   globs,
  collection 
)
static

Definition at line 126 of file confdb.py.

References python.rootplot.root2matplotlib.replace().

126  def expandWildcards(globs, collection):
127  # expand a list of unix-style wildcards matching a given collection
128  # wildcards with no matches are silently discarded
129  matches = []
130  for glob in globs:
131  negate = ''
132  if glob[0] == '-':
133  negate = '-'
134  glob = glob[1:]
135  # translate a unix-style glob expression into a regular expression
136  filter = re.compile(r'^' + glob.replace('?', '.').replace('*', '.*').replace('[!', '[^') + r'$')
137  matches.extend( negate + element for element in collection if filter.match(element) )
138  return matches
139 
140 
def replace(string, replacements)

◆ fixPrescales()

def confdb.HLTProcess.fixPrescales (   self)

Definition at line 360 of file confdb.py.

References confdb.HLTProcess.all_paths, geometry.Alignables.config, confdb.HLTProcess.config, preexistingValidation.PreexistingValidation.config, plottingOptions.BasePlottingOptions.config, pkg.AbstractPkg.config, cscdqm::EventProcessorMutex.config, genericValidation.GenericValidation.config, CSCOfflineClient.config, validateAlignments.ValidationBase.config, CSCMonitorModule.config, cscdqm::Collection.config, cscdqm::Dispatcher.config, cscdqm::EventProcessor.config, validateAlignments.ValidationJobMultiIOV.config, mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, ecal::raw::InputDataCPU.data, ecal::DigisCollection< StoragePolicy >.data, TrackProbabilityCalibration.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, KDTreeNodeInfo< DATA, DIM >.data, calo::multifit::MapSymM< T, Stride, Order >.data, hcal::raw::InputDataCPU.data, LASModuleProfile.data, dqmTnP::AbstractFitter.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, GeometryComparisonPlotter.data, calo::multifit::MapMForPM< T >.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, edm::MapOfVectors< K, T >::Iter.data, hcal::raw::InputDataGPU.data, EcalShowerContainmentCorrections::Coefficients.data, data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, edmStreamStallGrapher.Stack.data, DTCalibrationWorker.DTCalibrationWorker.options, DTWorkflow.DTWorkflow.options, TestProcess.TestProcess.options, confdb.HLTProcess.options, Config.Process.options, and validateAlignments.ValidationJobMultiIOV.options.

Referenced by confdb.HLTProcess.customize().

360  def fixPrescales(self):
361  # update the PrescaleService to match the new list of paths
362  if self.options['paths']:
363  if self.options['paths'][0][0] == '-':
364  # drop requested paths
365  for minuspath in self.options['paths']:
366  path = minuspath[1:]
367  self.data = re.sub(r' cms.PSet\( pathName = cms.string\( "%s" \),\n prescales = cms.vuint32\( .* \)\n \),?\n' % path, '', self.data)
368  else:
369  # keep requested paths
370  for path in self.all_paths:
371  if path not in self.options['paths']:
372  self.data = re.sub(r' cms.PSet\( pathName = cms.string\( "%s" \),\n prescales = cms.vuint32\( .* \)\n \),?\n' % path, '', self.data)
373 
374  if self.config.prescale and (self.config.prescale.lower() != 'none'):
375  # TO DO: check that the requested prescale column is valid
376  self.data += """
377 # force the use of a specific HLT prescale column
378 if 'PrescaleService' in %(dict)s:
379  %(process)s.PrescaleService.forceDefault = True
380  %(process)s.PrescaleService.lvl1DefaultLabel = '%(prescale)s'
381 """
382 
383 

◆ getPathList()

def confdb.HLTProcess.getPathList (   self)

Definition at line 101 of file confdb.py.

References geometry.Alignables.config, confdb.HLTProcess.config, preexistingValidation.PreexistingValidation.config, plottingOptions.BasePlottingOptions.config, pkg.AbstractPkg.config, cscdqm::EventProcessorMutex.config, genericValidation.GenericValidation.config, CSCOfflineClient.config, validateAlignments.ValidationBase.config, CSCMonitorModule.config, cscdqm::Collection.config, cscdqm::Dispatcher.config, cscdqm::EventProcessor.config, validateAlignments.ValidationJobMultiIOV.config, and confdb.HLTProcess.converter.

101  def getPathList(self):
102  if self.config.menu.run:
103  args = ['--runNumber', self.config.menu.run]
104  else:
105  args = ['--configName', self.config.menu.name]
106  args.extend( (
107  '--cff',
108  '--noedsources',
109  '--noes',
110  '--noservices',
111  '--nosequences',
112  '--nomodules'
113  ) )
114 
115  data, err = self.converter.query( *args )
116  if 'ERROR' in err or 'Exhausted Resultset' in err or 'CONFIG_NOT_FOUND' in err:
117  sys.stderr.write("%s: error while retrieving the list of paths from the HLT menu\n\n" % os.path.basename(sys.argv[0]))
118  sys.stderr.write(err + "\n\n")
119  sys.exit(1)
120  filter = re.compile(r' *= *cms.(End|Final)?Path.*')
121  paths = [ filter.sub('', line) for line in data.splitlines() if filter.search(line) ]
122  return paths
123 
124 
Definition: query.py:1

◆ getRawConfigurationFromDB()

def confdb.HLTProcess.getRawConfigurationFromDB (   self)

Definition at line 83 of file confdb.py.

References geometry.Alignables.config, confdb.HLTProcess.config, preexistingValidation.PreexistingValidation.config, plottingOptions.BasePlottingOptions.config, pkg.AbstractPkg.config, cscdqm::EventProcessorMutex.config, genericValidation.GenericValidation.config, CSCOfflineClient.config, validateAlignments.ValidationBase.config, CSCMonitorModule.config, cscdqm::Collection.config, cscdqm::Dispatcher.config, cscdqm::EventProcessor.config, validateAlignments.ValidationJobMultiIOV.config, confdb.HLTProcess.converter, mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ecal::raw::InputDataCPU.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, calo::multifit::MapSymM< T, Stride, Order >.data, KDTreeNodeInfo< DATA, DIM >.data, hcal::raw::InputDataCPU.data, LASModuleProfile.data, dqmTnP::AbstractFitter.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, calo::multifit::MapMForPM< T >.data, GeometryComparisonPlotter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, edm::MapOfVectors< K, T >::Iter.data, hcal::raw::InputDataGPU.data, data, EcalShowerContainmentCorrections::Coefficients.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, edmStreamStallGrapher.Stack.data, mps_monitormerge.items, join(), DTCalibrationWorker.DTCalibrationWorker.options, DTWorkflow.DTWorkflow.options, TestProcess.TestProcess.options, confdb.HLTProcess.options, Config.Process.options, and validateAlignments.ValidationJobMultiIOV.options.

83  def getRawConfigurationFromDB(self):
84  if self.config.menu.run:
85  args = ['--runNumber', self.config.menu.run]
86  else:
87  args = ['--configName', self.config.menu.name ]
88  if not self.config.hilton:
89  # keep the original Source when running on Hilton
90  args.append('--noedsources')
91  for key, vals in self.options.items():
92  if vals:
93  args.extend(('--'+key, ','.join(vals)))
94  data, err = self.converter.query( *args )
95  if 'ERROR' in err or 'Exhausted Resultset' in err or 'CONFIG_NOT_FOUND' in err:
96  sys.stderr.write("%s: error while retrieving the HLT menu\n\n" % os.path.basename(sys.argv[0]))
97  sys.stderr.write(err + "\n\n")
98  sys.exit(1)
99  self.data = data
100 
Definition: query.py:1
static std::string join(char **cmd)
Definition: RemoteFile.cc:19

◆ getSetupConfigurationFromDB()

def confdb.HLTProcess.getSetupConfigurationFromDB (   self)

Definition at line 60 of file confdb.py.

References geometry.Alignables.config, confdb.HLTProcess.config, preexistingValidation.PreexistingValidation.config, plottingOptions.BasePlottingOptions.config, pkg.AbstractPkg.config, cscdqm::EventProcessorMutex.config, genericValidation.GenericValidation.config, CSCOfflineClient.config, validateAlignments.ValidationBase.config, CSCMonitorModule.config, cscdqm::Collection.config, cscdqm::Dispatcher.config, cscdqm::EventProcessor.config, validateAlignments.ValidationJobMultiIOV.config, confdb.HLTProcess.converter, mps_monitormerge.items, join(), DTCalibrationWorker.DTCalibrationWorker.options, DTWorkflow.DTWorkflow.options, TestProcess.TestProcess.options, confdb.HLTProcess.options, Config.Process.options, validateAlignments.ValidationJobMultiIOV.options, and python.rootplot.root2matplotlib.replace().

60  def getSetupConfigurationFromDB(self):
61  if not self.config.setup:
62  return
63 
64  if ".py" in self.config.setup:
65  self.config.setupFile = self.config.setup.split(".py")[0]
66  return
67  args = ['--configName', self.config.setup ]
68  args.append('--noedsources')
69  args.append('--nopaths')
70  for key, vals in self.options.items():
71  if vals:
72  args.extend(('--'+key, ','.join(vals)))
73  args.append('--cff')
74  data, err = self.converter.query( *args )
75  if 'ERROR' in err or 'Exhausted Resultset' in err or 'CONFIG_NOT_FOUND' in err:
76  sys.stderr.write("%s: error while retrieving the HLT setup menu\n\n" % os.path.basename(sys.argv[0]))
77  sys.stderr.write(err + "\n\n")
78  sys.exit(1)
79  self.config.setupFile = "setup_"+self.config.setup[1:].replace("/","_")+"_cff"
80  outfile = open(self.config.setupFile+".py","w+")
81  outfile.write("# This file is automatically generated by hltGetConfiguration.\n" + data)
82 
def replace(string, replacements)
Definition: query.py:1
static std::string join(char **cmd)
Definition: RemoteFile.cc:19

◆ instrumentDQM()

def confdb.HLTProcess.instrumentDQM (   self)

Definition at line 690 of file confdb.py.

References geometry.Alignables.config, confdb.HLTProcess.config, preexistingValidation.PreexistingValidation.config, plottingOptions.BasePlottingOptions.config, pkg.AbstractPkg.config, cscdqm::EventProcessorMutex.config, genericValidation.GenericValidation.config, CSCOfflineClient.config, validateAlignments.ValidationBase.config, CSCMonitorModule.config, cscdqm::Collection.config, cscdqm::Dispatcher.config, cscdqm::EventProcessor.config, validateAlignments.ValidationJobMultiIOV.config, mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, ecal::raw::InputDataCPU.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, calo::multifit::MapSymM< T, Stride, Order >.data, KDTreeNodeInfo< DATA, DIM >.data, hcal::raw::InputDataCPU.data, LASModuleProfile.data, dqmTnP::AbstractFitter.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, GeometryComparisonPlotter.data, calo::multifit::MapMForPM< T >.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, edm::MapOfVectors< K, T >::Iter.data, hcal::raw::InputDataGPU.data, data, EcalShowerContainmentCorrections::Coefficients.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, edmStreamStallGrapher.Stack.data, confdb.HLTProcess.loadCffCommand(), and confdb.HLTProcess.removeElementFromSequencesTasksAndPaths().

Referenced by confdb.HLTProcess.customize().

690  def instrumentDQM(self):
691  if not self.config.hilton:
692  # remove any reference to the hltDQMFileSaver and hltDQMFileSaverPB:
693  # note the convert options remove the module itself,
694  # here we are just removing the references in paths, sequences, etc
695  self.removeElementFromSequencesTasksAndPaths('hltDQMFileSaverPB')
696  self.removeElementFromSequencesTasksAndPaths('hltDQMFileSaver')
697 
698  # instrument the HLT menu with DQMStore and DQMRootOutputModule suitable for running offline
699  dqmstore = "\n# load the DQMStore and DQMRootOutputModule\n"
700  dqmstore += self.loadCffCommand('DQMServices.Core.DQMStore_cfi')
701  dqmstore += """
702 %(process)s.dqmOutput = cms.OutputModule("DQMRootOutputModule",
703  fileName = cms.untracked.string("DQMIO.root")
704 )
705 """
706  empty_path = re.compile(r'.*\b(process\.)?DQMOutput = cms\.(Final|End)Path\( *\).*')
707  other_path = re.compile(r'(.*\b(process\.)?DQMOutput = cms\.(Final|End)Path\()(.*)')
708  if empty_path.search(self.data):
709  # replace an empty DQMOutput path
710  self.data = empty_path.sub(dqmstore + '\n%(process)s.DQMOutput = cms.FinalPath( %(process)s.dqmOutput )\n', self.data)
711  elif other_path.search(self.data):
712  # prepend the dqmOutput to the DQMOutput path
713  self.data = other_path.sub(dqmstore + r'\g<1> %(process)s.dqmOutput +\g<4>', self.data)
714  else:
715  # create a new DQMOutput path with the dqmOutput module
716  self.data += dqmstore
717  self.data += '\n%(process)s.DQMOutput = cms.FinalPath( %(process)s.dqmOutput )\n'
718  self.data += '%(process)s.schedule.append( %(process)s.DQMOutput )\n'
719 
720 

◆ instrumentErrorEventType()

def confdb.HLTProcess.instrumentErrorEventType (   self)

Definition at line 398 of file confdb.py.

References confdb.HLTProcess._fix_parameter(), geometry.Alignables.config, confdb.HLTProcess.config, preexistingValidation.PreexistingValidation.config, plottingOptions.BasePlottingOptions.config, pkg.AbstractPkg.config, cscdqm::EventProcessorMutex.config, genericValidation.GenericValidation.config, CSCOfflineClient.config, validateAlignments.ValidationBase.config, CSCMonitorModule.config, cscdqm::Collection.config, cscdqm::Dispatcher.config, cscdqm::EventProcessor.config, and validateAlignments.ValidationJobMultiIOV.config.

Referenced by confdb.HLTProcess.customize().

398  def instrumentErrorEventType(self):
399  if self.config.errortype:
400  # change all HLTTriggerTypeFilter EDFilters to accept only error events (SelectedTriggerType = 0)
401  self._fix_parameter(name = 'SelectedTriggerType', type ='int32', value = '1', replace = '0')
402  self._fix_parameter(name = 'SelectedTriggerType', type ='int32', value = '2', replace = '0')
403  self._fix_parameter(name = 'SelectedTriggerType', type ='int32', value = '3', replace = '0')
404 
405 

◆ instrumentOpenMode()

def confdb.HLTProcess.instrumentOpenMode (   self)

Definition at line 384 of file confdb.py.

References geometry.Alignables.config, confdb.HLTProcess.config, preexistingValidation.PreexistingValidation.config, plottingOptions.BasePlottingOptions.config, pkg.AbstractPkg.config, cscdqm::EventProcessorMutex.config, genericValidation.GenericValidation.config, CSCOfflineClient.config, validateAlignments.ValidationBase.config, CSCMonitorModule.config, cscdqm::Collection.config, cscdqm::Dispatcher.config, cscdqm::EventProcessor.config, validateAlignments.ValidationJobMultiIOV.config, mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, ecal::raw::InputDataCPU.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, calo::multifit::MapSymM< T, Stride, Order >.data, KDTreeNodeInfo< DATA, DIM >.data, hcal::raw::InputDataCPU.data, dqmTnP::AbstractFitter.data, LASModuleProfile.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, GeometryComparisonPlotter.data, calo::multifit::MapMForPM< T >.data, edm::MapOfVectors< K, T >::Iter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, hcal::raw::InputDataGPU.data, data, EcalShowerContainmentCorrections::Coefficients.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, edmStreamStallGrapher.Stack.data, and join().

Referenced by confdb.HLTProcess.customize().

384  def instrumentOpenMode(self):
385  if self.config.open:
386  # find all EDfilters
387  filters = [ match[1] for match in re.findall(r'(process\.)?\b(\w+) = cms.EDFilter', self.data) ]
388  re_sequence = re.compile( r'cms\.(Path|Sequence)\((.*)\)' )
389  # remove existing 'cms.ignore' and '~' modifiers
390  self.data = re_sequence.sub( lambda line: re.sub( r'cms\.ignore *\( *((process\.)?\b(\w+)) *\)', r'\1', line.group(0) ), self.data )
391  self.data = re_sequence.sub( lambda line: re.sub( r'~', '', line.group(0) ), self.data )
392  # wrap all EDfilters with "cms.ignore( ... )", 1000 at a time (python 2.6 complains for too-big regular expressions)
393  for some in splitter(filters, 1000):
394  re_filters = re.compile( r'\b((process\.)?(' + r'|'.join(some) + r'))\b' )
395  self.data = re_sequence.sub( lambda line: re_filters.sub( r'cms.ignore( \1 )', line.group(0) ), self.data )
396 
397 
static std::string join(char **cmd)
Definition: RemoteFile.cc:19

◆ instrumentTiming()

def confdb.HLTProcess.instrumentTiming (   self)

Definition at line 627 of file confdb.py.

References geometry.Alignables.config, confdb.HLTProcess.config, preexistingValidation.PreexistingValidation.config, plottingOptions.BasePlottingOptions.config, pkg.AbstractPkg.config, cscdqm::EventProcessorMutex.config, genericValidation.GenericValidation.config, CSCOfflineClient.config, validateAlignments.ValidationBase.config, CSCMonitorModule.config, cscdqm::Collection.config, cscdqm::Dispatcher.config, cscdqm::EventProcessor.config, validateAlignments.ValidationJobMultiIOV.config, mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, ecal::raw::InputDataCPU.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, calo::multifit::MapSymM< T, Stride, Order >.data, KDTreeNodeInfo< DATA, DIM >.data, hcal::raw::InputDataCPU.data, dqmTnP::AbstractFitter.data, LASModuleProfile.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, GeometryComparisonPlotter.data, calo::multifit::MapMForPM< T >.data, edm::MapOfVectors< K, T >::Iter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, hcal::raw::InputDataGPU.data, data, EcalShowerContainmentCorrections::Coefficients.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, edmStreamStallGrapher.Stack.data, and confdb.HLTProcess.loadCff().

Referenced by confdb.HLTProcess.customize().

627  def instrumentTiming(self):
628 
629  if self.config.timing:
630  self.data += """
631 # instrument the menu with the modules and EndPath needed for timing studies
632 """
633 
634  self.data += '\n# configure the FastTimerService\n'
635  self.loadCff('HLTrigger.Timer.FastTimerService_cfi')
636 
637  self.data += """# print a text summary at the end of the job
638 %(process)s.FastTimerService.printEventSummary = False
639 %(process)s.FastTimerService.printRunSummary = False
640 %(process)s.FastTimerService.printJobSummary = True
641 
642 # enable DQM plots
643 %(process)s.FastTimerService.enableDQM = True
644 
645 # enable per-path DQM plots
646 %(process)s.FastTimerService.enableDQMbyPath = True
647 
648 # enable per-module DQM plots
649 %(process)s.FastTimerService.enableDQMbyModule = True
650 
651 # enable per-event DQM plots vs lumisection
652 %(process)s.FastTimerService.enableDQMbyLumiSection = True
653 %(process)s.FastTimerService.dqmLumiSectionsRange = 2500
654 
655 # set the time resolution of the DQM plots
656 %(process)s.FastTimerService.dqmTimeRange = 2000.
657 %(process)s.FastTimerService.dqmTimeResolution = 10.
658 %(process)s.FastTimerService.dqmPathTimeRange = 1000.
659 %(process)s.FastTimerService.dqmPathTimeResolution = 5.
660 %(process)s.FastTimerService.dqmModuleTimeRange = 200.
661 %(process)s.FastTimerService.dqmModuleTimeResolution = 1.
662 
663 # set the base DQM folder for the DQM plots
664 %(process)s.FastTimerService.dqmPath = 'HLT/TimerService'
665 %(process)s.FastTimerService.enableDQMbyProcesses = False
666 
667 # write a JSON file with the information to be displayed in a pie chart
668 %(process)s.FastTimerService.writeJSONSummary = True
669 %(process)s.FastTimerService.jsonFileName = 'resources.json'
670 """
671 
672  self.data += '\n# configure the ThroughputService\n'
673  self.loadCff('HLTrigger.Timer.ThroughputService_cfi')
674 
675  self.data += """# enable DQM plots
676 %(process)s.ThroughputService.enableDQM = True
677 
678 # set the resolution of the DQM plots
679 %(process)s.ThroughputService.eventRange = 10000
680 %(process)s.ThroughputService.eventResolution = 1
681 %(process)s.ThroughputService.timeRange = 60000
682 %(process)s.ThroughputService.timeResolution = 10
683 
684 # set the base DQM folder for the DQM plots
685 %(process)s.ThroughputService.dqmPath = 'HLT/Throughput'
686 %(process)s.ThroughputService.dqmPathByProcesses = False
687 """
688 
689 

◆ loadAdditionalConditions()

def confdb.HLTProcess.loadAdditionalConditions (   self,
  comment,
  conditions 
)

Definition at line 583 of file confdb.py.

References mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ecal::raw::InputDataCPU.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, KDTreeNodeInfo< DATA, DIM >.data, calo::multifit::MapSymM< T, Stride, Order >.data, hcal::raw::InputDataCPU.data, LASModuleProfile.data, dqmTnP::AbstractFitter.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, calo::multifit::MapMForPM< T >.data, GeometryComparisonPlotter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, edm::MapOfVectors< K, T >::Iter.data, hcal::raw::InputDataGPU.data, data, EcalShowerContainmentCorrections::Coefficients.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, and edmStreamStallGrapher.Stack.data.

583  def loadAdditionalConditions(self, comment, *conditions):
584  # load additional conditions
585  self.data += """
586 # %s
587 if 'GlobalTag' in %%(dict)s:
588 """ % comment
589  for condition in conditions:
590  self.data += """ %%(process)s.GlobalTag.toGet.append(
591  cms.PSet(
592  record = cms.string( '%(record)s' ),
593  tag = cms.string( '%(tag)s' ),
594  label = cms.untracked.string( '%(label)s' ),
595  )
596  )
597 """ % condition
598 
599 

◆ loadCff()

◆ loadCffCommand()

def confdb.HLTProcess.loadCffCommand (   self,
  module 
)

◆ loadSetupCff()

def confdb.HLTProcess.loadSetupCff (   self)

Definition at line 542 of file confdb.py.

References geometry.Alignables.config, confdb.HLTProcess.config, preexistingValidation.PreexistingValidation.config, plottingOptions.BasePlottingOptions.config, pkg.AbstractPkg.config, cscdqm::EventProcessorMutex.config, genericValidation.GenericValidation.config, CSCOfflineClient.config, validateAlignments.ValidationBase.config, CSCMonitorModule.config, cscdqm::Collection.config, cscdqm::Dispatcher.config, cscdqm::EventProcessor.config, validateAlignments.ValidationJobMultiIOV.config, mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, ecal::raw::InputDataCPU.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, calo::multifit::MapSymM< T, Stride, Order >.data, KDTreeNodeInfo< DATA, DIM >.data, hcal::raw::InputDataCPU.data, dqmTnP::AbstractFitter.data, LASModuleProfile.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, GeometryComparisonPlotter.data, calo::multifit::MapMForPM< T >.data, edm::MapOfVectors< K, T >::Iter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, hcal::raw::InputDataGPU.data, data, EcalShowerContainmentCorrections::Coefficients.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, edmStreamStallGrapher.Stack.data, and spr.find().

Referenced by confdb.HLTProcess.customize().

542  def loadSetupCff(self):
543  if self.config.setup is None:
544  return
545  processLine = self.data.find("\n",self.data.find("cms.Process"))
546  self.data = self.data[:processLine]+'\nprocess.load("%s")'%self.config.setupFile+self.data[processLine:]
547 
void find(edm::Handle< EcalRecHitCollection > &hits, DetId thisDet, std::vector< EcalRecHitCollection::const_iterator > &hit, bool debug=false)
Definition: FindCaloHit.cc:19

◆ overrideGlobalTag()

def confdb.HLTProcess.overrideGlobalTag (   self)

Definition at line 406 of file confdb.py.

References geometry.Alignables.config, confdb.HLTProcess.config, preexistingValidation.PreexistingValidation.config, plottingOptions.BasePlottingOptions.config, pkg.AbstractPkg.config, cscdqm::EventProcessorMutex.config, genericValidation.GenericValidation.config, CSCOfflineClient.config, validateAlignments.ValidationBase.config, CSCMonitorModule.config, cscdqm::Collection.config, cscdqm::Dispatcher.config, cscdqm::EventProcessor.config, validateAlignments.ValidationJobMultiIOV.config, mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, ecal::raw::InputDataCPU.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, calo::multifit::MapSymM< T, Stride, Order >.data, KDTreeNodeInfo< DATA, DIM >.data, hcal::raw::InputDataCPU.data, dqmTnP::AbstractFitter.data, LASModuleProfile.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, calo::multifit::MapMForPM< T >.data, GeometryComparisonPlotter.data, edm::MapOfVectors< K, T >::Iter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, hcal::raw::InputDataGPU.data, EcalShowerContainmentCorrections::Coefficients.data, data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, and edmStreamStallGrapher.Stack.data.

Referenced by confdb.HLTProcess.customize().

406  def overrideGlobalTag(self):
407  # overwrite GlobalTag
408  # the logic is:
409  # - if a GlobalTag is specified on the command line:
410  # - override the global tag
411  # - if the GT is "auto:...", insert the code to read it from Configuration.AlCa.autoCond
412  # - if a GlobalTag is NOT specified on the command line:
413  # - when running on data, do nothing, and keep the global tag in the menu
414  # - when running on mc, take the GT from the configuration.type
415 
416  # override the GlobalTag connection string and pfnPrefix
417 
418  # when running on MC, override the global tag even if not specified on the command line
419  if not self.config.data and not self.config.globaltag:
420  if self.config.type in globalTag:
421  self.config.globaltag = globalTag[self.config.type]
422  else:
423  self.config.globaltag = globalTag['GRun']
424 
425  # if requested, override the L1 menu from the GlobalTag
426  if self.config.l1.override:
427  self.config.l1.tag = self.config.l1.override
428  self.config.l1.record = 'L1TUtmTriggerMenuRcd'
429  self.config.l1.connect = ''
430  self.config.l1.label = ''
431  if not self.config.l1.snapshotTime:
432  self.config.l1.snapshotTime = '9999-12-31 23:59:59.000'
433  self.config.l1cond = '%(tag)s,%(record)s,%(connect)s,%(label)s,%(snapshotTime)s' % self.config.l1.__dict__
434  else:
435  self.config.l1cond = None
436 
437  if self.config.globaltag or self.config.l1cond:
438  text = """
439 # override the GlobalTag, connection string and pfnPrefix
440 if 'GlobalTag' in %(dict)s:
441  from Configuration.AlCa.GlobalTag import GlobalTag as customiseGlobalTag
442  %(process)s.GlobalTag = customiseGlobalTag(%(process)s.GlobalTag"""
443  if self.config.globaltag:
444  text += ", globaltag = %s" % repr(self.config.globaltag)
445  if self.config.l1cond:
446  text += ", conditions = %s" % repr(self.config.l1cond)
447  text += ")\n"
448  self.data += text
449 

◆ overrideL1MenuXml()

def confdb.HLTProcess.overrideL1MenuXml (   self)

Definition at line 450 of file confdb.py.

References geometry.Alignables.config, confdb.HLTProcess.config, preexistingValidation.PreexistingValidation.config, plottingOptions.BasePlottingOptions.config, pkg.AbstractPkg.config, cscdqm::EventProcessorMutex.config, genericValidation.GenericValidation.config, CSCOfflineClient.config, validateAlignments.ValidationBase.config, CSCMonitorModule.config, cscdqm::Collection.config, cscdqm::Dispatcher.config, cscdqm::EventProcessor.config, validateAlignments.ValidationJobMultiIOV.config, mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, ecal::raw::InputDataCPU.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, calo::multifit::MapSymM< T, Stride, Order >.data, KDTreeNodeInfo< DATA, DIM >.data, hcal::raw::InputDataCPU.data, dqmTnP::AbstractFitter.data, LASModuleProfile.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, calo::multifit::MapMForPM< T >.data, GeometryComparisonPlotter.data, edm::MapOfVectors< K, T >::Iter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, hcal::raw::InputDataGPU.data, EcalShowerContainmentCorrections::Coefficients.data, data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, and edmStreamStallGrapher.Stack.data.

Referenced by confdb.HLTProcess.customize().

450  def overrideL1MenuXml(self):
451  # if requested, override the GlobalTag's L1T menu from an Xml file
452  if self.config.l1Xml.XmlFile:
453  text = """
454 # override the GlobalTag's L1T menu from an Xml file
455 from HLTrigger.Configuration.CustomConfigs import L1XML
456 %%(process)s = L1XML(%%(process)s,"%s")
457 """ % (self.config.l1Xml.XmlFile)
458  self.data += text
459 

◆ overrideOutput()

def confdb.HLTProcess.overrideOutput (   self)

Definition at line 470 of file confdb.py.

References geometry.Alignables.config, confdb.HLTProcess.config, preexistingValidation.PreexistingValidation.config, plottingOptions.BasePlottingOptions.config, pkg.AbstractPkg.config, cscdqm::EventProcessorMutex.config, genericValidation.GenericValidation.config, CSCOfflineClient.config, validateAlignments.ValidationBase.config, CSCMonitorModule.config, cscdqm::Collection.config, cscdqm::Dispatcher.config, cscdqm::EventProcessor.config, validateAlignments.ValidationJobMultiIOV.config, mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, ecal::raw::InputDataCPU.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, calo::multifit::MapSymM< T, Stride, Order >.data, KDTreeNodeInfo< DATA, DIM >.data, hcal::raw::InputDataCPU.data, dqmTnP::AbstractFitter.data, LASModuleProfile.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, calo::multifit::MapMForPM< T >.data, GeometryComparisonPlotter.data, edm::MapOfVectors< K, T >::Iter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, hcal::raw::InputDataGPU.data, EcalShowerContainmentCorrections::Coefficients.data, data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, and edmStreamStallGrapher.Stack.data.

Referenced by confdb.HLTProcess.customize().

470  def overrideOutput(self):
471  # if not runnign on Hilton, override the "online" ShmStreamConsumer output modules with "offline" PoolOutputModule's
472  # note for Run3 ShmStreamConsumer has been replaced with EvFOutputModule and later GlobalEvFOutputModule
473  # so we also do a replace there
474  if not self.config.hilton:
475  self.data = re.sub(
476  r'\b(process\.)?hltOutput(\w+) *= *cms\.OutputModule\( *"(ShmStreamConsumer)" *,',
477  r'%(process)s.hltOutput\2 = cms.OutputModule( "PoolOutputModule",\n fileName = cms.untracked.string( "output\2.root" ),\n fastCloning = cms.untracked.bool( False ),\n dataset = cms.untracked.PSet(\n filterName = cms.untracked.string( "" ),\n dataTier = cms.untracked.string( "RAW" )\n ),',
478  self.data
479  )
480  self.data = re.sub(
481  r'\b(process\.)?hltOutput(\w+) *= *cms\.OutputModule\( *"EvFOutputModule" *,\n use_compression = cms.untracked.bool\( True \),\n compression_algorithm = cms.untracked.string\( "ZLIB" \),\n compression_level = cms.untracked.int32\( 1 \),\n lumiSection_interval = cms.untracked.int32\( 0 \),\n(.+?),\n psetMap = cms.untracked.InputTag\( "hltPSetMap" \)\n',
482  r'\1hltOutput\2 = cms.OutputModule( "PoolOutputModule",\n fileName = cms.untracked.string( "output\2.root" ),\n fastCloning = cms.untracked.bool( False ),\n dataset = cms.untracked.PSet(\n filterName = cms.untracked.string( "" ),\n dataTier = cms.untracked.string( "RAW" )\n ),\n\3\n',
483  self.data,0,re.DOTALL
484  )
485  self.data = re.sub(
486  r'\b(process\.)?hltOutput(\w+) *= *cms\.OutputModule\( *"GlobalEvFOutputModule" *,\n use_compression = cms.untracked.bool\( True \),\n compression_algorithm = cms.untracked.string\( "ZLIB" \),\n compression_level = cms.untracked.int32\( 1 \),\n lumiSection_interval = cms.untracked.int32\( 0 \),\n(.+?),\n psetMap = cms.untracked.InputTag\( "hltPSetMap" \)\n',
487  r'\1hltOutput\2 = cms.OutputModule( "PoolOutputModule",\n fileName = cms.untracked.string( "output\2.root" ),\n fastCloning = cms.untracked.bool( False ),\n dataset = cms.untracked.PSet(\n filterName = cms.untracked.string( "" ),\n dataTier = cms.untracked.string( "RAW" )\n ),\n\3\n',
488  self.data,0,re.DOTALL
489  )
490  if not self.config.fragment and self.config.output == 'minimal':
491  # add a single output to keep the TriggerResults and TriggerEvent
492  self.data += """
493 # add a single "keep *" output
494 %(process)s.hltOutputMinimal = cms.OutputModule( "PoolOutputModule",
495  fileName = cms.untracked.string( "output.root" ),
496  fastCloning = cms.untracked.bool( False ),
497  dataset = cms.untracked.PSet(
498  dataTier = cms.untracked.string( 'AOD' ),
499  filterName = cms.untracked.string( '' )
500  ),
501  outputCommands = cms.untracked.vstring( 'drop *',
502  'keep edmTriggerResults_*_*_*',
503  'keep triggerTriggerEvent_*_*_*',
504  'keep GlobalAlgBlkBXVector_*_*_*',
505  'keep GlobalExtBlkBXVector_*_*_*',
506  'keep l1tEGammaBXVector_*_EGamma_*',
507  'keep l1tEtSumBXVector_*_EtSum_*',
508  'keep l1tJetBXVector_*_Jet_*',
509  'keep l1tMuonBXVector_*_Muon_*',
510  'keep l1tTauBXVector_*_Tau_*',
511  )
512 )
513 %(process)s.MinimalOutput = cms.FinalPath( %(process)s.hltOutputMinimal )
514 %(process)s.schedule.append( %(process)s.MinimalOutput )
515 """
516  elif not self.config.fragment and self.config.output == 'full':
517  # add a single "keep *" output
518  self.data += """
519 # add a single "keep *" output
520 %(process)s.hltOutputFull = cms.OutputModule( "PoolOutputModule",
521  fileName = cms.untracked.string( "output.root" ),
522  fastCloning = cms.untracked.bool( False ),
523  dataset = cms.untracked.PSet(
524  dataTier = cms.untracked.string( 'RECO' ),
525  filterName = cms.untracked.string( '' )
526  ),
527  outputCommands = cms.untracked.vstring( 'keep *' )
528 )
529 %(process)s.FullOutput = cms.FinalPath( %(process)s.hltOutputFull )
530 %(process)s.schedule.append( %(process)s.FullOutput )
531 """
532 

◆ overrideParameters()

def confdb.HLTProcess.overrideParameters (   self,
  module,
  parameters 
)

Definition at line 611 of file confdb.py.

References mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ecal::raw::InputDataCPU.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, KDTreeNodeInfo< DATA, DIM >.data, calo::multifit::MapSymM< T, Stride, Order >.data, hcal::raw::InputDataCPU.data, LASModuleProfile.data, dqmTnP::AbstractFitter.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, calo::multifit::MapMForPM< T >.data, GeometryComparisonPlotter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, edm::MapOfVectors< K, T >::Iter.data, hcal::raw::InputDataGPU.data, data, EcalShowerContainmentCorrections::Coefficients.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, and edmStreamStallGrapher.Stack.data.

611  def overrideParameters(self, module, parameters):
612  # override a module's parameter if the module is present in the configuration
613  self.data += "if '%s' in %%(dict)s:\n" % module
614  for (parameter, value) in parameters:
615  self.data += " %%(process)s.%s.%s = %s\n" % (module, parameter, value)
616  self.data += "\n"
617 
618 

◆ overrideProcessName()

def confdb.HLTProcess.overrideProcessName (   self)

Definition at line 549 of file confdb.py.

References geometry.Alignables.config, confdb.HLTProcess.config, preexistingValidation.PreexistingValidation.config, plottingOptions.BasePlottingOptions.config, pkg.AbstractPkg.config, cscdqm::EventProcessorMutex.config, genericValidation.GenericValidation.config, CSCOfflineClient.config, validateAlignments.ValidationBase.config, CSCMonitorModule.config, cscdqm::Collection.config, cscdqm::Dispatcher.config, cscdqm::EventProcessor.config, validateAlignments.ValidationJobMultiIOV.config, mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, ecal::raw::InputDataCPU.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, calo::multifit::MapSymM< T, Stride, Order >.data, KDTreeNodeInfo< DATA, DIM >.data, hcal::raw::InputDataCPU.data, dqmTnP::AbstractFitter.data, LASModuleProfile.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, calo::multifit::MapMForPM< T >.data, GeometryComparisonPlotter.data, edm::MapOfVectors< K, T >::Iter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, hcal::raw::InputDataGPU.data, EcalShowerContainmentCorrections::Coefficients.data, data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, and edmStreamStallGrapher.Stack.data.

Referenced by confdb.HLTProcess.customize().

549  def overrideProcessName(self):
550  if self.config.name is None:
551  return
552 
553  # sanitise process name
554  self.config.name = self.config.name.replace("_","")
555  # override the process name
556  quote = '[\'\"]'
557  self.data = re.compile(r'^(process\s*=\s*cms\.Process\(\s*' + quote + r')\w+(' + quote + r'\s*\).*)$', re.MULTILINE).sub(r'\1%s\2' % self.config.name, self.data, 1)
558 
559  # when --setup option is used, remove possible errors from PrescaleService due to missing HLT paths.
560  if self.config.setup: self.data += """
561 # avoid PrescaleService error due to missing HLT paths
562 if 'PrescaleService' in process.__dict__:
563  for pset in reversed(process.PrescaleService.prescaleTable):
564  if not hasattr(process,pset.pathName.value()):
565  process.PrescaleService.prescaleTable.remove(pset)
566 """
567 
568 

◆ removeElementFromSequencesTasksAndPaths()

def confdb.HLTProcess.removeElementFromSequencesTasksAndPaths (   self,
  label 
)

Definition at line 619 of file confdb.py.

References mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ecal::raw::InputDataCPU.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, KDTreeNodeInfo< DATA, DIM >.data, calo::multifit::MapSymM< T, Stride, Order >.data, hcal::raw::InputDataCPU.data, LASModuleProfile.data, dqmTnP::AbstractFitter.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, calo::multifit::MapMForPM< T >.data, GeometryComparisonPlotter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, edm::MapOfVectors< K, T >::Iter.data, hcal::raw::InputDataGPU.data, data, EcalShowerContainmentCorrections::Coefficients.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, and edmStreamStallGrapher.Stack.data.

Referenced by confdb.HLTProcess.instrumentDQM().

619  def removeElementFromSequencesTasksAndPaths(self, label):
620  if label in self.data:
621  label_re = r'\b(process\.)?' + label
622  self.data = re.sub(r' *(\+|,) *' + label_re, '', self.data)
623  self.data = re.sub(label_re + r' *(\+|,) *', '', self.data)
624  self.data = re.sub(label_re, '', self.data)
625 
626 

◆ runL1Emulator()

def confdb.HLTProcess.runL1Emulator (   self)

Definition at line 460 of file confdb.py.

References geometry.Alignables.config, confdb.HLTProcess.config, preexistingValidation.PreexistingValidation.config, plottingOptions.BasePlottingOptions.config, pkg.AbstractPkg.config, cscdqm::EventProcessorMutex.config, genericValidation.GenericValidation.config, CSCOfflineClient.config, validateAlignments.ValidationBase.config, CSCMonitorModule.config, cscdqm::Collection.config, cscdqm::Dispatcher.config, cscdqm::EventProcessor.config, validateAlignments.ValidationJobMultiIOV.config, mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, ecal::raw::InputDataCPU.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, calo::multifit::MapSymM< T, Stride, Order >.data, KDTreeNodeInfo< DATA, DIM >.data, hcal::raw::InputDataCPU.data, dqmTnP::AbstractFitter.data, LASModuleProfile.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, calo::multifit::MapMForPM< T >.data, GeometryComparisonPlotter.data, edm::MapOfVectors< K, T >::Iter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, hcal::raw::InputDataGPU.data, EcalShowerContainmentCorrections::Coefficients.data, data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, and edmStreamStallGrapher.Stack.data.

Referenced by confdb.HLTProcess.customize().

460  def runL1Emulator(self):
461  # if requested, run the Full L1T emulator, then repack the data into a new RAW collection, to be used by the HLT
462  if self.config.emulator:
463  text = """
464 # run the Full L1T emulator, then repack the data into a new RAW collection, to be used by the HLT
465 from HLTrigger.Configuration.CustomConfigs import L1REPACK
466 %%(process)s = L1REPACK(%%(process)s,"%s")
467 """ % (self.config.emulator)
468  self.data += text
469 

◆ specificCustomize()

def confdb.HLTProcess.specificCustomize (   self)

Definition at line 176 of file confdb.py.

References geometry.Alignables.config, confdb.HLTProcess.config, preexistingValidation.PreexistingValidation.config, plottingOptions.BasePlottingOptions.config, pkg.AbstractPkg.config, cscdqm::EventProcessorMutex.config, genericValidation.GenericValidation.config, CSCOfflineClient.config, validateAlignments.ValidationBase.config, CSCMonitorModule.config, cscdqm::Collection.config, cscdqm::Dispatcher.config, cscdqm::EventProcessor.config, validateAlignments.ValidationJobMultiIOV.config, mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, ecal::raw::InputDataCPU.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, calo::multifit::MapSymM< T, Stride, Order >.data, KDTreeNodeInfo< DATA, DIM >.data, hcal::raw::InputDataCPU.data, dqmTnP::AbstractFitter.data, LASModuleProfile.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, calo::multifit::MapMForPM< T >.data, GeometryComparisonPlotter.data, edm::MapOfVectors< K, T >::Iter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, hcal::raw::InputDataGPU.data, data, EcalShowerContainmentCorrections::Coefficients.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, edmStreamStallGrapher.Stack.data, python.rootplot.root2matplotlib.replace(), confdb.HLTProcess.source, lhef::XMLDocument.source, DQMNet::Peer.source, Config.Process.source, and BeautifulSoup.ResultSet.source.

Referenced by confdb.HLTProcess.customize().

176  def specificCustomize(self):
177  # specific customizations now live in HLTrigger.Configuration.customizeHLTforALL.customizeHLTforAll(.,.)
178  if self.config.fragment:
179  self.data += """
180 # add specific customizations
181 from HLTrigger.Configuration.customizeHLTforALL import customizeHLTforAll
182 fragment = customizeHLTforAll(fragment,"%s")
183 """ % (self.config.type)
184  elif self.config.hilton:
185  # do not apply the STORM-specific customisation
186  pass
187  else:
188  if self.config.type=="Fake":
189  prefix = "run1"
190  elif self.config.type in ("Fake1","Fake2","2018"):
191  prefix = "run2"
192  else:
193  prefix = "run3"
194  _gtData = "auto:"+prefix+"_hlt_"+self.config.type
195  _gtMc = "auto:"+prefix+"_mc_" +self.config.type
196  self.data += """
197 # add specific customizations
198 _customInfo = {}
199 _customInfo['menuType' ]= "%s"
200 _customInfo['globalTags']= {}
201 _customInfo['globalTags'][True ] = "%s"
202 _customInfo['globalTags'][False] = "%s"
203 _customInfo['inputFiles']={}
204 _customInfo['inputFiles'][True] = "file:RelVal_Raw_%s_DATA.root"
205 _customInfo['inputFiles'][False] = "file:RelVal_Raw_%s_MC.root"
206 _customInfo['maxEvents' ]= %s
207 _customInfo['globalTag' ]= "%s"
208 _customInfo['inputFile' ]= %s
209 _customInfo['realData' ]= %s
210 
211 from HLTrigger.Configuration.customizeHLTforALL import customizeHLTforAll
212 %%(process)s = customizeHLTforAll(%%(process)s,"%s",_customInfo)
213 """ % (self.config.type,_gtData,_gtMc,self.config.type,self.config.type,self.config.events,self.config.globaltag,self.source,self.config.data,self.config.type)
214 
215  self.data += """
216 from HLTrigger.Configuration.customizeHLTforCMSSW import customizeHLTforCMSSW
217 %%(process)s = customizeHLTforCMSSW(%%(process)s,"%s")
218 """ % (self.config.type)
219 
220  # Eras-based customisations
221  self.data += """
222 # Eras-based customisations
223 from HLTrigger.Configuration.Eras import modifyHLTforEras
224 modifyHLTforEras(%(process)s)
225 """
226  # add the user-defined customization functions, if any
227  if self.config.customise:
228  self.data += "\n"
229  self.data += "#User-defined customization functions\n"
230  for customise in self.config.customise.split(","):
231  customiseValues = customise.split(".")
232  if len(customiseValues)>=3: raise Exception("--customise option cannot contain more than one dot.")
233  if len(customiseValues)==1:
234  customiseValues.append("customise")
235  customiseValues[0] = customiseValues[0].replace("/",".")
236  self.data += "from "+customiseValues[0]+" import "+customiseValues[1]+"\n"
237  self.data += "process = "+customiseValues[1]+"(process)\n"
238 
239 
def replace(string, replacements)

◆ updateMessageLogger()

def confdb.HLTProcess.updateMessageLogger (   self)

Definition at line 569 of file confdb.py.

References mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ecal::raw::InputDataCPU.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, KDTreeNodeInfo< DATA, DIM >.data, calo::multifit::MapSymM< T, Stride, Order >.data, hcal::raw::InputDataCPU.data, LASModuleProfile.data, dqmTnP::AbstractFitter.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, calo::multifit::MapMForPM< T >.data, GeometryComparisonPlotter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, edm::MapOfVectors< K, T >::Iter.data, hcal::raw::InputDataGPU.data, data, EcalShowerContainmentCorrections::Coefficients.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, and edmStreamStallGrapher.Stack.data.

Referenced by confdb.HLTProcess.customize().

569  def updateMessageLogger(self):
570  # request summary informations from the MessageLogger
571  self.data += """
572 # show summaries from trigger analysers used at HLT
573 if 'MessageLogger' in %(dict)s:
574  %(process)s.MessageLogger.TriggerSummaryProducerAOD = cms.untracked.PSet()
575  %(process)s.MessageLogger.L1GtTrigReport = cms.untracked.PSet()
576  %(process)s.MessageLogger.L1TGlobalSummary = cms.untracked.PSet()
577  %(process)s.MessageLogger.HLTrigReport = cms.untracked.PSet()
578  %(process)s.MessageLogger.FastReport = cms.untracked.PSet()
579  %(process)s.MessageLogger.ThroughputService = cms.untracked.PSet()
580 """
581 
582 

Member Data Documentation

◆ all_paths

confdb.HLTProcess.all_paths

Definition at line 729 of file confdb.py.

Referenced by confdb.HLTProcess.customize(), and confdb.HLTProcess.fixPrescales().

◆ config

◆ converter

◆ data

◆ labels

confdb.HLTProcess.labels

◆ options

confdb.HLTProcess.options

◆ parent

◆ source

confdb.HLTProcess.source