CMS 3D CMS Logo

 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Properties Friends Macros Groups Pages
List of all members | Public Member Functions | Static Public Member Functions | Public Attributes | Private Member Functions
confdb.HLTProcess Class Reference
Inheritance diagram for confdb.HLTProcess:

Public Member Functions

def __init__
 
def addEras
 
def addGlobalOptions
 
def append_filenames
 
def build_source
 
def buildOptions
 
def buildPathList
 
def customize
 
def dump
 
def expand_filenames
 
def fixPrescales
 
def getPathList
 
def getRawConfigurationFromDB
 
def getSetupConfigurationFromDB
 
def instrumentDQM
 
def instrumentErrorEventType
 
def instrumentOpenMode
 
def instrumentTiming
 
def loadAdditionalConditions
 
def loadCff
 
def loadCffCommand
 
def loadSetupCff
 
def overrideGlobalTag
 
def overrideL1MenuXml
 
def overrideOutput
 
def overrideParameters
 
def overrideProcessName
 
def removeElementFromSequencesTasksAndPaths
 
def runL1Emulator
 
def specificCustomize
 
def updateMessageLogger
 

Static Public Member Functions

def consolidateNegativeList
 
def consolidatePositiveList
 
def dumppaths
 
def expandWildcards
 

Public Attributes

 all_paths
 
 config
 
 converter
 
 data
 
 labels
 
 options
 
 parent
 
 source
 

Private Member Functions

def _fix_parameter
 

Detailed Description

Definition at line 21 of file confdb.py.

Constructor & Destructor Documentation

def confdb.HLTProcess.__init__ (   self,
  configuration 
)

Definition at line 23 of file confdb.py.

23 
24  def __init__(self, configuration):
25  self.config = configuration
26  self.data = None
27  self.source = []
28  self.parent = []
29 
30  self.options = {
31  'essources' : [],
32  'esmodules' : [],
33  'modules' : [],
34  'sequences' : [],
35  'services' : [],
36  'paths' : [],
37  'psets' : [],
38  'blocks' : [],
39  }
40 
41  self.labels = {}
42  if self.config.fragment:
43  self.labels['process'] = 'fragment'
44  self.labels['dict'] = 'fragment.__dict__'
45  else:
46  self.labels['process'] = 'process'
47  self.labels['dict'] = 'process.__dict__'
48 
49  if self.config.prescale and (self.config.prescale.lower() != 'none'):
50  self.labels['prescale'] = self.config.prescale
51 
52  # get the configuration from ConfdB
53  from .confdbOfflineConverter import OfflineConverter
54  self.converter = OfflineConverter(version = self.config.menu.version, database = self.config.menu.database, proxy = self.config.proxy, proxyHost = self.config.proxy_host, proxyPort = self.config.proxy_port)
55  self.buildPathList()
56  self.buildOptions()
59  self.customize()
def getSetupConfigurationFromDB
Definition: confdb.py:60
def getRawConfigurationFromDB
Definition: confdb.py:83

Member Function Documentation

def confdb.HLTProcess._fix_parameter (   self,
  args 
)
private
arguments:
name:     parameter name (optional)
type:     parameter type (look for tracked and untracked variants)
value:    original value
replace:  replacement value

Definition at line 337 of file confdb.py.

References mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, ecal::raw::InputDataCPU.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, KDTreeNodeInfo< DATA, DIM >.data, calo::multifit::MapSymM< T, Stride, Order >.data, hcal::raw::InputDataCPU.data, LASModuleProfile.data, dqmTnP::AbstractFitter.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, calo::multifit::MapMForPM< T >.data, GeometryComparisonPlotter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, edm::MapOfVectors< K, T >::Iter.data, hcal::raw::InputDataGPU.data, @775.data, EcalShowerContainmentCorrections::Coefficients.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, and edmStreamStallGrapher.Stack.data.

Referenced by confdb.HLTProcess.customize(), and confdb.HLTProcess.instrumentErrorEventType().

338  def _fix_parameter(self, **args):
339  """arguments:
340  name: parameter name (optional)
341  type: parameter type (look for tracked and untracked variants)
342  value: original value
343  replace: replacement value
344  """
345  if 'name' in args:
346  self.data = re.sub(
347  r'%(name)s = cms(?P<tracked>(?:\.untracked)?)\.%(type)s\( (?P<quote>["\']?)%(value)s(?P=quote)' % args,
348  r'%(name)s = cms\g<tracked>.%(type)s( \g<quote>%(replace)s\g<quote>' % args,
349  self.data)
350  else:
351  self.data = re.sub(
352  r'cms(?P<tracked>(?:\.untracked)?)\.%(type)s\( (?P<quote>["\']?)%(value)s(?P=quote)' % args,
353  r'cms\g<tracked>.%(type)s( \g<quote>%(replace)s\g<quote>' % args,
354  self.data)
355 
def _fix_parameter
Definition: confdb.py:337
def confdb.HLTProcess.addEras (   self)

Definition at line 526 of file confdb.py.

References mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, ecal::raw::InputDataCPU.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, calo::multifit::MapSymM< T, Stride, Order >.data, KDTreeNodeInfo< DATA, DIM >.data, hcal::raw::InputDataCPU.data, LASModuleProfile.data, dqmTnP::AbstractFitter.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, calo::multifit::MapMForPM< T >.data, GeometryComparisonPlotter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, edm::MapOfVectors< K, T >::Iter.data, hcal::raw::InputDataGPU.data, @775.data, EcalShowerContainmentCorrections::Coefficients.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, edmStreamStallGrapher.Stack.data, and join().

Referenced by confdb.HLTProcess.customize().

527  def addEras(self):
528  if self.config.eras is None:
529  return
530  from Configuration.StandardSequences.Eras import eras
531  erasSplit = self.config.eras.split(',')
532  self.data = re.sub(r'process = cms.Process\( *"\w+"', '\n'.join(eras.pythonCfgLines[era] for era in erasSplit)+'\n\g<0>, '+', '.join(era for era in erasSplit), self.data)
static std::string join(char **cmd)
Definition: RemoteFile.cc:19
def confdb.HLTProcess.addGlobalOptions (   self)

Definition at line 319 of file confdb.py.

References mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, ecal::raw::InputDataCPU.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, KDTreeNodeInfo< DATA, DIM >.data, calo::multifit::MapSymM< T, Stride, Order >.data, hcal::raw::InputDataCPU.data, LASModuleProfile.data, dqmTnP::AbstractFitter.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, calo::multifit::MapMForPM< T >.data, GeometryComparisonPlotter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, edm::MapOfVectors< K, T >::Iter.data, hcal::raw::InputDataGPU.data, @775.data, EcalShowerContainmentCorrections::Coefficients.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, and edmStreamStallGrapher.Stack.data.

Referenced by confdb.HLTProcess.customize().

320  def addGlobalOptions(self):
321  # add global options
322  self.data += """
323 # limit the number of events to be processed
324 %%(process)s.maxEvents = cms.untracked.PSet(
325  input = cms.untracked.int32( %d )
326 )
327 """ % self.config.events
328 
329  self.data += """
330 # enable TrigReport, TimeReport and MultiThreading
331 %(process)s.options = cms.untracked.PSet(
332  wantSummary = cms.untracked.bool( True ),
333  numberOfThreads = cms.untracked.uint32( 4 ),
334  numberOfStreams = cms.untracked.uint32( 0 ),
335 )
336 """
def addGlobalOptions
Definition: confdb.py:319
def confdb.HLTProcess.append_filenames (   self,
  name,
  filenames 
)

Definition at line 863 of file confdb.py.

References mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, ecal::raw::InputDataCPU.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, KDTreeNodeInfo< DATA, DIM >.data, calo::multifit::MapSymM< T, Stride, Order >.data, hcal::raw::InputDataCPU.data, LASModuleProfile.data, dqmTnP::AbstractFitter.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, calo::multifit::MapMForPM< T >.data, GeometryComparisonPlotter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, edm::MapOfVectors< K, T >::Iter.data, hcal::raw::InputDataGPU.data, @775.data, EcalShowerContainmentCorrections::Coefficients.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, and edmStreamStallGrapher.Stack.data.

Referenced by confdb.HLTProcess.build_source().

864  def append_filenames(self, name, filenames):
865  if len(filenames) > 255:
866  token_open = "( *("
867  token_close = ") )"
868  else:
869  token_open = "("
870  token_close = ")"
871 
872  self.data += " %s = cms.untracked.vstring%s\n" % (name, token_open)
873  for line in filenames:
874  self.data += " '%s',\n" % line
875  self.data += " %s,\n" % (token_close)
876 
def append_filenames
Definition: confdb.py:863
def confdb.HLTProcess.build_source (   self)

Definition at line 889 of file confdb.py.

References confdb.HLTProcess.append_filenames(), mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, ecal::raw::InputDataCPU.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, KDTreeNodeInfo< DATA, DIM >.data, calo::multifit::MapSymM< T, Stride, Order >.data, hcal::raw::InputDataCPU.data, dqmTnP::AbstractFitter.data, LASModuleProfile.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, GeometryComparisonPlotter.data, calo::multifit::MapMForPM< T >.data, edm::MapOfVectors< K, T >::Iter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, hcal::raw::InputDataGPU.data, EcalShowerContainmentCorrections::Coefficients.data, @775.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, edmStreamStallGrapher.Stack.data, confdb.HLTProcess.expand_filenames(), confdb.HLTProcess.parent, FWPSetTableManager::PSetData.parent, SlimmedCount.parent, dqm::implementation::IBooker::UseScope< SCOPE >.parent, confdb.HLTProcess.source, lhef::XMLDocument.source, DQMNet::Peer.source, Config.Process.source, gen::Hydjet2Hadronizer.source, and BeautifulSoup.ResultSet.source.

Referenced by confdb.HLTProcess.customize().

890  def build_source(self):
891  if self.config.hilton:
892  # use the DAQ source
893  return
894 
895  if self.config.input:
896  # if a dataset or a list of input files was given, use it
897  self.source = self.expand_filenames(self.config.input)
898  elif self.config.data:
899  # offline we can run on data...
900  self.source = [ "file:RelVal_Raw_%s_DATA.root" % self.config.type ]
901  else:
902  # ...or on mc
903  self.source = [ "file:RelVal_Raw_%s_MC.root" % self.config.type ]
904 
905  if self.config.parent:
906  # if a dataset or a list of input files was given for the parent data, use it
907  self.parent = self.expand_filenames(self.config.parent)
908 
909  self.data += """
910 # source module (EDM inputs)
911 %(process)s.source = cms.Source( "PoolSource",
912 """
913  self.append_filenames("fileNames", self.source)
914  if (self.parent):
915  self.append_filenames("secondaryFileNames", self.parent)
916  self.data += """\
917  inputCommands = cms.untracked.vstring(
918  'keep *'
919  )
920 )
921 """
def expand_filenames
Definition: confdb.py:877
def append_filenames
Definition: confdb.py:863
def confdb.HLTProcess.buildOptions (   self)

Definition at line 765 of file confdb.py.

References bitset_utilities.append(), DTCalibrationWorker.DTCalibrationWorker.options, DTWorkflow.DTWorkflow.options, TestProcess.TestProcess.options, confdb.HLTProcess.options, Config.Process.options, and validateAlignments.ValidationJobMultiIOV.options.

766  def buildOptions(self):
767  # common configuration for all scenarios
768  self.options['services'].append( "-DQM" )
769  self.options['services'].append( "-FUShmDQMOutputService" )
770  self.options['services'].append( "-MicroStateService" )
771  self.options['services'].append( "-ModuleWebRegistry" )
772  self.options['services'].append( "-TimeProfilerService" )
773 
774  # remove the DAQ modules and the online definition of the DQMStore and DQMFileSaver
775  # unless a hilton-like configuration has been requested
776  if not self.config.hilton:
777  self.options['services'].append( "-EvFDaqDirector" )
778  self.options['services'].append( "-FastMonitoringService" )
779  self.options['services'].append( "-DQMStore" )
780  self.options['modules'].append( "-hltDQMFileSaver" )
781  self.options['modules'].append( "-hltDQMFileSaverPB" )
782 
783  if self.config.fragment:
784  # extract a configuration file fragment
785  self.options['essources'].append( "-GlobalTag" )
786  self.options['essources'].append( "-HepPDTESSource" )
787  self.options['essources'].append( "-XMLIdealGeometryESSource" )
788  self.options['essources'].append( "-eegeom" )
789  self.options['essources'].append( "-es_hardcode" )
790  self.options['essources'].append( "-magfield" )
791 
792  self.options['esmodules'].append( "-SlaveField0" )
793  self.options['esmodules'].append( "-SlaveField20" )
794  self.options['esmodules'].append( "-SlaveField30" )
795  self.options['esmodules'].append( "-SlaveField35" )
796  self.options['esmodules'].append( "-SlaveField38" )
797  self.options['esmodules'].append( "-SlaveField40" )
798  self.options['esmodules'].append( "-VBF0" )
799  self.options['esmodules'].append( "-VBF20" )
800  self.options['esmodules'].append( "-VBF30" )
801  self.options['esmodules'].append( "-VBF35" )
802  self.options['esmodules'].append( "-VBF38" )
803  self.options['esmodules'].append( "-VBF40" )
804  self.options['esmodules'].append( "-CSCGeometryESModule" )
805  self.options['esmodules'].append( "-CaloGeometryBuilder" )
806  self.options['esmodules'].append( "-CaloTowerHardcodeGeometryEP" )
807  self.options['esmodules'].append( "-CastorHardcodeGeometryEP" )
808  self.options['esmodules'].append( "-DTGeometryESModule" )
809  self.options['esmodules'].append( "-EcalBarrelGeometryEP" )
810  self.options['esmodules'].append( "-EcalElectronicsMappingBuilder" )
811  self.options['esmodules'].append( "-EcalEndcapGeometryEP" )
812  self.options['esmodules'].append( "-EcalLaserCorrectionService" )
813  self.options['esmodules'].append( "-EcalPreshowerGeometryEP" )
814  self.options['esmodules'].append( "-GEMGeometryESModule" )
815  self.options['esmodules'].append( "-HcalHardcodeGeometryEP" )
816  self.options['esmodules'].append( "-HcalTopologyIdealEP" )
817  self.options['esmodules'].append( "-MuonNumberingInitialization" )
818  self.options['esmodules'].append( "-ParametrizedMagneticFieldProducer" )
819  self.options['esmodules'].append( "-RPCGeometryESModule" )
820  self.options['esmodules'].append( "-SiStripGainESProducer" )
821  self.options['esmodules'].append( "-SiStripRecHitMatcherESProducer" )
822  self.options['esmodules'].append( "-SiStripQualityESProducer" )
823  self.options['esmodules'].append( "-StripCPEfromTrackAngleESProducer" )
824  self.options['esmodules'].append( "-TrackerAdditionalParametersPerDetESModule" )
825  self.options['esmodules'].append( "-TrackerDigiGeometryESModule" )
826  self.options['esmodules'].append( "-TrackerGeometricDetESModule" )
827  self.options['esmodules'].append( "-VolumeBasedMagneticFieldESProducer" )
828  self.options['esmodules'].append( "-ZdcHardcodeGeometryEP" )
829  self.options['esmodules'].append( "-hcal_db_producer" )
830  self.options['esmodules'].append( "-L1GtTriggerMaskAlgoTrigTrivialProducer" )
831  self.options['esmodules'].append( "-L1GtTriggerMaskTechTrigTrivialProducer" )
832  self.options['esmodules'].append( "-hltESPEcalTrigTowerConstituentsMapBuilder" )
833  self.options['esmodules'].append( "-hltESPGlobalTrackingGeometryESProducer" )
834  self.options['esmodules'].append( "-hltESPMuonDetLayerGeometryESProducer" )
835  self.options['esmodules'].append( "-hltESPTrackerRecoGeometryESProducer" )
836  self.options['esmodules'].append( "-trackerTopology" )
837 
838  self.options['esmodules'].append( "-CaloTowerGeometryFromDBEP" )
839  self.options['esmodules'].append( "-CastorGeometryFromDBEP" )
840  self.options['esmodules'].append( "-EcalBarrelGeometryFromDBEP" )
841  self.options['esmodules'].append( "-EcalEndcapGeometryFromDBEP" )
842  self.options['esmodules'].append( "-EcalPreshowerGeometryFromDBEP" )
843  self.options['esmodules'].append( "-HcalGeometryFromDBEP" )
844  self.options['esmodules'].append( "-ZdcGeometryFromDBEP" )
845  self.options['esmodules'].append( "-XMLFromDBSource" )
846  self.options['esmodules'].append( "-sistripconn" )
847 
848  self.options['services'].append( "-MessageLogger" )
849 
850  self.options['psets'].append( "-maxEvents" )
851  self.options['psets'].append( "-options" )
852 
853  # remove Scouting OutputModules even though the EndPaths are kept
854  self.options['modules'].append( "-hltOutputScoutingCaloMuon" )
855  self.options['modules'].append( "-hltOutputScoutingPF" )
856 
857  if self.config.fragment or (self.config.prescale and (self.config.prescale.lower() == 'none')):
858  self.options['services'].append( "-PrescaleService" )
859 
860  if self.config.fragment or self.config.timing:
861  self.options['services'].append( "-FastTimerService" )
862 
boost::dynamic_bitset append(const boost::dynamic_bitset<> &bs1, const boost::dynamic_bitset<> &bs2)
this method takes two bitsets bs1 and bs2 and returns result of bs2 appended to the end of bs1 ...
def confdb.HLTProcess.buildPathList (   self)

Definition at line 700 of file confdb.py.

701  def buildPathList(self):
702  self.all_paths = self.getPathList()
703 
704  if self.config.paths:
705  # no path list was requested, dump the full table, minus unsupported / unwanted paths
706  paths = self.config.paths.split(',')
707  else:
708  # dump only the requested paths, plus the eventual output endpaths
709  paths = []
710 
711  # 'none' should remove all outputs
712  # 'dqm' should remove all outputs but DQMHistograms
713  # 'minimal' should remove all outputs but DQMHistograms, and add a single output module to keep the TriggerResults and TriggerEvent
714  # 'full' should remove all outputs but DQMHistograms, and add a single output module to "keep *"
715  # See also the `overrideOutput` method
716  if self.config.fragment or self.config.output in ('none', ):
717  if self.config.paths:
718  # keep only the Paths and EndPaths requested explicitly
719  pass
720  else:
721  # drop all output EndPaths but the Scouting ones, and drop the RatesMonitoring and DQMHistograms
722  paths.append( "-*Output" )
723  paths.append( "-RatesMonitoring")
724  paths.append( "-DQMHistograms")
725  if self.config.fragment: paths.append( "Scouting*Output" )
726 
727  elif self.config.output in ('dqm', 'minimal', 'full'):
728  if self.config.paths:
729  # keep only the Paths and EndPaths requested explicitly, and the DQMHistograms
730  paths.append( "DQMHistograms" )
731  else:
732  # drop all output EndPaths but the Scouting ones, and drop the RatesMonitoring
733  paths.append( "-*Output" )
734  paths.append( "-RatesMonitoring")
735  if self.config.fragment: paths.append( "Scouting*Output" )
736 
737  else:
738  if self.config.paths:
739  # keep all output EndPaths, including the DQMHistograms
740  paths.append( "*Output" )
741  paths.append( "DQMHistograms" )
742  else:
743  # keep all Paths and EndPaths
744  pass
745 
746  # drop unwanted paths for profiling (and timing studies)
747  if self.config.profiling:
748  paths.append( "-HLTAnalyzerEndpath" )
749 
750  # this should never be in any dump (nor online menu)
751  paths.append( "-OfflineOutput" )
752 
753  # expand all wildcards
754  paths = self.expandWildcards(paths, self.all_paths)
755 
756  if self.config.paths:
757  # do an "additive" consolidation
758  paths = self.consolidatePositiveList(paths)
759  if not paths:
760  raise RuntimeError('Error: option "--paths %s" does not select any valid paths' % self.config.paths)
761  else:
762  # do a "subtractive" consolidation
763  paths = self.consolidateNegativeList(paths)
764  self.options['paths'] = paths
def consolidateNegativeList
Definition: confdb.py:142
def consolidatePositiveList
Definition: confdb.py:154
def expandWildcards
Definition: confdb.py:126
def confdb.HLTProcess.consolidateNegativeList (   elements)
static

Definition at line 142 of file confdb.py.

143  def consolidateNegativeList(elements):
144  # consolidate a list of path exclusions and re-inclusions
145  # the result is the list of paths to be removed from the dump
146  result = set()
147  for element in elements:
148  if element[0] == '-':
149  result.add( element )
150  else:
151  result.discard( '-' + element )
152  return sorted( element for element in result )
def consolidateNegativeList
Definition: confdb.py:142
def confdb.HLTProcess.consolidatePositiveList (   elements)
static

Definition at line 154 of file confdb.py.

155  def consolidatePositiveList(elements):
156  # consolidate a list of path selection and re-exclusions
157  # the result is the list of paths to be included in the dump
158  result = set()
159  for element in elements:
160  if element[0] == '-':
161  result.discard( element[1:] )
162  else:
163  result.add( element )
164  return sorted( element for element in result )
165 
def consolidatePositiveList
Definition: confdb.py:154
def confdb.HLTProcess.customize (   self)

Definition at line 241 of file confdb.py.

References confdb.HLTProcess._fix_parameter(), confdb.HLTProcess.addEras(), confdb.HLTProcess.addGlobalOptions(), confdb.HLTProcess.all_paths, confdb.HLTProcess.build_source(), mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, ecal::raw::InputDataCPU.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, KDTreeNodeInfo< DATA, DIM >.data, calo::multifit::MapSymM< T, Stride, Order >.data, hcal::raw::InputDataCPU.data, dqmTnP::AbstractFitter.data, LASModuleProfile.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, calo::multifit::MapMForPM< T >.data, GeometryComparisonPlotter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, edm::MapOfVectors< K, T >::Iter.data, hcal::raw::InputDataGPU.data, @775.data, EcalShowerContainmentCorrections::Coefficients.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, edmStreamStallGrapher.Stack.data, confdb.HLTProcess.fixPrescales(), confdb.HLTProcess.instrumentDQM(), confdb.HLTProcess.instrumentErrorEventType(), confdb.HLTProcess.instrumentOpenMode(), confdb.HLTProcess.instrumentTiming(), confdb.HLTProcess.loadSetupCff(), confdb.HLTProcess.overrideGlobalTag(), confdb.HLTProcess.overrideL1MenuXml(), confdb.HLTProcess.overrideOutput(), confdb.HLTProcess.overrideProcessName(), confdb.HLTProcess.runL1Emulator(), confdb.HLTProcess.specificCustomize(), and confdb.HLTProcess.updateMessageLogger().

242  def customize(self):
243 
244  # adapt the source to the current scenario
245  if not self.config.fragment:
246  self.build_source()
247 
248  # manual override some parameters
249  if self.config.type in ('HIon', ):
250  if self.config.data:
251  if not self.config.fragment:
252  self._fix_parameter( type = 'InputTag', value = 'rawDataCollector', replace = 'rawDataRepacker')
253 
254  # if requested, remove the HLT prescales
255  self.fixPrescales()
256 
257  # if requested, override all ED/HLTfilters to always pass ("open" mode)
258  self.instrumentOpenMode()
259 
260  # if requested, change all HLTTriggerTypeFilter EDFilters to accept only error events (SelectedTriggerType = 0)
262 
263  # if requested, instrument the self with the modules and EndPath needed for timing studies
264  self.instrumentTiming()
265 
266  # if requested, override the L1 self from the GlobalTag (Xml)
267  self.overrideL1MenuXml()
268 
269  # if requested, run the L1 emulator
270  self.runL1Emulator()
271 
272  # add process.load("setup_cff")
273  self.loadSetupCff()
274 
275  if self.config.fragment:
276  self.data += """
277 # dummify hltGetConditions in cff's
278 if 'hltGetConditions' in %(dict)s and 'HLTriggerFirstPath' in %(dict)s :
279  %(process)s.hltDummyConditions = cms.EDFilter( "HLTBool",
280  result = cms.bool( True )
281  )
282  %(process)s.HLTriggerFirstPath.replace(%(process)s.hltGetConditions,%(process)s.hltDummyConditions)
283 """
284 
285  # fix the Scouting EndPaths
286  for path in self.all_paths:
287  match = re.match(r'(Scouting\w+)Output$', path)
288  if match:
289  module = 'hltOutput' + match.group(1)
290  self.data = self.data.replace(path+' = cms.EndPath', path+' = cms.Path')
291  self.data = self.data.replace(' + process.'+module, '')
292 
293  else:
294 
295  # override the process name and adapt the relevant filters
296  self.overrideProcessName()
297 
298  # select specific Eras
299  self.addEras()
300 
301  # override the output modules to output root files
302  self.overrideOutput()
303 
304  # add global options
305  self.addGlobalOptions()
306 
307  # if requested or necessary, override the GlobalTag and connection strings (incl. L1!)
308  self.overrideGlobalTag()
309 
310  # request summary informations from the MessageLogger
311  self.updateMessageLogger()
312 
313  # replace DQMStore and DQMRootOutputModule with a configuration suitable for running offline
314  self.instrumentDQM()
315 
316  # add specific customisations
317  self.specificCustomize()
318 
def _fix_parameter
Definition: confdb.py:337
def overrideL1MenuXml
Definition: confdb.py:446
def addGlobalOptions
Definition: confdb.py:319
def overrideOutput
Definition: confdb.py:466
def overrideProcessName
Definition: confdb.py:541
def specificCustomize
Definition: confdb.py:176
def instrumentErrorEventType
Definition: confdb.py:394
def overrideGlobalTag
Definition: confdb.py:402
def instrumentTiming
Definition: confdb.py:619
def instrumentOpenMode
Definition: confdb.py:380
def updateMessageLogger
Definition: confdb.py:561
def confdb.HLTProcess.dump (   self)

Definition at line 167 of file confdb.py.

References mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, ecal::raw::InputDataCPU.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, KDTreeNodeInfo< DATA, DIM >.data, calo::multifit::MapSymM< T, Stride, Order >.data, hcal::raw::InputDataCPU.data, LASModuleProfile.data, dqmTnP::AbstractFitter.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, calo::multifit::MapMForPM< T >.data, GeometryComparisonPlotter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, edm::MapOfVectors< K, T >::Iter.data, hcal::raw::InputDataGPU.data, @775.data, EcalShowerContainmentCorrections::Coefficients.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, edmStreamStallGrapher.Stack.data, confdb.HLTProcess.labels, ecaldqm::binning::AxisSpecs.labels, and svgfig.Ticks.labels.

168  def dump(self):
169  self.data = self.data % self.labels
170  if self.config.fragment:
171  self.data = re.sub( r'\bprocess\b', 'fragment', self.data )
172  self.data = re.sub( r'\bProcess\b', 'ProcessFragment', self.data )
173  return self.data
174 
def confdb.HLTProcess.dumppaths (   paths)
static

Definition at line 694 of file confdb.py.

695  def dumppaths(paths):
696  sys.stderr.write('Path selection:\n')
697  for path in paths:
698  sys.stderr.write('\t%s\n' % path)
699  sys.stderr.write('\n\n')
def confdb.HLTProcess.expand_filenames (   self,
  input 
)

Definition at line 877 of file confdb.py.

Referenced by confdb.HLTProcess.build_source().

878  def expand_filenames(self, input):
879  # check if the input is a dataset or a list of files
880  if input[0:8] == 'dataset:':
881  from .dasFileQuery import dasFileQuery
882  # extract the dataset name, and use DAS to fine the list of LFNs
883  dataset = input[8:]
884  files = dasFileQuery(dataset)
885  else:
886  # assume a comma-separated list of input files
887  files = input.split(',')
888  return files
def expand_filenames
Definition: confdb.py:877
def confdb.HLTProcess.expandWildcards (   globs,
  collection 
)
static

Definition at line 126 of file confdb.py.

References python.rootplot.root2matplotlib.replace().

127  def expandWildcards(globs, collection):
128  # expand a list of unix-style wildcards matching a given collection
129  # wildcards with no matches are silently discarded
130  matches = []
131  for glob in globs:
132  negate = ''
133  if glob[0] == '-':
134  negate = '-'
135  glob = glob[1:]
136  # translate a unix-style glob expression into a regular expression
137  filter = re.compile(r'^' + glob.replace('?', '.').replace('*', '.*').replace('[!', '[^') + r'$')
138  matches.extend( negate + element for element in collection if filter.match(element) )
139  return matches
140 
def expandWildcards
Definition: confdb.py:126
def confdb.HLTProcess.fixPrescales (   self)

Definition at line 356 of file confdb.py.

References confdb.HLTProcess.all_paths, mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, ecal::DigisCollection< StoragePolicy >.data, TrackProbabilityCalibration.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, ecal::raw::InputDataCPU.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, KDTreeNodeInfo< DATA, DIM >.data, calo::multifit::MapSymM< T, Stride, Order >.data, hcal::raw::InputDataCPU.data, LASModuleProfile.data, dqmTnP::AbstractFitter.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, GeometryComparisonPlotter.data, calo::multifit::MapMForPM< T >.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, edm::MapOfVectors< K, T >::Iter.data, hcal::raw::InputDataGPU.data, EcalShowerContainmentCorrections::Coefficients.data, @775.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, edmStreamStallGrapher.Stack.data, DTCalibrationWorker.DTCalibrationWorker.options, DTWorkflow.DTWorkflow.options, TestProcess.TestProcess.options, confdb.HLTProcess.options, Config.Process.options, and validateAlignments.ValidationJobMultiIOV.options.

Referenced by confdb.HLTProcess.customize().

357  def fixPrescales(self):
358  # update the PrescaleService to match the new list of paths
359  if self.options['paths']:
360  if self.options['paths'][0][0] == '-':
361  # drop requested paths
362  for minuspath in self.options['paths']:
363  path = minuspath[1:]
364  self.data = re.sub(r' cms.PSet\( pathName = cms.string\( "%s" \),\n prescales = cms.vuint32\( .* \)\n \),?\n' % path, '', self.data)
365  else:
366  # keep requested paths
367  for path in self.all_paths:
368  if path not in self.options['paths']:
369  self.data = re.sub(r' cms.PSet\( pathName = cms.string\( "%s" \),\n prescales = cms.vuint32\( .* \)\n \),?\n' % path, '', self.data)
370 
371  if self.config.prescale and (self.config.prescale.lower() != 'none'):
372  # TO DO: check that the requested prescale column is valid
373  self.data += """
374 # force the use of a specific HLT prescale column
375 if 'PrescaleService' in %(dict)s:
376  %(process)s.PrescaleService.forceDefault = True
377  %(process)s.PrescaleService.lvl1DefaultLabel = '%(prescale)s'
378 """
379 
def confdb.HLTProcess.getPathList (   self)

Definition at line 101 of file confdb.py.

102  def getPathList(self):
103  if self.config.menu.run:
104  args = ['--runNumber', self.config.menu.run]
105  else:
106  args = ['--configName', self.config.menu.name]
107  args.extend( (
108  '--cff',
109  '--noedsources',
110  '--noes',
111  '--noservices',
112  '--nosequences',
113  '--nomodules'
114  ) )
115 
116  data, err = self.converter.query( *args )
117  if 'ERROR' in err or 'Exhausted Resultset' in err or 'CONFIG_NOT_FOUND' in err:
118  sys.stderr.write("%s: error while retrieving the list of paths from the HLT menu\n\n" % os.path.basename(sys.argv[0]))
119  sys.stderr.write(err + "\n\n")
120  sys.exit(1)
121  filter = re.compile(r' *= *cms.(End)?Path.*')
122  paths = [ filter.sub('', line) for line in data.splitlines() if filter.search(line) ]
123  return paths
124 
def confdb.HLTProcess.getRawConfigurationFromDB (   self)

Definition at line 83 of file confdb.py.

References mat4.data, CombinedTauTagCalibration.data, CombinedSVCalibration.data, ecal::DigisCollection< StoragePolicy >.data, TrackProbabilityCalibration.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, ecal::raw::InputDataCPU.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, calo::multifit::MapSymM< T, Stride, Order >.data, KDTreeNodeInfo< DATA, DIM >.data, hcal::raw::InputDataCPU.data, dqmTnP::AbstractFitter.data, LASModuleProfile.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, GeometryComparisonPlotter.data, calo::multifit::MapMForPM< T >.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, edm::MapOfVectors< K, T >::Iter.data, hcal::raw::InputDataGPU.data, EcalShowerContainmentCorrections::Coefficients.data, @775.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, edmStreamStallGrapher.Stack.data, and join().

83 
84  def getRawConfigurationFromDB(self):
85  if self.config.menu.run:
86  args = ['--runNumber', self.config.menu.run]
87  else:
88  args = ['--configName', self.config.menu.name ]
89  if not self.config.hilton:
90  # keep the original Source when running on Hilton
91  args.append('--noedsources')
92  for key, vals in self.options.items():
93  if vals:
94  args.extend(('--'+key, ','.join(vals)))
95  data, err = self.converter.query( *args )
96  if 'ERROR' in err or 'Exhausted Resultset' in err or 'CONFIG_NOT_FOUND' in err:
97  sys.stderr.write("%s: error while retrieving the HLT menu\n\n" % os.path.basename(sys.argv[0]))
98  sys.stderr.write(err + "\n\n")
99  sys.exit(1)
100  self.data = data
static std::string join(char **cmd)
Definition: RemoteFile.cc:19
def getRawConfigurationFromDB
Definition: confdb.py:83
def confdb.HLTProcess.getSetupConfigurationFromDB (   self)

Definition at line 60 of file confdb.py.

References join(), and python.rootplot.root2matplotlib.replace().

60 
62  if not self.config.setup:
63  return
64  ## if --setup is a python file, use directly that file as setup_cff.py
65  if ".py" in self.config.setup:
66  self.config.setupFile = self.config.setup.split(".py")[0]
67  return
68  args = ['--configName', self.config.setup ]
69  args.append('--noedsources')
70  args.append('--nopaths')
71  for key, vals in self.options.items():
72  if vals:
73  args.extend(('--'+key, ','.join(vals)))
74  args.append('--cff')
75  data, err = self.converter.query( *args )
76  if 'ERROR' in err or 'Exhausted Resultset' in err or 'CONFIG_NOT_FOUND' in err:
77  sys.stderr.write("%s: error while retrieving the HLT setup menu\n\n" % os.path.basename(sys.argv[0]))
78  sys.stderr.write(err + "\n\n")
79  sys.exit(1)
80  self.config.setupFile = "setup_"+self.config.setup[1:].replace("/","_")+"_cff"
81  outfile = open(self.config.setupFile+".py","w+")
82  outfile.write("# This file is automatically generated by hltGetConfiguration.\n" + data)
def getSetupConfigurationFromDB
Definition: confdb.py:60
static std::string join(char **cmd)
Definition: RemoteFile.cc:19
def confdb.HLTProcess.instrumentDQM (   self)

Definition at line 661 of file confdb.py.

References mat4.data, CombinedTauTagCalibration.data, CombinedSVCalibration.data, ecal::DigisCollection< StoragePolicy >.data, TrackProbabilityCalibration.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, ecal::raw::InputDataCPU.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, calo::multifit::MapSymM< T, Stride, Order >.data, KDTreeNodeInfo< DATA, DIM >.data, hcal::raw::InputDataCPU.data, LASModuleProfile.data, dqmTnP::AbstractFitter.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, calo::multifit::MapMForPM< T >.data, GeometryComparisonPlotter.data, edm::MapOfVectors< K, T >::Iter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, hcal::raw::InputDataGPU.data, @775.data, EcalShowerContainmentCorrections::Coefficients.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, edmStreamStallGrapher.Stack.data, confdb.HLTProcess.loadCffCommand(), and confdb.HLTProcess.removeElementFromSequencesTasksAndPaths().

Referenced by confdb.HLTProcess.customize().

662  def instrumentDQM(self):
663  if not self.config.hilton:
664  # remove any reference to the hltDQMFileSaver and hltDQMFileSaverPB:
665  # note the convert options remove the module itself,
666  # here we are just removing the references in paths, sequences, etc
667  self.removeElementFromSequencesTasksAndPaths('hltDQMFileSaverPB')
668  self.removeElementFromSequencesTasksAndPaths('hltDQMFileSaver')
669 
670  # instrument the HLT menu with DQMStore and DQMRootOutputModule suitable for running offline
671  dqmstore = "\n# load the DQMStore and DQMRootOutputModule\n"
672  dqmstore += self.loadCffCommand('DQMServices.Core.DQMStore_cfi')
673  dqmstore += """
674 %(process)s.dqmOutput = cms.OutputModule("DQMRootOutputModule",
675  fileName = cms.untracked.string("DQMIO.root")
676 )
677 """
678 
679  empty_path = re.compile(r'.*\b(process\.)?DQMOutput = cms\.EndPath\( *\).*')
680  other_path = re.compile(r'(.*\b(process\.)?DQMOutput = cms\.EndPath\()(.*)')
681  if empty_path.search(self.data):
682  # replace an empty DQMOutput path
683  self.data = empty_path.sub(dqmstore + '\n%(process)s.DQMOutput = cms.EndPath( %(process)s.dqmOutput )\n', self.data)
684  elif other_path.search(self.data):
685  # prepend the dqmOutput to the DQMOutput path
686  self.data = other_path.sub(dqmstore + r'\g<1> %(process)s.dqmOutput +\g<3>', self.data)
687  else:
688  # create a new DQMOutput path with the dqmOutput module
689  self.data += dqmstore
690  self.data += '\n%(process)s.DQMOutput = cms.EndPath( %(process)s.dqmOutput )\n'
691  self.data += '%(process)s.schedule.append( %(process)s.DQMOutput )\n'
692 
def removeElementFromSequencesTasksAndPaths
Definition: confdb.py:611
def loadCffCommand
Definition: confdb.py:592
def confdb.HLTProcess.instrumentErrorEventType (   self)

Definition at line 394 of file confdb.py.

References confdb.HLTProcess._fix_parameter().

Referenced by confdb.HLTProcess.customize().

395  def instrumentErrorEventType(self):
396  if self.config.errortype:
397  # change all HLTTriggerTypeFilter EDFilters to accept only error events (SelectedTriggerType = 0)
398  self._fix_parameter(name = 'SelectedTriggerType', type ='int32', value = '1', replace = '0')
399  self._fix_parameter(name = 'SelectedTriggerType', type ='int32', value = '2', replace = '0')
400  self._fix_parameter(name = 'SelectedTriggerType', type ='int32', value = '3', replace = '0')
401 
def _fix_parameter
Definition: confdb.py:337
def instrumentErrorEventType
Definition: confdb.py:394
def confdb.HLTProcess.instrumentOpenMode (   self)

Definition at line 380 of file confdb.py.

References mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, ecal::raw::InputDataCPU.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, calo::multifit::MapSymM< T, Stride, Order >.data, KDTreeNodeInfo< DATA, DIM >.data, hcal::raw::InputDataCPU.data, LASModuleProfile.data, dqmTnP::AbstractFitter.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, calo::multifit::MapMForPM< T >.data, GeometryComparisonPlotter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, edm::MapOfVectors< K, T >::Iter.data, hcal::raw::InputDataGPU.data, @775.data, EcalShowerContainmentCorrections::Coefficients.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, edmStreamStallGrapher.Stack.data, and join().

Referenced by confdb.HLTProcess.customize().

381  def instrumentOpenMode(self):
382  if self.config.open:
383  # find all EDfilters
384  filters = [ match[1] for match in re.findall(r'(process\.)?\b(\w+) = cms.EDFilter', self.data) ]
385  re_sequence = re.compile( r'cms\.(Path|Sequence)\((.*)\)' )
386  # remove existing 'cms.ignore' and '~' modifiers
387  self.data = re_sequence.sub( lambda line: re.sub( r'cms\.ignore *\( *((process\.)?\b(\w+)) *\)', r'\1', line.group(0) ), self.data )
388  self.data = re_sequence.sub( lambda line: re.sub( r'~', '', line.group(0) ), self.data )
389  # wrap all EDfilters with "cms.ignore( ... )", 1000 at a time (python 2.6 complains for too-big regular expressions)
390  for some in splitter(filters, 1000):
391  re_filters = re.compile( r'\b((process\.)?(' + r'|'.join(some) + r'))\b' )
392  self.data = re_sequence.sub( lambda line: re_filters.sub( r'cms.ignore( \1 )', line.group(0) ), self.data )
393 
static std::string join(char **cmd)
Definition: RemoteFile.cc:19
def instrumentOpenMode
Definition: confdb.py:380
def confdb.HLTProcess.instrumentTiming (   self)

Definition at line 619 of file confdb.py.

References mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, ecal::raw::InputDataCPU.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, calo::multifit::MapSymM< T, Stride, Order >.data, KDTreeNodeInfo< DATA, DIM >.data, hcal::raw::InputDataCPU.data, LASModuleProfile.data, dqmTnP::AbstractFitter.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, calo::multifit::MapMForPM< T >.data, GeometryComparisonPlotter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, edm::MapOfVectors< K, T >::Iter.data, hcal::raw::InputDataGPU.data, @775.data, EcalShowerContainmentCorrections::Coefficients.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, edmStreamStallGrapher.Stack.data, and confdb.HLTProcess.loadCff().

Referenced by confdb.HLTProcess.customize().

620  def instrumentTiming(self):
621 
622  if self.config.timing:
623  self.data += """
624 # instrument the menu with the modules and EndPath needed for timing studies
625 """
626 
627  self.data += '\n# configure the FastTimerService\n'
628  self.loadCff('HLTrigger.Timer.FastTimerService_cfi')
629 
630  self.data += """# print a text summary at the end of the job
631 %(process)s.FastTimerService.printEventSummary = False
632 %(process)s.FastTimerService.printRunSummary = False
633 %(process)s.FastTimerService.printJobSummary = True
634 
635 # enable DQM plots
636 %(process)s.FastTimerService.enableDQM = True
637 
638 # enable per-path DQM plots (starting with CMSSW 9.2.3-patch2)
639 %(process)s.FastTimerService.enableDQMbyPath = True
640 
641 # enable per-module DQM plots
642 %(process)s.FastTimerService.enableDQMbyModule = True
643 
644 # enable per-event DQM plots vs lumisection
645 %(process)s.FastTimerService.enableDQMbyLumiSection = True
646 %(process)s.FastTimerService.dqmLumiSectionsRange = 2500
647 
648 # set the time resolution of the DQM plots
649 %(process)s.FastTimerService.dqmTimeRange = 2000.
650 %(process)s.FastTimerService.dqmTimeResolution = 10.
651 %(process)s.FastTimerService.dqmPathTimeRange = 1000.
652 %(process)s.FastTimerService.dqmPathTimeResolution = 5.
653 %(process)s.FastTimerService.dqmModuleTimeRange = 200.
654 %(process)s.FastTimerService.dqmModuleTimeResolution = 1.
655 
656 # set the base DQM folder for the plots
657 %(process)s.FastTimerService.dqmPath = 'HLT/TimerService'
658 %(process)s.FastTimerService.enableDQMbyProcesses = False
659 """
660 
def instrumentTiming
Definition: confdb.py:619
def confdb.HLTProcess.loadAdditionalConditions (   self,
  comment,
  conditions 
)

Definition at line 575 of file confdb.py.

References mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, ecal::raw::InputDataCPU.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, KDTreeNodeInfo< DATA, DIM >.data, calo::multifit::MapSymM< T, Stride, Order >.data, hcal::raw::InputDataCPU.data, LASModuleProfile.data, dqmTnP::AbstractFitter.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, calo::multifit::MapMForPM< T >.data, GeometryComparisonPlotter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, edm::MapOfVectors< K, T >::Iter.data, hcal::raw::InputDataGPU.data, @775.data, EcalShowerContainmentCorrections::Coefficients.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, and edmStreamStallGrapher.Stack.data.

576  def loadAdditionalConditions(self, comment, *conditions):
577  # load additional conditions
578  self.data += """
579 # %s
580 if 'GlobalTag' in %%(dict)s:
581 """ % comment
582  for condition in conditions:
583  self.data += """ %%(process)s.GlobalTag.toGet.append(
584  cms.PSet(
585  record = cms.string( '%(record)s' ),
586  tag = cms.string( '%(tag)s' ),
587  label = cms.untracked.string( '%(label)s' ),
588  )
589  )
590 """ % condition
591 
def loadAdditionalConditions
Definition: confdb.py:575
def confdb.HLTProcess.loadCff (   self,
  module 
)

Definition at line 599 of file confdb.py.

References mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, ecal::raw::InputDataCPU.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, calo::multifit::MapSymM< T, Stride, Order >.data, KDTreeNodeInfo< DATA, DIM >.data, hcal::raw::InputDataCPU.data, LASModuleProfile.data, dqmTnP::AbstractFitter.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, calo::multifit::MapMForPM< T >.data, GeometryComparisonPlotter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, edm::MapOfVectors< K, T >::Iter.data, hcal::raw::InputDataGPU.data, @775.data, EcalShowerContainmentCorrections::Coefficients.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, edmStreamStallGrapher.Stack.data, and confdb.HLTProcess.loadCffCommand().

Referenced by confdb.HLTProcess.instrumentTiming().

600  def loadCff(self, module):
601  self.data += self.loadCffCommand(module)
602 
def loadCffCommand
Definition: confdb.py:592
def confdb.HLTProcess.loadCffCommand (   self,
  module 
)

Definition at line 592 of file confdb.py.

Referenced by confdb.HLTProcess.instrumentDQM(), and confdb.HLTProcess.loadCff().

593  def loadCffCommand(self, module):
594  # load a cfi or cff module
595  if self.config.fragment:
596  return 'from %s import *\n' % module
597  else:
598  return 'process.load( "%s" )\n' % module
def loadCffCommand
Definition: confdb.py:592
def confdb.HLTProcess.loadSetupCff (   self)

Definition at line 534 of file confdb.py.

References mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, ecal::raw::InputDataCPU.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, KDTreeNodeInfo< DATA, DIM >.data, calo::multifit::MapSymM< T, Stride, Order >.data, hcal::raw::InputDataCPU.data, LASModuleProfile.data, dqmTnP::AbstractFitter.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, calo::multifit::MapMForPM< T >.data, GeometryComparisonPlotter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, edm::MapOfVectors< K, T >::Iter.data, hcal::raw::InputDataGPU.data, @775.data, EcalShowerContainmentCorrections::Coefficients.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, and edmStreamStallGrapher.Stack.data.

Referenced by confdb.HLTProcess.customize().

535  def loadSetupCff(self):
536  if self.config.setup is None:
537  return
538  processLine = self.data.find("\n",self.data.find("cms.Process"))
539  self.data = self.data[:processLine]+'\nprocess.load("%s")'%self.config.setupFile+self.data[processLine:]
def confdb.HLTProcess.overrideGlobalTag (   self)

Definition at line 402 of file confdb.py.

References mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, ecal::raw::InputDataCPU.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, KDTreeNodeInfo< DATA, DIM >.data, calo::multifit::MapSymM< T, Stride, Order >.data, hcal::raw::InputDataCPU.data, LASModuleProfile.data, dqmTnP::AbstractFitter.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, calo::multifit::MapMForPM< T >.data, GeometryComparisonPlotter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, edm::MapOfVectors< K, T >::Iter.data, hcal::raw::InputDataGPU.data, @775.data, EcalShowerContainmentCorrections::Coefficients.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, and edmStreamStallGrapher.Stack.data.

Referenced by confdb.HLTProcess.customize().

403  def overrideGlobalTag(self):
404  # overwrite GlobalTag
405  # the logic is:
406  # - if a GlobalTag is specified on the command line:
407  # - override the global tag
408  # - if the GT is "auto:...", insert the code to read it from Configuration.AlCa.autoCond
409  # - if a GlobalTag is NOT specified on the command line:
410  # - when running on data, do nothing, and keep the global tag in the menu
411  # - when running on mc, take the GT from the configuration.type
412 
413  # override the GlobalTag connection string and pfnPrefix
414 
415  # when running on MC, override the global tag even if not specified on the command line
416  if not self.config.data and not self.config.globaltag:
417  if self.config.type in globalTag:
418  self.config.globaltag = globalTag[self.config.type]
419  else:
420  self.config.globaltag = globalTag['GRun']
421 
422  # if requested, override the L1 menu from the GlobalTag
423  if self.config.l1.override:
424  self.config.l1.tag = self.config.l1.override
425  self.config.l1.record = 'L1TUtmTriggerMenuRcd'
426  self.config.l1.connect = ''
427  self.config.l1.label = ''
428  if not self.config.l1.snapshotTime:
429  self.config.l1.snapshotTime = '9999-12-31 23:59:59.000'
430  self.config.l1cond = '%(tag)s,%(record)s,%(connect)s,%(label)s,%(snapshotTime)s' % self.config.l1.__dict__
431  else:
432  self.config.l1cond = None
433 
434  if self.config.globaltag or self.config.l1cond:
435  text = """
436 # override the GlobalTag, connection string and pfnPrefix
437 if 'GlobalTag' in %(dict)s:
438  from Configuration.AlCa.GlobalTag import GlobalTag as customiseGlobalTag
439  %(process)s.GlobalTag = customiseGlobalTag(%(process)s.GlobalTag"""
440  if self.config.globaltag:
441  text += ", globaltag = %s" % repr(self.config.globaltag)
442  if self.config.l1cond:
443  text += ", conditions = %s" % repr(self.config.l1cond)
444  text += ")\n"
445  self.data += text
def overrideGlobalTag
Definition: confdb.py:402
def confdb.HLTProcess.overrideL1MenuXml (   self)

Definition at line 446 of file confdb.py.

References mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, ecal::raw::InputDataCPU.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, KDTreeNodeInfo< DATA, DIM >.data, calo::multifit::MapSymM< T, Stride, Order >.data, hcal::raw::InputDataCPU.data, LASModuleProfile.data, dqmTnP::AbstractFitter.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, calo::multifit::MapMForPM< T >.data, GeometryComparisonPlotter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, edm::MapOfVectors< K, T >::Iter.data, hcal::raw::InputDataGPU.data, @775.data, EcalShowerContainmentCorrections::Coefficients.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, and edmStreamStallGrapher.Stack.data.

Referenced by confdb.HLTProcess.customize().

447  def overrideL1MenuXml(self):
448  # if requested, override the GlobalTag's L1T menu from an Xml file
449  if self.config.l1Xml.XmlFile:
450  text = """
451 # override the GlobalTag's L1T menu from an Xml file
452 from HLTrigger.Configuration.CustomConfigs import L1XML
453 %%(process)s = L1XML(%%(process)s,"%s")
454 """ % (self.config.l1Xml.XmlFile)
455  self.data += text
def overrideL1MenuXml
Definition: confdb.py:446
def confdb.HLTProcess.overrideOutput (   self)

Definition at line 466 of file confdb.py.

References mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, ecal::raw::InputDataCPU.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, KDTreeNodeInfo< DATA, DIM >.data, calo::multifit::MapSymM< T, Stride, Order >.data, hcal::raw::InputDataCPU.data, LASModuleProfile.data, dqmTnP::AbstractFitter.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, calo::multifit::MapMForPM< T >.data, GeometryComparisonPlotter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, edm::MapOfVectors< K, T >::Iter.data, hcal::raw::InputDataGPU.data, @775.data, EcalShowerContainmentCorrections::Coefficients.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, and edmStreamStallGrapher.Stack.data.

Referenced by confdb.HLTProcess.customize().

467  def overrideOutput(self):
468  # if not runnign on Hilton, override the "online" ShmStreamConsumer output modules with "offline" PoolOutputModule's
469  # note for Run3 ShmStreamConsumer has been replaced with EvFOutputModule
470  # so we also do a replace there
471  if not self.config.hilton:
472  self.data = re.sub(
473  r'\b(process\.)?hltOutput(\w+) *= *cms\.OutputModule\( *"(ShmStreamConsumer)" *,',
474  r'%(process)s.hltOutput\2 = cms.OutputModule( "PoolOutputModule",\n fileName = cms.untracked.string( "output\2.root" ),\n fastCloning = cms.untracked.bool( False ),\n dataset = cms.untracked.PSet(\n filterName = cms.untracked.string( "" ),\n dataTier = cms.untracked.string( "RAW" )\n ),',
475  self.data
476  )
477  self.data = re.sub(
478  r'\b(process\.)?hltOutput(\w+) *= *cms\.OutputModule\( *"EvFOutputModule" *,\n use_compression = cms.untracked.bool\( True \),\n compression_algorithm = cms.untracked.string\( "ZLIB" \),\n compression_level = cms.untracked.int32\( 1 \),\n lumiSection_interval = cms.untracked.int32\( 0 \),\n(.+?),\n psetMap = cms.untracked.InputTag\( "hltPSetMap" \)\n',
479  r'\1hltOutput\2 = cms.OutputModule( "PoolOutputModule",\n fileName = cms.untracked.string( "output\2.root" ),\n fastCloning = cms.untracked.bool( False ),\n dataset = cms.untracked.PSet(\n filterName = cms.untracked.string( "" ),\n dataTier = cms.untracked.string( "RAW" )\n ),\n\3\n',
480  self.data,0,re.DOTALL
481  )
482 
483  if not self.config.fragment and self.config.output == 'minimal':
484  # add a single output to keep the TriggerResults and TriggerEvent
485  self.data += """
486 # add a single "keep *" output
487 %(process)s.hltOutputMinimal = cms.OutputModule( "PoolOutputModule",
488  fileName = cms.untracked.string( "output.root" ),
489  fastCloning = cms.untracked.bool( False ),
490  dataset = cms.untracked.PSet(
491  dataTier = cms.untracked.string( 'AOD' ),
492  filterName = cms.untracked.string( '' )
493  ),
494  outputCommands = cms.untracked.vstring( 'drop *',
495  'keep edmTriggerResults_*_*_*',
496  'keep triggerTriggerEvent_*_*_*',
497  'keep GlobalAlgBlkBXVector_*_*_*',
498  'keep GlobalExtBlkBXVector_*_*_*',
499  'keep l1tEGammaBXVector_*_EGamma_*',
500  'keep l1tEtSumBXVector_*_EtSum_*',
501  'keep l1tJetBXVector_*_Jet_*',
502  'keep l1tMuonBXVector_*_Muon_*',
503  'keep l1tTauBXVector_*_Tau_*',
504  )
505 )
506 %(process)s.MinimalOutput = cms.EndPath( %(process)s.hltOutputMinimal )
507 %(process)s.schedule.append( %(process)s.MinimalOutput )
508 """
509  elif not self.config.fragment and self.config.output == 'full':
510  # add a single "keep *" output
511  self.data += """
512 # add a single "keep *" output
513 %(process)s.hltOutputFull = cms.OutputModule( "PoolOutputModule",
514  fileName = cms.untracked.string( "output.root" ),
515  fastCloning = cms.untracked.bool( False ),
516  dataset = cms.untracked.PSet(
517  dataTier = cms.untracked.string( 'RECO' ),
518  filterName = cms.untracked.string( '' )
519  ),
520  outputCommands = cms.untracked.vstring( 'keep *' )
521 )
522 %(process)s.FullOutput = cms.EndPath( %(process)s.hltOutputFull )
523 %(process)s.schedule.append( %(process)s.FullOutput )
524 """
def overrideOutput
Definition: confdb.py:466
def confdb.HLTProcess.overrideParameters (   self,
  module,
  parameters 
)

Definition at line 603 of file confdb.py.

References mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, ecal::raw::InputDataCPU.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, KDTreeNodeInfo< DATA, DIM >.data, calo::multifit::MapSymM< T, Stride, Order >.data, hcal::raw::InputDataCPU.data, LASModuleProfile.data, dqmTnP::AbstractFitter.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, calo::multifit::MapMForPM< T >.data, GeometryComparisonPlotter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, edm::MapOfVectors< K, T >::Iter.data, hcal::raw::InputDataGPU.data, @775.data, EcalShowerContainmentCorrections::Coefficients.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, and edmStreamStallGrapher.Stack.data.

604  def overrideParameters(self, module, parameters):
605  # override a module's parameter if the module is present in the configuration
606  self.data += "if '%s' in %%(dict)s:\n" % module
607  for (parameter, value) in parameters:
608  self.data += " %%(process)s.%s.%s = %s\n" % (module, parameter, value)
609  self.data += "\n"
610 
def overrideParameters
Definition: confdb.py:603
def confdb.HLTProcess.overrideProcessName (   self)

Definition at line 541 of file confdb.py.

References mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, ecal::raw::InputDataCPU.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, KDTreeNodeInfo< DATA, DIM >.data, calo::multifit::MapSymM< T, Stride, Order >.data, hcal::raw::InputDataCPU.data, LASModuleProfile.data, dqmTnP::AbstractFitter.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, calo::multifit::MapMForPM< T >.data, GeometryComparisonPlotter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, edm::MapOfVectors< K, T >::Iter.data, hcal::raw::InputDataGPU.data, @775.data, EcalShowerContainmentCorrections::Coefficients.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, and edmStreamStallGrapher.Stack.data.

Referenced by confdb.HLTProcess.customize().

542  def overrideProcessName(self):
543  if self.config.name is None:
544  return
545 
546  # sanitise process name
547  self.config.name = self.config.name.replace("_","")
548  # override the process name
549  quote = '[\'\"]'
550  self.data = re.compile(r'^(process\s*=\s*cms\.Process\(\s*' + quote + r')\w+(' + quote + r'\s*\).*)$', re.MULTILINE).sub(r'\1%s\2' % self.config.name, self.data, 1)
551 
552  # when --setup option is used, remove possible errors from PrescaleService due to missing HLT paths.
553  if self.config.setup: self.data += """
554 # avoid PrescaleService error due to missing HLT paths
555 if 'PrescaleService' in process.__dict__:
556  for pset in reversed(process.PrescaleService.prescaleTable):
557  if not hasattr(process,pset.pathName.value()):
558  process.PrescaleService.prescaleTable.remove(pset)
559 """
560 
def overrideProcessName
Definition: confdb.py:541
def confdb.HLTProcess.removeElementFromSequencesTasksAndPaths (   self,
  label 
)

Definition at line 611 of file confdb.py.

References mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, ecal::raw::InputDataCPU.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, KDTreeNodeInfo< DATA, DIM >.data, calo::multifit::MapSymM< T, Stride, Order >.data, hcal::raw::InputDataCPU.data, LASModuleProfile.data, dqmTnP::AbstractFitter.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, calo::multifit::MapMForPM< T >.data, GeometryComparisonPlotter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, edm::MapOfVectors< K, T >::Iter.data, hcal::raw::InputDataGPU.data, @775.data, EcalShowerContainmentCorrections::Coefficients.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, and edmStreamStallGrapher.Stack.data.

Referenced by confdb.HLTProcess.instrumentDQM().

613  if label in self.data:
614  label_re = r'\b(process\.)?' + label
615  self.data = re.sub(r' *(\+|,) *' + label_re, '', self.data)
616  self.data = re.sub(label_re + r' *(\+|,) *', '', self.data)
617  self.data = re.sub(label_re, '', self.data)
618 
def removeElementFromSequencesTasksAndPaths
Definition: confdb.py:611
def confdb.HLTProcess.runL1Emulator (   self)

Definition at line 456 of file confdb.py.

References mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, ecal::raw::InputDataCPU.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, KDTreeNodeInfo< DATA, DIM >.data, calo::multifit::MapSymM< T, Stride, Order >.data, hcal::raw::InputDataCPU.data, LASModuleProfile.data, dqmTnP::AbstractFitter.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, calo::multifit::MapMForPM< T >.data, GeometryComparisonPlotter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, edm::MapOfVectors< K, T >::Iter.data, hcal::raw::InputDataGPU.data, @775.data, EcalShowerContainmentCorrections::Coefficients.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, and edmStreamStallGrapher.Stack.data.

Referenced by confdb.HLTProcess.customize().

457  def runL1Emulator(self):
458  # if requested, run the Full L1T emulator, then repack the data into a new RAW collection, to be used by the HLT
459  if self.config.emulator:
460  text = """
461 # run the Full L1T emulator, then repack the data into a new RAW collection, to be used by the HLT
462 from HLTrigger.Configuration.CustomConfigs import L1REPACK
463 %%(process)s = L1REPACK(%%(process)s,"%s")
464 """ % (self.config.emulator)
465  self.data += text
def confdb.HLTProcess.specificCustomize (   self)

Definition at line 176 of file confdb.py.

References mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, ecal::raw::InputDataCPU.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, KDTreeNodeInfo< DATA, DIM >.data, calo::multifit::MapSymM< T, Stride, Order >.data, hcal::raw::InputDataCPU.data, LASModuleProfile.data, dqmTnP::AbstractFitter.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, calo::multifit::MapMForPM< T >.data, GeometryComparisonPlotter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, edm::MapOfVectors< K, T >::Iter.data, hcal::raw::InputDataGPU.data, @775.data, EcalShowerContainmentCorrections::Coefficients.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, edmStreamStallGrapher.Stack.data, python.rootplot.root2matplotlib.replace(), confdb.HLTProcess.source, lhef::XMLDocument.source, DQMNet::Peer.source, Config.Process.source, gen::Hydjet2Hadronizer.source, and BeautifulSoup.ResultSet.source.

Referenced by confdb.HLTProcess.customize().

177  def specificCustomize(self):
178  # specific customizations now live in HLTrigger.Configuration.customizeHLTforALL.customizeHLTforAll(.,.)
179  if self.config.fragment:
180  self.data += """
181 # add specific customizations
182 from HLTrigger.Configuration.customizeHLTforALL import customizeHLTforAll
183 fragment = customizeHLTforAll(fragment,"%s")
184 """ % (self.config.type)
185  elif self.config.hilton:
186  # do not apply the STORM-specific customisation
187  pass
188  else:
189  if self.config.type=="Fake":
190  prefix = "run1"
191  elif self.config.type in ("Fake1","Fake2","2018"):
192  prefix = "run2"
193  else:
194  prefix = "run3"
195  _gtData = "auto:"+prefix+"_hlt_"+self.config.type
196  _gtMc = "auto:"+prefix+"_mc_" +self.config.type
197  self.data += """
198 # add specific customizations
199 _customInfo = {}
200 _customInfo['menuType' ]= "%s"
201 _customInfo['globalTags']= {}
202 _customInfo['globalTags'][True ] = "%s"
203 _customInfo['globalTags'][False] = "%s"
204 _customInfo['inputFiles']={}
205 _customInfo['inputFiles'][True] = "file:RelVal_Raw_%s_DATA.root"
206 _customInfo['inputFiles'][False] = "file:RelVal_Raw_%s_MC.root"
207 _customInfo['maxEvents' ]= %s
208 _customInfo['globalTag' ]= "%s"
209 _customInfo['inputFile' ]= %s
210 _customInfo['realData' ]= %s
211 
212 from HLTrigger.Configuration.customizeHLTforALL import customizeHLTforAll
213 %%(process)s = customizeHLTforAll(%%(process)s,"%s",_customInfo)
214 """ % (self.config.type,_gtData,_gtMc,self.config.type,self.config.type,self.config.events,self.config.globaltag,self.source,self.config.data,self.config.type)
215 
216  self.data += """
217 from HLTrigger.Configuration.customizeHLTforCMSSW import customizeHLTforCMSSW
218 %%(process)s = customizeHLTforCMSSW(%%(process)s,"%s")
219 """ % (self.config.type)
220 
221  # Eras-based customisations
222  self.data += """
223 # Eras-based customisations
224 from HLTrigger.Configuration.Eras import modifyHLTforEras
225 modifyHLTforEras(%(process)s)
226 """
227  # add the user-defined customization functions, if any
228  if self.config.customise:
229  self.data += "\n"
230  self.data += "#User-defined customization functions\n"
231  for customise in self.config.customise.split(","):
232  customiseValues = customise.split(".")
233  if len(customiseValues)>=3: raise Exception("--customise option cannot contain more than one dot.")
234  if len(customiseValues)==1:
235  customiseValues.append("customise")
236  customiseValues[0] = customiseValues[0].replace("/",".")
237  self.data += "from "+customiseValues[0]+" import "+customiseValues[1]+"\n"
238  self.data += "process = "+customiseValues[1]+"(process)\n"
239 
def specificCustomize
Definition: confdb.py:176
def confdb.HLTProcess.updateMessageLogger (   self)

Definition at line 561 of file confdb.py.

References mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, ecal::raw::InputDataCPU.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, KDTreeNodeInfo< DATA, DIM >.data, calo::multifit::MapSymM< T, Stride, Order >.data, hcal::raw::InputDataCPU.data, LASModuleProfile.data, dqmTnP::AbstractFitter.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, calo::multifit::MapMForPM< T >.data, GeometryComparisonPlotter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, edm::MapOfVectors< K, T >::Iter.data, hcal::raw::InputDataGPU.data, @775.data, EcalShowerContainmentCorrections::Coefficients.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, and edmStreamStallGrapher.Stack.data.

Referenced by confdb.HLTProcess.customize().

562  def updateMessageLogger(self):
563  # request summary informations from the MessageLogger
564  self.data += """
565 # show summaries from trigger analysers used at HLT
566 if 'MessageLogger' in %(dict)s:
567  %(process)s.MessageLogger.TriggerSummaryProducerAOD = cms.untracked.PSet()
568  %(process)s.MessageLogger.L1GtTrigReport = cms.untracked.PSet()
569  %(process)s.MessageLogger.L1TGlobalSummary = cms.untracked.PSet()
570  %(process)s.MessageLogger.HLTrigReport = cms.untracked.PSet()
571  %(process)s.MessageLogger.FastReport = cms.untracked.PSet()
572  %(process)s.MessageLogger.ThroughputService = cms.untracked.PSet()
573 """
574 
def updateMessageLogger
Definition: confdb.py:561

Member Data Documentation

confdb.HLTProcess.all_paths

Definition at line 701 of file confdb.py.

Referenced by confdb.HLTProcess.customize(), and confdb.HLTProcess.fixPrescales().

confdb.HLTProcess.config

Definition at line 24 of file confdb.py.

Referenced by zMuMuValidation.ZMuMuValidation.trackcollection().

confdb.HLTProcess.converter

Definition at line 53 of file confdb.py.

confdb.HLTProcess.data

Definition at line 25 of file confdb.py.

Referenced by data_sources.node.__str__(), confdb.HLTProcess._fix_parameter(), confdb.HLTProcess.addEras(), confdb.HLTProcess.addGlobalOptions(), confdb.HLTProcess.append_filenames(), data_sources.json_list.as_dicts(), data_sources.json_list.as_table(), confdb.HLTProcess.build_source(), confdb.HLTProcess.customize(), confdb.HLTProcess.dump(), confdb.HLTProcess.fixPrescales(), data_sources.json_list.get_members(), confdb.HLTProcess.getRawConfigurationFromDB(), confdb.HLTProcess.instrumentDQM(), confdb.HLTProcess.instrumentOpenMode(), confdb.HLTProcess.instrumentTiming(), data_sources.json_list.last(), confdb.HLTProcess.loadAdditionalConditions(), confdb.HLTProcess.loadCff(), confdb.HLTProcess.loadSetupCff(), confdb.HLTProcess.overrideGlobalTag(), confdb.HLTProcess.overrideL1MenuXml(), confdb.HLTProcess.overrideOutput(), confdb.HLTProcess.overrideParameters(), confdb.HLTProcess.overrideProcessName(), confdb.HLTProcess.removeElementFromSequencesTasksAndPaths(), confdb.HLTProcess.runL1Emulator(), confdb.HLTProcess.specificCustomize(), cuy.FindIssue.startElement(), and confdb.HLTProcess.updateMessageLogger().

confdb.HLTProcess.labels

Definition at line 40 of file confdb.py.

Referenced by confdb.HLTProcess.dump().

confdb.HLTProcess.options

Definition at line 29 of file confdb.py.

Referenced by betterConfigParser.BetterConfigParser.__updateDict(), submitPVValidationJobs.BetterConfigParser.__updateDict(), confdb.HLTProcess.buildOptions(), betterConfigParser.BetterConfigParser.checkInput(), CrabHelper.CrabHelper.crab_config_filename(), CrabHelper.CrabHelper.crab_taskname(), CrabHelper.CrabHelper.fill_options_from_crab_config(), confdb.HLTProcess.fixPrescales(), edmIntegrityCheck.IntegrityCheck.query(), cmsswPreprocessor.CmsswPreprocessor.run(), production_tasks.BaseDataset.run(), production_tasks.GenerateMask.run(), production_tasks.RunCMSBatch.run(), production_tasks.MonitorJobs.run(), production_tasks.CheckJobStatus.run(), and production_tasks.CleanJobFiles.run().

confdb.HLTProcess.parent

Definition at line 27 of file confdb.py.

Referenced by BeautifulSoup.PageElement._invert(), confdb.HLTProcess.build_source(), edmIntegrityCheck.PublishToFileSystem.get(), and edmIntegrityCheck.PublishToFileSystem.publish().

confdb.HLTProcess.source

Definition at line 26 of file confdb.py.

Referenced by confdb.HLTProcess.build_source(), and confdb.HLTProcess.specificCustomize().