CMS 3D CMS Logo
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Properties Friends Macros Groups Pages
List of all members | Public Member Functions | Static Public Member Functions | Public Attributes | Private Member Functions
confdb.HLTProcess Class Reference
Inheritance diagram for confdb.HLTProcess:

Public Member Functions

def __init__
 
def addEras
 
def addGlobalOptions
 
def append_filenames
 
def build_source
 
def buildOptions
 
def buildPathList
 
def customize
 
def dump
 
def expand_filenames
 
def fixPrescales
 
def getPathList
 
def getRawConfigurationFromDB
 
def getSetupConfigurationFromDB
 
def instrumentDQM
 
def instrumentErrorEventType
 
def instrumentOpenMode
 
def instrumentTiming
 
def loadAdditionalConditions
 
def loadCff
 
def loadCffCommand
 
def loadSetupCff
 
def overrideGlobalTag
 
def overrideL1MenuXml
 
def overrideOutput
 
def overrideParameters
 
def overrideProcessName
 
def removeElementFromSequencesTasksAndPaths
 
def runL1Emulator
 
def specificCustomize
 
def updateMessageLogger
 

Static Public Member Functions

def consolidateNegativeList
 
def consolidatePositiveList
 
def dumppaths
 
def expandWildcards
 

Public Attributes

 all_paths
 
 config
 
 converter
 
 data
 
 labels
 
 options
 
 parent
 
 source
 

Private Member Functions

def _fix_parameter
 

Detailed Description

Definition at line 21 of file confdb.py.

Constructor & Destructor Documentation

def confdb.HLTProcess.__init__ (   self,
  configuration 
)

Definition at line 23 of file confdb.py.

23 
24  def __init__(self, configuration):
25  self.config = configuration
26  self.data = None
27  self.source = []
28  self.parent = []
29 
30  self.options = {
31  'essources' : [],
32  'esmodules' : [],
33  'modules' : [],
34  'sequences' : [],
35  'services' : [],
36  'paths' : [],
37  'psets' : [],
38  'blocks' : [],
39  }
40 
41  self.labels = {}
42  if self.config.fragment:
43  self.labels['process'] = 'fragment'
44  self.labels['dict'] = 'fragment.__dict__'
45  else:
46  self.labels['process'] = 'process'
47  self.labels['dict'] = 'process.__dict__'
48 
49  if self.config.prescale and (self.config.prescale.lower() != 'none'):
50  self.labels['prescale'] = self.config.prescale
51 
52  # get the configuration from ConfdB
53  from .confdbOfflineConverter import OfflineConverter
54  self.converter = OfflineConverter(version = self.config.menu.version, database = self.config.menu.database, proxy = self.config.proxy, proxyHost = self.config.proxy_host, proxyPort = self.config.proxy_port)
55  self.buildPathList()
56  self.buildOptions()
57  self.getSetupConfigurationFromDB()
58  self.getRawConfigurationFromDB()
59  self.customize()
confdb.HLTProcess.getSetupConfigurationFromDB
def getSetupConfigurationFromDB
Definition: confdb.py:60
confdb.HLTProcess.getRawConfigurationFromDB
def getRawConfigurationFromDB
Definition: confdb.py:83

Member Function Documentation

def confdb.HLTProcess._fix_parameter (   self,
  args 
)
private 
arguments:
name:     parameter name (optional)
type:     parameter type (look for tracked and untracked variants)
value:    original value
replace:  replacement value

Definition at line 343 of file confdb.py.

References mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, ecal::raw::InputDataCPU.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, KDTreeNodeInfo< DATA, DIM >.data, calo::multifit::MapSymM< T, Stride, Order >.data, hcal::raw::InputDataCPU.data, LASModuleProfile.data, dqmTnP::AbstractFitter.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, calo::multifit::MapMForPM< T >.data, GeometryComparisonPlotter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, edm::MapOfVectors< K, T >::Iter.data, hcal::raw::InputDataGPU.data, @775.data, EcalShowerContainmentCorrections::Coefficients.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, and edmStreamStallGrapher.Stack.data.

Referenced by confdb.HLTProcess.customize(), and confdb.HLTProcess.instrumentErrorEventType().

343 
344  def _fix_parameter(self, **args):
345  """arguments:
346  name: parameter name (optional)
347  type: parameter type (look for tracked and untracked variants)
348  value: original value
349  replace: replacement value
350  """
351  if 'name' in args:
352  self.data = re.sub(
353  r'%(name)s = cms(?P<tracked>(?:\.untracked)?)\.%(type)s\( (?P<quote>["\']?)%(value)s(?P=quote)' % args,
354  r'%(name)s = cms\g<tracked>.%(type)s( \g<quote>%(replace)s\g<quote>' % args,
355  self.data)
356  else:
357  self.data = re.sub(
358  r'cms(?P<tracked>(?:\.untracked)?)\.%(type)s\( (?P<quote>["\']?)%(value)s(?P=quote)' % args,
359  r'cms\g<tracked>.%(type)s( \g<quote>%(replace)s\g<quote>' % args,
360  self.data)
361 
confdb.HLTProcess._fix_parameter
def _fix_parameter
Definition: confdb.py:343
def confdb.HLTProcess.addEras (   self)

Definition at line 536 of file confdb.py.

References mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, ecal::raw::InputDataCPU.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, calo::multifit::MapSymM< T, Stride, Order >.data, KDTreeNodeInfo< DATA, DIM >.data, hcal::raw::InputDataCPU.data, LASModuleProfile.data, dqmTnP::AbstractFitter.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, calo::multifit::MapMForPM< T >.data, GeometryComparisonPlotter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, edm::MapOfVectors< K, T >::Iter.data, hcal::raw::InputDataGPU.data, @775.data, EcalShowerContainmentCorrections::Coefficients.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, edmStreamStallGrapher.Stack.data, and join().

Referenced by confdb.HLTProcess.customize().

536 
537  def addEras(self):
538  if self.config.eras is None:
539  return
540  from Configuration.StandardSequences.Eras import eras
541  erasSplit = self.config.eras.split(',')
542  self.data = re.sub(r'process = cms.Process\( *"\w+"', '\n'.join(eras.pythonCfgLines[era] for era in erasSplit)+'\n\g<0>, '+', '.join(era for era in erasSplit), self.data)
join
static std::string join(char **cmd)
Definition: RemoteFile.cc:19
def confdb.HLTProcess.addGlobalOptions (   self)

Definition at line 325 of file confdb.py.

References mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, ecal::raw::InputDataCPU.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, KDTreeNodeInfo< DATA, DIM >.data, calo::multifit::MapSymM< T, Stride, Order >.data, hcal::raw::InputDataCPU.data, LASModuleProfile.data, dqmTnP::AbstractFitter.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, calo::multifit::MapMForPM< T >.data, GeometryComparisonPlotter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, edm::MapOfVectors< K, T >::Iter.data, hcal::raw::InputDataGPU.data, @775.data, EcalShowerContainmentCorrections::Coefficients.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, and edmStreamStallGrapher.Stack.data.

Referenced by confdb.HLTProcess.customize().

325 
326  def addGlobalOptions(self):
327  # add global options
328  self.data += """
329 # limit the number of events to be processed
330 %%(process)s.maxEvents = cms.untracked.PSet(
331  input = cms.untracked.int32( %d )
332 )
333 """ % self.config.events
334 
335  self.data += """
336 # enable TrigReport, TimeReport and MultiThreading
337 %(process)s.options = cms.untracked.PSet(
338  wantSummary = cms.untracked.bool( True ),
339  numberOfThreads = cms.untracked.uint32( 4 ),
340  numberOfStreams = cms.untracked.uint32( 0 ),
341 )
342 """
confdb.HLTProcess.addGlobalOptions
def addGlobalOptions
Definition: confdb.py:325
def confdb.HLTProcess.append_filenames (   self,
  name,
  filenames 
)

Definition at line 872 of file confdb.py.

References mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, ecal::raw::InputDataCPU.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, KDTreeNodeInfo< DATA, DIM >.data, calo::multifit::MapSymM< T, Stride, Order >.data, hcal::raw::InputDataCPU.data, LASModuleProfile.data, dqmTnP::AbstractFitter.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, calo::multifit::MapMForPM< T >.data, GeometryComparisonPlotter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, edm::MapOfVectors< K, T >::Iter.data, hcal::raw::InputDataGPU.data, @775.data, EcalShowerContainmentCorrections::Coefficients.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, and edmStreamStallGrapher.Stack.data.

Referenced by confdb.HLTProcess.build_source().

872 
873  def append_filenames(self, name, filenames):
874  if len(filenames) > 255:
875  token_open = "( *("
876  token_close = ") )"
877  else:
878  token_open = "("
879  token_close = ")"
880 
881  self.data += " %s = cms.untracked.vstring%s\n" % (name, token_open)
882  for line in filenames:
883  self.data += " '%s',\n" % line
884  self.data += " %s,\n" % (token_close)
885 
confdb.HLTProcess.append_filenames
def append_filenames
Definition: confdb.py:872
def confdb.HLTProcess.build_source (   self)

Definition at line 898 of file confdb.py.

References confdb.HLTProcess.append_filenames(), mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, ecal::raw::InputDataCPU.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, KDTreeNodeInfo< DATA, DIM >.data, calo::multifit::MapSymM< T, Stride, Order >.data, hcal::raw::InputDataCPU.data, dqmTnP::AbstractFitter.data, LASModuleProfile.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, GeometryComparisonPlotter.data, calo::multifit::MapMForPM< T >.data, edm::MapOfVectors< K, T >::Iter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, hcal::raw::InputDataGPU.data, EcalShowerContainmentCorrections::Coefficients.data, @775.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, edmStreamStallGrapher.Stack.data, confdb.HLTProcess.expand_filenames(), confdb.HLTProcess.parent, FWPSetTableManager::PSetData.parent, SlimmedCount.parent, dqm::implementation::IBooker::UseScope< SCOPE >.parent, confdb.HLTProcess.source, lhef::XMLDocument.source, DQMNet::Peer.source, Config.Process.source, gen::Hydjet2Hadronizer.source, and BeautifulSoup.ResultSet.source.

Referenced by confdb.HLTProcess.customize().

898 
899  def build_source(self):
900  if self.config.hilton:
901  # use the DAQ source
902  return
903 
904  if self.config.input:
905  # if a dataset or a list of input files was given, use it
906  self.source = self.expand_filenames(self.config.input)
907  elif self.config.data:
908  # offline we can run on data...
909  self.source = [ "file:RelVal_Raw_%s_DATA.root" % self.config.type ]
910  else:
911  # ...or on mc
912  self.source = [ "file:RelVal_Raw_%s_MC.root" % self.config.type ]
913 
914  if self.config.parent:
915  # if a dataset or a list of input files was given for the parent data, use it
916  self.parent = self.expand_filenames(self.config.parent)
917 
918  self.data += """
919 # source module (EDM inputs)
920 %(process)s.source = cms.Source( "PoolSource",
921 """
922  self.append_filenames("fileNames", self.source)
923  if (self.parent):
924  self.append_filenames("secondaryFileNames", self.parent)
925  self.data += """\
926  inputCommands = cms.untracked.vstring(
927  'keep *'
928  )
929 )
930 """
confdb.HLTProcess.expand_filenames
def expand_filenames
Definition: confdb.py:886
confdb.HLTProcess.append_filenames
def append_filenames
Definition: confdb.py:872
def confdb.HLTProcess.buildOptions (   self)

Definition at line 774 of file confdb.py.

References bitset_utilities.append(), DTCalibrationWorker.DTCalibrationWorker.options, DTWorkflow.DTWorkflow.options, TestProcess.TestProcess.options, confdb.HLTProcess.options, Config.Process.options, and validateAlignments.ValidationJobMultiIOV.options.

774 
775  def buildOptions(self):
776  # common configuration for all scenarios
777  self.options['services'].append( "-DQM" )
778  self.options['services'].append( "-FUShmDQMOutputService" )
779  self.options['services'].append( "-MicroStateService" )
780  self.options['services'].append( "-ModuleWebRegistry" )
781  self.options['services'].append( "-TimeProfilerService" )
782 
783  # remove the DAQ modules and the online definition of the DQMStore and DQMFileSaver
784  # unless a hilton-like configuration has been requested
785  if not self.config.hilton:
786  self.options['services'].append( "-EvFDaqDirector" )
787  self.options['services'].append( "-FastMonitoringService" )
788  self.options['services'].append( "-DQMStore" )
789  self.options['modules'].append( "-hltDQMFileSaver" )
790  self.options['modules'].append( "-hltDQMFileSaverPB" )
791 
792  if self.config.fragment:
793  # extract a configuration file fragment
794  self.options['essources'].append( "-GlobalTag" )
795  self.options['essources'].append( "-HepPDTESSource" )
796  self.options['essources'].append( "-XMLIdealGeometryESSource" )
797  self.options['essources'].append( "-eegeom" )
798  self.options['essources'].append( "-es_hardcode" )
799  self.options['essources'].append( "-magfield" )
800 
801  self.options['esmodules'].append( "-SlaveField0" )
802  self.options['esmodules'].append( "-SlaveField20" )
803  self.options['esmodules'].append( "-SlaveField30" )
804  self.options['esmodules'].append( "-SlaveField35" )
805  self.options['esmodules'].append( "-SlaveField38" )
806  self.options['esmodules'].append( "-SlaveField40" )
807  self.options['esmodules'].append( "-VBF0" )
808  self.options['esmodules'].append( "-VBF20" )
809  self.options['esmodules'].append( "-VBF30" )
810  self.options['esmodules'].append( "-VBF35" )
811  self.options['esmodules'].append( "-VBF38" )
812  self.options['esmodules'].append( "-VBF40" )
813  self.options['esmodules'].append( "-CSCGeometryESModule" )
814  self.options['esmodules'].append( "-CaloGeometryBuilder" )
815  self.options['esmodules'].append( "-CaloTowerHardcodeGeometryEP" )
816  self.options['esmodules'].append( "-CastorHardcodeGeometryEP" )
817  self.options['esmodules'].append( "-DTGeometryESModule" )
818  self.options['esmodules'].append( "-EcalBarrelGeometryEP" )
819  self.options['esmodules'].append( "-EcalElectronicsMappingBuilder" )
820  self.options['esmodules'].append( "-EcalEndcapGeometryEP" )
821  self.options['esmodules'].append( "-EcalLaserCorrectionService" )
822  self.options['esmodules'].append( "-EcalPreshowerGeometryEP" )
823  self.options['esmodules'].append( "-GEMGeometryESModule" )
824  self.options['esmodules'].append( "-HcalHardcodeGeometryEP" )
825  self.options['esmodules'].append( "-HcalTopologyIdealEP" )
826  self.options['esmodules'].append( "-MuonNumberingInitialization" )
827  self.options['esmodules'].append( "-ParametrizedMagneticFieldProducer" )
828  self.options['esmodules'].append( "-RPCGeometryESModule" )
829  self.options['esmodules'].append( "-SiStripGainESProducer" )
830  self.options['esmodules'].append( "-SiStripRecHitMatcherESProducer" )
831  self.options['esmodules'].append( "-SiStripQualityESProducer" )
832  self.options['esmodules'].append( "-StripCPEfromTrackAngleESProducer" )
833  self.options['esmodules'].append( "-TrackerAdditionalParametersPerDetESModule" )
834  self.options['esmodules'].append( "-TrackerDigiGeometryESModule" )
835  self.options['esmodules'].append( "-TrackerGeometricDetESModule" )
836  self.options['esmodules'].append( "-VolumeBasedMagneticFieldESProducer" )
837  self.options['esmodules'].append( "-ZdcHardcodeGeometryEP" )
838  self.options['esmodules'].append( "-hcal_db_producer" )
839  self.options['esmodules'].append( "-L1GtTriggerMaskAlgoTrigTrivialProducer" )
840  self.options['esmodules'].append( "-L1GtTriggerMaskTechTrigTrivialProducer" )
841  self.options['esmodules'].append( "-hltESPEcalTrigTowerConstituentsMapBuilder" )
842  self.options['esmodules'].append( "-hltESPGlobalTrackingGeometryESProducer" )
843  self.options['esmodules'].append( "-hltESPMuonDetLayerGeometryESProducer" )
844  self.options['esmodules'].append( "-hltESPTrackerRecoGeometryESProducer" )
845  self.options['esmodules'].append( "-trackerTopology" )
846 
847  self.options['esmodules'].append( "-CaloTowerGeometryFromDBEP" )
848  self.options['esmodules'].append( "-CastorGeometryFromDBEP" )
849  self.options['esmodules'].append( "-EcalBarrelGeometryFromDBEP" )
850  self.options['esmodules'].append( "-EcalEndcapGeometryFromDBEP" )
851  self.options['esmodules'].append( "-EcalPreshowerGeometryFromDBEP" )
852  self.options['esmodules'].append( "-HcalGeometryFromDBEP" )
853  self.options['esmodules'].append( "-ZdcGeometryFromDBEP" )
854  self.options['esmodules'].append( "-XMLFromDBSource" )
855  self.options['esmodules'].append( "-sistripconn" )
856 
857  self.options['services'].append( "-MessageLogger" )
858 
859  self.options['psets'].append( "-maxEvents" )
860  self.options['psets'].append( "-options" )
861 
862  # remove Scouting OutputModules even though the EndPaths are kept
863  self.options['modules'].append( "-hltOutputScoutingCaloMuon" )
864  self.options['modules'].append( "-hltOutputScoutingPF" )
865 
866  if self.config.fragment or (self.config.prescale and (self.config.prescale.lower() == 'none')):
867  self.options['services'].append( "-PrescaleService" )
868 
869  if self.config.fragment or self.config.timing:
870  self.options['services'].append( "-FastTimerService" )
871 
bitset_utilities::append
boost::dynamic_bitset append(const boost::dynamic_bitset<> &bs1, const boost::dynamic_bitset<> &bs2)
this method takes two bitsets bs1 and bs2 and returns result of bs2 appended to the end of bs1 ...
Definition: bitset_append.cc:10
def confdb.HLTProcess.buildPathList (   self)

Definition at line 709 of file confdb.py.

709 
710  def buildPathList(self):
711  self.all_paths = self.getPathList()
712 
713  if self.config.paths:
714  # no path list was requested, dump the full table, minus unsupported / unwanted paths
715  paths = self.config.paths.split(',')
716  else:
717  # dump only the requested paths, plus the eventual output endpaths
718  paths = []
719 
720  # 'none' should remove all outputs
721  # 'dqm' should remove all outputs but DQMHistograms
722  # 'minimal' should remove all outputs but DQMHistograms, and add a single output module to keep the TriggerResults and TriggerEvent
723  # 'full' should remove all outputs but DQMHistograms, and add a single output module to "keep *"
724  # See also the `overrideOutput` method
725  if self.config.fragment or self.config.output in ('none', ):
726  if self.config.paths:
727  # keep only the Paths and EndPaths requested explicitly
728  pass
729  else:
730  # drop all output EndPaths but the Scouting ones, and drop the RatesMonitoring and DQMHistograms
731  paths.append( "-*Output" )
732  paths.append( "-RatesMonitoring")
733  paths.append( "-DQMHistograms")
734  if self.config.fragment: paths.append( "Scouting*Output" )
735 
736  elif self.config.output in ('dqm', 'minimal', 'full'):
737  if self.config.paths:
738  # keep only the Paths and EndPaths requested explicitly, and the DQMHistograms
739  paths.append( "DQMHistograms" )
740  else:
741  # drop all output EndPaths but the Scouting ones, and drop the RatesMonitoring
742  paths.append( "-*Output" )
743  paths.append( "-RatesMonitoring")
744  if self.config.fragment: paths.append( "Scouting*Output" )
745 
746  else:
747  if self.config.paths:
748  # keep all output EndPaths, including the DQMHistograms
749  paths.append( "*Output" )
750  paths.append( "DQMHistograms" )
751  else:
752  # keep all Paths and EndPaths
753  pass
754 
755  # drop unwanted paths for profiling (and timing studies)
756  if self.config.profiling:
757  paths.append( "-HLTAnalyzerEndpath" )
758 
759  # this should never be in any dump (nor online menu)
760  paths.append( "-OfflineOutput" )
761 
762  # expand all wildcards
763  paths = self.expandWildcards(paths, self.all_paths)
764 
765  if self.config.paths:
766  # do an "additive" consolidation
767  paths = self.consolidatePositiveList(paths)
768  if not paths:
769  raise RuntimeError('Error: option "--paths %s" does not select any valid paths' % self.config.paths)
770  else:
771  # do a "subtractive" consolidation
772  paths = self.consolidateNegativeList(paths)
773  self.options['paths'] = paths
confdb.HLTProcess.consolidateNegativeList
def consolidateNegativeList
Definition: confdb.py:142
confdb.HLTProcess.consolidatePositiveList
def consolidatePositiveList
Definition: confdb.py:154
confdb.HLTProcess.expandWildcards
def expandWildcards
Definition: confdb.py:126
def confdb.HLTProcess.consolidateNegativeList (   elements)
static

Definition at line 142 of file confdb.py.

142 
143  def consolidateNegativeList(elements):
144  # consolidate a list of path exclusions and re-inclusions
145  # the result is the list of paths to be removed from the dump
146  result = set()
147  for element in elements:
148  if element[0] == '-':
149  result.add( element )
150  else:
151  result.discard( '-' + element )
152  return sorted( element for element in result )
confdb.HLTProcess.consolidateNegativeList
def consolidateNegativeList
Definition: confdb.py:142
def confdb.HLTProcess.consolidatePositiveList (   elements)
static

Definition at line 154 of file confdb.py.

154 
155  def consolidatePositiveList(elements):
156  # consolidate a list of path selection and re-exclusions
157  # the result is the list of paths to be included in the dump
158  result = set()
159  for element in elements:
160  if element[0] == '-':
161  result.discard( element[1:] )
162  else:
163  result.add( element )
164  return sorted( element for element in result )
165 
confdb.HLTProcess.consolidatePositiveList
def consolidatePositiveList
Definition: confdb.py:154
def confdb.HLTProcess.customize (   self)

Definition at line 241 of file confdb.py.

References confdb.HLTProcess._fix_parameter(), confdb.HLTProcess.addEras(), confdb.HLTProcess.addGlobalOptions(), confdb.HLTProcess.all_paths, confdb.HLTProcess.build_source(), mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, ecal::raw::InputDataCPU.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, KDTreeNodeInfo< DATA, DIM >.data, calo::multifit::MapSymM< T, Stride, Order >.data, hcal::raw::InputDataCPU.data, dqmTnP::AbstractFitter.data, LASModuleProfile.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, calo::multifit::MapMForPM< T >.data, GeometryComparisonPlotter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, edm::MapOfVectors< K, T >::Iter.data, hcal::raw::InputDataGPU.data, @775.data, EcalShowerContainmentCorrections::Coefficients.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, edmStreamStallGrapher.Stack.data, confdb.HLTProcess.fixPrescales(), confdb.HLTProcess.instrumentDQM(), confdb.HLTProcess.instrumentErrorEventType(), confdb.HLTProcess.instrumentOpenMode(), confdb.HLTProcess.instrumentTiming(), confdb.HLTProcess.loadSetupCff(), confdb.HLTProcess.overrideGlobalTag(), confdb.HLTProcess.overrideL1MenuXml(), confdb.HLTProcess.overrideOutput(), confdb.HLTProcess.overrideProcessName(), confdb.HLTProcess.runL1Emulator(), confdb.HLTProcess.specificCustomize(), and confdb.HLTProcess.updateMessageLogger().

241 
242  def customize(self):
243 
244  # adapt the source to the current scenario
245  if not self.config.fragment:
246  self.build_source()
247 
248  # manual override some parameters
249  if self.config.type in ('HIon', ):
250  if self.config.data:
251  if not self.config.fragment:
252  self._fix_parameter( type = 'InputTag', value = 'rawDataCollector', replace = 'rawDataRepacker')
253 
254  # if requested, remove the HLT prescales
255  self.fixPrescales()
256 
257  # if requested, override all ED/HLTfilters to always pass ("open" mode)
258  self.instrumentOpenMode()
259 
260  # if requested, change all HLTTriggerTypeFilter EDFilters to accept only error events (SelectedTriggerType = 0)
261  self.instrumentErrorEventType()
262 
263  # if requested, instrument the self with the modules and EndPath needed for timing studies
264  self.instrumentTiming()
265 
266  # if requested, override the L1 self from the GlobalTag (Xml)
267  self.overrideL1MenuXml()
268 
269  # if requested, run the L1 emulator
270  self.runL1Emulator()
271 
272  # add process.load("setup_cff")
273  self.loadSetupCff()
274 
275  if self.config.fragment:
276  self.data += """
277 # dummify hltGetConditions in cff's
278 if 'hltGetConditions' in %(dict)s and 'HLTriggerFirstPath' in %(dict)s :
279  %(process)s.hltDummyConditions = cms.EDFilter( "HLTBool",
280  result = cms.bool( True )
281  )
282  %(process)s.HLTriggerFirstPath.replace(%(process)s.hltGetConditions,%(process)s.hltDummyConditions)
283 """
284 
285  # the scouting path issue:
286  # 1) for config fragments, we remove all output modules
287  # 2) however in old style datasets, the scouting output paths also run the unpackers which are needed
288  # 3) therefore they have to keep the scouting path but remove the scouting output module
289  # 4) in new style datasets, aka datasetpaths & finalpaths, the scouting unpackers are on another path and all of this is unnecessary
290  # 5) however its hard to detect whether we have new style or old style so we run this for both
291  # 6) therefore we end up with a superfluous Scouting*OutputPaths which are empty
292  for path in self.all_paths:
293  match = re.match(r'(Scouting\w+)Output$', path)
294  if match:
295  module = 'hltOutput' + match.group(1)
296  self.data = self.data.replace(path+' = cms.EndPath', path+' = cms.Path')
297  self.data = self.data.replace(' + process.'+module, '')
298  self.data = self.data.replace(' process.'+module, '')
299  else:
300 
301  # override the process name and adapt the relevant filters
302  self.overrideProcessName()
303 
304  # select specific Eras
305  self.addEras()
306 
307  # override the output modules to output root files
308  self.overrideOutput()
309 
310  # add global options
311  self.addGlobalOptions()
312 
313  # if requested or necessary, override the GlobalTag and connection strings (incl. L1!)
314  self.overrideGlobalTag()
315 
316  # request summary informations from the MessageLogger
317  self.updateMessageLogger()
318 
319  # replace DQMStore and DQMRootOutputModule with a configuration suitable for running offline
320  self.instrumentDQM()
321 
322  # add specific customisations
323  self.specificCustomize()
324 
confdb.HLTProcess._fix_parameter
def _fix_parameter
Definition: confdb.py:343
confdb.HLTProcess.overrideL1MenuXml
def overrideL1MenuXml
Definition: confdb.py:452
confdb.HLTProcess.addGlobalOptions
def addGlobalOptions
Definition: confdb.py:325
confdb.HLTProcess.overrideOutput
def overrideOutput
Definition: confdb.py:472
confdb.HLTProcess.overrideProcessName
def overrideProcessName
Definition: confdb.py:551
confdb.HLTProcess.specificCustomize
def specificCustomize
Definition: confdb.py:176
confdb.HLTProcess.instrumentErrorEventType
def instrumentErrorEventType
Definition: confdb.py:400
confdb.HLTProcess.overrideGlobalTag
def overrideGlobalTag
Definition: confdb.py:408
confdb.HLTProcess.instrumentTiming
def instrumentTiming
Definition: confdb.py:629
confdb.HLTProcess.instrumentOpenMode
def instrumentOpenMode
Definition: confdb.py:386
confdb.HLTProcess.updateMessageLogger
def updateMessageLogger
Definition: confdb.py:571
def confdb.HLTProcess.dump (   self)

Definition at line 167 of file confdb.py.

References mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, ecal::raw::InputDataCPU.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, KDTreeNodeInfo< DATA, DIM >.data, calo::multifit::MapSymM< T, Stride, Order >.data, hcal::raw::InputDataCPU.data, LASModuleProfile.data, dqmTnP::AbstractFitter.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, calo::multifit::MapMForPM< T >.data, GeometryComparisonPlotter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, edm::MapOfVectors< K, T >::Iter.data, hcal::raw::InputDataGPU.data, @775.data, EcalShowerContainmentCorrections::Coefficients.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, edmStreamStallGrapher.Stack.data, confdb.HLTProcess.labels, ecaldqm::binning::AxisSpecs.labels, and svgfig.Ticks.labels.

167 
168  def dump(self):
169  self.data = self.data % self.labels
170  if self.config.fragment:
171  self.data = re.sub( r'\bprocess\b', 'fragment', self.data )
172  self.data = re.sub( r'\bProcess\b', 'ProcessFragment', self.data )
173  return self.data
174 
def confdb.HLTProcess.dumppaths (   paths)
static

Definition at line 703 of file confdb.py.

703 
704  def dumppaths(paths):
705  sys.stderr.write('Path selection:\n')
706  for path in paths:
707  sys.stderr.write('\t%s\n' % path)
708  sys.stderr.write('\n\n')
def confdb.HLTProcess.expand_filenames (   self,
  input 
)

Definition at line 886 of file confdb.py.

Referenced by confdb.HLTProcess.build_source().

886 
887  def expand_filenames(self, input):
888  # check if the input is a dataset or a list of files
889  if input[0:8] == 'dataset:':
890  from .dasFileQuery import dasFileQuery
891  # extract the dataset name, and use DAS to fine the list of LFNs
892  dataset = input[8:]
893  files = dasFileQuery(dataset)
894  else:
895  # assume a comma-separated list of input files
896  files = input.split(',')
897  return files
confdb.HLTProcess.expand_filenames
def expand_filenames
Definition: confdb.py:886
def confdb.HLTProcess.expandWildcards (   globs,
  collection 
)
static

Definition at line 126 of file confdb.py.

References python.rootplot.root2matplotlib.replace().

126 
127  def expandWildcards(globs, collection):
128  # expand a list of unix-style wildcards matching a given collection
129  # wildcards with no matches are silently discarded
130  matches = []
131  for glob in globs:
132  negate = ''
133  if glob[0] == '-':
134  negate = '-'
135  glob = glob[1:]
136  # translate a unix-style glob expression into a regular expression
137  filter = re.compile(r'^' + glob.replace('?', '.').replace('*', '.*').replace('[!', '[^') + r'$')
138  matches.extend( negate + element for element in collection if filter.match(element) )
139  return matches
140 
confdb.HLTProcess.expandWildcards
def expandWildcards
Definition: confdb.py:126
def confdb.HLTProcess.fixPrescales (   self)

Definition at line 362 of file confdb.py.

References confdb.HLTProcess.all_paths, mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, ecal::DigisCollection< StoragePolicy >.data, TrackProbabilityCalibration.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, ecal::raw::InputDataCPU.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, KDTreeNodeInfo< DATA, DIM >.data, calo::multifit::MapSymM< T, Stride, Order >.data, hcal::raw::InputDataCPU.data, LASModuleProfile.data, dqmTnP::AbstractFitter.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, GeometryComparisonPlotter.data, calo::multifit::MapMForPM< T >.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, edm::MapOfVectors< K, T >::Iter.data, hcal::raw::InputDataGPU.data, EcalShowerContainmentCorrections::Coefficients.data, @775.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, edmStreamStallGrapher.Stack.data, DTCalibrationWorker.DTCalibrationWorker.options, DTWorkflow.DTWorkflow.options, TestProcess.TestProcess.options, confdb.HLTProcess.options, Config.Process.options, and validateAlignments.ValidationJobMultiIOV.options.

Referenced by confdb.HLTProcess.customize().

362 
363  def fixPrescales(self):
364  # update the PrescaleService to match the new list of paths
365  if self.options['paths']:
366  if self.options['paths'][0][0] == '-':
367  # drop requested paths
368  for minuspath in self.options['paths']:
369  path = minuspath[1:]
370  self.data = re.sub(r' cms.PSet\( pathName = cms.string\( "%s" \),\n prescales = cms.vuint32\( .* \)\n \),?\n' % path, '', self.data)
371  else:
372  # keep requested paths
373  for path in self.all_paths:
374  if path not in self.options['paths']:
375  self.data = re.sub(r' cms.PSet\( pathName = cms.string\( "%s" \),\n prescales = cms.vuint32\( .* \)\n \),?\n' % path, '', self.data)
376 
377  if self.config.prescale and (self.config.prescale.lower() != 'none'):
378  # TO DO: check that the requested prescale column is valid
379  self.data += """
380 # force the use of a specific HLT prescale column
381 if 'PrescaleService' in %(dict)s:
382  %(process)s.PrescaleService.forceDefault = True
383  %(process)s.PrescaleService.lvl1DefaultLabel = '%(prescale)s'
384 """
385 
def confdb.HLTProcess.getPathList (   self)

Definition at line 101 of file confdb.py.

101 
102  def getPathList(self):
103  if self.config.menu.run:
104  args = ['--runNumber', self.config.menu.run]
105  else:
106  args = ['--configName', self.config.menu.name]
107  args.extend( (
108  '--cff',
109  '--noedsources',
110  '--noes',
111  '--noservices',
112  '--nosequences',
113  '--nomodules'
114  ) )
115 
116  data, err = self.converter.query( *args )
117  if 'ERROR' in err or 'Exhausted Resultset' in err or 'CONFIG_NOT_FOUND' in err:
118  sys.stderr.write("%s: error while retrieving the list of paths from the HLT menu\n\n" % os.path.basename(sys.argv[0]))
119  sys.stderr.write(err + "\n\n")
120  sys.exit(1)
121  filter = re.compile(r' *= *cms.(End|Final)?Path.*')
122  paths = [ filter.sub('', line) for line in data.splitlines() if filter.search(line) ]
123  return paths
124 
def confdb.HLTProcess.getRawConfigurationFromDB (   self)

Definition at line 83 of file confdb.py.

References mat4.data, CombinedTauTagCalibration.data, CombinedSVCalibration.data, ecal::DigisCollection< StoragePolicy >.data, TrackProbabilityCalibration.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, ecal::raw::InputDataCPU.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, calo::multifit::MapSymM< T, Stride, Order >.data, KDTreeNodeInfo< DATA, DIM >.data, hcal::raw::InputDataCPU.data, dqmTnP::AbstractFitter.data, LASModuleProfile.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, GeometryComparisonPlotter.data, calo::multifit::MapMForPM< T >.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, edm::MapOfVectors< K, T >::Iter.data, hcal::raw::InputDataGPU.data, EcalShowerContainmentCorrections::Coefficients.data, @775.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, edmStreamStallGrapher.Stack.data, and join().

83 
84  def getRawConfigurationFromDB(self):
85  if self.config.menu.run:
86  args = ['--runNumber', self.config.menu.run]
87  else:
88  args = ['--configName', self.config.menu.name ]
89  if not self.config.hilton:
90  # keep the original Source when running on Hilton
91  args.append('--noedsources')
92  for key, vals in self.options.items():
93  if vals:
94  args.extend(('--'+key, ','.join(vals)))
95  data, err = self.converter.query( *args )
96  if 'ERROR' in err or 'Exhausted Resultset' in err or 'CONFIG_NOT_FOUND' in err:
97  sys.stderr.write("%s: error while retrieving the HLT menu\n\n" % os.path.basename(sys.argv[0]))
98  sys.stderr.write(err + "\n\n")
99  sys.exit(1)
100  self.data = data
join
static std::string join(char **cmd)
Definition: RemoteFile.cc:19
confdb.HLTProcess.getRawConfigurationFromDB
def getRawConfigurationFromDB
Definition: confdb.py:83
def confdb.HLTProcess.getSetupConfigurationFromDB (   self)

Definition at line 60 of file confdb.py.

References join(), and python.rootplot.root2matplotlib.replace().

60 
61  def getSetupConfigurationFromDB(self):
62  if not self.config.setup:
63  return
64  ## if --setup is a python file, use directly that file as setup_cff.py
65  if ".py" in self.config.setup:
66  self.config.setupFile = self.config.setup.split(".py")[0]
67  return
68  args = ['--configName', self.config.setup ]
69  args.append('--noedsources')
70  args.append('--nopaths')
71  for key, vals in self.options.items():
72  if vals:
73  args.extend(('--'+key, ','.join(vals)))
74  args.append('--cff')
75  data, err = self.converter.query( *args )
76  if 'ERROR' in err or 'Exhausted Resultset' in err or 'CONFIG_NOT_FOUND' in err:
77  sys.stderr.write("%s: error while retrieving the HLT setup menu\n\n" % os.path.basename(sys.argv[0]))
78  sys.stderr.write(err + "\n\n")
79  sys.exit(1)
80  self.config.setupFile = "setup_"+self.config.setup[1:].replace("/","_")+"_cff"
81  outfile = open(self.config.setupFile+".py","w+")
82  outfile.write("# This file is automatically generated by hltGetConfiguration.\n" + data)
confdb.HLTProcess.getSetupConfigurationFromDB
def getSetupConfigurationFromDB
Definition: confdb.py:60
join
static std::string join(char **cmd)
Definition: RemoteFile.cc:19
def confdb.HLTProcess.instrumentDQM (   self)

Definition at line 671 of file confdb.py.

References mat4.data, CombinedTauTagCalibration.data, CombinedSVCalibration.data, ecal::DigisCollection< StoragePolicy >.data, TrackProbabilityCalibration.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, ecal::raw::InputDataCPU.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, calo::multifit::MapSymM< T, Stride, Order >.data, KDTreeNodeInfo< DATA, DIM >.data, hcal::raw::InputDataCPU.data, LASModuleProfile.data, dqmTnP::AbstractFitter.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, calo::multifit::MapMForPM< T >.data, GeometryComparisonPlotter.data, edm::MapOfVectors< K, T >::Iter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, hcal::raw::InputDataGPU.data, @775.data, EcalShowerContainmentCorrections::Coefficients.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, edmStreamStallGrapher.Stack.data, confdb.HLTProcess.loadCffCommand(), and confdb.HLTProcess.removeElementFromSequencesTasksAndPaths().

Referenced by confdb.HLTProcess.customize().

671 
672  def instrumentDQM(self):
673  if not self.config.hilton:
674  # remove any reference to the hltDQMFileSaver and hltDQMFileSaverPB:
675  # note the convert options remove the module itself,
676  # here we are just removing the references in paths, sequences, etc
677  self.removeElementFromSequencesTasksAndPaths('hltDQMFileSaverPB')
678  self.removeElementFromSequencesTasksAndPaths('hltDQMFileSaver')
679 
680  # instrument the HLT menu with DQMStore and DQMRootOutputModule suitable for running offline
681  dqmstore = "\n# load the DQMStore and DQMRootOutputModule\n"
682  dqmstore += self.loadCffCommand('DQMServices.Core.DQMStore_cfi')
683  dqmstore += """
684 %(process)s.dqmOutput = cms.OutputModule("DQMRootOutputModule",
685  fileName = cms.untracked.string("DQMIO.root")
686 )
687 """
688  empty_path = re.compile(r'.*\b(process\.)?DQMOutput = cms\.(Final|End)Path\( *\).*')
689  other_path = re.compile(r'(.*\b(process\.)?DQMOutput = cms\.(Final|End)Path\()(.*)')
690  if empty_path.search(self.data):
691  # replace an empty DQMOutput path
692  self.data = empty_path.sub(dqmstore + '\n%(process)s.DQMOutput = cms.FinalPath( %(process)s.dqmOutput )\n', self.data)
693  elif other_path.search(self.data):
694  # prepend the dqmOutput to the DQMOutput path
695  self.data = other_path.sub(dqmstore + r'\g<1> %(process)s.dqmOutput +\g<4>', self.data)
696  else:
697  # create a new DQMOutput path with the dqmOutput module
698  self.data += dqmstore
699  self.data += '\n%(process)s.DQMOutput = cms.FinalPath( %(process)s.dqmOutput )\n'
700  self.data += '%(process)s.schedule.append( %(process)s.DQMOutput )\n'
701 
confdb.HLTProcess.removeElementFromSequencesTasksAndPaths
def removeElementFromSequencesTasksAndPaths
Definition: confdb.py:621
confdb.HLTProcess.loadCffCommand
def loadCffCommand
Definition: confdb.py:602
def confdb.HLTProcess.instrumentErrorEventType (   self)

Definition at line 400 of file confdb.py.

References confdb.HLTProcess._fix_parameter().

Referenced by confdb.HLTProcess.customize().

400 
401  def instrumentErrorEventType(self):
402  if self.config.errortype:
403  # change all HLTTriggerTypeFilter EDFilters to accept only error events (SelectedTriggerType = 0)
404  self._fix_parameter(name = 'SelectedTriggerType', type ='int32', value = '1', replace = '0')
405  self._fix_parameter(name = 'SelectedTriggerType', type ='int32', value = '2', replace = '0')
406  self._fix_parameter(name = 'SelectedTriggerType', type ='int32', value = '3', replace = '0')
407 
confdb.HLTProcess._fix_parameter
def _fix_parameter
Definition: confdb.py:343
confdb.HLTProcess.instrumentErrorEventType
def instrumentErrorEventType
Definition: confdb.py:400
def confdb.HLTProcess.instrumentOpenMode (   self)

Definition at line 386 of file confdb.py.

References mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, ecal::raw::InputDataCPU.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, calo::multifit::MapSymM< T, Stride, Order >.data, KDTreeNodeInfo< DATA, DIM >.data, hcal::raw::InputDataCPU.data, LASModuleProfile.data, dqmTnP::AbstractFitter.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, calo::multifit::MapMForPM< T >.data, GeometryComparisonPlotter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, edm::MapOfVectors< K, T >::Iter.data, hcal::raw::InputDataGPU.data, @775.data, EcalShowerContainmentCorrections::Coefficients.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, edmStreamStallGrapher.Stack.data, and join().

Referenced by confdb.HLTProcess.customize().

386 
387  def instrumentOpenMode(self):
388  if self.config.open:
389  # find all EDfilters
390  filters = [ match[1] for match in re.findall(r'(process\.)?\b(\w+) = cms.EDFilter', self.data) ]
391  re_sequence = re.compile( r'cms\.(Path|Sequence)\((.*)\)' )
392  # remove existing 'cms.ignore' and '~' modifiers
393  self.data = re_sequence.sub( lambda line: re.sub( r'cms\.ignore *\( *((process\.)?\b(\w+)) *\)', r'\1', line.group(0) ), self.data )
394  self.data = re_sequence.sub( lambda line: re.sub( r'~', '', line.group(0) ), self.data )
395  # wrap all EDfilters with "cms.ignore( ... )", 1000 at a time (python 2.6 complains for too-big regular expressions)
396  for some in splitter(filters, 1000):
397  re_filters = re.compile( r'\b((process\.)?(' + r'|'.join(some) + r'))\b' )
398  self.data = re_sequence.sub( lambda line: re_filters.sub( r'cms.ignore( \1 )', line.group(0) ), self.data )
399 
join
static std::string join(char **cmd)
Definition: RemoteFile.cc:19
confdb.HLTProcess.instrumentOpenMode
def instrumentOpenMode
Definition: confdb.py:386
def confdb.HLTProcess.instrumentTiming (   self)

Definition at line 629 of file confdb.py.

References mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, ecal::raw::InputDataCPU.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, calo::multifit::MapSymM< T, Stride, Order >.data, KDTreeNodeInfo< DATA, DIM >.data, hcal::raw::InputDataCPU.data, LASModuleProfile.data, dqmTnP::AbstractFitter.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, calo::multifit::MapMForPM< T >.data, GeometryComparisonPlotter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, edm::MapOfVectors< K, T >::Iter.data, hcal::raw::InputDataGPU.data, @775.data, EcalShowerContainmentCorrections::Coefficients.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, edmStreamStallGrapher.Stack.data, and confdb.HLTProcess.loadCff().

Referenced by confdb.HLTProcess.customize().

629 
630  def instrumentTiming(self):
631 
632  if self.config.timing:
633  self.data += """
634 # instrument the menu with the modules and EndPath needed for timing studies
635 """
636 
637  self.data += '\n# configure the FastTimerService\n'
638  self.loadCff('HLTrigger.Timer.FastTimerService_cfi')
639 
640  self.data += """# print a text summary at the end of the job
641 %(process)s.FastTimerService.printEventSummary = False
642 %(process)s.FastTimerService.printRunSummary = False
643 %(process)s.FastTimerService.printJobSummary = True
644 
645 # enable DQM plots
646 %(process)s.FastTimerService.enableDQM = True
647 
648 # enable per-path DQM plots (starting with CMSSW 9.2.3-patch2)
649 %(process)s.FastTimerService.enableDQMbyPath = True
650 
651 # enable per-module DQM plots
652 %(process)s.FastTimerService.enableDQMbyModule = True
653 
654 # enable per-event DQM plots vs lumisection
655 %(process)s.FastTimerService.enableDQMbyLumiSection = True
656 %(process)s.FastTimerService.dqmLumiSectionsRange = 2500
657 
658 # set the time resolution of the DQM plots
659 %(process)s.FastTimerService.dqmTimeRange = 2000.
660 %(process)s.FastTimerService.dqmTimeResolution = 10.
661 %(process)s.FastTimerService.dqmPathTimeRange = 1000.
662 %(process)s.FastTimerService.dqmPathTimeResolution = 5.
663 %(process)s.FastTimerService.dqmModuleTimeRange = 200.
664 %(process)s.FastTimerService.dqmModuleTimeResolution = 1.
665 
666 # set the base DQM folder for the plots
667 %(process)s.FastTimerService.dqmPath = 'HLT/TimerService'
668 %(process)s.FastTimerService.enableDQMbyProcesses = False
669 """
670 
confdb.HLTProcess.instrumentTiming
def instrumentTiming
Definition: confdb.py:629
def confdb.HLTProcess.loadAdditionalConditions (   self,
  comment,
  conditions 
)

Definition at line 585 of file confdb.py.

References mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, ecal::raw::InputDataCPU.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, KDTreeNodeInfo< DATA, DIM >.data, calo::multifit::MapSymM< T, Stride, Order >.data, hcal::raw::InputDataCPU.data, LASModuleProfile.data, dqmTnP::AbstractFitter.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, calo::multifit::MapMForPM< T >.data, GeometryComparisonPlotter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, edm::MapOfVectors< K, T >::Iter.data, hcal::raw::InputDataGPU.data, @775.data, EcalShowerContainmentCorrections::Coefficients.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, and edmStreamStallGrapher.Stack.data.

585 
586  def loadAdditionalConditions(self, comment, *conditions):
587  # load additional conditions
588  self.data += """
589 # %s
590 if 'GlobalTag' in %%(dict)s:
591 """ % comment
592  for condition in conditions:
593  self.data += """ %%(process)s.GlobalTag.toGet.append(
594  cms.PSet(
595  record = cms.string( '%(record)s' ),
596  tag = cms.string( '%(tag)s' ),
597  label = cms.untracked.string( '%(label)s' ),
598  )
599  )
600 """ % condition
601 
confdb.HLTProcess.loadAdditionalConditions
def loadAdditionalConditions
Definition: confdb.py:585
def confdb.HLTProcess.loadCff (   self,
  module 
)

Definition at line 609 of file confdb.py.

References mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, ecal::raw::InputDataCPU.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, calo::multifit::MapSymM< T, Stride, Order >.data, KDTreeNodeInfo< DATA, DIM >.data, hcal::raw::InputDataCPU.data, LASModuleProfile.data, dqmTnP::AbstractFitter.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, calo::multifit::MapMForPM< T >.data, GeometryComparisonPlotter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, edm::MapOfVectors< K, T >::Iter.data, hcal::raw::InputDataGPU.data, @775.data, EcalShowerContainmentCorrections::Coefficients.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, edmStreamStallGrapher.Stack.data, and confdb.HLTProcess.loadCffCommand().

Referenced by confdb.HLTProcess.instrumentTiming().

609 
610  def loadCff(self, module):
611  self.data += self.loadCffCommand(module)
612 
confdb.HLTProcess.loadCffCommand
def loadCffCommand
Definition: confdb.py:602
def confdb.HLTProcess.loadCffCommand (   self,
  module 
)

Definition at line 602 of file confdb.py.

Referenced by confdb.HLTProcess.instrumentDQM(), and confdb.HLTProcess.loadCff().

602 
603  def loadCffCommand(self, module):
604  # load a cfi or cff module
605  if self.config.fragment:
606  return 'from %s import *\n' % module
607  else:
608  return 'process.load( "%s" )\n' % module
confdb.HLTProcess.loadCffCommand
def loadCffCommand
Definition: confdb.py:602
def confdb.HLTProcess.loadSetupCff (   self)

Definition at line 544 of file confdb.py.

References mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, ecal::raw::InputDataCPU.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, KDTreeNodeInfo< DATA, DIM >.data, calo::multifit::MapSymM< T, Stride, Order >.data, hcal::raw::InputDataCPU.data, LASModuleProfile.data, dqmTnP::AbstractFitter.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, calo::multifit::MapMForPM< T >.data, GeometryComparisonPlotter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, edm::MapOfVectors< K, T >::Iter.data, hcal::raw::InputDataGPU.data, @775.data, EcalShowerContainmentCorrections::Coefficients.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, and edmStreamStallGrapher.Stack.data.

Referenced by confdb.HLTProcess.customize().

544 
545  def loadSetupCff(self):
546  if self.config.setup is None:
547  return
548  processLine = self.data.find("\n",self.data.find("cms.Process"))
549  self.data = self.data[:processLine]+'\nprocess.load("%s")'%self.config.setupFile+self.data[processLine:]
def confdb.HLTProcess.overrideGlobalTag (   self)

Definition at line 408 of file confdb.py.

References mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, ecal::raw::InputDataCPU.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, KDTreeNodeInfo< DATA, DIM >.data, calo::multifit::MapSymM< T, Stride, Order >.data, hcal::raw::InputDataCPU.data, LASModuleProfile.data, dqmTnP::AbstractFitter.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, calo::multifit::MapMForPM< T >.data, GeometryComparisonPlotter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, edm::MapOfVectors< K, T >::Iter.data, hcal::raw::InputDataGPU.data, @775.data, EcalShowerContainmentCorrections::Coefficients.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, and edmStreamStallGrapher.Stack.data.

Referenced by confdb.HLTProcess.customize().

408 
409  def overrideGlobalTag(self):
410  # overwrite GlobalTag
411  # the logic is:
412  # - if a GlobalTag is specified on the command line:
413  # - override the global tag
414  # - if the GT is "auto:...", insert the code to read it from Configuration.AlCa.autoCond
415  # - if a GlobalTag is NOT specified on the command line:
416  # - when running on data, do nothing, and keep the global tag in the menu
417  # - when running on mc, take the GT from the configuration.type
418 
419  # override the GlobalTag connection string and pfnPrefix
420 
421  # when running on MC, override the global tag even if not specified on the command line
422  if not self.config.data and not self.config.globaltag:
423  if self.config.type in globalTag:
424  self.config.globaltag = globalTag[self.config.type]
425  else:
426  self.config.globaltag = globalTag['GRun']
427 
428  # if requested, override the L1 menu from the GlobalTag
429  if self.config.l1.override:
430  self.config.l1.tag = self.config.l1.override
431  self.config.l1.record = 'L1TUtmTriggerMenuRcd'
432  self.config.l1.connect = ''
433  self.config.l1.label = ''
434  if not self.config.l1.snapshotTime:
435  self.config.l1.snapshotTime = '9999-12-31 23:59:59.000'
436  self.config.l1cond = '%(tag)s,%(record)s,%(connect)s,%(label)s,%(snapshotTime)s' % self.config.l1.__dict__
437  else:
438  self.config.l1cond = None
439 
440  if self.config.globaltag or self.config.l1cond:
441  text = """
442 # override the GlobalTag, connection string and pfnPrefix
443 if 'GlobalTag' in %(dict)s:
444  from Configuration.AlCa.GlobalTag import GlobalTag as customiseGlobalTag
445  %(process)s.GlobalTag = customiseGlobalTag(%(process)s.GlobalTag"""
446  if self.config.globaltag:
447  text += ", globaltag = %s" % repr(self.config.globaltag)
448  if self.config.l1cond:
449  text += ", conditions = %s" % repr(self.config.l1cond)
450  text += ")\n"
451  self.data += text
confdb.HLTProcess.overrideGlobalTag
def overrideGlobalTag
Definition: confdb.py:408
def confdb.HLTProcess.overrideL1MenuXml (   self)

Definition at line 452 of file confdb.py.

References mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, ecal::raw::InputDataCPU.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, KDTreeNodeInfo< DATA, DIM >.data, calo::multifit::MapSymM< T, Stride, Order >.data, hcal::raw::InputDataCPU.data, LASModuleProfile.data, dqmTnP::AbstractFitter.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, calo::multifit::MapMForPM< T >.data, GeometryComparisonPlotter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, edm::MapOfVectors< K, T >::Iter.data, hcal::raw::InputDataGPU.data, @775.data, EcalShowerContainmentCorrections::Coefficients.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, and edmStreamStallGrapher.Stack.data.

Referenced by confdb.HLTProcess.customize().

452 
453  def overrideL1MenuXml(self):
454  # if requested, override the GlobalTag's L1T menu from an Xml file
455  if self.config.l1Xml.XmlFile:
456  text = """
457 # override the GlobalTag's L1T menu from an Xml file
458 from HLTrigger.Configuration.CustomConfigs import L1XML
459 %%(process)s = L1XML(%%(process)s,"%s")
460 """ % (self.config.l1Xml.XmlFile)
461  self.data += text
confdb.HLTProcess.overrideL1MenuXml
def overrideL1MenuXml
Definition: confdb.py:452
def confdb.HLTProcess.overrideOutput (   self)

Definition at line 472 of file confdb.py.

References mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, ecal::raw::InputDataCPU.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, KDTreeNodeInfo< DATA, DIM >.data, calo::multifit::MapSymM< T, Stride, Order >.data, hcal::raw::InputDataCPU.data, LASModuleProfile.data, dqmTnP::AbstractFitter.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, calo::multifit::MapMForPM< T >.data, GeometryComparisonPlotter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, edm::MapOfVectors< K, T >::Iter.data, hcal::raw::InputDataGPU.data, @775.data, EcalShowerContainmentCorrections::Coefficients.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, and edmStreamStallGrapher.Stack.data.

Referenced by confdb.HLTProcess.customize().

472 
473  def overrideOutput(self):
474  # if not runnign on Hilton, override the "online" ShmStreamConsumer output modules with "offline" PoolOutputModule's
475  # note for Run3 ShmStreamConsumer has been replaced with EvFOutputModule and later GlobalEvFOutputModule
476  # so we also do a replace there
477  if not self.config.hilton:
478  self.data = re.sub(
479  r'\b(process\.)?hltOutput(\w+) *= *cms\.OutputModule\( *"(ShmStreamConsumer)" *,',
480  r'%(process)s.hltOutput\2 = cms.OutputModule( "PoolOutputModule",\n fileName = cms.untracked.string( "output\2.root" ),\n fastCloning = cms.untracked.bool( False ),\n dataset = cms.untracked.PSet(\n filterName = cms.untracked.string( "" ),\n dataTier = cms.untracked.string( "RAW" )\n ),',
481  self.data
482  )
483  self.data = re.sub(
484  r'\b(process\.)?hltOutput(\w+) *= *cms\.OutputModule\( *"EvFOutputModule" *,\n use_compression = cms.untracked.bool\( True \),\n compression_algorithm = cms.untracked.string\( "ZLIB" \),\n compression_level = cms.untracked.int32\( 1 \),\n lumiSection_interval = cms.untracked.int32\( 0 \),\n(.+?),\n psetMap = cms.untracked.InputTag\( "hltPSetMap" \)\n',
485  r'\1hltOutput\2 = cms.OutputModule( "PoolOutputModule",\n fileName = cms.untracked.string( "output\2.root" ),\n fastCloning = cms.untracked.bool( False ),\n dataset = cms.untracked.PSet(\n filterName = cms.untracked.string( "" ),\n dataTier = cms.untracked.string( "RAW" )\n ),\n\3\n',
486  self.data,0,re.DOTALL
487  )
488  self.data = re.sub(
489  r'\b(process\.)?hltOutput(\w+) *= *cms\.OutputModule\( *"GlobalEvFOutputModule" *,\n use_compression = cms.untracked.bool\( True \),\n compression_algorithm = cms.untracked.string\( "ZLIB" \),\n compression_level = cms.untracked.int32\( 1 \),\n lumiSection_interval = cms.untracked.int32\( 0 \),\n(.+?),\n psetMap = cms.untracked.InputTag\( "hltPSetMap" \)\n',
490  r'\1hltOutput\2 = cms.OutputModule( "PoolOutputModule",\n fileName = cms.untracked.string( "output\2.root" ),\n fastCloning = cms.untracked.bool( False ),\n dataset = cms.untracked.PSet(\n filterName = cms.untracked.string( "" ),\n dataTier = cms.untracked.string( "RAW" )\n ),\n\3\n',
491  self.data,0,re.DOTALL
492  )
493  if not self.config.fragment and self.config.output == 'minimal':
494  # add a single output to keep the TriggerResults and TriggerEvent
495  self.data += """
496 # add a single "keep *" output
497 %(process)s.hltOutputMinimal = cms.OutputModule( "PoolOutputModule",
498  fileName = cms.untracked.string( "output.root" ),
499  fastCloning = cms.untracked.bool( False ),
500  dataset = cms.untracked.PSet(
501  dataTier = cms.untracked.string( 'AOD' ),
502  filterName = cms.untracked.string( '' )
503  ),
504  outputCommands = cms.untracked.vstring( 'drop *',
505  'keep edmTriggerResults_*_*_*',
506  'keep triggerTriggerEvent_*_*_*',
507  'keep GlobalAlgBlkBXVector_*_*_*',
508  'keep GlobalExtBlkBXVector_*_*_*',
509  'keep l1tEGammaBXVector_*_EGamma_*',
510  'keep l1tEtSumBXVector_*_EtSum_*',
511  'keep l1tJetBXVector_*_Jet_*',
512  'keep l1tMuonBXVector_*_Muon_*',
513  'keep l1tTauBXVector_*_Tau_*',
514  )
515 )
516 %(process)s.MinimalOutput = cms.FinalPath( %(process)s.hltOutputMinimal )
517 %(process)s.schedule.append( %(process)s.MinimalOutput )
518 """
519  elif not self.config.fragment and self.config.output == 'full':
520  # add a single "keep *" output
521  self.data += """
522 # add a single "keep *" output
523 %(process)s.hltOutputFull = cms.OutputModule( "PoolOutputModule",
524  fileName = cms.untracked.string( "output.root" ),
525  fastCloning = cms.untracked.bool( False ),
526  dataset = cms.untracked.PSet(
527  dataTier = cms.untracked.string( 'RECO' ),
528  filterName = cms.untracked.string( '' )
529  ),
530  outputCommands = cms.untracked.vstring( 'keep *' )
531 )
532 %(process)s.FullOutput = cms.FinalPath( %(process)s.hltOutputFull )
533 %(process)s.schedule.append( %(process)s.FullOutput )
534 """
confdb.HLTProcess.overrideOutput
def overrideOutput
Definition: confdb.py:472
def confdb.HLTProcess.overrideParameters (   self,
  module,
  parameters 
)

Definition at line 613 of file confdb.py.

References mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, ecal::raw::InputDataCPU.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, KDTreeNodeInfo< DATA, DIM >.data, calo::multifit::MapSymM< T, Stride, Order >.data, hcal::raw::InputDataCPU.data, LASModuleProfile.data, dqmTnP::AbstractFitter.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, calo::multifit::MapMForPM< T >.data, GeometryComparisonPlotter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, edm::MapOfVectors< K, T >::Iter.data, hcal::raw::InputDataGPU.data, @775.data, EcalShowerContainmentCorrections::Coefficients.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, and edmStreamStallGrapher.Stack.data.

613 
614  def overrideParameters(self, module, parameters):
615  # override a module's parameter if the module is present in the configuration
616  self.data += "if '%s' in %%(dict)s:\n" % module
617  for (parameter, value) in parameters:
618  self.data += " %%(process)s.%s.%s = %s\n" % (module, parameter, value)
619  self.data += "\n"
620 
confdb.HLTProcess.overrideParameters
def overrideParameters
Definition: confdb.py:613
def confdb.HLTProcess.overrideProcessName (   self)

Definition at line 551 of file confdb.py.

References mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, ecal::raw::InputDataCPU.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, KDTreeNodeInfo< DATA, DIM >.data, calo::multifit::MapSymM< T, Stride, Order >.data, hcal::raw::InputDataCPU.data, LASModuleProfile.data, dqmTnP::AbstractFitter.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, calo::multifit::MapMForPM< T >.data, GeometryComparisonPlotter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, edm::MapOfVectors< K, T >::Iter.data, hcal::raw::InputDataGPU.data, @775.data, EcalShowerContainmentCorrections::Coefficients.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, and edmStreamStallGrapher.Stack.data.

Referenced by confdb.HLTProcess.customize().

551 
552  def overrideProcessName(self):
553  if self.config.name is None:
554  return
555 
556  # sanitise process name
557  self.config.name = self.config.name.replace("_","")
558  # override the process name
559  quote = '[\'\"]'
560  self.data = re.compile(r'^(process\s*=\s*cms\.Process\(\s*' + quote + r')\w+(' + quote + r'\s*\).*)$', re.MULTILINE).sub(r'\1%s\2' % self.config.name, self.data, 1)
561 
562  # when --setup option is used, remove possible errors from PrescaleService due to missing HLT paths.
563  if self.config.setup: self.data += """
564 # avoid PrescaleService error due to missing HLT paths
565 if 'PrescaleService' in process.__dict__:
566  for pset in reversed(process.PrescaleService.prescaleTable):
567  if not hasattr(process,pset.pathName.value()):
568  process.PrescaleService.prescaleTable.remove(pset)
569 """
570 
confdb.HLTProcess.overrideProcessName
def overrideProcessName
Definition: confdb.py:551
def confdb.HLTProcess.removeElementFromSequencesTasksAndPaths (   self,
  label 
)

Definition at line 621 of file confdb.py.

References mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, ecal::raw::InputDataCPU.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, KDTreeNodeInfo< DATA, DIM >.data, calo::multifit::MapSymM< T, Stride, Order >.data, hcal::raw::InputDataCPU.data, LASModuleProfile.data, dqmTnP::AbstractFitter.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, calo::multifit::MapMForPM< T >.data, GeometryComparisonPlotter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, edm::MapOfVectors< K, T >::Iter.data, hcal::raw::InputDataGPU.data, @775.data, EcalShowerContainmentCorrections::Coefficients.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, and edmStreamStallGrapher.Stack.data.

Referenced by confdb.HLTProcess.instrumentDQM().

621 
622  def removeElementFromSequencesTasksAndPaths(self, label):
623  if label in self.data:
624  label_re = r'\b(process\.)?' + label
625  self.data = re.sub(r' *(\+|,) *' + label_re, '', self.data)
626  self.data = re.sub(label_re + r' *(\+|,) *', '', self.data)
627  self.data = re.sub(label_re, '', self.data)
628 
confdb.HLTProcess.removeElementFromSequencesTasksAndPaths
def removeElementFromSequencesTasksAndPaths
Definition: confdb.py:621
def confdb.HLTProcess.runL1Emulator (   self)

Definition at line 462 of file confdb.py.

References mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, ecal::raw::InputDataCPU.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, KDTreeNodeInfo< DATA, DIM >.data, calo::multifit::MapSymM< T, Stride, Order >.data, hcal::raw::InputDataCPU.data, LASModuleProfile.data, dqmTnP::AbstractFitter.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, calo::multifit::MapMForPM< T >.data, GeometryComparisonPlotter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, edm::MapOfVectors< K, T >::Iter.data, hcal::raw::InputDataGPU.data, @775.data, EcalShowerContainmentCorrections::Coefficients.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, and edmStreamStallGrapher.Stack.data.

Referenced by confdb.HLTProcess.customize().

462 
463  def runL1Emulator(self):
464  # if requested, run the Full L1T emulator, then repack the data into a new RAW collection, to be used by the HLT
465  if self.config.emulator:
466  text = """
467 # run the Full L1T emulator, then repack the data into a new RAW collection, to be used by the HLT
468 from HLTrigger.Configuration.CustomConfigs import L1REPACK
469 %%(process)s = L1REPACK(%%(process)s,"%s")
470 """ % (self.config.emulator)
471  self.data += text
def confdb.HLTProcess.specificCustomize (   self)

Definition at line 176 of file confdb.py.

References mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, ecal::raw::InputDataCPU.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, KDTreeNodeInfo< DATA, DIM >.data, calo::multifit::MapSymM< T, Stride, Order >.data, hcal::raw::InputDataCPU.data, LASModuleProfile.data, dqmTnP::AbstractFitter.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, calo::multifit::MapMForPM< T >.data, GeometryComparisonPlotter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, edm::MapOfVectors< K, T >::Iter.data, hcal::raw::InputDataGPU.data, @775.data, EcalShowerContainmentCorrections::Coefficients.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, edmStreamStallGrapher.Stack.data, python.rootplot.root2matplotlib.replace(), confdb.HLTProcess.source, lhef::XMLDocument.source, DQMNet::Peer.source, Config.Process.source, gen::Hydjet2Hadronizer.source, and BeautifulSoup.ResultSet.source.

Referenced by confdb.HLTProcess.customize().

176 
177  def specificCustomize(self):
178  # specific customizations now live in HLTrigger.Configuration.customizeHLTforALL.customizeHLTforAll(.,.)
179  if self.config.fragment:
180  self.data += """
181 # add specific customizations
182 from HLTrigger.Configuration.customizeHLTforALL import customizeHLTforAll
183 fragment = customizeHLTforAll(fragment,"%s")
184 """ % (self.config.type)
185  elif self.config.hilton:
186  # do not apply the STORM-specific customisation
187  pass
188  else:
189  if self.config.type=="Fake":
190  prefix = "run1"
191  elif self.config.type in ("Fake1","Fake2","2018"):
192  prefix = "run2"
193  else:
194  prefix = "run3"
195  _gtData = "auto:"+prefix+"_hlt_"+self.config.type
196  _gtMc = "auto:"+prefix+"_mc_" +self.config.type
197  self.data += """
198 # add specific customizations
199 _customInfo = {}
200 _customInfo['menuType' ]= "%s"
201 _customInfo['globalTags']= {}
202 _customInfo['globalTags'][True ] = "%s"
203 _customInfo['globalTags'][False] = "%s"
204 _customInfo['inputFiles']={}
205 _customInfo['inputFiles'][True] = "file:RelVal_Raw_%s_DATA.root"
206 _customInfo['inputFiles'][False] = "file:RelVal_Raw_%s_MC.root"
207 _customInfo['maxEvents' ]= %s
208 _customInfo['globalTag' ]= "%s"
209 _customInfo['inputFile' ]= %s
210 _customInfo['realData' ]= %s
211 
212 from HLTrigger.Configuration.customizeHLTforALL import customizeHLTforAll
213 %%(process)s = customizeHLTforAll(%%(process)s,"%s",_customInfo)
214 """ % (self.config.type,_gtData,_gtMc,self.config.type,self.config.type,self.config.events,self.config.globaltag,self.source,self.config.data,self.config.type)
215 
216  self.data += """
217 from HLTrigger.Configuration.customizeHLTforCMSSW import customizeHLTforCMSSW
218 %%(process)s = customizeHLTforCMSSW(%%(process)s,"%s")
219 """ % (self.config.type)
220 
221  # Eras-based customisations
222  self.data += """
223 # Eras-based customisations
224 from HLTrigger.Configuration.Eras import modifyHLTforEras
225 modifyHLTforEras(%(process)s)
226 """
227  # add the user-defined customization functions, if any
228  if self.config.customise:
229  self.data += "\n"
230  self.data += "#User-defined customization functions\n"
231  for customise in self.config.customise.split(","):
232  customiseValues = customise.split(".")
233  if len(customiseValues)>=3: raise Exception("--customise option cannot contain more than one dot.")
234  if len(customiseValues)==1:
235  customiseValues.append("customise")
236  customiseValues[0] = customiseValues[0].replace("/",".")
237  self.data += "from "+customiseValues[0]+" import "+customiseValues[1]+"\n"
238  self.data += "process = "+customiseValues[1]+"(process)\n"
239 
confdb.HLTProcess.specificCustomize
def specificCustomize
Definition: confdb.py:176
def confdb.HLTProcess.updateMessageLogger (   self)

Definition at line 571 of file confdb.py.

References mat4.data, CombinedSVCalibration.data, CombinedTauTagCalibration.data, TrackProbabilityCalibration.data, ecal::DigisCollection< StoragePolicy >.data, ElectronLikelihoodCalibration.data, CSCTMBScope.data, ecal::raw::InputDataCPU.data, SimpleVFATFrameCollection.data, confdb.HLTProcess.data, classes.PlotData.data, dqm::DQMFileSaverOnline::SnapshotFiles.data, QGLikelihoodObject.data, KDTreeNodeInfo< DATA, DIM >.data, calo::multifit::MapSymM< T, Stride, Order >.data, hcal::raw::InputDataCPU.data, LASModuleProfile.data, dqmTnP::AbstractFitter.data, QGLikelihoodSystematicsObject.data, TkLasBeam.data, KDTreeNodes< DATA, DIM >.data, ecal::raw::InputDataGPU.data, calo::multifit::MapMForPM< T >.data, GeometryComparisonPlotter.data, EcalGlobalShowerContainmentCorrectionsVsEta::Coefficients.data, edm::MapOfVectors< K, T >::Iter.data, hcal::raw::InputDataGPU.data, @775.data, EcalShowerContainmentCorrections::Coefficients.data, edm::DetSet< T >.data, CTPPSCompositeESSource::BinData< T >.data, cscdqm::Cache.data, DQMNet::Bucket.data, hcal::DigiCollectionBase< StoragePolicy >.data, VFATFrame.data, and edmStreamStallGrapher.Stack.data.

Referenced by confdb.HLTProcess.customize().

571 
572  def updateMessageLogger(self):
573  # request summary informations from the MessageLogger
574  self.data += """
575 # show summaries from trigger analysers used at HLT
576 if 'MessageLogger' in %(dict)s:
577  %(process)s.MessageLogger.TriggerSummaryProducerAOD = cms.untracked.PSet()
578  %(process)s.MessageLogger.L1GtTrigReport = cms.untracked.PSet()
579  %(process)s.MessageLogger.L1TGlobalSummary = cms.untracked.PSet()
580  %(process)s.MessageLogger.HLTrigReport = cms.untracked.PSet()
581  %(process)s.MessageLogger.FastReport = cms.untracked.PSet()
582  %(process)s.MessageLogger.ThroughputService = cms.untracked.PSet()
583 """
584 
confdb.HLTProcess.updateMessageLogger
def updateMessageLogger
Definition: confdb.py:571

Member Data Documentation

confdb.HLTProcess.all_paths

Definition at line 710 of file confdb.py.

Referenced by confdb.HLTProcess.customize(), and confdb.HLTProcess.fixPrescales().

confdb.HLTProcess.config

Definition at line 24 of file confdb.py.

Referenced by zMuMuValidation.ZMuMuValidation.trackcollection().

confdb.HLTProcess.converter

Definition at line 53 of file confdb.py.

confdb.HLTProcess.data

Definition at line 25 of file confdb.py.

Referenced by data_sources.node.__str__(), confdb.HLTProcess._fix_parameter(), confdb.HLTProcess.addEras(), confdb.HLTProcess.addGlobalOptions(), confdb.HLTProcess.append_filenames(), data_sources.json_list.as_dicts(), data_sources.json_list.as_table(), confdb.HLTProcess.build_source(), confdb.HLTProcess.customize(), confdb.HLTProcess.dump(), confdb.HLTProcess.fixPrescales(), data_sources.json_list.get_members(), confdb.HLTProcess.getRawConfigurationFromDB(), confdb.HLTProcess.instrumentDQM(), confdb.HLTProcess.instrumentOpenMode(), confdb.HLTProcess.instrumentTiming(), data_sources.json_list.last(), confdb.HLTProcess.loadAdditionalConditions(), confdb.HLTProcess.loadCff(), confdb.HLTProcess.loadSetupCff(), confdb.HLTProcess.overrideGlobalTag(), confdb.HLTProcess.overrideL1MenuXml(), confdb.HLTProcess.overrideOutput(), confdb.HLTProcess.overrideParameters(), confdb.HLTProcess.overrideProcessName(), confdb.HLTProcess.removeElementFromSequencesTasksAndPaths(), confdb.HLTProcess.runL1Emulator(), confdb.HLTProcess.specificCustomize(), cuy.FindIssue.startElement(), and confdb.HLTProcess.updateMessageLogger().

confdb.HLTProcess.labels

Definition at line 40 of file confdb.py.

Referenced by confdb.HLTProcess.dump().

confdb.HLTProcess.options

Definition at line 29 of file confdb.py.

Referenced by betterConfigParser.BetterConfigParser.__updateDict(), submitPVValidationJobs.BetterConfigParser.__updateDict(), confdb.HLTProcess.buildOptions(), betterConfigParser.BetterConfigParser.checkInput(), CrabHelper.CrabHelper.crab_config_filename(), CrabHelper.CrabHelper.crab_taskname(), CrabHelper.CrabHelper.fill_options_from_crab_config(), confdb.HLTProcess.fixPrescales(), edmIntegrityCheck.IntegrityCheck.query(), cmsswPreprocessor.CmsswPreprocessor.run(), production_tasks.BaseDataset.run(), production_tasks.GenerateMask.run(), production_tasks.RunCMSBatch.run(), production_tasks.MonitorJobs.run(), production_tasks.CheckJobStatus.run(), and production_tasks.CleanJobFiles.run().

confdb.HLTProcess.parent

Definition at line 27 of file confdb.py.

Referenced by BeautifulSoup.PageElement._invert(), confdb.HLTProcess.build_source(), edmIntegrityCheck.PublishToFileSystem.get(), and edmIntegrityCheck.PublishToFileSystem.publish().

confdb.HLTProcess.source

Definition at line 26 of file confdb.py.

Referenced by confdb.HLTProcess.build_source(), and confdb.HLTProcess.specificCustomize().

Generated for CMSSW Reference Manual by doxygen 1.8.5