CMS 3D CMS Logo
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Properties Friends Macros Pages
List of all members | Classes | Public Member Functions | Public Attributes | Private Attributes
cmsPerfSuite.PerfSuite Class Reference

Classes

class  PerfTest
 
class  simpleGenReportThread
 

Public Member Functions

def __init__
 
def benchmarks
 Run cmsScimark benchmarks a number of times. More...
 
def cprootfile
 Copy root file from another candle's directory ! Again this is messy. More...
 
def displayErrors
 Display G4 cerr errors and CMSExceptions in the logfile. More...
 
def getDate
 
def mkCandleDir
 Make directory for a particular candle and profiler. More...
 
def optionParse
 
def printDate
 
def printFlush
 Print and flush a string (for output to a log file) More...
 
def runcmd
 Run a command and return the exit status. More...
 
def runCmdSet
 Run a list of commands using system ! We should rewrite this not to use system (most cases it is unnecessary) More...
 
def runCmsInput
 Wrapper for cmsRelvalreportInput. More...
 
def runCmsReport
 This function is a wrapper around cmsRelvalreport. More...
 
def runPerfSuite
 Runs benchmarking, cpu spinlocks on spare cores and profiles selected candles. More...
 
def simpleGenReport
 Prepares the profiling directory and runs all the selected profiles (if this is not a unit test) More...
 
def testCmsDriver
 Test cmsDriver.py (parses the simcandles file, removing duplicate lines, and runs the cmsDriver part) More...
 
def valFilterReport
 Filter lines in the valgrind report that match GEN,SIM. More...
 

Public Attributes

 AuxiliaryScripts
 
 cmssw_arch
 
 cmssw_version
 
 Commands
 
 ERRORS
 
 HEPSPEC06
 
 host
 
 logh
 
 PerfTestTotalTimer
 FIXME: We may want to introduce a switch here or agree on a different default (currently 10 cmsScimark and 10 cmsScimarkLarge) More...
 
 Scripts
 
 user
 

Private Attributes

 _CASTOR_DIR
 
 _debug
 Developer options. More...
 
 _dryrun
 
 _noexec
 
 _unittest
 
 _verbose
 Check logfile option. More...
 

Detailed Description

Definition at line 60 of file cmsPerfSuite.py.

Constructor & Destructor Documentation

def cmsPerfSuite.PerfSuite.__init__ (   self)

Definition at line 61 of file cmsPerfSuite.py.

61 
62  def __init__(self):
63 
64  self.ERRORS = 0
65  self._CASTOR_DIR = "/castor/cern.ch/cms/store/relval/performance/"
66  self._dryrun = False
67  self._debug = False
68  self._unittest = False
69  self._noexec = False
70  self._verbose = True
71  self.logh = sys.stdout
72 
73  #Get some environment variables to use
74  try:
75  self.cmssw_arch = os.environ["SCRAM_ARCH"]
76  self.cmssw_version= os.environ["CMSSW_VERSION"]
77  self.host = os.environ["HOST"]
78  self.user = os.environ["USER"]
79  except KeyError:
80  self.logh.write('Error: An environment variable either SCRAM_ARCH, CMSSW_VERSION, HOST or USER is not available.\n')
81  self.logh.write(' Please run eval `scramv1 runtime -csh` to set your environment variables\n')
82  self.logh.flush()
83  sys.exit()
84 
85  #Scripts used by the suite:
86  self.Scripts =["cmsDriver.py","cmsRelvalreport.py","cmsRelvalreportInput.py","cmsScimark2"]
87  self.AuxiliaryScripts=["cmsScimarkLaunch.csh","cmsScimarkParser.py","cmsScimarkStop.py"]
88 
cmsPerfSuite.PerfSuite._verbose
_verbose
Check logfile option.
Definition: cmsPerfSuite.py:69
cmsPerfSuite.PerfSuite._debug
_debug
Developer options.
Definition: cmsPerfSuite.py:66

Member Function Documentation

def cmsPerfSuite.PerfSuite.benchmarks (   self,
  cpu,
  pfdir,
  name,
  bencher,
  large = False 
)

Run cmsScimark benchmarks a number of times.

Definition at line 678 of file cmsPerfSuite.py.

References cmsPerfSuite.PerfSuite.Commands, cmsPerfSuite.PerfSuite.printFlush(), and cmsPerfSuite.PerfSuite.runcmd().

678 
679  def benchmarks(self,cpu,pfdir,name,bencher,large=False):
680  cmd = self.Commands[cpu][3]
681  redirect = ""
682  if large:
683  redirect = " -large >>"
684  else:
685  redirect = " >>"
686 
687  for i in range(bencher):
688  #Check first for the existence of the file so that we can append:
689  if not os.path.exists(os.path.join(pfdir,os.path.basename(name))):
690  #Equivalent of touch to make sure the file exist to be able to append to it.
691  open(os.path.join(pfdir,os.path.basename(name)))
692 
693  command= cmd + redirect + os.path.join(pfdir,os.path.basename(name))
694  self.printFlush(command + " [%s/%s]" % (i+1,bencher))
695  self.runcmd(command)
696  self.logh.flush()
cmsPerfSuite.PerfSuite.runcmd
def runcmd
Run a command and return the exit status.
Definition: cmsPerfSuite.py:553
cmsPerfSuite.PerfSuite.benchmarks
def benchmarks
Run cmsScimark benchmarks a number of times.
Definition: cmsPerfSuite.py:678
cmsPerfSuite.PerfSuite.printFlush
def printFlush
Print and flush a string (for output to a log file)
Definition: cmsPerfSuite.py:545
def cmsPerfSuite.PerfSuite.cprootfile (   self,
  dir,
  candle,
  NumOfEvents,
  cmsdriverOptions = "" 
)

Copy root file from another candle's directory ! Again this is messy.

Definition at line 599 of file cmsPerfSuite.py.

References cmsPerfSuite.PerfSuite.printFlush(), and cmsPerfSuite.PerfSuite.runCmdSet().

599 
600  def cprootfile(self,dir,candle,NumOfEvents,cmsdriverOptions=""):
601  cmds = ("cd %s" % dir,
602  "cp -pR ../%s_IgProf/%s_GEN,SIM.root ." % (candle,CandFname[candle]))
603 
604  if self.runCmdSet(cmds):
605  self.logh.write("Since there was no ../%s_IgProf/%s_GEN,SIM.root file it will be generated first\n"%(candle,CandFname[candle]))
606 
607  cmd = "cd %s ; cmsDriver.py %s -s GEN,SIM -n %s --fileout %s_GEN,SIM.root %s>& %s_GEN_SIM_for_valgrind.log" % (dir,KeywordToCfi[candle],str(NumOfEvents),candle,cmsdriverOptions,candle)
608 
609  self.printFlush(cmd)
610  #Obsolete popen4-> subprocess.Popen
611  #cmdout=os.popen3(cmd)[2].read()
612  cmdout=subprocess.Popen(cmd,shell=True,stdout=subprocess.PIPE,stderr=subprocess.STDOUT).stdout.read()
613  if cmdout:
614  self.printFlush(cmdout)
615  return cmdout
cmsPerfSuite.PerfSuite.runCmdSet
def runCmdSet
Run a list of commands using system ! We should rewrite this not to use system (most cases it is unne...
Definition: cmsPerfSuite.py:527
cmsPerfSuite.PerfSuite.printFlush
def printFlush
Print and flush a string (for output to a log file)
Definition: cmsPerfSuite.py:545
cmsPerfSuite.PerfSuite.cprootfile
def cprootfile
Copy root file from another candle's directory ! Again this is messy.
Definition: cmsPerfSuite.py:599
def cmsPerfSuite.PerfSuite.displayErrors (   self,
  file 
)

Display G4 cerr errors and CMSExceptions in the logfile.

Definition at line 619 of file cmsPerfSuite.py.

References cmsPerfSuite.PerfSuite.ERRORS.

Referenced by cmsPerfSuite.PerfSuite.simpleGenReport().

619 
620  def displayErrors(self,file):
621  try:
622  for line in open(file,"r"):
623  if "cerr" in line or "CMSException" in line:
624  self.logh.write("ERROR: %s\n" % line)
625  self.ERRORS += 1
626  except OSError, detail:
627  self.logh.write("WARNING: %s\n" % detail)
628  self.ERRORS += 1
629  except IOError, detail:
630  self.logh.write("WARNING: %s\n" % detail)
631  self.ERRORS += 1
cmsPerfSuite.PerfSuite.displayErrors
def displayErrors
Display G4 cerr errors and CMSExceptions in the logfile.
Definition: cmsPerfSuite.py:619
def cmsPerfSuite.PerfSuite.getDate (   self)

Definition at line 579 of file cmsPerfSuite.py.

Referenced by cmsPerfSuite.PerfSuite.printDate(), and cmsPerfSuite.PerfSuite.runCmdSet().

579 
580  def getDate(self):
581  return time.ctime()
def cmsPerfSuite.PerfSuite.mkCandleDir (   self,
  pfdir,
  candle,
  profiler 
)

Make directory for a particular candle and profiler.

! This is really unnecessary code and should be replaced with a os.mkdir() call

Definition at line 588 of file cmsPerfSuite.py.

References helpers.MassSearchReplaceParamVisitor._verbose, HiHelperTools.MassSearchReplaceParamVisitor._verbose, HTMHTAnalyzer._verbose, helpers.MassSearchReplaceAnyInputTagVisitor._verbose, HiHelperTools.MassSearchReplaceAnyInputTagVisitor._verbose, cmsPerfSuite.PerfSuite._verbose, TcMETAnalyzer._verbose, MuCorrMETAnalyzer._verbose, CaloMETAnalyzer._verbose, PFMETAnalyzer._verbose, METAnalyzer._verbose, ConfigBuilder.ConfigBuilder.MassSearchReplaceProcessNameVisitor._verbose, cmsPerfSuite.PerfSuite.printDate(), and cmsPerfSuite.PerfSuite.runcmd().

Referenced by cmsPerfSuite.PerfSuite.simpleGenReport().

588 
589  def mkCandleDir(self,pfdir,candle,profiler):
590  adir = os.path.join(pfdir,"%s_%s" % (candle,profiler))
591  self.runcmd( "mkdir -p %s" % adir )
592  if self._verbose:
593  self.printDate()
594  return adir
cmsPerfSuite.PerfSuite.runcmd
def runcmd
Run a command and return the exit status.
Definition: cmsPerfSuite.py:553
cmsPerfSuite.PerfSuite.mkCandleDir
def mkCandleDir
Make directory for a particular candle and profiler.
Definition: cmsPerfSuite.py:588
cmsPerfSuite.PerfSuite._verbose
_verbose
Check logfile option.
Definition: cmsPerfSuite.py:69
def cmsPerfSuite.PerfSuite.optionParse (   self,
  argslist = None 
)

Definition at line 146 of file cmsPerfSuite.py.

References cmsPerfSuite.PerfSuite._CASTOR_DIR, EcalMatacqAnalyzer._debug, LMFDefFabric._debug, cmsPerfSuite.PerfSuite._debug, EcalABAnalyzer._debug, EcalLaserAnalyzer2._debug, EcalLaserAnalyzer._debug, DTBtiCard._debug, DTTracoCard._debug, DTTrigGeom._debug, DTTrig._debug, cmsPerfSuite.PerfSuite._dryrun, cmsPerfSuite.PerfSuite._noexec, cmsPerfSuite.PerfSuite._unittest, helpers.MassSearchReplaceParamVisitor._verbose, HiHelperTools.MassSearchReplaceParamVisitor._verbose, HTMHTAnalyzer._verbose, helpers.MassSearchReplaceAnyInputTagVisitor._verbose, HiHelperTools.MassSearchReplaceAnyInputTagVisitor._verbose, cmsPerfSuite.PerfSuite._verbose, MuCorrMETAnalyzer._verbose, TcMETAnalyzer._verbose, CaloMETAnalyzer._verbose, PFMETAnalyzer._verbose, METAnalyzer._verbose, ConfigBuilder.ConfigBuilder.MassSearchReplaceProcessNameVisitor._verbose, cmsCpuInfo.get_NumOfCores(), join(), list(), python.multivaluedict.map(), EcalTPGParamReaderFromDB.user, popcon::RPCObPVSSmapData.user, popcon::RpcObGasData.user, popcon::RpcDataT.user, popcon::RpcDataV.user, popcon::RpcDataGasMix.user, popcon::RpcDataFebmap.user, popcon::RpcDataI.user, popcon::RpcDataUXC.user, popcon::RpcDataS.user, cmsPerfSuite.PerfSuite.user, and EcalDBConnection.user.

146 
147  def optionParse(self,argslist=None):
148  parser = opt.OptionParser(usage='''./cmsPerfSuite.py [options]
149 
150  Examples:
151 
152  cmsPerfSuite.py --step GEN-HLT -t 5 -i 2 -c 1 -m 5 --RunTimeSize MinBias,TTbar --RunIgProf TTbar --RunCallgrind TTbar --RunMemcheck TTbar --RunDigiPileUp TTbar --PUInputFile /store/relval/CMSSW_2_2_1/RelValMinBias/GEN-SIM-DIGI-RAW-HLTDEBUG/IDEAL_V9_v2/0001/101C84AF-56C4-DD11-A90D-001D09F24EC0.root --cmsdriver="--eventcontent FEVTDEBUGHLT --conditions FrontierConditions_GlobalTag,IDEAL_V9::All"
153  (this will run the suite with 5 events for TimeSize tests on MinBias and TTbar, 2 for IgProf tests on TTbar only, 1 for Callgrind tests on TTbar only, 5 for Memcheck on MinBias and TTbar, it will also run DIGI PILEUP for all TTbar tests defined, i.e. 5 TimeSize, 2 IgProf, 1 Callgrind, 5 Memcheck. The file /store/relval/CMSSW_2_2_1/RelValMinBias/GEN-SIM-DIGI-RAW-HLTDEBUG/IDEAL_V9_v2/0001/101C84AF-56C4-DD11-A90D-001D09F24EC0.root will be copied locally as INPUT_PILEUP_EVENTS.root and it will be used as the input file for the MixingModule pile up events. All these tests will be done for the step GEN-HLT, i.e. GEN,SIM,DIGI,L1,DIGI2RAW,HLT at once)
154  OR
155  cmsPerfSuite.py --step GEN-HLT -t 5 -i 2 -c 1 -m 5 --RunTimeSize MinBias,TTbar --RunIgProf TTbar --RunCallgrind TTbar --RunMemcheck TTbar --RunTimeSizePU TTbar --PUInputFile /store/relval/CMSSW_2_2_1/RelValMinBias/GEN-SIM-DIGI-RAW-HLTDEBUG/IDEAL_V9_v2/0001/101C84AF-56C4-DD11-A90D-001D09F24EC0.root
156  (this will run the suite with 5 events for TimeSize tests on MinBias and TTbar, 2 for IgProf tests on TTbar only, 1 for Callgrind tests on TTbar only, 5 for Memcheck on MinBias and TTbar, it will also run DIGI PILEUP on TTbar but only for 5 TimeSize events. All these tests will be done for the step GEN-HLT, i.e. GEN,SIM,DIGI,L1,DIGI2RAW,HLT at once)
157  OR
158  cmsPerfSuite.py --step GEN-HLT -t 5 -i 2 -c 1 -m 5 --RunTimeSize MinBias,TTbar --RunIgProf TTbar --RunCallgrind TTbar --RunMemcheck TTbar --RunTimeSizePU TTbar --PUInputFile /store/relval/CMSSW_2_2_1/RelValMinBias/GEN-SIM-DIGI-RAW-HLTDEBUG/IDEAL_V9_v2/0001/101C84AF-56C4-DD11-A90D-001D09F24EC0.root --cmsdriver="--eventcontent RAWSIM --conditions FrontierConditions_GlobalTag,IDEAL_V9::All"
159  (this will run the suite with 5 events for TimeSize tests on MinBias and TTbar, 2 for IgProf tests on TTbar only, 1 for Callgrind tests on TTbar only, 5 for Memcheck on MinBias and TTbar, it will also run DIGI PILEUP on TTbar but only for 5 TimeSize events. All these tests will be done for the step GEN-HLT, i.e. GEN,SIM,DIGI,L1,DIGI2RAW,HLT at once. It will also add the options "--eventcontent RAWSIM --conditions FrontierConditions_GlobalTag,IDEAL_V9::All" to all cmsDriver.py commands executed by the suite. In addition it will run only 2 cmsDriver.py "steps": "GEN,SIM" and "DIGI". Note the syntax GEN-SIM for combined cmsDriver.py steps)
160 
161  Legal entries for individual candles (--RunTimeSize, --RunIgProf, --RunCallgrind, --RunMemcheck options):
162  %s
163  ''' % ("\n".join(Candles)))
164 
165  parser.set_defaults(TimeSizeEvents = 0 ,
166  IgProfEvents = 0 ,
167  CallgrindEvents = 0 ,
168  MemcheckEvents = 0 ,
169  cmsScimark = 10 ,
170  cmsScimarkLarge = 10 ,
171  cmsdriverOptions = "--eventcontent FEVTDEBUGHLT", # Decided to avoid using the automatic parsing of cmsDriver_highstats_hlt.txt: cmsRelValCmd.get_cmsDriverOptions(), #Get these options automatically now!
172  #"Release Integrators" will create another file relative to the performance suite and the operators will fetch from that file the --cmsdriver option... for now just set the eventcontent since that is needed in order for things to run at all now...
173  stepOptions = "" ,
174  profilers = "" ,
175  outputdir = "" ,
176  logfile = os.path.join(os.getcwd(),"cmsPerfSuite.log"),
177  runonspare = True ,
178  bypasshlt = False ,
179  quicktest = False ,
180  unittest = False ,
181  noexec = False ,
182  dryrun = False ,
183  verbose = True ,
184  previousrel = "" ,
185  castordir = self._CASTOR_DIR,
186  cores = cmsCpuInfo.get_NumOfCores(), #Get Number of cpu cores on the machine from /proc/cpuinfo
187  cpu = "1" , #Cpu core on which the suite is run:
188  RunTimeSize = "" ,
189  RunIgProf = "" ,
190  RunCallgrind = "" ,
191  RunMemcheck = "" ,
192  RunDigiPileUP = "" ,
193  RunTimeSizePU = "" ,
194  RunIgProfPU = "" ,
195  RunCallgrindPU = "" ,
196  RunMemcheckPU = "" ,
197  PUInputFile = "" ,
198  userInputFile = "" )
199  parser.add_option('-q', '--quiet' , action="store_false", dest='verbose' ,
200  help = 'Output less information' )
201  parser.add_option('-b', '--bypass-hlt' , action="store_true" , dest='bypasshlt' ,
202  help = 'Bypass HLT root file as input to RAW2DIGI')
203  parser.add_option('-n', '--notrunspare', action="store_false", dest='runonspare',
204  help = 'Do not run cmsScimark on spare cores')
205  parser.add_option('-t', '--timesize' , type='int' , dest='TimeSizeEvents' , metavar='<#EVENTS>' ,
206  help = 'specify the number of events for the TimeSize tests' )
207  parser.add_option('-i', '--igprof' , type='int' , dest='IgProfEvents' , metavar='<#EVENTS>' ,
208  help = 'specify the number of events for the IgProf tests' )
209  parser.add_option('-c', '--callgrind' , type='int' , dest='CallgrindEvents' , metavar='<#EVENTS>' ,
210  help = 'specify the number of events for the Callgrind tests' )
211  parser.add_option('-m', '--memcheck' , type='int' , dest='MemcheckEvents' , metavar='<#EVENTS>' ,
212  help = 'specify the number of events for the Memcheck tests' )
213  parser.add_option('--cmsScimark' , type='int' , dest='cmsScimark' , metavar='' ,
214  help = 'specify the number of times the cmsScimark benchmark is run before and after the performance suite on cpu1')
215  parser.add_option('--cmsScimarkLarge' , type='int' , dest='cmsScimarkLarge' , metavar='' ,
216  help = 'specify the number of times the cmsScimarkLarge benchmark is run before and after the performance suite on cpu1')
217  parser.add_option('--cores' , type='int', dest='cores' , metavar='<CORES>' ,
218  help = 'specify the number of cores of the machine (can be used with 0 to stop cmsScimark from running on the other cores)')
219  parser.add_option('--cmsdriver' , type='string', dest='cmsdriverOptions', metavar='<OPTION_STR>',
220  help = 'specify special options to use with the cmsDriver.py commands (designed for integration build use')
221  parser.add_option('-a', '--archive' , type='string', dest='castordir' , metavar='<DIR>' ,
222  help = 'specify the wanted CASTOR directory where to store the results tarball')
223  parser.add_option('-L', '--logfile' , type='string', dest='logfile' , metavar='<FILE>' ,
224  help = 'file to store log output of the script')
225  parser.add_option('-o', '--output' , type='string', dest='outputdir' , metavar='<DIR>' ,
226  help = 'specify the directory where to store the output of the script')
227  parser.add_option('-r', '--prevrel' , type='string', dest='previousrel' , metavar='<DIR>' ,
228  help = 'Top level dir of previous release for regression analysis')
229  parser.add_option('--step' , type='string', dest='stepOptions' , metavar='<STEPS>' ,
230  help = 'specify the processing steps intended (instead of the default ones)' )
231  parser.add_option('--cpu' , type='string', dest='cpu' , metavar='<CPU>' ,
232  help = 'specify the core on which to run the performance suite')
233 
234  #Adding new options to put everything configurable at command line:
235  parser.add_option('--RunTimeSize' , type='string', dest='RunTimeSize' , metavar='<CANDLES>' ,
236  help = 'specify on which candles to run the TimeSize tests')
237  parser.add_option('--RunIgProf' , type='string', dest='RunIgProf' , metavar='<CANDLES>' ,
238  help = 'specify on which candles to run the IgProf tests')
239  parser.add_option('--RunCallgrind' , type='string', dest='RunCallgrind' , metavar='<CANDLES>' ,
240  help = 'specify on which candles to run the Callgrind tests')
241  parser.add_option('--RunMemcheck' , type='string', dest='RunMemcheck' , metavar='<CANDLES>' ,
242  help = 'specify on which candles to run the Memcheck tests')
243  parser.add_option('--RunDigiPileUp' , type='string', dest='RunDigiPileUp' , metavar='<CANDLES>' ,
244  help = 'specify the candle on which to run DIGI PILE UP and repeat all the tests set to run on that candle with PILE UP')
245  parser.add_option('--PUInputFile' , type='string', dest='PUInputFile' , metavar='<FILE>' ,
246  help = 'specify the root file to pick the pile-up events from')
247  parser.add_option('--RunTimeSizePU' , type='string', dest='RunTimeSizePU' , metavar='<CANDLES>' ,
248  help = 'specify on which candles to run the TimeSize tests with PILE UP')
249  parser.add_option('--RunIgProfPU' , type='string', dest='RunIgProfPU' , metavar='<CANDLES>' ,
250  help = 'specify on which candles to run the IgProf tests with PILE UP')
251  parser.add_option('--RunCallgrindPU' , type='string', dest='RunCallgrindPU' , metavar='<CANDLES>' ,
252  help = 'specify on which candles to run the Callgrind tests with PILE UP')
253  parser.add_option('--RunMemcheckPU' , type='string', dest='RunMemcheckPU' , metavar='<CANDLES>' ,
254  help = 'specify on which candles to run the Memcheck tests with PILE UP')
255 
256  #Adding a filein option to use pre-processed RAW file for RECO and HLT:
257  parser.add_option('--filein' , type='string', dest='userInputFile' , metavar='<FILE>', #default="",
258  help = 'specify input RAW root file for HLT and RAW2DIGI-RECO (list the files in the same order as the candles for the tests)')
259 
260  #Adding an option to handle additional (to the default user) email addresses to the email notification list (that sends the cmsPerfSuite.log once the performance suite is done running):
261  parser.add_option('--mail', type='string', dest='MailLogRecipients', metavar='<EMAIL ADDRESS>', default=self.user, help='specify valid email address(es) name@domain in order to receive notification at the end of the performance suite running with the cmsPerfSuite.log file')
262 
263  #Adding option to turn off tarball creation at the end of the execution of the performance suite:
264  parser.add_option('--no_tarball', action="store_false", dest='tarball', default=True, help='Turn off automatic tarball creation at the end of the performance suite execution')
265 
266  #####################
267  #
268  # Developer options
269  #
270 
271  devel = opt.OptionGroup(parser, "Developer Options",
272  "Caution: use these options at your own risk."
273  "It is believed that some of them bite.\n")
274 
275  devel.add_option('-p', '--profile' , type="str" , dest='profilers', metavar="<PROFILERS>" ,
276  help = 'Profile codes to use for cmsRelvalInput' )
277  devel.add_option('-f', '--false-run', action="store_true", dest='dryrun' ,
278  help = 'Dry run' )
279  devel.add_option('-d', '--debug' , action='store_true', dest='debug' ,
280  help = 'Debug' )
281  devel.add_option('--quicktest' , action="store_true", dest='quicktest',
282  help = 'Quick overwrite all the defaults to small numbers so that we can run a quick test of our chosing.' )
283  devel.add_option('--test' , action="store_true", dest='unittest' ,
284  help = 'Perform a simple test, overrides other options. Overrides verbosity and sets it to false.' )
285  devel.add_option('--no_exec' , action="store_true", dest='noexec' ,
286  help = 'Run the suite without executing the cmsRelvalreport.py commands in the various directories. This is a useful debugging tool.' )
287  parser.add_option_group(devel)
288  (options, args) = parser.parse_args(argslist)
289 
290 
291  self._debug = options.debug
292  self._unittest = options.unittest
293  self._noexec = options.noexec
294  self._verbose = options.verbose
295  self._dryrun = options.dryrun
296  castordir = options.castordir
297  TimeSizeEvents = options.TimeSizeEvents
298  IgProfEvents = options.IgProfEvents
299  CallgrindEvents = options.CallgrindEvents
300  MemcheckEvents = options.MemcheckEvents
301  cmsScimark = options.cmsScimark
302  cmsScimarkLarge = options.cmsScimarkLarge
303  cmsdriverOptions = options.cmsdriverOptions
304  stepOptions = options.stepOptions
305  quicktest = options.quicktest
306  #candleoption = options.candleOptions
307  runonspare = options.runonspare
308  profilers = options.profilers.strip()
309  cpu = options.cpu.strip()
310  bypasshlt = options.bypasshlt
311  cores = options.cores
312  logfile = options.logfile
313  prevrel = options.previousrel
314  outputdir = options.outputdir
315  RunTimeSize = options.RunTimeSize
316  RunIgProf = options.RunIgProf
317  RunCallgrind = options.RunCallgrind
318  RunMemcheck = options.RunMemcheck
319  RunDigiPileUp = options.RunDigiPileUp
320  RunTimeSizePU = options.RunTimeSizePU
321  RunIgProfPU = options.RunIgProfPU
322  RunCallgrindPU = options.RunCallgrindPU
323  RunMemcheckPU = options.RunMemcheckPU
324  PUInputFile = options.PUInputFile
325  userInputFile = options.userInputFile
326  if options.MailLogRecipients !="" and self.user not in options.MailLogRecipients: #To allow for the --mail "" case of suppressing the email and the default user case
327  MailLogRecipients= self.user+","+options.MailLogRecipients #Add the user by default if there is a mail report
328  else:
329  MailLogRecipients=options.MailLogRecipients
330  tarball = options.tarball
331 
332  #################
333  # Check logfile option
334  #
335  if not logfile == None:
336  logfile = os.path.abspath(logfile)
337  logdir = os.path.dirname(logfile)
338  if not os.path.exists(logdir):
339  parser.error("Directory to output logfile does not exist")
340  sys.exit()
341  logfile = os.path.abspath(logfile)
342 
343  #############
344  # Check step Options
345  #
346  if "GEN,SIM" in stepOptions:
347  self.logh.write("WARNING: Please use GEN-SIM with a hypen not a \",\"!\n")
348  #Using the step option as a switch between different dictionaries for:
349  #RunTimeSize,RunIgProf,RunCallgrind,RunMemCheck,RunDigiPileUp:
350  if stepOptions == "" or stepOptions == 'Default':
351  pass
352  else:
353  stepOptions='--usersteps=%s' % (stepOptions)
354 
355  ###############
356  # Check profile option
357  #
358  isnumreg = re.compile("^-?[0-9]*$")
359  found = isnumreg.search(profilers)
360  if not found :
361  parser.error("profile codes option contains non-numbers")
362  sys.exit()
363 
364  ###############
365  # Check output directory option
366  #
367  if outputdir == "":
368  outputdir = os.getcwd()
369  else:
370  outputdir = os.path.abspath(outputdir)
371 
372  if not os.path.isdir(outputdir):
373  parser.error("%s is not a valid output directory" % outputdir)
374  sys.exit()
375 
376  ################
377  # Check cpu option
378  #
379  numetcomreg = re.compile("^[0-9,]*")
380  if not numetcomreg.search(cpu):
381  parser.error("cpu option needs to be a comma separted list of ints or a single int")
382  sys.exit()
383 
384  cpustr = cpu
385  cpu = []
386  if "," in cpustr:
387  cpu = map(lambda x: int(x),cpustr.split(","))
388  else:
389  cpu = [ int(cpustr) ]
390 
391  ################
392  # Check previous release directory
393  #
394  if not prevrel == "":
395  prevrel = os.path.abspath(prevrel)
396  if not os.path.exists(prevrel):
397  self.logh.write("ERROR: Previous release dir %s could not be found" % prevrel)
398  sys.exit()
399 
400  #############
401  # Setup quicktest option
402  #
403  if quicktest:
404  TimeSizeEvents = 1
405  IgProfEvents = 1
406  CallgrindEvents = 0
407  MemcheckEvents = 0
408  cmsScimark = 1
409  cmsScimarkLarge = 1
410 
411  #############
412  # Setup unit test option
413  #
414  if self._unittest:
415  self._verbose = False
416  if stepOptions == "":
417  stepOptions = "GEN-SIM,DIGI,L1,DIGI2RAW,HLT,RAW2DIGI-RECO"
418  cmsScimark = 0
419  cmsScimarkLarge = 0
420  CallgrindEvents = 0
421  MemcheckEvents = 0
422  IgProfEvents = 0
423  TimeSizeEvents = 1
424 
425  #Split all the RunTimeSize etc candles in lists:
426  TimeSizeCandles=[]
427  IgProfCandles=[]
428  CallgrindCandles=[]
429  MemcheckCandles=[]
430  TimeSizePUCandles=[]
431  IgProfPUCandles=[]
432  CallgrindPUCandles=[]
433  MemcheckPUCandles=[]
434  userInputRootFiles=[]
435  if RunTimeSize:
436  TimeSizeCandles = RunTimeSize.split(",")
437  if RunIgProf:
438  IgProfCandles = RunIgProf.split(",")
439  if RunCallgrind:
440  CallgrindCandles = RunCallgrind.split(",")
441  if RunMemcheck:
442  MemcheckCandles = RunMemcheck.split(",")
443  if RunDigiPileUp:
444  for candle in RunDigiPileUp.split(","):
445  if candle in TimeSizeCandles:
446  TimeSizePUCandles.append(candle)
447  if candle in IgProfCandles:
448  IgProfPUCandles.append(candle)
449  if candle in CallgrindCandles:
450  CallgrindPUCandles.append(candle)
451  if candle in MemcheckCandles:
452  MemcheckPUCandles.append(candle)
453  if RunTimeSizePU:
454  TimeSizePUCandles.extend(RunTimeSizePU.split(","))
455  #Some smart removal of duplicates from the list!
456  temp=set(TimeSizePUCandles)
457  TimeSizePUCandles=list(temp) #Doing it in 2 steps to avoid potential issues with type of arguments
458  if RunIgProfPU:
459  IgProfPUCandles.extend(RunIgProfPU.split(","))
460  #Some smart removal of duplicates from the list!
461  temp=set(IgProfPUCandles)
462  IgProfPUCandles=list(temp) #Doing it in 2 steps to avoid potential issues with type of arguments
463  if RunCallgrindPU:
464  CallgrindPUCandles.extend(RunCallgrindPU.split(","))
465  #Some smart removal of duplicates from the list!
466  temp=set(CallgrindPUCandles)
467  CallgrindPUCandles=list(temp) #Doing it in 2 steps to avoid potential issues with type of arguments
468  if RunMemcheckPU:
469  MemcheckPUCandles.extend(RunMemcheckPU.split(","))
470  #Some smart removal of duplicates from the list!
471  temp=set(MemcheckPUCandles)
472  MemcheckPUCandles=list(temp) #Doing it in 2 steps to avoid potential issues with type of arguments
473  if userInputFile:
474  userInputRootFiles=userInputFile.split(",")
475 
476 
477 
478  #############
479  # Setup cmsdriver and eventual cmsdriverPUoption
480  #
481  cmsdriverPUOptions=""
482  if cmsdriverOptions:
483  #Set the eventual Pile Up cmsdriver options first:
484  if TimeSizePUCandles or IgProfPUCandles or CallgrindPUCandles or MemcheckPUCandles:
485  #Bug fixed: no space between --pileup= and LowLumiPileUp (otherwise could omit the =)
486  cmsdriverPUOptions = '--cmsdriver="%s %s%s"'%(cmsdriverOptions," --pileup=",cmsDriverPileUpOption)
487  #Set the regular ones too:
488  cmsdriverOptions = '--cmsdriver="%s"'%cmsdriverOptions
489 
490  return (castordir ,
491  TimeSizeEvents ,
492  IgProfEvents ,
493  CallgrindEvents ,
494  MemcheckEvents ,
495  cmsScimark ,
496  cmsScimarkLarge ,
497  cmsdriverOptions,
498  cmsdriverPUOptions,
499  stepOptions ,
500  quicktest ,
501  profilers ,
502  cpu ,
503  cores ,
504  prevrel ,
505  bypasshlt ,
506  runonspare ,
507  outputdir ,
508  logfile ,
509  TimeSizeCandles ,
510  IgProfCandles ,
511  CallgrindCandles,
512  MemcheckCandles ,
513  TimeSizePUCandles ,
514  IgProfPUCandles ,
515  CallgrindPUCandles,
516  MemcheckPUCandles ,
517  PUInputFile ,
518  userInputRootFiles,
519  MailLogRecipients,
520  tarball)
cmsPerfSuite.PerfSuite._verbose
_verbose
Check logfile option.
Definition: cmsPerfSuite.py:69
cmsCpuInfo.get_NumOfCores
def get_NumOfCores
Definition: cmsCpuInfo.py:6
join
static std::string join(char **cmd)
Definition: RemoteFile.cc:18
cmsPerfSuite.PerfSuite._debug
_debug
Developer options.
Definition: cmsPerfSuite.py:66
list
How EventSelector::AcceptEvent() decides whether to accept an event for output otherwise it is excluding the probing of A single or multiple positive and the trigger will pass if any such matching triggers are PASS or EXCEPTION[A criterion thatmatches no triggers at all is detected and causes a throw.] A single negative with an expectation of appropriate bit checking in the decision and the trigger will pass if any such matching triggers are FAIL or EXCEPTION A wildcarded negative criterion that matches more than one trigger in the trigger list("!*","!HLTx*"if it matches 2 triggers or more) will accept the event if all the matching triggers are FAIL.It will reject the event if any of the triggers are PASS or EXCEPTION(this matches the behavior of"!*"before the partial wildcard feature was incorporated).Triggers which are in the READY state are completely ignored.(READY should never be returned since the trigger paths have been run
def cmsPerfSuite.PerfSuite.printDate (   self)

Definition at line 582 of file cmsPerfSuite.py.

References FittedEntriesSet.getDate(), and cmsPerfSuite.PerfSuite.getDate().

Referenced by cmsPerfSuite.PerfSuite.mkCandleDir().

582 
583  def printDate(self):
584  self.logh.write(self.getDate() + "\n")
self.logh.flush()
def cmsPerfSuite.PerfSuite.printFlush (   self,
  command 
)

Print and flush a string (for output to a log file)

Definition at line 545 of file cmsPerfSuite.py.

References helpers.MassSearchReplaceParamVisitor._verbose, HiHelperTools.MassSearchReplaceParamVisitor._verbose, HTMHTAnalyzer._verbose, helpers.MassSearchReplaceAnyInputTagVisitor._verbose, HiHelperTools.MassSearchReplaceAnyInputTagVisitor._verbose, cmsPerfSuite.PerfSuite._verbose, MuCorrMETAnalyzer._verbose, TcMETAnalyzer._verbose, CaloMETAnalyzer._verbose, PFMETAnalyzer._verbose, METAnalyzer._verbose, and ConfigBuilder.ConfigBuilder.MassSearchReplaceProcessNameVisitor._verbose.

Referenced by cmsPerfSuite.PerfSuite.benchmarks(), cmsPerfSuite.PerfSuite.cprootfile(), cmsPerfSuite.PerfSuite.runCmdSet(), and cmsPerfSuite.PerfSuite.simpleGenReport().

545 
546  def printFlush(self,command):
547  if self._verbose:
548  self.logh.write(str(command) + "\n")
549  self.logh.flush()
cmsPerfSuite.PerfSuite._verbose
_verbose
Check logfile option.
Definition: cmsPerfSuite.py:69
cmsPerfSuite.PerfSuite.printFlush
def printFlush
Print and flush a string (for output to a log file)
Definition: cmsPerfSuite.py:545
def cmsPerfSuite.PerfSuite.runcmd (   self,
  command 
)

Run a command and return the exit status.

Definition at line 553 of file cmsPerfSuite.py.

References helpers.MassSearchReplaceParamVisitor._verbose, HiHelperTools.MassSearchReplaceParamVisitor._verbose, HTMHTAnalyzer._verbose, helpers.MassSearchReplaceAnyInputTagVisitor._verbose, HiHelperTools.MassSearchReplaceAnyInputTagVisitor._verbose, cmsPerfSuite.PerfSuite._verbose, MuCorrMETAnalyzer._verbose, TcMETAnalyzer._verbose, CaloMETAnalyzer._verbose, PFMETAnalyzer._verbose, METAnalyzer._verbose, and ConfigBuilder.ConfigBuilder.MassSearchReplaceProcessNameVisitor._verbose.

Referenced by cmsPerfSuite.PerfSuite.benchmarks(), cmsPerfSuite.PerfSuite.mkCandleDir(), and cmsPerfSuite.PerfSuite.runCmdSet().

553 
554  def runcmd(self,command):
555  #Substitute popen with subprocess.Popen!
556  #Using try/except until Popen becomes thread safe (it seems that everytime it is called
557  #all processes are checked to reap the ones that are done, this creates a race condition with the wait()... that
558  #results into an error with "No child process".
559  #os.popen(command)
560  try:
561  process = subprocess.Popen(command,shell=True,stdout=subprocess.PIPE,stderr=subprocess.STDOUT)
562  pid=process.pid
563  exitstat= process.wait()
564  cmdout = process.stdout.read()
565  exitstat = process.returncode
566  except OSError, detail:
567  self.logh.write("Race condition in subprocess.Popen has robbed us of the exit code of the %s process (PID %s).Assume it failed!\n %s\n"%(command,pid,detail))
568  self.logh.flush()
569  exitstat=999
570  cmdout="Race condition in subprocess.Popen has robbed us of the exit code of the %s process (PID %s).Assume it failed!\n %s"%(command,pid,detail)
571  if self._verbose:
572  self.logh.write(cmdout)# + "\n") No need of extra \n!
573  self.logh.flush()
574  if exitstat == None:
575  self.logh.write("Something strange is going on! Exit code was None for command %s: check if it really ran!"%command)
576  self.logh.flush()
577  exitstat=0
578  return exitstat
cmsPerfSuite.PerfSuite.runcmd
def runcmd
Run a command and return the exit status.
Definition: cmsPerfSuite.py:553
cmsPerfSuite.PerfSuite._verbose
_verbose
Check logfile option.
Definition: cmsPerfSuite.py:69
def cmsPerfSuite.PerfSuite.runCmdSet (   self,
  cmd 
)

Run a list of commands using system ! We should rewrite this not to use system (most cases it is unnecessary)

Definition at line 527 of file cmsPerfSuite.py.

References helpers.MassSearchReplaceParamVisitor._verbose, HiHelperTools.MassSearchReplaceParamVisitor._verbose, HTMHTAnalyzer._verbose, helpers.MassSearchReplaceAnyInputTagVisitor._verbose, HiHelperTools.MassSearchReplaceAnyInputTagVisitor._verbose, cmsPerfSuite.PerfSuite._verbose, TcMETAnalyzer._verbose, MuCorrMETAnalyzer._verbose, CaloMETAnalyzer._verbose, PFMETAnalyzer._verbose, METAnalyzer._verbose, ConfigBuilder.ConfigBuilder.MassSearchReplaceProcessNameVisitor._verbose, FittedEntriesSet.getDate(), cmsPerfSuite.PerfSuite.getDate(), join(), cmsPerfSuite.PerfSuite.printFlush(), and cmsPerfSuite.PerfSuite.runcmd().

Referenced by cmsPerfSuite.PerfSuite.cprootfile(), cmsPerfSuite.PerfSuite.runCmsInput(), cmsPerfSuite.PerfSuite.runCmsReport(), and cmsPerfSuite.PerfSuite.testCmsDriver().

527 
528  def runCmdSet(self,cmd):
529  exitstat = 0
530  if len(cmd) <= 1:
531  exitstat = self.runcmd(cmd)
532  if self._verbose:
533  self.printFlush(cmd)
534  else:
535  for subcmd in cmd:
536  if self._verbose:
537  self.printFlush(subcmd)
538  exitstat = self.runcmd(" && ".join(cmd))
539  if self._verbose:
540  self.printFlush(self.getDate())
541  return exitstat
cmsPerfSuite.PerfSuite.runcmd
def runcmd
Run a command and return the exit status.
Definition: cmsPerfSuite.py:553
cmsPerfSuite.PerfSuite.runCmdSet
def runCmdSet
Run a list of commands using system ! We should rewrite this not to use system (most cases it is unne...
Definition: cmsPerfSuite.py:527
cmsPerfSuite.PerfSuite._verbose
_verbose
Check logfile option.
Definition: cmsPerfSuite.py:69
cmsPerfSuite.PerfSuite.printFlush
def printFlush
Print and flush a string (for output to a log file)
Definition: cmsPerfSuite.py:545
join
static std::string join(char **cmd)
Definition: RemoteFile.cc:18
def cmsPerfSuite.PerfSuite.runCmsInput (   self,
  cpu,
  dir,
  numevents,
  candle,
  cmsdrvopts,
  stepopt,
  profiles,
  bypasshlt,
  userInputFile 
)

Wrapper for cmsRelvalreportInput.

Definition at line 750 of file cmsPerfSuite.py.

References cmsPerfSuite.PerfSuite._unittest, cmsPerfSuite.PerfSuite.Commands, and cmsPerfSuite.PerfSuite.runCmdSet().

Referenced by cmsPerfSuite.PerfSuite.simpleGenReport().

750 
751  def runCmsInput(self,cpu,dir,numevents,candle,cmsdrvopts,stepopt,profiles,bypasshlt,userInputFile):
752 
753  #Crappy fix for optional options with special synthax (bypasshlt and userInputFile)
754  bypass = ""
755  if bypasshlt:
756  bypass = "--bypass-hlt"
757  userInputFileOption=""
758  if userInputFile:
759  userInputFileOption = "--filein %s"%userInputFile
760  cmd = self.Commands[cpu][2]
761  cmds=[]
762  #print cmds
763  cmds = ("cd %s" % (dir),
764  "%s %s \"%s\" %s %s %s %s %s" % (cmd,
765  numevents,
766  candle,
767  profiles,
768  cmsdrvopts,
769  stepopt,
770  bypass,userInputFileOption))
771  exitstat=0
772  exitstat = self.runCmdSet(cmds)
773  if self._unittest and (not exitstat == 0):
774  self.logh.write("ERROR: CMS Report Input returned a non-zero exit status \n" )
return exitstat
cmsPerfSuite.PerfSuite.runCmdSet
def runCmdSet
Run a list of commands using system ! We should rewrite this not to use system (most cases it is unne...
Definition: cmsPerfSuite.py:527
cmsPerfSuite.PerfSuite.runCmsInput
def runCmsInput
Wrapper for cmsRelvalreportInput.
Definition: cmsPerfSuite.py:750
def cmsPerfSuite.PerfSuite.runCmsReport (   self,
  cpu,
  dir,
  candle 
)

This function is a wrapper around cmsRelvalreport.

Definition at line 700 of file cmsPerfSuite.py.

References EcalMatacqAnalyzer._debug, LMFDefFabric._debug, cmsPerfSuite.PerfSuite._debug, EcalABAnalyzer._debug, EcalLaserAnalyzer2._debug, EcalLaserAnalyzer._debug, DTBtiCard._debug, DTTracoCard._debug, DTTrigGeom._debug, DTTrig._debug, cmsPerfSuite.PerfSuite._unittest, cmsPerfSuite.PerfSuite.cmssw_version, cmsPerfSuite.PerfSuite.Commands, mathSSE.return(), and cmsPerfSuite.PerfSuite.runCmdSet().

Referenced by cmsPerfSuite.PerfSuite.simpleGenReport().

700 
701  def runCmsReport(self,cpu,dir,candle):
702  cmd = self.Commands[cpu][1]
703  cmds = ("cd %s" % (dir),
704  "%s -i SimulationCandles_%s.txt -t perfreport_tmp -R -P >& %s.log" % (cmd,self.cmssw_version,candle))
705  exitstat = 0
706  if not self._debug:
707  exitstat = self.runCmdSet(cmds)
708 
709  if self._unittest and (not exitstat == 0):
710  self.logh.write("ERROR: CMS Report returned a non-zero exit status \n")
711  sys.exit(exitstat)
712  else:
713  return(exitstat) #To return the exit code of the cmsRelvalreport.py commands to the runPerfSuite function
cmsPerfSuite.PerfSuite.runCmdSet
def runCmdSet
Run a list of commands using system ! We should rewrite this not to use system (most cases it is unne...
Definition: cmsPerfSuite.py:527
mathSSE::return
return((rh^lh)&mask)
cmsPerfSuite.PerfSuite._debug
_debug
Developer options.
Definition: cmsPerfSuite.py:66
cmsPerfSuite.PerfSuite.runCmsReport
def runCmsReport
This function is a wrapper around cmsRelvalreport.
Definition: cmsPerfSuite.py:700
def cmsPerfSuite.PerfSuite.runPerfSuite (   self,
  castordir = "/castor/cern.ch/cms/store/relval/performance/",
  TimeSizeEvents = 100,
  IgProfEvents = 5,
  CallgrindEvents = 1,
  MemcheckEvents = 5,
  cmsScimark = 10,
  cmsScimarkLarge = 10,
  cmsdriverOptions = "",
  cmsdriverPUOptions = "",
  stepOptions = "",
  quicktest = False,
  profilers = "",
  cpus = [1],
  cores = 4,
  prevrel = "",
  bypasshlt = False,
  runonspare = True,
  perfsuitedir = os.getcwd(),
  logfile = None,
  TimeSizeCandles = "",
  IgProfCandles = "",
  CallgrindCandles = "",
  MemcheckCandles = "",
  TimeSizePUCandles = "",
  IgProfPUCandles = "",
  CallgrindPUCandles = "",
  MemcheckPUCandles = "",
  PUInputFile = "",
  userInputFile = "",
  MailLogRecipients = "",
  tarball = "" 
)

Runs benchmarking, cpu spinlocks on spare cores and profiles selected candles.

FIXME: Could redesign interface of functions to use keyword arguments: def runPerfSuite(**opts): then instead of using castordir variable, would use opts['castordir'] etc

Definition at line 908 of file cmsPerfSuite.py.

References cmsPerfSuite.PerfSuite.logh.

Referenced by cmsPerfSuite.PerfSuite.simpleGenReport().

908 
909  tarball = "" ):
910 
911  #Set up a variable for the FinalExitCode to be used as the sum of exit codes:
912  FinalExitCode=0
913 
914  #Set up the logfile first!
915  if not logfile == None:
916  try:
917  self.logh = open(logfile,"a")
918  except (OSError, IOError), detail:
919  self.logh.write(detail + "\n")
920  self.logh.flush()
921 
922  #Adding HEPSPEC06 score if available in /build/HEPSPEC06.score file
923  self.HEPSPEC06 = 0 #Set it to 0 by default (so it is easy to catch in the DB too)
924  try:
925  HEPSPEC06_file=open("/build/HEPSPEC06.score","r")
926  for line in HEPSPEC06_file.readlines():
927  if not line.startswith("#") and "HEPSPEC06" in line:
928  self.HEPSPEC06= line.split()[2]
929  except IOError:
930  self.logh.write("***Warning***: Could not find file /build/HEPSPEC06.score file on this machine!\n")
931  self.logh.flush()
932 
933  #Adding a copy of /proc/cpuinfo and /proc/meminfo in the working directory so it can be kept in the tarball on CASTOR:
934  localcpuinfo=os.path.join(perfsuitedir,"cpuinfo")
935  cpuinfo_exitcode=-1
936  if os.path.exists(localcpuinfo):
937  cpuinfo_exitcode=0
938  else:
939  self.logh.write("Copying /proc/cpuinfo in current working directory (%s)\n"%perfsuitedir)
940  cpuinfo_exitcode=self.runcmd("cp /proc/cpuinfo %s"%perfsuitedir)
941  localmeminfo=os.path.join(perfsuitedir,"meminfo")
942  meminfo_exitcode=-1
943  if os.path.exists(localmeminfo):
944  meminfo_exitcode=0
945  else:
946  self.logh.write("Copying /proc/meminfo in current working directory (%s)\n"%perfsuitedir)
947  meminfo_exitcode=self.runcmd("cp /proc/meminfo %s"%perfsuitedir)
948  if cpuinfo_exitcode or meminfo_exitcode:
949  self.logh.write("There was an issue copying the cpuinfo or meminfo files!\n")
950  self.logh.flush()
951 
952  try:
953  if not prevrel == "":
954  self.logh.write("Production of regression information has been requested with release directory %s\n" % prevrel)
955  if not cmsdriverOptions == "":
956  self.logh.write("Running cmsDriver.py with user defined options: %s\n" % cmsdriverOptions)
957  #Attach the full option synthax for cmsRelvalreportInput.py:
958  cmsdriverOptionsRelvalInput="--cmsdriver="+cmsdriverOptions
959  #FIXME: should import cmsRelvalreportInput.py and avoid these issues...
960  if not stepOptions == "":
961  self.logh.write("Running user defined steps only: %s\n" % stepOptions)
962  #Attach the full option synthax for cmsRelvalreportInput.py:
963  setpOptionsRelvalInput="--usersteps="+stepOptions
964  #FIXME: should import cmsRelvalreportInput.py and avoid these issues...
965  if bypasshlt:
966  #Attach the full option synthax for cmsRelvalreportInput.py:
967  bypasshltRelvalInput="--bypass-hlt"
968  #FIXME: should import cmsRelvalreportInput.py and avoid these issues...
969  self.logh.write("Current Architecture is %s\n"%self.cmssw_arch)
970  self.logh.write("Current CMSSW version is %s\n"%self.cmssw_version)
971  self.logh.write("This machine ( %s ) is assumed to have %s cores, and the suite will be run on cpu %s\n" %(self.host,cores,cpus))
972  self.logh.write("This machine's HEPSPEC06 score is: %s \n"%self.HEPSPEC06)
973  path=os.path.abspath(".")
974  self.logh.write("Performance Suite started running at %s on %s in directory %s, run by user %s\n" % (self.getDate(),self.host,path,self.user))
975  #Start the timer for the total performance suite running time:
976  TotalTime=PerfSuiteTimer(start=datetime.datetime.now())
977  #Also initialize the dictionary that will contain all the timing information:
978  global TimerInfo
979  TimerInfo={'TotalTime':{'TotalTime':TotalTime}} #Structure will be {'key':[PerfSuiteTimerInstance,...],...}
980  #Obsolete popen4-> subprocess.Popen
981  #showtags=os.popen4("showtags -r")[1].read()
982  showtags=subprocess.Popen("showtags -r",shell=True,stdout=subprocess.PIPE,stderr=subprocess.STDOUT).stdout.read()
983  self.logh.write(showtags) # + "\n") No need for extra \n!
984  self.logh.flush()
985  #For the log:
986  if self._verbose:
987  self.logh.write("The performance suite results tarball will be stored in CASTOR at %s\n" % self._CASTOR_DIR)
988  self.logh.write("%s TimeSize events\n" % TimeSizeEvents)
989  self.logh.write("%s IgProf events\n" % IgProfEvents)
990  self.logh.write("%s Callgrind events\n" % CallgrindEvents)
991  self.logh.write("%s Memcheck events\n" % MemcheckEvents)
992  self.logh.write("%s cmsScimark benchmarks before starting the tests\n" % cmsScimark)
993  self.logh.write("%s cmsScimarkLarge benchmarks before starting the tests\n" % cmsScimarkLarge)
994  self.logh.flush()
995  #Actual script actions!
996  #Will have to fix the issue with the matplotlib pie-charts:
997  #Used to source /afs/cern.ch/user/d/dpiparo/w0/perfreport2.1installation/share/perfreport/init_matplotlib.sh
998  #Need an alternative in the release
999 
1000  #Code for the architecture benchmarking use-case
1001  if len(cpus) > 1:
1002  for cpu in cpus:
1003  cpupath = os.path.join(perfsuitedir,"cpu_%s" % cpu)
1004  if not os.path.exists(cpupath):
1005  os.mkdir(cpupath)
1006 
1007  self.Commands = {}
1008  AllScripts = self.Scripts + self.AuxiliaryScripts
1009 
1010  for cpu in range(cmsCpuInfo.get_NumOfCores()): #FIXME use the actual number of cores of the machine here!
1011  self.Commands[cpu] = []
1012 
1013  #Information for the log:
1014  self.logh.write("Full path of all the scripts used in this run of the Performance Suite:\n")
1015  for script in AllScripts:
1016  which="which " + script
1017 
1018  #Logging the actual version of cmsDriver.py, cmsRelvalreport.py, cmsSimPyRelVal.pl
1019  #Obsolete popen4-> subprocess.Popen
1020  #whichstdout=os.popen4(which)[1].read()
1021  whichstdout=subprocess.Popen(which,shell=True,stdout=subprocess.PIPE,stderr=subprocess.STDOUT).stdout.read()
1022  self.logh.write(whichstdout) # + "\n") No need of the extra \n!
1023  if script in self.Scripts:
1024  for cpu in range(cmsCpuInfo.get_NumOfCores()):#FIXME use the actual number of cores of the machine here!
1025  command="taskset -c %s %s" % (cpu,script)
1026  self.Commands[cpu].append(command)
1027 
1028  #First submit the cmsScimark benchmarks on the unused cores:
1029  scimark = ""
1030  scimarklarge = ""
1031  if not (self._unittest or self._noexec):
1032  for core in range(cores):
1033  if (not core in cpus) and runonspare:
1034  self.logh.write("Submitting cmsScimarkLaunch.csh to run on core cpu "+str(core) + "\n")
1035  subcmd = "cd %s ; cmsScimarkLaunch.csh %s" % (perfsuitedir, str(core))
1036  command="taskset -c %s sh -c \"%s\" &" % (str(core), subcmd)
1037  self.logh.write(command + "\n")
1038 
1039  #cmsScimarkLaunch.csh is an infinite loop to spawn cmsScimark2 on the other
1040  #cpus so it makes no sense to try reading its stdout/err
1041  #Obsolete popen4-> subprocess.Popen
1042  #os.popen4(command)
1043  subprocess.Popen(command,shell=True,stdout=subprocess.PIPE,stderr=subprocess.STDOUT)
1044 
1045  self.logh.flush()
1046 
1047  #Don't do benchmarking if in debug mode... saves time
1048  benching = not self._debug
1049  ##FIXME:
1050  #We may want to introduce a switch here or agree on a different default (currently 10 cmsScimark and 10 cmsScimarkLarge)
1051  if benching and not (self._unittest or self._noexec):
1052  #Submit the cmsScimark benchmarks on the cpu where the suite will be run:
1053  for cpu in cpus:
1054  scimark = open(os.path.join(perfsuitedir,"cmsScimark2.log") ,"w")
1055  scimarklarge = open(os.path.join(perfsuitedir,"cmsScimark2_large.log"),"w")
1056  if cmsScimark > 0:
1057  self.logh.write("Starting with %s cmsScimark on cpu%s\n" % (cmsScimark,cpu))
1058  cmsScimarkInitialTime=PerfSuiteTimer(start=datetime.datetime.now()) #Create the cmsScimark PerfSuiteTimer
1059  TimerInfo.update({'cmsScimarkTime':{'cmsScimarkInitial':cmsScimarkInitialTime}}) #Add the cmsScimarkInitialTime information to the general TimerInfo dictionary
1060  self.benchmarks(cpu,perfsuitedir,scimark.name,cmsScimark)
1061  cmsScimarkInitialTime.set_end(datetime.datetime.now()) #Stop the cmsScimark initial timer
1062 
1063  if cmsScimarkLarge > 0:
1064  self.logh.write("Following with %s cmsScimarkLarge on cpu%s\n" % (cmsScimarkLarge,cpu))
1065  cmsScimarkLargeInitialTime=PerfSuiteTimer(start=datetime.datetime.now()) #Create the cmsScimarkLarge PerfSuiteTimer
1066  TimerInfo['cmsScimarkTime'].update({'cmsScimarkLargeInitial':cmsScimarkLargeInitialTime}) #Add the cmsScimarkLargeInitialTime information to the general TimerInfo dictionary
1067  self.benchmarks(cpu,perfsuitedir,scimarklarge.name,cmsScimarkLarge, large=True)
1068  cmsScimarkLargeInitialTime.set_end(datetime.datetime.now()) #Stop the cmsScimarkLarge Initial timer
1069  self.logh.flush()
1070  #Handling the Pile up input file here:
1071  if (TimeSizePUCandles or IgProfPUCandles or CallgrindPUCandles or MemcheckPUCandles) and not ("FASTSIM" in stepOptions):
1072  #Note the FASTSIM exclusion... since there is no need to copy the file for FASTSIM.
1073  PUInputName=os.path.join(perfsuitedir,"INPUT_PILEUP_EVENTS.root")
1074  if PUInputFile:
1075  #Define the actual command to copy the file locally:
1076  #Allow the file to be mounted locally (or accessible via AFS)
1077  copycmd="cp"
1078  #Allow the file to be on CASTOR (taking a full CASTOR path)
1079  if '/store/relval/' in PUInputFile:
1080  copycmd="rfcp"
1081  #Accept plain LFNs from DBS for RelVal CASTOR files:
1082  #Minor fix to allow the case of user using the full path /castor/cern.ch/cms...
1083  if PUInputFile.startswith('/store/relval/'):
1084  PUInputFile="/castor/cern.ch/cms"+PUInputFile
1085  #Copy the file locally
1086  self.logh.write("Copying the file %s locally to %s\n"%(PUInputFile,PUInputName))
1087  self.logh.flush()
1088  GetPUInput=subprocess.Popen("%s %s %s"%(copycmd,PUInputFile,PUInputName), shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
1089  GetPUInputExitCode=GetPUInput.wait()
1090  #Allow even the potential copy of a local file (even one already named INPUT_PILEUP_EVENTS.root!)
1091  if GetPUInputExitCode:
1092  self.logh.write("The copying of the pile-up input file returned a non-zero exit code: %s \nThis is the stdout+stderr if the command:\n%s\n"%(GetPUInputExitCode,GetPUInput.stdout))
1093  #Ultimately accept the case of the file being already there and not being specified in the --PUInputFile option
1094  if not os.path.exists(PUInputName):
1095  self.logh.write("The necessary INPUT_PILEUP_EVENTS.root file was not found in the working directory %s\nExiting now!"%perfsuitedir)
1096  self.logh.flush()
1097  sys.exit(1)
1098  else:
1099  #Set up here the DIGI PILE UP options
1100  self.printFlush("Some PILE UP tests will be run!")
1101  #Actually setting them earlier... when handling options... May not need this else after all... or just as a log entry.
1102  self.printFlush("cmsdriverPUOptions is %s"%cmsdriverPUOptions)
1103  pass
1104 
1105  #TimeSize tests:
1106  if TimeSizeEvents > 0:
1107  TimeSizeTime=PerfSuiteTimer(start=datetime.datetime.now()) #Start the TimeSize timer
1108  TimerInfo.update({'TimeSize':{'TotalTime':TimeSizeTime}}) #Add the TimeSize timer to the dictionary
1109  if TimeSizeCandles:
1110  self.logh.write("Launching the TimeSize tests (TimingReport, TimeReport, SimpleMemoryCheck, EdmSize) with %s events each\n" % TimeSizeEvents)
1111  NoPileUpTime=PerfSuiteTimer(start=datetime.datetime.now()) #Start the TimeSize timer
1112  TimerInfo['TimeSize'].update({'NoPileUpTime':NoPileUpTime}) #Add the TimeSize No Pile Up tests timer to the list
1113  self.printDate()
1114  self.logh.flush()
1115  ReportExit=self.simpleGenReport(cpus,perfsuitedir,TimeSizeEvents,TimeSizeCandles,cmsdriverOptions,stepOptions,"TimeSize",profilers,bypasshlt,userInputFile)
1116  FinalExitCode=FinalExitCode+ReportExit
1117  #Adding a time stamp here to parse for performance suite running time data
1118  self.printFlush("Regular TimeSize tests were finished at %s"%(self.getDate()))
1119  NoPileUpTime.set_end(datetime.datetime.now()) #Stop TimeSize timer
1120 
1121  #Launch eventual Digi Pile Up TimeSize too:
1122  if TimeSizePUCandles:
1123  self.logh.write("Launching the PILE UP TimeSize tests (TimingReport, TimeReport, SimpleMemoryCheck, EdmSize) with %s events each\n" % TimeSizeEvents)
1124  PileUpTime=PerfSuiteTimer(start=datetime.datetime.now()) #Start the TimeSize timer
1125  TimerInfo['TimeSize'].update({'PileUpTime':PileUpTime}) #Add the TimeSize Pile Up tests timer to the list
1126  self.printDate()
1127  self.logh.flush()
1128  ReportExit=self.simpleGenReport(cpus,perfsuitedir,TimeSizeEvents,TimeSizePUCandles,cmsdriverPUOptions,stepOptions,"TimeSize",profilers,bypasshlt,userInputFile)
1129  FinalExitCode=FinalExitCode+ReportExit
1130  #Adding a time stamp here to parse for performance suite running time data
1131  self.printFlush("Pileup TimeSize tests were finished at %s"%(self.getDate()))
1132  PileUpTime.set_end(datetime.datetime.now()) #Stop TimeSize timer
1133 
1134  #Check for issue with
1135  if not (TimeSizeCandles or TimeSizePUCandles):
1136  self.printFlush("A number of events (%s) for TimeSize tests was selected, but no candle for regular or pileup tests was selected!"%(TimeSizeEvents))
1137  #Adding a time stamp here to parse for performance suite running time data
1138  self.printFlush("All TimeSize tests were finished at %s"%(self.getDate()))
1139  TimeSizeTime.set_end(datetime.datetime.now()) #Stop TimeSize timer
1140 
1141  #Stopping all cmsScimark jobs and analysing automatically the logfiles
1142  #No need to waste CPU while the load does not affect Valgrind measurements!
1143  if not (self._unittest or self._noexec):
1144  self.logh.write("Stopping all cmsScimark jobs now\n")
1145  subcmd = "cd %s ; %s" % (perfsuitedir,self.AuxiliaryScripts[2])
1146  stopcmd = "sh -c \"%s\"" % subcmd
1147  self.printFlush(stopcmd)
1148  #os.popen(stopcmd)
1149  #Obsolete popen4-> subprocess.Popen
1150  #self.printFlush(os.popen4(stopcmd)[1].read())
1151  self.printFlush(subprocess.Popen(stopcmd,shell=True,stdout=subprocess.PIPE,stderr=subprocess.STDOUT).stdout.read())
1152 
1153  #From here on we can use all available cores to speed up the performance suite remaining tests:
1154  if cores==0: #When specifying the cpu to run the suite on, one has to set cores to 0 to avoid threading of PerfSuite itself...
1155  #So we need to catch this case for the IB tests case where we assign the test to a specific cpu.
1156  AvailableCores=cpus
1157  else:
1158  AvailableCores=range(cores)
1159  #Initialize a list that will contain all the simpleGenReport keyword arguments (1 dictionary per test):
1160  TestsToDo=[]
1161  #IgProf tests:
1162  if IgProfEvents > 0:
1163  if IgProfCandles:
1164  self.printFlush("Preparing IgProf tests")
1165  #Special case for IgProf: user could pick with the option --profilers to run only IgProf perf or Mem (or Mem_Total alone etc)
1166  #So in general we want to be able to split the perf and mem tests...
1167  #For the case of --profiler option we will run only 1 test (i.e. it will get one core slot until it is done with whatever profiling choosen)
1168  if profilers:
1169  self.printFlush("Special profiler option for IgProf was indicated by the user: %s"%profilers)
1170  #Prepare the simpleGenReport arguments for this test:
1171  IgProfProfilerArgs={
1172  'perfdir':perfsuitedir,
1173  'NumEvents':IgProfEvents,
1174  'candles':IgProfCandles,
1175  'cmsdriverOptions':cmsdriverOptions,
1176  'stepOptions':stepOptions,
1177  'Name':"IgProf",
1178  'profilers':profilers,
1179  'bypasshlt':bypasshlt,
1180  'userInputRootFiles':userInputFile
1181  }
1182  #Append the test to the TestsToDo list:
1183  TestsToDo.append(IgProfProfilerArgs)
1184  self.printFlush("Appended IgProf test with profiler option %s to the TestsToDo list"%profilers)
1185  #For the default case (4,5,6,7) we split the tests into 2 jobs since they naturally are 2 cmsRun jobs and for machines with many cores this will
1186  #make the performance suite run faster.
1187  else:
1188  self.printFlush("Splitting the IgProf tests into Perf and Mem to parallelize the cmsRun execution as much as possible:")
1189  ##PERF##
1190  #Prepare the simpleGenReport arguments for this test:
1191  IgProfPerfArgs={
1192  'perfdir':perfsuitedir,
1193  'NumEvents':IgProfEvents,
1194  'candles':IgProfCandles,
1195  'cmsdriverOptions':cmsdriverOptions,
1196  'stepOptions':stepOptions,
1197  'Name':"IgProf_Perf",
1198  'profilers':profilers,
1199  'bypasshlt':bypasshlt,
1200  'userInputRootFiles':userInputFile
1201  }
1202  #Append the test to the TestsToDo list:
1203  TestsToDo.append(IgProfPerfArgs)
1204  self.printFlush("Appended IgProf PERF test to the TestsToDo list")
1205  ##MEM##
1206  #Prepare the simpleGenReport arguments for this test:
1207  IgProfMemArgs={
1208  'perfdir':perfsuitedir,
1209  'NumEvents':IgProfEvents,
1210  'candles':IgProfCandles,
1211  'cmsdriverOptions':cmsdriverOptions,
1212  'stepOptions':stepOptions,
1213  'Name':"IgProf_Mem",
1214  'profilers':profilers,
1215  'bypasshlt':bypasshlt,
1216  'userInputRootFiles':userInputFile
1217  }
1218  #Append the test to the TestsToDo list:
1219  TestsToDo.append(IgProfMemArgs)
1220  self.printFlush("Appended IgProf MEM test to the TestsToDo list")
1221  #The following will be handled in the while loop that handles the starting of the threads:
1222  #ReportExit=self.simpleGenReport(cpus,perfsuitedir,IgProfEvents,IgProfCandles,cmsdriverOptions,stepOptions,"IgProf",profilers,bypasshlt,userInputFile)
1223  #FinalExitCode=FinalExitCode+ReportExit
1224  #Launch eventual Digi Pile Up IgProf too:
1225  if IgProfPUCandles:
1226  self.printFlush("Preparing IgProf PileUp tests")
1227  #Special case for IgProf: user could pick with the option --profilers to run only IgProf perf or Mem (or Mem_Total alone etc)
1228  #So in general we want to be able to split the perf and mem tests...
1229  #For the case of --profiler option we will run only 1 test (i.e. it will get one core slot until it is done with whatever profiling choosen)
1230  if profilers:
1231  self.printFlush("Special profiler option for IgProf was indicated by the user: %s"%profilers)
1232  #Prepare the simpleGenReport arguments for this test:
1233  IgProfProfilerPUArgs={
1234  'perfdir':perfsuitedir,
1235  'NumEvents':IgProfEvents,
1236  'candles':IgProfPUCandles,
1237  'cmsdriverOptions':cmsdriverPUOptions,
1238  'stepOptions':stepOptions,
1239  'Name':"IgProf",
1240  'profilers':profilers,
1241  'bypasshlt':bypasshlt,
1242  'userInputRootFiles':userInputFile
1243  }
1244  #Append the test to the TestsToDo list:
1245  TestsToDo.append(IgProfProfilerPUArgs)
1246  self.printFlush("Appended IgProf PileUp test with profiler option %s to the TestsToDo list"%profilers)
1247  else:
1248  self.printFlush("Splitting the IgProf tests into Perf and Mem to parallelize the cmsRun execution as much as possible:")
1249  ##PERF##
1250  #Prepare the simpleGenReport arguments for this test:
1251  IgProfPerfPUArgs={
1252  'perfdir':perfsuitedir,
1253  'NumEvents':IgProfEvents,
1254  'candles':IgProfPUCandles,
1255  'cmsdriverOptions':cmsdriverPUOptions,
1256  'stepOptions':stepOptions,
1257  'Name':"IgProf_Perf",
1258  'profilers':profilers,
1259  'bypasshlt':bypasshlt,
1260  'userInputRootFiles':userInputFile
1261  }
1262  #Append the test to the TestsToDo list:
1263  TestsToDo.append(IgProfPerfPUArgs)
1264  self.printFlush("Appended IgProf MEM PileUp test to the TestsToDo list")
1265  ##MEM##
1266  #Prepare the simpleGenReport arguments for this test:
1267  IgProfMemPUArgs={
1268  'perfdir':perfsuitedir,
1269  'NumEvents':IgProfEvents,
1270  'candles':IgProfPUCandles,
1271  'cmsdriverOptions':cmsdriverPUOptions,
1272  'stepOptions':stepOptions,
1273  'Name':"IgProf_Mem",
1274  'profilers':profilers,
1275  'bypasshlt':bypasshlt,
1276  'userInputRootFiles':userInputFile
1277  }
1278  #Append the test to the TestsToDo list:
1279  TestsToDo.append(IgProfMemPUArgs)
1280  self.printFlush("Appended IgProf MEM PileUp test to the TestsToDo list")
1281  if not (IgProfCandles or IgProfPUCandles):
1282  self.printFlush("A number of events (%s) for IgProf tests was selected, but no candle for regular or pileup tests was selected!"%(IgProfEvents))
1283 
1284 
1285  #Valgrind tests:
1286  if CallgrindEvents > 0:
1287  if CallgrindCandles:
1288  self.printFlush("Preparing Callgrind tests")
1289  CallgrindArgs={
1290  'perfdir':perfsuitedir,
1291  'NumEvents':CallgrindEvents,
1292  'candles':CallgrindCandles,
1293  'cmsdriverOptions':cmsdriverOptions,
1294  'stepOptions':stepOptions,
1295  'Name':"Callgrind",
1296  'profilers':profilers,
1297  'bypasshlt':bypasshlt,
1298  'userInputRootFiles':userInputFile
1299  }
1300  #Append the test to the TestsToDo list:
1301  TestsToDo.append(CallgrindArgs)
1302  self.printFlush("Appended Callgrind test to the TestsToDo list")
1303  #Launch eventual Digi Pile Up Callgrind too:
1304  if CallgrindPUCandles:
1305  self.printFlush("Preparing Callgrind PileUp tests")
1306  CallgrindPUArgs={
1307  'perfdir':perfsuitedir,
1308  'NumEvents':CallgrindEvents,
1309  'candles':CallgrindPUCandles,
1310  'cmsdriverOptions':cmsdriverPUOptions,
1311  'stepOptions':stepOptions,
1312  'Name':"Callgrind",
1313  'profilers':profilers,
1314  'bypasshlt':bypasshlt,
1315  'userInputRootFiles':userInputFile
1316  }
1317  #Append the test to the TestsToDo list:
1318  TestsToDo.append(CallgrindPUArgs)
1319  self.printFlush("Appended Callgrind PileUp test to the TestsToDo list")
1320  if not (CallgrindCandles or CallgrindPUCandles):
1321  self.printFlush("A number of events (%s) for Callgrind tests was selected, but no candle for regular or pileup tests was selected!"%(CallgrindEvents))
1322 
1323  if MemcheckEvents > 0:
1324  if MemcheckCandles:
1325  self.printFlush("Preparing Memcheck tests")
1326  MemcheckArgs={
1327  'perfdir':perfsuitedir,
1328  'NumEvents':MemcheckEvents,
1329  'candles':MemcheckCandles,
1330  'cmsdriverOptions':cmsdriverOptions,
1331  'stepOptions':stepOptions,
1332  'Name':"Memcheck",
1333  'profilers':profilers,
1334  'bypasshlt':bypasshlt,
1335  'userInputRootFiles':userInputFile
1336  }
1337  #Append the test to the TestsToDo list:
1338  TestsToDo.append(MemcheckArgs)
1339  self.printFlush("Appended Memcheck test to the TestsToDo list")
1340  #Launch eventual Digi Pile Up Memcheck too:
1341  if MemcheckPUCandles:
1342  self.printFlush("Preparing Memcheck PileUp tests")
1343  MemcheckPUArgs={
1344  'perfdir':perfsuitedir,
1345  'NumEvents':MemcheckEvents,
1346  'candles':MemcheckPUCandles,
1347  'cmsdriverOptions':cmsdriverPUOptions,
1348  'stepOptions':stepOptions,
1349  'Name':"Memcheck",
1350  'profilers':profilers,
1351  'bypasshlt':bypasshlt,
1352  'userInputRootFiles':userInputFile
1353  }
1354  #Append the test to the TestsToDo list:
1355  TestsToDo.append(MemcheckPUArgs)
1356  self.printFlush("Appended Memcheck PileUp test to the TestsToDo list")
1357  if not (MemcheckCandles or MemcheckPUCandles):
1358  self.printFlush("A number of events (%s) for Memcheck tests was selected, but no candle for regular or pileup tests was selected!"%(MemcheckEvents))
1359 
1360  #Here if there are any IgProf, Callgrind or MemcheckEvents to be run,
1361  #run the infinite loop that submits the PerfTest() threads on the available cores:
1362  if IgProfEvents or CallgrindEvents or MemcheckEvents:
1363  #FIXME:We should consider what behavior makes most sense in case we use the --cores option at this time only the cores=0 care is considered...
1364  self.printFlush("Threading all remaining tests on all %s available cores!"%len(AvailableCores))
1365  self.printDate()
1366  self.logh.flush()
1367  #Save the original AvailableCores list to use it as a test to break the infinite loop:
1368  #While in the regular RelVal use-case it makes sense to use the actual number of cores of the machines, in
1369  #the IB case the AvailableCores will always consist of only 1 single core..
1370  OriginalAvailableCores=list(AvailableCores) #Tricky list copy bug! without the list() OriginalAvalaibleCores would point to AvailableCores!
1371  #Print this out in the log for debugging reasons
1372  self.printFlush("Original available cores list: %s"%AvailableCores)
1373 
1374  #Create a dictionary to keep track of running threads on the various cores:
1375  activePerfTestThreads={}
1376  #Flag for waiting messages:
1377  Waiting=False
1378  while 1:
1379  #Check if there are tests to run:
1380  if TestsToDo:
1381  #Using the Waiting flag to avoid writing this message every 5 seconds in the case
1382  #of having more tests to do than available cores...
1383  if not Waiting:
1384  self.printFlush("Currently %s tests are scheduled to be run:"%len(TestsToDo))
1385  self.printFlush(TestsToDo)
1386  #Check the available cores:
1387  if AvailableCores:
1388  #Set waiting flag to False since we'll be doing something
1389  Waiting=False
1390  self.printFlush("There is/are %s core(s) available"%len(AvailableCores))
1391  cpu=AvailableCores.pop()
1392  self.printFlush("Let's use cpu %s"%cpu)
1393  simpleGenReportArgs=TestsToDo.pop()
1394  self.printFlush("Let's submit %s test on core %s"%(simpleGenReportArgs['Name'],cpu))
1395  #Adding a Total timer for each of the threaded tests:
1396  if simpleGenReportArgs['Name'] not in TimerInfo.keys():
1397  #if 'TotalTime' not in TimerInfo[simpleGenReportArgs['Name']].keys():
1398  self.PerfTestTotalTimer=PerfSuiteTimer(start=datetime.datetime.now()) #Start the TimeSize timer
1399  TimerInfo.update({simpleGenReportArgs['Name']:{'TotalTime':self.PerfTestTotalTimer}}) #Add the TimeSize timer to the dictionary
1400  threadToDo=self.simpleGenReportThread(cpu,self,**simpleGenReportArgs) #Need to send self too, so that the thread has access to the PerfSuite.simpleGenReport() function
1401  self.printFlush("Starting thread %s"%threadToDo)
1402  ReportExitCode=threadToDo.start()
1403  self.printFlush("Adding thread %s to the list of active threads"%threadToDo)
1404  activePerfTestThreads[cpu]=threadToDo
1405  #If there is no available core, pass, there will be some checking of activeThreads, a little sleep and then another check.
1406  else:
1407  pass
1408  #Test activePerfThreads:
1409  activeTestNames=[]
1410  activeTestNamesPU=[]
1411  for cpu in activePerfTestThreads.keys():
1412  if activePerfTestThreads[cpu].isAlive():
1413  #print "%% cpu %s activerPerfTestThreads[cpu] %s activePerfTestThreads[cpu].simpleGenReportArgs['cmsdriverOptions'] %s"%(cpu,activePerfTestThreads[cpu],activePerfTestThreads[cpu].simpleGenReportArgs['cmsdriverOptions'])
1414  if "--pileup" in activePerfTestThreads[cpu].simpleGenReportArgs['cmsdriverOptions']:
1415  activeTestNamesPU.append(activePerfTestThreads[cpu].simpleGenReportArgs['Name'])
1416  else:
1417  activeTestNames.append(activePerfTestThreads[cpu].simpleGenReportArgs['Name'])
1418  pass
1419  elif cpu not in AvailableCores:
1420  #Set waiting flag to False since we'll be doing something
1421  Waiting=False
1422  self.printFlush(time.ctime())
1423  self.printFlush("%s test, in thread %s is done running on core %s"%(activePerfTestThreads[cpu].simpleGenReportArgs['Name'],activePerfTestThreads[cpu],cpu) )
1424  self.printFlush("About to append cpu %s to AvailableCores list"%cpu)
1425  AvailableCores.append(cpu)
1426  #Eliminate from activeTestNames lists:
1427  #print activeTestNames
1428  #print activeTestNamesPU
1429  #print activePerfTestThreads[cpu].simpleGenReportArgs['Name']
1430  if "--pileup" in activePerfTestThreads[cpu].simpleGenReportArgs['cmsdriverOptions']:
1431  try:
1432  activeTestNamesPU.remove(activePerfTestThreads[cpu].simpleGenReportArgs['Name'])
1433  except:
1434  pass
1435  else:
1436  try:
1437  activeTestNames.remove(activePerfTestThreads[cpu].simpleGenReportArgs['Name'])
1438  except:
1439  pass
1440  #Eliminate also from activePErfTestThreads dictionary:
1441  activePerfTestThreads.pop(cpu)
1442  #FIXME:
1443  #Delicate check to stop the timer on the individual threaded test!
1444  #Need to thik about it still...
1445  #FIXME:
1446  #Delicate check to stop the timers on the threaded tests:
1447  #Check activePerfTestThreads dictionary for "Name" if any name is missing, the total can be stopped for that name.
1448  #self.PerfTestTotalTimer
1449  for TestName in ["IgProf_Perf","IgProf_Mem","Memcheck","Valgrind"]:
1450  if (TestName not in activeTestNames) and (TestName not in activeTestNamesPU) :
1451  try:
1452  TimerInfo[TestName]['TotalTime'].set_end(datetime.datetime.now())
1453  except:
1454  #print "No %s test was running"%TestName
1455  pass
1456  #Buggy if... it seems we don't wait for the running thread to be finished...
1457  #We should request:
1458  #-All OriginalAvailableCores should be actually available.
1459  if not AvailableCores==[] and (set(AvailableCores)==set(range(cmsCpuInfo.get_NumOfCores())) or set(AvailableCores)==set(OriginalAvailableCores)) and not TestsToDo:
1460  self.printFlush("PHEW! We're done... all TestsToDo are done... at %s "%(self.getDate()))
1461  #Debug printouts:
1462  #print "AvailableCores",AvailableCores
1463  #print "set(AvailableCores)",set(AvailableCores)
1464  #print "set(range(cmsCpuInfo.get_NumOfCores())",set(range(cmsCpuInfo.get_NumOfCores()))
1465  #print "OriginalAvailableCores",OriginalAvailableCores
1466  #print "set(OriginalAvailableCores)",set(OriginalAvailableCores)
1467  #print "TestsToDo",TestsToDo
1468  break
1469  else:
1470  #Putting the sleep statement first to avoid writing Waiting... before the output of the started thread reaches the log...
1471  time.sleep(5)
1472  #Use Waiting flag to writing 1 waiting message while waiting and avoid having 1 message every 5 seconds...
1473  if not Waiting:
1474  self.printFlush(time.ctime())
1475  self.printFlush("Waiting for tests to be done...")
1476  sys.stdout.flush()
1477  Waiting=True
1478  #End of the if for IgProf, Callgrind, Memcheck tests
1479 
1480  if benching and not (self._unittest or self._noexec):
1481  #Ending the performance suite with the cmsScimark benchmarks again:
1482  for cpu in cpus:
1483  if cmsScimark > 0:
1484  self.logh.write("Ending with %s cmsScimark on cpu%s\n" % (cmsScimark,cpu))
1485  cmsScimarkFinalTime=PerfSuiteTimer(start=datetime.datetime.now()) #Create the cmsScimark PerfSuiteTimer
1486  TimerInfo['cmsScimarkTime'].update({'cmsScimarkFinal':cmsScimarkFinalTime}) #Add the cmsScimarkFinalTime information to the general TimerInfo dictionary
1487 
1488  self.benchmarks(cpu,perfsuitedir,scimark.name,cmsScimark)
1489  cmsScimarkFinalTime.set_end(datetime.datetime.now()) #Stop the cmsScimarkLarge Initial timer
1490  if cmsScimarkLarge > 0:
1491  self.logh.write("Following with %s cmsScimarkLarge on cpu%s\n" % (cmsScimarkLarge,cpu))
1492  cmsScimarkLargeFinalTime=PerfSuiteTimer(start=datetime.datetime.now()) #Create the cmsScimarkLargePerfSuiteTimer
1493  TimerInfo['cmsScimarkTime'].update({'cmsScimarkLargeFinal':cmsScimarkLargeFinalTime}) #Add the cmsScimarkLargeFinalTime information to the general TimerInfo dictionary
1494  self.benchmarks(cpu,perfsuitedir,scimarklarge.name,cmsScimarkLarge,large=True)
1495  cmsScimarkLargeFinalTime.set_end(datetime.datetime.now()) #Stop the cmsScimarkLarge Initial timer
1496 
1497  if prevrel:
1498  self.logh.write("Running the regression analysis with respect to %s\n"%getVerFromLog(prevrel))
1499  self.logh.write(time.ctime(time.time()))
1500  self.logh.flush()
1501 
1502  crr.regressReports(prevrel,os.path.abspath(perfsuitedir),oldRelName = getVerFromLog(prevrel),newRelName=self.cmssw_version)
1503 
1504  #Create a tarball of the work directory
1505  if tarball:
1506  tarballTime=PerfSuiteTimer(start=datetime.datetime.now()) #Create the tarball PerfSuiteTimer
1507  TimerInfo.update({'tarballTime':{'TotalTime':tarballTime}})
1508  # Adding the str(stepOptions to distinguish the tarballs for 1 release
1509  # (GEN->DIGI, L1->RECO will be run in parallel)
1510 
1511  # Cleaning the stepOptions from the --usersteps=:
1512  if "=" in str(stepOptions):
1513  fileStepOption=str(stepOptions).split("=")[1]
1514  else:
1515  fileStepOption=str(stepOptions)
1516  if fileStepOption=="":
1517  fileStepOption="UnknownStep"
1518  # Add the working directory used to avoid overwriting castor files (also put a check...)
1519  fileWorkingDir=os.path.basename(perfsuitedir)
1520 
1521  # Also add the --conditions and --eventcontent options used in the --cmsdriver options since it
1522  # is possible that the same tests will be run with different conditions and/or event content:
1523  # Parse it out of --cmsdriver option:
1524  fileEventContentOption="UnknownEventContent"
1525  fileConditionsOption="UnknownConditions"
1526  for token in cmsdriverOptions.split("--"):
1527  if token!='' and 'cmsdriver' not in token:
1528  if "=" in token:
1529  fileOption=token.split("=")[0]
1530  fileOptionValue=token.split("=")[1].strip("'").strip('"')
1531  else:
1532  fileOption=token.split()[0]
1533  fileOptionValue=token.split()[1].strip("'").strip('"')
1534  if "eventcontent" or "conditions" in fileOption:
1535  if "eventcontent" in fileOption:
1536  fileEventContentOption=fileOptionValue
1537  elif "conditions" in fileOption:
1538  # check if we are using the autoCond style of flexible conditions
1539  # if so, expand the condition here so that the file names contain the real conditions
1540  if "auto:" in fileOptionValue:
1541  from Configuration.PyReleaseValidation.autoCond import autoCond
1542  fileConditionsOption = autoCond[ fileOptionValue.split(':')[1] ]
1543  else:
1544  # "old style" conditions, hardcoded values ...
1545  # FIXME:
1546  # Should put at least the convention in cmsPerfCommons to know how to parse it...
1547  # Potential weak point if the conditions tag convention changes...
1548  if "," in fileOptionValue: #Since 330, conditions don't have FrontierConditions_GlobalTag, in front of them anymore...
1549  fileConditionsOption=fileOptionValue.split("::")[0].split(",")[1] #"Backward" compatibility
1550  else:
1551  fileConditionsOption=fileOptionValue.split("::")[0]
1552  else: # empty token
1553  #print "Print this is the token: %s"%token
1554  pass
1555 
1556  #self.printFlush("Conditions label to add to the tarball name is %s"%fileConditionsOption)
1557  #self.printFlush("Eventcontent label to add to the tarball name is %s"%fileEventContentOption)
1558  #FIXME:
1559  #Could add the allowed event contents in the cmsPerfCommons.py file and use those to match in the command line options... This assumes maintenance of cmsPerfCommons.py
1560 
1561 
1562  #Create a tarball with just the logfiles
1563  subprocess.Popen("ls -R | grep .root > rootFiles",shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE).stderr.read()
1564  LogFile = "%s_%s_%s_%s_%s_%s_%s_%s_log.tgz" % (self.cmssw_arch, self.cmssw_version, fileStepOption, fileConditionsOption, fileEventContentOption.split()[0], fileWorkingDir, self.host, self.user)
1565  AbsTarFileLOG = os.path.join(perfsuitedir,LogFile)
1566  tarcmd = "tar zcfX %s %s %s" %(AbsTarFileLOG, "rootFiles", os.path.join(perfsuitedir,"*"))
1567  self.printFlush("Creating a tarball for the logfiles")
1568  self.printFlush(tarcmd)
1569  self.printFlush(subprocess.Popen(tarcmd,shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE).stderr.read())
1570  self.printFlush(subprocess.Popen("rm rootFiles",shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE).stderr.read())
1571 
1572  fullcastorpathlog=os.path.join(castordir,LogFile)
1573 
1574 
1575  #Create the tarball with the contents of the directory + md5 checksum
1576  TarFile = "%s_%s_%s_%s_%s_%s_%s_%s.tgz" % (self.cmssw_arch, self.cmssw_version, fileStepOption, fileConditionsOption, fileEventContentOption.split()[0], fileWorkingDir, self.host, self.user)
1577  AbsTarFile = os.path.join(perfsuitedir,TarFile)
1578  tarcmd = "tar -zcf %s %s" %(AbsTarFile, os.path.join(perfsuitedir,"*"))
1579  md5cmd = "md5sum %s" %(AbsTarFile)
1580  self.printFlush("Creating a tarball with the content of the directory")
1581  self.printFlush(tarcmd)
1582  self.printFlush(md5cmd)
1583  #FIXME:
1584  #Anything that will be logged after the tar command below will not enter the cmsPerfSuite.log in the tarball (by definition)...
1585  #To remain backward compatible the harvesting script needs to be based on the command above to identify the tarball location.
1586  #Obsolete popen4-> subprocess.Popen
1587  #self.printFlush(os.popen3(tarcmd)[2].read()) #Using popen3 to get only stderr we don't want the whole stdout of tar!
1588  self.printFlush(subprocess.Popen(tarcmd,shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE).stderr.read())
1589  md5sum = subprocess.Popen(md5cmd,shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE).stdout.read().split()[0]
1590  self.printFlush("The md5 checksum of the tarball: %s" %(md5sum))
1591  AbsTarFileMD5 = AbsTarFile + ".md5"
1592  md5filecmd = "echo %s > %s" % (md5sum, AbsTarFileMD5)
1593  self.printFlush(subprocess.Popen(md5filecmd,shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE).stderr.read())
1594 
1595  #Archive it on CASTOR
1596  #Before archiving check if it already exist if it does print a message, but do not overwrite, so do not delete it from local dir:
1597  fullcastorpathfile=os.path.join(castordir,TarFile)
1598  fullcastorpathmd5=os.path.join(castordir,TarFile + ".md5")
1599 
1600  checkcastor="nsls %s" % fullcastorpathfile
1601  #Obsolete os.popen-> subprocess.Popen
1602  #checkcastorout=os.popen3(checkcastor)[1].read()
1603  checkcastorout=subprocess.Popen(checkcastor,shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE).stdout.read()
1604  if checkcastorout.rstrip()==fullcastorpathfile:
1605  castorcmdstderr="File %s is already on CASTOR! Will NOT OVERWRITE!!!"%fullcastorpathfile
1606  else:
1607  castorcmd="rfcp %s %s" % (AbsTarFile,fullcastorpathfile)
1608  castormd5cmd="rfcp %s %s" % (AbsTarFileMD5,fullcastorpathmd5)
1609  castorlogcmd="rfcp %s %s" % (AbsTarFileLOG,fullcastorpathlog)
1610  self.printFlush(castorcmd)
1611  self.printFlush(castormd5cmd)
1612  self.printFlush(castorlogcmd)
1613  #Obsolete os.popen-> subprocess.Popen
1614  #castorcmdstderr=os.popen3(castorcmd)[2].read()
1615  castorcmdstderr=subprocess.Popen(castorcmd,shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE).stderr.read()
1616  subprocess.Popen(castormd5cmd,shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE).stderr.read()
1617  subprocess.Popen(castorlogcmd,shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE).stderr.read()
1618  #Checking the stderr of the rfcp command to copy the tarball (.tgz) on CASTOR:
1619  if castorcmdstderr:
1620  #If it failed print the stderr message to the log and tell the user the tarball (.tgz) is kept in the working directory
1621  self.printFlush(castorcmdstderr)
1622  self.printFlush("Since the CASTOR archiving for the tarball failed the file %s is kept in directory %s"%(TarFile, perfsuitedir))
1623  else:
1624  #If it was successful then remove the tarball from the working directory:
1625  self.printFlush("Successfully archived the tarball %s in CASTOR!"%(TarFile))
1626  self.printFlush("The tarball can be found: %s"%(fullcastorpathfile))
1627  self.printFlush("The logfile can be found: %s"%(fullcastorpathlog))
1628  self.printFlush("Deleting the local copy of the tarballs")
1629  rmtarballcmd="rm -Rf %s"%(AbsTarFile)
1630  rmtarballmd5cmd="rm -Rf %s"%(AbsTarFileMD5)
1631  rmtarballlogcmd="rm -Rf %s"%(AbsTarFileLOG)
1632  self.printFlush(rmtarballcmd)
1633  self.printFlush(rmtarballmd5cmd)
1634  self.printFlush(rmtarballlogcmd)
1635  #Obsolete os.popen-> subprocess.Popen
1636  #self.printFlush(os.popen4(rmtarballcmd)[1].read())
1637  self.printFlush(subprocess.Popen(rmtarballcmd,shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT).stdout.read() )
1638  self.printFlush(subprocess.Popen(rmtarballmd5cmd,shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT).stdout.read() )
1639  self.printFlush(subprocess.Popen(rmtarballlogcmd,shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT).stdout.read() )
1640  tarballTime.set_end(datetime.datetime.now())
1641  else:
1642  self.printFlush("Performance Suite directory will not be archived in a tarball since --no_tarball option was chosen")
1643 
1644  #End of script actions!
1645 
1646  #Print a time stamp at the end:
1647  date=time.ctime(time.time())
1648  self.logh.write("Performance Suite finished running at %s on %s in directory %s\n" % (date,self.host,path))
1649  if self.ERRORS == 0:
1650  self.logh.write("There were no errors detected in any of the log files!\n")
1651  else:
1652  self.logh.write("ERROR: There were %s errors detected in the log files, please revise!\n" % self.ERRORS)
1653  #print "No exit code test"
1654  #sys.exit(1)
1655  except exceptions.Exception, detail:
1656  self.logh.write(str(detail) + "\n")
1657  self.logh.flush()
1658  if not self.logh.isatty():
1659  self.logh.close()
1660  raise
1661  #Add the possibility to send as an email the execution logfile to the user and whoever else interested:
1662  if MailLogRecipients != "": #Basically leave the option to turn it off too.. --mail ""
1663  self.printFlush("Sending email notification for this execution of the performance suite with command:")
1664  sendLogByMailcmd='cat cmsPerfSuite.log |mail -s "Performance Suite finished running on %s" '%self.host + MailLogRecipients
1665  self.printFlush(sendLogByMailcmd)
1666  #Obsolete os.popen-> subprocess.Popen
1667  #self.printFlush(os.popen4(sendLogByMailcmd)[1].read())
1668  self.printFlush(subprocess.Popen(sendLogByMailcmd,shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT).stdout.read() )
1669  else:
1670  self.printFlush('No email notification will be sent for this execution of the performance suite since option --mail "" was used')
1671 
1672  TotalTime.set_end(datetime.datetime.now())
1673  self.printFlush("Total Running Time\t%s hrs (%s mins)"%(TotalTime.get_duration()['hours'],TotalTime.get_duration()['minutes']))
1674 
1675  #Dump of the TimerInfo information
1676  #First dump it as a pickleq file...
1677  #Well in order to do so without the complication of serializing a custom class instance need to make the dictionary fully string-made:
1678  TimerInfoStr={}
1679  PerfSuiteTimerInfo=open("PerfSuiteTimerInfo.pkl","wb")
1680  #pickle.dump(TimerInfo,PerfSuiteTimerInfo)
1681  #PerfSuiteTimerInfo.close()
1682  #For now print it at the bottom of the log:
1683  self.logh.write("Test type\tActual Test\tDuration\tStart Time\tEnd Time\n")
1684  for key in TimerInfo.keys():
1685  #self.printFlush(key)
1686  TimerInfoStr.update({key:{}})
1687  for test in TimerInfo[key].keys():
1688  TimerInfoStr[key].update({test:[str(TimerInfo[key][test].get_duration()['hours'])+" hrs ("+str(TimerInfo[key][test].get_duration()['minutes'])+" mins)",TimerInfo[key][test].get_start(),TimerInfo[key][test].get_end()]})
1689  self.logh.write(key+"\t"+test+"\t")
1690  self.logh.write("%s hrs (%s mins)\t"%(TimerInfo[key][test].get_duration()['hours'],TimerInfo[key][test].get_duration()['minutes']))
1691  self.logh.write("%s\t"%TimerInfo[key][test].get_start())
1692  self.logh.write("%s\n"%TimerInfo[key][test].get_end())
1693  pickle.dump(TimerInfoStr,PerfSuiteTimerInfo)
1694  PerfSuiteTimerInfo.close()
1695 
1696  self.logh.write("Final Performance Suite exit code was %s"%FinalExitCode)
1697  self.logh.flush()
sys.exit(FinalExitCode)
cmsPerfSuite.PerfSuite.runcmd
def runcmd
Run a command and return the exit status.
Definition: cmsPerfSuite.py:553
strip
void strip(std::string &input, const std::string &blanks=" \n\t")
Definition: stringTools.cc:16
cmsPerfSuite.PerfSuite.benchmarks
def benchmarks
Run cmsScimark benchmarks a number of times.
Definition: cmsPerfSuite.py:678
cmsPerfSuite.PerfSuite._verbose
_verbose
Check logfile option.
Definition: cmsPerfSuite.py:69
cmsPerfSuite.PerfSuite.printFlush
def printFlush
Print and flush a string (for output to a log file)
Definition: cmsPerfSuite.py:545
cmsPerfSuite.PerfSuite.simpleGenReport
def simpleGenReport
Prepares the profiling directory and runs all the selected profiles (if this is not a unit test) ...
Definition: cmsPerfSuite.py:779
cmsCpuInfo.get_NumOfCores
def get_NumOfCores
Definition: cmsCpuInfo.py:6
cmsPerfSuite.PerfSuite.PerfTestTotalTimer
PerfTestTotalTimer
FIXME: We may want to introduce a switch here or agree on a different default (currently 10 cmsScimar...
Definition: cmsPerfSuite.py:1397
update
#define update(a, b)
Definition: TrackClassifier.cc:10
cmsPerfSuite.PerfSuite._debug
_debug
Developer options.
Definition: cmsPerfSuite.py:66
split
double split
Definition: MVATrainer.cc:139
list
How EventSelector::AcceptEvent() decides whether to accept an event for output otherwise it is excluding the probing of A single or multiple positive and the trigger will pass if any such matching triggers are PASS or EXCEPTION[A criterion thatmatches no triggers at all is detected and causes a throw.] A single negative with an expectation of appropriate bit checking in the decision and the trigger will pass if any such matching triggers are FAIL or EXCEPTION A wildcarded negative criterion that matches more than one trigger in the trigger list("!*","!HLTx*"if it matches 2 triggers or more) will accept the event if all the matching triggers are FAIL.It will reject the event if any of the triggers are PASS or EXCEPTION(this matches the behavior of"!*"before the partial wildcard feature was incorporated).Triggers which are in the READY state are completely ignored.(READY should never be returned since the trigger paths have been run
def cmsPerfSuite.PerfSuite.simpleGenReport (   self,
  cpus,
  perfdir = os.getcwd(),
  NumEvents = 1,
  candles = ['MinBias'],
  cmsdriverOptions = '',
  stepOptions = '',
  Name = '',
  profilers = '',
  bypasshlt = '',
  userInputRootFiles = '' 
)

Prepares the profiling directory and runs all the selected profiles (if this is not a unit test)

Making parameters named to facilitate the handling of arguments (especially with the threading use case)

Definition at line 779 of file cmsPerfSuite.py.

References cmsPerfSuite.PerfSuite._noexec, cmsPerfSuite.PerfSuite._unittest, cmsPerfSuite.PerfSuite.displayErrors(), cmsPerfSuite.PerfSuite.mkCandleDir(), cmsPerfSuite.PerfSuite.printFlush(), cmsPerfSuite.PerfSuite.runCmsInput(), cmsPerfSuite.PerfSuite.runCmsReport(), cmsPerfSuite.PerfSuite.runPerfSuite(), cmsPerfSuite.PerfSuite.testCmsDriver(), and update.

779 
780  def simpleGenReport(self,cpus,perfdir=os.getcwd(),NumEvents=1,candles=['MinBias'],cmsdriverOptions='',stepOptions='',Name='',profilers='',bypasshlt='',userInputRootFiles=''):
781  callgrind = Name == "Callgrind"
782  memcheck = Name == "Memcheck"
783 
784  profCodes = {"TimeSize" : "0123",
785  "IgProf" : "4567",
786  "IgProf_Perf":"47", #Added the Analyse to IgProf_Perf #FIXME: At the moment Analyse is always run whether 7 is selected or not! Issue to solve in cmsRelvalreportInput.py... but not really important (it's always been there, not impacting our use-cases).
787  "IgProf_Mem":"567",
788  "Callgrind": "8",
789  "Memcheck" : "9",
790  None : "-1"}
791 
792  profiles = profCodes[Name]
793  if not profilers == "":
794  profiles = profilers
795 
796  RelvalreportExitCode=0
797 
798  for cpu in cpus:
799  pfdir = perfdir
800  if len(cpus) > 1:
801  pfdir = os.path.join(perfdir,"cpu_%s" % cpu)
802  for candle in candles:
803  #Create the directory for cmsRelvalreport.py running (e.g. MinBias_TimeSize, etc)
804  #Catch the case of PILE UP:
805  if "--pileup" in cmsdriverOptions:
806  candlename=candle+"_PU"
807  else:
808  candlename=candle
809  adir=self.mkCandleDir(pfdir,candlename,Name)
810  if self._unittest:
811  # Run cmsDriver.py
812  if userInputRootFiles:
813  self.logh.write(userInputRootFiles)
814  userInputFile=userInputRootFiles[0]
815  else:
816  userInputFile=""
817  self.logh.flush()
818  self.runCmsInput(cpu,adir,NumEvents,candle,cmsdriverOptions,stepOptions,profiles,bypasshlt,userInputFile)
819  self.testCmsDriver(cpu,adir,candle)
820  else:
821  if userInputRootFiles:
822  self.logh.write("Variable userInputRootFiles is %s\n"%userInputRootFiles)
823  #Need to use regexp, cannot rely on the order... since for different tests there are different candles...
824  #userInputFile=userInputRootFiles[candles.index(candle)]
825  #FIXME:
826  #Note the issue that the input files HAVE to have in their name the candle as is used in cmsPerfSuite.py command line!
827  #This is currently caught by a printout in the log: should be either taken care of with some exception to throw?
828  #Will put this in the documentation
829  userInputFile=""
830  candleregexp=re.compile(candle)
831  for file in userInputRootFiles:
832  if candleregexp.search(file):
833  userInputFile=file
834  self.logh.write("For these %s %s tests will use user input file %s\n"%(candlename,Name,userInputFile))
835  if userInputFile == "":
836  self.logh.write("***WARNING: For these %s %s tests could not find a matching input file in %s: will try to do without it!!!!!\n"%(candlename,Name,userInputRootFiles))
837  self.logh.flush()
838  else:
839  userInputFile=""
840  DummyTestName=candlename+"_"+stepOptions.split("=")[1]
841  DummyTimer=PerfSuiteTimer(start=datetime.datetime.now()) #Start the timer (DummyTimer is just a reference, but we will use the dictionary to access this later...
842  TimerInfo[Name].update({DummyTestName:DummyTimer}) #Add the TimeSize timer to the dictionary
843  #The following command will create the appropriate SimulationCandlesX.txt file in the relevant directory, ready to run cmsRelvalreport.py on it.
844  self.runCmsInput(cpu,adir,NumEvents,candle,cmsdriverOptions,stepOptions,profiles,bypasshlt,userInputFile)
845  #Here where the no_exec option kicks in (do everything but do not launch cmsRelvalreport.py, it also prevents cmsScimark spawning...):
846  if self._noexec:
847  self.logh.write("Running in debugging mode, without executing cmsRelvalreport.py\n")
848  self.logh.flush()
849  pass
850  else:
851  #The following command will launch cmsRelvalreport.py on the SimulationCandlesX.txt input file created above.
852  ExitCode=self.runCmsReport(cpu,adir,candle)
853  self.logh.write("Individual cmsRelvalreport.py ExitCode %s\n"%ExitCode)
854  RelvalreportExitCode=RelvalreportExitCode+ExitCode
855  self.logh.write("Summed cmsRelvalreport.py ExitCode %s\n"%RelvalreportExitCode)
856  self.logh.flush()
857  DummyTimer.set_end(datetime.datetime.now())
858 
859  #for proflog in proflogs:
860  #With the change from 2>1&|tee to >& to preserve exit codes, we need now to check all logs...
861  #less nice... we might want to do this externally so that in post-processing its a re-usable tool
862  globpath = os.path.join(adir,"*.log") #"%s.log"%candle)
863  self.logh.write("Looking for logs that match %s\n" % globpath)
864  logs = glob.glob(globpath)
865  for log in logs:
866  self.logh.write("Found log %s\n" % log)
867  self.displayErrors(log)
868  self.printFlush("Returned cumulative RelvalreportExitCode is %s"%RelvalreportExitCode)
869  return RelvalreportExitCode
cmsPerfSuite.PerfSuite.mkCandleDir
def mkCandleDir
Make directory for a particular candle and profiler.
Definition: cmsPerfSuite.py:588
cmsPerfSuite.PerfSuite.runCmsInput
def runCmsInput
Wrapper for cmsRelvalreportInput.
Definition: cmsPerfSuite.py:750
cmsPerfSuite.PerfSuite.printFlush
def printFlush
Print and flush a string (for output to a log file)
Definition: cmsPerfSuite.py:545
cmsPerfSuite.PerfSuite.simpleGenReport
def simpleGenReport
Prepares the profiling directory and runs all the selected profiles (if this is not a unit test) ...
Definition: cmsPerfSuite.py:779
cmsPerfSuite.PerfSuite.testCmsDriver
def testCmsDriver
Test cmsDriver.py (parses the simcandles file, removing duplicate lines, and runs the cmsDriver part)...
Definition: cmsPerfSuite.py:717
update
#define update(a, b)
Definition: TrackClassifier.cc:10
cmsPerfSuite.PerfSuite.displayErrors
def displayErrors
Display G4 cerr errors and CMSExceptions in the logfile.
Definition: cmsPerfSuite.py:619
cmsPerfSuite.PerfSuite.runCmsReport
def runCmsReport
This function is a wrapper around cmsRelvalreport.
Definition: cmsPerfSuite.py:700
def cmsPerfSuite.PerfSuite.testCmsDriver (   self,
  cpu,
  dir,
  cmsver,
  candle 
)

Test cmsDriver.py (parses the simcandles file, removing duplicate lines, and runs the cmsDriver part)

Definition at line 717 of file cmsPerfSuite.py.

References cmsPerfSuite.PerfSuite._dryrun, cmsPerfSuite.PerfSuite.Commands, and cmsPerfSuite.PerfSuite.runCmdSet().

Referenced by cmsPerfSuite.PerfSuite.simpleGenReport().

717 
718  def testCmsDriver(self,cpu,dir,cmsver,candle):
719  cmsdrvreg = re.compile("^cmsDriver.py")
720  cmd = self.Commands[cpu][0]
721  noExit = True
722  stepreg = re.compile("--step=([^ ]*)")
723  previousCmdOnline = ""
724  for line in open(os.path.join(dir,"SimulationCandles_%s.txt" % (cmsver))):
725  if (not line.lstrip().startswith("#")) and not (line.isspace() or len(line) == 0):
726  cmdonline = line.split("@@@",1)[0]
727  if cmsdrvreg.search(cmdonline) and not previousCmdOnline == cmdonline:
728  stepbeingrun = "Unknown"
729  matches = stepreg.search(cmdonline)
730  if not matches == None:
731  stepbeingrun = matches.groups()[0]
732  if "PILEUP" in cmdonline:
733  stepbeingrun += "_PILEUP"
734  self.logh.write(cmdonline + "\n")
735  cmds = ("cd %s" % (dir),
736  "%s >& ../cmsdriver_unit_test_%s_%s.log" % (cmdonline,candle,stepbeingrun))
737  if self._dryrun:
738  self.logh.write(cmds + "\n")
739  else:
740  out = self.runCmdSet(cmds)
741  if not out == None:
742  sig = out >> 16 # Get the top 16 bits
743  xstatus = out & 0xffff # Mask out all bits except the first 16
744  self.logh.write("FATAL ERROR: CMS Driver returned a non-zero exit status (which is %s) when running %s for candle %s. Signal interrupt was %s\n" % (xstatus,stepbeingrun,candle,sig))
745  sys.exit()
746  previousCmdOnline = cmdonline
cmsPerfSuite.PerfSuite.runCmdSet
def runCmdSet
Run a list of commands using system ! We should rewrite this not to use system (most cases it is unne...
Definition: cmsPerfSuite.py:527
cmsPerfSuite.PerfSuite.testCmsDriver
def testCmsDriver
Test cmsDriver.py (parses the simcandles file, removing duplicate lines, and runs the cmsDriver part)...
Definition: cmsPerfSuite.py:717
def cmsPerfSuite.PerfSuite.valFilterReport (   self,
  dir 
)

Filter lines in the valgrind report that match GEN,SIM.

Definition at line 635 of file cmsPerfSuite.py.

References cmsPerfSuite.PerfSuite.cmssw_version.

635 
636  def valFilterReport(self,dir):
637  #cmds = ("cd %s" % dir,
638  # "grep -v \"step=GEN,SIM\" SimulationCandles_%s.txt > tmp" % (self.cmssw_version),
639  # "mv tmp SimulationCandles_%s.txt" % (self.cmssw_version))
640  #FIXME:
641  #Quick and dirty hack to have valgrind MemCheck run on 5 events on both GEN,SIM and DIGI in QCD_80_120, while removing the line for GEN,SIM for Callgrind
642  InputFileName=os.path.join(dir,"SimulationCandles_%s.txt"%(self.cmssw_version))
643  InputFile=open(InputFileName,"r")
644  InputLines=InputFile.readlines()
645  InputFile.close()
646  Outputfile=open(InputFileName,"w")
647  simRegxp=re.compile("step=GEN,SIM")
648  digiRegxp=re.compile("step=DIGI")
649  CallgrindRegxp=re.compile("ValgrindFCE")
650  MemcheckRegxp=re.compile("Memcheck")
651  NumEvtRegxp=re.compile("-n 1")#FIXME Either use the ValgrindEventNumber or do a more general match!
652  for line in InputLines:
653  if simRegxp.search(line) and CallgrindRegxp.search(line):
654  continue
655  elif simRegxp.search(line) and MemcheckRegxp.search(line):
656  #Modify
657  if NumEvtRegxp.search(line):
658  line=NumEvtRegxp.sub(r"-n 5",line)
659  else:
660  self.logh.write("The number of Memcheck event was not changed since the original number of Callgrind event was not 1!\n")
661  Outputfile.write(line)
662  elif digiRegxp.search(line) and MemcheckRegxp.search(line):
663  #Modify
664  if NumEvtRegxp.search(line):
665  line=NumEvtRegxp.sub(r"-n 5",line)
666  else:
667  self.logh.write("The number of Memcheck event was not changed since the original number of Callgrind event was not 1!\n")
668  Outputfile.write(line)
669  else:
670  Outputfile.write(line)
671  self.logh.flush()
672  Outputfile.close()
673 
674  #self.runCmdSet(cmds)
cmsPerfSuite.PerfSuite.valFilterReport
def valFilterReport
Filter lines in the valgrind report that match GEN,SIM.
Definition: cmsPerfSuite.py:635

Member Data Documentation

cmsPerfSuite.PerfSuite._CASTOR_DIR
private

Definition at line 64 of file cmsPerfSuite.py.

Referenced by cmsPerfSuite.PerfSuite.optionParse().

cmsPerfSuite.PerfSuite._debug
private

Developer options.

Definition at line 66 of file cmsPerfSuite.py.

Referenced by cmsPerfSuite.PerfSuite.optionParse(), and cmsPerfSuite.PerfSuite.runCmsReport().

cmsPerfSuite.PerfSuite._dryrun
private

Definition at line 65 of file cmsPerfSuite.py.

Referenced by cmsPerfSuite.PerfSuite.optionParse(), and cmsPerfSuite.PerfSuite.testCmsDriver().

cmsPerfSuite.PerfSuite._noexec
private

Definition at line 68 of file cmsPerfSuite.py.

Referenced by cmsPerfSuite.PerfSuite.optionParse(), and cmsPerfSuite.PerfSuite.simpleGenReport().

cmsPerfSuite.PerfSuite._unittest
private

Definition at line 67 of file cmsPerfSuite.py.

Referenced by cmsPerfSuite.PerfSuite.optionParse(), cmsPerfSuite.PerfSuite.runCmsInput(), cmsPerfSuite.PerfSuite.runCmsReport(), and cmsPerfSuite.PerfSuite.simpleGenReport().

cmsPerfSuite.PerfSuite._verbose
private

Check logfile option.

Check step Options Check profile option Check output directory option Check cpu option Check previous release directory Setup quicktest option Setup unit test option

Definition at line 69 of file cmsPerfSuite.py.

Referenced by cmsPerfSuite.PerfSuite.mkCandleDir(), cmsPerfSuite.PerfSuite.optionParse(), cmsPerfSuite.PerfSuite.printFlush(), cmsPerfSuite.PerfSuite.runcmd(), and cmsPerfSuite.PerfSuite.runCmdSet().

cmsPerfSuite.PerfSuite.AuxiliaryScripts

Definition at line 86 of file cmsPerfSuite.py.

cmsPerfSuite.PerfSuite.cmssw_arch

Definition at line 74 of file cmsPerfSuite.py.

cmsPerfSuite.PerfSuite.cmssw_version

Definition at line 75 of file cmsPerfSuite.py.

Referenced by cmsPerfSuite.PerfSuite.runCmsReport(), and cmsPerfSuite.PerfSuite.valFilterReport().

cmsPerfSuite.PerfSuite.Commands

Definition at line 1006 of file cmsPerfSuite.py.

Referenced by cmsPerfSuite.PerfSuite.benchmarks(), cmsPerfSuite.PerfSuite.runCmsInput(), cmsPerfSuite.PerfSuite.runCmsReport(), and cmsPerfSuite.PerfSuite.testCmsDriver().

cmsPerfSuite.PerfSuite.ERRORS

Definition at line 63 of file cmsPerfSuite.py.

Referenced by cmsPerfSuite.PerfSuite.displayErrors().

cmsPerfSuite.PerfSuite.HEPSPEC06

Definition at line 922 of file cmsPerfSuite.py.

cmsPerfSuite.PerfSuite.host

Definition at line 76 of file cmsPerfSuite.py.

cmsPerfSuite.PerfSuite.logh

Definition at line 70 of file cmsPerfSuite.py.

Referenced by cmsPerfSuite.PerfSuite.runPerfSuite().

cmsPerfSuite.PerfSuite.PerfTestTotalTimer

FIXME: We may want to introduce a switch here or agree on a different default (currently 10 cmsScimark and 10 cmsScimarkLarge)

PERF## Prepare the simpleGenReport arguments for this test: MEM## Prepare the simpleGenReport arguments for this test: PERF## Prepare the simpleGenReport arguments for this test: MEM## Prepare the simpleGenReport arguments for this test:

Definition at line 1397 of file cmsPerfSuite.py.

cmsPerfSuite.PerfSuite.Scripts

Definition at line 85 of file cmsPerfSuite.py.

cmsPerfSuite.PerfSuite.user

Definition at line 77 of file cmsPerfSuite.py.

Referenced by cmsPerfSuite.PerfSuite.optionParse().

Generated for CMSSW Reference Manual by doxygen 1.8.5