test
CMS 3D CMS Logo
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Properties Friends Macros Pages
List of all members | Public Member Functions | Public Attributes
MatrixReader.MatrixReader Class Reference
Inheritance diagram for MatrixReader.MatrixReader:

Public Member Functions

def __init__
 
def createWorkFlows
 
def makeCmd
 
def makeStep
 
def prepare
 
def readMatrix
 
def reset
 
def show
 
def showRaw
 
def showWorkFlows
 
def updateDB
 
def workFlowsByLocation
 

Public Attributes

 addCommand
 
 apply
 
 commandLineWf
 
 files
 
 filesDefault
 
 filesPrefMap
 
 nameList
 
 noRun
 
 overWrite
 
 relvalModule
 
 revertDqmio
 maybe we want too level deep input More...
 
 what
 
 wm
 
 workFlows
 
 workFlowSteps
 

Detailed Description

Definition at line 16 of file MatrixReader.py.

Constructor & Destructor Documentation

def MatrixReader.MatrixReader.__init__ (   self,
  opt 
)

Definition at line 18 of file MatrixReader.py.

References PuppiAlgo.reset(), BinningPointByMap.reset(), TPedResult.reset(), ora::NamedReference.reset(), AlcaBeamSpotManager.reset(), ora::IRelationalOperation.reset(), ApvLatencyAnalysis.reset(), VEcalCalibBlock.reset(), ora::Handle< T >.reset(), MatrixReader.MatrixReader.reset(), TB06Tree.reset(), ora::IteratorBuffer.reset(), EcalCondHeader.reset(), ora::ContainerIterator.reset(), IMACalibBlock.reset(), ora::OId.reset(), SamplingAnalysis.reset(), L3CalibBlock.reset(), FactorizedJetCorrectorCalculator::VariableValues.reset(), DaqScopeModeAnalysis.reset(), ora::MultiRecordInsertOperation.reset(), pos::PixelROCStatus.reset(), FedTimingAnalysis.reset(), TB06TreeH2.reset(), L1MuDTTFMasks.reset(), L1MuDTTFParameters.reset(), L1MuDTPhiLut.reset(), L1MuDTPtaLut.reset(), FedCablingAnalysis.reset(), L1MuDTEtaPatternLut.reset(), L1MuDTQualPatternLut.reset(), PedsOnlyAnalysis.reset(), TB06Reco.reset(), L1MuDTExtLut.reset(), VpspScanAnalysis.reset(), PhysicsTools::Calibration::Histogram< Value_t, Axis_t >.reset(), coral_bridge::AuthenticationCredentialSet.reset(), PhysicsTools::Calibration::Histogram2D< Value_t, AxisX_t, AxisY_t >.reset(), NoiseAnalysis.reset(), PedestalsAnalysis.reset(), OptoScanAnalysis.reset(), ApvTimingAnalysis.reset(), CalibrationAnalysis.reset(), PhysicsTools::Calibration::Histogram3D< Value_t, AxisX_t, AxisY_t, AxisZ_t >.reset(), FastFedCablingAnalysis.reset(), PedsFullNoiseAnalysis.reset(), ora::UniqueRef< T >.reset(), cond::persistency::GTProxy.reset(), cond::persistency::IOVProxy.reset(), CommissioningAnalysis.reset(), ora::InsertOperation.reset(), ora::Ptr< T >.reset(), TB06RecoH2.reset(), ora::BulkInsertOperation.reset(), cond::XMLAuthenticationService::XMLAuthenticationService.reset(), TrackerMap.reset(), ora::UpdateOperation.reset(), ora::DeleteOperation.reset(), data_sources.json_list.reset(), DTTFBitArray< N >.reset(), ora::QueryableVector< Tp >.reset(), and mps_create_file_lists._DasCache.reset().

18 
19  def __init__(self, opt):
20 
21  self.reset(opt.what)
22 
23  self.wm=opt.wmcontrol
24  self.revertDqmio=opt.revertDqmio
25  self.addCommand=opt.command
26  self.apply=opt.apply
27  self.commandLineWf=opt.workflow
28  self.overWrite=opt.overWrite
29 
30  self.noRun = opt.noRun
31  return
MatrixReader.MatrixReader.revertDqmio
revertDqmio
maybe we want too level deep input
Definition: MatrixReader.py:23

Member Function Documentation

def MatrixReader.MatrixReader.createWorkFlows (   self,
  fileNameIn 
)

Definition at line 453 of file MatrixReader.py.

References MatrixReader.MatrixReader.filesPrefMap, MatrixReader.MatrixReader.nameList, and MatrixReader.MatrixReader.workFlowSteps.

Referenced by MatrixReader.MatrixReader.prepare().

453 
454  def createWorkFlows(self, fileNameIn):
455 
456  prefixIn = self.filesPrefMap[fileNameIn]
457 
458  # get through the list of items and update the requested workflows only
459  keyList = self.workFlowSteps.keys()
460  ids = []
461  for item in keyList:
462  id, pref = item
463  if pref != prefixIn : continue
464  ids.append(id)
465  ids.sort()
466  for key in ids:
467  val = self.workFlowSteps[(key,prefixIn)]
468  num, name, commands, stepList = val
469  nameId = str(num)+'_'+name
470  if nameId in self.nameList:
471  print "==> duplicate name found for ", nameId
472  print ' keeping : ', self.nameList[nameId]
473  print ' ignoring : ', val
474  else:
475  self.nameList[nameId] = val
476 
477  self.workFlows.append(WorkFlow(num, name, commands=commands))
478 
479  return
def MatrixReader.MatrixReader.makeCmd (   self,
  step 
)

Definition at line 93 of file MatrixReader.py.

Referenced by MatrixReader.MatrixReader.readMatrix().

93 
94  def makeCmd(self, step):
95 
96  cmd = ''
97  cfg = None
98  input = None
99  for k,v in step.items():
100  if 'no_exec' in k : continue # we want to really run it ...
101  if k.lower() == 'cfg':
102  cfg = v
103  continue # do not append to cmd, return separately
104  if k.lower() == 'input':
105  input = v
106  continue # do not append to cmd, return separately
107 
108  #chain the configs
109  #if k.lower() == '--python':
110  # v = 'step%d_%s'%(index,v)
111  cmd += ' ' + k + ' ' + str(v)
112  return cfg, input, cmd
def MatrixReader.MatrixReader.makeStep (   self,
  step,
  overrides 
)

Definition at line 113 of file MatrixReader.py.

Referenced by MatrixReader.MatrixReader.readMatrix().

113 
114  def makeStep(self,step,overrides):
115  from Configuration.PyReleaseValidation.relval_steps import merge
116  if len(overrides.keys()) > 0:
117  copyStep=merge([overrides]+[step])
118  return copyStep
119  else:
120  return step
merge
Definition: merge.py:1
relval_steps
def MatrixReader.MatrixReader.prepare (   self,
  useInput = None,
  refRel = '',
  fromScratch = None 
)

Definition at line 480 of file MatrixReader.py.

References MatrixReader.MatrixReader.createWorkFlows(), MatrixReader.MatrixReader.files, geometryComparison.GeometryComparison.files, MatrixReader.MatrixReader.filesDefault, MatrixReader.MatrixReader.readMatrix(), and MatrixReader.MatrixReader.what.

480 
481  def prepare(self, useInput=None, refRel='', fromScratch=None):
482 
483  for matrixFile in self.files:
484  if self.what != 'all' and self.what not in matrixFile:
485  print "ignoring non-requested file",matrixFile
486  continue
487  if self.what == 'all' and not self.filesDefault[matrixFile]:
488  print "ignoring",matrixFile,"from default matrix"
489  continue
490 
491  try:
492  self.readMatrix(matrixFile, useInput, refRel, fromScratch)
493  except Exception as e:
494  print "ERROR reading file:", matrixFile, str(e)
495  raise
496 
497  try:
498  self.createWorkFlows(matrixFile)
499  except Exception as e:
500  print "ERROR creating workflows :", str(e)
501  raise
502 
def MatrixReader.MatrixReader.readMatrix (   self,
  fileNameIn,
  useInput = None,
  refRel = None,
  fromScratch = None 
)

Definition at line 121 of file MatrixReader.py.

References MatrixReader.MatrixReader.addCommand, MatrixReader.MatrixReader.apply, MatrixReader.MatrixReader.filesPrefMap, join(), list(), MatrixReader.MatrixReader.makeCmd(), MatrixReader.MatrixReader.makeStep(), MatrixReader.MatrixReader.noRun, MatrixReader.MatrixReader.overWrite, MatrixReader.MatrixReader.relvalModule, MatrixReader.MatrixReader.revertDqmio, MatrixReader.MatrixReader.wm, MatrixReader.MatrixReader.workFlowSteps, and ComparisonHelper.zip().

Referenced by MatrixReader.MatrixReader.prepare(), and MatrixReader.MatrixReader.showRaw().

121 
122  def readMatrix(self, fileNameIn, useInput=None, refRel=None, fromScratch=None):
123 
124  prefix = self.filesPrefMap[fileNameIn]
125 
126  print "processing", fileNameIn
127 
128  try:
129  _tmpMod = __import__( 'Configuration.PyReleaseValidation.'+fileNameIn )
130  self.relvalModule = sys.modules['Configuration.PyReleaseValidation.'+fileNameIn]
131  except Exception as e:
132  print "ERROR importing file ", fileNameIn, str(e)
133  return
134 
135  if useInput is not None:
136  print "request for INPUT for ", useInput
137 
138 
139  fromInput={}
140 
141  if useInput:
142  for i in useInput:
143  if ':' in i:
144  (ik,il)=i.split(':')
145  if ik=='all':
146  for k in self.relvalModule.workflows.keys():
147  fromInput[float(k)]=int(il)
148  else:
149  fromInput[float(ik)]=int(il)
150  else:
151  if i=='all':
152  for k in self.relvalModule.workflows.keys():
153  fromInput[float(k)]=0
154  else:
155  fromInput[float(i)]=0
156 
157  if fromScratch:
158  fromScratch=map(float,fromScratch)
159  for num in fromScratch:
160  if num in fromInput:
161  fromInput.pop(num)
162  #overwrite steps
163  if self.overWrite:
164  for p in self.overWrite:
165  self.relvalModule.steps.overwrite(p)
166 
167  #change the origin of dataset on the fly
168  if refRel:
169  if ',' in refRel:
170  refRels=refRel.split(',')
171  if len(refRels)!=len(self.relvalModule.baseDataSetRelease):
172  return
173  self.relvalModule.changeRefRelease(
174  self.relvalModule.steps,
175  list(zip(self.relvalModule.baseDataSetRelease,refRels))
176  )
177  else:
178  self.relvalModule.changeRefRelease(
179  self.relvalModule.steps,
180  [(x,refRel) for x in self.relvalModule.baseDataSetRelease]
181  )
182 
183 
184  for num, wfInfo in self.relvalModule.workflows.items():
185  commands=[]
186  wfName = wfInfo[0]
187  stepList = wfInfo[1]
188  stepOverrides=wfInfo.overrides
189  # if no explicit name given for the workflow, use the name of step1
190  if wfName.strip() == '': wfName = stepList[0]
191  # option to specialize the wf as the third item in the WF list
192  addTo=None
193  addCom=None
194  if len(wfInfo)>=3:
195  addCom=wfInfo[2]
196  if not type(addCom)==list: addCom=[addCom]
197  #print 'added dict',addCom
198  if len(wfInfo)>=4:
199  addTo=wfInfo[3]
200  #pad with 0
201  while len(addTo)!=len(stepList):
202  addTo.append(0)
203 
204  name=wfName
205  stepIndex=0
206  ranStepList=[]
207 
208  #first resolve INPUT possibilities
209  if num in fromInput:
210  ilevel=fromInput[num]
211  #print num,ilevel
212  for (stepIr,step) in enumerate(reversed(stepList)):
213  stepName=step
214  stepI=(len(stepList)-stepIr)-1
215  #print stepIr,step,stepI,ilevel
216  if stepI>ilevel:
217  #print "ignoring"
218  continue
219  if stepI!=0:
220  testName='__'.join(stepList[0:stepI+1])+'INPUT'
221  else:
222  testName=step+'INPUT'
223  #print "JR",stepI,stepIr,testName,stepList
224  if testName in self.relvalModule.steps.keys():
225  #print "JR",stepI,stepIr
226  stepList[stepI]=testName
227  #pop the rest in the list
228  #print "\tmod prepop",stepList
229  for p in range(stepI):
230  stepList.pop(0)
231  #print "\t\tmod",stepList
232  break
233 
234 
235  for (stepI,step) in enumerate(stepList):
236  stepName=step
237  if self.wm:
238  #cannot put a certain number of things in wm
239  if stepName in [
240  #'HARVEST','HARVESTD','HARVESTDreHLT',
241  'RECODFROMRAWRECO','SKIMD','SKIMCOSD','SKIMDreHLT'
242  ]:
243  continue
244 
245  #replace stepName is needed
246  #if stepName in self.replaceStep
247  if len(name) > 0 : name += '+'
248  #any step can be mirrored with INPUT
249  ## maybe we want too level deep input
250  """
251  if num in fromInput:
252  if step+'INPUT' in self.relvalModule.steps.keys():
253  stepName = step+"INPUT"
254  stepList.remove(step)
255  stepList.insert(stepIndex,stepName)
256  """
257  name += stepName
258  if addCom and (not addTo or addTo[stepIndex]==1):
259  from Configuration.PyReleaseValidation.relval_steps import merge
260  copyStep=merge(addCom+[self.makeStep(self.relvalModule.steps[stepName],stepOverrides)])
261  cfg, input, opts = self.makeCmd(copyStep)
262  else:
263  cfg, input, opts = self.makeCmd(self.makeStep(self.relvalModule.steps[stepName],stepOverrides))
264 
265  if input and cfg :
266  msg = "FATAL ERROR: found both cfg and input for workflow "+str(num)+' step '+stepName
267  raise MatrixException(msg)
268 
269  if input:
270  cmd = input
271  if self.noRun:
272  cmd.run=[]
273  else:
274  if cfg:
275  cmd = 'cmsDriver.py '+cfg+' '+opts
276  else:
277  cmd = 'cmsDriver.py step'+str(stepIndex+1)+' '+opts
278  if self.wm:
279  cmd+=' --io %s.io --python %s.py'%(stepName,stepName)
280  if self.addCommand:
281  if self.apply:
282  if stepIndex in self.apply or stepName in self.apply:
283  cmd +=' '+self.addCommand
284  else:
285  cmd +=' '+self.addCommand
286  if self.wm and self.revertDqmio=='yes':
287  cmd=cmd.replace('DQMIO','DQM')
288  cmd=cmd.replace('--filetype DQM','')
289  commands.append(cmd)
290  ranStepList.append(stepName)
291  stepIndex+=1
292 
293  self.workFlowSteps[(num,prefix)] = (num, name, commands, ranStepList)
294 
295  return
296 
merge
Definition: merge.py:1
MatrixReader.MatrixReader.revertDqmio
revertDqmio
maybe we want too level deep input
Definition: MatrixReader.py:23
relval_steps
ComparisonHelper::zip
OutputIterator zip(InputIterator1 first1, InputIterator1 last1, InputIterator2 first2, InputIterator2 last2, OutputIterator result, Compare comp)
Definition: L1TStage2CaloLayer1.h:38
join
static std::string join(char **cmd)
Definition: RemoteFile.cc:18
list
How EventSelector::AcceptEvent() decides whether to accept an event for output otherwise it is excluding the probing of A single or multiple positive and the trigger will pass if any such matching triggers are PASS or EXCEPTION[A criterion thatmatches no triggers at all is detected and causes a throw.] A single negative with an expectation of appropriate bit checking in the decision and the trigger will pass if any such matching triggers are FAIL or EXCEPTION A wildcarded negative criterion that matches more than one trigger in the trigger list("!*","!HLTx*"if it matches 2 triggers or more) will accept the event if all the matching triggers are FAIL.It will reject the event if any of the triggers are PASS or EXCEPTION(this matches the behavior of"!*"before the partial wildcard feature was incorporated).Triggers which are in the READY state are completely ignored.(READY should never be returned since the trigger paths have been run
def MatrixReader.MatrixReader.reset (   self,
  what = 'all' 
)

Definition at line 32 of file MatrixReader.py.

Referenced by MatrixReader.MatrixReader.__init__(), and MatrixReader.MatrixReader.showRaw().

32 
33  def reset(self, what='all'):
34 
35  self.what = what
36 
37  #a bunch of information, but not yet the WorkFlow object
38  self.workFlowSteps = {}
39  #the actual WorkFlow objects
40  self.workFlows = []
41  self.nameList = {}
42 
43  self.filesPrefMap = {'relval_standard' : 'std-' ,
44  'relval_highstats': 'hi-' ,
45  'relval_pileup': 'PU-' ,
46  'relval_generator': 'gen-',
47  'relval_extendedgen': 'genExt-',
48  'relval_production': 'prod-' ,
49  'relval_ged': 'ged-',
50  'relval_upgrade':'upg-',
51  'relval_2017':'2017-',
52  'relval_2023':'2023-',
53  'relval_identity':'id-',
54  'relval_machine': 'mach-',
55  'relval_unsch': 'unsch-',
56  'relval_premix': 'premix-'
57  }
58 
59  self.files = ['relval_standard' ,
60  'relval_highstats',
61  'relval_pileup',
62  'relval_generator',
63  'relval_extendedgen',
64  'relval_production',
65  'relval_ged',
66  'relval_upgrade',
67  'relval_2017',
68  'relval_2023',
69  'relval_identity',
70  'relval_machine',
71  'relval_unsch',
72  'relval_premix'
73  ]
74  self.filesDefault = {'relval_standard':True ,
75  'relval_highstats':True ,
76  'relval_pileup':True,
77  'relval_generator':True,
78  'relval_extendedgen':True,
79  'relval_production':True,
80  'relval_ged':True,
81  'relval_upgrade':False,
82  'relval_2017':True,
83  'relval_2023':True,
84  'relval_identity':False,
85  'relval_machine':True,
86  'relval_unsch':True,
87  'relval_premix':True
88  }
89 
90  self.relvalModule = None
91 
92  return
def MatrixReader.MatrixReader.show (   self,
  selected = None,
  extended = True,
  cafVeto = True 
)

Definition at line 503 of file MatrixReader.py.

References MatrixReader.MatrixReader.showWorkFlows().

Referenced by Vispa.Main.AboutDialog.AboutDialog.onScreen(), Vispa.Gui.BoxContentDialog.BoxContentDialog.onScreen(), and Vispa.Gui.FindDialog.FindDialog.onScreen().

503 
504  def show(self, selected=None, extended=True, cafVeto=True):
505 
506  self.showWorkFlows(selected, extended, cafVeto)
507  print '\n','-'*80,'\n'
508 
def MatrixReader.MatrixReader.showRaw (   self,
  useInput,
  refRel = None,
  fromScratch = None,
  what = 'all',
  step1Only = False,
  selected = None 
)

Definition at line 297 of file MatrixReader.py.

References Clusterizer1DCommons.add(), MatrixReader.MatrixReader.files, geometryComparison.GeometryComparison.files, MatrixReader.MatrixReader.filesDefault, join(), MatrixReader.MatrixReader.readMatrix(), PuppiAlgo.reset(), TPedResult.reset(), BinningPointByMap.reset(), ora::NamedReference.reset(), AlcaBeamSpotManager.reset(), ora::IRelationalOperation.reset(), VEcalCalibBlock.reset(), ApvLatencyAnalysis.reset(), MatrixReader.MatrixReader.reset(), ora::Handle< T >.reset(), TB06Tree.reset(), ora::IteratorBuffer.reset(), EcalCondHeader.reset(), ora::ContainerIterator.reset(), IMACalibBlock.reset(), ora::OId.reset(), SamplingAnalysis.reset(), L3CalibBlock.reset(), FactorizedJetCorrectorCalculator::VariableValues.reset(), DaqScopeModeAnalysis.reset(), ora::MultiRecordInsertOperation.reset(), pos::PixelROCStatus.reset(), FedTimingAnalysis.reset(), TB06TreeH2.reset(), L1MuDTTFMasks.reset(), L1MuDTTFParameters.reset(), L1MuDTPhiLut.reset(), L1MuDTPtaLut.reset(), FedCablingAnalysis.reset(), L1MuDTEtaPatternLut.reset(), L1MuDTQualPatternLut.reset(), PedsOnlyAnalysis.reset(), TB06Reco.reset(), VpspScanAnalysis.reset(), L1MuDTExtLut.reset(), PhysicsTools::Calibration::Histogram< Value_t, Axis_t >.reset(), coral_bridge::AuthenticationCredentialSet.reset(), PhysicsTools::Calibration::Histogram2D< Value_t, AxisX_t, AxisY_t >.reset(), NoiseAnalysis.reset(), PedestalsAnalysis.reset(), OptoScanAnalysis.reset(), ApvTimingAnalysis.reset(), CalibrationAnalysis.reset(), PhysicsTools::Calibration::Histogram3D< Value_t, AxisX_t, AxisY_t, AxisZ_t >.reset(), FastFedCablingAnalysis.reset(), PedsFullNoiseAnalysis.reset(), ora::UniqueRef< T >.reset(), cond::persistency::GTProxy.reset(), cond::persistency::IOVProxy.reset(), CommissioningAnalysis.reset(), ora::InsertOperation.reset(), ora::Ptr< T >.reset(), TB06RecoH2.reset(), ora::BulkInsertOperation.reset(), cond::XMLAuthenticationService::XMLAuthenticationService.reset(), TrackerMap.reset(), ora::UpdateOperation.reset(), ora::DeleteOperation.reset(), data_sources.json_list.reset(), DTTFBitArray< N >.reset(), ora::QueryableVector< Tp >.reset(), mps_create_file_lists._DasCache.reset(), MatrixReader.MatrixReader.revertDqmio, MatrixReader.MatrixReader.what, and MatrixReader.MatrixReader.workFlowSteps.

297 
298  def showRaw(self, useInput, refRel=None, fromScratch=None, what='all',step1Only=False,selected=None):
299 
300  if selected:
301  selected=map(float,selected)
302  for matrixFile in self.files:
303 
304  self.reset(what)
305 
306  if self.what != 'all' and self.what not in matrixFile:
307  print "ignoring non-requested file",matrixFile
308  continue
309 
310  if self.what == 'all' and not self.filesDefault[matrixFile]:
311  print "ignoring file not used by default (enable with -w)",matrixFile
312  continue
313 
314  try:
315  self.readMatrix(matrixFile, useInput, refRel, fromScratch)
316  except Exception as e:
317  print "ERROR reading file:", matrixFile, str(e)
318  raise
319 
320  if not self.workFlowSteps: continue
321 
322  dataFileName = matrixFile.replace('relval_', 'cmsDriver_')+'_hlt.txt'
323  outFile = open(dataFileName,'w')
324 
325  print "found ", len(self.workFlowSteps.keys()), ' workflows for ', dataFileName
326  ids = self.workFlowSteps.keys()
327  ids.sort()
328  indexAndSteps=[]
329 
330  writtenWF=0
331  for key in ids:
332  if selected and not (key[0] in selected):
333  continue
334  #trick to skip the HImix IB test
335  if key[0]==203.1 or key[0]==204.1 or key[0]==205.1 or key[0]==4.51 or key[0]==4.52: continue
336  num, name, commands, stepList = self.workFlowSteps[key]
337 
338  wfName,stepNames= name.split('+',1)
339 
340  stepNames=stepNames.replace('+RECODFROMRAWRECO','')
341  stepNames=stepNames.replace('+SKIMCOSD','')
342  stepNames=stepNames.replace('+SKIMD','')
343  if 'HARVEST' in stepNames:
344  #find out automatically what to remove
345  exactb=stepNames.index('+HARVEST')
346  exacte=stepNames.index('+',exactb+1) if ('+' in stepNames[exactb+1:]) else (len(stepNames))
347  stepNames=stepNames.replace(stepNames[exactb:exacte],'')
348  otherSteps = None
349  if '+' in stepNames:
350  step1,otherSteps = stepNames.split('+',1)
351 
352  line = str(num) + ' ++ '+ wfName
353  if otherSteps and not step1Only:
354  line += ' ++ ' +otherSteps.replace('+',',')
355  else:
356  line += ' ++ none'
357  inputInfo=None
358  if not isinstance(commands[0],str):
359  inputInfo=commands[0]
360  if otherSteps:
361  for (i,c) in enumerate(otherSteps.split('+')):
362  #pad with set
363  for p in range(len(indexAndSteps),i+2):
364  indexAndSteps.append(set())
365  indexAndSteps[i+1].add((c,commands[i+1]))
366 
367  if inputInfo :
368  #skip the samples from INPUT when step1Only is on
369  if step1Only: continue
370  line += ' ++ REALDATA: '+inputInfo.dataSet
371  if inputInfo.run!=[]: line += ', RUN:'+'|'.join(map(str,inputInfo.run))
372  line += ', FILES: ' +str(inputInfo.files)
373  line += ', EVENTS: '+str(inputInfo.events)
374  if inputInfo.label!='':
375  line += ', LABEL: ' +inputInfo.label
376  line += ', LOCATION:'+inputInfo.location
377  line += ' @@@'
378  else:
379  line += ' @@@ '+commands[0]
380  if self.revertDqmio=='yes':
381  line=line.replace('DQMIO','DQM')
382  writtenWF+=1
383  outFile.write(line+'\n')
384 
385 
386  outFile.write('\n'+'\n')
387  if step1Only: continue
388 
389  for (index,s) in enumerate(indexAndSteps):
390  for (stepName,cmd) in s:
391  stepIndex=index+1
392  if 'dasquery.log' in cmd: continue
393  line = 'STEP%d ++ '%(stepIndex,) +stepName + ' @@@ '+cmd
394  if self.revertDqmio=='yes':
395  line=line.replace('DQMIO','DQM')
396  outFile.write(line+'\n')
397  outFile.write('\n'+'\n')
398  outFile.close()
399  print "wrote ",writtenWF, ' workflow'+('s' if (writtenWF!=1) else ''),' to ', outFile.name
400  return
MatrixReader.MatrixReader.revertDqmio
revertDqmio
maybe we want too level deep input
Definition: MatrixReader.py:23
Clusterizer1DCommons::add
void add(const std::vector< const T * > &source, std::vector< const T * > &dest)
Definition: Clusterizer1DCommons.h:26
join
static std::string join(char **cmd)
Definition: RemoteFile.cc:18
def MatrixReader.MatrixReader.showWorkFlows (   self,
  selected = None,
  extended = True,
  cafVeto = True 
)

Definition at line 416 of file MatrixReader.py.

References MatrixReader.MatrixReader.workFlowsByLocation().

Referenced by MatrixReader.MatrixReader.show().

416 
417  def showWorkFlows(self, selected=None, extended=True, cafVeto=True):
418  if selected: selected = map(float,selected)
419  wfs = self.workFlowsByLocation(cafVeto)
420  maxLen = 100 # for summary, limit width of output
421  fmt1 = "%-6s %-35s [1]: %s ..."
422  fmt2 = " %35s [%d]: %s ..."
423  print "\nfound a total of ", len(wfs), ' workflows:'
424  if selected:
425  print " of which the following", len(selected), 'were selected:'
426  #-ap for now:
427  maxLen = -1 # for individual listing, no limit on width
428  fmt1 = "%-6s %-35s [1]: %s "
429  fmt2 = " %35s [%d]: %s"
430 
431  N=[]
432  for wf in wfs:
433  if selected and float(wf.numId) not in selected: continue
434  if extended: print ''
435  #pad with zeros
436  for i in range(len(N),len(wf.cmds)): N.append(0)
437  N[len(wf.cmds)-1]+=1
438  wfName, stepNames = wf.nameId.split('+',1)
439  for i,s in enumerate(wf.cmds):
440  if extended:
441  if i==0:
442  print fmt1 % (wf.numId, stepNames, (str(s)+' ')[:maxLen])
443  else:
444  print fmt2 % ( ' ', i+1, (str(s)+' ')[:maxLen])
445  else:
446  print "%-6s %-35s "% (wf.numId, stepNames)
447  break
448  print ''
449  for i,n in enumerate(N):
450  if n: print n,'workflows with',i+1,'steps'
451 
452  return
def MatrixReader.MatrixReader.updateDB (   self)

Definition at line 509 of file MatrixReader.py.

References MatrixReader.MatrixReader.workFlows.

509 
510  def updateDB(self):
511 
512  import pickle
513  pickle.dump(self.workFlows, open('theMatrix.pkl', 'w') )
514 
515  return
516 
def MatrixReader.MatrixReader.workFlowsByLocation (   self,
  cafVeto = True 
)

Definition at line 401 of file MatrixReader.py.

References MatrixReader.MatrixReader.workFlows.

Referenced by MatrixReader.MatrixReader.showWorkFlows().

401 
402  def workFlowsByLocation(self, cafVeto=True):
403  # Check if we are on CAF
404  onCAF = False
405  if 'cms/caf/cms' in os.environ['CMS_PATH']:
406  onCAF = True
407 
408  workflows = []
409  for workflow in self.workFlows:
410  if isinstance(workflow.cmds[0], InputInfo):
411  if cafVeto and (workflow.cmds[0].location == 'CAF' and not onCAF):
412  continue
413  workflows.append(workflow)
414 
415  return workflows

Member Data Documentation

MatrixReader.MatrixReader.addCommand

Definition at line 24 of file MatrixReader.py.

Referenced by MatrixReader.MatrixReader.readMatrix().

MatrixReader.MatrixReader.apply

Definition at line 25 of file MatrixReader.py.

Referenced by heavyIonTools.ConfigureHeavyIons.__call__(), editorTools.UserCodeTool.__call__(), HiCoreTools.RestrictInputToAOD.__call__(), coreTools.RunOnData.__call__(), trackTools.MakeAODTrackCandidates.__call__(), runJetUncertainties.RunJetUncertainties.__call__(), metTools.AddMETCollection.__call__(), heavyIonTools.ProductionDefaults.__call__(), editorTools.ChangeSource.__call__(), HiCoreTools.RemoveMCMatching.__call__(), cmsswVersionTools.PickRelValInputFiles.__call__(), coreTools.RemoveMCMatching.__call__(), trackTools.MakePATTrackCandidates.__call__(), trigTools.SwitchOnTrigger.__call__(), runMETCorrectionsAndUncertainties.RunMETCorrectionsAndUncertainties.__call__(), heavyIonTools.SelectionDefaults.__call__(), HiCoreTools.RemoveAllPATObjectsBut.__call__(), heavyIonTools.DisbaleMonteCarloDeps.__call__(), HiCoreTools.RemoveSpecificPATObjects.__call__(), trigTools.SwitchOnTriggerStandAlone.__call__(), trackTools.MakeTrackCandidates.__call__(), tauTools.AddTauCollection.__call__(), trigTools.SwitchOnTriggerMatching.__call__(), HiCoreTools.RemoveCleaning.__call__(), HiCoreTools.AddCleaning.__call__(), trigTools.SwitchOnTriggerMatchingStandAlone.__call__(), jetTools.AddJetCollection.__call__(), trigTools.SwitchOnTriggerMatchEmbedding.__call__(), jetTools.SwitchJetCollection.__call__(), jetTools.UpdateJetCollection.__call__(), jetTools.AddJetID.__call__(), jetTools.SetTagInfos.__call__(), and MatrixReader.MatrixReader.readMatrix().

MatrixReader.MatrixReader.commandLineWf

Definition at line 26 of file MatrixReader.py.

MatrixReader.MatrixReader.files

Definition at line 58 of file MatrixReader.py.

Referenced by chain.Chain._guessTreeName(), dataset.BaseDataset.listOfFiles(), dataset.BaseDataset.listOfGoodFiles(), MatrixReader.MatrixReader.prepare(), dataset.BaseDataset.printFiles(), and MatrixReader.MatrixReader.showRaw().

MatrixReader.MatrixReader.filesDefault

Definition at line 73 of file MatrixReader.py.

Referenced by MatrixReader.MatrixReader.prepare(), and MatrixReader.MatrixReader.showRaw().

MatrixReader.MatrixReader.filesPrefMap

Definition at line 42 of file MatrixReader.py.

Referenced by MatrixReader.MatrixReader.createWorkFlows(), and MatrixReader.MatrixReader.readMatrix().

MatrixReader.MatrixReader.nameList

Definition at line 40 of file MatrixReader.py.

Referenced by MatrixReader.MatrixReader.createWorkFlows().

MatrixReader.MatrixReader.noRun

Definition at line 29 of file MatrixReader.py.

Referenced by MatrixReader.MatrixReader.readMatrix().

MatrixReader.MatrixReader.overWrite

Definition at line 27 of file MatrixReader.py.

Referenced by MatrixReader.MatrixReader.readMatrix().

MatrixReader.MatrixReader.relvalModule

Definition at line 89 of file MatrixReader.py.

Referenced by MatrixReader.MatrixReader.readMatrix().

MatrixReader.MatrixReader.revertDqmio

maybe we want too level deep input

Definition at line 23 of file MatrixReader.py.

Referenced by MatrixReader.MatrixReader.readMatrix(), and MatrixReader.MatrixReader.showRaw().

MatrixReader.MatrixReader.what

Definition at line 34 of file MatrixReader.py.

Referenced by MatrixReader.MatrixReader.prepare(), and MatrixReader.MatrixReader.showRaw().

MatrixReader.MatrixReader.wm

Definition at line 22 of file MatrixReader.py.

Referenced by MatrixReader.MatrixReader.readMatrix().

MatrixReader.MatrixReader.workFlows

Definition at line 39 of file MatrixReader.py.

Referenced by MatrixRunner.MatrixRunner.runTests(), MatrixReader.MatrixReader.updateDB(), and MatrixReader.MatrixReader.workFlowsByLocation().

MatrixReader.MatrixReader.workFlowSteps

Definition at line 37 of file MatrixReader.py.

Referenced by MatrixReader.MatrixReader.createWorkFlows(), MatrixReader.MatrixReader.readMatrix(), and MatrixReader.MatrixReader.showRaw().

Generated for CMSSW Reference Manual by doxygen 1.8.5