CMS 3D CMS Logo
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Properties Friends Macros Pages
List of all members | Public Member Functions | Private Member Functions | Private Attributes
validation.Validation Class Reference
Inheritance diagram for validation.Validation:
trackingPlots.TrackingValidation vertexPlots.VertexValidation

Public Member Functions

def __init__
 
def doPlots
 
def download
 

Private Member Functions

def _doPlots
 
def _doPlotsFastFull
 
def _doPlotsPileup
 
def _getDirectoryName
 
def _getSelectionName
 

Private Attributes

 _fastsimSamples
 
 _fullsimSamples
 
 _newFileModifier
 
 _newRelease
 
 _newRepository
 
 _plotter
 
 _plotterDrawArgs
 
 _refRelease
 
 _refRepository
 
 _selectionName
 

Detailed Description

Base class for Tracking/Vertex validation.

Definition at line 279 of file validation.py.

Constructor & Destructor Documentation

def validation.Validation.__init__ (   self,
  fullsimSamples,
  fastsimSamples,
  newRelease,
  newFileModifier = None,
  selectionName = "" 
) 
Constructor.

Arguments:
fullsimSamples -- List of Sample objects for FullSim samples (may be empty)
fastsimSamples -- List of Sample objects for FastSim samples (may be empty)
newRelease     -- CMSSW release to be validated
newFileModifier -- If given, a function to modify the names of the new files (function takes a string and returns a string)
selectionName  -- If given, use this string as the selection name (appended to GlobalTag for directory names)

Definition at line 281 of file validation.py.

Referenced by vertexPlots.VertexValidation._init__(), and trackingPlots.TrackingValidation._init__().

281 
282  def __init__(self, fullsimSamples, fastsimSamples, newRelease, newFileModifier=None, selectionName=""):
283  """Constructor.
284 
285  Arguments:
286  fullsimSamples -- List of Sample objects for FullSim samples (may be empty)
287  fastsimSamples -- List of Sample objects for FastSim samples (may be empty)
288  newRelease -- CMSSW release to be validated
289  newFileModifier -- If given, a function to modify the names of the new files (function takes a string and returns a string)
290  selectionName -- If given, use this string as the selection name (appended to GlobalTag for directory names)
291  """
292  try:
293  self._newRelease = os.environ["CMSSW_VERSION"]
294  except KeyError:
295  print >>sys.stderr, 'Error: CMSSW environment variables are not available.'
296  print >>sys.stderr, ' Please run cmsenv'
297  sys.exit()
298 
299  self._fullsimSamples = fullsimSamples
300  self._fastsimSamples = fastsimSamples
301  if newRelease != "":
302  self._newRelease = newRelease
303  self._newFileModifier = newFileModifier
304  self._selectionName = selectionName

Member Function Documentation

def validation.Validation._doPlots (   self,
  algo,
  quality,
  sample 
)
private 
Do the real plotting work for a given algorithm, quality flag, and sample.

Definition at line 417 of file validation.py.

References validation._copySubDir(), validation.Validation._getDirectoryName(), trackingPlots.TrackingValidation._getDirectoryName(), validation._getGlobalTag(), validation.Validation._getSelectionName(), trackingPlots.TrackingValidation._getSelectionName(), validation.Validation._newRelease, validation.Validation._newRepository, validation.Validation._plotterDrawArgs, validation.Validation._refRelease, validation.Validation._refRepository, and validation._stripRelease().

417 
418  def _doPlots(self, algo, quality, sample):
419  """Do the real plotting work for a given algorithm, quality flag, and sample."""
420  # Get GlobalTags
421  refGlobalTag = _getGlobalTag(sample, self._refRelease)
422  newGlobalTag = _getGlobalTag(sample, self._newRelease)
423 
424  # Construct selection string
425  tmp = ""
426  if sample.hasScenario():
427  tmp += "_"+sample.scenario()
428  tmp += "_"+sample.pileup()
429  tmp += self._getSelectionName(quality, algo)
430  refSelection = refGlobalTag+tmp
431  newSelection = newGlobalTag+tmp
432  if sample.hasPileup() and not sample.fastsim():
433  refSelection += "_"+sample.pileupType(self._refRelease)
434  newSelection += "_"+sample.pileupType(self._newRelease)
435 
436  # Check that the new DQM file exists
437  harvestedfile = sample.filename(self._newRelease)
438  if not os.path.exists(harvestedfile):
439  print "Harvested file %s does not exist!" % harvestedfile
440  sys.exit(1)
441 
442  valname = "val.{sample}.root".format(sample=sample.name())
443 
444  # Construct reference directory name
445  tmp = [self._refRepository, self._refRelease]
446  if sample.fastsim():
447  tmp.extend(["fastsim", self._refRelease])
448  tmp.extend([refSelection, sample.name()])
449  refdir = os.path.join(*tmp)
450 
451  # Construct new directory name
452  tmp = [self._newRepository, self._newRelease]
453  if sample.fastsim():
454  tmp.extend(["fastsim", self._newRelease])
455  tmp.extend([newSelection, sample.name()])
456  newdir = os.path.join(*tmp)
457 
458  # Open reference file if it exists
459  refValFilePath = os.path.join(refdir, valname)
460  if not os.path.exists(refValFilePath):
461  print "Reference file %s not found" % refValFilePath
462  if not plotting.missingOk:
463  print "If this is acceptable, please set 'plotting.missingOk=True'"
464  sys.exit(1)
465  else:
466  refValFile = None
467  else:
468  refValFile = ROOT.TFile.Open(refValFilePath)
469 
470  # Copy the relevant histograms to a new validation root file
471  newValFile = _copySubDir(harvestedfile, valname, self._plotter.getPossibleDirectoryNames(), self._getDirectoryName(quality, algo))
472  fileList = [valname]
473 
474  # Do the plots
475  print "Comparing ref and new {sim} {sample} {algo} {quality}".format(
476  sim="FullSim" if not sample.fastsim() else "FastSim",
477  sample=sample.name(), algo=algo, quality=quality)
478  self._plotter.create([refValFile, newValFile], [
479  "%s, %s %s" % (sample.name(), _stripRelease(self._refRelease), refSelection),
480  "%s, %s %s" % (sample.name(), _stripRelease(self._newRelease), newSelection)
481  ],
482  subdir = self._getDirectoryName(quality, algo))
483  fileList.extend(self._plotter.draw(algo, **self._plotterDrawArgs))
484 
485  newValFile.Close()
486  if refValFile is not None:
487  refValFile.Close()
488 
489  # Move plots to new directory
490  print "Moving plots and %s to %s" % (valname, newdir)
491  if not os.path.exists(newdir):
492  os.makedirs(newdir)
493  for f in fileList:
494  shutil.move(f, os.path.join(newdir, f))
validation._copySubDir
def _copySubDir
Definition: validation.py:593
validation._getGlobalTag
def _getGlobalTag
Definition: validation.py:70
validation._stripRelease
def _stripRelease
Definition: validation.py:63
def validation.Validation._doPlotsFastFull (   self,
  algo,
  quality,
  fastSample,
  fullSample 
)
private 
Do the real plotting work for FastSim vs. FullSim for a given algorithm, quality flag, and sample.

Definition at line 495 of file validation.py.

References validation.Validation._getDirectoryName(), trackingPlots.TrackingValidation._getDirectoryName(), validation._getGlobalTag(), validation.Validation._getSelectionName(), trackingPlots.TrackingValidation._getSelectionName(), validation.Validation._newRelease, validation.Validation._newRepository, validation.Validation._plotterDrawArgs, and validation._stripRelease().

495 
496  def _doPlotsFastFull(self, algo, quality, fastSample, fullSample):
497  """Do the real plotting work for FastSim vs. FullSim for a given algorithm, quality flag, and sample."""
498  # Get GlobalTags
499  fastGlobalTag = _getGlobalTag(fastSample, self._newRelease)
500  fullGlobalTag = _getGlobalTag(fullSample, self._newRelease)
501 
502  # Construct selection string
503  tmp = self._getSelectionName(quality, algo)
504  fastSelection = fastGlobalTag+"_"+fastSample.pileup()+tmp
505  fullSelection = fullGlobalTag+"_"+fullSample.pileup()+tmp
506  if fullSample.hasPileup():
507  fullSelection += "_"+fullSample.pileupType(self._newRelease)
508 
509  # Construct directories for FastSim, FullSim, and for the results
510  fastdir = os.path.join(self._newRepository, self._newRelease, "fastsim", self._newRelease, fastSelection, fastSample.name())
511  fulldir = os.path.join(self._newRepository, self._newRelease, fullSelection, fullSample.name())
512  newdir = os.path.join(self._newRepository, self._newRelease, "fastfull", self._newRelease, fastSelection, fastSample.name())
513 
514  # Open input root files
515  valname = "val.{sample}.root".format(sample=fastSample.name())
516  fastValFilePath = os.path.join(fastdir, valname)
517  if not os.path.exists(fastValFilePath):
518  print "FastSim file %s not found" % fastValFilePath
519  fullValFilePath = os.path.join(fulldir, valname)
520  if not os.path.exists(fullValFilePath):
521  print "FullSim file %s not found" % fullValFilePath
522 
523  fastValFile = ROOT.TFile.Open(fastValFilePath)
524  fullValFile = ROOT.TFile.Open(fullValFilePath)
525 
526  # Do plots
527  print "Comparing FullSim and FastSim {sample} {algo} {quality}".format(
528  sample=fastSample.name(), algo=algo, quality=quality)
529  self._plotter.create([fullValFile, fastValFile], [
530  "FullSim %s, %s %s" % (fullSample.name(), _stripRelease(self._newRelease), fullSelection),
531  "FastSim %s, %s %s" % (fastSample.name(), _stripRelease(self._newRelease), fastSelection),
532  ],
533  subdir = self._getDirectoryName(quality, algo))
534  fileList = self._plotter.draw(algo, **self._plotterDrawArgs)
535 
536  fullValFile.Close()
537  fastValFile.Close()
538 
539  # Move plots to new directory
540  print "Moving plots to %s" % (newdir)
541  if not os.path.exists(newdir):
542  os.makedirs(newdir)
543  for f in fileList:
544  shutil.move(f, os.path.join(newdir, f))
validation._getGlobalTag
def _getGlobalTag
Definition: validation.py:70
validation._stripRelease
def _stripRelease
Definition: validation.py:63
def validation.Validation._doPlotsPileup (   self,
  algo,
  quality,
  sample 
)
private 
Do the real plotting work for Old vs. New pileup scenarios for a given algorithm, quality flag, and sample.

Definition at line 545 of file validation.py.

References validation.Validation._getDirectoryName(), trackingPlots.TrackingValidation._getDirectoryName(), validation._getGlobalTag(), validation.Validation._getSelectionName(), trackingPlots.TrackingValidation._getSelectionName(), validation.Validation._newRelease, validation.Validation._newRepository, validation.Validation._plotterDrawArgs, and validation._stripRelease().

545 
546  def _doPlotsPileup(self, algo, quality, sample):
547  """Do the real plotting work for Old vs. New pileup scenarios for a given algorithm, quality flag, and sample."""
548  # Get GlobalTags
549  newGlobalTag = _getGlobalTag(sample, self._newRelease)
550  refGlobalTag = newGlobalTag + "_OldPU"
551 
552  # Construct selection string
553  tmp = self._getSelectionName(quality, algo)
554  refSelection = refGlobalTag+"_"+sample.pileup()+tmp+"_"+sample.pileupType(self._newRelease)
555  newSelection = newGlobalTag+"_"+sample.pileup()+tmp+"_"+sample.pileupType(self._newRelease)
556 
557  # Construct directories for FastSim, FullSim, and for the results
558  refdir = os.path.join(self._newRepository, self._newRelease, refSelection, sample.name())
559  newdir = os.path.join(self._newRepository, self._newRelease, newSelection, sample.name())
560  resdir = os.path.join(self._newRepository, self._newRelease, "pileup", self._newRelease, newSelection, sample.name())
561 
562  # Open input root files
563  valname = "val.{sample}.root".format(sample=sample.name())
564  refValFilePath = os.path.join(refdir, valname)
565  if not os.path.exists(refValFilePath):
566  print "Ref pileup file %s not found" % refValFilePath
567  newValFilePath = os.path.join(newdir, valname)
568  if not os.path.exists(newValFilePath):
569  print "New pileup file %s not found" % newValFilePath
570 
571  refValFile = ROOT.TFile.Open(refValFilePath)
572  newValFile = ROOT.TFile.Open(newValFilePath)
573 
574  # Do plots
575  print "Comparing Old and New pileup {sample} {algo} {quality}".format(
576  sample=sample.name(), algo=algo, quality=quality)
577  self._plotter.create([refValFile, newValFile], [
578  "%d BX %s, %s %s" % ({"25ns": 10, "50ns": 20}[sample.pileupType(self._newRelease)], sample.name(), _stripRelease(self._newRelease), refSelection),
579  "35 BX %s, %s %s" % (sample.name(), _stripRelease(self._newRelease), newSelection),
580  ],
581  subdir = self._getDirectoryName(quality, algo))
582  fileList = self._plotter.draw(algo, **self._plotterDrawArgs)
583 
584  newValFile.Close()
585  refValFile.Close()
586 
587  # Move plots to new directory
588  print "Moving plots to %s" % (resdir)
589  if not os.path.exists(resdir):
590  os.makedirs(resdir)
591  for f in fileList:
592  shutil.move(f, os.path.join(resdir, f))
validation._getGlobalTag
def _getGlobalTag
Definition: validation.py:70
validation._stripRelease
def _stripRelease
Definition: validation.py:63
def validation.Validation._getDirectoryName (   self,
  args,
  kwargs 
)
private

Definition at line 305 of file validation.py.

Referenced by validation.Validation._doPlots(), validation.Validation._doPlotsFastFull(), and validation.Validation._doPlotsPileup().

305 
306  def _getDirectoryName(self, *args, **kwargs):
307  return None
def validation.Validation._getSelectionName (   self,
  args,
  kwargs 
)
private

Definition at line 308 of file validation.py.

References validation.Validation._selectionName.

Referenced by validation.Validation._doPlots(), validation.Validation._doPlotsFastFull(), and validation.Validation._doPlotsPileup().

308 
309  def _getSelectionName(self, *args, **kwargs):
310  return self._selectionName
def validation.Validation.doPlots (   self,
  algos,
  qualities,
  refRelease,
  refRepository,
  newRepository,
  plotter,
  plotterDrawArgs = {} 
) 
Create validation plots.

Arguments:
algos         -- List of strings for algoritms
qualities     -- List of strings for quality flags (can be None)
refRelease    -- String for reference CMSSW release
refRepository -- String for directory where reference root files are
newRepository -- String for directory whete to put new files
plotter       -- plotting.Plotter object that does the plotting

Keyword arguments:
plotterDrawArgs -- Dictionary for additional arguments to Plotter.draw() (default: {})

Definition at line 359 of file validation.py.

359 
360  def doPlots(self, algos, qualities, refRelease, refRepository, newRepository, plotter, plotterDrawArgs={}):
361  """Create validation plots.
362 
363  Arguments:
364  algos -- List of strings for algoritms
365  qualities -- List of strings for quality flags (can be None)
366  refRelease -- String for reference CMSSW release
367  refRepository -- String for directory where reference root files are
368  newRepository -- String for directory whete to put new files
369  plotter -- plotting.Plotter object that does the plotting
370 
371  Keyword arguments:
372  plotterDrawArgs -- Dictionary for additional arguments to Plotter.draw() (default: {})
373  """
374  self._refRelease = refRelease
375  self._refRepository = refRepository
376  self._newRepository = newRepository
377  self._plotter = plotter
378  self._plotterDrawArgs = plotterDrawArgs
379 
380  if qualities is None:
381  qualities = [None]
382  if algos is None:
383  algos = [None]
384 
385  # New vs. Ref
386  for s in self._fullsimSamples+self._fastsimSamples:
387  for q in qualities:
388  for a in algos:
389  self._doPlots(a, q, s)
390 # if s.fullsim() and s.hasPileup():
391 # self._doPlotsPileup(a, q, s)
392 
393  # Fast vs. Full in New
394  for fast in self._fastsimSamples:
395  correspondingFull = None
396  for full in self._fullsimSamples:
397  if fast.name() != full.name():
398  continue
399  if fast.hasPileup():
400  if not full.hasPileup():
401  continue
402  if fast.fastsimCorrespondingFullsimPileup() != full.pileupType():
403  continue
404  else:
405  if full.hasPileup():
406  continue
407 
408  if correspondingFull is None:
409  correspondingFull = full
410  else:
411  raise Exception("Got multiple compatible FullSim samples for FastSim sample %s %s" % (fast.name(), fast.pileup()))
412  if correspondingFull is None:
413  raise Exception("Did not find compatible FullSim sample for FastSim sample %s %s" % (fast.name(), fast.pileup()))
414  for q in qualities:
415  for a in algos:
416  self._doPlotsFastFull(a, q, fast, correspondingFull)
def validation.Validation.download (   self) 
Download DQM files. Requires grid certificate and asks your password for it.

Definition at line 311 of file validation.py.

References validation.Validation._fastsimSamples, validation.Validation._fullsimSamples, validation._getRelValUrl(), validation.Validation._newFileModifier, validation.Validation._newRelease, alcazmumu_cfi.filter, join(), and python.multivaluedict.map().

311 
312  def download(self):
313  """Download DQM files. Requires grid certificate and asks your password for it."""
314  filenames = [s.filename(self._newRelease) for s in self._fullsimSamples+self._fastsimSamples]
315  if self._newFileModifier is not None:
316  filenames = map(self._newFileModifier, filenames)
317  filenames = filter(lambda f: not os.path.exists(f), filenames)
318  if len(filenames) == 0:
319  print "All files already downloaded"
320  return
321 
322  relvalUrl = _getRelValUrl(self._newRelease)
323  urls = [relvalUrl+f for f in filenames]
324  certfile = os.path.join(os.environ["HOME"], ".globus", "usercert.pem")
325  if not os.path.exists(certfile):
326  print "Certificate file {certfile} does not exist, unable to download RelVal files from {url}".format(certfile=certfile, url=relvalUrl)
327  sys.exit(1)
328  keyfile = os.path.join(os.environ["HOME"], ".globus", "userkey.pem")
329  if not os.path.exists(certfile):
330  print "Private key file {keyfile} does not exist, unable to download RelVal files from {url}".format(keyfile=keyfile, url=relvalUrl)
331  sys.exit(1)
332 
333  cmd = ["curl", "--cert-type", "PEM", "--cert", certfile, "--key", keyfile, "-k"]
334  for u in urls:
335  cmd.extend(["-O", u])
336  print "Downloading %d files from RelVal URL %s:" % (len(filenames), relvalUrl)
337  print " "+"\n ".join(filenames)
338  print "Please provide your private key pass phrase when curl asks it"
339  ret = subprocess.call(cmd)
340  if ret != 0:
341  print "Downloading failed with exit code %d" % ret
342  sys.exit(1)
343 
344  # verify
345  allFine = True
346  for f in filenames:
347  p = subprocess.Popen(["file", f], stdout=subprocess.PIPE)
348  stdout = p.communicate()[0]
349  if p.returncode != 0:
350  print "file command failed with exit code %d" % p.returncode
351  sys.exit(1)
352  if not "ROOT" in stdout:
353  print "File {f} is not ROOT, please check the correct version, GobalTag etc. from {url}".format(f=f, url=relvalUrl)
354  allFine = False
355  if os.path.exists(f):
356  os.remove(f)
357  if not allFine:
358  sys.exit(1)
alcazmumu_cfi.filter
tuple filter
Definition: alcazmumu_cfi.py:34
join
static std::string join(char **cmd)
Definition: RemoteFile.cc:18
validation._getRelValUrl
def _getRelValUrl
Definition: validation.py:106

Member Data Documentation

validation.Validation._fastsimSamples
private

Definition at line 299 of file validation.py.

Referenced by validation.Validation.download().

validation.Validation._fullsimSamples
private

Definition at line 298 of file validation.py.

Referenced by validation.Validation.download().

validation.Validation._newFileModifier
private

Definition at line 302 of file validation.py.

Referenced by validation.Validation.download().

validation.Validation._newRelease
private

Definition at line 292 of file validation.py.

Referenced by validation.Validation._doPlots(), validation.Validation._doPlotsFastFull(), validation.Validation._doPlotsPileup(), and validation.Validation.download().

validation.Validation._newRepository
private

Definition at line 375 of file validation.py.

Referenced by validation.Validation._doPlots(), validation.Validation._doPlotsFastFull(), and validation.Validation._doPlotsPileup().

validation.Validation._plotter
private

Definition at line 376 of file validation.py.

validation.Validation._plotterDrawArgs
private

Definition at line 377 of file validation.py.

Referenced by validation.Validation._doPlots(), validation.SimpleValidation._doPlots(), validation.Validation._doPlotsFastFull(), and validation.Validation._doPlotsPileup().

validation.Validation._refRelease
private

Definition at line 373 of file validation.py.

Referenced by validation.Validation._doPlots().

validation.Validation._refRepository
private

Definition at line 374 of file validation.py.

Referenced by validation.Validation._doPlots().

validation.Validation._selectionName
private

Definition at line 303 of file validation.py.

Referenced by validation.Validation._getSelectionName().

Generated for CMSSW Reference Manual by doxygen 1.8.5