test
CMS 3D CMS Logo
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Properties Friends Macros Pages
Functions | Variables
cmsPerfHarvest Namespace Reference

Functions

def harvest
 Harvest all data from a perfsuite directory (if cpu subdirectories exist, it will examine them) More...
 
def optionParse
 Parse options. More...
 
def visit
 Retrieve data from a perf suite output (sub) directory, only examines TimeSize at the moment. More...
 
def visit_timesize_steps
 Harvest information from a timesize directory, currently only retrieves TimingReport information. More...
 

Variables

tuple _PROG_NAME = os.path.basename(sys.argv[0])
 

Function Documentation

def cmsPerfHarvest.harvest (   perfdir)

Harvest all data from a perfsuite directory (if cpu subdirectories exist, it will examine them)

Definition at line 99 of file cmsPerfHarvest.py.

References optionParse(), and visit().

99 
100 def harvest(perfdir):
101  cpureg = re.compile("cpu_([0-9][0-9]*)")
102  out = {}
103  globpath = os.path.join(perfdir, "cpu_*")
104  globs = glob.glob(globpath)
105  if len(globs) > 0:
106  for globule in globs:
107  base = os.path.basename(globule)
108  found = cpureg.search(base)
109  if found:
110  cpuid = found.groups()[0]
111  if cpuid in out:
112  print "Error: we already have a cpu run with this id %s ! Skipping..." % cpuid
113  else:
114  if cpuid == None:
115  print "Error: could not resolve cpuid something is wrong"
116  cpuid = "None"
117  out[cpuid] = visit(globule)
118  else:
119  print "Error: could not determine valid cpu id from %s ! Skipping..." % base
120 
121  else:
122  out["None"] = visit(perfdir)
123 
124  return out
cmsPerfHarvest.visit
def visit
Retrieve data from a perf suite output (sub) directory, only examines TimeSize at the moment...
Definition: cmsPerfHarvest.py:67
cmsPerfHarvest.harvest
def harvest
Harvest all data from a perfsuite directory (if cpu subdirectories exist, it will examine them) ...
Definition: cmsPerfHarvest.py:99
def cmsPerfHarvest.optionParse ( )

Parse options.

Definition at line 12 of file cmsPerfHarvest.py.

Referenced by harvest().

12 
13 def optionParse():
14  parser = opt.OptionParser(usage="""./%s [perf dir] [options]""" % _PROG_NAME)
15 
16  (options, args) = parser.parse_args()
17 
18  if not len(args) == 1:
19  parser.error("You have not supplied a perfsuite directory.")
20  sys.exit()
21 
22  args[0] = os.path.abspath(args[0])
23 
24  ###########
25  # Check existance of the directory supplied
26  #
27  if not os.path.isdir(args[0]):
28  parser.error("You have not provided a valid perfsuite output directory")
29  sys.exit()
30 
31  perfdir = args[0]
32 
33  return (perfdir)
cmsPerfHarvest.optionParse
def optionParse
Parse options.
Definition: cmsPerfHarvest.py:12
def cmsPerfHarvest.visit (   visitdir)

Retrieve data from a perf suite output (sub) directory, only examines TimeSize at the moment.

Definition at line 67 of file cmsPerfHarvest.py.

References visit_timesize_steps().

Referenced by Config.Process._insertPaths(), harvest(), ConfigBuilder.ConfigBuilder.PrintAllModules.leave(), ConfigBuilder.ConfigBuilder.renameHLTprocessInSequence(), and rerunParticleFlow.updateInputTags().

67 
68 def visit(visitdir):
69  out = {}
70  for candle in Candles:
71  globpath = os.path.join(visitdir,"%s_*" % (candle))
72  globs = glob.glob(globpath)
73  if len(globs) > 0:
74  profsetreg = re.compile("%s_(.*)" % candle)
75  for globule in globs:
76  base = os.path.basename(globule)
77  found = profsetreg.search(base)
78  if found:
79  profset = found.groups()[0]
80  if profset == "TimeSize": # Just do timesize for now!
81  if candle == None:
82  print "Error: could not resolve candle something is wrong"
83  candle = "None"
84  if profset == None:
85  print "Error: could not resolve profset something is wrong"
86  profset = "None"
87  if candle not in out:
88  out[candle] = {}
89  candledict = out[candle]
90  if profset in candledict:
91  print "Error: we already have a profset that matches %s" % str(profset)
92  else:
93  candledict[profset] = visit_timesize_steps(candle,globule)
94  out[candle] = candledict
95  return out
cmsPerfHarvest.visit_timesize_steps
def visit_timesize_steps
Harvest information from a timesize directory, currently only retrieves TimingReport information...
Definition: cmsPerfHarvest.py:37
cmsPerfHarvest.visit
def visit
Retrieve data from a perf suite output (sub) directory, only examines TimeSize at the moment...
Definition: cmsPerfHarvest.py:67
def cmsPerfHarvest.visit_timesize_steps (   candle,
  profsetdir 
)

Harvest information from a timesize directory, currently only retrieves TimingReport information.

Definition at line 37 of file cmsPerfHarvest.py.

Referenced by visit().

37 
38 def visit_timesize_steps(candle,profsetdir):
39  out = {}
40  # Just do timing report for now
41  globpath = os.path.join(profsetdir,"%s_*_TimingReport.log" % CandFname[candle])
42  globs = glob.glob(globpath)
43  if len(globs) > 0:
44  stepreg = re.compile("%s_([^_]*)_TimingReport.log" % CandFname[candle])
45  for globule in globs:
46  base = os.path.basename(globule)
47  found = stepreg.search(base)
48  if found:
49  step = found.groups()[0]
50  try:
51  if step == None:
52  print "Error: could not resolve step something is wrong"
53  step = "None"
54  if "TimingReport" not in out:
55  out["TimingReport"] = {}
56  stepdict = out["TimingReport"]
57  stepdict[step] = cpr.getTimingLogData(globule)
58  out["TimingReport"] = stepdict
59  except (OSError, IOError) as detail:
60  print detail
61  else:
62  print "Error: Could not determine step from %s" % base
63  return out
cmsPerfHarvest.visit_timesize_steps
def visit_timesize_steps
Harvest information from a timesize directory, currently only retrieves TimingReport information...
Definition: cmsPerfHarvest.py:37

Variable Documentation

tuple cmsPerfHarvest._PROG_NAME = os.path.basename(sys.argv[0])

Definition at line 7 of file cmsPerfHarvest.py.

Generated for CMSSW Reference Manual by doxygen 1.8.5