CMS 3D CMS Logo
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Properties Friends Macros Pages
Namespaces | Classes | Functions | Variables
heppy Namespace Reference

Namespaces

 mt2w_bisect
 

Classes

struct  AlphaT
 
class  BTagSF
 
class  CMGMuonCleanerBySegmentsAlgo
 
class  Davismt2
 
class  EGammaMvaEleEstimatorFWLite
 
class  FSRWeightAlgo
 
class  Hemisphere
 
class  MuScleFitCorrector
 
class  ReclusterJets
 
class  RochCor
 
class  RochCor2012
 
class  TriggerBitChecker
 

Functions

def callBack
 
def chunks
 
def createOutputDir
 
def main
 
def runLoop
 
def runLoopAsync
 
def split
 

Variables

string action = 'store_true'
 
 default = None)
 
string dest = "nevents"
 
string help = "number of events to process"
 
 loop = None
 
list oldv = sys.argv[:]
 
tuple parser = OptionParser()
 

Detailed Description

MuScleFitCorrector class Author M. De Mattia - 18/11/2008 Author S. Casasso - 25/10/2012 Author E. Migliore - 25/10/2012 This is used to have a common set of functions for the specialized templates to use. The constructor receives the name identifying the parameters for the correction function. It reads the parameters from a txt file in data/.

Function Documentation

def heppy.callBack (   result)

Definition at line 28 of file heppy.py.

28 
29 def callBack( result ):
30  pass
31  print 'production done:', str(result)
heppy.callBack
def callBack
Definition: heppy.py:28
def heppy.chunks (   l,
  n 
)

Definition at line 92 of file heppy.py.

Referenced by split().

92 
93 def chunks(l, n):
94  return [l[i:i+n] for i in range(0, len(l), n)]
heppy.chunks
def chunks
Definition: heppy.py:92
def heppy.createOutputDir (   dir,
  components,
  force 
) 
Creates the output dir, dealing with the case where dir exists.

Definition at line 65 of file heppy.py.

References join().

Referenced by main().

65 
66 def createOutputDir(dir, components, force):
67  '''Creates the output dir, dealing with the case where dir exists.'''
68  answer = None
69  try:
70  os.mkdir(dir)
71  return True
72  except OSError:
73  print 'directory %s already exists' % dir
74  print 'contents: '
75  dirlist = [path for path in os.listdir(dir) if os.path.isdir( '/'.join([dir, path]) )]
76  pprint( dirlist )
77  print 'component list: '
78  print [comp.name for comp in components]
79  if force is True:
80  print 'force mode, continue.'
81  return True
82  else:
83  while answer not in ['Y','y','yes','N','n','no']:
84  answer = raw_input('Continue? [y/n]')
85  if answer.lower().startswith('n'):
86  return False
87  elif answer.lower().startswith('y'):
88  return True
89  else:
90  raise ValueError( ' '.join(['answer can not have this value!',
91  answer]) )
join
static std::string join(char **cmd)
Definition: RemoteFile.cc:18
heppy.createOutputDir
def createOutputDir
Definition: heppy.py:65
def heppy.main (   options,
  args 
)

Definition at line 125 of file heppy.py.

References createOutputDir(), min(), runLoop(), and split().

125 
126 def main( options, args ):
127 
128  if len(args) != 2:
129  parser.print_help()
130  print 'ERROR: please provide the processing name and the component list'
131  sys.exit(1)
132 
133  outDir = args[0]
134  if os.path.exists(outDir) and not os.path.isdir( outDir ):
135  parser.print_help()
136  print 'ERROR: when it exists, first argument must be a directory.'
137  sys.exit(2)
138  cfgFileName = args[1]
139  if not os.path.isfile( cfgFileName ):
140  parser.print_help()
141  print 'ERROR: second argument must be an existing file (your input cfg).'
142  sys.exit(3)
143 
144  file = open( cfgFileName, 'r' )
145  cfg = imp.load_source( 'PhysicsTools.HeppyCore.__cfg_to_run__', cfgFileName, file)
146 
147  selComps = [comp for comp in cfg.config.components if len(comp.files)>0]
148  selComps = split(selComps)
149  for comp in selComps:
150  print comp
151  if len(selComps)>10:
152  print "WARNING: too many threads {tnum}, will just use a maximum of 10.".format(tnum=len(selComps))
153  if not createOutputDir(outDir, selComps, options.force):
154  print 'exiting'
155  sys.exit(0)
156  if len(selComps)>1:
157  shutil.copy( cfgFileName, outDir )
158  pool = Pool(processes=min(len(selComps),10))
159  ## workaround for a scoping problem in ipython+multiprocessing
160  import PhysicsTools.HeppyCore.framework.heppy as ML
161  for comp in selComps:
162  print 'submitting', comp.name
163  pool.apply_async( ML.runLoopAsync, [comp, outDir, 'PhysicsTools.HeppyCore.__cfg_to_run__', options],
164  callback=ML.callBack)
165  pool.close()
166  pool.join()
167  else:
168  # when running only one loop, do not use multiprocessor module.
169  # then, the exceptions are visible -> use only one sample for testing
170  global loop
171  loop = runLoop( comp, outDir, cfg.config, options )
172 
173 
heppy.main
def main
Definition: heppy.py:125
heppy.runLoop
def runLoop
Definition: heppy.py:44
heppy.split
def split
Definition: heppy.py:95
min
T min(T a, T b)
Definition: MathUtil.h:58
heppy.createOutputDir
def createOutputDir
Definition: heppy.py:65
def heppy.runLoop (   comp,
  outDir,
  config,
  options 
)

Definition at line 44 of file heppy.py.

References join().

Referenced by main(), and runLoopAsync().

44 
45 def runLoop( comp, outDir, config, options):
46  fullName = '/'.join( [outDir, comp.name ] )
47  # import pdb; pdb.set_trace()
48  config.components = [comp]
49  loop = Looper( fullName,
50  config,
51  options.nevents, 0,
52  nPrint = options.nprint,
53  timeReport = options.timeReport)
54  print loop
55  if options.iEvent is None:
56  loop.loop()
57  loop.write()
58  print loop
59  else:
60  # loop.InitOutput()
61  iEvent = int(options.iEvent)
62  loop.process( iEvent )
63  return loop
64 
heppy.runLoop
def runLoop
Definition: heppy.py:44
join
static std::string join(char **cmd)
Definition: RemoteFile.cc:18
def heppy.runLoopAsync (   comp,
  outDir,
  configName,
  options 
)

Definition at line 32 of file heppy.py.

References runLoop().

32 
33 def runLoopAsync(comp, outDir, configName, options):
34  try:
35  loop = runLoop( comp, outDir, copy.copy(sys.modules[configName].config), options)
36  return loop.name
37  except Exception:
38  import traceback
39  print "ERROR processing component %s" % comp.name
40  print comp
41  print "STACK TRACE: "
42  print traceback.format_exc()
43  raise
heppy.runLoop
def runLoop
Definition: heppy.py:44
heppy.runLoopAsync
def runLoopAsync
Definition: heppy.py:32
def heppy.split (   comps)

Definition at line 95 of file heppy.py.

References chunks().

Referenced by main().

95 
96 def split(comps):
97  # import pdb; pdb.set_trace()
98  splitComps = []
99  for comp in comps:
100  if hasattr( comp, 'fineSplitFactor') and comp.fineSplitFactor>1:
101  subchunks = range(comp.fineSplitFactor)
102  for ichunk, chunk in enumerate([(f,i) for f in comp.files for i in subchunks]):
103  newComp = copy.deepcopy(comp)
104  newComp.files = [chunk[0]]
105  newComp.fineSplit = ( chunk[1], comp.fineSplitFactor )
106  newComp.name = '{name}_Chunk{index}'.format(name=newComp.name,
107  index=ichunk)
108  splitComps.append( newComp )
109  elif hasattr( comp, 'splitFactor') and comp.splitFactor>1:
110  chunkSize = len(comp.files) / comp.splitFactor
111  if len(comp.files) % comp.splitFactor:
112  chunkSize += 1
113  # print 'chunk size',chunkSize, len(comp.files), comp.splitFactor
114  for ichunk, chunk in enumerate( chunks( comp.files, chunkSize)):
115  newComp = copy.deepcopy(comp)
116  newComp.files = chunk
117  newComp.name = '{name}_Chunk{index}'.format(name=newComp.name,
118  index=ichunk)
119  splitComps.append( newComp )
120  else:
121  splitComps.append( comp )
122  return splitComps
123 
124 
heppy.chunks
def chunks
Definition: heppy.py:92
heppy.split
def split
Definition: heppy.py:95

Variable Documentation

string heppy.action = 'store_true'

Definition at line 198 of file heppy.py.

int heppy.default = None)

Definition at line 187 of file heppy.py.

string heppy.dest = "nevents"

Definition at line 185 of file heppy.py.

string heppy.help = "number of events to process"

Definition at line 186 of file heppy.py.

heppy.loop = None

Definition at line 26 of file heppy.py.

list heppy.oldv = sys.argv[:]

Definition at line 16 of file heppy.py.

tuple heppy.parser = OptionParser()

Definition at line 177 of file heppy.py.

Generated for CMSSW Reference Manual by doxygen 1.8.5