CMS 3D CMS Logo

List of all members | Public Member Functions | Public Attributes
pkg.AbstractPkg Class Reference
Inheritance diagram for pkg.AbstractPkg:

Public Member Functions

def __init__ (self, config=None)
 
def generate (self)
 
def get_kwds (self)
 
def parse_etags (self, line)
 
def print_etags (self)
 
def print_tags (self)
 
def tmpl_etags (self)
 
def tmpl_tags (self)
 
def write (self, fname, tmpl_name, kwds)
 

Public Attributes

 author
 
 config
 
 date
 
 debug
 
 not_in_dir
 
 pname
 
 tdir
 
 tmpl
 

Detailed Description

AbstractPkg takes care how to generate code from template/PKG
package area. The PKG can be any directory which may include
any types of files, e.g. C++ (.cc), python (.py), etc.
This class relies on specific logic which we outline here:

    - each template may use tags defined with double underscores
      enclosure, e.g. __class__, __record__, etc.
    - each template may have example tags, such tags should
      start with @example_. While processing template user may
      choose to strip them off or keep the code behind those tags
    - in addition user may specify pure python code which can
      operate with user defined tags. This code snipped should
      be enclosed with #python_begin and #python_end lines
      which declares start and end of python block

Definition at line 21 of file pkg.py.

Constructor & Destructor Documentation

def pkg.AbstractPkg.__init__ (   self,
  config = None 
)

Definition at line 38 of file pkg.py.

38  def __init__(self, config=None):
39  super(AbstractPkg, self).__init__()
40  if not config:
41  self.config = {}
42  else:
43  self.config = config
44  self.pname = self.config.get('pname', None)
45  self.tmpl = self.config.get('tmpl', None)
46  self.debug = self.config.get('debug', 0)
47  self.tdir = self.config.get('tmpl_dir')
48  self.author = user_info(self.config.get('author', None))
49  self.date = time.strftime("%a, %d %b %Y %H:%M:%S GMT", time.gmtime())
50  self.not_in_dir = self.config.get('not_in_dir', [])
51 
def __init__(self, config=None)
Definition: pkg.py:38
def user_info(ainput=None)
Definition: utils.py:96

Member Function Documentation

def pkg.AbstractPkg.generate (   self)

Definition at line 158 of file pkg.py.

References DTDigiToRawModule.debug, CSCRPCData.debug, CSCDCCUnpacker.debug, ALIUtils.debug, CSCTMBScope.debug, DTDigiToRaw.debug, EgammaProbeSelector.debug, DTTTrigCorrectionFirst.debug, DTSegmentsTask.debug, pkg.AbstractPkg.debug, CSCDCCEventData.debug, IsolatedParticlesGeneratedJets.debug, DTTPDeadWriter.debug, DTTTrigWriter.debug, SiPixelRawToDigi.debug, SiPixelDigiToRaw.debug, DTDDUUnpacker.debug, DTT0Calibration.debug, DTT0CalibrationNew.debug, DTOccupancyPoint.debug, util.rrapi.RRApi.debug, DTNoiseComputation.debug, DTROS25Unpacker.debug, DTTTrigSyncT0Only.debug, DTTTrigCalibration.debug, DTEfficiencyTask.debug, CSCDDUEventData.debug, DTRunConditionVar.debug, HCALResponse.debug, DTChamberEfficiencyTask.debug, DTTMax.debug, DTChamberEfficiency.debug, CSCCLCTData.debug, DTVDriftCalibration.debug, PixelDataFormatter.debug, DTTTrigSyncFromDB.debug, CSCTMBData.debug, DTTTrigSyncTOFCorr.debug, BaseParticlePropagator.debug, RPCEfficiency.debug, CSCEventData.debug, CSCTMBHeader.debug, CSCALCTTrailer.debug, CSCALCTHeader.debug, spr.find(), pkg.AbstractPkg.get_kwds(), join(), pkg.AbstractPkg.not_in_dir, pkg.AbstractPkg.pname, python.rootplot.root2matplotlib.replace(), split, pkg.AbstractPkg.tdir, pkg.AbstractPkg.tmpl, edm::ThreadSafeOutputFileStream.write(), pos::PixelConfig.write(), Json::Writer.write(), AlignmentCorrelationsIO.write(), FileBlob.write(), edm::RunLumiEntryInfo.write(), HcalSiPMParametersPopConAnalyzer.write(), HcalTPChannelParametersPopConAnalyzer.write(), AlignmentCorrelationsIORoot.write(), edm::EventEntryInfo.write(), SurveyOutput.write(), HcalTPParametersPopConAnalyzer.write(), HcalSiPMCharacteristicsPopConAnalyzer.write(), edm::AsciiOutputModule.write(), ExTestEcalChannelStatusAnalyzer.write(), personalPlayback.Applet.write(), HcalFrontEndMapPopConAnalyzer.write(), HcalDcsValuesPopConAnalyzer.write(), HcalPedestalWidthsPopConAnalyzer.write(), HcalRecoParamsPopConAnalyzer.write(), MultiFileBlob.write(), HcalGainsPopConAnalyzer.write(), CastorElectronicsMapPopConAnalyzer.write(), CastorGainsPopConAnalyzer.write(), CastorGainWidthsPopConAnalyzer.write(), CastorPedestalsPopConAnalyzer.write(), CastorPedestalWidthsPopConAnalyzer.write(), CastorQIEDataPopConAnalyzer.write(), CastorRecoParamsPopConAnalyzer.write(), CastorSaturationCorrsPopConAnalyzer.write(), HcalChannelQualityPopConAnalyzer.write(), HcalDcsMapPopConAnalyzer.write(), HcalElectronicsMapPopConAnalyzer.write(), HcalFlagHFDigiTimeParamsPopConAnalyzer.write(), CastorChannelQualityPopConAnalyzer.write(), HcalGainWidthsPopConAnalyzer.write(), HcalL1TriggerObjectsPopConAnalyzer.write(), HcalLongRecoParamsPopConAnalyzer.write(), HcalValidationCorrsPopConAnalyzer.write(), HcalLutMetadataPopConAnalyzer.write(), HcalMCParamsPopConAnalyzer.write(), HcalPedestalsPopConAnalyzer.write(), HcalPFCorrsPopConAnalyzer.write(), HcalQIEDataPopConAnalyzer.write(), HcalQIETypesPopConAnalyzer.write(), HcalLUTCorrsPopConAnalyzer.write(), HcalRespCorrsPopConAnalyzer.write(), HcalTimeCorrsPopConAnalyzer.write(), HcalTimingParamsPopConAnalyzer.write(), HcalZSThresholdsPopConAnalyzer.write(), edm::RunAuxiliary.write(), HcalZDCLowGainFractionsPopConAnalyzer.write(), AlignmentParametersIO.write(), edm::EventEntryDescription.write(), SiPixelConfigWriter.write(), Json::FastWriter.write(), AlignmentUserVariablesIO.write(), SiStripConfigWriter.write(), edm::LuminosityBlockAuxiliary.write(), RawEventOutputModuleForBU< Consumer >.write(), edm::GetProductCheckerOutputModule.write(), popcon::PopConAnalyzer< S >.write(), edm::ProvenanceCheckerOutputModule.write(), edm::ProductProvenance.write(), edm::Parentage.write(), popcon::PopCon.write(), edm::ModuleDescription.write(), electronCompare.flushfile.write(), MuonAlignmentOutputXML.write(), l1t::LUT.write(), electronStore.flushfile.write(), edm::StableProvenance.write(), CSCPairResidualsConstraint.write(), edm::EventAuxiliary.write(), XMLDOMBlock.write(), edmplugin::CacheParser.write(), Benchmark.write(), edm::BranchDescription.write(), XMLDocument.write, uploads.output.write(), edm::Provenance.write(), Json::StyledWriter.write(), amc13::Packet.write(), ztee.GZipLog.write(), MuonResidualsTwoBin.write(), presentation.SubsectionBase.write(), edm::FileInPath.write(), pkg.AbstractPkg.write(), Json::StyledStreamWriter.write(), edm::ProductResolverBase.write(), XMLProcessor.write(), esMonitoring.LineHistoryEnd.write(), DQMRootOutputModule.write(), MuonResidualsFitter.write(), esMonitoring.LineHistoryStart.write(), edm::OutputModule.write(), betterConfigParser.BetterConfigParser.write(), edm::one::OutputModuleBase.write(), edm::global::OutputModuleBase.write(), and querying.connection.write().

158  def generate(self):
159  "Generate package templates in a given directory"
160 
161  # keep current location, since generate will switch directories
162  cdir = os.getcwd()
163 
164  # read from configutation which template files to create
165  tmpl_files = self.config.get('tmpl_files', 'all')
166 
167  # setup keyword arguments which we'll pass to write method
168  kwds = self.get_kwds()
169 
170  # create template package dir and cd into it
171  if tmpl_files == 'all' and self.tmpl not in self.not_in_dir:
172  if os.path.isdir(self.pname):
173  msg = "Can't create package '%s'\n" % self.pname
174  msg += "Directory %s is already exists" % self.pname
175  print msg
176  sys.exit(1)
177  os.makedirs(self.pname)
178  os.chdir(self.pname)
179 
180  # read directory driver information and create file list to generate
181  sdir = os.path.join(self.tdir, self.tmpl)
182  sources = [s for s in os.listdir(sdir) \
183  if s != 'Driver.dir' and s.find('~') == -1]
184  driver = os.path.join(sdir, 'Driver.dir')
185  if os.path.isfile(driver):
186  sources = [s.replace('\n', '') for s in open(driver, 'r').readlines()]
187  if 'CVS' in sources:
188  sources.remove('CVS')
189 
190  # special case of Skeleton, which requires to generate only given
191  # file type if self.pname has extension of that type
192  names = set([s.split('.')[0] for s in sources])
193  if names == set(['Skeleton']):
194  if self.pname.find('.') != -1:
195  _, ext = os.path.splitext(self.pname)
196  sources = [s for s in sources if s.rfind(ext) != -1]
197  self.pname = self.pname.replace(ext, '')
198  kwds = self.get_kwds()
199  if not sources:
200  msg = 'Unable to find skeleton for extension "%s"' % ext
201  print msg
202  sys.exit(1)
203  bdir = os.environ.get('CMSSW_BASE', '')
204  dirs = os.getcwd().replace(bdir, '').split('/')
205  ldir = os.getcwd().split('/')[-1]
206  idir = ''
207  subsys = kwds['__subsys__']
208  pkgname = kwds['__pkgname__']
209  if sources == ['Skeleton.cc', 'Skeleton.h']:
210  if ldir == 'interface' and os.getcwd().find(bdir) != -1:
211  idir = '%s/%s/interface/' % (subsys, pkgname)
212  # run within some directory of the Sybsystem/Pkg area
213  # and only for mkskel <file>.cc
214  elif sources == ['Skeleton.cc'] and \
215  len(dirs) == 5 and dirs[0] == '' and dirs[1] == 'src':
216  idir = '%s/%s/interface/' % (subsys, pkgname)
217  elif sources == ['Skeleton.h'] and ldir == 'interface' and \
218  len(dirs) == 5 and dirs[0] == '' and dirs[1] == 'src':
219  idir = '%s/%s/interface/' % (subsys, pkgname)
220  kwds.update({'__incdir__': idir})
221 
222  # loop over source files, create dirs as necessary and generate files
223  # names for writing templates
224  gen_files = []
225  for src in sources:
226  if tmpl_files != 'all':
227  fname, ext = os.path.splitext(src)
228  if tmpl_files != ext:
229  continue
230  src = src.split('/')[-1]
231  if self.debug:
232  print "Read", src
233  items = src.split('/')
234  if items[-1] == '/':
235  items = items[:-1]
236  tname = items[-1] # template file name
237  tmpl_name = os.path.join(sdir, items[-1]) # full tmpl file name
238  if os.path.isfile(tmpl_name):
239  ftype = 'file'
240  else:
241  ftype = 'dir'
242  name2gen = src # new file we'll create
243  if tname.split('.')[0] == self.tmpl: # need to substitute
244  name2gen = name2gen.replace(self.tmpl, self.pname)
245  name2gen = os.path.join(os.getcwd(), name2gen)
246  if self.debug:
247  print "Create", name2gen
248  if ftype == 'dir':
249  if not os.path.isdir(name2gen):
250  os.makedirs(name2gen)
251  continue # we're done with dir
252  fdir = os.path.dirname(name2gen)
253  if not os.path.isdir(fdir):
254  os.makedirs(fdir)
255  self.write(name2gen, tmpl_name, kwds)
256  gen_files.append(name2gen.split('/')[-1])
257  if tmpl_files == 'all' and self.tmpl not in self.not_in_dir:
258  msg = 'New package "%s" of %s type is successfully generated' \
259  % (self.pname, self.tmpl)
260  else:
261  msg = 'Generated %s file' % ', '.join(gen_files)
262  if len(gen_files) > 1:
263  msg += 's'
264  print msg
265  # return back where we started
266  os.chdir(cdir)
267  if msg.find('New package') != -1:
268  tree(self.pname)
269 
def get_kwds(self)
Definition: pkg.py:142
def replace(string, replacements)
void find(edm::Handle< EcalRecHitCollection > &hits, DetId thisDet, std::vector< EcalRecHitCollection::const_iterator > &hit, bool debug=false)
Definition: FindCaloHit.cc:20
def generate(self)
Definition: pkg.py:158
static std::string join(char **cmd)
Definition: RemoteFile.cc:18
def write(self, fname, tmpl_name, kwds)
Definition: pkg.py:115
Definition: tree.py:1
double split
Definition: MVATrainer.cc:139
def pkg.AbstractPkg.get_kwds (   self)

Definition at line 142 of file pkg.py.

References pkg.AbstractPkg.author, pkg.AbstractPkg.date, DTDigiToRawModule.debug, CSCRPCData.debug, CSCDCCUnpacker.debug, ALIUtils.debug, CSCTMBScope.debug, DTDigiToRaw.debug, EgammaProbeSelector.debug, DTTTrigCorrectionFirst.debug, DTSegmentsTask.debug, pkg.AbstractPkg.debug, IsolatedParticlesGeneratedJets.debug, DTTPDeadWriter.debug, CSCDCCEventData.debug, DTTTrigWriter.debug, SiPixelDigiToRaw.debug, SiPixelRawToDigi.debug, DTDDUUnpacker.debug, DTT0Calibration.debug, DTT0CalibrationNew.debug, DTOccupancyPoint.debug, util.rrapi.RRApi.debug, DTNoiseComputation.debug, DTROS25Unpacker.debug, DTTTrigSyncT0Only.debug, DTTTrigCalibration.debug, DTEfficiencyTask.debug, CSCDDUEventData.debug, DTRunConditionVar.debug, HCALResponse.debug, DTChamberEfficiencyTask.debug, DTTMax.debug, DTChamberEfficiency.debug, CSCCLCTData.debug, DTVDriftCalibration.debug, PixelDataFormatter.debug, DTTTrigSyncFromDB.debug, CSCTMBData.debug, DTTTrigSyncTOFCorr.debug, BaseParticlePropagator.debug, RPCEfficiency.debug, CSCEventData.debug, CSCTMBHeader.debug, CSCALCTTrailer.debug, CSCALCTHeader.debug, and pkg.AbstractPkg.pname.

Referenced by pkg.AbstractPkg.generate().

142  def get_kwds(self):
143  "Return keyword arguments to be used in methods"
144  kwds = {'__pkgname__': self.config.get('pkgname', 'Package'),
145  '__author__': self.author,
146  '__user__': os.getlogin(),
147  '__date__': self.date,
148  '__class__': self.pname,
149  '__name__': self.pname,
150  '__subsys__': self.config.get('subsystem', 'Subsystem')}
151  args = self.config.get('args', None)
152  kwds.update(args)
153  if self.debug:
154  print "Template tags:"
155  pprint.pprint(kwds)
156  return kwds
157 
def get_kwds(self)
Definition: pkg.py:142
def pkg.AbstractPkg.parse_etags (   self,
  line 
)
Determine either skip or keep given line based on class tags 
meta-strings

Definition at line 95 of file pkg.py.

References pkg.AbstractPkg.tmpl_etags().

Referenced by pkg.AbstractPkg.write().

95  def parse_etags(self, line):
96  """
97  Determine either skip or keep given line based on class tags
98  meta-strings
99  """
100  tmpl_etags = self.tmpl_etags()
101  keep_etags = self.config.get('tmpl_etags', [])
102  for tag in tmpl_etags:
103  if keep_etags:
104  for valid_tag in keep_etags:
105  if line.find(valid_tag) != -1:
106  line = line.replace(valid_tag, '')
107  return line
108  else:
109  if line.find(tag) != -1:
110  line = line.replace(tag, '')
111  line = ''
112  return line
113  return line
114 
def parse_etags(self, line)
Definition: pkg.py:95
def tmpl_etags(self)
Definition: pkg.py:52
def pkg.AbstractPkg.print_etags (   self)

Definition at line 69 of file pkg.py.

References pkg.AbstractPkg.tmpl_etags().

69  def print_etags(self):
70  "Print out template example tags"
71  for key in self.tmpl_etags():
72  print key
73 
def print_etags(self)
Definition: pkg.py:69
def tmpl_etags(self)
Definition: pkg.py:52
def pkg.AbstractPkg.print_tags (   self)

Definition at line 90 of file pkg.py.

References pkg.AbstractPkg.tmpl_tags().

90  def print_tags(self):
91  "Print out template keys"
92  for key in self.tmpl_tags():
93  print key
94 
def print_tags(self)
Definition: pkg.py:90
def tmpl_tags(self)
Definition: pkg.py:74
def pkg.AbstractPkg.tmpl_etags (   self)

Definition at line 52 of file pkg.py.

References pkg.AbstractPkg.tdir, and pkg.AbstractPkg.tmpl.

Referenced by pkg.AbstractPkg.parse_etags(), and pkg.AbstractPkg.print_etags().

52  def tmpl_etags(self):
53  "Scan template files and return example tags"
54  keys = []
55  sdir = '%s/%s' % (self.tdir, self.tmpl)
56  for name in os.listdir(sdir):
57  if name[-1] == '~':
58  continue
59  if name == 'CVS':
60  continue
61  fname = os.path.join(sdir, name)
62  with open(fname, 'r') as stream:
63  for line in stream.readlines():
64  if line.find('@example_') != -1: # possible tag
65  keys += [k for k in line.split() if \
66  k.find('@example_') != -1]
67  return set(keys)
68 
def tmpl_etags(self)
Definition: pkg.py:52
def pkg.AbstractPkg.tmpl_tags (   self)

Definition at line 74 of file pkg.py.

References utils.parse_word(), pkg.AbstractPkg.tdir, and pkg.AbstractPkg.tmpl.

Referenced by pkg.AbstractPkg.print_tags().

74  def tmpl_tags(self):
75  "Scan template files and return template tags"
76  keys = []
77  sdir = '%s/%s' % (self.tdir, self.tmpl)
78  for name in os.listdir(sdir):
79  if name[-1] == '~':
80  continue
81  if name == 'CVS':
82  continue
83  fname = os.path.join(sdir, name)
84  with open(fname, 'r') as stream:
85  for line in stream.readlines():
86  if line.find('__') != -1: # possible key
87  keys += [k for k in parse_word(line)]
88  return set(keys)
89 
def tmpl_tags(self)
Definition: pkg.py:74
def parse_word(word)
Definition: utils.py:21
def pkg.AbstractPkg.write (   self,
  fname,
  tmpl_name,
  kwds 
)

Definition at line 115 of file pkg.py.

References DTDigiToRawModule.debug, CSCRPCData.debug, CSCDCCUnpacker.debug, CSCTMBScope.debug, ALIUtils.debug, DTDigiToRaw.debug, EgammaProbeSelector.debug, DTTTrigCorrectionFirst.debug, DTSegmentsTask.debug, pkg.AbstractPkg.debug, IsolatedParticlesGeneratedJets.debug, DTTPDeadWriter.debug, CSCDCCEventData.debug, DTTTrigWriter.debug, SiPixelDigiToRaw.debug, SiPixelRawToDigi.debug, DTDDUUnpacker.debug, DTT0Calibration.debug, DTT0CalibrationNew.debug, DTOccupancyPoint.debug, util.rrapi.RRApi.debug, DTNoiseComputation.debug, DTROS25Unpacker.debug, DTTTrigSyncT0Only.debug, DTTTrigCalibration.debug, DTEfficiencyTask.debug, DTRunConditionVar.debug, CSCDDUEventData.debug, HCALResponse.debug, DTChamberEfficiencyTask.debug, DTTMax.debug, DTChamberEfficiency.debug, CSCCLCTData.debug, DTVDriftCalibration.debug, PixelDataFormatter.debug, DTTTrigSyncFromDB.debug, CSCTMBData.debug, DTTTrigSyncTOFCorr.debug, BaseParticlePropagator.debug, RPCEfficiency.debug, CSCEventData.debug, CSCTMBHeader.debug, CSCALCTTrailer.debug, CSCALCTHeader.debug, utils.functor(), and pkg.AbstractPkg.parse_etags().

Referenced by pkg.AbstractPkg.generate().

115  def write(self, fname, tmpl_name, kwds):
116  "Create new file from given template name and set of arguments"
117  code = ""
118  read_code = False
119  with open(fname, 'w') as stream:
120  for line in open(tmpl_name, 'r').readlines():
121  line = self.parse_etags(line)
122  if not line:
123  continue
124  if line.find('#python_begin') != -1:
125  read_code = True
126  continue
127  if line.find('#python_end') != -1:
128  read_code = False
129  if read_code:
130  code += line
131  if code and not read_code:
132  res = functor(code, kwds, self.debug)
133  stream.write(res)
134  code = ""
135  continue
136  if not read_code:
137  for key, val in kwds.items():
138  if isinstance(val, basestring):
139  line = line.replace(key, val)
140  stream.write(line)
141 
def parse_etags(self, line)
Definition: pkg.py:95
def write(self, fname, tmpl_name, kwds)
Definition: pkg.py:115
def functor(code, kwds, debug=0)
Definition: utils.py:54

Member Data Documentation

pkg.AbstractPkg.author

Definition at line 48 of file pkg.py.

Referenced by pkg.AbstractPkg.get_kwds().

pkg.AbstractPkg.config

Definition at line 41 of file pkg.py.

pkg.AbstractPkg.date

Definition at line 49 of file pkg.py.

Referenced by pkg.AbstractPkg.get_kwds().

pkg.AbstractPkg.debug
pkg.AbstractPkg.not_in_dir

Definition at line 50 of file pkg.py.

Referenced by pkg.AbstractPkg.generate().

pkg.AbstractPkg.pname

Definition at line 44 of file pkg.py.

Referenced by pkg.AbstractPkg.generate(), and pkg.AbstractPkg.get_kwds().

pkg.AbstractPkg.tdir
pkg.AbstractPkg.tmpl