CMS 3D CMS Logo

 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Properties Friends Macros Pages
List of all members | Public Member Functions | Public Attributes
pkg.AbstractPkg Class Reference
Inheritance diagram for pkg.AbstractPkg:

Public Member Functions

def __init__
 
def generate
 
def get_kwds
 
def parse_etags
 
def print_etags
 
def print_tags
 
def tmpl_etags
 
def tmpl_tags
 
def write
 

Public Attributes

 author
 
 config
 
 date
 
 debug
 
 not_in_dir
 
 pname
 
 tdir
 
 tmpl
 

Detailed Description

AbstractPkg takes care how to generate code from template/PKG
package area. The PKG can be any directory which may include
any types of files, e.g. C++ (.cc), python (.py), etc.
This class relies on specific logic which we outline here:

    - each template may use tags defined with double underscores
      enclosure, e.g. __class__, __record__, etc.
    - each template may have example tags, such tags should
      start with @example_. While processing template user may
      choose to strip them off or keep the code behind those tags
    - in addition user may specify pure python code which can
      operate with user defined tags. This code snipped should
      be enclosed with #python_begin and #python_end lines
      which declares start and end of python block

Definition at line 21 of file pkg.py.

Constructor & Destructor Documentation

def pkg.AbstractPkg.__init__ (   self,
  config = None 
)

Definition at line 38 of file pkg.py.

38 
39  def __init__(self, config=None):
40  super(AbstractPkg, self).__init__()
41  if not config:
42  self.config = {}
43  else:
44  self.config = config
45  self.pname = self.config.get('pname', None)
46  self.tmpl = self.config.get('tmpl', None)
47  self.debug = self.config.get('debug', 0)
48  self.tdir = self.config.get('tmpl_dir')
49  self.author = user_info(self.config.get('author', None))
50  self.date = time.strftime("%a, %d %b %Y %H:%M:%S GMT", time.gmtime())
51  self.not_in_dir = self.config.get('not_in_dir', [])
def __init__
Definition: pkg.py:38
def user_info
Definition: utils.py:96

Member Function Documentation

def pkg.AbstractPkg.generate (   self)

Definition at line 158 of file pkg.py.

References DTDigiToRawModule.debug, CSCRPCData.debug, ALIUtils.debug, CSCDCCUnpacker.debug, CSCTMBScope.debug, BxNumberFilter.debug, DTDigiToRaw.debug, EgammaProbeSelector.debug, DTTTrigCorrectionFirst.debug, DTSegmentsTask.debug, pkg.AbstractPkg.debug, DTTriggerCheck.debug, IsolatedParticlesGeneratedJets.debug, CSCDCCEventData.debug, DTTPDeadWriter.debug, DTTTrigWriter.debug, SiPixelDigiToRaw.debug, SiPixelRawToDigi.debug, DTDDUUnpacker.debug, DTT0Calibration.debug, DTT0CalibrationNew.debug, DTOccupancyPoint.debug, LumiblockFilter.debug, DTNoiseComputation.debug, DTROS25Unpacker.debug, DTTTrigSyncT0Only.debug, DTAlbertoBenvenutiTask.debug, DTTTrigCalibration.debug, DTEfficiencyTask.debug, DTRunConditionVar.debug, CSCDDUEventData.debug, HCALResponse.debug, DTDigiForNoiseTask.debug, DTChamberEfficiencyTask.debug, DTNoiseTest.debug, DTTMax.debug, DTChamberEfficiency.debug, CSCCLCTData.debug, DTVDriftCalibration.debug, PixelDataFormatter.debug, DTTTrigSyncFromDB.debug, CSCTMBData.debug, DTTTrigSyncTOFCorr.debug, BaseParticlePropagator.debug, RPCEfficiency.debug, CSCEventData.debug, CSCTMBHeader.debug, CSCALCTTrailer.debug, AlCaHOCalibProducer.debug, CSCALCTHeader.debug, spr.find(), pkg.AbstractPkg.get_kwds(), join(), pkg.AbstractPkg.not_in_dir, pkg.AbstractPkg.pname, python.rootplot.root2matplotlib.replace(), split, pkg.AbstractPkg.tdir, pkg.AbstractPkg.tmpl, ora::PVectorWriter.write(), pos::PixelConfig.write(), Json::Writer.write(), ora::IBlobStreamingService.write(), AlignmentCorrelationsIO.write(), FileBlob.write(), edm::RunLumiEntryInfo.write(), edm::EventEntryInfo.write(), ora::IRelationalWriter.write(), AlignmentCorrelationsIORoot.write(), SurveyOutput.write(), edm::AsciiOutputModule.write(), ora::QueryableVectorWriter.write(), cond::BlobStreamingService.write(), ExTestEcalChannelStatusAnalyzer.write(), HcalPedestalsPopConAnalyzer.write(), KalmanAlignmentDataCollector.write(), HcalRespCorrsPopConAnalyzer.write(), HcalTimingParamsPopConAnalyzer.write(), HcalZSThresholdsPopConAnalyzer.write(), HcalLongRecoParamsPopConAnalyzer.write(), HcalLutMetadataPopConAnalyzer.write(), HcalTimeCorrsPopConAnalyzer.write(), HcalCovarianceMatricesPopConAnalyzer.write(), ora::CArrayWriter.write(), HcalValidationCorrsPopConAnalyzer.write(), ora::OraPtrWriter.write(), HcalRecoParamsPopConAnalyzer.write(), CastorGainWidthsPopConAnalyzer.write(), MultiFileBlob.write(), CastorChannelQualityPopConAnalyzer.write(), CastorElectronicsMapPopConAnalyzer.write(), CastorGainsPopConAnalyzer.write(), CastorPedestalsPopConAnalyzer.write(), CastorPedestalWidthsPopConAnalyzer.write(), CastorQIEDataPopConAnalyzer.write(), CastorSaturationCorrsPopConAnalyzer.write(), HcalChannelQualityPopConAnalyzer.write(), HcalCholeskyMatricesPopConAnalyzer.write(), HcalFlagHFDigiTimeParamsPopConAnalyzer.write(), HcalDcsMapPopConAnalyzer.write(), HcalDcsValuesPopConAnalyzer.write(), HcalElectronicsMapPopConAnalyzer.write(), HcalGainsPopConAnalyzer.write(), HcalGainWidthsPopConAnalyzer.write(), HcalL1TriggerObjectsPopConAnalyzer.write(), CastorRecoParamsPopConAnalyzer.write(), HcalLUTCorrsPopConAnalyzer.write(), HcalMCParamsPopConAnalyzer.write(), HcalPedestalWidthsPopConAnalyzer.write(), HcalPFCorrsPopConAnalyzer.write(), HcalQIEDataPopConAnalyzer.write(), edm::RunAuxiliary.write(), HcalZDCLowGainFractionsPopConAnalyzer.write(), edm::Parentage.write(), cond::TBufferBlobStreamingService.write(), ora::UniqueRefWriter.write(), edm::EventEntryDescription.write(), SiPixelConfigWriter.write(), AlignmentParametersIO.write(), RawEventOutputModuleForBU< Consumer >.write(), AlignmentUserVariablesIO.write(), Json::FastWriter.write(), edm::ProvenanceCheckerOutputModule.write(), ora::STLContainerWriter.write(), SiStripConfigWriter.write(), edm::LuminosityBlockAuxiliary.write(), edm::ProductProvenance.write(), popcon::PopConAnalyzer< S >.write(), edm::GetProductCheckerOutputModule.write(), ora::ObjectWriter.write(), popcon::PopCon.write(), edm::ModuleDescription.write(), ora::PrimitiveWriter.write(), MuonAlignmentOutputXML.write(), RootStreamBuffer.write(), electronCompare.flushfile.write(), l1t::LUT.write(), electronStore.flushfile.write(), ora::OraReferenceWriter.write(), ora::InlineCArrayWriter.write(), ora::DependentClassWriter.write(), ora::BlobWriter.write(), cond::RootOutputArchive.write(), edm::EventAuxiliary.write(), CSCPairResidualsConstraint.write(), edmplugin::CacheParser.write(), DDStreamer.write(), ora::NamedRefWriter.write(), XMLDOMBlock.write(), ora::ITokenWriter.write(), Benchmark.write(), edm::BranchDescription.write(), XMLDocument.write, edm::Provenance.write(), Json::StyledWriter.write(), MuonResidualsTwoBin.write(), edm::FileInPath.write(), pkg.AbstractPkg.write(), cond::PoolTokenWriter.write(), edm::ProductHolderBase.write(), Json::StyledStreamWriter.write(), amc13::Packet.write(), XMLProcessor.write(), esMonitoring.ElasticReport.write(), DQMRootOutputModule.write(), esMonitoring.History.write(), edm::OutputModule.write(), MuonResidualsFitter.write(), edm::one::OutputModuleBase.write(), and esMonitoring.JsonInput.write().

159  def generate(self):
160  "Generate package templates in a given directory"
161 
162  # keep current location, since generate will switch directories
163  cdir = os.getcwd()
164 
165  # read from configutation which template files to create
166  tmpl_files = self.config.get('tmpl_files', 'all')
167 
168  # setup keyword arguments which we'll pass to write method
169  kwds = self.get_kwds()
170 
171  # create template package dir and cd into it
172  if tmpl_files == 'all' and self.tmpl not in self.not_in_dir:
173  if os.path.isdir(self.pname):
174  msg = "Can't create package '%s'\n" % self.pname
175  msg += "Directory %s is already exists" % self.pname
176  print msg
177  sys.exit(1)
178  os.makedirs(self.pname)
179  os.chdir(self.pname)
180 
181  # read directory driver information and create file list to generate
182  sdir = os.path.join(self.tdir, self.tmpl)
183  sources = [s for s in os.listdir(sdir) \
184  if s != 'Driver.dir' and s.find('~') == -1]
185  driver = os.path.join(sdir, 'Driver.dir')
186  if os.path.isfile(driver):
187  sources = [s.replace('\n', '') for s in open(driver, 'r').readlines()]
188  if 'CVS' in sources:
189  sources.remove('CVS')
190 
191  # special case of Skeleton, which requires to generate only given
192  # file type if self.pname has extension of that type
193  names = set([s.split('.')[0] for s in sources])
194  if names == set(['Skeleton']):
195  if self.pname.find('.') != -1:
196  _, ext = os.path.splitext(self.pname)
197  sources = [s for s in sources if s.rfind(ext) != -1]
198  self.pname = self.pname.replace(ext, '')
199  kwds = self.get_kwds()
200  if not sources:
201  msg = 'Unable to find skeleton for extension "%s"' % ext
202  print msg
203  sys.exit(1)
204  bdir = os.environ.get('CMSSW_BASE', '')
205  dirs = os.getcwd().replace(bdir, '').split('/')
206  ldir = os.getcwd().split('/')[-1]
207  idir = ''
208  subsys = kwds['__subsys__']
209  pkgname = kwds['__pkgname__']
210  if sources == ['Skeleton.cc', 'Skeleton.h']:
211  if ldir == 'interface' and os.getcwd().find(bdir) != -1:
212  idir = '%s/%s/interface/' % (subsys, pkgname)
213  # run within some directory of the Sybsystem/Pkg area
214  # and only for mkskel <file>.cc
215  elif sources == ['Skeleton.cc'] and \
216  len(dirs) == 5 and dirs[0] == '' and dirs[1] == 'src':
217  idir = '%s/%s/interface/' % (subsys, pkgname)
218  elif sources == ['Skeleton.h'] and ldir == 'interface' and \
219  len(dirs) == 5 and dirs[0] == '' and dirs[1] == 'src':
220  idir = '%s/%s/interface/' % (subsys, pkgname)
221  kwds.update({'__incdir__': idir})
222 
223  # loop over source files, create dirs as necessary and generate files
224  # names for writing templates
225  gen_files = []
226  for src in sources:
227  if tmpl_files != 'all':
228  fname, ext = os.path.splitext(src)
229  if tmpl_files != ext:
230  continue
231  src = src.split('/')[-1]
232  if self.debug:
233  print "Read", src
234  items = src.split('/')
235  if items[-1] == '/':
236  items = items[:-1]
237  tname = items[-1] # template file name
238  tmpl_name = os.path.join(sdir, items[-1]) # full tmpl file name
239  if os.path.isfile(tmpl_name):
240  ftype = 'file'
241  else:
242  ftype = 'dir'
243  name2gen = src # new file we'll create
244  if tname.split('.')[0] == self.tmpl: # need to substitute
245  name2gen = name2gen.replace(self.tmpl, self.pname)
246  name2gen = os.path.join(os.getcwd(), name2gen)
247  if self.debug:
248  print "Create", name2gen
249  if ftype == 'dir':
250  if not os.path.isdir(name2gen):
251  os.makedirs(name2gen)
252  continue # we're done with dir
253  fdir = os.path.dirname(name2gen)
254  if not os.path.isdir(fdir):
255  os.makedirs(fdir)
256  self.write(name2gen, tmpl_name, kwds)
257  gen_files.append(name2gen.split('/')[-1])
258  if tmpl_files == 'all' and self.tmpl not in self.not_in_dir:
259  msg = 'New package "%s" of %s type is successfully generated' \
260  % (self.pname, self.tmpl)
261  else:
262  msg = 'Generated %s file' % ', '.join(gen_files)
263  if len(gen_files) > 1:
264  msg += 's'
265  print msg
266  # return back where we started
267  os.chdir(cdir)
268  if msg.find('New package') != -1:
269  tree(self.pname)
def write
Definition: pkg.py:115
void find(edm::Handle< EcalRecHitCollection > &hits, DetId thisDet, std::vector< EcalRecHitCollection::const_iterator > &hit, bool debug=false)
Definition: FindCaloHit.cc:7
static std::string join(char **cmd)
Definition: RemoteFile.cc:18
def get_kwds
Definition: pkg.py:142
Definition: tree.py:1
def generate
Definition: pkg.py:158
double split
Definition: MVATrainer.cc:139
def pkg.AbstractPkg.get_kwds (   self)

Definition at line 142 of file pkg.py.

References pkg.AbstractPkg.author, pkg.AbstractPkg.date, DTDigiToRawModule.debug, CSCRPCData.debug, ALIUtils.debug, CSCDCCUnpacker.debug, CSCTMBScope.debug, BxNumberFilter.debug, DTDigiToRaw.debug, EgammaProbeSelector.debug, DTTTrigCorrectionFirst.debug, DTSegmentsTask.debug, pkg.AbstractPkg.debug, DTTriggerCheck.debug, CSCDCCEventData.debug, DTTPDeadWriter.debug, IsolatedParticlesGeneratedJets.debug, DTTTrigWriter.debug, SiPixelDigiToRaw.debug, SiPixelRawToDigi.debug, DTDDUUnpacker.debug, DTT0Calibration.debug, DTT0CalibrationNew.debug, DTOccupancyPoint.debug, LumiblockFilter.debug, DTNoiseComputation.debug, DTROS25Unpacker.debug, DTTTrigSyncT0Only.debug, DTTTrigCalibration.debug, DTAlbertoBenvenutiTask.debug, DTEfficiencyTask.debug, CSCDDUEventData.debug, DTRunConditionVar.debug, HCALResponse.debug, DTDigiForNoiseTask.debug, DTChamberEfficiencyTask.debug, DTNoiseTest.debug, DTTMax.debug, DTChamberEfficiency.debug, CSCCLCTData.debug, DTVDriftCalibration.debug, PixelDataFormatter.debug, DTTTrigSyncFromDB.debug, CSCTMBData.debug, DTTTrigSyncTOFCorr.debug, BaseParticlePropagator.debug, RPCEfficiency.debug, CSCEventData.debug, CSCTMBHeader.debug, CSCALCTTrailer.debug, AlCaHOCalibProducer.debug, CSCALCTHeader.debug, and pkg.AbstractPkg.pname.

Referenced by pkg.AbstractPkg.generate().

143  def get_kwds(self):
144  "Return keyword arguments to be used in methods"
145  kwds = {'__pkgname__': self.config.get('pkgname', 'Package'),
146  '__author__': self.author,
147  '__user__': os.getlogin(),
148  '__date__': self.date,
149  '__class__': self.pname,
150  '__name__': self.pname,
151  '__subsys__': self.config.get('subsystem', 'Subsystem')}
152  args = self.config.get('args', None)
153  kwds.update(args)
154  if self.debug:
155  print "Template tags:"
156  pprint.pprint(kwds)
157  return kwds
def get_kwds
Definition: pkg.py:142
def pkg.AbstractPkg.parse_etags (   self,
  line 
)
Determine either skip or keep given line based on class tags 
meta-strings

Definition at line 95 of file pkg.py.

References pkg.AbstractPkg.tmpl_etags().

Referenced by pkg.AbstractPkg.write().

95 
96  def parse_etags(self, line):
97  """
98  Determine either skip or keep given line based on class tags
99  meta-strings
100  """
101  tmpl_etags = self.tmpl_etags()
102  keep_etags = self.config.get('tmpl_etags', [])
103  for tag in tmpl_etags:
104  if keep_etags:
105  for valid_tag in keep_etags:
106  if line.find(valid_tag) != -1:
107  line = line.replace(valid_tag, '')
108  return line
109  else:
110  if line.find(tag) != -1:
111  line = line.replace(tag, '')
112  line = ''
113  return line
114  return line
def tmpl_etags
Definition: pkg.py:52
def parse_etags
Definition: pkg.py:95
def pkg.AbstractPkg.print_etags (   self)

Definition at line 69 of file pkg.py.

References pkg.AbstractPkg.tmpl_etags().

69 
70  def print_etags(self):
71  "Print out template example tags"
72  for key in self.tmpl_etags():
73  print key
def tmpl_etags
Definition: pkg.py:52
def print_etags
Definition: pkg.py:69
def pkg.AbstractPkg.print_tags (   self)

Definition at line 90 of file pkg.py.

References pkg.AbstractPkg.tmpl_tags().

90 
91  def print_tags(self):
92  "Print out template keys"
93  for key in self.tmpl_tags():
94  print key
def tmpl_tags
Definition: pkg.py:74
def print_tags
Definition: pkg.py:90
def pkg.AbstractPkg.tmpl_etags (   self)

Definition at line 52 of file pkg.py.

References pkg.AbstractPkg.tdir, and pkg.AbstractPkg.tmpl.

Referenced by pkg.AbstractPkg.parse_etags(), and pkg.AbstractPkg.print_etags().

52 
53  def tmpl_etags(self):
54  "Scan template files and return example tags"
55  keys = []
56  sdir = '%s/%s' % (self.tdir, self.tmpl)
57  for name in os.listdir(sdir):
58  if name[-1] == '~':
59  continue
60  if name == 'CVS':
61  continue
62  fname = os.path.join(sdir, name)
63  with open(fname, 'r') as stream:
64  for line in stream.readlines():
65  if line.find('@example_') != -1: # possible tag
66  keys += [k for k in line.split() if \
67  k.find('@example_') != -1]
68  return set(keys)
def tmpl_etags
Definition: pkg.py:52
def pkg.AbstractPkg.tmpl_tags (   self)

Definition at line 74 of file pkg.py.

References utils.parse_word(), pkg.AbstractPkg.tdir, and pkg.AbstractPkg.tmpl.

Referenced by pkg.AbstractPkg.print_tags().

74 
75  def tmpl_tags(self):
76  "Scan template files and return template tags"
77  keys = []
78  sdir = '%s/%s' % (self.tdir, self.tmpl)
79  for name in os.listdir(sdir):
80  if name[-1] == '~':
81  continue
82  if name == 'CVS':
83  continue
84  fname = os.path.join(sdir, name)
85  with open(fname, 'r') as stream:
86  for line in stream.readlines():
87  if line.find('__') != -1: # possible key
88  keys += [k for k in parse_word(line)]
89  return set(keys)
def parse_word
Definition: utils.py:21
def tmpl_tags
Definition: pkg.py:74
def pkg.AbstractPkg.write (   self,
  fname,
  tmpl_name,
  kwds 
)

Definition at line 115 of file pkg.py.

References DTDigiToRawModule.debug, CSCRPCData.debug, CSCDCCUnpacker.debug, ALIUtils.debug, CSCTMBScope.debug, BxNumberFilter.debug, DTDigiToRaw.debug, EgammaProbeSelector.debug, DTTTrigCorrectionFirst.debug, DTSegmentsTask.debug, pkg.AbstractPkg.debug, DTTriggerCheck.debug, IsolatedParticlesGeneratedJets.debug, CSCDCCEventData.debug, DTTPDeadWriter.debug, DTTTrigWriter.debug, SiPixelDigiToRaw.debug, SiPixelRawToDigi.debug, DTDDUUnpacker.debug, DTT0Calibration.debug, DTT0CalibrationNew.debug, DTOccupancyPoint.debug, DTNoiseComputation.debug, LumiblockFilter.debug, DTROS25Unpacker.debug, DTTTrigSyncT0Only.debug, DTTTrigCalibration.debug, DTAlbertoBenvenutiTask.debug, DTEfficiencyTask.debug, CSCDDUEventData.debug, DTRunConditionVar.debug, HCALResponse.debug, DTDigiForNoiseTask.debug, DTChamberEfficiencyTask.debug, DTNoiseTest.debug, DTTMax.debug, DTChamberEfficiency.debug, CSCCLCTData.debug, DTVDriftCalibration.debug, PixelDataFormatter.debug, DTTTrigSyncFromDB.debug, CSCTMBData.debug, DTTTrigSyncTOFCorr.debug, BaseParticlePropagator.debug, RPCEfficiency.debug, CSCEventData.debug, CSCTMBHeader.debug, CSCALCTTrailer.debug, AlCaHOCalibProducer.debug, CSCALCTHeader.debug, utils.functor(), and pkg.AbstractPkg.parse_etags().

Referenced by pkg.AbstractPkg.generate().

116  def write(self, fname, tmpl_name, kwds):
117  "Create new file from given template name and set of arguments"
118  code = ""
119  read_code = False
120  with open(fname, 'w') as stream:
121  for line in open(tmpl_name, 'r').readlines():
122  line = self.parse_etags(line)
123  if not line:
124  continue
125  if line.find('#python_begin') != -1:
126  read_code = True
127  continue
128  if line.find('#python_end') != -1:
129  read_code = False
130  if read_code:
131  code += line
132  if code and not read_code:
133  res = functor(code, kwds, self.debug)
134  stream.write(res)
135  code = ""
136  continue
137  if not read_code:
138  for key, val in kwds.items():
139  if isinstance(val, basestring):
140  line = line.replace(key, val)
141  stream.write(line)
def write
Definition: pkg.py:115
def parse_etags
Definition: pkg.py:95
def functor
Definition: utils.py:54

Member Data Documentation

pkg.AbstractPkg.author

Definition at line 48 of file pkg.py.

Referenced by pkg.AbstractPkg.get_kwds().

pkg.AbstractPkg.config

Definition at line 41 of file pkg.py.

pkg.AbstractPkg.date

Definition at line 49 of file pkg.py.

Referenced by pkg.AbstractPkg.get_kwds().

pkg.AbstractPkg.debug

Definition at line 46 of file pkg.py.

Referenced by rrapi.RRApi.dprint(), pkg.AbstractPkg.generate(), rrapi.RRApi.get(), pkg.AbstractPkg.get_kwds(), and pkg.AbstractPkg.write().

pkg.AbstractPkg.not_in_dir

Definition at line 50 of file pkg.py.

Referenced by pkg.AbstractPkg.generate().

pkg.AbstractPkg.pname

Definition at line 44 of file pkg.py.

Referenced by pkg.AbstractPkg.generate(), and pkg.AbstractPkg.get_kwds().

pkg.AbstractPkg.tdir

Definition at line 47 of file pkg.py.

Referenced by pkg.AbstractPkg.generate(), pkg.AbstractPkg.tmpl_etags(), and pkg.AbstractPkg.tmpl_tags().

pkg.AbstractPkg.tmpl

Definition at line 45 of file pkg.py.

Referenced by pkg.AbstractPkg.generate(), pkg.AbstractPkg.tmpl_etags(), and pkg.AbstractPkg.tmpl_tags().