CMS 3D CMS Logo

 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Properties Friends Macros Groups Pages
List of all members | Public Member Functions | Public Attributes
pkg.AbstractPkg Class Reference
Inheritance diagram for pkg.AbstractPkg:

Public Member Functions

def __init__
 
def generate
 
def get_kwds
 
def parse_etags
 
def print_etags
 
def print_tags
 
def tmpl_etags
 
def tmpl_tags
 
def write
 

Public Attributes

 author
 
 config
 
 date
 
 debug
 
 not_in_dir
 
 pname
 
 tdir
 
 tmpl
 
 working_dir
 

Detailed Description

AbstractPkg takes care how to generate code from template/PKG
package area. The PKG can be any directory which may include
any types of files, e.g. C++ (.cc), python (.py), etc.
This class relies on specific logic which we outline here:

    - each template may use tags defined with double underscores
      enclosure, e.g. __class__, __record__, etc.
    - each template may have example tags, such tags should
      start with @example_. While processing template user may
      choose to strip them off or keep the code behind those tags
    - in addition user may specify pure python code which can
      operate with user defined tags. This code snipped should
      be enclosed with #python_begin and #python_end lines
      which declares start and end of python block

Definition at line 22 of file pkg.py.

Constructor & Destructor Documentation

def pkg.AbstractPkg.__init__ (   self,
  config = None 
)

Definition at line 39 of file pkg.py.

39 
40  def __init__(self, config=None):
41  super(AbstractPkg, self).__init__()
42  if not config:
43  self.config = {}
44  else:
45  self.config = config
46  self.pname = self.config.get('pname', None)
47  self.tmpl = self.config.get('tmpl', None)
48  self.debug = self.config.get('debug', 0)
50  self.author = user_info(self.config.get('author', None))
51  self.date = time.strftime("%a, %d %b %Y %H:%M:%S GMT", time.gmtime())
52  self.not_in_dir = self.config.get('not_in_dir', [])
53  self.working_dir = self.config.get('working_dir')
def __init__
Definition: pkg.py:39
def template_directory
Definition: utils.py:23
def user_info
Definition: utils.py:112

Member Function Documentation

def pkg.AbstractPkg.generate (   self)

Definition at line 159 of file pkg.py.

References DTDigiToRawModule.debug, CSCRPCData.debug, CSCTMBScope.debug, ALIUtils.debug, DTDigiToRaw.debug, crabFunctions.CrabController.debug, DTSegmentsTask.debug, DTOccupancyEfficiency.debug, CSCGEMData.debug, DTTTrigCorrectionFirst.debug, DTDDUUnpacker.debug, pkg.AbstractPkg.debug, CSCDCCEventData.debug, DTT0CalibrationRMS.debug, DTTTrigWriter.debug, DTTPDeadWriter.debug, PPSAlignmentConfigurationESSource.debug, DTOccupancyPoint.debug, DTNoiseComputation.debug, DTROS25Unpacker.debug, util.rrapi.RRApi.debug, DTT0Calibration.debug, DTTTrigSyncT0Only.debug, DTTTrigCalibration.debug, DTEfficiencyTask.debug, CSCDDUEventData.debug, DTRunConditionVar.debug, HCALResponse.debug, DTChamberEfficiencyTask.debug, DTChamberEfficiency.debug, CSCComparatorData.debug, CSCDCCUnpacker.debug, DTTMax.debug, DTVDriftCalibration.debug, DTTTrigSyncFromDB.debug, CSCTMBData.debug, DTTTrigSyncTOFCorr.debug, CSCEventData.debug, BaseParticlePropagator.debug, CSCTMBHeader.debug, CSCALCTTrailer.debug, CSCALCTHeader.debug, crabFunctions.CrabTask.debug, spr.find(), pkg.AbstractPkg.get_kwds(), join(), pkg.AbstractPkg.not_in_dir, pkg.AbstractPkg.pname, print(), python.rootplot.root2matplotlib.replace(), submitPVValidationJobs.split(), pkg.AbstractPkg.tdir, pkg.AbstractPkg.tmpl, pkg.AbstractPkg.working_dir, edm::ThreadSafeOutputFileStream.write(), Json::Writer.write(), AlignmentCorrelationsIO.write(), FRDOutputModule.write(), edm::RunAuxiliary.write(), edm::RunLumiEntryInfo.write(), ExTestEcalChannelStatusAnalyzer.write(), SurveyOutput.write(), AlignmentCorrelationsIORoot.write(), FileBlob.write(), HcalFrontEndMapPopConAnalyzer.write(), HcalTPParametersPopConAnalyzer.write(), HcalSiPMCharacteristicsPopConAnalyzer.write(), HcalSiPMParametersPopConAnalyzer.write(), HcalTPChannelParametersPopConAnalyzer.write(), edm::EventEntryInfo.write(), HcalPFCorrsPopConAnalyzer.write(), CastorSaturationCorrsPopConAnalyzer.write(), CastorChannelQualityPopConAnalyzer.write(), HcalGainWidthsPopConAnalyzer.write(), HcalTimingParamsPopConAnalyzer.write(), CastorElectronicsMapPopConAnalyzer.write(), CastorGainsPopConAnalyzer.write(), CastorGainWidthsPopConAnalyzer.write(), CastorPedestalsPopConAnalyzer.write(), CastorPedestalWidthsPopConAnalyzer.write(), CastorQIEDataPopConAnalyzer.write(), CastorRecoParamsPopConAnalyzer.write(), HcalChannelQualityPopConAnalyzer.write(), HcalDcsMapPopConAnalyzer.write(), HcalDcsValuesPopConAnalyzer.write(), HcalFlagHFDigiTimeParamsPopConAnalyzer.write(), HcalGainsPopConAnalyzer.write(), HcalL1TriggerObjectsPopConAnalyzer.write(), HcalLongRecoParamsPopConAnalyzer.write(), HcalLUTCorrsPopConAnalyzer.write(), HcalMCParamsPopConAnalyzer.write(), HcalPedestalsPopConAnalyzer.write(), HcalPedestalWidthsPopConAnalyzer.write(), HcalQIEDataPopConAnalyzer.write(), HcalQIETypesPopConAnalyzer.write(), HcalRecoParamsPopConAnalyzer.write(), HcalTimeCorrsPopConAnalyzer.write(), HcalLutMetadataPopConAnalyzer.write(), HcalRespCorrsPopConAnalyzer.write(), HcalValidationCorrsPopConAnalyzer.write(), HcalZSThresholdsPopConAnalyzer.write(), HcalElectronicsMapPopConAnalyzer.write(), HcalZDCLowGainFractionsPopConAnalyzer.write(), edm::LuminosityBlockAuxiliary.write(), MultiFileBlob.write(), SiPixelConfigWriter.write(), SiStripConfigWriter.write(), personalPlayback.Applet.write(), edm::AsciiOutputModule.write(), AlignmentParametersIO.write(), edm::CompactEventAuxiliaryVector::CompactEventAuxiliaryExtra.write(), popcon::PopConAnalyzer< S >.write(), AlignmentUserVariablesIO.write(), Json::FastWriter.write(), edm::ProductProvenance.write(), edm::EventEntryDescription.write(), popcon::PopCon.write(), edm::ModuleDescription.write(), edm::ProvenanceCheckerOutputModule.write(), edm::GetProductCheckerOutputModule.write(), RawEventOutputModuleForBU< Consumer >.write(), edm::Parentage.write(), ProduceDropBoxMetadata.write, electronCompare.flushfile.write(), edm::StableProvenance.write(), l1t::LUT.write(), electronStore.flushfile.write(), MuonAlignmentOutputXML.write(), XMLDOMBlock.write(), Benchmark.write(), edmplugin::CacheParser.write(), edm::EventAuxiliary.write(), edm::BranchDescription.write(), CSCPairResidualsConstraint.write(), uploads.output.write(), Json::StyledWriter.write(), edm::Provenance.write(), edm::CompactEventAuxiliaryVector::CompactEventAuxiliary.write(), evf::EvFOutputModule.write(), ztee.GZipLog.write(), amc13::Packet.write(), presentation.SubsectionBase.write(), MuonResidualsTwoBin.write(), DTWorkflow.DTWorkflow.write(), pkg.AbstractPkg.write(), Json::StyledStreamWriter.write(), evf::GlobalEvFOutputModule.write(), edm::ProductResolverBase.write(), XMLProcessor.write(), esMonitoring.LineHistoryEnd.write(), MuonResidualsFitter.write(), esMonitoring.LineHistoryStart.write(), StraightTrackAlignment::RPSetPlots.write(), DQMRootOutputModule.write(), edm::core::OutputModuleCore.write(), betterConfigParser.BetterConfigParser.write(), and querying.connection.write().

160  def generate(self):
161  "Generate package templates in a given directory"
162 
163  # keep current location, since generate will switch directories
164  cdir = os.getcwd()
165 
166  # read from configutation which template files to create
167  tmpl_files = self.config.get('tmpl_files', 'all')
168 
169  # setup keyword arguments which we'll pass to write method
170  kwds = self.get_kwds()
171 
172  # create template package dir and cd into it
173  if tmpl_files == 'all' and self.tmpl not in self.not_in_dir:
174  if os.path.isdir(self.pname):
175  msg = "Can't create package '%s'\n" % self.pname
176  msg += "Directory %s is already exists" % self.pname
177  print(msg)
178  sys.exit(1)
179  os.makedirs(self.pname)
180  os.chdir(self.pname)
181 
182  # read directory driver information and create file list to generate
183  sdir = os.path.join(self.tdir, self.tmpl)
184  sources = [s for s in os.listdir(sdir) \
185  if s != 'Driver.dir' and s.find('~') == -1]
186  driver = os.path.join(sdir, 'Driver.dir')
187  if os.path.isfile(driver):
188  sources = [s.replace('\n', '') for s in open(driver, 'r').readlines()]
189  if 'CVS' in sources:
190  sources.remove('CVS')
191 
192  # special case of Skeleton, which requires to generate only given
193  # file type if self.pname has extension of that type
194  names = set([s.split('.')[0] for s in sources])
195  if names == set(['Skeleton']):
196  if self.pname.find('.') != -1:
197  _, ext = os.path.splitext(self.pname)
198  sources = [s for s in sources if s.rfind(ext) != -1]
199  self.pname = self.pname.replace(ext, '')
200  kwds = self.get_kwds()
201  if not sources:
202  msg = 'Unable to find skeleton for extension "%s"' % ext
203  print(msg)
204  sys.exit(1)
205  bdir = os.environ.get('CMSSW_BASE', '')
206  dirs = os.getcwd().replace(bdir, '').split('/')
207  ldir = os.getcwd().split('/')[-1]
208  idir = ''
209  subsys = kwds['__subsys__']
210  pkgname = kwds['__pkgname__']
211  if sources == ['Skeleton.cc', 'Skeleton.h']:
212  if ldir == 'interface' and os.getcwd().find(bdir) != -1:
213  idir = '%s/%s/interface/' % (subsys, pkgname)
214  # run within some directory of the Sybsystem/Pkg area
215  # and only for mkskel <file>.cc
216  elif sources == ['Skeleton.cc'] and \
217  len(dirs) == 5 and dirs[0] == '' and dirs[1] == 'src':
218  idir = '%s/%s/interface/' % (subsys, pkgname)
219  elif sources == ['Skeleton.h'] and ldir == 'interface' and \
220  len(dirs) == 5 and dirs[0] == '' and dirs[1] == 'src':
221  idir = '%s/%s/interface/' % (subsys, pkgname)
222  kwds.update({'__incdir__': idir})
223 
224  # loop over source files, create dirs as necessary and generate files
225  # names for writing templates
226  gen_files = []
227  for src in sources:
228  if tmpl_files != 'all':
229  fname, ext = os.path.splitext(src)
230  if tmpl_files != ext:
231  continue
232  #also reject if this is the wrong directory
233  if self.working_dir and src.split('/')[-2] != self.working_dir:
234  continue
235  src = src.split('/')[-1]
236  if self.debug:
237  print("Read", src)
238  items = src.split('/')
239  if items[-1] == '/':
240  items = items[:-1]
241  tname = items[-1] # template file name
242  tmpl_name = os.path.join(sdir, items[-1]) # full tmpl file name
243  if os.path.isfile(tmpl_name):
244  ftype = 'file'
245  else:
246  ftype = 'dir'
247  name2gen = src # new file we'll create
248  if items[-1] == 'testBuildFile.xml':
249  name2gen = '/'.join(src.split('/')[:-1])+'/BuildFile.xml'
250  if -1 !=tname.split('.')[0].find(self.tmpl): # need to substitute
251  name2gen = name2gen.replace(self.tmpl, self.pname)
252  name2gen = os.path.join(os.getcwd(), name2gen)
253  if self.debug:
254  print("Create", name2gen)
255  if ftype == 'dir':
256  if not os.path.isdir(name2gen):
257  os.makedirs(name2gen)
258  continue # we're done with dir
259  fdir = os.path.dirname(name2gen)
260  if not os.path.isdir(fdir):
261  os.makedirs(fdir)
262  self.write(name2gen, tmpl_name, kwds)
263  gen_files.append(name2gen.split('/')[-1])
264  if tmpl_files == 'all' and self.tmpl not in self.not_in_dir:
265  msg = 'New package "%s" of %s type is successfully generated' \
266  % (self.pname, self.tmpl)
267  else:
268  msg = 'Generated %s file' % ', '.join(gen_files)
269  if len(gen_files) > 1:
270  msg += 's'
271  print(msg)
272  # return back where we started
273  os.chdir(cdir)
274  if msg.find('New package') != -1:
275  tree(self.pname)
def write
Definition: pkg.py:114
void find(edm::Handle< EcalRecHitCollection > &hits, DetId thisDet, std::vector< EcalRecHitCollection::const_iterator > &hit, bool debug=false)
Definition: FindCaloHit.cc:19
void print(TMatrixD &m, const char *label=nullptr, bool mathematicaFormat=false)
Definition: Utilities.cc:47
static std::string join(char **cmd)
Definition: RemoteFile.cc:19
def get_kwds
Definition: pkg.py:143
Definition: tree.py:1
def generate
Definition: pkg.py:159
def pkg.AbstractPkg.get_kwds (   self)

Definition at line 143 of file pkg.py.

References pkg.AbstractPkg.author, pkg.AbstractPkg.date, DTDigiToRawModule.debug, CSCRPCData.debug, CSCTMBScope.debug, ALIUtils.debug, DTDigiToRaw.debug, crabFunctions.CrabController.debug, DTSegmentsTask.debug, DTOccupancyEfficiency.debug, CSCGEMData.debug, DTDDUUnpacker.debug, DTTTrigCorrectionFirst.debug, pkg.AbstractPkg.debug, CSCDCCEventData.debug, DTT0CalibrationRMS.debug, DTTTrigWriter.debug, DTTPDeadWriter.debug, PPSAlignmentConfigurationESSource.debug, DTOccupancyPoint.debug, DTNoiseComputation.debug, DTROS25Unpacker.debug, util.rrapi.RRApi.debug, DTT0Calibration.debug, DTTTrigSyncT0Only.debug, DTTTrigCalibration.debug, DTEfficiencyTask.debug, CSCDDUEventData.debug, DTRunConditionVar.debug, HCALResponse.debug, DTChamberEfficiencyTask.debug, DTChamberEfficiency.debug, CSCComparatorData.debug, CSCDCCUnpacker.debug, DTTMax.debug, DTVDriftCalibration.debug, DTTTrigSyncFromDB.debug, CSCTMBData.debug, DTTTrigSyncTOFCorr.debug, CSCEventData.debug, BaseParticlePropagator.debug, CSCTMBHeader.debug, CSCALCTTrailer.debug, CSCALCTHeader.debug, crabFunctions.CrabTask.debug, pkg.AbstractPkg.pname, and print().

Referenced by pkg.AbstractPkg.generate().

144  def get_kwds(self):
145  "Return keyword arguments to be used in methods"
146  kwds = {'__pkgname__': self.config.get('pkgname', 'Package'),
147  '__author__': self.author,
148  '__date__': self.date,
149  '__class__': self.pname,
150  '__class_lowercase__': self.pname.lower(),
151  '__name__': self.pname,
152  '__subsys__': self.config.get('subsystem', 'Subsystem')}
153  args = self.config.get('args', None)
154  kwds.update(args)
155  if self.debug:
156  print("Template tags:")
157  pprint.pprint(kwds)
158  return kwds
void print(TMatrixD &m, const char *label=nullptr, bool mathematicaFormat=false)
Definition: Utilities.cc:47
def get_kwds
Definition: pkg.py:143
def pkg.AbstractPkg.parse_etags (   self,
  line 
)
Determine either skip or keep given line based on class tags
meta-strings

Definition at line 97 of file pkg.py.

References pkg.AbstractPkg.tmpl_etags().

Referenced by pkg.AbstractPkg.write().

97 
98  def parse_etags(self, line):
99  """
100  Determine either skip or keep given line based on class tags
101  meta-strings
102  """
103  tmpl_etags = self.tmpl_etags()
104  keep_etags = self.config.get('tmpl_etags', [])
105  for tag in tmpl_etags:
106  for valid_tag in keep_etags:
107  if line.find(valid_tag) != -1:
108  line = line.replace(valid_tag, '')
109  return line
110  if line.find(tag) != -1:
111  line = ''
112  return line
113  return line
def tmpl_etags
Definition: pkg.py:54
def parse_etags
Definition: pkg.py:97
def pkg.AbstractPkg.print_etags (   self)

Definition at line 71 of file pkg.py.

References print(), and pkg.AbstractPkg.tmpl_etags().

71 
72  def print_etags(self):
73  "Print out template example tags"
74  for key in self.tmpl_etags():
75  print(key)
def tmpl_etags
Definition: pkg.py:54
void print(TMatrixD &m, const char *label=nullptr, bool mathematicaFormat=false)
Definition: Utilities.cc:47
def print_etags
Definition: pkg.py:71
def pkg.AbstractPkg.print_tags (   self)

Definition at line 92 of file pkg.py.

References print(), and pkg.AbstractPkg.tmpl_tags().

92 
93  def print_tags(self):
94  "Print out template keys"
95  for key in self.tmpl_tags():
96  print(key)
void print(TMatrixD &m, const char *label=nullptr, bool mathematicaFormat=false)
Definition: Utilities.cc:47
def tmpl_tags
Definition: pkg.py:76
def print_tags
Definition: pkg.py:92
def pkg.AbstractPkg.tmpl_etags (   self)

Definition at line 54 of file pkg.py.

References pkg.AbstractPkg.tdir, and pkg.AbstractPkg.tmpl.

Referenced by pkg.AbstractPkg.parse_etags(), and pkg.AbstractPkg.print_etags().

54 
55  def tmpl_etags(self):
56  "Scan template files and return example tags"
57  keys = []
58  sdir = '%s/%s' % (self.tdir, self.tmpl)
59  for name in os.listdir(sdir):
60  if name[-1] == '~':
61  continue
62  if name == 'CVS':
63  continue
64  fname = os.path.join(sdir, name)
65  with open(fname, 'r') as stream:
66  for line in stream.readlines():
67  if line.find('@example_') != -1: # possible tag
68  keys += [k for k in line.split() if \
69  k.find('@example_') != -1]
70  return set(keys)
def tmpl_etags
Definition: pkg.py:54
def pkg.AbstractPkg.tmpl_tags (   self)

Definition at line 76 of file pkg.py.

References utils.parse_word(), pkg.AbstractPkg.tdir, and pkg.AbstractPkg.tmpl.

Referenced by pkg.AbstractPkg.print_tags().

76 
77  def tmpl_tags(self):
78  "Scan template files and return template tags"
79  keys = []
80  sdir = '%s/%s' % (self.tdir, self.tmpl)
81  for name in os.listdir(sdir):
82  if name[-1] == '~':
83  continue
84  if name == 'CVS':
85  continue
86  fname = os.path.join(sdir, name)
87  with open(fname, 'r') as stream:
88  for line in stream.readlines():
89  if line.find('__') != -1: # possible key
90  keys += [k for k in parse_word(line)]
91  return set(keys)
def parse_word
Definition: utils.py:37
def tmpl_tags
Definition: pkg.py:76
def pkg.AbstractPkg.write (   self,
  fname,
  tmpl_name,
  kwds 
)

Definition at line 114 of file pkg.py.

References DTDigiToRawModule.debug, CSCRPCData.debug, CSCTMBScope.debug, ALIUtils.debug, DTDigiToRaw.debug, crabFunctions.CrabController.debug, DTSegmentsTask.debug, DTOccupancyEfficiency.debug, CSCGEMData.debug, DTTTrigCorrectionFirst.debug, DTDDUUnpacker.debug, pkg.AbstractPkg.debug, CSCDCCEventData.debug, DTT0CalibrationRMS.debug, DTTTrigWriter.debug, DTTPDeadWriter.debug, PPSAlignmentConfigurationESSource.debug, DTOccupancyPoint.debug, DTNoiseComputation.debug, DTROS25Unpacker.debug, util.rrapi.RRApi.debug, DTT0Calibration.debug, DTTTrigSyncT0Only.debug, DTTTrigCalibration.debug, DTEfficiencyTask.debug, CSCDDUEventData.debug, DTRunConditionVar.debug, HCALResponse.debug, DTChamberEfficiencyTask.debug, DTChamberEfficiency.debug, CSCComparatorData.debug, CSCDCCUnpacker.debug, DTTMax.debug, DTVDriftCalibration.debug, DTTTrigSyncFromDB.debug, CSCTMBData.debug, DTTTrigSyncTOFCorr.debug, CSCEventData.debug, BaseParticlePropagator.debug, CSCTMBHeader.debug, CSCALCTTrailer.debug, CSCALCTHeader.debug, crabFunctions.CrabTask.debug, utils.functor(), and pkg.AbstractPkg.parse_etags().

Referenced by pkg.AbstractPkg.generate().

115  def write(self, fname, tmpl_name, kwds):
116  "Create new file from given template name and set of arguments"
117  code = ""
118  read_code = False
119  if os.path.exists(fname):
120  return
121  with open(fname, 'w') as stream:
122  for line in open(tmpl_name, 'r').readlines():
123  line = self.parse_etags(line)
124  if not line:
125  continue
126  if line.find('#python_begin') != -1:
127  read_code = True
128  continue
129  if line.find('#python_end') != -1:
130  read_code = False
131  if read_code:
132  code += line
133  if code and not read_code:
134  res = functor(code, kwds, self.debug)
135  stream.write(res)
136  code = ""
137  continue
138  if not read_code:
139  for key, val in kwds.items():
140  if isinstance(val, str):
141  line = line.replace(key, val)
142  stream.write(line)
def write
Definition: pkg.py:114
def parse_etags
Definition: pkg.py:97
def functor
Definition: utils.py:70

Member Data Documentation

pkg.AbstractPkg.author

Definition at line 49 of file pkg.py.

Referenced by pkg.AbstractPkg.get_kwds().

pkg.AbstractPkg.config

Definition at line 42 of file pkg.py.

Referenced by zMuMuValidation.ZMuMuValidation.trackcollection().

pkg.AbstractPkg.date

Definition at line 50 of file pkg.py.

Referenced by pkg.AbstractPkg.get_kwds().

pkg.AbstractPkg.debug

Definition at line 47 of file pkg.py.

Referenced by rrapi.RRApi.dprint(), pkg.AbstractPkg.generate(), rrapi.RRApi.get(), pkg.AbstractPkg.get_kwds(), runTauIdMVA.TauIDEmbedder.loadMVA_WPs_run2_2017(), runTauIdMVA.TauIDEmbedder.runTauID(), and pkg.AbstractPkg.write().

pkg.AbstractPkg.not_in_dir

Definition at line 51 of file pkg.py.

Referenced by pkg.AbstractPkg.generate().

pkg.AbstractPkg.pname

Definition at line 45 of file pkg.py.

Referenced by pkg.AbstractPkg.generate(), and pkg.AbstractPkg.get_kwds().

pkg.AbstractPkg.tdir

Definition at line 48 of file pkg.py.

Referenced by pkg.AbstractPkg.generate(), pkg.AbstractPkg.tmpl_etags(), and pkg.AbstractPkg.tmpl_tags().

pkg.AbstractPkg.tmpl

Definition at line 46 of file pkg.py.

Referenced by pkg.AbstractPkg.generate(), pkg.AbstractPkg.tmpl_etags(), and pkg.AbstractPkg.tmpl_tags().

pkg.AbstractPkg.working_dir

Definition at line 52 of file pkg.py.

Referenced by pkg.AbstractPkg.generate().