CMS 3D CMS Logo
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Properties Friends Macros Pages
List of all members | Public Member Functions | Private Attributes | Static Private Attributes
cmsswVersionTools.PickRelValInputFiles Class Reference

Automatic pick-up of RelVal input files

More...

Inheritance diagram for cmsswVersionTools.PickRelValInputFiles:

Public Member Functions

def __call__
 
def __init__
 
def apply
 
def getDefaultParameters
 
def messageEmptyList
 

Private Attributes

 _comment
 
 _parameters
 

Static Private Attributes

tuple _defaultParameters = dicttypes.SortedKeysDict()
 
string _label = 'pickRelValInputFiles'
 

Detailed Description

Automatic pick-up of RelVal input files 

Picks up RelVal input files automatically and
  returns a vector of strings with the paths to be used in [PoolSource].fileNames
PickRelValInputFiles( cmsswVersion, relVal, dataTier, condition, globalTag, maxVersions, skipFiles, numberOfFiles, debug )
- useDAS       : switch to perform query in DAS rather than in DBS
                 optional; default: False
- cmsswVersion : CMSSW release to pick up the RelVal files from
                 optional; default: the current release (determined automatically from environment)
- formerVersion: use the last before the last valid CMSSW release to pick up the RelVal files from
                 applies also, if 'cmsswVersion' is set explicitly
                 optional; default: False
- relVal       : RelVal sample to be used
                 optional; default: 'RelValTTbar'
- dataTier     : data tier to be used
                 optional; default: 'GEN-SIM-RECO'
- condition    : identifier of GlobalTag as defined in Configurations/PyReleaseValidation/python/autoCond.py
                 possibly overwritten, if 'globalTag' is set explicitly
                 optional; default: 'startup'
- globalTag    : name of GlobalTag as it is used in the data path of the RelVals
                 optional; default: determined automatically as defined by 'condition' in Configurations/PyReleaseValidation/python/autoCond.py
  !!!            Determination is done for the release one runs in, not for the release the RelVals have been produced in.
  !!!            Example of deviation: data RelVals (CMSSW_4_1_X) might not only have the pure name of the GlobalTag 'GR_R_311_V2' in the full path,
                 but also an extension identifying the data: 'GR_R_311_V2_RelVal_wzMu2010B'
- maxVersions  : max. versioning number of RelVal to check
                 optional; default: 9
- skipFiles    : number of files to skip for a found RelVal sample
                 optional; default: 0
- numberOfFiles: number of files to pick up
                 setting it to negative values, returns all found ('skipFiles' remains active though)
                 optional; default: -1
- debug        : switch to enable enhanced messages in 'stdout'
                 optional; default: False

Definition at line 55 of file cmsswVersionTools.py.

Constructor & Destructor Documentation

def cmsswVersionTools.PickRelValInputFiles.__init__ (   self)

Definition at line 95 of file cmsswVersionTools.py.

References editorTools.UserCodeTool._defaultParameters, cmsswVersionTools.Run52xOn51xTrigger._defaultParameters, Vispa.Plugins.ConfigEditor.ToolDataAccessor.ImportTool._defaultParameters, editorTools.ChangeSource._defaultParameters, cmsswVersionTools.PickRelValInputFiles._defaultParameters, root::helper::RootFunctionAdapter< F, args >.addParameter(), root::helper::RootFunctionHelper< F, args, Tag >.addParameter(), ParameterMgr.addParameter(), fit::RootMinuit< Function >.addParameter(), DropBoxMetadata::Parameters.addParameter(), PythonParameterSet.addParameter(), LutXml.addParameter(), ConfigToolBase.ConfigToolBase.addParameter(), edm::ParameterSet.addParameter(), cmsswVersionTools.Run52xOn51xTrigger.getDefaultParameters(), editorTools.ChangeSource.getDefaultParameters(), and cmsswVersionTools.PickRelValInputFiles.getDefaultParameters().

95 
96  def __init__( self ):
97  ConfigToolBase.__init__( self )
98  self.addParameter( self._defaultParameters, 'useDAS' , False , '' )
99  self.addParameter( self._defaultParameters, 'cmsswVersion' , os.getenv( "CMSSW_VERSION" ) , 'auto from environment' )
100  self.addParameter( self._defaultParameters, 'formerVersion', False , '' )
101  self.addParameter( self._defaultParameters, 'relVal' , 'RelValTTbar' , '' )
102  self.addParameter( self._defaultParameters, 'dataTier' , 'GEN-SIM-RECO' , '' )
103  self.addParameter( self._defaultParameters, 'condition' , 'startup' , '' )
104  self.addParameter( self._defaultParameters, 'globalTag' , autoCond[ self.getDefaultParameters()[ 'condition' ].value ][ : -5 ], 'auto from \'condition\'' )
105  self.addParameter( self._defaultParameters, 'maxVersions' , 3 , '' )
106  self.addParameter( self._defaultParameters, 'skipFiles' , 0 , '' )
107  self.addParameter( self._defaultParameters, 'numberOfFiles', -1 , 'all' )
108  self.addParameter( self._defaultParameters, 'debug' , False , '' )
109  self._parameters = copy.deepcopy( self._defaultParameters )
110  self._comment = ""

Member Function Documentation

def cmsswVersionTools.PickRelValInputFiles.__call__ (   self,
  useDAS = None,
  cmsswVersion = None,
  formerVersion = None,
  relVal = None,
  dataTier = None,
  condition = None,
  globalTag = None,
  maxVersions = None,
  skipFiles = None,
  numberOfFiles = None,
  debug = None 
)

Definition at line 123 of file cmsswVersionTools.py.

References gen::PhotosInterfaceBase.apply(), SurveyParameters.apply(), FWPSetCellEditor.apply(), gen::PhotosInterface.apply(), gen::PhotosInterface53XLegacy.apply(), cmsswVersionTools.Run52xOn51xTrigger.apply(), FWGUIEventFilter.apply(), DeviationsFromFileSensor2D.apply(), BeamSpotAlignmentParameters.apply(), RigidBodyAlignmentParameters.apply(), ConfigToolBase.ConfigToolBase.apply(), AlignmentParameters.apply(), BowedSurfaceAlignmentParameters.apply(), Vispa.Plugins.ConfigEditor.ToolDataAccessor.ImportTool.apply(), Vispa.Plugins.ConfigEditor.ToolDataAccessor.ApplyTool.apply(), TwoBowedSurfacesAlignmentParameters.apply(), Vispa.Gui.BoxContentDialog.BoxContentDialog.apply(), Vispa.Plugins.ConfigEditor.ToolDialog.ToolDialog.apply(), cmsswVersionTools.PickRelValInputFiles.apply(), cmsswVersionTools.Run52xOn51xTrigger.getDefaultParameters(), editorTools.ChangeSource.getDefaultParameters(), cmsswVersionTools.PickRelValInputFiles.getDefaultParameters(), ora::OraMainTable.setParameter(), ora::IMainTable.setParameter(), PedeReader.setParameter(), ora::PoolMainTable.setParameter(), fit::RootMinuit< Function >.setParameter(), and ConfigToolBase.ConfigToolBase.setParameter().

123 
124  ):
125  if useDAS is None:
126  useDAS = self.getDefaultParameters()[ 'useDAS' ].value
127  if cmsswVersion is None:
128  cmsswVersion = self.getDefaultParameters()[ 'cmsswVersion' ].value
129  if formerVersion is None:
130  formerVersion = self.getDefaultParameters()[ 'formerVersion' ].value
131  if relVal is None:
132  relVal = self.getDefaultParameters()[ 'relVal' ].value
133  if dataTier is None:
134  dataTier = self.getDefaultParameters()[ 'dataTier' ].value
135  if condition is None:
136  condition = self.getDefaultParameters()[ 'condition' ].value
137  if globalTag is None:
138  globalTag = autoCond[ condition ][ : -5 ] # auto from 'condition'
139  if maxVersions is None:
140  maxVersions = self.getDefaultParameters()[ 'maxVersions' ].value
141  if skipFiles is None:
142  skipFiles = self.getDefaultParameters()[ 'skipFiles' ].value
143  if numberOfFiles is None:
144  numberOfFiles = self.getDefaultParameters()[ 'numberOfFiles' ].value
145  if debug is None:
146  debug = self.getDefaultParameters()[ 'debug' ].value
147  self.setParameter( 'useDAS' , useDAS )
148  self.setParameter( 'cmsswVersion' , cmsswVersion )
149  self.setParameter( 'formerVersion', formerVersion )
150  self.setParameter( 'relVal' , relVal )
151  self.setParameter( 'dataTier' , dataTier )
152  self.setParameter( 'condition' , condition )
153  self.setParameter( 'globalTag' , globalTag )
154  self.setParameter( 'maxVersions' , maxVersions )
155  self.setParameter( 'skipFiles' , skipFiles )
156  self.setParameter( 'numberOfFiles', numberOfFiles )
157  self.setParameter( 'debug' , debug )
158  return self.apply()
def cmsswVersionTools.PickRelValInputFiles.apply (   self)

Definition at line 163 of file cmsswVersionTools.py.

References editorTools.UserCodeTool._label, MagneticFieldMapESProducer._label, TrackerInteractionGeometryESProducer._label, Vispa.Plugins.ConfigEditor.ConfigDataAccessor.ConfigFolder._label, cmsswVersionTools.Run52xOn51xTrigger._label, Vispa.Plugins.ConfigEditor.ToolDataAccessor.ImportTool._label, ConfigToolBase.ConfigToolBase._label, editorTools.ChangeSource._label, LHECOMWeightProducer._label, Vispa.Plugins.ConfigEditor.ToolDataAccessor.ApplyTool._label, cmsswVersionTools.PickRelValInputFiles._label, Vispa.Views.LineDecayView.DecayLine._label, editorTools.UserCodeTool._parameters, Vispa.Plugins.ConfigEditor.ConfigDataAccessor.ConfigFolder._parameters, cmsswVersionTools.Run52xOn51xTrigger._parameters, Vispa.Plugins.ConfigEditor.ToolDataAccessor.ImportTool._parameters, ConfigToolBase.ConfigToolBase._parameters, editorTools.ChangeSource._parameters, TAbsFitConstraint._parameters, TAbsFitParticle._parameters, cmsswVersionTools.PickRelValInputFiles._parameters, communicate(), cmsRelvalreport.exit, cmsswVersionTools.Run52xOn51xTrigger.getDefaultParameters(), editorTools.ChangeSource.getDefaultParameters(), cmsswVersionTools.PickRelValInputFiles.getDefaultParameters(), cmsswVersionTools.PickRelValInputFiles.messageEmptyList(), and split.

Referenced by electronTools.AddElectronUserIsolation.__call__(), photonTools.AddPhotonUserIsolation.__call__(), heavyIonTools.ConfigureHeavyIons.__call__(), metTools.AddTcMET.__call__(), muonTools.AddMuonUserIsolation.__call__(), editorTools.UserCodeTool.__call__(), coreTools.RestrictInputToAOD.__call__(), HiCoreTools.RestrictInputToAOD.__call__(), trackTools.MakeAODTrackCandidates.__call__(), heavyIonTools.ProductionDefaults.__call__(), editorTools.ChangeSource.__call__(), metTools.AddPfMET.__call__(), HiCoreTools.RemoveMCMatching.__call__(), coreTools.RunOnData.__call__(), trackTools.MakePATTrackCandidates.__call__(), trigTools.SwitchOnTrigger.__call__(), cmsswVersionTools.PickRelValInputFiles.__call__(), coreTools.RemoveMCMatching.__call__(), heavyIonTools.SelectionDefaults.__call__(), HiCoreTools.RemoveAllPATObjectsBut.__call__(), heavyIonTools.DisbaleMonteCarloDeps.__call__(), jetTools.RunBTagging.__call__(), HiCoreTools.RemoveSpecificPATObjects.__call__(), trigTools.SwitchOnTriggerStandAlone.__call__(), coreTools.RemoveAllPATObjectsBut.__call__(), trackTools.MakeTrackCandidates.__call__(), coreTools.RemoveSpecificPATObjects.__call__(), HiCoreTools.RemoveCleaning.__call__(), trigTools.SwitchOnTriggerMatching.__call__(), HiCoreTools.AddCleaning.__call__(), coreTools.RemoveCleaning.__call__(), tauTools.AddTauCollection.__call__(), trigTools.SwitchOnTriggerMatchingStandAlone.__call__(), coreTools.AddCleaning.__call__(), jetTools.AddJetCollection.__call__(), trigTools.SwitchOnTriggerMatchEmbedding.__call__(), trigTools.RemoveCleaningFromTriggerMatching.__call__(), jetTools.SwitchJetCollection.__call__(), jetTools.AddJetID.__call__(), jetTools.SetTagInfos.__call__(), jetTools.SwitchJetCorrLevels.__call__(), and metUncertaintyTools.RunMEtUncertainties.__call__().

163 
164  def apply( self ):
165  useDAS = self._parameters[ 'useDAS' ].value
166  cmsswVersion = self._parameters[ 'cmsswVersion' ].value
167  formerVersion = self._parameters[ 'formerVersion' ].value
168  relVal = self._parameters[ 'relVal' ].value
169  dataTier = self._parameters[ 'dataTier' ].value
170  condition = self._parameters[ 'condition' ].value # only used for GT determination in initialization, if GT not explicitly given
171  globalTag = self._parameters[ 'globalTag' ].value
172  maxVersions = self._parameters[ 'maxVersions' ].value
173  skipFiles = self._parameters[ 'skipFiles' ].value
174  numberOfFiles = self._parameters[ 'numberOfFiles' ].value
175  debug = self._parameters[ 'debug' ].value
176 
177  filePaths = []
178 
179  # Determine corresponding CMSSW version for RelVals
180  preId = '_pre'
181  patchId = '_patch' # patch releases
182  hltPatchId = '_hltpatch' # HLT patch releases
183  dqmPatchId = '_dqmpatch' # DQM patch releases
184  slhcId = '_SLHC' # SLHC releases
185  rootId = '_root' # ROOT test releases
186  ibId = '_X_' # IBs
187  if patchId in cmsswVersion:
188  cmsswVersion = cmsswVersion.split( patchId )[ 0 ]
189  elif hltPatchId in cmsswVersion:
190  cmsswVersion = cmsswVersion.split( hltPatchId )[ 0 ]
191  elif dqmPatchId in cmsswVersion:
192  cmsswVersion = cmsswVersion.split( dqmPatchId )[ 0 ]
193  elif rootId in cmsswVersion:
194  cmsswVersion = cmsswVersion.split( rootId )[ 0 ]
195  elif slhcId in cmsswVersion:
196  cmsswVersion = cmsswVersion.split( slhcId )[ 0 ]
197  elif ibId in cmsswVersion or formerVersion:
198  outputTuple = Popen( [ 'scram', 'l -c CMSSW' ], stdout = PIPE, stderr = PIPE ).communicate()
199  if len( outputTuple[ 1 ] ) != 0:
200  print '%s INFO : SCRAM error'%( self._label )
201  if debug:
202  print ' from trying to determine last valid releases before \'%s\''%( cmsswVersion )
203  print
204  print outputTuple[ 1 ]
205  print
206  self.messageEmptyList()
207  return filePaths
208  versions = { 'last' :''
209  , 'lastToLast':''
210  }
211  for line in outputTuple[ 0 ].splitlines():
212  version = line.split()[ 1 ]
213  if cmsswVersion.split( ibId )[ 0 ] in version or cmsswVersion.rpartition( '_' )[ 0 ] in version:
214  if not ( patchId in version or hltPatchId in version or dqmPatchId in version or slhcId in version or ibId in version or rootId in version ):
215  versions[ 'lastToLast' ] = versions[ 'last' ]
216  versions[ 'last' ] = version
217  if version == cmsswVersion:
218  break
219  # FIXME: ordering of output problematic ('XYZ_pre10' before 'XYZ_pre2', no "formerVersion" for 'XYZ_pre1')
220  if formerVersion:
221  # Don't use pre-releases as "former version" for other releases than CMSSW_X_Y_0
222  if preId in versions[ 'lastToLast' ] and not preId in versions[ 'last' ] and not versions[ 'last' ].endswith( '_0' ):
223  versions[ 'lastToLast' ] = versions[ 'lastToLast' ].split( preId )[ 0 ] # works only, if 'CMSSW_X_Y_0' esists ;-)
224  # Use pre-release as "former version" for CMSSW_X_Y_0
225  elif versions[ 'last' ].endswith( '_0' ) and not ( preId in versions[ 'lastToLast' ] and versions[ 'lastToLast' ].startswith( versions[ 'last' ] ) ):
226  versions[ 'lastToLast' ] = ''
227  for line in outputTuple[ 0 ].splitlines():
228  version = line.split()[ 1 ]
229  versionParts = version.partition( preId )
230  if versionParts[ 0 ] == versions[ 'last' ] and versionParts[ 1 ] == preId:
231  versions[ 'lastToLast' ] = version
232  elif versions[ 'lastToLast' ] != '':
233  break
234  # Don't use CMSSW_X_Y_0 as "former version" for pre-releases
235  elif preId in versions[ 'last' ] and not preId in versions[ 'lastToLast' ] and versions[ 'lastToLast' ].endswith( '_0' ):
236  versions[ 'lastToLast' ] = '' # no alternative :-(
237  cmsswVersion = versions[ 'lastToLast' ]
238  else:
239  cmsswVersion = versions[ 'last' ]
240 
241  # Debugging output
242  if debug:
243  print '%s DEBUG: Called with...'%( self._label )
244  for key in self._parameters.keys():
245  print ' %s:\t'%( key ),
246  print self._parameters[ key ].value,
247  if self._parameters[ key ].value is self.getDefaultParameters()[ key ].value:
248  print ' (default)'
249  else:
250  print
251  if key == 'cmsswVersion' and cmsswVersion != self._parameters[ key ].value:
252  if formerVersion:
253  print ' ==> modified to last to last valid release %s (s. \'formerVersion\' parameter)'%( cmsswVersion )
254  else:
255  print ' ==> modified to last valid release %s'%( cmsswVersion )
256 
257  # Check domain
258  domain = socket.getfqdn().split( '.' )
259  domainSE = ''
260  if len( domain ) == 0:
261  print '%s INFO : Cannot determine domain of this computer'%( self._label )
262  if debug:
263  self.messageEmptyList()
264  return filePaths
265  elif os.uname()[0] == "Darwin":
266  print '%s INFO : Running on MacOSX without direct access to RelVal files.'%( self._label )
267  if debug:
268  self.messageEmptyList()
269  return filePaths
270  elif len( domain ) == 1:
271  print '%s INFO : Running on local host \'%s\' without direct access to RelVal files'%( self._label, domain[ 0 ] )
272  if debug:
273  self.messageEmptyList()
274  return filePaths
275  if not ( ( domain[ -2 ] == 'cern' and domain[ -1 ] == 'ch' ) or ( domain[ -2 ] == 'fnal' and domain[ -1 ] == 'gov' ) ):
276  print '%s INFO : Running on site \'%s.%s\' without direct access to RelVal files'%( self._label, domain[ -2 ], domain[ -1 ] )
277  if debug:
278  self.messageEmptyList()
279  return filePaths
280  if domain[ -2 ] == 'cern':
281  domainSE = 'T2_CH_CERN'
282  elif domain[ -2 ] == 'fnal':
283  domainSE = 'T1_US_FNAL_MSS'
284  if debug:
285  print '%s DEBUG: Running at site \'%s.%s\''%( self._label, domain[ -2 ], domain[ -1 ] )
286  print '%s DEBUG: Looking for SE \'%s\''%( self._label, domainSE )
287 
288  # Find files
289  validVersion = 0
290  dataset = ''
291  datasetAll = '/%s/%s-%s-v*/%s'%( relVal, cmsswVersion, globalTag, dataTier )
292  if useDAS:
293  if debug:
294  print '%s DEBUG: Using DAS query'%( self._label )
295  dasLimit = numberOfFiles
296  if dasLimit <= 0:
297  dasLimit += 1
298  for version in range( maxVersions, 0, -1 ):
299  filePaths = []
300  filePathsTmp = []
301  fileCount = 0
302  dataset = '/%s/%s-%s-v%i/%s'%( relVal, cmsswVersion, globalTag, version, dataTier )
303  dasQuery = 'file dataset=%s | grep file.name'%( dataset )
304  if debug:
305  print '%s DEBUG: Querying dataset \'%s\' with'%( self._label, dataset )
306  print ' \'%s\''%( dasQuery )
307  # partially stolen from das_client.py for option '--format=plain', needs filter ("grep") in the query
308  jsondict = das_client.get_data( 'https://cmsweb.cern.ch', dasQuery, 0, dasLimit, False )
309  if debug:
310  print '%s DEBUG: Received DAS JSON dictionary:'%( self._label )
311  print ' \'%s\''%( jsondict )
312  if jsondict[ 'status' ] != 'ok':
313  print 'There was a problem while querying DAS with query \'%s\'. Server reply was:\n %s' % (dasQuery, jsondict)
314  exit( 1 )
315  mongo_query = jsondict[ 'mongo_query' ]
316  filters = mongo_query[ 'filters' ]
317  data = jsondict[ 'data' ]
318  if debug:
319  print '%s DEBUG: Query in JSON dictionary:'%( self._label )
320  print ' \'%s\''%( mongo_query )
321  print '%s DEBUG: Filters in query:'%( self._label )
322  print ' \'%s\''%( filters )
323  print '%s DEBUG: Data in JSON dictionary:'%( self._label )
324  print ' \'%s\''%( data )
325  for row in data:
326  filePath = [ r for r in das_client.get_value( row, filters[ 'grep' ] ) ][ 0 ]
327  if debug:
328  print '%s DEBUG: Testing file entry \'%s\''%( self._label, filePath )
329  if len( filePath ) > 0:
330  if validVersion != version:
331  jsontestdict = das_client.get_data( 'https://cmsweb.cern.ch', 'site dataset=%s | grep site.name'%( dataset ), 0, 999, False )
332  mongo_testquery = jsontestdict[ 'mongo_query' ]
333  testfilters = mongo_testquery[ 'filters' ]
334  testdata = jsontestdict[ 'data' ]
335  if debug:
336  print '%s DEBUG: Received DAS JSON dictionary (site test):'%( self._label )
337  print ' \'%s\''%( jsontestdict )
338  print '%s DEBUG: Query in JSON dictionary (site test):'%( self._label )
339  print ' \'%s\''%( mongo_testquery )
340  print '%s DEBUG: Filters in query (site test):'%( self._label )
341  print ' \'%s\''%( testfilters )
342  print '%s DEBUG: Data in JSON dictionary (site test):'%( self._label )
343  print ' \'%s\''%( testdata )
344  foundSE = False
345  for testrow in testdata:
346  siteName = [ tr for tr in das_client.get_value( testrow, testfilters[ 'grep' ] ) ][ 0 ]
347  if siteName == domainSE:
348  foundSE = True
349  break
350  if not foundSE:
351  if debug:
352  print '%s DEBUG: Possible version \'v%s\' not available on SE \'%s\''%( self._label, version, domainSE )
353  break
354  validVersion = version
355  if debug:
356  print '%s DEBUG: Valid version set to \'v%i\''%( self._label, validVersion )
357  if numberOfFiles == 0:
358  break
359  # protect from double entries ( 'unique' flag in query does not work here)
360  if not filePath in filePathsTmp:
361  filePathsTmp.append( filePath )
362  if debug:
363  print '%s DEBUG: File \'%s\' found'%( self._label, filePath )
364  fileCount += 1
365  # needed, since and "limit" overrides "idx" in 'get_data' (==> "idx" set to '0' rather than "skipFiles")
366  if fileCount > skipFiles:
367  filePaths.append( filePath )
368  elif debug:
369  print '%s DEBUG: File \'%s\' found again'%( self._label, filePath )
370  if validVersion > 0:
371  if numberOfFiles == 0 and debug:
372  print '%s DEBUG: No files requested'%( self._label )
373  break
374  else:
375  if debug:
376  print '%s DEBUG: Using DBS query'%( self._label )
377  print '%s WARNING: DBS query disabled for DBS3 transition to new API'%( self._label )
378  #for version in range( maxVersions, 0, -1 ):
379  #filePaths = []
380  #fileCount = 0
381  #dataset = '/%s/%s-%s-v%i/%s'%( relVal, cmsswVersion, globalTag, version, dataTier )
382  #dbsQuery = 'find file where dataset = %s'%( dataset )
383  #if debug:
384  #print '%s DEBUG: Querying dataset \'%s\' with'%( self._label, dataset )
385  #print ' \'%s\''%( dbsQuery )
386  #foundSE = False
387  #for line in os.popen( 'dbs search --query="%s"'%( dbsQuery ) ).readlines():
388  #if line.find( '.root' ) != -1:
389  #if validVersion != version:
390  #if not foundSE:
391  #dbsSiteQuery = 'find dataset where dataset = %s and site = %s'%( dataset, domainSE )
392  #if debug:
393  #print '%s DEBUG: Querying site \'%s\' with'%( self._label, domainSE )
394  #print ' \'%s\''%( dbsSiteQuery )
395  #for lineSite in os.popen( 'dbs search --query="%s"'%( dbsSiteQuery ) ).readlines():
396  #if lineSite.find( dataset ) != -1:
397  #foundSE = True
398  #break
399  #if not foundSE:
400  #if debug:
401  #print '%s DEBUG: Possible version \'v%s\' not available on SE \'%s\''%( self._label, version, domainSE )
402  #break
403  #validVersion = version
404  #if debug:
405  #print '%s DEBUG: Valid version set to \'v%i\''%( self._label, validVersion )
406  #if numberOfFiles == 0:
407  #break
408  #filePath = line.replace( '\n', '' )
409  #if debug:
410  #print '%s DEBUG: File \'%s\' found'%( self._label, filePath )
411  #fileCount += 1
412  #if fileCount > skipFiles:
413  #filePaths.append( filePath )
414  #if not numberOfFiles < 0:
415  #if numberOfFiles <= len( filePaths ):
416  #break
417  #if validVersion > 0:
418  #if numberOfFiles == 0 and debug:
419  #print '%s DEBUG: No files requested'%( self._label )
420  #break
421 
422  # Check output and return
423  if validVersion == 0:
424  print '%s WARNING : No RelVal file(s) found at all in datasets \'%s*\' on SE \'%s\''%( self._label, datasetAll, domainSE )
425  if debug:
426  self.messageEmptyList()
427  elif len( filePaths ) == 0:
428  print '%s WARNING : No RelVal file(s) picked up in dataset \'%s\''%( self._label, dataset )
429  if debug:
430  self.messageEmptyList()
431  elif len( filePaths ) < numberOfFiles:
432  print '%s INFO : Only %i RelVal file(s) instead of %i picked up in dataset \'%s\''%( self._label, len( filePaths ), numberOfFiles, dataset )
433 
434  if debug:
435  print '%s DEBUG: returning %i file(s):\n%s'%( self._label, len( filePaths ), filePaths )
436  return filePaths
communicate
static void * communicate(void *obj)
Definition: DQMNet.cc:1246
split
double split
Definition: MVATrainer.cc:139
def cmsswVersionTools.PickRelValInputFiles.getDefaultParameters (   self)

Definition at line 92 of file cmsswVersionTools.py.

References editorTools.UserCodeTool._defaultParameters, cmsswVersionTools.Run52xOn51xTrigger._defaultParameters, Vispa.Plugins.ConfigEditor.ToolDataAccessor.ImportTool._defaultParameters, editorTools.ChangeSource._defaultParameters, and cmsswVersionTools.PickRelValInputFiles._defaultParameters.

Referenced by cmsswVersionTools.PickRelValInputFiles.__call__(), cmsswVersionTools.PickRelValInputFiles.__init__(), cmsswVersionTools.PickRelValInputFiles.apply(), trigTools.SwitchOnTrigger.toolCode(), trigTools.SwitchOnTriggerStandAlone.toolCode(), trigTools.SwitchOnTriggerMatching.toolCode(), and trigTools.SwitchOnTriggerMatchingStandAlone.toolCode().

92 
93  def getDefaultParameters( self ):
94  return self._defaultParameters
def cmsswVersionTools.PickRelValInputFiles.messageEmptyList (   self)

Definition at line 159 of file cmsswVersionTools.py.

References editorTools.UserCodeTool._label, MagneticFieldMapESProducer._label, TrackerInteractionGeometryESProducer._label, Vispa.Plugins.ConfigEditor.ConfigDataAccessor.ConfigFolder._label, cmsswVersionTools.Run52xOn51xTrigger._label, Vispa.Plugins.ConfigEditor.ToolDataAccessor.ImportTool._label, editorTools.ChangeSource._label, ConfigToolBase.ConfigToolBase._label, LHECOMWeightProducer._label, Vispa.Plugins.ConfigEditor.ToolDataAccessor.ApplyTool._label, cmsswVersionTools.PickRelValInputFiles._label, and Vispa.Views.LineDecayView.DecayLine._label.

Referenced by cmsswVersionTools.PickRelValInputFiles.apply().

159 
160  def messageEmptyList( self ):
161  print '%s DEBUG: Empty file list returned'%( self._label )
162  print ' This might be overwritten by providing input files explicitly to the source module in the main configuration file.'

Member Data Documentation

cmsswVersionTools.PickRelValInputFiles._comment
private

Definition at line 109 of file cmsswVersionTools.py.

Referenced by metUncertaintyTools.RunMEtUncertainties.__init__(), and jetTools.SwitchJetCollection.__init__().

tuple cmsswVersionTools.PickRelValInputFiles._defaultParameters = dicttypes.SortedKeysDict()
staticprivate

Definition at line 90 of file cmsswVersionTools.py.

Referenced by electronTools.AddElectronUserIsolation.__call__(), photonTools.AddPhotonUserIsolation.__call__(), metTools.AddTcMET.__call__(), muonTools.AddMuonUserIsolation.__call__(), coreTools.RestrictInputToAOD.__call__(), HiCoreTools.RestrictInputToAOD.__call__(), trackTools.MakeAODTrackCandidates.__call__(), metTools.AddPfMET.__call__(), HiCoreTools.RemoveMCMatching.__call__(), coreTools.RunOnData.__call__(), trackTools.MakePATTrackCandidates.__call__(), trigTools.SwitchOnTrigger.__call__(), coreTools.RemoveMCMatching.__call__(), HiCoreTools.RemoveAllPATObjectsBut.__call__(), jetTools.RunBTagging.__call__(), HiCoreTools.RemoveSpecificPATObjects.__call__(), trigTools.SwitchOnTriggerStandAlone.__call__(), coreTools.RemoveAllPATObjectsBut.__call__(), trackTools.MakeTrackCandidates.__call__(), coreTools.RemoveSpecificPATObjects.__call__(), HiCoreTools.RemoveCleaning.__call__(), trigTools.SwitchOnTriggerMatching.__call__(), HiCoreTools.AddCleaning.__call__(), coreTools.RemoveCleaning.__call__(), tauTools.AddTauCollection.__call__(), trigTools.SwitchOnTriggerMatchingStandAlone.__call__(), coreTools.AddCleaning.__call__(), jetTools.AddJetCollection.__call__(), trigTools.SwitchOnTriggerMatchEmbedding.__call__(), trigTools.RemoveCleaningFromTriggerMatching.__call__(), jetTools.SwitchJetCollection.__call__(), jetTools.AddJetID.__call__(), jetTools.SetTagInfos.__call__(), jetTools.SwitchJetCorrLevels.__call__(), metUncertaintyTools.RunMEtUncertainties.__call__(), electronTools.AddElectronUserIsolation.__init__(), photonTools.AddPhotonUserIsolation.__init__(), trackTools.MakeAODTrackCandidates.__init__(), metTools.AddTcMET.__init__(), muonTools.AddMuonUserIsolation.__init__(), coreTools.RestrictInputToAOD.__init__(), HiCoreTools.RestrictInputToAOD.__init__(), metUncertaintyTools.RunMEtUncertainties.__init__(), HiCoreTools.RemoveMCMatching.__init__(), metTools.AddPfMET.__init__(), coreTools.RunOnData.__init__(), trackTools.MakePATTrackCandidates.__init__(), trigTools.SwitchOnTrigger.__init__(), cmsswVersionTools.PickRelValInputFiles.__init__(), coreTools.RemoveMCMatching.__init__(), jetTools.RunBTagging.__init__(), HiCoreTools.RemoveAllPATObjectsBut.__init__(), HiCoreTools.RemoveSpecificPATObjects.__init__(), trigTools.SwitchOnTriggerStandAlone.__init__(), trackTools.MakeTrackCandidates.__init__(), coreTools.RemoveAllPATObjectsBut.__init__(), coreTools.RemoveSpecificPATObjects.__init__(), HiCoreTools.RemoveCleaning.__init__(), trigTools.SwitchOnTriggerMatching.__init__(), HiCoreTools.AddCleaning.__init__(), coreTools.RemoveCleaning.__init__(), tauTools.AddTauCollection.__init__(), trigTools.SwitchOnTriggerMatchingStandAlone.__init__(), coreTools.AddCleaning.__init__(), jetTools.AddJetCollection.__init__(), trigTools.SwitchOnTriggerMatchEmbedding.__init__(), trigTools.RemoveCleaningFromTriggerMatching.__init__(), jetTools.SwitchJetCollection.__init__(), jetTools.AddJetID.__init__(), jetTools.SetTagInfos.__init__(), jetTools.SwitchJetCorrLevels.__init__(), metUncertaintyTools.RunMEtUncertainties._initializeInputTag(), ConfigToolBase.ConfigToolBase.dumpPython(), ConfigToolBase.ConfigToolBase.getAllowedValues(), photonTools.AddPhotonUserIsolation.getDefaultParameters(), electronTools.AddElectronUserIsolation.getDefaultParameters(), heavyIonTools.ConfigureHeavyIons.getDefaultParameters(), metTools.AddTcMET.getDefaultParameters(), muonTools.AddMuonUserIsolation.getDefaultParameters(), coreTools.RestrictInputToAOD.getDefaultParameters(), HiCoreTools.RestrictInputToAOD.getDefaultParameters(), trackTools.MakeAODTrackCandidates.getDefaultParameters(), heavyIonTools.ProductionDefaults.getDefaultParameters(), metTools.AddPfMET.getDefaultParameters(), HiCoreTools.RemoveMCMatching.getDefaultParameters(), coreTools.RunOnData.getDefaultParameters(), metUncertaintyTools.RunMEtUncertainties.getDefaultParameters(), trackTools.MakePATTrackCandidates.getDefaultParameters(), cmsswVersionTools.PickRelValInputFiles.getDefaultParameters(), trigTools.SwitchOnTrigger.getDefaultParameters(), coreTools.RemoveMCMatching.getDefaultParameters(), heavyIonTools.SelectionDefaults.getDefaultParameters(), HiCoreTools.RemoveAllPATObjectsBut.getDefaultParameters(), heavyIonTools.DisbaleMonteCarloDeps.getDefaultParameters(), jetTools.RunBTagging.getDefaultParameters(), HiCoreTools.RemoveSpecificPATObjects.getDefaultParameters(), trigTools.SwitchOnTriggerStandAlone.getDefaultParameters(), coreTools.RemoveAllPATObjectsBut.getDefaultParameters(), trackTools.MakeTrackCandidates.getDefaultParameters(), coreTools.RemoveSpecificPATObjects.getDefaultParameters(), HiCoreTools.RemoveCleaning.getDefaultParameters(), trigTools.SwitchOnTriggerMatching.getDefaultParameters(), HiCoreTools.AddCleaning.getDefaultParameters(), coreTools.RemoveCleaning.getDefaultParameters(), tauTools.AddTauCollection.getDefaultParameters(), trigTools.SwitchOnTriggerMatchingStandAlone.getDefaultParameters(), coreTools.AddCleaning.getDefaultParameters(), jetTools.AddJetCollection.getDefaultParameters(), trigTools.SwitchOnTriggerMatchEmbedding.getDefaultParameters(), trigTools.RemoveCleaningFromTriggerMatching.getDefaultParameters(), jetTools.SwitchJetCollection.getDefaultParameters(), jetTools.AddJetID.getDefaultParameters(), jetTools.SetTagInfos.getDefaultParameters(), jetTools.SwitchJetCorrLevels.getDefaultParameters(), ConfigToolBase.ConfigToolBase.reset(), and ConfigToolBase.ConfigToolBase.setParameter().

string cmsswVersionTools.PickRelValInputFiles._label = 'pickRelValInputFiles'
staticprivate

Definition at line 89 of file cmsswVersionTools.py.

Referenced by cmsswVersionTools.PickRelValInputFiles.apply(), cmsswVersionTools.PickRelValInputFiles.messageEmptyList(), trigTools.SwitchOnTrigger.toolCode(), trigTools.SwitchOnTriggerStandAlone.toolCode(), trigTools.SwitchOnTriggerMatching.toolCode(), trigTools.SwitchOnTriggerMatchingStandAlone.toolCode(), trigTools.SwitchOnTriggerMatchEmbedding.toolCode(), and trigTools.RemoveCleaningFromTriggerMatching.toolCode().

cmsswVersionTools.PickRelValInputFiles._parameters
private

Definition at line 108 of file cmsswVersionTools.py.

Referenced by coreTools.RemoveMCMatching.__call__(), metUncertaintyTools.RunMEtUncertainties.__init__(), jetTools.SwitchJetCollection.__init__(), cmsswVersionTools.PickRelValInputFiles.apply(), jetTools.RunBTagging.apply(), electronTools.AddElectronUserIsolation.toolCode(), photonTools.AddPhotonUserIsolation.toolCode(), metTools.AddTcMET.toolCode(), muonTools.AddMuonUserIsolation.toolCode(), coreTools.RestrictInputToAOD.toolCode(), HiCoreTools.RestrictInputToAOD.toolCode(), trackTools.MakeAODTrackCandidates.toolCode(), metTools.AddPfMET.toolCode(), HiCoreTools.RemoveMCMatching.toolCode(), coreTools.RunOnData.toolCode(), trackTools.MakePATTrackCandidates.toolCode(), trigTools.SwitchOnTrigger.toolCode(), coreTools.RemoveMCMatching.toolCode(), HiCoreTools.RemoveAllPATObjectsBut.toolCode(), HiCoreTools.RemoveSpecificPATObjects.toolCode(), trigTools.SwitchOnTriggerStandAlone.toolCode(), coreTools.RemoveAllPATObjectsBut.toolCode(), trackTools.MakeTrackCandidates.toolCode(), coreTools.RemoveSpecificPATObjects.toolCode(), HiCoreTools.RemoveCleaning.toolCode(), trigTools.SwitchOnTriggerMatching.toolCode(), HiCoreTools.AddCleaning.toolCode(), coreTools.RemoveCleaning.toolCode(), tauTools.AddTauCollection.toolCode(), coreTools.AddCleaning.toolCode(), trigTools.SwitchOnTriggerMatchingStandAlone.toolCode(), jetTools.AddJetCollection.toolCode(), trigTools.SwitchOnTriggerMatchEmbedding.toolCode(), trigTools.RemoveCleaningFromTriggerMatching.toolCode(), jetTools.SwitchJetCollection.toolCode(), jetTools.AddJetID.toolCode(), jetTools.SetTagInfos.toolCode(), jetTools.SwitchJetCorrLevels.toolCode(), and metUncertaintyTools.RunMEtUncertainties.toolCode().

Generated for CMSSW Reference Manual by doxygen 1.8.5