CMS 3D CMS Logo

List of all members | Public Member Functions
production_tasks.CleanJobFiles Class Reference
Inheritance diagram for production_tasks.CleanJobFiles:
production_tasks.Task

Public Member Functions

def __init__ (self, dataset, user, options)
 
def run (self, input)
 
- Public Member Functions inherited from production_tasks.Task
def __init__ (self, name, dataset, user, options, instance=None)
 
def addOption (self, parser)
 
def getname (self)
 
def run (self, input)
 

Additional Inherited Members

- Public Attributes inherited from production_tasks.Task
 dataset
 
 instance
 
 name
 
 options
 
 user
 

Detailed Description

Removes and compresses auto-generated files from the job directory to save space.

Definition at line 845 of file production_tasks.py.

Constructor & Destructor Documentation

def production_tasks.CleanJobFiles.__init__ (   self,
  dataset,
  user,
  options 
)

Definition at line 847 of file production_tasks.py.

847  def __init__(self, dataset, user, options):
848  Task.__init__(self,'CleanJobFiles', dataset, user, options)
def __init__(self, dataset, user, options)

Member Function Documentation

def production_tasks.CleanJobFiles.run (   self,
  input 
)

Definition at line 849 of file production_tasks.py.

References mps_setup.append, production_tasks.Task.dataset, edmIntegrityCheck.IntegrityCheck.dataset, genericValidation.GenericValidationData.dataset, FileExportPlugin.FileExportPlugin.options, cmsswPreprocessor.CmsswPreprocessor.options, DTCalibrationWorker.DTCalibrationWorker.options, DTWorkflow.DTWorkflow.options, DOTExport.DotProducer.options, TestProcess.TestProcess.options, confdb.HLTProcess.options, production_tasks.Task.options, edmIntegrityCheck.IntegrityCheck.options, validateAlignments.ValidationJobMultiIOV.options, EcalTPGParamReaderFromDB.user, dataset.BaseDataset.user, production_tasks.Task.user, popcon::RpcObGasData.user, popcon::RpcDataT.user, popcon::RPCObPVSSmapData.user, popcon::RpcDataV.user, popcon::RpcDataGasMix.user, popcon::RpcDataUXC.user, popcon::RpcDataI.user, popcon::RpcDataFebmap.user, popcon::RpcDataS.user, MatrixInjector.MatrixInjector.user, and EcalDBConnection.user.

849  def run(self, input):
850 
851  jobdir = input['CreateJobDirectory']['JobDir']
852  jobs = input['MonitorJobs']['LSFJobIDs']
853  job_status = input['MonitorJobs']['LSFJobStatus']
854 
855  actions = {'FilesToCompress':{'Files':[]},'FilesToClean':{'Files':[]}}
856 
857  actions['FilesToClean']['Files'].append(input['ExpandConfig']['ExpandedFullCFG'])
858  if 'RunTestEvents' in input:
859  actions['FilesToClean']['Files'].append(input['RunTestEvents']['TestCFG'])
860 
861  for rt in glob.iglob('%s/*.root' % jobdir):
862  actions['FilesToClean']['Files'].append(rt)
863  for pyc in glob.iglob('%s/*.pyc' % jobdir):
864  actions['FilesToClean']['Files'].append(pyc)
865 
866  for j in jobs:
867  status = job_status[j]
868  if os.path.exists(status) and not status.endswith('.gz'):
869  actions['FilesToCompress']['Files'].append(status)
870 
871  compress = GZipFiles(self.dataset,self.user,self.options)
872  compressed = compress.run(actions)
873 
874  clean = CleanFiles(self.dataset,self.user,self.options)
875  removed = clean.run(actions)
876  return {'Cleaned':removed,'Compressed':compressed}
877