CMS 3D CMS Logo

List of all members | Public Member Functions
production_tasks.CheckJobStatus Class Reference
Inheritance diagram for production_tasks.CheckJobStatus:
production_tasks.Task

Public Member Functions

def __init__ (self, dataset, user, options)
 
def addOption (self, parser)
 
def run (self, input)
 
- Public Member Functions inherited from production_tasks.Task
def __init__ (self, name, dataset, user, options, instance=None)
 
def addOption (self, parser)
 
def getname (self)
 
def run (self, input)
 

Additional Inherited Members

- Public Attributes inherited from production_tasks.Task
 dataset
 
 instance
 
 name
 
 options
 
 user
 

Detailed Description

Checks the job STDOUT to catch common problems like exceptions, CPU time exceeded. Sets the job status in the report accordingly.

Definition at line 736 of file production_tasks.py.

Constructor & Destructor Documentation

def production_tasks.CheckJobStatus.__init__ (   self,
  dataset,
  user,
  options 
)

Definition at line 738 of file production_tasks.py.

738  def __init__(self, dataset, user, options):
739  Task.__init__(self,'CheckJobStatus', dataset, user, options)
def __init__(self, dataset, user, options)

Member Function Documentation

def production_tasks.CheckJobStatus.addOption (   self,
  parser 
)

Definition at line 740 of file production_tasks.py.

740  def addOption(self, parser):
741  parser.add_option("--output_wildcard", dest="output_wildcard", help="The wildcard to use when testing the output of this production (defaults to same as -w)", default=None)
def production_tasks.CheckJobStatus.run (   self,
  input 
)

Definition at line 742 of file production_tasks.py.

References FrontierConditions_GlobalTag_cff.file, FileExportPlugin.FileExportPlugin.options, cmsswPreprocessor.CmsswPreprocessor.options, DTCalibrationWorker.DTCalibrationWorker.options, DTWorkflow.DTWorkflow.options, DOTExport.DotProducer.options, TestProcess.TestProcess.options, confdb.HLTProcess.options, production_tasks.Task.options, edmIntegrityCheck.IntegrityCheck.options, and validateAlignments.ValidationJobMultiIOV.options.

742  def run(self, input):
743 
744  job_status = input['MonitorJobs']['LSFJobStatus']
745 
746  result = {}
747  for j, status in six.iteritems(job_status):
748  valid = True
749  if os.path.exists(status):
750 
751  fileHandle = None
752  if status.endswith('.gz') or status.endswith('.GZ'):
753  fileHandle = gzip.GzipFile(status)
754  else:
755  fileHandle = file(status)
756 
757  open_count = 0
758  close_count = 0
759  for line in fileHandle:
760  #start by counting files opened and closed
761  #suggestion from Enrique
762  if 'pened file' in line:
763  open_count += 1
764  if 'losed file' in line:
765  close_count += 1
766 
767  if 'Exception' in line:
768  result[j] = 'Exception'
769  valid = False
770  break
771  elif 'CPU time limit exceeded' in line:
772  result[j] = 'CPUTimeExceeded'
773  valid = False
774  break
775  elif 'Killed' in line:
776  result[j] = 'JobKilled'
777  valid = False
778  break
779  elif 'A fatal system signal has occurred' in line:
780  result[j] = 'SegFault'
781  valid = False
782  break
783 
784  if valid and open_count != close_count:
785  result[j] = 'FileOpenCloseMismatch'
786  valid = False
787  if valid:
788  result[j] = 'VALID'
789  else:
790  result[j] = status
791 
792  #allows a different wildcard in the final check.
793  options = copy.deepcopy(self.options)
794  if self.options.output_wildcard is not None:
795  options.wildcard = self.options.output_wildcard
796 
797  mask = GenerateMask(input['RunCMSBatch']['SampleDataset'],self.options.batch_user,options)
798  report = mask.run({'CheckForMask':{'MaskPresent':False}})
799  report['LSFJobStatusCheck'] = result
800  return report