test
CMS 3D CMS Logo

 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Properties Friends Macros Pages
List of all members | Classes | Public Types | Public Member Functions | Protected Member Functions | Private Attributes
edm::JobReport Class Reference

#include <JobReport.h>

Inheritance diagram for edm::JobReport:
edm::service::JobReportService

Classes

class  AtomicLongLong
 
struct  InputFile
 
struct  JobReportImpl
 
struct  OutputFile
 
struct  RunReport
 

Public Types

typedef unsigned int RunNumber
 
typedef std::size_t Token
 

Public Member Functions

void childAfterFork (std::string const &jobReportFile, unsigned int childIndex, unsigned int numberOfChildren)
 New output file for child. More...
 
std::string dumpFiles (void)
 debug/test util More...
 
void eventReadFromFile (InputType inputType, Token fileToken)
 
void eventWrittenToFile (Token fileToken, unsigned int run, unsigned int event)
 
void inputFileClosed (InputType inputType, Token fileToken)
 
Token inputFileOpened (std::string const &physicalFileName, std::string const &logicalFileName, std::string const &catalog, std::string const &inputType, std::string const &inputSourceClassName, std::string const &moduleLabel, std::string const &guid, std::vector< std::string > const &branchNames)
 
 JobReport ()
 
 JobReport (std::ostream *outputStream)
 
 JobReport (JobReport const &)=delete
 
JobReportoperator= (JobReport const &)=delete
 
void outputFileClosed (Token fileToken)
 
Token outputFileOpened (std::string const &physicalFileName, std::string const &logicalFileName, std::string const &catalog, std::string const &outputModuleClassName, std::string const &moduleLabel, std::string const &guid, std::string const &dataType, std::string const &branchHash, std::vector< std::string > const &branchNames)
 
void parentAfterFork (std::string const &jobReportFile)
 
void parentBeforeFork (std::string const &jobReportFile, unsigned int numberOfChildren)
 
void reportAnalysisFile (std::string const &fileName, std::map< std::string, std::string > const &fileData)
 
void reportDataType (Token fileToken, std::string const &dataType)
 
void reportError (std::string const &shortDesc, std::string const &longDesc, int const &exitCode)
 
void reportFallbackAttempt (std::string const &pfn, std::string const &lfn, std::string const &err)
 
void reportFastClonedBranches (std::set< std::string > const &fastClonedBranches, long long nEvents)
 Inform the job report that branches have been fast Cloned. More...
 
void reportFastCopyingStatus (Token t, std::string const &inputFileName, bool fastCopying)
 
void reportInputLumiSection (unsigned int run, unsigned int lumiSectId)
 
void reportInputRunNumber (unsigned int run)
 
void reportLumiSection (JobReport::Token token, unsigned int run, unsigned int lumiSectId)
 
void reportMemoryInfo (std::vector< std::string > const &memoryData)
 
void reportMessageInfo (std::map< std::string, double > const &messageData)
 
void reportPerformanceForModule (std::string const &metricClass, std::string const &moduleName, std::map< std::string, std::string > const &metrics)
 
void reportPerformanceSummary (std::string const &metricClass, std::map< std::string, std::string > const &metrics)
 
void reportRandomStateFile (std::string const &name)
 
void reportReadBranch (InputType inputType, std::string const &branchName)
 Inform the job report that a branch has been read. More...
 
void reportReadBranches ()
 
void reportRunNumber (JobReport::Token token, unsigned int run)
 
void reportSkippedEvent (unsigned int run, unsigned int event)
 
void reportSkippedFile (std::string const &pfn, std::string const &lfn)
 
 ~JobReport ()
 

Protected Member Functions

boost::scoped_ptr
< JobReportImpl > & 
impl ()
 

Private Attributes

boost::scoped_ptr< JobReportImplimpl_
 
std::mutex write_mutex
 

Detailed Description

Description: A service that collections job handling information.

Usage: The JobReport service collects 'job handling' information (currently file handling) from several sources, collates the information, and at appropriate intervals, reports the information to the job report, through the MessageLogger.

CMS-THREADING Some notes about thread safety:

  1. It is assumed non-experts will turn the job report on and look at the XML output and NOT call the available public functions. Many of the available service functions can be called at times that are not thread safe (causing data races or interleaved output). The following notes are for Framework experts and detail the assumptions made when modifying JobReport to run in the multithreaded Framework.
  2. We assume the primary input source performs its activity serially. There can be multiple secondary input sources running concurrently.
  3. We assume the following sequence of activities where the lines of asterisks indicate synchronization points:

open primary input file nothing else running concurrently Also respondToOpenInputFiles serially

open output files serially nothing else running concurrently

The primary source runs its other tasks concurrently with the secondary sources running their tasks and modules running their tasks.

close primary input file nothing else running concurrently

close output files serially nothing else running concurrently

repeat the above (the output file opens and closes are optional except for the first and last)

All endJob and postEndJob activities occur serially not concurrently

  1. We assume that a single instance of an OutputModule will only be running on one thread at a time. Other instances of that type of OutputModule may be running concurrently. There are several functions where this is an issue. We have discussed that in the future we might decide to run the OutputModule for multiple events concurrently. That would require further modifications of either the JobReport or the OutputModule.
  2. For Primary and SecondaryFile input sources (not SecondarySource) the calls to reportBranchName from the delayed reader need to be serialized.

Definition at line 101 of file JobReport.h.

Member Typedef Documentation

typedef unsigned int edm::JobReport::RunNumber

Definition at line 104 of file JobReport.h.

typedef std::size_t edm::JobReport::Token

Definition at line 105 of file JobReport.h.

Constructor & Destructor Documentation

edm::JobReport::JobReport ( )

Definition at line 326 of file JobReport.cc.

326  :
327  impl_(new JobReportImpl(0)) {
328  }
boost::scoped_ptr< JobReportImpl > impl_
Definition: JobReport.h:436
edm::JobReport::JobReport ( std::ostream *  outputStream)

Definition at line 330 of file JobReport.cc.

References impl_.

330  : impl_(new JobReportImpl(iOstream)) {
331  if(impl_->ost_) {
332  *(impl_->ost_) << "<FrameworkJobReport>\n";
333  }
334  }
boost::scoped_ptr< JobReportImpl > impl_
Definition: JobReport.h:436
edm::JobReport::JobReport ( JobReport const &  )
delete
edm::JobReport::~JobReport ( )

Definition at line 319 of file JobReport.cc.

References impl_.

319  {
320  impl_->flushFiles();
321  if(impl_->ost_) {
322  *(impl_->ost_) << "</FrameworkJobReport>\n" << std::flush;
323  }
324  }
boost::scoped_ptr< JobReportImpl > impl_
Definition: JobReport.h:436

Member Function Documentation

void edm::JobReport::childAfterFork ( std::string const &  jobReportFile,
unsigned int  childIndex,
unsigned int  numberOfChildren 
)

New output file for child.

Definition at line 383 of file JobReport.cc.

References impl_, and AlCaHLTBitMon_ParallelJobs::p.

383  {
384  std::ofstream* p = dynamic_cast<std::ofstream*>(impl_->ost_);
385  if(!p) return;
386  std::ostringstream ofilename;
387  toFileName(jobReportFile, childIndex, numberOfChildren, ofilename);
388  p->open(ofilename.str().c_str());
389  *p << "<FrameworkJobReport>\n";
390  }
boost::scoped_ptr< JobReportImpl > impl_
Definition: JobReport.h:436
std::string edm::JobReport::dumpFiles ( void  )

debug/test util

Definition at line 777 of file JobReport.cc.

References f, edm::findOrDefault(), impl_, edm::JobReport::InputFile::logicalFileName, producerFileCleanner::msg, and edm::JobReport::InputFile::physicalFileName.

777  {
778  std::ostringstream msg;
779 
780  typedef std::vector<JobReport::OutputFile>::iterator iterator;
781 
782  for(iterator f = impl_->outputFiles_.begin(), fEnd = impl_->outputFiles_.end(); f != fEnd; ++f) {
783 
784  msg << "\n<File>";
785  msg << *f;
786 
787  msg << "\n<LumiSections>";
788  msg << "\n<Inputs>";
789  typedef std::vector<JobReport::Token>::iterator iterator;
790  for(iterator iInput = f->contributingInputs.begin(),
791  iInputEnd = f->contributingInputs.end();
792  iInput != iInputEnd; ++iInput) {
793  JobReport::InputFile inpFile = impl_->inputFiles_[*iInput];
794  msg << "\n<Input>";
795  msg << "\n <LFN>" << TiXmlText(inpFile.logicalFileName) << "</LFN>";
796  msg << "\n <PFN>" << TiXmlText(inpFile.physicalFileName) << "</PFN>";
797  msg << "\n <FastCopying>" << findOrDefault(f->fastCopyingInputs, inpFile.physicalFileName) << "</FastCopying>";
798  msg << "\n</Input>";
799  }
800  msg << "\n</Inputs>";
801  msg << "\n</File>";
802 
803  }
804  return msg.str();
805  }
tuple InputFile
Open Root file and provide MEs ############.
double f[11][100]
boost::scoped_ptr< JobReportImpl > impl_
Definition: JobReport.h:436
Value const & findOrDefault(std::map< Key, Value > const &m, Key const &k, Value const &defaultValue)
Definition: Map.h:28
void edm::JobReport::eventReadFromFile ( InputType  inputType,
JobReport::Token  fileToken 
)

Report that an event has been read from the file identified by the given Token.

Definition at line 441 of file JobReport.cc.

References f, impl_, and edm::JobReport::InputFile::numEventsRead.

441  {
442  JobReport::InputFile& f = impl_->getInputFileForToken(inputType, fileToken);
443  ++f.numEventsRead;
444  }
tuple InputFile
Open Root file and provide MEs ############.
double f[11][100]
boost::scoped_ptr< JobReportImpl > impl_
Definition: JobReport.h:436
void edm::JobReport::eventWrittenToFile ( JobReport::Token  fileToken,
unsigned int  run,
unsigned int  event 
)

Report that the event with the given id has been written to the file identified by the given Token.

Definition at line 507 of file JobReport.cc.

References f, impl_, and edm::JobReport::OutputFile::numEventsWritten.

507  {
508  JobReport::OutputFile& f = impl_->getOutputFileForToken(fileToken);
509  ++f.numEventsWritten;
510  }
double f[11][100]
boost::scoped_ptr< JobReportImpl > impl_
Definition: JobReport.h:436
boost::scoped_ptr<JobReportImpl>& edm::JobReport::impl ( )
inlineprotected

Definition at line 433 of file JobReport.h.

References impl_.

Referenced by edm::service::JobReportService::frameworkShutdownOnFailure(), and edm::service::JobReportService::postEndJob().

433 {return impl_;}
boost::scoped_ptr< JobReportImpl > impl_
Definition: JobReport.h:436
void edm::JobReport::inputFileClosed ( InputType  inputType,
JobReport::Token  fileToken 
)

Report that the input file identified by the given Token has been closed. An exception will be thrown if the given Token was not obtained from inputFileOpened.

Definition at line 453 of file JobReport.cc.

References f, edm::JobReport::InputFile::fileHasBeenClosed, impl_, fff_deleter::lock(), edm::Primary, and write_mutex.

Referenced by DQMRootSource::closeFile_().

453  {
454  JobReport::InputFile& f = impl_->getInputFileForToken(inputType, fileToken);
455  f.fileHasBeenClosed = true;
456  if(inputType == InputType::Primary) {
457  impl_->writeInputFile(f);
458  } else {
459  {
460  std::lock_guard<std::mutex> lock(write_mutex);
461  impl_->writeInputFile(f);
462  }
463  }
464  }
std::mutex write_mutex
Definition: JobReport.h:437
tuple InputFile
Open Root file and provide MEs ############.
double f[11][100]
boost::scoped_ptr< JobReportImpl > impl_
Definition: JobReport.h:436
JobReport::Token edm::JobReport::inputFileOpened ( std::string const &  physicalFileName,
std::string const &  logicalFileName,
std::string const &  catalog,
std::string const &  inputType,
std::string const &  inputSourceClassName,
std::string const &  moduleLabel,
std::string const &  guid,
std::vector< std::string > const &  branchNames 
)

Report that an input file has been opened. The returned Token should be used for later identification of this file.

Definition at line 393 of file JobReport.cc.

References edm::JobReport::InputFile::branchNames, validate_alignment_devdb10_cfg::catalog, edm::JobReport::InputFile::catalog, edm::JobReport::InputFile::fileHasBeenClosed, edm::JobReport::InputFile::guid, impl_, RPC_Client_on_RootFile::InputFile, edm::JobReport::InputFile::inputSourceClassName, edm::JobReport::InputFile::inputType, edm::JobReport::InputFile::logicalFileName, edm::JobReport::InputFile::moduleLabel, edm::JobReport::InputFile::numEventsRead, edm::JobReport::InputFile::physicalFileName, edm::Primary, edm::SecondaryFile, and edm::SecondarySource.

Referenced by DQMRootSource::readFile_().

400  {
401 
402  InputType theInputType = InputType::Primary;
403  InputFile* newFile = nullptr;
404  JobReport::Token newToken = 0;
405 
406  if (inputType == "mixingFiles") {
407  theInputType = InputType::SecondarySource;
408  impl_->inputFilesSecSource_.push_back(InputFile());
409  newFile = &impl_->inputFilesSecSource_.back();
410  newToken = impl_->inputFilesSecSource_.size() - 1;
411  } else {
412  if (inputType == "secondaryFiles") {
413  theInputType = InputType::SecondaryFile;
414  }
415  impl_->inputFiles_.emplace_back();
416  newFile = &impl_->inputFiles_.back();
417  newToken = impl_->inputFiles_.size() - 1;
418  }
419 
420  if(theInputType == InputType::Primary) {
421  impl_->lastOpenedPrimaryInputFile_ = impl_->inputFiles_.size() - 1;
422  }
423  newFile->logicalFileName = logicalFileName;
424  newFile->physicalFileName = physicalFileName;
425  newFile->catalog = catalog;
426  newFile->inputType = inputType;
427  newFile->inputSourceClassName = inputSourceClassName;
428  newFile->moduleLabel = moduleLabel;
429  newFile->guid = guid;
430  newFile->numEventsRead = 0;
431  newFile->branchNames = branchNames;
432  newFile->fileHasBeenClosed = false;
433 
434  // Add the new input file token to all output files
435  // currently open.
436  impl_->insertInputForOutputs(theInputType, newToken);
437  return newToken;
438  }
InputType
Definition: InputType.h:5
tuple InputFile
Open Root file and provide MEs ############.
std::size_t Token
Definition: JobReport.h:105
boost::scoped_ptr< JobReportImpl > impl_
Definition: JobReport.h:436
JobReport& edm::JobReport::operator= ( JobReport const &  )
delete
void edm::JobReport::outputFileClosed ( JobReport::Token  fileToken)

Report that the output file identified by the given Token has been closed. An exception will be thrown if the given Token was not obtained from outputFileOpened.

Definition at line 513 of file JobReport.cc.

References f, edm::JobReport::OutputFile::fileHasBeenClosed, and impl_.

Referenced by DQMRootOutputModule::finishEndFile().

513  {
514  JobReport::OutputFile& f = impl_->getOutputFileForToken(fileToken);
515  f.fileHasBeenClosed = true;
516  impl_->writeOutputFile(f);
517  }
double f[11][100]
boost::scoped_ptr< JobReportImpl > impl_
Definition: JobReport.h:436
JobReport::Token edm::JobReport::outputFileOpened ( std::string const &  physicalFileName,
std::string const &  logicalFileName,
std::string const &  catalog,
std::string const &  outputModuleClassName,
std::string const &  moduleLabel,
std::string const &  guid,
std::string const &  dataType,
std::string const &  branchHash,
std::vector< std::string > const &  branchNames 
)

Report that an output file has been opened. The returned Token should be used for later identification of this file.

Definition at line 467 of file JobReport.cc.

References edm::JobReport::OutputFile::branchHash, edm::JobReport::OutputFile::branchNames, validate_alignment_devdb10_cfg::catalog, edm::JobReport::OutputFile::catalog, edm::JobReport::OutputFile::contributingInputs, edm::JobReport::OutputFile::contributingInputsSecSource, dtTPAnalyzer_cfg::dataType, edm::JobReport::OutputFile::dataType, edm::JobReport::OutputFile::fileHasBeenClosed, edm::JobReport::OutputFile::guid, i, impl_, edm::JobReport::OutputFile::logicalFileName, edm::JobReport::OutputFile::moduleLabel, edm::JobReport::OutputFile::numEventsWritten, edm::JobReport::OutputFile::outputModuleClassName, edm::JobReport::OutputFile::physicalFileName, and alignCSCRings::r.

Referenced by DQMRootOutputModule::openFile().

475  {
476  impl_->outputFiles_.emplace_back();
477  JobReport::OutputFile& r = impl_->outputFiles_.back();
478 
479  r.logicalFileName = logicalFileName;
480  r.physicalFileName = physicalFileName;
481  r.catalog = catalog;
482  r.outputModuleClassName = outputModuleClassName;
483  r.moduleLabel = moduleLabel;
484  r.guid = guid;
485  r.dataType = dataType;
486  r.branchHash = branchHash;
487  r.numEventsWritten = 0;
488  r.branchNames = branchNames;
489  r.fileHasBeenClosed = false;
490  //
491  // Init list of contributors to list of open input file Tokens
492  //
493  for(std::vector<Token>::size_type i = 0, iEnd = impl_->inputFiles_.size(); i < iEnd; ++i) {
494  if(!impl_->inputFiles_[i].fileHasBeenClosed) {
495  r.contributingInputs.push_back(i);
496  }
497  }
498  for(tbb::concurrent_vector<Token>::size_type i = 0, iEnd = impl_->inputFilesSecSource_.size(); i < iEnd; ++i) {
499  if(!impl_->inputFilesSecSource_[i].fileHasBeenClosed) {
500  r.contributingInputsSecSource.push_back(i);
501  }
502  }
503  return impl_->outputFiles_.size()-1;
504  }
int i
Definition: DBlmapReader.cc:9
uint16_t size_type
boost::scoped_ptr< JobReportImpl > impl_
Definition: JobReport.h:436
void edm::JobReport::parentAfterFork ( std::string const &  jobReportFile)

Definition at line 379 of file JobReport.cc.

379  {
380  }
void edm::JobReport::parentBeforeFork ( std::string const &  jobReportFile,
unsigned int  numberOfChildren 
)

Definition at line 361 of file JobReport.cc.

References i, impl_, and AlCaHLTBitMon_ParallelJobs::p.

361  {
362  if(impl_->ost_) {
363  *(impl_->ost_) << "<ChildProcessFiles>\n";
364  for(unsigned int i = 0; i < numberOfChildren; ++i) {
365  std::ostringstream ofilename;
366  toFileName(jobReportFile, i, numberOfChildren, ofilename);
367  *(impl_->ost_) << " <ChildProcessFile>" << ofilename.str() << "</ChildProcessFile>\n";
368  }
369  *(impl_->ost_) << "</ChildProcessFiles>\n";
370  *(impl_->ost_) << "</FrameworkJobReport>\n";
371  std::ofstream* p = dynamic_cast<std::ofstream *>(impl_->ost_);
372  if(p) {
373  p->close();
374  }
375  }
376  }
int i
Definition: DBlmapReader.cc:9
boost::scoped_ptr< JobReportImpl > impl_
Definition: JobReport.h:436
void edm::JobReport::reportAnalysisFile ( std::string const &  fileName,
std::map< std::string, std::string > const &  fileData 
)

Definition at line 559 of file JobReport.cc.

References impl_, fff_deleter::lock(), producerFileCleanner::msg, and write_mutex.

Referenced by TFileService::afterBeginJob(), DQMStoreStats::calcstats(), PCLMetadataWriter::endRun(), and DQMFileSaver::saveJobReport().

559  {
560  if(impl_->ost_) {
561  std::ostream& msg = *(impl_->ost_);
562  {
563  std::lock_guard<std::mutex> lock(write_mutex);
564  msg << "<AnalysisFile>\n"
565  << " <FileName>" << TiXmlText(fileName) << "</FileName>\n";
566 
567  typedef std::map<std::string, std::string>::const_iterator const_iterator;
568  for(const_iterator pos = fileData.begin(), posEnd = fileData.end(); pos != posEnd; ++pos) {
569  msg << " <" << pos->first
570  << " Value=\"" << pos->second << "\" />"
571  << "\n";
572  }
573  msg << "</AnalysisFile>\n";
574  msg << std::flush;
575  }
576  }
577  }
std::mutex write_mutex
Definition: JobReport.h:437
boost::scoped_ptr< JobReportImpl > impl_
Definition: JobReport.h:436
void edm::JobReport::reportDataType ( Token  fileToken,
std::string const &  dataType 
)

Report the data type of a file after it has been opened Needed since the data type isn't known until an event has been read

Definition at line 447 of file JobReport.cc.

References dtTPAnalyzer_cfg::dataType, edm::JobReport::OutputFile::dataType, f, and impl_.

447  {
448  JobReport::OutputFile& f = impl_->getOutputFileForToken(fileToken);
449  f.dataType = dataType;
450  }
double f[11][100]
boost::scoped_ptr< JobReportImpl > impl_
Definition: JobReport.h:436
void edm::JobReport::reportError ( std::string const &  shortDesc,
std::string const &  longDesc,
int const &  exitCode 
)

Report an exception, providing details of the problem as a short description (Eg "XXXError") and a long description (Eg "XXX crashed because...") Also overload this method to accept an optional standard exit code

Definition at line 580 of file JobReport.cc.

References impl_, fff_deleter::lock(), producerFileCleanner::msg, and write_mutex.

Referenced by edm::printCmsException(), and edm::printCmsExceptionWarning().

582  {
583  if(impl_->ost_) {
584  {
585  std::lock_guard<std::mutex> lock(write_mutex);
586  std::ostream& msg = *(impl_->ost_);
587  msg << "<FrameworkError ExitStatus=\""<< exitCode
588  << "\" Type=\"" << shortDesc << "\" >\n";
589  msg << "<![CDATA[\n" << longDesc << "\n]]>\n";
590  msg << "</FrameworkError>\n";
591  msg << std::flush;
592  }
593  }
594  }
std::mutex write_mutex
Definition: JobReport.h:437
boost::scoped_ptr< JobReportImpl > impl_
Definition: JobReport.h:436
void edm::JobReport::reportFallbackAttempt ( std::string const &  pfn,
std::string const &  lfn,
std::string const &  err 
)

Definition at line 613 of file JobReport.cc.

References impl_, fff_deleter::lock(), producerFileCleanner::msg, TiXmlElement::SetAttribute(), and write_mutex.

613  {
614  if(impl_->ost_) {
615  std::ostream& msg = *(impl_->ost_);
616  TiXmlElement fallback("FallbackAttempt");
617  fallback.SetAttribute("Pfn", pfn);
618  fallback.SetAttribute("Lfn", lfn);
619  {
620  std::lock_guard<std::mutex> lock(write_mutex);
621  msg << fallback << "\n";
622  msg << "<![CDATA[\n" << err << "\n]]>\n";
623  msg << std::flush;
624  }
625  }
626  }
list pfn
Definition: dbtoconf.py:76
std::mutex write_mutex
Definition: JobReport.h:437
boost::scoped_ptr< JobReportImpl > impl_
Definition: JobReport.h:436
void edm::JobReport::reportFastClonedBranches ( std::set< std::string > const &  fastClonedBranches,
long long  nEvents 
)

Inform the job report that branches have been fast Cloned.

Definition at line 709 of file JobReport.cc.

References impl_, and nEvents.

709  {
710  std::set<std::string>& clonedBranches = impl_->inputFiles_.at(impl_->lastOpenedPrimaryInputFile_).fastClonedBranches;
711  for(std::set<std::string>::const_iterator it = fastClonedBranches.begin(), itEnd = fastClonedBranches.end();
712  it != itEnd; ++it) {
713  if(clonedBranches.insert(*it).second) {
714  impl_->readBranches_[*it] += nEvents;
715  }
716  }
717  }
boost::scoped_ptr< JobReportImpl > impl_
Definition: JobReport.h:436
UInt_t nEvents
Definition: hcalCalib.cc:42
void edm::JobReport::reportFastCopyingStatus ( JobReport::Token  fileToken,
std::string const &  inputFileName,
bool  fastCopying 
)

Definition at line 533 of file JobReport.cc.

References f, edm::JobReport::OutputFile::fastCopyingInputs, and impl_.

533  {
534  JobReport::OutputFile& f = impl_->getOutputFileForToken(fileToken);
535  f.fastCopyingInputs.insert(std::make_pair(inputFileName, fastCopying));
536  }
double f[11][100]
boost::scoped_ptr< JobReportImpl > impl_
Definition: JobReport.h:436
void edm::JobReport::reportInputLumiSection ( unsigned int  run,
unsigned int  lumiSectId 
)

API for reporting a Lumi Section to the job report. for input files, call only if lumi section is physically read from the input file

Definition at line 544 of file JobReport.cc.

References impl_.

Referenced by dqmservices::DQMProtobufReader::readLuminosityBlock_(), and DQMRootSource::readLuminosityBlock_().

544  {
545  impl_->associateInputLumiSection(run, lumiSectId);
546  }
boost::scoped_ptr< JobReportImpl > impl_
Definition: JobReport.h:436
void edm::JobReport::reportInputRunNumber ( unsigned int  run)

API to report a run read from input

Definition at line 554 of file JobReport.cc.

References impl_.

Referenced by DQMRootSource::readRun_().

554  {
555  impl_->associateInputRun(run);
556  }
boost::scoped_ptr< JobReportImpl > impl_
Definition: JobReport.h:436
void edm::JobReport::reportLumiSection ( JobReport::Token  token,
unsigned int  run,
unsigned int  lumiSectId 
)

API for reporting a Lumi Section to the job report. for output files, call only if lumi section is written to the output file

Definition at line 539 of file JobReport.cc.

References impl_.

Referenced by DQMRootOutputModule::writeLuminosityBlock().

539  {
540  impl_->associateLumiSection(token, run, lumiSectId);
541  }
boost::scoped_ptr< JobReportImpl > impl_
Definition: JobReport.h:436
void edm::JobReport::reportMemoryInfo ( std::vector< std::string > const &  memoryData)

Report Memory statistics ALTERNATE FORM - USING THIS MAY NECESSITATE CHANGES IN PARSING XML! Invoked by the Memory service to send an end of job summary about memory usage for inclusion in the job report

Definition at line 629 of file JobReport.cc.

References impl_, and producerFileCleanner::msg.

629  {
630  if(impl_->ost_) {
631  std::ostream& msg = *(impl_->ost_);
632  msg << "<MemoryService>\n";
633 
634  typedef std::vector<std::string>::const_iterator const_iterator;
635  for(const_iterator pos = memoryData.begin(), posEnd = memoryData.end(); pos != posEnd; ++pos) {
636  msg << *pos << "\n";
637  }
638  msg << "</MemoryService>\n";
639  msg << std::flush;
640  }
641  }
boost::scoped_ptr< JobReportImpl > impl_
Definition: JobReport.h:436
void edm::JobReport::reportMessageInfo ( std::map< std::string, double > const &  messageData)

Report Message statistics Invoked by the MessageLogger service to send an end of job summary about numbers of various categories messages issued for inclusion in the job report

Definition at line 644 of file JobReport.cc.

References impl_, and producerFileCleanner::msg.

Referenced by edm::service::MessageLogger::SummarizeInJobReport().

644  {
645  if(impl_->ost_) {
646  std::ostream& msg = *(impl_->ost_);
647  msg << "<MessageSummary>\n";
648  typedef std::map<std::string, double>::const_iterator const_iterator;
649  for(const_iterator pos = messageData.begin(), posEnd = messageData.end(); pos != posEnd; ++pos) {
650  msg << " <" << pos->first
651  << " Value=\"" << pos->second << "\" />"
652  << "\n";
653  }
654  msg << "</MessageSummary>\n";
655  msg << std::flush;
656  }
657  }
boost::scoped_ptr< JobReportImpl > impl_
Definition: JobReport.h:436
void edm::JobReport::reportPerformanceForModule ( std::string const &  metricClass,
std::string const &  moduleName,
std::map< std::string, std::string > const &  metrics 
)

Definition at line 754 of file JobReport.cc.

References impl_, getDQMSummary::iter, and producerFileCleanner::msg.

756  {
757  if(impl_->ost_) {
758  std::ostream& msg = *(impl_->ost_);
759  msg << "<PerformanceReport>\n"
760  << " <PerformanceModule Metric=\"" << metricClass << "\" "
761  << " Module=\"" << moduleName << "\" >\n";
762 
763  typedef std::map<std::string, std::string>::const_iterator const_iterator;
764  for(const_iterator iter = metrics.begin(), iterEnd = metrics.end(); iter != iterEnd; ++iter) {
765  msg << " <Metric Name=\"" << iter->first << "\" "
766  << "Value=\"" << iter->second << "\"/>\n";
767  }
768 
769  msg << " </PerformanceModule>\n"
770  << "</PerformanceReport>\n";
771  msg << std::flush;
772  //LogInfo("FwkJob") << msg.str();
773  }
774  }
std::string moduleName(Provenance const &provenance)
Definition: Provenance.cc:27
boost::scoped_ptr< JobReportImpl > impl_
Definition: JobReport.h:436
void edm::JobReport::reportPerformanceSummary ( std::string const &  metricClass,
std::map< std::string, std::string > const &  metrics 
)

Performance Reports

Two categories: Summary for entire job and module for a specific module Each one requires a performance metric class such as Timing, Memory, CPU, Trigger etc.

Definition at line 733 of file JobReport.cc.

References impl_, getDQMSummary::iter, and producerFileCleanner::msg.

Referenced by TFileAdaptor::termination().

734  {
735  if(impl_->ost_) {
736  std::ostream& msg = *(impl_->ost_);
737  msg << "<PerformanceReport>\n"
738  << " <PerformanceSummary Metric=\"" << metricClass << "\">\n";
739 
740  typedef std::map<std::string, std::string>::const_iterator const_iterator;
741  for(const_iterator iter = metrics.begin(), iterEnd = metrics.end(); iter != iterEnd; ++iter) {
742  msg << " <Metric Name=\"" << iter->first << "\" "
743  << "Value=\"" << iter->second << "\"/>\n";
744  }
745 
746  msg << " </PerformanceSummary>\n"
747  << "</PerformanceReport>\n";
748  msg << std::flush;
749  //LogInfo("FwkJob") << msg.str();
750  }
751  }
boost::scoped_ptr< JobReportImpl > impl_
Definition: JobReport.h:436
void edm::JobReport::reportRandomStateFile ( std::string const &  name)

Report the name of the random engine persistency file

Definition at line 719 of file JobReport.cc.

References impl_, fff_deleter::lock(), producerFileCleanner::msg, and write_mutex.

719  {
720  if(impl_->ost_) {
721  std::ostream& msg = *(impl_->ost_);
722  {
723  std::lock_guard<std::mutex> lock(write_mutex);
724  msg << "<RandomServiceStateFile>\n"
725  << TiXmlText(name) << "\n"
726  << "</RandomServiceStateFile>\n";
727  msg << std::flush;
728  }
729  }
730  }
std::mutex write_mutex
Definition: JobReport.h:437
boost::scoped_ptr< JobReportImpl > impl_
Definition: JobReport.h:436
void edm::JobReport::reportReadBranch ( InputType  inputType,
std::string const &  branchName 
)

Inform the job report that a branch has been read.

Definition at line 694 of file JobReport.cc.

References impl_, edm::Primary, edm::SecondaryFile, and edm::SecondarySource.

694  {
695  if(inputType == InputType::Primary) {
696  // Fast cloned branches have already been reported.
697  std::set<std::string> const& clonedBranches = impl_->inputFiles_.at(impl_->lastOpenedPrimaryInputFile_).fastClonedBranches;
698  if(clonedBranches.find(branchName) == clonedBranches.end()) {
699  ++impl_->readBranches_[branchName];
700  }
701  } else if (inputType == InputType::SecondaryFile) {
702  ++impl_->readBranchesSecFile_[branchName];
703  } else if (inputType == InputType::SecondarySource) {
704  ++impl_->readBranchesSecSource_[branchName].value();
705  }
706  }
boost::scoped_ptr< JobReportImpl > impl_
Definition: JobReport.h:436
void edm::JobReport::reportReadBranches ( )

Report read branches Invoked by the source that reads ROOT/EDM files at the end of a job to report which product branches in the Events tree have been read, with a count of the number of events for which the branch was read.

Definition at line 660 of file JobReport.cc.

References impl_, and TiXmlElement::SetAttribute().

660  {
661  if(impl_->printedReadBranches_) return;
662  impl_->printedReadBranches_ = true;
663  if(impl_->ost_) {
664  std::ostream& ost = *(impl_->ost_);
665  ost << "<ReadBranches>\n";
666  for(auto const& iBranch : impl_->readBranches_) {
667  TiXmlElement branch("Branch");
668  branch.SetAttribute("Name", iBranch.first);
669  branch.SetAttribute("ReadCount", iBranch.second);
670  ost << branch << "\n";
671  }
672  for(auto const& iBranch : impl_->readBranchesSecFile_) {
673  TiXmlElement branch("Branch");
674  branch.SetAttribute("Name", iBranch.first);
675  branch.SetAttribute("ReadCount", iBranch.second);
676  ost << branch << "\n";
677  }
678  ost << "</ReadBranches>\n";
679  if(!impl_->readBranchesSecSource_.empty()) {
680  ost << "<SecondarySourceReadBranches>\n";
681  for(auto const& iBranch : impl_->readBranchesSecSource_) {
682  TiXmlElement branch("Branch");
683  branch.SetAttribute("Name", iBranch.first);
684  branch.SetAttribute("ReadCount", iBranch.second.value().load());
685  ost << branch << "\n";
686  }
687  ost << "</SecondarySourceReadBranches>\n";
688  }
689  ost << std::flush;
690  }
691  }
boost::scoped_ptr< JobReportImpl > impl_
Definition: JobReport.h:436
void edm::JobReport::reportRunNumber ( JobReport::Token  token,
unsigned int  run 
)

API for reporting a Run to the job report. for output files, call only if Run is written to the output file

Definition at line 549 of file JobReport.cc.

References impl_.

Referenced by DQMRootOutputModule::writeRun().

549  {
550  impl_->associateRun(token, run);
551  }
boost::scoped_ptr< JobReportImpl > impl_
Definition: JobReport.h:436
void edm::JobReport::reportSkippedEvent ( unsigned int  run,
unsigned int  event 
)

Definition at line 520 of file JobReport.cc.

References impl_, fff_deleter::lock(), producerFileCleanner::msg, and write_mutex.

520  {
521  if(impl_->ost_) {
522  std::ostream& msg = *(impl_->ost_);
523  {
524  std::lock_guard<std::mutex> lock(write_mutex);
525  msg << "<SkippedEvent Run=\"" << run << "\"";
526  msg << " Event=\"" << event << "\" />\n";
527  msg << std::flush;
528  }
529  }
530  }
std::mutex write_mutex
Definition: JobReport.h:437
boost::scoped_ptr< JobReportImpl > impl_
Definition: JobReport.h:436
void edm::JobReport::reportSkippedFile ( std::string const &  pfn,
std::string const &  lfn 
)

Report Skipped File

Report that a file has been skipped due to it not being found.

Definition at line 597 of file JobReport.cc.

References impl_, fff_deleter::lock(), producerFileCleanner::msg, TiXmlElement::SetAttribute(), and write_mutex.

598  {
599  if(impl_->ost_) {
600  std::ostream& msg = *(impl_->ost_);
601  TiXmlElement skipped("SkippedFile");
602  skipped.SetAttribute("Pfn", pfn);
603  skipped.SetAttribute("Lfn", lfn);
604  {
605  std::lock_guard<std::mutex> lock(write_mutex);
606  msg << skipped << "\n";
607  msg << std::flush;
608  }
609  }
610  }
list pfn
Definition: dbtoconf.py:76
std::mutex write_mutex
Definition: JobReport.h:437
boost::scoped_ptr< JobReportImpl > impl_
Definition: JobReport.h:436

Member Data Documentation

boost::scoped_ptr<JobReportImpl> edm::JobReport::impl_
private
std::mutex edm::JobReport::write_mutex
private