#include <JobReport.h>
Classes | |
class | AtomicLongLong |
struct | InputFile |
struct | JobReportImpl |
struct | OutputFile |
struct | RunReport |
Public Types | |
typedef unsigned int | RunNumber |
typedef std::size_t | Token |
Public Member Functions | |
void | childAfterFork (std::string const &jobReportFile, unsigned int childIndex, unsigned int numberOfChildren) |
New output file for child. More... | |
std::string | dumpFiles (void) |
debug/test util More... | |
void | eventReadFromFile (InputType inputType, Token fileToken) |
void | eventWrittenToFile (Token fileToken, RunNumber_t run, EventNumber_t event) |
void | inputFileClosed (InputType inputType, Token fileToken) |
Token | inputFileOpened (std::string const &physicalFileName, std::string const &logicalFileName, std::string const &catalog, std::string const &inputType, std::string const &inputSourceClassName, std::string const &moduleLabel, std::string const &guid, std::vector< std::string > const &branchNames) |
JobReport () | |
JobReport (std::ostream *outputStream) | |
JobReport (JobReport const &)=delete | |
JobReport & | operator= (JobReport const &)=delete |
void | outputFileClosed (Token fileToken) |
Token | outputFileOpened (std::string const &physicalFileName, std::string const &logicalFileName, std::string const &catalog, std::string const &outputModuleClassName, std::string const &moduleLabel, std::string const &guid, std::string const &dataType, std::string const &branchHash, std::vector< std::string > const &branchNames) |
void | parentAfterFork (std::string const &jobReportFile) |
void | parentBeforeFork (std::string const &jobReportFile, unsigned int numberOfChildren) |
void | reportAnalysisFile (std::string const &fileName, std::map< std::string, std::string > const &fileData) |
void | reportDataType (Token fileToken, std::string const &dataType) |
void | reportError (std::string const &shortDesc, std::string const &longDesc, int const &exitCode) |
void | reportFallbackAttempt (std::string const &pfn, std::string const &lfn, std::string const &err) |
void | reportFastClonedBranches (std::set< std::string > const &fastClonedBranches, long long nEvents) |
Inform the job report that branches have been fast Cloned. More... | |
void | reportFastCopyingStatus (Token t, std::string const &inputFileName, bool fastCopying) |
void | reportInputLumiSection (unsigned int run, unsigned int lumiSectId) |
void | reportInputRunNumber (unsigned int run) |
void | reportLumiSection (JobReport::Token token, unsigned int run, unsigned int lumiSectId, unsigned long nEvents=0) |
void | reportMemoryInfo (std::vector< std::string > const &memoryData) |
void | reportMessageInfo (std::map< std::string, double > const &messageData) |
void | reportPerformanceForModule (std::string const &metricClass, std::string const &moduleName, std::map< std::string, std::string > const &metrics) |
void | reportPerformanceSummary (std::string const &metricClass, std::map< std::string, std::string > const &metrics) |
void | reportRandomStateFile (std::string const &name) |
void | reportReadBranch (InputType inputType, std::string const &branchName) |
Inform the job report that a branch has been read. More... | |
void | reportReadBranches () |
void | reportRunNumber (JobReport::Token token, unsigned int run) |
void | reportSkippedEvent (RunNumber_t run, EventNumber_t event) |
void | reportSkippedFile (std::string const &pfn, std::string const &lfn) |
~JobReport () | |
Protected Member Functions | |
edm::propagate_const< std::unique_ptr< JobReportImpl > > & | impl () |
Private Attributes | |
edm::propagate_const< std::unique_ptr< JobReportImpl > > | impl_ |
std::mutex | write_mutex |
Description: A service that collections job handling information.
Usage: The JobReport service collects 'job handling' information (currently file handling) from several sources, collates the information, and at appropriate intervals, reports the information to the job report, through the MessageLogger.
CMS-THREADING Some notes about thread safety:
open primary input file nothing else running concurrently Also respondToOpenInputFiles serially
open output files serially nothing else running concurrently
The primary source runs its other tasks concurrently with the secondary sources running their tasks and modules running their tasks.
close primary input file nothing else running concurrently
close output files serially nothing else running concurrently
repeat the above (the output file opens and closes are optional except for the first and last)
All endJob and postEndJob activities occur serially not concurrently
Definition at line 103 of file JobReport.h.
typedef unsigned int edm::JobReport::RunNumber |
Definition at line 106 of file JobReport.h.
typedef std::size_t edm::JobReport::Token |
Definition at line 107 of file JobReport.h.
edm::JobReport::JobReport | ( | ) |
Definition at line 330 of file JobReport.cc.
edm::JobReport::JobReport | ( | std::ostream * | outputStream | ) |
Definition at line 334 of file JobReport.cc.
References objects.autophobj::filler, impl_, PFRecoTauDiscriminationByIsolation_cfi::offset, AlCaHLTBitMon_QueryRunRegistry::string, and mitigatedMETSequence_cff::U.
|
delete |
edm::JobReport::~JobReport | ( | ) |
Definition at line 323 of file JobReport.cc.
References impl_.
void edm::JobReport::childAfterFork | ( | std::string const & | jobReportFile, |
unsigned int | childIndex, | ||
unsigned int | numberOfChildren | ||
) |
New output file for child.
Definition at line 387 of file JobReport.cc.
References impl_, and AlCaHLTBitMon_ParallelJobs::p.
std::string edm::JobReport::dumpFiles | ( | void | ) |
debug/test util
Definition at line 781 of file JobReport.cc.
References f, edm::findOrDefault(), impl_, and mps_alisetup::msg.
void edm::JobReport::eventReadFromFile | ( | InputType | inputType, |
JobReport::Token | fileToken | ||
) |
Report that an event has been read from the file identified by the given Token.
Definition at line 445 of file JobReport.cc.
References f, impl_, and edm::JobReport::InputFile::numEventsRead.
void edm::JobReport::eventWrittenToFile | ( | JobReport::Token | fileToken, |
RunNumber_t | run, | ||
EventNumber_t | event | ||
) |
Report that the event with the given id has been written to the file identified by the given Token.
Definition at line 511 of file JobReport.cc.
References f, impl_, and edm::JobReport::OutputFile::numEventsWritten.
|
inlineprotected |
Definition at line 438 of file JobReport.h.
References impl_.
Referenced by edm::service::JobReportService::frameworkShutdownOnFailure(), and edm::service::JobReportService::postEndJob().
void edm::JobReport::inputFileClosed | ( | InputType | inputType, |
JobReport::Token | fileToken | ||
) |
Report that the input file identified by the given Token has been closed. An exception will be thrown if the given Token was not obtained from inputFileOpened.
Definition at line 457 of file JobReport.cc.
References f, edm::JobReport::InputFile::fileHasBeenClosed, impl_, CommonMethods::lock(), edm::Primary, and write_mutex.
Referenced by DQMRootSource::closeFile_().
JobReport::Token edm::JobReport::inputFileOpened | ( | std::string const & | physicalFileName, |
std::string const & | logicalFileName, | ||
std::string const & | catalog, | ||
std::string const & | inputType, | ||
std::string const & | inputSourceClassName, | ||
std::string const & | moduleLabel, | ||
std::string const & | guid, | ||
std::vector< std::string > const & | branchNames | ||
) |
Report that an input file has been opened. The returned Token should be used for later identification of this file.
Definition at line 397 of file JobReport.cc.
References edm::JobReport::InputFile::branchNames, getEcalConditions_orcoffint2r_cff::catalog, edm::JobReport::InputFile::catalog, edm::JobReport::InputFile::fileHasBeenClosed, edm::JobReport::InputFile::guid, impl_, ElectronMcFakePostValidator_cfi::InputFile, edm::JobReport::InputFile::inputSourceClassName, edm::JobReport::InputFile::inputType, edm::JobReport::InputFile::logicalFileName, edm::JobReport::InputFile::moduleLabel, edm::JobReport::InputFile::numEventsRead, edm::JobReport::InputFile::physicalFileName, edm::Primary, edm::SecondaryFile, and edm::SecondarySource.
Referenced by DQMRootSource::readFile_().
void edm::JobReport::outputFileClosed | ( | JobReport::Token | fileToken | ) |
Report that the output file identified by the given Token has been closed. An exception will be thrown if the given Token was not obtained from outputFileOpened.
Definition at line 517 of file JobReport.cc.
References f, edm::JobReport::OutputFile::fileHasBeenClosed, and impl_.
Referenced by DQMRootOutputModule::finishEndFile().
JobReport::Token edm::JobReport::outputFileOpened | ( | std::string const & | physicalFileName, |
std::string const & | logicalFileName, | ||
std::string const & | catalog, | ||
std::string const & | outputModuleClassName, | ||
std::string const & | moduleLabel, | ||
std::string const & | guid, | ||
std::string const & | dataType, | ||
std::string const & | branchHash, | ||
std::vector< std::string > const & | branchNames | ||
) |
Report that an output file has been opened. The returned Token should be used for later identification of this file.
Definition at line 471 of file JobReport.cc.
References edm::JobReport::OutputFile::branchHash, edm::JobReport::OutputFile::branchNames, getEcalConditions_orcoffint2r_cff::catalog, edm::JobReport::OutputFile::catalog, edm::JobReport::OutputFile::contributingInputs, edm::JobReport::OutputFile::contributingInputsSecSource, MergeFilesAndCalculateEfficiencies_cfg::dataType, edm::JobReport::OutputFile::dataType, edm::JobReport::OutputFile::fileHasBeenClosed, edm::JobReport::OutputFile::guid, mps_fire::i, impl_, edm::JobReport::OutputFile::logicalFileName, edm::JobReport::OutputFile::moduleLabel, edm::JobReport::OutputFile::numEventsWritten, edm::JobReport::OutputFile::outputModuleClassName, edm::JobReport::OutputFile::physicalFileName, and alignCSCRings::r.
Referenced by DQMRootOutputModule::openFile().
void edm::JobReport::parentAfterFork | ( | std::string const & | jobReportFile | ) |
Definition at line 383 of file JobReport.cc.
void edm::JobReport::parentBeforeFork | ( | std::string const & | jobReportFile, |
unsigned int | numberOfChildren | ||
) |
Definition at line 365 of file JobReport.cc.
References mps_fire::i, impl_, and AlCaHLTBitMon_ParallelJobs::p.
void edm::JobReport::reportAnalysisFile | ( | std::string const & | fileName, |
std::map< std::string, std::string > const & | fileData | ||
) |
Definition at line 563 of file JobReport.cc.
References impl_, CommonMethods::lock(), mps_alisetup::msg, and write_mutex.
Referenced by TFileService::afterBeginJob(), DQMStoreStats::calcstats(), PCLMetadataWriter::endRun(), DQMFileSaver::saveJobReport(), and dqm::DQMFileSaverBase::saveJobReport().
void edm::JobReport::reportDataType | ( | Token | fileToken, |
std::string const & | dataType | ||
) |
Report the data type of a file after it has been opened Needed since the data type isn't known until an event has been read
Definition at line 451 of file JobReport.cc.
References MergeFilesAndCalculateEfficiencies_cfg::dataType, edm::JobReport::OutputFile::dataType, f, and impl_.
void edm::JobReport::reportError | ( | std::string const & | shortDesc, |
std::string const & | longDesc, | ||
int const & | exitCode | ||
) |
Report an exception, providing details of the problem as a short description (Eg "XXXError") and a long description (Eg "XXX crashed because...") Also overload this method to accept an optional standard exit code
Definition at line 584 of file JobReport.cc.
References impl_, CommonMethods::lock(), mps_alisetup::msg, and write_mutex.
Referenced by edm::printCmsException().
void edm::JobReport::reportFallbackAttempt | ( | std::string const & | pfn, |
std::string const & | lfn, | ||
std::string const & | err | ||
) |
Definition at line 617 of file JobReport.cc.
References trackingPlots::fallback, impl_, CommonMethods::lock(), mps_alisetup::msg, and write_mutex.
void edm::JobReport::reportFastClonedBranches | ( | std::set< std::string > const & | fastClonedBranches, |
long long | nEvents | ||
) |
Inform the job report that branches have been fast Cloned.
Definition at line 713 of file JobReport.cc.
References impl_, and nEvents.
void edm::JobReport::reportFastCopyingStatus | ( | JobReport::Token | fileToken, |
std::string const & | inputFileName, | ||
bool | fastCopying | ||
) |
Definition at line 537 of file JobReport.cc.
References f, edm::JobReport::OutputFile::fastCopyingInputs, and impl_.
void edm::JobReport::reportInputLumiSection | ( | unsigned int | run, |
unsigned int | lumiSectId | ||
) |
API for reporting a Lumi Section to the job report. for input files, call only if lumi section is physically read from the input file
Definition at line 548 of file JobReport.cc.
References impl_.
Referenced by dqmservices::DQMProtobufReader::readLuminosityBlock_(), and DQMRootSource::readLuminosityBlock_().
void edm::JobReport::reportInputRunNumber | ( | unsigned int | run | ) |
API to report a run read from input
Definition at line 558 of file JobReport.cc.
References impl_.
Referenced by DQMRootSource::readRun_().
void edm::JobReport::reportLumiSection | ( | JobReport::Token | token, |
unsigned int | run, | ||
unsigned int | lumiSectId, | ||
unsigned long | nEvents = 0 |
||
) |
API for reporting a Lumi Section to the job report. for output files, call only if lumi section is written to the output file
Definition at line 543 of file JobReport.cc.
References impl_.
Referenced by DQMRootOutputModule::writeLuminosityBlock().
void edm::JobReport::reportMemoryInfo | ( | std::vector< std::string > const & | memoryData | ) |
Report Memory statistics ALTERNATE FORM - USING THIS MAY NECESSITATE CHANGES IN PARSING XML! Invoked by the Memory service to send an end of job summary about memory usage for inclusion in the job report
Definition at line 633 of file JobReport.cc.
References impl_, and mps_alisetup::msg.
void edm::JobReport::reportMessageInfo | ( | std::map< std::string, double > const & | messageData | ) |
Report Message statistics Invoked by the MessageLogger service to send an end of job summary about numbers of various categories messages issued for inclusion in the job report
Definition at line 648 of file JobReport.cc.
References impl_, and mps_alisetup::msg.
Referenced by edm::service::MessageLogger::SummarizeInJobReport().
void edm::JobReport::reportPerformanceForModule | ( | std::string const & | metricClass, |
std::string const & | moduleName, | ||
std::map< std::string, std::string > const & | metrics | ||
) |
Definition at line 758 of file JobReport.cc.
References impl_, mps_alisetup::msg, and AlCaHLTBitMon_QueryRunRegistry::string.
Referenced by XrdAdaptor::XrdStatisticsService::postEndJob().
void edm::JobReport::reportPerformanceSummary | ( | std::string const & | metricClass, |
std::map< std::string, std::string > const & | metrics | ||
) |
Performance Reports
Two categories: Summary for entire job and module for a specific module Each one requires a performance metric class such as Timing, Memory, CPU, Trigger etc.
Definition at line 737 of file JobReport.cc.
References impl_, and mps_alisetup::msg.
Referenced by TFileAdaptor::termination().
void edm::JobReport::reportRandomStateFile | ( | std::string const & | name | ) |
Report the name of the random engine persistency file
Definition at line 723 of file JobReport.cc.
References impl_, CommonMethods::lock(), mps_alisetup::msg, and write_mutex.
void edm::JobReport::reportReadBranch | ( | InputType | inputType, |
std::string const & | branchName | ||
) |
Inform the job report that a branch has been read.
Definition at line 698 of file JobReport.cc.
References impl_, edm::Primary, edm::SecondaryFile, and edm::SecondarySource.
void edm::JobReport::reportReadBranches | ( | ) |
Report read branches Invoked by the source that reads ROOT/EDM files at the end of a job to report which product branches in the Events tree have been read, with a count of the number of events for which the branch was read.
Definition at line 664 of file JobReport.cc.
References impl_.
void edm::JobReport::reportRunNumber | ( | JobReport::Token | token, |
unsigned int | run | ||
) |
API for reporting a Run to the job report. for output files, call only if Run is written to the output file
Definition at line 553 of file JobReport.cc.
References impl_.
Referenced by DQMRootOutputModule::writeRun().
void edm::JobReport::reportSkippedEvent | ( | RunNumber_t | run, |
EventNumber_t | event | ||
) |
Definition at line 524 of file JobReport.cc.
References impl_, CommonMethods::lock(), mps_alisetup::msg, and write_mutex.
void edm::JobReport::reportSkippedFile | ( | std::string const & | pfn, |
std::string const & | lfn | ||
) |
Report Skipped File
Report that a file has been skipped due to it not being found.
Definition at line 601 of file JobReport.cc.
References impl_, CommonMethods::lock(), mps_alisetup::msg, runEdmFileComparison::skipped, and write_mutex.
|
private |
Definition at line 441 of file JobReport.h.
Referenced by childAfterFork(), dumpFiles(), eventReadFromFile(), eventWrittenToFile(), impl(), inputFileClosed(), inputFileOpened(), JobReport(), outputFileClosed(), outputFileOpened(), parentBeforeFork(), reportAnalysisFile(), reportDataType(), reportError(), reportFallbackAttempt(), reportFastClonedBranches(), reportFastCopyingStatus(), reportInputLumiSection(), reportInputRunNumber(), reportLumiSection(), reportMemoryInfo(), reportMessageInfo(), reportPerformanceForModule(), reportPerformanceSummary(), reportRandomStateFile(), reportReadBranch(), reportReadBranches(), reportRunNumber(), reportSkippedEvent(), reportSkippedFile(), and ~JobReport().
|
private |
Definition at line 442 of file JobReport.h.
Referenced by inputFileClosed(), reportAnalysisFile(), reportError(), reportFallbackAttempt(), reportRandomStateFile(), reportSkippedEvent(), and reportSkippedFile().