edm::JobReport Class Reference

#include <JobReport.h>

Inheritance diagram for edm::JobReport:

Classes
class	AtomicLongLong
struct	InputFile
struct	JobReportImpl
struct	OutputFile
struct	RunReport

Public Types
typedef unsigned int	RunNumber
typedef std::size_t	Token

Public Member Functions
std::string	dumpFiles (void)
	debug/test util More...
void	eventReadFromFile (InputType inputType, Token fileToken)
void	eventWrittenToFile (Token fileToken, RunNumber_t run, EventNumber_t event)
void	inputFileClosed (InputType inputType, Token fileToken)
Token	inputFileOpened (std::string const &physicalFileName, std::string const &logicalFileName, std::string const &catalog, std::string const &inputType, std::string const &inputSourceClassName, std::string const &moduleLabel, std::string const &guid, std::vector< std::string > const &branchNames)
	JobReport ()
	JobReport (std::ostream *outputStream)
	JobReport (JobReport const &)=delete
JobReport &	operator= (JobReport const &)=delete
void	outputFileClosed (Token fileToken)
Token	outputFileOpened (std::string const &physicalFileName, std::string const &logicalFileName, std::string const &catalog, std::string const &outputModuleClassName, std::string const &moduleLabel, std::string const &guid, std::string const &dataType, std::string const &branchHash, std::vector< std::string > const &branchNames)
void	reportAnalysisFile (std::string const &fileName, std::map< std::string, std::string > const &fileData)
void	reportDataType (Token fileToken, std::string const &dataType)
void	reportError (std::string const &shortDesc, std::string const &longDesc, int const &exitCode)
void	reportFallbackAttempt (std::string const &pfn, std::string const &lfn, std::string const &err)
void	reportFastClonedBranches (std::set< std::string > const &fastClonedBranches, long long nEvents)
	Inform the job report that branches have been fast Cloned. More...
void	reportFastCopyingStatus (Token t, std::string const &inputFileName, bool fastCopying)
void	reportInputLumiSection (unsigned int run, unsigned int lumiSectId)
void	reportInputRunNumber (unsigned int run)
void	reportLumiSection (JobReport::Token token, unsigned int run, unsigned int lumiSectId, unsigned long nEvents=0)
void	reportMemoryInfo (std::vector< std::string > const &memoryData)
void	reportMessageInfo (std::map< std::string, double > const &messageData)
void	reportPerformanceForModule (std::string const &metricClass, std::string const &moduleName, std::map< std::string, std::string > const &metrics)
void	reportPerformanceSummary (std::string const &metricClass, std::map< std::string, std::string > const &metrics)
void	reportRandomStateFile (std::string const &name)
void	reportReadBranch (InputType inputType, std::string const &branchName)
	Inform the job report that a branch has been read. More...
void	reportReadBranches ()
void	reportRunNumber (JobReport::Token token, unsigned int run)
void	reportShutdownSignal ()
void	reportSkippedEvent (RunNumber_t run, EventNumber_t event)
void	reportSkippedFile (std::string const &pfn, std::string const &lfn)
	~JobReport ()

Protected Member Functions
edm::propagate_const< std::unique_ptr< JobReportImpl > > &	impl ()

Private Member Functions
void	temporarilyCloseXML ()

Private Attributes
bool	errorLogged_ = false
edm::propagate_const< std::unique_ptr< JobReportImpl > >	impl_
std::mutex	write_mutex

Detailed Description

Description: A service that collections job handling information.

Usage: The JobReport service collects 'job handling' information (currently file handling) from several sources, collates the information, and at appropriate intervals, reports the information to the job report, through the MessageLogger.

CMS-THREADING Some notes about thread safety:

1. It is assumed non-experts will turn the job report on and look at the XML output and NOT call the available public functions. Many of the available service functions can be called at times that are not thread safe (causing data races or interleaved output). The following notes are for Framework experts and detail the assumptions made when modifying JobReport to run in the multithreaded Framework.
2. We assume the primary input source performs its activity serially. There can be multiple secondary input sources running concurrently.
3. We assume the following sequence of activities where the lines of asterisks indicate synchronization points:

open primary input file nothing else running concurrently Also respondToOpenInputFiles serially

open output files serially nothing else running concurrently

The primary source runs its other tasks concurrently with the secondary sources running their tasks and modules running their tasks.

close primary input file nothing else running concurrently

close output files serially nothing else running concurrently

repeat the above (the output file opens and closes are optional except for the first and last)

All endJob and postEndJob activities occur serially not concurrently

1. We assume that a single instance of an OutputModule will only be running on one thread at a time. Other instances of that type of OutputModule may be running concurrently. There are several functions where this is an issue. We have discussed that in the future we might decide to run the OutputModule for multiple events concurrently. That would require further modifications of either the JobReport or the OutputModule.
2. For Primary and SecondaryFile input sources (not SecondarySource) the calls to reportBranchName from the delayed reader need to be serialized.

Definition at line 103 of file JobReport.h.

Member Typedef Documentation

RunNumber

typedef unsigned int edm::JobReport::RunNumber

Definition at line 105 of file JobReport.h.

Token

typedef std::size_t edm::JobReport::Token

Definition at line 106 of file JobReport.h.

Constructor & Destructor Documentation

JobReport() [1/3]

edm::JobReport::JobReport

(

)

Definition at line 345 of file JobReport.cc.

: impl_(new JobReportImpl(nullptr)) {}

JobReport() [2/3]

edm::JobReport::JobReport

(

std::ostream *

outputStream

)

Definition at line 347 of file JobReport.cc.

References impl_, and temporarilyCloseXML().

                                           : impl_(new JobReportImpl(iOstream)) {
    if (impl_->ost_) {
      *(impl_->ost_) << "<FrameworkJobReport>\n";
    }
    temporarilyCloseXML();
  }

JobReport() [3/3]

edm::JobReport::JobReport ( JobReport const &  )

delete

~JobReport()

edm::JobReport::~JobReport

(

)

Definition at line 308 of file JobReport.cc.

References mps_fire::end, mps_fire::i, impl_, edm::kJobReportEndElement, and edm::kMinSizeOfComment.

                        {
    impl_->flushFiles();
    if (impl_->ost_) {
      //are we actually at the end of the file?
      auto pos = impl_->ost_->tellp();
      impl_->ost_->seekp(0, std::ios_base::end);
      auto endpos = impl_->ost_->tellp();
      impl_->ost_->seekp(pos);
      if ((endpos - pos) > static_cast<long int>(kJobReportEndElement.size())) {
        //need to add some padding so use a comment element
        // comment is used since white spaces are converted to a special node
        // while comments are usually ignored by xml parsers
        auto padding = (endpos - pos) - (kJobReportEndElement.size() + kMinSizeOfComment);
        *(impl_->ost_) << "<!--";
        for (int i = padding; i > 0; --i) {
          (*impl_->ost_) << ' ';
        }
        *(impl_->ost_) << "-->\n";
      }
      *(impl_->ost_) << kJobReportEndElement << std::flush;
    }
  }

Member Function Documentation

dumpFiles()

std::string edm::JobReport::dumpFiles

(

void

)

debug/test util

Definition at line 743 of file JobReport.cc.

References hcalRecHitTable_cff::doc, f, edm::findOrDefault(), impl_, and mps_check::msg.

                                     {
    std::ostringstream msg;

    tinyxml2::XMLDocument doc;
    for (auto const& f : impl_->outputFiles_) {
      msg << "\n<File>";
      msg << f;

      msg << "\n<LumiSections>";
      msg << "\n<Inputs>";
      typedef std::vector<JobReport::Token>::iterator iterator;
      for (auto const& iInput : f.contributingInputs) {
        auto const& inpFile = impl_->inputFiles_[iInput];
        msg << "\n<Input>";
        msg << "\n  <LFN>" << doc.NewText(inpFile.logicalFileName.c_str())->Value() << "</LFN>";
        msg << "\n  <PFN>" << doc.NewText(inpFile.physicalFileName.c_str())->Value() << "</PFN>";
        msg << "\n  <FastCopying>" << findOrDefault(f.fastCopyingInputs, inpFile.physicalFileName) << "</FastCopying>";
        msg << "\n</Input>";
        doc.DeleteChildren();
      }
      msg << "\n</Inputs>";
      msg << "\n</File>";
    }
    return msg.str();
  }

eventReadFromFile()

void edm::JobReport::eventReadFromFile	(	InputType	inputType,
		JobReport::Token	fileToken
	)

Report that an event has been read from the file identified by the given Token.

Definition at line 400 of file JobReport.cc.

References f, and impl_.

                                                                                 {
    JobReport::InputFile& f = impl_->getInputFileForToken(inputType, fileToken);
    ++f.numEventsRead;
  }

eventWrittenToFile()

void edm::JobReport::eventWrittenToFile	(	JobReport::Token	fileToken,
		RunNumber_t	run,
		EventNumber_t	event
	)

Report that the event with the given id has been written to the file identified by the given Token.

Definition at line 462 of file JobReport.cc.

References f, and impl_.

Referenced by NanoAODRNTupleOutputModule::write(), and NanoAODOutputModule::write().

                                                                                           {
    JobReport::OutputFile& f = impl_->getOutputFileForToken(fileToken);
    ++f.numEventsWritten;
  }

impl()

edm::propagate_const<std::unique_ptr<JobReportImpl> >& edm::JobReport::impl ( )

inlineprotected

Definition at line 432 of file JobReport.h.

References impl_.

Referenced by edm::service::JobReportService::frameworkShutdownOnFailure(), and edm::service::JobReportService::postEndJob().

{ return impl_; }

inputFileClosed()

void edm::JobReport::inputFileClosed	(	InputType	inputType,
		JobReport::Token	fileToken
	)

Report that the input file identified by the given Token has been closed. An exception will be thrown if the given Token was not obtained from inputFileOpened.

Definition at line 410 of file JobReport.cc.

References f, impl_, CommonMethods::lock(), edm::Primary, temporarilyCloseXML(), and write_mutex.

Referenced by DQMRootSource::OpenFileInfo::~OpenFileInfo().

                                                                               {
    JobReport::InputFile& f = impl_->getInputFileForToken(inputType, fileToken);
    f.fileHasBeenClosed = true;
    std::lock_guard<std::mutex> lock(write_mutex);
    if (inputType == InputType::Primary) {
      impl_->writeInputFile(f);
    } else {
      impl_->writeInputFile(f);
    }
    temporarilyCloseXML();
  }

inputFileOpened()

JobReport::Token edm::JobReport::inputFileOpened	(	std::string const &	physicalFileName,
		std::string const &	logicalFileName,
		std::string const &	catalog,
		std::string const &	inputType,
		std::string const &	inputSourceClassName,
		std::string const &	moduleLabel,
		std::string const &	guid,
		std::vector< std::string > const &	branchNames
	)

Report that an input file has been opened. The returned Token should be used for later identification of this file.

Definition at line 354 of file JobReport.cc.

References haddnano::branchNames, edm::JobReport::InputFile::branchNames, getEcalConditions_orcoffint2r_cff::catalog, edm::JobReport::InputFile::catalog, edm::JobReport::InputFile::fileHasBeenClosed, edm::JobReport::InputFile::guid, impl_, RPC_Client_on_RootFile::InputFile, edm::JobReport::InputFile::inputSourceClassName, edm::JobReport::InputFile::inputType, edm::JobReport::InputFile::logicalFileName, HerwigMaxPtPartonFilter_cfi::moduleLabel, edm::JobReport::InputFile::moduleLabel, edm::JobReport::InputFile::numEventsRead, edm::JobReport::InputFile::physicalFileName, edm::Primary, edm::SecondaryFile, and edm::SecondarySource.

Referenced by DQMRootSource::readFile_().

                                                                                       {
    InputType theInputType = InputType::Primary;
    InputFile* newFile = nullptr;
    JobReport::Token newToken = 0;

    if (inputType == "mixingFiles") {
      theInputType = InputType::SecondarySource;
      auto itr = impl_->inputFilesSecSource_.push_back(InputFile());
      newFile = &(*itr);
      newToken = itr - impl_->inputFilesSecSource_.begin();
    } else {
      if (inputType == "secondaryFiles") {
        theInputType = InputType::SecondaryFile;
      }
      impl_->inputFiles_.emplace_back();
      newFile = &impl_->inputFiles_.back();
      newToken = impl_->inputFiles_.size() - 1;
    }

    if (theInputType == InputType::Primary) {
      impl_->lastOpenedPrimaryInputFile_ = impl_->inputFiles_.size() - 1;
    }
    newFile->logicalFileName = logicalFileName;
    newFile->physicalFileName = physicalFileName;
    newFile->catalog = catalog;
    newFile->inputType = inputType;
    newFile->inputSourceClassName = inputSourceClassName;
    newFile->moduleLabel = moduleLabel;
    newFile->guid = guid;
    newFile->numEventsRead = 0;
    newFile->branchNames = branchNames;
    newFile->fileHasBeenClosed = false;

    // Add the new input file token to all output files
    //  currently open.
    impl_->insertInputForOutputs(theInputType, newToken);
    return newToken;
  }

operator=()

JobReport& edm::JobReport::operator= ( JobReport const &  )

delete

outputFileClosed()

void edm::JobReport::outputFileClosed

(

JobReport::Token

fileToken

)

Report that the output file identified by the given Token has been closed. An exception will be thrown if the given Token was not obtained from outputFileOpened.

Definition at line 467 of file JobReport.cc.

References f, impl_, CommonMethods::lock(), temporarilyCloseXML(), and write_mutex.

Referenced by DQMRootOutputModule::finishEndFile(), NanoAODRNTupleOutputModule::reallyCloseFile(), and NanoAODOutputModule::reallyCloseFile().

                                                           {
    JobReport::OutputFile& f = impl_->getOutputFileForToken(fileToken);
    f.fileHasBeenClosed = true;
    std::lock_guard<std::mutex> lock(write_mutex);
    impl_->writeOutputFile(f);
    temporarilyCloseXML();
  }

outputFileOpened()

JobReport::Token edm::JobReport::outputFileOpened	(	std::string const &	physicalFileName,
		std::string const &	logicalFileName,
		std::string const &	catalog,
		std::string const &	outputModuleClassName,
		std::string const &	moduleLabel,
		std::string const &	guid,
		std::string const &	dataType,
		std::string const &	branchHash,
		std::vector< std::string > const &	branchNames
	)

Report that an output file has been opened. The returned Token should be used for later identification of this file.

Definition at line 422 of file JobReport.cc.

References haddnano::branchNames, getEcalConditions_orcoffint2r_cff::catalog, DTskim_cfg::dataType, mps_fire::i, impl_, and HerwigMaxPtPartonFilter_cfi::moduleLabel.

                                                                                        {
    auto itr = impl_->outputFiles_.emplace_back();
    JobReport::OutputFile& r = *itr;

    r.logicalFileName = logicalFileName;
    r.physicalFileName = physicalFileName;
    r.catalog = catalog;
    r.outputModuleClassName = outputModuleClassName;
    r.moduleLabel = moduleLabel;
    r.guid = guid;
    r.dataType = dataType;
    r.branchHash = branchHash;
    r.numEventsWritten = 0;
    r.branchNames = branchNames;
    r.fileHasBeenClosed = false;
    //
    // Init list of contributors to list of open input file Tokens
    //
    for (std::vector<Token>::size_type i = 0, iEnd = impl_->inputFiles_.size(); i < iEnd; ++i) {
      if (!impl_->inputFiles_[i].fileHasBeenClosed) {
        r.contributingInputs.push_back(i);
      }
    }
    for (oneapi::tbb::concurrent_vector<Token>::size_type i = 0, iEnd = impl_->inputFilesSecSource_.size(); i < iEnd;
         ++i) {
      if (!impl_->inputFilesSecSource_[i].fileHasBeenClosed) {
        r.contributingInputsSecSource.push_back(i);
      }
    }
    return itr - impl_->outputFiles_.begin();
  }

reportAnalysisFile()

void edm::JobReport::reportAnalysisFile	(	std::string const &	fileName,
		std::map< std::string, std::string > const &	fileData
	)

Definition at line 509 of file JobReport.cc.

References hcalRecHitTable_cff::doc, MillePedeFileConverter_cfg::fileName, impl_, CommonMethods::lock(), mps_check::msg, temporarilyCloseXML(), and write_mutex.

Referenced by TFileService::afterBeginJob(), DQMStoreStats::calcstats(), AlcaBeamSpotHarvester::endRun(), PCLMetadataWriter::endRun(), DQMFileSaver::saveJobReport(), and dqm::DQMFileSaverBase::saveJobReport().

                                                                                                            {
    tinyxml2::XMLDocument doc;
    if (impl_->ost_) {
      std::ostream& msg = *(impl_->ost_);
      {
        std::lock_guard<std::mutex> lock(write_mutex);
        msg << "<AnalysisFile>\n"
            << "  <FileName>" << doc.NewText(fileName.c_str())->Value() << "</FileName>\n";

        typedef std::map<std::string, std::string>::const_iterator const_iterator;
        for (const_iterator pos = fileData.begin(), posEnd = fileData.end(); pos != posEnd; ++pos) {
          msg << "  <" << pos->first << "  Value=\"" << pos->second << "\" />"
              << "\n";
        }
        msg << "</AnalysisFile>\n";
        temporarilyCloseXML();
      }
    }
  }

reportDataType()

void edm::JobReport::reportDataType	(	Token	fileToken,
		std::string const &	dataType
	)

Report the data type of a file after it has been opened Needed since the data type isn't known until an event has been read

Definition at line 405 of file JobReport.cc.

References DTskim_cfg::dataType, f, and impl_.

                                                                           {
    JobReport::OutputFile& f = impl_->getOutputFileForToken(fileToken);
    f.dataType = dataType;
  }

reportError()

void edm::JobReport::reportError	(	std::string const &	shortDesc,
		std::string const &	longDesc,
		int const &	exitCode
	)

Report an exception, providing details of the problem as a short description (Eg "XXXError") and a long description (Eg "XXX crashed because...") Also overload this method to accept an optional standard exit code

Definition at line 529 of file JobReport.cc.

References errorLogged_, impl_, CommonMethods::lock(), mps_check::msg, temporarilyCloseXML(), and write_mutex.

Referenced by edm::printCmsException().

                                                                                                        {
    if (impl_->ost_) {
      {
        std::lock_guard<std::mutex> lock(write_mutex);
        errorLogged_ = true;
        std::ostream& msg = *(impl_->ost_);
        msg << "<FrameworkError ExitStatus=\"" << exitCode << "\" Type=\"" << shortDesc << "\" >\n";
        msg << "<![CDATA[\n" << longDesc << "\n]]>\n";
        msg << "</FrameworkError>\n";
        temporarilyCloseXML();
      }
    }
  }

reportFallbackAttempt()

void edm::JobReport::reportFallbackAttempt	(	std::string const &	pfn,
		std::string const &	lfn,
		std::string const &	err
	)

Definition at line 571 of file JobReport.cc.

References hcalRecHitTable_cff::doc, submitPVResolutionJobs::err, trackingPlots::fallback, impl_, CommonMethods::lock(), mps_check::msg, temporarilyCloseXML(), and write_mutex.

                                                                                                        {
    if (impl_->ost_) {
      std::ostream& msg = *(impl_->ost_);
      tinyxml2::XMLDocument doc;
      tinyxml2::XMLPrinter printer;
      tinyxml2::XMLElement* fallback = doc.NewElement("FallbackAttempt");
      fallback->SetAttribute("Pfn", pfn.c_str());
      fallback->SetAttribute("Lfn", lfn.c_str());
      {
        std::lock_guard<std::mutex> lock(write_mutex);
        fallback->Accept(&printer);
        msg << printer.CStr();
        msg << "<![CDATA[\n" << err << "\n]]>\n";
        temporarilyCloseXML();
      }
    }
  }

reportFastClonedBranches()

void edm::JobReport::reportFastClonedBranches	(	std::set< std::string > const &	fastClonedBranches,
		long long	nEvents
	)

Inform the job report that branches have been fast Cloned.

Definition at line 677 of file JobReport.cc.

References impl_, and createIOVlist::nEvents.

                                                                                                         {
    std::set<std::string>& clonedBranches =
        impl_->inputFiles_.at(impl_->lastOpenedPrimaryInputFile_).fastClonedBranches;
    for (std::set<std::string>::const_iterator it = fastClonedBranches.begin(), itEnd = fastClonedBranches.end();
         it != itEnd;
         ++it) {
      if (clonedBranches.insert(*it).second) {
        impl_->readBranches_[*it] += nEvents;
      }
    }
  }

reportFastCopyingStatus()

void edm::JobReport::reportFastCopyingStatus	(	JobReport::Token	fileToken,
		std::string const &	inputFileName,
		bool	fastCopying
	)

Definition at line 487 of file JobReport.cc.

References f, impl_, and InefficientDoubleROC::inputFileName.

                                                            {
    JobReport::OutputFile& f = impl_->getOutputFileForToken(fileToken);
    f.fastCopyingInputs.insert(std::make_pair(inputFileName, fastCopying));
  }

reportInputLumiSection()

void edm::JobReport::reportInputLumiSection	(	unsigned int	run,
		unsigned int	lumiSectId
	)

API for reporting a Lumi Section to the job report. for input files, call only if lumi section is physically read from the input file

Definition at line 501 of file JobReport.cc.

References impl_, and writedatasetfile::run.

Referenced by dqmservices::DQMProtobufReader::readLuminosityBlock_(), and DQMRootSource::readLuminosityBlock_().

                                                                                  {
    impl_->associateInputLumiSection(run, lumiSectId);
  }

reportInputRunNumber()

void edm::JobReport::reportInputRunNumber

(

unsigned int

run

)

API to report a run read from input

Definition at line 507 of file JobReport.cc.

References impl_, and writedatasetfile::run.

Referenced by DQMRootSource::readRun_().

{ impl_->associateInputRun(run); }

reportLumiSection()

void edm::JobReport::reportLumiSection	(	JobReport::Token	token,
		unsigned int	run,
		unsigned int	lumiSectId,
		unsigned long	nEvents = 0
	)

API for reporting a Lumi Section to the job report. for output files, call only if lumi section is written to the output file

Definition at line 494 of file JobReport.cc.

References impl_, createIOVlist::nEvents, writedatasetfile::run, and unpackBuffers-CaloStage2::token.

Referenced by NanoAODRNTupleOutputModule::writeLuminosityBlock(), NanoAODOutputModule::writeLuminosityBlock(), and DQMRootOutputModule::writeLuminosityBlock().

                                                           {
    impl_->associateLumiSection(token, run, lumiSectId, nEvents);
  }

reportMemoryInfo()

void edm::JobReport::reportMemoryInfo

(

std::vector< std::string > const &

memoryData

)

Report Memory statistics ALTERNATE FORM - USING THIS MAY NECESSITATE CHANGES IN PARSING XML! Invoked by the Memory service to send an end of job summary about memory usage for inclusion in the job report

Definition at line 589 of file JobReport.cc.

References impl_, CommonMethods::lock(), mps_check::msg, temporarilyCloseXML(), and write_mutex.

                                                                         {
    if (impl_->ost_) {
      std::lock_guard<std::mutex> lock(write_mutex);
      std::ostream& msg = *(impl_->ost_);
      msg << "<MemoryService>\n";

      typedef std::vector<std::string>::const_iterator const_iterator;
      for (const_iterator pos = memoryData.begin(), posEnd = memoryData.end(); pos != posEnd; ++pos) {
        msg << *pos << "\n";
      }
      msg << "</MemoryService>\n";
      temporarilyCloseXML();
    }
  }

reportMessageInfo()

void edm::JobReport::reportMessageInfo

(

std::map< std::string, double > const &

messageData

)

Report Message statistics Invoked by the MessageLogger service to send an end of job summary about numbers of various categories messages issued for inclusion in the job report

Definition at line 604 of file JobReport.cc.

References impl_, CommonMethods::lock(), mps_check::msg, temporarilyCloseXML(), and write_mutex.

Referenced by edm::service::MessageLogger::summarizeInJobReport().

                                                                                {
    if (impl_->ost_) {
      std::lock_guard<std::mutex> lock(write_mutex);
      std::ostream& msg = *(impl_->ost_);
      msg << "<MessageSummary>\n";
      typedef std::map<std::string, double>::const_iterator const_iterator;
      for (const_iterator pos = messageData.begin(), posEnd = messageData.end(); pos != posEnd; ++pos) {
        msg << "  <" << pos->first << "  Value=\"" << pos->second << "\" />"
            << "\n";
      }
      msg << "</MessageSummary>\n";
      temporarilyCloseXML();
    }
  }

reportPerformanceForModule()

void edm::JobReport::reportPerformanceForModule	(	std::string const &	metricClass,
		std::string const &	moduleName,
		std::map< std::string, std::string > const &	metrics
	)

Definition at line 722 of file JobReport.cc.

References impl_, PostProcessorHGCAL_cfi::metrics, edm::moduleName(), mps_check::msg, and temporarilyCloseXML().

Referenced by XrdAdaptor::XrdStatisticsService::postEndJob().

                                                                                          {
    if (impl_->ost_) {
      std::ostream& msg = *(impl_->ost_);
      msg << "<PerformanceReport>\n"
          << "  <PerformanceModule Metric=\"" << metricClass << "\" "
          << " Module=\"" << moduleName << "\" >\n";

      typedef std::map<std::string, std::string>::const_iterator const_iterator;
      for (const_iterator iter = metrics.begin(), iterEnd = metrics.end(); iter != iterEnd; ++iter) {
        msg << "    <Metric Name=\"" << iter->first << "\" "
            << "Value=\"" << iter->second << "\"/>\n";
      }

      msg << "  </PerformanceModule>\n"
          << "</PerformanceReport>\n";
      temporarilyCloseXML();
    }
  }

reportPerformanceSummary()

void edm::JobReport::reportPerformanceSummary	(	std::string const &	metricClass,
		std::map< std::string, std::string > const &	metrics
	)

Performance Reports

Two categories: Summary for entire job and module for a specific module Each one requires a performance metric class such as Timing, Memory, CPU, Trigger etc.

Definition at line 703 of file JobReport.cc.

References impl_, PostProcessorHGCAL_cfi::metrics, mps_check::msg, and temporarilyCloseXML().

Referenced by TFileAdaptor::termination().

                                                                                        {
    if (impl_->ost_) {
      std::ostream& msg = *(impl_->ost_);
      msg << "<PerformanceReport>\n"
          << "  <PerformanceSummary Metric=\"" << metricClass << "\">\n";

      typedef std::map<std::string, std::string>::const_iterator const_iterator;
      for (const_iterator iter = metrics.begin(), iterEnd = metrics.end(); iter != iterEnd; ++iter) {
        msg << "    <Metric Name=\"" << iter->first << "\" "
            << "Value=\"" << iter->second << "\"/>\n";
      }

      msg << "  </PerformanceSummary>\n"
          << "</PerformanceReport>\n";
      temporarilyCloseXML();
    }
  }

reportRandomStateFile()

void edm::JobReport::reportRandomStateFile

(

std::string const &

name

)

Report the name of the random engine persistency file

Definition at line 689 of file JobReport.cc.

References hcalRecHitTable_cff::doc, impl_, CommonMethods::lock(), mps_check::msg, Skims_PA_cff::name, temporarilyCloseXML(), and write_mutex.

                                                             {
    tinyxml2::XMLDocument doc;
    if (impl_->ost_) {
      std::ostream& msg = *(impl_->ost_);
      {
        std::lock_guard<std::mutex> lock(write_mutex);
        msg << "<RandomServiceStateFile>\n"
            << doc.NewText(name.c_str())->Value() << "\n"
            << "</RandomServiceStateFile>\n";
        temporarilyCloseXML();
      }
    }
  }

reportReadBranch()

void edm::JobReport::reportReadBranch	(	InputType	inputType,
		std::string const &	branchName
	)

Inform the job report that a branch has been read.

Definition at line 662 of file JobReport.cc.

References electrons_cff::branchName, impl_, edm::Primary, edm::SecondaryFile, and edm::SecondarySource.

                                                                                   {
    if (inputType == InputType::Primary) {
      // Fast cloned branches have already been reported.
      std::set<std::string> const& clonedBranches =
          impl_->inputFiles_.at(impl_->lastOpenedPrimaryInputFile_).fastClonedBranches;
      if (clonedBranches.find(branchName) == clonedBranches.end()) {
        ++impl_->readBranches_[branchName];
      }
    } else if (inputType == InputType::SecondaryFile) {
      ++impl_->readBranchesSecFile_[branchName];
    } else if (inputType == InputType::SecondarySource) {
      ++impl_->readBranchesSecSource_[branchName].value();
    }
  }

reportReadBranches()

void edm::JobReport::reportReadBranches

(

)

Report read branches Invoked by the source that reads ROOT/EDM files at the end of a job to report which product branches in the Events tree have been read, with a count of the number of events for which the branch was read.

Definition at line 619 of file JobReport.cc.

References MicroEventContent_cff::branch, hcalRecHitTable_cff::doc, impl_, CommonMethods::lock(), temporarilyCloseXML(), and write_mutex.

                                     {
    bool expected = false;
    if (not impl_->printedReadBranches_.compare_exchange_strong(expected, true))
      return;
    if (impl_->ost_) {
      std::lock_guard<std::mutex> lock(write_mutex);
      std::ostream& ost = *(impl_->ost_);
      ost << "<ReadBranches>\n";
      tinyxml2::XMLDocument doc;
      tinyxml2::XMLPrinter printer;
      for (auto const& iBranch : impl_->readBranches_) {
        tinyxml2::XMLElement* branch = doc.NewElement("Branch");
        branch->SetAttribute("Name", iBranch.first.c_str());
        branch->SetAttribute("ReadCount", int64_t(iBranch.second));
        branch->Accept(&printer);
        ost << printer.CStr();
        printer.ClearBuffer();
      }
      for (auto const& iBranch : impl_->readBranchesSecFile_) {
        tinyxml2::XMLElement* branch = doc.NewElement("Branch");
        branch->SetAttribute("Name", iBranch.first.c_str());
        branch->SetAttribute("ReadCount", int64_t(iBranch.second));
        branch->Accept(&printer);
        ost << printer.CStr();
        printer.ClearBuffer();
      }
      ost << "</ReadBranches>\n";
      if (!impl_->readBranchesSecSource_.empty()) {
        ost << "<SecondarySourceReadBranches>\n";
        for (auto const& iBranch : impl_->readBranchesSecSource_) {
          tinyxml2::XMLElement* branch = doc.NewElement("Branch");
          branch->SetAttribute("Name", iBranch.first.c_str());
          branch->SetAttribute("ReadCount", int64_t(iBranch.second.value().load()));
          branch->Accept(&printer);
          ost << printer.CStr();
          printer.ClearBuffer();
        }
        ost << "</SecondarySourceReadBranches>\n";
      }
      temporarilyCloseXML();
    }
  }

reportRunNumber()

void edm::JobReport::reportRunNumber	(	JobReport::Token	token,
		unsigned int	run
	)

API for reporting a Run to the job report. for output files, call only if Run is written to the output file

Definition at line 505 of file JobReport.cc.

References impl_, writedatasetfile::run, and unpackBuffers-CaloStage2::token.

Referenced by NanoAODRNTupleOutputModule::writeRun(), NanoAODOutputModule::writeRun(), and DQMRootOutputModule::writeRun().

{ impl_->associateRun(token, run); }

reportShutdownSignal()

void edm::JobReport::reportShutdownSignal

(

)

Report a unix signal sent to early terminate the job

Definition at line 543 of file JobReport.cc.

References impl_, CommonMethods::lock(), mps_check::msg, temporarilyCloseXML(), and write_mutex.

Referenced by edm::EventProcessor::checkForAsyncStopRequest().

                                       {
    if (impl_->ost_) {
      {
        std::lock_guard<std::mutex> lock(write_mutex);
        std::ostream& msg = *(impl_->ost_);
        msg << "<ShutdownSignal/>\n";
        temporarilyCloseXML();
      }
    }
  }

reportSkippedEvent()

void edm::JobReport::reportSkippedEvent	(	RunNumber_t	run,
		EventNumber_t	event
	)

Definition at line 475 of file JobReport.cc.

References impl_, CommonMethods::lock(), mps_check::msg, writedatasetfile::run, temporarilyCloseXML(), and write_mutex.

                                                                         {
    if (impl_->ost_) {
      std::ostream& msg = *(impl_->ost_);
      {
        std::lock_guard<std::mutex> lock(write_mutex);
        msg << "<SkippedEvent Run=\"" << run << "\"";
        msg << " Event=\"" << event << "\" />\n";
        temporarilyCloseXML();
      }
    }
  }

reportSkippedFile()

void edm::JobReport::reportSkippedFile	(	std::string const &	pfn,
		std::string const &	lfn
	)

Report Skipped File

Report that a file has been skipped due to it not being found.

Definition at line 554 of file JobReport.cc.

References hcalRecHitTable_cff::doc, impl_, CommonMethods::lock(), mps_check::msg, runEdmFileComparison::skipped, temporarilyCloseXML(), and write_mutex.

                                                                              {
    if (impl_->ost_) {
      std::ostream& msg = *(impl_->ost_);
      tinyxml2::XMLDocument doc;
      tinyxml2::XMLPrinter printer;
      tinyxml2::XMLElement* skipped = doc.NewElement("SkippedFile");
      skipped->SetAttribute("Pfn", pfn.c_str());
      skipped->SetAttribute("Lfn", lfn.c_str());
      {
        std::lock_guard<std::mutex> lock(write_mutex);
        skipped->Accept(&printer);
        msg << printer.CStr();
        temporarilyCloseXML();
      }
    }
  }

temporarilyCloseXML()

void edm::JobReport::temporarilyCloseXML ( )

private

Definition at line 331 of file JobReport.cc.

References errorLogged_, impl_, and edm::kJobReportEndElement.

Referenced by inputFileClosed(), JobReport(), outputFileClosed(), reportAnalysisFile(), reportError(), reportFallbackAttempt(), reportMemoryInfo(), reportMessageInfo(), reportPerformanceForModule(), reportPerformanceSummary(), reportRandomStateFile(), reportReadBranches(), reportShutdownSignal(), reportSkippedEvent(), and reportSkippedFile().

                                      {
    if (impl_->ost_) {
      //remember where we were
      auto pos = impl_->ost_->tellp();
      if (not errorLogged_) {
        *(impl_->ost_) << "<FrameworkError ExitStatus=\"8901\" Type=\"UnexpectedJobTermination\"/>\n";
      }
      *(impl_->ost_) << kJobReportEndElement << std::flush;

      //overwrite above during next write.
      impl_->ost_->seekp(pos);
    }
  }

Member Data Documentation

errorLogged_

bool edm::JobReport::errorLogged_ = false

private

Definition at line 438 of file JobReport.h.

Referenced by reportError(), and temporarilyCloseXML().

impl_

edm::propagate_const<std::unique_ptr<JobReportImpl> > edm::JobReport::impl_

private

Definition at line 436 of file JobReport.h.

Referenced by dumpFiles(), eventReadFromFile(), eventWrittenToFile(), impl(), inputFileClosed(), inputFileOpened(), JobReport(), outputFileClosed(), outputFileOpened(), reportAnalysisFile(), reportDataType(), reportError(), reportFallbackAttempt(), reportFastClonedBranches(), reportFastCopyingStatus(), reportInputLumiSection(), reportInputRunNumber(), reportLumiSection(), reportMemoryInfo(), reportMessageInfo(), reportPerformanceForModule(), reportPerformanceSummary(), reportRandomStateFile(), reportReadBranch(), reportReadBranches(), reportRunNumber(), reportShutdownSignal(), reportSkippedEvent(), reportSkippedFile(), temporarilyCloseXML(), and ~JobReport().

write_mutex

std::mutex edm::JobReport::write_mutex

private

Definition at line 437 of file JobReport.h.

Referenced by inputFileClosed(), outputFileClosed(), reportAnalysisFile(), reportError(), reportFallbackAttempt(), reportMemoryInfo(), reportMessageInfo(), reportRandomStateFile(), reportReadBranches(), reportShutdownSignal(), reportSkippedEvent(), and reportSkippedFile().