CMS 3D CMS Logo

 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Properties Friends Macros Groups Pages
Functions | Variables
mps_update Namespace Reference

Functions

def fill_time_info
 

Variables

tuple condor_h
 
tuple condor_q
 
string disabled = "DISABLED"
 
dictionary htcondor_jobstatus
 mapping of HTCondor status codes to MPS status More...
 
dictionary job_status = {}
 deal with submitted jobs by looking into output of shell (condor_q) More...
 
tuple lib = mpslib.jobdatabase()
 collect submitted jobs (use 'in' to handle composites, e.g. More...
 
tuple mps_index = submitted_jobs.get(job_id, -1)
 
list status = htcondor_jobstatus[status]
 
 submitted = True
 
dictionary submitted_jobs = {}
 
dictionary submitted_jobs_copy = { k:v for k,v in submitted_jobs.items() }
 loop over remaining jobs to see whether they are done More...
 
tuple userlog = os.path.join("jobData", lib.JOBDIR[mps_index], "HTCJOB")
 

Function Documentation

def mps_update.fill_time_info (   mps_index,
  status,
  cpu_time 
)
Fill timing info in the database for `mps_index`.

Arguments:
- `mps_index`: index in the MPS database
- `status`: job status
- `cpu_time`: extracted CPU timing information

Definition at line 10 of file mps_update.py.

References str.

10 
11 def fill_time_info(mps_index, status, cpu_time):
12  """Fill timing info in the database for `mps_index`.
13 
14  Arguments:
15  - `mps_index`: index in the MPS database
16  - `status`: job status
17  - `cpu_time`: extracted CPU timing information
18  """
19 
20  cpu_time = int(round(cpu_time)) # care only about seconds for now
21  if status in ("RUN", "DONE"):
22  if cpu_time > 0:
23  diff = cpu_time - lib.JOBRUNTIME[mps_index]
24  lib.JOBRUNTIME[mps_index] = cpu_time
25  lib.JOBHOST[mps_index] = "+"+str(diff)
26  lib.JOBINCR[mps_index] = diff
27  else:
28  lib.JOBRUNTIME[mps_index] = 0
29  lib.JOBINCR[mps_index] = 0
30 
31 
def fill_time_info
Definition: mps_update.py:10
#define str(s)

Variable Documentation

tuple mps_update.condor_h
Initial value:
1 = subprocess.check_output(["condor_history", job_id, "-limit", "1",
2  "-userlog", userlog,
3  "-af:j", "JobStatus", "RemoteSysCpu"],
4  stderr = subprocess.STDOUT)

Definition at line 101 of file mps_update.py.

tuple mps_update.condor_q
Initial value:
1 = subprocess.check_output(["condor_q", "-af:j",
2  "JobStatus", "RemoteSysCpu"],
3  stderr = subprocess.STDOUT)

Definition at line 64 of file mps_update.py.

string mps_update.disabled = "DISABLED"

Definition at line 75 of file mps_update.py.

dictionary mps_update.htcondor_jobstatus
Initial value:
1 = {"1": "PEND", # Idle
2  "2": "RUN", # Running
3  "3": "EXIT", # Removed
4  "4": "DONE", # Completed
5  "5": "PEND", # Held
6  "6": "RUN", # Transferring output
7  "7": "PEND"}

mapping of HTCondor status codes to MPS status

Definition at line 34 of file mps_update.py.

dictionary mps_update.job_status = {}

deal with submitted jobs by looking into output of shell (condor_q)

Definition at line 63 of file mps_update.py.

tuple mps_update.lib = mpslib.jobdatabase()

collect submitted jobs (use 'in' to handle composites, e.g.

DISABLEDFETCH)

Definition at line 45 of file mps_update.py.

tuple mps_update.mps_index = submitted_jobs.get(job_id, -1)

Definition at line 73 of file mps_update.py.

list mps_update.status = htcondor_jobstatus[status]

Definition at line 107 of file mps_update.py.

Referenced by XMLProcessor._toXMLCh(), addFilesWithFork(), cms::cuda::HostAllocator< T, FLAGS >.allocate(), ESTimingTask.analyze(), CSCTFAnalyzer.analyze(), BeamSpotProblemMonitor.analyze(), TKStatus.analyze(), SiPixelGenErrorDBObjectReader.analyze(), BasicHepMCValidation.analyze(), DTNoiseCalibration.analyze(), TotemRPDQMSource.analyze(), WriteL1TriggerObjectsTxt.analyze(), L1TCSCTF.analyze(), L1Validator.analyze(), CTPPSDiamondDQMSource.analyze(), HLTriggerJSONMonitoring.analyze(), L1TrackObjectNtupleMaker.analyze(), FWCaloDataHistProxyBuilder.assertCaloDataSlice(), gen::PhotosInterface.attachParticles(), edm::EventProcessor.beginLumiAsync(), EcalBoundaryInfoCalculator< EcalDetId >.boundaryRecHits(), gen::AMPTHadronizer.build_ampt(), TtFullHadHypKinFit.buildHypo(), TtSemiLepHypHitFit.buildHypo(), TtSemiLepHypKinFit.buildHypo(), CaloGeometryHelper.buildNeighbourArray(), PFMETBenchmark.calculateQuantities(), EcalRecHitWorkerRecover.checkChannelStatus(), pat::UserData.checkDictionaries(), edm.checkForModuleDependencyCorrectness(), lhef::LHEEvent.checkHepMCTree(), HFTimingTrust.checkHFTimErr(), SiPixelFedCablingTree.checkNumbering(), EcalBoundaryInfoCalculator< EBDetId >.checkRecHitHasDeadNeighbour(), EgammaHLTNxNClusterProducer.checkStatusOfEcalRecHit(), HLTEcalResonanceFilter.checkStatusOfEcalRecHit(), HLTRegionalEcalResonanceFilter.checkStatusOfEcalRecHit(), l1tVertexFinder::VertexNTupler::GenParticlesBranchData.clear(), tensorflow.closeSession(), edm::service.cmssw_stacktrace_fork(), compare_l1candidate(), dtCalibration::DTVDriftSegment.compute(), HFShower.compute(), HDShower.compute(), EMShower.compute(), EcalHitMaker.configureGeometry(), edm::EventProcessor.continueLumiAsync(), dtCalibration::DTTTrigMatchRPhi.correction(), dtCalibration::DTTTrigFillWithAverage.correction(), dtCalibration::DTTTrigConstantShift.correction(), dtCalibration::DTT0FillChamberFromDB.correction(), dtCalibration::DTT0FillDefaultFromDB.correction(), dtCalibration::DTT0AbsoluteReferenceCorrection.correction(), dtCalibration::DTTTrigT0SegCorrection.correction(), dtCalibration::DTT0ChamberReferenceCorrection.correction(), dtCalibration::DTT0FEBPathCorrection.correction(), dtCalibration::DTT0WireInChamberReferenceCorrection.correction(), dtCalibration::DTTTrigResidualCorrection.correction(), TMatacq.countBadPulses(), tensorflow.createSession(), CUDAService.CUDAService(), DDIsValid(), cms::cuda::HostAllocator< T, FLAGS >.deallocate(), gen::TauolappInterface.decay(), LzmaFile.DecodeAll(), LzmaFile.DecodeBuffer(), dtCalibration::DTT0AbsoluteReferenceCorrection.defaultT0(), dtCalibration::DTT0ChamberReferenceCorrection.defaultT0(), dtCalibration::DTT0WireInChamberReferenceCorrection.defaultT0(), dtCalibration::DTT0FEBPathCorrection.defaultT0(), Demangle.Demangle(), EcalHitMaker.diagonalEdge(), doBZ2Compression(), edm::EDLooperBase.doDuringLoop(), PulseChiSqSNNLS.DoFit(), SiPixelGainCalibrationAnalysis.doFits(), EmissionVetoHook1.doVetoFSREmission(), EmissionVetoHook1.doVetoISREmission(), doZlibCompression(), TKStatus.dumpTkDcsStatus(), PFECALHashNavigator.ecalNeighbArray(), DTT0Correction.endJob(), DTVDriftWriter.endJob(), DTTTrigCorrection.endJob(), edm::EventProcessor.endOfLoop(), TMatacq.enterdata(), ErrorChecker.errorDetId(), ErrorCheckerPhase0.errorDetId(), ESRecHitAnalyticAlgo.EvalAmplitude(), ESRecHitFitAlgo.EvalAmplitude(), ESRecHitSimAlgo.evalAmplitude(), PhysicsTools::MVAComputer.evalInternal(), ExternalLHEProducer.executeScript(), EcalHitMaker.fastInsideCell(), MuScleFitPlotter.fillGen(), Py8toJetInput.fillJetAlgoInput(), LHETablesProducer.fillLHEObjectTable(), PFElectronTranslator.fillMVAValueMap(), TrackingQualityChecker.fillTrackingStatus(), TrackingQualityChecker.fillTrackingStatusAtLumi(), PythiaFilterTTBar.filter(), PythiaFilterIsolatedTrack.filter(), EcalDeadCellBoundaryEnergyFilter.filter(), Phase2Tracker::Phase2TrackerFEDBuffer.findChannels(), HepMCValidationHelper.findFSRPhotons(), L1GtVhdlWriterCore.findObjectType(), edm::Path.finished(), UnbinnedLikelihoodFit.fit(), TtFullHadKinFitter::KinFit.fit(), BSFitter.Fit_d0phi(), for(), EcalLaserCondTools.from_hdf_to_db(), Phase2Tracker::Phase2TrackerFEDHeader.frontendStatus(), EcalBoundaryInfoCalculator< EcalDetId >.gapRecHits(), GaussNoiseProducerFP420.generate(), gen::BaseHadronizer.generateLHE(), pat::PATObject< ObjectType >.genParticleById(), DTTtrig.get(), DTMtime.get(), Py8toJetInput.getAncestor(), dtCalibration::DTTTrigFillWithAverage.getAverage(), ESTrivialConditionRetriever.getChannelStatusFromConfiguration(), EcalTrivialConditionRetriever.getChannelStatusFromConfiguration(), EcalDeadCellTriggerPrimitiveFilter.getChannelStatusMaps(), EcalDeadCellDeltaRFilter.getChannelStatusMaps(), ErrorCheckerBase.getConversionErrorTypeAndIssueLogMessage(), pat::EventHypothesis.getDemangledSymbol(), cscdqm::Summary.GetEfficiencyHW(), PixelSLinkDataInputSource.getEventNumberFromFillWords(), HLTTauMCProducer.getGenDecayProducts(), CrossingPtBasedLinearizationPointFinder.getLinearizationPoint(), TrackingUtility.getMEStatus(), SiStripUtility.getMEStatus(), DTPerformanceHandler.getNewObjects(), DTStatusFlagHandler.getNewObjects(), DTT0Handler.getNewObjects(), DTTPGParametersHandler.getNewObjects(), DTRangeT0Handler.getNewObjects(), DTDeadFlagHandler.getNewObjects(), DTReadOutMappingHandler.getNewObjects(), FedRawDataInputSource.getNextEvent(), cscdqm::Summary.GetReportingArea(), SiPixelUtility.getStatus(), EcalTrivialConditionRetriever.getTrgChannelStatusFromConfiguration(), edm::EventProcessor.globalEndLumiAsync(), lhef::H5Handler.H5Handler(), XrdAdaptor::Source.handle(), edm::EventProcessor.handleNextEventForStreamAsync(), XrdAdaptor::ClientRequest.HandleResponse(), SendMonitoringInfoHandler.HandleResponse(), DelayedClose.HandleResponseWithHosts(), XrdAdaptor::RequestManager::OpenHandler.HandleResponseWithHosts(), egamma.hasActiveHcal(), GenParticlesHelper.hasAncestor(), CalorimetryManager.HDShowerSimulation(), Generator.HepMC2G4(), HcalHFStatusBitFromRecHits.hfSetFlagFromRecHits(), InduceChargeFP420.induce(), PFBadHcalPseudoClusterProducer.init(), fireworks::Context.initEveElements(), RBCBasicConfig.initialise(), TTUBasicConfig.initialise(), LogicTool< GenLogic >.initialise(), RBCEmulator.initialise(), RPCWheel.initialise(), TTUEmulator.initialise(), XrdAdaptor::RequestManager.initialize(), Pythia8Hadronizer.initializeForExternalPartons(), Pythia8Hadronizer.initializeForInternalPartons(), edm::ParameterDescriptionCases< T >.insert(), edm::ParameterSwitchBase.insertAndCheckLabels(), MBUEandQCDValidation.isCharged(), HepMCValidationHelper.isChargedLepton(), EcalDeadCellDeltaRFilter.isCloseToBadEcalChannel(), AnomalousECALVariables.isDeadEcalCluster(), GenParticlesHelper.isDirect(), MBUEandQCDValidation.isNeutral(), MBUEandQCDValidation.isNeutrino(), HepMCValidationHelper.isNeutrino(), npstat.isNonDecreasing(), npstat.isNonIncreasing(), cscdqm::Summary.IsPhysicsReady(), npstat.isStrictlyDecreasing(), AbsHcalFunctor.isStrictlyDecreasing(), npstat.isStrictlyIncreasing(), AbsHcalFunctor.isStrictlyIncreasing(), FFTJetProducer.iterateJetReconstruction(), tensorflow.loadGraphDef(), tensorflow.loadMetaGraphDef(), main(), lhef::LHEEvent.makeHepMCParticle(), gen.GeneratorAnalyzer.GeneratorAnalyzer.makeMCInfo(), EcalUncalibRecHitMultiFitAlgo.makeRecHit(), reco::MCMatchSelector< T1, T2 >.MCMatchSelector(), MergeRootfile(), PulseChiSqSNNLS.Minimize(), CaloGeometryHelper.move(), FedRawDataInputSource.nextEvent(), ESRecHitSimAlgo.oldEvalAmplitude(), ESRecHitSimAlgo.oldreconstruct(), edm::storage::XrdFile.open(), XrdAdaptor::RequestManager::OpenHandler.open(), TEcnaRead.OpenRootFile(), TEcnaRun.OpenRootFile(), TSCBLBuilderNoMaterial.operator()(), GenJetParticleSelector.operator()(), FastTimerService::ResourcesPerPath.operator+=(), trigger::TriggerObject.particle(), FFTJetCorrectionProducer.performPileupSubtraction(), pos::PixelDetectorConfig.PixelDetectorConfig(), PixelTemplateSmearerBase.PixelTemplateSmearerBase(), SymmetryFit.pol2_from_pol2(), SymmetryFit.pol2_from_pol3(), edm::OccurrenceTraits< EventPrincipal, BranchActionStreamBegin >.postPathSignal(), HepMCFileReader.printEvent(), TMatacq.printitermatacqData(), RBCChamberORLogic.process(), RPCWheel.process(), edm::EventProcessor.processEventWithLooper(), TtSemiLepKinFitProducer< LeptonCollection >.produce(), PseudoTopProducer.produce(), EcalRecHitParametersGPUESProducer.produce(), PFElectronTranslator.produce(), RPCUnpackingModule.produce(), CSCTFUnpacker.produce(), PFPhotonTranslator.produce(), Onia2MuMuPAT.produce(), L1TkFastVertexProducer.produce(), FlavorHistoryProducer.produce(), l1t::GenToInputProducer.produce(), RPCTechnicalTrigger.produce(), FFTJetProducer.produce(), HcalHardcodeCalibrations.produceChannelQuality(), EcalTrivialConditionRetriever.produceEcalDAQTowerStatus(), EcalTrivialConditionRetriever.produceEcalDCSTowerStatus(), ecaldqm::CalibrationSummaryClient.producePlots(), ecaldqm::SummaryClient.producePlots(), Herwig6Hadronizer.pythiaStatusCode(), StoreEcalCondition.readEcalChannelStatusFromFile(), StoreESCondition.readESChannelStatusFromFile(), DTReadOutMapping.readOutToGeometry(), FedRawDataInputSource.readSupervisor(), edm::storage::Storage.readv(), EcalDCSTowerStatusXMLTranslator.readXML(), EcalDAQTowerStatusXMLTranslator.readXML(), ESRecHitSimAlgo.reconstruct(), ESRecHitAnalyticAlgo.reconstruct(), ESRecHitFitAlgo.reconstruct(), XrdAdaptor::RequestManager.requestFailure(), FastTimerService::ResourcesPerPath.reset(), gen::Cascade2Hadronizer.residualDecay(), gen::Pythia6Hadronizer.residualDecay(), TEcnaRootFile.ReStart(), L1GtVhdlWriterCore.returnConditionsOfOneClass(), dqmservices::TriggerSelector::TreeElement.returnStatus(), Pythia8PartonSelector.run(), tensorflow.run(), run_script(), ecaldqm::RawDataTask.runOnRawData(), HcalSourcePositionData.set(), DTStatusFlag.setCellDead(), DTDeadFlag.setCellDead_HV(), DTDeadFlag.setCellDead_RO(), DTDeadFlag.setCellDead_TP(), DTDeadFlag.setCellDiscCat(), DTStatusFlag.setCellFEMask(), DTStatusFlag.setCellNoHV(), DTStatusFlag.setCellNoise(), DTStatusFlag.setCellTDCMask(), DTStatusFlag.setCellTrigMask(), ESRecHitSimAlgo.setChannelStatus(), ESRecHitAnalyticAlgo.setChannelStatus(), ESRecHitFitAlgo.setChannelStatus(), FWLegoViewBase.setContext(), EcalDeadCellTriggerPrimitiveFilter.setEvtRecHitstatus(), EcalDeadCellTriggerPrimitiveFilter.setEvtTPstatus(), ESDigitizer.setGain(), EcalTBHodoscopePlaneRawHits.setHit(), reco::CSCHaloData.SetHLTBit(), FFTJetProducer.setJetStatusBit(), MonPedestalsOnlineDat.setTaskStatus(), MonDelaysTTDat.setTaskStatus(), MonPedestalOffsetsDat.setTaskStatus(), MonLed1Dat.setTaskStatus(), MonLed2Dat.setTaskStatus(), MonLaserBlueDat.setTaskStatus(), MonLaserIRedDat.setTaskStatus(), MonLaserGreenDat.setTaskStatus(), MonPNPedDat.setTaskStatus(), MonLaserRedDat.setTaskStatus(), MonMemChConsistencyDat.setTaskStatus(), MonCrystalConsistencyDat.setTaskStatus(), MonPedestalsDat.setTaskStatus(), MonMemTTConsistencyDat.setTaskStatus(), MonTTConsistencyDat.setTaskStatus(), MonTestPulseDat.setTaskStatus(), MonPNRedDat.setTaskStatus(), MonPNLed1Dat.setTaskStatus(), MonPNIRedDat.setTaskStatus(), MonPNBlueDat.setTaskStatus(), MonPNLed2Dat.setTaskStatus(), MonPNGreenDat.setTaskStatus(), MonPNMGPADat.setTaskStatus(), ITimingDat.setTaskStatus(), HBHETimingShapedFlagSetter.SetTimingShapedFlags(), edm::EventProcessor.streamEndLumiAsync(), SymmetryFit.symmetryChi2(), TEcnaRootFile.TEcnaRootFile(), cms::cudatest.testDevices(), CSCFitAFEBThr.ThresholdNoise(), HBHETimingShapedFlagSetter.timingStatus(), TMatacq.TMatacq(), HLTPerformanceInfo.totalPathCPUTime(), HLTPerformanceInfo.totalPathTime(), edm.typeDemangle(), pat::UserData.typeNameFor(), HcaluLUTTPGCoder.update(), edm::service::CondorStatusService.updateChirpImpl(), PulseChiSqSNNLS.updateCov(), CrossingPtBasedLinearizationPointFinder.useAllTracks(), cscdqm::Summary.Write(), npstat::LinInterpolatedTableND< Numeric, Axis >.write(), EPOS::IO_EPOS.write_event(), gs.writeCompressedStringArchive(), and gs.writeStringArchive().

mps_update.submitted = True

Definition at line 50 of file mps_update.py.

dictionary mps_update.submitted_jobs = {}

Definition at line 48 of file mps_update.py.

dictionary mps_update.submitted_jobs_copy = { k:v for k,v in submitted_jobs.items() }

loop over remaining jobs to see whether they are done

Definition at line 93 of file mps_update.py.

tuple mps_update.userlog = os.path.join("jobData", lib.JOBDIR[mps_index], "HTCJOB")

Definition at line 100 of file mps_update.py.