Functions | |
def | fill_time_info (mps_index, status, cpu_time) |
Variables | |
bjobs | |
condor_h | |
condor_q | |
cpu_time = float(match.group(1))ifmatchelse0 | |
cputime_regex = re.compile(r"TheCPUtimeusedis(\d+(\.\d+)?)seconds") | |
string | disabled = "DISABLED" |
loop over remaining jobs to see whether they are done More... | |
dictionary | htcondor_jobstatus |
mapping of HTCondor status codes to MPS status More... | |
job_id = job_regex.search(line).group(1) | |
job_regex = re.compile(r"Job<(\d+?)>,") | |
dictionary | job_status = {} |
deal with submitted jobs by looking into output of shell (bjobs/condor_q) More... | |
lib = mpslib.jobdatabase() | |
collect submitted jobs (use 'in' to handle composites, e.g. More... | |
match = cputime_regex.search(line) | |
mps_index = submitted_jobs.get(job_id, -1) | |
results = bjobs.replace(" ","").split("-----------------------") | |
status = status_regex.search(line).group(1) | |
status_regex = re.compile(r"Status<([A-Z]+?)>") | |
bool | submitted = True |
dictionary | submitted_jobs = {} |
string | theBatchDirectory = "LSFJOB_" |
userlog = os.path.join("jobData", lib.JOBDIR[mps_index], "HTCJOB") | |
def mps_update.fill_time_info | ( | mps_index, | |
status, | |||
cpu_time | |||
) |
Fill timing info in the database for `mps_index`. Arguments: - `mps_index`: index in the MPS database - `status`: job status - `cpu_time`: extracted CPU timing information
Definition at line 8 of file mps_update.py.
References createfilelist.int, and harvestTrackValidationPlots.str.
mps_update.bjobs |
Definition at line 71 of file mps_update.py.
mps_update.condor_h |
Definition at line 131 of file mps_update.py.
mps_update.condor_q |
Definition at line 63 of file mps_update.py.
int mps_update.cpu_time = float(match.group(1))ifmatchelse0 |
Definition at line 67 of file mps_update.py.
mps_update.cputime_regex = re.compile(r"TheCPUtimeusedis(\d+(\.\d+)?)seconds") |
Definition at line 77 of file mps_update.py.
string mps_update.disabled = "DISABLED" |
loop over remaining jobs to see whether they are done
Definition at line 96 of file mps_update.py.
dictionary mps_update.htcondor_jobstatus |
mapping of HTCondor status codes to MPS status
Definition at line 32 of file mps_update.py.
mps_update.job_id = job_regex.search(line).group(1) |
Definition at line 67 of file mps_update.py.
mps_update.job_regex = re.compile(r"Job<(\d+?)>,") |
Definition at line 75 of file mps_update.py.
dictionary mps_update.job_status = {} |
deal with submitted jobs by looking into output of shell (bjobs/condor_q)
Definition at line 61 of file mps_update.py.
mps_update.lib = mpslib.jobdatabase() |
collect submitted jobs (use 'in' to handle composites, e.g.
DISABLEDFETCH)
Definition at line 43 of file mps_update.py.
mps_update.match = cputime_regex.search(line) |
Definition at line 87 of file mps_update.py.
mps_update.mps_index = submitted_jobs.get(job_id, -1) |
Definition at line 94 of file mps_update.py.
mps_update.results = bjobs.replace(" ","").split("-----------------------") |
Definition at line 79 of file mps_update.py.
Referenced by TriggerBxMonitor.analyze(), GetTrackTrajInfo.analyze(), TriggerRatesMonitor.analyze(), L1TriggerJSONMonitoring.analyze(), HLTriggerJSONMonitoring.analyze(), HcalTrigTowerGeometry.detIds(), ESElectronicsSim.encode(), LA_Filler_Fitter.ensemble_results(), edm::storage::StatisticsSenderService.filePreCloseEvent(), PulseShapeFitOOTPileupCorrection.fit(), MuScleFitUtils.fitMass(), MuScleFitUtils.fitReso(), CSCWireHitSim.getIonizationClusters(), edm::LuminosityBlock.getManyByType(), edm::Run.getManyByType(), edm::PrincipalGetAdapter.getManyByType(), edm::Event.getManyByType(), PhysicsTools::LeastSquares.getMeans(), OMTFReconstruction.getProcessorCandidates(), edm::Event.getRefBeforePut(), GlobalMuonRefitter.getRidOfSelectStationHits(), PhysicsTools::LeastSquares.getWeights(), pftools::Utils.Glob(), LA_Filler_Fitter.layer_results(), LA_Filler_Fitter.module_results(), Utils.myGlob(), CSCTFAlignmentOnlineProd.newObject(), L1MuGMTChannelMaskOnlineProducer.newObject(), L1GctJetFinderParamsOnlineProd.newObject(), CSCTFConfigOnlineProd.newObject(), L1MuCSCPtLutConfigOnlineProd.newObject(), L1GtPrescaleFactorsTechTrigConfigOnlineProd.newObject(), L1GtTriggerMaskAlgoTrigConfigOnlineProd.newObject(), L1GtTriggerMaskTechTrigConfigOnlineProd.newObject(), L1GtTriggerMaskVetoTechTrigConfigOnlineProd.newObject(), L1GtParametersConfigOnlineProd.newObject(), L1GtPrescaleFactorsAlgoTrigConfigOnlineProd.newObject(), ESRecHitSimAlgo.oldEvalAmplitude(), ESRecHitSimAlgo.oldreconstruct(), edm::Principal.prodGetter(), CalibratedPhotonProducerT< T >.produce(), CalibratedElectronProducerT< T >.produce(), MuIsolatorResultProducer< BT >.produce(), NuclearInteractionSimulator.read(), ESRecHitAnalyticAlgo.reconstruct(), ESRecHitSimAlgo.reconstruct(), ESRecHitFitAlgo.reconstruct(), PVFitter.runFitter(), edm::PrincipalGetAdapter.setProducer(), SiStripClusterInfo.stripNoisesRescaledByGain(), LA_Filler_Fitter.summarize_ensembles(), L1GtTriggerMenuConfigOnlineProd.tableMenuGeneralFromDB(), CSCCathodeLCTProcessor.testPatterns(), topLevelPSet(), HcalTrigTowerGeometry.towerIds(), sistrip::MeasureLA.write_report_text(), MuIsolatorResultProducer< BT >.writeOut(), and XMLConfigWriter.writeResultsData().
mps_update.status = status_regex.search(line).group(1) |
Definition at line 67 of file mps_update.py.
Referenced by XMLProcessor._toXMLCh(), addFilesWithFork(), ESTimingTask.analyze(), CSCTFAnalyzer.analyze(), DTNoiseCalibration.analyze(), BasicHepMCValidation.analyze(), CTPPSDiamondDQMSource.analyze(), TotemRPDQMSource.analyze(), WriteL1TriggerObjectsTxt.analyze(), MCEfficiencyAnalyzer.analyze(), FixTrackHitPattern.analyze(), L1TCSCTF.analyze(), L1Validator.analyze(), HLTriggerJSONMonitoring.analyze(), FWCaloDataHistProxyBuilder.assertCaloDataSlice(), gen::PhotosInterface.attachParticles(), EcalBoundaryInfoCalculator< EcalDetId >.boundaryRecHits(), gen::AMPTHadronizer.build_ampt(), TtFullHadHypKinFit.buildHypo(), TtSemiLepHypHitFit.buildHypo(), TtSemiLepHypKinFit.buildHypo(), CaloGeometryHelper.buildNeighbourArray(), PFMETBenchmark.calculateQuantities(), EcalRecHitWorkerRecover.checkChannelStatus(), pat::UserData.checkDictionaries(), QTestStatusChecker.checkGlobalStatus(), lhef::LHEEvent.checkHepMCTree(), HFTimingTrust.checkHFTimErr(), SiPixelFedCablingTree.checkNumbering(), EcalBoundaryInfoCalculator< EBDetId >.checkRecHitHasDeadNeighbour(), EgammaHLTNxNClusterProducer.checkStatusOfEcalRecHit(), HLTEcalResonanceFilter.checkStatusOfEcalRecHit(), HLTRegionalEcalResonanceFilter.checkStatusOfEcalRecHit(), tensorflow.closeSession(), NuclearVertexBuilder.closestApproach(), edm::service.cmssw_stacktrace_fork(), compare_l1candidate(), dtCalibration::DTVDriftSegment.compute(), HFShower.compute(), HDShower.compute(), EMShower.compute(), EcalHitMaker.configureGeometry(), ErrorChecker.conversionError(), dtCalibration::DTTTrigFillWithAverage.correction(), dtCalibration::DTTTrigMatchRPhi.correction(), dtCalibration::DTT0FillDefaultFromDB.correction(), dtCalibration::DTTTrigConstantShift.correction(), dtCalibration::DTT0AbsoluteReferenceCorrection.correction(), dtCalibration::DTT0ChamberReferenceCorrection.correction(), dtCalibration::DTT0FillChamberFromDB.correction(), dtCalibration::DTT0FEBPathCorrection.correction(), dtCalibration::DTT0WireInChamberReferenceCorrection.correction(), dtCalibration::DTTTrigT0SegCorrection.correction(), dtCalibration::DTTTrigResidualCorrection.correction(), TMatacq.countBadPulses(), CounterChecker.CounterChecker(), tensorflow.createSession(), CSCDCCFormatStatusDigi.CSCDCCFormatStatusDigi(), DDIsValid(), gen::TauolappInterface.decay(), LzmaFile.DecodeAll(), LzmaFile.DecodeBuffer(), dtCalibration::DTT0AbsoluteReferenceCorrection.defaultT0(), dtCalibration::DTT0ChamberReferenceCorrection.defaultT0(), dtCalibration::DTT0WireInChamberReferenceCorrection.defaultT0(), dtCalibration::DTT0FEBPathCorrection.defaultT0(), Demangle.Demangle(), EcalHitMaker.diagonalEdge(), doBZ2Compression(), edm::EDLooperBase.doDuringLoop(), PulseChiSqSNNLS.DoFit(), SiPixelGainCalibrationAnalysis.doFits(), EmissionVetoHook1.doVetoFSREmission(), EmissionVetoHook1.doVetoISREmission(), doZlibCompression(), PFECALHashNavigator.ecalNeighbArray(), DTVDriftWriter.endJob(), DTT0Correction.endJob(), DTTTrigCorrection.endJob(), LogicTool< RBCLogic >.endjob(), edm::EventProcessor.endOfLoop(), TMatacq.enterdata(), ErrorChecker.errorDetId(), ESRecHitAnalyticAlgo.EvalAmplitude(), ESRecHitFitAlgo.EvalAmplitude(), ESRecHitSimAlgo.evalAmplitude(), PhysicsTools::MVAComputer.evalInternal(), ExternalLHEProducer.executeScript(), EcalHitMaker.fastInsideCell(), MuScleFitPlotter.fillGen(), Py8toJetInput.fillJetAlgoInput(), LHETablesProducer.fillLHEObjectTable(), PFElectronTranslator.fillMVAValueMap(), TrackingQualityChecker.fillTrackingStatus(), TrackingQualityChecker.fillTrackingStatusAtLumi(), PythiaFilterIsolatedTrack.filter(), PythiaFilterTTBar.filter(), EcalDeadCellBoundaryEnergyFilter.filter(), Phase2Tracker::Phase2TrackerFEDBuffer.findChannels(), HepMCValidationHelper.findFSRPhotons(), L1GtVhdlWriterCore.findObjectType(), GenParticlesHelper.findParticles(), edm::Path.finished(), UnbinnedLikelihoodFit.fit(), TtFullHadKinFitter::KinFit.fit(), BSFitter.Fit_d0phi(), PixelDataFormatter.formatRawData(), reco::GenParticle.fromHardProcessFinalState(), Phase2Tracker::Phase2TrackerFEDHeader.frontendStatus(), EcalBoundaryInfoCalculator< EcalDetId >.gapRecHits(), GaussianTailNoiseGenerator.GaussianTailNoiseGenerator(), GaussNoiseProducerFP420.generate(), GaussianTailNoiseGenerator.generate(), gen::BaseHadronizer.generateLHE(), reco::GenParticle.GenParticle(), pat::PATObject< ObjectType >.genParticleById(), pat::PATObject< reco::Muon >.genParticleRef(), DTTtrig.get(), DTMtime.get(), XrdAdaptor::ClientRequest.get_future(), Py8toJetInput.getAncestor(), dtCalibration::DTTTrigFillWithAverage.getAverage(), ESTrivialConditionRetriever.getChannelStatusFromConfiguration(), EcalTrivialConditionRetriever.getChannelStatusFromConfiguration(), EcalDeadCellTriggerPrimitiveFilter.getChannelStatusMaps(), EcalDeadCellDeltaRFilter.getChannelStatusMaps(), FWGUIManager.getContext(), pat::EventHypothesis.getDemangledSymbol(), cscdqm::Summary.GetEfficiencyHW(), PixelSLinkDataInputSource.getEventNumberFromFillWords(), HLTTauMCProducer.getGenDecayProducts(), XrdAdaptor::RequestManager.getInstance(), SiPixelInformationExtractor.getItemValue(), CrossingPtBasedLinearizationPointFinder.getLinearizationPoint(), TrackingUtility.getMEStatus(), SiStripUtility.getMEStatus(), DTTPGParametersHandler.getNewObjects(), DTDeadFlagHandler.getNewObjects(), DTPerformanceHandler.getNewObjects(), DTRangeT0Handler.getNewObjects(), DTReadOutMappingHandler.getNewObjects(), DTT0Handler.getNewObjects(), DTStatusFlagHandler.getNewObjects(), FedRawDataInputSource.getNextEvent(), cscdqm::Summary.GetReportingArea(), SiPixelUtility.getStatus(), DQMStore.getStatus(), SiPixelUtility.getStatusColor(), EcalTrivialConditionRetriever.getTrgChannelStatusFromConfiguration(), XrdAdaptor::Source.handle(), edm::Path.handleEarlyFinish(), MakerResponseHandler.HandleResponse(), XrdAdaptor::ClientRequest.HandleResponse(), SendMonitoringInfoHandler.HandleResponse(), DelayedClose.HandleResponseWithHosts(), XrdAdaptor::RequestManager::OpenHandler.HandleResponseWithHosts(), GenParticlesHelper.hasAncestor(), CalorimetryManager.HDShowerSimulation(), Generator.HepMC2G4(), HcalHFStatusBitFromRecHits.hfSetFlagFromRecHits(), InduceChargeFP420.induce(), fireworks::Context.initEveElements(), RBCBasicConfig.initialise(), TTUBasicConfig.initialise(), RBCLogicUnit.initialise(), TTULogicUnit.initialise(), LogicTool< RBCLogic >.initialise(), RPCWheel.initialise(), RBCEmulator.initialise(), TTUEmulator.initialise(), XrdAdaptor::RequestManager.initialize(), Pythia8Hadronizer.initializeForExternalPartons(), Pythia8Hadronizer.initializeForInternalPartons(), edm::ParameterDescriptionCases< T >.insert(), edm::ParameterSwitchBase.insertAndCheckLabels(), MBUEandQCDValidation.isCharged(), HepMCValidationHelper.isChargedLepton(), EcalDeadCellDeltaRFilter.isCloseToBadEcalChannel(), AnomalousECALVariables.isDeadEcalCluster(), GenParticlesHelper.isDirect(), reco::GenParticle.isDirectHardProcessTauDecayProductFinalState(), reco::GenParticle.isDirectPromptTauDecayProductFinalState(), MBUEandQCDValidation.isNeutral(), HepMCValidationHelper.isNeutrino(), MBUEandQCDValidation.isNeutrino(), npstat.isNonDecreasing(), npstat.isNonIncreasing(), cscdqm::Summary.IsPhysicsReady(), reco::GenParticle.isPromptFinalState(), npstat.isStrictlyDecreasing(), AbsHcalFunctor.isStrictlyDecreasing(), npstat.isStrictlyIncreasing(), AbsHcalFunctor.isStrictlyIncreasing(), FFTJetProducer.iterateJetReconstruction(), reco::PFCandidateElectronExtra.kfTrackRef(), tensorflow.loadGraphDef(), tensorflow.loadMetaGraph(), PFRecoTauDiscriminationAgainstElectronDeadECAL.loopXtals(), main(), lhef::LHEEvent.makeHepMCParticle(), gen.GeneratorAnalyzer.GeneratorAnalyzer.makeMCInfo(), EcalUncalibRecHitMultiFitAlgo.makeRecHit(), reco::MCMatchSelector< T1, T2 >.MCMatchSelector(), MergeRootfile(), PulseChiSqSNNLS.Minimize(), CaloGeometryHelper.move(), FedRawDataInputSource.nextEvent(), tensorflow::NTSession.NTSession(), ESRecHitSimAlgo.oldEvalAmplitude(), ESRecHitSimAlgo.oldreconstruct(), XrdFile.open(), XrdAdaptor::RequestManager::OpenHandler.open(), TEcnaRead.OpenRootFile(), TEcnaRun.OpenRootFile(), TSCBLBuilderNoMaterial.operator()(), GenJetParticleSelector.operator()(), FastTimerService::ResourcesPerPath.operator+=(), trigger::TriggerObject.particle(), FFTJetCorrectionProducer.performPileupSubtraction(), pos::PixelDetectorConfig.PixelDetectorConfig(), SymmetryFit.pol2_from_pol2(), SymmetryFit.pol2_from_pol3(), edm::OccurrenceTraits< EventPrincipal, BranchActionStreamBegin >.postPathSignal(), EPOS::EPOS_Wrapper.print_hepcom_particle(), HepMCFileReader.printEvent(), TMatacq.printitermatacqData(), RBCChamberORLogic.process(), RPCWheel.process(), edm::EventProcessor.processEventWithLooper(), WeakEffectsWeightProducer.produce(), PseudoTopProducer.produce(), TtSemiLepKinFitProducer< LeptonCollection >.produce(), ISRGammaWeightProducer.produce(), ISRWeightProducer.produce(), PdfWeightProducer.produce(), PFElectronTranslator.produce(), RPCUnpackingModule.produce(), CSCTFUnpacker.produce(), Onia2MuMuPAT.produce(), PFPhotonTranslator.produce(), FlavorHistoryProducer.produce(), l1t::GenToInputProducer.produce(), RPCTechnicalTrigger.produce(), FFTJetProducer.produce(), EcalTrivialConditionRetriever.produceEcalDAQTowerStatus(), EcalTrivialConditionRetriever.produceEcalDCSTowerStatus(), ecaldqm::CalibrationSummaryClient.producePlots(), ecaldqm::SummaryClient.producePlots(), Herwig6Hadronizer.pythiaStatusCode(), StoreEcalCondition.readEcalChannelStatusFromFile(), StoreESCondition.readESChannelStatusFromFile(), DTReadOutMapping.readOutToGeometry(), FedRawDataInputSource.readSupervisor(), IOInput.readv(), EcalDCSTowerStatusXMLTranslator.readXML(), EcalDAQTowerStatusXMLTranslator.readXML(), ESRecHitAnalyticAlgo.reconstruct(), ESRecHitSimAlgo.reconstruct(), ESRecHitFitAlgo.reconstruct(), XrdAdaptor::RequestManager.requestFailure(), FastTimerService::ResourcesPerPath.reset(), gen::Cascade2Hadronizer.residualDecay(), gen::Pythia6Hadronizer.residualDecay(), TEcnaRootFile.ReStart(), L1GtVhdlWriterCore.returnConditionsOfOneClass(), dqmservices::TriggerSelector::TreeElement.returnStatus(), Pythia8PartonSelector.run(), tensorflow.run(), run_script(), ecaldqm::RawDataTask.runOnRawData(), DQMStore.save(), SiStripInformationExtractor.selectColor(), SiStripInformationExtractor.selectImage(), HcalSourcePositionData.set(), DTStatusFlag.setCellDead(), DTDeadFlag.setCellDead_HV(), DTDeadFlag.setCellDead_RO(), DTDeadFlag.setCellDead_TP(), DTDeadFlag.setCellDiscCat(), DTStatusFlag.setCellFEMask(), DTStatusFlag.setCellNoHV(), DTStatusFlag.setCellNoise(), DTStatusFlag.setCellTDCMask(), DTStatusFlag.setCellTrigMask(), ESRecHitSimAlgo.setChannelStatus(), ESRecHitAnalyticAlgo.setChannelStatus(), ESRecHitFitAlgo.setChannelStatus(), FWLegoViewBase.setContext(), EcalDeadCellTriggerPrimitiveFilter.setEvtRecHitstatus(), EcalDeadCellTriggerPrimitiveFilter.setEvtTPstatus(), ESDigitizer.setGain(), EcalTBHodoscopePlaneRawHits.setHit(), reco::CSCHaloData.SetHLTBit(), FFTJetProducer.setJetStatusBit(), gen::TauolappInterface.setRandomEngine(), MonDelaysTTDat.setTaskStatus(), MonPedestalsOnlineDat.setTaskStatus(), MonPedestalOffsetsDat.setTaskStatus(), MonPNPedDat.setTaskStatus(), MonLed2Dat.setTaskStatus(), MonLed1Dat.setTaskStatus(), MonLaserIRedDat.setTaskStatus(), MonLaserBlueDat.setTaskStatus(), MonLaserRedDat.setTaskStatus(), MonLaserGreenDat.setTaskStatus(), MonMemChConsistencyDat.setTaskStatus(), MonCrystalConsistencyDat.setTaskStatus(), MonPedestalsDat.setTaskStatus(), MonTTConsistencyDat.setTaskStatus(), MonTestPulseDat.setTaskStatus(), MonMemTTConsistencyDat.setTaskStatus(), MonPNLed2Dat.setTaskStatus(), MonPNIRedDat.setTaskStatus(), MonPNGreenDat.setTaskStatus(), MonPNLed1Dat.setTaskStatus(), MonPNBlueDat.setTaskStatus(), MonPNRedDat.setTaskStatus(), MonPNMGPADat.setTaskStatus(), ITimingDat.setTaskStatus(), HBHETimingShapedFlagSetter.SetTimingShapedFlags(), reco::PFCandidateEGammaExtra.singleLegConversionRef(), XrdStorageMaker.stagein(), pftools::Utils.StringMatch(), Utils.stringMatch(), SymmetryFit.symmetryChi2(), tensorflow::TBBSession.TBBSession(), TEcnaRootFile.TEcnaRootFile(), CSCFitAFEBThr.ThresholdNoise(), HBHETimingShapedFlagSetter.timingStatus(), TMatacq.TMatacq(), HLTPerformanceInfo.totalPathCPUTime(), HLTPerformanceInfo.totalPathTime(), edm.typeDemangle(), pat::UserData.typeNameFor(), HcaluLUTTPGCoder.update(), edm::service::CondorStatusService.updateChirpImpl(), PulseChiSqSNNLS.updateCov(), CrossingPtBasedLinearizationPointFinder.useAllTracks(), tensorflow::TBBSession.WaitForNotification(), tensorflow::NTSession.WaitForNotification(), PixelDataFormatter.word2digi(), cscdqm::Summary.Write(), npstat::LinInterpolatedTableND< Numeric, Axis >.write(), EPOS::IO_EPOS.write_event(), gs.writeCompressedStringArchive(), gs.writeStringArchive(), and CrystalWindowMap.~CrystalWindowMap().
mps_update.status_regex = re.compile(r"Status<([A-Z]+?)>") |
Definition at line 76 of file mps_update.py.
bool mps_update.submitted = True |
Definition at line 48 of file mps_update.py.
dictionary mps_update.submitted_jobs = {} |
Definition at line 46 of file mps_update.py.
string mps_update.theBatchDirectory = "LSFJOB_" |
Definition at line 121 of file mps_update.py.
mps_update.userlog = os.path.join("jobData", lib.JOBDIR[mps_index], "HTCJOB") |
Definition at line 130 of file mps_update.py.