dqmiodatasetharvest Namespace Reference

Functions
def	dasquery (dataset)
def	harvestfile (fname)
def	tosqlite (x)

Variables
	args
	ctr
	db
	default
	description
	dumpmes
	end
	files
	help
	inf
	insertinto
	int
	interesting_mes
	interesting_types
	makeindex
	maketable
	nan
	parser
	pool
	processes
	ROOTPREFIX
	sqlite2tree
	treenames
	type

Function Documentation

def dqmiodatasetharvest.dasquery

(

dataset

)

Definition at line 126 of file dqmiodatasetharvest.py.

References edm.print().

def dasquery(dataset):
    if not dataset.endswith("DQMIO"):
        raise Exception("This tool probably cannot read the dataset you specified. The name should end with DQMIO.")
    dasquery = ["dasgoclient",  "-query=file dataset=%s" % dataset]
    print("Querying das ... %s" % dasquery)
    files = subprocess.check_output(dasquery)
    files = files.splitlines()
    print("Got %d files." % len(files))
    return files

def dqmiodatasetharvest.harvestfile

(

fname

)

Definition at line 179 of file dqmiodatasetharvest.py.

References SiStripPI.max, min(), FastTimerService_cff.range, str, and tosqlite().

def harvestfile(fname):
    f = ROOT.TFile.Open(ROOTPREFIX + fname)
    idxtree = getattr(f, "Indices")
    #idxtree.GetEntry._threaded = True # now the blocking call should release the GIL...

    # we have no good way to find out which lumis where processed in a job.
    # so we watch the per-lumi indices and assume that all mentioned lumis 
    # are covered in the end-of-job MEs. This might fail if there are no 
    # per-lumi MEs.
    knownlumis = set()
    mes_to_store = []

    for i in range(idxtree.GetEntries()):
        idxtree.GetEntry(i)
        run, lumi, metype = idxtree.Run, idxtree.Lumi, idxtree.Type
        if lumi != 0:
            knownlumis.add(lumi)

        if not treenames[metype] in interesting_types:
            continue


        endrun = run # assume no multi-run files for now
        if lumi == 0: # per-job ME
            endlumi = max(knownlumis)
            lumi = min(knownlumis)
        else: 
            endlumi = lumi

        # inclusive range -- for 0 entries, row is left out
        firstidx, lastidx = idxtree.FirstIndex, idxtree.LastIndex
        metree = getattr(f, treenames[metype])
        metree.SetBranchStatus("*",0)
        metree.SetBranchStatus("FullName",1)

        for x in range(firstidx, lastidx+1):
            metree.GetEntry(x)
            mename = str(metree.FullName)
            if mename in interesting_mes:
                metree.GetEntry(x, 1)
                value = metree.Value

                mes_to_store.append((
                  mename,
                  run, lumi, endrun, endlumi,
                  metype,
                  tosqlite(value),
                ))

    return mes_to_store

def dqmiodatasetharvest.tosqlite

(

x

)

Definition at line 103 of file dqmiodatasetharvest.py.

References edmScanValgrind.buffer.

Referenced by harvestfile().

def tosqlite(x):
    if isinstance(x, ROOT.string):
        try:
            return unicode(x.data())
        except:
            return buffer(x.data())
    if isinstance(x, int):
        return x
    if isinstance(x, float):
        return x
    if isinstance(x, long):
        return x
    else:
        try: 
            rootobj = unicode(ROOT.TBufferJSON.ConvertToJSON(x))
            # turns out ROOT does not generate valid JSON for NaN/inf
            clean = nan.sub('\\g<1>0\\g<2>', inf.sub('\\g<1>1e38\\g<2>', rootobj))
            obj = json.loads(clean)
            jsonobj = json.dumps(obj, allow_nan=False)
            return jsonobj
        except Exception as e:
            return json.dumps({"root2sqlite_error": e.__repr__(), "root2sqlite_object": x.__repr__()})

Variable Documentation

dqmiodatasetharvest.args

Definition at line 21 of file dqmiodatasetharvest.py.

dqmiodatasetharvest.ctr

Definition at line 234 of file dqmiodatasetharvest.py.

Referenced by CaloCellCrossing.CaloCellCrossing(), Tracker_OldtoNewConverter.createMap(), MultiVertexFitter.fit(), cms::MuonNumbering.geoHistoryToBaseNumber(), PreshowerStrip.initCorners(), EcalTBHodoscopeGeometryLoaderFromDDD.makeGeometry(), HLTPerformanceInfo.moduleIndexInPath(), MuonGeometryArrange.MuonGeometryArrange(), MultiClusterizer1D< T >.operator()(), HLTPerformanceInfo.setStatusOfModulesFromPath(), TrackerGeometryCompare.TrackerGeometryCompare(), and AdaptiveVertexReconstructor.vertices().

dqmiodatasetharvest.db

Definition at line 175 of file dqmiodatasetharvest.py.

Referenced by EcalSRCondTools.analyze(), l1t::CaloConfigHelper.CaloConfigHelper(), SiStripCommissioningOfflineDbClient.createHistos(), SiStripCondObjBuilderFromDb.dbParams(), DEcompare< T >.DEcompare(), TEveEllipsoidProjectedGL.DrawRhoZ(), EcalCondDBReader.EcalCondDBReader(), EcalCondDBWriter.EcalCondDBWriter(), Fit_MaximumPoint(), HcalLutManager.get_brickSet_from_oracle(), HcalFeatureHFEMBit.getE(), HcalQIEManager.getHfQieTable(), HcalLutManager.getLutFromXml_old(), HcalQIEManager.getTableFromDb(), HcalLutManager.init(), edm::service::LoadAllDictionaries.LoadAllDictionaries(), EcalUncalibRecHitFixedAlphaBetaAlgo< C >.PerformAnalyticFit(), PFTauElecRejectionBenchmarkAnalyzer.PFTauElecRejectionBenchmarkAnalyzer(), HcalZSAlgoEnergy.prepare(), HiEvtPlaneFlatProducer.produce(), EvtPlaneProducer.produce(), publishObjects(), SiPixelGenError.pushfile(), SiPixelTemplate2D.pushfile(), SiPixelTemplate.pushfile(), HcalLutManager.read_luts(), HcalZeroSuppressionAlgo.setDbService(), HcalLutManager.test_xml_access(), TotemTimingConversions.timeOfFirstSample(), TotemTimingConversions.timePrecision(), l1t::CaloConfigHelper.UpdatePayload(), TotemTimingConversions.voltSamples(), and HcalLutManager.~HcalLutManager().

dqmiodatasetharvest.default

Definition at line 18 of file dqmiodatasetharvest.py.

dqmiodatasetharvest.description

Definition at line 15 of file dqmiodatasetharvest.py.

dqmiodatasetharvest.dumpmes

Definition at line 171 of file dqmiodatasetharvest.py.

dqmiodatasetharvest.end

Definition at line 240 of file dqmiodatasetharvest.py.

dqmiodatasetharvest.files

Definition at line 230 of file dqmiodatasetharvest.py.

dqmiodatasetharvest.help

Definition at line 17 of file dqmiodatasetharvest.py.

dqmiodatasetharvest.inf

Definition at line 100 of file dqmiodatasetharvest.py.

Referenced by HFShower.compute(), HDShower.compute(), main(), ParametrizedSubtractor.ParametrizedSubtractor(), PFResolutionMap.ReadMapFile(), and FedRawDataInputSource.readSupervisor().

dqmiodatasetharvest.insertinto

Definition at line 162 of file dqmiodatasetharvest.py.

dqmiodatasetharvest.int

Definition at line 19 of file dqmiodatasetharvest.py.

dqmiodatasetharvest.interesting_mes

Definition at line 29 of file dqmiodatasetharvest.py.

dqmiodatasetharvest.interesting_types

Definition at line 25 of file dqmiodatasetharvest.py.

dqmiodatasetharvest.makeindex

Definition at line 159 of file dqmiodatasetharvest.py.

dqmiodatasetharvest.maketable

Definition at line 152 of file dqmiodatasetharvest.py.

dqmiodatasetharvest.nan

Definition at line 101 of file dqmiodatasetharvest.py.

Referenced by L1TMuon::TTGeometryTranslator.calculateBendAngle(), L1TMuon::GeometryTranslator.calculateBendAngle(), L1TMuon::TTGeometryTranslator.calculateGlobalEta(), L1TMuon::GeometryTranslator.calculateGlobalEta(), L1TMuon::TTGeometryTranslator.calculateGlobalPhi(), L1TMuon::GeometryTranslator.calculateGlobalPhi(), L1TMuon::TTGeometryTranslator.getGlobalPoint(), L1TMuon::GeometryTranslator.getGlobalPoint(), and RPCFw.UTtoT().

dqmiodatasetharvest.parser

Definition at line 15 of file dqmiodatasetharvest.py.

dqmiodatasetharvest.pool

Definition at line 233 of file dqmiodatasetharvest.py.

Referenced by FWDialogBuilder.addLabel(), fireworks.boldGC(), CmsShowCommonPopup.CmsShowCommonPopup(), CentralityTableProducer.endJob(), fireworks.italicGC(), and CmsShowMainFrame.showFWorksInfo().

dqmiodatasetharvest.processes

Definition at line 233 of file dqmiodatasetharvest.py.

Referenced by GenLumiInfoProduct.mergeProduct(), GenLumiInfoProduct.setProcessInfo(), and TauDQMHistPlotter.TauDQMHistPlotter().

dqmiodatasetharvest.ROOTPREFIX

Definition at line 12 of file dqmiodatasetharvest.py.

dqmiodatasetharvest.sqlite2tree

Definition at line 243 of file dqmiodatasetharvest.py.

dqmiodatasetharvest.treenames

Definition at line 137 of file dqmiodatasetharvest.py.

dqmiodatasetharvest.type

Definition at line 19 of file dqmiodatasetharvest.py.