Functions
def	check_interesting (mename)

def	dasquery (dataset)

def	harvestfile (fname)

def	tosqlite (x)

Variables
	args

	ctr

	db

	default

	description

	dumpmes

	end

	files

	help

	inf

	insertinto

	int

	interesting_mes

	interesting_types

	makeindex

	maketable

	nan

	parser

	pool

	processes

	ROOTPREFIX

	sqlite2tree

	treenames

	type

Function Documentation

◆ check_interesting()

def dqmiodatasetharvest.check_interesting ( mename )

Definition at line 41 of file dqmiodatasetharvest.py.

Referenced by harvestfile().

 def check_interesting(mename):
   for pattern in interesting_mes:
     if fnmatch.fnmatch(mename,pattern):
       return True
   return False
 

◆ dasquery()

def dqmiodatasetharvest.dasquery ( dataset )

Definition at line 70 of file dqmiodatasetharvest.py.

References print().

 def dasquery(dataset):
     if not dataset.endswith("DQMIO"):
         raise Exception("This tool probably cannot read the dataset you specified. The name should end with DQMIO.")
     dasquery = ["dasgoclient",  "-query=file dataset=%s" % dataset]
     print("Querying das ... %s" % dasquery)
     files = subprocess.check_output(dasquery)
     files = files.splitlines()
     print("Got %d files." % len(files))
     return files
 
 

◆ harvestfile()

def dqmiodatasetharvest.harvestfile ( fname )

Definition at line 123 of file dqmiodatasetharvest.py.

References check_interesting(), WZElectronSkims53X_cff.max, SiStripPI.min, isotrackApplyRegressor.range, str, and tosqlite().

 def harvestfile(fname):
     f = ROOT.TFile.Open(ROOTPREFIX + fname)
     idxtree = getattr(f, "Indices")
     #idxtree.GetEntry._threaded = True # now the blocking call should release the GIL...
 
     # we have no good way to find out which lumis where processed in a job.
     # so we watch the per-lumi indices and assume that all mentioned lumis 
     # are covered in the end-of-job MEs. This might fail if there are no 
     # per-lumi MEs.
     knownlumis = set()
     mes_to_store = []
 
     for i in range(idxtree.GetEntries()):
         idxtree.GetEntry(i)
         run, lumi, metype = idxtree.Run, idxtree.Lumi, idxtree.Type
         if lumi != 0:
             knownlumis.add(lumi)
 
         if not treenames[metype] in interesting_types:
           continue
 
         endrun = run # assume no multi-run files for now
         if lumi == 0: # per-job ME
             endlumi = max(knownlumis)
             lumi = min(knownlumis)
         else: 
             endlumi = lumi
 
         # inclusive range -- for 0 entries, row is left out
         firstidx, lastidx = idxtree.FirstIndex, idxtree.LastIndex
         metree = getattr(f, treenames[metype])
         metree.GetEntry(0)
         metree.SetBranchStatus("*",0)
         metree.SetBranchStatus("FullName",1)
 
         for x in range(firstidx, lastidx+1):
             metree.GetEntry(x)
             mename = str(metree.FullName)
 
             if mename.find("AlCaReco") != -1: 
               continue
 
             if mename.find("Isolated") != -1:
               continue
             
             if mename.find("HLT") != -1:
               continue
             
             if not ((mename.find("SiStrip") >= 0) or (mename.find("OfflinePV") >= 0) or (mename.find("PixelPhase1") >= 0) or (mename.find("Tracking") >= 0 )):    
               continue
 
             if check_interesting(mename):
                 metree.GetEntry(x, 1)
                 value = metree.Value
 
                 mes_to_store.append((
                   mename,
                   run, lumi, endrun, endlumi,
                   metype,
                   tosqlite(value),
                 ))
 
     return mes_to_store
 

◆ tosqlite()

def dqmiodatasetharvest.tosqlite ( x )

Definition at line 47 of file dqmiodatasetharvest.py.

References edmScanValgrind.buffer, and betterConfigParser.unicode.

Referenced by harvestfile().

 def tosqlite(x):
     if isinstance(x, ROOT.string):
         try:
             return unicode(x.data())
         except:
             return buffer(x.data())
     if isinstance(x, int):
         return x
     if isinstance(x, float):
         return x
     if isinstance(x, int):
         return x
     else:
         try: 
             rootobj = unicode(ROOT.TBufferJSON.ConvertToJSON(x))
             # turns out ROOT does not generate valid JSON for NaN/inf
             clean = nan.sub('\\g<1>0\\g<2>', inf.sub('\\g<1>1e38\\g<2>', rootobj))
             obj = json.loads(clean)
             jsonobj = json.dumps(obj, allow_nan=False)
             return jsonobj
         except Exception as e:
             return json.dumps({"root2sqlite_error": e.__repr__(), "root2sqlite_object": x.__repr__()})
 

Variable Documentation

◆ args

dqmiodatasetharvest.args

Definition at line 21 of file dqmiodatasetharvest.py.

◆ ctr

dqmiodatasetharvest.ctr

Definition at line 191 of file dqmiodatasetharvest.py.

Referenced by CaloCellCrossing.CaloCellCrossing(), Tracker_OldtoNewConverter.createMap(), MultiVertexFitter.fit(), MuonGeometryNumbering.geoHistoryToBaseNumber(), cms::MuonNumbering.geoHistoryToBaseNumber(), PreshowerStrip.initCorners(), EcalTBHodoscopeGeometryLoaderFromDDD.makeGeometry(), HLTPerformanceInfo.moduleIndexInPath(), MultiClusterizer1D< T >.operator()(), MuonPathFitter.RomDataConvert(), HLTPerformanceInfo.setStatusOfModulesFromPath(), edm::service::tracer.setupFile(), and edm::service::moduleAlloc.setupFile().

◆ db

dqmiodatasetharvest.db

Definition at line 119 of file dqmiodatasetharvest.py.

◆ default

dqmiodatasetharvest.default

Definition at line 18 of file dqmiodatasetharvest.py.

◆ description

dqmiodatasetharvest.description

Definition at line 15 of file dqmiodatasetharvest.py.

◆ dumpmes

dqmiodatasetharvest.dumpmes

Definition at line 115 of file dqmiodatasetharvest.py.

◆ end

dqmiodatasetharvest.end

Definition at line 197 of file dqmiodatasetharvest.py.

◆ files

dqmiodatasetharvest.files

Definition at line 187 of file dqmiodatasetharvest.py.

◆ help

dqmiodatasetharvest.help

Definition at line 17 of file dqmiodatasetharvest.py.

◆ inf

dqmiodatasetharvest.inf

Definition at line 38 of file dqmiodatasetharvest.py.

◆ insertinto

dqmiodatasetharvest.insertinto

Definition at line 106 of file dqmiodatasetharvest.py.

◆ int

dqmiodatasetharvest.int

Definition at line 19 of file dqmiodatasetharvest.py.

◆ interesting_mes

dqmiodatasetharvest.interesting_mes

Definition at line 32 of file dqmiodatasetharvest.py.

◆ interesting_types

dqmiodatasetharvest.interesting_types

Definition at line 25 of file dqmiodatasetharvest.py.

◆ makeindex

dqmiodatasetharvest.makeindex

Definition at line 103 of file dqmiodatasetharvest.py.

◆ maketable

dqmiodatasetharvest.maketable

Definition at line 96 of file dqmiodatasetharvest.py.

◆ nan

dqmiodatasetharvest.nan

Definition at line 39 of file dqmiodatasetharvest.py.

Referenced by L1TMuon::GeometryTranslator.calculateBendAngle(), L1TMuon::GeometryTranslator.calculateGlobalEta(), L1TMuon::GeometryTranslator.calculateGlobalPhi(), L1TMuon::GeometryTranslator.getGlobalPoint(), PlotAlignmentValidation.plotDMR(), pat::XGBooster.reset(), PlotAlignmentValidation.setDMRHistStyleAndLegend(), and RPCFw.UTtoT().

◆ parser

dqmiodatasetharvest.parser

Definition at line 15 of file dqmiodatasetharvest.py.

◆ pool

dqmiodatasetharvest.pool

Definition at line 190 of file dqmiodatasetharvest.py.

◆ processes

dqmiodatasetharvest.processes

Definition at line 190 of file dqmiodatasetharvest.py.

Referenced by edm::RootFile.dropOnInputAndReorder(), edm.insertSelectedProcesses(), GenLumiInfoProduct.mergeProduct(), FastTimerService::ResourcesPerJob.operator+=(), FastTimerService::ResourcesPerJob.reset(), edm::RootFile.RootFile(), edm::RootFile.setPresenceInProductRegistry(), edm::PoolOutputModule.setProcessesWithSelectedMergeableRunProducts(), GenLumiInfoProduct.setProcessInfo(), and TauDQMHistPlotter.TauDQMHistPlotter().

◆ ROOTPREFIX

dqmiodatasetharvest.ROOTPREFIX

Definition at line 12 of file dqmiodatasetharvest.py.

◆ sqlite2tree

dqmiodatasetharvest.sqlite2tree

Definition at line 200 of file dqmiodatasetharvest.py.

◆ treenames

dqmiodatasetharvest.treenames

Definition at line 81 of file dqmiodatasetharvest.py.

◆ type

dqmiodatasetharvest.type