TkMap_script_phase1 Namespace Reference

Functions
def	downloadnlineDQMhisto (run, Run_type)
def	downloadOfflineDQMhisto (run, Run_type, rereco)
def	downloadOfflinePCLhisto (run, Run_type)
def	getGT (DQMfile, RunNumber, globalTagVar)
def	setRunDirectory (runNumber)

Variables
	check_command
	Check if run is complete##############. More...
	Check_output
	checkfolder
	Start making TkMaps################. More...
	CMSSW_BASE
	DataLocalDir
	deadRocMap
	dest
	Producing the list of bad modules. More...
	detIdInfoFileName
	File_Name
	File_Name_online
	File_Name_PCL
	filepath
	full_pixelfile_name
	full_stripfile_name
	globalTag
	Getting GT############## After switch production to 10_X_X release, the clean up section need to be reviewed and modified ##########. More...
	input_var
	nnnOut
	Downloading DQM file############################. More...
	output
	put color legend in the TrackerMap################### More...
	pixel_files
	Start making pixel maps#####################. More...
	pixelTreeFileName
	rereco
	Check if user enter the right run type######################. More...
	Run_Number
	Run_type
	runDirval
	sefile
	rename bad module list file###################### More...
	shell
	strip_files
	create merged list of BadComponent from (PCL, RunInfo and FED Errors) ignore for now More...
	workPath

Function Documentation

downloadnlineDQMhisto()

def TkMap_script_phase1.downloadnlineDQMhisto	(		run,
			Run_type
	)

Definition at line 93 of file TkMap_script_phase1.py.

def downloadnlineDQMhisto(run, Run_type):
    runDirval=setRunDirectory(run)
    DataLocalDir=runDirval[0]
    DataOfflineDir=runDirval[1]
    nnn=run/100
    nnnOnline = run/10000
    File_Name_online=''
    deadRocMap = False
    
    url1 = 'https://cmsweb.cern.ch/dqm/online/data/browse/Original/000'+str(nnnOnline)+'xxxx/000'+str(nnn)+'xx/'
    os.popen("curl -k --cert /data/users/cctrkdata/current/auth/proxy/proxy.cert --key /data/users/cctrkdata/current/auth/proxy/proxy.cert -X GET "+url1+" > index_online.html")
    
    url2 = 'https://cmsweb.cern.ch/dqm/offline/data/browse/ROOT/OnlineData/original/000'+str(nnnOnline)+'xxxx/000'+str(nnn)+'xx/'
    os.popen("curl -k --cert /data/users/cctrkdata/current/auth/proxy/proxy.cert --key /data/users/cctrkdata/current/auth/proxy/proxy.cert -X GET "+url2+" > index_online_backup.html")
    f_online_backup=codecs.open("index_online_backup.html", 'r')
    index_online_backup = f_online_backup.readlines()
 
    f_online=codecs.open("index_online.html", 'r')
    index_online = f_online.readlines()
    if any(str(run) in x for x in index_online):
        for x in index_online:
            if (str(run) in x) and ("_PixelPhase1_" in x):
                File_Name_online=str(str(x).split(".root'>")[1].split("</a></td><td>")[0])
                deadRocMap = True
    else:
        print("Can't find any file in offline server, trying the online server")
        if any(str(run) in y for y in index_online_backup):
            for y in index_online:
                if (str(run) in y) and ("_PixelPhase1_" in y):
                    File_Name_online=str(str(y).split(".root'>")[1].split("</a></td><td>")[0])
                    deadRocMap = True
        else:
            print('No Online DQM file available. Skip dead roc map')
            deadRocMap = False
 
    print('Downloading DQM file:'+File_Name_online)
 
 
    os.system('curl -k --cert /data/users/cctrkdata/current/auth/proxy/proxy.cert --key /data/users/cctrkdata/current/auth/proxy/proxy.cert -X GET https://cmsweb.cern.ch/dqm/online/data/browse/Original/000'+str(nnnOnline)+'xxxx/000'+str(nnn)+'xx/'+File_Name_online+' > /tmp/'+File_Name_online)
 
    os.remove('index_online.html')
    os.remove('index_online_backup.html')
    return deadRocMap, File_Name_online;
 
 

References any(), print(), setRunDirectory(), submitPVValidationJobs.split(), and str.

downloadOfflineDQMhisto()

def TkMap_script_phase1.downloadOfflineDQMhisto	(		run,
			Run_type,
			rereco
	)

Definition at line 37 of file TkMap_script_phase1.py.

def downloadOfflineDQMhisto(run, Run_type,rereco):
    runDirval=setRunDirectory(run)
    DataLocalDir=runDirval[0]
    DataOfflineDir=runDirval[1]
    nnn=run/100
    print('Processing '+ Run_type + ' in '+DataOfflineDir+"...")
    File_Name = ''
    print('Directory to fetch the DQM file from: https://cmsweb.cern.ch/dqm/offline/data/browse/ROOT/OfflineData/'+DataOfflineDir+'/'+Run_type+'/000'+str(nnn)+'xx/')
    url = 'https://cmsweb.cern.ch/dqm/offline/data/browse/ROOT/OfflineData/'+DataOfflineDir+'/'+Run_type+'/000'+str(nnn)+'xx/'
    os.popen("curl -k --cert /data/users/cctrkdata/current/auth/proxy/proxy.cert --key /data/users/cctrkdata/current/auth/proxy/proxy.cert -X GET "+url+" > index.html") 
    f=codecs.open("index.html", 'r')
    index = f.readlines()
    if any(str(Run_Number[i]) in s for s in index):
        for s in index:
            if rereco:
                if (str(Run_Number[i]) in s) and ("__DQMIO.root" in s) and ("17Sep2018" in s):
                    File_Name = str(str(s).split("xx/")[1].split("'>DQM")[0])
            else:
                if (str(Run_Number[i]) in s) and ("__DQMIO.root" in s):
                    File_Name = str(str(s).split("xx/")[1].split("'>DQM")[0])
 
    else:
        print('No DQM file available. Please check the Offline server')
        sys.exit(0)
 
    print('Downloading DQM file:'+File_Name)
    os.system('curl -k --cert /data/users/cctrkdata/current/auth/proxy/proxy.cert --key /data/users/cctrkdata/current/auth/proxy/proxy.cert -X GET https://cmsweb.cern.ch/dqm/offline/data/browse/ROOT/OfflineData/'+DataOfflineDir+'/'+Run_type+'/000'+str(nnn)+'xx/'+File_Name+' > /tmp/'+File_Name)
    
    return File_Name
 

References any(), print(), setRunDirectory(), submitPVValidationJobs.split(), and str.

downloadOfflinePCLhisto()

def TkMap_script_phase1.downloadOfflinePCLhisto	(		run,
			Run_type
	)

Definition at line 67 of file TkMap_script_phase1.py.

def downloadOfflinePCLhisto(run, Run_type):
    runDirval=setRunDirectory(run)
    DataLocalDir=runDirval[0]
    DataOfflineDir=runDirval[1]
    nnn=run/100
    print('Processing '+ Run_type + ' in '+DataOfflineDir+"...")
    File_Name = 'Temp'
    print('Directory to fetch the DQM file from: https://cmsweb.cern.ch/dqm/offline/data/browse/ROOT/OfflineData/'+DataOfflineDir+'/'+Run_type+'/000'+str(nnn)+'xx/')
    url = 'https://cmsweb.cern.ch/dqm/offline/data/browse/ROOT/OfflineData/'+DataOfflineDir+'/'+Run_type+'/000'+str(nnn)+'xx/'
    os.popen("curl -k --cert /data/users/cctrkdata/current/auth/proxy/proxy.cert --key /data/users/cctrkdata/current/auth/proxy/proxy.cert -X GET "+url+" > index.html") 
    f=codecs.open("index.html", 'r')
    index = f.readlines()
    if any(str(Run_Number[i]) in s for s in index):
        for s in index:
            if (str(Run_Number[i]) in s) and ("PromptCalibProdSiPixel-Express" in s) and ("__ALCAPROMPT.root" in s):
                File_Name = str(str(s).split("xx/")[1].split("'>DQM")[0])
    else:
        print('No DQM file available. Please check the Offline server')
        sys.exit(0)
    if File_Name!='Temp':
        print('Downloading DQM file:'+File_Name)
        os.system('curl -k --cert /data/users/cctrkdata/current/auth/proxy/proxy.cert --key /data/users/cctrkdata/current/auth/proxy/proxy.cert -X GET https://cmsweb.cern.ch/dqm/offline/data/browse/ROOT/OfflineData/'+DataOfflineDir+'/'+Run_type+'/000'+str(nnn)+'xx/'+File_Name+' > /tmp/'+File_Name)
    
    return File_Name
 
 

References any(), print(), setRunDirectory(), submitPVValidationJobs.split(), and str.

getGT()

def TkMap_script_phase1.getGT	(		DQMfile,
			RunNumber,
			globalTagVar
	)

Definition at line 138 of file TkMap_script_phase1.py.

def getGT(DQMfile, RunNumber, globalTagVar):
    globalTag_v0 = getGTfromDQMFile(DQMfile, RunNumber, globalTagVar)
    print("Global Tag: " + globalTag_v0)
    globalTag = globalTag_v0
 
    for z in range(len(globalTag_v0)-2):#clean up the garbage string in the GT
        if (globalTag_v0[z].isdigit()) and  (globalTag_v0[z+1].isdigit()) and (globalTag_v0[z+2].isdigit()) and(globalTag_v0[z+3].isupper()):
            globalTag = globalTag_v0[z:]
            break
    if globalTag == "":
        print(" No GlobalTag found: trying from DAS.... ")
        globalTag = str(os.popen('getGTscript.sh '+filepath+ File_Name+' ' +str(Run_Number[i])));
        if globalTag == "":
            print(" No GlobalTag found for run: "+str(Run_Number[i]))
 
    return globalTag
 
 

References print(), FastTimerService_cff.range, and str.

setRunDirectory()

def TkMap_script_phase1.setRunDirectory

(

runNumber

)

Definition at line 16 of file TkMap_script_phase1.py.

def setRunDirectory(runNumber):
    # Don't forget to add an entry when there is a new era
    dirDict = { 325799:['Data2018', 'HIRun2018'],\
                315252:['Data2018', 'Run2018'],\
                308336:['Data2018', 'Commissioning2018'],\
                294644:['Data2017', 'Run2017'],\
                290123:['Data2017', 'Commissioning2017'],\
                284500:['Data2016', 'PARun2016'],\
                271024:['Data2016', 'Run2016'],\
                264200:['Data2016', 'Commissioning2016'],\
                246907:['Data2015', 'Run2015'],\
                232881:['Data2015', 'Commissioning2015'],\
                211658:['Data2013', 'Run2013'],\
                209634:['Data2013', 'HIRun2013'],\
                190450:['Data2012', 'Run2012']}
    runKey=0
    for key in sorted(dirDict):
        if runNumber > key:
            runKey=key
    return dirDict[runKey]  
 

Referenced by downloadnlineDQMhisto(), downloadOfflineDQMhisto(), and downloadOfflinePCLhisto().

Variable Documentation

check_command

TkMap_script_phase1.check_command

Check if run is complete##############.

Definition at line 191 of file TkMap_script_phase1.py.

Check_output

TkMap_script_phase1.Check_output

Definition at line 192 of file TkMap_script_phase1.py.

checkfolder

TkMap_script_phase1.checkfolder

Start making TkMaps################.

Definition at line 217 of file TkMap_script_phase1.py.

CMSSW_BASE

TkMap_script_phase1.CMSSW_BASE

Definition at line 158 of file TkMap_script_phase1.py.

Referenced by hitfit::JetTranslatorBase< pat::Jet >.JetTranslatorBase(), and hitfit::LeptonTranslatorBase< AElectron >.LeptonTranslatorBase().

DataLocalDir

TkMap_script_phase1.DataLocalDir

Definition at line 183 of file TkMap_script_phase1.py.

deadRocMap

TkMap_script_phase1.deadRocMap

Definition at line 180 of file TkMap_script_phase1.py.

dest

TkMap_script_phase1.dest

Producing the list of bad modules.

if Run_type != "StreamExpress": shutil.copyfile(sefile, checkdir+'/'+sefile) os.system('/usr/bin/python ${CMSSW_BASE}/src/DQM/SiStripMonitorClient/scripts/findBadModT9.py -p '+sefile+' -s /'+checkdir+'/'+sefile); Producing the run certification by lumisection Producing the PrimaryVertex/BeamSpot quality test by LS.. if (Run_type != "Cosmics") and ( Run_type != "StreamExpress") and (Run_type != "StreamExpressCosmics"): print(" Creating the BeamSpot Calibration certification summary:") os.system('lsbs_cert '+filepath+'/'+File_Name) .. and harvest the bad beamspot LS with automatic emailing (if in period and if bad LS found) Producing the Module difference for ExpressStream

Definition at line 297 of file TkMap_script_phase1.py.

detIdInfoFileName

TkMap_script_phase1.detIdInfoFileName

Definition at line 233 of file TkMap_script_phase1.py.

File_Name

TkMap_script_phase1.File_Name

Definition at line 177 of file TkMap_script_phase1.py.

File_Name_online

TkMap_script_phase1.File_Name_online

Definition at line 180 of file TkMap_script_phase1.py.

File_Name_PCL

TkMap_script_phase1.File_Name_PCL

Definition at line 179 of file TkMap_script_phase1.py.

filepath

TkMap_script_phase1.filepath

Definition at line 176 of file TkMap_script_phase1.py.

full_pixelfile_name

TkMap_script_phase1.full_pixelfile_name

Definition at line 383 of file TkMap_script_phase1.py.

full_stripfile_name

TkMap_script_phase1.full_stripfile_name

Definition at line 359 of file TkMap_script_phase1.py.

globalTag

TkMap_script_phase1.globalTag

Getting GT############## After switch production to 10_X_X release, the clean up section need to be reviewed and modified ##########.

Definition at line 226 of file TkMap_script_phase1.py.

input_var

TkMap_script_phase1.input_var

Definition at line 199 of file TkMap_script_phase1.py.

nnnOut

TkMap_script_phase1.nnnOut

Downloading DQM file############################.

Definition at line 174 of file TkMap_script_phase1.py.

output

TkMap_script_phase1.output

put color legend in the TrackerMap###################

Copying the template html file to index.html########################

Definition at line 265 of file TkMap_script_phase1.py.

pixel_files

TkMap_script_phase1.pixel_files

Start making pixel maps#####################.

copy ouput files##########

Definition at line 381 of file TkMap_script_phase1.py.

pixelTreeFileName

TkMap_script_phase1.pixelTreeFileName

Definition at line 395 of file TkMap_script_phase1.py.

rereco

TkMap_script_phase1.rereco

Check if user enter the right run type######################.

Definition at line 159 of file TkMap_script_phase1.py.

Run_Number

TkMap_script_phase1.Run_Number

Definition at line 157 of file TkMap_script_phase1.py.

Run_type

TkMap_script_phase1.Run_type

Definition at line 156 of file TkMap_script_phase1.py.

runDirval

TkMap_script_phase1.runDirval

Definition at line 182 of file TkMap_script_phase1.py.

sefile

TkMap_script_phase1.sefile

rename bad module list file######################

Definition at line 242 of file TkMap_script_phase1.py.

shell

TkMap_script_phase1.shell

Definition at line 192 of file TkMap_script_phase1.py.

strip_files

TkMap_script_phase1.strip_files

create merged list of BadComponent from (PCL, RunInfo and FED Errors) ignore for now

counting dead pixel####################### Counting Dead ROCs and Inefficient DC during the run######################### Merge Dead ROCs and Occupoancy Plot ################### Merge PCL and DQM Plot (only StreamExpress) ################### copy ouput files###################

Definition at line 357 of file TkMap_script_phase1.py.

workPath

TkMap_script_phase1.workPath

Definition at line 234 of file TkMap_script_phase1.py.