CMS 3D CMS Logo
List of all members | Public Member Functions | Public Attributes | Private Member Functions
InefficientDoubleROC.InefficientDeadROCs Class Reference

Public Member Functions

def __init__ (self, inputDQMName, outputFileName, noiseOutputFileName, dirs)
 
def ReadHistograms (self)
 

Public Attributes

 barrelInefficientDColTh
 
 barrelNoisyColumnTh
 
 barrelNoisyColumnTh2
 
 detDict
 
More...
 
 dicOfModuleHistograms
 
 dirs
 
 endcapInefficientDColTh
 
 endcapNoisyColumnTh
 
 inputFile
 
 inputFileName
 
 lookForStr
 
 noiseOutputFileName
 
 outputFileName
 
 pixelNoisynessTh
 THRESHOLDS SECTION. More...
 
 rocMaxCol
 
 rocMaxRow
 
 rocOccupancyTh
 
 rocsInCol
 
 rocsInRow
 

Private Member Functions

def __convertCoordinatesFromHistToROCSpace (self, histName, histXpos, histRow)
 
def __customMedianFilter (self, array, radius=2)
 
def __determineBarrelDColInefficiencyAndNoise (self, medFiltRes, histName, pixelArr, pixelArrWithoutDrops, startPixel, rocCol, rocRow, outputFile, columnsWithSuspiciouslyNoisyPixels, noiseFile)
 
def __determineBarrelNoise (self, noiseFile, columnsWithSuspiciouslyNoisyPixels, histName, meanOfPixels, maxMed, val, pos, rocCol, rocRow)
 
def __determineBarrelNoise2 (self, noiseFile, columnsWithSuspiciouslyNoisyPixels, histName, meanOfPixels, normMeanOfPixels, normVal, pos, rocCol, rocRow)
 
def __determineEndcapDColInefficiencyAndNoise (self, medFiltRes, histName, pixelArr, startPixel, rocCol, rocRow, outputFile, columnsWithSuspiciouslyNoisyPixels, noiseFile)
 
def __determineEndcapNoise (self, noiseFile, columnsWithSuspiciouslyNoisyPixels, histName, meanOfPixels, linVal, val, pos, rocCol, rocRow)
 
def __getPixelArrWithRemovedDrops (self, pixelArr, medFiltRes)
 
def __getROCData (self, hist, startPixel, endPixel, row, repeatFilter=3, filterKernelSize=5)
 
def __lmsExp (self, data, xMin, xMax)
 
def __lmsLin (self, data, xMin, xMax)
 
def __normalizeArray (self, pixelArr)
 
def __setNormalizedArrayZeroInThePoint (self, pixelArr, pt)
 
def __TraverseDirTree (self, dir)
 

Detailed Description

Definition at line 12 of file InefficientDoubleROC.py.

Constructor & Destructor Documentation

◆ __init__()

def InefficientDoubleROC.InefficientDeadROCs.__init__ (   self,
  inputDQMName,
  outputFileName,
  noiseOutputFileName,
  dirs 
)

Definition at line 45 of file InefficientDoubleROC.py.

45  def __init__(self, inputDQMName, outputFileName, noiseOutputFileName, dirs):
46 
47  self.inputFileName = inputDQMName
48  self.outputFileName = outputFileName
49  self.noiseOutputFileName = noiseOutputFileName
50  self.dirs = dirs
51 
52  self.lookForStr = "digi_occupancy_per_col_per_row_"
53 
54  self.rocMaxCol = 52
55  self.rocMaxRow = 80
56  self.rocsInRow = 8
57  self.rocsInCol = 2
58 
59  self.inputFile = TFile(self.inputFileName)
60  self.dicOfModuleHistograms = {}
61 
62 
63  self.pixelNoisynessTh = 6
64  self.rocOccupancyTh = 200
65 
66  self.barrelNoisyColumnTh = 1.35
67  self.barrelNoisyColumnTh2 = 4.5
68  self.endcapNoisyColumnTh = 1.5
69 
70  self.barrelInefficientDColTh = 8#2.5
71  self.endcapInefficientDColTh = 30#8
72 
73 
74 
75  if self.inputFile.IsOpen():
76  print("%s opened successfully!" % (self.inputFileName))
77  #Get all neeeded histograms
78  for dir in self.dirs:
79  self.__TraverseDirTree(self.inputFile.Get(dir))
80  print("Histograms to read: %d" % (len(self.dicOfModuleHistograms)))
81 
82  self.detDict = {}
83 
84  else:
85  print("Unable to open file %s" % (self.inputFileName))
86 
submitPVValidationJobs.__init__
def __init__(self, dataset, job_number, job_id, job_name, isDA, isMC, applyBOWS, applyEXTRACOND, extraconditions, runboundary, lumilist, intlumi, maxevents, gt, allFromGT, alignmentDB, alignmentTAG, apeDB, apeTAG, bowDB, bowTAG, vertextype, tracktype, refittertype, ttrhtype, applyruncontrol, ptcut, CMSSW_dir, the_dir)
Definition: submitPVValidationJobs.py:365
print
void print(TMatrixD &m, const char *label=nullptr, bool mathematicaFormat=false)
Definition: Utilities.cc:47
Get
T * Get(Args... args)
Definition: Trend.h:122

Member Function Documentation

◆ __convertCoordinatesFromHistToROCSpace()

def InefficientDoubleROC.InefficientDeadROCs.__convertCoordinatesFromHistToROCSpace (   self,
  histName,
  histXpos,
  histRow 
)
private

Definition at line 238 of file InefficientDoubleROC.py.

References InefficientDoubleROC.InefficientDeadROCs.rocMaxCol.

Referenced by InefficientDoubleROC.InefficientDeadROCs.__determineBarrelDColInefficiencyAndNoise(), InefficientDoubleROC.InefficientDeadROCs.__determineBarrelNoise(), InefficientDoubleROC.InefficientDeadROCs.__determineBarrelNoise2(), InefficientDoubleROC.InefficientDeadROCs.__determineEndcapDColInefficiencyAndNoise(), and InefficientDoubleROC.InefficientDeadROCs.__determineEndcapNoise().

238  def __convertCoordinatesFromHistToROCSpace(self, histName, histXpos, histRow):
239  tempXROC = (histXpos / self.rocMaxCol) # 0,...,7
240  tempYROC = histRow
241 
242  tempXCoordInROC = histXpos % self.rocMaxCol
243 
244  realXROC, realYROC = tempXROC, tempYROC
245  xCoordInROC = tempXCoordInROC
246 
247  rocNum = 0
248 
249  if histName.find("BPix_Bp") != -1: #zero ROC is in top left corner
250  realYROC = 1 - tempYROC
251  if realYROC == 1:
252  rocNum = 15 - realXROC
253  xCoordInROC = self.rocMaxCol - 1 - xCoordInROC
254  else:
255  rocNum = realXROC
256  else: # zero ROC is in bottom right corner
257  realXROC = 7 - tempXROC
258  if realYROC == 1:
259  rocNum = 15 - realXROC
260  else:
261  rocNum = realXROC
262  xCoordInROC = self.rocMaxCol - 1 - xCoordInROC
263 
264  return rocNum, xCoordInROC
265 
266 

◆ __customMedianFilter()

def InefficientDoubleROC.InefficientDeadROCs.__customMedianFilter (   self,
  array,
  radius = 2 
)
private

Definition at line 122 of file InefficientDoubleROC.py.

References FastTimerService_cff.range.

Referenced by InefficientDoubleROC.InefficientDeadROCs.__getROCData().

122  def __customMedianFilter(self, array, radius = 2):
123  # contrary to scipy implementation it provides adaptive kernel size instead of copying data on boundaries
124  filtered = [0 for i in range(len(array))]
125  currArray = []
126  for i in range(len(array)):
127  if i - radius < 0:
128  currArray = array[0 : i + radius + 1]
129  elif i + radius + 1 >= len(array):
130  currArray = array[i - radius : ]
131 
132  currArray.sort()
133  filtered[i] = currArray[len(currArray) // 2]
134 
135  return filtered
136 
FastTimerService_cff.range
range
Definition: FastTimerService_cff.py:34

◆ __determineBarrelDColInefficiencyAndNoise()

def InefficientDoubleROC.InefficientDeadROCs.__determineBarrelDColInefficiencyAndNoise (   self,
  medFiltRes,
  histName,
  pixelArr,
  pixelArrWithoutDrops,
  startPixel,
  rocCol,
  rocRow,
  outputFile,
  columnsWithSuspiciouslyNoisyPixels,
  noiseFile 
)
private

Definition at line 267 of file InefficientDoubleROC.py.

References InefficientDoubleROC.InefficientDeadROCs.__convertCoordinatesFromHistToROCSpace(), InefficientDoubleROC.InefficientDeadROCs.__determineBarrelNoise2(), InefficientDoubleROC.InefficientDeadROCs.barrelInefficientDColTh, SiStripPI.max, SiStripPI.min, FastTimerService_cff.range, and InefficientDoubleROC.InefficientDeadROCs.rocMaxCol.

Referenced by InefficientDoubleROC.InefficientDeadROCs.ReadHistograms().

267  def __determineBarrelDColInefficiencyAndNoise(self, medFiltRes, histName, pixelArr, pixelArrWithoutDrops, startPixel, rocCol, rocRow, outputFile, columnsWithSuspiciouslyNoisyPixels, noiseFile):
268  meanOfPixels = sum(medFiltRes) / len(medFiltRes)
269  maxMed = max(medFiltRes)
270  minMed = min(medFiltRes)
271 
272  normMeanOfPixels = sum(pixelArrWithoutDrops) / len(pixelArrWithoutDrops)
273  # print( meanOfPixels, maxMed, minMed )
274 
275  doubleDeadCols = 0
276  noisyColsNum = 0
277  noisyROC = 0
278 
279  # for x in range(startPixel, endPixel, 1):
280  for i in range(1, len(pixelArr) - 2):
281  # print(i , i + 1)
282  bin1valDiff = minMed - pixelArr[i + 0]#hist.GetBinContent(x+0)
283  bin2valDiff = minMed - pixelArr[i + 1]
284  # WE ONLY WANT A SET OF TWO COLUMNS SO ADJACENT COLUMNS HAVE TO BE NORMAL
285  bin0valDiff = minMed - pixelArr[i - 1]
286  bin3valDiff = minMed - pixelArr[i + 2]
287 
288  # currentDoubleBinThreshold = minMed / math.sqrt(meanOfPixels) * self.barrelInefficientDColTh # error in bin entry grows as sqrt(N)
289  currentDoubleBinThreshold = math.sqrt(meanOfPixels) * self.barrelInefficientDColTh # error in bin entry grows as sqrt(N)
290 
291  if bin1valDiff > currentDoubleBinThreshold and bin2valDiff > currentDoubleBinThreshold and not bin3valDiff > currentDoubleBinThreshold and not bin0valDiff > currentDoubleBinThreshold:
292 
293  doubleColInRoc = ((i + startPixel) % (self.rocMaxCol)) // 2 + 1
294  doubleDeadCols = doubleDeadCols + 1
295 
296  # outputFile.write("%s,\tX: %d-%d\tROC COLUMN: %d\tROC ROW: %d\tDOUBLE COL IN ROC: %d\tTH: %f\tMIN IN ROC: %f\tBINVAL: %f\n" % (histName, startPixel + (i + 0), startPixel + (i + 1), rocCol, rocRow, doubleColInRoc, currentDoubleBinThreshold, minMed, pixelArr[i]))
297  rocNum, xCoordInROC = self.__convertCoordinatesFromHistToROCSpace(histName, startPixel + i, rocRow)
298  outputFile.write("%s\t(x, row)->[rocNum, doubleXPixelColInROC]\t(%d, %d)->[%d, %d];\t{MIN - VAL, TH}\t{%f, %f}\n" % (histName, startPixel + i, rocRow + 1, rocNum, xCoordInROC / 2, bin1valDiff, currentDoubleBinThreshold))
299 
300  # HANDLE NOISY PIXELS
301  if noisyROC == True: #don't go inside if noisyness was determined already
302  continue
303 
304  # res = self.__determineBarrelNoise(noiseFile, columnsWithSuspiciouslyNoisyPixels, histName, meanOfPixels, maxMed, pixelArr[i], startPixel + i, rocCol, rocRow)
305  # noisyColsNum, noisyROC = noisyColsNum + res[0], res[1]
306  # if i == len(pixelArr) - 3: # CHECK NOISYNESS IN THE RIGHTMOST INNER COL
307  # res = self.__determineBarrelNoise(noiseFile, columnsWithSuspiciouslyNoisyPixels, histName, meanOfPixels, maxMed, pixelArr[i + 1], startPixel + i + 1, rocCol, rocRow)
308  # noisyColsNum, noisyROC = noisyColsNum + res[0], res[1]
309 
310  # NORMALIZED MEAN NOISE DETERMINATION METHOD
311  res = self.__determineBarrelNoise2(noiseFile, columnsWithSuspiciouslyNoisyPixels, histName, meanOfPixels, normMeanOfPixels, pixelArrWithoutDrops[i], startPixel + i, rocCol, rocRow)
312  noisyColsNum, noisyROC = noisyColsNum + res[0], res[1]
313  if i == len(pixelArr) - 3: # CHECK NOISYNESS IN THE RIGHTMOST INNER COL
314  res = self.__determineBarrelNoise2(noiseFile, columnsWithSuspiciouslyNoisyPixels, histName, meanOfPixels, normMeanOfPixels, pixelArrWithoutDrops[i + 1], startPixel + i + 1, rocCol, rocRow)
315  noisyColsNum, noisyROC = noisyColsNum + res[0], res[1]
316 
317 
318 
319  return doubleDeadCols, noisyColsNum
320 
FastTimerService_cff.range
range
Definition: FastTimerService_cff.py:34

◆ __determineBarrelNoise()

def InefficientDoubleROC.InefficientDeadROCs.__determineBarrelNoise (   self,
  noiseFile,
  columnsWithSuspiciouslyNoisyPixels,
  histName,
  meanOfPixels,
  maxMed,
  val,
  pos,
  rocCol,
  rocRow 
)
private

Definition at line 183 of file InefficientDoubleROC.py.

References InefficientDoubleROC.InefficientDeadROCs.__convertCoordinatesFromHistToROCSpace(), InefficientDoubleROC.InefficientDeadROCs.barrelNoisyColumnTh, and InefficientDoubleROC.InefficientDeadROCs.rocOccupancyTh.

183  def __determineBarrelNoise(self, noiseFile, columnsWithSuspiciouslyNoisyPixels, histName, meanOfPixels, maxMed, val, pos, rocCol, rocRow):
184  noisyROC = False;
185  if meanOfPixels < self.rocOccupancyTh:
186  #print("Very low mean occupancy: %f in %s in (col, row) (%d, %d)...\tSkipping noisy ROC calculation" % (meanOfPixels, histName, rocCol, rocRow) )
187  noisyROC = True
188  else:
189  th = self.barrelNoisyColumnTh * maxMed
190  if val > th:
191  if pos not in columnsWithSuspiciouslyNoisyPixels:
192  rocNum, xCoordInROC = self.__convertCoordinatesFromHistToROCSpace(histName, pos, rocRow)
193  noiseFile.write("%s\t(x, row)->[rocNum, xRoc]\t(%d, %d)->[%d, %d];\t{VAL, TH}\t{%f, %f}\n" % (histName, pos, rocRow+1, rocNum, xCoordInROC, val, th))
194 
195  return 1, noisyROC
196  # else:
197  # print("WARNING: rejecting %s (x, row) (%d, %d) as being affected by a few noisy pixel(s)" % (histName, pos, rocRow+1))
198 
199  return 0, noisyROC
200 

◆ __determineBarrelNoise2()

def InefficientDoubleROC.InefficientDeadROCs.__determineBarrelNoise2 (   self,
  noiseFile,
  columnsWithSuspiciouslyNoisyPixels,
  histName,
  meanOfPixels,
  normMeanOfPixels,
  normVal,
  pos,
  rocCol,
  rocRow 
)
private

Definition at line 201 of file InefficientDoubleROC.py.

References InefficientDoubleROC.InefficientDeadROCs.__convertCoordinatesFromHistToROCSpace(), InefficientDoubleROC.InefficientDeadROCs.barrelNoisyColumnTh2, and InefficientDoubleROC.InefficientDeadROCs.rocOccupancyTh.

Referenced by InefficientDoubleROC.InefficientDeadROCs.__determineBarrelDColInefficiencyAndNoise().

201  def __determineBarrelNoise2(self, noiseFile, columnsWithSuspiciouslyNoisyPixels, histName, meanOfPixels, normMeanOfPixels, normVal, pos, rocCol, rocRow):
202  noisyROC = False;
203  if meanOfPixels < self.rocOccupancyTh:
204  #print("Very low mean occupancy: %f in %s in (col, row) (%d, %d)...\tSkipping noisy ROC calculation" % (meanOfPixels, histName, rocCol, rocRow) )
205  noisyROC = True
206  else:
207  th = self.barrelNoisyColumnTh2 * normMeanOfPixels
208  if normVal > th:
209  if pos not in columnsWithSuspiciouslyNoisyPixels:
210  rocNum, xCoordInROC = self.__convertCoordinatesFromHistToROCSpace(histName, pos, rocRow)
211  noiseFile.write("%s\t(x, row)->[rocNum, xRoc]\t(%d, %d)->[%d, %d];\t{NORMVAL, TH}\t{%f, %f}\n" % (histName, pos, rocRow+1, rocNum, xCoordInROC, normVal, th))
212 
213  return 1, noisyROC
214  # else:
215  # print("WARNING: rejecting %s (x, row) (%d, %d) as being affected by a few noisy pixel(s)" % (histName, pos, rocRow+1))
216 
217  return 0, noisyROC
218 

◆ __determineEndcapDColInefficiencyAndNoise()

def InefficientDoubleROC.InefficientDeadROCs.__determineEndcapDColInefficiencyAndNoise (   self,
  medFiltRes,
  histName,
  pixelArr,
  startPixel,
  rocCol,
  rocRow,
  outputFile,
  columnsWithSuspiciouslyNoisyPixels,
  noiseFile 
)
private

Definition at line 321 of file InefficientDoubleROC.py.

References InefficientDoubleROC.InefficientDeadROCs.__convertCoordinatesFromHistToROCSpace(), InefficientDoubleROC.InefficientDeadROCs.__determineEndcapNoise(), InefficientDoubleROC.InefficientDeadROCs.__lmsLin(), InefficientDoubleROC.InefficientDeadROCs.endcapInefficientDColTh, FastTimerService_cff.range, and InefficientDoubleROC.InefficientDeadROCs.rocMaxCol.

Referenced by InefficientDoubleROC.InefficientDeadROCs.ReadHistograms().

321  def __determineEndcapDColInefficiencyAndNoise(self, medFiltRes, histName, pixelArr, startPixel, rocCol, rocRow, outputFile, columnsWithSuspiciouslyNoisyPixels, noiseFile):
322  doubleDeadCols = 0
323  noisyColsNum = 0
324  noisyROC = 0
325 
326  useLin = True
327  # <D> might be used for high noise ROC recognition
328  a, b, D = self.__lmsLin(medFiltRes, startPixel, len(medFiltRes) + startPixel)
329 
330  meanOfPixels = sum(medFiltRes) / len(medFiltRes)
331 
332  # for x in range(startPixel, endPixel, 1):
333  for i in range(1, len(pixelArr) - 2):
334 
335  if useLin == True:
336  linVal1 = a * (i + startPixel + 0) + b
337  linVal2 = a * (i + startPixel + 1) + b
338 
339  linVal0 = a * (i + startPixel - 1) + b
340  linVal3 = a * (i + startPixel + 2) + b
341  else:
342  linVal1 = b * math.exp(a * (i + startPixel + 0))
343  linVal2 = b * math.exp(a * (i + startPixel + 1))
344 
345  linVal0 = b * math.exp(a * (i + startPixel - 1))
346  linVal3 = b * math.exp(a * (i + startPixel + 2))
347 
348  bin1valDiff = linVal1 - pixelArr[i + 0]
349  bin2valDiff = linVal2 - pixelArr[i + 1]
350  # WE ONLY WANT A SET OF TWO COLUMNS SO ADJACENT COLUMNS HAVE TO BE NORMAL
351  bin0valDiff = linVal0 - pixelArr[i - 1]
352  bin3valDiff = linVal3 - pixelArr[i + 2]
353 
354  try:
355  currentDoubleBinThreshold = math.sqrt((linVal1 + linVal2) * 0.5) * self.endcapInefficientDColTh
356  except:
357  # print(a, b, startPixel, i, linVal1, linVal2)
358  continue
359 
360  if bin1valDiff > currentDoubleBinThreshold and bin2valDiff > currentDoubleBinThreshold and not bin3valDiff > currentDoubleBinThreshold and not bin0valDiff > currentDoubleBinThreshold:
361 
362  doubleColInRoc = ((i + startPixel) % (self.rocMaxCol)) // 2 + 1
363  doubleDeadCols = doubleDeadCols + 1
364 
365  # outputFile.write("%s,\tX: %d-%d\tROC COLUMN: %d\tROC ROW: %d\tDOUBLE COL IN ROC: %d\tTH: %f\tLINVAL: %f\tBINVAL: %f\n" % (histName, startPixel + (i + 0), startPixel + (i + 1), rocCol, rocRow, doubleColInRoc, currentDoubleBinThreshold, linVal1, pixelArr[i]))
366  rocNum, xCoordInROC = self.__convertCoordinatesFromHistToROCSpace(histName, startPixel + i, rocRow)
367  outputFile.write("%s\t(x, row)->[rocNum, doubleXPixelColInROC]\t(%d, %d)->[%d, %d];\t{LIN(x) - VAL, TH}\t{%f, %f}\n" % (histName, startPixel + i, rocRow + 1, rocNum, xCoordInROC / 2, bin1valDiff, currentDoubleBinThreshold))
368 
369 
370  # HANDLE NOISY PIXELS
371  if noisyROC == True: #don't go inside if noisyness was determined already
372  continue
373 
374  res = self.__determineEndcapNoise(noiseFile, columnsWithSuspiciouslyNoisyPixels, histName, meanOfPixels, linVal1, pixelArr[i], i + startPixel, rocCol, rocRow)
375  noisyColsNum, noisyROC = noisyColsNum + res[0], res[1]
376  if i == len(pixelArr) - 3: # CHECK NOISYNESS IN THE RIGHTMOST INNER COL
377  res = self.__determineEndcapNoise(noiseFile, columnsWithSuspiciouslyNoisyPixels, histName, meanOfPixels, linVal2, pixelArr[i + 1], i + 1 + startPixel, rocCol, rocRow)
378  noisyColsNum, noisyROC = noisyColsNum + res[0], res[1]
379 
380  return doubleDeadCols, noisyColsNum
381 
FastTimerService_cff.range
range
Definition: FastTimerService_cff.py:34

◆ __determineEndcapNoise()

def InefficientDoubleROC.InefficientDeadROCs.__determineEndcapNoise (   self,
  noiseFile,
  columnsWithSuspiciouslyNoisyPixels,
  histName,
  meanOfPixels,
  linVal,
  val,
  pos,
  rocCol,
  rocRow 
)
private

Definition at line 219 of file InefficientDoubleROC.py.

References InefficientDoubleROC.InefficientDeadROCs.__convertCoordinatesFromHistToROCSpace(), InefficientDoubleROC.InefficientDeadROCs.endcapNoisyColumnTh, and InefficientDoubleROC.InefficientDeadROCs.rocOccupancyTh.

Referenced by InefficientDoubleROC.InefficientDeadROCs.__determineEndcapDColInefficiencyAndNoise().

219  def __determineEndcapNoise(self, noiseFile, columnsWithSuspiciouslyNoisyPixels, histName, meanOfPixels, linVal, val, pos, rocCol, rocRow):
220  noisyROC = False;
221  if meanOfPixels < self.rocOccupancyTh:
222  # print("Very low mean occupancy: %f in %s in (col, row) (%d, %d)...\tSkipping noisy ROC calculation" % (meanOfPixels, histName, rocCol, rocRow) )
223  noisyROC = True
224 
225  else:
226  th = self.endcapNoisyColumnTh * linVal
227  if val > th:
228  if pos not in columnsWithSuspiciouslyNoisyPixels:
229  rocNum, xCoordInROC = self.__convertCoordinatesFromHistToROCSpace(histName, pos, rocRow)
230  noiseFile.write("%s\t(x, row)->[rocNum, xRoc]\t(%d, %d)->[%d, %d];\t{VAL, TH}\t{%f, %f}\n" % (histName, pos, rocRow+1, rocNum, xCoordInROC, val, th))
231 
232  return 1, noisyROC
233  # else:
234  # print("WARNING: rejecting %s (x, row) (%d, %d) as being affected by a few noisy pixel(s)" % (histName, pos, rocRow+1))
235 
236  return 0, noisyROC
237 

◆ __getPixelArrWithRemovedDrops()

def InefficientDoubleROC.InefficientDeadROCs.__getPixelArrWithRemovedDrops (   self,
  pixelArr,
  medFiltRes 
)
private

Definition at line 168 of file InefficientDoubleROC.py.

References SiStripPI.min, and FastTimerService_cff.range.

Referenced by InefficientDoubleROC.InefficientDeadROCs.ReadHistograms().

168  def __getPixelArrWithRemovedDrops(self, pixelArr, medFiltRes):
169  return [ (pixelArr[i] if pixelArr[i] > medFiltRes[i] else medFiltRes[i]) if 0 < i < len(pixelArr) - 1 else min(medFiltRes) for i in range(len(pixelArr))]
170 
FastTimerService_cff.range
range
Definition: FastTimerService_cff.py:34

◆ __getROCData()

def InefficientDoubleROC.InefficientDeadROCs.__getROCData (   self,
  hist,
  startPixel,
  endPixel,
  row,
  repeatFilter = 3,
  filterKernelSize = 5 
)
private

Definition at line 137 of file InefficientDoubleROC.py.

References InefficientDoubleROC.InefficientDeadROCs.__customMedianFilter(), InefficientDoubleROC.InefficientDeadROCs.pixelNoisynessTh, FastTimerService_cff.range, and InefficientDoubleROC.InefficientDeadROCs.rocMaxRow.

Referenced by InefficientDoubleROC.InefficientDeadROCs.ReadHistograms().

137  def __getROCData(self, hist, startPixel, endPixel, row, repeatFilter = 3, filterKernelSize = 5):
138  pixelArr = []
139  columnsWithSuspiciouslyNoisyPixels = []
140 
141  for x in range(startPixel, endPixel):
142 
143  columnPixels = [hist.GetBinContent(x, y + 1) for y in range(row * self.rocMaxRow, (row + 1) * self.rocMaxRow)]
144  columnSum = sum(columnPixels)
145 
146  columnMean = columnSum / len(columnPixels)
147  for i in range(len(columnPixels)):
148  if columnPixels[i] > self.pixelNoisynessTh * columnMean:
149  # col = (startPixel % self.rocMaxCol) + 1
150 
151  columnsWithSuspiciouslyNoisyPixels.append(x)
152 
153  # print("WARNING:\t %s : %dx%d:%d may contain NOISY PIXELS instead of NOISY COLUMNS" % (hist.GetName(), col, row + 1, startPixel + i))
154  break
155 
156  pixelArr.append(columnSum)
157 
158  if len(pixelArr) == 0:
159  return None, None, None # ROC down
160 
161  medFiltRes, sciPyMedFiltRes = deepcopy(pixelArr), deepcopy(pixelArr)
162  for i in range(repeatFilter):
163  sciPyMedFiltRes = signal.medfilt(sciPyMedFiltRes, filterKernelSize) # 5 is obligatory to filter doublets!!!
164  medFiltRes = self.__customMedianFilter(medFiltRes, filterKernelSize // 2)
165 
166  return pixelArr, medFiltRes, columnsWithSuspiciouslyNoisyPixels, sciPyMedFiltRes
167 
FastTimerService_cff.range
range
Definition: FastTimerService_cff.py:34

◆ __lmsExp()

def InefficientDoubleROC.InefficientDeadROCs.__lmsExp (   self,
  data,
  xMin,
  xMax 
)
private

Definition at line 87 of file InefficientDoubleROC.py.

References FastTimerService_cff.range.

87  def __lmsExp(self, data, xMin, xMax):
88  meanOfX = (xMax + xMin) * 0.5
89  meanOfY = sum( [math.log(data[i]) for i in range(len(data))] ) / len(data)
90 
91  D = 0
92  for i in range(xMin, xMax + 1):
93  D = D + (i - meanOfX)**2
94  # print(D)
95 
96  a = 0
97  for i in range(len(data)):
98  a = a + math.log(data[i]) * (xMin + i - meanOfX)
99  a = a/D
100 
101  lnb = meanOfY - a * meanOfX
102 
103  return a, math.exp(lnb)
104 
FastTimerService_cff.range
range
Definition: FastTimerService_cff.py:34

◆ __lmsLin()

def InefficientDoubleROC.InefficientDeadROCs.__lmsLin (   self,
  data,
  xMin,
  xMax 
)
private

Definition at line 105 of file InefficientDoubleROC.py.

References FastTimerService_cff.range.

Referenced by InefficientDoubleROC.InefficientDeadROCs.__determineEndcapDColInefficiencyAndNoise().

105  def __lmsLin(self, data, xMin, xMax):
106  meanOfX = (xMax + xMin) * 0.5
107  meanOfY = sum(data) / len(data)
108 
109  D = 0
110  for i in range(xMin, xMax + 1):
111  D = D + (i - meanOfX)**2
112 
113  a = 0
114  for i in range(len(data)):
115  a = a + data[i] * (xMin + i - meanOfX)
116  a = a/D
117 
118  b = meanOfY - a * meanOfX
119 
120  return a, b, D
121 
FastTimerService_cff.range
range
Definition: FastTimerService_cff.py:34

◆ __normalizeArray()

def InefficientDoubleROC.InefficientDeadROCs.__normalizeArray (   self,
  pixelArr 
)
private

Definition at line 171 of file InefficientDoubleROC.py.

References SiStripPI.max, SiStripPI.min, and FastTimerService_cff.range.

Referenced by InefficientDoubleROC.InefficientDeadROCs.ReadHistograms().

171  def __normalizeArray(self, pixelArr):
172  c_min, c_max = min(pixelArr), max(pixelArr)
173  if c_min != c_max:
174  c_diff_inv = 1.0 / (c_max - c_min)
175  return [ (pixelArr[i] - c_min) * c_diff_inv for i in range(len(pixelArr))]
176  return [0 for i in range(len(pixelArr))]
177 
FastTimerService_cff.range
range
Definition: FastTimerService_cff.py:34

◆ __setNormalizedArrayZeroInThePoint()

def InefficientDoubleROC.InefficientDeadROCs.__setNormalizedArrayZeroInThePoint (   self,
  pixelArr,
  pt 
)
private

Definition at line 178 of file InefficientDoubleROC.py.

References FastTimerService_cff.range.

178  def __setNormalizedArrayZeroInThePoint(self, pixelArr, pt):
179  c_diff_inv = 1.0 / (1.0 - pt)
180 
181  return [ (pixelArr[i] - pt) * c_diff_inv for i in range(len(pixelArr))]
182 
FastTimerService_cff.range
range
Definition: FastTimerService_cff.py:34

◆ __TraverseDirTree()

def InefficientDoubleROC.InefficientDeadROCs.__TraverseDirTree (   self,
  dir 
)
private

Definition at line 15 of file InefficientDoubleROC.py.

References InefficientDoubleROC.InefficientDeadROCs.__TraverseDirTree(), mps_setup.append, InefficientDoubleROC.InefficientDeadROCs.dicOfModuleHistograms, InefficientDoubleROC.InefficientDeadROCs.lookForStr, and update.

Referenced by InefficientDoubleROC.InefficientDeadROCs.__TraverseDirTree(), PhaseITreeProducer.ModuleLvlValuesReader.__TraverseDirTree(), and TH2PolyOfflineMaps.TH2PolyOfflineMaps.__TraverseDirTree().

15  def __TraverseDirTree(self, dir):
16 
17  for obj in dir.GetListOfKeys():
18  if not obj.IsFolder():
19  if obj.ReadObjectAny(TClass.GetClass("TH2")):
20  th1 = deepcopy(obj.ReadObj())
21  name = th1.GetName()
22  if name.startswith(self.lookForStr): #take only module lvl plots
23  # print(''.join([dir.GetPath(), '/', name]))
24  newName = name.split(self.lookForStr)[1]
25  th1.SetName(newName)
26 
27  # used to sort outputs by disk/layer
28  layer = 0
29  # print(newName)
30  if newName.startswith("B"):
31  layer = "B" + ((newName.split("_LYR"))[1])[0]
32  else:
33  layer = ((newName.split("_D"))[1])[0]
34  if newName.startswith("FPix_Bm"):
35  layer = "-" + layer
36  layer = "F" + layer
37 
38  if layer in self.dicOfModuleHistograms:
39  self.dicOfModuleHistograms[layer].append(th1)
40  else:
41  self.dicOfModuleHistograms.update({layer : [th1]})
42  else:
43  self.__TraverseDirTree(obj.ReadObj())
44 
mps_setup.append
append
Definition: mps_setup.py:85
update
#define update(a, b)
Definition: TrackClassifier.cc:10

◆ ReadHistograms()

def InefficientDoubleROC.InefficientDeadROCs.ReadHistograms (   self)

Definition at line 382 of file InefficientDoubleROC.py.

References InefficientDoubleROC.InefficientDeadROCs.__determineBarrelDColInefficiencyAndNoise(), InefficientDoubleROC.InefficientDeadROCs.__determineEndcapDColInefficiencyAndNoise(), InefficientDoubleROC.InefficientDeadROCs.__getPixelArrWithRemovedDrops(), InefficientDoubleROC.InefficientDeadROCs.__getROCData(), InefficientDoubleROC.InefficientDeadROCs.__normalizeArray(), InefficientDoubleROC.InefficientDeadROCs.dicOfModuleHistograms, InefficientDoubleROC.InefficientDeadROCs.noiseOutputFileName, DTSurveyConvert.outputFileName, InefficientDoubleROC.InefficientDeadROCs.outputFileName, SiStripGainCosmicCalculator.outputFileName, CSCSkim.outputFileName, print(), FastTimerService_cff.range, InefficientDoubleROC.InefficientDeadROCs.rocMaxCol, and InefficientDoubleROC.InefficientDeadROCs.rocsInRow.

382  def ReadHistograms(self):
383  doubleDeadCols, noisyColsNum = 0, 0
384 
385  with open(self.noiseOutputFileName, "w") as noiseFile:
386  with open(self.outputFileName, "w") as outputFile:
387  for layer in self.dicOfModuleHistograms:
388 
389  doubleDeadColsInLayer, noisyColsNumInLayer = 0, 0
390 
391  outputFile.write("-> " + layer + "\n\n")
392  noiseFile.write("-> " + layer + "\n\n")
393 
394  for hist in self.dicOfModuleHistograms[layer]:
395  for row in range(2):
396  for rocNum in range(self.rocsInRow):
397  startPixel = rocNum * self.rocMaxCol + 1
398  endPixel = (rocNum + 1) * self.rocMaxCol + 1 # - 1 ???
399 
400  rocCol = rocNum + 1
401 
402  pixelArr, medFiltRes, columnsWithSuspiciouslyNoisyPixels, sciPyMedFiltRes = self.__getROCData(hist, startPixel, endPixel, row, 3, 5)
403 
404  if pixelArr == None:
405  continue
406 
407  # meanOfPixels = sum(pixelArr) / len(pixelArr)
408  # pixelArrSorted = deepcopy(pixelArr)
409  # pixelArrSorted.sort()
410  # outputFile.write("%s: <x> <med_min> VS. <med_max> | x_min:\t%f %f %f | %f, %f, %f, %f\n" % (hist.GetName(), meanOfPixels, min(medFiltRes), max(medFiltRes), pixelArrSorted[0], pixelArrSorted[1], pixelArrSorted[2], pixelArrSorted[3]))
411 
412  if "F" not in layer:
413  # pixelArrWithoutDrops = self.__getPixelArrWithRemovedDrops(pixelArr, medFiltRes)
414  pixelArrWithoutDrops = self.__getPixelArrWithRemovedDrops(pixelArr, sciPyMedFiltRes)
415  pixelArrWithoutDropsNormalized = self.__normalizeArray(pixelArrWithoutDrops)
416  # tmp_mean = sum(pixelArrWithoutDropsNormalized) / len(pixelArrWithoutDropsNormalized)
417  # pixelArrWithoutDropsNormalized = self.__setNormalizedArrayZeroInThePoint(pixelArrWithoutDropsNormalized, tmp_mean)
418 
419  # print(min(pixelArrWithoutDropsNormalized), max(pixelArrWithoutDropsNormalized))
420  result = self.__determineBarrelDColInefficiencyAndNoise(medFiltRes, hist.GetName(), pixelArr, pixelArrWithoutDropsNormalized, startPixel, rocCol, row, outputFile, columnsWithSuspiciouslyNoisyPixels, noiseFile)
421  else:
422  result = self.__determineEndcapDColInefficiencyAndNoise(medFiltRes, hist.GetName(), pixelArr, startPixel, rocCol, row, outputFile, columnsWithSuspiciouslyNoisyPixels, noiseFile)
423 
424  doubleDeadCols, doubleDeadColsInLayer = doubleDeadCols + result[0], doubleDeadColsInLayer + result[0]
425  noisyColsNum, noisyColsNumInLayer = noisyColsNum + result[1], noisyColsNumInLayer + result[1]
426 
427  outputFile.write("\n\tTOTAL IN LAYER/DISK: %d\n\n" % (doubleDeadColsInLayer))
428  noiseFile.write("\n\tTOTAL IN LAYER/DISK: %d\n\n" % (noisyColsNumInLayer))
429 
430  print("Number of inefficient double columns: %d"%(doubleDeadCols))
431  print("Number of noisy cols: %d"%(noisyColsNum))
432 
433 
434 #--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--
print
void print(TMatrixD &m, const char *label=nullptr, bool mathematicaFormat=false)
Definition: Utilities.cc:47
FastTimerService_cff.range
range
Definition: FastTimerService_cff.py:34

Member Data Documentation

◆ barrelInefficientDColTh

InefficientDoubleROC.InefficientDeadROCs.barrelInefficientDColTh

Definition at line 70 of file InefficientDoubleROC.py.

Referenced by InefficientDoubleROC.InefficientDeadROCs.__determineBarrelDColInefficiencyAndNoise().

◆ barrelNoisyColumnTh

InefficientDoubleROC.InefficientDeadROCs.barrelNoisyColumnTh

Definition at line 66 of file InefficientDoubleROC.py.

Referenced by InefficientDoubleROC.InefficientDeadROCs.__determineBarrelNoise().

◆ barrelNoisyColumnTh2

InefficientDoubleROC.InefficientDeadROCs.barrelNoisyColumnTh2

Definition at line 67 of file InefficientDoubleROC.py.

Referenced by InefficientDoubleROC.InefficientDeadROCs.__determineBarrelNoise2().

◆ detDict

InefficientDoubleROC.InefficientDeadROCs.detDict

Definition at line 82 of file InefficientDoubleROC.py.

Referenced by PhaseITreeProducer.ModuleLvlValuesReader.ReadHistograms(), and TH2PolyOfflineMaps.TH2PolyOfflineMaps.ReadHistograms().

◆ dicOfModuleHistograms

InefficientDoubleROC.InefficientDeadROCs.dicOfModuleHistograms

Definition at line 60 of file InefficientDoubleROC.py.

Referenced by InefficientDoubleROC.InefficientDeadROCs.__TraverseDirTree(), and InefficientDoubleROC.InefficientDeadROCs.ReadHistograms().

◆ dirs

InefficientDoubleROC.InefficientDeadROCs.dirs

Definition at line 50 of file InefficientDoubleROC.py.

Referenced by PhaseITreeProducer.ModuleLvlValuesReader.__TraverseDirTree(), and TH2PolyOfflineMaps.TH2PolyOfflineMaps.__TraverseDirTree().

◆ endcapInefficientDColTh

InefficientDoubleROC.InefficientDeadROCs.endcapInefficientDColTh

Definition at line 71 of file InefficientDoubleROC.py.

Referenced by InefficientDoubleROC.InefficientDeadROCs.__determineEndcapDColInefficiencyAndNoise().

◆ endcapNoisyColumnTh

InefficientDoubleROC.InefficientDeadROCs.endcapNoisyColumnTh

Definition at line 68 of file InefficientDoubleROC.py.

Referenced by InefficientDoubleROC.InefficientDeadROCs.__determineEndcapNoise().

◆ inputFile

InefficientDoubleROC.InefficientDeadROCs.inputFile

Definition at line 59 of file InefficientDoubleROC.py.

Referenced by PhaseITreeProducer.ModuleLvlValuesReader.__del__(), TH2PolyOfflineMaps.TH2PolyOfflineMaps.__del__(), PhaseITreeProducer.ModuleLvlValuesReader.ReadHistograms(), and TH2PolyOfflineMaps.TH2PolyOfflineMaps.ReadHistograms().

◆ inputFileName

InefficientDoubleROC.InefficientDeadROCs.inputFileName

Definition at line 47 of file InefficientDoubleROC.py.

◆ lookForStr

InefficientDoubleROC.InefficientDeadROCs.lookForStr

Definition at line 52 of file InefficientDoubleROC.py.

Referenced by InefficientDoubleROC.InefficientDeadROCs.__TraverseDirTree().

◆ noiseOutputFileName

InefficientDoubleROC.InefficientDeadROCs.noiseOutputFileName

Definition at line 49 of file InefficientDoubleROC.py.

Referenced by InefficientDoubleROC.InefficientDeadROCs.ReadHistograms().

◆ outputFileName

InefficientDoubleROC.InefficientDeadROCs.outputFileName

Definition at line 48 of file InefficientDoubleROC.py.

Referenced by PhaseITreeProducer.ModuleLvlValuesReader.CreateTree2(), and InefficientDoubleROC.InefficientDeadROCs.ReadHistograms().

◆ pixelNoisynessTh

InefficientDoubleROC.InefficientDeadROCs.pixelNoisynessTh

THRESHOLDS SECTION.

Definition at line 63 of file InefficientDoubleROC.py.

Referenced by InefficientDoubleROC.InefficientDeadROCs.__getROCData().

◆ rocMaxCol

InefficientDoubleROC.InefficientDeadROCs.rocMaxCol

Definition at line 54 of file InefficientDoubleROC.py.

Referenced by InefficientDoubleROC.InefficientDeadROCs.__convertCoordinatesFromHistToROCSpace(), InefficientDoubleROC.InefficientDeadROCs.__determineBarrelDColInefficiencyAndNoise(), InefficientDoubleROC.InefficientDeadROCs.__determineEndcapDColInefficiencyAndNoise(), and InefficientDoubleROC.InefficientDeadROCs.ReadHistograms().

◆ rocMaxRow

InefficientDoubleROC.InefficientDeadROCs.rocMaxRow

Definition at line 55 of file InefficientDoubleROC.py.

Referenced by InefficientDoubleROC.InefficientDeadROCs.__getROCData().

◆ rocOccupancyTh

InefficientDoubleROC.InefficientDeadROCs.rocOccupancyTh

Definition at line 64 of file InefficientDoubleROC.py.

Referenced by InefficientDoubleROC.InefficientDeadROCs.__determineBarrelNoise(), InefficientDoubleROC.InefficientDeadROCs.__determineBarrelNoise2(), and InefficientDoubleROC.InefficientDeadROCs.__determineEndcapNoise().

◆ rocsInCol

InefficientDoubleROC.InefficientDeadROCs.rocsInCol

Definition at line 57 of file InefficientDoubleROC.py.

◆ rocsInRow

InefficientDoubleROC.InefficientDeadROCs.rocsInRow

Definition at line 56 of file InefficientDoubleROC.py.

Referenced by InefficientDoubleROC.InefficientDeadROCs.ReadHistograms().

Generated for CMSSW Reference Manual by doxygen 1.8.14