CMS 3D CMS Logo
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Properties Friends Macros Groups Pages
List of all members | Public Member Functions | Public Attributes | Private Member Functions
InefficientDoubleROC.InefficientDeadROCs Class Reference

Public Member Functions

def __init__
 
def ReadHistograms
 

Public Attributes

 barrelInefficientDColTh
 
 barrelNoisyColumnTh
 
 barrelNoisyColumnTh2
 
 detDict
 
More...
 
 dicOfModuleHistograms
 
 dirs
 
 endcapInefficientDColTh
 
 endcapNoisyColumnTh
 
 inputFile
 
 inputFileName
 
 lookForStr
 
 noiseOutputFileName
 
 outputFileName
 
 pixelNoisynessTh
 THRESHOLDS SECTION. More...
 
 rocMaxCol
 
 rocMaxRow
 
 rocOccupancyTh
 
 rocsInCol
 
 rocsInRow
 

Private Member Functions

def __convertCoordinatesFromHistToROCSpace
 
def __customMedianFilter
 
def __determineBarrelDColInefficiencyAndNoise
 
def __determineBarrelNoise
 
def __determineBarrelNoise2
 
def __determineEndcapDColInefficiencyAndNoise
 
def __determineEndcapNoise
 
def __getPixelArrWithRemovedDrops
 
def __getROCData
 
def __lmsExp
 
def __lmsLin
 
def __normalizeArray
 
def __setNormalizedArrayZeroInThePoint
 
def __TraverseDirTree
 

Detailed Description

Definition at line 12 of file InefficientDoubleROC.py.

Constructor & Destructor Documentation

def InefficientDoubleROC.InefficientDeadROCs.__init__ (   self,
  inputDQMName,
  outputFileName,
  noiseOutputFileName,
  dirs 
)

Definition at line 45 of file InefficientDoubleROC.py.

45 
46  def __init__(self, inputDQMName, outputFileName, noiseOutputFileName, dirs):
47 
48  self.inputFileName = inputDQMName
49  self.outputFileName = outputFileName
50  self.noiseOutputFileName = noiseOutputFileName
51  self.dirs = dirs
52 
53  self.lookForStr = "digi_occupancy_per_col_per_row_"
54 
55  self.rocMaxCol = 52
56  self.rocMaxRow = 80
57  self.rocsInRow = 8
58  self.rocsInCol = 2
59 
60  self.inputFile = TFile(self.inputFileName)
61  self.dicOfModuleHistograms = {}
62 
63  ### THRESHOLDS SECTION
64  self.pixelNoisynessTh = 6
65  self.rocOccupancyTh = 200
66 
67  self.barrelNoisyColumnTh = 1.35
68  self.barrelNoisyColumnTh2 = 4.5
69  self.endcapNoisyColumnTh = 1.5
70 
71  self.barrelInefficientDColTh = 8#2.5
72  self.endcapInefficientDColTh = 30#8
73 
74  ### ###################
75 
76  if self.inputFile.IsOpen():
77  print("%s opened successfully!" % (self.inputFileName))
78  #Get all neeeded histograms
79  for dir in self.dirs:
80  self.__TraverseDirTree(self.inputFile.Get(dir))
81  print("Histograms to read: %d" % (len(self.dicOfModuleHistograms)))
82 
83  self.detDict = {}
84 
85  else:
86  print("Unable to open file %s" % (self.inputFileName))
print
void print(TMatrixD &m, const char *label=nullptr, bool mathematicaFormat=false)
Definition: Utilities.cc:47

Member Function Documentation

def InefficientDoubleROC.InefficientDeadROCs.__convertCoordinatesFromHistToROCSpace (   self,
  histName,
  histXpos,
  histRow 
)
private

Definition at line 238 of file InefficientDoubleROC.py.

References InefficientDoubleROC.InefficientDeadROCs.rocMaxCol.

Referenced by InefficientDoubleROC.InefficientDeadROCs.__determineBarrelDColInefficiencyAndNoise(), InefficientDoubleROC.InefficientDeadROCs.__determineBarrelNoise(), InefficientDoubleROC.InefficientDeadROCs.__determineBarrelNoise2(), InefficientDoubleROC.InefficientDeadROCs.__determineEndcapDColInefficiencyAndNoise(), and InefficientDoubleROC.InefficientDeadROCs.__determineEndcapNoise().

238 
239  def __convertCoordinatesFromHistToROCSpace(self, histName, histXpos, histRow):
240  tempXROC = (histXpos / self.rocMaxCol) # 0,...,7
241  tempYROC = histRow
242 
243  tempXCoordInROC = histXpos % self.rocMaxCol
244 
245  realXROC, realYROC = tempXROC, tempYROC
246  xCoordInROC = tempXCoordInROC
247 
248  rocNum = 0
249 
250  if histName.find("BPix_Bp") != -1: #zero ROC is in top left corner
251  realYROC = 1 - tempYROC
252  if realYROC == 1:
253  rocNum = 15 - realXROC
254  xCoordInROC = self.rocMaxCol - 1 - xCoordInROC
255  else:
256  rocNum = realXROC
257  else: # zero ROC is in bottom right corner
258  realXROC = 7 - tempXROC
259  if realYROC == 1:
260  rocNum = 15 - realXROC
261  else:
262  rocNum = realXROC
263  xCoordInROC = self.rocMaxCol - 1 - xCoordInROC
264 
265  return rocNum, xCoordInROC
266 
def InefficientDoubleROC.InefficientDeadROCs.__customMedianFilter (   self,
  array,
  radius = 2 
)
private

Definition at line 122 of file InefficientDoubleROC.py.

References sistrip::SpyUtilities.range().

Referenced by InefficientDoubleROC.InefficientDeadROCs.__getROCData().

122 
123  def __customMedianFilter(self, array, radius = 2):
124  # contrary to scipy implementation it provides adaptive kernel size instead of copying data on boundaries
125  filtered = [0 for i in range(len(array))]
126  currArray = []
127  for i in range(len(array)):
128  if i - radius < 0:
129  currArray = array[0 : i + radius + 1]
130  elif i + radius + 1 >= len(array):
131  currArray = array[i - radius : ]
132 
133  currArray.sort()
134  filtered[i] = currArray[len(currArray) // 2]
135 
136  return filtered
sistrip::SpyUtilities::range
const uint16_t range(const Frame &aFrame)
Definition: SiStripSpyUtilities.cc:76
def InefficientDoubleROC.InefficientDeadROCs.__determineBarrelDColInefficiencyAndNoise (   self,
  medFiltRes,
  histName,
  pixelArr,
  pixelArrWithoutDrops,
  startPixel,
  rocCol,
  rocRow,
  outputFile,
  columnsWithSuspiciouslyNoisyPixels,
  noiseFile 
)
private

Definition at line 267 of file InefficientDoubleROC.py.

References InefficientDoubleROC.InefficientDeadROCs.__convertCoordinatesFromHistToROCSpace(), InefficientDoubleROC.InefficientDeadROCs.__determineBarrelNoise2(), InefficientDoubleROC.InefficientDeadROCs.barrelInefficientDColTh, SiStripPI.max, min(), sistrip::SpyUtilities.range(), and InefficientDoubleROC.InefficientDeadROCs.rocMaxCol.

Referenced by InefficientDoubleROC.InefficientDeadROCs.ReadHistograms().

267 
268  def __determineBarrelDColInefficiencyAndNoise(self, medFiltRes, histName, pixelArr, pixelArrWithoutDrops, startPixel, rocCol, rocRow, outputFile, columnsWithSuspiciouslyNoisyPixels, noiseFile):
269  meanOfPixels = sum(medFiltRes) / len(medFiltRes)
270  maxMed = max(medFiltRes)
271  minMed = min(medFiltRes)
272 
273  normMeanOfPixels = sum(pixelArrWithoutDrops) / len(pixelArrWithoutDrops)
274  # print( meanOfPixels, maxMed, minMed )
275 
276  doubleDeadCols = 0
277  noisyColsNum = 0
278  noisyROC = 0
279 
280  # for x in range(startPixel, endPixel, 1):
281  for i in range(1, len(pixelArr) - 2):
282  # print(i , i + 1)
283  bin1valDiff = minMed - pixelArr[i + 0]#hist.GetBinContent(x+0)
284  bin2valDiff = minMed - pixelArr[i + 1]
285  # WE ONLY WANT A SET OF TWO COLUMNS SO ADJACENT COLUMNS HAVE TO BE NORMAL
286  bin0valDiff = minMed - pixelArr[i - 1]
287  bin3valDiff = minMed - pixelArr[i + 2]
288 
289  # currentDoubleBinThreshold = minMed / math.sqrt(meanOfPixels) * self.barrelInefficientDColTh # error in bin entry grows as sqrt(N)
290  currentDoubleBinThreshold = math.sqrt(meanOfPixels) * self.barrelInefficientDColTh # error in bin entry grows as sqrt(N)
291 
292  if bin1valDiff > currentDoubleBinThreshold and bin2valDiff > currentDoubleBinThreshold and not bin3valDiff > currentDoubleBinThreshold and not bin0valDiff > currentDoubleBinThreshold:
293 
294  doubleColInRoc = ((i + startPixel) % (self.rocMaxCol)) // 2 + 1
295  doubleDeadCols = doubleDeadCols + 1
296 
297  # outputFile.write("%s,\tX: %d-%d\tROC COLUMN: %d\tROC ROW: %d\tDOUBLE COL IN ROC: %d\tTH: %f\tMIN IN ROC: %f\tBINVAL: %f\n" % (histName, startPixel + (i + 0), startPixel + (i + 1), rocCol, rocRow, doubleColInRoc, currentDoubleBinThreshold, minMed, pixelArr[i]))
298  rocNum, xCoordInROC = self.__convertCoordinatesFromHistToROCSpace(histName, startPixel + i, rocRow)
299  outputFile.write("%s\t(x, row)->[rocNum, doubleXPixelColInROC]\t(%d, %d)->[%d, %d];\t{MIN - VAL, TH}\t{%f, %f}\n" % (histName, startPixel + i, rocRow + 1, rocNum, xCoordInROC / 2, bin1valDiff, currentDoubleBinThreshold))
300 
301  # HANDLE NOISY PIXELS
302  if noisyROC == True: #don't go inside if noisyness was determined already
303  continue
304 
305  # res = self.__determineBarrelNoise(noiseFile, columnsWithSuspiciouslyNoisyPixels, histName, meanOfPixels, maxMed, pixelArr[i], startPixel + i, rocCol, rocRow)
306  # noisyColsNum, noisyROC = noisyColsNum + res[0], res[1]
307  # if i == len(pixelArr) - 3: # CHECK NOISYNESS IN THE RIGHTMOST INNER COL
308  # res = self.__determineBarrelNoise(noiseFile, columnsWithSuspiciouslyNoisyPixels, histName, meanOfPixels, maxMed, pixelArr[i + 1], startPixel + i + 1, rocCol, rocRow)
309  # noisyColsNum, noisyROC = noisyColsNum + res[0], res[1]
310 
311  # NORMALIZED MEAN NOISE DETERMINATION METHOD
312  res = self.__determineBarrelNoise2(noiseFile, columnsWithSuspiciouslyNoisyPixels, histName, meanOfPixels, normMeanOfPixels, pixelArrWithoutDrops[i], startPixel + i, rocCol, rocRow)
313  noisyColsNum, noisyROC = noisyColsNum + res[0], res[1]
314  if i == len(pixelArr) - 3: # CHECK NOISYNESS IN THE RIGHTMOST INNER COL
315  res = self.__determineBarrelNoise2(noiseFile, columnsWithSuspiciouslyNoisyPixels, histName, meanOfPixels, normMeanOfPixels, pixelArrWithoutDrops[i + 1], startPixel + i + 1, rocCol, rocRow)
316  noisyColsNum, noisyROC = noisyColsNum + res[0], res[1]
317 
318 
319 
320  return doubleDeadCols, noisyColsNum
sistrip::SpyUtilities::range
const uint16_t range(const Frame &aFrame)
Definition: SiStripSpyUtilities.cc:76
min
T min(T a, T b)
Definition: MathUtil.h:58
def InefficientDoubleROC.InefficientDeadROCs.__determineBarrelNoise (   self,
  noiseFile,
  columnsWithSuspiciouslyNoisyPixels,
  histName,
  meanOfPixels,
  maxMed,
  val,
  pos,
  rocCol,
  rocRow 
)
private

Definition at line 183 of file InefficientDoubleROC.py.

References InefficientDoubleROC.InefficientDeadROCs.__convertCoordinatesFromHistToROCSpace(), InefficientDoubleROC.InefficientDeadROCs.barrelNoisyColumnTh, and InefficientDoubleROC.InefficientDeadROCs.rocOccupancyTh.

183 
184  def __determineBarrelNoise(self, noiseFile, columnsWithSuspiciouslyNoisyPixels, histName, meanOfPixels, maxMed, val, pos, rocCol, rocRow):
185  noisyROC = False;
186  if meanOfPixels < self.rocOccupancyTh:
187  #print("Very low mean occupancy: %f in %s in (col, row) (%d, %d)...\tSkipping noisy ROC calculation" % (meanOfPixels, histName, rocCol, rocRow) )
188  noisyROC = True
189  else:
190  th = self.barrelNoisyColumnTh * maxMed
191  if val > th:
192  if pos not in columnsWithSuspiciouslyNoisyPixels:
193  rocNum, xCoordInROC = self.__convertCoordinatesFromHistToROCSpace(histName, pos, rocRow)
194  noiseFile.write("%s\t(x, row)->[rocNum, xRoc]\t(%d, %d)->[%d, %d];\t{VAL, TH}\t{%f, %f}\n" % (histName, pos, rocRow+1, rocNum, xCoordInROC, val, th))
195 
196  return 1, noisyROC
197  # else:
198  # print("WARNING: rejecting %s (x, row) (%d, %d) as being affected by a few noisy pixel(s)" % (histName, pos, rocRow+1))
199 
200  return 0, noisyROC
def InefficientDoubleROC.InefficientDeadROCs.__determineBarrelNoise2 (   self,
  noiseFile,
  columnsWithSuspiciouslyNoisyPixels,
  histName,
  meanOfPixels,
  normMeanOfPixels,
  normVal,
  pos,
  rocCol,
  rocRow 
)
private

Definition at line 201 of file InefficientDoubleROC.py.

References InefficientDoubleROC.InefficientDeadROCs.__convertCoordinatesFromHistToROCSpace(), InefficientDoubleROC.InefficientDeadROCs.barrelNoisyColumnTh2, and InefficientDoubleROC.InefficientDeadROCs.rocOccupancyTh.

Referenced by InefficientDoubleROC.InefficientDeadROCs.__determineBarrelDColInefficiencyAndNoise().

201 
202  def __determineBarrelNoise2(self, noiseFile, columnsWithSuspiciouslyNoisyPixels, histName, meanOfPixels, normMeanOfPixels, normVal, pos, rocCol, rocRow):
203  noisyROC = False;
204  if meanOfPixels < self.rocOccupancyTh:
205  #print("Very low mean occupancy: %f in %s in (col, row) (%d, %d)...\tSkipping noisy ROC calculation" % (meanOfPixels, histName, rocCol, rocRow) )
206  noisyROC = True
207  else:
208  th = self.barrelNoisyColumnTh2 * normMeanOfPixels
209  if normVal > th:
210  if pos not in columnsWithSuspiciouslyNoisyPixels:
211  rocNum, xCoordInROC = self.__convertCoordinatesFromHistToROCSpace(histName, pos, rocRow)
212  noiseFile.write("%s\t(x, row)->[rocNum, xRoc]\t(%d, %d)->[%d, %d];\t{NORMVAL, TH}\t{%f, %f}\n" % (histName, pos, rocRow+1, rocNum, xCoordInROC, normVal, th))
213 
214  return 1, noisyROC
215  # else:
216  # print("WARNING: rejecting %s (x, row) (%d, %d) as being affected by a few noisy pixel(s)" % (histName, pos, rocRow+1))
217 
218  return 0, noisyROC
def InefficientDoubleROC.InefficientDeadROCs.__determineEndcapDColInefficiencyAndNoise (   self,
  medFiltRes,
  histName,
  pixelArr,
  startPixel,
  rocCol,
  rocRow,
  outputFile,
  columnsWithSuspiciouslyNoisyPixels,
  noiseFile 
)
private

Definition at line 321 of file InefficientDoubleROC.py.

References InefficientDoubleROC.InefficientDeadROCs.__convertCoordinatesFromHistToROCSpace(), InefficientDoubleROC.InefficientDeadROCs.__determineEndcapNoise(), InefficientDoubleROC.InefficientDeadROCs.__lmsLin(), InefficientDoubleROC.InefficientDeadROCs.endcapInefficientDColTh, sistrip::SpyUtilities.range(), and InefficientDoubleROC.InefficientDeadROCs.rocMaxCol.

Referenced by InefficientDoubleROC.InefficientDeadROCs.ReadHistograms().

321 
322  def __determineEndcapDColInefficiencyAndNoise(self, medFiltRes, histName, pixelArr, startPixel, rocCol, rocRow, outputFile, columnsWithSuspiciouslyNoisyPixels, noiseFile):
323  doubleDeadCols = 0
324  noisyColsNum = 0
325  noisyROC = 0
326 
327  useLin = True
328  # <D> might be used for high noise ROC recognition
329  a, b, D = self.__lmsLin(medFiltRes, startPixel, len(medFiltRes) + startPixel)
330 
331  meanOfPixels = sum(medFiltRes) / len(medFiltRes)
332 
333  # for x in range(startPixel, endPixel, 1):
334  for i in range(1, len(pixelArr) - 2):
335 
336  if useLin == True:
337  linVal1 = a * (i + startPixel + 0) + b
338  linVal2 = a * (i + startPixel + 1) + b
339 
340  linVal0 = a * (i + startPixel - 1) + b
341  linVal3 = a * (i + startPixel + 2) + b
342  else:
343  linVal1 = b * math.exp(a * (i + startPixel + 0))
344  linVal2 = b * math.exp(a * (i + startPixel + 1))
345 
346  linVal0 = b * math.exp(a * (i + startPixel - 1))
347  linVal3 = b * math.exp(a * (i + startPixel + 2))
348 
349  bin1valDiff = linVal1 - pixelArr[i + 0]
350  bin2valDiff = linVal2 - pixelArr[i + 1]
351  # WE ONLY WANT A SET OF TWO COLUMNS SO ADJACENT COLUMNS HAVE TO BE NORMAL
352  bin0valDiff = linVal0 - pixelArr[i - 1]
353  bin3valDiff = linVal3 - pixelArr[i + 2]
354 
355  try:
356  currentDoubleBinThreshold = math.sqrt((linVal1 + linVal2) * 0.5) * self.endcapInefficientDColTh
357  except:
358  # print(a, b, startPixel, i, linVal1, linVal2)
359  continue
360 
361  if bin1valDiff > currentDoubleBinThreshold and bin2valDiff > currentDoubleBinThreshold and not bin3valDiff > currentDoubleBinThreshold and not bin0valDiff > currentDoubleBinThreshold:
362 
363  doubleColInRoc = ((i + startPixel) % (self.rocMaxCol)) // 2 + 1
364  doubleDeadCols = doubleDeadCols + 1
365 
366  # outputFile.write("%s,\tX: %d-%d\tROC COLUMN: %d\tROC ROW: %d\tDOUBLE COL IN ROC: %d\tTH: %f\tLINVAL: %f\tBINVAL: %f\n" % (histName, startPixel + (i + 0), startPixel + (i + 1), rocCol, rocRow, doubleColInRoc, currentDoubleBinThreshold, linVal1, pixelArr[i]))
367  rocNum, xCoordInROC = self.__convertCoordinatesFromHistToROCSpace(histName, startPixel + i, rocRow)
368  outputFile.write("%s\t(x, row)->[rocNum, doubleXPixelColInROC]\t(%d, %d)->[%d, %d];\t{LIN(x) - VAL, TH}\t{%f, %f}\n" % (histName, startPixel + i, rocRow + 1, rocNum, xCoordInROC / 2, bin1valDiff, currentDoubleBinThreshold))
369 
370 
371  # HANDLE NOISY PIXELS
372  if noisyROC == True: #don't go inside if noisyness was determined already
373  continue
374 
375  res = self.__determineEndcapNoise(noiseFile, columnsWithSuspiciouslyNoisyPixels, histName, meanOfPixels, linVal1, pixelArr[i], i + startPixel, rocCol, rocRow)
376  noisyColsNum, noisyROC = noisyColsNum + res[0], res[1]
377  if i == len(pixelArr) - 3: # CHECK NOISYNESS IN THE RIGHTMOST INNER COL
378  res = self.__determineEndcapNoise(noiseFile, columnsWithSuspiciouslyNoisyPixels, histName, meanOfPixels, linVal2, pixelArr[i + 1], i + 1 + startPixel, rocCol, rocRow)
379  noisyColsNum, noisyROC = noisyColsNum + res[0], res[1]
380 
381  return doubleDeadCols, noisyColsNum
sistrip::SpyUtilities::range
const uint16_t range(const Frame &aFrame)
Definition: SiStripSpyUtilities.cc:76
def InefficientDoubleROC.InefficientDeadROCs.__determineEndcapNoise (   self,
  noiseFile,
  columnsWithSuspiciouslyNoisyPixels,
  histName,
  meanOfPixels,
  linVal,
  val,
  pos,
  rocCol,
  rocRow 
)
private

Definition at line 219 of file InefficientDoubleROC.py.

References InefficientDoubleROC.InefficientDeadROCs.__convertCoordinatesFromHistToROCSpace(), InefficientDoubleROC.InefficientDeadROCs.endcapNoisyColumnTh, and InefficientDoubleROC.InefficientDeadROCs.rocOccupancyTh.

Referenced by InefficientDoubleROC.InefficientDeadROCs.__determineEndcapDColInefficiencyAndNoise().

219 
220  def __determineEndcapNoise(self, noiseFile, columnsWithSuspiciouslyNoisyPixels, histName, meanOfPixels, linVal, val, pos, rocCol, rocRow):
221  noisyROC = False;
222  if meanOfPixels < self.rocOccupancyTh:
223  # print("Very low mean occupancy: %f in %s in (col, row) (%d, %d)...\tSkipping noisy ROC calculation" % (meanOfPixels, histName, rocCol, rocRow) )
224  noisyROC = True
225 
226  else:
227  th = self.endcapNoisyColumnTh * linVal
228  if val > th:
229  if pos not in columnsWithSuspiciouslyNoisyPixels:
230  rocNum, xCoordInROC = self.__convertCoordinatesFromHistToROCSpace(histName, pos, rocRow)
231  noiseFile.write("%s\t(x, row)->[rocNum, xRoc]\t(%d, %d)->[%d, %d];\t{VAL, TH}\t{%f, %f}\n" % (histName, pos, rocRow+1, rocNum, xCoordInROC, val, th))
232 
233  return 1, noisyROC
234  # else:
235  # print("WARNING: rejecting %s (x, row) (%d, %d) as being affected by a few noisy pixel(s)" % (histName, pos, rocRow+1))
236 
237  return 0, noisyROC
def InefficientDoubleROC.InefficientDeadROCs.__getPixelArrWithRemovedDrops (   self,
  pixelArr,
  medFiltRes 
)
private

Definition at line 168 of file InefficientDoubleROC.py.

References min(), and sistrip::SpyUtilities.range().

Referenced by InefficientDoubleROC.InefficientDeadROCs.ReadHistograms().

168 
169  def __getPixelArrWithRemovedDrops(self, pixelArr, medFiltRes):
170  return [ (pixelArr[i] if pixelArr[i] > medFiltRes[i] else medFiltRes[i]) if 0 < i < len(pixelArr) - 1 else min(medFiltRes) for i in range(len(pixelArr))]
sistrip::SpyUtilities::range
const uint16_t range(const Frame &aFrame)
Definition: SiStripSpyUtilities.cc:76
min
T min(T a, T b)
Definition: MathUtil.h:58
def InefficientDoubleROC.InefficientDeadROCs.__getROCData (   self,
  hist,
  startPixel,
  endPixel,
  row,
  repeatFilter = 3,
  filterKernelSize = 5 
)
private

Definition at line 137 of file InefficientDoubleROC.py.

References InefficientDoubleROC.InefficientDeadROCs.__customMedianFilter(), InefficientDoubleROC.InefficientDeadROCs.pixelNoisynessTh, sistrip::SpyUtilities.range(), and InefficientDoubleROC.InefficientDeadROCs.rocMaxRow.

Referenced by InefficientDoubleROC.InefficientDeadROCs.ReadHistograms().

137 
138  def __getROCData(self, hist, startPixel, endPixel, row, repeatFilter = 3, filterKernelSize = 5):
139  pixelArr = []
140  columnsWithSuspiciouslyNoisyPixels = []
141 
142  for x in range(startPixel, endPixel):
143 
144  columnPixels = [hist.GetBinContent(x, y + 1) for y in range(row * self.rocMaxRow, (row + 1) * self.rocMaxRow)]
145  columnSum = sum(columnPixels)
146 
147  columnMean = columnSum / len(columnPixels)
148  for i in range(len(columnPixels)):
149  if columnPixels[i] > self.pixelNoisynessTh * columnMean:
150  # col = (startPixel % self.rocMaxCol) + 1
151 
152  columnsWithSuspiciouslyNoisyPixels.append(x)
153 
154  # print("WARNING:\t %s : %dx%d:%d may contain NOISY PIXELS instead of NOISY COLUMNS" % (hist.GetName(), col, row + 1, startPixel + i))
155  break
156 
157  pixelArr.append(columnSum)
158 
159  if len(pixelArr) == 0:
160  return None, None, None # ROC down
161 
162  medFiltRes, sciPyMedFiltRes = deepcopy(pixelArr), deepcopy(pixelArr)
163  for i in range(repeatFilter):
164  sciPyMedFiltRes = signal.medfilt(sciPyMedFiltRes, filterKernelSize) # 5 is obligatory to filter doublets!!!
165  medFiltRes = self.__customMedianFilter(medFiltRes, filterKernelSize // 2)
166 
167  return pixelArr, medFiltRes, columnsWithSuspiciouslyNoisyPixels, sciPyMedFiltRes
sistrip::SpyUtilities::range
const uint16_t range(const Frame &aFrame)
Definition: SiStripSpyUtilities.cc:76
def InefficientDoubleROC.InefficientDeadROCs.__lmsExp (   self,
  data,
  xMin,
  xMax 
)
private

Definition at line 87 of file InefficientDoubleROC.py.

References sistrip::SpyUtilities.range().

87 
88  def __lmsExp(self, data, xMin, xMax):
89  meanOfX = (xMax + xMin) * 0.5
90  meanOfY = sum( [math.log(data[i]) for i in range(len(data))] ) / len(data)
91 
92  D = 0
93  for i in range(xMin, xMax + 1):
94  D = D + (i - meanOfX)**2
95  # print(D)
96 
97  a = 0
98  for i in range(len(data)):
99  a = a + math.log(data[i]) * (xMin + i - meanOfX)
100  a = a/D
101 
102  lnb = meanOfY - a * meanOfX
103 
104  return a, math.exp(lnb)
sistrip::SpyUtilities::range
const uint16_t range(const Frame &aFrame)
Definition: SiStripSpyUtilities.cc:76
def InefficientDoubleROC.InefficientDeadROCs.__lmsLin (   self,
  data,
  xMin,
  xMax 
)
private

Definition at line 105 of file InefficientDoubleROC.py.

References sistrip::SpyUtilities.range().

Referenced by InefficientDoubleROC.InefficientDeadROCs.__determineEndcapDColInefficiencyAndNoise().

105 
106  def __lmsLin(self, data, xMin, xMax):
107  meanOfX = (xMax + xMin) * 0.5
108  meanOfY = sum(data) / len(data)
109 
110  D = 0
111  for i in range(xMin, xMax + 1):
112  D = D + (i - meanOfX)**2
113 
114  a = 0
115  for i in range(len(data)):
116  a = a + data[i] * (xMin + i - meanOfX)
117  a = a/D
118 
119  b = meanOfY - a * meanOfX
120 
121  return a, b, D
sistrip::SpyUtilities::range
const uint16_t range(const Frame &aFrame)
Definition: SiStripSpyUtilities.cc:76
def InefficientDoubleROC.InefficientDeadROCs.__normalizeArray (   self,
  pixelArr 
)
private

Definition at line 171 of file InefficientDoubleROC.py.

References SiStripPI.max, min(), and sistrip::SpyUtilities.range().

Referenced by InefficientDoubleROC.InefficientDeadROCs.ReadHistograms().

171 
172  def __normalizeArray(self, pixelArr):
173  c_min, c_max = min(pixelArr), max(pixelArr)
174  if c_min != c_max:
175  c_diff_inv = 1.0 / (c_max - c_min)
176  return [ (pixelArr[i] - c_min) * c_diff_inv for i in range(len(pixelArr))]
177  return [0 for i in range(len(pixelArr))]
sistrip::SpyUtilities::range
const uint16_t range(const Frame &aFrame)
Definition: SiStripSpyUtilities.cc:76
min
T min(T a, T b)
Definition: MathUtil.h:58
def InefficientDoubleROC.InefficientDeadROCs.__setNormalizedArrayZeroInThePoint (   self,
  pixelArr,
  pt 
)
private

Definition at line 178 of file InefficientDoubleROC.py.

References sistrip::SpyUtilities.range().

178 
179  def __setNormalizedArrayZeroInThePoint(self, pixelArr, pt):
180  c_diff_inv = 1.0 / (1.0 - pt)
181 
182  return [ (pixelArr[i] - pt) * c_diff_inv for i in range(len(pixelArr))]
sistrip::SpyUtilities::range
const uint16_t range(const Frame &aFrame)
Definition: SiStripSpyUtilities.cc:76
def InefficientDoubleROC.InefficientDeadROCs.__TraverseDirTree (   self,
  dir 
)
private

Definition at line 15 of file InefficientDoubleROC.py.

References InefficientDoubleROC.InefficientDeadROCs.__TraverseDirTree(), bitset_utilities.append(), InefficientDoubleROC.InefficientDeadROCs.dicOfModuleHistograms, and InefficientDoubleROC.InefficientDeadROCs.lookForStr.

Referenced by InefficientDoubleROC.InefficientDeadROCs.__TraverseDirTree(), PhaseITreeProducer.ModuleLvlValuesReader.__TraverseDirTree(), and TH2PolyOfflineMaps.TH2PolyOfflineMaps.__TraverseDirTree().

15 
16  def __TraverseDirTree(self, dir):
17 
18  for obj in dir.GetListOfKeys():
19  if not obj.IsFolder():
20  if obj.ReadObjectAny(TClass.GetClass("TH2")):
21  th1 = deepcopy(obj.ReadObj())
22  name = th1.GetName()
23  if name.startswith(self.lookForStr): #take only module lvl plots
24  # print(''.join([dir.GetPath(), '/', name]))
25  newName = name.split(self.lookForStr)[1]
26  th1.SetName(newName)
27 
28  # used to sort outputs by disk/layer
29  layer = 0
30  # print(newName)
31  if newName.startswith("B"):
32  layer = "B" + ((newName.split("_LYR"))[1])[0]
33  else:
34  layer = ((newName.split("_D"))[1])[0]
35  if newName.startswith("FPix_Bm"):
36  layer = "-" + layer
37  layer = "F" + layer
38 
39  if layer in self.dicOfModuleHistograms:
40  self.dicOfModuleHistograms[layer].append(th1)
41  else:
42  self.dicOfModuleHistograms.update({layer : [th1]})
43  else:
44  self.__TraverseDirTree(obj.ReadObj())
bitset_utilities::append
boost::dynamic_bitset append(const boost::dynamic_bitset<> &bs1, const boost::dynamic_bitset<> &bs2)
this method takes two bitsets bs1 and bs2 and returns result of bs2 appended to the end of bs1 ...
Definition: bitset_append.cc:10
def InefficientDoubleROC.InefficientDeadROCs.ReadHistograms (   self)

Definition at line 382 of file InefficientDoubleROC.py.

References InefficientDoubleROC.InefficientDeadROCs.__determineBarrelDColInefficiencyAndNoise(), InefficientDoubleROC.InefficientDeadROCs.__determineEndcapDColInefficiencyAndNoise(), InefficientDoubleROC.InefficientDeadROCs.__getPixelArrWithRemovedDrops(), InefficientDoubleROC.InefficientDeadROCs.__getROCData(), InefficientDoubleROC.InefficientDeadROCs.__normalizeArray(), InefficientDoubleROC.InefficientDeadROCs.dicOfModuleHistograms, InefficientDoubleROC.InefficientDeadROCs.noiseOutputFileName, DTSurveyConvert.outputFileName, InefficientDoubleROC.InefficientDeadROCs.outputFileName, SiStripGainCosmicCalculator.outputFileName, CSCSkim.outputFileName, print(), sistrip::SpyUtilities.range(), InefficientDoubleROC.InefficientDeadROCs.rocMaxCol, and InefficientDoubleROC.InefficientDeadROCs.rocsInRow.

382 
383  def ReadHistograms(self):
384  doubleDeadCols, noisyColsNum = 0, 0
385 
386  with open(self.noiseOutputFileName, "w") as noiseFile:
387  with open(self.outputFileName, "w") as outputFile:
388  for layer in self.dicOfModuleHistograms:
389 
390  doubleDeadColsInLayer, noisyColsNumInLayer = 0, 0
391 
392  outputFile.write("-> " + layer + "\n\n")
393  noiseFile.write("-> " + layer + "\n\n")
394 
395  for hist in self.dicOfModuleHistograms[layer]:
396  for row in range(2):
397  for rocNum in range(self.rocsInRow):
398  startPixel = rocNum * self.rocMaxCol + 1
399  endPixel = (rocNum + 1) * self.rocMaxCol + 1 # - 1 ???
400 
401  rocCol = rocNum + 1
402 
403  pixelArr, medFiltRes, columnsWithSuspiciouslyNoisyPixels, sciPyMedFiltRes = self.__getROCData(hist, startPixel, endPixel, row, 3, 5)
404 
405  if pixelArr == None:
406  continue
407 
408  # meanOfPixels = sum(pixelArr) / len(pixelArr)
409  # pixelArrSorted = deepcopy(pixelArr)
410  # pixelArrSorted.sort()
411  # outputFile.write("%s: <x> <med_min> VS. <med_max> | x_min:\t%f %f %f | %f, %f, %f, %f\n" % (hist.GetName(), meanOfPixels, min(medFiltRes), max(medFiltRes), pixelArrSorted[0], pixelArrSorted[1], pixelArrSorted[2], pixelArrSorted[3]))
412 
413  if "F" not in layer:
414  # pixelArrWithoutDrops = self.__getPixelArrWithRemovedDrops(pixelArr, medFiltRes)
415  pixelArrWithoutDrops = self.__getPixelArrWithRemovedDrops(pixelArr, sciPyMedFiltRes)
416  pixelArrWithoutDropsNormalized = self.__normalizeArray(pixelArrWithoutDrops)
417  # tmp_mean = sum(pixelArrWithoutDropsNormalized) / len(pixelArrWithoutDropsNormalized)
418  # pixelArrWithoutDropsNormalized = self.__setNormalizedArrayZeroInThePoint(pixelArrWithoutDropsNormalized, tmp_mean)
419 
420  # print(min(pixelArrWithoutDropsNormalized), max(pixelArrWithoutDropsNormalized))
421  result = self.__determineBarrelDColInefficiencyAndNoise(medFiltRes, hist.GetName(), pixelArr, pixelArrWithoutDropsNormalized, startPixel, rocCol, row, outputFile, columnsWithSuspiciouslyNoisyPixels, noiseFile)
422  else:
423  result = self.__determineEndcapDColInefficiencyAndNoise(medFiltRes, hist.GetName(), pixelArr, startPixel, rocCol, row, outputFile, columnsWithSuspiciouslyNoisyPixels, noiseFile)
424 
425  doubleDeadCols, doubleDeadColsInLayer = doubleDeadCols + result[0], doubleDeadColsInLayer + result[0]
426  noisyColsNum, noisyColsNumInLayer = noisyColsNum + result[1], noisyColsNumInLayer + result[1]
427 
428  outputFile.write("\n\tTOTAL IN LAYER/DISK: %d\n\n" % (doubleDeadColsInLayer))
429  noiseFile.write("\n\tTOTAL IN LAYER/DISK: %d\n\n" % (noisyColsNumInLayer))
430 
431  print("Number of inefficient double columns: %d"%(doubleDeadCols))
432  print("Number of noisy cols: %d"%(noisyColsNum))
433 
434 
#--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--
sistrip::SpyUtilities::range
const uint16_t range(const Frame &aFrame)
Definition: SiStripSpyUtilities.cc:76
print
void print(TMatrixD &m, const char *label=nullptr, bool mathematicaFormat=false)
Definition: Utilities.cc:47

Member Data Documentation

InefficientDoubleROC.InefficientDeadROCs.barrelInefficientDColTh

Definition at line 70 of file InefficientDoubleROC.py.

Referenced by InefficientDoubleROC.InefficientDeadROCs.__determineBarrelDColInefficiencyAndNoise().

InefficientDoubleROC.InefficientDeadROCs.barrelNoisyColumnTh

Definition at line 66 of file InefficientDoubleROC.py.

Referenced by InefficientDoubleROC.InefficientDeadROCs.__determineBarrelNoise().

InefficientDoubleROC.InefficientDeadROCs.barrelNoisyColumnTh2

Definition at line 67 of file InefficientDoubleROC.py.

Referenced by InefficientDoubleROC.InefficientDeadROCs.__determineBarrelNoise2().

InefficientDoubleROC.InefficientDeadROCs.detDict

Definition at line 82 of file InefficientDoubleROC.py.

Referenced by PhaseITreeProducer.ModuleLvlValuesReader.ReadHistograms(), and TH2PolyOfflineMaps.TH2PolyOfflineMaps.ReadHistograms().

InefficientDoubleROC.InefficientDeadROCs.dicOfModuleHistograms

Definition at line 60 of file InefficientDoubleROC.py.

Referenced by InefficientDoubleROC.InefficientDeadROCs.__TraverseDirTree(), and InefficientDoubleROC.InefficientDeadROCs.ReadHistograms().

InefficientDoubleROC.InefficientDeadROCs.dirs

Definition at line 50 of file InefficientDoubleROC.py.

Referenced by PhaseITreeProducer.ModuleLvlValuesReader.__TraverseDirTree(), and TH2PolyOfflineMaps.TH2PolyOfflineMaps.__TraverseDirTree().

InefficientDoubleROC.InefficientDeadROCs.endcapInefficientDColTh

Definition at line 71 of file InefficientDoubleROC.py.

Referenced by InefficientDoubleROC.InefficientDeadROCs.__determineEndcapDColInefficiencyAndNoise().

InefficientDoubleROC.InefficientDeadROCs.endcapNoisyColumnTh

Definition at line 68 of file InefficientDoubleROC.py.

Referenced by InefficientDoubleROC.InefficientDeadROCs.__determineEndcapNoise().

InefficientDoubleROC.InefficientDeadROCs.inputFile

Definition at line 59 of file InefficientDoubleROC.py.

InefficientDoubleROC.InefficientDeadROCs.inputFileName

Definition at line 47 of file InefficientDoubleROC.py.

InefficientDoubleROC.InefficientDeadROCs.lookForStr

Definition at line 52 of file InefficientDoubleROC.py.

Referenced by InefficientDoubleROC.InefficientDeadROCs.__TraverseDirTree().

InefficientDoubleROC.InefficientDeadROCs.noiseOutputFileName

Definition at line 49 of file InefficientDoubleROC.py.

Referenced by InefficientDoubleROC.InefficientDeadROCs.ReadHistograms().

InefficientDoubleROC.InefficientDeadROCs.outputFileName

Definition at line 48 of file InefficientDoubleROC.py.

Referenced by PhaseITreeProducer.ModuleLvlValuesReader.CreateTree2(), and InefficientDoubleROC.InefficientDeadROCs.ReadHistograms().

InefficientDoubleROC.InefficientDeadROCs.pixelNoisynessTh

THRESHOLDS SECTION.

Definition at line 63 of file InefficientDoubleROC.py.

Referenced by InefficientDoubleROC.InefficientDeadROCs.__getROCData().

InefficientDoubleROC.InefficientDeadROCs.rocMaxCol

Definition at line 54 of file InefficientDoubleROC.py.

Referenced by InefficientDoubleROC.InefficientDeadROCs.__convertCoordinatesFromHistToROCSpace(), InefficientDoubleROC.InefficientDeadROCs.__determineBarrelDColInefficiencyAndNoise(), InefficientDoubleROC.InefficientDeadROCs.__determineEndcapDColInefficiencyAndNoise(), and InefficientDoubleROC.InefficientDeadROCs.ReadHistograms().

InefficientDoubleROC.InefficientDeadROCs.rocMaxRow

Definition at line 55 of file InefficientDoubleROC.py.

Referenced by InefficientDoubleROC.InefficientDeadROCs.__getROCData().

InefficientDoubleROC.InefficientDeadROCs.rocOccupancyTh

Definition at line 64 of file InefficientDoubleROC.py.

Referenced by InefficientDoubleROC.InefficientDeadROCs.__determineBarrelNoise(), InefficientDoubleROC.InefficientDeadROCs.__determineBarrelNoise2(), and InefficientDoubleROC.InefficientDeadROCs.__determineEndcapNoise().

InefficientDoubleROC.InefficientDeadROCs.rocsInCol

Definition at line 57 of file InefficientDoubleROC.py.

InefficientDoubleROC.InefficientDeadROCs.rocsInRow

Definition at line 56 of file InefficientDoubleROC.py.

Referenced by InefficientDoubleROC.InefficientDeadROCs.ReadHistograms().

Generated for CMSSW Reference Manual by doxygen 1.8.5