CMS 3D CMS Logo

All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Properties Friends Macros Modules Pages
List of all members | Public Member Functions | Private Attributes
conddb2hdf5.DBPayloadIterator Class Reference
Inheritance diagram for conddb2hdf5.DBPayloadIterator:

Public Member Functions

def __init__ (self, args, payloads)
 
def __iter__ (self)
 
def __next__ (self)
 

Private Attributes

 _args
 
 _cacheChunking
 
 _nextIndex
 
 _payloadCache
 
 _payloadHashs
 
 _payloadHashsIndex
 
 _safeChunkingSize
 

Detailed Description

Definition at line 157 of file conddb2hdf5.py.

Constructor & Destructor Documentation

◆ __init__()

def conddb2hdf5.DBPayloadIterator.__init__ (   self,
  args,
  payloads 
)

Definition at line 158 of file conddb2hdf5.py.

158  def __init__(self, args, payloads):
159  self._args = args
160  self._payloadHashs = payloads
161  self._payloadCache = {}
162  self._payloadHashsIndex = 0
163  self._cacheChunking = 1
164  self._safeChunkingSize = 1
165  self._nextIndex = 0
def __init__(self, dataset, job_number, job_id, job_name, isDA, isMC, applyBOWS, applyEXTRACOND, extraconditions, runboundary, lumilist, intlumi, maxevents, gt, allFromGT, alignmentDB, alignmentTAG, apeDB, apeTAG, bowDB, bowTAG, vertextype, tracktype, refittertype, ttrhtype, applyruncontrol, ptcut, CMSSW_dir, the_dir)

Member Function Documentation

◆ __iter__()

def conddb2hdf5.DBPayloadIterator.__iter__ (   self)

Definition at line 166 of file conddb2hdf5.py.

166  def __iter__(self):
167  return self

◆ __next__()

def conddb2hdf5.DBPayloadIterator.__next__ (   self)

Definition at line 168 of file conddb2hdf5.py.

References mps_alisetup.SetupAlignment._args, tkal_create_file_lists.FileListCreator._args, conddb2hdf5.DBPayloadIterator._args, conddb2hdf5.DBPayloadIterator._cacheChunking, conddb2hdf5.DBPayloadIterator._nextIndex, conddb2hdf5.DBPayloadIterator._payloadCache, conddb2hdf5.DBPayloadIterator._payloadHashs, conddb2hdf5.DBPayloadIterator._payloadHashsIndex, conddb2hdf5.DBPayloadIterator._safeChunkingSize, and conddb2hdf5.canonicalProductName().

168  def __next__(self):
169  if self._nextIndex >= len(self._payloadHashs):
170  raise StopIteration()
171  payloadHash = self._payloadHashs[self._nextIndex]
172  if not self._payloadCache:
173  self._cacheChunking = self._safeChunkingSize
174  queue = mp.Queue()
175  p=mp.Process(target=external_process_get_payloads_objtype_data, args=(queue, self._args, self._payloadHashs[self._payloadHashsIndex:self._payloadHashsIndex+self._cacheChunking]))
176  p.start()
177  table = queue.get()
178  p.join()
179  #table = get_payloads_objtype_data(session, payloadHashs[payloadHashsIndex:payloadHashsIndex+cacheChunking])
180  #print(table)
181  self._payloadHashsIndex +=self._cacheChunking
182  for r in table:
183  self._payloadCache[r[0]] = (r[1],r[2])
184  objtype,data = self._payloadCache[payloadHash]
185  if len(data) < 1000000:
186  self._safeChunkingSize = 10
187  del self._payloadCache[payloadHash]
188  self._nextIndex +=1
189  return DBPayload(payloadHash, canonicalProductName(objtype.encode("ascii")), data)
190 
191 
def canonicalProductName(product)
Definition: conddb2hdf5.py:541

Member Data Documentation

◆ _args

conddb2hdf5.DBPayloadIterator._args
private

◆ _cacheChunking

conddb2hdf5.DBPayloadIterator._cacheChunking
private

Definition at line 163 of file conddb2hdf5.py.

Referenced by conddb2hdf5.DBPayloadIterator.__next__().

◆ _nextIndex

conddb2hdf5.DBPayloadIterator._nextIndex
private

Definition at line 165 of file conddb2hdf5.py.

Referenced by conddb2hdf5.DBPayloadIterator.__next__().

◆ _payloadCache

conddb2hdf5.DBPayloadIterator._payloadCache
private

Definition at line 161 of file conddb2hdf5.py.

Referenced by conddb2hdf5.DBPayloadIterator.__next__().

◆ _payloadHashs

conddb2hdf5.DBPayloadIterator._payloadHashs
private

◆ _payloadHashsIndex

conddb2hdf5.DBPayloadIterator._payloadHashsIndex
private

Definition at line 162 of file conddb2hdf5.py.

Referenced by conddb2hdf5.DBPayloadIterator.__next__().

◆ _safeChunkingSize

conddb2hdf5.DBPayloadIterator._safeChunkingSize
private

Definition at line 164 of file conddb2hdf5.py.

Referenced by conddb2hdf5.DBPayloadIterator.__next__().