CMS 3D CMS Logo

 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Properties Friends Macros Pages
List of all members | Public Member Functions | Public Attributes
fff_deleter.FileDeleter Class Reference
Inheritance diagram for fff_deleter.FileDeleter:

Public Member Functions

def __init__
 
def delete
 
def make_report
 
def rename
 
def run
 
def send_smg
 

Public Attributes

 email_to
 
 fake
 
 hostname
 
 last_email
 
 min_interval
 
 report_directory
 
 sequence
 
 thresholds
 
 top
 

Detailed Description

Definition at line 86 of file fff_deleter.py.

Constructor & Destructor Documentation

def fff_deleter.FileDeleter.__init__ (   self,
  top,
  thresholds,
  email_to,
  report_directory,
  fake = True 
)

Definition at line 87 of file fff_deleter.py.

87 
88  def __init__(self, top, thresholds, email_to, report_directory, fake=True, ):
89  self.top = top
90  self.fake = fake
91  self.email_to = email_to
92  self.thresholds = thresholds
93  self.report_directory = report_directory
94  self.sequence = 0
95 
96  self.last_email = None
97  self.min_interval = datetime.timedelta(seconds=60*10)
98  self.hostname = socket.gethostname()

Member Function Documentation

def fff_deleter.FileDeleter.delete (   self,
  f 
)

Definition at line 114 of file fff_deleter.py.

References ValidationMisalignedTracker.fake, and fff_deleter.FileDeleter.fake.

Referenced by Vispa.Gui.PortConnection.PointToPointConnection.keyPressEvent(), Vispa.Views.LineDecayView.LineDecayContainer.keyPressEvent(), Vispa.Gui.VispaWidget.VispaWidget.keyPressEvent(), and fff_deleter.FileDeleter.run().

115  def delete(self, f):
116  if not f.endswith(".deleted"):
117  return
118 
119  if self.fake:
120  log.warning("Truncating file (fake): %s", f)
121  else:
122  log.warning("Truncating file: %s", f)
123  open(f, "w").close()
def fff_deleter.FileDeleter.make_report (   self,
  logout 
)

Definition at line 146 of file fff_deleter.py.

References python.multivaluedict.dict, fff_deleter.diskusage_bytes(), fff_deleter.FileDeleter.hostname, list(), python.multivaluedict.map(), fff_deleter.FileDeleter.report_directory, fff_deleter.FileDeleter.sequence, ConfigBuilder.ConfigBuilder.sequence, and fff_deleter.FileDeleter.top.

Referenced by esMonitoring.ElasticReport.try_update().

147  def make_report(self, logout):
148  if not os.path.isdir(self.report_directory):
149  log.warning("Directory %s does not exists. Reports disabled.", self.report_directory)
150  return
151 
152  meminfo = list(open("/proc/meminfo", "r").readlines())
153  def entry_to_dict(line):
154  key, value = line.split()[:2]
155  value = int(value)
156  return (key.strip(":"), value, )
157  meminfo = dict(map(entry_to_dict, meminfo))
158 
159  if os.path.isdir(self.top):
160  used, free, total = diskusage_bytes(self.top)
161  else:
162  used, free, total = -1, -1, -1
163 
164  doc = {
165  "sequence": self.sequence,
166  "memory_used": (meminfo["MemTotal"] - meminfo["MemFree"]) * 1024,
167  "memory_free": meminfo["MemFree"] * 1024,
168  "memory_total": meminfo["MemTotal"] * 1024,
169  "disk_used": used,
170  "disk_free": free,
171  "disk_total": total,
172  "hostname": self.hostname,
173  "extra": {
174  "meminfo": meminfo,
175  "last_log": logout.split("\n"),
176  },
177  "pid": os.getpid(),
178  "_id": "dqm-diskspace-%s" % self.hostname,
179  "type": "dqm-diskspace"
180  }
181 
182  fn = "dqm-diskspace-seq%06d" % (doc["sequence"], )
183  tmp_fp = os.path.join(self.report_directory, "." + fn + ".tmp")
184  final_fp = os.path.join(self.report_directory, fn + ".jsn")
185  fd = open(tmp_fp, "w")
186 
187  json.dump(doc, fd, indent=True)
188  fd.write("\n")
189  fd.close()
190 
191  os.rename(tmp_fp, final_fp)
def diskusage_bytes
Definition: fff_deleter.py:78
How EventSelector::AcceptEvent() decides whether to accept an event for output otherwise it is excluding the probing of A single or multiple positive and the trigger will pass if any such matching triggers are PASS or EXCEPTION[A criterion thatmatches no triggers at all is detected and causes a throw.] A single negative with an expectation of appropriate bit checking in the decision and the trigger will pass if any such matching triggers are FAIL or EXCEPTION A wildcarded negative criterion that matches more than one trigger in the trigger list("!*","!HLTx*"if it matches 2 triggers or more) will accept the event if all the matching triggers are FAIL.It will reject the event if any of the triggers are PASS or EXCEPTION(this matches the behavior of"!*"before the partial wildcard feature was incorporated).Triggers which are in the READY state are completely ignored.(READY should never be returned since the trigger paths have been run
def fff_deleter.FileDeleter.rename (   self,
  f 
)

Definition at line 99 of file fff_deleter.py.

References ValidationMisalignedTracker.fake, and fff_deleter.FileDeleter.fake.

Referenced by fff_deleter.FileDeleter.run().

99 
100  def rename(self, f):
101  if f.endswith(".deleted"):
102  return
103 
104  fn = f + ".deleted"
105 
106  if self.fake:
107  log.warning("Renaming file (fake): %s -> %s", f,
108  os.path.relpath(fn, os.path.dirname(f)))
109  else:
110  log.warning("Renaming file: %s -> %s", f,
111  os.path.relpath(fn, os.path.dirname(f)))
112 
113  os.rename(f, fn)
def fff_deleter.FileDeleter.run (   self)

Definition at line 192 of file fff_deleter.py.

References fff_deleter.cleanup_threshold(), fff_deleter.FileDeleter.delete(), fff_deleter.diskusage(), fff_deleter.FileDeleter.rename(), fff_deleter.FileDeleter.sequence, ConfigBuilder.ConfigBuilder.sequence, fff_deleter.FileDeleter.thresholds, and fff_deleter.FileDeleter.top.

Referenced by Types.EventID.cppID(), and Types.LuminosityBlockID.cppID().

193  def run(self):
194  self.sequence += 1
195  if not os.path.isdir(self.top):
196  log.warning("Directory %s does not exists.", self.top)
197  return
198 
199  cleanup_threshold(self.top, self.thresholds['rename'], self.rename, "rename")
200  cleanup_threshold(self.top, self.thresholds['delete'], self.delete, "delete")
201 
202  du = diskusage(self.top)
203  if du > self.thresholds['email']:
204  deleter.send_smg(du)
205 
206 # use a named socket check if we are running
207 # this is very clean and atomic and leave no files
# from: http://stackoverflow.com/a/7758075
def cleanup_threshold
Definition: fff_deleter.py:54
def fff_deleter.FileDeleter.send_smg (   self,
  used_pc 
)

Definition at line 124 of file fff_deleter.py.

References fff_deleter.FileDeleter.email_to, fff_deleter.FileDeleter.hostname, fff_deleter.FileDeleter.last_email, and fff_deleter.FileDeleter.min_interval.

125  def send_smg(self, used_pc):
126  now = datetime.datetime.now()
127 
128  if (self.last_email is not None):
129  if (now - self.last_email) < self.min_interval:
130  return
131 
132  self.last_email = now
133 
134  # sms service does not accept an email with a several recipients
135  # so we send one-by-one
136  for email in self.email_to:
137  subject = "Disk out of space (%.02f%%) on %s." % (used_pc, self.hostname)
138  if "mail2sms" in email:
139  text = ""
140  else:
141  text = subject
142 
143  log.info("Sending email: %s", repr(["/bin/mail", "-s", subject, email]))
144  p = subprocess.Popen(["/bin/mail", "-s", subject, email], stdin=subprocess.PIPE, shell=False)
145  p.communicate(input=text)

Member Data Documentation

fff_deleter.FileDeleter.email_to

Definition at line 90 of file fff_deleter.py.

Referenced by fff_deleter.FileDeleter.send_smg().

fff_deleter.FileDeleter.fake

Definition at line 89 of file fff_deleter.py.

Referenced by fff_deleter.FileDeleter.delete(), and fff_deleter.FileDeleter.rename().

fff_deleter.FileDeleter.hostname

Definition at line 97 of file fff_deleter.py.

Referenced by fff_deleter.FileDeleter.make_report(), and fff_deleter.FileDeleter.send_smg().

fff_deleter.FileDeleter.last_email

Definition at line 95 of file fff_deleter.py.

Referenced by fff_deleter.FileDeleter.send_smg().

fff_deleter.FileDeleter.min_interval

Definition at line 96 of file fff_deleter.py.

Referenced by fff_deleter.FileDeleter.send_smg().

fff_deleter.FileDeleter.report_directory

Definition at line 92 of file fff_deleter.py.

Referenced by fff_deleter.FileDeleter.make_report().

fff_deleter.FileDeleter.sequence

Definition at line 93 of file fff_deleter.py.

Referenced by fff_deleter.FileDeleter.make_report(), and fff_deleter.FileDeleter.run().

fff_deleter.FileDeleter.thresholds

Definition at line 91 of file fff_deleter.py.

Referenced by fff_deleter.FileDeleter.run().

fff_deleter.FileDeleter.top

Definition at line 88 of file fff_deleter.py.

Referenced by fff_deleter.FileDeleter.make_report(), and fff_deleter.FileDeleter.run().