CMS 3D CMS Logo
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Properties Friends Macros Pages
List of all members | Public Member Functions | Public Attributes | Private Member Functions
dirstructure.Directory Class Reference
Inheritance diagram for dirstructure.Directory:
dirstructure.Weighted

Public Member Functions

def __init__
 
def __repr__
 
def calcStats
 
def get_fail_rate
 
def get_null_rate
 
def get_skiped_rate
 
def get_subdirs_dict
 
def get_subdirs_names
 
def get_success_rate
 
def get_summary_chart_ajax
 
def get_summary_chart_name
 
def is_empty
 
def print_report
 
def prune
 
- Public Member Functions inherited from dirstructure.Weighted
def __init__
 

Public Attributes

 comparisons
 
 different_histograms
 
 do_pngs
 
 draw_success
 
 filename1
 
 filename2
 
 full_path
 
 meta
 
 mother_dir
 
 n_comp_fails
 
 n_comp_nulls
 
 n_comp_skiped
 
 n_comp_successes
 
 n_fails
 
 n_missing_objs
 
 n_nulls
 
 n_skiped
 
 n_successes
 
 rank_histo
 
 stats_calculated
 
 subdirs
 
 weight
 
- Public Attributes inherited from dirstructure.Weighted
 name
 
 weight
 

Private Member Functions

def __create_on_disk
 
def __create_pie_image
 
def __get_full_path
 

Detailed Description

Definition at line 59 of file dirstructure.py.

Constructor & Destructor Documentation

def dirstructure.Directory.__init__ (   self,
  name,
  mother_dir = "",
  meta = CompInfo(),
  draw_success = False,
  do_pngs = False 
)

Definition at line 60 of file dirstructure.py.

60 
61  def __init__(self,name,mother_dir="",meta=CompInfo(),draw_success=False,do_pngs=False):
62  self.mother_dir=mother_dir
63  self.meta=meta
64  self.subdirs=[]
65  self.comparisons=[]
66  self.n_fails=0
67  self.n_successes=0
68  self.n_nulls=0
69  self.n_skiped = 0
70  self.n_comp_skiped = 0
71  self.n_comp_fails=0
72  self.n_comp_successes=0
73  self.n_comp_nulls=0
74  self.weight=0
75  self.stats_calculated=False
76  Weighted.__init__(self,name)
77  self.draw_success=draw_success
78  self.do_pngs=do_pngs
79  self.rank_histo=TH1I("rh%s"%name,"",50,-0.01,1.001)
80  self.rank_histo.SetDirectory(0)
81  self.different_histograms = {}
82  self.different_histograms['file1']= {}
83  self.different_histograms['file2']= {}
84  self.filename1 = ""
85  self.filename2 = ""
86  self.n_missing_objs = 0
87  self.full_path = ""

Member Function Documentation

def dirstructure.Directory.__create_on_disk (   self)
private

Definition at line 223 of file dirstructure.py.

References dirstructure.Directory.__get_full_path(), and dirstructure.Directory.mother_dir.

Referenced by dirstructure.Directory.__create_pie_image().

223 
224  def __create_on_disk(self):
225  if not exists(self.mother_dir) and len(self.mother_dir)!=0:
226  makedirs(self.mother_dir)
227  full_path=self.__get_full_path()
228  if not exists(full_path) and len(full_path)>0:
229  makedirs(full_path)
def dirstructure.Directory.__create_pie_image (   self)
private

Definition at line 233 of file dirstructure.py.

References dirstructure.Directory.__create_on_disk(), dirstructure.Directory.get_fail_rate(), dirstructure.Directory.get_null_rate(), dirstructure.Directory.get_skiped_rate(), dirstructure.Directory.get_success_rate(), dirstructure.Directory.get_summary_chart_name(), dirstructure.Directory.n_fails, dirstructure.Directory.n_nulls, dirstructure.Directory.n_skiped, dirstructure.Directory.n_successes, entry.name, genericValidation.GenericValidation.name, TmModule.name, EventMonitor::Entry.name, ora::RecordSpecImpl::Item.name, cond::persistency::GTEditorData.name, alignment.Alignment.name, FWTGeoRecoGeometry::Info.name, Types._Untracked.name, ParameterSet.name, fit::RootMinuitCommand.name, CombinedMVAJetTagComputer::Computer.name, PixelDCSObject< class >::Item.name, MagCylinder.name, DQMRivetClient::LumiOption.name, PFTauMVAInputDiscriminantTranslator::DiscriminantInfo.name, PrintSensitive.name, cond::persistency::GTProxyData.name, RHStopTracer::StopPoint.name, DQMRivetClient::ScaleFactorOption.name, SingleObjectCondition.name, EgHLTOfflineSummaryClient::SumHistBinData.name, XMLRBXPedestalsLoader::_loaderBaseConfig.name, PhysicsTools::Source.name, XMLHTRZeroSuppressionLoader::_loaderBaseConfig.name, DQMGenericClient::EfficOption.name, MyWatcher.name, edm::PathTimingSummary.name, dirstructure.Weighted.name, cond::TimeTypeSpecs.name, lumi::TriggerInfo.name, PrintMaterialBudgetInfo.name, edm::PathSummary.name, perftools::EdmEventSize::BranchRecord.name, PixelEndcapLinkMaker::Item.name, FWTableViewManager::TableEntry.name, PixelBarrelLinkMaker::Item.name, Mapper::definition< ScannerT >.name, EcalLogicID.name, options.ConnectionHLTMenu.name, ExpressionHisto< T >.name, McSelector.name, python.rootplot.utilities.Hist2D.name, SensitiveDetector.name, RecoSelector.name, DQMGenericClient::NormOption.name, XMLProcessor::_loaderBaseConfig.name, TreeCrawler.Package.name, CaloTrkProcessing::Detector.name, TrajectorySeedProducer::LayerSpec.name, HcalForwardLibWriter::FileHandle.name, PrintGeomInfoAction.name, MagGeoBuilderFromDDD::volumeHandle.name, DQMGenericClient::CDOption.name, PrintGeomMatInfo.name, PhysicsTools::Calibration::Variable.name, cond::TagInfo_t.name, h4DSegm.name, PhysicsTools::Variable::Value.name, options.HLTProcessOptions.name, EDMtoMEConverter.name, ProcTMVA::Method.name, TreeSaver::Var.name, python.rootplot.tree2hists.Plot.name, MEtoEDM< T >::MEtoEDMObject.name, PhysicsTools::TreeReader::Value.name, MuonGeometrySanityCheckPoint.name, PhysicsTools::TrainProcessor.name, TotemSD.name, PhysicsTools::MVAModuleHelper< Record, Object, Filler >::Value.name, PhysicsTools::ProcessRegistry< Base_t, CalibBase_t, Parent_t >.name, PhysicsTools::MVATrainer.name, h2DSegm.name, python.rootplot.utilities.Hist.name, BscSD.name, DQMNet::WaitObject.name, AlpgenParameterName.name, SiStripMonitorDigi.name, FP420SD.name, public_plots_tools.ColorScheme.name, PhysicsTools::Variable.name, PhysicsTools::TrainerMonitoring::Object.name, FastTimerService::LuminosityDescription.name, conddblib.Tag.name, conddblib.GlobalTag.name, plotscripts.SawTeethFunction.name, @16252::Id.name, FastTimerService::ProcessDescription.name, python.rootplot.utilities.RootFile.name, hTMaxCell.name, cscdqm::ParHistoDef.name, BeautifulSoup.Tag.name, @16248::Id.name, TiXmlAttribute.name, BeautifulSoup.SoupStrainer.name, and archive.zip.

233 
234  def __create_pie_image(self):
235  self.__create_on_disk()
236  vals=[]
237  colors=[]
238  for n,col in zip((self.n_fails,self.n_nulls,self.n_successes,self.n_skiped),(kRed,kYellow,kGreen,kBlue)):
239  if n!=0:
240  vals.append(n)
241  colors.append(col)
242  valsa=array('f',vals)
243  colorsa=array('i',colors)
244  can = TCanvas("cpie","TPie test",100,100);
245  try:
246  pie = TPie("ThePie",self.name,len(vals),valsa,colorsa);
247  label_n=0
248  if self.n_fails!=0:
249  pie.SetEntryLabel(label_n, "Fail: %.1f(%i)" %(self.get_fail_rate(),self.n_fails) );
250  label_n+=1
251  if self.n_nulls!=0:
252  pie.SetEntryLabel(label_n, "Null: %.1f(%i)" %(self.get_null_rate(),self.n_nulls) );
253  label_n+=1
254  if self.n_successes!=0:
255  pie.SetEntryLabel(label_n, "Success: %.1f(%i)" %(self.get_success_rate(),self.n_successes) );
256  if self.n_skiped!=0:
257  pie.SetEntryLabel(label_n, "Skipped: %.1f(%i)" %(self.get_skiped_rate(),self.n_skiped));
258  pie.SetY(.52);
259  pie.SetAngularOffset(0.);
260  pie.SetLabelsOffset(-.3);
261  #pie.SetLabelFormat("#splitline{%val (%perc)}{%txt}");
262  pie.Draw("3d nol");
263  can.Print(self.get_summary_chart_name());
264  except:
265  print "self.name = %s" %self.name
266  print "len(vals) = %s (vals=%s)" %(len(vals),vals)
267  print "valsa = %s" %valsa
268  print "colorsa = %s" %colorsa
archive.zip
tuple zip
Definition: archive.py:476
def dirstructure.Directory.__get_full_path (   self)
private

Definition at line 217 of file dirstructure.py.

References join(), dirstructure.Directory.mother_dir, entry.name, genericValidation.GenericValidation.name, TmModule.name, EventMonitor::Entry.name, ora::RecordSpecImpl::Item.name, cond::persistency::GTEditorData.name, alignment.Alignment.name, FWTGeoRecoGeometry::Info.name, Types._Untracked.name, fit::RootMinuitCommand.name, CombinedMVAJetTagComputer::Computer.name, ParameterSet.name, PixelDCSObject< class >::Item.name, DQMRivetClient::LumiOption.name, MagCylinder.name, PFTauMVAInputDiscriminantTranslator::DiscriminantInfo.name, PrintSensitive.name, cond::persistency::GTProxyData.name, RHStopTracer::StopPoint.name, DQMRivetClient::ScaleFactorOption.name, SingleObjectCondition.name, EgHLTOfflineSummaryClient::SumHistBinData.name, PhysicsTools::Source.name, DQMGenericClient::EfficOption.name, XMLHTRZeroSuppressionLoader::_loaderBaseConfig.name, XMLRBXPedestalsLoader::_loaderBaseConfig.name, MyWatcher.name, edm::PathTimingSummary.name, dirstructure.Weighted.name, cond::TimeTypeSpecs.name, lumi::TriggerInfo.name, edm::PathSummary.name, PrintMaterialBudgetInfo.name, PixelEndcapLinkMaker::Item.name, perftools::EdmEventSize::BranchRecord.name, FWTableViewManager::TableEntry.name, PixelBarrelLinkMaker::Item.name, options.ConnectionHLTMenu.name, Mapper::definition< ScannerT >.name, EcalLogicID.name, ExpressionHisto< T >.name, McSelector.name, python.rootplot.utilities.Hist2D.name, SensitiveDetector.name, RecoSelector.name, DQMGenericClient::NormOption.name, TreeCrawler.Package.name, XMLProcessor::_loaderBaseConfig.name, CaloTrkProcessing::Detector.name, TrajectorySeedProducer::LayerSpec.name, HcalForwardLibWriter::FileHandle.name, PrintGeomInfoAction.name, MagGeoBuilderFromDDD::volumeHandle.name, DQMGenericClient::CDOption.name, PrintGeomMatInfo.name, PhysicsTools::Calibration::Variable.name, h4DSegm.name, cond::TagInfo_t.name, PhysicsTools::Variable::Value.name, options.HLTProcessOptions.name, EDMtoMEConverter.name, ProcTMVA::Method.name, TreeSaver::Var.name, python.rootplot.tree2hists.Plot.name, MEtoEDM< T >::MEtoEDMObject.name, PhysicsTools::TreeReader::Value.name, MuonGeometrySanityCheckPoint.name, PhysicsTools::TrainProcessor.name, TotemSD.name, PhysicsTools::MVAModuleHelper< Record, Object, Filler >::Value.name, PhysicsTools::ProcessRegistry< Base_t, CalibBase_t, Parent_t >.name, PhysicsTools::MVATrainer.name, h2DSegm.name, python.rootplot.utilities.Hist.name, BscSD.name, DQMNet::WaitObject.name, AlpgenParameterName.name, SiStripMonitorDigi.name, FP420SD.name, public_plots_tools.ColorScheme.name, PhysicsTools::Variable.name, PhysicsTools::TrainerMonitoring::Object.name, FastTimerService::LuminosityDescription.name, conddblib.Tag.name, conddblib.GlobalTag.name, plotscripts.SawTeethFunction.name, @16252::Id.name, FastTimerService::ProcessDescription.name, python.rootplot.utilities.RootFile.name, hTMaxCell.name, cscdqm::ParHistoDef.name, BeautifulSoup.Tag.name, @16248::Id.name, TiXmlAttribute.name, and BeautifulSoup.SoupStrainer.name.

Referenced by dirstructure.Directory.__create_on_disk(), and dirstructure.Directory.get_summary_chart_name().

217 
218  def __get_full_path(self):
219  #print "Mother is %s" %self.mother_dir
220  if len(self.mother_dir)==0:
221  return self.name
222  return join(self.mother_dir,self.name)
join
static std::string join(char **cmd)
Definition: RemoteFile.cc:18
def dirstructure.Directory.__repr__ (   self)

Definition at line 313 of file dirstructure.py.

References dirstructure.Directory.comparisons, dirstructure.Directory.get_fail_rate(), dirstructure.Directory.get_null_rate(), dirstructure.Directory.get_success_rate(), DEutils< T >.is_empty(), DEcompare< T >.is_empty(), dirstructure.Directory.is_empty(), create_public_lumi_plots.LumiDataBlock.is_empty(), dirstructure.Directory.n_fails, dirstructure.Directory.n_nulls, dirstructure.Directory.n_successes, entry.name, genericValidation.GenericValidation.name, TmModule.name, EventMonitor::Entry.name, ora::RecordSpecImpl::Item.name, cond::persistency::GTEditorData.name, alignment.Alignment.name, FWTGeoRecoGeometry::Info.name, Types._Untracked.name, ParameterSet.name, fit::RootMinuitCommand.name, CombinedMVAJetTagComputer::Computer.name, PixelDCSObject< class >::Item.name, DQMRivetClient::LumiOption.name, MagCylinder.name, PFTauMVAInputDiscriminantTranslator::DiscriminantInfo.name, PrintSensitive.name, cond::persistency::GTProxyData.name, RHStopTracer::StopPoint.name, DQMRivetClient::ScaleFactorOption.name, SingleObjectCondition.name, EgHLTOfflineSummaryClient::SumHistBinData.name, XMLHTRZeroSuppressionLoader::_loaderBaseConfig.name, XMLRBXPedestalsLoader::_loaderBaseConfig.name, PhysicsTools::Source.name, DQMGenericClient::EfficOption.name, MyWatcher.name, edm::PathTimingSummary.name, dirstructure.Weighted.name, cond::TimeTypeSpecs.name, lumi::TriggerInfo.name, PrintMaterialBudgetInfo.name, edm::PathSummary.name, perftools::EdmEventSize::BranchRecord.name, PixelEndcapLinkMaker::Item.name, FWTableViewManager::TableEntry.name, PixelBarrelLinkMaker::Item.name, EcalLogicID.name, options.ConnectionHLTMenu.name, Mapper::definition< ScannerT >.name, ExpressionHisto< T >.name, McSelector.name, SensitiveDetector.name, python.rootplot.utilities.Hist2D.name, RecoSelector.name, DQMGenericClient::NormOption.name, XMLProcessor::_loaderBaseConfig.name, TreeCrawler.Package.name, CaloTrkProcessing::Detector.name, TrajectorySeedProducer::LayerSpec.name, HcalForwardLibWriter::FileHandle.name, PrintGeomInfoAction.name, MagGeoBuilderFromDDD::volumeHandle.name, DQMGenericClient::CDOption.name, PrintGeomMatInfo.name, PhysicsTools::Calibration::Variable.name, cond::TagInfo_t.name, h4DSegm.name, PhysicsTools::Variable::Value.name, options.HLTProcessOptions.name, EDMtoMEConverter.name, ProcTMVA::Method.name, TreeSaver::Var.name, python.rootplot.tree2hists.Plot.name, MEtoEDM< T >::MEtoEDMObject.name, PhysicsTools::TreeReader::Value.name, PhysicsTools::TrainProcessor.name, MuonGeometrySanityCheckPoint.name, TotemSD.name, PhysicsTools::MVAModuleHelper< Record, Object, Filler >::Value.name, PhysicsTools::ProcessRegistry< Base_t, CalibBase_t, Parent_t >.name, PhysicsTools::MVATrainer.name, h2DSegm.name, python.rootplot.utilities.Hist.name, BscSD.name, DQMNet::WaitObject.name, AlpgenParameterName.name, SiStripMonitorDigi.name, FP420SD.name, public_plots_tools.ColorScheme.name, PhysicsTools::Variable.name, PhysicsTools::TrainerMonitoring::Object.name, FastTimerService::LuminosityDescription.name, conddblib.Tag.name, conddblib.GlobalTag.name, plotscripts.SawTeethFunction.name, @16252::Id.name, FastTimerService::ProcessDescription.name, python.rootplot.utilities.RootFile.name, hTMaxCell.name, cscdqm::ParHistoDef.name, BeautifulSoup.Tag.name, @16248::Id.name, TiXmlAttribute.name, BeautifulSoup.SoupStrainer.name, and dirstructure.Directory.subdirs.

313 
314  def __repr__(self):
315  if self.is_empty():
316  return "%s seems to be empty. Please check!" %self.name
317  content="%s , Rates: Success %.2f%%(%s) - Fail %.2f%%(%s) - Null %.2f%%(%s)\n" %(self.name,self.get_success_rate(),self.n_successes,self.get_fail_rate(),self.n_fails,self.get_null_rate(),self.n_nulls)
318  for subdir in self.subdirs:
319  content+=" %s\n" % subdir
320  for comp in self.comparisons:
321  content+=" %s\n" % comp
322  return content
323 
#-------------------------------------------------------------------------------
def dirstructure.Directory.calcStats (   self,
  make_pie = True 
) 
Walk all subdirs and calculate weight,fails and successes.
Moreove propagate the sample and releases names.

Definition at line 93 of file dirstructure.py.

References dirstructure.Directory.comparisons, dirstructure.Directory.different_histograms, dirstructure.Directory.filename1, dirstructure.Directory.filename2, alcazmumu_cfi.filter, dirstructure.Directory.full_path, join(), relativeConstraints.keys, dirstructure.Directory.meta, dirstructure.Directory.mother_dir, dirstructure.Directory.n_comp_fails, dirstructure.Directory.n_comp_nulls, dirstructure.Directory.n_comp_skiped, dirstructure.Directory.n_comp_successes, dirstructure.Directory.n_fails, dirstructure.Directory.n_missing_objs, dirstructure.Directory.n_nulls, dirstructure.Directory.n_skiped, dirstructure.Directory.n_successes, entry.name, genericValidation.GenericValidation.name, TmModule.name, EventMonitor::Entry.name, ora::RecordSpecImpl::Item.name, cond::persistency::GTEditorData.name, alignment.Alignment.name, FWTGeoRecoGeometry::Info.name, Types._Untracked.name, ParameterSet.name, fit::RootMinuitCommand.name, CombinedMVAJetTagComputer::Computer.name, PixelDCSObject< class >::Item.name, DQMRivetClient::LumiOption.name, MagCylinder.name, PFTauMVAInputDiscriminantTranslator::DiscriminantInfo.name, PrintSensitive.name, cond::persistency::GTProxyData.name, RHStopTracer::StopPoint.name, DQMRivetClient::ScaleFactorOption.name, SingleObjectCondition.name, EgHLTOfflineSummaryClient::SumHistBinData.name, DQMGenericClient::EfficOption.name, XMLHTRZeroSuppressionLoader::_loaderBaseConfig.name, XMLRBXPedestalsLoader::_loaderBaseConfig.name, PhysicsTools::Source.name, MyWatcher.name, edm::PathTimingSummary.name, dirstructure.Weighted.name, cond::TimeTypeSpecs.name, lumi::TriggerInfo.name, edm::PathSummary.name, PrintMaterialBudgetInfo.name, PixelEndcapLinkMaker::Item.name, perftools::EdmEventSize::BranchRecord.name, FWTableViewManager::TableEntry.name, PixelBarrelLinkMaker::Item.name, options.ConnectionHLTMenu.name, EcalLogicID.name, Mapper::definition< ScannerT >.name, McSelector.name, ExpressionHisto< T >.name, SensitiveDetector.name, RecoSelector.name, python.rootplot.utilities.Hist2D.name, DQMGenericClient::NormOption.name, XMLProcessor::_loaderBaseConfig.name, TreeCrawler.Package.name, TrajectorySeedProducer::LayerSpec.name, CaloTrkProcessing::Detector.name, PrintGeomInfoAction.name, HcalForwardLibWriter::FileHandle.name, MagGeoBuilderFromDDD::volumeHandle.name, PrintGeomMatInfo.name, DQMGenericClient::CDOption.name, PhysicsTools::Calibration::Variable.name, h4DSegm.name, cond::TagInfo_t.name, PhysicsTools::Variable::Value.name, options.HLTProcessOptions.name, EDMtoMEConverter.name, ProcTMVA::Method.name, TreeSaver::Var.name, python.rootplot.tree2hists.Plot.name, MEtoEDM< T >::MEtoEDMObject.name, PhysicsTools::TreeReader::Value.name, MuonGeometrySanityCheckPoint.name, PhysicsTools::TrainProcessor.name, TotemSD.name, PhysicsTools::MVAModuleHelper< Record, Object, Filler >::Value.name, PhysicsTools::ProcessRegistry< Base_t, CalibBase_t, Parent_t >.name, PhysicsTools::MVATrainer.name, h2DSegm.name, python.rootplot.utilities.Hist.name, BscSD.name, DQMNet::WaitObject.name, AlpgenParameterName.name, SiStripMonitorDigi.name, FP420SD.name, public_plots_tools.ColorScheme.name, PhysicsTools::Variable.name, PhysicsTools::TrainerMonitoring::Object.name, FastTimerService::LuminosityDescription.name, conddblib.Tag.name, conddblib.GlobalTag.name, plotscripts.SawTeethFunction.name, @16252::Id.name, FastTimerService::ProcessDescription.name, python.rootplot.utilities.RootFile.name, hTMaxCell.name, cscdqm::ParHistoDef.name, BeautifulSoup.Tag.name, @16248::Id.name, TiXmlAttribute.name, BeautifulSoup.SoupStrainer.name, python.rootplot.root2matplotlib.replace(), dirstructure.Directory.stats_calculated, dirstructure.Directory.subdirs, RPCDQMObject::DQMObjectItem.weight, JetAnaPythia< Jet >.weight, TtFullLepKinSolver::NeutrinoSolution.weight, ZIterativeAlgorithmWithFit::ZIterativeAlgorithmWithFitPlots.weight, SeedCandidate.weight, materialEffect::Effect.weight, dirstructure.Weighted.weight, ExtendedPerigeeTrajectoryError.weight, PrimaryVertexMonitor.weight, LHEProducer.weight, MuIsolatorResultProducer< BT >::DepositConf.weight, DTTrigTest.weight, L1PromptAnalysis.weight, TreeSaver.weight, ProcTMVA.weight, and areaInfo.weight.

Referenced by dirstructure.Directory.print_report().

93 
94  def calcStats(self,make_pie=True):
95  '''Walk all subdirs and calculate weight,fails and successes.
96  Moreove propagate the sample and releases names.
97  '''
98  if self.stats_calculated:
99  return 0
100 
101  self.n_fails=0
102  self.n_successes=0
103  self.n_nulls=0
104  self.n_comp_fails=0
105  self.n_comp_successes=0
106  self.n_comp_nulls=0
107  self.weight=0
108 
109  self.n_skiped = 0
110  self.n_comp_skiped = 0
111  self.n_missing_objs = len(self.different_histograms['file1'].keys())+len(self.different_histograms['file2'].keys())
112  if self.n_missing_objs != 0:
113  print " [*] Missing in %s: %s" %(self.filename1, self.different_histograms['file1'])
114  print " [*] Missing in %s: %s" %(self.filename2, self.different_histograms['file2'])
115  # clean from empty dirs
116  self.subdirs = filter(lambda subdir: not subdir.is_empty(),self.subdirs)
117 
118  for comp in self.comparisons:
119  if comp.status == SKIPED: #in case its in black list & skiped
120  self.n_skiped += 1
121  self.n_comp_skiped += 1
122  self.weight+=1
123  else: #else original code -> to check for Fails and Successes
124  self.rank_histo.Fill(comp.rank)
125  self.weight+=1
126  if comp.status == FAIL:
127  self.n_fails+=1
128  self.n_comp_fails+=1
129  elif comp.status == SUCCESS:
130  self.n_successes+=1
131  self.n_comp_successes+=1
132  else:
133  self.n_nulls+=1
134  self.n_comp_nulls+=1
135 
136  for subdir in self.subdirs:
137  subdir.mother_dir=join(self.mother_dir,self.name)
138  subdir.full_path = join(self.mother_dir,self.name).replace("/Run summary","")
139  subdir.calcStats(make_pie)
140  subdir.meta=self.meta
141  self.weight+=subdir.weight
142  self.n_fails+=subdir.n_fails
143  self.n_successes+=subdir.n_successes
144  self.n_nulls+=subdir.n_nulls
145 
146  self.n_skiped+=subdir.n_skiped
147  self.n_missing_objs += subdir.n_missing_objs
148 
149  self.rank_histo.Add(subdir.rank_histo)
150 
151  self.stats_calculated=True
152  self.full_path = join(self.mother_dir,self.name).replace("/Run summary","")
153  #if make_pie:
154  #self.__create_pie_image()
alcazmumu_cfi.filter
tuple filter
Definition: alcazmumu_cfi.py:34
join
static std::string join(char **cmd)
Definition: RemoteFile.cc:18
def dirstructure.Directory.get_fail_rate (   self)

Definition at line 205 of file dirstructure.py.

References dirstructure.Directory.n_fails, RPCDQMObject::DQMObjectItem.weight, JetAnaPythia< Jet >.weight, TtFullLepKinSolver::NeutrinoSolution.weight, ZIterativeAlgorithmWithFit::ZIterativeAlgorithmWithFitPlots.weight, SeedCandidate.weight, materialEffect::Effect.weight, dirstructure.Weighted.weight, ExtendedPerigeeTrajectoryError.weight, PrimaryVertexMonitor.weight, LHEProducer.weight, MuIsolatorResultProducer< BT >::DepositConf.weight, DTTrigTest.weight, TreeSaver.weight, L1PromptAnalysis.weight, ProcTMVA.weight, and areaInfo.weight.

Referenced by dirstructure.Directory.__create_pie_image(), dirstructure.Directory.__repr__(), dirstructure.Directory.get_summary_chart_ajax(), and dirstructure.Directory.print_report().

205 
206  def get_fail_rate(self):
207  if self.weight == 0:return 0
208  return 100.*self.n_fails/self.weight
def dirstructure.Directory.get_null_rate (   self)

Definition at line 213 of file dirstructure.py.

References dirstructure.Directory.n_nulls, RPCDQMObject::DQMObjectItem.weight, JetAnaPythia< Jet >.weight, TtFullLepKinSolver::NeutrinoSolution.weight, ZIterativeAlgorithmWithFit::ZIterativeAlgorithmWithFitPlots.weight, SeedCandidate.weight, materialEffect::Effect.weight, dirstructure.Weighted.weight, ExtendedPerigeeTrajectoryError.weight, PrimaryVertexMonitor.weight, LHEProducer.weight, MuIsolatorResultProducer< BT >::DepositConf.weight, DTTrigTest.weight, TreeSaver.weight, L1PromptAnalysis.weight, ProcTMVA.weight, and areaInfo.weight.

Referenced by dirstructure.Directory.__create_pie_image(), dirstructure.Directory.__repr__(), dirstructure.Directory.get_summary_chart_ajax(), and dirstructure.Directory.print_report().

213 
214  def get_null_rate(self):
215  if self.weight == 0:return 0
216  return 100.*self.n_nulls/self.weight
def dirstructure.Directory.get_skiped_rate (   self)

Definition at line 202 of file dirstructure.py.

References dirstructure.Directory.n_skiped, RPCDQMObject::DQMObjectItem.weight, JetAnaPythia< Jet >.weight, TtFullLepKinSolver::NeutrinoSolution.weight, ZIterativeAlgorithmWithFit::ZIterativeAlgorithmWithFitPlots.weight, SeedCandidate.weight, materialEffect::Effect.weight, dirstructure.Weighted.weight, ExtendedPerigeeTrajectoryError.weight, PrimaryVertexMonitor.weight, LHEProducer.weight, MuIsolatorResultProducer< BT >::DepositConf.weight, DTTrigTest.weight, TreeSaver.weight, L1PromptAnalysis.weight, ProcTMVA.weight, and areaInfo.weight.

Referenced by dirstructure.Directory.__create_pie_image(), dirstructure.Directory.get_summary_chart_ajax(), and dirstructure.Directory.print_report().

202 
203  def get_skiped_rate(self):
204  if self.weight == 0: return 0
return 100.*self.n_skiped/self.weight
def dirstructure.Directory.get_subdirs_dict (   self)

Definition at line 155 of file dirstructure.py.

References dirstructure.Directory.subdirs.

155 
156  def get_subdirs_dict(self):
157  subdirdict={}
158  for subdir in self.subdirs:
159  subdirdict[subdir.name]=subdir
160  return subdirdict
def dirstructure.Directory.get_subdirs_names (   self)

Definition at line 161 of file dirstructure.py.

References dirstructure.Directory.subdirs.

161 
162  def get_subdirs_names(self):
163  subdirnames=[]
164  for subdir in self.subdirs:
165  subdirnames.append(subdir.name)
166  return subdirnames
def dirstructure.Directory.get_success_rate (   self)

Definition at line 209 of file dirstructure.py.

References dirstructure.Directory.n_successes, RPCDQMObject::DQMObjectItem.weight, JetAnaPythia< Jet >.weight, TtFullLepKinSolver::NeutrinoSolution.weight, ZIterativeAlgorithmWithFit::ZIterativeAlgorithmWithFitPlots.weight, SeedCandidate.weight, materialEffect::Effect.weight, dirstructure.Weighted.weight, ExtendedPerigeeTrajectoryError.weight, PrimaryVertexMonitor.weight, LHEProducer.weight, MuIsolatorResultProducer< BT >::DepositConf.weight, DTTrigTest.weight, TreeSaver.weight, L1PromptAnalysis.weight, ProcTMVA.weight, and areaInfo.weight.

Referenced by dirstructure.Directory.__create_pie_image(), dirstructure.Directory.__repr__(), dirstructure.Directory.get_summary_chart_ajax(), and dirstructure.Directory.print_report().

209 
210  def get_success_rate(self):
211  if self.weight == 0:return 1
212  return 100.*self.n_successes/self.weight
def dirstructure.Directory.get_summary_chart_ajax (   self,
  w = 400,
  h = 300 
) 
Emit the ajax to build a pie chart using google apis...

Definition at line 167 of file dirstructure.py.

References dirstructure.Directory.get_fail_rate(), dirstructure.Directory.get_null_rate(), dirstructure.Directory.get_skiped_rate(), and dirstructure.Directory.get_success_rate().

167 
168  def get_summary_chart_ajax(self,w=400,h=300):
169  """Emit the ajax to build a pie chart using google apis...
170  """
171  url = "https://chart.googleapis.com/chart?"
172  url+= "cht=p3" # Select the 3d chart
173  #url+= "&chl=Success|Null|Fail" # give labels
174  url+= "&chco=00FF00|FFFF00|FF0000|7A7A7A" # give colours to labels
175  url+= "&chs=%sx%s" %(w,h)
176  #url+= "&chtt=%s" %self.name
177  url+= "&chd=t:%.2f,%.2f,%.2f,%.2f"%(self.get_success_rate(),self.get_null_rate(),self.get_fail_rate(),self.get_skiped_rate())
178 
179  return url
def dirstructure.Directory.get_summary_chart_name (   self)

Definition at line 230 of file dirstructure.py.

References dirstructure.Directory.__get_full_path(), and join().

Referenced by dirstructure.Directory.__create_pie_image().

230 
231  def get_summary_chart_name(self):
232  return join(self.__get_full_path(),"summary_chart.png")
join
static std::string join(char **cmd)
Definition: RemoteFile.cc:18
def dirstructure.Directory.is_empty (   self)

Definition at line 88 of file dirstructure.py.

References dirstructure.Directory.comparisons, and dirstructure.Directory.subdirs.

Referenced by dirstructure.Directory.__repr__(), and utils_v2.StatisticalTest.do_test().

88 
89  def is_empty(self):
90  if len(self.subdirs)==0 and len(self.comparisons)==0:
91  return True
92  return False
def dirstructure.Directory.print_report (   self,
  indent = "",
  verbose = False 
)

Definition at line 180 of file dirstructure.py.

References dirstructure.Directory.calcStats(), dirstructure.Directory.comparisons, alcazmumu_cfi.filter, dirstructure.Directory.get_fail_rate(), dirstructure.Directory.get_null_rate(), dirstructure.Directory.get_skiped_rate(), dirstructure.Directory.get_success_rate(), dirstructure.Directory.mother_dir, dirstructure.Directory.n_fails, dirstructure.Directory.n_missing_objs, dirstructure.Directory.n_nulls, dirstructure.Directory.n_skiped, dirstructure.Directory.n_successes, entry.name, genericValidation.GenericValidation.name, TmModule.name, EventMonitor::Entry.name, ora::RecordSpecImpl::Item.name, cond::persistency::GTEditorData.name, alignment.Alignment.name, FWTGeoRecoGeometry::Info.name, Types._Untracked.name, ParameterSet.name, fit::RootMinuitCommand.name, CombinedMVAJetTagComputer::Computer.name, PixelDCSObject< class >::Item.name, DQMRivetClient::LumiOption.name, MagCylinder.name, PFTauMVAInputDiscriminantTranslator::DiscriminantInfo.name, PrintSensitive.name, cond::persistency::GTProxyData.name, RHStopTracer::StopPoint.name, DQMRivetClient::ScaleFactorOption.name, SingleObjectCondition.name, EgHLTOfflineSummaryClient::SumHistBinData.name, XMLHTRZeroSuppressionLoader::_loaderBaseConfig.name, XMLRBXPedestalsLoader::_loaderBaseConfig.name, PhysicsTools::Source.name, DQMGenericClient::EfficOption.name, MyWatcher.name, edm::PathTimingSummary.name, dirstructure.Weighted.name, cond::TimeTypeSpecs.name, lumi::TriggerInfo.name, PrintMaterialBudgetInfo.name, edm::PathSummary.name, perftools::EdmEventSize::BranchRecord.name, PixelEndcapLinkMaker::Item.name, FWTableViewManager::TableEntry.name, PixelBarrelLinkMaker::Item.name, EcalLogicID.name, options.ConnectionHLTMenu.name, Mapper::definition< ScannerT >.name, ExpressionHisto< T >.name, McSelector.name, python.rootplot.utilities.Hist2D.name, SensitiveDetector.name, RecoSelector.name, DQMGenericClient::NormOption.name, XMLProcessor::_loaderBaseConfig.name, TreeCrawler.Package.name, CaloTrkProcessing::Detector.name, TrajectorySeedProducer::LayerSpec.name, HcalForwardLibWriter::FileHandle.name, PrintGeomInfoAction.name, MagGeoBuilderFromDDD::volumeHandle.name, DQMGenericClient::CDOption.name, PrintGeomMatInfo.name, PhysicsTools::Calibration::Variable.name, cond::TagInfo_t.name, h4DSegm.name, PhysicsTools::Variable::Value.name, options.HLTProcessOptions.name, EDMtoMEConverter.name, ProcTMVA::Method.name, TreeSaver::Var.name, python.rootplot.tree2hists.Plot.name, MEtoEDM< T >::MEtoEDMObject.name, PhysicsTools::TreeReader::Value.name, PhysicsTools::TrainProcessor.name, MuonGeometrySanityCheckPoint.name, TotemSD.name, PhysicsTools::MVAModuleHelper< Record, Object, Filler >::Value.name, PhysicsTools::ProcessRegistry< Base_t, CalibBase_t, Parent_t >.name, PhysicsTools::MVATrainer.name, h2DSegm.name, python.rootplot.utilities.Hist.name, BscSD.name, DQMNet::WaitObject.name, AlpgenParameterName.name, SiStripMonitorDigi.name, FP420SD.name, public_plots_tools.ColorScheme.name, PhysicsTools::Variable.name, PhysicsTools::TrainerMonitoring::Object.name, FastTimerService::LuminosityDescription.name, conddblib.Tag.name, conddblib.GlobalTag.name, plotscripts.SawTeethFunction.name, @16252::Id.name, FastTimerService::ProcessDescription.name, python.rootplot.utilities.RootFile.name, hTMaxCell.name, cscdqm::ParHistoDef.name, BeautifulSoup.Tag.name, @16248::Id.name, TiXmlAttribute.name, BeautifulSoup.SoupStrainer.name, dirstructure.Directory.subdirs, RPCDQMObject::DQMObjectItem.weight, JetAnaPythia< Jet >.weight, TtFullLepKinSolver::NeutrinoSolution.weight, ZIterativeAlgorithmWithFit::ZIterativeAlgorithmWithFitPlots.weight, SeedCandidate.weight, materialEffect::Effect.weight, dirstructure.Weighted.weight, ExtendedPerigeeTrajectoryError.weight, PrimaryVertexMonitor.weight, LHEProducer.weight, MuIsolatorResultProducer< BT >::DepositConf.weight, DTTrigTest.weight, TreeSaver.weight, L1PromptAnalysis.weight, ProcTMVA.weight, and areaInfo.weight.

180 
181  def print_report(self,indent="",verbose=False):
182  if len(indent)==0:
183  self.calcStats(make_pie=False)
184  # print small failure report
185  if verbose:
186  fail_comps=filter(lambda comp:comp.status==FAIL,self.comparisons)
187  fail_comps=sorted(fail_comps,key=lambda comp:comp.name )
188  if len(fail_comps)>0:
189  print indent+"* %s/%s:" %(self.mother_dir,self.name)
190  for comp in fail_comps:
191  print indent+" - %s: %s Test Failed (pval = %s) " %(comp.name,comp.test_name,comp.rank)
192  for subdir in self.subdirs:
193  subdir.print_report(indent+" ",verbose)
194 
195  if len(indent)==0:
196  print "\n%s - summary of %s tests:" %(self.name,self.weight)
197  print " o Failiures: %.2f%% (%s/%s)" %(self.get_fail_rate(),self.n_fails,self.weight)
198  print " o Nulls: %.2f%% (%s/%s) " %(self.get_null_rate(),self.n_nulls,self.weight)
199  print " o Successes: %.2f%% (%s/%s) " %(self.get_success_rate(),self.n_successes,self.weight)
200  print " o Skipped: %.2f%% (%s/%s) " %(self.get_skiped_rate(),self.n_skiped,self.weight)
201  print " o Missing objects: %s" %(self.n_missing_objs)
alcazmumu_cfi.filter
tuple filter
Definition: alcazmumu_cfi.py:34
def dirstructure.Directory.prune (   self,
  expandable_dir 
) 
Eliminate from the tree the directory the expandable ones.

Definition at line 269 of file dirstructure.py.

References dirstructure.Directory.comparisons, dirstructure.Directory.n_comp_fails, dirstructure.Directory.n_comp_nulls, dirstructure.Directory.n_comp_skiped, dirstructure.Directory.n_comp_successes, TtEventPartons.prune(), dirstructure.Directory.prune(), Config.Process.prune(), and dirstructure.Directory.subdirs.

Referenced by dirstructure.Directory.prune().

269 
270  def prune(self,expandable_dir):
271  """Eliminate from the tree the directory the expandable ones.
272  """
273  #print "pruning %s" %self.name
274  exp_index=-1
275  counter=0
276  for subdir in self.subdirs:
277  # Eliminate any trace of the expandable path in the mother directories
278  # for depths higher than 1
279  subdir.mother_dir=subdir.mother_dir.replace("/"+expandable_dir,"")
280  if subdir.name==expandable_dir:
281  exp_index=counter
282  counter+=1
283 
284  # Did we find an expandable?
285  if exp_index>=0:
286  exp_dir=self.subdirs[exp_index]
287  for subsubdir in exp_dir.subdirs:
288  #print "*******",subsubdir.mother_dir,
289  subsubdir.mother_dir=subsubdir.mother_dir.replace("/"+expandable_dir,"")
290  while "//" in subsubdir.mother_dir:
291  print subsubdir.mother_dir
292  subsubdir.mother_dir=subsubdir.mother_dir.replace("//","/")
293  #print "*******",subsubdir.mother_dir
294  self.subdirs.append(subsubdir)
295 
296  for comp in exp_dir.comparisons:
297  comp.mother_dir=comp.mother_dir.replace("/"+expandable_dir,"")
298  while "//" in comp.mother_dir:
299  comp.mother_dir
300  comp.mother_dir=comp.mother_dir.replace("/")
301  if not comp in self.comparisons: #in case not to append same comparisons few times
302  self.comparisons.append(comp) # add a comparison
303  self.n_comp_fails = exp_dir.n_comp_fails #copy to-be removed directory
304  self.n_comp_nulls = exp_dir.n_comp_nulls # numbers to parent directory
305  self.n_comp_successes = exp_dir.n_comp_successes
306  self.n_comp_skiped = exp_dir.n_comp_skiped
307 
308  del self.subdirs[exp_index]
309  self.prune(expandable_dir)
310 
311  for subdir in self.subdirs:
312  subdir.prune(expandable_dir)

Member Data Documentation

dirstructure.Directory.comparisons

Definition at line 64 of file dirstructure.py.

Referenced by dirstructure.Directory.__repr__(), dirstructure.Directory.calcStats(), dirstructure.Directory.is_empty(), dirstructure.Directory.print_report(), and dirstructure.Directory.prune().

dirstructure.Directory.different_histograms

Definition at line 80 of file dirstructure.py.

Referenced by dirstructure.Directory.calcStats(), and dqm_interfaces.DirWalkerFile.ls().

dirstructure.Directory.do_pngs

Definition at line 77 of file dirstructure.py.

Referenced by dqm_interfaces.DirWalkerDB.run().

dirstructure.Directory.draw_success

Definition at line 76 of file dirstructure.py.

dirstructure.Directory.filename1

Definition at line 83 of file dirstructure.py.

Referenced by dirstructure.Directory.calcStats().

dirstructure.Directory.filename2

Definition at line 84 of file dirstructure.py.

Referenced by dirstructure.Directory.calcStats().

dirstructure.Directory.full_path

Definition at line 86 of file dirstructure.py.

Referenced by dirstructure.Directory.calcStats().

dirstructure.Directory.meta

Definition at line 62 of file dirstructure.py.

Referenced by dirstructure.Directory.calcStats().

dirstructure.Directory.mother_dir

Definition at line 61 of file dirstructure.py.

Referenced by dirstructure.Directory.__create_on_disk(), dirstructure.Directory.__get_full_path(), dirstructure.Comparison.__get_img_name(), dirstructure.Comparison.__make_img_dir(), dirstructure.Directory.calcStats(), and dirstructure.Directory.print_report().

dirstructure.Directory.n_comp_fails

Definition at line 70 of file dirstructure.py.

Referenced by dirstructure.Directory.calcStats(), and dirstructure.Directory.prune().

dirstructure.Directory.n_comp_nulls

Definition at line 72 of file dirstructure.py.

Referenced by dirstructure.Directory.calcStats(), and dirstructure.Directory.prune().

dirstructure.Directory.n_comp_skiped

Definition at line 69 of file dirstructure.py.

Referenced by dirstructure.Directory.calcStats(), and dirstructure.Directory.prune().

dirstructure.Directory.n_comp_successes

Definition at line 71 of file dirstructure.py.

Referenced by dirstructure.Directory.calcStats(), and dirstructure.Directory.prune().

dirstructure.Directory.n_fails

Definition at line 65 of file dirstructure.py.

Referenced by dirstructure.Directory.__create_pie_image(), dirstructure.Directory.__repr__(), dirstructure.Directory.calcStats(), dirstructure.Directory.get_fail_rate(), and dirstructure.Directory.print_report().

dirstructure.Directory.n_missing_objs

Definition at line 85 of file dirstructure.py.

Referenced by dirstructure.Directory.calcStats(), and dirstructure.Directory.print_report().

dirstructure.Directory.n_nulls

Definition at line 67 of file dirstructure.py.

Referenced by dirstructure.Directory.__create_pie_image(), dirstructure.Directory.__repr__(), dirstructure.Directory.calcStats(), dirstructure.Directory.get_null_rate(), and dirstructure.Directory.print_report().

dirstructure.Directory.n_skiped

Definition at line 68 of file dirstructure.py.

Referenced by dirstructure.Directory.__create_pie_image(), dirstructure.Directory.calcStats(), dirstructure.Directory.get_skiped_rate(), and dirstructure.Directory.print_report().

dirstructure.Directory.n_successes

Definition at line 66 of file dirstructure.py.

Referenced by dirstructure.Directory.__create_pie_image(), dirstructure.Directory.__repr__(), dirstructure.Directory.calcStats(), dirstructure.Directory.get_success_rate(), and dirstructure.Directory.print_report().

dirstructure.Directory.rank_histo

Definition at line 78 of file dirstructure.py.

dirstructure.Directory.stats_calculated

Definition at line 74 of file dirstructure.py.

Referenced by dirstructure.Directory.calcStats().

dirstructure.Directory.subdirs

Definition at line 63 of file dirstructure.py.

Referenced by dirstructure.Directory.__repr__(), dirstructure.Directory.calcStats(), dirstructure.Directory.get_subdirs_dict(), dirstructure.Directory.get_subdirs_names(), dirstructure.Directory.is_empty(), dirstructure.Directory.print_report(), and dirstructure.Directory.prune().

dirstructure.Directory.weight

Definition at line 73 of file dirstructure.py.

Referenced by cuy.ValElement.__init__(), cuy.additionElement.__init__(), cuy.superimposeElement.__init__(), and cuy.graphElement.__init__().

Generated for CMSSW Reference Manual by doxygen 1.8.5