CMS 3D CMS Logo
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Properties Friends Macros Groups Pages
List of all members | Public Member Functions | Public Attributes | Private Member Functions
dirstructure.Directory Class Reference
Inheritance diagram for dirstructure.Directory:
dirstructure.Weighted

Public Member Functions

def __init__
 
def __repr__
 
def calcStats
 
def get_fail_rate
 
def get_null_rate
 
def get_skiped_rate
 
def get_subdirs_dict
 
def get_subdirs_names
 
def get_success_rate
 
def get_summary_chart_ajax
 
def get_summary_chart_name
 
def is_empty
 
def print_report
 
def prune
 
- Public Member Functions inherited from dirstructure.Weighted
def __init__
 

Public Attributes

 comparisons
 
 different_histograms
 
 do_pngs
 
 draw_success
 
 filename1
 
 filename2
 
 full_path
 
 meta
 
 mother_dir
 
 n_comp_fails
 
 n_comp_nulls
 
 n_comp_skiped
 
 n_comp_successes
 
 n_fails
 
 n_missing_objs
 
 n_nulls
 
 n_skiped
 
 n_successes
 
 rank_histo
 
 stats_calculated
 
 subdirs
 
 weight
 
- Public Attributes inherited from dirstructure.Weighted
 name
 
 weight
 

Private Member Functions

def __create_on_disk
 
def __create_pie_image
 
def __get_full_path
 

Detailed Description

Definition at line 62 of file dirstructure.py.

Constructor & Destructor Documentation

def dirstructure.Directory.__init__ (   self,
  name,
  mother_dir = "",
  meta = CompInfo(),
  draw_success = False,
  do_pngs = False 
)

Definition at line 63 of file dirstructure.py.

63 
64  def __init__(self,name,mother_dir="",meta=CompInfo(),draw_success=False,do_pngs=False):
65  self.mother_dir=mother_dir
66  self.meta=meta
67  self.subdirs=[]
68  self.comparisons=[]
69  self.n_fails=0
70  self.n_successes=0
71  self.n_nulls=0
72  self.n_skiped = 0
73  self.n_comp_skiped = 0
74  self.n_comp_fails=0
75  self.n_comp_successes=0
76  self.n_comp_nulls=0
77  self.weight=0
78  self.stats_calculated=False
79  Weighted.__init__(self,name)
80  self.draw_success=draw_success
81  self.do_pngs=do_pngs
82  self.rank_histo=ROOT.TH1I("rh%s"%name,"",50,-0.01,1.001)
83  self.rank_histo.SetDirectory(0)
84  self.different_histograms = {}
85  self.different_histograms['file1']= {}
86  self.different_histograms['file2']= {}
87  self.filename1 = ""
88  self.filename2 = ""
89  self.n_missing_objs = 0
90  self.full_path = ""

Member Function Documentation

def dirstructure.Directory.__create_on_disk (   self)
private

Definition at line 226 of file dirstructure.py.

References dirstructure.Directory.__get_full_path(), and dirstructure.Directory.mother_dir.

Referenced by dirstructure.Directory.__create_pie_image().

226 
227  def __create_on_disk(self):
228  if not exists(self.mother_dir) and len(self.mother_dir)!=0:
229  makedirs(self.mother_dir)
230  full_path=self.__get_full_path()
231  if not exists(full_path) and len(full_path)>0:
232  makedirs(full_path)
def dirstructure.Directory.__create_pie_image (   self)
private

Definition at line 236 of file dirstructure.py.

References dirstructure.Directory.__create_on_disk(), mps_check.array, dirstructure.Directory.get_fail_rate(), dirstructure.Directory.get_null_rate(), dirstructure.Directory.get_skiped_rate(), dirstructure.Directory.get_success_rate(), dirstructure.Directory.get_summary_chart_name(), dirstructure.Directory.n_fails, dirstructure.Directory.n_nulls, dirstructure.Directory.n_skiped, dirstructure.Directory.n_successes, ElectronMVAID.ElectronMVAID.name, AlignableObjectId::entry.name, average.Average.name, counter.Counter.name, histograms.Histograms.name, cond::persistency::GLOBAL_TAG::NAME.name, cond::persistency::RUN_INFO::RUN_NUMBER.name, cond::persistency::TAG::NAME.name, FWLite.ElectronMVAID.name, TmModule.name, cond::persistency::GTEditorData.name, cond::persistency::GLOBAL_TAG::VALIDITY.name, cond::persistency::RUN_INFO::START_TIME.name, cond::persistency::TAG::TIME_TYPE.name, cond::persistency::GLOBAL_TAG::DESCRIPTION.name, core.autovars.NTupleVariable.name, cond::persistency::RUN_INFO::END_TIME.name, cond::persistency::TAG::OBJECT_TYPE.name, cond::persistency::GLOBAL_TAG::RELEASE.name, cond::persistency::TAG::SYNCHRONIZATION.name, DQMRivetClient::NormOption.name, cond::persistency::GLOBAL_TAG::SNAPSHOT_TIME.name, cond::persistency::TAG::END_OF_VALIDITY.name, MEPSet.name, cond::persistency::O2O_RUN::JOB_NAME.name, cond::persistency::GLOBAL_TAG::INSERTION_TIME.name, cms::dd::NameValuePair< T >.name, cond::persistency::TAG::DESCRIPTION.name, cond::persistency::O2O_RUN::START_TIME.name, cond::persistency::TAG::LAST_VALIDATED_TIME.name, cond::persistency::O2O_RUN::END_TIME.name, cond::persistency::TAG::INSERTION_TIME.name, FWTGeoRecoGeometry::Info.name, cond::persistency::O2O_RUN::STATUS_CODE.name, cond::persistency::TAG::MODIFICATION_TIME.name, ParameterSet.name, cond::persistency::O2O_RUN::LOG.name, nanoaod::MergeableCounterTable::SingleColumn< T >.name, cond::persistency::TAG::PROTECTION_CODE.name, preexistingValidation.PreexistingValidation.name, OutputMEPSet.name, PixelDCSObject< class >::Item.name, dataset.BaseDataset.name, SummaryTableOutputBranches::NamedBranchPtr.name, AlignmentConstraint.name, fit::RootMinuitCommand.name, cms::dd::ValuePair< T, U >.name, personalPlayback.Applet.name, EventStringOutputBranches::NamedBranchPtr.name, Types._Untracked.name, MagCylinder.name, analyzer.Analyzer.name, production_tasks.Task.name, HcalForwardLibWriter::FileHandle.name, heppy::ParSet.name, DQMRivetClient::LumiOption.name, o2olib.O2OJob.name, cond::persistency::GTProxyData.name, CharmTagger::MVAVar.name, TableOutputBranches::NamedBranchPtr.name, TriggerOutputBranches::NamedBranchPtr.name, HeavyIonCSVTagger::MVAVar.name, SingleObjectCondition.name, EgHLTOfflineSummaryClient::SumHistBinData.name, edm::PathTimingSummary.name, DQMRivetClient::ScaleFactorOption.name, cms::DDAlgoArguments.name, RHStopTracer::StopPoint.name, Barrel.name, perftools::EdmEventSize::BranchRecord.name, core.autovars.NTupleObjectType.name, cond::TimeTypeSpecs.name, edm::PathSummary.name, EcalLogicID.name, alignment.Alignment.name, lumi::TriggerInfo.name, XMLProcessor::_loaderBaseConfig.name, dirstructure.Weighted.name, PixelEndcapLinkMaker::Item.name, MEtoEDM< T >::MEtoEDMObject.name, FWTableViewManager::TableEntry.name, PixelBarrelLinkMaker::Item.name, DQMGenericClient::EfficOption.name, ExpressionHisto< T >.name, python.rootplot.utilities.Hist2D.name, Supermodule.name, TreeCrawler.Package.name, genericValidation.GenericValidation.name, cond::persistency::GLOBAL_TAG_MAP::GLOBAL_TAG_NAME.name, CaloTrkProcessing::Detector.name, options.ConnectionHLTMenu.name, cond::persistency::GLOBAL_TAG_MAP::RECORD.name, MVAVar.name, cond::persistency::GLOBAL_TAG_MAP::LABEL.name, DisplayManager.DisplayManager.name, cms::DDParsingContext::CompositeMaterial.name, cond::persistency::GLOBAL_TAG_MAP::TAG_NAME.name, FWLite.WorkingPoints.name, cond::Tag_t.name, dqmoffline::l1t::HistDefinition.name, DQMGenericClient::ProfileOption.name, magneticfield::BaseVolumeHandle.name, nanoaod::MergeableCounterTable::VectorColumn< T >.name, FastHFShowerLibrary.name, PhysicsTools::Variable::Value.name, emtf::Node.name, PrintMaterialBudgetInfo.name, h4DSegm.name, BTagProbabilityToDiscriminator::Discriminator.name, DQMGenericClient::NormOption.name, DQMGenericClient::CDOption.name, CounterChecker.name, TriggerObjectTableProducer::SelectedObject.name, PhysicsTools::Calibration::Variable.name, cond::TagInfo_t.name, TrackerSectorStruct.name, MuonGeometrySanityCheckPoint.name, DQMGenericClient::NoFlowOption.name, cond::persistency::PAYLOAD::HASH.name, looper.Looper.name, Mapper::definition< ScannerT >.name, cond::persistency::PAYLOAD::OBJECT_TYPE.name, cond::persistency::PAYLOAD::DATA.name, EDMtoMEConverter.name, python.rootplot.tree2hists.Plot.name, cond::persistency::PAYLOAD::STREAMER_INFO.name, cond::persistency::PAYLOAD::VERSION.name, cond::persistency::PAYLOAD::INSERTION_TIME.name, PhysicsTools::TreeReader::Value.name, classes.MonitorData.name, PhysicsTools::ProcessRegistry< Base_t, CalibBase_t, Parent_t >.name, HistogramManager.name, classes.OutputData.name, Crystal.name, h2DSegm.name, options.HLTProcessOptions.name, cond::persistency::IOV::TAG_NAME.name, cond::persistency::IOV::SINCE.name, cond::persistency::IOV::PAYLOAD_HASH.name, cond::persistency::IOV::INSERTION_TIME.name, DQMNet::WaitObject.name, python.rootplot.utilities.Hist.name, AlpgenParameterName.name, config.Analyzer.name, geometry.Structure.name, hDigis.name, core.autovars.NTupleSubObject.name, Capsule.name, core.autovars.NTupleObject.name, Ceramic.name, PhysicsTools::Variable.name, SiStripMonitorDigi.name, BulkSilicon.name, config.Service.name, CaloSD::Detector.name, APD.name, nanoaod::FlatTable::Column.name, core.autovars.NTupleCollection.name, NanoAODDQM::SelGroupConfig.name, BPHRecoBuilder::BPHRecoSource.name, BPHRecoBuilder::BPHCompSource.name, StraightTrackAlignment::RPSetPlots.name, cond::persistency::TAG_AUTHORIZATION::TAG_NAME.name, cond::persistency::TAG_AUTHORIZATION::ACCESS_TYPE.name, cond::persistency::TAG_AUTHORIZATION::CREDENTIAL.name, cond::persistency::TAG_AUTHORIZATION::CREDENTIAL_TYPE.name, InnerLayerVolume.name, cond::payloadInspector::TagReference.name, cond::persistency::TAG_LOG::TAG_NAME.name, cond::persistency::TAG_LOG::EVENT_TIME.name, cond::persistency::TAG_LOG::USER_NAME.name, cond::persistency::TAG_LOG::HOST_NAME.name, cond::persistency::TAG_LOG::COMMAND.name, cond::persistency::TAG_LOG::ACTION.name, cond::persistency::TAG_LOG::USER_TEXT.name, personalPlayback.FrameworkJob.name, Grid.name, Grille.name, BackPipe.name, plotscripts.SawTeethFunction.name, PatchPanel.name, BackCoolTank.name, DryAirTube.name, python.rootplot.utilities.RootFile.name, crabFunctions.CrabTask.name, MBCoolTube.name, hHits.name, MBManif.name, cscdqm::ParHistoDef.name, hDeltaR.name, hTMaxCell.name, BeautifulSoup.Tag.name, hParam.name, hMuonStat.name, hTOF.name, hTDelay.name, hTimes< hTime >.name, SummaryOutputProducer::GenericSummary.name, BeautifulSoup.SoupStrainer.name, print(), and ComparisonHelper.zip().

236 
237  def __create_pie_image(self):
238  self.__create_on_disk()
239  vals=[]
240  colors=[]
241  for n,col in zip((self.n_fails,self.n_nulls,self.n_successes,self.n_skiped),(ROOT.kRed,ROOT.kYellow,ROOT.kGreen,ROOT.kBlue)):
242  if n!=0:
243  vals.append(n)
244  colors.append(col)
245  valsa=array('f',vals)
246  colorsa=array('i',colors)
247  can = ROOT.TCanvas("cpie","TPie test",100,100);
248  try:
249  pie = TPie("ThePie",self.name,len(vals),valsa,colorsa);
250  label_n=0
251  if self.n_fails!=0:
252  pie.SetEntryLabel(label_n, "Fail: %.1f(%i)" %(self.get_fail_rate(),self.n_fails) );
253  label_n+=1
254  if self.n_nulls!=0:
255  pie.SetEntryLabel(label_n, "Null: %.1f(%i)" %(self.get_null_rate(),self.n_nulls) );
256  label_n+=1
257  if self.n_successes!=0:
258  pie.SetEntryLabel(label_n, "Success: %.1f(%i)" %(self.get_success_rate(),self.n_successes) );
259  if self.n_skiped!=0:
260  pie.SetEntryLabel(label_n, "Skipped: %.1f(%i)" %(self.get_skiped_rate(),self.n_skiped));
261  pie.SetY(.52);
262  pie.SetAngularOffset(0.);
263  pie.SetLabelsOffset(-.3);
264  #pie.SetLabelFormat("#splitline{%val (%perc)}{%txt}");
265  pie.Draw("3d nol");
266  can.Print(self.get_summary_chart_name());
267  except:
268  print("self.name = %s" %self.name)
269  print("len(vals) = %s (vals=%s)" %(len(vals),vals))
270  print("valsa = %s" %valsa)
271  print("colorsa = %s" %colorsa)
mps_check.array
tuple array
Definition: mps_check.py:216
ComparisonHelper::zip
OutputIterator zip(InputIterator1 first1, InputIterator1 last1, InputIterator2 first2, InputIterator2 last2, OutputIterator result, Compare comp)
Definition: L1TStage2CaloLayer1.h:41
print
void print(TMatrixD &m, const char *label=nullptr, bool mathematicaFormat=false)
Definition: Utilities.cc:47
def dirstructure.Directory.__get_full_path (   self)
private

Definition at line 220 of file dirstructure.py.

References join(), dirstructure.Directory.mother_dir, ElectronMVAID.ElectronMVAID.name, AlignableObjectId::entry.name, average.Average.name, counter.Counter.name, histograms.Histograms.name, TmModule.name, cond::persistency::TAG::NAME.name, cond::persistency::RUN_INFO::RUN_NUMBER.name, FWLite.ElectronMVAID.name, cond::persistency::GTEditorData.name, cond::persistency::GLOBAL_TAG::NAME.name, cond::persistency::TAG::TIME_TYPE.name, cond::persistency::RUN_INFO::START_TIME.name, cond::persistency::GLOBAL_TAG::VALIDITY.name, cond::persistency::RUN_INFO::END_TIME.name, core.autovars.NTupleVariable.name, cond::persistency::TAG::OBJECT_TYPE.name, cond::persistency::GLOBAL_TAG::DESCRIPTION.name, cond::persistency::TAG::SYNCHRONIZATION.name, DQMRivetClient::NormOption.name, cond::persistency::GLOBAL_TAG::RELEASE.name, cond::persistency::TAG::END_OF_VALIDITY.name, MEPSet.name, cond::persistency::GLOBAL_TAG::SNAPSHOT_TIME.name, cond::persistency::O2O_RUN::JOB_NAME.name, cond::persistency::GLOBAL_TAG::INSERTION_TIME.name, cond::persistency::TAG::DESCRIPTION.name, cms::dd::NameValuePair< T >.name, cond::persistency::O2O_RUN::START_TIME.name, cond::persistency::TAG::LAST_VALIDATED_TIME.name, cond::persistency::O2O_RUN::END_TIME.name, cond::persistency::TAG::INSERTION_TIME.name, FWTGeoRecoGeometry::Info.name, cond::persistency::O2O_RUN::STATUS_CODE.name, cond::persistency::TAG::MODIFICATION_TIME.name, cond::persistency::O2O_RUN::LOG.name, ParameterSet.name, nanoaod::MergeableCounterTable::SingleColumn< T >.name, cond::persistency::TAG::PROTECTION_CODE.name, preexistingValidation.PreexistingValidation.name, OutputMEPSet.name, AlignmentConstraint.name, PixelDCSObject< class >::Item.name, dataset.BaseDataset.name, SummaryTableOutputBranches::NamedBranchPtr.name, fit::RootMinuitCommand.name, cms::dd::ValuePair< T, U >.name, personalPlayback.Applet.name, EventStringOutputBranches::NamedBranchPtr.name, Types._Untracked.name, MagCylinder.name, analyzer.Analyzer.name, production_tasks.Task.name, HcalForwardLibWriter::FileHandle.name, heppy::ParSet.name, DQMRivetClient::LumiOption.name, o2olib.O2OJob.name, cond::persistency::GTProxyData.name, TableOutputBranches::NamedBranchPtr.name, CharmTagger::MVAVar.name, TriggerOutputBranches::NamedBranchPtr.name, HeavyIonCSVTagger::MVAVar.name, SingleObjectCondition.name, EgHLTOfflineSummaryClient::SumHistBinData.name, edm::PathTimingSummary.name, DQMRivetClient::ScaleFactorOption.name, cms::DDAlgoArguments.name, RHStopTracer::StopPoint.name, Barrel.name, perftools::EdmEventSize::BranchRecord.name, core.autovars.NTupleObjectType.name, cond::TimeTypeSpecs.name, edm::PathSummary.name, EcalLogicID.name, alignment.Alignment.name, lumi::TriggerInfo.name, PixelEndcapLinkMaker::Item.name, XMLProcessor::_loaderBaseConfig.name, dirstructure.Weighted.name, MEtoEDM< T >::MEtoEDMObject.name, FWTableViewManager::TableEntry.name, PixelBarrelLinkMaker::Item.name, ExpressionHisto< T >.name, DQMGenericClient::EfficOption.name, python.rootplot.utilities.Hist2D.name, TreeCrawler.Package.name, Supermodule.name, cond::persistency::GLOBAL_TAG_MAP::GLOBAL_TAG_NAME.name, genericValidation.GenericValidation.name, CaloTrkProcessing::Detector.name, options.ConnectionHLTMenu.name, cond::persistency::GLOBAL_TAG_MAP::RECORD.name, cond::persistency::GLOBAL_TAG_MAP::LABEL.name, MVAVar.name, cond::persistency::GLOBAL_TAG_MAP::TAG_NAME.name, cms::DDParsingContext::CompositeMaterial.name, DisplayManager.DisplayManager.name, FWLite.WorkingPoints.name, cond::Tag_t.name, dqmoffline::l1t::HistDefinition.name, DQMGenericClient::ProfileOption.name, magneticfield::BaseVolumeHandle.name, nanoaod::MergeableCounterTable::VectorColumn< T >.name, FastHFShowerLibrary.name, PhysicsTools::Variable::Value.name, emtf::Node.name, h4DSegm.name, PrintMaterialBudgetInfo.name, DQMGenericClient::NormOption.name, BTagProbabilityToDiscriminator::Discriminator.name, DQMGenericClient::CDOption.name, CounterChecker.name, TriggerObjectTableProducer::SelectedObject.name, PhysicsTools::Calibration::Variable.name, cond::TagInfo_t.name, TrackerSectorStruct.name, MuonGeometrySanityCheckPoint.name, cond::persistency::PAYLOAD::HASH.name, DQMGenericClient::NoFlowOption.name, looper.Looper.name, cond::persistency::PAYLOAD::OBJECT_TYPE.name, Mapper::definition< ScannerT >.name, cond::persistency::PAYLOAD::DATA.name, EDMtoMEConverter.name, cond::persistency::PAYLOAD::STREAMER_INFO.name, python.rootplot.tree2hists.Plot.name, cond::persistency::PAYLOAD::VERSION.name, cond::persistency::PAYLOAD::INSERTION_TIME.name, PhysicsTools::TreeReader::Value.name, classes.MonitorData.name, PhysicsTools::ProcessRegistry< Base_t, CalibBase_t, Parent_t >.name, HistogramManager.name, classes.OutputData.name, Crystal.name, h2DSegm.name, options.HLTProcessOptions.name, cond::persistency::IOV::TAG_NAME.name, cond::persistency::IOV::SINCE.name, cond::persistency::IOV::PAYLOAD_HASH.name, cond::persistency::IOV::INSERTION_TIME.name, DQMNet::WaitObject.name, python.rootplot.utilities.Hist.name, AlpgenParameterName.name, config.Analyzer.name, geometry.Structure.name, hDigis.name, core.autovars.NTupleSubObject.name, Capsule.name, core.autovars.NTupleObject.name, Ceramic.name, PhysicsTools::Variable.name, SiStripMonitorDigi.name, config.Service.name, BulkSilicon.name, CaloSD::Detector.name, APD.name, nanoaod::FlatTable::Column.name, core.autovars.NTupleCollection.name, NanoAODDQM::SelGroupConfig.name, BPHRecoBuilder::BPHRecoSource.name, BPHRecoBuilder::BPHCompSource.name, StraightTrackAlignment::RPSetPlots.name, cond::persistency::TAG_AUTHORIZATION::TAG_NAME.name, cond::persistency::TAG_AUTHORIZATION::ACCESS_TYPE.name, cond::persistency::TAG_AUTHORIZATION::CREDENTIAL.name, cond::persistency::TAG_AUTHORIZATION::CREDENTIAL_TYPE.name, InnerLayerVolume.name, cond::payloadInspector::TagReference.name, cond::persistency::TAG_LOG::TAG_NAME.name, cond::persistency::TAG_LOG::EVENT_TIME.name, cond::persistency::TAG_LOG::USER_NAME.name, cond::persistency::TAG_LOG::HOST_NAME.name, cond::persistency::TAG_LOG::COMMAND.name, cond::persistency::TAG_LOG::ACTION.name, cond::persistency::TAG_LOG::USER_TEXT.name, personalPlayback.FrameworkJob.name, Grid.name, Grille.name, BackPipe.name, plotscripts.SawTeethFunction.name, PatchPanel.name, BackCoolTank.name, DryAirTube.name, python.rootplot.utilities.RootFile.name, crabFunctions.CrabTask.name, MBCoolTube.name, hHits.name, MBManif.name, cscdqm::ParHistoDef.name, hDeltaR.name, hTMaxCell.name, BeautifulSoup.Tag.name, hParam.name, hMuonStat.name, hTOF.name, hTDelay.name, hTimes< hTime >.name, SummaryOutputProducer::GenericSummary.name, and BeautifulSoup.SoupStrainer.name.

Referenced by dirstructure.Directory.__create_on_disk(), and dirstructure.Directory.get_summary_chart_name().

220 
221  def __get_full_path(self):
222  #print "Mother is %s" %self.mother_dir
223  if len(self.mother_dir)==0:
224  return self.name
225  return join(self.mother_dir,self.name)
join
static std::string join(char **cmd)
Definition: RemoteFile.cc:19
def dirstructure.Directory.__repr__ (   self)

Definition at line 316 of file dirstructure.py.

References dirstructure.Directory.comparisons, dirstructure.Directory.get_fail_rate(), dirstructure.Directory.get_null_rate(), dirstructure.Directory.get_success_rate(), DEutils< T >.is_empty(), DEcompare< T >.is_empty(), dirstructure.Directory.is_empty(), dirstructure.Directory.n_fails, dirstructure.Directory.n_nulls, dirstructure.Directory.n_successes, ElectronMVAID.ElectronMVAID.name, AlignableObjectId::entry.name, average.Average.name, counter.Counter.name, histograms.Histograms.name, cond::persistency::TAG::NAME.name, cond::persistency::RUN_INFO::RUN_NUMBER.name, TmModule.name, FWLite.ElectronMVAID.name, cond::persistency::GTEditorData.name, cond::persistency::GLOBAL_TAG::NAME.name, cond::persistency::RUN_INFO::START_TIME.name, cond::persistency::TAG::TIME_TYPE.name, cond::persistency::GLOBAL_TAG::VALIDITY.name, cond::persistency::RUN_INFO::END_TIME.name, core.autovars.NTupleVariable.name, cond::persistency::TAG::OBJECT_TYPE.name, cond::persistency::GLOBAL_TAG::DESCRIPTION.name, cond::persistency::GLOBAL_TAG::RELEASE.name, cond::persistency::TAG::SYNCHRONIZATION.name, DQMRivetClient::NormOption.name, cond::persistency::GLOBAL_TAG::SNAPSHOT_TIME.name, cond::persistency::TAG::END_OF_VALIDITY.name, MEPSet.name, cond::persistency::GLOBAL_TAG::INSERTION_TIME.name, cond::persistency::O2O_RUN::JOB_NAME.name, cond::persistency::TAG::DESCRIPTION.name, cms::dd::NameValuePair< T >.name, cond::persistency::O2O_RUN::START_TIME.name, cond::persistency::TAG::LAST_VALIDATED_TIME.name, cond::persistency::O2O_RUN::END_TIME.name, cond::persistency::TAG::INSERTION_TIME.name, FWTGeoRecoGeometry::Info.name, cond::persistency::O2O_RUN::STATUS_CODE.name, cond::persistency::TAG::MODIFICATION_TIME.name, ParameterSet.name, cond::persistency::O2O_RUN::LOG.name, nanoaod::MergeableCounterTable::SingleColumn< T >.name, cond::persistency::TAG::PROTECTION_CODE.name, OutputMEPSet.name, preexistingValidation.PreexistingValidation.name, AlignmentConstraint.name, PixelDCSObject< class >::Item.name, dataset.BaseDataset.name, SummaryTableOutputBranches::NamedBranchPtr.name, fit::RootMinuitCommand.name, cms::dd::ValuePair< T, U >.name, personalPlayback.Applet.name, EventStringOutputBranches::NamedBranchPtr.name, Types._Untracked.name, MagCylinder.name, analyzer.Analyzer.name, production_tasks.Task.name, HcalForwardLibWriter::FileHandle.name, heppy::ParSet.name, DQMRivetClient::LumiOption.name, o2olib.O2OJob.name, cond::persistency::GTProxyData.name, TableOutputBranches::NamedBranchPtr.name, CharmTagger::MVAVar.name, HeavyIonCSVTagger::MVAVar.name, TriggerOutputBranches::NamedBranchPtr.name, SingleObjectCondition.name, EgHLTOfflineSummaryClient::SumHistBinData.name, edm::PathTimingSummary.name, DQMRivetClient::ScaleFactorOption.name, cms::DDAlgoArguments.name, RHStopTracer::StopPoint.name, Barrel.name, perftools::EdmEventSize::BranchRecord.name, core.autovars.NTupleObjectType.name, cond::TimeTypeSpecs.name, edm::PathSummary.name, EcalLogicID.name, alignment.Alignment.name, lumi::TriggerInfo.name, PixelEndcapLinkMaker::Item.name, dirstructure.Weighted.name, XMLProcessor::_loaderBaseConfig.name, MEtoEDM< T >::MEtoEDMObject.name, FWTableViewManager::TableEntry.name, PixelBarrelLinkMaker::Item.name, ExpressionHisto< T >.name, DQMGenericClient::EfficOption.name, python.rootplot.utilities.Hist2D.name, TreeCrawler.Package.name, Supermodule.name, cond::persistency::GLOBAL_TAG_MAP::GLOBAL_TAG_NAME.name, genericValidation.GenericValidation.name, CaloTrkProcessing::Detector.name, options.ConnectionHLTMenu.name, cond::persistency::GLOBAL_TAG_MAP::RECORD.name, MVAVar.name, cond::persistency::GLOBAL_TAG_MAP::LABEL.name, cond::persistency::GLOBAL_TAG_MAP::TAG_NAME.name, DisplayManager.DisplayManager.name, cms::DDParsingContext::CompositeMaterial.name, FWLite.WorkingPoints.name, cond::Tag_t.name, dqmoffline::l1t::HistDefinition.name, DQMGenericClient::ProfileOption.name, magneticfield::BaseVolumeHandle.name, nanoaod::MergeableCounterTable::VectorColumn< T >.name, FastHFShowerLibrary.name, emtf::Node.name, PhysicsTools::Variable::Value.name, PrintMaterialBudgetInfo.name, h4DSegm.name, BTagProbabilityToDiscriminator::Discriminator.name, DQMGenericClient::NormOption.name, DQMGenericClient::CDOption.name, CounterChecker.name, PhysicsTools::Calibration::Variable.name, TriggerObjectTableProducer::SelectedObject.name, cond::TagInfo_t.name, TrackerSectorStruct.name, MuonGeometrySanityCheckPoint.name, DQMGenericClient::NoFlowOption.name, cond::persistency::PAYLOAD::HASH.name, cond::persistency::PAYLOAD::OBJECT_TYPE.name, looper.Looper.name, Mapper::definition< ScannerT >.name, cond::persistency::PAYLOAD::DATA.name, EDMtoMEConverter.name, python.rootplot.tree2hists.Plot.name, cond::persistency::PAYLOAD::STREAMER_INFO.name, cond::persistency::PAYLOAD::VERSION.name, cond::persistency::PAYLOAD::INSERTION_TIME.name, PhysicsTools::TreeReader::Value.name, classes.MonitorData.name, PhysicsTools::ProcessRegistry< Base_t, CalibBase_t, Parent_t >.name, HistogramManager.name, classes.OutputData.name, Crystal.name, h2DSegm.name, options.HLTProcessOptions.name, cond::persistency::IOV::TAG_NAME.name, cond::persistency::IOV::SINCE.name, cond::persistency::IOV::PAYLOAD_HASH.name, cond::persistency::IOV::INSERTION_TIME.name, DQMNet::WaitObject.name, python.rootplot.utilities.Hist.name, AlpgenParameterName.name, config.Analyzer.name, geometry.Structure.name, hDigis.name, core.autovars.NTupleSubObject.name, Capsule.name, core.autovars.NTupleObject.name, Ceramic.name, PhysicsTools::Variable.name, SiStripMonitorDigi.name, BulkSilicon.name, config.Service.name, CaloSD::Detector.name, APD.name, nanoaod::FlatTable::Column.name, core.autovars.NTupleCollection.name, NanoAODDQM::SelGroupConfig.name, BPHRecoBuilder::BPHRecoSource.name, BPHRecoBuilder::BPHCompSource.name, StraightTrackAlignment::RPSetPlots.name, cond::persistency::TAG_AUTHORIZATION::TAG_NAME.name, cond::persistency::TAG_AUTHORIZATION::ACCESS_TYPE.name, cond::persistency::TAG_AUTHORIZATION::CREDENTIAL.name, cond::persistency::TAG_AUTHORIZATION::CREDENTIAL_TYPE.name, InnerLayerVolume.name, cond::payloadInspector::TagReference.name, cond::persistency::TAG_LOG::TAG_NAME.name, cond::persistency::TAG_LOG::EVENT_TIME.name, cond::persistency::TAG_LOG::USER_NAME.name, cond::persistency::TAG_LOG::HOST_NAME.name, cond::persistency::TAG_LOG::COMMAND.name, cond::persistency::TAG_LOG::ACTION.name, cond::persistency::TAG_LOG::USER_TEXT.name, personalPlayback.FrameworkJob.name, Grid.name, Grille.name, BackPipe.name, plotscripts.SawTeethFunction.name, PatchPanel.name, BackCoolTank.name, DryAirTube.name, python.rootplot.utilities.RootFile.name, crabFunctions.CrabTask.name, MBCoolTube.name, hHits.name, MBManif.name, cscdqm::ParHistoDef.name, hDeltaR.name, hTMaxCell.name, BeautifulSoup.Tag.name, hParam.name, hMuonStat.name, hTOF.name, hTDelay.name, hTimes< hTime >.name, SummaryOutputProducer::GenericSummary.name, BeautifulSoup.SoupStrainer.name, and dirstructure.Directory.subdirs.

316 
317  def __repr__(self):
318  if self.is_empty():
319  return "%s seems to be empty. Please check!" %self.name
320  content="%s , Rates: Success %.2f%%(%s) - Fail %.2f%%(%s) - Null %.2f%%(%s)\n" %(self.name,self.get_success_rate(),self.n_successes,self.get_fail_rate(),self.n_fails,self.get_null_rate(),self.n_nulls)
321  for subdir in self.subdirs:
322  content+=" %s\n" % subdir
323  for comp in self.comparisons:
324  content+=" %s\n" % comp
325  return content
326 
#-------------------------------------------------------------------------------
def dirstructure.Directory.calcStats (   self,
  make_pie = True 
) 
Walk all subdirs and calculate weight,fails and successes.
Moreove propagate the sample and releases names.

Definition at line 96 of file dirstructure.py.

References dirstructure.Directory.comparisons, dirstructure.Directory.different_histograms, dirstructure.Directory.filename1, dirstructure.Directory.filename2, dirstructure.Directory.full_path, join(), dqm::DQMFileSaverOnline::SnapshotFiles.meta, dirstructure.Directory.meta, dirstructure.Directory.mother_dir, dirstructure.Directory.n_comp_fails, dirstructure.Directory.n_comp_nulls, dirstructure.Directory.n_comp_skiped, dirstructure.Directory.n_comp_successes, dirstructure.Directory.n_fails, dirstructure.Directory.n_missing_objs, dirstructure.Directory.n_nulls, dirstructure.Directory.n_skiped, dirstructure.Directory.n_successes, ElectronMVAID.ElectronMVAID.name, AlignableObjectId::entry.name, average.Average.name, counter.Counter.name, histograms.Histograms.name, cond::persistency::GLOBAL_TAG::NAME.name, cond::persistency::TAG::NAME.name, cond::persistency::RUN_INFO::RUN_NUMBER.name, FWLite.ElectronMVAID.name, TmModule.name, cond::persistency::GTEditorData.name, cond::persistency::GLOBAL_TAG::VALIDITY.name, cond::persistency::TAG::TIME_TYPE.name, cond::persistency::RUN_INFO::START_TIME.name, cond::persistency::GLOBAL_TAG::DESCRIPTION.name, cond::persistency::TAG::OBJECT_TYPE.name, core.autovars.NTupleVariable.name, cond::persistency::RUN_INFO::END_TIME.name, cond::persistency::GLOBAL_TAG::RELEASE.name, cond::persistency::TAG::SYNCHRONIZATION.name, DQMRivetClient::NormOption.name, MEPSet.name, cond::persistency::GLOBAL_TAG::SNAPSHOT_TIME.name, cond::persistency::TAG::END_OF_VALIDITY.name, cond::persistency::GLOBAL_TAG::INSERTION_TIME.name, cond::persistency::TAG::DESCRIPTION.name, cond::persistency::O2O_RUN::JOB_NAME.name, cms::dd::NameValuePair< T >.name, cond::persistency::TAG::LAST_VALIDATED_TIME.name, cond::persistency::O2O_RUN::START_TIME.name, FWTGeoRecoGeometry::Info.name, cond::persistency::TAG::INSERTION_TIME.name, cond::persistency::O2O_RUN::END_TIME.name, cond::persistency::TAG::MODIFICATION_TIME.name, cond::persistency::O2O_RUN::STATUS_CODE.name, cond::persistency::TAG::PROTECTION_CODE.name, ParameterSet.name, cond::persistency::O2O_RUN::LOG.name, nanoaod::MergeableCounterTable::SingleColumn< T >.name, OutputMEPSet.name, preexistingValidation.PreexistingValidation.name, dataset.BaseDataset.name, SummaryTableOutputBranches::NamedBranchPtr.name, AlignmentConstraint.name, PixelDCSObject< class >::Item.name, personalPlayback.Applet.name, fit::RootMinuitCommand.name, cms::dd::ValuePair< T, U >.name, Types._Untracked.name, EventStringOutputBranches::NamedBranchPtr.name, MagCylinder.name, analyzer.Analyzer.name, production_tasks.Task.name, HcalForwardLibWriter::FileHandle.name, DQMRivetClient::LumiOption.name, heppy::ParSet.name, o2olib.O2OJob.name, cond::persistency::GTProxyData.name, TableOutputBranches::NamedBranchPtr.name, CharmTagger::MVAVar.name, SingleObjectCondition.name, EgHLTOfflineSummaryClient::SumHistBinData.name, edm::PathTimingSummary.name, DQMRivetClient::ScaleFactorOption.name, TriggerOutputBranches::NamedBranchPtr.name, HeavyIonCSVTagger::MVAVar.name, RHStopTracer::StopPoint.name, cms::DDAlgoArguments.name, Barrel.name, perftools::EdmEventSize::BranchRecord.name, core.autovars.NTupleObjectType.name, cond::TimeTypeSpecs.name, edm::PathSummary.name, EcalLogicID.name, alignment.Alignment.name, lumi::TriggerInfo.name, XMLProcessor::_loaderBaseConfig.name, PixelEndcapLinkMaker::Item.name, dirstructure.Weighted.name, FWTableViewManager::TableEntry.name, MEtoEDM< T >::MEtoEDMObject.name, PixelBarrelLinkMaker::Item.name, ExpressionHisto< T >.name, DQMGenericClient::EfficOption.name, python.rootplot.utilities.Hist2D.name, TreeCrawler.Package.name, Supermodule.name, cond::persistency::GLOBAL_TAG_MAP::GLOBAL_TAG_NAME.name, genericValidation.GenericValidation.name, CaloTrkProcessing::Detector.name, options.ConnectionHLTMenu.name, cond::persistency::GLOBAL_TAG_MAP::RECORD.name, cond::persistency::GLOBAL_TAG_MAP::LABEL.name, MVAVar.name, cond::persistency::GLOBAL_TAG_MAP::TAG_NAME.name, DisplayManager.DisplayManager.name, cms::DDParsingContext::CompositeMaterial.name, cond::Tag_t.name, FWLite.WorkingPoints.name, dqmoffline::l1t::HistDefinition.name, DQMGenericClient::ProfileOption.name, FastHFShowerLibrary.name, magneticfield::BaseVolumeHandle.name, nanoaod::MergeableCounterTable::VectorColumn< T >.name, emtf::Node.name, PhysicsTools::Variable::Value.name, PrintMaterialBudgetInfo.name, h4DSegm.name, BTagProbabilityToDiscriminator::Discriminator.name, DQMGenericClient::NormOption.name, DQMGenericClient::CDOption.name, CounterChecker.name, PhysicsTools::Calibration::Variable.name, cond::TagInfo_t.name, TriggerObjectTableProducer::SelectedObject.name, TrackerSectorStruct.name, MuonGeometrySanityCheckPoint.name, DQMGenericClient::NoFlowOption.name, cond::persistency::PAYLOAD::HASH.name, cond::persistency::PAYLOAD::OBJECT_TYPE.name, looper.Looper.name, Mapper::definition< ScannerT >.name, cond::persistency::PAYLOAD::DATA.name, EDMtoMEConverter.name, python.rootplot.tree2hists.Plot.name, cond::persistency::PAYLOAD::STREAMER_INFO.name, cond::persistency::PAYLOAD::VERSION.name, cond::persistency::PAYLOAD::INSERTION_TIME.name, PhysicsTools::TreeReader::Value.name, classes.MonitorData.name, PhysicsTools::ProcessRegistry< Base_t, CalibBase_t, Parent_t >.name, HistogramManager.name, classes.OutputData.name, Crystal.name, h2DSegm.name, options.HLTProcessOptions.name, cond::persistency::IOV::TAG_NAME.name, cond::persistency::IOV::SINCE.name, cond::persistency::IOV::PAYLOAD_HASH.name, cond::persistency::IOV::INSERTION_TIME.name, DQMNet::WaitObject.name, python.rootplot.utilities.Hist.name, AlpgenParameterName.name, config.Analyzer.name, geometry.Structure.name, hDigis.name, core.autovars.NTupleSubObject.name, Capsule.name, core.autovars.NTupleObject.name, Ceramic.name, PhysicsTools::Variable.name, SiStripMonitorDigi.name, BulkSilicon.name, config.Service.name, CaloSD::Detector.name, APD.name, nanoaod::FlatTable::Column.name, core.autovars.NTupleCollection.name, NanoAODDQM::SelGroupConfig.name, BPHRecoBuilder::BPHRecoSource.name, BPHRecoBuilder::BPHCompSource.name, StraightTrackAlignment::RPSetPlots.name, cond::persistency::TAG_AUTHORIZATION::TAG_NAME.name, cond::persistency::TAG_AUTHORIZATION::ACCESS_TYPE.name, cond::persistency::TAG_AUTHORIZATION::CREDENTIAL.name, cond::persistency::TAG_AUTHORIZATION::CREDENTIAL_TYPE.name, InnerLayerVolume.name, cond::payloadInspector::TagReference.name, cond::persistency::TAG_LOG::TAG_NAME.name, cond::persistency::TAG_LOG::EVENT_TIME.name, cond::persistency::TAG_LOG::USER_NAME.name, cond::persistency::TAG_LOG::HOST_NAME.name, cond::persistency::TAG_LOG::COMMAND.name, cond::persistency::TAG_LOG::ACTION.name, cond::persistency::TAG_LOG::USER_TEXT.name, personalPlayback.FrameworkJob.name, Grid.name, Grille.name, BackPipe.name, plotscripts.SawTeethFunction.name, PatchPanel.name, BackCoolTank.name, DryAirTube.name, python.rootplot.utilities.RootFile.name, crabFunctions.CrabTask.name, MBCoolTube.name, hHits.name, MBManif.name, cscdqm::ParHistoDef.name, hDeltaR.name, hTMaxCell.name, BeautifulSoup.Tag.name, hParam.name, hMuonStat.name, hTOF.name, hTDelay.name, hTimes< hTime >.name, SummaryOutputProducer::GenericSummary.name, BeautifulSoup.SoupStrainer.name, print(), python.rootplot.root2matplotlib.replace(), dirstructure.Directory.stats_calculated, dirstructure.Directory.subdirs, RPCDQMObject::DQMObjectItem.weight, TtFullLepKinSolver::NeutrinoSolution.weight, JetAnaPythia< Jet >.weight, lheh5::EventHeader.weight, ZIterativeAlgorithmWithFit::ZIterativeAlgorithmWithFitPlots.weight, ExtendedPerigeeTrajectoryError.weight, SeedCandidate.weight, LutXml::_Config.weight, materialEffect::Effect.weight, L1Analysis::L1AnalysisGeneratorDataFormat.weight, dirstructure.Weighted.weight, PrimaryVertexMonitor.weight, MuIsolatorResultProducer< BT >::DepositConf.weight, DTTrigTest.weight, FastLineRecognition::Cluster.weight, classes.MonitorData.weight, HLTBitAnalyzer.weight, HGCalImagingAlgo::Hexel.weight, HGCalCLUEAlgoT< TILE >::CellsOnLayer.weight, and areaInfo.weight.

Referenced by dirstructure.Directory.print_report().

96 
97  def calcStats(self,make_pie=True):
98  '''Walk all subdirs and calculate weight,fails and successes.
99  Moreove propagate the sample and releases names.
100  '''
101  if self.stats_calculated:
102  return 0
103 
104  self.n_fails=0
105  self.n_successes=0
106  self.n_nulls=0
107  self.n_comp_fails=0
108  self.n_comp_successes=0
109  self.n_comp_nulls=0
110  self.weight=0
111 
112  self.n_skiped = 0
113  self.n_comp_skiped = 0
114  self.n_missing_objs = len(self.different_histograms['file1'])+len(self.different_histograms['file2'])
115  if self.n_missing_objs != 0:
116  print(" [*] Missing in %s: %s" %(self.filename1, self.different_histograms['file1']))
117  print(" [*] Missing in %s: %s" %(self.filename2, self.different_histograms['file2']))
118  # clean from empty dirs
119  self.subdirs = [subdir for subdir in self.subdirs if not subdir.is_empty()]
120 
121  for comp in self.comparisons:
122  if comp.status == SKIPED: #in case its in black list & skiped
123  self.n_skiped += 1
124  self.n_comp_skiped += 1
125  self.weight+=1
126  else: #else original code -> to check for Fails and Successes
127  self.rank_histo.Fill(comp.rank)
128  self.weight+=1
129  if comp.status == FAIL:
130  self.n_fails+=1
131  self.n_comp_fails+=1
132  elif comp.status == SUCCESS:
133  self.n_successes+=1
134  self.n_comp_successes+=1
135  else:
136  self.n_nulls+=1
137  self.n_comp_nulls+=1
138 
139  for subdir in self.subdirs:
140  subdir.mother_dir=join(self.mother_dir,self.name)
141  subdir.full_path = join(self.mother_dir,self.name).replace("/Run summary","")
142  subdir.calcStats(make_pie)
143  subdir.meta=self.meta
144  self.weight+=subdir.weight
145  self.n_fails+=subdir.n_fails
146  self.n_successes+=subdir.n_successes
147  self.n_nulls+=subdir.n_nulls
148 
149  self.n_skiped+=subdir.n_skiped
150  self.n_missing_objs += subdir.n_missing_objs
151 
152  self.rank_histo.Add(subdir.rank_histo)
153 
154  self.stats_calculated=True
155  self.full_path = join(self.mother_dir,self.name).replace("/Run summary","")
156  #if make_pie:
157  #self.__create_pie_image()
print
void print(TMatrixD &m, const char *label=nullptr, bool mathematicaFormat=false)
Definition: Utilities.cc:47
join
static std::string join(char **cmd)
Definition: RemoteFile.cc:19
def dirstructure.Directory.get_fail_rate (   self)

Definition at line 208 of file dirstructure.py.

References dirstructure.Directory.n_fails, RPCDQMObject::DQMObjectItem.weight, TtFullLepKinSolver::NeutrinoSolution.weight, JetAnaPythia< Jet >.weight, lheh5::EventHeader.weight, ZIterativeAlgorithmWithFit::ZIterativeAlgorithmWithFitPlots.weight, ExtendedPerigeeTrajectoryError.weight, SeedCandidate.weight, LutXml::_Config.weight, materialEffect::Effect.weight, L1Analysis::L1AnalysisGeneratorDataFormat.weight, dirstructure.Weighted.weight, PrimaryVertexMonitor.weight, MuIsolatorResultProducer< BT >::DepositConf.weight, DTTrigTest.weight, FastLineRecognition::Cluster.weight, classes.MonitorData.weight, HLTBitAnalyzer.weight, HGCalImagingAlgo::Hexel.weight, HGCalCLUEAlgoT< TILE >::CellsOnLayer.weight, and areaInfo.weight.

Referenced by dirstructure.Directory.__create_pie_image(), dirstructure.Directory.__repr__(), dirstructure.Directory.get_summary_chart_ajax(), and dirstructure.Directory.print_report().

208 
209  def get_fail_rate(self):
210  if self.weight == 0:return 0
211  return 100.*self.n_fails/self.weight
def dirstructure.Directory.get_null_rate (   self)

Definition at line 216 of file dirstructure.py.

References dirstructure.Directory.n_nulls, RPCDQMObject::DQMObjectItem.weight, TtFullLepKinSolver::NeutrinoSolution.weight, JetAnaPythia< Jet >.weight, lheh5::EventHeader.weight, ZIterativeAlgorithmWithFit::ZIterativeAlgorithmWithFitPlots.weight, ExtendedPerigeeTrajectoryError.weight, SeedCandidate.weight, LutXml::_Config.weight, materialEffect::Effect.weight, L1Analysis::L1AnalysisGeneratorDataFormat.weight, dirstructure.Weighted.weight, PrimaryVertexMonitor.weight, MuIsolatorResultProducer< BT >::DepositConf.weight, DTTrigTest.weight, FastLineRecognition::Cluster.weight, classes.MonitorData.weight, HLTBitAnalyzer.weight, HGCalImagingAlgo::Hexel.weight, HGCalCLUEAlgoT< TILE >::CellsOnLayer.weight, and areaInfo.weight.

Referenced by dirstructure.Directory.__create_pie_image(), dirstructure.Directory.__repr__(), dirstructure.Directory.get_summary_chart_ajax(), and dirstructure.Directory.print_report().

216 
217  def get_null_rate(self):
218  if self.weight == 0:return 0
219  return 100.*self.n_nulls/self.weight
def dirstructure.Directory.get_skiped_rate (   self)

Definition at line 205 of file dirstructure.py.

References dirstructure.Directory.n_skiped, RPCDQMObject::DQMObjectItem.weight, TtFullLepKinSolver::NeutrinoSolution.weight, JetAnaPythia< Jet >.weight, lheh5::EventHeader.weight, ZIterativeAlgorithmWithFit::ZIterativeAlgorithmWithFitPlots.weight, ExtendedPerigeeTrajectoryError.weight, SeedCandidate.weight, LutXml::_Config.weight, materialEffect::Effect.weight, L1Analysis::L1AnalysisGeneratorDataFormat.weight, dirstructure.Weighted.weight, PrimaryVertexMonitor.weight, MuIsolatorResultProducer< BT >::DepositConf.weight, DTTrigTest.weight, FastLineRecognition::Cluster.weight, classes.MonitorData.weight, HLTBitAnalyzer.weight, HGCalImagingAlgo::Hexel.weight, HGCalCLUEAlgoT< TILE >::CellsOnLayer.weight, and areaInfo.weight.

Referenced by dirstructure.Directory.__create_pie_image(), dirstructure.Directory.get_summary_chart_ajax(), and dirstructure.Directory.print_report().

205 
206  def get_skiped_rate(self):
207  if self.weight == 0: return 0
return 100.*self.n_skiped/self.weight
def dirstructure.Directory.get_subdirs_dict (   self)

Definition at line 158 of file dirstructure.py.

References dirstructure.Directory.subdirs.

158 
159  def get_subdirs_dict(self):
160  subdirdict={}
161  for subdir in self.subdirs:
162  subdirdict[subdir.name]=subdir
163  return subdirdict
def dirstructure.Directory.get_subdirs_names (   self)

Definition at line 164 of file dirstructure.py.

References dirstructure.Directory.subdirs.

164 
165  def get_subdirs_names(self):
166  subdirnames=[]
167  for subdir in self.subdirs:
168  subdirnames.append(subdir.name)
169  return subdirnames
def dirstructure.Directory.get_success_rate (   self)

Definition at line 212 of file dirstructure.py.

References dirstructure.Directory.n_successes, RPCDQMObject::DQMObjectItem.weight, TtFullLepKinSolver::NeutrinoSolution.weight, JetAnaPythia< Jet >.weight, lheh5::EventHeader.weight, ZIterativeAlgorithmWithFit::ZIterativeAlgorithmWithFitPlots.weight, ExtendedPerigeeTrajectoryError.weight, SeedCandidate.weight, LutXml::_Config.weight, materialEffect::Effect.weight, L1Analysis::L1AnalysisGeneratorDataFormat.weight, dirstructure.Weighted.weight, PrimaryVertexMonitor.weight, MuIsolatorResultProducer< BT >::DepositConf.weight, DTTrigTest.weight, FastLineRecognition::Cluster.weight, classes.MonitorData.weight, HLTBitAnalyzer.weight, HGCalImagingAlgo::Hexel.weight, HGCalCLUEAlgoT< TILE >::CellsOnLayer.weight, and areaInfo.weight.

Referenced by dirstructure.Directory.__create_pie_image(), dirstructure.Directory.__repr__(), dirstructure.Directory.get_summary_chart_ajax(), and dirstructure.Directory.print_report().

212 
213  def get_success_rate(self):
214  if self.weight == 0:return 1
215  return 100.*self.n_successes/self.weight
def dirstructure.Directory.get_summary_chart_ajax (   self,
  w = 400,
  h = 300 
) 
Emit the ajax to build a pie chart using google apis...

Definition at line 170 of file dirstructure.py.

References dirstructure.Directory.get_fail_rate(), dirstructure.Directory.get_null_rate(), dirstructure.Directory.get_skiped_rate(), and dirstructure.Directory.get_success_rate().

170 
171  def get_summary_chart_ajax(self,w=400,h=300):
172  """Emit the ajax to build a pie chart using google apis...
173  """
174  url = "https://chart.googleapis.com/chart?"
175  url+= "cht=p3" # Select the 3d chart
176  #url+= "&chl=Success|Null|Fail" # give labels
177  url+= "&chco=00FF00|FFFF00|FF0000|7A7A7A" # give colours to labels
178  url+= "&chs=%sx%s" %(w,h)
179  #url+= "&chtt=%s" %self.name
180  url+= "&chd=t:%.2f,%.2f,%.2f,%.2f"%(self.get_success_rate(),self.get_null_rate(),self.get_fail_rate(),self.get_skiped_rate())
181 
182  return url
def dirstructure.Directory.get_summary_chart_name (   self)

Definition at line 233 of file dirstructure.py.

References dirstructure.Directory.__get_full_path(), and join().

Referenced by dirstructure.Directory.__create_pie_image().

233 
234  def get_summary_chart_name(self):
235  return join(self.__get_full_path(),"summary_chart.png")
join
static std::string join(char **cmd)
Definition: RemoteFile.cc:19
def dirstructure.Directory.is_empty (   self)

Definition at line 91 of file dirstructure.py.

References dirstructure.Directory.comparisons, and dirstructure.Directory.subdirs.

Referenced by dirstructure.Directory.__repr__(), and utils_v2.StatisticalTest.do_test().

91 
92  def is_empty(self):
93  if len(self.subdirs)==0 and len(self.comparisons)==0:
94  return True
95  return False
def dirstructure.Directory.print_report (   self,
  indent = "",
  verbose = False 
)

Definition at line 183 of file dirstructure.py.

References dirstructure.Directory.calcStats(), dirstructure.Directory.comparisons, dirstructure.Directory.get_fail_rate(), dirstructure.Directory.get_null_rate(), dirstructure.Directory.get_skiped_rate(), dirstructure.Directory.get_success_rate(), dirstructure.Directory.mother_dir, dirstructure.Directory.n_fails, dirstructure.Directory.n_missing_objs, dirstructure.Directory.n_nulls, dirstructure.Directory.n_skiped, dirstructure.Directory.n_successes, ElectronMVAID.ElectronMVAID.name, AlignableObjectId::entry.name, average.Average.name, counter.Counter.name, histograms.Histograms.name, cond::persistency::RUN_INFO::RUN_NUMBER.name, cond::persistency::TAG::NAME.name, FWLite.ElectronMVAID.name, TmModule.name, cond::persistency::GTEditorData.name, cond::persistency::GLOBAL_TAG::NAME.name, cond::persistency::RUN_INFO::START_TIME.name, cond::persistency::TAG::TIME_TYPE.name, cond::persistency::GLOBAL_TAG::VALIDITY.name, cond::persistency::GLOBAL_TAG::DESCRIPTION.name, core.autovars.NTupleVariable.name, cond::persistency::RUN_INFO::END_TIME.name, cond::persistency::TAG::OBJECT_TYPE.name, cond::persistency::GLOBAL_TAG::RELEASE.name, cond::persistency::TAG::SYNCHRONIZATION.name, DQMRivetClient::NormOption.name, cond::persistency::GLOBAL_TAG::SNAPSHOT_TIME.name, cond::persistency::TAG::END_OF_VALIDITY.name, MEPSet.name, cond::persistency::GLOBAL_TAG::INSERTION_TIME.name, cond::persistency::O2O_RUN::JOB_NAME.name, cond::persistency::TAG::DESCRIPTION.name, cms::dd::NameValuePair< T >.name, cond::persistency::O2O_RUN::START_TIME.name, cond::persistency::TAG::LAST_VALIDATED_TIME.name, cond::persistency::O2O_RUN::END_TIME.name, cond::persistency::TAG::INSERTION_TIME.name, FWTGeoRecoGeometry::Info.name, cond::persistency::O2O_RUN::STATUS_CODE.name, cond::persistency::TAG::MODIFICATION_TIME.name, ParameterSet.name, cond::persistency::O2O_RUN::LOG.name, nanoaod::MergeableCounterTable::SingleColumn< T >.name, cond::persistency::TAG::PROTECTION_CODE.name, OutputMEPSet.name, preexistingValidation.PreexistingValidation.name, AlignmentConstraint.name, dataset.BaseDataset.name, PixelDCSObject< class >::Item.name, SummaryTableOutputBranches::NamedBranchPtr.name, fit::RootMinuitCommand.name, cms::dd::ValuePair< T, U >.name, personalPlayback.Applet.name, EventStringOutputBranches::NamedBranchPtr.name, Types._Untracked.name, MagCylinder.name, analyzer.Analyzer.name, production_tasks.Task.name, HcalForwardLibWriter::FileHandle.name, heppy::ParSet.name, DQMRivetClient::LumiOption.name, o2olib.O2OJob.name, cond::persistency::GTProxyData.name, TableOutputBranches::NamedBranchPtr.name, CharmTagger::MVAVar.name, TriggerOutputBranches::NamedBranchPtr.name, HeavyIonCSVTagger::MVAVar.name, SingleObjectCondition.name, EgHLTOfflineSummaryClient::SumHistBinData.name, edm::PathTimingSummary.name, DQMRivetClient::ScaleFactorOption.name, cms::DDAlgoArguments.name, RHStopTracer::StopPoint.name, Barrel.name, perftools::EdmEventSize::BranchRecord.name, core.autovars.NTupleObjectType.name, cond::TimeTypeSpecs.name, edm::PathSummary.name, EcalLogicID.name, alignment.Alignment.name, lumi::TriggerInfo.name, PixelEndcapLinkMaker::Item.name, dirstructure.Weighted.name, XMLProcessor::_loaderBaseConfig.name, MEtoEDM< T >::MEtoEDMObject.name, FWTableViewManager::TableEntry.name, PixelBarrelLinkMaker::Item.name, ExpressionHisto< T >.name, DQMGenericClient::EfficOption.name, python.rootplot.utilities.Hist2D.name, TreeCrawler.Package.name, Supermodule.name, cond::persistency::GLOBAL_TAG_MAP::GLOBAL_TAG_NAME.name, CaloTrkProcessing::Detector.name, genericValidation.GenericValidation.name, options.ConnectionHLTMenu.name, cond::persistency::GLOBAL_TAG_MAP::RECORD.name, cond::persistency::GLOBAL_TAG_MAP::LABEL.name, MVAVar.name, cond::persistency::GLOBAL_TAG_MAP::TAG_NAME.name, cms::DDParsingContext::CompositeMaterial.name, DisplayManager.DisplayManager.name, FWLite.WorkingPoints.name, cond::Tag_t.name, dqmoffline::l1t::HistDefinition.name, DQMGenericClient::ProfileOption.name, magneticfield::BaseVolumeHandle.name, nanoaod::MergeableCounterTable::VectorColumn< T >.name, FastHFShowerLibrary.name, emtf::Node.name, PhysicsTools::Variable::Value.name, h4DSegm.name, PrintMaterialBudgetInfo.name, DQMGenericClient::NormOption.name, BTagProbabilityToDiscriminator::Discriminator.name, DQMGenericClient::CDOption.name, CounterChecker.name, PhysicsTools::Calibration::Variable.name, TriggerObjectTableProducer::SelectedObject.name, cond::TagInfo_t.name, TrackerSectorStruct.name, MuonGeometrySanityCheckPoint.name, cond::persistency::PAYLOAD::HASH.name, DQMGenericClient::NoFlowOption.name, looper.Looper.name, Mapper::definition< ScannerT >.name, cond::persistency::PAYLOAD::OBJECT_TYPE.name, cond::persistency::PAYLOAD::DATA.name, EDMtoMEConverter.name, python.rootplot.tree2hists.Plot.name, cond::persistency::PAYLOAD::STREAMER_INFO.name, cond::persistency::PAYLOAD::VERSION.name, cond::persistency::PAYLOAD::INSERTION_TIME.name, PhysicsTools::TreeReader::Value.name, classes.MonitorData.name, PhysicsTools::ProcessRegistry< Base_t, CalibBase_t, Parent_t >.name, HistogramManager.name, classes.OutputData.name, Crystal.name, h2DSegm.name, options.HLTProcessOptions.name, cond::persistency::IOV::TAG_NAME.name, cond::persistency::IOV::SINCE.name, cond::persistency::IOV::PAYLOAD_HASH.name, cond::persistency::IOV::INSERTION_TIME.name, DQMNet::WaitObject.name, python.rootplot.utilities.Hist.name, AlpgenParameterName.name, config.Analyzer.name, geometry.Structure.name, hDigis.name, core.autovars.NTupleSubObject.name, Capsule.name, core.autovars.NTupleObject.name, Ceramic.name, PhysicsTools::Variable.name, SiStripMonitorDigi.name, BulkSilicon.name, config.Service.name, CaloSD::Detector.name, APD.name, nanoaod::FlatTable::Column.name, core.autovars.NTupleCollection.name, NanoAODDQM::SelGroupConfig.name, BPHRecoBuilder::BPHRecoSource.name, BPHRecoBuilder::BPHCompSource.name, StraightTrackAlignment::RPSetPlots.name, cond::persistency::TAG_AUTHORIZATION::TAG_NAME.name, cond::persistency::TAG_AUTHORIZATION::ACCESS_TYPE.name, cond::persistency::TAG_AUTHORIZATION::CREDENTIAL.name, cond::persistency::TAG_AUTHORIZATION::CREDENTIAL_TYPE.name, InnerLayerVolume.name, cond::payloadInspector::TagReference.name, cond::persistency::TAG_LOG::TAG_NAME.name, cond::persistency::TAG_LOG::EVENT_TIME.name, cond::persistency::TAG_LOG::USER_NAME.name, cond::persistency::TAG_LOG::HOST_NAME.name, cond::persistency::TAG_LOG::COMMAND.name, cond::persistency::TAG_LOG::ACTION.name, cond::persistency::TAG_LOG::USER_TEXT.name, personalPlayback.FrameworkJob.name, Grid.name, Grille.name, BackPipe.name, plotscripts.SawTeethFunction.name, PatchPanel.name, BackCoolTank.name, DryAirTube.name, python.rootplot.utilities.RootFile.name, crabFunctions.CrabTask.name, MBCoolTube.name, hHits.name, MBManif.name, cscdqm::ParHistoDef.name, hDeltaR.name, hTMaxCell.name, BeautifulSoup.Tag.name, hParam.name, hMuonStat.name, hTOF.name, hTDelay.name, hTimes< hTime >.name, SummaryOutputProducer::GenericSummary.name, BeautifulSoup.SoupStrainer.name, print(), dirstructure.Directory.subdirs, RPCDQMObject::DQMObjectItem.weight, TtFullLepKinSolver::NeutrinoSolution.weight, JetAnaPythia< Jet >.weight, lheh5::EventHeader.weight, ZIterativeAlgorithmWithFit::ZIterativeAlgorithmWithFitPlots.weight, ExtendedPerigeeTrajectoryError.weight, SeedCandidate.weight, LutXml::_Config.weight, materialEffect::Effect.weight, L1Analysis::L1AnalysisGeneratorDataFormat.weight, dirstructure.Weighted.weight, PrimaryVertexMonitor.weight, MuIsolatorResultProducer< BT >::DepositConf.weight, DTTrigTest.weight, FastLineRecognition::Cluster.weight, classes.MonitorData.weight, HLTBitAnalyzer.weight, HGCalImagingAlgo::Hexel.weight, HGCalCLUEAlgoT< TILE >::CellsOnLayer.weight, and areaInfo.weight.

183 
184  def print_report(self,indent="",verbose=False):
185  if len(indent)==0:
186  self.calcStats(make_pie=False)
187  # print small failure report
188  if verbose:
189  fail_comps=[comp for comp in self.comparisons if comp.status==FAIL]
190  fail_comps=sorted(fail_comps,key=lambda comp:comp.name )
191  if len(fail_comps)>0:
192  print(indent+"* %s/%s:" %(self.mother_dir,self.name))
193  for comp in fail_comps:
194  print(indent+" - %s: %s Test Failed (pval = %s) " %(comp.name,comp.test_name,comp.rank))
195  for subdir in self.subdirs:
196  subdir.print_report(indent+" ",verbose)
197 
198  if len(indent)==0:
199  print("\n%s - summary of %s tests:" %(self.name,self.weight))
200  print(" o Failiures: %.2f%% (%s/%s)" %(self.get_fail_rate(),self.n_fails,self.weight))
201  print(" o Nulls: %.2f%% (%s/%s) " %(self.get_null_rate(),self.n_nulls,self.weight))
202  print(" o Successes: %.2f%% (%s/%s) " %(self.get_success_rate(),self.n_successes,self.weight))
203  print(" o Skipped: %.2f%% (%s/%s) " %(self.get_skiped_rate(),self.n_skiped,self.weight))
204  print(" o Missing objects: %s" %(self.n_missing_objs))
print
void print(TMatrixD &m, const char *label=nullptr, bool mathematicaFormat=false)
Definition: Utilities.cc:47
def dirstructure.Directory.prune (   self,
  expandable_dir 
) 
Eliminate from the tree the directory the expandable ones.

Definition at line 272 of file dirstructure.py.

References dirstructure.Directory.comparisons, dirstructure.Directory.n_comp_fails, dirstructure.Directory.n_comp_nulls, dirstructure.Directory.n_comp_skiped, dirstructure.Directory.n_comp_successes, print(), TtEventPartons.prune(), PFMultiDepthClusterizer.prune(), dirstructure.Directory.prune(), Config.Process.prune(), and dirstructure.Directory.subdirs.

Referenced by dirstructure.Directory.prune().

272 
273  def prune(self,expandable_dir):
274  """Eliminate from the tree the directory the expandable ones.
275  """
276  #print "pruning %s" %self.name
277  exp_index=-1
278  counter=0
279  for subdir in self.subdirs:
280  # Eliminate any trace of the expandable path in the mother directories
281  # for depths higher than 1
282  subdir.mother_dir=subdir.mother_dir.replace("/"+expandable_dir,"")
283  if subdir.name==expandable_dir:
284  exp_index=counter
285  counter+=1
286 
287  # Did we find an expandable?
288  if exp_index>=0:
289  exp_dir=self.subdirs[exp_index]
290  for subsubdir in exp_dir.subdirs:
291  #print "*******",subsubdir.mother_dir,
292  subsubdir.mother_dir=subsubdir.mother_dir.replace("/"+expandable_dir,"")
293  while "//" in subsubdir.mother_dir:
294  print(subsubdir.mother_dir)
295  subsubdir.mother_dir=subsubdir.mother_dir.replace("//","/")
296  #print "*******",subsubdir.mother_dir
297  self.subdirs.append(subsubdir)
298 
299  for comp in exp_dir.comparisons:
300  comp.mother_dir=comp.mother_dir.replace("/"+expandable_dir,"")
301  while "//" in comp.mother_dir:
302  comp.mother_dir
303  comp.mother_dir=comp.mother_dir.replace("/")
304  if not comp in self.comparisons: #in case not to append same comparisons few times
305  self.comparisons.append(comp) # add a comparison
306  self.n_comp_fails = exp_dir.n_comp_fails #copy to-be removed directory
307  self.n_comp_nulls = exp_dir.n_comp_nulls # numbers to parent directory
308  self.n_comp_successes = exp_dir.n_comp_successes
309  self.n_comp_skiped = exp_dir.n_comp_skiped
310 
311  del self.subdirs[exp_index]
312  self.prune(expandable_dir)
313 
314  for subdir in self.subdirs:
315  subdir.prune(expandable_dir)
print
void print(TMatrixD &m, const char *label=nullptr, bool mathematicaFormat=false)
Definition: Utilities.cc:47

Member Data Documentation

dirstructure.Directory.comparisons

Definition at line 67 of file dirstructure.py.

Referenced by dirstructure.Directory.__repr__(), dirstructure.Directory.calcStats(), dirstructure.Directory.is_empty(), dirstructure.Directory.print_report(), and dirstructure.Directory.prune().

dirstructure.Directory.different_histograms

Definition at line 83 of file dirstructure.py.

Referenced by dirstructure.Directory.calcStats(), and dqm_interfaces.DirWalkerFile.ls().

dirstructure.Directory.do_pngs

Definition at line 80 of file dirstructure.py.

Referenced by dqm_interfaces.DirWalkerDB.run().

dirstructure.Directory.draw_success

Definition at line 79 of file dirstructure.py.

dirstructure.Directory.filename1

Definition at line 86 of file dirstructure.py.

Referenced by dirstructure.Directory.calcStats().

dirstructure.Directory.filename2

Definition at line 87 of file dirstructure.py.

Referenced by dirstructure.Directory.calcStats().

dirstructure.Directory.full_path

Definition at line 89 of file dirstructure.py.

Referenced by dirstructure.Directory.calcStats().

dirstructure.Directory.meta

Definition at line 65 of file dirstructure.py.

Referenced by dirstructure.Directory.calcStats().

dirstructure.Directory.mother_dir

Definition at line 64 of file dirstructure.py.

Referenced by dirstructure.Directory.__create_on_disk(), dirstructure.Directory.__get_full_path(), dirstructure.Comparison.__get_img_name(), dirstructure.Comparison.__make_img_dir(), dirstructure.Directory.calcStats(), and dirstructure.Directory.print_report().

dirstructure.Directory.n_comp_fails

Definition at line 73 of file dirstructure.py.

Referenced by dirstructure.Directory.calcStats(), and dirstructure.Directory.prune().

dirstructure.Directory.n_comp_nulls

Definition at line 75 of file dirstructure.py.

Referenced by dirstructure.Directory.calcStats(), and dirstructure.Directory.prune().

dirstructure.Directory.n_comp_skiped

Definition at line 72 of file dirstructure.py.

Referenced by dirstructure.Directory.calcStats(), and dirstructure.Directory.prune().

dirstructure.Directory.n_comp_successes

Definition at line 74 of file dirstructure.py.

Referenced by dirstructure.Directory.calcStats(), and dirstructure.Directory.prune().

dirstructure.Directory.n_fails

Definition at line 68 of file dirstructure.py.

Referenced by dirstructure.Directory.__create_pie_image(), dirstructure.Directory.__repr__(), dirstructure.Directory.calcStats(), dirstructure.Directory.get_fail_rate(), and dirstructure.Directory.print_report().

dirstructure.Directory.n_missing_objs

Definition at line 88 of file dirstructure.py.

Referenced by dirstructure.Directory.calcStats(), and dirstructure.Directory.print_report().

dirstructure.Directory.n_nulls

Definition at line 70 of file dirstructure.py.

Referenced by dirstructure.Directory.__create_pie_image(), dirstructure.Directory.__repr__(), dirstructure.Directory.calcStats(), dirstructure.Directory.get_null_rate(), and dirstructure.Directory.print_report().

dirstructure.Directory.n_skiped

Definition at line 71 of file dirstructure.py.

Referenced by dirstructure.Directory.__create_pie_image(), dirstructure.Directory.calcStats(), dirstructure.Directory.get_skiped_rate(), and dirstructure.Directory.print_report().

dirstructure.Directory.n_successes

Definition at line 69 of file dirstructure.py.

Referenced by dirstructure.Directory.__create_pie_image(), dirstructure.Directory.__repr__(), dirstructure.Directory.calcStats(), dirstructure.Directory.get_success_rate(), and dirstructure.Directory.print_report().

dirstructure.Directory.rank_histo

Definition at line 81 of file dirstructure.py.

dirstructure.Directory.stats_calculated

Definition at line 77 of file dirstructure.py.

Referenced by dirstructure.Directory.calcStats().

dirstructure.Directory.subdirs

Definition at line 66 of file dirstructure.py.

Referenced by dirstructure.Directory.__repr__(), dirstructure.Directory.calcStats(), dirstructure.Directory.get_subdirs_dict(), dirstructure.Directory.get_subdirs_names(), dirstructure.Directory.is_empty(), dirstructure.Directory.print_report(), and dirstructure.Directory.prune().

dirstructure.Directory.weight

Definition at line 76 of file dirstructure.py.

Generated for CMSSW Reference Manual by doxygen 1.8.5