dirstructure.Directory Class Reference

Inheritance diagram for dirstructure.Directory:

Public Member Functions
def	__init__ (self, name, mother_dir="", meta=CompInfo(), draw_success=False, do_pngs=False)
def	__repr__ (self)
def	calcStats (self, make_pie=True)
def	get_fail_rate (self)
def	get_null_rate (self)
def	get_skiped_rate (self)
def	get_subdirs_dict (self)
def	get_subdirs_names (self)
def	get_success_rate (self)
def	get_summary_chart_ajax (self, w=400, h=300)
def	get_summary_chart_name (self)
def	is_empty (self)
def	print_report (self, indent="", verbose=False)
def	prune (self, expandable_dir)
Public Member Functions inherited from dirstructure.Weighted
def	__init__ (self, name, weight=1)

Public Attributes
	comparisons
	different_histograms
	do_pngs
	draw_success
	filename1
	filename2
	full_path
	meta
	mother_dir
	n_comp_fails
	n_comp_nulls
	n_comp_skiped
	n_comp_successes
	n_fails
	n_missing_objs
	n_nulls
	n_skiped
	n_successes
	rank_histo
	stats_calculated
	subdirs
	weight
Public Attributes inherited from dirstructure.Weighted
	name
	weight

Private Member Functions
def	__create_on_disk (self)
def	__create_pie_image (self)
def	__get_full_path (self)

Detailed Description

Definition at line 60 of file dirstructure.py.

Constructor & Destructor Documentation

def dirstructure.Directory.__init__	(		self,
			name,
			mother_dir = "",
			meta = CompInfo(),
			draw_success = False,
			do_pngs = False
	)

Definition at line 61 of file dirstructure.py.

  def __init__(self,name,mother_dir="",meta=CompInfo(),draw_success=False,do_pngs=False):
    self.mother_dir=mother_dir
    self.meta=meta
    self.subdirs=[]
    self.comparisons=[]   
    self.n_fails=0
    self.n_successes=0
    self.n_nulls=0
    self.n_skiped = 0
    self.n_comp_skiped = 0
    self.n_comp_fails=0
    self.n_comp_successes=0
    self.n_comp_nulls=0 
    self.weight=0
    self.stats_calculated=False
    Weighted.__init__(self,name)
    self.draw_success=draw_success
    self.do_pngs=do_pngs
    self.rank_histo=TH1I("rh%s"%name,"",50,-0.01,1.001)
    self.rank_histo.SetDirectory(0)
    self.different_histograms = {}
    self.different_histograms['file1']= {}
    self.different_histograms['file2']= {}
    self.filename1 = ""
    self.filename2 = ""
    self.n_missing_objs = 0
    self.full_path = ""
    

Member Function Documentation

def dirstructure.Directory.__create_on_disk (   self )

private

Definition at line 224 of file dirstructure.py.

References dirstructure.Directory.__get_full_path(), and dirstructure.Directory.mother_dir.

Referenced by dirstructure.Directory.__create_pie_image().

  def __create_on_disk(self):
    if not exists(self.mother_dir) and len(self.mother_dir)!=0:
      makedirs(self.mother_dir)
    full_path=self.__get_full_path()    
    if not exists(full_path) and len(full_path)>0:
      makedirs(full_path)

def dirstructure.Directory.__create_pie_image (   self )

private

Definition at line 234 of file dirstructure.py.

References dirstructure.Directory.__create_on_disk(), mps_check.array, dirstructure.Directory.get_fail_rate(), dirstructure.Directory.get_null_rate(), dirstructure.Directory.get_skiped_rate(), dirstructure.Directory.get_success_rate(), dirstructure.Directory.get_summary_chart_name(), dirstructure.Directory.n_fails, dirstructure.Directory.n_nulls, dirstructure.Directory.n_skiped, dirstructure.Directory.n_successes, ElectronMVAID.ElectronMVAID.name, counter.Counter.name, average.Average.name, AlignableObjectId::entry.name, histograms.Histograms.name, FWLite.ElectronMVAID.name, cond::persistency::RUN_INFO::RUN_NUMBER.name, cond::persistency::TAG::NAME.name, TmModule.name, cond::persistency::GLOBAL_TAG::NAME.name, core.autovars.NTupleVariable.name, cond::persistency::GLOBAL_TAG::VALIDITY.name, cond::persistency::RUN_INFO::START_TIME.name, cond::persistency::TAG::TIME_TYPE.name, cond::persistency::GLOBAL_TAG::DESCRIPTION.name, cond::persistency::RUN_INFO::END_TIME.name, cond::persistency::TAG::OBJECT_TYPE.name, cond::persistency::GLOBAL_TAG::RELEASE.name, cond::persistency::TAG::SYNCHRONIZATION.name, cond::persistency::GLOBAL_TAG::SNAPSHOT_TIME.name, MEPSet.name, cond::persistency::TAG::END_OF_VALIDITY.name, cond::persistency::GLOBAL_TAG::INSERTION_TIME.name, cond::persistency::TAG::DESCRIPTION.name, cond::persistency::GTEditorData.name, nanoaod::MergeableCounterTable::SingleColumn< T >.name, cond::persistency::TAG::LAST_VALIDATED_TIME.name, cond::persistency::TAG::INSERTION_TIME.name, preexistingValidation.PreexistingValidation.name, cond::persistency::TAG::MODIFICATION_TIME.name, FWTGeoRecoGeometry::Info.name, Types._Untracked.name, dataset.BaseDataset.name, OutputMEPSet.name, SummaryTableOutputBranches::NamedBranchPtr.name, fit::RootMinuitCommand.name, personalPlayback.Applet.name, ParameterSet.name, PixelDCSObject< T >::Item.name, production_tasks.Task.name, PFTauMVAInputDiscriminantTranslator::DiscriminantInfo.name, analyzer.Analyzer.name, DQMRivetClient::LumiOption.name, MagCylinder.name, CharmTagger::MVAVar.name, HcalForwardLibWriter::FileHandle.name, TriggerOutputBranches::NamedBranchPtr.name, PrintSensitive.name, ParSet.name, TableOutputBranches::NamedBranchPtr.name, DQMRivetClient::ScaleFactorOption.name, RHStopTracer::StopPoint.name, EgHLTOfflineSummaryClient::SumHistBinData.name, SingleObjectCondition.name, PhysicsTools::Source.name, cond::persistency::GTProxyData.name, core.autovars.NTupleObjectType.name, MyWatcher.name, edm::PathTimingSummary.name, nanoaod::MergeableCounterTable::VectorColumn< T >.name, cond::TimeTypeSpecs.name, lumi::TriggerInfo.name, dirstructure.Weighted.name, alignment.Alignment.name, PrintMaterialBudgetInfo.name, edm::PathSummary.name, cond::persistency::GLOBAL_TAG_MAP::GLOBAL_TAG_NAME.name, PixelEndcapLinkMaker::Item.name, perftools::EdmEventSize::BranchRecord.name, DQMGenericClient::EfficOption.name, FWTableViewManager::TableEntry.name, cond::persistency::GLOBAL_TAG_MAP::RECORD.name, PixelBarrelLinkMaker::Item.name, cms::DDAlgoArguments.name, EcalLogicID.name, cond::persistency::GLOBAL_TAG_MAP::LABEL.name, validateAlignments.ParallelMergeJob.name, cond::persistency::GLOBAL_TAG_MAP::TAG_NAME.name, MEtoEDM< T >::MEtoEDMObject.name, ExpressionHisto< T >.name, XMLProcessor::_loaderBaseConfig.name, python.rootplot.utilities.Hist2D.name, cond::persistency::PAYLOAD::HASH.name, cond::persistency::PAYLOAD::OBJECT_TYPE.name, CaloTrkProcessing::Detector.name, genericValidation.GenericValidation.name, cond::persistency::PAYLOAD::DATA.name, TreeCrawler.Package.name, PrintGeomInfoAction.name, cond::persistency::PAYLOAD::STREAMER_INFO.name, cond::persistency::PAYLOAD::VERSION.name, options.ConnectionHLTMenu.name, MagGeoBuilderFromDDD::volumeHandle.name, cond::persistency::PAYLOAD::INSERTION_TIME.name, DisplayManager.DisplayManager.name, MVAVar.name, DQMGenericClient::ProfileOption.name, dqmoffline::l1t::HistDefinition.name, FWLite.WorkingPoints.name, DQMGenericClient::NormOption.name, emtf::Node.name, BTagProbabilityToDiscriminator::Discriminator.name, h4DSegm.name, PhysicsTools::Calibration::Variable.name, core.TriggerMatchAnalyzer.TriggerMatchAnalyzer.name, PhysicsTools::Variable::Value.name, FastHFShowerLibrary.name, DQMGenericClient::CDOption.name, TriggerObjectTableProducer::SelectedObject.name, CounterChecker.name, cond::TagInfo_t.name, ProcTMVA::Method.name, looper.Looper.name, TreeSaver::Var.name, DQMGenericClient::NoFlowOption.name, python.rootplot.tree2hists.Plot.name, cond::persistency::IOV::TAG_NAME.name, EDMtoMEConverter.name, TrackerSectorStruct.name, cond::persistency::IOV::SINCE.name, Mapper::definition< ScannerT >.name, cond::persistency::IOV::PAYLOAD_HASH.name, cond::persistency::IOV::INSERTION_TIME.name, classes.MonitorData.name, PhysicsTools::TreeReader::Value.name, HistogramManager.name, PhysicsTools::TrainProcessor.name, MuonGeometrySanityCheckPoint.name, classes.OutputData.name, PhysicsTools::MVAModuleHelper< Record, Object, Filler >::Value.name, PhysicsTools::ProcessRegistry< Base_t, CalibBase_t, Parent_t >.name, PhysicsTools::MVATrainer.name, options.HLTProcessOptions.name, h2DSegm.name, core.TriggerBitAnalyzer.TriggerBitAnalyzer.name, python.rootplot.utilities.Hist.name, nanoaod::FlatTable::Column.name, config.Analyzer.name, geometry.Structure.name, core.autovars.NTupleSubObject.name, DQMNet::WaitObject.name, AlpgenParameterName.name, NanoAODDQM::SelGroupConfig.name, SiStripMonitorDigi.name, core.autovars.NTupleObject.name, config.Service.name, public_plots_tools.ColorScheme.name, PhysicsTools::Variable.name, cond::persistency::TAG_LOG::TAG_NAME.name, cond::persistency::TAG_LOG::EVENT_TIME.name, cond::persistency::TAG_LOG::USER_NAME.name, cond::persistency::TAG_LOG::HOST_NAME.name, cond::persistency::TAG_LOG::COMMAND.name, cond::persistency::TAG_LOG::ACTION.name, cond::persistency::TAG_LOG::USER_TEXT.name, PhysicsTools::TrainerMonitoring::Object.name, core.autovars.NTupleCollection.name, BPHRecoBuilder::BPHRecoSource.name, BPHRecoBuilder::BPHCompSource.name, personalPlayback.FrameworkJob.name, plotscripts.SawTeethFunction.name, python.rootplot.utilities.RootFile.name, crabFunctions.CrabTask.name, hTMaxCell.name, cscdqm::ParHistoDef.name, BeautifulSoup.Tag.name, Id.name, SummaryOutputProducer::GenericSummary.name, BeautifulSoup.SoupStrainer.name, edm.print(), and ComparisonHelper.zip().

  def __create_pie_image(self):
    self.__create_on_disk()
    vals=[]
    colors=[]
    for n,col in zip((self.n_fails,self.n_nulls,self.n_successes,self.n_skiped),(kRed,kYellow,kGreen,kBlue)):
      if n!=0:
        vals.append(n)
        colors.append(col)
    valsa=array('f',vals)
    colorsa=array('i',colors)
    can = TCanvas("cpie","TPie test",100,100);
    try:
      pie = TPie("ThePie",self.name,len(vals),valsa,colorsa);
      label_n=0
      if self.n_fails!=0:
        pie.SetEntryLabel(label_n, "Fail: %.1f(%i)" %(self.get_fail_rate(),self.n_fails) );
        label_n+=1
      if self.n_nulls!=0:
        pie.SetEntryLabel(label_n, "Null: %.1f(%i)" %(self.get_null_rate(),self.n_nulls) );      
        label_n+=1
      if self.n_successes!=0:
        pie.SetEntryLabel(label_n, "Success: %.1f(%i)" %(self.get_success_rate(),self.n_successes) );
      if self.n_skiped!=0:
        pie.SetEntryLabel(label_n, "Skipped: %.1f(%i)" %(self.get_skiped_rate(),self.n_skiped));
      pie.SetY(.52);
      pie.SetAngularOffset(0.);    
      pie.SetLabelsOffset(-.3);
      #pie.SetLabelFormat("#splitline{%val (%perc)}{%txt}");
      pie.Draw("3d  nol");
      can.Print(self.get_summary_chart_name());    
    except:
      print("self.name = %s" %self.name)
      print("len(vals) = %s (vals=%s)" %(len(vals),vals))
      print("valsa = %s" %valsa)
      print("colorsa = %s" %colorsa)

def dirstructure.Directory.__get_full_path (   self )

private

Definition at line 218 of file dirstructure.py.

References join(), dirstructure.Directory.mother_dir, ElectronMVAID.ElectronMVAID.name, counter.Counter.name, average.Average.name, histograms.Histograms.name, AlignableObjectId::entry.name, FWLite.ElectronMVAID.name, TmModule.name, cond::persistency::TAG::NAME.name, cond::persistency::GLOBAL_TAG::NAME.name, core.autovars.NTupleVariable.name, cond::persistency::RUN_INFO::RUN_NUMBER.name, cond::persistency::TAG::TIME_TYPE.name, cond::persistency::GLOBAL_TAG::VALIDITY.name, cond::persistency::RUN_INFO::START_TIME.name, cond::persistency::TAG::OBJECT_TYPE.name, cond::persistency::GLOBAL_TAG::DESCRIPTION.name, cond::persistency::RUN_INFO::END_TIME.name, cond::persistency::TAG::SYNCHRONIZATION.name, cond::persistency::GLOBAL_TAG::RELEASE.name, MEPSet.name, cond::persistency::TAG::END_OF_VALIDITY.name, cond::persistency::GLOBAL_TAG::SNAPSHOT_TIME.name, cond::persistency::GTEditorData.name, cond::persistency::TAG::DESCRIPTION.name, cond::persistency::GLOBAL_TAG::INSERTION_TIME.name, nanoaod::MergeableCounterTable::SingleColumn< T >.name, cond::persistency::TAG::LAST_VALIDATED_TIME.name, cond::persistency::TAG::INSERTION_TIME.name, cond::persistency::TAG::MODIFICATION_TIME.name, preexistingValidation.PreexistingValidation.name, FWTGeoRecoGeometry::Info.name, Types._Untracked.name, dataset.BaseDataset.name, OutputMEPSet.name, SummaryTableOutputBranches::NamedBranchPtr.name, fit::RootMinuitCommand.name, personalPlayback.Applet.name, ParameterSet.name, PixelDCSObject< T >::Item.name, production_tasks.Task.name, PFTauMVAInputDiscriminantTranslator::DiscriminantInfo.name, DQMRivetClient::LumiOption.name, MagCylinder.name, analyzer.Analyzer.name, CharmTagger::MVAVar.name, HcalForwardLibWriter::FileHandle.name, TriggerOutputBranches::NamedBranchPtr.name, PrintSensitive.name, ParSet.name, TableOutputBranches::NamedBranchPtr.name, DQMRivetClient::ScaleFactorOption.name, EgHLTOfflineSummaryClient::SumHistBinData.name, RHStopTracer::StopPoint.name, PhysicsTools::Source.name, cond::persistency::GTProxyData.name, core.autovars.NTupleObjectType.name, SingleObjectCondition.name, MyWatcher.name, edm::PathTimingSummary.name, nanoaod::MergeableCounterTable::VectorColumn< T >.name, cond::TimeTypeSpecs.name, lumi::TriggerInfo.name, dirstructure.Weighted.name, alignment.Alignment.name, PrintMaterialBudgetInfo.name, edm::PathSummary.name, PixelEndcapLinkMaker::Item.name, perftools::EdmEventSize::BranchRecord.name, cond::persistency::GLOBAL_TAG_MAP::GLOBAL_TAG_NAME.name, DQMGenericClient::EfficOption.name, FWTableViewManager::TableEntry.name, cond::persistency::GLOBAL_TAG_MAP::RECORD.name, PixelBarrelLinkMaker::Item.name, cms::DDAlgoArguments.name, EcalLogicID.name, cond::persistency::GLOBAL_TAG_MAP::LABEL.name, validateAlignments.ParallelMergeJob.name, MEtoEDM< T >::MEtoEDMObject.name, cond::persistency::GLOBAL_TAG_MAP::TAG_NAME.name, ExpressionHisto< T >.name, XMLProcessor::_loaderBaseConfig.name, python.rootplot.utilities.Hist2D.name, cond::persistency::PAYLOAD::HASH.name, cond::persistency::PAYLOAD::OBJECT_TYPE.name, CaloTrkProcessing::Detector.name, cond::persistency::PAYLOAD::DATA.name, genericValidation.GenericValidation.name, TreeCrawler.Package.name, PrintGeomInfoAction.name, cond::persistency::PAYLOAD::STREAMER_INFO.name, cond::persistency::PAYLOAD::VERSION.name, options.ConnectionHLTMenu.name, MagGeoBuilderFromDDD::volumeHandle.name, cond::persistency::PAYLOAD::INSERTION_TIME.name, DisplayManager.DisplayManager.name, MVAVar.name, DQMGenericClient::ProfileOption.name, dqmoffline::l1t::HistDefinition.name, FWLite.WorkingPoints.name, DQMGenericClient::NormOption.name, BTagProbabilityToDiscriminator::Discriminator.name, emtf::Node.name, h4DSegm.name, FastHFShowerLibrary.name, PhysicsTools::Calibration::Variable.name, PhysicsTools::Variable::Value.name, core.TriggerMatchAnalyzer.TriggerMatchAnalyzer.name, DQMGenericClient::CDOption.name, CounterChecker.name, TriggerObjectTableProducer::SelectedObject.name, cond::TagInfo_t.name, ProcTMVA::Method.name, looper.Looper.name, TreeSaver::Var.name, DQMGenericClient::NoFlowOption.name, python.rootplot.tree2hists.Plot.name, cond::persistency::IOV::TAG_NAME.name, EDMtoMEConverter.name, cond::persistency::IOV::SINCE.name, TrackerSectorStruct.name, Mapper::definition< ScannerT >.name, cond::persistency::IOV::PAYLOAD_HASH.name, PhysicsTools::TreeReader::Value.name, classes.MonitorData.name, cond::persistency::IOV::INSERTION_TIME.name, HistogramManager.name, PhysicsTools::TrainProcessor.name, MuonGeometrySanityCheckPoint.name, classes.OutputData.name, PhysicsTools::MVAModuleHelper< Record, Object, Filler >::Value.name, PhysicsTools::ProcessRegistry< Base_t, CalibBase_t, Parent_t >.name, PhysicsTools::MVATrainer.name, options.HLTProcessOptions.name, h2DSegm.name, python.rootplot.utilities.Hist.name, core.TriggerBitAnalyzer.TriggerBitAnalyzer.name, nanoaod::FlatTable::Column.name, geometry.Structure.name, config.Analyzer.name, core.autovars.NTupleSubObject.name, DQMNet::WaitObject.name, AlpgenParameterName.name, NanoAODDQM::SelGroupConfig.name, SiStripMonitorDigi.name, core.autovars.NTupleObject.name, public_plots_tools.ColorScheme.name, config.Service.name, PhysicsTools::Variable.name, cond::persistency::TAG_LOG::TAG_NAME.name, cond::persistency::TAG_LOG::EVENT_TIME.name, cond::persistency::TAG_LOG::USER_NAME.name, cond::persistency::TAG_LOG::HOST_NAME.name, cond::persistency::TAG_LOG::COMMAND.name, cond::persistency::TAG_LOG::ACTION.name, PhysicsTools::TrainerMonitoring::Object.name, cond::persistency::TAG_LOG::USER_TEXT.name, core.autovars.NTupleCollection.name, BPHRecoBuilder::BPHRecoSource.name, BPHRecoBuilder::BPHCompSource.name, personalPlayback.FrameworkJob.name, plotscripts.SawTeethFunction.name, python.rootplot.utilities.RootFile.name, crabFunctions.CrabTask.name, hTMaxCell.name, cscdqm::ParHistoDef.name, BeautifulSoup.Tag.name, Id.name, SummaryOutputProducer::GenericSummary.name, and BeautifulSoup.SoupStrainer.name.

Referenced by dirstructure.Directory.__create_on_disk(), and dirstructure.Directory.get_summary_chart_name().

  def __get_full_path(self):
    #print "Mother is %s" %self.mother_dir
    if len(self.mother_dir)==0:
      return self.name
    return join(self.mother_dir,self.name)
    

def dirstructure.Directory.__repr__

(

self

)

Definition at line 314 of file dirstructure.py.

References dirstructure.Directory.comparisons, dirstructure.Directory.get_fail_rate(), dirstructure.Directory.get_null_rate(), dirstructure.Directory.get_success_rate(), DEutils< T >.is_empty(), DEcompare< T >.is_empty(), dirstructure.Directory.is_empty(), create_public_lumi_plots.LumiDataBlock.is_empty(), dirstructure.Directory.n_fails, dirstructure.Directory.n_nulls, dirstructure.Directory.n_successes, ElectronMVAID.ElectronMVAID.name, counter.Counter.name, average.Average.name, histograms.Histograms.name, AlignableObjectId::entry.name, FWLite.ElectronMVAID.name, cond::persistency::RUN_INFO::RUN_NUMBER.name, cond::persistency::TAG::NAME.name, TmModule.name, cond::persistency::GLOBAL_TAG::NAME.name, core.autovars.NTupleVariable.name, cond::persistency::RUN_INFO::START_TIME.name, cond::persistency::TAG::TIME_TYPE.name, cond::persistency::GLOBAL_TAG::VALIDITY.name, cond::persistency::RUN_INFO::END_TIME.name, cond::persistency::TAG::OBJECT_TYPE.name, cond::persistency::GLOBAL_TAG::DESCRIPTION.name, cond::persistency::TAG::SYNCHRONIZATION.name, cond::persistency::GLOBAL_TAG::RELEASE.name, cond::persistency::GLOBAL_TAG::SNAPSHOT_TIME.name, MEPSet.name, cond::persistency::TAG::END_OF_VALIDITY.name, cond::persistency::GLOBAL_TAG::INSERTION_TIME.name, cond::persistency::GTEditorData.name, cond::persistency::TAG::DESCRIPTION.name, nanoaod::MergeableCounterTable::SingleColumn< T >.name, cond::persistency::TAG::LAST_VALIDATED_TIME.name, cond::persistency::TAG::INSERTION_TIME.name, cond::persistency::TAG::MODIFICATION_TIME.name, FWTGeoRecoGeometry::Info.name, Types._Untracked.name, preexistingValidation.PreexistingValidation.name, dataset.BaseDataset.name, OutputMEPSet.name, SummaryTableOutputBranches::NamedBranchPtr.name, fit::RootMinuitCommand.name, personalPlayback.Applet.name, ParameterSet.name, PixelDCSObject< T >::Item.name, production_tasks.Task.name, PFTauMVAInputDiscriminantTranslator::DiscriminantInfo.name, analyzer.Analyzer.name, DQMRivetClient::LumiOption.name, MagCylinder.name, CharmTagger::MVAVar.name, HcalForwardLibWriter::FileHandle.name, TriggerOutputBranches::NamedBranchPtr.name, PrintSensitive.name, ParSet.name, TableOutputBranches::NamedBranchPtr.name, DQMRivetClient::ScaleFactorOption.name, RHStopTracer::StopPoint.name, EgHLTOfflineSummaryClient::SumHistBinData.name, PhysicsTools::Source.name, SingleObjectCondition.name, cond::persistency::GTProxyData.name, core.autovars.NTupleObjectType.name, MyWatcher.name, edm::PathTimingSummary.name, nanoaod::MergeableCounterTable::VectorColumn< T >.name, dirstructure.Weighted.name, cond::TimeTypeSpecs.name, lumi::TriggerInfo.name, alignment.Alignment.name, PrintMaterialBudgetInfo.name, edm::PathSummary.name, cond::persistency::GLOBAL_TAG_MAP::GLOBAL_TAG_NAME.name, PixelEndcapLinkMaker::Item.name, perftools::EdmEventSize::BranchRecord.name, DQMGenericClient::EfficOption.name, FWTableViewManager::TableEntry.name, cond::persistency::GLOBAL_TAG_MAP::RECORD.name, PixelBarrelLinkMaker::Item.name, cms::DDAlgoArguments.name, EcalLogicID.name, cond::persistency::GLOBAL_TAG_MAP::LABEL.name, validateAlignments.ParallelMergeJob.name, cond::persistency::GLOBAL_TAG_MAP::TAG_NAME.name, MEtoEDM< T >::MEtoEDMObject.name, ExpressionHisto< T >.name, XMLProcessor::_loaderBaseConfig.name, python.rootplot.utilities.Hist2D.name, cond::persistency::PAYLOAD::HASH.name, cond::persistency::PAYLOAD::OBJECT_TYPE.name, cond::persistency::PAYLOAD::DATA.name, TreeCrawler.Package.name, genericValidation.GenericValidation.name, CaloTrkProcessing::Detector.name, cond::persistency::PAYLOAD::STREAMER_INFO.name, PrintGeomInfoAction.name, cond::persistency::PAYLOAD::VERSION.name, options.ConnectionHLTMenu.name, MagGeoBuilderFromDDD::volumeHandle.name, cond::persistency::PAYLOAD::INSERTION_TIME.name, DisplayManager.DisplayManager.name, DQMGenericClient::ProfileOption.name, MVAVar.name, dqmoffline::l1t::HistDefinition.name, FWLite.WorkingPoints.name, DQMGenericClient::NormOption.name, BTagProbabilityToDiscriminator::Discriminator.name, emtf::Node.name, h4DSegm.name, PhysicsTools::Calibration::Variable.name, FastHFShowerLibrary.name, PhysicsTools::Variable::Value.name, core.TriggerMatchAnalyzer.TriggerMatchAnalyzer.name, DQMGenericClient::CDOption.name, TriggerObjectTableProducer::SelectedObject.name, CounterChecker.name, cond::TagInfo_t.name, ProcTMVA::Method.name, looper.Looper.name, TreeSaver::Var.name, DQMGenericClient::NoFlowOption.name, python.rootplot.tree2hists.Plot.name, cond::persistency::IOV::TAG_NAME.name, TrackerSectorStruct.name, EDMtoMEConverter.name, cond::persistency::IOV::SINCE.name, Mapper::definition< ScannerT >.name, cond::persistency::IOV::PAYLOAD_HASH.name, PhysicsTools::TreeReader::Value.name, classes.MonitorData.name, cond::persistency::IOV::INSERTION_TIME.name, HistogramManager.name, MuonGeometrySanityCheckPoint.name, PhysicsTools::TrainProcessor.name, classes.OutputData.name, PhysicsTools::MVAModuleHelper< Record, Object, Filler >::Value.name, PhysicsTools::ProcessRegistry< Base_t, CalibBase_t, Parent_t >.name, PhysicsTools::MVATrainer.name, options.HLTProcessOptions.name, h2DSegm.name, python.rootplot.utilities.Hist.name, core.TriggerBitAnalyzer.TriggerBitAnalyzer.name, nanoaod::FlatTable::Column.name, geometry.Structure.name, config.Analyzer.name, core.autovars.NTupleSubObject.name, DQMNet::WaitObject.name, AlpgenParameterName.name, SiStripMonitorDigi.name, NanoAODDQM::SelGroupConfig.name, core.autovars.NTupleObject.name, config.Service.name, public_plots_tools.ColorScheme.name, PhysicsTools::Variable.name, cond::persistency::TAG_LOG::TAG_NAME.name, cond::persistency::TAG_LOG::EVENT_TIME.name, cond::persistency::TAG_LOG::USER_NAME.name, cond::persistency::TAG_LOG::HOST_NAME.name, cond::persistency::TAG_LOG::COMMAND.name, cond::persistency::TAG_LOG::ACTION.name, PhysicsTools::TrainerMonitoring::Object.name, cond::persistency::TAG_LOG::USER_TEXT.name, core.autovars.NTupleCollection.name, BPHRecoBuilder::BPHRecoSource.name, BPHRecoBuilder::BPHCompSource.name, personalPlayback.FrameworkJob.name, plotscripts.SawTeethFunction.name, python.rootplot.utilities.RootFile.name, crabFunctions.CrabTask.name, hTMaxCell.name, cscdqm::ParHistoDef.name, BeautifulSoup.Tag.name, Id.name, SummaryOutputProducer::GenericSummary.name, BeautifulSoup.SoupStrainer.name, and dirstructure.Directory.subdirs.

  def __repr__(self):
    if self.is_empty():
      return "%s seems to be empty. Please check!" %self.name
    content="%s , Rates: Success %.2f%%(%s) - Fail %.2f%%(%s) - Null %.2f%%(%s)\n" %(self.name,self.get_success_rate(),self.n_successes,self.get_fail_rate(),self.n_fails,self.get_null_rate(),self.n_nulls)   
    for subdir in self.subdirs:
      content+=" %s\n" % subdir
    for comp in self.comparisons:
      content+=" %s\n" % comp
    return content
    
#-------------------------------------------------------------------------------

def dirstructure.Directory.calcStats	(		self,
			make_pie = True
	)

Walk all subdirs and calculate weight,fails and successes.
Moreove propagate the sample and releases names.

Definition at line 94 of file dirstructure.py.

References dirstructure.Directory.comparisons, dirstructure.Directory.different_histograms, dirstructure.Directory.filename1, dirstructure.Directory.filename2, dirstructure.Directory.full_path, join(), dqm::DQMFileSaverOnline::SnapshotFiles.meta, dirstructure.Directory.meta, dirstructure.Directory.mother_dir, dirstructure.Directory.n_comp_fails, dirstructure.Directory.n_comp_nulls, dirstructure.Directory.n_comp_skiped, dirstructure.Directory.n_comp_successes, dirstructure.Directory.n_fails, dirstructure.Directory.n_missing_objs, dirstructure.Directory.n_nulls, dirstructure.Directory.n_skiped, dirstructure.Directory.n_successes, ElectronMVAID.ElectronMVAID.name, counter.Counter.name, average.Average.name, AlignableObjectId::entry.name, histograms.Histograms.name, FWLite.ElectronMVAID.name, cond::persistency::TAG::NAME.name, core.autovars.NTupleVariable.name, cond::persistency::RUN_INFO::RUN_NUMBER.name, TmModule.name, cond::persistency::GLOBAL_TAG::NAME.name, cond::persistency::TAG::TIME_TYPE.name, cond::persistency::RUN_INFO::START_TIME.name, cond::persistency::GLOBAL_TAG::VALIDITY.name, cond::persistency::GLOBAL_TAG::DESCRIPTION.name, cond::persistency::TAG::OBJECT_TYPE.name, cond::persistency::RUN_INFO::END_TIME.name, cond::persistency::GLOBAL_TAG::RELEASE.name, cond::persistency::TAG::SYNCHRONIZATION.name, cond::persistency::GLOBAL_TAG::SNAPSHOT_TIME.name, cond::persistency::TAG::END_OF_VALIDITY.name, MEPSet.name, cond::persistency::GLOBAL_TAG::INSERTION_TIME.name, cond::persistency::TAG::DESCRIPTION.name, cond::persistency::GTEditorData.name, cond::persistency::TAG::LAST_VALIDATED_TIME.name, nanoaod::MergeableCounterTable::SingleColumn< T >.name, cond::persistency::TAG::INSERTION_TIME.name, FWTGeoRecoGeometry::Info.name, Types._Untracked.name, cond::persistency::TAG::MODIFICATION_TIME.name, preexistingValidation.PreexistingValidation.name, dataset.BaseDataset.name, OutputMEPSet.name, SummaryTableOutputBranches::NamedBranchPtr.name, personalPlayback.Applet.name, ParameterSet.name, fit::RootMinuitCommand.name, PixelDCSObject< T >::Item.name, production_tasks.Task.name, PFTauMVAInputDiscriminantTranslator::DiscriminantInfo.name, DQMRivetClient::LumiOption.name, MagCylinder.name, analyzer.Analyzer.name, CharmTagger::MVAVar.name, HcalForwardLibWriter::FileHandle.name, TriggerOutputBranches::NamedBranchPtr.name, PrintSensitive.name, ParSet.name, TableOutputBranches::NamedBranchPtr.name, DQMRivetClient::ScaleFactorOption.name, EgHLTOfflineSummaryClient::SumHistBinData.name, RHStopTracer::StopPoint.name, core.autovars.NTupleObjectType.name, PhysicsTools::Source.name, SingleObjectCondition.name, cond::persistency::GTProxyData.name, MyWatcher.name, edm::PathTimingSummary.name, nanoaod::MergeableCounterTable::VectorColumn< T >.name, dirstructure.Weighted.name, alignment.Alignment.name, cond::TimeTypeSpecs.name, lumi::TriggerInfo.name, edm::PathSummary.name, PrintMaterialBudgetInfo.name, cond::persistency::GLOBAL_TAG_MAP::GLOBAL_TAG_NAME.name, perftools::EdmEventSize::BranchRecord.name, PixelEndcapLinkMaker::Item.name, FWTableViewManager::TableEntry.name, DQMGenericClient::EfficOption.name, cond::persistency::GLOBAL_TAG_MAP::RECORD.name, PixelBarrelLinkMaker::Item.name, EcalLogicID.name, cms::DDAlgoArguments.name, cond::persistency::GLOBAL_TAG_MAP::LABEL.name, validateAlignments.ParallelMergeJob.name, cond::persistency::GLOBAL_TAG_MAP::TAG_NAME.name, MEtoEDM< T >::MEtoEDMObject.name, ExpressionHisto< T >.name, XMLProcessor::_loaderBaseConfig.name, cond::persistency::PAYLOAD::HASH.name, python.rootplot.utilities.Hist2D.name, cond::persistency::PAYLOAD::OBJECT_TYPE.name, TreeCrawler.Package.name, cond::persistency::PAYLOAD::DATA.name, genericValidation.GenericValidation.name, CaloTrkProcessing::Detector.name, cond::persistency::PAYLOAD::STREAMER_INFO.name, PrintGeomInfoAction.name, options.ConnectionHLTMenu.name, MagGeoBuilderFromDDD::volumeHandle.name, cond::persistency::PAYLOAD::VERSION.name, cond::persistency::PAYLOAD::INSERTION_TIME.name, DisplayManager.DisplayManager.name, MVAVar.name, DQMGenericClient::ProfileOption.name, dqmoffline::l1t::HistDefinition.name, FWLite.WorkingPoints.name, DQMGenericClient::NormOption.name, emtf::Node.name, BTagProbabilityToDiscriminator::Discriminator.name, h4DSegm.name, FastHFShowerLibrary.name, core.TriggerMatchAnalyzer.TriggerMatchAnalyzer.name, PhysicsTools::Variable::Value.name, PhysicsTools::Calibration::Variable.name, DQMGenericClient::CDOption.name, CounterChecker.name, TriggerObjectTableProducer::SelectedObject.name, cond::TagInfo_t.name, ProcTMVA::Method.name, looper.Looper.name, TreeSaver::Var.name, DQMGenericClient::NoFlowOption.name, python.rootplot.tree2hists.Plot.name, cond::persistency::IOV::TAG_NAME.name, TrackerSectorStruct.name, cond::persistency::IOV::SINCE.name, EDMtoMEConverter.name, cond::persistency::IOV::PAYLOAD_HASH.name, Mapper::definition< ScannerT >.name, classes.MonitorData.name, PhysicsTools::TreeReader::Value.name, cond::persistency::IOV::INSERTION_TIME.name, HistogramManager.name, PhysicsTools::TrainProcessor.name, MuonGeometrySanityCheckPoint.name, classes.OutputData.name, PhysicsTools::MVAModuleHelper< Record, Object, Filler >::Value.name, PhysicsTools::ProcessRegistry< Base_t, CalibBase_t, Parent_t >.name, options.HLTProcessOptions.name, PhysicsTools::MVATrainer.name, h2DSegm.name, core.TriggerBitAnalyzer.TriggerBitAnalyzer.name, python.rootplot.utilities.Hist.name, nanoaod::FlatTable::Column.name, config.Analyzer.name, geometry.Structure.name, core.autovars.NTupleSubObject.name, DQMNet::WaitObject.name, AlpgenParameterName.name, NanoAODDQM::SelGroupConfig.name, SiStripMonitorDigi.name, core.autovars.NTupleObject.name, public_plots_tools.ColorScheme.name, config.Service.name, PhysicsTools::Variable.name, cond::persistency::TAG_LOG::TAG_NAME.name, cond::persistency::TAG_LOG::EVENT_TIME.name, cond::persistency::TAG_LOG::USER_NAME.name, cond::persistency::TAG_LOG::HOST_NAME.name, cond::persistency::TAG_LOG::COMMAND.name, cond::persistency::TAG_LOG::ACTION.name, cond::persistency::TAG_LOG::USER_TEXT.name, PhysicsTools::TrainerMonitoring::Object.name, core.autovars.NTupleCollection.name, BPHRecoBuilder::BPHRecoSource.name, BPHRecoBuilder::BPHCompSource.name, personalPlayback.FrameworkJob.name, plotscripts.SawTeethFunction.name, python.rootplot.utilities.RootFile.name, crabFunctions.CrabTask.name, hTMaxCell.name, cscdqm::ParHistoDef.name, BeautifulSoup.Tag.name, Id.name, SummaryOutputProducer::GenericSummary.name, BeautifulSoup.SoupStrainer.name, edm.print(), python.rootplot.root2matplotlib.replace(), dirstructure.Directory.stats_calculated, dirstructure.Directory.subdirs, RPCDQMObject::DQMObjectItem.weight, JetAnaPythia< Jet >.weight, TtFullLepKinSolver::NeutrinoSolution.weight, ZIterativeAlgorithmWithFit::ZIterativeAlgorithmWithFitPlots.weight, SeedCandidate.weight, materialEffect::Effect.weight, dirstructure.Weighted.weight, ExtendedPerigeeTrajectoryError.weight, L1Analysis::L1AnalysisGeneratorDataFormat.weight, PrimaryVertexMonitor.weight, MuIsolatorResultProducer< BT >::DepositConf.weight, DTTrigTest.weight, FastLineRecognition::Cluster.weight, classes.MonitorData.weight, TreeSaver.weight, L1PromptAnalysis.weight, ProcTMVA.weight, HLTBitAnalyzer.weight, HGCalImagingAlgo::Hexel.weight, and areaInfo.weight.

Referenced by dirstructure.Directory.print_report().

  def calcStats(self,make_pie=True):
    '''Walk all subdirs and calculate weight,fails and successes.
    Moreove propagate the sample and releases names.
    '''
    if self.stats_calculated:
      return 0
    
    self.n_fails=0
    self.n_successes=0
    self.n_nulls=0
    self.n_comp_fails=0
    self.n_comp_successes=0
    self.n_comp_nulls=0  
    self.weight=0
    
    self.n_skiped = 0
    self.n_comp_skiped = 0
    self.n_missing_objs = len(self.different_histograms['file1'])+len(self.different_histograms['file2'])
    if self.n_missing_objs != 0:
      print("    [*] Missing in %s: %s" %(self.filename1, self.different_histograms['file1']))
      print("    [*] Missing in %s: %s" %(self.filename2, self.different_histograms['file2']))
    # clean from empty dirs    
    self.subdirs = [subdir for subdir in self.subdirs if not subdir.is_empty()]    
    
    for comp in self.comparisons:
      if comp.status == SKIPED: #in case its in black list & skiped 
          self.n_skiped += 1
          self.n_comp_skiped += 1
          self.weight+=1
      else: #else original code -> to check for Fails and Successes
          self.rank_histo.Fill(comp.rank)
          self.weight+=1
          if comp.status == FAIL:
              self.n_fails+=1
              self.n_comp_fails+=1
          elif comp.status == SUCCESS:
              self.n_successes+=1
              self.n_comp_successes+=1
          else:
              self.n_nulls+=1
              self.n_comp_nulls+=1

    for subdir in self.subdirs:
      subdir.mother_dir=join(self.mother_dir,self.name)
      subdir.full_path = join(self.mother_dir,self.name).replace("/Run summary","")
      subdir.calcStats(make_pie)
      subdir.meta=self.meta 
      self.weight+=subdir.weight
      self.n_fails+=subdir.n_fails
      self.n_successes+=subdir.n_successes
      self.n_nulls+=subdir.n_nulls
      
      self.n_skiped+=subdir.n_skiped
      self.n_missing_objs += subdir.n_missing_objs
      
      self.rank_histo.Add(subdir.rank_histo)

    self.stats_calculated=True
    self.full_path = join(self.mother_dir,self.name).replace("/Run summary","")
    #if make_pie:
      #self.__create_pie_image()

def dirstructure.Directory.get_fail_rate

(

self

)

Definition at line 206 of file dirstructure.py.

References dirstructure.Directory.n_fails, RPCDQMObject::DQMObjectItem.weight, JetAnaPythia< Jet >.weight, TtFullLepKinSolver::NeutrinoSolution.weight, ZIterativeAlgorithmWithFit::ZIterativeAlgorithmWithFitPlots.weight, SeedCandidate.weight, materialEffect::Effect.weight, dirstructure.Weighted.weight, ExtendedPerigeeTrajectoryError.weight, L1Analysis::L1AnalysisGeneratorDataFormat.weight, PrimaryVertexMonitor.weight, MuIsolatorResultProducer< BT >::DepositConf.weight, DTTrigTest.weight, FastLineRecognition::Cluster.weight, classes.MonitorData.weight, L1PromptAnalysis.weight, TreeSaver.weight, ProcTMVA.weight, HLTBitAnalyzer.weight, HGCalImagingAlgo::Hexel.weight, and areaInfo.weight.

Referenced by dirstructure.Directory.__create_pie_image(), dirstructure.Directory.__repr__(), dirstructure.Directory.get_summary_chart_ajax(), and dirstructure.Directory.print_report().

  def get_fail_rate(self):
    if self.weight == 0:return 0
    return 100.*self.n_fails/self.weight
    

def dirstructure.Directory.get_null_rate

(

self

)

Definition at line 214 of file dirstructure.py.

References dirstructure.Directory.n_nulls, RPCDQMObject::DQMObjectItem.weight, JetAnaPythia< Jet >.weight, TtFullLepKinSolver::NeutrinoSolution.weight, ZIterativeAlgorithmWithFit::ZIterativeAlgorithmWithFitPlots.weight, SeedCandidate.weight, materialEffect::Effect.weight, dirstructure.Weighted.weight, ExtendedPerigeeTrajectoryError.weight, L1Analysis::L1AnalysisGeneratorDataFormat.weight, PrimaryVertexMonitor.weight, MuIsolatorResultProducer< BT >::DepositConf.weight, DTTrigTest.weight, FastLineRecognition::Cluster.weight, classes.MonitorData.weight, L1PromptAnalysis.weight, TreeSaver.weight, ProcTMVA.weight, HLTBitAnalyzer.weight, HGCalImagingAlgo::Hexel.weight, and areaInfo.weight.

Referenced by dirstructure.Directory.__create_pie_image(), dirstructure.Directory.__repr__(), dirstructure.Directory.get_summary_chart_ajax(), and dirstructure.Directory.print_report().

  def get_null_rate(self):
    if self.weight == 0:return 0    
    return 100.*self.n_nulls/self.weight

def dirstructure.Directory.get_skiped_rate

(

self

)

Definition at line 203 of file dirstructure.py.

References dirstructure.Directory.n_skiped, RPCDQMObject::DQMObjectItem.weight, JetAnaPythia< Jet >.weight, TtFullLepKinSolver::NeutrinoSolution.weight, ZIterativeAlgorithmWithFit::ZIterativeAlgorithmWithFitPlots.weight, SeedCandidate.weight, materialEffect::Effect.weight, dirstructure.Weighted.weight, ExtendedPerigeeTrajectoryError.weight, L1Analysis::L1AnalysisGeneratorDataFormat.weight, PrimaryVertexMonitor.weight, MuIsolatorResultProducer< BT >::DepositConf.weight, DTTrigTest.weight, FastLineRecognition::Cluster.weight, classes.MonitorData.weight, L1PromptAnalysis.weight, TreeSaver.weight, ProcTMVA.weight, HLTBitAnalyzer.weight, HGCalImagingAlgo::Hexel.weight, and areaInfo.weight.

Referenced by dirstructure.Directory.__create_pie_image(), dirstructure.Directory.get_summary_chart_ajax(), and dirstructure.Directory.print_report().

  def get_skiped_rate(self):
    if self.weight == 0: return 0
    return 100.*self.n_skiped/self.weight

def dirstructure.Directory.get_subdirs_dict

(

self

)

Definition at line 156 of file dirstructure.py.

References dirstructure.Directory.subdirs.

  def get_subdirs_dict(self):
    subdirdict={}
    for subdir in self.subdirs:
      subdirdict[subdir.name]=subdir
    return subdirdict

def dirstructure.Directory.get_subdirs_names

(

self

)

Definition at line 162 of file dirstructure.py.

References dirstructure.Directory.subdirs.

  def get_subdirs_names(self):
    subdirnames=[]
    for subdir in self.subdirs:
      subdirnames.append(subdir.name)
    return subdirnames

def dirstructure.Directory.get_success_rate

(

self

)

Definition at line 210 of file dirstructure.py.

References dirstructure.Directory.n_successes, RPCDQMObject::DQMObjectItem.weight, JetAnaPythia< Jet >.weight, TtFullLepKinSolver::NeutrinoSolution.weight, ZIterativeAlgorithmWithFit::ZIterativeAlgorithmWithFitPlots.weight, SeedCandidate.weight, materialEffect::Effect.weight, dirstructure.Weighted.weight, ExtendedPerigeeTrajectoryError.weight, L1Analysis::L1AnalysisGeneratorDataFormat.weight, PrimaryVertexMonitor.weight, MuIsolatorResultProducer< BT >::DepositConf.weight, DTTrigTest.weight, FastLineRecognition::Cluster.weight, classes.MonitorData.weight, L1PromptAnalysis.weight, TreeSaver.weight, ProcTMVA.weight, HLTBitAnalyzer.weight, HGCalImagingAlgo::Hexel.weight, and areaInfo.weight.

Referenced by dirstructure.Directory.__create_pie_image(), dirstructure.Directory.__repr__(), dirstructure.Directory.get_summary_chart_ajax(), and dirstructure.Directory.print_report().

  def get_success_rate(self):
    if self.weight == 0:return 1    
    return 100.*self.n_successes/self.weight
    

def dirstructure.Directory.get_summary_chart_ajax	(		self,
			w = 400,
			h = 300
	)

Emit the ajax to build a pie chart using google apis...

Definition at line 168 of file dirstructure.py.

References dirstructure.Directory.get_fail_rate(), dirstructure.Directory.get_null_rate(), dirstructure.Directory.get_skiped_rate(), and dirstructure.Directory.get_success_rate().

  def get_summary_chart_ajax(self,w=400,h=300):
    """Emit the ajax to build a pie chart using google apis...
    """
    url = "https://chart.googleapis.com/chart?"
    url+= "cht=p3" # Select the 3d chart
    #url+= "&chl=Success|Null|Fail" # give labels
    url+= "&chco=00FF00|FFFF00|FF0000|7A7A7A" # give colours to labels
    url+= "&chs=%sx%s" %(w,h)
    #url+= "&chtt=%s" %self.name
    url+= "&chd=t:%.2f,%.2f,%.2f,%.2f"%(self.get_success_rate(),self.get_null_rate(),self.get_fail_rate(),self.get_skiped_rate())
    
    return url

def dirstructure.Directory.get_summary_chart_name

(

self

)

Definition at line 231 of file dirstructure.py.

References dirstructure.Directory.__get_full_path(), and join().

Referenced by dirstructure.Directory.__create_pie_image().

  def get_summary_chart_name(self):
    return join(self.__get_full_path(),"summary_chart.png") 

def dirstructure.Directory.is_empty

(

self

)

Definition at line 89 of file dirstructure.py.

References dirstructure.Directory.comparisons, and dirstructure.Directory.subdirs.

Referenced by dirstructure.Directory.__repr__(), and utils_v2.StatisticalTest.do_test().

  def is_empty(self):
    if len(self.subdirs)==0 and len(self.comparisons)==0:
      return True
    return False
  

def dirstructure.Directory.print_report	(		self,
			indent = "",
			verbose = False
	)

Definition at line 181 of file dirstructure.py.

References dirstructure.Directory.calcStats(), dirstructure.Directory.comparisons, dirstructure.Directory.get_fail_rate(), dirstructure.Directory.get_null_rate(), dirstructure.Directory.get_skiped_rate(), dirstructure.Directory.get_success_rate(), dirstructure.Directory.mother_dir, dirstructure.Directory.n_fails, dirstructure.Directory.n_missing_objs, dirstructure.Directory.n_nulls, dirstructure.Directory.n_skiped, dirstructure.Directory.n_successes, ElectronMVAID.ElectronMVAID.name, counter.Counter.name, average.Average.name, histograms.Histograms.name, AlignableObjectId::entry.name, FWLite.ElectronMVAID.name, cond::persistency::RUN_INFO::RUN_NUMBER.name, cond::persistency::TAG::NAME.name, TmModule.name, cond::persistency::GLOBAL_TAG::NAME.name, core.autovars.NTupleVariable.name, cond::persistency::RUN_INFO::START_TIME.name, cond::persistency::TAG::TIME_TYPE.name, cond::persistency::GLOBAL_TAG::VALIDITY.name, cond::persistency::RUN_INFO::END_TIME.name, cond::persistency::TAG::OBJECT_TYPE.name, cond::persistency::GLOBAL_TAG::DESCRIPTION.name, cond::persistency::TAG::SYNCHRONIZATION.name, cond::persistency::GLOBAL_TAG::RELEASE.name, cond::persistency::GLOBAL_TAG::SNAPSHOT_TIME.name, MEPSet.name, cond::persistency::TAG::END_OF_VALIDITY.name, cond::persistency::GLOBAL_TAG::INSERTION_TIME.name, cond::persistency::TAG::DESCRIPTION.name, cond::persistency::GTEditorData.name, nanoaod::MergeableCounterTable::SingleColumn< T >.name, cond::persistency::TAG::LAST_VALIDATED_TIME.name, cond::persistency::TAG::INSERTION_TIME.name, cond::persistency::TAG::MODIFICATION_TIME.name, FWTGeoRecoGeometry::Info.name, Types._Untracked.name, preexistingValidation.PreexistingValidation.name, dataset.BaseDataset.name, OutputMEPSet.name, SummaryTableOutputBranches::NamedBranchPtr.name, fit::RootMinuitCommand.name, personalPlayback.Applet.name, ParameterSet.name, PixelDCSObject< T >::Item.name, production_tasks.Task.name, PFTauMVAInputDiscriminantTranslator::DiscriminantInfo.name, analyzer.Analyzer.name, DQMRivetClient::LumiOption.name, MagCylinder.name, CharmTagger::MVAVar.name, HcalForwardLibWriter::FileHandle.name, TriggerOutputBranches::NamedBranchPtr.name, PrintSensitive.name, ParSet.name, TableOutputBranches::NamedBranchPtr.name, DQMRivetClient::ScaleFactorOption.name, RHStopTracer::StopPoint.name, EgHLTOfflineSummaryClient::SumHistBinData.name, PhysicsTools::Source.name, SingleObjectCondition.name, cond::persistency::GTProxyData.name, core.autovars.NTupleObjectType.name, MyWatcher.name, edm::PathTimingSummary.name, nanoaod::MergeableCounterTable::VectorColumn< T >.name, cond::TimeTypeSpecs.name, lumi::TriggerInfo.name, dirstructure.Weighted.name, alignment.Alignment.name, PrintMaterialBudgetInfo.name, edm::PathSummary.name, cond::persistency::GLOBAL_TAG_MAP::GLOBAL_TAG_NAME.name, PixelEndcapLinkMaker::Item.name, perftools::EdmEventSize::BranchRecord.name, DQMGenericClient::EfficOption.name, FWTableViewManager::TableEntry.name, cond::persistency::GLOBAL_TAG_MAP::RECORD.name, PixelBarrelLinkMaker::Item.name, cms::DDAlgoArguments.name, EcalLogicID.name, cond::persistency::GLOBAL_TAG_MAP::LABEL.name, validateAlignments.ParallelMergeJob.name, MEtoEDM< T >::MEtoEDMObject.name, cond::persistency::GLOBAL_TAG_MAP::TAG_NAME.name, ExpressionHisto< T >.name, XMLProcessor::_loaderBaseConfig.name, python.rootplot.utilities.Hist2D.name, cond::persistency::PAYLOAD::HASH.name, cond::persistency::PAYLOAD::OBJECT_TYPE.name, cond::persistency::PAYLOAD::DATA.name, CaloTrkProcessing::Detector.name, TreeCrawler.Package.name, genericValidation.GenericValidation.name, PrintGeomInfoAction.name, cond::persistency::PAYLOAD::STREAMER_INFO.name, cond::persistency::PAYLOAD::VERSION.name, options.ConnectionHLTMenu.name, MagGeoBuilderFromDDD::volumeHandle.name, cond::persistency::PAYLOAD::INSERTION_TIME.name, DisplayManager.DisplayManager.name, MVAVar.name, DQMGenericClient::ProfileOption.name, dqmoffline::l1t::HistDefinition.name, FWLite.WorkingPoints.name, DQMGenericClient::NormOption.name, BTagProbabilityToDiscriminator::Discriminator.name, emtf::Node.name, h4DSegm.name, PhysicsTools::Calibration::Variable.name, PhysicsTools::Variable::Value.name, FastHFShowerLibrary.name, core.TriggerMatchAnalyzer.TriggerMatchAnalyzer.name, DQMGenericClient::CDOption.name, TriggerObjectTableProducer::SelectedObject.name, CounterChecker.name, cond::TagInfo_t.name, ProcTMVA::Method.name, looper.Looper.name, TreeSaver::Var.name, DQMGenericClient::NoFlowOption.name, python.rootplot.tree2hists.Plot.name, cond::persistency::IOV::TAG_NAME.name, TrackerSectorStruct.name, EDMtoMEConverter.name, cond::persistency::IOV::SINCE.name, Mapper::definition< ScannerT >.name, cond::persistency::IOV::PAYLOAD_HASH.name, PhysicsTools::TreeReader::Value.name, classes.MonitorData.name, cond::persistency::IOV::INSERTION_TIME.name, HistogramManager.name, PhysicsTools::TrainProcessor.name, MuonGeometrySanityCheckPoint.name, classes.OutputData.name, PhysicsTools::MVAModuleHelper< Record, Object, Filler >::Value.name, PhysicsTools::ProcessRegistry< Base_t, CalibBase_t, Parent_t >.name, options.HLTProcessOptions.name, PhysicsTools::MVATrainer.name, h2DSegm.name, python.rootplot.utilities.Hist.name, core.TriggerBitAnalyzer.TriggerBitAnalyzer.name, nanoaod::FlatTable::Column.name, geometry.Structure.name, config.Analyzer.name, core.autovars.NTupleSubObject.name, DQMNet::WaitObject.name, AlpgenParameterName.name, NanoAODDQM::SelGroupConfig.name, SiStripMonitorDigi.name, core.autovars.NTupleObject.name, config.Service.name, public_plots_tools.ColorScheme.name, PhysicsTools::Variable.name, cond::persistency::TAG_LOG::TAG_NAME.name, cond::persistency::TAG_LOG::EVENT_TIME.name, cond::persistency::TAG_LOG::USER_NAME.name, cond::persistency::TAG_LOG::HOST_NAME.name, cond::persistency::TAG_LOG::COMMAND.name, cond::persistency::TAG_LOG::ACTION.name, PhysicsTools::TrainerMonitoring::Object.name, cond::persistency::TAG_LOG::USER_TEXT.name, core.autovars.NTupleCollection.name, BPHRecoBuilder::BPHRecoSource.name, BPHRecoBuilder::BPHCompSource.name, personalPlayback.FrameworkJob.name, plotscripts.SawTeethFunction.name, python.rootplot.utilities.RootFile.name, crabFunctions.CrabTask.name, hTMaxCell.name, cscdqm::ParHistoDef.name, BeautifulSoup.Tag.name, Id.name, SummaryOutputProducer::GenericSummary.name, BeautifulSoup.SoupStrainer.name, edm.print(), dirstructure.Directory.subdirs, RPCDQMObject::DQMObjectItem.weight, JetAnaPythia< Jet >.weight, TtFullLepKinSolver::NeutrinoSolution.weight, ZIterativeAlgorithmWithFit::ZIterativeAlgorithmWithFitPlots.weight, SeedCandidate.weight, materialEffect::Effect.weight, dirstructure.Weighted.weight, ExtendedPerigeeTrajectoryError.weight, L1Analysis::L1AnalysisGeneratorDataFormat.weight, PrimaryVertexMonitor.weight, MuIsolatorResultProducer< BT >::DepositConf.weight, DTTrigTest.weight, FastLineRecognition::Cluster.weight, classes.MonitorData.weight, L1PromptAnalysis.weight, TreeSaver.weight, ProcTMVA.weight, HLTBitAnalyzer.weight, HGCalImagingAlgo::Hexel.weight, and areaInfo.weight.

  def print_report(self,indent="",verbose=False):
    if len(indent)==0:
      self.calcStats(make_pie=False)
    # print small failure report
    if verbose:
      fail_comps=[comp for comp in self.comparisons if comp.status==FAIL]
      fail_comps=sorted(fail_comps,key=lambda comp:comp.name )    
      if len(fail_comps)>0:
        print(indent+"* %s/%s:" %(self.mother_dir,self.name))
        for comp in fail_comps:
          print(indent+" - %s: %s Test Failed (pval = %s) " %(comp.name,comp.test_name,comp.rank))
      for subdir in self.subdirs:
        subdir.print_report(indent+"  ",verbose)
    
    if len(indent)==0:
      print("\n%s - summary of %s tests:" %(self.name,self.weight))
      print(" o Failiures: %.2f%% (%s/%s)" %(self.get_fail_rate(),self.n_fails,self.weight))
      print(" o Nulls: %.2f%% (%s/%s) " %(self.get_null_rate(),self.n_nulls,self.weight))
      print(" o Successes: %.2f%% (%s/%s) " %(self.get_success_rate(),self.n_successes,self.weight))
      print(" o Skipped: %.2f%% (%s/%s) " %(self.get_skiped_rate(),self.n_skiped,self.weight))
      print(" o Missing objects: %s" %(self.n_missing_objs))

def dirstructure.Directory.prune	(		self,
			expandable_dir
	)

Eliminate from the tree the directory the expandable ones.

Definition at line 270 of file dirstructure.py.

References dirstructure.Directory.comparisons, dirstructure.Directory.n_comp_fails, dirstructure.Directory.n_comp_nulls, dirstructure.Directory.n_comp_skiped, dirstructure.Directory.n_comp_successes, edm.print(), TtEventPartons.prune(), PFMultiDepthClusterizer.prune(), dirstructure.Directory.prune(), Config.Process.prune(), and dirstructure.Directory.subdirs.

Referenced by dirstructure.Directory.prune().

  def prune(self,expandable_dir):
    """Eliminate from the tree the directory the expandable ones.
    """
    #print "pruning %s" %self.name
    exp_index=-1
    counter=0
    for subdir in self.subdirs:      
      # Eliminate any trace of the expandable path in the mother directories
      # for depths higher than 1
      subdir.mother_dir=subdir.mother_dir.replace("/"+expandable_dir,"")
      if subdir.name==expandable_dir:        
        exp_index=counter
      counter+=1
    
    # Did we find an expandable?
    if exp_index>=0:
      exp_dir=self.subdirs[exp_index]
      for subsubdir in exp_dir.subdirs:
        #print "*******",subsubdir.mother_dir,
        subsubdir.mother_dir=subsubdir.mother_dir.replace("/"+expandable_dir,"")
        while "//" in subsubdir.mother_dir:
          print(subsubdir.mother_dir)
          subsubdir.mother_dir=subsubdir.mother_dir.replace("//","/") 
        #print "*******",subsubdir.mother_dir
        self.subdirs.append(subsubdir)
          
        for comp in exp_dir.comparisons:
          comp.mother_dir=comp.mother_dir.replace("/"+expandable_dir,"")        
          while "//" in comp.mother_dir:
              comp.mother_dir
              comp.mother_dir=comp.mother_dir.replace("/")
          if not comp in self.comparisons:  #in case not to  append same comparisons few times
              self.comparisons.append(comp)  # add a comparison
              self.n_comp_fails = exp_dir.n_comp_fails  #copy to-be removed directory
              self.n_comp_nulls = exp_dir.n_comp_nulls  # numbers to parent directory
              self.n_comp_successes = exp_dir.n_comp_successes
              self.n_comp_skiped = exp_dir.n_comp_skiped
        
      del self.subdirs[exp_index]
      self.prune(expandable_dir)
    
    for subdir in self.subdirs:
      subdir.prune(expandable_dir)

Member Data Documentation

dirstructure.Directory.comparisons

Definition at line 65 of file dirstructure.py.

Referenced by dirstructure.Directory.__repr__(), dirstructure.Directory.calcStats(), dirstructure.Directory.is_empty(), dirstructure.Directory.print_report(), and dirstructure.Directory.prune().

dirstructure.Directory.different_histograms

Definition at line 81 of file dirstructure.py.

Referenced by dirstructure.Directory.calcStats(), and dqm_interfaces.DirWalkerFile.ls().

dirstructure.Directory.do_pngs

Definition at line 78 of file dirstructure.py.

Referenced by dqm_interfaces.DirWalkerDB.run().

dirstructure.Directory.draw_success

Definition at line 77 of file dirstructure.py.

dirstructure.Directory.filename1

Definition at line 84 of file dirstructure.py.

Referenced by dirstructure.Directory.calcStats().

dirstructure.Directory.filename2

Definition at line 85 of file dirstructure.py.

Referenced by dirstructure.Directory.calcStats().

dirstructure.Directory.full_path

Definition at line 87 of file dirstructure.py.

Referenced by dirstructure.Directory.calcStats().

dirstructure.Directory.meta

Definition at line 63 of file dirstructure.py.

Referenced by dirstructure.Directory.calcStats().

dirstructure.Directory.mother_dir

Definition at line 62 of file dirstructure.py.

Referenced by dirstructure.Directory.__create_on_disk(), dirstructure.Directory.__get_full_path(), dirstructure.Comparison.__get_img_name(), dirstructure.Comparison.__make_img_dir(), dirstructure.Directory.calcStats(), and dirstructure.Directory.print_report().

dirstructure.Directory.n_comp_fails

Definition at line 71 of file dirstructure.py.

Referenced by dirstructure.Directory.calcStats(), and dirstructure.Directory.prune().

dirstructure.Directory.n_comp_nulls

Definition at line 73 of file dirstructure.py.

Referenced by dirstructure.Directory.calcStats(), and dirstructure.Directory.prune().

dirstructure.Directory.n_comp_skiped

Definition at line 70 of file dirstructure.py.

Referenced by dirstructure.Directory.calcStats(), and dirstructure.Directory.prune().

dirstructure.Directory.n_comp_successes

Definition at line 72 of file dirstructure.py.

Referenced by dirstructure.Directory.calcStats(), and dirstructure.Directory.prune().

dirstructure.Directory.n_fails

Definition at line 66 of file dirstructure.py.

Referenced by dirstructure.Directory.__create_pie_image(), dirstructure.Directory.__repr__(), dirstructure.Directory.calcStats(), dirstructure.Directory.get_fail_rate(), and dirstructure.Directory.print_report().

dirstructure.Directory.n_missing_objs

Definition at line 86 of file dirstructure.py.

Referenced by dirstructure.Directory.calcStats(), and dirstructure.Directory.print_report().

dirstructure.Directory.n_nulls

Definition at line 68 of file dirstructure.py.

Referenced by dirstructure.Directory.__create_pie_image(), dirstructure.Directory.__repr__(), dirstructure.Directory.calcStats(), dirstructure.Directory.get_null_rate(), and dirstructure.Directory.print_report().

dirstructure.Directory.n_skiped

Definition at line 69 of file dirstructure.py.

Referenced by dirstructure.Directory.__create_pie_image(), dirstructure.Directory.calcStats(), dirstructure.Directory.get_skiped_rate(), and dirstructure.Directory.print_report().

dirstructure.Directory.n_successes

Definition at line 67 of file dirstructure.py.

Referenced by dirstructure.Directory.__create_pie_image(), dirstructure.Directory.__repr__(), dirstructure.Directory.calcStats(), dirstructure.Directory.get_success_rate(), and dirstructure.Directory.print_report().

dirstructure.Directory.rank_histo

Definition at line 79 of file dirstructure.py.

dirstructure.Directory.stats_calculated

Definition at line 75 of file dirstructure.py.

Referenced by dirstructure.Directory.calcStats().

dirstructure.Directory.subdirs

Definition at line 64 of file dirstructure.py.

Referenced by dirstructure.Directory.__repr__(), dirstructure.Directory.calcStats(), dirstructure.Directory.get_subdirs_dict(), dirstructure.Directory.get_subdirs_names(), dirstructure.Directory.is_empty(), dirstructure.Directory.print_report(), and dirstructure.Directory.prune().

dirstructure.Directory.weight

Definition at line 74 of file dirstructure.py.