Functions
def	auth_wget (url, chunk_size=1048576)

Variables
	action

	all_files

	args

	base_url

	default

	dest

	False

	file_list_re

	file_res
	Fetch the files, using multi-processing. More...

	filedir_html

	filedir_url

	help

	mthreads

	options
	Parse sys.argv. More...

	parser
	Define options. More...

	pool

	regexp

	release

	releasedir

	relvaldir
	Check for option errors. More...

	selected_files

	usage

Detailed Description

Script fetches files matching specified RegExps from DQM GUI.

Author:  Albertas Gimbutas,  Vilnius University (LT)
e-mail:  albertasgim@gmail.com

Function Documentation

def fetchall_from_DQM_v2.auth_wget	(	url,
		chunk_size = `1048576`
	)

Returns the content of specified URL, which requires authentication.
If the content is bigger than 1MB, then save it to file.

Definition at line 38 of file fetchall_from_DQM_v2.py.

References estimatePileup.basename, and createfilelist.int.

 def auth_wget(url, chunk_size=1048576):
     """Returns the content of specified URL, which requires authentication.
     If the content is bigger than 1MB, then save it to file.
     """
     opener = build_opener(X509CertOpen())
     url_file = opener.open(Request(url))
     size = int(url_file.headers["Content-Length"])
 
     if size < 1048576:   # if File size < 1MB
         filename = basename(url)    #still download
         readed = url_file.read()    ## and then check if its not an empty dir (parent directory)
         if filename != '':
             outfile = open(filename, 'wb')  #then write File to local system
             outfile.write(readed)
         return readed
 
     filename = basename(url)
     file_id = selected_files.index(filename)
 
     if isfile("./%s" % filename):
         print '%d. Exsits on disk. Skipping.' % (file_id +1)
         return
 
     print '%d. Downloading...' % (file_id +1)
     file = open(filename, 'wb')
     # progress = 0
     chunk = url_file.read(chunk_size)
     while chunk:
         file.write(chunk)
         # progress += chunk_size
         chunk = url_file.read(chunk_size)
     print '%d.  Done.' % (file_id +1)
     file.close()
 
 

Variable Documentation

fetchall_from_DQM_v2.action

Definition at line 75 of file fetchall_from_DQM_v2.py.

fetchall_from_DQM_v2.all_files

Definition at line 123 of file fetchall_from_DQM_v2.py.

fetchall_from_DQM_v2.args

Definition at line 90 of file fetchall_from_DQM_v2.py.

fetchall_from_DQM_v2.base_url

Definition at line 115 of file fetchall_from_DQM_v2.py.

fetchall_from_DQM_v2.default

Definition at line 81 of file fetchall_from_DQM_v2.py.

fetchall_from_DQM_v2.dest

Definition at line 75 of file fetchall_from_DQM_v2.py.

fetchall_from_DQM_v2.False

Definition at line 87 of file fetchall_from_DQM_v2.py.

fetchall_from_DQM_v2.file_list_re

Definition at line 122 of file fetchall_from_DQM_v2.py.

fetchall_from_DQM_v2.file_res

Fetch the files, using multi-processing.

Definition at line 130 of file fetchall_from_DQM_v2.py.

fetchall_from_DQM_v2.filedir_html

Definition at line 117 of file fetchall_from_DQM_v2.py.

fetchall_from_DQM_v2.filedir_url

Definition at line 116 of file fetchall_from_DQM_v2.py.

fetchall_from_DQM_v2.help

Definition at line 76 of file fetchall_from_DQM_v2.py.

fetchall_from_DQM_v2.mthreads

Definition at line 125 of file fetchall_from_DQM_v2.py.

fetchall_from_DQM_v2.options

Parse sys.argv.

Definition at line 90 of file fetchall_from_DQM_v2.py.

fetchall_from_DQM_v2.parser

Define options.

Definition at line 74 of file fetchall_from_DQM_v2.py.

fetchall_from_DQM_v2.pool

Definition at line 139 of file fetchall_from_DQM_v2.py.

fetchall_from_DQM_v2.regexp

Definition at line 92 of file fetchall_from_DQM_v2.py.

Referenced by DQMGenericTnPClient.findAllSubdirectories(), DQMGenericClient.findAllSubdirectories(), edm.glob2reg(), HLTConfigProvider.matched(), edm.regexMatch(), and HLTConfigProvider.removeVersion().

fetchall_from_DQM_v2.release

Definition at line 91 of file fetchall_from_DQM_v2.py.

Referenced by XMLConfigReader.buildGP(), ConfigurationDBHandler.cvt2String(), edm::DataMixingModule.DataMixingModule(), cms::xerces.dispose(), edm::stream::EDAnalyzerAdaptor< T >.EDAnalyzerAdaptor(), SaxToDom.error(), SaxToDom2.error(), L1GtTriggerMenuXmlParser.findXMLChild(), RPAlignmentCorrectionsMethods.GetCorrectionsDataFromFile(), XMLDOMBlock.getNewDocument(), XMLConfigReader.getPatternsVersion(), L1GtTriggerMenuXmlParser.getXMLAttribute(), L1GtTriggerMenuXmlParser.getXMLTextValue(), MonitorXMLParser.handleElement(), cond::XMLAuthenticationService::XMLAuthenticationService.initialize(), L1GtTriggerMenuXmlParser.initXML(), MonitorXMLParser.load(), RPAlignmentCorrectionsDataSequence.LoadXMLFile(), MonitorXMLParser.MonitorXMLParser(), MuonAlignmentInputXML.newAlignableMuon(), XMLDOMBlock.parse(), L1GtTriggerMenuXmlParser.parseConditions(), MuonAlignmentInputXML.parseDouble(), SummaryPlotXmlParser.parseXML(), cond::XMLAuthenticationService::XMLAuthenticationService.processFile(), TotemDAQMappingESSourceXML.produce(), edm::stream::ProducingModuleAdaptor< T, M, B >.ProducingModuleAdaptor(), HcalItemArrayColl< Item, N >.push_back(), CalibrationXML.readAttribute(), XMLConfigReader.readConfig(), l1t::XmlConfigParser.readContext(), l1t::XmlConfigParser.readHwDescription(), XMLConfigReader.readPatterns(), XMLProcessor.serializeDOM(), HcalItemArrayCollById< Item, N >.setDefault(), SaxToDom.startElement(), SaxToDom2.startElement(), edm::eventsetup::DataKey.swap(), cscdqm::XMLFileErrorHandler.warning(), L1GtTriggerMenuXmlParser.workXML(), CalibrationXML.writeAttribute(), xMLCh2String(), XMLDOMBlock.XMLDOMBlock(), ConfigurationDBHandler.~ConfigurationDBHandler(), cond::persistency::IGTTable.~IGTTable(), LutXml.~LutXml(), MonitorXMLParser.~MonitorXMLParser(), MuonAlignmentInputXML.~MuonAlignmentInputXML(), edm::SingleConsumerQ::OperateBuffer< T >.~OperateBuffer(), cond::persistency::GLOBAL_TAG::Table.~Table(), TagNames.~TagNames(), XMLSimpleStr.~XMLSimpleStr(), lhef::XMLSimpleStr.~XMLSimpleStr(), XMLUniStr.~XMLUniStr(), lhef::XMLUniStr.~XMLUniStr(), and XStr.~XStr().

fetchall_from_DQM_v2.releasedir

Definition at line 113 of file fetchall_from_DQM_v2.py.

fetchall_from_DQM_v2.relvaldir

Check for option errors.

Use options

Definition at line 106 of file fetchall_from_DQM_v2.py.

fetchall_from_DQM_v2.selected_files

Definition at line 131 of file fetchall_from_DQM_v2.py.

fetchall_from_DQM_v2.usage

Definition at line 74 of file fetchall_from_DQM_v2.py.

Functions

Variables

Detailed Description

Function Documentation

Variable Documentation