parallelization Namespace Reference

Functions
def	adjust_pset (cmsrunfilename, savefilename, par_list)
def	cleanupandexit (filelist)
def	uint (string)

Variables
	action
	actual_int_jobs
	Integrate ##. More...
	args
	build_log
	build_name
	choices
	cleanupfiles
	default
	help
	int
	integration_log
	integration_name
	parameters
	Execute the different run modes. More...
	parser
	Get command line arguments. More...
	process
	processes
	Run mode ##. More...
	run_log
	run_name
	shell
	stderr
	stdout
	template_name
	type
	uint

Function Documentation

adjust_pset()

def parallelization.adjust_pset	(		cmsrunfilename,
			savefilename,
			par_list
	)

Takes the cmsRun filem, removes all occurences of runMode, jobs,
   maxJobs and integrationList parameters in the process.generator
   part.
   The the parameters in par_list are set instead and saved.

Definition at line 60 of file parallelization.py.

def adjust_pset(cmsrunfilename, savefilename, par_list):
    """Takes the cmsRun filem, removes all occurences of runMode, jobs,
       maxJobs and integrationList parameters in the process.generator
       part.
       The the parameters in par_list are set instead and saved.
    """ 
 
    with open(cmsrunfilename, 'r') as readfile:
        parsestring = readfile.read()
 
        # get first opening bracket after process.generator
        begin_gen_step = parsestring.find('(', parsestring.find('process.generator'))
 
        # find matching bracket
        end_gen_step = begin_gen_step
        bracket_counter = 1
        for position in range(begin_gen_step+1, len(parsestring)):
            if parsestring[position] == '(':
                bracket_counter += 1
            if parsestring[position] == ')':
                bracket_counter -= 1
            if not bracket_counter:
                end_gen_step = position
                break
 
        # get string between brackets
        gen_string = parsestring[begin_gen_step+1:end_gen_step]
 
        # remove all parameters that would interfere
        gen_string = re.sub(r',\s*runModeList\s*=\s*cms.untracked.string\((.*?)\)', '', gen_string)
        gen_string = re.sub(r',\s*jobs\s*=\s*cms.untracked.int32\((.*?)\)', '', gen_string)
        gen_string = re.sub(r',\s*integrationList\s*=\s*cms.untracked.string\((.*?)\)', '', gen_string)
        gen_string = re.sub(r',\s*maxJobs\s*=\s*cms.untracked.uint32\((.*?)\)', '', gen_string)
        gen_string = re.sub(r',\s*seed\s*=\s*cms.untracked.int32\((.*?)\)', '', gen_string)
 
 
    # write the savefile with all parameters given in par_list
    with open(savefilename,'w') as savefile:
        savefile.write(parsestring[:begin_gen_step+1])
        savefile.write(gen_string)
        for item in par_list:
            savefile.write(',\n')
            savefile.write(item)
        savefile.write(parsestring[end_gen_step:])
 
 
 

References FastTimerService_cff.range.

cleanupandexit()

def parallelization.cleanupandexit

(

filelist

)

Delete the files in filelist and exit

Definition at line 107 of file parallelization.py.

def cleanupandexit(filelist):
    """Delete the files in filelist and exit"""
    for filename in filelist:
        os.remove(filename)
    sys.exit(0)
 
 
 
 

uint()

def parallelization.uint

(

string

)

Unsigned int type

Definition at line 50 of file parallelization.py.

def uint(string):
    """Unsigned int type"""
    value = int(string)
    if value < 0:
        msg = '{0} is negative'.format(string)
        raise argparse.ArgumentTypeError(msg)
    return value
 
 
 

References int, and uint.

Variable Documentation

action

parallelization.action

Definition at line 126 of file parallelization.py.

actual_int_jobs

parallelization.actual_int_jobs

Integrate ##.

Definition at line 192 of file parallelization.py.

args

parallelization.args

Definition at line 132 of file parallelization.py.

build_log

parallelization.build_log

Definition at line 172 of file parallelization.py.

build_name

parallelization.build_name

Definition at line 159 of file parallelization.py.

choices

parallelization.choices

Definition at line 123 of file parallelization.py.

cleanupfiles

parallelization.cleanupfiles

Definition at line 135 of file parallelization.py.

default

parallelization.default

Definition at line 123 of file parallelization.py.

help

parallelization.help

Definition at line 122 of file parallelization.py.

int

parallelization.int

Definition at line 123 of file parallelization.py.

Referenced by uint().

integration_log

parallelization.integration_log

Definition at line 220 of file parallelization.py.

integration_name

parallelization.integration_name

Definition at line 211 of file parallelization.py.

parameters

parallelization.parameters

Execute the different run modes.

Build ##

Definition at line 154 of file parallelization.py.

parser

parallelization.parser

Get command line arguments.

Definition at line 120 of file parallelization.py.

process

parallelization.process

Definition at line 172 of file parallelization.py.

processes

parallelization.processes

Run mode ##.

This part uses the parallelization of the run step provided by Herwig. At the moment it is not usable. if args.run != 0: parameters = ['runModeList = cms.untracked.string(\'run\')'] parameters.append('jobs = cms.untracked.int32(' + str(args.run) + ')')

run_name = template_name + '_run.py' adjust_pset(args.cmsRunfile, run_name, parameters) cleanupfiles.append(run_name)

print 'Setting up {0} run jobs.'.format(str(args.run)) print 'Calling
\t\'cmsRun ' + run_name + '\'\nfor the Herwig run step.'.format(str(args.run)) process = subprocess.Popen(['cmsRun ' + run_name], shell=True) process.wait() print '---------------—' print 'Run step finished.' print '---------------—' This is the alternative for a paralellized run step. cmsRun is called as often as give with the option -r/–run. So the total number of generated events is a corresponding multiple of the number of events given in the cmsRun file.

Definition at line 205 of file parallelization.py.

run_log

parallelization.run_log

Definition at line 295 of file parallelization.py.

run_name

parallelization.run_name

Definition at line 272 of file parallelization.py.

shell

parallelization.shell

Definition at line 174 of file parallelization.py.

stderr

parallelization.stderr

Definition at line 172 of file parallelization.py.

stdout

parallelization.stdout

Definition at line 172 of file parallelization.py.

template_name

parallelization.template_name

Definition at line 138 of file parallelization.py.

type

parallelization.type

Definition at line 123 of file parallelization.py.

uint

parallelization.uint

Definition at line 124 of file parallelization.py.

Referenced by SiStripMonitorDigi.AddApvShotsToSubDet(), FWRecoGeometryESProducer.addCaloGeometry(), L1Scalers.analyze(), SiStripMonitorHLT.analyze(), HLTFiltersDQMonitor.analyze(), SiStripMonitorDigi.analyze(), MonitorTrackResidualsBase< pixel_or_strip >.analyze(), SiPixelTrackResidualSource.analyze(), SiPixelHitEfficiencySource.analyze(), SiStripMonitorCluster.analyze(), TkConvValidator.analyze(), L1TS2PFJetInputPatternWriter.analyze(), L1Validator.analyze(), MuIsoValidation.analyze(), TrackingMonitor.analyze(), SplitVertexResolution.analyze(), OverlapValidation.analyze(), PhotonAnalyzer.analyze(), ZToMuMuGammaAnalyzer.analyze(), reco::tau::PFRecoTauDiscriminationByMVAIsolationRun2.beginEvent(), PhotonOfflineClient.book2DHistoVector(), PhotonAnalyzer.book2DHistoVector(), PhotonOfflineClient.book3DHistoVector(), PhotonAnalyzer.book3DHistoVector(), L1Scalers.bookHistograms(), TrackingMonitor.bookHistograms(), PhotonAnalyzer.bookHistogramsEfficiency(), TriggerDQMBase.bookME(), TrackerOfflineValidation.bookSummaryHists(), L1TkMuonProducer.build_tkMuons_from_idxs(), CmsMTDConstruction< FilteredView >.buildETLModule(), L1TMuonBarrelKalmanStubProcessor.buildStub(), L1METPFProducer.CalcMetHLS(), tmtt::ChiSquaredFitBase.calculateChiSq(), L1TMuonBarrelKalmanAlgo.calculateEta(), GsfElectronAlgo.calculateShowerShape(), EffectiveAreas.checkConsistency(), riemannFit.circleFit(), FlagsCleanerECAL.clean(), L1TMuonBarrelKalmanAlgo.clean(), L1TMuonBarrelKalmanRegionModule.cleanHigher(), L1TMuonBarrelKalmanRegionModule.cleanLower(), L1TMuonBarrelKalmanLUTs.coarseEta(), riemannFit.computeRadLenUniformMaterial(), EERecHitFromSoA.convert_soa_data_to_collection_(), HEFRecHitFromSoA.convert_soa_data_to_collection_(), HEBRecHitFromSoA.convert_soa_data_to_collection_(), JetCoreMCtruthSeedGenerator.coreTracksFilling(), JetCoreMCtruthSeedGenerator.coreTracksFillingDeltaR(), riemannFit.cov_carttorad(), riemannFit.cov_carttorad_prefit(), riemannFit.cov_radtocart(), GsfElectronAlgo.createElectron(), PFPhotonTranslator.createPhotons(), PFRecoTauDiscriminationByNProngs.discriminate(), CSCOfflineMonitor.doEfficiencies(), HLTMuonCertSummary.dqmEndJob(), EcalDAQTowerStatusXMLTranslator.dumpXML(), EcalDCSTowerStatusXMLTranslator.dumpXML(), L1TStage2InputPatternWriter.endJob(), L1TS2PFJetInputPatternWriter.endJob(), GlobalHitsProdHistStripper.endRun(), L1TMuonBarrelKalmanAlgo.estimateChiSquare(), L1TMuonBarrelKalmanAlgo.estimateCompatibility(), VariableNTupler.fill(), StringBasedNTupler.fill(), SiStripMonitorDigi.FillApvShotsMap(), PixelVertexCollectionTrimmer.fillDescriptions(), PuppiProducer.fillDescriptions(), CTPPSPixelLocalTrackProducer.fillDescriptions(), TriggerDQMBase.fillHistoLSPSetDescription(), TriggerDQMBase.fillHistoPSetDescription(), PhotonProducer.fillPhotonCollection(), GEDPhotonProducer.fillPhotonCollection(), PythiaFilterMultiAncestor.filter(), L1TkMuMantra.find_match(), L1Validator.FindBest(), L1TkMuCorrDynamicWindows.findBin(), L1TkMuMantra.findBin(), L1TMuonBarrelKalmanAlgo.getByCode(), EffectiveAreas.getEffectiveArea(), ZElectronsSelector.getEffectiveArea(), TrackerOfflineValidation.getHistStructFromMap(), PixelInactiveAreaFinder.getPhiSpanBarrel(), L1TTauOffline.getProbeTaus(), reco::GsfElectron.hcalToRun2EffDepth(), riemannFit.helixFit(), L1TowerCalibrator.L1TowerCalibrator(), riemannFit.lineFit(), PixelCPEGeneric.localError(), L1TMuonBarrelKalmanStubProcessor.makeStubs(), LowPtConversion.match(), L1TMuonBarrelKalmanAlgo.match(), SiStripTrackerMapCreator.paintTkMapFromAlarm(), EcalDAQTowerStatusXMLTranslator.plot(), EcalDCSTowerStatusXMLTranslator.plot(), TrackerOfflineValidation.prepareSummaryHists(), EffectiveAreas.printEffectiveAreas(), ZElectronsSelector.printEffectiveAreas(), riemannFit.printIt(), l1t::Stage2Layer2DemuxSumsAlgoFirmwareImp1.processEvent(), L1TMuonBarrelKalmanTrackProducer.produce(), L1TMuonBarrelKalmanStubProducer.produce(), PuppiProducer.produce(), HiEvtPlaneFlatProducer.produce(), JetCoreMCtruthSeedGenerator.produce(), EvtPlaneProducer.produce(), L1TkMuonProducer.product_to_muvec(), L1TkMuonProducer.product_to_trkvec(), PuppiProducer.PuppiProducer(), L1TMuonBarrelKalmanAlgo.rank(), L1TCaloParamsOnlineProd.readCaloLayer2OnlineSettings(), CompleteNTupler.registerleaves(), VariableNTupler.registerleaves(), StringBasedNTupler.registerleaves(), MuonTestSummary.ResidualCheck(), PhotonOfflineClient.retrieveHisto(), RPixPlaneCombinatoryTracking.RPixPlaneCombinatoryTracking(), PhotonOfflineClient.runClient(), L1TkMuonProducer.runOnMTFCollection_v2(), riemannFit.scatter_cov_rad(), riemannFit.scatterCovLine(), JetCoreMCtruthSeedGenerator.seedParFilling(), HGCalConcentratorAutoEncoderImpl.select(), StringCutEventSelector< Object, any >.select(), StringCutsEventSelector< Object, existenceMatter >.select(), L1TMuonBarrelKalmanRegionModule.selfClean(), TrackingMonitor.setMaxMinBin(), TrackingMonitor.setNclus(), StringBasedNTupler.StringBasedNTupler(), StringBranchHelper< Object, Collection >.StringBranchHelper(), TauDiscriminantCutMultiplexerT< TauType, TauTypeRef, ParentClass >.TauDiscriminantCutMultiplexerT(), trklet::FitTrack.trackFitChisq(), uint(), l1t::stage1.unpack_em(), l1t::stage1.unpack_region(), RctRawToDigi.unpackCTP7(), L1TMuonBarrelKalmanAlgo.updateLUT(), VariableNTupler.VariableNTupler(), L1TMuonBarrelKalmanAlgo.vertexConstraintLUT(), and L1TMuonBarrelKalmanSectorProcessor.wedgeSort().