cmsHarvester.CMSHarvester Class Reference

CMSHarvester class. More...

Inheritance diagram for cmsHarvester.CMSHarvester:

Public Member Functions
def	__init__
def	cleanup
def	config_file_header
def	db_account_name_cms_cond_dqm_summary
def	db_account_name_cms_cond_globaltag
def	format_conditions_string
def	ident_string
def	option_handler_book_keeping_file
def	option_handler_debug
def	option_handler_force
def	option_handler_frontier_connection
def	option_handler_globaltag
def	option_handler_harvesting_mode
def	option_handler_harvesting_type
def	option_handler_input_Jsonfile
def	option_handler_input_Jsonrunfile
def	option_handler_input_spec
def	option_handler_input_todofile
def	option_handler_no_ref_hists
def	option_handler_quiet
def	option_handler_ref_hist_mapping_file
def	set_output_level
def	time_stamp

Public Attributes
	book_keeping_file_name
	force_running
	frontier_connection_name
	frontier_connection_overridden
	globaltag
	harvesting_info
	harvesting_mode
	harvesting_mode_default
	harvesting_modes
	harvesting_type
	harvesting_types
	input_method
	input_name
	Jsonfilename
	Jsonrunfilename
	log_level
	ref_hist_mappings_file_name
	todofile
	use_ref_hists
	version

Static Public Attributes
list	cmd_line_opts = sys.argv[1:]
tuple	log_formatter = logging.Formatter("%(message)s")
tuple	log_handler = logging.StreamHandler()
tuple	logger = logging.getLogger()

Detailed Description

CMSHarvester class.

Class to perform CMS harvesting.

More documentation `obviously' to follow.

Definition at line 292 of file cmsHarvester.py.

Constructor & Destructor Documentation

def cmsHarvester.CMSHarvester.__init__	(		self,
			cmd_line_opts = None
	)

Definition at line 301 of file cmsHarvester.py.

    def __init__(self, cmd_line_opts=None):
        "Initialize class and process command line options."

        self.version = __version__

        # These are the harvesting types allowed. See the head of this
        # file for more information.
        self.harvesting_types = [
            "RelVal",
            "RelValFS",
            "MC",
            "DQMOffline",
            ]

        # These are the possible harvesting modes:
        #   - Single-step: harvesting takes place on-site in a single
        #   step. For each samples only a single ROOT file containing
        #   the harvesting results is returned.
        #   - Single-step-allow-partial: special hack to allow
        #   harvesting of partial statistics using single-step
        #   harvesting on spread-out samples.
        #   - Two-step: harvesting takes place in two steps. The first
        #   step returns a series of monitoring elenent summaries for
        #   each sample. The second step then merges these summaries
        #   locally and does the real harvesting. This second step
        #   produces the ROOT file containing the harvesting results.
        self.harvesting_modes = [
            "single-step",
            "single-step-allow-partial",
            "two-step"
            ]

        # It is possible to specify a GlobalTag that will override any
        # choices (regarding GlobalTags) made by the cmsHarvester.
        # BUG BUG BUG
        # For the moment, until I figure out a way to obtain the
        # GlobalTag with which a given data (!) dataset was created,
        # it is necessary to specify a GlobalTag for harvesting of
        # data.
        # BUG BUG BUG end
        self.globaltag = None

        # It's also possible to switch off the use of reference
        # histograms altogether.
        self.use_ref_hists = True

        # The database name and account are hard-coded. They are not
        # likely to change before the end-of-life of this tool. But of
        # course there is a way to override this from the command
        # line. One can even override the Frontier connection used for
        # the GlobalTag and for the reference histograms
        # independently. Please only use this for testing purposes.
        self.frontier_connection_name = {}
        self.frontier_connection_name["globaltag"] = "frontier://" \
                                                     "FrontierProd/"
        self.frontier_connection_name["refhists"] = "frontier://" \
                                                    "FrontierProd/"
        self.frontier_connection_overridden = {}
        for key in self.frontier_connection_name.keys():
            self.frontier_connection_overridden[key] = False

        # This contains information specific to each of the harvesting
        # types. Used to create the harvesting configuration. It is
        # filled by setup_harvesting_info().
        self.harvesting_info = None

        ###

        # These are default `unused' values that will be filled in
        # depending on the command line options.

        # The type of harvesting we're doing. See
        # self.harvesting_types for allowed types.
        self.harvesting_type = None

        # The harvesting mode, popularly known as single-step
        # vs. two-step. The thing to remember at this point is that
        # single-step is only possible for samples located completely
        # at a single site (i.e. SE).
        self.harvesting_mode = None
        # BUG BUG BUG
        # Default temporarily set to two-step until we can get staged
        # jobs working with CRAB.
        self.harvesting_mode_default = "single-step"
        # BUG BUG BUG end

        # The input method: are we reading a dataset name (or regexp)
        # directly from the command line or are we reading a file
        # containing a list of dataset specifications. Actually we
        # keep one of each for both datasets and runs.
        self.input_method = {}
        self.input_method["datasets"] = {}
        self.input_method["datasets"]["use"] = None
        self.input_method["datasets"]["ignore"] = None
        self.input_method["runs"] = {}
        self.input_method["runs"]["use"] = None
        self.input_method["runs"]["ignore"] = None
        self.input_method["runs"]["ignore"] = None
        # The name of whatever input we're using.
        self.input_name = {}
        self.input_name["datasets"] = {}
        self.input_name["datasets"]["use"] = None
        self.input_name["datasets"]["ignore"] = None
        self.input_name["runs"] = {}
        self.input_name["runs"]["use"] = None
        self.input_name["runs"]["ignore"] = None