cmsHarvester.CMSHarvester Class Reference

CMSHarvester class. More...

Inheritance diagram for cmsHarvester.CMSHarvester:

Public Member Functions
def	__init__ (self, cmd_line_opts=None)
def	cleanup (self)
def	config_file_header (self)
def	db_account_name_cms_cond_dqm_summary (self)
def	db_account_name_cms_cond_globaltag (self)
def	format_conditions_string (self, globaltag)
def	ident_string (self)
def	option_handler_book_keeping_file (self, option, opt_str, value, parser)
def	option_handler_debug (self, option, opt_str, value, parser)
def	option_handler_force (self, option, opt_str, value, parser)
def	option_handler_frontier_connection (self, option, opt_str, value, parser)
def	option_handler_globaltag (self, option, opt_str, value, parser)
def	option_handler_harvesting_mode (self, option, opt_str, value, parser)
def	option_handler_harvesting_type (self, option, opt_str, value, parser)
def	option_handler_input_Jsonfile (self, option, opt_str, value, parser)
def	option_handler_input_Jsonrunfile (self, option, opt_str, value, parser)
def	option_handler_input_spec (self, option, opt_str, value, parser)
def	option_handler_input_todofile (self, option, opt_str, value, parser)
def	option_handler_no_ref_hists (self, option, opt_str, value, parser)
def	option_handler_quiet (self, option, opt_str, value, parser)
def	option_handler_ref_hist_mapping_file (self, option, opt_str, value, parser)
def	set_output_level (self, output_level)
def	time_stamp (self)

Public Attributes
	book_keeping_file_name
	force_running
	frontier_connection_name
	frontier_connection_overridden
	globaltag
	harvesting_info
	harvesting_mode
	harvesting_mode_default
	harvesting_modes
	harvesting_type
	harvesting_types
	input_method
	input_name
	Jsonfilename
	Jsonrunfilename
	log_level
	ref_hist_mappings_file_name
	todofile
	use_ref_hists
	version

Detailed Description

CMSHarvester class.

Class to perform CMS harvesting.

More documentation `obviously' to follow.

Definition at line 293 of file cmsHarvester.py.

Constructor & Destructor Documentation

def cmsHarvester.CMSHarvester.__init__	(		self,
			cmd_line_opts = None
	)

Definition at line 302 of file cmsHarvester.py.

    def __init__(self, cmd_line_opts=None):
        "Initialize class and process command line options."

        self.version = __version__

        # These are the harvesting types allowed. See the head of this
        # file for more information.
        self.harvesting_types = [
            "RelVal",
            "RelValFS",
            "MC",
            "DQMOffline",
            ]

        # These are the possible harvesting modes:
        #   - Single-step: harvesting takes place on-site in a single
        #   step. For each samples only a single ROOT file containing
        #   the harvesting results is returned.
        #   - Single-step-allow-partial: special hack to allow
        #   harvesting of partial statistics using single-step
        #   harvesting on spread-out samples.
        #   - Two-step: harvesting takes place in two steps. The first
        #   step returns a series of monitoring elenent summaries for
        #   each sample. The second step then merges these summaries
        #   locally and does the real harvesting. This second step
        #   produces the ROOT file containing the harvesting results.
        self.harvesting_modes = [
            "single-step",
            "single-step-allow-partial",
            "two-step"
            ]

        # It is possible to specify a GlobalTag that will override any
        # choices (regarding GlobalTags) made by the cmsHarvester.
        # BUG BUG BUG
        # For the moment, until I figure out a way to obtain the
        # GlobalTag with which a given data (!) dataset was created,
        # it is necessary to specify a GlobalTag for harvesting of
        # data.
        # BUG BUG BUG end
        self.globaltag = None

        # It's also possible to switch off the use of reference
        # histograms altogether.
        self.use_ref_hists = True

        # The database name and account are hard-coded. They are not
        # likely to change before the end-of-life of this tool. But of
        # course there is a way to override this from the command
        # line. One can even override the Frontier connection used for
        # the GlobalTag and for the reference histograms
        # independently. Please only use this for testing purposes.
        self.frontier_connection_name = {}
        self.frontier_connection_name["globaltag"] = "frontier://" \
                                                     "FrontierProd/"
        self.frontier_connection_name["refhists"] = "frontier://" \
                                                    "FrontierProd/"
        self.frontier_connection_overridden = {}
        for key in self.frontier_connection_name.keys():
            self.frontier_connection_overridden[key] = False

        # This contains information specific to each of the harvesting
        # types. Used to create the harvesting configuration. It is
        # filled by setup_harvesting_info().
        self.harvesting_info = None

        ###

        # These are default `unused' values that will be filled in
        # depending on the command line options.

        # The type of harvesting we're doing. See
        # self.harvesting_types for allowed types.
        self.harvesting_type = None

        # The harvesting mode, popularly known as single-step
        # vs. two-step. The thing to remember at this point is that
        # single-step is only possible for samples located completely
        # at a single site (i.e. SE).
        self.harvesting_mode = None
        # BUG BUG BUG
        # Default temporarily set to two-step until we can get staged
        # jobs working with CRAB.
        self.harvesting_mode_default = "single-step"
        # BUG BUG BUG end

        # The input method: are we reading a dataset name (or regexp)
        # directly from the command line or are we reading a file
        # containing a list of dataset specifications. Actually we
        # keep one of each for both datasets and runs.
        self.input_method = {}
        self.input_method["datasets"] = {}
        self.input_method["datasets"]["use"] = None
        self.input_method["datasets"]["ignore"] = None
        self.input_method["runs"] = {}
        self.input_method["runs"]["use"] = None
        self.input_method["runs"]["ignore"] = None
        self.input_method["runs"]["ignore"] = None
        # The name of whatever input we're using.
        self.input_name = {}
        self.input_name["datasets"] = {}
        self.input_name["datasets"]["use"] = None
        self.input_name["datasets"]["ignore"] = None
        self.input_name["runs"] = {}
        self.input_name["runs"]["use"] = None
        self.input_name["runs"]["ignore"] = None

Member Function Documentation

def cmsHarvester.CMSHarvester.cleanup

(

self

)

Definition at line 506 of file cmsHarvester.py.

Referenced by esMonitoring.FDJsonServer.handle_close().

    def cleanup(self):
        "Clean up after ourselves."

        # NOTE: This is the safe replacement of __del__.

        #self.logger.debug("All done -> shutting down")
        logging.shutdown()

        # End of cleanup.

def cmsHarvester.CMSHarvester.config_file_header

(

self

)

Definition at line 600 of file cmsHarvester.py.

References cmsHarvester.CMSHarvester.ident_string(), join(), and cmsHarvester.CMSHarvester.time_stamp().

    def config_file_header(self):
        "Create a nice header to be used to mark the generated files."

        tmp = []

        time_stamp = self.time_stamp()
        ident_str = self.ident_string()
        tmp.append("# %s" % time_stamp)
        tmp.append("# WARNING: This file was created automatically!")
        tmp.append("")
        tmp.append("# Created by %s" % ident_str)

        header = "\n".join(tmp)

        # End of config_file_header.
        return header

def cmsHarvester.CMSHarvester.db_account_name_cms_cond_dqm_summary

(

self

)

See db_account_name_cms_cond_globaltag.

Definition at line 585 of file cmsHarvester.py.

References cmsPerfSuite.PerfSuite.cmssw_version.

    def db_account_name_cms_cond_dqm_summary(self):
        """See db_account_name_cms_cond_globaltag."""

        account_name = None
        version = self.cmssw_version[6:11]
        if version < "3_4_0":
            account_name = "CMS_COND_31X_DQM_SUMMARY"
        else:
            account_name = "CMS_COND_34X"

        # End of db_account_name_cms_cond_dqm_summary.
        return account_name

def cmsHarvester.CMSHarvester.db_account_name_cms_cond_globaltag

(

self

)

Return the database account name used to store the GlobalTag.

The name of the database account depends (albeit weakly) on
the CMSSW release version.

Definition at line 568 of file cmsHarvester.py.

    def db_account_name_cms_cond_globaltag(self):
        """Return the database account name used to store the GlobalTag.

        The name of the database account depends (albeit weakly) on
        the CMSSW release version.

        """

        # This never changed, unlike the cms_cond_31X_DQM_SUMMARY ->
        # cms_cond_34X_DQM transition.
        account_name = "CMS_COND_31X_GLOBALTAG"

        # End of db_account_name_cms_cond_globaltag.
        return account_name

def cmsHarvester.CMSHarvester.format_conditions_string	(		self,
			globaltag
	)

Create the conditions string needed for `cmsDriver'.

Just glueing the FrontierConditions bit in front of it really.

Definition at line 543 of file cmsHarvester.py.

References spr.find().

    def format_conditions_string(self, globaltag):
        """Create the conditions string needed for `cmsDriver'.

        Just glueing the FrontierConditions bit in front of it really.

        """

        # Not very robust but okay. The idea is that if the user
        # specified (since this cannot happen with GlobalTags coming
        # from DBS) something containing `conditions', they probably
        # know what they're doing and we should not muck things up. In
        # all other cases we just assume we only received the
        # GlobalTag part and we built the usual conditions string from
        # that .
        if globaltag.lower().find("conditions") > -1:
            conditions_string = globaltag
        else:
            conditions_string = "FrontierConditions_GlobalTag,%s" % \
                                globaltag

        # End of format_conditions_string.
        return conditions_string

def cmsHarvester.CMSHarvester.ident_string

(

self

)

Definition at line 531 of file cmsHarvester.py.

Referenced by cmsHarvester.CMSHarvester.config_file_header().

    def ident_string(self):
        "Spit out an identification string for cmsHarvester.py."

        ident_str = "`cmsHarvester.py " \
                    "version %s': cmsHarvester.py %s" % \
                    (__version__,
                     reduce(lambda x, y: x+' '+y, sys.argv[1:]))

        return ident_str

def cmsHarvester.CMSHarvester.option_handler_book_keeping_file	(		self,
			option,
			opt_str,
			value,
			parser
	)

Store the name of the file to be used for book keeping.

The only check done here is that only a single book keeping
file is specified.

Definition at line 960 of file cmsHarvester.py.

References cmsHarvester.CMSHarvester.book_keeping_file_name.

    def option_handler_book_keeping_file(self, option, opt_str, value, parser):
        """Store the name of the file to be used for book keeping.

        The only check done here is that only a single book keeping
        file is specified.

        """

        file_name = value

        if not self.book_keeping_file_name is None:
            msg = "Only one book keeping file should be specified"
            self.logger.fatal(msg)
            raise Usage(msg)
        self.book_keeping_file_name = file_name

        self.logger.info("Book keeping file to be used: `%s'" % \
                         self.book_keeping_file_name)

        # End of option_handler_book_keeping_file.

def cmsHarvester.CMSHarvester.option_handler_debug	(		self,
			option,
			opt_str,
			value,
			parser
	)

Switch to debug mode.

This both increases the amount of output generated, as well as
changes the format used (more detailed information is given).

Definition at line 658 of file cmsHarvester.py.

References cmsHarvester.CMSHarvester.set_output_level().

    def option_handler_debug(self, option, opt_str, value, parser):
        """Switch to debug mode.

        This both increases the amount of output generated, as well as
        changes the format used (more detailed information is given).

        """

        # Switch to a more informative log formatter for debugging.
        log_formatter_debug = logging.Formatter("[%(levelname)s] " \
                                                # NOTE: funcName was
                                                # only implemented
                                                # starting with python
                                                # 2.5.
                                                #"%(funcName)s() " \
                                                #"@%(filename)s:%(lineno)d " \
                                                "%(message)s")
        # Hmmm, not very nice. This assumes there's only a single
        # handler associated with the current logger.
        log_handler = self.logger.handlers[0]
        log_handler.setFormatter(log_formatter_debug)
        self.set_output_level("DEBUG")

        # End of option_handler_debug.

def cmsHarvester.CMSHarvester.option_handler_force	(		self,
			option,
			opt_str,
			value,
			parser
	)

Switch on `force mode' in which case we don't brake for nobody.

In so-called `force mode' all sanity checks are performed but
we don't halt on failure. Of course this requires some care
from the user.

Definition at line 694 of file cmsHarvester.py.

    def option_handler_force(self, option, opt_str, value, parser):
        """Switch on `force mode' in which case we don't brake for nobody.

        In so-called `force mode' all sanity checks are performed but
        we don't halt on failure. Of course this requires some care
        from the user.

        """

        self.logger.debug("Switching on `force mode'.")
        self.force_running = True

        # End of option_handler_force.

def cmsHarvester.CMSHarvester.option_handler_frontier_connection	(		self,
			option,
			opt_str,
			value,
			parser
	)

Override the default Frontier connection string.

Please only use this for testing (e.g. when a test payload has
been inserted into cms_orc_off instead of cms_orc_on).

This method gets called for three different command line
options:
- --frontier-connection,
- --frontier-connection-for-globaltag,
- --frontier-connection-for-refhists.
Appropriate care has to be taken to make sure things are only
specified once.

Definition at line 822 of file cmsHarvester.py.

References cmsHarvester.CMSHarvester.frontier_connection_name, and cmsHarvester.CMSHarvester.frontier_connection_overridden.

Referenced by cmsHarvester.CMSHarvester.option_handler_no_ref_hists().

                                           value, parser):
        """Override the default Frontier connection string.

        Please only use this for testing (e.g. when a test payload has
        been inserted into cms_orc_off instead of cms_orc_on).

        This method gets called for three different command line
        options:
        - --frontier-connection,
        - --frontier-connection-for-globaltag,
        - --frontier-connection-for-refhists.
        Appropriate care has to be taken to make sure things are only
        specified once.

        """

        # Figure out with which command line option we've been called.
        frontier_type = opt_str.split("-")[-1]
        if frontier_type == "connection":
            # Main option: change all connection strings.
            frontier_types = self.frontier_connection_name.keys()
        else:
            frontier_types = [frontier_type]

        # Make sure that each Frontier connection is specified only
        # once. (Okay, in a bit of a dodgy way...)
        for connection_name in frontier_types:
            if self.frontier_connection_overridden[connection_name] == True:
                msg = "Please specify either:\n" \
                      "  `--frontier-connection' to change the " \
                      "Frontier connection used for everything, or\n" \
                      "either one or both of\n" \
                      "  `--frontier-connection-for-globaltag' to " \
                      "change the Frontier connection used for the " \
                      "GlobalTag and\n" \
                      "  `--frontier-connection-for-refhists' to change " \
                      "the Frontier connection used for the " \
                      "reference histograms."
                self.logger.fatal(msg)
                raise Usage(msg)

        frontier_prefix = "frontier://"
        if not value.startswith(frontier_prefix):
            msg = "Expecting Frontier connections to start with " \
                  "`%s'. You specified `%s'." % \
                  (frontier_prefix, value)
            self.logger.fatal(msg)
            raise Usage(msg)
        # We also kind of expect this to be either FrontierPrep or
        # FrontierProd (but this is just a warning).
        if value.find("FrontierProd") < 0 and \
               value.find("FrontierProd") < 0:
            msg = "Expecting Frontier connections to contain either " \
                  "`FrontierProd' or `FrontierPrep'. You specified " \
                  "`%s'. Are you sure?" % \
                  value
            self.logger.warning(msg)

        if not value.endswith("/"):
            value += "/"

        for connection_name in frontier_types:
            self.frontier_connection_name[connection_name] = value
            self.frontier_connection_overridden[connection_name] = True

            frontier_type_str = "unknown"
            if connection_name == "globaltag":
                frontier_type_str = "the GlobalTag"
            elif connection_name == "refhists":
                frontier_type_str = "the reference histograms"

            self.logger.warning("Overriding default Frontier " \
                                "connection for %s " \
                                "with `%s'" % \
                                (frontier_type_str,
                                 self.frontier_connection_name[connection_name]))

        # End of option_handler_frontier_connection

def cmsHarvester.CMSHarvester.option_handler_globaltag	(		self,
			option,
			opt_str,
			value,
			parser
	)

Set the GlobalTag to be used, overriding our own choices.

By default the cmsHarvester will use the GlobalTag with which
a given dataset was created also for the harvesting. The
--globaltag option is the way to override this behaviour.

Definition at line 787 of file cmsHarvester.py.

References alignment.Alignment.globaltag, and cmsHarvester.CMSHarvester.globaltag.

    def option_handler_globaltag(self, option, opt_str, value, parser):
        """Set the GlobalTag to be used, overriding our own choices.

        By default the cmsHarvester will use the GlobalTag with which
        a given dataset was created also for the harvesting. The
        --globaltag option is the way to override this behaviour.

        """

        # Make sure that this flag only occurred once.
        if not self.globaltag is None:
            msg = "Only one GlobalTag should be specified"
            self.logger.fatal(msg)
            raise Usage(msg)
        self.globaltag = value

        self.logger.info("GlobalTag to be used: `%s'" % \
                         self.globaltag)

        # End of option_handler_globaltag.

def cmsHarvester.CMSHarvester.option_handler_harvesting_mode	(		self,
			option,
			opt_str,
			value,
			parser
	)

Set the harvesting mode to be used.

Single-step harvesting can be used for samples that are
located completely at a single site (= SE). Otherwise use
two-step mode.

Definition at line 754 of file cmsHarvester.py.

References cmsHarvester.CMSHarvester.harvesting_mode, cmsHarvester.CMSHarvester.harvesting_modes, and join().

    def option_handler_harvesting_mode(self, option, opt_str, value, parser):
        """Set the harvesting mode to be used.

        Single-step harvesting can be used for samples that are
        located completely at a single site (= SE). Otherwise use
        two-step mode.

        """

        # Check for valid mode.
        harvesting_mode = value.lower()
        if not harvesting_mode in self.harvesting_modes:
            msg = "Unknown harvesting mode `%s'" % harvesting_mode
            self.logger.fatal(msg)
            self.logger.fatal("  possible modes are: %s" % \
                              ", ".join(self.harvesting_modes))
            raise Usage(msg)

        # Check if we've been given only a single mode, otherwise
        # complain.
        if not self.harvesting_mode is None:
            msg = "Only one harvesting mode should be specified"
            self.logger.fatal(msg)
            raise Usage(msg)
        self.harvesting_mode = harvesting_mode

        self.logger.info("Harvesting mode to be used: `%s'" % \
                         self.harvesting_mode)

        # End of option_handler_harvesting_mode.

def cmsHarvester.CMSHarvester.option_handler_harvesting_type	(		self,
			option,
			opt_str,
			value,
			parser
	)

Set the harvesting type to be used.

This checks that no harvesting type is already set, and sets
the harvesting type to be used to the one specified. If a
harvesting type is already set an exception is thrown. The
same happens when an unknown type is specified.

Definition at line 710 of file cmsHarvester.py.

References cmsHarvester.CMSHarvester.harvesting_type, cmsHarvester.CMSHarvester.harvesting_types, and join().

    def option_handler_harvesting_type(self, option, opt_str, value, parser):
        """Set the harvesting type to be used.

        This checks that no harvesting type is already set, and sets
        the harvesting type to be used to the one specified. If a
        harvesting type is already set an exception is thrown. The
        same happens when an unknown type is specified.

        """

        # Check for (in)valid harvesting types.
        # NOTE: The matching is done in a bit of a complicated
        # way. This allows the specification of the type to be
        # case-insensitive while still ending up with the properly
        # `cased' version afterwards.
        value = value.lower()
        harvesting_types_lowered = [i.lower() for i in self.harvesting_types]
        try:
            type_index = harvesting_types_lowered.index(value)
            # If this works, we now have the index to the `properly
            # cased' version of the harvesting type.
        except ValueError:
            self.logger.fatal("Unknown harvesting type `%s'" % \
                              value)
            self.logger.fatal("  possible types are: %s" %
                              ", ".join(self.harvesting_types))
            raise Usage("Unknown harvesting type `%s'" % \
                        value)

        # Check if multiple (by definition conflicting) harvesting
        # types are being specified.
        if not self.harvesting_type is None:
            msg = "Only one harvesting type should be specified"
            self.logger.fatal(msg)
            raise Usage(msg)
        self.harvesting_type = self.harvesting_types[type_index]

        self.logger.info("Harvesting type to be used: `%s'" % \
                         self.harvesting_type)

        # End of option_handler_harvesting_type.

def cmsHarvester.CMSHarvester.option_handler_input_Jsonfile	(		self,
			option,
			opt_str,
			value,
			parser
	)

Definition at line 910 of file cmsHarvester.py.

    def option_handler_input_Jsonfile(self, option, opt_str, value, parser):

        self.Jsonfilename = value
        # End of option_handler_input_Jsonfile.

def cmsHarvester.CMSHarvester.option_handler_input_Jsonrunfile	(		self,
			option,
			opt_str,
			value,
			parser
	)

Definition at line 917 of file cmsHarvester.py.

    def option_handler_input_Jsonrunfile(self, option, opt_str, value, parser):

        self.Jsonrunfilename = value
        # End of option_handler_input_Jsonrunfile.

def cmsHarvester.CMSHarvester.option_handler_input_spec	(		self,
			option,
			opt_str,
			value,
			parser
	)

TODO TODO TODO
Document this...

Definition at line 924 of file cmsHarvester.py.

References spr.find(), cmsHarvester.CMSHarvester.input_method, cmsHarvester.CMSHarvester.input_name, and python.rootplot.root2matplotlib.replace().

    def option_handler_input_spec(self, option, opt_str, value, parser):
        """TODO TODO TODO
        Document this...

        """

        # Figure out if we were called for the `use these' or the
        # `ignore these' case.
        if opt_str.lower().find("ignore") > -1:
            spec_type = "ignore"
        else:
            spec_type = "use"

        # Similar: are we being called for datasets or for runs?
        if opt_str.lower().find("dataset") > -1:
            select_type = "datasets"
        else:
            select_type = "runs"

        if not self.input_method[select_type][spec_type] is None:
            msg = "Please only specify one input method " \
                  "(for the `%s' case)" % opt_str
            self.logger.fatal(msg)
            raise Usage(msg)

        input_method = opt_str.replace("-", "").replace("ignore", "")
        self.input_method[select_type][spec_type] = input_method
        self.input_name[select_type][spec_type] = value

        self.logger.debug("Input method for the `%s' case: %s" % \
                          (spec_type, input_method))

        # End of option_handler_input_spec

def cmsHarvester.CMSHarvester.option_handler_input_todofile	(		self,
			option,
			opt_str,
			value,
			parser
	)

Definition at line 903 of file cmsHarvester.py.

    def option_handler_input_todofile(self, option, opt_str, value, parser):

        self.todofile = value
        # End of option_handler_input_todofile.

def cmsHarvester.CMSHarvester.option_handler_no_ref_hists	(		self,
			option,
			opt_str,
			value,
			parser
	)

Definition at line 810 of file cmsHarvester.py.

References cmsHarvester.CMSHarvester.option_handler_frontier_connection(), and cmsHarvester.CMSHarvester.use_ref_hists.

    def option_handler_no_ref_hists(self, option, opt_str, value, parser):
        "Switch use of all reference histograms off."

        self.use_ref_hists = False

        self.logger.warning("Switching off all use of reference histograms")

        # End of option_handler_no_ref_hists.
        
     ##########

def cmsHarvester.CMSHarvester.option_handler_quiet	(		self,
			option,
			opt_str,
			value,
			parser
	)

Definition at line 685 of file cmsHarvester.py.

References cmsHarvester.CMSHarvester.set_output_level().

    def option_handler_quiet(self, option, opt_str, value, parser):
        "Switch to quiet mode: less verbose."

        self.set_output_level("QUIET")

        # End of option_handler_quiet.

def cmsHarvester.CMSHarvester.option_handler_ref_hist_mapping_file	(		self,
			option,
			opt_str,
			value,
			parser
	)

Store the name of the file for the ref. histogram mapping.

Definition at line 983 of file cmsHarvester.py.

References cmsHarvester.CMSHarvester.ref_hist_mappings_file_name.

    def option_handler_ref_hist_mapping_file(self, option, opt_str, value, parser):
        """Store the name of the file for the ref. histogram mapping.

        """

        file_name = value

        if not self.ref_hist_mappings_file_name is None:
            msg = "Only one reference histogram mapping file " \
                  "should be specified"
            self.logger.fatal(msg)
            raise Usage(msg)
        self.ref_hist_mappings_file_name = file_name

        self.logger.info("Reference histogram mapping file " \
                         "to be used: `%s'" % \
                         self.ref_hist_mappings_file_name)

        # End of option_handler_ref_hist_mapping_file.

def cmsHarvester.CMSHarvester.set_output_level	(		self,
			output_level
	)

Adjust the level of output generated.

Choices are:
  - normal  : default level of output
  - quiet   : less output than the default
  - verbose : some additional information
  - debug   : lots more information, may be overwhelming

NOTE: The debug option is a bit special in the sense that it
      also modifies the output format.

Definition at line 619 of file cmsHarvester.py.

Referenced by cmsHarvester.CMSHarvester.option_handler_debug(), and cmsHarvester.CMSHarvester.option_handler_quiet().

    def set_output_level(self, output_level):
        """Adjust the level of output generated.

        Choices are:
          - normal  : default level of output
          - quiet   : less output than the default
          - verbose : some additional information
          - debug   : lots more information, may be overwhelming

        NOTE: The debug option is a bit special in the sense that it
              also modifies the output format.

        """

        # NOTE: These levels are hooked up to the ones used in the
        #       logging module.
        output_levels = {
            "NORMAL"  : logging.INFO,
            "QUIET"   : logging.WARNING,
            "VERBOSE" : logging.INFO,
            "DEBUG"   : logging.DEBUG
            }

        output_level = output_level.upper()

        try:
            # Update the logger.
            self.log_level = output_levels[output_level]
            self.logger.setLevel(self.log_level)
        except KeyError:
            # Show a complaint
            self.logger.fatal("Unknown output level `%s'" % ouput_level)
            # and re-raise an exception.
            raise Exception

        # End of set_output_level.

def cmsHarvester.CMSHarvester.time_stamp

(

self

)

Definition at line 518 of file cmsHarvester.py.

Referenced by cmsHarvester.CMSHarvester.config_file_header().

    def time_stamp(self):
        "Create a timestamp to use in the created config files."

        time_now = datetime.datetime.utcnow()
        # We don't care about the microseconds.
        time_now = time_now.replace(microsecond = 0)
        time_stamp = "%sUTC" % datetime.datetime.isoformat(time_now)

        # End of time_stamp.
        return time_stamp

Member Data Documentation

cmsHarvester.CMSHarvester.book_keeping_file_name

Definition at line 974 of file cmsHarvester.py.

Referenced by cmsHarvester.CMSHarvester.option_handler_book_keeping_file().

cmsHarvester.CMSHarvester.force_running

Definition at line 704 of file cmsHarvester.py.

cmsHarvester.CMSHarvester.frontier_connection_name

Definition at line 354 of file cmsHarvester.py.

Referenced by cmsHarvester.CMSHarvester.option_handler_frontier_connection().

cmsHarvester.CMSHarvester.frontier_connection_overridden

Definition at line 359 of file cmsHarvester.py.

Referenced by cmsHarvester.CMSHarvester.option_handler_frontier_connection().

cmsHarvester.CMSHarvester.globaltag

Definition at line 342 of file cmsHarvester.py.

Referenced by cmsHarvester.CMSHarvester.option_handler_globaltag().

cmsHarvester.CMSHarvester.harvesting_info

Definition at line 366 of file cmsHarvester.py.

cmsHarvester.CMSHarvester.harvesting_mode

Definition at line 381 of file cmsHarvester.py.

Referenced by cmsHarvester.CMSHarvester.option_handler_harvesting_mode().

cmsHarvester.CMSHarvester.harvesting_mode_default

Definition at line 385 of file cmsHarvester.py.

cmsHarvester.CMSHarvester.harvesting_modes

Definition at line 328 of file cmsHarvester.py.

Referenced by cmsHarvester.CMSHarvester.option_handler_harvesting_mode().

cmsHarvester.CMSHarvester.harvesting_type

Definition at line 375 of file cmsHarvester.py.

Referenced by cmsHarvester.CMSHarvester.option_handler_harvesting_type().

cmsHarvester.CMSHarvester.harvesting_types

Definition at line 309 of file cmsHarvester.py.

Referenced by cmsHarvester.CMSHarvester.option_handler_harvesting_type().

cmsHarvester.CMSHarvester.input_method

Definition at line 392 of file cmsHarvester.py.

Referenced by cmsHarvester.CMSHarvester.option_handler_input_spec().

cmsHarvester.CMSHarvester.input_name

Definition at line 401 of file cmsHarvester.py.

Referenced by cmsHarvester.CMSHarvester.option_handler_input_spec().

cmsHarvester.CMSHarvester.Jsonfilename

Definition at line 912 of file cmsHarvester.py.

cmsHarvester.CMSHarvester.Jsonrunfilename

Definition at line 919 of file cmsHarvester.py.

cmsHarvester.CMSHarvester.log_level

Definition at line 646 of file cmsHarvester.py.

cmsHarvester.CMSHarvester.ref_hist_mappings_file_name

Definition at line 995 of file cmsHarvester.py.

Referenced by cmsHarvester.CMSHarvester.option_handler_ref_hist_mapping_file().

cmsHarvester.CMSHarvester.todofile

Definition at line 905 of file cmsHarvester.py.

cmsHarvester.CMSHarvester.use_ref_hists

Definition at line 346 of file cmsHarvester.py.

Referenced by cmsHarvester.CMSHarvester.option_handler_no_ref_hists().

cmsHarvester.CMSHarvester.version

Definition at line 305 of file cmsHarvester.py.

Referenced by python.rootplot.argparse._VersionAction.__call__(), validation.Sample.datasetpattern(), validation.Sample.filename(), argparse.ArgumentParser.format_version(), and python.rootplot.argparse.ArgumentParser.format_version().