HTTP.RequestManager Class Reference

Public Member Functions
def	__init__
def	process
def	put

Public Attributes
	cm
	free
	handles
	queue
	request_error
	request_init
	request_respond

Private Member Functions
def	_request_error
def	_request_init
def	_request_respond

Detailed Description

Manager of multiple concurrent or overlapping HTTP requests.

This is a utility class acting as a pump of several overlapping
HTTP requests against any number of HTTP or HTTPS servers. It
uses a configurable number of simultaneous connections, ten by
default. The actual connection layer is handled using curl, and
the client classes need to aware of this to a limited degree.

The client supplies optional callback methods for initialising,
responding and handling errors on connections. At the very least
the request response callback should be defined.

This class is not designed for multi-threaded use. It employs
overlapping requests, but in a single thread. Only one thread
at a time should be calling `process()`; several threads may
call `.put()` provided the caller uses a mutex so that only one
thread calls into the method at a time.

Definition at line 5 of file HTTP.py.

Constructor & Destructor Documentation

def HTTP.RequestManager.__init__	(		self,
			num_connections = 10,
			ssl_opts = None,
			user_agent = None,
			request_headers = None,
			request_init = None,
			request_respond = None,
			request_error = None,
			handle_init = None
	)

Initialise the request manager. The arguments are:

:arg num_connections: maximum number of simultaneous connections.
:arg ssl_opts: optional SSLOptions (Monitoring.Core.X509) for SSL
X509 parametre values, e.g. for X509 client authentication.
:arg user_agent: sets user agent identification string if defined.
:arg request_headers: if defined, specifies list of additional HTTP
request headers to be added to each request.
:arg request_init: optional callback to initialise requests; the
default assumes each task is a URL to access and sets the `URL`
property on the curl object to the task value.
:arg request_respond: callback for handling responses; at the very
minimum this should be defined as the default one does nothing.
:arg request_error: callback for handling connection errors; the
default one raises a RuntimeException.
:arg handle_init: callback for customising connection handles at
creation time; the callback will be invoked for each connection
object as it's created and queued to the idle connection list.

Definition at line 27 of file HTTP.py.

               request_error = None, handle_init = None):
    """Initialise the request manager. The arguments are:

:arg num_connections: maximum number of simultaneous connections.
:arg ssl_opts: optional SSLOptions (Monitoring.Core.X509) for SSL
X509 parametre values, e.g. for X509 client authentication.
:arg user_agent: sets user agent identification string if defined.
:arg request_headers: if defined, specifies list of additional HTTP
request headers to be added to each request.
:arg request_init: optional callback to initialise requests; the
default assumes each task is a URL to access and sets the `URL`
property on the curl object to the task value.
:arg request_respond: callback for handling responses; at the very
minimum this should be defined as the default one does nothing.
:arg request_error: callback for handling connection errors; the
default one raises a RuntimeException.
:arg handle_init: callback for customising connection handles at
creation time; the callback will be invoked for each connection
object as it's created and queued to the idle connection list."""
    self.request_respond = request_respond or self._request_respond
    self.request_error = request_error or self._request_error
    self.request_init = request_init or self._request_init
    self.cm = CurlMulti()
    self.handles = [Curl() for i in range(0, num_connections)]
    self.free = [c for c in self.handles]
    self.queue = []

    for c in self.handles:
      c.buffer = None
      c.setopt(NOSIGNAL, 1)
      c.setopt(TIMEOUT, 300)
      c.setopt(CONNECTTIMEOUT, 30)
      c.setopt(FOLLOWLOCATION, 1)
      c.setopt(MAXREDIRS, 5)
      if user_agent:
        c.setopt(USERAGENT, user_agent)
      if ssl_opts:
        c.setopt(CAPATH, ssl_opts.ca_path)
        c.setopt(SSLCERT, ssl_opts.cert_file)
        c.setopt(SSLKEY, ssl_opts.key_file)
        if ssl_opts.key_pass:
          c.setopt(SSLKEYPASSWD, ssl_opts.key_pass)
      if request_headers:
        c.setopt(HTTPHEADER, request_headers)
      if handle_init:
        handle_init(c)