comect.sidecar.processors package

class comect.sidecar.processors.Processor

Bases: abc.ABC

Processors are in charge of handling previously parsed data.

Buffer it, ship it off to a webservice, or whatever you need. As usual, you could use a function instead of a class instance if that makes more sense for your project.

If you’re subclassing, then you must implement __call__().

abstract __call__(line, data, path)

Perform final processing of the data - ship it off to a database, for example.

Parameters

  • line (Union[str, bytes]) – The original unprocessed line

  • path (Path) – The path to the file the line is from

  • data (Any) – The data to be processed

class comect.sidecar.processors.HTTPProcessor(url, *, method='POST', use_json=False)

Bases: comect.sidecar.processors.base.Processor

A processor that takes processed lines and sends them off to some HTTP server.

This processor requires Requests to be installed. Install comect-sidecar[http] to also grab the version of Requests that is used to test Sidecar.

Configure this processor using its parameters, as described below. Note that combining the GET HTTP method with the use_json option is unsupported and will raise a ValueError early. Any other exceptions will be raised by Requests.

This is a naiive implementation, and you may want to write your own HTTP processor for anything more specialised than very basic data submission. Feel free to browse the source of this class if you’re curious just how basic it really is!

Parameters

  • url (str) – The URL to send the request to

  • method (str) – The HTTP method to use

  • use_json (bool) – Whether the data should be considered JSON data