comect.sidecar package

Comect Sidecar - an easy, fast library for creating logging sidecars for Docker environments.

To get started, take a look at the Sidecar class below.

class comect.sidecar.Sidecar

Bases: object

Logging sidecar class that aims to enable easier log collection and processing.

This is designed for use with Nuitka to keep things fast.

This is a threaded sidecar - one thread per file - which should help to keep things responsive.

Sidecar instances track a set of files and each line is fed through a pipeline. Pipelines consist of a FilterType, a tuple of ParserType, and a ProcessorType. Each line is given to these in order - filters decide whether the line should be processed by the current pipeline, parsers transform the line in some way - either by modifying the string or returning a different object entirely - and processors are in charge of deciding what to do with the parsed line.

For more information on filters, parsers and processors, please see their respective modules.

add_file(path)

Add a file to keep track of. You may only add a file once - no duplicates!

Parameters

path (Union[Path, str]) – A Path object or a string (to be converted to a Path) pointing to the file

Return type

bool

Returns

True if the file was added (no duplicate), False otherwise

add_pipeline(name, pipeline)

Add a pipeline to this Sidecar instance.

Pipelines operate on every line emitted from each tracked logfile. Note: You may not have two pipelines with the same name.

Parameters

  • name (str) – The name of the pipeline to add

  • pipeline (Pipeline) – the pipeline tuple to add

Return type

bool

Returns

True if the pipeline was added (no duplicate name), False otherwise

block()

If the threads have been started, this method will block until they’ve all stopped.

You can use this to occupy a thread if you need to.

get_pipeline(name)

Get a pipeline by name.

Parameters

name (str) – The name of the pipeline to retrieve

Returns

The pipeline tuple, or None if it doesn’t exist

Return type

Optional[Pipeline]

remove_file(path)

Remove a file that this sidecar currently tracks.

Parameters

path (Union[Path, str]) – A Path object or a string (to be converted to a Path) pointing to the file

Return type

bool

Returns

True if the file was removed (it exists), False otherwise

remove_pipeline(name)

Remove a pipeline from this Sidecar instance.

Parameters

name (str) – The name of the pipeline to remove

Return type

bool

Returns

True if the pipeline existed (and was removed), False otherwise

start()

Start processing lines.

This starts the thread created for each file, which will begin the process of emitting lines to the pipelines.

This method does not block.

stop()

Attempt to stop the Sidecar.

This will remove all of the files from this Sidecar (and thus tell them to stop tailing), and make block() return. The threads used for tailing the logfiles are daemon threads and will be destroyed as soon as all other threads have stopped, but they cannot be forced to stop. This simply asks them to stop as soon as any more log data comes through.