Logger

Global logger for fedbiomed

Written above origin Logger class provided by python.

Following features were added from to the original module:

provides a logger instance of FedLogger, which is also a singleton, so it can be used "as is"
provides a dedicated file handler
provides a JSON/MQTT handler: all messages with priority greater than error are sent to the MQQT handler (this permit to send error messages from a node to a researcher)
works on python scripts / ipython / notebook
manages a dictionary of handlers. Default keys are 'CONSOLE', 'MQTT', 'FILE', but any key is allowed (only one handler by key)
allow changing log level globally, or on a specific handler (using its key)
log levels can be provided as string instead of logging.* levels (no need to import logging in caller's code) just as in the initial python logger

A typical usage is:

from fedbiomed.common.logger import logger

logger.info("information message")

All methods of the original python logger are provided. To name a few:

logger.debug()
logger.info()
logger.warning()
logger.error()
logger.critical()

Contrary to other Fed-BioMed classes, the API of FedLogger is compliant with the coding conventions used for logger (lowerCameCase)

Dependency issue

Please pay attention to not create dependency loop then importing other fedbiomed package

Attributes

DEFAULT_LOG_FILE `module-attribute`

DEFAULT_LOG_FILE = 'mylog.log'

DEFAULT_LOG_LEVEL `module-attribute`

DEFAULT_LOG_LEVEL = logging.WARNING

DEFAULT_LOG_TOPIC `module-attribute`

DEFAULT_LOG_TOPIC = 'general/logger'

logger `module-attribute`

logger = FedLogger()

Classes

FedLogger

FedLogger(level=DEFAULT_LOG_LEVEL)

Base class for the logger. it uses python logging module by composition (only log() method is overwritten)

All methods from the logging module can be accessed through the _logger member of the class if necessary (instead of overloading all the methods) (ex: logger._logger.getEffectiveLevel() )

Should not be imported

An initial console logger is installed (so the logger has at minimum one handler)

Parameters:

Name	Type	Description	Default
`level`	`str`	initial loglevel. This loglevel will be the default for all handlers, if called without the default level	`DEFAULT_LOG_LEVEL`

Source code in fedbiomed/common/logger.py

def __init__(self, level: str = DEFAULT_LOG_LEVEL):
    """Constructor of base class

    An initial console logger is installed (so the logger has at minimum one handler)

    Args:
        level: initial loglevel. This loglevel will be the default for all handlers, if called
            without the default level


    """

    # internal tables
    # transform string to logging.level
    self._nameToLevel = {
        "DEBUG": logging.DEBUG,
        "INFO": logging.INFO,
        "WARNING": logging.WARNING,
        "ERROR": logging.ERROR,
        "CRITICAL": logging.CRITICAL,
    }

    # transform logging.level to string
    self._levelToName = {
        logging.DEBUG: "DEBUG",
        logging.INFO: "INFO",
        logging.WARNING: "WARNING",
        logging.ERROR: "ERROR",
        logging.CRITICAL: "CRITICAL"
    }

    # name this logger
    self._logger = logging.getLogger("fedbiomed")

    # Do not propagate (avoids log duplication when third party libraries uses logging module)
    self._logger.propagate = False

    self._default_level = DEFAULT_LOG_LEVEL  # MANDATORY ! KEEP THIS PLEASE !!!
    self._default_level = self._internalLevelTranslator(level)

    self._logger.setLevel(self._default_level)

    # init the handlers list and add a console handler on startup
    self._handlers = {}
    self.addConsoleHandler()

    pass

Functions

addConsoleHandler

addConsoleHandler(format='%(asctime)s %(name)s %(levelname)s - %(message)s', level=DEFAULT_LOG_LEVEL)

Adds a console handler

Parameters:

Name	Type	Description	Default
`format`	`str`	the format string of the logger	`'%(asctime)s %(name)s %(levelname)s - %(message)s'`
`level`	`Any`	initial level of the logger for this handler (optional) if not given, the default level is set	`DEFAULT_LOG_LEVEL`

Source code in fedbiomed/common/logger.py

def addConsoleHandler(self,
                      format: str = '%(asctime)s %(name)s %(levelname)s - %(message)s',
                      level: Any = DEFAULT_LOG_LEVEL):

    """Adds a console handler

    Args:
        format: the format string of the logger
        level: initial level of the logger for this handler (optional) if not given, the default level is set
    """

    handler = logging.StreamHandler()

    handler.setLevel(self._internalLevelTranslator(level))

    formatter = logging.Formatter(format)
    handler.setFormatter(formatter)
    self._internalAddHandler("CONSOLE", handler)

    pass

addFileHandler

addFileHandler(filename=DEFAULT_LOG_FILE, format='%(asctime)s %(name)s %(levelname)s - %(message)s', level=DEFAULT_LOG_LEVEL)

Adds a file handler

Parameters:

Name	Type	Description	Default
`filename`	`str`	File to log to	`DEFAULT_LOG_FILE`
`format`	`str`	Log format	`'%(asctime)s %(name)s %(levelname)s - %(message)s'`
`level`	`any`	Initial level of the logger (optionnal)	`DEFAULT_LOG_LEVEL`

Source code in fedbiomed/common/logger.py

def addFileHandler(self,
                   filename: str = DEFAULT_LOG_FILE,
                   format: str = '%(asctime)s %(name)s %(levelname)s - %(message)s',
                   level: any = DEFAULT_LOG_LEVEL):
    """Adds a file handler

    Args:
        filename: File to log to
        format: Log format
        level: Initial level of the logger (optionnal)
    """

    handler = logging.FileHandler(filename=filename, mode='a')
    handler.setLevel(self._internalLevelTranslator(level))

    formatter = logging.Formatter(format)
    handler.setFormatter(formatter)

    self._internalAddHandler("FILE", handler)
    pass

addMqttHandler

addMqttHandler(mqtt=None, node_id=None, topic=DEFAULT_LOG_TOPIC, level=logging.ERROR)

Adds a mqtt handler, to publish error message on a topic

Parameters:

Name	Type	Description	Default
`mqtt`	`Any`	already opened MQTT object	`None`
`node_id`	`str`	id of the caller (necessary for msg formatting to the researcher)	`None`
`topic`	`Any`	topic to publish to (non-mandatory)	`DEFAULT_LOG_TOPIC`
`level`	`Any`	level of this handler (non-mandatory) level must be lower than ERROR to ensure that the research get all ERROR/CRITICAL messages	`ERROR`

Source code in fedbiomed/common/logger.py

def addMqttHandler(self,
                   mqtt: Any = None,
                   node_id: str = None,
                   topic: Any = DEFAULT_LOG_TOPIC,
                   level: Any = logging.ERROR
                   ):

    """Adds a mqtt handler, to publish error message on a topic

    Args:
        mqtt: already opened MQTT object
        node_id: id of the caller (necessary for msg formatting to the researcher)
        topic: topic to publish to (non-mandatory)
        level: level of this handler (non-mandatory) level must be lower than ERROR to ensure that the
            research get all ERROR/CRITICAL messages
    """

    handler = _MqttHandler(
        mqtt=mqtt,
        node_id=node_id,
        topic=topic
    )

    # may be not necessary ?
    handler.setLevel(self._internalLevelTranslator(level))
    formatter = _MqttFormatter(node_id)

    handler.setFormatter(formatter)
    self._internalAddHandler("MQTT", handler)

    # as a side effect this will set the minimal level to ERROR
    self.setLevel(level, "MQTT")
    pass

delMqttHandler

delMqttHandler()

Source code in fedbiomed/common/logger.py

def delMqttHandler(self):
    self._internalAddHandler("MQTT", None)

log

log(level, msg)

Overrides the logging.log() method to allow the use of string instead of a logging.* level

Source code in fedbiomed/common/logger.py

def log(self, level: Any, msg: str):
    """Overrides the logging.log() method to allow the use of string instead of a logging.* level """

    level = logger._internalLevelTranslator(level)
    self._logger.log(
        level,
        msg
    )

setLevel

setLevel(level, htype=None)

Overrides the setLevel method, to deal with level given as a string and to change le level of one or all known handlers

This also change the default level for all future handlers.

Remark

Level should not be lower than CRITICAL (meaning CRITICAL errors are always displayed)

Example:

setLevel( logging.DEBUG, 'FILE')

Parameters:

Name	Type	Description	Default
`level`		level to modify, can be a string or a logging.* level (mandatory)	required
`htype`		if provided (non-mandatory), change the level of the given handler. if not provided (or None), change the level of all known handlers	`None`

Source code in fedbiomed/common/logger.py

def setLevel(self, level: Any, htype: Any = None):
    """Overrides the setLevel method, to deal with level given as a string and to change le level of
    one or all known handlers

    This also change the default level for all future handlers.

    !!! info "Remark"

        Level should not be lower than CRITICAL (meaning CRITICAL errors are always displayed)

        Example:
        ```python
        setLevel( logging.DEBUG, 'FILE')
        ```

    Args:
        level : level to modify, can be a string or a logging.* level (mandatory)
        htype : if provided (non-mandatory), change the level of the given handler. if not  provided (or None),
            change the level of all known handlers
    """

    level = self._internalLevelTranslator(level)

    if htype is None:
        # store this level (for future handler adding)
        self._logger.setLevel(level)

        for h in self._handlers:
            self._handlers[h].setLevel(level)
        return

    if htype in self._handlers:
        self._handlers[htype].setLevel(level)
        return

    # htype provided but no handler for this type exists
    self._logger.warning(htype + " handler not initialized yet")

Attributes

DEFAULT_LOG_FILE module-attribute

DEFAULT_LOG_LEVEL module-attribute

DEFAULT_LOG_TOPIC module-attribute

logger module-attribute

Classes

FedLogger

Functions

addConsoleHandler

addFileHandler

addMqttHandler

delMqttHandler

log

setLevel

DEFAULT_LOG_FILE `module-attribute`

DEFAULT_LOG_LEVEL `module-attribute`

DEFAULT_LOG_TOPIC `module-attribute`

logger `module-attribute`