Logger

Global logger for fedbiomed

Written above origin Logger class provided by python.

Following features were added from to the original module:

provides a logger instance of FedLogger, which is also a singleton, so it can be used "as is"
provides a dedicated file handler
provides a JSON/gRPC handler (this permit to send error messages from a node to a researcher)
works on python scripts / ipython / notebook
manages a dictionary of handlers. Default keys are 'CONSOLE', 'GRPC', 'FILE', but any key is allowed (only one handler by key)
allow changing log level globally, or on a specific handler (using its key)
log levels can be provided as string instead of logging.* levels (no need to import logging in caller's code) just as in the initial python logger

A typical usage is:

from fedbiomed.common.logger import logger

logger.info("information message")

All methods of the original python logger are provided. To name a few:

logger.debug()
logger.info()
logger.warning()
logger.error()
logger.critical()

Contrary to other Fed-BioMed classes, the API of FedLogger is compliant with the coding conventions used for logger (lowerCameCase)

Dependency issue

Please pay attention to not create dependency loop then importing other fedbiomed package

Attributes

DEFAULT_FORMAT `module-attribute`

DEFAULT_FORMAT = '%(asctime)s %(name)s %(levelname)s - %(message)s'

DEFAULT_LOG_FILE `module-attribute`

DEFAULT_LOG_FILE = 'mylog.log'

DEFAULT_LOG_LEVEL `module-attribute`

DEFAULT_LOG_LEVEL = WARNING

logger `module-attribute`

logger = FedLogger()

Classes

FedLogger

FedLogger(level=DEFAULT_LOG_LEVEL)

Base class for the logger.

It uses python logging module by composition (only log() method is overwritten)

All methods from the logging module can be accessed through the _logger member of the class if necessary (instead of overloading all the methods) (ex: logger._logger.getEffectiveLevel() )

Should not be imported

An initial console logger is installed (so the logger has at minimum one handler)

Parameters:

Name	Type	Description	Default
`level`	`str`	initial loglevel. This loglevel will be the default for all handlers, if called without the default level	`DEFAULT_LOG_LEVEL`

Source code in fedbiomed/common/logger.py

def __init__(self, level: str = DEFAULT_LOG_LEVEL):
    """Constructor of base class

    An initial console logger is installed (so the logger has at minimum one handler)

    Args:
        level: initial loglevel. This loglevel will be the default for all handlers, if called
            without the default level


    """

    # internal tables
    # transform string to logging.level
    self._nameToLevel = {
        "DEBUG": logging.DEBUG,
        "INFO": logging.INFO,
        "WARNING": logging.WARNING,
        "ERROR": logging.ERROR,
        "CRITICAL": logging.CRITICAL,
    }

    # transform logging.level to string
    self._levelToName = {
        logging.DEBUG: "DEBUG",
        logging.INFO: "INFO",
        logging.WARNING: "WARNING",
        logging.ERROR: "ERROR",
        logging.CRITICAL: "CRITICAL"
    }

    # name this logger
    self._logger = logging.getLogger("fedbiomed")

    # Do not propagate (avoids log duplication when third party libraries uses logging module)
    self._logger.propagate = False

    self._default_level = DEFAULT_LOG_LEVEL  # MANDATORY ! KEEP THIS PLEASE !!!
    self._default_level = self._internal_level_translator(level)

    self._logger.setLevel(self._default_level)

    # init the handlers list and add a console handler on startup
    self._handlers = {}
    self.add_console_handler()

    pass

Functions

add_console_handler

add_console_handler(format=DEFAULT_FORMAT, level=DEFAULT_LOG_LEVEL)

Adds a console handler

Parameters:

Name	Type	Description	Default
`format`	`str`	the format string of the logger	`DEFAULT_FORMAT`
`level`	`Any`	initial level of the logger for this handler (optional) if not given, the default level is set	`DEFAULT_LOG_LEVEL`

Source code in fedbiomed/common/logger.py

def add_console_handler(self,
                        format: str = DEFAULT_FORMAT,
                        level: Any = DEFAULT_LOG_LEVEL):

    """Adds a console handler

    Args:
        format: the format string of the logger
        level: initial level of the logger for this handler (optional) if not given, the default level is set
    """
    if is_ipython():
        handler = _IpythonConsoleHandler()
    else:
        handler = logging.StreamHandler()

    handler.setLevel(self._internal_level_translator(level))

    formatter = logging.Formatter(format)
    handler.setFormatter(formatter)
    self._internal_add_handler("CONSOLE", handler)

    pass

add_file_handler

add_file_handler(filename=DEFAULT_LOG_FILE, format=DEFAULT_FORMAT, level=DEFAULT_LOG_LEVEL)

Adds a file handler

Parameters:

Name	Type	Description	Default
`filename`	`str`	File to log to	`DEFAULT_LOG_FILE`
`format`	`str`	Log format	`DEFAULT_FORMAT`
`level`	`any`	Initial level of the logger	`DEFAULT_LOG_LEVEL`

Source code in fedbiomed/common/logger.py

def add_file_handler(
        self,
        filename: str = DEFAULT_LOG_FILE,
        format: str = DEFAULT_FORMAT,
        level: any = DEFAULT_LOG_LEVEL):
    """Adds a file handler

    Args:
        filename: File to log to
        format: Log format
        level: Initial level of the logger
    """

    handler = logging.FileHandler(filename=filename, mode='a')
    handler.setLevel(self._internal_level_translator(level))

    formatter = logging.Formatter(format)
    handler.setFormatter(formatter)

    self._internal_add_handler("FILE", handler)

add_grpc_handler

add_grpc_handler(on_log=None, node_id=None, level=logging.INFO)

Adds a gRPC handler, to publish error message on a topic

Parameters:

Name	Type	Description	Default
`on_log`	`Callable`	Provided by higher level GRPC implementation	`None`
`node_id`	`str`	id of the caller (necessary for msg formatting to the researcher)	`None`
`level`	`Any`	level of this handler (non-mandatory) level must be lower than ERROR to ensure that the research get all ERROR/CRITICAL messages	`INFO`

Source code in fedbiomed/common/logger.py

def add_grpc_handler(self,
                     on_log: Callable = None,
                     node_id: str = None,
                     level: Any = logging.INFO
                     ):

    """Adds a gRPC handler, to publish error message on a topic

    Args:
        on_log: Provided by higher level GRPC implementation
        node_id: id of the caller (necessary for msg formatting to the researcher)
        level: level of this handler (non-mandatory) level must be lower than ERROR to ensure that the
            research get all ERROR/CRITICAL messages
    """

    handler = _GrpcHandler(
        on_log=on_log,
        node_id=node_id,
    )

    # may be not necessary ?
    handler.setLevel(self._internal_level_translator(level))
    formatter = _GrpcFormatter(node_id)

    handler.setFormatter(formatter)
    self._internal_add_handler("GRPC", handler)

    # as a side effect this will set the minimal level to ERROR
    self.setLevel(level, "GRPC")

    pass

critical

critical(msg, *args, broadcast=False, researcher_id=None, **kwargs)

Same as info message

Source code in fedbiomed/common/logger.py

def critical(self, msg, *args, broadcast=False, researcher_id=None, **kwargs):
    """Same as info message"""
    self._logger.critical(msg, *args, **kwargs,
                          extra={"researcher_id": researcher_id, 'broadcast': broadcast})

debug

debug(msg, *args, broadcast=False, researcher_id=None, **kwargs)

Same as info message

Source code in fedbiomed/common/logger.py

def debug(self, msg, *args, broadcast=False, researcher_id=None, **kwargs):
    """Same as info message"""
    self._logger.debug(msg, *args, **kwargs,
                       extra={"researcher_id": researcher_id, 'broadcast': broadcast})

error

error(msg, *args, broadcast=False, researcher_id=None, **kwargs)

Same as info message

Source code in fedbiomed/common/logger.py

def error(self, msg, *args, broadcast=False, researcher_id=None, **kwargs):
    """Same as info message"""
    self._logger.error(msg, *args, **kwargs,
                       extra={"researcher_id": researcher_id, 'broadcast': broadcast})

info

info(msg, *args, broadcast=False, researcher_id=None, **kwargs)

Extends arguments of info message.

Valid only GrpcHandler is existing

Parameters:

Name	Description	Default
`msg`	Message to log	required
`broadcast`	Broadcast message to all available researchers	`False`
`researcher_id`	ID of the researcher that the message will be sent. If broadcast True researcher id will be ignored	`None`

Source code in fedbiomed/common/logger.py

def info(self, msg, *args, broadcast=False, researcher_id=None, **kwargs):
    """Extends arguments of info message.

    Valid only GrpcHandler is existing

    Args:
        msg: Message to log
        broadcast: Broadcast message to all available researchers
        researcher_id: ID of the researcher that the message will be sent.
            If broadcast True researcher id will be ignored
    """
    self._logger.info(msg, *args, **kwargs,
                      extra={"researcher_id": researcher_id, 'broadcast': broadcast})

log

log(level, msg)

Overrides the logging.log() method to allow the use of string instead of a logging.* level

Source code in fedbiomed/common/logger.py

def log(self, level: Any, msg: str):
    """Overrides the logging.log() method to allow the use of string instead of a logging.* level """

    level = logger._internal_level_translator(level)
    self._logger.log(
        level,
        msg
    )

setLevel

setLevel(level, htype=None)

Overrides the setLevel method, to deal with level given as a string and to change le level of one or all known handlers

This also change the default level for all future handlers.

Remark

Level should not be lower than CRITICAL (meaning CRITICAL errors are always displayed)

Example:

setLevel( logging.DEBUG, 'FILE')

Parameters:

Name	Type	Description	Default
`level`		level to modify, can be a string or a logging.* level (mandatory)	required
`htype`		if provided (non-mandatory), change the level of the given handler. if not provided (or None), change the level of all known handlers	`None`

Source code in fedbiomed/common/logger.py

def setLevel(self, level: Any, htype: Any = None):
    """Overrides the setLevel method, to deal with level given as a string and to change le level of
    one or all known handlers

    This also change the default level for all future handlers.

    !!! info "Remark"

        Level should not be lower than CRITICAL (meaning CRITICAL errors are always displayed)

        Example:
        ```python
        setLevel( logging.DEBUG, 'FILE')
        ```

    Args:
        level : level to modify, can be a string or a logging.* level (mandatory)
        htype : if provided (non-mandatory), change the level of the given handler. if not  provided (or None),
            change the level of all known handlers
    """

    level = self._internal_level_translator(level)

    if htype is None:
        # store this level (for future handler adding)
        self._logger.setLevel(level)

        for h in self._handlers:
            self._handlers[h].setLevel(level)
        return

    if htype in self._handlers:
        self._handlers[htype].setLevel(level)
        return

    # htype provided but no handler for this type exists
    self._logger.warning(htype + " handler not initialized yet")

warning

warning(msg, *args, broadcast=False, researcher_id=None, **kwargs)

Same as info message

Source code in fedbiomed/common/logger.py

def warning(self, msg, *args, broadcast=False, researcher_id=None, **kwargs):
    """Same as info message"""
    self._logger.warning(msg, *args, **kwargs,
                         extra={"researcher_id": researcher_id, 'broadcast': broadcast})

Attributes

DEFAULT_FORMAT module-attribute

DEFAULT_LOG_FILE module-attribute

DEFAULT_LOG_LEVEL module-attribute

logger module-attribute

Classes

FedLogger

Functions

add_console_handler

add_file_handler

add_grpc_handler

critical

debug

error

info

log

setLevel

warning

Functions

DEFAULT_FORMAT `module-attribute`

DEFAULT_LOG_FILE `module-attribute`

DEFAULT_LOG_LEVEL `module-attribute`

logger `module-attribute`