Logger

Global logger for fedbiomed

Written above origin Logger class provided by python.

Following features were added from to the original module:

  • provides a logger instance of FedLogger, which is also a singleton, so it can be used "as is"
  • provides a dedicated file handler
  • provides a JSON/gRPC handler (this permit to send error messages from a node to a researcher)
  • works on python scripts / ipython / notebook
  • manages a dictionary of handlers. Default keys are 'CONSOLE', 'GRPC', 'FILE', but any key is allowed (only one handler by key)
  • allow changing log level globally, or on a specific handler (using its key)
  • log levels can be provided as string instead of logging.* levels (no need to import logging in caller's code) just as in the initial python logger

A typical usage is:

from fedbiomed.common.logger import logger

logger.info("information message")

All methods of the original python logger are provided. To name a few:

  • logger.debug()
  • logger.info()
  • logger.warning()
  • logger.error()
  • logger.critical()

Contrary to other Fed-BioMed classes, the API of FedLogger is compliant with the coding conventions used for logger (lowerCameCase)

Dependency issue

Please pay attention to not create dependency loop then importing other fedbiomed package

Attributes

DEFAULT_FORMAT module-attribute

DEFAULT_FORMAT = '%(asctime)s %(name)s %(levelname)s - %(message)s'

DEFAULT_LOG_FILE module-attribute

DEFAULT_LOG_FILE = 'mylog.log'

DEFAULT_LOG_LEVEL module-attribute

DEFAULT_LOG_LEVEL = WARNING

logger module-attribute

logger = FedLogger()

Classes

FedLogger

FedLogger(level=DEFAULT_LOG_LEVEL)

Base class for the logger.

It uses python logging module by composition (only log() method is overwritten)

All methods from the logging module can be accessed through the _logger member of the class if necessary (instead of overloading all the methods) (ex: logger._logger.getEffectiveLevel() )

Should not be imported

An initial console logger is installed (so the logger has at minimum one handler)

Parameters:

Name Type Description Default
level str

initial loglevel. This loglevel will be the default for all handlers, if called without the default level

DEFAULT_LOG_LEVEL
Source code in fedbiomed/common/logger.py
def __init__(self, level: str = DEFAULT_LOG_LEVEL):
    """Constructor of base class

    An initial console logger is installed (so the logger has at minimum one handler)

    Args:
        level: initial loglevel. This loglevel will be the default for all handlers, if called
            without the default level


    """

    # internal tables
    # transform string to logging.level
    self._nameToLevel = {
        "DEBUG": logging.DEBUG,
        "INFO": logging.INFO,
        "WARNING": logging.WARNING,
        "ERROR": logging.ERROR,
        "CRITICAL": logging.CRITICAL,
    }

    # transform logging.level to string
    self._levelToName = {
        logging.DEBUG: "DEBUG",
        logging.INFO: "INFO",
        logging.WARNING: "WARNING",
        logging.ERROR: "ERROR",
        logging.CRITICAL: "CRITICAL"
    }

    # name this logger
    self._logger = logging.getLogger("fedbiomed")

    # Do not propagate (avoids log duplication when third party libraries uses logging module)
    self._logger.propagate = False

    self._default_level = DEFAULT_LOG_LEVEL  # MANDATORY ! KEEP THIS PLEASE !!!
    self._default_level = self._internal_level_translator(level)

    self._logger.setLevel(self._default_level)

    # init the handlers list and add a console handler on startup
    self._handlers = {}
    self.add_console_handler()

    pass

Functions

add_console_handler
add_console_handler(format=DEFAULT_FORMAT, level=DEFAULT_LOG_LEVEL)

Adds a console handler

Parameters:

Name Type Description Default
format str

the format string of the logger

DEFAULT_FORMAT
level Any

initial level of the logger for this handler (optional) if not given, the default level is set

DEFAULT_LOG_LEVEL
Source code in fedbiomed/common/logger.py
def add_console_handler(self,
                        format: str = DEFAULT_FORMAT,
                        level: Any = DEFAULT_LOG_LEVEL):

    """Adds a console handler

    Args:
        format: the format string of the logger
        level: initial level of the logger for this handler (optional) if not given, the default level is set
    """
    if is_ipython():
        handler = _IpythonConsoleHandler()
    else:
        handler = logging.StreamHandler()

    handler.setLevel(self._internal_level_translator(level))

    formatter = logging.Formatter(format)
    handler.setFormatter(formatter)
    self._internal_add_handler("CONSOLE", handler)

    pass
add_file_handler
add_file_handler(filename=DEFAULT_LOG_FILE, format=DEFAULT_FORMAT, level=DEFAULT_LOG_LEVEL)

Adds a file handler

Parameters:

Name Type Description Default
filename str

File to log to

DEFAULT_LOG_FILE
format str

Log format

DEFAULT_FORMAT
level any

Initial level of the logger

DEFAULT_LOG_LEVEL
Source code in fedbiomed/common/logger.py
def add_file_handler(
        self,
        filename: str = DEFAULT_LOG_FILE,
        format: str = DEFAULT_FORMAT,
        level: any = DEFAULT_LOG_LEVEL):
    """Adds a file handler

    Args:
        filename: File to log to
        format: Log format
        level: Initial level of the logger
    """

    handler = logging.FileHandler(filename=filename, mode='a')
    handler.setLevel(self._internal_level_translator(level))

    formatter = logging.Formatter(format)
    handler.setFormatter(formatter)

    self._internal_add_handler("FILE", handler)
add_grpc_handler
add_grpc_handler(on_log=None, node_id=None, level=logging.INFO)

Adds a gRPC handler, to publish error message on a topic

Parameters:

Name Type Description Default
on_log Callable

Provided by higher level GRPC implementation

None
node_id str

id of the caller (necessary for msg formatting to the researcher)

None
level Any

level of this handler (non-mandatory) level must be lower than ERROR to ensure that the research get all ERROR/CRITICAL messages

INFO
Source code in fedbiomed/common/logger.py
def add_grpc_handler(self,
                     on_log: Callable = None,
                     node_id: str = None,
                     level: Any = logging.INFO
                     ):

    """Adds a gRPC handler, to publish error message on a topic

    Args:
        on_log: Provided by higher level GRPC implementation
        node_id: id of the caller (necessary for msg formatting to the researcher)
        level: level of this handler (non-mandatory) level must be lower than ERROR to ensure that the
            research get all ERROR/CRITICAL messages
    """

    handler = _GrpcHandler(
        on_log=on_log,
        node_id=node_id,
    )

    # may be not necessary ?
    handler.setLevel(self._internal_level_translator(level))
    formatter = _GrpcFormatter(node_id)

    handler.setFormatter(formatter)
    self._internal_add_handler("GRPC", handler)

    # as a side effect this will set the minimal level to ERROR
    self.setLevel(level, "GRPC")

    pass
critical
critical(msg, *args, broadcast=False, researcher_id=None, **kwargs)

Same as info message

Source code in fedbiomed/common/logger.py
def critical(self, msg, *args, broadcast=False, researcher_id=None, **kwargs):
    """Same as info message"""
    self._logger.critical(msg, *args, **kwargs,
                          extra={"researcher_id": researcher_id, 'broadcast': broadcast})
debug
debug(msg, *args, broadcast=False, researcher_id=None, **kwargs)

Same as info message

Source code in fedbiomed/common/logger.py
def debug(self, msg, *args, broadcast=False, researcher_id=None, **kwargs):
    """Same as info message"""
    self._logger.debug(msg, *args, **kwargs,
                       extra={"researcher_id": researcher_id, 'broadcast': broadcast})
error
error(msg, *args, broadcast=False, researcher_id=None, **kwargs)

Same as info message

Source code in fedbiomed/common/logger.py
def error(self, msg, *args, broadcast=False, researcher_id=None, **kwargs):
    """Same as info message"""
    self._logger.error(msg, *args, **kwargs,
                       extra={"researcher_id": researcher_id, 'broadcast': broadcast})
info
info(msg, *args, broadcast=False, researcher_id=None, **kwargs)

Extends arguments of info message.

Valid only GrpcHandler is existing

Parameters:

Name Type Description Default
msg

Message to log

required
broadcast

Broadcast message to all available researchers

False
researcher_id

ID of the researcher that the message will be sent. If broadcast True researcher id will be ignored

None
Source code in fedbiomed/common/logger.py
def info(self, msg, *args, broadcast=False, researcher_id=None, **kwargs):
    """Extends arguments of info message.

    Valid only GrpcHandler is existing

    Args:
        msg: Message to log
        broadcast: Broadcast message to all available researchers
        researcher_id: ID of the researcher that the message will be sent.
            If broadcast True researcher id will be ignored
    """
    self._logger.info(msg, *args, **kwargs,
                      extra={"researcher_id": researcher_id, 'broadcast': broadcast})
log
log(level, msg)

Overrides the logging.log() method to allow the use of string instead of a logging.* level

Source code in fedbiomed/common/logger.py
def log(self, level: Any, msg: str):
    """Overrides the logging.log() method to allow the use of string instead of a logging.* level """

    level = logger._internal_level_translator(level)
    self._logger.log(
        level,
        msg
    )
setLevel
setLevel(level, htype=None)

Overrides the setLevel method, to deal with level given as a string and to change le level of one or all known handlers

This also change the default level for all future handlers.

Remark

Level should not be lower than CRITICAL (meaning CRITICAL errors are always displayed)

Example:

setLevel( logging.DEBUG, 'FILE')

Parameters:

Name Type Description Default
level

level to modify, can be a string or a logging.* level (mandatory)

required
htype

if provided (non-mandatory), change the level of the given handler. if not provided (or None), change the level of all known handlers

None
Source code in fedbiomed/common/logger.py
def setLevel(self, level: Any, htype: Any = None):
    """Overrides the setLevel method, to deal with level given as a string and to change le level of
    one or all known handlers

    This also change the default level for all future handlers.

    !!! info "Remark"

        Level should not be lower than CRITICAL (meaning CRITICAL errors are always displayed)

        Example:
        ```python
        setLevel( logging.DEBUG, 'FILE')
        ```

    Args:
        level : level to modify, can be a string or a logging.* level (mandatory)
        htype : if provided (non-mandatory), change the level of the given handler. if not  provided (or None),
            change the level of all known handlers
    """

    level = self._internal_level_translator(level)

    if htype is None:
        # store this level (for future handler adding)
        self._logger.setLevel(level)

        for h in self._handlers:
            self._handlers[h].setLevel(level)
        return

    if htype in self._handlers:
        self._handlers[htype].setLevel(level)
        return

    # htype provided but no handler for this type exists
    self._logger.warning(htype + " handler not initialized yet")
warning
warning(msg, *args, broadcast=False, researcher_id=None, **kwargs)

Same as info message

Source code in fedbiomed/common/logger.py
def warning(self, msg, *args, broadcast=False, researcher_id=None, **kwargs):
    """Same as info message"""
    self._logger.warning(msg, *args, **kwargs,
                         extra={"researcher_id": researcher_id, 'broadcast': broadcast})

Functions