使用来自多个模块的python日志记录并写入文件和RotatingFileHandler

问题描述

我正在使用以下模块在我的模块中记录事件.我这样称呼它:

I'm using the below module to log events in my modules. I call it as follows:

模块1

from tools.debug_logger import debug_logger
self.logger = debug_logger().start_logger('module1')
self.logger.debug("Top left corner found")

module2:

from tools.debug_logger import debug_logger
self.logger = debug_logger().start_logger('module2')
self.logger.debug("Top left corner found")

，这里是文件/tools/debug_logger.py

and here the file /tools/debug_logger.py

import logging, logging.handlers
import sys
class debug_logger(object):
    def start_logger(self,name):
        logger = logging.getLogger(name)
        logger.setLevel(logging.DEBUG)
        if not len(logger.handlers):
            fh = logging.handlers.RotatingFileHandler('log/pokerprogram.log', maxBytes=1000000, backupCount=10)
            fh.setLevel(logging.DEBUG)
            fh2 = logging.handlers.RotatingFileHandler('log/pokerprogram_info_only.log', maxBytes=1000000, backupCount=5)
            fh2.setLevel(logging.INFO)
            er = logging.handlers.RotatingFileHandler('log/errors.log', maxBytes=2000000, backupCount=2)
            er.setLevel(logging.WARNING)
            ch = logging.StreamHandler(sys.stdout)
            ch.setLevel(1)
            fh.setFormatter(logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s'))
            fh2.setFormatter(logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s'))
            er.setFormatter(logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s'))
            ch.setFormatter(logging.Formatter('%(name)s - %(levelname)s - %(message)s'))
            logger.addHandler(fh)
            logger.addHandler(fh2)
            logger.addHandler(ch)
            logger.addHandler(er)
        return logger

一切正常，我得到了各个级别的日志文件，但是当调用RotatingFileHandler时，有时会出错.就像各种实例都希望同时进行轮换一样，尽管我非常确定这不会发生，因为我确保只有一个处理程序带有if not len(logger.handlers)，如此处建议的那样:.

Everything works fine and I get log files for the respective levels, but when the RotatingFileHandler is called I sometimes get an error. It is as if various instances want to do the rotation at the same time, even though I'm quite sure this shouldn't happen as I make sure there's only 1 handler with if not len(logger.handlers) as recommended here: Duplicate log output when using Python logging module.

任何建议都会在轮换期间引起此文件访问冲突.

Any suggestion what could be causing this file access violation during the rotation would be appreciated.

PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\Nicolas\\Dropbox\\PythonProjects\\Poker\\log\\pokerprogram.log' -> 'C:\\Users\\Nicolas\\Dropbox\\PythonProjects\\Poker\\log\\pokerprogram.log.1'

我相信会发生权限错误，因为发生轮换时，其他模块仍在写入文件.

I believe the permission error occur because when the rotation occurs other modules are still writing to the file.

当我写入文件并使用此RotatingFileHandler时，从多个模块进行日志记录的最佳方法是什么?有什么最佳实践吗?

What would be the best way to do logging from multiple modules when I write to a file and use this RotatingFileHandler? Is there any best practice?

推荐答案

我相信您输入的日志记录设置有误.推荐的设置日志记录的方法是不定义任何处理程序或将级别记录到模块中，而是在主文件中定义所有配置.

I believe you are getting your logging setup wrong. The recommended way to setup logging is to not define any handlers nor logging levels into the modules, but define all the configuration in the main file.

例如在module1.py中:

import logging

logger = logging.getLogger(__name__)

# use logger.info/logger.debug etc.

在module2.py中，您输入了完全相同的代码:

In module2.py you put exactly the same code:

import logging

logger = logging.getLogger(__name__)

# use logger.info/logger.debug etc.

请注意，__name__是模块名称，因此它将类似于package.module1或package.module2.使用点名会自动创建记录器的层次结构，因此这就是习惯使用模块的__name__来获取记录器的原因.

Note that __name__ is the module name, so it will be something like package.module1 or package.module2. Using dotted names automatically creates hierarchies of loggers, so that's why it's customary to just use the __name__ of the module to get the logger.

不需要module1和module2包含与日志记录有关的任何其他内容.他们不应该确定日志输出的位置或级别，因为这是启动应用程序的人应该控制的东西.因此，最好在主可执行文件中进行处理.

There is no need for module1 and module2 to contain anything else related to logging. They should not decide where the logging output goes or its level, because this is something that who launches the application should control. Hence it is best handled in the main executable.

现在在主可执行文件中定义处理程序:

Now in your main executable you define your handlers:

import logging, logging.handlers

fh = logging.handlers.RotatingFileHandler('log/pokerprogram.log', maxBytes=1000000, backupCount=10)
fh.setLevel(logging.DEBUG)
fh2 = logging.handlers.RotatingFileHandler('log/pokerprogram_info_only.log', maxBytes=1000000, backupCount=5)
fh2.setLevel(logging.INFO)
er = logging.handlers.RotatingFileHandler('log/errors.log', maxBytes=2000000, backupCount=2)
er.setLevel(logging.WARNING)
ch = logging.StreamHandler(sys.stdout)
ch.setLevel(1)
fh.setFormatter(logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s'))
fh2.setFormatter(logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s'))
er.setFormatter(logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s'))
ch.setFormatter(logging.Formatter('%(name)s - %(levelname)s - %(message)s'))

最后，您只需将处理程序添加到 root logger 中，并将root logger的级别设置为处理程序中的最低级别:

And finally you just add the handlers to the root logger and set the root logger's level to the lowest level among the handlers:

root = logging.getLogger()
root.setLevel(logging.DEBUG)
# alternatively:
# root.setLevel(min([fh.level, fh2.level, ch.level, er.level])

root.addHandler(fh)
root.addHandler(fh2)
root.addHandler(ch)
root.addHandler(er)

这是由于记录器的分层性质而起作用.调用module1.logger.debug时，如果记录器没有处理程序，它将把日志记录传递给它的父记录器，父记录器将继续这样做，直到根记录器最终使用它的处理程序来处理日志记录.

This works due to the hierarchical nature of loggers. When module1.logger.debug is called, if the logger has no handlers it will pass the log record to its parent logger, which will continue to do so up until the root logger, which finally uses its handlers to handle the log record.

还必须设置根记录器级别，因为它默认为WARNING，而其他记录器默认为NOTSET(这将导致前面提到的委派).

It is necessary to also set the root logger level because it defaults to WARNING, while other loggers default to NOTSET (which causes the delegation mentioned before).

或者，您将相同的处理程序显式添加到两个模块记录器中:

Alternatively you add the same handlers to both module loggers explicitly:

from <package> import module1, module2

module1.logger.setLevel(logging.DEBUG)
module2.logger.setLevel(logging.DEBUG)

module1.logger.addHandler(fh)
module2.logger.addHandler(fh)
module1.logger.addHandler(fh2)
module2.logger.addHandler(fh2)
module1.logger.addHandler(ch)
module2.logger.addHandler(ch)
module1.logger.addHandler(er)
module2.logger.addHandler(er)

将相同的处理程序对象添加到多个记录器中没有什么害处.这样可以确保处理程序不会尝试同时旋转文件.

There is no harm in adding the same handler object to multiple loggers. This makes sure that the handlers do not try to concurrently rotate the file.

这篇关于使用来自多个模块的python日志记录并写入文件和RotatingFileHandler的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！