Skip to content

Conversation

MikolajBalcerek
Copy link
Contributor

@MikolajBalcerek MikolajBalcerek commented Aug 27, 2025

json_logger doesn't work with Python 3.8=+ due to (last working is Python 3.7)

  • renamed field fmt to format in logging
  • strict % formatting being needed
raise ValueError("Invalid format '%s' for '%s' style" % (self._fmt, self.default_format[0]))

ValueError: Invalid format '(@timestamp) (level) (name) (@message) (message)' for '%' style

This fixes the usage to newer logging behaviour


This fix won't make it enough to work with newer Python (3.12=+) releases, because this entire package uses pkg_resources, which is deprecated and/or removed in new Python releases (removed in Python 3.12) - this would also would have to be fixed

    from wikia.common.logger import json_logger
  File "/tmp/recommendationimport/src/wikia/__init__.py", line 4, in <module>
    __import__('pkg_resources').declare_namespace(__name__)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^
ModuleNotFoundError: No module named 'pkg_resources'

On older Python releases you will get a warning instead


/usr/local/lib/python3.12/site-packages/wikia/common/logger/__init__.py:9: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.

This is thus fixable with limiting support to old python release, rewriting python-commons to use importlibs (preferred), or installing setuptools as dependency.

I don't have the time to fix it further

@mikulskibartosz
Copy link

Use namespace packages (https://packaging.python.org/en/latest/guides/packaging-namespace-packages/) instead of __import__('pkg_resources').declare_namespace(__name__)

import time

import pythonjsonlogger.jsonlogger
import pythonjsonlogger.json
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Package moved structure

super(WikiaJsonLogsFormatter, self).add_fields(log_record, record, message_dict)
if not log_record.get('@timestamp'):
log_record['@timestamp'] = datetime.utcfromtimestamp(record.created).strftime('%Y-%m-%dT%H:%M:%S.%fZ')
log_record['@timestamp'] = datetime.fromtimestamp(record.created).strftime('%Y-%m-%dT%H:%M:%S.%fZ')
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

datetime.utcfromtimestamp is deprecated

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants