UnicodeDecodeError: 'charmap' codec can't decode byte 0x88 in position 1097: character maps to <undefined>

[daliboris]

Running python ./run.py create_config --path semantsum/config/boris_config.yaml throws an UnicodeDecodeError error.

OS: Windows 10 Czech
Python: 3.11.0
Terminál Windows: 1.21.10351.0

python ./run.py create_config --path semantsum/config/boris_config.yaml
[?] Which configuration do you want to create?:
 > summarization
   API

[?] Select summarizer:
   HFQueryBasedMultiDocSummarizer
 > HFSingleDocSummarizer
   HFWithPromptBuilder
   OpenAIQueryBasedMultiDocSummarizer
   OpenAISingleDocSummarizer
   OpenAIWithPromptBuilder

[?] Enter configuration name: boris
Traceback (most recent call last):
  File "V:\Projekty\Github\SemANT\semant-summarization\run.py", line 6, in <module>
    main()
  File "V:\Projekty\Github\SemANT\semant-summarization\semantsum\__main__.py", line 198, in main
    args.func(args)
  File "V:\Projekty\Github\SemANT\semant-summarization\semantsum\__main__.py", line 159, in create_config
    create_summarizer_config(args)
  File "V:\Projekty\Github\SemANT\semant-summarization\semantsum\__main__.py", line 81, in create_summarizer_config
    while name is None or name in list_summarizers():
                                  ^^^^^^^^^^^^^^^^^^
  File "V:\Projekty\Github\SemANT\semant-summarization\semantsum\summarizator_factory.py", line 29, in list_summarizers
    config = YAML().load(f)
             ^^^^^^^^^^^^^^
  File "C:\Users\Boris\AppData\Roaming\Python\Python311\site-packages\ruamel\yaml\main.py", line 424, in load
    constructor, parser = self.get_constructor_parser(stream)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Boris\AppData\Roaming\Python\Python311\site-packages\ruamel\yaml\main.py", line 473, in get_constructor_parser
    self.reader.stream = stream
    ^^^^^^^^^^^^^^^^^^
  File "C:\Users\Boris\AppData\Roaming\Python\Python311\site-packages\ruamel\yaml\reader.py", line 118, in stream
    self.determine_encoding()
  File "C:\Users\Boris\AppData\Roaming\Python\Python311\site-packages\ruamel\yaml\reader.py", line 172, in determine_encoding
    self.update_raw()
  File "C:\Users\Boris\AppData\Roaming\Python\Python311\site-packages\ruamel\yaml\reader.py", line 261, in update_raw
    data = self.stream.read(size)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python311\Lib\encodings\cp1250.py", line 23, in decode
    return codecs.charmap_decode(input,self.errors,decoding_table)[0]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'charmap' codec can't decode byte 0x88 in position 1097: character maps to <undefined>
PS V:\Projekty\Github\SemANT\semant-summarization>

[mdocekal]

The problem is that we have developed this project for Linux, where the default encoding is UTF8. However, your Windows is trying to use cp1250. I think that the easiest solution would be to use -Xutf8 python parameter, which will activate the UTF8.

python -Xutf8 .\run.py create_config --path semantsum\config\boris_config.yaml

[daliboris]

Thank you, that worked.

It would be worth it to improve the software (to open files directly in UTF-8 encoding) or the help.