task_migrate_and_publish_articles_by_issue: django.db.utils.DataError:Unicode low surrogate must follow a high surrogate /scielo_www/revenf/bases/pdf/ccs/v4n2/v4n2 Sum\\udce1

[patymori]

Descrição do problema

Executando migração e publicação de artigo de um periódico, alguns registros em ArticleProc ficaram com todos os status TODO e algumas das as tarefas em TaskTracker ficaram com status interrupted com o seguinte erro nos detalhes:

{..., 'traceback': '[\' File "/app/proc/tasks.py", line 1041, in task_migrate_and_publish_articles_by_issue\\n total_migrated_files = issue_proc.get_files_from_classic_website(\\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\\n\', \' File "/app/proc/models.py", line 1262, in get_files_from_classic_website\\n operation.finish(\\n\', \' File "/app/proc/models.py", line 177, in finish\\n self.save()\\n\', \' File "/usr/local/lib/python3.11/site-packages/django/db/models/base.py", line 902, in save\\n self.save_base(\\n\', \' File "/usr/local/lib/python3.11/site-packages/django/db/models/base.py", line 1005, in save_base\\n parent_inserted = self._save_parents(\\n ^^^^^^^^^^^^^^^^^^^\\n\', \' File "/usr/local/lib/python3.11/site-packages/django/db/models/base.py", line 1058, in _save_parents\\n updated = self._save_table(\\n ^^^^^^^^^^^^^^^^^\\n\', \' File "/usr/local/lib/python3.11/site-packages/django/db/models/base.py", line 1138, in _save_table\\n updated = self._do_update(\\n ^^^^^^^^^^^^^^^^\\n\', \' File "/usr/local/lib/python3.11/site-packages/django/db/models/base.py", line 1203, in _do_update\\n return filtered._update(values) > 0\\n ^^^^^^^^^^^^^^^^^^^^^^^^\\n\', \' File "/usr/local/lib/python3.11/site-packages/django/db/models/query.py", line 1286, in _update\\n return query.get_compiler(self.db).execute_sql(ROW_COUNT)\\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\\n\', \' File "/usr/local/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 2060, in execute_sql\\n row_count = super().execute_sql(result_type)\\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\\n\', \' File "/usr/local/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 1623, in execute_sql\\n cursor.execute(sql, params)\\n\', \' File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 79, in execute\\n return self._execute_with_wrappers(\\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^\\n\', \' File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 92, in _execute_with_wrappers\\n return executor(sql, params, many, context)\\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\\n\', \' File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 100, in _execute\\n with self.db.wrap_database_errors:\\n\', \' File "/usr/local/lib/python3.11/site-packages/django/db/utils.py", line 91, in __exit__\\n raise dj_exc_value.with_traceback(traceback) from exc_value\\n\', \' File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 105, in _execute\\n return self.cursor.execute(sql, params)\\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\\n\']', 'exceptions': [], 'exception_type': "<class 'django.db.utils.DataError'>", 'exception_message': 'invalid input syntax for type json\nLINE 1: ...classic_website\', "completed" = false, "detail" = \'{"failure...\n ^\nDETAIL: Unicode low surrogate must follow a high surrogate.\nCONTEXT: JSON data, line 1: ...ielo_www/revenf/bases/pdf/ccs/v4n2/v4n2 Sum\\udce1...\n'} 

As tarefas interrompidas foram exatamente as dos seguintes issues, com os PDFs nos diretórios:

Passos para reproduzir o problema

1. Execute a tarefa migrate_and_publish_articles para o periódico acrônimo CCS.
2. Acesse o Article Proc e observe que há artigos deste periódicos que estão com todos os status TODO
3. Acesse o Event Tracker e observe que há tarefas para este periódico com status interrupted
4. Observe o erro apresentado nos detalhes das tarefas interrompidas

Comportamento esperado

A tarefa proc.tasks.task_migrate_and_publish_articles_by_issue não deveria ser interrompida pelo erro e os artigos deveriam ser processados.

Screenshots ou vídeos

n/a

Anexos

.

Ambiente utilizado

Coleção RVE, versão do Upload 2.12.0rc1

[patymori]

@robertatakenaka testado na versão 2.12.0rc2 e o erro persiste: