Using the substring "flip" in the text for freetype.putText duplicates the last character

[ichu-lsa]

System information (version)

• OpenCV => 4.2
• Operating System / Platform => Ubuntu 18.04
• Compiler => gcc

Detailed description

Having the substring "flip" anywhere in the text parameter for cv2.freetype.putText() causes the last character to be duplicated. Including it twice will cause the last two characters to be duplicated (i.e "xflipflipx" will cause "xflipflipxpx" to appear on screen).
This is a pretty low-priority bug since it doesn't seem to cause any additional problems and is just a graphical glitch, but it's definitely one of the weirdest ones I've come across with opencv.

Steps to reproduce

import cv2
import numpy as np

# load font
font = cv2.freetype.createFreeType2();
font.loadFontData("Font/JosefinSansBold-OVA7o.ttf", 0); # replace with whatever font

# make empty image
img = np.zeros((600,600,3), np.uint8);

# draw text
img = font.putText(img, "x flip y flip z", (50,50), 25, (255, 255, 255), -1, cv2.LINE_AA, True);

# show image
cv2.imshow("Image", img);
cv2.waitKey(0);

Edits

This seems to happen with the substring 'find' as well. There are probably more keywords that trigger this behavior.

[Kumataro]

Thank you for your report.
I'm sorry this is bug of contrib/freetype2 module.

Summary

Sequence to calls harfbuzz library has trouble.

Sample Code

https://github.com/Kumataro/opencv_contrib/blob/fix_contrib_issue_%232627/modules/freetype/src/freetype.cpp

Incorrect Call Sequence (Currently implementation )

1. create buffer.
2. guess segment properties.
3. add utf8 string into buffer.
4. Get information and text length.
5. shaping. (convert from utf8 to codepoint)
6. loop with text length

What is problem?

There are 2 problems.

• before adding utf8 string, guess segment properties has no meaningful.
• in step 4 it returns "length of utf8 strings". However, at step 5 some utf8 substrings are composite to one codepoint. So "length of codepoint array" are shorten than it.

It is need to fix it as following.

Correct Call Sequence

1. create buffer.
2. add utf8 string into buffer.
3. guess segment properties.
4. shaping. (convert from utf8 to codepoint)
5. Get information and text length.
6. loop with text length

Testing

• Using "NotoSerifCJKjp-Medium.otf" which is one of Google Noto Font.
• Drawing"affinity" text, which includes "ffi" .
  • using other font file, the same problem may occur with "fli", "ff", "tt" ,...

Result of Incorrect Call Sequence (Currently implementation )

The number of Iteration is 8.

[0]66 // a
[1]59874 // ffi
[2]79 // n
[3]74 // i
[4]85 // t
[5]90 // y
[6]85 // t
[7]90 // y

Result of Correct Call Sequence

The number of Iteration is 6.

[0]66 // a
[1]59874 // ffi
[2]79 // n
[3]74 // i
[4]85 // t
[5]90 // y

Reference

• https://harfbuzz.github.io/ch03s03.html.

[ichu-lsa]

Thanks for addressing this. It's reassuring to see even relatively unimportant bugs like this get fixed.

[Kumataro]

Thank you very much!
And I'm sorry that pull request for this issue will be delayed 2-3 weeks.

This fix will change some output images, and regression test is needed.

Because this module has no test code, to confirm some effects and side-effects of changing code is not easy.

( Until now, I have been confirmed it by seeing many output images with changing some font file, some text, some height of font , some thickness and etc. It is heuristic. )

I’m trying to add opencv_contrib/module/freetype/test and opencv_extra/testdata/freetype for regression test.

[Kumataro]

Thank you for waiting and this is only progress report..

I’m trying to add opencv_contrib/module/freetype/test and opencv_extra/testdata/freetype for regression test.

Last and this weekend, I wrote test program, it seems to work well in x86-64 linux.
After this fix, some output are changed because some glyphs are combined.
I will challenge this test in other environment (ARM Linux) next week.
If there is no trouble, I will create pull request.

Repository

• https://github.com/Kumataro/opencv_contrib/tree/fix_contrib_issue_2627_with_test/modules/freetype/test
• https://github.com/Kumataro/opencv_extra/tree/fix_contrib_issue_2627_with_test/testdata/cv/freetype

[Kumataro]

The results in the ARM environment were NG.

If there is no trouble, I will create pull request.

I ran the test program on the Tinker Board (ASUS), which has ARM Cortex Quad A17.
The library and test program could be compiled without any problem.
However, the test pointed out a number of errors, and some are regression test for this fix (testLigature).

I'm sorry I need time to resolve.

[----------] Global test environment tear-down
[==========] 21 tests from 7 test cases ran. (61676 ms total)
[  PASSED  ] 16 tests.
[  FAILED  ] 5 tests, listed below:
[  FAILED  ] CV_freetype_putText_Mono.testForFontHeight
[  FAILED  ] CV_freetype_putText_Outline.testMultilingual
[  FAILED  ] CV_freetype_putText_Mono_Blend.testRegression
[  FAILED  ] CV_freetype_putText_Mono_Blend.testLigature
[  FAILED  ] CV_freetype_putText_Mono_Blend.testMultilingual

 5 FAILED TESTS

[Kumataro]

I'm sorry, it was fixed by #3079 .