AttributeError: module 'vim' has no attribute 'find_module' (python 3.7)

[sebastianst]

Probably after upgrading my Arch Linux system to python 3.7, thesaurus_query stopped working. I get the following errors:

Error detected while processing /home/stammler/.vim/bundle/thesaurus_query.vim/autoload/thesaurus_query.vim:
line  214:
Traceback (most recent call last):
  File "<string>", line 9, in <module>
  File "/home/stammler/.vim/bundle/thesaurus_query.vim/autoload/thesaurus_query/thesaurus_query.py", line 17, in <module>
    from . import backends as tq_backends
  File "/home/stammler/.vim/bundle/thesaurus_query.vim/autoload/thesaurus_query/backends/__init__.py", line 10, in <module>
    from . import *
  File "/home/stammler/.vim/bundle/thesaurus_query.vim/autoload/thesaurus_query/backends/thesaurus_com_lookup.py", line 27, i
n <module>
    import pkg_resources
  File "/usr/lib/python3.7/site-packages/pkg_resources/__init__.py", line 3112, in <module>
    @_call_aside
  File "/usr/lib/python3.7/site-packages/pkg_resources/__init__.py", line 3096, in _call_aside
    f(*args, **kwargs)
  File "/usr/lib/python3.7/site-packages/pkg_resources/__init__.py", line 3140, in _initialize_master_working_set
    for dist in working_set
  File "/usr/lib/python3.7/site-packages/pkg_resources/__init__.py", line 3140, in <genexpr>
    for dist in working_set
  File "/usr/lib/python3.7/site-packages/pkg_resources/__init__.py", line 2647, in activate
    declare_namespace(pkg)
  File "/usr/lib/python3.7/site-packages/pkg_resources/__init__.py", line 2184, in declare_namespace
    _handle_ns(packageName, path_item)
  File "/usr/lib/python3.7/site-packages/pkg_resources/__init__.py", line 2101, in _handle_ns
    loader = importer.find_module(packageName)
AttributeError: module 'vim' has no attribute 'find_module'
Error detected while processing function thesaurus_query#Thesaurus_Query_Init:
line    1:
Traceback (most recent call last):
  File "<string>", line 1, in <module>
NameError: name 'tq_interface' is not defined
Error detected while processing function thesaurus_query#Thesaurus_Query_Lookup:
line   40:
Traceback (most recent call last):
  File "<string>", line 2, in <module>
NameError: name 'tq_framework' is not defined
line   44:
Traceback (most recent call last):
  File "<string>", line 1, in <module>
NameError: name 'tq_synonym_result' is not defined 

Up-to-date Arch Linux, python 3.7.0, vim 8.1
It seems that other projects have similar problems, e.g., python-mode/python-mode#937

[Ron89]

Hi, since Python 3.7 and Python 3.6 use separate site-packages folder, the upgrade to Python3.7 does require a reinstallation of package thesaurus. But other than that, if thesaurus package is installed, I seems to be able to use the affected backend "thesaurus_com" as normal.

For your case, I find it quite curious why import pkg_resources could eventually lead to error AttributeError: module 'vim' has no attribute 'find_module'. I mean, vim is not even explicitly mentioned here.

If you open python3 in the terminal, in the interactive prompt, invoke import pkg_resources, does it return something weird?

[sebastianst]

Thanks you, import pkg_resources invoked in a fresh python 3.7 terminal works without any error.

[Ron89]

How about in Vim 8? Invoke :py3 import pkg_resources. The reason I am asking for this is that the error message specifically said vim has no attribute find_module, the existence of which is assumed by pkg_resources.

[sebastianst]

That causes an error!

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python3.7/site-packages/pkg_resources/__init__.py", line 3112, in
 <module>
    @_call_aside
  File "/usr/lib/python3.7/site-packages/pkg_resources/__init__.py", line 3096, in
 _call_aside
    f(*args, **kwargs)
  File "/usr/lib/python3.7/site-packages/pkg_resources/__init__.py", line 3140, in
 _initialize_master_working_set
    for dist in working_set
  File "/usr/lib/python3.7/site-packages/pkg_resources/__init__.py", line 3140, in
 <genexpr>
    for dist in working_set
  File "/usr/lib/python3.7/site-packages/pkg_resources/__init__.py", line 2647, in
 activate
    declare_namespace(pkg)
  File "/usr/lib/python3.7/site-packages/pkg_resources/__init__.py", line 2184, in
 declare_namespace
    _handle_ns(packageName, path_item)
  File "/usr/lib/python3.7/site-packages/pkg_resources/__init__.py", line 2101, in
 _handle_ns
    loader = importer.find_module(packageName)
AttributeError: module 'vim' has no attribute 'find_module'

my full vim --version

VIM - Vi IMproved 8.1 (2018 May 18, compiled Sep 12 2018 23:25:13)
Included patches: 1-374
Compiled by Arch Linux
Huge version with GTK3 GUI.  Features included (+) or not (-):
+acl               +extra_search      +mouse_netterm     +tag_old_static
+arabic            +farsi             +mouse_sgr         -tag_any_white
+autocmd           +file_in_path      -mouse_sysmouse    +tcl/dyn
+autochdir         +find_in_path      +mouse_urxvt       +termguicolors
-autoservername    +float             +mouse_xterm       +terminal
+balloon_eval      +folding           +multi_byte        +terminfo
+balloon_eval_term -footer            +multi_lang        +termresponse
+browse            +fork()            -mzscheme          +textobjects
++builtin_terms    +gettext           +netbeans_intg     +timers
+byte_offset       -hangul_input      +num64             +title
+channel           +iconv             +packages          +toolbar
+cindent           +insert_expand     +path_extra        +user_commands
+clientserver      +job               +perl/dyn          +vartabs
+clipboard         +jumplist          +persistent_undo   +vertsplit
+cmdline_compl     +keymap            +postscript        +virtualedit
+cmdline_hist      +lambda            +printer           +visual
+cmdline_info      +langmap           +profile           +visualextra
+comments          +libcall           +python/dyn        +viminfo
+conceal           +linebreak         +python3/dyn       +vreplace
+cryptv            +lispindent        +quickfix          +wildignore
+cscope            +listcmds          +reltime           +wildmenu
+cursorbind        +localmap          +rightleft         +windows
+cursorshape       +lua/dyn           +ruby/dyn          +writebackup
+dialog_con_gui    +menu              +scrollbind        +X11
+diff              +mksession         +signs             -xfontset
+digraphs          +modify_fname      +smartindent       +xim
+dnd               +mouse             +startuptime       -xpm
-ebcdic            +mouseshape        +statusline        +xsmp_interact
+emacs_tags        +mouse_dec         -sun_workshop      +xterm_clipboard
+eval              +mouse_gpm         +syntax            -xterm_save
+ex_extra          -mouse_jsbterm     +tag_binary        
   system vimrc file: "/etc/vimrc"
     user vimrc file: "$HOME/.vimrc"
 2nd user vimrc file: "~/.vim/vimrc"
      user exrc file: "$HOME/.exrc"
  system gvimrc file: "/etc/gvimrc"
    user gvimrc file: "$HOME/.gvimrc"
2nd user gvimrc file: "~/.vim/gvimrc"
       defaults file: "$VIMRUNTIME/defaults.vim"
    system menu file: "$VIMRUNTIME/menu.vim"
  fall-back for $VIM: "/usr/share/vim"
Compilation: gcc -c -I. -Iproto -DHAVE_CONFIG_H -DFEAT_GUI_GTK  -I/usr/include/gtk-3.0 -I/usr/include/pango-1.0 -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include -I/usr/lib/libffi-3.2.1/include -I/usr/include/fribidi -I/usr/include/cairo -I/usr/include/pixman-1 -I/usr/include/freetype2 -I/usr/include/libpng16 -I/usr/include/harfbuzz -I/usr/include/uuid -I/usr/include/gdk-pixbuf-2.0 -I/usr/include/gio-unix-2.0 -I/usr/include/libmount -I/usr/include/blkid -I/usr/include/libdrm -I/usr/include/atk-1.0 -I/usr/include/at-spi2-atk/2.0 -I/usr/include/at-spi-2.0 -I/usr/include/dbus-1.0 -I/usr/lib/dbus-1.0/include -pthread  -D_FORTIFY_SOURCE=2  -march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong -fno-plt -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1       
Linking: gcc   -L. -Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now -fstack-protector -rdynamic -Wl,-export-dynamic -Wl,-E -Wl,-rpath,/usr/lib/perl5/5.28/core_perl/CORE  -Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now -L/usr/local/lib -Wl,--as-needed -o vim   -lgtk-3 -lgdk-3 -lpangocairo-1.0 -lpango-1.0 -latk-1.0 -lcairo-gobject -lcairo -lgdk_pixbuf-2.0 -lgio-2.0 -lgobject-2.0 -lglib-2.0  -lSM -lICE -lXt -lX11 -lXdmcp -lSM -lICE  -lm -ltinfo -lelf -lnsl    -lacl -lattr -lgpm -ldl   -Wl,-E -Wl,-rpath,/usr/lib/perl5/5.28/core_perl/CORE -Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now -fstack-protector-strong -L/usr/local/lib  -L/usr/lib/perl5/5.28/core_perl/CORE -lperl -lpthread -ldl -lm -lcrypt -lutil -lc   -L/usr/lib -ltclstub8.6 -ldl -lz -lpthread -lm     

[Ron89]

Hi @sebastianst , sorry for the long delay. From what we have conversed, it seems like your Vim build's python integration has some issue. I cannot determine if the source was caused by Vim's build or Python's minor version upgrade, though.

[sebastianst]

Thanks! I reported this on the Arch bug tracker

[sebastianst]

Got a response already. I quote:
This has nothing to do with a faulty vim build, this is what you get when you build vim with python 3.7:

https://github.com/vim/vim/blob/HEAD/src/if_py_both.h#L91
vim/vim@79a494d

only find_spec is exposed as that's what should be used in python >= 3.7 so there is no find_module exposed in vim's module and hence the error message. Calling it from terminal does something totally different.

pkg_resources from https://github.com/pypa/setuptools should use find_spec instead of imp find_module when running on >= 3.7 that would be the place to get a proper fix.

[Ron89]

I see, so the issue is with PyPa kept on using the deprecated find_module in Python for 3.7 and VIm decided to drop it...
Strangely, on my Manjaro with Python3.7, Vim seems to work fine with my Python with pkg_resources using find_module(). But it could be because the issue starts with some very specific fix version 0x030700f0.

[t89]

I get this error since I use pyenv. I reinstalled (mac)vim using the --with-python3 flag. All other plugins including the notorious YCM are working again. I haven't found a fix for this one in almost half a year now. Any recommendations?

[Ron89]

Removing package dependency checking could work. But this would potentially make new users not knowing what to do when their thesaurus.com backend refuses to work.

Technically, this issue is caused by the conflicting between Vim8 and Pypa, so I am not certain what I can do from my side. Maybe using :!python to force install on each startup could work. But that has its own issues following.

[Ron89]

I suggest you give a try to neovim. So far I have no issue using my plugin (and many others that I rely on), with it.

[t89]

I suggest you give a try to neovim. So far I have no issue using my plugin (and many others that I rely on), with it.

I guess it's time to spend an afternoon migrating, then. Thank you for your time, and also this wonderful plugin. :)