running word_freqs throws ValueError for all submissions

[dotnwat]

I'm having some trouble running word_freqs. When I run the tool each submission is skipped due to ValueError, and the output files are empty. Example:

(env) MacBook-Air:reddit-analysis john$ word_freqs -l 1 -p day johndoe /r/news
Analyzing /r/news
.
Skipping submission https://www.reddit.com/r/news/comments/4zkn83/itt_tech_banned_from_enrolling_new_students_who/ due to ValueError.

[rhiever]

Darn. Maybe Reddit changed their API and that's messing up the script. I've filed this as a probable bug, but I probably won't find time to look into this anytime soon. Add a comment here if you're interested in digging into this bug.

cc @bboe

[dotnwat]

I can dig into it it a bit. What I'm seeing is that in parse_text an exception is being thrown when BeautifulSoup object is created.

The input that I printed out was

I just sold all my things and bought a one way ticket to the other side of the world. I consider 
myself opting out; engaged withdrawal. Experimenting with shedding the ways in which I've 
been conditioned by my culture and my comforts. I'll be camping down the coast by myself for 
the next few weeks before catching a flight with nothing but the pack on my back. I have very 
few solid plans, just trust. It's thrilling and extremely uncomfortable.

And looks like the exception that is thrown is <class 'bs4.FeatureNotFound'>

[bboe]

@rhiever I don't have any insight to add from Reddit's API side. Perhaps there is a new bs4 version that is backwards incompatible?

[mikeygeeman]

Try changing findAll to find_all

I think that changed somewhere

[bboe]

Also this one, is it still an issue?

[rhiever]

Never got around to digging into this one to see if it is indeed a bug.