Designed to scrap internet content for the main project Medbot.
npm installto install packages.- Define your own config file
yourconfig.json. It should contain which file to write to and which urls to crawl. - Write your own parsing methods using cheerios with
objecttypeas method name. Refer toscrape.jsfor movies example. - Call from commandline
node main.js yourconfig.json objecttype.