feat(academy): add advanced crawling section with sitemaps and search

[metalwarrior665]

No description provided.

[metalwarrior665]

Will process the lint issues soon

[metalwarrior665]

@TC-MO If we change URL of an article, do I need to contact web team to set a hard redirect?

[TC-MO]

I think we do redirects in nginx.conf file not sure if there is any other way

[metalwarrior665]

TODO redirect
..._web_scraping/scraping_paginated_sites.md → ...scraping/crawling/crawling-with-search.md

[honzajavorek]

Will review, but I think I will wait for @TC-MO's comments to be addressed first.

[metalwarrior665]

@TC-MO I addressed the remarks and rebased to master, pls take a look. I will see the tests

[honzajavorek]

🔴 I think the front matter contains double semicolons, which could be a syntax error or a silent error causing unexpected problems.

🟠 I made a few suggestions on improvements. My aim was to add to readability and flow of the sentences.

• I steered the text towards being language-agnostic, but I didn't check the whole context of the course to see if it's necessary or not.
• I suggested to remove certain emotional phrases, such as "fortunately".
• Saying "next" together with "lessons" didn't feel right, so I suggest using "following lessons" and to keep "next" for just one lesson.
• You mention framework several times, and while it's perfectly valid, my idea would be to keep that word only for Crawlee (or Scrapy and others), and find other words to call the rest, such as algorithm, tool, or approach. I think separating those two clearly will remove some amount of possible confusion.
• When linking to content, I prefer to put actual text and names to the link text instead of "see this article", and I think it's also a web document accessibility rule.

⚪️ I could imagine a bit more polishing of the flow of the sentences, and sometimes they could be more brief, but I don't think that's something we should do in PR reviews, it's highly subjective and a lot of back and forth. For minimalistic Czenglish check I can highly recommend "Is this correct English?" prompt to ChatGPT, followed by a Markdown block of whatever you want to check. It keeps your writing as is, but provides basic grammar and flow fixes. The text then sounds like you, but is more smooth. I use it a lot.

[metalwarrior665]

@TC-MO @honzajavorek Thanks for the suggestions, I hope I applied everything

[TC-MO]

LGTM