Web Scraping using Scrapy in Python for Data Science (Learn to Extract Data from Websites)

Welcome to the Web Scraping using Scrapy in Python for Data Science course! In the contemporary setting, information has become abundant; however, much of it constitutes big data, which is unrefined and useless. That is when web scraping turns out to be very helpful. It is like a magic foil hat that collects and sorts data from websites that interest us in pressing a button. And Scrapy, this faithful companion for this journey, is all made possible!

In this course, “Web Scraping using Scrapy in Python for Data Science,” to begin with, we need to understand what web scraping is all about and why it is useful in today's world, especially when doing data science. Next, you will discover how to get Scrapy up and running and prepare your coding environment. Once we are established, we will understand selectors, which are tools helping to attract the specific data we want from web pages. Once we have that sorted, we can go for more ‘advanced items, such as using spiders (not the insect kind) to schedule our scraping tasks and the formats in which we can store the data we scrape.

You will be very competent at web scraping when you complete this Web Scraping using Scrapy in Python for Data Science course. This class is for you if you’ve never attempted web scraping or if you are a pro coder who would like to improve your data scraping skills. Further, do come to this class: let us delve into the world of web scraping together!

Web Scraping using Scrapy in Python for Data Science Table of Contents:

1. Introduction to web scraping and Scrapy library - 09:48
2. Setting up Scrapy library - 04:21
3. Doing a quick crawling of the website using Scrapy - 06:23
4. Scrapy CSS selectors introduction - 06:35
5. More CSS selector examples - 08:29
6. Scrapy XPath selectors - 09:46
7. Using Scrapy inside a Python program - Part 1 - 06:29
8. Using Scrapy inside a Python program - Part 2 - 07:54
9. Creating a new Scrapy project - Part 1 - 12:56
10. Creating a new Scrapy project - Part 2 - 05:01
11. Creating expression to get quote and author - 07:21
12. Adding expression to Scrapy project - 07:50
13. Scraping multiple pages automatically - 09:30
14. Saving scrapped data in SQLite DB - Part 1 - 07:14
15. Saving scrapped data in SQLite DB - Part 2 - 05:02
16. Getting author details links from all pages - Part 1 - 06:59
17. Getting author details links from all pages - Part 2 - 05:11
18. Getting author details and parse it - 09:29
19. Infinite scrolling - Create expression - Part 1 - 08:10
20. Infinite scrolling - Create expression - Part 2 - 05:17
21. Infinite scrolling - Add expression to Scrapy project - Part 1 - 06:24
22. Infinite scrolling - Add expression to Scrapy project - Part 2 - 07:38
23. Deploy spider in Scrapyd server - Part 1 - 07:52
24. Deploy spider in Scrapyd server - Part 2 - 06:10
25. Advanced scraping - JS generated HTML pages - Part 1 - 07:28
26. Advanced scraping - JS generated HTML pages - Part 2 - 10:37
27. Advanced scraping - Post form requests using Scrapy - Part 1 - 08:37
28. Advanced scraping - Post form requests using Scrapy - Part 2 - 08:33
29. Advanced scraping - Post form requests using Scrapy - Part 3 - 08:51
30. Full source code attached - 00:02

Who is this course for?

1. People currently working as data analysts or who wish to become data scientists in the near future.
2. Software developers and programmers who want to add another programming language to what they already know.
3. Digital marketers and SEO professionals will look to tap that data for insights and to optimize.
4. Any professional in the business world who wishes to improve how they collect their data.
5. Any individual who is interested in web scraping application and wish to have more information about it.

Click on the links below to Download Web Scraping using Scrapy in Python for Data Science!