25Mar 2020 by Andrew Treadway
Python for Web Scraping and APIs Online Course

Python for Web Scraping and APIs Online Course

Python, Web Scraping
I recently worked with a great team at 365 Data Science to create an extensive online Python for Web Scraping and APIs course! The course is now available on Udemy. Check it out here! The course covers many important topics, including the following: All about APIs: Get and Post requests HTML syntax - what's it for and why is it useful for web scraping BeautifulSoup - scraping links, text, and other elements from a webpage How to easily scrape tables from webpages How to automatically download files from the web Handling modern challenges: how to scrape JavaScript-rendered webpages Exploring the requests-html package, a powerful alternative to BeautifulSoup Click here to learn more about Python for Web Scraping and APIs Fundamentals! For other resources on learning Python or R, also check…
Read More
23Jul 2019 by Andrew Treadway
BeautifulSoup vs. Rvest

BeautifulSoup vs. Rvest

Python, R, Web Scraping
This post will compare Python's BeautifulSoup package to R's rvest package for web scraping. We'll also talk about additional functionality in rvest (that doesn't exist in BeautifulSoup) in comparison to a couple of other Python packages (including pandas and RoboBrowser). Getting started BeautifulSoup and rvest both involve creating an object that we can use to parse the HTML from a webpage. However, one immediate difference is that BeautifulSoup is just a web parser, so it doesn't connect to webpages. rvest, on the other hand, can connect to a webpage and scrape / parse its HTML in a single package. In BeautifulSoup, our initial setup looks like this: [code lang="python"] # load packages from bs4 import BeautifulSoup import requests # connect to webpage resp = requests.get(""https://www.azlyrics.com/b/beatles.html"") # get BeautifulSoup object soup…
Read More