Scraping Medium with Python & Beautiful Soup
Extract data about 6K+ articles from 7 different publications
Whether you want to do an analysis on blogging and want to know what factors may determine some articles to be more successful than others, or whether you just want to practice your web scraping skills, this project of scraping data about Medium articles is one that’s not so trivial and I think it’s worth sharing it here.
This article is pure informative; I do not encourage that all of you, readers, start running the code and exhaust Medium’s servers.
There is a link to the scraped data at the end of the article for those who are interested in downloading/using it.
For those who are new to web scraping, it may be helpful to first read my introductory article about web scraping:
… and then continue with this one.
That being said, now let’s get started.
In order to be useful in drawing some statistical conclusions or to be used in machine learning, we…
