Python Archives - Page 2 of 6 - Open Source Automation

12Oct 2020 by Andrew Treadway

How to create a progress bar in Python

Python

Creating a progress bar is really useful if you have a long-running task and you'd like to know how far along you are. In this post we'll talk about two packages for creating progress bars in Python: tqdm and progressbar2. Creating a progress bar with Python's tqdm package Let's get started by installing tqdm. We can do that using pip: [code] pip install tqdm [/code] Once we have tqdm installed, it's straightforward to use. Essentially, we just need to wrap whatever object we're looping over (i.e. any object that is iterable) inside tqdm. Then, when the code runs, Python will print out a progress bar so that you can know how far along you are. [code lang="python"] from tqdm import tqdm result = 0 for i in tqdm(range(10000000)): result +=…

05Aug 2020 by Andrew Treadway

How to scrape news articles with Python

Python, Web Scraping

In this post we're going to discuss how to scrape news articles with Python. This can be done using the handy newspaper package. Introduction to Python's newspaper package The newspaper package can be installed using pip: [code] pip install newspaper [/code] Once its installed, we can get started. newspaper can work by either scraping a single article from a given URL, or by finding the links on a webpage to other news articles. Let's start with handling a single article. First, we need to import the Article class. Next, we use this class to download the content from the URL to our news article. Then, we use the parse method to parse the HTML. Lastly, we can print out the text of the article using .text. Scraping a single article…

14Jul 2020 by Andrew Treadway

yahoo_fin 0.8.6

Python, Web Scraping

yahoo_fin, a package for scraping stock prices and financial data, was recently updated to version 0.8.6. The latest version has several new features, including the following: Users can now pull quarterly fundamentals data (in addition to yearly). This includes balance sheets, income statements, and cash flow statements. Earnings data can be extracted with the new get_earnings method. If you're looking to get multiple pieces of fundamentals data for the same stock - i.e. cash flows, income statements, and balance sheets, then make sure to check out a new method called get_financials. This method allows you to easily pull data points from each of these sources in a single request. New methods were added to retrieve historical dividend payouts and splits information. A bug in the ticker extraction methods causing a…

09Jul 2020 by Andrew Treadway

The for/else statement in Python

Python

One of the first things you usually learn when starting a new language is the if-else construct of that language. Python, like some other languages, offers an extended variation of this called the "for/else" statement. Let's check out a couple examples to see how it works! The for/else statement The main use case for the for/else statement is when you need to loop through a list with the potential to break out of the loop. If a break occurs in the loop, the code within the else clause will not be executed. However, if a break does not occur, then the code within the else clause will be executed. For example, in the code below we initialize test = False. Then we loop through each element in the list, nums.…

05May 2020 by Andrew Treadway

How to download fundamentals data with Python

Python, Web Scraping

How can we download fundamentals data with Python? In this post we will explore how to download fundamentals data with Python. We'll be extracting fundamentals data from Yahoo Finance using the yahoo_fin package. For more on yahoo_fin, including installation instructions, check out its full documentation here or my YouTube video tutorials here. Getting started Now, let's import the stock_info module from yahoo_fin. This will provide us with the functionality we need to scrape fundamentals data from Yahoo Finance. We'll also import the pandas package as we'll be using that later to work with data frames. [code lang="python"] import yahoo_fin.stock_info as si import pandas as pd [/code] Next, we'll dive into getting common company metrics, starting with P/E ratios. How to get P/E (Price-to-Earnings) Ratios There's a couple ways to get…

28Apr 2020 by Andrew Treadway

How to hide a password in a Python script

Python, System Administration

Background Python is widely used for web scraping and APIs, dealing with databases, sending emails, and other tasks that may involve credentials, like a username or password. Luckily, there are several ways to hide a password in a Python script, as well as to store multiple passwords for many users if needed. In this post, we'll talk about three such packages - keyring, passlib, and cryptography, with a focus on the first two. How to hide a password in a Python script with keyring keyring is an easy-to-use package that works by allowing you to store a password in your operating system's credential store. It's great in that it can work across multiple different operating systems. To get started, just use pip to install: [code] pip install keyring [/code] Next,…

15Apr 2020 by Andrew Treadway

All About Python Dictionaries

Python

In this article, we will explore Python dictionaries. For my previous tutorials on Python data structures, please see the below links. Lists List Comprehensions Sets What are Python dictionaries? A dictionary is one of Python's primary data structures. In essence, a dictionary (also shortened as dict) is a collection of key-value pairs, similar to an actual dictionary (like the one pictured above). When you look up a word in a dictionary, you find a corresponding definition. Similarly, in a Python dictionary, each key has a corresponding value. In dictionaries, one key can have only one value. However, the value can be anything from a single number or character, to lists, tuples, other dictionaries, or even functions! Creating a Dictionary Dictionaries can be created several ways. The most straightforward way is…

25Mar 2020 by Andrew Treadway

Python for Web Scraping and APIs Online Course

Python, Web Scraping

I recently worked with a great team at 365 Data Science to create an extensive online Python for Web Scraping and APIs course! The course is now available on Udemy. Check it out here! The course covers many important topics, including the following: All about APIs: Get and Post requests HTML syntax - what's it for and why is it useful for web scraping BeautifulSoup - scraping links, text, and other elements from a webpage How to easily scrape tables from webpages How to automatically download files from the web Handling modern challenges: how to scrape JavaScript-rendered webpages Exploring the requests-html package, a powerful alternative to BeautifulSoup Click here to learn more about Python for Web Scraping and APIs Fundamentals! For other resources on learning Python or R, also check…

24Mar 2020 by Andrew Treadway

What to study if you’re under quarantine

Python, R

If you're staying indoors more often recently because of the current COVID-19 outbreak and looking for new things to study, here's a few ideas! Free 365 Data Science Courses 365 Data Science is making all of their courses free until April 15. They have a variety of courses across R, Python, SQL, and more. Their platform also has courses that give a great mathematical foundation behind machine learning, which helps you a lot as you get deeper into data science. They also have courses on deep learning, which is a hot field right now. In addition to pure data science, 365 Data Science also covers material on Git / Github, which is essential for any data scientist nowadways. Another nice feature of 365 Data Science is that they also offer…

17Mar 2020 by Andrew Treadway

How to create decorators in R

Python, R

Introduction One of the coolest features of Python is its nice ability to create decorators. In short, decorators allow us to modify how a function behaves without changing the function's source code. This can often make code cleaner and easier to modify. For instance, decorators are also really useful if you have a collection of functions where each function has similar repeating code. Fortunately, decorators can now also be created in R! The first example below is in Python - we'll get to R in a moment. If you already know about decorators in Python, feel free to skip below to the R section. Below we have a function that prints today's date. In our example, we create two functions. The first - print_start_end takes another function as input. It…

Tag: Python