How to get options data with Python

How to get options data with Python

Python, Web Scraping
In a previous post, we talked about how to get real-time stock prices with Python. This post will go through how to download financial options data with Python. We will be using the yahoo_fin package. The yahoo_fin package comes with a module called options. This module allows you to scrape option chains and get option expiration dates. To get started we'll just import this module from yahoo_fin. [code lang="python"] from yahoo_fin import options [/code] How to get options expiration dates Any option contract has an expiration date. To get all of the option expiration dates for a particular stock, we can use the get_expiration_dates method in the options package. This method is equivalent to scraping all of the date selection boxes on the options page for an individual stock (e.g.…
Read More

Don’t forget the “utils” package in R

R
With thousands of powerful packages, it's easy to glaze over the libraries that come preinstalled with R. Thus, this post will talk about some of the cool functions in the utils package, which comes with a standard installation of R. While utils comes with several familiar functions, like read.csv, write.csv, and help, it also contains over 200 other functions. readClipboard and writeClipboard One of my favorite duo of functions from utils is readCLipboard and writeClipboard. If you're doing some manipulation to get a quick answer between R and Excel, these functions can come in handy. readClipboard reads in whatever is currently on the Clipboard. For example, let's copy a column of cells from Excel. We can now run readClipboard() in R. The result of running this command is a vector…
Read More
Speed Test: Sapply vs. Vectorization

Speed Test: Sapply vs. Vectorization

R
The apply functions in R are awesome (see this post for some lesser known apply functions). However, if you can use pure vectorization, then you'll probably end up making your code run a lot faster than just depending upon functions like sapply and lapply. This is because apply functions like these still rely on looping through elements in a vector or list behind the scenes - one at a time. Vectorization, on the other hand, allows parallel operations under the hood - allowing much faster computation. This posts runs through a couple such examples involving string substitution and fuzzy matching. String substitution For example, let's create a vector that looks like this: test1, test2, test3, test4, ..., test1000000 with one million elements. With sapply, the code to create this would…
Read More
Why defining constants is important – a Python example

Why defining constants is important – a Python example

Python
This post will walk through an example of why defining a known constant can save lots of computational time. How to find the key with the maximum value in a Python dictionary There's a few ways to go about getting the key associated with the max value in a Python dictionary. The two ways we'll show each involve using a list comprehension. First, let's set the scene by creating a dictionary with 100,000 key-value pairs. We'll just make the keys the integers between 0 and 99,999 and we'll use the random package to randomly assign values for each of these keys based off the uniform distribution between 0 and 100,000. [code lang="python"] import random import time vals = [random.uniform(0, 100000) for x in range(100000)] mapping = dict(zip(range(100000), vals)) [/code] Now,…
Read More
Creating a word cloud on R-bloggers posts

Creating a word cloud on R-bloggers posts

R, Web Scraping
This post will go through how to create a word cloud of article titles scraped from the awesome R-bloggers. Our goal will be to use R's rvest package to search through 50 successive pages on the site for article titles. The stringr and tm packages will be used for string cleaning and for creating a term document frequency matrix (with tm). We will then create a word cloud based off the words comprising these titles. First, we'll load the packages we need. [code lang="R"] # load packages library(rvest) library(stringr) library(tm) library(wordcloud) [/code] Let's write a function that will take a webpage as input and return all the scraped article titles. [code lang="R"] scrape_post_titles <- function(site) { # scrape HTML from input site source_html <- read_html(site) # grab the title attributes…
Read More
Scraping data from a JavaScript webpage with Python

Scraping data from a JavaScript webpage with Python

Python, Web Scraping
This post will walk through how to use the requests_html package to scrape options data from a JavaScript-rendered webpage. requests_html serves as an alternative to Selenium and PhantomJS, and provides a clear syntax similar to the awesome requests package. The code we'll walk through is packaged into functions in the options module in the yahoo_fin package, but this article will show how to write the code from scratch using requests_html so that you can use the same idea to scrape other JavaScript-rendered webpages. Note: requests_html requires Python 3.6+. If you don't have requests_html installed, you can download it using pip: [cc] pip install requests_html [/cc] Motivation Let's say we want to scrape options data for a particular stock. As an example, let's look at Netflix (since it's well known). If…
Read More
2 packages for extracting dates from a string of text in Python

2 packages for extracting dates from a string of text in Python

Pandas, Python
This post will cover two different ways to extract a date from a string of text in Python. The main purpose here is that the strings we will parse contain additional text - not just the date. Scraping a date out of text can be useful in many different situations. Option 1) dateutil The first option we'll show is using the dateutil package. Here's an example: [code lang="python"] from dateutil.parser import parse parse("Today is 12-01-18", fuzzy_with_tokens=True) [/code] Above, we use a method in dateutil called parse. The first parameter to this method is the string of the text we want to use to search for a date. The second parameter, fuzzy_with_tokens, is set equal to True - this causes the method to return where the date is found in the…
Read More

How to change a file’s last modified date with R

File Manipulation, R
This relatively quick post goes through how to change a file's last modified date with base R. How to change a file's modified time with R Let's say we have a file, test.txt. What if we want to change the last modified date of the file (let's suppose the file's not that important)? Let's say, for instance, we want to make a file have a last modified date back in the 1980's. We can do that with one line of code. First, let's use file.info to check the current modified date of some file called test.txt. [code lang="R"] file.info("test.txt") [/code] We can see above by looking at mtime that this file was last modified December 4th, 2018. Now, we can use a function called Sys.setFileTime to change the modified date…
Read More
10 R functions for Linux commands and vice-versa

10 R functions for Linux commands and vice-versa

File Manipulation, R, System Administration
This post will go through 10 different Linux commands and their R alternatives. If you're interested in learning more R functions for working with files like some of those below, also check out this post. How to list all the files in a directory Linux R What does it do? ls list.files() Lists all the files in a directory ls -R list.files(recursive = TRUE) Recursively lists all the files in a directory and all sub-directories ls | grep "something" list.files(pattern = "something") Lists all the files in a directory containing the regex "something" R [code lang="R"] list.files("/path/to/directory") list.files("/path/to/do/directory", recursive = TRUE) # search for files containing "something" in their name list.files("/path/to/do/directory", pattern = "something") # search for all CSV files list.files("/path/to/do/directory", pattern = ".csv") [/code] Linux [code lang="bash"] ls /path/to/directory…
Read More

Intro to Python Course

Python
For anyone in the NYC area, I am offering an in-person introductory Python course on January 7th, 2019. The description of the workshop is below. Please see here to register on Eventbrite. Want to learn Python? Consider attending this workshop! This hands-on class will be a great introduction for how to code in Python, the important features of the language, and will help you build a strong foundation for learning more in the future! Overview This course provides a workshop for introducing you to Python. We'll walk through how to write and run Python programs, when to use particular data structures, how to handle different data types, and more. The class will be a great start in learning one of the most versatile, powerful programming languages being used today! All…
Read More