04Apr 2023 by Andrew Treadway

Software Engineering for Data Scientists (New book!)

Machine Learning, Python
Very excited to announce the early-access preview (MEAP) of my upcoming book, Software Engineering for Data Scientists is available now! Check it out at this link. Use promo code au35tre to save 30% on this book and any products sold from Manning. Why Software Engineering for Data Scientists? Data science and software engineering have been merging more and more, especially over the last decade. Software Engineering for Data Scientists is my upcoming book that will help you learn more about software engineering and how it can make your life easier as a data scientist! This book covers the following key topics: Source control How to implement exception handling and write robust code Object-oriented programming for data scientists How to monitor the progress of training machine learning models Scaling your Python…
Read More
28Apr 2021 by Andrew Treadway
How to plot XGBoost trees in R

How to plot XGBoost trees in R

Machine Learning, R
In this post, we're going to cover how to plot XGBoost trees in R. XGBoost is a very popular machine learning algorithm, which is frequently used in Kaggle competitions and has many practical use cases. Let's start by loading the packages we'll need. Note that plotting XGBoost trees requires the DiagrammeR package to be installed, so even if you have xgboost installed already, you'll need to make sure you have DiagrammeR also. [code lang="R"] # load libraries library(xgboost) library(caret) library(dplyr) library(DiagrammeR) [/code] Next, let's read in our dataset. In this post, we'll be using this customer churn dataset. The label we'll be trying to predict is called "Exited" and is a binary variable with 1 meaning the customer churned (canceled account) vs. 0 meaning the customer did not churn (did…
Read More
18Feb 2020 by Andrew Treadway
How is information gain calculated?

How is information gain calculated?

Machine Learning, R
This post will explore the mathematics behind information gain. We'll start with the base intuition behind information gain, but then explain why it has the calculation that it does. What is information gain? Information gain is a measure frequently used in decision trees to determine which variable to split the input dataset on at each step in the tree. Before we formally define this measure we need to first understand the concept of entropy. Entropy measures the amount of information or uncertainty in a variable's possible values. How to calculate entropy Entropy of a random variable X is given by the following formula: -Σi[p(Xi) * log2(p(Xi))] Here, each Xi represents each possible (ith) value of X. p(xi) is the probability of a particular (the ith) possible value of X. Why…
Read More
29Jan 2020 by Andrew Treadway
Evaluate your R model with MLmetrics

Evaluate your R model with MLmetrics

Machine Learning, R
This post will explore using R's MLmetrics to evaluate machine learning models. MLmetrics provides several functions to calculate common metrics for ML models, including AUC, precision, recall, accuracy, etc. Building an example model Firstly, we need to build a model to use as an example. For this post, we'll be using a dataset on pulsar stars from Kaggle. Let's save the file as "pulsar_stars.csv". Each record in the file represents a pulsar star candidate. The goal will be to predict if a record is a pulsar star based upon the attributes available. To get started, let's load the packages we'll need and read in our dataset. [code lang="R"] library(MLmetrics) library(dplyr) stars = read.csv("pulsar_stars.csv") [/code] Next, let's split our data into train vs. test. We'll do a standard 70/30 split here.…
Read More
20Aug 2019 by Andrew Treadway
How to get an AUC confidence interval

How to get an AUC confidence interval

Machine Learning, R
Background AUC is an important metric in machine learning for classification. It is often used as a measure of a model's performance. In effect, AUC is a measure between 0 and 1 of a model's performance that rank-orders predictions from a model. For a detailed explanation of AUC, see this link. Since AUC is widely used, being able to get a confidence interval around this metric is valuable to both better demonstrate a model's performance, as well as to better compare two or more models. For example, if model A has an AUC higher than model B, but the 95% confidence interval around each AUC value overlaps, then the models may not be statistically different in performance. We can get a confidence interval around AUC using R's pROC package, which…
Read More
02Oct 2018 by Andrew Treadway
How to build a logistic regression model from scratch in R

How to build a logistic regression model from scratch in R

Machine Learning, R
Background In a previous post, we showed how using vectorization in R can vastly speed up fuzzy matching. Here, we will show you how to use vectorization to efficiently build a logistic regression model from scratch in R. Now we could just use the caret or stats packages to create a model, but building algorithms from scratch is a great way to develop a better understanding of how they work under the hood. Definitions & Assumptions In developing our code for the logistic regression algorithm, we will consider the following definitions and assumptions: x = A dxn matrix of d predictor variables, where each column xi represents the vector of predictors corresponding to one data point (with n such columns i.e. n data points) d = The number of predictor…
Read More
23Jun 2018 by Andrew Treadway
ICA on Images with Python

ICA on Images with Python

Machine Learning, Python
Click here to see my recommended reading list. What is Independent Component Analysis (ICA)? If you're already familiar with ICA, feel free to skip below to how we implement it in Python. ICA is a type of dimensionality reduction algorithm that transforms a set of variables to a new set of components; it does so such that that the statistical independence between the new components is maximized. This is similar to Principle Component Analysis (PCA), which maps a collection of variables to statistically uncorrelated components, except that ICA goes a step further by maximizing statistical independence rather than just developing components that are uncorrelated. Like other dimensionality reduction methods, ICA seeks to reduce the number of variables in a set of data, while retaining key information. In the example we…
Read More