Industry recognized certification enables you to add this credential to your resume upon completion of all courses

Need Custom Training for Your Team?
Get Quote
Call Us

Toll Free (844) 397-3739

Inquire About This Course
Benjamin Holmes, Instructor - Model Tuning for Machine Learning

Benjamin Holmes

Data Scientist and a Machine Learning Engineer. Worked in scientific, governmental, and business contexts. Drove insights for research in biotechnology, genetics, neuroscience, and clinical technology.

Instructor: Benjamin Holmes

Learn how to make sure you are getting the best predictions your model can provide.

This course includes real-world examples, coding assignments, and lots of in-depth exploration of how and why model tuning should be done. 

Instructor is a Data Scientist and a Machine Learning Engineer. Worked in scientific, governmental, and business contexts.

Course Description

This course will help you slingshot the predictive capabilities of your models, far out-pacing the limits of out-of-box ML. From a ground-up perspective, we'll understand how a model functions, the part of the model that is able to fit the data on its' own, and how important additional tuning and fitting by a trained ML engineer is. This module includes real-world examples, coding assignments, and lots of in-depth exploration of how and why model tuning should be done. If you understand the material in this course, your models will improve, and the results you will be able to deliver will as well!

What am I going to get from this course?

Exceed the performance of out-of-box machine learning models - drastically!

You will learn how to make sure you are getting the best predictions your model can

provide, if you should change your model, or even if you could benefit from

packaging multiple models together to get the best fit for your data.

This course is a crucial step towards becoming an expert data scientist - moving from

simply knowing how machine learning works, to understanding on an intuitive

level how these algorithms model data, and the state of the art techniques that

are available to improve their performance. 


Prerequisites and Target Audience

What will students need to know or do before starting this course?

Students should have taken a basic course in machine learning, either in Experfy, or

have studied independently enough to understand the prerequisites listed above.

Additionally, they should have access to and ability with a basic programming

environment, such as python or R. 


Who should take this course? Who should not?

This course is appropriate for anyone who will be involved in the planning and

implementation of machine learning or AI algorithms. Knowing how to tune a

model is a crucial step in using machine learning, so this course should be

taken by anyone who plans on using, maintaining, or understanding these models.

This course is hands-on and technical, and so would be of limited use to those who don't

plan on understanding the details of how a machine learning system is




Module 1: Model Tuning Introduction

Lecture 1 Introduction and expectation-setting

Students will learn about model tuning - why it is important, and how it separates real machine learning engineers from beginners, or those just implementing out-of-box libraries.

Lecture 2 Hyperparameters

This lecture will introduce the first topic in model tuning - hyper-parameters. We will go in detail about what they are, how they're used, and how understanding them can result in early, easy wins for the model.

Lecture 3 Intro to Bayesianism

Bayesian reasoning is a powerful tool, especially when it comes to tuning machine learning models in complicated, incomplete environments. In this module we will review bayesian statistics, and discuss how it can be applied to ML models.

Lecture 4 Intro to Bayesian Model Averaging

Bayesian model averaging is a powerful tool that can be used to drastically increase the performance of ML models. We'll give a theoretical framework for it in this module.

Lecture 5 Bayesian Model Averaging- Specification

Continuing our delve into Bayesian Model Averaging, we'll go over a practical example, and show how it might be applied to problems you encounter.

Lecture 6 Occam's Window

Discussing the theoretical difficulties surrounding BMA, we'll show some tricks used to overcome these difficulties, including Occam's Window.

Lecture 7 Computing the Integral

This module finishing up our discussion of the theoretical difficulties with BMA, discussing the difficulties in computing complex integrals, and reviewing some techniques used to either compute or approximate it.

Lecture 8 Bayesian Model Averaging-Worked Example

Finally, we'll delve into a real example of BMA and show how it can deliver great results - and set you up to try one yourself!

Module 2: Bagging

Lecture 9 Intro to Bootstrap Aggregation

Bootstrap Aggregation is a key concept in the construction of more sophisticated and powerful machine learning models. We'll discuss it's concept and implementation here.

Lecture 10 Intro to Bootstrap Aggregation- CART

We'll work an example of bagging using it's application to classification and regression trees, or CARTs. This is a good intro to how bagging is used in the real world.

Module 3: Boosting

Lecture 11 Problem with Bagged Decision Trees

Why not be content with bagged decision trees? We'll discuss problems with that framework, and suggest a simple way they can be fixed with the intro to boosting.

Lecture 12 Random Forests- Start to Finish

Random Forests by themselves can be powerful ML tools, so let's first understand how they work, and then show how boosting can make them truly powerful!

Lecture 13 Random Forests: Time-Accuracy Tradeoff

One concept to always keep in mine when tuning a model is the time-accuracy tradeoff. At this point in the lecture series, we know enough to illustrate how if we used every tool exhaustively, this can be a prohibitive time investment. Let's review it here!

Lecture 14 Boosted Trees- Differences from Random Forest

Building on the framework we have for understanding random forests, let's now delve deeply into how this basic setup can be used to accommodate boosting!

Lecture 15 Boosted trees- Adaboost Procedure

Adaboost is very close to being best-in-class, and it's a simple implementation of boosted trees. This module clearly explains the whats and hows of Adaboost.

Lecture 16 XG Boost- Gradient Boosting

The final piece of the puzzle needed to get us to the best-in-class XGBoost algorithm is gradient boosting. We'll explore that concept here, and start to see how it can apply to our algorithm.

Lecture 17 Boosted Trees- Final Decision

Apply everything we've learned, let's see how XGBoost and boosted trees can work on real-world problems.

Module 4: Hyper-Parameters

Lecture 18 Introduction to hyper-parameters- Basics

To truly understand a model, you must know which parts of the algorithm are tuned by the model itself, and which you must adjust. Let's review hyper-parameters, and how important they are to model tuning.

Lecture 19 Hyperparameters in decision trees

Decision trees are an easily-understandable ML model (and the basis of much of what we've discussed!). Let's look at hyper-parameters in the context of these models.

Lecture 20 Hyperparamters in decision trees- Levels

Understanding tradeoffs is a major part of model tuning. Explore how adjusting a hyper-parameter gives us very different results in a decision tree.

Lecture 21 Hyperparameters in decision trees- AUC

With so many potential values for hyper-parameters, how do we know which to pick? Let's investigate AUC as a measure of model fitness.

Lecture 22 Finding optimal hyperparameters- Brute Force

How to find the optimal hyper-parameter? Sometimes the best ideas to start with are the simplest - let's consider the pros and cons of the brute force method.

Lecture 23 Finding optimal hyperparameters- Sanity Check

Understanding tradeoffs is a recurring theme. Let's understand the choices we're making, and think how to best find the happy medium that makes for good hyper-parameters.

Module 5: Stacking

Lecture 24 Intro to Stacking

Model stacking is a competition-winner, suitable for problems that would otherwise be intractable. Let's lay down the foundation and understand the concepts behind it.

Lecture 25 Intro to stacking

The best way to understand is to see the basic concepts in action - here we'll review those concepts, and begin to see their application.

Lecture 26 Intro to stacking- Motivation

Why should model stacking be able to deliver such dramatic results? Some examples show precisely why this technique is so powerful.

Lecture 27 Stacking- Pedigree

Now that we understand the theory, let's see some of the real-world results model stacking has been able to accomplish.

Module 6: Finale

Lecture 28 Conceptual Overview

Let's remember the basics of model tuning - why we need to tune, the first steps, and how it sets us apart from those who don't fully understand it.

Lecture 29 Start Simple

Even the most complicated models we saw started with a comprehensible framework. Let's discuss the solid foundation required for effective model tuning.

Lecture 30 Know your data

One of the most important mantras for any data scientist, "know your data" is a crucial step before tuning the data - models should fit the data!

Lecture 31 Time/Value tradeoff

We review the tradeoff that we've been discussing for this lecture series. How do we intelligently decide when the model is fit enough, and the rewards are diminishing?

Lecture 32 Example scenario- Network transactions

Finally, let's end with another comprehensive example - this should tie together the course and illustrate our points!