setup
1
Introduction
2
Moving Beyond Linearity
2.1
Models Beyond Linearity
2.1.1
Polynomial Regression
2.1.1.1
Beta coefficients and variance
2.1.1.2
Application procedure
2.1.2
Step Functions
2.1.3
Regression Splines
2.1.3.1
Piecewise Polynomials
2.1.3.2
Constraints and Splines
2.1.3.3
Choosing the number and location of the Knots
2.1.3.4
Degrees of freedom
2.1.3.5
Basis splines vs. natural splines
2.1.3.6
Comparison with Polynomial Regression
2.1.4
Smoothing Splines
2.1.4.1
Choosing optimal tuning parameter
2.1.5
Local Regression
2.1.6
Generalized Additive Models
2.1.6.1
Feature Selection
2.1.6.2
GAM for regression problems
2.1.6.2.1
Interpretation of output
2.1.6.3
GAM for classification problems
2.1.7
Model assessment
2.2
Lecture notes
2.3
Lab section
2.3.1
Polynomial Regression and Step Functions
2.3.1.1
Continous model
2.3.1.2
Logarithmic model
2.3.1.3
Step function
2.3.2
Splines
2.3.2.1
Basis Function Splines
2.3.2.2
Natural Splines
2.3.2.3
Smooth Splines
2.3.2.4
Local Regression
2.3.3
GAMs
2.3.3.1
With only natural splines
2.3.3.2
With different splines
2.3.3.3
But what variables to include?
2.3.3.4
GAM with local regression
2.3.3.5
Logistic Regression
2.4
Exercises
2.4.1
Exercise 6
2.4.1.1
6.a Polynomial Regression
2.4.1.2
6.b Step function
2.4.2
Exercise 7
2.4.3
Exercise 8
2.4.4
Exercise 9
2.4.4.1
(a) using poly function to fit cubic polynomial regression
2.4.4.2
(b) Plotting polynomial fits for a range of polynomials
2.4.4.3
(c) Using CV to select best degree of d
2.4.4.4
(d) Use
bs()
to fit a regression spline
2.4.4.5
(e) Now fit a regression spline
2.4.4.6
(f) Perform cross-validation, to select degrees
2.4.5
Exercise 10
2.4.5.1
(a) Partitioning the data
2.4.5.2
(b) Fitting a GAM
2.4.5.3
(c) Evaluating on the test set
2.4.5.4
(d) Which variables appear to have a non linear relationship?
2.5
Casestudy - Predicting the Return on Advertising Spent
2.5.1
1. Background
2.5.2
2. Case study (Business Understanding Phase)
2.5.3
3. The data (Data Understanding Phase)
2.5.4
4. Specific requirements:
2.5.4.1
4.1 Task 1 - Import and overview
2.5.4.2
4.2 Task 2 - Data inspection
2.5.4.3
4.3 Task 3 - Building different models
2.5.4.3.1
A Generalized Additive Model (GAM) to predict ROAS
2.5.4.3.1.1
c1) Feature selection using regsubsets()
2.5.4.3.1.2
c2) Feature selection using step.GAM
2.5.4.3.1.3
c3) Fetaure selection using random forest
3
Tree Based Methods
3.1
The Basics of Decision Trees
3.1.1
Regression Trees
3.1.1.1
How to make the decision trees
3.1.1.1.1
The goal of regression
3.1.1.1.2
Tree Pruning & Algoritm
3.1.1.1.3
Setting contrains of the tree sise
3.1.2
Classification Trees
3.1.3
Tree vs. Lienar Models
3.1.4
Advantages and Disadvantages of Trees
3.2
Bagging, Random Forests, Boosting
3.2.1
Bagging (Bootstrap Aggregation)
3.2.2
Random Forests
3.2.3
Boosting (i.e. Gradient Boosting)
3.3
XGBoost
3.4
Application in R
3.4.1
Decision trees
3.4.2
Bagging
3.4.3
Random Forests
3.4.4
Boosting
3.4.5
XGBoost
3.5
Lab section
3.5.1
Fitting Classification Trees
3.5.2
Fitting Regression Trees
3.5.3
Bagging and Random Forests
3.5.3.1
Bagging
3.5.3.2
Random Forest
3.5.4
Boosting
3.6
Exercies
3.6.1
Exercise 7 - Decision Tree Assessment
3.6.2
But what does it really show?, ntrees are fixed at 500
3.6.3
Exercise 8 - DT/RF/Boosting
3.6.4
Exercise 9 - Decision Tree / Pruning
3.6.5
Exercise 10 - Boosting/GAM/LinearReg/Bagging - Comparison
3.6.6
Exercise 11 - Boosting
3.6.7
Exercise 12
3.7
Casestudy - Predicting Algae Blooms
3.7.1
1. Background
3.7.2
2. Case study (Business Understanding Phase)
3.7.3
3. The data (Data Understanding Phase)
3.7.4
4. Requirements:
3.7.4.1
4.1 Task 1: Understand and import data properly
3.7.4.2
4.2 Task 2: Inspect your data and do the required variable adaptations and transformations
3.7.4.3
4.3 Task 3: Build one or several predictive models and evaluate their performance.
3.7.4.3.1
4.3.1. Growing a regression tree
3.7.4.3.2
4.3.2 Pruning the tree
3.7.4.3.3
4.3.3.Model evaluation and selection
3.7.4.3.4
4.3.4. Ensemble methods: Random Forests
3.7.4.3.5
4.3.5. Predicting in the test sample
4
Support Vector Machines
4.1
Maximal Vector Machines
4.1.1
What is a hyperplane?
4.1.2
Classification Using a Separating Hyperplane
4.1.3
The Maximal Margin Classifier
4.1.4
Construction of the Maximal Margin Classifer
4.1.5
The Non-separable Case
4.2
Support Vector Classifiers
4.2.1
Overview of the Support Vector Classifier
4.2.2
Details of the Support Vector Classifer
4.3
Support Vector Machines
4.3.1
Classification with non linear decision boundary
4.3.2
The Support Vector Machine
4.3.3
SVMs with More than Two Classes
4.3.4
Relationship to logistic regression
4.4
Process of kernels methods
4.5
Lab
4.5.1
Support Vector Classifier
4.5.2
Support Vector Machine
4.5.3
ROC Curves
4.5.4
SVM with Multiple Classes
4.5.5
Application to Gene Expression Data
4.6
Exercises
5
Deep Learning Fundamentals
5.1
Basic Deep Learning
5.1.1
Terms
5.1.2
Optimizers, Loss, Metrics and Activation rules
5.1.2.1
Gradient Descents
5.1.3
Example with image recognizion
5.1.4
Model building
5.1.5
Validating the model
5.1.6
Overfitting / Underfitting
5.1.6.1
Hyperparameters:
5.1.6.2
Regularization:
5.1.6.3
Dropout
5.1.6.4
Then how do we control for a good fit?
5.1.7
Dealing with missing data
5.2
The workflow for building the neural network
5.2.1
Being aware of the process
5.3
When Neural Networks will fail
5.3.1
Hyper parameter tuning
6
Chapter 3 - Getting Started With Neural Networks
6.1
Positive / Negative IMDB reviews
6.1.1
Extracting the data
6.1.2
Preparing the data
6.1.3
Building the model
6.1.4
Assessing model performance
6.2
Multiclass classification - Classifying newswires
6.2.1
Loading the data
6.2.2
Preparing the data
6.2.3
Building the model
6.2.4
Validating the model + model assessment
6.2.5
Training model with optimal epochs
6.2.6
Predictions on new data
6.3
Continous prediction / a regression example - Predicting houseprices
6.3.1
Loading the data
6.3.2
Preparing the data
6.3.3
Building the model
6.3.3.1
Validating the approach using K-fold validation
6.3.3.2
Validation with more iterations
6.3.3.3
Tuning amount fo hidden layers
7
Chapter 5 - Deep learning for computer vision
7.1
Definition of convoluted network
7.2
Spetial heirarchical
7.2.1
Tuning parameters
7.2.2
Data modeling techniques
7.2.2.1
Padding
7.2.2.2
Strides
7.3
The max-pooling operation
7.4
Examples
7.4.1
Training a cats and dogs classifier from scratch.
7.4.1.1
Loading data
7.4.1.2
Building the model
7.4.1.3
Data preprocessing
7.4.1.4
Fitting the model
7.4.1.5
Dealing with overfitting
7.4.1.5.1
Randomized transformations / Data augmentation
7.4.1.5.2
Adding dropout
7.4.1.5.3
Max pooling
7.4.1.6
Training with dropout and random image transformations
7.5
Using a pretrained convnet
7.5.1
Feature extraction
7.5.1.1
Feature extraction without data augmentation
7.5.1.2
Feature extraction with data augmentation
7.5.2
Fine-tuning
7.6
Visualizing what convnets learn
8
Deep Learning for Text and Sequences
8.1
Working with Text Data
8.1.1
One-hot encoding of words and characters
8.1.2
Using word embeddings
8.1.2.1
The first approach
8.1.2.2
The second approach - using pretrained word embeddings
8.1.3
Putting it all together: from raw text to word embeddings
8.1.3.1
Preprocessing the embeddings
8.1.3.2
Defining a model
8.1.3.3
Loading GloVe embeddings in the model
8.1.3.4
Training and evaluating the model
8.1.3.5
Training and evaluating without GloVe
8.1.3.6
Using test data
8.2
Understanding Recurrent Neural Networks (RNN)
8.2.1
A recurrent layer in Keras
8.2.2
Understanding the LSTM and GRU layers
8.2.2.1
Units inside GRU and LSTM
8.2.3
A concrete LSTM example in Keras
8.3
Advanced use of Recurrent neural networks
8.3.1
A temperature-forecasting problem
8.3.2
Preparing the data
8.3.3
A common-sense, non-machine-learning baseline
8.3.4
A basic machine-learning approach
8.3.5
A first recurrent baseline
8.3.6
using recurrent dropout to fight overfitting
8.3.7
Stacking recurrent layers
8.3.8
Using bidirectional RNNs
8.3.9
Going even further
8.3.10
Wrap up
8.4
Sequence processing with convnets
8.4.1
Implementing a 1D convnet
8.4.2
Combining CNNs and RNNs to process long sequences
9
Advanced Deep-Learning Best Practices
9.1
Going beyond the sequential model: the Keras functino API
9.1.1
Introduction to the functional API
9.1.2
Multi-input models
9.1.3
Multi-output
9.1.4
Directed acyclic graphs of layers (DAG)
9.1.4.1
Inception modules
9.1.4.2
Residual Connection
9.1.5
Layer weight sharing
9.1.6
Models as layers
9.2
Inspecting and monitoring deep-learning models using Keras callba- acks and TensorBoard
9.2.1
Using callbacks to act on a model during training
9.2.1.1
The model-checkpoint and early-stopping callbacks
9.2.1.2
The reduce-learning-rate-on-plateau callback
9.2.1.3
Writing your own callback functions
9.2.2
Introduction to tensorBoard: the TensorFlow visualization framework
9.3
Getting the most of your models
9.3.1
Advanced architecture patterns
9.3.1.1
Batch normalization
9.3.1.2
Depthwise separable convolution
9.3.2
Hyperparameter optimization
9.3.3
Model ensembling
10
Structuring data transformation and model assessments
11
Github and CSS styling
11.1
Managing GitHub
11.2
CSS Styling
12
References
Machine Learning for Business Intelligence 2
1
Introduction
library
(dplyr)