arthur/ml_exercises
1
0
Fork 0
mirror of https://github.com/ArthurDanjou/ml_exercises.git synced 2026-01-29 11:20:30 +01:00
Code Issues Packages Projects Releases Wiki Activity

further explanations in the notebooks

Browse Source
This commit is contained in:
franzi
2021-09-25 15:28:19 +02:00
parent f6017bc1ed
commit 44e88622a1
11 changed files with 236 additions and 108 deletions
Download Patch File Download Diff File Expand all files Collapse all files

41
exercises/3_supervised_comparison.ipynb
View File

@@ -24,10 +24,7 @@
"from sklearn.datasets import make_moons\n",
"# don't get unneccessary warnings\n",
"import warnings\n",
"warnings.simplefilter(action='ignore', category=FutureWarning)\n",
"\n",
"%load_ext autoreload\n",
"%autoreload 2"
"warnings.simplefilter(action='ignore', category=FutureWarning)"
]
},
{
@@ -148,7 +145,10 @@
"source": [
"## Datasets\n",
"\n",
"Here you can have a look at the 3 regression and 3 classification datasets on which we'll compare the different models. The regression dataset only has one input feature, while the classification dataset has two and the target (i.e. class label) is indicated by the color of the dots."
"Here you can have a look at the 3 regression and 3 classification datasets on which we'll compare the different models. The regression dataset only has one input feature, while the classification dataset has two and the target (i.e., class label) is indicated by the color of the dots.\n",
"\n",
"**Questions:**\n",
"- Why are the first two regression and classification datasets linear and the last ones non-linear?"
]
},
{
@@ -240,7 +240,7 @@
"source": [
"# Logistic Regression (for classification problems!):\n",
"# C (> 0): regularization (smaller values = more regularization)\n",
"# penalty: change to \"l1\" to get sparse weights (only if you have many features)\n",
"# penalty: change to \"l1\" to get sparse weights (only if you have many features; needs a different solver)\n",
"X, y = X_clf_2, y_clf_2\n",
"model = LogisticRegression(penalty=\"l2\", C=100.)\n",
"model.fit(X, y)\n",
@@ -278,8 +278,8 @@
"outputs": [],
"source": [
"# Decision Tree for regression:\n",
"# max_depth (> 1): depth of the tree (i.e. how many decisions are made before the final prediction)\n",
"# min_samples_leaf (> 1): how many training points are in one prediction bucket\n",
"# max_depth (>= 1): depth of the tree (i.e. how many decisions are made before the final prediction)\n",
"# min_samples_leaf (>= 1): how many training points are in one prediction bucket\n",
"X, y = X_reg_3, y_reg_3\n",
"model = DecisionTreeRegressor(max_depth=2, min_samples_leaf=10)\n",
"model.fit(X, y)\n",
@@ -293,8 +293,8 @@
"outputs": [],
"source": [
"# Decision Tree for classification:\n",
"# max_depth (> 1): depth of the tree (i.e. how many decisions are made before the final prediction)\n",
"# min_samples_leaf (> 1): how many training points are in one prediction bucket\n",
"# max_depth (>= 1): depth of the tree (i.e. how many decisions are made before the final prediction)\n",
"# min_samples_leaf (>= 1): how many training points are in one prediction bucket\n",
"X, y = X_clf_1, y_clf_1\n",
"model = DecisionTreeClassifier(max_depth=2, min_samples_leaf=10)\n",
"model.fit(X, y)\n",
@@ -329,9 +329,9 @@
"outputs": [],
"source": [
"# Random Forest for regression:\n",
"# n_estimators (> 1): how many decision trees to train (don't set this too high, gets computationally expensive)\n",
"# max_depth (> 1): depth of the tree (i.e. how many decisions are made before the final prediction)\n",
"# min_samples_leaf (> 1): how many training points are in one prediction bucket\n",
"# n_estimators (>= 1): how many decision trees to train (don't set this too high, gets computationally expensive)\n",
"# max_depth (>= 1): depth of the tree (i.e. how many decisions are made before the final prediction)\n",
"# min_samples_leaf (>= 1): how many training points are in one prediction bucket\n",
"X, y = X_reg_3, y_reg_3\n",
"model = RandomForestRegressor(n_estimators=100, max_depth=2, min_samples_leaf=10)\n",
"model.fit(X, y)\n",
@@ -345,9 +345,9 @@
"outputs": [],
"source": [
"# Random Forest for classification:\n",
"# n_estimators (> 1): how many decision trees to train\n",
"# max_depth (> 1): depth of the tree (i.e. how many decisions are made before the final prediction)\n",
"# min_samples_leaf (> 1): how many training points are in one prediction bucket\n",
"# n_estimators (>= 1): how many decision trees to train\n",
"# max_depth (>= 1): depth of the tree (i.e. how many decisions are made before the final prediction)\n",
"# min_samples_leaf (>= 1): how many training points are in one prediction bucket\n",
"X, y = X_clf_2, y_clf_2\n",
"model = RandomForestClassifier(n_estimators=100, max_depth=2, min_samples_leaf=10)\n",
"model.fit(X, y)\n",
@@ -363,7 +363,8 @@
"After reading the chapter on k-nearest neighbors, test the method here on different datasets and experiment with the hyperparameter settings.\n",
"\n",
"**Questions:**\n",
"- On the 3rd regression dataset for a larger number of nearest neighbors (e.g. 20), what do you observe for the prediction at the edges of the input domain and why?"
"- On the 3rd regression dataset for a larger number of nearest neighbors (e.g., 20), what do you observe for the prediction at the edges of the input domain and why?\n",
"- Especially for binary classification problems, why does it make sense to always use an odd number of nearest neighbors?"
]
},
{
@@ -382,7 +383,7 @@
"outputs": [],
"source": [
"# k-Nearest Neighbors for regression:\n",
"# n_neighbors (> 1): how many nearest neighbors are used for the prediction\n",
"# n_neighbors (>= 1): how many nearest neighbors are used for the prediction\n",
"X, y = X_reg_3, y_reg_3\n",
"model = KNeighborsRegressor(n_neighbors=10)\n",
"model.fit(X, y)\n",
@@ -396,9 +397,9 @@
"outputs": [],
"source": [
"# k-Nearest Neighbors for classification:\n",
"# n_neighbors (> 1): how many nearest neighbors are used for the prediction\n",
"# n_neighbors (>= 1): how many nearest neighbors are used for the prediction\n",
"X, y = X_clf_3, y_clf_3\n",
"model = KNeighborsClassifier(n_neighbors=12)\n",
"model = KNeighborsClassifier(n_neighbors=11)\n",
"model.fit(X, y)\n",
"plot_classification(X, y, model)"
]