Update libraries to latest version, including TensorFlow 2.4.1 and Scikit-Learn 0.24.1

11_training_deep_neural_networks.ipynb

@@ -1039,7 +1039,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Great! We got quite a bit of transfer: the error rate dropped by a factor of 4!"
+    "Great! We got quite a bit of transfer: the error rate dropped by a factor of 4.5!"
    ]
   },
   {
@@ -1048,7 +1048,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "(100 - 96.95) / (100 - 99.25)"
+    "(100 - 97.05) / (100 - 99.35)"
    ]
   },
   {
@@ -2274,7 +2274,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "The model with the lowest validation loss gets about 47% accuracy on the validation set. It took 39 epochs to reach the lowest validation loss, with roughly 10 seconds per epoch on my laptop (without a GPU). Let's see if we can improve performance using Batch Normalization."
+    "The model with the lowest validation loss gets about 47.6% accuracy on the validation set. It took 27 epochs to reach the lowest validation loss, with roughly 8 seconds per epoch on my laptop (without a GPU). Let's see if we can improve performance using Batch Normalization."
    ]
   },
   {
@@ -2339,9 +2339,9 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "* *Is the model converging faster than before?* Much faster! The previous model took 39 epochs to reach the lowest validation loss, while the new model with BN took 18 epochs. That's more than twice as fast as the previous model. The BN layers stabilized training and allowed us to use a much larger learning rate, so convergence was faster.\n",
-    "* *Does BN produce a better model?* Yes! The final model is also much better, with 55% accuracy instead of 47%. It's still not a very good model, but at least it's much better than before (a Convolutional Neural Network would do much better, but that's a different topic, see chapter 14).\n",
-    "* *How does BN affect training speed?* Although the model converged twice as fast, each epoch took about 16s instead of 10s, because of the extra computations required by the BN layers. So overall, although the number of epochs was reduced by 50%, the training time (wall time) was shortened by 30%. Which is still pretty significant!"
+    "* *Is the model converging faster than before?* Much faster! The previous model took 27 epochs to reach the lowest validation loss, while the new model achieved that same loss in just 5 epochs and continued to make progress until the 16th epoch. The BN layers stabilized training and allowed us to use a much larger learning rate, so convergence was faster.\n",
+    "* *Does BN produce a better model?* Yes! The final model is also much better, with 54.0% accuracy instead of 47.6%. It's still not a very good model, but at least it's much better than before (a Convolutional Neural Network would do much better, but that's a different topic, see chapter 14).\n",
+    "* *How does BN affect training speed?* Although the model converged much faster, each epoch took about 12s instead of 8s, because of the extra computations required by the BN layers. But overall the training time (wall time) was shortened significantly!"
    ]
   },
   {
@@ -2412,7 +2412,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "We get 51.4% accuracy, which is better than the original model, but not quite as good as the model using batch normalization. Moreover, it took 13 epochs to reach the best model, which is much faster than both the original model and the BN model, plus each epoch took only 10 seconds, just like the original model. So it's by far the fastest model to train (both in terms of epochs and wall time)."
+    "We get 47.9% accuracy, which is not much better than the original model (47.6%), and not as good as the model using batch normalization (54.0%). However, convergence was almost as fast as with the BN model, plus each epoch took only 7 seconds. So it's by far the fastest model to train so far."
    ]
   },
   {
@@ -2473,7 +2473,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "The model reaches 50.8% accuracy on the validation set. That's very slightly worse than without dropout (51.4%). With an extensive hyperparameter search, it might be possible to do better (I tried dropout rates of 5%, 10%, 20% and 40%, and learning rates 1e-4, 3e-4, 5e-4, and 1e-3), but probably not much better in this case."
+    "The model reaches 48.9% accuracy on the validation set. That's very slightly better than without dropout (47.6%). With an extensive hyperparameter search, it might be possible to do better (I tried dropout rates of 5%, 10%, 20% and 40%, and learning rates 1e-4, 3e-4, 5e-4, and 1e-3), but probably not much better in this case."
    ]
   },
   {
@@ -2561,7 +2561,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "We only get virtually no accuracy improvement in this case (from 50.8% to 50.9%).\n",
+    "We get no accuracy improvement in this case (we're still at 48.9% accuracy).\n",
     "\n",
     "So the best model we got in this exercise is the Batch Normalization model."
    ]
@@ -2655,7 +2655,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "One cycle allowed us to train the model in just 15 epochs, each taking only 3 seconds (thanks to the larger batch size). This is over 3 times faster than the fastest model we trained so far. Moreover, we improved the model's performance (from 50.8% to 52.8%). The batch normalized model reaches a slightly better performance, but it's much slower to train."
+    "One cycle allowed us to train the model in just 15 epochs, each taking only 2 seconds (thanks to the larger batch size). This is several times faster than the fastest model we trained so far. Moreover, we improved the model's performance (from 47.6% to 52.0%). The batch normalized model reaches a slightly better performance (54%), but it's much slower to train."
    ]
   },
   {
@@ -2682,7 +2682,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.7.6"
+   "version": "3.7.9"
   },
   "nav_menu": {
    "height": "360px",