arthur/handson-ml3
1
0
Fork 0
mirror of https://github.com/ArthurDanjou/handson-ml3.git synced 2026-01-14 12:14:36 +01:00
Code Issues Packages Projects Releases Wiki Activity

Update notebook to latest Colab library versions

Browse Source
This commit is contained in:
Aurélien Geron
2024-10-04 22:23:05 +13:00
parent d2b4c73e97
commit c694f7cfe4
1 changed files with 98 additions and 70 deletions
Download Patch File Download Diff File Expand all files Collapse all files

168
10_neural_nets_with_keras.ipynb
View File

@@ -1819,6 +1819,13 @@
" outputs=[output, aux_output])" " outputs=[output, aux_output])"
] ]
}, },
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Warning**: in recent versions, Keras requires one metric per output, so I replaced `metrics=[\"RootMeanSquaredError\"]` with `metrics=[\"RootMeanSquaredError\", \"RootMeanSquaredError\"]` in the code below."
]
},
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 62, "execution_count": 62,
@@ -1827,7 +1834,7 @@
"source": [ "source": [
"optimizer = tf.keras.optimizers.Adam(learning_rate=1e-3)\n", "optimizer = tf.keras.optimizers.Adam(learning_rate=1e-3)\n",
"model.compile(loss=(\"mse\", \"mse\"), loss_weights=(0.9, 0.1), optimizer=optimizer,\n", "model.compile(loss=(\"mse\", \"mse\"), loss_weights=(0.9, 0.1), optimizer=optimizer,\n",
" metrics=[\"RootMeanSquaredError\"])" " metrics=[\"RootMeanSquaredError\", \"RootMeanSquaredError\"])"
] ]
}, },
{ {
@@ -1906,7 +1913,7 @@
], ],
"source": [ "source": [
"eval_results = model.evaluate((X_test_wide, X_test_deep), (y_test, y_test))\n", "eval_results = model.evaluate((X_test_wide, X_test_deep), (y_test, y_test))\n",
"weighted_sum_of_losses, main_loss, aux_loss, main_rmse, aux_rmse = eval_results" "weighted_sum_of_losses, main_loss, aux_loss = eval_results"
] ]
}, },
{ {
@@ -1974,6 +1981,13 @@
"model = WideAndDeepModel(30, activation=\"relu\", name=\"my_cool_model\")" "model = WideAndDeepModel(30, activation=\"relu\", name=\"my_cool_model\")"
] ]
}, },
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Warning**: as explained above, Keras now requires one metric per output, so I replaced `metrics=[\"RootMeanSquaredError\"]` with `metrics=[\"RootMeanSquaredError\", \"RootMeanSquaredError\"]` in the code below."
]
},
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 68, "execution_count": 68,
@@ -2011,14 +2025,13 @@
"source": [ "source": [
"optimizer = tf.keras.optimizers.Adam(learning_rate=1e-3)\n", "optimizer = tf.keras.optimizers.Adam(learning_rate=1e-3)\n",
"model.compile(loss=\"mse\", loss_weights=[0.9, 0.1], optimizer=optimizer,\n", "model.compile(loss=\"mse\", loss_weights=[0.9, 0.1], optimizer=optimizer,\n",
" metrics=[\"RootMeanSquaredError\"])\n", " metrics=[\"RootMeanSquaredError\", \"RootMeanSquaredError\"])\n",
"model.norm_layer_wide.adapt(X_train_wide)\n", "model.norm_layer_wide.adapt(X_train_wide)\n",
"model.norm_layer_deep.adapt(X_train_deep)\n", "model.norm_layer_deep.adapt(X_train_deep)\n",
"history = model.fit(\n", "history = model.fit(\n",
" (X_train_wide, X_train_deep), (y_train, y_train), epochs=10,\n", " (X_train_wide, X_train_deep), (y_train, y_train), epochs=10,\n",
" validation_data=((X_valid_wide, X_valid_deep), (y_valid, y_valid)))\n", " validation_data=((X_valid_wide, X_valid_deep), (y_valid, y_valid)))\n",
"eval_results = model.evaluate((X_test_wide, X_test_deep), (y_test, y_test))\n", "eval_results = model.evaluate((X_test_wide, X_test_deep), (y_test, y_test))\n",
"weighted_sum_of_losses, main_loss, aux_loss, main_rmse, aux_rmse = eval_results\n",
"y_pred_main, y_pred_aux = model.predict((X_new_wide, X_new_deep))" "y_pred_main, y_pred_aux = model.predict((X_new_wide, X_new_deep))"
] ]
}, },
@@ -2029,6 +2042,13 @@
"## Saving and Restoring a Model" "## Saving and Restoring a Model"
] ]
}, },
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Warning**: Keras now recommends using the `.keras` format to save models, and the `h5` format for weights. Therefore I have updated the code in this section to first show what you need to change if you still want to use TensorFlow's `SavedModel` format, and then how you can use the recommended formats."
]
},
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 69, "execution_count": 69,
@@ -2042,21 +2062,20 @@
"shutil.rmtree(\"my_keras_model\", ignore_errors=True)" "shutil.rmtree(\"my_keras_model\", ignore_errors=True)"
] ]
}, },
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Warning**: Keras's `model.save()` method no longer supports TensorFlow's `SavedModel` format. However, you can still export models to the `SavedModel` format using `model.export()` like this:"
]
},
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 70, "execution_count": 70,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [],
{
"name": "stdout",
"output_type": "stream",
"text": [
"INFO:tensorflow:Assets written to: my_keras_model/assets\n"
]
}
],
"source": [ "source": [
"model.save(\"my_keras_model\", save_format=\"tf\")" "model.export(\"my_keras_model\")"
] ]
}, },
{ {
@@ -2083,14 +2102,28 @@
" print(path)" " print(path)"
] ]
}, },
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Warning**: In Keras 3, it is no longer possible to load a TensorFlow `SavedModel` as a Keras model. However, you can load a `SavedModel` as a `tf.keras.layers.TFSMLayer` layer, but be aware that this layer can only be used for inference: no training."
]
},
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 72, "execution_count": 72,
"metadata": {}, "metadata": {},
"outputs": [], "outputs": [],
"source": [ "source": [
"model = tf.keras.models.load_model(\"my_keras_model\")\n", "tfsm_layer = tf.keras.layers.TFSMLayer(\"my_keras_model\")\n",
"y_pred_main, y_pred_aux = model.predict((X_new_wide, X_new_deep))" "y_pred_main, y_pred_aux = tfsm_layer((X_new_wide, X_new_deep))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Warning**: Keras now requires the saved weights to have the `.weights.h5` extension. There are no longer saved using the `SavedModel` format."
] ]
}, },
{ {
@@ -2099,7 +2132,7 @@
"metadata": {}, "metadata": {},
"outputs": [], "outputs": [],
"source": [ "source": [
"model.save_weights(\"my_weights\")" "model.save_weights(\"my_weights.weights.h5\")"
] ]
}, },
{ {
@@ -2119,27 +2152,42 @@
} }
], ],
"source": [ "source": [
"model.load_weights(\"my_weights\")" "model.load_weights(\"my_weights.weights.h5\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"To save a model using the `.keras` format, simply use `model.save()`:"
] ]
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 75, "execution_count": 75,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [],
{
"name": "stdout",
"output_type": "stream",
"text": [
"my_weights.data-00000-of-00001\n",
"my_weights.index\n"
]
}
],
"source": [ "source": [
"# extra code – show the list of my_weights.* files\n", "model.save(\"my_model.keras\")"
"for path in sorted(Path().glob(\"my_weights.*\")):\n", ]
" print(path)" },
{
"cell_type": "markdown",
"metadata": {},
"source": [
"To load a `.keras` model, use the `tf.keras.models.load_model()` function. If the model uses any custom object, you must pass them to the function via the `custom_objects` argument:"
]
},
{
"cell_type": "code",
"execution_count": 76,
"metadata": {},
"outputs": [],
"source": [
"loaded_model = tf.keras.models.load_model(\n",
" \"my_model.keras\",\n",
" custom_objects={\"WideAndDeepModel\": WideAndDeepModel}\n",
")"
] ]
}, },
{ {
@@ -2151,7 +2199,7 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 76, "execution_count": 77,
"metadata": {}, "metadata": {},
"outputs": [], "outputs": [],
"source": [ "source": [
@@ -2159,39 +2207,17 @@
] ]
}, },
{ {
"cell_type": "code", "cell_type": "markdown",
"execution_count": 77,
"metadata": {}, "metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 1/10\n",
"363/363 [==============================] - 1s 2ms/step - loss: 0.3775 - output_1_loss: 0.3706 - output_2_loss: 0.4402 - output_1_root_mean_squared_error: 0.6088 - output_2_root_mean_squared_error: 0.6635 - val_loss: 0.3369 - val_output_1_loss: 0.3234 - val_output_2_loss: 0.4587 - val_output_1_root_mean_squared_error: 0.5687 - val_output_2_root_mean_squared_error: 0.6773\n",
"Epoch 2/10\n",
"363/363 [==============================] - 1s 1ms/step - loss: 0.3556 - output_1_loss: 0.3480 - output_2_loss: 0.4242 - output_1_root_mean_squared_error: 0.5899 - output_2_root_mean_squared_error: 0.6513 - val_loss: 0.4940 - val_output_1_loss: 0.4650 - val_output_2_loss: 0.7551 - val_output_1_root_mean_squared_error: 0.6819 - val_output_2_root_mean_squared_error: 0.8689\n",
"Epoch 3/10\n",
"363/363 [==============================] - 1s 1ms/step - loss: 0.3612 - output_1_loss: 0.3547 - output_2_loss: 0.4198 - output_1_root_mean_squared_error: 0.5956 - output_2_root_mean_squared_error: 0.6480 - val_loss: 0.3443 - val_output_1_loss: 0.3355 - val_output_2_loss: 0.4241 - val_output_1_root_mean_squared_error: 0.5792 - val_output_2_root_mean_squared_error: 0.6512\n",
"Epoch 4/10\n",
"363/363 [==============================] - 1s 1ms/step - loss: 0.3493 - output_1_loss: 0.3425 - output_2_loss: 0.4110 - output_1_root_mean_squared_error: 0.5852 - output_2_root_mean_squared_error: 0.6411 - val_loss: 0.4676 - val_output_1_loss: 0.4635 - val_output_2_loss: 0.5046 - val_output_1_root_mean_squared_error: 0.6808 - val_output_2_root_mean_squared_error: 0.7104\n",
"Epoch 5/10\n",
"363/363 [==============================] - 1s 1ms/step - loss: 0.3525 - output_1_loss: 0.3465 - output_2_loss: 0.4069 - output_1_root_mean_squared_error: 0.5886 - output_2_root_mean_squared_error: 0.6379 - val_loss: 1.3020 - val_output_1_loss: 1.3842 - val_output_2_loss: 0.5623 - val_output_1_root_mean_squared_error: 1.1765 - val_output_2_root_mean_squared_error: 0.7499\n",
"Epoch 6/10\n",
"363/363 [==============================] - 1s 1ms/step - loss: 0.3512 - output_1_loss: 0.3453 - output_2_loss: 0.4039 - output_1_root_mean_squared_error: 0.5876 - output_2_root_mean_squared_error: 0.6356 - val_loss: 1.6719 - val_output_1_loss: 1.7502 - val_output_2_loss: 0.9670 - val_output_1_root_mean_squared_error: 1.3230 - val_output_2_root_mean_squared_error: 0.9833\n",
"Epoch 7/10\n",
"363/363 [==============================] - 1s 1ms/step - loss: 0.3533 - output_1_loss: 0.3477 - output_2_loss: 0.4038 - output_1_root_mean_squared_error: 0.5897 - output_2_root_mean_squared_error: 0.6355 - val_loss: 0.6855 - val_output_1_loss: 0.7149 - val_output_2_loss: 0.4210 - val_output_1_root_mean_squared_error: 0.8455 - val_output_2_root_mean_squared_error: 0.6488\n",
"Epoch 8/10\n",
"363/363 [==============================] - 1s 1ms/step - loss: 0.3409 - output_1_loss: 0.3348 - output_2_loss: 0.3965 - output_1_root_mean_squared_error: 0.5786 - output_2_root_mean_squared_error: 0.6297 - val_loss: 2.0126 - val_output_1_loss: 1.9280 - val_output_2_loss: 2.7742 - val_output_1_root_mean_squared_error: 1.3885 - val_output_2_root_mean_squared_error: 1.6656\n",
"Epoch 9/10\n",
"363/363 [==============================] - 1s 1ms/step - loss: 0.3441 - output_1_loss: 0.3375 - output_2_loss: 0.4028 - output_1_root_mean_squared_error: 0.5810 - output_2_root_mean_squared_error: 0.6347 - val_loss: 1.6894 - val_output_1_loss: 1.8009 - val_output_2_loss: 0.6859 - val_output_1_root_mean_squared_error: 1.3420 - val_output_2_root_mean_squared_error: 0.8282\n",
"Epoch 10/10\n",
"363/363 [==============================] - 1s 1ms/step - loss: 0.3517 - output_1_loss: 0.3468 - output_2_loss: 0.3962 - output_1_root_mean_squared_error: 0.5889 - output_2_root_mean_squared_error: 0.6294 - val_loss: 1.2969 - val_output_1_loss: 1.3365 - val_output_2_loss: 0.9407 - val_output_1_root_mean_squared_error: 1.1561 - val_output_2_root_mean_squared_error: 0.9699\n"
]
}
],
"source": [ "source": [
"checkpoint_cb = tf.keras.callbacks.ModelCheckpoint(\"my_checkpoints\",\n", "**Warning**: as explained earlier, Keras now requires the checkpoint files to have a `.weights.h5` extension:"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"checkpoint_cb = tf.keras.callbacks.ModelCheckpoint(\"my_checkpoints.weights.h5\",\n",
" save_weights_only=True)\n", " save_weights_only=True)\n",
"history = model.fit(\n", "history = model.fit(\n",
" (X_train_wide, X_train_deep), (y_train, y_train), epochs=10,\n", " (X_train_wide, X_train_deep), (y_train, y_train), epochs=10,\n",
@@ -2719,7 +2745,7 @@
"outputs": [], "outputs": [],
"source": [ "source": [
"if \"google.colab\" in sys.modules:\n", "if \"google.colab\" in sys.modules:\n",
" %pip install -q -U keras_tuner" " %pip install -q -U keras_tuner~=1.4.6"
] ]
}, },
{ {
@@ -3343,10 +3369,12 @@
" self.factor = factor\n", " self.factor = factor\n",
" self.rates = []\n", " self.rates = []\n",
" self.losses = []\n", " self.losses = []\n",
" def on_batch_end(self, batch, logs):\n", "\n",
" self.rates.append(K.get_value(self.model.optimizer.learning_rate))\n", " def on_batch_end(self, batch, logs=None):\n",
" self.losses.append(logs[\"loss\"])\n", " lr = self.model.optimizer.learning_rate.numpy() * self.factor\n",
" K.set_value(self.model.optimizer.learning_rate, self.model.optimizer.learning_rate * self.factor)" " self.model.optimizer.learning_rate = lr\n",
" self.rates.append(lr)\n",
" self.losses.append(logs[\"loss\"])"
] ]
}, },
{ {
@@ -3593,7 +3621,7 @@
], ],
"source": [ "source": [
"early_stopping_cb = tf.keras.callbacks.EarlyStopping(patience=20)\n", "early_stopping_cb = tf.keras.callbacks.EarlyStopping(patience=20)\n",
"checkpoint_cb = tf.keras.callbacks.ModelCheckpoint(\"my_mnist_model\", save_best_only=True)\n", "checkpoint_cb = tf.keras.callbacks.ModelCheckpoint(\"my_mnist_model.keras\", save_best_only=True)\n",
"tensorboard_cb = tf.keras.callbacks.TensorBoard(run_logdir)\n", "tensorboard_cb = tf.keras.callbacks.TensorBoard(run_logdir)\n",
"\n", "\n",
"history = model.fit(X_train, y_train, epochs=100,\n", "history = model.fit(X_train, y_train, epochs=100,\n",
@@ -3625,7 +3653,7 @@
} }
], ],
"source": [ "source": [
"model = tf.keras.models.load_model(\"my_mnist_model\") # rollback to best model\n", "model = tf.keras.models.load_model(\"my_mnist_model.keras\") # rollback to best model\n",
"model.evaluate(X_test, y_test)" "model.evaluate(X_test, y_test)"
] ]
}, },