arthur/handson-ml3
1
0
Fork 0
mirror of https://github.com/ArthurDanjou/handson-ml3.git synced 2026-01-14 12:14:36 +01:00
Code Issues Packages Projects Releases Wiki Activity

Replace handson-ml2 with handson-ml3, and fix figure chapter numbers

Browse Source
This commit is contained in:
Aurélien Geron
2021-11-23 15:42:16 +13:00
parent e38983d595
commit 5bb0366125
23 changed files with 137 additions and 137 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
docker/.env
View File

@@ -1 +1 @@
COMPOSE_PROJECT_NAME=handson-ml2
COMPOSE_PROJECT_NAME=handson-ml3

4
docker/Dockerfile
View File

@@ -31,7 +31,7 @@ ARG username
ARG userid
ARG home=/home/${username}
ARG workdir=${home}/handson-ml2
ARG workdir=${home}/handson-ml3
RUN adduser ${username} --uid ${userid} --gecos '' --disabled-password \
&& echo "${username} ALL=(root) NOPASSWD:ALL" > /etc/sudoers.d/${username} \
@@ -47,7 +47,7 @@ ENV PATH /opt/conda/envs/tf2/bin:$PATH
# The config below enables diffing notebooks with nbdiff (and nbdiff support
# in git diff command) after connecting to the container by "make exec" (or
# "docker-compose exec handson-ml2 bash")
# "docker-compose exec handson-ml3 bash")
# You may also try running:
# nbdiff NOTEBOOK_NAME.ipynb
# to get nbdiff between checkpointed version and current version of the

10
docker/Dockerfile.gpu
View File

@@ -27,7 +27,7 @@ ARG LIBNVINFER_MAJOR_VERSION=7
# Needed for string substitution
SHELL ["/bin/bash", "-c"]
# Pick up some TF dependencies
# [HOML2] Tweaked for handson-ml2: added all the libs before build-essentials
# [HOML2] Tweaked for handson-ml3: added all the libs before build-essentials
# and call apt clean + remove apt cache.
RUN apt-get update -q && apt-get install -q -y --no-install-recommends \
bzip2 \
@@ -87,7 +87,7 @@ RUN ln -s /usr/local/cuda/lib64/stubs/libcuda.so /usr/local/cuda/lib64/stubs/lib
&& echo "/usr/local/cuda/lib64/stubs" > /etc/ld.so.conf.d/z-cuda-stubs.conf \
&& ldconfig
# [HOML2] Tweaked for handson-ml2: removed Python3 & TensorFlow installation using pip
# [HOML2] Tweaked for handson-ml3: removed Python3 & TensorFlow installation using pip
#################################################
#### End of tensorflow/tensorflow:latest-gpu ####
@@ -102,7 +102,7 @@ ENV PATH /opt/conda/bin:/opt/conda/envs/tf2/bin:$PATH
#### FROM continuumio/miniconda3:latest ####
############################################
# [HOML2] Tweaked for handson-ml2: removed the beginning of the Dockerfile
# [HOML2] Tweaked for handson-ml3: removed the beginning of the Dockerfile
CMD [ "/bin/bash" ]
# Leave these args here to better use the Docker build cache
@@ -141,7 +141,7 @@ ARG username
ARG userid
ARG home=/home/${username}
ARG workdir=${home}/handson-ml2
ARG workdir=${home}/handson-ml3
RUN adduser ${username} --uid ${userid} --gecos '' --disabled-password \
&& echo "${username} ALL=(root) NOPASSWD:ALL" > /etc/sudoers.d/${username} \
@@ -156,7 +156,7 @@ WORKDIR ${workdir}
# The config below enables diffing notebooks with nbdiff (and nbdiff support
# in git diff command) after connecting to the container by "make exec" (or
# "docker-compose exec handson-ml2 bash")
# "docker-compose exec handson-ml3 bash")
# You may also try running:
# nbdiff NOTEBOOK_NAME.ipynb
# to get nbdiff between checkpointed version and current version of the

4
docker/Makefile
View File

@@ -4,11 +4,11 @@ help:
run:
docker-compose up
exec:
docker-compose exec handson-ml2 bash
docker-compose exec handson-ml3 bash
build: stop .FORCE
docker-compose build
rebuild: stop .FORCE
docker-compose build --no-cache
stop:
docker stop handson-ml2 || true; docker rm handson-ml2 || true;
docker stop handson-ml3 || true; docker rm handson-ml3 || true;
.FORCE:

34
docker/README.md
View File

@@ -16,34 +16,34 @@ Some general knowledge about `docker` infrastructure might be useful (that's an
The first option is to pull the image from Docker Hub (this will download about 1.9 GB of compressed data):
```bash
$ docker pull ageron/handson-ml2
$ docker pull ageron/handson-ml3
```
**Note**: this is the CPU-only image. For GPU support, read the GPU section below.
Alternatively, you can build the image yourself. This will be slower, but it will ensure the image is up to date, with the latest libraries. For this, assuming you already downloaded this project into the directory `/path/to/project/handson-ml2`:
Alternatively, you can build the image yourself. This will be slower, but it will ensure the image is up to date, with the latest libraries. For this, assuming you already downloaded this project into the directory `/path/to/project/handson-ml3`:
```bash
$ cd /path/to/project/handson-ml2/docker
$ cd /path/to/project/handson-ml3/docker
$ docker-compose build
```
This will take quite a while, but is only required once.
After the process is finished you have an `ageron/handson-ml2:latest` image, that will be the base for your experiments. You can confirm that by running the following command:
After the process is finished you have an `ageron/handson-ml3:latest` image, that will be the base for your experiments. You can confirm that by running the following command:
```bash
$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
ageron/handson-ml2 latest 3ebafebc604a 2 minutes ago 4.87GB
ageron/handson-ml3 latest 3ebafebc604a 2 minutes ago 4.87GB
```
### Run the notebooks
Still assuming you already downloaded this project into the directory `/path/to/project/handson-ml2`, run the following commands to start the Jupyter server inside the container, which is named `handson-ml2`:
Still assuming you already downloaded this project into the directory `/path/to/project/handson-ml3`, run the following commands to start the Jupyter server inside the container, which is named `handson-ml3`:
```bash
$ cd /path/to/project/handson-ml2/docker
$ cd /path/to/project/handson-ml3/docker
$ docker-compose up
```
@@ -55,13 +55,13 @@ You can close the server just by pressing `Ctrl-C` in the terminal window.
### Using `make` (optional)
If you have `make` installed on your computer, you can use it as a thin layer to run `docker-compose` commands. For example, executing `make rebuild` will actually run `docker-compose build --no-cache`, which will rebuild the image without using the cache. This ensures that your image is based on the latest version of the `continuumio/miniconda3` image which the `ageron/handson-ml2` image is based on.
If you have `make` installed on your computer, you can use it as a thin layer to run `docker-compose` commands. For example, executing `make rebuild` will actually run `docker-compose build --no-cache`, which will rebuild the image without using the cache. This ensures that your image is based on the latest version of the `continuumio/miniconda3` image which the `ageron/handson-ml3` image is based on.
If you don't have `make` (and you don't want to install it), just examine the contents of `Makefile` to see which `docker-compose` commands you can run instead.
### Run additional commands in the container
Run `make exec` (or `docker-compose exec handson-ml2 bash`) while the server is running to run an additional `bash` shell inside the `handson-ml2` container. Now you're inside the environment prepared within the image.
Run `make exec` (or `docker-compose exec handson-ml3 bash`) while the server is running to run an additional `bash` shell inside the `handson-ml3` container. Now you're inside the environment prepared within the image.
One of the useful things that can be done there would be starting TensorBoard (for example with simple `tb` command, see bashrc file).
@@ -81,12 +81,12 @@ If you're running on Linux, and you have a TensorFlow-compatible GPU card (NVidi
Next, edit the `docker-compose.yml` file:
```bash
$ cd /path/to/project/handson-ml2/docker
$ cd /path/to/project/handson-ml3/docker
$ edit docker-compose.yml # use your favorite editor
```
* Replace `dockerfile: ./docker/Dockerfile` with `dockerfile: ./docker/Dockerfile.gpu`
* Replace `image: ageron/handson-ml2:latest` with `image: ageron/handson-ml2:latest-gpu`
* Replace `image: ageron/handson-ml3:latest` with `image: ageron/handson-ml3:latest-gpu`
* If you want to use `docker-compose`, you will need version 1.28 or above for GPU support, and you must uncomment the whole `deploy` section in `docker-compose.yml`.
### Prepare the image (once)
@@ -94,13 +94,13 @@ $ edit docker-compose.yml # use your favorite editor
If you want to pull the prebuilt image from Docker Hub (this will download over 3.5 GB of compressed data):
```bash
$ docker pull ageron/handson-ml2:latest-gpu
$ docker pull ageron/handson-ml3:latest-gpu
```
If you prefer to build the image yourself:
```bash
$ cd /path/to/project/handson-ml2/docker
$ cd /path/to/project/handson-ml3/docker
$ docker-compose build
```
@@ -109,7 +109,7 @@ $ docker-compose build
If you have `docker-compose` version 1.28 or above, that's great! You can simply run:
```bash
$ cd /path/to/project/handson-ml2/docker
$ cd /path/to/project/handson-ml3/docker
$ docker-compose up
[...]
or http://127.0.0.1:8888/?token=[...]
@@ -132,8 +132,8 @@ If you have a version of `docker-compose` earlier than 1.28, you will have to us
If you are using Docker 19.03 or above, you can run:
```bash
$ cd /path/to/project/handson-ml2
$ docker run --name handson-ml2 --gpus all -p 8888:8888 -p 6006:6006 --log-opt mode=non-blocking --log-opt max-buffer-size=50m -v `pwd`:/home/devel/handson-ml2 ageron/handson-ml2:latest-gpu /opt/conda/envs/tf2/bin/jupyter notebook --ip='0.0.0.0' --port=8888 --no-browser
$ cd /path/to/project/handson-ml3
$ docker run --name handson-ml3 --gpus all -p 8888:8888 -p 6006:6006 --log-opt mode=non-blocking --log-opt max-buffer-size=50m -v `pwd`:/home/devel/handson-ml3 ageron/handson-ml3:latest-gpu /opt/conda/envs/tf2/bin/jupyter notebook --ip='0.0.0.0' --port=8888 --no-browser
```
If you are using an older version of Docker, then replace `--gpus all` with `--runtime=nvidia`.
@@ -143,7 +143,7 @@ Now point your browser to the displayed URL: Jupyter should appear, and you can
Lastly, to interrupt the server, press Ctrl-C, then run:
```bash
$ docker rm handson-ml2
$ docker rm handson-ml3
```
This will remove the container so you can start a new one later (but it will not remove the image or the notebooks, don't worry!).

8
docker/docker-compose.yml
View File

@@ -1,14 +1,14 @@
version: "3"
services:
handson-ml2:
handson-ml3:
build:
context: ../
dockerfile: ./docker/Dockerfile #Dockerfile.gpu
args:
- username=devel
- userid=1000
container_name: handson-ml2
image: ageron/handson-ml2:latest #latest-gpu
container_name: handson-ml3
image: ageron/handson-ml3:latest #latest-gpu
restart: unless-stopped
logging:
driver: json-file
@@ -18,7 +18,7 @@ services:
- "8888:8888"
- "6006:6006"
volumes:
- ../:/home/devel/handson-ml2
- ../:/home/devel/handson-ml3
command: /opt/conda/envs/tf2/bin/jupyter notebook --ip='0.0.0.0' --port=8888 --no-browser
#deploy:
# resources: