antisaWiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
{{tag>docker troubleshooting ansible}}

====== Docker troubleshooting ======
===== DOCKER-ISOLATION-STAGE-1 =====

  An unexpected docker error occurred: 500 Server Error: Internal Server Error ("unable to insert jump to DOCKER-ISOLATION-STAGE-1 rule in FORWARD chain:  (iptables failed: iptables --wait -I FORWARD -j DOCKER-ISOLATION-STAGE-1: iptables v1.8.7 (nf_tables): Chain 'DOCKER-ISOLATION-STAGE-1' does not exist
    Try `iptables -h' or 'iptables --help' for more information.
     (exit status 2))")
  An exception occurred during task execution. To see the full traceback, use -vvv. The error was:  (exit status 2))")

Restart the docker service:
  systemctl restart docker.service

Or create that chain in filter table
  iptables -t filter -N DOCKER-ISOLATION-STAGE-1

This is [[https://www.mail-archive.com/debian-bugs-dist@lists.debian.org/msg1768269.html|supposed]] to be fixed in version 5.2.3.7


==== Tested on ====

  * Docker 20.10.12
  * Debian 11
  * shorewall 5.2.3.4

===== Volume shadowing =====
This is a tricky behaviour from Docker. Say you have a volume ///app/my_data// with some preexisting content which you mount in the container in the Dockerfile. This volume has some files there and folders

  a.txt
  b.txt
  private/
  private/c.txt
  private/d.txt

Your Dockerfile might look like this:
<code>
FROM python:3-alpine

# create folders
RUN mkdir -p /app/my_data/public && \
    mkdir -p /app/my_data/backup && \
    mkdir -p /app/my_data/private && \


WORKDIR /app
VOLUME /app/my_data

</code>

When you run this dockerfile everything under the //private/// directory will be shadowed from the volume meaning let's say you want to create another folder in Dockerfile, you add a line
  mkdir -p /app/my_data/private/extra && \

like so

<code>
FROM python:3-alpine

# create folders
RUN mkdir -p /app/my_data/public && \
    mkdir -p /app/my_data/backup && \
    mkdir -p /app/my_data/private && \
    mkdir -p /app/my_data/private/extra && \

WORKDIR /app
VOLUME /app/my_data

</code>
When the container builds now there won't be a directory called "extra" under the //private/// directory because the contents of the volume will be shown inside the container! There is no //extra// directory in the volume so it won't be in the container also.

To get around this you will have to create this extra directory on the host and change it's permissions accordingly or define a new volume for this extra directory only.

===== Error pulling image ... 404 Client Error=====

Might happen when trying to create the container without building the image first.

Full error:
  Error pulling image myimage:dev-999 - 404 Client Error for http+docker://localhost/v1.42/images/create?tag=dev-999&fromImage=myimage: Not Found ("pull access denied for myimage, repository does not exist or may require ''docker login'': denied: requested access to the resource is denied")

Check if the image is built and exist before creating container

  docker image ls

===== --chmod option requires BuildKit =====
Might happen when using ansible's docker_image module. It does not support the ''--chmod'' option in the ''ADD'' instruction. Just remove the chmod part and add a separat ''RUN'' command that runs ''chmod''

=====   stderr: 'failed to solve: failed to compute cache key: failed to calculate checksum ... no such file or directory' =====
Make sure that the files (app files, folders etc.) that the Dockerfile is suppose to copy are in the project root directory, where the Dockerfile file is (or docker compose file). Also check ''.dockerignore''

===== No internet access when building an image =====
In an alpine image this might manifest for example (run inside the virtualbox VM) :

<code>
fatal: [10.21.21.11]: FAILED! => changed=false                                                                                                                                                                     
  msg: 'Error building my/container - code: 2, message: The command ''/bin/sh -c apk add --no-cache libc6-compat git'' returned a non-zero code: 2, logs: [''Step 1/31 : FROM node:18.18.2-alpine AS deps''
, ''\n'', '' ---> 1646380c3156\n'', ''Step 2/31 : RUN apk add --no-cache libc6-compat git'', ''\n'', '' ---> Running in 567cff6b368f\n'', ''fetch https://dl-cdn.alpinelinux.org/alpine/v3.18/main/x86_64/APKINDEX.
tar.gz\n'', ''fetch https://dl-cdn.alpinelinux.org/alpine/v3.18/community/x86_64/APKINDEX.tar.gz\n'', ''\x1b[91mWARNING: fetching https://dl-cdn.alpinelinux.org/alpine/v3.18/main: temporary error (try again late
r)\n\x1b[0m'', ''\x1b[91mWARNING: fetching https://dl-cdn.alpinelinux.org/alpine/v3.18/community: temporary error (try again later)\n\x1b[0m'', ''\x1b[91mERROR: unable to select packages:\n  git (no such package
):\n    required by: world[git]\n  libc6-compat (no such package):\n    required by: world[libc6-compat]\n\x1b[0m'', ''Removing intermediate container 567cff6b368f\n'']'                                          
           
</code>

To test try adding ping in Dockerfile:
  
  ...
  RUN ping -c 4 8.8.8.8
  ...
If you get a packet loss that means the docker can't access the Internet. Try first restarting the docker e.g.

  systemctl restart docker
  
If this doesn't work you can build the image with ''--network host'' argument:

  docker build --network host -t test1 -f Dockerfile .

In ansible task this looks like:

<code>
...
- name: "Build the docker image"
  docker_image:
    name: "{{ full_app_name }}:{{ buildNo }}"
    repository: "{{ full_app_name }}:latest"
    state: present
    build:
      path: "{{ deploy_helper.new_release_path }}"
      network: host
    force: "{{ not (disable_image_force_build | default(false)) }}"
...
</code>
==== Tested on ====
  * Virtualbox VM Debian 11 Jessie (building inside of this VM)
  * ansible 2.9.27
  * docker 24.0.7


===== Containers lose access to internet suddenly =====
Try docker version downgrade. I had issues with version 5:28.0.4-1~debian.11~bullseye. Downgrading to 23.0.1 seems to have resolved the problem.

==== Tested on ====
  * Debian 11

====== See also ======

====== References ======
  * https://medium.com/@yaofei/docker-volume-what-i-learned-27134081d6d9
  * https://docs.ansible.com/ansible/latest/collections/community/docker/docker_image_module.html#notes