User Tools

Site Tools


wiki:docker_troubleshooting

Docker troubleshooting

DOCKER-ISOLATION-STAGE-1

An unexpected docker error occurred: 500 Server Error: Internal Server Error ("unable to insert jump to DOCKER-ISOLATION-STAGE-1 rule in FORWARD chain:  (iptables failed: iptables --wait -I FORWARD -j DOCKER-ISOLATION-STAGE-1: iptables v1.8.7 (nf_tables): Chain 'DOCKER-ISOLATION-STAGE-1' does not exist
  Try `iptables -h' or 'iptables --help' for more information.
   (exit status 2))")
An exception occurred during task execution. To see the full traceback, use -vvv. The error was:  (exit status 2))")

Restart the docker service:

systemctl restart docker.service

Or create that chain in filter table

iptables -t filter -N DOCKER-ISOLATION-STAGE-1

This is supposed to be fixed in version 5.2.3.7

Tested on

  • Docker 20.10.12
  • Debian 11
  • shorewall 5.2.3.4

Volume shadowing

This is a tricky behaviour from Docker. Say you have a volume /app/my_data with some preexisting content which you mount in the container in the Dockerfile. This volume has some files there and folders

a.txt
b.txt
private/
private/c.txt
private/d.txt

Your Dockerfile might look like this:

FROM python:3-alpine

# create folders
RUN mkdir -p /app/my_data/public && \
    mkdir -p /app/my_data/backup && \
    mkdir -p /app/my_data/private && \


WORKDIR /app
VOLUME /app/my_data

When you run this dockerfile everything under the private/ directory will be shadowed from the volume meaning let's say you want to create another folder in Dockerfile, you add a line

mkdir -p /app/my_data/private/extra && \

like so

FROM python:3-alpine

# create folders
RUN mkdir -p /app/my_data/public && \
    mkdir -p /app/my_data/backup && \
    mkdir -p /app/my_data/private && \
    mkdir -p /app/my_data/private/extra && \

WORKDIR /app
VOLUME /app/my_data

When the container builds now there won't be a directory called “extra” under the private/ directory because the contents of the volume will be shown inside the container! There is no extra directory in the volume so it won't be in the container also.

To get around this you will have to create this extra directory on the host and change it's permissions accordingly or define a new volume for this extra directory only.

Error pulling image ... 404 Client Error

Might happen when trying to create the container without building the image first.

Full error:

Error pulling image myimage:dev-999 - 404 Client Error for http+docker://localhost/v1.42/images/create?tag=dev-999&fromImage=myimage: Not Found ("pull access denied for myimage, repository does not exist or may require ''docker login'': denied: requested access to the resource is denied")

Check if the image is built and exist before creating container

docker image ls

--chmod option requires BuildKit

Might happen when using ansible's docker_image module. It does not support the –chmod option in the ADD instruction. Just remove the chmod part and add a separat RUN command that runs chmod

stderr: 'failed to solve: failed to compute cache key: failed to calculate checksum ... no such file or directory'

Make sure that the files (app files, folders etc.) that the Dockerfile is suppose to copy are in the project root directory, where the Dockerfile file is (or docker compose file).

No internet access when building an image

In an alpine image this might manifest for example (run inside the virtualbox VM) :

fatal: [10.21.21.11]: FAILED! => changed=false                                                                                                                                                                     
  msg: 'Error building my/container - code: 2, message: The command ''/bin/sh -c apk add --no-cache libc6-compat git'' returned a non-zero code: 2, logs: [''Step 1/31 : FROM node:18.18.2-alpine AS deps''
, ''\n'', '' ---> 1646380c3156\n'', ''Step 2/31 : RUN apk add --no-cache libc6-compat git'', ''\n'', '' ---> Running in 567cff6b368f\n'', ''fetch https://dl-cdn.alpinelinux.org/alpine/v3.18/main/x86_64/APKINDEX.
tar.gz\n'', ''fetch https://dl-cdn.alpinelinux.org/alpine/v3.18/community/x86_64/APKINDEX.tar.gz\n'', ''\x1b[91mWARNING: fetching https://dl-cdn.alpinelinux.org/alpine/v3.18/main: temporary error (try again late
r)\n\x1b[0m'', ''\x1b[91mWARNING: fetching https://dl-cdn.alpinelinux.org/alpine/v3.18/community: temporary error (try again later)\n\x1b[0m'', ''\x1b[91mERROR: unable to select packages:\n  git (no such package
):\n    required by: world[git]\n  libc6-compat (no such package):\n    required by: world[libc6-compat]\n\x1b[0m'', ''Removing intermediate container 567cff6b368f\n'']'                                          
           

To test try adding ping in Dockerfile:

...
RUN ping -c 4 8.8.8.8
...

If you get a packet loss that means the docker can't access the Internet. Try first restarting the docker e.g.

systemctl restart docker

If this doesn't work you can build the image with –network host argument:

docker build --network host -t test1 -f Dockerfile .

In ansible task this looks like:

...
- name: "Build the docker image"
  docker_image:
    name: "{{ full_app_name }}:{{ buildNo }}"
    repository: "{{ full_app_name }}:latest"
    state: present
    build:
      path: "{{ deploy_helper.new_release_path }}"
      network: host
    force: "{{ not (disable_image_force_build | default(false)) }}"
...

Tested on

  • Virtualbox VM Debian 11 Jessie (building inside of this VM)
  • ansible 2.9.27
  • docker 24.0.7

See also

References

wiki/docker_troubleshooting.txt · Last modified: 2023/12/01 14:04 by antisa

Except where otherwise noted, content on this wiki is licensed under the following license: CC0 1.0 Universal
CC0 1.0 Universal Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki