Hey all! I’m having an issue that’s probably simple but I can’t seem to work it out.
For some history (just in case it matters): I have a simple server running docker and all services being defined in docker-compose files. Probably doesn’t matter, but I’ve switched between a few management UIs (Portainer, Dokemon, currently Dockge). Initially, I set everything up in Portainer (including the main network) and migrated everything over to Dockge. I was using Traefik labels but was getting a bit annoying since I tend to tinker on a tablet. I wanted something a bit more UI-focused so I switched to NPM.
Now I’m going through all of my compose files and cleaning up a bunch of things like Traefik labels, homepage labels, etc… but I’m also trying to clean up my Docker network situation.
My containers are all on the same network, and I want to slice things up a little better, e.g. I have the Cloudflared container and want to be selective about what containers it has access to network-wise.
So, the meat of my issue is that my original network (call it old_main
) seems to be the only one that can access the internet outbound. I added a new network called cloudflared
and put just my Cloudflared container and another service on it and I get the 1033 ARGO Tunnel
error when accessing the service and Cloudflare says the tunnel is down. Same thing for other containers I try to move from old_main
, SearXNG can’t connect, Audiobookshelf can’t search for author info, etc… I can connect to these services but they can’t reach anything on the web.
I have my docker daemon.json set to use my Pi-hole for DNS and I only see my services like audiobookshelf.old_main
coming through. I also see the IP address of the old_main
gateway coming into Pi-hole as docker-host
. My goal is to add all of my services to new, more-specific networks then remove old_main
but I don’t want to drop the only network that seems to be able to communicate with the web until I have another that can.
I’m not sure what else to look for, any suggestions? Let me know if you need more info.
Try another dns provider. Put dns: 1.1.1.1 or something in your compose file
Any chance you’ve defined the new networks as “internal”? (using
docker network create --internal
on the CLI orinternal: true
in your docker-compose.yaml).Because the symptoms you’re describing (no connectivity to stuff outside the new network, including the wider Internet) sound exactly like you did, but didn’t realize what that option does…
It sounds like your issue might be related to how your Docker networks are configured for DNS and internet access. Try these:
-
Check Network Configuration: Ensure your new networks are correctly configured to allow internet access. Docker networks should be able to route traffic to the internet by default unless specified otherwise.
-
DNS Configuration: Since you’re using Pi-hole for DNS, make sure the new networks are properly configured to use Pi-hole as their DNS server.
-
Inspect Network Settings: Compare the settings of
old_main
with the new networks. Use the following command to inspect the network configuration:docker network inspect old_main docker network inspect cloudflared
Pay attention to the gateway, subnet, and any custom DNS settings.
-
Check Docker Daemon Configuration: Verify that your
daemon.json
file is correctly set up to use Pi-hole for DNS. It should look something like this:{ "dns": ["<Pi-hole IP>"] }
-
Verify Container Configuration: Ensure that your containers are correctly configured to use the new network. This can be specified in your
docker-compose
files like this:version: '3.7' services: cloudflared: image: cloudflare/cloudflared networks: - cloudflared networks: cloudflared: external: true
-
Check Firewall Rules: Ensure there are no firewall rules on your host or network equipment that might be blocking traffic from the new networks.
-
Test Connectivity: Run a simple connectivity test from within a container on the new network to check internet access:
docker run --rm -it --network cloudflared alpine ping -c 4 google.com
If this fails, the issue is likely with network configuration rather than the containers themselves.
-
Docker Network Restart: Sometimes, Docker networks need to be restarted to apply changes correctly. Try removing and recreating the problematic networks:
docker network rm cloudflared docker network create cloudflared
If none of the above steps resolve the issue, there might be a deeper configuration problem. At this point, it might be helpful to see the exact configuration of your
docker-compose
files and the output of the network inspection commands.This sounds like a chat gpt answer.
-