Docker network internet access

Selfhosted@lemmy.world – 18 points – 4 months ago

Hey all! I'm having an issue that's probably simple but I can't seem to work it out.

For some history (just in case it matters): I have a simple server running docker and all services being defined in docker-compose files. Probably doesn't matter, but I've switched between a few management UIs (Portainer, Dokemon, currently Dockge). Initially, I set everything up in Portainer (including the main network) and migrated everything over to Dockge. I was using Traefik labels but was getting a bit annoying since I tend to tinker on a tablet. I wanted something a bit more UI-focused so I switched to NPM.

Now I'm going through all of my compose files and cleaning up a bunch of things like Traefik labels, homepage labels, etc... but I'm also trying to clean up my Docker network situation.

My containers are all on the same network, and I want to slice things up a little better, e.g. I have the Cloudflared container and want to be selective about what containers it has access to network-wise.

So, the meat of my issue is that my original network (call it old_main) seems to be the only one that can access the internet outbound. I added a new network called cloudflared and put just my Cloudflared container and another service on it and I get the 1033 ARGO Tunnel error when accessing the service and Cloudflare says the tunnel is down. Same thing for other containers I try to move from old_main, SearXNG can't connect, Audiobookshelf can't search for author info, etc... I can connect to these services but they can't reach anything on the web.

I have my docker daemon.json set to use my Pi-hole for DNS and I only see my services like audiobookshelf.old_main coming through. I also see the IP address of the old_main gateway coming into Pi-hole as docker-host. My goal is to add all of my services to new, more-specific networks then remove old_main but I don't want to drop the only network that seems to be able to communicate with the web until I have another that can.

I'm not sure what else to look for, any suggestions? Let me know if you need more info.

Any chance you've defined the new networks as "internal"? (using docker network create --internal on the CLI or internal: true in your docker-compose.yaml).

Because the symptoms you're describing (no connectivity to stuff outside the new network, including the wider Internet) sound exactly like you did, but didn't realize what that option does...

It sounds like your issue might be related to how your Docker networks are configured for DNS and internet access. Try these:

Check Network Configuration: Ensure your new networks are correctly configured to allow internet access. Docker networks should be able to route traffic to the internet by default unless specified otherwise.
DNS Configuration: Since you're using Pi-hole for DNS, make sure the new networks are properly configured to use Pi-hole as their DNS server.
Inspect Network Settings: Compare the settings of old_main with the new networks. Use the following command to inspect the network configuration:
```
docker network inspect old_main
docker network inspect cloudflared
```
Pay attention to the gateway, subnet, and any custom DNS settings.
Check Docker Daemon Configuration: Verify that your daemon.json file is correctly set up to use Pi-hole for DNS. It should look something like this:
```
{
  "dns": [""]
}
```
Verify Container Configuration: Ensure that your containers are correctly configured to use the new network. This can be specified in your docker-compose files like this:
```
version: '3.7'
services:
  cloudflared:
    image: cloudflare/cloudflared
    networks:
      - cloudflared

networks:
  cloudflared:
    external: true
```
Check Firewall Rules: Ensure there are no firewall rules on your host or network equipment that might be blocking traffic from the new networks.
Test Connectivity: Run a simple connectivity test from within a container on the new network to check internet access:
```
docker run --rm -it --network cloudflared alpine ping -c 4 google.com
```
If this fails, the issue is likely with network configuration rather than the containers themselves.
Docker Network Restart: Sometimes, Docker networks need to be restarted to apply changes correctly. Try removing and recreating the problematic networks:
```
docker network rm cloudflared
docker network create cloudflared
```

If none of the above steps resolve the issue, there might be a deeper configuration problem. At this point, it might be helpful to see the exact configuration of your docker-compose files and the output of the network inspection commands.

This sounds like a chat gpt answer.

Definitely.

Try another dns provider. Put dns: 1.1.1.1 or something in your compose file

What does "old network" and "new network" mean? What are they, LAN setup? Docker setup? Describe them better (netmasks, routing etc.)

I'm referring to docker bridge networks. old_main is in the 10.2.1.0/24 subnet and i'm trying to move everything to a new bridge network on a subnet of 10.0.0.0/24. sorry, i'm not exactly sure what other info would be useful