奇怪的问题: docker 容器和宿主机物理网卡 IP 不通,求教 - V2EX
V2EX = way to explore
V2EX 是一个关于分享和探索的地方
现在注册
已注册用户请  登录
watara
V2EX    Docker

奇怪的问题: docker 容器和宿主机物理网卡 IP 不通,求教

  •  
  •   watara 2023-02-11 18:37:54 +08:00 1348 次点击
    这是一个创建于 974 天前的主题,其中的信息可能已经有所发展或是发生改变。

    前置说明

    宿主机系统:CentOS 7.2009
    Docker 版本:20.10.19
    防火墙: 停止并禁用了 firewalld ,并启用了 iptables ,
    物理网卡 IP: 公网 IP (下面用 1.1.1.1 替代)
    docker 网络:172.17.0.0/16 ,宿主机 IP 172.17.0.1 ,容器 IP:172.17.0.2
    其他补充说明:由于架构设计,哪怕在本机容器中,依然需要访问物理网卡的公网 IP 访问才行。

    现象

    1 、在容器中 ping 不通宿主机公网 IP1.1.1.1 ,相应的,业务端口也不通。
    2 、但在容器中可以 ping 通其他任意公网 IP ,包括和宿主机通网段的 1.1.1.2 等,端口访问也正常。
    3 、用 docker 网卡的 IP 进行在容器以及宿主机间互 ping ,都是通的,端口访问也正常。

    测试机复现以及一些想法

    说实话我没能成功复现,在测试环境中发现如果仅仅是容器和真实网卡的 IP 之间的互通而言,和 iptables 的规则关系不大(当然,刻意写规则去禁 ping 啥的那肯定是有影响的),清空了 iptables 规则,甚至停止了 iptables ,网络依然是通的,所以我想问题应该是出在 docker0 这个网卡上,但照着这个思路查了半天也查不出一个所以然,所以特发帖求教。

    一些配置或结果

    brctl show

    brctl show bridge name bridge id STP enabled interfaces docker0 8000.02423f9b3058 no veth29942ea veth5726f6f veth7eca3df 

    docker ntwork ls

    root# docker network ls NETWORK ID NAME DRIVER SCOPE df8dc4223152 bridge bridge local dc4beaa42e37 host host local b9230ceb5861 none null local 

    容器详情(截取了网络相关部分)

    docker inspect 7c395050d09b [ #省略了一些我认为无用的部分,不然太长了 "NetworkSettings": { "Bridge": "", "SandboxID": "801c42feac51b082e9947d86f38175d1c3b5bc6295385b6d06ce6f100be95ddf", "HairpinMode": false, "LinkLocalIPv6Address": "", "LinkLocalIPv6PrefixLen": 0, "Ports": {}, "SandboxKey": "/var/run/docker/netns/801c42feac51", "SecondaryIPAddresses": null, "SecondaryIPv6Addresses": null, "EndpointID": "8318ab5e93f6c4a02e788d300d4c31164c84a9eddc3a870db7bfb788d8e187c9", "Gateway": "172.17.0.1", "GlobalIPv6Address": "", "GlobalIPv6PrefixLen": 0, "IPAddress": "172.17.0.4", "IPPrefixLen": 16, "IPv6Gateway": "", "MacAddress": "02:42:ac:11:00:04", "Networks": { "bridge": { "IPAMConfig": null, "Links": null, "Aliases": null, "NetworkID": "df8dc4223152b8fbc3676a4a86aa3c4c0a9f5528a2c3fabc9e74dd0bec5e0b06", "EndpointID": "8318ab5e93f6c4a02e788d300d4c31164c84a9eddc3a870db7bfb788d8e187c9", "Gateway": "172.17.0.1", "IPAddress": "172.17.0.4", "IPPrefixLen": 16, "IPv6Gateway": "", "GlobalIPv6Address": "", "GlobalIPv6PrefixLen": 0, "MacAddress": "02:42:ac:11:00:04", "DriverOpts": null } } } } ] 

    iptables 配置

    # Generated by iptables-save v1.4.21 on Fri Feb 10 18:13:01 2023 *filter :INPUT ACCEPT [8077:11154340] :FORWARD ACCEPT [0:0] :OUTPUT ACCEPT [20891:12086812] :DOCKER - [0:0] :DOCKER-ISOLATION-STAGE-1 - [0:0] :DOCKER-ISOLATION-STAGE-2 - [0:0] :DOCKER-USER - [0:0] iptables -A INPUT -p tcp -m tcp --dport 443 -j ACCEPT iptables -A INPUT -p tcp -m tcp --dport 80 -j ACCEPT iptables -A INPUT -p tcp -m tcp --dport 22 -j ACCEPT iptables -A FORWARD -j DOCKER-USER iptables -A FORWARD -j DOCKER-ISOLATION-STAGE-1 iptables -A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT iptables -A FORWARD -o docker0 -j DOCKER iptables -A FORWARD -i docker0 ! -o docker0 -j ACCEPT iptables -A FORWARD -i docker0 -o docker0 -j ACCEPT iptables -A DOCKER -d 172.17.0.2/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 9090 -j ACCEPT iptables -A DOCKER -d 172.17.0.2/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 9000 -j ACCEPT iptables -A DOCKER-ISOLATION-STAGE-1 -i docker0 ! -o docker0 -j DOCKER-ISOLATION-STAGE-2 iptables -A DOCKER-ISOLATION-STAGE-1 -j RETURN iptables -A DOCKER-ISOLATION-STAGE-2 -o docker0 -j DROP iptables -A DOCKER-ISOLATION-STAGE-2 -j RETURN iptables -A DOCKER-USER -j RETURN COMMIT # Completed on Fri Feb 10 18:13:01 2023 # Generated by iptables-save v1.4.21 on Fri Feb 10 18:13:01 2023 *nat :PREROUTING ACCEPT [6168:356557] :INPUT ACCEPT [4862:272405] :OUTPUT ACCEPT [4222:245620] :POSTROUTING ACCEPT [4223:245672] :DOCKER - [0:0] iptables -A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER iptables -A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j DOCKER iptables -A POSTROUTING -s 172.17.0.0/16 ! -o docker0 -j MASQUERADE iptables -A POSTROUTING -s 172.17.0.2/32 -d 172.17.0.2/32 -p tcp -m tcp --dport 9090 -j MASQUERADE iptables -A POSTROUTING -s 172.17.0.2/32 -d 172.17.0.2/32 -p tcp -m tcp --dport 9000 -j MASQUERADE iptables -A DOCKER -i docker0 -j RETURN iptables -A DOCKER -d 127.0.0.1/32 ! -i docker0 -p tcp -m tcp --dport 9090 -j DNAT --to-destination 172.17.0.2:9090 iptables -A DOCKER -d 127.0.0.1/32 ! -i docker0 -p tcp -m tcp --dport 9000 -j DNAT --to-destination 172.17.0.2:9000 COMMIT # Completed on Fri Feb 10 18:13:01 2023 
    4 条回复    2023-02-14 15:46:03 +08:00
    julyclyde
        1
    julyclyde  
       2023-02-13 09:53:54 +08:00
    docker inspect 容器显示其所属 bridge 为
    建议你运行一下
    julyclyde
        2
    julyclyde  
       2023-02-13 09:54:05 +08:00
    docker network inspect df8dc422312
    看看情况?
    watara
        3
    watara  
    OP
       2023-02-13 11:48:32 +08:00
    @julyclyde #2 多谢帮忙出谋划策,这个我对比看了,好像没啥异常,由于后续重启了 docker ,所以资源 ID 变了,下面是结果:

    ```bash
    docker network inspect dc1dc15bd744
    [
    {
    "Name": "bridge",
    "Id": "dc1dc15bd7448b829dbde584f5b3d6aedbf8b6e14bb4b8b6fcc2dbb80b81ea3f",
    "Created": "2023-02-13T10:01:31.771962878+08:00",
    "Scope": "local",
    "Driver": "bridge",
    "EnableIPv6": false,
    "IPAM": {
    "Driver": "default",
    "Options": null,
    "Config": [
    {
    "Subnet": "172.17.0.0/16",
    "Gateway": "172.17.0.1"
    }
    ]
    },
    "Internal": false,
    "Attachable": false,
    "Ingress": false,
    "ConfigFrom": {
    "Network": ""
    },
    "ConfigOnly": false,
    "Containers": {
    "066344f76f69bee353fe2751383fe9fa7a8ae0815aeec89c837a4795724f48ab": {
    "Name": "apffd_dghg",
    "EndpointID": "8e30360569e438bb9943a8ad542909e8de6e78e66e8cf6f77ac9ea946d265867",
    "MacAddress": "02:42:ac:11:00:02",
    "IPv4Address": "172.17.0.2/16",
    "IPv6Address": ""
    },
    "bddf8b8719d44ff77fe95b33a264852a34795134050fb188913db04883407203": {
    "Name": "sleepy_neumann",
    "EndpointID": "f7ef3cfcb2afdbdea998d64eb750ed0f399e54f5e759869a40318641cf1ae793",
    "MacAddress": "02:42:ac:11:00:03",
    "IPv4Address": "172.17.0.3/16",
    "IPv6Address": ""
    }
    },
    "Options": {
    "com.docker.network.bridge.default_bridge": "true",
    "com.docker.network.bridge.enable_icc": "true",
    "com.docker.network.bridge.enable_ip_masquerade": "true",
    "com.docker.network.bridge.host_binding_ipv4": "0.0.0.0",
    "com.docker.network.bridge.name": "docker0",
    "com.docker.network.driver.mtu": "1500"
    },
    "Labels": {}
    }
    ]
    ```
    dengkj
        4
    dengkj  
       2023-02-14 15:46:03 +08:00
    应该是系统内核的网桥模块加载失败,升级内核可解决
    关于     帮助文档     自助推广系统     博客     API     FAQ     Solana     2854 人在线   最高记录 6679       Select Language
    创意工作者们的社区
    World is powered by solitude
    VERSION: 3.9.8.5 33ms UTC 13:38 PVG 21:38 LAX 06:38 JFK 09:38
    Do have faith in what you're doing.
    ubao snddm index pchome yahoo rakuten mypaper meadowduck bidyahoo youbao zxmzxm asda bnvcg cvbfg dfscv mmhjk xxddc yybgb zznbn ccubao uaitu acv GXCV ET GDG YH FG BCVB FJFH CBRE CBC GDG ET54 WRWR RWER WREW WRWER RWER SDG EW SF DSFSF fbbs ubao fhd dfg ewr dg df ewwr ewwr et ruyut utut dfg fgd gdfgt etg dfgt dfgd ert4 gd fgg wr 235 wer3 we vsdf sdf gdf ert xcv sdf rwer hfd dfg cvb rwf afb dfh jgh bmn lgh rty gfds cxv xcv xcs vdas fdf fgd cv sdf tert sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf shasha9178 shasha9178 shasha9178 shasha9178 shasha9178 liflif2 liflif2 liflif2 liflif2 liflif2 liblib3 liblib3 liblib3 liblib3 liblib3 zhazha444 zhazha444 zhazha444 zhazha444 zhazha444 dende5 dende denden denden2 denden21 fenfen9 fenf619 fen619 fenfe9 fe619 sdf sdf sdf sdf sdf zhazh90 zhazh0 zhaa50 zha90 zh590 zho zhoz zhozh zhozho zhozho2 lislis lls95 lili95 lils5 liss9 sdf0ty987 sdft876 sdft9876 sdf09876 sd0t9876 sdf0ty98 sdf0976 sdf0ty986 sdf0ty96 sdf0t76 sdf0876 df0ty98 sf0t876 sd0ty76 sdy76 sdf76 sdf0t76 sdf0ty9 sdf0ty98 sdf0ty987 sdf0ty98 sdf6676 sdf876 sd876 sd876 sdf6 sdf6 sdf9876 sdf0t sdf06 sdf0ty9776 sdf0ty9776 sdf0ty76 sdf8876 sdf0t sd6 sdf06 s688876 sd688 sdf86