如何使Docker容器看到真正的用户IP？
问题
在nginx-proxy Docker容器（下面的更多信息）里面，我总是看到每个连接都有相同的IP地址： 172.18.0.1 （这是nginx-proxy网关）。 例如：
nginx.1 | www.my-site.tld 172.18.0.1 - - [28/Nov/2017:17:22:21 +0000] "GET /some/path HTTP/2.0" 200 46576 "https://www.my-site.tld/some/path" "Mozilla/5.0 (Linux; Android 4.4.2; PSP5507DUO Build/KVT49L) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 YaBrowser/17.10.0.446.00 Mobile Safari/537.36"
无论是从外部networking，本地机器还是从服务器（主机）本身发出请求，IP地址始终为172.18.0.1 。
我需要的是看到真正的客户端IP。
尝试修复
我search了很多，尝试了不同的解决scheme，但都没有为我工作。 我尝试了以下内容：
- 禁用firewalld（在这种情况下什么都不起作用，当然iptables是空的）。
- 使用
--userland-proxy=false（没有运气）; - 使用
--net=host（根本不是解决scheme，也不工作）。
相关链接：
- https://github.com/jwilder/nginx-proxy/issues/130
- https://github.com/jwilder/nginx-proxy/issues/133
那么，有什么想法？ 似乎是iptables路由问题或内部的Docker错误。 无论如何，它肯定与Docker有关，因为以下显示正确的远程用户IP：
[root@server]# nc -lv 12345 [user@remote-client]$ nc -vz MY.IP.ADDRESS.HERE 12345
很奇怪！
现在我要提供有关系统的完整信息，请耐心等待。 🙂
基本
我安装了Centos 7.4和Docker：
# cat /etc/centos-release CentOS Linux release 7.4.1708 (Core) # uname -r 3.10.0-693.5.2.el7.x86_64 # docker version Client: Version: 17.09.0-ce API version: 1.32 Go version: go1.8.3 Git commit: afdb6d4 Built: Tue Sep 26 22:41:23 2017 OS/Arch: linux/amd64 Server: Version: 17.09.0-ce API version: 1.32 (minimum version 1.12) Go version: go1.8.3 Git commit: afdb6d4 Built: Tue Sep 26 22:42:49 2017 OS/Arch: linux/amd64 Experimental: false
firewalld被用作防火墙。
networkingconfiguration
接口
有三个物理接口：
# ifconfig | grep -A 7 enp enp1s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet6 fe80::1ad6:c7ff:fe00:fc98 prefixlen 64 scopeid 0x20<link> ether 18:d6:c7:00:fc:98 txqueuelen 1000 (Ethernet) RX packets 11530516 bytes 2220554890 (2.0 GiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 24237462 bytes 31702254967 (29.5 GiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 enp2s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet6 fe80::1ad6:c7ff:fe06:ef36 prefixlen 64 scopeid 0x20<link> ether 18:d6:c7:06:ef:36 txqueuelen 1000 (Ethernet) RX packets 948513 bytes 143294797 (136.6 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 2371584 bytes 3207775040 (2.9 GiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 enp3s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 31.172.141.111 netmask 255.255.255.128 broadcast MY.EXTERNAL.IP.HERE inet6 fe80::127b:44ff:fe46:c2d1 prefixlen 64 scopeid 0x20<link> ether 10:7b:44:46:c2:d1 txqueuelen 1000 (Ethernet) RX packets 46772225 bytes 57416420859 (53.4 GiB) RX errors 0 dropped 1 overruns 0 frame 0 TX packets 28814037 bytes 10943786995 (10.1 GiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
和一个虚拟桥（ enp1s0和enp2s0 ）：
# ifconfig | grep -A 7 virbr virbr0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 192.168.0.1 netmask 255.255.255.0 broadcast 192.168.0.255 inet6 fe80::1ad6:c7ff:fe00:fc98 prefixlen 64 scopeid 0x20<link> ether 18:d6:c7:00:fc:98 txqueuelen 1000 (Ethernet) RX packets 11542053 bytes 2042945874 (1.9 GiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 26586017 bytes 34919742970 (32.5 GiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
防火墙区域
enp3s0接口连接到Internet（– --zone=public ），而virbr0连接到内部本地networking（ --zone=internal ）。
转发和伪装
IPv4转发已启用：
# cat /proc/sys/net/ipv4/ip_forward 1
并在public区域冒充configurationfirewalld：
# firewall-cmd --zone=public --list-all success
Dockerconfiguration
Docker有默认configuration。
联网
我只创build了一个networking：
# docker network create -d bridge nginx-proxy
这是它的configuration：
# docker network inspect nginx-proxy [ { "Name": "nginx-proxy", "Id": "192821446c9a5891fd1a7e240533cefb81ba0548033acb605eea805abec83505", "Created": "2017-11-24T14:25:10.687199435+02:00", "Scope": "local", "Driver": "bridge", "EnableIPv6": false, "IPAM": { "Driver": "default", "Options": {}, "Config": [ { "Subnet": "172.18.0.0/16", "Gateway": "172.18.0.1" } ] }, "Internal": false, "Attachable": false, "Ingress": false, "ConfigFrom": { "Network": "" }, "ConfigOnly": false, "Containers": { "1c563c33a45c9feabee4cb5ec3194464c66bfad1e233dedb797e269649067159": { "Name": "nginxproxy_nginx-proxy_1", "EndpointID": "683ceaced8eb416b485775734ca6828e9eced5ea6e3c7a76960c4eb807b521d9", "MacAddress": "02:42:ac:12:00:02", "IPv4Address": "172.18.0.2/16", "IPv6Address": "" } }, "Options": {}, "Labels": {} } ]
图像和容器
我正在使用jwilder / nginx-proxy映像。 这是我docker-compose.yml ：
version: '3' services: nginx-proxy: image: jwilder/nginx-proxy:alpine labels: - 'com.github.jrcs.letsencrypt_nginx_proxy_companion.nginx_proxy=true' volumes: - /var/run/docker.sock:/tmp/docker.sock:ro ports: - '80:80' - '443:443' networks: nginx-proxy: networks: nginx-proxy: depends_on: - nginx-proxy networks: nginx-proxy: external: name: nginx-proxy
如您所见， nginx-proxy容器在所有接口上监听端口80和443 。
# docker ps | grep nginx-proxy 1c563c33a45c jwilder/nginx-proxy:alpine "/app/docker-entry..." 8 hours ago Up 8 hours 0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp nginxproxy_nginx-proxy_1
iptables的
与容器相关的规则
这里是与容器相关的iptables规则：
# iptables-save | grep -e br-192821446c9a -e 172.18.0 -e '\*' *mangle *security *raw *filter -A FORWARD -o br-192821446c9a -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT -A FORWARD -o br-192821446c9a -j DOCKER -A FORWARD -i br-192821446c9a ! -o br-192821446c9a -j ACCEPT -A FORWARD -i br-192821446c9a -o br-192821446c9a -j ACCEPT -A DOCKER -d 172.18.0.2/32 ! -i br-192821446c9a -o br-192821446c9a -p tcp -m tcp --dport 443 -j ACCEPT -A DOCKER -d 172.18.0.2/32 ! -i br-192821446c9a -o br-192821446c9a -p tcp -m tcp --dport 80 -j ACCEPT *nat -A POSTROUTING -s 172.18.0.0/16 ! -o br-192821446c9a -j MASQUERADE -A POSTROUTING -s 172.18.0.2/32 -d 172.18.0.2/32 -p tcp -m tcp --dport 443 -j MASQUERADE -A POSTROUTING -s 172.18.0.2/32 -d 172.18.0.2/32 -p tcp -m tcp --dport 80 -j MASQUERADE -A DOCKER -i br-192821446c9a -j RETURN -A DOCKER ! -i br-192821446c9a -p tcp -m tcp --dport 443 -j DNAT --to-destination 172.18.0.2:443 -A DOCKER ! -i br-192821446c9a -p tcp -m tcp --dport 80 -j DNAT --to-destination 172.18.0.2:80
完整的iptables规则列表
这里是iptables-save的完整输出：
# iptables-save # Generated by iptables-save v1.4.21 on Tue Nov 28 19:16:31 2017 *mangle :PREROUTING ACCEPT [9063259:7480026035] :INPUT ACCEPT [250719:30179859] :FORWARD ACCEPT [8795875:7445096680] :OUTPUT ACCEPT [1078071:1430559812] :POSTROUTING ACCEPT [9842972:8867742437] :FORWARD_direct - [0:0] :INPUT_direct - [0:0] :OUTPUT_direct - [0:0] :POSTROUTING_direct - [0:0] :PREROUTING_ZONES - [0:0] :PREROUTING_ZONES_SOURCE - [0:0] :PREROUTING_direct - [0:0] :PRE_internal - [0:0] :PRE_internal_allow - [0:0] :PRE_internal_deny - [0:0] :PRE_internal_log - [0:0] :PRE_public - [0:0] :PRE_public_allow - [0:0] :PRE_public_deny - [0:0] :PRE_public_log - [0:0] -A PREROUTING -j PREROUTING_direct -A PREROUTING -j PREROUTING_ZONES_SOURCE -A PREROUTING -j PREROUTING_ZONES -A INPUT -j INPUT_direct -A FORWARD -j FORWARD_direct -A OUTPUT -j OUTPUT_direct -A POSTROUTING -j POSTROUTING_direct -A PREROUTING_ZONES -i enp3s0 -g PRE_public -A PREROUTING_ZONES -i virbr0 -g PRE_internal -A PREROUTING_ZONES -g PRE_public -A PRE_internal -j PRE_internal_log -A PRE_internal -j PRE_internal_deny -A PRE_internal -j PRE_internal_allow -A PRE_public -j PRE_public_log -A PRE_public -j PRE_public_deny -A PRE_public -j PRE_public_allow COMMIT # Completed on Tue Nov 28 19:16:31 2017 # Generated by iptables-save v1.4.21 on Tue Nov 28 19:16:31 2017 *security :INPUT ACCEPT [207361:25027156] :FORWARD ACCEPT [8764800:7437165960] :OUTPUT ACCEPT [1078077:1430561824] :FORWARD_direct - [0:0] :INPUT_direct - [0:0] :OUTPUT_direct - [0:0] -A INPUT -j INPUT_direct -A FORWARD -j FORWARD_direct -A OUTPUT -j OUTPUT_direct COMMIT # Completed on Tue Nov 28 19:16:31 2017 # Generated by iptables-save v1.4.21 on Tue Nov 28 19:16:31 2017 *raw :PREROUTING ACCEPT [9063267:7480026451] :OUTPUT ACCEPT [1078080:1430562828] :OUTPUT_direct - [0:0] :PREROUTING_ZONES - [0:0] :PREROUTING_ZONES_SOURCE - [0:0] :PREROUTING_direct - [0:0] :PRE_internal - [0:0] :PRE_internal_allow - [0:0] :PRE_internal_deny - [0:0] :PRE_internal_log - [0:0] :PRE_public - [0:0] :PRE_public_allow - [0:0] :PRE_public_deny - [0:0] :PRE_public_log - [0:0] -A PREROUTING -j PREROUTING_direct -A PREROUTING -j PREROUTING_ZONES_SOURCE -A PREROUTING -j PREROUTING_ZONES -A OUTPUT -j OUTPUT_direct -A PREROUTING_ZONES -i enp3s0 -g PRE_public -A PREROUTING_ZONES -i virbr0 -g PRE_internal -A PREROUTING_ZONES -g PRE_public -A PRE_internal -j PRE_internal_log -A PRE_internal -j PRE_internal_deny -A PRE_internal -j PRE_internal_allow -A PRE_public -j PRE_public_log -A PRE_public -j PRE_public_deny -A PRE_public -j PRE_public_allow COMMIT # Completed on Tue Nov 28 19:16:31 2017 # Generated by iptables-save v1.4.21 on Tue Nov 28 19:16:31 2017 *filter :INPUT ACCEPT [0:0] :FORWARD ACCEPT [0:0] :OUTPUT ACCEPT [1078082:1430564354] :DOCKER - [0:0] :DOCKER-ISOLATION - [0:0] :FORWARD_IN_ZONES - [0:0] :FORWARD_IN_ZONES_SOURCE - [0:0] :FORWARD_OUT_ZONES - [0:0] :FORWARD_OUT_ZONES_SOURCE - [0:0] :FORWARD_direct - [0:0] :FWDI_internal - [0:0] :FWDI_internal_allow - [0:0] :FWDI_internal_deny - [0:0] :FWDI_internal_log - [0:0] :FWDI_public - [0:0] :FWDI_public_allow - [0:0] :FWDI_public_deny - [0:0] :FWDI_public_log - [0:0] :FWDO_internal - [0:0] :FWDO_internal_allow - [0:0] :FWDO_internal_deny - [0:0] :FWDO_internal_log - [0:0] :FWDO_public - [0:0] :FWDO_public_allow - [0:0] :FWDO_public_deny - [0:0] :FWDO_public_log - [0:0] :INPUT_ZONES - [0:0] :INPUT_ZONES_SOURCE - [0:0] :INPUT_direct - [0:0] :IN_internal - [0:0] :IN_internal_allow - [0:0] :IN_internal_deny - [0:0] :IN_internal_log - [0:0] :IN_public - [0:0] :IN_public_allow - [0:0] :IN_public_deny - [0:0] :IN_public_log - [0:0] :OUTPUT_direct - [0:0] -A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT -A INPUT -i lo -j ACCEPT -A INPUT -j INPUT_direct -A INPUT -j INPUT_ZONES_SOURCE -A INPUT -j INPUT_ZONES -A INPUT -m conntrack --ctstate INVALID -j DROP -A INPUT -j REJECT --reject-with icmp-host-prohibited -A FORWARD -j DOCKER-ISOLATION -A FORWARD -o br-828c38e3582b -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT -A FORWARD -o br-828c38e3582b -j DOCKER -A FORWARD -i br-828c38e3582b ! -o br-828c38e3582b -j ACCEPT -A FORWARD -i br-828c38e3582b -o br-828c38e3582b -j ACCEPT -A FORWARD -o br-7cbbdaf3d8fe -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT -A FORWARD -o br-7cbbdaf3d8fe -j DOCKER -A FORWARD -i br-7cbbdaf3d8fe ! -o br-7cbbdaf3d8fe -j ACCEPT -A FORWARD -i br-7cbbdaf3d8fe -o br-7cbbdaf3d8fe -j ACCEPT -A FORWARD -o br-92f8d769de0b -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT -A FORWARD -o br-92f8d769de0b -j DOCKER -A FORWARD -i br-92f8d769de0b ! -o br-92f8d769de0b -j ACCEPT -A FORWARD -i br-92f8d769de0b -o br-92f8d769de0b -j ACCEPT -A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT -A FORWARD -o docker0 -j DOCKER -A FORWARD -i docker0 ! -o docker0 -j ACCEPT -A FORWARD -i docker0 -o docker0 -j ACCEPT -A FORWARD -o br-6a38b645b1c7 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT -A FORWARD -o br-6a38b645b1c7 -j DOCKER -A FORWARD -i br-6a38b645b1c7 ! -o br-6a38b645b1c7 -j ACCEPT -A FORWARD -i br-6a38b645b1c7 -o br-6a38b645b1c7 -j ACCEPT -A FORWARD -o br-2d90be830c58 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT -A FORWARD -o br-2d90be830c58 -j DOCKER -A FORWARD -i br-2d90be830c58 ! -o br-2d90be830c58 -j ACCEPT -A FORWARD -i br-2d90be830c58 -o br-2d90be830c58 -j ACCEPT -A FORWARD -o br-192821446c9a -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT -A FORWARD -o br-192821446c9a -j DOCKER -A FORWARD -i br-192821446c9a ! -o br-192821446c9a -j ACCEPT -A FORWARD -i br-192821446c9a -o br-192821446c9a -j ACCEPT -A FORWARD -o br-b61364bf1724 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT -A FORWARD -o br-b61364bf1724 -j DOCKER -A FORWARD -i br-b61364bf1724 ! -o br-b61364bf1724 -j ACCEPT -A FORWARD -i br-b61364bf1724 -o br-b61364bf1724 -j ACCEPT -A FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT -A FORWARD -i lo -j ACCEPT -A FORWARD -j FORWARD_direct -A FORWARD -j FORWARD_IN_ZONES_SOURCE -A FORWARD -j FORWARD_IN_ZONES -A FORWARD -j FORWARD_OUT_ZONES_SOURCE -A FORWARD -j FORWARD_OUT_ZONES -A FORWARD -m conntrack --ctstate INVALID -j DROP -A FORWARD -j REJECT --reject-with icmp-host-prohibited -A OUTPUT -j OUTPUT_direct -A DOCKER -d 172.18.0.2/32 ! -i br-192821446c9a -o br-192821446c9a -p tcp -m tcp --dport 443 -j ACCEPT -A DOCKER -d 172.18.0.2/32 ! -i br-192821446c9a -o br-192821446c9a -p tcp -m tcp --dport 80 -j ACCEPT -A DOCKER -d 172.20.0.2/32 ! -i br-2d90be830c58 -o br-2d90be830c58 -p tcp -m tcp --dport 1194 -j ACCEPT -A DOCKER-ISOLATION -j RETURN -A FORWARD_IN_ZONES -i enp3s0 -g FWDI_public -A FORWARD_IN_ZONES -i virbr0 -g FWDI_internal -A FORWARD_IN_ZONES -g FWDI_public -A FORWARD_OUT_ZONES -o enp3s0 -g FWDO_public -A FORWARD_OUT_ZONES -o virbr0 -g FWDO_internal -A FORWARD_OUT_ZONES -g FWDO_public -A FWDI_internal -j FWDI_internal_log -A FWDI_internal -j FWDI_internal_deny -A FWDI_internal -j FWDI_internal_allow -A FWDI_internal -p icmp -j ACCEPT -A FWDI_public -j FWDI_public_log -A FWDI_public -j FWDI_public_deny -A FWDI_public -j FWDI_public_allow -A FWDI_public -p icmp -j ACCEPT -A FWDO_internal -j FWDO_internal_log -A FWDO_internal -j FWDO_internal_deny -A FWDO_internal -j FWDO_internal_allow -A FWDO_public -j FWDO_public_log -A FWDO_public -j FWDO_public_deny -A FWDO_public -j FWDO_public_allow -A FWDO_public_allow -m conntrack --ctstate NEW -j ACCEPT -A INPUT_ZONES -i enp3s0 -g IN_public -A INPUT_ZONES -i virbr0 -g IN_internal -A INPUT_ZONES -g IN_public -A IN_internal -j IN_internal_log -A IN_internal -j IN_internal_deny -A IN_internal -j IN_internal_allow -A IN_internal -p icmp -j ACCEPT -A IN_internal_allow -p tcp -m tcp --dport 22 -m conntrack --ctstate NEW -j ACCEPT -A IN_internal_allow -d 224.0.0.251/32 -p udp -m udp --dport 5353 -m conntrack --ctstate NEW -j ACCEPT -A IN_internal_allow -p udp -m udp --dport 137 -m conntrack --ctstate NEW -j ACCEPT -A IN_internal_allow -p udp -m udp --dport 138 -m conntrack --ctstate NEW -j ACCEPT -A IN_internal_allow -p tcp -m tcp --dport 2049 -m conntrack --ctstate NEW -j ACCEPT -A IN_internal_allow -p tcp -m tcp --dport 20048 -m conntrack --ctstate NEW -j ACCEPT -A IN_internal_allow -p udp -m udp --dport 20048 -m conntrack --ctstate NEW -j ACCEPT -A IN_internal_allow -p tcp -m tcp --dport 111 -m conntrack --ctstate NEW -j ACCEPT -A IN_internal_allow -p udp -m udp --dport 111 -m conntrack --ctstate NEW -j ACCEPT -A IN_internal_allow -p tcp -m tcp --dport 139 -m conntrack --ctstate NEW -j ACCEPT -A IN_internal_allow -p tcp -m tcp --dport 445 -m conntrack --ctstate NEW -j ACCEPT -A IN_internal_allow -p tcp -m tcp --dport 58846 -m conntrack --ctstate NEW -j ACCEPT -A IN_internal_allow -p tcp -m tcp --dport 8112 -m conntrack --ctstate NEW -j ACCEPT -A IN_internal_allow -p tcp -m tcp --dport 53 -m conntrack --ctstate NEW -j ACCEPT -A IN_internal_allow -p udp -m udp --dport 53 -m conntrack --ctstate NEW -j ACCEPT -A IN_public -j IN_public_log -A IN_public -j IN_public_deny -A IN_public -j IN_public_allow -A IN_public -p icmp -j ACCEPT -A IN_public_allow -p tcp -m tcp --dport 22 -m conntrack --ctstate NEW -j ACCEPT -A IN_public_allow -p udp -m udp --dport 67 -m conntrack --ctstate NEW -j ACCEPT -A IN_public_allow -p tcp -m tcp --dport 80 -m conntrack --ctstate NEW -j ACCEPT -A IN_public_allow -p tcp -m tcp --dport 443 -m conntrack --ctstate NEW -j ACCEPT -A IN_public_allow -p udp -m udp --dport 1194 -m conntrack --ctstate NEW -j ACCEPT -A IN_public_allow -p tcp -m tcp --dport 1195 -m conntrack --ctstate NEW -j ACCEPT COMMIT # Completed on Tue Nov 28 19:16:31 2017 # Generated by iptables-save v1.4.21 on Tue Nov 28 19:16:31 2017 *nat :PREROUTING ACCEPT [118631:13941821] :INPUT ACCEPT [6147:464301] :OUTPUT ACCEPT [18836:1989867] :POSTROUTING ACCEPT [2593:197858] :DOCKER - [0:0] :OUTPUT_direct - [0:0] :POSTROUTING_ZONES - [0:0] :POSTROUTING_ZONES_SOURCE - [0:0] :POSTROUTING_direct - [0:0] :POST_internal - [0:0] :POST_internal_allow - [0:0] :POST_internal_deny - [0:0] :POST_internal_log - [0:0] :POST_public - [0:0] :POST_public_allow - [0:0] :POST_public_deny - [0:0] :POST_public_log - [0:0] :PREROUTING_ZONES - [0:0] :PREROUTING_ZONES_SOURCE - [0:0] :PREROUTING_direct - [0:0] :PRE_internal - [0:0] :PRE_internal_allow - [0:0] :PRE_internal_deny - [0:0] :PRE_internal_log - [0:0] :PRE_public - [0:0] :PRE_public_allow - [0:0] :PRE_public_deny - [0:0] :PRE_public_log - [0:0] -A PREROUTING -j PREROUTING_direct -A PREROUTING -j PREROUTING_ZONES_SOURCE -A PREROUTING -j PREROUTING_ZONES -A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER -A OUTPUT -j OUTPUT_direct -A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j DOCKER -A POSTROUTING -s 172.101.0.0/24 ! -o br-828c38e3582b -j MASQUERADE -A POSTROUTING -s 172.102.0.0/24 ! -o br-7cbbdaf3d8fe -j MASQUERADE -A POSTROUTING -s 172.101.0.0/24 ! -o br-92f8d769de0b -j MASQUERADE -A POSTROUTING -s 172.17.0.0/16 ! -o docker0 -j MASQUERADE -A POSTROUTING -s 172.102.0.0/24 ! -o br-6a38b645b1c7 -j MASQUERADE -A POSTROUTING -s 172.20.0.0/16 ! -o br-2d90be830c58 -j MASQUERADE -A POSTROUTING -s 172.18.0.0/16 ! -o br-192821446c9a -j MASQUERADE -A POSTROUTING -s 172.19.0.0/16 ! -o br-b61364bf1724 -j MASQUERADE -A POSTROUTING -j POSTROUTING_direct -A POSTROUTING -j POSTROUTING_ZONES_SOURCE -A POSTROUTING -j POSTROUTING_ZONES -A POSTROUTING -s 172.18.0.2/32 -d 172.18.0.2/32 -p tcp -m tcp --dport 443 -j MASQUERADE -A POSTROUTING -s 172.18.0.2/32 -d 172.18.0.2/32 -p tcp -m tcp --dport 80 -j MASQUERADE -A POSTROUTING -s 172.20.0.2/32 -d 172.20.0.2/32 -p tcp -m tcp --dport 1194 -j MASQUERADE -A DOCKER -i br-828c38e3582b -j RETURN -A DOCKER -i br-7cbbdaf3d8fe -j RETURN -A DOCKER -i br-92f8d769de0b -j RETURN -A DOCKER -i docker0 -j RETURN -A DOCKER -i br-6a38b645b1c7 -j RETURN -A DOCKER -i br-2d90be830c58 -j RETURN -A DOCKER -i br-192821446c9a -j RETURN -A DOCKER -i br-b61364bf1724 -j RETURN -A DOCKER ! -i br-192821446c9a -p tcp -m tcp --dport 443 -j DNAT --to-destination 172.18.0.2:443 -A DOCKER ! -i br-192821446c9a -p tcp -m tcp --dport 80 -j DNAT --to-destination 172.18.0.2:80 -A DOCKER ! -i br-2d90be830c58 -p tcp -m tcp --dport 1195 -j DNAT --to-destination 172.20.0.2:1194 -A POSTROUTING_ZONES -o enp3s0 -g POST_public -A POSTROUTING_ZONES -o virbr0 -g POST_internal -A POSTROUTING_ZONES -g POST_public -A POST_internal -j POST_internal_log -A POST_internal -j POST_internal_deny -A POST_internal -j POST_internal_allow -A POST_public -j POST_public_log -A POST_public -j POST_public_deny -A POST_public -j POST_public_allow -A POST_public_allow ! -o lo -j MASQUERADE -A PREROUTING_ZONES -i enp3s0 -g PRE_public -A PREROUTING_ZONES -i virbr0 -g PRE_internal -A PREROUTING_ZONES -g PRE_public -A PRE_internal -j PRE_internal_log -A PRE_internal -j PRE_internal_deny -A PRE_internal -j PRE_internal_allow -A PRE_public -j PRE_public_log -A PRE_public -j PRE_public_deny -A PRE_public -j PRE_public_allow COMMIT # Completed on Tue Nov 28 19:16:31 2017
最后，回答我自己的问题。
经过一番研究，我发现这是造成Docker无法正确识别客户端地址的IP伪装。
原因
在我的情况下，firewalld被configuration为与本地networking中的其他机器共享互联网连接。 这是通过将外部接口（ enp3s0 ）设置为public区域，将本地接口（ enp1s0 ， enp2s0 ）设置为internal enp2s0 ，以及为传出包启用IP伪装来实现的。
就是这样。 伪装也适用于从publicnetworking到dockernetworking的软件包，就好像软件包正在（另一个）外部接口一样。
解
据我所知，所有iptables和Docker的东西，有一个可以接受的解决方法：将所有由Docker创build的虚拟networking接口添加到internal区域，所以伪装规则不适用于去往相应networking的包。 例如：
# firewall-cmd --permanent -zone=internal --change-interface=br-192821446c9a
要使解决scheme持久化，需要在/etc/sysconfig/network-scripts目录中创build相应的脚本。 例如，对于br-192821446c9a接口，应该创build一个文件/etc/sysconfig/network-scripts/ifcfg-br-192821446c9a ，其内容如下：
DEVICE=br-192821446c9a TYPE=Bridge BOOTPROTO=none IPADDR=172.18.0.1 PREFIX=16 DEFROUTE=yes IPV4_DNS_PRIORITY=100 NAME=br-192821446c9a ONBOOT=no ZONE=internal
希望这个答案会节省一些人的时间。