Why can't my VPC-native pods connect to my Droplets?
Overview
VPC-native DOKS clusters communicate with outside resources differently than clusters using legacy networking.
VPC-native networking allows for communication directly from Pods to Droplets and other resources within VPC networks, but for this to work the Droplets need to have the necessary routes enabled.
If your pods in VPC-native clusters are unable to connect to Droplets created on or before 2 October 2024, but can connect with hostNetwork: true, you may need to enable route-injection for the Droplets
Read on to understand why.
Understanding VPC-Native vs Legacy Cluster Networking
VPC-native DOKS clusters work differently to DOKS clusters on legacy networking. One of the key differences is that outbound connections to VPC resources do not have the source IP substituted for the Node IP.
In practice, this means that outside resources see the Pod IP, rather than the Node IP. This also means that outside connections can be made directly to resources inside a cluster, such as making a connection from a Droplet to a Pod.
To provide a deeper understanding of the differences between VPC-native DOKS clusters and legacy (non-VPC-native) clusters, take the following example:
Droplets
| Droplet Name | Route Injection | Droplet IP |
|---|---|---|
native-vpc-droplet |
Yes | 10.2.0.2 |
legacy-vpc-droplet |
No | 10.2.0.4 |
The example VPC contains two Droplets, one with route-injection configured, and one without.
The routing table of the native-vpc-droplet is as follows. The highlighted lines are the routes configured by route-injection.
root@native-vpc-droplet:~# ip route
default via <GATEWAY-IP> dev eth0 proto static
10.0.0.0/8 via 10.2.0.1 dev eth1 metric 101 mtu 1500
10.2.0.0/24 dev eth1 proto kernel scope link src 10.2.0.2
10.16.0.0/16 dev eth0 proto kernel scope link src 10.16.0.5
167.99.80.0/20 dev eth0 proto kernel scope link src <GATEWAY-IP>
172.16.0.0/12 via 10.2.0.1 dev eth1 metric 101 mtu 1500
192.168.0.0/16 via 10.2.0.1 dev eth1 metric 101 mtu 1500
The legacy-vpc-droplet does not have these routes.
DOKS Clusters
In this example we also have two clusters in the same VPC, one configured with VPC-native networking, and one without.
| Cluster Name | VPC-Native | Pod Subnet | Node Subnet |
|---|---|---|---|
vpc-native |
Yes | 10.150.0.0/16 | 10.2.0.0/24 |
vpc-legacy |
No | 10.244.0.0/16 | 10.2.0.0/24 |
Pods
Finally, these clusters have one pod, each located on the vpc-native and vpc-legacy cluster respectively.
| Pod Name | Cluster Name | Pod IP | Node IP |
|---|---|---|---|
vpc-native-pod |
vpc-native |
10.150.0.103 | 10.2.0.3 |
vpc-legacy-pod |
vpc-legacy |
10.244.0.86 | 10.2.0.5 |
If the vpc-native-pod (10.150.0.103) attempts to reach out to native-vpc-droplet (10.2.0.2), the connection succeeds and the native-vpc-droplet sees an inbound connection from 10.150.0.103 (The Pod IP).
# VPC-native Pod
root@vpc-native-pod:~# curl --connect-timeout 1 native-vpc-droplet
I am a Droplet
# VPC-native Droplet Log
10.150.0.103 - - [03/Feb/2025 12:29:12] "GET / HTTP/1.1" 200 -
However, if the pod attempts to reach the legacy-vpc-droplet Droplet, the connection fails instead.
# VPC-native Pod
root@vpc-native-pod:~# curl --connect-timeout 1 legacy-vpc-droplet
curl: (28) Failed to connect to legacy-vpc-droplet port 80 after 1000 ms: Timeout was reached
Using tcpdump we can see that the packet does indeed reach the Droplet.
root@legacy-vpc-droplet:~# tcpdump -i eth1 -s 0 'tcp port http'
12:35:51.865393 IP 10.150.0.103.44260 > legacy-vpc-droplet.http
The connection fails because the legacy-vpc-droplet Droplet does not have the necessary routes to handle a connection from 10.150.0.103, and instead sends it via the default gateway (towards the internet, rather than through the VPC).
Related Topics
