A Primer on AWS Networking for Data Engineers

As a data engineer, I've found that understanding the basics of AWS networking is crucial for building reliable and performant data pipelines. Even though AWS abstracts away much of the complexity of networking, it's still important to have a solid understanding of how it works in order to troubleshoot issues that might arise.
I recently got into a conversation about AWS RDS(Relational Database Service) networking with a colleague, and I realized that it would help if I wrote a primer on the topic, more as a reference for myself in the future.

In this post, I'll cover the basics of AWS networking, including VPCs, subnets, security groups, and route tables. I'll also touch on some advanced topics like VPC endpoints and NAT gateways. By the end of this post, you should have a good understanding of how networking works in AWS and how to design a secure and scalable network for your data workloads.

---

Update (2025-02-22): Thanks to Jamie for pointing out that I missed explaining availability zones in the VPC section. I've updated the post to include that information.

---

Contents

1. VPCs(Virtual Private Clouds)
   1. Key Components of a VPC
   2. VPC CIDR Blocks
   3. VPC Design Considerations
2. Subnets
   1. Private subnets
   2. Public subnets
   3. Subnet CIDR Blocks
3. Route Tables
4. Internet Gateway(IGW)
5. NAT Gateway
6. NACL (Network Access Control List)
7. VPC Endpoints
8. Security Groups (SG)
9. Transit Gateway
10. Load balancers
11. Route 53
12. Conclusion

VPCs(Virtual Private Clouds)

A VPC or Virtual Private cloud is a private network that isolates your AWS resources from the public internet. It's similar to a traditional data center network but AWS manages it for you. When you create an AWS account, you get a default VPC. However, it's a best practice to create your own VPCs to have more control over your network configuration. AWS allows 5 VPCs per region by default, but you can request more if needed.

An availability zone (AZ) can be thought of as an AWS data center. Each region has multiple availability zones connected via low-latency links. AZs are important for high availability, fault tolerance, and disaster recovery. When you create a VPC, you can choose to spread your subnets across multiple AZs for redundancy. This ensures that if one AZ goes down, your resources in the other AZ can still function and serve traffic.

Key Components of a VPC

A VPC consists of several key components:

1. CIDR Block: This defines the IP address range for your VPC. For example, 10.0.0.0/16 gives you 65,536 IP addresses.

2. Subnets: These are segments of your VPC's IP address range where you can place AWS resources. Subnets can be public (with internet access) or private.

3. Route Tables: These control traffic routing between subnets and to/from the internet.

4. Internet Gateway: This allows communication between your VPC and the internet.

We'll be diving into each of these in more detail in the following sections.

VPC CIDR Blocks

CIDR(Classless Inter-Domain Routing) blocks are used to define the IP address range for your VPC. The CIDR block you choose will determine the number of IP addresses available to your VPC.
For example: 10.0.0.0/16 gives you 65,536 IP addresses. The /16 denotes the number of bits in the network portion of the IP address.

Each part of the block is an 8-bit number (octet), so /16 means the first 16 bits are fixed, and the remaining 32-16=16 bits are variable. This gives you 2^16=65,536 IP addresses.


Total size of the CIDR block = 2^(32 - prefix_length). I find the method of thinking about CIDR blocks in one of TravisMedia's videos very helpful.

Examples:

1. CIDR block: 10.0.0.0/24 --> The first 24 bits are fixed. As the CIDR block is 32 bits long, the remaining 32-24=8 bits are variable.
   
   Therefore, the total size of the CIDR block = 2^(32-24) = 2^8 = 256 IP addresses.
2. CIDR block: 10.0.0.0/16 --> The first 16 bits are fixed. The total size of the CIDR block = 2^(32-16) = 2^16 = 65,536 IP addresses.

VPC Design Considerations

When designing VPCs for data workloads, consider:

1. IP address space planning

   • Choose CIDR blocks that don't overlap with other networks
   • Allocate enough IP addresses for future growth
   • Reserve IP ranges for different functions (databases, compute, etc.)

2. Subnet distribution across Availability Zones

   • Spread subnets across multiple AZs for high availability
   • Maintain consistent subnet sizes across AZs
   • Consider placing data-intensive workloads in the same AZ to reduce costs

3. Network segmentation for different environments

   • Separate production and development resources
   • Use different subnets for web, application, and database tiers
   • Implement proper routing between segments as needed

4. Connectivity requirements with on-premises networks

   • Plan for VPN or Direct Connect connections
   • Ensure CIDR ranges don't conflict with on-premises networks

A common pattern is to create separate VPCs for development and production environments to maintain isolation and security. These VPCs can be connected using VPC peering or Transit Gateway for communication between environments.

Subnets

Subnets are segments/parts of your VPC that you can use to organize your resources. They are associated with a specific CIDR block within the VPC's IP address range. Subnets can be public or private, depending on whether they have internet access.

Private subnets

Private subnets are used to isolate resources from the public internet. They can still access the internet through a NAT Gateway or NAT Instance (more on this later), but they can't be accessed from the internet directly. Private subnets are typically used for databases, internal APIs, and other sensitive workloads.

Public subnets

Public subnets have a route to an Internet Gateway, allowing resources within the subnet to communicate with the internet. Public subnets are used for web servers, load balancers, and other resources that need to be accessible from the internet.

Subnet CIDR Blocks

Each subnet is associated with a CIDR block that is a subset of the VPC's CIDR block.

The size of the CIDR blocks for subnets should be chosen carefully to ensure you have enough IP addresses for the resources in that subnet.

A common practice is to have 2 subnets per Availability Zone (AZ) - one public and one private. This provides high availability and fault tolerance for your resources.

For example:

• VPC CIDR block: 10.0.0.0/16 = 65,536 IP addresses
• Subnet 1 CIDR block: 10.0.0.0/24 = 256 IP addresses
• Subnet 2 CIDR block: 10.0.1.0/24 = 256 IP addresses
  and so on.

With this setup, you can have 2 subnets per AZ, each with 256 IP addresses, for a total of 512 IP addresses per AZ. Most regions have at least 2 AZs, so you would have 1024 IP addresses in total.

AWS reserves 5 IP addresses in each subnet for its own use. For a CIDR block 10.0.0.0/24:

• 10.0.0.0: Network address
• 10.0.0.1: Reserved for the AWS VPC Router (handles traffic routing between subnets)
• 10.0.0.2: Reserved for the AWS DNS server
• 10.0.0.3: Reserved for future use by AWS
• 10.0.0.255: Network broadcast address. AWS does not support broadcast in a VPC, so this address is reserved.

Which leaves you with 251 usable IP addresses out of the 256 in the CIDR block.

Now that we have a VPC with subnets, let's look at how traffic is routed between them.

Route Tables

Route tables are used to control the flow of traffic between subnets within a VPC. Each subnet is associated with a route table that determines how traffic is routed to and from its resources.

By default, AWS creates a main route table for your VPC that allows traffic to flow between the subnets in the VPC. You can create custom route tables to control traffic flow more granularly.

A best practice is to create separate route tables for public and private subnets to enforce network segmentation.

They're useful for:

• Internet access: Route traffic to an Internet Gateway(IGW) for public subnets
• NAT Gateway/Instance: Route traffic to a NAT Gateway or NAT Instance for private subnets
• VPC Peering: Route traffic to another VPC through a VPC Peering Connection
• Transit Gateway: Route traffic to other VPCs or on-premises networks through a Transit Gateway

I like to think of it as a set of instructions that tells AWS where to send traffic based on the routes defined in the table. For example, traffic destined for the internet should go through the Internet Gateway, while traffic between subnets should be routed internally.

Internet Gateway(IGW)

A public subnet cannot access the internet by default as it is isolated within a VPC. To enable internet access, you need to attach an Internet Gateway (IGW) to the VPC.

When you attach an IGW to a VPC, it becomes the default route for all traffic going out of the VPC. This allows resources in public subnets to communicate with the internet.

AWS allows only 1 IGW per VPC. The IGW does not have any security groups or NACLs associated with it. It's simply a gateway that allows traffic to flow between your VPC and the internet, and does not perform any NAT(Network Address Translation) or firewall functions either.

To enable internet access for resources in a public subnet, you need to add a route to the public subnet's route table that points to the IGW. This tells AWS to send all internet-bound traffic through the IGW.

NAT Gateway

Network address translation (NAT) allows instances in a private subnet to initiate outbound connections to the internet while preventing inbound connections from the internet. This is useful for scenarios where you want your resources to access the internet for updates, patches, etc., but you don't want them to be directly accessible from the internet.

NAT gateways are managed by AWS and translate private IP addresses to public IP addresses when sending traffic to the internet. These are created in a public subnet and require an Elastic IP address to function.

All outbound traffic from instances in a private subnet is routed through the NAT gateway. The NAT gateway then sends the traffic to the internet and forwards the response back to the instances in the private subnet.

Something to keep in mind is that NAT gateways are not free. You pay an hourly rate for the NAT gateway, as well as data processing charges for the data transferred through the gateway.

NACL (Network Access Control List)

Network Access Control Lists (NACLs) act as a firewall for controlling traffic in and out of subnets. They are stateless and evaluate rules in order, from lowest to highest.

NACLs are associated with subnets and contain a list of rules that allow or deny traffic based on IP addresses, protocols, and ports. They are evaluated before security groups and can be used to block specific IP addresses or protocols from entering or leaving a subnet.

VPC Endpoints

VPC endpoints allow you to privately connect your VPC to supported AWS services without requiring an internet gateway, NAT device, VPN connection, or Direct Connect connection. This keeps traffic within the AWS network and avoids exposure to the public internet.

There are 2 types of VPC endpoints:

1. Gateway Endpoints: Used for S3 and DynamoDB. They allow you to connect to these services from your VPC without going over the internet. It creates a VPC route table prefix-list entry for these services. These act as virtual routers that direct traffic to S3 and DynamoDB privately through AWS networks.
2. Interface Endpoints: Used for other AWS services like SNS, SQS, and KMS. They provide a private interface to the service within your VPC using AWS PrivateLink.

VPC endpoints are a great way to secure your traffic and reduce data transfer costs by keeping traffic within the AWS network.

Example usage:
AWS Glue can leverage S3 VPC endpoints to access data in S3 without going over the public internet. This is useful for ETL jobs that need to read/write data from S3 securely. You can read more about how Glue uses VPC endpoints here.

Security Groups (SG)

Security groups control traffic that is allowed to reach/leave the resources they are associated with. They act as a virtual firewall that filters traffic based on rules you define.

A security group has inbound and outbound rules that allow you to control traffic based on IP address, protocol and port number. They are stateful, meaning if you allow inbound traffic on a specific port, the response traffic is automatically allowed back out on the same port.

Some examples of SG rules for data workloads:

• Allow inbound traffic on port 5432 (Postgres) from a specific IP range (AWS RDS). This could be used to allow your application servers/company network to access the database.
• For EMR clusters, allow inbound traffic on port 22 (SSH) from a specific IP range (your company network) for debugging purposes.
• For Redshift clusters, allow inbound traffic on port 5439 (Redshift) from a specific IP range (ex. company VPN, Tableau servers) for querying the warehouse.

Security groups rules are evaluated in order, and the most specific rule that matches the traffic is applied. If no rule matches, the default rule (deny all) is applied. SG rules are evaluated after NACL rules.

Transit Gateway

Transit Gateway is a service that allows you to connect multiple VPCs, on-premises networks, and remote offices through a single gateway. It simplifies network architecture by providing a hub-and-spoke model for routing traffic between networks.

At one of my previous companies, we had an AWS sub-account per team and each team had their own VPC. We used Transit Gateway to connect all the VPCs to a central network where shared services like databases, data lakes, and monitoring tools were hosted. This allowed teams to communicate with each other securely and reduced the complexity of managing VPC peering connections.

Load balancers

Load balancers distribute incoming traffic across multiple instances to ensure high availability and fault tolerance. They can be internal (private) or external (public) facing, depending on the use case.

They're commonly created in the public subnets as they have an attached internet gateway(IGW) and route traffic to instances in private subnets.

It is usually accessible using the DNS name of the load balancer or a CNAME record in Route 53 that points to the LB's DNS name. LB then use target groups as destinations. Target groups can be EC2 instances, IP addresses, or Lambda functions.

AWS offers 3 types of load balancers:

1. Application Load Balancer (ALB): Best suited for HTTP/HTTPS traffic. This is the most common type of load balancer used for web applications and the one I have the most experience with.

2. Network Load Balancer (NLB): Best suited for TCP/UDP traffic. This is used for high-performance workloads that require ultra-low latency. I have not used this type of load balancer personally but it's best suited for real-time processing applications.

3. Classic Load Balancer (CLB): The original type of load balancer that supports HTTP, HTTPS, TCP, and SSL traffic. This is the least recommended type of load balancer as it has been deprecated in favor of ALB and NLB.

Route 53

Route 53 is AWS's managed DNS service that allows you to route traffic to different resources based on routing policies. It can be used to route traffic to EC2 instances, load balancers, S3 buckets, and other AWS resources.

Some common use cases for Route 53:

• Domains or subdomains pointing to an Application Load Balancer (ALB) for microservices
• Hosted zones for internal services like databases, monitoring tools, etc.

Route 53 can be used to create public and private hosted zones. Public hosted zones are accessible from the internet, while private hosted zones are accessible only from within the VPC.

Conclusion

In this post, we covered the basics of AWS Networking for data engineers. We discussed VPCs, subnets, route tables, security groups and touched on advanced topics like VPC endpoints, NAT gateways, and Transit Gateway.

As data engineers, we don't need to be networking experts, but having a solid understanding of how networking works in AWS can help us design reliable and secure infrastructure for our workloads. I hope this post has given you a good foundation to build on and explore more advanced networking topics in AWS if needed.

If you have any questions, feedback, corrections or suggestions, please drop a comment down below. I'd love to hear from you!