Documentation
System StatusPortal Login

Recommended Network Configuration

Suggest Edits

This section describes the necessary scaffolding and processes to create the initial AWS network environment for Rackspace customers using AWS through the Fanatical Support for AWS offering. A CloudFormation template and additional supporting scripts will be used to create the initial network and all of its necessary components, thus providing Public and Private subnets for EC2 instances and other AWS services.

This includes:

  • A Single VPC
  • Availability Zones (AZ) Options
    • Two AZ deployments are the standard
    • Three AZ deployment to address specific application requirements
  • Subnets
    • Public Tier - could be accessible from the Internet
    • Private Tier - could access the Internet via a NAT environment
  • Highly Available Outbound NAT (HA-NAT) with Elastic IP - for EC2 gateways in the Private Subnets
  • Security Groups - primary method to isolate and secure workloads
  • Tagging - to address Rackspace billing and operational processes

You can access the template by downloading it from here.

📘

The template will create AWS resources for which you will be charged (for example, EC2 NAT gateways).

CloudFormation

There are two important concepts to understand when using AWS CloudFormation: templates and stacks. A template is used to describe your AWS resources and their properties. When you create a stack, AWS CloudFormation provisions the resources that are described in the template.

To learn more, view the AWS documentation on stacks and templates.

Rackspace CloudFormation Template: BaseNetwork

We help customers create custom CloudFormation templates to describe their environments. We make available a standardized CloudFormation Template, BaseNetwork, that creates the initial network and all of its necessary components. The rest of this section describes the elements that are part of the BaseNetwork CloudFormation Template and the associated components. You can download the BaseNetwork template here.

Parameters

  • VPCCIDR - CIDR for the VPC
  • SubnetPublicAZ1 - CIDR for Public subnet
  • SubnetPublicAZ2 - CIDR for Public subnet
  • SubnetPrivateAZ1 - CIDR for Private subnet
  • SubnetPrivateAZ2 - CIDR for Private subnet
  • InstanceTenancy - Single or Multi-Tenant Hypervisor
  • Environment - Dev, Test, Prod etc.

Networking

  • The CloudFormation template has two major options:
    • 2 Availability Zones with 4 Subnets
    • 3 Availability Zones with 6 Subnets
  • Defaults to using CIDR: 172.18.0.0/16
    • Public Ranges
      • 172.18.0.0/22 - 1,022 Hosts - Public AZ1
      • 172.18.4.0/22 - 1,022 Hosts - Public AZ2
      • 172.18.8.0/22 - 1,022 Hosts - Public AZ3
      • 172.18.12.0/22 - 1,022 Hosts - Public AZx
      • 172.18.16.0/20 - 4,094 Hosts - Additional public (4 more public with size listed above)
    • Private Ranges
      • 172.18.32.0/21 - 2,046 Hosts - Private AZ1
      • 172.18.40.0/21 - 2,046 Hosts - Private AZ2
      • 172.18.48.0/21 - 2,046 Hosts - Private AZ3
      • 172.18.56.0/21 - 2,046 Hosts - Private AZx
      • 172.18.64.0/18 - 16,382 Hosts - - Additional private (8 more private using size above)
      • 172.18.128.0/17 - 32,766 Hosts - Special needs (16 more private using size above)
  • Route Tables
    • RouteTablePublic - route table for Public subnets
    • RouteTablePrivateAZ1 - route table for Subnet Private AZ1
    • RouteTablePrivateAZ2 - route table for Subnet Private AZ2
  • Default Gateways
    • Internet Gateway (IGW) - Default GW for the Public Subnets
    • ASGNatAZ1 Instance ID - Default GW for Subnet Private AZ1
    • ASGNatAZ2 Instance ID - Default GW for Subnet Private AZ2
    • ASGNatAZ3 Instance ID - Default GW for Subnet Private AZ3 (if necessary)

HA NAT

  • High Availability NAT gateways get created in the public subnets (1 per AZ)
    • NatAZ1
    • NatAZ2
    • NatAZ3 (if necessary)

Tags

  • Service Provider - “Rackspace”
  • Environment - from Parameter Environment
  • Name - Resource name (e.g. IGWBase, SubnetPublicAZ2)

Outputs

  • outputVPCID
  • outputSubnetPublicAZ1
  • outputSubnetPublicAZ2
  • outputSubnetPublicAZ3 (if necessary)
  • outputSubnetPrivateAZ1
  • outputSubnetPrivateAZ2
  • outputSubnetPrivateAZ3 (if necessary)

Virtual Private Cloud (VPC)

Amazon Virtual Private Cloud (Amazon VPC) lets you provision a logically isolated section of the Amazon Web Services (AWS) Cloud where you can launch AWS resources in a virtual network that you define. Click here to learn more about VPC.

Rackspace Base Network VPC

For most Fanatical Support for AWS customers, Rackspace recommends the deployment of a single VPC per AWS account to provide operational simplicity while meeting stringent security requirements. Segregation will be accomplished by creating Public and Private subnets, and by relying on carefully created Security Groups that only allow the required granular access (further details in the Security section).

If further segregation were required to control access (e.g. Production vs. Test vs. Development), Rackspace’s recommendation is to create a separate AWS accounts, and not a separate VPC in the same AWS account. This is because a second VPC in the same AWS account does not provide control plane isolation of resources, and could complicate ongoing operational processes.

You can assign a single CIDR block to a VPC. The allowed block size is between a /28 netmask and /16 netmask. In other words, the VPC can contain from 16 to 65,536 IP addresses. You cannot change the size of a VPC after you have created it. If your VPC is too small to meet your needs, you will need to create a new, larger VPC, and then migrate your instances to the new VPC.

The VPC requires an RFC 1918 CIDR range (Private addresses). The default is a /16 network, however, you need to carefully consider the range you select to ensure the range does not overlap with your other environments (on premise, other AWS VPCs, Rackspace dedicated environments, etc.).

The CloudFormation template captures the CIDR range in the Parameter: VPCCIDR. Detailed IP addressing recommendations will be discussed in the Subnets section . The BaseNetwork CloudFormation template’s default VPC CIDR is 172.18.0.0/16.

Availability Zones (AZs)

Each region contains multiple distinct locations called Availability Zones, or AZs. Each Availability Zone is engineered to be isolated from failures in other Availability Zones, and to provide inexpensive, low-latency network connectivity to other AZs in the same region.

An Availability Zone (AZ) is one or more data centers in close geographic proximity connected together over low-latency/high-speed links.

By launching instances in separate Availability Zones, you can protect your applications from the failure of a single location. Note: Each AWS region provides a minimum of two AZs.

Rackspace Availability Zone Recommendations

Rackspace typically recommends a two AZ deployment, which provides availability and redundancy while reducing complexity, operational overhead, and cost.

There are situations where a third AZ may be required to address specific application-centric requirements:

  • Example 1: MongoDB’s Election and Quorum constraints require three AZs to survive a single AZ failure that contains the primary and a secondary in a three-node cluster.

  • Example 2: Applications that have strict load and availability requirements that cannot be met by relying on Auto Scaling Groups require over-provisioning. Adding a third AZ could be considered to reduce costs by lowering needed the over-provisioning.

Subnets

You can create a VPC that spans multiple Availability Zones. After creating a VPC, you can add one or more subnets in each Availability Zone. Each subnet must reside exclusively within one Availability Zone and cannot span zones. AWS assigns a unique ID to each subnet.

If a subnet’s traffic is routed to an Internet gateway, the subnet is known as a Public subnet. If the instance in a Public subnet needs to communicate with the Internet, it must have a public IP address or an Elastic IP address. If a subnet doesn’t have a direct route to the Internet gateway, the subnet is known as a Private subnet.

When you create a subnet, you specify the CIDR block for the subnet. The CIDR block of a subnet can be the same as the CIDR block for the VPC (for a single subnet in the VPC), or a subset (to enable multiple subnets). The allowed block size is between a /28 netmask and /16 netmask. You can create more than one subnet in a VPC, but the CIDR blocks of the subnets must not overlap.

Rackspace Subnet Recommendations

For most deployments, Rackspace recommends having two tiers of Subnets: Public and Private.

EC2 instances in Public Subnets have public IP addresses associated with them and have a direct route to an AWS Internet Gateway (IGW), thus having the capability (if required) to access or be accessed by the Internet.

EC2 instances in Private Subnets only have private IP addresses and cannot be accessed by the Internet. These EC2 instances have the capability to access the Internet via a NAT Gateway in the Public subnets (further info in the NAT section).

Assuming a typical two AZ deployment, four subnets would be required (two for Public and two for Private).

In situations where a third AZ is required (e.g. MongoDB servers in the Private subnets) then six subnets would be required (three for Public and three for Private).

It is important to note that within each tier, all the subnets will have the same network mask to simplify the operational processes (e.g. /22 for all Public subnets and /21 for all Private subnets).

Unlike traditional networking segmentation approaches that requires separate subnets (VLANs) for web, batch, application, and data tiers, AWS’s use of Security Groups allows you to leverage just the Public and Private subnets, applying specific Security Groups to each tier (further info in the Security section). Thus a deployment would looks like:

  • Public Subnets
    • Bastion servers
    • NAT servers (if not using a NAT Gateway)
    • VPN servers (if not using a Virtual Private Gateway)
    • Web servers not behind any ELB
  • Private Subnets
    • Web servers behind an ELB
    • Batch-tier instances
    • App-tier instance
    • Data-tier instances

The CloudFormation template uses the built-in “GetAZs” function to map the first or second AZ to the specified subnet in a particular region (e.g. us-west-1a and us-west-1b). The CloudFormation template also captures the CIDR range for the subnet in the parameters:

  • SubnetPublicAZ1 - CIDR for Public subnet
  • SubnetPublicAZ2 - CIDR for Public subnet
  • SubnetPrivateAZ1 - CIDR for Private subnet
  • SubnetPrivateAZ2 - CIDR for Private subnet
  • SubnetProtectedAZ1 - CIDR for Protected subnet
  • SubnetProtectedAZ2 - CIDR for Protected subnet

It is recommend that you choose the CIDRs carefully to map with the applications’ requirements; however, most AWS customers typically allocate roughly double the IP addresses for private subnets than public subnets. The default CIDRs in the BaseNetwork CloudFormation template are detailed in the CloudFormation section.

Highly Available Network Address Translation (HA NAT)

In a VPC, you can use private subnets for instances that do not require directly accessible Internet-facing IP addresses. Instances in a private subnet can access the Internet without exposing their private IP address by routing their traffic through a Network Address Translation (NAT) gateway in a public subnet. Each NAT gateway is created in a specific Availability Zone (AZ) and has built-in redundancy in that AZ.

Rackspace NAT Recommendations

As mentioned above, NAT gateways are required for instances in a Private subnet to access the Internet. In the recommended two AZ deployment, Rackspace recommends leveraging one NAT gateway in each AZ - not sharing a NAT gateway with more than one AZ.

NAT gateways are created via the CloudFormation template which:

  • Creates an Elastic IP Address (EIP) for each NAT gateway to be reachable on public networks
  • Creates a route for each private network in each AZ to route all traffic through the corresponding NAT gateway in each AZ.

Rackspace does not recommend creating resources in one AZ that rely exclusively on a NAT gateway in a different AZ. In the event of a NAT gateway failure, resources in any AZ that depend on that single NAT gateway will be unable to access the Internet.

Migrating from a NAT instance

If you were previously using NAT instances for allowing resources on private networks to access the Internet, you should create a NAT gateway in each AZ, and change the routing tables for your private networks to use the new NAT gateway. Then, existing resources associated with your NAT instances (autoscale groups, NAT instances in EC2) can be removed; the change will only impact connections open at the time of the change to the routing table.

You should also take care to ensure that any existing whitelist of IPs from the NAT instances are also adjusted to reflect the new IPs of your NAT gateways.

Security

AWS provides a scalable, highly reliable platform that helps customers deploy applications and data quickly and securely.

When customers build systems on the AWS infrastructure, security responsibilities are shared between the customer and AWS. This shared model can reduce the customer’s operational burden as AWS operates, manages, and controls the components from the host operating system and virtualization layer down to the physical security of the facilities in which the services operate. In turn, the customer assumes responsibility and management of the guest operating system (including updates and security patches), other associated applications, as well as the configuration of the AWS-provided security group firewall.

The Rackspace Fanatical Support for AWS offering takes some of the security burden from the customer by leveraging AWS security best practices and providing additional security capabilities. These include using/enabling Security Groups, Config, CloudTrail, CloudWatch, etc. In this section, we will focus on Security Groups.

Security Groups

A Security Groups acts as a virtual firewall that controls inbound and outbound traffic for one or more instances. When an instance is launched in a VPC, the instance can be assigned up to five security groups that are associated to the VPC. Specific inbound and outbound rules are then added to each security group that allows defined traffic to or from its associated instances. Rules can be modified at any time; the new rules are automatically applied to all instances that are associated with the security group.

📘

By default, outbound rules allow all traffic to egress the instance and inbound rules allow nothing (implicit deny).

Security groups act at the instance level, not the subnet level. Therefore, each instance in a subnet in a VPC could be assigned to a different set of security groups, thus easily creating isolation within the same subnet. When AWS is deciding whether to allow traffic to reach an instance, all the rules from all the security groups that are associated with the instance are evaluated at the same time. This is very different to the way Network ACLs (NACLs) work.

Network ACLs (NACLs)

AWS also offers network ACLs with rules similar to your security groups. NACLs act as a firewall for associated subnets, controlling both inbound and outbound traffic at the subnet level. The following summarizes the basic differences between security groups and network ACLs:

Security Group

  • Operates at the instance level (first layer of defense)
  • Supports allow rules only
  • Is stateful: Return traffic is automatically allowed, regardless of any rules
  • We evaluate all rules before deciding whether to allow traffic
  • Applies to an instance only if someone specifies the security group when launching the instance, or associates the security group with the instance later on

Network ACL

  • Operates at the subnet level
  • Supports allow rules and deny rules
  • Is stateless: Return traffic must be explicitly allowed by rules
  • We process rules in number order when deciding whether to allow traffic
  • Automatically applies to all instances in the subnets it’s associated with (backup layer of defense, so you don’t have to rely on someone specifying the security group)

Rackspace Security Model

As a general best practice, Rackspace advises customers to use Security Groups as their primary method of securing workloads within AWS. While Network ACLs (NACLs) are typically more familiar to networking engineers, they often introduce complexity into AWS architectures.

Security Groups provide more granular control, are stateful (therefore more intelligent in allowing appropriate traffic) and apply only to the instance level. By using NACLs as well as Security Groups, you must consider all traffic in a stateless context (specifying inbound and outbound ports, including any ephemeral ports used by a given application) and these rules are applied at a subnet level; the “blast radius” or potential for impact when a NACL is incorrect or changed is significantly higher, without providing any tangible benefit over the use of a Security Group.

Rackspace and AWS recommend avoiding NACLs due to potential conflicts with Security Groups and performance degradation. If there are compliance requirements (e.g. PCI) that specifically call for NACLs, they will be used sparingly and with coarse controls to mitigate potential issues.

Tagging

AWS customers use tags to organize their EC2 resources (instances, images, load balancers, security groups, and so forth), RDS resources (DB instances, option groups, and more), VPC resources (gateways, option sets, network ACLS, subnets, and the like), Route 53 health checks, and S3 buckets. Tags are used to label, collect, and organize resources and become increasingly important as customers use AWS in larger and more sophisticated ways.

For example, customers can tag relevant resources and then take advantage cost allocation via tagging.

Rackspace CloudFormation Tagging

The BaseNetwork CloudFormation template makes use of tagging to drive many of the operational functions associated with the Fanatical Support for AWS offering. These include:

  • Service Provider - “Rackspace”
  • Environment - from Parameter Environment
  • Name - Resource name (e.g. IGWBase, SubnetPublicAZ2)

Updated about 1 year ago