OSS - Netflix ICE deployment using a CloudFormation template
Last updated - 2018/01/31
Who is this page for
If you are trying to start NetflixOSS ICE (now TeevityOSS ICE) on your own AWS account using a CloudFormation, this page is for you.
The CloudFormation template is maintained by Teevity and:
- relies on docker-ice
- lets you store your
ice.propertiesfile on an S3 bucket for easy update
- [NOT YET] - supports reloading of the
ice.propertiesfile from the S3 bucket, after you have updated it, via a simple reboot of the EC2 instance. This makes it easy to experiment with various parameters
Step A. Gather the following information
In order to configure and deploy ICE using this CloudFormation, you will need to gather the following information. This will be needed to update some of the snipets given below and to pass parameters to the CloudFormation stack.
##ACCOUNT_WHERE_NETFLIXICE_IS_RUNNING## = 077....86732
##S3_PROGRAMMATIC_BILLING_BUCKET_NAME## = ...-billing
##BILLINGACCOUNT## = 431....66902
##CHILDACCOUNT_001## = 913....90804
##CHILDACCOUNT_002## = 351....79527
##CHILDACCOUNT_...## = ...
##S3_PATH_TO_YOUR_ICE_PROPERTIES## = ...-iceproperties-bucket/teevity-oss-ice.properties
##YOUR_SSH_KEYPAIR## = ...KeyPair
Step B. Create the various IAM Roles needed by the CloudFormation
The CloudFormation itself will define some permissions. But you need to create some IAM Roles ahead of the deployment of the stack.
To help you understand the goal of each of these roles, we have created a set of 3 scripts that decompose the creation of those Roles:
- Step 1 - Creation of the Role used by the EC2 instance
- Step 2 - Creation of the Role used to give access to the billing buckets (multiple billing buckets are supported)
- Step 3 - Creation of the Roles, on each "AWS child account", to let ICE poll ec2 reserved instances on these accounts
Each step has an associated bash file (which contains a few AWS CLI commands) and a few dependency files (policy and assume-role-document):
You can find the scripts and their dependant files in this zip file.
Step C. Create an ice.properties file and upload it to an S3 bucket
You will then need to define the following values inside your
ice.properties file (the name of the roles given below are the ones you should use if you have followed the scripts). You then need to upload the file to an S3 bucket.
WARNING - Below is just an extract of an
ice.properties file. You can download a full ice.properties template here.
Step D. Instanciate the CloudFormation to create a 'Netflix ICE instance'
To create the stack using the CloudFormation template, you can use the following AWS CLI command.
aws cloudformation create-stack \
--stack-name "NetflixOSS-Ice" \
--template-body "file://teevity-oss-ice-CloudFormationTemplate-v2.json" \
--capabilities "CAPABILITY_IAM" \
Investigating what's happening inside the Docker container
If you need to check the ICE logs, or inspect the content of the ice.properties file (that has been copied from the S3 bucket you have specified as a parameter of the stack), you can SSH into the instance.
You can inspect the logs of the Docker composition (started by docker-ice) where you will find the ICE log and the ngnix logs.
# You need the environment of the root user so that docker-compose
# can reach the docker daemon
docker-compose logs -f
The ICE properties file can be found inside
Here is the directory structure under /opt/teevityoss-ice