Last updated - 2018/01/31

Who is this page for

If you are trying to start NetflixOSS ICE (now TeevityOSS ICE) on your own AWS account using a CloudFormation, this page is for you.

The CloudFormation template is maintained by Teevity and:

• relies on docker-ice

• lets you store your ice.properties file on an S3 bucket for easy update

• [NOT YET] - supports reloading of the ice.properties file from the S3 bucket, after you have updated it, via a simple reboot of the EC2 instance. This makes it easy to experiment with various parameters

Step A. Gather the following information

In order to configure and deploy ICE using this CloudFormation, you will need to gather the following information. This will be needed to update some of the snipets given below and to pass parameters to the CloudFormation stack.

##ACCOUNT_WHERE_NETFLIXICE_IS_RUNNING## = 077....86732

##S3_PROGRAMMATIC_BILLING_BUCKET_NAME## = ...-billing

##BILLINGACCOUNT##                      = 431....66902

##CHILDACCOUNT_001##                    = 913....90804

##CHILDACCOUNT_002##                    = 351....79527

##CHILDACCOUNT_...##                    = ...

##S3_PATH_TO_YOUR_ICE_PROPERTIES##      = ...-iceproperties-bucket/teevity-oss-ice.properties

##YOUR_SSH_KEYPAIR##                    = ...KeyPair

Step B. Create the various IAM Roles needed by the CloudFormation

The CloudFormation itself will define some permissions. But you need to create some IAM Roles ahead of the deployment of the stack.

To help you understand the goal of each of these roles, we have created a set of 3 scripts that decompose the creation of those Roles:

• Step 1 - Creation of the Role used by the EC2 instance

• Step 2 - Creation of the Role used to give access to the billing buckets (multiple billing buckets are supported)

• Step 3 - Creation of the Roles, on each "AWS child account", to let ICE poll ec2 reserved instances on these accounts

Each step has an associated bash file (which contains a few AWS CLI commands) and a few dependency files (policy and assume-role-document):

step1.create.NetflixIce_InstancesExecution_Role.sh

step2.create.NetflixIce_BillingBucketAccess_Role.s

step3.create.NetflixIce_ChildAccountAccess_Role.sh

You can find the scripts and their dependant files in this zip file.

Step C. Create an ice.properties file and upload it to an S3 bucket

You will then need to define the following values inside your ice.properties file (the name of the roles given below are the ones you should use if you have followed the scripts). You then need to upload the file to an S3 bucket.

WARNING - Below is just an extract of an ice.properties file.  You can download a full ice.properties template here.

ice.billing_s3bucketregion=eu-west-1

ice.billing_s3bucketname=...-billing

ice.billing_payerAccountId=431....66902    

ice.customTags=user:Application,user:Environment,user:Owner

ice.billing_accessRoleName=NetflixIce_BillingBucketAccess_Role

ice.companyName=Acme

ice.account.root=431....66902

ice.account.dev3=913....90804

ice.account.internal=351....79527

Step D. Instanciate the CloudFormation to create a 'Netflix ICE instance'

To create the stack using the CloudFormation template, you can use the following AWS CLI command.

aws cloudformation create-stack \

    --stack-name "NetflixOSS-Ice" \

    --template-body "file://teevity-oss-ice-CloudFormationTemplate-v2.json" \

    --capabilities "CAPABILITY_IAM" \

    --parameters \

         "ParameterKey=IcePropertiesConfigurationFileS3Path,ParameterValue=netflixoss-ice-iceproperties-bucket/teevity-oss-ice.properties" \

         "ParameterKey=TimeZone,ParameterValue=Europe/Paris" \

         "ParameterKey=NetflixIceInstancesExecutionRoleName,ParameterValue=NetflixIce_InstancesExecution_Role" \

         "ParameterKey=EC2InstanceSize,ParameterValue=t2.medium" \

         "ParameterKey=IceInstanceSSHKeyPairName,ParameterValue=...KEYPAIR" \

    --tags \

         "Key=Application,Value=NetflixOSS-ICE" \

         "Key=Environment,Value=Test" \

         "Key=Owner,Value=..."  \

    --region "ap-northeast-2"

Investigating what's happening inside the Docker container

If you need to check the ICE logs, or inspect the content of the ice.properties file (that has been copied from the S3 bucket you have specified as a parameter of the stack), you can SSH into the instance.

You can inspect the logs of the Docker composition (started by docker-ice) where you will find the ICE log and the ngnix logs.

# You need the environment of the root user so that docker-compose

# can reach the docker daemon

sudo su

cd /opt/teevityoss-ice/docker-ice

docker-compose logs -f

The ICE properties file can be found inside 

/opt/teevityoss-ice/docker-ice/ice/assets/ice.properties

Here is the directory structure under /opt/teevityoss-ice