NetflixOSS: Installing and Running ICE

NetflixOSS ICE is AWS COST monitoring solution. Similar to cloud watch but IMHO is better. There are some issues with ICE first of all is kinda of buggy at least right now - something you got some NPE and is hard to figure out what you did wrong.

Second - its kinda of slow to run first time it take some time because it does lots of processing to figure out the costs.  You need have your billing configure right on AWS and also you need a S3 bucket to store the billing data so ICE and read and show the charts for you. We will install ICE on Amazon Linux OS(CentOS based).

Installing and Running ICE on Amazon Linux OS

Beware of the policies - you need add some policies in S3 in order to get this working right - in aws you can get your tight policies this are the policies are generated for me for you might be different so keep that in mind. You will need to have your amazon Key and Secret as well so when we run ice you will pass that by parameter.

Netflix/ICE

goto https://console.aws.amazon.com/s3/home?region=us-west-2#
create a bucket: awscostice
add policy more or less like this:

 { 
 	  "Version": "2008-10-17", 
 	  "Id": "Policy1335892530063", 
 	  "Statement": [ 
 		 { 
 			   "Sid": "Stmt1335892150622",
 			   "Effect": "Allow", 
 			   "Principal": { 
 				        "AWS": "arn:aws:iam::386209384616:root" 
 		    	},
 			"Action": [ 
 				"s3:GetBucketAcl", 
 				"s3:GetBucketPolicy" 
 			], 
 			"Resource": "arn:aws:s3:::awscostice" 
 		}, 
 		{ 
 			"Sid": "Stmt1335892526596", 
 			"Effect": "Allow", 
 			"Principal": { 
 				"AWS": "arn:aws:iam::386209384616:root" 
 			}, 
 			"Action": "s3:PutObject", 
 			"Resource": "arn:aws:s3:::awscostice/*" 
 		} 
 	] 
 }

Goto: https://console.aws.amazon.com/billing/home#/preferences
last checkbox: awscostice

Create EC2 Instance Amazon Linux
Install Java 8

sudo yum install -y git vim 
sudo yum remove -y java 
cd /opt 
sudo wget --no-cookies --no-check-certificate --header "Cookie: gpw_e24=http%3A%2F%2Fwww.oracle.com%2F; oraclelicense=accept-securebackup-cookie" "http://download.oracle.com/otn-pub/java/jdk/8u45-b14/jdk-8u45-linux-x64.tar.gz"
sudo tar -xzvf jdk-8u45-linux-x64.tar.gz 
sudo rm -rf jdk-8u45-linux-x64.tar.gz

Add to ~/.bashrc and /root/.bashrc
sudo vim ~/.bashrc 
alias cls='clear' 
export JAVA_HOME=/opt/jdk1.8.0_45 
export JRE_HOME=/opt/jdk1.8.0_45/jre 
export PATH=$PATH:/opt/jdk1.8.0_45/bin:/opt/jdk1.8.0_45/jre/bin

$ source ~/.bashrc
Install Grails 2.4.4 on CentOs

cd opt/ 
sudo wget https://github.com/grails/grails-core/releases/download/v2.4.4/grails-2.4.4.zip
sudo unzip grails-2.4.4.zip
sudo rm -rf grails-2.4.4.zip

Add to ~/.bashrc and /root/.bashrc
$ sudo vim ~/.bashrc

export GRAILS_HOME=/opt/grails-2.4.4/ <BR>
export PATH=$PATH:$GRAILS_HOME/bin <BR>

source ~/.bashrc

You will need get your AWS_ACCOUNT_ID you cab get it here => https://console.aws.amazon.com/billing/home#/account them replace on the properties file where it says YOU_NEED_CHANGE_THIS_AND_PUT_YOUR_AWS_ACCOUNT_ID_HERE

Create ice.properties where you downloaded ice from github

ice.processor=true
ice.processor.localDir=/mnt/ice_processor
ice.work_s3bucketname=awscostice
ice.work_s3bucketprefix=aws
#ice.startmillis=1364774400000

ice.reader=true 
ice.reader.localDir=/mnt/ice_reader
ice.companyName=CompanyX

# s3 bucket name where the billing files are. multiple bucket names are delimited by ",". Ice must have read access to billing s3 bucket.
ice.billing_s3bucketname=awscostice

# prefix of the billing files. multiple prefixes are delimited by ","
ice.billing_s3bucketprefix=,

# specify your payer account id here if across-accounts IAM role access is used. multiple account ids are delimited by ",". "ice.billing_payerAccountId=,222222222222" means assumed role access is only used for the second bucket.
ice.billing_payerAccountId=,YOU_NEED_CHANGE_THIS_AND_PUT_YOUR_AWS_ACCOUNT_ID_HERE

# specify the assumed role name here if you use IAM role access to read from billing s3 bucket. multiple role names are delimited by ",". "ice.billing_accessRoleName=,ice" means assumed role access is only used for the second bucket.
ice.billing_accessRoleName=,ice

Create Processor/Reader folders

sudo mkdir /mnt/ice_processor
sudo mkdir /mnt/ice_reader

Get ice from github and build and run

sudo git clone https://github.com/Netflix/ice.git
cd ice/
grails wrapper
./grailsw run-app
./grailsw -Dice.s3AccessKeyId=xxx -Dice.s3SecretKey=xxx

GOTO http://HOST:8080/ice/dashboard/summary

view raw ice.md hosted with ❤ by GitHub
When you run ice you will see something like this:



Have Fun :-)

Cheers,
Diego Pacheco

Popular posts from this blog

Having fun with Zig Language

C Unit Testing with Check

HMAC in Java