Background

Optima uses two primary data sources for helping you manage costs:

  • Bill data - Bill information is collected from your public cloud provider to enable an accurate view of all of your costs across your accounts and services
  • Usage data - Usage data is collected from RightScale Cloud Management to provide additional detail for slicing and dicing costs across many different dimensions

This page describes how to connect bill data for GCE. Learn about the data sources for Optima, how to connect bills for other clouds, and how to connect usage data.

If you have any questions and would like live assistance, please join us on our chat channel on chat.rightscale.com or email us at support@rightscale.com.

Steps

  1. Enable Billing Data Export to CSV in GCE
  2. Allow RightScale to access the CSV billing bucket
  3. Update existing billing CSV permissions (optional)
  4. Send configuration information to RightScale

Enable Billing Data Export to CSV in GCE

RightScale consumes the billing data via the CSV export method in Google. This can be enabled in GCE for each Billing Account and will contain data for all projects in that Billing Acccount.

To configure the data export to CSV, please follow the instructions provided by Google for your Google billing accounts.

Be sure to select the Export to File option and use the CSV file format.

Update the Permissions on the Google Storage Bucket

Once your billing CSVs are being exported to a Google Bucket, the RightScale service account needs permission to access those files.

  1. Log into Google Cloud and navigate to the storage bucket used for billing exports. Click on the actions on the right and select Edit bucket permissions google-storage-nav.gif

  2. On the Permissions tab, add RightScale's service account address (5295748725-ni3a6uhuioneglrbostvk43tvgi21ei6@developer.gserviceaccount.com) in the textbox and select both the Storage Legacy Bucket Reader and Storage Legacy Object Reader under the Storage Legacy menu. google-edit-perms.gif

  3. Click Add to complete the role granting process.

Update Permissions on Existing Files in the Bucket

Optional - If you already have billing CSV files in the bucket, this step enables Optima to import them. This is not necessary if there are no existing CSV files in the bucket.

  1. Download and install gsutil and run gsutil config to set it up.
  2. Copy/paste the following script into a new file called permission_google_bucket.sh. This script uses gsutil to update the permission for all of the existing files in the bucket.

    #!/bin/bash
    # This script will accept a bucket name and give RightScale read permission on every item in it.
    # This is an idempotent script; it's safe to re-run it multiple times if it fails.
    if which gsutil >/dev/null; then
      echo "Please enter the name of the bucket that needs to be updated, followed by [ENTER]:"
      read bucket
      files=`gsutil list gs://$bucket`
      total=$((`echo $files | wc -w`))
      echo "Adding reader permission on $total files in $bucket for RightScale Optima. This can take a while..."
      counter=0
      for file in $files
      do
        acl_file=/tmp/google_acl.txt
        gsutil acl get $file > $acl_file
        echo $acl_file | python -c $'import sys, json; email = "5295748725-ni3a6uhuioneglrbostvk43tvgi21ei6@developer.gserviceaccount.com"; file = sys.stdin.read().strip(); data = json.loads(open(file).read()); role = {"role": "READER", "email": email, "entity": "user-"+email};\nif role not in data: data.append(role); open(file, "w").write(json.dumps(data))'
        gsutil acl set $acl_file $file
        counter=$((counter + 1))
        echo -ne $(echo 'scale=2; 100 * '$counter' / '$total'' | bc -l)'% done: '
        rm $acl_file
      done
      echo "$total files updated. You can now delete this script."
    else
      echo "This script requires gsutil, please install and configure using instructions from https://cloud.google.com/storage/docs/gsutil_install"
    fi
    
  3. Run the script as follows; it will ask for your bucket name and output its progress as it updates all file permissions in the bucket.

    chmod u+x permission_google_bucket.sh
    ./permission_google_bucket.sh
    

Give RightScale Details of your Google Accounts

Note: the following process is temporary - the below will be built-in to the platform soon.

To finalize the setup, please email the following information to support@rightscale.com:

  • The Google storage bucket name that has your billing files.
  • Please provide the RightScale organization ID as billing creds can only be associated with an Organization. You can find the ID from the page url in the Governance module. It will be the numeric value after /org/ in the url. e.g.: /org/<orgid>/
  • For each GCE project included in your bill, we require additional information about the project due to limitations of the bill data from GCE. For each project contained in your bill, please provide:
    • The GCE Project ID
    • The GCE Project Name
    • The GCE Project Number - Some billing files only expose the Project Number in the bill, not the Project ID. So we need this information submitted to ensure we can map the Project Number in the bill to the Project ID
  • Support costs in the GCE bill are not associated to a Project, so you must select which GCE Project you want those costs associated with and notify us.

Example:

Project ID Project Name Project Number
My-id-123 My Project Name 1 123123123
My-id-456 My Project Name 2 234234234

You can find your Project Name, ID and Number in the Google Cloud Console as shown below. Alternatively you can use the gcloud beta projects list command to output these, see this page for more details.

google-dev-console.png