Background

Cloud Analytics uses two primary data sources for helping you manage costs:

  • Bill data - Bill information is collected from your public cloud provider to enable an accurate view of all of your costs across your accounts and services
  • Usage data - Usage data is collected from RightScale Cloud Management to provide additional detail for slicing and dicing costs across many different dimensions

This page describes how to connect bill data for GCE. Learn about the data sources for Cloud Analytics, how to connect bills for other clouds, and how to connect usage data.

If you have any questions and would like live assistance, please join us on our chat channel on chat.rightscale.com or email us at support@rightscale.com.

Steps

  1. Enable Billing Data Export to CSV in GCE
  2. Allow RightScale to access the CSV billing bucket
  3. Update existing billing CSV permissions (optional)
  4. Send configuration information to RightScale

Enable Billing Data Export to CSV in GCE

RightScale consumes the billing data via the CSV export method in Google. This can be enabled in GCE for each Billing Account and will contain data for all projects in that Billing Acccount.

To configure the data export to CSV, please follow the instructions provided by Google for your Google billing accounts.

Be sure to select the Export to File option and use the CSV file format.

Update the Permissions on the Google Storage Bucket

Once your billing CSVs are being exported to a Google Bucket, the RightScale service account needs permission to access those files. The following steps walk through the setup of these permissions.

  1. Log into https://console.developers.google.com.
  2. From the dropdown, select the project to use for accessing your cloud storage bucket (where your bill CSV or JSON files live). google-select-project.gif

  3. Navigate to Storage > Cloud Storage > Browser from the left-hand navigation bar. google-storage-nav.gif

  4. Find the billing bucket and click the three dots on the far right, then select Edit object default permissions. Click on Add item and select User from the Entity dropdown. Add the RightScale email address 5295748725-ni3a6uhuioneglrbostvk43tvgi21ei6@developer.gserviceaccount.com in the Name field and select Reader on the Access dropdown, then click Save. google-edit-perms.gif

  5. Repeat the previous step but this time select Edit bucket permissions in the three-dot dropdown for the bucket. google-repeat.gif

Update Permissions on Existing Files in the Bucket

Optional - If you already have billing CSV files in the bucket, this step enables Cloud Analytics to import them. This is not necessary if there are no existing CSV files in the bucket.

  1. Download and install gsutil and run gsutil config to set it up.
  2. Copy/paste the following script into a new file called permission_google_bucket.sh. This script uses gsutil to update the permission for all of the existing files in the bucket.

    #!/bin/bash
    # This script will accept a bucket name and give RightScale read permission on every item in it.
    # This is an idempotent script; it's safe to re-run it multiple times if it fails.
    if which gsutil >/dev/null; then
      echo "Please enter the name of the bucket that needs to be updated, followed by [ENTER]:"
      read bucket
      files=`gsutil list gs://$bucket`
      total=$((`echo $files | wc -w`))
      echo "Adding reader permission on $total files in $bucket for RightScale Cloud Analytics. This can take a while..."
      counter=0
      for file in $files
      do
        acl_file=/tmp/google_acl.txt
        gsutil acl get $file > $acl_file
        echo $acl_file | python -c $'import sys, json; email = "5295748725-ni3a6uhuioneglrbostvk43tvgi21ei6@developer.gserviceaccount.com"; file = sys.stdin.read().strip(); data = json.loads(open(file).read()); role = {"role": "READER", "email": email, "entity": "user-"+email};\nif role not in data: data.append(role); open(file, "w").write(json.dumps(data))'
        gsutil acl set $acl_file $file
        counter=$((counter + 1))
        echo -ne $(echo 'scale=2; 100 * '$counter' / '$total'' | bc -l)'% done: '
        rm $acl_file
      done
      echo "$total files updated. You can now delete this script."
    else
      echo "This script requires gsutil, please install and configure using instructions from https://cloud.google.com/storage/docs/gsutil_install"
    fi
    
  3. Run the script as follows; it will ask for your bucket name and output its progress as it updates all file permissions in the bucket.

    chmod u+x permission_google_bucket.sh
    ./permission_google_bucket.sh
    

Give RightScale Details of your Google Accounts

Note: the following process is temporary - the below will be built-in to the platform soon.

To finalize the setup, please email the following information to support@rightscale.com:

  • The Google storage bucket name that has your billing files.
  • The RightScale account ID that your billing files should be tied to. Users with access to this RightScale account will be able to see all of the cloud bills in the bucket. This should be the account ID of your Enterprise Master account.
  • For each GCE project included in your bill, we require additional information about the project due to limitations of the bill data from GCE. For each project contained in your bill, please provide:
    • The GCE Project ID
    • The GCE Project Name
    • The GCE Project Number - Some billing files only expose the Project Number in the bill, not the Project ID. So we need this information submitted to ensure we can map the Project Number in the bill to the Project ID
  • Support costs in the GCE bill are not associated to a Project, so you must select which GCE Project you want those costs associated with and notify us.

Example:

Project ID Project Name Project Number
My-id-123 My Project Name 1 123123123
My-id-456 My Project Name 2 234234234

You can find your Project Name, ID and Number in the Google Cloud Console as shown below. Alternatively you can use the gcloud beta projects list command to output these, see this page for more details.

google-dev-console.png