Overview

Recent security audits of the bash source code have discovered a broad class of security vulnerabilities in the bash shell. These vulnerabilities vary in severity but their characteristics include remote shell code execution, denial-of-service, and heap/stack overflow leading to arbitrary machine code execution. All versions of bash, and therefore all major Linux distributions, are affected.

As a quick test to see if a system is affected, you can run these commands:

  env x='() { :;}; echo vulnerable' bash -c "echo test"
  env foo='() { echo vulnerable; }' bash -c foo
  rm echo; env X='() { (a)=>\' sh -c "echo date"; cat echo

If the output includes the word vulnerable, or if a file named echo exists in the current directory, then your system is vulnerable.

RightScale has deployed rolling update to our affected infrastructure systems as the individual issues were disclosed and patches became available; no downtime or customer impact is expected. We are contacting our public-cloud partners to assess their responses to the vulnerability disclosure and will update our dashboard with any salient information.

The remainder of this document is devoted to helping you remediate the vulnerability in your own systems.

Resolution

RightScale updates its mirrors of vendor package repositories every day at approximately 06:00 UTC. In addition, whenever we become aware of a vendor-issued fix for a critical issue, we perform an on-demand mirror sync and post a system notification on our dashboard.

In order to install fixed bash versions, your machines must be configured to use the latest date of our frozen repositories. This document describes how to temporarily unfreeze your systems' repos, apply the bash fix, and restore repos to their original freeze dates.

Prerequisite

To familiarize yourself with your OS vendor's response to these disclosures, refer to your vendor's security information portal and search by CVE name:

  • RedHat -- online CVE tracker
  • CentOS -- centos-announce mailing list
  • Ubuntu -- online CVE tracker

For specific information on CVE-2014-6271:

For specific information on CVE-2014-7169:

Note that the patched package version, when available, is noted for every vulnerability. In most cases a direct link is provided to the updated package files. This may be useful if automatic patching fails.

Patch Running Servers

We have created a RightScript that you can execute on any running instance to remediate the vulnerability without rebooting. The script is idempotent (safe to run multiple times) and it verifies that the vulnerability has been patched before exiting with success.

This script has been tested with CentOS, RHEL, and Ubuntu RightImages from the v13 lineage. It should work with any yum- or apt-based RightImage, may work with other systems, and should not harm systems in any case.

There are two versions of this script:

Note: This script tests for both known vulnerabilities, but bash is being audited and more disclosures may be forthcoming. To address potential unknowns, we have added a $FORCE option to the scripts, which will force the package update to run regardless of the vulnerability test output. You can import the latest revision (Rev 4 for 12H1, v13 and 14 and Rev 5 for 11H1) and set the $FORCE option to true which will force the package update to run anyway.

To use the auto-patching script, verify your instances' lineage, import the appropriate script from the library, and run it as an Any script on all affected instances.

Manually Patch Servers

To patch the server without using the RightScript as above, you can unfreeze the repositories from the command line by following these instructions:

  • Edit each file in your package manager's sources directory
  • for Ubuntu, Debian and other apt-based systems, this is usually /etc/apt/sources.list.d

  • for CentOS, RHEL and other yum-based systems, this is usually /etc/yum.repos.d

  • Find URLs in each file that point to rightscale.com and end with a date string such as 20140307

  • Replace all occurrences of date string with latest

  • Optional: you can choose to only modify repositories that are update-related (contain security or update in the filename or URL).

After following these steps, follow your OS vendor's installation instructions to apply the update. Some sample references below.

Patch Newly-Launched Servers

We strongly recommend updating your ServerTemplates as follows:

  • Re-freeze all repositories to latest or 20140926
  • Ensure that your ServerTemplate has an apply updates script or recipe in its boot sequence

If you cannot unfreeze repositories, then we recommend adding the automatic patching script to your ServerTemplate's boot sequence.

After you have updated your ServerTemplates, make sure to commit a new revision if necessary and update the next of every server and array to use the new revision.