How to Manually Update Bash to Patch Shellshock Bug on Older Fedora-Based Linux Systems

With the announcement of the Shellshock Bash Bug, Linux admins around the world have been scrambling to patch their Bash shells so that they’re no longer vulnerable to the exploit. If you have a Fedora, RHEL, or CentOS system that hasn’t reached End-Of-Life, then updating to a patched version of Bash is as simple as:

sudo yum update -y bash

But what if you have a system running Fedora 12, Fedora 13, Fedora 14, Fedora 15, Fedora 16, Fedora 17, Fedora 18, or Fedora 19… or even RHEL/CentOS 3 or RHEL/CentOS 4, or an older SUSE Linux box? I have a Fedora 12 box I keep around for testing, and an updated version of Bash isn’t available in the repos. In that case, you can actually download the Bash source, manually apply all the patches, and compile and install it manually. It’s not as hard as you think! In fact, check out the comments for reports of people successfully patching all the way back to Fedora 4 using this method!

Ubuntu users: There’s actually no reason this approach won’t work with any Linux-based OS, including Ubuntu. You just need to use your system’s package manager (which is apt if you’re using Ubuntu) to install any required packages you’re missing in Step 3. But if you don’t need to install any additional packages, then these instructions will work just fine on Ubuntu, too.

Before You Start

It’s likely you’ve already done this step before coming here, but here’s how to check to see if you’re vulnerable to the Shellshock Bug. Check your vulnerability by running the following tests from your shell:

Exploit 1 (CVE-2014-6271)

Running the following command in a shell.

env x='() { :;}; echo vulnerable' bash -c "echo this is a test"

If you see “vulnerable” you need to update bash. Otherwise, you should be good to go.

Exploit 2 (CVE-2014-7169)

Even after upgrading bash you may still be vulnerable to this exploit. Try running the following code.

cd /tmp; env X='() { (a)=>\' bash -c "echo date"; cat echo

If the above command outputs the current date (it may also show errors), you are still vulnerable. If it just spits out the word “date,” then you’re fine against this exploit. It’s possible after running this test that you’ll have new file named “echo” in your /tmp directory. If that’s the case, it’s fine to delete it. IMPORTANT: Some of the tests floating around out there for this exploit (including the one I used to reference) include the command “rm -f echo” and could potentially delete the /bin/echo file on your system… which (ironically) is required when building a patched version of Bash. Don’t attempt any tests that include “rm -f echo.” If you did accidentally remove your /bin/echo file, you can restore it by searching for an archive of the core-utils RPM package for your particular system, then use “rpm -iv –replacepkgs” when installing it to force it to re-install and restore the deleted /bin/echo file.

Exploit 3 (???)

Here is another variation of the exploit.

env -i X=' () { }; echo hello' bash -c 'date'

If the above command outputs “hello”, you are vulnerable.

Exploit 4 (CVE-2014-7186)

bash -c 'true <<EOF <<EOF <<EOF <<EOF <<EOF <<EOF <<EOF <<EOF <<EOF <<EOF <<EOF <<EOF <<EOF <<EOF' || echo "CVE-2014-7186 vulnerable, redir_stack"

If you see “CVE-2014-7186 vulnerable, redir_stack” as part of the output, you are vulnerable to this exploit.

Exploit 5 (CVE-2014-7187)

(for x in {1..200} ; do echo "for x$x in ; do :"; done; for x in {1..200} ; do echo done ; done) | bash || echo "CVE-2014-7187 vulnerable, word_lineno"

If you see “CVE-2014-7187 vulnerable, word_lineno” as part of the output, you are vulnerable to this exploit. IMPORTANT: Keep in mind that as new Bash exploits are being discovered, new patches are being released by the Bash developers. For example, this article was first published on Sep 26, 2014, and additional patches were released within 2 days (which are now included in the instructions below). Still another patch was released on October 1, 2014. I will continue to update this post as new patches are available. Keep in mind that if the current patches don’t fix all the above known vulnerabilities on your system, you are still better off patching to the highest level possible, and then re-patching later (by simply following these instructions again) when new patches are released.

Step 1: Back up your existing Bash binary

Find out where your existing bash binary is located on your system with:

which bash

You’ll get a response like:

/bin/bash

Backup that file with:

sudo cp /bin/bash /bin/bash.old

Step 2: Determine which version of Bash you’re running

Now you’ll need to determining which version of bash your system is running. If you’re running Fedora 12, for example, it’s probably version 4.0. You can find out your version with:

bash --version

which will spit out a version number that looks something like this:

GNU bash, version 4.0.1(1)-release (i686-redhat-linux-gnu)

The first two numbers in 4.0.1 means you’re running Bash version 4.0. Remember your version number, as you’ll need that later. The third number in 4.0.1 can be a bit confusing, because it will mean something different based on where it came from. In this example, because “redhat” appears in the description, the third number is the build number in the RedHat repositories. However, if your output looked like this:

GNU bash, version 4.0.45(1)-release (i686-pc-linux-gnu)

the fact that it says only “pc” in that part of the description means Bash was manually compiled on your system (probably by you) and so in that case, the third number refers to the patch level of Bash version 4.0. Your goal at the end of this post is to have Bash report a version that looks like this, with the highest possible patch level. Do not confuse the Bash version with the patch level! Your goal during this fix should be to keep your version number the same, but increase your patch level to the highest possible number. A higher version number doesn’t necessarily mean you’re protected. Being patched to the highest patch level, regardless of your version number, is what you care about in this case. Finally, as you move through the following steps, resist the urge to move to a newer version number of Bash versions… as you’ll probably end up causing more problems than you’ll fix. Patching your current version of Bash is the best option to ensure things keep working on your system.

Step 3: Set up your fix environment

Whenever I’m working with source code on a Linux box, I like to keep everything in the /usr/local/src directory. So create a new subdirectory for fixing bash, and then jump into that directory, with:

mkdir /usr/local/src/bashfix
cd /usr/local/src/bashfix

You should also make sure you have a few required packages that will come in handy later (like patch, byacc, bison, autoconf, etc.) do:

sudo yum install patch byacc textinfo bison autoconf gettext ncurses-devel

Step 4: Download the Bash source

Locate the matching source code for the version of Bash you’re already running on the GNU.org FTP server. Since my test system was using 4.0, that’s what I’ll download in this example, but you should obviously download the one that’s appropriate for your system. Again, resist the urge to upgrade to a newer version (such as 4.1, 4.2, or 4.3 in this example). This can potentially create serious problems. Just stick with what you’ve already got for now. Download and extract the appropriate Bash source code into your fix directory with:

wget https://ftp.gnu.org/pub/gnu/bash/bash-4.0.tar.gz
tar zxvf bash-4.0.tar.gz

You should now have a new sub-directory containing the bash source. In this example, that directory is /usr/local/src/bashfix/bash-4.0. Move yourself into the newly extracted bash source code directory with:

cd bash-4.0

Step 5: Download and Apply the Patches

If you check the GNU.org FTP server where you downloaded the source code, you’ll also see a few sub-directories for each major version that contain all the patches for that version. Different versions of Bash have a different number of patches. In our example, the patches are located in https://ftp.gnu.org/pub/gnu/bash/bash-4.0-patches/. Checking that directory (as of Oct 5, 2014) shows a total of 44 patches for version 4.0, from bash40-001 to bash40-044. Your first option is to download the first patch, apply it to the source code, then download the second patch, apply it to the source code, and so on. Don’t do this just yet, because I’m going to show you a faster way to do it. But you should at least understand what’s happening before you automate it. The command you’d use to download the first patch and apply it in a single step would be (again, don’t do this… it’s just for illustration):

curl https://ftp.gnu.org/pub/gnu/bash/bash-4.0-patches/bash40-001 | patch -p0

That command uses curl to download the patch 001, then pipes the downloaded patch into the patch command, which patches the source code in that directory (you can check the patch man page for more details, if you want). If you did this manually, you’d have to repeat this command for each individual patch file, changing the 001 to 002, and then again changing it to 003, and so on until you reached the final patch. But my buddy Steve Cook helped me write a script (I stored it on GitHub as a Gist) that will automate all the patching for you. You just need to tell it the bash version you’re patching, and the total number of patches available for that version. Check it out:

Make sure you’re in the Bash source code directory you extracted, then download the “raw” version of the bash-multipatch.sh script we wrote with:

wget https://gist.githubusercontent.com/stevejenkins/3d64d3543060c1bcac92/raw/1ab592f5c8b584e9a0debf8e2ccbcac50cbf6e73/bash-multipatch.sh

Edit the file with your favorite text editor and set the version, nodotversion, and lastpatch variables in the script to the appropriate values for your situation (the nodotversion is simply the version number of bash without a dot in the middle). In our example, the variables are 4.0 (because we’re using Bash 4.0), 40 (same as the version without the dot), and 44 (since there are 44 total patches available for this version of Bash). Depending on your version, the number of patches will be different.

I do my best to stay on top of this issue, but It’s possible that even more patches are available in the patches directory before I’ve had a chance to update this article. You should always set the lastpatch variable in the script to the last patch you see in the directory to ensure the highest level of vulnerability protection. Save your edited file, then make it executable with:

chmod 755 bash-multipatch.sh

Now run it inside the Bash source code directory with:

./bash-multipatch.sh

Depending on your connection speed, it shouldn’t take very long to download all the patches and apply them. You’ll see each download and patch happen on your screen as the script runs. Keep an eye out for any error messages. The very last one should look something like this:

https://ftp.gnu.org/pub/gnu/bash/bash-4.0-patches/bash40-044
 % Total % Received % Xferd Average Speed Time Time Time Current
 Dload Upload Total Spent Left Speed
102 3882 102 3882 0 0 10438 0 --:--:-- --:--:-- --:--:-- 53178
patching file builtins/evalstring.c
patching file parse.y
patching file shell.h
patching file patchlevel.h

You can verify your source is patched to the level you want with:

cat patchlevel.h

If you look near the end of that file and see #define PATCHLEVEL followed by the last patch available for your version, then your source is patched to the highest level and should address the Shellshock Bug. Now you’re ready to build the patched version of your bash binary.

Step 6: Build and Install your Patched Bash Binary

It’s best if the “configure” and “make” steps in this section are performed as a regular, non-root user. However, on particularly older systems, if you’re getting errors other than missing dependencies when running “configure,” you may just have to do them as root.

In the source code directory, do:

./configure

You’ll see your system check to make sure everything is ready for your build. If you don’t see any errors, go ahead and make the new binary with:

make

Then test with:

make test

When everything is done you should be able to do this command:

ls -la bash

And you’ll see a newly build Bash binary with a timestamp of just a few seconds ago, like this:

-rwxrwxr-x 1 root root 2273014 2014-09-28 08:37 bash

Now copy the new binary to where your old bash binary was located in Step 1 (which is almost certainly /bin/bash) with:

sudo cp -f bash /bin/bash

Step 7: Test Your Fix

Now that you’ve manually downloaded, patched, compiled, and installed a new bash, you should test it to make sure you’re no longer vulnerable. Make sure your current shell session is using your newly compiled bash by simply running the new location from the command line. In this example, that would be:

/bin/bash

First, check to make sure you’re running the newly compiled version with:

bash --version

The output should look like:

GNU bash, version 4.0.44(1)-release (i686-pc-linux-gnu)

The 4.0.44 means you’re running Bash 4.0 at patch level 44. If that was the highest patch you applied to your source, then you are running the version you just built. Now run the vulnerability tests at the top of the article again. As I stated earlier, it’s possible that patches aren’t available to address all of them yet, but you should still patch to the highest level available, and then check back frequently to see if newer patches are available. Also, make sure you log out of any current shell sessions, and log in again using your new shell.

UPDATE! Step 8: Check For (and Kill) Any Old Bash Binaries in Memory

My buddy and fellow Linux sysadmin Todd Lyons emailed me a few days after I wrote this post, and informed me that (particularly on Fedora-based systems) he’d discovered that the old vulnerable Bash binary might still be in memory — even after building and installing a patched version. When I tested his theory on my system, I found he was absolutely right. So once you’re finished with this patch, read my post on how to prevent your vulnerable Bash binary from being “resurrected.”

If this article was helpful to you, I’d appreciate you sharing or tweeting it! 🙂

Congratulations on a successful fix! As always, I welcome your questions, comments, and feedback below!

UPDATE #2: Fully Automated Bash Update Script

A generous reader of my blog named Mike Marino was kind enough to write a script based on the steps in this article to cleverly automate the manual Bash update process (including the patching functions of the bash-multipatch.sh script). I’ve created a Gist of the script he emailed to me, and named it bash-autopatch.sh. It essentially follows all the steps in this post, including figuring out your current version of Bash and which patches you need. Mike reports that he’s “been using it to resolve Shellshock on hundreds of hosts which are stuck running officially unsupported versions of OSes.” It works great! I still recommend that you understand all the steps it’s doing before you pull the trigger, but if you’ve read this far, I’m assuming you already do.

When I replied to Mike to thank him, and ask if I could link to his blog or Twitter feed, his response was “I do not have a blog or anything. Thanks for the offer though. Just my way of giving back on all that you gave us. You can just mention a reader submitted it based off all your awesomeness.”

So thanks, Mike! Your script is awesomeness. 🙂