Linux (CentOS 7) - Increasing Available Entropy using Software Tools

If you are running repeated Monte Carlo simulations on Linux that use a kernel random device to seed a random number generator (like, say, generating a large number of pseudo-random lottery tickets), you may run out of available kernel entropy.  This may result in less random values being used in your simulation (if using /dev/urandom), or your application may pause entirely (if using /dev/random, which blocks if the pool runs out), which is undesirable. 

You can view the amount of available entropy (in bits) by entering the following from a command prompt in Linux: cat /proc/sys/kernel/random/entropy_avail .  The exact number of available bits will constantly vary.  If you're running repeated simulations that seed from a random device, in my experience the available entropy should remain consistently above 1000 bits, otherwise you're going to run into issues with the entropy pool being exhausted.

My Linux box is a freshly-installed CentOS 7 virtual machine running in VMWare Player.  The VMWare Tools drivers appear to be very efficient at abstracting/accelerating system device access.  While this makes the UI and overall system run smoother, this has a side effect of low available entropy in the kernel (entropy in the kernel is generated in part via interrupts and timings from device access like accessing disks).  On my system without modification the output of cat /proc/sys/kernel/random/entropy_avail is consistently only 100-200 bits, which can be a problem.

The best option to remedy this is to use a hardware random number generator (hwrng) to fill the kernel entropy pool, such as this USB key.  This will provide "true" random input.  However, like me, if you don't want to go out and buy something, and you are not depending on absolute randomness for cryptographic applications, there are a couple of software options that are "random enough" for Monte Carlo simulations.  By this I mean the entropy pool gets filled, and repeated runs of the testing utility rngtest on the kernel random devices (such as by running the command rngtest -c 2000 < /dev/urandom) consistently return only a couple of FIPS testing failures, which is normal and a generally acceptable measure of "random".  Note: there are some online sources of true random bits for seeding, such as random.org and HotBits, however, if you running repeated Monte Carlo simulations and seeding the application's prng frequently, you will likely exceed these services' daily quotas.

In a nutshell, I implemented haveged, as well as rngd set to use /dev/erandom (from the "frandom" package) as its rng device.  Now the available entropy is consistently above 2000 bits.  This begs the question - once frandom is installed, why not use /dev/frandom or /dev/erandom directly as the PRNG seed in my application, instead of using rngd to add /dev/erandom to the kernel random seed and using /dev/urandom to seed my application?  Well, using rngd with /dev/erandom *adds* the output of /dev/erandom to the kernel random device.  So, as configured, the kernel random device is now using its default entropy collection routine (which, as I'm using a system with a 4th generation Intel i3 processor, also includes the hardware RDRAND RNG component), haveged, and /dev/erandom, instead of just /dev/erandom itself.  Theoretically, this method is "more" random than just using /dev/erandom itself.

Now, on to the implementation.  As indicated, my environment is CentOS 7, so some of these options may be different or not required in other distributions/versions of Linux.  I am documenting this because CentOS 7 in particular changed the fundamental way some components work, particularly the move to "systemd"-style startup that doesn't run rc.local anymore by default, and the user management is a bit different/more locked down.  None of these changes are bad, just setting certain things up is no longer straightforward or at least wasn't obvious to me.

Note:  I like nano as a text editor (the open-source clone of the pico editor).  It's small, simple (you don't need to enter a key combo to edit a file once it's open, and to save and close a file it's a simple ctrl-W, enter).  I believe nano is installed by default with most CentOS installation types - this is easy to test - simply enter nano at a command prompt - if it doesn't open and you get a "command not found" message, type sudo yum install nano at the command prompt to install it.

Note 2:  When I say "from a command prompt", I mean a terminal window, opened from Applications > Favorites > Terminal.

Prepare the user account

Some of the processes that follow will require "root" or superuser access.  CentOS 7, like many modern *nix variants, really, really, really doesn't want you to log in as the "root" user.  It forces you to create a non-root user.  If you select the "Make administrative user" checkbox for this new user, it adds this user account to the "wheel" group (which is the administrative group).  However, and this is a pet peeve of mine with Linux/unix security in general, this is deceptive - even as a member of the wheel group, you still need to use the sudo command with most commands that touch a system folder or update a kernel parameter/module.  In addition to being annoying, it causes a problem: by default, when invoking sudo somecommand, sudo will prompt you for a password, which makes running shell scripts that include sudo as a non-root user a problem (the sudo command doesn't like passwords that are echo'd to it - I tried that - likely this is a security measure).  As this process involves running a shell command that needs to be sudo'd, we need to change this behaviour.

(Side note:  before trying the procedure below, I tried an older *nix trick of editing the /etc/passwd file and changing the UID and GID fields of the newly-created user to "0" so that the system would be fooled into thinking you are logged in as "root" when actually logged in as this new user.  This completely hosed the user authentication mechanism in CentOS 7 - on reboot it wouldn't even load the login page and I couldn't get to a shell prompt.  I had to reinstall the OS at that point.  Lesson learned.)

While you can't eliminate using sudo as a non-root user, you can get sudo to not prompt for a password, which makes it suitable for scripting.  To do this:

1. Open the /etc/sudoers file by typing sudo nano /etc/sudoers .
2. Scroll down in the file to the line that says %wheel ALL=(ALL) ALL . Add a # in front of the %wheel to comment it out, so the line now looks like #%wheel ALL=(ALL) ALL .
3. Scroll down to the next parameter line that says #%wheel ALL=(ALL) NOPASSWD: ALL . Remove the #, so the line now says %wheel ALL=(ALL) NOPASSWD: ALL .  
4. Enter ctrl-W, then Enter to save and close the file.
5. Log off and log back in.  Test the sudo functionality by entering sudo nano at a command prompt.  You should not be prompted for an additional password.  If you are, reboot and try again.  If it still does not work, review steps 1-3.

Enable Execution of rc.local

By default in CentOS 7, rc.local is not executable, as the move has been made to a "systemd"-style of startup.  However, simply adding a command to rc.local is much more convenient for 1-off items, like the loading of the frandom module (dealt with later).

To enable execution of rc.local, at a command prompt, enter:

• sudo chmod +x /etc/rc.d/rc.local . 
• sudo systemctl enable rc-local.service (you will see an error after this indicating "The unit files have no [Install] section. They are not meant to be enabled using systemctl." - this is OK.  If you see any other error, there is another issue that will require further troubleshooting.)

Install haveged

Haveged attempts to gather more entropy than the kernel itself does, looking more in depth at user-based I/O.  I've found it increases available entropy a bit in a virtualized environment, but the frandom method below is still necessary to get a substantial increase in available entropy.

At time of writing, there is no prepared package for haveged for CentOS 7 (ex, sudo yum install haveged comes back with no package found).  However, it is easily to compile and install it from source.  Open a web browser and navigate to http://www.issihosts.com/haveged/downloads.html . Download the latest version (1.9.1 at time of writing).  Extract it somewhere convenient.  In a command prompt, navigate to the folder where it was extracted.  Enter the following commands:

• sudo ./configure
• sudo make
• sudo make install

Install frandom

Download the frandom package from http://billauer.co.il/download/frandom-1.1.tar.gz .  Extract it somewhere convenient.  Following the instructions in the included INSTALL file, you need to open a command prompt, navigate to the directory you just extracted the files to, and run:

• sudo make
• sudo install -m 644 frandom.ko /lib/modules/`uname -r`/kernel/drivers/char/
• sudo depmod -a
• sudo cp 10-frandom.rules /etc/udev/rules.d
• sudo /sbin/modprobe frandom (at this point there shouldn't be an error; if there is, it will need to be troubleshot further)

Note: This kernel module is compiled against the current kernel headers.  So, if you run updates in the future, and your kernel version gets updated (including minor revision numbers), you will need to run these commands again to recompile and install the module again.

Once the frandom module is loaded, you can test to make sure you are getting random output.  From a command prompt, enter head -c 100 /dev/frandom  and head -c 100 /dev/erandom.  You will see random characters appear (the terminal window may try to interpret these as different language characters and may prompt to install character sets - just click cancel).

(Side note:  The frandom package installs both /dev/frandom and /dev/erandom kernel devices.  From the webpage, /dev/frandom takes a small random seed from /dev/random, and then uses an RC4 algorithm to generate further random output.  /dev/erandom also uses the RC4 algorithm, but uses an internal PRNG to generate the seed - so effectively /dev/erandom does not reduce available entropy at all. /dev/erandom also fairs well with the rngtest utility for randomness.)

To make frandom load automatically on boot, the rc.local file needs to be modified.  Enter sudo nano /etc/rc.d/rc.local .  Add the line /sbin/modprobe frandom  to the bottom of the file.  Ctrl-W, Enter to save and exit.

Verify rng-tools is installed

The package rng-tools contains rngd, which we need.  From a command prompt, enter sudo yum install rng-tools .  It will advise if it is already installed, or will install it if it's not.

Create a shell script to launch haveged and rngd

For some reason, rngd does not work with /dev/erandom if launched from rc.local (but rc.local is definitely being executed, as the modprobe command added above gets executed).  For an unknown reason it simply does not add anything to the available entropy when launched this way.

Launching rngd with the appropriate options does function correctly when run from a local shell script that is run when the user logs in (available entropy is increased).  I have no idea why this way works and the rc.local method fails (using the exact same command, except prefaced by "sudo" when run as the user).  I also added haveged to this script for convenience.

Create the shell script by entering sudo nano /var/rng.sh.  Add the following lines: 

sudo /usr/local/sbin/haveged --run=0

sudo /usr/sbin/rngd --no-tpm=1 --rng-device=/dev/erandom

Save and close (ctrl-W, Enter).

Make the script executable: chmod +x /var/rng.sh .

Make this script run automatically on user login

Navigate (in the GUI) to Applications > System Tools > Startup Applications.  Click Add, give it a name (like "rng1"), and enter the following into the Command box: gnome-terminal -e /var/rng.sh .

Log off, log back in.  If all worked correctly, you can run cat /proc/sys/kernel/random/entropy_avail again, and it should come back with a larger number of bits (usually somewhere between 2500 - 4096 - the max is 4096).

You can now run your simulations as needed and you won't run out of available entropy.