I run XWindows on a CentOS 5.9 box as a non-root user. To perform administrative tasks, I occasionally log in as root using “su” in a terminal window. However, I found that I was unable to start any GUI applications as root. I got the following error:

Xlib: connection to ":0.0" refused by server
Xlib: No protocol specified
E233: cannot open display Xlib: connection to ":0.0" refused by server
Xlib: No protocol specified

This can happen when xhost isn’t configured to allow local, non-network connections. As the user who started the xwindows session (not root) run:

I recently found the following message in the system logs for one of the compute nodes in the STOKES cluster:

init: Id "co" respawning too fast: disabled for 5 minutes

This caught my attention, because this OS image on this particular node should be identical to the image that is deployed on the rest of the nodes in the cluster. Why was it the only one producing this strange warning message? I searched the web and learned that the following line in /etc/inittab is causing the warning:

GROMACS produces graphical output in the form of .xvg files. These are designed to be viewed with a classic UNIX/Linux plotting program called Grace. If you happen to be using Linux and you have Grace installed, it is very easy to plot the data with the command

xmgrace my_file.xvg

If you aren’t using Linux, plotting .xvg files is quite a bit more difficult. Gnuplot is the only free plotting program that I have found that can handle .xvg files. Gnuplot is available for Windows (there is a direct download link near the top of the download page), but I will caution you that Gnuplot is not easy to learn. It has a command-line interface, and there are no point-and-click options to do basic operations such as labeling the axes of the plot. If you are using Gnuplot, you can plot the contents of an .xvg file with the command:

Most of the time, RPM (especially in conjunction with yum) is a decent package management solution. However, I can think of two common circumstances when you don’t want to let RPM install a package:

You don’t have root permissions on a system such as a shared cluster You are an administrator on a shared cluster and you can’t risk having a package over-write system-critical files

One approach is to extract the files from the RPM package and install them manually. I used this method to install FDTD Solutions from Lumerical on the STOKES Linux cluster. The application is distributed as an RPM package. I never install a third-party RPM as root, because a badly constructed package might over-write a system-critical file that some user is depending on. I downloaded the appropriate TAR file from Lumerical, uncompressed it, and looked at the contents. There is a simple script called install.sh which checks to make sure the user is root and then tries to install a hardware key driver and an RPM that contains the FDTD software. Here are my recommendations for installing Lumerical products on a cluster:

I found an interesting quirk when trying to build an OpenMPI application on a visualization node with a “stock” version of Red Hat Enterprise Linux 5.8.  I used mpicc to compile the application and got the following error:

$ mpicc hello_world_mpi.c -o hello_world
/usr/bin/ld: cannot find -lnuma

This was rather surprising, since this node mounts a directory via NFS that contains OpenMPI and Intel Composer 2013, and these applications are known to work on other nodes. To find the source of the problem, I used the showme option to see the command that is actually run by mpicc:

Since Python is widely used as a high-productivity language for scientific computing, Intel has created a page showing how to build NumPy with Intel compilers and the Math Kernel Library (MKL). I would like to clarify a few items regarding building NumPy on a 64-bit Red Hat Enterprise Linux 5.4 system. Since this is a production system, I don’t want to replace the Python 2.4 binary -2.7.3-intel-composer-2013that ships with RHEL 5.4. Instead, I created a directory called

This article from Admin Magazine has a handy list of profiling tools.

This post will take you through the installation and configuration of an Infiniband card on a server running Red Hat Enterprise Linux 5.4.  These steps are applicable to any version of Red Hat 5, and will probably work with version 6 as well.  It has been surprisingly hard to find all of these steps in one document.

Required packages

openib-1.4.1-6.el5.noarch libibverbs-1.1.3-2.el5.x86_64 libnes-0.9.0-2.el5.x86_64 libibumad-1.3.3-1.el5.x86_64 opensm-libs-3.3.3-2.el5.x86_64 swig-1.3.29-2.el5.x86_64 ibutils-libs-1.2-11.1.el5.x86_64 ibutils-1.2-11.1.el5.x86_64 (provides ibdiagnet and others) opensm-3.3.3-2.el5.x86_64 libibmad-1.3.3-1.el5.x86_64 infiniband-diags-1.5.3-1.el5.x86_64 (provides handy tools like ibstat and ibstatus) libibverbs-utils-1.1.3-2.el5.x86_64 (provides handy tools ibv_devinfo and ibv_devices) libibverbs-devel-1.1.3-2.el5.x86_64

Glossary of Infiniband Terminology

GID: Global Identifier GUID: Global Unique Identifiers (also known as Direct Address) HCA: Host Channel Adapter LID: Local Identifier TCA: Target Channel Adapter SM: Subnet Manager

Infiniband: The Host Perspective

Every host on an Infiniband fabric has three identifiers: GUID, GID, and LID. A GUID is similar in concept to a MAC address because it consists of a 24-bit manufacturer’s prefix and a 40-bit device identifier (64 bits total). The Global Identifier (GID) is a 128-bit identifier similar to an IPv6 address (technically, a GID is a valid IPv6 identifier with restrictions). The GID consists of the 64-bit GUID plus an additional 64-bit EUI-64 identifier, for a total of 128 bits. The GID is used for routing between subnets. The default GID prefix is 0xfe80::0. Finally, there is the local identifier (LID), which is assigned by the subnet manager. The LID is a 16-bit identifier that is unique within a subnet. Hosts have an LID between 0 and 48,000, usually expressed in hexadecimal notation (such as 0xb1). Routing within a subnet is managed by LID. The GUID, GID, and LID for a Linux server are stored in text files. The exact path to this text file will depend upon which Infiniband driver is used on the system. For a system with the MLX4 driver (such as RedHat/CentOS 5.x), the commands are: