In a previous post, I explained how to use the Python profiler.  The profile is great for finding out which parts of the code run the slowest, or are called most often.  However, the profiler doesn’t give any information about how much RAM is being consumed, or where it’s being consumed.  If your program needs so much memory that it starts swapping to disk, its speed can be reduced by orders of magnitude.  On the positive side, your code may run much faster if it fits entirely in the processor cache.  In this post, I will introduce two tools that can help you understand the RAM usage of your Python code.

It’s a real pain to surf the Web when the batteries die in your wireless mouse or trackball.  I use some old rechargeables that are no longer fit for digital camera service, so this happens to me fairly often.   My batteries just died again, so I had to figure out how to configure X.org to use a backup mouse.  It’s actually not hard to have an old 3-button mouse plugged into the PS/2 port and tell X.org to use it as a backup.  First, define two input devices:

My simulation library, which is written mostly in Python, needs a lot of data and parameters in order to run. In some cases, I just hard-code the values in the script that calls the library, and in other cases I load a pickle file containing a Python objext. What if I want to read in data or configuration parameters from a human-editable text file? If the information is extensive or complex, XML might be a good choice, but XML is overkill for simple configuration or data files. Fortunately, a standard Python library called ConfigParser has already defined a configuration file format, and provides methods to interact with such files. Here is a sample of the format used in a config file:

Writing applications in Python on a Linux system is almost too easy.  Deploying Python apps on other Linux systems is not hard, because most Linux systems already have Python, with its core libraries and tools, installed.  Most Linux systems also have package managers that make it easy to find and install required components.  But, what happens when your co-workers who use Windows need to use your app?  When you tell them to “go to the command line and…” you’ve pretty much lost them at “command line.” How do you package Python in a way that’s easy for a Windows user to install? Here is a process that worked for me:

I recently found a solution to a problem that had been vexing me for about a year. In order to successfully import anything from scipy.integrate, I had edit the file scipy/integrate/__init__.py and comment out the line

from odepack import *

If not, I would get various import errors such as

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.5/site-packages/scipy/integrate/__init__.py", line 10,
in <module>
    from odepack import *
  File "/usr/lib/python2.5/site-packages/scipy/integrate/odepack.py", line 7,
in <module>
    import _odepack
ImportError: /usr/lib/python2.5/site-packages/scipy/integrate/_odepack.so:
undefined symbol: daxpy_

I found the solution in Gentoo bug 251165. The problem only occurs when Scipy is built with non-reference versions of BLAS and CBLAS. The solution is to install the reference implementation of BLAS and CBLAS, rebuild Scipy, and then use whichever implementation of BLAS or CBLAS you want. I don’t know why this happens, and I don’t know if it affects distributions other than Gentoo. By the way, Gentoo has a really useful system tool called eselect, which has various modules that are used to choose between different versions or implementations of tools on your system (BLAS, Java virtual machine, kernel sources, OpenGL, etc.)

I have found a few tools over the years that I find extremely useful for developing software. Python is my language of choice at the moment, but I’m sure these tools will be handy for any language.

• Subversion is an open-source version-control system. Version control was designed to allow multiple programmers to work on the same project at the same time without stepping all over one another. However, even though I am a solo developer, I find version control to be extremely helpful.
  1. When I commit changes to the repository, I can document what I’ve changed and why. This is a great help when I introduce a bug and have to go back and find it.
  2. The repository is stored on a remote server that is backed up nightly.
  3. It’s easy to make an “unstable” branch for implementing new features. When I make changes that don’t work, it’s easy to revert to a previous version that works.
  4. It is easy to deploy my code to the Linux cluster and make sure that the cluster is running the latest version of my software.

Go read the documentation on the Subversion web site to find out what it can do for you.RapidSVN is a GUI client for a Subversion server. By default, Subversion comes with a command-line client that does everything you need. However, sometimes it’s easier to stay organized when everything is presented visually. Here is a screenshot of RapidSVN: