Posts
Deploying Python applications on Windows
Writing applications in Python on a Linux system is almost too easy. Deploying Python apps on other Linux systems is not hard, because most Linux systems already have Python, with its core libraries and tools, installed. Most Linux systems also have package managers that make it easy to find and install required components. But, what happens when your co-workers who use Windows need to use your app? When you tell them to “go to the command line and…” you’ve pretty much lost them at “command line.” How do you package Python in a way that’s easy for a Windows user to install? Here is a process that worked for me:
Scipy.integrate ODEPACK import error solved!
I recently found a solution to a problem that had been vexing me for about a year. In order to successfully import anything from scipy.integrate, I had edit the file scipy/integrate/__init__.py and comment out the line
from odepack import *If not, I would get various import errors such as
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.5/site-packages/scipy/integrate/__init__.py", line 10,
in <module>
from odepack import *
File "/usr/lib/python2.5/site-packages/scipy/integrate/odepack.py", line 7,
in <module>
import _odepack
ImportError: /usr/lib/python2.5/site-packages/scipy/integrate/_odepack.so:
undefined symbol: daxpy_I found the solution in Gentoo bug 251165. The problem only occurs when Scipy is built with non-reference versions of BLAS and CBLAS. The solution is to install the reference implementation of BLAS and CBLAS, rebuild Scipy, and then use whichever implementation of BLAS or CBLAS you want. I don’t know why this happens, and I don’t know if it affects distributions other than Gentoo. By the way, Gentoo has a really useful system tool called eselect, which has various modules that are used to choose between different versions or implementations of tools on your system (BLAS, Java virtual machine, kernel sources, OpenGL, etc.)
Optimizing Python code for fast math
I spent some time today profiling a Brownian dynamics simulation written in Python to see how I could make it faster before starting some long runs on a Linux cluster. In the sections below, I have attempted to quantify the speed improvements due to various changes. Keep in mind that the actual speed improvement in your code will vary, depending on where the actual bottlenecks occur. See my post about profiling Python code. Another caveat: I am running Python 2.4.4 because it’s installed on our cluster.
Redirecting standard output from Python: another example
I wrote a previous post about how to redirect standard output from a Python script to a GUI window. In this post, I will give an even simpler example of to redirect standard output to a log file. During the early development and debugging of Python programs, I use print statements to keep me informed of what’s happening. However, printing to the terminal is not always practical–for example, when I run numerical code on a parallel cluster, there is no way to determine which output came from which instance of the program. Here is a class that you can use to redirect standard output to a log file:
Tools for Python software development
I have found a few tools over the years that I find extremely useful for developing software. Python is my language of choice at the moment, but I’m sure these tools will be handy for any language.
- Subversion is an open-source version-control system. Version control was designed to allow multiple programmers to work on the same project at the same time without stepping all over one another. However, even though I am a solo developer, I find version control to be extremely helpful.
- When I commit changes to the repository, I can document what I’ve changed and why. This is a great help when I introduce a bug and have to go back and find it.
- The repository is stored on a remote server that is backed up nightly.
- It’s easy to make an “unstable” branch for implementing new features. When I make changes that don’t work, it’s easy to revert to a previous version that works.
- It is easy to deploy my code to the Linux cluster and make sure that the cluster is running the latest version of my software.
Go read the documentation on the Subversion web site to find out what it can do for you.RapidSVN is a GUI client for a Subversion server. By default, Subversion comes with a command-line client that does everything you need. However, sometimes it’s easier to stay organized when everything is presented visually. Here is a screenshot of RapidSVN: RapidSVN screenshot
A lookup table for fast Python math
Numerical programming frequently requires the use of look-up tables. A look-up table is a collection of pre-computed values. When given an “x” value, the table returns a pre-computed “y” value. Look-up tables can be used to speed up numerical codes, when it is faster to look up a value in the table than it is to compute the value. They are also used when the data in the table cannot be computed–for example, experimental data or averaged results from an ensemble of Monte Carlo simulations. Another application is to compute a value when a function cannot be solved algebraically. Assume that you have a formula for a function q(h). You need the value of h for a given value of q, but the formula cannot be algebraically solved to get h(q). Instead, choose a range of h values, compute the function q(h), and store each value in a look-up table. Now you can get h(q) for any value stored in the table. The major limitation of a look-up table is that it cannot return valid results for any value of q which is outside the range of those stored in the table. Depending on its implementation, the table may be able to interpolate to return values between known points.
Update 2: building 64-bit Numpy with Intel compilers and MKL
NOTE: these instructions are obsolete. Please see Building NumPy on a 64-bit Red Hat Cluster with Intel MKL. In a previous post I described how I built Numpy with Intel compilers and the Math Kernel Library on a 64-bit cluster. Today I upgraded to Numpy-1.2.1 and I made a few improvements to my install process. Please read the previous post, since I will not duplicate some important information, and then read on. This time, I made use of a site.cfg file. Copy the file site.cfg.exampletosite.cfg and edit. At the end of the file, uncomment the [mkl] section and set the path to your library. Mine looks like:
Calculating the Pair Correlation Function in Python
The pair correlation function, also known as the radial distribution function, is a way to characterize the distribution of particles on a two-dimensional plane or in a three-dimensional space. Please check out Eric Weeks’ web site for an introduction to pair correlation functions. He has written some routines in IDL to compute these functions. Using his foundation, I have written some simple routines in Python to compute 2D and 3D pair correlation functions.
Visualizing trajectories with Python, VMD, and .vtf files
As a computational scientist who develops Brownian Dynamics (BD) simulations, I have had a difficult time finding software to visualize trajectories from my simulations. There are a lot of software tools for molecular visualization, but the majority of these tools are designed to view “static” molecules that do not change over time. For storing the output of molecular dynamics (MD) simulations, this is logical, since the simulations often run for millions of time steps and it would be impractical to store (and view) all of them. The user is often interested only in finding the lowest-energy configuration, so all the earlier steps can be discarded. While this is also somewhat true for BD simulations, for debugging purposes it is helpful to visualize the trajectory of the particles.