I gave a presentation at BarCamp Orlando on 5 April 2008. Here is a link to my presentation, Data Analysis with Python.

This post is a reminder to myself more than it is intended for anyone else.  Well, that’s most of this blog.  Gentoo is a great distribution of Linux, but you have to install everything yourself. This is a good thing, because you get a very lean system, but the downside is that you are re-inventing the wheel somewhat with every install. So, without further ado, here is my current favorite configuration: Global USE flags:X cups dbus hal jpeg png gs dvi gif tiff mpeg bzip2 gtk lapack tetex -gnome -kdePackage list (basic software):* dbus (message bus system)* hal  (Hardware Abstraction Layer) gentoolkit vim/gvim x.org xfce4 thunar mozilla-firefox acroreadPackage list (technical software): blas-atlas* lapack-reference numpy (unstable 0.90.1) matplotlib (Matlab-like plotting package for Python) texlive (experimental, hard-masked) lyx evince (dvi, ps, pdf viewer) paraview (3D visualization of datasets, unstable)

Minutes ago I uploaded a tarball containing my re-release of ODEPACK, a standard numerical tool for the solution of systems of ordinary differential equations. The algorithms and numerical code are identical to the distribution found on Netlib, but everything has been placed into an organized “package” with Makefiles, examples, documentation, etc. The code is set up to be compiled as a shared library that you can link to your code. Download and enjoy! And tell me if you’re using it!

In previous examples, I’ve shown how to pass Python lists into C++ using Boost.python. Because Python lists can contain a mixture of objects of different types, C++ has to use extract<type> to determine what kind of data to get from a list item. This approach works well, but it has its faults. In the C++ code, you might need to have duplicate data structures (a boost.python list to interchange data with Python and a C++ container such as a vector to process the data) and the code to convert between them. If you want to use your C++ library with languages other than Python, you want a non-Python-specific interface. Wouldn’t it be great to define your core data structures using a standard C++ container (like vector) and then access and modify them in Python? You can, but it’s non-obvious and it’s not well explained in the Boost.python docs. Read the docs for the indexing suite and take a look at my demonstration code for exposing C++ vectors to Python, and you should be able to figure it out. Once you get it working, it’s a very clean way of doing things. Just remember, because you’re working with C++ containers that accept only the specified type of object Python will throw a TypeError if you try to append the wrong kind of object.

Demonstration Code–mixing objects of different types in a Boost.python list In Python, a list is allowed to contain objects of various types. Using Python’s introspection capabilities, it’s easy to process a mixed list because you can just test each list member to see what kind it is, using isinstance(objectA, TypeB). This creates a problem when the list is passed into C++, where arrays and containers are designed to hold objects of one type. I encountered this problem when writing a small module to parse mathematical expressions. The expression is entered as a string in Python, which is parsed and “compiled” into a stack of objects. The compiled stack is passed to C++ using boost.python, where it is processed when needed. The stack consists of two or more fundamentally different objects: numerical constants and operators. The evaluation routine pops an object. If it’s a numerical constant, the value is placed on the operand stack. If it’s an operator, the appropriate operation is performed on the values at the top of the operand stack. With help from Alex Mohr and Christopher Woods on the C++-sig mailing list, I developed a method that is illustrated in the example. In C++, define a polymorphic base class (has at least one virtual method) and several classes that are derived from the base class. These classes are exposed to Python using Boost.python so that objects can be created in Python. A list of mixed objects is then passed back to C++ for processing. When trying to access a member of the list, C++ doesn’t know what kind of object to expect, so we have to use the Boost.python function extract<>() (see previous post for an introduction to extract). Now we extract a pointer to the base class using: object = extract <BaseClass*> (list[index]); This works because every object in the list is derived from the base class. As shown in the example, I first tried to do this with references (to avoid the whole pointer mess) but it did not work, so it seems that pointers are unavoidable here. It seems that this use of introspection is relatively rare in OOP, and is sometimes actively discouraged. However, for this type of problem it seems like the most logical design. Is there a different design pattern that would work better?

The US Patent and Trademark Office (USPTO) has launched a trial program to subject software patents to peer review online.  Naturally, the big software companies aren’t exactly publicizing this.  We all know that a lot of stupid patents get issued.  Well, stop complaining and start reviewing!  Let’s make this trial successful!

The documentation for the Boost.Python extractor class is not terribly enlightening for a new user. The examples given in the tutorial are code fragments, and it’s never clear exactly what the context is or what they are supposed to accomplish. That’s a shame, because the extractor classes are really cool and easy to use, so people should use them! Here is a Boost Python extractor example that should make things clear.

The Dapper-to-Edgy upgrade was surprisingly smooth, once I figured out how to do it. Unfortunately, the new version overwrote my “/boot/grub/menu.lst” so I could not boot Windows. I have enough experience with Linux that this was a five-minute fix, but it could be pretty scary for an inexperienced person. Who knows what else got over-written and I just didn’t realize it. I was further disappointed to find out that the version of PIDA in Edgy is broken.

When it comes to Linux, I’m a Gentoo guy. Go ahead, make fun of me and call me a ricer. Then go try to do something difficult on Redhat or Ubuntu, and come back and tell me what you think. I needed a Linux development environment at work, so I took my Windows machine and added a Linux partition. Since Ubuntu has been getting a lot of good press, and I wanted something that would install fast, I tried Ubuntu. Specifically, I installed Kubuntu, because Linus Torvalds doesn’t use Gnome. It really did install quickly and everything worked great out of the box. I installed a number of packages, and they worked too! Too good to be true? Absolutely! It all started when I wanted to compile something. Silly me, I thought gcc was a basic part of a basic Linux installation…but no, that must be a Gentoo bias, so I installed gcc. Then I decided to install numpy, the latest numerical toolkit for Python. It’s been around for a couple of years as a beta, so I figured there must be an “unstable” build out there. Nope. To get numpy, you have to upgrade from Ubuntu 6.06 “Dapper” to 6.10 “Edgy”. <sarcasm>Because it makes sense to upgrade gcc, KDE, Firefox, etc just to get a new Python package.</sarcasm> Now, go find instructions on how to upgrade the Ubuntu distribution. Then try those instructrions and realize that they won’t work at all for Kubuntu because the package managers are totally different. While you’re there, read the horror stories on the forums from people who followed the “official” upgrade instructions and ended up with less-than-functioning systems. FYI, the Kubuntu upgrade instructions are actually easy to find once you know where to look. Kubuntu 6.10 is installing right now, so tomorrow we’ll see how it went.