Python Pickle: Painless binary storage for Python objects

The pickle module provided with Python is so useful that I’m surprised I haven’t used it before. Pickle allows you to save an entire data structure (such as an object) to disk as a binary file in a effortless (and fairly efficient) manner. For example, in my latest project I have created a Monte Carlo simulation that can take quite a bit of time to run. I also need to make multiple runs to get statistics on the results. At the end of each run, I need to dump the resulting data to disk so that it can be read in later by an analysis program. If I had to write data in a format that could be interchanged with other scientific software, I’d use the hdf5 format with the pytables package. However, right now I just need to get something working, and the pickle module is perfect. Here is how I save an object called box:

pickleFileName = "Pickles/boxData_run" + str(run) + ".pickle"
pickleFile = open(pickleFileName, 'wb')
pickle.dump(box, pickleFile, pickle.HIGHEST_PROTOCOL)
pickleFile.close()

First, open a file object for binary writing. Then use pickle.dump() to write the object to a pickle file. That’s all! To read in a pickled object, do the following:

pickleFileName = "Pickles/boxData_run.pickle"
pickleFile = open(pickleFileName, 'rb')
data = pickle.load(pickleFile)
pickleFile.close()

The object is “reconstituted” exactly as it was saved–with methods, class and object data, etc. This probably isn’t ideal for a “finished” application written in Python, but it sure works well for the quick and dirty apps that I am always writing.