Python Pickle: Painless binary storage for Python objects

The pickle module provided with Python is so useful that I’m surprised I haven’t used it before. Pickle allows you to save an entire data structure (such as an object) to disk as a binary file in a effortless (and fairly efficient) manner. For example, in my latest project I have created a Monte Carlo simulation that can take quite a bit of time to run. I also need to make multiple runs to get statistics on the results. At the end of each run, I need to dump the resulting data to disk so that it can be read in later by an analysis program. If I had to write data in a format that could be interchanged with other scientific software, I’d use the hdf5 format with the pytables package. However, right now I just need to get something working, and the pickle module is perfect. Here is how I save an object called box:

pickleFileName = "Pickles/boxData_run" + str(run) + ".pickle"
pickleFile = open(pickleFileName, 'wb')
pickle.dump(box, pickleFile, pickle.HIGHEST_PROTOCOL)

First, open a file object for binary writing. Then use pickle.dump() to write the object to a pickle file. That’s all! To read in a pickled object, do the following:

pickleFileName = "Pickles/boxData_run.pickle"
pickleFile = open(pickleFileName, 'rb')
data = pickle.load(pickleFile)

The object is “reconstituted” exactly as it was saved–with methods, class and object data, etc. This probably isn’t ideal for a “finished” application written in Python, but it sure works well for the quick and dirty apps that I am always writing.

4 thoughts on “Python Pickle: Painless binary storage for Python objects”

  1. Pingback: The Python configparser: a way to read simple data files | as through a mirror dimly

  2. Pingback: Storing large Numpy arrays on disk: Python Pickle vs. HDF5adsf | scientific computing, modeling, and simulation

  3. Pingback: Storing large Numpy arrays on disk: Python Pickle vs. HDF5 –

Leave a Comment

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.