Debugging Python in ipython and ipdb

In this blog, I'm going to demonstrate some very useful features of ipython, especially when debugging python using ipdb; examining objects and step into pdb/ipdb from Python code, etc.

You should know how to use pdb to debug python, if not, please visit my other blog article.

I'll briefly describe how to exam objects and name space in iPython and then demonstrate how to debug Python in iPython/ipdb.

Note: a quick link to ipython. (make sure you come back here after reading the external page)

(*** Very Important ***)
If your ipython is 0.10 or earlier version, you need to do the following to fix a bug in checkline() method in IPython/Debugger.py.

shan@ub1:~/code$ python
Python 2.6.4 (r264:75706, Dec 7 2009, 18:45:15)
[GCC 4.4.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import IPython
>>> IPython.Debugger.__file__
'/usr/local/lib/python2.6/dist-packages/IPython/Debugger.py'

wget the updated Debugger.py from here and replace the above Debugger.py with the new one. That's it.


Let's start with examining object and name space in ipython:

1. Start ipdb from ipython: (a very useful way to debug python code)

First, let's start ipython

shan@ub1:~/code$ ipython
Python 2.6.4 (r264:75706, Dec 7 2009, 18:45:15)
Type "copyright", "credits" or "license" for more information.

IPython 0.10 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object'. ?object also works, ?? prints more.

In [1]:
The first few lines already show the most important tricks of ipython. The problem is most people rarely read it!


Now, let's import test code and see some of the features python IDLE doesn't offer.

shan@ub1:~/code$ ipython
Python 2.6.4 (r264:75706, Dec 7 2009, 18:45:15)
Type "copyright", "credits" or "license" for more information.

IPython 0.10 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object'. ?object also works, ?? prints more.

In [1]: import test2

In [2]: import sys

In [3]: import os

In [4]: who
os sys test2

In [5]: whos
Variable Type Data/Info
------------------------------
os module
sys module
test2 module

In [6]: test2?
Type: module
Base Class:
String Form:
Namespace: Interactive
File: /home/shan/code/test2.py
Docstring:



In [7]: test2??
Type: module
Base Class:
String Form:
Namespace: Interactive
File: /home/shan/code/test2.py
Source:
#/usr/bin/env python

class Employee:
def __init__(self, name, title=None, phone=5551212):
self.name = name
self.title = title
self.phone = phone

def employee():
shan = Employee("Shan Jing","UNIX ADMIN","2720748")
import pdb;pdb.set_trace()
joe = Employee("Joe Smith","Application Developer","7207837")
print (shan.name, shan.title, shan.phone)
print (joe.name, joe.title, joe.phone)


if __name__ == '__main__':
employee()

In [8]:


In the above, we typed 7 commands (3 imports, who, whos, test2? and test2??):

From the output, I'm sure you already know what each command does. But just in case you need to know the detail about these commands, type '%cmd?' in ipython. (see below). By the way, this was printed in the last line of the ipython's startup message - "object? -> Details about 'object'. ?? prints more details about the object."

In [9]: %who?
Type: Magic function
Base Class:
String Form: >
Namespace: IPython internal
File: /usr/local/lib/python2.6/dist-packages/IPython/Magic.py
Definition: %who(self, parameter_s='')
Docstring:
Print all interactive variables, with some minimal formatting.

If any arguments are given, only variables whose type matches one of
these are printed. For example:

%who function str

will only list functions and strings, excluding all other types of
variables. To find the proper type names, simply use type(var) at a
command line to see how python prints type names. For example:

In [1]: type('hello')
Out[1]:

indicates that the type name for strings is 'str'.

%who always excludes executed names loaded through your configuration
file and things which are internal to IPython.

This is deliberate, as typically you may load many modules and the
purpose of %who is to show you only what you've manually defined.


whos gives extra information about an object.



Next, let's see how to use ipython to debug our code.

If you are already inside ipython, to start pdb/ipdb, all you need to do is run the code with -d option:

In [14]: run -d test2.py
*** Blank or comment
*** Blank or comment
Breakpoint 1 at /home/shan/code/test2.py:3
NOTE: Enter 'c' at the ipdb> prompt to start your script.
> (1)()

ipdb> c
> /home/shan/code/test2.py(3)()
2
1---> 3 class Employee:
4 def __init__(self, name, title=None, phone=5551212):

ipdb> l
1 #/usr/bin/env python

2
1---> 3 class Employee:
4 def __init__(self, name, title=None, phone=5551212):
5 self.name = name
6 self.title = title
7 self.phone = phone
8
9 def employee():
10 shan = Employee("Shan Jing","UNIX ADMIN","2720748")
11 import pdb;pdb.set_trace()
ipdb> h

Documented commands (type help ):
========================================
EOF bt cont enable jump pdef r tbreak w
a c continue exit l pdoc restart u whatis
alias cl d h list pinfo return unalias where
args clear debug help n pp run unt
b commands disable ignore next q s until
break condition down j p quit step up

Miscellaneous help topics:
==========================
exec pdb

Undocumented commands:
======================
retval rv

As we can see, 'run -d code' brings us into ipdb. The rest is same as pdb except ipdb offers good features like command line completion, etc.


2. Set ipdb breakpoints from Python source code

First, you can always set pdb breakpoints as I described in a previous blog. After you pdb.set_trace(), when you run codes from ipython, it will open pdb instead of ipdb. The differences between pdb and ipdb is ipdb offers some features that ipython offers while pdb doesn't.

Now, if you want to set breakpoint and opens ipdb inside ipython when your code reaches the breakpoint, all you need to do is:

insert the following code at your breakpoint:

from IPython.Debugger import Tracer; breakpoint1 = Tracer()


Challenge: Write a vim key map to map F1 key in insert mode to insert a breakpoint in your source code without the need of typing the above import code. Also, map F6 key to open a shell in a split window then start ipython/ipdb to debug the code you are editting. Hint: vim's % variable and ConqueShell or Vimshell.

Answer: please refer to my previous blog Debugging Python with pdb.