Watchpoint debugging for OpenERP

A few thoughts about watch points and why they'd be really useful in the realm of OpenERP/OpenObject
par Georges Racinet, mis à jour le 09/09/2012

(Originally published on 2011-06-12 on my personal technical blog)

While debugging some end-of-chain OpenERP module, I felt that watch points would have save me a lot of time.

In this post, I gather some thoughts about why specifically it would be useful in the OpenERP context. So far, I haven't found anything similar to what's described below, but feel free to point me at it if you know one.

As a side note, debugging some foreign code is usually work I actually like, maybe because it's puzzle solving for which one has to mix information gathered through different tools:

  • code reading
  • advanced code searching
  • log lines (built in and set for the occasion)
  • debugger

I should also say that understanding how something works is very akin to debugging. In other words, one doesn't need bugs to use debugging tools.

What is a watch point ?

The gdb documentation has a very broad definition:

You can use a watchpoint to stop execution whenever the value of an expression changes, without having to predict a particular place where this may happen.

The most common definition I found on the internet is a bit more restrictive:

a watchpoint is a breakpoint that suspends execution whenever a specified field is accessed or modified.

Replace of course "field" by "attribute", "column value" according to context. The first page of results had this definition in the context of IBM SQLJ, Eclipse and… SAP debugging tools.

It is well known that pdb does not have watch points, even in the simple sense of tracking object attributes. I've always felt that it didn't really need them, because it's run from within the language, and one can set very flexible conditional break points that can do the job in most situations. Not to say that Eclipse Java debugger doesn't have conditional breakpoints, mind you, but it's a different feeling.

Simplest example of conditional break points:

if self.x == 1234: pdb.set_trace()

Why pdb conditional break points aren't enough to debug openobject application code

First, I should state the obvious : it's a matter of saving time.

Downstream application code is about putting things together, and hence involves lots of different objects. It can be also hard to track, because that's where you get to read those methods with several hundred of lines, five levels of if/then/else blocks, redundant loops, counter-intuitive method namings…

So, let's say the value of some model's column is set and you don't know where in the code that may have happened. It could a side effect of about anything. In openobject, as well as most SQL based applications, lots is done through foreign ids. In general, if the object one's interested in is the identical between different runs of the software, one may very well find its id with pdb or a log line, and restart the server with a simple conditional breakpoint in the write method of the appropriate osv subclass.

But there are many cases where the object cannot be reused in subsequent runs, therefore the interesting id changes all the time. This happens e.g if you have to set the workflow state of some intermediate model in the reproducing process and can't undo that. Imagine for instance a bug in the accounting lines produced by an invoice confirmation. This is frustrating, because you know very well the interesting object's id from the code : it's typically stored in a variable that sits right in front of your eyes.

Another slow down for pdb conditional breakpoints in the openobject context is the models inheritance system, making it not so obvious to find the correct class where to put them : the write method can be overridden from about anywhere, and you'll lose time finding it, too.

How could a watch point system for openerp look like

I'm thinking of something like this:

account_line.watchpoint(meth='write', ids=interesting_ids)

This would intercept all calls to the write method for the account_line model and set a conditional breakpoint related to the interesting ids. Then you can climb up the call stack.

I don't imagine this in the main openobject-server code, but rather keep it separated by using a dedicated launcher for the server. It looks like a typical exercise in metaclass programming.

I did some experiments already and am very close to a working proof-of-concept. Probably more on that later, have to try it in real life.

Why not IDE ?

Added on 2011-06-13

Granted, a debugger running from outside of the process being debugged would also be useful for that kind of introspection, since they don't require restarting the server. That's typically the IDE way of doing. But…

  • I think pdb's ubiquity is valuable, and adding the kind of watchpoints I propose here is lightweight and can work no matter the context, e.g, on a headless development server
  • there's lots of freedom in the above proposed approach, and hence room for potential further powerful developments
  • I don't like IDEs overall. Yes, that's a matter of personal taste and I love python for being manageable with no IDE, but it also comes to its own laziness : I haven't investigated python IDEs much :-)