Showing posts with label Python. Show all posts
Showing posts with label Python. Show all posts

Tuesday, May 6, 2014

Don't stop! I yield!

Riffing on the topic I touched on a few weeks ago, of questions on high-pressure exams, I wanted to share some thoughts about a fun programming problem that I'm not sure makes a great interview question -- but maybe that's an argument for why it does.

The problem was mentioned in a thread about programming interview questions.
Consider the digits 123456789, in that order. Now consider the value of the expression resulting from inserting + or * between some of those digits: for example, you could insert + after 2 and 4 and * after 1 and 8, resulting in
>>> 1*2+34+5678*9
<<<   51148
or insert + after 3, 6, and 8, resulting in
>>> 123+456+78+9
<<<   666
Write a program which prints out all the expressions of this kind which evaluate to 2003. (Hint: there are exactly four.)

Thursday, August 22, 2013

Debugging C/API python extensions

I feel really fracking dumb right about now. But at least I now know how to use GDB (the Gnu DeBugger) on this kind of project. So I've got that going for me. Which is nice.

Monday, May 13, 2013

Notes on installing IPython

Notes to self on installing IPython on Puppy Linux:

First: when the site says the QTconsole is "built on QT", that does in fact mean that QT has to be installed first.

Second: QT's "configure" script has to be called with the '-qt-xcb' option (which tells the compiler to use its own version of some library or other).

Third: 'make'ing QT takes a couple of hours. Thank Eris it compiled successfully, or else I'd have kicked something.

=====
Fourth: I spoke too soon. QT's 'make install' terminated with an error the first time I ran it. This is a rare but known bug, and it fixes itself magically by running 'make -j4 install' instead.

Fifth: After installing QT, figure out where the 'qmake' program lives. For me, it was /usr/local/Qt-X.Y.Z/bin. If you're like me and that's not on your path, you'll need that location later.

Sixth: Install SIP. Now, PIP/the PyPI knows about SIP, but can't install it automatically because it uses a weird build procedure. Instead of the usual
$python setup.py install
instead you do
$python configure.py --options
$make
$make install
after you've downloaded and unpacked the tarball.

Seventh: same for PySide. Oh, and don't waste time with PyQt5. Apparently IPython's QT stuff needs 4 anyway. As in, it's hard-coded in.

Monday, February 18, 2013

Triangulations...

... or how I learned that eventually I'll have to stop worrying and love a compiler.

Doubly-indexed comprehensions

A nonintuitive (for me) feature of comprehensions in Python:

Python supports doubly (triply...) indexed list/generator comprehensions. Let's say we want to do something equivalent to the following:
foo = []
for iter1 in range(6):
 for iter2 in range(7):
    foo.append(bar(iter1,iter2))
but without the double loop or initialization. This can be done in just one list comprehension as follows:
foo = [bar(iter1,iter2) for iter1 in range(6) for iter2 in range(7)]
What's surprising about this to me is that, when trying to parse this, my instinct is to associate the for loops in such a way that the range(7) loop is outermost, because the whole statement (bar(iter1,iter2) for iter1 in range(6) looks to me like the body of the range(7) loop. However, the Python team decided to do it the other way: loops in a comprehension evaluate left to right outer to inner.

The reason for this, I'm sure, is so that a developer can take code written with old-style loops and refactor it into a generator expression without the added pain and suffering of reversing the order the loops are written in. A noble goal, to be sure.

Thursday, February 14, 2013

On objects

I haven't had cause to blog about it much at all, but the only course I'm taking this semester is a for-fun course on approximation theory, or more specifically splines.

The guy teaching it, Larry, is... shall we say, old-school. I don't mean that in any way negatively; just that he remembers the days of punch-cards first-hand. However, I do mean to say that his way of computer programming is very different from my own -- at least, as I am now. The computational part of the course has given me an opportunity to reflect on my own changing thoughts, intuitions and preferences when it comes to programming.

Friday, February 8, 2013

Changing the viewing angle of a Matplotlib plot from command line

One annoying feature/bug of working with IPython in the notebook environment is that, when it draws a graphic for you, it freezes the image so that it's no longer possible to interact with it. What to do if, for example, you're plotting a 3d surface, and the default viewing angle doesn't display what you think of as the interesting features of the surface?

Thanks to a nice question on stackexchange, I now know how to not settle for the defaults.

Let's use a specific example: here's the graph of a function which I want to plot:
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
sigmoid = lambda x,y: 1/sqrt(1 + 2*exp(20.1-27*(x**2 + y**2)))
ts = arange(0,1.001,.01)
fig = plt.figure()
ax = fig.gca(projection='3d')
(XX,YY) = meshgrid(ts,ts)
ZZ = vectorize(sigmoid)(XX,YY)
ax.plot_surface(XX,YY,ZZ,rstride = 5, cstride = 5, cmap = cm.jet)

This produces the following graphic:
Now, this isn't awful, but it's not the angle that I really want to be looking at this function from. What I'd really like is to be looking approximately down the line \(y = x \) and not be nearly so far "above" the plot.

The way to change this is to change the azim(uth) and elev(ation) attributes of the axis (the object called ax in the code above). Elevation refers to the angle between the top plane of the axis (the red stuff in the image) and the viewer's eye; in the picture above, it's about 30 degrees. (Python uses degrees instead of radians for these parameters, btw.) Azimuth refers to the direction that the viewer is facing, but be careful! In the image above, we're looking along the line \(y = - \frac{1}{\sqrt{3}} x \), which is a 120-degree rotation from the \(x\)-axis; however, if you type in ax.azim you'll get -60 out, not 120. That's because ax.azim is the angular position of the viewer, and not of their gaze.

ax.plot_surface(XX,YY,ZZ,rstride = 3, cstride = 3, cmap = cm.jet)
ax.view_init(azim = 180+40,elev = 22) 



I've adjusted the rstride and cstride parameters also, which determines how densely spaced the grid lines on the surface are. Notice how in this image, our view goes out along the line with slope \( \tan 40^\circ \), corresponding to the azimuth entered above.

Thursday, January 31, 2013

A precompiled Scipy from Activestate?

So, I've been procrastinating installing Scipy, since a manual install requires having (and manually using?) C and FORTRAN compilers. I tried using pip to do a quick install, but that failed because it needed the FORTRAN libraries (specifically BLAS). None of the off-the-shelf tools, it seemed, were actually off-the-shelf for Windows users. I swear to Loki I will turn this laptop into a Linux box when I have the money to upgrade... but if I'm going to use Numpy/Scipy to teach undergrads, which I still intend to do, then "get Linux" really isn't a solution.

But I ran across a Stackoverflow thread this afternoon where somebody mentioned that ActiveState has a precompiled standalone build of Scipy. My ears did verily perk up at this! After trying a few things and getting very strange error messages, here's what appears to have worked:


Step 0: close all Python sessions.
Step 1: Open ActiveState's PYPM from the Start menu. (This will open a glorified command shell.)
Step 2: enter pypm -g install numpy
Step 3: enter pypm -g install scipy

(The -g flag is an instruction for PYPM to install to the global Python installation.) After doing this, I opened up IPython and both numpy and scipy imported correctly! Yay!

The foregoing is recorded so that I can reproduce this install (or give my undergrads instructions to install) later.


Thursday, January 17, 2013

Notes to self: useful Emacs commands in Python mode

Switch buffers: C-x b (minibuffer then suggests a default buffer to switch to, waits for text input)

Comment/uncomment area: M-; (works with marked region, but to uncomment you should have the '#' marked that you want to remove)

Indent: C-c >
Unindent: C-c <

Tuesday, October 30, 2012

"Has the package installed correctly?"

This post will either be a staid and serious discussion of what is probably a completely n00b-tastic issue I'm having with Python, or more likely devolve into a rant. Important note: nothing in this post should be regarded as factual without further checking. If you find that I have made a factual mistake, please let me know in comments.

Tuesday, October 23, 2012

Upgrades are strictly optional

Can I just say that it annoys the living fuck out of me that, after having basically made the mental switch to Python 3, it turns out that a lot of the powerful non-commercial packages for doing math and science haven't been ported/translated and are only compatible up to Python 2.x!!!

Specifically, dealing with mathematical graphics: the standard sharp object appears to be matplotlib; the examples show some very nice graphics, but it won't talk to Python 3. It's been like two years, people! I understand that some projects aren't actively maintained, but if you do, and there's a new version of the scripting language out, don't ignore that fact!

Bah. Anyway, the main point of this post is to record for myself and my alteri nos that IDLE doesn't deal with matplotlib/pylab in interactive mode. So don't try, just use the basic shell (or, apparently, switch IDEs to something called ipython) if you need interactive. Which kinda sucks, since the whole reason I'm going down this particular yellow brick road is to try to program a very basic visualization tool for finite posets, something similar in feel to the congruence lattice/subalgebra lattice tool in Ralph Freese's universal algebra calculator.

Thursday, September 13, 2012

Subclassing immutable types in Python 3

I've been hacking around in Python 3 for a while now, writing (as I mentioned a while back) a package for implementing arbitrary finite first-order structures.

The natural way to build such a thing is to subclass sets in some way: the traditional way of describing a first-order structure is as a set with additional information attached. But Python has two native set types: set and frozenset, the former mutable and the later im-.

Ideally, we'd like to enforce that you can't add or subtract elements from a model -- mean, if you plan to iterate through a model M, it would be a poor choice to pop elements off one at a time and throw them away, so a good programmer should make that an action that a user can't do by accident. Hence, I'm going to subclass from frozenset instead of the mutable set class.

There's only one problem: initializing an immutable object, or a mutable object which inherits from an immutable class, requires doing things a little differently.

I won't go through the whole setup process for my Model class, since that would involve a lot of explaining, but I'll do a simplified example: a derived class from frozenset with an extra data attribute foo.

But first, where does the problem crop up anyway? Let's say that we didn't know about this whole business: how would we usually program a class inheritance?
class SetWithFoo(frozenset):
    def __init__(self,X,foo_in):
        frozenset.__init__(self,X)
        self.foo = foo_in
 But when you compile this code we might get something like
>>> S = SetWithFoo({1,2},"bar")
>>> S.foo
    'bar'
>>>1 in S
    False
What's gone wrong? Well, remember how frozensets are immutable? And remember how __init__(self,...) isn't a constructor, because the object self already exists? What that means here is that self gets summoned into existence from the void with certain elements -- and those are the only elements which will ever belong to it. By the time __init__ sees the object, it can't change its members.

The method which does the summoning is where we need to work. That method is called __new__(...), and it's the only method in your class which doesn't take self as an argument -- because self doesn't exist yet! Instead, the first argument to __new__ is the class of the object it's creating. You the programmer don't have to worry about this at all -- just put cls as the first argname, and Python will take care of the rest:
def __new__(cls,X,foo_in):
Now, at this point there are two schools of thought on what to do next. One of these schools says that if you're going to bother overriding __new__, you should code the whole initialization in there and just leave __init__ alone (don't even explicitly override it). I'm more in the other side, which thinks that only that which has to be done in __new__ (that is, what has to be done before freezing the basic data of your object, in this case the immutable members of the set) should be done there; everything else can be handled profitably in __init__. The one caveat is that the arglists (including default arguments) of the two methods must be the same (except for cls and self, of course), or else Python will throw a fit when you call SetWithFoo(args)
Long story short, either of the two following code blocks will do just fine.
def __new__(cls,X,foo_in):
    s = frozenset.__new__(cls,X)
    s.foo = foo_in
or
def __new__(cls,X,foo_in):
    return frozenset.__new__(cls,X)

def __init__(self,X,foo_in)
    self.foo = foo_in
but obviously not both ;)

Sunday, August 26, 2012

Mark Pilgrim vs the Android

Discovered completely by accident that the up-to-date version of the book I'm learning Python from is available, for free, from the Android appketplace.

Also, now that I have a handy reference for Python 3, I think it's time to upgrade my IDE too.

Tuesday, August 14, 2012

Blogging learning Python: Equivalence Relations II

Part I

Last time we looked at the basic class methods of the eqrel class. Python knows these are class methods because their definition blocks are indented under the main class header. What this means in practice is that a method like union is called on an instance EQ of the equivalence relation class, modifies that instance, and doesn't return anything.

The methods we'll see this time are different: they don't belong to the class but to the overall module, and they return a new instance of the class.

As mentioned, the two main functions we want to be able to compute are the equivalence relation join and meet. Join doesn't require any new technology:

def eqjoin(EQa,EQb):
    """Returns an instance of eqrel coding the least equivalence relation
    containing both EQa and EQb.

    Both arguments must be of the same length."""
    if len(EQa) != len(EQb): raise IndexMismatchError()
    
    N = len(EQa)
    EQr = eqrel(N)

    for iter1 in range(N):
        for iter2 in [y for y in [EQa[iter1],EQb[iter1]] if y >= 0]:
            EQr.union(iter1,iter2)

    return EQr

I've included a custom error message in case __name__ passes stupid data to the method; I'm sure that there are other possible errors I could anticipate, but this is the only one I can remember accidentally tripping over in practice. Here's the custom exception definition:

class IndexMismatchError(Exception):
    def __init__(self):
        self.msg = "Index Mismatch!\n\nAll equivalence relations must have the same length!"

    def __str__(self):
        return repr(self.msg)

Notice that a Python exception is a class (which for obvious reasons inherits from Exception). Basically all I've written this one to do is print a message detailing why it got called; nothing fancy.

Now let's program equivalence relation meet. But to do this, we need a low-wattage implementation of breadth-first search. (Recall that we already have a class method EQ.alists(), shown in the last post, for creating an adjacency-list representation of the tree out of the original parent data.)

def bfs_component(alists,origin):
    """bfs_component(alists) returns a list, the connected component
    of the element origin in the graph coded by alists."""
    N = len(alists)
    is_searched = [False] * N
    queue = [origin]
    ret_component = []

    while len(queue) > 0:
        active_element = queue.pop()
        if is_searched[active_element]:
            pass
        else:
            is_searched[active_element] = True
            ret_component.append(active_element)

            for iter1 in [x for x in alists[active_element] if not is_searched[x]]:
                queue.insert(0,iter1)

    return ret_component

All this implementation does is compute the connected component of whatever vertex number is passed as the origin argument. This will allow us to search downwards, where the array-of-parents data representation allows easy upward search but no downward search. (The tradeoff is that it is more difficult to tend a tree represented as adjacency lists, cut off and reattach elements, etc. Not an insurmountable problem, but annoying.)

Finally we can meet the meet:

def eqmeet(EQa,EQb):
    """eqmeet(EQa,EQb) returns an instance of eqrel coding the greatest
    equivalence relation contained in both EQa and EQb. Both arguments must
    have the same length."""
    if len(EQa) != len(EQb): raise IndexMismatchError()
    
    N = len(EQa)
    alistsa = EQa.alists()
    alistsb = EQb.alists()
    components_b = dict([[root,bfs_component(alistsb,root)] for root in range(N) if EQb[root] < 0])
    ## components_b is a dictionary with items root : root/EQb, where root
    ## takes on all root values in EQb and root/EQb is a list of the elements
    ## in root's tree.
    
    component_list_r = []
    is_handled = [False] * N

    ## Basic procedure: 1: pop an element from the main queue.
    ## 2: find its EQa component and its EQb component.
    ## 2a: the intersection of these two is a component of EQr.
    ## 3. exhaust the remaining elements of the EQa component,
    ## treating it as a queue.

    for x in range(N):
        if is_handled[x]:
            pass
        else:
            cmp_a = bfs_component(alistsa,x)

            while len(cmp_a) > 0:
                x1 = cmp_a[0]
                cmp_b = components_b[EQb.find(x1)]

                cmp_r = [y for y in cmp_a if y in cmp_b]
                ## cmp_r is the intersection of cmp_a and cmp_b
                component_list_r.append(cmp_r)
                for y in cmp_r:
                    is_handled[y] = True

                cmp_a = [y for y in cmp_a if y not in cmp_r]
                ## deletes all elements of cmp_r from cmp_a

    return eqrel(N,blocks = component_list_r)

Python's list comprehension scheme is right up my alley, as you've no doubt already seen. I do sometimes worry about construction like the assignment of cmp_r near the end; abstractly, that could be \( \mathcal{O}(n^2) \), but I think Python has ways of making comprehensions like that run faster than the naive algorithm. (If someone knows if that's true, I'd love to know details. Maybe use some kind of hash and only check clashing pairs?)

So that's it for the methods I planned to write when I sat down to program. But wandering around in the documentation, I found something really fun, which I couldn't resist including. But that's for next time.

Sunday, August 5, 2012

Blogging learning Python: equivalence relations I

 (Continuation here)

So I've finally broken down and started climbing up the learning curve for Python, just like everyone has been telling me to do for a couple of years now. Last summer's debacles with Octave should have been the last straw, but it's amazing what one can't get oneself to do when one actually got the result (despite the algorithm's taking a week to finally fail to compute what I asked for) and can move over to writing a paper instead of laboriously constructing examples.