Python and XML — coming to a meeting near you

Blaine and I have been talking this week and then he tells me he put me on the schedule. Yes, I am doing a talk for the Omaha Dynamic Language Group on September 5. The topic is “Python and XML” My focus will not be on a intro to Python, but rather what Python can do for XML processing. Hopefully, it will whet the group’s appetite to learn more about Python.

This is the first talk I’ve ever given to group like this (though I’ve conducting hands-on training at work). So I’m excited about getting my feet wet in the informal language talk realm.

Now, I should get started sometime . . . .

My first “apply” call

After many years of using Python, I finally had to use the apply function.

I need to find the path name of a file underneath it’s parent. Yep, it’s confusing. Here is the code:

def findName(fpath,folder):

    fsplit=fpath.splitall()

    parentSplit=folder.splitall()

    startJoin=len(parentSplit)
    return apply(os.path.join,fsplit[startJoin:])

IronPy 1.0Beta7 — the goodness continues

I haven’t tried out the last few betas of IronPython because of a lack of time, but getting it to work with our application will a priority for me latter this summer. So today I decided to take a wack at Beta7.

I first tried to get it to use our DLL’s but that wasn’t working. I broke down and looked at the docs and they added the AddReferenceToFileAndPath to the clr library. That’s what they recommend for scripts now instead of Beta1’s AddReference. After changing that, things were working wonderfully! Even better — Beta7 can load ConfigParser, which Beta1 couldn’t (yes, I filed the bug on that one). That is a Good Thing(tm) for me, since I use ConfigParser a lot in my scripts at work.

The big test was making it work with the most-excellent path module. Not just because I use it alot, but also because it combines a lot of standard Python modules together. If path works, it implies that os.path, glob, and even shutil are all working.

And the initial verdict is good:

IronPython 1.0.60523 (Beta) on .NET 2.0.50727.42
Copyright (c) Microsoft Corporation. All rights reserved.
>>> from path import path
>>> parent = path("..")
>>> for dll in parent.glob("*.dll"):
...    print dll
...
..\IronMath.dll
..\IronPython.dll
..\MSEDAF.dll
..\MSEL.dll
..\WPOBJAPILib.dll
>>>

Let the banging-the-crap-out-of-it continue!

Using Sets on files

Problem: I had files from a directory I had on one machine to a directory on another. I didn’t care about subdirectories (there are none) nor did I care about files on the remote machine not being on the local side. What I did care about was not overwriting the files that were already on the local machine, even if they were on the remote side. Note that this is quite different than Unison, which checks time stamps, size, etc. I really just wanted to check filenames and move them around if need-be.

I used the almighty sets package. I actually made three Set objects: one for files I didn’t want to move at all, one of all the files on the remote site, and one of all the files on the local side. I then did a difference on the local set to the remote set, and then another different from that set to the ignored set. This sounds waay complicated, but it’s really not.

Here is the code. Note that I’m also using the almighty path module, which has become a standard for me.

#!/usr/bin/env python

import sys,os,string
from path import path
from sets import Set

host="foo.com"
rdir="/directory/to/copy/to."

if __name__=='__main__':

    ignoreFiles = Set(["list","of","files"])
    rfiles = []
    for f in os.popen('ssh %s "find %s"' %(host,rdir)):
        rfiles.append(path(f.strip()))

    rfileSet = Set( [x.basename() for x in rfiles])

    lfiles = []
    for f in path(".").listdir():
        lfiles.append(path(f))

    lfileSet = Set( [x.basename() for x in lfiles])

    localOnly = lfileSet-rfileSet
    localOnly = localOnly-ignoreFiles

    copyCmd = 'scp -r "%s" "%s:%s"'
    for f in localOnly:
        os.system(copyCmd %(f,host,rdir))

Moving from de.lirio.us to del.icio.us

One of my worries of moving from de.lirio.us to del.icio.us is that I wanted move my bookmarks to del.icio.us. I thought it would be impossible, but after I remembered how to use del.icio.us API and that you can fetch any of your de.lirio.us’s pages with “page=num&format=rss”, then it was easy.

Hence comes der2del.py, a badly-name Python script that reads the RSS pages from de.lirio.us and posts them to del.icio.us. I tried to put the posted time in, but del.icio.us seems to ignore it. There is no also no way to get a tag from de.lirio.us’s RSS feed, but the bookmarks are now in del.icio.us now. That’s the main thing.

der2del.py uses a customized version of my RestConnect library. I need to put in authenication, and I was posting things, not fetching them, so the return code was much different. I also ended up going with ElementTree as the XML parser, since it’s faster and I was doing a lot of XML parsing (parsing the RSS feeds, parsing the response, etc.) The script doesn’t read the response code correctly — it thinks that something went wrong with every post. But the post is there.

Yes, the script is messy, but it’s better than nothing.

On another note, my PC decided that it didn’t want to reboot this morning. I think it’s a long-ago upgrade that finally went awry. It may just be easier to redo my Linux install than fix the issue, baring any data loss. This has nothing to do with moving to del.icio.us — I just thought I’d share my grief.