Build a Path for Elementtree Namespaces

I really like using ElementTree and it’s lxml and xml.etree brethern.  But one of my pet peeves is how it deals with namespaces. I understand the reasoning — it’s just difficult and bulky.  Combining the syntax of {namespace-uri} with an XPath-like search string can be confusing.

I’m muddling over this while I’m writing yet another utility script to help me with some XML.  I stop and decided I’m going to build a help class, wittingly called PathBuilder.

class PathBuilder:

    def __init__(self,namespace_uri):

        self._template=string.Template("{%s}$tag" %namespace_uri)

    def all_tags(self,tag):
        return "//"+self._template.substitute(tag=tag)

    def all_decendents(self,tag):
        return ".//"+self._template.substitute(tag=tag)

    def children(self,tag):

        return self._template.substitute(tag=tag)

My methods above are all that I need at this point, but you can see how to build them all. I can put all my path building knowledge in that class and then just tell it what I want and it will give it to me.

Adding your custom fonts to Cygwin’s XWin

I decided to finally make the leap and install Cygwin’s XWindows and run Emacs out of that instead of NTEmacs.  There wasn’t one big thing, but a bunch of little things, including:

• I wanted to stop using Plink in TRAMP, because that was a mess.  I know that it works better with a command line SSH client, like what I was using for Cygwin.
• They took my Win32 grep away!  How else am I going to use grep-mode?
• Pasting a path/filename from my Cygwin prompt to NTEmacs always involved an extra step — calling cygpath and the copying the value.
• I wanted to mess around with BicycleRepairMan but my Python environment is in Cygwin and my Emacs environment wasn’t.

If you read between the lines, you see a common theme — control.  In my entire Windows workstation, I have very little control over the environment.  In my little Cygwin world,  I have almost complete control.

I had installed and used Emacs within Cygwin/X before, but found that it was slow and clusmey.  Now they’ve had a few years to work out the kinks and I have a much more powerful workstation. And so I tried it again. Yep, things are much better!  However, I like some of the functionality of Emacs 22 but Cygwin doesn’t have Emacs 22 — they still have Emacs 21 as “stable” and Emacs 23 as experimental!  So I jumped through the hoops to install the “unstable” Emacs 23 along with X-windows (XWin, the Cygwin/X world.)

And thus our story begins.  Since I now have complete control on one part of my GUI and one of the things I’m picky about is fonts.  I like to use ProggyClean and, now, heck I can install it. So I downloaded it, move it to the right file, did the magic “xset fp rehash” and  . . . nothing.  Nadda.  XWin couldn’t find it.

I won’t depress you with a play-by-play, but this is an overview:

• I made ~/.fonts a font directory
• fonconfig ignores any addition to the /etc/fonts/local.conf file even though it says I loaded it.
• I decided to kick up old school and edit the ~/.xinitrc file — only to find out that if XWin starts in Rootless most it doesn’t even look at the ~/.xinitirc.

What ended up working was that I added the following to the /usr/bin/startxwin.bat file just before the last command (an xterm, I believe):

%RUN% /usr/bin/xset +fp /cygdrive/h/.fonts &

Then the font is loaded! Then I just make sure my ~/.emacs says:

(set-default-font
"-windows-proggyclean-medium-r-normal--13-80-96-96-c-70-iso8859-1"
)

And then you see such beautiful things as:

Emacs using Proggy Clean

Formatting Date in Lucene Queries

Little did I know that my adventures with Terracotta would soon require me to dip my toes into Lucene so, you know, we could actually find the objects in Terracotta.

So I had an odd problem where Lucene was only searching by the year. For example, I had an object with the date of 20090915. And so I made my date range as:

datevalue:[19000101 TO 20091231]

And that worked fine. Then I changed the query to:

datevalue:[20090101 TO 20091231]

And it didn’t find it! Why not?

Even though it’s not spelled out in the documentation, you have to use the ISO Format for date/time objects. So I changed the query to:

datevalue:[2009-01-01 TO 2009-12-31]

And it started working

Using Jython as a Terracotta Command Line

My new position at work was best described to me from my AVP: “Figure out how to make it work.”  The first thing that I need to see if it will work is Terracotta.  Terracotta is cool stuff — more to follow on it later.  Probably.

If you know me well enough, or have been reading this blog for a while, you know that I’m a big fan of Python.  Well, if I need to figure out how to “make it work” it generally involves Jython.  That way, I can open a prompt up and noodle around at the object and see what makes them tick.  To me, this could be especially important in Terracotta since you have objects floating around everywhere.

Getting Jython as a command-line interface into a Terracotta cluster is trivial — almost not worth blogging about.  And the result is oh-so cool.

Here is my Jython script. It just imports some common things and a Spring application context, and then get an object from the context. That object contains my Terracotta root object, which means that it stays in sync across my client and server JVM’s.

import sys,os

from org.springframework.context.support  import ClassPathXmlApplicationContext

if __name__ == '__main__':

    ctx = ClassPathXmlApplicationContext("applicationContext.xml")

    worker=ctx.getBean("worker")

Not that there isn’t anything specific to Terracotta in there — just like it should be.

I then wrote a shell script to setup my environment and run my Jython script in a Terracotta-instrumented environment. I actually run everything in Windows (unfortunately) so I have some Cygwin magic in there to change the classpath and to run the dso-java.bat in a cmd window:

#!/bin/sh

cp="dowork.jar:/c/Projects/jython2.5.1rc2/jython.jar"
cygstart bin/dso-java.bat -classpath `cygpath -mp $cp` org.python.util.jython -i $1 $2 $3 $4 $5

This allows me to run any script in a Terracotta cluster and then keep a command line open to I can muck around with stuff. Thejython.jar I use is a stand-alone install. I’ve just found that easier than worrying about where all modules are.

So now I just enter the shell script name with my Jython script as an argument and away I go!

And this works exactly as I expected. I can see the state of the root in the worker object, update that state in another clustered JVM, and then see that state immediately in my Jython prompt.

I’m sure you can do the same with Groovy, JRUby and their ilk. But I use Jython. It’s just how I roll

The Harrowing Journey From Tiger to Snow Leopard

Unlike most Apple users, I didn’t make the quick jump from Tiger (10.4) to Leopard (10.5).  Mostly because I’ve learned the hard way not to be the first in line for upgrades.  And when I read about the changes they were making, I thought “I’ll wait until they work out the kinks.” And then they announced Snow Leopard (10.6)  and touted the $24 upgrade but, if you looked close at it, that was only from Leopard to Snow Leopard, where little was changed on the service (but much under the hood was redone).  A little looking and I found that I had to get the Mac Box Set with 10.6, iWork, and the new iLife, which I wanted anyway.  And lots of my tools that wanted to use only worked with Leopard on up anyway.  So Snow Leopard it is.

I ordered my Box Set and waited for Amazon to ship it (it was $20 cheaper there and no taxes. Yes, I’m that cheap!).  When I got it in my hands, and got ready for the upgrade . . . I did a backup first.  SuperDuper is my friend.  Before this process was over, it became my lifesaver.

So I stuck the Snow Leopard disc in, and told it that I wanted to upgrade.  The machine rebooted, the Snow Leopard install came up, and said it was starting and then . . . it quit, telling me that there was not enough room left on the drive.  Which was very possible — there was a lot of junk on that drive.  So I took the disk out and rebooted, thinking I would remove some more junk and then do the upgrade.  And then it happened . . .

The machine wouldn’t boot.

My Mac would start up just fine, give me the Apple logo and then shut down.  I put the Snow Leopard disc back in and that didn’t boot either!   A little research showed my assumption about it booting from the DVD drive if it couldn’t boot anything else was wrong — instead it just stops.  You have to hold down “C” during the boot sequence to get it to boot from the DVD drive.  I went to find my Tiger install discs and booted with that.  I went to run the DiskUtility and did “Repair” but it said it couldn’t.   Arrgggghhhh.

I tried the DiskUtility with the Snow Leopard and it wouldn’t even Repair it — it was not a Snow Leopard hard disk!!  Arrggghhhh again!

Now I had a choice.  SuperDuper makes my USB hard drive bootable.  I could boot off of it but that doesn’t solve my problem — the boot info on the hard drive was messed up. If I booted from the USB drive and removed enough stuff to make Snow Leopard install.  But, still, nothing can repair my drive.  My data was safely backed up and I know that I didn’t need it all anyway and, with SuperDuper, I can go and copy the files that I wanted off of it anyway.  So I took a leap and erased the drive.

You read that right. I wiped the drive clean with the Snow Leopard installer and installed from scratch. Of course, the installer was more than willing to do that.

After that, things went mostly well.  I copied our Music and Preferences folders over and Safari, iTunes, and Mail all saw the changes and updated their databases.  The copying part took a while, but after that it was all smooth.

But I had lots of problems with MacPorts.  Emacs.app needs some manual guidence , Python2.4 has some weirdness, and PostgreSQL/PostGIS are always a pain to install.  But I got them going.

A clean install was a good thing — it got rid of the junk and I was able to move just the files that I needed.  And, thankfull,  SuperDuper demostrated that it’s worth 10x it’s $28 price tag.

So do your backup kids.