Where Are The Wise Men?

Mike's Ramblings

Neat ZShell Trick

| Comments

I've just discovered on of the neatest shell tricks I've seen since [Ctrl-r][]. If you are in [zsh][]and hit Alt-. (or ESC-., if you have a misconfigured terminal like I do) then the last argument from the last command will appear on the command line.

This may seem like no big deal, but it saves me a lot of editing on the command line itself. If you aren't using zsh now, maybe this will convince you to give it a try.

Powered by [ScribeFire][].

LifeLight 2007

| Comments

It's Labor Day again, and so we had another trip to LifeLight. Technically, it's still going on as write this, but due to the heat in Sioux Falls and the long list of things to do at home, the Hostetler's left in the late afternoon. But we still had a good time. These are my highlights. Other people will have different ones.

* I was very excited when I learned that Derek Webb was coming. But, when Gina and I saw the schedule we were extremely disappointed that he was playing at 2pm at the Main Stage and then playing again 90 minutes later at the Coffeehouse. But I wasn't disappointed in his Main Stage performance. He just appeared by himself with his guitar and played his heart out. He is just as outspoken in his talk between songs than he is in his music. At one point, when he was talking about loving and taking care of people in Africa he said, "And most people in American churches don't give a shit what is happening in Africa." There was a shocked silence. Didn't he know that this was a Christian music festival? And he what is happening in Africa."

* Gina and I headed over to the Coffeehouse to see Derek's next set and a seminar was just finishing. It was a captain who finished serving in Iraq. The audience was engaged in the subject but it probably wasn't the ideal audience for the guy who wrote My Enemies Are Men Like Me. When the soldier finished, a band started setting up. The singer of that band announced, "I was just asked to say that Derek Webb will not be performing tonight in the Coffeehouse." Being a conspiracy theorist, I feel that Derek saying "shit" and he sudden non-performance is connected, but maybe not. Everfound seemed very ready to go for an hour and a half notice so it could have been planned. We listened to Everfound for a little while and they were pretty good.

* Leeland was just as good live this year as last year. Probably even better. And they moved up in the world -- instead of playing at 4pm on Saturday (like last year) they played at 8pm this year -- just before the big artist of the day. Good for them! They deserve it.

* Gina really like Phil Joel. I was taking Leah back to sleep when he was singing, but what I heard made was good.

Seems short this year, huh? I guess that it is a little bit. Oh well . . . I had a good time.

More Joys Of Python

| Comments

I haven't had a chance to do anything in Python lately. But yesterday I needed to do some screen scraping and, after looking at the HTML I needed to scrap from, I knew that I needed some good tools.

The HTML is table in a form with several rows in it. In cell in each rows has hidden items representing that data. And, inexplicably, each row also has a couple JavaScript functions embedded in it. I did a File-Save on a page with ~150 rows in that table, which is large for what I am doing. You won't believe the file size if I told you, so I'll just copy-and-paste it in:

$ ls -lh file.htm-rw-r--r-- 1 me mkgroup-l-d 2.9M Aug 31 10:39 file.htm

Yes, one HTML file with one table is a little shy of 3MB. There is no images embedded in this file -- just pure text. This is why I needed to get out the Big Guns for this exercise.

The Big Gun for this is [Beautiful Soup][]. It can parse anything that might resemble HTML and give you what you need. It is not speedy, but it works well. And for the hunk of HTML I need, correctness was more important than speed.

One thing I discovered about Beautiful Soup is that you can query with a regex. Remember what I said that each cell in the table had a hidden form field in it? The ID for each row was in a hidden input named "id0" for the first ID, "id1" for the second, and so on. Similar for the customer column -- "CustName0","CustName1", etc. So, really, this was easy to find:

soup = BeautifulSoup(file(fname))ids = soup.findAll("input", attrs={'name':re.compile("id\d+$")})custs = soup.findAll("input", attrs={'name':re.compile("CustName\d+$")})

What became a horrid parsing problem quickly became a three-liner. Wow.

I needed to spit data out from this HTML into different files. Since I know what kind of a pain file management can be, I decided it would be great to put them in a Zip file. I could have done a system call to zip but instead I used Python's standard [zipfile][]module.

I was storing the data files temporarily in a data directory, but I didn't want to put that into the Zip file. Luckily that, too, was easy using the zipfile module and the non-standard yet wonderful [path.py module][]:

zfile = ZipFile(zip_file,"w")

Coding in Python has been a refreshing change of pace, because I can use these wonderful modules to bend the data to my will and all I have to it tell it how. In Java, I have to worry about making sure the right kind of object is being passed and that things are casting right, etc., etc. Although this is a highly complex problem it was easy in Python.

Powered by [ScribeFire][].

A Late-Blooming Garden

| Comments

I saw a vegetable stand yesterday that was filled with pumpkins and watermelon -- you know, late summer and fall items from the garden. I thought of our garden at home and I chuckled to myself.

You see, our small garden is a late bloomer. I got started a little late and planted three tomato plants (one Early Girl, on Big Boy, and one Grape) two [pan squash][] (I thought that there was one plant in the pot, but instead there were two!) and two green people plants. There was some hope that this year would be better than last, where we got about two tomatoes and one little green pepper.

And, for most of the summer, our hopes were dashed. Our plants would grow and we would get buds, and that's it. Last year we had a lot of green tomatoes that never turned red and it looked like the same thing happening again. We got one green pepper, one tomato out of our Early Girl, and them some squash.

Then the August rains came. It's unusual to have this much rain in August, but since it didn't rain at all in June, I guess it evens out. And our garden has taken off. Gina pulled some pan squash out, including a giant one. And quite a few grape tomatoes and said that many green tomatoes are coming. When I checked the garden last night, I saw our Big Boy plant as almost taken over the whole little plot. And it has some nice sized green tomatoes. And two green peppers are coming in!

I think the [composting][]worked. The small place in our garden area was where the compost pile was. When I turned it this spring, I put it on the other side of the garden and worked what was left in that place in the ground. It is still a late-bloomer, I think because of the lack of rain in June, the little attention in July (we were gone all the time) but monsoons we have had this August helped it catch up.

Next year I place on spacing things out a little more. And, maybe, add a couple more plants. But this late-blooming stuff is very weird. I wasn't that late in putting in the garden. I blame the lack of rain for stunting things off. And I can't explain why we have a decent crop of green peppers -- everyone says they are the easiest thing to grow and yet I get excited when I have two.

Well, I'm happy for some fresh produce anyway. Maybe this is just practice for next years . . . .

Powered by [ScribeFire][].


| Comments

I sneaked out of bed at about 4:40am this morning. I tried the backyard first, but we had too many trees in the way. I ended up walking a block or so to the main street where there was less tree cover. I tried to take a picture, but my camera and my abilities weren't up to the challenge. Luckily, someone else was and got a great shot of what I saw:


Powered by [ScribeFire][].