Using Dataclass as a constructor

 — 

This article from Glyph talks about not using the __init__ dunder function to initialize a class and instead use the dataclass in the standard library. I read his post and experimented with this new way to make a constructor and decided that I totally agreed – this was a much cleaner way to initialize a class. I will let you read his arguments about why this is so much better.

But I hit a gotcha as I was implementing a new class, and it was more to do with my understanding of how Dataclasses work than actually using it in an constructor. Here is an example:

from dataclasses import dataclass
from initializers import some_expensive_client

@dataclass
class MyService:
    client = some_expensive_client()

    ## more methods

some_expensive_client is just a wrapper around an outside API. In my case, it was an AWS/boto3 client. I'm not sure what type it was to make Python's typing system happy and, to be frank, I don't really care that much (yes, I'm a bit old-fashioned in my Python ways). But that was a mistake.

When I was writing a test, I wanted to client via MyService(client=mock_client). But then I got an AWS error that client wasn't valid. I scratched my head … my mock_client didn't have anything to do with AWS. Then noticed that I didn't specify a type, which a dataclass expects. So I just added Any.

from dataclasses import dataclass
from typing import Any
from initializers import some_expensive_client

@dataclass
class MyService:
    client : Any = some_expensive_client()

    ## more methods

And then I could do MyService(client=mock_client) with no problem. So what seems to happen is if you don't specify a type, the @dataclass gets confused and doesn't actually initialize it (or treat it like a normal property). Once you add the type (even a generic Any) the magic happens.

Category: tech Tags:

Lotus Agenda: modern ideas from an old app

 — 

A couple of weeks ago I read a long article about Lotus Agenda. I had heard of Agenda before – I had read man times that it was a much beloved niche product that was way ahead of it's time. You can read this particular article yourself – I'm not going to repeat its thoughts and conclusions here.

But if you know me well enough, you know I went down the rabbit hole and I read more and found a copy to download. Not because I was going to move to it – but because I wanted to try it out. I like it a lot but in the recesses of forums and articles, people said that the instaling the Presidental Planner macros added… well, so much more. So I grabbed that as well and played with that integration.

My experiments with Lotus Agenda and the Presidential Planner add-on was enlightening. It does things that no other modern app does now. I mean, it's an old DOS app but the functionality of the whole thing was terrific, despite showing it's age with function keys to do most of the stuff.

To use it was easy: just start typing.

pay the light bill tomorrow

And… just leave it there. Soon it will turn into a task:

<tomorrow date> pay the light bill  TODO

I did nothing to make that happen – I just typed that in.

If I have put in:

expense the gas for the business trip

Soon (5-10 mins) I'll see something in the expense view for that. I still have to put the amount in (something I've little perplexed over) but it just makes it into a list.

A fantastic one was this:

get new paint for House

Note the capital letter. Not only will that be a Todo item but I will magicially have a category called House that I can search for and just see the list of things in that category.

You are thinking "Well that doesn't seem too hard" but it apparently it is, because none of the todo apps I've ever tried has done things like this. Todoist gets close but it doesn't auto-categorize. Amazing Marvin (my current app), Remember the Milk, even TickTick (which I experimented with after my Agenda experiment). Remember: this is a DOS application written in 1990. There is no LLM, it doesn't even know what a network is, let alone the Internet. It just runs locally.

This honestly is as close to my ultimate wish in a Todo app as I have ever seen. I can put in a notes that are a stream of consciousness and out of it will come a distilled list of things that I can do. Later on, I can organize them, schedule them and put them on my timeblock. I keep thinking that something will come out that will do all this and it doesn't. But maybe it already has – in 1990.

After being in Agenda and playing with things a bit, the magic kind of faded a bit. Overall the parser is great but then you realize Agenda only takes a simple statement and then uses the wording to see if it's a Todo or a note-to-self, an expense or whatever else it supports. It doesn't learn from your text and it doesn't get smarter. Mostly it teaches you how to insert items. And there is nothing wrong with that but at end of the day, it's not different than Amazing Marvin or Todoist…only that Agenda has a much smarter parser.

As I started thinking about how much I would like something I could have a stream of consciousness to a perfect group of tasks, I realized that my current workflow is already doing all that. For the past few months, I've been utilizing my Bullet Journal. When I have a thought, a todo or even a feeling, it jot it down and then at the end of the day I go through it and schedule tasks my tasks in Amazing Marvin or simply read my thoughts or maybe cancel something. I like the fact that it isn't automatically put on my working list… that filter of "is this really want I want to do?" is very helpful.

I think productivity tools should hit a sweet spot between Agenda's intelligence and a Bullet Journal's friction. We need tools that can figure out our goals based on what we write, yet still help us reflect on whether those goals align with our true intentions.

Maybe it took a 30-year-old DOS application to help me realize that, for now, my notebook and pen will continue to be my first draft.

What tools have you found that balance automation with intentionality?

Category: productivity Tags:

Emacs Works For You

 — 

This is my submission for the August Emacs Carnival – the Emacs Elevator Pitch.

My pitch is simple, but then it takes a little while to explain:

Emacs works for you, you won't work for Emacs.

Overall I think the heart of my pitch reflects on my own personal belief about Emacs. I believe there is no one version of Emacs. Every Emacs user has it tweaked to their own workflow so each of user has a different Emacs – sometimes slightly different, sometimes greatly different, from yours.

I still use Intellij IDEA when I'm in Javaland. There are extensions that do a lot for you, but you still have to put up with how the app and those extensions work. For example, one of my pet peeves in IDEA is how it places tabs. The file I was just in previously might be buried among how many other tabs there are. It annoys me to no end. I realized that I don't think about this in Emacs. I easily switch buffers in Emacs – it's simply C-x b, I type a vague substring of the buffer name and Helm gets me the lists of buffers that match it. I barely have to think about it, where in IDEA, I have to do a lot of conscious work. I put up with in IDEA because there is nothing I can do. In Emacs, I would not put up with that for very long (and in this case, I have never had to).

I do almost everything in Emacs now but that is recent development. I used to use Obsidian for personal notes and I wanted to move to Emacs. I wrote about that migration here but the point I want to make is that I could. Was it easy to get exactly what I wanted? No it took some trial and error and some creative ELisp (that I'm not afraid to say that AI helped me write). Even putting in my old Obsidian vaults was possible. And I can link write a new note and link to an old one because orgmode lets you make a link to any other file.

This didn't come out of the box but I was able to configure Emacs to do it for me. Not always easy, but always possible. I get an idea of something I want and then make Emacs work the way I want to.

As I said, Emacs works for me, I don't work for Emacs.

Category: tech Tags:

Self-hosting on a NAS

 — 

The shutdown of Pocket (that I didn't hadn't used in a long while) got me into self-hosting my RSS Reader and my Read-It Later systems. When I bought my Synology NAS a few years ago, I unknowingly bought a model that has an ARM processor, which does not support Docker. But thanks to SynoCommunity packages, I can install things that do work for my system – they just don't run in a container.

I chose Selfoss as the RSS reader. It has a great mobile site and only uses a sqlite database by default. The username/password is set in a config file which is kinda weird but it makes is less database-intensive and makes it easy for you to change.

For read-it-later, it was an easier decision – I chose Wallybag, which is in very active development and has a well-documented API and mobile site. I find that it's very slow to add items sometimes but overall the formatting is very good (except for Six at 6, which I should report).

Both apps work really well on desktop and mobile, without separate apps and they both have good and documented APIs. I should state that Wallbag has good apps on iOS and Android but they are totally not necessary.

Part of my problem in setting these up is that SynoCommunity doesn't provide a lot documentation on how these packages are setup and you really need to do some configuration on your own to get them working. Here are some of my notes:

Selfoss

The installation folder is in /volume1/web_packages/selfoss and is only readable root. The URL is https://<ip or host>/selfoss/

The installation documentation is here: https://selfoss.aditu.de/docs/administration/installation/basic/

But the only thing you really need to look at is the configuration options: https://selfoss.aditu.de/docs/administration/options/

  1. Copy config-example.ini to config.ini
  2. Need to set the username/password in config.ini

I don't use the MariaDB database option but just use the SQLite database for simplicity – it's just me using this.

One thing you need to do it to have a scheduled job to update the feeds. You can do this manually in Selfoss by hitting a button but why do that? I simply wrote this shell script then scheduled that hourly in Task Scheduler. You don't want to put it in cron (though that is installed) because it will disappear when you reboot.

Wallabag

I found Wallabag to be the most frustrating of the two because the package sets some defaults that aren't necessarily documented as well as knowing a few more things about your Synology system.

The install folder is /volume1/web_packages/wallabag and the URL is https://<ip or name>/wallabag/web/. You will need the root password of the MariaDB for the package to install. You will also have to set a database password for the wallabag user – the install script will create that user and a wallabag database.

The default Wallabag admin name is wallabag and that is the password as well – it should make you change them as you log in.

Wallabag was quite slow to load when I first installed it on my Synology but after I did the latest package update to Wallabag, it's quite a bit faster.

Merging the Two

A nice thing about self-hosting is that you can merge the apps together in different sort of ways and not have to worry about what happens in the connections. Both Wallabag and Selfoss have APIs both documented in Swagger docs: Wallabag, Selfoss. Authenicating to the APIs are very different tho: in Wallabag you use their OAuth API, so you have to make Client IDs and Client Secrets for your user within Wallabag. Selfoss seems more mysterious but actually is very simple, especially when using Python:

 # use a session, not the raw request object
 session = requests.Session()
 response = session.post(f"{SELFOSS_HOST}/login", data={"username": SELFOSS_USERNAME, "password": SELFOSS_PASSWORD })

 # then reuse the session object and you are auth'ed
response = session.get(f"{SELFOSS_HOST}/items?type=starred")

I posted my scripts on GitLab, in case anyone finds them useful or simply wants some inspiration. I have a few more notes in the readme there.

This system has served me well for months so I hope these notes are useful for others.

Category: tech Tags:

From Obsidian to Emacs

 — 

This is to join in on this month's Emacs Blog Carnival: The Writing Experience.

The "Writing Experience" is a timely theme because I recently changed from using Obsidian for my note habit to Emacs. I'm not going into a long list of reasons why but it all comes down to: I like Obsidian a lot, but I like Emacs and Org more. I found that the things I really wanted to do in Obsidian were already working in Emacs/Org, especially with after some config work (more on that below). Sure, Obsidian can make a graph of my notes, I could write sql-like queries to display my notes, etc, etc. I spent a lot of time working on those things instead actually making notes. It took me years to realize this simple fact: Emacs and Org did everything that I really needed. And even more, like making a link to any file (and a line in that file) and putting it in an Org file.

(Ironically, I found Denote Explore while I was writing this post that adds things from Obsidian like graphs, stats, and random walks that could be useful. I haven't played with any of it yet but also proves… Emacs can basically already do what Obsidian is doing.)

What really inspired me to do this was watching some videos of Denote. Yeah I could have used OrgRoam or something else but Denote gives me what I want with little overhead. There is a lot of commands in Denote and I had a hard time remembering them all. I ended up making a Transient menu to help me keep them straight:

(transient-define-prefix mh/denote-transient ()
  "Denote"
  [["Common Denote Commands"
    ("n" "new note" denote)
    ("j" "journal" denote-journal-new-or-existing-entry)
    ("l"  "link, create" denote-link-or-create)
    ("r"  "rename-note" denote-rename-file)
    ("f" "file path"  copy-full-path-to-kill-ring)
    ("k" "search old vaults"  my-xeft-transient)]
    ])


(global-set-key (kbd "C-c c" ) 'mh/denote-transient)

I also installed the Olivetti package, which helps reformat the screen, which I enabled in all OrgMode files. I thought it would be a temporary thing but I figured out that I really liked it – and made it really easy to know if I'm in an Org file or a Markdown file. The screenshot of this post is exactly how that looks

I'm shockingly very happy with this setup. I realized that I hadn't opened up Obsidian for weeks … accept when I wanted to look something up. Well they are just Markdown files, I should be able to integrate them in to my Denote structure (without converting them all to Denote files). Without having to give every step of the journey, I ended up choosing Xeft, which has a Deft-like interface but doesn't open every file to search. Instead, under the covers, it uses Xapian to index and search the files. The hard part was to get Xeft to switch indexes, depending on which Obsidian vault I wanted to search (yes I had two!). Here is my Xeft setup. It's seemingly simple but it took a while to get to a usable point

The reason I used Obsidian for so many years was mobile. My biggest problem was that I couldn't put my Org notes on my mobile device and have them sync cleanly, nor did I have a way to edit them. Then I realized I had the solution all along – DEVONThink and, more appropriately, DEVONThink To Go. What I did was setup DEVONThink to index my org folder . Then that folder is monitored when I create or edit new Org notes. DEVONThink To Go then gets the files from my Mac running DEVONThink and I can edit them or read them on my phone or iPad. Sure, DEVONThink doesn't understand the Org format like it does Markdown, but it's just text files anyway… I can read and edit the files on my mobile devices… that is the end game for me.

So it seems I'm having my cake and eat it too … I can write my notes in my beloved Emacs and OrgMode and have them on my mobile. And even my "old" Obsidian notes are there. It's a great arrangement.

Category: tech Tags:

Emacs: Take 2

 — 

This is part of the Emacs: Carnival: Take Two challenge.

This may be hard to believe, but when I first exposed to Emacs, I dismissed it almost immediately. I was taking a Unix class while at in the university and the horrible professor was an Emacs fan. He tried to teach us the basics and failed miserably. The TA, who lectured a few times was a big vi time (note: not vim, but vi) and he was much better at explaining vi than his boss was at explaining Emacs.

A few months later, I got my first tech job (while attending the same University) and we were all on Solaris Sparc workstations (how much does this age me?). The Admin in charge was a zealot vi fan… so much so that when one of the new employees started and installed Emacs in his home folder, the Admin deleted it over the weekend. With a beginning like this, it's no wonder I never even tried Emacs.

At my first post-collegiate job, I helped support a product that ran on SCO Unix (OK, now I'm aging myself). I used to help people set through configs with standard vi over the phone. It was madness. This time of the rise of vim and I was in love.

So why I did switch from Vim to Emacs? Well fast-forward some years and I had switched positions and my needs changed. I don't remember exactly what happened but VimScript was in it's infancy and it wasn't quite living up to my expectations. And then I remember opening several files at once, trying to compare them and Vim wasn't cutting it. I think I had been seeing smart people talk about Emacs and it could do what I wanted. So, I decided to take a week and go cold-turkey on it. I choose XEmacs because that was better on Windows (where I was subjected to then) and I didn't even choose the Vi-emulation but the normal, standard Emacs keys. It was painful at the beginning but then it got a lot better. I went back and forth for a bit after my cold-turkey run but eventually used XEmacs full-time.

I don't know when I switched to GNU Emacs but I think it was because XEmacs hadn't been updated in a while so I gritted my teeth and decided to use Stallman's version. And there I have been ever since. I've changed a lot of configs and the way I write configs and even distros (my own, Prelude, Doom and now back to Prelude). Literally my configs for years was copy-pasted snippets from the Internet and a few packages, the big one I remember was Icicles, an early completion framework.

I've seen a lot of positive changes in Emacs recently (package management being the big one) and, surprisingly a large uptake in activity in Emacs. Maybe that is just be being more aware of things like r/emacs, which is where a lot of Emacs activity takes place. And of course apps like Magit which makes an entire workflow easier.

So this is my history… what is my Take 2? I have a lot…I wish I would have listened to the bad prof early on. More importantly, I also think I should have spent time early learning Elisp instead of blindly copying snippets. I'm still not very good at Elisp – at all. But honestly coding with an AI has helped me a lot in trying to get things configured the way I want – and to teach me some. And I have learned a lot. I also wish I wouldn't have spent so much time declaring configuration bankruptcy and struggled more with my setup..I would have learned more.

Category: tech Tags:

Small Emacs Tweaks

 — 

Emacs Tweaks

The last few months I really been heavy into software development and, honestly, that is my happy place. I started to get into the habit of writing down my nits or annoyances of my workflow or my Emacs setup to try to streamline things. When I get a chance, I go through the list and start figuring out how to tune things. The following is a list of what things I have figured out in the last few months. Most of them are small, but the small things add up.

dotenv creds

I'm on a project using AWS and my AWS creds refresh twice a day. Luckily we are using dotenv so I just need to paste the values into a .env file in my project. But AWS gives me export statements while dotenv just wants VAR=value . And I have to do this twice a day! So a little Emacs and Hyperbole to the rescue.

I worked with my favorite AI and we came up with the following ELisp:

(defun my-clean-env ()
  "Clean environment variables and format text below the button."
  (interactive)
  (let ((kill-content (current-kill 0)))  ; Get content first
    (save-excursion
      ;; Move to end of current line (after the button)
      (end-of-line)
      (insert "\n")
      (let ((start-point (point)))
        ;; Delete everything from start-point to end of buffer
        (delete-region start-point (point-max))
        ;; Insert and clean the new content
        (insert kill-content)
        (let ((end-point (point)))
          ;; Remove lines starting with AWS_
          (goto-char start-point)
          (while (re-search-forward "^AWS_.*\n" end-point t)
            (replace-match ""))
          ;; Replace all "export " instances
          (goto-char start-point)
          (while (search-forward "export " end-point t)
            (replace-match "")))))))

;; Create the Hyperbole button
(defact clean-env ()
  "Hyperbole action to clean environment variables."
  (my-clean-env))

Then make a Global Button to run the Lisp code when activated. I called this button clean-env. The bottom part of my .env file looks like:

## other stuff
# <(clean-env)>
AWS_TOKEN=<old>
AWS_SECRET=<old>
AWS_SESSION=<old>

When I need to refresh my creds:

  1. goto the AWS console, copy them into the system clipboard (which is a button on the AWS interface)
  2. Open the .env file in my project
  3. Go to the clean-env in the comment (which is a Hyperbole button) and hit M-enter.
  4. The old values are replaced by the new values (without the export charaters).
  5. Save the file

This seems simple but has saved me a lot of brain cycles.

Using eshell to run tests

I am on Windows for my current project and mostly because I use native Python in Msys2 Emacs, I can't run my Python test with project-run-test. This makes it hard to go exactly to the failed tests. But I remembered that I can use eshell, which I generally forget about. Then I can run the pytests within eshell and then use Hypberbole's action key to go to the failed test. Easy!

Navigating between variables, function call, etc.

This doesn't have anything to do with Hyperbole, funny enough. I stumbled onto someone talking about highlight-symbol, which does what it states -- will highlight all instances of a symbol that you are one. But it also has highlight-symbol-nav-mode, which let's you use M-p and M-n to goto the next instance of that symbol in a file. This is the kind of thing I've been looking for but didn't know how to describe it. Before used helm-occur for this type of navigation but that was very quickly replaced.

Editing the results of helm-occur

Speaking of helm-occur, I still use it and generally for tweaking a value or a pattern in my file. You can edit the search results directly, but I don't do it every day so I tend to forget the step. I finally wrote them down:

  1. Find candidates with helm-occur (it's C-c h o in my setup)
  2. Save the buffer with C-x C-s
  3. C-c C-p to change to wgrep mode
  4. Now it's a normal buffer! So I can make my changes
  5. C-x C-s to save the changes
  6. q to quit
  7. You may want to kill the *occur* buffer or Emacs will keep it around and ask you about it later

It's amazing what kind of little things can really add up to make you more productive.

Category: programming Tags:

Treating a SQLAlchemy query like a table

 — 

sqlachemy AI image that I like

I've been doing a lot of work in SQLAlchemy lately and I'm continually impressed with how easy it is for the basic things but also how it can also handle complicated items. One of the things I have had to do again and again is to have a complicated query with a lot of joins. I could fetch the top-items and the all the other rows and then all the other tables... but then I would have a lot of round trips and a mess of objects. I could use a session.query(Table1,Table2...).join(...) type of structure but then I wouldn't necessarily the object that I wanted. What I discovered is that you can map a SQLAlchemy subquery to an object, just like you do with tables. This is very much like a view in a database...only that you are doing all the work in SQLAlchemy, and not have to let the DBA know!

I have a contrived example on Github that shows how this work. The tables.py file defines the tables... there is a Person table, and Authors and Fans point to Person. Author's have Books and a Fan can have many Books (favorite_books). I wanted to create a Fandom query, which will show what each Fan likes (even if they like more than one! The query.py file shows that:

from tables import *
from sqlalchemy import select
from sqlalchemy.orm import aliased

FanPerson = aliased(Person, name="fan_person")
AuthorPerson = aliased(Person, name="author_person")

fandom_q = (
    select(
        Book.id,
        FanPerson.name.label("fan_name"),
        Book.title,
        AuthorPerson.name.label("author_name"),
    )
    .select_from(Fan)  # explicitly say where to start
    .join(FanPerson, Fan.person_id == FanPerson.id)  # Join to fan's person details
    .join(Fan.favorite_books)  # Join to books
    .join(Book.author)  # Join to author
    .join(
        AuthorPerson, Author.person_id == AuthorPerson.id
    )  # Join to author's person details
    .subquery()
)


class Fandom(Base):
    __table__ = fandom_q
    __mapper_args__ = {
        "primary_key": [Book.id, Fan.id]
    }  # using Book.id and Fan.id as the primary ensures we see all the rows

The fandom_q is the actual subquery and you can see that it isn't trivial. You have to make aliases for the Fans and the Authors to get their names and manually do joins. The SQLAlchemy-table object is the one that we are familiar with but note that it's slightly different -- instead of a __tablename__ attribute that takes the name of the table, we have a __table__ attribute that we set to the fandom_q subquery. We also set the primary key to make sure we get all the people back. You can download the project, install the deps with poetry and run the table_query.py file to see it in action (and play around with it.)

Now, in our code, we can use Fandom as table to do queries (ie filters in SqlAlchmey speak) on it, just like a normal table:

    with Session(engine) as session:
        for x in (
            session.query(Fandom).filter(Fandom.author_name == "Stephen King").all()
        ):
            print(x.fan_name, x.author_name, x.title)

So from an API perspective, the Fandom object is just another table that you query from but an insert, update, or delete operation because it's a select not a table. If you try one of those, you will get an error:

sqlalchemy.exc.ArgumentError: subject table for an INSERT, UPDATE or DELETE expected, got <sqlalchemy.sql.annotation.AnnotatedSubquery at 0x105babbb0; anon_1sqlalchemy.exc.ArgumentError: subject table for an INSERT, UPDATE or DELETE expected, got <sqlalchemy.sql.annotation.AnnotatedSubquery at 0x105babbb0; anon_1

Regardless, I find this way to use and re-use a long, complicated query really powerful, because I use all the power of a SQLAlchemy table object.

Category: programming Tags:

git worktrees

 — 

git-worktree

I am not sure how I discovered git-worktrees but I'm sure glad that I did. It has made a big difference in how many branches I can work on at once, while still remaining simple

Git uses worktrees internally but you can use them to make clone from your local repo instead of clone from the origin. However, the big feature is that it shares history with your local repo. So you act on it independently (even push and pull from remotes ) but it's kind of an extension from the local "main" repo. You can think of it as a local fork, if you will, but with a shared history. A good explanation from the main git documentation:

A git repository can support multiple working trees, allowing you to check out more than one branch at a time. With `git worktree add` a new working tree is associated with the repository, along with additional metadata that differentiates that working tree from others in the same repository. The working tree, along with this metadata, is called a "worktree".

This new worktree is called a "linked worktree" as opposed to the "main worktree" prepared by git-init or git-clone. A repository has one main worktree (if it is not a bare repository) and zero or more linked worktrees.

My common use is this: I have a develop branch checked out locally. I then make new worktree and make a new feature branch in that new worktree to work on that feature. If a bug comes up that needs fixing quickly, I either make a new bugfix branch from my main develop worktree or I make a new fix worktree, push and merge it from that folder. Once it is merged into develop, I can use git merge <bugfix-branch> in my feature worktree. I'll give a step-by-step example below.

Note that I don't push, or pull develop from my branches around develop. That is because they are sharing the same history and metadata files. So it simulates doing everything in that folder...but you are actually in different folders. This allows you to have different branches in different folder yet still have the name metadata/history in each. One interesting quirk is that you can't have have the same branch checked out by two different worktrees. So if you are working on BranchA on one worktree repo, you can't work on BranchA in another worktree folder . This seems weird but worktrees are not stand-alone - they all share the same git history.

I like Tomups git-worktree scripts. They not only manage creating the folders, but they also copy over ~.env~ files, node_modules folders, etc. I'll give an example of how I use them with my general workflow:

  1. I have a repo cloned locally and make sure that the develop branch is checked-out.
  2. I run git wtadd ../proj-new-feature which creates a worktree in the parent folder above my project called proj-new-feature
  3. I change to the proj-new-feature folder and make a feature branch just like normal: git checkout -b feature/fancy-new-thing
  4. I start doing my normal development work.
  5. Soon I'm told about a bug in the dev environment. I go to the folder where my develop branch is. It's a one-line fix, so I make a new branch there : git checkout -b bugfix/easy-bug
  6. Once I fix the bug and push it up, I can do back to my proj-new-feature folder.
  7. There I can merge in my bugfix/easy-bug branch easily with git merge bugfix/easy-bug. No pulling from remote, etc because the worktrees share the same history
  8. When I'm done I push the feature/fancy-new-thing branch to remote
  9. After the PR is merged in, I go back to development, pull in remote and then I can delete the feature worktree with git wtremove ../proj-new-feature

The cycle then continues.

With git worktrees I can have different features/branches active and they don't get in the way. It seems like a simple idea but really has made things easier in my workflow

Category: programming Tags:

The Emacs Hyperbole

 — 

Hyperbole is super

I saw a few posts on r/emacs about Hyperbole, including some videos, and I -put it in a tab- save the URL to look at it later. When I finally got down to it and watched some of the videos and read some of the documentation... well I was intrigued but still confused on what it was and how I could use it. So I installed it and have used it for a while. I have to say I like it and it has improved my workflow, but I don't know if I can describe Hyperbole succinctly. So I'm writing this up, in case someone else is confused and perhaps I can get some clarity as to what the Hyperbole package is.

Let me make a quick disclaimer -- there is a lot of functionality in Hyperbole and I'm only covering the things I have found useful in the few weeks I've been experimenting with. There are things that I keep uncovering (even in the middle of writing this post).

The Magic Key Combination

Hyperbole introduces the key combo of M-return that becomes magic. The easiest way to describe it is "M-return does what you want in the context that you are currently in`". Hyperbole calls this the Smart Key and it's assigned both the "Action Key" and the "Assist Key". I'm assuming they did this because you can configure the "Magic Key" to be what you want, and the Smart and Assist Keys could be different. But what do I mean about "does what you want"? Here are examples from the Hyperbole documentation.:

    Smart Key - Magit Mode
     If pressed within a Magit buffer and not on a button:
      ACTION KEY
       (1) on the last line, quit from the magit mode ({q} key binding);
       (2) at the end of a line, scroll up a windowful;
       (3) on an initial read-only header line, cycle visibility of diff sections;
       (4) anywhere else, hide/show the thing at point ({TAB} key binding)
          unless that does nothing in the mode, then jump to the thing at point
          ({RET} key binding) but display based on the value of
          hpath:display-where.
      ASSIST KEY
      (1) on the last line, quit from the magit mode ({q} key binding);
      (2) at the end of a line, scroll down a windowful;
      (3) on an initial read-only header line, cycle visibility of all sections;
      (4) anywhere else, jump to the thing at point ({RET} key binding)
         but display based on the value of hpath:display-where."
q

    Smart Key - Org Mode
     When in an Org mode context and hsys-org-enable-smart-keys is non-nil:
     ACTION KEY
     (1) If on an Org todo keyword, cycle through the keywords in
         that set or if final done keyword, remove it.

     (2) If on an Org agenda view item, jump to the item for editing.

     (3) Within a radio or internal target or a link to it, jump between
         the target and the first link to it, allowing two-way navigation.

     (4) Follow other internal links and ID references in Org mode files.

     (5) Follow Org mode external links.

     (6) When on a Hyperbole button, activate the button.

     (7) With point on the :dir path of a code block definition, display the
         directory given by the path.

     (8) With point on any #+BEGIN_SRC, #+END_SRC, #+RESULTS, #+begin_example
         or #+end_example header, execute the code block via the Org mode
         standard binding of {C-c C-c}, org-ctrl-c-ctrl-c.

     (9) When point is on an Org mode heading, cycle the view of the subtree
         at point.

     (10) In any other context besides the end of a line, invoke the Org mode
          standard binding of {M-RET}, org-meta-return.

So these are just examples from the docs but it goes with what I am saying: - Are you in a Magit buffer online that can cycle visibility? M-return will do that. - Are you at the end of Magit buffer? Then M-return will quit the buffer - Are you on a OrgMode Todo item? Then M-return will change status - Want to execute code in a org-src section? Get on the begin_src and hit M-return

But probably the most important things that M-return executes are buttons -- both Implicit and Explicit.

Button Pushing

I think Buttons are the most important concept in Hyperbole and yet they took me a while to get my brain around. They aren't really that confusing but they are abstract concepts because a button can be literally anything. I will present some ways I have utilized them that have helped my workflow more in a short time than I would have thought. It's easier to start with Implicit Buttons and then talk about Explicit ones.

Implicit Buttons

These are by far the easiest ones to understand... you have some in your codebase now! It's pretty basic - this, for example, is an Implicit Button:

~/Projects/my-project/ReadMe.md

Just put your cursor on that file path, hit M-return and if the file exists, it will open it in Emacs. "Wait," you say, "that's just a text filepath?" And you are right! This is what makes it implicit because it's just text link! Hyperbole tries to open up or follow any string that looks like a filepath. But what if just used a directory?

~/Projects/my-project

If you hit M-return on that, it will open up Dired in that folder, if the path exists! So, as I said above, M-return does what you expect it to do.

So "Well great, hyperlinks! So sorta like a wiki." and that is true! But wait... there is more! Let say you have this:

"!poetry run pytest"

So that seems like a shell command... and you are 100% correct! If you put your cursor between the quotes (which are important), hit M-return, Hyperbole will start a terminal in Emacs (ansi-term, in my case), cd to the folder that file is in, and run the command. But here is another one, with < >.

<describe-function 'describe-function>

"Wait, " you think, "is that Elisp?" Yes, yes it is. Hit M-return and watch the magic happen. Yes you can run arbitrary Elisp functions with Hyperbole anywhere in your file. This opens up a lot of possibilities. I quickly came up with a solution to something that always causes me some frustration: in Django, a View can return an HTML template, but that template file could be under the current directory or in the main template folder. So there can be a journey to figure out where it is. So I used Direnv to set TEMPLATE_PATH to a list of locations where my templates could be:

export TEMPLATE_PATH=$HOME/Projects/my-project/project/templates:$HOME/Projects/my-project/apps/app1/templates

and then I wrote this little Elisp function

(defun mh/dj-open-template-file (filename)
  "Use Emacs to find and open a file named FILENAME under TEMPLATE_PATH."
  (interactive "sEnter filename to search for: ")
  (let ((template-path (getenv "TEMPLATE_PATH")))
    (if template-path
        (let* ((files (directory-files-recursively template-path (regexp-quote filename)))
               (file-to-open (car files))) ; Just take the first match
          (if file-to-open
              (find-file-other-window file-to-open)
            (message "File not found under %s" template-path)))
      (message "TEMPLATE_PATH environment variable is not set"))))

Now in my Django views, I can put in this implicit button:

# <mh/dj-open-template-file 'home.html'>
return render(request, "home.html")

Then I can hit M-return in that comment, and another window opens in the frame with the home.html open. Similarly, you can do keystrokes:

{C-x 3 C-x o }

So that will just simply run those keystrokes to split vertically and switch to that other window.

Explicit Buttons

The Explicit Buttons are similar only they are formal and look like buttons. They are really made to run Elisp function easily (or, at least that is what I think). To make a button, you have to use the Hyperbole menu, which I haven't had to mention yet. C-h h e will bring up the Explicit Button Menu. From there, c will be created. I will use my mh/dj-open-template-file as an example.

So I would create a button with a label called home because that is the file that I want to load. Next, I would choose my elisp function (mh/dj-open-template-file) and then it will ask me for a parameter (because the function requires one) and I put in home.html. Then where I was in the file says to write <(home)>. There! That is the Explicit Button! Now, anywhere in that file, you can type <(home)> that button will be a way to run mh/dj-open-template-file to find home.html. Note that I said anywhere in that file... from what I have found the explicit buttons only exist in the current file. So if you want to use the same explicit button in another file, you have to re-create it with the same process. It's because of this I find explicit buttons to be less useful than implicit ones (why remake the same buttons over and over again?).

From a utility standpoint, the information about the explicit button is written to $PWD/.hypb in the same directory. It's not editable (and not meant to be) but that enables the buttons to persist after you restart Emacs.

Window/Frame control

This is something that I didn't think I would use much, but I found that I really like it. C-h h s w will bring up a menu to tweak all kinds of things about the current window -- size, position in the frame, change frame, and more! The similar C-h h s f then you have a similar thing for the frame. I find the Frame controls a lot more useful than the Window one. You can also create a grid of windows in the frame with the @

A lot is going on in this so check out the docs. But what is nice about this is that it makes small adjustments very quick and easy without taking your hands off the keyboard. Also makes it easy to load the file in a new frame, which is a command I can never remember.

Window Config

This name is similar to the above but it's very different. In essence you can save the opened buffers and the window formation to use later. So, if you are like me and get 3 different windows just so and then mess that all up when debugging, this is the fix for you! You can save it into a "window-ring" (that is my term) where you can just pop off the configs. Or you can save it by name and then just restore it by name later. You get to this by C-h h w.

There is a lot to be said when you have a bunch of windows saved in a circular ring and you just pop them off until you find the right one. Or save one by name so you can get at it again later. The big caveat to this, however, is that it doesn't persist. So you can set up a bunch of saved windows but lose them when you restart Emacs. Generally not a huge deal but I was excited to save some window configs in my Emacs configuration but no cigar.

My Config

There is a lot that I liked here and some that I didn't. And then there are some that I tweaked. :)

One of the things that didn't work is M-return didn't follow the definition in my Python code. I figured out that it had to do with Hyperbole assuming that I was using Etags and my Prelude config is using anaconda-mode. I did some digging and figured out what the function in Hyperbole is smart-python-tag and then I put some elisp advice around it:

(defun mh/override-smart-python-tag (original-function &rest args)
  "Conditionally override `smart-python-tag` to use `anaconda-mode-find-definitions` in Python mode."
  ;; Check if we're in python-mode and anaconda-mode is active
  (if (and (eq major-mode 'python-mode)
           (bound-and-true-p anaconda-mode))
      (call-interactively 'anaconda-mode-find-definitions) ;; Call anaconda-mode-find-definitions interactively
    (apply original-function args))) ;; Otherwise, call the original function


(advice-add 'smart-python-tag :around #'mh/override-smart-python-tag)

Now M-return works in Python like expected.

The next things are less drastic but also important to me. I liked the functionality with Windows and Frames but I didn't want to have to dig into the Hyperbole menu each time (C-h h w s to save a window? No thanks!). Of course, this is Emacs so I can configure things how I want. I do like Magit's Transient interface (and Hyperbole's menus essentially work the same way) I decided to dip my toes into making my own Tranient menu. And it was a lot easier than I thought:

(transient-define-prefix mh/hyperbole-transient ()
  ["Hyperbole commands"
   ("s" "save window" hywconfig-ring-save)
   ("y" "yank window" hywconfig-yank-pop)
   ("w" "window control" hycontrol-enable-windows-mode)
   ("f" "frame control" hycontrol-enable-frames-mode)
   ])

(global-set-key (kbd "C-h j") 'mh/hyperbole-transient)

Yes, I assigned it to C-h j, because it's close to Hyperbole. And it was an open keystroke on my system. Now when I use that, I have a list of Hyperbole options that I use the most.

Related, I wanted to be able to save a lot of windows and just move through them. Both my Transient and the Hyperbole menus quit after you get a window off the ring. What I needed was a Hydra -- something I had tried before but couldn't figure out a use. But I did now!

(defhydra mh/window-config-hydra (:color blue :hint nil)
  "
^Window Configurations^
-----------------------
_s_: Save Window Config  _y_: Yank Window Config
"
  ("s" hywconfig-ring-save "save" )
  ("y" hywconfig-yank-pop  "yank" :color red))


(global-set-key (kbd "C-c m") 'mh/window-config-hydra/body)

I have a hard time finding open key sequences but C-c m is fast and was unused. Now I can simply type that and hit y until I find the window that I want.

You can see my Hyperbole specific things (including my two Django-based functions I use for buttons) on my Gitlab page. You can look at the rest of my configuration as well, if that is useful.

Epilogue

When I started this post, I didn't plan on a 2,000+ words about Hyperbole but there is so much to talk about and there are things that I didn't even touch on like Koutliner, which is an outliner system sorta like OrgMode, and Hyrolo is a record management system (think Rolodex but apparently for anything). Most of these are because I haven't needed to dive into it.

If nothing else, I think the Hyperbole is a big package but you can use whatever piece you want. I use Hyperbole as a way to reduce some cognitive load while working, like making it easy to link filenames to their locations, quickly jump between windows, etc.

Category: programming Tags:

© Mike Hostetler 2016

Powered by Pelican