How I manage Emacs packages: The Old, Modern, and Dream Way

Posted on

Though Emacs is one of the best text editors, I mean without any packages, nearly every emacser has a bunch of packages they collect which fits their needs.

With those packages, Emacs becomes the best text editor, no more "one of".

OK, Vim is a good editor, too. I had tried it before addicted to Emacs, but couldn't get used to it. I felt uncomfortable of the philosophy, the separatation of editing and operation, and the switch key ESC, which is really hard to type.

Someone may also say that Emacs is an operating system, it can be used to write programs on any languages, to read or send emails, to play games, even as a twitter client.

Like always, I see some interesting tools, Google whether Emacs can do so, then, turns out it can, just by some packages.

The more powerful Emacs is, the more huge number of packages it has. Someone may call it extensions, plugins, or add-on. In this post, I use "package" for generality.

I'll share some tips of how to manage those packages.

The Old Way

When I began to use Emacs, I searched a lot to find packages, not for a long time, I fell in love with emacswiki.

And just one or two months later, I got a habit: search, download, then configure, all according to emacswiki.

For files arragement, at first, all packages were put in the default .emacs.d folder, except for some single file packages, which I put in .emacs.d/site-lisp.

To be honest, collected those packages was not an easy job, also needed to configure them to fit my needs, I didn't want to lose them. Then I moved all of them to my Dropbox.

All contents of .emacs were moved to init.el, which was saved to my Dropbox folder, then, loaded init.el in the old .emacs using load-file.

It was really a great experience, I could use exactly same configurations and packages on my Macbook Pro and Ubuntu.

The Modern Way

Since 2010, I use git a lot, I considered put those files under git control, but some packages were not small, and nearly never got changed, version control them seemed work in vain.

I worked out one solution, version control the package file before I needed to change it, for example, version control the snippets before began to modify, then add a new version after changes were made in Yasnppet, it helped me a lot, to track the changes I made to the original packages.

When I first looked into mark-multiple (I blogged it here), I realized that git submodule might be a proper way to manage all my packages, because more and more Emacs packages were settled in Github.

Then I used git submodule to track all packages which can be found in Github. This is my .gitmodules file:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
[submodule "emacs.d/mark-multiple"]
	path = emacs.d/mark-multiple
	url = https://github.com/magnars/mark-multiple.el.git
[submodule "emacs.d/expand-region"]
	path = emacs.d/expand-region
	url = https://github.com/magnars/expand-region.el.git
[submodule "emacs.d/yasnippet"]
	path = emacs.d/yasnippet
	url = https://github.com/capitaomorte/yasnippet.git
[submodule "emacs.d/auto-complete"]
	path = emacs.d/auto-complete
	url = https://github.com/m2ym/auto-complete.git
[submodule "emacs.d/org-mode"]
	path = emacs.d/org-mode
	url = git://orgmode.org/org-mode.git
[submodule "emacs.d/emacs-nav"]
	path = emacs.d/emacs-nav
	url = https://github.com/ctrochalakis/emacs-nav.git
[submodule "emacs.d/emacs-java"]
	path = emacs.d/emacs-java
	url = https://github.com/skeeto/emacs-java.git

By adapting this strategy, I could keep using the newest feature of all packages in my submodule list, but it has some problems.

First, developing branch often has some bugs, most of the time, able to work is more important than trying newest features.

Second, git submodule is not easy to cope with.

** How to delete one submodule already added by git submodule add? **

You must remove submodule info in .gitmodules and .git/config, and then git rm --cached $module_path.

Even you git submodule add one repository, but didn't check in, you thought git checkout -- . will undo everything.

It's wrong, you must remove the .git/modules/package-name directory, or, you could never add the same submodule again.

The Dream Way

I don't like the apt-get package management in Ubuntu (Debian) and maven in Java dev, but I like Homebrew very much, also the ruby gem, they are neat and elegant, apt-get and maven seems kinda messy as my point of view.

When I first heard of Emacs package management, like el-get and package.el, I really didn't give a shit, I prefered to treat all packages as treasures of my own by managing them manually.

But last weekend, I persuaded myself to give it a try.

It opened a brand new world.

With it, packages themselves are no longer important, only the configurations are.

Delegate all things to package.el and version control the configurations is the dream way.

** How am I supposed to sync all packages installed in one machine to others? **

It can't be called the dream way if this problem can't be solved. Watch this,

1
2
3
4
(require 'package)
(package-initialize)
(add-to-list 'package-archives
             '("marmalade" . "http://marmalade-repo.org/packages/"))

Because official package repository doesn't have enough packages, it's better to add marmalade-repo to package-archives.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
(require 'cl)
;; Guarantee all packages are installed on start
(defvar packages-list
  '(rainbow-mode
    fill-column-indicator
    clojure-mode
    cursor-chg
    highlight-indentation
    highlight-symbol
    markdown-mode
    php-mode
    protobuf-mode
    rvm)
  "List of packages needs to be installed at launch")

(defun has-package-not-installed ()
  (loop for p in packages-list
        when (not (package-installed-p p)) do (return t)
        finally (return nil)))
(when (has-package-not-installed)
  ;; Check for new packages (package versions)
  (message "%s" "Get latest versions of all packages...")
  (package-refresh-contents)
  (message "%s" " done.")
  ;; Install the missing packages
  (dolist (p packages-list)
    (when (not (package-installed-p p))
      (package-install p))))

Below scripts gurantees all packages in package-list are installed when Emacs is started.

It first check is there any package not installed, if it has, install the package, or, nothing's missing, start Emacs as normal.

Doubt this would slow down Emacs start because the checking part? Come on, it's super fast, I tested.

It has another benefit, some packages you downloaded from package repository before might be shipped inside Emacs new version, it will automatically use the inside one, instead of fetching from package repository again. And you do not need to require them manually, since the packages ships auto-load, if you want to configure the package, use eval-after-load, this will decrease emacs starting time dramatically.

With this magic scripts, I no longer need git submodule, no longer need emacswiki, just list-packages, choose some packages I like, add them to my package-list, then restart Emacs, everything works so perfect.

I am kidding, if I could, I would not call it "The Dream Way".

  • Not enough packages in package repository

    There are lots of packages which is not changes for years, and authors of them don't care about them, or don't want to put them to package repositories like marmalade-repo.

    The active packages are almost all in Github, still, not every author put their stuffs to package repository, many of them still think package.el is weak and useless as I thought not so long ago.

  • Hard to get packages from package repository

    Packages download speed is much slower than apt-get or ruby gem.

    Even I need to wait a long time before start looking for packages by executing list-packages, it's annoy.

    When I first opened my Emacs in another machine using same configuration, the time cost to install all packages I need is huge, I tried today, Emacs hung for more than ten minutes, I closed it by force, and started again, another ten more minutes.

    So, I have to create a elpa-mirror link to the elpa folder created by package.el in .emacs.d/elpa, when I want a fast start, dropbox them, instead of fetching from online package repository.

Here I recommend us to use the package.el, and hope a new, fast, stable online package repository will appear soon.

Updated:

There is an terrific package repository rising these days, you do not want to miss it, I stopped using marmalade for the first time I saw MELPA.