--- /dev/null
+Title: New address book
+Slug: new-address-book
+Date: 2017-06-27 12:57:27 +0100
+Tags: planet-debian, planet-ubuntu
+
+I've had a kludgy mess of electronic address books for most of two decades,
+and have got rather fed up with it. My stack consisted of:
+
+ * `~/.mutt/aliases`, a flat text file consisting of `mutt` `alias` commands
+ * [lbdb](http://www.spinnaker.de/lbdb/) configuration to query
+ `~/.mutt/aliases`, Debian's LDAP database, and Canonical's LDAP database,
+ so that I can search by name with Ctrl-t in `mutt` when composing a new
+ message
+ * Google Contacts, which I used from Android and was completely separate
+ from all of the above
+
+The biggest practical problem with this was that I had the address book that
+was most convenient for me to add things to (Google Contacts) and the one I
+used when sending email, and no sensible way to merge them or move things
+between them. I also wasn't especially comfortable with having all my
+contact information in a proprietary web service.
+
+My goals for a replacement address book system were:
+
+ * free software throughout
+ * storage under my control
+ * single common database
+ * minimal manual transcription when consolidating existing databases
+ * integration with Android such that I can continue using the same
+ contacts, messaging, etc. apps
+ * integration with `mutt` such that I can continue using the same query
+ interface
+ * not having to write my own software, because honestly
+
+I think I have all this now!
+
+## New stack
+
+The obvious basic technology to use is
+[CardDAV](https://en.wikipedia.org/wiki/CardDAV): it's fairly complex,
+admittedly, but lots of software supports it and one of my goals was not
+having to write my own thing. This meant I needed a CardDAV server, some
+way to sync the database to and from both Android and the system where I run
+`mutt`, and whatever query glue was necessary to get `mutt` to understand
+vCards.
+
+There are lots of different alternatives here, and if anything the problem
+was an embarrassment of choice. In the end I just decided to go for things
+that looked roughly the right shape for me and tried not to spend too much
+time in analysis paralysis.
+
+### CardDAV server
+
+I went with [Xandikos](https://www.jelmer.uk/xandikos-intro.html) for the
+server, largely because I know Jelmer and have generally had pretty good
+experiences with their software, but also because using Git for history of
+the backend storage seems like something my future self will thank me for.
+
+It isn't packaged in stretch, but it's in Debian unstable, so I installed it
+from there.
+
+Rather than the standalone mode suggested on the web page, I decided to set
+it up in what felt like a more robust way using WSGI. I installed `uwsgi`,
+`uwsgi-plugin-python3`, and `libapache2-mod-proxy-uwsgi`, and created the
+following file in `/etc/uwsgi/apps-available/xandikos.ini` which I then
+symlinked into `/etc/uwsgi/apps-enabled/xandikos.ini`:
+
+ :::ini
+ [uwsgi]
+ socket = 127.0.0.1:8801
+ uid = xandikos
+ gid = xandikos
+ umask = 022
+ master = true
+ cheaper = 2
+ processes = 4
+ plugin = python3
+ module = xandikos.wsgi:app
+ env = XANDIKOSPATH=/srv/xandikos/collections
+
+The port number was arbitrary, as was the path. You need to create the
+`xandikos` user and group first (`adduser --system --group --no-create-home
+--disabled-login xandikos`). I created `/srv/xandikos` owned by
+`xandikos:xandikos` and mode 0700, and I recommend [setting a
+umask](https://bugs.debian.org/866058) as shown above since uwsgi's default
+umask is 000 (!). You should also run `sudo -u xandikos xandikos -d
+/srv/xandikos/collections --autocreate` and then Ctrl-c it after a short
+time (I think it would be nicer if there were a way to [ask the WSGI wrapper
+to do this](https://bugs.debian.org/866093)).
+
+For Apache setup, I kept it reasonably simple: I ran `a2enmod proxy_uwsgi`,
+used `htpasswd` to create `/etc/apache2/xandikos.passwd` with a username and
+password for myself, added a virtual host in
+`/etc/apache2/sites-available/xandikos.conf`, and enabled it with `a2ensite
+xandikos`:
+
+ :::apache
+ <VirtualHost *:443>
+ ServerName xandikos.example.org
+ ServerAdmin me@example.org
+
+ ErrorLog /var/log/apache2/xandikos-error.log
+ TransferLog /var/log/apache2/xandikos-access.log
+
+ <Location />
+ ProxyPass "uwsgi://127.0.0.1:8801/"
+ AuthType Basic
+ AuthName "Xandikos"
+ AuthBasicProvider file
+ AuthUserFile "/etc/apache2/xandikos.passwd"
+ Require valid-user
+ </Location>
+ </VirtualHost>
+
+Then `service apache2 reload`, set the new virtual host up with [Let's
+Encrypt](https://letsencrypt.org/), reloaded again, and off we go.
+
+### Android integration
+
+I installed [DAVdroid](https://davdroid.bitfire.at/) from the Play Store: it
+cost a few pounds, but I was OK with that since it's GPLv3 and I'm happy to
+help fund free software. I created two accounts, one for my existing Google
+Contacts database (and in fact calendaring as well, although I don't intend
+to switch over to self-hosting that just yet), and one for the new Xandikos
+instance. The [Google
+setup](https://davdroid.bitfire.at/configuration/google/) was a bit fiddly
+because I have two-step verification turned on so I had to create an
+app-specific password. The Xandikos setup was straightforward: base URL,
+username, password, and done.
+
+Since I didn't completely trust the new setup yet, I followed what seemed
+like the most robust option from the [DAVdroid contacts syncing
+documentation](https://davdroid.bitfire.at/faq/entry/existing-contacts-are-not-synced/),
+and used the stock contacts app to export my Google Contacts account to a
+`.vcf` file and then import that into the appropriate DAVdroid account
+(which showed up automatically). This seemed straightforward and everything
+got pushed to Xandikos. There are some weird delays in syncing contacts
+that I don't entirely understand, but it all seems to get there in the end.
+
+### mutt integration
+
+First off I needed to sync the contacts. (In fact I happen to run `mutt` on
+the same system where I run Xandikos at the moment, but I don't want to rely
+on that, and going through the CardDAV server means that I don't have to
+poke holes for myself using filesystem permissions.) I used
+[vdirsyncer](https://vdirsyncer.pimutils.org/) for this. In
+`~/.vdirsyncer/config`:
+
+ :::ini
+ [general]
+ status_path = "~/.vdirsyncer/status/"
+
+ [pair contacts]
+ a = "contacts_local"
+ b = "contacts_remote"
+ collections = ["from a", "from b"]
+
+ [storage contacts_local]
+ type = "filesystem"
+ path = "~/.contacts/"
+ fileext = ".vcf"
+
+ [storage contacts_remote]
+ type = "carddav"
+ url = "<Xandikos base URL>"
+ username = "<my username>"
+ password = "<my password>"
+
+Running `vdirsyncer discover` and `vdirsyncer sync` then synced everything
+into `~/.contacts/`. I added an hourly `crontab` entry to run `vdirsyncer
+-v WARNING sync`.
+
+Next, I needed a command-line address book tool based on this.
+[khard](https://github.com/scheibler/khard) looked about right and is in
+stretch, so I installed that. In `~/.config/khard/khard.conf` (this is
+mostly just the example configuration, but I preferred to sort by first name
+since not all my contacts have neat first/last names):
+
+ :::ini
+ [addressbooks]
+ [[contacts]]
+ path = ~/.contacts/<UUID of my contacts collection>/
+
+ [general]
+ debug = no
+ default_action = list
+ editor = vim
+ merge_editor = vimdiff
+
+ [contact table]
+ # display names by first or last name: first_name / last_name
+ display = first_name
+ # group by address book: yes / no
+ group_by_addressbook = no
+ # reverse table ordering: yes / no
+ reverse = no
+ # append nicknames to name column: yes / no
+ show_nicknames = no
+ # show uid table column: yes / no
+ show_uids = yes
+ # sort by first or last name: first_name / last_name
+ sort = first_name
+
+ [vcard]
+ # extend contacts with your own private objects
+ # these objects are stored with a leading "X-" before the object name in the vcard files
+ # every object label may only contain letters, digits and the - character
+ # example:
+ # private_objects = Jabber, Skype, Twitter
+ private_objects = Jabber, Skype, Twitter
+ # preferred vcard version: 3.0 / 4.0
+ preferred_version = 3.0
+ # Look into source vcf files to speed up search queries: yes / no
+ search_in_source_files = no
+ # skip unparsable vcard files: yes / no
+ skip_unparsable = no
+
+Now `khard list` shows all my contacts. So far so good. Apparently there
+are some [awkward vCard compatibility
+issues](https://github.com/scheibler/khard#khard) with creating or modifying
+contacts from the `khard` end. I've tried adding one address from
+`~/.mutt/aliases` using `khard` and it seems to at least minimally work for
+me, but I haven't explored this very much yet.
+
+Finally, `mutt` integration. I already had `set query_command="lbdbq '%s'"`
+in `~/.muttrc`, and I wanted to keep that in place since I still wanted to
+use LDAP querying as well. I had to write a very small amount of code for
+this (perhaps I should contribute this to `lbdb` upstream?), in
+`~/.lbdb/modules/m_khard`:
+
+ :::sh
+ #! /bin/sh
+
+ m_khard_query () {
+ khard email --parsable --remove-first-line --search-in-source-files "$1"
+ }
+
+My full `~/.lbdb/rc` now reads as follows (you probably won't want the LDAP
+stuff, but I've included it here for completeness):
+
+ MODULES_PATH="$MODULES_PATH $HOME/.lbdb/modules"
+ METHODS='m_muttalias m_khard m_ldap'
+ LDAP_NICKS='debian canonical'
+
+## Next steps
+
+I've deleted one account from Google Contacts just to make sure that
+everything still works (e.g. I can still search for it when composing a new
+message), but I haven't yet deleted everything. I won't be adding anything
+new there though.
+
+I need to push everything from `~/.mutt/aliases` into the new system. This
+is only about 30 contacts so shouldn't take too long.
+
+Overall this feels like a big improvement! It wasn't a trivial amount of
+setup for just me, but it means I have both better usability for myself and
+more independence from proprietary services, and I think I can add extra
+users with much less effort if I need to.
~cjwatson / blog.git / commitdiff