From: Colin Watson Date: Tue, 27 Jun 2017 11:58:52 +0000 (+0100) Subject: New address book X-Git-Url: https://www.chiark.greenend.org.uk/ucgi/~cjwatson/git?a=commitdiff_plain;h=663b427f83d920053c06ba3d87d46b404a488fab;p=blog.git New address book --- diff --git a/content/new-address-book.md b/content/new-address-book.md new file mode 100644 index 00000000..830b9d4d --- /dev/null +++ b/content/new-address-book.md @@ -0,0 +1,258 @@ +Title: New address book +Slug: new-address-book +Date: 2017-06-27 12:57:27 +0100 +Tags: planet-debian, planet-ubuntu + +I've had a kludgy mess of electronic address books for most of two decades, +and have got rather fed up with it. My stack consisted of: + + * `~/.mutt/aliases`, a flat text file consisting of `mutt` `alias` commands + * [lbdb](http://www.spinnaker.de/lbdb/) configuration to query + `~/.mutt/aliases`, Debian's LDAP database, and Canonical's LDAP database, + so that I can search by name with Ctrl-t in `mutt` when composing a new + message + * Google Contacts, which I used from Android and was completely separate + from all of the above + +The biggest practical problem with this was that I had the address book that +was most convenient for me to add things to (Google Contacts) and the one I +used when sending email, and no sensible way to merge them or move things +between them. I also wasn't especially comfortable with having all my +contact information in a proprietary web service. + +My goals for a replacement address book system were: + + * free software throughout + * storage under my control + * single common database + * minimal manual transcription when consolidating existing databases + * integration with Android such that I can continue using the same + contacts, messaging, etc. apps + * integration with `mutt` such that I can continue using the same query + interface + * not having to write my own software, because honestly + +I think I have all this now! + +## New stack + +The obvious basic technology to use is +[CardDAV](https://en.wikipedia.org/wiki/CardDAV): it's fairly complex, +admittedly, but lots of software supports it and one of my goals was not +having to write my own thing. This meant I needed a CardDAV server, some +way to sync the database to and from both Android and the system where I run +`mutt`, and whatever query glue was necessary to get `mutt` to understand +vCards. + +There are lots of different alternatives here, and if anything the problem +was an embarrassment of choice. In the end I just decided to go for things +that looked roughly the right shape for me and tried not to spend too much +time in analysis paralysis. + +### CardDAV server + +I went with [Xandikos](https://www.jelmer.uk/xandikos-intro.html) for the +server, largely because I know Jelmer and have generally had pretty good +experiences with their software, but also because using Git for history of +the backend storage seems like something my future self will thank me for. + +It isn't packaged in stretch, but it's in Debian unstable, so I installed it +from there. + +Rather than the standalone mode suggested on the web page, I decided to set +it up in what felt like a more robust way using WSGI. I installed `uwsgi`, +`uwsgi-plugin-python3`, and `libapache2-mod-proxy-uwsgi`, and created the +following file in `/etc/uwsgi/apps-available/xandikos.ini` which I then +symlinked into `/etc/uwsgi/apps-enabled/xandikos.ini`: + + :::ini + [uwsgi] + socket = 127.0.0.1:8801 + uid = xandikos + gid = xandikos + umask = 022 + master = true + cheaper = 2 + processes = 4 + plugin = python3 + module = xandikos.wsgi:app + env = XANDIKOSPATH=/srv/xandikos/collections + +The port number was arbitrary, as was the path. You need to create the +`xandikos` user and group first (`adduser --system --group --no-create-home +--disabled-login xandikos`). I created `/srv/xandikos` owned by +`xandikos:xandikos` and mode 0700, and I recommend [setting a +umask](https://bugs.debian.org/866058) as shown above since uwsgi's default +umask is 000 (!). You should also run `sudo -u xandikos xandikos -d +/srv/xandikos/collections --autocreate` and then Ctrl-c it after a short +time (I think it would be nicer if there were a way to [ask the WSGI wrapper +to do this](https://bugs.debian.org/866093)). + +For Apache setup, I kept it reasonably simple: I ran `a2enmod proxy_uwsgi`, +used `htpasswd` to create `/etc/apache2/xandikos.passwd` with a username and +password for myself, added a virtual host in +`/etc/apache2/sites-available/xandikos.conf`, and enabled it with `a2ensite +xandikos`: + + :::apache + + ServerName xandikos.example.org + ServerAdmin me@example.org + + ErrorLog /var/log/apache2/xandikos-error.log + TransferLog /var/log/apache2/xandikos-access.log + + + ProxyPass "uwsgi://127.0.0.1:8801/" + AuthType Basic + AuthName "Xandikos" + AuthBasicProvider file + AuthUserFile "/etc/apache2/xandikos.passwd" + Require valid-user + + + +Then `service apache2 reload`, set the new virtual host up with [Let's +Encrypt](https://letsencrypt.org/), reloaded again, and off we go. + +### Android integration + +I installed [DAVdroid](https://davdroid.bitfire.at/) from the Play Store: it +cost a few pounds, but I was OK with that since it's GPLv3 and I'm happy to +help fund free software. I created two accounts, one for my existing Google +Contacts database (and in fact calendaring as well, although I don't intend +to switch over to self-hosting that just yet), and one for the new Xandikos +instance. The [Google +setup](https://davdroid.bitfire.at/configuration/google/) was a bit fiddly +because I have two-step verification turned on so I had to create an +app-specific password. The Xandikos setup was straightforward: base URL, +username, password, and done. + +Since I didn't completely trust the new setup yet, I followed what seemed +like the most robust option from the [DAVdroid contacts syncing +documentation](https://davdroid.bitfire.at/faq/entry/existing-contacts-are-not-synced/), +and used the stock contacts app to export my Google Contacts account to a +`.vcf` file and then import that into the appropriate DAVdroid account +(which showed up automatically). This seemed straightforward and everything +got pushed to Xandikos. There are some weird delays in syncing contacts +that I don't entirely understand, but it all seems to get there in the end. + +### mutt integration + +First off I needed to sync the contacts. (In fact I happen to run `mutt` on +the same system where I run Xandikos at the moment, but I don't want to rely +on that, and going through the CardDAV server means that I don't have to +poke holes for myself using filesystem permissions.) I used +[vdirsyncer](https://vdirsyncer.pimutils.org/) for this. In +`~/.vdirsyncer/config`: + + :::ini + [general] + status_path = "~/.vdirsyncer/status/" + + [pair contacts] + a = "contacts_local" + b = "contacts_remote" + collections = ["from a", "from b"] + + [storage contacts_local] + type = "filesystem" + path = "~/.contacts/" + fileext = ".vcf" + + [storage contacts_remote] + type = "carddav" + url = "" + username = "" + password = "" + +Running `vdirsyncer discover` and `vdirsyncer sync` then synced everything +into `~/.contacts/`. I added an hourly `crontab` entry to run `vdirsyncer +-v WARNING sync`. + +Next, I needed a command-line address book tool based on this. +[khard](https://github.com/scheibler/khard) looked about right and is in +stretch, so I installed that. In `~/.config/khard/khard.conf` (this is +mostly just the example configuration, but I preferred to sort by first name +since not all my contacts have neat first/last names): + + :::ini + [addressbooks] + [[contacts]] + path = ~/.contacts// + + [general] + debug = no + default_action = list + editor = vim + merge_editor = vimdiff + + [contact table] + # display names by first or last name: first_name / last_name + display = first_name + # group by address book: yes / no + group_by_addressbook = no + # reverse table ordering: yes / no + reverse = no + # append nicknames to name column: yes / no + show_nicknames = no + # show uid table column: yes / no + show_uids = yes + # sort by first or last name: first_name / last_name + sort = first_name + + [vcard] + # extend contacts with your own private objects + # these objects are stored with a leading "X-" before the object name in the vcard files + # every object label may only contain letters, digits and the - character + # example: + # private_objects = Jabber, Skype, Twitter + private_objects = Jabber, Skype, Twitter + # preferred vcard version: 3.0 / 4.0 + preferred_version = 3.0 + # Look into source vcf files to speed up search queries: yes / no + search_in_source_files = no + # skip unparsable vcard files: yes / no + skip_unparsable = no + +Now `khard list` shows all my contacts. So far so good. Apparently there +are some [awkward vCard compatibility +issues](https://github.com/scheibler/khard#khard) with creating or modifying +contacts from the `khard` end. I've tried adding one address from +`~/.mutt/aliases` using `khard` and it seems to at least minimally work for +me, but I haven't explored this very much yet. + +Finally, `mutt` integration. I already had `set query_command="lbdbq '%s'"` +in `~/.muttrc`, and I wanted to keep that in place since I still wanted to +use LDAP querying as well. I had to write a very small amount of code for +this (perhaps I should contribute this to `lbdb` upstream?), in +`~/.lbdb/modules/m_khard`: + + :::sh + #! /bin/sh + + m_khard_query () { + khard email --parsable --remove-first-line --search-in-source-files "$1" + } + +My full `~/.lbdb/rc` now reads as follows (you probably won't want the LDAP +stuff, but I've included it here for completeness): + + MODULES_PATH="$MODULES_PATH $HOME/.lbdb/modules" + METHODS='m_muttalias m_khard m_ldap' + LDAP_NICKS='debian canonical' + +## Next steps + +I've deleted one account from Google Contacts just to make sure that +everything still works (e.g. I can still search for it when composing a new +message), but I haven't yet deleted everything. I won't be adding anything +new there though. + +I need to push everything from `~/.mutt/aliases` into the new system. This +is only about 30 contacts so shouldn't take too long. + +Overall this feels like a big improvement! It wasn't a trivial amount of +setup for just me, but it means I have both better usability for myself and +more independence from proprietary services, and I think I can add extra +users with much less effort if I need to.