From 9e2de243163b75f8408b3c85cd88a62b843dd139 Mon Sep 17 00:00:00 2001 From: Colin Watson Date: Tue, 4 Dec 2018 01:37:55 +0000 Subject: [PATCH] Deploying Swift --- content/deploying-swift.md | 232 +++++++++++++++++++++++++++++++++++++ 1 file changed, 232 insertions(+) create mode 100644 content/deploying-swift.md diff --git a/content/deploying-swift.md b/content/deploying-swift.md new file mode 100644 index 00000000..fd4be7c0 --- /dev/null +++ b/content/deploying-swift.md @@ -0,0 +1,232 @@ +Title: Deploying Swift +Slug: deploying-swift +Date: 2018-12-04 01:37:11 +00:00 +Category: ubuntu +Tags: juju, openstack, ubuntu, planet-debian, planet-ubuntu + +Sometimes I want to deploy [Swift](https://docs.openstack.org/swift/), the +OpenStack object storage system. + +Well, no, that's not true. I basically never actually want to deploy Swift +as such. What I generally want to do is to debug some bit of production +service deployment machinery that relies on Swift for getting build +artifacts into the right place, or maybe the parts of the +[Launchpad](https://launchpad.net/) librarian (our blob storage service) +that use Swift. I could find an existing private or public cloud that +offers the right API and test with that, but sometimes I need to test with +particular versions, and in any case I have a terribly slow internet +connection and shuffling large build artifacts back and forward over the +relevant bit of wet string makes it painfully slow to test things. + +For a while I've had an Ubuntu 12.04 VM lying around with an +[Icehouse](https://releases.openstack.org/icehouse/)-based Swift deployment +that I put together by hand. It works, but I didn't keep good notes and +have no real idea how to reproduce it, not that I really want to keep +limping along with manually-constructed VMs for this kind of thing anyway; +and I don't want to be dependent on obsolete releases forever. For the +sorts of things I'm doing I need to make sure that authentication works +broadly the same way as it does in a real production deployment, so I want +to have [Keystone](https://docs.openstack.org/keystone/) too. At the same +time, I definitely don't want to do anything close to a full OpenStack +deployment of my own: it's much too big a sledgehammer for this particular +nut, and I don't really have the hardware for it. + +Here's my solution to this, which is compact enough that I can run it on my +laptop, and while it isn't completely automatic it's close enough that I can +spin it up for a test and discard it when I'm finished (so I haven't worried +very much about producing something that runs efficiently). It relies on +[Juju](https://docs.jujucharms.com/) and +[LXD](https://linuxcontainers.org/lxd/). I've only tested it on Ubuntu +18.04, using [Queens](https://releases.openstack.org/queens/); for anything +else you're on your own. In general, I probably can't help you if you run +into trouble with the directions here: this is provided "as is", without +warranty of any kind, and all that kind of thing. + +First, install Juju and LXD if necessary, following the instructions +provided by those projects, and also install the `python-openstackclient` +package as you'll need it later. You'll want to [set Juju up to use +LXD](https://docs.jujucharms.com/2.4/en/tut-lxd), and you should probably +make sure that the shells you're working in don't have `http_proxy` set as +it's quite likely to confuse things unless you've arranged for your proxy to +be able to cope with your local LXD containers. Then add a +[model](https://docs.jujucharms.com/2.4/en/juju-concepts#model): + + :::sh + juju add-model swift + +At this point there's a bit of complexity that you normally don't have to +worry about with Juju. The [swift-storage +charm](https://jujucharms.com/swift-storage) wants to mount something to use +for storage, which with the LXD provider in practice ends up being some kind +of loopback mount. Unfortunately, being able to perform loopback mounts +exposes too much kernel attack surface, so LXD doesn't allow unprivileged +containers to do it. +([Ideally](https://bugs.launchpad.net/charm-swift-storage/+bug/1250965) the +swift-storage charm would just let you use directory storage instead.) To +make the containers we're about to create privileged enough for this to +work, run: + + :::sh + lxc profile set juju-swift security.privileged true + lxc profile device add juju-swift loop-control unix-char \ + major=10 minor=237 path=/dev/loop-control + for i in $(seq 0 255); do + lxc profile device add juju-swift loop$i unix-block \ + major=7 minor=$i path=/dev/loop$i + done + +Now we can start deploying things! Save this to a file, e.g. +`swift.bundle`: + + :::yaml + series: bionic + description: "Swift in a box" + applications: + mysql: + charm: "cs:mysql-62" + channel: candidate + num_units: 1 + options: + dataset-size: 512M + keystone: + charm: "cs:keystone" + num_units: 1 + swift-storage: + charm: "cs:swift-storage" + num_units: 1 + options: + block-device: "/etc/swift/storage.img|5G" + swift-proxy: + charm: "cs:swift-proxy" + num_units: 1 + options: + zone-assignment: auto + replicas: 1 + relations: + - ["keystone:shared-db", "mysql:shared-db"] + - ["swift-proxy:swift-storage", "swift-storage:swift-storage"] + - ["swift-proxy:identity-service", "keystone:identity-service"] + +And run: + + :::sh + juju deploy swift.bundle + +This will take a while. You can run `juju status` to see how it's going in +general terms, or `juju debug-log` for detailed logs from the individual +containers as they're putting themselves together. When it's all done, it +should look something like this: + + Model Controller Cloud/Region Version SLA + swift lxd localhost 2.3.1 unsupported + + App Version Status Scale Charm Store Rev OS Notes + keystone 13.0.1 active 1 keystone jujucharms 290 ubuntu + mysql 5.7.24 active 1 mysql jujucharms 62 ubuntu + swift-proxy 2.17.0 active 1 swift-proxy jujucharms 75 ubuntu + swift-storage 2.17.0 active 1 swift-storage jujucharms 250 ubuntu + + Unit Workload Agent Machine Public address Ports Message + keystone/0* active idle 0 10.36.63.133 5000/tcp Unit is ready + mysql/0* active idle 1 10.36.63.44 3306/tcp Ready + swift-proxy/0* active idle 2 10.36.63.75 8080/tcp Unit is ready + swift-storage/0* active idle 3 10.36.63.115 Unit is ready + + Machine State DNS Inst id Series AZ Message + 0 started 10.36.63.133 juju-d3e703-0 bionic Running + 1 started 10.36.63.44 juju-d3e703-1 bionic Running + 2 started 10.36.63.75 juju-d3e703-2 bionic Running + 3 started 10.36.63.115 juju-d3e703-3 bionic Running + +At this point you have what should be a working installation, but with only +administrative privileges set up. Normally you want to create at least one +normal user. To do this, start by creating a configuration file granting +administrator privileges (this one comes verbatim from the [openstack-base +bundle](https://api.jujucharms.com/charmstore/v5/openstack-base/archive/openrc)): + + :::sh + _OS_PARAMS=$(env | awk 'BEGIN {FS="="} /^OS_/ {print $1;}' | paste -sd ' ') + for param in $_OS_PARAMS; do + if [ "$param" = "OS_AUTH_PROTOCOL" ]; then continue; fi + if [ "$param" = "OS_CACERT" ]; then continue; fi + unset $param + done + unset _OS_PARAMS + + _keystone_unit=$(juju status keystone --format yaml | \ + awk '/units:$/ {getline; gsub(/:$/, ""); print $1}') + _keystone_ip=$(juju run --unit ${_keystone_unit} 'unit-get private-address') + _password=$(juju run --unit ${_keystone_unit} 'leader-get admin_passwd') + + export OS_AUTH_URL=${OS_AUTH_PROTOCOL:-http}://${_keystone_ip}:5000/v3 + export OS_USERNAME=admin + export OS_PASSWORD=${_password} + export OS_USER_DOMAIN_NAME=admin_domain + export OS_PROJECT_DOMAIN_NAME=admin_domain + export OS_PROJECT_NAME=admin + export OS_REGION_NAME=RegionOne + export OS_IDENTITY_API_VERSION=3 + # Swift needs this: + export OS_AUTH_VERSION=3 + # Gnocchi needs this + export OS_AUTH_TYPE=password + +Source this into a shell: for instance, if you saved this to +`~/.swiftrc.juju-admin`, then run: + + . ~/.swiftrc.juju-admin + +You should now be able to run `openstack endpoint list` and see a table for +the various services exposed by your deployment. Then you can create a +dummy project and a user with enough privileges to use Swift: + + :::sh + USERNAME=your-username + PASSWORD=your-password + openstack domain create SwiftDomain + openstack project create --domain SwiftDomain --description Swift \ + SwiftProject + openstack user create --domain SwiftDomain --project-domain SwiftDomain \ + --project SwiftProject --password "$PASSWORD" "$USERNAME" + openstack role add --project SwiftProject --user-domain SwiftDomain \ + --user "$USERNAME" Member + +(This is intended for testing rather than for doing anything particularly +sensitive. If you cared about keeping the password secret then you'd use +the `--password-prompt` option to `openstack user create` instead of +supplying the password on the command line.) + +Now create a configuration file granting privileges for the user you just +created. I felt like automating this to at least some degree: + + touch ~/.swiftrc.juju + chmod 600 ~/.swiftrc.juju + sed '/^_password=/d; + s/\( OS_PROJECT_DOMAIN_NAME=\).*/\1SwiftDomain/; + s/\( OS_PROJECT_NAME=\).*/\1SwiftProject/; + s/\( OS_USER_DOMAIN_NAME=\).*/\1SwiftDomain/; + s/\( OS_USERNAME=\).*/\1'"$USERNAME"'/; + s/\( OS_PASSWORD=\).*/\1'"$PASSWORD"'/' \ + <~/.swiftrc.juju-admin >~/.swiftrc.juju + +Source this into a shell. For example: + + . ~/.swiftrc.juju + +You should now find that `swift list` works. Success! Now you can `swift +upload` files, or just start testing whatever it was that you were actually +trying to test in the first place. + +This is not a setup I expect to leave running for a long time, so to tear it +down again: + + juju destroy-model swift + +This will probably get stuck trying to remove the `swift-storage` unit, +since nothing deals with detaching the loop device. If that happens, find +the relevant device in `losetup -a` from another window and use `losetup -d` +to detach it; `juju destroy-model` should then be able to proceed. + +Credit to the Juju and LXD teams and to the maintainers of the various +charms used here, as well as of course to the OpenStack folks: their work +made it very much easier to put this together. -- 2.30.2