Swift/Setup New Swift Cluster (labs)

This document is incomplete - it is a list of differences from Swift/Setup New Swift Cluster rather than a complete set of instructions.

Any heading that is empty should be used unaltered from the regular setup page.

update DNS

skip this step

Launch the instances

new step

• Create a project for your cluster (you may need help from a labs admin for this - ask in IRC
• Create a security group for the proxy in your project
  • accept traffic on port 80 (from everyone or a specific other project or ...)
• Launch the instances for your cluster - minimum one front end proxy and 3 back end storage nodes
  • m1.small is sufficient as the instance size
  • put the proxy in the proxy security group

Set up filesystems

Do this on all back end storage nodes

• unmount /dev/vdb from /mnt
• reformat it as xfs
  • sudo mkfs -t xfs -f -i size=512 -L swift-vdb /dev/vdb
• make the mountpoint swift expects
  • mkdir -p /srv/swift-storage/vdb
• mount the partition with the appropriate options
  • sudo mount /dev/vdb /srv/swift-storage/vdb/ -o rw,noatime,nodiratime,nobarrier,logbufs=8
• set the ownership to swift
  • sudo chown -R swift:swift /srv/swift-storage/vdb

create the cluster hash

update puppet

build the rings

When building the rings, there's only one device per storage node - /srv/swift-storage/vdb. This simplifies adding devices to the rings a little bit. This example has four storage nodes. Everything else about building the rings remains the same.

for num in {1..4}; do 
  host="swift-be${num}.pmtpa.wmflabs"; 
  hostip=$(dig +short $host); 
  zone="z${num}-${hostip}"; 
  weight=100; 
  for dev in vdb; do
     sudo swift-ring-builder account.builder add z${zone}:6002/${dev} $weight;
     sudo swift-ring-builder container.builder add z${zone}:6001/${dev} $weight;
     sudo swift-ring-builder object.builder add z${zone}:6000/${dev} $weight;
  done;
done
swift-ring-builder account.builder rebalance
swift-ring-builder container.builder rebalance
swift-ring-builder object.builder rebalance
chown swift:swift *.ring.gz 

distribute the rings

reboot

skip this step. instead:

• sudo swift-init all stop; sleep 1; sudo pkill swift; sleep 1; sudo swift-init all start

set up auth tokens for the cluster

tell puppet about the auth tokens

update the proxies and restart the proxy service

make the containers necessary for thumbnails

make the containers readable by anonymous users

test the cluster