Cluster Building - Ubuntu 12.04

This is an incomplete article, and has been SUPERCEDED BY http://burning-midnight.blogspot.com/2012/07/cluster-building-ubuntu-1204-revised.html

 After learning a considerable amount about the ins and outs of cluster configuration and upkeep, it's not unreasonable to move to CMAN.  I still have yet to understand why CMAN is so beneficial when Pacemaker seems to figure things out all by itself, but then that's my inexperience talking. 

 What To Do When The Romance Is Gone...

Well, it seems Canonical has decided that bugs are fixed via full release upgrades.  What does that mean for my clusters?  11.04 ain't gonna cut it.  11.10 will probably work, but 12.04 is now out also.  And since 12.04 is their new LTS target (if they even MEAN IT), I'll probably end up migrating to that.  Begrudgingly.

I ran into a nasty issue where iscsitarget-dkms, which is (from what I gather) required to get the ocf:heartbeat:iSCSITarget plugin to work, won't compile in 11.04.  Suck it, DKMS.  iSCSI was my golden scepter of greatness.  Now it's a trash-heap.

So, I've got one spare 11.04 machine (l5 is its name) going through a do-release-upgrade procedure.  We'll see how it fares.  Aside from a few pointed questions, it seemed to do its job rather successfully. YMMV.  It's a VM for me, so I don't care - I did it over SSH.

On to 12.04...

At least there is already a draft document outlining the major steps.  I'll write down as I go the things I do to bring my new l6 and l7 machines up to snuff.  Our goals, as usual: Pacemaker, Corosync, CMAN (probably), DRBD, OCFS2, iSCSI.  If things go well, we may also explore a full host migration, which would only be necessary for a MASSIVE upgrade or in the event of Total Meltdown.  Or just because it sounds like fun.

Install the essentials - make sure your package manager is up-to-date with the latest sources, or it will probably puke everywhere.
Both

apt-get install drbd8-utils iscsitarget ocfs2-tools pacemaker corosync libdlm3 openais ocfs2-tools-pacemaker iscsitarget-dkms lm-sensors ocfs2-tools-cman resource-agents fence-agents

Without CMAN...

I first started with creating some basic LVM volumes for all my resources.  I'll pretend we're gonna virtualize off these hosts, even though they themselves are virtuals.  Two devices will be needed:

• a datastore (ds00), to house vm images and their definitions
• an iscsi config share (is00)

Both

lvcreate -L+2G -n ds00 vg00; lvcreate -L+500M -n is00 vg00

After setting up the DRBD resources, I configured the corosync.conf file according to previous instructions.  Got rid of startup links as follows:

for X in {drbd,o2cb,ocfs2}; do update-rc.d -f ${X} disable; done

Don't forget to modify /etc/default/corosync and set START=yes.  As one of my two new 12.04 virtuals ate itself, I had to configure one first.  I pretty much copy/pasted my config from the old cluster to this new one.

At this point, I'm snagged and tired.  12.04 insists on letting resolvconf stomp my resolv.conf file - thanks!  Some of the dlm-pmck stuff is missing now, meaning I'll probably have to break here and configure CMAN.  As I mentioned before, l7 got nuked due to a bad virtual drive definition.  Reinstall is finally done, configuration will happen tomorrow.

Sigh.