20140528

The Importance of umask

This is subtle and covered in countless other places.  However, for the sake of it, this is a brief illustration of umask and its (non-)interaction with the "SGID" bit.  This can be especially relevant when setting up something like a Subversion server, with user access done via SSH.

I have created a directory called test.  It's empty, and owned by me and under the group svn (my default group).

admin@svnsrv:~/test$ ls -al
total 8
drwxr-xr-x 2 admin svn 4096 May 28 15:10 .
drwxr-xr-x 3 admin svn 4096 May 28 15:10 ..

I decide to use a different group, and I set the SGID bit so that all files and directories created under "test" will get the same group ID.  I also give group write permission.

admin@svnsrv:~/test$ chgrp sshusers .
admin@svnsrv:~/test$ chmod g+ws .
admin@svnsrv:~/test$ ls -al
total 8
drwxrwsr-x 2 admin sshusers 4096 May 28 15:10 .
drwxr-xr-x 3 admin svn      4096 May 28 15:10 ..

Now we can try making another directory.

admin@svnsrv:~/test$ mkdir foo
admin@svnsrv:~/test$ ls -la
total 12
drwxrwsr-x 3 admin sshusers 4096 May 28 15:10 .
drwxr-xr-x 3 admin svn      4096 May 28 15:10 ..
drwxr-sr-x 2 admin sshusers 4096 May 28 15:10 foo

As you can see, foo inherited the correct group ID, but did not inherit group write permission.  This is where umask comes in.  We first look at our current umask, and then set a new one:

admin@svnsrv:~/test$ umask
0022
admin@svnsrv:~/test$ umask 0002
admin@svnsrv:~/test$ mkdir bar
admin@svnsrv:~/test$ ls -la
total 16
drwxrwsr-x 4 admin sshusers 4096 May 28 15:10 .
drwxr-xr-x 3 admin svn      4096 May 28 15:10 ..
drwxrwsr-x 2 admin sshusers 4096 May 28 15:10 bar
drwxr-sr-x 2 admin sshusers 4096 May 28 15:10 foo
admin@svnsrv:~/test$ umask
0002

The latest directory was created with the right permissions.  It is important to note that permissions do not inherit (to the best of my knowledge), whereas ownership does.

umask itself is designed to mask out permission bits, so it is the logical negation of the permissions we want to ultimately allow.  This does not imply that files will be created with mode 775; rather, that if a file is created with 664, a mask of 002 will retain 664 whereas a mask of 022 will reduce permissions to 644.  Likewise, a file created 700 will retain 700 with both of these example umasks.  It is, exactly, a mask: applied as a logical AND (NOT mask).  It is not a mathematical subtraction.  As an exercise to the reader, look through this Wikipedia article for more info on permissions and then work out some examples by hand, applying masks using AND (NOT mask):

http://en.wikipedia.org/wiki/File_system_permissions#Symbolic_notation

And on that note, consider this final command and its result:

admin@svnsrv:~/test$ touch baz
admin@svnsrv:~/test$ ls -la
total 16
drwxrwsr-x 4 admin sshusers 4096 May 28 15:14 .
drwxr-xr-x 3 admin svn      4096 May 28 15:10 ..
drwxrwsr-x 2 admin sshusers 4096 May 28 15:10 bar
-rw-rw-r-- 1 admin sshusers    0 May 28 15:14 baz
drwxr-sr-x 2 admin sshusers 4096 May 28 15:10 foo

See?  No execute bits on that file, even though the umask is still 0002.

N.B. The initial zero is code for "octal."  In case you didn't realize we were using octal notation here...  In the big paragraph above, the modes themselves would be properly represented as 0775 etc.


20140527

ProxMox 3.x RAID-1 Transfiguration

Proxmox 3.2 RAID-1 Conversion - GPT Mode - With Bonus Features!

The latest Proxmox can install as GPT.  Not sure if we have any say in the matter.  BUT, in case you had a die-hard need to convert your Proxmox server to RAID-1, this is a cleaned-up set of instructions.  YMMV, make sure you backup your data, check your particular partition configuration, and READ CAREFULLY.  Also it is important to note that gdisk and sgdisk will be your friend, fdisk and sfdisk will not.

Basis: http://boffblog.wordpress.com/2013/08/22/how-to-install-proxmox-ve-3-0-on-software-raid/

EDIT and RAID-5 NOTE:
This site describes some changes you need to make to get RAID-5 working.  The below is good for RAID-1 only.  The important distinctions come with the initramfs modules.  I'll try to put together another post that sums all this up....or you can just go here:




Notes:
It is necessary to either have a valid license or to manually configure apt to use another set of repositories.  Otherwise, mdadm will not be available.  Details at:
https://pve.proxmox.com/wiki/Package_repositories

Instructions:

  • Duplicate partition info onto new drive:
    • sgdisk -b sda-part.txt /dev/sda
    • sgdisk -l sda-part.txt /dev/sdb
  • Configure partition types as RAID members:
    • sgdisk -t 2:fd00 -t 3:fd00 /dev/sdb
    • NOTE:  partition 1 must be type EF02 to support GRUB.  DO NOT TOUCH THIS PARTITION!
  • Create RAIDs:
    • boot
      • mdadm --create /dev/md0 -l 1 -n 2 missing /dev/sdb2
    • root
      • mdadm --create /dev/md1 -l 1 -n 2 missing /dev/sdb3
  • Copy /boot
    • mkfs.ext3 /dev/md0
    • mount /dev/md0 /mnt
    • rsync -avsHAXS /boot/ /mnt/
    • ## Update fstab with new boot mount: /dev/md0 (don’t use UUID)
    • reboot
  • Configure mdadm.conf
    • mdadm -D --brief /dev/md0 >> /etc/mdadm/mdadm.conf
    • mdadm -D --brief /dev/md1 >> /etc/mdadm/mdadm.conf
  • Update GRUB & initramfs
    • echo 'GRUB_DISABLE_LINUX_UUID=true' >> /etc/default/grub
    • echo 'GRUB_PRELOAD_MODULES="raid dmraid"' >> /etc/default/grub
    • echo raid1 >> /etc/modules
    • echo raid1 >> /etc/initramfs-tools/modules
    • grub-install /dev/sda
    • grub-install /dev/sdb
    • update-grub
    • update-initramfs -u
  • Convert BOOT partition to RAID-1
    • sgdisk -t 2:fd00 /dev/sda
    • mdadm --add /dev/md0 /dev/sda2
  • Migrate all logical extents to /dev/md1
    • pvcreate /dev/md1
    • vgextend pve /dev/md1
    • pvmove /dev/sda3 /dev/md1
    • …. wait a long time ….
    • vgreduce pve /dev/sda3
    • pvremove /dev/sda3
  • Convert sda3 to RAID
    • sgdisk -t 3:fd00 /dev/sda
    • mdadm --add /dev/md1 /dev/sda3
    • …. wait a long time ….
  • Reboot for final test - everything should work!
  • Configure monitoring, device scans, emails, etc
  • Configure SMART
  • SUCCESS!!

Now you have a server that can tell you a little sooner that it might die, and have a better chance of recovering from said-death.

Why This Works:
Ordinarily, pv-moving from one partition to another of the exact same size doesn’t work, except (in our case) when the partition isn’t actually 100% in use.  As it turns out, Proxmox leaves a little extra at the end, maybe for snapshots, and consequently we can move from our original PV to one that is 1 LE smaller without issue.

Using the above technique, it should be perfectly possible to migrate to a different array configuration, such as RAID-5 or RAID-6, if you have the drives for it.  Just make sure that you load the appropriate modules into GRUB and initramfs.  All other commands should be similar, and of course there will be more of them (because there will be more drives to configure).


Why You Might Want This:

Software RAID is not officially supported by Proxmox, so you are on your own there.  BUT, if you do trust mdadm (and who doesn't?), you can have your RAID and use it, too.  There are no hardware adapters to buy, or to fail, or to curse at when they fail to read the metadata from one or more of your disks during some random reboot.  If you are very concerned about data integrity, then you should be looking at ZFS-backed iSCSI stores instead of mucking around underneath Proxmox's hood.  That said, having this all running RAID-1 does make recovery a heck of a lot easier, and especially if you couple it with a ZFS iSCSI store for all your actual VM data.

And remember:  RAID is NOT a backup solution.  If you treat it like a backup solution, you have no one but yourself to blame when all your data gets toasted after a multi-drive failure, fool!