Network backup

How to Backup (step by step)

You will need to be on a wired connection for this to work

On the old machine/drive run the following commands in the terminal.

Read this before proceeding

• Use reverse ISO format for the date (e.g. 20121223 for Dec 23 2012) not American date format. Failing to do so may result in files being accidentally deleted.
• Always rsync the entire home directory even if there is only one user otherwise files and users may not be restored correctly.
• Errors are logged in the file rsync.log.

All these commands will need to be run as root (via sudo). Because sudo does not effect operations to the right of '>' dpkg --get-selections > /home/dpkg.out will fail if you try to run it as sudo on its own. To get around this it is wrapped in sh -c .

 cp /etc/passwd /etc/shadow /etc/group /home/
 sudo   sh -c "dpkg --get-selections > /home/dpkg.out"
 tar -czf   /home/etc.backup.tar.gz /etc/
 rsync -avzh /home/ tsbackup@tsbackup:/var/tsbackup/[date-ticketnumber] 2>rsync.log

This should back up enough to create an effective clone of the old machine.

How to Restore (step by step)

On the new machine/drive run the following commands in the terminal.

All these commands will need to be run via sudo:

 sudo rsync -avzh  tsbackup@tsbackup:/var/tsbackup/[date-ticketnumber]/ /home
 sudo dpkg --set-selections < /home/dpkg.out
 sudo apt-get -u dselect-upgrade
 sudo cp /home/passwd /home/shadow /home/group /etc/  <-- breaks things DO NOT DO THIS UNLESS YOU KNOW IT WILL WORK

The last line restores the original password files etc. Before that you can operate as oem (as root) but it might not work properly after this stage. Don't forget to check to see if file ownership on the home directories matches properly. It might break things. To make sure it doesn't compare the version in /home and the installed version before copying over. You will need to make sure that UID and GUID (the numbers in passwd and group(users start at 1000) match up, especially for system users. Probably a better way is to edit the in place /etc/passwd and /etc/shadow adding the lines from /home/password and /home/shadow that correspond to the users on the original system. Then edit /etc/shadow to make sure they are in the relevant groups.

Reference

The sections below are included only to explain the preceding commands; please consider the above procedure definitive.

Backing up User Accounts

The three files you usually need are:

• /etc/passwd (list of users and home directories)
• /etc/shadow (the encrypted password file)
• /etc/group (list of groups and what users are in them)

Copy these into the home folder to make sure you remember to back them up:

sudo cp /etc/passwd /etc/shadow /etc/group /home/

It can be good practice to back up the whole /etc directory and /var/www as well, but its beyond the scope of what we do. The best practice is to keep the drive around for the users to check afterwards if possible.

Backup and Restore Installed Packages

You can also generate a list of the user's installed packages so you can reinstall them all at once later using a single command.

sudo dpkg --get-selections > dpkg.out 

Don't forget to include this file in the backup!

Then, when you have installed a new system/replaced the box/etc, you can do:

sudo dpkg --set-selections < dpkg.out

then

sudo apt-get -u dselect-upgrade

on the new box to install the same set of packages as before.

(This can be done over a network using ssh, but setting up ssh is beyond the scope of this document).

Backup and Restore User Data

To back up the home folder(s) over the network using rsync (don't actually be root or else funny things happen).

sudo rsync -avzh /home/ tsbackup@tsbackup:/var/tsbackup/[date-ticketnumber] 2>rsync.out

N.B.

• Use reverse ISO format for the date (e.g. 20121223 for Dec 23 2012) not American date format. Failing to do so may result in files being accidentally deleted.
• Always rsync the entire home directory even if there is only one user otherwise files and users may not be restored correctly.
• This is our standard: the date part so that we don't accidentally overwrite existing directories on the backup box; the entire home directory so that we can ensure that all of the files that were backed up are restored.
• Errors will be logged to file rsync.out

To copy it back again:

sudo rsync -avzh  tsbackup@tsbackup:/var/tsbackup/[date-ticketnumber]/ /home

N.B. The trailing slash on the source (first part) is important in rsync. A trailing slash means copy the contents of this folder without creating the folder itself. If you are uncertain about anything you can test it using the -n option. This will do a dry run and nothing will actually be copied over. It is a good habit to do a sanity check first with rsync; it is an immensely powerful tool and you want to make sure it is going to do what you think it is.

Rysnc is a more efficient protocol to use than scp. It will ensure file permissions and ownerships are preserved and will compress data on the fly to make for a faster transfer. (it will also only copy the needed files so if you add files, then you can run the command again and rsync will only copy the additional files. You can also remove files and use the --delete option and it will remove the files on the remote host).

If you don't want to copy hidden files add --exclude=".*/" as an option. Its a good idea to at least copy over users .mozilla files, and any .evolution or .thunderbird folders as a minimum. It shouldn't cause too many problems copying over all hidden files.

For even more information, you can have a look at

man rsync

Backup and Restore User's Desktop Background

You can also restore the user's desktop background (as long as you have the file backed up as well. To check, run the command without >desktop.pict).

gconftool-2 --get /desktop/gnome/background/picture_filename >desktop.pict

copy over desktop.pict and on the new machine

gconftool-2 --type string --set  /desktop/gnome/background/picture_filename "$(cat desktop.pict)"

N.B. copying over a user's /home folder wholesale with all the hidden files and folders will do this anyway.

Recreating User Accounts and Restoring Permissions

If you backed up the User Accounts from the old system as detailed above, you can simply restore those files to /etc now, and you'll have the same user/group setup as on the old system. Just do:

sudo cp /home/passwd /home/shadow /home/group /etc/

If you didn't back them up, you will need to recreate those accounts manually. For each username, do:

adduser [username]

You might need to change the ownership to match that of the user if you find that the ownership and groups have changed to a numeric value (check using ls -lh).

• For multiple users, do:

find /home/username/ -type f -exec chown -R username.username {} \;

'Remember to include the semicolon at the end.'

• For a single user, you can just do:

chown -R username.username /home/username/

You might also need to apply proper permissions, though this step should be unnecessary if you used rsync.

find /home/username/ -type f -exec chmod 644 {} \;
find /home/username/ -type d -exec chmod 754 {} \;

or, for more private users:

find ./ -type f -exec chmod 640 {} \;
find ./ -type d -exec chmod 750 {} \;

When you are finished with a backup and you've made sure that all the data is properly transferred and that the machine works, you can (optionally) remove your backup from tsbackup:

rm -r /var/tsbackup/[date-ticketnumber]