Network backup
How to Backup (step by step)
You will need to be on a wired connection for this to work
On the old machine/drive run the following commands in the terminal.
Read this before proceeding
- Use reverse ISO format for the date (e.g. 20121223 for Dec 23 2012) not American date format. Failing to do so may result in files being accidentally deleted.
- Always rsync the entire home directory even if there is only one user otherwise files and users may not be restored correctly.
- Errors are logged in the file rsync.log.
All these commands will need to be run as root (via sudo). Because sudo does not effect operations to the right of '>' dpkg --get-selections > /home/dpkg.out will fail if you try to run it as sudo on its own. To get around this out put the file and move it in place .
sudo cp /etc/passwd /etc/shadow /etc/group /home/ dpkg --get-selections > dpkg.out sudo mv dpkg.out /home/ sudo tar -czf /home/etc.backup.tar.gz /etc/ sudo rsync -avzh /home/ tsbackup@tsbackup:/var/tsbackup/[date-ticketnumber] 2>rsync.log
This should back up enough to create an effective clone of the old machine.
Note: on the data recovery box you will need to add /mnt/sdb1 in front of /home. For the second command use
dpkg --get-selections --admindir=/mnt/sdb1/var/lib/dpkg >dpkg.out
this avoids the need to chroot.
there is also an --installdir=[targetdir] option that will allow you to install packages elsewhere the directory you point to is treated as the root of the file system. The --root option affects both the admindir, where package info is taken from, and the target directory for installs.
How to Restore (step by step)
On the new machine/drive run the following commands in the terminal.
All these commands will need to be run via sudo:
sudo rsync -avzh tsbackup@tsbackup:/var/tsbackup/[date-ticketnumber]/ /home sudo dpkg --set-selections < /home/dpkg.out sudo apt-get -u dselect-upgrade sudo cp /home/passwd /home/shadow /home/group /etc/ <-- breaks things DO NOT DO THIS UNLESS YOU KNOW IT WILL WORK
The last line restores the original password files etc. Before that you can operate as oem (as root) but it might not work properly after this stage. Don't forget to check to see if file ownership on the home directories matches properly. It might break things. To make sure it doesn't compare the version in /home and the installed version before copying over. You will need to make sure that UID and GUID (the numbers in passwd and group(users start at 1000) match up, especially for system users. Probably a better way is to edit the in place /etc/passwd and /etc/shadow adding the lines from /home/password and /home/shadow that correspond to the users on the original system. Then edit /etc/shadow to make sure they are in the relevant groups.
Reference
The sections below are included only to explain the preceding commands; please consider the above procedure definitive.
Backing up User Accounts
The three files you usually need are:
- /etc/passwd (list of users and home directories)
- /etc/shadow (the encrypted password file)
- /etc/group (list of groups and what users are in them)
Copy these into the home folder to make sure you remember to back them up:
sudo cp /etc/passwd /etc/shadow /etc/group /home/
It can be good practice to back up the whole /etc directory and /var/www as well, but its beyond the scope of what we do. The best practice is to keep the drive around for the users to check afterwards if possible.
Backup and Restore Installed Packages
You can also generate a list of the user's installed packages so you can reinstall them all at once later using a single command.
sudo dpkg --get-selections > dpkg.out
Don't forget to include this file in the backup!
Then, when you have installed a new system/replaced the box/etc, you can do:
sudo dpkg --set-selections < dpkg.out
then
sudo apt-get -u dselect-upgrade
on the new box to install the same set of packages as before.
(This can be done over a network using ssh, but setting up ssh is beyond the scope of this document).
Backup and Restore User Data
To back up the home folder(s) over the network using rsync (don't actually be root or else funny things happen).
sudo rsync -avzh /home/ tsbackup@tsbackup:/var/tsbackup/[date-ticketnumber] 2>rsync.out
N.B.
- Use reverse ISO format for the date (e.g. 20121223 for Dec 23 2012) not American date format. Failing to do so may result in files being accidentally deleted.
- Always rsync the entire home directory even if there is only one user otherwise files and users may not be restored correctly.
- This is our standard: the date part so that we don't accidentally overwrite existing directories on the backup box; the entire home directory so that we can ensure that all of the files that were backed up are restored.
- Errors will be logged to file rsync.out
To copy it back again:
sudo rsync -avzh tsbackup@tsbackup:/var/tsbackup/[date-ticketnumber]/ /home
N.B. The trailing slash on the source (first part) is important in rsync. A trailing slash means copy the contents of this folder without creating the folder itself. If you are uncertain about anything you can test it using the -n option. This will do a dry run and nothing will actually be copied over. It is a good habit to do a sanity check first with rsync; it is an immensely powerful tool and you want to make sure it is going to do what you think it is.
Rysnc is a more efficient protocol to use than scp. It will ensure file permissions and ownerships are preserved and will compress data on the fly to make for a faster transfer. (it will also only copy the needed files so if you add files, then you can run the command again and rsync will only copy the additional files. You can also remove files and use the --delete option and it will remove the files on the remote host).
If you don't want to copy hidden files add --exclude=".*/" as an option. Its a good idea to at least copy over users .mozilla files, and any .evolution or .thunderbird folders as a minimum. It shouldn't cause too many problems copying over all hidden files.
For even more information, you can have a look at
man rsync
Backup and Restore User's Desktop Background
You can also restore the user's desktop background (as long as you have the file backed up as well. To check, run the command without >desktop.pict).
gconftool-2 --get /desktop/gnome/background/picture_filename >desktop.pict
copy over desktop.pict and on the new machine
gconftool-2 --type string --set /desktop/gnome/background/picture_filename "$(cat desktop.pict)"
N.B. copying over a user's /home folder wholesale with all the hidden files and folders will do this anyway.
Recreating User Accounts and Restoring Permissions
If you backed up the User Accounts from the old system as detailed above, you can simply restore those files to /etc now, and you'll have the same user/group setup as on the old system. Just do:
sudo cp /home/passwd /home/shadow /home/group /etc/
If you didn't back them up, you will need to recreate those accounts manually. For each username, do:
adduser [username]
You might need to change the ownership to match that of the user if you find that the ownership and groups have changed to a numeric value (check using ls -lh).
- For multiple users, do:
find /home/username/ -type f -exec chown -R username.username {} \;
'Remember to include the semicolon at the end.'
- For a single user, you can just do:
chown -R username.username /home/username/
You might also need to apply proper permissions, though this step should be unnecessary if you used rsync.
find /home/username/ -type f -exec chmod 644 {} \;
find /home/username/ -type d -exec chmod 754 {} \;
or, for more private users:
find ./ -type f -exec chmod 640 {} \;
find ./ -type d -exec chmod 750 {} \;
When you are finished with a backup and you've made sure that all the data is properly transferred and that the machine works, you can (optionally) remove your backup from tsbackup:
rm -r /var/tsbackup/[date-ticketnumber]