Windows Client Quick Start

I've found an online reference that's very helpful when it comes to resolving specific errors. Take a look at this and let me know if it works for you!

Note that there have been some changes in the rsync solution landscape since this was originally written so I might want to evaluate Grsync as a useful rsync tool. (And rdiff-backup. And anything else interesting here.)

On Client; install cwRsync from here. Download cwRsync_<version>_Installer.zip then run the .exe program inside. To backup run something like this:

pushd C:\Documents and Settings\DaleM\Some folder to backup
rsync --modify-window=2 --chmod=u+rwx,g+rx,o+x --verbose  --progress --stats --compress --recursive --relative --times --perms --links --delete . ravmud@MooreWorks.DynDNS.Org::bills/someBackup

To restore just turn the source and target around "ravmud@MooreWorks.DynDNS.Org::bills/thisBackup .". (Read the gotchas in this website about tuning for Windows.) Revise C:\Program Files\cwRsync\cwrsync.cmd to hold the correct command lines then schedule it to run in the Windows task scheduler. I use the pushd command line because the cygwin path references are odd, put yourself in the folder that needs backup (with "pushd C:\Folder") then just backup the current folder using "." on the command line.

Overview

rSync is a tool for copying folders and files from one place to another. You can copy from your local hard drive to a different location on your local hard drive. You can also copy from your local hard drive to another computer on your LAN or the Internet. The copy that happens can be very smart so that only the changed pieces of very large files is copied.

For more information about rSync read from the following list:

  1. http://en.wikipedia.org/wiki/Rsync is always a good place to start.
  2. http://www.rsync.net/ Main web site. Windows information here.
  3. http://samba.anu.edu.au/rsync/ SAMBA started the rSync project.
  4. http://www.gaztronics.net/rsync.php If you are a Linux SysAdmin looking for a way to Rsync a Windows machine to your Linux server, or you are a Windows user who wants to use Rsync as a Server, then you have come to the right place!
  5. http://everythinglinux.org/rsync/ Server and Client notes.
  6. http://transamrit.net/docs/rsync/ Linux server setup notes.
  7. http://sourceforge.net/projects/sereds/ SEREDS is a toolbox for secure distribution/replication of software/files on multiple platforms/systems. Perl, openssh and rsync are key components. More files here.

Please let us know of your experiences!

Create new backup repository for BobIsYourUncle and do a backup.

  1. On MooreWorks.DynDNS.Org:873 server "mkdir /mnt/sdb1/BobIsYourUncle"
  2. chown ravmud:ravmud /mnt/sdb1/BobIsYourUncle
  3. vi /etc/rsyncd.conf
    [BobIsYourUncle]
       path = /mnt/sdb1/BobIsYourUncle
       comment = Moore Works rSync Server
       uid = user1
       gid = user1
       read only = no
       list = yes
       auth users = ravmud
       secrets file = /etc/rsyncd.secrets
  4. vi /etc/rsyncd.secrets
    user1:password
  5. On Client install cwRsync then run "rsync --verbose  --progress --stats --compress --recursive --relative --times --perms --links --delete . user1@LocalHost::BobIsYourUncle/thisBackup"
  6. To restore just turn the source and target around "user1@LocalHost::BobIsYourUncle/thisBackup .".

Data flow analysis

To MooreWorks.DynDNS.Org:873 which port forwards to 192.168.1.98 PC Bubbles a 265.2 GB HD.
/etc/rsyncd.conf has repositories(?) definitions.
/etc/rsyncd.secrets has account access information.
Starts at 2 AM, ends about 2:15 everyday about 15 minutes for 15 MB.
Notice that some logged times are GMT and others local!
See http://www.mail-archive.com/rsync@lists.samba.org/msg23519.html for one fix.
/var/log/rsyncd.log is the recent activity log.
Notice that some log entries are GMT and others local time. It's 5 hours off GMT here in San Antonio, Texas.
2009/04/29 21:05:22 [17904] connect from adsl-70-248-12-38.dsl.snantx.swbell.net (70.248.12.38)
2009/04/30 02:05:23 [17904] rsync to SASmax/ from ravmud@adsl-70-248-12-38.dsl.snantx.swbell.net (70.248.12.38)
2009/04/30 02:05:23 [17904] receiving file list
2009/04/30 02:05:31 [17904] MAX/DAT/
2009/04/30 02:18:49 [17904] sent 1073238 bytes  received 15595250 bytes  total size 13003962668
About 15 MB in 13 minutes.

/mnt/sdb1/SAS is 29.3 GB and is the SAS rsync backup target repository.
The data transfer is about 1 MB a minute.

 

DIAGNOSTICS

 

 

rsync occasionally produces error messages that may seem a little cryptic.  

The one that  seems to cause the most confusion is  "protocol version 

mismatch - is your shell clean?".   This message is usually caused by 

your startup scripts or remote shell facility producing  unwanted garbage 

on the stream that rsync is using for its transport.  The way to diagnose  

this problem is to run your remote shell like this:     

rsh remotehost /bin/true > out.dat   then look at out.dat. 

If everything is working correctly then out.dat should be a zero  

length file. If you are getting the above error from rsync then you 

will probably find  that out.dat contains some text or data.  

Look at the contents and try to work out what  is producing it.  

The most common cause is incorrectly configured shell startup scripts   

(such as .cshrc or .profile) that contain output statements for 

non-interactive logins.   If you are having trouble debugging include 

and exclude patterns, then try specifying the  -vv option. At this level 

of verbosity rsync will show why each individual file is included  or 

excluded.  

 

root@RyCycle:~/bin# rsync -vv 4rsync.sh ravmud@LakeNap::MWdale/rycycle/

opening tcp connection to LakeNap port 873

sending daemon args: --server -vve.Lsf . MWdale/rycycle/ 

Change is the firmament.

Password: 

created directory /rycycle

4rsync.sh

total: matches=0  hash_hits=0  false_alarms=0 data=512

sent 581 bytes  received 27 bytes  243.20 bytes/sec

total size is 512  speedup is 0.84

More Setup and Diagnostics here, and here.
 

 

Example Command Lines/Batch Files

AAMHA

pushd C:\Documents and Settings\Amy
echo rsync-ing "C:\Documents and Settings\Amy"
"C:\Program Files\cwRsync\bin\rsync" --chmod=u+rwx,g+rx,o+x --verbose  --progress --stats --compress --recursive --relative --times --perms --links --password-file="p1" . dalem@10.50.0.212::AAMHA/AMY

CJF

set log="K:\Logs\rsync2DaleM.log"
call K:\Scripts\bu0.bat K:\Logs rsync2DaleM log K:\Logs
echo Look in %log% for results.
echo Backing up with rsync >%log%
\\2003Server\Apps\Scripts\LogIt.VBS Begin: %0 %1 %2 %3 %4 %5 %6 %7 %8 %9

\\2003Server\Apps\Scripts\LogIt.VBS %0 rsync-ing C:\Documents and Settings\Brucew\My Documents
pushd C:\Documents and Settings\Brucew\My Documents
echo rsync-ing "C:\Documents and Settings\Brucew\My Documents">>%log%
"C:\Program Files\cwRsync\bin\rsync">>%log% 2>&1 --verbose  --progress --stats --compress --recursive --relative --times --perms --links  --password-file="/cygdrive/c/Documents and Settings/Brucew/p1" . brucew@192.168.0.51::bruceREPO/mydocs/bruce
 
\\2003Server\Apps\Scripts\LogIt.VBS End: %0 %1 %2 %3 %4 %5 %6 %7 %8 %9
 

Persyn

@Echo Begin: %0 %1 %2 %3 %4 %5 %6 %7 %8 %9
if %mwPath%.==. Set mwPath=S:
cscript %mwPath%\scripts\logit.vbs Begin: %0 %1 %2 %3 %4 %5 %6 %7 %8 %9

pushd "C:\Program Files\cwRsync\bin\"
"C:\Program Files\cwRsync\bin\rsync.exe" --verbose  --progress --stats --compress --recursive --times --perms --links --exclude "*bak" --exclude "*~" --password-file="pass.txt" /cygdrive/r/ persyn@MooreWorks.DynDNS.Org::Persyn


cscript %mwPath%\scripts\logit.vbs End: %0 %1 %2 %3 %4 %5 %6 %7 %8 %9
@Echo End: %0 %1 %2 %3 %4 %5 %6 %7 %8 %9
 

 

The best place to host an rSync Server is on Ubuntu; the software there is Open Source, powerful and safe.

The hard drive you are backing up to might should be setup in /etc/fstab to auto-mount somewhere like /mnt/buHD or some such. I use blkid to figure out the UUID for fstab.

Installation and configuration on Ubuntu 8.10

$ sudo apt-get install rsync
$ sudo apt-get install xinetd
$ sudo vi /etc/xinetd.d/rsync
  service rsync
    {
        disable         = no
        socket_type     = stream
        wait            = no
        user            = root
        server          = /usr/bin/rsync
        server_args     = --daemon
        log_on_failure  += USERID
    }
$ sudo /etc/init.d/xinetd restart

Forward TCP port 873

root@Bubbles:/etc# ls rsyncd.
rsyncd.conf     rsyncd.motd     rsyncd.secrets

Below are some samples of key rsync files in your /etc folder:

root@Bubbles:/etc# cat rsyncd.conf
max connections = 2
timeout = 300
motd file = /etc/rsyncd.motd
log file = /var/log/rsyncd.log
pid file = /var/run/rsyncd.pid
lock file = /var/run/rsync.lock

[MW]
   path = /mnt/sdb1/MW
   comment = Moore Works rSync Server
   uid = ravmud
   gid = ravmud
   read only = no
   list = yes
   auth users = ravmud
   secrets file = /etc/rsyncd.secrets

[MWdale]
   path = /mnt/sdb1/MWdale
   comment = Moore Works rSync Server
   uid = ravmud
   gid = ravmud
   read only = no
   list = yes
   auth users = ravmud
   secrets file = /etc/rsyncd.secrets

[sean1]
   path = /mnt/sdb1/sean1
   comment = Moore Works rSync Server
   uid = ravmud
   gid = ravmud
   read only = no
   list = yes
   auth users = sean
   secrets file = /etc/rsyncd1.secrets

root@Bubbles:/etc# cat rsyncd.secrets
user1:password1
user2:password2

# chmod 600 /etc/rsyncd.secrets

This will help debug why rsync client is failing:
# tail /var/log/rsyncd.log

Log rotation

The log file /var/log/rsyncd.log might grow without bounds if you don't turn on log rotation.

# vi /etc/logrotate.d/rsync
/var/log/rsyncd.log {
       daily
       rotate 14
       compress
       notifempty
       missingok
}

 

And it should start rotating daily.

Server backup rotation to keep multiple copies can be setup as described here.

Checkout a recent latest client download here. (Version 4.0.5 here shows client.)

Most Windows users will prefer rSync.Nets Windows Backup Agent which is a simple, easy to use GUI tool that enables powerful and flexible backups. Alternatively; Windows users might want cwrSync which can be found here, here, here or here.

There is good information about on Windows here; be sure to look at the gotchas section!

Note that you might want to add "--relative"  in the following command lines  so the destination contains all the source folders.

root@Bubbles:~# pushd /home/dalem/bin/
root@Bubbles:~/bin# cat rsync1.sh
#!/bin/sh
rsync --verbose  --progress --stats --compress \
      --recursive --relative --times --perms --links --delete \
      --exclude "*bak" --exclude "*~" \
      --password-file=/home/dalem/bin/p \
      /home/dalem/* ravmud@LocalHost::MWdale

rsync --verbose  --progress --stats --compress \
      --recursive --relative --times --perms --links --delete \
      --exclude "*bak" --exclude "*~" \
      * sean@MooreWorks.DynDNS.Org::sean2

rsync --verbose  --progress --stats --compress \
      --recursive --relative --times --perms --links --delete \
      --exclude "*bak" --exclude "*~" \
      * ravmud@LocalHost::MWdale/thisOne
 

dalem@SaveSeth:~/bin$ rsync mooreworks.dyndns.org::

The firmament is change.

bb                 Moore Works rSync Server
MW                 Moore Works rSync Server
MWdale             Moore Works rSync Server
SASmax             Moore Works rSync Server
sean1              Moore Works rSync Server
 

 

 

rsync DaleHome backups - MANUAL

 
rsync LakeNap to MW.

root@LakeNap:~/bin# time sh rsync3.sh
This completed Wed Dec 10 19:19:37 CST 2008 after 2 weeks of backup.
This should be run as root user!
Tap ENTER to continue...

 

rsync BrainIac to MW.

C:\>pushd "C:\Program Files\cwRsync"
C:\Program Files\cwRsync>cwrsync1.cmd

Another sample

rsync --verbose  --progress --stats --compress --recursive --relative --times --perms --links --delete --exclude "*bak" --exclude "*~" --password-file=p.txt  * ravmud@MooreWorks.SpaceTechnology.Net::Apple/test2