FTP Mirroring in Windows

Just backing up my empeg. I usually ftp so that more recent files are overwritten, identical files skipped, and the rest downloaded regularly.
It's a backup and it's enough to be safe, but not perfect .

Now, does anybody know if there's any ftp cliend which actually performs directory mirroring, i.e. that it also REMOVES files locally which are not anymore on the remote site (empeg, in our case)?

Thank you!

wget might have a mode to do that; I normally use rsync to do it.

For Windows, yes?

I believe WebSynchronizer will do what you want.

Sorry, I forgot to mention. Yes, for windows. Thanks for WebSynchronizer. I am playing with it, but I am still not sure it is doing what I want. It seems so but it is actually deleting too many files locally, it seems to me...

I guess rsync is fro Linux, am I right?

You can run rsync on Windows as well. It is entirely command line controlled.

You can get rsync for Windows as part of Cygwin's toolkit:

http://www.cygwin.com/

If all you want is rsync though, this might be better (it also looks like it deals with some of the issues of getting rsync running as a service on Windows):

http://www.itefix.no/cwrsync/

Here is the manual page for rsync:

http://samba.anu.edu.au/ftp/rsync/rsync.html

Ok, with a little delay I finally found the time to try this. Thank you all very much, by the way.

Now, I am getting confused in spite of the manual you sent me. Basically, it seems to me that a resync cannot be done on other ports than the 873. Am I correct? If not, I must be doing something wrong, since I am getting these few different errors:

[ ILQUEN HQ ] G:\P\cwrsync>rsync -v -r -t --delete --force --stats --progress 10.10.1.108::ftp/drive0 g:\taympeg-backup\drive0

rsync: failed to connect to 10.10.1.108: Connection refused

rsync error: error in socket IO (code 10) at clientserver.c(88)

or

[ ILQUEN HQ ] G:\P\cwrsync>rsync -v -r -t --delete --force --stats --progress 10.10.1.108::/drive0 g:\taympeg-backup\drive0

ERROR: The remote path must start with a module name not a /

rsync error: error starting client-server protocol (code 5) at main.c(1062)

or

[ ILQUEN HQ ] G:\P\cwrsync>rsync -v -r -t --delete --force --stats --progress rsync://10.10.1.108::/drive0 g:\taympeg-backup\drive0

rsync: failed to connect to 10.10.1.108: Cannot assign requested address

rsync error: error in socket IO (code 10) at clientserver.c(88)

As you noticed , it is stil not clear to me which command line I should use.

Basically, my empeg is 10.10.1.108 . Hijack's ftp server is on port 21 (but I'll be happy to change it if necessary).
I am trying to create a mirror of drive0 and drive1 in this local dir:
g:\taympeg-backup\

so that i get

g:\taympeg-backup\drive0
g:\taympeg-backup\drive1


Can anybody provide me with a correct command line? Thanks!!

Hijack's ftp server is on port 21

Rsync is not an ftp client. You need an rsync server on the empeg (or on your computer and then an rsync client on the empeg).

-Mike

You don't actually need an rsync server as such on either (though you can do it this way), as rsync can connect via a shell (either telnet or ssh) and then run the local rsync to start the transfer.

But if you want to set up rsync as a server on the empeg, look here.

Right, I've got a little more time to reply to this fully.

First of all, rsync has two distinct modes of operation. In one mode you set one host up as an rsync server, so that rsync is always running and listening on port 873. In the other mode you simply install rsync on each host and when you run rsync it connects to the remote host using a shell (telnet, rsh or ssh) and runs the remote copy of rsync that way.

I have never set rsync up as a server on either Windows or Linux, so I can't help much on that side of things. In many cases however you don't want rsync set up as a server, as it has a couple of drawbacks. The first is that there is more setup to do, especcially on Windows. The second is that you rsync can't communicate securely in server mode, if you want secure comms you need to use the shell mode and ssh.

However, in this case you probably aren't too worried about security and using a shell is complicated because the empeg doesn't ship with telnet or ssh.

So I think you are best advised to follow the instuctions Roger has just posted for getting rsync running in server mode on your empeg.


[ ILQUEN HQ ] G:\P\cwrsync>rsync -v -r -t --delete --force --stats --progress 10.10.1.108::/drive0 g:\taympeg-backup\drive0
ERROR: The remote path must start with a module name not a /
rsync error: error starting client-server protocol (code 5) at main.c(1062)

[ ILQUEN HQ ] G:\P\cwrsync>rsync -v -r -t --delete --force --stats --progress rsync://10.10.1.108::/drive0 g:\taympeg-backup\drive0
rsync: failed to connect to 10.10.1.108: Cannot assign requested address
rsync error: error in socket IO (code 10) at clientserver.c(88)


Looks like you are getting the source format slightly wrong. I think what you are missing is that when using an rsync server you don't add a "/" to the front of the path. So when you say "rsync://10.10.1.108::/drive0" I think you should be saying "rsync://10.10.1.108::drive0", but I don't use rsync in server mode so I just guessing from reading the rsync man page.

To connect to the drive0 module (or whatever they call it) on 10.10.1.108 it'd be:
10.10.1.108::drive0
or using a rsync URL:
rsync://10.10.1.108/drive0/

Of course, first you'd have to have the rsync server running on the empeg.

Whoever chose the term "module" to refer to a rsync mount point/share needs shooting.

mod·ule n.
A standard or unit of measurement.
Architecture. The dimensions of a structural component, such as the base of a column, used as a unit of measurement or standard for determining the proportions of the rest of the construction.
A standardized, often interchangeable component of a system or construction that is designed for easy assembly or flexible use: a sofa consisting of two end modules.
Electronics. A self-contained assembly of electronic components and circuitry, such as a stage in a computer, that is installed as a unit.
Computer Science. A portion of a program that carries out a specific function and may be used alone or combined with other modules of the same program.
A self-contained unit of a spacecraft that performs a specific task or class of tasks in support of the major function of the craft.
A unit of education or instruction with a relatively low student-to-teacher ratio, in which a single topic or a small section of a broad topic is studied for a given period of time.
Mathematics. A system with scalars coming from a ring.

A term that was in someway related to its use would have been more helpful...

Yeah, I originally typed "mount point/share" and then remembered that it was called a module and changed it. I think it's stupid too, but that's what it's called. Didn't want to confuse; although, I think using that term is confusing to begin with.

Thank you very much for your help guys. Anyway I am now testing this Synchromagic Pro, which seems to be synchronizing quite well and does not need to install anything on the empeg. Very feature rich.

does not need to install anything on the empeg

So if it doesn't install anything on the empeg it must be reading all the data from it via ftp every time you sync ? Isn't this going to be very slow ?

The benefit of rsync is that because it runs locally at both ends of the syncronisation it only copies changes rather than just copying all the data each time.

Mmm... which brings to mind something new we can now do:

We can write PC scripts/programs to upload, chmod, and site-exec rsync to do the mirror, and then remove it again after it completes.. giving the appearance of not uploading anything to do the entire operation in one slick motion..

Cheers

Isn't this going to be very slow ?

I would assume so, but I just tested the software with few files. Soon I'll be using it on the empeg for a full backup first, and a sync later with just few extra files to be downloaded. I don't think it's going to be a mojor problem since I don't plan to backup more often than when I put new files on the empeg, which does not happen frequently. However, Mark makes me think of a suggestion for Hijack, if it is possible/convenient:

what about a rsync server in hijack, startable when needed from a new hijack menu item? Too much mem?

Just thinking about how to refine the idea: what if we
1) installed the necessary files in any dir on drive0, as Roger recommends (maybe we could create a distributable zip with the files ready to go, so that it is more user-friendly to those who are not failiar with linux)
2) configure config.ini with the proper path to the files
3) Have a hijack menu item that starts/stops the rsync server (from the above mentioned path?)

?

As usual, not an expert in linux, sorry if this does not make sense

No need. See my other post, just above yours, for a totally external (non-hijack) solution approach.

And the new MENU_EXEC functionality in Hijack can be used to launch rsync (or anything else) from the Hijack menu. Once I add the back-end implementation of MENU_EXEC, that is.

Now, that sounds interesting!