Sorry, coming back to this thread a bit late ...
On 27 Feb 98 at 19:15, Steve Pacenka wrote:
Regarding writing a FTP push-mirror, the most complexity seems to be in the
heterogeneity of the FTP servers. Ones on different platforms send back
different formats of directory listings that have to be parsed to obtain
file timestamps (and sizes if those are used in comparison; not useful in
cross-platform text files).
I hit this issue last autumn when I started using mHonarc, and found
no satisfactory tools available for my platform: I use KA9Q under DOS
for most of TCP/IP stuff, because its so reliable and so flexibly
configurable, though I use Winsock undr Win 3.1 for WWW stuff.
So I sat down and wrote something myself, in Turbo C (having since
learnnt Perl, I'd use that if I was doing it again, but I haven't
bothered with a rewrite).
Basically, my proggie (FTPmir) generates scripts for KA9Q's
scriptable FTP client, which means that I can do all my mirroring
unattended: in the early hours of the morning, a batch file updates
the archives, FTPmir updates the scripts, and then KA9Q does all the
transfers at off-peak phone rates.
FTPmir only processes sigle directories at a time, but by using
multiple configs, it can generate scripts for each dir that needs to
be mirrored. Since my mHonarc directories only change monthly,
that's fine by me.
I wrote it for a UNIX server with my homepages account, mnut it
needed no modification when I pointed it at a commercial server,
also UNIX. Basically, its methodology is:
First invocation: just get directory listing
Then build script. Files are compared by time and size: newer local
files are always uploaded. Other files are compared by size:
different sizes cause a file to go into the update queue, and ASCII
conversion issues are handled by speccing which extensions are to be
treated as ASCII -- local files in that category are scanned to
calculate an ASCII size before comparison with remote size.
Finally, it generates an upload script, with a cmd line flag to
enable deletion of redundant remote files. The first and last tasks
in each script are retrieving a remote dir listing, so it is
self-correcting, though with some potential for lag (discussed in the
docs).
I haven't found a scriptable FTP client for Windows, but if there is
one, I'm sure that FTPmir could be easily adjusted to generate the
appropriate scripts.
Anyways, binary and source code and some docs available to anyone who
asks in private mail: as I say, its in Turbo C 2, but with ANSO
compliance checking on, so it should (<g>) compile under other
compilers.
Hugs,
Claire
--
Claire McNab -- Claire(_at_)siberia(_dot_)demon(_dot_)co(_dot_)uk
