I've frequently run into a problem, and I'd like to know how to work around it. As always, this BBS is a gold mine of minds, and I'm sure someone here can help me.

Often, I want to exchange large files with a friend. Only one friend, directly, not share them with the world.

The following options work, but they all have limitations:

- We can email the files. But the limitation is that many people's mail servers have caps on the size of file attachments, either incoming or outgoing, or both. I hit that problem far too often, and splitting the files up into multiple emails is unwieldy.

- We can use the direct-file-transfer feature of instant messaging applications (or heck, IRC DCC for that matter). However, these don't work behind firewalls, and rarely work behind NAT. Often, the person on the other end of the line is behind a firewall/NAT.

- We can run an FTP daemon on our local machines, but those also run into the problem of being behind NAT or behind a firewall.

- We can use some FTP space that's been kindly loaned to us from one or the other's ISP. But these usually have size caps on them, so really big files can't be exchanged that way.

In most cases, the person at the other end of the line can chat with me via one of the IM services (I use Trillian so I can be on all of them), but the direct file transfer feature rarely works.

Ideally, I'd like for the IM clients to have file transfers that just worked behind NAT/Firewalls. But failing that, what other options to I have for file swapping big files? Anyone?

Hmm... Tricky...

I usually use FTP for anything that big, as long as it does not go over about 2GB...

Is there a way to encapsulate a file transfer and send it through as plain ascii within your IM session? Not using a file transfer mechanism, but through the regular text stream - just like the "send text file" option in hyperterminal.

I'm behind firewall/NAT and I don't have problems sending/receiving files using AOL IM (Trillian actually, but using IM via Trillian.) I think you need port 5190 open on both sides.

This link has some more info on setting up my particular router.

Text messages in the IM clients usually have a limitation per-message of just a couple of K or so.

And being the FTP server is nice if you're not behind NAT/Firewall. If you are the FTP server, and you're behind a firewall, no one can see you or connect to you. And borrowing space from someone else's server has limitations.

I think you need port 5190 open on both sides.

Like I said. Doesn't work behind a firewall.

You're assuming, with that statement, that both people on each end of the transaction have control over their firewalls. Assume that one or both do NOT have control over the firewall and that it's a draconian corporate firewall that only allows HTTP to get through.

Oh geez, then you're screwed.

Heh, thanks, Tony.

Why not install Apache, and set the file to transfer in the http-doc directory then kill Apache when you don't need to transfer a file? Unless you want to get fancy, you can have Apache running on whichever machine is sending out the file. That is a simple, but pretty ugly solution.

If HTTP gets through, that means port 80 is open, right? Why not just set up a FTP daemon (only when you want to send or receive a file) that listens on port 80?

Chances are that if the firewall is paranoid enough to only allow HTTP traffic in then it's going to allow access to the official webserver only. This is also not taking account the problems caused by NAT or masquerading.

- Trevor

The last time I used Yahoo IM, the file transfer service was indirect. It would upload to a Yahoo server completely before the recipient could start downloading it. And, the download was through http.

However, I would imagine that there is a file size limit due to it being stored first. Also, this was in the early days of Yahoo IM, so things may have changed.

Also, AIM transfers usually only fail if both people are behind NAT/firewall. I am using NAT and I can always transfer files to my 56K friends (but why would you want to).

Have you tried Fed Ex?

Sure, encode the file as a bunch of airbill numbers and recipients.

I usually use drive crypt to make a container that is triple blowfish encrypted, use a 128 bit key, name it something weird and put it on kazaa until the transfer is done, then remove. Klugey, but it works.

Only done it a couple of times. I usually just mail a CD. Unless of course, time is critical.

How big are these files?

Why not install Apache, and set the file to transfer in the http-doc directory then kill Apache when you don't need to transfer a file?

If I'm behind a NAT/Firewall, the person on the other side has no way of seeing or connecting to a web server or an FTP server on my machine.

use a 128 bit key, name it something weird and put it on kazaa until the transfer is done, then remove. Klugey, but it works.

Except when my recipient is behind a draconian corporate NAT/Firewall that doesn't allow things like Kazaa to work through it.

How big are these files?

Doesn't matter exactly. Files could be a couple megs up to hundreds of megs. This is something I just run into occasionally in my life with various people... I'm talking about finding a long-term solution to internet file transfers from behind firewalls in general.

If SSH is open on the firewall, you might be able to tunnel a web or FTP server over it. Or just use SCP.

Can you describe how those things are done, in more detail?

Tony, if this is just you and one other person, you don't do this every single day, and you don't mind using a slightly slower connection (640/256), I'd be happy to create an account for you on my server that you and your friend can share. You will not be able to use FTP, only SSH, but you can get a free wintel ssh client for non-commercial use that works just like a GUI FTP client.

Like I said, if it isn't 10's of Gigs every day and as long as it doesn't totally hammer my gaming bandwidth all the time, it's the least I can do for our heroic FAQmaster. Oh, and you'll have to clean up after yourself by deleting the files when you've exchanged them.

Also, being behind NAT/firewall is not that big of a deal, and only one of you will need to open the port. If you want to talk about doing this, I'd be happy to help. You can have an old machine running OpenBSD and your very own SSH file server in under a couple of hours. Very easily. With a proper OS (UNIX), an old, free P133 can totally fill a 10Mbps pipe, and I doubt you have that kind of connectivity... I can give you some suggestions if you want to go this route, too.

Send me an email or PM if interested.

Jim

Greetings!

And some further good stuff on ssh.

Thanks for the offer of FTP space. At the moment, I'm trying to find a long-term solution that doesn't depend on someone else's FTP server. However I will consider your offer.

How about using freedrive/xdrive/what-not harddrive-on-internet? They are (at least used to be) free up to so many MB and you can share files with other people. There might be some regulations on the use nowadays, but earlier they were very popular for sharing warez.

Stig

Freedrive redirects to Xdrive, which doesn't appear to be free. Although it does look like it would work for what I want if it were free.

What you have here Tony, to give a slightly different analogy is effectively the same problem as having two USB "slave devices" and trying to get them to talk to each other.
[in that you and your friend are behind NAT'ed and gods knows what else firewalls, and therefore cannot accept incoming connections,you can only make them - sort of how a USB slave device works]

The only way to achieve that will be by using the equivalent of a USB "master" as a proxy to communicate to both slaves.
[BTW: There is a USB experienced person on the board here who is proposing to do exactly this to add USB slave to slave capability to the Empeg - look in the projects board/section for more details]

Of course, the analogy runs out here as its actually a software issue not a hardware one, but then the USB slave to USB slave problem is mostly a software problem too.

But you could imagine a "IM" type of place, like a Telephone Exchange that does voice conferencing where two incoming calls are routed to a common "place" where both parties can exchange files and whatever.

I would have thought that one of the existing IM services would offer this sort of thing.

Hey wait a minute - what about "Microsoft NetMeeting" or similar "service" - yes its MS software/protocols (well sort of), but from memory the Netmeeting protocol has up to 65535 "channels" that can be used (with the right software) - so surely you could concoct something up using Netmeeting - assuming both parties are using NetMeeting compliant software - not sure about Linux, but windows has lots of NM clients.
It also supports file transfer, real time chat, shared white board amoungst other features (including shared audio/video).

You're right in your assessment.

Yes, the IM programs should, in theory, just do this natively, but I've not yet been able to get the file-transfer features to work from behind firewalls.

I could try netmeeting. I wonder if it would have the same problems that the IM programs have from behind firewalls.

50megs.com?

Just checked 50megs.com and it looked good until I hit this.

most zip programs can split, i'm told.

And I can split files into multiple emails. As I said at the top of this thread, that's unwieldy and I'd like to avoid it.