I recently decided to try Mercurial for version control. I’m most familiar with Subversion, but I like the idea of a distributed version control system. I tried Git first, but my desktop runs Windows 7, and the Windows interface for Git (TortoiseGit) is a little wonky compared to its SVN brother (TortoiseSVN). I stumbled onto Mercurial as an alternative to Git, and I love it. The Windows interface (TortoiseHg) is great; I actually like it better than TortoiseSVN!
Anyway, my initial repo size was 3.8gb. I pushed it, and after several hours of uploading, I got this error message:
1 2 3 4 5 6 | searching for changes remote: adding changesets remote: adding manifests remote: adding file changes abort: unexpected response: empty string [command returned code 255 Mon Dec 12 07:19:00 2011] |
I figured something happened during the transfer, so I wiped the repo and tried again but got the same error message. After some Google-fu attempts, I discovered that this error doesn’t seem very popular. I figured it might just be choking on the repo size, so I went through and deleted a bunch of data I didn’t need. I got it down to 1.9gb. I tried again and got the same error message.
At this point, I was starting to get pissed. Pushing a repo that large on a connection with 2mbit upload takes a while, and this was wasting a bunch of time (and bandwidth). I decided to try breaking down the push into several smaller pushes (each around 400-500mb). This worked for the first several pushes, but by push 4, I got the same error. Looking at the files I was pushing, I saw that one of the files was a 500mb ZIP file. I figured this might be the issue, and since I didn’t actually need the file, I decided to get rid of it and try again. I wiped the repo and tried once more to push the entire repo in one go. Voilà, it worked!
So what was the problem? After looking around, I stumbled onto http://mercurial.selenic.com/wiki/HandlingLargeFiles which describes how Mercurial handles large files. Specifically, this sentence described my issue:
all files are currently handled as single Python strings in memory
Well of course! I use Linode for hosting, and I have the Linode 512 option. Since Mercurial requires memory proportional the the size of the file it’s processing, I ran out of memory (both physical and swap). I’m certain a server with more memory could’ve handled the troublesome 500mb ZIP file with ease, but mine couldn’t.
So the moral of the story is this: Mercurial is awesome as long as you recognize its memory limits when it comes to large files. If you get the “empty string” error 255 on a push, make sure you have enough memory to handle the largest files in your repo.
