I almost dumped MongoDB.
I have a new instance running in Chicago that I’ve been trying to setup as a mirror of what I have running in NYC. My progress (with everything) was drastically slowed when MongoDB would crash when trying to setup a replica set between the two. The problem is that both of my instances are 2G VMs running on OpenVZ without any virtual memory. Mongo would try to allocate memory, and then horribly crash when setting up the replica set.
The problem was that Mongo tries to reserve 5% of available disk space as the log space of the replica set. This is more memory than my servers had, and therefore a big crash with Mongo trying to mmap everything in. The solution that I found, that I could not find documented anywhere was…. setting a static oplogSize in the config.
So, hopefully this will get out onto google for other people to find. When you are using MongoDB in a replica set in a low memory environment, set oplogSize low.
I’ve currently set oplogSize to 256M, which has made my available memory decrease dramatically. I’ll have to tune it, but now I have a mirror of my data replicating with automatic failover!!!! Super sweet!
Here’s a graph from Cacti that shows the spike in memory from the main server.