Nov 212013
 

When configuring the attributes of members of a Mongo DB replica set, either in the mongo shell or a script, the typical procedure is to set each attribute by its array reference. The weakness in this method is that it’s necessary to know what the array index is of the host whose attribute you need to configure. This may not be possible for a generalized script.

cfg = rs.config();
cfg.members[1].priority = 2;
rs.reconfig(cfg);

However, with a bit of mongo javascript it’s possible to write a small function to cycle through the member config array, determine the index of a host and then set an attribute.

In this example, we want to configure the Mongo DB nodedb02.example.com as a hidden node. This can be done like this:

cfg = rs.config();
cfg.members.forEach( 
	function(item, i) {
		if ( item.host == "nodedb02.example.com" ) 
		{ 
			cfg.members[i].priority = 0;
			cfg.members[i].hidden = true;
			rs.reconfig(cfg);
		}  
	} 
)

If, on the other hand, you wanted to change the slave delay on a read-only reporting node, this could be accomplished by testing each node to find which has the “hidden” attribute set to “true”:

		if ( item.hidden == "true" ) 
		{ 
			cfg.members[i].slaveDelay=0
			rs.reconfig(cfg)
		}  

Use this function in Mongo JS scripts, be it with Capistrano, Puppet or some other automation technique, and everything got just a bit more hands-off.


Matt Parsons is a freelance Linux specialist who has designed, built and supported Unix and Linux systems in the finance, telecommunications and media industries.

He lives and works in London.

  4 Responses to “Reconfigure Mongo DB Replica Set Member by Attribute”

  1. Matt, I have a single 3 member replication set on the same network. 2 members of the replica set (1 primary and 1 secondary) are at the Data Center location 1, and the 3rd member (2nd secondary) at Data Center location 2. What would be the configuration settings I use for adding a 4th member (server) to the replication set as a hidden node, to be used as a backup server, and configured for delayed replication?
    Here are the priority settings for the 3 members of the replication current set:

    DC Location1:
    Primary server; Priority = 1
    Secondary server; Primary = 1

    DC Location2:
    Secondary server; Primary = 2

    Thanks Matt!

    • The MongoDB manual has an example for exactly this, here: http://docs.mongodb.org/manual/tutorial/configure-a-delayed-replica-set-member/

      In short, the way to configure a hidden delayed node is like this:

      cfg = rs.conf()
      cfg.members[0].priority = 0
      cfg.members[0].hidden = true
      cfg.members[0].slaveDelay = 3600
      rs.reconfig(cfg)
      

      However, bear in mind that a hidden node still votes in elections when a primary needs to be determined. You’re proposing a 4-node cluster which is a bad idea, the reason being that the cluster can only vote a new primary if there is a majority of nodes available. With a 4-node cluster, if two nodes go down an (n/2 + 1) majority cannot be reached and the cluster will be read-only. It would be preferable to have an arbiter to act as fifth vote in the cluster – preferably in a third location.

      • Thanks Matt!!

        This is very helpful. A third site is not an option for the arbiter, so I will put it in the primary data center and keep the hidden node backup server at the secondary site.

        From a backup perspective, what is the down side to this hidden node configuration using delayed replication? From a Point in Time (PIT) I am limited to just recovery within the delay (3600) window -correct?

        Thanks,
        Ian

        • The delayed replication simply means that while the oplog is shipped to that replica in real time, the updates are not applied until the time delay has passed. Therefore when you back up this node, the data is an hour behind your primary.

          The real advantage of the delayed replica is that it protects against data deletion by providing a hot standby – provided you can detach the primary in time and break the replication before the updates are applied.

          On the subject of arbiters, if you have the arbiter in one of two data centres, this will protect you from the loss of multiple nodes, but if you lose the entire primary data centre, then the nodes in the secondary data centre will not constitute a majority of the cluster (n/2 + 1) and so the cluster will be rendered read-only. So having two sites doesn’t give you true high availability in that only the primary site is resilient.

 Leave a Reply

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>