Thursday, September 15, 2011

How does [mongoDB] concurrency work

This is a short post to call attention to this [mongoDB concurrency]. I do not doubt, for a second, that concurrency is difficult. It's probably very hard. Part of which depends on the overall architecture, resultset size, record version collision, replication, sharding, and conflict resolution; among others.

I do not have a particular concern other than it is an important problem and that it's going to effect write performance and potentially some reads. (This is very similar to the GIL problem in Python). There is a ticket on their system that references this issue and plans to resolve the issue by pushing the lock to the collection instead of the DB. While this is more granular I'm not sure it resolves the real issue with a workable solution. (And I do not have a formal recommendation)

It's not clear to me whether this problem references a lock across all shards or just the one shard. Therefore, could the problem be alleviated by adding shards? (doubt that)

I can say this, however, most RDBMS servers have addressed this problem by setting the locking level via a pragma and that it is tunable. In order to know whether this is an issue for you; you will need to know the distribution of calls [read, insert, update and delete] and the TPS rates too.

I'm less convinced that the day of the RDBMS is over. (Recently PostgreSQL 9.1 was released)

No comments:

Post a Comment

prod, staging, QA, dev in your CI/CD?

I've been developing with CI/CD since before it was a straw, let alone a pipeline. No, graduates of 2020 you did not design or discover ...