Tech learnings

Spring Mongodb Connection Pool Limit I had started to observe performance degradation in my app, when the traffic spikes up in very few minutes. The current connections of front facing Apache will spike up during this period of time. Upon investigation found that mongodb primary server experiences high loadavg during this time. My mongo primary is CPU bound, as the entire working set fits comfortably in to the main memory. During the high loadavg, there were about 3000 active client connections, served by 4 cores. It is evident that 4 cores is struggling to deal with 3000 client connections at the same time. Just when I was thinking about increasing the no.of cores, I had came across this excellent article on pool sizing  About-Pool-Sizing . Instead of scaling up the CPU, I decided to limit the pool size in each server. By default, Java Mongo driver can open 100 connections in a pool, but it can be customised with additional options in the connection string. spring.data....

Reading from secondary mongodb is one of the simplest way to scale number of reads from the application. By default, all the reads and writes goes to primary. Although mongodb docs caution against going ahead with this option, there are certain scenarios where it's useful, Data is mostly static and changes if at all, rarely Reporting systems, where lagging data is fine High number of reads, relatively low writes The risk is that the data read from secondary might not be latest, so we have to be careful in choosing which queries are to be sent to secondary.  It's best to consider setting SecondaryPreferred with maxStaleness for the driver to determine, whether to send to secondary or fallback to primary. We can achieve this behaviour by making use of Read Preference Spring Data MongoDB Spring provides two API to set Read Preference. One is via MongoTemplate and the other is through Query flags(slave ok). Option 1: MongTemplate: This approach is slightly tedio...