1/30/2024 0 Comments Robo 3t search commandsThis section will describe the tools and methods we used to debug slow-running queries in real time. Number of indexes to decrease the utilization when writing.How often certain queries are run to decrease utilization when reading.We did some experiments that confirmed the correlation between the white-page problem and over-utilized MongoDB drive, then started working on optimization. These queries would take the longest when MongoDB drive utilization was at its peak. We found out that certain database queries make the Meteor backend unresponsive until the query is resolved. Our main application is built using the Meteor framework. Internally, this became known as the white-page problem, as after refreshing the browser, users got stuck on a white page.ĭuring this incident, not all the application pods were affected at the same time, so we started to dig into our framework first. It all started with users reporting that our UI sometimes freezes for a couple of seconds or even minutes. Throughout the article, we will be using the Robo 3T MongoDB client and tooling provided by MongoDB Atlas, which is our MongoDB provider. 6k reads and 1k writes per second with >150MB/s transfers.260 million objects amounting to 200GB of data with 40GB indexes.We store metadata about our users, their actors, actor runs, and storage in MongoDB, but for actual data, we have separate data stores: key-value store, dataset, and request queue (see Apify Storage for more information).This article describes some of the techniques and MongoDB Cloud features we used to debug performance issues and expose sub-optimal queries. It was time to take action and improve our overall usage of MongoDB. The cause? Over-utilized drives in our MongoDB cluster. In early 2021, our users started reporting degraded performance of our UI. It is a crucial element that can affect both user experience and our platform's performance. For Apify, MongoDB is not just the main source of truth for all our systems.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |