How to Use MongoDB Atlas to Make Your CRM More Efficient

As part of digital transformation, many companies want to optimize their internal business processes, gain more visibility into important business metrics, and create new automation routines. Data is always at the core of business processes and metrics, and most business-critical data is often located in one or a few repositories, such as a customer relationship management system (CRM).

Historically business users have relied on spreadsheets and enterprise data warehouses for bringing the data together and making decisions. These solutions can range from a disjointed set of dashboards to an all-in-one central console. But businesses that need to move fast need to iterate on their data and processes fast, and they can’t do that if implementing a change in CRM takes months or if the things are done manually in spreadsheets.

This article describes how MongoDB Professional Services created an internal solution to address these issues.

Our approach

In MongoDB Professional Services, we also needed to streamline our business processes and get out of spreadsheets for business management, especially for revenue forecasting. As the organization grew, the amount of manual labor associated with spreadsheet maintenance became untenable, and making sense of the data became more difficult, especially when the data might be inconsistent, stale, or even inaccurate.

Ordinarily, a good CRM or Professional Services Automation (PSA) system can help solve this problem. At MongoDB, for example, we use Salesforce, which provides decent flexibility, but also requires heavy customization and has limitations.

We’ve also seen MongoDB customers address the problem by building ETL pipelines into MongoDB Atlas and taking advantage of MongoDB’s flexible schema, query language and aggregation framework, and Atlas Search. The data from source systems is ingested as-is or remapped to create a single view.

The best approach we’ve found, however, is to optimize the schema for how the data will be consumed, with different parts of documents potentially coming from different source systems. Atlas App Services provides a serverless abstraction layer that allows fine-grained but flexible control over the schema to help you avoid conflicts and iterate without breaking compatibility.

After considering alternatives, we created an internal CRM/PSA-augmenting system that is built on top of the MongoDB Atlas platform to provide us with additional capabilities and flexibility. This solution allows Professional Services to rapidly deliver advanced functionality, such as revenue forecasting, automation, and visibility into complex business metrics. The solution also allows Professional Services to address business systems' needs and promptly react to changes, with functionality beyond what is typically provided by other systems.

MongoDB’s internal solution, at its core, is serverless and data-centric, leveraging Atlas App Services functions and triggers for processing the data and Atlas Search for full-text search. It uses Connector for BI, Atlas GraphQL API, and App Services wire protocol and Atlas Functions to access and manipulate data from other components. Its components include a React-based console application, Atlas Charts, Tableau dashboards, Google Sheets, and microservices for data import and integrations.

Solution architecture

The data architecture in our internal solution builds on the single view approach and the data-mart concept. The main idea is to ingest relevant data from Salesforce and other systems, enrich it, and build on it quickly, as shown in the following image.

We followed these eight key principles to help enable this functionality:

1. Focus on bringing in data in the form that makes the most sense for the business. And, find the right balance between making the ETL easy and optimizing for the foreseen application use cases.
2. Apply transformations in the ETL process to make the ingested data intuitive, including document hierarchy, field names, and data types.
3. Clearly define the data lifecycle in terms of data producers and consumers. Data producers can only overwrite documents and fields that they “own” - and only those. For example, the ETL process from the source system should overwrite the data in MongoDB documents as needed, but it should only modify those fields that are actually coming from the pipeline.
   1. Aim to structure MongoDB documents in a way that makes it clear which fields are owned by what producer.
   2. Atlas App Services schema and rules can help ensure that the most critical documents and fields are correctly accessed and modified.
4. Use the Atlas Functions and App Services wire protocol in applications and services, as opposed to directly connecting to the Atlas instance.
   1. This allowed us to use Google SSO in the console without requiring any sophisticated security mechanisms when we need to do regular CRUD operations from within the application.
5. For complex data logic and on-the-fly calculations, use App Functions.
6. Use database triggers for propagating changes and generating data-driven events. Use scheduled triggers for generating aggregated views and periodic work.
7. Use external services for communicating with the outside world (e.g., email sender, ETL job).
   1. The external services are invoked asynchronously by listening on change streams from their respective namespaces (pub-sub model).
   2. All external services work independently of each other.
8. Don’t overthink. MongoDB Atlas’s Developer Data Platform offers a lot of flexibility and, if these principles are followed, making changes and iterating on a working system is surprisingly easy.

To reiterate the last point, our internal solution is easy to modify and extend because of the flexible schema concept in MongoDB and the independence of external components. Users can access the data through available tools and integrations, and developers can update specific parts of the system or introduce new ones without delays, making this solution efficient in terms of both cost and effort.

Conclusion

Through this example of our internal solution, we demonstrated that by leveraging MongoDB Atlas in full force, you can solve seemingly intractable business problems with speed, efficiency, and robustness on top of what regular systems can do. Whether you’re optimizing your company’s business processes, building business dashboards, or improving automation, the MongoDB Atlas developer data platform can help make the process easier.