Large Data Volumes: Mitigation Techniques

Aloha Trailblazers!

In the last blog, we discussed Large Data Volume in Salesforce and its impact on our business and efficiency. Continuing from that, let’s now look at different ways to handle the challenges of dealing with a large amount of data.

Large Data Volume Mitigation Techniques

When looking at Large Data Volumes below are the mitigation techniques

1. Indexing
2. Skinny Tables
3. Divisioning

What is Data Indexing?

Indexing is a sorted column or a combination of columns that can be used to uniquely identify the rows of data. An index contains sorted columns and pointers to the rows containing the data. When we index a particular field, the query usually uses the sorted column to quickly identify data rows, thus eliminating the overhead of a full data table scan to fetch the required rows. Salesforce usually takes around 15 mins to index any new field.

Types of Indexing :

Standard Indexing: This type of indexing is the one that is offered directly by Salesforce. Typically the fields that are indexed by Salesforce automatically are as follows:

1. Record TypeId: A unique identifier for record type in Salesforce.
2. Division: Used to categorize data within an organization.
3. System Mod Stamp: Represents the last modification timestamp for a record.
4. Name: Often associated with personal or business names.
5. Email: Facilitates quick and efficient searches based on email addresses.
6. Salesforce Record IDs: Unique identifiers assigned to each record in Salesforce.

Usage Criteria

The Manufacturer’s Guide to Marketing Through Partner Enablement in 2021

Learn how partnerships allow manufacturers to scale revenue growth beyond what’s possible with direct sales alone.

Get the Guide

Standard Indexing is a powerful feature, but there are specific criteria to consider when opting for it. Choose Standard Indexing only when the filter conditions match less than or equal to 30% of the first million records and less than or equal to 15% of additional records. This ensures that the indexing operation remains efficient and aligns with Salesforce’s performance guidelines.

We can opt for Standard Indexing only if the filter matches less than or equal to 30% of the first million records and less than or equal to 15% of additional records.

Custom Indexing: This is a valuable tool that allows you to optimize the performance of your queries and reports by indexing frequently used custom fields, including formula fields. However, it’s crucial to know the limitations and best practices associated with this feature.

1. Fields Eligible for Custom Indexing: You can create custom indexed for commonly used fields, particularly those used in reports and queries. This includes Formula fields, providing flexibility in optimizing performance even for calculated values.
2. Fields Ineligible for Custom Indexing: Certain fields cannot be indexed. The fields that cannot be indexed are as follows Multiselect picklists, Currency fields, long text fields, and Binary fields. These restrictions stem from the nature of the data types, making traditional indexing less feasible.

Usage Criteria

It’s important to exercise caution and adhere to guidelines when implementing custom indexing. This feature is most effective when applied to fields where the filter conditions match a relatively small percentage of total records, ideally 10% or less, up to a maximum of 333,333 records. This ensures that the indexing operation remains efficient and doesn’t overly burden the system.

What are Skinny Tables in Data Management?

Salesforce optimizes data retrieval by creating Skinny Tables and consolidating essential Standard and Custom fields. They’re supported on objects with record sizes exceeding 1 million, enhancing data fetching speed. Skinny Tables exclude soft-deleted records, maintaining a streamlined data retrieval process.

However, Skinny Tables have limitations. They’re object-specific, containing a maximum of 100 columns, including encrypted data. They can’t include fields from other objects. Importantly, after a Sandbox refresh, Skinny Tables are not automatically copied, except in Full Copy Sandboxes, requiring manual synchronization. Overall, Skinny Tables in Salesforce boost performance for large datasets but come with specific considerations.

Usage Criteria

Evaluate LDV objects, identify frequently queried fields, compile a list, and request Skinny Tables from Salesforce for optimization. Adjust existing queries to utilize Skinny Tables for faster results retrieval, strategically enhancing overall performance and streamlining data access.

What is Divisioning of Data?

Divisioning lets us segment our organization’s data into logical sections, making searches, reports, and list views more meaningful to users. Divisions are useful for organizations with extremely large amounts of data. It’s only available in SF Classic.

Types of Divisioning

• Record Level Division: Division is a field on individual records that marks the record as belonging to a particular division. A record can belong to a division created by the administrator or the standard “global” division. The standard global division is created automatically when your organization enables divisions. A record can belong to only one division at a time.
• Default Division: Users are assigned a default division that applies to their newly created accounts, leads, and custom objects that are enabled for divisions.
• Working Division: If you have the “Affected by Divisions” permission, you can set the division using a drop-down list in the sidebar. Then, searches show only the data for the current working division. You can change your working division at any time. If you don’t have the “Affected by Divisions” permission, you always see records in all divisions.

You can also check the below blogs for added information on Large Data Volumes: Introduction to Large Data Volume in Salesforce