The key challenge in migrating/restoring data is that the act of moving data between environments often times transforms the data. These transformations happen through a variety of mechanisms, from system updates to automation and custom code. But what if you don’t want your data to change? How do you minimize changes to your data? This article answers those questions and also helps you understand what the possible transformations are, and how to manage and mitigate them effectively.
This article is part 4 of a series written by Sovan Bin from Odaseva. This series details Odaseva and Sovan’s experiences and best-practice recommendations for anyone working with enterprise backup and restore processes. For more information about backup and recovery see part 1, part 2, and part 3 of this series.
La Joconde, Entropy and Salesforce data
The increase in entropy accounts for the irreversibility of natural processes, and the asymmetry between future and past. (Second law of thermodynamics.)
Let’s take the following example of restoring a masterpiece painting after severe damage. No matter how good your restoration goes, it might appear close to the original, but the restored painting will never be the same as the original. The differences may or may not be noticed by the novice, but the changes made by a different artist, at a different time in history, with different materials will lead to micro alterations.
Salesforce data is transformed during the restoration process and creates an asymmetry between the backup version and the restored version of the data.
The main questions to address in this process are: how to measure the amount of transformation, how to minimize it as much as possible, and what is the degree of transformation that we are willing to tolerate (e.g. 0.0001 < δ < 0.001)?
Before we answer those questions, let’s review why, and how, data is transformed.
Migrating or Restoring Data Transforms Data
The key challenge in migrating and restoring data is that it transforms the data, and you will need to minimize the effects of the transformations and adapt your process to take them into account.
Let’s take a look at a typical Salesforce restore scenario which illustrates how much data is transformed during data restoration.
What about your Salesforce organization? Can you evaluate how much Salesforce data you have and how much you can restore without any transformations? And which data and functionality can be considered business critical and thus require your full attention to minimize transformations?
Measuring the Amount of Transformation During a Restore
Let’s look at the backup and restore of a single record to see how it is transformed.
Case record from backup
Case record after restore
Transformation Type
Id
500A000000CS8UrIAL
500A00000079TO4IAM
1
Subject
Package 065412ITL was lost
Package 065412ITL was lost
N/A
CaseNumber
00301052
02191061
2
CreatedDate
2011-02-08T11:09:48.000Z
2015-11-24T19:19:28.000Z
3
LastModifiedDate
2012-04-01T10:39:49.000Z
2015-11-24T19:19:28.000Z
3
CreatedById
005A0000000aIydIAE
005A0000001s4NUIAY
3
LastModifiedBy
005A0000000aDBGIA2
005A0000001s4NUIAY
3
NewFormulaField__c (formula cross object that was created since backup)
null
Platinum Customer
4
OldTextField__c (text field that was deleted since backup)
Priority8
null
5
Various transformations were involved between the backup (query() api call) and the restore (create() api call).
– Transformation Type 1: Record Id was modified
– Transformation Type 2: Autonumbers look different
– Transformation Type 3: Audit fields (createdDate, lastModifiedDate, createdBy, lastModifiedBy, systemModStamp) were updated by Salesforce
– Transformation Type 4: Fields that did not exist when the backup was done could not be restored and would require post-process actions.
– Transformation Type 5: Fields that existed when the backup was done but that were deleted since the backup could not be restored as is and would require post-process actions.
Multiple Transformations Can Occur During Restore
By design, Salesforce may transform your data during the restore process (audit fields for example), and your customizations (e.g. workflow field updates) may also add data transformations, thus compounding the overall scope of transformation. For example, workflows, Apex triggers, and validation rules in your customizations may add transformations during a restore.
How to Minimize or Mitigate Transformations Related to Restore
As explained in the Extreme Force.com Data Loading series by Bud Vieira, some pre-processing steps can help enhance the performance of loading Salesforce data into another Salesforce instance, and post-processing steps can help complete the data enrichment and configuration tasks that have been deferred.
Regardless of whether you want to enhance performance or minimize data transformations, we can still apply the same concepts:
Pre-processing Steps: Minimize transformations that will occur during the restore process by tweaking the Salesforce system.
Post-processing Steps: Adjust transformations that have occurred during the restore process to get closer to the original data.
Pre-processing Steps that Reduce Transformations During Restore
Some of the pre-processing tips below suggest that you disable automation (workflow rules, validation rules, etc.) in your system when restoring data, and while it may increase performance and ensure a smooth data load, don’t forget to add these automation steps back in during post-processing!
Tip 1: Minimize transformation on Audit fields
By activating the “Create Audit Fields” feature that appeared in the Spring ’15 Release, you will be able to override the audit fields in certain cases. (See Considerations before having Create Audit Fields enabled for details.)
Tip 2: Minimize transformation on Autonumbers
Switch the Autonumber field type to a text field and then switch it back after the restore. By switching the field type to text you will be able to restore an autonumber value to what it used to be in the backup. When switching the field type back to Autonumber, be sure to correctly reset the starting auto number. (See How do I reset/restart an auto number field? for details.)
Tip 3: Minimize transformation generated by your workflows and triggers
By deactivating your workflows or triggers, not only will you improve performance, you will also reduce transformations. While you are expected to use workflows and triggers for daily operations, these will probably not apply when you are restoring your data.
Tip 4: Prevent validation rules from blocking your restore process
By deactivating your validation rules, you will both improve performance, and also prevent restore failures that are related to data quality or business process changes.
Tip 5: Minimize transformation due to data and metadata gaps between your backups and your application schema
By identifying and understanding the data and metadata gaps between your backup and your application schema, you will be able to identify specific fields that may exist in your backup that don’t exist in your current schema (and vice versa). You will need to create these additional fields in your organization before starting your restore process (if the data is relevant to you). Enterprise-level tools such as Odaseva Metadata Compare (part of the Odaseva Data Management Suite) can help to automate this process for you.
Post-processing Steps
After the restore is completed, you want to set any configuration and automation settings you modified back to their original state.
De-activate (if necessary) the “Create Audit Fields” feature.
Re-activate any trigger, workflow or validation rules that you have deactivated during the pre-processing steps.
Switch autonumber/text fields back to Autonumbers. You will need to switch the field type back to Autonumber and reinitialize the counter. (See How do I reset/restart an auto number field? for more information.)
Don’t Forget to Evaluate the Success of Your Restore Process
Once you have run your restore process including pre-processing and post-processing steps, you might want to evaluate how successful the operation was and how much transformation it generated. You will then be able to mitigate data transformations that the business will consider not acceptable.
Should you need help regarding this topic, have a look at tools like Odaseva Compare Data (part of the Odaseva Data Management Suite) that automates this process for you.
Reduce Transformation by 80%!
If we apply the different tips mentioned above, the initial simple example will look quite different as we will have minimized transformation by 80%.
Case record in backup
Case record after restore
Id
500A000000CS8UrIAL
500A00000079TO4IAM
Subject
Package 065412ITL was lost
Package 065412ITL was lost
CaseNumber
00301052
00301052
CreatedDate
2011-02-08T11:09:48.000Z
2011-02-08T11:09:48.000Z
LastModifiedDate
2012-04-01T10:39:49.000Z
2012-04-01T10:39:49.000Z
CreatedById
005A0000000aIydIAE
005A0000000aIydIAE
LastModifiedBy
005A0000000aDBGIA2
005A0000000aDBGIA2
NewFormulaField__c (formula cross object that was created since backup)
null
Platinum Customer
OldTextField__c (text field that was deleted since backup)
Priority8
Priority8
Conclusion
In this article, we saw how data can be more or less transformed when pushed to Salesforce. We’ve covered how multiple transformations can occur when restoring records, how we can minimize them, and how important it is to evaluate the actual success of the operation. These tips should also help you when you migrate data to Salesforce, as the same concepts apply.
What Next?
As you’ve discovered in this series, backup and restore is not just about data, it’s also about metadata. In our next article, we will start a new series and share insights about how to dramatically increase the quality of your Release Management process with Salesforce, and we’ll start by covering one of the most important topics: migrating/transporting metadata from one sandbox to another.
Resources
Trailhead Modules:
Data Quality
Data Management
Lightning Experience Data Management
Help and Training:
How do I reset/restart an auto number field?
Enabling Create Audit Fields
Articles:
Extreme Force.com Data Loading, Part 1: Tune Your Data Model
Salesforce Backup and Restore Essentials Part 2: Restore Strategies and Processes
Extreme Force.com Data Loading, Part 3: Suspending Events that Fire on Insert
About odaseva.com
Odaseva (http://odaseva.com) is the leading Business Continuity solution for Salesforce enterprise customers. Their products address the challenges of Salesforce backup & restore, Salesforce high availability, Salesforce archiving, and Salesforce release management automation, and eliminate the risks of data loss and service outage for over 1 Million Salesforce users. Customers like GE, Robert Half, Heineken, and Schneider Electric trust Odaseva’s technology and unique expertise to ensure business continuity.
About the Author and CCE Technical Enablement
Sovan Bin holds 8 Salesforce certifications including the Certified Technical Architect (CTA). He is a specialist on Salesforce platform governance, security, and performance, and served as a Lead Technical Architect at Salesforce for several years before founding Odaseva in 2012.
This post was published in conjunction with the Technical Enablement team of the Salesforce Customer-Centric Engineering (CCE) group. The team’s mission is to help customers understand how to implement technically sound Salesforce solutions. Check out all of the resources that this team maintains on the Architect Core Resources page of Salesforce Developers.