Monday, February 18, 2013

Merging Two Salesforce Instances -- Why is it so hard?

Has anyone been faced with the need to merge one Salesforce organization into another? This has been my task for the past few weeks and now that I am nearing completion I wonder if there was a better way.  Yet the best that SF provides is a PPT outlining the many considerations and steps
And then other searches on the web generally lead to consulting companies.  

For my record, here are some observations.  This is a huge task so this blog posting will need to be released in stages and will require many postings.

First step was to conduct a thorough analysis of the other company's business practices to enable us to understand the type of data in their system and the quality of data.   Let me call this other company "Company X" or just X for short.   So, Company X has a number of custom business processes with custom SF objects.  My company considered these and decided to not import the process. That was good because it left me with the task of "just" importing the standard objects.

Second task was to examine all the customizations on the standard objects and determine which needed to be implemented in our company's instance, and which just needed to be imported as data.  

There was a lot more analysis but eventually came the time to perform the import.  I started, of course, with the Account object.

First, add three custom fields to your target instance Account object:
  1. X_Id: Text 30 chars; external Id.  this will hold the ID of the Account record in Company X's instance.  This ID is essential for many reasons. First, it allows you to run the import again and locate the exact record imported before and perform any necessary updates.   Second, it allows you to locate the Account record in the target for any record in the source that links to the account. For example Contact records in X need to be linked to the new record in your target based on this X_Id field.
  2. X_Status:  Text 30 chars.  To hold either "imported" or "merged".  The "merge" value is used when an account already exists in the target instance.  (We are merging two businesses that had some synergy so it is common to use this merge value.)  Otherwise, if the account is newly created in the target then I used the value "imported".   I only use this field to help my end users know where the account record came from.  We imported over 5,000 accounts so this information is quite helpful.
  3. X_Data_Transfer:  Text 2,000 chars. This long text field holds data from X that is not pushed directly into the fields of the Account.  For example, this includes 
    1. Custom fields found in X that were not recreated in the target. 
    2. The name of the users who created the record in X.  I copied the user name because there is no way to (a) set the CreatedById or LastUpdatedById fields and (b) the user does not exist in my company.
    3. If the record is merged then I did not overwrite the previous value in our company record.  Instead I put the value from X into this transfer field.
Second, export the data from X into a CSV file, using your favorite SOQL data exported.  I'm a Mac user so I am dependent on  SOQLExplorer

Third, write code.   There is no way around it.  You need a programmer to write a program that will do the following.  First, let me say that I exclusively use the Partner API while coding.  Sure the Enterprise API gives you tight code-level type checking and makes the auto-completion tools work great. But, it is such a burden to rebuild the WSDL after any change in your instance.
  1. Read in the CSV file.  Break the data into a header row and content. Break the content into a set of LINES each with an array of values.
    1. Contend with fields that contain a double quote.
    2. Contend with fields that contain a double quote followed by a comma.
  2. For each row in the import search the target for possible merges by Account Name.  (I forgot to mention above that we did a thorough attempt to clean up these names before starting the merge using data exports, spreadsheets, and bulk updates.)  If found then we merge  otherwise we import.
  3. Create a new SObject.  
    1. If importing then set the fields in the new record according to the values in the old.  If merging then only update fields that are empty. 
      1. Convert the CSV date into the appropriate data types: text (easy), Boolean (easy), Date (tricky!), Numeric (easy)
    2. For the rest add into the transfer data. Compose the transfer data field to hold the original field label and its value.
  4. Collect a sets of new SObjects and import blocks of these avoiding the SF governor limits.  I choose to import when I have 50 records.
That's all I have time for now.  Sorry, no code at this time.  That is because this is a long story and the code that I finished with did a lot more than just the above. 

No comments:

Post a Comment