How to generate synthetic data for Salesforce?

Salesforce being one of the best CRM platforms on the cloud, it is highly configurable allowing companies to tailor workflows, screens and so on … One of the most challenging activities while developing on Salesforce is Data! In fact, we need Test Data for each development use case. Test Data should not be using production data due to GDPR regulation as well as internal company policies, therefore the need for Synthetic/Realistic Test Data.

In this article we are going to discuss the following:

• What is Test Data Management?
• Key challenges for Test Data Management
• Available options to create test data for Salesforce
• Creating Test Data with CloudTDMS

What is Test Data management?

Test Data is the key input given to a software program or SaaS solution during Dev & Test phases. Test data is used for both positive testing to verify that functions produce expected results for given inputs and for negative testing to test software ability to handle unusual, exceptional, or unexpected inputs. Test data management is the process of planning, designing, storing, and managing Test Data modeling and workflows.

Key challenges for Test Data Management

Regulatory Compliance

• Over 55% of companies are not fully compliant with Data Privacy Policies due to access to all or subset of production data by development & test teams.
• Impact: GDPR penalties range from 2% of annual revenue or €10 million, whichever is the greater, to 4% or €20 million, depending on the severity of the breach.

Test Data Readiness

• Up to 45% of dev/testing time is spent generating by hand or waiting for test data.
• Impact: Testing Delays as well as increased defects.

Data profiling

• Up to 85% of data is still profiled manually.
• Impact: Delivery Delays, Incomplete/inconsistent data discovery & profiling.

Automation

• Over 70% of test data is still created manually.
• Impact: Delivery delays & Incomplete testing of key use cases, inefficient Dev Ops processes.

Available options to create test data for Salesforce

• Manual (xls) or Specific development
• Create Test Data using Apex Tests
• Automated tools: Inhouse, on-prem or Cloud

In this article we shall be discussing about

• Creating Test Data using automated tool (CloudTDMS)

Creating Test Data with CloudTDMS

CloudTDMS is a one stop solution for Test Data Management, CloudTDMS automates the process of test data generation for non-production purposes such as development, testing, training, upgrading or profiling. While at the same time ensuring compliance to regulatory and organizational policies & standards. CloudTDMS involves manufacturing and provisioning test data for multiple testing environments by Synthetic Test Data Generation as well as Data Discovery & Profiling.

The process of using CloudTDMS is simple as follows:

• Go to CloudTDMS and create a free account (Starter Plan)

• Create an Application (Salesforce in our case)

• Create Streams & Attributes. Salesforce translations

  Salesforce Object = Stream

  Salesforce field = Attribute

• Create Workflows

• Get Test data. You have two options:

  • Free plan (Starter): Download your Test Data from CloudTDMS and upload it into your Salesforce dev instance
  • Paid plan (Pro or Team): Create your Storage connection as Salesforce, CloudTDMS will automatically load your test data into linked Salesforce Objects

Use Case: Test data for some Salesforce objects

Salesforce data model is well documented, you need to know Objects & fields you need for your test cases. In this use case, we shall address the Contact Object (other objects could be added similarly to the same CloudTDMS Application)

Creating Application

So let's create an application first, by going to the Application tab from the dashboard and then click on Create New.

• Go to Applications and click on create new.
• You will have to provide the Application name (e.g., 'Salesforce')

Click on Save and Next to Continue.

Creating Stream

Here you can define a few things for your data

• Application: The name of the application which you created earlier (e.g Salesforce)
• Completeness: Percentage of data to be populated with empty values
• Correctness: Percentage of data to be populated with incorrect values

  Note: This stream will generate data for the Contact table, so your stream name will be Contact.

Now click on Save and Next.

Creating Attributes

Here we will define columns for our data, since Salesforce columns are pre-defined, we will use some default columns provided for Salesforce 'contact' table. Click on Add Attributes

Here we have 3 sections:

• Name: The name of your column
• Category: The category of the type of data you want to generate
• Provider: The type of data to generate

  We can add more fields by clicking on 'Add Row'

Here is the data model which we are going to upload to Salesforce

After creating all your attributes click on 'Save'

Free users (Starter Plan) — Saving data to local storage

Creating Workflow

1. Navigate to Workflows
2. Click on 'Create New'

Here we will define some parameters for our Workflow

• Application: The name of your application within which your stream exists, Select your application from the list (e.g., 'Salesforce')

  Note: 1 application can have multiple streams, All the active streams will be executed by workflow`

• Storage: for Free Plan (Starter), you can use 'Local Storage'
• Scheduled At: If you want to generate data on a specific date and time

  Note: By default the workflow will be executed at a maximum delay of 1 minute

• Format: The output format of the file. (Default format is 'CSV')

  Available formats: — CSV, JSON, XML1

Click on 'Save' to create Workflow

After the Workflow is 'Executed' Navigate to Data Generated section You will see your file in the section where you can download the file by clicking on it and then you can upload your file to Salesforce.

Advanced users (Pro & Team plans) — loading data directly into Salesforce

Creating a Salesforce storage in CloudTDMS

1. Go to storages, click on 'Create storage'
2. Select Salesforce from the list
3. For creating a Salesforce storage we need to provide
   • Name for storage
   • Username
   • Password
   • Security Token
   • Description (optional)

Click on 'Save' to save your Storage

Creating workflow to load data directly into Salesforce

1. Navigate to Workflows
2. Click on 'Create New'

Here our workflow parameters will be

• Application: Salesforce
• Storage: Salesforce Storage
• Scheduled At: If you want to generate data on a specific date and time

  Note: As we are using Salesforce storage, other parameters are not applicable here

Click on 'Save' to create the workflow

After a couple of seconds, we can see the data uploaded to Salesforce Contact object. If by any means CloudTDMS does not load data to our Salesforce Contact object, the reason could be the invalid credentials for Salesforce, regardless in any case of failure the generated data will be stored in the CloudTDMS Local Storage which we can access through the Data Generated tab and then load the data into Salesforce manually. For more information, we can check the logs after the workflow is executed.