By Saurabh GuptaFebruary 9, 2018
This is the first of a monthly blog series we call “Ask an Architect.” In this series, we will be answering common questions from some of our most ambitious customers using Salesforce.
“We have millions of records and constituents with high expectations around service and engagement; how do we develop an effective data management strategy?”
While it doesn’t sound sexy, effective CRM data management is an integral part of delivering a remarkable constituent experience across all channels, particularly for organizations with large data volumes. And the Salesforce platform offers a number of features that make it easy to develop a common sense approach to data management that can deliver happier constituents, a more effective user experience, improved organizational agility, and reduced maintenance & cost.
An effective CRM data management strategy is founded on a solid understanding of your business process, user behavior and technology, and succeeds when you combine it with governance and disciplined execution.
Consider these 5 steps when building your Salesforce Data Management Strategy:
- 1. Take Only What You Need: CRM-Relevant Data
- 2. Optimize your Big Objects: Large Data Volume Optimization
- 3. Use Data where it Lives: Federate and Integrate non-CRM data
- 4. Travel Light: Data Archiving
- 5. Govern with Discipline: Master Data Management
In this blog, we are focused on Steps 1 and 2:
1. Take Only What you Need: CRM-Relevant Data
Limit the data imported into CRM by clearly differentiating and importing only core CRM data. The following is a summary of various types of data, and a potential approach you to manage it.
Result: Reduces data clutter, improves user experience and may reduce cost.
|Types of Data||Examples||High level Approach|
Data that drives constituent engagement for university programs like recruitment, admissions, students success, employee enablement, alumni relations, athletics and advancement.
Typical must-have data for all constituents e.g. biographical or demographic data and communication preferences.
For example, in a student success solution, it is valuable to import degree and GPA data, but not donations.
|Import all, or enough data to run daily operations|
This data populates standard or custom Salesforce objects (or “tables”) and is
|Non-CRM Transactional Data|
Data that supports CRM operations but is mastered elsewhere
|Coursework/LMS data, external system data such as event details, donation amounts and other financial transactions.||Manage data in its system of records|
External Data/ Apps – Integrate/Federate, Mashups, Einstein Analytics etc.
Create CRM integrated app via Heroku/Heroku Connect
|Non-CRM Additional Data|
Data that offers additional insights/research into constituent relationship
Usage statistics from dining hall, gym, study room reservation systems – other indicators of student success on campus.
Also includes system generated – Web Analytics IoT – Device/Sensor data etc.
|Manage data in its system of records|
External Data/ Apps – Integrate/Federate, Mashups, Einstein Analytics etc.
Create CRM integrated app via Heroku/Heroku Connect OR
Salesforce IoT Cloud
Data that is outdated and cannot be used in operational context
|Typically 5-7+ years old data, also includes data such as Individual Email Result (IER) data from Marketing Cloud and others that may be irrelevant after 3-6 months.||Move data out of CRM|
Data Archiving, use of Warehouse/Data Lake
Import summary/roll-up of transactional historic data
What makes any data a good fit for CRM data?
Answering “Yes” to one or more of the following implies that this data may be CRM-relevant, and requires a more careful consideration.
|Considerations||Tell me more||Examples|
|Will Salesforce be the “System of Record” for this data?||Typically data that is mastered in other systems can be shown in Salesforce without replication (Refer section #3)||Data mastered in an institution’s source system such as SIS, Advancement, HR etc.|
|Will this data be required for key business processes that are implemented on Salesforce?||Data that is referenced for key Service processes such as in Case object, Campaigns etc.||Academic programs, periods, assignments, inventory data, Do Not Call preferences etc.|
|Are there automation use cases (within Salesforce) that are based on changes to this data?||Data that is used to drive Salesforce workflows, lightning process builders, email templates, Apex triggers etc.||Total Current or Fiscal Year Giving, Total # Enrolled Credits|
|Are there any Salesforce specific reports, dashboards, and KPI (Key Performance Indicators) that use this data?||Reports and dashboards in Salesforce require data to be stored within Salesforce.||Fundraising Goals, Total New Donors, Total New Students, Total Returning Students|
|Will users need to have activities, tasks or chatter feeds around this data?||Activities/Tasks/Chatter feeds require the parent record data to be stored within Salesforce.||Team collaboration on a major or restricted gift|
2. Optimize your Big Objects: Large Data Volume Optimization
Proactively identify and optimize your CRM for objects with millions of records (typically referred to as Large Data Volume or LDV). Use the LDV best practices guide to optimize performance.
Result: Improves CRM’s internal performance and user experience.
|LDV Techniques||What is it?||When to use it?|
|Force.com Query Plan||To understand effective execution plans for SOQL queries.||Use the Query Plan tool to optimize and speed up queries done over large volumes.|
|Database Statistics||A nightly process to collect statistics from the database in order to know and access data better.||Use it to understand data growth, and plan for data maintenance, LDV strategy accordingly.|
|Skinny Tables||Salesforce can create skinny tables to contain frequently used fields and to avoid joins. Can be requested for custom and some standard objects.||Useful to resolve specific long running queries – typically for objects with millions of records. They can enhance performance for reports, list views, and SOQL.|
|Indexes||Salesforce supports custom indexes to speed up queries. They can be requested by contacting Customer Support.||Useful for specific SOQL queries that need to work selectively using a non-indexed field.|
|Divisions||Divisions can segment organization’s data into logical sections, making searches, reports, and list views more meaningful to users.||For organizations with extremely large amounts of data that can be logically segregated (by region, territories or others).|
Here is some additional information on LDV techniques.
|LDV Techniques||Level of effort/Complexity||Examples|
|Force.com Query Plan||Complexity: Low to Medium|
Effort: Low (Days to Week)
Requires being code-savvy to get meaningful analysis. Accessed via Dev. console, and fairly simple to use.
|Reports or list views that time out or take a long time to return results. Run SOQL from report to troubleshoot.|
|Database Statistics||Complexity: Low|
Effort: Low (Days)
Fairly simple to access via Setup menu, and understand the results from.
|Use to make informed decisions about growth of their database by object.|
|Skinny Tables||Complexity: Low to Medium|
Effort: Low to Medium (Week)
Analysis can take time. Contact Salesforce support to determine if this is appropriate for your instance.
|User registration that may require additional info (from custom non-indexed fields) that is taking too long to return results.|
Effort: Low to Medium (Weeks)
Analysis can take time. Utilize query plans to determine need for Index. Contact Salesforce Customer support to get these implemented (on a case by case basis).
|Most commonly used custom fields, membership ID,|
|Divisions||Complexity: Medium to High|
Effort: High (Weeks to Months)
Can require extensive analysis and expertise from experienced professionals, and is irreversible. Testing in a sandbox may be needed.
|Breaking out Contact/Account information by School or College.|
This is part 1 of a 2-part blog series on data management for large data volumes. Keep an eye out for the next installment of this series on February 19, 2018. Or join us for our “5 Steps to an Effective Salesforce Data Management Strategy” webinar on February 27, 2018.
This blog is also part of our larger “Ask an Architect” content series. To learn more about engaging a Salesforce.org Customer Success Architect in your organization, please contact your Account Executive.
- Ask an Architect
- User Tips and Tricks
You Might Also Like
Higher EducationFeeding the World Together: Our First Zero Hunger Summit
Read about the three main takeaways from the first ever Zero Hunger Summit, hosted by Salesforce during Dreamforce.
Higher EducationHow to Double Down on People-Powered Philanthropy
In this blog, you’ll read about a list of principles for optimizing people-powered philanthropy in a post-pandemic world.
Learn more about the exciting product announcements from the education and nonprofit keynotes at Dreamforce 2022.
The data management process includes a wide range of tasks and procedures, such as: Collecting, processing, validating, and storing data. Integrating different types of data from disparate sources, including structured and unstructured data. Ensuring high data availability and disaster recovery.
- Business Requirements. Data must address specific business needs in order to achieve strategic goals and generate real value. ...
- Sourcing and Gathering Data. ...
- Technology Infrastructure Requirements. ...
- Turning Data into Insights. ...
- People and Processes. ...
- Data Governance. ...
- The Roadmap.
MDM stands for Master Data Management. To put it simply, it means creating a single source of truth, or one master “golden record” for all versions of a record across all systems. This could be a person, an object, or place, and a CRM like Salesforce can be one point of entry for this data.
Provide high availability and disaster recovery. Use data in a growing variety of apps, analytics, and algorithms. Ensure data privacy and security.
The stages through which information passes, typically characterized as creation or collection, processing, dissemination, use, storage, and disposition, to include destruction and deletion.
Definitions: Data Architecture & Data Strategy
Data Architecture defines how data is acquired, stored, processed, distributed, and consumed. On the other hand, the term Data Strategy implies the overall vision and underlying framework of an organization's data-centric capabilities and activities.
They are volume, velocity, variety, veracity and value.
- Business strategy. Your data strategy should reinforce and advance your overall business strategy, which refers to the processes you use to operate and improve your business. ...
- Organizational roles. ...
- Data architecture. ...
- Data management.
for Salesforce™ Scalable 360° View Automation. Data Quality Protection.
- Identify sources of master data. ...
- Identify the producers and consumers of the master data. ...
- Collect and analyze metadata for your master data. ...
- Appoint data stewards. ...
- Implement a data governance program and data governance council. ...
- Develop the master data model. ...
- Choose a toolset.
MDM provides a unified view of critical business data with a single master dataset and ensuring consistency of data used in analytical and operational processes. It provides up-to-date data for businesses. MDM collects, transforms, and corrects data and creates a golden record for businesses.
An effective data strategy improves data security by implementing measures to limit unauthorized data access. You can consider all data governance rules and regulations while defining policies and processes.
Only considering internal, traditional data: Today, data comes in many forms and from many sources. A good data strategy should consider all avenues for accessing data, including options like photo data and video data, and external sources such as social media platforms and big data brokers.
Data management in Salesforce deals with Import/Export of data or records to/from a Salesforce organization.
While there is no industry standard for enterprise DLM, most experts agree that the data lifecycle includes these six stages: creation, storage, use, sharing, archiving, and destruction.
The constant cycling of data generation, analysis, integration, storage, and elimination gives Executives the quality data they need to make decisions.
Information life cycle management is the consistent management of information from creation to final disposition. It is comprised of strategy, process, and technology to effectively manage information which, when combined, drives improved control over information in the enterprise.
The data life cycle is the sequence of stages that a particular unit of data goes through from its initial generation or capture to its eventual archival and/or deletion at the end of its useful life.
Sensory memory is the first stage of Information Processing Theory. It refers to what we are experiencing through our senses at any given moment. This includes what we can see, hear, touch, taste and smell. Sight and hearing are generally thought to be the two most important ones.
By definition, data architecture is a blanket term that covers both the policies, rules, standards, and models that govern data collection and how data is stored, managed, processed, and used within the organization.
A data architect is an IT professional responsible for defining the policies, procedures, models and technologies to be used in collecting, organizing, storing and accessing company information. The position is often confused with a database architect and data engineer.
Data Architecture is a framework built to transfer data from one location to another, efficiently. It is full of models and rules that govern what data is to be collected. It also controls how the collected data should be stored, arranged, integrated and put to use in data systems of an organisation.
The 5 V's of big data (velocity, volume, value, variety and veracity) are the five main and innate characteristics of big data. Knowing the 5 V's allows data scientists to derive more value from their data while also allowing the scientists' organization to become more customer-centric.
It refers to the assurance of quality/integrity/credibility/accuracy of the data. Since the data is collected from multiple sources, we need to check the data for accuracy before using it for business insights.
A comprehensive DMP clearly articulates the roles and responsibilities of every named individual and organization associated with the project. Roles may include data collection, data entry, QA/QC, metadata creation and management, backup, data preparation and submission to an archive, and systems administration.
Data first organizations prioritize business innovation and risk models built around multiple sources of digital intelligence.
- Your Current Data Maturity Level. ...
- Your Industry and Company Size. ...
- Your Data Management Team. ...
- Outline your data architecture. ...
- Define the relationship between BI and your teams. ...
- Assign ownership. ...
- Establish data governance. ...
- Reassess regularly.
Specifically, there are four major pillars to keep in mind for good data management: Strategy and Governance, Standards, Integration, and Quality. Most importantly, in order to be data-driven, an organization must embrace data as a corporate asset.
Transactional data, in the context of data management, is the information recorded from transactions. A transaction, in this context, is a sequence of information exchange and related work (such as database updating) that is treated as a unit for the purposes of satisfying a request.
Data Governance and Master Data Management
Master Data Management includes processes from the creation of master data through to its disposal. Data Governance creates the rules and adjudication of the operational processes that are executed within those processes.
You need to contact Salesforce Customer Support for skinny table. You can't create, access, or modify skinny tables yourself. This table shows an Account view, a corresponding database table, and a skinny table that would speed up Account queries.
There are four master data management (MDM) implementation styles, and their different characteristics suit different organizational needs. These include consolidation, registry, centralized and, ultimately, coexistence.
Data management helps minimize potential errors by establishing processes and policies for usage and building trust in the data being used to make decisions across your organization. With reliable, up-to-date data, companies can respond more efficiently to market changes and customer needs.
The most commonly found categories of master data are parties (individuals and organisations, and their roles, such as customers, suppliers, employees), products, financial structures (such as ledgers and cost centers) and locational concepts.
- Extends Existing Data Governance Program and Tools.
- Cleansing and Correction of Erroneous Data.
- Data Quality Monitoring and Reporting.
- Business Taxonomy and Hierarchy Management.
- Concept Standardization (e.g. Address)
- Deduplication, Matching and Unique Keying.
Most prominently the Data Owner and the Data Steward. Probably several people would be allocated to each role, each person responsible for a subset of Master Data (e.g. one data owner for employee master data, another for customer master data).
Data management includes storage, data security, data sharing, data governance, data architecture, database management, and records management.
- Understand the potential of the data you have.
- Build a company data management team.
- Be sure to comply with global data privacy regulations.
- Make sure your company data is secure.
- Turn your company data management strategy into profit.
Data architecture is a framework for how IT infrastructure supports your data strategy. The goal of any data architecture is to show the company's infrastructure how data is acquired, transported, stored, queried, and secured. A data architecture is the foundation of any data strategy.
There are five core components of a data strategy that work together as building blocks to comprehensively support data management across an organization: identify, store, provision, integrate and govern.
- What do I need to know or what business problem do I need to solve? ...
- What data do I need to answer my questions? ...
- How will I analyse that data? ...
- How will I report and present insights? ...
- What software and hardware do I need?
Strategic analysis (sometimes referred to as a strategic market analysis) is the process of gathering data that helps a company's leaders decide on priorities and goals, shaping (or shifting) a long-term strategy for the business.
Salesforce Data Model is essentially the manner in which tables of data are represented within your Salesforce database to make them understandable to anyone who views them. Data modeling helps you make sense of the data residing within your system.
The standard Salesforce Data Loader is the tool of choice for most, business users and consultants alike. It's powerful, convenient and well documented. This tool is best used for repetitive data loads by business users or data migration projects.
The Salesforce Database creates records for leads, tasks, opportunities, accounts, and notes. This is where the actual data is stored. There are a variety of record types that allow linking different business processes to users, customers, and admins based on their user profiles.
Data management is the process of ingesting, storing, organizing and maintaining the data created and collected by an organization.
- Define a Data Architecture. First, it's crucial to define a data architecture. ...
- Assign Responsibilities. ...
- Define How You'll Name Things. ...
- Collect Data. ...
- Prepare Data. ...
- Process Data. ...
- Analyze Data. ...
- Interpret Data.
Students will apply methods for organizing and analysing large amounts of information; solve problems involving probability and statistics; and carry out a culminating investigation that integrates statistical concepts and skills.
Data management is the practice of collecting, organizing, protecting, and storing an organization's data so it can be analyzed for business decisions. As organizations create and consume data at unprecedented rates, data management solutions become essential for making sense of the vast quantities of data.
Data management in Salesforce deals with Import/Export of data or records to/from a Salesforce organization.
What is a data strategy? A data strategy is a long-term plan that defines the technology, processes, people, and rules required to manage an organization's information assets. All types of businesses collect large amounts of raw data today.
- Build strong file naming and cataloging conventions. ...
- Carefully consider metadata for data sets. ...
- Data Storage. ...
- Documentation. ...
- Commitment to data culture. ...
- Data quality trust in security and privacy. ...
- Invest in quality data-management software.
- Define goals. Defining clear goals will help businesses determine the type of data to collect and analyze.
- Integrate tools for data analysis. ...
- Collect the data. ...
- Clean the data. ...
- Analyze the data. ...
- Draw conclusions. ...
- Visualize the data.
- Recording. Recording refers to the transfer of data onto some form of documents. ...
- Verifying. Since recording is usually a manual operation, it is important that recorded data be carefully checked for any errors. ...
- Duplicating. ...
- Classifying. ...
- Sorting. ...
- Calculating. ...
- Summarizing and Reporting. ...
- Interactive computing or Interactive processing, historically introduced as Time-sharing.
- Transaction processing.
- Batch processing.
- Real-time processing.
This course examines the core concepts of Data Management, including: definitions, why Data Management is essential to organizational success, the value of data as a whole, key Data Management practices, and more. Data Management is the overarching umbrella around pretty much everything we do with data.
Data management is important because the data your organization creates is a very valuable resource. The last thing you want to do is spend time and resources collecting data and business intelligence, only to lose or misplace that information.
This course comprises four major parts: Counting Techniques and Probability, Organization of Data for Analysis, Statistics, and Integration & Application of Data Management Techniques. Through classroom instruction, assignments and a culminating project, students will study the mathematics required for data management.
- 1) Organizational Ownership. First, you need to determine which internal team (or individual) will “own” customer data. ...
- 2) Data Types. ...
- 3) Data Storage. ...
- 4) Security and Privacy. ...
- 5) Data Quality. ...
- 6) Activation.
Four types of database management systems
hierarchical database systems. network database systems. object-oriented database systems.