Redpoint Logo
Redpoint Logo

May 3, 2021

Data Lake vs. Data Swamp: Why Enterprise-Class Businesses Can’t Afford Bad Data

To meet the expectations of the always-on, connected customer for a relevant experience at the precise moment of every interaction requires brands to move at the cadence of the customer. Until recently, when customer journeys were linear, sequential and consisted of a limited number of channels, a basic customer data platform (CDP) or data lake could sufficiently handle the data needs to deliver rudimentary personalization. The same is not true today. Dynamic, omnichannel customer journeys and rapidly changing consumer behaviors far outpace the abilities of basic CDPs or data lakes to deliver highly relevant experiences – in real time – at scale.

Marketers who need easy access to large data streams may need to rethink their current data practices, many of which likely rest on antiquated notions that a code-centric approach to data warehousing, in which large amounts of data pool into a data lake and ‘stagnate’ – creating inertia that resists large amounts of daily transactional updates because the data must be processed using coded ETL’s as with Databricks. Furthermore, a lack of governance tools and a failure of embedding data quality into the process at the point of ingestion leaves marketers with data unfit for purpose – resulting in poorly executed identity resolution, minimal behavior information and a dearth of transformations (e.g., a year-over-year change in spend).

The problem, of course, is that poor data quality inhibits the ability of the enterprise to deliver the personalized experiences customers expect in today’s real-time world. Consider a recent Dynata survey commissioned by Redpoint, where 70 percent of customers said they will only shop with brands that demonstrate a personal understanding of them. This means the brand knows they are the same customer across all channels and is able to deliver a highly relevant experience at the moment of interaction.

Data Lake or Data Swamp?

Meeting this expectation requires access to an enterprise-class dynamic data repository that continuously refreshes and links the totality of an organization’s customer data in a central hub with a single point of operational control: a robust, dynamic CDP.

Knowing the key differences between a data lake and a dynamic CDP will help marketers understand why the latter is often the No. 1 revenue-generating solution of the enterprise, and why high-engagement brands in industries such as retail, financial services, telco, travel and hospitality, and healthcare are choosing the Redpoint rg1 customer experience platform as their no-code, data management platform to manage the brand experience.

One of the biggest limitations of a data lake and other highly code-dependent systems is the inability to scale – providing personalized experiences for tens or hundreds of millions of customers. By itself, this makes a data lake entirely inadequate for enterprise-wide adoption and CX use cases. It also limits the power of AI and machine learning to provide differentiated experiences; offline, coded models go stale over time. Data scientists build models that, once in production, become outdated as soon as there is new data or as soon as the business decides to optimize a different metric.

Because transformations are pushed downstream, marketers must build their own aggregate values, derived attributes and other critical information. And with these data values driving segmentation decisions, audience selections, campaign triggers and real-time personalization decisions, missing it, doing it wrong or inconsistently strips data of much of its value and suboptimizes the overall return on investment.

Client Side vs. Server Side

A robust, dynamic CDP also solves for the problem in the marketing cloud world where an enterprise’s technology stack is composed of tools that were standalone packages and acquired and integrated over time. This introduces the potential for errors, data latency, and complexity for marketing programs that need to work across the marketing cloud products.

Similarly, many of the client-side personalization tools that enable building web-based customer profiles may use an enormous volume of web behavior to drive personalization but lack access to other data sources that provide a complete picture of a customer. As a result, users need to take a Lego approach and link building blocks of tools that provide access to other data. The Lego approach isn’t inherently bad; it’s just not scalable nor able to deliver repeatable results.

Plus, individuals in the process might think they’re doing a great job—especially because many of the success metrics in that approach are operationally focused versus focused on marketing outcomes or ROI. But it’s virtually impossible to tell if, for example, one marketer’s A/B tests are advancing the overall marketing cause and improving return on investment. This situation conceals the strategic shortcomings hidden behind the technical limitations of the environment and the busy work of employees doing jobs that should have been automated.

The Lego approach also creates a significant data security problem, starting all the way back at the data lake and through to the front of the website. Understanding customers is very difficult, for instance, so learning about them over time to improve the relevancy of communications and offers becomes nearly impossible. An enterprise-class CDP can resolve this because it can handle front- and backend data for personalization.

Embrace the Data Challenge

The Lego approach is fine for businesses sending out occasional email blasts and doing light personalization on their website. It enables companies to start in limited ways and grow their way to a larger capability. However, ambitious marketers need enterprise-class software for marketing.

So, what does enterprise-grade marketing software look like? It embraces the data challenge and provides the appropriate level of precision processing automation, cleansing, matching, de-duplication, governance and data mastering needed to know everything that is knowable about the customer and preparing it for use within seconds of its arrival. It enables companies to build a complete Golden Record that links together all the proxy identities for each possible customer—even unknown customers—and provides a robust long-tail of transactional information that includes everything from granular behavior to KPIs to transformations summaries; everything that is needed to know the customer and properly treat and message the customer with exceptional relevance. In addition, all that data needs to be ingested and processed, and those Golden Records updated, in milliseconds.

The result is a marketing data store that has a complete contact graph and an extensive data story that is valid and current up to the millisecond. Nothing short of this level of data perfection is suitable for a large enterprise or ambitious marketing leader who wants to make a boardroom level impact in the organization.

Most important, an enterprise-class customer data management platform creates a single-brain approach (i.e., one point of operational control for all channels and messages). It eliminates fragmented communication that goes out based on local brains (distributed in the channels) that only have a local scope of data. Instead, you can control the utilization and orchestration of all channels, easily selecting the optimal messages, timing, and frequency from that single point of operational control. It enables marketers to strategically engineer a holistic engagement journey for individual customers at scale.

Think of it from the point of view of a customer. Would you prefer to speak with a committee of representatives where each member has its own style, agenda or cadence or would you rather speak with just one person, consistently over your journey through the brand? In which case would there be a better understanding and intimacy?  Which case best reflects the commitment to honoring a first party relationship? That is the difference between marketing with fragmented channels vs. the single-brain approach.

Enterprise-class organizations that adopt a robust customer data processing platform are leading the way with an exceptional level of personalized customer engagement. The predictable data security and data perfection of the rg1 platform, for example, enables large enterprises to engineer their customer experiences down to the millisecond – creating the most impactful moment of truth for each and every customer, and at scale. That level of relevance is what customers expect today—and is one of the few points of true differentiation left to build a lasting competitive edge.

Related Content

Embrace Complexity: That’s Where the Magic Happens

Customer Journeys are Dynamic: Your Engagement Technology Should be as Well

The Sky is the Limit with AI and Machine Learning: Capitalize on the Moment of Interaction

Be in-the-know with all the latest customer engagement, data management, and Redpoint Global news by following us on LinkedInTwitter, and Facebook.