What is Data Observability? Gartner, Forrester and the CDP Institute all describe Data Observability in some shape or form as the continuous monitoring of data quality and data pipelines to ensure data is reliable, trustworthy and compliant with regulations.
Marketers and business users should care about Data Observability for several reasons. Greater transparency through the continual monitoring of data quality and data pipelines improves trust in customer data, helps reduce the negative impact of data issues and allows for proactive data management. The validity of data has always been of interest to marketers. One reason Data Observability is gaining traction is that it is easier for marketers to monitor the health of data than it used to be. Traditionally, marketers sought the help of IT or data analysts in vetting customer data, a process that took time away from being able to execute campaigns with the confidence in the underlying data. Even today, the default protocol for many operational marketers is to summon IT. In doing so, they are making the calculation that having the utmost confidence in the health of their data is worth the delay.
Time, though, is a luxury marketers no longer have when it comes to keeping pace with the always-on, connected consumer. A dynamic customer journey across multiple online and offline channels is fast and unpredictable. Meeting the right customer with the right message on the right channel and at the right time is a delicate operation that requires accurate, timely data that is fit for its intended business purpose. Marketers who accept a delay, however short, in running a campaign risk alienating customers with irrelevant or untimely messages, offers or content.
Data Observability: Key to Confidence
Data Observability allows marketers to confidently make decisions about how to proceed in their day-to-day activities. Data issues – as any marketer can attest – can and often do happen; Data Observability is what allows marketers to head those issues off at the pass before they become bigger issues, or before they negatively impact customer experience.
Data Observability is like being able to see the flow of water from source to destination. If the only time you notice a problem is when you see murky, brown water coming out of your tap, you will likely put a plumber on speed dial and shut everything down until however long it takes a plumber to diagnose and fix the problem. But if you could continually check the pipes and the water flowing through it, perhaps you detect the issue before it flows to the tap, make a quick repair yourself and not skip a beat.
By avoiding having to call the plumber – the homeowner version of calling on a data analyst team or building out SQL code – marketers not only save valuable time, they also begin to learn about data – where and why are the common issues occurring, is there one feed in particular that’s causing a problem? Is another feed always late? Does another return wildly inconsistent numbers? By providing access to data pipelines, Data Observability not only vets the validity of data, it allows marketers to gain trust and confidence in the actual data feeds; this one has never had an issue, I see this one is always on time, etc.
Data Observability in the Redpoint CDP
With an understanding of what Data Observability is, how exactly does transparency and visibility manifest itself? In the case of the Redpoint CDP, a single UI provides graphical visualizations that allow marketers to detect patterns, discover issues and validate the health of customer data as it moves through the CDP. Coding is never needed. Redpoint provides a simple visual representation of different aspects of data quality. (See Figure 1)
FIGURE 1 – A Data Observability dashboard in the Redpoint CDP showing feed state and status and the quality of a data match of a CRM system.
In another Data Observability dashboard (See Figure 2) marketers can dive into the data feeds, checking the status, the number of records, whether there are duplicates, and easily gauge the validity, timeliness and completeness of each feed. If there is an issue, marketers can then drill down further to investigate the cause. These are simple visual representations that would normally require marketers and business users to have access to the database, pull data, write SQL.
FIGURE 2 – A Data Observability dashboard in the Redpoint CDP showing the validity, timeliness and completeness of data feeds.
Having continuous daily access to data in a Data Observability dashboard also provides an instance history. If there is, say, an unusual number of customer records in a particular feed, it’s simple to check whether or not there is an actual anomaly. In the types of instances that might not ring an alarm bell, it’s doubtful that a marketer would ever ask IT to look into the matter. But with easy access to the daily ebb and flow of a particular feed and how it flows into the rest of the CDP, Data Observability dashboards in the Redpoint CDP give marketers the ability to ask questions of data that might otherwise be ignored.
Does Your CDP Offer Data Observability?
Data Observability is not a replacement for performing data quality, data enrichment and identity resolution as soon as data enters the system, all of which the Redpoint CDP takes care of as core functionality. Because many CDPs outsource those key functions, they are unable to provide details about the health of data that flows through the system. How can they provide an updated data hygiene report (without writing a line of code) if data quality steps are offloaded to a third party, or completed somewhere downstream?
The answer is – they can’t. Completing all data quality processes and creating and updating a Golden Record as data is ingested is a differentiating feature of the Redpoint CDP. Data Observability provides marketers with a simple way to validate that data are performing as expected as these processes occur. Great marketing campaigns start with great data. With Redpoint, marketers know in advance that all systems are go.
