Page 8 - Data Transparency White Paper_FINAL_Neat

Addressing Biases in Multicultural & Inclusive Identity Data
8

ARE THESE IDENTITY APPROACHES ACCURATE AND VALID?
Data providers that took part in the 2018 RFI were open with AIMM about describing their approaches and
the science behind their identity assignments. But when AIMM asked how accurate these approaches are,
many providers said they are “95 percent accurate” without providing substantiation or proof. Many declined
to share validation studies, saying that they were proprietary to clients. Only one provider shared a validation
study with AIMM, but it had been conducted in 2012.

The lack of validation of multicultural data drove the rest of the AIMM data transparency initiative.

SYSTEMIC BIAS IN MARKETING DATA

SIZING THE PROBLEM: UNDERREPRESENTATION OF

MULTICULTURAL CONSUMERS IN THIRD-PARTY DATASETS
As part of this effort, AIMM obtained data from two independent sources that provide insights into the quality
levels of multicultural data in third-party datasets. Both studies involved evaluating the accuracy and cov-
erage of identity data from a variety of providers at different points in time. The studies compared provid-
ers’ identity assignments to independent representative datasets and offered preliminary indications of the
current state of the practice. It is AIMM’s goal to benchmark and quantitatively document improvements in
multicultural data quality over time.

BENCHMARK STUDY 1: NIELSEN, 2011-18
Nielsen shared with AIMM the aggregated results from a number of third-party
data validations studies conducted from 2011 through 2018. Nielsen blinded, aggre-
gated and averaged the results of these studies for AIMM. As part of standard pro-
cedures, Nielsen evaluates the quality of third-party data to correct and calibrate
its data for bias. Nielsen’s insights shed light on how well multicultural consumers are covered and accurately
identified in a variety of commercially available offline data sources.

Approach
Nielsen used the same methodology in each of these validation studies (conducted at different points in time
across several years), and compared the demographic profile assigned by third-party providers to its own
directly collected demographics from the same household. Nielsen has the advantage of collecting, verifying,
and regularly updating demographic information directly from its panel households. This process allows
Nielsen to compare its own up-to-date demographic information to that which is assigned by the third-party
data providers, allowing them to measure the missing data (coverage) and accuracy.