The importance of data accuracy is well documented. A Federal Trade Commission study of the U.S. credit reporting industry found that one in five consumers had an error on at least one of their three credit reports. More significantly, 5% of consumers had errors that could result in less favorable terms for financial products such as auto loans and insurance.
The findings demonstrate how the presence of information does not automatically establish its accuracy or usefulness.
That distinction should influence how collection agencies, creditors, debt buyers, and recovery law firms evaluate data providers. Decision-makers also need to understand whether the information is accurate, current, relevant, and capable of improving the specific recovery process being tested.
A well-designed collection data test provides a structured way to answer those questions.
Start With the Business Problem, Not the Product
One common mistake in vendor evaluation is allowing product capabilities to define the test. A provider may offer hundreds of attributes, extensive consumer coverage, monitoring capabilities, or sophisticated technology. Those features can be valuable, but they should not determine the business objective.
The organization should define the problem first.
- Is the goal to improve contactability?
- Identify verified employment?
- Strengthen post-judgment recovery?
- Reduce manual research?
- Improve account segmentation?
- Or find actionable information within a specific portfolio?
Each objective requires a different test.
This principle emerged during a recent Receivables Podcast conversation with Dane Mauldin, President of RNN Group. Our discussion explored how organizations can evaluate collection data based on what happens after information is returned, including whether it supports contact, agreements, settlements, employment identification, or other meaningful outcomes.
So, how should an organization determine whether a new data source warrants scaling?
The Define-Isolate-Measure-Refine Framework
A practical data collection and testing process can be organized into four stages: Define, Isolate, Measure, and Refine.
Define the Objective
“Testing a new data provider” is not specific enough. The organization should identify what it expects the information to change.
For example, a test could determine whether verified employment information improves outcomes within a defined post-judgment population. Another could evaluate whether updated contact information improves successful consumer engagement within accounts that previously lacked usable contact data.
Success criteria should also be established before the test begins. Clearly defined benchmarks reduce uncertainty by providing an objective framework for evaluating results against established success criteria.
Without those benchmarks, teams can easily interpret the same results differently.
Isolate the Right Population
The quality of a test depends heavily on the population selected.
Combining accounts with substantially different balances, ages, jurisdictions, treatment histories, or recovery characteristics can make it difficult to determine why results changed. A more disciplined approach selects accounts that correspond directly to the business question being evaluated.
Where appropriate, comparable test and control populations can strengthen the analysis.
Maintaining consistency throughout the evaluation period also enables organizations to isolate the impact of the new data or technology and assess its effectiveness with greater confidence.
The goal is to remove enough unnecessary variation that the organization can make a reasonable assessment of what produced the observed outcome.
Measure More Than Match Rates
Match rates and record counts remain useful indicators. They show whether a provider can locate information across a selected population. They should not, however, become the final definition of success.
Imagine one provider returns information on 80% of tested accounts while another returns information on 60%. Based exclusively on coverage, the first provider appears stronger.
Now consider what happens if the second provider’s information contributes to more successful contacts, verified employment results, payment arrangements, settlements, or recoveries.
The business conclusion changes.
Vendor evaluations should therefore prioritize measurable improvements in recovery performance, with match rates and record counts serving as only part of the overall assessment.
The appropriate downstream metrics will depend on the original objective. What matters is creating a measurable connection between the information received and the business result the organization wanted to improve.
Information Quality Belongs in the Vendor Evaluation
Accuracy deserves particular attention because erroneous consumer information can create consequences beyond operational inefficiency.
The FTC study illustrates the potential significance of inaccuracies within consumer information systems. The CFPB has also reported that information associated with collections has historically generated substantial consumer disputes. In discussing the credit reporting system, the Bureau reported that more than one-third of consumer disputes involved collection items and that information provided by the collections industry was five times more likely to be disputed than mortgage information.
These findings concern credit reporting and should not be interpreted as measurements of commercial collection-data-provider accuracy. They do, however, reinforce the need for organizations that handle consumer information to have disciplined processes for assessing reliability.
Turn Testing Into a Feedback Loop
A solution may perform exceptionally well in one account segment but deliver limited value in another. Data that works for a post-judgment population may have a different value proposition for earlier-stage accounts. Another solution may produce its strongest results only when triggered at a specific point in the account lifecycle.
Performance feedback can reveal those differences.
A more mature testing process therefore follows a continuous cycle:
Test → Measure → Learn → Adjust → Retest
The objective is to determine not simply whether a data provider “works,” but where, when, and under what circumstances the information creates value.
For collection leaders, the larger lesson extends beyond any individual vendor. Data strategies become stronger when organizations continuously learn from their own results.
Scaling Should Follow Evidence
Pressure to improve productivity makes investments in data and technology increasingly important. It also raises the cost of scaling a solution before understanding whether it produces meaningful value.
Organizations do not need endless pilot programs. They need sufficient, disciplined evidence to determine whether a solution achieves the intended outcome and where it performs best.
Only then should scaling be the next topic of conversation.
The competitive advantage is unlikely to come from simply having access to more data. It will come from developing the discipline to distinguish between available information and information that actually changes decisions.
For more perspectives on collecting data, operational strategy, technology, and the changing receivables environment, explore the latest industry resources and thought leadership at Receivables Info news.
About Adam Parks
Adam Parks has become a voice for the accounts receivable industry. With almost 20 years of experience in debt portfolio purchasing, debt sales, consulting, and technology systems, Adam now produces industry news, hosts hundreds of episodes of Receivables Podcasts, and manages branding, websites, and marketing for over 100 companies in the industry.
