Anticipating Err0r Rates in the CAT
In this article we look at error rates and how they will be applied under the Consolidated Audit Trail (CAT). We will be assessing what areas firms should focus on to reduce error rates and the tools and processes clients have at their disposal.
Firstly the SEC Rule 613 defines Error Rate as “the percentage of reportable events collected by the central repository in which the data reported does not fully and accurately reflect the order event that occurred in the market1."
Every CAT Report will be given an error rate calculated using three criteria 1) timeliness of reporting, 2) data accuracy, and 3) data linkages. CAT Reporters error rates will be reviewed at least annually, with an initial benchmark of 5% errors being set by the CAT processor as an maximum acceptable rate2. This will be calculated during the CAT Processors data and linkage validation processes and will be based on the number of erroneous records divided by total number of records received on any given day3. This acceptable benchmark rate will go down from 5% to 1% once that category of reporter (i.e. participants, large broker-dealers, small broker-dealers) has been live for a year or more4. CAT Reporters will be notified each time that they have exceeded the maximum allowable error rate and will be informed as to the reason(s) they did not comply.
The error rate stands as one of the fundamental components to monitoring a CAT reporter’s compliance. CAT reporters will also be required to meet separate Compliance Thresholds as set by the Operating Committee.
In order to remain below the error rate threshold firms will need to ensure they have effective means of monitoring their reporting activities. To aid CAT Reporters in monitoring overall compliance, the CAT Processor will publish CAT Reporter compliance report cards; these reports will contain information containing number of inaccurate, incomplete and late transactions5. But what more can firms do to remain compliant and improve and maintain an appropriate error rate?
If we take a look at existing reporting requirements such as Order Audit Trail System (OATS) and Electronic Blue Sheets (EBS) reporting, we can see that reporting practices have been closely monitored. Failure to comply with the reporting requirements can result in high remediation costs for a firm. A study conducted in 2011 by Oxford academics found that poor reporting practices cause losses to shareholders that on average are almost 9 times the size of a penalty for non-compliance.6
New Areas to Assess
What areas should firms highlight when looking at the implementation of CAT? The quick answer is the areas today that are not covered by OATs reporting; which is options data, allocations data and personally identifiable information (PII) data.
Each of these areas and requirements under CAT, that did not exist under OATs, will pose new requirements such as the reporting of options. For example, reporting options could require clients to undergo a significant program of work to capture reports for orders, quotes and executions thus increasing the risk of higher error rates if not properly tested.
As Options historically have not been reported under OATs for Broker-Dealers, firms will likely be stepping on new ground in this area. Allocations data, although not mandated to be linked to its order data under CAT, will need to be reconciled and matched internally with the order/trade data and reported on T+1 like other order events.
This could present issues if mismatches or exceptions are exposed during the trade confirmation process. PII data, although not explicitly captured within the scope of the error rates definition may become an area that could be explored in the future to gain further transparency and understanding of market participants.
How to Mitigate and to avoid errors from day one
The immediate advice is for firms to start early with impact assessment and a gap analysis between OATs and CAT. Firms should also analyze reporting procedures and assess which systems will need to be brought in scope to comply with CAT. It is key to understand to what extent do these systems need to be upgraded to provide accurate reporting using the new specification, and better visibility over the reporting process.
Firms should also be checking for data integrity issues in their static and reference data for example a review of customer account mappings across different groups within the firm and an assessment to the completeness of entity & customer data should be undertaken.
Broker-dealers will need to put in place plans for a rigorous testing cycle before go-live to manage the increased scope and volume of data, but should also consider putting in place a set of unit tests to cover data linkage requirements which is a key component in generating a CAT Reporters error rate.
Once live, clients should conduct periodic reviews of processes in order to improve standards. This can demonstrate a pro-active approach to their regulatory reporting operations fostering a culture of improvement. Under the current NMS plan post-production support includes:
- Issuing a monthly Report Card on reporting statistics with information on how reporters stand against similar entities 7
- Publishing of daily reporting statistics , these key performance indicators (KPIs) should be utilized by firms as a benchmark to their own regulatory infrastructures performance but also where they’re benchmarked against their peers.8
More applied measures include validating outgoing transactional data before it leaves the firms reporting infrastructure. Catching exceptions so they can be corrected before a report is submitted, or to warn operations teams of errors ahead of time so they more easily meet the timeline requirements imposed around error-handling within CAT.
Finally, firms should implement data reconciliation procedures to confirm that data in source systems matches with data sent to the CAT Processor and data that will continue to be sent to FINRA under OATS. This will ensure that both OATs and CAT reports are aligned and improve data accuracy under both regimes.
Clients should take a pro-active approach in tackling CAT to put in place a comprehensive framework to keep error rates low. This should start before go-live with assessment and GAP analysis, moving through to live using tools to validate submissions before they leave a firm’s reporting ecosystem. Finally, error rates & Compliance Report Cards can provide useful benchmarks to performance and should be utilized by firms to encourage a philosophy of continuous improvement in reporting procedures.
1 SEC Rule 613(e)(6)(i)
2 SEC Rule 613(e)(6)(ii)
3 CAT NMS Plan 3 (b) Error Communication, Correction, and Processing Appendix C – 23 (p115)
4 CAT NMS Plan 3 (b) Error Communication, Correction, and Processing Appendix C – 25 (p117)
5 CAT NMS Plan 10.4 CAT Reporter Compliance, Appendix D – 41 (p265)
7 CAT NMS Plan 10.4 CAT Reporter Compliance, Appendix D – 41 (p265)
8 CAT NMS Plan 10.1 CAT Reporter Support, Appendix D – 39 (pg 263)