Patrick Flannery
The need for exchange data of very high quality, either raw or lightly normalised, has never been greater than it is today. Tick History – PCAP meets these robust standards, and Refinitiv, an LSEG business, is investing further to meet tomorrow’s evolving customer requirements.
- Data quality standards for historical data are more rigorous for use cases aligned with the electronic trading lifecycle.
- Ensuring lossless capture – the hallmark of high data quality – requires superior technology, processes and people.
- Refinitiv is investing in Tick-History – PCAP and its capture technology, Real-Time – Ultra Direct to expand coverage and other aspects of quality to meet the evolving needs of the marketplace.
As the use of technology in trading grows – with electronification, algorithms, artificial intelligence, machine learning and more – the importance of deploying high-quality data is increasing dramatically.
So, packet-captured (PCAP) historical data that supports use cases aligned with ultra-low latency trading needs to be both extremely robust and precise. With the acquisition of MayStreet in May 2022 by LSEG – Refinitiv’s parent company – there is now significant potential to grow Refinitiv Tick History – PCAP data across several dimensions.
What is data quality?
For electronic trading needs, data quality for market data is determined by five key factors:
- Accuracy – To be accurate, the market data needs to be valid. That is, the data must contain the values disseminated by the exchange and it needs to be of the highest fidelity.
- Accessibility – Quality data is accessible when and where you want it.
- Completeness – This means that data sets must contain all of the data elements provided by the exchange and that all data required for a use case is available.
- Latency – Data is captured as it comes out of the exchange, with minimal network hops and high-performance network infrastructure (switches/routers) to minimise network latency.
- Usability – The ability of financial services firms to use data in the format that they prefer.
Electronic trading needs require historical data with the same robust quality standards as the original feed.
Refinitiv Tick History – PCAP provides data captured directly from exchanges using Real-Time – Ultra Direct, Refinitiv’s ultra-low latency feed handler – with high levels of accuracy, accessibility, completeness, and usability, as well as low levels of latency in the capture process. This ensures that financial services firms can trust Tick History – PCAP for even the most sensitive use cases.
Refinitiv Tick History — PCAP data is captured at exchange data centres with nanosecond precision, and in a lossless manner to ensure the highest quality.
Patrick Flannery
How are gaps in data addressed?
While Tick History – PCAP provides robust historical data today, data quality can always be evolved further to support the developing demands of customers’ use cases.
For example, during the T+1 processing, Tick History – PCAP automatically uses primary and backup captures at the primary data centre only where available and when there would be no impact on processing time. Arbitrated data is examined for drops the next morning – if any are found, this is investigated to determine whether including backup sites would be beneficial.
Many drops occur at the exchange level, which TCP replays – utilities for editing and replaying previously captured network traffic – can help remedy.
Over the coming months, the Tick History – PCAP team plans to include TCP replays with all the data captures. If there is a drop, the technology will automatically capture the replays and deliver those with the feed.
Longer term, the technology will automatically perform the drop examination processes before arbitration, allowing us to determine the best mix of sites to include in that night’s arbitration. Ultimately, this will deliver even more accurate data to customers for deployment in their most sensitive use cases.
End-to-end process flow to achieve data quality
What else does the acquisition of MayStreet by LSEG mean for the data quality of Refinitiv Tick History – PCAP?
Refinitiv is deeply committed to investing in the technology that supports Refinitiv Tick History – PCAP and innovating to enhance the Real-Time – Ultra Direct feed handler – all with a focus on data quality.
In addition to expanding coverage, ongoing improvements include:
- Switching to independent time sources to add redundancy, reduce latency and improve time accuracy.
- Undertaking implementation improvements, including enhancements in backhaul timing at all sites worldwide.
- Upgrading our infrastructure so that data can be captured as near to an exchange’s matching engine as possible using capture switches that are 100 nanoseconds from the wire.
Many other improvements are planned or are underway.
Legal Disclaimer
Republication or redistribution of LSE Group content is prohibited without our prior written consent.
The content of this publication is for informational purposes only and has no legal effect, does not form part of any contract, does not, and does not seek to constitute advice of any nature and no reliance should be placed upon statements contained herein. Whilst reasonable efforts have been taken to ensure that the contents of this publication are accurate and reliable, LSE Group does not guarantee that this document is free from errors or omissions; therefore, you may not rely upon the content of this document under any circumstances and you should seek your own independent legal, investment, tax and other advice. Neither We nor our affiliates shall be liable for any errors, inaccuracies or delays in the publication or any other content, or for any actions taken by you in reliance thereon.
Copyright © 2023 London Stock Exchange Group. All rights reserved.
The content of this publication is provided by London Stock Exchange Group plc, its applicable group undertakings and/or its affiliates or licensors (the “LSE Group” or “We”) exclusively.
Neither We nor our affiliates guarantee the accuracy of or endorse the views or opinions given by any third party content provider, advertiser, sponsor or other user. We may link to, reference, or promote websites, applications and/or services from third parties. You agree that We are not responsible for, and do not control such non-LSE Group websites, applications or services.
The content of this publication is for informational purposes only. All information and data contained in this publication is obtained by LSE Group from sources believed by it to be accurate and reliable. Because of the possibility of human and mechanical error as well as other factors, however, such information and data are provided "as is" without warranty of any kind. You understand and agree that this publication does not, and does not seek to, constitute advice of any nature. You may not rely upon the content of this document under any circumstances and should seek your own independent legal, tax or investment advice or opinion regarding the suitability, value or profitability of any particular security, portfolio or investment strategy. Neither We nor our affiliates shall be liable for any errors, inaccuracies or delays in the publication or any other content, or for any actions taken by you in reliance thereon. You expressly agree that your use of the publication and its content is at your sole risk.
To the fullest extent permitted by applicable law, LSE Group, expressly disclaims any representation or warranties, express or implied, including, without limitation, any representations or warranties of performance, merchantability, fitness for a particular purpose, accuracy, completeness, reliability and non-infringement. LSE Group, its subsidiaries, its affiliates and their respective shareholders, directors, officers employees, agents, advertisers, content providers and licensors (collectively referred to as the “LSE Group Parties”) disclaim all responsibility for any loss, liability or damage of any kind resulting from or related to access, use or the unavailability of the publication (or any part of it); and none of the LSE Group Parties will be liable (jointly or severally) to you for any direct, indirect, consequential, special, incidental, punitive or exemplary damages, howsoever arising, even if any member of the LSE Group Parties are advised in advance of the possibility of such damages or could have foreseen any such damages arising or resulting from the use of, or inability to use, the information contained in the publication. For the avoidance of doubt, the LSE Group Parties shall have no liability for any losses, claims, demands, actions, proceedings, damages, costs or expenses arising out of, or in any way connected with, the information contained in this document.
LSE Group is the owner of various intellectual property rights ("IPR”), including but not limited to, numerous trademarks that are used to identify, advertise, and promote LSE Group products, services and activities. Nothing contained herein should be construed as granting any licence or right to use any of the trademarks or any other LSE Group IPR for any purpose whatsoever without the written permission or applicable licence terms.
