Nell’iperspazio con Rocket: il Framework Web di Rust!
Extent 2013 Obninsk LSE - The Focus Beyond Low Latency
1. The focus beyond Low Latency
EXTENT Trading Technology Trends & Quality Assurance
Eric Benedetti, Luciano Zanetti
Head of Technology Products Development & Head of Technology Equity Products LSEG
3 March 2013
1
2. Agenda
1. Introduction
2. Key features from the Borsa Italiana migration to MIT
3. MIT convergence road map for LSE, Borsa and Turquoise
4. Group Ticker Plant
5. Group Trading Interface
6. Sponsored Access
7. Conclusion
2
3. Introduction
April 2012
• LSE
– Trading latency (Order ack): 148 us (99% avg)
– Market data latency (Tick): 50 us (99% max)
– Current Capacity (tps): 300,000 (scalable)
• Turquoise:
– Trading latency (Order ack): 103 us (99% avg)
– Market data latency (Tick): 60 us (99% max)
– Current Capacity (tps): 300,000 (lit) + 150,000 (dark) both scalable
after rebalancing and the addition of a new partition
• LSE
– Trading latency (Order ack): 137 us (99% avg)
Consistency improved significantly after rebalance (max 99%)
– Current Capacity (tps): 450,000 (scalable)
3
4. Introduction
LSE Market Share (calculated on value traded)
Migration
to MIT
New
Pricing
Trend in MTFs % on Trading in LSE Total Shares (calculated on value traded)
Source:- LSEG-MA analysis on LSEG, BATS figures. Latest update end August 2012.
4
5. Introduction
BIT Market Share (calculated on value traded)
Migration to
MIT
First TUG
Trend in MTFs % on Trading in BIt Total Shares (calculated on value traded)
Source:- LSEG-MA analysis on LSEG, BATS figures. Latest update end August 2012.
5
6. Agenda
1. Introduction
2. Key features from the Borsa Italiana migration to MIT
3. MIT convergence road map for LSE, Turquoise and Oslo
4. The Group Ticker Plant
5. Group Trading Interface
6. Sponsored Access
7. Conclusion
6
7. New Trading Functionalities
• Order Imbalance during auction calls
• Stamping VWAP in Execution Reports of Orders & Quotes
• Stop and Stop Limit Orders
• Market to Limit Orders
• Cancel on disconnect
• Named Orders
• Cross Orders & BTFs
7
8. Reliability – Hot-Hot Architecture
• Focus on reliability vs performance – target 400 us
• 3 market partitions (equities, bonds, warrants) + trade reporting & after hours
• MIT synchronous mode was chosen for implementing a hot-hot configuration
• The system runs in a hot-hot configuration in a distributed and synchronous
mode in the PDC an SDC
• Every system’s component is running on 2 instances in PDC and in SDC for a
total of 4 instances per each component (hardware and software)
• Customers are able to access both sites cuncurrently; network failover is
automatic and transparent
• For latency reasons, SDC was moved from Turin to Milan
8
9. Reliability – MIT Synch mode
• PDC and SDC are synchronized at transaction level. Every transaction is
committed on both sites before the ack is returned to clients
• Market data is disseminated by each site independently
• Each single logical or physical component, market partition, or the full site
can failover independently and automatically to SDC in a completely
transparent way
• The system is able to automatically recover:
the fail of any primary hardware or software system component of PDC
the fail of any primary AND secondary hardware or software component
the fail of one or more single market partitions
the fail of the entire PDC
the partial or full fail of SDC without impacting PDC
• The system guarantees no loss of trades or orders
9
10. Reliability – Trading consistency
< 300 us
2 Sync copy
MIT Exchange MIT Exchange
SDC PDC
Milan Milan
3 Sync confirmation 5 Async Trade
Order entry Execution report Capture Report
1 4
Customer Customer
EMS OMS
10
11. Reliability – Market Data consistency
Channel B
t+150 us
MIT Exchange MIT Exchange
SDC PDC
Customer
Milan Customer
Milan front-ends front-ends
Channel A
Incoming order t ms
t+300 us 1. Channel B delay 300 us
2. Arbitration A vs B
3. Request retrasmission
Customers
11
12. Agenda
1. Introduction
2. Key features from the Borsa Italiana migration to MIT
3. MIT convergence road map for LSE, Turquoise and Oslo
4. The Group Ticker Plant
5. Group Trading Interface
6. Sponsored Access
7. Conclusion
12
13. LSEG convergence roadmap
2012 2013
May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug
LSE Release 9
decommission
Go live
LSE MIT Build QA
CDS LSE (MIT Ex release 9)
TNP Release 9
decommission
QA Go live
TQ MIT Build TNP (MIT Ex release 9)
CDS
BIT migration
to release 18
BITTarget BIT (MIT Ex release 14) BIT (MIT release 18)
QA
Go live
OB Target BIT + OB (MIT Ex release 16)
CDS LSE migration
backward
compatibile
LSE Target BA MIT Build QA
BIT + OB + LSE (release 18)
CDS
TQ Target MIT Build QA
CDS Release 18
13
14. Benefits of the Convergence
• Customers and ISVs have one protocol for all the equities markets costs for
Customers can be reduced
• Functionalities developed for one market can be re-used for other markets better
time to market & product development cost can be shared
• Each market can maintain its own market model no need to harmonize
• Releases can be deployed at predefined time windows for all markets improvement
of Customers planning and budgeting
• MIT has one Exchange release to maintain for all markets Implementation cost
efficiency
• Same Technology Operations team can operate different markets in different
geographical locations Operational cost efficiency
14
15. Additional enhancements
• New Business requirements - These include:
o iceberg order enhancements
o optional self-execution prevention
o visible or hidden indicator on passive order Execution Reports.
• Group Ticker Plant Integration (GTP) – Changes will be introduced into the
MIT Exchange system in order to be tightly integrated with the new version
of the GTP system. Thanks to hardware accelerated components, GTP 2 will
be able to disseminate tick by tick price increments (L2 market data) faster
than the current ITCH market data feed.
• Group Trading Interface Integration (GTI) – Ahead of planned availability
of new interface for actual use in December 2013, we need to build a
connection into the matching engine. Release 8 provides this link which,
when activated, will allow GTI to run initially in parallel to the existing Fix
and Native Gateways.
15
16. Additional Enhancements – continued
• Backward Compatibility of Client Facing Interfaces – In order to minimize the
changes that clients have to implement, the client facing interfaces will be made
backward compatible for specifically agreed gaps.
• New Operational requirements – Specific Market Operations requirements will be
introduced as part of the scope of this release (for example, halting instruments
across multiple days, enhance the Order Query functionality in the Service Desk
tool, introducing Members Mnemonics, etc). These are intended to improve the
efficiency of the Market Operations team.
• Enhancements to the MIT Surveillance system – The release will complete the de-
coupling of the MIT Surveillance system from the Trading system. This will allow for
greater flexibility in the implementation of future Surveillance releases.
16 16
17. Agenda
1. Introduction
2. Key features from the Borsa Italiana migration to MIT
3. MIT convergence road map for LSE, Turquoise and Oslo
4. The Group Ticker Plant
5. Group Trading Interface
6. Sponsored Access
7. Conclusion
17
18. Current Market data Architecture
Equities Derivs Bonds
Millennium Exchange SOLA Trade Impact
London Cash Equities Borsa Italiana Equities
Turquoise Equities Borsa Derivs Turquoise Derivs MTS Cash, Repo &
& Retail Bonds Oslo Equities & Bonds & Fixed Income
Lit & Dark IDEM, IDEX, AGREX TQD & OB Bondvision
SETS & ORB MTA,MOT,ETF,SEDEX
Level 2-ITCH FIX-FAST Level 2-ITCH Level 2-ITCH FIX-FAST Level 2-ITCH HSVF HSVF SDP
(20 depth) (20 depth) (5 depth) (5 depth) (5 depth)
User User User User User User User
DDM-Plus (Only Borsa Italiana)
EMAPI EMAPI
(full depth) 5,10,20 depth)
User
18
19. Introducing our Ticker Plant
• All LSEG market data will be delivered on a single bespoke protocol
– Binary, bandwidth efficient 10 level snapshot feed (intelligent throttling)
– Binary, performant protocol for tick by tick full depth OB feed
– Binary, flexible TCP/IP protocol for recovery and reference data download
• Data delivered by load balanced IP Multicast channels
• Hardware assisted data delivery (FPGA) technology
• Hot-Hot architecture disseminating from Primary and Secondary sites
• Independent Primary and Secondary Data Centres in London and Milan
19
20. Phase 1: live in Nov 2012
• Real Time incremental Level 1 Best Bid and Offer Service (Top Of the Book)
• Snapshot service for 10 levels MBO and MBP (150 ms time interval)
• Intelligent throttling: non-latency sensitive messages delivered in multi-
message UDP packets whereas latency sensitive messages delivered in single
message UDP packets
• Enriched reference data package delivered on the wire (unicast)
• Rich statistics package
• FTSE indices disseminated
20
21. Phase 1.1: live in Feb 2013
• Launch of FTSE First – the most popular FTSE indices including the FTSE100,
FTSE250, FTSE All Share and FTSE MIB calculated incrementally by the Ticker
Plant on next generation technology and delivered directly to Exchange hosted
customers
• market gateways on FPGA for latency reduction
21
22. Phase 2: planned for Jul 2013
• Turquoise equities integration
• Real Time full depth incremental service (full depth)
• New levels of technical performance (introduction of FPGA fr cliens data
dissemination)
22
23. GTP Target Architecture
Equities Derivs Bonds
Millennium Exchange SOLA Trade Impact
London Cash Equities Borsa Italiana Equities
Turquoise Equities Turquoise Derivs Borsa Derivs MTS Cash, Repo &
& Retail Bonds & Fixed Income Oslo Equities & Bonds
Lit & Dark EDX & OB IDEM & IDEX Bondvision
SETS & ORB MTA,MOT,ETF,SEDEX
HSVF HSVF SDP
FIX-FAST
(5 depth) (5 depth) (5 depth)
FPGA feed FPGA feed FPGA feed
User User User User
GTP
Level 2 Replay &
Level 1 & 2
snapshot Recovery
MBO (full depth)
(10 depth) Ref Data
FTSE
User
23
24. And this is only the beginning (Phase 3) …
• Extend the coverage to Europe and US markets in a simple,
normalised format
• Market wide statistics to ensure our trading community have a
total view of a securities’ trading patterns and real time
behaviour
• Collapse other external products into our Group market data
feeds – diversifying our product to facilitate additional revenues
and delivering more information to our trading community
• Provide clients a vision into our technology through real time
performance statistics allowing our HFT trading community to
fine tune their market interactions based on our real time
performance
24
25. Agenda
1. Introduction
2. Key features from the Borsa Italiana migration to MIT
3. MIT convergence road map for LSE, Turquoise and Oslo
4. The Group Ticker Plant
5. Group Trading Interface
6. Sponsored Access
7. Conclusion
25
26. Introducing the Group Trading Interface
• The Group Trading Interface (GTI) will provide our customers with a single trading interface allowing
them to manage their orders across the existing LSEG cash and derivative markets.
• The project will be delivered in a phased approach as clients move across to the new interfaces in as
short as time period as possible.
– Q4 2013 Phase 1- covering the LSEG cash markets (LSE, Turquoise and Borsa Italiana)
– Q2 2014 Phase 1.1- Introducing Prescriptive Order Routing strategies within LSEG markets
– Q3 2014 Phase 1.2- Expand the GTI coverage to Derivatives
– Q4 2014 Phase 2- The new service will connect into the GTP and become “market data aware”,
allowing order routing strategies. FPGA technologies will be introduced to improve performance.
• Customer Benefits
– Single order entry API/Session to access all markets, driving lower technical/development costs, and/or
lower third party fees from EMS/SOR vendors
– Operational efficiencies through unified symbology, common risk-controls, common
reporting/reconciliation, etc.
26
27. Market access current architecture
Equities Derivs Bonds
Millennium Exchange SOLA Trade Impact
London Cash Equities Borsa Italiana Equities MTS Cash, Repo &
Turquoise Equities Borsa Derivs Turquoise Derivs
& Retail Bonds & Fixed Income Bondvision
Lit & Dark IDEM, IDEX, AGREX TQD & OB
SETS & ORB MTA,MOT,ETF,SEDEX
Native & FIX Native & FIX Native & FIX SAIL & FIX SAIL & FIX SDP
User User User User User User
27
28. GTI Market access Target architecture
Equities Derivs Bonds
Millennium Exchange SOLA Trade Impact
London Cash Equities Borsa Italiana Equities MTS Cash, Repo &
Turquoise Equities Borsa Derivs Turquoise Derivs
& Retail Bonds & Fixed Income Bondvision
Lit & Dark IDEM, IDEX, AGREX TQD & OB
SETS & ORB MTA,MOT,ETF,SEDEX
GTi
Native & FIX
User
28
29. Agenda
1. Introduction
2. Key features from the Borsa Italiana migration to MIT
3. MIT convergence road map for LSE, Turquoise and Oslo
4. The Group Ticker Plant
5. Group Trading Interface
6. Sponsored Access
7. Conclusion
29
30. Introducing the new Sponsored Access
• Exchange participants (Members) are allowed to provide trading access to
non-exchange members through their membership. However Members must
demonstrate that they have adequate risk controls measures set up to prevent
these Sponsored participants (i.e. non-exchange members) from trading
beyond a limit the member can be financially responsible for.
• Borsa Italiana requires Members to enforce these risk control mechanisms.
• In order to support the above mentioned requirements Borsa Italiana wants to
introduce the Sponsored Access functionality which supports validations and
other features required by Borsa Italiana Trading System to allow Sponsored
Access trading.
• Borsa Italiana should provide to the Clients a Sponsor Portal that allows, via
GUI, the configuration in real time of all the parameters needed to manage
the risk control mechanism.
30
31. New Sponsored Access validations
The following risk validations are required by Borsa Italiana:
• Price Band Validation – Prevent orders from Sponsored Users with an overly aggressive
price from entering the order book and trading.
• Maximum Gross Consideration – Prevent Sponsored Users from trading beyond a threshold
specified in a configurable currency (e.g. Euros, Great Britain Pounds etc). The gross
consideration is the sum of all trades and value of all open orders of the current trading
day.
• Maximum Order Value – Prevent orders (limit and market) with an uncommonly large
order value from entering the order book.
• Maximum Order Quantity – Prevent orders from Sponsored Users with an uncommonly
large order size from entering the order book.
• Order Capacity - Prevent orders from Sponsored Users with an unexpected Capacity of
the order.
• Account Type - Prevent orders from Sponsored Users with an unexpected Account Type of
the order.
• Restricted Instrument List – With this feature, the Sponsor User would be able to update
their Sponsored Users’ restricted instrument lists.
• Maximum Message Rate – it’s requested, as technical requirement, to validate that the
incoming rate injected by the Sponsored User does not exceed the Maximum Message
Rate (number of transactions per second) configured by the Sponsor User.
31
32. Sponsored Access Target solution
Equities Derivs Bonds
Millennium Exchange SOLA Trade Impact
London Cash Equities Borsa Italiana Equities MTS Cash, Repo &
Turquoise Equities Borsa Derivs Turquoise Derivs
& Retail Bonds & Fixed Income Bondvision
Lit & Dark IDEM, IDEX, AGREX TQD & OB
SETS & ORB MTA,MOT,ETF,SEDEX
Sponsored Access
Execution Native & FIX
report
Member Sponsored
32
33. Sponsored Access logical architecture
MIT Trading
and Information
FIX 5.0 NATIVE
PostTrade
GTP Sponsor Access Service
Drop copy SA GTW SA GTW SA GTW SA GTW
FIX 5.0
Member
Member
Supervisor Wkst
Sponsored
User
33
34. Agenda
1. Introduction
2. Key features from the Borsa Italiana migration to MIT
3. MIT convergence road map for LSE, Turquoise and Oslo
4. The Group Ticker Plant
5. Group Trading Interface
6. Sponsored Access
7. Conclusion
34
35. Conclusion
• LSEG continues to lead in trading systems performance thanks to its
Millennium IT low latency technology
• Performance is relevant, however other aspects can make the difference
between success and failure. We have seen only some of them:
1. Functionally rich systems to support the launch of new products
2. Top reliability
3. Architecture flexibility
4. Market data consolidation (GTP)
5. Market access standardization (GTI & GSPO)
6. Efficient development and operational models
• LSEG is active on each one of these fronts as it is on Performance to continue
to improve the quality of our services
35