The Gateway to Algorithmic and Automated Trading

Timestamp resolution under MiFID 2

Published in Automated Trader Magazine Issue 39 Q2 2016

Many production systems fall short of the new MiFID 2 requirements when it comes to handling and storing timestamp data. We look at the most commonly used systems and examine their suitability going forward.

The new MiFID 2 regulations are not yet set in stone, but are approaching completion with what should be minor changes between now and June 2016. Although the regulations will not come into effect until January 2017, now is a good time to start updating relevant systems to ensure they are compliant with the prospective new rules. A potential further delay to January 2018 is still actively being discussed at the time of writing. Readers may want to refer to page 24 for additional insights into this.

There is one section of the latest revision of the RTS (Regulatory Technical Standards) that is entirely new:

"RTS 25: Draft regulatory technical standards on clock synchronization" - 2015-ESMA-1464

The previous revision of these regulations was codified in 2004 and lacked specificity with regards to timestamp synchronisation, divergence and granularity.

The latest proposed regulations seek to address this. In the table opposite, each type of trading activity is color-coded according to its requirements:

Red for requirements that allow timestamps with a resolution of at least 1 second (such as manual trading activities)

Yellow for activities that require 1 millisecond resolution

Green for applications that require microsecond resolution timestamps

The designations 'category 1, 2 and 3' are not mentioned in the directive text. It is something we have introduced here to easily identify the various levels of requirements.

Note that any data type that satisfies category 3 also satisfies categories 2 and 1. Correspondingly, data types for category 2 automatically satisfy category 1.

Category 1 Timestamp resolution: 1 second
Divergence: 1 second
Activity on voice trading systems Voice trading systems as defined in Article 5(5) of RTS 2 transparency requirements in respects of bonds, structured financial products etc
Activity on request for quote systems where the response requires human intervention or where the system does not allow algorithmic trading Request for quotes systems as defined in Article 5(4) of RTS 2 transparency requirements in respects of bonds, structured financial products etc.
Activity of concluding negotiated transactions Negotiated transaction as defined under Article 4(1)(b) of Regulation (EU) 600/2014
Category 2 Timestamp resolution: 1 millisecond
Divergence: 1 millisecond
All other activity not covered by Category 1 or Category 3.
Category 3 Timestamp resolution: 1 microsecond
Divergence: 100 microseconds
Activity under the vague reference to 'High frequency algorithmic trading technique', a term currently not further defined by ESMA

It is worthwhile pointing out that even though the new regulations can demand up to microsecond resolution for 'high frequency algorithmic trading', this term is in fact not defined in the standard as of yet. There have been drafts leaked out of Brussels that indicate that the number of orders sent per second by a firm may be used as criteria to be classified for category 3 requirements: either sending two orders per second per liquid instrument or sending four orders per second per venue (for sure an odd way to define things in the eyes of any high-frequency trader. But with MiFID 2 oddities are the norm rather than the exception).

There appears to be some wiggle room for agency brokers in the sense that pure customer flow by definition does not come under the 'high frequency' label. Only proprietary flow seems to be able to fall under this classification, at least that is what we are hearing in unofficial discussions with various parties. Simple order routing activity, including Smart Order Routers, is exempt. However, algorithmic trading (e.g. the processing of larger orders into child orders to minimize market impact) is not exempt and would fall under category 3.

Given the way the wind is blowing, it seems reasonable to assume that for most firms, a large fraction of their automated trading activity will be caught by this definition. In other words, if in doubt as to whether the high frequency designation applies (category 3), then assume that it does.

Additionally, the MiFID 2 proposals require a strategy for how timestamps are generated in a system, to guarantee that they are collected with their correct sequence ordering. These obligations are beyond the scope of this article and will be covered in a future issue of Automated Trader.

The MiFID 2 proposals also stipulate a maximum divergence from UTC (Coordinated Universal Time) of 1 second, 1 millisecond or 100 microseconds depending on the type of trading conducted. This is another issue that relates to the accuracy of the timestamps.

For now, we will focus on possible implementation issues surrounding the new requirements on timestamp resolution, especially as they relate to storage (i.e. persistence).

Not every runtime or relational database system will support these new specifications, so the tables on the next page give an overview of which systems should be compliant and which will require upgrading or re-architecting. Despite it being 2016, there are still a great many systems in production use today that do not support native temporal types that are precise enough. Worse still, there are also systems that appear to support certain resolutions at first glance, yet do not actually support the required resolution.

For example, Microsoft SQL Server legacy DATETIME type appears to support milliseconds on casual observation, that means it might appear as 10:43:29.253. In fact, it only supports increments of .000, .003 or .007 seconds, thereby technically falling outside some of the millisecond resolution requirements of MiFID 2 (see category 2).

Runtime Data Type Description
.NET Framework Class Library
  • TimeSpan
  • DateTime
  • DateTimeOffset
  • Supports 100 nanosecond resolution
  • Supports 100 nanosecond resolution
  • Supports 100 nanosecond resolution
Java
  • java.util.Date
  • java.sql.Timestamp
  • java.time.LocalDateTime
  • java.time.LocalTime
  • java.time.ZonedDateTime
  • org.joda.time.LocalDateTime
  • org.joda.time.DateTime
  • Supports 1 millisecond resolution
  • Supports 1 millisecond resolution
  • Supports 1 millisecond resolution
  • Supports 1 millisecond resolution
  • Supports 1 millisecond resolution
  • Part of Joda-Time, supports 1 millisecond resolution
  • Part of Joda-Time, supports 1 millisecond resolution
Python
  • datetime
  • time.struct_time
  • Supports 1 microsecond resolution
  • Only supports 1 second resolution. Returned from gmtime(), localtime(), and strptime()
Win32
  • SYSTEMTIME
  • FILETIMEM
  • Supports 1 millisecond resolution
  • Supports 100 nanosecond resolution
C++ STL
  • tm
  • time_t
  • Only supports 1 second resolution
  • Only supports 1 second resolution

Table 01: Common Runtime Libraries

Table 01 and Table 02 detail the most commonly used Relational Database Management Systems and Runtime Environments used in finance, and their respective compatibilities for microsecond timestamps.

Each data type is color-coded with reference to the requirements laid out above. Data types colored green support the most stringent requirements and therefore are compatible with storing timestamps for all three categories. Those in yellow are somewhat more restrictive and are only able to store values for category 1 and 2 requirements. Finally, data types in red will only support storing values in category 1, which is the most permissive category. Alarm bells should ring if one of these more restrictive data types is encountered in a production system, as it is likely to be necessary to upgrade the system. Most financial firms will not be able to get away with this level of resolution under MiFID 2.

Conclusion

In summary, many issues can arise when trying to ensure compliance with the proposed MiFID 2 standards, particularly on the microsecond resolution scale. What appears to be a relatively minor issue can become more problematic when considering the full software stack or legacy applications.

Even though many commonly-used runtimes and database engines support the proposed resolution in temporal data types, some might not if they are running on legacy data types or on older versions.

There are a plethora of potential issues and those caught unprepared will quickly realize how much time and effort it can take to update and migrate large and complex systems, to ensure support once the new regulations are in full effect. Further implications arise when considering synchronisation and divergence from the UTC reference time that all timestamps will have to guarantee. These will be covered in a future article.

Data Type Version Description
Oracle Database
TIMESTAMP(n) Oracle9i Extension of the DATE datatype. Supports up to 1 nanosecond resolution via optional 'fractional_seconds_resolution' type attribute. Default is 1 microsecond resolution.
INTERVAL DAY TO SECOND(n) Oracle9i Interval expression. Supports up to 1 nanosecond resolution via optional 'fractional_seconds_resolution' type attribute. Default is 1 microsecond resolution.
DATE Oracle7 Supports 1 second resolution
SQL Server
DATETIME2 SQL Server 2008 Supports 100 nanoseconds resolution. Default is 100 nanoseconds resolution.
DATETIME Previous versions Should not be used. Though it may appear at first glance that this type supports 1 millisecond resolution, the fractional seconds are actually rounded to increments of .000, .003, or .007 seconds. Upgrade and use DATETIME2 or use BIGINT from an epoch that makes sense in your system.
MySQL
DATETIME(n) v5.6.4 Type upgraded to support fractional seconds, up to 1 microsecond resolution. Default is 1 second resolution.
TIMESTAMP(n)
TIME(n)
DATETIME Previous versions Only supports 1 second resolution in previous versions
TIMESTAMP
TIME
KDB+
TIMESTAMP v2.6 Backed by a 64-bit integer number providing 1 nanosecond resolution
TIMESPAN
TIME Previous versions Only supports 1 millisecond resolution
IBM Informix
DATETIME(n) 10 microsecond resolution via the fraction(n) attribute
IBM DB2 LUW (Linux, UNIX and Windows)
TIMESTAMP Supports 1 microsecond resolution
TIME Supports 1 second resolution
IBM DB2 LUW (z/OS)
TIMESTAMP Supports 1 microsecond resolution
TIME Supports 1 second resolution

Table 02: Common Database Systems