TSE’s Arrowhead system was launched with much fanfare in 2010 and was billed as a modern-day solution.
At 7:04 a.m. Thursday autumn in Tokyo, regulators of the world’s third-largest equity market realized they had a problem. The important data device for the trading system of the Tokyo Stock Exchange was missing, and the automatic backup failed to kick. Less than an hour before the system, called Arrowhead, was due to begin processing orders in the 6 6 trillion equity market. Exchange officials could not see any compromise.
The entire day has been shut down since the exchange switched to a full-fledged electronic trading system in 1999. Criticism from market participants and executives focused on the under-discussed weakness in global financial plumbing – not software. Or security risks but the risk is when one of the pieces of hardware that makes up a trading system decides to leave a ghost.
Members of the media gather inside the Tokyo Stock Exchange in Tokyo on October 1st.
“Exchange is a crucial part of the market infrastructure and it is unacceptable to deny trade opportunities,” Finance Minister Taro Aso told reporters in Tokyo. “You’re working with machines, so it’s always possible for them to break down. They need to build infrastructure keeping in mind the possibility of a breakdown.”
TSE’s arrowhead system began with much fanfare in 2000, and was billed as a modern-day solution in 2000, following a series of outages on the old system. The “arrow” symbolizes the speed of order processing, while the “head” indicates strength and reliability, according to the exchange. There were a few setbacks in the system of about 350 servers processing orders to buy and sell, but there was no major disruption in its first decade.
That all changed on Thursday, when a piece of hardware called the No. 1 shared disk device, one of two square-shaped data storage boxes, was found to have a memory error. These devices store management data used in servers and distribute information such as commands and ID and password attachments to terminals that monitor trades.
When an error occurs, the system fails – what is called automatic switching on the No. 2 device should be carried out. But for some reason exchange officials could not explain, the process also failed. This did not have a knock-on effect on the servers known as the distribution gateway to send market information to merchants.
Disappearing data
At 8 a.m., traders who are ready at their desks when the market opens an hour later should be looking for indicative prices at their terminals as orders are processed. But many saw nothing, while others reported the data appearing and disappearing. They had no idea if the information was accurate.
A minute later, Borse held his first conversation, informing systems managers at securities companies that a problem had occurred. On some brokerages, it does not immediately filter out the filtered trading desk.
At about 8:05 a.m., Twitter – frequently used by merchants to communicate outside of the more communicative facial communication channels monitored by compliance – began to buzz with rumors of an issue. Traders described a growing sense of incompetence as few answers were forthcoming.
“We didn’t know it was our system or the exchange,” said Masaya Akiba, a broker in the stock-trading division of Marusan Securities Co. “We only confirmed it when the release was released by the exchange.”
At 8:36 a.m., Bosser finally informed the securities companies that trading would be halted. Three minutes later, he made a press release on his public website – albeit only in Japanese. The confusedly translated English publication will not follow for more than 90 minutes.
For the first time in almost fifteen years, the exchange was hit by disasters in the entire trade. The Tokyo Bourse has a policy of not closing even during natural disasters, so this experience was a first for many who trade in the capital.
.Historical decision
Some market participants shouted as they closed. Others, who had nothing to do, devoted their time reading research notes or trading commodities.
“I haven’t thought about it before,” said Kyoshi Ishigan, chief fund manager at Mitsubishi UFJ Kokusai Asset Management Co. in Tokyo. “Previous outages were resolved quickly so I assumed the order would only be late.”
In 2012, after a switchover on Arrowhead, the exchange quickly resolved limited issues. Many expected to do the same this time.
But as the hours passed, Hazim Sakai, chief fund manager at Mito Securities Co., became increasingly upset.
“I couldn’t really pay much attention to anything else,” he said. “I didn’t like, ‘Open the market!’ It was more like, ‘Call it whatever it is, fast.’ “
Kal was a desperate one. After a failed switch to backup, the exchange manually forced a switchover on the No. 2 shared disk device. At this point, the administrators had a choice: they could try resuming trading, but this would force a complete reset of the system – turning off the power and rebooting.
Data for orders already received from securities companies would have been lost, without cancellation. This would have caused chaos, securities companies told the exchange. After talking to market participants, the exchange made its decision: Trading will be closed for the entire day.
Many in the market say they are relieved. A Tokyo-based brokerage worker said there was no way to tell which orders from existing customers were active, while also trying to process new inquiries and bids.
Technical discussion
At 4.30pm local time, four TSE officials, including Chief Executive Officer Koichiro Miyahara and Chief Information Officer Raisuk Yokoyama, will confront reporters at the exchange to explain the outage. In a briefing that lasted about 100 minutes, they apologized in front of a crowded room before going into a detailed technical discussion of the crash.
If Boras was criticized for the conversation earlier that day, he was praised for how he handled the press conference. Executives discussed areas such as systems architecture in easy, highly technical terms regarding answers to media questions. Rather than try to blame system vendor Fujitsu Ltd., they accepted responsibility for the incident, as there have been some similarities between GAF-filled briefings by other Japanese companies in the past. On Twitter, the Japanese public voiced its approval.
“The management explained very clearly during last night’s briefing,” said Magumi Takarada, senior analyst at Toyo Securities Kunwar in Tokyo. “The briefing gave a bit of assurance that management understands the issue clearly.”
Later in the evening, it was announced that the bourse would resume trading from Friday. When it passed without issue, many questions remained unanswered. The Financial Services Agency has ordered the exchange to issue a report on the outage, according to local media, which may provide more insight on some of the issues.
But the biggest of them is whether the failure to run from the same stock hardware could happen in other stock markets. For a strategist, he can almost certainly – but not too much to worry about.
“There’s nothing special Japanese about this,” said Nicholas Smith of CLSA Ltd. in Tokyo. “I think we call it ‘things happen.’ No. These things happen. They shouldn’t do it, but they should do it. “
(Except for the headline, this story has not been edited by NDTV staff and has been published from Syndicate Feed.)
.
