So earlier this week while browsing the transit blogosphere, I read on Subway Blogger how the MetroCard Vending Machine’s (MVM) ability to take electronic payments crashed. One would figure that the problem would be taken care of quickly but not so as it continued into the next day. Days after the MTA decided to issue a press release about the entire fiasco:
During three separate rush periods (Monday a.m. and p.m., and Tuesday a.m.), MTA New York City Transit’s MetroCard Vending Machines (MVMs) and MetroCard Express Machines (MEMs) had difficulty processing credit-and debit-card sales transactions, inconveniencing thousands of our customers. As customer usage increased, it took an increased period of time to receive authorizations for card usage from our bank. In such cases, the machines are programmed to cancel the transaction after too much time has elapsed without authorization.
After an investigation, it was discovered that one of the encryption devices was not operating. The devices are required by Credit Card Industry security protocols and are necessary to process transactions. The one functioning device could not handle the load during peak usage periods.
Normally the system is served by at least two encrypters (sometimes three). In this case, however, our monitoring system told us that two were active when in fact only one of them was working. This “false positive” was the main reason why it took so long to identify the cause.
Following Tuesday’s a.m. rush period, when this issue was identified, another device was activated to distribute the workload and, as we expected, the change corrected the “time out” issues encountered over those three rush hours.
To eliminate a recurrence of this issue, we have requested that the contractor who provides the interface from the MTA data center to the bank that authorizes credit-and-debit-card transactions investigate the addition of a triggering feature that will display a warning message when only one encryption device is active. This will allow NYC Transit to take immediate action to prevent possible future outages.
It should be noted that on Wednesday, with the problem corrected, we processed 211,000 credit/debit purchase requests at MVM’s (about 20,000 above the normal figure) and successfully processed 99.98% of them. Only 35 attempted transactions didn’t succeed, and only 14 of those “timed out.”
Things like this are bound to happen as that is the price we pay when dealing with technology. I however would be worried as this is something that has happened before within the system albeit on the Long Island Rail Road. I have always heard it is good to carry a card in case of an emergency. I’m inclined to think the opposite as it is good to have cash on you in case something like this happens. Imagine how people who only had what was on their card to depend on for traveling felt when they were screwed. They seriously need to find a way to issue temporary credits or cards in case of such an emergency in the future.
xoxo Transit Blogger