Keysight Uncovers Critical Issues in Early MACsec Implementations
While there are different encryption technologies to secure data in motion, MACsec brings line-rate encryption throughput for high-speed Ethernet. MACsec has become extremely popular as an encryption technology that is shipped with next-generation chips, routers, and switches.
When MACsec is implemented in a switch, the switch receives encrypted traffic, decrypts it, and then re-encrypts the packets before forwarding. This additional processing may impact the forwarding throughput of the device, especially when the traffic is flowing at 100% line rate with small-size frames.
Engineers here at Keysight used our IxNetwork MACsec Test Solution to run tests against three devices from the leading network equipment manufacturers (NEMs) (DUT A, DUT B, DUT C). Our functional and stress tests revealed critical issues in the early MACsec implementations. Let’s take a look at the details for a few of the trouble spots we found.
Impact of a Broken MACsec Key Agreement (MKA) Control Plane
The MKA protocol (802.1X – 2010) is used to discover members in a connectivity association (CA) and establish a secured association among members to coordinate crypto algorithms and keys used for encryption/decryption to protect data transfer over the local area network (LAN) network. Validation is critical to a successful MACsec operation. Read here to know more about MACsec MKA Validation techniques.
- Sending the wrong short secure channel identifier (SSCI) when MKA is used and not conforming to IEEE specs. In a pairwise CA configuration and using the XPN cipher suite, DUT A and C sent SSCI as 0 but DUT B sent an invalid number like 119. As a consequence, the receiving MKA device was not be able to calculate its SSCI and the SSCI of the other devices in the CA, which caused a problem with encrypting and decrypting traffic with the XPN cipher suite.
- Wrong key server ID. After rekey, DUT A replied with SAK_USE for Key Number 2, AN 1 but Key Server ID was all 0s! If the Key Server ID is not correct, the key server cannot determine if the sender of the MKPDU is able to transmit or receive traffic with the new key or not. This will delay switchover to the new key by 6 seconds (transmitDelay) and can result in traffic loss if packet number (PN) exhaustion occurs for the old key in those 6 seconds.
- No MKPDU with Clear Text VLAN. DUT C does not send MKPDU when clear text VLAN is enabled. As a result, the MKA session is not coming up and there is no cipher suite and SAK distribution, hence no encryption traffic can flow.
- Stops sending MKPDU under stress. DUT C stopped sending MKPDU when line-rate traffic is sent with frame size <128 bytes. This leads to a down MKA session and then traffic loss.
Data Plane Forwarding with Small Frame Size
We ran tests to measure the performance when forwarding with encrypted traffic. IxNetwork was used to created different stress conditions with various traffic patterns, such as frame sizes, mixed traffic flow ratio, and bursty or constant traffic rate. Following are our observation:
- Packet drop at lower frame sizes. Analysis showed that DUT B was erroneously padding lower-sized packets of 64 bytes to 96 bytes while forwarding on the egress. As a result there was packet drop in the test.
- Sending fully meshed imix traffic to DUT B results in ~24% loss and CRC errors.
- Sending increment frame size (98-1518) with re-key triggered from DUT B results in consistent loss of 0.169% and around ~18-second packet loss duration during re-key.
- DUT A, when under stress, sends the traffic to a port that belongs to different CA. This leak causes serious security concerns and is absolutely a “NO GO” in a production network.
Comparison of Encrypted vs. Non-Encrypted Traffic
As with any encryption technique, MACsec imparts some overhead on the forwarding engine, which can adversely impact the latency through the switch. The chipset vendors implement encryption algorithms in the hardware to minimize the processing overhead at high-speed line rates. However, algorithms may have pitfalls when it comes to handling different packet sizes, patterns, and flow rates.
We ran a benchmarking test to find out the performance comparison between encrypted and non-encrypted traffic. Our observation with all the 3 devices showed that jumbo frames result in more latency for encrypted packets.
The results of our validation clearly show the need for testing and performance benchmarking at every stage of MACsec implementation before it is ready for deployment. Any issues discovered in a production network means security risk with unprotected data, which may result in lost trust and revenue for your cloud provider, data center operator, and enterprise customers.
While the leading chipset vendors and NEMs are competing to bring out winning products with their MACsec solutions, their network validation engineers need sophisticated tools to overcome MACsec test challenges and avoid major issues like we found above in the field. Performance at high scale and stability under all conditions can become key differentiation factors. It is obvious from our validation results that testing back-to-back between a vendor’s own devices will not uncover all the critical issues.
As a leader in the network test and measurement market, we at Keysight are committed to provide tools and techniques that enable the adoption of new technologies like MACsec. Read the following blogs to know more about MACsec validation techniques.