SONiC Testbed in a Box
By Nitesh Jha | Testing is one of the integral components of any software or hardware product. It has taken various forms and evolved over a period of time from manual test, automated test, unit test, and dev-test. No matter what kind of product or services you have, testing is CRITICAL.
Most of the software companies are adopting open-source tools and solutions in some way or the other. Open-source tools give them liberty to use an existing framework that’s widely adopted and supported by the communities and eliminates the need of a dedicated team to support them. Lots of these tools are used for test automation framework to support CI/CD pipeline for agile development.
Is this as easy as it looks? Let’s take the example of setting up the community test bed to test SONiC (Software for Open Networking in the Cloud). SONiC is an open-source network operating system (NOS) based on Linux that is developed and open-sourced by Microsoft. It’s one of the most promising open NOS adopted by hyperscalers and cloud companies and market research is indicating to have exponential adoption by enterprise, service providers in coming years.
SONiC community started with developing community test bed by using open-source Packet Test Framework (PTF), Arista vEOS running on multiple VMs in one or more test servers, root and leaf fan-out switches by using vlan-trunking to test one or more SONiC switches for multiple topologies. The complexities of the test bed are derived from setting up so many different hardware, software, and their configuration to get it up and running to a stable state. Multiple virtual and physical cable connections among multiple hardware and software add another layer of complexity. The following is an image of the community test bed:
Figure 1. Community test bed
Source: https://github.com/Azure/sonic-mgmt/blob/master/docs/testbed/img/physical_connection.png
In the last five years, more than 2000 test cases have been developed and contributed by the community members to test multiple features of SONiC. These test cases are written by using the Pytest framework and run from sonic-mgmt infrastructure. However, it has been a challenge to get consistent results for the same SONiC version across multiple vendors. This inconsistency is because of lack of robust SONiC software stack multiples services, ASICs/SAI, and the associated minimal test coverage. There is a huge variation in the test results across two different vendors for the same SONiC version. Debug information is not comparable to any commercial tool so that users can identify the root cause and fix the issue quickly. This results in high engineering spend to debug the problems.
Hence, complexity of the test bed and inconsistent test results are preventing many users from adopting SONiC. Keeping both items in mind, Keysight has come up with product and services that would address them and accelerate the SONiC adoption.
Setup Simplification
Keysight has been able to put the whole test bed and run the tests from a single box. The following image shows all the necessary components that are required for the test bed execution.
Keysight would provide the orchestration that would enable the users to install PTF, Arista EOS, and sonic-mgmt dockers automatically on UHD and kick off the test by using a simple CLI in less than hour.
Hardening sonic-mgmt infrastructure
Keysight understands the durability issues of the test cases, and hence, have come up with services offering. These include setup, training, test cases selection, and debugging of the test results. Customers can buy additional services to harden these cases and have robust, stable, and predictable test results.
To summarize, the advantages are as follows:
- Get a turnkey sonic-mgmt test bed
- Replace the test bed server, root and leaf fanout switches by one box
- Set up easily, with no unnecessary cabling, power, and space requirement
- Scale number of test beds easily
- Experience a simple CLI approach from configuring to running test cases
- Obtain Keysight's services and training to get to a stable sonic-mgmt run
- Get periodic software updates to keep up with the community repository changes
- Get Keysight support
Select SONiC Community Testbed in a Box to watch a demo video.
But we are not done here. Having UHD not only solves the previously highlighted issues but covers the entire spectrum of SONiC testing as shown:
Figure 3. SONiC Testing
In addition, we can do RFC benchmark tests (2544, 2889), data center tests as it supports all the necessary protocols (BGP, VxLAN/EVPN, LACP, PFC, OSPF, ISIS) and 100G traffic generation.
In summary, an OCP form factor 1U UHD100T32 offers everything that’s required to test SONiC on a switch and to get it production ready.