While interoperability testing has been one of the hallmarks of open standards and Internet protocol development for more than three decades, the process itself has been limited by manual and inconsistent testing.
To address these shortcomings, we created QuicInteropRunner (QIR), a framework that enables implementers to maintain and update their own endpoints, making it easier for them to set up and run interoperability tests using carefully constructed and repeatable network scenarios. We have also set up continuous testing with QIR and a page where the results are available.
As implementations of the emerging QUIC protocol being developed in the Internet Engineering Task Force (IETF) began to mature, interoperability testing became increasingly important to ensure that independently-developed implementations worked as expected. However, the existing approach to testing was limited in a number of ways, including:
As a result, identifying issues with the protocol and implementations took longer than it should have, and the existing tests could miss scenarios likely to be found once they were widely in use.
The QuicInteropRunner (QIR) is our attempt at overcoming these limitations. Using simple container orchestration and network simulation, QIR makes it possible to meet the constraints of implementers while automating this process for on-demand, continuous, and repeatable interoperability and performance testing. A more complete explanation how QIR uses Docker containers and the ns-3 network simulator is available in our paper, which was recently presented at the EPIQ ‘20 conference.
Currently, 13 QUIC implementations (two of which only implement a QUIC server and one which only implements a QUIC client) are included in QIR. Importantly, any implementer can include their implementation in QIR by building a compatible container image, making it publicly available, and adding it to the list of implementations.
To enable continuous interoperability testing, we have set up a QIR instance that continuously pulls the implementation images and runs all the tests in the framework. The results of this continuous testing are posted on this webpage.
As a network of networks, the global Internet we know today works because independently deployed and managed components work well together through open protocols. Internet interoperability testing traces its history back to at least 1986 and remains a key part of new protocol development and deployment. While QIR was developed for testing QUIC implementations, we believe the central ideas, components, and even code can be repurposed for testing other protocols that will help the Internet evolve and work better.
We recently ran a survey to understand why people who participated in an IETF meeting in 2018/2019 did not participate in IETF 108 to see if there are any issues that we need to fix. The results are now available and this blog post explains some of the insights in the data.
While interoperability testing has been one of the hallmarks of open standards and Internet protocol development for more than three decades, the process itself has been limited by manual and inconsistent testing.
The Internet Research Task Force (IRTF) Open Meeting at the IETF 101 in London had a rather curious point on the agenda: “Vision for a QIRG: Quantum Internet Research Group”. With the hype around quantum computing growing every year, it was only natural that some people would start asking whether the Internet could also benefit from the quantum revolution.
The IETF Administration LLC has reforecast its 2020 budget to address the significant changes in the underlying assumptions since the original budget was created.
Seven of the IRTF research groups and the ACM/IRTF Applied Networking Research Workshop (ANRW) met co-located the IETF 108 meeting, and three Applied Networking Research Prizes were awarded.
Show all
